From owner-mpi-core@CS.UTK.EDU Mon Jan 25 13:23:41 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA22386; Mon, 25 Jan 93 13:23:41 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA12278; Mon, 25 Jan 93 13:23:35 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 25 Jan 1993 13:23:33 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from THUD.CS.UTK.EDU by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA12267; Mon, 25 Jan 93 13:23:31 -0500 Received: from LOCALHOST.cs.utk.edu by thud.cs.utk.edu with SMTP (5.61++/2.7c-UTK) id AA27302; Mon, 25 Jan 93 13:23:31 -0500 Message-Id: <9301251823.AA27302@thud.cs.utk.edu> To: mpi-core@cs.utk.edu Cc: wade@cs.utk.edu Subject: test Date: Mon, 25 Jan 93 13:23:30 EST From: Reed Wade testing archive -reed From owner-mpi-core@CS.UTK.EDU Tue Jan 26 11:10:55 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA12746; Tue, 26 Jan 93 11:10:55 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA08431; Tue, 26 Jan 93 11:10:27 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 26 Jan 1993 11:10:25 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA08412; Tue, 26 Jan 93 11:10:20 -0500 Received: from donner.mcs.anl.gov by antares.mcs.anl.gov (4.1/SMI-GAR) id AA10009; Tue, 26 Jan 93 10:10:16 CST Received: by donner.mcs.anl.gov (4.1/GCF-5.8) id AA15597; Tue, 26 Jan 93 10:10:10 CST Message-Id: <9301261610.AA15597@donner.mcs.anl.gov> To: mpi-core@cs.utk.edu Subject: minutes of January MPI meeting Cc: lusk@antares.mcs.anl.gov Date: Tue, 26 Jan 93 10:10:09 CST From: Rusty Lusk To all aficionados of the non-blocking wait: These are the minutes of the January meeting in Dallas, as taken by Bob Knighten and myself. Before sending them out to the world, I am circulating them to the people at the meeting, just in case you wish to disavow any opinions I have attributed to you. I would like to distribute these widely later today, if possible, so please give them a quick check. Regards, Rusty Lusk _____________________________________________________________________________ Minutes of the Message Passing Interface Standard Meeting Dallas, January 6-8, 1992 The MPI Standards Committee met in Dallas on January 6-8, 1992, at the Bristol Suites Hotel in North Dallas. This was the third meeting of the MPI committee, but the first following the format used by the High Performance Fortran Forum. There were both general meetings of the committee as a whole and meetings of several of the subcommittees. Because interest in the Point-to-Point communications and the Collective communications was so general, these met as committees of the whole. No formal decisions were taken at this meeting, but a number of straw votes were taken in the subcommittees. These are included as part of the reports on the work of the subcommittees. These minutes were taken by Rusty Lusk (lusk@mcs.anl.gov) with some additions by Bob Knighten. Marc Snir's notes on the point-to-point subcommittee meetings are included here as well. These minutes are quite long. If you want to see the important topics you can search for --- and this will quickly the lead to each topic (and a few other things.) January 6 --------- ------------------------------------------------------------------------------- General Meeting ------------------------------------------------------------------------------- The meeting was called to order by Jack Dongarra at 1:30. Jack Dongarra presented the rules and procedures that had been circulated in the mailing list. In general, they say that we intend to operate in very open fashion, following the example set by the High-Performance Fortran Committee. He also described the subcommittee structure. For details, see the mailing list, A tentative schedule for future meetings was presented, which was amended on the last day (see there). All meetings will be in Dallas at the Bristol Suites. Steve Otto will coordinate the production of the document. He will obtain a set of LaTeX macros from the HPF Committee and distribute them to the subcommittee heads. It was suggested by Bob Knighten that the Executive Director arrange for copies of all pertinent documents be provided at the meetings. Dennis Weeks, who is somewhat local (Convex), volunteered to help with the relevant copying. The attendees were: Ed Anderson Cray Research eca@cray.com James Cownie Meiko jim@meiko.co.uk Jack Dongarra UT/ORNL dongarra@cs.utk.edu Jim Feeney IBM-Endicott feeneyj@gdlvm6.vnet.ibm.com Jon Flower ParaSoft jwf@parasoft.com Daniel Frye IBM-Kingston danielf@kgnvma.vnet.ibm.com Al Geist ORNL gst@ornl.gov Ian Glendinning Univ. of Southampton igl@ecs.soton.ac.uk Adam Greenberg TMC moose@think.com Bill Gropp ANL gropp@mcs.anl.gov Robert Harrison PNL rj_harrison@pnl.gov Leslie Hart NOAA/FSL hart@fsl.noaa.gov Tom Haupt Syracuse U. haupt@npac.syr.edu Rolf Hempel GMD hempel@gmd.de Tom Henderson NOAA/FSL hender@fsl.noaa.gov C. T. Howard Ho IBM Almaden ho@almaden.ibm.com Steven Huss-Lederman SRC lederman@super.org John Kapenga Western Michigan Univ. john@cs.wmich.edu Bob Knighten Intel SSD knighten@ssd.intel.com Bob Leary SDSC leary@sdsc.edu Rik Littlefield PNL rj_littlefield@pnl.gov Rusty Lusk ANL lusk@mcs.anl.gov Barney Maccabe Sandia abmacca@cs.sandia.gov Phil McKinley Michigan State mckinlehy@cps.msu.edu Chuck Mosher ARCO ccm@arco.com Dan Nessett LLNL nessett@llnl.gov Steve Otto Oregon Graduate Institute otto@cse.ogi.edu Paul Pierce Intel prp@ssd.intel.com Peter Rigsbee Cray Research par@cray.com Ambuj Singh UC Santa Barbara ambuj@cs.ucsb.edu Marc Snir IBM snir@watson.ibm.com Robert G. Voigt NSF rvoigt@nsf.gov David Walker ORNL walker@msr.epm.ornl.gov Dennis Weeks Convex weeks@convex.com Stephen Wheat Sandia NL srwheat@cs.sandia.gov ------------------------------------------------------------------------------- Point-to-point subcommittee ------------------------------------------------------------------------------- Mark Snir called the meeting to order at 1:40 p.m. It adjourned at 4:10 p.m. It resumed the following morning at 9:10 a.m. and adjourned at 4:15 p.m. Marc Snir began by summarizing the decisions that we have to make: * which operations? send receive channels? sendreceive? info arguments operations on queues probe? * operation modes sync async local and/or global termination interrupt-driven? * message types (data types) structure of data in core buffer packing * send-receive matching type (We later decided to call this "tag".) sender? * correctness criteria (See Marc Snir's paper in handouts) * heterogeneous operations * name space how processes are addressed flat? structured? implicit/explicit * error handling * interaction with threads, interrupt handlers, remote signalling * special operations for high performance ready receiver? * process startup * syntax/style (The plan is to postpone this for this meeting.) We will prioritize this list and then go through them one by one. (The priorities assigned were more or less in the order listed above.) Two preliminary questions were then discussed: A. Must we worry about multithreaded environments? James Cownie pointed out that threads were coming, in almost all new systems. Most systems have threads now. It was proposed that a process, which could send and receive messages, should be an address space, so that individual threads would not be (MPI-) addressable. B. What about signals? Paul Pierce suggested that we discuss signals first: do we want to support send/receive from interrupt handlers? These two questions were then discussed at length. Dealing with threads argues against the notion of "last message", since that implies state is maintained by the system. There was general agreement that "state" was a ` bad thing, but arguments in favor of state are: Sometimes one doesn't want all of the information available after an operations, so it shouldn't be returned. Having lots of arguments to calls is bad, especially inout arguments. Ways to avoid state are: Structures could be returned Return individual arguments Return tag to do queries on (but they one needs to free it.) Additional out arguments (OK in Fortran 90, but not in C or f77) User passes in storage to be used (so he knows the address), and MPI provides inquiry functions [For more details, see Jim Cownie's mail message of January 4, 1993 entitled: Multifarious] There was a general agreement that system state decreases portability and manageability, and we should decrease it when we can. James Cownie said that We need a reentrant style, and Mark Snir suggested that we try to make all function calls reentrant. When queried no one in the group objected to trying to make all the functions that are introduced in MPI reentrant. Now we began going through the above-mentioned major topics. Which Operations? ----- ---------- We have send and receive. How about send-receive (also called shift)? It can be efficiently implemented, and buffer can be reused. There was a discussion of the "two-body" send-receive (exchange) and the "three-body" version (ring-shift). Variations include those in which the send-buffer is required to be the same as the received- buffer and those in which is is required to be disjoint from the receive-buffer. Al Geist: We should focus on *required* operations. Steve Otto replied that send-receive *is* a required operation. Using "exchange" can help avoid deadlock. It was agreed that there was no consensus on these issues and it was decided to defer this to the collective communication subcommittee. Operation Modes --------- ----- The next topic that Marc Snir raised for discussion was when do send and receive return. Marc described several options: For send: 1) return as soon as possible 2) return when send-buffer is clear 3) return when the corresponding receive has completed For receive: 1) return as soon as possible 2) return when the receive-buffer is full "Receive has completed" means "when the user knows". In other words, when the sender has returned from send, the receiver has returned from receive. There was a general discussion about whether 3) was necessary? dangerous? Robert Harrison said he believed that 3) was the minimal version that was truly portable. Steve Otto pointed out that 3) is CSP-like. Rusty Lusk said that 3) would be easier to prove things about than the others. Adam Greenberg and Paul Pierce pointed out that neither TMC nor Intel have implemented an operation depending on the behavior of the receiver. A straw vote was taken and the vote was 17-3 in favor of having 3) as an option. Marc Snir pointed out that in his original proposal send returns a handle and the status of the handle is then tested for completion of the send operation, and asked if this is this desirable. There was general agreement that something of this sort was desirable, but a variety of alternatives were mentioned It was pointed out that sometimes one wants to wait on multiple outstanding operations. Al Geist prefers separating "wait" into "sendwait" and receivewait" for code readability. Bill Gropp suggested that instead of using handles, one could supply a routine to be called when an operation completes. James Cownie: "This gets really hairy in Fortran". There was a discussion of probing multiple outstanding receives: If the receives return handles, h1 = recv( ... ) h2 = recv( ... ) wait ( h1 or h2) ? wait ( h1 and h2 ) is not needed. Jame Cownie proposed that we supply an operations to *wait* on a vector of handles, which would return one of those that have succeeded. It would return the handle, not the status. A straw vote as taken on this proposal, which passed 17 - 0. So we have: status (handle) wait (array of handles) The send specifies what completion of send means. Handles need to be freed. It was pointed out the only the existence of such an operation has been decided, the semantics are yet unspecified - e.g. issues such as fairness or what wait returns when several complete are not yet specified. There was a long discussion of cancellation of send and receive. It was observed that there are serious implementation problems because of race conditions, freeing resources, etc. A straw poll was taken on including cancel in the initial MPI. It failed 7-19. This was the end of the Wednesday afternoon point-to-point meeting. January 7 --------- The point-to-point subcommittee (now a Committee of the Whole) resumed at 9:15 a.m. on Thursday morning. Marc Snir opened the meeting and summarized the progress so far: 3 ways in which send can terminate sendreceive postponed no cancel of incomplete send operation status and wait (successful status accomplishes same as wait) We did not get to: channels (the idea of trying to bind as soon as possible as many parameters as possible, so that they can be reused.) probe readyreceive Marc noted that channels and readyrecv address similar issues. Probably want only one of these. Do we want either? Rolf Hempel observed that we don't need channels - can depend on operating system to cache the connection information when doing synchronous communication. Adam Greenberg replied: NO! Want to be able to do this all at user level without "smart" OS. Channel creation and use might look like: handle = send_init( ... ) start(handle) wait(handle) free(handle) This is an intermediate point between bundled send/receive and full named channels. Indeed there are many intermediate points based on various early bindings. Is there enough experience to justify a standard? Bob Knighten observed that there has been substantial experience with channels on the iWarp system. There was next a discussion of the ready-receiver semantics proposed by Lusk and Gropp in the handouts. Steve Huss-Lederman said that such operations could make a difference of as much as 25% for matrix multiplication on the Delta. Some doubt was expressed about the universality of this optimization. Question of use of readyrecv by naive users again. Cownie mentions experience again. Greenberg: facilities for efficiency should not make it difficult to write correct programs. Wheat: Don't penalize users who do understand and can take advantage of efficient procedures. General back and forth discussion. Two straw votes were taken: Ready-receiver operations passed 13 - 10 Channels passed 19 - 2 (Marc Snir will write up a detailed proposal) The next topic discussed was the probe operation. Do we want such an operation, and if so, what should be its semantics? Probing must "lock" the message that it finds, else the information returned by the probe may be unreliable. (Consider the multithreaded environment.) Bill Gropp pointed out that probe is often used to find out the length of a message so that a buffer of the appropriate size can be allocated. Marc Snir pointed out that this is a problem with the November document, that we need to know the length of a message ahead of time. Jon Flowers suggested the need for a blocking probe. What is needed is to probe and then to receive the message found via the probe: handle = probe(params) . . . recv(handle) release(handle) Marc Snir pointed out that the handle serves as a lock on the message. James Cownie pointed out that while we agreed to not have a cancel for a send, we do need to be able to cancel receives, since an outstanding receive is permission for the system to write in the user's address space, which is a permission the user may want to revoke. A straw vote was taken on the existence of some form of probe, and it passed 25 to 0. Send-Receive Matching ------------ -------- The next topic is the matching of send and receive. Currently we have to discuss matching on: tag sender group id context id We will also need to discuss the name space issue for messages. Here are three proposals for the predicate that determine whether a message matches a particular receive: 1) simple matching on fields 2) more general, with mask, etc. 3) user defined function Adam Greenberg said that at TMC: A user defined function is used by the system whenever a message is received by a node to decide if it is to actually be received by the application. The parameters to the user defined receive predicate are tag and group. Issue: If most information is encoded in the tag, then the tag protocol must be understood by all users involved in writing a particular application. True, but not a serious problem. Best to identify small class of specific matching parameters (e.g. group) and use tag for everything else. James Cownie pointed out that the matching function, if not too complicated, can (and is, on many systems) done by special communications processors. There was further discussion of the difficulties of having the system call user code for screening messages. Paul Pierce pointed out that receipt of a message by the hardware is a crucial point for performance. There was general discussion of alternative approaches to getting at least some of this. The question of need for this generality was also raised. TMC has a user who wants and uses his own predicate function. Possibilities: (a) select on mask for fields (including a don't care); (b) simple static logical operations on fields; (c) user defined (b) might be match = AND (( message(i) = pattern(i) ) OR mask(i)) fields A straw vote was taken on whether to pursue allowing user-defines predicates. It was decided 26-1 not to allow user-defined functions for this purpose. (b) deferred until a proposal is available. Marc Snir summarized that matching by tag is generally agreed on and that this is not the only item for selection. After some discussion, matching by sender was also generally agreed on. So now, how do we identify a sender? Rusty Lusk spoke in favor of a flat name space, so that processes could be addressed independently of group, etc. There ensued a general discussion of groups, contexts, and the name space. It was pointed out that the name space expected by send could be flat and groups could be implemented by a function that converted any structured name into a flat integer id. Other proposals were to to have name=(rank,gid) with the restriction that this name be usable only within the given group (gid) and the sender must be a member of this group. By default the group would be ALL. Other alternatives mentioned were name=(rank,ALL)=pid and name=(pid,context). This led to a general discussion of context and the relation to groups. Marc Snir pointed out that we could have pid pid,context in which context did not change the meaning of pid. Paul Pierce said that tags and contexts should be separated since they need to be handled in different ways. Marc Snir pointed out that there should be no "don't care" on context. There was a discussion of servers that can process "any" message. This also led into discussion of flat name space vs. hierarchical name space where we would have pid(group, rank) function. Can use context to define groups, but there are other uses as well. Why groups as well as context? What is the difference between context and groups? Cownie: Context is just another integer used in the same manner as tag. Not quite - it is reserved, but what is the meaning of "reserved"? Greenberg was concerned about connecting send/receive behavior with groups. Snir: Suppose a users wants to have two independently written subroutines that use the usual rank notation. Wheat: Similarly want to use rank notation when partitioning machine. Snir: Both contexts and groups are nice, but do we need both? Gropp: Problem with mixing two applications both of which use 0-based indexing will need a larger common name space when they need to communicate. There was a general discussion of the cost of contexts. Cownie observed that context is cheap if only used to distinguish code - obtain a unique context id for the code by means of the "one-dollar random number generator": each author obtains a one-dollar bill, copies the serial number, and then burns the bill. But in general context is not cheaper than groups. Someone asked about spawning additional processes while program is running? Various people raised the question: If use name=(pid,context), does context change the meaning of the pid (i.e. is pid context {or gid or ???} relative.) There was some discussion of message registration. Paul Pierce observed that tag vs. context is only matter of registration. He wants to divorce tag and context for safety. This implies that one cannot use wild card for selecting on context. Various people noted difficulties with mixing tag and context. Adam Greenberg offered: Proposal - always separate tag and context. Have a context, NONE, so that pid with context NONE is unmodified, but with other contexts the pid may be relative. [NONE, GLOBAL, BASE] tag, context - must match on context Several people noted that there are two very different uses of context - identification of distinct code and identification of a group of processors. There is state, even distributed state, associated with remapping of processors with groups. POSSIBLE FIELDS FOR SEND/RECEIVE: tag context id group - no wild card - set of processes - registration management - receive only from group - managed by system Marc Snir asked whether we could agree on what would be carried with a message: tag context (like tag, except no wild card; management to be determined) Two straw votes were taken: Having contexts passed unanimously. Having the context *not* modify the process id passed unanimously. Groups ------ Three alternatives: no groups (use send(pid(group, rank), ...) instead) group as explicit parameter in operations use contexts to implement groups The basic difference is: do we want to be able to select on group? Straw vote: yes: 10 no:11 on the capability of selecting by group. (Thursday lunch occurred here) Message Data Types ------- ---- ----- WHAT IS A BUFFER? (Language bindings are going to be important here.) There are many options to consider: a) contiguous bytes (non-controversial) General agreement that 0-length messages should be allowed. b) contiguous buffers of other (implementation specific?) units? b) stride? (parameters: base-address, element-length, stride, number-of-elements) c) basic data types? d) arbitrary structures? e) How will we specify data to be communicated in a heterogeneous environment? f) iovec structures (array of pointers and lengths, as in un*x iovec for reads and writes) Marc Snir pointed out that one possibility is to have separate pack/unpack routines and then just send contiguous buffers. Rusty Lusk pointed out that this requires a copy that may be unnecessary on certain machines. Two choices - pack scattered buffer and send OR send scattered buffer. If the second, then may need a pack that produces the descriptor of a scattered buffer to be used by the send scattered buffer. Straw poll: Use IOVEC type send. Passed 18-1 Basic data types were deferred. Marc Snir observed that up to this point, a message is a set of bytes in storage, but now we are about to consider more meanings: message = sequence of *values* Should we use the same calls for homogeneous and heterogeneous calls? Can we have a fast homogeneous implementation of the heterogeneous calls? Bill Gropp pointed out that the current testbed implementation does this. SEND vs. SENDCHAR, SENDREAL, . . . To be compliant with F77 need to have at least SENDCHAR for correctness (and this is a real issue, e.g. on VAX.) Strictly need to have different for each basic data type (but in practice this is not an issue.) But for other than CHARACTER there is also an efficiency issue. 1. F77 conformance 2. Special problem of CHARACTER 3. Performance 4. Heterogeneity (?) Postpone to language binding discussion. This led into the issue of the general problem of converting types between languages and machines! This in turn led to a discussion of XDR (and mention of other systems such as NDR, ...) XDR supports the basic types (INT, REAL, COMPLEX, CHAR, etc.), array constructors, pack/unpack routines, etc. Do we use the same calls for homogeneous and heterogeneous systems? Can we have a fast implementation of heterogeneous procedures for a homogeneous system? What about a "message envelope" that specifies the environmental aspects of messages (e.g. heterogeneity features such as XDR.) When we talk about heterogeneity, do we expect MPI libraries from different vendors on different machines to cooperate? Include general SEND as SENDBYTES? Agreed that do not want SEND in homogeneous to require type information needed for heterogeneous environment. There was a discussion of whether we have to pick an interchange format, for example XDR. There seemed to be some agreement that we do (as MPI implementations from different vendors have to be able to communicate with one another), but no vote was taken. Error Handling ----- -------- The main issue here is whether an error detected by an MPI routine should result in the calling of an error-handler or return of a return code. Other issues are how much of error handling should be standardized as opposed to implementation-dependent, and how much user control there should be over error-handling. There are two types of error environments - soft (recoverable) and hard (unrecoverable). In a soft error environment there is the opportunity for cleanup on the part of both the "application" and the system, while in the hard error environment the system will cleanup and terminate the application. Choices: An mpi routine always returns (though it may return with an error code.) An mpi routine may call an exception handler There may be a default exception handler and there could be a user-installable one as well. Library writers may want to handle errors differently from how a user program wants to handle them (or have them handled by the system). Robert Harrison described the error modes used in TCGMSG and p4: A process has a user-settable state that determines whether an error should result in a (hopefully) graceful takedown of the program or in a error return code. Paul Pierce described the Intel method which uses two syntactically distinct classes of functions. For one class an error results in a message being printed and the process in which the error occurred terminating. For the other class an error code is set. There was some discussion of the problem of maintaining state in a multithreaded environment. Two straw votes were taken: Do we want a version of MPI that calls an exception handler: yes: 23 no: 0 Do we want a version with return codes: yes: 19 no: 1 Specific discussion of modes or "shadow" routines was deferred. Correctness Criteria ----------- -------- This concerns defining what is a correct implementation of MPI An assumption that had to be restated several times during the meeting is that MPI assumes a reliable underlying communication system, i.e. MPI does NOT address what happens if that fails. Two specific topics are order of messages and resource bounds. There was discussion about whether order preservation is required; that is, for messages from one process to another, messages are received in the order they are sent. Maintaining message ordering is troublesome, but seems essential for conveniently writing reliable portable programs. But then comes the question of what exactly this means, particularly with multithreaded processes! What is the effect of probe on the ordering of messages? Straw vote in favor of requiring order preservation: yes: 23 no: 4 On the issue of correctness with regard to resource exhaustion, Marc Snir suggested the following example: Process 1 Process 2 --------- --------- send to 2 send to 1 recv recv What should an implementation be required to do with this program? On the CM-5 this will always deadlock. On Intel and Meiko machines this will "usually" work (but how does one specify exactly when it will work.) Exchange is an even nastier case. ------------------------------------------------------------------------------ Summary of both Wednesday and Thursday point-to-point subgroup meetings by Marc Snir 1. Multithreaded systems and signal handlers. Should these be of concern to us? No vote was taken, but the general feeling was that we should try to define the various communication calls so that they do not rule out the case where the communicating process is multithreaded. The implications seems to be that all calls should be made reentrant, and the communication subsystem is, from the view-point of the application code, stateless. (With one with one obvious exception, namely that due to posted receive or send buffers, and perhaps additional exceptions having to do with global "modes", like error handling mode. 2. Small or large library? No vote taken. The general feeling is that we should provide many options for the advanced programmer that wants to optimize code (otherwise, all "interesting" codes will use non-portable features, but set the syntax so that the use that uses only the "core" functions need not be burdened by the availability of advanced options. 3. What functions? Clearly, SEND and RECEIVE General sentiment that a combined send-receive would be nice ("most used function on CM"), but discussion postponed until we have a proposed definition: Do we we want an exchange (source=dest), or a 3-body function (source != dest), or allow for both? do we want send_buffer identical to receive_buffer, or disjoint from receive_buffer, or allow arbitrary overlap between the two? What attributes are shared by sent message and received message, if at all? WAIT, STATUS and PROBE functions, and persistent handles are discussed later. 4. What modes? We want blocking and nonblocking sends and receives (blocking -- returns when operation terminated; nonblocking -- returns as soon as possible and a different call is needed to terminate the operation). We want synchronous and asynchronous modes (Synchronous -- operation terminated when terminated at all participating nodes. Asynchronous -- operation terminated when terminated at the calling node; e.g. a send terminates asynchronously when the sender buffer can be reused. Please let me know if you dislike this terminology and prefer something like "local" and "global".) The vote went 17-2 toward having a synchronous SEND (completes when RECEIVE has completed, i.e. when the corresponding WAIT has returned, or STATUS has returned successfully.) We did not discuss whether we want all 4 combinations of blocking-nonblocking and synchronous-asynchronous, or just 3 (blocking synchronous, blocking asynchronous and nonblocking asynchronous). We did not discussed explicitly, but "kind of assumed" that any SEND mode can match any RECEIVE mode. 5. How does one complete a nonblocking operation? The SEND and RECEIVE nonblocking operations return a handle that can be used to query for completion. WAIT(handle) blocks until operation completed; STATUS(handle) returns as soon as possible, and returns an indication for successful completion. In addition, these operations return information on completed RECEIVES: tag, message length, etc. for the received message. The information is returned in a structure provided by the caller. After return of a WAIT or successful return of a STATUS the operation handle is freed; the system has no more information on the completed operation, and has freed all associated resources. A more complex WAIT is needed, that waits for the completion of one out of several pending operations. Proposed syntax is WAIT(array_of_handles) that returns information on which operation succeeded and its parameters (voted 17 to 0). No CANCEL operation -- Once a SEND or RECEIVE is posted, it must complete. (Voted 19 to 7. Some peoples asked to reconsider at least canceling posted RECEIVEs, even if posted SENDs must complete). 6. Additional operations "ready-receive" SEND. SEND with a promise that a matching RECEIVE is already posted (A program where such SEND occurs with no preceding matching RECEIVE is erroneous and, hopefully, the implementation detects this error.) The justification is "it exists on some machine" and "it can improve performance by 25% on Delta". Accepted by 13 against 10. Persistent handles. Created by SEND_INIT(params) (resp RECV_INIT(params). can now be repeatedly used to send/receive messages with these parameters, and then explicitly destroyed. Supported by 19 against 2. PROBE. Allows probing for messages available to receive. Justification - "provides a mechanism to allocate memory to a message of unknown length, before it is received". The proposed mechanism is PROBE(params) that returns a lock to a matching message if there is a matching message that can be received. This message is now locked and can only be received using this lock. This was voted 25 to 0. Some level of uncertainty whether we should also allow to unlock without receiving (why should one want to do this?) 7. What is the buffer argument in SENDs and RECEIVEs? A message is a sequence of values, and as a particular case which is of most interest for homogeneous systems, and for which the syntax ought be simpler, a message is a sequence of bytes. There are various ways of specifying this sequence of bytes. a. Contiguous message: Starting address and length b. Regular stride message: Starting address, number blocks, length of blocks, stride. Voted with no opposition. c. IOVEC: a list of entries, each of which describes a type a or type b message. Voted 18 against 1. There was no discussion on a concrete proposal for typed messages, short of agreement that there should be such. The standard is not going to propose a concrete encoding of typed messages, and a concrete mechanism for message exchange in heterogeneous systems. 8. Matching of SENDs and RECEIVEs. A SEND operation associates with a message the following attributes. a. Sender id. b. Tag c. Context The idea of associated a group id, too, was rejected 11 to 10. The RECEIVE criterion is a Boolean predicate on these attributes of the form. (SENDER_ID = param1) and (TAG = param2) and (CONTEXT = param3). Don't cares are allowed for sender_id and tag, but not for context. Sender_id is determined by system, in the obvious manner, and is absolute (not relative to a group or a context). Tag is under sender control. Context is under sender control, but a yet to be determined mechanism is used to allocate valid context values to processes so as to prevent conflicts. All this was approved with no opposition. The idea of allowing the user to provide its own Boolean function as a receive predicate was rejected 26 to 1 (Reason: "hard to do if the matching is done by a communication coprocessor".) 9. Error handling a. We need a version of MPI where errors generate exceptions (user program halts when an error is detected in an MPI call, or a specific exception handling mechanism is invoked). Voted 19 to 1. b. we need to provide a version of MPI where calls return error codes, and do not cause exceptions, whenever possible. Voted 23 to 0. 10. Ordering of messages Messages sent from the same source to the same destination "arrive in the order they where sent". Voted 23 to 0. The exact implications in terms of order in which RECEIVEs can occur has to be worked out. It was pointed out that this condition may be somewhat hard to define in a multithreaded environment. End of Marc Snir's summary --------------------------------------------------------------------------- Collective Communication Subcommittee --------------------------------------------------------------------------- The Collective Communication Subcommittee was called to order by Al Geist at 4:30 p.m. on Wednesday. It continued until 6:40 p.m. when there was a break for dinner. The meeting resumed at 8:25 p.m. and finally adjourned at 10:10 p.m. Al Geist introduced this as the first meeting, since no real discussion on groups and collective communication took place in Minneapolis. One goal of this committee is to maintain consistency with the point-to-point operations. Any discussion of groups necessarily involves this subcommittee. Collective communication operations can be constructed out of the point-to-point primitives, but are desired because they can be implemented efficiently they are convenient for programmers. The committee then went through the set of collective communication primitives that had been proposed by Al Geist during the email discussions. Broadcast: info = MPI_BCAST(buf,bytes,tag,gid,root) On return, contents of buf for root is in buf for all processes. Al Geist pointed out that the group id here is explicit. Root has to be a member of the group. It was at this point that the committee decided that it would use the word "tag" for message type from now on to distinguish it from "type", which will now always mean type of data. Marc Snir pointed out that for consistency with point-to-point operations, there should be both local termination (the operation returns when the local process has done its part) and global termination (the operation returns when all processes have finished their participation) versions. There followed a discussion of the fact that the point-to-point committee seems to be adopting many different versions of send and receive, and that total compatibility will require many different versions of broadcast. There was a discussion of the reason for the tag parameter in the call. It is needed to disentangle multiple broadcasts occurring at approximately the same time. Paul Pierce described how the system can do this by generating sequence numbers. Others argued that the tag was useful for the programmer in any case, particularly for verifying program correctness. Marc Snir argued that there is a problem (because of the intuition that bcast provides barrier) 1 2 3 send(3) bcast rec(don't care) bcast send(3) bcast rec(don't care) Note that 3 may receive from 2 before 1, i.e. no barrier. Al Geist replied that we need barrier, but broadcast is NOT a barrier. James Cownie initiated a general discussion of whether broadcasts could be received by general receives. This would make it simpler to inherit some of the point-to-point semantics. Al Geist said that broadcast should be consistent with the other collective operations, all of which are symmetric. Paul Pierce suggested we specify collective communication routines in terms of model P-P implementation. This has consequences in terms of what options can be supported. Marc Snir pointed out that one can't actually specify collective communication in terms of point-to-point operations because they need dynamically-allocated additional space. It was decided to postpone a straw vote on whether all processes participating in a broadcast should do "broadcast" or only the root should "broadcast" and the others should "receive" because of concern about remaining issues, e.g. different varieties of recieves. The discussion of "error code" was deferred until the issue is settled in the Point-to-point communication subcommittee. MPI_GATHER: (see mail archives for details) It was proposed to have a version in which each participant contributes a different amount of information (a general "concatenate" function). Issues raised: How handle the situation where the number of bytes on each processor is different. How specify the type of data? For example one needs to know the size of the data type for various purposes, e.g. when doing recursive bisection. MPI_GLOBAL_OP: (see archives for definition) This does not include the data types. There was a discussion of how the forwarding processors know where to break buffers if the data type is not specified. Paul Pierce suggested that we should separate the case of user-defined combining operations from the system ones, which could be optimized. Robert Harrison suggested that the buffer be specified as (#items, length) at least for the user-defined operations. (Tag would be retained) Someone noted that "bytes" would be different on each processor in the heterogeneous case. Back to GATHER. Many agreed that the interface should be changed, but no proposal was offered. Straw vote on having separate general concatenation, to go along with the gather operation: yes: 18 no: 0 MPI_SYNCH There was general agreement that "BARRIER" would be a better name. James Cownie suggested that a tag argument would be helpful for debugging. There was also some discussion of failure of such a barrier, e.g. because some node fails. It was agreed that this was not a problem peculiar to this particular function. One individual nonetheless argued strongly for some kind of timeout for the barrier. Groups ------ gid = MPI_MKGRP (list of processes) There was much discussion of the format of the process list. As defined MKGRP defines a group as a subset of a pre-existing group. One alternative would be to allow creating a group consisting of processes from a number of other groups. (NB Identification of processes is unspecified. This is a task for the Point-to-point Communication Subcommittee.) MKGRP provides an implicit barrier among the processes joining the group. There are a number of problems about making sure that gid is uniform and known across the system. This is an efficiency issue. Should it be possible to SEND to a (gid,rank) pair? Marc argued that one should do Point-to-point communication only within a group, not between groups. Note that groups are constant - cannot add or delete members from a group. Also group creation is a barrier for the processes that are part of the group. This raises the question of how the processes joining the group know that they are joining. What is utility of groups? Certainly at present the only commonly used group is ALL. MPI_FREEGRP(gid) MPI_GRPSIZE MPI_MYRANK There was a general discussion of how group id's would be generated. Also a discussion of the mapping information: How to map back from my_rank and gid to rank in ALL? (In order to actually do a SEND.) ----- At this point the group broke for dinner ----- The continuation after dinner was an informal general discussion. There were some general question about experience from Al Geist to Paul Pierce. Adam Greenberg expressed interest in discussing channels. Channels are seen as an early binding, (Curryification) of various of the SEND/RECV functions which offer a number of gains in efficiency. There was a discussion of Fortran language bindings (F77, F90, HPF) of MPI. It was agreed by those knowledgable in the area that there are no special issues in regard to HPF. Steve Wheat discussed the Sandia implementation of channels on the Ncube. Sounds very similar to iWarp channels except that they are dynamic in creation. Jim Cownie noted that global-ops are going to result in non-determinism in numeric routines. Jim also elaborated on Meiko's BAD experience with ready_receive function - lots of user problems. Commonly user's try it on small problems and it works and speeds up. But then on large problems things erratically break and the user bitches. Paul Pierce noted that this is essentially Intel's force type and the Intel experience has not been so bad. In particular it is harder to use and does not generally work easily on small problems. Cownie: In general what to do when a ready_receive fails? No reasonable way to raise error. Response: Use a signal. Cownie: GAACK! This is implementation and not viable on all systems. John Kapenga listed six collective communication issues that he considers particularly important. [Missed the list] Other desirable collective communication features that were mentioned: global-exchange; all-to-all communication. What are criteria for including? Proposal: Difficulty of implementation; frequency of use; efficiency gain John Kapenga asked about 2-D and 3-D mesh operations - e.g. shifts? Adam Greenberg said this should be left to compilers. John: No Way! Adam argued that the compiler can recognize opportunity to avoid memory copies. Unless that same facility is available to user the compiler can do much better. The group adjourned at 10:10 p.m. --------------------------------------------------------------------------- Topologies Subcommittee --------------------------------------------------------------------------- The Topologies Subcommittee was called to order by Rolf Hempel at 4:00 on Wednesday. It lasted until dinner. --------------------------------------------------------------------------- Other Subcommittees --------------------------------------------------------------------------- The other subcommittees (Introduction, Formal Semantics, Environmental Enquiry, Language Binding) met informally after dinner on Wednesday. --------------------------------------------------------------------------- Meeting of the Whole Committee --------------------------------------------------------------------------- Thursday, January 7, 4:30 The Agenda for the rest of the meeting was presented: Introduction subgroup report Collective-communications subgroup report Process Topology subgroup report Environmental Inquiry subgroup report Formal Language subgroup report Language Binding subgroup report Profiling (Jim Cownie) Dates for future meetings Report of the Introduction Subcommittee: ------ -- --- ------------ ------------ Jack Dongarra presented the results of the subcommittee meeting that took place Wednesday night. This is essentially the draft that has been available from netlib for the last six weeks. There was some on-the-fly editing by the group at large. The goal of the Message Passing Interface simply stated is to develop a *de facto* standard for writing message-passing programs. As such the interface should establishing a practical, portable, efficient, and flexible standard for message passing. Goals ----- Design an application programming interface (not necessarily for compilers or a system implementation library). Allow efficient communication: Avoid memory to memory copying and allow overlap of computation and communication and offload to communication coprocessor, where available. Allow (but not mandate) extensions for use in heterogeneous environment. Allow convenient C, Fortran 77, Fortran 90, and C++ bindings for interface. Provide a reliable communication interface. User need not cope with communication failures. Such failures are dealt by the underlying communication subsystem. Define an interface that is not too different from current practice, such as PVM, Express, P4, etc. Define an interface that can be quickly implemented on many vendor's platforms, with no significant changes in the underlying communication and system software. The interface should not contain more functions than are really necessary. (Based on the latest count of send/receive variants, this drew a large laugh from the crowd.) Focus on a proposal that can be agreed upon in 6 months. Added: Semantics of the MPI should be programming language independent. Who Should Use This Standard? --- ------ --- ---- --------- This standard is intended for use by all those who want to write portable message-passing programs in Fortran 77 and/or C. This includes individual application programmers, developers of software designed to run on parallel machines, and creators of higher-level programming languages, environments, and tools. In order to be attractive to this wide audience, the standard must provide a simple, easy-to-use interface for the basic user while not semantically precluding the high-performance message-passing operations available on advanced machines. What Platforms Are Targets For Implementation? ---- --------- --- ------- --- --------------- The attractiveness of the message-passing paradigm at least partially stems from its wide portability. Programs expressed this way can run on distributed-memory multiprocessors, networks of workstations, and combinations of all of these. In addition, shared-memory implementations are possible. The paradigm will not be made obsolete by architectures combining the shared- and distributed-memory views, or by increases in network speeds. It thus should be both possible and useful to implement this standard on a great variety of machines, including those ``machines" consisting of collections of other machines, parallel or not, connected by a communication network. It was agreed that explicit remarks that MPI is intended to be usable with multithreaded processes and with MIMD (not just SPMD) programs should be added somewhere. What Is Included In The Standard? ---- -- -------- -- --- --------- The standard includes: Point-to-point communication in a variety of modes, including modes that allow fast communication and heterogeneous communication Collective operations Process groups Communication contexts A simple way to create processes for the SPMD model Bindings for both Fortran and C In addition A model implementation and A formal specification. will be provided. It was proposed that explanation and rationale for the standard would also be provided as would sample programs and a validation suite. This is getting very ambitious. Jim Cownie also wants wrappers available for use by, for example, profiling. The suggestion is to provide "name shift", e.g. __MPI_SEND, etc. so the profiler can have MPI_SEND call __MPI_SEND after doing whatever is useful for profiling. What Is Not Included In The Standard? ---- -- --- -------- -- --- --------- The standard does not specify: Explicit shared-memory operations Operations that require more operating system support than is currently standard; for example, interrupt-driven receives, remote execution, or active messages Program construction tools Debugging facilities Tracing facilities Features that are not included can always be offered as extensions by specific implementations. Report of the Collective Communication Subcommittee: ------ -- --- ---------- ------------- ------------ Al Geist summarized the meeting that took place Wednesday afternoon (described above). Global functions beyond those discussed by the subcommittee, such as all2all or total_exchange, await written proposals. The (whole) committee added that Fortran 90 and HPF would be a good place to look for more combining functions (other than max, min, sum, etc.) It was agreed that a way to supply user-supplied functions would be useful. Issues mentioned include: What is a group? How are groups formed? Are group elements addressable, if so how? Are groups ordered (e.g. for prefix/suffix operations)? Group always an ordered subset of the ALL group? Partitioning? Connection with virtual topologies? This will be discussed when topology group reports. Friday, January 8 ------ ------- - Jack Dongarra called the meeting to order at 9:00. Report of the Process Topologies Subcommittee: ------ -- --- ------- ---------- ------------ Rolf Hempel reported on the meeting held Wednesday afternoon: Motivation: Applications have structures of processes Most natural way to address processes Processor topology is valuable to user Creation of subgroups is a natural way to implement topologies Draft proposal for MPI functions in support of process topologies (by Rolf Hempel) is in the handout bundle. The subcommittee made some changes to the draft. What functions should MPI contain? specification of logical process structure lookup functions for process id's clean interface to other parts of MPI (process groups) What should it not contain? any reference to particular hardware architectures algorithms for mapping of processes to processors If it does this, the user program will be portable, but will contain full information for processes mapping at the logical level. Claim: The use of process topologies is not an obstruction to quick implementation of MPI, since the first implementation can make random assignments. A process topology is assigned to a process group. Copying groups can be used to overlay different topologies on the same processes. All processes in a group call the topology definition function. Inquiry functions provide the translation of logical process location to process id. Supported Topologies: General graph structure: For each process, define the complete set of neighbors for each node. In principle this is sufficient as it covers all topologies. But it is not scalable as all processes have knowledge of all others. we should investigate a scalable version. However, important special cases should be treated explicitly, because regular structures can be specified in a scalable way easier to implement the mapping they cover a large number of applications. A special case: Cartesian structures grids/tori hypercube is a special case Support for creation of subgroups for regular structures will be useful. Special treatment for trees? deferred User-defined topology definition functions? deferred It will be necessary for the inquiry functions to provide information on the hardware topology, so that a user can provide his own mapping function. Marc Snir: We need to consider consistency of mapping alignments, for example an octtree for image processing with a grid structure. Al Geist: What is connection between group and topology. Recall that a group is a linear *ordered* array which is a kind of topology. General discussion of copying topologies and groups Proposal is to have at most one topology per group so can use group id as name for topology. This is reason that there must be a group copy. David Walker: We need closer coordination between the collective communication subcommittee and the topology subcommittee, since groups are central to both. Report of the Environmental Enquiry Subcommittee: ------ -- --- ------------- ------- ------------ Bill Gropp reported that the Environmental Enquiry subcommittee needs to wait and get a better picture of what MPI will contain. Jon Flower again asked for cpu_time. This was discussed, and we were reminded that these were more-or-less rejected at the Minneapolis meeting as not being part of MPI. Standardization should come from POSIX. Marc Snir: Part of the subcommittee's job should be to decide *what* can be enquired about as well as how it will be done. There was general discussion about inquiring about both MPI parameters and implementation parameters. Also if parameter *setting* as well as enquiry should be supported. (Buffer pool sizes, for example). Jon Flower also asked about system hints. He suggested it should be possible to tell the system about implementation specific tuning in a system independent way. Report of the Formal Specification Subcommittee: ------ -- --- ------ ------------- ------------ Rusty Lusk reported the committee was without its chairman, Steven Zenith, but that it viewed its mission as to try to formalize what the other subcommittees decide on. It will probably use CSP, for lack of experience with any other formal specification language. Bob Knighten suggested that the subcommittee look into LIS (Language Independent Specification) that POSIX defined in order to separate semantics from language bindings. Report on MPI -1 (minus one) ------ -- ------ ----------- James Cownie presented an MPI anti-specification. Ya hadda be there, but in case you weren't or just want to be reminded, here is a transcription of Jim's slides. MPI -1 (Jim Cownie) In the spirit of LPF (Low Performance Fortran) * Bindings ONLY for Mathematica Occam ML * No function take arguments or returns result * Point to Pointless communication * 1024 different sends NO receives * Full support for 0 dimensional topologies * User data in a message limited to 1 byte (of 6 data types) BUT 1 KByte of TAG, CONTEXT * Informal semantics - Formal Syntax * All groups are contexts * All contexts are groups * Non blocking wait * Non blocking barrier * All user programs are unsafe & erroneous, they therefore do all their work in the exception handler. --------------------------------------------------------------------------- A Profile/Instrumentation subgroup was formed with Jim Cownie as chairman. Steve Otto, as general editor, will contact subgroup chairmen to begin discussion of editing concerns. Discussion of meeting format. The following was proposed as a format for subsequent meetings, based on the experience with this meeting. Wed. afternoon: point-to-point Wed. night: all subcommittees other than pt-to-pt and collective comm. Thurs. morning: collective communication Thurs. afternoon: subcommittee reports Fri. afternoon: subcommittee reports Meeting Dates: It was decided to moved the next two meetings up a week from when they were tentatively scheduled. The next meeting will be Feb 17-19. The next one after that will be Mar 31-Apr 2 The currently-scheduled May 19-21 and June 30-July 2 meetings may also be moved up as well. Note that July 2 will be a holiday in the United States. From owner-mpi-core@CS.UTK.EDU Tue Feb 2 15:48:24 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA28929; Tue, 2 Feb 93 15:48:24 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA09979; Tue, 2 Feb 93 15:47:52 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 2 Feb 1993 15:47:51 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA09971; Tue, 2 Feb 93 15:47:50 -0500 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA18338; Tue, 2 Feb 1993 15:47:49 -0500 Date: Tue, 2 Feb 1993 15:47:49 -0500 From: walker@rios2.epm.ornl.gov (David Walker) Message-Id: <9302022047.AA18338@rios2.epm.ornl.gov> To: mpi-core@cs.utk.edu Subject: Next MPI meeting in Dallas Dear MPI core member, I need to know how many people to expect at the Dallas so please let me know if you intend to show up by sending email to walker@msr.epm.ornl.gov MPI Committee Meeting, Feb. 17-19, 1993 ======================================= The next meeting of the MPI subcommittees will take place at Bristol Suites Hotel 7800 Alpha Road Dallas, Texas The meeting will start at 1pm on Wednesday February 17, 1993, and finish at noon on Friday, February 19, 1993. Rooms are $84 per night and reservations may be made by calling (214) 233-7600 (mention MPI meeting). The meeting registration fee will be $75. Please make checks and POs payable to University of Tennessee. The registration fee will be collected at the meeting. The registration fee will go for coffee breaks, meeting rooms, AV and printer rentals. TBS Shuttle Service will be providing complimentary shuttle service to and from the airports. If you fly into DFW, use their courtesy telephone and dial 03. If you fly into Love Field, you'll have to use a pay phone. They can be reached at 817-267-5150. Upon boarding the shuttle refer to the MPI meeting. We have NOT been able to make any special arrangements with airlines to get reduced fares. If you intend to come to both the February and March/April meetings (March 31-April 2) you might consider buying two tickets now as follows: Ticket #1: Home to Dallas on Feb. 17 Dallas to home on April 2 Ticket #2: Dallas to home on Feb. 19 Home to Dallas on March 31 We have secured limited funding from DARPA/NSF for travel expenses of MPI meeting participants who are from U.S. universities. If you would like to apply for financial support to attend the next MPI meeting please send email to me at walker@msr.epm.ornl.gov, with justification of why you need support and an estimate of your travel expenses. Provisional Agenda for MPI Meeting, 17-19 February, 1993 Wednesday 1:30-6:00 Point-to-Point meeting (everyone) 6:00-7:30 Unofficial dinner break 7:30-10:30 Break up for subcommittee meetings Thursday 9:00-12:00 Collective communication meeting (everyone) 12:00-1:30 Lunch (provided) 1:30-3:00 Full group meeting for presentation of point-to-point subcommittee ideas and proposals. 3:00-6:00 Full group meeting for presentation of collective communication subcommittee ideas and proposals. 6:00-8:00 Dinner (attendees pay, but hotel provids transport to area restaurant) 8:00-10:00 Continued informal subcommittee meetings if necessary Friday 9:00-10:00 Full group meeting for presentation of contexts subcommittee ideas and proposals. 10:00-11:00 Full group meeting for presentation of process topology subcommittee ideas and proposals. 11:00-12:00 Full group meeting for presentation of other subcommittee ideas and proposals. From owner-mpi-core@CS.UTK.EDU Wed Mar 17 09:58:14 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA16424; Wed, 17 Mar 93 09:58:14 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA21476; Wed, 17 Mar 93 09:57:52 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 17 Mar 1993 09:57:52 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA21468; Wed, 17 Mar 93 09:57:51 -0500 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA15962; Wed, 17 Mar 1993 09:57:50 -0500 Date: Wed, 17 Mar 1993 09:57:50 -0500 From: walker@rios2.epm.ornl.gov (David Walker) Message-Id: <9303171457.AA15962@rios2.epm.ornl.gov> To: mpi-core@cs.utk.edu Subject: NEXT MPI MEETING The next meeting of the Message Passing Interface (MPI) Forum will take place in Dallas, March 31-April 2, 1993. Details are given below. The minutes of the last meeting in February are not yet available electronically, but should be soon. The ongoing email discussion on MPI standardization issues can be obtained from netlib. To find out what is available via email send the following message to netlib@ornl.gov: send index from mpi or use xnetlib ("send index from xnetlib" for info). I need to know how many people to expect at the Dallas so please let me know if you intend to show up by sending email to walker@msr.epm.ornl.gov MPI Forum Meeting, March 31-April 2, 1993 ========================================= The next meeting of the MPI forum will take place at Bristol Suites Hotel 7800 Alpha Road Dallas, Texas The meeting will start at 1pm on Wednesday March 31, 1993, and finish at noon on Friday, April 2, 1993. Rooms are $89 per night and reservations may be made by calling (214) 233-7600 (mention MPI meeting). The meeting registration fee will be $75. Please make checks and POs payable to University of Tennessee. The registration fee will be collected at the meeting. The registration fee will go for coffee breaks, meeting rooms, AV and printer rentals. Certain organizations need to see a registration form before giving their employees money for meetings like this. You can get such a form in PostScript by sending the following message to netlib@ornl.gov: send mpi-form.ps from mpi There is no need to send me the registration form or bring it to the meeting. TBS Shuttle Service will be providing complimentary shuttle service to and from the airports. If you fly into DFW, use their courtesy telephone and dial 03. If you fly into Love Field, you'll have to use a pay phone. They can be reached at 817-267-5150. Upon boarding the shuttle refer to the MPI meeting. We have NOT been able to make any special arrangements with airlines to get reduced fares. We have secured limited funding from DARPA/NSF for travel expenses of MPI meeting participants who are from U.S. universities. If you would like to apply for financial support to attend the next MPI meeting please send email to me at walker@msr.epm.ornl.gov, with justification of why you need support and an estimate of your travel expenses. Please send comments and/or suggested changes to the agenda below to me at walker@msr.epm.ornl.gov Provisional Agenda for MPI Meeting, March 31-April 2, 1993 Wednesday 1:30-6:00 Discussion of Snir, Gropp, Lusk point-to-point proposal (everyone) (Snir) 6:00-7:30 Unofficial dinner break 7:30-10:30 Break up for subcommittee meetings Thursday 9:00-12:00 Discussion of Snir & Geist collection communication proposal (everyone) (Otto?) 12:00-1:30 Lunch (provided) 1:30-3:00 Full group meeting for presentation of alternate approaches to groups and contexts, dynamic vs. static process models, and other issues (Volunteer?) 3:00-6:00 Full group meeting for presentation of process topology subcommittee ideas and proposals. (Hempel) 6:00-8:00 Dinner (attendees pay, but hotel provides transport to area restaurant) 8:00-10:00 Continued informal subcommittee meetings if necessary Friday 9:00-11:00 Full group meeting with the intent of taking binding votes on point-to-point and collective communication proposals, or sending proposals back to subcommittees for revision. (Snir?) 11:00-12:00 Full group meeting for defining timetable for producing MPI (or subset) by deadline in July. (Dongarra) From owner-mpi-core@CS.UTK.EDU Wed Mar 17 10:57:03 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA18573; Wed, 17 Mar 93 10:57:03 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA24930; Wed, 17 Mar 93 10:56:27 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 17 Mar 1993 10:56:26 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA24922; Wed, 17 Mar 93 10:56:24 -0500 Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA05362; Wed, 17 Mar 93 10:56:22 EST Received: by b125.super.org (4.1/SMI-4.1) id AA00994; Wed, 17 Mar 93 10:56:22 EST Date: Wed, 17 Mar 93 10:56:22 EST From: lederman@b125.super.org (Steve Huss-Lederman) Message-Id: <9303171556.AA00994@b125.super.org> To: mpi-core@cs.utk.edu Subject: agenda for MPI meeting Hello all, I would like to propose that we have a meeting of all participants at the start of the next meeting to settle the issue of what MPI-1 will cover. Several mail messages have expressed concern that we cannot conclude by June if we try to incorporate everything that everyone wants. Since the scope and time frame will effect most of the other discussions I think it would be best to begin with this. My only concern is that it will take too long to decide the global picture. I would hope it could be done in 1-2 hours and help to refocus the group. Steve From owner-mpi-core@CS.UTK.EDU Wed Mar 17 16:31:07 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA25549; Wed, 17 Mar 93 16:31:07 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA13185; Wed, 17 Mar 93 16:30:04 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 17 Mar 1993 16:30:03 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA13177; Wed, 17 Mar 93 16:30:02 -0500 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA21299; Wed, 17 Mar 1993 16:30:00 -0500 Date: Wed, 17 Mar 1993 16:30:00 -0500 From: walker@rios2.epm.ornl.gov (David Walker) Message-Id: <9303172130.AA21299@rios2.epm.ornl.gov> To: mpi-core@cs.utk.edu Subject: agenda for MPI meeting Regarding Steve's message appended hereto, the basic idea of the agenda I sent out was that we should focus one Wednesday strictly on the concrete pt2pt and collcomm proposals that have been put together by. At these sessions we should not add anything to what has been proposed, though unresolved issues and inconsistencies will no doubt be discussed, and some things may be removed from the proposals. The 1:30-3:00pm meeting on Thursday is meant to look at issues related to the scope of MPI. This meeting might decide that things need to be added to the pt2pt and collcomm proposals or defer defining a more comprehensive MPI till later (after July). Finally, on Friday we have a meeting that ejects or accepts the pt2pt and collcomm proposals, or sends the proposals back to the subcommittees for changes. So that's the rationale behind the agenda. I think it's OK as is, but am willing to be swayed if a reasonable number of people want it changed. David From owner-mpi-core@CS.UTK.EDU Wed Mar 17 16:42:26 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA25693; Wed, 17 Mar 93 16:42:26 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA13806; Wed, 17 Mar 93 16:41:08 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 17 Mar 1993 16:41:07 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from sun2.nsfnet-relay.ac.uk by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA13794; Wed, 17 Mar 93 16:41:03 -0500 Via: uk.ac.southampton.ecs; Wed, 17 Mar 1993 19:47:58 +0000 Via: brewery.ecs.soton.ac.uk; Wed, 17 Mar 93 19:18:07 GMT From: Ian Glendinning Received: from holt.ecs.soton.ac.uk by brewery.ecs.soton.ac.uk; Wed, 17 Mar 93 19:26:26 GMT Date: Wed, 17 Mar 93 19:26:29 GMT Message-Id: <25009.9303171926@holt.ecs.soton.ac.uk> To: mpi-core@cs.utk.edu Subject: Re: agenda for MPI meeting lederman@org.super.b125 (Steve Huss-Lederman) writes: > I would like to propose that we have a meeting of all participants at > the start of the next meeting to settle the issue of what MPI-1 will > cover. Several mail messages have expressed concern that we cannot > conclude by June if we try to incorporate everything that everyone > wants. Since the scope and time frame will effect most of the other > discussions I think it would be best to begin with this. My only > concern is that it will take too long to decide the global picture. > I would hope it could be done in 1-2 hours and help to refocus the > group. We, that is Tony Hey and myself, couldn't agree more. I expressed concern about the lack of focus within MPI in a message to mpi-comm a week ago, and Tony has asked me to add his voice to this. We think it imperative that the next meeting begins by deciding on the scope of MPI, keeping it within reasonable bounds, as was originally intended. The latest draft of the point to point specification by Marc Snir, Bill Gropp and Rusty Lusk seems to us to strike just about the right balance on what to include, though of course there is still room for discussion about details. I'd like to see some form of `secure' send mode, for example, and I'm sure there will be much debate about the context == group idea. We also think some set of collective communication routines definitely falls within the brief of MPI, although we haven't had time to digest the latest draft proposal yet, so have no firm opinions on how appropriate it is. Ian Glendinning and Tony Hey From owner-mpi-core@CS.UTK.EDU Wed Mar 17 17:20:26 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA26228; Wed, 17 Mar 93 17:20:26 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA16096; Wed, 17 Mar 93 17:19:37 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 17 Mar 1993 17:19:36 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA16086; Wed, 17 Mar 93 17:19:35 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA27214; Wed, 17 Mar 93 16:14:45 CST Date: Wed, 17 Mar 93 16:14:45 CST From: Tony Skjellum Message-Id: <9303172214.AA27214@Aurora.CS.MsState.Edu> To: mpi-core@cs.utk.edu, walker@rios2.epm.ornl.gov Subject: Re: agenda for MPI meeting We will have context proposals by then as well, and it would be bad to have the pt2pt accepted without even looking at the context proposals. So, I would say the agenda is sub-optimal, because it does not embrace the concept of reading pt2pt and context at the next meeting, where we had also decided that collcomm would not be read. I do not see a problem with a first reading of collcomm, except that most of my efforts have been focused not on viewing that (since first reading was not expected), but on the context issues. I expect context proposals to be in order by March 24. - Tony From owner-mpi-core@CS.UTK.EDU Thu Mar 18 02:02:19 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA04779; Thu, 18 Mar 93 02:02:19 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA11680; Thu, 18 Mar 93 02:01:49 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 18 Mar 1993 02:01:48 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA11672; Thu, 18 Mar 93 02:01:43 -0500 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA06844 (5.65c/IDA-1.4.4 for ); Thu, 18 Mar 1993 08:00:10 +0100 Received: by f1neuman.gmd.de id AA14757; Thu, 18 Mar 1993 08:01:35 GMT Date: Thu, 18 Mar 1993 08:01:35 GMT From: Rolf.Hempel@gmd.de Message-Id: <9303180801.AA14757@f1neuman.gmd.de> To: mpi-core@cs.utk.edu Subject: Re: agenda for MPI meeting Cc: gmap10@f1neuman.gmd.de Thanks to Steve for his suggestion to start the next meeting with a general discussion on the scope of MPI-1. I completely agree. If we take the current agenda, the risk is that some of the Wednesday discussions on details of PT2PT and COLLCOMM become obsolete by the more fundamental discussion the next day. Rolf Hempel From owner-mpi-core@CS.UTK.EDU Thu Mar 18 02:12:33 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA06092; Thu, 18 Mar 93 02:12:33 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA11980; Thu, 18 Mar 93 02:12:16 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 18 Mar 1993 02:12:15 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA11972; Thu, 18 Mar 93 02:12:14 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA28359; Thu, 18 Mar 93 01:07:25 CST Date: Thu, 18 Mar 93 01:07:25 CST From: Tony Skjellum Message-Id: <9303180707.AA28359@Aurora.CS.MsState.Edu> To: mpi-core@cs.utk.edu, Rolf.Hempel@gmd.de Subject: Re: agenda for MPI meeting Cc: gmap10@f1neuman.gmd.de Yes, it is wise to do things that way. - Tony From owner-mpi-core@CS.UTK.EDU Thu Mar 18 08:50:41 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA10820; Thu, 18 Mar 93 08:50:41 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA05053; Thu, 18 Mar 93 08:49:51 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 18 Mar 1993 08:49:49 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from daedalus.epcc.ed.ac.uk by CS.UTK.EDU with SMTP (5.61++/2.8s-UTK) id AA05045; Thu, 18 Mar 93 08:49:46 -0500 Date: Thu, 18 Mar 93 13:49:39 GMT Message-Id: <9182.9303181349@subnode.epcc.ed.ac.uk> From: L J Clarke Subject: AGENDA! To: walker@rios2.epm.ornl.gov (David Walker), mpi-core@cs.utk.edu In-Reply-To: David Walker's message of Wed, 17 Mar 1993 09:57:50 -0500 Reply-To: lyndon@epcc.ed.ac.uk Hi all I should like to suggest that this agenda is modified such that the session > 11:00-12:00 Full group meeting for defining timetable for producing MPI (or > subset) by deadline in July. > (Dongarra) be the first held, and that the subject of the session should also include formal (and binding) agreement on the scope of this invocation of MPI. I suggest that this is held immediately on Wednesday afternoon. The subjects of process models and dynamic vs static group membership etc should be addressed in this whole committee session. I should also like to suggest that the agenda is further modified such that the session > 1:30-3:00 Full group meeting for presentation of alternate approaches to > groups and contexts, dynamic vs. static process models, and other > issues > (Volunteer?) be held BEFORE the point-to-point and collective subcommittee meetings, and the subject be modified such that it is a discussion of the report and proposal of the context subcomittee, led by the chair of that committee Tony Skjellum. I suggest that this is the first meeting on Thursday morning, which will allow the context subcommittee to meet on the Wednesday evening. I am entirely confident that the justification is transparent. Best Wishes Lyndon /--------------------------------------------------------\ e||) | Lyndon J Clarke Edinburgh Parallel Computing Centre | e||) c||c | Tel: 031 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | c||c \--------------------------------------------------------/ From owner-mpi-core@CS.UTK.EDU Mon Mar 29 00:43:06 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA27148; Mon, 29 Mar 93 00:43:06 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA17449; Mon, 29 Mar 93 00:42:14 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 29 Mar 1993 00:42:12 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from ssd.intel.com by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA17393; Mon, 29 Mar 93 00:42:03 -0500 Received: from tualatin.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA25755; Sun, 28 Mar 93 21:41:57 PST Date: Sun, 28 Mar 93 21:41:57 PST Message-Id: <9303290541.AA25755@SSD.intel.com> Received: by tualatin.SSD.intel.com (4.1/SMI-4.0) id AA14795; Sun, 28 Mar 93 21:41:53 PST From: Bob Knighten Sender: knighten@SSD.intel.com To: mpi-core@cs.utk.edu Subject: YES! MINUTES (and before the next meeting.) Reply-To: knighten@SSD.intel.com (Bob Knighten) Below are the minutes from the last MPI Forum. I am initially sending these to the mpi-core mailing list to give a short opportunity for you to correct grevious errors in them before they are posted to the full MPI mailing list. I will post them widely on Tuesday morning. -- Bob Robert L. Knighten | knighten@ssd.intel.com Intel Supercomputer Systems Division | 15201 N.W. Greenbrier Pkwy. | (503) 629-4315 Beaverton, Oregon 97006 | (503) 629-9147 [FAX] ============================================================================== Minutes of the Message Passing Interface Standard Meeting Dallas, Texas February 17-19, 1993 The MPI Standards Committee met in Dallas on February 17-19, 1993, at the Bristol Suites Hotel in North Dallas. This was the fourth meeting of the MPI committee and the second of the now regular meetings in Dallas. There were both general meetings of the committee as a whole and meetings of several of the subcommittees. Because interest in the Point-to-Point communications and the Collective communications was so general, these met as committees of the whole. No formal decisions were taken at this meeting, but a number of straw votes were taken in the subcommittees. These are included as part of the reports on the work of the subcommittees. These minutes were taken by Rusty Lusk (lusk@mcs.anl.gov) and Bob Knighten (knighten@ssd.intel.com). These minutes are quite long. If you want to see the important topics you can search for --- and this will quickly the lead to each topic (and a few other things.) Wednesday, February 17 --------- ----------- ------------------------------------------------------------------------------- General Meeting ------------------------------------------------------------------------------- Jack Dongarra called the meeting to order at 1:30, There was a discussion of the agenda. Other topics included the possibility of some DARPA funding and a tutorial for Supercomputing '93. The next meeting will be March 31-April 2 at same place (the Bristol Suites in Dallas). The following meetings are tentatively scheduled for May 12-14 and June 23-25. Bob Knighten proposed that we set a definite schedule, particularly if we are going to follow the example of the HPF committee. This was discussed more fully on Friday. (Search for "Schedule" below.) Attendees: --------- Joe Baron IBM Austin jbaron@vnet.ibm.com Harry Scott Berryman Yale Univ. berryman@cs.yale.edu Lyndon Clarke EPCC, U. Edinburgh lyndon@epcc.ed.ac.uk James Cownie Meiko jim@meiko.co.uk Jack Dongarra UT/ORNL dongarra@cs.utk.edu Vince Fernando NAG & UC Berkeley fernando@jaguar.berkeley.com Jon Flower ParaSoft jwf@parasoft.com Daniel Frye IBM-Kingston danielf@kgnvma.vnet.ibm.com Al Geist ORNL gst@ornl.gov Adam Greenberg TMC moose@think.com Bill Gropp ANL gropp@mcs.anl.gov Leslie Hart NOAA/FSL hart@fsl.noaa.gov Tom Haupt Syracuse U. haupt@npac.syr.edu Don Heller Shell Development heller@shell.com Rolf Hempel GMD hempel@gmd.de Tom Henderson NOAA/FSL hender@fsl.noaa.gov Steven Huss-Lederman SRC lederman@super.org John Kapenga Western Michigan U. john@cs.wmich.edu Bob Knighten Intel SSD knighten@ssd.intel.com Rik Littlefield PNL rj_littlefield@pnl.gov Rusty Lusk ANL lusk@mcs.anl.gov Peter Madams nCube pmadams@ncube.com Alan Mainwaring TMC amm@think.com Oliver McBryan U. Colorado mcbryan@cs.colorado.edu Barney Maccabe Sandia abmacca@cs.sandia.gov Dan Nessett LLNL nessett@llnl.gov Steve Otto Oregon Graduate Institute otto@cse.ogi.edu Peter Pacheco U. of San Francisco peter@sun.math.usfca.edu Howard Palmer nCube hep@ncube.com Paul Pierce Intel prp@ssd.intel.com Sanjay Ranka Syracuse U. ranka@top.cis.syr.edu Peter Rigsbee Cray Research par@cray.com Mark Sears Sandia mpsears@cs.sandia.gov Anthony Skjellum Mississippi State U. tony@cs.msstate.edu Marc Snir IBM, T.J. Watson snir@watson.ibm.com Alan Sussman U. of Maryland als@cs.umd.edu Bob Tomlinson LANL bob@lanl.gov Dennis Weeks Convex weeks@convex.com Stephen Wheat Sandia NL srwheat@cs.sandia.gov Stephen Ericsson Zenith Kuck & Associates zenith@kai.com The group then became a committee of the whole to meet as the Point-to-Point Communications Subcommittee. ------------------------------------------------------------------------------- Point-to-Point Subcommittee ------------------------------------------------------------------------------- Marc Snir opened point-to-point subcommittee meeting, and asked for discussion of his draft ("Point-to-Point Communication" by Marc Snir, Feb 8, 1993; this is also included in the overall MPI draft dated February 16, 1993.) He asked about additions to his draft. Cancel was mentioned and was discussed later. Alignment of "sequence of bytes" buffers: --------- -- --------- -- ------ ------- Marc began discussion of the draft by asking about alignment. There followed a discussion of whether messages of type "sequence of bytes" should be restricted to be of length a multiple of 4 or 8, or should be aligned. Jim Cownie proposed that we also vote on requiring that all data types start on their natural boundaries. We decided that this was too restrictive, given that some Fortran compilers do not deliver this. Straw vote: String of bytes can start on any byte address and can be of any ---------- length (including 0.) Yes: 34 No: 0 After a question from Bob Knighten, it was agreed that bytes have 8 bits. (Bob Knighten reminded us that few other standards require that. Named Constants for Options: ----- --------- --- ------- Discussion of whether we should use named constants or specific values for various options. Fortran 77 does not specify an "include" facility. Straw vote: Use named constants? ---------- Yes: 27 No: 2 Scott Berryman spoke in favor of using fixed constants, because of existing Fortran practice. Joe Baron also spoke in favor. This was deferred to the language binding committee. Structure of Buffer: --------- -- ------ Note the message is a sequence of bytes (at this point). There is no requirement that the structures on send and receive side match. (Can move contiguous area into scattered area, etc.) Paul Pierce raised the issue that we need to be clear about whether we want to use *real* iovecs or not. General discussion of how general the datavec should be. Tony Skjellum spoke in favor of generality (mentioned BLAS, which don't have most general striding, so people invent their own.) Jim Cownie suggested that we include data type in the descriptor vectors (that is, there would always be a data type, which might be "byte"). Bill Gropp suggested that the data descriptor vector be an opaque data type, both to help Fortran binding, to allow taking advantages of real iovecs where they apply, and to allow extensibility. Baron mentioned the FORTRAN favorite way of specifying these vectors, with a[b[i]]. This was postponed to the language binding committee. (Note: a concrete proposal by Gropp and Lusk on how data description vectors might be handled was made later, and is described below.) Receive Criteria: ------- -------- (See proposal) selection by tag and by source, and by context. Tony Skjellum proposed that an AND mask be available to deal with ranges of bits in the types and sources. Scott Berryman objected to "bit-twiddling" in standards. Truncation of buffer: ---------- -- ------ Rolf spoke in favor of allowing buffer to be longer than the message received, although some thought this should be an error. There was general agreement that "too short messages" should not be erroneous. Al Geist said truncation (messages too long) should be an error. Paul Pierce said that experience was on this side. Jim Cownie said that we should be clear about whether the first part of a truncated message appears in the buffer. Marc Snir observed that standards seldom specify the behavior of an erroneous program. Straw vote: Matched messages must fit in buffer, otherwise an error. ---------- Yes: 26 No: 0 Send and Receive: ---- --- ------- Marc Snir noted that all the point-to-point operations can be defined in terms of the four low level operations: INIT, START, COMPLETE, FREE. See Section 1.5 (Communication Handles) of Snir's document. Discussion of how restrictive the init routines should be (difference between Snir and Gropp-Lusk proposals.) Much discussion of efficiency of handle creation and modification, whether handle would be in system space or user space,and whether there should be default values. Some discussion of whether we should have this Level 1 (of the Gropp-Lusk multilevel proposal) at all. (General agreement that it should.) Handles: Do we have handles? Do handles have default values? Are ------- handles malleable? Adam Greenberg argued that modifiable handles make channels much harder. They are hard to start with. [NOTE: If so, then must be a single "create handle with attributes" operation.] Straw vote: Unmodifiable handles? ---------- Yes: 1 No: the rest Bill Gropp suggested that handles could be created, then repeatedly modified, and finally "committed to", after which time they cannot be modified without recommitment before use. This allows creation to proceed by modifying defaults, and some sort of compilation to take place on the commit operation. Jim Cownie proposed a "dup" function for handles. Straw vote: This section should be rewritten to allow modification, ---------- followed by commit operation. (Can even modify and reuse repeatedly.) Yes: 28 No: 0 What about defaults. One argument against defaults - want to be able to catch unset fields. Gropp/Lusk went with defaults because extensibility and requiring setting of attributes of handles are conflicting as new attributes break old programs. Also did not want to be able to create handles that cannot be used. Straw vote: Have Defaults ---------- Yes: 20 No: 5 Parameters of a handle: buffer start mode Some more discussion of ready receive semantics. Marc says he added ready-send semantics more for symmetry, but also to allow for the "pull" model of communication, to go along with the currently prevalent "push" model. General discussion followed about whether it provides a way to write erroneous programs. It was pointed out that the call always works; on machines which don't support a special protocol it merely provides no performance improvement. But on some machines the ready-receiver semantics of send provides a performance win. Ready-receiver (send message with fore knowledge that receiver has posted the receive) passed narrowly last time, so another vote was taken. Straw vote: Have ready-receive? ---------- Yes: 20 No: 12 When does an operation complete? (1.5.4 of Snir proposal) ---- ---- -- --------- -------- Adam Greenberg began by asking whether we need synchronous mode (send does not complete until receive has completed at the other end) if we are assuming reliable communication. General discussion of whether we need synch mode. Lusk retracts previous arguments in favor of synch mode. Greenberg asks why. Only argument in favor is that it can use no buffering. Jim Cownie pointed out that it is a way of forcing the effect of "no system buffering". Paul Pierce argued that there be a global method of specifying that no system buffering is to be used. Straw vote: Have synchronous mode? ---------- Yes: 10 No: 15 As a consequence section 1.5.4 goes away. Extracting information from handle on completion: (Section 1.5.6) ---------- ----------- ---- ------ -- ---------- Separate COMPLETE-RECEIVE and COMPLETE-SEND? Al Geist: separate improves clarity. Jim Cownie: same allows for waiting on a set of completions of mixed sends and receives. Marc Snir: the completes have different parameters. Paul Pierce: could have query between complete and free, so that complete could have same parameters. Jim Cownie spoke in favor of this: it is similar to the modifying the attributes of a handle and then committing it. But he proposes that the user pass to the complete routine an area (in user space) where all the parameters are stashed. Marc says that the parameters then become different again, since these structures are not the same. Jim: it still could be. Rehash of commit and handles in general. Should the commit return a system handle? Straw vote: Have a separate query function. ---------- Yes: 21 No: 3 Straw vote: Have a single complete function? ---------- Yes: 23 No: 0 Rik Littlefield noted that the phrase at the top of page 12: "Note that it is correct, but inefficient, to implement MPI_CHECK via a call to MPI_COMPLETE, in which case, MPI_CHECK always returns true." is wrong. This will change to remove comment at top of p. 12 that says mpi_check might block. Higher-Level Operations: (Section 1.6) ------------ ---------- Now that synchronous operations have been discarded,there are only 6 operations for each send and receive, as opposed to 12. Jim Cownie objected to in/out arguments. Discussion: Jim Cownie reminded us of "solution number 5": (For more information see minutes of last meeting.) 1: handle + inquiry. 2: pass in arguments where unwanted information is 5: pass in structure where things are stashed (in user space) (out arguments are replaced by one structure pointer to opaque structure) This is also thread-safe. Three proposals: in/out arguments, multiple parameters, opaque structures. Straw vote: Do not use in/out arguments? ---------- Yes: 32 No: 2 Straw vote: Package the arguments in a structure rather than using list of ---------- arguments. Yes: 24 No: 8 The Point-to-Point Communication Subcommittee meeting ended at this point. It continued on Thursday. ------------------------------------------------------------------------------ General Meeting ------------------------------------------------------------------------------ There was a brief general meeting with Marc Snir presiding. Short discussion of next meeting dates: Next meeting March 31, April 1 - 2 as scheduled 6 weeks later is May 12 - 14; this was approved. and June 23 - 25 was approved. The subcommittees meeting Wednesday night were: Collective, Topology, Context, Introduction, Environment and Profiling. ______________________________________________________________________ ______________________________________________________________________ Some subcommittee meetings took place Wednesday evening. Reports on those meetings are part of the General Meeting minutes for Friday. ______________________________________________________________________ ______________________________________________________________________ Thursday, February 18: -------- -------- -- Marc reminded us that we need to move quickly toward readings and approval --------------------------------------------------------------------------- Collective Communication Subcommittee --------------------------------------------------------------------------- Al Geist opened the collective communication discussion at 9:15. He reviewed where we were at the end of the last meeting and urged people to send in written proposals. Broadcast: --------- Discussion of syntax of broadcast -- do both the sender and receivers of a broadcast call mpi_bcast or do the receivers call mpi-receive. If the receive must handle broadcasts, it puts an extra burden on them. Suggestion that there are applications, such as discrete-event simulation, where it would be convenient if broadcasts were received by normal receives. Marc Snir: Do we have the same data types as for point-to-point messages? Al Geist: Yes. Discussion of whether the broadcast should be synchronous or not. Straw vote: Broadcast not required to be synchronous? ---------- Yes: 11 No: 5 (There were 37 people in the room.) Straw vote: Have a non-blocking broadcast? ---------- Yes: 21 No: 7 A non-blocking broadcast returns as soon as possible, but buffers are invalid until the operation is complete (as verified by some inquiry routine.) Buffer options will be like point to point. Straw vote: Broadcast is received by broadcast, not receive. ---------- Yes: unanimous Barrier: ------- Discussion of tag on barrier. Jim Cownie suggested a non-blocking barrier, so that one could initiate a test for whether processes have reached a certain point, and test later. Non-blocking barrier: Each process entering the barrier posts that fact. There is an inquiry function to check that everyone in group has entered barrier. Non-blocking barriers would then *require* tags, since one could be participating in multiple barriers. Scott Berryman said that he has had to implement this. Adam Greenberg noted that CM-5 implements in hardware (with limitations). Users would like with fewer limitations. Straw vote: Should we have a non-blocking barrier? ---------- Yes: No: 1 Straw vote: Should we have a tag for both blocking and nonblocking barriers? ---------- Yes: 31 No: 1 gather/concat: ------------- There is a need for a gather-then-broadcast. Discussion of whether there should be in and out buffers on the gather. Observation that buf is an IN/OUT parameter and for root is actually used both IN and OUT. Proposal to separate the IN and OUT buffers. General discussion: How much buffer space and where needed. GENERAL_GATHER(in_buf, out_buf, bytes, out_bytes, tag, context, gid) out_bytes = 0 indicating if result is to be delivered in out_buf Weeks: Note that bytes must be the same in every call. If non-blocking, one call may return without any error indication even though call is erroneous. Weeks: Proposal to specify out_bytes with it being 0 if result buffer fills in out_buf. In any case should have in_buf and out_buf because root is using buf for two very different purposes. Greenberg, Flower: Agree but for different reasons. Alternative proposal: GENERAL_GATHER(in_buf, out_buf, bytes, flag, tag, context, gid) flag indicates if result is to be delivered in out_buf Oliver McBryan: Question of whether gen_gather should replace concat. Straw vote: Use two buffers (inbuf, outbuf), not a single in/out-buf on gather ---------- Yes: 33 No: 0 There is a problem of when it is know where flag is set. A straw vote was proposed, but Adam Greenberg argued that we need further discussion before voting and the vote was postponed. Straw vote: Have a gather function to a single "root node"? ---------- Yes: 32 No: 0 Straw vote: Have an all-to-all gather? ---------- Yes: 26 No: 0 Straw vote: Should we have any further discussion of including flag? ---------- Yes: 9 No: 15 [So absent a new proposal flag is out.] Straw vote: Have non-blocking versions of gather and all-to-all? ---------- Yes: 18 No: 5 Straw vote: Have a gen_gather and all-to all, with different buffer size ---------- on each process. Yes: 29 No: 0 Creating and Freeing Groups: -------- --- ------- ------ Al Geist opened discussion on creating and freeing groups. He pointed out that this is still tangled up with process id's and contexts. mkgroup (list of processes, specified somehow) returns group id. called by all processes, Cownie and McBryan proposed that all in a group call this, passing flag to say whether they want to join new group or not. Rolf says that we don't want to have to have topologies to have groups. Marc pointed out that the "flag" version subsumes the list version, given that everyone calls it, and pointed out that some sort of global synch is desirable in order to have system-global gids. Discussion of whether gids are globally known (i.e. known to all processes in a group) and valid or not. gids could be valid only in one process only among members of a group system wide Discussion of only making subgroups vs. creating union of groups. Discussion of whether we want to deal with dynamic process creation or not. In dynamic situation, there is no common ancestor for group creation. Tony suggested that people are going to want dynamic groups. MPI must be competitive with PVM. Paul Pierce called for a concrete proposal, since dynamic processes goes considerably beyond what we have seriously considered so far. Oliver McBryan proposed a special join operation for forming unions of groups. Marc proposed that we vote at least on an operation on partitioning an existing group by key. Straw vote: Should MPI have an operation to partition an existing group? ---------- Yes: 32 No: 0 Should we throw out the list version? Straw vote: Should MPI only provide partitions of an existing group? ---------- Yes: 10 No: 17 [The alternative is to keep the list form of mkgroup.] This was the end of the Collective Communications Subcommittee meeting. ------------------------------------------------------------------------------- Point-to-Point Subcommittee ------------------------------------------------------------------------------- Marc Snir called the Point-to-Point Subcommittee to order at 1:30 p.m. He started with a review of the previous day and reminded us that we were now discussing Level II rather than Level I. (Page numbers from here on in the minutes refer to the "Draft Document for a Standard Message-Passing Interface" of February 16, 1993 [prepared and distributed by Steve Otto] whereas previous page numbers referred to the draft "Point-to-Point Communication" by Marc Snir, Feb 8, 1993.) Discussion of how to get information about completion of calls: opaque structures used for both blocking and non-blocking. Discussion that a program in which if there is a posted send on one process and a matching posted receive on another node, then the operations will eventually complete. Jim Cownie proposed that there be only one type of handle, and then only one kind of wait: wait(handle) together with query functions. Discussion of having a uniform wait routine. We don't have concrete proposals for what the query functions would look like. wait(handle, opaque_return) query (opaque_return,.... ) or could have layered higher-level specialized query function. Tony Skjellum pointed out that you will first have to query the opaque_return to find out what the type of the handle is, in order to determine which query function to use on it. (Applicable to wait_any.) Straw vote: wait_send, wait_recv with different parameters? ---------- Yes: 2 No: 29 (The preferred alternative is to have a uniform wait, handles, opaque_return and query function(s)). Steven Zenith asked that the word "alternation" be replace by "choice" - this was accepted without further discussion. Waiting on Set of Events (wait_any): (pages 20-21). ------- -- --- -- ------ -------- The two functions discussed are wait_any and wait_all. Wait_any completes a single operation, and the handle is freed. There was a discussion of which handle is selected and the issue of fairness. It was observed that it would be responsibility of programmer to pass always a list of valid handles. Another possibility would be to have wait_any modify the list of handles on return (and would change the matched handle to a magic "null" handle that would match nothing but always be accepted.) What would happen if only these null values were passed? Rik Littlefield suggested that the in/out argument problem doesn't apply here, so it would be better to have the handle list modified. The specific freed handle replaced by NULL, say. Straw vote: wait_any(list, index, opaque_return) returning index and ---------- opaque_return with index identifying the handle returned Yes: 28 Con: 1 Straw vote: "null" handles (for deletion from list)? ---------- Yes: 19 No: 3 Straw vote: wait_any to set handle matched to null? ---------- Yes: 21 No: 6 (Bob Knighten pointed out that this simplifies handling of a shared list when there are multiple threads.) Is all-null an error or is it a no-op? Postponed. Straw vote: Should we have a wait_all? ---------- Yes: 15 No: 9 wait_all(list-of-handles, list-of-opaque-returns) thus approved. Bill Gropp suggested that we should worry about an error during this operation. Should we have a wait_all for *all* operations? Jim Cownie suggested that if we know what contexts are, we might want to have a wait_all for all events in the context. Other questions: What happens with multiple wait_all with overlapping lists? What happens in a multithreaded environment? Probe: ----- We want to receive a message without knowing its length, for example. (probe returns the envelope of the message and locks the message). Then you can receive it. We should also have an unlock operation. This is a different sort of handle than at level 1, since at level one the buffer has been associated with the handle, but for this one you learn about the buffer. Therefore this section (middle of p 21. ) should have "handle" replaced by, say, "lock". This requires a separate receive in order to receive on a lock. Also the in/out parameters should have input parameters, plus an opaque return object. The revised functions are: mpi_probe(source, tag, context, opaque_return, lock) mpi_precvx(lock,...) mpi_unlock(lock) Peter Rigsbee suggested that there be a wait version and a status version. (wait version doesn't return until it returns with a lock). Discussion of blocking, nonblocking version of the precvx. Jim Cownie argued against unlock, since probing is sort of a contract to receive the message. There was a counter that it is often desirable to decide NOT to read this message but rather toss it back into the pool. What to do about message order after unlocking? Several possibilities: head of queue, original location, tail of queue, unspecified. PROBE clearly perturbs the order of receipt of messages. Skjellum and Lusk argue that the only reason for this is because of other problems - we ought to fix those problems instead. (The problems here are the specification of fixed size buffers all over the place rather than just providing the needed buffers to the extent possible.) Steve Wheat proposed that we do away with the lock and the new receive. Have PROBE(source,tag,context,info) and then using a blocking receive to get the particular message the probe located. Dealing with thread safety can be dealt with by using critical regions. Note that you cannot use this to look at the entire message queue. Paul Pierce proposed that "you get the buffer and give me the address" be one of the buffer types. Summarize: 1. lock, unlock precv 2. probe gives you info, then you receive from that tag-source. 3. get rid of probes. and mpi gets buffer for you. Buffer Descriptors: ------ ----------- Bill Gropp described a buffer-descriptor proposal, in which there is a function create_bd( ... ) returning a buffer-descriptor. It is then possible to append different kinds of descriptors. The idea is to be able to build arbitrary structures for mixed data types and gather-scatter operations. The append operations might look like: bd_contig(bd, address, datatype, numitems) (so can build mixed-type heterogeneous messages) bd_stride(bd, address, datatype, stride, numitems {, itemlen?}) for strided data bd_abi( bd, address, index_array, datatype, numitems) for indirect address vectors {abi stands for A[B[I]]} free_bd(bd). or have send free it. Note probably want to reuse, so free should be explicit. Al Geist noted that this is for sophisticated users, so likely can get by with just bd_stride. Data types? Gropp's current view is that these are only primitive data types, not derived data types, e.g. no structures. Can build buffer descriptor for structure by multiple calls. It would be nice to have a program/function that would do that for the programmer. That would not be hard. Note that this is very different from existing practice. There should certainly be a level close to existing practice. After a discussion of this, there was a Straw vote: Get a fleshed-out proposal for this? ---------- Yes: 29 No: 2 Straw vote: Should there also be a simple version for contiguous messages ---------- of fixed types (to better conform to existing practice)? Yes: 28 No: 3 This was the end of the data type discussion and Marc Snir took the floor again. Cancel: ------ Someone brought up that all the operations for which we can wait, require perhaps a cancel. Jim Cownie argued (again) that the worrisome case is the wait for an outstanding nonblocking receive. MPI_cancel(handle) guarantees that the buffer will not be written on. Here is a sample program that illustrates the usefulness of cancel: IRECV(...) REPEAT(...) WAIT(HANDLE) . . . IRECV(...) UNTIL(converge) CANCEL Berryman: this could be handled with an appropriate use of tags. Snir: Cancel send could be done with free_handle. Pierce: That should be an error. Marc withdrew the suggestion. This led us into a discussion of whether MPI will require a fixed number of processes. Note that introduction does not discuss the (fixed number of processes) requirement. Correctness: (p. 21) ----------- What is a correct MPI program? What is done with erroneous MPI programs? Review of message-order preservation. In the case of threads, there may not be an order to the messages. So *if* there is an order on messages, it is preserved. Note that this is for *matching* receives. What about receive any with two messages from same source? They should be received in order. There is also a fairness question. Paul Pierce said that the important order is the order in which the receives are posted, not the order of the receives themselves, so that messages land in the correct buffers. General agreement. Jim Cownie brought up the fairness issue with respect to receive-any. Progress and Fairness: -------- --- -------- This brings up the resources issue. Discussion of minimum resource requirements. number of handles, etc. Bob Knighten proposed that the bounds be implicit in the test suite. It was proposed that there be an appendix to describe implementation profiles, which will be an agreement on what an implementation will try to support. Mark Snir's current document attempts to specify the weakest possible requirement - no system buffering buffering is explicitly required. Rather there is just the requirement that if a matching send and receive have been posted, the the operation will complete. Marc noted that this can be extended to collective communications. Oliver McBryan suggested that the user could supply some space to the system that it could use for MPI, even on machines that supply no buffering. Rik Littlefield suggested that the user could provide the buffer space to the system, and declare its requirements. This discussion was deferred until there is a concrete proposal from the Environment Subcommittee (which has renamed itself from the Environmental Subcommittee to the Environmental Management Subcommittee). Error Handling: ----- -------- Snir: Two communities: program writers and system programmers System programmers not likely to write in Fortran. Single mechanism may not be suitable across languages. One approach, especially appropriate for Fortran application programmers: an error should bring the system down and maybe help you debug, the other, for system programmers writing in C, test return codes for errors. Marc suggested that the default be to blow up. Jim Cownie suggested the solution of having an alternate set of routines, and Rick Littlefield pointed out that this is the only possible thread-safe mechanism. Paul Pierce suggested having the syntactic alternatives, as in NX. It was suggested that the error-handling mode should be attached to the levels. It was proposed that C routines return negative values on error, while Fortran routines are used to having extra out parameter. Marc summarized the alternatives: both f77 and C always return error code. only C 2 different libraries can select to signal when error occurs At what granularity? per job per context Straw vote: F77 code should {always/never} return an error code? ---------- Always: 22 Never: 3 Straw vote: Should there be alternate libraries to select between these ---------- alternatives? Yes: 6 No: 18 Without debate it was assumed that the same result would prevail in a vote for C. Straw vote: Can select (in some way) what happens when an error occurs? ---------- In F77: Yes: 21 No: 6 Without debate it was assumed the same vote would prevail in C. ------------------------------------------------------------------------------- General Meeting ------------------------------------------------------------------------------- Friday, February 19: ------ -------- -- The meeting started at 9:00 with: Report of the Context Subcommittee: ------ -- --- ------ ------------ Tony Skjellum reported on the meeting of the previous night (see notes above.) There is a Contexts Draft 1.0 that is available. Contexts are a partition of the tag space for matching, no wild cards on context. Contexts chosen by users, tags by users. MPI_NEW_CONTEXT is executed by one process to obtain a context. It is then broadcast to those who need it. If there is a group all there is an associated context, to deal with bootstrap problem. Pairwise message ordering is preserved within context. The context subcommittee will try to come next time with proposals that are consistent and resolve the circular interaction among groups, contexts, and process identifiers. Rick Littlefield noted that there is no mechanism for statically created context. He prefers static name_server model. Discussion of whether groups can be used to replace contexts. Marc Snir pointed out that the real question is that of another parameter one sends, and whether we call it gid or context is irrelevant. Rolf Hempel pointed out that the intentioned use is quite different, so we should have both concepts, like we do selectivity on source although it can be encoded in the tag, Steven Wheat said that his users understand contexts quite clearly as types you can't wildcard, while groups are more confusing. Jim Cownie suggested that there is a definite bootstrap problem with getting the context to the processes that need it. Tony Skjellum said that in zipcode contexts are associated with groups, so obtaining a context is a synchronized, group operation. John Kapenga spoke in favor of a way for a group to obtain an associated context. Marc Snir spoke in favor of the name-server approach for libraries. Rik LIttlefield said that it would be nice to have guidelines on how to write an MPI-safe library. Marc Snir summarized that the extra match field can be: separate in the tag (tag range registration) in the pid (send to ports instead of processes) Paul Pierce summarized issues that need to be addressed in concrete proposals: are groups local, are contexts global or local how to implement service. General agreement that the context subcommittee will produce a new white paper clarifying these issues. Report of the Process Topologies Subcommittee: ------ -- --- ------- ---------- ------------ Rolf Hempel reported on the work of the Process Topologies Subcommittee. (See the proposal in the "Draft Document for a Standard Message-Passing Interface" of February 16, 1993) the Process Topologies and Collective Communication Subcommittees met jointly, as both deal with groups. Division of Responsibilities: -------- -- ---------------- Process Topologies Collective Communications topology group creation basic group creation group partitioning along group partitioning by key coordinate lines Topology Functions: -------- --------- Topology definition function always creates a new group/ advantage: ranks in parent group so not change ranks in new group are aligned with the Topology supported topologies: agreed on cartesian structures (grids, tori) arbitrary graphs This is all we should aim at for MPI-1. We can do trees later. {McBryan: What is relation of order to topology? There should be translation functions. What is order of a tree?} Standard case: MPI decides which process in group gets which position in topology. {Geist: Does this mean the topology must be encoded in the gid? Hemple: No.} Additional option: User assigns topology position to each process explicitly. (Marc Snir will write a proposal) Mapping: ------- MPI implementation may try to efficiently place processes. Option: user can explicitly ask for random mapping. This would be more efficient. Also could be used to explicitly request a random pattern of communication. This might be redundant, since it could be a user-requested mapping. {Random or arbitrary? Random. Why random? Because it is sometimes the correct answer. May reduce contention over any systematic placement.} Indexing in MPI: -------- -- --- There is a general problem in MPI: How are n object numbered? 0, 1, ..., n-1 C style 1, 2, ..., n Fortran style For inter-language compatibility we should pick one, but this issue was deferred to the Language-Binding Subcommittee. Applications: ------------ rank in group node numbers in graph structures MPI_WAITANY there was a possibility of having this alternative selectability, but Tony pointed out that this breaks libraries. Straw vote: MPI to number objects using the C convention (0, ..., n-1)? ---------- Yes: 32 No: 1 (So (0,0) is the first element of a two-dimensional arrays.) Another issue, row-major vs. column-major in arrays. visible in ranks in groups and order in buffering. Alignment of groups and subgroups. General discussion. is (0,1) the second element or (1,0) the second element? Discussion of usefulness of elaborate mappings, and whether vendors will offer support for this. Otherwise, topologies are lightweight (function mesh(i,j) returns process id) Snir: General comments on proposal. One purpose is to renumber processes so that they can be more efficiently placed. But will this actually be used by any vendor. Hempel: This is not part of the standard and creating a new group solves other problems. Snir/McBryan advantage of topology is ability to express communication in terms of topology. Minor advantage. Geist: Relation of collective communication to topolgy? Want SHIFT to be relative to topology? But argument is a gid and so how is topology encoded? Topologies are implemented on top of (augmented) groups. I.e. the group associated with a topology can "know" the process order for "shift left". Marc Snir suggested that Topology functions be local (who is my left neighbor, right neighbor) McBryan: Sounds like this should be in a library, not in the language. Hempel: MPI is not a language. Snir: Need an interesting example of a system where this will actually be used, otherwise this is only a convenience feature. Cownie: Based on experience with Parmacs on machines where this placement is important. Have vendors moved onto machines where this is no longer relevant. Hempel: It could be a serious mistake to believe that situation with newest machines shows that placement is not relevant to future very large machines. Skjellum: With very large machines, the entire model will have to change. Topology is not the right way to provide information. Not relevant to program running several distinct kernels. Snir: Global communication vs. local communication. Berryman: Do not accept idea that topology is not relevant. User needs to be aware of machine topology and able to use this in program. Jim Cownie pointed out that on machines where the processes are on the nodes of the switch network, and there there is an important performance benefit to mapping correctly. But new machines are getting away from this, and process placement may become less of an issue. Machines are becoming "flatter". End of Rolf's discussion of topologies. MPI Tutorial at Supercomputing `93: --- -------- -- ------------------ Rusty Lusk asked those interested in participating in a tutorial on MPI at Supercomputing '93 contact Jack Dongarra or himself. Validation of MPI: ---------- -- --- Oliver McBryan suggested that we have an effort to write, port, provide and share applications programs for MPI. Rusty Lusk noted the ongoing implementation by Bill Gropp and himself that will allow people to test applications soon. There will be an effort to port some HPF programs to MPI. Schedule: -------- Bob Knighten asked about the schedule. Marc said that we would have a reading about the language-independent stuff, and the C and fortran bindings separately. The point-to-point proposals will be ready for the next meeting. Profiling should be ready. Introduction should be ready. Others should be ready for first reading at the following meeting. Report on Environmental Management Subcommittee (Bill Gropp): ------ -- ------------- ---------- ------------ three classes of routines MPI Parallel-related Non-MPI useful, but not specifically MPI, like high-resolution timers. 3 routines agreed on (no syntax) mytid numtid validtags Management Hints: ---------- ----- provide requested value return actual value for implementation limits and characteristics. Exact choices of items that can be managed not determined yet. We aren't doing error handling. Report of the Language Binding Subcommittee ------ -- --- -------- ------- ------------ Scott Berryman reported on the Language Binding Subcommittee. This subcommittee has been biding its time. There will be a "Thou shalt not" list on the network soon. A proposal for standards used and exceptions allowed will be presented at next meeting. For F77 the basic proposal will be F77 plus long names plus underscores plus include. We will vote on these at the next meeting. John Kapenga asked whether we want to say anything about I/O? Jon Flower pointed out that we need some minimal requirement, driven by the need to write a test suite. Bill Gropp pointed out that minimally we should be able to run this program: if (master) printf ("hello"). Jim Cownie and Rusty Lusk said that all one really needs is a requirement that at least one node be able to do stdio. Marc Snir pointed out that there needs to be an enquiry function to find out which node can do I/O. ------------------------------------------------------------------------------ MPI -1.1 ------------------------------------------------------------------------------ Jim Cownie presented MPI -1.1. A revision from -1.0 was required since many of the concepts humorously presented there have now been adopted into MPI (e.g., non-blocking barrier.) Design approach: macc o (opposite of Occam!) h "Entia sunt multiplicanda." Objective: To be as complex as possible with no coherent subsets. Developments since MPI -1.0 Non blocking barrier remove to MPI-1 Handles added - needed for opening doors NPROCS redefined - now guaranteed never to return the same answer twice. Number of collective routines increased - need more to keep all procs busy. Another two versions of all functions probably erroneous & guaranteed erroneous All errors are opaque (following industry practice) Non blocking exit added {But can it be canceled?} Preserved from MPI -1 All groups are contexts All contexts are groups Environmental management: I require the hardware to be ... I require the vendor to be ... ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ Thursday night there were meetings of at several subcommittees. Notes from the meeting of the Communication Contexts Subcommittee are included below. ------------------------------------------------------------------------------ Communication Contexts February 18, 1993 Anthony Skjellum opened the discussion [Started with 12, another 10 or so came in about 9:20] Contexts are to make it feasible to build scalable software that can be mixed. TAGS - unstructured bits (at least 32) 1. definition of tags 2. matching of tags CONTEXT - unstructured integer, system assigned NEW_CONTEXTS(number_of_contexts, array_of_contexts) number specified, array returned FREE_CONTEXTS() CONTEXTS_AVAILABLE() minimum in system >= 16K contexts are gotten by one process and then distributed to others Contexts -------- 1) Avoid crossing messages between libraries and user code Select: by source, by tag (and mask), by context - "safe" Relation to groups and global operations Use of contexts in a 3-D grid model: context for each full two dimensional array section => big numbers Alternative need for multiple name spaces apart from groups: Wheat: Server with multiple contributors to a package of data. Also, context as a means of separating stages in a software pipeline without need barrier. Cownie: Proposes using many fewer contexts a very fast implementation by using an array of queues with indexing via the context. An alternative to context is using copies of group ids, but then need point-to-point routines to be aware of groups, i.e. able to select on gids. Purpose of contexts is to keep separate part of program from interfering without preventing users from doing anything they want with tags. Context as an "endpoint of communication" (Cownie as "queue index"). Connection with groups, i.e. collective communications. Lusk: What are cost/complication of using instead tag registration, i.e. the user requests a range of tags? Change in "don't care". Rik Littlefield agreed with need for large number of contexts - situations where intersections of groups are complicated. Paul Pierce argued that semantics of using a part of tag as context implies enough difference that there is likely only a syntactic distinction. Sandia - user's group had no problem with idea of context; groups caused them a great deal of confusion and disinterest. Some situations, e.g. separating software libraries, need only a very small number of contexts, but using contexts to separate intersecting groups can lead to situations where you need many contexts. Skjellum asks Cownie about performance impact. What about using index for small number and switching for large number? Well . . . Paul Pierce: "IF" in performance critical code is always a problem. There followed a discussion of the relationship between contexts and the problem of implementing collective operations using P-P operations. Paul Pierce pointed out that some implementations will want to do this! [Intel will do this; Meiko will not do this.] Rik Littlefield proposed to have context associated with a "code package" in some fashion, then structured use of tags within the code is sufficient. For this to work we need to have some essentially static assignment of context. But only modest number of contexts needed. Groups vs. contexts again. What is a pid? Etc. Zipcode bases everything on groups. Proposal: Make groups and contexts the same. [Of course this is part of MPI -1, rejected last time] ------------------------------------------------------------------------------ ============================================================================== From owner-mpi-core@CS.UTK.EDU Tue Apr 6 20:21:12 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA29037; Tue, 6 Apr 93 20:21:12 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA23665; Tue, 6 Apr 93 20:20:30 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 6 Apr 1993 20:20:28 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from ssd.intel.com by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA23646; Tue, 6 Apr 93 20:20:25 -0400 Received: from tualatin.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA10804; Tue, 6 Apr 93 17:20:22 PDT Date: Tue, 6 Apr 93 17:20:22 PDT Message-Id: <9304070020.AA10804@SSD.intel.com> Received: by tualatin.SSD.intel.com (4.1/SMI-4.0) id AA00545; Tue, 6 Apr 93 17:20:19 PDT From: Bob Knighten Sender: knighten@SSD.intel.com To: mpi-core@cs.utk.edu Subject: Brief summary of votes taken at the MPI Forum meetings Reply-To: knighten@SSD.intel.com (Bob Knighten) What follows is a summary of all of the votes take at the MPI. All of the Formal Votes (all from the March-April meeting of last week) are listed first, followed by the Straw Votes in reverse cronological order. If there is interest, I will elaborate this list to provide fuller explanation of the individual votes and will update it after future meetings. Please let me know if there is such interest. Even more important, please let me know of errors in this list. -- Bob Knighten ============================================================================== March-April Meeting Formal Votes Separate mpi_create for each type Yes: 20 No: 0 Abstain: 4 MPI_FREE is valid even if handle is in use, but effect is to free the object when the operation completes. Yes: 7 No: 8 LIST_LENGTH explicit (rather than included in array or EOL)? Yes: 22 No: 0 Accept Section 2.2? Yes: 23 No: 1 BYTE data type? Yes: 24 No: 0 Accept Section 2.5 (minus 2.5.2) Yes: unanimous Proposals for units of bytes and for consistency. 1. Units are elements 2. bytes in C and elements in F77 (Yes: 0 No: 14) 3. bytes everywhere (and provide some way of getting size of basic types) a) index stays as is, but bytes elsewhere Yes: 9 Con: 13 (vote 1) b) truly bytes everywhere, including indices Yes: 10 Con: 8 (vote 2) In vector, stride may be negative? Yes: 10 No: 2 Allow repetition? Yes: 13 No: 1 Can multiple components overlap? vote not taken Total length is an integer multiple of the block size Yes: 4 No: 10 Commit? Yes: 11 No: 6 Accept 2.6 Yes: 12 No: 4 Typed DONTCARE? Yes: 19 No: 1 (1) RECEIVE(..., tag, info, ...) no DONTCARE for tag (2) RECEIVE(..., tag, mask, ...) no DONTCARE for tag (3) RECEIVE(..., tag, ...) DONTCARE for tag (1) fails for lack of second. (2) Yes: 6 No: 12 so (3) remains. Do we want a "secure" communication mode? Yes: 7 No: 6 Delete send/ready_receive? Yes: 10 No: 10 [Amendment tabled] Accept amended versions of 2.7 & 2.8 Yes: 25 No: 1 The return status for a receive operation should be the number of bytes received Yes: 17 No: 3 Abstain: 7 Accept 2.9 (with amendments) excluding 2.9.1 and WAIT_ANY Yes: 19 No: 2 Abstain: 3 Add both blocking and immediate secure-send/secure-receive with failure of program to match being erroneous. Yes: 10 No: 8 Abstain: 9 2.10 as amended Yes: 20 No: 4 Abstain: 3 2.11 Yes: 26 No: 0 Abstain: 1 Use exactly the same parameters in MPI_ADD_BLOCK as amended Yes: 26 No: 0 Abstain: 1 Have strided block message functions. Yes: 5 No: 9 Abstain: 12 Include non-blocking c-c? Yes: 0 No: uncounted Abstain: 9 Remove tag in all c-c? Yes: 15 No: 5 Abstain: 5 Accept just semantics of 3.4 Yes: 23 No: 0 Abstain: 2 Allow 2nd cshift with only one buffer. No partial overlap on orig cshift Yes: 21 No: 0 Abstain: 6 Accept cshift, eoshift proposals with amendment of 2nd form of each. Yes: 20 No: 2 Abstain: 4 Accept bcast Yes: 22 No: 0 Abstain: 2 VOTE: Provide Hxx functions; Use the element count except for ---- displacements in Hxx buffer components. Yes: 19 No: 0 Abstain: 3 ============================================================================ March-April Meeting Straw Votes Topologies in MPI? If yes integrated with groups (e.g. eoshift), OR separate library, environmental inquiry In: 25 Out: 4 Abstain: 5 integrated: 4 Separate: 23 Abstain: 7 ============================================================================ February Meeting POINT-TO-POINT Straw Votes Should all data types be required to start on their natural boundaries? No by acclamation Can messages have 0 length? Yes by acclamation Can a string of bytes start on any byte address and be of any length (including 0)? Yes: 34 Con: 0 Unanimous that byte types can start on any alignment. Unanimous that bytes have 8 bits. Use named constants? Yes: 27 No: 2 Must matched messages fit in buffer? (Otherwise an error.) Yes: 26 No: 0 Unmodifiable handles? Only one in favor Section ?? should be rewritten to allow modification, followed by commit operation. (This includes use modification and recommit.) Yes: 28 No: 0 Defaults? {for what} Yes: 20 No: 5 Ready-receive? Yes: 20 No: 12 Synchronized functions? Yes 10 No 15 (So 1.5.4 goes away) Separate query functio(s)? Yes: 21 No: 3 Single complete function? Yes: 23 No: 0 Three proposals: inout arguments, multiple parameters, opaque structures. inout arguments?: Yes: 2 No: 32 Lists of out arguments and in arguments?: Yes: 24 No: 8 Straw vote: wait_send, wait_recv with different parameters? Yes: 2 No: 29 (The preferred alternative is to have a uniform WAIT, handles, opaque_return and query function(s)). WAIT_ANY? Yes: 28 No: 1 Do we want null-value handle (so can delete)? Yes: 19 No: 3 Do we want wait_any to set handle matched to null? Yes: 21 No: 6 Is all-null an error or is it a no-op? Vote postponed. Should we have a WAIT_ALL? Yes: 15 No: 9 In favor of get full form of buffer-descriptor presented by Bill Gropp? Yes: 29 No: 2 Should there also be a simple version for contig messages of fixed types, in order to conform more to existing practice? Yes: 28 No: 3 User can select (in some way) what happens when an error occurs? in f77 Yes: 21 No: 6 in C assume same in C ============================================================================== February Meeting COLLECTIVE COMMUNICATION Straw Votes Should broadcast be synchronous? Yes: 5 No: 5 (37 people in room) Non-blocking broadcast? (A non-blocking broadcast returns ASAP, but buffers are invalid until the operation is waited on.) Yes: 21 No: 7 broadcast is received by broadcast, not receive. Unanimous Non-blocking barrier? Yes: 27 No: 1 Include tag for both blocking and nonblocking barrier. Yes: 31 No: 1 Flag to indicate whether gather receives data (on non-root)? Postponed Later: Yes: 9 No: 15 Function that gathers to root of operation? Yes: 32 No: 0 All-to-all gather? Yes: 26 No: 0 Non-blocking versions of gather and all-to-all? Yes: 18 No: 5 Can gather and all-to all have different buf size on each process? Yes: 29 No: 0 At least provide an operation that partitions an existing group by key? Yes: 32 No: 0 Only partition existing group or keep list form of mkgroup? Only partition: 10 keep list form: 17 ============================================================================== February Meeting MPI FORUM Straw Votes Numbering of objects: 32-1, in favor of 0,,N-1 (Therefore (0,0) is the first element of a two-dimensional arrays.) ============================================================================= January Meeting POINT-TO-POINT Straw Votes Support for synchronous point to point communication? Yes: 17 No: 2 Wait for a vector of handlers (this is an OR operation)? Yes: 17 No: 0 Cancel in the initial MPI? Yes: 7 No: 19 Ready_receive? Yes: 13 No: 10 Write point-to-point operations in terms of init, start, wait and free? Yes: 19 No: 2 Probe on handle as lock? Yes: 25 No: 0 User-defined select criteria? Yes: 1 No: 26 May context modify pid? NO Context? YES Selection by group? Yes: 10 No: 11 Buffer with stride? YES IOVEC buffer? Yes: 18 No: 1 Do not want send in homogeneous environment to require the type information need for the heterogeneous environment? YES General send is a send-bytes? YES Option so that MPI routines cause exception on error? Yes: 23 No: 0 Option to have MPI routines set error code? Yes: 19 No: 1 Maintain order of messages? Yes: 23 No: 4 ============================================================================== January Meeting COLLECTIVE COMMUNICATION Straw Votes General CONCAT function with the number of bytes different on each node? Yes: 18 No: 0 ============================================================================= Robert L. Knighten | knighten@ssd.intel.com Intel Supercomputer Systems Division | 15201 N.W. Greenbrier Pkwy. | (503) 629-4315 Beaverton, Oregon 97006 | (503) 629-9147 [FAX] From owner-mpi-core@CS.UTK.EDU Wed Apr 7 10:42:11 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA12294; Wed, 7 Apr 93 10:42:11 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA02644; Wed, 7 Apr 93 09:55:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 7 Apr 1993 09:55:07 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from fslg8.fsl.noaa.gov by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA02627; Wed, 7 Apr 93 09:55:04 -0400 Received: by fslg8.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA05355; Wed, 7 Apr 93 13:54:59 GMT Received: by macaw.fsl.noaa.gov (4.1/SMI-4.1) id AA11477; Wed, 7 Apr 93 07:53:39 MDT Date: Wed, 7 Apr 93 07:53:39 MDT From: hender@macaw.fsl.noaa.gov (Tom Henderson) Message-Id: <9304071353.AA11477@macaw.fsl.noaa.gov> To: mpi-core@cs.utk.edu Subject: Re: Brief summary of votes taken at the MPI Forum meetings Bob Knighten writes, > What follows is a summary of all of the votes take at the MPI. All of the > Formal Votes (all from the March-April meeting of last week) are listed > first, followed by the Straw Votes in reverse cronological order. > > If there is interest, I will elaborate this list to provide fuller > explanation of the individual votes and will update it after future > meetings. Please let me know if there is such interest. > > Even more important, please let me know of errors in this list. > > -- Bob Knighten Bob, Thanks for getting the vote summary out so quickly! I would like to see more detailed explanations of the individual votes, when you find time to add them. I'd like to check the details on one of the more critical votes: > ============================================================================== > > March-April Meeting > Formal Votes > > > VOTE: Provide Hxx functions; Use the element count except for > ---- displacements in Hxx buffer components. > Yes: 19 No: 0 Abstain: 3 My understanding is that MPI_ADD_BLOCK(), MPI_ADD_VEC(), and MPI_ADD_INDEX() would remain as in the March 25 draft (all "lengths", "strides" and "indices" measured in "elements"). Two new routines, MPI_ADD_HVEC() and MPI_ADD_HINDEX(), would be added. MPI_ADD_HVEC() is identical to MPI_ADD_VEC() except that "stride" is measured in bytes. MPI_ADD_HINDEX() is identical to MPI_ADD_INDEX() except that "indices" are byte offsets. Is this correct? Tom From owner-mpi-core@CS.UTK.EDU Fri Apr 30 09:55:33 1993 Received: from CS.UTK.EDU by surfer.EPM.ORNL.GOV (5.61/1.34) id AA12529; Fri, 30 Apr 93 09:55:33 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA13002; Fri, 30 Apr 93 09:54:56 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 30 Apr 1993 09:54:55 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA12994; Fri, 30 Apr 93 09:54:54 -0400 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA18916; Fri, 30 Apr 1993 09:54:52 -0400 Date: Fri, 30 Apr 1993 09:54:52 -0400 From: walker@rios2.epm.ornl.gov (David Walker) Message-Id: <9304301354.AA18916@rios2.epm.ornl.gov> To: mpi-core@cs.utk.edu Subject: Next MPI meeting The next meeting of the Message Passing Interface (MPI) Forum will take place in Dallas, May 12-14, 1993. Details are given below. The minutes of the last meeting in March/April are not yet available electronically, but should be soon. The ongoing email discussion on MPI standardization issues can be obtained from netlib. To find out what is available via email send the following message to netlib@ornl.gov: send index from mpi or use xnetlib ("send index from xnetlib" for info). I need to know how many people to expect at the Dallas so please let me know if you intend to show up by sending email to walker@msr.epm.ornl.gov MPI Forum Meeting, May 12-14, 1993 ================================== The next meeting of the MPI forum will take place at Bristol Suites Hotel 7800 Alpha Road Dallas, Texas The meeting will start at 1pm on Wednesday May 12, 1993, and finish at noon on Friday, May 14, 1993. Rooms are $89 per night and reservations may be made by calling (214) 233-7600 (mention MPI meeting). The meeting registration fee will be $75. Please make checks and POs payable to University of Tennessee. The registration fee will be collected at the meeting. The registration fee will go for coffee breaks, meeting rooms, AV and printer rentals. Certain organizations need to see a registration form before giving their employees money for meetings like this. You can get such a form in PostScript by sending the following message to netlib@ornl.gov: send mpi-form.ps from mpi There is no need to send me the registration form or bring it to the meeting. TBS Shuttle Service will be providing complimentary shuttle service to and from the airports. If you fly into DFW, use their courtesy telephone and dial 03. If you fly into Love Field, you'll have to use a pay phone. They can be reached at 817-267-5150. Upon boarding the shuttle refer to the MPI meeting. We have NOT been able to make any special arrangements with airlines to get reduced fares. We have secured limited funding from ARPA/NSF for travel expenses of MPI meeting participants who are from U.S. universities. If you would like to apply for financial support to attend the next MPI meeting please send email to me at walker@msr.epm.ornl.gov, with justification of why you need support and an estimate of your travel expenses. Please send comments and/or suggested changes to the agenda below to me at walker@msr.epm.ornl.gov Provisional Agenda for MPI Meeting, May 12-14, 1993 Wednesday 1:30-6:00 Discussion of communication contexts proposal(s) (Skjellum) 6:00-7:30 Unofficial dinner break 7:30-10:30 Break up for subcommittee meetings Thursday 9:00-12:00 Second reading of the point-to-point draft (Snir) 12:00-1:30 Lunch (provided) 1:30-4:30 First reading of the collective communication draft (Geist) 4:30-6:00 Full group meeting for presentation of the integration of process topologies into the current draft (Hempel) 6:00-8:00 Dinner (attendees pay, but hotel provides transport to area restaurant) 8:00-10:00 Continued informal subcommittee meetings if necessary Friday 9:00-10:30 Full group meeting to revisit contexts, and to select a single proposal if the approach to be taken in MPI has not been decided earlier. (Skjellum) 10:30-12:00 Full group meeting for reports from the other subcommittee chairs. (Dongarra) >From owner-mpi-core@CS.UTK.EDU Mon Jun 7 10:22:41 1993 Received: from convex.convex.com by mozart.convex.com (5.64/1.28) id AA17701; Mon, 7 Jun 93 10:22:39 -0500 Received: from CS.UTK.EDU by convex.convex.com (5.64/1.35) id AA03317; Mon, 7 Jun 93 10:20:07 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA01429; Mon, 7 Jun 93 11:12:36 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 7 Jun 1993 11:12:34 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA01419; Mon, 7 Jun 93 11:12:33 -0400 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA18426; Mon, 7 Jun 1993 11:12:32 -0400 Date: Mon, 7 Jun 1993 11:12:32 -0400 From: walker@rios2.epm.ornl.gov (David Walker) Message-Id: <9306071512.AA18426@rios2.epm.ornl.gov> To: mpi-core@cs.utk.edu Subject: Next MPI meeting Status: R The next meeting of the Message Passing Interface (MPI) Forum will take place in Dallas, June 23-25, 1993. Details are given below. The minutes of the last meeting in May are not yet available electronically, but should be soon. The ongoing email discussion on MPI standardization issues can be obtained from netlib. To find out what is available via email send the following message to netlib@ornl.gov: send index from mpi or use xnetlib ("send index from xnetlib" for info). The schedule for future meetings is as follows: August 11-13, 1993 Dallas September 22-24, 1993 Dallas October 27-29, 1993 Europe MPI Forum Meeting, June 23-25, 1993 =================================== The next meeting of the MPI forum will take place at Bristol Suites Hotel 7800 Alpha Road Dallas, Texas The meeting will start at 1pm on Wednesday June 23, 1993, and finish at noon on Friday, June 25, 1993. Rooms are $89 per night and reservations may be made by calling (214) 233-7600 (mention MPI meeting). The meeting registration fee will be $75. Please make checks and POs payable to University of Tennessee. The registration fee will be collected at the meeting. The registration fee will go for coffee breaks, meeting rooms, AV and printer rentals. Certain organizations need to see a registration form before giving their employees money for meetings like this. You can get such a form in PostScript by sending the following message to netlib@ornl.gov: send mpi-form.ps from mpi There is no need to send me the registration form or bring it to the meeting. TBS Shuttle Service will be providing complimentary shuttle service to and from the airports. If you fly into DFW, use their courtesy telephone and dial 03. If you fly into Love Field, you'll have to use a pay phone. They can be reached at 817-267-5150. Upon boarding the shuttle refer to the MPI meeting. We have NOT been able to make any special arrangements with airlines to get reduced fares. We have secured limited funding from ARPA/NSF for travel expenses of MPI meeting participants who are from U.S. universities. If you would like to apply for financial support to attend the June MPI meeting please send email to me at walker@msr.epm.ornl.gov, with justification of why you need support and an estimate of your travel expenses. Please send comments and/or suggested changes to the agenda below to me at walker@msr.epm.ornl.gov Provisional Agenda for MPI Meeting, June 23-25, 1993 Wednesday 1:30-6:00 First Reading of the contexts draft. (Skjellum) 6:00-7:30 Unofficial dinner break 7:30-10:30 Break up for subcommittee meetings Thursday 9:00-12:00 First reading of the process topologies draft. (Hempel) 12:00-1:30 Lunch (provided) 1:30-4:30 Second reading of the collective communication draft (Geist) 4:30-6:00 Second reading of the profiling draft (Cownie) 6:00-8:00 Dinner (attendees pay, but hotel provides transport to area restaurant) 8:00-10:00 Continued informal subcommittee meetings if necessary Friday 9:00-10:30 First reading of the subset draft. (Huss-Lederman) 10:30-12:00 First reading of the formal specifications draft. (???) From owner-mpi-core@CS.UTK.EDU Fri Jun 18 14:37:35 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA10139; Fri, 18 Jun 93 14:37:35 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA13430; Fri, 18 Jun 93 14:37:02 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 18 Jun 1993 14:37:01 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA13422; Fri, 18 Jun 93 14:36:57 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA12781; Fri, 18 Jun 93 14:37:23 EDT Received: by b125.super.org (4.1/SMI-4.1) id AA01213; Fri, 18 Jun 93 14:37:23 EDT Date: Fri, 18 Jun 93 14:37:23 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9306181837.AA01213@b125.super.org> To: mpi-core@cs.utk.edu Subject: IAC chapter Hi All, Attached is the LaTeX version of the "subset" chapter. It is scheduled to be presented on Friday of the upcoming meeting. I thought I would pass it along in case anyone wants to read it before then. A LaTeX and a PostScript copy of this file are available via anonymous ftp from ftp.super.org in pub/mpi. If you are on the IAC subcommittee this is the same as the copy I send out yesterday (Thursday). I look forward to seeing you soon in Dallas. Steve ---------------------------------------------------------------------- \documentstyle[12pt]{article} \newcommand{\discuss}[1]{ \ \\ {\small {\bf Discussion:} #1} } \newcommand{\missing}[1]{ \ \\ {\small {\bf Missing:} #1} } \begin{document} \title{ Initial Implementation Subset } \author{Steven Huss-Lederman} \maketitle \section{Initial Implementation Subset} \subsection{Introduction} \par This chapter defines a minimal subset of MPI for initial implementation. This subset is being defined so that consistent implementations can appear more rapidly. It was recognized early in the process that MPI needed to appear as quickly as possible and practical. The creation of a subset will hopefully allow users earlier access to the standard and still allow for the writing of portable message passing codes. \par This subset should not in any way be construed as a limitation on MPI implementations. It is strictly the minimum necessary to have an initial MPI subset conforming implementation. It is hoped that an officially sanctioned subset will encourage and allow implementors to introduce MPI in a more timely fashion. It should be noted, however, that implementation of the subset does not make an implementation conform with MPI. The subset is only a potential first step in the process. All implementations must ultimately conform to the entire standard; implementations are encouraged to do the full standard as rapidly as possible. \par The subset presented is consistent with the complete MPI standard. This was an important goal so the additional MPI features could be added without changing any functionality from the user's perspective. Thus, users can be assured that programs written now for the subset will run without modification under the full MPI standard. Furthermore, using additional features of the full MPI standard in the future will not require changes to code already written. Users may use MPI features outside this subset that are offered by various implementors. However, people who require portability during the early development of MPI may experience some difficulties until later in the development process. \subsection{Criteria and Rational} \par Having the subset be consistent with the entire MPI standard was considered critical to the effort. In addition to this, there were many possible criteria to use in determining the elements of MPI to include in the subset. The main criteria used were that the MPI subset should \begin{enumerate} \item contain routines that are as close as possible to current standard practice to minimize the effort to port codes. \item contain as many new and important MPI features as possible. \item allow developers to be able to have the subset show up as rapidly as possible while still meeting the other criteria. \end{enumerate} \par There were many rationales for these criteria. It was recognized that it is impossible to come up with an ideal subset and compromises were necessary. It was felt that current users should be comfortable in migrating to MPI. Thus, the subset should contain the message passing elements that represent common practice today. Furthermore, it was felt that the initial (and later) version of MPI should maintain efficiency. Otherwise users could suffer from disenchantment with MPI on their first usage and may never give the standard a second try. Having the standard routines in the subset should encourage developers to optimized these routines in the initial version of MPI. Also, though some felt that the portability offered by the MPI subset was sufficient to entice users, many felt that the subset should contain some new feature(s) over and above current common practice as an added incentive. The additional items selected were meant to represent some of the significant new features of MPI. Balanced against these first two goals is the need for the subset to show up in a timely fashion. Thus, each feature chosen for inclusion in the subset was deemed of sufficient importance to outweigh its added complexity in implementing the subset. Though some functions seem easy to implement there are often overlooked costs in testing, documentation and development that was considered before a feature was added to the MPI subset. Inclusion of too many routines might lead to moderately efficient implementation of them all instead of a very efficient and possibly more useful implementation of a smaller subset. \discuss{ Some felt that we should aim the subset at typical users. Should this be another criteria? Is the compatibility with current practice enough? } \discuss{ Should it be a criteria that the subset and common practice features be implemented efficiently. The text touches on this but it is not a specific item. I did not give it a bullet since it seems like a quality of implementation issue. Poorly implemented subsets will be forced out by the market. } \discuss{ Should it be a criteria that the rest of MPI be able to be layered on top so users might get early (though lower performance) access to the complete standard? This is aimed at allowing easy and efficient layering of the rest so that a ``machine independent'' package could be created without having the developer jump through lots of hops. This might happen if a few functions were left out that didn't meet the other criteria. However, some felt it was not a strong enough reason to leave it in and my lead to poor MPI implementations of these layered routines. } \subsection{Subset Functionality} \discuss{ This section only discuss the pt-to-pt and collcomm chapters. The other chapters will be added after they are read at meetings and finalized. } \subsubsection{Itemized Functionality} The Table below summarizes the specific functionality included and excluded from the subset. \goodbreak \vskip 0.2truein \def\chpt{1} \def\chcol{2} \vbox{ \halign{\hfil#\hfil\quad &\hfil#\hfil\quad &\hfil#\hfil\quad &\hfil#\hfil \cr Section&Operation&Included&Excluded \cr \noalign{\kern 2pt} \noalign{\hrule} \noalign{\kern 10pt} \multispan 4 \hfil \underline{Point to Point Communication} \hfil \cr \noalign{\kern 4pt} \chpt.5.2&Buffer Operations&``All'' functions&Multiple appends, h versions(?) \cr \chpt.7&Comm. Mode&Standard&Ready, Synchronous \cr \chpt.8.1&Comm. Object Creation&All(?)& \cr \chpt.8.2&Comm. Start&START(if have INIT)& \cr \chpt.8.3&Comm. Completion&All& \cr \chpt.8.4&Multiple Completion&All& \cr \chpt.9&Blocking Comm.&Std. version&Ready, Synch. version \cr \chpt.10&Nonblocking Comm.&Std. version&Ready, Synch. version \cr \chpt.11&Contiguous Buffer Comm.&Std. version&Ready, Synch. version \cr \chpt.12&&PROBE&CANCEL,GET\_LEN \cr \noalign{\kern 10pt} \multispan 4 \hfil \underline{Collective Communication} \hfil \cr \noalign{\kern 4pt} \chcol.4&Synchronization&All& \cr \chcol.5&Data Move&All(?)& \cr \chcol.6&Global Compute&MPI Ops&User Ops \cr } } \vskip 0.2truein \noindent All other general functionality is included in the subset. Thus, all matching criteria, order, etc. are part of the subset. \subsubsection{Point to Point Functionality} The operations involving contiguous buffers are included since they represent standard practice today. In addition, the communications buffers are also included because they offer an important new feature in MPI. However, to simplify the implementation, the communication buffer will be limited to one component, e.g., only one append is allowed. This limitation will still allow the user access to the strided and indexed cases but require more complex cases to be done by the user. \discuss{ If we eliminated communication buffers then the number of routines would be greatly reduced. We would only have the ``C'' routines in collective communications and contiguous in pt-2-pt. Right now I think we should leave it in but I point this out } \discuss{ The heterogeneous cases (hvector, hindexed) seem very easy to code but are another set of functions. They allow elements of a C type structure to be pulled out in the subset. It has been pointed out that even simple routines have a cost of testing, documentation, etc. that can easily outweigh the coding cost. } Of the three types of communication modes, the STANDARD mode represents standard practice and is included. The other two modes are not used as commonly and therefore excluded. This applies to all of the point-to-point communications routines. The common versions of blocking and nonblocking send and receive are included. In addition, the low level routines of INIT and START are included. These should help users who wish greater control over communications for optimization purposes. For example, this would allow for the reuse of a persistent handle. \discuss{ Using INIT, START, COMPLETE, FREE is not what most people do today but does allow for greater speed. It seems as if you would have this in a system anyway and it would be useful to the user. Should we include it? } \subsubsection{Collective Communication Functionality} The collective communication routines have the same general restrictions as the point to point routines. Therefore, the buffer descriptors will be limited to one entry. A collective communication operates on a group. This defines the scope of the operation. Many current users only operate on all the processes. This is the MPI\_ALL group in MPI. To simplify the subset, groups will be limited to be MPI\_ALL. \discuss{ There are a lot of data move functions. I could not think of a way to draw a line to exclude some. Do we want all of these in the subset. How often are ALLCAST, ALLSCATTER used? } \discuss{ For the subset do we want both the REDUCE and ALLREDUCE versions? } \end{document} From owner-mpi-core@CS.UTK.EDU Fri Jun 18 15:30:32 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA10665; Fri, 18 Jun 93 15:30:32 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA17439; Fri, 18 Jun 93 15:30:38 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 18 Jun 1993 15:30:38 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from gw1.fsl.noaa.gov by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA17428; Fri, 18 Jun 93 15:30:36 -0400 Received: by gw1.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA02358; Fri, 18 Jun 93 19:31:03 GMT Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA09313; Fri, 18 Jun 93 13:34:27 MDT Date: Fri, 18 Jun 93 13:34:27 MDT From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9306181934.AA09313@nipmuc.fsl.noaa.gov> To: mpi-core@cs.utk.edu Subject: MPI-2? Hi All, There have been a few references to MPI-2 in discussions at the meetings. Not any formal references, but in discussion someone may say "that's an issue for MPI-2". There have been a few references in email as well. Tom Henderson's survey in March had a question asking whether there should be an MPI-n (n > 1) and the response was 13 yes and 0 no (a small subset of people admittedly). So, should we discuss at one of the next two meetings whether there should be an MPI-2. If the decision is yes, should we discuss the timing for meetings and a schedule for an MPI-2 draft? The questions come up for me here as we are looking at next fiscal year's travel planning (government fiscal year -- Oct-Sept). Thanks, Leslie Hart (hart@fsl.noaa.gov) From owner-mpi-core@CS.UTK.EDU Fri Jun 18 18:04:07 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA11901; Fri, 18 Jun 93 18:04:07 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA26537; Fri, 18 Jun 93 18:04:03 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 18 Jun 1993 18:04:02 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA26529; Fri, 18 Jun 93 18:04:01 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA04885; Fri, 18 Jun 93 17:02:57 CDT Date: Fri, 18 Jun 93 17:02:57 CDT From: Tony Skjellum Message-Id: <9306182202.AA04885@Aurora.CS.MsState.Edu> To: mpi-core@cs.utk.edu, hart@nipmuc.fsl.noaa.gov Subject: Re: MPI-2? I would say 'yes' to MPI-2, to scheduling the next working group meetings, to a review of the successful implmenetations of MPI-1, etc. - Tony From owner-mpi-core@CS.UTK.EDU Sat Jun 19 15:42:23 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA13814; Sat, 19 Jun 93 15:42:23 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA13573; Sat, 19 Jun 93 15:42:17 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 19 Jun 1993 15:42:16 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from daedalus.epcc.ed.ac.uk by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA13565; Sat, 19 Jun 93 15:42:15 -0400 Date: Sat, 19 Jun 93 20:42:44 BST Message-Id: <1712.9306191942@subnode.epcc.ed.ac.uk> From: L J Clarke Subject: Re: MPI-2? To: mpi-core@cs.utk.edu In-Reply-To: Tony Skjellum's message of Fri, 18 Jun 93 17:02:57 CDT Reply-To: lyndon@epcc.ed.ac.uk > I would say 'yes' to MPI-2, to scheduling the next working group meetings, > to a review of the successful implmenetations of MPI-1, etc. > I agree. For one I will have to make the case within EPCC, so planning, detail and justification should make this easier for me. Regards Lyndon /--------------------------------------------------------\ e||) | Lyndon J Clarke Edinburgh Parallel Computing Centre | e||) c||c | Tel: 031 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | c||c \--------------------------------------------------------/ From owner-mpi-core@CS.UTK.EDU Mon Jun 21 14:48:48 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA20360; Mon, 21 Jun 93 14:48:48 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA22861; Mon, 21 Jun 93 14:48:37 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 21 Jun 1993 14:48:35 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from ssd.intel.com by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA22823; Mon, 21 Jun 93 14:48:24 -0400 Received: from tualatin.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA16077; Mon, 21 Jun 93 11:48:35 PDT Received: by tualatin.SSD.intel.com (4.1/SMI-4.1) id AA01765; Mon, 21 Jun 93 11:48:51 PDT Date: Mon, 21 Jun 93 11:48:51 PDT Message-Id: <9306211848.AA01765@tualatin.SSD.intel.com> From: Bob Knighten Sender: knighten@tualatin.SSD.intel.com To: mpi-core@cs.utk.edu Subject: Minutes from the May meeting of MPIF Reply-To: knighten@SSD.intel.com (Bob Knighten) PLEASE REVIEW THESE MINUTES FOR ERRORS AS I PLAN TO POST THEM TO COMP.PARALLEL SHORTLY. -- BOB ============================================================================== Minutes of the Message Passing Interface Forum Dallas, Texas May 12 - 14, 1993 The MPI Forum met May 12 - 14, 1993, at the Bristol Suites Hotel in North Dallas. This was the sixth meeting of the MPIF and the fourth of the now regular meetings in Dallas. There were both general meetings of the committee as a whole and meetings of several of the subcommittees. This meeting included the second reading of the Point-to-Point Communication and Collective Communication proposals, so a substantial number of formal votes were taken at this meeting. All of these are recorded in these minutes (and can be found by searching for VOTE) and have also been published (to the mpi-core mailing list) in a summary of all the formal votes and all of the straw votes for the committee as a whole. The notes for these minutes were taken by Rusty Lusk (lusk@mcs.anl.gov) and Bob Knighten (knighten@ssd.intel.com). These minutes are quite long. If you want to see the important topics you can search for --- and this will quickly lead to each topic (and a few other things.) The basic documents that were used at this meeting are: + DRAFT Document for a Standard Message-Passing Interface (May 5, 1993) [This was distributed at the meeting. Although the date on the cover is May 5, it actually contains the material submitted to Steve Otto through May 10 and was produced on May 10. Unfortunately the version of Chapter 2 is not the correct version.] + Point to Point communication (Marc Snir, William Gropp and Ewing Lusk) [A new version was distributed by Marc Snir at the meeting.] + Contexts and Groups - Proposal VIII (Mark Sears, May 11, 1993) [Distributed at the meeting] + Contexts - Proposal X (Lyndon Clarke, Tony Skjellum and Rik Littlefield, May 10, 1993) [Distributed at the meeting] + MPI_LOAD_INFO: Implementing load-balanced applications on distributed-memory systems (Anne Elster, May 11, 1993) [Distributed at the meeting] + DRAFT Maps, Groups and Contexts (Lyndon Clarke, Mark Sears, Tony Skjellum, Marc Snir; May 13, 1993) [Created and distributed at the meeting] The minutes from the last meeting were finally distributed on at the meeting on Friday morning. Attendees: --------- Rob Bjornson SCA bjornson@sca.com Lyndon Clarke EPCC, U. Edinburgh lyndon@epcc.ed.ac.uk James Cownie Meiko jim@meiko.co.uk Jack Dongarra UT/ORNL dongarra@cs.utk.edu Anne C. Elster Cornell U. elster@cs.cornell.edu Sam Fineberg NASA Ames fineberg@nas.nasa.gov Al Geist ORNL gst@ornl.gov Ian Glendinning U. of Southampton igl@ecs.soton.ac.uk Adam Greenberg TMC moose@think.com Bill Gropp ANL gropp@mcs.anl.gov Leslie Hart NOAA/FSL hart@fsl.noaa.gov Tom Haupt Syracuse U. haupt@npac.syr.edu Rolf Hempel GMD hempel@gmd.de Tom Henderson NOAA/FSL hender@fsl.noaa.gov C. T. Howard Ho IBM Almaden ho@almaden.ibm.com Steven Huss-Lederman SRC lederman@super.org Rusty Lusk ANL lusk@mcs.anl.gov Bob Knighten Intel SSD knighten@ssd.intel.com Peter Madams nCube pmadams@ncube.com Arthur B. Maccabe U. of New Mexico maccabe@cs.sandia.edu Oliver McBryan U. Colorado mcbryan@cs.colorado.edu Dan Nessett LLNL nessett@llnl.gov Steve Otto Oregon Graduate Institute otto@cse.ogi.edu Peter Pacheco U. of San Francisco peter@sun.math.usfca.edu Paul Pierce Intel prp@ssd.intel.com Erich Schikuta University of Vienna schiki@cs.rice.edu Mark Sears Sandia mpsears@cs.sandia.gov Anthony Skjellum Mississippi State U. tony@cs.msstate.edu Marc Snir IBM, T.J. Watson snir@watson.ibm.com Alan Sussman U. of Maryland als@cs.umd.edu David Walker ORNL walker@msr.epm.ornl.gov Dennis Weeks Convex weeks@convex.com Stephen Wheat Sandia NL srwheat@cs.sandia.gov Wednesday, May 12 --------- ------ ------------------------------------------------------------------------------- General Meeting ------------------------------------------------------------------------------- Jack Dongarra opened the meeting with a brief discussion of the agenda: Wednesday Presentations: Robert van Renese - MPI on top of HOROS (Isis group, Cornell) Anne Elster - MPI_LOAD_INFO (Cornell) Report from Communication Context Subcommittee Thursday 9-12 P-P 2nd reading 12-1:30 lunch 1:30-4:30 Collective Communications 1st 4:30-6:30 Top 6:45 Dinner 8:45 Sub Friday 9-10:30 Contexts 10:30-12 Reports from other subcommittees Other presentations: Gropp - Environment virtual Berryman - Language Binding Greenberg - Testing/Validation Lusk - implementation There was also a reminder of planned future meetings: June 23-25 August 11-13 September 22-24 (Nov 15-19 SuperComputing '93) Robert van Renese did not make it to the MPIF meeting, so his presentation was cancelled. Anne Elster briefly presented her proposal for an MPI_LOAD_INFO function. There was then the report for the Profiling Subcommittee. This was followed by a long break so that the Communication Context Subcommittee could meet before their presentation. ------------------------------------------------------------------------------- Report From the Profiling Subcommittee ------------------------------------------------------------------------------- Jim Cownie presented. The proposal is in Chapter 8 of the May 5, 1993 draft which was distributed at the meeting. This constitutes the First Reading of the proposal. [Note that Chapter 2 (Point-to-Point Communication) is not the correct version.] Jim Cownie gave a brief overview of the proposal pointing out the (small) changes from the proposal he presented at the last meeting. VOTE: Accept the Profiling Chapter? Yes: 18 No: 0 Abstain 0 ------------------------------------------------------------------------------- Report on the Reference Implementation ------------------------------------------------------------------------------- Rusty Lusk spoke about the work that he and Bill Gropp have done on a reference implementation of MPI. The second version of reference implementation was started after the last meeting as the first version is now quite obsolete. They implemented the basic low level functions and have implemented significant parts of the point-to-point communications on top of this. Implementation Outline A design: ------------------------------------------------------------------------------- User code ------------------------------------------------------------------------------- portable MPI code +---------------------+ | Vendor code | +---------------------+ ------------------------------------------------------------------------------- This is designed to allow maximal reuse of code in various implementations of MPI while also allowing replacement of functions where there is a high performance vendor version. The current design is ------------------------------------------------------------------------------- User code ------------------------------------------------------------------------------- portable MPI code +---------------------+ | Chameleon code | | __ ___ | | |p4| |PVM| | +---------------------+ ------------------------------------------------------------------------------- to allow an easy, portable implementation. This will soon be made available through Netlib. ------------------------------------------------------------------------------- Report From the Communication Contexts Subcommittee ------------------------------------------------------------------------------- Anthony Skjellum presented. Just before this presentation the Context Subcommittee spent about two hours meeting to discuss the various proposals before them. Before they went there were basically three proposals: I Marc Snir VIII Mark Sears X Lyndon Clarke/Tony Skjellum and these are in the MPI draft considered at this meeting. The words: context group communicators: C1 and C2 Differences: What are the coordinates used in sending messages? C1 - intergroup communication C2 - purpose is intergroup communication (Sears) On creation when can use group? (Snir) When is it possible to send message? {{Confusion on my part as to what the subcommittee voted}} Erroneous to receive a message whose context has not been enabled on the receiver. (vote in subcommittee) High level objects vs. opaque group/context vs. transparent context/group Propose to replace MPI_ALL by MPI_INIT to emphasize that it is a group valid only at initialization. Marc Snir: Why context? + Parallel invocation - executed in a loosely synchronous style by a subset of processes + A parallel invocation is _active_ if it is executing on some processes + Two parallel invocations _conflict_ at p if - both are active at the same time - node p is in the execution set of both (p need not simultaneously execute both) + If two parallel invocations conflict at p, then p needs two different contexts to distinguish them. Examples: + A library called within a library (recursion?) + Two partially overlapping collective communications + Multiple parallel execution threads on same processes + Need mechanism for generating context with subset of processes + Need both static & dynamic context allocation => use explicit context + Context should not restrict point-to-point performance Tony came back: Subcommittee agreed that both static and dynamic context allocation needed. Also agreed that because of the desire to support network computing it is necessary to support dynamic groups. Subcommittee met again Wednesday night to try and reduce to a single proposal. Issues to be resolved are whether groups and contexts are completely orthogonal concepts, related concepts (since context formation is a collective operation), or can be merged into a single concept. ------------------------------------------------------------------------------- MPI_LOAD_INFO proposal ------------------------------------------------------------------------------- Anne Elster distributed and spoke briefly on her proposal to have a function to get load information. Jim Cownie remarked that this is very similar to the time functions that we have already agreed not to include. He also noted that it is important than any inquiry function be local, i.e. restricted to getting information from a single node. General observation that this deals with an object (namely the machine) that is not part of MPI. Bill Gropp, as chairman of the Environment Subcommittee, asked for both a precise description of what the function would provide and also verification that there are not other emerging standards that would provide this function. -- Broke at 4:25 -- Subcommittees meeting this afternoon/tonight Topology [room 1] Context [room 2] Collective (after dinner) [here] There are new version of Context proposal VIII and X which were handed out at the meeting. Will break for dinner at 6pm ============================================================================= Thursday, May 13, 1993 -------- ------ ---- ------------------------------------------------------------------------------- General Meeting ------------------------------------------------------------------------------- Jack Dongarra presided. He began by discussing the August and September meeting dates. It was agreed that they would be: August 11-13 September 22-24 There was discussion of the proposal to hold a meeting in Europe. The main disadvantage is that many of the regular U.S. participants would not be able to attend and the new European attendees would not be able to vote. The advantage is publicity of the effort in Europe and showing that the development is truly international. As an alternative Jack Dongarra proposed a additional meeting on November 3-5 in Germany. This was quickly agreed and Rolf Hempel agreed to coordinate the meeting. There was also discussion of the desire of some to find some other location for later meetings. The June meeting is already committed to being at the usual Dallas location. Many wanted something else, but there was no concrete proposal so no action was taken. Simplicity of arrangements is a strong argument for continuing with the current setup. ------------------------------------------------------------------------------- Report From the Point-to-Point Communication Subcommittee ------------------------------------------------------------------------------- Marc Snir presided. This was the second formal reading of the point-to-point communication proposal. Formal votes were take on each of the points in the proposal. A revised draft (dated May 10, 1993) was distributed at the meeting. Marc Snir noted that this is a minor revision of the draft he distributed a few weeks previously. There is no proposal yet from the Language Binding subcommittee. This needs to be discussed we must get this done soon. Rusty Lusk agreed to make sure that there was at least a partial draft for the next meeting, so that discussions could begin. Section 1.1 (Data Types) ------------------------ Marc reminded us that Marc argued for putting responsibility on the user to guarantee that there are no pending operations when a FREE is called. This is contrary to the decision at the last meeting and what is in the draft. Marc's argument against the current proposal is complexity of implementation - it requires the system to keep a reference count and do garbage collection. Paul Pierce noted that the reference count can be kept on each object and there is a natural place where this reference count is decremented and can be freed when the reference count goes to zero, an extremely simple form of garbage collection. After a little more discussion it was agreed to leave the draft as it is. Paul also noted that there is an error in the description - not equivalent to converting to ephemeral. The text will be corrected Array of handles ----- -- ------- Al Geist pointed out that the collective communications need lists of handles all over the place, so exactly how this is done is very important to collective communications. There was some general discussion, but no changes were made. VOTE: Accept Section 1.2 as corrected? ---- Yes: 22 No: 0 Abstain: 0 Section 1.3 (Processes) ----------------------- Error handling has not yet been discussed and should be moved to another section in time. Should this be discussed? Yes because there is a proposal here. Bill Gropp moved to delete the material in Section 1.3.1 recommending the use of signals in a POSIX environment. This is because of the extreme difficulty in using signals with independent libraries. Several others suggested leaving a content-free place holder.. STRAW VOTE: Environmental committee should provide something to replace this? ---------- Yes: 25 No: 0 Abstain: 2 Section 1.4 (Messages) ---------------------- Marc summarized impact of context discussions on point-to-point. The context committee promises not to change the number of parameters. The name context might change to communicator. The communicator provides both protection and it also determines the meaning of source and destination. (It binds group and context). Marc Snir noted that while we have not agreed on the names and details of context, the generalities in 1.4 should be valid whatever we do. There was a discussion of making threads separately addressable, initiated by Dennis Weeks. This was argued against. Bob Knighten mentioned that the threads world in general says that threads are not visible outside a process. No change was made to the draft in this regard. VOTE: Accept Section 1.4 subject to changes in the meaning of "context"? ---- Yes: 21 No: 0 Abstain: 1 Section 1.5 (Communication Buffers) ----------------------------------- There were a number of changes in this section in this draft. Terminology: "contiguous components" rather than "blocks"; "vector component" rather than "index"{??}; change in the units of lengths - now normally in elements; changed ADD to APPEND. Suggestion to change heterogeneous to e.g. general - other suggestions welcomed. There is also the new MPI_COMMIT_BUFFER function. Because there has been confusion about exactly what is being committed, Marc provided examples of usage. Because of the length of this section it was decided to consider the subsections one by one. Jim Cownie said that he and Al Geist in discussion had decided that what would be very useful would be some kind of method for specifying replication and asked that a slot be held open for this purpose. He will submit a proposal before the next meeting. 1.5.1 ----- Paul Pierce proposed that the names for MPI types should directly match the names in the language being used. E.g in Fortran use MPI_INTEGER. Lyndon Clarke proposed a change (or addition) to indexing. [Array of displacements and associated array of lengths.] There was a general discussion of the need/desirability of complex buffers. As usual some favor simplicity and some favor greater function. Arguments had to do with current practice, performance, programming ease, etc. No changes were made. VOTE: Lyndon proposal like I/O vec ---- Yes: 8 No: 9 Abstain: 10 VOTE: Accept Section 1.5.1? ---- Yes: 22 No: 2 Abstain: 1 1.5.2 ----- Mark Sears proposed adding an MPI_CREATE_CONTIGUOUS_BUFFER (and corresponding COMMIT) so that have a persistent handle for contiguous buffers. This is just an accelerator to provide persistent handles for contiguous buffers. MPI_COMMIT_C(buffer, addr, len, datatype, persistence) Arguments: May be more efficient than using the contiguous send/receive. VOTE: Add contiguous-buffer creation functions to this section? ---- Yes: 2 No: 14 Abstain: 7 Paul Pierce proposed that in the MPI_CREATE_BUFFER there be added an additional parameter that specifies the maximum number of appends that will be used in creating this buffer. The discussion dealt with reasons for this (performance because of how the implementation likely to handle append), how onerous this is on the VOTE: Pierce proposal? ---- Yes: 4 No: 7 Abstain: 16 [[Took a count of organizations: 24]] VOTE: Accept Section 1.5.2? ---- Yes: 23 No: 0 Abstain: 1 1.5.3 (Type Matching) ----- Marc Snir gave a brief overview. He separated Type Matching and Data Conversion into subsections 1.5.3 and 1.5.4 so the name of this subsection in the document is wrong. There was essentially no discussion. VOTE: Accept Section 1.5.3? ---- Yes: 23 No: 0 Abstain: 1 1.5.4 ----- Marc Snir gave a brief overview. Note that this Discussion in this section is not a formal part of this section and cannot be as the relevant bindings are not yet done. VOTE: Accept Section 1.5.4? ---- Yes: 23 No: 0 Abstain: 1 1.6 (Receive Criteria) ---------------------- Tony Skjellum again raised the issue of using a mask as well as a tag for receive criteria. Proposal: Receive selection based on (tag & mask) = (intag & mask) Paul Pierce argued strongly against this (and in favor of only exact match and DONTCARE) on the base of performance (because this disallows hash based matching schemes.) Tony Skjellum argued in favor of this largely on the basis of programming utility. Adam Greenberg noted that TMC already implements an arbitrary matching function and so the argument for efficiency is not justified. Tony admitted that much of the functionality can be obtained as long as there is a large number of contexts. VOTE: Accept the proposal? ---- Yes: 4 No:15 Abstain: 5 VOTE: Accept Section 1.6? ---- Yes: 23 No: 3 Abstain: 0 Section 1.7 (Communication Mode) -------------------------------- Secure communication was added in this draft. There were discussions of debates regarding both secure mode and ready-receive Adam Greenberg noted that the vote to accept the secure mode at the last meeting was very close (7-6) and should be revisited. So he moved to delete this mode from the draft and asked for explanation of why this should be part of MPI. Ian Glendinning began by noting that this is for the benefit of the European community who are used to programming with CSP. This led to a long discussion of exactly the meaning of secure mode, what it should be, what it should be named, etc. Marc pointed out that it does not, as currently formulated, supply CSP semantics. Lyndon suggested that we change the name to synchronous. Marc pointed out that our receives are atomic. Lyndon pointed out that we do not need the secure receive; secure send is enough. Lyndon said that we need this to write programs that are safe regardless of buffer resources. David Walker asked for an explanation of why we don't require the more stringent semantics, of having the send suspend until the receive is *complete* rather than *guaranteed to complete*, which is the current description. VOTE: Remove secure communication mode? ---- Yes: 3 No: 12 Abstain: 11 VOTE: Remove secure receive, i.e. have only one receive? ---- Yes: 18 No: 0 Abstain: 7 -- Break for lunch 12-1:35 -- VOTE: Remove READY-RECEIVE? ---- Yes: 6 No: 8 Abstain: 11 Discussion of the name for "secure communication" and it became synch VOTE: Approve Section 1.7 as amended (to remove secure-receive)? ---- Yes: 23 No: 0 Abstain: 2 Discussion of Section 1.8 was postponed at the request of Lyndon Clarke. Section 1.9 (Blocking Communication) ------------------------------------ Lyndon Clarke proposed putting the mode into the parameter list instead of in the function name. Discussion of this: makes smaller number of routines, but makes manual deal with complex ideas necessary in early part. Rusty Lusk pointed out there are errors (missing COMMITs) in the examples. VOTE: Move the modes from the names of ---- Yes: 3 No: 15 Abstain 5 VOTE: Accept Section 1.9? ---- Yes: 22 No: 1 Abstain: 1 Section 1.10 (Nonblocking Communication) ---------------------------------------- The only changes to the draft are adding the features approved at the last meeting. Implicitly the secure sends have now been removed. Rusty Lusk sent mail to Marc Snir about this section before this meeting pointing out that for messages that arrive before they are waited on, it would be useful to have the status object passed in on the init_receive instead of the wait. At any rate the current draft is inconsistent. Currently there is an inconsistency in the functions. Paul Pierce said that he had considered this as part of the NX implementation (which is essentially as in the draft) but found it not a serious problem. But there are similar problems elsewhere. Change is to pass handle for return status to init-receive, etc. Long discussion Rusty flinched at idea of stashing data knowing that will need it elsewhere. Paul notes that problem is primarily stashing data knowing that it will never be used. STRAW VOTE: Move the return-status handle from the I functions to the corresponding init functions? Yes: 5 No: 7 Clearly this will go somewhere only a if concrete proposal is submitted. Note that IRECV should not have a return_status_handle VOTE: Accept Section 1.10 with the corrections discussed? ---- Yes: 20 No: 0 Abstain: 2 Section 1.11 (Contiguous Block Communication Operations) Corrections - Blocking functions should not have handles; the secure receive functions are gone. VOTE: Accept Section 1.11 with corrections? ---- Yes: 23 No: 0 Abstain: 0 Section 1.8 (Communication Objects) ----------------------------------- The subsections were considered separately, but there was then a single vote on section 1.8 as a whole. 1.8.1 (communication Object Creation) ----- Note this is a first reading of this subsection. It was not considered earlier because there was some interest in partially specified message handles, but no proposals were ever received. Paul Pierce proposed having replacing the mode argument on the init_send by having three init_sends. VOTE: Pierce proposal ---- Yes: 11 No: 4 Abstain: 6 1.8.2 (Communication Start) ----- No comments 1.8.3 (Communication Completion) ----- No comments 1.8.4 (Multiple Completions) ----- There has been no resolution of the fairness issue, but that is deferred to the subcommittee dealing with the issue of correctness. Danny Nessett proposed adding a timeout parameter to the MPI_WAITANY and MPI_WAITALL functions. (This was posted to mpi-p2p between the meetings and there was discussion there.) As an alternative Mark Sears proposes adding a function that sends a message (e.g. a timeout message) after some amount of time. A third proposal is to use probe and some external time facilities. Yet another proposal is to use MPI_STATUS and external time and scheduling facilities. Several people noted that timeout requires adding the concept of time to the MPI programming model and this is a very heavy concept. VOTE: Add a timeout parameter to the MPI_WAITANY and MPI_WAITALL functions? ---- Yes: 5 No: 13 Abstain: 3 STRAW VOTE: Should Mark Sears write up a proposal for a timed send? ----- ---- Yes: 4 No: 16 Abstain: 5 Steve Wheat raises strongly the question of the behavior of MPI_WAIT (and, indeed, other MPI functions) in the presence of signals. This led to a substantial discussion of what can/should be specified about the interaction of MPI functions with asynchronous interrupts. Currently MPI does not specify any such interaction. No change was made. VOTE: Accept Section 1.8 with the one amendment? ---- Yes: 23 No: 0 Abstain: 0 Section 1.12 (Probe and Cancel) -------------------------------- Marc Snir first gave an overview of the contents of this section. Jim Cownie noted that because of the necessity of dealing with multithreaded process a CANCEL of a RECEIVE does not remove the necessity for doing a WAIT. Moreover in this case the return-status needs to contain the information that the RECEIVE was canceled. Paul Pierce proposes as an alternative that a WAIT is not needed and if a CANCEL is posted while a WAIT is pending, then the program is erroneous. STRAW VOTE: (1) Pierce (2) Cownie ---------- Paul: 4 Cownie: 15 Jim Cownie proposed to write up a new version of CANCEL for the next meeting. Paul Pierce suggested a restriction on GET_LEN: it can only be used when PROBE has been used with the MPI_BYTE data type. There was confusion and disinterest. Jim Cownie noted that for consistency the data type should be removed from MPI_PROBE and it does not return length. Alternatives: MPI_PROBE as in the draft MPI_PROBE does not return length MPI_PROBE returns the length in bytes MPI_GET_LEN as in draft MPI_GET_LEN works only if MPI_PROBE called with data type MPI_BYTE STRAW VOTE: Do we leave the draft as is or remove data type argument? ---------- Yes: 12 No: 9 STRAW VOTE: Do we leave the draft as is or Paul's proposal? ---------- Yes: 8 No: 7 Kopetzky proposes a blocking probe STRAW VOTE: Add a blocking probe? ---------- Yes: 18 No: 2 VOTE: Accept Section 1.12? ---- Yes: 16 No: 5 Abstain: 1 -- break -- ------------------------------------------------------------------------------- Report From the Collective Communication Subcommittee ------------------------------------------------------------------------------- Collective Communications - Second Reading ----------------------------------------- Al Geist presided The proposal discussed is that in the full draft dated May 10, 1993 (though the version distributed at the meeting had a date of May 5, 1993 on the cover.) Note that this was actually the first reading of most of the actual functions being proposed including everything in Sections 3.5-3.6. Section 3.4 (Synchronization) ----------------------------- VOTE: Accept Section 3.4 with editorial correction? ---- Yes: 21 No: 0 Abstain: 0 Section 3.5 (Data move functions) --------------------------------- VOTE: Accept MPI_BCAST and MPI_BCASTC descriptions? ---- Yes: 19 No: 0 Abstain: 1 VOTE: Accept MPI_GATHER and MPI_GATHERC descriptions (after return_status ---- removed from MPI_GATHER)? Yes: 19 No: 0 Abstain: 0 Adam Greenberg noted that descriptions of error behavior is needed. Al Geist responded that as written the descriptions are inconsistent in their use of return status. VOTE: Accept MPI_SCATTER and MPI_SCATTERC description? ---- Yes: 20 No: 0 Abstain: 0 VOTE: Accept MPI_ALLSCATTER and MPI_ALLSCATTERC description (after ---- return_status removed from MPI_ALLSCATTER)? Yes: 21 No: 0 Abstain: 0 VOTE: Accept MPI_ALLCAST and MPI_ALLCASTC descriptions (after return_status ---- removed from MPI_ALLCAST)? Yes: 20 No: 0 Abstain: 0 Section 3.6 (Global Compute Operations) --------------------------------------- Discussion of numeric properties of the global compute operations. Nothing is specified about the order of the operations performed. In particular there is no guarantee that MPI_PROD and MPI_SUM will even be repeatable. What is the effect of MPI_MAXLOC and MPI_MINLOC on VOTE: When the value MPI_MAXLOC and MPI_MINLOC might be ambiguous, the ---- smallest group rank is returned? Yes: 15 No: 0 Abstain: 6 Adam Greenberg asked about interest in a function that returns both a max and the location of the max. STRAW VOTE: Want such a combined function? ---------- Yes: 13 No: 0 Abstain: 10 VOTE: Accept descriptions of MPI_REDUCE and MPI_REDUCEC? ---- Yes: 21 No: 0 Abstain: 0 There was a moderate amount of discussion of the USER_REDUCE functions involved in clarifying their use. As part of this Jim Cownie proposed that that these two functions should have an additional parameter, unitsize, used by the operation when operating on complicated elements. Further discussion of this raised again the issue of dealing with buffers for arrays of structures in a heterogeneous environment. Jim Cownie raised this earlier when discussing general buffers and intends to provide a proposal for this purpose. STRAW VOTE: Write up a full proposal? ---------- Yes: 20 No: 0 Abstain: 0 On to considering version of reduce that broadcasts the result to all processes in the group. Paul Pierce noted that for consistency the word ALL should be used as part of these names rather than the letter G. This was accepted. VOTE: Have MPI_ALLOP, MPI_ALLOPC, MPI_USER_ALLOP, MPI_USER_ALLOPC? ---- Yes: 15 No: 1 Abstain: 2 Note that MPI_USER_SCAN has the same problems as MPI_USER_REDUCE and VOTE: Accept the description of MPI_SCAN? ---- Yes: 16 No: 0 Abstain: 4 Finally discussed the various SHIFT functions. This started with an explanation of why the new shift functions are in the draft. Adam Greenberg noted that he proposed a general send_and_receive operation in email as an alternative to the shift functions. It's only appearance in the draft is in Section 1.14 of the Point-to-point section. There was a long discussion of the relation of shift functions to process topologies as well. STRAW VOTE: Have SEND-RECEIVE function? ---------- Yes: 15 No: 0 Abstain: 6 The question of the relation of SEND-RECEIVE to the SHIFT functions and further consideration of the SHIFT functions was postponed until the Topology Subcommittee reports. ------------------------------------------------------------------------------- Report From the Process Topology Subcommittee ------------------------------------------------------------------------------- Rolf Hempel presented Rolf presented the current status of the Process Topology proposal. The subcommittee has reached consensus in response to the decisions of the MPIF at the last meeting. The current proposal is in Chapter 4 (Process Topologies) of the draft. Topologies have been reworked to reflect last meeting's vote to use mpi low-level group creation mechanisms to build topologies. He summarized the current draft. It uses group-creation mechanisms, but has functions to assist in the creation of the orderings of ranks in the groups. The rest of the draft is implementable on top of the previous two low-level functions to create groups, and then using groups. The following is essentially a transcription of the foils that Rolf used in his presentation. Basis for topology functions: Group creation with: MPI_CREATE_GROUP(gid_old, gid_new, group_key, order_key) with gid_old: group id of existing group (input) gid_new: group id of new group (output) group_key: key for group partitioning (input) order_key: key for ranking within partition (rank in gid_old for tie breaking) -- Topology functions arranged in two levels: Low Level Functions: --- ----- --------- Together with MPI_CREATE_GROUP need: Mapping functions for cartesian structures and arbitrary graphs newrank = MPI_MAP_CART(gid_old, ndims, dims, period) and newrank = MPI_MAP_GRAPH(gid_old, nodes, index, nglist) If the calling process is not in the topology: newrank = MPI_NORANK -- The newrank value can be used as order_key in the group creation. ==> Potentially this can be used for mapping optimization If for a particular implementation the process placement does not make any difference, there is a trivial implementation: Just return the rank in the parent group. -- High Level Functions: ---- ----- --------- In principle, low level functions are sufficient for topology mapping, but they are not convenient to use. The advantages of high level routines are: + Do mapping and group creation in a single call + Internally store topology information associated with a group + provide look-up functions for topology information The high level functions are defined in terms of the low level functions. High Level Topology Definitions ---- ----- -------- ----------- newrank = MPI_CART(gid_old, gid_cart, ndims, dims, period, reorder) and newrank = MPI_GRAPH(gid_old, gid_graph, nodes, index, nglist, reorder) Question: If more processes are available than are used in the topology, should two groups be created? The advantage is that the REST group could then be used for other purposes. -- For Cartesian topologies, there are two obvious ranking schemes: row-major order and column-major order. These could be selected with an additional argument to MPI_CART. -- Question: How should topology information for a group be stored for later use by inquiry functions. One options is caching as part of the group as has been discussed in the Context Subcommittee. Alternatively it could be done in an implementation specific manner, e.g. additional fields in the group structure. An ugly alternative would be to pass a handle to a suitable opaque object to every topology function. -- Divide a group among ndim Cartesian dimensions via code = MPI_MAKDIM(gid, ndim, dims) input arguments gid: id of process group ndim: number of dimensions dims(*): number of processes per coordinate direction (integer vector) output arguments dims(*): number of processes per coordinate direction (missing entries filled in) return value if an error occurred, code is -1, otherwise code is 0 Example: 60 processes in process group Input Output dims dims ndims=3 |0|0|0| |5|4|3| dims dims ndims=3 |0|10|0| |3|10|2| dims ndims=2 |0|7| ERROR dims ndims=3 |-1|0| ERROR -- Topology Inquiry Functions -------- ------- --------- Find type of topology for a group map = MPI_INQMAP(gid) Find size of cartesian structure and own position dim = MPI_INQCART(gid, maxdim, size, period, pos) Translate coordinate vector to pid pid = MPI_INQPID(gid, location) Depending on what is finally done regarding process identifiers, this may be rank rather than pid. Translate group rank into topology location MPI_INQLOC(gid, rank, location, maxdim) -- Split topology group into subgroups ----- -------- ----- ---- --------- Creation of hyperplanes gid_new = MPI_PARTC(gid, lostdim) Creation of 1-dimensional subgroups gid_new = MPI_SHIFTGR(gid, direction) New groups inherit the topology information (e.g. periodicity) from their parent group. -- Should there be a special shift function for Cartesian structures? This would be similar to the the collective communication shift function with an additional direction argument. The shift function could choose circular or end-off behavior automatically depending of the periodicity of the group in the selected direction. -- STRAW VOTE: Is this a reasonable approach for the Process Topology material? ---------- Yes: 10 No: 0 Abstain: 5 Discussion of shift functions in the context of topology. Rolf began by remarking that putting the shift functions in the topology section seems unnatural because it is not basically concerned with communication and all of the other communication functions are in either Point-to-point or Collective Communication. But the very meaning of shift is associated with a topology and associating them with groups is just to assume some particular topology is associated with a group. There was a long discussion of possible relations between groups, topologies and shifts. A general proposal that emerged was to leave shifts in collective communications but to get the information needed to choose the behavior from any topology on the group. A complementary proposal from Paul Pierce was to have shifts defined for topologies, while in collective communication there would be only send-receive. Proposals: (1) Leave shift in collective communications (2) Put shifts in topology (3) Remove shift from collective communication, add send-receive to point-to-point and provide convenience functions in topology to implement shift relevant to the topology. These proposals will have to be written. ============================================================================= Friday, May 14, 1993 ------ ------ ---- ------------------------------------------------------------------------------- Report From the Communication Context Subcommittee ------------------------------------------------------------------------------- Tony Skjellum presented. A unified proposal is now available. A limited number of of the draft proposal were distributed at the meeting. Concepts -------- Map: function from (0,..,n-1) to (0,..,m-1) (i.e. an ordered set of --- integers) Process Group: an ordered set of processes. ------- ----- A map defines a process group relative to another group. In particular a map can be used to define a process group relative to a universal group which will be PREDEFINED. Context: Partitioning of communications space. ------- Context ID: an identifier for a context; this can be passed between ------- -- processes. Communicator (C1): is a process group + context. It is use for ------------ ---- communication within a single process group. (C2 will be defined later) New opaque objects: + Map + Communicator The communicator replaces context. Operations: ---------- Map --- Map operations (apply, . . . ) Local constructor (concat, . . . ) Collective constructor (split, . . . ) Context id ALLOCATE (unique within group) FREE Communicator MAKE (local constructor) SAFEMAKE (allocate + make) ACCESSORS (group of; context of; parent of) + MPI_ALLOCATE_CONTEXTID(communicator, map, array_of_ids, len) - collective call executed by all process in group defined by (communicator, map) - synchronization is between these processes in the map - generates ids that are UNIQUE within the new group + MPI_MAKE_COMM(communicator, map, id, new_communicator) - local call - generates new communicator - OK if id is unique (no guarantee otherwise) - Has be to called by "both" sides before use, i.e. by by all of the potential senders and receivers. + MPI_SAFEMAKE_COMM(communicator, map, new_communicator) + MPI_ALLOCATED_CONTEXTID(communicator, map, id, 1) + MPI_MAKE(communicator, map, id, new_communicator) + "MPI_SYNCH(communicator, map)" [this needs to be added to the collective communications] MPI_SAFEMAKE_COMM guarantees synchronization before the communicator can be used. The currently defined function in this proposal (i.e. C1) is sufficient for point-to-point and collective communication. It will be extended for use in an "open universe", server model. etc. It is compatible with the planned extensions. There is a pre-defined MPI_ALL communicator and at least a predefined MPI_IDENTITY_MAP map. Discussion: Strong desire to see examples of (1) how to bootstrap, (2) how to create "subgroups" and (3) how to create the communicator for use by a library. Mark Sears: Making new (global) context ids c1 = MPI_ALL_C1 m = MPI_IDENTITYMAP MPI_ALLOC_CTXT(c1, m, 57, return_list) ^^ any number up to some implementation limit Now need to build mA and mB, the maps of interest. There are a couple of ways of doing this: mA = MPI_MAP_BUILD(. . ., (2,4,6,...),. . .) mB = MPI_MAP_BUILD(. . ., (1,3,5,...),. . .) MPI_MAKE(c1, mA, return_list1, new_communicator1) MPI_MAKE(c1, mB, return_list2, new_communicator2) { Note that if, as in this example, the maps are disjoint, then it is not necessary to have two different communicators (new_communicator1 and new_communicator2), rather could write if (rank is even) then mA = MPI_MAKE(c1, mA, return_list, new_communicator) else mB = MPI_MAKE(c1, mB, return_list, new_communicator) } An alternative to the use of MPI_MAKE is to write MPI_SAFEMAKE(c1, mA, new_communicator1) MPI_SAFEMAKE(c1, mB, new_communicator2) In this case the user is guaranteed that the communicators are unique, but the user does not yet know the communicators. Discussion: Note that maps are, as in Mark Sears earlier proposal, just a function from an initial segment of the natural numbers to the natural numbers. There is no direct connection between a map and processes - such a connection is provided by a communicator. Note that WHO_AM_I will necessarily take a communicator as an argument and this is the right thing to do. In a particular situation a process may want to get this information for several communicators. Plans are to elaborate this common proposal via the network between now and the next meeting when there will be a first reading of the proposal. Paul Pierce noted that there is something that he asked for shortly after the last meeting but which he does not see in this proposal. Consider the following situation: PROGRAM INIT LibA INIT LibB . . COMMUNICATOR1 COMMUNICATOR2 . . /* above is all setup */ LibA(COMMUNICATOR1) LibA(COMMUNICATOR2) LibB(COMMUNICATOR2) . . . In the setup synchronization is not an issue, but Paul wants to be sure that no synchronization is required during the calls to the libraries. Marc Snir argued that if the libraries are written in such a way that there are no "dangling messages", then there is no problem. If this is not true, e.g. if LibA calls LibB, then need an additional communicator to protect in that situation. This is a problem because it implies a knowledge of the architecture of the libraries at the time the communicators are allocated. Paul offered an existence proof that it is possible to get what he wants. {{EXTRACT FROM HIS MAIL MESSAGE}} There was a fair amount of discussion of various possible solutions to this problem. In particular Marc Snir argued that After some discussion this issue was referred back to the Communication Contexts Subcommittee. Al Geist asked for an explanation of MPI_MAP_FLATTEN. The point is that a map may be described other than a list of integer, so this just returns the list of integers in the range of the map. A better name is needed. {{I proposed MPI_LIST_MAP}} ------------------------------------------------------------------------------- Report on MPI -1 ------------------------------------------------------------------------------- Steve Huss presented MPI_JACK_TIME which returns the time Jack leaves the meeting Name changes: STANDARD --> PASSIVE SECURE --> HOPEFUL READY --> AGRESSIVE ------------------------------------------------------------------------------- Testing and Validation ------------------------------------------------------------------------------- Adam Greenberg Implementations will need to be validated. This means: + test for completeness + test for correctness Testing + test interface (the user view) + test the implementation (i.e. the inner boundary conditions) Validation may well be a subset of Testing Want to begin cooperative development of a Test System + Test Driver - Means for running tests - Consistent test interface Uniform return values for success Consistent reporting of failures + Reporting of test results + F77, C + Structured (functional) tests - test specific function - small - meet the requirements of the test interface + Application kernels - complete program - known results - contains everything to build program (including input data) ---------------------------------------------------------------------------- Starting Processes ---------------------------------------------------------------------------- Bill Gropp discussed alternatives for how the beginning of an MPI program might look. alternatives: MPI_main(arc,argv) MPI_init( ...) MPI_exit() MPI_call() These are all equivalent. These will be discussed, and a proposal made, for the environmental management section of the draft for the next meeting. Marc pointed out that we need to see how to get libraries initialized. =========================================================================== It was discovered that this hotel is already full for the time of the June meeting though some rooms have been reserved for MPI. Book your room SOON! From owner-mpi-core@CS.UTK.EDU Mon Jun 21 17:33:12 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA21191; Mon, 21 Jun 93 17:33:12 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA03418; Mon, 21 Jun 93 17:32:46 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 21 Jun 1993 17:32:45 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from ssd.intel.com by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA03407; Mon, 21 Jun 93 17:32:38 -0400 Received: from tualatin.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA23760; Mon, 21 Jun 93 14:32:51 PDT Received: by tualatin.SSD.intel.com (4.1/SMI-4.1) id AA01827; Mon, 21 Jun 93 14:33:07 PDT Date: Mon, 21 Jun 93 14:33:07 PDT Message-Id: <9306212133.AA01827@tualatin.SSD.intel.com> From: Bob Knighten Sender: knighten@tualatin.SSD.intel.com To: mpi-core@cs.utk.edu Subject: VOTES at the MPIF meetings Reply-To: knighten@SSD.intel.com (Bob Knighten) Just after the May MPIF meeting I sent out a summary of all the votes that have been taken at MPIF meetings. It now seems that many of you did not get this summary, so I am sending it again. Please check it over to see if there are any errors. I have tried to make the description of the votes at the May meeting self-contained for those who attend the meetings. If clarification is needed, please tell me about that as well. I have made a few corrections to votes at earlier meetings, but have not significantly improved the descriptions of the votes at the earlier meetings. If this is important, please let me know about this as well. -- Bob ============================================================================== May Meeting Formal Votes Accept the Profiling Chapter? Yes: 18 No: 0 Abstain 0 [[Chapter 2 is the Point-to-Point Communication]] Accept Section 2.2 as corrected? Yes: 22 No: 0 Abstain: 0 Accept Section 2.4 subject to changes in the meaning of "context"? Yes: 21 No: 0 Abstain: 1 Add a facility to indexing using an array of displacements and an associated array of lengths (such as with IO vec)? [Proposal from Lyndon Clarke] Yes: 8 No: 9 Abstain: 10 Accept Section 2.5.1 (Communication Buffer Components)? Yes: 22 No: 2 Abstain: 1 Add contiguous-buffer creation functions to section 2.5.2 (Buffer operations)? Yes: 2 No: 14 Abstain: 7 In MPI_CREATE_BUFFER, add an additional parameter specifying the maximum number of appends that will be used in creating the buffer? [Proposal from Paul Pierce]? Yes: 4 No: 7 Abstain: 16 [[There was probably some double voting in the last couple of votes - there were only 24 organizations represented.]] Accept Section 2.5.2 (Buffer Operations)? Yes: 23 No: 0 Abstain: 1 Accept Section 2.5.3 (Type Matching and Data Conversion)? Yes: 23 No: 0 Abstain: 1 Accept Section 2.5.4 (Data Conversion)? Yes: 23 No: 0 Abstain: 1 Have receive selection based on (tag & mask) = (intag & mask)? Yes: 4 No: 15 Abstain: 5 Accept Section 2.6 (Receive Criteria)? Yes: 23 No: 3 Abstain: 0 Remove secure communication mode? Yes: 3 No: 12 Abstain: 11 Remove secure receive, i.e. have only one receive? Yes: 18 No: 0 Abstain: 7 Remove READY-RECEIVE? Yes: 6 No: 8 Abstain: 11 Approve Section 2.7 (Communication Mode) as amended (to remove secure-receive)? Yes: 23 No: 0 Abstain: 2 Move the modes from the names of communication functions to a parameter? Yes: 3 No: 15 Abstain 5 Accept Section 2.9 (Blocking Communication)? Yes: 22 No: 1 Abstain: 1 Accept Section 2.10 (Nonblocking Communication) with the necessary corrections of the errors in the text? Yes: 20 No: 0 Abstain: 2 Accept Section 2.11 (Contiguous Block Communication Operations) with the necessary corrections of the errors in the text? [Blocking functions should not have handles; the secure receive functions removed.] Yes: 23 No: 0 Abstain: 0 Replace the mode argument on the INIT_SEND by having three variants of INIT_SEND? Yes: 11 No: 4 Abstain: 6 Add a timeout parameter to the MPI_WAITANY and MPI_WAITALL functions? Yes: 5 No: 13 Abstain: 3 Accept Section 2.8 (Communication Objects) with the one amendment of having three variants of INIT_SEND? Yes: 23 No: 0 Abstain: 0 Accept Section 2.12 (Probe and Cancel)? Yes: 16 No: 5 Abstain: 1 [[Chapter 3 is Collective Communication]] Accept Section 3.4 (Synchronization) with editorial correction? Yes: 21 No: 0 Abstain: 0 Accept MPI_BCAST and MPI_BCASTC descriptions? Yes: 19 No: 0 Abstain: 1 Accept MPI_GATHER and MPI_GATHERC descriptions (with return_status removed from MPI_GATHER)? Yes: 19 No: 0 Abstain: 0 Accept MPI_SCATTER and MPI_SCATTERC descriptions? Yes: 20 No: 0 Abstain: 0 Accept MPI_ALLSCATTER and MPI_ALLSCATTERC descriptions (with return_status removed from MPI_ALLSCATTER)? Yes: 21 No: 0 Abstain: 0 Accept MPI_ALLCAST and MPI_ALLCASTC descriptions (with return_status removed from MPI_ALLCAST)? Yes: 20 No: 0 Abstain: 0 When the value MPI_MAXLOC and MPI_MINLOC might be ambiguous, the smallest group grank is returned? Yes: 15 No: 0 Abstain: 6 Accept descriptions of MPI_REDUCE and MPI_REDUCEC? Yes: 21 No: 0 Abstain: 0 Have MPI_ALLOP, MPI_ALLOPC, MPI_USER_ALLOP, MPI_USER_ALLOPC in Collective Communications? Yes: 15 No: 1 Abstain: 2 Accept the description of MPI_SCAN? Yes: 16 No: 0 Abstain: 4 ============================================================================ May Meeting Straw Votes Move the return-status handle from the I(mmediate) nonblocking functions to the corresponding init functions? Yes: 5 No: 7 Environmental Subcommittee should provide something to replace Section 2.3 (Processes)? Yes: 25 No: 0 Abstain: 2 Should Mark Sears write up a proposal for a timed send? Yes: 4 No: 16 Abstain: 5 Two Proposals dealing with CANCEL in a multithreaded program: (1) [Cownie] A CANCEL of a RECEIVE does not remove the necessity for doing a WAIT. Moreover in this case the return-status needs to contain the information that the RECEIVE was canceled. (2) [Pierce] A CANCEL of a RECEIVE does not need a WAIT and if a CANCEL is posted while a WAIT is pending, then the program is erroneous. (1) 15 (2) 4 Should the data type argument be removed from MPI_PROBE (for consistency)? Yes: 9 No: 12 Restrict GET_LEN to be used only when PROBE has been used with the MPI_BYTE data type? No: 8 Yes: 7 Add a blocking probe? Yes: 18 No: 2 Want a function that returns both the location and value of the max in an array? [Greenberg] Yes: 13 No: 0 Abstain: 10 Write up a proposal for user_reduce functions using general buffers? [Cownie] Yes: 20 No: 0 Abstain: 0 Have SEND-RECEIVE function? Yes: 15 No: 0 Abstain: 6 Is this a reasonable approach for the Process Topology material? Yes: 10 No: 0 Abstain: 5 ============================================================================ March-April Meeting Formal Votes Separate mpi_create for each type Yes: 20 No: 0 Abstain: 4 MPI_FREE is valid even if handle is in use, but effect is to free the object when the operation completes. Yes: 7 No: 8 LIST_LENGTH explicit (rather than included in array or EOL)? Yes: 22 No: 0 Accept Section 2.2? Yes: 23 No: 1 BYTE data type? Yes: 24 No: 0 Accept Section 2.5 (minus 2.5.2) Yes: unanimous Proposals for units of bytes and for consistency. 1. Units are elements 2. bytes in C and elements in F77 (Yes: 0 No: 14) 3. bytes everywhere (and provide some way of getting size of basic types) a) index stays as is, but bytes elsewhere Yes: 9 Con: 13 (vote 1) b) truly bytes everywhere, including indices Yes: 10 Con: 8 (vote 2) In vector, stride may be negative? Yes: 10 No: 2 Allow repetition? Yes: 13 No: 1 Can multiple components overlap? vote not taken Total length is an integer multiple of the block size Yes: 4 No: 10 Commit? Yes: 11 No: 6 Accept 2.6 Yes: 12 No: 4 Typed DONTCARE? Yes: 19 No: 1 (1) RECEIVE(..., tag, info, ...) no DONTCARE for tag (2) RECEIVE(..., tag, mask, ...) no DONTCARE for tag (3) RECEIVE(..., tag, ...) DONTCARE for tag (1) fails for lack of second. (2) Yes: 6 No: 12 so (3) remains. Do we want a "secure" communication mode? Yes: 7 No: 6 Delete send/ready-receive? Yes: 9 No: 10 Accept amended versions of 2.7 & 2.8 Yes: 25 No: 1 The return status for a receive operation should be the number of bytes received Yes: 17 No: 3 Abstain: 7 Accept 2.9 (with amendments) excluding 2.9.1 and WAIT_ANY Yes: 19 No: 2 Abstain: 3 Add both blocking and immediate secure-send/secure-receive with failure of program to match being erroneous. Yes: 10 No: 8 Abstain: 9 2.10 as amended Yes: 20 No: 4 Abstain: 3 2.11 Yes: 26 No: 0 Abstain: 1 Use exactly the same parameters in MPI_ADD_BLOCK as amended Yes: 26 No: 0 Abstain: 1 Have strided block message functions. Yes: 5 No: 9 Abstain: 12 Include non-blocking c-c? Yes: 0 No: uncounted Abstain: 9 Remove tag in all c-c? Yes: 15 No: 5 Abstain: 5 Accept just semantics of 3.4 Yes: 23 No: 0 Abstain: 2 Allow 2nd cshift with only one buffer. No partial overlap on orig cshift Yes: 21 No: 0 Abstain: 6 Accept cshift, eoshift proposals with amendment of 2nd form of each. Yes: 20 No: 2 Abstain: 4 Accept bcast Yes: 22 No: 0 Abstain: 2 Provide two new routines: MPI_ADD_HVEC() and MPI_ADD_HINDEX(). MPI_ADD_HVEC() is identical to MPI_ADD_VEC() except that "stride" is measured in bytes. MPI_ADD_HINDEX() is identical to MPI_ADD_INDEX() except that "indices" are byte offsets Yes: 19 No: 0 Abstain: 3 ============================================================================ March Meeting Straw Votes Topologies in MPI? If yes integrated with groups (e.g. eoshift), OR separate library, environmental inquiry In: 25 Out: 4 Abstain: 5 integrated: 4 Separate: 23 Abstain: 7 ============================================================================ February Meeting POINT-TO-POINT Straw Votes Should all data types be required to start on their natural boundaries? No by acclamation Can messages have 0 length? Yes by acclamation Can a string of bytes start on any byte address and be of any length (including 0)? Yes: 34 Con: 0 Unanimous that byte types can start on any alignment. Unanimous that bytes have 8 bits. Use named constants? Yes: 27 No: 2 Must matched messages fit in buffer? (Otherwise an error.) Yes: 26 No: 0 Unmodifiable handles? Only one in favor Section ?? should be rewritten to allow modification, followed by commit operation. (This includes use modification and recommit.) Yes: 28 No: 0 Defaults? {for what} Yes: 20 No: 5 Ready-receive? Yes: 20 No: 12 Synchronized functions? Yes 10 No 15 (So 2.5.4 goes away) Separate query functio(s)? Yes: 21 No: 3 Single complete function? Yes: 23 No: 0 Three proposals: inout arguments, multiple parameters, opaque structures. inout arguments?: Yes: 2 No: 32 Lists of out arguments and in arguments?: Yes: 24 No: 8 Straw vote: wait_send, wait_recv with different parameters? Yes: 2 No: 29 (The preferred alternative is to have a uniform WAIT, handles, opaque_return and query function(s)). WAIT_ANY? Yes: 28 No: 1 Do we want null-value handle (so can delete)? Yes: 19 No: 3 Do we want wait_any to set handle matched to null? Yes: 21 No: 6 Is all-null an error or is it a no-op? Vote postponed. Should we have a WAIT_ALL? Yes: 15 No: 9 In favor of get full form of buffer-descriptor presented by Bill Gropp? Yes: 29 No: 2 Should there also be a simple version for contig messages of fixed types, in order to conform more to existing practice? Yes: 28 No: 3 User can select (in some way) what happens when an error occurs? in f77 Yes: 21 No: 6 in C assume same in C ============================================================================== February Meeting COLLECTIVE COMMUNICATION Straw Votes Should broadcast be synchronous? Yes: 5 No: 5 (37 people in room) Non-blocking broadcast? (A non-blocking broadcast returns ASAP, but buffers are invalid until the operation is waited on.) Yes: 21 No: 7 broadcast is received by broadcast, not receive. Unanimous Non-blocking barrier? Yes: 27 No: 1 Include tag for both blocking and nonblocking barrier. Yes: 31 No: 1 Flag to indicate whether gather receives data (on non-root)? Postponed Later: Yes: 9 No: 15 Function that gathers to root of operation? Yes: 32 No: 0 All-to-all gather? Yes: 26 No: 0 Non-blocking versions of gather and all-to-all? Yes: 18 No: 5 Can gather and all-to all have different buf size on each process? Yes: 29 No: 0 At least provide an operation that partitions an existing group by key? Yes: 32 No: 0 Only partition existing group or keep list form of mkgroup? Only partition: 10 keep list form: 17 ============================================================================== February Meeting MPI FORUM Straw Votes Numbering of objects: 32-1, in favor of 0,,N-1 (Therefore (0,0) is the first element of a two-dimensional arrays.) ============================================================================= January Meeting POINT-TO-POINT Straw Votes Support for synchronous point to point communication? Yes: 17 No: 2 Wait for a vector of handlers (this is an OR operation)? Yes: 17 No: 0 Cancel in the initial MPI? Yes: 7 No: 19 Ready_receive? Yes: 13 No: 10 Write point-to-point operations in terms of init, start, wait and free? Yes: 19 No: 2 Probe on handle as lock? Yes: 25 No: 0 User-defined select criteria? Yes: 1 No: 26 May context modify pid? NO Context? YES Selection by group? Yes: 10 No: 11 Buffer with stride? YES IOVEC buffer? Yes: 18 No: 1 Do not want send in homogeneous environment to require the type information need for the heterogeneous environment? YES General send is a send-bytes? YES Option so that MPI routines cause exception on error? Yes: 23 No: 0 Option to have MPI routines set error code? Yes: 19 No: 1 Maintain order of messages? Yes: 23 No: 4 ============================================================================== January Meeting COLLECTIVE COMMUNICATION Straw Votes General CONCAT function with the number of bytes different on each node? Yes: 18 No: 0 ============================================================================= Robert L. Knighten | knighten@ssd.intel.com Intel Supercomputer Systems Division | 15201 N.W. Greenbrier Pkwy. | (503) 629-4315 Beaverton, Oregon 97006 | (503) 629-9147 [FAX] From owner-mpi-core@CS.UTK.EDU Mon Jun 28 12:58:30 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA12885; Mon, 28 Jun 93 12:58:30 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA25645; Mon, 28 Jun 93 12:57:54 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 28 Jun 1993 12:57:53 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from ssd.intel.com by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA25637; Mon, 28 Jun 93 12:57:51 -0400 Received: from tualatin.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA17717; Mon, 28 Jun 93 09:58:50 PDT Received: by tualatin.SSD.intel.com (4.1/SMI-4.1) id AA03654; Mon, 28 Jun 93 09:58:48 PDT Date: Mon, 28 Jun 93 09:58:48 PDT Message-Id: <9306281658.AA03654@tualatin.SSD.intel.com> From: Bob Knighten Sender: knighten@tualatin.SSD.intel.com To: mpi-core@cs.utk.edu Subject: Notes from the last meeting Reply-To: knighten@SSD.intel.com (Bob Knighten) This is addressed to those of you who were at the meeting of the MPIF last week and particularly those who were in the second reading of the Collective Communications draft. I lost somewhere between 1 and 2 hours of notes that I need for the minutes. This was most of the discussion on the reduce functions. Danny Nesset had the main votes, but there were even some of the votes that I have lost. If you have any notes from that period, it would be very helpful if you would send them to me. Either e-mail or fax would be fine. Thank you, Bob Robert L. Knighten | knighten@ssd.intel.com Intel Supercomputer Systems Division | 15201 N.W. Greenbrier Pkwy. | (503) 629-4315 Beaverton, Oregon 97006 | (503) 629-9147 [FAX] From owner-mpi-core@CS.UTK.EDU Sun Jul 4 05:16:22 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA27689; Sun, 4 Jul 93 05:16:22 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA24081; Sun, 4 Jul 93 05:15:10 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sun, 4 Jul 1993 05:15:08 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from ens.ens-lyon.fr by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA24052; Sun, 4 Jul 93 05:15:06 -0400 Received: from lip.ens-lyon.fr (lip-gw.ens-lyon.fr) by ens.ens-lyon.fr (4.1/SMI-4.1) id AA18967; Sun, 4 Jul 93 11:16:26 +0200 Received: from sake.ens-lyon.fr by lip.ens-lyon.fr (4.1/SMI-4.1) id AA07816; Sun, 4 Jul 93 11:16:24 +0200 Date: Sun, 4 Jul 93 11:16:24 +0200 From: Jack.Dongarra@lip.ens-lyon.fr (Jack Dongarra) Message-Id: <9307040916.AA07816@lip.ens-lyon.fr> To: mpi-core@cs.utk.edu Subject: MPI workshop at SC93 I just received word from the Supercompuer 93 people that the workshop we proposed for the meeting in Portland in November has been accepted. We are schedule to go on Friday at 8:30-Noon. The original plan called for each subcommittee chair to present a part of the standard. If there are other suggestions please send them to me. Best wishes, Jack From owner-mpi-core@CS.UTK.EDU Sun Jul 18 11:22:53 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA08257; Sun, 18 Jul 93 11:22:53 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA03332; Sun, 18 Jul 93 11:23:43 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sun, 18 Jul 1993 11:23:42 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA03185; Sun, 18 Jul 93 11:21:08 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA19755; Sun, 18 Jul 93 10:21:07 CDT Date: Sun, 18 Jul 93 10:21:07 CDT From: Tony Skjellum Message-Id: <9307181521.AA19755@Aurora.CS.MsState.Edu> To: mpi-collcomm@cs.utk.edu, mpi-context@cs.utk.edu, mpi-core@cs.utk.edu, mpi-pt2pt@cs.utk.edu Subject: All about threads Topic: Tacit Thread Safety Requirement of MPI1, Context Chapter, etc. Dear colleagues: In reviewing comments about the latest context draft, it has been repeatedly told me that we are at a crucial stage in MPI, because we have to agree on the context model, etc, as soon as possible. I concur with that assessment. In trying to find a consistent way to acquire safe communication space for groups, the issue of thread safety arises, because overlapping concurrent threads would have to work correctly. I am currently confident about the single-threaded case, and I am NOT CONFIDENT about the multi-threaded case. Does anyone have real experience with multi-threaded message passing (has it been done in an application setting, like we assume for MPI)? I need immediate guidance (specific guidance) about what multi-threaded programming MEANS in MPI, if this was in fact a reasonable requirement for MPI in the first place, and how multi-threading impacts point-to-point and collective communication (that is, real programs with examples). For instance, do we assume named threads or unnamed threads (and would this help). Is there an examplar threads package ??? Here is one problem in a nutshell. We discussed "statically initialized libraries" from time to time. Well, if there are multiple overlapping threads, then one would need to have separate contexts statically initialized for each concurrent thread. Such threads have group scope. Hence, groups would have to cache contexts for each concurrent thread (notice: groups cacheing contexts). I propose that we have a serious discussion on what thread safety really means for MPI1. I need for there to be well-formulated guidelines and in-depth debate immediately, so that the context committee can work effectively within these requirements, or give feedback as to why they are unreasonable. Otherwise, I/we can't really make the context chapter bullet-proof in time for the next meeting (except for the single-thread case). We have discussed how contexts provide group safety, but not temporal safety from multiple invocations of operations on a context (for which a programming paradihm must be described; e.g., synchronizing or implicitly synchronizing ... also could be called quiescent-at-exit). Now we need to have a notion of how to provide safety with multiple threads, or how to program the multi-threaded environment consistently, with interspersesd MPI calls. - - - To summarize, I seriously propose that in absence of an in-depth debate and specification of what thread safety means in MPI1, that we abandon this requirement altogether (analogous to the abandonment of non-blocking collective operations). If thread safety were to remain a de jure requirement of MPI1, then I ask that there be examples (analogous to or supersets of our contexts examples, pt2pt examples, and collective examples) illustrating same. If this is to be an added task of my subcommittee [which makes reasonablee sense to me] then I am eager for assistance nonetheless. I would want to see what people think existing thread practice is, what the design choices are, and which we choose to support, as well. It is not obvious to me that we really know what we mean (formally, practically) by "thread safety" for SPMD/MPMD message passing applications. Recall that there are at least three kinds of threads: O/S threads, compiler threads, user threads (we seeem to really mean the latter in our discussions). Thanks + please advise soonest. Tony Skjellum PS References to accessible texts or papers or software (eg, portable thread packages) are acceptable forms of advice. PPS I would like to have a new draft of the context chapter out by August 1 (with possible revisions by August 5). I am getting one extremely negative set of feedback from a single vendor representative, and one more balanced feedback (ie, only two people are communicating with me on the context chapter). I am not seeing widespread debate over the context chapter. This MUST happen now, between the meetings, since we have our best current draft available. We will not be successful if we are debating it all again at the next meeting without careful thought now (eg, on the threads issue). From owner-mpi-core@CS.UTK.EDU Sun Jul 18 11:29:46 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA08283; Sun, 18 Jul 93 11:29:46 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA03699; Sun, 18 Jul 93 11:30:26 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sun, 18 Jul 1993 11:30:25 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA03600; Sun, 18 Jul 93 11:28:50 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA19772; Sun, 18 Jul 93 10:28:48 CDT Date: Sun, 18 Jul 93 10:28:48 CDT From: Tony Skjellum Message-Id: <9307181528.AA19772@Aurora.CS.MsState.Edu> To: mpi-collcomm@cs.utk.edu, mpi-context@cs.utk.edu, mpi-core@cs.utk.edu, mpi-pt2pt@cs.utk.edu Subject: Heterogeneous communication proposal Dear colleagues: In order to make inter-vendor MPI implementations and cluster computing with MPI even a reasonable possibility, I suggest that we need to adopt the requirement that data formats follow IEEE Std 1596.5-1993 Data Transfer Formats Optimized for SCI. I propose that debate be started on this topic, and that a presentation be made at MPI in which the features of 1596.5-1993 are discussed and elaborated. Currently, there is little hope for standardization between vendors (or home-brew heterogeneous MPI) implementations. We recognize that XDR is inefficient, so this IEEE standard seems the logical alternative. If we say nothing, implementations will surely become incompatible. I volunteer to champion this effort, but only after the context chapter issues are resolved (so for September meeting or later). It is very important, to my mind, that we embrace other reasonable standards in creating MPI, such as this data standard. - Tony Skjellum Enclosure: From dbg@SLAC.Stanford.EDU Thu Jul 15 15:37:11 1993 Date: Thu, 15 Jul 1993 12:56:25 -0800 From: dbg@SLAC.Stanford.EDU (Dave Gustavson) Subject: SCI Data Transfer Formats standard approved To: sci_announce@hplsci.hpl.hp.com X-Envelope-To: sci_announce@hplsci.hpl.hp.com Content-Transfer-Encoding: 7BIT X-Sender: dbg@scs.slac.stanford.edu Content-Length: 4329 X-Lines: 86 Status: RO In its June 1993 meeting, the IEEE Standards Board approved: IEEE Std 1596.5-1993 Data Transfer Formats Optimized for SCI. (The approved document was Draft 1.0 8Dec92, but with significant edits to clarify the vendor-dependent formats listed in the appendix.) Congratulations to the working group, and especially to working group chairman David James! This new standard defines a set of data types and formats that will work efficiently on SCI for transferring data among heterogeneous processors in a multiprocessor SCI system. This work has attracted much interest, even beyond the SCI community. It solves a difficult problem that must be faced in heterogeneous systems. Over the years a great amount of effort has been invested in translating data among dissimilar computers. Computer-bus bridges have incorporated byte swappers to try to handle the big-endian/little-endian conversion. Software and hardware have been used to convert floating point formats. It was always tempting to have the hardware swap byte addresses to preserve full-bus-width integers, which seem to look the same on big- and little-endian machines, and then not swap bytes when passing character strings. But finally we understood that this problem cannot be solved by the hardware (at least until some far-future day when we all use standardized fully tagged self-describing data structures!). The magnitude of the problem became clearer during work on Futurebus+, where we had to deal with multiple bus widths and their interfaces with other standards like VME and SCI. When you observe data flowing along paths of various widths through a connected system, you see how hardware byte-swappers can arbitrarily scramble the data bytes of various number formats such as long integer or floating point. Furthermore, the scrambling may depend on the particular path used and on the state of the bridge hardware at the time the data passed through! Finally the solution became clear: first, keep the relative byte address of each component of a data item fixed as it flows through the complex system. (This is now referred to as the "address invariance" principle.) Thus, character strings arrive unchanged, but other data items may have been created with their bytes in inconvenient (but well-defined) places. Then provide the descriptive tools needed to tell the compiler what the original format of the data was. (That is what this standard does.) The compiler knows the properties of the machine for which it is compiling, and thus now has enough information to allow it to generate code to perform the needed conversions before trying to do arithmetic on the foreign data. For example, when the compiler loads a long integer into a register it may swap bytes to convert from little-endian to big-endian significance, so that the register will contain the correct arithmetic value for use in calculations. Similarly, when an arithmetic result is stored back into a structure that is declared with foreign data types the compiler ensures that the conversions are done appropriately before the data are stored. This capability is critical for work in heterogeneous multiprocessors, but it is also useful for interpreting data tapes or disk files that were written on a different machine as well. The IEEE Std 1596.5-defined descriptors include type (character, integer, floating), sizes, alignment, endian-ness, and atomic properties (can I be certain this long integer is always changed as a unit, never by a series of narrower loads and stores that might allow inconsistent data to be momentarily visible to a sharing machine). The standard also includes a C-code test suite that can be used to check the degree of compliance of a given implementation. The chairman is Dr. David V. James, MS 301-4G, Apple Computer, 20525 Mariani Avenue, Cupertino, CA 95014, 408-974-1321, fax 408-974-9793, dvj@apple.com. Again, my hearty congratulations on a job well done! David Gustavson, SCI (IEEE Std 1596-1992 Scalable Coherent Interface) chair David B. Gustavson phone 415/961-3539 SCI (ANSI/IEEE Std 1596 Scalable Coherent Interface) chairman SLAC Computation Research Group, Stanford University fax 415/961-3530 POB 4349, MS 88, Stanford, CA 94309 dbg@slac.stanford.edu From owner-mpi-core@CS.UTK.EDU Thu Aug 5 16:02:07 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA29548; Thu, 5 Aug 93 16:02:07 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA16077; Thu, 5 Aug 93 16:01:27 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 5 Aug 1993 16:01:26 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK) id AA16069; Thu, 5 Aug 93 16:01:25 -0400 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA12219; Thu, 5 Aug 1993 16:01:24 -0400 Date: Thu, 5 Aug 1993 16:01:24 -0400 From: walker@rios2.epm.ornl.gov (David Walker) Message-Id: <9308052001.AA12219@rios2.epm.ornl.gov> To: mpi-core@cs.utk.edu Subject: Next MPI meeting The next meeting of the Message Passing Interface (MPI) Forum will take place in Dallas, August 11-13, 1993. Details are given below. The minutes of the last meeting in May are not yet available electronically, but should be soon. The ongoing email discussion on MPI standardization issues can be obtained from netlib. To find out what is available via email send the following message to netlib@ornl.gov: send index from mpi or use xnetlib ("send index from xnetlib" for info). The schedule for future meetings is as follows: August 11-13, 1993 Dallas September 22-24, 1993 Dallas October 27-29, 1993 Europe (date tentative) MPI Forum Meeting, August 11-13, 1993 ===================================== The next meeting of the MPI forum will take place at Bristol Suites Hotel 7800 Alpha Road Dallas, Texas The meeting will start at 1pm on Wednesday August 11, 1993, and finish at noon on Friday, August 13, 1993. Rooms are $89 per night and reservations may be made by calling (214) 233-7600 (mention MPI meeting). The meeting registration fee will be $75. Please make checks and POs payable to University of Tennessee. The registration fee will be collected at the meeting. The registration fee will go for coffee breaks, meeting rooms, AV and printer rentals. Certain organizations need to see a registration form before giving their employees money for meetings like this. You can get such a form in PostScript by sending the following message to netlib@ornl.gov: send mpi-form.ps from mpi There is no need to send me the registration form or bring it to the meeting. TBS Shuttle Service will be providing complimentary shuttle service to and from the airports. If you fly into DFW, use their courtesy telephone and dial 03. If you fly into Love Field, you'll have to use a pay phone. They can be reached at 817-267-5150. Upon boarding the shuttle refer to the MPI meeting. We have NOT been able to make any special arrangements with airlines to get reduced fares. We have secured limited funding from ARPA/NSF for travel expenses of MPI meeting participants who are from U.S. universities. If you would like to apply for financial support to attend the June MPI meeting please send email to me at walker@msr.epm.ornl.gov, with justification of why you need support and an estimate of your travel expenses. Please send comments and/or suggested changes to the agenda below to me at walker@msr.epm.ornl.gov Provisional Agenda for MPI Meeting, August 11-13, 1993 Wednesday 1:00-3:00 Subcommittee meetings 3:30-6:00 Discussion of revised point-to-point draft (Snir) 7:30-10:30 More subcommittee meetings if necessary Thursday 9:00-11:00 Collective Communication (Geist) 11:00-12:00 Subset general discussion (Huss-Lederman) 12:00-1:30 Lunch (provided) 1:30-6:00 First reading of contexts chapter (Skjellum) 6:00-8:00 Dinner (attendees pay, but hotel provides transport to area restaurant) 8:00-10:00 Continued informal subcommittee meetings if necessary Friday 9:00-10:30 Environmental Inquiry (Gropp) 10:30-12:00 Topologies second reading (Huss-Lederman) From owner-mpi-core@CS.UTK.EDU Fri Oct 8 10:30:16 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA00668; Fri, 8 Oct 93 10:30:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA07018; Fri, 8 Oct 93 10:28:57 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 8 Oct 1993 10:28:54 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from DASHER.CS.UTK.EDU by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA06975; Fri, 8 Oct 93 10:28:52 -0400 From: Jack Dongarra Received: by dasher.cs.utk.edu (5.61+IDA+UTK-930922/2.7c-UTK) id AA09438; Fri, 8 Oct 93 10:28:51 -0400 Date: Fri, 8 Oct 93 10:28:51 -0400 Message-Id: <9310081428.AA09438@dasher.cs.utk.edu> To: mpi-core@cs.utk.edu Subject: schedule for mpi Here is the schedule for completing MPI. No technical or conceptual changes are expected to be made to the draft after 10/8/93. 1) 10/9/93 Steve Otto will take control of all sections of the MPI draft as of Saturday, 10/9/93. 2) 10/13-14/93 A small group, (Al Geist, Rusty Lusk, Steve Otto, Tony Skjellum and Marc Snir) are planning to meet at Argonne on October 13th and 14th to unify and complete the draft in preparation for the Supercomputing '93 meeting in Portland. 3) 10/15-29/93 The full committee will have a chance to proof-read the draft from October 15th through October 29th. Only typos and snafus can be corrected. (Steve Otto will post the draft.) 4) 11/1/93 On November 1st, Steve Otto will send the draft to the vendor's for copying. The vendors participating in MPI have agreed to each make 400 copies of the draft and have it available during the Supercomputing '93 meeting and at the workshop on Friday, November 19th. Steve Otto will send the tex and postscript of the draft to: Marc Snir, IBM Paul Pierce, Intel James Cownie, Meiko Peter Madams, Ncube Adam Greenberg, TMC Joel Williamson, Convex on November 1st. (Steve Otto will have details on the specification for printing and where to send the report.) 5) 11/15/93-1/19/94 Supercomputing'93 Meeting Workshop on MPI: Friday, November 19th, 8:30-12:00. 6) 11/19/93-1/15/94 There will be an open comment period for MPI from November 19th until January 15th. Comments can be sent to mpi-forum@cs.utk.edu. 7) David Walker will be the point of collection for comments. He will feed the comments to the appropriate subcommittees. 8) 2/?/94 We will have a meeting sometime in February, perhaps in Europe, to finish the draft and make it official. We may consider further changes/additions at the Europe meeting. (We are looking for dates and a host site.) Hope this covers most things, Regards, Jack From owner-mpi-core@CS.UTK.EDU Wed Oct 20 20:06:23 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA02837; Wed, 20 Oct 93 20:06:23 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA03558; Wed, 20 Oct 93 20:05:16 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 20 Oct 1993 20:05:15 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from win31.nas.nasa.gov by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA03535; Wed, 20 Oct 93 20:05:09 -0400 Received: by win31.nas.nasa.gov (5.67-NAS.6/NAS.2-sgi) id AA25351; Wed, 20 Oct 93 17:05:07 -0700 Date: Wed, 20 Oct 93 17:05:07 -0700 From: fineberg@nas.nasa.gov (Samuel A. Fineberg) Message-Id: <9310210005.AA25351@win31.nas.nasa.gov> To: mpi-core@cs.utk.edu Subject: Draft According to Jack's schedule, there was supposed to be a draft available as of the 15th? Is someone going to post it? Thanks, Sam From owner-mpi-core@CS.UTK.EDU Wed Oct 20 20:14:37 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA02859; Wed, 20 Oct 93 20:14:37 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA03906; Wed, 20 Oct 93 20:14:05 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 20 Oct 1993 20:14:04 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from gw1.fsl.noaa.gov by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA03898; Wed, 20 Oct 93 20:14:02 -0400 Received: by gw1.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA22104; Thu, 21 Oct 93 00:11:34 GMT Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA08819; Wed, 20 Oct 93 18:18:33 MDT Message-Id: <9310210018.AA08819@nipmuc.fsl.noaa.gov> To: fineberg@nas.nasa.gov (Samuel A. Fineberg) Cc: mpi-core@cs.utk.edu Subject: Re: Draft In-Reply-To: Your message of "Wed, 20 Oct 1993 17:05:07 PDT." <9310210005.AA25351@win31.nas.nasa.gov> Date: Wed, 20 Oct 1993 18:18:33 -0600 From: Leslie Hart (NOAA/FSL) I found one on netlib (ftp netlib2.cs.utk.edu) that was dated 19 Oct. (It was in mpi/draft1019.ps). Regards, Leslie Hart (hart@fsl.noaa.gov) From owner-mpi-core@CS.UTK.EDU Thu Oct 21 04:32:25 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA04271; Thu, 21 Oct 93 04:32:25 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA28975; Thu, 21 Oct 93 04:31:07 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 21 Oct 1993 04:31:06 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA28909; Thu, 21 Oct 93 04:31:00 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA23381 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Thu, 21 Oct 1993 09:30:56 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA00578; Thu, 21 Oct 1993 09:29:55 +0000 Date: Thu, 21 Oct 1993 09:29:55 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9310210829.AA00578@tycho.co.uk> To: mpi-core@cs.utk.edu In-Reply-To: <9310210018.AA08819@nipmuc.fsl.noaa.gov> (hart@nipmuc.fsl.noaa.gov) Subject: Re: Draft Content-Length: 637 Leslie says > I found one on netlib (ftp netlib2.cs.utk.edu) that was dated > 19 Oct. (It was in mpi/draft1019.ps). I was expecting at least an announcement (and maybe even a copy of the document) on the mail list. So what is the status ? Is this silently appearing netlib version definitive ? If it's not going to appear by mail can I ftp it from somewhere ? James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Thu Oct 21 04:50:07 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA04510; Thu, 21 Oct 93 04:50:07 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA01379; Thu, 21 Oct 93 04:49:34 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 21 Oct 1993 04:49:33 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from THUD.CS.UTK.EDU by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA01373; Thu, 21 Oct 93 04:49:32 -0400 From: Jack Dongarra Received: by thud.cs.utk.edu (5.61+IDA+UTK-930922/2.7c-UTK) id AA05948; Thu, 21 Oct 93 04:49:31 -0400 Date: Thu, 21 Oct 93 04:49:31 -0400 Message-Id: <9310210849.AA05948@thud.cs.utk.edu> To: mpi-core@cs.utk.edu Subject: Re: Draft OK, the current draft is out on netlib. You can retrieve an up-to-date copy in postscript form by: 1) from any machine on the internet type: rcp anon@netlib2.cs.utk.edu:mpi/draft1019.ps draft1019.ps 2) anonymous ftp to netlib2.cs.utk.edu cd mpi get draft1019.ps quit 3) sending email to netlib@ornl.gov and in the message type: send draft1019.ps from mpi 4) use Xnetlib and click "library", click "mpi", click "mpi/draft1019.ps", click "download", click "Get Files Now". (Xnetlib is an X-window interface to the netlib software based on a client-server model. The software can be found in netlib.) A compressed tar file of the latex source is there as well in draft1019-tar.Z. Send comments to Steve Otto, otto@cse.ogi.edu. The schedule follows... 3) 10/15-29/93 The full committee will have a chance to proof-read the draft from October 15th through October 29th. Only typos and snafus can be corrected. (Steve Otto will post the draft.) 4) 11/1/93 On November 1st, Steve Otto will send the draft to the vendor's for copying. The vendors participating in MPI have agreed to each make 400 copies of the draft and have it available during the Supercomputing '93 meeting and at the workshop on Friday, November 19th. Steve Otto will send the tex and postscript of the draft to: Marc Snir, IBM Paul Pierce, Intel James Cownie, Meiko Peter Madams, Ncube Adam Greenberg, TMC Joel Williamson, Convex on November 1st. (Steve Otto will have details on the specification for printing and where to send the report.) 5) 11/15/93-1/19/94 Supercomputing'93 Meeting Workshop on MPI: Friday, November 19th, 8:30-12:00. 6) 11/19/93-1/15/94 There will be an open comment period for MPI from November 19th until January 15th. Comments can be sent to mpi-forum@cs.utk.edu. 7) David Walker will be the point of collection for comments. He will feed the comments to the appropriate subcommittees. 8) 2/?/94 We will have a meeting sometime in February, perhaps in Europe, to finish the draft and make it official. We may consider further changes/additions at the Europe meeting. (We are looking for dates and a host site.) 9) 3/94? We may need another meeting to fix and make it official. Hope this covers most things, Regards, Jack From owner-mpi-core@CS.UTK.EDU Thu Oct 21 14:55:26 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA09846; Thu, 21 Oct 93 14:55:26 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA13602; Thu, 21 Oct 93 14:54:05 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 21 Oct 1993 14:54:03 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA13594; Thu, 21 Oct 93 14:54:01 -0400 Received: by merckx.cse.ogi.edu (16.8/16.2) id AA25274; Thu, 21 Oct 93 11:52:38 -0700 Date: Thu, 21 Oct 93 11:52:38 -0700 From: Steve Otto Message-Id: <9310211852.AA25274@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: MPI DRAFT Apparently, I screwed up. I sent this out a few days ago, and I thought everyone had received it. I was mistaken. --Steve Dear MPIF, Sometime today the current version of the MPI Draft will be made available to you through netlib. You are requested to study it *quickly*, since we have only until Nov 1. At this date, the draft will be printed for SuperComputing '93. Anyway, send all your comments to me. I will act as a filter and route the comments, as necessary, to the correct author. Unfortunately, the document is not nearly as clean and pristine as I would like it to be. All of us have been quite busy trying to straighten out more substantive issues, and so various stylistic points have been overlooked. Many words are wrong or have been mis-spelled. Go ahead and tell me about them. I include a set of notes below -- these are meant to draw your attention to particular changes that have occurred in the last few days. No we are not trying to decoy you away from our *real* changes! ;^) OK, read it and talk to me. --Steve ------------------------------------------------------------------------ Dr Steve Otto | Dept of Computer Science and Engineering *___________ | Oregon Graduate Institute of Science & Technology _____) | 20000 NW Walker Rd, PO Box 91000 / | Portland, OR 97291-1000 / | email: otto@cse.ogi.edu /o | Phone: (503) 690-1486 | FAX: (503) 690-1553 ------------------------------------------------------------------------ Substantative changes made in the Mtg of Oct 12-14, R. Lusk, M. Snir, S. Otto, A. Skjellum, A. Geist (in random order) 1. collective: mixed type return from REDUCE/MIN_LOC and MAX_LOC hard to use in a portable manner. Definition changed so that ranks are coerced to type of data values and carried along in that form; int ranks can be coerced back to ints after call is complete. 2. Type matching requirement for gather/scatter functions changed to a pairwise matching requirement, and also, made more restrictive than pt-pt by requiring EXACT match in terms of number of elements. 3. displs argument added to gatherv, scatterv, etc Allows easy control of placement of data. 4. Logical and Bitwise versions of AND, OR, XOR REDUCE operators made explicit. Type-validity table added. 5. Name of ALLCAST changed to ALLGATHER. Thought to be more clear, since "V" version of function naturally follows from thinking of GATHERV, not BCAST. 6. added HOST INQ function 7. deleted TYPE DECODE INQ function 8. Changed BUFFER CONTROL INQ function 9. Order of topology chapter changed so that high-level is presented first. This re-work is unfinished and the chapter is currently in an inconsistent state...Snir then worked on it and reports the following changes: % changes made by Marc Snir: % * changed all names % * low-level topology functions moved out of standard (in the spirit of the % univeral point-to-point functions) % * added two new functions that extract neighborhood info fro general graph % topology (otherwise, general grphas look silly: you can just cahce and % retrieve the info) % * added several examples % * changed several values from integer to logical % * clarified how graphs are represented (taking into account the difference % between C arrays and Fortran arrays) 10. iis is out of date in several ways: Collective names changed, topology names changed, some functions are gone... 11. Language bindings do not yet exist for Topologies Chapter. -- Just didn't get around to it. 12. \discuss macro was made NULL. If your "discuss" said something important, then put it in the text. 13. Simple, point to point versions of the collective functions (many of them) were written as an instructive aid to the committee. They were, however, left out of the draft document since they are deadlock-prone and to make them deadlock-free but keep them illustrative...is problematical. Did I hear someone say JOD? They are still there (commented out) in coll.tex if you are interested. From owner-mpi-core@CS.UTK.EDU Mon Oct 25 13:24:47 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA08569; Mon, 25 Oct 93 13:24:47 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA18700; Mon, 25 Oct 93 13:23:09 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 25 Oct 1993 13:23:08 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA18686; Mon, 25 Oct 93 13:23:07 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA07206; Mon, 25 Oct 93 12:22:47 CDT Date: Mon, 25 Oct 93 12:22:47 CDT From: Tony Skjellum Message-Id: <9310251722.AA07206@Aurora.CS.MsState.Edu> To: fineberg@nas.nasa.gov Subject: Re: Draft Cc: mpi-core@cs.utk.edu I believe a draft is available now. Please ask Steve Otto. -Tony From owner-mpi-core@CS.UTK.EDU Wed Oct 27 09:09:36 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA05034; Wed, 27 Oct 93 09:09:36 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA08416; Wed, 27 Oct 93 08:08:36 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 27 Oct 1993 08:08:34 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA08380; Wed, 27 Oct 93 08:08:12 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA14272 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Wed, 27 Oct 1993 12:08:08 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA04766; Wed, 27 Oct 1993 13:07:57 +0000 Date: Wed, 27 Oct 1993 13:07:57 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9310271207.AA04766@tycho.co.uk> To: otto@merckx.cse.ogi.edu Cc: mpi-core@cs.utk.edu In-Reply-To: <9310211852.AA25274@merckx.cse.ogi.edu> (message from Steve Otto on Thu, 21 Oct 93 11:52:38 -0700) Subject: Re: MPI DRAFT Content-Length: 6214 Steve, Thanks to you and the rest of the editorial group. (Especially whoever did the scatter/gather derived datatype pictures !). Significant issue ================= Collective (Reduction operations) We have only a weak specification of the semantic requirement for these. Do we want reproducibility ? (I can't remember what we concluded. Who can ?) In particular for the ALL REDUCE operations we don't currently say that everyone gets the SAME result. I think we should (at least) say this, even if we want to be relaxed about the reproducibility of that result. Global edits ============ It would be nice to change "parameter" to "argument" throughout. (Possibly with a minor discussion in the terms and conventions section, something like ... Since the word PARAMETER is a keyword in the Fortran language, we use the word argument to denote the arguments to a subroutine. These are normally referred to as parameters in C, however we expect that C programmers will understand the word arguments (which has no specific meaning in C), thus allowing us to avoid unnecessary confusion for Fortran programmers. ) Typos ===== I've included here some changes to the Fortran binding for routines which need to return/take character strings. I believe that these should take CHARACTER*(*) arguments, in accordance with standard Fortran practice. The current draft has these as INTEGER and also passes (a now unnecessary) length. Here are a set of typos in Oct 19 draft... Page Line Change Anonymous (Acknowledgements) Line 21 I think it is Mark Sears, (not Marc) 9 25 delete the : 28 "allocated on user storage or on system" change both "on"s to "in"s 10 3 Add "an" at beginning of the line 8 "is the user resposibility" => "is the user's responsibility" 20 "begginning" => "beginning" 20 "indecates" => "indicates" 12 19 "spedify" => "sepcify" 39 "processor" => "process" 44 "implicit global states" => "implicit state" 47 "or other events" => "or with other events" 13 11 "to always" => "always to" 12 "consecutive integers" => "consecutive integers in the range 0 .. groupsize-1" 14 5 ":" => "." 6 "however" => "However" 15-18 Was this what we said ? I thought we were much stronger than this. (see Significant issues above). 43 "As an" => "An" 15 19 "SIGFPE,and" => "SIGFPE or" 16 21 "manaipulation" => "manipulation" 22 23 delete "and is seen as a quality of implementation issue" (Or Moose will be very unhappy !) 30 "MPI implementation" => "An MPI implementation" 26 11 "types as" => "types from" 27 10 "part of" => "some of" 20 "either ends" => "either end" 29 29 "Such call" => "Such a call" 31 40 "associates" => "associate" 41 delete "about" 43 "buf copying" => "copy" 33 7 "In multi-threaded" => In a multi-threaded" 34 2 "either receives" => "either receive" 39 4 "user" => "user's" 42 39 "semantics as" => "semantics to" 43 48 "correctly the sends and receives" => "the sends and receives correctly" 46 27 "allows" => "allow" 36 "have used" => "have been used" 58 15 "Each of the sends can match each of the receives" => "From a type matching viewpoint any of the sends legally matches any of the receives" 16 "Overlaping" => "Overlapping" 20 "Or one of them have" => "Or one of them has" 22 "overlaping" => "overlapping" 68 27 "This section is not part of the MPI definition" should be made MUCH more significant (Bold, or margin or something) 83 48 "exectuted" => "executed" 88 45 Lots of (unnecessary) blank space 89 40 Ditto 106 28 "allows to invoke" => "allows the invocation of" 43 "users that" => "users who" 44 "and ignore the rest of this chapter" should be in bold or CAPS 107 9 "users that" => "users who" 108 26 "The groups corresponding to this communicator do not appear as pre-defined constants," => "The group embedded in this communicator does not appear as a pre-defined constant," 115 1 "with all its cahed information". This is WRONG. It should say something like "For each piece of cahed information on the original communicator the relevant copy_fn is called. This can either copy the cached information, or not, as is appropriate." 122 6 "in size or" => "in size to, or" 123 40 "put into this attribute set" => "attached to this communicator" 124 13 "are invoked." => "are invoked in an arbitrary order." 128 42 "if ((me % 2) == 0)" => "if ((me % 2) == 0 && (me + 1) < size))" So don't send if I don't have a neighbour. Should also remove all the modulusing in the sends and receives. So 43 "(...,((me+1)%size)," => "...,me+1," 45 "((me-1+size)%size)" => "me-1" 152 47,48 Should read MPI_GET_PROCESSOR_NAME(NAME,IERROR) CHARACTER*(*) NAME INTEGER IERROR 154 41 "no to" => "not to" 156 20 "an algorithm" => "a program" 157 31 "can not" => "cannot" 43 "Such handler" => "Such a handler" 48 "MPI_WORLD" => "MPI_COMM_WORLD" 158 1 "with a communicator;" => "to a communicator." 159 16,17 Should read MPI_ERROR_STRING(ERRORCODE, STRING, IERROR) INTEGER ERRORCODE, IERROR CHARACTER *(*) STRING 40 "before any other MPI routines are called" => "before any other MPI routine (apart from MPI_INITIALIZED) is called" -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Wed Oct 27 11:08:43 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA05644; Wed, 27 Oct 93 11:08:43 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA18116; Wed, 27 Oct 93 10:06:41 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 27 Oct 1993 10:06:31 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from sun2.nsfnet-relay.ac.uk by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA17917; Wed, 27 Oct 93 10:06:08 -0400 Via: uk.ac.southampton; Wed, 27 Oct 1993 14:02:31 +0000 Via: brewery.ecs.soton.ac.uk; Wed, 27 Oct 93 12:07:26 GMT From: David Pritchard Received: from marston.ecs.soton.ac.uk by brewery.ecs.soton.ac.uk; Wed, 27 Oct 93 12:19:30 GMT Date: Wed, 27 Oct 93 12:19:31 GMT Message-Id: <12024.9310271219@marston.ecs.soton.ac.uk> To: otto@merckx.cse.ogi.edu, jim@meiko.co.uk Subject: Re: MPI DRAFT Cc: mpi-core@cs.utk.edu > Significant issue > ================= > Collective (Reduction operations) > We have only a weak specification of the semantic requirement for these. > Do we want reproducibility ? (I can't remember what we concluded. Who can ?) > As I recall we concluded: Reproducibility: Yes Determinacy: No From owner-mpi-core@CS.UTK.EDU Wed Oct 27 11:26:00 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA05730; Wed, 27 Oct 93 11:26:00 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA20497; Wed, 27 Oct 93 10:25:11 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 27 Oct 1993 10:25:10 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from msr.EPM.ORNL.GOV by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA20482; Wed, 27 Oct 93 10:25:08 -0400 Received: by msr.EPM.ORNL.GOV (4.1/1.34) id AA22052; Wed, 27 Oct 93 10:24:41 EDT Date: Wed, 27 Oct 93 10:24:41 EDT From: geist@msr.EPM.ORNL.GOV (Al Geist) Message-Id: <9310271424.AA22052@msr.EPM.ORNL.GOV> To: D.J.Pritchard@ecs.soton.ac.uk, jim@meiko.co.uk, otto@merckx.cse.ogi.edu Subject: Re: MPI DRAFT Cc: mpi-core@cs.utk.edu > Significant issue > ================= > Collective (Reduction operations) > We have only a weak specification of the semantic requirement for these. > Do we want reproducibility ? (I can't remember what we concluded. Who can ?) > I just checked my notes from the last meeting. The decision then was that if any of the reduce functions are called multiple times with the same communicator and data, then the same result is seen by all the group members every time. I placed a statement to this effect in a draft of October 14, seems to have vanished in the latest draft. Al Geist From owner-mpi-core@CS.UTK.EDU Wed Oct 27 18:56:33 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA08982; Wed, 27 Oct 93 18:56:33 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA00728; Wed, 27 Oct 93 18:55:40 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 27 Oct 1993 18:55:38 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA00696; Wed, 27 Oct 93 18:55:30 -0400 Received: by merckx.cse.ogi.edu (16.8/16.2) id AA03277; Wed, 27 Oct 93 15:53:56 -0700 Date: Wed, 27 Oct 93 15:53:56 -0700 From: Steve Otto Message-Id: <9310272253.AA03277@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: MPI Draft; new version; actions taken Hello All, I have received many useful comments and suggestions; I am now putting out a new version of the draft with most of your suggestions in it so that we don't start repeating the same comments. I have asked Jack to put the new version of the draft in netlib. Appended below is my attempt to document what I have done. You will see comments, followed by an explanation of what I have (or have not yet) done about it in the new version. I have not acted on the comments pertaining to context or environmental, since these are currently in the hands of Tony and Bill, respectively. Please continue sending your comments...we have till next Monday. Note that I will be out on Thursday, so don't be alarmed when I don't immediately answer back. Whew! --Steve Actions List for MPI Comments received. Steve Otto *************** Point to Point: *************** >From snir@watson.ibm.com Mon Oct 25 21:23 PDT 1993 Received: from iliamna.cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA05917; Mon, 25 Oct 93 21:23:53 -0700 Return-Path: Received: by iliamna.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.17) id ; Mon, 25 Oct 93 21:25 PDT Message-Id: Received: from YKTVMV by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 5425; Tue, 26 Oct 93 00:25:28 EDT Date: Tue, 26 Oct 93 00:25:09 EDT >From: "Marc Snir" X-Addr: (914) 945-3204 (862-3204) 28-226 IBM T.J. Watson Research Center P.O. Box 218 Yorktown Heights NY 10598 To: otto@iliamna.cse.ogi.edu Subject: Note from PCTERM3 Reply-To: SNIR@watson.ibm.com Status: RO ------------------------------- Referenced Note --------------------------- First batch of corrections I use px,ly for page x, line y Explanations for changes I propose are in square brackets [] Global changes are followed by a (probably incomplete) list of places where they occur. ******************************** GLOBAL CHANGES FLAG arguments have type LOGICAL in Fortran replace INTEGER FLAG by LOGICAL FLAG p32,l44 p36,l1 p38,l40 done status argument has type (Status) add (Status) at end of description of status parameter p19,l26 p32,l20 p32,l40 p35,l17 p35,l43 p37,l35 ?? p38,l12 ?? p44,l33 p45,l14 done Datatype argument has type (handle) add (handle) after description of {\tt datatype} argument. or {\tt sendtype}, {\tt recvtype} p17,l6 p44,l15 p21,l5 p44,l25 p27,l33 p28,l5 p30,l10 p30,l32 p31,l5 p31,l25 p40,l37 p41,l10 p41,l32 p42,l5 p45,l6 done. AND ALSO: p19,l21 communication object (handle) -> communication request (handle) handle to communication object -> communication request (handle) p30,l16 p32,l18 p30,l39 p32,l37 p31,l11 p31,l31 p40,l42 p41,l17 p41,l39 p42,l22 p43,l19 done. array of handles to communication objects -> array of requests array of communication objects -> array of requests p35,l14 p42,l44 p35,l38 p36,l16 p36,l36 done. I put in: "array of requests (array of handle)" ************************************ p6,l17 No support -> Support done. p6,l36 recv -> receive done p7,l18 Fortran -> Fortran 77 done p9,l11 Such an operation -> Usually, such an operation [Cancel is a local operation, but may require communication] *NOT* done: unsettling to the reader...does CANCEL require communication with the user-level process? Perhaps communications need to occur, but it is hidden from the user and so may as well be considered local, no? p9,l42 as as -> as done p10,l6 MPI_GROUP_FREE -> free the object done p10, l15 Array of handles -> Array arguments done [they are not always handles] p10,l16 that is an array-of-handles. The ... -> that is an array of opaque objects, or an array of handles [no italics]. An ... done p10,l21 [Add at end of paragraph] -> The same approach is followed for other array arguments. done p12,l12 The same MPI subroutine -> An MPI subroutine with a choice argument done p12,l13 Figure 2.4.2 -> Figure 2.1 done p12,l19 spedify -> specify done p12,l30 Buffer pointers are -> Choice arguments are pointers done p13,l32 The recommended -> A high quality done p13,l35 an error value -> a code done p16,l21 manaipulation -> manipulation done p17,l38 delete MPI_DOUBLE_COMPLEX line done p18,l23 ... C; MPI_REAL2 ... -> ... C; \const{MPI\_DOUBLE\_COMPLEX} for double precision complex in Fortran; MPI_REAL2 ... [DOUBLE PRECISION COMPLEX is not standard Fortran 77] done p18,l45 TagRange -> \mpifunc{MPI\_GET\_VALID\_TAG\_RANGE} done p18,l20 (state) -> (handle) done -- actually p19. p21,l48 [add at end of paragraph] -> Note that the receiver is not required to receive messages in the order they were sent, if they have distinct envelopes. done, but I'm unhappy about the placement here...it puts this statement in the same paragraph with threads, while this is not necessarily a threads issue. p22,l30 provide information -> provide information and control done p27,l4 posted; Otherwise -> posted; otherwise done p29,l39 communication objects -> communication request objects [communication objects are request objects] done p29,38 An opaque communication object -> A request object done p30,l3 nonblocking communication -> nonblocking send. done p30,l20 *request -> request [The call modifes the opaque object pointed at by request, but does not modify the handle] done p30,25 nonblocking communication -> nonblocking send. done p30,l43 *request -> request done p30,l48 nonblocking communication -> nonblocking send. done p31,l11 *request -> request done p31,l31 *request -> request done p31,40 communication object and associates it with the request. -> communication request and associates it with the handle. done p31,l43 buf -> start done p32,l5 [delete] or any other location that can be referenced by the send operation [there are none] done p32,l5 [delete] or any other location that can be referenced by the receive operation done p32,l17 IN request ... -> {\INOUT}{request}{communication request (handle)} done p32,l22 request -> *request [WAIT nullifies the handle] done p32,l37 IN request ... -> {\INOUT}{request}{communication request (handle)} done p32,l38 logical -> (logical) done p32,l41 request -> *request done p33,l26 they will be also used to complete any other nonblocking communication call -> they are also used to complete all other nonblocking communication calls. done p35,l26 Returns in ... with the status. -> Returns in \mpiarg{index} the index of that request in the array and returns in \mpiarg{status} the status of the completing communication. done p35,l30 is is -> is done p35,l39 index of handle for operation -> index of operation done p35,l42 true ... -> \const{true} if one of the operations is complete (logical) done p36,l16 IN -> INOUT done p37,l31 mix sends -> mix of sends done p37,l33 and the array_of_statuses -> and the entries of the array_of_statuses done p38,l7 first message -> first arriving message done p38,l9 second message -> second arriving message done p38,l20 [add new paragraph] An MPI implementation of \mpifunc{MPI\_PROBE} need to guarantee progress: if a call to \mpifunc{MPI\_PROBE} has been issued by a process, and a send that matches the probe has been initated by some process, then the call to \mpifunc{MPI\_PROBE} will return, unless the message is received by another concurrent receive operation (that is executed by another thread at the probing process). Similarly, if a process busy waits with \mpifunc{MPI\_IPROBE} and a matching message has been issued, then the call to \mpifunc{MPI\_IPROBE} will eventually return {\tt flag = true} unless the message is received by another concurrent receive operation. done p40,l15 add implementation note \implement{ If a send operation uses an ``eager'' protocol (data is transfered to the receiver before a matching receive is posted), then the cancelation of this send may require communication with the intended receiver, in order to free allocated buffers. On some systems this may require to interupt the intended receiver. Therefore, it is expected that \mpifunc{MPI\_CANCEL} will be an expensive operation on many systems, to be used only exceptionally. Note that, while communication may be needed to implement \mpifunc{MPI\_CANCEL}, this is still a local operation, in the sense that its completion does not depend on the code executed by other processes. If processing is required on another process, this should be transparent to the application (hence the need for an interupt and an interupt handler). } done p40,l20 to the communication object -> to a communication request done p40,l23 The communication object -> The request done p40,l28 A communication object -> A request done p40,l30 a communication object -> a request done p40,l45 *request -> request done p40,l48 INTEGER BUF -> BUF(*) done p41,l1 The call ... -> [delete sentence] done p41,l21 *request -> request done p41,l23 INTEGER BUF -> BUF(*) done p41,l42 *request -> request done p41,l45 INTEGER BUF -> BUF(*) done p42,l13 *request -> request done p42,l15 INTEGER BUF -> BUF(*) done p42,l17 communication object -> request done p42,l33 updated -> accessed done p42,l44 requestss -> requests done p43,l8 communication object -> request done p43,l10 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l12 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l17 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l21 MPI_Commreq_FREE -> MPI_Request_free (or MPI_Comm_request_free) done p43,l22 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l27 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done [to be consistent with current terminology -- change need to be propagated to appendices] yes -- done. p42,l34 communication object -> request done p44,l46 the same tag ... [end of paragraph] -> the same communicator, but possibly different tags. The send buffer and receive buffers are disjoint, and may have different lengths and datatypes. [lengths] done! *************** Collective chapter: *************** 1. MPI_SEND --> MPI_Send in several places 2. Comments from David Walker: -------------------------------------------------------------------------- Steve and Al, I think the removal of the MPI_USER_REDUCE_SCATTER and MPI_USER_REDUCE_SCATTERA is illogical, since we still have the corrsponding routines for REDUCE and SCAN. If you want to get rid of routines here are 2 possibilities: 1) Remove all REDUCE_SCATTER routines. 2) Remove all noncommutative REDUCE and SCAN routines (the ones ending in "A") At this late stage I think it best to leave things as they were at the end of the last MPI meeting, i.e. keep MPI_USER_REDUCE_SCATTER and MPI_USER_REDUCE_SCATTERA. David -------------------------------------------------------------------------- Action taken: I have followed David's last remark, and: I think it best to leave things as they were at the end of the last MPI meeting, i.e. keep MPI_USER_REDUCE_SCATTER and MPI_USER_REDUCE_SCATTERA. done. NOTE: these now need language binding. 3. Fixed indent in "Example of Reduce" 4. Comments from Steven Huss-Lederman: - I can see how it happened, but it seems a little funny that MPI_ALLREDUCE is in section 4.5.2 "User-Reduce". It isn't a user reduce function just the result know to all. I've given them they're own section: \subsection{All-Reduce} \label{subsec:coll-all-reduce} - I just want to make sure that the MPI_USER_REDUCE_SCATTER(A) functions were actually removed from the chapter. they're back! *************** Topologies: *************** 1. Fixed Overfull on page 150 2. New language bindings from Rusty. I will try and integrate these with Rolf's new chapter, see below. done 3. Comments from Rolf Hempel: -------------------------------------------------------------------------- ... In general, I appreciate the work you have put into editing the MPI document, to improve its consistency. In particular, Marc has done a number of improvements to the topology chapter. I never had strong feelings about function names, and the new names are fine with me. Also, the added examples make the details easier to understand for the reader. It's difficult to decide whether the original or the new definitions for vectors "index" and "edges" are better. The new definitions save the additional entry in vector index, but they introduce the special case for accessing the first element. This problem is somehow alleviated by the introduction of the two new lookup functions for the graph neighbors, so that the user code does not have to deal with the special case. So, I have no strong objections against the change. The only thing I really don't like is the removal of the low-level mapping functions from the standard. The analogy with the low-level functions in PT2PT is only partially correct. There the functions mainly serve for the semantic definition of the higher-level functions, and good MPI implementations are expected to bypass them. In the case of the topology construction functions the situation is different. Here the layered implementation will be the general choice, and at our last meeting I even had the impression that the vendors are waiting for me to provide the (portable) high-level layer. This only works if every vendor provides the low-level functions for their system, so that the high-level functions can use them. In this sense, a better analogy for the low-level functions is the cacheing facility which (for good reason) is also part of the standard. I think this issue goes a bit beyond pure editing of the final document. After all, we have had several readings of the topology chapter, and the low-level functions passed them all. I don't want to insist on formal rules, but what was the purpose of all those readings, if we can now ignore them? As always happens, I found a number of mis-spellings and other inconsistencies in the current topology chapter version. How can I do the necessary changes without interfering with others? Do you control the update right for individual chapters? Does Marc still work on the topology chapter? Rolf -------------------------------------------------------------------------- Well put, Rolf! I will incorporate Rolf's version of topol.tex, which he has just sent me. And Rolf has sent a new version: -------------------------------------------------------------------------- ... Below you will find the new LaTex source and two entries in the bibliography which I changed slightly. Marc told me that he has no strong feeling about the low-level functions (he removed them from the standard), but that he would like to present them in the order they are presented now. So, I moved them back into the standard, but kept the order. -------------------------------------------------------------------------- Yes, I think this new order of presentation is much improved from the old version. done. 4. Comment from Steven Huss-Lederman: While doing the subset chapter I noticed the two new routines MPI_GRAPH_NNEIGHBORS and MPI_GRAPH_NEIGHBORS. I personally think it is a bad idea to have two names that are so close. Maybe put in the phrase DIM as in the Cartesian case. I don't really care what, just the similarity will probably cause people to get confused. *************** Contexts: *************** 1. In Tony's hands at the moment --- >From hender@macaw.fsl.noaa.gov Fri Oct 22 09:54 PDT 1993 Received: from gw1.fsl.noaa.gov by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA11305; Fri, 22 Oct 93 09:54:06 -0700 Return-Path: Received: by gw1.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA13627; Fri, 22 Oct 93 16:53:00 GMT Received: by macaw.fsl.noaa.gov (4.1/SMI-4.1) id AA17176; Fri, 22 Oct 93 10:55:31 MDT Date: Fri, 22 Oct 93 10:55:31 MDT >From: hender@macaw.fsl.noaa.gov (Tom Henderson) Message-Id: <9310221655.AA17176@macaw.fsl.noaa.gov> To: mpi-context@cs.utk.edu, otto@merckx.cse.ogi.edu Subject: Comments on Oct. 19 draft Status: RO Hi all, A few comments on the Inter-Communication section (5.5) and the Inter-Communication Examples section (5.8.7) of the October 19 draft... On page 118, line 46, the first sentence reads: "Intercommunicators can be created using the function MPI_COMM_DUP." We had a long discussion following the last meeting about how to describe rules for "safe" use of the inter-communicator construction routines. The result was a decision to drop the blocking versions: MPI_INTERCOMM_MAKE() and MPI_INTERCOMM_NAME(). It seems to me that using MPI_COMM_DUP() to duplicate an inter-communicator has the same synchronization problems as MPI_INTERCOMM_MAKE()-- both groups need to know what's going on. It's just another blocking inter-communicator construction routine. To be consistent, I think we need to delete this sentence or replace "can" with "cannot". On page 119, lines 6-12: "The explicit synchronization can cause deadlock in modular programs with cyclic communication graphs, even if: * within each local group calls are executed in the same order, * the local and remote operations are decoupled and the construction is performed "loosely synchronously" (by calling the two routines MPI_INTERCOMM_START() and MPI_INTERCOMM_FINISH())." The previous draft (9/27) read: "The explicit synchronization can cause deadlock in modular programs with cyclic communication graphs, even if within each local group calls are executed in the same order. So, the local and remote operations can be decoupled and the construction performed "loosely synchronously" by calling the two routines MPI_INTERCOMM_START() and MPI_INTERCOMM_FINISH()." I have two comments about this change: 1) In the current draft, the ONLY WAY to construct an inter-communicator is "loosely synchronously". The second bullet doesn't make sense any more. The local and remote operations are always decoupled. 2) My interpretation of the current draft is that there is no "safe" way to construct an inter-communicator. I think it may also be true that we don't know how to describe sufficient conditions for non-deadlocking for any series of collective operations on groups that overlap. If this is true, then we should make this statement a general warning about using any collective operations and not hide it here. Comments please? I don't have a good idea for fixing this text without knowing what lead to the change. What am I missing? On page 119, line 41 (typo): "MPI _arg" appears, looks like a typo. On page 120, lines 8-9 (typo): "This call starts off an inter-communicator creation operation, returning a handle for the completion of the operation in handle; ..." ^^^^^^ "This call starts off an inter-communicator creation operation, returning a handle for the completion of the operation in inter_request; ..." ^^^^^^^^^^^^^ On page 121, line 28 (clarification): I would like to add the following sentence: "All processes in the local and remote groups must provide the same name." This is not stated anywhere else. It used to be in an implementation note. On page 134, line 26 (typo): "group B" should be "group 1" This one's my fault, left over from several drafts ago... :-) On page 137, lines 24-40 (typos): "if (color == 0) { /* Group 0 communicates with group 1. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); MPI_Intercomm_finish (inter_request1, &myFirstComm); } else if (color == 1) { /* Group 1 communicates with groups 0 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 10", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); MPI_Intercomm_finish (inter_request1, &myFirstComm); MPI_Intercomm_finish (inter_request2, &mySecondComm); } else if (color == 2) { /* Group 2 communicates with group 1. */ MPI_Intercomm_name_start (myComm, "Connect 21", &inter_request1); MPI_Intercomm_finish (inter_request1, &myFirstComm); }" Names must match for all processes in local and remote groups. Fix is: if (color == 0) { /* Group 0 communicates with group 1. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); MPI_Intercomm_finish (inter_request1, &myFirstComm); } else if (color == 1) { /* Group 1 communicates with groups 0 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); ^^ MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); MPI_Intercomm_finish (inter_request1, &myFirstComm); MPI_Intercomm_finish (inter_request2, &mySecondComm); } else if (color == 2) { /* Group 2 communicates with group 1. */ MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request1); ^^ MPI_Intercomm_finish (inter_request1, &myFirstComm); } On page 138, lines 39-48 and page 139, lines 1-5 (typos): "if (color == 0) { /* Group 0 communicates with groups 1 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 02", &inter_request2); } else if (color == 1) { /* Group 1 communicates with groups 0 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 10", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); } else if () { /* Group 2 communicates with groups 0 and 1. */ MPI_Intercomm_name_start (myComm, "Connect 20", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 21", &inter_request2); }" Names must match for all processes in local and remote groups. Also, last elseif is a dinosaur. Fixes are: if (color == 0) { /* Group 0 communicates with groups 1 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 02", &inter_request2); } else if (color == 1) { /* Group 1 communicates with groups 0 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); ^^ MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); } else if (color == 2) ^^^^^^^^^^ { /* Group 2 communicates with groups 0 and 1. */ MPI_Intercomm_name_start (myComm, "Connect 02", &inter_request1); ^^ MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); ^^ } By the way, whoever updated all these examples to the current draft, thanks! Bye for now... Tom *************** Env Management (inquiry.tex): *************** 1. In Bill and Rusty's hands at the moment --- >From hart@nipmuc.fsl.noaa.gov Fri Oct 22 12:37 PDT 1993 Received: from gw1.fsl.noaa.gov by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA11487; Fri, 22 Oct 93 12:37:45 -0700 Return-Path: Received: by gw1.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA14955; Fri, 22 Oct 93 19:36:43 GMT Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA23637; Fri, 22 Oct 93 13:43:35 MDT Message-Id: <9310221943.AA23637@nipmuc.fsl.noaa.gov> To: otto@merckx.cse.ogi.edu Cc: hart@nipmuc.fsl.noaa.gov, hender@nipmuc.fsl.noaa.gov Subject: MPI Environmental Management (Ch 7) Date: Fri, 22 Oct 1993 13:43:35 -0600 >From: Leslie Hart (NOAA/FSL) Status: RO Steve, On page 152 (lines 45-48) there seems to be a typo. I believe that the profile for the Fortran binding of MPI_GET_PROCESSOR_NAME should be: MPI_GET_PROCESSOR_NAME (NAME, IERROR) CHARACTER*(*) NAME INTEGER IERROR (Length of a character string in Fortran is passed implicitly). Regards, Leslie Hart (hart@fsl.noaa.gov) >From hart@nipmuc.fsl.noaa.gov Fri Oct 22 12:42 PDT 1993 Received: from gw1.fsl.noaa.gov by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA11494; Fri, 22 Oct 93 12:42:17 -0700 Return-Path: Received: by gw1.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA15014; Fri, 22 Oct 93 19:41:15 GMT Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA23704; Fri, 22 Oct 93 13:48:07 MDT Message-Id: <9310221948.AA23704@nipmuc.fsl.noaa.gov> To: otto@merckx.cse.ogi.edu Cc: hart@nipmuc.fsl.noaa.gov, hender@nipmuc.fsl.noaa.gov Subject: MPI Environmental Management (more) (Ch 7) Date: Fri, 22 Oct 1993 13:48:06 -0600 >From: Leslie Hart (NOAA/FSL) Status: RO Steve, On pages 156 (line 39) to 157 (line 1) I noticed that MPI_USER_SPECIFIES_BUFFER takes three arguments: communicatior, buffer, size. In my notes I have a vote (8-4-7) for the function with four arguments: communicator, byte count of buffer, number of blocking sends outstanding and a success/failure flag. I recall a desire to have this function to be similar to the suggest function. I also recall a problem with supplying a buffer since you may not know when it is actually available (a free communicator may not complete immendiately). Do you have any recollections of this? Regards, Leslie Hart (hart@fsl.noaa.gov) >From lyndon@epcc.ed.ac.uk Tue Oct 26 05:34 PDT 1993 Received: from daedalus.epcc.ed.ac.uk by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA06480; Tue, 26 Oct 93 05:34:46 -0700 Return-Path: Date: Tue, 26 Oct 93 12:36:07 GMT Message-Id: <7710.9310261236@subnode.epcc.ed.ac.uk> >From: L J Clarke Subject: Comments on MPI Draft To: Steve Otto Reply-To: lyndon@epcc.ed.ac.uk Status: RO Hi Steve The draft is impressive, to say the least. Obviously you guys have been busy! I hope the observations below, typos and such, are helpful. Of course I will undoubtedly have missed much. Best Wishes Lyndon o--------------------o 1. Page 29 line 28/29. Text is "... may fail because because ..." Word repitition. Replace with "... may fail because ..." done 2. Page 37 line 31. Text is "... calls, including a mix sends and receives ..." Missing word. Replace with "... calls, including a mix of sends and receives ..." done 3. Page 43 lines 17-24 and possible global. The MPI function MPI_COMMOBJ_FREE has a C binding MPI_Commreq_free and a Fortran binding MPI_COMMOBJ_FREE, which is inconsistent. The C object type is MPI_Comm_request. Is there a mix-up here? names are now different and consistent. 4. Page 74 line 16. The BUFFER argument to the Fortran binding of MPI_BCAST is written as INTEGER whereas it should be . Replace "INTEGER BUFFER, COUNT, DATATYPE, ..." with "INTEGER COUNT, DATATYPE, ... BUFFER(*)" right. done. 5. Page 75 line 35. Text is "... the displacement (relative to ...". Extraneous left bracket. Replace with "... the displacement relative to ...". done 6. Page 96 lines 29-42. The example declares C function "void myProd(Complex invec[], Complex inoutvec[], int *len)" However ANSI C does not require a pointer to void to be coerced into a pointer to type (other than void) - it only coerces from pointer to type to pointer to void - which would be required if the user function in MPI_USER_REDUCE has prototype "void func(void *, void *, int *)" and it should have this prototype since user reduce has the corresponding pair of "void *" arguments. This is really a detailed language binding issue. Suggestion is that (a) prototype for user function is given in the draft; (b) example is changed to accept "void *" rather than "Complex []", and explicit cast from "void *" to "Complex *" required in the example. I'm not going to change it right now since I'm afraid there may be even more issues here. What I have in mind is this: If we just declare everything as void *, then MPI has no idea how to do the "chunking" of the data. Eg, in the example, we have to make sure we are handed complete complexes inside of myProd, not just half of one. I tried to represent this in the example by saying: /* explain to MPI what a "complex" is (remember, this is C!) * Note we are assuming compiler implements type Complex as * two contiguous doubles */ MPI_Type_contiguous( 2, MPI_DOUBLE, &ctype ); MPI_Type_commit( ctype ); MPI_User_reduce( a, answer, 100, ctype, myProd, root, comm ); and we probably need to say that USER_REDUCE hands complete copies of sendtype to the user-function. 7. Page 120 line 10. Text is "... execute MPI_INTERCOMM_start()." Inconsistent case. Replace with "... execute MPI_INTERCOMM_START()." defer -- Tony has chapter 8. Page 120 line 15. Text is "... execute MPI_INTERCOMM_finish()." Inconsistent case. Replace with "... execute MPI_INTERCOMM_FINISH()." defer 9. Page 120 line 29/30. Text is "... creation operation, This routine ..." Punctuation. Replace with "... creation operation. This routine ..." defer 10. Page 120 line 40 Text is "MPI_INTERCOMM_MERGE(intercomm, key, newintracomm". Missing right bracket. Replace with "MPI_INTERCOMM_MERGE(intercomm, key, newintracomm)". defer 11. Page 121 line 27. Text is "INTEGER COMM, NAME, NEW_COMM, ...". The NAME argument of the Fortran binding to MPI_INTERCOMM_NAME_START is declared as INTEGER whereas it should be a character string. Replace with "INTEGER COMM, NEW_COMM, ... CHARACTER *(*) NAME" although note that *(*) is not standard and raises a language binding query. defer 12. Page 123 line 18/19 Text is "... MPI_ATTR_get_key( ..." Inconsistent case. Replace with "... MPI_ATTR_GET_KEY( ..." defer 13. Page 151 line 47. Text is "INTEGER LEN, NAME, IERROR". The NAME argument of the Fortran binding to MPI_GET_PROCESSOR_NAME is declared as INTEGER whereas is looks like it should be a character string. Replace with "INTEGER LEN, IERROR ... CHARACTER *(*) NAME" although note that *(*) is not standard and raises a language binding query (cf 11. above). defer -- Bill Gropp has chapter 14. Page 159 line 32/33. Text is "int MPI_INIT(int argc, char **argv)". Case inconsistent with C binding guidelines. Function name should be "MPI_Init". Looks like also a problem with the arguments. Should these not be "int *argc, char ***argv". I ask this because (a) typically these get passed in because the system has some positions at the beginning or at the end booked for special use, and wants to read them then strip them out (b) this is inconsistent with the arguments in the context chapter examples which seem to assume arguments of "int *argc, char **argv". defer 15. Page 159 line 42/43. Text is "MPI_init( argc, argv );" Case inconsistent with C binding guidelines. Function name should be "MPI_Init". Also above comment on arguments. defer o--------------------o /--------------------------------------------------------\ e||) | Lyndon J Clarke Edinburgh Parallel Computing Centre | e||) c||c | Tel: 031 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | c||c \--------------------------------------------------------/ >From jim@meiko.co.uk Wed Oct 27 05:07 PDT 1993 Received: from hub.meiko.co.uk by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA00456; Wed, 27 Oct 93 05:07:06 -0700 Return-Path: Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA14272 (5.65c/IDA-1.4.4 for otto@merckx.cse.ogi.edu); Wed, 27 Oct 1993 12:08:08 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA04766; Wed, 27 Oct 1993 13:07:57 +0000 Date: Wed, 27 Oct 1993 13:07:57 +0000 >From: jim@meiko.co.uk (James Cownie) Message-Id: <9310271207.AA04766@tycho.co.uk> To: otto@merckx.cse.ogi.edu Cc: mpi-core@cs.utk.edu In-Reply-To: <9310211852.AA25274@merckx.cse.ogi.edu> (message from Steve Otto on Thu, 21 Oct 93 11:52:38 -0700) Subject: Re: MPI DRAFT Content-Length: 6214 Status: RO Steve, Thanks to you and the rest of the editorial group. (Especially whoever did the scatter/gather derived datatype pictures !). Me. Significant issue ================= Collective (Reduction operations) We have only a weak specification of the semantic requirement for these. Do we want reproducibility ? (I can't remember what we concluded. Who can ?) reproducibility/repeatability is now mentioned in the terms chapter. In terms: "However, repeatability is a desirable quality-of-implementation goal. In particular, it is expected that MPI implementations will avoid introducing additional non-determinism. For example, while MPI can take advantage of associativity and commutativity in reordering the execution of a global reduce operation, it is desired (but not required) that the same operations order be used in repeated executions." In particular for the ALL REDUCE operations we don't currently say that everyone gets the SAME result. I think we should (at least) say this, even if we want to be relaxed about the reproducibility of that result. good point. done. Global edits ============ It would be nice to change "parameter" to "argument" throughout. (Possibly with a done, except for context/inquiry which are "checked out" at the moment. minor discussion in the terms and conventions section, something like ... Since the word PARAMETER is a keyword in the Fortran language, we use the word argument to denote the arguments to a subroutine. These are normally referred to as parameters in C, however we expect that C programmers will understand the word arguments (which has no specific meaning in C), thus allowing us to avoid unnecessary confusion for Fortran programmers. ) done, placed in terms.tex. Typos ===== I've included here some changes to the Fortran binding for routines which need to return/take character strings. I believe that these should take CHARACTER*(*) arguments, in accordance with standard Fortran practice. The current draft has these as INTEGER and also passes (a now unnecessary) length. Here are a set of typos in Oct 19 draft... Page Line Change Anonymous (Acknowledgements) Line 21 I think it is Mark Sears, (not Marc) done 9 25 delete the : done 28 "allocated on user storage or on system" change both "on"s to "in"s done 10 3 Add "an" at beginning of the line done 8 "is the user resposibility" => "is the user's responsibility" done 20 "begginning" => "beginning" done 20 "indecates" => "indicates" done 12 19 "spedify" => "sepcify" done 39 "processor" => "process" done 44 "implicit global states" => "implicit state" done 47 "or other events" => "or with other events" done 13 11 "to always" => "always to" done 12 "consecutive integers" => "consecutive integers in the range 0 .. groupsize-1" done 14 5 ":" => "." 6 "however" => "However" done 15-18 Was this what we said ? I thought we were much stronger than this. (see Significant issues above). ???Hmmm... 43 "As an" => "An" done 15 19 "SIGFPE,and" => "SIGFPE or" done 16 21 "manaipulation" => "manipulation" done 22 23 delete "and is seen as a quality of implementation issue" (Or Moose will be very unhappy !) done 30 "MPI implementation" => "An MPI implementation" done 26 11 "types as" => "types from" done 27 10 "part of" => "some of" done 20 "either ends" => "either end" done 29 29 "Such call" => "Such a call" done 31 40 "associates" => "associate" done 41 delete "about" done 43 "buf copying" => "copy" done 33 7 "In multi-threaded" => In a multi-threaded" done 34 2 "either receives" => "either receive" done 39 4 "user" => "user's" done 42 39 "semantics as" => "semantics to" done 43 48 "correctly the sends and receives" => "the sends and receives correctly" done 46 27 "allows" => "allow" done 36 "have used" => "have been used" done 58 15 "Each of the sends can match each of the receives" => "From a type matching viewpoint any of the sends legally matches any of the receives" done 16 "Overlaping" => "Overlapping" done 20 "Or one of them have" => "Or one of them has" done 22 "overlaping" => "overlapping" done 68 27 "This section is not part of the MPI definition" should be made MUCH more significant (Bold, or margin or something) done (bold) 83 48 "exectuted" => "executed" done 88 45 Lots of (unnecessary) blank space Not if you're LaTeX! 89 40 Ditto ditto ************tony has... 106 28 "allows to invoke" => "allows the invocation of" 43 "users that" => "users who" 44 "and ignore the rest of this chapter" should be in bold or CAPS 107 9 "users that" => "users who" 108 26 "The groups corresponding to this communicator do not appear as pre-defined constants," => "The group embedded in this communicator does not appear as a pre-defined constant," 115 1 "with all its cahed information". This is WRONG. It should say something like "For each piece of cahed information on the original communicator the relevant copy_fn is called. This can either copy the cached information, or not, as is appropriate." 122 6 "in size or" => "in size to, or" 123 40 "put into this attribute set" => "attached to this communicator" 124 13 "are invoked." => "are invoked in an arbitrary order." 128 42 "if ((me % 2) == 0)" => "if ((me % 2) == 0 && (me + 1) < size))" So don't send if I don't have a neighbour. Should also remove all the modulusing in the sends and receives. So 43 "(...,((me+1)%size)," => "...,me+1," 45 "((me-1+size)%size)" => "me-1" *************** Bill Gropp has 152 47,48 Should read MPI_GET_PROCESSOR_NAME(NAME,IERROR) CHARACTER*(*) NAME INTEGER IERROR 154 41 "no to" => "not to" 156 20 "an algorithm" => "a program" 157 31 "can not" => "cannot" 43 "Such handler" => "Such a handler" 48 "MPI_WORLD" => "MPI_COMM_WORLD" 158 1 "with a communicator;" => "to a communicator." 159 16,17 Should read MPI_ERROR_STRING(ERRORCODE, STRING, IERROR) INTEGER ERRORCODE, IERROR CHARACTER *(*) STRING 40 "before any other MPI routines are called" => "before any other MPI routine (apart from MPI_INITIALIZED) is called" ****************** -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com >From doss@ERC.MsState.Edu Wed Oct 27 08:09 PDT 1993 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA00647; Wed, 27 Oct 93 08:08:56 -0700 Return-Path: Received: by ogicse.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.4) id ; Wed, 27 Oct 93 08:10 PDT Received: from Athena.ERC.MsState.Edu by Phoenix.ERC.MsState.Edu (4.1/6.0s-FWP); id AA24780; Wed, 27 Oct 93 10:14:32 CDT >From: Nathan E. Doss Received: by Athena.ERC.MsState.Edu (4.1/6.0c-FWP); id AA16589; Wed, 27 Oct 93 10:14:05 CDT Message-Id: <9310271514.AA16589@Athena.ERC.MsState.Edu> Subject: Minor inconsistencies in collective chapter To: otto@cse.ogi.edu Date: Wed, 27 Oct 1993 10:14:04 -0500 (CDT) Cc: tony@cs X-Mailer: ELM [version 2.4 PL17] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1177 Status: RO Dear Steve, Below is a list of minor inconsistencies that I've found in the Collective Communication chapter in the Oct. 19, 1993 version of the MPI draft. Tony Skjellum suggested that I send this list to you. - In section 4.5.4 Scan, all arguments to the MPI_SCAN function are listed as "IN" arguments. "recvbuf" should be "OUT". done pg 100, line 25 - For most (all?) of the Fortran bindings, IERROR is listed as an argument, but is not included in the argument list. done - For many of the C bindings, the argument list should include "MPI_Datatype recvtype", but "recvtype" has been left out. yep. done. - The argument list for the C binding of MPI_GATHER should contain "sendcount" instead of "sendcnt" pg 74, line 41 done - The last argument in the generic description of MPI_SCATTER and MPI_SCATTERV should be "comm" instead of "group". pg 83, line 36 pg 84, line 35 done - There are a few places where "recvcount[i]" should be "recvcounts[i]" and where "sendcount[i]" should be "sendcounts[i]". pg 76, line 13 pg 85, line 5 pg 91, line 45 done -- Nathan Doss doss@ERC.MsState.Edu NOTE: I have not gotten to Steven H-L's comments on topology that appear below. --Steve Otto >From lederman@super.org Wed Oct 27 06:51 PDT 1993 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA00566; Wed, 27 Oct 93 06:51:49 -0700 Return-Path: Received: by ogicse.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.4) id ; Wed, 27 Oct 93 06:53 PDT Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA12950; Wed, 27 Oct 93 09:53:19 EDT Received: by b125.super.org (4.1/SMI-4.1) id AA01775; Wed, 27 Oct 93 09:53:18 EDT Date: Wed, 27 Oct 93 09:53:18 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9310271353.AA01775@b125.super.org> To: hempel@gmd.de Subject: topology chapter comments Cc: otto@cse.ogi.edu Status: RO Rolf, Last night I babysat for some friends. After they went to bed I finally had the peace and quiet to read your chapter. The following is the comments I would like to make. I know a number of changes were made to your chapter during the editing session and I suspect a number of these thing occurred then. I am putting them in order of the pages so editing might be easier. I give the page/line number of the draft I got from ogi last week. I have not had time to carefully check everything so if some of the comments do not make sense either ignore it or ask me for clarification. You may have already caught these changes. Sorry I did not get to this sooner but life is always busy. I know there are a lot and some are small but since this is the last draft I wanted to pass them all along. Steve ---------------------------------------------------------------------- - p.140/44: "processes is be represented" should be "processes can be represented" - p. 142/7: "processes along a given" I prefer: "processes among a given" - p. 142/22+23: "outlines such implementation" should be "outlines such an implementation" - p. 142/34-37+: This is a general comment. You refer to the logical type. In chapter 2 is a definition of the choice type, etc but not the logical type. In FORTRAN I know what this is but not in C. Maybe MPI should make this an integer with the standard 0/1 definition as C does. - p. 142/46: "than the the group" should be "than the group" - p. 143/25,26,28,29: MPI_MAKE_DIM should be MPI_MAKE_DIMS - p. 144/1: "are not member of the grid" should be "are not members of the grid" - p. 144/2: "larger than then the group" should be "larger than the group" - p. 144/25: "i, i=1,..." should be "i,i=0,..." since this is 0 up numbering in C. - p. 144/29: "i, i=1,....,nnodes-1" should be "i, i=1,....,nnodes" got the 1 start for FORTRAN but left the -1 accidentially - p. 144/30: "1<=j<=index[0]" should be "1<=j<=index[1]" again forgot the 1 up numbering in FORTRAN - p. 144/31: "index[i-1]+1 <= j <= index[i]" should be "index[i]+1 <= j <= index[i+1]" same one up numbering problem - p. 145/27 + p. 146/1: there is MPI_GET_GRAPHDIMS and MPI_GET_CARTDIM. Either have the S or not. Be consistent with DIM or DIMS. You already used MPI_MAKE_DIMS so maybe it should have the S. - p. 145/44: "index, edges vectors containing" should be "index, edges array of integers containing" - p. 146/9: "MPI_GET_CART(....,period,...)" should be "MPI_GET_CART(....,periods,...)" since you use periods (not period) most other places and is what the input argument is defined as. - p. 147/1+8: At my request you changed MPI_GRAPH_NNEIGHBORS to MPI_GRAPH_NEIGHBOR_COUNT. The only concern I have now is that MPI_GRAPH_NEIGHBORS has an S but the new one doesn't. I don't care if NEIGHBOR or NEIGHBORS but it need to be consistent. - p. 147/2+9: the comm argument needs to be identified as a (handle). - p. 147/12: maxneighbors needs to be identified as a (integer). - p. 147/22: I would add a row right below the one with "exchange shuffle unshuffle" in the table which gives "neighbor(1) neighbor(2) neighbor(3)" so it is clearer which values in the example correspond to what in the code. - p. 149/24: "MPI_MAP_CART(...,period,..)" should be "MPI_MAP_CART(...,periods,..)". - p. 149/43+150/17: I don't understand how using color=MPI_UNDEFINED is safe. How do you know that this constant is not 0 in some implementation. I may have missed something here. - p. 149/45-48: I don't think this definition works when MPI_SUBCART is called with multiple values in array remain_dims = false. It seems to me that you need multiple number (not a single number) encoding to deal with this case. - p. 150/34: This is a picky point but would you use MPI_SENDRECV instead of MPI_SEND (with an MPI_RECV) to do an exchange. - p. 151/17: "MPI_MAKE_CART(...,rank,...)" should have no rank argument. - p. 151/18: "MPI_GAT_CART" should be "MPI_GET_CART" That's all folks .... :-) From owner-mpi-core@CS.UTK.EDU Wed Oct 27 21:38:52 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA09632; Wed, 27 Oct 93 21:38:52 -0400 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA11786; Wed, 27 Oct 93 21:37:59 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 27 Oct 1993 21:37:57 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from THUD.CS.UTK.EDU by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA11777; Wed, 27 Oct 93 21:37:56 -0400 From: Jack Dongarra Received: by thud.cs.utk.edu (5.61+IDA+UTK-930922/2.7c-UTK) id AA16905; Wed, 27 Oct 93 21:38:04 -0400 Date: Wed, 27 Oct 93 21:38:04 -0400 Message-Id: <9310280138.AA16905@thud.cs.utk.edu> To: mpi-core@cs.utk.edu Subject: new draft The new draft of MPI is out in netlib, it is under: draft1027.ps and draft1027-tar.Z in the mpi directory. You can retrieve an up-to-date copy of the postscript from any machine on the internet by typing: rcp anon@netlib2.cs.utk.edu:mpi/draft1027.ps draft1027.ps or the compressed tar file by typing: rcp anon@netlib2.cs.utk.edu:mpi/draft1027-tar.Z draft1027-tar.Z Regards, Jack From owner-mpi-core@CS.UTK.EDU Tue Nov 2 04:48:47 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA11741; Tue, 2 Nov 93 04:48:47 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA19769; Tue, 2 Nov 93 04:47:40 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 2 Nov 1993 04:47:39 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA19761; Tue, 2 Nov 93 04:47:37 -0500 Received: by merckx.cse.ogi.edu (16.8/16.2) id AA25960; Tue, 2 Nov 93 01:46:20 -0800 Date: Tue, 2 Nov 93 01:46:20 -0800 From: Steve Otto Message-Id: <9311020946.AA25960@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: the draft... Hello All, The draft for SuperComputing 93 will be available later today. Sorry that it's ending up a day late, but Marc Snir kept me very busy! ;^) (and so did a few others) Rusty is (hopefully) going to re-build the Language binding Annex this (Tuesday) morning, then I will LaTeX one more time and send the ftp info out to the persons responsible for printing. Whew! Someone owes me a beer, and as for Marc, I estimate that someone owes him a six-pack! --Steve From owner-mpi-core@CS.UTK.EDU Tue Nov 2 09:27:51 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA12332; Tue, 2 Nov 93 09:27:51 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA12033; Tue, 2 Nov 93 09:27:04 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 2 Nov 1993 09:27:03 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA12021; Tue, 2 Nov 93 09:26:57 -0500 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA00625 (5.65c8/IDA-1.4.4 for ); Tue, 2 Nov 1993 15:25:42 +0100 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA16196; Tue, 2 Nov 1993 15:27:15 GMT Date: Tue, 2 Nov 1993 15:27:15 GMT From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9311021527.AA16196@f1neuman.gmd.de> To: mpi-core@cs.utk.edu Subject: Deadline for feedback Cc: gmap10@f1neuman.gmd.de In the provisional time table for future MPI activities Jan. 15th has been chosen as the deadline for user feedback. In the meantime I have talked to many people here in Europe, and the general feeling was that this is much too early. At the moment many people know hardly more about MPI than its name. Of course this will change after SC'93, but it will take some time until potential users understand the subtleties of MPI fully enough to know where they run into trouble with their applications. This problem will be aggravated by the fact that the days around Christmas are a favourite holiday time (I guess not only in Europe). Don't you think we should give people more time? I would suggest end of February as a more realistic deadline. We don't improve the quality of the standard if we push too hard. An extended feedback period in particular would give us the chance to organize things like user workshops. Some people over here would be interested to do such a thing on an European level, but it could not be before Jan. 18th. If we cannot decide in this matter on email, I think we should discuss it at SC'93. - Rolf Hempel From owner-mpi-core@CS.UTK.EDU Tue Nov 2 14:54:38 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA15272; Tue, 2 Nov 93 14:54:38 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA06030; Tue, 2 Nov 93 14:53:29 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 2 Nov 1993 14:53:26 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA06020; Tue, 2 Nov 93 14:53:15 -0500 Received: by merckx.cse.ogi.edu (16.8/16.2) id AA28441; Tue, 2 Nov 93 11:51:57 -0800 Date: Tue, 2 Nov 93 11:51:57 -0800 From: Steve Otto Message-Id: <9311021951.AA28441@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: MPI Draft Actions List Hello All, Appended below is a crude attempt to document what has happened with the MPI draft over the last week. You will see comments, followed by an explanation of what I have (or have not yet) done about it in the new version. One use of the below is that if you sent in some comments, and you like to find out what I did about it. Search for your comment. Some comments re: context, inquiry, topology, chapters were handled by the chapter authors. I have no "actions" documentation for these (though your comments do appear below), but they assured me that they took every comment into account! When studying Marc Snir's comments, keep in mind that his page numbers refer to the draft of October 19. If I mislaid your comments somehow, I am sorry. I did handle some things on the fly without properly documenting them. As for those "Overfulls" that have crept back into the document, they occurred because the text was changed. I have not had a chance to prrof-read this thing myself, since I've been so busy processing your comments! We will be able to pretty-up the doc over the next few weeks, but it didn't seem too bad as I rapidly scanned it this morning. --Steve Actions List for MPI Comments received. Steve Otto By the way, "cacheing" is spelled as "caching"! ************** Intro.tex ************** Steve, Just a few small points about the draft: p. 1/20: I would change "vendor has implemented his own variant" -> "vendor has implemented its own variant" p. 74/13: rectype -> recvtype Steve done *************** Point to Point: *************** >From snir@watson.ibm.com Mon Oct 25 21:23 PDT 1993 Received: from iliamna.cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA05917; Mon, 25 Oct 93 21:23:53 -0700 Return-Path: Received: by iliamna.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.17) id ; Mon, 25 Oct 93 21:25 PDT Message-Id: Received: from YKTVMV by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 5425; Tue, 26 Oct 93 00:25:28 EDT Date: Tue, 26 Oct 93 00:25:09 EDT >From: "Marc Snir" X-Addr: (914) 945-3204 (862-3204) 28-226 IBM T.J. Watson Research Center P.O. Box 218 Yorktown Heights NY 10598 To: otto@iliamna.cse.ogi.edu Subject: Note from PCTERM3 Reply-To: SNIR@watson.ibm.com Status: RO ------------------------------- Referenced Note --------------------------- First batch of corrections I use px,ly for page x, line y Explanations for changes I propose are in square brackets [] Global changes are followed by a (probably incomplete) list of places where they occur. ******************************** GLOBAL CHANGES FLAG arguments have type LOGICAL in Fortran replace INTEGER FLAG by LOGICAL FLAG p32,l44 p36,l1 p38,l40 done status argument has type (Status) add (Status) at end of description of status parameter p19,l26 p32,l20 p32,l40 p35,l17 p35,l43 p37,l35 ?? p38,l12 ?? p44,l33 p45,l14 done Datatype argument has type (handle) add (handle) after description of {\tt datatype} argument. or {\tt sendtype}, {\tt recvtype} p17,l6 p44,l15 p21,l5 p44,l25 p27,l33 p28,l5 p30,l10 p30,l32 p31,l5 p31,l25 p40,l37 p41,l10 p41,l32 p42,l5 p45,l6 done. AND ALSO: p19,l21 communication object (handle) -> communication request (handle) handle to communication object -> communication request (handle) p30,l16 p32,l18 p30,l39 p32,l37 p31,l11 p31,l31 p40,l42 p41,l17 p41,l39 p42,l22 p43,l19 done. array of handles to communication objects -> array of requests array of communication objects -> array of requests p35,l14 p42,l44 p35,l38 p36,l16 p36,l36 done. I put in: "array of requests (array of handle)" ************************************ p6,l17 No support -> Support done. p6,l36 recv -> receive done p7,l18 Fortran -> Fortran 77 done p9,l11 Such an operation -> Usually, such an operation [Cancel is a local operation, but may require communication] *NOT* done: unsettling to the reader...does CANCEL require communication with the user-level process? Perhaps communications need to occur, but it is hidden from the user and so may as well be considered local, no? p9,l42 as as -> as done p10,l6 MPI_GROUP_FREE -> free the object done p10, l15 Array of handles -> Array arguments done [they are not always handles] p10,l16 that is an array-of-handles. The ... -> that is an array of opaque objects, or an array of handles [no italics]. An ... done p10,l21 [Add at end of paragraph] -> The same approach is followed for other array arguments. done p12,l12 The same MPI subroutine -> An MPI subroutine with a choice argument done p12,l13 Figure 2.4.2 -> Figure 2.1 done p12,l19 spedify -> specify done p12,l30 Buffer pointers are -> Choice arguments are pointers done p13,l32 The recommended -> A high quality done p13,l35 an error value -> a code done p16,l21 manaipulation -> manipulation done p17,l38 delete MPI_DOUBLE_COMPLEX line done p18,l23 ... C; MPI_REAL2 ... -> ... C; \const{MPI\_DOUBLE\_COMPLEX} for double precision complex in Fortran; MPI_REAL2 ... [DOUBLE PRECISION COMPLEX is not standard Fortran 77] done p18,l45 TagRange -> \mpifunc{MPI\_GET\_VALID\_TAG\_RANGE} done p18,l20 (state) -> (handle) done -- actually p19. p21,l48 [add at end of paragraph] -> Note that the receiver is not required to receive messages in the order they were sent, if they have distinct envelopes. done, but I'm unhappy about the placement here...it puts this statement in the same paragraph with threads, while this is not necessarily a threads issue.? p22,l30 provide information -> provide information and control done p27,l4 posted; Otherwise -> posted; otherwise done p29,l39 communication objects -> communication request objects [communication objects are request objects] done p29,38 An opaque communication object -> A request object done p30,l3 nonblocking communication -> nonblocking send. done p30,l20 *request -> request [The call modifes the opaque object pointed at by request, but does not modify the handle] done p30,25 nonblocking communication -> nonblocking send. done p30,l43 *request -> request done p30,l48 nonblocking communication -> nonblocking send. done p31,l11 *request -> request done p31,l31 *request -> request done p31,40 communication object and associates it with the request. -> communication request and associates it with the handle. done p31,l43 buf -> start done p32,l5 [delete] or any other location that can be referenced by the send operation [there are none] done p32,l5 [delete] or any other location that can be referenced by the receive operation done p32,l17 IN request ... -> {\INOUT}{request}{communication request (handle)} done p32,l22 request -> *request [WAIT nullifies the handle] done p32,l37 IN request ... -> {\INOUT}{request}{communication request (handle)} done p32,l38 logical -> (logical) done p32,l41 request -> *request done p33,l26 they will be also used to complete any other nonblocking communication call -> they are also used to complete all other nonblocking communication calls. done p35,l26 Returns in ... with the status. -> Returns in \mpiarg{index} the index of that request in the array and returns in \mpiarg{status} the status of the completing communication. done p35,l30 is is -> is done p35,l39 index of handle for operation -> index of operation done p35,l42 true ... -> \const{true} if one of the operations is complete (logical) done p36,l16 IN -> INOUT done p37,l31 mix sends -> mix of sends done p37,l33 and the array_of_statuses -> and the entries of the array_of_statuses done p38,l7 first message -> first arriving message done p38,l9 second message -> second arriving message done p38,l20 [add new paragraph] An MPI implementation of \mpifunc{MPI\_PROBE} need to guarantee progress: if a call to \mpifunc{MPI\_PROBE} has been issued by a process, and a send that matches the probe has been initated by some process, then the call to \mpifunc{MPI\_PROBE} will return, unless the message is received by another concurrent receive operation (that is executed by another thread at the probing process). Similarly, if a process busy waits with \mpifunc{MPI\_IPROBE} and a matching message has been issued, then the call to \mpifunc{MPI\_IPROBE} will eventually return {\tt flag = true} unless the message is received by another concurrent receive operation. done p40,l15 add implementation note \implement{ If a send operation uses an ``eager'' protocol (data is transfered to the receiver before a matching receive is posted), then the cancelation of this send may require communication with the intended receiver, in order to free allocated buffers. On some systems this may require to interupt the intended receiver. Therefore, it is expected that \mpifunc{MPI\_CANCEL} will be an expensive operation on many systems, to be used only exceptionally. Note that, while communication may be needed to implement \mpifunc{MPI\_CANCEL}, this is still a local operation, in the sense that its completion does not depend on the code executed by other processes. If processing is required on another process, this should be transparent to the application (hence the need for an interupt and an interupt handler). } done p40,l20 to the communication object -> to a communication request done p40,l23 The communication object -> The request done p40,l28 A communication object -> A request done p40,l30 a communication object -> a request done p40,l45 *request -> request done p40,l48 INTEGER BUF -> BUF(*) done p41,l1 The call ... -> [delete sentence] done p41,l21 *request -> request done p41,l23 INTEGER BUF -> BUF(*) done p41,l42 *request -> request done p41,l45 INTEGER BUF -> BUF(*) done p42,l13 *request -> request done p42,l15 INTEGER BUF -> BUF(*) done p42,l17 communication object -> request done p42,l33 updated -> accessed done p42,l44 requestss -> requests done p43,l8 communication object -> request done p43,l10 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l12 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l17 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l21 MPI_Commreq_FREE -> MPI_Request_free (or MPI_Comm_request_free) done p43,l22 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l27 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done [to be consistent with current terminology -- change need to be propagated to appendices] yes -- done. p42,l34 communication object -> request done p44,l46 the same tag ... [end of paragraph] -> the same communicator, but possibly different tags. The send buffer and receive buffers are disjoint, and may have different lengths and datatypes. [lengths] done! *************** Collective chapter: *************** 1. MPI_SEND --> MPI_Send in several places 2. Comments from David Walker: -------------------------------------------------------------------------- Steve and Al, I think the removal of the MPI_USER_REDUCE_SCATTER and MPI_USER_REDUCE_SCATTERA is illogical, since we still have the corrsponding routines for REDUCE and SCAN. If you want to get rid of routines here are 2 possibilities: 1) Remove all REDUCE_SCATTER routines. 2) Remove all noncommutative REDUCE and SCAN routines (the ones ending in "A") At this late stage I think it best to leave things as they were at the end of the last MPI meeting, i.e. keep MPI_USER_REDUCE_SCATTER and MPI_USER_REDUCE_SCATTERA. David -------------------------------------------------------------------------- Action taken: I have followed David's last remark, and: I think it best to leave things as they were at the end of the last MPI meeting, i.e. keep MPI_USER_REDUCE_SCATTER and MPI_USER_REDUCE_SCATTERA. done. NOTE: these now need language binding. 3. Fixed indent in "Example of Reduce" 4. Comments from Steven Huss-Lederman: - I can see how it happened, but it seems a little funny that MPI_ALLREDUCE is in section 4.5.2 "User-Reduce". It isn't a user reduce function just the result know to all. I've given them they're own section: \subsection{All-Reduce} \label{subsec:coll-all-reduce} - I just want to make sure that the MPI_USER_REDUCE_SCATTER(A) functions were actually removed from the chapter. they're back! *************** Topologies: *************** 1. Fixed Overfull on page 150 2. New language bindings from Rusty. I will try and integrate these with Rolf's new chapter, see below. done 3. Comments from Rolf Hempel: -------------------------------------------------------------------------- ... In general, I appreciate the work you have put into editing the MPI document, to improve its consistency. In particular, Marc has done a number of improvements to the topology chapter. I never had strong feelings about function names, and the new names are fine with me. Also, the added examples make the details easier to understand for the reader. It's difficult to decide whether the original or the new definitions for vectors "index" and "edges" are better. The new definitions save the additional entry in vector index, but they introduce the special case for accessing the first element. This problem is somehow alleviated by the introduction of the two new lookup functions for the graph neighbors, so that the user code does not have to deal with the special case. So, I have no strong objections against the change. The only thing I really don't like is the removal of the low-level mapping functions from the standard. The analogy with the low-level functions in PT2PT is only partially correct. There the functions mainly serve for the semantic definition of the higher-level functions, and good MPI implementations are expected to bypass them. In the case of the topology construction functions the situation is different. Here the layered implementation will be the general choice, and at our last meeting I even had the impression that the vendors are waiting for me to provide the (portable) high-level layer. This only works if every vendor provides the low-level functions for their system, so that the high-level functions can use them. In this sense, a better analogy for the low-level functions is the cacheing facility which (for good reason) is also part of the standard. I think this issue goes a bit beyond pure editing of the final document. After all, we have had several readings of the topology chapter, and the low-level functions passed them all. I don't want to insist on formal rules, but what was the purpose of all those readings, if we can now ignore them? As always happens, I found a number of mis-spellings and other inconsistencies in the current topology chapter version. How can I do the necessary changes without interfering with others? Do you control the update right for individual chapters? Does Marc still work on the topology chapter? Rolf -------------------------------------------------------------------------- Well put, Rolf! I will incorporate Rolf's version of topol.tex, which he has just sent me. And Rolf has sent a new version: -------------------------------------------------------------------------- ... Below you will find the new LaTex source and two entries in the bibliography which I changed slightly. Marc told me that he has no strong feeling about the low-level functions (he removed them from the standard), but that he would like to present them in the order they are presented now. So, I moved them back into the standard, but kept the order. -------------------------------------------------------------------------- Yes, I think this new order of presentation is much improved from the old version. done. 4. Comment from Steven Huss-Lederman: While doing the subset chapter I noticed the two new routines MPI_GRAPH_NNEIGHBORS and MPI_GRAPH_NEIGHBORS. I personally think it is a bad idea to have two names that are so close. Maybe put in the phrase DIM as in the Cartesian case. I don't really care what, just the similarity will probably cause people to get confused. 5. Another Comment from Steven Huss-Lederman: >From lederman@super.org Wed Oct 27 17:27 PDT 1993 Received: from iliamna.cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA04596; Wed, 27 Oct 93 17:27:44 -0700 Return-Path: Received: by iliamna.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.17) id ; Wed, 27 Oct 93 17:29 PDT Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA14947; Wed, 27 Oct 93 13:17:54 EDT Received: by b125.super.org (4.1/SMI-4.1) id AA01895; Wed, 27 Oct 93 13:17:53 EDT Date: Wed, 27 Oct 93 13:17:53 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9310271717.AA01895@b125.super.org> To: Rolf.Hempel@gmd.de Cc: otto@iliamna.cse.ogi.edu In-Reply-To: <9310271646.AA16464@f1neuman.gmd.de> (Rolf.Hempel@gmd.de) Subject: Re: Topology chapter Status: RO (Steve: please read point 1. because it relates to chapter 2 :-) Rolf, 1. As for logicals, I only read your chapter carefully so forgot about the pt-2-pt reference. I think Steve needs to add a section to 2.3.* to deal with logicals to state what they are. 2. I still have a problem with the index & edge stuff on p. 144. Let me try to explain: The paragraph on FORTRAN (lines 28-31) seems to use 0 up numbering for index even though it start by saying "index[1] is the degree of node zero" which means 1 up numbering. I don't see how node zero can have edges[j], 1<=j<=index[0] I think it has to be index[1] in FORTRAN. This is clearly 1 up numbering because of the 1<=j part. The comment I made before on the other part with node != 0 was a similar problem. I think this whole discussion is complex and obscures some of the basic point: the C program use 0 as the base index and FORTRAN uses 1. In all other regards they are the same: that is the values in index and edges are identical in the two languages. Any clearer? I may well be missing something here. 3. The DIM vs DIMS is ok. How I understand the difference. 4. As for MPI_CART_SUB on p. 149 (line 45) (I just noticed it is MPI_SUBCART but should be MPI_CART_SUB): I misunderstood the intent of this statement. I thought it meant a singe value of color but what it means is a single number that can have multiple values. The stuff I put about multiple falses was a mistake. 5. Yes, MPI_SEND will work fine. I was just pointing out that MPI_SENDRECV is the new "official" way of doing exchanges. The example does not need to be changed. Steve Rolf has addressed these points as follows: % % changes made by Rolf Hempel: (10/25/93) % % * low-level topology functions moved back into standard % % * added a paragraph about out-of-range coordinates on input to % function MPI_CART_RANK. % % * general clean-up and text formatting % % changes made by Rolf Hempel: (10/27/93) % % * change of function name MPI_GRAPH_NNEIGHBORS --> % MPI_GRAPH_NEIGHBORS_COUNT I also changed appLang.tex... (SO) % % * used a spell checker for correcting several typos % % changes made by Rolf Hempel: (10/28/93) % % * change of function name MPI_GRAPH_COORD --> % MPI_GRAPH_COORDS I can't find either of these names in the chapter!! Where are they? --Steve % % * corrected two citations of function names (MPI_SUBPART and % MPI_SUBCART --> MPI_CART_SUB) % % * corrected the explanation of vectors index and edges in the % Fortran case. % % * inserted a label in subsection "Cartesian shift coordinates". % % * some minor text formatting (Overfulls, alignments, ...) % *************** Contexts: *************** 1. In Tony's hands at the moment --- "Here it comes... (all known problems are addressed). - Tony" >From hender@macaw.fsl.noaa.gov Fri Oct 22 09:54 PDT 1993 Received: from gw1.fsl.noaa.gov by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA11305; Fri, 22 Oct 93 09:54:06 -0700 Return-Path: Received: by gw1.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA13627; Fri, 22 Oct 93 16:53:00 GMT Received: by macaw.fsl.noaa.gov (4.1/SMI-4.1) id AA17176; Fri, 22 Oct 93 10:55:31 MDT Date: Fri, 22 Oct 93 10:55:31 MDT >From: hender@macaw.fsl.noaa.gov (Tom Henderson) Message-Id: <9310221655.AA17176@macaw.fsl.noaa.gov> To: mpi-context@cs.utk.edu, otto@merckx.cse.ogi.edu Subject: Comments on Oct. 19 draft Status: RO Hi all, A few comments on the Inter-Communication section (5.5) and the Inter-Communication Examples section (5.8.7) of the October 19 draft... On page 118, line 46, the first sentence reads: "Intercommunicators can be created using the function MPI_COMM_DUP." We had a long discussion following the last meeting about how to describe rules for "safe" use of the inter-communicator construction routines. The result was a decision to drop the blocking versions: MPI_INTERCOMM_MAKE() and MPI_INTERCOMM_NAME(). It seems to me that using MPI_COMM_DUP() to duplicate an inter-communicator has the same synchronization problems as MPI_INTERCOMM_MAKE()-- both groups need to know what's going on. It's just another blocking inter-communicator construction routine. To be consistent, I think we need to delete this sentence or replace "can" with "cannot". On page 119, lines 6-12: "The explicit synchronization can cause deadlock in modular programs with cyclic communication graphs, even if: * within each local group calls are executed in the same order, * the local and remote operations are decoupled and the construction is performed "loosely synchronously" (by calling the two routines MPI_INTERCOMM_START() and MPI_INTERCOMM_FINISH())." The previous draft (9/27) read: "The explicit synchronization can cause deadlock in modular programs with cyclic communication graphs, even if within each local group calls are executed in the same order. So, the local and remote operations can be decoupled and the construction performed "loosely synchronously" by calling the two routines MPI_INTERCOMM_START() and MPI_INTERCOMM_FINISH()." I have two comments about this change: 1) In the current draft, the ONLY WAY to construct an inter-communicator is "loosely synchronously". The second bullet doesn't make sense any more. The local and remote operations are always decoupled. 2) My interpretation of the current draft is that there is no "safe" way to construct an inter-communicator. I think it may also be true that we don't know how to describe sufficient conditions for non-deadlocking for any series of collective operations on groups that overlap. If this is true, then we should make this statement a general warning about using any collective operations and not hide it here. Comments please? I don't have a good idea for fixing this text without knowing what lead to the change. What am I missing? On page 119, line 41 (typo): "MPI _arg" appears, looks like a typo. On page 120, lines 8-9 (typo): "This call starts off an inter-communicator creation operation, returning a handle for the completion of the operation in handle; ..." ^^^^^^ "This call starts off an inter-communicator creation operation, returning a handle for the completion of the operation in inter_request; ..." ^^^^^^^^^^^^^ On page 121, line 28 (clarification): I would like to add the following sentence: "All processes in the local and remote groups must provide the same name." This is not stated anywhere else. It used to be in an implementation note. On page 134, line 26 (typo): "group B" should be "group 1" This one's my fault, left over from several drafts ago... :-) On page 137, lines 24-40 (typos): "if (color == 0) { /* Group 0 communicates with group 1. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); MPI_Intercomm_finish (inter_request1, &myFirstComm); } else if (color == 1) { /* Group 1 communicates with groups 0 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 10", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); MPI_Intercomm_finish (inter_request1, &myFirstComm); MPI_Intercomm_finish (inter_request2, &mySecondComm); } else if (color == 2) { /* Group 2 communicates with group 1. */ MPI_Intercomm_name_start (myComm, "Connect 21", &inter_request1); MPI_Intercomm_finish (inter_request1, &myFirstComm); }" Names must match for all processes in local and remote groups. Fix is: if (color == 0) { /* Group 0 communicates with group 1. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); MPI_Intercomm_finish (inter_request1, &myFirstComm); } else if (color == 1) { /* Group 1 communicates with groups 0 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); ^^ MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); MPI_Intercomm_finish (inter_request1, &myFirstComm); MPI_Intercomm_finish (inter_request2, &mySecondComm); } else if (color == 2) { /* Group 2 communicates with group 1. */ MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request1); ^^ MPI_Intercomm_finish (inter_request1, &myFirstComm); } On page 138, lines 39-48 and page 139, lines 1-5 (typos): "if (color == 0) { /* Group 0 communicates with groups 1 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 02", &inter_request2); } else if (color == 1) { /* Group 1 communicates with groups 0 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 10", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); } else if () { /* Group 2 communicates with groups 0 and 1. */ MPI_Intercomm_name_start (myComm, "Connect 20", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 21", &inter_request2); }" Names must match for all processes in local and remote groups. Also, last elseif is a dinosaur. Fixes are: if (color == 0) { /* Group 0 communicates with groups 1 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); MPI_Intercomm_name_start (myComm, "Connect 02", &inter_request2); } else if (color == 1) { /* Group 1 communicates with groups 0 and 2. */ MPI_Intercomm_name_start (myComm, "Connect 01", &inter_request1); ^^ MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); } else if (color == 2) ^^^^^^^^^^ { /* Group 2 communicates with groups 0 and 1. */ MPI_Intercomm_name_start (myComm, "Connect 02", &inter_request1); ^^ MPI_Intercomm_name_start (myComm, "Connect 12", &inter_request2); ^^ } By the way, whoever updated all these examples to the current draft, thanks! Bye for now... Tom *************** Env Management (inquiry.tex): *************** 1. In Bill and Rusty's hands at the moment --- >From hart@nipmuc.fsl.noaa.gov Fri Oct 22 12:37 PDT 1993 Received: from gw1.fsl.noaa.gov by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA11487; Fri, 22 Oct 93 12:37:45 -0700 Return-Path: Received: by gw1.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA14955; Fri, 22 Oct 93 19:36:43 GMT Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA23637; Fri, 22 Oct 93 13:43:35 MDT Message-Id: <9310221943.AA23637@nipmuc.fsl.noaa.gov> To: otto@merckx.cse.ogi.edu Cc: hart@nipmuc.fsl.noaa.gov, hender@nipmuc.fsl.noaa.gov Subject: MPI Environmental Management (Ch 7) Date: Fri, 22 Oct 1993 13:43:35 -0600 >From: Leslie Hart (NOAA/FSL) Status: RO Steve, On page 152 (lines 45-48) there seems to be a typo. I believe that the profile for the Fortran binding of MPI_GET_PROCESSOR_NAME should be: MPI_GET_PROCESSOR_NAME (NAME, IERROR) CHARACTER*(*) NAME INTEGER IERROR (Length of a character string in Fortran is passed implicitly). Regards, Leslie Hart (hart@fsl.noaa.gov) >From hart@nipmuc.fsl.noaa.gov Fri Oct 22 12:42 PDT 1993 Received: from gw1.fsl.noaa.gov by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA11494; Fri, 22 Oct 93 12:42:17 -0700 Return-Path: Received: by gw1.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA15014; Fri, 22 Oct 93 19:41:15 GMT Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA23704; Fri, 22 Oct 93 13:48:07 MDT Message-Id: <9310221948.AA23704@nipmuc.fsl.noaa.gov> To: otto@merckx.cse.ogi.edu Cc: hart@nipmuc.fsl.noaa.gov, hender@nipmuc.fsl.noaa.gov Subject: MPI Environmental Management (more) (Ch 7) Date: Fri, 22 Oct 1993 13:48:06 -0600 >From: Leslie Hart (NOAA/FSL) Status: RO Steve, On pages 156 (line 39) to 157 (line 1) I noticed that MPI_USER_SPECIFIES_BUFFER takes three arguments: communicatior, buffer, size. In my notes I have a vote (8-4-7) for the function with four arguments: communicator, byte count of buffer, number of blocking sends outstanding and a success/failure flag. I recall a desire to have this function to be similar to the suggest function. I also recall a problem with supplying a buffer since you may not know when it is actually available (a free communicator may not complete immendiately). Do you have any recollections of this? Regards, Leslie Hart (hart@fsl.noaa.gov) >From lyndon@epcc.ed.ac.uk Tue Oct 26 05:34 PDT 1993 Received: from daedalus.epcc.ed.ac.uk by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA06480; Tue, 26 Oct 93 05:34:46 -0700 Return-Path: Date: Tue, 26 Oct 93 12:36:07 GMT Message-Id: <7710.9310261236@subnode.epcc.ed.ac.uk> >From: L J Clarke Subject: Comments on MPI Draft To: Steve Otto Reply-To: lyndon@epcc.ed.ac.uk Status: RO Hi Steve The draft is impressive, to say the least. Obviously you guys have been busy! I hope the observations below, typos and such, are helpful. Of course I will undoubtedly have missed much. Best Wishes Lyndon o--------------------o 1. Page 29 line 28/29. Text is "... may fail because because ..." Word repitition. Replace with "... may fail because ..." done 2. Page 37 line 31. Text is "... calls, including a mix sends and receives ..." Missing word. Replace with "... calls, including a mix of sends and receives ..." done 3. Page 43 lines 17-24 and possible global. The MPI function MPI_COMMOBJ_FREE has a C binding MPI_Commreq_free and a Fortran binding MPI_COMMOBJ_FREE, which is inconsistent. The C object type is MPI_Comm_request. Is there a mix-up here? names are now different and consistent. 4. Page 74 line 16. The BUFFER argument to the Fortran binding of MPI_BCAST is written as INTEGER whereas it should be . Replace "INTEGER BUFFER, COUNT, DATATYPE, ..." with "INTEGER COUNT, DATATYPE, ... BUFFER(*)" right. done. 5. Page 75 line 35. Text is "... the displacement (relative to ...". Extraneous left bracket. Replace with "... the displacement relative to ...". done 6. Page 96 lines 29-42. The example declares C function "void myProd(Complex invec[], Complex inoutvec[], int *len)" However ANSI C does not require a pointer to void to be coerced into a pointer to type (other than void) - it only coerces from pointer to type to pointer to void - which would be required if the user function in MPI_USER_REDUCE has prototype "void func(void *, void *, int *)" and it should have this prototype since user reduce has the corresponding pair of "void *" arguments. This is really a detailed language binding issue. Suggestion is that (a) prototype for user function is given in the draft; (b) example is changed to accept "void *" rather than "Complex []", and explicit cast from "void *" to "Complex *" required in the example. I'm not going to change it right now since I'm afraid there may be even more issues here. What I have in mind is this: If we just declare everything as void *, then MPI has no idea how to do the "chunking" of the data. Eg, in the example, we have to make sure we are handed complete complexes inside of myProd, not just half of one. I tried to represent this in the example by saying: /* explain to MPI what a "complex" is (remember, this is C!) * Note we are assuming compiler implements type Complex as * two contiguous doubles */ MPI_Type_contiguous( 2, MPI_DOUBLE, &ctype ); MPI_Type_commit( ctype ); MPI_User_reduce( a, answer, 100, ctype, myProd, root, comm ); and we probably need to say that USER_REDUCE hands complete copies of sendtype to the user-function. 7. Page 120 line 10. Text is "... execute MPI_INTERCOMM_start()." Inconsistent case. Replace with "... execute MPI_INTERCOMM_START()." defer -- Tony has chapter 8. Page 120 line 15. Text is "... execute MPI_INTERCOMM_finish()." Inconsistent case. Replace with "... execute MPI_INTERCOMM_FINISH()." defer 9. Page 120 line 29/30. Text is "... creation operation, This routine ..." Punctuation. Replace with "... creation operation. This routine ..." defer 10. Page 120 line 40 Text is "MPI_INTERCOMM_MERGE(intercomm, key, newintracomm". Missing right bracket. Replace with "MPI_INTERCOMM_MERGE(intercomm, key, newintracomm)". defer 11. Page 121 line 27. Text is "INTEGER COMM, NAME, NEW_COMM, ...". The NAME argument of the Fortran binding to MPI_INTERCOMM_NAME_START is declared as INTEGER whereas it should be a character string. Replace with "INTEGER COMM, NEW_COMM, ... CHARACTER *(*) NAME" although note that *(*) is not standard and raises a language binding query. defer 12. Page 123 line 18/19 Text is "... MPI_ATTR_get_key( ..." Inconsistent case. Replace with "... MPI_ATTR_GET_KEY( ..." defer 13. Page 151 line 47. Text is "INTEGER LEN, NAME, IERROR". The NAME argument of the Fortran binding to MPI_GET_PROCESSOR_NAME is declared as INTEGER whereas is looks like it should be a character string. Replace with "INTEGER LEN, IERROR ... CHARACTER *(*) NAME" although note that *(*) is not standard and raises a language binding query (cf 11. above). defer -- Bill Gropp has chapter 14. Page 159 line 32/33. Text is "int MPI_INIT(int argc, char **argv)". Case inconsistent with C binding guidelines. Function name should be "MPI_Init". Looks like also a problem with the arguments. Should these not be "int *argc, char ***argv". I ask this because (a) typically these get passed in because the system has some positions at the beginning or at the end booked for special use, and wants to read them then strip them out (b) this is inconsistent with the arguments in the context chapter examples which seem to assume arguments of "int *argc, char **argv". defer 15. Page 159 line 42/43. Text is "MPI_init( argc, argv );" Case inconsistent with C binding guidelines. Function name should be "MPI_Init". Also above comment on arguments. defer o--------------------o /--------------------------------------------------------\ e||) | Lyndon J Clarke Edinburgh Parallel Computing Centre | e||) c||c | Tel: 031 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | c||c \--------------------------------------------------------/ >From jim@meiko.co.uk Wed Oct 27 05:07 PDT 1993 Received: from hub.meiko.co.uk by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA00456; Wed, 27 Oct 93 05:07:06 -0700 Return-Path: Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA14272 (5.65c/IDA-1.4.4 for otto@merckx.cse.ogi.edu); Wed, 27 Oct 1993 12:08:08 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA04766; Wed, 27 Oct 1993 13:07:57 +0000 Date: Wed, 27 Oct 1993 13:07:57 +0000 >From: jim@meiko.co.uk (James Cownie) Message-Id: <9310271207.AA04766@tycho.co.uk> To: otto@merckx.cse.ogi.edu Cc: mpi-core@cs.utk.edu In-Reply-To: <9310211852.AA25274@merckx.cse.ogi.edu> (message from Steve Otto on Thu, 21 Oct 93 11:52:38 -0700) Subject: Re: MPI DRAFT Content-Length: 6214 Status: RO Steve, Thanks to you and the rest of the editorial group. (Especially whoever did the scatter/gather derived datatype pictures !). Me. Significant issue ================= Collective (Reduction operations) We have only a weak specification of the semantic requirement for these. Do we want reproducibility ? (I can't remember what we concluded. Who can ?) reproducibility/repeatability is now mentioned in the terms chapter. In terms: "However, repeatability is a desirable quality-of-implementation goal. In particular, it is expected that MPI implementations will avoid introducing additional non-determinism. For example, while MPI can take advantage of associativity and commutativity in reordering the execution of a global reduce operation, it is desired (but not required) that the same operations order be used in repeated executions." In particular for the ALL REDUCE operations we don't currently say that everyone gets the SAME result. I think we should (at least) say this, even if we want to be relaxed about the reproducibility of that result. good point. done. Global edits ============ It would be nice to change "parameter" to "argument" throughout. (Possibly with a done, except for context/inquiry which are "checked out" at the moment. minor discussion in the terms and conventions section, something like ... Since the word PARAMETER is a keyword in the Fortran language, we use the word argument to denote the arguments to a subroutine. These are normally referred to as parameters in C, however we expect that C programmers will understand the word arguments (which has no specific meaning in C), thus allowing us to avoid unnecessary confusion for Fortran programmers. ) done, placed in terms.tex. Typos ===== I've included here some changes to the Fortran binding for routines which need to return/take character strings. I believe that these should take CHARACTER*(*) arguments, in accordance with standard Fortran practice. The current draft has these as INTEGER and also passes (a now unnecessary) length. Here are a set of typos in Oct 19 draft... Page Line Change Anonymous (Acknowledgements) Line 21 I think it is Mark Sears, (not Marc) done 9 25 delete the : done 28 "allocated on user storage or on system" change both "on"s to "in"s done 10 3 Add "an" at beginning of the line done 8 "is the user resposibility" => "is the user's responsibility" done 20 "begginning" => "beginning" done 20 "indecates" => "indicates" done 12 19 "spedify" => "sepcify" done 39 "processor" => "process" done 44 "implicit global states" => "implicit state" done 47 "or other events" => "or with other events" done 13 11 "to always" => "always to" done 12 "consecutive integers" => "consecutive integers in the range 0 .. groupsize-1" done 14 5 ":" => "." 6 "however" => "However" done 15-18 Was this what we said ? I thought we were much stronger than this. (see Significant issues above). ???Hmmm... 43 "As an" => "An" done 15 19 "SIGFPE,and" => "SIGFPE or" done 16 21 "manaipulation" => "manipulation" done 22 23 delete "and is seen as a quality of implementation issue" (Or Moose will be very unhappy !) done 30 "MPI implementation" => "An MPI implementation" done 26 11 "types as" => "types from" done 27 10 "part of" => "some of" done 20 "either ends" => "either end" done 29 29 "Such call" => "Such a call" done 31 40 "associates" => "associate" done 41 delete "about" done 43 "buf copying" => "copy" done 33 7 "In multi-threaded" => In a multi-threaded" done 34 2 "either receives" => "either receive" done 39 4 "user" => "user's" done 42 39 "semantics as" => "semantics to" done 43 48 "correctly the sends and receives" => "the sends and receives correctly" done 46 27 "allows" => "allow" done 36 "have used" => "have been used" done 58 15 "Each of the sends can match each of the receives" => "From a type matching viewpoint any of the sends legally matches any of the receives" done 16 "Overlaping" => "Overlapping" done 20 "Or one of them have" => "Or one of them has" done 22 "overlaping" => "overlapping" done 68 27 "This section is not part of the MPI definition" should be made MUCH more significant (Bold, or margin or something) done (bold) 83 48 "exectuted" => "executed" done 88 45 Lots of (unnecessary) blank space Not if you're LaTeX! 89 40 Ditto ditto ************tony has... 106 28 "allows to invoke" => "allows the invocation of" 43 "users that" => "users who" 44 "and ignore the rest of this chapter" should be in bold or CAPS 107 9 "users that" => "users who" 108 26 "The groups corresponding to this communicator do not appear as pre-defined constants," => "The group embedded in this communicator does not appear as a pre-defined constant," 115 1 "with all its cahed information". This is WRONG. It should say something like "For each piece of cahed information on the original communicator the relevant copy_fn is called. This can either copy the cached information, or not, as is appropriate." 122 6 "in size or" => "in size to, or" 123 40 "put into this attribute set" => "attached to this communicator" 124 13 "are invoked." => "are invoked in an arbitrary order." 128 42 "if ((me % 2) == 0)" => "if ((me % 2) == 0 && (me + 1) < size))" So don't send if I don't have a neighbour. Should also remove all the modulusing in the sends and receives. So 43 "(...,((me+1)%size)," => "...,me+1," 45 "((me-1+size)%size)" => "me-1" *************** Bill Gropp has 152 47,48 Should read MPI_GET_PROCESSOR_NAME(NAME,IERROR) CHARACTER*(*) NAME INTEGER IERROR 154 41 "no to" => "not to" 156 20 "an algorithm" => "a program" 157 31 "can not" => "cannot" 43 "Such handler" => "Such a handler" 48 "MPI_WORLD" => "MPI_COMM_WORLD" 158 1 "with a communicator;" => "to a communicator." 159 16,17 Should read MPI_ERROR_STRING(ERRORCODE, STRING, IERROR) INTEGER ERRORCODE, IERROR CHARACTER *(*) STRING 40 "before any other MPI routines are called" => "before any other MPI routine (apart from MPI_INITIALIZED) is called" ****************** -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com >From doss@ERC.MsState.Edu Wed Oct 27 08:09 PDT 1993 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA00647; Wed, 27 Oct 93 08:08:56 -0700 Return-Path: Received: by ogicse.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.4) id ; Wed, 27 Oct 93 08:10 PDT Received: from Athena.ERC.MsState.Edu by Phoenix.ERC.MsState.Edu (4.1/6.0s-FWP); id AA24780; Wed, 27 Oct 93 10:14:32 CDT >From: Nathan E. Doss Received: by Athena.ERC.MsState.Edu (4.1/6.0c-FWP); id AA16589; Wed, 27 Oct 93 10:14:05 CDT Message-Id: <9310271514.AA16589@Athena.ERC.MsState.Edu> Subject: Minor inconsistencies in collective chapter To: otto@cse.ogi.edu Date: Wed, 27 Oct 1993 10:14:04 -0500 (CDT) Cc: tony@cs X-Mailer: ELM [version 2.4 PL17] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1177 Status: RO Dear Steve, Below is a list of minor inconsistencies that I've found in the Collective Communication chapter in the Oct. 19, 1993 version of the MPI draft. Tony Skjellum suggested that I send this list to you. - In section 4.5.4 Scan, all arguments to the MPI_SCAN function are listed as "IN" arguments. "recvbuf" should be "OUT". done pg 100, line 25 - For most (all?) of the Fortran bindings, IERROR is listed as an argument, but is not included in the argument list. done - For many of the C bindings, the argument list should include "MPI_Datatype recvtype", but "recvtype" has been left out. yep. done. - The argument list for the C binding of MPI_GATHER should contain "sendcount" instead of "sendcnt" pg 74, line 41 done - The last argument in the generic description of MPI_SCATTER and MPI_SCATTERV should be "comm" instead of "group". pg 83, line 36 pg 84, line 35 done - There are a few places where "recvcount[i]" should be "recvcounts[i]" and where "sendcount[i]" should be "sendcounts[i]". pg 76, line 13 pg 85, line 5 pg 91, line 45 done -- Nathan Doss doss@ERC.MsState.Edu NOTE: I have not gotten to Steven H-L's comments on topology that appear below. --Steve Otto >From lederman@super.org Wed Oct 27 06:51 PDT 1993 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA00566; Wed, 27 Oct 93 06:51:49 -0700 Return-Path: Received: by ogicse.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.4) id ; Wed, 27 Oct 93 06:53 PDT Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA12950; Wed, 27 Oct 93 09:53:19 EDT Received: by b125.super.org (4.1/SMI-4.1) id AA01775; Wed, 27 Oct 93 09:53:18 EDT Date: Wed, 27 Oct 93 09:53:18 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9310271353.AA01775@b125.super.org> To: hempel@gmd.de Subject: topology chapter comments Cc: otto@cse.ogi.edu Status: RO Rolf, Last night I babysat for some friends. After they went to bed I finally had the peace and quiet to read your chapter. The following is the comments I would like to make. I know a number of changes were made to your chapter during the editing session and I suspect a number of these thing occurred then. I am putting them in order of the pages so editing might be easier. I give the page/line number of the draft I got from ogi last week. I have not had time to carefully check everything so if some of the comments do not make sense either ignore it or ask me for clarification. You may have already caught these changes. Sorry I did not get to this sooner but life is always busy. I know there are a lot and some are small but since this is the last draft I wanted to pass them all along. Steve ---------------------------------------------------------------------- - p.140/44: "processes is be represented" should be "processes can be represented" - p. 142/7: "processes along a given" I prefer: "processes among a given" - p. 142/22+23: "outlines such implementation" should be "outlines such an implementation" - p. 142/34-37+: This is a general comment. You refer to the logical type. In chapter 2 is a definition of the choice type, etc but not the logical type. In FORTRAN I know what this is but not in C. Maybe MPI should make this an integer with the standard 0/1 definition as C does. - p. 142/46: "than the the group" should be "than the group" - p. 143/25,26,28,29: MPI_MAKE_DIM should be MPI_MAKE_DIMS - p. 144/1: "are not member of the grid" should be "are not members of the grid" - p. 144/2: "larger than then the group" should be "larger than the group" - p. 144/25: "i, i=1,..." should be "i,i=0,..." since this is 0 up numbering in C. - p. 144/29: "i, i=1,....,nnodes-1" should be "i, i=1,....,nnodes" got the 1 start for FORTRAN but left the -1 accidentially - p. 144/30: "1<=j<=index[0]" should be "1<=j<=index[1]" again forgot the 1 up numbering in FORTRAN - p. 144/31: "index[i-1]+1 <= j <= index[i]" should be "index[i]+1 <= j <= index[i+1]" same one up numbering problem - p. 145/27 + p. 146/1: there is MPI_GET_GRAPHDIMS and MPI_GET_CARTDIM. Either have the S or not. Be consistent with DIM or DIMS. You already used MPI_MAKE_DIMS so maybe it should have the S. - p. 145/44: "index, edges vectors containing" should be "index, edges array of integers containing" - p. 146/9: "MPI_GET_CART(....,period,...)" should be "MPI_GET_CART(....,periods,...)" since you use periods (not period) most other places and is what the input argument is defined as. - p. 147/1+8: At my request you changed MPI_GRAPH_NNEIGHBORS to MPI_GRAPH_NEIGHBOR_COUNT. The only concern I have now is that MPI_GRAPH_NEIGHBORS has an S but the new one doesn't. I don't care if NEIGHBOR or NEIGHBORS but it need to be consistent. - p. 147/2+9: the comm argument needs to be identified as a (handle). - p. 147/12: maxneighbors needs to be identified as a (integer). - p. 147/22: I would add a row right below the one with "exchange shuffle unshuffle" in the table which gives "neighbor(1) neighbor(2) neighbor(3)" so it is clearer which values in the example correspond to what in the code. - p. 149/24: "MPI_MAP_CART(...,period,..)" should be "MPI_MAP_CART(...,periods,..)". - p. 149/43+150/17: I don't understand how using color=MPI_UNDEFINED is safe. How do you know that this constant is not 0 in some implementation. I may have missed something here. - p. 149/45-48: I don't think this definition works when MPI_SUBCART is called with multiple values in array remain_dims = false. It seems to me that you need multiple number (not a single number) encoding to deal with this case. - p. 150/34: This is a picky point but would you use MPI_SENDRECV instead of MPI_SEND (with an MPI_RECV) to do an exchange. - p. 151/17: "MPI_MAKE_CART(...,rank,...)" should have no rank argument. - p. 151/18: "MPI_GAT_CART" should be "MPI_GET_CART" That's all folks .... :-) ************ IIS ************ Steve, I have updated the iis chapter to reflect the changes made in other chapters. The only change of substance is: p. 170/34: MPI_USER_SPECIFIES_BUFFER was removed from the subset. The only things that my version still has a problem with is the two new references in the topol.tex and inquiry.tex chapter that I already wrote you about. Have fun :-) Steve >From snir@watson.ibm.com Mon Oct 25 21:23 PDT 1993 Received: from iliamna.cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA05917; Mon, 25 Oct 93 21:23:53 -0700 Return-Path: Received: by iliamna.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.17) id ; Mon, 25 Oct 93 21:25 PDT Message-Id: Received: from YKTVMV by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 5425; Tue, 26 Oct 93 00:25:28 EDT Date: Tue, 26 Oct 93 00:25:09 EDT >From: "Marc Snir" X-Addr: (914) 945-3204 (862-3204) 28-226 IBM T.J. Watson Research Center P.O. Box 218 Yorktown Heights NY 10598 To: otto@iliamna.cse.ogi.edu Subject: Note from PCTERM3 Reply-To: SNIR@watson.ibm.com Status: RO ------------------------------- Referenced Note --------------------------- First batch of corrections I use px,ly for page x, line y Explanations for changes I propose are in square brackets [] Global changes are followed by a (probably incomplete) list of places where they occur. ******************************** GLOBAL CHANGES FLAG arguments have type LOGICAL in Fortran replace INTEGER FLAG by LOGICAL FLAG p32,l44 p36,l1 p38,l40 done status argument has type (Status) add (Status) at end of description of status parameter p19,l26 p32,l20 p32,l40 p35,l17 p35,l43 p37,l35 ?? p38,l12 ?? p44,l33 p45,l14 done Datatype argument has type (handle) add (handle) after description of {\tt datatype} argument. or {\tt sendtype}, {\tt recvtype} p17,l6 p44,l15 p21,l5 p44,l25 p27,l33 p28,l5 p30,l10 p30,l32 p31,l5 p31,l25 p40,l37 p41,l10 p41,l32 p42,l5 p45,l6 done. AND ALSO: p19,l21 communication object (handle) -> communication request (handle) handle to communication object -> communication request (handle) p30,l16 p32,l18 p30,l39 p32,l37 p31,l11 p31,l31 p40,l42 p41,l17 p41,l39 p42,l22 p43,l19 done. array of handles to communication objects -> array of requests array of communication objects -> array of requests p35,l14 p42,l44 p35,l38 p36,l16 p36,l36 done. I put in: "array of requests (array of handle)" ************************************ p6,l17 No support -> Support done. p6,l36 recv -> receive done p7,l18 Fortran -> Fortran 77 done p9,l11 Such an operation -> Usually, such an operation [Cancel is a local operation, but may require communication] *NOT* done: unsettling to the reader...does CANCEL require communication with the user-level process? Perhaps communications need to occur, but it is hidden from the user and so may as well be considered local, no? p9,l42 as as -> as done p10,l6 MPI_GROUP_FREE -> free the object done p10, l15 Array of handles -> Array arguments done [they are not always handles] p10,l16 that is an array-of-handles. The ... -> that is an array of opaque objects, or an array of handles [no italics]. An ... done p10,l21 [Add at end of paragraph] -> The same approach is followed for other array arguments. done p12,l12 The same MPI subroutine -> An MPI subroutine with a choice argument done p12,l13 Figure 2.4.2 -> Figure 2.1 done p12,l19 spedify -> specify done p12,l30 Buffer pointers are -> Choice arguments are pointers done p13,l32 The recommended -> A high quality done p13,l35 an error value -> a code done p16,l21 manaipulation -> manipulation done p17,l38 delete MPI_DOUBLE_COMPLEX line done p18,l23 ... C; MPI_REAL2 ... -> ... C; \const{MPI\_DOUBLE\_COMPLEX} for double precision complex in Fortran; MPI_REAL2 ... [DOUBLE PRECISION COMPLEX is not standard Fortran 77] done p18,l45 TagRange -> \mpifunc{MPI\_GET\_VALID\_TAG\_RANGE} done p18,l20 (state) -> (handle) done -- actually p19. p21,l48 [add at end of paragraph] -> Note that the receiver is not required to receive messages in the order they were sent, if they have distinct envelopes. done, but I'm unhappy about the placement here...it puts this statement in the same paragraph with threads, while this is not necessarily a threads issue. p22,l30 provide information -> provide information and control done p27,l4 posted; Otherwise -> posted; otherwise done p29,l39 communication objects -> communication request objects [communication objects are request objects] done p29,38 An opaque communication object -> A request object done p30,l3 nonblocking communication -> nonblocking send. done p30,l20 *request -> request [The call modifes the opaque object pointed at by request, but does not modify the handle] done p30,25 nonblocking communication -> nonblocking send. done p30,l43 *request -> request done p30,l48 nonblocking communication -> nonblocking send. done p31,l11 *request -> request done p31,l31 *request -> request done p31,40 communication object and associates it with the request. -> communication request and associates it with the handle. done p31,l43 buf -> start done p32,l5 [delete] or any other location that can be referenced by the send operation [there are none] done p32,l5 [delete] or any other location that can be referenced by the receive operation done p32,l17 IN request ... -> {\INOUT}{request}{communication request (handle)} done p32,l22 request -> *request [WAIT nullifies the handle] done p32,l37 IN request ... -> {\INOUT}{request}{communication request (handle)} done p32,l38 logical -> (logical) done p32,l41 request -> *request done p33,l26 they will be also used to complete any other nonblocking communication call -> they are also used to complete all other nonblocking communication calls. done p35,l26 Returns in ... with the status. -> Returns in \mpiarg{index} the index of that request in the array and returns in \mpiarg{status} the status of the completing communication. done p35,l30 is is -> is done p35,l39 index of handle for operation -> index of operation done p35,l42 true ... -> \const{true} if one of the operations is complete (logical) done p36,l16 IN -> INOUT done p37,l31 mix sends -> mix of sends done p37,l33 and the array_of_statuses -> and the entries of the array_of_statuses done p38,l7 first message -> first arriving message done p38,l9 second message -> second arriving message done p38,l20 [add new paragraph] An MPI implementation of \mpifunc{MPI\_PROBE} need to guarantee progress: if a call to \mpifunc{MPI\_PROBE} has been issued by a process, and a send that matches the probe has been initated by some process, then the call to \mpifunc{MPI\_PROBE} will return, unless the message is received by another concurrent receive operation (that is executed by another thread at the probing process). Similarly, if a process busy waits with \mpifunc{MPI\_IPROBE} and a matching message has been issued, then the call to \mpifunc{MPI\_IPROBE} will eventually return {\tt flag = true} unless the message is received by another concurrent receive operation. done p40,l15 add implementation note \implement{ If a send operation uses an ``eager'' protocol (data is transfered to the receiver before a matching receive is posted), then the cancelation of this send may require communication with the intended receiver, in order to free allocated buffers. On some systems this may require to interupt the intended receiver. Therefore, it is expected that \mpifunc{MPI\_CANCEL} will be an expensive operation on many systems, to be used only exceptionally. Note that, while communication may be needed to implement \mpifunc{MPI\_CANCEL}, this is still a local operation, in the sense that its completion does not depend on the code executed by other processes. If processing is required on another process, this should be transparent to the application (hence the need for an interupt and an interupt handler). } done p40,l20 to the communication object -> to a communication request done p40,l23 The communication object -> The request done p40,l28 A communication object -> A request done p40,l30 a communication object -> a request done p40,l45 *request -> request done p40,l48 INTEGER BUF -> BUF(*) done p41,l1 The call ... -> [delete sentence] done p41,l21 *request -> request done p41,l23 INTEGER BUF -> BUF(*) done p41,l42 *request -> request done p41,l45 INTEGER BUF -> BUF(*) done p42,l13 *request -> request done p42,l15 INTEGER BUF -> BUF(*) done p42,l17 communication object -> request done p42,l33 updated -> accessed done p42,l44 requestss -> requests done p43,l8 communication object -> request done p43,l10 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l12 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l17 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l21 MPI_Commreq_FREE -> MPI_Request_free (or MPI_Comm_request_free) done p43,l22 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done p43,l27 MPI_COMMOBJ_FREE -> MPI_REQUEST_FREE (or MPI_COMM_REQUEST_FREE) done [to be consistent with current terminology -- change need to be propagated to appendices] yes -- done. p42,l34 communication object -> request done p44,l46 the same tag ... [end of paragraph] -> the same communicator, but possibly different tags. The send buffer and receive buffers are disjoint, and may have different lengths and datatypes. [lengths] done! >From snir@watson.ibm.com Wed Oct 27 21:30 PDT 1993 Received: from iliamna.cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA05075; Wed, 27 Oct 93 21:30:36 -0700 Return-Path: Received: by iliamna.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.17) id ; Wed, 27 Oct 93 21:32 PDT Message-Id: Received: from YKTVMV by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 1101; Thu, 28 Oct 93 00:32:24 EDT Date: Thu, 28 Oct 93 00:31:56 EDT >From: "Marc Snir" X-Addr: (914) 945-3204 (862-3204) 28-226 IBM T.J. Watson Research Center P.O. Box 218 Yorktown Heights NY 10598 To: otto@iliamna.cse.ogi.edu Subject: Note from PCTERM3 Reply-To: SNIR@watson.ibm.com Status: RO Third batch of corrections (collective communication chapter). ------------------------------- Referenced Note --------------------------- GOING BACK TO POINT TO POINT [I assume C is smart enough to do type conversion when arguments are passed; the coertion of pointers to type (void *) is superfluous] Ansi C is but ... (void *) -> [delete] p62,l26 p62,l27 p62,l28 p62,l42 p62,l43 p62,l44 p62,l45 p63,l40 p64,l42 p64,l43 p64,l44 p65,l23 p65,l24 p65,l25 p66,l14 p66,l46 p66,l47 done ************************* COLLECTIVE ************************* GLOBAL add (handle) to description of datatype arguments in collective communication calls p72,l7 p72,l29 p72,l35 p74,l8 p74,l29 p74,l36 p75,l30 p75,l41 p83,l27 p83,l33 p84,l28 p84,l32 p88,l12 p88,l22 p89,l7 p89,l19 p90,l11 p90,l19 p91,l10 p91,l20 p95,l36 p97,l26 p98,l16 p98,l35 p99,l6 p99,l38 p100,l29 p101,l7 p101,l31 done add IERROR as last argument in F77 binding of collective communication calls. p72,l15 p72,l46 p74,l15 p74,l46 p76,l4 p83,l42 p84,l41 p88,l32 p89,l28 p90,l28 p91,l27 p93,l1 p95,l45 p97,l36 p98,l23 p98,l42 p100,l40 p101,l15 p101,l38 done reverse displs and recvcounts (sendcounts) in ..V collective communication functions; also, replace displs by recvdispls or sendispls. Propagate change to \funcarg items, and to C and Fortran binding. p75,l25 (p75,l33 p75,l47 p76,l3) NOTE Fortran binding had other errors p78,l24 p81,l8 p82,l18 p83,l16 p84,l19 (p84,l23 p84,l37 p84,l40) p86,l37 p87,l44 p89,l1 (p89,l11 p89,l23 p89,l30) p89,l39 p91,l1 (p91,l4 p91,l13 p91,l23 p91,l27) done Examples 1-8 on pages 77-83 and examples 9-11 on pages 85-87: It seems the use of group is superfluous yep... MPI_Comm_group( comm, &mygroup) MPI_GROUP_size( mygroup, &gsize) can be replaced by MPI_Comm_size( comm, &gsize) done I would also suggest to do the malloc only at the root. not done...I like a more SPMD program... In examples 5-8: sptr should be computed used pointer arithmetic, not byte arithmetic [the & C operator is not guaranteed to return a byte address]. -> sptr = sendarray + myrank OR, sptr = &sendarray[0][myrank] p80,l18 p81,l7 p82,l17 p83,l15 good point! done Similarly, rptr in example 11 p87,l44 -> rptr = &rbuf[0][myrank] OR rbuf+myrank (and why notr rbuf -> recvarray, for symmetry?) done Examples 7,11 Would be nicer to compute first displs[gsize-1] and then set bufsize = displs[gsize-1]+rcounts[gsize-1] yes, I had done this somewhere else. done. ************************************************** p71,l37 [add at end of sentence] -> and a few auxiliary functions. done p72,l7 ,or any other .. operation -> [delete] done p72,l14 guarnatees -> guarantees done p75,l3 to, -> to done, and elsewhere p75,l10 i \times \extent(recvtype) -> i \times recvcount \times extent(recvtype) done -- ouch! p75,l16 must exactly match -> must be equal to done p75,l17 ...,and the type .. processes -> at the root. done p75,l18 Exact matching means... -> [delete sentence] done p75,l35 (relative to recvbuf -> (relative to recvbuf) done p75,l22 -> The type signature implied by sendcount, sendtype at each process must be equal to the type signature implied by recvcount, recvtype at the root (however, the type maps may be different). [the typemap is the sequence of types,displacements; the type signature is the sequence of basic types of the message components.] yes. done. p75,l22 -> The type signature implied by sendcount, sendtype at process i must be equal to the type signature implied by recvcount[i], recvtype at the root (however, the type maps may be different). done p78,l28 [replace sentence] -> Note that the program is erroneous if stride < 100. done p79,l18 150,1 -> 1, 150 done, (it's more than just this) p80,l14 150,1 -> 1, 150 done [We don't do address arithmetic -- The & C operator is not guaranteed to return byte addresses. Instead, we do pointer arithmetic] p80,l32 int disp[2] -> MPI_Addr disp[2] wait. p80,l33 types -> type done p81,l5 type,disp,blocklen -> blocklen, disp, type done p82,l31 int disp[2] -> MPI_Addr disp[2] wait p83,l13 type,disp,blocklen -> blocklen, disp, type done p83,l30 [add at end of line] -> (choice) done p83,l48 exectutued -> executed done p84,l3 i \times \extent(sendtype) -> i \times sendcount \times extent(sendtype) done p84,l12 -> the type signature associated with sendcount, sendtype at the root must be equal to the type signature associated with recvcount, recvtype at all processes (however, the type maps may be different). done p85,l10 -> The type signature implied by sendcount[i], sendtype at the root must be equal to the type signature implied by recvcount, recvtype at process i (however, the type maps may be different). done p86,l37 Scatter -> Scatterv done p87,l42 150, 1 -> 1, 150 done p88,l38 The ... recvtype. -> The type signature associated with sendcount, sendtype at a process must be equal to the type signature associated with recvcount, recvtype at any other process. done p89,l35 The ... (processes). -> The type signature associated with sendcount, sendtype at process j must be equal to the type signature associated with recvcounts[j], recvtype at any other process. done p89,l39 [delete superfluous space] can't -- latex problem p90,l34 The ... recvtype. -> The type signature associated with sendcount, sendtype at a process must be equal to the type signature associated with recvcount, recvtype at any other process. done p90,l41 i \times \extent(sendtype) -> i \times sendcount \times extent(sendtype) done p90,l48 i \times \extent(recvtype) -> i \times recvcount \times extent(recvtype) done p91,l36 ,but the ... on process j-> The type signature associated with sendcount[j], sendtype at process i must be equal to the type signature associated with recvcount[i], recvtype at process j. done p92,l42 MPI_defined constant -> state done p92,l48 void op -> MPI_Op op done [MINLOC and MAXLOC are uglier by the day -- But I am not sure I have a solution -- Unless we delete them, and add a predefined "user-define" function.] p94,l32 MPI_LONG_DOUBLE -> [delete] done p95,l16 30 -> 60 [Am I right?] I don't think so -- we are still "maxloc"ing over 30 items, and that is the count. The locs just land in the upper 30 of the arrays, and that's why they are declared as [60] p95,l37 [add at end of line (function) done p98,l1 [delete implementation note] done p98,l17 [add at end of line] -> (state) done p98,l22 void op -> MPI_Op op done p98,l36 [add at end of line] -> (function) done p99,l11 eint -> int done p99,l40 [add at end of line] -> (state) done p99,l44 void op -> MPI_Op op done p100,l21 [delete superfluous empty lines in between list of arguments] can't -- latex problem p100,l38 void op -> MPI_Op op done p101,l9 [add at end of line] -> (function) done p101,l32 [add at end of line] -> (function) done p103,l2 -21 -> [It would be nice to replace with a figure] I'll see.... done. In fact -- as drawn in the document it was wrong!!! >From snir@watson.ibm.com Fri Oct 29 12:03 PDT 1993 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA11590; Fri, 29 Oct 93 12:03:22 -0700 Return-Path: Received: by ogicse.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.4) id ; Fri, 29 Oct 93 12:05 PDT Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9951; Fri, 29 Oct 93 15:05:12 EDT Received: from YKTVMH by watson.vnet.ibm.com with "VAGENT.V1.0" id 5467; Fri, 29 Oct 1993 15:04:50 EDT Received: from snir.watson.ibm.com by yktvmh.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 29 Oct 93 15:04:49 EDT Received: by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA21159; Fri, 29 Oct 1993 15:04:57 -0400 >From: snir@watson.ibm.com (Marc Snir) Message-Id: <9310291904.AA21159@snir.watson.ibm.com> To: otto@cse.ogi.edu Reply-To: snir@watson.ibm.com Date: Fri, 29 Oct 93 15:04:56 -0500 Status: RO -:) -:) -:) -:) -:) GLOBAL [I propose change in names of cache functions , in order to bring them in line with other functions, and shorten them] MPI_ATTR_GET_KEY -> MPI_ATTR_CREATE_KEY p122,l17 p122,l26 p122,l27 p124,l44 done MPI_ATTR_PUT_VALUE -> MPI_PUT_ATTR p123,l13 p123,l23 p123,l24 p123,l29 p125,l26 done MPI_ATTR_GET_VALUE -> MPI_GET_ATTR p123,l32 p123,l42 p123,l45 p123,l28 p125,l5 done MPI_ATTR_DELETE_VALUE -> MPI_DELETE_ATTR p123,l30 done (actually a few more) In all examples MPI_Init(&argc,argv) -> MPI_Init(argc, argv) p128,l37 p129,l12 p129,l39 p131,l17 p132,l39 done *********************************************************************8 p108,l27 do not appear -> does not appear done p109,l30 RANKS1, GROUP2, RANKS2 -> RANKS1(*), GROUP2, RANKS2(*) done p111,l5 [add] -> (integer) done p111,l12 [add] -> (handle) done p111,l17 RANKS -> RANKS(*) done p111,l30 [add] -> (integer) done p111,l42 RANKS -> RANKS(*) done p118,l31 INTEGER FLAG -> LOGICAL FLAG done p118,l40-42 [uppercase] -> MPI_COMM_SIZE MPI_COMM_GROUP MPI_COMM_RANK done p119,l35 [delete superfluous blanks] done p119,l41 [correct mpiarg\] done p120,l5 NEW_COMM -> [delete] done p120,l11 requesthandle -> request handle done p120,l20 IN -> INOUT done [to prevent any misunderstanding, I make key into a two-valued logical argument high] p120,l40 key -> high done p120 l43 -> IN high (logical) done p120,l47 key -> high done p121,l1 INTEGER KEY -> LOGICAL HIGH done p121,l3 All processes ... -> All processes should provide the same \mpiarg{high} value within each of the two groups. If processes in one group provided the value \mpiarg{high = false} and processes in the other group provided the value \mpiarg{high = true} then the union orders the ``low'' group before the ``high'' group. If all processes provided the same \mpiuarg{high} argument then the order of the union is arbitrary. done p121,l27 INTEGER NAME -> CHARACTER(MPI\_NAME\_SIZE) NAME I followed Jim Cownie's suggestion... [add MPI_NAME_SIZE to the list of MPI constants] p121,l35 {\em attributes} -> {\bf attributes} done p122,l11 group or communicator -> communicator done p122,l26 copy_fn, delete_fn -> void copy_fn, void delete_fn no -- these functions need to return values, so I put them as int p122,l28 INTEGER COPY_FN, DELETE_FN -> EXTERNAL COPY_FN, DELETE_FN done p122,l37 olcomm -> oldcomm done p122,l46 [replace sentence] This function is called by \mpifunc{MPI\_COMM\_FREE} and \mpifunc{MPI\_ATTR\_DELETE} to do whatever is needed to remove an attribute. done p123,l11 on a process -> on the process done p123,l11 [add a sentence] -> These references need to be explictly freed by the program, either via calls to \mpifunc{MPI\_ATTR\_DELETE} that free one attribute instance, or by calls to \mpifunc{MPI\_COMM\_FREE} that free all attribute instances associated with the freed communicator. done p123,l28 If the value ... -> If the value is already present, then the outcome is as if \mpifunc{MPI\_DELETE\_ATTR} was first called to delete the previous value (and the callback function \mpifunc{delete\_fn} was executed), and a new value was next stored. The call is erroneous if there is no key with value \mpiarg{keyval}. done p123,l39 true or false ... -> {\tt true} if an attribute value was extracted; {\tt false} if no attribute is associated with the key. done p123,l48 [add sentence] -> The call is erroneous if there is no key with value \mpiarg{keyval}. (On the other hand, the call is correct if the key value exists, but no attribute is attached on {\tt comm} for that key; in such case, the call returns {\tt found = false}.) done p123,l46 INTEGER FOUND -> LOGICAL FOUND done p124,l4 The integer ... comm -> The key value of the deleted attribute (integer) done p125,l6 if (foundflag == MPI_FOUND) -> if (foundflag) done [DELETE SUBSECTION 5.7.2. (It is still written for the model where contexts are full blown objects.)] I'm uncomfortable taking this all out -- it's a lot. ...the section was approved... I will leave it in; we can still remove it for Jan 15. p129,l43,44 [replace with] MPI_Group_excl(MPI_GROUP_WORLD,1, ranks, &grprem); /* local */ done p130,l43 MPI_UNDEFINED_RANK -> MPI_UNDEFINED done [add to example #4 send operations, so that send and receives are matched] p130,l47 [replace with] -> MPI_Irecv(buff1, count, MPI_DOUBLE, MPI_ANY_SOURCE, TAG_ARBITRARY, the_comm, request) MPI_Isend(buff2, count, MPI_DOUBLE, (me+1)%4, TAG_ARBITRARY, the_comm, request+1) done p130,l3 [insert after] -> MPI_Waitall(2, request, status) done, except you had wrong page number (I think) -- should be 131? p131,l32 libh_a -> libh_b done -- you meant line 22... p131,l40 void -> MPI_Comm done p133,l3 MPI_UNDEFINED_RANK -> MPI_UNDEFINED done p133,l6 MPI_UNDEFINED_RANK -> MPI_UNDEFINED done [all undefined integers are equal to MPI_UNDEFINED] whew. >From snir@watson.ibm.com Sat Oct 30 11:07 PDT 1993 Received: from iliamna.cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA16719; Sat, 30 Oct 93 11:07:28 -0700 Return-Path: Received: by iliamna.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.17) id ; Sat, 30 Oct 93 11:09 PDT Message-Id: Received: from YKTVMV by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 7039; Sat, 30 Oct 93 14:09:19 EDT Date: Sat, 30 Oct 93 14:07:36 EDT >From: "Marc Snir" X-Addr: (914) 945-3204 (862-3204) 28-226 IBM T.J. Watson Research Center P.O. Box 218 Yorktown Heights NY 10598 To: otto@iliamna.cse.ogi.edu Reply-To: SNIR@watson.ibm.com Subject: my last contrigubtion to this round Status: RO Reference: Attached information This is for chapter 7. I did not go again over chapter 6 (I don't have a printout with the binding. I can be reached home 914 762 9614 for last minute emergencies. *************** Referenced Info *************** p152,l27 [add] -> integer done p152,l28 [add] -> integer done p152,l47 INTEGER NAME -> CHARACTER(*) NAME done p153,l13 [add] -> (integer) done p154,l3 l26 l28 l30 totalnumsednd -> totalnumsend done p154, l4 l26 l29 l31 p155,l19 l43 totalsizesen -> totalsizesend done p154,l46 totalnumsend-k -> totalnumsend=k done p154,l23 l38 node -> process done p155,l26 and calling process -> at calling process done p156,l2 INTEGER FLAG -> LOGICAL FLAG done p156,l3 const -> flag done p156,l48 SENDBUF -> BUFFER Someone has messed this up, I think. Damn it. I don't have time to fix this... p157,l48 MPI_WORLD -> MPI_COMM_WORLD done p158,l15 mode -> *mode done p158,l38 MPI_Errhandler -> void done [all user supplied functions have been of type void. I see no reason for erro handlers to be any different] p158,l39 INTEGER HANDLER -> EXTERNAL HANDLER done p158,l43 typedef int -> typedef void done p159,l17 INTEGER STRING -> CHARACTER(*) STRING done p160,l4 [add sentence] -> The user must ensure that all pending communications involving a process complete before the process calls \mpifunc{MPI\_FINALIZE}. done p160,l13 INTEGER FLAG -> LOGICAL FLAG done whew. >From snir@watson.ibm.com Mon Nov 1 14:30 PST 1993 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA22252; Mon, 1 Nov 93 14:29:58 -0800 Return-Path: Received: by ogicse.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.4) id ; Mon, 1 Nov 93 14:31 PST Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 7979; Mon, 01 Nov 93 17:31:37 EST Received: from YKTVMH by watson.vnet.ibm.com with "VAGENT.V1.0" id 2906; Mon, 1 Nov 1993 17:31:35 EST Received: from snir.watson.ibm.com by yktvmh.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 01 Nov 93 17:31:35 EST Received: by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA36367; Mon, 1 Nov 1993 17:31:34 -0500 >From: snir@watson.ibm.com (Marc Snir) Message-Id: <9311012231.AA36367@snir.watson.ibm.com> To: otto@cse.ogi.edu Date: Mon, 01 Nov 93 17:31:34 -0500 Status: RO ------- Forwarded Message Received: from yktvmv.watson.ibm.com by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA31981; Thu, 21 Oct 1993 11:16:27 -0400 Received: from watson.vnet.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9425; Thu, 21 Oct 93 11:16:37 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id for snir@watson; Thu, 21 Oct 93 11:16:37 -0400 Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 7211; Thu, 21 Oct 1993 11:14:27 EDT Received: from bubba.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Thu, 21 Oct 93 11:14:26 EDT Received: by bubba.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA31308; Thu, 21 Oct 1993 11:14:14 -0400 Message-Id: <9310211514.AA31308@bubba.watson.ibm.com> To: snir@yktvmv.vnet.ibm.com Cc: pratap@yktvmv.vnet.ibm.com, girwin@yktvmv.vnet.ibm.com Date: Thu, 21 Oct 93 11:14:13 -0500 >From: "Hubertus Franke" Some suggestions for MPI: I propose to make in MPI_WAIT, MPI_TEST , ..... all handle parameters to be references that is IN/OUT The doucmentation requires that the handles are set to NULL after completion and that the internal objects are deallocated. This is not reflected in the parameter status. so the prototype would be MPI_WAIT(MPI_HANDLE *hd, MPI_STATUS *stat) Also that allows on implementation for FORTRAN and C ..... The nice side effect is that upon return the handle in the user space is set to NULL and errors are much easier recognizable. Please let me know what the final decision is so that the test programs can be corrected accordingly - -- Hubertus ------- End of Forwarded Message I have not done anything about this. >From ho@almaden.ibm.com Fri Oct 29 11:14 PDT 1993 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (16.8/16.2) id AA11185; Fri, 29 Oct 93 11:14:04 -0700 Return-Path: Received: by ogicse.cse.ogi.edu (/\==/\ Smail3.1.25.1 #25.4) id ; Fri, 29 Oct 93 11:15 PDT Received: from ALMADEN by almaden.ibm.com (IBM VM SMTP V2R2) with BSMTP id 4138; Fri, 29 Oct 93 11:15:52 PDT Received: by ALMADEN (XAGENTA 3.0) id 0814; Fri, 29 Oct 1993 11:15:51 -0700 Received: by ho.almaden.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA25229; Fri, 29 Oct 1993 11:15:33 -0700 >From: (Howard Ho) Message-Id: <9310291815.AA25229@ho.almaden.ibm.com> To: otto@cse.ogi.edu Subject: MPI comments. Date: Fri, 29 Oct 93 11:15:30 -0800 Status: RO Comments from 10/27/93 version. -- Howard ========================================== p6,l22-26: "a argument" ==> "an argument" (3 places). done p7,l43: "there there" done p10,l44: "each execute" ==> "each executes" changed p17,l38: "less then or equal to" ==> "less than or equal to" done p18,l18: "where used" ==> "were used" done p19: It may be nice to add after MPI_GET_COUNT a sentence like this "Although the datatype argument seems redundant here, it is needed when the status is returned from MPI_PROBE described later" I'm not sure about this... p44,l39: "beeing" ==> "being" done p45,l44: add a parenthesis for "disp_j + sizeof (type_j)" to be explicit as the term subject to \max_j. done p51,l22: "Let type have ..." ==> "Let type1 have ..." done p51,l36: "array_of_indices" ==> "array_of_displacements" done p51,l35 till end of this page: Change all n_i to n_{i}-1 throughout except the one at line 37 with a summation term. Also, change "n_{{\rm count}-1}" (which mostly appear as subscripts) to "n_{{\rm count}-1}-1". At any rate, this will be very ugly to read. I'm afraid you're right. p52,l30: "1009" ==> "909" yes. done. p53,l36: "a new datatype that" ==> "a new datatype {\bf type1} that" left it p53,l37: I suggest drop "(from -3 to 5)" as it is somewhat misleading and confusing. left it p53,l46-48: What if $lb > \min_j disp_j$ for some $j$? Is it an error? no, it is not an error, and the definition holds as written. The user can "override" the normal type extent by using UB and LB if he/she chooses to do so. p54,l8 :"is is" done thank god you have no coll.tex changes. I'm tired! In Chapter 5: replace all occurrences "MPI_group" by "MPI_Group". There are many of them. Do the same thing in Appendix A.5. done p113,l28: I suggest to add a sentence like "It is not required to free all groups before MPI_FINALIZE." (I suppose this is true.) left it -- Tony and other Communicator hackers can mull this over p116,l29: "It is the user responsibility to free each copy of a communicator object." I suppose you meant if the user likes to free all copies of communicators (that were generated by MPI_COMM_DUP), then the user has to free each copy one by one. Note that I suppose the user is not required to free all communicators he/she creates before calling MPI_FINALIZE? left it -- Tony and other Communicator hackers can mull this over p120,l40: missing ")". done p121,l14: bad Latex format. done p122,l26: missing "int" before each of the first two arguments. done p128,l41-45: I thought Jim Cownie commented on this already? As it is, the code simply does not work when size is odd. A simple fix looks something like: if ((size % 2) != 0 && me == size-1) exit; elseif ((me % 2) == 0) MPI_Send (..., me + 1, ...); else MPI_Recv (..., me - 1, ...); .... I put in an even simpler fix. p151,l20-30: "i--1" or "j--1" (4 places). done p154,l26: "*totalnumsednd" ==> "*totalnumsend". Also, "*totalsizesen" ==> missind "*totalsizesend". (Same thing needs to be fixed in Appendix.) done p155,l5-7: many "old" argument names forgot to be changed to the new ones here. left it. Bill can fix. p155,l43: missing "d" in the last argument name. (Same thing is the Appendix.) done p156,l25: missing a ",". done P170: 2 Latex references errors in the Section Column. done P181-182, A.6: "MPI_comm" ==> "MPI_Comm" (many places). done p192: one obvious indexing error there. done From owner-mpi-core@CS.UTK.EDU Tue Nov 2 18:42:09 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA18976; Tue, 2 Nov 93 18:42:09 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA23725; Tue, 2 Nov 93 18:41:46 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 2 Nov 1993 18:41:44 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from DASHER.CS.UTK.EDU by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA23719; Tue, 2 Nov 93 18:41:43 -0500 From: Jack Dongarra Received: by dasher.cs.utk.edu (5.61+IDA+UTK-930922/2.7c-UTK) id AA16000; Tue, 2 Nov 93 18:41:25 -0500 Date: Tue, 2 Nov 93 18:41:25 -0500 Message-Id: <9311022341.AA16000@dasher.cs.utk.edu> To: mpi-core@cs.utk.edu Subject: new draft The draft of MPI for SC93 is out in netlib, it is under: draft1102.ps and draft1102-tar.Z in the mpi directory. You can retrieve an up-to-date copy of the postscript by typing from any machine on the internet: rcp anon@netlib2.cs.utk.edu:mpi/draft1102.ps draft1102.ps or the compressed tar file by typing: rcp anon@netlib2.cs.utk.edu:mpi/draft1102-tar.Z draft1102-tar.Z Regards, Jack From owner-mpi-core@CS.UTK.EDU Thu Nov 4 13:06:37 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA04618; Thu, 4 Nov 93 13:06:37 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA19306; Thu, 4 Nov 93 09:27:01 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 4 Nov 1993 09:27:00 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA19275; Thu, 4 Nov 93 09:26:51 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA07758 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Thu, 4 Nov 1993 14:26:49 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA23059; Thu, 4 Nov 1993 14:26:26 +0000 Date: Thu, 4 Nov 1993 14:26:26 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9311041426.AA23059@tycho.co.uk> To: knighten@SSD.intel.com Cc: mpi-core@cs.utk.edu In-Reply-To: <9311040130.AA12491@tualatin.SSD.intel.com> (message from Bob Knighten on Wed, 3 Nov 93 17:30:14 PST) Subject: Re: Address for comments Content-Length: 1452 > There is an omission in the DRAFT that we should rectify when we distribute > copies. The address for comments is not mentioned. For e-mail I believe it > is mpi-forum@cs.utk.edu. Is there also an address for non-electronic mail -- > there are people who do not have e-mail access! I agree, we need to make clear how to comment. Who is going to field the e-mail ? (David ?) HPF tried to get people to submit separate comments per chapter with a separate e-mail address for each. This would perhaps save some admin... To get over this omission I suggest we have a single sheet of errata, to go with the document. This could contain 1) The instructions for feedback 2) The fixes to the Fortran binding for the routines on page 111 3) The instructions for getting the document off the net. (Some people will then just take the errata sheet, and grab the document electronically). 4) Anything else we notice real soon... Meiko hasn't yet gone to press, so we would bind this in. For those who already have, a single sheet add on shouldn't be too hard. p.s. What are we supposed to do with the 400 copies ? Hand them out on the stand, bring them to the seminar ? ... James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Thu Nov 4 13:06:49 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA04633; Thu, 4 Nov 93 13:06:49 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA18637; Thu, 4 Nov 93 09:19:26 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 4 Nov 1993 09:19:19 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA18617; Thu, 4 Nov 93 09:18:39 -0500 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA28662 (5.65c8/IDA-1.4.4 for ); Thu, 4 Nov 1993 15:16:38 +0100 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA17134; Thu, 4 Nov 1993 15:17:27 GMT Date: Thu, 4 Nov 1993 15:17:27 GMT From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9311041517.AA17134@f1neuman.gmd.de> To: mpi-core@cs.utk.edu Subject: meeting in Europe Cc: gmap10@f1neuman.gmd.de, karls@pallas-gmbh.de Dear All, a few days ago I suggested to extend the feedback period until end of February. Some of you responded positively, whereas Marc Snir was not so happy because he would like to have the definitions finished as soon as possible, in order to have a safe basis for implementation work. The current plan is to hold a meeting in Europe in Jan/Feb 94, where we discuss the feedback and work it into the document. As I understand our discussions at the last Dallas meeting, the European meeting would have to be followed by a U.S. meeting, anyway. One reason is that it is very unlikely that one meeting will be sufficient. Also, some of the U.S. participants will not get funding for going to Europe, but they should have the chance to vote at the last (and decisive) meeting. So, the final MPI meeting could not take place before early March. What about the following compromise: We extend the deadline for feedback until end of February, but hold the European meeting in January. The purpose of this meeting would be twofold: On one day we could repeat the presentations of SC'93 and invite all interested people from Europe. A second (and third?) day could then be an MPI working meeting where we discuss the feedback gathered so far. Since most of the feedback is expected to come from regular MPI participants, we should have most of that part available by then. The final MPI meeting could then still be held in the States around early March. The advantage of this plan is that the Europeans get a chance to give their input to MPI after they have learned the details at the meeting. Currently we have a discussion within a European project (PPPE) which will have a meeting on Jan 18/19, anyway. Since this project funds the participation of three European MPI participants (Jim, Ian, and me), it would make sense to combine that with our MPI meeting. Also, this combination would ensure a minimum attendence of interested applications people. The PPPE meeting is scheduled to take place at Sophia Antipolis near Niece. It is not clear at the moment whether the local organizators could handle a much larger number of attendees, but we would try. By the way: Niece is not too far away from Val d'Isere. As an alternative, the meeting could be held in the Cologne/Bonn area. What do you think? Rolf From owner-mpi-core@CS.UTK.EDU Thu Nov 4 13:06:57 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA04658; Thu, 4 Nov 93 13:06:57 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA20869; Wed, 3 Nov 93 20:30:21 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 3 Nov 1993 20:30:19 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from tualatin.ssd.intel.com by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA20855; Wed, 3 Nov 93 20:30:17 -0500 Received: by tualatin.SSD.intel.com (4.1/SMI-4.1) id AA12491; Wed, 3 Nov 93 17:30:14 PST Date: Wed, 3 Nov 93 17:30:14 PST Message-Id: <9311040130.AA12491@tualatin.SSD.intel.com> From: Bob Knighten Sender: knighten@SSD.intel.com To: mpi-core@cs.utk.edu Subject: Address for comments Reply-To: knighten@SSD.intel.com (Bob Knighten) There is an omission in the DRAFT that we should rectify when we distribute copies. The address for comments is not mentioned. For e-mail I believe it is mpi-forum@cs.utk.edu. Is there also an address for non-electronic mail -- there are people who do not have e-mail access! -- Bob Robert L. Knighten | knighten@ssd.intel.com Intel Supercomputer Systems Division | 15201 N.W. Greenbrier Pkwy. | (503) 629-4315 Beaverton, Oregon 97006 | (503) 629-9147 [FAX] From owner-mpi-core@CS.UTK.EDU Thu Nov 4 17:13:07 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA08931; Thu, 4 Nov 93 17:13:07 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA28369; Thu, 4 Nov 93 17:12:33 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 4 Nov 1993 17:12:32 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA28354; Thu, 4 Nov 93 17:12:22 -0500 Received: by merckx.cse.ogi.edu (16.8/16.2) id AA16078; Thu, 4 Nov 93 14:10:27 -0800 Date: Thu, 4 Nov 93 14:10:27 -0800 From: Steve Otto Message-Id: <9311042210.AA16078@merckx.cse.ogi.edu> To: knighten@SSD.intel.com, owner-mpi-core@CS.UTK.EDU Subject: Re: Address for comments Cc: mpi-core@CS.UTK.EDU I agree, we need to make clear how to comment. Who is going to field the e-mail ? (David ?) I volunteer, but we will set up an abstract name for the email; I suggest mpi-comments@... HPF tried to get people to submit separate comments per chapter with a separate e-mail address for each. This would perhaps save some admin... Yes, I will try and lift the statement from HPF To get over this omission I suggest we have a single sheet of errata, to go with the document. This could contain 1) The instructions for feedback 2) The fixes to the Fortran binding for the routines on page 111 3) The instructions for getting the document off the net. (Some people will then just take the errata sheet, and grab the document electronically). 4) Anything else we notice real soon... Yes, this is good. I will try and make one up in the next few days (is this soon enough?) or if you already have one, Jim, that would be fine by me! Meiko hasn't yet gone to press, so we would bind this in. For those who already have, a single sheet add on shouldn't be too hard. Most everyone else *has* gone to press however, so we must do it as an errata. Sorry for these glitches (the email address is serious) but the editing team were concentrating on technical issues... p.s. What are we supposed to do with the 400 copies ? Hand them out on the stand, bring them to the seminar ? ... I'm not sure.. --Steve ------------------------------------------------------------------------ Dr Steve Otto | Dept of Computer Science and Engineering *___________ | Oregon Graduate Institute of Science & Technology _____) | 20000 NW Walker Rd, PO Box 91000 / | Portland, OR 97291-1000 / | email: otto@cse.ogi.edu /o | Phone: (503) 690-1486 | FAX: (503) 690-1553 ------------------------------------------------------------------------ From owner-mpi-core@CS.UTK.EDU Thu Nov 4 19:59:35 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA10018; Thu, 4 Nov 93 19:59:35 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA09278; Thu, 4 Nov 93 19:59:15 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 4 Nov 1993 19:59:14 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA09269; Thu, 4 Nov 93 19:59:11 -0500 Received: by merckx.cse.ogi.edu (16.8/16.2) id AA18030; Thu, 4 Nov 93 16:57:24 -0800 Date: Thu, 4 Nov 93 16:57:24 -0800 From: Steve Otto Message-Id: <9311050057.AA18030@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: errata sheet so far Here is what I have for an errata sheet so far -- I'm open to suggestions. Have we reached closure yet on our final draft date? --Steve %!PS-Adobe-2.0 %%Creator: dvips 5.516 Copyright 1986, 1993 Radical Eye Software %%Title: errata.dvi %%CreationDate: Thu Nov 4 16:54:49 1993 %%Pages: 1 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips errata %DVIPSSource: TeX output 1993.11.04:1654 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (/ogi/staff/otto/papers/Editor/MPI/draft7/errata.dvi) @start /Fa 27 118 df<007000F001E003C007800F001E001C00380038007000700070 007000E000E000E000E000E000E000E000E0007000700070007000380038001C001E000F 00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001 C001C000E000E000E000E00070007000700070007000700070007000E000E000E000E001 C001C0038007800F001E003C007800F00060000C247C9F18>I<1C3E7E7F3F1F070E1E7C F860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC783006 06778518>I<07F8001FFE003FFF007807807803C07801C03001C00001C00003C0000380 000F0003FF0003FE0003FF000007800003C00001C00000E00000E00000E0F000E0F000E0 F001C0F003C07C07803FFF001FFE0003F800131C7E9B18>51 D<007C0001FE0007FF000F 87801E03C03C1DC0387FC070FFE071E3E071C1E0E1C1E0E380E0E380E0E380E0E380E0E3 80E0E380E0E1C1C071C1C071E3C070FF80387F003C1C001E00E00F83E007FFC001FF8000 7E00131C7E9B18>64 D<00700000F80000F80000D80000D80001DC0001DC0001DC00018C 00038E00038E00038E00038E000306000707000707000707000707000FFF800FFF800FFF 800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I69 D<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001 C07001C0700000E00000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001 C07003C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>71 D<7FFF00FFFF807FFF0001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C0007FFF00FFFF807FFF00111C7D9B18>73 D<7E07F0FF0FF87F07F01D81C01D81C0 1D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C0 1C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F 9B18>78 D<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C 03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C 039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>82 D<03F3801FFF803FFF807C0F 80700780E00380E00380E00380E000007000007800003F00001FF00007FE0000FF00000F 800003C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F8 00131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E0703800700000700000 700000700000700000700000700000700000700000700000700000700000700000700000 700000700000700000700007FF0007FF0007FF00151C7F9B18>I<01FE0007FF001FFF80 3E0780380300700000700000E00000E00000E00000E00000E00000E000007000007001C0 3801C03E03C01FFF8007FF0001FC0012147D9318>99 D<001F80003F80001F8000038000 038000038000038000038003E3800FFB801FFF803C1F80380F80700780700380E00380E0 0380E00380E00380E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F015 1C7E9B18>I<01F00007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0 FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318> I<03800007C00007C00007C0000380000000000000000000000000007FC000FFC0007FC0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C000FFFF00FFFF80FFFF00111D7C9C18>105 D107 D<7CE0E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C 1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C 1C001C1C1C007F1F1F00FFBFBF807F1F1F001914819318>109 D<7E3E00FEFF807FFFC0 0FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803 807001C07001C0E000E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F 801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E 00700E00380E00380E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00E FF800E3E000E00000E00000E00000E00000E00000E00000E00007FC000FFE0007FC00015 1E809318>I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC00 03FE00001F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318> 115 D<0180000380000380000380000380007FFFC0FFFFC0FFFFC0038000038000038000 0380000380000380000380000380000380000380400380E00380E00380E001C1C001FFC0 00FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FC FC1714809318>I E /Fb 42 122 df<003F0000E0C001C0C00381E00701E00701E00700 00070000070000070000070000070000FFFFE00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E07FC3FE1720809F19>12 D<70F8FCFC74040404080810102040060E7C840D>44 DI<70F8F8F87005057C840D>I<03F0000E1C001C0E0018060038 0700700380700380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C0F003C0F003C07003807003807003807807803807001806001C 0E000E1C0003F000121F7E9D17>48 D<018003800F80F380038003800380038003800380 038003800380038003800380038003800380038003800380038003800380038003800380 07C0FFFE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F8 03C0F803C02007C00007C0000780000780000F00000E00001C0000380000700000600000 C0000180000300000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I< 000600000600000E00000E00001E00002E00002E00004E00008E00008E00010E00020E00 020E00040E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E00000E00 000E00000E00000E00000E00000E0000FFE0141E7F9D17>52 D<03F0000E18001C0C0038 0600380700700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007C070 07C03807C0180BC00E13C003E3C0000380000380000380000700300700780600780E0070 0C002018001070000FC000121F7E9D17>57 D<70F8F8F8700000000000000000000070F8 F8F87005147C930D>I<0FC0307040384038E03CF03CF03C603C0038007000E000C00180 0180010003000200020002000200020002000000000000000000000007000F800F800F80 07000E207D9F15>63 D<000FC040007030C001C009C0038005C0070003C00E0001C01E00 00C01C0000C03C0000C07C0000407C00004078000040F8000000F8000000F8000000F800 0000F8000000F8000000F8000000F8000000F8000000780000007C0000407C0000403C00 00401C0000401E0000800E000080070001000380020001C0040000703800000FC0001A21 7D9F21>67 D70 D73 D<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00203C 00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>I77 D<001F800000F0F00001C0380007801E000F000F000E 0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F8 0001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C 0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F00000 1F80001C217D9F23>79 DI<07E0 800C1980100780300380600180600180E00180E00080E00080E00080F00000F000007800 007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C08001C08001 C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19>83 D86 D89 D<1FE000303000781800781C 00300E00000E00000E00000E0000FE00078E001E0E00380E00780E00F00E10F00E10F00E 10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E3E000EC3800F01C00F00E00E00E0 0E00700E00700E00780E00780E00780E00780E00780E00780E00700E00700E00E00F00E0 0D01C00CC300083E0015207F9F19>I<03F80E0C1C1E381E380C70007000F000F000F000 F000F000F00070007000380138011C020E0C03F010147E9314>I<000380003F80000380 00038000038000038000038000038000038000038000038000038003E380061B801C0780 380380380380700380700380F00380F00380F00380F00380F00380F00380700380700380 3803803807801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E003807003807 00700700700380F00380F00380FFFF80F00000F00000F000007000007000003800801800 800C010007060001F80011147F9314>I<007C00C6018F038F0706070007000700070007 0007000700FFF00700070007000700070007000700070007000700070007000700070007 000700070007007FF01020809F0E>I<0000E003E3300E3C301C1C30380E00780F00780F 00780F00780F00780F00380E001C1C001E380033E0002000002000003000003000003FFE 001FFF800FFFC03001E0600070C00030C00030C00030C000306000603000C01C038003FC 00141F7F9417>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E 00000E00000E00000E3E000E43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F 19>I<1C001E003E001E001C000000000000000000000000000E007E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A1F809E0C>I< 0E00FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>108 D<0E1F01F000FE618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I< 0E3E00FE43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E 001C03803801C03801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000 E07000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F 00E00E00E00E00F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E 00E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E 0000FFE000151D7F9319>I<0E78FE8C0F1E0F1E0F0C0E000E000E000E000E000E000E00 0E000E000E000E000E000E000E00FFE00F147F9312>114 D<1F9030704030C010C010C0 10E00078007F803FE00FF00070803880188018C018C018E030D0608F800D147E9312>I< 020002000200060006000E000E003E00FFF80E000E000E000E000E000E000E000E000E00 0E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E03C00603C0030DC001F1FC16147F9319>III121 D E /Fc 15 122 df<001F0000001F0000 003F8000003F8000003B8000007BC0000073C0000071C00000F1E00000F1E00000E0E000 01E0F00001E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C00 0F001E000F001E000FFFFE001FFFFF001FFFFF001C000F003C0007803C00078038000780 780003C0780003C0700003C0F00001E0F00001E0E00001E01B237EA220>65 D<000FF000007FFC0000FFFF0001F01F0003C00700078000000F0000001E0000003E0000 003C0000003C000000780000007800000078000000F0000000F0000000F0000000F00000 00F0000000F0000000F0000000F0000000F00000007800000078000000780000003C0000 003C0000003E0000001E0000000F0000000780008003C0038001F00F8000FFFF00007FFC 00000FF00019257DA31F>67 D70 D89 D<07E01FF83FFC381E201E000F000F 000F000F00FF07FF1FFF3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E9517> 97 D<03F00007FC001FFE003E0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0 F00000F00000F000007000007800007800003C00801F07800FFF8007FF0001F80012167E 9516>101 D<01F07807FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C0780 1E0F001F1F000FFE001FFC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF0 7801F07800F8F00078F00078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F 9518>103 D109 DI<01FC0007FF000FFF801F07C03C01E07800F07800 F0700070F00078F00078F00078F00078F00078F000787800F07800F07C01F03E03E01F07 C00FFF8007FF0001FC0015167F9518>I114 D<07F01FFC3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC0 0FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F00 0F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E0 0E1C7F9B12>II121 D E /Fd 7 117 df69 D73 D77 D80 D<01FC00000E0780001001C0003C00E0003E00F0003E00 78001C00780008007800000078000000780000007800007FF80003E078000F8078001F00 78003E0078007C00780078007820F8007820F8007820F8007820F800F8207C00F8203C01 3C401F063FC007F80F001B1A7E991E>97 D<0787C0FF98E0FF91F00FA1F007C1F007C0E0 07C000078000078000078000078000078000078000078000078000078000078000078000 07800007800007800007800007800007C000FFFE00FFFE00141A7F9917>114 D<0080000080000080000080000180000180000180000380000380000780000F80001FFF 80FFFF800780000780000780000780000780000780000780000780000780000780000780 0007800007800007804007804007804007804007804007804007804003C08001C08000E1 00003E0012257FA417>116 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 821 45 a Fd(MPI)19 b(Errata)75 167 y Fc(Y)l(our)d(Comments)75 253 y Fb(MPIF)i(in)o(vites)h(commen)o(ts)e(on)h(the)g(tec)o(hnical)i (con)o(ten)o(t)d(of)h(MPI,)g(as)f(w)o(ell)i(as)f(on)g(the)g(editorial)i (pre-)75 309 y(sen)o(tation)15 b(in)h(the)g(do)q(cumen)o(t.)k(Commen)o (ts)14 b(receiv)o(ed)j(b)q(efore)f(Jan)o(uary)f(??,)g(1994)g(will)i(b)q (e)f(considered)75 366 y(in)i(pro)q(ducing)h(the)f(\014nal)g(draft)f (of)g(V)l(ersion)h(1.0)f(of)g(the)g(Message)g(P)o(assing)h(In)o (terface)f(Sp)q(eci\014cation.)75 422 y(Please)g(send)g(commen)o(ts)e (on)i(MPI)f(to)g Fa(mpi-comments@cs.utk.edu)o Fb(.)k(Y)l(our)d(commen)o (t)f(will)h(b)q(e)g(for-)75 478 y(w)o(arded)e(to)g(MPIF)f(committee)i (mem)o(b)q(ers)f(who)g(will)i(attempt)d(to)g(resp)q(ond.)75 600 y Fc(F)o(o)o(rtran)h(Arra)o(y)h(Arguments)75 686 y Fb(On)g(pages)f(111)f(and)h(112,)f(the)i(F)l(ortran)e(argumen)o(t)g Fa(RANGES)g Fb(should)j(b)q(e)f(declared)g(as:)266 792 y Fa(INTEGER)47 b(RANGES\(3,N\))964 2828 y Fb(1)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Fri Nov 5 04:48:25 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA14309; Fri, 5 Nov 93 04:48:25 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA18086; Fri, 5 Nov 93 04:47:54 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 5 Nov 1993 04:47:53 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA18078; Fri, 5 Nov 93 04:47:41 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA10547 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Fri, 5 Nov 1993 09:47:40 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA23604; Fri, 5 Nov 1993 09:47:14 +0000 Date: Fri, 5 Nov 1993 09:47:14 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9311050947.AA23604@tycho.co.uk> To: otto@merckx.cse.ogi.edu Cc: mpi-core@CS.UTK.EDU In-Reply-To: <9311050057.AA18030@merckx.cse.ogi.edu> (message from Steve Otto on Thu, 4 Nov 93 16:57:24 -0800) Subject: Re: errata sheet so far Content-Length: 1666 I'd extend this a bit, somethnig like ... +++++++++++ 1) Please send comments on mpi to mpi-comments@cs.utk.edu . To ease our administrative load we woudl be grateful if you could send separate mail messages for comments on each chapter, and make the Subject: line of each message have the form "Chapter nn" Comment by e-mail is the preferred route, however for those of you who cannot achive this comments may be sent to :- $Suitable snail Mail Address$ 2) Additional copies of the MPI report can be obtained from netlib, under: draft1102.ps and draft1102-tar.Z in the mpi directory. You can retrieve an up-to-date copy of the postscript by typing from any machine on the internet: rcp anon@netlib2.cs.utk.edu:mpi/draft1102.ps draft1102.ps or the compressed tar file by typing: rcp anon@netlib2.cs.utk.edu:mpi/draft1102-tar.Z draft1102-tar.Z or by using anonymous ftp to netlib2.cs.utk.edu and getting the same files. (Don't forget to use binary mode if copying the compressed file !) 3) If you want to participate in the detailed MPI discussions, then $Include how to join MPI info here $ +++++++++++ HPF also had some words about "All comments become the property of Rice University", maybe we need something like that too. I'm not up to date with copyright and IPR paranoia, do we really risk being sued if we include someone's re-wording in the document without this ? James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Fri Nov 5 08:43:04 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA17910; Fri, 5 Nov 93 08:43:04 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA07042; Fri, 5 Nov 93 08:42:32 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 5 Nov 1993 08:42:31 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA07031; Fri, 5 Nov 93 08:42:28 -0500 Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA05168; Fri, 5 Nov 93 08:42:24 EST Received: by b125.super.org (4.1/SMI-4.1) id AA01826; Fri, 5 Nov 93 08:42:24 EST Date: Fri, 5 Nov 93 08:42:24 EST From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9311051342.AA01826@b125.super.org> To: mpi-core@CS.UTK.EDU In-Reply-To: <9311050947.AA23604@tycho.co.uk> (jim@meiko.co.uk) Subject: Re: errata sheet so far As someone who sent in a comment to HPF, I would like to have some statement as to what a submitter can expect back from MPIF. Originally, HPF said each person would get back an individual comment. When too many comments arrived it was changed so that you did not. I would like the following (and have the document say this): 1) Have the reflector generate an automatic response so that the sender knows that it was received. (HPF did this) 2) An anticipated date at which those who made comments will be able to get a new draft. (HPF also did this though they ran late) If we are going to have two more meeting will we be releasing another draft before the last meeting? If not, we should quickly release the minutes of the first meeting so those who made comments that were address can know how they were received. I don't think we should promise a personal response to each person. You get off the wall ones and it will take lots of time. 3) I think we ought to tell people to reference the document by page#/line # so we can clearly tell what they are talking about. Steve From owner-mpi-core@CS.UTK.EDU Sat Nov 6 04:14:25 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with SMTP (5.61+IDA+UTK-930125/2.8t-netlib) id AA23835; Sat, 6 Nov 93 04:14:25 -0500 Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA21885; Sat, 6 Nov 93 04:13:47 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 6 Nov 1993 04:13:46 EST Errors-To: owner-mpi-core@CS.UTK.EDU Received: from THUD.CS.UTK.EDU by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930922/2.8s-UTK) id AA21879; Sat, 6 Nov 93 04:13:45 -0500 From: Jack Dongarra Received: by thud.cs.utk.edu (5.61+IDA+UTK-930922/2.7c-UTK) id AA28802; Sat, 6 Nov 93 04:13:40 -0500 Date: Sat, 6 Nov 93 04:13:40 -0500 Message-Id: <9311060913.AA28802@thud.cs.utk.edu> To: mpi-core@cs.utk.edu Subject: reference for MPI Here is the `official' reference for the MPI report: Document for a Standard Message-Passing Interface, Message Passing Interface Forum, University of Tennessee, Technical Report No. CS-93-214, November, 1993. Jack From owner-mpi-core@CS.UTK.EDU Tue Nov 9 14:05:45 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id OAA17022; Tue, 9 Nov 1993 14:05:45 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id TAA03712; Tue, 9 Nov 1993 19:05:14 GMT X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 9 Nov 1993 19:05:12 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.epm.ornl.gov by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id OAA03695; Tue, 9 Nov 1993 14:05:11 -0500 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA17150; Tue, 9 Nov 1993 14:05:19 -0500 Message-Id: <9311091905.AA17150@rios2.epm.ornl.gov> To: Steve Otto Cc: mpi-core@CS.UTK.EDU Subject: Re: comment period date; MPI In-Reply-To: (Your message of Tue, 09 Nov 93 10:25:30 PST.) <9311091825.AA02752@merckx.cse.ogi.edu> Date: Tue, 09 Nov 93 14:05:19 -0500 From: "David W. Walker" Steve, I would suggest the following compromise; Jan 18-19 Meeting in Europe Feb 11 Deadline for comments Feb 23-25 Meeting in United States Dsvid From owner-mpi-core@CS.UTK.EDU Wed Nov 10 10:00:58 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id KAA23825; Wed, 10 Nov 1993 10:00:56 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id PAA04228; Wed, 10 Nov 1993 15:00:10 GMT X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 10 Nov 1993 15:00:08 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.epm.ornl.gov by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id KAA04210; Wed, 10 Nov 1993 10:00:04 -0500 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA18076; Wed, 10 Nov 1993 10:00:15 -0500 Message-Id: <9311101500.AA18076@rios2.epm.ornl.gov> To: Steve Otto , dongarra@rios2.epm.ornl.gov Cc: mpi-core@CS.UTK.EDU Subject: Re: MPI comment period In-Reply-To: (Your message of Tue, 09 Nov 93 21:42:41 PST.) <9311100542.AA05414@merckx.cse.ogi.edu> Date: Wed, 10 Nov 93 10:00:15 -0500 From: "David W. Walker" Steve, Jack, Jack, has the mpi-comments@cs.utk.edu address been set up yet. This mail should be forwarded to me walker@msr.epm.ornl.gov. I think it would be useful if the questions and responses were made available via anonymous ftp. Thus, the mechanism is as follows: 1. Questions are sent to mpi-comments and forwarded to me, 2. I send the question to the appropriate person for response 3. That person sends a reply to mpi-responses@cs.utk.edu and to the person who sent in the question. The reply to mpi-responses is forwarded to me. The response should include the original question with each line of the original question starting with ">". 4. I will make the response available by anonymous ftp at UTK, or perhaps it should be on netlib. I will maintain an index so that people can quickly find out what sort of questions have been asked. Maybe there should also be a FAQ. Can netlib allow you to view the index and/or FAQ without having to download the corresponding files to a local machine? David From owner-mpi-core@CS.UTK.EDU Wed Nov 10 13:13:39 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id NAA25199; Wed, 10 Nov 1993 13:13:36 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id SAA21409; Wed, 10 Nov 1993 18:13:18 GMT X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 10 Nov 1993 18:13:17 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id NAA21402; Wed, 10 Nov 1993 13:13:10 -0500 Received: by merckx.cse.ogi.edu (16.8/16.2) id AA07055; Wed, 10 Nov 93 10:11:25 -0800 Date: Wed, 10 Nov 93 10:11:25 -0800 From: Steve Otto Message-Id: <9311101811.AA07055@merckx.cse.ogi.edu> To: snir@watson.ibm.com, prp@ssd.intel.com, jim@meiko.co.uk, peterm@ncube.com, moose@think.com, joelw@mozart.convex.com, tony@cs.msstate.edu Subject: Errata Sheet; Here it is. Cc: knighten@ssd.intel.com, mpi-core@CS.UTK.EDU Dear Vendors, Here is the promised Supplement/Errata sheet in postscript. It managed to fit on one page. Please include this sheet with your copies of the draft; I will bring some copies of this myself and of course we can make more at the conference. --Steve Otto %!PS-Adobe-2.0 %%Creator: dvips 5.516 Copyright 1986, 1993 Radical Eye Software %%Title: errata.dvi %%CreationDate: Wed Nov 10 10:06:39 1993 %%Pages: 1 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips errata %DVIPSSource: TeX output 1993.11.10:1006 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (/ogi/staff/otto/papers/Editor/MPI/draft7/errata.dvi) @start /Fa 38 122 df<787878781830306060E0050A7D830D>44 DII<000100030003000600060006 000C000C000C00180018001800300030003000600060006000C000C000C0018001800180 0300030003000600060006000C000C000C00180018001800300030003000600060006000 C000C000C000102D7DA117>I<03F00007F8001FFE001E1E003C0F003807007807807807 80700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003 C0F003C0F003C0F003C07807807807807807803807003C0F001E1E001FFE000FFC0003F0 0012207E9E17>I<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F 7C9E17>I<07F0000FFC001FFE00383F00700F00600780E00780E003C04003C04003C000 03C00003C00003C0000780000780000F00000E00001C00003C0000780000E00001C00003 80000700000E00001C0000380000700000FFFFC0FFFFC0FFFFC0121F7E9E17>I<03F000 0FFC001FFE003C1F00780F00300780200780000780000780000780000F00000F00003E00 03FC0003F80003FC00001E00000F000007800003800003C00003C00003C00003C08003C0 C003C0C00780700F807C1F003FFE000FFC0003F00012207E9E17>I<007C0001FE0003FE 000782000F00001E00003C00003C000078000078000078F800F3FC00F7FE00FE1F00FC0F 80F80780F80780F003C0F003C0F003C0F003C0F003C07003C07803C07803C07807803807 803C0F001E1E000FFE0007F80001F00012207E9E17>54 DI<03F0000FFC001FFE003E1F003C0F007807807807807807 807807807807807807803C0F001E1E000FFC0007F8000FFC001F3E003C0F007807807807 80F003C0F003C0F003C0F003C0F003C0F003C07807807C0F803E1F001FFE000FFC0003F0 0012207E9E17>I66 D68 D76 DII<003F000000FFC00003FFF00007 E1F8000F807C001F003E001E001E003C000F003C000F00780007807800078078000780F0 0003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C07800078078000780780007803C000F003C000F001E001E001F003E000F807C0007 E1F80003FFF00000FFC000003F00001A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C00007800007800 007800007800007800007C00003C00003F00001FE0000FFC0007FE0001FF00003F800007 C00003C00003E00001E00001E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF 001FFE0003F80013227EA019>II87 D<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7F FF3FEF1F8F10147E9316>97 DI<00078000078000078000078000078000078000078000078000078000 078000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F0 0780F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F 17>100 D<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F0 0000F00000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I< 03E0F00FFFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E00 1FFC003FF80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0 F000F0F801F07E07E03FFFC00FFF0003FC00141E7F9317>103 D105 D107 DI110 D<01F80007FE001FFF803F0FC03C03C07801E078 01E0F000F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007 FE0001F80014147F9317>I114 D<1E001E001E001E001E001E 00FFF0FFF0FFF01E001E001E001E001E001E001E001E001E001E001E001E001E001E201F F00FF007C00C1A7F9910>116 DII<7801E07C03C03E07801E0F000F0F0007 9E0003FC0003F80001F80000F00001F00001F80003FC00079E000F0F000E0F001E07803C 03C07801E0F801F01414809315>120 DI E /Fb 40 119 df<007000F001E003C007800F001E001C00380038007000700070 007000E000E000E000E000E000E000E000E0007000700070007000380038001C001E000F 00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001 C001C000E000E000E000E00070007000700070007000700070007000E000E000E000E001 C001C0038007800F001E003C007800F00060000C247C9F18>I<1C3E7E7F3F1F070E1E7C F860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC783006 06778518>I<000300000780000780000F80000F00001F00001E00001E00003E00003C00 007C0000780000780000F80000F00001F00001E00003E00003C00003C00007C000078000 0F80000F00000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000 F0000060000011247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001 C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001 C07001C07803C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>I<01800380 038007800F803F80FF80FB80438003800380038003800380038003800380038003800380 038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03F0000FFE003FFF007C0F8070 03C0E001C0F000E0F000E06000E00000E00000E00001C00001C00003C0000780000F0000 1E00003C0000780000F00001E00007C0000F80001E00E03C00E07FFFE0FFFFE07FFFE013 1C7E9B18>I<07F8001FFE003FFF007807807803C07801C03001C00001C00003C0000380 000F0003FF0003FE0003FF000007800003C00001C00000E00000E00000E0F000E0F000E0 F001C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<3078FCFC78300000000000 0000003078FCFC78300614779318>58 D<007C0001FE0007FF000F87801E03C03C1DC038 7FC070FFE071E3E071C1E0E1C1E0E380E0E380E0E380E0E380E0E380E0E380E0E1C1C071 C1C071E3C070FF80387F003C1C001E00E00F83E007FFC001FF80007E00131C7E9B18>64 D<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E00038E 00038E000306000707000707000707000707000FFF800FFF800FFF800E03800E03801C01 C01C01C07F07F0FF8FF87F07F0151C7F9B18>I69 D<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001C0700000E000 00E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001C07003C03803C03803 C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>71 D<7FFF00FFFF807FFF0001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF807FFF00 111C7D9B18>73 D<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01C E1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C 1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>78 D<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F 801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C039C1C03 9C1C039C7F01F8FF81F87F00F0161C7F9B18>82 D<03F3801FFF803FFF807C0F80700780 E00380E00380E00380E000007000007800003F00001FF00007FE0000FF00000F800003C0 0001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E 9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E070380070000070000070000070 000070000070000070000070000070000070000070000070000070000070000070000070 0000700000700007FF0007FF0007FF00151C7F9B18>I<3FFFE07FFFE07FFFE07001C070 03C0700780700700000F00001E00001C00003C0000780000700000F00001E00001C00003 C0000780000700000F00001E00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE013 1C7E9B18>90 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF 007F0700780700E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318 >97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC0 0FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F0070 0F80E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E07803803 00700000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03 C01FFF8007FF0001FC0012147D9318>I<001F80003F80001F8000038000038000038000 038000038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E0 0380E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I< 01F00007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0 E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007F C000FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF 007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E0038070038070038 07003807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03F FFE07801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC0015 1F7F9318>I<03800007C00007C00007C0000380000000000000000000000000007FC000 FFC0007FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>105 D107 D<7FE000FFE0007FE00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C 7E9B18>I<7CE0E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C 007F1F1F00FFBFBF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07F C3FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0 E000E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE00 01F00013147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00 380E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00 000E00000E00000E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<7F 87E0FF9FF07FBFF803F87803F03003E00003C00003C00003800003800003800003800003 80000380000380000380000380007FFE00FFFF007FFE0015147F9318>114 D<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE00001F 00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<01800003 80000380000380000380007FFFC0FFFFC0FFFFC003800003800003800003800003800003 80000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E0013 197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318> I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E00038E 00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>I E /Fc 29 125 df<001F0000001F0000003F8000003F8000003B8000007BC0000073C000 0071C00000F1E00000F1E00000E0E00001E0F00001E0F00001C0F00003C0780003C07800 0380780007803C0007803C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF00 1C000F003C0007803C00078038000780780003C0780003C0700003C0F00001E0F00001E0 E00001E01B237EA220>65 D<000FF000007FFC0000FFFF0001F01F0003C0070007800000 0F0000001E0000003E0000003C0000003C000000780000007800000078000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000078000000 78000000780000003C0000003C0000003E0000001E0000000F0000000780008003C00380 01F00F8000FFFF00007FFC00000FF00019257DA31F>67 DIII73 D<000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000FC01EF03E FFFC7FF80FE010247EA217>I77 D80 D82 D89 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF00FF0 0FF83F7FFF3FEF1F8F10167E9517>97 D<01FC0007FF000FFF801F03803C018078000078 0000700000F00000F00000F00000F00000F00000F000007800007800007800003C00401F 03C00FFFC007FF8001FC0012167E9516>99 D<03F00007FC001FFE003E0F003C07807803 80780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F000007000007800007800003C00 801F07800FFF8007FF0001F80012167E9516>101 D<003F00FF01FF03C0038007800780 078007800780078007800780FFF8FFF8FFF8078007800780078007800780078007800780 0780078007800780078007800780078007800780102380A20F>I<01F07807FFF80FFFF8 1F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F000FFE001FFC0019F000 3800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8F00078F00078F00078 F000787800F03E03E01FFFC00FFF8001FC0015217F9518>III< F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F004 237DA20B>108 DII<01FC0007FF000FFF801F07C03C01E07800F07800F0700070F000 78F00078F00078F00078F00078F000787800F07800F07C01F03E03E01F07C00FFF8007FF 0001FC0015167F9518>I114 D<07F01FFC3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC0 0FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F00 0F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E0 0E1C7F9B12>III121 D124 D E /Fd 49 123 df<003F0000E0C001C0C00381E00701E00701 E0070000070000070000070000070000070000FFFFE00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E07FC3FE1720809F19>12 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<70F8FCFC74040404080810102040060E7C840D>44 DI<70F8 F8F87005057C840D>I<000100030003000600060006000C000C000C0018001800180030 0030003000600060006000C000C000C00180018001800300030003000600060006000C00 0C000C00180018001800300030003000600060006000C000C000C000102D7DA117>I<03 F0000E1C001C0E00180600380700700380700380700380700380F003C0F003C0F003C0F0 03C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C070038070038070 03807807803807001806001C0E000E1C0003F000121F7E9D17>I<018003800F80F38003 800380038003800380038003800380038003800380038003800380038003800380038003 800380038003800380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E002007004007 80800780F007C0F803C0F803C0F803C02007C00007C0000780000780000F00000E00001C 0000380000700000600000C0000180000300000600400C00401800401000803FFF807FFF 80FFFF80121E7E9D17>I<03F0000C1C00100E00200F00780F80780780780780380F8000 0F80000F00000F00000E00001C0000380003F000003C00000E00000F0000078000078000 07C02007C0F807C0F807C0F807C0F00780400780400F00200E001C3C0003F000121F7E9D 17>I<000600000600000E00000E00001E00002E00002E00004E00008E00008E00010E00 020E00020E00040E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E00 000E00000E00000E00000E00000E00000E0000FFE0141E7F9D17>I<03F0000E18001C0C 00380600380700700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007 C07007C03807C0180BC00E13C003E3C0000380000380000380000700300700780600780E 00700C002018001070000FC000121F7E9D17>57 D<70F8F8F87000000000000000000000 70F8F8F87005147C930D>I<000FC040007030C001C009C0038005C0070003C00E0001C0 1E0000C01C0000C03C0000C07C0000407C00004078000040F8000000F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F8000000780000007C0000407C000040 3C0000401C0000401E0000800E000080070001000380020001C0040000703800000FC000 1A217D9F21>67 DI70 D73 D77 DI<001F800000F0F00001C03800 07801E000F000F000E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0 F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0 780001E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E00 01C0380000F0F000001F80001C217D9F23>II<07E0800C1980100780300380600180600180E00180E00080E00080E00080 F00000F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C0 0001C08001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D 9F19>83 D86 DI89 D<1FE000303000781800781C00300E00000E00000E00000E0000FE0007 8E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E93 17>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00 780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03 F80E0C1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E 0C03F010147E9314>I<000380003F800003800003800003800003800003800003800003 8000038000038000038003E380061B801C0780380380380380700380700380F00380F003 80F00380F00380F00380F003807003807003803803803807801C07800E1B8003E3F81520 7E9F19>I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F0 0000F00000F000007000007000003800801800800C010007060001F80011147F9314>I< 007C00C6018F038F07060700070007000700070007000700FFF007000700070007000700 07000700070007000700070007000700070007000700070007007FF01020809F0E>I<00 00E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E 380033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C0 0030C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E8180 0F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000 00000000000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000 0000000000007007F000F000700070007000700070007000700070007000700070007000 70007000700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E00 00FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0F F00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E 000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE6186 18000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E 000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E8180 0F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801 C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801 C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E 00700E00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00E C3000E3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F93 19>I<0E78FE8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E000E 000E000E00FFE00F147F9312>114 D<1F9030704030C010C010C010E00078007F803FE0 0FF00070803880188018C018C018E030D0608F800D147E9312>I<020002000200060006 000E000E003E00FFF80E000E000E000E000E000E000E000E000E000E000E000E080E080E 080E080E080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0 030DC001F1FC16147F9319>III121 D<3FFF380E200E201C40384078407000E001E001C00380078007 010E011E011C0338027006700EFFFE10147F9314>I E /Fe 14 118 df68 D73 D77 D80 D<00FE010003FF83000F81E3001E0037003C001F0038000F007800070070000700F00003 00F0000300F0000300F0000100F8000100F8000100FC0000007C0000007F0000003FE000 001FFF00000FFFE00007FFF80003FFFC00007FFE000007FF0000007F0000001F8000000F 80000007C0000007C0800003C0800003C0800003C0800003C0C00003C0C0000380C00003 80E0000780F0000700F8000E00EE001C00C3C07800C1FFF000803FC0001A2B7DA921>83 D<007F8001C0700780080F003C1E007C3C007C3C00387C0010780000F80000F80000F800 00F80000F80000F80000F80000F800007800007C00003C00043C00041E00080F00100780 2001C0C0007F00161A7E991B>99 D<007E0003C3800700E00E00F01C00703C00783C0038 78003C78003CF8003CF8003CFFFFFCF80000F80000F80000F80000F800007800007C0000 3C00043C00041E00080E001007002001C0C0007F00161A7E991B>101 D<0780FF80FF800F80078007800780078007800780078007800780078007800780078007 800780078007800780078007800780078007800780078007800780078007800780078007 800780078007800780FFFCFFFC0E2A7FA911>108 D<0781F800FC00FF860E030700FF98 070C03800FA0079003C007A003D001E007C003E001E007C003E001E0078003C001E00780 03C001E0078003C001E0078003C001E0078003C001E0078003C001E0078003C001E00780 03C001E0078003C001E0078003C001E0078003C001E0078003C001E0078003C001E00780 03C001E0078003C001E0078003C001E0078003C001E0FFFC7FFE3FFFFFFC7FFE3FFF301A 7F9933>I<0783F800FF8C1C00FF900E000FA0070007A0078007C0078007C00780078007 800780078007800780078007800780078007800780078007800780078007800780078007 8007800780078007800780078007800780078007800780078007800780FFFCFFFCFFFCFF FC1E1A7F9921>I<007F000001C1C000070070000E0038001C001C003C001E003C001E00 78000F0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80 F8000F8078000F0078000F003C001E003C001E001E003C000E0038000700700001C1C000 007F0000191A7E991E>I<0783F000FF8C1C00FFB00F0007A0078007C003C0078003E007 8001E0078001F0078001F0078000F8078000F8078000F8078000F8078000F8078000F807 8000F8078000F0078001F0078001F0078001E0078003C007C003C007C0078007A00E0007 983C000787E0000780000007800000078000000780000007800000078000000780000007 8000000780000007800000FFFC0000FFFC00001D267F9921>I<00800000800000800000 80000180000180000180000380000380000780000F80001FFF80FFFF8007800007800007 800007800007800007800007800007800007800007800007800007800007800007804007 804007804007804007804007804007804003C08001C08000E100003E0012257FA417> 116 D<07800780FF80FF80FF80FF800F800F800780078007800780078007800780078007 800780078007800780078007800780078007800780078007800780078007800780078007 800780078007800780078007800F8007800F800380178001C027C000E047FC003F87FC1E 1A7F9921>I E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 572 45 a Fe(Supplemen)n(t)19 b(to)i(MPI)d(Do)r(cumen)n(t)786 102 y Fd(No)o(v)o(em)o(b)q(er)c(10,)h(1993)75 223 y Fc(Y)l(our)h (Comments)75 309 y Fd(MPIF)d(in)o(vites)i(commen)o(ts)f(on)f(the)h(tec) o(hnical)i(con)o(ten)o(t)d(of)g(MPI,)h(as)f(w)o(ell)i(as)e(on)h(the)g (editorial)h(presen-)75 366 y(tation)j(in)h(the)f(do)q(cumen)o(t.)29 b(Commen)o(ts)17 b(receiv)o(ed)j(b)q(efore)e(F)l(ebruary)g(11,)g(1994)f (will)j(b)q(e)f(considered)75 422 y(in)f(pro)q(ducing)h(the)f(\014nal)g (draft)f(of)g(V)l(ersion)h(1.0)f(of)g(the)g(Message)g(P)o(assing)h(In)o (terface)f(Sp)q(eci\014cation.)75 478 y(Please)g(send)g(commen)o(ts)e (on)i(MPI)f(to)g Fb(mpi-comments@cs.utk.edu)o Fd(.)k(Y)l(our)d(commen)o (t)f(will)h(b)q(e)g(for-)75 535 y(w)o(arded)e(to)f(MPIF)h(committee)g (mem)o(b)q(ers)h(who)f(will)h(attempt)f(to)f(resp)q(ond.)21 b(Commen)o(t)14 b(b)o(y)h(e-mail)h(is)75 591 y(the)d(preferred)h (route,)f(ho)o(w)o(ev)o(er)f(for)h(those)g(of)g(y)o(ou)g(who)g(cannot)g (ac)o(hiv)o(e)g(this)h(commen)o(ts)f(ma)o(y)f(b)q(e)i(sen)o(t)75 648 y(to)189 742 y Fa(David)h(W.)f(W)o(alk)o(er)189 798 y(Oak)h(Ridge)h(National)f(Lab)q(o)o(rato)o(ry)189 855 y(Building)h(6012/MS-6367)189 911 y(P)l(.)f(O.)g(Bo)o(x)g(2008)189 967 y(Oak)g(Ridge,)h(TN)f(37831-6367)166 1061 y Fd(When)e(making)f (commen)o(ts,)h(please)g(reference)g(the)g(do)q(cumen)o(t)g(b)o(y)f (page)g(n)o(um)o(b)q(er)h(/)f(line)j(n)o(um)o(b)q(er)75 1118 y(so)g(as)g(to)f(minimize)k(p)q(ossible)f(confusion.)75 1239 y Fc(Electronic)f(Retrieval)h(of)f(Do)q(cument)75 1325 y Fd(Copies)22 b(of)f(the)h(MPI)f(draft)g(do)q(cumen)o(t)h(can)f (b)q(e)h(obtained)h(from)d(netlib,)25 b(under:)33 b Fb(draft1102.ps)75 1382 y Fd(and)20 b Fb(draft1102-tar.Z)d Fd(in)k(the)e Fb(mpi)g Fd(directory)l(.)34 b(Y)l(ou)20 b(can)f(retriev)o(e)h(an)g (up-to-date)f(cop)o(y)h(of)f(the)75 1438 y(p)q(ostscript)c(b)o(y)h(t)o (yping)f(from)f(an)o(y)h(mac)o(hine)h(on)g(the)f(in)o(ternet:)194 1532 y Fb(rcp)24 b(anon@netlib2.cs.utk.edu:)o(mpi/draf)o(t1102.p)o(s)d (draft1102.ps)75 1626 y Fd(or)15 b(the)g(compressed)g(tar)g(\014le)h(b) o(y)f(t)o(yping:)194 1720 y Fb(rcp)24 b(anon@netlib2.cs.utk.edu:)o (mpi/draf)o(t1102-t)o(ar.Z)d(draft1102-tar.Z)75 1813 y Fd(or)12 b(b)o(y)g(using)h(anon)o(ymous)f(ftp)g(to)g Fb(netlib2.cs.utk.edu)e Fd(and)j(getting)f(the)h(same)f(\014les.)20 b(Don't)11 b(forget)75 1870 y(to)k(use)g(binary)h(mo)q(de)f(if)h(cop)o (ying)g(the)f(compressed)h(\014le.)75 1992 y Fc(Joining)i(the)d(MPI)i (F)o(o)o(rum)75 2077 y Fd(Information)10 b(on)g(joining)h(the)g(MPI)f (F)l(orum)f(can)i(b)q(e)f(obtained)h(b)o(y)f(sending)i(email)f(to)e Fb(netlib@ornl.gov)75 2134 y Fd(with)16 b(a)e(message)h(b)q(o)q(dy)h (of:)75 2228 y Fb(send)23 b(mpi-info)g(from)g(mpi)75 2349 y Fc(Erro)o(r)15 b(|)h(F)o(o)o(rtran)f(Arra)o(y)h(Arguments)75 2435 y Fd(On)g(pages)f(111)f(and)h(112,)f(the)i(F)l(ortran)e(argumen)o (t)g Fb(RANGES)g Fd(should)j(b)q(e)f(declared)g(as:)266 2541 y Fb(INTEGER)47 b(RANGES\(3,N\))p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Thu Nov 11 12:56:00 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id MAA04398; Thu, 11 Nov 1993 12:55:59 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id RAA14113; Thu, 11 Nov 1993 17:55:25 GMT X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 11 Nov 1993 17:55:23 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id MAA14092; Thu, 11 Nov 1993 12:55:19 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA02428 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Thu, 11 Nov 1993 17:55:26 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA28932; Thu, 11 Nov 1993 17:54:46 +0000 Date: Thu, 11 Nov 1993 17:54:46 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9311111754.AA28932@tycho.co.uk> To: mpi-core@CS.UTK.EDU Subject: [chk@cs.rice.edu: HPFF Photo] Content-Length: 975 This sounds like a neat idea of HPFF, should we rip it off too ? p.s. what happened about T shirts ? -- Jim > Return-Path: > Date: Thu, 11 Nov 93 11:27:43 CST > From: chk@cs.rice.edu (Charles Koelbel) > To: hpff-core@erato.cs.rice.edu > Subject: HPFF Photo > > HPFF Committee Members - > > We would like to get a publicity shot of as many of the HPF > designers as possible, for dressing up reports (and future grant > applications, etc.). Since most of you will be at Supercomputing '93 > next week, that seems like a good opportunity. If you attended any of > the HPFF technical meetings and aren't camera-shy, meet on Concourse A > of the Oregon Convention Center (under the big pendulum) at 7:30pm > Tuesday, November 16 (i.e. right after the poster session). We'll > keep the picture-taking short so you can get to dinner/hospitality > suites/whatever. Dress is informal -- HPF t-shirts are OK, but not > mandatory. > > Chuck Koelbel From owner-mpi-core@CS.UTK.EDU Thu Nov 11 13:28:32 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id NAA04565; Thu, 11 Nov 1993 13:28:32 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id SAA17190; Thu, 11 Nov 1993 18:28:13 GMT X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 11 Nov 1993 18:28:11 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id NAA17172; Thu, 11 Nov 1993 13:28:10 -0500 From: Jack Dongarra Received: by dasher.cs.utk.edu (5.61+IDA+UTK-930922/2.7c-UTK) id AA00625; Thu, 11 Nov 93 13:27:30 -0500 Date: Thu, 11 Nov 93 13:27:30 -0500 Message-Id: <9311111827.AA00625@dasher.cs.utk.edu> To: jim@meiko.co.uk, mpi-core@CS.UTK.EDU Subject: Re: [chk@cs.rice.edu: HPFF Photo] In-Reply-To: Mail from 'jim@meiko.co.uk (James Cownie)' dated: Thu, 11 Nov 1993 17:54:46 +0000 The T-shirts should appear at SC'93. Jack From owner-mpi-core@CS.UTK.EDU Fri Nov 12 01:36:59 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id BAA09572; Fri, 12 Nov 1993 01:36:58 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id GAA11190; Fri, 12 Nov 1993 06:36:21 GMT X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 12 Nov 1993 06:36:20 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id BAA11182; Fri, 12 Nov 1993 01:36:08 -0500 Received: from donner.mcs.anl.gov by antares.mcs.anl.gov with SMTP id AA09690 (5.65c/IDA-1.4.4 for ); Fri, 12 Nov 1993 00:36:22 -0600 Message-Id: <199311120636.AA09690@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: MPI implementation status Cc: frankeh@watson.ibm.com Date: Fri, 12 Nov 1993 00:36:15 -0600 From: Rusty Lusk MPI Implementation News The initial public, portable implementation of MPI, scheduled for release at SuperComputing '93, is just going to make it, subject to lots of reservations. The purpose of this note is to fill you in on the current state of the implementation effort. This has become a joint-effort project between Argonne (Bill Gropp and Rusty Lusk) and Mississippi State (Tony Skjellum and Nathan Doss). Hubertus Franke of IBM has also made major contributions. Of course the hardest work was getting the spec right, which was done by the MPI Forum as a whole. We are very much in the middle of the implementation effort, and wish we had another couple of weeks before having to give this out, but now is the time, given the publicity that will be generated at SuperComputing '93 and the MPI workshop. We plan to tell everyone how to ftp the implementation at SC'93. The implementation at this point contains the C library for lots of the point-to-point chapter, most of the collective and context chapters, the topology chapter, and a small amount of the environment chapter. A small number of Fortran-callable versions of the library functions have been generated. It does not have the fancy datatypes yet. It contains several test programs, but has not been thoroughly tested. One major application at Argonne has been ported to this version of MPI, and other applications have been ported at IBM. The implementation is not complete, robust, or particularly efficient. On the other hand, enough of it is there that applications can begin to be ported to it, and it is becoming more complete, robust, and efficient every day. Things are moving rapidly. Details follow below. Our implementation is based on the abstract device interface that we talked about at the MPI meetings several times. MPI is implemented on top of the device interface, and we have implemented the device interface on top of Chameleon, Bill Gropp's lightweight portability package. It in turn uses vendor libraries on individual parallel machines, and either p4 or PVM on networks of workstations. Our testing so far has been with the p4 version; we hope to have the PVM version, and the Intel nX version, working by Saturday. As of right now (Thursday night, Nov 11) the implementation has been tested only on networks of Suns, networks of RS/6000's, and the IBM SP-1. Maybe we (or others) can hack the CM-5, Meiko, and nCube versions during the Conference. No obstacles are foreseen, given that Chameleon and p4 already run on those machines. The purpose of this initial implementation is to advance the cause of MPI by: 1. demonstrating the implementability of MPI and contributing to shaking down the specifications. 2. providing an avenue for applications to begin the porting process very early. 3. providing a shortcut path for vendor implementations. Vendors need provide only the device part of the implementation. Longer-term goals are to provide a portable MPI implementation for heterogeneous networks of workstations and to motivate research into device abstractions for high-performance message-passing. The current implementation has the following limitations, as of November 11, 1993. 1. All the code written so far is written for clarity and simplicity rather than efficiency. 2. The code is not thoroughly or systematically tested. 3. Only elementary datatypes are supported. 4. Messages longer than 100K are not yet supported. 5. Synchronous and Ready modes are not yet supported. 6. Assorted functions are still missing. New functions are added every few days. 7. Only a very few of the functions have Fortran-callable versions. On the other hand, here are a few good aspects of the current state: 1. Enough functions have been implemented both to port existing codes and to experiment with the new ideas in MPI. 2. The C code has been prepared for automatic generation of Fortran wrappers, so the Fortran-callable versions should appear quickly. 3. The Chameleon-p4 (and soon, Chameleon-PVM) implementation is portable to virtually any parallel machine and to heterogeneous networks of workstations. The implementation is not yet highly optimized but it is not naive. 4. A major application (a nuclear-structure code from Argonne's Physics Division) has been ported to MPI and runs on the IBM SP-1. So that's where we stand. I hope to see many of you in Portland. Encourage people you see to come to the MPI Workshop on Friday morning. I have inside information that it will be great! Regards, Rusty (and Bill and Tony and Nathan) From owner-mpi-core@CS.UTK.EDU Thu Nov 25 08:03:56 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id IAA15827; Thu, 25 Nov 1993 08:03:56 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id IAA02686; Thu, 25 Nov 1993 08:03:13 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 25 Nov 1993 08:03:12 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id IAA02678; Thu, 25 Nov 1993 08:03:06 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA12549 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Thu, 25 Nov 1993 13:02:45 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA01815; Thu, 25 Nov 1993 13:01:32 +0000 Date: Thu, 25 Nov 1993 13:01:32 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9311251301.AA01815@tycho.co.uk> To: mpi-core@CS.UTK.EDU Subject: HPCWire Content-Length: 1345 > Globalization of HPC Beckons as Export Barriers Fade Nov 24 > Commentary by Norris Parker Smith, Editor at Large HPCwire > ============================================================================= ... > Unfortunately, Kahaner's extremely useful session -- which also included > informative presentations on supercomputing in Singapore, Norway, and the > European Economic Community -- was scheduled on Friday, the final rump of the > Portland schedule, and was sparsely attended. > The Friday timing was, however, not a sufficient factor. Downstairs, > several hundred persons attended a workshop on MPI, the new message passing > interface that has undergone a rapid but extensive process of development and > is now available for public review and comment. > MPI is certainly an important topic, because it could give a substantial > boost to distributed computing -- one of the keys to the future of parallel > processing. The MPI workshop was an illuminating report from one forefront of > high-performance computing. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Fri Nov 26 03:43:46 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id DAA19779; Fri, 26 Nov 1993 03:43:46 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id DAA08116; Fri, 26 Nov 1993 03:43:17 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 26 Nov 1993 03:43:16 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id DAA08109; Fri, 26 Nov 1993 03:43:03 -0500 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA07124 (5.65c8/IDA-1.4.4 for ); Fri, 26 Nov 1993 09:41:15 +0100 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA16850; Fri, 26 Nov 1993 09:42:52 GMT Date: Fri, 26 Nov 1993 09:42:52 GMT From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9311260942.AA16850@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: MPI meeting in Europe Cc: gmap10@f1neuman.gmd.de, karls@pallas-gmbh.de, loriot@simpolis.fr, owen@meiko.co.uk Dear Collegue, at the Supercomputing Conference last week I already discussed with some of you our plans for a feedback meeting in Europe. Since then things have become more concrete, so we can now make definitive plans for the meeting. The general idea is to combine the MPI meeting with a project meeting of the CEC project PPPE (which is funding the MPI participation of Jim, Ian, and me) which is taking place in southern France on January 18/19. Additionally we will try to invite as many interested people from Europe as possible, given the short time frame for preparations. The meeting will be organized by INRIA, Pallas GmbH, and GMD. We are especially grateful that INRIA offers to do the local organization which includes the reservation of meeting rooms. We propose the following tentative schedule: Jan 17 afternoon: Re-run of SC93 MPI presentations Jan 17 evening: Social event with room for technical discussions Jan 18 morning: Comments of European code developers Jan 18 afternoon: MPI committee meeting Jan 19 morning: Continuation of MPI committee meeting In parallel to the MPI committee meetings the PPPE project will hold its meetings. We thought it would be a good idea to start the meeting on Monday at noon. This gives the Europeans a chance to arrive that same day, whereas the U.S. participants can spend some holidays in France before the MPI meeting. After all, the meeting place is as pleasent as you can expect in Europe in January. The location will be a place near Nice (at the Cote d'Azur), close to INRIA Sophia Antipolis. There is an airport at Nice, but you will probably have to go through Paris if you come from the States. It is not yet clear what transportation from the airport will be provided, and which hotel will be available for the meeting, but we will find this out soon. We will probably have to charge a fee for the provision of meeting rooms, refreshments, etc., which will be in the range of the $ 75,- we had to pay for the previous MPI meetings. The exact amount has not been fixed yet, and it will depend on the cost estimates of INRIA. I need from you as soon as possible your confirmation that you plan to attend the European workshop (or a short note that you can't). We need to get this information at latest by Dec. 10, but it would be good if you could send it before that. Please send me your confirmation by email to hempel@gmd.de. Also, please tell me if the above meeting schedule is okay with you, or if you have any other suggestions. As soon as I have more details for the meeting, I'll let you know. Best regards, Rolf From owner-mpi-core@CS.UTK.EDU Sat Nov 27 08:36:06 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id IAA29250; Sat, 27 Nov 1993 08:36:06 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id IAA07045; Sat, 27 Nov 1993 08:35:45 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 27 Nov 1993 08:35:43 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from thud.cs.utk.edu by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id IAA07033; Sat, 27 Nov 1993 08:35:43 -0500 From: Jack Dongarra Received: by thud.cs.utk.edu (5.61+IDA+UTK-930922/2.7c-UTK) id AA17826; Sat, 27 Nov 93 08:35:36 -0500 Date: Sat, 27 Nov 93 08:35:36 -0500 Message-Id: <9311271335.AA17826@thud.cs.utk.edu> To: mpi-core@CS.UTK.EDU Subject: T-shirts The MPI T-shirts are in. If you didn't get one at the Supercomputer 93 Meeting send me your mailing address and I will send you one. Jack From owner-mpi-core@CS.UTK.EDU Mon Nov 29 14:18:03 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id OAA15672; Mon, 29 Nov 1993 14:18:03 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id OAA12543; Mon, 29 Nov 1993 14:17:16 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 29 Nov 1993 14:17:14 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from zach.fit.edu by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id OAA12531; Mon, 29 Nov 1993 14:17:12 -0500 Received: by zach.fit.edu (5.61/1.34/900923wah) id AA04776; Mon, 29 Nov 93 14:15:00 -0500 Date: Mon, 29 Nov 93 14:15:00 -0500 From: howell@zach.fit.edu ( Gary Howell) Message-Id: <9311291915.AA04776@zach.fit.edu> To: dongarra@CS.UTK.EDU, mpi-core@CS.UTK.EDU Subject: Re: T-shirts Please send me an MPI T-shirt at Dept. of Applied Math, 150 W. university Blvd. Florida Institute of Technology Melbourne, FL 32901 Gary Howell From owner-mpi-core@CS.UTK.EDU Wed Dec 1 04:17:03 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id EAA27376; Wed, 1 Dec 1993 04:17:02 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id EAA28677; Wed, 1 Dec 1993 04:16:02 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 1 Dec 1993 04:16:00 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id EAA28642; Wed, 1 Dec 1993 04:15:48 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA03330 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Wed, 1 Dec 1993 09:16:10 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA05187; Wed, 1 Dec 1993 09:14:44 +0000 Date: Wed, 1 Dec 1993 09:14:44 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9312010914.AA05187@tycho.co.uk> To: mpi-core@CS.UTK.EDU Subject: Ptools consortiun announcement Content-Length: 6086 People, You or your organization may be interested in the Parallel Tools consortium that is being formed. The following annoucement was made at the recent SC93 conference in Portland Oregon. Since the announcement does not spell it out, I will add that the initial intent of the consortium is that membership is defined by participation in projects - not by payment of fees. If you would like your name on the mailing list - send email to ptools-request@llnl.gov. This same address is to be used for requesting your name be deleted from the list. Mail to the general consortium list should be addressed to ptools@llnl.gov. My apologies if you get more than one copy of this announcement - it is being posted in multiple places. (Thanks to Mary Zosel who's text I have "borrowed" above !) -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com ----------------------------------------------------------------------- ----------------------------------------------------------------------- Lawrence Livermore National Laboratory and Oregon State University announce the formation of the PARALLEL TOOLS CONSORTIUM Several recent conferences (including the DARPA/NSF 1993 Workshop on Instrumentation for Parallel Computer Systems, the HPCC Workshop on Grand Challenge Applications and Software Technology, and the 1993 ACM/ONR Workshop on Parallel and Distributed Debugging) have exposed the fact that tool use is appallingly low among the high-performance computing community, in spite of increasingly vocal demands for software support. Three factors have been cited as critical: * Current tools do not respond to the specific needs of scientific users, who must become quasi-computer scientists to understand and use the tools. * Tools vary widely across current parallel platforms, so the steep learning curve must be repeated each time a user migrates to a new machine. * The lack of specialized support for heterogeneous or scalable applications also deters users from investing the effort needed to parallelize scientific libraries or applications that could be used by third parties. The Parallel Tools Consortium brings together representatives from the federal, industrial, and academic communities to address these issues. Our mission is to take a leadership role in the definition, development, and promotion of parallel tools that meet the specific requirements of users who develop scalable applications on a variety of platforms. We plan to do this by: 1) Encouraging active collaboration on specific tool projects among users, developers and researchers. 2) Incorporating user input into tool requirements and user feedback on tool suitability. 3) Establishing consensus among users and developers of parallel tools. 4) Promoting parallel tools that help developers create robust, high-performance, scalable applications. 5) Sponsoring parallel tool development projects that: - offer a common look and feel, published interfaces, and public-domain prototypes - leverage existing standards and technology to shorten development time and to achieve wider user acceptance - support the development of applications for a variety of parallel architectures and programming paradigms. 6) Serving the parallel computing community by encouraging the dissemination and transfer of parallel tool technology. The organizers welcome the participation of all persons and organizations interested in achieving these goals. Official membership is by organization, not by individuals, although specific individuals from member organizations will be identified as voting representatives. The first general meeting of the group will be held in spring 93. The location and date will be announced in January. To join the Parallel Tools Consortium, or to receive announcements and other information, contact Sam Coleman (scoleman@llnl.gov). Organizing Committee: Cherri M. Pancake, Oregon State University, Chair (pancake@cs.orst.edu) Sam Coleman, Lawrence Livermore National Laboratory, Secretary (scoleman@llnl.gov) Don Breazeal, Intel Corporation (donb@ssd.intel.com) Gary Brooks, CONVEX Computer Corporation (gbrooks@convex.com) Michael L. Koszykowski, Sandia National Lab (mikek@magus.ca.sandia.gov) James Cownie, Meiko Scientific Corporation (jim@meiko.com) David Culler, University of California, Berkeley (culler@cs.berkeley.edu) Daniel D. Frye, IBM Corporation (danielf@kgnvmt.vnet.ibm.com) Rusty Lusk, Argonne National Lab (lusk@mcs.anl.gov) Allen D. Malony, University of Oregon (malony@cs.uoregon.edu) Douglas M. Pase, Cray Research Inc. (pase@cray.com) Mark Seager, Lawrence Livermore National Lab (seager@llnl.gov) Steven J. Sistare, Thinking Machines Corporation (sistare@think.com) Jerry Yan, NASA Ames Research Center (jerry@ptolemy.arc.nasa.gov) Mary Zosel, Lawrence Livermore National Lab (zosel@llnl.gov) ========================================================================= Sam Coleman Lawrence Livermore National Laboratory (510) 422-4323 Mail Stop L-60 (510) 423-8715 (FAX) 7000 East Avenue scoleman@llnl.gov Livermore, CA 94550 ========================================================================= From owner-mpi-core@CS.UTK.EDU Thu Dec 2 11:20:38 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id LAA09044; Thu, 2 Dec 1993 11:20:37 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id LAA21670; Thu, 2 Dec 1993 11:19:34 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 2 Dec 1993 11:19:32 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id LAA21660; Thu, 2 Dec 1993 11:19:24 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA09299 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Thu, 2 Dec 1993 16:19:37 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA06092; Thu, 2 Dec 1993 16:18:07 +0000 Date: Thu, 2 Dec 1993 16:18:07 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9312021618.AA06092@tycho.co.uk> To: mpi-core@CS.UTK.EDU Cc: owen@meiko.co.uk Subject: [rehmann@serd.cscs.ch: Copy of official announcement of Call for Papers for IFIP10.3 94 conference] Content-Length: 5880 People, It seems like there ought to be at least one MPI paper at the IFIP conference (see below for details), and ideally more than one. Is anyone intending to submit one ? If so can you let me know so that I don't need to (which I view as a positive result !) Note that they have just extended the deadlines for submission... -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com Subject: CFP: IFIP WG10.3 conference on programming environments, Switzerland Keywords: massive parallelism, programming, tools, working conference, CFP CALL FOR PAPERS IFIP WG10.3 WORKING CONFERENCE ON PROGRAMMING ENVIRONMENTS FOR MASSIVELY PARALLEL DISTRIBUTED SYSTEMS April 25 - 30, 1994 Monte Verita, Ascona, Switzerland Massively parallel systems with distributed resources will play a very important role for the future of high performance computing. One of the current obstacles of these systems is their difficult programming. The proposed conference will bring together active researchers who are working on ways how to help programmers to exploit the performance potential of massively parallel systems. The working conference will consist of sessions for full and short papers, interleaved with poster and demonstration sessions. The Conference will be held April 25 - 30, 1994 at the Centro Stefano Franscini, located in the hills above Ascona at Lago Maggiore, in the southern part of Switzerland. It is organized by the Swiss Scientific Computing Center CSCS ETH Zurich. The conference is the forthcoming event of the working group WG 10.3 of the International Federation for Information Processing (IFIP) on Programming Environments for Parallel Computing. The conference succeeds the 1992 Edinburgh conference on Programming Environments for Parallel Computing. SUBMISSION OF PAPERS Submission of papers is invited in the following areas: -- Programming models for parallel distributed computing -- Computational models for parallel distributed computing -- Program transformation tools -- Concepts and tools for the design of parallel distributed algorithms -- Reusability in parallel distributed programming -- Concepts and tools for debugging massively parallel systems (100+ processing nodes) -- Concepts and tools for performance monitoring of massively parallel systems (100+ processing nodes) -- Tools for application development on massively parallel systems -- Support for computational scientists: what do they really need ? -- Application libraries (e.g., BLAS, etc.) for parallel distributed systems: what do they really offer ? -- Problem solving environments for parallel distributed programming Authors are invited to submit complete, original, papers reflecting their current research results. All submitted papers will be refereed for quality and originality. The program committee reserves the right to accept a submission as a long, short, or poster presentation paper. Manuscripts should be double spaced, should include an abstract, and should be limited to 5000 words (20 double spaced pages); The contact authors are requested to list e-mail addresses if available. Fax or electronic submissions will not be considered. Please submit 5 copies of the complete paper to the following address: PD Dr. Karsten M. Decker IFIP 94 CSCS-ETH Zurich Via Cantonale CH-6928 Manno Switzerland IMPORTANT DATES Deadline for submission: December 1, 1993 (extended to 17 Dec) Notification of acceptance: February 1, 1994 (extended to 15 Feb) Final versions: March 1, 1994 (extended to 15 Mar) CONFERENCE CHAIR Karsten M. Decker CSCS-ETH Zurich Via Cantonale CH-6928 Manno Switzerland phone: +41 (91) 50 8233 fax: +41 (91) 50 6711 e-mail: decker@serd.cscs.ch ORGANIZATION COMMITTEE CHAIR Rene M. Rehmann CSCS-ETH Zurich Via Cantonale CH-6928 Manno Switzerland phone: +41 (91) 50 8234 fax: +41 (91) 50 6711 e-mail: rehmann@serd.cscs.ch PROGRAM COMMITTEE Francoise Andre, IRISA, France Thomas Bemmerl, Intel Corporation, Germany Arndt Bode, Technical University Muenchen, Germany Helmar Burkhart, University Basel, Switzerland Lyndon J. Clarke, University of Edinburgh, UK Michel Cosnard, Ecole Normale Superieure de Lyon, France Karsten M. Decker, CSCS-ETH Zurich, Switzerland Thomas Fahringer, University of Vienna, Austria Claude Girault, University P. et M. Curie Paris, France Anthony J. G. Hey, University of Southhampton, UK Roland N. Ibbett, University of Edinburgh, UK Nobuhiko Koike, NEC Corporation, Japan Peter B. Ladkin, University of Stirling, UK Juerg Nievergelt, ETH Zurich, Switzerland Edwin Paalvast, TNO-TPD, The Netherlands Gerard Reijns, Delft University of Technology, The Netherlands Eugen Schenfeld, NEC Research Institute, USA Clemens-August Thole, GMD, Germany Owen Thomas, Meiko, UK Marco Vanneschi, University of Pisa, Italy Francis Wray, Cambridge, UK MONTE VERITA, ASCONA, SWITZERLAND Centro Stefano Franscini, Monte Verita, located in the scenic hills above Ascona, with a beautiful view on Lago Maggiore, has excellent conference and housing facilities for about sixty participants. Monte Verita enjoys a sub-alpine/mediterranean climate with mean temperatures between 15 and 18 C in April. The closest airport to Centro Stefano Franscini is Lugano-Agno which is connected to Zurich, Geneva and Basle and many other cities in Europe by air. Centro Stefano Franscini can also be reached conveniently by train from any of the three major airports in Switzerland to Locarno by a few hours scenic trans-alpine train ride. It can also be reached from Milano in less than three hours. From owner-mpi-core@CS.UTK.EDU Tue Dec 14 09:01:58 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id JAA02902; Tue, 14 Dec 1993 09:01:58 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id JAA02694; Tue, 14 Dec 1993 09:03:22 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 14 Dec 1993 09:03:21 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id JAA02687; Tue, 14 Dec 1993 09:03:15 -0500 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA19102 (5.65c8/IDA-1.4.4 for ); Tue, 14 Dec 1993 15:01:24 +0100 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA16931; Tue, 14 Dec 1993 15:03:09 GMT Date: Tue, 14 Dec 1993 15:03:09 GMT From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9312141503.AA16931@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: MPI Europe meeting Cc: gmap10@f1neuman.gmd.de Dear Collegue, with the time of the European MPI meeting approaching quickly, with this mail I send you more details on the meeting and a registration form. Please fill out the form as soon as possible and send it to the given email address (simoneti@sophia.inria.fr). The local organizers will handle the hotel reservations, if you return the form soon enough. Also, please send me a short note indicating whether you will be able to come to Nice or not. So far I only got response from 14 people. It looks that there are a lot of people coming to the meeting. Pallas already got some 50 registrations. It would be good if the MPI committee itself were represented by more than just a few of us. The agenda which you will find in this note is only tentative. Any suggestions for improvements are welcome. In particular, we have to discuss who is giving which talk. Since not all the speakers of the Portland workshop will be present, we have to switch some tasks. I suggest the following changes: 1. Since Jack is not coming, I would like to ask Rusty to include Jack's general introduction in his talk, which concentrated on the technical introduction at SC'93. 2. Lyndon seems to me an obvious replacement of Tony Skjellum (who will not come either). 3. I didn't hear anything from Al or Bill yet. If they don't show up in Nice, we have to get some speakers to replace them. So, Al and Bill, please drop ma a note. Any other suggestions, questions? I'm looking forward to seeing most of you at Sophia Antipolis! - Rolf ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Second Announcement E U R O P E A N M P I W O R K S H O P MPI, the new standard for message-passing programming, has been published recently. It is the aim of the European MPI Workshop to organize the dissemination of MPI in Europe and to collect the European developers' view on MPI. The MPI Workshop will bring together European software developers with experience in parallel computing, in particular message passing. Date: January 17/18, 1994 January 18/19, 1994 (MPI Committee only) Location: INRIA Sophia Antipolis (near Nice) Organized by: PALLAS, GMD, INRIA (for the ESPRIT project PPPE) Registration fee: 70 ECU or 450 FF (75 US$) (to be paid cash at registration) Accomodation: A block reservation has been made in the Hotel OMEGA (3 stars) in Sophia Antipolis (800 m from INRIA). Prices are 390 FF (65 US$) for a single room, 500 FF (83 US$) for a double room. The block reservation and prices are valid until January 5, 1994. After this time accomodation in the Hotel Omega is not guaranteed. Transport: A shuttle bus will be going from Nice airport to INRIA on Monday (17th) morning at 11.00, and from INRIA to Nice airport on Tuesday and Wednesday afternoon. Otherwise taxi transport is recommended. Bus tranfer from Hotel OMEGA to INRIA and back is provided. Registration fee: The registration fee covers refreshments, lunches, transport and the reception on Monday night. Due to the short notice no cheque or bank transfer payments can be accepted. As usual at the preceeding MPI meetings in the US, please pay the registration fee (450 FF) in cash. How to register: Please fill out the enclosed registration form and send it to Monique Simonetti, INRIA Sophia Antipolis (full address is on the enclosed registration form). PPPE project partners should also register. You will be informed if your registration could not be accepted. (Because of capacity limitations, the maximum number of participants is 80.) More information: For more information about the European MPI Workshop and to receive the MPI standard document, please contact PALLAS: mpi- ws@pallas-gmbh.de. ------------------------------------------------------------------------------ Tentative Agenda E U R O P E A N M P I W O R K S H O P Monday, January 17: 11:00 - 13:00 Registration 13:00 L U N C H 14:00 Welcome and introduction 14:30 Basic MPI 15:00 Advanced concepts in MPI message-passing 15:30 Writing libraries with MPI 16:00 Process topologies 16:30 Implementing MPI 17:00 Discussion 18:00 Outlook: MPI-2 19:30 R E C E P T I O N and B U F F E T D I N N E R Tuesday, January 18: 09:00 Requirements of message-passing code developers* 11:00 Statements of hardware and software vendors* 11:30 Migration from existing message-passing libraries to MPI* 11:45 Discussion 13:00 End of European MPI Workshop 13:00 L U N C H 14:00 MPI Committee Meeting 20:00 D I N N E R (cost not included in registration fee) * Contributions from code developers, software companies and hardware vendors are highly welcome. Please contact karls@pallas-gmbh.de. Wednesday, January 19: 09:00 MPI Committee Meeting continued 13:00 End of MPI Committee Meeting 13:00 L U N C H For PPPE partners: The PPPE meeting will start on Tuesday, January 18 at 14.00. ----------------------------------------------------------------------------- Registration Form E U R O P E A N M P I W O R K S H O P January 17-19, 1994, INRIA Sophia Antipolis Name ______________________________________________________________ Organization ______________________________________________________________ Full address ______________________________________________________________ Phone_______________ fax _______________ email _____________ Hotel reservation I book in the Hotel OMEGA, Sophia Antipolis __ single room from Jan __ until Jan __ (= _ nights) (390 FF per night) __ double room from Jan __ until Jan __ (= _ nights) (500 FF per night) I want to participate at the dinner on Tuesday, January 18 night: yes ___ no ___ Travel plans To organize transport from Nice airport to INRIA, please indicate your travel plans Arrival at Nice Airport: date/time: __________ flight from: ________________ Departure from Nice Airport: date/time: __________ flight to: __________________ Please send this form to Monique Simonetti Phone: +33-93 65 78 64 Relations Exterieurs Fax: +33-93 65 79 55 Bureau des Colloques email: simoneti@sophia.inria.fr INRIA Sophia Antipolis 2004, Route de Lucioles BP 93 F-06902 SOPHIA ANTIPOLIS CEDEX France ----------------------------------------------------------------------------- Flight Information E U R O P E A N M P I W O R K S H O P The most convenient airport to get to INRIA Sophia Antipolis is Nice. Here are some flight connections (other direct flights to these and other destinations are available) Paris (Orly) - Nice Nice - Paris (Orly) Jan 16: 17.55 - 19.15 Jan 18: 18.00 - 19.30 18.25 - 19.45 19.05 - 20.25 Jan 17: 09.20 - 10.40 Jan 19: 08.05 - 09.25 09.35 - 11.05 09.05 - 10.25 Brussels - Nice Nice - Brussels Jan 16: 14.30 - 16.10 Jan 18: 17.00 - 18.40 Jan 17: 09.35 - 11.15 Jan 19: 12.05 - 13.45 Frankfurt - Nice Nice - Frankfurt Jan 16: 16.20 - 17.55 Jan 18: 18.35 - 20.20 Jan 17: 08.30 - 10.00 Jan 19: 10.40 - 12.25 Amsterdam - Nice Nice - Amsterdam Jan 16: 13.30 - 15.20 Jan 18: 16.05 - 18.10 Jan 17: 09.05 - 10.55 Jan 19: 11.40 - 13.45 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ From owner-mpi-core@CS.UTK.EDU Tue Dec 14 09:46:52 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id JAA03085; Tue, 14 Dec 1993 09:46:52 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id JAA06521; Tue, 14 Dec 1993 09:48:35 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 14 Dec 1993 09:48:34 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id JAA06514; Tue, 14 Dec 1993 09:48:30 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA23317 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Tue, 14 Dec 1993 14:48:22 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA12688; Tue, 14 Dec 1993 14:46:27 +0000 Date: Tue, 14 Dec 1993 14:46:27 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9312141446.AA12688@tycho.co.uk> To: Rolf.Hempel@gmd.de Cc: owen@meiko.co.uk, mpi-core@CS.UTK.EDU In-Reply-To: <9312141503.AA16931@f1neuman.gmd.de> (Rolf.Hempel@gmd.de) Subject: Re: MPI Europe meeting Content-Length: 675 Rolf, I will almost certainly be at the MPI meeting (despite my broken wrist). I'm happy to stand in for someone else if speakers are required. Note that there are also flights to/from London Heathrow on BA (every day) as below LHR Nice 09-00 11-50 12-30 15-20 18-25 21-15 Nice LHR 09-00 10-05 12-50 13-45 16-20 17-20 --Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Thu Dec 16 04:58:00 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id EAA18381; Thu, 16 Dec 1993 04:57:59 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id EAA20668; Thu, 16 Dec 1993 04:59:34 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 16 Dec 1993 04:59:33 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id EAA20660; Thu, 16 Dec 1993 04:59:28 -0500 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA13480 (5.65c8/IDA-1.4.4 for ); Thu, 16 Dec 1993 10:57:37 +0100 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA16414; Thu, 16 Dec 1993 10:59:20 GMT Date: Thu, 16 Dec 1993 10:59:20 GMT From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9312161059.AA16414@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: European MPI meeting Cc: gmap10@f1neuman.gmd.de Dear Collegues, as you have seen from the second announcement, the program for the European MPI meeting is taking shape. All the local preparations seem to run smoothly, now it's up to us to make the event a success. There seems to be considerable interest in the meeting here in Europe, and we expect more than 50 participants. Since not all of you are able to come to Europe, we had to replace some of the speakers of the original MPI presentation in Portland. Here is the actual (preliminary) list of talks and speakers: 14:00 Welcome and introduction (Rusty Lusk) 14:30 Basic MPI (David Walker ?) 15:00 Advanced concepts in MPI message-passing (Marc Snir) 15:30 Writing libraries with MPI (Lyndon Clarke) 16:00 Process topologies (Rolf Hempel) 16:30 Implementing MPI (Bill Gropp/Rusty Lusk) 17:00 Discussion 18:00 Outlook: MPI-2 We don't have any names for the last two items. Perhaps it would be better to shift the outlook item before the discussion, and it could be made relatively short. Any ideas how to do the discussion? Should we do it as in Portland, with the speakers on the podium? Then we need a moderator, perhaps Steve Otto or Jim Cownie? Also I would like to remind you that I got feedback from only 17 people yet. Those of you who didn't do it yet, please tell me whether you will attend the meeting or not. Even if you can't decide yet, this information would be more than nothing. Something else we should start thinking about is the MPI committee meeting which is scheduled for the afternoon of Jan. 18 and the morning of Jan. 19. I don't see it as my task to set up an agenda for that part. Does anybody else? - Rolf Hempel From owner-mpi-core@CS.UTK.EDU Fri Dec 17 06:19:58 1993 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id GAA03705; Fri, 17 Dec 1993 06:19:58 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id GAA04573; Fri, 17 Dec 1993 06:21:17 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 17 Dec 1993 06:21:16 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id GAA04566; Fri, 17 Dec 1993 06:21:10 -0500 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA17696 (5.65c8/IDA-1.4.4 for ); Fri, 17 Dec 1993 12:19:16 +0100 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA17305; Fri, 17 Dec 1993 12:20:59 GMT Date: Fri, 17 Dec 1993 12:20:59 GMT From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9312171220.AA17305@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: MPI European meeting Cc: gmap10@f1neuman.gmd.de, karls@pallas-gmbh.de Dear Friends, here is an update on the program for the Nice meeting in January. Now that I got feedback from all speakers, we have the complete list of talks: 14:00 Welcome and introduction (Rusty Lusk) 14:30 Basic MPI (David Walker) 15:00 Advanced concepts in MPI message-passing (Marc Snir) 15:30 Writing libraries with MPI (Lyndon Clarke) 16:00 Process topologies (Rolf Hempel) 16:30 Implementing MPI (Bill Gropp/Rusty Lusk) 17:00 Outlook: MPI-2 (Leslie Hart) 17:30 Discussion (Moderation: Jim Cownie) Leslie volunteered to give the most difficult presentation on what we need to do after MPI-1, and I think it's a good idea to give that talk from an applications programmer's point of view. Of course that cannot be an official announcement of MPI-2, including dates and deadlines. I asked Jim Cownie to moderate the discussion after the talks. I hope that many people have read the standard document by then, which is a little facilitated since every participant gets a copy of it mailed. I don't know whether they mail it to us, too! I would like to remind you that you should send your registration form directly to Monique Simonetti (email: simoneti@sophia.inria.fr). The registration fee of $75 includes both the workshop and the MPI meeting. If you want to call the hotel directly, here is the data you need: OMEGA HOTEL 49 rue Ludwig Van Beethoven 06560 Valbonne Sophia Antipolis - France - Tel : (33) 92.96.07.07 Fax : (33) 92.96.03.03 So far I got feedback from 20 of you, that's only half of the core group. Please tell me whether you plan to attend or not. Best Wishes, Rolf From owner-mpi-core@CS.UTK.EDU Fri Jan 28 16:20:37 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id QAA17810; Fri, 28 Jan 1994 16:20:37 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id QAA13490; Fri, 28 Jan 1994 16:20:15 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Jan 1994 16:20:13 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.epm.ornl.gov by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id QAA13476; Fri, 28 Jan 1994 16:20:12 -0500 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA12654; Fri, 28 Jan 1994 16:20:16 -0500 Date: Fri, 28 Jan 1994 16:20:16 -0500 From: walker@rios2.epm.ornl.gov (David Walker) Message-Id: <9401282120.AA12654@rios2.epm.ornl.gov> To: mpi-core@CS.UTK.EDU Subject: Article Cc: walker@rios2.epm.ornl.gov UNICOM Seminars Ltd. of Uxbridge, England is putting ou a book on Concurrent Distributed Computing edited by Vaidy Sunderam. They would like to include the SC '93 paper on MPI in this book, and have reached an agreement with IEEE regarding the copyright. I am writing to check whether anyone involved in writing the SC 93 article has any objections to it being published in this book. Also does anyone think that (minor) alterations to the article need to be made. If I don't hear back from you by the end of next week I'll assume you have no objections and that you think the article is OK as is. Regards, David From owner-mpi-core@CS.UTK.EDU Thu Feb 3 16:52:51 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id QAA16352; Thu, 3 Feb 1994 16:52:51 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id QAA21866; Thu, 3 Feb 1994 16:52:16 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 3 Feb 1994 16:52:14 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from pnlg.pnl.gov by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id QAA21859; Thu, 3 Feb 1994 16:52:10 -0500 Received: from snacker.pnl.gov. (130.20.186.18) by pnlg.pnl.gov; Thu, 3 Feb 94 13:48 PST Received: by snacker.pnl.gov. (4.1/SMI-4.1) id AA06800; Thu, 3 Feb 94 13:48:28 PST Date: Thu, 3 Feb 94 13:48:28 PST From: Rik Littlefield Subject: MPI meeting To: mpi-context@CS.UTK.EDU, mpi-core@CS.UTK.EDU Cc: rj_littlefield@pnlg.pnl.gov Message-id: <9402032148.AA06800@snacker.pnl.gov.> X-Envelope-to: mpi-context@cs.utk.edu, mpi-core@cs.utk.edu I will not be able to attend the MPI meeting in Knoxville, due to a program review by my funding agency on the same days. I will attempt to review materials that come out beforehand, so please consider this a plea to get that stuff out early. Thanks much, --Rik ---------------------------------------------------------------------- rj_littlefield@pnl.gov (alias 'd39135') Rik Littlefield Tel: 509-375-3927 Pacific Northwest Lab, MS K1-87 Fax: 509-375-6631 P.O.Box 999, Richland, WA 99352 From owner-mpi-core@CS.UTK.EDU Thu Feb 10 16:12:17 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id QAA24641; Thu, 10 Feb 1994 16:12:17 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id QAA29425; Thu, 10 Feb 1994 16:11:33 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 10 Feb 1994 16:11:30 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.epm.ornl.gov by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id QAA29418; Thu, 10 Feb 1994 16:11:30 -0500 Received: by rios2.epm.ornl.gov (AIX 3.2/UCB 5.64/4.03) id AA19467; Thu, 10 Feb 1994 16:11:35 -0500 Message-Id: <9402102111.AA19467@rios2.epm.ornl.gov> To: mpi-core@CS.UTK.EDU Subject: user-supplied buffers Date: Thu, 10 Feb 94 16:11:34 -0500 From: "David W. Walker" I'm a bit confused by the discussion that arose at the MPI meeting in Nice about creating a lot of new point-to-point routines in which the user specifies the buffer space. This was proposed to take into account concerns expressed by Europeans about the fact that MPI does not guarantee some (large) minimum amount of buffering. Creating these new routines seems pointless, since instead of passing the buffer space into (say) a send routine for the system to use, the user could equally well copy the message into his/her scratch buffer, send it with mpi_isend, and do a wait on the send some time later. It seems that the slightly greater convenience to the user of having the system do the buffer copy is not worth the hassle of adding a load of new routines to MPI. If it argued that the system might do the copy more efficiently, perhaps we should just supply an mpi_buffer_to_buffer_copy routine - at least this adds only one routine. Regards, David From owner-mpi-core@CS.UTK.EDU Mon Feb 28 12:15:39 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id MAA10625; Mon, 28 Feb 1994 12:15:39 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id MAA00878; Mon, 28 Feb 1994 12:15:00 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 28 Feb 1994 12:14:59 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id MAA00870; Mon, 28 Feb 1994 12:14:53 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA05415; Mon, 28 Feb 94 11:13:41 CST Date: Mon, 28 Feb 94 11:13:41 CST From: Tony Skjellum Message-Id: <9402281713.AA05415@Aurora.CS.MsState.Edu> To: Rolf.Hempel@gmd.de, mpi-core@CS.UTK.EDU Subject: Clarifications of MPI_MAKE_CART Rolf, hello. We need to make the following clarification on MPI_MAKE_CART and MPI_MAKE_GRAPH. These functions are analogous to MPI_COMM_MAKE, except that they also attach the topology capability to the output communicator. The issue of caching semantics therefore arises. I propose that caching of attributes not be carried across by these functions, so they are symmetric with MPI_COMM_MAKE, and that this be explicitly stated in the text of section 6.5. I consider this to be strictly an interpretation of what we have already agreed upon, and invite your comments. -Tony PS Can you remind me why we do not have a means to attach a topology to an already existing communicator, which could be a non-communicating collective operation? This is mainly curiosity at this point. I recognize that the functionality of MPI_MAKE_GRAPH/CART is superior to that, but the functionality I mention is also without need of communication, and analogous to adding other attributes to a communicator (that is to say, user-defined topology functions could provide this type of capability, so why not the MPI-defined approach?) From owner-mpi-core@CS.UTK.EDU Thu Mar 10 22:41:16 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id WAA28408; Thu, 10 Mar 1994 22:41:16 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id WAA16286; Thu, 10 Mar 1994 22:40:49 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 10 Mar 1994 22:40:46 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id WAA16273; Thu, 10 Mar 1994 22:40:44 -0500 From: Jack Dongarra Received: by dasher.cs.utk.edu (5.61+IDA+UTK-930922/2.7c-UTK) id AA12027; Thu, 10 Mar 94 22:40:31 -0500 Date: Thu, 10 Mar 94 22:40:31 -0500 Message-Id: <9403110340.AA12027@dasher.cs.utk.edu> To: mpi-core@CS.UTK.EDU Subject: Publishing MPI MIT Press has accepted our request for a joint copyright agreement with the University of Tennnessee for the publication of the MPI Standard in the International Journal of Supercomputer Applications. We will be able to provide an electronic version of the Standard and have it appear in the Journal. We are scheduled to provide the manuscript to the Journal toward the end of April and it will appear in the Fall 94 issue of the Journal and should be available at the SC'94 meeting. Jack From owner-mpi-core@CS.UTK.EDU Fri Mar 11 04:21:48 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id EAA29309; Fri, 11 Mar 1994 04:21:47 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id EAA08896; Fri, 11 Mar 1994 04:21:43 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 11 Mar 1994 04:21:42 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id EAA08887; Fri, 11 Mar 1994 04:21:39 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA22256 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Fri, 11 Mar 1994 09:21:05 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA00913; Fri, 11 Mar 1994 09:19:50 +0000 Date: Fri, 11 Mar 1994 09:19:50 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9403110919.AA00913@tycho.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: <9403110340.AA12027@dasher.cs.utk.edu> (message from Jack Dongarra on Thu, 10 Mar 94 22:40:31 -0500) Subject: Re: Publishing MPI > We are scheduled to provide the manuscript to the Journal > toward the end of April and it will appear in the Fall 94 > issue of the Journal and should be available at the SC'94 meeting. You mean we won't hit a formal release date of April 1 ? (I'd really like to see that as the date on the cover :-)) -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Mon Mar 14 18:20:47 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (8.6.4/2.8t-netlib) id SAA22059; Mon, 14 Mar 1994 18:20:46 -0500 Received: from localhost by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id SAA22651; Mon, 14 Mar 1994 18:20:21 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 14 Mar 1994 18:20:19 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (8.6.4/2.8s-UTK) id SAA22632; Mon, 14 Mar 1994 18:20:16 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 1223; Mon, 14 Mar 94 18:20:28 EST Received: from YKTVMH by watson.vnet.ibm.com with "VAGENT.V1.0" id 2600; Mon, 14 Mar 1994 18:22:09 EST Received: from snir.watson.ibm.com by yktvmh.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 14 Mar 94 18:22:08 EST Received: by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA17170; Mon, 14 Mar 1994 18:20:23 -0500 From: snir@watson.ibm.com (Marc Snir) Message-Id: <9403142320.AA17170@snir.watson.ibm.com> To: mpi-core@CS.UTK.EDU Subject: might be interested in this insight Reply-To: snir@watson.ibm.com Date: Mon, 14 Mar 94 18:20:22 -0500 -:) -:) -:) -:) -:) Date: Thu, 10 Mar 1994 14:11:25 -0600 To: hpff-core@titan.cs.rice.edu From: chk@cs.rice.edu (Chuck Koelbel) Subject: MIL SPEC - --========================_7415460==_ Content-Type: text/plain; charset="us-ascii" Just a reminder why standards are so important to get right... - --========================_7415460==_ Content-Type: text/plain; name="MILSPEC"; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="MILSPEC" =46rom: Bill Innanen US Standard Railroad Gauge or How MilSpecs Live Forever - -------------------------- The US standard railroad gauge (distance between the rails) is 4 ft 8 1/2 in (1.44 m). That's an exceedingly odd number. Why is that gauge used? Because that's the way they built them in England, and the US railroads were built by English ex patriots. Why did the English build 'em like that? Because the first rail lines were built by the same people who built the pre-railroad tramways, and that's the gauge they used. Why did *they* use that gauge then? Because the people who built the tramways used the same jigs and tools as they used for building wagons, which used that wheel spacing. OK! Why did the wagons use that wheel spacing? Well, if they tried to use any other spacing the wagons would break on some of the old, long distance roads, because that's the spacing of the ruts. So who built these old rutted roads? The first long distance roads in Europe were built by Imperial Rome for the benefit of their legions. The roads have been used ever since. And the ruts? The initial ruts, which everyone else had to match for fear of breaking their wagons, were first made by Roman war chariots. Since the chariots were made by or for Imperial Rome they were all alike in the matter of wheel spacing (ruts again). Thus we have the answer to the original question. The United States standard railroad gauge of 4 ft 8 1/2 in derives from the original military specification (MilSpec) for an Imperial Roman army war chariot. MisSpecs (and bureaucracies) live forever! =46rom: tooch@auspex.com (Mike Tuciarone) Subject: Re: How MilSpecs Live Forever =46ascinating. I showed this to my wife, medieval studies major and horsewoman, who points out that the spacing of wheels on the Roman chariot was like as not dictated by the width of the yoke that attached the chariot to the horse, and the need to keep the wheel ruts well out of the path of the loose earth the hooves are kicking up. Thus, the gauge of the Iron Horse might be in fact derived from the width of the standard Roman warhorse. - --========================_7415460==_-- ------- End of Forwarded Message From owner-mpi-core@CS.UTK.EDU Wed Mar 30 15:24:46 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id PAA17193; Wed, 30 Mar 1994 15:24:46 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id PAA13151; Wed, 30 Mar 1994 15:24:07 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 30 Mar 1994 15:24:02 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with ESMTP (cf v2.8s-UTK) id PAA13141; Wed, 30 Mar 1994 15:24:01 -0500 From: Jack Dongarra Received: by dasher.cs.utk.edu (cf v2.9c-UTK) id PAA06106; Wed, 30 Mar 1994 15:23:21 -0500 Date: Wed, 30 Mar 1994 15:23:21 -0500 Message-Id: <199403302023.PAA06106@dasher.cs.utk.edu> To: mpi-core@CS.UTK.EDU Subject: Birds of a feather session at SC94 I have put in a request for a Birds-of-a-Feather session on MPI at the Supercompting 94 meeting in DC this November. (SC 94 dropped the workshop concept in favor of BOF.) Jack From owner-mpi-core@CS.UTK.EDU Thu Mar 31 16:35:42 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA26099; Thu, 31 Mar 1994 16:35:41 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA09287; Thu, 31 Mar 1994 16:35:14 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 31 Mar 1994 16:35:12 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA09280; Thu, 31 Mar 1994 16:35:10 -0500 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA11573; Thu, 31 Mar 94 13:35:08 -0800 Date: Thu, 31 Mar 94 13:35:08 -0800 From: Steve Otto Message-Id: <9403312135.AA11573@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Version 1.0 MPI ****************************************************************************** Version 1.0 of "MPI: A Message-Passing Interface Standard" ****************************************************************************** is available for your perusal! It is on netlib in mpi under draft-final.ps Several of us, Rusty Lusk, Bill Gropp, Steve Otto, Tony Skjellum, and of course, Marc Snir, have been working on the document for the past two weeks and we have it in a presentable form. We have some time for MPIF to look it over, comment, and catch any residual errors. We plan to hand the document over to IJSA on April 15, so send your cards and letters soon. Please send your comments and corrections to mpi-core@cs.utk.edu so that we can all rapidly see them. In any event, make sure I see it so that I can correct the "true" copy. Don't bother telling me about line "Overfulls" (text spilling beyond the margin) --- I have fixed most of these, but a few remain, usually associated with some of our long names (LaTeX doesn't break up words in some fonts), and with some of the coding examples. I will probably make some more cosmetic improvements to the document in the next week. Some (potentially) significant changes *were* made to the spec by us. Here are the ones we can remember and you may want to have a look at those sections. * Function names were changed to a consistent style (or, almost always so) The style is: MPI noun verb, or: MPI noun attribute e.g., MPI_GROUP_COMPARE (lang indep.) MPI_Group_compare (C) MPI_GROUP_COMPARE (Fortran) and MPI_GROUP_SIZE Names were changed to conform with this style, but a few were felt to be very clumsy and were left as is. * Point to point Much more text discussing the various communication modes, their semantics, and what users are supposed to do with them...have a look. Pack/unpack (I believe there may be some minor problems in here) Buffered communication mode More examples * Collective communication Definition of MPI_MINLOC, MPI_MAXLOC -- we changed the def'n to (we think) a more elegant form. Section on user defined reduce functions -- was significantly expanded for the sake of precision. Did we succeed? Even more examples in collective. * Context Section on attribute caching and callback functions * Inquiry Section on predefined attributes for MPI_COMM_WORLD * Language Binding Tried to track name changes and the like --Steve Otto --------------------------------------------------------------------------- Dr Steve Otto | Dept of Computer Science and Engineering *___________ | Oregon Graduate Institute of Science & Technology _____) | Mailing: 20000 NW Walker Rd, PO Box 91000 / | Portland, OR 97291-1000 USA / | email: otto@cse.ogi.edu /o | Phone: (503) 690-1486 (office) | Phone: (503) 641-8399 (home) | FAX: (503) 690-1553 | Shipping: 20000 NW Walker Rd, Beaverton, OR 97006 USA --------------------------------------------------------------------------- From owner-mpi-core@CS.UTK.EDU Wed Apr 6 14:59:43 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id OAA06700; Wed, 6 Apr 1994 14:59:43 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id OAA02160; Wed, 6 Apr 1994 14:59:29 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 6 Apr 1994 14:59:27 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id OAA02153; Wed, 6 Apr 1994 14:59:25 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2117; Wed, 06 Apr 94 14:59:07 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 9149; Wed, 6 Apr 1994 14:59:07 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Wed, 06 Apr 94 14:59:07 EDT Received: by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA28293; Wed, 6 Apr 1994 14:59:06 -0400 From: snir@watson.ibm.com (Marc Snir) Message-Id: <9404061859.AA28293@snir.watson.ibm.com> To: mpi-core@CS.UTK.EDU Cc: otto@cse.ogi.edu, lusk@mcs.anl.gov, Rolf.Hempel@gmd.de Subject: MPI draft -- language binding Reply-To: snir@watson.ibm.com Date: Wed, 06 Apr 94 14:59:06 -0500 :-) :-) :-) *** (-: (-: (-: 1. Need also Fortran prototypes for user-defined functions (really, example of definitions) in the language binding. The prptotypes are in the main text. 2. The naming scheme for these prototypes is not consistent. I suggest MPI_Copy_function MPI_Delete_function MPI_Handler_function MPI_User_function All capitalized in the same way, and all with suffix "function". 3. I got some objections about MPI_Graphdims_get and MPI_Cartdim_get. One suggestion is MPI_Graph_get_dim and MPI_Cart_get_dim. Another is MPI_Graph_size and MPI_Cart_size, which is more in line with MPI_Comm_size and MPI_Group_size. 4. We may also want to replace MPI_Graph_get with MPI_Comm_graph, in analogy with MPI_Comm_group, and MPI_Cart_get with MPI_Comm_cart. P.S. Is it my mailer that does not work, or is it that everybody is happy with the draft? From owner-mpi-core@CS.UTK.EDU Thu Apr 7 04:27:10 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id EAA11424; Thu, 7 Apr 1994 04:27:10 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id EAA02153; Thu, 7 Apr 1994 04:27:40 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 7 Apr 1994 04:27:39 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id EAA02131; Thu, 7 Apr 1994 04:27:34 -0400 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA09532 (5.65c8/IDA-1.4.4 for ); Thu, 7 Apr 1994 10:27:12 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA16993; Thu, 7 Apr 1994 10:27:07 +0200 Date: Thu, 7 Apr 1994 10:27:07 +0200 From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9404070827.AA16993@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: Topology chapter Marc, with this message I would like to show you that your mailer is still working okay. Also, I have had a look at the topology chapter. It seems to be impossible to look into the MPI document without finding at least one error. Here is what I found this time: 1. On page 177, line 34, the sentence has been corrupted in the editing process. My suggestion is to change it to: "..... helps the user to select a balanced distribution of processes per coordinate direction, ....." 2. On page 184, the Example 6.4 does not seem to work. I don't know who originally added this example, and what exactly it is supposed to do. The problem is the use of variable coords(2) as the displacement. A process with coords(2)=1, for example, will send a message to its neighbor process for which coords(2)=2. This neighbor, however, will try to receive a message from the process with coords(2)=0, so a deadlock will occur. Do I understand something wrong? My solution would be to make the example a little less sophisticated, but correct, by changing the sentence in line 39 to: "One wishes to shift its columns forwards by 3 steps." On line 47, then, "coords(2)" is replaced by "3". I don't know who does the editing at this point. Should I grab the chapter from Steve Otto's file server, do the changes, and send it back? On another vein, I have no strong opinion about the function names of MPI_GRAPHDIMS_GET etc. However, I don't like MPI_CART_SIZE because it is a little misleading. The function does only return the number of dimensions, not the size of the topology which would be given by the dimensions themselves. So, people might confuse MPI_CART_SIZE with MPI_CART_GET. The names MPI_GRAPH_GET_DIMS and MPI_CART_GET_DIM would be okay, only that they are a little long and complicated. I don't think we should replace MPI_GRAPH_GET with MPI_COMM_GRAPH. Assume one wants to retrieve the topology information. The first thing is to look up the dimensions by calling MPI_GRAPH_GET_DIMS or however this function will be called. This suggests that the topology is already retrieved from the communicator. Thus, a later call to MPI_COMM_GRAPH looks strange to me. Again, I'm not religious about function names, but we should change what we have now only if we make the life of the users easier. It should be easy to find the correspondence between complementary functions by looking at their names, as is the case now with MPI_CARTDIM_GET and MPI_CART_GET. Personally I would vote for leaving the names as they are. - Rolf From owner-mpi-core@CS.UTK.EDU Fri Apr 8 08:20:40 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id IAA23216; Fri, 8 Apr 1994 08:20:40 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id IAA15242; Fri, 8 Apr 1994 08:20:58 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 8 Apr 1994 08:20:57 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id IAA15220; Fri, 8 Apr 1994 08:20:50 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA04528 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Fri, 8 Apr 1994 13:19:47 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA09146; Fri, 8 Apr 1994 13:17:45 +0000 Date: Fri, 8 Apr 1994 13:17:45 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9404081217.AA09146@tycho.co.uk> To: mpi-core@CS.UTK.EDU Cc: otto@cse.ogi.edu In-Reply-To: <9404061859.AA28293@snir.watson.ibm.com> Subject: Re: MPI draft (Various) People, Here are some comments on the March 31 draft. Mainly typos, a few more substantive issues. I'll put these first, and follow with the typos. Real issues to argue over... Page:line Issue 68:34 MPI_TYPE_SIZE. What does this actually return ? An example would be useful. The current definition is sort of recursive and hard to understand (at least I find it so !) I assume that it means the number of bytes of actual data described by the datatype. (But what about a type which references the same location more than once ???) 165 We should say which MPI calls are valid in copy and delete functions, and whether they can refer to the communicator on which they are being applied. (I believe that there need be no restrictions, apart from deleting the communicator, and the communicator should still be valid to use. I think this is OK). 190 and 193 Fortran binding of MPI_GET_PROCESSOR_NAME and MPI_ERROR_STRING is wrong. (It is a bad idea to put the null character into a Fortran string, since then you can't just print it) I believe it should look something like this. ---- MPI_GET_PROCESSOR_NAME(NAME, IERROR) CHARACTER*(*) NAME INTEGER IERROR The name is a character string for maximum flexibility. From this value it should be possible to identify a specific piece of hardware; possible values include such things as "processor 9 in rack 4 of mpp.cs.org", "231", "c0m4b6p0" and so on. An error code of MPI_UNDEFINED is allowed, but strongly discouraged. The processor name returned is that of the processor on which the process is currently executing, if a system were to support process migration, then the value returned should change if the process migrated. The value MPI_MAX_PROCESSOR_NAME is the maximum length (in bytes) which is required for the NAME argument, this length includes any termination characters which may be required in a particular language binding. If the length of NAME is less than that required, then the leading part of the result is returned in NAME (with a terminator if the language's string convention requires it), and the error MPI_ERR_TRUNCATE is raised. In the Fortran binding the result string is assigned to the NAME variable as if by a normal Fortran character assignment, therefore truncation can occur if NAME is too short (but note that MPI_ERR_TRUNCATE should be raised in this case), and NAME will be blank filled if it is larger than required. This follows normal Fortran practice, and allows the result to be printed immediately, with no further processing. ---- A similar change should be made for MPI_ERROR_STRING. (Though maybe we don't want to raise MPI_ERR_TRUNCATE here ? Since we're already in an error handler, and could easily go recursive...) Typos ===== Page:line Change All fortran examples have THEN on next line from IF. This is incorrect unless there is a continuation marker. (if without a then on the same line is a single statement if, presence of the "then" turns it into a block if). Cover Please add "(PPPE)" after P6643 Acknowledgements:43 "Steve Zenith" should read "Steven Zenith" (!) 13:18 "erroneuous" should read "erroneous" 16:16 "to efficiently" should read "efficiently to" 18:34 :" should (maybe) be ": 29:33 A spare "," after "mechanism.)" 39:11/12 Missing "does" over line break. (Currently reads "It not indicate") 41:9 "the request will complete successfully" should read "the request will be allowed to complete" (We can't guarantee successful completion of the request !) 49:22 "until it is received" should read "before it is received" 50: Labels in Fortran examples are not sufficiently outdented (at least if this is fixed format...) 51:27 "using a call to MPI_WAIT or MPI_TEST" should read "using a call to MPI_REQUEST_FREE, MPI_WAIT or MPI_TEST" 62:5 "starting 4.16 apart." I don't know what this should read ! 68:15 "the common case" should read "commonly the case" 83:13 "to be send" should read "to be sent" 83:20 "operations facilitate" should read "operations facilitates" 85:7 "contains now" should read "now contains" 85:11 "bytes that consists" should read "bytes that forms" 85:22 "two packing unit" should read "two packing units" 90:17 "are the following." should read "are the following :" 112:47 Table heading needs to move to next page. 117:24 Fortran example is missing "END DO" "END IF" 118:38 "a op" should read "an op" 125:25 etc Tabs in \verbatim don't work ! 130:20 "help ensure" should read "help to ensure" 131:19 "on par with" should read "on a par with" 131:30 "used to invoke" should read "used in" 131:36,46 Inconsistent use of font for bullet items (and over page) 146:3 "attributed" should read "attributes" 152:36 "backmasking cannot occur" should read "backmasking cannot be required" 156:16 "exists where both" should read "exists to which both" 156:26 "new children processes" should read "new child processes" 166:12 "arbitrary communicator" should read "arbitrary communicators" 166:40 Some \_ have escaped into a verbatim section 167:34 Ditto 169:43 Empty advice to implementors 190 See above re MPI_GET_PROCESSOR_NAME 191:29 "In such case" should read "in this case" 192:16 "Such call" should read "This call" 193 See above re MPI_ERROR_STRING 195:10 C binding for MPI_Wtime should read double MPI_Wtime(void) 195:34 ditto for MPI_Wtick double MPI_Wtick(void) There is a difference in ANSI C between a prototype without any arguments "foo()" (assumed to be a K&R external declaration, and so accepting ANY argument list, and a declaration just of "foo(void)" an ANSI declaration of a function with NO arguments. We want the latter.) 195:24 Same for MPI_Finalize That's all folks -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@cs.utk.edu Fri Apr 8 14:05:24 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id OAA24770; Fri, 8 Apr 1994 14:05:24 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id MAA09446; Fri, 8 Apr 1994 12:45:43 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 8 Apr 1994 12:45:34 EDT Errors-to: owner-mpi-core@cs.utk.edu Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id MAA09411; Fri, 8 Apr 1994 12:45:26 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA13525 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Fri, 8 Apr 1994 17:44:26 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA00351; Fri, 8 Apr 1994 17:42:21 +0000 Date: Fri, 8 Apr 1994 17:42:21 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9404081642.AA00351@tycho.co.uk> To: mpi-core@cs.utk.edu Cc: otto@cse.ogi.edu Subject: One more typo Page 195 line 18 ''ticks'' should read ``ticks'' -- Jim From owner-mpi-core@CS.UTK.EDU Fri Apr 8 16:27:28 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA26691; Fri, 8 Apr 1994 16:27:28 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA28601; Fri, 8 Apr 1994 16:27:54 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 8 Apr 1994 16:27:53 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA28593; Fri, 8 Apr 1994 16:27:51 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 6621; Fri, 08 Apr 94 16:27:49 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 1331; Fri, 8 Apr 1994 16:27:49 EDT Received: from bubba.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 08 Apr 94 16:27:48 EDT Received: by bubba.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA33831; Fri, 8 Apr 1994 16:27:47 -0400 Message-Id: <9404082027.AA33831@bubba.watson.ibm.com> To: mpi-core@CS.UTK.EDU Cc: otto@cs.ogi.edu Subject: Draft 3/31 comments Date: Fri, 08 Apr 94 16:27:47 -0500 From: "Hubertus Franke" I have a few comments on the 3/31 draft. pg 139: MPI_Group_range_incl and MPI_Group_range_excl should have the following prototype. ...(MPI_Group group, int n, int ranges[][3], MPI_Group newgroup); int** is not the same as int[][3] !!!!! pg 195/3 MPI defines on timer. and NOT two !!! pg 208/13 should be MPI_ERRORS_ARE_FATAL and MPI_ERRORS_RETURN pg 210/13..18 should be deleted as they are specified in line 27..34 pg 210/27..34 all names must be in brackets and names should be corrected. MPI_copy_function -> (MPI_Copy_function) MPI_delete_function -> (MPI_Delete_function) MPI_Handler_function -> (MPI_Handler_function) MPI_Uop -> (MPI_Uop) pg 118/48 same as above MPI_Uop -> (MPI_Uop) pg 207/25: propose MPI_OK as a synonym for MPI_SUCCESS why all these long names. pg 208..: missing definitions for MPI_2REAL, MPI_2DOUBLE_PRECISION, MPI_2INTEGER, MPI_FLOAT_INT, MPI_DOUBLE_INT, MPI_LONG_INT, MPI_2INT. pg 51/20: There seems no obvious reason for being passed by reference. This would only make sense if the object behind the request will be exchanged. If that is possible (why not) then I suggest an advice to the that he could do so. pg 55/20: There seems no obvious reason for being a pointer, in this case the object can not be exchanged in the background. That would be far fetched. From owner-mpi-core@CS.UTK.EDU Fri Apr 8 18:49:06 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id SAA27240; Fri, 8 Apr 1994 18:49:05 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id SAA08817; Fri, 8 Apr 1994 18:49:32 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 8 Apr 1994 18:49:29 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id SAA08809; Fri, 8 Apr 1994 18:49:27 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA17229; Fri, 8 Apr 94 15:49:14 -0700 Date: Fri, 8 Apr 94 15:49:14 -0700 From: Steve Otto Message-Id: <9404082249.AA17229@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Topology Chapter; Actions taken. Marc, with this message I would like to show you that your mailer is still working okay. Also, I have had a look at the topology chapter. It seems to be impossible to look into the MPI document without finding at least one error. Here is what I found this time: 1. On page 177, line 34, the sentence has been corrupted in the editing process. My suggestion is to change it to: "..... helps the user to select a balanced distribution of processes per coordinate direction, ....." Done. 2. On page 184, the Example 6.4 does not seem to work. I don't know who originally added this example, and what exactly it is supposed to do. The problem is the use of variable coords(2) as the displacement. A process with coords(2)=1, for example, will send a message to its neighbor process for which coords(2)=2. This neighbor, however, will try to receive a message from the process with coords(2)=0, so a deadlock will occur. Do I understand something wrong? This isn't my example, but my understanding was that the example was trying to shift column i by i steps, ****in the row direction, or "downwards"****. The example makes the mistake of not stating the direction of the shift. So, I have changed line 47, page 184 to: CALL MPI_CART_SHIFT(comm, 1, coords(2), source, dest, ierr) Now it doesn't deadlock. Correct? My solution would be to make the example a little less sophisticated, but correct, by changing the sentence in line 39 to: "One wishes to shift its columns forwards by 3 steps." On line 47, then, "coords(2)" is replaced by "3". I don't know who does the editing at this point. Should I grab the chapter from Steve Otto's file server, do the changes, and send it back? No, I do it. Handing files out at this point is too chaotic. On another vein, I have no strong opinion about the function names of MPI_GRAPHDIMS_GET etc. However, I don't like MPI_CART_SIZE because it is a little misleading. The function does only return the number of dimensions, not the size of the topology which would be given by the dimensions themselves. So, people might confuse MPI_CART_SIZE with MPI_CART_GET. The names MPI_GRAPH_GET_DIMS and MPI_CART_GET_DIM would be okay, only that they are a little long and complicated. I don't think we should replace MPI_GRAPH_GET with MPI_COMM_GRAPH. Assume one wants to retrieve the topology information. The first thing is to look up the dimensions by calling MPI_GRAPH_GET_DIMS or however this function will be called. This suggests that the topology is already retrieved from the communicator. Thus, a later call to MPI_COMM_GRAPH looks strange to me. Again, I'm not religious about function names, but we should change what we have now only if we make the life of the users easier. It should be easy to find the correspondence between complementary functions by looking at their names, as is the case now with MPI_CARTDIM_GET and MPI_CART_GET. Personally I would vote for leaving the names as they are. - Rolf I am not getting a clear signal as to whether we should change names. Since I am somewhat uncomfortable with it (it's error prone and other, unforseen problems may arise) I would like to leave them as they are. --Steve Otto From owner-mpi-core@CS.UTK.EDU Mon Apr 11 09:09:37 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id JAA15515; Mon, 11 Apr 1994 09:09:37 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA15715; Mon, 11 Apr 1994 09:10:14 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Apr 1994 09:10:13 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA15708; Mon, 11 Apr 1994 09:10:11 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9579; Mon, 11 Apr 94 09:10:10 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 8699; Mon, 11 Apr 1994 09:10:10 EDT Received: from bubba.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 11 Apr 94 09:10:10 EDT Received: by bubba.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA33732; Mon, 11 Apr 1994 09:10:07 -0400 Message-Id: <9404111310.AA33732@bubba.watson.ibm.com> To: mpi-core@CS.UTK.EDU Cc: otto@cse.ogi.edu, snir@watson.ibm.com Subject: Draft 3/31 omments Date: Mon, 11 Apr 94 09:10:07 -0500 From: "Hubertus Franke" More comments on the 3/31 draft: --------------------------------------------------- | A) TYPOS and inconsistency in cross referencing | --------------------------------------------------- pg. 208: MPI_PACKED missing for the Fortran interface pg. 70/14 && 70/26: The displacement type should be coherent with the one in MPI_Type_struct etc. -> both or both . pg. 214/39: MPI_op_free -> MPI_Op_free ----------------------------------- | B) SERIOUS semantical question | ----------------------------------- pg. 39. Interaction between MPI_Wait/MPI_Test .. and MPI_BSEND is not clear. I can justify several interpretations from the draft. MPI_BSEND: 1) immediately buffer and return, the message will be completed in the background. Q: But for what do we need MPI_IBSEND then? 2) try to send and if (wait is issued or the message is blocking) and the message has not been completed then allocate a buffer, copy and complete in the background. Wait immediately returns with done. In this scenario MPI_IBSEND could make sense. => this model guarantees that buffer can be reused after wait. Also in question is the interaction between wait, buffered send and persistent handles. Let's suppose the wait is issued on the persistent handle. If we buffer and return as described in (2) then the handle has to be duped in the message kernel, as to allow to reuse the persistent handle for the next communication and to have the current communication complete in the background. If however MPI_WAIT waits till the communication is actually completed, why do we need persistent buffered sends? Regardless of how I look at this buffered beast, I can't get clean semantics => needs work !!!! Hope this can be clarified before G-Day. ------------------------------------------------------------------ Hubertus Franke IBM T.J. Watson Research Center P.O.-Box 218 Yorktown Heights, NY 10598 From owner-mpi-core@CS.UTK.EDU Mon Apr 11 09:12:16 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id JAA15540; Mon, 11 Apr 1994 09:12:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA16103; Mon, 11 Apr 1994 09:13:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Apr 1994 09:13:07 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA16096; Mon, 11 Apr 1994 09:13:05 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA25440; Mon, 11 Apr 94 09:13:03 EDT Date: Mon, 11 Apr 94 09:13:03 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404111313.AA25440@super> Received: by b125.super.org (4.1/SMI-4.1) id AA07402; Mon, 11 Apr 94 09:13:02 EDT To: mpi-core@CS.UTK.EDU Subject: draft comments #1 Howdy MPIers, With time running short, I will send in my comments as I finish. Attached is what I saw from a slow reading of the first 50 pages. Jim and I found only one mistake that was the same (not duplicated below). I don't think the rest overlap. This is a little scary about how good my slow reading was. Steve ---------------------------------------------------------------------- 9:4 - innaccessible -> inaccesible 9:33 - MPI_ERRORFATAL -> MPI_ERRORS_ARE_FATAL as defined on 191:18 9:34 - MPI_ERRORSRETURN -> MPI_ERRORS_RETURN as defined on 191:22 9:45 - This is a general comment that I think goes here. In the FORTRAN bindings, the document uses (ex. 16:41) for a choice variable; (void *) in C (11:36). I did not notice anywhere that we clearly state what this means. I wonder if it should go here for completeness. 10:28 - Did we say that implementors could not use MPI_ for other routines? Excluding this would leave all the names open for future use by MPIF. On the other hand, a logical extension of MPI might use the MPI_ name. I thought we voted on this but cannot recall the outcome for sure. 10:31/11:29 - The FORTRAN binding (10:31) say "error codes are only potentially implementation dependent; see Chapter 7." The C binding say (11:29) "failure return codes are implentation dependent." These are somewhat different. p. 194 defines error codes/classes so they are both "standard" and implementation dependent. It isn't real important but I think these statements should be the same since they are the same in the two languages. 14:5 - neither requirements or -> neither requirements nor 14:22 - form -> from 15:37,48 - I am a little confused by the message[20]. on line 37 it has an MPI_Recv with a size of 21. Since message is size [20] should the receive also be 20 (or is the \n somehow extra). Also, line 48 says the message sent is 21 characters but the send on line 33 uses strlen(message) so I think it only sends 13 characters with the \n. 19:41 - Do we want to make a statement that if the message is shorter then the buf, then values beyond the receive count are not touched in buf? 25:3 - Should "No conversion occurs" -> "No conversion need occur". It might be possible on an implementation designed for heterogeneous systems that they alway use XDR (for example). I think it would be inefficient but I think it would be ok. 33:22,23,26 - It is not clear to me why MPI_BUFFER_DETACH has buffer and size as OUT arguments. It seems like you are telling the system the start address and size of the message queue (I would think the system has to know the size). I would not think MPI changes the pointer (it does not deallocate it) and certainly not the size. Along the same lines, I think there is an extra * in the C binding. Thus, "void** buffer, int* size" -> "void* buffer, int size" 34:5 - etc.. -> etc. ? 34:20 - Should backward -> forward. I'm not completely sure what is being done, but I would think you start at the head (the oldest) and search forward for messages that are done in the buffer. 35:30-33 - It says "On the other hand..". It was confusing to me about what was different between the first and second sentence since they refer to send-complete and send. I may just be confused myself. 52:44 - send -> sent. From owner-mpi-core@CS.UTK.EDU Mon Apr 11 10:13:47 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA16214; Mon, 11 Apr 1994 10:13:47 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA21988; Mon, 11 Apr 1994 10:14:30 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Apr 1994 10:14:29 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA21974; Mon, 11 Apr 1994 10:14:24 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA14137 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Mon, 11 Apr 1994 15:13:29 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA01645; Mon, 11 Apr 1994 15:11:18 +0000 Date: Mon, 11 Apr 1994 15:11:18 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9404111411.AA01645@tycho.co.uk> To: frankeh@watson.ibm.com Cc: mpi-core@CS.UTK.EDU, otto@cse.ogi.edu, snir@watson.ibm.com In-Reply-To: <9404111310.AA33732@bubba.watson.ibm.com> (frankeh@watson.ibm.com) Subject: Re: Draft 3/31 omments > ----------------------------------- > | B) SERIOUS semantical question | > ----------------------------------- > > pg. 39. Interaction between MPI_Wait/MPI_Test .. and MPI_BSEND > is not clear. I can justify several interpretations > from the draft. > > MPI_BSEND: > 1) immediately buffer and return, the > message will be completed in the background. > > Q: But for what do we need MPI_IBSEND then? Suppose that you are on a multi-processor machine, then you could return in the IBSEND case BEFORE copying the data, and awake another thread to do the copying. IBSEND will then return sooner than BSEND would even if you always copy. As ever WAIT waits for the send buffer to be re-usable. It says NOTHING about the message having actually been sent. (Exactly the same semantics as a standard mode send). > 2) try to send and if (wait is issued or the message > is blocking) and the message > has not been completed then allocate a buffer, > copy and complete in the background. > Wait immediately returns with done. > In this scenario MPI_IBSEND could make sense. > => this model guarantees that buffer can be reused > after wait. I think this is just an optimisation of the previous case to avoid the copy if it is not (actually) required. The user model should be as above. (This is necessary for the explanation of the buffer exhaustion case to be simple). In either case the IBSEND variant potentially has value. (Though as you point out if the implementation really is as in case 1 above it may not be a great benefit, aside from the fact that it completes the table of send/isend pairs, and would be hard to justify it NOT being there). > Also in question is the interaction between wait, buffered > send and persistent handles. > Let's suppose the wait is issued on the persistent handle. > If we buffer and return as described in (2) > then the handle has to > be duped in the message kernel, as to allow to > reuse the persistent handle for the next communication > and to have the current communication complete in the > background. > > If however MPI_WAIT waits till the > communication is actually completed, why do we need > persistent buffered sends? > > Regardless of how I look at this buffered beast, I can't > get clean semantics => needs work !!!! The wait here should mean the same as it always did, i.e. it is a LOCAL property, solely meaning that the input buffer can be trashed. I believe that the effect is like this MPI_Start_Bsend(...) { if (can send without blocking) { mpi_isend( same args ); save descriptor for this isend in original bsend block. } else { do the copy (or schedule it); mpi_isend(copied buffer); save descriptor for isend on global list, will be picked up by the buffer allocator later. } } MPI_Bsend_complete(...) { if (there's a send descriptor here) return MPI_Status(that one) else return TRUE; /* It was the copy case, so it must be done... */ } Is this too horrible ? -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Mon Apr 11 10:25:06 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA16241; Mon, 11 Apr 1994 10:25:06 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA23037; Mon, 11 Apr 1994 10:25:54 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Apr 1994 10:25:52 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA23021; Mon, 11 Apr 1994 10:25:49 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA14515 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Mon, 11 Apr 1994 15:24:55 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA01667; Mon, 11 Apr 1994 15:22:48 +0000 Date: Mon, 11 Apr 1994 15:22:48 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9404111422.AA01667@tycho.co.uk> To: lederman@super.org Cc: mpi-core@CS.UTK.EDU In-Reply-To: <9404111313.AA25440@super> (lederman@super.org) Subject: Re: draft comments #1 > 33:22,23,26 - It is not clear to me why MPI_BUFFER_DETACH has buffer > and size as OUT arguments. It seems like you are telling the system > the start address and size of the message queue (I would think the > system has to know the size). I would not think MPI changes the > pointer (it does not deallocate it) and certainly not the size. Along > the same lines, I think there is an extra * in the C binding. Thus, > "void** buffer, int* size" -> "void* buffer, int size" I think the idea here is that you (the user) may need to free the buffer after MPI has finished using it, and this may happen in a different piece of code than where you allocated it (so you may no longer have the buffer in your hand). Therefore it makes sense for MPI to give you back as much information as you gave it about the buffer. (Otherwise you have to remember this somewhere yourself). That's why these are OUT arguments, you're not telling MPI anything about the buffer at this point, rather it is giving you back the info you gave it in the BUFFER_ATTACH call. If so the arguments are correct both in intenta and type. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Tue Apr 12 10:08:42 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA24897; Tue, 12 Apr 1994 10:08:41 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA14985; Tue, 12 Apr 1994 10:09:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 12 Apr 1994 10:09:07 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA14975; Tue, 12 Apr 1994 10:09:04 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA07879; Tue, 12 Apr 94 10:09:02 EDT Date: Tue, 12 Apr 94 10:09:02 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404121409.AA07879@super> Received: by b125.super.org (4.1/SMI-4.1) id AA07501; Tue, 12 Apr 94 10:09:02 EDT To: mpi-core@CS.UTK.EDU Subject: draft comments #2 Hello again, I finished pt-2-pt last night. The comments below are on pages 53-89. I give Marc credit for lots of interesting examples; I thought I would never finish understanding them all! Given the number of things I saw, there must be more. Hope others are giving it a careful reading. I'm sure I have made some mistakes in my comments. Steve ---------------------------------------------------------------------- 55:6 - dest -> source 55:35 - four -> five? I think there is one more from adding MPI_BSEND_INIT. 58:27 - "is what would obtain" -> "is what would be obtained" 63:19 - STRIDE in the FORTRAN definition is an address (MPI_Aint in the C definition). 11:15 says that mpif.h in FORTRAN should define MPI_ADDRESS if possible. Since a default integer and MPI_ADDRESS might be different, I think we should use MPI_ADDRESS for things that are addresses in the FORTRAN binding. I think it is much easier to to take MPI_ADDRESS into integer than visa versa sine there are lots of intergers this could happen to. This comment also holds for: 65:30,41 - array_of_displacements 66:29,40 - array_of_displacements 67:41,46 - address 68:25,30 - extent 68:37,41 - size 69:39 - I'm not sure I gotten the definition of lb(Typemap) figured out. My concern is with the second condition (otherwise) where type_j = lb. Supposae one combines several derived datatypes into a new datatype and some had lb as a type and some did not. If a disp_j of a datatype where type_j != lb is smaller than any of the disp_k where type_k=lb, then should the lb(Typemap) be that disp_j? If so, then the otherwise clause which restricts type=lb seems too restrictive. This comment also holds (for max) to 69:43 for ub(Typemap). Is this right or am I confused as usual? 70:11,14 - MPI_TYPE_LB returns a displacement. The other functions for extent and derived datatypes return MPI_Aint*. Here it returns int*. Should it be MPI_Aint for consistancy. (The MPI_ADDRESS also holds for FORTRAN definition if changed to MPI_Aint in C.) 70:23,26 - same comment for MPI_TYPE_UB as above. 72:9 - datatype. -> datatype, 75:34 - "MPI_STATUS status" -> "int status(MPI_STATUS_SIZE)" as defined for MPI_SENDRECV on 57:33. This raises a point I missed earlier. I cannot find where MPI_STATUS_SIZE is definied. I think it is first used in MPI_WAIT on p. 39. I cannot find a discussion of MPI_STATUS_SIZE and MPI_Status in Ch. 2. This comment also applies to staus on: 76:12 76:36 77:12 78:18 - I think the code: for(i=0; i <3; i++) disp[i] -= disp[0]; is incorrect. It has a classic vector type dependence. What is intended is to make all the displacements relative to disp[0] and disp[0]=0. However, i=0 makes disp[0]=0 first, and all the other disp[i] are therefore unchanged. I think it will work if you run the loop backward: for(i=2; i >=0; i--) disp[i] -= disp[0]; The same comment applies to: 78:35 80:35 80:20 - MPI_REAL -> MPI_DOUBLE since particle.d is double 80:22: I would change particle.d -> particle[0].d. This makes it consisant with other places (80:33, 81:14, ...) and is a little clearer. 82:39 - need a ; to terminate the C expression: disp[1] = j-i -> disp[1] = j-i; 85:11 - "consists the packing" -> "consists of the packing" 86:2 - MPI_PACK_SIZE(incount, datatype, size) -> MPI_PACK_SIZE(incount, datatype, comm, &size) 86:4 - MPI_PACK(...position) -> MPI_PACK(...&position, comm) 86:10 - The int ... should be on a separate line then Example 3.36. Also applies to: 86:30 86:21,22: It now reads: MPI_PACK(buff, 1000, position, i, 1, MPI_INT); MPI_PACK(buff, 1000, position, j, 1, MPI_INT); It is clearly wrong. My guess is it should be: MPI_PACK(&i, 1, MPI_INT, buff, 1000, &position, MPI_COMM_WORLD); MPI_PACK(&j, 1, MPI_INT, buff, 1000, &position, MPI_COMM_WORLD); 87:11 - MPI_PACK(MPI_BOTTOM, 1, newtype, buff, position, MPI_COMM_WORLD) -> MPI_PACK(MPI_BOTTOM, 1, newtype, buff, 1000, &position, MPI_COMM_WORLD) 87:38 - This is really a general comment. The examples use status in MPI_Recv, etc. It is of type MPI_Status*. Are we assuming it is declared as MPI_Status *status? If you do this then using MPI_GET_COUNT is ugly since it uses MPI_Status (not MPI_Status *). The sample programs distributed so far (from ANL/MSU, mpif) use MPI_Status status and then use MPI_Recv(..., &status). I hate to say this, but this has been done throughout the chapter. Other variables are like this too. If people agree with my interpretation, I can go back and try and find them all. Ugh :-(. 87:43 - MPI_Unpack(buff, 1000, position, i, 1, MPI_INT) -> MPI_Unpack(buff, 1000, &position, &i, 1, MPI_INT, MPI_COMM_WORLD) 87:47 - MPI_Unpack(buff, 1000, position, a, i, MPI_FLOAT) -> MPI_Unpack(buff, 1000, &position, a, i, MPI_FLOAT, MPI_COMM_WORLD) 88:8 - MPI_Comm_size(comm, gsize) -> MPI_Comm_size(comm, &gsize) 88:9 - MPI_Comm_rank(comm, myrank) -> MPI_Comm_rank(comm, &myrank) 88:13 - MPI_Pack_size(1, MPI_INT, comm, k1) -> MPI_Pack_size(1, MPI_INT, comm, &k1) 88:14 - MPI_Pack_size(count, MPI_CHAR, k2) -> MPI_Pack_size(count, MPI_CHAR, comm, &k2) 88:26 - MPI_GATHER(&position, 1, MPI_INT, NULL, NULL, root, comm) -> MPI_GATHER(&position, 1, MPI_INT, NULL, NULL, NULL, root, comm) (need another NULL). 88:34 - MPI_GATHER(&position, 1, MPI_INT, counts, MPI_INT, root, comm) -> MPI_GATHER(&position, 1, MPI_INT, counts, 1, MPI_INT, root, comm) 88:39 - do -> for (who let those FORTRAN people in here :-). Also applies to: 89:1 89:4 - MPI_Unpack(....,position, count,...) -> MPI_Unpack(....,&position, &count,...) 89:6 - MPI_Unpack(..., position, ..) -> MPI_Unpack(..., &position, ..) From owner-mpi-core@CS.UTK.EDU Wed Apr 13 08:40:57 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id IAA03484; Wed, 13 Apr 1994 08:40:56 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id IAA00759; Wed, 13 Apr 1994 08:41:09 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 13 Apr 1994 08:41:06 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id IAA00742; Wed, 13 Apr 1994 08:41:01 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA19738; Wed, 13 Apr 94 08:40:51 EDT Date: Wed, 13 Apr 94 08:40:51 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404131240.AA19738@super> Received: by b125.super.org (4.1/SMI-4.1) id AA07556; Wed, 13 Apr 94 08:40:50 EDT To: mpi-core@CS.UTK.EDU Subject: comments #3 Hello another time! Here are my comments on Chapters 6, 7 and 8. I assume Steve Otto is really starting to hate my e-mail. Steve ---------------------------------------------------------------------- 176:34,36 - This paragraph refers to the last section as having the *_MAP functions. The last section is actually the example. These are in the next to last section (6.5.7). (Now I'm getting picky!) 178:12 - "dims[i] will be ordered in non-increasing order." Does this mean decreasing? If so, I prefer decreasing over non-increasing. 180:36 - This is a question: Is nnodes returned by MPI_GRAPHDIMS_GET the same value returned by MPI_COMM_SIZE? If so, it would be nice to note. 182:22 - "0<=coords(i)<=dims(i)" -> "0<=coords(i) "MPI_CART_COORDS(comm, rank, maxdims, coords, ierr)" 188:13,17,18,25,28,31,34 - All the MPI_* calls need to have ierr added as the last argument since these are FORTRAN calls. Also need to declare ierr as an integer. 189:30 - MPI_UNDEFINED -> MPI_PROC_NULL as on 190:3 192:1,12 - definition refers to function (as do language binding ones) but the description says handler_function. Need to chose one. 193:41/208:10 - The value MPI_MAX_ERROR_STRING is not listed in the appendix at 208:17. 194:32 - I think it would be nice to give the reader the use of MPI_ERR_LASTCODE. It isn't a code returned by an MPI error. 196:28 - "This routine cleans all MPI state" -> "This routine cleans up all MPI state" 200:41 - uses "int extent". By definition of MPI_Type_size on 68:39 this should be MPI_Aint. (I had to find something in the profile chapter :-) From owner-mpi-core@CS.UTK.EDU Wed Apr 13 11:39:50 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id LAA05344; Wed, 13 Apr 1994 11:39:49 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id LAA16288; Wed, 13 Apr 1994 11:39:42 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 13 Apr 1994 11:39:41 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id LAA16239; Wed, 13 Apr 1994 11:39:36 -0400 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA03554 (5.65c8/IDA-1.4.4 for ); Wed, 13 Apr 1994 17:39:27 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA16962; Wed, 13 Apr 1994 17:39:25 +0200 Date: Wed, 13 Apr 1994 17:39:25 +0200 From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9404131539.AA16962@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: Steve's corrections Cc: gmap10@f1neuman.gmd.de Hi there, Steve really did a great job at detecting mistakes in the document. I read over the text so many times, but it seems that after a while I don't look at the details carefully enough any more. Steve's corrections are okay except for the following comments: 178:12 - Non-increasing does not necessarily mean decreasing, since it also includes the case where entries are identical. 180:36 - Yes, that's true. Perhaps we should add something like (same as number of processes in the group). 184:37 - Sorry, but I still have a problem with the example. We have to be very precise in what we mean with "shift column i by i steps". Steve seems to understand the example such that the topology consists of a number of columns, and in each column the data is shifted by a certain number of steps. This step width is set to be the column index, so that it is different in each column. However, if 2 is the shift direction (direction of the column), then the index of the column is orthogonal to that direction, namely coords(1). There is always the confusion between Fortran and C as to the definition of rows and columns, but my point is independent of that. One only has to replace the word "column" with "row". A way to save the old example would be to change line 184:47 to CALL MPI_CART_SHIFT(comm, 1, coords(2), source, dest, ierr). I changed it this way because it seems to be a Fortran example, and for a Fortran programmer the first direction is the column direction. I really hope that I'm right with this explanation. Can anybody serve as an arbiter? At this point I could be convinced of anything. - Rolf From owner-mpi-core@CS.UTK.EDU Wed Apr 13 12:06:46 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id MAA05648; Wed, 13 Apr 1994 12:06:46 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id MAA20105; Wed, 13 Apr 1994 12:07:40 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 13 Apr 1994 12:07:39 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id MAA20098; Wed, 13 Apr 1994 12:07:37 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA22079; Wed, 13 Apr 94 12:07:36 EDT Date: Wed, 13 Apr 94 12:07:36 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404131607.AA22079@super> Received: by b125.super.org (4.1/SMI-4.1) id AA07646; Wed, 13 Apr 94 12:07:33 EDT To: mpi-core@CS.UTK.EDU In-Reply-To: <9404131539.AA16962@f1neuman.gmd.de> (Rolf.Hempel@gmd.de) Subject: Re: Steve's corrections As for 178:12 - I still think decreasing is ok. As a matter of fact, monotonically decreasing might be better. I believe this means that the next value is less than or equal to the previous value. This is what you get back. I think I now agree with Rolf about the example 6.4 (184:37). By shifting on coord #1 you shift along the column. Thanks for checking these for me Rolf. Steve From owner-mpi-core@CS.UTK.EDU Thu Apr 14 10:04:00 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA13920; Thu, 14 Apr 1994 10:03:59 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA09019; Thu, 14 Apr 1994 10:04:30 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 14 Apr 1994 10:04:28 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA08992; Thu, 14 Apr 1994 10:04:22 -0400 Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA09540; Thu, 14 Apr 94 10:04:16 EDT Date: Thu, 14 Apr 94 10:04:16 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404141404.AA09540@super.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA00313; Thu, 14 Apr 94 10:04:15 EDT To: mpi-core@CS.UTK.EDU Subject: comments #4 - collective comm Good morning, afternoon or evening! Here are my comments on Chap. 4, collective communications. Sorry for any mistakes below, I have been staring at the document too long. Steve ---------------------------------------------------------------------- 93:43 - says that MPI_BCAST broadcast a message, "itself included". This sounds a little strange since there is only one INOUT buffer. On the root the answer is already "there" at the start. Should we remove the "itself included"? 106:16 - MPI_Scatterv(sendbuf, displs, scounts, ...) -> MPI_Scatterv(sendbuf, scounts, displs, ...) 108:44 - MPI_GATHERV(..., displs, recvcounts, ...) -> MPI_GATHERV(..., recvcounts, displs, ...) 110:22 - says "specifying the maximum number of elements that can be received from each processor". 110:46 says the sendcount[j], sendtype must be equal to recvcount[i], recvtype and this implies that the amount of data send must equal the amount of data received. I think this implies that recvcounts is the amount of data received. If true, I would reword it to be "specifying the number of elements that will be received at each processor". 111:4 - MPI_Send(sendbuf + displs[i], ...) -> MPI_Send(sendbuf + displs[i] * extent(sendtype), ...)? Same comment on: 111:7 for MPI_RECV and extent(recvtype) 113:23 - Was MPI_LONG_DOUBLE deliberately left out from the allowed floating point types? 113:35 - The MPI_B* ops can be applied to C integer but not Fortran integer. I was wondering why we had this asymmetry. Some Fortran (Cray for example) allows bit ops on integers. I suggest we add Fortran integers since it is not clear what is the "right" datatype for doing bit ops. 113:41 - My Fortran is rusty, but should: SUBROUTINE PAR_BLAS1(m, a(m), b(m), c, comm) -> SUBROUTINE PAR_BLAS1(m, a, b, c, comm)? This also applies to: 114:13 for PAR_BLAS2 114:8 - First, can a Fortran subroutine return a value? Also, I didn't think the value returned in Fortran was done via the return statement but by setting the name of the function to the value returned. This also applies to: 114:32 114:13 - I am confused by what this routine is doing. One problem I see is that sum(j) is for j=1,n but the declaration of sum is sum(m). I assume vector * array = vector is being computed. However, I cannot figure out how sum, a, b are distributed so that the MPI_REDUCE works. It seems as if each node must have a full length of a, c and sum so that the local value of m is the full length of the vectors. Otherwise the REDUCE would add different indices of the vector. Is this the right distribution? I really got myself confused on this one. Even with this, I'm not sure it does the right thing. 114:36 - MPI_MINLOC -> MPI_MAXLOC 115:38 - none of the types from MINLOC and MAXLOC are given in Appendix A. 115:38 - How did we decide what pairs to include/exclude. Fortran does not have COMPLEX and C does not have LONG_DOUBLE, SHORT and UNSIGNED. We probably voted on this and I forgot. 116:22 - I think a[30] -> ain[30]? Same comment applies to 117:3 117:16 - MPI_DOUBLE -> MPI_2DOUBLE_PRECISION 117:16 - add error return as last argument to MPI_REDUCE 117:20-24 - The DO loop is missing an end. I'm also not sure if a DO inside an IF is a single statement. I would change to: IF (myrank .EQ. root) then ! .... DO I=1,30 ..... END DO ENDIF 117:44 - if (in.value < val[i]) { ->if (in.value > val[i]) { i.e., < -> > since looking for min. 118:48,119:3 - Is MPI_Uop and UOP supposed to be function? Also, my C isn't deep enough to understand what MPI_Uop on 118:32 is for. The typedef is a void function but the input for MPI_Op_create is MPI_Uop. I just want a C hacker to make sure this was correct. 122:33 - I have the same problems here that I had with Ex. 4.16 p. 114. 123:29 - INTEGER RECVCOUNTS, .... -> INTEGER RECVCOUNTS(*), .... 126:2 - This has the vector dependancy I talked about in note #2 78:18. I think it needs to be changed to: for (i=1; i >= 0; i--) .... From owner-mpi-core@CS.UTK.EDU Thu Apr 14 22:42:53 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id WAA18878; Thu, 14 Apr 1994 22:42:53 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id WAA15676; Thu, 14 Apr 1994 22:43:27 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 14 Apr 1994 22:43:27 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id WAA15669; Thu, 14 Apr 1994 22:43:25 -0400 Message-Id: <199404150243.WAA15669@CS.UTK.EDU> Received: from YKTVMV by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 4861; Thu, 14 Apr 94 22:43:20 EDT Date: Thu, 14 Apr 94 22:35:49 EDT From: "Marc Snir ((914) 945-3204)" To: mpi-core@CS.UTK.EDU, cc:otto@cse.ogi.edu Subject: MPI errata Reply-To: SNIR@watson.ibm.com Sorry for the last minute reply. This is a feedback on the comments that were sent on the forum. ------ 15:37,48 20 and 21 should be 13. 19:41. We should probably state, indeed, that if a message that is shorter than the receive buffer arrives, than only the locations set to contain incoming data are modified. I can think of situations where a more lenient statement could allow for some optimizations but, in any case, the implementation should be ready to end a copy into the receiver memory exactly at the end of the receive buffer, even if it is an odd address. 34:20 We traverse the queue from the head toward the tail. Is this backward or forward? 35:30-31 Should be "On the other hand, the send-complete..." 39 Successful return of WAIT after IBSEND implies that the user send buffer can be reused -- i.e. data has been sent out or copied into buffer attached with BUFFER_ATTACH. Note that, at this point, we cannot cancel anymore the send. If a matching receive is not posted ever, then the buffer cannot be freed. This runs somewhat accounter the stated goal of MPI_CANCEL (being always able to free program space that was committed to the communication subsystem), but may be an acceptable behavior as BUFFER_ATTACH is providing static storage out of the heap. 62:5 4 . 16 should be 4 x 16. 68:34 MPI_TYPE_SIZE returns the total size of the entries in the type signature associated with the datatype argument; i.e. the total size of the data in a message that would be created with such datatype. Entries that occur multiple times in the datatype are counted with their multiplicity. 68:27, 70:14, 70:26 Should all be int and NOT MPI_Aint: MPI_Aint is used for variables that may store absolute addresses; type extent and displacements to start and end of datatype are relative displacements. Note that the rule on 75:5 implies that it is erroneous to call these three functions with a datatype argument that contains absoluted addresses, unless all these addreses are within the same sequential storage. We should state this explicitly. 184, example 6.4, as corrected by Steve looks correct to me: CALL MPI_CART_SHIFT(comm, 1, coords(2), source, dest, ierr) 190, 193 I think we are getting too sophisticated with the LEN argument. We have a MPI_MAX_ERROR_STRING constant, and an MPI_MAX_PROCESSOR_NAME constant. It should be the programmer responsibility to provide a string/CHARACTER argument that has at least that length (including, in C, the terminating null). I suggest the following text: "The routine returns the name of the processor ... In C this string is null terminated. In Fortran, the name is blank padded, if shorter than the character argument. The call is erroneous if the input string has length less than MPI_MAX_PROCESSOR_NAME." And similarly, for error strings. Of course, a good quality Fortran implementation will just truncate the name if input string is too short. My rationale is that: 1. It is not too painful to provide oversized strings. 2. It is more error-prone to make sure that the input string has length >= LEN as to make sure it has length >= MPI_MAX_PROCESSOR_NAME (constant). 3. If somebody wants to do pretty printing in Fortran, let him use a function that deletes trailing blanks. Good quality Fortran implementation will truncate. From owner-mpi-core@CS.UTK.EDU Fri Apr 15 04:16:06 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id EAA19601; Fri, 15 Apr 1994 04:16:03 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id EAA08081; Fri, 15 Apr 1994 04:16:45 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 15 Apr 1994 04:16:43 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id EAA08069; Fri, 15 Apr 1994 04:15:58 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA17042; Fri, 15 Apr 94 01:15:56 -0700 Date: Fri, 15 Apr 94 01:15:56 -0700 From: Steve Otto Message-Id: <9404150815.AA17042@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: MPI Version 1.001 Dear MPIers, OK, I've tried mighty hard to process all the comments on the spec that came across mpi-core. I have put together a new version and it should be available later today, April 15. I enclose my "stream of consciousness" as I worked on it --- I tried to reply to all remarks. Todays BIG LESSON: Code examples are bug-ridden and must be checked out with compilers and a working MPI implmentation! The number of errors in the examples is disheartening --- and there are no doubt many remaining. It also tells me something else about MPI -- we need higher-level tools for it, but that's another story. For the ARM: We MUST compile/run every code fragment. I volunteer to do C, who will do Fortran? BIG Thanks to reviewers: Steve Huss-Lederman, Marc Snir, Rolf Hempel, Jim Cownie, Hubertus Franke, Tony Skjellum, Nathan Doss. --Steve encl: stream of consciousness >From snir@watson.ibm.com Wed Apr 6 11:59 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA06014; Wed, 6 Apr 94 11:59:13 -0700 Return-Path: Received: from watson.ibm.com by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pocoh-000KshC; Wed, 6 Apr 94 11:59 PDT Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2123; Wed, 06 Apr 94 14:59:09 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 9149; Wed, 6 Apr 1994 14:59:07 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Wed, 06 Apr 94 14:59:07 EDT Received: by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA28293; Wed, 6 Apr 1994 14:59:06 -0400 >From: snir@watson.ibm.com (Marc Snir) Message-Id: <9404061859.AA28293@snir.watson.ibm.com> To: mpi-core@cs.utk.edu Cc: otto@cse.ogi.edu, lusk@mcs.anl.gov, Rolf.Hempel@gmd.de Subject: MPI draft -- language binding Reply-To: snir@watson.ibm.com Date: Wed, 06 Apr 94 14:59:06 -0500 Status: RO :-) :-) :-) *** (-: (-: (-: 1. Need also Fortran prototypes for user-defined functions (really, example of definitions) in the language binding. The prptotypes are in the main text. Done, except for MPI_Handler_function. I'm not sure I understand this case. Even in C, just because the args are implementation dependent, it doesn't necessarily mean they are variable or "stdargs" 2. The naming scheme for these prototypes is not consistent. I suggest MPI_Copy_function MPI_Delete_function MPI_Handler_function MPI_User_function All capitalized in the same way, and all with suffix "function". done Naming: Marc says --- 3. I got some objections about MPI_Graphdims_get and MPI_Cartdim_get. One suggestion is MPI_Graph_get_dim and MPI_Cart_get_dim. Another is MPI_Graph_size and MPI_Cart_size, which is more in line with MPI_Comm_size and MPI_Group_size. 4. We may also want to replace MPI_Graph_get with MPI_Comm_graph, in analogy with MPI_Comm_group, and MPI_Cart_get with MPI_Comm_cart. Rolf says -------- On another vein, I have no strong opinion about the function names of MPI_GRAPHDIMS_GET etc. However, I don't like MPI_CART_SIZE because it is a little misleading. The function does only return the number of dimensions, not the size of the topology which would be given by the dimensions themselves. So, people might confuse MPI_CART_SIZE with MPI_CART_GET. The names MPI_GRAPH_GET_DIMS and MPI_CART_GET_DIM would be okay, only that they are a little long and complicated. I don't think we should replace MPI_GRAPH_GET with MPI_COMM_GRAPH. Assume one wants to retrieve the topology information. The first thing is to look up the dimensions by calling MPI_GRAPH_GET_DIMS or however this function will be called. This suggests that the topology is already retrieved from the communicator. Thus, a later call to MPI_COMM_GRAPH looks strange to me. Again, I'm not religious about function names, but we should change what we have now only if we make the life of the users easier. It should be easy to find the correspondence between complementary functions by looking at their names, as is the case now with MPI_CARTDIM_GET and MPI_CART_GET. Personally I would vote for leaving the names as they are. I think we should leave the names as they are. I remember feeling that the names were somewhat confusing when I read the topology chapter (but they weren't wrong), and I'm not convinced the schemes above are really any clearer. It might have been just as clear to name them "MPI_42," MPI_43, etc! ------------------------------------------------------------- Jim Cownie: Real issues to argue over... Page:line Issue 68:34 MPI_TYPE_SIZE. What does this actually return ? An example would be useful. The current definition is sort of recursive and hard to understand (at least I find it so !) I assume that it means the number of bytes of actual data described by the datatype. (But what about a type which references the same location more than once ???) Marc answered this below (far below!) 165 We should say which MPI calls are valid in copy and delete functions, and whether they can refer to the communicator on which they are being applied. (I believe that there need be no restrictions, apart from deleting the communicator, and the communicator should still be valid to use. I think this is OK). Tony, do you want to suggest a sentence or two? 190 and 193 Fortran binding of MPI_GET_PROCESSOR_NAME and MPI_ERROR_STRING is wrong. (It is a bad idea to put the null character into a Fortran string, since then you can't just print it) I believe it should look something like this. After talking to Bill Gropp, we have decided upon a "compromise" sol'n that has the advantage of having the same argument lists in both C and Fortran and I think it also addresses Jim Cownie's concerns. MPI\_ERROR\_STRING was handled in a similar way. Here is what the spec now says: --------------------------------------- \begin{funcdef}{MPI\_GET\_PROCESSOR\_NAME( name, resultlen )} \funcarg{\OUT}{name}{A unique specifier for the actual (as opposed to virtual) node.} \funcarg{\OUT}{resultlen}{Length (in printable characters) of the result returned in \code{name}} \end{funcdef} \mpibind{MPI\_Get\_processor\_name(char~*name, int~*resultlen)} \mpifbind{MPI\_GET\_PROCESSOR\_NAME( NAME, RESULTLEN, IERROR)\fargs CHARACTER*(*) NAME\\INTEGER RESULTLEN,IERROR} This routine returns the name of the processor on which it was called at the moment of the call. The name is a character string for maximum flexibility. From this value it must be possible to identify a specific piece of hardware; possible values include ``processor 9 in rack 4 of mpp.cs.org'' and ``231'' (where 231 is the actual processor number in the running homogeneous system). The argument \mpiarg{name} must represent storage that is at least \const{MPI\_MAX\_PROCESSOR\_NAME} characters long. \func{MPI\_GET\_PROCESSOR\_NAME} may write up to this many characters into \mpiarg{name}. The number of characters actually written is returned in the output argument, \mpiarg{resultlen}. \begin{rationale} This function allows \MPI/ implementations that do process migration to return the current processor. Note that nothing in \MPI/ {\em requires} or defines process migration; this definition of \mpifunc{MPI\_GET\_PROCESSOR\_NAME} simply allows such an implementation. \end{rationale} \begin{users} The user must provide at least \const{MPI\_MAX\_PROCESSOR\_NAME} space to write the processor name --- processor names can be this long. The user should examine the ouput argument, \mpiarg{resultlen}, to determine the actual length of the name. \end{users} --------------------------------------- Typos ===== Page:line Change All fortran examples have THEN on next line from IF. This is incorrect unless there is a continuation marker. (if without a then on the same line is a single statement if, presence of the "then" turns it into a block if). done Cover Please add "(PPPE)" after P6643 done Acknowledgements:43 "Steve Zenith" should read "Steven Zenith" (!) Oh darn I was hoping to upset him. 13:18 "erroneuous" should read "erroneous" done 16:16 "to efficiently" should read "efficiently to" done sorta: "to transfer efficiently" is what I chose. 18:34 :" should (maybe) be ": didn't 29:33 A spare "," after "mechanism.)" done 39:11/12 Missing "does" over line break. (Currently reads "It not indicate") done 41:9 "the request will complete successfully" should read "the request will be allowed to complete" (We can't guarantee successful completion of the request !) done 49:22 "until it is received" should read "before it is received" done. 50: Labels in Fortran examples are not sufficiently outdented (at least if this is fixed format...) yeah. haven't done anything here yet. 51:27 "using a call to MPI_WAIT or MPI_TEST" should read "using a call to MPI_REQUEST_FREE, MPI_WAIT or MPI_TEST" done 62:5 "starting 4.16 apart." I don't know what this should read ! done. "with a stride of 4 elements (4*16 bytes) between the blocks." 68:15 "the common case" should read "commonly the case" done 83:13 "to be send" should read "to be sent" done 83:20 "operations facilitate" should read "operations facilitates" done 85:7 "contains now" should read "now contains" done 85:11 "bytes that consists" should read "bytes that forms" done 85:22 "two packing unit" should read "two packing units" done 90:17 "are the following." should read "are the following :" done 112:47 Table heading needs to move to next page. done 117:24 Fortran example is missing "END DO" "END IF" done 118:38 "a op" should read "an op" done 125:25 etc Tabs in \verbatim don't work ! yes and I try to be so careful. sigh. 130:20 "help ensure" should read "help to ensure" done 131:19 "on par with" should read "on a par with" done 131:30 "used to invoke" should read "used in" done 131:36,46 Inconsistent use of font for bullet items (and over page) done 146:3 "attributed" should read "attributes" done 152:36 "backmasking cannot occur" should read "backmasking cannot be required" done 156:16 "exists where both" should read "exists to which both" done 156:26 "new children processes" should read "new child processes" done 166:12 "arbitrary communicator" should read "arbitrary communicators" done 166:40 Some \_ have escaped into a verbatim section done 167:34 Ditto done 169:43 Empty advice to implementors done 190 See above re MPI_GET_PROCESSOR_NAME done 191:29 "In such case" should read "in this case" done 192:16 "Such call" should read "This call" done 193 See above re MPI_ERROR_STRING done 195:10 C binding for MPI_Wtime should read double MPI_Wtime(void) done 195:34 ditto for MPI_Wtick double MPI_Wtick(void) There is a difference in ANSI C between a prototype without any arguments "foo()" (assumed to be a K&R external declaration, and so accepting ANY argument list, and a declaration just of "foo(void)" an ANSI declaration of a function with NO arguments. We want the latter.) good point. 195:24 Same for MPI_Finalize done That's all folks -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com >From jim@meiko.co.uk Fri Apr 8 09:45 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA15249; Fri, 8 Apr 94 09:45:21 -0700 Return-Path: Received: from hub.meiko.co.uk by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0ppJgE-000KsaC; Fri, 8 Apr 94 09:45 PDT Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA13525 (5.65c/IDA-1.4.4 for otto@cse.ogi.edu); Fri, 8 Apr 1994 17:44:26 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA00351; Fri, 8 Apr 1994 17:42:21 +0000 Date: Fri, 8 Apr 1994 17:42:21 +0000 >From: jim@meiko.co.uk (James Cownie) Message-Id: <9404081642.AA00351@tycho.co.uk> To: mpi-core@CS.UTK.EDU Cc: otto@cse.ogi.edu Subject: One more typo Content-Length: 59 Status: RO Page 195 line 18 ''ticks'' should read ``ticks'' done -- Jim >From tony@Aurora.CS.MsState.Edu Fri Apr 8 13:39 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA16548; Fri, 8 Apr 94 13:39:55 -0700 Return-Path: Received: from Aurora.CS.MsState.Edu by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0ppNLF-000KspC; Fri, 8 Apr 94 13:39 PDT Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA23665; Fri, 8 Apr 94 15:34:42 CDT Date: Fri, 8 Apr 94 15:34:42 CDT >From: Tony Skjellum Message-Id: <9404082034.AA23665@Aurora.CS.MsState.Edu> To: otto@cse.ogi.edu Subject: F binding of MPI_UNPACK Status: RO I have a source argument in the slightly old version of the F binding of MPI_UNPACK. It appears only in argument list. It should go if still present. -Tony done >From owner-mpi-core@CS.UTK.EDU Fri Apr 8 13:36 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA16538; Fri, 8 Apr 94 13:36:50 -0700 Return-Path: Received: from CS.UTK.EDU by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0ppNIG-000KsqC; Fri, 8 Apr 94 13:36 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA28601; Fri, 8 Apr 1994 16:27:54 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 8 Apr 1994 16:27:53 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA28593; Fri, 8 Apr 1994 16:27:51 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 6621; Fri, 08 Apr 94 16:27:49 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 1331; Fri, 8 Apr 1994 16:27:49 EDT Received: from bubba.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 08 Apr 94 16:27:48 EDT Received: by bubba.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA33831; Fri, 8 Apr 1994 16:27:47 -0400 Message-Id: <9404082027.AA33831@bubba.watson.ibm.com> To: mpi-core@CS.UTK.EDU Cc: otto@cs.ogi.edu Subject: Draft 3/31 comments Date: Fri, 08 Apr 94 16:27:47 -0500 >From: "Hubertus Franke" Status: RO I have a few comments on the 3/31 draft. pg 139: MPI_Group_range_incl and MPI_Group_range_excl should have the following prototype. ...(MPI_Group group, int n, int ranges[][3], MPI_Group newgroup); int** is not the same as int[][3] !!!!! yes, I'm amazed that's still in there. Depressing. done. pg 195/3 MPI defines on timer. and NOT two !!! yes, done. Again I am impressed. pg 208/13 should be MPI_ERRORS_ARE_FATAL and MPI_ERRORS_RETURN done pg 210/13..18 should be deleted as they are specified in line 27..34 done pg 210/27..34 all names must be in brackets and names should be corrected. MPI_copy_function -> (MPI_Copy_function) MPI_delete_function -> (MPI_Delete_function) MPI_Handler_function -> (MPI_Handler_function) MPI_Uop -> (MPI_Uop) I've already corrected the names; I believe the extra ()'s are unnecessary pg 118/48 same as above MPI_Uop -> (MPI_Uop) nope pg 207/25: propose MPI_OK as a synonym for MPI_SUCCESS why all these long names. too late! pg 208..: missing definitions for MPI_2REAL, MPI_2DOUBLE_PRECISION, MPI_2INTEGER, MPI_FLOAT_INT, MPI_DOUBLE_INT, MPI_LONG_INT, MPI_2INT. done pg 51/20: There seems no obvious reason for being passed by reference. This would only make sense if the object behind the request will be exchanged. consistency with other calls If that is possible (why not) then I suggest an advice to the that he could do so. pg 55/20: There seems no obvious reason for being a pointer, in this case the object can not be exchanged in the background. That would be far fetched. >From frankeh@watson.ibm.com Mon Apr 11 06:10 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA25343; Mon, 11 Apr 94 06:10:13 -0700 Return-Path: Received: from watson.ibm.com by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pqLkh-000KsGC; Mon, 11 Apr 94 06:10 PDT Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9581; Mon, 11 Apr 94 09:10:11 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 8699; Mon, 11 Apr 1994 09:10:10 EDT Received: from bubba.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 11 Apr 94 09:10:10 EDT Received: by bubba.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA33732; Mon, 11 Apr 1994 09:10:07 -0400 Message-Id: <9404111310.AA33732@bubba.watson.ibm.com> To: mpi-core@cs.utk.edu Cc: otto@cse.ogi.edu, snir@watson.ibm.com Subject: Draft 3/31 omments Date: Mon, 11 Apr 94 09:10:07 -0500 >From: "Hubertus Franke" Status: RO More comments on the 3/31 draft: --------------------------------------------------- | A) TYPOS and inconsistency in cross referencing | --------------------------------------------------- pg. 208: MPI_PACKED missing for the Fortran interface done pg. 70/14 && 70/26: The displacement type should be coherent with the one in MPI_Type_struct etc. -> both or both . No -- see Marc Snir's rebuttal, below. pg. 214/39: MPI_op_free -> MPI_Op_free done ----------------------------------- | B) SERIOUS semantical question | ----------------------------------- Jim gives a good answer to this down below. pg. 39. Interaction between MPI_Wait/MPI_Test .. and MPI_BSEND is not clear. I can justify several interpretations from the draft. MPI_BSEND: 1) immediately buffer and return, the message will be completed in the background. Q: But for what do we need MPI_IBSEND then? 2) try to send and if (wait is issued or the message is blocking) and the message has not been completed then allocate a buffer, copy and complete in the background. Wait immediately returns with done. In this scenario MPI_IBSEND could make sense. => this model guarantees that buffer can be reused after wait. Also in question is the interaction between wait, buffered send and persistent handles. Let's suppose the wait is issued on the persistent handle. If we buffer and return as described in (2) then the handle has to be duped in the message kernel, as to allow to reuse the persistent handle for the next communication and to have the current communication complete in the background. If however MPI_WAIT waits till the communication is actually completed, why do we need persistent buffered sends? Regardless of how I look at this buffered beast, I can't get clean semantics => needs work !!!! Hope this can be clarified before G-Day. ------------------------------------------------------------------ Hubertus Franke IBM T.J. Watson Research Center P.O.-Box 218 Yorktown Heights, NY 10598 >From jim@meiko.co.uk Mon Apr 11 07:14 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA25407; Mon, 11 Apr 94 07:14:37 -0700 Return-Path: Received: from hub.meiko.co.uk by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pqMkz-000KsNC; Mon, 11 Apr 94 07:14 PDT Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA14137 (5.65c/IDA-1.4.4 for otto@cse.ogi.edu); Mon, 11 Apr 1994 15:13:29 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA01645; Mon, 11 Apr 1994 15:11:18 +0000 Date: Mon, 11 Apr 1994 15:11:18 +0000 >From: jim@meiko.co.uk (James Cownie) Message-Id: <9404111411.AA01645@tycho.co.uk> To: frankeh@watson.ibm.com Cc: mpi-core@CS.UTK.EDU, otto@cse.ogi.edu, snir@watson.ibm.com In-Reply-To: <9404111310.AA33732@bubba.watson.ibm.com> (frankeh@watson.ibm.com) Subject: Re: Draft 3/31 omments Content-Length: 3577 Status: RO > ----------------------------------- > | B) SERIOUS semantical question | > ----------------------------------- > > pg. 39. Interaction between MPI_Wait/MPI_Test .. and MPI_BSEND > is not clear. I can justify several interpretations > from the draft. > > MPI_BSEND: > 1) immediately buffer and return, the > message will be completed in the background. > > Q: But for what do we need MPI_IBSEND then? Suppose that you are on a multi-processor machine, then you could return in the IBSEND case BEFORE copying the data, and awake another thread to do the copying. IBSEND will then return sooner than BSEND would even if you always copy. As ever WAIT waits for the send buffer to be re-usable. It says NOTHING about the message having actually been sent. (Exactly the same semantics as a standard mode send). > 2) try to send and if (wait is issued or the message > is blocking) and the message > has not been completed then allocate a buffer, > copy and complete in the background. > Wait immediately returns with done. > In this scenario MPI_IBSEND could make sense. > => this model guarantees that buffer can be reused > after wait. I think this is just an optimisation of the previous case to avoid the copy if it is not (actually) required. The user model should be as above. (This is necessary for the explanation of the buffer exhaustion case to be simple). In either case the IBSEND variant potentially has value. (Though as you point out if the implementation really is as in case 1 above it may not be a great benefit, aside from the fact that it completes the table of send/isend pairs, and would be hard to justify it NOT being there). > Also in question is the interaction between wait, buffered > send and persistent handles. > Let's suppose the wait is issued on the persistent handle. > If we buffer and return as described in (2) > then the handle has to > be duped in the message kernel, as to allow to > reuse the persistent handle for the next communication > and to have the current communication complete in the > background. > > If however MPI_WAIT waits till the > communication is actually completed, why do we need > persistent buffered sends? > > Regardless of how I look at this buffered beast, I can't > get clean semantics => needs work !!!! The wait here should mean the same as it always did, i.e. it is a LOCAL property, solely meaning that the input buffer can be trashed. I believe that the effect is like this MPI_Start_Bsend(...) { if (can send without blocking) { mpi_isend( same args ); save descriptor for this isend in original bsend block. } else { do the copy (or schedule it); mpi_isend(copied buffer); save descriptor for isend on global list, will be picked up by the buffer allocator later. } } MPI_Bsend_complete(...) { if (there's a send descriptor here) return MPI_Status(that one) else return TRUE; /* It was the copy case, so it must be done... */ } Is this too horrible ? -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com >From owner-mpi-core@CS.UTK.EDU Mon Apr 11 06:19 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA25355; Mon, 11 Apr 94 06:19:17 -0700 Return-Path: Received: from smoked.cse.ogi.edu by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pqLtU-000KsJC; Mon, 11 Apr 94 06:19 PDT Received: from CS.UTK.EDU by smoked.cse.ogi.edu with smtp (Smail3.1.28.1 #1) id m0pqLtS-0000daC; Mon, 11 Apr 94 06:19 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA16103; Mon, 11 Apr 1994 09:13:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Apr 1994 09:13:07 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA16096; Mon, 11 Apr 1994 09:13:05 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA25440; Mon, 11 Apr 94 09:13:03 EDT Date: Mon, 11 Apr 94 09:13:03 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404111313.AA25440@super> Received: by b125.super.org (4.1/SMI-4.1) id AA07402; Mon, 11 Apr 94 09:13:02 EDT To: mpi-core@CS.UTK.EDU Subject: draft comments #1 Status: RO Howdy MPIers, With time running short, I will send in my comments as I finish. Attached is what I saw from a slow reading of the first 50 pages. Jim and I found only one mistake that was the same (not duplicated below). I don't think the rest overlap. This is a little scary about how good my slow reading was. Steve ---------------------------------------------------------------------- 9:4 - innaccessible -> inaccesible you got it wrong ;^) but done 9:33 - MPI_ERRORFATAL -> MPI_ERRORS_ARE_FATAL as defined on 191:18 done 9:34 - MPI_ERRORSRETURN -> MPI_ERRORS_RETURN as defined on 191:22 done 9:45 - This is a general comment that I think goes here. In the FORTRAN bindings, the document uses (ex. 16:41) for a choice variable; (void *) in C (11:36). I did not notice anywhere that we clearly state what this means. I wonder if it should go here for completeness. done 10:28 - Did we say that implementors could not use MPI_ for other routines? Excluding this would leave all the names open for future use by MPIF. On the other hand, a logical extension of MPI might use the MPI_ name. I thought we voted on this but cannot recall the outcome for sure. I don't know; I punt. 10:31/11:29 - The FORTRAN binding (10:31) say "error codes are only potentially implementation dependent; see Chapter 7." The C binding say (11:29) "failure return codes are implentation dependent." These are somewhat different. p. 194 defines error codes/classes so they are both "standard" and implementation dependent. It isn't real important but I think these statements should be the same since they are the same in the two languages. I made it more consistent -- thanks for seeing this. 14:5 - neither requirements or -> neither requirements nor done 14:22 - form -> from done 15:37,48 - I am a little confused by the message[20]. on line 37 it has an MPI_Recv with a size of 21. Since message is size [20] should the receive also be 20 (or is the \n somehow extra). Also, line 48 says the message sent is 21 characters but the send on line 33 uses strlen(message) so I think it only sends 13 characters with the \n. it was wrong, you should be confused! the recv is now: recv(message, 20, ...) and the text says that 13 characters are sent. (This is C, and the null termination is there) 19:41 - Do we want to make a statement that if the message is shorter then the buf, then values beyond the receive count are not touched in buf? yep. see Snir's response below. 25:3 - Should "No conversion occurs" -> "No conversion need occur". It might be possible on an implementation designed for heterogeneous systems that they alway use XDR (for example). I think it would be inefficient but I think it would be ok. ok. done 33:22,23,26 - It is not clear to me why MPI_BUFFER_DETACH has buffer and size as OUT arguments. It seems like you are telling the system the start address and size of the message queue (I would think the system has to know the size). I would not think MPI changes the pointer (it does not deallocate it) and certainly not the size. Along the same lines, I think there is an extra * in the C binding. Thus, "void** buffer, int* size" -> "void* buffer, int size" Jim gave (I think) the right answer. see below. 34:5 - etc.. -> etc. ? done. 34:20 - Should backward -> forward. I'm not completely sure what is being done, but I would think you start at the head (the oldest) and search forward for messages that are done in the buffer. "traverse from head toward tail" 35:30-33 - It says "On the other hand..". It was confusing to me about what was different between the first and second sentence since they refer to send-complete and send. I may just be confused myself. we changed second "send" to "send-complete" 52:44 - send -> sent. done >From owner-mpi-core@CS.UTK.EDU Thu Apr 14 19:47 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA16320; Thu, 14 Apr 94 19:47:58 -0700 Return-Path: Received: from CS.UTK.EDU by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0prdwi-000KrvC; Thu, 14 Apr 94 19:47 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id WAA15676; Thu, 14 Apr 1994 22:43:27 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 14 Apr 1994 22:43:27 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id WAA15669; Thu, 14 Apr 1994 22:43:25 -0400 Message-Id: <199404150243.WAA15669@CS.UTK.EDU> Received: from YKTVMV by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 4861; Thu, 14 Apr 94 22:43:20 EDT Date: Thu, 14 Apr 94 22:35:49 EDT >From: "Marc Snir ((914) 945-3204)" To: mpi-core@CS.UTK.EDU, cc:otto@cse.ogi.edu Subject: MPI errata Reply-To: SNIR@watson.ibm.com Status: R Sorry for the last minute reply. This is a feedback on the comments that were sent on the forum. ------ 15:37,48 20 and 21 should be 13. I changed the recv to 20; and the text now says that 13 characters are sent. 19:41. We should probably state, indeed, that if a message that is shorter than the receive buffer arrives, than only the locations set to contain incoming data are modified. I can think of situations where a more lenient statement could allow for some optimizations but, in any case, the implementation should be ready to end a copy into the receiver memory exactly at the end of the receive buffer, even if it is an odd address. statements added. 34:20 We traverse the queue from the head toward the tail. Is this backward or forward? ok. 35:30-31 Should be "On the other hand, the send-complete..." done 39 Successful return of WAIT after IBSEND implies that the user send buffer can be reused -- i.e. data has been sent out or copied into buffer attached with BUFFER_ATTACH. Note that, at this point, we cannot cancel anymore the send. If a matching receive is not posted ever, then the buffer cannot be freed. This runs somewhat accounter the stated goal of MPI_CANCEL (being always able to free program space that was committed to the communication subsystem), but may be an acceptable behavior as BUFFER_ATTACH is providing static storage out of the heap. paragraph added to rationale on page 39 62:5 4 . 16 should be 4 x 16. \cdot used. 68:34 MPI_TYPE_SIZE returns the total size of the entries in the type signature associated with the datatype argument; i.e. the total size of the data in a message that would be created with such datatype. Entries that occur multiple times in the datatype are counted with their multiplicity. This explanation added. 68:27, 70:14, 70:26 Should all be int and NOT MPI_Aint: MPI_Aint is used for variables that may store absolute addresses; type extent and displacements to start and end of datatype are relative displacements. Note that the rule on 75:5 implies that it is erroneous to call these three functions with a datatype argument that contains absoluted addresses, unless all these addreses are within the same sequential storage. We should state this explicitly. OK. I have added an explicit statement (in a rationale) here it is: \begin{rationale} Note that the rules given in Sec. \ref{subsec:pt2pt-segmented} imply that it is erroneous to call \func{MPI\_TYPE\_EXTENT}, \func{MPI\_TYPE\_LB}, and \func{MPI\_TYPE\_UB} with a datatype argument that contains absolute addresses, unless all these addreses are within the same sequential storage. For this reason, the \mpiarg{displacement} for the C binding in \func{MPI\_TYPE\_UB} is an \ctype{int} and not of type \ctype{MPI\_Aint}. \end{rationale} I must side a bit with Franke, however...it looks asymmetric as it now is: displacements fed into HVECTOR are of type MPI_Aint, while the displacement that comes out of MPI\_TYPE\_UB is not a MPI_Aint. 184, example 6.4, as corrected by Steve looks correct to me: CALL MPI_CART_SHIFT(comm, 1, coords(2), source, dest, ierr) Yes, I think so...let me see how Rolf likes it...he seems to like it. 190, 193 I think we are getting too sophisticated with the LEN argument. We have a MPI_MAX_ERROR_STRING constant, and an MPI_MAX_PROCESSOR_NAME constant. It should be the programmer responsibility to provide a string/CHARACTER argument that has at least that length (including, in C, the terminating null). I suggest the following text: "The routine returns the name of the processor ... In C this string is null terminated. In Fortran, the name is blank padded, if shorter than the character argument. The call is erroneous if the input string has length less than MPI_MAX_PROCESSOR_NAME." And similarly, for error strings. YES. Good, Bill Gropp and I agreed on this today. It is the programmer responsibility to provide a buffer of the MAX_... size. We also added an OUT argument that tells you the actual size. Of course, a good quality Fortran implementation will just truncate the name if input string is too short. My rationale is that: 1. It is not too painful to provide oversized strings. 2. It is more error-prone to make sure that the input string has length >= LEN as to make sure it has length >= MPI_MAX_PROCESSOR_NAME (constant). 3. If somebody wants to do pretty printing in Fortran, let him use a function that deletes trailing blanks. Good quality Fortran implementation will truncate. >From gmap10@f1neuman.gmd.de Wed Apr 13 23:21 PDT 1994 Received: from iliamna.cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA10309; Wed, 13 Apr 94 23:21:23 -0700 Return-Path: Received: from gmdzi.gmd.de by iliamna.cse.ogi.edu with smtp (Smail3.1.28.1 #1) id m0prKnf-0002F2C; Wed, 13 Apr 94 23:21 PDT Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA04706 (5.65c8/IDA-1.4.4 for ); Thu, 14 Apr 1994 08:21:11 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA17013; Thu, 14 Apr 1994 08:21:10 +0200 Date: Thu, 14 Apr 1994 08:21:10 +0200 >From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9404140621.AA17013@f1neuman.gmd.de> To: otto@iliamna.cse.ogi.edu Subject: topology chapter Cc: gmap10@f1neuman.gmd.de, lederman@b125.super.org Status: RO Steve, I was relieved to see that we now agree on the funny topology example. Perhaps the best thing will be to correct it as I indicated in my previous mail, i.e., by replacing line 184:47 with CALL MPI_CART_SHIFT(comm, 1, coords(2), source, dest, ierr). This way we have a more sophisticated example than I suggested in my mail of April 7, and we can leave the text as it is. Yes, I'm sticking with the code above. it works... Steve Huss-Lederman's suggestion to replace the words "non-increasing" in 178:12 with "monotonically decreasing" seems to me the best solution. I'm leaving it --- non-increasing was correct. So, now everything seems to be fine. I hope there are no more bugs in the text. Many thanks to Steve Huss-Lederman for reading the text so carefully! - Rolf >From owner-mpi-core@CS.UTK.EDU Mon Apr 11 07:29 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA25435; Mon, 11 Apr 94 07:29:47 -0700 Return-Path: Received: from CS.UTK.EDU by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pqMzh-000KsQC; Mon, 11 Apr 94 07:29 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA23037; Mon, 11 Apr 1994 10:25:54 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Apr 1994 10:25:52 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA23021; Mon, 11 Apr 1994 10:25:49 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA14515 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Mon, 11 Apr 1994 15:24:55 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA01667; Mon, 11 Apr 1994 15:22:48 +0000 Date: Mon, 11 Apr 1994 15:22:48 +0000 >From: jim@meiko.co.uk (James Cownie) Message-Id: <9404111422.AA01667@tycho.co.uk> To: lederman@super.org Cc: mpi-core@CS.UTK.EDU In-Reply-To: <9404111313.AA25440@super> (lederman@super.org) Subject: Re: draft comments #1 Status: RO > 33:22,23,26 - It is not clear to me why MPI_BUFFER_DETACH has buffer > and size as OUT arguments. It seems like you are telling the system > the start address and size of the message queue (I would think the > system has to know the size). I would not think MPI changes the > pointer (it does not deallocate it) and certainly not the size. Along > the same lines, I think there is an extra * in the C binding. Thus, > "void** buffer, int* size" -> "void* buffer, int size" I think the idea here is that you (the user) may need to free the buffer after MPI has finished using it, and this may happen in a different piece of code than where you allocated it (so you may no longer have the buffer in your hand). Therefore it makes sense for MPI to give you back as much information as you gave it about the buffer. (Otherwise you have to remember this somewhere yourself). That's why these are OUT arguments, you're not telling MPI anything about the buffer at this point, rather it is giving you back the info you gave it in the BUFFER_ATTACH call. If so the arguments are correct both in intenta and type. Sounds good. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com >From owner-mpi-core@CS.UTK.EDU Tue Apr 12 07:17 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA00296; Tue, 12 Apr 94 07:17:25 -0700 Return-Path: Received: from CS.UTK.EDU by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pqjHA-000KrpC; Tue, 12 Apr 94 07:17 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA14985; Tue, 12 Apr 1994 10:09:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 12 Apr 1994 10:09:07 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA14975; Tue, 12 Apr 1994 10:09:04 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA07879; Tue, 12 Apr 94 10:09:02 EDT Date: Tue, 12 Apr 94 10:09:02 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404121409.AA07879@super> Received: by b125.super.org (4.1/SMI-4.1) id AA07501; Tue, 12 Apr 94 10:09:02 EDT To: mpi-core@CS.UTK.EDU Subject: draft comments #2 Status: RO Hello again, I finished pt-2-pt last night. The comments below are on pages 53-89. I give Marc credit for lots of interesting examples; I thought I would never finish understanding them all! Given the number of things I saw, there must be more. Hope others are giving it a careful reading. I'm sure I have made some mistakes in my comments. Steve ---------------------------------------------------------------------- 55:6 - dest -> source Yes! done 55:35 - four -> five? I think there is one more from adding MPI_BSEND_INIT. done 58:27 - "is what would obtain" -> "is what would be obtained" done 63:19 - STRIDE in the FORTRAN definition is an address (MPI_Aint in the C definition). 11:15 says that mpif.h in FORTRAN should define MPI_ADDRESS if possible. Since a default integer and MPI_ADDRESS might be different, I think we should use MPI_ADDRESS for things that are addresses in the FORTRAN binding. I think it is much easier to to take MPI_ADDRESS into integer than visa versa sine there are lots of intergers this could happen to. This comment also holds for: 65:30,41 - array_of_displacements 66:29,40 - array_of_displacements 67:41,46 - address 68:25,30 - extent 68:37,41 - size Oh groan. More of this Aint trouble. The trouble is that it isn't standard Fortran to say MPI_ADDRESS... and so we haven't. Therefore we leave it as it is...it's a bit fuzzy, I agree, but Fortran is a pain in this way. 69:39 - I'm not sure I gotten the definition of lb(Typemap) figured out. My concern is with the second condition (otherwise) where type_j = lb. Supposae one combines several derived datatypes into a new datatype and some had lb as a type and some did not. If a disp_j of a datatype where type_j != lb is smaller than any of the disp_k where type_k=lb, then should the lb(Typemap) be that disp_j? If so, then No -- the new lb will be from the set of lbs. the otherwise clause which restricts type=lb seems too restrictive. This comment also holds (for max) to 69:43 for ub(Typemap). Is this right or am I confused as usual? The key is that the Typemap is the set of basic types, not derived types. Therefore, one should think of the LBs to be entries at the basic type level. If we combine some types, some of which have LBs and some of which do not, then at the low level, the basic types in the typemap, it is unambiguous -- we look only at the LB entries to define the new LB. I think it's ok. 70:11,14 - MPI_TYPE_LB returns a displacement. The other functions for extent and derived datatypes return MPI_Aint*. Here it returns int*. Should it be MPI_Aint for consistancy. (The MPI_ADDRESS also holds for FORTRAN definition if changed to MPI_Aint in C.) See discussion elsewhere. 70:23,26 - same comment for MPI_TYPE_UB as above. See discussion elsewhere. 72:9 - datatype. -> datatype, done 75:34 - "MPI_STATUS status" -> "int status(MPI_STATUS_SIZE)" as yes. good. It was (and is) defined in the section \subsection{Return status}. defined for MPI_SENDRECV on 57:33. This raises a point I missed earlier. I cannot find where MPI_STATUS_SIZE is definied. I think it is first used in MPI_WAIT on p. 39. I cannot find a discussion of MPI_STATUS_SIZE and MPI_Status in Ch. 2. This comment also applies to staus on: 76:12 76:36 77:12 done 78:18 - I think the code: for(i=0; i <3; i++) disp[i] -= disp[0]; is incorrect. It has a classic vector type dependence. What is intended is to make all the displacements relative to disp[0] and disp[0]=0. However, i=0 makes disp[0]=0 first, and all the other disp[i] are therefore unchanged. I think it will work if you run the loop backward: for(i=2; i >=0; i--) disp[i] -= disp[0]; Oh christ. You've earned another gold star!! I've changed the code to: int base; ... base = disp[0]; for (i=0; i <3; i++) disp[i] -= base; The same comment applies to: 78:35 80:35 done 80:20 - MPI_REAL -> MPI_DOUBLE since particle.d is double done, and because we are writing in C, and MPI_REAL is only for Fortran. Also needed: MPI_Type_extent( Particletype, &sizeofentry); just above ... 80:22: I would change particle.d -> particle[0].d. This makes it consisant with other places (80:33, 81:14, ...) and is a little clearer. It also makes it correct. I'm afraid this code is buggy. 82:39 - need a ; to terminate the C expression: disp[1] = j-i -> disp[1] = j-i; done 85:11 - "consists the packing" -> "consists of the packing" done 86:2 - MPI_PACK_SIZE(incount, datatype, size) -> MPI_PACK_SIZE(incount, datatype, comm, &size) Nope. this wasn't a C statement. 86:4 - MPI_PACK(...position) -> MPI_PACK(...&position, comm) ditto 86:10 - The int ... should be on a separate line then Example 3.36. Also applies to: 86:30 done 86:21,22: It now reads: MPI_PACK(buff, 1000, position, i, 1, MPI_INT); MPI_PACK(buff, 1000, position, j, 1, MPI_INT); It is clearly wrong. My guess is it should be: probably... MPI_PACK(&i, 1, MPI_INT, buff, 1000, &position, MPI_COMM_WORLD); MPI_PACK(&j, 1, MPI_INT, buff, 1000, &position, MPI_COMM_WORLD); done 87:11 - MPI_PACK(MPI_BOTTOM, 1, newtype, buff, position, MPI_COMM_WORLD) -> MPI_PACK(MPI_BOTTOM, 1, newtype, buff, 1000, &position, MPI_COMM_WORLD) done 87:38 - This is really a general comment. The examples use status in MPI_Recv, etc. It is of type MPI_Status*. Are we assuming it is declared as MPI_Status *status? If you do this then using no, status is in user space, hence it is declared as MPI_Status status; we are just missing an & in the Recv call: MPI_Recv( buff, 1000, MPI_PACKED, 0, 0, &status); is what we want. MPI_GET_COUNT is ugly since it uses MPI_Status (not MPI_Status *). The sample programs distributed so far (from ANL/MSU, mpif) use MPI_Status status and then use MPI_Recv(..., &status). I hate to say this, but this has been done throughout the chapter. Other variables are like this too. If people agree with my interpretation, I can go back and try and find them all. Ugh :-(. No don't panic (at least *I'm* trying not to!) --- Statuses don't live in MPI space hence we don't have the usual confusions about handles. They are just normal things that we declare like: MPI_Status status; 87:43 - MPI_Unpack(buff, 1000, position, i, 1, MPI_INT) -> MPI_Unpack(buff, 1000, &position, &i, 1, MPI_INT, MPI_COMM_WORLD) Yes, Marc forgets his &'s (sorry Marc! ;^) ) 87:47 - MPI_Unpack(buff, 1000, position, a, i, MPI_FLOAT) -> MPI_Unpack(buff, 1000, &position, a, i, MPI_FLOAT, MPI_COMM_WORLD) done 88:8 - MPI_Comm_size(comm, gsize) -> MPI_Comm_size(comm, &gsize) done 88:9 - MPI_Comm_rank(comm, myrank) -> MPI_Comm_rank(comm, &myrank) done 88:13 - MPI_Pack_size(1, MPI_INT, comm, k1) -> MPI_Pack_size(1, MPI_INT, comm, &k1) done 88:14 - MPI_Pack_size(count, MPI_CHAR, k2) -> MPI_Pack_size(count, MPI_CHAR, comm, &k2) done 88:26 - MPI_GATHER(&position, 1, MPI_INT, NULL, NULL, root, comm) -> MPI_GATHER(&position, 1, MPI_INT, NULL, NULL, NULL, root, comm) (need another NULL). Gather??! In pt2pt? Ok... 88:34 - MPI_GATHER(&position, 1, MPI_INT, counts, MPI_INT, root, comm) -> MPI_GATHER(&position, 1, MPI_INT, counts, 1, MPI_INT, root, comm) done. Your patience is impressive, Dr Huss-Lederman! 88:39 - do -> for (who let those FORTRAN people in here :-). Also applies to: 89:1 done 89:4 - MPI_Unpack(....,position, count,...) -> MPI_Unpack(....,&position, &count,...) done 89:6 - MPI_Unpack(..., position, ..) -> MPI_Unpack(..., &position, ..) done >From owner-mpi-core@CS.UTK.EDU Wed Apr 13 05:45 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA04888; Wed, 13 Apr 94 05:45:43 -0700 Return-Path: Received: from CS.UTK.EDU by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pr4K4-000KryC; Wed, 13 Apr 94 05:45 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id IAA00759; Wed, 13 Apr 1994 08:41:09 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 13 Apr 1994 08:41:06 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id IAA00742; Wed, 13 Apr 1994 08:41:01 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA19738; Wed, 13 Apr 94 08:40:51 EDT Date: Wed, 13 Apr 94 08:40:51 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404131240.AA19738@super> Received: by b125.super.org (4.1/SMI-4.1) id AA07556; Wed, 13 Apr 94 08:40:50 EDT To: mpi-core@CS.UTK.EDU Subject: comments #3 Status: RO Hello another time! Here are my comments on Chapters 6, 7 and 8. I assume Steve Otto is really starting to hate my e-mail. Steve ---------------------------------------------------------------------- 176:34,36 - This paragraph refers to the last section as having the *_MAP functions. The last section is actually the example. These are in the next to last section (6.5.7). (Now I'm getting picky!) that's fine. thanks. done. 178:12 - "dims[i] will be ordered in non-increasing order." Does this mean decreasing? If so, I prefer decreasing over non-increasing. and I prefer non-increasing. 180:36 - This is a question: Is nnodes returned by MPI_GRAPHDIMS_GET the same value returned by MPI_COMM_SIZE? If so, it would be nice to note. 182:22 - "0<=coords(i)<=dims(i)" -> "0<=coords(i) "MPI_CART_COORDS(comm, rank, maxdims, coords, ierr)" done 188:13,17,18,25,28,31,34 - All the MPI_* calls need to have ierr added as the last argument since these are FORTRAN calls. Also need to declare ierr as an integer. done 189:30 - MPI_UNDEFINED -> MPI_PROC_NULL as on 190:3 done 192:1,12 - definition refers to function (as do language binding ones) but the description says handler_function. Need to chose one. done 193:41/208:10 - The value MPI_MAX_ERROR_STRING is not listed in the appendix at 208:17. done 194:32 - I think it would be nice to give the reader the use of MPI_ERR_LASTCODE. It isn't a code returned by an MPI error. not done 196:28 - "This routine cleans all MPI state" -> "This routine cleans up all MPI state" done 200:41 - uses "int extent". By definition of MPI_Type_size on 68:39 this should be MPI_Aint. (I had to find something in the profile chapter :-) not done. >From owner-mpi-core@CS.UTK.EDU Wed Apr 13 08:47 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA05105; Wed, 13 Apr 94 08:47:32 -0700 Return-Path: Received: from CS.UTK.EDU by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pr7A2-000Ks4C; Wed, 13 Apr 94 08:47 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id LAA16288; Wed, 13 Apr 1994 11:39:42 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 13 Apr 1994 11:39:41 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id LAA16239; Wed, 13 Apr 1994 11:39:36 -0400 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA03554 (5.65c8/IDA-1.4.4 for ); Wed, 13 Apr 1994 17:39:27 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA16962; Wed, 13 Apr 1994 17:39:25 +0200 Date: Wed, 13 Apr 1994 17:39:25 +0200 >From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9404131539.AA16962@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: Steve's corrections Cc: gmap10@f1neuman.gmd.de Status: RO Hi there, Steve really did a great job at detecting mistakes in the document. I read over the text so many times, but it seems that after a while I don't look at the details carefully enough any more. Steve's corrections are okay except for the following comments: 178:12 - Non-increasing does not necessarily mean decreasing, since it also includes the case where entries are identical. You guys are really hung up on this, aren't you? 180:36 - Yes, that's true. Perhaps we should add something like (same as number of processes in the group). done 184:37 - Sorry, but I still have a problem with the example. We have to be very precise in what we mean with "shift column i by i steps". Steve seems to understand the example such that the topology consists of a number of columns, and in each column the data is shifted by a certain number of steps. This step width is set to be the column index, so that it is different in each column. However, if 2 is the shift direction (direction of the column), then the index of the column is orthogonal to that direction, namely coords(1). There is always the confusion between Fortran and C as to the definition of rows and columns, but my point is independent of that. One only has to replace the word "column" with "row". --------------------------------------------------------------- Another point of confusion is that people often mean different things by "in the column direction". Do you mean: 1) *along* a column 2) in the direction in which the column index is changing. --Steve O. ---------------------------------------------------------------- A way to save the old example would be to change line 184:47 to CALL MPI_CART_SHIFT(comm, 1, coords(2), source, dest, ierr). I changed it this way because it seems to be a Fortran example, and for a Fortran programmer the first direction is the column direction. I really hope that I'm right with this explanation. Can anybody serve as an arbiter? At this point I could be convinced of anything. - Rolf >From owner-mpi-core@CS.UTK.EDU Wed Apr 13 09:11 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA05351; Wed, 13 Apr 94 09:11:05 -0700 Return-Path: Received: from CS.UTK.EDU by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0pr7Wp-000Ks3C; Wed, 13 Apr 94 09:11 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id MAA20105; Wed, 13 Apr 1994 12:07:40 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 13 Apr 1994 12:07:39 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id MAA20098; Wed, 13 Apr 1994 12:07:37 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA22079; Wed, 13 Apr 94 12:07:36 EDT Date: Wed, 13 Apr 94 12:07:36 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404131607.AA22079@super> Received: by b125.super.org (4.1/SMI-4.1) id AA07646; Wed, 13 Apr 94 12:07:33 EDT To: mpi-core@CS.UTK.EDU In-Reply-To: <9404131539.AA16962@f1neuman.gmd.de> (Rolf.Hempel@gmd.de) Subject: Re: Steve's corrections Status: RO As for 178:12 - I still think decreasing is ok. As a matter of fact, monotonically decreasing might be better. I believe this means that the next value is less than or equal to the previous value. This is what you get back. Like I said, you guys are hung up on this! I think I now agree with Rolf about the example 6.4 (184:37). By shifting on coord #1 you shift along the column. OK. Thanks for checking these for me Rolf. Steve >From lederman@super.org Thu Apr 14 07:11 PDT 1994 Received: from iliamna.cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA11236; Thu, 14 Apr 94 07:11:21 -0700 Return-Path: Received: from super.super.org by iliamna.cse.ogi.edu with smtp (Smail3.1.28.1 #1) id m0prS8V-0002F2C; Thu, 14 Apr 94 07:11 PDT Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA09627; Thu, 14 Apr 94 10:11:17 EDT Date: Thu, 14 Apr 94 10:11:17 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404141411.AA09627@super.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA00316; Thu, 14 Apr 94 10:11:17 EDT To: otto@iliamna.cse.ogi.edu Subject: context comments Status: RO Steve, Given what I have to get done in the next two days, it seems unlikely I will be able to carefully look over the whole context chapter (or much). It has been taking me about 1 hour per 10 pages. I could get to it this weekend, but the deadline was Friday, April 15. Is that really the deadline? If so, I won't bother reading it this weekend. I'll see tomorrow what we're going to do...I'd like to end for now...BUT, your efforts wouldn't be wasted since we would like that chapter proofed for the ARM. On another note. I have sent in quite a few comments. If many are correct (I hope I wasn't that asleep that they are all wrong), it seems that a lot of changes need to be made to the document. Will everyone (or at least me and/or other commentors) get another look to make sure all the changes do what our comments intended. Sometimes e-mail doesn't get your point across. I'm trying to give you feedback as to what I've done; you can ftp over the doc from my place too. --Steve Otto Finally, you can be happy that I have not found a x-ref mistake and all the refs and Biblio seem perfect. The editor must have done a great job! Of course. ;^) Steve >From owner-mpi-core@CS.UTK.EDU Thu Apr 14 07:11 PDT 1994 Received: from cse.ogi.edu by merckx.cse.ogi.edu with SMTP (1.37.109.4/16.2) id AA11240; Thu, 14 Apr 94 07:11:35 -0700 Return-Path: Received: from CS.UTK.EDU by church.cse.ogi.edu with smtp (Smail3.1.28.1 #5) id m0prS8b-000KrpC; Thu, 14 Apr 94 07:11 PDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA09019; Thu, 14 Apr 1994 10:04:30 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 14 Apr 1994 10:04:28 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA08992; Thu, 14 Apr 1994 10:04:22 -0400 Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA09540; Thu, 14 Apr 94 10:04:16 EDT Date: Thu, 14 Apr 94 10:04:16 EDT >From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404141404.AA09540@super.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA00313; Thu, 14 Apr 94 10:04:15 EDT To: mpi-core@CS.UTK.EDU Subject: comments #4 - collective comm Status: RO Good morning, afternoon or evening! Here are my comments on Chap. 4, collective communications. Sorry for any mistakes below, I have been staring at the document too long. Steve ---------------------------------------------------------------------- 93:43 - says that MPI_BCAST broadcast a message, "itself included". This sounds a little strange since there is only one INOUT buffer. On the root the answer is already "there" at the start. Should we remove the "itself included"? I think we should leave it...I guess I like it because it sounds consistent with the other collective functions, where it DOES matter. 106:16 - MPI_Scatterv(sendbuf, displs, scounts, ...) -> MPI_Scatterv(sendbuf, scounts, displs, ...) yes. done. 108:44 - MPI_GATHERV(..., displs, recvcounts, ...) -> MPI_GATHERV(..., recvcounts, displs, ...) done 110:22 - says "specifying the maximum number of elements that can be received from each processor". 110:46 says the sendcount[j], sendtype must be equal to recvcount[i], recvtype and this implies that the amount of data send must equal the amount of data received. I think this implies that recvcounts is the amount of data received. If true, I would reword it to be "specifying the number of elements that will be received at each processor". the word maximum was historical and was wrong. Thanks! done 111:4 - MPI_Send(sendbuf + displs[i], ...) -> MPI_Send(sendbuf + displs[i] * extent(sendtype), ...)? Same comment on: 111:7 for MPI_RECV and extent(recvtype) done 113:23 - Was MPI_LONG_DOUBLE deliberately left out from the allowed floating point types? I don't think so... 113:35 - The MPI_B* ops can be applied to C integer but not Fortran integer. I was wondering why we had this asymmetry. Some Fortran (Cray for example) allows bit ops on integers. I suggest we add Fortran integers since it is not clear what is the "right" datatype for doing bit ops. Yes, I don't see why we should rule out B* ops for Fortran INTEGERs I will add it. 113:41 - My Fortran is rusty, but should: SUBROUTINE PAR_BLAS1(m, a(m), b(m), c, comm) -> SUBROUTINE PAR_BLAS1(m, a, b, c, comm)? This also applies to: 114:13 for PAR_BLAS2 I don't do Fortran...but I *think* your right. I will change. 114:8 - First, can a Fortran subroutine return a value? Also, I didn't think the value returned in Fortran was done via the return statement but by setting the name of the function to the value returned. This also applies to: 114:32 Not sure 114:13 - I am confused by what this routine is doing. One problem I see is that sum(j) is for j=1,n but the declaration of sum is sum(m). I assume vector * array = vector is being computed. However, I cannot figure out how sum, a, b are distributed so that the MPI_REDUCE works. It seems as if each node must have a full length of a, c and sum so that the local value of m is the full length of the vectors. Otherwise the REDUCE would add different indices of the vector. Is this the right distribution? I really got myself confused on this one. Even with this, I'm not sure it does the right thing. I think it's screwed up... 114:36 - MPI_MINLOC -> MPI_MAXLOC done 115:38 - none of the types from MINLOC and MAXLOC are given in Appendix A. they are now 115:38 - How did we decide what pairs to include/exclude. Fortran does not have COMPLEX and C does not have LONG_DOUBLE, SHORT and UNSIGNED. We probably voted on this and I forgot. No, Marc and I decided...this is new! We picked the obviously needed ones. MAX by itself doesn't even allow COMPLEX. 116:22 - I think a[30] -> ain[30]? Same comment applies to 117:3 aww come on -- they're comments! Yes, you are right. done 117:16 - MPI_DOUBLE -> MPI_2DOUBLE_PRECISION done 117:16 - add error return as last argument to MPI_REDUCE done 117:20-24 - The DO loop is missing an end. I'm also not sure if a DO inside an IF is a single statement. I would change to: IF (myrank .EQ. root) then ! .... DO I=1,30 ..... END DO ENDIF done 117:44 - if (in.value < val[i]) { ->if (in.value > val[i]) { i.e., < -> > since looking for min. done 118:48,119:3 - Is MPI_Uop and UOP supposed to be function? Also, my C isn't deep enough to understand what MPI_Uop on 118:32 is for. The typedef is a void function but the input for MPI_Op_create is MPI_Uop. I just want a C hacker to make sure this was correct. It's a typedef --- showing the prototype for the user function, ie, what the function should look like I think its ok. 122:33 - I have the same problems here that I had with Ex. 4.16 p. 114. yep. 123:29 - INTEGER RECVCOUNTS, .... -> INTEGER RECVCOUNTS(*), .... yep. 126:2 - This has the vector dependancy I talked about in note #2 78:18. I think it needs to be changed to: for (i=1; i >= 0; i--) .... done. From owner-mpi-core@CS.UTK.EDU Mon Apr 18 09:05:43 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id JAA05119; Mon, 18 Apr 1994 09:05:42 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA04703; Mon, 18 Apr 1994 09:05:12 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 18 Apr 1994 09:05:10 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA04662; Mon, 18 Apr 1994 09:04:59 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA24714 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Mon, 18 Apr 1994 14:03:53 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA01194; Mon, 18 Apr 1994 14:01:33 +0000 Date: Mon, 18 Apr 1994 14:01:33 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9404181301.AA01194@tycho.co.uk> To: SNIR@watson.ibm.com Cc: mpi-core@CS.UTK.EDU, cc:otto@cse.ogi.edu In-Reply-To: <199404150243.WAA15669@CS.UTK.EDU> (snir@watson.ibm.com) Subject: Re: MPI errata > 190, 193 > > I think we are getting too sophisticated with the LEN argument. We have a > MPI_MAX_ERROR_STRING constant, and an MPI_MAX_PROCESSOR_NAME constant. It > should be the programmer responsibility to provide a string/CHARACTER argument > that has at least that length (including, in C, the terminating null). > I suggest the following text: > "The routine returns the name of the processor ... In C this string is null > terminated. In Fortran, the name is blank padded, if shorter than the > character argument. The call is erroneous if the input string has length less > than MPI_MAX_PROCESSOR_NAME." > And similarly, for error strings. > > Of course, a good quality Fortran implementation will just truncate the name if > input string is too short. > > My rationale is that: > 1. It is not too painful to provide oversized strings. > 2. It is more error-prone to make sure that the input string has length >= LEN > as to make sure it has length >= MPI_MAX_PROCESSOR_NAME (constant). > 3. If somebody wants to do pretty printing in Fortran, let him use a function > that deletes trailing blanks. > > Good quality Fortran implementation will truncate. I am happy not to have the result length. I would prefer to REQUIRE that the result string always contains something printable after a call to these routines in either C or Fortran. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@CS.UTK.EDU Mon Apr 18 12:16:34 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id MAA07801; Mon, 18 Apr 1994 12:16:34 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id MAA20222; Mon, 18 Apr 1994 12:15:58 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 18 Apr 1994 12:15:56 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id MAA20211; Mon, 18 Apr 1994 12:15:55 -0400 Received: by rios2.EPM.ORNL.GOV (AIX 3.2/UCB 5.64/4.03) id AA16190; Mon, 18 Apr 1994 12:15:52 -0400 Message-Id: <9404181615.AA16190@rios2.EPM.ORNL.GOV> To: mpi-core@CS.UTK.EDU Subject: MPI talk Date: Mon, 18 Apr 94 12:15:52 -0500 From: "David W. Walker" If you've seen the slides of my talk on MPI in the proceedings of the 1993 Intel SUG meeting, you may be a little puzzled. This is because (for reasons unbeknownst to me) only the last 8 of my slides appear, and these are in reverse order! Regards, David From owner-mpi-core@CS.UTK.EDU Mon Apr 18 16:23:22 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA11089; Mon, 18 Apr 1994 16:23:22 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA11063; Mon, 18 Apr 1994 16:22:52 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 18 Apr 1994 16:22:51 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.8s-UTK) id QAA11056; Mon, 18 Apr 1994 16:22:49 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id PAA11436; Mon, 18 Apr 1994 15:22:46 -0500 Message-Id: <199404182022.PAA11436@antares.mcs.anl.gov> To: otto@cse.ogi.edu Subject: Latest MPI Draft on Web Cc: mpi-core@CS.UTK.EDU Date: Mon, 18 Apr 1994 15:22:45 -0500 From: Rusty Lusk The April 15 version of the draft recently announced by Steve is now up on our Web site. The URL is http://www.mcs.anl.gov/mpi/mpi-report/mpi-report.html It is an imperfectly converted (we are still working on it) but usable version of the draft. The parent page ...mpi/index.html has a pointer to the postscript as well. A new release of the implementation is coming soon (maybe within hours). Let me know if you want to be on the implementation-announcement mailing list. Bill & Rusty From owner-mpi-core@CS.UTK.EDU Tue Apr 19 10:18:15 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA18733; Tue, 19 Apr 1994 10:18:15 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA04020; Tue, 19 Apr 1994 10:16:52 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 19 Apr 1994 10:16:48 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with ESMTP (cf v2.8s-UTK) id KAA04013; Tue, 19 Apr 1994 10:16:47 -0400 From: Jack Dongarra Received: by dasher.cs.utk.edu (cf v2.9c-UTK) id KAA17298; Tue, 19 Apr 1994 10:16:18 -0400 Date: Tue, 19 Apr 1994 10:16:18 -0400 Message-Id: <199404191416.KAA17298@dasher.cs.utk.edu> To: lusk@mcs.anl.gov, otto@cse.ogi.edu Subject: Re: Latest MPI Draft on Web Cc: mpi-core@CS.UTK.EDU The latest draft is also available on netlib (ftp www.netlib.org, cd mpi). Look for mpi-report.ps or tarfile.Z. Jack From owner-mpi-core@CS.UTK.EDU Tue Apr 19 17:59:48 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id RAA21136; Tue, 19 Apr 1994 17:59:48 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id RAA13010; Tue, 19 Apr 1994 17:59:42 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 19 Apr 1994 17:59:41 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id RAA13001; Tue, 19 Apr 1994 17:59:37 -0400 Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA04988; Tue, 19 Apr 94 17:14:42 EDT Date: Tue, 19 Apr 94 17:14:42 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404192114.AA04988@super.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA01171; Tue, 19 Apr 94 17:13:19 EDT To: mpi-core@CS.UTK.EDU Subject: last comments? Hello! I finally got a copy of the 4/15 draft. First, I take my hat off the Steve for his great editing. I know how must time and effort he has put into this. Here are my comments. I know some of my previous ones were out to lunch, so I hope these are correct. We need to finish up these points (pulling them out to encourage a response): ---------------------------------------------------------------------- 10: I recall discussing if people could use the MPI_ for their own functions. I thought we said no. Does anyone recall????? ---------------------------------------------------------------------- 50: Labels in Fortran examples are not sufficiently outdented (at least if this is fixed format...) yeah. haven't done anything here yet. ---------------------------------------------------------------------- 165 We should say which MPI calls are valid in copy and delete functions, and whether they can refer to the communicator on which they are being applied. (I believe that there need be no restrictions, apart from deleting the communicator, and the communicator should still be valid to use. I think this is OK). Tony, do you want to suggest a sentence or two? ---------------------------------------------------------------------- 193:32 - Bill: could you describe what MPI_ERR_CODE is for? I think it is confusing since it is different from the other error codes. ---------------------------------------------------------------------- I have a couple of comments (all page:line apply to the 4/15 draft): 86:14 - I agree with Steve that I confused C and Fortran. However, the calls are still missing arguments: MPI_PACK_SIZE(incount, datatype, size) -> MPI_PACK_SIZE(incount, datatype, comm, size, ierror) 86:16 - The same applies to: MPI_PACK(...,position) -> MPI_PACK(...,position, comm, ierror) 87:25 - Says got edited but I think it slipped through: MPI_PACK(MPI_BOTTOM, 1, newtype, buff, position, MPI_COMM_WORLD) -> MPI_PACK(MPI_BOTTOM, 1, newtype, buff, 1000, &position, MPI_COMM_WORLD) 113:26 - MPI_LONG_DOUBLE is still missing from the allowed floating point types. 113:36 - Steve's response to a previous comment raised a point. I am pretty sure we agreed to allow min, max for complex. There is a clear mathematical definition involving the absolute value. This is something I think would be useful in physics, chemistry, etc. Could we please add this. 114:12 - I double checked on this. you do not say return(c) in Fortran. It is just return. the array c was passed in the argument list and since Fortran is by reference, it is changed for the caller too. How quickly I forget..... Also applies to: 114:34 123:6 183:39 - I agree with Steve's comment about column shift being confusing to some. How about: by shifting column i -> by shifting column i (vertically, i.e., along column) 194:34 - int double MPI_Wtick -> double MPI_Wtick 223:26-30 - I think COPY_FN and DELETE_FN are left from before. I think the new ones are on 210:2-10. I think this should be deleted. Steve From owner-mpi-core@CS.UTK.EDU Tue Apr 19 18:16:15 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id SAA21183; Tue, 19 Apr 1994 18:16:15 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id SAA14314; Tue, 19 Apr 1994 18:15:38 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 19 Apr 1994 18:15:37 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id SAA14297; Tue, 19 Apr 1994 18:15:34 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA03339; Tue, 19 Apr 94 15:15:26 -0700 Date: Tue, 19 Apr 94 15:15:26 -0700 From: Steve Otto Message-Id: <9404192215.AA03339@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Steve H-L's Comments Thank you, Steven, for your comments. I admire your patience and doggedness in keeping after this! I will attempt to do your work justice by processing them tonight. If anyone has significant thoughts on the points that Steven has raised, please let me know soon -- like tonight (Tuesday) or tomorrow (Wednesday). I would like to wrap this up soon as we can't let it drag on any longer. (remember there's always the ARM) As always, I will broadcast out my actions that I have taken on Steven H-L's points. --Steve Otto From owner-mpi-core@CS.UTK.EDU Wed Apr 20 02:21:25 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id CAA22470; Wed, 20 Apr 1994 02:21:25 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id CAA16885; Wed, 20 Apr 1994 02:21:09 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 20 Apr 1994 02:21:08 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id CAA16874; Wed, 20 Apr 1994 02:21:06 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA12909; Tue, 19 Apr 94 23:20:49 -0700 Date: Tue, 19 Apr 94 23:20:49 -0700 From: Steve Otto Message-Id: <9404200620.AA12909@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Actions taken; FINAL CALL! From owner-mpi-core@CS.UTK.EDU Wed Apr 20 02:21:31 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id CAA22475; Wed, 20 Apr 1994 02:21:31 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id CAA16910; Wed, 20 Apr 1994 02:21:34 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 20 Apr 1994 02:21:33 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id CAA16901; Wed, 20 Apr 1994 02:21:30 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA12995; Tue, 19 Apr 94 23:21:24 -0700 Date: Tue, 19 Apr 94 23:21:24 -0700 From: Steve Otto Message-Id: <9404200621.AA12995@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Actions taken; FINAL CALL! Hello! I finally got a copy of the 4/15 draft. First, I take my hat off the Steve for his great editing. I know how must time and effort he has put into this. Here are my comments. I know some of my previous ones were out to lunch, so I hope these are correct. We need to finish up these points (pulling them out to encourage a response): ---------------------------------------------------------------------- 10: I recall discussing if people could use the MPI_ for their own functions. I thought we said no. Does anyone recall????? Here is the statement I've added in terms. If someone objects, tell me. Programs must not declare variables or functions with names beginning with the prefix, {\tt MPI\_}. This is mandated to avoid possible name collisions. ---------------------------------------------------------------------- 50: Labels in Fortran examples are not sufficiently outdented (at least if this is fixed format...) yeah. haven't done anything here yet. done ---------------------------------------------------------------------- 165 We should say which MPI calls are valid in copy and delete functions, and whether they can refer to the communicator on which they are being applied. (I believe that there need be no restrictions, apart from deleting the communicator, and the communicator should still be valid to use. I think this is OK). Tony, do you want to suggest a sentence or two? Tony?? ---------------------------------------------------------------------- 193:32 - Bill: could you describe what MPI_ERR_CODE is for? I think it is confusing since it is different from the other error codes. Do you mean LASTCODE? Bill?? ---------------------------------------------------------------------- I have a couple of comments (all page:line apply to the 4/15 draft): 86:14 - I agree with Steve that I confused C and Fortran. However, the calls are still missing arguments: MPI_PACK_SIZE(incount, datatype, size) -> MPI_PACK_SIZE(incount, datatype, comm, size, ierror) done. I left off ierror since this is a language-independent statement. 86:16 - The same applies to: MPI_PACK(...,position) -> MPI_PACK(...,position, comm, ierror) done. ditto. 87:25 - Says got edited but I think it slipped through: MPI_PACK(MPI_BOTTOM, 1, newtype, buff, position, MPI_COMM_WORLD) -> MPI_PACK(MPI_BOTTOM, 1, newtype, buff, 1000, &position, MPI_COMM_WORLD) done. 113:26 - MPI_LONG_DOUBLE is still missing from the allowed floating point types. done. 113:36 - Steve's response to a previous comment raised a point. I am pretty sure we agreed to allow min, max for complex. There is a clear mathematical definition involving the absolute value. This is you mean the modulus... something I think would be useful in physics, chemistry, etc. Could we please add this. ok...any objections, anyone? done. So: MAX and MIN work for COMPLEXes. On a related point, I added more types for MAXLOC, MINLOC: Fortran: \constitem{\const{MPI\_2COMPLEX}}{pair of \ftype{COMPLEX}es} C: \constitem{\const{MPI\_SHORT\_INT}}{\type{short} and \type{int} } \constitem{\const{MPI\_LONG\_DOUBLE\_INT}}{\type{long double} and \type{int} } I left out the C unsigned types, however, as it's just getting ridiculous. 114:12 - I double checked on this. you do not say return(c) in Fortran. It is just return. the array c was passed in the argument list and since Fortran is by reference, it is changed for the caller too. How quickly I forget..... Also applies to: 114:34 123:6 done 183:39 - I agree with Steve's comment about column shift being confusing to some. How about: by shifting column i -> by shifting column i (vertically, i.e., along column) done 194:34 - int double MPI_Wtick -> double MPI_Wtick done. Wtime also had this problem. "int" was embedded in macro... 223:26-30 - I think COPY_FN and DELETE_FN are left from before. I think the new ones are on 210:2-10. I think this should be deleted. done Steve From owner-mpi-core@CS.UTK.EDU Wed Apr 20 07:54:51 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id HAA24197; Wed, 20 Apr 1994 07:54:50 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id HAA20904; Wed, 20 Apr 1994 07:54:36 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 20 Apr 1994 07:54:33 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.8s-UTK) id HAA20893; Wed, 20 Apr 1994 07:54:30 -0400 Received: from godzilla (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id GAA01287; Wed, 20 Apr 1994 06:54:16 -0500 From: William Gropp Date: Wed, 20 Apr 1994 06:54:15 -0500 Message-Id: <199404201154.GAA23050@godzilla> To: otto@cse.ogi.edu CC: mpi-core@CS.UTK.EDU In-reply-to: <9404200621.AA12995@merckx.cse.ogi.edu> (message from Steve Otto on Tue, 19 Apr 94 23:21:24 -0700) Subject: Re: Actions taken; FINAL CALL! 193:32 - Bill: could you describe what MPI_ERR_CODE is for? I think it is confusing since it is different from the other error codes. Do you mean LASTCODE? Bill?? The intent was that the error classes satisfy 0 == MPI_SUCCESS < MPI_ERR_... <= MPI_ERR_LASTCODE Perhaps we should say this. Note that MPI_SUCCESS == 0 is necessary to be consistent with C practice; the separation of error classes and error codes allows us to define the error CLASSES this way. Having a known LASTCODE is often a nice sanity check as well. Bill From owner-mpi-core@CS.UTK.EDU Wed Apr 20 16:56:21 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA27590; Wed, 20 Apr 1994 16:56:21 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA12311; Wed, 20 Apr 1994 16:55:15 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 20 Apr 1994 16:55:12 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA12267; Wed, 20 Apr 1994 16:55:07 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA23002; Wed, 20 Apr 94 13:54:57 -0700 Date: Wed, 20 Apr 94 13:54:57 -0700 From: Steve Otto Message-Id: <9404202054.AA23002@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Action taken 193:32 - Bill: could you describe what MPI_ERR_CODE is for? I think it is confusing since it is different from the other error codes. Do you mean LASTCODE? Bill?? The intent was that the error classes satisfy 0 == MPI_SUCCESS < MPI_ERR_... <= MPI_ERR_LASTCODE Perhaps we should say this. Note that MPI_SUCCESS == 0 is necessary to be consistent with C practice; the separation of error classes and error codes allows us to define the error CLASSES this way. Having a known LASTCODE is often a nice sanity check as well. Bill I've added some of Bill's text. Here is what it now reads (after the table of error classes) (excuse the latex): An implementation is free to define more error classes; however, these must be used where appropriate. The error classes satisfy $0 = $ \const{MPI\_SUCCESS} $<$ \const{MPI\_ERR\_...} $\leq$ \const{MPI\_ERR\_LASTCODE}. \begin{rationale} The difference between \const{MPI\_ERR\_UNKNOWN} and \const{MPI\_ERR\_OTHER} is that \func{MPI\_ERROR-} \func{\_STRING} %mansplit can return useful information about \const{MPI\_ERR\_OTHER}. Note that \const{MPI\_SUCCESS} $= 0$ is necessary to be consistent with C practice; the separation of error classes and error codes allows us to define the error classes this way. Having a known \const{LASTCODE} is often a nice sanity check as well. \end{rationale} --Steve From owner-mpi-core@CS.UTK.EDU Wed Apr 20 17:00:06 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id RAA27620; Wed, 20 Apr 1994 17:00:06 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id RAA12791; Wed, 20 Apr 1994 17:00:02 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 20 Apr 1994 17:00:00 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA12777; Wed, 20 Apr 1994 16:59:57 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA23618; Wed, 20 Apr 94 13:59:45 -0700 Date: Wed, 20 Apr 94 13:59:45 -0700 From: Steve Otto Message-Id: <9404202059.AA23618@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: sentence from Tony... Steve H-L's query: >---------------------------------------------------------------------- >165 We should say which MPI calls are valid in copy and > delete functions, and whether they can refer to the > communicator on which they are being applied. > (I believe that there need be no restrictions, apart > from deleting the communicator, and the communicator > should still be valid to use. I think this is OK). > > Tony, do you want to suggest a sentence or two? > > Tony?? > Tony says this: Here the potential problem is MPI re-calling a user program. In general, a windy path of calls between user and MPI can result from this if a copy_fn or delete_fn is permitted to call any MPI calls. However, since MPI is required to be thread-safe, this seems to be allowable (ie, copy- and delete-callbacks are allowed to call anything). Furthermore, it should be legal to use the current communicator being copied or deleted as part of the function, as noted above. This doesn't seem like a definitive statement that I can add to the document, so I haven't done anything yet. --Steve Otto From owner-mpi-core@CS.UTK.EDU Thu Apr 21 04:48:07 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id EAA29892; Thu, 21 Apr 1994 04:48:07 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id EAA08911; Thu, 21 Apr 1994 04:47:44 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 21 Apr 1994 04:47:43 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id EAA08857; Thu, 21 Apr 1994 04:47:28 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA13840 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Thu, 21 Apr 1994 09:46:22 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA02935; Thu, 21 Apr 1994 09:43:58 +0000 Date: Thu, 21 Apr 1994 09:43:58 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9404210843.AA02935@tycho.co.uk> To: otto@cse.ogi.edu Cc: mpi-core@CS.UTK.EDU In-Reply-To: <9404202054.AA23002@merckx.cse.ogi.edu> (message from Steve Otto on Wed, 20 Apr 94 13:54:57 -0700) Subject: Re: Action taken > An implementation is free to define more error classes; however, these must be > used where appropriate. Shouldn't this read... An implementation is free to define more error classes; however, the standard error classes must be used where appropriate. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com From owner-mpi-core@cs.utk.edu Thu Apr 21 15:08:01 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id PAA09227; Thu, 21 Apr 1994 15:08:01 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id NAA23240; Thu, 21 Apr 1994 13:10:37 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 21 Apr 1994 13:10:35 EDT Errors-to: owner-mpi-core@cs.utk.edu Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id NAA23216; Thu, 21 Apr 1994 13:10:28 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA20343; Thu, 21 Apr 94 10:10:03 -0700 Date: Thu, 21 Apr 94 10:10:03 -0700 From: Steve Otto Message-Id: <9404211710.AA20343@merckx.cse.ogi.edu> To: mpi-core@cs.utk.edu Subject: Action >From: jim@meiko.co.uk (James Cownie) Message-Id: <9404210843.AA02935@tycho.co.uk> To: otto@cse.ogi.edu Cc: mpi-core@CS.UTK.EDU In-Reply-To: <9404202054.AA23002@merckx.cse.ogi.edu> (message from Steve Otto on Wed, 20 Apr 94 13:54:57 -0700) Subject: Re: Action taken Status: R > An implementation is free to define more error classes; however, these must be > used where appropriate. Shouldn't this read... An implementation is free to define more error classes; however, the standard error classes must be used where appropriate. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West Reservoir Place Bristol BS12 4SD 1601 Trapelo Road England Waltham MA 02154 Phone : +44 454 616171 +1 617 890 7676 FAX : +44 454 618188 +1 617 890 5042 E-Mail: jim@meiko.co.uk or jim@meiko.com OK. --Steve From owner-mpi-core@CS.UTK.EDU Fri Apr 22 09:00:46 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id JAA17386; Fri, 22 Apr 1994 09:00:45 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id IAA02101; Fri, 22 Apr 1994 08:59:36 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 22 Apr 1994 08:59:35 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with ESMTP (cf v2.8s-UTK) id IAA02095; Fri, 22 Apr 1994 08:59:34 -0400 From: Jack Dongarra Received: by dasher.cs.utk.edu (cf v2.9c-UTK) id IAA01722; Fri, 22 Apr 1994 08:59:02 -0400 Date: Fri, 22 Apr 1994 08:59:02 -0400 Message-Id: <199404221259.IAA01722@dasher.cs.utk.edu> To: mpi-core@CS.UTK.EDU Subject: mpi reference Here is the reference for MPI. The tech report number for the final version is different than the draft. Jack @TECHREPORT{mpi, AUTHOR = { Message Passing Interface Forum }, TITLE = { MPI: A Message-Passing Interface Standard }, INSTITUTION = {University of Tennessee}, YEAR = {1994}, TYPE = {Computer {S}cience {D}ept. {T}echnical {R}eport {CS}-94-230}, MONTH = {April}, NOTE = {(To appear in the International Journal of Supercomputer Applications, Volume 8, Number 3/4, 1994)}, ADDRESS = {Knoxville, TN}} From owner-mpi-core@cs.utk.edu Wed Apr 27 19:17:16 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id TAA23806; Wed, 27 Apr 1994 19:17:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id TAA22084; Wed, 27 Apr 1994 19:05:57 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 27 Apr 1994 19:05:52 EDT Errors-to: owner-mpi-core@cs.utk.edu Received: from sun2.nsfnet-relay.ac.uk by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id TAA22063; Wed, 27 Apr 1994 19:05:46 -0400 Via: uk.ac.southampton.parallel-computing-support; Thu, 28 Apr 1994 00:05:40 +0100 From: Ian Glendinning Date: Wed, 27 Apr 94 16:58:43 BST Message-Id: <29428.9404271558@par.soton.ac.uk> Via: aloisius.sucs.soton.ac.uk; Wed, 27 Apr 94 16:58:43 BST To: mpi-core@cs.utk.edu Subject: Communicator argument to MPI_PACK/MPI_UNPACK Hi, while preparing a talk on MPI, I decided to look at the section in the final report on the recently added functions MPI_PACK and MPI_UNPACK, and discovered that I had no idea why they have a communicator argument. Unfortunately I haven't been able to devote much (any) time to MPI lately, due to a change in employment, and I wasn't at the final meeting in Knoxville to hear the discussion about them. However, recourse to my archive of mpi-comm e-mail messages, most of which I haven't had time to read since Christmas, revealed that representation conversion was the motivation for their presence. I realise that it's a little late now, and I wish I'd been able to contribute to the proof reading of the final draft, but might I suggest adding some text to section 3.13, to explain the reason for the presence of these arguments? I'm sure I won't be the only one to be confused by this! Ian -- igl@par.soton.ac.uk Ian Glendinning Tel: +44 703 592594 Southampton HPC Centre, Computing Services Fax: +44 703 593939 University of Southampton, SO17 1BJ, UK From owner-mpi-core@CS.UTK.EDU Thu Apr 28 00:55:27 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id AAA25016; Thu, 28 Apr 1994 00:55:26 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id AAA15429; Thu, 28 Apr 1994 00:54:14 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 28 Apr 1994 00:54:10 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id AAA15420; Thu, 28 Apr 1994 00:54:08 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA02689; Wed, 27 Apr 94 23:51:43 CDT Date: Wed, 27 Apr 94 23:51:43 CDT From: Tony Skjellum Message-Id: <9404280451.AA02689@Aurora.CS.MsState.Edu> To: igl@par.soton.ac.uk, mpi-core@CS.UTK.EDU Subject: Re: Communicator argument to MPI_PACK/MPI_UNPACK Because communicators are used to cache information about conversion in a heterogeneous environment, or for protocol information in a hierarchical memory environment, pack information is useful and we decided to be "symmetric". For the average user, as with other MPI calls, when this is not being exploited, use MPI_COMM_WORLD as the argument! -Tony From owner-mpi-core@CS.UTK.EDU Thu Apr 28 10:22:45 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA02824; Thu, 28 Apr 1994 10:22:44 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA29110; Thu, 28 Apr 1994 10:22:39 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 28 Apr 1994 10:22:38 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA29102; Thu, 28 Apr 1994 10:22:37 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA02924; Thu, 28 Apr 94 09:22:05 CDT Date: Thu, 28 Apr 94 09:22:05 CDT From: Tony Skjellum Message-Id: <9404281422.AA02924@Aurora.CS.MsState.Edu> To: igl@par.soton.ac.uk, mpi-core@CS.UTK.EDU Subject: Re: Communicator argument to MPI_PACK/MPI_UNPACK Ian, Let me clarify previous comments by replacing "pack information is useful and we decided to be symmetric..." by The information cached in a communicator determines the behavior of pack, and allows a symmetric design in that there is a reasonable equivalence to the use of pack's followed by sends (resp, receives followed by unpacks) as their is to the combined call, except for the tacit buffering implied by the use of pack/unpack. The same functionality, namely, support for multiple protocol message passing, hiding of heterogeneous conversion, etc, are still available to the advanced user and to the advanced MPI implementation. The default communicator, MPI_COMM_WORLD, serves as in other instances to provide the most conservative of packing conversions (equivalently, a dup of this comunicator). If these calls (as we concluded) are merely to help port PVM codes to MPI, then using MPI_COMM_WORLD is a reasonable choice. IF they are also useful to help simplify other programming in MPI, then retaining the ability to use any MPI_Comm is valuable. From owner-mpi-core@CS.UTK.EDU Fri Apr 29 07:26:42 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id HAA12141; Fri, 29 Apr 1994 07:26:42 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id HAA09246; Fri, 29 Apr 1994 07:26:23 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 29 Apr 1994 07:26:22 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from sun2.nsfnet-relay.ac.uk by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id HAA09239; Fri, 29 Apr 1994 07:26:18 -0400 Via: uk.ac.southampton.parallel-computing-support; Fri, 29 Apr 1994 11:45:42 +0100 From: Ian Glendinning Date: Fri, 29 Apr 94 11:39:49 BST Message-Id: <3957.9404291039@par.soton.ac.uk> Via: aloisius.sucs.soton.ac.uk; Fri, 29 Apr 94 11:39:49 BST To: mpi-core@CS.UTK.EDU, tony@Aurora.CS.MsState.Edu Subject: Re: Communicator argument to MPI_PACK/MPI_UNPACK > From tony@Edu.MsState.CS.Aurora Thu Apr 28 15:47:14 1994 > Let me clarify previous comments by replacing "pack information is > useful and we decided to be symmetric..." by Tony, Thanks for your explanation of the rationale behind the communicator argument to pack and unpack. It's a pity there aren't any such words in the MPI document. Is the document now frozen? What happens if (when) mistakes are found? Perhaps we could have an accompanying errata/set of clarifying notes, which could be merged into a revision of the document somewhere down the road? Ian -- igl@par.soton.ac.uk Ian Glendinning Tel: +44 703 592594 Southampton HPC Centre, Computing Services Fax: +44 703 593939 University of Southampton, SO17 1BJ, UK From owner-mpi-core@CS.UTK.EDU Fri Apr 29 09:26:56 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id JAA13843; Fri, 29 Apr 1994 09:26:56 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA17698; Fri, 29 Apr 1994 09:26:38 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 29 Apr 1994 09:26:36 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA17691; Fri, 29 Apr 1994 09:26:34 -0400 Received: from b125.super.org by super.super.org (4.1/SMI-4.1) id AA28533; Fri, 29 Apr 94 09:26:26 EDT Date: Fri, 29 Apr 94 09:26:26 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9404291326.AA28533@super.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA02894; Fri, 29 Apr 94 09:26:24 EDT To: mpi-core@CS.UTK.EDU Subject: mpi corrections I have already wondered what Ian asked. What about small corrections? I send Steve Otto mail but he is away in Cancun until next week. Here, for example, is a small correction I found: 17:27 - signed short -> signed short int These never seem to end. Our only hope is that nobody ever reads the MPI document :-) One another note: I am giving a internal seminar about MPI here at SRC. I have a few set of slides from previous talks given by David. If anyone else has slides from a general overview talk that they would be willing to share with me (or others) I would appreciate receiving them. Thanks, Steve From owner-mpi-core@CS.UTK.EDU Sat Apr 30 16:53:43 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA24775; Sat, 30 Apr 1994 16:53:43 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA29474; Sat, 30 Apr 1994 16:53:37 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 30 Apr 1994 16:53:35 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from thialfi.cs.cornell.edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id QAA29467; Sat, 30 Apr 1994 16:53:34 -0400 Received: from CLOYD.CS.CORNELL.EDU by thialfi.cs.cornell.edu (5.67/I-1.99E) id AA16218; Sat, 30 Apr 94 16:53:29 -0400 Received: from OTTAR.CS.CORNELL.EDU by cloyd.cs.cornell.edu (5.67/I-1.99D) id AA18614; Sat, 30 Apr 94 16:53:34 -0400 From: elster@cs.cornell.edu (Anne C. Elster) Date: Sat, 30 Apr 94 16:53:26 -0400 Message-Id: <9404302053.AA23376@ottar.cs.cornell.edu> Received: by ottar.cs.cornell.edu (5.67/N-0.13) id AA23376; Sat, 30 Apr 94 16:53:26 -0400 To: lederman@super.org, mpi-core@CS.UTK.EDU Subject: Re: MPI slides (was: mpi corrections) Cc: elster@cs.cornell.edu >One another note: I am giving a internal seminar about MPI here at >SRC. I have a few set of slides from previous talks given by David. >If anyone else has slides from a general overview talk that they >would be willing to share with me (or others) I would appreciate >receiving them. I would be interested in sharing some overview slides, too. I have a few (color PS) from a talk I gave last spring here at Cornell, but they are getting old... Other MPI members may also be interested? Maybe we could set up a ftp directory for slides (Ps files or the like) that MPI members could access when we need to give overview presentations? Could save us from all doing the same work over-and-over ... What do you think? Anne ------------------------------------------------------------------------------- Anne C. Elster | (607) 254-8653 [office/ answ.mach] Xerox Design Research Institute | 254-4742 [fax] Cornell University, 502 Theory Center | Ithaca, NY 14853 | e-mail: elster@cs.cornell.edu USA | na.elster@na-net.ornl.gov From owner-mpi-core@CS.UTK.EDU Tue May 10 09:49:31 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id JAA10981; Tue, 10 May 1994 09:49:31 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id JAA15474; Tue, 10 May 1994 09:49:45 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 10 May 1994 09:49:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares10.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.8s-UTK) id JAA15466; Tue, 10 May 1994 09:49:41 -0400 Received: from godzilla (godzilla.mcs.anl.gov [140.221.5.136]) by antares10.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id IAA24329 for ; Tue, 10 May 1994 08:49:38 -0500 From: William Gropp Date: Tue, 10 May 1994 08:49:37 -0500 Message-Id: <199405101349.IAA15619@godzilla> To: mpi-core@CS.UTK.EDU Subject: MPI_KEYVAL_CREATE functions There are some inconsistencies in the description of the use of MPI_NULL_FN when used with MPI_KEYVAL_CREATE. In particular, when describing the "copy_function", we say "MPI_NULL_FN is a function that does nothing other than returning flag = 0." (flag is the sixth argument!) This is inconsistent with a null function (by definition, a null function should do nothing) and will fail when used as the "delete_function", where we say "It may be specified as the null function pointer in C or as MPI_NULL_FN from either C or FORTRAN, in which case no delete callback occurs for keyval." There are two obvious fixes: 1) Replace MPI_NULL_FN with MPI_NULL_COPY_FN (for correctness) and MPI_NULL_DELETE_FN (for consistency) in the discussion of MPI_KEYVAL_CREATE. 2) Change the wording under "copy_function" to "MPI_NULL_FN is a function that does nothing other; when used as the copy function, it has the same effect as a function that does nothing other than returning flag = 0." I prefer (1) because it is clearer as to what is going on and (naturally) makes the implementation simpler. Implementing (2) is straightforward but unpleasant. Comments? Bill From owner-mpi-core@CS.UTK.EDU Tue May 10 10:26:44 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA11176; Tue, 10 May 1994 10:26:44 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA19969; Tue, 10 May 1994 10:27:05 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 10 May 1994 10:27:03 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA19945; Tue, 10 May 1994 10:27:00 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA10213; Tue, 10 May 94 09:26:41 CDT Date: Tue, 10 May 94 09:26:41 CDT From: Tony Skjellum Message-Id: <9405101426.AA10213@Aurora.CS.MsState.Edu> To: gropp@mcs.anl.gov, mpi-core@CS.UTK.EDU Subject: Re: MPI_KEYVAL_CREATE functions Bill, I prefer your option #1. When Marc and I revised this section, we worked hard to get it right, but obviously failed. Marc wanted only to permit the MPI_NULL_FN option from either C or Fortran, and never to permit NULL in C. I argued that it was natural for a C programmer to pose NULL as an argument. In your option #2, the first clause ends at a semi-colon preceded by the word "other" and I can't parse it precisely. -Tony From owner-mpi-core@CS.UTK.EDU Tue May 10 10:51:35 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA11325; Tue, 10 May 1994 10:51:35 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.8s-UTK) id KAA23253; Tue, 10 May 1994 10:51:53 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 10 May 1994 10:51:50 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.8s-UTK) id KAA23246; Tue, 10 May 1994 10:51:46 -0400 Received: from godzilla (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id JAA01461 for ; Tue, 10 May 1994 09:51:44 -0500 From: William Gropp Date: Tue, 10 May 1994 09:51:41 -0500 Message-Id: <199405101451.JAA19557@godzilla> To: mpi-core@CS.UTK.EDU An extra word crept into the description of option 2; it should read: 2) Change the wording under "copy_function" to "MPI_NULL_FN is a function that does nothing; when used as the copy function, it has the same effect as a function that does nothing other than returning flag = 0." From owner-mpi-core@CS.UTK.EDU Fri May 20 10:20:11 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id KAA13792; Fri, 20 May 1994 10:20:11 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA09024; Fri, 20 May 1994 10:19:27 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 20 May 1994 10:19:25 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id KAA08999; Fri, 20 May 1994 10:19:23 -0400 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.8/8.6.6) id KAA16280; Fri, 20 May 1994 10:19:20 -0400 From: David Walker Message-Id: <199405201419.KAA16280@rios2.EPM.ORNL.GOV> To: mpi-core@CS.UTK.EDU Subject: Error in example 3.38 Date: Fri, 20 May 94 10:19:20 -0500 In lines 26 and 27, page 88 of the May 5 MPI draft, shouldn't "&lbuf" just be "lbuf". David From owner-mpi-core@CS.UTK.EDU Fri May 20 15:27:29 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id PAA15774; Fri, 20 May 1994 15:27:29 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA14655; Fri, 20 May 1994 15:26:57 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 20 May 1994 15:26:56 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares10.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA14648; Fri, 20 May 1994 15:26:55 -0400 Received: from godzilla (godzilla.mcs.anl.gov [140.221.5.136]) by antares10.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id OAA07445 for ; Fri, 20 May 1994 14:26:50 -0500 From: William Gropp Date: Fri, 20 May 1994 14:26:46 -0500 Message-Id: <199405201926.OAA06772@godzilla> To: mpi-core@CS.UTK.EDU Subject: MPI bindings There is one error: Page 215, line 31: int MPI_Comm_compare( MPI_Comm comm1, comm2, int *result ) should be int MPI_Comm_compare( MPI_Comm comm1, MPI_Comm comm2, int *result ) (in the first form, ANSI C makes comm2 an int!). This probably needs to be changed in the text as well. There are two bindings that may not be as intended. Page 211, line 24, has int MPI_Get_count( MPI_Status status, MPI_Datatype datatype, int *count ) Page 213, line 35, has int MPI_Get_elements( MPI_Status status, MPI_Datatype datatype, int *count ) (there is also MPI_Test_cancelled( MPI_Status, etc )) Now, since MPI_Status is not an opaque object but is a specified structure, we are passing a structure to these routines rather than a pointer to the structure. This isn't wrong, but it may not be what we intended (it certainly isn't what *I* intended). Any comments? Bill From owner-mpi-core@CS.UTK.EDU Mon May 23 04:45:32 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id EAA29037; Mon, 23 May 1994 04:45:32 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA03300; Mon, 23 May 1994 04:45:25 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 23 May 1994 04:45:25 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA03245; Mon, 23 May 1994 04:45:04 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA16966 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Mon, 23 May 1994 09:43:46 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA22842; Mon, 23 May 1994 09:42:57 +0000 Date: Mon, 23 May 1994 09:42:57 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9405230842.AA22842@tycho.co.uk> To: gropp@mcs.anl.gov Cc: mpi-core@CS.UTK.EDU In-Reply-To: <199405201926.OAA06772@godzilla> (message from William Gropp on Fri, 20 May 1994 14:26:46 -0500) Subject: Re: MPI bindings > There are two bindings that may not be as intended. > Page 211, line 24, has > > int MPI_Get_count( MPI_Status status, MPI_Datatype datatype, int *count ) > > Page 213, line 35, has > > int MPI_Get_elements( MPI_Status status, MPI_Datatype datatype, int *count ) > > (there is also MPI_Test_cancelled( MPI_Status, etc )) > > Now, since MPI_Status is not an opaque object but is a specified structure, > we are passing a structure to these routines rather than a pointer to the > structure. This isn't wrong, but it may not be what we intended (it certainly > isn't what *I* intended). Any comments? > Bill Yes, Bill is right. We definitely want MPI_Status * here. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 454 616171 +1 508 371 0088 FAX : +44 454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com:8080/welcome.html From owner-mpi-core@CS.UTK.EDU Mon Jun 20 09:16:46 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id JAA10178; Mon, 20 Jun 1994 09:16:46 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA17065; Mon, 20 Jun 1994 09:16:14 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 20 Jun 1994 09:16:13 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id JAA17058; Mon, 20 Jun 1994 09:16:12 -0400 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.8/8.6.6) id JAA19847; Mon, 20 Jun 1994 09:15:58 -0400 From: David Walker Message-Id: <199406201315.JAA19847@rios2.EPM.ORNL.GOV> To: mpi-core@CS.UTK.EDU, gcf@nova.npac.syr.edu Subject: PVM wins award Date: Mon, 20 Jun 94 09:15:58 -0500 Well, maybe MPI will be getting one of these awards in a couple of years!?! Congratulations to the PVM team! David > From: sincovec@sirius.epm.ornl.gov > Subject: PVM Wins 1994 R&D 100 Award > Date: Fri, 17 Jun 94 08:52:20 -0400 > > PVM 3.2 has won a coveted 1994 R&D 100 Award for significant technology. > > The R&D 100 Awards, now in its 32nd year, recognize the 100 most > technological significant new products of the year. The awards are called > "The Oscars of Invention" by the Chicago Tribune and other have referred > to the R&D 100 Award as the "Nobel Prize of Applied Research." > > The award will be received by Adam Beguelin, Jack Dongarra, Al Geist, Bob > Manchek, and Vaidy Sunderam at the ceremonial black-tie Awards > Banquet on September 22, 1994, at the Museum of Science & Industry in > Chicago. > > Congratulations to Al and all who are on the PVM Team. > > Rich > > > From owner-mpi-core@CS.UTK.EDU Fri Jun 24 14:25:19 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id OAA11026; Fri, 24 Jun 1994 14:25:19 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA13034; Fri, 24 Jun 1994 14:24:21 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 24 Jun 1994 14:24:20 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id OAA13027; Fri, 24 Jun 1994 14:24:18 -0400 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.8/8.6.6) id OAA19585; Fri, 24 Jun 1994 14:24:17 -0400 Date: Fri, 24 Jun 1994 14:24:17 -0400 From: David Walker Message-Id: <199406241824.OAA19585@rios2.EPM.ORNL.GOV> To: mpi-core@CS.UTK.EDU Subject: Caching and collective communication Cc: walker@rios2.EPM.ORNL.GOV I'm trying to understand better how MPI's caching facility can be used to implement collective communication routines. Suppose that every time we create a new communicator we also create a "secret" communicator for colelctive communication. This secret communicator would be cached with the new communicator. Then inside of a collective communication routine call MPI_ATTR_GET to get the communicator for collective communication, like this: SUBROUTINE MY_COLLECTIVE_CALL (.....,comm) . call mpi_attr_get (comm, keyval, collect_comm, flag, ierr) . now use collect_comm to do the communication safely Is this the right idea?? If so, what is keyval. I suppose there has to be a predefined MPI_COLLECTIVE_KEY to use in mpi_attr_get. Without this I can't see how MPI "..permits routines like collective communication and application topologies to be implemented as portable code, and also [allow] user-written routines to use standard MPI calling sequences." as stated on page 165 of the May 5 spec. Any clarification would be appreciated. David From owner-mpi-core@CS.UTK.EDU Fri Jun 24 15:49:12 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id PAA11416; Fri, 24 Jun 1994 15:49:11 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA19028; Fri, 24 Jun 1994 15:48:47 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 24 Jun 1994 15:48:46 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA19020; Fri, 24 Jun 1994 15:48:43 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA14632; Fri, 24 Jun 94 14:47:28 CDT Date: Fri, 24 Jun 94 14:47:28 CDT From: Tony Skjellum Message-Id: <9406241947.AA14632@Aurora.CS.MsState.Edu> To: walker@rios2.EPM.ORNL.GOV Subject: Re: Caching and collective communication Cc: mpi-core@CS.UTK.EDU ----- Begin Included Message ----- From owner-mpi-core@CS.UTK.EDU Fri Jun 24 13:31:16 1994 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 24 Jun 1994 14:24:20 EDT Date: Fri, 24 Jun 1994 14:24:17 -0400 From: David Walker To: mpi-core@CS.UTK.EDU Subject: Caching and collective communication Cc: walker@rios2.EPM.ORNL.GOV Content-Length: 1084 I'm trying to understand better how MPI's caching facility can be used to implement collective communication routines. Suppose that every time we create a new communicator we also create a "secret" communicator for colelctive communication. This secret communicator would be cached with the new communicator. Then inside of a collective communication routine call MPI_ATTR_GET to get the communicator for collective communication, like this: SUBROUTINE MY_COLLECTIVE_CALL (.....,comm) . call mpi_attr_get (comm, keyval, collect_comm, flag, ierr) . now use collect_comm to do the communication safely Is this the right idea?? If so, what is keyval. I suppose there has to be a predefined MPI_COLLECTIVE_KEY to use in mpi_attr_get. Without this I can't see how MPI "..permits routines like collective communication and application topologies to be implemented as portable code, and also [allow] user-written routines to use standard MPI calling sequences." as stated on page 165 of the May 5 spec. Any clarification would be appreciated. David ----- End Included Message ----- David, let us call the new collective operation MY_COLLECTIVE_CALL. Since you define the code for this, and once and for all, you get a KEYVAL for this call; it will be process local, but the same for all communicators in that process. So, you might have int my_collective_op_keyval = MPI_KEYVAL_INVALID; defined in the same file as your collective operation. Whenever you choose to endow a communicator with MY_COLLECTIVE_CALL, you do the following: 1) See if the communicator already has an attribute with value my_collective_op_keyval. 2) If not, register an attribute with that keyval, and make the value of that keyval be a pointer to an appropriate structure (see below); The structure would contain, at the very least, a communicator: struct walker_op_struct { MPI_Comm *comm; /* other stuff */ }; BTW, that communicator would be made by doing MPI_COMM_CREATE() on your current communicator, so that no attributes would pass on to it. 3) If it does, look up the current value of that attribute (the same structure), and extract the information you need to make your collective operation work, such as the duplicate communicator. -Tony From owner-mpi-core@CS.UTK.EDU Mon Jun 27 04:54:10 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id EAA25879; Mon, 27 Jun 1994 04:54:09 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA19712; Mon, 27 Jun 1994 04:53:52 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 27 Jun 1994 04:53:47 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA19704; Mon, 27 Jun 1994 04:53:27 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA02367 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Mon, 27 Jun 1994 09:50:54 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA00963; Mon, 27 Jun 1994 09:49:49 +0000 Date: Mon, 27 Jun 1994 09:49:49 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9406270849.AA00963@tycho.co.uk> To: tony@Aurora.CS.MsState.Edu Cc: walker@rios2.EPM.ORNL.GOV, mpi-core@CS.UTK.EDU In-Reply-To: <9406241947.AA14632@Aurora.CS.MsState.Edu> (message from Tony Skjellum on Fri, 24 Jun 94 14:47:28 CDT) Subject: Re: Caching and collective communication You missed a minor detail, Tony. > int my_collective_op_keyval = MPI_KEYVAL_INVALID; > > defined in the same file as your collective operation. > > Whenever you choose to endow a communicator with > MY_COLLECTIVE_CALL, you do the following: 0) See if we have already allocated a key and if not allocate it. (This will only happen the first time the collective op is ever used in this process.) if (my_collective_op_keyval == MPI_KEYVAL_INVALID) { if (MPI_Keyval_create( mydelete, NULL, &my_collective_op_keyval, NULL) != MPI_SUCESS) MPI_Abort(MPI_COMM_WORLD, -1); } > 1) See if the communicator already has an attribute with value > my_collective_op_keyval. etc... -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 454 616171 +1 508 371 0088 FAX : +44 454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com:8080/welcome.html From owner-mpi-core@cs.utk.edu Thu Jun 30 15:38:17 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id PAA13718; Thu, 30 Jun 1994 15:38:17 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA12584; Thu, 30 Jun 1994 11:16:43 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 30 Jun 1994 11:16:41 EDT Errors-to: owner-mpi-core@cs.utk.edu Received: from epcc.ed.ac.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA12410; Thu, 30 Jun 1994 11:14:30 -0400 Date: Thu, 30 Jun 94 16:14:14 BST Message-Id: <23291.9406301514@subnode.epcc.ed.ac.uk> From: L J Clarke Subject: Opaque objects and cross-calling To: mpi-core@cs.utk.edu Reply-To: lyndon@epcc.ed.ac.uk In MPI we have defined the handle to opaque objects in Fortran to be an INTEGER and in C as a special type, for example MPI_Comm for a communicator. The expectation is that in Fortran the INTEGER can be a table index whereas in C this can be an opaque pointer (although could again be the same kind of table index). Assume I want to write a library in C, with a C user interface, and a Fortran user interface provided by cross-calling. I have an immediate problem if the library interface accepts MPI opaque objects, for example communicators. The Fortran code passes in the handle as an INTEGER. This hits a C cross-call interface routine which has the a long (or int, depending). But this long is of no use to the C code, since it needs an MPI_Comm (for example) to pass into the MPI calls. The type rule can be overcome by casting the Fortran INTEGER appearing as C long into an MPI_Comm, but this is hardly guaranteed to give the desired effect. The integer thing could be a table index and the MPI_Comm thing expecting to be the pointer. We dont have anything in the draft (as far as I can see) to specify relations and/or translations between Fortran environment handles to opaque objects and C environment handles to opaque objects. This effectively denies the possibility of this cross-calling. Sad really since its exactly the kind of thing the library programmer wants to do, which is suppoed to be an MPI strong point. I guess we'd better fix this. Here are two options. 1) Specify that a cast of the Fortran INTEGER handle to an object as manifested in C (dont specify the manifestation as this is what we wanted to avoid all along I think) to the relevant MPI_object produces the C MPI_object handle to the same object. [For completeness, a cast of the MPI_object into an integer gives the Fortran handle corresponding to the C handle.] This seems about the same for the implementor as both being integer types which index into a table, or the Fortran INTEGER having a pointer stuffed into it and the C MPI_object being a pointer. This is my preferred option. It doesnt change the size of the spec and reflects some simple implementation options. 2) Define functions or (pref) macros in the C environment which accept the Fortran INTEGER handle and return the C MPI_object handle [and for completeness vica versa]. To be consistent we'd have to have a different one for each kind of handle. I dont like this one. It adds a swathe of functions to the spec. So the proposal is to add text into section 2.4.1 which mandates the first option above. Best Wishes Lyndon /--------------------------------------------------------\ e||) | Lyndon J Clarke Edinburgh Parallel Computing Centre | e||) c||c | Tel: 031 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | c||c \--------------------------------------------------------/ From owner-mpi-core@CS.UTK.EDU Thu Jun 30 16:18:33 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA14016; Thu, 30 Jun 1994 16:18:33 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA00699; Thu, 30 Jun 1994 16:17:52 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 30 Jun 1994 16:17:50 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id QAA00691; Thu, 30 Jun 1994 16:17:48 -0400 Received: from godzilla (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id PAA05138; Thu, 30 Jun 1994 15:17:46 -0500 From: William Gropp Date: Thu, 30 Jun 1994 15:17:45 -0500 Message-Id: <199406302017.PAA17153@godzilla> To: lyndon@epcc.ed.ac.uk CC: mpi-core@CS.UTK.EDU In-reply-to: <23291.9406301514@subnode.epcc.ed.ac.uk> (message from L J Clarke on Thu, 30 Jun 94 16:14:14 BST) Subject: Re: Opaque objects and cross-calling One problem to be aware of when considering cross-language environment. Consider a hypothetical system with 64 bit ints (in C and in the architecture) and 64 bit IEEE floating point. Assume that, in Fortran, the decision has been made that double precision is 64 bits. Then the Fortran 77 standard REQUIRES that integers occupy only 32 bits (see 4.3, 4.5, and "numeric storage unit"). If we further assume that pointers on this system occupy an int (64 bits) instead of an integer (32 bits), then the first option eliminates the use of pointers to the MPI objects when called from C (on this system). This is not necessarily bad, but it is a consequence of this choice. Bill From owner-mpi-core@CS.UTK.EDU Thu Jun 30 17:16:37 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id RAA14588; Thu, 30 Jun 1994 17:16:36 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA03404; Thu, 30 Jun 1994 17:16:03 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 30 Jun 1994 17:16:02 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA03397; Thu, 30 Jun 1994 17:16:00 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA11661; Thu, 30 Jun 94 14:15:58 -0700 Date: Thu, 30 Jun 94 14:15:58 -0700 From: Steve Otto Message-Id: <9406302115.AA11661@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Opaque objects and cross-calling I'm probably being thick, but it is unclear to me what Lyndon wants the MPI Spec to "specify". It seems to me that his 1st (preferred) proposal amounts to just this: >This seems about the same for the implementor as both being integer >types which index into a table, or the Fortran INTEGER having a pointer >stuffed into it and the C MPI_object being a pointer. that is, the implementor of MPI should just be consistent, for both C and Fortran, as to what the handle actually is -- ie, it is either an index into a table, or a pointer. Now what is it that the standard needs to specify that you don't have? The implementor needs to be consistent, and he or she knows what is going on and can write the casts appropriately. What am I not understanding? Gropp's mail points out, however, that it isn't so simple to just choose the pointer way of doing things, since the Fortran INTEGER may not hold a pointer. --Steve Otto --------------------------------------------------------------------------- Dr Steve Otto | Dept of Computer Science and Engineering *___________ | Oregon Graduate Institute of Science & Technology _____) | Mailing: 20000 NW Walker Rd, PO Box 91000 / | Portland, OR 97291-1000 USA / | email: otto@cse.ogi.edu /o | Phone: (503) 690-1486 (office) | Phone: (503) 641-8399 (home) | FAX: (503) 690-1553 | Shipping: 20000 NW Walker Rd, Beaverton, OR 97006 USA | URL: http://www.cse.ogi.edu/~otto/ --------------------------------------------------------------------------- From owner-mpi-core@CS.UTK.EDU Fri Jul 1 05:31:33 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id FAA16851; Fri, 1 Jul 1994 05:31:32 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id FAA08789; Fri, 1 Jul 1994 05:30:48 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 1 Jul 1994 05:30:46 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from epcc.ed.ac.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id FAA08754; Fri, 1 Jul 1994 05:30:41 -0400 Date: Fri, 1 Jul 94 10:30:35 BST Message-Id: <24352.9407010930@subnode.epcc.ed.ac.uk> From: L J Clarke Subject: Re: Opaque objects and cross-calling To: Steve Otto , mpi-core@CS.UTK.EDU In-Reply-To: Steve Otto's message of Thu, 30 Jun 94 14:15:58 -0700 Reply-To: lyndon@epcc.ed.ac.uk > I'm probably being thick, but it is unclear to me what > Lyndon wants the MPI Spec to "specify". What I'd like the spec to "specify" is (approx) that > that is, the implementor of MPI should just be consistent, > for both C and Fortran, I certainly dont want to specify > as to what the handle actually is -- ie, it is either an > index into a table, or a pointer. Now what is it that the > standard needs to specify that you don't have? > What the standard does not have is anything at all that requires the implemementor to avoid, for example, the situation where in C we have a pointer and in Fortran we have an index, which means that we pretty well cant write mixed language programs which use MPI. > The implementor needs to be consistent, and he or she knows > what is going on and can write the casts appropriately. It looks to me like you are thinking of the MPI implementor here, whereas I am thinking of the MPI user who must do mixed language. C xxfdoit is a Fortran wrapper for the C routine xxcdoit. Its takes a C communicator (INTEGER) and some other stuff as arguments. C SUBROUTINE xxfdoit(comm, ... stuff ...) INTEGER comm C Here we call the C routine xxcdoit, passing on the arguments. CALL xxcdoit(comm, ... stuff ...) RETURN END /* xxcdoit is a C routine which does it, whatever it is, with the stuff, * and the communicator from the fortran language environment. I've * deliberately left out whatever _ appending or prepending or case * apply for cross-calling. */ void xxcdoit(long fcomm, ... stuff ...) { MPI_Comm comm; /* HELP! I cant call the MPI routines with this `long fcomm'. So I'll * just cast it into an MPI_Comm and hope that works fine, but the MPI * standard does not GUARANTEE it works fine, I wish the standard did! */ comm = (MPI_Comm)fcomm; /* * now do it */ } Well, I dont know if the example helps. I'll try to give the literal text I'd like added to the standard, and where. It wont be fast, as I'm slow writing this kinda formal stuff. Best Wishes Lyndon /--------------------------------------------------------\ e||) | Lyndon J Clarke Edinburgh Parallel Computing Centre | e||) c||c | Tel: 031 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | c||c \--------------------------------------------------------/ From owner-mpi-core@CS.UTK.EDU Fri Jul 1 06:36:48 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id GAA17033; Fri, 1 Jul 1994 06:36:47 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id GAA13294; Fri, 1 Jul 1994 06:36:31 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 1 Jul 1994 06:36:30 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id GAA13287; Fri, 1 Jul 1994 06:36:22 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA02435 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Fri, 1 Jul 1994 11:34:33 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA04295; Fri, 1 Jul 1994 11:33:22 +0000 Date: Fri, 1 Jul 1994 11:33:22 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9407011033.AA04295@tycho.co.uk> To: lyndon@epcc.ed.ac.uk Cc: otto@cse.ogi.edu, mpi-core@CS.UTK.EDU In-Reply-To: <24352.9407010930@subnode.epcc.ed.ac.uk> (message from L J Clarke on Fri, 1 Jul 94 10:30:35 BST) Subject: Re: Opaque objects and cross-calling > What the standard does not have is anything at all that requires the > implemementor to avoid, for example, the situation where in C we have a > pointer and in Fortran we have an index, which means that we pretty well > cant write mixed language programs which use MPI. But this is so ANYWAY. There is explicitly no guarantee that data sent from C (using C datatypes) can be received (in C) and then used in Fortran. I do see the problem you are pointing out, BUT if we're going to solve it I much favour adding the necessary routines. I can see no reason to penalise one implementation (C) because of the limitations of another (Fortran). > /* xxcdoit is a C routine which does it, whatever it is, with the stuff, > * and the communicator from the fortran language environment. I've > * deliberately left out whatever _ appending or prepending or case > * apply for cross-calling. > */ Yes, but these trailing underscores and argument changing are at the heart of your problem. C and Fortran do have different calling conventions, and it will be necessary for the interface routine (the one with the trailing _) to understand this, just to be able to handle integer constants (for instance). (This is because of the call by reference in F77 and by value in C). So the structure of the code will (in any case) be Fortran calls foo_ foo_ written in C converts arguments, calls foo written in C, the thing which does the work returns to foo_ to convert results if necessary returns to user level Fortran You are pointing out that we don't provide the user with routines to allow her to write this interface routine, which is indeed true. If we do want to solve this, though, I'd rather provide them than force the implementation to do the same thing in C and Fortran. (These routines (or better macros) will very likely exist anyway in the MPI implementation, since it probably works this way itself). Note once again, though, that for this to be useful at all you are already making STRONG presumptions about the ability to call C from Fortran, and the compatibility of the data representations. MPI does not guarantee any of that ! -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 454 616171 +1 508 371 0088 FAX : +44 454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com:8080/welcome.html From owner-mpi-core@CS.UTK.EDU Fri Jul 1 14:44:37 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id OAA19863; Fri, 1 Jul 1994 14:44:37 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA14597; Fri, 1 Jul 1994 14:44:06 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 1 Jul 1994 14:44:04 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA14583; Fri, 1 Jul 1994 14:44:01 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA14304; Fri, 1 Jul 94 11:43:22 -0700 Date: Fri, 1 Jul 94 11:43:22 -0700 From: Steve Otto Message-Id: <9407011843.AA14304@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: MPI_Comm / INTEGER conversion OK, now I understand what you are asking for. As both Bill and Jim have pointed out, mandating cross-language behavior is tricky, at best. I suggest that this is a discussion for MPI2. We aren't going to change the current spec. --Steve From owner-mpi-core@CS.UTK.EDU Tue Jul 5 18:27:16 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id SAA00191; Tue, 5 Jul 1994 18:27:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id SAA08132; Tue, 5 Jul 1994 18:26:38 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 5 Jul 1994 18:26:37 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id SAA08125; Tue, 5 Jul 1994 18:26:36 -0400 Received: from godzilla (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id RAA20495 for ; Tue, 5 Jul 1994 17:26:34 -0500 From: William Gropp Date: Tue, 5 Jul 1994 17:26:33 -0500 Message-Id: <199407052226.RAA01326@godzilla> To: mpi-core@CS.UTK.EDU Subject: MPI_BOTTOM in Fortran There is a bug (or unintended effect) in the definition of MPI_BOTTOM for Fortran. Specifically, MPI_BOTTOM can be used as a buffer address; in C the appropriate value will often be (void *)0. But in Fortran, there are no pointer types; in most implementations, all values are passed by reference. Now consider the following two code fragments: -------------------------- C: void *p; p = (void *)MPI_BOTTOM; MPI_Send( p, ... ); -------------------------- Fortran: integer p p = MPI_BOTTOM; MPI_Send( p, ... ); -------------------------- The C case is clearly allowed; the Fortran case must not be. The only way that I can see to provide MPI_BOTTOM to Fortran is to make it a special constant that is like a reserved word - assigning it to a variable is erroneous (that is, there is a special location that is known as MPI_BOTTOM; the routines can test for that special location). This may seem obvious, but it is easily overlooked. We certainly did while writing the model implementation. I believe that this merits some comment in the standard, since to Fortran, MPI_BOTTOM is not a "constant" as defined by that standard. Bill From owner-mpi-core@CS.UTK.EDU Sun Jul 10 16:44:38 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA04884; Sun, 10 Jul 1994 16:44:38 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA29231; Sun, 10 Jul 1994 16:44:21 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sun, 10 Jul 1994 16:44:20 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA29224; Sun, 10 Jul 1994 16:44:19 -0400 Message-Id: <199407102044.QAA29224@CS.UTK.EDU> Received: from YKTVMV by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 0235; Sun, 10 Jul 94 16:44:12 EDT Date: Sun, 10 Jul 94 16:29:35 EDT From: "Marc Snir ((914) 945-3204)" To: mpi-core@CS.UTK.EDU Subject: Cross calling Reply-To: SNIR@watson.ibm.com Lyndon wants MPI to specify the cross-language conversions that need be done to move MPI datatypes from Fortran to C and back. I would like to point out that this, by itself, is not sufficient. One also needs to specify the rules for converting the basic C and Fortran datatypes (is INTEGER int or long?). We decided, at some point, not to tackle this issue, because "there are no agreed standards for the correspondence between C types and Fortran types". I suggest we leave it there. On the other hand, it is reasonable to require an implementation dependent mechanism for cross-language calls: Each implementer need specify how MPI datatypes can be passed form Fortran to C and back. I would put that in the "quality of implementation" bag. From owner-mpi-core@CS.UTK.EDU Mon Jul 11 17:46:47 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id RAA13415; Mon, 11 Jul 1994 17:46:46 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA05337; Mon, 11 Jul 1994 17:46:27 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Jul 1994 17:46:26 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA05330; Mon, 11 Jul 1994 17:46:24 -0400 Received: from b125.super.org by super (4.1/SMI-4.1) id AA24842; Mon, 11 Jul 94 17:46:23 EDT Date: Mon, 11 Jul 94 17:46:22 EDT From: lederman@super.org (Steve Huss-Lederman) Message-Id: <9407112146.AA24842@super> Received: by b125.super.org (4.1/SMI-4.1) id AA03209; Mon, 11 Jul 94 17:46:22 EDT To: mpi-core@CS.UTK.EDU Subject: reduce with complex values Hello all, While I am here at Argonne I was asked about the meaning of the reduce operations on complex datatypes. Since I believe I proposed this functionality at the very end, I will propose how to clarify the document. The first question is the meaning of max, min on a complex value. We intended it to be the max, min modulus value. I assume this is ok with everyone. The second question was how this value is returned. It is a little different from most of the reduce values since the modulus is real but acts on a complex value. Here are the possible ways that we thought of: 1) Return the complex value for which the max/min modulus occured. 2) Return the max/min modulus in a real or double variable (as appropriate for how the complex is stored). 3) Return the max/min modulus in the real part of an imaginary value. 4) Return the max/min modulus in the real part of the imaginary value and return the angle in the imaginary part (i.e., make it into polar coordinates). The feeling was that 2) was bad because of the mixture of complex and real/double variables and the fact that you cannot figure out the original value from the modulus. 4) has an appeal from the point of maximizing information returned but will probably lead to user errors. 1) and 3) have their merits but we lean toward 3) since you can get the modulus from the original value but not the other way around. The negative we saw was that you get back the value and not the max/min modulus. We therefore propose that 1) becomes the basis of the clarified text. Any thoughts/comments. Steve and the Argonne Gang (who will remain nameless to protect the innocent). From owner-mpi-core@CS.UTK.EDU Mon Jul 11 22:06:38 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id WAA15479; Mon, 11 Jul 1994 22:06:37 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id WAA21639; Mon, 11 Jul 1994 22:06:11 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Jul 1994 22:06:08 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id WAA21628; Mon, 11 Jul 1994 22:06:06 -0400 Received: from tualatin.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA16943; Mon, 11 Jul 94 19:05:27 PDT Received: by tualatin.SSD.intel.com (4.1/SMI-4.1) id AA01186; Mon, 11 Jul 94 19:05:26 PDT Date: Mon, 11 Jul 94 19:05:26 PDT Message-Id: <9407120205.AA01186@tualatin.SSD.intel.com> From: Bob Knighten To: lederman@super.org Cc: mpi-core@CS.UTK.EDU Subject: Re: reduce with complex values In-Reply-To: <9407112146.AA24842@super> References: <9407112146.AA24842@super> Reply-To: knighten@ssd.intel.com (Bob Knighten) What value is returned when there is a tie? Notice that this is extremely different from the situation with real numbers where tie implies equality. I hope this is readable. I am trying to edit over a very noisy phone line. -- Bob From owner-mpi-core@CS.UTK.EDU Mon Jul 11 23:41:16 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id XAA15556; Mon, 11 Jul 1994 23:41:15 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id XAA29427; Mon, 11 Jul 1994 23:40:56 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Jul 1994 23:40:53 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id XAA29410; Mon, 11 Jul 1994 23:40:51 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA10965; Mon, 11 Jul 94 22:36:24 CDT Date: Mon, 11 Jul 94 22:36:24 CDT From: Tony Skjellum Message-Id: <9407120336.AA10965@Aurora.CS.MsState.Edu> To: knighten@ssd.intel.com, lederman@super.org Subject: Re: reduce with complex values Cc: mpi-core@CS.UTK.EDU One can enforce a stable sort requirement, such as returning the result from the process of lowest rank, whenever a tie occurs. A tie of real numbers also needs a stable ordering rule, so this does not seem too different, and is certainly not a generic happening, because of rounding! From owner-mpi-core@CS.UTK.EDU Mon Jul 11 21:06:57 1994 Received: from Walt.CS.MsState.Edu by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA10749; Mon, 11 Jul 94 21:06:56 CDT Received: from CS.UTK.EDU by Walt.CS.MsState.Edu (4.1/6.0s-FWP); id AA01201; Mon, 11 Jul 94 21:10:41 CDT Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id WAA21639; Mon, 11 Jul 1994 22:06:11 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 11 Jul 1994 22:06:08 EDT Errors-To: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id WAA21628; Mon, 11 Jul 1994 22:06:06 -0400 Received: from tualatin.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA16943; Mon, 11 Jul 94 19:05:27 PDT Received: by tualatin.SSD.intel.com (4.1/SMI-4.1) id AA01186; Mon, 11 Jul 94 19:05:26 PDT Date: Mon, 11 Jul 94 19:05:26 PDT Message-Id: <9407120205.AA01186@tualatin.SSD.intel.com> From: Bob Knighten To: lederman@super.org Cc: mpi-core@CS.UTK.EDU Subject: Re: reduce with complex values In-Reply-To: <9407112146.AA24842@super> References: <9407112146.AA24842@super> Reply-To: knighten@ssd.intel.com (Bob Knighten) Status: R What value is returned when there is a tie? Notice that this is extremely different from the situation with real numbers where tie implies equality. I hope this is readable. I am trying to edit over a very noisy phone line. -- Bob -Tony From owner-mpi-core@CS.UTK.EDU Tue Jul 12 05:02:25 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id FAA16919; Tue, 12 Jul 1994 05:02:25 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id FAA23504; Tue, 12 Jul 1994 05:01:43 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 12 Jul 1994 05:01:40 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id FAA23482; Tue, 12 Jul 1994 05:01:33 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA28029 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Tue, 12 Jul 1994 10:01:25 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA01362; Tue, 12 Jul 1994 09:58:10 +0000 Date: Tue, 12 Jul 1994 09:58:10 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9407120858.AA01362@tycho.co.uk> To: tony@Aurora.CS.MsState.Edu Cc: mpi-core@CS.UTK.EDU Subject: Congratulations RESULT: comp.parallel.mpi passes 506:14 Well done, Tony. -- Jim From owner-mpi-core@CS.UTK.EDU Tue Jul 12 16:25:15 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA26648; Tue, 12 Jul 1994 16:25:14 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA25040; Tue, 12 Jul 1994 16:24:12 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 12 Jul 1994 16:24:10 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA25016; Tue, 12 Jul 1994 16:23:41 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA16216; Tue, 12 Jul 94 13:23:39 -0700 Date: Tue, 12 Jul 94 13:23:39 -0700 From: Steve Otto Message-Id: <9407122023.AA16216@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Errata Sheet Dear MPI, I enclose my first cut at a readable errata sheet. Note that it is six sheets. Please tell me about anything I have missed. No, there won't be an errata to the errata. I will post a new one. I have organized items into two categories: clear errors that will be patched, and unresolved issues (or at least, they look unresolved to me). --Steve Otto %!PS-Adobe-2.0 %%Creator: dvips 5.516 Copyright 1986, 1993 Radical Eye Software %%Title: paper.dvi %%CreationDate: Tue Jul 12 13:20:16 1994 %%Pages: 6 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o paper.ps paper %DVIPSSource: TeX output 1994.07.12:1319 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /@rigin{isls{[ 0 -1 1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{Resolution hsize -72 div mul 0 TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/ogi/staff/otto/papers/Editor/MPI/errata/paper.dvi) @start /Fa 1 16 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE 7FFE3FFC1FF80FF003C010127D9317>15 D E /Fb 2 79 df73 D78 D E /Fc 65 123 df<030600078F00078F00078F00078F00078F 00078F007FFFC0FFFFE0FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFF C0FFFFE0FFFFE07FFFC01E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18 >35 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F0 0FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070 E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F001E 001C00380038007000700070007000E000E000E000E000E000E000E000E0007000700070 007000380038001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070007000 70007000700070007000E000E000E000E001C001C0038007800F001E003C007800F00060 000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007 F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011 047D8F18>I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00 001E00001E00003E00003C00007C0000780000780000F80000F00001F00001E00003E000 03C00003C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C0000 7C0000780000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F 001C07003803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000 E0E000E0E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F0 00131C7E9B18>I<01800380038007800F803F80FF80FB80438003800380038003800380 038003800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03 F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C000 01C00003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C 00E07FFFE0FFFFE07FFFE0131C7E9B18>I<3078FCFC783000000000000000003078FCFC 78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060 071A789318>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFF E0FFFFE07FFFC0130C7E9318>61 D<600000F00000FC00007E00003F00001FC00007E000 03F00001FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC000 3F00007E0000FC0000F0000060000011187D9918>I<00700000F80000F80000D80000D8 0001DC0001DC0001DC00018C00038E00038E00038E00038E000306000707000707000707 000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C 7F9B18>65 DI<00F8E003FEE007FFE00F07 E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E000 00E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F8 00131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C 00F01C00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C 01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00F FFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E0 0000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003 FDC001F1C0141C7E9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>73 D<7F07F0FF87F87F07F01C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0 001DF0001DF0001FB8001FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03 801C01C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F 9B18>II<7E07F0FF0FF87F07F01D81C01D 81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C 39C01C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C015 1C7F9B18>I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 F00780700700780F007FFF003FFE000FF800111C7D9B18>II<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C 03C01C03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C 03801C039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>82 D<03F3801FFF803FFF807C0F80700780E00380E00380E00380E000007000007800003F00 001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E000E0E001E0F001 C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E0 7038E0703800700000700000700000700000700000700000700000700000700000700000 700000700000700000700000700000700000700000700007FF0007FF0007FF00151C7F9B 18>II87 D<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F800 01F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F00070700 0E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F00001E0000 1C00003C0000780000700000F00001E00001C00003C0000780000700000F00001E00E01C 00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC0078 1E00300E0000070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F0 0F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00 000E00000E00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00 380E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C 809B18>I<01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E0 0000E00000E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I< 001F80003F80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80 380F80700780700380E00380E00380E00380E00380E00380E00380700780700780380F80 3C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F003807807003 80700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF 8007FF0001FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001 C0007FFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F0 07FFF80FFFF81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC00 1FF80039E0003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038 E000387800F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00 000E00000E00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3 FC171C809B18>I<03800007C00007C00007C0000380000000000000000000000000007F C000FFC0007FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0 E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C 1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBF BF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3 FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E0 00E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001F00013147E93 18>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E0038 0E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E0000 0E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<7F87E0FF9FF07FBF F803F87803F03003E00003C00003C0000380000380000380000380000380000380000380 000380000380007FFE00FFFF007FFE0015147F9318>114 D<07F7003FFF007FFF00780F 00E00700E00700E007007C00007FE0001FFC0003FE00001F00600780E00380E00380F003 80F80F00FFFF00FFFC00E7F00011147D9318>I<0180000380000380000380000380007F FFC0FFFFC0FFFFC003800003800003800003800003800003800003800003800003800003 80400380E00380E00380E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE0 7E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03 C00E03800E03800E0380070700070700070700038E00038E00038E00038E0001DC0001DC 0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC00 01F80000F80000700000F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF8 7F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03800703800707000707 00038700038600038E0001CE0001CE0000CC0000CC0000DC000078000078000078000070 0000700000700000F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<3F FFF07FFFF07FFFF07001E07003C0700780000F00001E00003C0000F80001F00003C00007 80000F00701E00703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I E /Fd 12 119 df69 D73 D85 D<00003FF80000003FF80000003FF800000003F800000003F800000003F800000003 F800000003F800000003F800000003F800000003F800000003F800000003F800000003F8 00000003F800001FE3F80000FFFBF80003F03FF80007E00FF8000FC007F8001F8003F800 3F8003F8007F0003F8007F0003F8007F0003F800FF0003F800FF0003F800FF0003F800FF 0003F800FF0003F800FF0003F800FF0003F8007F0003F8007F0003F8007F0003F8003F80 03F8001F8003F8000F8007F80007C00FF80003F03BFF8000FFF3FF80003FC3FF80212A7E A926>100 D<003FE00001FFF80003F07E0007C01F000F801F801F800F803F800FC07F00 0FC07F0007C07F0007E0FF0007E0FF0007E0FFFFFFE0FFFFFFE0FF000000FF000000FF00 00007F0000007F0000007F0000003F8000E01F8000E00FC001C007E0038003F81F0000FF FE00001FF0001B1B7E9A20>I108 D110 D<003FE00001FFFC0003F07E000FC01F801F800FC03F800FE03F0007E07F0007F07F0007 F07F0007F0FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007 F87F0007F07F0007F03F800FE03F800FE01F800FC00FC01F8007F07F0001FFFC00003FE0 001D1B7E9A22>I114 D<03FE300FFFF01E03F038 00F0700070F00070F00070F80070FC0000FFE0007FFE007FFF803FFFE01FFFF007FFF800 FFF80003FC0000FC60007CE0003CF0003CF00038F80038FC0070FF01E0F7FFC0C1FF0016 1B7E9A1B>I117 DI E /Fe 18 118 df<00000200000006000000060000000E0000001E0000001E00 00003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F0000020F00 00040F00000C0F0000080F0000100F0000100F0000200F80003FFF800040078000C00780 0080078001000780010007800200078002000780060007801E000F80FF807FF81D207E9F 22>65 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078 0000780000780000F00000F00000F00000F00001E00001E00001E00001E00003C00003C0 0003C00003C0000780000780000780000780000F8000FFF800141F7D9E12>73 D<001FFF0000F80000F00000F00000F00001E00001E00001E00001E00003C00003C00003 C00003C0000780000780000780000780000F00000F00000F00000F00001E00001E00301E 00781E00F83C00F83C00F0780080700040E00021C0001F000018207D9E18>I<0007E040 001C18C0003005800060038000C0038001C0018001800100038001000380010003800100 0380000003C0000003C0000003F8000001FF800001FFE000007FF000001FF0000001F800 000078000000780000003800000038002000380020003800200030006000700060006000 6000E0007000C000E8038000C606000081F800001A217D9F1A>83 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F000 00F00000F00000F0000070010070020030040018380007C00011147C9315>99 D<007C01C207010E011C013C013802780C7BF07C00F000F000F000F00070007001700238 04183807C010147C9315>101 D<00007800019C00033C00033C00071800070000070000 0E00000E00000E00000E00000E0001FFE0001C00001C00001C00001C0000380000380000 380000380000380000700000700000700000700000700000700000E00000E00000E00000 E00000C00001C00001C0000180003180007B0000F300006600003C00001629829F0E>I< 00C001E001E001C000000000000000000000000000000E00330023004380430047008700 0E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E>105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030 000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I E /Ff 75 123 df<001F83E000F06E3001C078780380F8780300F030 07007000070070000700700007007000070070000700700007007000FFFFFF8007007000 070070000700700007007000070070000700700007007000070070000700700007007000 07007000070070000700700007007000070070000700700007007000070070007FE3FF00 1D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007000007000007000007 0000070000070000FFFFE00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F 19>I<003FE000E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E0 0700E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0 0700E00700E00700E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<7038 F87CFC7EFC7E743A0402040204020804080410081008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C 000C00180018003000300030007000600060006000E000E000E000E000E000E000E000E0 00E000E000E000E0006000600060007000300030003000180018000C000C000600020001 000080004000200B2E7DA112>I<800040002000100008000C0006000600030003000180 0180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E0 00C000C000C001C001800180018003000300060006000C00080010002000400080000B2E 7DA112>I<01800180018001800180C183F18F399C0FF003C003C00FF0399CF18FC18301 80018001800180018010147DA117>I<70F8FCFC74040404080810102040060E7C840D> 44 DI<70F8F8F87005057C840D>I<0001000300030006000600 06000C000C000C00180018001800300030003000600060006000C000C000C00180018001 800300030003000600060006000C000C000C001800180018003000300030006000600060 00C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070038070 0380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003F00012 1F7E9D17>I<018003800F80F38003800380038003800380038003800380038003800380 038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C9D17> I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007C00007 C0000780000780000F00000E00001C0000380000700000600000C0000180000300000600 400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E0020 0F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003F00000 3C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040078040 0F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00002E00 002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00200E00 200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0 141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000001000 0010000011F000161C00180E001007001007800003800003800003C00003C00003C07003 C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17>I<00 7C000182000701000E03800C07801C0780380300380000780000700000700000F1F000F2 1C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003C070 03803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF80 7FFF80400100800200800200800400000800000800001000002000002000004000004000 00C00000C00001C000018000038000038000038000038000078000078000078000078000 078000078000078000030000121F7D9D17>I<03F0000C0C001006003003002001806001 806001806001807001807803003E03003F06001FC8000FF00003F80007FC000C7E00103F 00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C 0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700700380F00380F0 0380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C000 0380000380000380000700300700780600780E00700C002018001070000FC000121F7E9D 17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F8700000 000000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0FFFFFF F00000000000000000000000000000000000000000000000000000000000000000FFFFFF F07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000 C001800180010003000200020002000200020002000000000000000000000007000F800F 800F8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C0 000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800002078 000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F 0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F 9F22>65 DI<000FC040007030C0 01C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C000040 78000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007000100 0380020001C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0 070001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000 F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E0 7C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E00460 00781820000FE0001E217D9F24>III75 DIII<001F800000F0F00001C0380007801E00 0F000F000E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0 F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E0 7C0003E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C03800 00F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F00000F000 007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C08001 C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7F FFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F001080 0F0010000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003 C0020001E0040001F00C0000F0080000781000007C1000003C2000003E4000001E400000 0F8000000F8000000780000003C0000007E0000005E0000009F0000018F8000010780000 207C0000603C0000401E0000801F0001800F0001000780020007C0070003C01F8007E0FF E01FFE1F1F7F9E22>II<1FE000 303000781800781C00300E00000E00000E00000E0000FE00078E001E0E00380E00780E00 F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>I E /Fg 15 122 df<70F8FCFC7404040404080810102040060F7C840E>44 D<008003800F80F380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038007C0FFFE0F217CA018 >49 D<03F0000C1C001007002007804003C04003C08003E0F003E0F801E0F801E0F801E0 2003E00003E00003C00003C0000780000700000E00001C0000180000300000600000C000 0180000100000200200400200800201800603000403FFFC07FFFC0FFFFC013217EA018> I<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>52 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0F001 E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C00001 C00001C0000380000380300300780700780600700C002018001030000FC00013227EA018 >57 D<03FFF0001F00000F00000F00000F00000F00000F00000F00000F00000F00000F00 000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00 000F00000F00000F00700F00F80F00F80F00F80E00F01E00401C0020380018700007C000 14237EA119>74 D<000FE00000783C0000E00E0003C00780078003C00F0001E00E0000E0 1E0000F03C0000783C0000787C00007C7C00007C7800003C7800003CF800003EF800003E F800003EF800003EF800003EF800003EF800003EF800003EF800003E7800003C7C00007C 7C00007C3C0000783E0000F81E0000F00F0001E00F0001E0078003C003C0078000E00E00 00783C00000FE0001F247DA226>79 D<03F0200C0C601802603001E07000E0600060E000 60E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF 80003FC00007E00001E00000F00000F0000070800070800070800070800070C00060C000 60E000C0F000C0C80180C6070081FC0014247DA21B>83 D<01FC000707000C03801C01C0 3801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000700000780020 3800201C00400E008007030000FC0013157F9416>101 D<0E00FE001E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>108 D<01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F000 78F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>111 D<02000200020002000600060006000E001E003E00FFF80E000E000E000E000E000E000E 000E000E000E000E000E040E040E040E040E040E040708030801F00E1F7F9E13>116 D<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00F00E00F006017003827800FC7F18157F941B>II121 D E /Fh 17 117 df<0001FE00000007018000001C004000007800200000F000F00000E001 F00001E001F00003C001F00003C000E00003C000000003C000000003C000000003C00000 0003C000000003C000000003C000000003C000000003C000000003C0000000FFFFFFF000 FFFFFFF00003C001F00003C000F00003C000F00003C000F00003C000F00003C000F00003 C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C0 00F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000 F00003C000F00003C000F00003C000F00003C000F00003C000F00007E001F8007FFE1FFF 807FFE1FFF8021327FB124>12 D69 D73 D77 D80 D<007F802001FFE02007C078600F001C601E00 06E03C0003E0380001E0780000E0700000E070000060F0000060F0000060F0000020F000 0020F0000020F8000020F80000007C0000007E0000003F0000003FC000001FF800000FFF 800007FFF80003FFFC0000FFFF00000FFF800000FFC000001FE0000007E0000003F00000 01F0000000F0000000F8000000F88000007880000078800000788000007880000078C000 0078C0000070E00000F0E00000E0F00000E0F80001C0EC000380C7000700C1F01E00807F FC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C00 3C003E001C003E001E003E001E0008001E0000001E0000001E0000001E00000FFE0000FC 1E0003E01E000F801E001F001E003E001E003C001E007C001E00F8001E04F8001E04F800 1E04F8003E04F8003E0478003E047C005E043E008F080F0307F003FC03E01E1F7D9E21> 97 D<003F8000E0600380180700040F00041E001E1C003E3C003E7C003E7C0008780000 F80000F80000F80000F80000F80000F80000F80000F80000F800007800007C00007C0000 3C00011E00011E00020F000207000403801800E060003F80181F7D9E1D>99 D<003F800000E0E0000380380007003C000E001E001E001E001C000F003C000F007C000F 0078000F8078000780F8000780F8000780FFFFFF80F8000000F8000000F8000000F80000 00F8000000F8000000780000007C0000003C0000003C0000801E0000800E0001000F0002 000780020001C00C0000F03000001FC000191F7E9E1D>101 D<0007E0001C1000383800 707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007FFF0016327FB1 14>I<0780000000FF80000000FF800000000F8000000007800000000780000000078000 000007800000000780000000078000000007800000000780000000078000000007800000 00078000000007800000000780000000078000000007800000000780FE00000783078000 078C03C000079001E00007A001E00007A000F00007C000F00007C000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF 8021327EB125>104 D<07000F801F801F800F8007000000000000000000000000000000 00000000000000000780FF80FF800F800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800FC0FFF8FFF80D30 7EAF12>I<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007 C000F00007C000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 000FC001F800FFFC1FFF80FFFC1FFF80211F7E9E25>110 D<001FC00000F0780001C01C 00070007000F0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000 F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000 F07C0001F03C0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F078 00001FC0001D1F7E9E21>I<0781FC00FF860700FF8803C00F9001E007A000F007C00078 078000780780003C0780003C0780003E0780001E0780001F0780001F0780001F0780001F 0780001F0780001F0780001F0780001F0780001F0780003E0780003E0780003C0780007C 0780007807C000F007A000F007A001E00798038007860F000781F8000780000007800000 078000000780000007800000078000000780000007800000078000000780000007800000 0FC00000FFFC0000FFFC0000202D7E9E25>I<0783E0FF8C18FF907C0F907C07A07C07C0 3807C00007C00007C0000780000780000780000780000780000780000780000780000780 000780000780000780000780000780000780000780000780000780000780000FC000FFFE 00FFFE00161F7E9E19>114 D<00400000400000400000400000400000C00000C00000C0 0001C00001C00003C00007C0000FC0001FFFE0FFFFE003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 1003C01003C01003C01003C01003C01003C01003C01001C02001E02000E0400078C0001F 00142C7FAB19>116 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 421 278 a Fh(Errata)23 b(Sheet)f(for)f(MPI)i(Sp)r (eci\014cation)806 404 y Fg(Stev)o(e)15 b(Otto)780 506 y(July)g(12,)i(1994)46 664 y Ff(This)12 b(do)q(cumen)o(t)g(lists)g (errors)f(in)i(the)e(Ma)o(y)g(5th)g(v)o(ersion)h(of)g(the)f(MPI)h(sp)q (eci\014cation.)21 b(Some)11 b(of)g(these)h(will)-25 721 y(b)q(e)17 b(\014xed)g(\(at)e(the)h(galley)h(stage\))f(in)h(the)f (v)o(ersion)g(to)g(b)q(e)h(published)i(b)o(y)d(MIT)g(Press)g(in)h(the)f Fe(International)-25 777 y(Journal)g(of)h(Sup)n(er)n(c)n(omputer)g (Applic)n(ations)p Ff(.)46 833 y(I)e(will)i(add)e(sp)q(eci\014c)j(page) d(and)g(line)i(n)o(um)o(b)q(ers)e(when)h(I)g(get)e(the)i(c)o(hance.)-25 974 y Fd(Errors)-25 1076 y Ff(The)g(follo)o(wing)f(are)g(kno)o(wn)g (errors)g(or)f(c)o(hanges)i(that)e(w)o(e)h(ha)o(v)o(e)g(made)g(to)g (the)g(sp)q(eci\014cation.)31 1166 y(1.)22 b Fc(MPI)p 164 1166 15 2 v 17 w(Status)15 b Ff(passed)i(as)f(p)q(oin)o(ter)g(ev)o (erywhere)h(\(an)f(issue)h(for)f(the)g(C)h(binding)h(only\).)23 b(The)17 b(curren)o(t)89 1222 y(sp)q(eci\014cation)g(has:)89 1321 y Fc(int)23 b(MPI_Get_count\()f(MPI_Status)h(status,)g (MPI_Datatype)f(datatype,)g(int)i(*count)f(\))89 1377 y(int)g(MPI_Get_elements\()f(MPI_Status)g(status,)h(MPI_Datatype)f (datatype,)h(int)g(*count)g(\))89 1434 y(int)g(MPI_Test_cancelled\()e (MPI_Status)i(status,)g(int)g(*flag)g(\))89 1532 y Ff(A)16 b(decision)j(w)o(as)d(reac)o(hed)g(that)g(the)h(\014rst)f(argumen)o(t)g (should)i(b)q(e)f(passed)g(in)g(as)g(a)f(p)q(oin)o(ter,)h(that)f(is,)89 1589 y(the)f(function)h(protot)o(yp)q(es)e(should)j(b)q(e:)89 1687 y Fc(int)23 b(MPI_Get_count\()f(MPI_Status)h(*status,)f (MPI_Datatype)h(datatype,)f(int)i(*count)f(\))89 1744 y(int)g(MPI_Get_elements\()f(MPI_Status)g(*status,)h(MPI_Datatype)f (datatype,)h(int)g(*count)g(\))89 1800 y(int)g(MPI_Test_cancelled\()e (MPI_Status)i(*status,)g(int)g(*flag)g(\))89 1899 y Ff(This)16 b(w)o(as)f(done)h(b)q(ecause)h Fc(MPI)p 634 1899 V 17 w(Status)e Ff(is)h(a)g Fc(struct)f Ff(and)h(passing)g(suc)o(h)g(ob)s (jects)f(as)g(argumen)o(ts)g(is)89 1955 y(a)j(bit)i(of)e(a)h(problem.) 32 b(Some)19 b(compilers)h(still)h(do)e(strange)f(things)h(with)h Fc(struct)p Ff(s)e(as)g(argumen)o(ts.)89 2012 y(Note)c(that)f(no)o(w,)h (there)h(is)f(a)h(sligh)o(t)f(inconsistency)j(with)d(the)h (language-indep)q(enden)o(t)i(sp)q(eci\014cation)89 2068 y(of)d(the)i(functions.)k(These)c(are)f Fb(IN)g Ff(argumen)o(ts,)f(y)o (et)h(w)o(e)g(are)g(passing)h(them)f(b)o(y)g(reference)h(in)g(C.)31 2157 y(2.)22 b(The)14 b(di\013erence)i(b)q(et)o(w)o(een)e(t)o(w)o(o)f (p)q(oin)o(ters)i(to)e(the)h(same)g(storage)f(area)h(should)h(b)q(e)g (of)f(t)o(yp)q(e)g Fc(MPI)p 1746 2157 V 17 w(Aint)p Ff(,)89 2214 y(rather)i(than)g Fc(int)p Ff(.)23 b(This)17 b(is)g(the)f (assumption)h(w)o(e)f(go)g(b)o(y)g(in)i(the)e(de\014nition)j(of)d Fc(MPI)p 1575 2214 V 16 w(Type)p 1687 2214 V 17 w(hvector)89 2270 y Ff(and)h Fc(MPI)p 254 2270 V 17 w(Type)p 367 2270 V 16 w(hindexed)p Ff(.)24 b(If)17 b(w)o(e)g(w)o(an)o(t)f(to)g(b)q(e)i (consisten)o(t,)f(then)g(w)o(e)g(should)h(use)f(the)g(same)g(t)o(yp)q (e)89 2327 y Fc(MPI)p 164 2327 V 17 w(Aint)d Ff(in)i(the)f(functions)h (that)f(return)g(suc)o(h)h(di\013erences;)f(th)o(us,)89 2425 y Fc(MPI_Type_extent\(...,)21 b(MPI_Aint)i(*extent\))89 2482 y(MPI_Type_size\(...,)e(MPI_Aint)i(*size\))89 2538 y(MPI_Type_lb\(...,)e(MPI_Aint)i(*displacement\))89 2594 y(MPI_Type_ub\(...,)e(MPI_Aint)i(*displacement\))910 2719 y Ff(1)p eop %%Page: 2 2 2 1 bop 31 95 a Ff(3.)22 b Fc(MPI_PACK\(...,)g(OUTCOUNT,...\))89 171 y Ff(should)16 b(b)q(e)89 283 y Fc(MPI_PACK\(...,)22 b(OUTSIZE,...\))89 396 y Ff(\(analogous)16 b(to)g Fc(MPI)p 450 396 15 2 v 17 w(UNPACK\(...,)23 b(INSIZE,...\))p Ff(\).)g Fc(OUTCOUNT)16 b Ff(is)i(not)e(a)h(coun)o(t)g(of)f(elemen)o (ts)i(in-)89 452 y(serted)d(in)h(the)f(bu\013er,)g(but)g(the)h(total)e (bu\013er)h(size.)31 546 y(4.)22 b(In)16 b(section)f(3.2.2,)f(the)h (sen)o(tence)h(fragmen)o(t:)189 640 y(MPI)p 281 640 14 2 v 16 w(DOUBLE)p 492 640 V 17 w(COMPLEX)e(for)g(double)h(precision)h (complex)f(in)f(F)l(ortran)f(declared)j(to)189 696 y(b)q(e)g(of)e(t)o (yp)q(e)i(DOUBLE)f(PRECISION;)89 790 y(should)h(b)q(e)189 884 y(MPI)p 281 884 V 16 w(DOUBLE)p 492 884 V 17 w(COMPLEX)e(for)g (double)h(precision)h(complex)f(in)f(F)l(ortran)f(declared)j(to)189 940 y(b)q(e)g(of)e(t)o(yp)q(e)i(DOUBLE)f(COMPLEX;)31 1034 y(5.)22 b(The)11 b(w)o(ording)h(of)f(the)g(description)i(of)e Fc(MPI)p 822 1034 15 2 v 17 w(GROUP)p 959 1034 V 17 w(RANGE)p 1096 1034 V 16 w(EXCL)g Ff(is)h(confusing)g(but)g(I)f(ha)o(v)o(en't)g (decided)89 1090 y(what)j(to)h(c)o(hange)g(it)h(to,)e(y)o(et.)19 b(The)d(function)g(protot)o(yp)q(es)e(are)h(correct,)f(I)i(b)q(eliev)o (e.)31 1184 y(6.)22 b(The)15 b(example)h(on)f(page)g(148.)k(The)d (example)g(w)o(ould)f(b)q(e)h(more)f(illustrious)i(if)89 1297 y Fc(MPI_Reduce\(...,MPI_SUM,)j(1,)k(commslave\))f(->)g (MPI_Reduce\(...,MPI_SUM,)e(0,)i(commslave\))89 1409 y Ff(The)12 b(comm)o(unicator)g Fc(commslave)f Ff(is)i(comp)q(osed)f (of)g(the)g(group)g Fc(grprem)g Ff(whic)o(h)h(is)f Fc(MPI)p 1598 1409 V 17 w(GROUP)p 1735 1409 V 17 w(WORLD)89 1466 y Ff(without)f(pro)q(cess)g(0.)19 b(As)11 b(suc)o(h,)h Fc(grprem)e Ff(is)i(n)o(um)o(b)q(ered)f(from)g(0)g(to)f(\(Numpro)q(cs)h (-)h(2\))e(where)i(Numpro)q(cs)89 1522 y(is)i(the)g(size)h(of)e Fc(MPI)p 420 1522 V 17 w(GROUP)p 557 1522 V 17 w(WORLD)p Ff(.)g(Making)g(the)h(reduce)h(go)f(to)f(pro)q(cess)h(0)g(mak)o(es)f (the)h(ren)o(um)o(b)q(ering)89 1579 y(clear.)25 b(It)17 b(also)g(mak)o(es)f(it)h(w)o(ork)f(for)g(only)h(2)g(no)q(des)g(-)g(the) g(curren)o(t)g(co)q(de)g(fails)h(since)g Fc(grprem)e Ff(w)o(ould)89 1635 y(only)f(ha)o(v)o(e)g(one)h(no)q(de/pro)q(cess.)31 1729 y(7.)22 b(P)o(age)14 b(150:)89 1841 y Fc(init_user_lib\(MPI_Comm) 21 b(*comm,)i(user_lib_t)f(**handle\))h(->)89 1898 y (init_user_lib\(MPI_Comm)e(comm,)i(user_lib_t)f(**handle\))89 2010 y Ff(The)15 b(calls)h(on)g(page)f(149)f(don't)h(pass)g Fc(comm)f Ff(as)h(a)g(p)q(oin)o(ter.)31 2104 y(8.)22 b(P)o(age)12 b(151.)18 b(This)13 b(example)g(seems)g(to)f(imply)h(that) f(MPI)p 1065 2104 14 2 v 17 w(Comm)p 1214 2104 V 15 w(rank)h(returns)f (MPI)p 1575 2104 V 16 w(UNDEFINED)89 2161 y(in)18 b(ma)f(and)g(m)o(b)h (if)g(the)f(calling)i(pro)q(cess)f(is)g(not)f(in)h(comm)p 1131 2161 V 16 w(a)f(or)g(comm)p 1364 2161 V 16 w(b,)h(resp)q(ectiv)o (ely)l(.)28 b(142)17 b(sa)o(ys)89 2217 y(nothing)f(ab)q(out)f(this.)22 b(If)15 b(this)h(w)o(as)f(the)h(in)o(ten)o(t)f(\(that)g(is)h(w)o(ould)g (return)f(MPI)p 1453 2217 V 17 w(UNDEFINED\))g(then)89 2274 y(the)j(text)f(ab)q(out)g(MPI)p 491 2274 V 17 w(COMM)p 660 2274 V 16 w(RANK)h(should)h(sa)o(y)e(this.)28 b(This)18 b(commen)o(t)g(migh)o(t)f(also)h(apply)h(to)89 2330 y(MPI)p 181 2330 V 16 w(GR)o(OUP)p 365 2330 V 17 w(RANK)13 b(on)f(135)f(if)i(w) o(e)f(w)o(an)o(t)f(it)h(to)g(act)g(the)g(same.)18 b(Their)13 b(b)q(eing)h(the)e(same)g(is)h(implied)89 2386 y(in)j(the)f(commen)o (ts)g(in)h(the)f(text.)31 2480 y(9.)22 b(P)o(age)14 b(159.)19 b(It)c(w)o(ould)h(b)q(e)g(a)f(little)h(cleaner)h(to)d(add:)89 2593 y Fc(MPI_Comm_free\(&myComm\);)910 2719 y Ff(2)p eop %%Page: 3 3 3 2 bop 89 95 a Ff(This)16 b(is)f(done)h(for)e(Example)i(2)f(on)g(page) g(160.)8 189 y(10.)22 b(P)o(age)16 b(162)f(Instead)j(of)e(\(MPI)p 611 189 14 2 v 16 w(Comm\))f(0)i(I)g(think)g(it)g(should)h(b)q(e)f(MPI) p 1331 189 V 16 w(COMM)p 1499 189 V 16 w(NULL.)h(145:8)d(sa)o(ys)89 246 y(MPI)p 181 246 V 16 w(Comm)p 329 246 V 16 w(split)f(returns)f(MPI) p 688 246 V 16 w(COMM)p 856 246 V 16 w(NULL)h(when)f(it)h(has)e(funn)o (y)i(cases.)19 b(I)13 b(don't)g(recall)h(read-)89 302 y(ing)h(0)g(and)h(NULL)g(w)o(ere)f(the)g(same)g(for)g(this)g(case.)8 396 y(11.)22 b(P)o(age)14 b(162.)89 509 y Fc(MPI_Attr_keyval_create)21 b(->)i(MPI_Keyval_create)89 621 y Ff(as)15 b(p)q(er)g(page)g(166.)k (Same)c(commen)o(t)g(applies)i(to)e(page)g(170.)8 715 y(12.)22 b(P)o(age)14 b(165.)89 827 y Fc(MPI_Intercomm_create\(loc)o (al_lead)o(er,)21 b(local_comm,)h(buffer[0],new_world,)f(..\))j(->)89 884 y(MPI_Intercomm_create\(loc)o(al_comm)o(,)d(local_leader,)h (new_world,)h(buffer[0],)f(..\))89 996 y Ff(as)15 b(de\014ned)h(on)f (157.)8 1090 y(13.)22 b(P)o(age)10 b(167)g(It)i(seems)f(to)f(me)i(that) e(the)h(F)l(ortran)f(de\014nition)j(needs)f(IERR)o(OR.)h(167)d(sa)o(ys) g(that)h(it)g(returns)89 1147 y(MPI)p 181 1147 V 16 w(SUCCESS)17 b(if)g(it)g(w)o(orks.)23 b(Only)17 b(the)g(C)f(v)o(ersion)h(seems)f(to) g(ha)o(v)o(e)g(a)h(w)o(a)o(y)e(to)h(return)g(this.)25 b(The)89 1203 y(same)15 b(commen)o(t)f(applies)j(to:)i(167)c(for)f (DELETE)p 968 1203 V 16 w(FUNCTION)8 1297 y(14.)22 b(P)o(age)13 b(167)g(Do)h(w)o(e)g(w)o(an)o(t)f(MPI)p 620 1297 V 16 w(NULL)p 760 1297 V 18 w(FN,)g(MPI)p 957 1297 V 16 w(DUP)p 1073 1297 V 17 w(FN)g(in)i(App)q(endix)i(A?)d(It)g(is)h(a)f(function)h (but)89 1353 y(some)g(are)g(listed)8 1447 y(15.)22 b(P)o(age)14 b(167)h(MPI)p 375 1447 V 16 w(KEYV)-5 b(AL)p 582 1447 V 17 w(INV)g(ALID)17 b(seems)e(to)f(b)q(e)i(missing)g(from)f(App)q (endix)i(A.)8 1541 y(16.)22 b(P)o(age)14 b(169)h(A)g(n)o(ull)h (Rational)g(should)h(b)q(e)e(remo)o(v)o(ed.)8 1635 y(17.)22 b(P)o(age)14 b(170)h(MPI)p 375 1635 V 16 w(Ab)q(ort)g(tak)o(es)f(a)h (comm)g(and)g(error)g(co)q(de,)g(not)g(string)g(as)g(de\014ned)i(P)o (age)d(197.)8 1729 y(18.)22 b(P)o(age)14 b(171)89 1841 y Fc(MPI_Comm)23 b(comm)g(->)g(MPI_Comm)g(*comm.)89 1954 y Ff(See)16 b(de\014nition)h(on)e(167.)k(And,)c(F)l(or)g(the)g(same)g (reason,)89 2066 y Fc(int)23 b(keyval)g(->)h(int)f(*keyval)89 2178 y Ff(And)89 2291 y Fc(gop_stuff_copier\(...,)e(gop_stuff,)h (extra\))h(->)89 2347 y(gop_stuff_copier\(...,)e(extra,)i (gop_stuff_in,)f(gop_stuff_out,)g(flag\))89 2460 y Ff(p)q(er)15 b(de\014nition)i(on)f(page)f(166.)k(And)89 2572 y Fc(MPI_Comm)k(comm)g (->)g(MPI_Comm)g(*comm)910 2719 y Ff(3)p eop %%Page: 4 4 4 3 bop 89 95 a Ff(And)89 205 y Fc(int)23 b(keyval)g(->)h(int)f (*keyval.)89 314 y Ff(Also)15 b(need)h(to)f(add)g Fc(*flag)g Ff(for)f(ab)q(o)o(v)o(e)h(c)o(hange)89 423 y Fc(gop_stuff_type)22 b(*gop_stuff)g(->)i(gop_stuff_type)e(*gop_stuff_in,)g(**gop_stuff_out) 89 532 y Ff(And,)15 b(I)h(think)g(w)o(e)f(need)h(to)e(add)i(the)f (line:)89 641 y Fc(*gop_stuff_out)22 b(=)h(gop_stuff_in;)89 751 y Ff(Righ)o(t)18 b(no)o(w,)f(the)g(routine)h(do)q(es)g(not)f(app)q (ear)h(to)f(cop)o(y)g(an)o(ything;)h(only)g(incremen)o(ts)h(the)e (reference)89 807 y(coun)o(t.)i(And,)d(need)g(to)e(mak)o(e)89 916 y Fc(gop_stuff)22 b(->)i(gop_stuff_in)89 1025 y Ff(if)15 b(y)o(ou)g(mak)o(e)g(the)g(other)g(c)o(hanges.)8 1118 y(19.)22 b(P)o(age)14 b(215,)g(line)j(31:)89 1227 y Fc(int)23 b(MPI_Comm_compare\()f(MPI_Comm)g(comm1,)i(comm2,)f(int)g(*result)g(\)) 89 1336 y Ff(should)16 b(b)q(e)89 1446 y Fc(int)23 b (MPI_Comm_compare\()f(MPI_Comm)g(comm1,)i(MPI_Comm)e(comm2,)h(int)h (*result)f(\))-25 1588 y Fd(Unresolv)n(ed)g(Issues)-25 1690 y Ff(The)18 b(follo)o(wing)g(are)f(some)h(issues)g(that)f(are)g (still)i(b)q(eing)g(discussed)g(and)f(that)f(ha)o(v)o(en't)g(y)o(et)g (reac)o(hed)h(con-)-25 1746 y(sensus.)31 1849 y(1.)k(An)15 b(issue)i(related)f(to)e(error)h(n)o(um)o(b)q(er)h(2:)k(Should)d(the)e (argumen)o(ts)g(POSITION)i(and)f(OUTCOUNT)89 1905 y(in)h Fc(MPI)p 218 1905 15 2 v 17 w(Pack)f Ff(b)q(e)h(of)f(t)o(yp)q(e)h Fc(int)f Ff(\(as)f(they)i(are)f(no)o(w\))g(or)g(of)g(t)o(yp)q(e)h Fc(MPI)p 1310 1905 V 16 w(Aint)p Ff(?)24 b(That)16 b(is,)h(do)g(w)o(e)f (w)o(an)o(t)89 1962 y(to)e(allo)o(w)i(for)e(a)h(bu\013er)g(with)h(more) f(than)g(2)g(Gb)o(ytes?)31 2054 y(2.)22 b(In)16 b(lines)g(26)f(and)g (27,)g(page)g(88)f(of)h(the)g(Ma)o(y)g(5)f(MPI)i(draft,)e(shouldn't)h Fc(&lbuf)g Ff(just)g(b)q(e)h Fc(lbuf)e Ff(?)31 2147 y(3.)22 b(There)15 b(are)g(some)f(inconsistencies)k(in)e(the)f(description)h (of)f(the)g(use)g(of)f(MPI)p 1427 2147 14 2 v 17 w(NULL)p 1568 2147 V 17 w(FN)h(when)g(used)89 2203 y(with)e(MPI)p 282 2203 V 16 w(KEYV)-5 b(AL)p 489 2203 V 17 w(CREA)l(TE.)13 b(In)g(particular,)h(when)f(describing)h(the)f("cop)o(y)p 1516 2203 V 16 w(function",)g(w)o(e)g(sa)o(y)189 2296 y(MPI)p 281 2296 V 16 w(NULL)p 421 2296 V 17 w(FN)i(is)h(a)f(function)h (that)e(do)q(es)i(nothing)g(other)e(than)i(returning)f(\015ag)g(=)h(0.) 89 2389 y(This)f(is)g(inconsisten)o(t)h(with)f(a)g(n)o(ull)h(function)f (\(b)o(y)g(de\014nition,)h(a)f(n)o(ull)h(function)f(should)h(do)f (nothing\))89 2445 y(and)g(will)i(fail)f(when)g(used)f(as)g(the)h ("delete)p 838 2445 V 17 w(function",)f(where)h(w)o(e)f(sa)o(y)189 2538 y(It)g(ma)o(y)f(b)q(e)i(sp)q(eci\014ed)h(as)e(the)g(n)o(ull)h (function)g(p)q(oin)o(ter)f(in)h(C)f(or)g(as)f(MPI)p 1442 2538 V 17 w(NULL)p 1583 2538 V 17 w(FN)h(from)189 2594 y(either)h(C)f(or)f(F)o(OR)l(TRAN,)i(in)g(whic)o(h)g(case)f(no)g (delete)i(callbac)o(k)f(o)q(ccurs)f(for)g(k)o(eyv)m(al.)910 2719 y(4)p eop %%Page: 5 5 5 4 bop 89 95 a Ff(There)15 b(are)g(t)o(w)o(o)f(ob)o(vious)h(\014xes:) 20 b(1\))15 b(Replace)i(MPI)p 984 95 14 2 v 16 w(NULL)p 1124 95 V 17 w(FN)e(with)h(MPI)p 1413 95 V 16 w(NULL)p 1553 95 V 17 w(COPY)p 1703 95 V 17 w(FN)f(\(for)89 152 y(correctness\))9 b(and)i(MPI)p 510 152 V 16 w(NULL)p 650 152 V 18 w(DELETE)p 857 152 V 16 w(FN)e(\(for)h(consistency\))g(in) h(the)g(discussion)h(of)d(MPI)p 1745 152 V 17 w(KEYV)-5 b(AL)p 1953 152 V 17 w(CREA)l(TE.)89 208 y(2\))14 b(Change)h(the)h(w)o (ording)f(under)h("cop)o(y)p 806 208 V 16 w(function")f(to)189 298 y(MPI)p 281 298 V 16 w(NULL)p 421 298 V 17 w(FN)e(is)h(a)e (function)i(that)e(do)q(es)h(nothing;)h(when)g(used)f(as)g(the)g(cop)o (y)f(function,)189 354 y(it)k(has)f(the)h(same)f(e\013ect)g(as)h(a)f (function)h(that)f(do)q(es)h(nothing)g(other)g(than)f(returning)h (\015ag)189 411 y(=)f(0.)89 501 y(I)i(prefer)g(\(1\))e(b)q(ecause)j(it) f(is)g(clearer)g(as)f(to)g(what)h(is)g(going)f(on)h(and)g (\(naturally\))f(mak)o(es)h(the)f(imple-)89 557 y(men)o(tation)f (simpler.)21 b(Implemen)o(ting)c(\(2\))d(is)i(straigh)o(tforw)o(ard)d (but)i(unpleasan)o(t.)31 647 y(4.)22 b(Related)16 b(to)f(previous)h (item,)f(w)o(e)g(ha)o(v)o(e)f(\(F)l(rom)h(Hub)q(ertus)g(F)l(rank)o (e\):)89 720 y(I)g(extract)g(this)g(out)g(of)g(the)g(MPI-F)g(include)j (\014les.)89 820 y Fc(#define)23 b(MPI_DUP_FN)70 b(mpi_dup_fn)89 876 y(/*)23 b(note)h(the)f(draft)g(provides)g(MPI_NULL_FN,)f(however)h (this)g(violates)113 932 y(*)g(the)h(prototype)e(differences)h(in)g (usage)h(for)f(copy)g(and)h(delete)f(function)113 989 y(*)g(In)h(agreement)f(with)g(the)g(public)g(domain)g(libary)g (developers)g(we)113 1045 y(*)g(distinguish)g(these)g(functions)160 1102 y(#define)g(MPI_NULL_FN)113 1158 y(*/)89 1215 y(#define)g (MPI_NULL_COPY_FN)93 b(...)89 1271 y(#define)23 b(MPI_NULL_DELETE_FN)45 b(...)89 1371 y Ff(I'v)o(e)19 b(discussed)i(this)f(with)g(Nathan)g (Doss)f(and)g(I)h(b)q(eliev)o(e)i(this)e(has)f(b)q(een)i(incorp)q (orated)f(in)o(to)g(the)89 1428 y(public)d(domain)f(v)o(ersion)f(as)g (w)o(ell.)31 1517 y(5.)22 b(There)14 b(is)h(a)e(long,)i(still)g (unresolv)o(ed)g(discussion)h(cen)o(tering)f(around)f(P)l(A)o(CK/UNP)l (A)o(CK.)g(A)o(t)f(the)h(end)89 1574 y(of)g(it,)i(\(at)e(least)h(the)g (last)h(message)e(I)i(see\),)f(Marc)f(Snir)i(sa)o(ys:)89 1647 y(Conclusion:)143 1736 y Fa(\017)23 b Ff(a\))16 b(A)g(message)g(that)g(w)o(as)f(created)i(b)o(y)f(one)h(or)e(sev)o (eral)i(sequences)h(of)e(related)g(P)l(A)o(CK)h(calls)g(is)189 1793 y(sen)o(t)e(with)g(t)o(yp)q(e)g(MPI)p 581 1793 V 17 w(P)l(A)o(CKED.)143 1862 y Fa(\017)23 b Ff(b\))c(A)h(message)f(that) g(is)h(going)g(to)f(b)q(e)i(unpac)o(k)o(ed)f(b)o(y)g(one)f(or)h(sev)o (eral)f(sequences)i(of)f(related)189 1918 y(UNP)l(A)o(CK)15 b(calls)h(is)g(receiv)o(ed)h(with)e(t)o(yp)q(e)g(MPI)p 1019 1918 V 17 w(P)l(A)o(CKED.)143 1987 y Fa(\017)23 b Ff(c\))d(A)g(Message)f(created)h(b)o(y)g(one)h(sequence)g(of)f (related)g(P)l(A)o(CK)g(calls,)i(and)f(sen)o(t)e(with)i(t)o(yp)q(e)189 2043 y(MPI)p 281 2043 V 16 w(P)l(A)o(CKED)15 b(is)h(iden)o(tical)h(to)d (a)h(message)g(sen)o(t)g(with)h(a)e(regular)i(send.)89 2133 y(I)f(don't)g(understand)h(wh)o(y)e(y)o(ou)h(came)g(to)g(the)g (conclusion)i(it)e(is)h(wrong)e(to)h(send)h(the)f(concatenation)89 2189 y(of)f(sev)o(eral)i(pac)o(king)g(units)f(with)h(t)o(yp)q(e)f(MPI)p 868 2189 V 17 w(P)l(A)o(CKED.)89 2262 y(\(end)g(of)g(Marc\).)k(Where)c (do)g(w)o(e)g(stand)g(here?)31 2352 y(6.)22 b(MPI)p 181 2352 V 16 w(BOTTOM)15 b(in)i(F)l(ortran.)89 2425 y(There)k(is)h(a)f (bug)h(\(or)f(unin)o(tended)i(e\013ect\))e(in)h(the)g(de\014nition)h (of)e(MPI)p 1385 2425 V 16 w(BOTTOM)h(for)e(F)l(ortran.)89 2482 y(Sp)q(eci\014cally)l(,)g(MPI)p 429 2482 V 17 w(BOTTOM)d(can)h(b)q (e)f(used)h(as)f(a)g(bu\013er)g(address;)h(in)g(C)f(the)g(appropriate)h (v)m(alue)89 2538 y(will)12 b(often)e(b)q(e)h(\(v)o(oid)g(*\)0.)17 b(But)10 b(in)i(F)l(ortran,)e(there)g(are)g(no)h(p)q(oin)o(ter)g(t)o (yp)q(es;)g(in)g(most)f(implemen)o(tations,)89 2594 y(all)16 b(v)m(alues)g(are)f(passed)h(b)o(y)f(reference.)20 b(No)o(w)15 b(consider)h(the)f(follo)o(wing)h(t)o(w)o(o)e(co)q(de)i(fragmen)o(ts:) 910 2719 y(5)p eop %%Page: 6 6 6 5 bop 89 95 a Fc(C:)89 152 y(void)23 b(*p;)89 265 y(p)g(=)h(\(void)f (*\)MPI_BOTTOM;)89 321 y(MPI_Send\()f(p,)i(...)f(\);)89 419 y(Fortran:)89 476 y(integer)g(p)89 589 y(p)g(=)h(MPI_BOTTOM;)89 645 y(MPI_Send\()e(p,)i(...)f(\);)89 743 y Ff(The)15 b(C)g(case)g(is)h(clearly)g(allo)o(w)o(ed;)g(the)f(F)l(ortran)f(case)h (m)o(ust)g(not)g(b)q(e.)89 815 y(The)g(only)h(w)o(a)o(y)e(that)h(I)h (can)f(see)h(to)f(pro)o(vide)g(MPI)p 971 815 14 2 v 17 w(BOTTOM)g(to)g(F)l(ortran)f(is)i(to)f(mak)o(e)g(it)g(a)g(sp)q(ecial)89 871 y(constan)o(t)20 b(that)h(is)g(lik)o(e)i(a)d(reserv)o(ed)i(w)o(ord) e(-)i(assigning)f(it)h(to)e(a)h(v)m(ariable)i(is)f(erroneous)f(\(that)f (is,)89 928 y(there)15 b(is)g(a)f(sp)q(ecial)j(lo)q(cation)e(that)f(is) i(kno)o(wn)e(as)h(MPI)p 1042 928 V 16 w(BOTTOM;)f(the)h(routines)g(can) g(test)g(for)f(that)89 984 y(sp)q(ecial)20 b(lo)q(cation\).)31 b(This)19 b(ma)o(y)f(seem)h(ob)o(vious,)h(but)e(it)h(is)h(easily)f(o)o (v)o(erlo)q(ok)o(ed.)30 b(W)l(e)19 b(certainly)h(did)89 1041 y(while)d(writing)g(the)f(mo)q(del)h(implemen)o(tation.)24 b(I)17 b(b)q(eliev)o(e)h(that)e(this)g(merits)g(some)g(commen)o(t)g(in) h(the)89 1097 y(standard,)11 b(since)h(to)e(F)l(ortran,)h(MPI)p 713 1097 V 16 w(BOTTOM)g(is)h(not)e(a)h("constan)o(t")f(as)g(de\014ned) j(b)o(y)d(that)h(standard.)31 1184 y(7.)22 b(The)14 b(requiremen)o(t)h (for)e(an)h(En)o(vironmen)o(tal)h(Enquiry)g(that)e(allo)o(ws)h(to)g (\014nd)h(whether)f(t)o(w)o(o)f(no)q(des)i(use)89 1241 y(the)f(same)f(data)g(represen)o(tation)h(is)h(reasonable,)f(irresp)q (ectiv)o(e)h(of)f(the)g(sp)q(eci\014c)h(use)g(that)e(Rolf)h(has)g(in)89 1297 y(mind,)k(for)f(P)o(armacs)g(em)o(ulation.)27 b(More)17 b(generally)l(,)i(users)e(ma)o(y)g(w)o(an)o(t)g(enquiry)h(functions)g (to)f(\014nd)89 1354 y(the)e(t)o(yp)q(e)g(of)f(no)q(de)i(on)f(whic)o(h) h(a)e(pro)q(cess)i(runs.)k(This)15 b(w)o(ould)h(help)g(in)g(situations) f(where)g(the)g(user)g(is)89 1410 y(willing)i(to)d(run)i(on)f(an)o(y)f (a)o(v)m(ailable)j(mac)o(hine)f(on)f(a)f(net)o(w)o(ork,)g(but)h(ma)o(y) f(w)o(an)o(t)g(to)h(use)g(di\013eren)o(t)g(co)q(de)89 1467 y(on)g(di\013eren)o(t)g(mac)o(hine)h(t)o(yp)q(es.)89 1538 y(It)g(is)g(not)f(reasonable)i(to)e(assume)g(that)g(the)h(MPI)g (forum)f(will)j(dev)o(elop)f(its)f(o)o(wn)f(nomenclature)i(for)89 1595 y(data)g(represen)o(tations)g(\(size)h(of)f(eac)o(h)h(basic)g (language)g(data)f(t)o(yp)q(e,)h(big)g(vs)f(little)i(endian,)g (\015oating)89 1651 y(p)q(oin)o(t)14 b(format,)e(c)o(haracter)h(co)q (des,)h(etc.\).)k(First)c(b)q(ecause)g(other)f(standards)g(try)g(to)g (handle)i(this)f(issue)89 1708 y(and,)22 b(second,)g(b)q(ecause)f(a)g (complete)g(c)o(haracterization)g(of)f(the)h(data)f(represen)o(tation)g (used)i(b)o(y)e(a)89 1764 y(pro)q(cess)e(\(whic)o(h)h(dep)q(ends)h(on)e (the)h(underlying)h(mac)o(hine)f(arc)o(hitecture)g(and)f(on)h(the)f (compiler\))h(is)89 1821 y(lik)o(ely)e(to)d(b)q(e)i(quite)g(length)o(y) l(.)89 1892 y(W)l(e)f(agreed)f(that)h(v)o(endors)f(can)h(attac)o(h)f (implemen)o(tation)i(sp)q(eci\014c)h(prede\014ned)f(attributes)f(that)f (are)89 1949 y(asso)q(ciated)h(with)h(MPI)p 501 1949 V 16 w(COMM)p 669 1949 V 16 w(W)o(ORLD.)f(I)h(w)o(ould)f(suggest)g(the) g(follo)o(wing:)89 2021 y(Asso)q(ciate)f(a)f(prede\014ned)i(attribute)f (k)o(ey)f(MPI)p 907 2021 V 17 w(PR)o(OC)p 1055 2021 V 16 w(TYPE)h(with)g(MPI)p 1405 2021 V 16 w(COMM)p 1573 2021 V 16 w(W)o(ORLD.)g(The)89 2077 y(v)m(alue)k(of)f(this)g(attribute) g(enco)q(des)h(the)f("t)o(yp)q(e")g(of)f(the)h(executing)i(pro)q(cess,) e(i.e.)26 b(the)17 b(t)o(yp)q(e)g(of)g(data)89 2134 y(represen)o (tation)e(used)i(b)o(y)e(this)h(pro)q(cess.)22 b(The)16 b(meaning)g(of)g(the)f(v)m(alues)i(of)e(this)i(attribute)e(is)h(imple-) 89 2190 y(men)o(tation)h(dep)q(enden)o(t,)j(except)e(that)f(if)h(t)o(w) o(o)f(pro)q(cesses)h(return)g(the)g(same)f(v)m(alue,)j(then)e(they)g (use)89 2246 y(the)d(same)g(data)f(represen)o(tation)i(\(i.e.,)e(data)h (can)g(b)q(e)h(mo)o(v)o(ed)f(un)o(t)o(yp)q(ed)g(b)q(et)o(w)o(een)h (these)f(pro)q(cesses\).)89 2318 y(Implemen)o(ters)e(of)f(MPI)g(for)g (homogeneous)g(SPMD)g(systems)g(just)g(need)h(to)f(alw)o(a)o(ys)g (return)g(the)g(same)89 2375 y(v)m(alue)17 b(\(0?\))k(with)15 b(this)i(attribute.)j(Implemen)o(ters)d(of)e(MPI)h(on)f(heterogeneous)h (systems)f(ma)o(y)g(need)89 2431 y(some)g(w)o(ork.)89 2503 y(If)k(this)h(suggestion)f(sounds)h(reasonable,)g(then)f(the)h(v)m (arious)f(MPI)g(implemen)o(ters)i(ma)o(y)d(accept)i(it)89 2559 y(no)o(w)14 b(as)h("common)g(practice",)g(and)g(w)o(e)g(can,)g(at) g(a)g(latter)g(time,)g(add)g(this)h(to)e(the)i(standard.)910 2719 y(6)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Tue Jul 12 16:53:16 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA26829; Tue, 12 Jul 1994 16:53:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA28048; Tue, 12 Jul 1994 16:52:30 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 12 Jul 1994 16:52:29 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA28034; Tue, 12 Jul 1994 16:52:26 -0400 From: Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 0373; Tue, 12 Jul 94 16:52:21 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 8657; Tue, 12 Jul 1994 16:52:21 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Tue, 12 Jul 94 16:52:19 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA16124; Tue, 12 Jul 1994 16:52:15 -0400 Message-Id: <9407122052.AA16124@snir.watson.ibm.com> To: knighten@ssd.intel.com (Bob Knighten) Subject: Re: reduce with complex values Cc: mpi-core@CS.UTK.EDU In-Reply-To: (Your message of Mon, 11 Jul 1994 19:05:26 PDT.) Date: Tue, 12 Jul 94 16:52:20 EDT : -) :-) :-) *** (-: (-: (-: Date: Tue, 12 Jul 1994 16:52:15 -0400 From: Marc Snir The question is how to define MIN(a,b) and Max(a,b), when a,b are complex numbers (who was the person that had the strange idea of supporting this operation, anyway?). The choice that I prefer is If |a| = |b| then Min(a,b) = a and Max(a,b) = a. If we pick this definition, then Reduce_min(a1,...,an) returns the first entry with minimal module and Reduce_max(a1,..,an) will return the first entry with maximal module. We need to give a deterministic rule in order to achieve repeatability and portability. The rule above works OK with MINLOC and MAXLOC -- it guarantees that if MINLOC(, ..., ) = then ai = MIN(a1,...,an), and likewise with MAXLOC and MAX. Another advantage: We have Min(a,b) = 1/Max(1/a, 1/b), assuming both nonzero. (Remember that Min(a,b) = -Max(-a,-b) does not work for complex numbers. Since we work with module, we need multiplicative inverse, not additive inverse.) Note that this makes MIN and MAX with complex arguments associative, but noncommutative operations. Comments? Btw, for complex numbers, need to change the formula defining MINLOC and MAXLOC: rather then uv, there should be |u|<|v|, |u|=|v|, |u|>|v|. From owner-mpi-core@CS.UTK.EDU Tue Jul 12 21:37:12 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id VAA28115; Tue, 12 Jul 1994 21:37:11 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id VAA18287; Tue, 12 Jul 1994 21:36:22 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 12 Jul 1994 21:36:20 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id VAA18226; Tue, 12 Jul 1994 21:35:45 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA17042; Tue, 12 Jul 94 18:35:40 -0700 Date: Tue, 12 Jul 94 18:35:40 -0700 From: Steve Otto Message-Id: <9407130135.AA17042@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Errata Sheet 1.1 Dear MPI, I enclose my first cut at a readable errata sheet. Note that it is six sheets. Please tell me about anything I have missed. No, there won't be an errata to the errata. I will post a new one. I have organized items into two categories: clear errors that will be patched, and unresolved issues (or at least, they look unresolved to me). --Steve Otto I added a few more things. %!PS-Adobe-2.0 %%Creator: dvips 5.516 Copyright 1986, 1993 Radical Eye Software %%Title: paper.dvi %%CreationDate: Tue Jul 12 13:37:25 1994 %%Pages: 7 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o paper.ps paper %DVIPSSource: TeX output 1994.07.12:1337 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /@rigin{isls{[ 0 -1 1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{Resolution hsize -72 div mul 0 TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/ogi/staff/otto/papers/Editor/MPI/errata/paper.dvi) @start /Fa 1 16 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE 7FFE3FFC1FF80FF003C010127D9317>15 D E /Fb 2 79 df73 D78 D E /Fc 65 123 df<030600078F00078F00078F00078F00078F 00078F007FFFC0FFFFE0FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFF C0FFFFE0FFFFE07FFFC01E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18 >35 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F0 0FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070 E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F001E 001C00380038007000700070007000E000E000E000E000E000E000E000E0007000700070 007000380038001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070007000 70007000700070007000E000E000E000E001C001C0038007800F001E003C007800F00060 000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007 F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011 047D8F18>I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00 001E00001E00003E00003C00007C0000780000780000F80000F00001F00001E00003E000 03C00003C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C0000 7C0000780000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F 001C07003803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000 E0E000E0E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F0 00131C7E9B18>I<01800380038007800F803F80FF80FB80438003800380038003800380 038003800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03 F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C000 01C00003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C 00E07FFFE0FFFFE07FFFE0131C7E9B18>I<3078FCFC783000000000000000003078FCFC 78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060 071A789318>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFF E0FFFFE07FFFC0130C7E9318>61 D<600000F00000FC00007E00003F00001FC00007E000 03F00001FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC000 3F00007E0000FC0000F0000060000011187D9918>I<00700000F80000F80000D80000D8 0001DC0001DC0001DC00018C00038E00038E00038E00038E000306000707000707000707 000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C 7F9B18>65 DI<00F8E003FEE007FFE00F07 E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E000 00E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F8 00131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C 00F01C00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C 01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00F FFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E0 0000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003 FDC001F1C0141C7E9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>73 D<7F07F0FF87F87F07F01C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0 001DF0001DF0001FB8001FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03 801C01C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F 9B18>II<7E07F0FF0FF87F07F01D81C01D 81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C 39C01C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C015 1C7F9B18>I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 F00780700700780F007FFF003FFE000FF800111C7D9B18>II<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C 03C01C03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C 03801C039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>82 D<03F3801FFF803FFF807C0F80700780E00380E00380E00380E000007000007800003F00 001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E000E0E001E0F001 C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E0 7038E0703800700000700000700000700000700000700000700000700000700000700000 700000700000700000700000700000700000700000700007FF0007FF0007FF00151C7F9B 18>II87 D<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F800 01F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F00070700 0E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F00001E0000 1C00003C0000780000700000F00001E00001C00003C0000780000700000F00001E00E01C 00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC0078 1E00300E0000070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F0 0F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00 000E00000E00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00 380E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C 809B18>I<01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E0 0000E00000E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I< 001F80003F80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80 380F80700780700380E00380E00380E00380E00380E00380E00380700780700780380F80 3C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F003807807003 80700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF 8007FF0001FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001 C0007FFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F0 07FFF80FFFF81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC00 1FF80039E0003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038 E000387800F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00 000E00000E00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3 FC171C809B18>I<03800007C00007C00007C0000380000000000000000000000000007F C000FFC0007FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0 E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C 1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBF BF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3 FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E0 00E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001F00013147E93 18>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E0038 0E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E0000 0E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<7F87E0FF9FF07FBF F803F87803F03003E00003C00003C0000380000380000380000380000380000380000380 000380000380007FFE00FFFF007FFE0015147F9318>114 D<07F7003FFF007FFF00780F 00E00700E00700E007007C00007FE0001FFC0003FE00001F00600780E00380E00380F003 80F80F00FFFF00FFFC00E7F00011147D9318>I<0180000380000380000380000380007F FFC0FFFFC0FFFFC003800003800003800003800003800003800003800003800003800003 80400380E00380E00380E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE0 7E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03 C00E03800E03800E0380070700070700070700038E00038E00038E00038E0001DC0001DC 0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC00 01F80000F80000700000F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF8 7F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03800703800707000707 00038700038600038E0001CE0001CE0000CC0000CC0000DC000078000078000078000070 0000700000700000F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<3F FFF07FFFF07FFFF07001E07003C0700780000F00001E00003C0000F80001F00003C00007 80000F00701E00703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I E /Fd 12 119 df69 D73 D85 D<00003FF80000003FF80000003FF800000003F800000003F800000003F800000003 F800000003F800000003F800000003F800000003F800000003F800000003F800000003F8 00000003F800001FE3F80000FFFBF80003F03FF80007E00FF8000FC007F8001F8003F800 3F8003F8007F0003F8007F0003F8007F0003F800FF0003F800FF0003F800FF0003F800FF 0003F800FF0003F800FF0003F800FF0003F8007F0003F8007F0003F8007F0003F8003F80 03F8001F8003F8000F8007F80007C00FF80003F03BFF8000FFF3FF80003FC3FF80212A7E A926>100 D<003FE00001FFF80003F07E0007C01F000F801F801F800F803F800FC07F00 0FC07F0007C07F0007E0FF0007E0FF0007E0FFFFFFE0FFFFFFE0FF000000FF000000FF00 00007F0000007F0000007F0000003F8000E01F8000E00FC001C007E0038003F81F0000FF FE00001FF0001B1B7E9A20>I108 D110 D<003FE00001FFFC0003F07E000FC01F801F800FC03F800FE03F0007E07F0007F07F0007 F07F0007F0FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007 F87F0007F07F0007F03F800FE03F800FE01F800FC00FC01F8007F07F0001FFFC00003FE0 001D1B7E9A22>I114 D<03FE300FFFF01E03F038 00F0700070F00070F00070F80070FC0000FFE0007FFE007FFF803FFFE01FFFF007FFF800 FFF80003FC0000FC60007CE0003CF0003CF00038F80038FC0070FF01E0F7FFC0C1FF0016 1B7E9A1B>I117 DI E /Fe 18 118 df<00000200000006000000060000000E0000001E0000001E00 00003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F0000020F00 00040F00000C0F0000080F0000100F0000100F0000200F80003FFF800040078000C00780 0080078001000780010007800200078002000780060007801E000F80FF807FF81D207E9F 22>65 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078 0000780000780000F00000F00000F00000F00001E00001E00001E00001E00003C00003C0 0003C00003C0000780000780000780000780000F8000FFF800141F7D9E12>73 D<001FFF0000F80000F00000F00000F00001E00001E00001E00001E00003C00003C00003 C00003C0000780000780000780000780000F00000F00000F00000F00001E00001E00301E 00781E00F83C00F83C00F0780080700040E00021C0001F000018207D9E18>I<0007E040 001C18C0003005800060038000C0038001C0018001800100038001000380010003800100 0380000003C0000003C0000003F8000001FF800001FFE000007FF000001FF0000001F800 000078000000780000003800000038002000380020003800200030006000700060006000 6000E0007000C000E8038000C606000081F800001A217D9F1A>83 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F000 00F00000F00000F0000070010070020030040018380007C00011147C9315>99 D<007C01C207010E011C013C013802780C7BF07C00F000F000F000F00070007001700238 04183807C010147C9315>101 D<00007800019C00033C00033C00071800070000070000 0E00000E00000E00000E00000E0001FFE0001C00001C00001C00001C0000380000380000 380000380000380000700000700000700000700000700000700000E00000E00000E00000 E00000C00001C00001C0000180003180007B0000F300006600003C00001629829F0E>I< 00C001E001E001C000000000000000000000000000000E00330023004380430047008700 0E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E>105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030 000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I E /Ff 75 123 df<001F83E000F06E3001C078780380F8780300F030 07007000070070000700700007007000070070000700700007007000FFFFFF8007007000 070070000700700007007000070070000700700007007000070070000700700007007000 07007000070070000700700007007000070070000700700007007000070070007FE3FF00 1D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007000007000007000007 0000070000070000FFFFE00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F 19>I<003FE000E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E0 0700E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0 0700E00700E00700E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<7038 F87CFC7EFC7E743A0402040204020804080410081008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C 000C00180018003000300030007000600060006000E000E000E000E000E000E000E000E0 00E000E000E000E0006000600060007000300030003000180018000C000C000600020001 000080004000200B2E7DA112>I<800040002000100008000C0006000600030003000180 0180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E0 00C000C000C001C001800180018003000300060006000C00080010002000400080000B2E 7DA112>I<01800180018001800180C183F18F399C0FF003C003C00FF0399CF18FC18301 80018001800180018010147DA117>I<70F8FCFC74040404080810102040060E7C840D> 44 DI<70F8F8F87005057C840D>I<0001000300030006000600 06000C000C000C00180018001800300030003000600060006000C000C000C00180018001 800300030003000600060006000C000C000C001800180018003000300030006000600060 00C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070038070 0380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003F00012 1F7E9D17>I<018003800F80F38003800380038003800380038003800380038003800380 038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C9D17> I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007C00007 C0000780000780000F00000E00001C0000380000700000600000C0000180000300000600 400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E0020 0F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003F00000 3C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040078040 0F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00002E00 002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00200E00 200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0 141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000001000 0010000011F000161C00180E001007001007800003800003800003C00003C00003C07003 C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17>I<00 7C000182000701000E03800C07801C0780380300380000780000700000700000F1F000F2 1C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003C070 03803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF80 7FFF80400100800200800200800400000800000800001000002000002000004000004000 00C00000C00001C000018000038000038000038000038000078000078000078000078000 078000078000078000030000121F7D9D17>I<03F0000C0C001006003003002001806001 806001806001807001807803003E03003F06001FC8000FF00003F80007FC000C7E00103F 00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C 0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700700380F00380F0 0380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C000 0380000380000380000700300700780600780E00700C002018001070000FC000121F7E9D 17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F8700000 000000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0FFFFFF F00000000000000000000000000000000000000000000000000000000000000000FFFFFF F07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000 C001800180010003000200020002000200020002000000000000000000000007000F800F 800F8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C0 000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800002078 000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F 0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F 9F22>65 DI<000FC040007030C0 01C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C000040 78000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007000100 0380020001C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0 070001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000 F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E0 7C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E00460 00781820000FE0001E217D9F24>III75 DIII<001F800000F0F00001C0380007801E00 0F000F000E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0 F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E0 7C0003E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C03800 00F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F00000F000 007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C08001 C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7F FFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F001080 0F0010000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003 C0020001E0040001F00C0000F0080000781000007C1000003C2000003E4000001E400000 0F8000000F8000000780000003C0000007E0000005E0000009F0000018F8000010780000 207C0000603C0000401E0000801F0001800F0001000780020007C0070003C01F8007E0FF E01FFE1F1F7F9E22>II<1FE000 303000781800781C00300E00000E00000E00000E0000FE00078E001E0E00380E00780E00 F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>I E /Fg 15 122 df<70F8FCFC7404040404080810102040060F7C840E>44 D<008003800F80F380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038007C0FFFE0F217CA018 >49 D<03F0000C1C001007002007804003C04003C08003E0F003E0F801E0F801E0F801E0 2003E00003E00003C00003C0000780000700000E00001C0000180000300000600000C000 0180000100000200200400200800201800603000403FFFC07FFFC0FFFFC013217EA018> I<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>52 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0F001 E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C00001 C00001C0000380000380300300780700780600700C002018001030000FC00013227EA018 >57 D<03FFF0001F00000F00000F00000F00000F00000F00000F00000F00000F00000F00 000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00 000F00000F00000F00700F00F80F00F80F00F80E00F01E00401C0020380018700007C000 14237EA119>74 D<000FE00000783C0000E00E0003C00780078003C00F0001E00E0000E0 1E0000F03C0000783C0000787C00007C7C00007C7800003C7800003CF800003EF800003E F800003EF800003EF800003EF800003EF800003EF800003EF800003E7800003C7C00007C 7C00007C3C0000783E0000F81E0000F00F0001E00F0001E0078003C003C0078000E00E00 00783C00000FE0001F247DA226>79 D<03F0200C0C601802603001E07000E0600060E000 60E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF 80003FC00007E00001E00000F00000F0000070800070800070800070800070C00060C000 60E000C0F000C0C80180C6070081FC0014247DA21B>83 D<01FC000707000C03801C01C0 3801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000700000780020 3800201C00400E008007030000FC0013157F9416>101 D<0E00FE001E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>108 D<01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F000 78F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>111 D<02000200020002000600060006000E001E003E00FFF80E000E000E000E000E000E000E 000E000E000E000E000E040E040E040E040E040E040708030801F00E1F7F9E13>116 D<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00F00E00F006017003827800FC7F18157F941B>II121 D E /Fh 17 117 df<0001FE00000007018000001C004000007800200000F000F00000E001 F00001E001F00003C001F00003C000E00003C000000003C000000003C000000003C00000 0003C000000003C000000003C000000003C000000003C000000003C0000000FFFFFFF000 FFFFFFF00003C001F00003C000F00003C000F00003C000F00003C000F00003C000F00003 C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C0 00F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000 F00003C000F00003C000F00003C000F00003C000F00003C000F00007E001F8007FFE1FFF 807FFE1FFF8021327FB124>12 D69 D73 D77 D80 D<007F802001FFE02007C078600F001C601E00 06E03C0003E0380001E0780000E0700000E070000060F0000060F0000060F0000020F000 0020F0000020F8000020F80000007C0000007E0000003F0000003FC000001FF800000FFF 800007FFF80003FFFC0000FFFF00000FFF800000FFC000001FE0000007E0000003F00000 01F0000000F0000000F8000000F88000007880000078800000788000007880000078C000 0078C0000070E00000F0E00000E0F00000E0F80001C0EC000380C7000700C1F01E00807F FC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C00 3C003E001C003E001E003E001E0008001E0000001E0000001E0000001E00000FFE0000FC 1E0003E01E000F801E001F001E003E001E003C001E007C001E00F8001E04F8001E04F800 1E04F8003E04F8003E0478003E047C005E043E008F080F0307F003FC03E01E1F7D9E21> 97 D<003F8000E0600380180700040F00041E001E1C003E3C003E7C003E7C0008780000 F80000F80000F80000F80000F80000F80000F80000F80000F800007800007C00007C0000 3C00011E00011E00020F000207000403801800E060003F80181F7D9E1D>99 D<003F800000E0E0000380380007003C000E001E001E001E001C000F003C000F007C000F 0078000F8078000780F8000780F8000780FFFFFF80F8000000F8000000F8000000F80000 00F8000000F8000000780000007C0000003C0000003C0000801E0000800E0001000F0002 000780020001C00C0000F03000001FC000191F7E9E1D>101 D<0007E0001C1000383800 707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007FFF0016327FB1 14>I<0780000000FF80000000FF800000000F8000000007800000000780000000078000 000007800000000780000000078000000007800000000780000000078000000007800000 00078000000007800000000780000000078000000007800000000780FE00000783078000 078C03C000079001E00007A001E00007A000F00007C000F00007C000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF 8021327EB125>104 D<07000F801F801F800F8007000000000000000000000000000000 00000000000000000780FF80FF800F800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800FC0FFF8FFF80D30 7EAF12>I<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007 C000F00007C000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 000FC001F800FFFC1FFF80FFFC1FFF80211F7E9E25>110 D<001FC00000F0780001C01C 00070007000F0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000 F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000 F07C0001F03C0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F078 00001FC0001D1F7E9E21>I<0781FC00FF860700FF8803C00F9001E007A000F007C00078 078000780780003C0780003C0780003E0780001E0780001F0780001F0780001F0780001F 0780001F0780001F0780001F0780001F0780001F0780003E0780003E0780003C0780007C 0780007807C000F007A000F007A001E00798038007860F000781F8000780000007800000 078000000780000007800000078000000780000007800000078000000780000007800000 0FC00000FFFC0000FFFC0000202D7E9E25>I<0783E0FF8C18FF907C0F907C07A07C07C0 3807C00007C00007C0000780000780000780000780000780000780000780000780000780 000780000780000780000780000780000780000780000780000780000780000FC000FFFE 00FFFE00161F7E9E19>114 D<00400000400000400000400000400000C00000C00000C0 0001C00001C00003C00007C0000FC0001FFFE0FFFFE003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 1003C01003C01003C01003C01003C01003C01003C01001C02001E02000E0400078C0001F 00142C7FAB19>116 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 421 278 a Fh(Errata)23 b(Sheet)f(for)f(MPI)i(Sp)r (eci\014cation)806 404 y Fg(Stev)o(e)15 b(Otto)780 506 y(July)g(12,)i(1994)46 664 y Ff(This)12 b(do)q(cumen)o(t)g(lists)g (errors)f(in)i(the)e(Ma)o(y)g(5th)g(v)o(ersion)h(of)g(the)f(MPI)h(sp)q (eci\014cation.)21 b(Some)11 b(of)g(these)h(will)-25 721 y(b)q(e)17 b(\014xed)g(\(at)e(the)h(galley)h(stage\))f(in)h(the)f (v)o(ersion)g(to)g(b)q(e)h(published)i(b)o(y)d(MIT)g(Press)g(in)h(the)f Fe(International)-25 777 y(Journal)g(of)h(Sup)n(er)n(c)n(omputer)g (Applic)n(ations)p Ff(.)46 833 y(I)e(will)i(add)e(sp)q(eci\014c)j(page) d(and)g(line)i(n)o(um)o(b)q(ers)e(when)h(I)g(get)e(the)i(c)o(hance.)-25 974 y Fd(Errors)-25 1076 y Ff(The)g(follo)o(wing)f(are)g(kno)o(wn)g (errors)g(or)f(c)o(hanges)i(that)e(w)o(e)h(ha)o(v)o(e)g(made)g(to)g (the)g(sp)q(eci\014cation.)31 1166 y(1.)22 b Fc(MPI)p 164 1166 15 2 v 17 w(Status)15 b Ff(passed)i(as)f(p)q(oin)o(ter)g(ev)o (erywhere)h(\(an)f(issue)h(for)f(the)g(C)h(binding)h(only\).)23 b(The)17 b(curren)o(t)89 1222 y(sp)q(eci\014cation)g(has:)89 1321 y Fc(int)23 b(MPI_Get_count\()f(MPI_Status)h(status,)g (MPI_Datatype)f(datatype,)g(int)i(*count)f(\))89 1377 y(int)g(MPI_Get_elements\()f(MPI_Status)g(status,)h(MPI_Datatype)f (datatype,)h(int)g(*count)g(\))89 1434 y(int)g(MPI_Test_cancelled\()e (MPI_Status)i(status,)g(int)g(*flag)g(\))89 1532 y Ff(A)16 b(decision)j(w)o(as)d(reac)o(hed)g(that)g(the)h(\014rst)f(argumen)o(t)g (should)i(b)q(e)f(passed)g(in)g(as)g(a)f(p)q(oin)o(ter,)h(that)f(is,)89 1589 y(the)f(function)h(protot)o(yp)q(es)e(should)j(b)q(e:)89 1687 y Fc(int)23 b(MPI_Get_count\()f(MPI_Status)h(*status,)f (MPI_Datatype)h(datatype,)f(int)i(*count)f(\))89 1744 y(int)g(MPI_Get_elements\()f(MPI_Status)g(*status,)h(MPI_Datatype)f (datatype,)h(int)g(*count)g(\))89 1800 y(int)g(MPI_Test_cancelled\()e (MPI_Status)i(*status,)g(int)g(*flag)g(\))89 1899 y Ff(This)16 b(w)o(as)f(done)h(b)q(ecause)h Fc(MPI)p 634 1899 V 17 w(Status)e Ff(is)h(a)g Fc(struct)f Ff(and)h(passing)g(suc)o(h)g(ob)s (jects)f(as)g(argumen)o(ts)g(is)89 1955 y(a)j(bit)i(of)e(a)h(problem.) 32 b(Some)19 b(compilers)h(still)h(do)e(strange)f(things)h(with)h Fc(struct)p Ff(s)e(as)g(argumen)o(ts.)89 2012 y(Note)c(that)f(no)o(w,)h (there)h(is)f(a)h(sligh)o(t)f(inconsistency)j(with)d(the)h (language-indep)q(enden)o(t)i(sp)q(eci\014cation)89 2068 y(of)d(the)i(functions.)k(These)c(are)f Fb(IN)g Ff(argumen)o(ts,)f(y)o (et)h(w)o(e)g(are)g(passing)h(them)f(b)o(y)g(reference)h(in)g(C.)31 2157 y(2.)22 b(The)14 b(di\013erence)i(b)q(et)o(w)o(een)e(t)o(w)o(o)f (p)q(oin)o(ters)i(to)e(the)h(same)g(storage)f(area)h(should)h(b)q(e)g (of)f(t)o(yp)q(e)g Fc(MPI)p 1746 2157 V 17 w(Aint)p Ff(,)89 2214 y(rather)i(than)g Fc(int)p Ff(.)23 b(This)17 b(is)g(the)f (assumption)h(w)o(e)f(go)g(b)o(y)g(in)i(the)e(de\014nition)j(of)d Fc(MPI)p 1575 2214 V 16 w(Type)p 1687 2214 V 17 w(hvector)89 2270 y Ff(and)h Fc(MPI)p 254 2270 V 17 w(Type)p 367 2270 V 16 w(hindexed)p Ff(.)24 b(If)17 b(w)o(e)g(w)o(an)o(t)f(to)g(b)q(e)i (consisten)o(t,)f(then)g(w)o(e)g(should)h(use)f(the)g(same)g(t)o(yp)q (e)89 2327 y Fc(MPI)p 164 2327 V 17 w(Aint)d Ff(in)i(the)f(functions)h (that)f(return)g(suc)o(h)h(di\013erences;)f(th)o(us,)89 2425 y Fc(MPI_Type_extent\(...,)21 b(MPI_Aint)i(*extent\))89 2482 y(MPI_Type_size\(...,)e(MPI_Aint)i(*size\))89 2538 y(MPI_Type_lb\(...,)e(MPI_Aint)i(*displacement\))89 2594 y(MPI_Type_ub\(...,)e(MPI_Aint)i(*displacement\))910 2719 y Ff(1)p eop %%Page: 2 2 2 1 bop 31 95 a Ff(3.)22 b Fc(MPI_PACK\(...,)g(OUTCOUNT,...\))89 171 y Ff(should)16 b(b)q(e)89 283 y Fc(MPI_PACK\(...,)22 b(OUTSIZE,...\))89 396 y Ff(\(analogous)16 b(to)g Fc(MPI)p 450 396 15 2 v 17 w(UNPACK\(...,)23 b(INSIZE,...\))p Ff(\).)g Fc(OUTCOUNT)16 b Ff(is)i(not)e(a)h(coun)o(t)g(of)f(elemen)o (ts)i(in-)89 452 y(serted)d(in)h(the)f(bu\013er,)g(but)g(the)h(total)e (bu\013er)h(size.)31 546 y(4.)22 b(In)16 b(section)f(3.2.2,)f(the)h (sen)o(tence)h(fragmen)o(t:)189 640 y(MPI)p 281 640 14 2 v 16 w(DOUBLE)p 492 640 V 17 w(COMPLEX)e(for)g(double)h(precision)h (complex)f(in)f(F)l(ortran)f(declared)j(to)189 696 y(b)q(e)g(of)e(t)o (yp)q(e)i(DOUBLE)f(PRECISION;)89 790 y(should)h(b)q(e)189 884 y(MPI)p 281 884 V 16 w(DOUBLE)p 492 884 V 17 w(COMPLEX)e(for)g (double)h(precision)h(complex)f(in)f(F)l(ortran)f(declared)j(to)189 940 y(b)q(e)g(of)e(t)o(yp)q(e)i(DOUBLE)f(COMPLEX;)31 1034 y(5.)22 b(The)11 b(w)o(ording)h(of)f(the)g(description)i(of)e Fc(MPI)p 822 1034 15 2 v 17 w(GROUP)p 959 1034 V 17 w(RANGE)p 1096 1034 V 16 w(EXCL)g Ff(is)h(confusing)g(but)g(I)f(ha)o(v)o(en't)g (decided)89 1090 y(what)j(to)h(c)o(hange)g(it)h(to,)e(y)o(et.)19 b(The)d(function)g(protot)o(yp)q(es)e(are)h(correct,)f(I)i(b)q(eliev)o (e.)31 1184 y(6.)22 b(The)15 b(example)h(on)f(page)g(148.)k(The)d (example)g(w)o(ould)f(b)q(e)h(more)f(illustrious)i(if)89 1297 y Fc(MPI_Reduce\(...,MPI_SUM,)j(1,)k(commslave\))f(->)g (MPI_Reduce\(...,MPI_SUM,)e(0,)i(commslave\))89 1409 y Ff(The)12 b(comm)o(unicator)g Fc(commslave)f Ff(is)i(comp)q(osed)f (of)g(the)g(group)g Fc(grprem)g Ff(whic)o(h)h(is)f Fc(MPI)p 1598 1409 V 17 w(GROUP)p 1735 1409 V 17 w(WORLD)89 1466 y Ff(without)f(pro)q(cess)g(0.)19 b(As)11 b(suc)o(h,)h Fc(grprem)e Ff(is)i(n)o(um)o(b)q(ered)f(from)g(0)g(to)f(\(Numpro)q(cs)h (-)h(2\))e(where)i(Numpro)q(cs)89 1522 y(is)i(the)g(size)h(of)e Fc(MPI)p 420 1522 V 17 w(GROUP)p 557 1522 V 17 w(WORLD)p Ff(.)g(Making)g(the)h(reduce)h(go)f(to)f(pro)q(cess)h(0)g(mak)o(es)f (the)h(ren)o(um)o(b)q(ering)89 1579 y(clear.)25 b(It)17 b(also)g(mak)o(es)f(it)h(w)o(ork)f(for)g(only)h(2)g(no)q(des)g(-)g(the) g(curren)o(t)g(co)q(de)g(fails)h(since)g Fc(grprem)e Ff(w)o(ould)89 1635 y(only)f(ha)o(v)o(e)g(one)h(no)q(de/pro)q(cess.)31 1729 y(7.)22 b(P)o(age)14 b(150:)89 1841 y Fc(init_user_lib\(MPI_Comm) 21 b(*comm,)i(user_lib_t)f(**handle\))h(->)89 1898 y (init_user_lib\(MPI_Comm)e(comm,)i(user_lib_t)f(**handle\))89 2010 y Ff(The)15 b(calls)h(on)g(page)f(149)f(don't)h(pass)g Fc(comm)f Ff(as)h(a)g(p)q(oin)o(ter.)31 2104 y(8.)22 b(P)o(age)12 b(151.)18 b(This)13 b(example)g(seems)g(to)f(imply)h(that) f(MPI)p 1065 2104 14 2 v 17 w(Comm)p 1214 2104 V 15 w(rank)h(returns)f (MPI)p 1575 2104 V 16 w(UNDEFINED)89 2161 y(in)18 b(ma)f(and)g(m)o(b)h (if)g(the)f(calling)i(pro)q(cess)f(is)g(not)f(in)h(comm)p 1131 2161 V 16 w(a)f(or)g(comm)p 1364 2161 V 16 w(b,)h(resp)q(ectiv)o (ely)l(.)28 b(142)17 b(sa)o(ys)89 2217 y(nothing)f(ab)q(out)f(this.)22 b(If)15 b(this)h(w)o(as)f(the)h(in)o(ten)o(t)f(\(that)g(is)h(w)o(ould)g (return)f(MPI)p 1453 2217 V 17 w(UNDEFINED\))g(then)89 2274 y(the)j(text)f(ab)q(out)g(MPI)p 491 2274 V 17 w(COMM)p 660 2274 V 16 w(RANK)h(should)h(sa)o(y)e(this.)28 b(This)18 b(commen)o(t)g(migh)o(t)f(also)h(apply)h(to)89 2330 y(MPI)p 181 2330 V 16 w(GR)o(OUP)p 365 2330 V 17 w(RANK)13 b(on)f(135)f(if)i(w) o(e)f(w)o(an)o(t)f(it)h(to)g(act)g(the)g(same.)18 b(Their)13 b(b)q(eing)h(the)e(same)g(is)h(implied)89 2386 y(in)j(the)f(commen)o (ts)g(in)h(the)f(text.)31 2480 y(9.)22 b(P)o(age)14 b(159.)19 b(It)c(w)o(ould)h(b)q(e)g(a)f(little)h(cleaner)h(to)d(add:)89 2593 y Fc(MPI_Comm_free\(&myComm\);)910 2719 y Ff(2)p eop %%Page: 3 3 3 2 bop 89 95 a Ff(This)16 b(is)f(done)h(for)e(Example)i(2)f(on)g(page) g(160.)8 189 y(10.)22 b(P)o(age)16 b(162)f(Instead)j(of)e(\(MPI)p 611 189 14 2 v 16 w(Comm\))f(0)i(I)g(think)g(it)g(should)h(b)q(e)f(MPI) p 1331 189 V 16 w(COMM)p 1499 189 V 16 w(NULL.)h(145:8)d(sa)o(ys)89 246 y(MPI)p 181 246 V 16 w(Comm)p 329 246 V 16 w(split)f(returns)f(MPI) p 688 246 V 16 w(COMM)p 856 246 V 16 w(NULL)h(when)f(it)h(has)e(funn)o (y)i(cases.)19 b(I)13 b(don't)g(recall)h(read-)89 302 y(ing)h(0)g(and)h(NULL)g(w)o(ere)f(the)g(same)g(for)g(this)g(case.)8 396 y(11.)22 b(P)o(age)14 b(162.)89 509 y Fc(MPI_Attr_keyval_create)21 b(->)i(MPI_Keyval_create)89 621 y Ff(as)15 b(p)q(er)g(page)g(166.)k (Same)c(commen)o(t)g(applies)i(to)e(page)g(170.)8 715 y(12.)22 b(P)o(age)14 b(165.)89 827 y Fc(MPI_Intercomm_create\(loc)o (al_lead)o(er,)21 b(local_comm,)h(buffer[0],new_world,)f(..\))j(->)89 884 y(MPI_Intercomm_create\(loc)o(al_comm)o(,)d(local_leader,)h (new_world,)h(buffer[0],)f(..\))89 996 y Ff(as)15 b(de\014ned)h(on)f (157.)8 1090 y(13.)22 b(P)o(age)10 b(167)g(It)i(seems)f(to)f(me)i(that) e(the)h(F)l(ortran)f(de\014nition)j(needs)f(IERR)o(OR.)h(167)d(sa)o(ys) g(that)h(it)g(returns)89 1147 y(MPI)p 181 1147 V 16 w(SUCCESS)17 b(if)g(it)g(w)o(orks.)23 b(Only)17 b(the)g(C)f(v)o(ersion)h(seems)f(to) g(ha)o(v)o(e)g(a)h(w)o(a)o(y)e(to)h(return)g(this.)25 b(The)89 1203 y(same)15 b(commen)o(t)f(applies)j(to:)i(167)c(for)f (DELETE)p 968 1203 V 16 w(FUNCTION)8 1297 y(14.)22 b(P)o(age)13 b(167)g(Do)h(w)o(e)g(w)o(an)o(t)f(MPI)p 620 1297 V 16 w(NULL)p 760 1297 V 18 w(FN,)g(MPI)p 957 1297 V 16 w(DUP)p 1073 1297 V 17 w(FN)g(in)i(App)q(endix)i(A?)d(It)g(is)h(a)f(function)h (but)89 1353 y(some)g(are)g(listed)8 1447 y(15.)22 b(P)o(age)14 b(167)h(MPI)p 375 1447 V 16 w(KEYV)-5 b(AL)p 582 1447 V 17 w(INV)g(ALID)17 b(seems)e(to)f(b)q(e)i(missing)g(from)f(App)q (endix)i(A.)8 1541 y(16.)22 b(P)o(age)14 b(169)h(A)g(n)o(ull)h (Rational)g(should)h(b)q(e)e(remo)o(v)o(ed.)8 1635 y(17.)22 b(P)o(age)14 b(170)h(MPI)p 375 1635 V 16 w(Ab)q(ort)g(tak)o(es)f(a)h (comm)g(and)g(error)g(co)q(de,)g(not)g(string)g(as)g(de\014ned)i(P)o (age)d(197.)8 1729 y(18.)22 b(P)o(age)14 b(171)89 1841 y Fc(MPI_Comm)23 b(comm)g(->)g(MPI_Comm)g(*comm.)89 1954 y Ff(See)16 b(de\014nition)h(on)e(167.)k(And,)c(F)l(or)g(the)g(same)g (reason,)89 2066 y Fc(int)23 b(keyval)g(->)h(int)f(*keyval)89 2178 y Ff(And)89 2291 y Fc(gop_stuff_copier\(...,)e(gop_stuff,)h (extra\))h(->)89 2347 y(gop_stuff_copier\(...,)e(extra,)i (gop_stuff_in,)f(gop_stuff_out,)g(flag\))89 2460 y Ff(p)q(er)15 b(de\014nition)i(on)f(page)f(166.)k(And)89 2572 y Fc(MPI_Comm)k(comm)g (->)g(MPI_Comm)g(*comm)910 2719 y Ff(3)p eop %%Page: 4 4 4 3 bop 89 95 a Ff(And)89 205 y Fc(int)23 b(keyval)g(->)h(int)f (*keyval.)89 314 y Ff(Also)15 b(need)h(to)f(add)g Fc(*flag)g Ff(for)f(ab)q(o)o(v)o(e)h(c)o(hange)89 423 y Fc(gop_stuff_type)22 b(*gop_stuff)g(->)i(gop_stuff_type)e(*gop_stuff_in,)g(**gop_stuff_out) 89 532 y Ff(And,)15 b(I)h(think)g(w)o(e)f(need)h(to)e(add)i(the)f (line:)89 641 y Fc(*gop_stuff_out)22 b(=)h(gop_stuff_in;)89 751 y Ff(Righ)o(t)18 b(no)o(w,)f(the)g(routine)h(do)q(es)g(not)f(app)q (ear)h(to)f(cop)o(y)g(an)o(ything;)h(only)g(incremen)o(ts)h(the)e (reference)89 807 y(coun)o(t.)i(And,)d(need)g(to)e(mak)o(e)89 916 y Fc(gop_stuff)22 b(->)i(gop_stuff_in)89 1025 y Ff(if)15 b(y)o(ou)g(mak)o(e)g(the)g(other)g(c)o(hanges.)8 1118 y(19.)22 b(P)o(age)14 b(215,)g(line)j(31:)89 1227 y Fc(int)23 b(MPI_Comm_compare\()f(MPI_Comm)g(comm1,)i(comm2,)f(int)g(*result)g(\)) 89 1336 y Ff(should)16 b(b)q(e)89 1446 y Fc(int)23 b (MPI_Comm_compare\()f(MPI_Comm)g(comm1,)i(MPI_Comm)e(comm2,)h(int)h (*result)f(\))-25 1588 y Fd(Unresolv)n(ed)g(Issues)-25 1690 y Ff(The)18 b(follo)o(wing)g(are)f(some)h(issues)g(that)f(are)g (still)i(b)q(eing)g(discussed)g(and)f(that)f(ha)o(v)o(en't)g(y)o(et)g (reac)o(hed)h(con-)-25 1746 y(sensus.)31 1849 y(1.)k(An)15 b(issue)i(related)f(to)e(error)h(n)o(um)o(b)q(er)h(2:)k(Should)d(the)e (argumen)o(ts)g(POSITION)i(and)f(OUTCOUNT)89 1905 y(in)h Fc(MPI)p 218 1905 15 2 v 17 w(Pack)f Ff(b)q(e)h(of)f(t)o(yp)q(e)h Fc(int)f Ff(\(as)f(they)i(are)f(no)o(w\))g(or)g(of)g(t)o(yp)q(e)h Fc(MPI)p 1310 1905 V 16 w(Aint)p Ff(?)24 b(That)16 b(is,)h(do)g(w)o(e)f (w)o(an)o(t)89 1962 y(to)e(allo)o(w)i(for)e(a)h(bu\013er)g(with)h(more) f(than)g(2)g(Gb)o(ytes?)31 2054 y(2.)22 b(In)16 b(lines)g(26)f(and)g (27,)g(page)g(88)f(of)h(the)g(Ma)o(y)g(5)f(MPI)i(draft,)e(shouldn't)h Fc(&lbuf)g Ff(just)g(b)q(e)h Fc(lbuf)e Ff(?)31 2147 y(3.)22 b(There)15 b(are)g(some)f(inconsistencies)k(in)e(the)f(description)h (of)f(the)g(use)g(of)f(MPI)p 1427 2147 14 2 v 17 w(NULL)p 1568 2147 V 17 w(FN)h(when)g(used)89 2203 y(with)e(MPI)p 282 2203 V 16 w(KEYV)-5 b(AL)p 489 2203 V 17 w(CREA)l(TE.)13 b(In)g(particular,)h(when)f(describing)h(the)f("cop)o(y)p 1516 2203 V 16 w(function",)g(w)o(e)g(sa)o(y)189 2296 y(MPI)p 281 2296 V 16 w(NULL)p 421 2296 V 17 w(FN)i(is)h(a)f(function)h (that)e(do)q(es)i(nothing)g(other)e(than)i(returning)f(\015ag)g(=)h(0.) 89 2389 y(This)f(is)g(inconsisten)o(t)h(with)f(a)g(n)o(ull)h(function)f (\(b)o(y)g(de\014nition,)h(a)f(n)o(ull)h(function)f(should)h(do)f (nothing\))89 2445 y(and)g(will)i(fail)f(when)g(used)f(as)g(the)h ("delete)p 838 2445 V 17 w(function",)f(where)h(w)o(e)f(sa)o(y)189 2538 y(It)g(ma)o(y)f(b)q(e)i(sp)q(eci\014ed)h(as)e(the)g(n)o(ull)h (function)g(p)q(oin)o(ter)f(in)h(C)f(or)g(as)f(MPI)p 1442 2538 V 17 w(NULL)p 1583 2538 V 17 w(FN)h(from)189 2594 y(either)h(C)f(or)f(F)o(OR)l(TRAN,)i(in)g(whic)o(h)g(case)f(no)g (delete)i(callbac)o(k)f(o)q(ccurs)f(for)g(k)o(eyv)m(al.)910 2719 y(4)p eop %%Page: 5 5 5 4 bop 89 95 a Ff(There)15 b(are)g(t)o(w)o(o)f(ob)o(vious)h(\014xes:) 20 b(1\))15 b(Replace)i(MPI)p 984 95 14 2 v 16 w(NULL)p 1124 95 V 17 w(FN)e(with)h(MPI)p 1413 95 V 16 w(NULL)p 1553 95 V 17 w(COPY)p 1703 95 V 17 w(FN)f(\(for)89 152 y(correctness\))9 b(and)i(MPI)p 510 152 V 16 w(NULL)p 650 152 V 18 w(DELETE)p 857 152 V 16 w(FN)e(\(for)h(consistency\))g(in) h(the)g(discussion)h(of)d(MPI)p 1745 152 V 17 w(KEYV)-5 b(AL)p 1953 152 V 17 w(CREA)l(TE.)89 208 y(2\))14 b(Change)h(the)h(w)o (ording)f(under)h("cop)o(y)p 806 208 V 16 w(function")f(to)189 298 y(MPI)p 281 298 V 16 w(NULL)p 421 298 V 17 w(FN)e(is)h(a)e (function)i(that)e(do)q(es)h(nothing;)h(when)g(used)f(as)g(the)g(cop)o (y)f(function,)189 354 y(it)k(has)f(the)h(same)f(e\013ect)g(as)h(a)f (function)h(that)f(do)q(es)h(nothing)g(other)g(than)f(returning)h (\015ag)189 411 y(=)f(0.)89 501 y(I)i(prefer)g(\(1\))e(b)q(ecause)j(it) f(is)g(clearer)g(as)f(to)g(what)h(is)g(going)f(on)h(and)g (\(naturally\))f(mak)o(es)h(the)f(imple-)89 557 y(men)o(tation)f (simpler.)21 b(Implemen)o(ting)c(\(2\))d(is)i(straigh)o(tforw)o(ard)d (but)i(unpleasan)o(t.)31 647 y(4.)22 b(Related)16 b(to)f(previous)h (item,)f(w)o(e)g(ha)o(v)o(e)f(\(F)l(rom)h(Hub)q(ertus)g(F)l(rank)o (e\):)89 720 y(I)g(extract)g(this)g(out)g(of)g(the)g(MPI-F)g(include)j (\014les.)89 820 y Fc(#define)23 b(MPI_DUP_FN)70 b(mpi_dup_fn)89 876 y(/*)23 b(note)h(the)f(draft)g(provides)g(MPI_NULL_FN,)f(however)h (this)g(violates)113 932 y(*)g(the)h(prototype)e(differences)h(in)g (usage)h(for)f(copy)g(and)h(delete)f(function)113 989 y(*)g(In)h(agreement)f(with)g(the)g(public)g(domain)g(libary)g (developers)g(we)113 1045 y(*)g(distinguish)g(these)g(functions)160 1102 y(#define)g(MPI_NULL_FN)113 1158 y(*/)89 1215 y(#define)g (MPI_NULL_COPY_FN)93 b(...)89 1271 y(#define)23 b(MPI_NULL_DELETE_FN)45 b(...)89 1371 y Ff(I'v)o(e)19 b(discussed)i(this)f(with)g(Nathan)g (Doss)f(and)g(I)h(b)q(eliev)o(e)i(this)e(has)f(b)q(een)i(incorp)q (orated)f(in)o(to)g(the)89 1428 y(public)d(domain)f(v)o(ersion)f(as)g (w)o(ell.)31 1517 y(5.)22 b(There)14 b(is)h(a)e(long,)i(still)g (unresolv)o(ed)g(discussion)h(cen)o(tering)f(around)f(P)l(A)o(CK/UNP)l (A)o(CK.)g(A)o(t)f(the)h(end)89 1574 y(of)g(it,)i(\(at)e(least)h(the)g (last)h(message)e(I)i(see\),)f(Marc)f(Snir)i(sa)o(ys:)89 1647 y(Conclusion:)143 1736 y Fa(\017)23 b Ff(a\))16 b(A)g(message)g(that)g(w)o(as)f(created)i(b)o(y)f(one)h(or)e(sev)o (eral)i(sequences)h(of)e(related)g(P)l(A)o(CK)h(calls)g(is)189 1793 y(sen)o(t)e(with)g(t)o(yp)q(e)g(MPI)p 581 1793 V 17 w(P)l(A)o(CKED.)143 1862 y Fa(\017)23 b Ff(b\))c(A)h(message)f(that) g(is)h(going)g(to)f(b)q(e)i(unpac)o(k)o(ed)f(b)o(y)g(one)f(or)h(sev)o (eral)f(sequences)i(of)f(related)189 1918 y(UNP)l(A)o(CK)15 b(calls)h(is)g(receiv)o(ed)h(with)e(t)o(yp)q(e)g(MPI)p 1019 1918 V 17 w(P)l(A)o(CKED.)143 1987 y Fa(\017)23 b Ff(c\))d(A)g(Message)f(created)h(b)o(y)g(one)h(sequence)g(of)f (related)g(P)l(A)o(CK)g(calls,)i(and)f(sen)o(t)e(with)i(t)o(yp)q(e)189 2043 y(MPI)p 281 2043 V 16 w(P)l(A)o(CKED)15 b(is)h(iden)o(tical)h(to)d (a)h(message)g(sen)o(t)g(with)h(a)e(regular)i(send.)89 2133 y(I)f(don't)g(understand)h(wh)o(y)e(y)o(ou)h(came)g(to)g(the)g (conclusion)i(it)e(is)h(wrong)e(to)h(send)h(the)f(concatenation)89 2189 y(of)f(sev)o(eral)i(pac)o(king)g(units)f(with)h(t)o(yp)q(e)f(MPI)p 868 2189 V 17 w(P)l(A)o(CKED.)89 2262 y(\(end)g(of)g(Marc\).)k(Where)c (do)g(w)o(e)g(stand)g(here?)31 2352 y(6.)22 b(MPI)p 181 2352 V 16 w(BOTTOM)15 b(in)i(F)l(ortran.)89 2425 y(There)k(is)h(a)f (bug)h(\(or)f(unin)o(tended)i(e\013ect\))e(in)h(the)g(de\014nition)h (of)e(MPI)p 1385 2425 V 16 w(BOTTOM)h(for)e(F)l(ortran.)89 2482 y(Sp)q(eci\014cally)l(,)g(MPI)p 429 2482 V 17 w(BOTTOM)d(can)h(b)q (e)f(used)h(as)f(a)g(bu\013er)g(address;)h(in)g(C)f(the)g(appropriate)h (v)m(alue)89 2538 y(will)12 b(often)e(b)q(e)h(\(v)o(oid)g(*\)0.)17 b(But)10 b(in)i(F)l(ortran,)e(there)g(are)g(no)h(p)q(oin)o(ter)g(t)o (yp)q(es;)g(in)g(most)f(implemen)o(tations,)89 2594 y(all)16 b(v)m(alues)g(are)f(passed)h(b)o(y)f(reference.)20 b(No)o(w)15 b(consider)h(the)f(follo)o(wing)h(t)o(w)o(o)e(co)q(de)i(fragmen)o(ts:) 910 2719 y(5)p eop %%Page: 6 6 6 5 bop 89 95 a Fc(C:)89 152 y(void)23 b(*p;)89 265 y(p)g(=)h(\(void)f (*\)MPI_BOTTOM;)89 321 y(MPI_Send\()f(p,)i(...)f(\);)89 431 y(Fortran:)89 487 y(integer)g(p)89 600 y(p)g(=)h(MPI_BOTTOM;)89 657 y(MPI_Send\()e(p,)i(...)f(\);)89 767 y Ff(The)15 b(C)g(case)g(is)h(clearly)g(allo)o(w)o(ed;)g(the)f(F)l(ortran)f(case)h (m)o(ust)g(not)g(b)q(e.)89 840 y(The)g(only)h(w)o(a)o(y)e(that)h(I)h (can)f(see)h(to)f(pro)o(vide)g(MPI)p 971 840 14 2 v 17 w(BOTTOM)g(to)g(F)l(ortran)f(is)i(to)f(mak)o(e)g(it)g(a)g(sp)q(ecial)89 896 y(constan)o(t)20 b(that)h(is)g(lik)o(e)i(a)d(reserv)o(ed)i(w)o(ord) e(-)i(assigning)f(it)h(to)e(a)h(v)m(ariable)i(is)f(erroneous)f(\(that)f (is,)89 953 y(there)15 b(is)g(a)f(sp)q(ecial)j(lo)q(cation)e(that)f(is) i(kno)o(wn)e(as)h(MPI)p 1042 953 V 16 w(BOTTOM;)f(the)h(routines)g(can) g(test)g(for)f(that)89 1009 y(sp)q(ecial)20 b(lo)q(cation\).)31 b(This)19 b(ma)o(y)f(seem)h(ob)o(vious,)h(but)e(it)h(is)h(easily)f(o)o (v)o(erlo)q(ok)o(ed.)30 b(W)l(e)19 b(certainly)h(did)89 1066 y(while)d(writing)g(the)f(mo)q(del)h(implemen)o(tation.)24 b(I)17 b(b)q(eliev)o(e)h(that)e(this)g(merits)g(some)g(commen)o(t)g(in) h(the)89 1122 y(standard,)11 b(since)h(to)e(F)l(ortran,)h(MPI)p 713 1122 V 16 w(BOTTOM)g(is)h(not)e(a)h("constan)o(t")f(as)g(de\014ned) j(b)o(y)d(that)h(standard.)31 1212 y(7.)22 b(The)14 b(requiremen)o(t)h (for)e(an)h(En)o(vironmen)o(tal)h(Enquiry)g(that)e(allo)o(ws)h(to)g (\014nd)h(whether)f(t)o(w)o(o)f(no)q(des)i(use)89 1269 y(the)f(same)f(data)g(represen)o(tation)h(is)h(reasonable,)f(irresp)q (ectiv)o(e)h(of)f(the)g(sp)q(eci\014c)h(use)g(that)e(Rolf)h(has)g(in)89 1325 y(mind,)k(for)f(P)o(armacs)g(em)o(ulation.)27 b(More)17 b(generally)l(,)i(users)e(ma)o(y)g(w)o(an)o(t)g(enquiry)h(functions)g (to)f(\014nd)89 1381 y(the)e(t)o(yp)q(e)g(of)f(no)q(de)i(on)f(whic)o(h) h(a)e(pro)q(cess)i(runs.)k(This)15 b(w)o(ould)h(help)g(in)g(situations) f(where)g(the)g(user)g(is)89 1438 y(willing)i(to)d(run)i(on)f(an)o(y)f (a)o(v)m(ailable)j(mac)o(hine)f(on)f(a)f(net)o(w)o(ork,)g(but)h(ma)o(y) f(w)o(an)o(t)g(to)h(use)g(di\013eren)o(t)g(co)q(de)89 1494 y(on)g(di\013eren)o(t)g(mac)o(hine)h(t)o(yp)q(es.)89 1568 y(It)g(is)g(not)f(reasonable)i(to)e(assume)g(that)g(the)h(MPI)g (forum)f(will)j(dev)o(elop)f(its)f(o)o(wn)f(nomenclature)i(for)89 1624 y(data)g(represen)o(tations)g(\(size)h(of)f(eac)o(h)h(basic)g (language)g(data)f(t)o(yp)q(e,)h(big)g(vs)f(little)i(endian,)g (\015oating)89 1681 y(p)q(oin)o(t)14 b(format,)e(c)o(haracter)h(co)q (des,)h(etc.\).)k(First)c(b)q(ecause)g(other)f(standards)g(try)g(to)g (handle)i(this)f(issue)89 1737 y(and,)22 b(second,)g(b)q(ecause)f(a)g (complete)g(c)o(haracterization)g(of)f(the)h(data)f(represen)o(tation)g (used)i(b)o(y)e(a)89 1793 y(pro)q(cess)e(\(whic)o(h)h(dep)q(ends)h(on)e (the)h(underlying)h(mac)o(hine)f(arc)o(hitecture)g(and)f(on)h(the)f (compiler\))h(is)89 1850 y(lik)o(ely)e(to)d(b)q(e)i(quite)g(length)o(y) l(.)89 1923 y(W)l(e)f(agreed)f(that)h(v)o(endors)f(can)h(attac)o(h)f (implemen)o(tation)i(sp)q(eci\014c)h(prede\014ned)f(attributes)f(that)f (are)89 1980 y(asso)q(ciated)h(with)h(MPI)p 501 1980 V 16 w(COMM)p 669 1980 V 16 w(W)o(ORLD.)f(I)h(w)o(ould)f(suggest)g(the) g(follo)o(wing:)89 2053 y(Asso)q(ciate)f(a)f(prede\014ned)i(attribute)f (k)o(ey)f(MPI)p 907 2053 V 17 w(PR)o(OC)p 1055 2053 V 16 w(TYPE)h(with)g(MPI)p 1405 2053 V 16 w(COMM)p 1573 2053 V 16 w(W)o(ORLD.)g(The)89 2109 y(v)m(alue)k(of)f(this)g(attribute) g(enco)q(des)h(the)f("t)o(yp)q(e")g(of)f(the)h(executing)i(pro)q(cess,) e(i.e.)26 b(the)17 b(t)o(yp)q(e)g(of)g(data)89 2166 y(represen)o (tation)e(used)i(b)o(y)e(this)h(pro)q(cess.)22 b(The)16 b(meaning)g(of)g(the)f(v)m(alues)i(of)e(this)i(attribute)e(is)h(imple-) 89 2222 y(men)o(tation)h(dep)q(enden)o(t,)j(except)e(that)f(if)h(t)o(w) o(o)f(pro)q(cesses)h(return)g(the)g(same)f(v)m(alue,)j(then)e(they)g (use)89 2279 y(the)d(same)g(data)f(represen)o(tation)i(\(i.e.,)e(data)h (can)g(b)q(e)h(mo)o(v)o(ed)f(un)o(t)o(yp)q(ed)g(b)q(et)o(w)o(een)h (these)f(pro)q(cesses\).)89 2352 y(Implemen)o(ters)e(of)f(MPI)g(for)g (homogeneous)g(SPMD)g(systems)g(just)g(need)h(to)f(alw)o(a)o(ys)g (return)g(the)g(same)89 2408 y(v)m(alue)17 b(\(0?\))k(with)15 b(this)i(attribute.)j(Implemen)o(ters)d(of)e(MPI)h(on)f(heterogeneous)h (systems)f(ma)o(y)g(need)89 2465 y(some)g(w)o(ork.)89 2538 y(If)k(this)h(suggestion)f(sounds)h(reasonable,)g(then)f(the)h(v)m (arious)f(MPI)g(implemen)o(ters)i(ma)o(y)d(accept)i(it)89 2594 y(no)o(w)14 b(as)h("common)g(practice",)g(and)g(w)o(e)g(can,)g(at) g(a)g(latter)g(time,)g(add)g(this)h(to)e(the)i(standard.)910 2719 y(6)p eop %%Page: 7 7 7 6 bop 31 95 a Ff(8.)22 b(Meaning)15 b(of)g(the)g(reduce)h(op)q (erations)g(on)f(complex)h(datat)o(yp)q(es.)89 171 y(The)g(\014rst)h (question)g(is)g(the)f(meaning)h(of)f(max,)g(min)h(on)g(a)f(complex)h (v)m(alue.)25 b(W)l(e)17 b(in)o(tended)h(it)e(to)g(b)q(e)89 227 y(the)f(max,)f(min)i(mo)q(dulus)h(v)m(alue.)89 302 y(The)c(second)h(question)g(w)o(as)e(ho)o(w)h(this)h(v)m(alue)g(is)g (returned.)19 b(It)14 b(is)f(a)g(little)i(di\013eren)o(t)e(from)g(most) f(of)h(the)89 359 y(reduce)i(v)m(alues)g(since)g(the)g(mo)q(dulus)g(is) g(real)f(but)g(acts)g(on)g(a)g(complex)h(v)m(alue.)21 b(Here)14 b(are)g(the)g(p)q(ossible)89 415 y(w)o(a)o(ys)g(that)g(w)o(e) h(though)o(t)g(of:)143 509 y Fa(\017)23 b Ff(1\))14 b(Return)i(the)f (complex)h(v)m(alue)h(for)d(whic)o(h)i(the)g(max/min)f(mo)q(dulus)h(o)q (ccured.)143 582 y Fa(\017)23 b Ff(2\))14 b(Return)h(the)f(max/min)g (mo)q(dulus)i(in)f(a)f(real)h(or)f(double)i(v)m(ariable)g(\(as)d (appropriate)i(for)e(ho)o(w)189 638 y(the)i(complex)h(is)g(stored\).) 143 711 y Fa(\017)23 b Ff(3\))14 b(Return)i(the)f(max/min)h(mo)q(dulus) g(in)g(the)f(real)h(part)e(of)h(an)g(imaginary)h(v)m(alue.)143 785 y Fa(\017)23 b Ff(4\))16 b(Return)h(the)f(max/min)h(mo)q(dulus)g (in)g(the)g(real)g(part)e(of)h(the)h(imaginary)g(v)m(alue)g(and)g (return)189 841 y(the)e(angle)h(in)g(the)f(imaginary)h(part)e(\(i.e.,)h (mak)o(e)f(it)i(in)o(to)f(p)q(olar)h(co)q(ordinates\).)89 935 y(The)g(feeling)h(w)o(as)e(that)f(2\))h(w)o(as)g(bad)h(b)q(ecause)h (of)e(the)h(mixture)g(of)f(complex)h(and)g(real/double)h(v)m(ari-)89 991 y(ables)j(and)h(the)f(fact)f(that)g(y)o(ou)h(cannot)g(\014gure)g (out)g(the)g(original)h(v)m(alue)g(from)e(the)h(mo)q(dulus.)36 b(4\))89 1048 y(has)16 b(an)h(app)q(eal)g(from)f(the)h(p)q(oin)o(t)g (of)f(maximizing)i(information)f(returned)g(but)g(will)h(probably)f (lead)89 1104 y(to)g(user)h(errors.)28 b(1\))17 b(and)h(3\))f(ha)o(v)o (e)h(their)g(merits)g(but)g(w)o(e)g(lean)h(to)o(w)o(ard)d(3\))i(since)h (y)o(ou)e(can)h(get)g(the)89 1161 y(mo)q(dulus)e(from)f(the)h(original) g(v)m(alue)h(but)e(not)g(the)h(other)f(w)o(a)o(y)f(around.)21 b(The)15 b(negativ)o(e)h(w)o(e)f(sa)o(w)g(w)o(as)89 1217 y(that)f(y)o(ou)h(get)g(bac)o(k)g(the)g(v)m(alue)i(and)e(not)g(the)g (max/min)g(mo)q(dulus.)89 1292 y(W)l(e)g(therefore)g(prop)q(ose)g(that) g(1\))f(b)q(ecomes)i(the)f(basis)h(of)f(the)g(clari\014ed)i(text.)910 2719 y(7)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Tue Jul 12 21:52:54 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id VAA28145; Tue, 12 Jul 1994 21:52:53 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id VAA19551; Tue, 12 Jul 1994 21:52:31 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 12 Jul 1994 21:52:29 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id VAA19376; Tue, 12 Jul 1994 21:51:46 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA17064; Tue, 12 Jul 94 18:51:38 -0700 Date: Tue, 12 Jul 94 18:51:38 -0700 From: Steve Otto Message-Id: <9407130151.AA17064@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: Of course, I sent the old version! Dear MPI, I enclose my first cut at a readable errata sheet. Note that it is six sheets. Please tell me about anything I have missed. No, there won't be an errata to the errata. I will post a new one. I have organized items into two categories: clear errors that will be patched, and unresolved issues (or at least, they look unresolved to me). --Steve Otto I added a few more things. %!PS-Adobe-2.0 %%Creator: dvips 5.516 Copyright 1986, 1993 Radical Eye Software %%Title: paper.dvi %%CreationDate: Tue Jul 12 18:49:33 1994 %%Pages: 7 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o paper.ps paper %DVIPSSource: TeX output 1994.07.12:1600 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /@rigin{isls{[ 0 -1 1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{Resolution hsize -72 div mul 0 TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/ogi/staff/otto/papers/Editor/MPI/errata/paper.dvi) @start /Fa 2 108 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE 7FFE3FFC1FF80FF003C010127D9317>15 D<4020C030C030C030C030C030C030C030C030 C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030 C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030 40200C2E7BA117>107 D E /Fb 6 119 df<70F8F8F87005057C840D>58 D<70F8FCFC74040404080810102040060E7C840D>I<000001C00000078000001E000000 78000001E00000078000000E00000038000000F0000003C000000F0000003C000000F000 0000F00000003C0000000F00000003C0000000F0000000380000000E0000000780000001 E0000000780000001E0000000780000001C01A1A7C9723>I62 D<0F00601180702180E021C0E041C0E04380E08381C00701C00701C00701C00E03800E03 800E03800E03840E07080C07080C07080E0F1006131003E1E016147E931A>117 D<0F01801183C02183E021C1E041C0E04380608380400700400700400700400E00800E00 800E00800E01000E01000C02000E04000E040006180001E00013147E9316>I E /Fc 2 79 df73 D78 D E /Fd 65 123 df<030600078F00078F00078F00078F00078F00078F007FFFC0FFFFE0 FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFFC0 1E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>35 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3 800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E078 7070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F001E001C 00380038007000700070007000E000E000E000E000E000E000E000E00070007000700070 00380038001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070007000 70007000700070007000E000E000E000E001C001C0038007800F001E003C007800F00060 000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007 F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011 047D8F18>I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00 001E00001E00003E00003C00007C0000780000780000F80000F00001F00001E00003E000 03C00003C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C0000 7C0000780000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F 001C07003803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000 E0E000E0E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F0 00131C7E9B18>I<01800380038007800F803F80FF80FB80438003800380038003800380 038003800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03 F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C000 01C00003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C 00E07FFFE0FFFFE07FFFE0131C7E9B18>I<3078FCFC783000000000000000003078FCFC 78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060 071A789318>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFF E0FFFFE07FFFC0130C7E9318>61 D<600000F00000FC00007E00003F00001FC00007E000 03F00001FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC000 3F00007E0000FC0000F0000060000011187D9918>I<00700000F80000F80000D80000D8 0001DC0001DC0001DC00018C00038E00038E00038E00038E000306000707000707000707 000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C 7F9B18>65 DI<00F8E003FEE007FFE00F07 E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E000 00E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F8 00131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C 00F01C00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C 01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00F FFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E0 0000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003 FDC001F1C0141C7E9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>73 D<7F07F0FF87F87F07F01C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0 001DF0001DF0001FB8001FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03 801C01C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F 9B18>II<7E07F0FF0FF87F07F01D81C01D 81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C 39C01C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C015 1C7F9B18>I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 F00780700700780F007FFF003FFE000FF800111C7D9B18>II<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C 03C01C03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C 03801C039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>82 D<03F3801FFF803FFF807C0F80700780E00380E00380E00380E000007000007800003F00 001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E000E0E001E0F001 C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E0 7038E0703800700000700000700000700000700000700000700000700000700000700000 700000700000700000700000700000700000700000700007FF0007FF0007FF00151C7F9B 18>II87 D<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F800 01F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F00070700 0E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F00001E0000 1C00003C0000780000700000F00001E00001C00003C0000780000700000F00001E00E01C 00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC0078 1E00300E0000070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F0 0F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00 000E00000E00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00 380E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C 809B18>I<01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E0 0000E00000E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I< 001F80003F80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80 380F80700780700380E00380E00380E00380E00380E00380E00380700780700780380F80 3C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F003807807003 80700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF 8007FF0001FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001 C0007FFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F0 07FFF80FFFF81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC00 1FF80039E0003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038 E000387800F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00 000E00000E00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3 FC171C809B18>I<03800007C00007C00007C0000380000000000000000000000000007F C000FFC0007FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0 E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C 1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBF BF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3 FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E0 00E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001F00013147E93 18>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E0038 0E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E0000 0E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<7F87E0FF9FF07FBF F803F87803F03003E00003C00003C0000380000380000380000380000380000380000380 000380000380007FFE00FFFF007FFE0015147F9318>114 D<07F7003FFF007FFF00780F 00E00700E00700E007007C00007FE0001FFC0003FE00001F00600780E00380E00380F003 80F80F00FFFF00FFFC00E7F00011147D9318>I<0180000380000380000380000380007F FFC0FFFFC0FFFFC003800003800003800003800003800003800003800003800003800003 80400380E00380E00380E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE0 7E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03 C00E03800E03800E0380070700070700070700038E00038E00038E00038E0001DC0001DC 0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC00 01F80000F80000700000F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF8 7F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03800703800707000707 00038700038600038E0001CE0001CE0000CC0000CC0000DC000078000078000078000070 0000700000700000F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<3F FFF07FFFF07FFFF07001E07003C0700780000F00001E00003C0000F80001F00003C00007 80000F00701E00703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I E /Fe 12 119 df69 D73 D85 D<00003FF80000003FF80000003FF800000003F800000003F800000003F800000003 F800000003F800000003F800000003F800000003F800000003F800000003F800000003F8 00000003F800001FE3F80000FFFBF80003F03FF80007E00FF8000FC007F8001F8003F800 3F8003F8007F0003F8007F0003F8007F0003F800FF0003F800FF0003F800FF0003F800FF 0003F800FF0003F800FF0003F800FF0003F8007F0003F8007F0003F8007F0003F8003F80 03F8001F8003F8000F8007F80007C00FF80003F03BFF8000FFF3FF80003FC3FF80212A7E A926>100 D<003FE00001FFF80003F07E0007C01F000F801F801F800F803F800FC07F00 0FC07F0007C07F0007E0FF0007E0FF0007E0FFFFFFE0FFFFFFE0FF000000FF000000FF00 00007F0000007F0000007F0000003F8000E01F8000E00FC001C007E0038003F81F0000FF FE00001FF0001B1B7E9A20>I108 D110 D<003FE00001FFFC0003F07E000FC01F801F800FC03F800FE03F0007E07F0007F07F0007 F07F0007F0FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007 F87F0007F07F0007F03F800FE03F800FE01F800FC00FC01F8007F07F0001FFFC00003FE0 001D1B7E9A22>I114 D<03FE300FFFF01E03F038 00F0700070F00070F00070F80070FC0000FFE0007FFE007FFF803FFFE01FFFF007FFF800 FFF80003FC0000FC60007CE0003CF0003CF00038F80038FC0070FF01E0F7FFC0C1FF0016 1B7E9A1B>I117 DI E /Ff 18 118 df<00000200000006000000060000000E0000001E0000001E00 00003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F0000020F00 00040F00000C0F0000080F0000100F0000100F0000200F80003FFF800040078000C00780 0080078001000780010007800200078002000780060007801E000F80FF807FF81D207E9F 22>65 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078 0000780000780000F00000F00000F00000F00001E00001E00001E00001E00003C00003C0 0003C00003C0000780000780000780000780000F8000FFF800141F7D9E12>73 D<001FFF0000F80000F00000F00000F00001E00001E00001E00001E00003C00003C00003 C00003C0000780000780000780000780000F00000F00000F00000F00001E00001E00301E 00781E00F83C00F83C00F0780080700040E00021C0001F000018207D9E18>I<0007E040 001C18C0003005800060038000C0038001C0018001800100038001000380010003800100 0380000003C0000003C0000003F8000001FF800001FFE000007FF000001FF0000001F800 000078000000780000003800000038002000380020003800200030006000700060006000 6000E0007000C000E8038000C606000081F800001A217D9F1A>83 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F000 00F00000F00000F0000070010070020030040018380007C00011147C9315>99 D<007C01C207010E011C013C013802780C7BF07C00F000F000F000F00070007001700238 04183807C010147C9315>101 D<00007800019C00033C00033C00071800070000070000 0E00000E00000E00000E00000E0001FFE0001C00001C00001C00001C0000380000380000 380000380000380000700000700000700000700000700000700000E00000E00000E00000 E00000C00001C00001C0000180003180007B0000F300006600003C00001629829F0E>I< 00C001E001E001C000000000000000000000000000000E00330023004380430047008700 0E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E>105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030 000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I E /Fg 75 123 df<001F83E000F06E3001C078780380F8780300F030 07007000070070000700700007007000070070000700700007007000FFFFFF8007007000 070070000700700007007000070070000700700007007000070070000700700007007000 07007000070070000700700007007000070070000700700007007000070070007FE3FF00 1D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007000007000007000007 0000070000070000FFFFE00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F 19>I<003FE000E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E0 0700E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0 0700E00700E00700E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<7038 F87CFC7EFC7E743A0402040204020804080410081008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C 000C00180018003000300030007000600060006000E000E000E000E000E000E000E000E0 00E000E000E000E0006000600060007000300030003000180018000C000C000600020001 000080004000200B2E7DA112>I<800040002000100008000C0006000600030003000180 0180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E0 00C000C000C001C001800180018003000300060006000C00080010002000400080000B2E 7DA112>I<01800180018001800180C183F18F399C0FF003C003C00FF0399CF18FC18301 80018001800180018010147DA117>I<70F8FCFC74040404080810102040060E7C840D> 44 DI<70F8F8F87005057C840D>I<0001000300030006000600 06000C000C000C00180018001800300030003000600060006000C000C000C00180018001 800300030003000600060006000C000C000C001800180018003000300030006000600060 00C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070038070 0380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003F00012 1F7E9D17>I<018003800F80F38003800380038003800380038003800380038003800380 038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C9D17> I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007C00007 C0000780000780000F00000E00001C0000380000700000600000C0000180000300000600 400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E0020 0F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003F00000 3C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040078040 0F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00002E00 002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00200E00 200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0 141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000001000 0010000011F000161C00180E001007001007800003800003800003C00003C00003C07003 C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17>I<00 7C000182000701000E03800C07801C0780380300380000780000700000700000F1F000F2 1C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003C070 03803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF80 7FFF80400100800200800200800400000800000800001000002000002000004000004000 00C00000C00001C000018000038000038000038000038000078000078000078000078000 078000078000078000030000121F7D9D17>I<03F0000C0C001006003003002001806001 806001806001807001807803003E03003F06001FC8000FF00003F80007FC000C7E00103F 00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C 0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700700380F00380F0 0380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C000 0380000380000380000700300700780600780E00700C002018001070000FC000121F7E9D 17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F8700000 000000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0FFFFFF F00000000000000000000000000000000000000000000000000000000000000000FFFFFF F07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000 C001800180010003000200020002000200020002000000000000000000000007000F800F 800F8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C0 000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800002078 000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F 0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F 9F22>65 DI<000FC040007030C0 01C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C000040 78000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007000100 0380020001C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0 070001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000 F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E0 7C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E00460 00781820000FE0001E217D9F24>III75 DIII<001F800000F0F00001C0380007801E00 0F000F000E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0 F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E0 7C0003E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C03800 00F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F00000F000 007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C08001 C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7F FFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F001080 0F0010000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003 C0020001E0040001F00C0000F0080000781000007C1000003C2000003E4000001E400000 0F8000000F8000000780000003C0000007E0000005E0000009F0000018F8000010780000 207C0000603C0000401E0000801F0001800F0001000780020007C0070003C01F8007E0FF E01FFE1F1F7F9E22>II<1FE000 303000781800781C00300E00000E00000E00000E0000FE00078E001E0E00380E00780E00 F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>I E /Fh 15 122 df<70F8FCFC7404040404080810102040060F7C840E>44 D<008003800F80F380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038007C0FFFE0F217CA018 >49 D<03F0000C1C001007002007804003C04003C08003E0F003E0F801E0F801E0F801E0 2003E00003E00003C00003C0000780000700000E00001C0000180000300000600000C000 0180000100000200200400200800201800603000403FFFC07FFFC0FFFFC013217EA018> I<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>52 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0F001 E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C00001 C00001C0000380000380300300780700780600700C002018001030000FC00013227EA018 >57 D<03FFF0001F00000F00000F00000F00000F00000F00000F00000F00000F00000F00 000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00 000F00000F00000F00700F00F80F00F80F00F80E00F01E00401C0020380018700007C000 14237EA119>74 D<000FE00000783C0000E00E0003C00780078003C00F0001E00E0000E0 1E0000F03C0000783C0000787C00007C7C00007C7800003C7800003CF800003EF800003E F800003EF800003EF800003EF800003EF800003EF800003EF800003E7800003C7C00007C 7C00007C3C0000783E0000F81E0000F00F0001E00F0001E0078003C003C0078000E00E00 00783C00000FE0001F247DA226>79 D<03F0200C0C601802603001E07000E0600060E000 60E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF 80003FC00007E00001E00000F00000F0000070800070800070800070800070C00060C000 60E000C0F000C0C80180C6070081FC0014247DA21B>83 D<01FC000707000C03801C01C0 3801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000700000780020 3800201C00400E008007030000FC0013157F9416>101 D<0E00FE001E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>108 D<01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F000 78F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>111 D<02000200020002000600060006000E001E003E00FFF80E000E000E000E000E000E000E 000E000E000E000E000E040E040E040E040E040E040708030801F00E1F7F9E13>116 D<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00F00E00F006017003827800FC7F18157F941B>II121 D E /Fi 17 117 df<0001FE00000007018000001C004000007800200000F000F00000E001 F00001E001F00003C001F00003C000E00003C000000003C000000003C000000003C00000 0003C000000003C000000003C000000003C000000003C000000003C0000000FFFFFFF000 FFFFFFF00003C001F00003C000F00003C000F00003C000F00003C000F00003C000F00003 C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C0 00F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000 F00003C000F00003C000F00003C000F00003C000F00003C000F00007E001F8007FFE1FFF 807FFE1FFF8021327FB124>12 D69 D73 D77 D80 D<007F802001FFE02007C078600F001C601E00 06E03C0003E0380001E0780000E0700000E070000060F0000060F0000060F0000020F000 0020F0000020F8000020F80000007C0000007E0000003F0000003FC000001FF800000FFF 800007FFF80003FFFC0000FFFF00000FFF800000FFC000001FE0000007E0000003F00000 01F0000000F0000000F8000000F88000007880000078800000788000007880000078C000 0078C0000070E00000F0E00000E0F00000E0F80001C0EC000380C7000700C1F01E00807F FC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C00 3C003E001C003E001E003E001E0008001E0000001E0000001E0000001E00000FFE0000FC 1E0003E01E000F801E001F001E003E001E003C001E007C001E00F8001E04F8001E04F800 1E04F8003E04F8003E0478003E047C005E043E008F080F0307F003FC03E01E1F7D9E21> 97 D<003F8000E0600380180700040F00041E001E1C003E3C003E7C003E7C0008780000 F80000F80000F80000F80000F80000F80000F80000F80000F800007800007C00007C0000 3C00011E00011E00020F000207000403801800E060003F80181F7D9E1D>99 D<003F800000E0E0000380380007003C000E001E001E001E001C000F003C000F007C000F 0078000F8078000780F8000780F8000780FFFFFF80F8000000F8000000F8000000F80000 00F8000000F8000000780000007C0000003C0000003C0000801E0000800E0001000F0002 000780020001C00C0000F03000001FC000191F7E9E1D>101 D<0007E0001C1000383800 707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007FFF0016327FB1 14>I<0780000000FF80000000FF800000000F8000000007800000000780000000078000 000007800000000780000000078000000007800000000780000000078000000007800000 00078000000007800000000780000000078000000007800000000780FE00000783078000 078C03C000079001E00007A001E00007A000F00007C000F00007C000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF 8021327EB125>104 D<07000F801F801F800F8007000000000000000000000000000000 00000000000000000780FF80FF800F800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800FC0FFF8FFF80D30 7EAF12>I<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007 C000F00007C000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 000FC001F800FFFC1FFF80FFFC1FFF80211F7E9E25>110 D<001FC00000F0780001C01C 00070007000F0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000 F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000 F07C0001F03C0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F078 00001FC0001D1F7E9E21>I<0781FC00FF860700FF8803C00F9001E007A000F007C00078 078000780780003C0780003C0780003E0780001E0780001F0780001F0780001F0780001F 0780001F0780001F0780001F0780001F0780001F0780003E0780003E0780003C0780007C 0780007807C000F007A000F007A001E00798038007860F000781F8000780000007800000 078000000780000007800000078000000780000007800000078000000780000007800000 0FC00000FFFC0000FFFC0000202D7E9E25>I<0783E0FF8C18FF907C0F907C07A07C07C0 3807C00007C00007C0000780000780000780000780000780000780000780000780000780 000780000780000780000780000780000780000780000780000780000780000FC000FFFE 00FFFE00161F7E9E19>114 D<00400000400000400000400000400000C00000C00000C0 0001C00001C00003C00007C0000FC0001FFFE0FFFFE003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 1003C01003C01003C01003C01003C01003C01003C01001C02001E02000E0400078C0001F 00142C7FAB19>116 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 421 278 a Fi(Errata)23 b(Sheet)f(for)f(MPI)i(Sp)r (eci\014cation)806 404 y Fh(Stev)o(e)15 b(Otto)780 506 y(July)g(12,)i(1994)46 664 y Fg(This)12 b(do)q(cumen)o(t)g(lists)g (errors)f(in)i(the)e(Ma)o(y)g(5th)g(v)o(ersion)h(of)g(the)f(MPI)h(sp)q (eci\014cation.)21 b(Some)11 b(of)g(these)h(will)-25 721 y(b)q(e)17 b(\014xed)g(\(at)e(the)h(galley)h(stage\))f(in)h(the)f (v)o(ersion)g(to)g(b)q(e)h(published)i(b)o(y)d(MIT)g(Press)g(in)h(the)f Ff(International)-25 777 y(Journal)g(of)h(Sup)n(er)n(c)n(omputer)g (Applic)n(ations)p Fg(.)46 833 y(I)e(will)i(add)e(sp)q(eci\014c)j(page) d(and)g(line)i(n)o(um)o(b)q(ers)e(when)h(I)g(get)e(the)i(c)o(hance.)-25 974 y Fe(Errors)-25 1076 y Fg(The)g(follo)o(wing)f(are)g(kno)o(wn)g (errors)g(or)f(c)o(hanges)i(that)e(w)o(e)h(ha)o(v)o(e)g(made)g(to)g (the)g(sp)q(eci\014cation.)31 1166 y(1.)22 b Fd(MPI)p 164 1166 15 2 v 17 w(Status)15 b Fg(passed)i(as)f(p)q(oin)o(ter)g(ev)o (erywhere)h(\(an)f(issue)h(for)f(the)g(C)h(binding)h(only\).)23 b(The)17 b(curren)o(t)89 1222 y(sp)q(eci\014cation)g(has:)89 1321 y Fd(int)23 b(MPI_Get_count\()f(MPI_Status)h(status,)g (MPI_Datatype)f(datatype,)g(int)i(*count)f(\))89 1377 y(int)g(MPI_Get_elements\()f(MPI_Status)g(status,)h(MPI_Datatype)f (datatype,)h(int)g(*count)g(\))89 1434 y(int)g(MPI_Test_cancelled\()e (MPI_Status)i(status,)g(int)g(*flag)g(\))89 1532 y Fg(A)16 b(decision)j(w)o(as)d(reac)o(hed)g(that)g(the)h(\014rst)f(argumen)o(t)g (should)i(b)q(e)f(passed)g(in)g(as)g(a)f(p)q(oin)o(ter,)h(that)f(is,)89 1589 y(the)f(function)h(protot)o(yp)q(es)e(should)j(b)q(e:)89 1687 y Fd(int)23 b(MPI_Get_count\()f(MPI_Status)h(*status,)f (MPI_Datatype)h(datatype,)f(int)i(*count)f(\))89 1744 y(int)g(MPI_Get_elements\()f(MPI_Status)g(*status,)h(MPI_Datatype)f (datatype,)h(int)g(*count)g(\))89 1800 y(int)g(MPI_Test_cancelled\()e (MPI_Status)i(*status,)g(int)g(*flag)g(\))89 1899 y Fg(This)16 b(w)o(as)f(done)h(b)q(ecause)h Fd(MPI)p 634 1899 V 17 w(Status)e Fg(is)h(a)g Fd(struct)f Fg(and)h(passing)g(suc)o(h)g(ob)s (jects)f(as)g(argumen)o(ts)g(is)89 1955 y(a)j(bit)i(of)e(a)h(problem.) 32 b(Some)19 b(compilers)h(still)h(do)e(strange)f(things)h(with)h Fd(struct)p Fg(s)e(as)g(argumen)o(ts.)89 2012 y(Note)c(that)f(no)o(w,)h (there)h(is)f(a)h(sligh)o(t)f(inconsistency)j(with)d(the)h (language-indep)q(enden)o(t)i(sp)q(eci\014cation)89 2068 y(of)d(the)i(functions.)k(These)c(are)f Fc(IN)g Fg(argumen)o(ts,)f(y)o (et)h(w)o(e)g(are)g(passing)h(them)f(b)o(y)g(reference)h(in)g(C.)31 2157 y(2.)22 b(The)14 b(di\013erence)i(b)q(et)o(w)o(een)e(t)o(w)o(o)f (p)q(oin)o(ters)i(to)e(the)h(same)g(storage)f(area)h(should)h(b)q(e)g (of)f(t)o(yp)q(e)g Fd(MPI)p 1746 2157 V 17 w(Aint)p Fg(,)89 2214 y(rather)i(than)g Fd(int)p Fg(.)23 b(This)17 b(is)g(the)f (assumption)h(w)o(e)f(go)g(b)o(y)g(in)i(the)e(de\014nition)j(of)d Fd(MPI)p 1575 2214 V 16 w(Type)p 1687 2214 V 17 w(hvector)89 2270 y Fg(and)h Fd(MPI)p 254 2270 V 17 w(Type)p 367 2270 V 16 w(hindexed)p Fg(.)24 b(If)17 b(w)o(e)g(w)o(an)o(t)f(to)g(b)q(e)i (consisten)o(t,)f(then)g(w)o(e)g(should)h(use)f(the)g(same)g(t)o(yp)q (e)89 2327 y Fd(MPI)p 164 2327 V 17 w(Aint)d Fg(in)i(the)f(functions)h (that)f(return)g(suc)o(h)h(di\013erences;)f(th)o(us,)89 2425 y Fd(MPI_Type_extent\(...,)21 b(MPI_Aint)i(*extent\))89 2482 y(MPI_Type_size\(...,)e(MPI_Aint)i(*size\))89 2538 y(MPI_Type_lb\(...,)e(MPI_Aint)i(*displacement\))89 2594 y(MPI_Type_ub\(...,)e(MPI_Aint)i(*displacement\))910 2719 y Fg(1)p eop %%Page: 2 2 2 1 bop 31 95 a Fg(3.)22 b Fd(MPI_PACK\(...,)g(OUTCOUNT,...\))89 171 y Fg(should)16 b(b)q(e)89 283 y Fd(MPI_PACK\(...,)22 b(OUTSIZE,...\))89 396 y Fg(\(analogous)16 b(to)g Fd(MPI)p 450 396 15 2 v 17 w(UNPACK\(...,)23 b(INSIZE,...\))p Fg(\).)g Fd(OUTCOUNT)16 b Fg(is)i(not)e(a)h(coun)o(t)g(of)f(elemen)o (ts)i(in-)89 452 y(serted)d(in)h(the)f(bu\013er,)g(but)g(the)h(total)e (bu\013er)h(size.)31 546 y(4.)22 b(In)16 b(section)f(3.2.2,)f(the)h (sen)o(tence)h(fragmen)o(t:)189 640 y(MPI)p 281 640 14 2 v 16 w(DOUBLE)p 492 640 V 17 w(COMPLEX)e(for)g(double)h(precision)h (complex)f(in)f(F)l(ortran)f(declared)j(to)189 696 y(b)q(e)g(of)e(t)o (yp)q(e)i(DOUBLE)f(PRECISION;)89 790 y(should)h(b)q(e)189 884 y(MPI)p 281 884 V 16 w(DOUBLE)p 492 884 V 17 w(COMPLEX)e(for)g (double)h(precision)h(complex)f(in)f(F)l(ortran)f(declared)j(to)189 940 y(b)q(e)g(of)e(t)o(yp)q(e)i(DOUBLE)f(COMPLEX;)31 1034 y(5.)22 b(The)11 b(w)o(ording)h(of)f(the)g(description)i(of)e Fd(MPI)p 822 1034 15 2 v 17 w(GROUP)p 959 1034 V 17 w(RANGE)p 1096 1034 V 16 w(EXCL)g Fg(is)h(confusing)g(but)g(I)f(ha)o(v)o(en't)g (decided)89 1090 y(what)j(to)h(c)o(hange)g(it)h(to,)e(y)o(et.)19 b(The)d(function)g(protot)o(yp)q(es)e(are)h(correct,)f(I)i(b)q(eliev)o (e.)31 1184 y(6.)22 b(The)15 b(example)h(on)f(page)g(148.)k(The)d (example)g(w)o(ould)f(b)q(e)h(more)f(illustrious)i(if)89 1297 y Fd(MPI_Reduce\(...,MPI_SUM,)j(1,)k(commslave\))f(->)g (MPI_Reduce\(...,MPI_SUM,)e(0,)i(commslave\))89 1409 y Fg(The)12 b(comm)o(unicator)g Fd(commslave)f Fg(is)i(comp)q(osed)f (of)g(the)g(group)g Fd(grprem)g Fg(whic)o(h)h(is)f Fd(MPI)p 1598 1409 V 17 w(GROUP)p 1735 1409 V 17 w(WORLD)89 1466 y Fg(without)f(pro)q(cess)g(0.)19 b(As)11 b(suc)o(h,)h Fd(grprem)e Fg(is)i(n)o(um)o(b)q(ered)f(from)g(0)g(to)f(\(Numpro)q(cs)h (-)h(2\))e(where)i(Numpro)q(cs)89 1522 y(is)i(the)g(size)h(of)e Fd(MPI)p 420 1522 V 17 w(GROUP)p 557 1522 V 17 w(WORLD)p Fg(.)g(Making)g(the)h(reduce)h(go)f(to)f(pro)q(cess)h(0)g(mak)o(es)f (the)h(ren)o(um)o(b)q(ering)89 1579 y(clear.)25 b(It)17 b(also)g(mak)o(es)f(it)h(w)o(ork)f(for)g(only)h(2)g(no)q(des)g(-)g(the) g(curren)o(t)g(co)q(de)g(fails)h(since)g Fd(grprem)e Fg(w)o(ould)89 1635 y(only)f(ha)o(v)o(e)g(one)h(no)q(de/pro)q(cess.)31 1729 y(7.)22 b(P)o(age)14 b(150:)89 1841 y Fd(init_user_lib\(MPI_Comm) 21 b(*comm,)i(user_lib_t)f(**handle\))h(->)89 1898 y (init_user_lib\(MPI_Comm)e(comm,)i(user_lib_t)f(**handle\))89 2010 y Fg(The)15 b(calls)h(on)g(page)f(149)f(don't)h(pass)g Fd(comm)f Fg(as)h(a)g(p)q(oin)o(ter.)31 2104 y(8.)22 b(P)o(age)12 b(151.)18 b(This)13 b(example)g(seems)g(to)f(imply)h(that) f(MPI)p 1065 2104 14 2 v 17 w(Comm)p 1214 2104 V 15 w(rank)h(returns)f (MPI)p 1575 2104 V 16 w(UNDEFINED)89 2161 y(in)18 b(ma)f(and)g(m)o(b)h (if)g(the)f(calling)i(pro)q(cess)f(is)g(not)f(in)h(comm)p 1131 2161 V 16 w(a)f(or)g(comm)p 1364 2161 V 16 w(b,)h(resp)q(ectiv)o (ely)l(.)28 b(142)17 b(sa)o(ys)89 2217 y(nothing)f(ab)q(out)f(this.)22 b(If)15 b(this)h(w)o(as)f(the)h(in)o(ten)o(t)f(\(that)g(is)h(w)o(ould)g (return)f(MPI)p 1453 2217 V 17 w(UNDEFINED\))g(then)89 2274 y(the)j(text)f(ab)q(out)g(MPI)p 491 2274 V 17 w(COMM)p 660 2274 V 16 w(RANK)h(should)h(sa)o(y)e(this.)28 b(This)18 b(commen)o(t)g(migh)o(t)f(also)h(apply)h(to)89 2330 y(MPI)p 181 2330 V 16 w(GR)o(OUP)p 365 2330 V 17 w(RANK)13 b(on)f(135)f(if)i(w) o(e)f(w)o(an)o(t)f(it)h(to)g(act)g(the)g(same.)18 b(Their)13 b(b)q(eing)h(the)e(same)g(is)h(implied)89 2386 y(in)j(the)f(commen)o (ts)g(in)h(the)f(text.)31 2480 y(9.)22 b(P)o(age)14 b(159.)19 b(It)c(w)o(ould)h(b)q(e)g(a)f(little)h(cleaner)h(to)d(add:)89 2593 y Fd(MPI_Comm_free\(&myComm\);)910 2719 y Fg(2)p eop %%Page: 3 3 3 2 bop 89 95 a Fg(This)16 b(is)f(done)h(for)e(Example)i(2)f(on)g(page) g(160.)8 189 y(10.)22 b(P)o(age)16 b(162)f(Instead)j(of)e(\(MPI)p 611 189 14 2 v 16 w(Comm\))f(0)i(I)g(think)g(it)g(should)h(b)q(e)f(MPI) p 1331 189 V 16 w(COMM)p 1499 189 V 16 w(NULL.)h(145:8)d(sa)o(ys)89 246 y(MPI)p 181 246 V 16 w(Comm)p 329 246 V 16 w(split)f(returns)f(MPI) p 688 246 V 16 w(COMM)p 856 246 V 16 w(NULL)h(when)f(it)h(has)e(funn)o (y)i(cases.)19 b(I)13 b(don't)g(recall)h(read-)89 302 y(ing)h(0)g(and)h(NULL)g(w)o(ere)f(the)g(same)g(for)g(this)g(case.)8 396 y(11.)22 b(P)o(age)14 b(162.)89 509 y Fd(MPI_Attr_keyval_create)21 b(->)i(MPI_Keyval_create)89 621 y Fg(as)15 b(p)q(er)g(page)g(166.)k (Same)c(commen)o(t)g(applies)i(to)e(page)g(170.)8 715 y(12.)22 b(P)o(age)14 b(165.)89 827 y Fd(MPI_Intercomm_create\(loc)o (al_lead)o(er,)21 b(local_comm,)h(buffer[0],new_world,)f(..\))j(->)89 884 y(MPI_Intercomm_create\(loc)o(al_comm)o(,)d(local_leader,)h (new_world,)h(buffer[0],)f(..\))89 996 y Fg(as)15 b(de\014ned)h(on)f (157.)8 1090 y(13.)22 b(P)o(age)10 b(167)g(It)i(seems)f(to)f(me)i(that) e(the)h(F)l(ortran)f(de\014nition)j(needs)f(IERR)o(OR.)h(167)d(sa)o(ys) g(that)h(it)g(returns)89 1147 y(MPI)p 181 1147 V 16 w(SUCCESS)17 b(if)g(it)g(w)o(orks.)23 b(Only)17 b(the)g(C)f(v)o(ersion)h(seems)f(to) g(ha)o(v)o(e)g(a)h(w)o(a)o(y)e(to)h(return)g(this.)25 b(The)89 1203 y(same)15 b(commen)o(t)f(applies)j(to:)i(167)c(for)f (DELETE)p 968 1203 V 16 w(FUNCTION)8 1297 y(14.)22 b(P)o(age)13 b(167)g(Do)h(w)o(e)g(w)o(an)o(t)f(MPI)p 620 1297 V 16 w(NULL)p 760 1297 V 18 w(FN,)g(MPI)p 957 1297 V 16 w(DUP)p 1073 1297 V 17 w(FN)g(in)i(App)q(endix)i(A?)d(It)g(is)h(a)f(function)h (but)89 1353 y(some)g(are)g(listed)8 1447 y(15.)22 b(P)o(age)14 b(167)h(MPI)p 375 1447 V 16 w(KEYV)-5 b(AL)p 582 1447 V 17 w(INV)g(ALID)17 b(seems)e(to)f(b)q(e)i(missing)g(from)f(App)q (endix)i(A.)8 1541 y(16.)22 b(P)o(age)14 b(169)h(A)g(n)o(ull)h (Rational)g(should)h(b)q(e)e(remo)o(v)o(ed.)8 1635 y(17.)22 b(P)o(age)14 b(170)h(MPI)p 375 1635 V 16 w(Ab)q(ort)g(tak)o(es)f(a)h (comm)g(and)g(error)g(co)q(de,)g(not)g(string)g(as)g(de\014ned)i(P)o (age)d(197.)8 1729 y(18.)22 b(P)o(age)14 b(171)89 1841 y Fd(MPI_Comm)23 b(comm)g(->)g(MPI_Comm)g(*comm.)89 1954 y Fg(See)16 b(de\014nition)h(on)e(167.)k(And,)c(F)l(or)g(the)g(same)g (reason,)89 2066 y Fd(int)23 b(keyval)g(->)h(int)f(*keyval)89 2178 y Fg(And)89 2291 y Fd(gop_stuff_copier\(...,)e(gop_stuff,)h (extra\))h(->)89 2347 y(gop_stuff_copier\(...,)e(extra,)i (gop_stuff_in,)f(gop_stuff_out,)g(flag\))89 2460 y Fg(p)q(er)15 b(de\014nition)i(on)f(page)f(166.)k(And)89 2572 y Fd(MPI_Comm)k(comm)g (->)g(MPI_Comm)g(*comm)910 2719 y Fg(3)p eop %%Page: 4 4 4 3 bop 89 95 a Fg(And)89 207 y Fd(int)23 b(keyval)g(->)h(int)f (*keyval.)89 318 y Fg(Also)15 b(need)h(to)f(add)g Fd(*flag)g Fg(for)f(ab)q(o)o(v)o(e)h(c)o(hange)89 429 y Fd(gop_stuff_type)22 b(*gop_stuff)g(->)i(gop_stuff_type)e(*gop_stuff_in,)g(**gop_stuff_out) 89 540 y Fg(And,)15 b(I)h(think)g(w)o(e)f(need)h(to)e(add)i(the)f (line:)89 652 y Fd(*gop_stuff_out)22 b(=)h(gop_stuff_in;)89 763 y Fg(Righ)o(t)18 b(no)o(w,)f(the)g(routine)h(do)q(es)g(not)f(app)q (ear)h(to)f(cop)o(y)g(an)o(ything;)h(only)g(incremen)o(ts)h(the)e (reference)89 819 y(coun)o(t.)i(And,)d(need)g(to)e(mak)o(e)89 931 y Fd(gop_stuff)22 b(->)i(gop_stuff_in)89 1042 y Fg(if)15 b(y)o(ou)g(mak)o(e)g(the)g(other)g(c)o(hanges.)8 1135 y(19.)22 b(P)o(age)14 b(215,)g(line)j(31:)89 1247 y Fd(int)23 b(MPI_Comm_compare\()f(MPI_Comm)g(comm1,)i(comm2,)f(int)g(*result)g(\)) 89 1358 y Fg(should)16 b(b)q(e)89 1469 y Fd(int)23 b (MPI_Comm_compare\()f(MPI_Comm)g(comm1,)i(MPI_Comm)e(comm2,)h(int)h (*result)f(\))8 1580 y Fg(20.)f(F)l(or)16 b(complex)h(n)o(um)o(b)q (ers,)g(w)o(e)f(need)i(to)d(c)o(hange)i(the)g(form)o(ula)f(de\014ning)i (MINLOC)f(and)g(MAXLOC:)89 1637 y(rather)d(than)786 1693 y Fb(u)f(<)g(v)r(;)8 b(u)k Fg(=)h Fb(v)r(;)8 b(u)j(>)i(v)r(;)89 1776 y Fg(these)i(should)h(b)q(e)650 1833 y Fa(k)p Fb(u)p Fa(k)c Fb(<)h Fa(k)p Fb(v)r Fa(k)p Fb(;)8 b Fa(k)p Fb(u)p Fa(k)j Fg(=)i Fa(k)p Fb(v)r Fa(k)p Fb(;)8 b Fa(k)p Fb(u)p Fa(k)i Fb(>)j Fa(k)p Fb(v)r Fa(k)p Fb(:)-25 1976 y Fe(Unresolv)n(ed)23 b(Issues)-25 2077 y Fg(The)18 b(follo)o(wing)g(are)f(some)h(issues)g (that)f(are)g(still)i(b)q(eing)g(discussed)g(and)f(that)f(ha)o(v)o (en't)g(y)o(et)g(reac)o(hed)h(con-)-25 2133 y(sensus.)31 2238 y(1.)k(An)15 b(issue)i(related)f(to)e(error)h(n)o(um)o(b)q(er)h (2:)k(Should)d(the)e(argumen)o(ts)g(POSITION)i(and)f(OUTCOUNT)89 2295 y(in)h Fd(MPI)p 218 2295 15 2 v 17 w(Pack)f Fg(b)q(e)h(of)f(t)o (yp)q(e)h Fd(int)f Fg(\(as)f(they)i(are)f(no)o(w\))g(or)g(of)g(t)o(yp)q (e)h Fd(MPI)p 1310 2295 V 16 w(Aint)p Fg(?)24 b(That)16 b(is,)h(do)g(w)o(e)f(w)o(an)o(t)89 2351 y(to)e(allo)o(w)i(for)e(a)h (bu\013er)g(with)h(more)f(than)g(2)g(Gb)o(ytes?)31 2445 y(2.)22 b(In)16 b(lines)g(26)f(and)g(27,)g(page)g(88)f(of)h(the)g(Ma)o (y)g(5)f(MPI)i(draft,)e(shouldn't)h Fd(&lbuf)g Fg(just)g(b)q(e)h Fd(lbuf)e Fg(?)31 2538 y(3.)22 b(There)15 b(are)g(some)f (inconsistencies)k(in)e(the)f(description)h(of)f(the)g(use)g(of)f(MPI)p 1427 2538 14 2 v 17 w(NULL)p 1568 2538 V 17 w(FN)h(when)g(used)89 2594 y(with)e(MPI)p 282 2594 V 16 w(KEYV)-5 b(AL)p 489 2594 V 17 w(CREA)l(TE.)13 b(In)g(particular,)h(when)f(describing)h(the) f("cop)o(y)p 1516 2594 V 16 w(function",)g(w)o(e)g(sa)o(y)910 2719 y(4)p eop %%Page: 5 5 5 4 bop 189 95 a Fg(MPI)p 281 95 14 2 v 16 w(NULL)p 421 95 V 17 w(FN)15 b(is)h(a)f(function)h(that)e(do)q(es)i(nothing)g(other) e(than)i(returning)f(\015ag)g(=)h(0.)89 189 y(This)f(is)g(inconsisten)o (t)h(with)f(a)g(n)o(ull)h(function)f(\(b)o(y)g(de\014nition,)h(a)f(n)o (ull)h(function)f(should)h(do)f(nothing\))89 246 y(and)g(will)i(fail)f (when)g(used)f(as)g(the)h("delete)p 838 246 V 17 w(function",)f(where)h (w)o(e)f(sa)o(y)189 340 y(It)g(ma)o(y)f(b)q(e)i(sp)q(eci\014ed)h(as)e (the)g(n)o(ull)h(function)g(p)q(oin)o(ter)f(in)h(C)f(or)g(as)f(MPI)p 1442 340 V 17 w(NULL)p 1583 340 V 17 w(FN)h(from)189 396 y(either)h(C)f(or)f(F)o(OR)l(TRAN,)i(in)g(whic)o(h)g(case)f(no)g (delete)i(callbac)o(k)f(o)q(ccurs)f(for)g(k)o(eyv)m(al.)89 490 y(There)g(are)g(t)o(w)o(o)f(ob)o(vious)h(\014xes:)20 b(1\))15 b(Replace)i(MPI)p 984 490 V 16 w(NULL)p 1124 490 V 17 w(FN)e(with)h(MPI)p 1413 490 V 16 w(NULL)p 1553 490 V 17 w(COPY)p 1703 490 V 17 w(FN)f(\(for)89 546 y(correctness\))9 b(and)i(MPI)p 510 546 V 16 w(NULL)p 650 546 V 18 w(DELETE)p 857 546 V 16 w(FN)e(\(for)h(consistency\))g(in)h(the)g(discussion)h(of) d(MPI)p 1745 546 V 17 w(KEYV)-5 b(AL)p 1953 546 V 17 w(CREA)l(TE.)89 603 y(2\))14 b(Change)h(the)h(w)o(ording)f(under)h ("cop)o(y)p 806 603 V 16 w(function")f(to)189 697 y(MPI)p 281 697 V 16 w(NULL)p 421 697 V 17 w(FN)e(is)h(a)e(function)i(that)e (do)q(es)h(nothing;)h(when)g(used)f(as)g(the)g(cop)o(y)f(function,)189 753 y(it)k(has)f(the)h(same)f(e\013ect)g(as)h(a)f(function)h(that)f(do) q(es)h(nothing)g(other)g(than)f(returning)h(\015ag)189 809 y(=)f(0.)89 903 y(I)i(prefer)g(\(1\))e(b)q(ecause)j(it)f(is)g (clearer)g(as)f(to)g(what)h(is)g(going)f(on)h(and)g(\(naturally\))f (mak)o(es)h(the)f(imple-)89 960 y(men)o(tation)f(simpler.)21 b(Implemen)o(ting)c(\(2\))d(is)i(straigh)o(tforw)o(ard)d(but)i (unpleasan)o(t.)31 1054 y(4.)22 b(Related)16 b(to)f(previous)h(item,)f (w)o(e)g(ha)o(v)o(e)f(\(F)l(rom)h(Hub)q(ertus)g(F)l(rank)o(e\):)89 1129 y(I)g(extract)g(this)g(out)g(of)g(the)g(MPI-F)g(include)j (\014les.)89 1241 y Fd(#define)23 b(MPI_DUP_FN)70 b(mpi_dup_fn)89 1298 y(/*)23 b(note)h(the)f(draft)g(provides)g(MPI_NULL_FN,)f(however)h (this)g(violates)113 1354 y(*)g(the)h(prototype)e(differences)h(in)g (usage)h(for)f(copy)g(and)h(delete)f(function)113 1411 y(*)g(In)h(agreement)f(with)g(the)g(public)g(domain)g(libary)g (developers)g(we)113 1467 y(*)g(distinguish)g(these)g(functions)160 1523 y(#define)g(MPI_NULL_FN)113 1580 y(*/)89 1636 y(#define)g (MPI_NULL_COPY_FN)93 b(...)89 1693 y(#define)23 b(MPI_NULL_DELETE_FN)45 b(...)89 1805 y Fg(I'v)o(e)19 b(discussed)i(this)f(with)g(Nathan)g (Doss)f(and)g(I)h(b)q(eliev)o(e)i(this)e(has)f(b)q(een)i(incorp)q (orated)f(in)o(to)g(the)89 1862 y(public)d(domain)f(v)o(ersion)f(as)g (w)o(ell.)31 1956 y(5.)22 b(There)14 b(is)h(a)e(long,)i(still)g (unresolv)o(ed)g(discussion)h(cen)o(tering)f(around)f(P)l(A)o(CK/UNP)l (A)o(CK.)g(A)o(t)f(the)h(end)89 2012 y(of)g(it,)i(\(at)e(least)h(the)g (last)h(message)e(I)i(see\),)f(Marc)f(Snir)i(sa)o(ys:)89 2087 y(Conclusion:)143 2181 y Fa(\017)23 b Fg(a\))16 b(A)g(message)g(that)g(w)o(as)f(created)i(b)o(y)f(one)h(or)e(sev)o (eral)i(sequences)h(of)e(related)g(P)l(A)o(CK)h(calls)g(is)189 2237 y(sen)o(t)e(with)g(t)o(yp)q(e)g(MPI)p 581 2237 V 17 w(P)l(A)o(CKED.)143 2311 y Fa(\017)23 b Fg(b\))c(A)h(message)f(that) g(is)h(going)g(to)f(b)q(e)i(unpac)o(k)o(ed)f(b)o(y)g(one)f(or)h(sev)o (eral)f(sequences)i(of)f(related)189 2367 y(UNP)l(A)o(CK)15 b(calls)h(is)g(receiv)o(ed)h(with)e(t)o(yp)q(e)g(MPI)p 1019 2367 V 17 w(P)l(A)o(CKED.)143 2440 y Fa(\017)23 b Fg(c\))d(A)g(Message)f(created)h(b)o(y)g(one)h(sequence)g(of)f (related)g(P)l(A)o(CK)g(calls,)i(and)f(sen)o(t)e(with)i(t)o(yp)q(e)189 2496 y(MPI)p 281 2496 V 16 w(P)l(A)o(CKED)15 b(is)h(iden)o(tical)h(to)d (a)h(message)g(sen)o(t)g(with)h(a)e(regular)i(send.)910 2719 y(5)p eop %%Page: 6 6 6 5 bop 89 95 a Fg(I)15 b(don't)g(understand)h(wh)o(y)e(y)o(ou)h(came)g (to)g(the)g(conclusion)i(it)e(is)h(wrong)e(to)h(send)h(the)f (concatenation)89 152 y(of)f(sev)o(eral)i(pac)o(king)g(units)f(with)h (t)o(yp)q(e)f(MPI)p 868 152 14 2 v 17 w(P)l(A)o(CKED.)89 227 y(\(end)g(of)g(Marc\).)k(Where)c(do)g(w)o(e)g(stand)g(here?)31 320 y(6.)22 b(MPI)p 181 320 V 16 w(BOTTOM)15 b(in)i(F)l(ortran.)89 395 y(There)k(is)h(a)f(bug)h(\(or)f(unin)o(tended)i(e\013ect\))e(in)h (the)g(de\014nition)h(of)e(MPI)p 1385 395 V 16 w(BOTTOM)h(for)e(F)l (ortran.)89 452 y(Sp)q(eci\014cally)l(,)g(MPI)p 429 452 V 17 w(BOTTOM)d(can)h(b)q(e)f(used)h(as)f(a)g(bu\013er)g(address;)h(in) g(C)f(the)g(appropriate)h(v)m(alue)89 508 y(will)12 b(often)e(b)q(e)h (\(v)o(oid)g(*\)0.)17 b(But)10 b(in)i(F)l(ortran,)e(there)g(are)g(no)h (p)q(oin)o(ter)g(t)o(yp)q(es;)g(in)g(most)f(implemen)o(tations,)89 564 y(all)16 b(v)m(alues)g(are)f(passed)h(b)o(y)f(reference.)20 b(No)o(w)15 b(consider)h(the)f(follo)o(wing)h(t)o(w)o(o)e(co)q(de)i (fragmen)o(ts:)89 675 y Fd(C:)89 732 y(void)23 b(*p;)89 845 y(p)g(=)h(\(void)f(*\)MPI_BOTTOM;)89 901 y(MPI_Send\()f(p,)i(...)f (\);)89 1024 y(Fortran:)89 1081 y(integer)g(p)89 1194 y(p)g(=)h(MPI_BOTTOM;)89 1250 y(MPI_Send\()e(p,)i(...)f(\);)89 1373 y Fg(The)15 b(C)g(case)g(is)h(clearly)g(allo)o(w)o(ed;)g(the)f(F)l (ortran)f(case)h(m)o(ust)g(not)g(b)q(e.)89 1448 y(The)g(only)h(w)o(a)o (y)e(that)h(I)h(can)f(see)h(to)f(pro)o(vide)g(MPI)p 971 1448 V 17 w(BOTTOM)g(to)g(F)l(ortran)f(is)i(to)f(mak)o(e)g(it)g(a)g(sp) q(ecial)89 1504 y(constan)o(t)20 b(that)h(is)g(lik)o(e)i(a)d(reserv)o (ed)i(w)o(ord)e(-)i(assigning)f(it)h(to)e(a)h(v)m(ariable)i(is)f (erroneous)f(\(that)f(is,)89 1561 y(there)15 b(is)g(a)f(sp)q(ecial)j (lo)q(cation)e(that)f(is)i(kno)o(wn)e(as)h(MPI)p 1042 1561 V 16 w(BOTTOM;)f(the)h(routines)g(can)g(test)g(for)f(that)89 1617 y(sp)q(ecial)20 b(lo)q(cation\).)31 b(This)19 b(ma)o(y)f(seem)h (ob)o(vious,)h(but)e(it)h(is)h(easily)f(o)o(v)o(erlo)q(ok)o(ed.)30 b(W)l(e)19 b(certainly)h(did)89 1674 y(while)d(writing)g(the)f(mo)q (del)h(implemen)o(tation.)24 b(I)17 b(b)q(eliev)o(e)h(that)e(this)g (merits)g(some)g(commen)o(t)g(in)h(the)89 1730 y(standard,)11 b(since)h(to)e(F)l(ortran,)h(MPI)p 713 1730 V 16 w(BOTTOM)g(is)h(not)e (a)h("constan)o(t")f(as)g(de\014ned)j(b)o(y)d(that)h(standard.)31 1824 y(7.)22 b(The)14 b(requiremen)o(t)h(for)e(an)h(En)o(vironmen)o (tal)h(Enquiry)g(that)e(allo)o(ws)h(to)g(\014nd)h(whether)f(t)o(w)o(o)f (no)q(des)i(use)89 1880 y(the)f(same)f(data)g(represen)o(tation)h(is)h (reasonable,)f(irresp)q(ectiv)o(e)h(of)f(the)g(sp)q(eci\014c)h(use)g (that)e(Rolf)h(has)g(in)89 1937 y(mind,)k(for)f(P)o(armacs)g(em)o (ulation.)27 b(More)17 b(generally)l(,)i(users)e(ma)o(y)g(w)o(an)o(t)g (enquiry)h(functions)g(to)f(\014nd)89 1993 y(the)e(t)o(yp)q(e)g(of)f (no)q(de)i(on)f(whic)o(h)h(a)e(pro)q(cess)i(runs.)k(This)15 b(w)o(ould)h(help)g(in)g(situations)f(where)g(the)g(user)g(is)89 2049 y(willing)i(to)d(run)i(on)f(an)o(y)f(a)o(v)m(ailable)j(mac)o(hine) f(on)f(a)f(net)o(w)o(ork,)g(but)h(ma)o(y)f(w)o(an)o(t)g(to)h(use)g (di\013eren)o(t)g(co)q(de)89 2106 y(on)g(di\013eren)o(t)g(mac)o(hine)h (t)o(yp)q(es.)89 2181 y(It)g(is)g(not)f(reasonable)i(to)e(assume)g (that)g(the)h(MPI)g(forum)f(will)j(dev)o(elop)f(its)f(o)o(wn)f (nomenclature)i(for)89 2237 y(data)g(represen)o(tations)g(\(size)h(of)f (eac)o(h)h(basic)g(language)g(data)f(t)o(yp)q(e,)h(big)g(vs)f(little)i (endian,)g(\015oating)89 2294 y(p)q(oin)o(t)14 b(format,)e(c)o (haracter)h(co)q(des,)h(etc.\).)k(First)c(b)q(ecause)g(other)f (standards)g(try)g(to)g(handle)i(this)f(issue)89 2350 y(and,)22 b(second,)g(b)q(ecause)f(a)g(complete)g(c)o(haracterization)g (of)f(the)h(data)f(represen)o(tation)g(used)i(b)o(y)e(a)89 2407 y(pro)q(cess)e(\(whic)o(h)h(dep)q(ends)h(on)e(the)h(underlying)h (mac)o(hine)f(arc)o(hitecture)g(and)f(on)h(the)f(compiler\))h(is)89 2463 y(lik)o(ely)e(to)d(b)q(e)i(quite)g(length)o(y)l(.)89 2538 y(W)l(e)f(agreed)f(that)h(v)o(endors)f(can)h(attac)o(h)f(implemen) o(tation)i(sp)q(eci\014c)h(prede\014ned)f(attributes)f(that)f(are)89 2594 y(asso)q(ciated)h(with)h(MPI)p 501 2594 V 16 w(COMM)p 669 2594 V 16 w(W)o(ORLD.)f(I)h(w)o(ould)f(suggest)g(the)g(follo)o (wing:)910 2719 y(6)p eop %%Page: 7 7 7 6 bop 89 95 a Fg(Asso)q(ciate)14 b(a)f(prede\014ned)i(attribute)f(k)o (ey)f(MPI)p 907 95 14 2 v 17 w(PR)o(OC)p 1055 95 V 16 w(TYPE)h(with)g(MPI)p 1405 95 V 16 w(COMM)p 1573 95 V 16 w(W)o(ORLD.)g(The)89 152 y(v)m(alue)k(of)f(this)g(attribute)g(enco)q (des)h(the)f("t)o(yp)q(e")g(of)f(the)h(executing)i(pro)q(cess,)e(i.e.) 26 b(the)17 b(t)o(yp)q(e)g(of)g(data)89 208 y(represen)o(tation)e(used) i(b)o(y)e(this)h(pro)q(cess.)22 b(The)16 b(meaning)g(of)g(the)f(v)m (alues)i(of)e(this)i(attribute)e(is)h(imple-)89 265 y(men)o(tation)h (dep)q(enden)o(t,)j(except)e(that)f(if)h(t)o(w)o(o)f(pro)q(cesses)h (return)g(the)g(same)f(v)m(alue,)j(then)e(they)g(use)89 321 y(the)d(same)g(data)f(represen)o(tation)i(\(i.e.,)e(data)h(can)g(b) q(e)h(mo)o(v)o(ed)f(un)o(t)o(yp)q(ed)g(b)q(et)o(w)o(een)h(these)f(pro)q (cesses\).)89 396 y(Implemen)o(ters)e(of)f(MPI)g(for)g(homogeneous)g (SPMD)g(systems)g(just)g(need)h(to)f(alw)o(a)o(ys)g(return)g(the)g (same)89 453 y(v)m(alue)17 b(\(0?\))k(with)15 b(this)i(attribute.)j (Implemen)o(ters)d(of)e(MPI)h(on)f(heterogeneous)h(systems)f(ma)o(y)g (need)89 509 y(some)g(w)o(ork.)89 584 y(If)k(this)h(suggestion)f (sounds)h(reasonable,)g(then)f(the)h(v)m(arious)f(MPI)g(implemen)o (ters)i(ma)o(y)d(accept)i(it)89 641 y(no)o(w)14 b(as)h("common)g (practice",)g(and)g(w)o(e)g(can,)g(at)g(a)g(latter)g(time,)g(add)g (this)h(to)e(the)i(standard.)31 735 y(8.)22 b(Meaning)15 b(of)g(the)g(reduce)h(op)q(erations)g(on)f(complex)h(datat)o(yp)q(es.) 89 810 y(The)g(\014rst)h(question)g(is)g(the)f(meaning)h(of)f(max,)g (min)h(on)g(a)f(complex)h(v)m(alue.)25 b(W)l(e)17 b(in)o(tended)h(it)e (to)g(b)q(e)89 866 y(the)f(max,)f(min)i(mo)q(dulus)h(v)m(alue.)89 941 y(The)c(second)h(question)g(w)o(as)e(ho)o(w)h(this)h(v)m(alue)g(is) g(returned.)19 b(It)14 b(is)f(a)g(little)i(di\013eren)o(t)e(from)g (most)f(of)h(the)89 998 y(reduce)i(v)m(alues)g(since)g(the)g(mo)q (dulus)g(is)g(real)f(but)g(acts)g(on)g(a)g(complex)h(v)m(alue.)21 b(Here)14 b(are)g(the)g(p)q(ossible)89 1054 y(w)o(a)o(ys)g(that)g(w)o (e)h(though)o(t)g(of:)143 1148 y Fa(\017)23 b Fg(1\))14 b(Return)i(the)f(complex)h(v)m(alue)h(for)d(whic)o(h)i(the)g(max/min)f (mo)q(dulus)h(o)q(ccured.)143 1221 y Fa(\017)23 b Fg(2\))14 b(Return)h(the)f(max/min)g(mo)q(dulus)i(in)f(a)f(real)h(or)f(double)i (v)m(ariable)g(\(as)d(appropriate)i(for)e(ho)o(w)189 1278 y(the)i(complex)h(is)g(stored\).)143 1351 y Fa(\017)23 b Fg(3\))14 b(Return)i(the)f(max/min)h(mo)q(dulus)g(in)g(the)f(real)h (part)e(of)h(an)g(imaginary)h(v)m(alue.)143 1424 y Fa(\017)23 b Fg(4\))16 b(Return)h(the)f(max/min)h(mo)q(dulus)g(in)g(the)g(real)g (part)e(of)h(the)h(imaginary)g(v)m(alue)g(and)g(return)189 1480 y(the)e(angle)h(in)g(the)f(imaginary)h(part)e(\(i.e.,)h(mak)o(e)f (it)i(in)o(to)f(p)q(olar)h(co)q(ordinates\).)89 1574 y(The)g(feeling)h(w)o(as)e(that)f(2\))h(w)o(as)g(bad)h(b)q(ecause)h(of) e(the)h(mixture)g(of)f(complex)h(and)g(real/double)h(v)m(ari-)89 1631 y(ables)j(and)h(the)f(fact)f(that)g(y)o(ou)h(cannot)g(\014gure)g (out)g(the)g(original)h(v)m(alue)g(from)e(the)h(mo)q(dulus.)36 b(4\))89 1687 y(has)16 b(an)h(app)q(eal)g(from)f(the)h(p)q(oin)o(t)g (of)f(maximizing)i(information)f(returned)g(but)g(will)h(probably)f (lead)89 1743 y(to)g(user)h(errors.)28 b(1\))17 b(and)h(3\))f(ha)o(v)o (e)h(their)g(merits)g(but)g(w)o(e)g(lean)h(to)o(w)o(ard)d(3\))i(since)h (y)o(ou)e(can)h(get)g(the)89 1800 y(mo)q(dulus)e(from)f(the)h(original) g(v)m(alue)h(but)e(not)g(the)h(other)f(w)o(a)o(y)f(around.)21 b(The)15 b(negativ)o(e)h(w)o(e)f(sa)o(w)g(w)o(as)89 1856 y(that)f(y)o(ou)h(get)g(bac)o(k)g(the)g(v)m(alue)i(and)e(not)g(the)g (max/min)g(mo)q(dulus.)89 1932 y(W)l(e)g(therefore)g(prop)q(ose)g(that) g(1\))f(b)q(ecomes)i(the)f(basis)h(of)f(the)g(clari\014ed)i(text.)89 2007 y(Marc)d(has)h(also)h(w)o(eighed)f(in)i(with)e(an)g(analysis)h (that)f(fa)o(v)o(ors)e(option)j(1\).)910 2719 y(7)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Tue Jul 12 23:27:36 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id XAA28266; Tue, 12 Jul 1994 23:27:36 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id XAA27175; Tue, 12 Jul 1994 23:27:10 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 12 Jul 1994 23:27:09 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares10.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id XAA27168; Tue, 12 Jul 1994 23:27:07 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares10.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id WAA22329; Tue, 12 Jul 1994 22:27:03 -0500 Message-Id: <199407130327.WAA22329@antares10.mcs.anl.gov> To: snir@watson.ibm.com Cc: mpi-core@CS.UTK.EDU, frankeh@watson.ibm.com, gropp@antares10.mcs.anl.gov, lusk@antares10.mcs.anl.gov, doss@ERC.MsState.Edu Subject: Re: Correction on status argument to 3 routines In-reply-to: Your message of "Tue, 12 Jul 1994 17:06:59 EDT." <9407122106.AA24353@snir.watson.ibm.com> Date: Tue, 12 Jul 1994 22:27:02 -0500 From: Rusty Lusk | > ___________________________________________________________________________ | > | > Erratum: | > | > The C bindings for MPI_Get_count, MPI_Get_elements, and MPI_Test_cancelled | > should be as follows: | > | > int MPI_Get_count(MPI_Status *status, MPI_Datatype datatype, int *count) | > int MPI_Get_elements(MPI_Status *status, MPI_Datatype datatype, int *count) | > int MPI_Test_cancelled(MPI_Status *status, int *flag) | > | > instead of: | > | > int MPI_Get_count(MPI_Status status, MPI_Datatype datatype, int *count) | > int MPI_Get_elements(MPI_Status status, MPI_Datatype datatype, int *count) | > int MPI_Test_cancelled(MPI_Status status, int *flag) | > | > ___________________________________________________________________________ | > | Marc>> Could you also post a message explaining why this is an error? Yes. I explained this briefly when Bill and I posted this about three weeks ago, along with the Aint problems, but I am glad to go into more detail now. At that time the only responses were in favor of treating this as an error. This is an error because it is not what was intended in the standard. The error was my fault, since I had responsibility for the bindings. When one passes a structure in C, common practice is to pass a pointer instead of putting the entire structure on the stack. This is more efficient, and because it is more efficient, it is considered bad practice in C to pass a whole structure on the stack. In fact, some compilers (Sun's, for example), will automatically replace the structure by a pointer to it in the call. For MPI's opaque objects, we pass the type, which is understood to be represented as a pointer, so in passing an object of the opaque type, we do not violate this principle. In fact, it is easy to distinguish input arguments from output arguments since the ouput ones *must* be pointers, so that MPI can allocate new objects and replace the pointers, etc. The input arguments (existing communicators, for example) are then passed as the type MPI_Comm itself, and pointers to MPI_Comm types are passed when the routine is to allocate a new communicator, for example. The bindings were first formalized when the MPI status object was an opaque object type like MPI_Comm, MPI_Group, etc. Then at some point, some 9 months into the MPI process, we decided (correctly, in my opinion) that the MPI_Status objects should be *not* opaque, since they were likely to be allocated in user memory, and thus completely visible to the user. We even reduced the number of MPI routines by giving the user access to the fields in the MPI_Status object directly (instead of having to call MPI_Get_source, etc.). Since the MPI_Status type was now no longer opaque (and thus no longer likely to be represented as a pointer) the three routines that take such an object as an *input* argument should have had their bindings changed at that point to accept a pointer to this object, since that is the way C programmers program. It was my oversight that the bindings were not changed at this point. I believe that it is very important that they be changed now. After getting no disagreement with my posting several weeks ago, and several notes of agreement, we went ahead and changed them in our implementation, changed them in the final draft of our book, and got Steve's agreement to post them as errata, so that this error could be nipped in the bud. I hope that it is not too late to change them as well in the galleys of the Journal version of the draft (The changes required are quite small). I presume they will be changed in the ARM. Steve's note in the Errata sheet refers to inconsistent compiler behavior. What he is referring to is that some compilers (Sun's, among others) automatically replace structures in parameter lists by pointers to them. Others (IBM's, for example) do not. Thus a program that incorrectly passes the structure instead of a pointer to it (when the called routine is expecting a pointer) will work fine on a Sun, but not on an RS6000. Of course this lack of portability is really the programmer's fault, but it is best to avoid this unexpected behavior by requiring the standard practice of passing pointers instead of visible structures as input parameters to C routines. I hope this makes it clear why the above change should be treated as an error (mine) that we should not hesitate to rectify immediately. Rusty From owner-mpi-core@CS.UTK.EDU Wed Jul 13 05:30:30 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id FAA29582; Wed, 13 Jul 1994 05:30:29 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id FAA01996; Wed, 13 Jul 1994 05:29:52 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 13 Jul 1994 05:29:51 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id FAA01988; Wed, 13 Jul 1994 05:29:48 -0400 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA23112 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Wed, 13 Jul 1994 10:29:33 +0100 Received: by tycho.co.uk (5.0/SMI-SVR4) id AA02925; Wed, 13 Jul 1994 10:26:16 +0000 Date: Wed, 13 Jul 1994 10:26:16 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9407130926.AA02925@tycho.co.uk> To: lusk@mcs.anl.gov Cc: snir@watson.ibm.com, mpi-core@CS.UTK.EDU, frankeh@watson.ibm.com, gropp@antares10.mcs.anl.gov, doss@ERC.MsState.Edu In-Reply-To: <199407130327.WAA22329@antares10.mcs.anl.gov> (message from Rusty Lusk on Tue, 12 Jul 1994 22:27:02 -0500) Subject: Re: Correction on status argument to 3 routines > When one passes a structure in C, common practice is to pass a > pointer instead of putting the entire structure on the stack. This is > more efficient, and because it is more efficient, it is considered bad > practice in C to pass a whole structure on the stack. In fact, some > compilers (Sun's, for example), will automatically replace the > structure by a pointer to it in the call. Just a slight clarification of this. The C standard requires the semantics of passing a structure to be BY VALUE (as for any other scalar argument). This ALWAYS necessitates a copy of the structure, but there are various possible implementations. On SPARC the ABI (and therefore the Sun compilers) code like this extern void foo(struct bah); void fubar(void) { struct bah c; foo(c); } is implemented as if it were written like this void fubar(void) { struct bah c; { struct bah compiler_temp = c; /* Structure assignment = memcpy */ foo(&compiler_temp); /* Call foo on the temp */ } } and foo is compiled AS IF it were declared void foo (struct bah *), and therefore will happen to work if (incorrectly) called with a struct bah * argument. Another possibility (which GCC often does) is to pass the structure in the (integer) registers if it is small enough. (In this case of course foo would NOT work if called with a struct bah *) In any case the compiler must ensure that the effect is call by value, which implies a copy of the whole structure somewhere (either into store, or into registers). Since the whole idea of the MPI_Status was to achieve efficiency (which is why we expose fields direct to the user, and make the allocation the user's responsibility), the last thing we want is to introduce a structure copy into routines which take an MPI_Status argument. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 454 616171 +1 508 371 0088 FAX : +44 454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com:8080/welcome.html From owner-mpi-core@CS.UTK.EDU Tue Jul 19 11:01:51 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id LAA24084; Tue, 19 Jul 1994 11:01:51 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA15996; Tue, 19 Jul 1994 11:00:26 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 19 Jul 1994 11:00:22 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA15977; Tue, 19 Jul 1994 11:00:20 -0400 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.8/8.6.6) id LAA15978 for mpi-core@cs.utk.edu; Tue, 19 Jul 1994 11:00:18 -0400 Date: Tue, 19 Jul 1994 11:00:18 -0400 From: David Walker Message-Id: <199407191500.LAA15978@rios2.EPM.ORNL.GOV> To: mpi-core@CS.UTK.EDU Subject: Error in MPI draft (UNPACK) On page 84, lines 40-41, the MPI draft says that "position is incremented...so that the output value of position is the first location in the output buffer after the locations occupied by the message that was unpacked." This is wrong I think. We unpack inbuf into outbuf, so position is an offset into inbuf, not into outbuf. David From owner-mpi-core@CS.UTK.EDU Fri Aug 5 14:35:19 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id OAA22070; Fri, 5 Aug 1994 14:35:18 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA15340; Fri, 5 Aug 1994 14:34:17 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 5 Aug 1994 14:34:13 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA15288; Fri, 5 Aug 1994 14:33:50 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA02136; Fri, 5 Aug 94 11:33:44 -0700 Date: Fri, 5 Aug 94 11:33:44 -0700 From: Steve Otto Message-Id: <9408051833.AA02136@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: URGENT msg; addendum and, I need to add to my suggested changing of the spec: also drop MPI_2COMPLEX from page 115, line 43, of the May 5 spec. AND on line 12, page 209 (Language Binbding annex). OK? --Steve Otto From owner-mpi-core@cs.utk.edu Fri Aug 5 14:35:37 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id OAA22075; Fri, 5 Aug 1994 14:35:36 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA14790; Fri, 5 Aug 1994 14:25:27 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 5 Aug 1994 14:25:23 EDT Errors-to: owner-mpi-core@cs.utk.edu Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA14773; Fri, 5 Aug 1994 14:25:17 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA02106; Fri, 5 Aug 94 11:25:09 -0700 Date: Fri, 5 Aug 94 11:25:09 -0700 From: Steve Otto Message-Id: <9408051825.AA02106@merckx.cse.ogi.edu> To: mpi-core@cs.utk.edu Subject: Min/Max Reduce on Complex; URGENT! Dear All, OK, we need to reach a decision on this issue of complex min/max reduce, REAL SOON, since I need to get corrections in to MIT Press for the IJSA-published version of the spec and I would like it to say the right thing. Therefore, I'm going to make (what I think is) a reasonable suggestion, and unless an outcry ensues, that's what I will send in to MIT Press. I am guilty of stirring some of this controversey up, because I assumed that Fortran *did* define MAX on COMPLEX. Bob Knighten, Leslie Hart, Peter Pacheco, and others, point out that this is not so. Therefore, despite the moral support thrown in by Cherri Pancake (thanks Cherri ;^) ), I retract the claim that this feature will be considered "natural" by Fortran programmers. So I think (and it seems that most will agree) that we should drop the requirement of max/min reduce for COMPLEX. Marc says this in one of his messages (included below). This will be one, very small change in the document, namely dropping the word "Complex" from line 36 of page 113 of the May 5 draft. OK? --Steve Otto PS Please note that we can revisit this issue (if we want) in MPI-2. MPI-1 cannot get everything absolutely perfect. It's doing pretty well, and it will not fail because of minor glitches (or *perceived* minor glitches which some could claim the lack of max on complex to be). Right now, I have to tell MIT Press what our corrections are and I don't have time for a lot of discussion -- they are going to press. Marc: >Let's not be carried away. MIN of complex can be either left out of MPI, or >if in, be defined to be an MPI reduction operation, which means an >associative binary operation that returns a value of the same type as the >inputs. The only reasonable definition of MIN of complex is the one Steve >and I proposed. Whether one should include it at all is a good >question. Note that Fortran defines MIN, MAX, MINLOC and MAXLOC only for >types INTEGER and REAL. Note also that a user can always provide it's own >user-defined MIN, MAX, MINLOC and MAXLOC for COMPLEX. Both are arguments >against inclusion. >I, personally, >vote for keeping it out. If we decide to put it in, then we should use >the text that was sent be Steve, also need to put in the corrsponding >versions of MINLOC and MAXLOC. From owner-mpi-core@CS.UTK.EDU Fri Aug 5 16:36:17 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA22643; Fri, 5 Aug 1994 16:36:17 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA25656; Fri, 5 Aug 1994 16:35:43 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 5 Aug 1994 16:35:37 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA25544; Fri, 5 Aug 1994 16:35:23 -0400 Received: from tualatin.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA28440; Fri, 5 Aug 94 13:34:46 PDT Received: by tualatin.SSD.intel.com (4.1/SMI-4.1) id AA02851; Fri, 5 Aug 94 13:34:45 PDT Date: Fri, 5 Aug 94 13:34:45 PDT Message-Id: <9408052034.AA02851@tualatin.SSD.intel.com> From: Bob Knighten To: otto@cse.ogi.edu Cc: mpi-core@CS.UTK.EDU Subject: Re: Min/Max Reduce on Complex; URGENT! In-Reply-To: <9408051825.AA02106@merckx.cse.ogi.edu> References: <9408051825.AA02106@merckx.cse.ogi.edu> Reply-To: knighten@ssd.intel.com (Bob Knighten) Can we do a quick mail poll? I would vote to NOT include this in the current MPI specification and so to make the editorial change that you (Steve) have mentioned. -- Bob From owner-mpi-core@cs.utk.edu Fri Aug 5 17:35:02 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id RAA23200; Fri, 5 Aug 1994 17:35:01 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA27985; Fri, 5 Aug 1994 17:01:01 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 5 Aug 1994 17:00:56 EDT Errors-to: owner-mpi-core@cs.utk.edu Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA27976; Fri, 5 Aug 1994 17:00:53 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA25174; Fri, 5 Aug 94 15:52:47 CDT Date: Fri, 5 Aug 94 15:52:47 CDT From: Tony Skjellum Message-Id: <9408052052.AA25174@Aurora.CS.MsState.Edu> To: otto@cse.ogi.edu, knighten@ssd.intel.com Subject: Re: Min/Max Reduce on Complex; URGENT! Cc: mpi-core@cs.utk.edu I would vote NOT to include as well. _Tony From owner-mpi-core@CS.UTK.EDU Fri Aug 5 18:25:04 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id SAA23537; Fri, 5 Aug 1994 18:25:03 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id SAA05309; Fri, 5 Aug 1994 18:24:41 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 5 Aug 1994 18:24:39 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from noc.usfca.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id SAA05302; Fri, 5 Aug 1994 18:24:36 -0400 Received: by noc.usfca.edu (5.65/DEC-Ultrix/4.3) id AA12233; Fri, 5 Aug 1994 15:26:55 -0700 Received: by mobydick.usfca.edu (4.1/SMI-4.1) id AA06818; Fri, 5 Aug 94 15:25:16 PDT Date: Fri, 5 Aug 94 15:25:16 PDT From: peter@mobydick.usfca.edu (Peter Pacheco) Message-Id: <9408052225.AA06818@mobydick.usfca.edu> To: knighten@ssd.intel.com, otto@cse.ogi.edu, tony@Aurora.CS.MsState.Edu Subject: Re: Min/Max Reduce on Complex; URGENT! Cc: mpi-core@CS.UTK.EDU I agree with Bob and Tony -- don't include. Peter Pacheco From owner-mpi-core@CS.UTK.EDU Mon Aug 8 04:29:02 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id EAA07798; Mon, 8 Aug 1994 04:29:02 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA17267; Mon, 8 Aug 1994 04:28:17 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 8 Aug 1994 04:28:15 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from epcc.ed.ac.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA17257; Mon, 8 Aug 1994 04:28:08 -0400 Date: Mon, 8 Aug 94 09:27:43 BST Message-Id: <27271.9408080827@subnode.epcc.ed.ac.uk> From: L J Clarke Subject: Re: Min/Max Reduce on Complex; URGENT! To: Steve Otto , mpi-core@CS.UTK.EDU In-Reply-To: Steve Otto's message of Fri, 5 Aug 94 11:25:09 -0700 Reply-To: lyndon@epcc.ed.ac.uk I'll vote NO for having complex min/max in MPI-1. Hope an email poll works out! Regards Lyndon /--------------------------------------------------------\ e||) | Lyndon J Clarke Edinburgh Parallel Computing Centre | e||) c||c | Tel: 031 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | c||c \--------------------------------------------------------/ From owner-mpi-core@CS.UTK.EDU Mon Aug 8 13:09:16 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id NAA12337; Mon, 8 Aug 1994 13:09:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA03808; Mon, 8 Aug 1994 13:08:13 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 8 Aug 1994 13:08:05 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from nipmuc.fsl.noaa.gov by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA03746; Mon, 8 Aug 1994 13:07:57 -0400 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA13017; Mon, 8 Aug 94 11:07:53 MDT Message-Id: <9408081707.AA13017@nipmuc.fsl.noaa.gov> To: mpi-core@CS.UTK.EDU, otto@cse.ogi.edu Subject: Re: Min/Max Reduce on Complex; URGENT! In-Reply-To: Your message of "Fri, 05 Aug 94 11:25:09 PDT." <9408051825.AA02106@merckx.cse.ogi.edu> Date: Mon, 08 Aug 94 11:07:52 MDT From: Leslie Hart (NOAA/FSL) I also vote NOT to include min/max as valid operations on type COMPLEX. Leslie From owner-mpi-core@CS.UTK.EDU Mon Aug 8 22:34:39 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id WAA17532; Mon, 8 Aug 1994 22:34:38 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id WAA13557; Mon, 8 Aug 1994 22:34:01 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 8 Aug 1994 22:33:59 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from mime.top.cis.syr.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id WAA13549; Mon, 8 Aug 1994 22:33:56 -0400 Message-Id: <199408090233.WAA13549@CS.UTK.EDU> Date: Mon, 8 Aug 94 22:31:23 EDT From: Sanjay Ranka To: mpi-core@CS.UTK.EDU Phone: 315-443-4457 Fax: 315-443-1122 Address: 4-116 CST, School of Computer Science, Syracuse University, Syracuse, NY 13244-4100 I vote against having complex min/max. sanjay From owner-mpi-core@CS.UTK.EDU Tue Aug 9 04:55:57 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id EAA18770; Tue, 9 Aug 1994 04:55:57 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA03657; Tue, 9 Aug 1994 04:55:06 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 9 Aug 1994 04:55:03 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from sun2.nsfnet-relay.ac.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA03640; Tue, 9 Aug 1994 04:54:58 -0400 Via: uk.ac.southampton.parallel-computing-support; Tue, 9 Aug 1994 09:54:33 +0100 From: Ian Glendinning Date: Tue, 9 Aug 94 09:53:53 BST Message-Id: <5269.9408090853@par.soton.ac.uk> Via: aloisius.sucs.soton.ac.uk; Tue, 9 Aug 94 09:53:53 BST To: mpi-core@CS.UTK.EDU I vote not to have min/max as valid operations on type COMPLEX. Ian From owner-mpi-core@CS.UTK.EDU Tue Aug 9 07:28:47 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id HAA22653; Tue, 9 Aug 1994 07:28:46 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id HAA19237; Tue, 9 Aug 1994 07:28:24 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 9 Aug 1994 07:28:22 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from convex.convex.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id HAA19230; Tue, 9 Aug 1994 07:28:21 -0400 Received: from mozart.convex.com by convex.convex.com (5.64/1.35) id AA26248; Tue, 9 Aug 94 06:21:11 -0500 Received: from localhost by mozart.convex.com (8.6.4/1.28) id GAA03758; Tue, 9 Aug 1994 06:26:32 -0500 From: joelw@mozart.convex.com (Joel Williamson) Message-Id: <199408091126.GAA03758@mozart.convex.com> Subject: No on min/max complex To: mpi-core@CS.UTK.EDU Date: Tue, 9 Aug 94 6:26:31 CDT X-Mailer: ELM [version 2.3 PL11] I vote against having complex min/max. Joel Williamson From owner-mpi-core@CS.UTK.EDU Tue Aug 9 12:14:35 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id MAA24642; Tue, 9 Aug 1994 12:14:35 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA15491; Tue, 9 Aug 1994 12:13:15 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 9 Aug 1994 12:13:07 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from thialfi.cs.cornell.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA15455; Tue, 9 Aug 1994 12:13:05 -0400 Received: from CLOYD.CS.CORNELL.EDU by thialfi.cs.cornell.edu (5.67/I-1.99G) id AA03406; Tue, 9 Aug 94 12:12:59 -0400 Received: from ELLI.CS.CORNELL.EDU by cloyd.cs.cornell.edu (5.67/I-1.99F) id AA24630; Tue, 9 Aug 94 12:13:03 -0400 From: elster@cs.cornell.edu (Anne C. Elster) Date: Tue, 9 Aug 94 12:12:57 -0400 Message-Id: <9408091612.AA26025@elli.cs.cornell.edu> Received: by elli.cs.cornell.edu (5.67/N-0.13aa) id AA26025; Tue, 9 Aug 94 12:12:57 -0400 To: knighten@ssd.intel.com, otto@cse.ogi.edu Subject: Re: Min/Max Reduce on Complex; URGENT! Cc: elster@cs.cornell.edu, mpi-core@CS.UTK.EDU I vote with Steve and Bob (and the other votes I've noticed) to leave out complex min/max for MPI1. I also do not think it is MPI1's place to define operations that FORTRAN yet doesn't. MPI2 on the other hand ... ;-) Anne From owner-mpi-core@CS.UTK.EDU Tue Aug 9 13:33:59 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id NAA25548; Tue, 9 Aug 1994 13:33:58 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA20168; Tue, 9 Aug 1994 13:29:48 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 9 Aug 1994 13:29:46 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from piper.cs.colorado.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id NAA20161; Tue, 9 Aug 1994 13:29:43 -0400 Received: from [128.138.198.243] (mac3bryan.cs.colorado.edu [128.138.198.243]) by piper.cs.colorado.edu (8.6.9/8.6.9) with SMTP id LAA22486 for ; Tue, 9 Aug 1994 11:29:29 -0600 Date: Tue, 9 Aug 1994 11:29:29 -0600 Message-Id: <199408091729.LAA22486@piper.cs.colorado.edu> X-Sender: mcbr@128.138.204.4 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: mpi-core@CS.UTK.EDU From: mcbryan@cs.colorado.edu (Oliver A. McBryan) Subject: complex min/max I vote against having complex min/max. Oliver McBryan; mcbryan@cs.colorado.edu; 303-6650544; Fax 303-4922844 Dept of Computer Science, University of Colorado, Boulder, CO 80309. WWW: http://www.cs.colorado.edu/home/mcbryan/Home.html From owner-mpi-core@CS.UTK.EDU Mon Aug 15 18:40:37 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id SAA00115; Mon, 15 Aug 1994 18:40:36 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id SAA27569; Mon, 15 Aug 1994 18:39:55 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 15 Aug 1994 18:39:53 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from merckx.cse.ogi.edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id SAA27562; Mon, 15 Aug 1994 18:39:48 -0400 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA13239; Mon, 15 Aug 94 15:39:36 -0700 Date: Mon, 15 Aug 94 15:39:36 -0700 From: Steve Otto Message-Id: <9408152239.AA13239@merckx.cse.ogi.edu> To: mpi-core@CS.UTK.EDU Subject: END of Complex Min/Max; errata sent to MIT Press Dear All, The voting turned out to be unanimous in taking Complex MIN/MAX/MINLOC/MAXLOC operations out of MPI. I saw the first half of the votes, then I told MIT Press to please take the relevant words out of the published version (I was leaving for vacation). In case you are interested, enclosed are the errata that I sent to MIT Press. The galley proofs of the Journal publication of the spec look quite nice. There will still be errata of the published version of the spec -- for instance, the issue of WAIT | TEST operations on null-lists. The Preface to the published version says where to get the current errata from netlib. ("send mpi_errata.ps from mpi" to netlib@ornl.gov) --Steve encl: to MIT Press, errata on galleys >From otto@merckx.cse.ogi.edu Fri Aug 5 13:27 PDT 1994 Received: by merckx.cse.ogi.edu (1.37.109.4/16.2) id AA02346; Fri, 5 Aug 94 13:27:16 -0700 Date: Fri, 5 Aug 94 13:27:16 -0700 >From: Steve Otto Return-Path: Message-Id: <9408052027.AA02346@merckx.cse.ogi.edu> To: LHORTON@MITVMA.MIT.EDU Subject: Re: SCMP 8:3/4 Cc: archetyp@uxh.cso.uiuc.edu, dongarra@cs.utk.edu, otto@merckx.cse.ogi.edu Status: RO >I hope you're pleased with the look of Supercomputer >Apps 8:3/4. The first pages will be going from the >proofreader back to type soon, so I will need any >corrections from you by this coming Tuesday, August >9. Thanks--please call me if you have any questions. > >Yours, > >Laura > >********************************************************** > >Laura Horton phone: 617-253-2805 >Production Coordinator >MIT Press Journals fax: 617-258-6779 >55 Hayward Street >Cambridge, MA 02142 e-mail: LHORTON@MITVMA.MIT.EDU > >********************************************************** Dear Laura (and Lori), Yes, I am pleased with the look of the galleys! The formatting has come out quite nice and has a better "look" than what we put out ourselves. One overall comment I have is that I would question the removal of the Table of Contents. It's a big document (!) and the TOC helps one to navigate. I guess it doesn't fit into your style; but if you can do it, consider adding the TOC back in. I've done a quick scan of the document and I found a few glitches. I list them below. There are also a few changes that we are making, all of which are minor (a few lines removed or a few words modified). 1) page iv of Preface: line 5: "on MPI" should read "of MPI" line 9: "that have made" should read "and have made" 2) There are several errors associated with the fact that I had some manually-inserted wordbreaks in my copy because I needed to tell LaTeX explicitly what I wanted. Now that it has been re-formatted, however, these breaks are no longer in the right places. I think Archetype found many of these, but here are some that were missed: page 11, line 4: "FA-TAL" should be "FATAL" page 43 "GET-_COU" should be "GET_COU" page 131 2nd line from bottom, "group- size" should be "groupsize" page 184 line 6 "ATTR-_DEL" should be "ATTR_DEL" 3) On line 8, page 104, the font changes to italics and it shouldn't. 4) Removing type MPI_2COMPLEX. This is a change that we have made to our document and we would like the published version to reflect this. The required changes are: page 124, middle of page, in the line just after "Op Allowed Types" that reads "MPI_MAX, MPI_MIN.....Complex" --- please REMOVE the last item, that is, remove the word "Complex". page 126, near bottom of page, in the list that begins "Fortran:", please REMOVE the entire line that reads: "MPI_2COMPLEX pair of COMPLEXes" page 227, line 16 remove the line that reads "MPI_2COMPLEX" 5) More changes that we have made. page 23, line -7, put an asterisk in front of "status". That is, "MPI_Status status" should read "MPI_Status *status" same thing on page 80, line 9 same thing on page 57, line 10 same thing on page 231, line 1 same thing on page 229, line -5 same thing on page 232, line 9 6) More changes that we have made. page 74, line -5: "int *extent" change to: "MPI_Aint *extent" same thing on page 75, line 4 same thing on page 76, line -3 same thing on page 76, line -10 same thing on page 232, line 2 same thing on page 232, line 3 same thing on page 232, line 5 same thing on page 232, line 6 7) More changes that we have made. page 91, lines 10, 14, and 15: the word "outcount" and "OUTCOUNT" should be changed to "outsize" and "OUTSIZE" same thing on page 240, lines 14 and 16 8) More changes that we have made. page 20, line 7: the word "PRECISION" should be changed to "COMPLEX" 9) More changes that we have made. page 164, line 21, "MPI_Comm *comm" should be changed to "MPI_Comm comm" That's all I have for you. THANKS!! --Steve Otto --------------------------------------------------------------------------- Dr Steve Otto | Dept of Computer Science and Engineering *___________ | Oregon Graduate Institute of Science & Technology _____) | Mailing: 20000 NW Walker Rd, PO Box 91000 / | Portland, OR 97291-1000 USA / | email: otto@cse.ogi.edu /o | Phone: (503) 690-1486 (office) | Phone: (503) 641-8399 (home) | FAX: (503) 690-1553 | Shipping: 20000 NW Walker Rd, Beaverton, OR 97006 USA | URL: http://www.cse.ogi.edu/~otto/ --------------------------------------------------------------------------- From owner-mpi-core@CS.UTK.EDU Mon Aug 22 08:14:20 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id IAA25435; Mon, 22 Aug 1994 08:14:20 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id IAA17783; Mon, 22 Aug 1994 08:13:51 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 22 Aug 1994 08:13:50 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from convex.convex.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id IAA17775; Mon, 22 Aug 1994 08:13:47 -0400 Received: from mozart.convex.com by convex.convex.com (5.64/1.35) id AA05657; Mon, 22 Aug 94 07:06:28 -0500 Received: from localhost by mozart.convex.com (8.6.4/1.28) id HAA01620; Mon, 22 Aug 1994 07:11:25 -0500 From: joelw@mozart.convex.com (Joel Williamson) Message-Id: <199408221211.HAA01620@mozart.convex.com> Subject: 3rd party native MPI implementations To: mpi-core@CS.UTK.EDU Date: Mon, 22 Aug 94 7:11:25 CDT X-Mailer: ELM [version 2.3 PL11] I am interested in learning of 3rd party software vendors that are targeting MPI implementations to machine-specific messaging rather than layering on top of PVM,etc. Please respond here or via email to joelw@convex.com. Thanks, Joel Williamson From owner-mpi-core@CS.UTK.EDU Tue Sep 13 14:45:08 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id OAA27403; Tue, 13 Sep 1994 14:45:08 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA19596; Tue, 13 Sep 1994 14:43:23 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 13 Sep 1994 14:43:22 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id OAA19589; Tue, 13 Sep 1994 14:43:20 -0400 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.8/8.6.6) id OAA13722; Tue, 13 Sep 1994 14:43:19 -0400 From: David Walker Message-Id: <199409131843.OAA13722@rios2.EPM.ORNL.GOV> To: mpi-core@CS.UTK.EDU Subject: Null communicators Date: Tue, 13 Sep 94 14:43:18 -0500 A couple of points: 1) Can I check if a communicator is null as follows: ierr = mpi_comm_compare (comm, MPI_COMM_NULL, result); If so, am I correct in assuming that result=MPI_IDENT if comm is the null communicator. 2) How do I set a communicator to the null communicator? Can I just say comm = MPI_COMM_NULL Thanks, David From owner-mpi-core@CS.UTK.EDU Tue Sep 13 16:22:20 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA24841; Tue, 13 Sep 1994 16:22:20 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA27269; Tue, 13 Sep 1994 16:21:39 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 13 Sep 1994 16:21:38 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA27262; Tue, 13 Sep 1994 16:21:36 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA17846; Tue, 13 Sep 94 15:21:34 CDT Date: Tue, 13 Sep 94 15:21:34 CDT From: Tony Skjellum Message-Id: <9409132021.AA17846@Aurora.CS.MsState.Edu> To: mpi-core@CS.UTK.EDU, walker@rios2.EPM.ORNL.GOV Subject: Re: Null communicators ----- Begin Included Message ----- From owner-mpi-core@CS.UTK.EDU Tue Sep 13 14:31:06 1994 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 13 Sep 1994 14:43:22 EDT From: David Walker To: mpi-core@CS.UTK.EDU Subject: Null communicators Date: Tue, 13 Sep 94 14:43:18 -0500 Content-Length: 353 A couple of points: 1) Can I check if a communicator is null as follows: ierr = mpi_comm_compare (comm, MPI_COMM_NULL, result); If so, am I correct in assuming that result=MPI_IDENT if comm is the null communicator. > Yes! 2) How do I set a communicator to the null communicator? Can I just say comm = MPI_COMM_NULL > Yes! Thanks, David ----- End Included Message ----- From owner-mpi-core@CS.UTK.EDU Tue Sep 13 16:55:58 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA27337; Tue, 13 Sep 1994 16:55:58 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA00737; Tue, 13 Sep 1994 16:55:31 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 13 Sep 1994 16:55:27 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA00690; Tue, 13 Sep 1994 16:55:22 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2557; Tue, 13 Sep 94 16:55:21 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 9991; Tue, 13 Sep 1994 16:55:21 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Tue, 13 Sep 94 16:55:19 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA35792; Tue, 13 Sep 1994 16:55:13 -0400 Message-Id: <9409132055.AA35792@snir.watson.ibm.com> To: mpi-core@CS.UTK.EDU Reply-To: snir@watson.ibm.com X-Mailer: exmh version 1.2 1/14/94 Date: Tue, 13 Sep 1994 16:55:12 -0400 From: Marc Snir -:) -:) -:) -:) -:) From: David Walker To: mpi-core@CS.UTK.EDU Subject: Null communicators Date: Tue, 13 Sep 94 14:43:18 -0500 Content-Length: 353 1) Can I check if a communicator is null as follows: ierr = mpi_comm_compare (comm, MPI_COMM_NULL, result); If so, am I correct in assuming that result=MPI_IDENT if comm is the null communicator. > Yes! ***** I disagree. One should check if a communicator is null using a plain comparison: comm == MPI_COMM_NULL. Since communicator handles are stated to be of an integer or pointer type, such comparison must work, and will be more efficient that a call to MPI_COMM_COMPARE (.EQ. can be used in Fortran). On the other hand, we did not specify, so far, the behavior of MPI_COMM_COMPARE on null handles. Strictly speaking, a null handle points to no object, and MPI_COMM_COMPARE applied to a pair of null handles should fail, or return MPI_UNEQUAL (otherwise clause...). If we want MPI_COMM_COMPARE to behave as suggested, the standard should say: MPI_IDENT results iff comm1 and comm2 are handles to the same object (...) or both are null. This is OK with me, but need a correction in the document. 2) How do I set a communicator to the null communicator? Can I just say comm = MPI_COMM_NULL > Yes! ***** I agree From owner-mpi-core@CS.UTK.EDU Tue Sep 13 17:09:49 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id RAA01197; Tue, 13 Sep 1994 17:09:48 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA02139; Tue, 13 Sep 1994 17:09:44 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 13 Sep 1994 17:09:42 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA02131; Tue, 13 Sep 1994 17:09:38 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA18025; Tue, 13 Sep 94 16:09:36 CDT Date: Tue, 13 Sep 94 16:09:36 CDT From: Tony Skjellum Message-Id: <9409132109.AA18025@Aurora.CS.MsState.Edu> To: mpi-core@CS.UTK.EDU, snir@watson.ibm.com Subject: MPI_COMM_COMPARE Cc: walker@msr.epm.ornl.gov ----- Begin Included Message ----- From owner-mpi-core@CS.UTK.EDU Tue Sep 13 16:04:46 1994 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 13 Sep 1994 16:55:27 EDT To: mpi-core@CS.UTK.EDU Reply-To: snir@watson.ibm.com X-Mailer: exmh version 1.2 1/14/94 Date: Tue, 13 Sep 1994 16:55:12 -0400 From: Marc Snir Content-Length: 1334 -:) -:) -:) -:) -:) From: David Walker To: mpi-core@CS.UTK.EDU Subject: Null communicators Date: Tue, 13 Sep 94 14:43:18 -0500 Content-Length: 353 1) Can I check if a communicator is null as follows: ierr = mpi_comm_compare (comm, MPI_COMM_NULL, result); If so, am I correct in assuming that result=MPI_IDENT if comm is the null communicator. > Yes! ***** I disagree. One should check if a communicator is null using a plain comparison: comm == MPI_COMM_NULL. Since communicator handles are stated to be of an integer or pointer type, such comparison must work, and will be more efficient that a call to MPI_COMM_COMPARE (.EQ. can be used in Fortran). On the other hand, we did not specify, so far, the behavior of MPI_COMM_COMPARE on null handles. Strictly speaking, a null handle points to no object, and MPI_COMM_COMPARE applied to a pair of null handles should fail, or return MPI_UNEQUAL (otherwise clause...). If we want MPI_COMM_COMPARE to behave as suggested, the standard should say: MPI_IDENT results iff comm1 and comm2 are handles to the same object (...) or both are null. This is OK with me, but need a correction in the document. > > I want a comparison of MPI_COMM_NULL and MPI_COMM_NULL to give TRUE! > I think of the comparison as being totally equivalent to == > > Your amendment seems to give this property, so I have no objection. > I don't care about performance here, it is clear that == can be used > if the person wants just to check for identity, versus the more > substantial comparisons of MPI_COMM_COMPARE. -Tony 2) How do I set a communicator to the null communicator? Can I just say comm = MPI_COMM_NULL > Yes! ***** I agree ----- End Included Message ----- From owner-mpi-core@CS.UTK.EDU Wed Sep 14 16:26:40 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA14678; Wed, 14 Sep 1994 16:26:40 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA05712; Wed, 14 Sep 1994 16:24:57 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 14 Sep 1994 16:24:57 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id QAA05706; Wed, 14 Sep 1994 16:24:56 -0400 From: Jack Dongarra Received: by dasher.cs.utk.edu (cf v2.9c-UTK) id QAA01654; Wed, 14 Sep 1994 16:24:52 -0400 Date: Wed, 14 Sep 1994 16:24:52 -0400 Message-Id: <199409142024.QAA01654@dasher.cs.utk.edu> To: mpi-core@CS.UTK.EDU Subject: SC94 and MPI We are set for a birds-of-a-feather session at the SC-94 meeting in DC. The meeting is schedule for Wednesday, November 16th between 3:30-5:00, location to be determined. Jack From owner-mpi-core@CS.UTK.EDU Fri Oct 7 16:27:16 1994 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.8t-netlib) id QAA22413; Fri, 7 Oct 1994 16:27:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA15673; Fri, 7 Oct 1994 16:26:12 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 7 Oct 1994 16:26:11 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id QAA15665; Fri, 7 Oct 1994 16:26:08 -0400 From: Jack Dongarra Received: by dasher.cs.utk.edu (cf v2.9c-UTK) id QAA20661; Fri, 7 Oct 1994 16:26:06 -0400 Date: Fri, 7 Oct 1994 16:26:06 -0400 Message-Id: <199410072026.QAA20661@dasher.cs.utk.edu> To: mpi-core@CS.UTK.EDU Subject: sc94 mpi meeting I was just informed that we have been assigned a room to hold 40 people for the MPI meeting at SC94 in November. Needless to say this needs some adjustment. The time and date for the MPI meeting at SC94 will most likely change. I'll keep you posted. Jack From owner-mpi-core@CS.UTK.EDU Wed Jan 18 23:06:36 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA03517; Wed, 18 Jan 1995 23:06:36 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id XAA01083; Wed, 18 Jan 1995 23:05:52 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 18 Jan 1995 23:05:50 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id XAA01076; Wed, 18 Jan 1995 23:05:48 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id WAA02908 for ; Wed, 18 Jan 1995 22:05:39 -0600 Message-Id: <199501190405.WAA02908@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: MPI Forum meeting Date: Wed, 18 Jan 1995 22:05:38 -0600 From: Rusty Lusk Dear Group, At SuperComputing '94 and since then, I have talked with a number of people about having an MPI Forum meeting early this spring. There seems to be a consensus that we should do so, for several reasons: 1. We need to fix some errors and make some decisions about ambiguities discovered in the current Standard document. There is a pretty long list of these in the "Errata" page on the Web. We need to get together, discuss, and vote to get these really settled. 2. We need to decide how to proceed from here. Should there be MPI-2? What should it include? When should it start? Where? 3. Although this would not be intended to be an official launch of MPI-2 but rather a finishing up of MPI-1, we might make some progress on some pressing issues, such as deciding, at least tentatively, on a version of MPI_Spawn and supporting functions. As many of you know, this is perceived as one of the things "missing" in MPI, especially by the workstation-network users. Although I have no official capacity other than MPI enthusiast, I hope that people will not mind if I begin organizing this. I propose that there be an MPI Forum meeting at the O'Hare Hilton in Chicago on Monday-Wednesday, March 13-15, 1995. The place: The O'Hare Hilton is where the HPF Forum met after it got fed up with Dallas. It is only a few steps beyond baggage claim and there would be no need for airport-hotel transportation at all. They have meeting rooms to support our usual style of meeting. I have checked with them and they can reserve them and a block of sleeping rooms for those dates. The best rate on the sleeping rooms I have gotten so far is $102. It looks like the usual $75 registration will cover the meeting rooms, one lunch, and assorted break food. The time: I know that the following week is bad for some people, and the hotel does not have the rooms for the week after that. Earlier might be too soon for some people to arrange their schedule and it might snow. April is getting into the conference season again. So this seems like a reasonable time. Please let me know as soon as possible whether you can come. I would also like to know if there are other people that should be invited (this message is going to mpi-core) and what other topics we should put on the agenda. If we get a critical mass I will go ahead and sign the contract with the hotel. If the dates are really impossible I will try to arrange alternate ones. Regards, Rusty From owner-mpi-core@CS.UTK.EDU Thu Jan 19 01:01:43 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id BAA04683; Thu, 19 Jan 1995 01:01:43 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id BAA08619; Thu, 19 Jan 1995 01:01:11 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 19 Jan 1995 01:01:09 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from noc.usfca.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id BAA08612; Thu, 19 Jan 1995 01:01:07 -0500 Received: from mobydick.usfca.edu (mobydick.usfca.edu [138.202.6.78]) by noc.usfca.edu (8.6.9/8.6.9) with SMTP id WAA13092; Wed, 18 Jan 1995 22:03:19 -0800 Received: by mobydick.usfca.edu (4.1/SMI-4.1) id AA03870; Wed, 18 Jan 95 22:04:22 PST Date: Wed, 18 Jan 95 22:04:22 PST From: peter@mobydick.usfca.edu (Peter Pacheco) Message-Id: <9501190604.AA03870@mobydick.usfca.edu> To: lusk@mcs.anl.gov, mpi-core@CS.UTK.EDU Subject: Re: MPI Forum meeting I just looked at my calendar. Isn't March 15 the first day of SIAM's Parallel Processing for Scientific Computing? If so, we might want to reschedule -- or else meet in San Francisco ;-) Peter Pacheco From owner-mpi-core@CS.UTK.EDU Thu Jan 19 07:39:25 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA12011; Thu, 19 Jan 1995 07:39:25 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id HAA13207; Thu, 19 Jan 1995 07:38:52 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 19 Jan 1995 07:38:50 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id HAA13201; Thu, 19 Jan 1995 07:38:48 -0500 From: Jack Dongarra Received: by dasher.cs.utk.edu (cf v2.9c-UTK) id HAA08715; Thu, 19 Jan 1995 07:39:04 -0500 Message-Id: <199501191239.HAA08715@dasher.cs.utk.edu> Subject: Re: MPI Forum meeting To: peter@mobydick.usfca.edu (Peter Pacheco) Date: Thu, 19 Jan 1995 07:39:03 -0500 (EST) Cc: lusk@mcs.anl.gov, mpi-core@CS.UTK.EDU In-Reply-To: <9501190604.AA03870@mobydick.usfca.edu> from "Peter Pacheco" at Jan 18, 95 10:04:22 pm X-Mailer: ELM [version 2.4 PL23] Content-Type: text > I just looked at my calendar. Isn't March 15 the first day of > SIAM's Parallel Processing for Scientific Computing? If so, we might > want to reschedule -- or else meet in San Francisco ;-) > > Peter Pacheco > I believe the SIAM meeting is 2/15-17. Jack From owner-mpi-core@CS.UTK.EDU Fri Jan 20 15:46:46 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA02526; Fri, 20 Jan 1995 15:46:46 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA20038; Fri, 20 Jan 1995 15:45:40 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 20 Jan 1995 15:45:38 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA20002; Fri, 20 Jan 1995 15:45:34 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id OAA14799 for ; Fri, 20 Jan 1995 14:45:21 -0600 Message-Id: <199501202045.OAA14799@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: MPI Forum meeting, March 13-15 Date: Fri, 20 Jan 1995 14:45:20 -0600 From: Rusty Lusk Thanks to those who have already responded, it looks like we now have critical mass, so the dates for the meeting can be nailed down. I now need to get a better idea of how many people will be coming, in order to sign a contract with the hotel. So could you please tell me by Monday noon whether you can come to the MPI Forum meeting at the O'Hare Hilton in Chicago from 1:30 on Monday, March 13 to 12:00, Wednesday, March 15? Reservation numbers for the hotel, etc. will follow. Please contact me if you have any questions. Regards, Rusty From owner-mpi-core@CS.UTK.EDU Wed Feb 1 15:43:02 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA10181; Wed, 1 Feb 1995 15:43:02 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA01480; Wed, 1 Feb 1995 15:42:00 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 1 Feb 1995 15:41:59 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA01473; Wed, 1 Feb 1995 15:41:57 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id OAA01445; Wed, 1 Feb 1995 14:41:35 -0600 Message-Id: <199502012041.OAA01445@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Cc: lusk@antares.mcs.anl.gov Subject: MPI Forum Meeting, March 13-15 Date: Wed, 01 Feb 1995 14:41:34 -0600 From: Rusty Lusk The plans for an MPI Forum meeting on March 13-15 are now complete. It looks like most of the group will be there. Here are some details. The O'Hare Hilton is the hotel *inside* the airport. You just walk across the street from the baggage-claim level. A block of rooms has been reserved, which will be held until March 1, so please make your reservations with the Hotel before then. The phone number for the hotel is (312) 686-8000 (FAX (312) 6601-2876. Mention the *MPI Forum* to get the special rate, which is $90.50 + tax per night. We will charge the usual $75 registration to take care of the meeting room, lunch on Tuesday, and coffee, etc. The meeting will start at 1:30 on Monday to allow people to fly in that morning, and end at noon on Wednesday. If you have any questions, don't hesitate to send me mail at lusk@mcs.anl.gov or call me at (708) 252-7852. Regards, Rusty From owner-mpi-core@CS.UTK.EDU Wed Feb 8 12:02:33 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA04926; Wed, 8 Feb 1995 12:02:33 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA20710; Wed, 8 Feb 1995 12:02:05 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 8 Feb 1995 12:02:04 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA20688; Wed, 8 Feb 1995 12:01:40 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA21509 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Wed, 8 Feb 1995 17:01:18 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA09267; Wed, 8 Feb 1995 17:01:12 +0000 Date: Wed, 8 Feb 1995 17:01:12 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9502081701.AA09267@tycho.co.uk> To: lusk@mcs.anl.gov Cc: mpi-core@CS.UTK.EDU In-Reply-To: <199502012041.OAA01445@antares.mcs.anl.gov> (message from Rusty Lusk on Wed, 01 Feb 1995 14:41:34 -0600) Subject: Re: MPI Forum Meeting, March 13-15 Reply-To: jim@meiko.com Rusty said : > The phone number for the O'Hare Hilton is (312) 686-8000 > FAX (312) 6601-2876. Bitter experience shows that the FAX number as given is WRONG. It should read FAX (312) 601 2873 Make your bookings now ! -- Jim From owner-mpi-core@CS.UTK.EDU Thu Feb 16 06:59:38 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA23066; Thu, 16 Feb 1995 06:59:38 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id HAA02019; Thu, 16 Feb 1995 07:00:14 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 16 Feb 1995 07:00:11 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id HAA02000; Thu, 16 Feb 1995 07:00:07 -0500 Received: from tycho.co.uk (tycho.meiko.co.uk) by hub with SMTP id AA02691 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Thu, 16 Feb 1995 11:59:57 GMT Received: by tycho.co.uk (5.0/SMI-SVR4) id AA01503; Thu, 16 Feb 1995 11:59:55 +0000 Date: Thu, 16 Feb 1995 11:59:55 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9502161159.AA01503@tycho.co.uk> To: mpi-core@CS.UTK.EDU Subject: Comp.parallel In case any of you missed it, this was a nice post in comp.parallel. It look as if someone appreciates our efforts at least ! -- Jim From Brian Wainscott Newsgroups: comp.parallel Subject: Re: [Q] which will be standard MPI or share memory Date: Mon, 13 Feb 1995 17:42:53 GMT In message <3h69ei$bc5@news.cais.com> - duffy@cais2.cais.com (Duffy Men) writes : >A IBM engineer told me in the future the MPI (Message passing interface) >will be the standard in parallel programming. Does anyone has option on >it? Thank you. We have a large application running on several machines: Paragon, SP2, T3D, workstation networks, and other ports in the works. I've seen/used NX (Intel), EUIH (IBM), PVM, and others. In my opinion (you asked for opinions!), MPI is significantly better than all of the above. In fact, it has a few features that I had to supply myself on the other machines. (reduction on a subgroup of the processors -- using MPI communicators it is trivial). It is also the first widly used specification (the first I ever heard of anyway) that will allow the development of good third party libraries, the coupling of disparate codes, etc. This is all possible because the communicators are insulated from each other, so building up pieces of large programs can be done without message passing conflicts. In fact, our code (a large Finite Element Analysis code, about 300K lines last time I checked) is now written in MPI, and runs on each of the above machines via translation libraries I wrote (MPI->NX, MPI->EUIH, etc.)Of course the libraries only supply those features I need. We are using MPICH on the workstations. I'm greatly looking forward to true native MPI support on a wide range of hardware. I expect it is comming, because I also believe that MPI will be the standard. At least it SHOULD be. -------------------------------------------------------------- Brian Wainscott brian@lstc.com From owner-mpi-core@CS.UTK.EDU Thu Feb 23 11:32:12 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA01777; Thu, 23 Feb 1995 11:32:12 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA29800; Thu, 23 Feb 1995 11:32:36 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 23 Feb 1995 11:32:30 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA29774; Thu, 23 Feb 1995 11:32:25 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.4/8.6.4) with ESMTP id KAA20449; Thu, 23 Feb 1995 10:32:17 -0600 Message-Id: <199502231632.KAA20449@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov, srwheat@ssd.intel.com Subject: MPI Forum Meeting, March 13-15 Date: Thu, 23 Feb 1995 10:32:14 -0600 From: Rusty Lusk Be sure to make your hotel reservations for the MPI Forum meeting. The deadline for getting the good rate is March 1, which is Wednesday. Many more people have told me they are coming than have made their reservations at this point. At first there was some problem with them recognizing the "MPI Forum" name, but I don't think that will happen again, not after the "discussion" I had with them last week. Please make the $75 checks payable to Argonne National Laboratory. Here is the information I sent out before (with the FAX number corrected): The O'Hare Hilton is the hotel *inside* the airport. You just walk across the street from the baggage-claim level. A block of rooms has been reserved, which will be held until March 1, so please make your reservations with the Hotel before then. The phone number for the hotel is (312) 686-8000 (FAX (312) 601-2876). Mention the *MPI Forum* to get the special rate, which is $90.50 + tax per night. We will charge the usual $75 registration to take care of the meeting room, lunch on Tuesday, and coffee, etc. The meeting will start at 1:30 on Monday to allow people to fly in that morning, and end at noon on Wednesday. If you have any questions, don't hesitate to send me mail at lusk@mcs.anl.gov or call me at (708) 252-7852. I will try to put together a trial agenda, or at least a topic list, later today. There will plenty to do. Regards, Rusty From owner-mpi-core@CS.UTK.EDU Fri Feb 24 18:20:48 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA17662; Fri, 24 Feb 1995 18:20:47 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id SAA27036; Fri, 24 Feb 1995 18:21:49 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 24 Feb 1995 18:21:48 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id SAA27029; Fri, 24 Feb 1995 18:21:46 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA19755; Fri, 24 Feb 1995 17:21:39 -0600 Message-Id: <199502242321.RAA19755@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: Topics for MPI Forum meeting Cc: lusk@antares.mcs.anl.gov Date: Fri, 24 Feb 1995 17:21:38 -0600 From: Rusty Lusk We should begin thinking about this meeting. My understanding is that it primarily to fill in lacunae in the current draft. That is why the meeting has not been widely advertised; we will be taking a fair number of votes on issues that are familiar to the mpi-core mailing list, and new people can't vote. A secondary purpose is to decide on what should happen next, and if there is time we might actually consider some new business. I am certainly open to any other opinions and suggestions. Here are some thoughts on what our topics might be. Please offer comments and suggestions; It would be nice to have at least a tentative agenda before we convene. 1. Assessment of what is happening with MPI a. publications b. presentations c. tutorials d. implementations e. endorsements 2. The current MPI Standard (the meat of the meeting) a. Ratify changes already made (in IJSA version) b. Debate issues described in errata and resolve them. (See http://www.mcs.anl.gov/mpi for the current draft and the errata.) c. Other issues relating to the current scope of MPI (please make suggestions.) 3. Brief, open discussion of possible future topics a. dynamic process management b. client/server c. parallel I/O d. interrupt-driven receive e. active messages f. remote put/get operations g. standardized user startup mechanism (mpirun) h. non-blocking collective operations i. collective operations on inter-communicators j. extended timer routines k. more topology functions (more topologies) l. debugging hooks (expose state of system, using portable abstraction) m. others 4. Should there be an MPI-2? a. Initial scope (actual participants will determine eventual scope) b. Participants, presumably open as before c. Should current MPI be an official subset of MPI-2? d. Location e. Schedule f. "Organizer" 5. If time permits, we could hear tentative proposals on any of the possible future topics. I know of three focused thought-out-and-written-up ones, a. IBM-NASA: parallel I/O b. ANL: dynamic processes and client-server c. MSU: inter-communicator operations and several others that make proposals in multiple areas: d. IBM: spawn, put/get, user-defined requests, others e. MSU: nonblocking collective operations, others f. ANL-NASA-Ohio mpirun (all different) Anyone else who has worked out useful MPI extensions is certainly welcome to bring them; the above are just the ones I know about. But again, the main business of this meeting is to finish MPI-1, and only to (possibly) make plans for MPI-2. Comments? Suggestions? Please make your hotel reservations this weekend: O'Hare Hilton (312) 686-8000. Rusty From owner-mpi-core@CS.UTK.EDU Fri Mar 10 09:46:36 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA03430; Fri, 10 Mar 1995 09:46:35 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA18047; Fri, 10 Mar 1995 09:45:58 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 10 Mar 1995 09:45:54 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA18025; Fri, 10 Mar 1995 09:45:47 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9555; Fri, 10 Mar 95 09:45:42 EST Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 0623; Fri, 10 Mar 1995 09:45:34 EST Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 10 Mar 95 09:45:26 EST Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA27106; Fri, 10 Mar 1995 09:45:24 -0500 Message-Id: <9503101445.AA27106@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: discussion items for meeting next week Reply-To: snir@watson.ibm.com Date: Fri, 10 Mar 1995 09:45:22 -0500 From: Marc Snir :-) :-) :-) *** (-: (-: (-: The postscript document below has two parts 1. Corrections to the current manual. Pls read carefully, as we shall need to decide on these corrections next week. 2. Suggestions for extensions. This is not a complete list of all extensions I would propose to consider, nor is it prioritized. -------------------------------------------------------- %!PS-Adobe-2.0 %%Creator: dvips 5.47 (RS/6000 1.0) Copyright 1986-91 Radical Eye Software %%Title: /u/snir/mpi2v09.dvi %%Pages: 12 1 %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N /def load def /B{bind def}N /S /exch load def /X{S N}B /TR /translate load N /isls false N /vsize 10 N /@rigin{ isls{[0 1 -1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale Resolution VResolution vsize neg mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@letter{/vsize 10 N}B /@landscape{/isls true N /vsize -1 N}B /@a4{/vsize 10.6929133858 N}B /@a3{ /vsize 15.5531 N}B /@ledger{/vsize 16 N}B /@legal{/vsize 13 N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder} N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail} B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get} B /ch-xoff{128 ch-data dup length 3 sub get sub} B /ch-yoff{ ch-data dup length 2 sub get 127 sub} B /ch-dx{ch-data dup length 1 sub get} B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image} imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto}N /eop{clear SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put} for}N /p /show load N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval(Display)eq}{pop false}ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /a{ moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{ S p tail}B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w }B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{clear SS restore}B end %%EndProcSet TeXDict begin 1000 300 300 @start /Fa 12 122 df<0FFFFFFC1FFFFFF81E03C0781803C0 383803C01830078018300780186007801860078030C00F0030C00F0030000F0000000F0000001E 0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000007800000078000000 7800000078000000F0000000F0000000F0000001F000007FFFC0007FFFC0001E1F7F9E1B>84 D<00F18003FDC0078F800E07801C07803C07803C0700780700780700780700F00E00F00E00F00E 00F00E18F01C30F03C30707C3078FC603FCFC00F078015147E9318>97 D<07803F803F00070007 0007000E000E000E000E001C001C001CF01FFC3F1E3E0E3C0F380F700F700F700F700FE01EE01E E01EE03CE03CE038607071E03FC01F0010207E9F14>I<007C01FE07870E031E033C033C06781C 7FF87FC0F000F000F000F000700070027007381E1FF807E010147E9315>101 D<03C01FC01F8003800380038007000700070007000E000E000E000E001C001C001C001C003800 3800380038007000700070007180E300E300E300E6007E003C000A207E9F0E>108 D<0F03E03E001F8FF0FF0031DC39C38061F03B038061E01E01C061C01C01C0C3C03C0380038038 0380038038038003803803800700700700070070070007007007000700700E0C0E00E00E180E00 E01C180E00E01C300E00E00C701C01C00FE00C00C0078026147F9328>I<0F07C0001F8FE00031 D8700061F0380061E0380061C03800C3C070000380700003807000038070000700E0000700E000 0700E0000701C1800E01C3000E0383000E0386000E018E001C01FC000C00F00019147F931B>I< 03C1E007E7F80C7E3C187C1C18781E18701E30E01E00E01E00E01E00E01E01C03C01C03C01C03C 01C07803C07803C07003C0E003E3C0077F80071E000700000700000E00000E00000E00000E0000 1C0000FFC000FFC000171D819317>112 D<00C000E001C001C001C001C003800380FFF8FFF807 000700070007000E000E000E000E001C001C001C001C1838303830386038E01FC00F000D1C7F9B 10>116 D<0780301FC03838C07030E07060E07061C070C1C0E00380E00380E00380E00701C007 01C00701C00701C3070386070386070786070F8C03F9F801F0F018147F931A>I<07C3C00FE7E0 187C703038706038F06038F0C070E000700000700000700000E00000E00000E00070E030F1C060 F1C060E1C0C0E3E1807E7F003C3E0014147E931A>120 D<0780301FC03838C07030E07060E070 61C070C1C0E00380E00380E00380E00701C00701C00701C00701C0070380070380070780070F80 03FF0001F700000700000700000E003C0C003C1C0038380030F0001FE0000F8000151D7F9316> I E /Fb 14 122 df<00F18003FDC0078F800E07801C07803C07803C0700780700780700780700 F00E00F00E00F00E00F00E30F01C60F03C60707C6078FCC03FCFC00F078014147C9317>97 D<0000780003F80003F00000700000700000700000E00000E00000E00000E00001C00001C000F1 C003FDC0078F800E07801C07803C07803C0700780700780700780700F00E00F00E00F00E00F00E 30F01C60F03C60707C6078FCC03FCFC00F078015207C9F17>100 D<007C0001FF000783000F01 801E01803C01803C0300780E007FFC007FE000F00000F00000F00000F000007000007002007807 003C1E001FF80007E00011147C9315>I<006000F000F000E00000000000000000000000000000 0F001F80318031C063806380C3800700070007000E000E000E001C301C601C6038C018C01F800F 000C1F7D9E0E>105 D<03C01FC01F8003800380038007000700070007000E000E000E000E001C 001C001C001C0038003800380038007000700070007180E300E300E300E6007E003C000A207C9F 0C>108 D<1E07C0F8003F1FE1FC0033B8730E0063E076070063C03C07006380380700C780780E 000700700E000700700E000700700E000E00E01C000E00E01C000E00E01C000E00E038601C01C0 38C01C01C038C01C01C071801C01C031803803803F001801801E0023147D9325>I<1E07C03F1F E033B87063E07063C038638038C780700700700700700700700E00E00E00E00E00E00E01C31C01 C61C01C61C038C1C018C3801F81800F018147D931A>I<007C0001FF000383800F01C01E01C01C 01E03C01E07801E07801E07801E0F003C0F003C0F003C0F00780F00700700F00701E003838001F F00007C00013147C9317>I<03C1E007E7F8067E3C0C7C1C0C781E0C701E18E01E00E01E00E01E 00E01E01C03C01C03C01C03C01C07803C07803C07003C0E003E3C0077F80071E00070000070000 0E00000E00000E00000E00001C0000FFC000FFC000171D809317>I<1E0F003F3F8033F1C063C1 C063C3C06383C0C783800700000700000700000E00000E00000E00000E00001C00001C00001C00 001C000038000018000012147D9313>114 D<00FC03FE07070E0F0E0F0E0E1E000F800FF007F8 03FC003E001E701EF01CF01CE03860703FE01F8010147D9313>I<018001C00380038003800380 07000700FFF0FFF00E000E000E000E001C001C001C001C0038003800380038307060706070C071 803F001E000C1C7C9B0F>I<0F00601F80703180E031C0E06380E06380E0C381C00701C00701C0 0701C00E03800E03800E03800E038C0E07180E07180E0F180E1F3007F3F003E1E016147D9318> I<0F00601F80703180E031C0E06380E06380E0C381C00701C00701C00701C00E03800E03800E03 800E03800E07000E07000E0F000E1F0007FE0003EE00000E00000E00001C00781C007838007070 0060E0003FC0001F0000141D7D9316>121 D E /Fc 6 104 df0 D<07E01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF807E0 10127D9317>15 D<0000000600000000060000000003000000000180000000018000000000C000 00000070FFFFFFFFFCFFFFFFFFF8000000007000000000C0000000018000000001800000000300 0000000600000000060026107D922D>33 D<00000C00000C00001C000018000038000030000070 0000600000E00000C00001C0000180000380000300000700000600000E00000C00001C00001800 00380000300000300000700000600000E00000C00001C000018000038000030000070000060000 0E00000C00001C0000180000380000300000700000600000E00000C00000C00000162C7AA000> 54 D<001F003F00F800E001C001C001C001C001C001C001C001C001C001C001C001C001C001C0 01C003800700FE00F800FE000700038001C001C001C001C001C001C001C001C001C001C001C001 C001C001C001C000E000F8003F001F102D7DA117>102 DI E /Fd 72 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F8 70051C779B18>33 D<03C00007E0000FF0001E78001C38001C38001C38001C38001C7BF01CF3F0 1EE3F00FE7800FC7000F87000F0F001F0E003F8E007B9E0073DC00F1DC00E1FC00E0F800E07870 F0F87071FE707FFFF03FCFE01F03C0141C7F9B18>38 D<387C7E7E3E0E0E0E1E3C78F8F0C0070E 789B18>I<007000F001F003C007800F001E001C003C003800780070007000F000E000E000E000 E000E000E000E000E000F00070007000780038003C001C001E000F00078003C001F000F000700C 24799F18>I<6000F00078003C001E000F000780038003C001C001E000E000E000F00070007000 700070007000700070007000F000E000E001E001C003C0038007800F001E003C007800F0006000 0C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007F00007F0 001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D9718>I<00600000 F00000F00000F00000F00000F00000F00000F0007FFFC0FFFFE0FFFFE07FFFC000F00000F00000 F00000F00000F00000F00000F00000600013147E9718>I<3C7E7F7F7F3F0F0E1E7CF870080C78 8518>I<78FCFCFCFC780606778518>46 D<000300000780000F80000F80000F00001F00001F00 003E00003E00003C00007C00007C0000F80000F80000F00001F00001F00003E00003E00007C000 07C0000780000F80000F80001F00001F00001E00003E00003E00007C00007C0000780000F80000 F80000F0000060000011247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001 C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C07001 C07803C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>I<01800380078007800F80 3F80FF80FB80638003800380038003800380038003800380038003800380038003800380038003 807FFCFFFE7FFC0F1C7B9B18>I<07F8001FFE003FFF007C0F807003C0F001E0F000E0F000E0F0 00E00000E00000E00001E00001C00003C0000780000F00001E00003C0000780000F00003E00007 C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<78FCFCFCFC78000000000000 000078FCFCFCFC780614779318>58 D<3C7E7E7E7E3C0000000000000000387C7E7E7E3E0E1E3C 78F060071A789318>I<000300000F80001F80003F0000FE0001FC0003F00007E0001FC0003F80 007E0000FC0000FC00007E00003F80001FC00007E00003F00001FC0000FE00003F00001F80000F 8000030011187D9918>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FF FFE0FFFFE07FFFC0130C7E9318>I<0FF8003FFE007FFF00F80F80F00380F00380F00780001F00 003E0000FC0001F00001E00003C00003C00003C00003C00003C000038000000000000000000000 00000000000003800007C00007C00007C000038000111C7D9B18>63 D<00700000F80000F80000 D80000D80001DC0001DC0001DC00018C00038E00038E00038E00038E0003060007070007070007 07000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B 18>65 DI<01FCE003FEE007FFE00F07E01E03E03C01E07800E070 00E07000E0F00000E00000E00000E00000E00000E00000E00000E00000E00000F000007000E070 00E07800E03C01E01E01C00F07C007FF8003FF0001FC00131C7E9B18>I<7FF800FFFE007FFF00 1C0F801C03C01C03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C00701C0070 1C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F 9B18>III<01F9C007FFC00FFFC0 1F0FC01C03C03C03C07801C07001C07001C0F00000E00000E00000E00000E00000E00000E00FF0 E01FF0E00FF0F001C07001C07003C07803C03C03C01C07C01F0FC00FFFC007FDC001F9C0141C7E 9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01 C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01 C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7FE000FFE0007FE000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F 9B18>76 DI<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C0 1CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C0 1C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF 00780F00700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C 7D9B18>II<0FF8003FFE007FFF00780F00700700F00780E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00001C00001E00000E00000F00000700 00070011227D9B18>I<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03 C01C03801C0F801FFF001FFE001FFE001C0F001C07801C03801C03801C03801C03801C03801C03 9C1C039C1C039C7F03FCFF81F87F00F0161C7F9B18>I<07F3801FFF803FFF807C1F80700780F0 0380E00380E00380E00000F000007800003F00001FF0000FFE0001FF00001F800003C00001E000 00E00000E06000E0E000E0E001E0F001C0FC07C0FFFF80FFFF00E7FC00131C7E9B18>I<7FFFF8 FFFFF8FFFFF8E07038E07038E07038E07038007000007000007000007000007000007000007000 00700000700000700000700000700000700000700000700000700000700000700007FF0007FF00 07FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F8 0001F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F000707000E07 800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II91 D93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<061E3E3C78F0E0E0E0F8FCFC7C38070E 789E18>I<1FE0003FF8007FFC00783E00300F0000070000070001FF000FFF003FFF007F070078 0700F00700E00700E00700F00F00783F007FFFF03FFBF00FE1F014147D9318>I<7E0000FE0000 7E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FE3E00F80F00F00700F0078 0E00380E00380E00380E00380E00380F00380F00780F00700F80F00FC3E00FFFC00EFF80067E00 151C809B18>I<01FE0007FF001FFF803F07803C0300780000700000F00000E00000E00000E000 00E00000F000007000007801C03C01C03F07C01FFF8007FF0001FC0012147D9318>I<001F8000 3F80001F8000038000038000038000038000038003F3800FFB801FFF803E1F80780F80700780F0 0780E00380E00380E00380E00380E00380E00780F00780700780780F803E3F801FFFF00FFBF803 E3F0151C7E9B18>I<03F0000FFC001FFE003E1F00780780700380F003C0E001C0E001C0FFFFC0 FFFFC0FFFFC0F000007000007801C03C01C03F07C01FFF8007FF0001FC0012147D9318>I<001F C0007FE000FFE001F1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007FFF 007FFF00131C7F9B18>I<03F1F007FFF80FFFF81E1F303C0F0038070038070038070038070038 07003C0F001E1E001FFC003FF8003BF0003800003C00001FFF001FFFC03FFFE07801F0F00078E0 0038E00038E00038F000787800F07E03F03FFFE00FFF8003FE00151F7F9318>I<7E0000FE0000 7E00000E00000E00000E00000E00000E00000E3F000EFF800FFFC00FE1E00F80E00F00E00F00E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC 171C809B18>I<03800007C00007C00007C0000380000000000000000000000000007FC000FFC0 007FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0007FFF C0FFFFE07FFFC0131C7E9B18>I<7DF1F000FFFBF8007FFFFC001F1F1C001E1E1C001E1E1C001C 1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C007F1F1F00FFBFBF807F1F1F001914819318>I<7E3F00FEFF807FFFC00FE1E00F80E00F 00E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFF E7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E000E0E000E0 E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318> I<7E3E00FEFF807FFFC00FE3E00F80F00F00700F00780E00380E00380E00380E00380E00380F00 380F00780F00700F80F00FC3E00FFFC00EFF800E7E000E00000E00000E00000E00000E00000E00 000E00007FC000FFE0007FC000151E809318>I<03F3800FFB801FFF803E1F80780F80700780F0 0780E00380E00380E00380E00380E00380E00380F00780700780780F803E1F801FFF800FFB8003 F380000380000380000380000380000380000380000380003FF8003FF8003FF8151E7E9318>I< 7F87E0FF9FF87FBFF803FC7803F03003E00003C00003C00003C000038000038000038000038000 0380000380000380000380007FFE00FFFF007FFE0015147F9318>I<0FF7003FFF007FFF00F81F 00E00700E00700F007007C00007FF0001FFC0007FE00001F00600780E00380F00380F00780FC0F 00FFFF00FFFE00E7F80011147D9318>I<0180000380000380000380000380007FFFC0FFFFC0FF FFC00380000380000380000380000380000380000380000380000380000380400380E00380E003 81E003C3C001FFC000FF80007E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F07E007FFFC03FFFE 01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E03800707000707000707 00038E00038E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700 078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E000707000F0780 7F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03800703800707 00070700038700038600038E0001CE0001CE0000CC0000CC0000DC000078000078000078000070 0000700000700000F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<0007E000 3FE0007FE000FC0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00001E0007FC000FF8000FF80007FC00001E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000FC00007FE0003FE00007E013247E9F18>123 D<60F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0600424 769F18>I<7C0000FF8000FFC00007E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000F000007FC0003FE0003FE0007FC000F00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00007E000FFC000FF80007C000013247E9F 18>I E /Fe 49 124 df<000F800000003FC00000007860000000F060000001F030000001E030 000003E030000003E030000003E060000003F060000003F0C0000003F180000003F380000003F7 001FFC01FE001FFC01FC00018001FC00018000FC00030000FE00060003FE000600077F000C000E 3F8018001E3F8018003C1FC030007C0FE06000FC0FF0C000FC07F8C000FC03FD8000FC01FF0000 FE00FF000C7E007F800C3F007FC0181F81E7F0780FFF81FFE001FC003F8026237EA22C>38 D45 D<387CFEFEFE7C3807077C8610>I<001800007800 01F800FFF800FFF80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F800 01F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F800 01F80001F8007FFFE07FFFE013207C9F1C>49 D<03FC000FFF003C1FC07007E07C07F0FE03F0FE 03F8FE03F8FE01F87C01F83803F80003F80003F00003F00007E00007C0000F80001F00003E0000 380000700000E01801C0180380180700180E00380FFFF01FFFF03FFFF07FFFF0FFFFF0FFFFF015 207D9F1C>I<01FE0007FFC00F07E01E03F03F03F03F01F83F81F83F01F81F03F80C03F00003F0 0003E00007C0001F8001FE0001FF000007C00001F00001F80000FC0000FC0000FE7C00FE7C00FE FE00FEFE00FEFE00FC7C01FC7801F83C07F00FFFC003FE0017207E9F1C>I<0000E00001E00003 E00003E00007E0000FE0001FE0001FE00037E00077E000E7E001C7E00187E00307E00707E00E07 E00C07E01807E03807E07007E0E007E0FFFFFEFFFFFE0007E00007E00007E00007E00007E00007 E00007E000FFFE00FFFE17207E9F1C>I<1000201E01E01FFFC01FFF801FFF001FFE001FF8001B C00018000018000018000018000019FC001FFF001E0FC01807E01803E00003F00003F00003F800 03F83803F87C03F8FC03F8FC03F8FC03F0F803F06007E03007C01C1F800FFF0003F80015207D9F 1C>I<001F8000FFE003F07007C0F00F01F81F01F83E01F83E01F87E00F07C00007C0000FC0800 FC7FC0FCFFE0FD80F0FF00F8FE007CFE007CFC007EFC007EFC007EFC007E7C007E7C007E7C007E 3C007C3E007C1E00F80F00F00783E003FFC000FF0017207E9F1C>I<6000007800007FFFFE7FFF FE7FFFFC7FFFF87FFFF87FFFF0E00060E000C0C00180C00300C00300000600000C00001C000018 0000380000780000780000F00000F00000F00001F00001F00001F00003F00003F00003F00003F0 0003F00003F00003F00001E00017227DA11C>I<00FE0003FFC00703E00E00F01C00F01C00783C 00783E00783F00783F80783FE0F01FF9E01FFFC00FFF8007FFC003FFE007FFF01E7FF83C1FFC78 07FC7801FEF000FEF0003EF0001EF0001EF0001CF8001C7800383C00381F01F00FFFC001FF0017 207E9F1C>I<01FE0007FF800F83E01E01F03E00F07C00F87C0078FC007CFC007CFC007CFC007E FC007EFC007EFC007E7C00FE7C00FE3E01FE1E037E0FFE7E07FC7E00207E00007C00007C1E007C 3F00F83F00F83F00F03F01E01E03C01C0F800FFE0003F80017207E9F1C>I66 D<0003FE0080001FFF818000FF01E38001F8003F8003E0001F80 07C0000F800F800007801F800007803F000003803F000003807F000001807E000001807E000001 80FE00000000FE00000000FE00000000FE00000000FE00000000FE00000000FE00000000FE0000 00007E000000007E000001807F000001803F000001803F000003801F800003000F8000030007C0 00060003F0000C0001F800380000FF00F000001FFFC0000003FE000021227DA128>III<0003FE0040001FFFC0C0007F00F1C001F8003FC003F0000FC007C00007C00FC00003C0 1F800003C03F000001C03F000001C07F000000C07E000000C07E000000C0FE00000000FE000000 00FE00000000FE00000000FE00000000FE00000000FE00000000FE000FFFFC7E000FFFFC7F0000 1FC07F00001FC03F00001FC03F00001FC01F80001FC00FC0001FC007E0001FC003F0001FC001FC 003FC0007F80E7C0001FFFC3C00003FF00C026227DA12C>71 DII< FFFFE000FFFFE00007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F000 0007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0 000007F0000007F0001807F0001807F0001807F0001807F0003807F0003807F0007007F0007007 F000F007F001F007F007F0FFFFFFF0FFFFFFF01D227EA122>76 DII80 D82 D<01FC0407FF8C1F03FC3C007C7C003C78001C78001CF8000CF8000CFC000CFC0000FF0000FFE0 007FFF007FFFC03FFFF01FFFF80FFFFC03FFFE003FFE0003FF00007F00003F00003FC0001FC000 1FC0001FE0001EE0001EF0003CFC003CFF00F8C7FFE080FF8018227DA11F>I<7FFFFFFF807FFF FFFF807E03F80F807803F807807003F803806003F80180E003F801C0E003F801C0C003F800C0C0 03F800C0C003F800C0C003F800C00003F800000003F800000003F800000003F800000003F80000 0003F800000003F800000003F800000003F800000003F800000003F800000003F800000003F800 000003F800000003F800000003F800000003F800000003F800000003F800000003F8000003FFFF F80003FFFFF80022227EA127>II<07FC001F FF803F0FC03F07E03F03E03F03F01E03F00003F00003F000FFF007FFF01FC3F03F03F07E03F0FC 03F0FC03F0FC03F0FC03F07E07F07E1DF81FF8FF07E07F18167E951B>97 DI<00FF8007FFE00F83 F01F03F03E03F07E03F07C01E07C0000FC0000FC0000FC0000FC0000FC0000FC00007C00007E00 007E00003E00301F00600FC0E007FF8000FE0014167E9519>I<0003FE000003FE0000007E0000 007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E00 01FC7E0007FF7E000F81FE001F00FE003E007E007E007E007C007E00FC007E00FC007E00FC007E 00FC007E00FC007E00FC007E00FC007E00FC007E007C007E007C007E003E007E001E00FE000F83 FE0007FF7FC001FC7FC01A237EA21F>I<00FE0007FF800F87C01E01E03E01F07C00F07C00F8FC 00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00007E00003E00181F00300FC07003 FFC000FF0015167E951A>I<01FE1F0007FFFF800F87E7801F03E7801E01E0003E01F0003E01F0 003E01F0003E01F0003E01F0001E01E0001F03E0000F87C0000FFF800019FE0000180000001C00 00001C0000001FFFE0001FFFF8000FFFFE000FFFFF003FFFFF007C003F80F8001F80F8000F80F8 000F80F8000F807C001F007E003F001F80FC000FFFF80001FFC00019217F951C>103 DI<0E001F003F803F80 3F801F000E000000000000000000000000000000FF80FF801F801F801F801F801F801F801F801F 801F801F801F801F801F801F801F801F801F801F80FFF0FFF00C247FA30F>I107 DIII<00FE0007FFC00F83E01E00F03E00F8 7C007C7C007C7C007CFC007EFC007EFC007EFC007EFC007EFC007EFC007E7C007C7C007C3E00F8 1F01F00F83E007FFC000FE0017167E951C>II114 D<07F3001FFF00780F00700700F00300F00300F80000FF0000FFF0007FFC003FFE001FFF0007FF 00003F80C00F80C00780E00780E00780F00700FC1E00EFFC00C7F00011167E9516>I<00C00000 C00000C00000C00001C00001C00003C00007C0000FC0001FC000FFFF00FFFF000FC0000FC0000F C0000FC0000FC0000FC0000FC0000FC0000FC0000FC0000FC0000FC1800FC1800FC1800FC1800F C18007C18007E30003FE0000FC0011207F9F16>III120 DI123 D E /Ff 39 122 df<001F000000007FC0000000F0C0000001F0E0000001E060000003E0600000 03E060000003E0E0000003E0C0000003E1C0000003E380000003E700000003FE007FE001FC007F E001F8000E0001F8000E0001FC001C0003FC003800077E0030000E7E0070001E3F00E0007C3F80 C0007C1FC1C000FC0FE38000FC07F70000FC03FE0000FC01FC0000FE00FC00607E00FF00E03F07 DF81C00FFF07FF8003F800FE0023207E9F29>38 D45 D<387CFEFEFE7C38000000000000387CFEFEFE7C3807147C930F>58 D<0000E000000000E00000 0001F000000001F000000001F000000003F800000003F800000007FC00000007FC0000000FFE00 00000CFE0000000CFE000000187F000000187F000000307F800000303F800000703FC00000601F C00000601FC00000C01FE00000C00FE00001FFFFF00001FFFFF000018007F000030003F8000300 03F800060003FC00060001FC000E0001FE00FFE01FFFE0FFE01FFFE0231F7E9E28>65 DI<0007FC02003FFF0E00FE03DE03F000FE07E0003E0FC0001E 1F80001E3F00000E3F00000E7F0000067E0000067E000006FE000000FE000000FE000000FE0000 00FE000000FE000000FE0000007E0000007E0000067F0000063F0000063F00000C1F80000C0FC0 001807E0003803F0007000FE01C0003FFF800007FC001F1F7D9E26>I70 D<0007FC0200003FFF0E0000FE03DE0003F000FE0007E0003E000FC0001E001F80001E 003F00000E003F00000E007F000006007E000006007E00000600FE00000000FE00000000FE0000 0000FE00000000FE00000000FE007FFFE0FE007FFFE07E0000FE007E0000FE007F0000FE003F00 00FE003F0000FE001F8000FE000FC000FE0007E000FE0003F001FE0000FE03BE00003FFF1E0000 07FC0600231F7D9E29>I76 DII<001FF80000FFFF0001 F81F8007E007E00FC003F01F8001F81F0000F83F0000FC7F0000FE7E00007E7E00007EFE00007F FE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007F7E00007E7F0000 FE7F0000FE3F0000FC3F8001FC1F8001F80FC003F007E007E001F81F8000FFFF00001FF800201F 7D9E27>I82 D<03FC080FFF381E03F83C00F8780078780038F80038F80018FC0018 FC0000FF0000FFF8007FFF007FFFC03FFFE01FFFF00FFFF803FFF8001FFC0001FC0000FC0000FC C0007CC0007CC0007CE00078E00078F800F0FE01E0E7FFC081FF00161F7D9E1D>I<7FFFFFFC7F FFFFFC7C0FE07C700FE01C600FE00C600FE00CE00FE00EC00FE006C00FE006C00FE006C00FE006 000FE000000FE000000FE000000FE000000FE000000FE000000FE000000FE000000FE000000FE0 00000FE000000FE000000FE000000FE000000FE000000FE000000FE00007FFFFC007FFFFC01F1E 7E9D24>I87 D<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003E001FFE00FFFE03F83E07E03E07C03 E0F803E0F803E0F803E0FC07E07E0DE03FF9FE07E07E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C0300FC00 00FC0000FC0000FC0000FC0000FC00007C00007E00003E00603F00C01F81C007FF0001FC001314 7E9317>I<0007F80007F80000F80000F80000F80000F80000F80000F80000F80000F80000F800 00F801F8F80FFEF81F83F83E01F87E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC 00F87C00F87C00F87E00F83E01F81F07F80FFEFF03F8FF18207E9F1D>I<01FE0007FF801F83E0 3F01F07E00F07E00F8FC00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007E00003E0018 3F00380F807007FFE000FF8015147F9318>I<001F8000FFC001F3E003E7E003C7E007C7E007C3 C007C00007C00007C00007C00007C000FFFC00FFFC0007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0003FFC003FFC001320 7F9F10>I<01FC3C07FFFE0F07DE1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800F FF0019FC001800001800001C00001FFF801FFFF00FFFF83FFFFC7C007C70003EF0001EF0001EF0 001E78003C78003C3F01F80FFFE001FF00171E7F931A>II<1C003F007F007F007F003F001C00000000000000000000000000FF00FF 001F001F001F001F001F001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B21 7EA00E>I107 DIII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC007E FC007EFC007EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A> II114 D<0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC007C007E0 07F006F81EFFFCC7F010147E9315>I<01800180018003800380038007800F803F80FFFCFFFC0F 800F800F800F800F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C 14>II III< FFE07F80FFE07F801F001C000F8018000F80180007C0300007C0300003E0600003E0600001F0C0 0001F0C00001F9C00000F9800000FF8000007F0000007F0000003E0000003E0000001C0000001C 0000001800000018000078300000FC300000FC600000C0E00000E1C000007F8000001E00000019 1D7F931C>I E /Fg 80 124 df<000FC3E0007FEFF000E07C7801C0F8780380F8300700700007 0070000700700007007000070070000700700007007000FFFFFF80FFFFFF800700700007007000 070070000700700007007000070070000700700007007000070070000700700007007000070070 00070070000700700007007000070070007FE3FF007FE3FF001D20809F1B>11 D<001F80007FC001E0E00381E00381E00701E0070000070000070000070000070000070000FFFF E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E07FC3FE7FC3FE1720809F19>I<001FE0007FE001E1E00381E003 80E00700E00700E00700E00700E00700E00700E00700E0FFFFE0FFFFE00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07F E7FE7FE7FE1720809F19>I<000F80F800007FE7FE0000E06E060001C0FC0F000380F80F000700 F00F00070070000007007000000700700000070070000007007000000700700000FFFFFFFF00FF FFFFFF000700700700070070070007007007000700700700070070070007007007000700700700 070070070007007007000700700700070070070007007007000700700700070070070007007007 0007007007007FE3FE3FF07FE3FE3FF02420809F26>I<70F8F8F8F8F8F8F8F870707070707070 70707070707070000000000070F8F8F87005217CA00D>33 D<7038F87CFC7EFC7E7C3E0C060C06 0C06180C180C381C3018603040200F0E7E9F17>I<0078000000FC000001860000038600000303 0000070300000703000007030000070300000706000007060000070C00000718000003B81FFC03 F01FFC03E003E003C0018003C0030007C003000EE006001CE0060038700C0078780C0070381800 F01C3000F01E3000F00E6000F007C000F003C0187801C0187807E0303E1E78701FF83FE007E00F 801E227EA023>38 D<70F8FCFC7C0C0C0C181838306040060E7C9F0D>I<006000C00180030007 0006000C001C00180038003800300070007000700060006000E000E000E000E000E000E000E000 E000E000E000E000E0006000600070007000700030003800380018001C000C0006000700030001 8000C000600B2E7DA112>II<000600000006 000000060000000600000006000000060000000600000006000000060000000600000006000000 060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600000006000000060000 000600000006000000060000000600000006000000060000000600000006000000060000000600 00000600001C207D9A23>43 D<70F8FCFC7C0C0C0C181838306040060E7C840D>II<70F8F8F87005057C840D>I<00030003000700060006000E000C000C001C0018 001800380030003000700060006000E000C000C001C00180018001800380030003000700060006 000E000C000C001C0018001800380030003000700060006000E000C000C000102D7DA117>I<03 F0000FFC001E1E001C0E00380700780780700380700380700380F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C070038070038070038078078038 07001C0E001E1E000FFC0003F000121F7E9D17>I<018003801F80FF80E3800380038003800380 0380038003800380038003800380038003800380038003800380038003800380038003800380FF FEFFFE0F1E7C9D17>I<03F0000FFC00183E00300F00600F80C00780F007C0F807C0F803C0F803 C02003C00007C0000780000780000F00000E00001C0000380000700000E00001C0000380000300 000600C00C00C01800C03001807FFF80FFFF80FFFF80121E7E9D17>I<03F0000FFC001C1E0030 0F00780F80780F80780780380F80000F80000F00000F00001E00003C0003F80003F000001C0000 0E00000F000007800007800007C02007C0F807C0F807C0F807C0F00780600F80700F003C1E001F FC0003F000121F7E9D17>I<000E00000E00001E00003E00003E00006E0000EE0000CE00018E00 038E00030E00060E000E0E000C0E00180E00380E00300E00600E00E00E00FFFFF0FFFFF0000E00 000E00000E00000E00000E00000E00000E0000FFE000FFE0141E7F9D17>I<3803003FFF003FFE 003FF8003FE00030000030000030000030000030000030000031F00037FC003E0E003C07003807 803003800003800003C00003C00003C06003C0F003C0F003C0F00380C00780600700700E003C1E 000FF80007E000121F7E9D17>I<007C0001FE000783000E03800C07801C078038030038000078 0000700000700000F3F800F7FE00FC0E00F80700F80380F80380F003C0F003C0F003C0F003C0F0 03C07003C07003C07003803803803807001C07000E1E0007FC0001F000121F7E9D17>I<600000 7FFFC07FFFC07FFF80600180C00300C00600C00600000C00001800001800003000006000006000 00E00000C00001C00001C00001C000038000038000038000038000078000078000078000078000 078000078000078000030000121F7D9D17>I<03F0000FFC001E1E003807003007007003807003 807003807003807807003E07003F0E001FDC000FF80003F80007FC001E7E00383F00700F807007 80E003C0E001C0E001C0E001C0E001C0F001807003803807001E0E000FFC0003F000121F7E9D17 >I<03F00007F8001E1C00380E00380700700700700380F00380F00380F003C0F003C0F003C0F0 03C0F003C07007C07007C0380FC01C0FC01FFBC007F3C000038000038000078000070030070078 0E00780E00701C003078001FF0000FC000121F7E9D17>I<70F8F8F87000000000000000000000 70F8F8F87005147C930D>I<70F8F8F8700000000000000000000070F8F8F87818181818303060 6040051D7C930D>I<7FFFFFE0FFFFFFF000000000000000000000000000000000000000000000 00000000000000000000FFFFFFF07FFFFFE01C0C7D9023>61 D<0FC03FF070786038E03CF03CF0 3C603C003C007800F000E001C00180038003000300030003000300030003000000000000000000 000007000F800F800F8007000E207D9F15>63 D<0003800000038000000380000007C0000007C0 000007C000000DE000000DE000000DE0000018F0000018F0000018F00000307800003078000030 780000603C0000603C0000603C0000E01E0000C01E0000FFFE0001FFFF0001800F0001800F0003 800F800300078003000780070007C0070003C00F8003C0FFE03FFEFFE03FFE1F207F9F22>65 DI<001FC040007FF0C001F839C003C00DC0078007C00F0003C0 1E0003C03E0001C03C0001C07C0001C07C0000C0780000C0F80000C0F8000000F8000000F80000 00F8000000F8000000F8000000F8000000F8000000780000C07C0000C07C0000C03C0000C03E00 01801E0001800F0003000780030003C00E0001F81C00007FF000001FC0001A217D9F21>III I<000FC020007FF86001F81CE003E006E0078003E00F0001E01E0001E01E0000E03C0000E07C00 00E07C00006078000060F8000060F8000000F8000000F8000000F8000000F8000000F8000000F8 007FFCF8007FFC780001E07C0001E07C0001E03C0001E01E0001E01E0001E00F0001E0078003E0 03E003E001F80E60007FFC20000FE0001E217D9F24>III75 DIII<001F800000FFF00001E0780007C03E000F801F000F000F001E0007803C0003C0 3C0003C07C0003E07C0003E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001 F0F80001F0F80001F0F80001F0780001E07C0003E07C0003E03C0003C03E0007C01E0007800F00 0F000F801F0007C03E0001F0F80000FFF000001F80001C217D9F23>II82 D<07E0800FF9801C1F80300F8070078060 0380E00380E00180E00180E00180F00000F000007800007F00003FF0001FFC000FFE0003FF0000 1F800007800003C00003C00001C0C001C0C001C0C001C0C001C0E00180F00380F80700FE0E00CF FC0081F80012217D9F19>I<7FFFFFE07FFFFFE0780F01E0700F00E0600F0060600F0060E00F00 70C00F0030C00F0030C00F0030C00F0030000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F000007FFFE0007FFFE001C1F7E9E21>IIII<7FF83FF87FF83FF807C00F80 03C0060001E00E0001F00C0000F0180000783800007C3000003C7000003E6000001EC000000FC0 00000F8000000780000007C0000007E000000DE000001DF0000018F8000038780000307C000060 3C0000E01E0000C01F0001800F0003800780030007C00F8007C0FFE01FFEFFE01FFE1F1F7F9E22 >II<0804180C3018703860306030C060C060C060F87CFC7EFC 7E7C3E381C0F0E7B9F17>92 D<1FE0003FF000783800781C00300E00000E00000E00000E0003FE 001FFE003E0E00780E00F80E00F00E30F00E30F00E30F01E30783F703FEFE00F878014147E9317 >97 D<0E0000FE0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E3F000EFF800FC3C00F00E00E00F00E00700E00780E00780E00780E00780E00780E00780E0078 0E00700E00F00F00F00F01E00FC3C00CFF800C3E0015207F9F19>I<03F80FFC1E1E3C1E380C78 007000F000F000F000F000F000F0007000780038033C031F0E0FFC03F010147E9314>I<000380 003F80003F8000038000038000038000038000038000038000038000038000038003E3800FFB80 1E0F803C0780780380780380700380F00380F00380F00380F00380F00380F00380700380700380 7807803807801E1F800FFBF803E3F815207E9F19>I<03F0000FFC001E1E003C0F003807007807 00700380F00380FFFF80FFFF80F00000F00000F000007000007800003801801C03800F070007FE 0001F80011147F9314>I<003C00FE01CF038F03060700070007000700070007000700FFF0FFF0 07000700070007000700070007000700070007000700070007000700070007007FF07FF0102080 9F0E>I<0001E003E3F00FFF701C1C703C1E00380E00780F00780F00780F00780F00380E003C1E 001C1C003FF80033E0003000003000003800003FFE001FFF803FFFC07003E07000F0E00070E000 70E00070F000F07801E03E07C00FFF0003FC00141F7F9417>I<0E0000FE0000FE00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E3E000EFF000FC3800F01C00F01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FF E7FCFFE7FC16207F9F19>I<1C001E003E001E001C000000000000000000000000000E007E007E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC0FFC00A1F809E 0C>I<00E001F001F001F000E0000000000000000000000000007007F007F000F0007000700070 007000700070007000700070007000700070007000700070007000700070007000706070F0E0F1 C07F803E000C28829E0E>I<0E0000FE0000FE00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E1FF00E1FF00E0F800E0E000E0C000E18000E30000E70000EF0000FF800 0F38000E1C000E1E000E0E000E07000E07800E03800E03C0FFCFF8FFCFF815207F9F18>I<0E00 FE00FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E00FFE0FFE00B20809F0C>I<0E3F03F000FEFF8FF8 00FFC1DC1C000F80F80E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E00FFE7FE7FE0FFE7FE7FE023147F9326>I<0E3E00FEFF00FFC3800F01C00F01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FCFF E7FC16147F9319>I<01F80007FE001E07803C03C03801C07000E07000E0F000F0F000F0F000F0 F000F0F000F0F000F07000E07801E03801C03C03C01E078007FE0001F80014147F9317>I<0E3F 00FEFF80FFC3C00F01E00E00F00E00F00E00780E00780E00780E00780E00780E00780E00780E00 700E00F00F00F00F01E00FC3C00EFF800E3E000E00000E00000E00000E00000E00000E00000E00 00FFE000FFE000151D7F9319>I<03E1800FF9801E1F803C0780780780780380700380F00380F0 0380F00380F00380F00380F003807003807803807807803C0F801E1F800FF38003E38000038000 0380000380000380000380000380000380003FF8003FF8151D7E9318>I<0E78FEFCFF9E0F1E0F 0C0F000E000E000E000E000E000E000E000E000E000E000E000E00FFE0FFE00F147F9312>I<1F 903FF07070E030E030E030F00078007F803FE00FF000F0C078C038C038E038E038F070DFE08F80 0D147E9312>I<06000600060006000E000E001E003E00FFF8FFF80E000E000E000E000E000E00 0E000E000E000E000E180E180E180E180E18073007E003C00D1C7F9B12>I<0E01C0FE1FC0FE1F C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03 C00E03C0070FC007FDFC01F1FC16147F9319>III<7FC7FC7FC7FC0703E007038003830001C70000EE00 00EC00007800003800003C00007C0000EE0001C7000187000303800701C00F01E0FF87FEFF87FE 1714809318>II<3FFF3FFF380E301C703C6038607060F0 01E001C0038007830F030E031C073C063806701EFFFEFFFE10147F9314>II E /Fh 20 121 df<0001FF0000001FFFC000007F81E00000FC01E00001F807F00003 F807F00007F007F00007F007F00007F007F00007F007F00007F001C00007F000000007F0000000 07F000000007F03FF800FFFFFFF800FFFFFFF800FFFFFFF80007F003F80007F003F80007F003F8 0007F003F80007F003F80007F003F80007F003F80007F003F80007F003F80007F003F80007F003 F80007F003F80007F003F80007F003F80007F003F80007F003F80007F003F80007F003F80007F0 03F80007F003F80007F003F8007FFF3FFF807FFF3FFF807FFF3FFF80212A7FA925>12 D<000E00001E00007E0007FE00FFFE00FFFE00F8FE0000FE0000FE0000FE0000FE0000FE0000FE 0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE 0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE007FFFFE7FFF FE7FFFFE17287BA722>49 D<007F800003FFF0000FFFFC001F03FE003C00FF0078007F807C003F C0FE003FC0FF001FE0FF001FE0FF001FE0FF001FE07E001FE03C001FE000001FE000001FC00000 3FC000003F8000007F0000007E000000FC000001F8000003F0000003E00000078000000F000000 1E0000003800E0007000E000E000E001C001C0038001C0060001C00FFFFFC01FFFFFC03FFFFFC0 7FFFFFC0FFFFFF80FFFFFF80FFFFFF801B287DA722>I<007F800001FFF00007FFFC000F81FE00 1E00FF001F007F003F807F803F807F803F807F803F807F801F807F801F007F8000007F000000FF 000000FE000001FC000001F8000007E00000FF800000FFC0000001F8000000FE0000007F000000 7F8000003FC000003FC000003FE000003FE03C003FE07E003FE0FF003FE0FF003FE0FF003FC0FF 003FC0FE007FC07C007F803C00FF001F01FE000FFFFC0007FFF00000FF80001B297DA722>I<00 003FE0030003FFFC07001FFFFF0F003FF00F9F00FF8001FF01FE0000FF03FC00007F07F800003F 0FF000001F1FE000001F1FE000000F3FC000000F3FC000000F7FC00000077FC00000077F800000 077F80000000FF80000000FF80000000FF80000000FF80000000FF80000000FF80000000FF8000 0000FF80000000FF800000007F800000007F800000007FC00000077FC00000073FC00000073FC0 0000071FE000000E1FE000000E0FF000001C07F800001C03FC00003801FE00007000FF8001E000 3FF007C0001FFFFF800003FFFE0000003FF000282B7CA931>67 D69 D73 D<03FF80000FFFF0001F01FC003F80FE003F807F 003F803F003F803F801F003F8000003F8000003F8000003F800000FF80007FFF8003FFBF800FF0 3F801F803F803F003F807E003F80FC003F80FC003F80FC003F80FC003F80FE007F807E00DFC03F 839FFC1FFF0FFC03FC03FC1E1B7E9A21>97 D<003FF00001FFFC0003F03E000FC07F001F807F00 3F007F003F007F007F003E007E0000007E000000FE000000FE000000FE000000FE000000FE0000 00FE000000FE0000007E0000007E0000007F0000003F0003803F8003801F8007000FE00E0003F8 3C0001FFF800003FC000191B7E9A1E>99 D<00007FF000007FF000007FF0000007F0000007F000 0007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0 003F87F001FFF7F007F03FF00FC00FF01F8007F03F0007F03F0007F07E0007F07E0007F07E0007 F0FE0007F0FE0007F0FE0007F0FE0007F0FE0007F0FE0007F0FE0007F0FE0007F07E0007F07E00 07F03F0007F03F0007F01F800FF00FC01FF007E07FFF01FFE7FF007F87FF202A7EA925>I<003F C00001FFF00007E07C000FC03E001F801F003F001F803F000F807F000F807E000FC0FE000FC0FE 0007C0FE0007C0FFFFFFC0FFFFFFC0FE000000FE000000FE000000FE0000007E0000007F000000 3F0001C03F8001C01F8003C00FC0078003F01F0000FFFC00003FE0001A1B7E9A1F>I<07001FC0 1FE03FE03FE03FE01FE01FC007000000000000000000000000000000FFE0FFE0FFE00FE00FE00F E00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE0FFFE FFFEFFFE0F2B7EAA12>105 D108 D110 D<003FE00001FFFC0003F07E 000FC01F801F800FC03F0007E03F0007E07E0003F07E0003F07E0003F0FE0003F8FE0003F8FE00 03F8FE0003F8FE0003F8FE0003F8FE0003F8FE0003F87E0003F07E0003F03F0007E03F0007E01F 800FC00FC01F8007F07F0001FFFC00003FE0001D1B7E9A22>I114 D<03FE300FFFF03E03F07800F07000F0F00070F00070F80070FE0000FFE0007FFF007FFFC03FFF E01FFFF007FFF800FFF80007FC0000FCE0007CE0003CF0003CF00038F80038FC0070FF01E0F7FF C0C1FF00161B7E9A1B>I<00700000700000700000700000F00000F00000F00001F00003F00003 F00007F0001FFFE0FFFFE0FFFFE007F00007F00007F00007F00007F00007F00007F00007F00007 F00007F00007F00007F00007F00007F07007F07007F07007F07007F07007F07007F07003F0E001 F8C000FFC0003F0014267FA51A>II120 D E /Fi 13 115 df<70F8FCFC7C0C0C0C0C181830306040060F7C 840E>44 D<01F00007FC000E0E001C07003803803803807803C07001C07001C07001C0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E070 01C07001C07001C07803C03803803803801C07000E0E0007FC0001F00013227EA018>48 D<018003800F80FF80F38003800380038003800380038003800380038003800380038003800380 038003800380038003800380038003800380038003800380FFFEFFFE0F217CA018>I<1000801C 07801FFF001FFE001FFC001FF00018000018000018000018000018000018000018F8001BFE001F 0F001C07801803801803C00001C00001E00001E00001E00001E06001E0F001E0F001E0F001C0C0 03C06003C06003803007001C1E000FFC0003F00013227EA018>53 D<01F00007FC000E0E001C07 00380380780380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003 E07803E03807E01C0DE00FF9E007F1E00081C00001C00003C0000380000380300700780700780E 00701C003838001FF0000FC00013227EA018>57 D77 D<03F0200FFC601C0EE03803E07001E07001E0E000E0E000E0E00060E00060E00060F000 00F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E00001E00000F00000F00000 70C00070C00070C00070C00070E00060E000E0F000C0F801C0EF0380C7FF0081FC0014247DA21B >83 D<0FE0001FF8003C1C003C0E00180700000700000700000F0003FF000FFF003F07007C0700 780700F00700F00718F00718F00F18780F187C3FB83FF3F00FC3C015157E9418>97 D<01FE0007FF000F07801C0780380300780000700000F00000F00000F00000F00000F00000F000 00F000007800007800C03C00C01E01800F030007FE0001F80012157E9416>99 D<0E0000FE0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E1F800E7FC00FC1E00F80F00F00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E0070FFE7FFFFE7FF18237FA21B>104 D<1C001E003E001E001C00000000000000000000000000000000000E00FE00FE001E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E00FFC0FFC00A227FA10E>I<0E1F 80FE7FC0FFC1E01F80F00F00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E0070FFE7FFFFE7FF18157F941B>110 D<0E3CFEFEFFCF1F8F0F060F000E000E000E000E000E000E000E000E000E000E000E000E000E00 FFF0FFF010157F9413>114 D E /Fj 12 116 df45 D<007F800001FFE0000781F8000E007C0018003E0018001F0030000F8060000FC0600007C06000 07C0F00007E0F80007E0FC0003E0FC0003E0FC0003E0780007E0000007E0000007C0000007C000 000FC000000F8000000F8000001F0000003E0000003C00000078000000F8000000F0000001E000 0003C00000070000000E0000001C000000180000003000000060000000C0006001800060038000 60070000C00E0000C00C0000C0180001C03FFFFFC07FFFFF80FFFFFF80FFFFFF801B2F7DAE21> 50 D 73 D77 D80 D<01FE00000FFFC0001C03E0003C00F0003E 0078003E003C003E001C0008001E0000001E0000001E0000001E0000001E000001FE00003FFE00 01FF1E0007F01E000F801E001F001E003E001E007C001E007C001E00F8001E0CF8001E0CF8001E 0CF8003E0CF8003E0C7C007E0C3E00DF1C1F038FB80FFF07F003F803C01E1F7D9E21>97 D<0003E0001FF8003C3800707C00F07C01E07C01E03803C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007FFF0016327FB114 >102 D<0780FF80FF800F80078007800780078007800780078007800780078007800780078007 800780078007800780078007800780078007800780078007800780078007800780078007800780 07800780078007800780078007800780078007800FC0FFFCFFFC0E327EB112>108 D<001FC00000FFF80001E03C0007800F000F0007801E0003C01E0003C03C0001E03C0001E07800 00F0780000F0780000F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F8 0000F8780000F07C0001F03C0001E03C0001E01E0003C01E0003C00F00078007C01F0001F07C00 00FFF800001FC0001D1F7E9E21>111 D<0780FC00FF87FF80FF9E07C00FB803E007E000F007C0 00F807C0007C0780007C0780003E0780003E0780003E0780001F0780001F0780001F0780001F07 80001F0780001F0780001F0780001F0780001F0780003E0780003E0780003E0780007C07C00078 07C000F807E001F007B803E0079E0FC00787FF000781FC00078000000780000007800000078000 00078000000780000007800000078000000780000007800000078000000FC00000FFFC0000FFFC 0000202D7E9E25>I<0783E0FF8FF8FF9C7C0FB07C07F07C07E03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100FFF301E03F03800F0700070E00070E00030E00030E00030F00030F800007E00003FF0 001FFF000FFF8003FFE0003FF00003F8000078C0003CC0003CC0001CE0001CE0001CF0001CF000 18F80038FC0070EF01E0C3FFC080FE00161F7E9E1A>I E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300 TeXDict begin %%EndSetup %%Page: 1 1 bop 684 231 a Fj(Prop)r(osals)20 b(for)i(MPI-2)869 358 y Fi(Marc)16 b(Snir)811 460 y(Marc)o(h)g(10,)g(1995)75 635 y Fh(1)67 b(In)n(tro)r(duction) 75 736 y Fg(The)18 b(\014rst)g(section)h(of)f(this)h(do)q(cumen)o(t)f(con)o (tains)g(prop)q(osed)h(corrections)f(and)h(clari\014cations)g(to)f(the)75 792 y(curren)o(t)11 b(MPI)g(do)q(cumen)o(t.)18 b(The)11 b(prop)q(osals)g(are) g(clear)g(enough)h(\(I)e(hop)q(e\))i(to)e(allo)o(w)h(us)g(to)f(tak)o(e)g (decisions)75 849 y(at)15 b(coming)g(meeting.)166 905 y(The)i(next)f(section) h(of)f(this)h(do)q(cumen)o(t)g(con)o(tains)g(prop)q(osals)f(for)g(p)q (ossible)i(extensions)g(to)d(MPI.)75 962 y(All)h(of)e(these)g(prop)q(osals)h (require)g(further)f(rounds)h(of)f(discussion)i(and)f(elab)q(oration,)f(b)q (efore)h(design)g(is)75 1018 y(frozen.)75 1136 y Ff(Ac)o(kno)o(wledgmen)o(t:) 43 b Fg(The)16 b(prop)q(osal)f(for)f(m)o(ultiple)j(completions)g(is)e(due)h (to)e(S)h(R)h(Chapple,)g(Edin-)75 1193 y(burgh.)k(Other)c(con)o(tributions)f (b)o(y)h(R)f(T)l(reumann,)g(Y-i)h(Cho)q(o,)f(M)g(T)g(Ragh)o(u,)g(H)g(F)l (rank)o(e)g(and)g(others.)75 1334 y Fh(2)67 b(Corrections)22 b(and)h(Clari\014cations)75 1437 y Fe(2.1)56 b(Non)18 b(blo)r(c)n(king)i (comm)n(unication)75 1523 y Fg(This)c(is)f(to)g(clarify)h(the)f(p)q (ersistence)i(of)e(request)g(handles.)166 1579 y(Problem:)75 1636 y(Request)k(argumen)o(ts)f(are)h(passed)g(b)o(y)g(reference)g(to)g(comm) o(unication)g(calls.)32 b(This)19 b(w)o(ould)h(seem)f(to)75 1692 y(indicate)e(that)f(all)h(suc)o(h)f(calls)h(can)f(mo)q(dify)h(the)f(v)m (alue)h(of)f(the)g(handle)h(\(the)f(lo)q(cation)g(of)g(the)g(request)75 1748 y(ob)s(ject\).)166 1805 y(Clari\014cation:)75 1861 y(MPI)d(comm)o (unication)g(calls)h(do)e(not)g(mo)q(dify)i(the)e(v)m(alue)i(of)f(a)f (request)g(handle,)j(except)e(as)f(to)g(assign)h(it)75 1918 y(a)f(v)m(alue)i(when)g(the)e(request)h(is)h(created)e(and)h(assign)g(it)g(v) m(alue)h(MPI)p 1241 1918 14 2 v 16 w(PR)o(OC)p 1388 1918 V 17 w(NULL)g(when)f(the)g(request)75 1974 y(is)k(deallo)q(cated.)27 b(In)18 b(particular,)f(the)g(v)m(alue)i(of)d(a)h(handle)h(to)e(a)h(p)q (ersisten)o(t)h(request)e(is)i(not)f(mo)q(di\014ed)75 2031 y(except)f(b)o(y)f(the)p 362 2031 V 32 w(INIT)h(and)p 582 2031 V 32 w(FREE)f(calls.)22 b(The)15 b(call)i(to)e(MPI)p 1181 2031 V 16 w(ST)l(AR)l(T)h(do)q(es)g(not)f(c)o(hange)g(the)h(v)m(alue)75 2087 y(of)f(the)g(request)g(handle)i(\(it)e(ma)o(y)f(c)o(hange)i(the)f(v)m (alue)h(of)f(the)g(request)h(ob)s(ject)e(itself)t(\).)166 2144 y(Example:)75 2226 y Fd(MPI_Bsend_init\(...,&reques)o(t\))75 2282 y(MPI_Start\(&request\))75 2339 y(MPI_Wait\(&request,)21 b(&status\))166 2421 y Fg(An)15 b(implemen)o(tation)h(that)d(implemen)o(ts)j (MPI)p 994 2421 V 17 w(Start)d(b)o(y)i(cop)o(ying)g(data)f(to)g(the)g (bu\013er,)h(p)q(osting)75 2478 y(a)g(standard)h(send)g(from)f(the)g (bu\013er)h(and)g(mo)q(difying)h(the)e(request)h(p)q(oin)o(ter)g(to)f(p)q (oin)o(t)h(to)f(the)h(request)75 2534 y(returned)g(b)o(y)f(this)g(send)h(is)g (incorrect.)166 2591 y(Discussion:)75 2647 y(W)l(e)c(ma)o(y)e(w)o(an)o(t)h (to)g(c)o(hange)g(the)h(C)f(binding)j(of)d Fd(MPI)p 953 2647 15 2 v 17 w(Start)g Fg(so)g(that)g(the)g(request)h(parameter)e(b)q(e)j (passed)75 2704 y(b)o(y)i(v)m(alue,)h(rather)f(than)g(reference.)20 b(This)c(will)h(clarify)f(that)f(this)g(parameter)g(is)g(not)g(up)q(dated.) 964 2828 y(1)p eop %%Page: 2 2 bop 75 49 a Fe(2.2)56 b(Multiple)20 b(Completions)75 139 y Fg(This)c(is)h(to)e(c)o(hange)h(the)g(b)q(eha)o(vior)g(of)f Fd(MPI)p 825 139 15 2 v 17 w Fc(f)p Fd(TEST|WAIT)p Fc(gf)22 b Fd(|ANY|SOME|ALL)p Fc(g)14 b Fg(on)h(an)h(empt)o(y)g(list)g(of)75 196 y(handles,)g(so)f(as)g(to)f(ac)o(hiev)o(e)i(a)f(more)g(consisten)o(t)g (de\014nition.)166 255 y(De\014nitions:)75 311 y(A)e(list)i(of)e(request)g (handles)i(is)f Fb(empty)g Fg(if)g(eac)o(h)f(handle)i(either)f(ha)o(v)o(e)f (v)m(alue)i Fd(MPI)p 1448 311 V 16 w(REQUEST)p 1632 311 V 17 w(NULL)d Fg(or)h(is)h(a)75 368 y(handle)h(to)d(a)h(p)q(ersisten)o(t)h (request)f(whic)o(h)h(is)g(inactiv)o(e)g(\(is)g(not)f(curren)o(tly)h(asso)q (ciated)f(with)h(an)f(ongoing)75 424 y(comm)o(unication.)166 483 y(A)i Fb(nul)r(l)h(status)g Fg(is)g(the)f(status)g(that)f(signals)i(\\no) g(message":)j Fd(tag)24 b(=)f(MPI)p 1462 483 V 17 w(ANY)p 1551 483 V 17 w(TAG,)g(source)g(=)75 539 y(MPI)p 150 539 V 17 w(ANY)p 239 539 V 17 w(SOURCE,)f(count)i(=)f(0)p Fg(.)166 598 y(Prop)q(osed)15 b(c)o(hanges:)166 657 y(1.)32 b Fd(MPI)p 309 657 V 17 w(TEST)18 b Fg(and)i Fd(MPI)p 605 657 V 17 w(TESTANY)e Fg(should)i(return)f Fd(flag)24 b(=)g(TRUE)18 b Fg(on)h(an)h(empt)o(y)f(list)h(\(rather)75 713 y(than)c(curren)o(t)h Fd(flag)23 b(=)h(FALSE)p Fg(\).)15 b(In)i(addition,)h Fd(MPI)p 1002 713 V 17 w(TEST)e Fg(will)i(return)f(a)f(n)o (ull)i(status)e(\(curren)o(tly)l(,)h(it)75 769 y(do)q(es)f(not)e(set)h(the)h (status\).)166 828 y(Note)d(that)g Fd(MPI)p 446 828 V 16 w(TESTANY)g Fg(will)i(return,)e(as)g(no)o(w,)g Fd(index)23 b(=)h(MPI)p 1300 828 V 17 w(UNDEFINED)12 b Fg(on)h(an)g(empt)o(y)h(list.)166 887 y(Th)o(us,)22 b(for)f Fd(MPI)p 452 887 V 17 w(TEST)p Fg(,)f Fd(flag=TRUE)g Fg(and)i Fd(status)h(=)g(``null'')e Fg(indicates)h(that)f (there)g(is)h(no)75 943 y(p)q(ending)g(comm)o(unication)f(to)f(w)o(ait)g (for;)i Fd(flag=TRUE)d Fg(and)h Fd(status)j Fc(6)p Fg(=)i Fd(``null'')19 b Fg(indicates)j(that)75 1000 y(a)g(comm)o(unication)g(has)g(completed;)k (and)c Fd(flag)h(=)h(FALSE)d Fg(indicates)j(that)d(there)h(is)g(an)g(ongoing) 75 1056 y(comm)o(unication)16 b(whic)o(h)g(has)f(not)g(completed)h(y)o(et.) 166 1115 y(F)l(or)11 b Fd(MPI)p 319 1115 V 17 w(TESTANY)p Fg(,)f Fd(flag)24 b(=)f(TRUE)12 b Fg(and)g Fd(index)23 b(=)h(MPI)p 1150 1115 V 17 w(UNDEFINED)10 b Fg(indicates)j(that)f(there)g(is)g(no)75 1171 y(p)q(ending)18 b(comm)o(unication)f(to)f(w)o(ait)g(for;)h Fd(flag)23 b(=)h(TRUE)15 b Fg(and)i Fd(index)23 b Fc(6)p Fg(=)i Fd(MPI)p 1456 1171 V 16 w(UNDEFINED)16 b Fg(indicates)75 1228 y(that)i(a)h(comm)o(unication)h(has)f(completed;)j(and)d Fd(flag)k(=)h(FALSE) 19 b Fg(indicates)h(that)e(there)i(is)f(at)g(least)75 1284 y(one)c(ongoing)h(comm)o(unication)f(and)h(no)f(ongoing)g(comm)o(unication)h (has)f(completed.)166 1343 y(In)h(b)q(oth)f(cases,)g Fd(flag)23 b(=)h(TRUE)14 b Fg(indicate)j(that)e(one)g(need)h(exit)g(a)f(busy-w)o(aiting) h(lo)q(op.)166 1402 y(Reasons)f(for)g(c)o(hange:)75 1458 y(Need)k(to)e (distinguish)j Fd(MPI)p 559 1458 V 17 w(TESTANY)d Fg(returning)i(b)q(ecause)g (none)g(of)e(the)h(p)q(ending)i(comm)o(unications)75 1515 y(completed,)c(and) f Fd(MPI)p 468 1515 V 17 w(TESTANY)f Fg(returning)i(b)q(ecause)g(there)f(is)h (no)f(comm)o(unication)h(to)f(complete.)166 1573 y(With)g(the)g(old)h (de\014nition,)h(in)e(b)q(oth)h(cases)f Fd(MPI)p 1001 1573 V 16 w(TESTANY)f Fg(returns)h Fd(flag)24 b(=)f(FALSE)p Fg(,)14 b(and)i Fd(index)75 1630 y Fg(v)m(alue)g(is)g(unde\014ned.)166 1688 y(With)d(the)h(new)f(de\014nition,)j(in)e(the)f(\014rst)g(case)g Fd(MPI)p 1051 1688 V 17 w(TESTANY)f Fg(returns)i(with)f Fd(flag)23 b(=)h(FALSE)13 b Fg(and)75 1745 y(in)i(the)g(second)g(case)g(it)f(returns)h (with)g Fd(flag)23 b(=)h(TRUE)p Fg(.)13 b(Successful)j(completion)g(of)e(a)g (comm)o(unication)75 1801 y(is)j(distinguished)h(from)d(the)i(case)f(where)g (there)g(w)o(as)f(no)h(comm)o(unication)h(to)f(w)o(ait)f(for)h(b)o(y)g(the)g (v)m(alue)75 1858 y(of)f Fd(index)p Fg(.)166 1917 y(Similarly)l(,)i(need)e (to)f(distinguish)j Fd(MPI)p 833 1917 V 16 w(TEST)d Fg(returning)i(b)q (ecause)f(the)g(p)q(ending)h(comm)o(unication)75 1973 y(has)f(not)g (completed,)h(and)f Fd(MPI)p 630 1973 V 17 w(TEST)f Fg(returning)i(b)q (ecause)g(there)g(is)f(no)g(p)q(ending)i(comm)o(unication.)166 2032 y(With)h(the)h(old)g(de\014nition)h Fd(MPI)p 727 2032 V 16 w(TEST)e Fg(w)o(ould)h(return)f(in)h(b)q(oth)g(cases)f Fd(flag)23 b(=)h(FALSE)17 b Fg(and)i(the)75 2088 y(v)m(alue)h(of)e Fd(status)g Fg(w)o(ould)h(b)q(e)g(unde\014ned.)33 b(With)19 b(the)f(new)h(de\014nition,)i Fd(MPI)p 1443 2088 V 17 w(TEST)d Fg(returns)h Fd(flag)k(=)75 2145 y(FALSE)18 b Fg(in)h(the)g(\014rst)f(case,)h (and)f Fd(flag)23 b(=)h(TRUE)18 b Fg(in)h(the)g(second)g(case.)30 b(Successful)20 b(completion)f(of)f(a)75 2201 y(comm)o(unication)f(is)g (distinguished)i(form)d(the)g(case)h(where)g(there)f(w)o(as)g(no)g(comm)o (unication)h(to)f(start)75 2258 y(with)g(b)o(y)f(the)g(v)m(alue)h(returned)g (in)g Fd(status)p Fg(.)166 2316 y(Example:)75 2421 y Fd(while\(1\))23 b({)147 2478 y(/*)g(busy)g(wait)h(until)f(flag=1)g(*/)147 2534 y(while)g(\(!{MPI_Testany\(count,)e(reqs,)i(&idx,)g(&flag,)g(&status\);)g (flag}\))g(;)147 2591 y(if\(idx)g(==)g(MPI_UNDEFINED\))f(break;)190 b(/*)24 b(no)f(more)h(requests)f(*/)147 2647 y(do_something\(reqs,)e(&idx,)i (&status\);)118 b(/*)24 b(else)f(comm)g(completed)g(*/)147 2704 y(})964 2828 y Fg(2)p eop %%Page: 3 3 bop 166 49 a Fg(2.)19 b Fd(MPI)p 296 49 15 2 v 17 w(WAITSOME)14 b Fg(and)i Fd(MPI)p 680 49 V 16 w(TESTSOME)e Fg(should)j(return)e Fd(outcount)23 b(=)g(MPI)p 1520 49 V 17 w(UNDEFINED)14 b Fg(on)h(an)75 106 y(empt)o(y)g(list)h(\(rather)e(than)h Fd(outcount)23 b(=)h(0)p Fg(\).)166 163 y(Reason)15 b(for)g(c)o(hange:)75 220 y(Need)20 b(to)e(distinguish)k Fd(MPI)p 563 220 V 16 w(TESTSOME)c Fg(returning)i(while) h(no)e(comm)o(unication)h(has)f(completed)h(but)75 276 y(there)c(are)g(still) h(p)q(ending)h(comm)o(unications,)e(and)g Fd(MPI)p 1044 276 V 17 w(TESTSOME)f Fg(returning)h(when)h(there)f(is)g(no)g(op-)75 333 y(eration)f(to)g(complete.)166 390 y(With)g(the)h(old)f(de\014nition,)i Fd(MPI)p 727 390 V 17 w(TESTSOME)d Fg(w)o(ould)h(return)h(in)g(b)q(oth)f (cases)g Fd(outcount)23 b(=)h(0)p Fg(.)166 447 y(With)c(the)f(new)h (de\014nition,)i Fd(MPI)p 763 447 V 17 w(TESTSOME)c Fg(returns)i Fd(outcount)i(=)i(0)19 b Fg(in)i(the)e(\014rst)h(case,)g(but)75 504 y Fd(outcount)j(=)g(MPI)p 412 504 V 17 w(UNDEFINED)14 b Fg(in)i(the)g(second)f(case.)166 561 y Fd(MPI)p 241 561 V 17 w(WAITSOME)f Fg(is)h(mo)q(di\014ed)i(in)f(order)f(to)g(b)q(e)g(consisten)o(t) h(with)f Fd(MPI)p 1371 561 V 17 w(TESTSOME)p Fg(.)166 619 y(Example:)20 b(a)15 b(lo)q(op)h(that)e(deals)i(for)f(all)h(comm)o(unications)g(in)g(a)f (list)h(could)g(b)q(e)g(written)f(as)75 731 y Fd(while\(1\))23 b({)147 787 y(MPI_Waitsome\(incnt,)e(reqs,)i(&outcnt,)g(indxs,)g(statuses\);) 147 844 y(if)g(\(outcnt)g(==)h(MPI_UNDEFINED\))e(break;)147 900 y(for\(i=0;)g(i To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann) Subject: proposals Cc: lusk@antares.mcs.anl.gov Date: Fri, 10 Mar 1995 14:35:41 -0600 From: Rusty Lusk If anyone has concrete proposals that they would like to discuss, or at least distribute, at next week's meeting, could they please send them to me? I will send them out by email today to everyone who is coming, for airplane reading, and I may be able to make paper copies for working with at the meeting. Regards, Rusty From owner-mpi-core@CS.UTK.EDU Fri Mar 10 17:03:10 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA18568; Fri, 10 Mar 1995 17:03:07 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA19534; Fri, 10 Mar 1995 17:03:23 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 10 Mar 1995 17:03:18 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id RAA19526; Fri, 10 Mar 1995 17:03:12 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA28861; Fri, 10 Mar 1995 16:03:03 -0600 Message-Id: <199503102203.QAA28861@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann), itf@mcs.anl.gov Subject: errata.ps Date: Fri, 10 Mar 1995 16:03:01 -0600 From: Rusty Lusk %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: errata.dvi %%CreationDate: Fri Oct 28 16:34:30 1994 %%Pages: 7 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips errata %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1994.10.28:1626 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/sparky/sparky5/gropp/MPI/errata.dvi) @start /Fa 66 123 df<030600078F00078F00078F00078F00078F00078F007FFFC0FFFFE0FFFF E07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFFC01E3C 001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>35 D<01C00007E0000FF000 0E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3800FC7000F87000F0700 1F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E0787070FC707FFFE03FCFE0 0F03C0141C7F9B18>38 D<007000F001E003C007800F001E001C00380038007000700070 007000E000E000E000E000E000E000E000E0007000700070007000380038001C001E000F 00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001 C001C000E000E000E000E00070007000700070007000700070007000E000E000E000E001 C001C0038007800F001E003C007800F00060000C247C9F18>I<01C00001C00001C00001 C000C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1 C18001C00001C00001C00001C00011147D9718>I<1C3E7E7F3F1F070E1E7CF860080C78 8518>44 D<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC78300606778518>I< 000300000780000780000F80000F00001F00001E00001E00003E00003C00007C00007800 00780000F80000F00001F00001E00003E00003C00003C00007C0000780000F80000F0000 0F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F00000600000 11247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001C07001C0E000 E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C07001C07803 C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>I<01800380038007800F80 3F80FF80FB80438003800380038003800380038003800380038003800380038003800380 038003807FFCFFFE7FFC0F1C7B9B18>I<03F0000FFE003FFF007C0F807003C0E001C0F0 00E0F000E06000E00000E00000E00001C00001C00003C0000780000F00001E00003C0000 780000F00001E00007C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I< 3078FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<7FFF C0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C 7E9318>61 D<600000F00000FC00007E00003F00001FC00007E00003F00001FC00007E00 003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000 F0000060000011187D9918>I<00700000F80000F80000D80000D80001DC0001DC0001DC 00018C00038E00038E00038E00038E000306000707000707000707000707000FFF800FFF 800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE00F07E01E03E03C 01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E00000E00000E0 00007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F800131C7E9B 18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C0070 1C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C01C0 1C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00FFFC01F0FC0 1C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E00000E00FF0 E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0 141C7E9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>73 D<7F07F0FF87F87F07F0 1C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0001DF0001DF0001FB800 1FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03801C01C07F03F0FF87F8 7F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E 00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1 C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19 C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003F FE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380F00780700700780F007F FF003FFE000FF800111C7D9B18>II<7FF8 00FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF 001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C039C1C039C1C03 9C7F01F8FF81F87F00F0161C7F9B18>82 D<03F3801FFF803FFF807C0F80700780E00380 E00380E00380E000007000007800003F00001FF00007FE0000FF00000F800003C00001C0 0000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18> I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E070380070000070000070000070000070 000070000070000070000070000070000070000070000070000070000070000070000070 0000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F80001 F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F000707000E 07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II91 D93 D<7FFF00FFFF80FFFF807FFF0011 047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF 007F0700780700E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318 >97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC0 0FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F0070 0F80E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E07803803 00700000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03 C01FFF8007FF0001FC0012147D9318>I<001F80003F80001F8000038000038000038000 038000038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E0 0380E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I< 01F00007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0 E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007F C000FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF 007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E0038070038070038 07003807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03F FFE07801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC0015 1F7F9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF80 0FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C0 000380000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF 80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF8001F1F1C00 1E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F001914819318>I<7E 3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0000FFE00 1FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001E07001C0 7803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E00007FC0 00FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780700780E0 0380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF800FFB8003 E380000380000380000380000380000380000380000380003FF8003FF8003FF8151E7E93 18>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C000038000038000038000 0380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318>I<07F7 003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE00001F006007 80E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<01800003800003 80000380000380007FFFC0FFFFC0FFFFC003800003800003800003800003800003800003 80000380000380000380400380E00380E00380E001C1C001FFC000FF80003E0013197F98 18>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318>I<7F8F F0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E00038E00038E 00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700 078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E00070700 0F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03 80070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000DC000078 0000780000780000700000700000700000F00000E00079E0007BC0007F80003F00001E00 00151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00003C0000 F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF014147F93 18>I E /Fb 20 121 df<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFF FE0F1F7C9E17>49 D<07F0000FFC001FFE00383F00700F00600780E00780E003C04003C0 4003C00003C00003C00003C0000780000780000F00000E00001C00003C0000780000E000 01C0000380000700000E00001C0000380000700000FFFFC0FFFFC0FFFFC0121F7E9E17> I<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C00007800007800 00780000F00000F00000F00000F00000F00000F00000F00000F000007800007800007800 007C00003C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D >67 DII73 D76 DII<003F000000FFC00003FFF00007 E1F8000F807C001F003E001E001E003C000F003C000F00780007807800078078000780F0 0003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C07800078078000780780007803C000F003C000F001E001E001F003E000F807C0007 E1F80003FFF00000FFC000003F00001A227DA021>II84 D<780007807C000F003E001F001E001E000F003C000F807C00078078 0003C0F00003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E00 00003F0000007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C 000F003E001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>88 D<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1F FF0FFE03F010147E9314>99 D<03F0000FFC001FFE003E1F003C0700780700700380FFFF 80FFFF80FFFF80F00000F00000F000007000007800003C01003E07001FFF0007FE0001F8 0011147F9314>101 D108 DI<01F80007FE00 1FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000F07801E0 7801E03C03C03F0FC01FFF8007FE0001F80014147F9317>111 DI<7801E07C03C03E07801E0F000F0F00079E0003FC0003F8 0001F80000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801 F01414809315>120 D E /Fc 1 16 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFF FFFFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D E /Fd 12 119 df 69 D73 D85 D<00003FF80000003FF80000003FF800000003F800000003F800000003F800000003F800 000003F800000003F800000003F800000003F800000003F800000003F800000003F80000 0003F800001FE3F80000FFFBF80003F03FF80007E00FF8000FC007F8001F8003F8003F80 03F8007F0003F8007F0003F8007F0003F800FF0003F800FF0003F800FF0003F800FF0003 F800FF0003F800FF0003F800FF0003F8007F0003F8007F0003F8007F0003F8003F8003F8 001F8003F8000F8007F80007C00FF80003F03BFF8000FFF3FF80003FC3FF80212A7EA926 >100 D<003FE00001FFF80003F07E0007C01F000F801F801F800F803F800FC07F000FC0 7F0007C07F0007E0FF0007E0FF0007E0FFFFFFE0FFFFFFE0FF000000FF000000FF000000 7F0000007F0000007F0000003F8000E01F8000E00FC001C007E0038003F81F0000FFFE00 001FF0001B1B7E9A20>I108 D110 D<003FE00001FFFC0003F07E000FC01F801F800FC03F800FE03F0007E07F0007F07F0007 F07F0007F0FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007 F87F0007F07F0007F03F800FE03F800FE01F800FC00FC01F8007F07F0001FFFC00003FE0 001D1B7E9A22>I114 D<03FE300FFFF01E03F038 00F0700070F00070F00070F80070FC0000FFE0007FFE007FFF803FFFE01FFFF007FFF800 FFF80003FC0000FC60007CE0003CF0003CF00038F80038FC0070FF01E0F7FFC0C1FF0016 1B7E9A1B>I117 DI E /Fe 18 118 df<00000200000006000000060000000E0000001E0000001E00 00003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F0000020F00 00040F00000C0F0000080F0000100F0000100F0000200F80003FFF800040078000C00780 0080078001000780010007800200078002000780060007801E000F80FF807FF81D207E9F 22>65 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078 0000780000780000F00000F00000F00000F00001E00001E00001E00001E00003C00003C0 0003C00003C0000780000780000780000780000F8000FFF800141F7D9E12>73 D<001FFF0000F80000F00000F00000F00001E00001E00001E00001E00003C00003C00003 C00003C0000780000780000780000780000F00000F00000F00000F00001E00001E00301E 00781E00F83C00F83C00F0780080700040E00021C0001F000018207D9E18>I<0007E040 001C18C0003005800060038000C0038001C0018001800100038001000380010003800100 0380000003C0000003C0000003F8000001FF800001FFE000007FF000001FF0000001F800 000078000000780000003800000038002000380020003800200030006000700060006000 6000E0007000C000E8038000C606000081F800001A217D9F1A>83 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F000 00F00000F00000F0000070010070020030040018380007C00011147C9315>99 D<007C01C207010E011C013C013802780C7BF07C00F000F000F000F00070007001700238 04183807C010147C9315>101 D<00007800019C00033C00033C00071800070000070000 0E00000E00000E00000E00000E0001FFE0001C00001C00001C00001C0000380000380000 380000380000380000700000700000700000700000700000700000E00000E00000E00000 E00000C00001C00001C0000180003180007B0000F300006600003C00001629829F0E>I< 00C001E001E001C000000000000000000000000000000E00330023004380430047008700 0E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E>105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030 000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I E /Ff 78 123 df<001F83E000F06E3001C078780380F8780300F030 07007000070070000700700007007000070070000700700007007000FFFFFF8007007000 070070000700700007007000070070000700700007007000070070000700700007007000 07007000070070000700700007007000070070000700700007007000070070007FE3FF00 1D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007000007000007000007 0000070000070000FFFFE00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F 19>I<003FE000E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E0 0700E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0 0700E00700E00700E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<7038 F87CFC7EFC7E743A0402040204020804080410081008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C 000C00180018003000300030007000600060006000E000E000E000E000E000E000E000E0 00E000E000E000E0006000600060007000300030003000180018000C000C000600020001 000080004000200B2E7DA112>I<800040002000100008000C0006000600030003000180 0180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E0 00C000C000C001C001800180018003000300060006000C00080010002000400080000B2E 7DA112>I<01800180018001800180C183F18F399C0FF003C003C00FF0399CF18FC18301 80018001800180018010147DA117>I<70F8FCFC74040404080810102040060E7C840D> 44 DI<70F8F8F87005057C840D>I<0001000300030006000600 06000C000C000C00180018001800300030003000600060006000C000C000C00180018001 800300030003000600060006000C000C000C001800180018003000300030006000600060 00C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070038070 0380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003F00012 1F7E9D17>I<018003800F80F38003800380038003800380038003800380038003800380 038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C9D17> I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007C00007 C0000780000780000F00000E00001C0000380000700000600000C0000180000300000600 400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E0020 0F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003F00000 3C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040078040 0F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00002E00 002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00200E00 200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0 141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000001000 0010000011F000161C00180E001007001007800003800003800003C00003C00003C07003 C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17>I<00 7C000182000701000E03800C07801C0780380300380000780000700000700000F1F000F2 1C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003C070 03803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF80 7FFF80400100800200800200800400000800000800001000002000002000004000004000 00C00000C00001C000018000038000038000038000038000078000078000078000078000 078000078000078000030000121F7D9D17>I<03F0000C0C001006003003002001806001 806001806001807001807803003E03003F06001FC8000FF00003F80007FC000C7E00103F 00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C 0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700700380F00380F0 0380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C000 0380000380000380000700300700780600780E00700C002018001070000FC000121F7E9D 17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F8700000 000000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0FFFFFF F00000000000000000000000000000000000000000000000000000000000000000FFFFFF F07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000 C001800180010003000200020002000200020002000000000000000000000007000F800F 800F8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C0 000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800002078 000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F 0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F 9F22>65 DI<000FC040007030C0 01C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C000040 78000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007000100 0380020001C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0 070001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000 F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E0 7C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E00460 00781820000FE0001E217D9F24>III<0F FFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0000 3C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00203C00F8 3C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>IIIII<001F800000F0F00001C0380007801E000F000F000E00 07001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F800 01F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C00 03E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F000001F 80001C217D9F23>II82 D<07E0800C1980100780300380 600180600180E00180E00080E00080E00080F00000F000007800007F00003FF0001FFC00 0FFE0003FF00001F800007800003C00003C00001C08001C08001C08001C08001C0C00180 C00380E00300F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060400F 0020400F0020C00F0030800F0010800F0010800F0010800F0010000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F800007FF FE001C1F7E9E21>II< FFF003FE1F8000F80F0000600F800060078000400780004003C0008003C0008003C00080 01E0010001E0010001F0010000F0020000F0020000F806000078040000780400003C0800 003C0800003C0800001E1000001E1000001F3000000F2000000F20000007C0000007C000 0007C000000380000003800000038000000100001F207F9E22>II<7FF83FF80FE00FC007C0070003C0020001E0040001F00C0000F0 080000781000007C1000003C2000003E4000001E4000000F8000000F8000000780000003 C0000007E0000005E0000009F0000018F8000010780000207C0000603C0000401E000080 1F0001800F0001000780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>II<7FFFF87C00F87000F06001E04001E0C0 03C0C003C0800780800F80800F00001E00001E00003C00003C0000780000F80000F00001 E00001E00003C00403C0040780040F80040F000C1E000C1E00083C00183C0018780038F8 01F8FFFFF8161F7D9E1C>I<080410082010201040204020804080408040B85CFC7EFC7E 7C3E381C0F0E7B9F17>92 D<1FE000303000781800781C00300E00000E00000E00000E00 00FE00078E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C0 14147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E 00000E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E 00780E00780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F 19>I<03F80E0C1C1E381E380C70007000F000F000F000F000F000F00070007000380138 011C020E0C03F010147E9314>I<000380003F8000038000038000038000038000038000 038000038000038000038000038003E380061B801C0780380380380380700380700380F0 0380F00380F00380F00380F00380F003807003807003803803803807801C07800E1B8003 E3F815207E9F19>I<03F0000E1C001C0E00380700380700700700700380F00380F00380 FFFF80F00000F00000F000007000007000003800801800800C010007060001F80011147F 9314>I<007C00C6018F038F07060700070007000700070007000700FFF0070007000700 0700070007000700070007000700070007000700070007000700070007007FF01020809F 0E>I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E00 1C1C001E380033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070 C00030C00030C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43 000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C00 0000000000000000000000000E007E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E0000000 000000000000000000007007F000F0007000700070007000700070007000700070007000 7000700070007000700070007000700070007000706070F060F0C061803F000C28829E0E >I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E 00000E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E 1C000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F0 00FE618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43 000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C038038 01C03801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E038 01C03801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E0 0E00F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E0 0F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000 151D7F9319>I<03E0800619801C05803C0780380380780380700380F00380F00380F003 80F00380F00380F003807003807803803803803807801C0B800E138003E3800003800003 80000380000380000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F 1E0F0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F 9312>I<1F9030704030C010C010C010E00078007F803FE00FF00070803880188018C018 C018E030D0608F800D147E9312>I<020002000200060006000E000E003E00FFF80E000E 000E000E000E000E000E000E000E000E000E000E080E080E080E080E080610031001E00D 1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319> III<7FC3FC0F01E00701C007018003810001C20000E40000EC0000 7800003800003C00007C00004E000087000107000303800201C00601E01E01E0FF07FE17 14809318>II<3FFF380E200E201C 40384078407000E001E001C00380078007010E011E011C0338027006700EFFFE10147F93 14>I E /Fg 15 119 df<70F8FCFC7404040404080810102040060F7C840E>44 D<008003800F80F380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038007C0FFFE0F217CA018 >49 D<03F0000C1C001007002007804003C04003C08003E0F003E0F801E0F801E0F801E0 2003E00003E00003C00003C0000780000700000E00001C0000180000300000600000C000 0180000100000200200400200800201800603000403FFFC07FFFC0FFFFC013217EA018> I<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>52 D<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000 E0C00060C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018 >56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C0 0001C00001C0000380000380300300780700780600700C002018001030000FC00013227E A018>I<000FE00000783C0000E00E0003C00780078003C00F0001E00E0000E01E0000F0 3C0000783C0000787C00007C7C00007C7800003C7800003CF800003EF800003EF800003E F800003EF800003EF800003EF800003EF800003EF800003E7800003C7C00007C7C00007C 3C0000783E0000F81E0000F00F0001E00F0001E0078003C003C0078000E00E0000783C00 000FE0001F247DA226>79 D<03F0200C0C601802603001E07000E0600060E00060E00060 E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC0 0007E00001E00000F00000F0000070800070800070800070800070C00060C00060E000C0 F000C0C80180C6070081FC0014247DA21B>83 D<0E0000FE00001E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C00E80600F 00300E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E 003C0E00380F00700C80600C41C0083F0017237FA21B>98 D<01FE000703000C07801C07 80380300780000700000F00000F00000F00000F00000F00000F00000F000007000007800 403800401C00800C010007060001F80012157E9416>I<01FC000707000C03801C01C038 01C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F0000070000078002038 00201C00400E008007030000FC0013157F9416>101 D<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>111 D<0E3CFE461E8F0F0F0F060F000E 000E000E000E000E000E000E000E000E000E000E000E000E000F00FFF010157F9413> 114 D<02000200020002000600060006000E001E003E00FFF80E000E000E000E000E000E 000E000E000E000E000E000E040E040E040E040E040E040708030801F00E1F7F9E13> 116 D 118 D E /Fh 17 117 df<0001FE00000007018000001C004000007800200000F000F000 00E001F00001E001F00003C001F00003C000E00003C000000003C000000003C000000003 C000000003C000000003C000000003C000000003C000000003C000000003C0000000FFFF FFF000FFFFFFF00003C001F00003C000F00003C000F00003C000F00003C000F00003C000 F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F0 0003C000F00003C000F00003C000F00003C000F00003C000F00003C000F00003C000F000 03C000F00003C000F00003C000F00003C000F00003C000F00003C000F00007E001F8007F FE1FFF807FFE1FFF8021327FB124>12 D69 D73 D77 D 80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E070 0000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF0000 0FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880 00007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0F0 0000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<003F800000E0E0000380380007003C000E00 1E001E001E001C000F003C000F007C000F0078000F8078000780F8000780F8000780FFFF FF80F8000000F8000000F8000000F8000000F8000000F8000000780000007C0000003C00 00003C0000801E0000800E0001000F0002000780020001C00C0000F03000001FC000191F 7E9E1D>101 D<0007E0001C1000383800707C00E07C01E07C01C03803C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00007E0007FFF007FFF0016327FB114>I<0780000000FF80000000FF800000000F 800000000780000000078000000007800000000780000000078000000007800000000780 000000078000000007800000000780000000078000000007800000000780000000078000 000007800000000780FE00000783078000078C03C000079001E00007A001E00007A000F0 0007C000F00007C000F000078000F000078000F000078000F000078000F000078000F000 078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F0000FC001F800FFFC1FFF80FFFC1FFF8021327EB125>104 D<07000F801F801F800F 800700000000000000000000000000000000000000000000000780FF80FF800F80078007 800780078007800780078007800780078007800780078007800780078007800780078007 8007800780078007800FC0FFF8FFF80D307EAF12>I<0780FE0000FF83078000FF8C03C0 000F9001E00007A001E00007A000F00007C000F00007C000F000078000F000078000F000 078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF80211F7E 9E25>110 D<001FC00000F0780001C01C00070007000F0007801E0003C01C0001C03C00 01E03C0001E0780000F0780000F0780000F0F80000F8F80000F8F80000F8F80000F8F800 00F8F80000F8F80000F8F80000F8780000F07C0001F03C0001E03C0001E01E0003C01E00 03C00F00078007800F0001C01C0000F07800001FC0001D1F7E9E21>I<0781FC00FF8607 00FF8803C00F9001E007A000F007C00078078000780780003C0780003C0780003E078000 1E0780001F0780001F0780001F0780001F0780001F0780001F0780001F0780001F078000 1F0780003E0780003E0780003C0780007C0780007807C000F007A000F007A001E0079803 8007860F000781F800078000000780000007800000078000000780000007800000078000 00078000000780000007800000078000000FC00000FFFC0000FFFC0000202D7E9E25>I< 0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000078000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<00400000400000400000400000400000C00000C00000C00001C00001C00003C00007C0 000FC0001FFFE0FFFFE003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C01003C01003C01003C01003C0 1003C01003C01003C01001C02001E02000E0400078C0001F00142C7FAB19>116 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 421 278 a Fh(Errata)23 b(Sheet)f(for)f(MPI)i(Sp)r (eci\014cation)806 404 y Fg(Stev)o(e)15 b(Otto)739 506 y(Octob)q(er)i(28,)f(1994)46 679 y Ff(This)11 b(do)q(cumen)o(t)f(lists) i(errors)d(in)j(the)e(Ma)o(y)g(5th)g(v)o(ersion)g(of)g(the)h(MPI)f(sp)q (eci\014cation.)20 b(Some)11 b(of)f(these)g(ha)o(v)o(e)-25 736 y(b)q(een)15 b(\014xed)f(\(at)f(the)g(galley)i(stage\))d(in)i(the)g (v)o(ersion)g(to)f(b)q(e)h(published)i(b)o(y)e(MIT)f(Press)g(in)i(the)e Fe(International)-25 792 y(Journal)j(of)h(Sup)n(er)n(c)n(omputer)g (Applic)n(ations)p Ff(,)c(and)j(are)f(noted)g(as)g(suc)o(h.)-25 935 y Fd(Errors)-25 1037 y Ff(The)h(follo)o(wing)f(are)g(kno)o(wn)g (errors)g(or)f(c)o(hanges)i(that)e(w)o(e)h(ha)o(v)o(e)g(made)g(to)g (the)g(sp)q(eci\014cation.)31 1142 y(1.)22 b(\(Fixed)14 b(in)h(IJSA)g(v)o(ersion\))f(Remo)o(v)m(al)h(of)e(complex)i(data)f(t)o (yp)q(es)g(from)f(reduce)i(op)q(erations.)k(Changes:)143 1235 y Fc(\017)k Ff(page)15 b(113,)f(line)j(36,)d(remo)o(v)o(e)g(\\)p Fb(Complex)p Ff(.")143 1308 y Fc(\017)23 b Ff(page)15 b(115,)f(line)j(42,)d(remo)o(v)o(e)g(the)i(en)o(tire)f(line)i(con)o (taining)f Fb(MPI)p 1303 1308 14 2 v 16 w(2COMPLEX)p Ff(.)143 1381 y Fc(\017)23 b Ff(page)15 b(209,)f(line)j(12,)d(remo)o(v) o(e)g(the)i(line)g(con)o(taining)g Fb(MPI)p 1175 1381 V 16 w(2COMPLEX)p Ff(.)31 1474 y(2.)22 b(\(Fixed)g(in)g(IJSA)h(v)o (ersion\))f Fa(MPI)p 674 1474 15 2 v 16 w(Status)f Ff(passed)h(as)f(p)q (oin)o(ter)i(ev)o(erywhere)e(\(an)h(issue)g(for)f(the)h(C)89 1530 y(binding)17 b(only\).)j(Changes:)143 1624 y Fc(\017)j Ff(page)15 b(21,)f(line)j(16,)d(should)i(sa)o(y:)k Fa(MPI_Status)i (*status)143 1697 y Fc(\017)h Ff(page)15 b(73,)f(line)j(33,)d(should)i (sa)o(y:)k Fa(MPI_Status)i(*status)143 1769 y Fc(\017)h Ff(page)15 b(52,)f(line)j(6,)d(should)j(sa)o(y:)i Fa(MPI_Status)j (*status)143 1842 y Fc(\017)h Ff(page)15 b(211,)f(line)j(24,)d(should)i (sa)o(y:)j Fa(MPI_Status)k(*status)143 1915 y Fc(\017)g Ff(page)15 b(212,)f(line)j(26,)d(should)i(sa)o(y:)j Fa(MPI_Status)k (*status)143 1987 y Fc(\017)g Ff(page)15 b(213,)f(line)j(35,)d(should)i (sa)o(y:)j Fa(MPI_Status)k(*status)31 2081 y Ff(3.)f(\(Fixed)16 b(in)g(IJSA)h(v)o(ersion\))f(P)o(oin)o(ters)f(to)g(storage)f(area)h (should)i(b)q(e)f(of)g(t)o(yp)q(e)f Fa(MPI)p 1502 2081 V 17 w(Aint)p Ff(,)g(rather)g(than)89 2137 y Fa(int)p Ff(.)k(Changes:)143 2231 y Fc(\017)k Ff(page)15 b(68,)f(line)j(27,)d (should)i(sa)o(y:)k Fa(MPI_Aint)i(*extent)143 2304 y Fc(\017)h Ff(page)15 b(68,)f(line)j(39,)d(should)i(sa)o(y:)k Fa(MPI_Aint)i(*size)143 2376 y Fc(\017)h Ff(page)15 b(70,)f(line)j(15,) d(should)i(sa)o(y:)k Fa(MPI_Aint)i(*displacement)143 2449 y Fc(\017)h Ff(page)15 b(70,)f(line)j(26,)d(should)i(sa)o(y:)k Fa(MPI_Aint)i(*displacement)143 2522 y Fc(\017)h Ff(page)15 b(213,)f(line)j(24,)d(should)i(sa)o(y:)j Fa(MPI_Aint)k(*extent)143 2594 y Fc(\017)g Ff(page)15 b(213,)f(line)j(25,)d(should)i(sa)o(y:)j Fa(MPI_Aint)k(*size)910 2719 y Ff(1)p eop %%Page: 2 2 2 1 bop 143 95 a Fc(\017)23 b Ff(page)15 b(213,)f(line)j(29,)d(should)i (sa)o(y:)j Fa(MPI_Aint)k(*displacement)143 169 y Fc(\017)g Ff(page)15 b(213,)f(line)j(30,)d(should)i(sa)o(y:)j Fa(MPI_Aint)k (*displacement)31 262 y Ff(4.)f(\(Fixed)15 b(in)h(IJSA)h(v)o(ersion\)) 143 356 y Fc(\017)23 b Ff(page)15 b(83,)f(lines)j(36,)d(42,)g(48,)h(c)o (hange)g("outcoun)o(t")f(to)g("outsize")143 429 y Fc(\017)23 b Ff(page)15 b(84,)f(lines)j(1,)d(4,)h(c)o(hange)g("OUTCOUNT")g(to)g ("OUTSIZE")143 502 y Fc(\017)23 b Ff(page)15 b(221,)f(lines)j(8,)d(11,) g(c)o(hange)i("OUTCOUNT")f(to)f("OUTSIZE")31 596 y(5.)22 b(\(Fixed)15 b(in)h(IJSA)h(v)o(ersion\))e(In)h(section)f(3.2.2,)e(the)j (sen)o(tence)g(fragmen)o(t:)189 690 y(MPI)p 281 690 14 2 v 16 w(DOUBLE)p 492 690 V 17 w(COMPLEX)e(for)g(double)h(precision)h (complex)f(in)f(F)l(ortran)f(declared)j(to)189 746 y(b)q(e)g(of)e(t)o (yp)q(e)i(DOUBLE)f(PRECISION;)89 840 y(should)h(b)q(e)189 934 y(MPI)p 281 934 V 16 w(DOUBLE)p 492 934 V 17 w(COMPLEX)e(for)g (double)h(precision)h(complex)f(in)f(F)l(ortran)f(declared)j(to)189 990 y(b)q(e)g(of)e(t)o(yp)q(e)i(DOUBLE)f(COMPLEX;)31 1084 y(6.)22 b(The)11 b(w)o(ording)h(of)f(the)g(description)i(of)e Fa(MPI)p 822 1084 15 2 v 17 w(GROUP)p 959 1084 V 17 w(RANGE)p 1096 1084 V 16 w(EXCL)g Ff(is)h(confusing)g(but)g(I)f(ha)o(v)o(en't)g (decided)89 1141 y(what)j(to)h(c)o(hange)g(it)h(to,)e(y)o(et.)19 b(The)d(function)g(protot)o(yp)q(es)e(are)h(correct,)f(I)i(b)q(eliev)o (e.)31 1235 y(7.)22 b(The)15 b(example)h(on)f(page)g(148.)k(The)d (example)g(w)o(ould)f(b)q(e)h(more)f(illustrious)i(if)89 1347 y Fa(MPI_Reduce\(...,MPI_SUM,)j(1,)k(commslave\))f(->)g (MPI_Reduce\(...,MPI_SUM,)e(0,)i(commslave\))89 1460 y Ff(The)12 b(comm)o(unicator)g Fa(commslave)f Ff(is)i(comp)q(osed)f (of)g(the)g(group)g Fa(grprem)g Ff(whic)o(h)h(is)f Fa(MPI)p 1598 1460 V 17 w(GROUP)p 1735 1460 V 17 w(WORLD)89 1516 y Ff(without)f(pro)q(cess)g(0.)19 b(As)11 b(suc)o(h,)h Fa(grprem)e Ff(is)i(n)o(um)o(b)q(ered)f(from)g(0)g(to)f(\(Numpro)q(cs)h (-)h(2\))e(where)i(Numpro)q(cs)89 1572 y(is)i(the)g(size)h(of)e Fa(MPI)p 420 1572 V 17 w(GROUP)p 557 1572 V 17 w(WORLD)p Ff(.)g(Making)g(the)h(reduce)h(go)f(to)f(pro)q(cess)h(0)g(mak)o(es)f (the)h(ren)o(um)o(b)q(ering)89 1629 y(clear.)25 b(It)17 b(also)g(mak)o(es)f(it)h(w)o(ork)f(for)g(only)h(2)g(no)q(des)g(-)g(the) g(curren)o(t)g(co)q(de)g(fails)h(since)g Fa(grprem)e Ff(w)o(ould)89 1685 y(only)f(ha)o(v)o(e)g(one)h(no)q(de/pro)q(cess.)31 1779 y(8.)22 b(\(Fixed)15 b(in)h(IJSA)h(v)o(ersion\))143 1873 y Fc(\017)23 b Ff(page)15 b(150,)f(line)j(16,)d(should)i(sa)o(y:)j Fa(MPI_Comm)k(comm)31 1967 y Ff(9.)f(P)o(age)12 b(151.)18 b(This)13 b(example)g(seems)g(to)f(imply)h(that)f(MPI)p 1065 1967 14 2 v 17 w(Comm)p 1214 1967 V 15 w(rank)h(returns)f(MPI)p 1575 1967 V 16 w(UNDEFINED)89 2023 y(in)18 b(ma)f(and)g(m)o(b)h(if)g (the)f(calling)i(pro)q(cess)f(is)g(not)f(in)h(comm)p 1131 2023 V 16 w(a)f(or)g(comm)p 1364 2023 V 16 w(b,)h(resp)q(ectiv)o (ely)l(.)28 b(142)17 b(sa)o(ys)89 2080 y(nothing)f(ab)q(out)f(this.)22 b(If)15 b(this)h(w)o(as)f(the)h(in)o(ten)o(t)f(\(that)g(is)h(w)o(ould)g (return)f(MPI)p 1453 2080 V 17 w(UNDEFINED\))g(then)89 2136 y(the)j(text)f(ab)q(out)g(MPI)p 491 2136 V 17 w(COMM)p 660 2136 V 16 w(RANK)h(should)h(sa)o(y)e(this.)28 b(This)18 b(commen)o(t)g(migh)o(t)f(also)h(apply)h(to)89 2193 y(MPI)p 181 2193 V 16 w(GR)o(OUP)p 365 2193 V 17 w(RANK)13 b(on)f(135)f(if)i(w) o(e)f(w)o(an)o(t)f(it)h(to)g(act)g(the)g(same.)18 b(Their)13 b(b)q(eing)h(the)e(same)g(is)h(implied)89 2249 y(in)j(the)f(commen)o (ts)g(in)h(the)f(text.)8 2343 y(10.)22 b(P)o(age)14 b(159.)19 b(It)c(w)o(ould)h(b)q(e)g(a)f(little)h(cleaner)h(to)d(add:)89 2455 y Fa(MPI_Comm_free\(&myComm\);)89 2568 y Ff(This)i(is)f(done)h (for)e(Example)i(2)f(on)g(page)g(160.)910 2719 y(2)p eop %%Page: 3 3 3 2 bop 8 95 a Ff(11.)22 b(P)o(age)16 b(162)f(Instead)j(of)e(\(MPI)p 611 95 14 2 v 16 w(Comm\))f(0)i(I)g(think)g(it)g(should)h(b)q(e)f(MPI)p 1331 95 V 16 w(COMM)p 1499 95 V 16 w(NULL.)h(145:8)d(sa)o(ys)89 152 y(MPI)p 181 152 V 16 w(Comm)p 329 152 V 16 w(split)f(returns)f(MPI) p 688 152 V 16 w(COMM)p 856 152 V 16 w(NULL)h(when)f(it)h(has)e(funn)o (y)i(cases.)19 b(I)13 b(don't)g(recall)h(read-)89 208 y(ing)h(0)g(and)h(NULL)g(w)o(ere)f(the)g(same)g(for)g(this)g(case.)8 302 y(12.)22 b(P)o(age)14 b(162.)89 415 y Fa(MPI_Attr_keyval_create)21 b(->)i(MPI_Keyval_create)89 527 y Ff(as)15 b(p)q(er)g(page)g(166.)k (Same)c(commen)o(t)g(applies)i(to)e(page)g(170.)8 621 y(13.)22 b(P)o(age)14 b(165.)89 733 y Fa(MPI_Intercomm_create\(loc)o (al_lead)o(er,)21 b(local_comm,)h(buffer[0],new_world,)f(..\))j(->)89 790 y(MPI_Intercomm_create\(loc)o(al_comm)o(,)d(local_leader,)h (new_world,)h(buffer[0],)f(..\))89 902 y Ff(as)15 b(de\014ned)h(on)f (157.)8 996 y(14.)22 b(P)o(age)10 b(167)g(It)i(seems)f(to)f(me)i(that)e (the)h(F)l(ortran)f(de\014nition)j(needs)f(IERR)o(OR.)h(167)d(sa)o(ys)g (that)h(it)g(returns)89 1053 y(MPI)p 181 1053 V 16 w(SUCCESS)17 b(if)g(it)g(w)o(orks.)23 b(Only)17 b(the)g(C)f(v)o(ersion)h(seems)f(to) g(ha)o(v)o(e)g(a)h(w)o(a)o(y)e(to)h(return)g(this.)25 b(The)89 1109 y(same)15 b(commen)o(t)f(applies)j(to:)i(167)c(for)f (DELETE)p 968 1109 V 16 w(FUNCTION)8 1203 y(15.)22 b(P)o(age)13 b(167)g(Do)h(w)o(e)g(w)o(an)o(t)f(MPI)p 620 1203 V 16 w(NULL)p 760 1203 V 18 w(FN,)g(MPI)p 957 1203 V 16 w(DUP)p 1073 1203 V 17 w(FN)g(in)i(App)q(endix)i(A?)d(It)g(is)h(a)f(function)h (but)89 1259 y(some)g(are)g(listed)8 1353 y(16.)22 b(P)o(age)14 b(167)h(MPI)p 375 1353 V 16 w(KEYV)-5 b(AL)p 582 1353 V 17 w(INV)g(ALID)17 b(seems)e(to)f(b)q(e)i(missing)g(from)f(App)q (endix)i(A.)8 1447 y(17.)22 b(P)o(age)14 b(169,)g(line)j(38,)d(A)i(n)o (ull)g(Rational)g(should)g(b)q(e)g(remo)o(v)o(ed.)8 1541 y(18.)22 b(P)o(age)14 b(170)h(MPI)p 375 1541 V 16 w(Ab)q(ort)g(tak)o (es)f(a)h(comm)g(and)g(error)g(co)q(de,)g(not)g(string)g(as)g (de\014ned)i(P)o(age)d(197.)8 1635 y(19.)22 b(P)o(age)14 b(171)89 1747 y Fa(MPI_Comm)23 b(comm)g(->)g(MPI_Comm)g(*comm.)89 1860 y Ff(See)16 b(de\014nition)h(on)e(167.)k(And,)c(F)l(or)g(the)g (same)g(reason,)89 1972 y Fa(int)23 b(keyval)g(->)h(int)f(*keyval)89 2085 y Ff(And)89 2197 y Fa(gop_stuff_copier\(...,)e(gop_stuff,)h (extra\))h(->)89 2254 y(gop_stuff_copier\(...,)e(extra,)i (gop_stuff_in,)f(gop_stuff_out,)g(flag\))89 2366 y Ff(p)q(er)15 b(de\014nition)i(on)f(page)f(166.)k(And)89 2479 y Fa(MPI_Comm)k(comm)g (->)g(MPI_Comm)g(*comm)89 2591 y Ff(And)910 2719 y(3)p eop %%Page: 4 4 4 3 bop 89 95 a Fa(int)23 b(keyval)g(->)h(int)f(*keyval.)89 205 y Ff(Also)15 b(need)h(to)f(add)g Fa(*flag)g Ff(for)f(ab)q(o)o(v)o (e)h(c)o(hange)89 315 y Fa(gop_stuff_type)22 b(*gop_stuff)g(->)i (gop_stuff_type)e(*gop_stuff_in,)g(**gop_stuff_out)89 425 y Ff(And,)15 b(I)h(think)g(w)o(e)f(need)h(to)e(add)i(the)f(line:)89 535 y Fa(*gop_stuff_out)22 b(=)h(gop_stuff_in;)89 644 y Ff(Righ)o(t)18 b(no)o(w,)f(the)g(routine)h(do)q(es)g(not)f(app)q(ear) h(to)f(cop)o(y)g(an)o(ything;)h(only)g(incremen)o(ts)h(the)e(reference) 89 701 y(coun)o(t.)i(And,)d(need)g(to)e(mak)o(e)89 811 y Fa(gop_stuff)22 b(->)i(gop_stuff_in)89 920 y Ff(if)15 b(y)o(ou)g(mak)o(e)g(the)g(other)g(c)o(hanges.)8 1013 y(20.)22 b(P)o(age)14 b(215,)g(line)j(31:)89 1123 y Fa(int)23 b(MPI_Comm_compare\()f(MPI_Comm)g(comm1,)i(comm2,)f(int)g(*result)g(\)) 89 1233 y Ff(should)16 b(b)q(e)89 1343 y Fa(int)23 b (MPI_Comm_compare\()f(MPI_Comm)g(comm1,)i(MPI_Comm)e(comm2,)h(int)h (*result)f(\))8 1452 y Ff(21.)f(P)o(age)14 b(142,)g(line)j(39.)i(The)d (sen)o(tence)g(should)g(read:)89 1527 y Fb(MPI)p 174 1527 14 2 v 16 w(IDENT)d Ff(results)h(if)g(and)g(only)h(if)f Fb(comm1)d Ff(and)j Fb(comm2)e Ff(are)h(handles)i(to)e(the)h(same)g(ob) s(ject)f(\(iden-)89 1584 y(tical)j(groups)f(and)g(same)g(con)o(texts\)) f(or)h(b)q(oth)g(are)g(n)o(ull.)8 1676 y(22.)76 b Fc(\017)23 b Ff(page)15 b(47)f(Example)i(3.14,)e(line)j(35)189 1759 y Fa(CALL)23 b(MPI_IRECV\(a\(i,1\),)f(n,)h(MPI_REAL,)g(0,)g(tag,)189 1842 y Ff(should)16 b(b)q(e)g(replaced)g(with)189 1924 y Fa(CALL)23 b(MPI_IRECV\(a\(i,1\),)f(n,)h(MPI_REAL,)g(i,)g(tag,)143 2007 y Fc(\017)g Ff(page)15 b(47)f(Example)i(3.14,)e(line)j(41)189 2089 y Fa(CALL)23 b(MPI_IRECV\(a\(1,index\),)e(n,)j(MPI_REAL,)e(0,)i (tag,)189 2172 y Ff(should)16 b(b)q(e)g(replaced)g(with)189 2254 y Fa(CALL)23 b(MPI_IRECV\(a\(1,)f(index\),)h(n,)g(MPI_REAL,)g (index,)g(tag,)143 2337 y Fc(\017)g Ff(page)15 b(48,)f(Example)i(3.15,) d(line)k(11)189 2419 y Fa(CALL)23 b(MPI_IRECV\(a\(i,1\),)f(n,)h (MPI_REAL,)g(0,)g(tag,)189 2502 y Ff(should)16 b(b)q(e)g(replaced)g (with)189 2584 y Fa(CALL)23 b(MPI_IRECV\(a\(i,1\),)f(n,)h(MPI_REAL,)g (i,)g(tag,)910 2719 y Ff(4)p eop %%Page: 5 5 5 4 bop 143 95 a Fc(\017)23 b Ff(page)15 b(48,)f(Example)i(3.15,)d (line)k(19)189 179 y Fa(CALL)23 b(MPI_IRECV\(a\(1,)f (index_list\(i\)\),)g(n,)h(MPI_REAL,)g(0,)h(tag,)451 235 y(comm,)f(request_list\(i\),)f(ierr\))189 319 y Ff(should)16 b(b)q(e)g(replaced)g(with)189 402 y Fa(CALL)23 b(MPI_IRECV\(a\(1,)f (index_list\(i\)\),)g(n,)h(MPI_REAL,)g(index_list\(i\),)f(tag,)451 459 y(comm,)h(request_list\(index_list\(i\)\),)d(ierr\))-25 602 y Fd(Unresolv)n(ed)j(Issues)-25 703 y Ff(The)18 b(follo)o(wing)g (are)f(some)h(issues)g(that)f(are)g(still)i(b)q(eing)g(discussed)g(and) f(that)f(ha)o(v)o(en't)g(y)o(et)g(reac)o(hed)h(con-)-25 760 y(sensus.)31 866 y(1.)k(An)15 b(issue)i(related)f(to)e(error)h(n)o (um)o(b)q(er)h(2:)k(Should)d(the)e(argumen)o(ts)g(POSITION)i(and)f (OUTCOUNT)89 923 y(in)h Fa(MPI)p 218 923 15 2 v 17 w(Pack)f Ff(b)q(e)h(of)f(t)o(yp)q(e)h Fa(int)f Ff(\(as)f(they)i(are)f(no)o(w\))g (or)g(of)g(t)o(yp)q(e)h Fa(MPI)p 1310 923 V 16 w(Aint)p Ff(?)24 b(That)16 b(is,)h(do)g(w)o(e)f(w)o(an)o(t)89 979 y(to)e(allo)o(w)i(for)e(a)h(bu\013er)g(with)h(more)f(than)g(2)g(Gb) o(ytes?)31 1073 y(2.)22 b(In)16 b(lines)g(26)f(and)g(27,)g(page)g(88)f (of)h(the)g(Ma)o(y)g(5)f(MPI)i(draft,)e(shouldn't)h Fa(&lbuf)g Ff(just)g(b)q(e)h Fa(lbuf)e Ff(?)31 1167 y(3.)22 b(There)15 b(are)g(some)f(inconsistencies)k(in)e(the)f(description)h(of)f(the)g (use)g(of)f(MPI)p 1427 1167 14 2 v 17 w(NULL)p 1568 1167 V 17 w(FN)h(when)g(used)89 1223 y(with)e(MPI)p 282 1223 V 16 w(KEYV)-5 b(AL)p 489 1223 V 17 w(CREA)l(TE.)13 b(In)g(particular,) h(when)f(describing)h(the)f("cop)o(y)p 1516 1223 V 16 w(function",)g(w)o(e)g(sa)o(y)189 1317 y(MPI)p 281 1317 V 16 w(NULL)p 421 1317 V 17 w(FN)i(is)h(a)f(function)h(that)e(do)q(es)i (nothing)g(other)e(than)i(returning)f(\015ag)g(=)h(0.)89 1411 y(This)f(is)g(inconsisten)o(t)h(with)f(a)g(n)o(ull)h(function)f (\(b)o(y)g(de\014nition,)h(a)f(n)o(ull)h(function)f(should)h(do)f (nothing\))89 1467 y(and)g(will)i(fail)f(when)g(used)f(as)g(the)h ("delete)p 838 1467 V 17 w(function",)f(where)h(w)o(e)f(sa)o(y)189 1561 y(It)g(ma)o(y)f(b)q(e)i(sp)q(eci\014ed)h(as)e(the)g(n)o(ull)h (function)g(p)q(oin)o(ter)f(in)h(C)f(or)g(as)f(MPI)p 1442 1561 V 17 w(NULL)p 1583 1561 V 17 w(FN)h(from)189 1617 y(either)h(C)f(or)f(F)o(OR)l(TRAN,)i(in)g(whic)o(h)g(case)f(no)g (delete)i(callbac)o(k)f(o)q(ccurs)f(for)g(k)o(eyv)m(al.)89 1711 y(There)g(are)g(t)o(w)o(o)f(ob)o(vious)h(\014xes:)20 b(1\))15 b(Replace)i(MPI)p 984 1711 V 16 w(NULL)p 1124 1711 V 17 w(FN)e(with)h(MPI)p 1413 1711 V 16 w(NULL)p 1553 1711 V 17 w(COPY)p 1703 1711 V 17 w(FN)f(\(for)89 1768 y(correctness\))9 b(and)i(MPI)p 510 1768 V 16 w(NULL)p 650 1768 V 18 w(DELETE)p 857 1768 V 16 w(FN)e(\(for)h(consistency\))g (in)h(the)g(discussion)h(of)d(MPI)p 1745 1768 V 17 w(KEYV)-5 b(AL)p 1953 1768 V 17 w(CREA)l(TE.)89 1824 y(2\))14 b(Change)h(the)h(w) o(ording)f(under)h("cop)o(y)p 806 1824 V 16 w(function")f(to)189 1918 y(MPI)p 281 1918 V 16 w(NULL)p 421 1918 V 17 w(FN)e(is)h(a)e (function)i(that)e(do)q(es)h(nothing;)h(when)g(used)f(as)g(the)g(cop)o (y)f(function,)189 1974 y(it)k(has)f(the)h(same)f(e\013ect)g(as)h(a)f (function)h(that)f(do)q(es)h(nothing)g(other)g(than)f(returning)h (\015ag)189 2031 y(=)f(0.)89 2125 y(I)i(prefer)g(\(1\))e(b)q(ecause)j (it)f(is)g(clearer)g(as)f(to)g(what)h(is)g(going)f(on)h(and)g (\(naturally\))f(mak)o(es)h(the)f(imple-)89 2181 y(men)o(tation)f (simpler.)21 b(Implemen)o(ting)c(\(2\))d(is)i(straigh)o(tforw)o(ard)d (but)i(unpleasan)o(t.)31 2275 y(4.)22 b(Related)16 b(to)f(previous)h (item,)f(w)o(e)g(ha)o(v)o(e)f(\(F)l(rom)h(Hub)q(ertus)g(F)l(rank)o (e\):)89 2350 y(I)g(extract)g(this)g(out)g(of)g(the)g(MPI-F)g(include)j (\014les.)89 2463 y Fa(#define)23 b(MPI_DUP_FN)70 b(mpi_dup_fn)89 2519 y(/*)23 b(note)h(the)f(draft)g(provides)g(MPI_NULL_FN,)f(however)h (this)g(violates)113 2576 y(*)g(the)h(prototype)e(differences)h(in)g (usage)h(for)f(copy)g(and)h(delete)f(function)910 2719 y Ff(5)p eop %%Page: 6 6 6 5 bop 113 95 a Fa(*)23 b(In)h(agreement)f(with)g(the)g(public)g (domain)g(libary)g(developers)g(we)113 152 y(*)g(distinguish)g(these)g (functions)160 208 y(#define)g(MPI_NULL_FN)113 265 y(*/)89 321 y(#define)g(MPI_NULL_COPY_FN)93 b(...)89 378 y(#define)23 b(MPI_NULL_DELETE_FN)45 b(...)89 490 y Ff(I'v)o(e)19 b(discussed)i(this)f(with)g(Nathan)g(Doss)f(and)g(I)h(b)q(eliev)o(e)i (this)e(has)f(b)q(een)i(incorp)q(orated)f(in)o(to)g(the)89 547 y(public)d(domain)f(v)o(ersion)f(as)g(w)o(ell.)31 641 y(5.)22 b(There)14 b(is)h(a)e(long,)i(still)g(unresolv)o(ed)g (discussion)h(cen)o(tering)f(around)f(P)l(A)o(CK/UNP)l(A)o(CK.)g(A)o(t) f(the)h(end)89 697 y(of)g(it,)i(\(at)e(least)h(the)g(last)h(message)e (I)i(see\),)f(Marc)f(Snir)i(sa)o(ys:)89 772 y(Conclusion:)143 866 y Fc(\017)23 b Ff(a\))16 b(A)g(message)g(that)g(w)o(as)f(created)i (b)o(y)f(one)h(or)e(sev)o(eral)i(sequences)h(of)e(related)g(P)l(A)o(CK) h(calls)g(is)189 922 y(sen)o(t)e(with)g(t)o(yp)q(e)g(MPI)p 581 922 14 2 v 17 w(P)l(A)o(CKED.)143 995 y Fc(\017)23 b Ff(b\))c(A)h(message)f(that)g(is)h(going)g(to)f(b)q(e)i(unpac)o(k)o (ed)f(b)o(y)g(one)f(or)h(sev)o(eral)f(sequences)i(of)f(related)189 1052 y(UNP)l(A)o(CK)15 b(calls)h(is)g(receiv)o(ed)h(with)e(t)o(yp)q(e)g (MPI)p 1019 1052 V 17 w(P)l(A)o(CKED.)143 1125 y Fc(\017)23 b Ff(c\))d(A)g(Message)f(created)h(b)o(y)g(one)h(sequence)g(of)f (related)g(P)l(A)o(CK)g(calls,)i(and)f(sen)o(t)e(with)i(t)o(yp)q(e)189 1181 y(MPI)p 281 1181 V 16 w(P)l(A)o(CKED)15 b(is)h(iden)o(tical)h(to)d (a)h(message)g(sen)o(t)g(with)h(a)e(regular)i(send.)89 1275 y(I)f(don't)g(understand)h(wh)o(y)e(y)o(ou)h(came)g(to)g(the)g (conclusion)i(it)e(is)h(wrong)e(to)h(send)h(the)f(concatenation)89 1332 y(of)f(sev)o(eral)i(pac)o(king)g(units)f(with)h(t)o(yp)q(e)f(MPI)p 868 1332 V 17 w(P)l(A)o(CKED.)89 1407 y(\(end)g(of)g(Marc\).)k(Where)c (do)g(w)o(e)g(stand)g(here?)31 1501 y(6.)22 b(MPI)p 181 1501 V 16 w(BOTTOM)15 b(in)i(F)l(ortran.)89 1576 y(There)k(is)h(a)f (bug)h(\(or)f(unin)o(tended)i(e\013ect\))e(in)h(the)g(de\014nition)h (of)e(MPI)p 1385 1576 V 16 w(BOTTOM)h(for)e(F)l(ortran.)89 1632 y(Sp)q(eci\014cally)l(,)g(MPI)p 429 1632 V 17 w(BOTTOM)d(can)h(b)q (e)f(used)h(as)f(a)g(bu\013er)g(address;)h(in)g(C)f(the)g(appropriate)h (v)m(alue)89 1689 y(will)12 b(often)e(b)q(e)h(\(v)o(oid)g(*\)0.)17 b(But)10 b(in)i(F)l(ortran,)e(there)g(are)g(no)h(p)q(oin)o(ter)g(t)o (yp)q(es;)g(in)g(most)f(implemen)o(tations,)89 1745 y(all)16 b(v)m(alues)g(are)f(passed)h(b)o(y)f(reference.)20 b(No)o(w)15 b(consider)h(the)f(follo)o(wing)h(t)o(w)o(o)e(co)q(de)i(fragmen)o(ts:) 89 1858 y Fa(C:)89 1914 y(void)23 b(*p;)89 2027 y(p)g(=)h(\(void)f (*\)MPI_BOTTOM;)89 2083 y(MPI_Send\()f(p,)i(...)f(\);)89 2208 y(Fortran:)89 2265 y(integer)g(p)89 2378 y(p)g(=)h(MPI_BOTTOM;)89 2434 y(MPI_Send\()e(p,)i(...)f(\);)89 2559 y Ff(The)15 b(C)g(case)g(is)h(clearly)g(allo)o(w)o(ed;)g(the)f(F)l(ortran)f(case)h (m)o(ust)g(not)g(b)q(e.)910 2719 y(6)p eop %%Page: 7 7 7 6 bop 89 95 a Ff(The)15 b(only)h(w)o(a)o(y)e(that)h(I)h(can)f(see)h (to)f(pro)o(vide)g(MPI)p 971 95 14 2 v 17 w(BOTTOM)g(to)g(F)l(ortran)f (is)i(to)f(mak)o(e)g(it)g(a)g(sp)q(ecial)89 152 y(constan)o(t)20 b(that)h(is)g(lik)o(e)i(a)d(reserv)o(ed)i(w)o(ord)e(-)i(assigning)f(it) h(to)e(a)h(v)m(ariable)i(is)f(erroneous)f(\(that)f(is,)89 208 y(there)15 b(is)g(a)f(sp)q(ecial)j(lo)q(cation)e(that)f(is)i(kno)o (wn)e(as)h(MPI)p 1042 208 V 16 w(BOTTOM;)f(the)h(routines)g(can)g(test) g(for)f(that)89 265 y(sp)q(ecial)20 b(lo)q(cation\).)31 b(This)19 b(ma)o(y)f(seem)h(ob)o(vious,)h(but)e(it)h(is)h(easily)f(o)o (v)o(erlo)q(ok)o(ed.)30 b(W)l(e)19 b(certainly)h(did)89 321 y(while)d(writing)g(the)f(mo)q(del)h(implemen)o(tation.)24 b(I)17 b(b)q(eliev)o(e)h(that)e(this)g(merits)g(some)g(commen)o(t)g(in) h(the)89 378 y(standard,)11 b(since)h(to)e(F)l(ortran,)h(MPI)p 713 378 V 16 w(BOTTOM)g(is)h(not)e(a)h("constan)o(t")f(as)g(de\014ned)j (b)o(y)d(that)h(standard.)31 472 y(7.)22 b(The)14 b(requiremen)o(t)h (for)e(an)h(En)o(vironmen)o(tal)h(Enquiry)g(that)e(allo)o(ws)h(to)g (\014nd)h(whether)f(t)o(w)o(o)f(no)q(des)i(use)89 528 y(the)f(same)f(data)g(represen)o(tation)h(is)h(reasonable,)f(irresp)q (ectiv)o(e)h(of)f(the)g(sp)q(eci\014c)h(use)g(that)e(Rolf)h(has)g(in)89 584 y(mind,)k(for)f(P)o(armacs)g(em)o(ulation.)27 b(More)17 b(generally)l(,)i(users)e(ma)o(y)g(w)o(an)o(t)g(enquiry)h(functions)g (to)f(\014nd)89 641 y(the)e(t)o(yp)q(e)g(of)f(no)q(de)i(on)f(whic)o(h)h (a)e(pro)q(cess)i(runs.)k(This)15 b(w)o(ould)h(help)g(in)g(situations)f (where)g(the)g(user)g(is)89 697 y(willing)i(to)d(run)i(on)f(an)o(y)f(a) o(v)m(ailable)j(mac)o(hine)f(on)f(a)f(net)o(w)o(ork,)g(but)h(ma)o(y)f (w)o(an)o(t)g(to)h(use)g(di\013eren)o(t)g(co)q(de)89 754 y(on)g(di\013eren)o(t)g(mac)o(hine)h(t)o(yp)q(es.)89 829 y(It)g(is)g(not)f(reasonable)i(to)e(assume)g(that)g(the)h(MPI)g (forum)f(will)j(dev)o(elop)f(its)f(o)o(wn)f(nomenclature)i(for)89 885 y(data)g(represen)o(tations)g(\(size)h(of)f(eac)o(h)h(basic)g (language)g(data)f(t)o(yp)q(e,)h(big)g(vs)f(little)i(endian,)g (\015oating)89 942 y(p)q(oin)o(t)14 b(format,)e(c)o(haracter)h(co)q (des,)h(etc.\).)k(First)c(b)q(ecause)g(other)f(standards)g(try)g(to)g (handle)i(this)f(issue)89 998 y(and,)22 b(second,)g(b)q(ecause)f(a)g (complete)g(c)o(haracterization)g(of)f(the)h(data)f(represen)o(tation)g (used)i(b)o(y)e(a)89 1055 y(pro)q(cess)e(\(whic)o(h)h(dep)q(ends)h(on)e (the)h(underlying)h(mac)o(hine)f(arc)o(hitecture)g(and)f(on)h(the)f (compiler\))h(is)89 1111 y(lik)o(ely)e(to)d(b)q(e)i(quite)g(length)o(y) l(.)89 1186 y(W)l(e)f(agreed)f(that)h(v)o(endors)f(can)h(attac)o(h)f (implemen)o(tation)i(sp)q(eci\014c)h(prede\014ned)f(attributes)f(that)f (are)89 1243 y(asso)q(ciated)h(with)h(MPI)p 501 1243 V 16 w(COMM)p 669 1243 V 16 w(W)o(ORLD.)f(I)h(w)o(ould)f(suggest)g(the) g(follo)o(wing:)89 1318 y(Asso)q(ciate)f(a)f(prede\014ned)i(attribute)f (k)o(ey)f(MPI)p 907 1318 V 17 w(PR)o(OC)p 1055 1318 V 16 w(TYPE)h(with)g(MPI)p 1405 1318 V 16 w(COMM)p 1573 1318 V 16 w(W)o(ORLD.)g(The)89 1374 y(v)m(alue)k(of)f(this)g(attribute) g(enco)q(des)h(the)f("t)o(yp)q(e")g(of)f(the)h(executing)i(pro)q(cess,) e(i.e.)26 b(the)17 b(t)o(yp)q(e)g(of)g(data)89 1431 y(represen)o (tation)e(used)i(b)o(y)e(this)h(pro)q(cess.)22 b(The)16 b(meaning)g(of)g(the)f(v)m(alues)i(of)e(this)i(attribute)e(is)h(imple-) 89 1487 y(men)o(tation)h(dep)q(enden)o(t,)j(except)e(that)f(if)h(t)o(w) o(o)f(pro)q(cesses)h(return)g(the)g(same)f(v)m(alue,)j(then)e(they)g (use)89 1544 y(the)d(same)g(data)f(represen)o(tation)i(\(i.e.,)e(data)h (can)g(b)q(e)h(mo)o(v)o(ed)f(un)o(t)o(yp)q(ed)g(b)q(et)o(w)o(een)h (these)f(pro)q(cesses\).)89 1619 y(Implemen)o(ters)e(of)f(MPI)g(for)g (homogeneous)g(SPMD)g(systems)g(just)g(need)h(to)f(alw)o(a)o(ys)g (return)g(the)g(same)89 1675 y(v)m(alue)17 b(\(0?\))k(with)15 b(this)i(attribute.)j(Implemen)o(ters)d(of)e(MPI)h(on)f(heterogeneous)h (systems)f(ma)o(y)g(need)89 1732 y(some)g(w)o(ork.)89 1807 y(If)k(this)h(suggestion)f(sounds)h(reasonable,)g(then)f(the)h(v)m (arious)f(MPI)g(implemen)o(ters)i(ma)o(y)d(accept)i(it)89 1863 y(no)o(w)14 b(as)h("common)g(practice",)g(and)g(w)o(e)g(can,)g(at) g(a)g(latter)g(time,)g(add)g(this)h(to)e(the)i(standard.)910 2719 y(7)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Fri Mar 10 17:03:49 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA18584; Fri, 10 Mar 1995 17:03:48 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA19629; Fri, 10 Mar 1995 17:04:14 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 10 Mar 1995 17:04:10 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id RAA19564; Fri, 10 Mar 1995 17:04:00 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA28885; Fri, 10 Mar 1995 16:03:56 -0600 Message-Id: <199503102203.QAA28885@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann), itf@mcs.anl.gov Subject: shortspawn.ps Date: Fri, 10 Mar 1995 16:03:55 -0600 From: Rusty Lusk %!PS-Adobe-2.0 %%Creator: dvips 5.47 Copyright 1986-91 Radical Eye Software %%Title: spawn_proposal.dvi %%Pages: 5 1 %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 200 dict def TeXDict begin /N /def load def /B{bind def}N /S /exch load def /X{S N}B /TR /translate load N /isls false N /vsize 10 N /@rigin{ isls{[0 1 -1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale Resolution VResolution vsize neg mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@letter{/vsize 10 N}B /@landscape{/isls true N /vsize -1 N}B /@a4{/vsize 10.6929133858 N}B /@a3{ /vsize 15.5531 N}B /@ledger{/vsize 16 N}B /@legal{/vsize 13 N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail} B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image} imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto}N /eop{clear SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for}N /p /show load N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval(Display)eq}{pop false}ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /a{ moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{ S p tail}B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w }B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{clear SS restore}B end %%EndProcSet TeXDict begin 1000 300 300 @start /Fa 11 122 df66 D<0007FFFE000000007FFFFFE0000001FFFFFFF8000003FFFFFFFE000007FF00 1FFF80000FFF8007FFC0000FFF8003FFE0000FFF8001FFF0000FFF8000FFF8000FFF80007FF800 0FFF80007FFC000FFF80007FFC0007FF00003FFC0003FE00003FFC0001FC00003FFC0000000000 3FFC00000000003FFC00000000003FFC00000000003FFC00000007FFFFFC000000FFFFFFFC0000 0FFFFFFFFC00003FFFF03FFC0001FFFE003FFC0003FFF0003FFC000FFFC0003FFC001FFF80003F FC003FFE00003FFC007FFE00003FFC007FFC00003FFC00FFF800003FFC00FFF800003FFC00FFF0 00003FFC00FFF000003FFC00FFF000003FFC00FFF000007FFC00FFF800007FFC00FFF80000FFFC 007FFC0001FFFC007FFE0003EFFC003FFF000FCFFF001FFFC07F8FFFFC0FFFFFFF07FFFC03FFFF FC03FFFC007FFFF001FFFC0007FF80007FFC362E7DAD3A>97 D<007FC00000000000FFFFC00000 000000FFFFC00000000000FFFFC00000000000FFFFC00000000000FFFFC0000000000003FFC000 0000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0 000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FF C0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001 FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC00000000000 01FFC01FFE00000001FFC0FFFFE0000001FFC3FFFFF8000001FFCFFFFFFE000001FFFFF01FFF80 0001FFFF8003FFC00001FFFE0001FFE00001FFFC0000FFF00001FFF800007FF80001FFF000003F F80001FFE000003FFC0001FFE000001FFE0001FFE000001FFE0001FFE000001FFF0001FFE00000 1FFF0001FFE000000FFF0001FFE000000FFF0001FFE000000FFF8001FFE000000FFF8001FFE000 000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE0 00000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FF E000000FFF0001FFE000001FFF0001FFE000001FFF0001FFE000001FFE0001FFE000001FFE0001 FFE000003FFC0001FFE000003FFC0001FFF000007FF80001FFF80000FFF00001FFFC0000FFF000 01FFFE0003FFE00001FFFF8007FF800001FF3FE03FFF000001FE0FFFFFFC000001FC03FFFFF000 0001F800FFFFC0000001F0001FFC00000039487CC742>I<00000000003F800000FFF800FFE000 07FFFF03FFF0001FFFFFCFFFF8007FFFFFFFEFF800FFE03FFF0FF803FF800FFE0FF803FF0007FE 0FF807FF0007FF0FF80FFE0003FF87F00FFE0003FF83E00FFC0001FF80001FFC0001FFC0001FFC 0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0 001FFC0001FFC0000FFC0001FF80000FFE0003FF80000FFE0003FF800007FF0007FF000003FF00 07FE000003FF800FFE000000FFE03FF8000001FFFFFFF0000003FFFFFFC0000003C7FFFF000000 07C0FFF800000007C0000000000007C0000000000007C0000000000007E0000000000007E00000 00000007F0000000000007FC000000000007FFFFFFF8000007FFFFFFFF800007FFFFFFFFF00003 FFFFFFFFFC0003FFFFFFFFFE0001FFFFFFFFFF0000FFFFFFFFFF8000FFFFFFFFFFC003FFFFFFFF FFC00FFFFFFFFFFFE03FF800001FFFE07FF0000001FFF07FE00000007FF0FFC00000003FF0FFC0 0000003FF0FF800000001FF0FF800000001FF0FF800000001FF0FFC00000003FF0FFC00000003F F07FE00000007FE07FF0000000FFE03FF8000001FFC01FFC000003FF800FFF00000FFF0007FFF0 00FFFE0001FFFFFFFFF800007FFFFFFFE000000FFFFFFF000000007FFFE0000035447DAE3B> 103 D<007FC00000000000FFFFC00000000000FFFFC00000000000FFFFC00000000000FFFFC000 00000000FFFFC0000000000003FFC0000000000001FFC0000000000001FFC0000000000001FFC0 000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FF C0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001 FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC00000000000 01FFC0000000000001FFC0000000000001FFC001FFC0000001FFC00FFFF8000001FFC03FFFFE00 0001FFC0FFFFFF800001FFC1FE07FFC00001FFC7F003FFC00001FFCFC001FFE00001FFCF8001FF E00001FFDF0001FFE00001FFFE0000FFF00001FFFC0000FFF00001FFF80000FFF00001FFF80000 FFF00001FFF00000FFF00001FFF00000FFF00001FFF00000FFF00001FFE00000FFF00001FFE000 00FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001 FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF000 01FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF000FFFFFFC07FFFFF E0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B487CC742> I<00FC0003FF0007FF800FFFC00FFFC01FFFE01FFFE01FFFE01FFFE01FFFE01FFFE00FFFC00FFF C007FF8003FF0000FC000000000000000000000000000000000000000000000000000000000000 00000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC0FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF18497CC820>I<007FC000FFFFC0 00FFFFC000FFFFC000FFFFC000FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FF C00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FF C00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FFFFFF80FFFFFF80FFFFFF80 FFFFFF80FFFFFF8019487CC720>108 D<00000FFF0000000000FFFFF000000007FFFFFE000000 1FFFFFFF8000003FFC03FFC00000FFE0007FF00001FF80001FF80003FF00000FFC0007FE000007 FE000FFE000007FF000FFC000003FF001FFC000003FF803FFC000003FFC03FF8000001FFC03FF8 000001FFC07FF8000001FFE07FF8000001FFE07FF8000001FFE0FFF8000001FFF0FFF8000001FF F0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF800 0001FFF0FFF8000001FFF0FFF8000001FFF07FF8000001FFE07FF8000001FFE07FF8000001FFE0 7FF8000001FFE03FFC000003FFC03FFC000003FFC01FFC000003FF801FFE000007FF800FFE0000 07FF0007FF00000FFE0003FF80001FFC0001FFC0003FF80000FFE0007FF000007FFC03FFE00000 1FFFFFFF80000007FFFFFE00000000FFFFF0000000000FFF000000342E7DAD3B>111 D<007FC01FFE000000FFFFC0FFFFE00000FFFFC3FFFFF80000FFFFCFFFFFFE0000FFFFFFF01FFF 8000FFFFFF8007FFC00003FFFE0003FFE00001FFFC0001FFF00001FFF80000FFF80001FFF00000 7FF80001FFE000007FFC0001FFE000003FFE0001FFE000003FFE0001FFE000003FFF0001FFE000 001FFF0001FFE000001FFF0001FFE000001FFF0001FFE000001FFF8001FFE000000FFF8001FFE0 00000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FF E000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000001FFF8001 FFE000001FFF0001FFE000001FFF0001FFE000001FFF0001FFE000003FFE0001FFE000003FFE00 01FFE000007FFC0001FFE000007FFC0001FFF00000FFF80001FFF80001FFF00001FFFC0001FFF0 0001FFFE0007FFE00001FFFF800FFF800001FFFFE03FFF000001FFEFFFFFFC000001FFE3FFFFF0 000001FFE0FFFFC0000001FFE01FFC00000001FFE0000000000001FFE0000000000001FFE00000 00000001FFE0000000000001FFE0000000000001FFE0000000000001FFE0000000000001FFE000 0000000001FFE0000000000001FFE0000000000001FFE0000000000001FFE0000000000001FFE0 000000000001FFE0000000000001FFE00000000000FFFFFFC000000000FFFFFFC000000000FFFF FFC000000000FFFFFFC000000000FFFFFFC00000000039427CAD42>I<00FF803FC000FFFF81FF F000FFFF83FFFC00FFFF87FFFE00FFFF8FE7FF00FFFF9F8FFF8003FFBF0FFF8001FFBE0FFF8001 FFFC0FFF8001FFF80FFF8001FFF80FFF8001FFF00FFF8001FFF007FF0001FFF003FE0001FFE001 FC0001FFE000000001FFE000000001FFE000000001FFC000000001FFC000000001FFC000000001 FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001 FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 0000FFFFFFE00000FFFFFFE00000FFFFFFE00000FFFFFFE00000FFFFFFE00000292E7CAD31> 114 D<7FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC0 00FFFF8000FFF000000FE00000FFF800000FC00000FFF800000FC000007FFC00000F8000007FFC 00001F8000003FFC00001F0000003FFE00003F0000001FFE00003E0000001FFF00007E0000000F FF00007C0000000FFF8000FC00000007FF8000F800000007FFC001F800000003FFC001F0000000 03FFE003F000000003FFE003F000000001FFF003E000000001FFF007E000000000FFF007C00000 0000FFF80FC0000000007FF80F80000000007FFC1F80000000003FFC1F00000000003FFE3F0000 0000001FFE3E00000000001FFF7E00000000000FFF7C00000000000FFFFC00000000000FFFFC00 0000000007FFF8000000000007FFF8000000000003FFF0000000000003FFF0000000000001FFE0 000000000001FFE0000000000000FFC0000000000000FFC00000000000007F800000000000007F 800000000000003F000000000000003F000000000000003F000000000000003E00000000000000 7E000000000000007C00000000000000FC000000001F8000F8000000003FC001F8000000007FE0 01F000000000FFF003F000000000FFF003E000000000FFF007E000000000FFF00FC000000000FF F01F8000000000FFF03F80000000007FE07F00000000007F43FE00000000003FFFF80000000000 1FFFF0000000000007FFC0000000000001FE00000000000039427EAD3F>121 D E /Fb 1 16 df<07E01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3F FC1FF807E010127D9317>15 D E /Fc 56 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8 000000000070F8F8F870051C779B18>33 D<4010F078F078F078E038E038E038E038E038E038E0 38E038E038E0380D0E7B9C18>I<03C0000FF0000FF0001E78001E78001C38001C38001C78001C 7BF01CF3F01FF3F01FE7800FC7800F87000F0F001F0F003F8E007FDE00FBDE00F1FC00E1FC00E0 F800E0F870F0FC70F9FEF07FFFF03FCFE01F03C0141C7F9B18>38 D<007000F003F007E00F800F 001E003E003C007800780070007000F000F000E000E000E000E000E000E000F000F00070007000 780078003C003E001E000F000F8007E003F000F000700C24799F18>40 DI<01C00001C00001C0 0001C000C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C1 8001C00001C00001C00001C00011147D9718>I<3C7E7F7F7F3F0F0F3EFEF8F0080C788518>44 D<78FCFCFCFC780606778518>46 D<000380000780000780000F80000F00001F00001E00001E00 003E00003C00007C0000780000780000F80000F00001F00001E00003E00003C00003C00007C000 0780000F80000F00000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000 F00000E0000011247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001C07001 C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C07001C07803 C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>I<0380038007800F800F803F80FF 80FB8063800380038003800380038003800380038003800380038003800380038003800380FFFE FFFEFFFE0F1C7B9B18>I<3C7E7E7E7E3C00000000000000003C7E7E7E7E3E1E1E3CFCF8E0071A 789318>59 D<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE0 7FFFC0130C7E9318>61 D<00700000F80000F80000D80000D80001DC0001DC0001DC00018C0003 8E00038E00038E00038E000306000707000707000707000707000FFF800FFF800FFF800E03800E 03801C01C01C01C0FF8FF8FF8FF8FF8FF8151C7F9B18>65 D<01FCE007FFE00FFFE01F07E03E03 E03C01E07801E07800E07000E0F00000F00000E00000E00000E00000E00000E00000E00000F000 00F000007000E07800E07800E03C01E03E03E01F07C00FFF8007FF0001FC00131C7E9B18>67 DII I<01F9C007FFC00FFFC01F0FC03E07C03C03C07803C07801C07001C0F00000F00000E00000E000 00E00000E00000E01FF0E01FF0F01FF0F001C07003C07803C07803C03C07C03E0FC01F1FC00FFF C007FDC001F9C0141C7E9B18>II I76 DII<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780700700780F007F FF003FFE000FF800111C7D9B18>II82 D<07F3801FFF803FFF807C1F80F80780F00780E00380E00380F00000F000007800007F0000 3FF0000FFE0001FF80001FC00003C00001E00001E00000E0E000E0E000E0E001E0F003E0FC07C0 FFFF80FFFF00E7FC00131C7E9B18>III87 D95 D<1FE0007FF8007FFE00783E00780F00000F0000070001FF000FFF003FFF007F0700F80700F007 00E00700E00700F00F00F83F007FFFF03FFFF00FE1F014147D9318>97 D<01FE000FFF801FFF80 3F07807C0780780000F00000F00000E00000E00000E00000E00000F00000F000007801C07C03C0 3F07C01FFF800FFF0001FC0012147D9318>99 D<003F80003F80003F8000038000038000038000 038000038003F3800FFF801FFF803E1F807C0F80780780F00780F00380E00380E00380E00380E0 0380F00780F00780780F80781F803E3F803FFFF80FFBF803E3F8151C7E9B18>I<03F0000FFE00 1FFF003E1F807C0780780380F003C0F003C0E001C0FFFFC0FFFFC0FFFFC0F00000F000007801C0 7C03C03F07C01FFF800FFF0001FC0012147D9318>I<001FC0007FE000FFE001F1E001E0C001C0 0001C00001C000FFFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<03F1F00F FFF81FFFF81E1F383C0F003C0F003807003807003807003C0F003C0F001E1E001FFE003FFC003F F0003C00003C00001FFF003FFFC07FFFF07801F0F00078F00078E00038E00038F00078F800F87E 03F03FFFE00FFF8003FE00151F7F9318>II<03800007C00007C0 0007C000038000000000000000000000000000FFC000FFC000FFC00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF80FFFF80FFFF 80111D7C9C18>I107 DIII<01F0000FFE 001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001E07001C07803 C03C07803E0F801FFF000FFE0001F00013147E9318>II114 D<0FF7003FFF007FFF00F81F00F00F00E00700F00700FC00007FF0001FFC0007FF00001F80E007 80E00380F00380F80780FC0F80FFFF00FFFE00E7F80011147D9318>I<03800003800003800003 8000038000FFFFC0FFFFC0FFFFC003800003800003800003800003800003800003800003800003 80000380400380E00380E003C1E003E3E001FFC000FF80007E0013197F9818>I118 DI<7F9FF07F9FF07F9FF0070700078E00039E0001DC0001F80000F8 0000700000F00000F80001DC00039E00038E000707000F0780FF8FF8FF8FF8FF8FF815147F9318 >II<7FFFF07FFFF07FFFF07003E07007C0700F80 001F00003E00007C0000F80001F00003E00007C0000F80701F00703E00707C0070FFFFF0FFFFF0 FFFFF014147F9318>I<0007E0003FE0007FE000FC0000F00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00001E0007FE000FFC000FFC0007FE00001E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000F00000FC00007FE0003FE00007 E013247E9F18>I125 D E /Fd 3 112 df<03FC0FFE1FFE3E1E780C7000F000E000E000E000E000F000 70077C073E1F1FFE0FFC03F010127D9116>99 D109 D<03E0000FF8001FFC003C1E00780F00700700E00380E00380E00380E00380E00380F0 0780700700780F003C1E001FFC000FF80003E00011127E9116>111 D E /Fe 38 123 df<003F0001FF8003C3C00783C00F03C00E03C00E00000E00000E00000E00000E00 00FFFFC0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C07F87F87F87F8151D809C17>12 D<7070F8F8FCFCFCFC7C7C0C0C0C0C1C 1C181838387070F0F060600E0D7F9C15>34 D<70F8F8F878181818383070E060050D7D840C>44 DI<70F8F8F87005057D840C>I<70F8F8F870000000000000000070 F8F8F878181818383070E060051A7D910C>59 D70 D73 D78 D<003F800001FFF00003E0 F80007001C000E000E001C0007003C00078038000380780003C0700001C0F00001E0F00001E0F0 0001E0F00001E0F00001E0F00001E0F00001E0F00001E0780003C0780003C0380003803C000780 1E000F000E000E0007803C0003E0F80001FFF000003F80001B1C7E9B20>II82 D<7FFFFFC07FFFFFC0780F03C0700F01C0600F00C0E00F00E0C00F0060C00F00 60C00F0060C00F0060000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000003FFFC0003 FFFC001B1C7F9B1E>84 DI<18183C3C383870706060E0E0C0C0C0C0F8F8FCFCFCFC7C7C3838 0E0D7B9C15>92 D<1FE0003FF8003C3C003C1E00180E00000E00001E0007FE003FFE007E0E00F8 0E00F80E00F00E60F00E60F81E607C7E607FFFC01FC78013127F9115>97 DI<07F80FFC3E3C3C3C78187800F000F000F000F000F000 F000780078063C0E3F1C0FF807F00F127F9112>I<001F80001F80000380000380000380000380 00038000038000038000038000038007F3801FFF803E1F807C0780780380F80380F00380F00380 F00380F00380F00380F00380F003807807807C0F803E1F801FFBF007E3F0141D7F9C17>I<07E0 1FF83E7C781C781EF01EFFFEFFFEF000F000F000F000780078063C0E3F1C0FF807F00F127F9112 >I<00FC03FE079E071E0F1E0E000E000E000E000E000E00FFE0FFE00E000E000E000E000E000E 000E000E000E000E000E000E000E000E007FE07FE00F1D809C0D>I<07E7C01FFFC03C3DC0781E 00781E00781E00781E00781E00781E003C3C003FF80037E0007000007000007800003FFC003FFF 007FFF807807C0F003C0E001C0E001C0F003C0F807C07C0F801FFE0007F800121B7F9115>II<3C007C007C007C003C00000000000000000000000000FC00 FC001C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80FF80091D7F9C0C >I108 DII<03F0000FFC001E 1E00380700780780700380F003C0F003C0F003C0F003C0F003C0F003C07003807807803807001E 1E000FFC0003F00012127F9115>II114 D<1FB07FF0F0F0E070E030F0 30F8007FC07FE01FF000F8C078C038E038F078F8F0FFF0CFC00D127F9110>I<0C000C000C000C 000C001C001C003C00FFE0FFE01C001C001C001C001C001C001C001C001C301C301C301C301C30 1E700FE007C00C1A7F9910>IIII<7F8FF07F8FF00F0F80070F00038E0001DC0001D8 0000F00000700000780000F80001DC00038E00030E000707001F0780FF8FF8FF8FF81512809116 >II<7FFC7FFC783C707860F061E061E063C00780078C0F0C1E0C1E1C3C187818 F078FFF8FFF80E127F9112>I E /Ff 25 117 df<00F001E003E003C007800F000F001E001E00 3C003C003C007800780078007800F800F000F000F000F000F000F000F000F000F000F000F000F0 00F80078007800780078003C003C003C001E001E000F000F00078003C003E001E000F00C2E7EA1 12>40 DI<787878783830307060E0050A7D83 0D>44 D<001F0000001F0000003F8000003B8000003B8000007BC0000073C0000071C00000F1E0 0000E1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C000780 3C0007003C000FFFFE000FFFFE000FFFFE001E000F001E000F003C000F803C0007803C00078078 0007C0780003C0780003C0F00003E01B207F9F1E>65 D<003FE000FFF803FFFC07F07C0FC01C1F 800C1F00003E00003C00007C0000780000780000F80000F00000F00000F00000F00000F00000F0 0000F00000F00000F800007800007800007C00003C00003E00001F00001F80060FC00E07F03E03 FFFC00FFF8003FE017227DA01D>67 D69 D73 D77 DI<003F000000FFC00003FFF00007E1F8000F807C001F003E001E001E003C00 0F003C000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F0 0003C0F00003C0F00003C0F00003C0F80007C07800078078000780780007803C000F003C000F00 1E001E001F003E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II82 D<01FE0007FF801FFFC01F07C03E01C03C00C07C000078000078000078000078 00007C00003E00003F00001FE0001FFC000FFF0003FF80003F80000FC00007C00003E00001E000 01E00001E00001E00001E0C003E0E003C0F007C0FE0F807FFF001FFE0007F80013227EA019>I< FFFFFFC0FFFFFFC0FFFFFFC0001E0000001E0000001E0000001E0000001E0000001E0000001E00 00001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E 0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E000000 1E0000001E0000001E00001A207E9F1F>I87 D<03F80FFC1FFE3E1E7C 067800F800F000F000F000F000F000F000F80078007C033E0F1FFF0FFE03F810147E9314>99 D<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F00000F000 007000007800003C03003E0F001FFF0007FE0001F80011147F9314>101 D<07F1F00FFFF01FFFF03E3E007C1F00780F00780F00780F00780F00780F007C1F003E3E003FFC 003FF80037F0003000003800003FFE003FFFC03FFFE07FFFE0F803F0F001F0F000F0F000F0F801 F07E07E03FFFC01FFF8003FC00141E7F9317>103 D105 D109 DI<01F80007FE001FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0 F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F80014147F9317>I114 D<0FF03FFC3FFC7C1C780078007C007E003FE03FF01FF803FC007C003C003CC0 3CF87CFFF87FF01FE00E147F9311>I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E 001E001E001E001E001E001E001E001E001E001E001F601FF00FF00FC00C1A7F9910>I E /Fg 24 122 df<70F8F8F0F005057B840E>46 D<0F0F1F1F0E0000000000000000000070F8F8 F0F008147B930E>58 D<01FFF001FFF0001F00001E00001E00003E00003C00003C00003C00007C 0000780000780000780000F80000F00000F00000F00001F00001E00001E00001E00003E00003C0 0003C00003C00007C0000780000780000F8000FFF800FFF800141F7D9E12>73 D<01FF0000FFC001FF0000FFC0001F0001FC00001F0001F800001F0003F800003F0003F8000037 0006F0000037000CF0000037000DF00000770019F00000638019E00000638031E00000638033E0 0000E38063E00000C380C3C00000C380C3C00000C38187C00001C38187C0000183830780000183 830780000183860F800003838C0F800003038C0F00000303980F00000301D81F00000701F01F00 000601F01E00000E01E01E00001F01C03E0000FFE1C3FFE000FFE183FFE0002A1F7D9E29>77 D<01FFFF8001FFFFE0001E00F0001E0078003E0038003E003C003C003C003C003C007C003C007C 0078007800780078007000F800E000F801E000F0078000FFFF0001FFF80001F0000001E0000001 E0000003E0000003E0000003C0000003C0000007C0000007C0000007800000078000000F800000 FFF80000FFF800001E1F7D9E1F>80 D<7FFC3FF87FFC3FF807C007C0078003000F8007000F8006 000F0006000F0006001F000E001F000C001E000C001E000C003E001C003E0018003C0018003C00 18007C0038007C003000780030007800300078007000F8006000F0006000F000E000F000C000F0 01C000F003800070070000780F00003C3C00001FF8000007E000001D20779E22>85 D<00F9C003FFC0078FC00F0F801E07803C07803C0780780700780700780F00F80F00F00E00F00E 00F01E30F01E70F03C60707C6078FEE03FCFC01F078014147C9317>97 D<07803F803F80070007 000F000F000E000E001E001E001C001CF83FFC3F1E3E0E3C0F780F780F700F700F701FF01FE01E E01EE03CE03CE07870F071E03FC01F0010207B9F15>I<007E0003FF0007C3800F07801E07803C 07803C0200780000780000780000F80000F00000F00000F00000F000007003007807003C3E003F FC000FE00011147C9315>I<00FE0003FF0007C7800F03801E01803C03807C0700781F007FFC00 7FF000F80000F00000F00000F000007000007003007807003C3E001FFC000FE00011147C9315> 101 D<0000F80001FC0003BC00073C00073C000700000700000E00000E00000E00000E00000E00 01FFE001FFE0001C00001C00001C00003C00003800003800003800003800007800007000007000 00700000700000F00000E00000E00000E00000E00001E00001C00001C00001C0007380007B8000 FF00007F00007C00001629829F0E>I<003E7000FFF001E3F003C3E00781E00F01E00F01E01E01 C01E01C01E03C03E03C03C03803C03803C07803C07803C0F001C1F001E3F000FFF0007CE00000E 00001E00001E00001C00703C00787800F8F0007FE0003F8000141D7E9315>I<01E0000FE0000F E00001C00001C00003C00003C000038000038000078000078000070000073F000FFF800FE3800F 83C00F01C01F03C01E03801E03801C03803C07803C0700380700380F00780E38780E30701E3070 1C70F01EE0F00FC0E0078015207D9F17>I<00F000F000F000E000000000000000000000000000 000F001F803BC071C063C06380E380078007000F000F000E001E001C701C603C6038E03DC01F80 0F000C1F7D9E0E>I<03C01FC01FC00380038007800780070007000F000F000E000E001E001E00 1C001C003C003C00380038007800780070007000F180F380E300E300F7007E003C000A207C9F0C >108 D<1E0FE0FC003F1FF3FE0077F8778E0063F07E0F0063E03C0700E7C07C0F00C780780E00 0780780E000700700E000F00F01E000F00F01C000E00E01C000E00E03C001E01E038E01E01E038 C01C01C078C01C01C071C03C03C07B803C03C03F003803801E0023147D9325>I<1E0FE03F1FF0 77F87063F07863E038E7C078C780700780700700700F00F00F00E00E00E00E01E01E01C71E01C6 1C03C61C038E3C03DC3C01F83800F018147D931A>I<007E0003FF8007C7800F03C01E01C03C01 E03C01E07801E07801E07803E0F803E0F003C0F003C0F00780F00780700F00781E003C7C003FF8 000FC00013147C9317>I<1E1F003F3F8077F1C063E3C063C3C0E783C0C783800700000700000F 00000F00000E00000E00001E00001E00001C00001C00003C00003C000038000012147D9313> 114 D<01FC03FE07870F0F0E0F0E0F1E000F800FF80FFC03FC007E001E701EF01CF01CF03CF078 7FF01FC010147D9313>I<01C003C003C003800380078007800700FFF0FFF00F000E000E001E00 1E001C001C003C003C00380038007870786070E070C079C03F801F000C1C7C9B0F>I<0F0070E0 1F80F1F03BC0E1F071C0E1F063C0E0F06381E070E381C0600781C0600701C0600703C0E00F03C0 C00E0380C00E0380C00E0381C00E0781800E0783800E0F83000F1FCF0007FDFE0001F0F8001C14 7D931E>119 D<07C7801FFFC03CFCE07079E07071E060F1E0E0E1C000E00000E00001E00001E0 0001C00001C00073C0E0F3C0C0F381C0F7C180EFC780FFFF007C7C0013147D9315>I<0F00701F 80F03BC0E071C0E063C0E06381E0E381C00781C00701C00703C00F03800E03800E03800E07800E 07000E0F000E0F000F3F0007FE0003EE00000E00001E00101C00783C0078380078700071E0007F C0001F0000141D7D9316>I E /Fh 59 123 df<001FC3F0007FEFF801F0FE7803C0FC780780F8 7807007000070070000700700007007000070070000700700007007000FFFFFF80FFFFFF800700 700007007000070070000700700007007000070070000700700007007000070070000700700007 00700007007000070070000700700007007000070070007FE3FF007FE3FF001D20809F1B>11 D<001F8000FFC001F1E003C1E00781E00701E0070000070000070000070000070000070000FFFF E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E07FC3FE7FC3FE1720809F19>I<7038F87CFC7EFC7E7C3E0C060C 060C061C0E180C381C7038F07860300F0E7E9F17>34 D<70F8FCFC7C0C0C0C1C183870F060060E 7C9F0D>39 D<00E001C003C0038007000E000E001C001C003800380030007000700070006000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000600070007000700030003800 38001C001C000E000E000700038003C001C000E00B2E7DA112>II<70F8FCFC7C0C0C0C1C183870F060060E7C840D>44 DI<70F8F8F87005057C840D>I<00030003000700060006000E000C 000C001C0018001800380030003000700060006000E000C000C001C00180018001800380030003 000700060006000E000C000C001C0018001800380030003000700060006000E000C000C000102D 7DA117>I<018003801F80FF80E380038003800380038003800380038003800380038003800380 03800380038003800380038003800380038003800380FFFEFFFE0F1E7C9D17>49 D<07F0001FFC003C3E00701F00600F80E00780F807C0F807C0F803C0F803C07007C00007C00007 C0000F80000F00001F00003E00003C0000780000F00001E0000380000700000E00C01C00C03800 C0700180FFFF80FFFF80FFFF80121E7E9D17>I<07F0001FFC003C3F00380F00780F80780F807C 0780780F80000F80000F80001F00001E00007E0003F80003F000003C00001F00000F80000F8000 07C00007C07007C0F807C0F807C0F807C0F80F80E00F80701F003C3E001FFC0007F000121F7E9D 17>I<000E00000E00001E00003E00003E00006E0000EE0000CE00018E00038E00030E00060E00 0E0E000C0E00180E00380E00300E00600E00E00E00FFFFF0FFFFF0000E00000E00000E00000E00 000E00000E00000E0000FFE000FFE0141E7F9D17>I<03F0000FFC001E1E003C0F007807007807 80F00780F00380F00380F003C0F003C0F003C0F003C0F003C07007C07807C0780FC03C1FC01FFB C00FF3C00183C0000380000780000780780700780F00781E00783C007878003FF0000FC000121F 7E9D17>57 D<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<000380000003 8000000380000007C0000007C0000007C000000DE000000DE000000DE0000018F0000018F00000 18F00000307800003078000030780000603C0000603C0000603C0000E01E0000C01E0000FFFE00 01FFFF0001800F0001800F0003800F800300078003000780070007C0070003C00F8003C0FFE03F FEFFE03FFE1F207F9F22>65 DI<001FC0C000FFF0C001F83DC0 07E00FC00F8007C00F0007C01F0003C03E0001C03C0001C07C0001C07C0000C07C0000C0F80000 C0F8000000F8000000F8000000F8000000F8000000F8000000F8000000F80000007C0000C07C00 00C07C0000C03C0001C03E0001801F0001800F0003800F80070007E00E0001F83C0000FFF80000 1FE0001A217D9F21>I69 D<001FE060007FF86001F83CE003E0 0FE007C007E00F8003E01F0001E03E0000E03E0000E07C0000E07C0000607C000060F8000060F8 000000F8000000F8000000F8000000F8000000F8000000F8007FFCF8007FFC7C0001E07C0001E0 7C0001E03E0001E03E0001E01F0001E00F8001E007C003E003E007E001FC1FE0007FFC60001FF0 001E217D9F24>71 D73 D76 DI<001F800000FFF00001 E0780007C03E000F801F000F000F001E0007803C0003C03C0003C07C0003E07C0003E0780001E0 F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001 E07C0003E07C0003E03C0003C03E0007C01E0007800F000F000F801F0007C03E0001F0F80000FF F000001F80001C217D9F23>79 DI<07E1801FF9803C3F80780F 80700780E00380E00380E00180E00180E00180F00000F800007C00007F80003FF8003FFE000FFF 0003FF00003F80000F800003C00003C00001C0C001C0C001C0C001C0E001C0E00380F00380F807 80FE0F00CFFE00C3F80012217D9F19>83 D<7FFFFFE07FFFFFE0780F01E0700F00E0600F006060 0F0060E00F0070C00F0030C00F0030C00F0030C00F0030000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F000007FFFE0007FFFE001C1F7E9E21>I86 DI91 D<180C3C1E381C703860 30E070C060C060C060F87CFC7EFC7E7C3E381C0F0E7B9F17>II< 3FE0007FF800787C00781E00781E00000E00000E00000E0007FE001FFE003F0E007C0E00F80E00 F00E30F00E30F01E30F81E307C7F703FFFE01FC78014147E9317>97 D<0E0000FE0000FE00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E3F000EFFC00FC3E00F81E00F 00F00E00F00E00780E00780E00780E00780E00780E00780E00780E00780E00F00F00F00F81E00F C3C00CFF800C7F0015207F9F19>I<03FC0FFE1E1E3C1E781E7800F000F000F000F000F000F000 F000F80078007C033E071F0E0FFC03F010147E9314>I<000380003F80003F8000038000038000 038000038000038000038000038000038000038007F3800FFF801E1F803C0780780780780380F0 0380F00380F00380F00380F00380F00380F00380F003807803807807803C0F803E1F801FFBF807 E3F815207E9F19>I<03F0000FFC001E1E003C0F00780700780780F00780F00380FFFF80FFFF80 F00000F00000F00000F800007800007C01803E03801F87000FFE0003F80011147F9314>I<007E 00FF01EF038F078F0700070007000700070007000700FFF0FFF007000700070007000700070007 000700070007000700070007000700070007007FF07FF01020809F0E>I<0001E007F7F00FFF70 3E3E703C1E00780F00780F00780F00780F00780F00780F003C1E003E3E003FF80037F000300000 3000003800003FFE001FFFC03FFFE07803E0F000F0E00070E00070E00070F000F07801E03E07C0 1FFF8003FC00141F7F9417>I<0E0000FE0000FE00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E7F000EFF800FC7C00F83C00F01C00F01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FCFFE7FC16207F9F19>I<1E 003E003E003E001E000000000000000000000000000E007E007E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E00FFC0FFC00A1F809E0C>I<00E001F001F001F000E0 000000000000000000000000007007F007F000F000700070007000700070007000700070007000 700070007000700070007000700070007000700070F0F0F0E0F1E0FFC03F000C28829E0E>I<0E 0000FE0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1FF00E 1FF00E0F800E0F000E1E000E3C000E78000EF0000FF0000FF8000FBC000F3C000E1E000E0F000E 0F000E07800E07800E03C0FFCFF8FFCFF815207F9F18>I<0E00FE00FE000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E00FFE0FFE00B20809F0C>I<0E3F03F000FEFFCFFC00FFC3DC3C000F81F81E000F00 F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE0FFE7FE7FE0 23147F9326>I<0E7F00FEFF80FFC7C00F83C00F01C00F01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FCFFE7FC16147F9319>I<01F80007 FE001E07803C03C03801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E078 01E03801C03C03C01E078007FE0001F80014147F9317>I<0E3F00FEFFC0FFC3E00F81E00F01F0 0E00F00E00F80E00780E00780E00780E00780E00780E00780E00F80E00F00F01F00F81E00FC7C0 0EFF800E7F000E00000E00000E00000E00000E00000E00000E0000FFE000FFE000151D7F9319> I<07F1800FF9801F1F803C0F807C0780780380F80380F00380F00380F00380F00380F00380F003 80F803807807807C07803C0F803E1F801FFB8007E3800003800003800003800003800003800003 80000380003FF8003FF8151D7E9318>I<0E7CFFFEFFDE0F9E0F1E0F000E000E000E000E000E00 0E000E000E000E000E000E000E00FFE0FFE00F147F9312>I<1FB07FF078F0E070E030E030F000 FC007FC03FE00FF000F8C078C038E038E038F078F8F0FFE0CFC00D147E9312>I<060006000600 06000E000E001E003E00FFF8FFF80E000E000E000E000E000E000E000E000E000E000E180E180E 180E180E180F3007F003E00D1C7F9B12>I<0E01C0FE1FC0FE1FC00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00E07C00F0FC007FDFC03F9FC 16147F9319>III<7FC7FC7FC7FC0F03E007038003830001C70000EE0000EC00007800003800003C0000 7C0000EE0001C7000187000303800701C01F01E0FF87FEFF87FE1714809318>II<3FFF3FFF381F301E703C6078607860F001E003C003C007830F031E031E 073C067806F81EFFFEFFFE10147F9314>I E /Fi 24 122 df46 D<007F000001FFC00007FFF0000FFFF8000FC1F8001F007C003F007E003E003E003C001E007C00 1F007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F8078000F00 7C001F007C001F007C001F003E003E003E003E003F007E001F80FC000FC1F8000FFFF80007FFF0 0001FFC000007F000019297EA71E>48 D<00180000380000F80007F800FFF800FFF800F8F80000 F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F8007FFFF07FFFF07FFFF014287BA71E>I<00FE0003FFC007FFE00FFFF0 1F03F83C00FC38007E78003E70003EF0001FF0001F60001F20001F00001F00001F00001F00003E 00003E00007C00007C0000F80001F00001E00003C0000780000F00001E00003C0000780000F000 01E00003C0000780000F00001E00003C00007FFFFF7FFFFF7FFFFF7FFFFF18287EA71E>I<007F 000001FFC00007FFF0000FFFF8001FC1F8003E007C003C003E0078003E0038003E0010003E0000 003E0000003E0000003C0000007C000000FC000001F8000007F00000FFE00000FFC00000FFE000 00FFF0000001FC0000007C0000003E0000001F0000001F0000000F8000000F8000000F8000000F 8000000F8040000F8060001F00F0001F00F8003F007E007E003F81FC001FFFF8000FFFF00003FF E000007F000019297EA71E>I<0003F0000007F0000005F000000DF000000DF000001DF0000039 F0000039F0000079F0000079F00000F1F00000F1F00001E1F00003E1F00003E1F00007C1F00007 C1F0000F81F0000F81F0001F01F0001F01F0003E01F0007C01F0007C01F000F801F000FFFFFF80 FFFFFF80FFFFFF80FFFFFF800001F0000001F0000001F0000001F0000001F0000001F0000001F0 000001F0000001F0000001F00019277EA61E>I<0001F000000003F800000003F800000007FC00 000007BC00000007BC0000000F3E0000000F1E0000000F1E0000001F1F0000001E1F0000003E0F 8000003E0F8000003C0F8000007C07C000007C07C000007807C00000F803E00000F803E00000F0 03E00001F001F00001F001F00003E001F80003E000F80003C000F80007C0007C0007FFFFFC0007 FFFFFC000FFFFFFE000F80003E000F00003E001F00001F001F00001F003E00001F803E00000F80 3C00000F807C000007C07C000007C078000007C0F8000003E0F8000003E023297EA828>65 D80 D<007FC00001FFF80007FFFE000FFFFF001FC07F003F00 0F007E0006007C000000F8000000F8000000F8000000F8000000F8000000FC0000007C0000007E 0000007F0000003FE000001FFE00000FFFC00007FFF00001FFF800003FFC000003FE0000007F00 00001F8000001F8000000FC0000007C0000007C0000007C0000007C0000007C0000007C000000F 8060000F80F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE00000FF80001A2B7DA921> 83 D<00FE0007FF801FFFC03FFFE03E03F03801F02001F80000F80000F80000F80000F80000F8 007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F800F8F800F8FC01F87E07F87FFFF83FFFF8 1FFCF80FE0F8151B7E9A1D>97 D<007F8001FFE007FFF80FFFFC1FC07C1F001C3E00087C00007C 00007C0000F80000F80000F80000F80000F80000F80000F800007C00007C00007E00003E00001F 000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>99 D<00003E00003E00003E00003E0000 3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00FC3E03FF3E07FFFE0FFF FE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003EF8003EF800 3EFC003E7C003E7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03FF3E00FC3E17297EA81F>I<00 7E0003FF8007FFC00FFFE01F83F03F00F03E00787C00787C003878003CFFFFFCFFFFFCFFFFFCFF FFFCF80000F80000F800007800007C00007C00003E00003F000C1FC07C0FFFFC07FFFC01FFF000 7F80161B7E9A1B>I105 D108 DII<007F00 0001FFC00007FFF0000FFFF8001FC1FC003F007E003E003E007C001F007C001F0078000F00F800 0F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F807C001F007C001F007E003F003E 003E003F007E001FC1FC000FFFF80007FFF00001FFC000007F0000191B7E9A1E>I114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F800 00F80000F80000FC00007F80007FF8003FFE001FFF0007FF8000FFC0000FE00007E00003E00003 E04003E0E007E0FC0FC0FFFFC07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007 C00007C00007C00007C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007 E1C003FFE003FFE001FF8000FC0013227FA116>III121 D E /Fj 29 122 df<70F8FCFC7C0C0C0C1C18183870E0E0060F7C840E>44 D<70F8F8F87005057C840E>46 D<018003800F80FF80F380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038003800380038003800380FFFE FFFE0F217CA018>49 D<03F8000FFE003C3F00380F807007C06007C0E003E0F803E0F803E0F801 E0F801E07003E00003E00003C00007C00007C0000F80000F00001E00003C0000780000F00000E0 0001C0000380000700600E00601C00603800E07000C0FFFFC0FFFFC0FFFFC013217EA018>I<18 01801E07801FFF801FFF001FFC001FF00018000018000018000018000018000018000019F8001F FE001F0F001E07801C03C01803C00001C00001E00001E00001E00001E07001E0F001E0F001E0F0 01E0E003C0E003C0700780380F803E1F000FFC0007F00013227EA018>53 D<007E0001FF8003C3C00703C00E03C01E03C03C03C0380000780000780000780000702000F3FC 00F7FF00FE0F80FC0780F803C0F803C0F801E0F001E0F001E0F001E0F001E0F001E07001E07001 E07801E07801C03803C03C03801E07800F1F0007FE0003F80013227EA018>I<03F8000FFC001F 1E003C07003807807803807003C0F003C0F001C0F001C0F001E0F001E0F001E0F001E0F001E0F0 03E07803E07803E03C07E03E0FE01FFDE007F9E00081E00001C00003C00003C000038078078078 0700780F00781E00787C003FF8000FE00013227EA018>57 D<000180000003C0000003C0000003 C0000007E0000007E0000007E000000FF000000CF000000CF000001CF800001878000018780000 383C0000303C0000303C0000601E0000601E0000601E0000C00F0000C00F0000C00F0001FFFF80 01FFFF8001800780030003C0030003C0030003C0060001E0060001E0060001E00E0000F01F0001 F0FFC00FFFFFC00FFF20237EA225>65 D68 DII78 D<07F0600FFE601E1FE03807E07003E07001E0E000E0E000E0E00060E00060F00060 F00000F800007C00007F00003FF0001FFE000FFF8003FFC0007FC00007E00001E00001F00000F0 0000F0C00070C00070C00070E00070E000F0F000E0F801E0FC01C0FF0780C7FF00C1FC0014247D A21B>83 D<1FF0003FFC003C3E003C0F003C0F00000700000700000F0003FF001FFF003F07007C 0700F80700F80700F00718F00718F00F18F81F187C3FB83FF3F01FC3C015157E9418>97 D<0E0000FE0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E3FC00EFFE00FE1F00F80780F003C0E003C0E001E0E001E0E001E0E001E0E001E0E00 1E0E001E0E001E0E003C0E003C0F007C0F80F80FE1F00CFFE00C3F8017237FA21B>I<0000E000 0FE0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E003 F8E00FFEE01F0FE03E03E07C01E07800E07800E0F000E0F000E0F000E0F000E0F000E0F000E0F0 00E07000E07801E07801E03C03E01F0FF00FFEFE03F0FE17237EA21B>100 D<01FC0007FF001F0F803E03C03C01C07801E07801E0FFFFE0FFFFE0F00000F00000F00000F000 00F000007800007800603C00E03E01C00F83C007FF0001FC0013157F9416>I<0E0000FE0000FE 00001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3F800E FFE00FE1E00F80F00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E0070FFE7FFFFE7FF18237FA21B>104 D<01C003E003E003E001C0 0000000000000000000000000000000001E00FE00FE001E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0F1E0F1C0F3C0FF803E00 0B2C82A10F>106 D<0E0000FE0000FE00001E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E0FFC0E0FFC0E07E00E07800E07000E1E000E3C000E78000EF800 0FFC000FFC000F1E000E0F000E0F800E07800E03C00E03E00E01E00E01F0FFE3FEFFE3FE17237F A21A>I<0E00FE00FE001E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE0FFE00B237FA2 0E>I<0E3FC0FF00FEFFF3FFC0FFE0F783C01F807E01E00F003C00E00F003C00E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E0 0E003800E00E003800E00E003800E00E003800E0FFE3FF8FFEFFE3FF8FFE27157F942A>I<0E3F 80FEFFE0FFE1E01F80F00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E0070FFE7FFFFE7FF18157F941B>I<01FC0007FF000F07801C 01C03800E07800F0700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00F078007FF0001FC0015157F9418>I<0E7EFEFFFFEF1F8F0F0F0F000F000E000E00 0E000E000E000E000E000E000E000E000E000E00FFF0FFF010157F9413>114 D<1FD83FF87878F038E018E018F018F8007F807FE01FF003F8007CC03CC01CE01CE01CF03CF878 FFF0CFE00E157E9413>I<060006000600060006000E000E000E001E003E00FFF8FFF80E000E00 0E000E000E000E000E000E000E000E000E0C0E0C0E0C0E0C0E0C0F1C073807F803E00E1F7F9E13 >I<0E0070FE07F0FE07F01E00F00E00700E00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00F00E00F00E01F00787F807FF7F01FC7F18157F941B>I121 D E /Fk 26 124 df<000003C00780000003C0078000 0003C00780000007C00F80000007800F00000007800F00000007800F0000000F801F0000000F00 1E0000000F001E0000001F003E0000001E003C0000001E003C0000001E003C0000003E007C0000 003C00780000003C00780000003C007800FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF7FFFFFFF FFFF0001F003E0000001E003C0000001E003C0000001E003C0000003E007C0000003C007800000 03C00780000003C00780000003C00780000007C00F80007FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF FFFFFFFFFFFFFFFF001E003C0000001E003C0000001E003C0000003E007C0000003C0078000000 3C00780000003C00780000007C00F80000007800F00000007800F0000000F801F0000000F001E0 000000F001E0000000F001E0000001F003E0000001E003C0000001E003C0000001E003C0000030 367CA939>35 D<000003800000000007C00000000007C0000000000FE0000000000FE000000000 0FE0000000001FF0000000001FF0000000003FF8000000003FF8000000003FF8000000007FFC00 00000077FC00000000F7FE00000000E3FE00000000E3FE00000001C1FF00000001C1FF00000003 C1FF8000000380FF8000000780FFC0000007007FC0000007007FC000000E007FE000000E003FE0 00001E003FF000001C001FF000001FFFFFF000003FFFFFF800003FFFFFF8000078000FFC000070 0007FC0000700007FC0000E00007FE0000E00003FE0001E00003FF0001C00001FF0001C00001FF 00FFFE003FFFFEFFFE003FFFFEFFFE003FFFFE2F297EA834>65 D<00007FF801800007FFFE0780 001FFFFF8F80007FF80FFF8000FF8001FF8003FE00007F8007FC00003F8007F800001F800FF000 000F801FE000000F803FE0000007803FC0000007807FC0000003807FC0000003807FC000000380 FF8000000000FF8000000000FF8000000000FF8000000000FF8000000000FF8000000000FF8000 000000FF8000000000FF8000000000FF8000000000FF80000000007FC0000000007FC000000380 7FC0000003803FC0000003803FE0000003801FE0000007800FF00000070007F800000F0007FC00 001E0003FE00003C0000FF8000F800007FF807F000001FFFFFC0000007FFFF000000007FF80000 29297CA832>67 DI 73 D77 D80 D<00FF806003FFF0E00FFFFFE01FC0FFE03F001FE03E0007E07E0003E07C0003E0FC 0001E0FC0001E0FC0000E0FE0000E0FF0000E0FF800000FFF80000FFFFC0007FFFF8007FFFFE00 3FFFFF801FFFFFC00FFFFFC007FFFFE001FFFFF0003FFFF00003FFF800001FF800000FF8000007 F8E00003F8E00001F8E00001F8E00001F8F00001F8F00001F0F80003F0FC0003E0FF0007E0FFE0 1FC0FFFFFF80E1FFFE00C03FF8001D297CA826>83 D<03FFC0000FFFF0001F81FC003FC0FE003F C07F003FC07F003FC03F803FC03F801F803F8000003F8000003F80001FFF8001FFFF8007FE3F80 1FE03F803FC03F807F803F807F003F80FE003F80FE003F80FE003F80FE007F80FF007F807F00FF C03FC3DFFC1FFF8FFC03FE07FC1E1B7E9A21>97 D<003FF80001FFFE0007F83F000FE07F801FC0 7F803F807F803F807F807F807F807F003F00FF000000FF000000FF000000FF000000FF000000FF 000000FF000000FF000000FF0000007F0000007F8000003F8001C03FC001C01FC003C00FE00780 07F83F0001FFFC00003FF0001A1B7E9A1F>99 D<00003FF80000003FF80000003FF800000003F8 00000003F800000003F800000003F800000003F800000003F800000003F800000003F800000003 F800000003F800000003F800000003F800003FE3F80001FFFBF80003F83FF8000FE00FF8001FC0 07F8003F8003F8003F8003F8007F8003F8007F0003F800FF0003F800FF0003F800FF0003F800FF 0003F800FF0003F800FF0003F800FF0003F800FF0003F800FF0003F8007F0003F8007F0003F800 3F8003F8003F8007F8001FC00FF8000FE01FF80007F07FFF8001FFFBFF80003FC3FF80212A7EA9 26>I<003FE00001FFFC0007F07E000FE03F001FC01F803F800FC03F800FC07F000FC07F0007E0 FF0007E0FF0007E0FF0007E0FFFFFFE0FFFFFFE0FF000000FF000000FF000000FF0000007F0000 007F8000003F8000E03F8001E01FC001C00FE007C003F81F8001FFFE00003FF8001B1B7E9A20> I<000FF800003FFE0000FF3F0001FC7F8003F87F8003F87F8007F07F8007F07F8007F03F0007F0 000007F0000007F0000007F0000007F0000007F00000FFFFC000FFFFC000FFFFC00007F0000007 F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F00000 07F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F0000007F000 007FFF80007FFF80007FFF8000192A7EA915>I<0F801FC01FE03FE03FE03FE01FE01FC00F8000 00000000000000000000000000FFE0FFE0FFE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE0 0FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE0FFFEFFFEFFFE0F2B7DAA14>105 D108 DII<003FE00001FFFC0003F07E000FC01F801F800FC03F800FE03F0007E07F0007F07F0007F0 7F0007F0FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F87F0007 F07F0007F03F800FE03F800FE01F800FC00FC01F8007F07F0001FFFC00003FE0001D1B7E9A22> II114 D<03FE701FFFF03E03F07801F07000F0F00070F0 0070F80070FE0000FFF000FFFF007FFFC03FFFE01FFFF007FFF800FFFC0007FC0000FCE0007CE0 003CF0003CF0003CF80078FC0078FF01F0FFFFC0E1FF00161B7E9A1B>I<007000007000007000 00700000F00000F00000F00001F00003F00003F00007F0001FFFF0FFFFF0FFFFF007F00007F000 07F00007F00007F00007F00007F00007F00007F00007F00007F00007F00007F00007F03807F038 07F03807F03807F03807F03807F03803F87001FCF000FFE0003FC015267FA51B>II119 D121 D123 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300 TeXDict begin %%EndSetup %%Page: 0 1 bop 85 980 a Fk(A)23 b(Prop)r(osal)h(for)f(Dynamic)f(Pro)r(cess)g(Creation)h (and)g(Destruction)531 1055 y(in)g(MPI)g({)g(Spa)n(wn)h(Prop)r(osal)f(#I)558 1249 y Fj(An)o(thon)o(y)16 b(Skjellum)d(and)k(Nathan)g(E.)f(Doss)783 1375 y(F)l(ebruary)g(26,)g(1995)p eop %%Page: 1 2 bop 75 45 a Fi(0.1)59 b(Pro)r(cess)19 b(creation)75 147 y Fh(T)l (raditionally)l(,)e(in)f(systems)f(suc)o(h)h(as)f(PVM)g(that)g(ha)o(v)o(e)g (supp)q(orted)h(a)g(dynamic)g(pro)q(cess)g(mo)q(del,)g(one)75 203 y(pro)q(cess)f(spa)o(wns)g(one)g(or)f(more)h(additional)h(pro)q(cesses)f (as)g(needed.)21 b(In)15 b Fg(Using)g(MPI)21 b Fh([1)o(],)15 b(the)g(authors)75 259 y(presen)o(t)f(a)f(mo)q(del)i(whereb)o(y)f(a)f(group)h (of)f(pro)q(cesses)h(collectiv)o(ely)j(spa)o(wn)c(pro)q(cesses.)20 b(Their)14 b(approac)o(h)75 316 y(encompasses)k(the)h(traditional)f(approac)o (h)g(\(the)g(spa)o(wning)h(group)f(can)g(b)q(e)h(of)e(size)i(one\))f(as)g(w)o (ell)h(as)75 372 y(broadens)h(it)g(\(the)g(spa)o(wning)g(group)g(can)g(con)o (tain)g(m)o(ultiple)i(pro)q(cesses\).)34 b(W)l(e)20 b(prop)q(ose)g(a)g (similar)75 429 y(function:)75 580 y Ff(MPI)p 160 580 14 2 v 16 w(COMM)p 318 580 V 16 w(SP)l(A)-5 b(WN\(string,)16 b(ro)q(ot,)e(comm)n (,)d(intercomm)n(\))117 657 y Fe(IN)155 b Ff(string)481 b Fe(Pro)q(cess)20 b(sp)q(eci\014cation)f(string)f(that)g(describ)q(es)j(\\what")c(is)905 713 y(to)h(b)q(e)h(created)h(and)e(\\where")h(it)e(is)h(to)g(b)q(e)h (created.)33 b(The)905 770 y(exact)16 b(format)e(of)g(this)i(argumen)o(t)e (is)h(not)g(sp)q(eci\014ed)i(b)o(y)e(the)905 826 y(standard,)j(but)f(is)g (implemen)o(tatio)o(n)d(dep)q(enden)o(t.)29 b(F)m(or)17 b(ex-)905 883 y(ample,)12 b(it)i(migh)o(t)e(b)q(e)j(a)e(\014lename)g(whic)o(h)h(con)o (tains)g(a)g(list)g(of)905 939 y(pro)q(cesses)f(to)d(create)h(or)f(simply)f (a)g(n)o(um)o(b)q(er)h(whic)o(h)g(indicates)905 996 y(ho)o(w)k(man)o(y)e(pro) q(cesses)k(to)e(create.)117 1071 y(IN)155 b Ff(ro)q(ot)508 b Fe(Pro)q(cess)24 b(in)d Fd(comm)g Fe(that)g(con)o(tains)h(a)f(v)n(alid)f (string.)42 b(The)905 1127 y(string)14 b(argumen)o(t)f(in)g(all)g(other)i (pro)q(cesses)h(is)e(ignored.)117 1202 y(IN)155 b Ff(comm)466 b Fe(Original)15 b(comm)o(unicator)f(whose)j(group)f(sync)o(hronizes)i(on)905 1259 y(the)d(spa)o(wning)e(of)g(the)i(c)o(hildren)f(pro)q(cesses.)117 1334 y(OUT)108 b Ff(intercomm)380 b Fe(Resulting)19 b(in)o(tercomm)o (unicator.)30 b(The)20 b(lo)q(cal)e(group)h(con-)905 1390 y(tains)e(the)g (pro)q(cesses)j(found)d(in)f Fd(comm)p Fe(;)h(the)h(remote)e(group)905 1447 y(con)o(tains)e(the)h(group)e(of)h(spa)o(wned)g(pro)q(cesses.)75 1571 y Fc(int)23 b(MPI)p 245 1571 15 2 v 17 w(Comm)p 358 1571 V 17 w(spawn\(char)f(*string,)h(int)h(root,)f(MPI)p 1163 1571 V 17 w(Comm)g(comm,)393 1628 y(MPI)p 468 1628 V 17 w(Comm)g(*intercomm\))75 1714 y(MPI)p 150 1714 V 17 w(COMM)p 263 1714 V 16 w(SPAWN\(STRING,)f(ROOT,)i (COMM,)f(INTERCOMM,)f(IERROR\))170 1771 y(CHARACTER*\(*\))h(STRING)170 1827 y(INTEGER)g(ROOT,)h(COMM,)f(INTERCOMM,)f(IERROR)75 1914 y Fh(The)14 b(return)g(v)m(alue)h(of)f(this)g(call)h(is)g Fc(MPI)p 761 1914 V 16 w(SUCCESS)e Fh(if)i(the)f(pro)q(cesses)g(w)o(ere)g(spa)o(wned.) 19 b(If)c(the)f(pro)q(cesses)75 1970 y(could)i(not)e(b)q(e)h(spa)o(wned)g(or) f(if)h(spa)o(wning)g(is)h(not)e(a)o(v)m(ailable)i(in)g(an)e(implemen)o (tation,)i Fc(MPI)p 1653 1970 V 17 w(ERR)p 1742 1970 V 17 w(SPAWN)75 2027 y Fh(is)g(returned.)166 2083 y(The)i(follo)o(wing)g(co)q(de)g(fragmen)o (t)f(illustrates)i(ho)o(w)e(this)h(call)h(migh)o(t)e(b)q(e)i(used.)28 b(The)17 b(spa)o(wn)h(call)75 2140 y(is)f(collectiv)o(e)h(o)o(v)o(er)d Fc(MPI)p 493 2140 V 17 w(COMM)p 606 2140 V 16 w(WORLD)h Fh(with)g(all)h(pro)q (cesses)g(in)g Fc(MPI)p 1250 2140 V 17 w(COMM)p 1363 2140 V 16 w(WORLD)f Fh(receiving)h(an)g(in)o(ter-)75 2196 y(comm)o(unicator)d(as)g (the)h(result)g(of)f(the)g(spa)o(wn)g(call.)21 b Fc(MPI)p 1071 2196 V 17 w(Intercomm)p 1304 2196 V 16 w(merge)14 b Fh(is)h(then)f(used)h(to) f(merge)75 2252 y(the)j(t)o(w)o(o)e(separate)h(w)o(orlds)h(\(represen)o(ted)g (b)o(y)g(the)g(t)o(w)o(o)e(sides)j(of)e(the)h(in)o(tercomm)o(unicator\))f(in) o(to)h(one)75 2309 y(in)o(tracomm)o(unicator)e(\()p Fg(galaxy)t Fh(\).)170 2415 y Fc(main)24 b(\(int)f(argc,)g(char)h(**argv\))170 2472 y({)218 2528 y(int)g(rank,)f(root)g(=)h(0;)242 2585 y(char)f(*string)g (=)h(\(char)f(*\)0;)242 2641 y(MPI_Comm)g(intercomm,)f(galaxy;)964 2828 y Fh(1)p eop %%Page: 2 3 bop 242 102 a Fc(MPI_Init)23 b(\(&argc,)g(&argv\);)242 158 y(MPI_Comm_rank)f(\(MPI_COMM_WORLD,)g(&rank\);)242 214 y(if)i(\(rank)f(==)g (root\))290 271 y(string)g(=)h("process)e(specification";)242 327 y(MPI_Comm_spawn\(string,root)o(,MPI_COM)o(M_WORLD)o(,&inter)o(comm\);) 242 384 y(MPI_Intercomm_merge\(interc)o(omm,)f(1,)i(&galaxy\);)242 440 y(/*)h(...)f(*/)170 497 y(})75 603 y Fh(The)14 b(spa)o(wned)g(pro)q (cesses)g(come)g(in)o(to)f(existence)i(with)g(an)e Fc(MPI)p 1180 603 15 2 v 17 w(COMM)p 1293 603 V 17 w(WORLD)g Fh(that)g(con)o(tains)h (only)g(the)75 659 y(newly)j(spa)o(wned)f(pro)q(cesses.)22 b(The)16 b(paren)o(ts)g(and)g(c)o(hildren)i(ha)o(v)o(e)d(di\013eren)o(t)h Fc(MPI)p 1485 659 V 17 w(COMM)p 1598 659 V 17 w(WORLD)p Fh('s.)21 b(The)75 716 y(follo)o(wing)g Fc(MPI)p 347 716 V 17 w(Comm)p 460 716 V 17 w(parent)f Fh(function)h(allo)o(ws)g(these)g(newly)h(spa)o(wned) f(pro)q(cesses)g(to)f(access)h(the)75 772 y(in)o(tercomm)o(unicator)15 b(that)g(con)o(tains)g(the)g(\\paren)o(t")f(group)h(of)g(pro)q(cesses:)75 923 y Ff(MPI)p 160 923 14 2 v 16 w(COMM)p 318 923 V 16 w(P)l (ARENT\(intercomm)o(\))117 1001 y Fe(OUT)108 b Ff(intercomm)380 b Fe(In)o(tercomm)o(unicator)7 b(whic)o(h)j(spans)f(b)q(oth)h(the)g(spa)o (wned)g(pro-)905 1057 y(cesses)17 b(and)c(the)i(pro)q(cesses)h(that)e(spa)o (wned)h(them.)75 1181 y Fc(int)23 b(MPI)p 245 1181 15 2 v 17 w(Comm)p 358 1181 V 17 w(parent\(MPI)p 615 1181 V 16 w(Comm)g(*intercomm\))75 1268 y(MPI)p 150 1268 V 17 w(COMM)p 263 1268 V 16 w(PARENT\(INTERCOMM,)f (IERROR\))170 1324 y(INTEGER)h(INTERCOMM,)g(IERROR)75 1411 y Fh(If)g Fc(intercomm)f Fh(is)i(not)f(a)f(v)m(alid)j(comm)o(unicator,)g (then)e(the)g(pro)q(cesses)h(w)o(ere)f(not)f(spa)o(wned)i(with)75 1467 y Fc(MPI)p 150 1467 V 17 w(Comm)p 263 1467 V 16 w(spawn)p Fh(.)29 b(The)19 b(follo)o(wing)g(illustrates)h(co)q(de)f(that)f(migh)o(t)g (b)q(e)h(used)g(b)o(y)g(the)f(spa)o(wned)h(pro-)75 1524 y(cesses.)170 1630 y Fc(main)24 b(\(int)f(argc,)g(char)h(**argv\))170 1686 y({)218 1743 y(int)g(rank,)f(root)g(=)h(0;)218 1799 y(char)f(*string)g(=)h (\(char)f(*\)0;)218 1856 y(MPI_Comm)g(intercomm,)f(galaxy;)218 1969 y(MPI_Init)h(\(&argc,)g(&argv\);)218 2025 y(MPI_Comm_parent)f(\()i (&intercomm)e(\);)218 2082 y(if)i(\(intercomm)e(!=)i(MPI_COMM_NULL\))266 2138 y(MPI_Intercomm_merge\(inter)o(comm,)d(0,)i(&galaxy\);)218 2195 y(/*)h(...)f(*/)170 2251 y(})75 2357 y Fh(The)15 b(host/no)q(de)f(pro)q (cess)g(mo)q(del)h(can)g(b)q(e)g(considered)g(as)f(a)g(sp)q(ecial)i(case)f (where)f(the)g(spa)o(wning)h(com-)75 2414 y(m)o(unicator)10 b(consists)g(of)g(only)h(one)f(pro)q(cess)g(\()p Fg(e.g.)p Fh(,)h Fc(MPI)p 1008 2414 V 16 w(COMM)p 1120 2414 V 17 w(SELF)p Fh(\).)e(W)l(e)h(note)g(that)f(the)i Fc(MPI)p 1681 2414 V 16 w(Intercomm)p 1913 2414 V 16 w(merge)75 2470 y Fh(function)18 b(used)h(in)f(the)g(t)o(w)o(o)e(co)q(de)i(fragmen)o(ts)f(is)h(not)f (necessary)h(since)h(the)e(resulting)i(in)o(tercomm)o(u-)75 2527 y(nicator)f(can)g(b)q(e)g(used)g(for)g(comm)o(unication.)28 b(It)18 b(is)g(sho)o(wn)f(to)g(demonstrate)g(ho)o(w)h(to)f(create)g(a)h(new) 75 2583 y(in)o(tracomm)o(unicator)d(that)f(con)o(tains)h(the)h(complete)g (set)f(of)f(existing)j(pro)q(cesses.)964 2828 y(2)p eop %%Page: 3 4 bop 75 45 a Fi(0.2)59 b(Pro)r(cess)19 b(destruction)75 147 y Fh(Pro)q(cesses)f(exit)f(collectiv)o(ely)j(just)e(as)f(they)g(are)g (created)h(collectiv)o(ely)l(.)29 b(The)18 b(call)g(to)f Fc(MPI)p 1670 147 15 2 v 17 w(Finalize)75 203 y Fh(is)g(tan)o(tamoun)o(t)f(to)g Fc(MPI)p 504 203 V 17 w(Comm)p 617 203 V 16 w(free\()23 b(MPI)p 848 203 V 17 w(COMM)p 961 203 V 17 w(WORLD)g(\))17 b Fh(at)f(whic)o(h)i(p)q (oin)o(t)f(the)g(pro)q(cesses)g(depart)75 259 y(from)f(the)h(MPI)f(univ)o (erse)i(of)e(kno)o(wn)h(pro)q(cesses.)25 b(This)17 b(ho)o(w)o(ev)o(er)f(do)q (es)h(lea)o(v)o(e)g(the)g(problem)g(of)g(what)75 316 y(to)e(do)g(with)g (\\dangling")h(comm)o(unicators.)j(W)l(e)d(see)f(t)o(w)o(o)f(alternativ)o (es:)131 410 y(1.)22 b(All)c(comm)o(unicators)e(to)g(whic)o(h)i(a)e(pro)q (cess)h(b)q(elongs)h(should)g(b)q(e)f(freed)g(b)q(efore)g Fc(MPI)p 1670 410 V 17 w(Finalize)189 466 y Fh(is)e(called.)131 560 y(2.)22 b(Comm)o(unicators)9 b(to)g(whic)o(h)i(a)f(pro)q(cess)h(b)q(elongs)g (do)f(not)g(ha)o(v)o(e)f(to)h(b)q(e)h(freed)f(b)q(efore)h Fc(MPI)p 1686 560 V 16 w(Finalize)189 616 y Fh(is)k(called.)75 710 y(Both)e(c)o (hoices)h(imply)g(that)f(all)h(pro)q(cesses)f(in)h(a)f(relationship)h(with)g (the)f(exiting)h(group)f(kno)o(w)g(that)f(the)75 767 y(group)17 b(is)i(indeed)g(exiting.)28 b(The)18 b(\014rst)f(c)o(hoice)i(implies)h(this)e (since)h Fc(MPI)p 1343 767 V 16 w(Comm)p 1455 767 V 17 w(free)e Fh(m)o(ust)g(b)q(e)h(called)75 823 y(b)o(y)j(all)i(pro)q(cesses)e (participating)i(in)f(a)f(comm)o(unicator.)39 b(The)21 b(second)h(c)o(hoice)g (implies)i(this)e(since)75 880 y(remaining)16 b(pro)q(cesses)g(m)o(ust)e(kno) o(w)h(to)g(a)o(v)o(oid)g(using)h(\\in)o(v)m(alid")h(comm)o(unicators.)75 1023 y Fi(0.3)59 b(Alternatives)75 1124 y Fh(One)16 b(p)q(ossibile)h (alternativ)o(e)f(is)f(for)g(the)g(spa)o(wned)g(group)g(of)f(pro)q(cesses)i (b)q(e)g(merged)f(in)o(to)g(the)g(original)75 1181 y Fc(MPI)p 150 1181 V 17 w(COMM)p 263 1181 V 16 w(WORLD)p Fh(.)g(This)i(implies)h(that)e (all)h(pro)q(cesses)f(in)h(the)g(original)g Fc(MPI)p 1419 1181 V 16 w(COMM)p 1531 1181 V 17 w(WORLD)f Fh(m)o(ust)f(par-)75 1237 y(ticipate)e(in)g(an)o(y)f(spa)o(wn)g(call.)20 b(This)13 b(w)o(ould)g(also)f(cause)h(further)f(problems)h(with)f(pro)q(cess)h (destruction.)166 1294 y(Another)h(alternativ)o(e)h(w)o(as)e(for)h(an)g(in)o (tracomm)o(unicator)g(instead)g(of)g(an)g(in)o(tercomm)o(unicator)h(to)75 1350 y(b)q(e)f(returned)g(b)o(y)f Fc(MPI)p 454 1350 V 17 w(Comm)p 567 1350 V 16 w(spawn)p Fh(.)19 b(Our)13 b(prop)q(osal)h(allo)o(ws)g(the)f (creation)g(of)g(an)g(in)o(tracomm)o(unicator)75 1407 y(\(as)h(sho)o(wn)g(in) i(the)f(example)g(co)q(de\))h(if)f(needed,)h(but)e(do)q(es)h(not)g(require)g (it.)20 b(It)15 b(is)g(relativ)o(ely)h(simple)h(\(a)75 1463 y(single)h(call)g(to)e Fc(MPI)p 420 1463 V 16 w(Intercomm)p 652 1463 V 16 w(merge)p Fh(\))g(to)g(create)g(the)h(spanning)g(in)o(tracomm)o (unicator,)g(but)f(quite)75 1520 y(a)f(bit)h(more)e(complicated)j(to)d (create)h(an)h(in)o(tercomm)o(unicator)f(from)f(an)h(in)o(tracomm)o (unicator.)75 1663 y Fi(0.4)59 b(Summa)n(ry)75 1764 y Fh(W)l(e)15 b(mak)o(e)g(the)g(follo)o(wing)h(observ)m(ations:)143 1858 y Fb(\017)23 b Fh(the)15 b(paren)o(t)g(spa)o(wners)g(get)f(an)h(in)o(tercomm) o(unicator)h(bac)o(k,)143 1952 y Fb(\017)23 b Fh(spa)o(wn)15 b(is)g(a)g(collectiv)o(e)i(op)q(eration)e(for)g(spa)o(wning)h(pro)q(cesses,) 143 2046 y Fb(\017)23 b Fh(implemen)o(tations)16 b(are)f(not)g(required)h(to) f(implemen)o(t)h(the)f(spa)o(wn)g(function,)143 2139 y Fb(\017)23 b Fh(the)15 b(c)o(hildren's)i(w)o(orld)e(is)g(the)h(remote)e(group)h(of)g (the)g(paren)o(t's)g(in)o(tercomm)o(unicator,)143 2233 y Fb(\017)23 b Fh(paren)o(ts)14 b(and)i(c)o(hildren)h(ha)o(v)o(e)e(a)g(collectiv)o(e)i (relationship)f(immediately)l(,)143 2327 y Fb(\017)23 b Fh(this)c(b)q(o)q (otstraps)f(to)h(bigger)g(\\)p Fc(comm)p 832 2327 V 17 w(worlds)p Fh(")f(with)h(w)o(ell-c)o(haracterized)i(seman)o(tics,)f(and)f(no)189 2384 y(race)c(conditions,)143 2477 y Fb(\017)23 b Fh(pro)q(cesses)15 b(are)g(destro)o(y)o(ed)g(collectiv)o(ely)l(.)964 2828 y(3)p eop %%Page: 4 5 bop 75 377 a Fa(Biblio)q(graph)m(y)75 600 y Fh([1])22 b(William)h(Gropp,)f (Ewing)g(Lusk,)h(and)f(An)o(thon)o(y)f(Skjellum.)41 b Fg(Using)21 b(MPI:)g(Portable)h(Par)n(al)r(lel)146 656 y(Pr)n(o)n(gr)n(amming)15 b(with)i(the)g(Message)e(Passing)g(Interfac)n(e)p Fh(.)j(MIT)d(Press,)g (1994.)964 2828 y(4)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Fri Mar 10 17:30:15 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA19159; Fri, 10 Mar 1995 17:30:14 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA21235; Fri, 10 Mar 1995 17:30:36 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 10 Mar 1995 17:30:32 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id RAA21175; Fri, 10 Mar 1995 17:30:14 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA29454; Fri, 10 Mar 1995 16:30:03 -0600 Message-Id: <199503102230.QAA29454@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann), itf@mcs.anl.gov Subject: longspawn.ps Date: Fri, 10 Mar 1995 16:30:00 -0600 From: Rusty Lusk %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: paper.dvi %%CreationDate: Sun Feb 26 16:36:38 1995 %%Pages: 24 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips paper %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1995.02.26:1636 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet %%BeginProcSet: special.pro TeXDict begin /SDict 200 dict N SDict begin /@SpecialDefaults{/hs 612 N /vs 792 N /ho 0 N /vo 0 N /hsc 1 N /vsc 1 N /ang 0 N /CLIP 0 N /rwiSeen false N /rhiSeen false N /letter{}N /note{}N /a4{}N /legal{}N}B /@scaleunit 100 N /@hscale{@scaleunit div /hsc X}B /@vscale{@scaleunit div /vsc X}B /@hsize{/hs X /CLIP 1 N}B /@vsize{/vs X /CLIP 1 N}B /@clip{ /CLIP 2 N}B /@hoffset{/ho X}B /@voffset{/vo X}B /@angle{/ang X}B /@rwi{ 10 div /rwi X /rwiSeen true N}B /@rhi{10 div /rhi X /rhiSeen true N}B /@llx{/llx X}B /@lly{/lly X}B /@urx{/urx X}B /@ury{/ury X}B /magscale true def end /@MacSetUp{userdict /md known{userdict /md get type /dicttype eq{userdict begin md length 10 add md maxlength ge{/md md dup length 20 add dict copy def}if end md begin /letter{}N /note{}N /legal{} N /od{txpose 1 0 mtx defaultmatrix dtransform S atan/pa X newpath clippath mark{transform{itransform moveto}}{transform{itransform lineto} }{6 -2 roll transform 6 -2 roll transform 6 -2 roll transform{ itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll curveto}}{{ closepath}}pathforall newpath counttomark array astore /gc xdf pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{PaintBlack}if}N /txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 -1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop 90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr 2 get ppr 0 get neg sub neg 0 S TR}if}ifelse scaleby96{ppr aload pop 4 -1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S TR}if}N /cp {pop pop showpage pm restore}N end}if}if}N /normalscale{Resolution 72 div VResolution 72 div neg scale magscale{DVImag dup scale}if 0 setgray} N /psfts{S 65781.76 div N}N /startTexFig{/psf$SavedState save N userdict maxlength dict begin /magscale false def normalscale currentpoint TR /psf$ury psfts /psf$urx psfts /psf$lly psfts /psf$llx psfts /psf$y psfts /psf$x psfts currentpoint /psf$cy X /psf$cx X /psf$sx psf$x psf$urx psf$llx sub div N /psf$sy psf$y psf$ury psf$lly sub div N psf$sx psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub TR /showpage{}N /erasepage{}N /copypage{}N /p 3 def @MacSetUp}N /doclip{ psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto closepath clip newpath moveto}N /endTexFig{end psf$SavedState restore}N /@beginspecial{SDict begin /SpecialSave save N gsave normalscale currentpoint TR @SpecialDefaults count /ocount X /dcount countdictstack N}N /@setspecial {CLIP 1 eq{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs neg 0 rlineto closepath clip}if ho vo TR hsc vsc scale ang rotate rwiSeen{rwi urx llx sub div rhiSeen{rhi ury lly sub div}{dup}ifelse scale llx neg lly neg TR }{rhiSeen{rhi ury lly sub div dup scale llx neg lly neg TR}if}ifelse CLIP 2 eq{newpath llx lly moveto urx lly lineto urx ury lineto llx ury lineto closepath clip}if /showpage{}N /erasepage{}N /copypage{}N newpath }N /@endspecial{count ocount sub{pop}repeat countdictstack dcount sub{ end}repeat grestore SpecialSave restore end}N /@defspecial{SDict begin} N /@fedspecial{end}B /li{lineto}B /rl{rlineto}B /rc{rcurveto}B /np{ /SaveX currentpoint /SaveY X N 1 setlinecap newpath}N /st{stroke SaveX SaveY moveto}N /fil{fill SaveX SaveY moveto}N /ellipse{/endangle X /startangle X /yrad X /xrad X /savematrix matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix}N end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/alriga/alriga9/MPI/papers/spawn/paper.dvi) @start /Fa 27 120 df<001C0000003E0000003E0000002E0000006700000067000000 E7800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F00007 00F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038 001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 D<003FC000FFF003C0F0 0780300F00001E00003C00003C0000780000780000780000F00000F00000F00000F00000 F00000F00000F00000F00000F000007800007800007800003C00003C00001E00000F0008 07801803C07800FFF0003F80151F7D9D1B>67 DII<003F8001FFF003C0 F80780380F00181E00003C00003C0000780000780000780000F00000F00000F00000F000 00F00000F00000F007F8F007F8F000387800387800387800383C00383C00381E00380F00 3807803803C0F801FFF0003F80151F7D9D1C>71 D73 D76 DII<003F000001FFE00003FFF00007C0F8000F807C 001E001E003E001F003C000F00780007807800078078000780F00003C0F00003C0F00003 C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C078000780780007807C000F 803C000F003E001F001F003E000F807C0007C0F80003FFF00001FFE000003F00001A1F7E 9D1F>II82 D<03F8000FFE001C0F00380700700300600000E00000E000 00E00000E00000F000007800007F00003FE0001FFC0007FE0001FF00001F800007800003 C00003C00001C00001C00001C00001C0C00180E00380F007007C0E001FFC0007F000121F 7E9D17>III87 D<78000E007C001E003C003C001E0038000F00 70000F00F0000781E00003C1C00001C3C00001E7800000F70000007E0000003E0000003C 0000003C0000007E00000077000000E7800001E3800003C1C0000381E0000700F0000F00 F8000E0078001C003C003C003E0078001F0070000F00F0000F80191D7F9C1C>I<0FC03F F07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC3F1C0E127E 9114>97 D<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C3F FC1FF807E00E127E9112>101 D110 D112 D<078E1FEE3FFE7C3E781E700EE00EE00EE00EE00EE00EE00E F00E701E7C3E3FFE1FEE0F8E000E000E000E000E000E000E000E000E0F1A7E9115>II<1FC03FF07FF0F030E000E000F0007F003FC01FE000F0003800388038F078 FFF07FE01FC00D127F9110>I<1C001C001C001C001C001C00FFE0FFE01C001C001C001C 001C001C001C001C001C001C001C001C001C201FF00FF007C00C187F970F>II119 D E /Fb 53 122 df<00F8F001F8F003F8F00780F00700F00F 00F00F00F00F00F00F00F00F00F00F00F00F00F0FFF8F0FFF8F0FFF8F00F00F00F00F00F 00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F01420809F18>13 D<007000E001C00380078007000E001E001E003C00 3C003C0078007800780078007000F000F000F000F000F000F000F000F000F000F000F000 F000700078007800780078003C003C003C001E001E000E0007000780038001C000E00070 0C2E7EA112>40 DI<78 7878781830306060E0050A7D830D>44 DI<001F0000001F 0000003F8000003B8000003B8000007BC0000073C0000071C00000F1E00000E1E00000E0 E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C0007803C000700 3C000FFFFE000FFFFE000FFFFE001E000F001E000F003C000F803C0007803C0007807800 07C0780003C0780003C0F00003E01B207F9F1E>65 D<001FC000FFF801FFFC03E03C0780 0C0F00001E00003E00003C00007C0000780000780000780000F00000F00000F00000F000 00F00000F00000F00000F000007800007800007800007C00003C00003E00001E00000F00 0207800E03E03E01FFFC00FFF0001FC017227DA01D>67 DIII<001FE000FFF801FFFE03E03E07800E0F00001E00003E00003C0000 7C0000780000780000780000F00000F00000F00000F00000F00000F00000F003FEF003FE 7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07801E03E03E01FFFE00FFF8 001FC017227DA01E>III<003C003C003C003C003C003C003C003C003C003C003C003C003C003C 003C003C003C003C003C003C003C003C003C003C003C003C003C003CC078E078FFF07FE0 0FC00E217E9F15>I76 DII<003F000000FFC00003 FFF00007E1F8000F807C001F003E001E001E003C000F003C000F00780007807800078078 000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F0 0003C0F80007C07800078078000780780007803C000F003C000F001E001E001F003E000F 807C0007E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC00003FFF00007E1F8 000F807C001F003E001E001E003C000F003C000F00780007807800078078000780F00003 C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003 C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE000F83FC0007E1F8 0003FFF00000FFF800003F780000007C0000003E0000001E0000001F0000000F801A277D A021>II<01 FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C00003C 00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001E000 01E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019>I< FFFFFFC0FFFFFFC0FFFFFFC0001E0000001E0000001E0000001E0000001E0000001E0000 001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000 001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000 001E0000001E0000001E0000001E0000001E00001A207E9F1F>IIII<780007807C000F003E001F001E001E000F003C000F807C000780780003C0F00003 E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000003F000000 7F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C000F003E001F 001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>III<07E03FF87FFC701E401F000F000F000F003F07FF 1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III<01E001E001E001E00000000000 000000000000000000000001E001E001E001E001E001E001E001E001E001E001E001E001 E001E001E001E001E001E001E001E001E001E001E001E001E0C3C0FFC0FF803F000B2983 9F0C>I108 DII<01 F80007FE001FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F0 00F07801E07801E03C03C03F0FC01FFF8007FE0001F80014147F9317>II<03C7800FF7801FFF803E1F807C0F807807807807 80F00780F00780F00780F00780F00780F00780F807807807807C0F803E1F801FFF800FF7 8007C780000780000780000780000780000780000780000780000780000780111D7E9317 >II<07F01FFC3FFC780C7800780078007C003FC01FF00FF803 F8007C003C003CC03CF07CFFF87FF00FC00E147F9311>I<1E001E001E001E001E001E00 FFF0FFF0FFF01E001E001E001E001E001E001E001E001E001E001E001E001E001E201FF0 0FF007C00C1A7F9910>IIII<7801E07C 03C03E07801E0F000F0F00079E0003FC0003F80001F80000F00001F00001F80003FC0007 9E000F0F000E0F001E07803C03C07801E0F801F01414809315>II E /Fc 80 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8 F8F870000000000070F8F8F870051C779B18>33 D<4010E038F078E038E038E038E038E0 38E038E038E038E038E03860300D0E7B9C18>I<030600078F00078F00078F00078F0007 8F00078F007FFFC0FFFFE0FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007F FFC0FFFFE0FFFFE07FFFC01E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B 18>I<3803007C07807C0780EE0F80EE0F00EE0F00EE1F00EE1E00EE1E00EE3E007C3C00 7C3C00387C0000780000780000F80000F00001F00001E00001E00003E00003C00003C000 07C0000783800787C00F87C00F0EE00F0EE01F0EE01E0EE01E0EE03E0EE03C07C03C07C0 18038013247E9F18>37 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C 73F01C73F01CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0 F800E0F800E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>I<007000F001E0 03C007800F001E001C00380038007000700070007000E000E000E000E000E000E000E000 E0007000700070007000380038001C001E000F00078003C001F000F000700C24799F18> 40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E00070007000 700070007000700070007000E000E000E000E001C001C0038007800F001E003C007800F0 0060000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC 0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C0001114 7D9718>I<00600000F00000F00000F00000F00000F00000F00000F0007FFFC0FFFFE0FF FFE07FFFC000F00000F00000F00000F00000F00000F00000F00000600013147E9718>I< 1C3E7E7F3F1F070E1E7CF860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D8F18> I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00001E0000 1E00003E00003C00007C0000780000780000F80000F00001F00001E00003E00003C00003 C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C00007C000078 0000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F001C0700 3803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0 E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F000131C7E 9B18>I<01800380038007800F803F80FF80FB8043800380038003800380038003800380 0380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03F0000FFE 003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C00001C00003 C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C00E07FFF E0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C07801C03001C000 01C00003C0000380000F0003FF0003FE0003FF000007800003C00001C00000E00000E000 00E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<001F00 003F0000770000770000E70001E70001C7000387000787000707000E07001E07003C0700 380700780700F00700FFFFF8FFFFF8FFFFF8000700000700000700000700000700000700 007FF000FFF8007FF0151C7F9B18>I<1FFF803FFF803FFF803800003800003800003800 003800003800003800003800003BF8003FFE003FFF003C07801803C00001C00000E00000 E06000E0F000E0F000E0E001C07003C07C0F803FFF001FFC0003F000131C7E9B18>I<30 78FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C18000000 0000000000183C7E7E3E1E0E1C3C78F060071A789318>I<000300000780001F80003F00 007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC00007E00003F00001FC000 07E00003F00001FC00007E00003F00001F8000078000030011187D9918>I<7FFFC0FFFF E0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C7E9318 >I<0FF0003FFC007FFF00700F00F00380F00380600780000F00003E00007C0001F00001 E00003C00003C00003C00003C00003C00003800000000000000000000000000000000003 800007C00007C00007C000038000111C7D9B18>63 D<00700000F80000F80000D80000D8 0001DC0001DC0001DC00018C00038E00038E00038E00038E000306000707000707000707 000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C 7F9B18>65 DI<00F8E003FEE007FFE00F07 E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E000 00E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F8 00131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C 00F01C00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C 01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00F FFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E0 0000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003 FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C01C0 1C01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C0 1C01C01C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF 807FFF0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF 00FFFF807FFF00111C7D9B18>I<01FFC003FFC001FFC0000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00F00E00F00E00F03C007FFC003FF0000FC000121C7D9B18>I<7FE000 FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00700E00700E00700E00700E0070 7FFFF0FFFFF07FFFF0141C7F9B18>76 DI< 7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C0 1C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC0 1C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F00700700F007 80E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C7D9B18 >II<0FF8003FFE007FFF00780F00700700 F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00001C00 001E00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF001C0F801C03 801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F001C07 001C03801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F87F00F0161C 7F9B18>I<03F3801FFF803FFF807C0F80700780E00380E00380E00380E0000070000078 00003F00001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E000E0E0 01E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038 E07038E07038E07038007000007000007000007000007000007000007000007000007000 00700000700000700000700000700000700000700000700000700007FF0007FF0007FF00 151C7F9B18>IIII<7F8FE07F9FE07F8F E00E07000F0700070E00078E00039C0003DC0001F80001F80000F00000F00000700000F0 0000F80001F80001DC00039E00038E00070F000707000E07800E03801E03C07F07F0FF8F F87F07F0151C7F9B18>II 91 D<600000F00000F00000F800007800007C00003C00003C00003E00001E00001F0000 0F00000F00000F800007800007C00003C00003C00003E00001E00001F00000F00000F800 007800007800007C00003C00003E00001E00001E00001F00000F00000F80000780000780 00030011247D9F18>II<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I<0038007C007C007C003800000000000000000FFC1FFC0FFC001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C6038F078FFF07FE03F800E277E9C18>II<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF800 1F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F0019148193 18>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0 000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001 E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807F FFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F 00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E 00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780 700780E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF80 0FFB8003E380000380000380000380000380000380000380000380003FF8003FF8003FF8 151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C0000380000380 000380000380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318 >I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE0000 1F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<018000 0380000380000380000380007FFFC0FFFFC0FFFFC0038000038000038000038000038000 0380000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E00 13197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318 >I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E0003 8E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8F F0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E 000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E 03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000 DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F80003F 00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00 003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF0 14147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000007800007F E0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003FE0003FE000 7FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00003C000FFC000FF00007C000013247E9F18>125 D E /Fd 52 123 df<387CFEFFFF7F3B030306060E0C18702008107C9F0F>39 D45 D<387CFEFEFE7C3807077C860F>I<0000600000 E00000E00001C00001C0000380000380000380000700000700000700000E00000E00001C 00001C00001C0000380000380000380000700000700000E00000E00000E00001C00001C0 000380000380000380000700000700000700000E00000E00001C00001C00001C00003800 00380000380000700000700000E00000E00000C00000132D7DA11A>I<00E00001E0000F E000FFE000F3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003 E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003 E000FFFF80FFFF80111D7C9C1A>49 D<07F0001FFE00383F007C1F80FE0FC0FE0FC0FE0F E0FE07E07C07E03807E0000FE0000FC0000FC0001F80001F00003E0000780000F00000E0 0001C0000380600700600E00601C00E01FFFC03FFFC07FFFC0FFFFC0FFFFC0131D7D9C1A >I<01FC0007FF000E0F801E0FC03F07E03F07E03F07E03F07E01E0FC0000FC0000F8000 1F0001FC0001FC00000F800007C00003E00003F00003F83803F87C03F8FE03F8FE03F8FE 03F0FC03F07807E03C0FC01FFF8003FC00151D7E9C1A>I<0001C00003C00007C00007C0 000FC0001FC0003BC00073C00063C000C3C00183C00383C00703C00E03C00C03C01803C0 3803C07003C0E003C0FFFFFEFFFFFE0007C00007C00007C00007C00007C00007C000FFFE 00FFFE171D7F9C1A>I<3803803FFF803FFF003FFE003FFC003FF0003F80003000003000 0030000030000033F80037FE003C1F00380F801007C00007C00007E00007E07807E0FC07 E0FC07E0FC07E0FC07C0780FC0600F80381F001FFC0007F000131D7D9C1A>I<003F0001 FFC007E0E00F81E01F03F01E03F03E03F07C03F07C01E07C0000FC1000FCFF00FDFFC0FD 03E0FE01F0FE01F0FC01F8FC01F8FC01F8FC01F87C01F87C01F87C01F83C01F03E01F01E 03E00F07C007FF8001FE00151D7E9C1A>I<0000E000000000E000000001F000000001F0 00000001F000000003F800000003F800000006FC00000006FC0000000EFE0000000C7E00 00000C7E000000183F000000183F000000303F800000301F800000701FC00000600FC000 00600FC00000C007E00000FFFFE00001FFFFF000018003F000018003F000030001F80003 0001F800060001FC00060000FC000E0000FE00FFE00FFFE0FFE00FFFE0231F7E9E28>65 DI<0007FC02003FFF0E00FE03DE 03F000FE07E0003E0FC0001E1F80001E3F00000E3F00000E7F0000067E0000067E000006 FE000000FE000000FE000000FE000000FE000000FE000000FE0000007E0000007E000006 7F0000063F0000063F00000C1F80000C0FC0001807E0003803F0007000FE01C0003FFF80 0007FC001F1F7D9E26>IIII73 D<03FFFC03FFFC000FC0000FC0000FC0000FC0000FC0000FC0000FC0000FC0000FC0000F C0000FC0000FC0000FC0000FC0000FC0000FC0000FC0000FC0000FC0000FC0100FC07C0F C0FE0FC0FE0FC0FE0F80FC1F80783F003FFE000FF000161F7F9E1B>I76 DII80 D<001FF80000FFFF0001F81F8007E007E00FC003F01F8001F81F8001F83F0000FC7F0000 FE7F0000FE7E00007EFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE0000 7FFE00007FFE00007F7E00007E7E00007E7F0000FE3F0000FC3F87C1FC1F8FE1F80FD833 F007F83FE001F81F8000FFFF00001FFE0300001E0300001F0700000FFF00000FFF000007 FE000007FE000003FC000001F8000000F020287D9E27>I<03FC080FFF381E03F83800F8 700078700038F00038F00018F00018F80000FC00007FC0007FFE003FFF801FFFE00FFFF0 07FFF000FFF80007F80000FC00007C00003CC0003CC0003CC0003CE00038E00078F80070 FE01E0E7FFC081FF00161F7D9E1D>83 D<7FFFFFFC7FFFFFFC7C07E07C7007E01C6007E0 0C6007E00CE007E00EC007E006C007E006C007E006C007E0060007E0000007E0000007E0 000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0 000007E0000007E0000007E0000007E0000007E00003FFFFC003FFFFC01F1E7E9D24>I< FFFE003FF8FFFE003FF807E000038007E000030007F000070003F000060003F8000E0001 F8000C0001FC000C0000FC00180000FC001800007E003000007E003000003F006000003F 006000003F80E000001F80C000001FC1C000000FC18000000FE180000007E300000007E3 00000003F600000003F600000003FE00000001FC00000001FC00000000F800000000F800 0000007000000000700000251F7F9E28>86 DI<07FC001FFF003F0F80 3F07C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0 F803E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC0 7C0300FC0000FC0000FC0000FC0000FC0000FC00007C00007E00003E00603F00C01F81C0 07FF0001FC0013147E9317>I<0007F80007F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F801F8F80FFEF81F83F83E01F87E00F87C00F87C00F8FC00 F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F87E00F83E01F81F07F80FFEFF03F8 FF18207E9F1D>I<01FE0007FF800F83C01E01E03E00F07C00F07C00F8FC00F8FFFFF8FF FFF8FC0000FC0000FC00007C00007C00003E00181E00180F807007FFE000FF8015147F93 18>I<001F8000FFC001F3E003E7E003C7E007C7E007C3C007C00007C00007C00007C000 07C000FFFC00FFFC0007C00007C00007C00007C00007C00007C00007C00007C00007C000 07C00007C00007C00007C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC 3C07FFFE0F079E1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC 001800001800001C00001FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF000 1E78003C78003C3F01F80FFFE001FF00171E7F931A>II<1C003E007F007F007F003E001C0000000000 0000000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F00 1F001F001F001F00FFE0FFE00B217EA00E>I<0038007C00FE00FE00FE007C0038000000 00000000000000000001FE01FE003E003E003E003E003E003E003E003E003E003E003E00 3E003E003E003E003E003E003E003E003E303E783EFC3CFC7C78783FF01FC00F2A83A010 >IIIII<01FF0007FFC01F83F03E 00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC007EFC007E7C007C7C007C3E 00F83E00F81F83F007FFC001FF0017147F931A>II114 D<0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC007C007E007 F006F81EFFFCC7F010147E9315>I<01800180018003800380038007800F803F80FFFCFF FC0F800F800F800F800F800F800F800F800F800F800F860F860F860F860F8607CC03F801 F00F1D7F9C14>IIIIII<3FFFE03FFFE03C07C0380F80701F80603F00603E00607C0000 F80001F80003F00003E06007C0600F80601F80E03F00C03E01C07C03C0FFFFC0FFFFC013 147F9317>I E /Fe 41 125 df<0E1F3F3F1D0102020404081020C0080E779F0E>39 D<7FF0FFE07FE00C037D8A10>45 D<001F800060E0008070010030020038042038042038 0410380420700460700380600000E00001C000030000FE00001C00000600000700000780 000780000780300780780780780780F00F00800F00401E00401C0040380020E0001F8000 151F7C9D17>51 D<001F0000718000C0C00180C00380E00700E00F00E00F01E01E01E01E 01E01E01E01E01C01C03C01C03C01C03C01C07C01C0F800C0F8006378003C700000F0000 0E00000E00001C00601C00F03800F07000E0600080C0004380003E0000131F7B9D17>57 D<070F1F1F0E0000000000000000000070F8F8F0E008147B930E>I<0000020000000600 0000060000000E0000001E0000001E0000003F0000002F0000004F0000004F0000008F00 00010F0000010F0000020F0000020F0000040F00000C0F0000080F0000100F0000100F00 00200F80003FFF800040078000C007800080078001000780010007800200078002000780 060007801E000F80FF807FF81D207E9F22>65 D<0000FE0200078186001C004C0038003C 0060003C00C0001C01C0001803800018070000180F0000181E0000101E0000103C000000 3C00000078000000780000007800000078000000F0000000F0000000F0000000F0000000 F00000807000008070000080700001003800010038000200180004000C00180006002000 0381C00000FE00001F217A9F21>67 D<0000FC040007030C001C00980030007800E00078 01C000380380003003800030070000300E0000301E0000201E0000203C0000003C000000 78000000780000007800000078000000F0000000F000FFF0F0000780F0000780F0000F00 70000F0070000F0070000F0070001E0038001E0018003E001C002E000E00CC0003830400 00FC00001E217A9F23>71 D<01FFF0001F00001E00001E00001E00003C00003C00003C00 003C0000780000780000780000780000F00000F00000F00000F00001E00001E00001E000 01E00003C00003C00003C00003C0000780000780000780000780000F8000FFF800141F7D 9E12>73 D<001FFF0000F80000F00000F00000F00001E00001E00001E00001E00003C000 03C00003C00003C0000780000780000780000780000F00000F00000F00000F00001E0000 1E00301E00781E00F83C00F83C00F0780080700040E00021C0001F000018207D9E18>I< 01FE00007FC0001E0000FC00001E0000F80000170001780000170001780000270002F000 00270004F00000270004F00000270008F00000470009E00000470011E00000470021E000 00470021E00000870043C00000838043C00000838083C00000838083C000010381078000 0103820780000103820780000103840780000203840F00000203880F00000203900F0000 0203900F00000401E01E00000401E01E00000401C01E00000C01801E00001C01803E0000 FF8103FFC0002A1F7D9E29>77 D<01FF007FE0001F000F00001F00040000178004000017 80040000278008000023C008000023C008000023C008000041E010000041E010000041F0 10000040F010000080F0200000807820000080782000008078200001003C400001003C40 0001003C400001001E400002001E800002001E800002000F800002000F800004000F0000 040007000004000700000C000700001C00020000FF80020000231F7D9E22>I<01FFFF80 001E00E0001E0070001E0038001E003C003C003C003C003C003C003C003C003C00780078 00780078007800F0007800E000F003C000F00F0000FFFC0000F0000001E0000001E00000 01E0000001E0000003C0000003C0000003C0000003C00000078000000780000007800000 078000000F800000FFF000001E1F7D9E1F>80 D<0007E040001C18C00030058000600380 00C0038001C00180018001000380010003800100038001000380000003C0000003C00000 03F8000001FF800001FFE000007FF000001FF0000001F800000078000000780000003800 0000380020003800200038002000300060007000600060006000E0007000C000E8038000 C606000081F800001A217D9F1A>83 D<0FFFFFF01E0780E0180780201007802020078020 200F0020600F0020400F0020400F0020801E0040001E0000001E0000001E0000003C0000 003C0000003C0000003C00000078000000780000007800000078000000F0000000F00000 00F0000000F0000001E0000001E0000001E0000001E0000003E00000FFFF00001C1F789E 21>I<7FFC1FF807C003C00780010007800100078001000F0002000F0002000F0002000F 0002001E0004001E0004001E0004001E0004003C0008003C0008003C0008003C00080078 001000780010007800100078001000F0002000F0002000F0002000F0004000F000400070 0080007001000030020000380400000C18000007E000001D20779E22>II<00F1800389C00707800E03801C0380 3C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40 705C40308C800F070013147C9317>97 D<07803F8007000700070007000E000E000E000E 001C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE01EE01CE03CE0 38607060E031C01F0010207B9F15>I<007E0001C1000300800E07801E07801C07003C02 00780000780000780000F00000F00000F00000F00000F000007001007002003004001838 0007C00011147C9315>I<0000780003F80000700000700000700000700000E00000E000 00E00000E00001C00001C000F1C00389C00707800E03801C03803C038038070078070078 0700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F070015 207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000F000F0007000 700170023804183807C010147C9315>I<00007800019C00033C00033C00071800070000 0700000E00000E00000E00000E00000E0001FFE0001C00001C00001C00001C0000380000 380000380000380000380000700000700000700000700000700000700000E00000E00000 E00000E00000C00001C00001C0000180003180007B0000F300006600003C00001629829F 0E>I<003C6000E27001C1E00380E00700E00F00E00E01C01E01C01E01C01E01C03C0380 3C03803C03803C03803C07003C07001C0F001C17000C2E0003CE00000E00000E00001C00 001C00301C00783800F0700060E0003F8000141D7E9315>I<01E0000FE00001C00001C0 0001C00001C000038000038000038000038000070000070000071E000763000E81800F01 C00E01C00E01C01C03801C03801C03801C0380380700380700380700380E10700E20700C 20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C00000000000000000 0000000000000E003300230043804300470087000E000E000E001C001C001C0038403880 30807080310033001C000B1F7C9E0E>I<0001800003C00003C000038000000000000000 0000000000000000000000000000003C00004600008700008700010700010700020E0000 0E00000E00000E00001C00001C00001C00001C0000380000380000380000380000700000 700000700000700000E00000E00030E00079C000F180006300003C00001228829E0E>I< 01E0000FE00001C00001C00001C00001C000038000038000038000038000070000070000 0703C00704200E08E00E11E00E21E00E40C01C80001D00001E00001FC00038E000387000 387000383840707080707080707080703100E03100601E0013207D9F15>I<03C01FC003 8003800380038007000700070007000E000E000E000E001C001C001C001C003800380038 0038007000700070007100E200E200E200E200640038000A207C9F0C>I<1C0F80F00026 30C318004740640C004780680E004700700E004700700E008E00E01C000E00E01C000E00 E01C000E00E01C001C01C038001C01C038001C01C038001C01C070803803807100380380 6100380380E10038038062007007006600300300380021147C9325>I<1C0F802630C047 40604780604700704700708E00E00E00E00E00E00E00E01C01C01C01C01C01C01C038438 03883803083807083803107003303001C016147C931A>I<007C0001C3000301800E01C0 1E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0F00780F00700700F00 700E0030180018700007C00013147C9317>I<01C1E002621804741C04781C04701E0470 1E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380780380700380E003C1 C0072380071E000700000700000E00000E00000E00000E00001C00001C0000FFC000171D 809317>I<1C1E002661004783804787804707804703008E00000E00000E00000E00001C 00001C00001C00001C000038000038000038000038000070000030000011147C9313> 114 D<00FC030206010C030C070C060C000F800FF007F803FC003E000E700EF00CF00CE0 08401020601F8010147D9313>I<018001C0038003800380038007000700FFF007000E00 0E000E000E001C001C001C001C003800380038003820704070407080708031001E000C1C 7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E03801C 07001C07001C07001C07101C0E20180E20180E201C1E200C264007C38014147C9318>I< 0E03803307802307C04383C04301C04700C08700800E00800E00800E00801C01001C0100 1C01001C02001C02001C04001C04001C08000E300003C00012147C9315>I<0E00C1C033 00E3C02301C3E04381C1E04301C0E04701C060870380400E0380400E0380400E0380401C 0700801C0700801C0700801C0701001C0701001C0602001C0F02000C0F04000E13080003 E1F0001B147C931E>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03 800E03801C07001C07001C07001C07001C0E00180E00180E001C1E000C3C0007DC00001C 00001C00003800F03800F07000E06000C0C0004380003E0000131D7C9316>121 D124 D E /Ff 36 122 df45 D<387CFEFEFE7C3807077C8610>I<00180000780001F800FFF800FFF80001F800 01F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F800 01F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F800 7FFFE07FFFE013207C9F1C>49 D<03FC000FFF003C1FC07007E07C07F0FE03F0FE03F8FE 03F8FE01F87C01F83803F80003F80003F00003F00007E00007C0000F80001F00003E0000 380000700000E01801C0180380180700180E00380FFFF01FFFF03FFFF07FFFF0FFFFF0FF FFF015207D9F1C>I<00FE0007FFC00F07E01E03F03F03F03F81F83F81F83F81F81F03F8 1F03F00003F00003E00007C0001F8001FE0001FF000007C00001F00001F80000FC0000FC 3C00FE7E00FEFF00FEFF00FEFF00FEFF00FC7E01FC7801F81E07F00FFFC001FE0017207E 9F1C>I<000070000000007000000000F800000000F800000000F800000001FC00000001 FC00000003FE00000003FE00000003FE00000006FF000000067F0000000E7F8000000C3F 8000000C3F800000183FC00000181FC00000381FE00000300FE00000300FE00000600FF0 00006007F00000E007F80000FFFFF80000FFFFF800018001FC00018001FC00038001FE00 030000FE00030000FE000600007F000600007F00FFE00FFFF8FFE00FFFF825227EA12A> 65 D<0003FE0080001FFF818000FF01E38001F8003F8003E0001F8007C0000F800F8000 07801F800007803F000003803F000003807F000001807E000001807E00000180FE000000 00FE00000000FE00000000FE00000000FE00000000FE00000000FE00000000FE00000000 7E000000007E000001807F000001803F000001803F000003801F800003000F8000030007 C000060003F0000C0001F800380000FF00F000001FFFC0000003FE000021227DA128>67 DII73 D77 D80 D82 D<01FC0407FF8C1F03FC3C007C7C003C78001C78001CF8000CF8000CFC000CFC0000FF00 00FFE0007FFF007FFFC03FFFF01FFFF80FFFFC03FFFE003FFE0003FF00007F00003F0000 3FC0001FC0001FC0001FE0001EE0001EF0003CFC003CFF00F8C7FFE080FF8018227DA11F >I86 D<07FC001FFF803F07C03F03E03F01E03F01F01E01F00001F00001F0003FF003FDF01FC1 F03F01F07E01F0FC01F0FC01F0FC01F0FC01F07E02F07E0CF81FF87F07E03F18167E951B >97 D<00FF8007FFE00F83F01F03F03E03F07E03F07C01E07C0000FC0000FC0000FC0000 FC0000FC0000FC00007C00007E00007E00003E00301F00600FC0E007FF8000FE0014167E 9519>99 D<00FE0007FF800F87C01E01E03E01F07C00F07C00F8FC00F8FC00F8FFFFF8FF FFF8FC0000FC0000FC00007C00007C00007E00003E00181F00300FC07003FFC000FF0015 167E951A>101 D<003F8000FFC001E3E003C7E007C7E00F87E00F83C00F80000F80000F 80000F80000F80000F8000FFFC00FFFC000F80000F80000F80000F80000F80000F80000F 80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80007F F8007FF80013237FA211>I<03FC1E0FFF7F1F0F8F3E07CF3C03C07C03E07C03E07C03E0 7C03E07C03E03C03C03E07C01F0F801FFF0013FC003000003000003800003FFF801FFFF0 0FFFF81FFFFC3800FC70003EF0001EF0001EF0001EF0001E78003C7C007C3F01F80FFFE0 01FF0018217E951C>II<1C003E007F007F007F003E001C00000000000000000000 0000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F001F 001F001F001F001F00FFE0FFE00B247EA310>I108 DII<00FE0007FFC00F83E01E00F03E00F87C007C7C007C7C007CFC007E FC007EFC007EFC007EFC007EFC007EFC007E7C007C7C007C3E00F81F01F00F83E007FFC0 00FE0017167E951C>I I<00FE030007FF87000FC1C7001F006F003F003F007E003F007E001F007C001F00FC001F 00FC001F00FC001F00FC001F00FC001F00FC001F00FC001F007E001F007E001F003E003F 001F007F000FC1DF0007FF9F0001FC1F0000001F0000001F0000001F0000001F0000001F 0000001F0000001F0000001F000000FFE00000FFE01B207E951E>II<0FF3003FFF0078 1F00600700E00300E00300F00300FC00007FE0007FF8003FFE000FFF0001FF00000F80C0 0780C00380E00380E00380F00700FC0E00EFFC00C7F00011167E9516>I<018000018000 0180000180000380000380000780000780000F80003F8000FFFF00FFFF000F80000F8000 0F80000F80000F80000F80000F80000F80000F80000F80000F80000F81800F81800F8180 0F81800F81800F830007C30003FE0000F80011207F9F16>IIIIII E /Fg 40 122 df<007E1F8001C170400703C060060380E00E0380400E0380000E0380000E0380000E03 80000E038000FFFFFFE00E0380E00E0380E00E0380E00E0380E00E0380E00E0380E00E03 80E00E0380E00E0380E00E0380E00E0380E00E0380E00E0380E00E0380E07F8FE3FC1E1A 809920>14 D<60F0F07010101020204080040B7D830B>44 DI< 60F0F06004047D830B>I<078018603030303060186018E01CE01CE01CE01CE01CE01CE0 1CE01CE01CE01CE01CE01C6018601870383030186007800E187E9713>48 D<03000700FF000700070007000700070007000700070007000700070007000700070007 0007000700070007000700FFF00C187D9713>I<0F8010E0207060787038203800780070 0070006000C00F8000E000700038003C003CE03CE03CC03C4038407030E00F800E187E97 13>51 D<078018603030201860186018601870103C303E600F8007C019F030F86038401C C00CC00CC00CC00C6008201018600FC00E187E9713>56 D<07801860303070306018E018 E018E01CE01CE01C601C603C303C185C0F9C001C00180018003870307060604021801F00 0E187E9713>I<000C0000000C0000000C0000001E0000001E0000003F00000027000000 2700000043800000438000004380000081C0000081C0000081C0000100E0000100E00001 FFE000020070000200700006007800040038000400380008001C0008001C001C001E00FF 00FFC01A1A7F991D>65 D68 DI77 D<007F000001C1C000070070000E0038001C001C003C001E0038000E007800 0F0070000700F0000780F0000780F0000780F0000780F0000780F0000780F0000780F000 078078000F0078000F0038000E003C001E001C001C000E0038000700700001C1C000007F 0000191A7E991E>79 D82 D<0FC21836200E6006C006C002C002C002E00070007E00 3FE01FF807FC003E000E00070003800380038003C002C006E004D81887E0101A7E9915> I<7FFFFF00701C0700401C0100401C0100C01C0180801C0080801C0080801C0080001C00 00001C0000001C0000001C0000001C0000001C0000001C0000001C0000001C0000001C00 00001C0000001C0000001C0000001C0000001C0000001C0000001C000003FFE000191A7F 991C>II87 D<3F8070C070E020700070007007F01C7030707070E070E071E0 71E0F171FB1E3C10107E8F13>97 DI<07F80C1C381C3008 7000E000E000E000E000E000E0007000300438080C1807E00E107F8F11>I<007E00000E 00000E00000E00000E00000E00000E00000E00000E00000E0003CE000C3E00380E00300E 00700E00E00E00E00E00E00E00E00E00E00E00E00E00600E00700E00381E001C2E0007CF C0121A7F9915>I<07C01C3030187018600CE00CFFFCE000E000E000E000600030043808 0C1807E00E107F8F11>I<01F0031807380E100E000E000E000E000E000E00FFC00E000E 000E000E000E000E000E000E000E000E000E000E000E000E007FE00D1A80990C>I<0FCE 187330307038703870387038303018602FC02000600070003FF03FFC1FFE600FC003C003 C003C0036006381C07E010187F8F13>II<18003C003C00 1800000000000000000000000000FC001C001C001C001C001C001C001C001C001C001C00 1C001C001C001C00FF80091A80990A>I107 DIII<07E01C38300C700E6006E007E007E007E007E007E0076006700E381C1C38 07E010107F8F13>II114 D<1F2060E04020C020C020F0007F003FC01FE0 00F080708030C030C020F0408F800C107F8F0F>I<0400040004000C000C001C003C00FF C01C001C001C001C001C001C001C001C001C201C201C201C201C200E4003800B177F960F >II119 D121 D E /Fh 1 4 df<0C000C008C40EDC07F800C007F80EDC08C400C000C000A0B7D8B10>3 D E /Fi 80 125 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<70F8F8F8F8F8F8F87070707070707070707020202020 20000000000070F8F8F87005217CA00D>33 D<7038F87CFC7EFC7E743A04020402040208 04080410081008201040200F0E7E9F17>I<70F8FCFC74040404080810102040060E7C9F 0D>39 D<0020004000800100020006000C000C0018001800300030003000700060006000 6000E000E000E000E000E000E000E000E000E000E000E000E00060006000600070003000 30003000180018000C000C000600020001000080004000200B2E7DA112>I<8000400020 00100008000C00060006000300030001800180018001C000C000C000C000E000E000E000 E000E000E000E000E000E000E000E000E000C000C000C001C00180018001800300030006 0006000C00080010002000400080000B2E7DA112>I<70F8FCFC74040404080810102040 060E7C840D>44 DI<70F8F8F87005057C840D>I<000100030003 000600060006000C000C000C00180018001800300030003000600060006000C000C000C0 0180018001800300030003000600060006000C000C000C00180018001800300030003000 600060006000C000C000C000102D7DA117>I<03F0000E1C001C0E001806003807007003 80700380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003 C0F003C0F003C0F003C0F003C07003807003807003807807803807001806001C0E000E1C 0003F000121F7E9D17>I<018003800F80F3800380038003800380038003800380038003 8003800380038003800380038003800380038003800380038003800380038007C0FFFE0F 1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C0 2007C00007C0000780000780000F00000E00001C0000380000700000600000C000018000 0300000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C 00100E00200F00780F80780780780780380F80000F80000F00000F00000E00001C000038 0003F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F007 80400780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E0000 1E00002E00002E00004E00008E00008E00010E00020E00020E00040E00080E00080E0010 0E00200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E0000 0E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE000100000100000100000 10000010000010000011F000161C00180E001007001007800003800003800003C00003C0 0003C07003C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E 9D17>I<007C000182000701000E03800C07801C07803803003800007800007000007000 00F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003 C07003C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007F FFC07FFF807FFF8040010080020080020080040000080000080000100000200000200000 400000400000C00000C00001C00001800003800003800003800003800007800007800007 8000078000078000078000078000030000121F7D9D17>I<03F0000C0C00100600300300 2001806001806001806001807001807803003E03003F06001FC8000FF00003F80007FC00 0C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0C00080600180200100 1002000C0C0003F000121F7E9D17>I<03F0000E18001C0C003806003807007007007003 80F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13 C003E3C0000380000380000380000700300700780600780E00700C002018001070000FC0 00121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8 F8F8700000000000000000000070F0F8F878080808101010202040051D7C930D>I<0001 00000003800000038000000380000007C0000007C0000007C0000009E0000009E0000009 E0000010F0000010F0000010F00000207800002078000020780000403C0000403C000040 3C0000801E0000801E0000FFFE0001000F0001000F0001000F0002000780020007800200 0780040003C00E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0038005C0070003C00E0001 C01E0000C01C0000C03C0000C07C0000407C00004078000040F8000000F8000000F80000 00F8000000F8000000F8000000F8000000F8000000F8000000780000007C0000407C0000 403C0000401C0000401E0000800E000080070001000380020001C0040000703800000FC0 001A217D9F21>IIII<000FE0200078186000E004E0038002E0070001E00F0000E01E0000601E0000 603C0000603C0000207C00002078000020F8000000F8000000F8000000F8000000F80000 00F8000000F8000000F8007FFCF80003E0780001E07C0001E03C0001E03C0001E01E0001 E01E0001E00F0001E0070001E0038002E000E0046000781820000FE0001E217D9F24>I< FFF8FFF80F800F8007800F0007800F0007800F0007800F0007800F0007800F0007800F00 07800F0007800F0007800F0007800F0007800F0007FFFF0007800F0007800F0007800F00 07800F0007800F0007800F0007800F0007800F0007800F0007800F0007800F0007800F00 07800F0007800F000F800F80FFF8FFF81D1F7E9E22>II<0FFFC0007C00003C00003C00003C0000 3C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0000 3C00003C00003C00003C00003C00003C00203C00F83C00F83C00F83C00F0380040780040 700030E0000F800012207E9E17>IIIII<00 1F800000F0F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C 0003E0780001E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F8 0001F0F80001F0F80001F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E 0007000F000F0007801E0001C0380000F0F000001F80001C217D9F23>II<001F800000F0F00001C0380007801E000F000F 000E0007001E0007803C0003C03C0003C07C0003E07C0003E0780001E0F80001F0F80001 F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E0780001 E07C0003E03C0003C03C0F03C01E1087800E2047000F204F0007A03E0001E0380000F0F0 10001FB01000003010000038300000387000003FF000001FE000001FE000000FC0000007 801C297D9F23>II<07 E0800C1980100780300380600180600180E00180E00080E00080E00080F00000F0000078 00007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C08001C080 01C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7FFFFF E0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F0010800F00 10000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003C002 0001E0040001F00C0000F0080000781000007C1000003C2000003E4000001E4000000F80 00000F8000000780000003C0000007E0000005E0000009F0000018F8000010780000207C 0000603C0000401E0000801F0001800F0001000780020007C0070003C01F8007E0FFE01F FE1F1F7F9E22>I91 D<0804100820102010402040 20804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E001E0E 00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>III E /Fj 35 122 df<1C007F007F00FF80FFC0 FFC07FC07FC01CC000C000C00180018001800300030006000C001800300020000A157B88 13>44 D<000E00001E00007E0007FE00FFFE00FFFE00F8FE0000FE0000FE0000FE0000FE 0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE 0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE0000FE 0000FE007FFFFE7FFFFE7FFFFE17277BA622>49 D<00FF800003FFF0000FFFFC001F03FE 003800FF007C007F80FE003FC0FF003FC0FF003FE0FF001FE0FF001FE07E001FE03C003F E000003FE000003FC000003FC000007F8000007F000000FE000000FC000001F8000003F0 000003E00000078000000F0000001E0000003C00E0007000E000E000E001C001C0038001 C0070001C00FFFFFC01FFFFFC03FFFFFC07FFFFFC0FFFFFF80FFFFFF80FFFFFF801B277D A622>I<007F800003FFF00007FFFC000F81FE001F00FF003F80FF003F807F803F807F80 3F807F801F807F800F007F800000FF000000FF000000FE000001FC000001F8000007F000 00FFC00000FFF0000001FC0000007E0000007F0000007F8000003FC000003FC000003FE0 00003FE03C003FE07E003FE0FF003FE0FF003FE0FF003FC0FF007FC07E007F807C007F00 3F01FE001FFFFC0007FFF00000FF80001B277DA622>I<00000E0000001E0000003E0000 007E000000FE000000FE000001FE000003FE0000077E00000E7E00000E7E00001C7E0000 387E0000707E0000E07E0000E07E0001C07E0003807E0007007E000E007E000E007E001C 007E0038007E0070007E00E0007E00FFFFFFF8FFFFFFF8FFFFFFF80000FE000000FE0000 00FE000000FE000000FE000000FE000000FE000000FE00007FFFF8007FFFF8007FFFF81D 277EA622>I<0C0003000F803F000FFFFE000FFFFC000FFFF8000FFFF0000FFFE0000FFF C0000FFE00000E0000000E0000000E0000000E0000000E0000000E0000000E7FC0000FFF F8000F80FC000E003E000C003F0000001F8000001FC000001FC000001FE000001FE01800 1FE07C001FE0FE001FE0FE001FE0FE001FE0FE001FC0FC001FC078003F8078003F803C00 7F001F01FE000FFFF80003FFF00000FF80001B277DA622>I<000003800000000007C000 00000007C0000000000FE0000000000FE0000000000FE0000000001FF0000000001FF000 0000003FF8000000003FF8000000003FF80000000073FC0000000073FC00000000F3FE00 000000E1FE00000000E1FE00000001C0FF00000001C0FF00000003C0FF80000003807F80 000007807FC0000007003FC0000007003FC000000E003FE000000E001FE000001E001FF0 00001C000FF000001FFFFFF000003FFFFFF800003FFFFFF80000780007FC0000700003FC 0000700003FC0000E00001FE0000E00001FE0001E00001FF0001C00000FF0001C00000FF 00FFFE001FFFFEFFFE001FFFFEFFFE001FFFFE2F297EA834>65 DI<00003FF001800003 FFFE0380000FFFFF8780003FF007DF8000FF8001FF8001FE00007F8003FC00003F8007F0 00001F800FF000000F801FE0000007801FE0000007803FC0000007803FC0000003807FC0 000003807F80000003807F8000000000FF8000000000FF8000000000FF8000000000FF80 00000000FF8000000000FF8000000000FF8000000000FF8000000000FF80000000007F80 000000007F80000000007FC0000003803FC0000003803FC0000003801FE0000003801FE0 000007000FF00000070007F000000E0003FC00001E0001FE00003C0000FF8000F800003F F007E000000FFFFFC0000003FFFF000000003FF8000029297CA832>I69 D73 D77 D80 D82 D<007F806003FFF0E007FFF9E0 0F807FE01F001FE03E0007E07C0003E07C0001E0FC0001E0FC0001E0FC0000E0FE0000E0 FE0000E0FF000000FFC000007FFE00007FFFE0003FFFFC001FFFFE000FFFFF8007FFFFC0 03FFFFE000FFFFE00007FFF000007FF000000FF8000007F8000003F8600001F8E00001F8 E00001F8E00001F8F00001F0F00001F0F80003F0FC0003E0FF0007C0FFE01F80F3FFFF00 E0FFFE00C01FF0001D297CA826>I<01FF800007FFF0000F81F8001FC07E001FC07E001F C03F000F803F8007003F8000003F8000003F8000003F80000FFF8000FFFF8007FC3F800F E03F803F803F803F003F807F003F80FE003F80FE003F80FE003F80FE003F807E007F807F 00DF803F839FFC0FFF0FFC01FC03FC1E1B7E9A21>97 DI<001FF80000FFFE0003F01F00 07E03F800FC03F801F803F803F801F007F800E007F0000007F000000FF000000FF000000 FF000000FF000000FF000000FF000000FF0000007F0000007F0000007F8000003F8001C0 1F8001C00FC0038007E0070003F01E0000FFFC00001FE0001A1B7E9A1F>I<00003FF800 00003FF80000003FF800000003F800000003F800000003F800000003F800000003F80000 0003F800000003F800000003F800000003F800000003F800000003F800000003F800001F E3F80000FFFBF80003F03FF80007E00FF8000FC007F8001F8003F8003F8003F8007F0003 F8007F0003F8007F0003F800FF0003F800FF0003F800FF0003F800FF0003F800FF0003F8 00FF0003F800FF0003F8007F0003F8007F0003F8007F0003F8003F8003F8001F8003F800 0F8007F80007C00FF80003F03BFF8000FFF3FF80003FC3FF80212A7EA926>I<003FE000 01FFF80003F07E0007C01F000F801F801F800F803F800FC07F000FC07F0007C07F0007E0 FF0007E0FF0007E0FFFFFFE0FFFFFFE0FF000000FF000000FF0000007F0000007F000000 7F0000003F8000E01F8000E00FC001C007E0038003F81F0000FFFE00001FF0001B1B7E9A 20>I<0007F0003FFC00FE3E01F87F03F87F03F07F07F07F07F03E07F00007F00007F000 07F00007F00007F00007F000FFFFC0FFFFC0FFFFC007F00007F00007F00007F00007F000 07F00007F00007F00007F00007F00007F00007F00007F00007F00007F00007F00007F000 07F00007F00007F00007F0007FFF807FFF807FFF80182A7EA915>I<00FF81F003FFE7F8 0FC1FE7C1F80FC7C1F007C383F007E107F007F007F007F007F007F007F007F007F007F00 7F007F003F007E001F007C001F80FC000FC1F8001FFFE00018FF80003800000038000000 3C0000003E0000003FFFF8001FFFFF001FFFFF800FFFFFC007FFFFE01FFFFFF03E0007F0 7C0001F8F80000F8F80000F8F80000F8F80000F87C0001F03C0001E01F0007C00FC01F80 03FFFE00007FF0001E287E9A22>I<07000F801FC03FE03FE03FE01FC00F800700000000 0000000000000000000000FFE0FFE0FFE00FE00FE00FE00FE00FE00FE00FE00FE00FE00F E00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE00FE0FFFEFFFEFFFE0F2B7DAA14> 105 D108 DII<003FE00001FFFC0003F07E000FC01F801F800FC03F800FE03F0007E07F0007F07F00 07F07F0007F0FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF0007F8FF00 07F87F0007F07F0007F03F800FE03F800FE01F800FC00FC01F8007F07F0001FFFC00003F E0001D1B7E9A22>II114 D<03FE300FFFF01E03F03800F0700070F00070F00070F800 70FC0000FFE0007FFE007FFF803FFFE01FFFF007FFF800FFF80003FC0000FC60007CE000 3CF0003CF00038F80038FC0070FF01E0F7FFC0C1FF00161B7E9A1B>I<00700000700000 700000700000F00000F00000F00001F00003F00003F00007F0001FFFF0FFFFF0FFFFF007 F00007F00007F00007F00007F00007F00007F00007F00007F00007F00007F00007F00007 F00007F03807F03807F03807F03807F03807F03803F03803F87001F86000FFC0001F8015 267FA51B>III 120 DI E /Fk 53 125 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0000E01C0 000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C0 0E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F8 151D809C17>I<6060F0F0F8F86868080808080808101010102020404080800D0C7F9C15> 34 D<60F0F8680808081010204080050C7C9C0C>39 D<004000800100020006000C000C 0018001800300030007000600060006000E000E000E000E000E000E000E000E000E000E0 00E000E000600060006000700030003000180018000C000C00060002000100008000400A 2A7D9E10>I<800040002000100018000C000C0006000600030003000380018001800180 01C001C001C001C001C001C001C001C001C001C001C001C0018001800180038003000300 060006000C000C00180010002000400080000A2A7E9E10>I<60F0F07010101010202040 80040C7C830C>44 DI<60F0F06004047C830C>I<60F0F0600000 000000000000000060F0F0701010101020204080041A7C910C>59 D<0FE03038401CE00EF00EF00EF00E000C001C0030006000C00080018001000100010001 00010001000000000000000000000003000780078003000F1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F00000017800000178000001780 000023C0000023C0000023C0000041E0000041E0000041E0000080F0000080F0000180F8 000100780001FFF80003007C0002003C0002003C0006003E0004001E0004001E000C001F 001E001F00FF80FFF01C1D7F9C1F>65 D68 D70 D72 DI76 DII<003F800000E0E0000380 380007001C000E000E001C0007003C00078038000380780003C0780003C0700001C0F000 01E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C07800 03C0780003C0380003803C0007801C0007000E000E0007001C000380380000E0E000003F 80001B1E7E9C20>II<003F800000E0E000 0380380007001C000E000E001C0007003C00078038000380780003C0780003C0700001C0 F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0 780003C0780003C0380003803C0E07801C1107000E208E0007205C0003A0780000F0E020 003FE0200000602000003060000038E000003FC000003FC000001F8000000F001B257E9C 20>I<07E0801C1980300580700380600180E00180E00080E00080E00080F00000F80000 7C00007FC0003FF8001FFE0007FF0000FF80000F800007C00003C00001C08001C08001C0 8001C0C00180C00180E00300D00200CC0C0083F800121E7E9C17>83 D<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F0020800F00 20000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F80 0003FFFC001B1C7F9B1E>II87 D<08081010202040404040808080808080B0B0F8F8787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C0001FC000F1C00381C00701C00601C 00E01C40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E0 0060007004300418080C3007C00E127E9112>I<003F0000070000070000070000070000 070000070000070000070000070000070003E7000C1700180F00300700700700600700E0 0700E00700E00700E00700E00700E00700600700700700300700180F000C370007C7E013 1D7E9C17>I<03E00C301818300C700E6006E006FFFEE000E000E000E000600070023002 18040C1803E00F127F9112>I<00F8018C071E061E0E0C0E000E000E000E000E000E00FF E00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E007FE00F 1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C00381C00381C00181800 1C38000C300013C0001000003000001800001FF8001FFF001FFF803003806001C0C000C0 C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C0018000000000000000000000000000000FC001C 001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80091D7F 9C0C>I<00C001E001E000C000000000000000000000000000000FE000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0 F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180E001C0E001C0E001C0E001C0 E001C0E001C06001807003803003001806000E1C0003F00012127F9115>II<03C1000C3300180B00300F00700700700700E00700E00700E00700E0 0700E00700E00700600700700700300F00180F000C370007C70000070000070000070000 0700000700000700000700003FE0131A7E9116>II<1F9030704030C010 C010E010F8007F803FE00FF000F880388018C018C018E010D0608FC00D127F9110>I<04 000400040004000C000C001C003C00FFE01C001C001C001C001C001C001C001C001C001C 101C101C101C101C100C100E2003C00C1A7F9910>III< FF3FCFE03C0F03801C0701801C0701001C0B01000E0B82000E0B82000E1182000711C400 0711C4000720C40003A0E80003A0E80003C0680001C0700001C070000180300000802000 1B127F911E>I<7F8FF00F03800F030007020003840001C80001D80000F0000070000078 0000F800009C00010E00020E000607000403801E07C0FF0FF81512809116>II124 D E /Fl 13 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D<00038000000380000007C0000007C0000007C000000FE000000FE000001FF000001BF0 00001BF0000031F8000031F8000061FC000060FC0000E0FE0000C07E0000C07E0001803F 0001FFFF0003FFFF8003001F8003001F8006000FC006000FC00E000FE00C0007E0FFC07F FEFFC07FFE1F1C7E9B24>65 D68 D<0FF8001C1E003E0F803E07 803E07C01C07C00007C0007FC007E7C01F07C03C07C07C07C0F807C0F807C0F807C0780B C03E13F80FE1F815127F9117>97 D I<03FC000E0E001C1F003C1F00781F00780E00F80000F80000F80000F80000F80000F800 007800007801803C01801C03000E0E0003F80011127E9115>I<1E003F003F003F003F00 1E00000000000000000000000000FF00FF001F001F001F001F001F001F001F001F001F00 1F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F8 00F8F800F8F800F8F800F8F800F87800F07800F03C01E01E03C00F078001FC0015127F91 18>I114 D<1FD830786018E018E018F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CF C00E127E9113>I<0300030003000300070007000F000F003FFCFFFC1F001F001F001F00 1F001F001F001F001F001F0C1F0C1F0C1F0C0F08079803F00E1A7F9913>II E /Fm 5 85 df<000000007800000000 00000000780000000000000000780000000000000000FC0000000000000000FC00000000 00000000FC0000000000000001FE0000000000000001FE0000000000000003FF00000000 00000003FF0000000000000003FF0000000000000007FF8000000000000007FF80000000 00000007FF800000000000000FFFC00000000000000E7FC00000000000000E7FC0000000 0000001E7FE00000000000001C3FE00000000000001C3FE00000000000003C3FF0000000 000000381FF0000000000000381FF0000000000000781FF8000000000000700FF8000000 000000700FF8000000000000E00FFC000000000000E007FC000000000000E007FC000000 000001C007FE000000000001C003FE000000000001C003FE0000000000038001FF000000 0000038001FF0000000000038001FF0000000000070000FF8000000000070000FF800000 0000070000FF80000000000E00007FC0000000000E00007FC0000000000E00007FC00000 00001C00003FE0000000001C00003FE0000000003C00003FF0000000003800001FF00000 00003800001FF0000000007800001FF8000000007FFFFFFFF8000000007FFFFFFFF80000 0000FFFFFFFFFC00000000E0000007FC00000000E0000007FC00000001E0000007FE0000 0001C0000003FE00000001C0000003FE00000003C0000003FF0000000380000001FF0000 000380000001FF0000000780000001FF8000000700000000FF8000000700000000FF8000 000E00000000FFC000000E000000007FC000000E000000007FC000001E000000003FE000 001E000000003FE000003E000000003FE000007F000000003FF00001FF800000003FF800 07FFE0000001FFFC00FFFFFC00003FFFFFFCFFFFFC00003FFFFFFCFFFFFC00003FFFFFFC 46497DC84D>65 D68 D70 D82 D<3FFFFFFFFFFFFFFF003FFFFFFFFFFFFFFF003FFFFF FFFFFFFFFF003FF8000FFE0007FF003FC00007FC0000FF003F000007FC00003F007E0000 07FC00001F807C000007FC00000F807C000007FC00000F8078000007FC00000780780000 07FC0000078078000007FC0000078070000007FC0000038070000007FC00000380700000 07FC0000038070000007FC0000038070000007FC00000380F0000007FC000003C0E00000 07FC000001C0E0000007FC000001C0E0000007FC000001C0E0000007FC000001C0E00000 07FC000001C000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC0000000000000007FC00000000000000 07FC0000000000000007FC0000000000000007FC000000000000000FFE00000000000000 3FFF800000000003FFFFFFFFF800000003FFFFFFFFF800000003FFFFFFFFF8000042467C C54B>84 D E /Fn 2 104 df<000F0038007000E001C001C001C001C001C001C001C001 C001C001C001C001C001C001C001C001C001C0038007001E00F0001E000700038001C001 C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C000E0007000 38000F10317CA419>102 DI E /Fo 36 122 df<70F8FCFC7404040404080810102040060F7C84 0E>44 D<70F8F8F87005057C840E>46 D<008003800F80F3800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 8003800380038007C0FFFE0F217CA018>49 D<03F0000C1C001007002007804003C04003 C08003E0F003E0F801E0F801E0F801E02003E00003E00003C00003C0000780000700000E 00001C0000180000300000600000C0000180000100000200200400200800201800603000 403FFFC07FFFC0FFFFC013217EA018>I<1000801E07001FFF001FFE001FF80013E00010 000010000010000010000010000010000010F800130E001407001803801003800001C000 01C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C04003C040038020 07001006000C1C0003F00013227EA018>53 D<007E0001C1000300800601C00E03C01C03 C0180180380000380000780000700000700000F0F800F30C00F40600F40300F80380F801 C0F001C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C03801C01803 801C03000C0600070C0001F00013227EA018>I<01F000060C000C060018070038038070 0380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003E038 03E01805E00C05E00619E003E1E00001C00001C00001C000038000038030030078070078 0600700C002018001030000FC00013227EA018>57 D<000FE00000701C00008002000300 01800400004008000020080000201007C01020183008203008084060040440C0078441C0 038481C003828380038283800382838003828380038283800382838003828380038281C0 038241C0038240C007824060078420300B84201831881007C0F008000000080000000400 00000300000E00800078007007C0000FFC001F237DA226>64 D<00018000000180000001 80000003C0000003C0000003C0000005E0000005E000000DF0000008F0000008F0000010 F800001078000010780000203C0000203C0000203C0000401E0000401E0000401E000080 0F0000800F0000FFFF000100078001000780030007C0020003C0020003C0040003E00400 01E0040001E00C0000F00C0000F03E0001F8FF800FFF20237EA225>I<0007E010003818 3000E0063001C00170038000F0070000F00E0000701E0000701C0000303C0000303C0000 307C0000107800001078000010F8000000F8000000F8000000F8000000F8000000F80000 00F8000000F800000078000000780000107C0000103C0000103C0000101C0000201E0000 200E000040070000400380008001C0010000E0020000381C000007E0001C247DA223>67 DI< FFFFFFC00F8007C0078001C0078000C00780004007800040078000600780002007800020 0780002007802020078020000780200007802000078060000780E00007FFE0000780E000 078060000780200007802000078020000780200007800000078000000780000007800000 07800000078000000780000007800000078000000FC00000FFFE00001B227EA120>70 D76 DII<03F020 0C0C601802603001E07000E0600060E00060E00060E00020E00020E00020F00000F00000 7800007F00003FF0001FFE000FFF0003FF80003FC00007E00001E00000F00000F0000070 800070800070800070800070C00060C00060E000C0F000C0C80180C6070081FC0014247D A21B>83 D<0FE0001838003C0C003C0E0018070000070000070000070000FF0007C7001E 07003C0700780700700700F00708F00708F00708F00F087817083C23900FC1E015157E94 18>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E1F000E61C00E80600F00300E00380E003C0E001C0E001E0E00 1E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C80600C41C0083F 0017237FA21B>I<01FE000703000C07801C0780380300780000700000F00000F00000F0 0000F00000F00000F00000F000007000007800403800401C00800C010007060001F80012 157E9416>I<0000E0000FE00001E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E001F8E00704E00C02E01C01E03800E07800E07000E0F000E0 F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800E01801E00C02E0070CF0 01F0FE17237EA21B>I<01FC000707000C03801C01C03801C07801E07000E0F000E0FFFF E0F00000F00000F00000F00000F000007000007800203800201C00400E008007030000FC 0013157F9416>I<00007001F198071E180E0E181C07001C07003C07803C07803C07803C 07801C07001C07000E0E000F1C0019F0001000001000001800001800001FFE000FFFC00F FFE03800F0600030400018C00018C00018C000186000306000303800E00E038003FE0015 217F9518>103 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E1F800E60C00E80E00F00700F00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E 0070FFE7FF18237FA21B>I<1C001E003E001E001C000000000000000000000000000000 00000E00FE001E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E00FFC00A227FA10E>I<0E0000FE00001E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E0200 0E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C0 0E01E00E00F00E00F8FFE3FE17237FA21A>107 D<0E00FE001E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00FE60E18380 1E807201C00F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60C0 1E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01 801800C03800E0700070700070F00078F00078F00078F00078F00078F00078F000787000 707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F 00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E 003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E 00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E00 0E000E000E000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>II121 D E /Fp 1 4 df<020002000200C218F2783AE00F800F803AE0F278C218020002000200 0D0E7E8E12>3 D E /Fq 17 120 df<00FFFFFF80000F000780000F000180000F000180 001E000180001E000180001E000100001E000100003C000100003C000100003C01010000 3C01000000780200000078020000007806000000780E000000FFFC000000F00C000000F0 0C000000F00C000001E008000001E008000001E008040001E000080003C000080003C000 080003C000100003C000100007800020000780006000078000C000078001C0000F8007C0 00FFFFFF800021227DA121>69 D<00007F00800003808100000E00630000380027000070 001F0000E0000E0001C0000E000380000E000700000E000F000004000E000004001E0000 04003C000004003C00000800780000000078000000007800000000F000000000F0000000 00F000000000F000000000F0003FFC00E00001E000E00001E000E00001E000E00003C000 E00003C000F00003C000700003C0007000078000380007800018000F80001C0013800006 002300000381C1000000FE000000212479A226>71 D<00FFFC00000F8000000F0000000F 0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C00000078 000000780000007800000078000000F0000000F0000000F0000000F0000001E0000001E0 000001E0002001E0002003C0004003C0004003C0008003C0008007800180078001000780 030007800F000F803E00FFFFFE001B227DA11F>76 D87 D<00F8C00185C00705C00E0380 0E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E10F01C20 701C20703C20305C40308C400F078014157B9419>97 D<001F180030B800E0B801C07001 C0700380700780700700E00F00E00F00E00F00E01E01C01E01C01E01C01E01C01E03800E 03800E0780060B8006170001E700000700000700000E00000E00000E00701C00F01800F0 300060E0003F8000151F7E9416>103 D<00C001E001C001C00000000000000000000000 00000000001C002300430043008700870087000E000E001C001C001C0038003800384070 8070807080710032001C000B217BA00F>105 D<00F0000FE00000E00000E00000E00001 C00001C00001C00001C0000380000380000380000380000700000701E0070210070C700E 10F00E10F00E20600E40001D80001E00001FC0001C7000383800383800381C00381C2070 3840703840703840701880E01880600F0014237DA216>107 D<01E01FC001C001C001C0 038003800380038007000700070007000E000E000E000E001C001C001C001C0038003800 380038007000700070007100E200E200E200E200640038000B237CA20C>I<1C0F80F800 2610C10C00476066060087807807008780780700870070070087007007000E00E00E000E 00E00E000E00E00E000E00E00E001C01C01C001C01C01C001C01C01C001C01C038203803 80384038038070403803807080380380308070070031003003001E0023157B9428>I<1C 0F002631C04740C08780E08780E08700E08700E00E01C00E01C00E01C00E01C01C03801C 03801C03801C0704380708380E08380E103806107006203003C016157B941B>I<007E00 01C3000381800701C00E01C01C01E03C01E03801E07801E07801E07801E0F003C0F003C0 F00380F00780700700700E00700C0030180018700007C00013157B9419>I<01C1F00262 1804741C08780C08700E08700E08701E00E01E00E01E00E01E00E01E01C03C01C03C01C0 3C01C07803807003807003C0E003C1C0072380071E000700000700000E00000E00000E00 000E00001C00001C00001C0000FFC000171F7F9419>I<1C1F002620804741C08783C087 03C08701808700000E00000E00000E00000E00001C00001C00001C00001C000038000038 000038000038000070000030000012157B9415>114 D<00FC000183000200800401800C 03800C03000C00000F00000FF00007FC0003FE00003E00000F00000700700700F00600F0 0600E004004008002030001FC00011157D9414>I<1E00602300E04380E04381C08381C0 8701C08701C00703800E03800E03800E03801C07001C07001C07001C07081C0E10180E10 1C0E101C1E200C262007C3C015157B941A>117 D<1E0060E02300E1F04380E1F04381C0 F08381C0708701C0308701C030070380200E0380200E0380200E0380201C0700401C0700 401C0700401C0700801C0700801C0701001C0F01000C0F020006138C0003E0F0001C157B 9420>119 D E /Fr 17 122 df68 D73 D77 D80 D<001FF0018000FFFF038003FFFFC78007F00FFF800F 8001FF801F00007F803F00001F803E00000F807E00000F807E00000780FE00000780FE00 000780FE00000380FF00000380FF00000380FF80000000FFE00000007FFC0000007FFFE0 00007FFFFE00003FFFFFC0001FFFFFF0001FFFFFF8000FFFFFFC0003FFFFFE0001FFFFFF 00007FFFFF80001FFFFF800000FFFFC0000007FFC0000000FFE00000003FE00000003FE0 0000001FE06000001FE0E000000FE0E000000FE0E000000FE0E000000FC0F000000FC0F0 00000FC0F800001F80FC00001F80FF00003F00FFC0007E00FFFC01FC00F1FFFFF800E03F FFE000C007FF000023317BB02E>83 D<007FF8000003FFFF000007FFFFC0000FE01FE000 1FF007F0001FF003F8001FF003FC001FF001FE000FE001FE0007C001FE00010001FE0000 0001FE00000001FE000001FFFE00003FFFFE0001FFF1FE0007FE01FE000FF001FE001FC0 01FE003F8001FE007F8001FE00FF0001FE00FF0001FE00FF0001FE00FF0001FE00FF0003 FE007F8003FE007FC00EFE003FF03CFF000FFFF87FF807FFF03FF800FF800FF825207E9F 28>97 D<0007FF00007FFFE000FFFFF003FC03F807F007FC0FE007FC1FE007FC3FC007FC 3FC003F87FC001F07F8000407F800000FF800000FF800000FF800000FF800000FF800000 FF800000FF800000FF8000007F8000007FC000007FC000003FC0000E3FE0000E1FE0001C 0FF0001C07F8007803FF01F000FFFFE0007FFF800007FC001F207D9F25>99 D<0007FC0000003FFF800000FFFFE00003FC07F00007F801F8000FE000FC001FE0007E00 3FC0007E003FC0003F007FC0003F007F80003F007F80003F80FF80003F80FF80003F80FF FFFFFF80FFFFFFFF80FFFFFFFF80FF80000000FF80000000FF800000007F800000007F80 0000003FC00000003FC00003801FC00003801FE00007800FF0000F0007F8001E0003FE00 FC0000FFFFF800003FFFE0000003FF000021207E9F26>101 D<001FF007E000FFFE3FF0 01FFFF7FF807F83FF1F80FE00FE1F80FE00FE0F01FC007F0601FC007F0003FC007F8003F C007F8003FC007F8003FC007F8003FC007F8001FC007F0001FC007F0000FE00FE0000FE0 0FE00007F83FC00007FFFF000006FFFE00000E1FF000000E000000001E000000001E0000 00001F000000001F800000001FFFFFC0000FFFFFF8000FFFFFFE0007FFFFFF0003FFFFFF 8007FFFFFFC01FFFFFFFE03F00007FE07E00000FF0FC000007F0FC000003F0FC000003F0 FC000003F0FC000003F07E000007E03F00000FC01FC0003F800FF801FF0007FFFFFE0000 FFFFF000001FFF8000252F7E9F29>103 D<01C00007F0000FF8000FF8001FFC001FFC00 1FFC000FF8000FF80007F00001C000000000000000000000000000000000000000000000 00000001F800FFF800FFF800FFF8000FF80007F80007F80007F80007F80007F80007F800 07F80007F80007F80007F80007F80007F80007F80007F80007F80007F80007F80007F800 07F80007F80007F80007F80007F80007F800FFFF80FFFF80FFFF8011337DB217>105 D<03F007F8000FF000FFF03FFF007FFE00FFF07FFF80FFFF00FFF0F03FC1E07F800FF1C0 1FE3803FC007F3000FE6001FC007F6000FFC001FE007FE000FFC001FE007FC000FF8001F E007FC000FF8001FE007F8000FF0001FE007F8000FF0001FE007F8000FF0001FE007F800 0FF0001FE007F8000FF0001FE007F8000FF0001FE007F8000FF0001FE007F8000FF0001F E007F8000FF0001FE007F8000FF0001FE007F8000FF0001FE007F8000FF0001FE007F800 0FF0001FE007F8000FF0001FE007F8000FF0001FE007F8000FF0001FE007F8000FF0001F E007F8000FF0001FE007F8000FF0001FE0FFFFC1FFFF83FFFFFFFFC1FFFF83FFFFFFFFC1 FFFF83FFFF40207D9F45>109 D<03F007F80000FFF03FFF0000FFF07FFF8000FFF0F03F C0000FF1C01FE00007F3000FE00007F6000FF00007FE000FF00007FC000FF00007FC000F F00007F8000FF00007F8000FF00007F8000FF00007F8000FF00007F8000FF00007F8000F F00007F8000FF00007F8000FF00007F8000FF00007F8000FF00007F8000FF00007F8000F F00007F8000FF00007F8000FF00007F8000FF00007F8000FF00007F8000FF00007F8000F F00007F8000FF000FFFFC1FFFF80FFFFC1FFFF80FFFFC1FFFF8029207D9F2E>I<0007FE 0000003FFFC00000FFFFF00003FC03FC0007F000FE000FE0007F001FC0003F803FC0003F C03FC0003FC07F80001FE07F80001FE07F80001FE0FF80001FF0FF80001FF0FF80001FF0 FF80001FF0FF80001FF0FF80001FF0FF80001FF0FF80001FF07F80001FE07F80001FE07F 80001FE03FC0003FC03FC0003FC01FE0007F800FE0007F0007F801FE0003FE07FC0001FF FFF800003FFFC0000007FE000024207E9F29>I<03F03F00FFF07FC0FFF1FFE0FFF3C7F0 0FF38FF807F70FF807F60FF807FE0FF807FC07F007FC03E007FC008007F8000007F80000 07F8000007F8000007F8000007F8000007F8000007F8000007F8000007F8000007F80000 07F8000007F8000007F8000007F8000007F8000007F8000007F80000FFFFE000FFFFE000 FFFFE0001D207E9F22>114 D<00FF870007FFEF001FFFFF003F007F003C001F0078000F 00F8000700F8000700F8000700FC000700FF000000FFF800007FFFC0003FFFF0003FFFFC 000FFFFE0007FFFF0001FFFF80001FFF800000FFC000001FC060000FC0E00007C0E00007 C0F00007C0F8000780F8000F80FE000F00FF803E00FFFFFC00F3FFF800C07FC0001A207D 9F21>I<00380000380000380000380000380000780000780000780000F80000F80001F8 0003F80007F8001FF800FFFFFEFFFFFEFFFFFE07F80007F80007F80007F80007F80007F8 0007F80007F80007F80007F80007F80007F80007F80007F80007F80007F80007F80707F8 0707F80707F80707F80707F80707F80703F80E03FC0E01FE1C00FFF8007FF0000FE0182E 7EAD20>I121 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 74 189 a Fr(Dynamic)26 b(Pro)r(cess)g(Managemen)n(t)h(in)g(an)f (MPI)i(Setting)815 346 y Fq(Wil)r(liam)18 b(Gr)n(opp)843 404 y(Ewing)g(Lusk)1106 386 y Fp(\003)493 462 y Fo(Mathematics)d(and)i (Computer)e(Science)g(Division)657 520 y(Argonne)h(National)g(Lab)q (oratory)696 578 y Fn(f)p Fo(gropp,lusk)p Fn(g)p Fo(@mcs.anl.go)o(v)782 710 y(F)l(ebruary)g(26,)h(1995)787 992 y Fm(DRAFT)883 1122 y Fl(Abstract)250 1215 y Fk(W)m(e)d(prop)q(ose)i(extensions)f(to)g (the)g(Message-P)o(assing)h(In)o(terface)f(\(MPI\))g(Standard)g(that)g (pro-)188 1265 y(vide)9 b(for)g(dynamic)f(pro)q(cess)j(managemen)o(t,)c (including)i(spa)o(wning)g(of)f(new)i(pro)q(cesses)i(b)o(y)e(a)f (running)188 1315 y(application.)16 b(Suc)o(h)d(extensions)g(are)g (needed)g(if)f(more)f(of)h(the)g(run)o(time)g(en)o(vironmen)o(t)f(for)h (parallel)188 1364 y(programs)h(is)i(to)g(b)q(e)g(accessible)h(to)f (MPI)g(programs)e(or)i(to)g(b)q(e)g(themselv)o(es)g(written)g(using)g (MPI.)188 1414 y(The)i(extensions)h(prop)q(osed)f(here)h(are)f(motiv)n (ated)e(b)o(y)h(real)h(applications)e(and)i(\014t)g(cleanly)f(with)188 1464 y(existing)d(concepts)i(of)d(MPI.)h(No)g(c)o(hanges)h(to)f(the)h (existing)f(MPI)g(Standard)h(are)f(prop)q(osed,)h(th)o(us)188 1514 y(all)e(presen)o(t)k(MPI)e(programs)f(will)f(run)i(unc)o(hanged.) 74 1687 y Fj(1)69 b(In)n(tro)r(duction)74 1819 y Fi(During)15 b(1993)e(and)i(1994)e(a)h(group)g(comp)q(osed)h(of)f(parallel)i (computer)e(v)o(endors,)g(library)h(writers,)f(and)74 1875 y(application)k(scien)o(tists)f(created)f(a)g(standard)g(message)g (passing)g(library)i(in)o(terface)e(sp)q(eci\014cation)i([1,)74 1931 y(2,)h(6)o(].)32 b(This)20 b(group,)g(whic)o(h)g(called)g(itself)h (the)e(MPI)g(F)l(orum,)h(c)o(hose)f(to)f(prop)q(ose)i(a)f(standard)g (only)74 1988 y(for)d(the)h(message-passing)g(library)l(,)h(attempting) e(to)g(unify)i(and)f(subsume)g(the)g(plethora)g(of)f(existing)74 2044 y(libraries.)34 b(They)19 b(delib)q(erately)j(and)d(explicitly)j (did)e(not)f(prop)q(ose)h(a)f(standard)f(for)h(ho)o(w)g(pro)q(cesses)74 2101 y(w)o(ould)e(b)q(e)g(created)g(in)g(the)g(\014rst)f(place,)i(only) f(for)f(ho)o(w)g(they)h(w)o(ould)g(comm)o(unicate)g(once)g(they)f(w)o (ere)74 2157 y(created.)145 2244 y(MPI)k(users)g(ha)o(v)o(e)g(ask)o(ed) f(that)h(the)g(F)l(orum)g(reconsider)h(this)f(issue)h(for)f(sev)o(eral) g(reasons.)34 b(The)74 2300 y(\014rst)20 b(is)h(that)f(w)o(orkstation)f (net)o(w)o(ork)h(users)g(migrating)h(from)f(PVM)g(to)g(MPI)g(are)g (accustomed)h(to)74 2357 y(using)14 b(PVM's)e(capabilities)j([4)o(])d (for)h(pro)q(cess)g(managemen)o(t.)18 b(\(On)13 b(the)g(other)f(hand,)h (dynamic)h(pro)q(cess)74 2413 y(creation)20 b(is)h(often)e(di\016cult)i (or)f(imp)q(ossible)i(on)e(MPP's,)f(limiting)j(the)e(p)q(ortabilit)o(y) h(of)e(suc)o(h)h(PVM)74 2470 y(programs.\))40 b(A)23 b(second)g(reason)f(is)h(that)e(imp)q(ortan)o(t)h(classes)h(of)f (message-passing)h(applications,)74 2526 y(suc)o(h)d(as)f(clien)o (t-serv)o(er)i(systems)e(and)h(task-farming)f(jobs,)h(require)g (dynamic)h(pro)q(cess)e(con)o(trol.)33 b(A)p 74 2566 720 2 v 125 2592 a Fh(\003)143 2608 y Fg(This)17 b(w)o(ork)f(w)o(as)g (supp)q(orted)i(b)o(y)f(the)f(Applied)i(Mathematical)g(Sciences)g (subprogram)g(of)e(the)g(O\016ce)g(of)g(Energy)74 2654 y(Researc)o(h,)e(U.S.)e(Departmen)o(t)i(of)f(Energy)m(,)g(under)h(con)o (tract)f(W-31-109-Eng-38.)963 2790 y Fi(1)p eop %%Page: 2 2 2 1 bop 74 7 a Fi(third)16 b(is)h(that)e(with)h(suc)o(h)g(extensions)g (it)g(w)o(ould)g(b)q(e)h(p)q(ossible)g(to)e(write)h(ma)s(jor)e(parts)h (of)h(the)g(parallel)74 64 y(programming)f(en)o(vironmen)o(t)g(in)h (MPI)f(itself.)145 150 y(In)i(this)f(pap)q(er)h(w)o(e)f(describ)q(e)i (an)e(arc)o(hitecture)h(of)f(the)g(system)g(run)o(time)h(en)o(vironmen) o(t)f(of)g(a)g(par-)74 207 y(allel)h(program)c(that)h(separates)g(the)h (functions)g(of)f(job)g(sc)o(heduler,)i(pro)q(cess)f(manager,)f(and)g (message-)74 263 y(passing)k(system.)26 b(W)l(e)17 b(sho)o(w)g(ho)o(w)g (the)g(existing)i(MPI)e(sp)q(eci\014cation,)j(whic)o(h)e(can)f(serv)o (e)g(handily)j(as)74 319 y(a)15 b(complete)h(message-passing)g(system,) f(can)g(b)q(e)h(extended)h(in)f(a)f(natural)g(w)o(a)o(y)g(to)f(include) k(an)e(appli-)74 376 y(cation)g(in)o(terface)g(to)g(the)g(system's)f (job)g(sc)o(heduler)i(and)g(pro)q(cess)f(manager,)f(or)g(ev)o(en)h(to)f (write)h(those)74 432 y(functions)j(if)g(they)g(are)f(not)g(already)g (pro)o(vided.)30 b(\(A)18 b(t)o(ypical)i(di\013erence)f(b)q(et)o(w)o (een)g(an)f(MPP)g(and)g(a)74 489 y(w)o(orkstation)12 b(net)o(w)o(ork)g(is)i(that)e(the)h(MPP)g(comes)g(with)h(a)e(built-in)k (sc)o(heduler)e(and)g(pro)q(cess)f(manager,)74 545 y(whereas)19 b(the)f(w)o(orkstation)g(net)o(w)o(ork)f(do)q(es)i(not.)30 b(W)l(e)18 b(will)i(mak)o(e)e(this)h(distinction)i(clearer)e(in)g(Sec-) 74 602 y(tion)f(2.\))27 b(The)18 b(extensions)h(are)e(straigh)o(tforw)o (ard)f(and)i(\014t)g(will)h(in)o(to)f(the)g(MPI)g(framew)o(ork,)e (reusing)74 658 y(sev)o(eral)g(of)e(its)i(existing)g(concepts)f(and)h (functions.)145 745 y(The)j(pap)q(er)h(is)g(organized)g(as)f(follo)o (ws.)33 b(In)20 b(Section)h(2)e(w)o(e)g(describ)q(e)i(in)f(detail)h (what)e(w)o(e)g(mean)74 801 y(b)o(y)i(eac)o(h)h(of)f(the)g(comp)q(onen) o(ts)g(of)g(the)h(parallel)g(run)o(time)g(en)o(vironmen)o(t|job)g(sc)o (heduler,)i(pro)q(cess)74 858 y(manager,)12 b(and)g(message-passing)h (system|and)f(giv)o(e)h(sev)o(eral)f(examples)h(of)f(complete)h (systems)f(with)74 914 y(v)o(ery)j(di\013eren)o(t)g(comp)q(onen)o(ts.) 20 b(Section)c(3)f(con)o(tains)g(the)g(basic)h(principles)h(b)q(ehind)g (the)e(design)h(of)f(the)74 970 y(extensions)f(and)g(the)f (de\014nitions)i(of)e(the)h(extensions)g(themselv)o(es.)20 b(It)13 b(concludes)i(with)f(a)f(comparison)74 1027 y(of)e(the)h(MPI)g (approac)o(h)f(de\014ned)i(here)f(with)g(the)g(PVM)f(approac)o(h)h(to)f (dynamic)h(pro)q(cess)g(managemen)o(t,)74 1083 y(whic)o(h)20 b(is)g(quite)h(di\013eren)o(t.)33 b(Section)20 b(4)f(con)o(tains)h (descriptions)h(of)e(sev)o(eral)g(complete)i(applications)74 1140 y(that)f(mak)o(e)h(use)g(of)f(the)h(extensions)h(describ)q(ed)g (here.)38 b(In)21 b(the)g(conclusion)h(w)o(e)f(address)g(issues)h(of)74 1196 y(implemen)o(tation)17 b(status.)145 1283 y(W)l(e)j(do)h(not)f (pro)o(vide)i(C)e(or)g(F)l(ortran)g(bindings)i(in)g(this)f(do)q(cumen)o (t,)h(instead)f(using)h(the)e(same)74 1339 y(de\014nition)d(st)o(yle)e (used)h(in)g(the)g(MPI)f(standard)g(itself.)74 1512 y Fj(2)69 b(Run)n(time)21 b(En)n(vironmen)n(ts)h(of)i(P)n(arallel)d (Programs)74 1644 y Fi(A)11 b(parallel)h(program)e(do)q(es)h(not)f (execute)h(in)h(isolation;)g(it)f(m)o(ust)f(ha)o(v)o(e)h(computing)g (and)g(other)f(resources)74 1700 y(allo)q(cated)16 b(to)f(it,)g(its)g (pro)q(cesses)g(m)o(ust)g(b)q(e)h(started)e(and)h(managed,)g(and)g (\(presumably\))g(its)h(pro)q(cesses)74 1757 y(m)o(ust)g(comm)o (unicate.)25 b(MPI)17 b(standardizes)g(the)g(comm)o(unication)g(asp)q (ect,)g(but)g(sa)o(ys)f(nothing)h(ab)q(out)74 1813 y(the)e(other)g(asp) q(ects)h(of)e(the)i(execution)g(en)o(vironmen)o(t.)145 1900 y(One)e(reason)f(that)g(the)h(MPI)f(forum)g(c)o(hose)h(to)f (\(temp)q(orarily\))g(ignore)h(these)g(asp)q(ects)f(is)h(that)f(they)74 1956 y(v)m(ary)j(so)g(greatly)g(in)i(curren)o(t)e(parallel)i(systems.)k (In)17 b(order)f(to)g(motiv)m(ate)g(the)h(structure)f(of)g(the)g(MPI)74 2013 y(extensions)c(that)f(w)o(e)g(are)g(going)h(to)f(prop)q(ose)g(in)i (Section)f(3,)f(w)o(e)h(describ)q(e)h(here)f(the)f(ma)s(jor)f(comp)q (onen)o(ts)74 2069 y(of)15 b(a)h(parallel)h(run)o(time)f(en)o(vironmen) o(t)g(and)g(giv)o(e)g(a)f(n)o(um)o(b)q(er)h(of)g(examples)g(of)g(v)m (arious)g(instan)o(tiations)74 2126 y(of)f(this)h(structure.)74 2277 y Ff(2.1)56 b(Comp)r(onen)n(ts)74 2393 y Fi(One)14 b(w)o(a)o(y)e(to)h(decomp)q(ose)h(the)f(complex)i(run)o(time)e(en)o (vironmen)o(t)h(at)e(a)h(high)i(lev)o(el)f(on)f(to)q(da)o(y's)g (parallel)74 2450 y(systems)h(is)h(to)f(separate)g(out)g(the)g (functions)h(of)f Fe(job)i(sche)n(duler)p Fi(,)d Fe(pr)n(o)n(c)n(ess)i (manager)p Fi(,)f Fe(message-p)n(assing)74 2506 y(libr)n(ary)p Fi(,)h(and)g Fe(se)n(curity)p Fi(.)963 2790 y(2)p eop %%Page: 3 3 3 2 bop 74 7 a Fd(Job)24 b(Sc)o(heduler)46 b Fi(By)21 b(the)g Fe(job)g(sche)n(duler)26 b Fi(w)o(e)20 b(mean)h(that)f(part)h (of)f(the)h(system)g(that)f(manages)74 64 y(resources.)g(It)14 b(decides)h(whic)o(h)g(pro)q(cessors)f(will)h(b)q(e)g(allo)q(cated)g (to)e(the)h(parallel)h(job)f(when)h(it)f(runs)g(and)74 120 y(when)h(it)f(will)i(run.)k(In)15 b(some)e(en)o(vironmen)o(ts)i(it) f(is)h(represen)o(ted)f(b)o(y)g(a)g(comp)q(onen)o(t)h(of)e(a)h (sophisticated)74 177 y(batc)o(h)j(queueing)h(system;)f(in)g(others)f (it)h(is)h(represen)o(ted)f(b)o(y)g(the)f(user)h(himself,)h(who)f(can)g (start)e(jobs)74 233 y(whenev)o(er)h(and)f(wherev)o(er)g(he)h(lik)o(es) g(on)f(a)g(net)o(w)o(ork.)74 383 y Fd(Pro)q(cess)23 b(Manager)46 b Fi(Once)22 b(pro)q(cessors)e(ha)o(v)o(e)h(b)q(een)h(allo)q(cated)f (to)g(a)f(program,)h(user)g(pro)q(cesses)74 440 y(m)o(ust)16 b(b)q(e)g(started)g(on)g(those)f(pro)q(cessors,)h(and)g(managed)g (after)f(startup.)22 b(By)16 b(\\managed")f(w)o(e)h(mean)74 496 y(that)d(signals)h(m)o(ust)f(b)q(e)i(deliv)o(erable,)h(that)c Fc(stdin)p Fi(,)h Fc(stdout)p Fi(,)g(and)h Fc(stderr)f Fi(m)o(ust)g(b)q(e)h(handled)h(in)f(some)74 552 y(reasonable)20 b(w)o(a)o(y)l(,)e(and)i(that)e(orderly)i(termination)f(can)g(b)q(e)h (guaran)o(teed.)31 b(A)19 b(minimal)i(example)f(is)74 609 y Fc(rsh)p Fi(,)12 b(whic)o(h)h(starts)e(pro)q(cesses)i(and)g (reroutes)f Fc(stdin)p Fi(,)f Fc(stdout)p Fi(,)h(and)h Fc(stderr)e Fi(bac)o(k)h(to)g(the)g(originating)74 665 y(pro)q(cess.)24 b(A)16 b(more)g(complex)h(example)g(is)g(giv)o(en)g(b) o(y)f Fc(poe)g Fi(on)h(the)f(IBM)h(SP2)f(or)g Fc(prun)f Fi(on)i(the)f(Meik)o(o)74 722 y(CS-2,)g(whic)o(h)h(start)e(pro)q (cesses)h(on)h(pro)q(cessors)e(giv)o(en)i(to)e(them)i(b)o(y)f(the)g (job)g(sc)o(heduler)h(and)g(manage)74 778 y(them)e(un)o(til)i(they)e (are)g(\014nished.)145 865 y(In)g(some)f(cases)g(the)h(situation)g(is)f (m)o(uddied)i(b)o(y)f(com)o(bining)g(the)g(functions)g(of)f(job)g(sc)o (heduler)i(and)74 921 y(pro)q(cess)d(manager)e(in)i(one)g(piece)h(of)e (soft)o(w)o(are.)17 b(Examples)c(of)e(this)i(approac)o(h)f(are)g(the)h (batc)o(h)f(queueing)74 978 y(systems)21 b(suc)o(h)h(as)f(Condor,)i (DQS,)e(and)h(LoadLev)o(eler.)40 b(Nonetheless,)24 b(it)e(will)h(b)q(e) g(con)o(v)o(enien)o(t)f(to)74 1034 y(consider)d(them)g(separately)l(,)g (since)g(although)g(they)f(m)o(ust)g(comm)o(unicate)h(with)f(one)h (another,)f(they)74 1091 y(are)d(separate)g(functions)h(that)e(can)h(b) q(e)h(indep)q(enden)o(tly)j(mo)q(di\014ed.)74 1241 y Fd(Message-P)o(assing)24 b(Library)45 b Fi(By)21 b(the)h Fe(message-p)n(assing)e(libr)n(ary)25 b Fi(w)o(e)20 b(mean)h(the)h (library)g(used)74 1297 y(b)o(y)16 b(the)h(application)g(program)f(for) f(its)i(in)o(terpro)q(cess)f(comm)o(unication.)24 b(Programs)15 b(con)o(taining)i(only)74 1353 y(calls)i(to)e(a)g(message-passing)h (library)h(can)e(b)q(e)i(extremely)f(p)q(ortable,)g(since)h(they)f (\014t)g(cleanly)h(in)o(to)e(a)74 1410 y(v)m(ariet)o(y)h(of)e(job)h(sc) o(heduler{pro)q(cess)i(manager)e(en)o(vironmen)o(ts.)26 b(MPI)17 b(de\014nes)h(a)f(standard)g(in)o(terface)74 1466 y(for)e(message-passing)g(libraries.)74 1616 y Fd(Securit)o(y)45 b Fi(An)21 b(imp)q(ortan)o(t)e(function)i(of)f(the)g(run)o(time)g(en)o (vironmen)o(t)h(is)f Fe(se)n(curity)p Fi(.)34 b(The)21 b(securit)o(y)74 1673 y(system)g(ensures)h(that)f(the)g(job)g(sc)o (heduler)i(do)q(es)f(not)f(allo)q(cate)h(resources)f(to)g(users)g(or)g (programs)74 1729 y(that)e(should)j(not)d(ha)o(v)o(e)h(them,)h(that)e (the)h(pro)q(cess)g(manager)g(do)q(es)g(indeed)i(con)o(trol)e(the)g (pro)q(cesses)74 1786 y(that)c(it)h(starts,)e(and)i(that)e(the)i (message-passing)g(library)g(deliv)o(ers)h(messages)e(only)h(to)f (their)h(prop)q(er)74 1842 y(destinations.)k(W)l(e)15 b(will)i(prop)q(ose)e(some)g(minimal)i(features)e(designed)h(to)f (enhance)h(securit)o(y)l(.)145 1929 y(These)g(comp)q(onen)o(ts)g(need)h (to)e(comm)o(unicate)i(among)e(themselv)o(es)h(and)h(with)f(the)g (user,)g(but)g(the)74 1985 y(timing)g(and)f(the)f(paths)h(of)f(suc)o(h) h(comm)o(unication)h(v)m(ary)f(from)f(one)h(en)o(vironmen)o(t)g(to)f (another.)19 b(Some)74 2042 y(of)c(the)g(paths)g(are)g(illustrated)i (in)f(Figure)f(1.)145 2128 y(F)l(or)e(example,)i(the)f(job)g(sc)o (heduler)i(and)e(the)h(pro)q(cess)f(manager)g(m)o(ust)f(comm)o(unicate) i(so)f(that)f(the)74 2185 y(pro)q(cess)20 b(manager)e(can)i(kno)o(w)e (where)i(to)e(start)g(the)i(user)f(pro)q(cesses.)32 b(The)20 b(pro)q(cess)f(manager)g(and)74 2241 y(the)d(message-passing)f(library) h(comm)o(unicate)g(in)g(order)f(for)g(the)g(message-passing)h(library)g (to)f(kno)o(w)74 2297 y(where)k(the)g(pro)q(cesses)g(are)g(and)g(ho)o (w)f(to)h(con)o(tact)f(them.)30 b(The)20 b(user)f(ma)o(y)f(in)o(teract) g(only)i(with)f(the)74 2354 y(job)d(sc)o(heduler)i(\(as)d(in)i(the)g (case)f(of)g(LoadLev)o(eler,)h(an)f(IBM)g(sc)o(heduler\),)i(directly)f (with)g(the)f(pro)q(cess)74 2410 y(manager)f(\()p Fc(poe)p Fi(,)f Fc(prun)p Fi(\),)g(or)h(only)h(with)g(the)f(application)i (program)d(\()p Fc(p4)p Fi(\).)20 b(Finally)l(,)d(it)e(ma)o(y)g(b)q(e)h (useful)74 2467 y(for)f(the)g(application)i(program)c(to)i(dynamically) i(request)e(more)g(resources)g(from)f(the)h(job)g(sc)o(heduler.)963 2790 y(3)p eop %%Page: 4 4 4 3 bop 374 841 a @beginspecial 145 @llx 445 @lly 415 @urx 642 @ury 2880 @rwi @setspecial %%BeginDocument: 3pieces.ps /arrowhead { 0 begin transform originalCTM itransform /taily exch def /tailx exch def transform originalCTM itransform /tipy exch def /tipx exch def /dy tipy taily sub def /dx tipx tailx sub def /angle dx 0 ne dy 0 ne or { dy dx atan } { 90 } ifelse def gsave originalCTM setmatrix tipx tipy translate angle rotate newpath arrowHeight neg arrowWidth 2 div moveto 0 0 lineto arrowHeight neg arrowWidth 2 div neg lineto patternNone not { originalCTM setmatrix /padtip arrowHeight 2 exp 0.25 arrowWidth 2 exp mul add sqrt brushWidth mul arrowWidth div def /padtail brushWidth 2 div def tipx tipy translate angle rotate padtip 0 translate arrowHeight padtip add padtail add arrowHeight div dup scale arrowheadpath ifill } if brushNone not { originalCTM setmatrix tipx tipy translate angle rotate arrowheadpath istroke } if grestore end } dup 0 9 dict put def /arrowheadpath { newpath arrowHeight neg arrowWidth 2 div moveto 0 0 lineto arrowHeight neg arrowWidth 2 div neg lineto } def /leftarrow { 0 begin y exch get /taily exch def x exch get /tailx exch def y exch get /tipy exch def x exch get /tipx exch def brushLeftArrow { tipx tipy tailx taily arrowhead } if end } dup 0 4 dict put def /rightarrow { 0 begin y exch get /tipy exch def x exch get /tipx exch def y exch get /taily exch def x exch get /tailx exch def brushRightArrow { tipx tipy tailx taily arrowhead } if end } dup 0 4 dict put def /arrowHeight 10 def /arrowWidth 5 def /IdrawDict 51 dict def IdrawDict begin /reencodeISO { dup dup findfont dup length dict begin { 1 index /FID ne { def }{ pop pop } ifelse } forall /Encoding ISOLatin1Encoding def currentdict end definefont } def /ISOLatin1Encoding [ /.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef /.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef /.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef /.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef /space/exclam/quotedbl/numbersign/dollar/percent/ampersand/quoteright /parenleft/parenright/asterisk/plus/comma/minus/period/slash /zero/one/two/three/four/five/six/seven/eight/nine/colon/semicolon /less/equal/greater/question/at/A/B/C/D/E/F/G/H/I/J/K/L/M/N /O/P/Q/R/S/T/U/V/W/X/Y/Z/bracketleft/backslash/bracketright /asciicircum/underscore/quoteleft/a/b/c/d/e/f/g/h/i/j/k/l/m /n/o/p/q/r/s/t/u/v/w/x/y/z/braceleft/bar/braceright/asciitilde /.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef /.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef /.notdef/dotlessi/grave/acute/circumflex/tilde/macron/breve /dotaccent/dieresis/.notdef/ring/cedilla/.notdef/hungarumlaut /ogonek/caron/space/exclamdown/cent/sterling/currency/yen/brokenbar /section/dieresis/copyright/ordfeminine/guillemotleft/logicalnot /hyphen/registered/macron/degree/plusminus/twosuperior/threesuperior /acute/mu/paragraph/periodcentered/cedilla/onesuperior/ordmasculine /guillemotright/onequarter/onehalf/threequarters/questiondown /Agrave/Aacute/Acircumflex/Atilde/Adieresis/Aring/AE/Ccedilla /Egrave/Eacute/Ecircumflex/Edieresis/Igrave/Iacute/Icircumflex /Idieresis/Eth/Ntilde/Ograve/Oacute/Ocircumflex/Otilde/Odieresis /multiply/Oslash/Ugrave/Uacute/Ucircumflex/Udieresis/Yacute /Thorn/germandbls/agrave/aacute/acircumflex/atilde/adieresis /aring/ae/ccedilla/egrave/eacute/ecircumflex/edieresis/igrave /iacute/icircumflex/idieresis/eth/ntilde/ograve/oacute/ocircumflex /otilde/odieresis/divide/oslash/ugrave/uacute/ucircumflex/udieresis /yacute/thorn/ydieresis ] def /Times-Roman reencodeISO def /none null def /numGraphicParameters 17 def /stringLimit 65535 def /Begin { save numGraphicParameters dict begin } def /End { end restore } def /SetB { dup type /nulltype eq { pop false /brushRightArrow idef false /brushLeftArrow idef true /brushNone idef } { /brushDashOffset idef /brushDashArray idef 0 ne /brushRightArrow idef 0 ne /brushLeftArrow idef /brushWidth idef false /brushNone idef } ifelse } def /SetCFg { /fgblue idef /fggreen idef /fgred idef } def /SetCBg { /bgblue idef /bggreen idef /bgred idef } def /SetF { /printSize idef /printFont idef } def /SetP { dup type /nulltype eq { pop true /patternNone idef } { dup -1 eq { /patternGrayLevel idef /patternString idef } { /patternGrayLevel idef } ifelse false /patternNone idef } ifelse } def /BSpl { 0 begin storexyn newpath n 1 gt { 0 0 0 0 0 0 1 1 true subspline n 2 gt { 0 0 0 0 1 1 2 2 false subspline 1 1 n 3 sub { /i exch def i 1 sub dup i dup i 1 add dup i 2 add dup false subspline } for n 3 sub dup n 2 sub dup n 1 sub dup 2 copy false subspline } if n 2 sub dup n 1 sub dup 2 copy 2 copy false subspline patternNone not brushLeftArrow not brushRightArrow not and and { ifill } if brushNone not { istroke } if 0 0 1 1 leftarrow n 2 sub dup n 1 sub dup rightarrow } if end } dup 0 4 dict put def /Circ { newpath 0 360 arc patternNone not { ifill } if brushNone not { istroke } if } def /CBSpl { 0 begin dup 2 gt { storexyn newpath n 1 sub dup 0 0 1 1 2 2 true subspline 1 1 n 3 sub { /i exch def i 1 sub dup i dup i 1 add dup i 2 add dup false subspline } for n 3 sub dup n 2 sub dup n 1 sub dup 0 0 false subspline n 2 sub dup n 1 sub dup 0 0 1 1 false subspline patternNone not { ifill } if brushNone not { istroke } if } { Poly } ifelse end } dup 0 4 dict put def /Elli { 0 begin newpath 4 2 roll translate scale 0 0 1 0 360 arc patternNone not { ifill } if brushNone not { istroke } if end } dup 0 1 dict put def /Line { 0 begin 2 storexyn newpath x 0 get y 0 get moveto x 1 get y 1 get lineto brushNone not { istroke } if 0 0 1 1 leftarrow 0 0 1 1 rightarrow end } dup 0 4 dict put def /MLine { 0 begin storexyn newpath n 1 gt { x 0 get y 0 get moveto 1 1 n 1 sub { /i exch def x i get y i get lineto } for patternNone not brushLeftArrow not brushRightArrow not and and { ifill } if brushNone not { istroke } if 0 0 1 1 leftarrow n 2 sub dup n 1 sub dup rightarrow } if end } dup 0 4 dict put def /Poly { 3 1 roll newpath moveto -1 add { lineto } repeat closepath patternNone not { ifill } if brushNone not { istroke } if } def /Rect { 0 begin /t exch def /r exch def /b exch def /l exch def newpath l b moveto l t lineto r t lineto r b lineto closepath patternNone not { ifill } if brushNone not { istroke } if end } dup 0 4 dict put def /Text { ishow } def /idef { dup where { pop pop pop } { exch def } ifelse } def /ifill { 0 begin gsave patternGrayLevel -1 ne { fgred bgred fgred sub patternGrayLevel mul add fggreen bggreen fggreen sub patternGrayLevel mul add fgblue bgblue fgblue sub patternGrayLevel mul add setrgbcolor eofill } { eoclip originalCTM setmatrix pathbbox /t exch def /r exch def /b exch def /l exch def /w r l sub ceiling cvi def /h t b sub ceiling cvi def /imageByteWidth w 8 div ceiling cvi def /imageHeight h def bgred bggreen bgblue setrgbcolor eofill fgred fggreen fgblue setrgbcolor w 0 gt h 0 gt and { l w add b translate w neg h scale w h true [w 0 0 h neg 0 h] { patternproc } imagemask } if } ifelse grestore end } dup 0 8 dict put def /istroke { gsave brushDashOffset -1 eq { [] 0 setdash 1 setgray } { brushDashArray brushDashOffset setdash fgred fggreen fgblue setrgbcolor } ifelse brushWidth setlinewidth originalCTM setmatrix stroke grestore } def /ishow { 0 begin gsave fgred fggreen fgblue setrgbcolor /fontDict printFont printSize scalefont dup setfont def /descender fontDict begin 0 [FontBBox] 1 get FontMatrix end transform exch pop def /vertoffset 1 printSize sub descender sub def { 0 vertoffset moveto show /vertoffset vertoffset printSize sub def } forall grestore end } dup 0 3 dict put def /patternproc { 0 begin /patternByteLength patternString length def /patternHeight patternByteLength 8 mul sqrt cvi def /patternWidth patternHeight def /patternByteWidth patternWidth 8 idiv def /imageByteMaxLength imageByteWidth imageHeight mul stringLimit patternByteWidth sub min def /imageMaxHeight imageByteMaxLength imageByteWidth idiv patternHeight idiv patternHeight mul patternHeight max def /imageHeight imageHeight imageMaxHeight sub store /imageString imageByteWidth imageMaxHeight mul patternByteWidth add string def 0 1 imageMaxHeight 1 sub { /y exch def /patternRow y patternByteWidth mul patternByteLength mod def /patternRowString patternString patternRow patternByteWidth getinterval def /imageRow y imageByteWidth mul def 0 patternByteWidth imageByteWidth 1 sub { /x exch def imageString imageRow x add patternRowString putinterval } for } for imageString end } dup 0 12 dict put def /min { dup 3 2 roll dup 4 3 roll lt { exch } if pop } def /max { dup 3 2 roll dup 4 3 roll gt { exch } if pop } def /midpoint { 0 begin /y1 exch def /x1 exch def /y0 exch def /x0 exch def x0 x1 add 2 div y0 y1 add 2 div end } dup 0 4 dict put def /thirdpoint { 0 begin /y1 exch def /x1 exch def /y0 exch def /x0 exch def x0 2 mul x1 add 3 div y0 2 mul y1 add 3 div end } dup 0 4 dict put def /subspline { 0 begin /movetoNeeded exch def y exch get /y3 exch def x exch get /x3 exch def y exch get /y2 exch def x exch get /x2 exch def y exch get /y1 exch def x exch get /x1 exch def y exch get /y0 exch def x exch get /x0 exch def x1 y1 x2 y2 thirdpoint /p1y exch def /p1x exch def x2 y2 x1 y1 thirdpoint /p2y exch def /p2x exch def x1 y1 x0 y0 thirdpoint p1x p1y midpoint /p0y exch def /p0x exch def x2 y2 x3 y3 thirdpoint p2x p2y midpoint /p3y exch def /p3x exch def movetoNeeded { p0x p0y moveto } if p1x p1y p2x p2y p3x p3y curveto end } dup 0 17 dict put def /storexyn { /n exch def /y n array def /x n array def n 1 sub -1 0 { /i exch def y i 3 2 roll put x i 3 2 roll put } for } def /SSten { fgred fggreen fgblue setrgbcolor dup true exch 1 0 0 -1 0 6 -1 roll matrix astore } def /FSten { dup 3 -1 roll dup 4 1 roll exch newpath 0 0 moveto dup 0 exch lineto exch dup 3 1 roll exch lineto 0 lineto closepath bgred bggreen bgblue setrgbcolor eofill SSten } def /Rast { exch dup 3 1 roll 1 0 0 -1 0 6 -1 roll matrix astore } def %I Idraw 10 Grid 8 8 Begin %I b u %I cfg u %I cbg u %I f u %I p u %I t [ 0.95606 0 0 0.95606 0 0 ] concat /originalCTM matrix currentmatrix def Begin %I Text %I cfg Black 0 0 0 SetCFg %I f -*-times-medium-r-normal-*-14-*-*-*-*-*-*-* Times-Roman 14 SetF %I t [ 1.04274 0 0 1.03064 172.502 552.666 ] concat %I [ ( Job) (Scheduler) ] Text End Begin %I Text %I cfg Black 0 0 0 SetCFg %I f -*-times-medium-r-normal-*-14-*-*-*-*-*-*-* Times-Roman 14 SetF %I t [ 1 0 0 1 270 552.5 ] concat %I [ ( Process) (Manager) ] Text End Begin %I Text %I cfg Black 0 0 0 SetCFg %I f -*-times-medium-r-normal-*-14-*-*-*-*-*-*-* Times-Roman 14 SetF %I t [ 1 0 0 1 359.5 552.5 ] concat %I [ (Application) ( Program) ] Text End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 181 419 229 419 Line %I 1 End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 229 403 181 403 Line %I 1 End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 277 419 325 419 Line %I 1 End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 325 403 277 403 Line %I 1 End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 181 507 325 507 Line %I 1 End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 325 491 181 491 Line %I 1 End Begin %I Text %I cfg Black 0 0 0 SetCFg %I f -*-times-medium-r-normal-*-14-*-*-*-*-*-*-* Times-Roman 14 SetF %I t [ 1 0 0 1 280 669 ] concat %I [ (User) ] Text End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 253 563 253 531 Line %I 1 End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 221 563 165 531 Line %I 1 End Begin %I Line %I b 65535 0 0 1 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 1 -0 -0 1 43 85 ] concat %I 277 563 341 531 Line %I 1 End Begin %I Rect %I b 65535 0 0 0 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 0.833334 0 0 0.833334 255.833 311.667 ] concat %I 101 186 213 354 Rect End Begin %I Rect %I b 65535 0 0 0 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 0.833334 0 0 0.833334 155.833 318.334 ] concat %I 109 178 221 346 Rect End Begin %I Rect %I b 65535 0 0 0 [] 0 SetB %I cfg Black 0 0 0 SetCFg %I cbg White 1 1 1 SetCBg none SetP %I p n %I t [ 0.833334 -0 -0 0.833334 -117.944 158.377 ] concat %I 325 370 437 538 Rect End End %I eop showpage end %%EndDocument @endspecial 487 939 a Fi(Figure)15 b(1:)20 b(Structure)15 b(of)g(the)g(Run)o(time)h(En)o(vironmen)o(t)74 1073 y Ff(2.2)56 b(Examples)16 b(of)j(Run)n(time)e(En)n(vironmen)n(ts)74 1188 y Fi(T)l(o)11 b(illustrate)i(ho)o(w)d(the)i(ab)q(o)o(v)o(e)f (framew)o(ork)e(allo)o(ws)j(us)f(to)g(describ)q(e)i(a)e(wide)h(v)m (ariet)o(y)g(of)f(actual)g(systems,)74 1245 y(w)o(e)k(giv)o(e)h(here)f (some)g(examples.)74 1395 y Fd(ANL's)i(SP2)46 b Fi(The)16 b(SP2)g(at)f(Argonne)h(National)g(Lab)q(oratory)f(is)h(sc)o(heduled)i (b)o(y)e(a)f(lo)q(cally)j(written)74 1451 y(job)10 b(sc)o(heduler)i (quite)f(di\013eren)o(t)f(from)f(the)i(LoadLev)o(eler)g(pro)q(duct)g (deliv)o(ered)h(with)e(the)g(SP2.)18 b(It)11 b(ensures)74 1508 y(that)k(only)h(one)g(user)g(has)f(access)h(to)f(an)o(y)g(SP)h(no) q(de)g(at)f(a)h(time)g(and)g(requires)g(users)g(to)f(pro)o(vide)h(time) 74 1564 y(limits)i(for)d(their)h(jobs)g(so)g(that)f(the)h(mac)o(hine)h (can)f(b)q(e)h(tigh)o(tly)f(sc)o(heduled.)25 b(Users)16 b(submit)g(scripts)h(to)74 1621 y(the)12 b(sc)o(heduler,)i(whic)o(h)f (sets)f(up)h(calls)g(to)f Fc(poe)p Fi(,)g(IBM's)g(pro)q(cess)g(manager) g(on)g(the)g(SP)l(.)g(The)h Fc(poe)f Fi(system)74 1677 y(in)o(teracts)j(with)h(a)f(v)m(ariet)o(y)g(of)g(message-passing)g (libraries,)h(includin)q(g)h(t)o(w)o(o)d(based)i(on)f(MPI.)74 1827 y Fd(The)h(Meik)o(o)f(CS-2)h(at)g(LLNL)47 b Fi(Job)14 b(sc)o(heduling)h(is)f(done)g(b)o(y)g(the)f(user)h(himself)h(who)e (insp)q(ects)i(the)74 1884 y(state)e(of)h(the)g(mac)o(hine)h(in)o (teractiv)o(ely)g(and)f(claims)h(a)e(partition)h(with)h(a)e(\014xed)i (n)o(um)o(b)q(er)f(of)g(pro)q(cessors.)74 1940 y(He)f(then)g(in)o(v)o (ok)o(es)f(the)g(pro)q(cess)h(manager)f(with)h(the)f Fc(prun)g Fi(command,)g(sp)q(ecifying)j(exactly)e(ho)o(w)f(man)o(y)74 1997 y(pro)q(cesses)h(he)g(wishes)g(to)f(execute)h(in)g(the)g(giv)o(en) g(partition.)19 b Fc(prun)12 b Fi(starts)f(pro)q(cesses)i(that)f(use)h (Meik)o(o's)74 2053 y(implemen)o(tation)k(of)d(In)o(tel's)i(NX)f (library)l(,)h(or)f(MPI)g(programs)f(that)h(run)g(on)g(top)g(of)g(this) g(library)l(.)74 2203 y Fd(P)o(aragon)g(at)h(Caltec)o(h)46 b Fi(There)14 b(are)f(three)g(sc)o(hedulers)h(for)f(the)g(P)o(aragons)f (op)q(erated)h(b)o(y)g(the)g(CSCC)74 2260 y(at)g(Caltec)o(h.)19 b(The)14 b(\014rst)f(t)o(w)o(o)f(are)h(for)f(in)o(teractiv)o(e)i(use.) 20 b(Programs)11 b(ma)o(y)i(b)q(e)h(started)e(b)o(y)i(simply)g(giving) 74 2316 y(the)g(n)o(um)o(b)q(er)g(of)f(no)q(des)h(as)g(an)f(argumen)o (t)g(or)g(b)o(y)h(creating)g(a)f(named)h(partition)g(of)f(a)g (particular)i(shap)q(e)74 2372 y(and)f(then)g(running)g(within)h(that)e (partition.)19 b(System)14 b(calls)g(to)f(create)g(partitions)h(and)g (run)f(programs)74 2429 y(are)i(pro)o(vided.)21 b(P)o(artitions)15 b(ma)o(y)f(b)q(e)i(gang-sc)o(heduled.)145 2515 y(The)d(other)g(is)h (the)g(NQS)g(batc)o(h)f(system,)g(whic)o(h)h(is)g(used)g(during)g(the)g (pro)q(duction)g(shift)g(\(ev)o(enings)74 2572 y(and)23 b(w)o(eek)o(ends\).)40 b(Users)23 b(submit)f(jobs)g(to)g(a)g (particular)h(queue;)j(NQS)d(allo)q(cates)g(the)f(necessary)74 2628 y(resources)14 b(and)h(starts)e(jobs.)19 b(The)c(jobs)f(are)f (usually)j(shell)g(scripts)e(b)q(ecause)i(they)e(start)f(in)i(the)f (user's)963 2790 y(4)p eop %%Page: 5 5 5 4 bop 74 7 a Fi(home)15 b(directory;)g(a)g(script)h(is)g(necessary)f (to)g(run)g(a)g(program)f(in)i(a)f(di\013eren)o(t)g(directory)l(.)74 157 y Fd(W)l(orkstation)g(net)o(w)o(ork)e(managed)i(b)o(y)f(DQS)46 b Fi(DQS)13 b([5)o(])f(is)h(a)g(batc)o(h)f(sc)o(heduler)i(for)e(w)o (orkstation)74 214 y(net)o(w)o(orks)h(dev)o(elop)q(ed)i(at)e(Florida)h (State)f(Univ)o(ersit)o(y)l(.)20 b(Users)14 b(submit)g(jobs)f(to)g(it)h (and)g(it)g(allo)q(cates)g(the)74 270 y(necessary)h(resources)f(and)h (starts)e(jobs.)19 b(It)c(has)f(an)h(in)o(terface)f(to)g Fc(p4)g Fi(that)g(allo)o(ws)h(it)g(to)e(start)h(parallel)74 327 y(jobs)20 b(written)h(using)g Fc(p4)f Fi(but)g(not)g(\(curren)o (tly\))g(an)o(y)g(other)g(library)l(.)36 b(Similarly)l(,)24 b(Condor,)c(a)g(batc)o(h)74 383 y(sc)o(heduler,)e(can)e(start)g(PVM)g (jobs)g(on)g(the)g(net)o(w)o(ork)g(it)g(manages)g(at)g(the)g(Univ)o (ersit)o(y)h(of)f(Wisconsin,)74 440 y(but)f(no)h(other)e(parallel)j (programs.)74 590 y Fd(Basic)d(w)o(orkstation)e(net)o(w)o(ork)f(with)j (PVM)44 b Fi(One)12 b(reason)e(for)h(PVM's)f(p)q(opularit)o(y)i(is)f (that)g(it)g(can)74 646 y(b)q(e)k(view)o(ed)h(as)e(a)g(completely)i (self-con)o(tained)g(system)e(that)g(supplies)i(its)f(o)o(wn)f(pro)q (cess)h(managemen)o(t)74 702 y(and)d(can)h(b)q(e)f(used)h(to)f (implemen)o(t)h(a)f(job)g(sc)o(heduler)h(as)f(w)o(ell.)20 b(On)12 b(systems)g(that)f(ha)o(v)o(e)h(neither)h(of)f(these)74 759 y(functions)20 b(pre-installed,)i(PVM)c(can)h(pro)o(vide)h(a)f (complete)g(solution.)32 b(A)19 b(user)g(creates)g(a)g(\\virtual)74 815 y(mac)o(hine")d(b)o(y)f(starting)g(\\daemons")g(on)g(an)h (assortmen)o(t)d(of)i(mac)o(hines)h(and)g(then)g(sc)o(hedules)h(jobs)e (to)74 872 y(run)g(on)g(it)g(and)h(manages)e(his)i(pro)q(cesses)f(with) g(the)g(help)i(of)d(of)h(the)g(daemons.)k(The)d(virtual)f(mac)o(hine)74 928 y(itself)21 b(can)e(b)q(e)h(recon\014gured)g(from)f(inside)j(the)d (user)h(program.)31 b(A)20 b(di\016cult)o(y)h(with)f(this)f(approac)o (h)74 985 y(is)h(that)e(the)h(user)g(is)h(assumed)f(to)g(ha)o(v)o(e)f (the)h(necessary)h(p)q(ermissions)g(to)f(execute)g(suc)o(h)h (functions.)74 1041 y(This)15 b(ma)o(y)f(b)q(e)h(the)f(case)h(on)f(a)g (w)o(orkstation)f(net)o(w)o(ork,)g(but)i(seldom)g(on)f(an)g(MPP)l(.)g (Con\015icts)h(b)q(et)o(w)o(een)74 1098 y(existing)e(pro)q(cess)g (managers)e(and)i(PVM)e(can)i(inhibit)h(the)f(p)q(ortabilit)o(y)g(\(to) e(MPP's\))g(of)h(self-con)o(tained)74 1154 y(programs)i(that)h(assume)g (all)h(functionalit)o(y)g(will)h(b)q(e)f(pro)o(vided)g(b)o(y)f(PVM.)74 1306 y Ff(2.3)56 b(Applications)12 b(Requiring)e(Direct)i(Comm)n (unication)e(with)i(the)g(Run)n(time)f(Sys-)202 1364 y(tem)74 1480 y Fi(The)22 b(existing)g(MPI)g(sp)q(eci\014cation)h(is)f (adequate)f(for)g(most)f(parallel)j(applications.)40 b(In)22 b(these)g(ap-)74 1536 y(plications,)g(the)d(job)h(sc)o(heduler) g(and)g(pro)q(cess)g(manager,)f(whether)g(simple)i(or)e(elab)q(orate,)h (allo)q(cate)74 1593 y(resources)15 b(and)h(manage)f(user)g(pro)q (cesses)h(without)f(in)o(teracting)h(with)f(the)h(application)h (program.)h(In)74 1649 y(other)c(applications,)i(ho)o(w)o(ev)o(er,)e (it)h(is)g(necessary)g(that)f(the)h Fe(user)h(level)i Fi(of)c(the)h(application)h(comm)o(uni-)74 1706 y(cate)g(with)h(the)g (job)f(sc)o(heduler)i(and)f(pro)q(cess)g(manager.)23 b(Here)16 b(w)o(e)h(describ)q(e)h(three)e(broad)h(classes)g(of)74 1762 y(suc)o(h)f(applications.)21 b(In)16 b(Section)g(4)f(w)o(e)g(will) i(giv)o(e)e(concrete)h(examples)g(of)e(eac)o(h)i(of)e(these)i(classes.) 74 1912 y Fd(T)l(ask)h(F)l(arming)45 b Fi(By)16 b(a)e(\\task)h(farm")f (application)j(w)o(e)d(mean)i(a)e(program)g(that)h(manages)f(the)i (exe-)74 1969 y(cution)f(of)e(a)g(set)h(of)f(other,)g(p)q(ossibly)i (sequen)o(tial,)g(programs.)j(This)c(situation)h(often)e(arises)h(when) g(one)74 2025 y(w)o(an)o(ts)j(to)h(run)g(the)g(same)g(sequen)o(tial)h (program)e(man)o(y)h(times)g(with)g(v)m(arying)h(input)g(data.)28 b(W)l(e)18 b(call)74 2081 y(eac)o(h)g(in)o(v)o(o)q(cation)g(of)f(the)h (sequen)o(tial)h(program)d(a)i Fe(task)p Fi(.)27 b(It)17 b(is)h(often)g(simplest)g(to)f(\\parallelize")j(the)74 2138 y(existing)c(sequen)o(tial)f(program)f(b)o(y)g(writing)h(a)f (parallel)i(\\harness")e(program)g(that)f(in)j(turn)e(dev)o(otes)g(a)74 2194 y(separate,)i(transien)o(t)h(pro)q(cess)g(to)f(eac)o(h)h(task.)23 b(When)17 b(one)g(task)f(\014nishes,)i(a)f(new)g(pro)q(cess)g(is)g (started)74 2251 y(to)e(execute)g(the)g(next)g(one.)20 b(Ev)o(en)15 b(if)h(the)f(resources)g(allo)q(cated)h(to)f(the)g(job)f (are)h(\014xed,)h(the)f(\\harness")74 2307 y(pro)q(cess)g(m)o(ust)e(in) o(teract)h(frequen)o(tly)h(with)g(the)f(pro)q(cess)g(manager)g(\(ev)o (en)g(if)h(this)g(is)f(just)g Fc(rsh)p Fi(,)g(to)f(start)74 2364 y(the)j(new)g(pro)q(cesses)g(with)g(the)g(new)g(input)g(data\).)21 b(In)16 b(man)o(y)f(cases)h(this)g(harness)f(can)h(b)q(e)h(written)e (in)74 2420 y(a)g(simple)i(scripting)f(language)f(lik)o(e)i Fc(csh)d Fi(or)h Fc(perl)p Fi(,)f(but)i(some)f(users)g(prefer)g(to)g (use)g(F)l(ortran)f(or)h(C.)74 2570 y Fd(Dynamic)k(n)o(um)o(b)q(er)e (of)h(pro)q(cesses)f(in)i(parallel)h(job)45 b Fi(The)16 b(program)f(wishes)h(to)g(decide)h Fe(inside)74 2627 y Fi(the)e(program)e(to)h(adjust)g(the)h(n)o(um)o(b)q(er)g(of)f(pro)q (cesses)h(to)f(\014t)g(the)h(size)g(of)f(the)h(problem.)20 b(F)l(urthermore,)963 2790 y(5)p eop %%Page: 6 6 6 5 bop 74 7 a Fi(it)12 b(ma)o(y)g(con)o(tin)o(ue)g(to)g(add)g(and)g (subtract)g(pro)q(cesses)g(during)h(the)f(computation)g(to)f(\014t)h (separate)g(phases)74 64 y(of)f(the)h(computation,)g(some)f(of)g(whic)o (h)h(ma)o(y)f(b)q(e)h(more)f(parallel)i(than)f(others.)18 b(In)12 b(order)f(to)g(do)h(this,)g(the)74 120 y(application)i(program) c(will)k(ha)o(v)o(e)d(to)h(in)o(teract)f(with)i(the)f(job)f(sc)o (heduler)i(\(ho)o(w)o(ev)o(er)e(it)h(is)h(implemen)o(ted\))74 177 y(to)e(request)g(and)h(acquire)g(or)f(return)g(computation)h (resources.)18 b(It)12 b(will)h(also)e(ha)o(v)o(e)g(to)g(in)o(teract)g (with)h(the)74 233 y(pro)q(cess)i(manager)e(to)h(request)g(that)f(pro)q (cess)i(b)q(e)f(started,)g(and)g(in)h(order)f(to)f(mak)o(e)h(the)g(new) h(pro)q(cesses)74 289 y(kno)o(wn)g(to)f(the)i(message-passing)f (library)h(so)f(that)f(the)h(larger)g(\(or)f(smaller\))i(group)f(of)f (pro)q(cesses)i(can)74 346 y(comm)o(unicate.)74 496 y Fd(Clien)o(t/Serv)o(er)45 b Fi(This)21 b(situation)f(is)h(the)g(opp)q (osite)g(of)f(the)g(situations)h(ab)q(o)o(v)o(e,)g(where)g(pro)q (cesses)74 552 y(come)e(and)h(go)f(up)q(on)h(request.)32 b(In)20 b(the)g(clien)o(t/serv)o(er)g(mo)q(del,)h(one)e(set)g(of)g(pro) q(cesses)h(is)g(relativ)o(ely)74 609 y(p)q(ermanen)o(t)h(\(the)g(serv)o (er,)g(whic)o(h)h(w)o(e)f(assume)f(here)i(is)f(a)g(parallel)h (program\).)36 b(A)o(t)20 b(unpredictable)74 665 y(times,)15 b(another)f(\(p)q(ossibly)h(parallel\))h(program)d(\(the)i(clien)o(t\)) g(b)q(egins)h(execution)f(and)g(m)o(ust)f(establish)74 722 y(comm)o(unication)k(with)f(the)h(serv)o(er.)25 b(In)17 b(this)h(case)f(the)g(pro)q(cess)g(manager)g(m)o(ust)f(pro)o(vide)i(a)f (w)o(a)o(y)f(for)74 778 y(the)e(clien)o(t)h(to)e(lo)q(cate)h(the)g (serv)o(er)f(and)h(comm)o(unicate)g(to)f(the)h(message-passing)f (library)i(that)e(it)h(m)o(ust)74 835 y(no)o(w)h(supp)q(ort)g(comm)o (unications)h(with)g(a)e(new)i(collection)h(of)e(pro)q(cesses.)145 921 y(It)20 b(is)h(curren)o(tly)g(p)q(ossible)h(to)e(write)g(the)h (parallel)g(clien)o(ts)h(and)f(serv)o(ers)e(in)j(MPI,)e(but)g(b)q (ecause)74 978 y(MPI)13 b(do)q(es)g(not)g(pro)o(vide)g(the)g(necessary) g(in)o(terfaces)g(b)q(et)o(w)o(een)g(the)g(application)i(program)c(and) i(the)g(job)74 1034 y(sc)o(heduler)k(or)e(pro)q(cess)h(manager,)f (other)g(nonp)q(ortable,)i(mac)o(hine)f(sp)q(eci\014c)i(libraries)f(m)o (ust)e(b)q(e)i(called)74 1091 y(in)i(order)f(for)g(the)g(clien)o(t)h (and)g(serv)o(er)e(to)h(comm)o(unicate)g(with)h(one)f(another.)28 b(On)19 b(the)f(other)g(hand,)74 1147 y(MPI)f(do)q(es)f(con)o(tain)h (sev)o(eral)f(features)g(that)g(mak)o(e)g(it)h(relativ)o(ely)g(easy)f (to)g(add)h(suc)o(h)f(in)o(terfaces,)h(and)74 1203 y(w)o(e)e(prop)q (ose)g(b)q(oth)h(a)f(simple)h(in)o(terface)g(and)f(a)g(more)g(complex)h (but)f(\015exible)i(one.)74 1377 y Fj(3)69 b(Extending)23 b(MPI)g(for)h(Pro)r(cess)f(Managemen)n(t)74 1508 y Fi(In)17 b(this)f(section)g(w)o(e)f(will)j(\014rst)d(describ)q(e)j(requiremen)o (ts)e(for)f(the)h(in)o(terface)g(whic)o(h)g(in\015uence)i(some)e(of)74 1565 y(the)j(decisions.)32 b(Then)19 b(w)o(e)f(will)j(\(\014nally\))e (prop)q(ose)g(a)f(set)h(of)f(MPI)h(extensions)g(that)f(will)i(meet)f (the)74 1621 y(requiremen)o(ts.)37 b(Note)20 b(that)g(w)o(e)g(think)i (of)e(ourselv)o(es)h(as)f(pro)o(viding)h(an)g(in)o(terface)g(to)f (existing)h(job)74 1677 y(sc)o(heduling)16 b(and)f(pro)q(cess)f (managemen)o(t)f(systems.)19 b(If)c(they)f(do)g(not)g(exist,)g(then)h (w)o(e)e(ma)o(y)h(w)o(an)o(t)f(to)g(b)q(e)74 1734 y(able)18 b(to)e(write)h(them)f(in)i(MPI.)e(Some)h(prop)q(osals)g(for)f(spa)o (wning)h(new)g(pro)q(cesses)g(in)g(an)g(\\MPI)f(w)o(a)o(y")74 1790 y(ha)o(v)o(e)g(previously)h(b)q(een)h(made)e(in)h([3)o(],)f([8)o (])g(and)32 b([6].)22 b(Our)17 b(prop)q(osals)f(here)g(o\013er)g (considerably)i(more)74 1847 y(functionalit)o(y)f(and)e(\015exibilit)o (y)l(.)74 1999 y Ff(3.1)56 b(Requireme)o(n)n(ts)74 2114 y Fi(Of)21 b(course)g(the)g(most)g(basic)g(requiremen)o(t)h(is)f(that)g (w)o(e)f(b)q(e)i(able)g(to)e(write)h(p)q(ortable)h(applications)74 2171 y(in)e(the)f(ab)q(o)o(v)o(e)f(classes,)i(that)e(can)h(run)g(in)h (a)f(v)m(ariet)o(y)g(of)f(job)h(sc)o(heduling)i(|)e(pro)q(cess)g (managemen)o(t)74 2227 y(en)o(vironmen)o(ts.)g(In)13 b(addition,)h(w)o(e)d(w)o(ould)i(lik)o(e)g(our)f(in)o(terface)h(to)e (ha)o(v)o(e)h(a)g(n)o(um)o(b)q(er)g(of)g(other)g(prop)q(erties.)74 2377 y Fd(Determinism)45 b Fi(The)16 b(seman)o(tics)h(of)e(dynamic)i (pro)q(cess)f(creation)g(m)o(ust)g(b)q(e)g(carefully)h(designed)h(to)74 2434 y(a)o(v)o(oid)h(race)g(conditions.)33 b(In)19 b(MPI,)g(ev)o(ery)g (pro)q(cess)g(is)h(a)f(mem)o(b)q(er)g(of)g(some)f(comm)o(unicator;)j (when)74 2490 y(w)o(e)e(allo)o(w)f(MPI)h(to)f(create)h(or)f(destro)o(y) g(pro)q(cesses,)h(all)h(of)e(the)h(comm)o(unicators)f(that)g(that)g (pro)q(cess)74 2547 y(b)q(elongs)j(to)f(c)o(hange.)34 b(In)21 b(order)f(to)f(k)o(eep)i(collectiv)o(e)h(op)q(erations)e(on)g (comm)o(unicators)g(meaningful)74 2603 y(\(for)d(example,)i(what)e(do)q (es)h(a)f(reduction)h(mean)g(when)g(a)f(pro)q(cess)h(joins)g(the)g (reduction)h(during)f(the)74 2660 y(op)q(eration;)24 b(for)c(that)g(matter,)h(ho)o(w)g(is)g(\\during")h(de\014ned\),)h(all)f (c)o(hanges)f(to)f(comm)o(unicators)g(are)963 2790 y(6)p eop %%Page: 7 7 7 6 bop 74 7 a Fi(collectiv)o(e)23 b(op)q(erations.)39 b(In)22 b(PVM)g(terms,)g(w)o(e)f(will)i(not)e(allo)o(w)h(a)f(new)h(pro) q(cess)f(to)g(join)h(a)f(group)74 64 y(while)f(a)e(collectiv)o(e)i(op)q (eration)f(o)o(v)o(er)e(that)h(group)g(is)g(in)i(progress.)28 b(\(Error)17 b(handling)j(is)f(dealt)g(with)74 120 y(separately)l(.\)) 74 270 y Fd(Scalability)k(and)e(p)q(erformance)45 b Fi(It)19 b(m)o(ust)e(b)q(e)i(p)q(ossible)h(to)e(deal)h(with)g(large)f(n)o(um)o (b)q(ers)g(of)g(pro-)74 327 y(cesses)e(b)o(y)g(exploiting)h(p)q(oten)o (tial)g(scalabilit)o(y)g(in)g(the)f(job)f(sc)o(heduler)j(or)d(pro)q (cess)h(manager.)21 b(In)16 b(addi-)74 383 y(tion,)f(since)h(eac)o(h)e (of)h(the)g(steps)f(of)h(allo)q(cating)g(resources)g(and)g(starting)f (pro)q(cesses)h(can)g(b)q(e)g(v)o(ery)g(time)74 440 y(consuming,)20 b(w)o(e)f(allo)o(w)g(eac)o(h)g(of)f(these)i(steps)e(to)h(b)q(e)g (non-blo)q(c)o(king)i(so)d(that)g(other)h(w)o(ork)f(can)h(tak)o(e)74 496 y(place)d(during)g(these)g(steps.)74 646 y Fd(Econom)o(y)45 b Fi(W)l(e)14 b(w)o(ould)f(lik)o(e)i(to)d(tak)o(e)h(adv)m(an)o(tage)g (of)g(MPI's)f(ric)o(h)i(set)f(of)g(functions)h(for)f(dealing)i(with)74 702 y(async)o(hronicit)o(y)l(,)h(and)f(a)o(v)o(oid)g(in)o(tro)q(ducing) i(new)e(ob)s(jects)g(or)f(functions)i(if)g(it)g(can)f(b)q(e)h(a)o(v)o (oided.)145 789 y(MPI's)21 b(curren)o(t)h(design)h(mak)o(es)e(it)h(far) f(easier)h(to)g(meet)f(these)h(requiremen)o(ts)h(than)e(for)g(other)74 845 y(systems)e(to)g(do)g(so.)32 b(As)19 b(will)i(b)q(e)f(seen)g(in)g (the)f(next)h(section,)g(w)o(e)f(will)i(b)q(e)f(able)g(to)f(eliminate)i (race)74 902 y(conditions)16 b(b)o(y)f(using)g(MPI's)f(comm)o (unicators)h(to)f(encapsulate)h(the)g(collectiv)o(e)h(act)f(of)f(c)o (hanging)h(the)74 958 y(n)o(um)o(b)q(er)e(of)f(pro)q(cesses)h(in)h(a)e (group.)19 b(By)13 b(adding)g(new)g(v)m(arian)o(ts)g(of)f(the)g (existing)i Fb(MPI)p 1563 958 14 2 v 16 w(Request)h Fi(ob)s(ject,)74 1015 y(w)o(e)e(will)i(b)q(e)f(able)g(to)e(tak)o(e)g(adv)m(an)o(tage)h (of)g(MPI's)f(extensiv)o(e)i(set)f(of)g(functions)h(for)e(testing)i (and)f(w)o(aiting)74 1071 y(on)g(n)o(um)o(b)q(ers)g(of)f(requests)g (and)h(th)o(us)g(add)f(a)h(ric)o(h)g(collection)h(of)f(non-blo)q(c)o (king)h(op)q(erations)f(for)f(pro)q(cess)74 1128 y(managemen)o(t.)74 1279 y Ff(3.2)56 b(MPI)19 b(Extensions)74 1395 y Fi(W)l(e)j(assume)f (that)g(reader)g(is)h(familiar)g(with)g(the)g(MPI)f(sp)q (eci\014cation,)k(particularly)e(comm)o(unica-)74 1452 y(tors)15 b(\(b)q(oth)g(in)o(tra)h(and)g(in)o(ter\),)f(p)q(ersisten)o (t)h(requests,)g(and)g(the)f(family)i(of)e Fb(MPI)p 1471 1452 V 16 w(Sta)o(rt)p Fi(,)g Fb(MPI)p 1691 1452 V 16 w(T)l(est)i Fi(and)74 1508 y Fb(MPI)p 159 1508 V 16 w(W)o(ait)e Fi(op)q(erations.)145 1595 y(The)f(MPI)h(extensions)g(w)o(e)f(need)i (will)g(b)q(e)f(dictated)g(b)o(y)f(the)h(view)g(w)o(e)f(ha)o(v)o(e)g (tak)o(en)g(of)g(the)h(run)o(time)74 1651 y(en)o(vironmen)o(t)g(as)f (consisting)h(of)f(a)h(job)f(sc)o(heduler,)h(pro)q(cess)g(manager,)f (and)g(comm)o(unication)h(library)l(.)74 1707 y(T)l(o)i(consider)h (clien)o(t-serv)o(er)h(applications)f(w)o(e)f(m)o(ust)g(also)g (consider)h(existing)g(running)h(parallel)f(jobs)74 1764 y(as)d(part)g(of)f(the)i(en)o(vironmen)o(t)f(as)g(w)o(ell.)145 1850 y(W)l(e)i(will)i(\014rst)d(presen)o(t)h(a)g(v)o(ery)g(simple)h(in) o(terface)g(that)e(com)o(bines)i(access)f(to)g(the)g(job)g(sc)o (heduler)74 1907 y(and)j(pro)q(cess)g(manager,)g(y)o(et)f(pro)o(vides)h (for)f(at)g(least)h(one)g(kind)h(of)e(dynamic)i(pro)q(cess)f(con)o (trol)f(and)74 1963 y(for)13 b(some)h(clien)o(t-serv)o(er)h (applications.)21 b(These)14 b(will)i(b)q(e)e(straigh)o(tforw)o(ard,)e (blo)q(c)o(king,)j(comm)o(unicator-)74 2020 y(based)d(op)q(erations.)19 b(Then)12 b(w)o(e)g(will)h(sho)o(w)e(ho)o(w)g(increased)i(\015exibilit) o(y)h(and)e(e\016ciency)h(can)f(b)q(e)g(obtained)74 2076 y(b)o(y)j(breaking)h(these)f(in)o(to)g(comp)q(onen)o(t)h(op)q (erations,)f(whic)o(h)h(are)f(non-blo)q(c)o(king)h(and)g(op)q(erate)f (directly)74 2133 y(on)g(dynamic)h(pro)q(cesses)g(represen)o(ted)g(b)o (y)f(a)g(new)g(v)m(ariet)o(y)g(of)g Fb(MPI)p 1240 2133 V 16 w(Request)j Fi(ob)s(ject.)74 2283 y Fd(3.2.1)52 b(MPI)16 b(In)o(ter-comm)o(unicators)74 2399 y Fi(Since)24 b(in)o(ter-comm)o(unicators)e(are)g(sometimes)h(though)o(t)e(of)h(as)g (one)h(of)f(the)g(\\exotic")h(features)f(of)74 2455 y(MPI,)g(in)h(this) f(section)h(w)o(e)e(brie\015y)j(remind)f(the)f(reader)g(of)f(the)h (main)h(features)f(of)f(MPI's)h(in)o(ter-)74 2511 y(comm)o(unicators.)e (F)l(or)14 b(more)h(on)g(in)o(ter-comm)o(unicators,)g(see)g([7].)145 2598 y(What)d(w)o(e)i(most)e(t)o(ypically)j(refer)e(to)g(as)g(a)g Fe(c)n(ommunic)n(ator)19 b Fi(in)14 b(MPI)f(is)h(more)f(precisely)i (referred)f(to)74 2654 y(as)e(an)h Fe(intr)n(a-c)n(ommunic)n(ator)p Fi(.)19 b(It)13 b(encapsulates)h(a)e(con)o(text)g(and)h(a)f(single)i (group)f(of)f(pro)q(cesses,)h(and)g(all)963 2790 y(7)p eop %%Page: 8 8 8 7 bop 74 7 a Fi(pro)q(cesses)16 b(men)o(tioned)g(in)g(comm)o (unication)g(op)q(erations)f(that)f(use)i(this)f(comm)o(unicator)g(are) g(referred)74 64 y(to)g(b)o(y)g(their)h(rank)e(in)j(the)e(group)g(asso) q(ciated)g(with)h(the)f(comm)o(unicator.)145 150 y(An)20 b Fe(inter-c)n(ommunic)n(ator)25 b Fi(encapsulates)20 b(a)g(con)o(text)f(and)h Fe(two)j Fi(groups)c(of)g(pro)q(cesses,)i (referred)74 207 y(to)15 b(\(from)f(the)i(p)q(oin)o(t)g(of)f(view)h(of) f(a)h(pro)q(cess)f(b)q(elonging)i(to)e(the)h(in)o(ter-comm)o (unicator's)f(lo)q(cal)i(group\))74 263 y(as)i(the)g Fe(lo)n(c)n(al)j Fi(group)d(and)g(the)g Fe(r)n(emote)k Fi(group.)31 b(The)19 b(lo)q(cal)h(group)f(is)g(the)g(\\normal")g (group)f(of)h(the)74 319 y(\(in)o(ter-\)comm)o(unicator,)i(whose)f (size)h(is)g(returned)g(b)o(y)f Fb(MPI)p 1157 319 14 2 v 16 w(COMM)p 1315 319 V 16 w(SIZE)p Fi(.)g(The)h(remote)f(group)g (is)74 376 y(another)14 b(group)h(of)f(pro)q(cesses,)h(whose)f(size)i (is)f(returned)g(b)o(y)f Fb(MPI)p 1224 376 V 16 w(COMM)p 1382 376 V 16 w(REMOTE)p 1585 376 V 17 w(SIZE)p Fi(.)g(When)h(a)74 432 y(pro)q(cess)e(is)g(referred)g(to)f(b)o(y)g(rank)g(in)i(an)e(comm)o (unication)i(op)q(eration)e(using)i(the)e(in)o(ter-comm)o(unicator,)74 489 y(that)k(pro)q(cess)g(is)h(the)f(one)h(with)f(that)g(rank)g(in)h (the)f(remote)g(group.)22 b(Th)o(us)16 b(in)o(tercomm)o(unicators)g (are)74 545 y(used)h(precisely)h(when)f(w)o(e)f(w)o(an)o(t)f(to)h(comm) o(unicate)h(b)q(et)o(w)o(een)f(pro)q(cesses)h(in)g(di\013eren)o(t)g (groups,)e(using)74 602 y(their)h(ranks)f(in)h(their)f(resp)q(ectiv)o (e)i(groups)e(to)f(iden)o(tify)j(them.)145 688 y(The)f(t)o(w)o(o)e (groups)i(can)g(easily)g(b)q(e)h(\\joined")f(in)o(to)g(an)g(in)o (tra-comm)o(unicator)f(whose)h(\(only\))f(group)74 745 y(is)h(the)f(union)h(of)f(the)g(t)o(w)o(o)f(groups,)h(b)o(y)g(the)g (collectiv)o(e)i(op)q(eration)f Fb(MPI)p 1327 745 V 15 w(INTERCOMM)p 1616 745 V 17 w(MERGE)p Fi(.)145 831 y(W)l(e)c(will)i (use)e(in)o(ter-comm)o(unicators)g(as)g(a)g(w)o(a)o(y)f(to)g(manage)h (the)g(distinction)i(b)q(et)o(w)o(een)f(t)o(w)o(o)d(groups)74 888 y(of)16 b(pro)q(cesses)i(when)f(one)g(group)f(collectiv)o(ely)j (creates)e(the)f(other)h(group)f(\(spa)o(wning\))g(or)h(else)g(w)o(an)o (ts)74 944 y(to)e(establish)i(comm)o(unication)f(with)g(an)f(existing)i (group)e(\(clien)o(t-serv)o(er\).)21 b(Details)16 b(are)g(giv)o(en)g (in)g(the)74 1000 y(follo)o(wing)g(sections.)74 1151 y Fd(3.2.2)52 b(Simple)18 b(In)o(terface)f(to)h(Pro)q(cess)f(Manager)74 1266 y Fi(In)22 b(this)g(section)h(w)o(e)e(pro)o(vide)h(high-lev)o(el)i (functions)e(b)q(oth)g(for)f(pro)q(cess)h(creation)g(and)g(for)f(clien) o(t-)74 1323 y(serv)o(er)13 b(requiremen)o(ts)g(that)f(are)h(the)g (easiest)g(to)f(use.)20 b(They)13 b(encapsulate)h(in)o(teractions)f (with)g(all)h(three)74 1379 y(comp)q(onen)o(ts)k(of)f(the)h(run)o(time) g(en)o(vironmen)o(t)f(at)g(once,)h(and)g(are)f(blo)q(c)o(king)i(calls.) 28 b(These)18 b(pro)o(vide)g(a)74 1436 y(useful)e(subset)e(of)h(op)q (erations;)f(routines)h(that)f(pro)o(vide)h(more)f(con)o(trol)g(and)h (\015exibilit)o(y)i(are)d(describ)q(ed)74 1492 y(later.)145 1579 y(The)h(follo)o(wing)i(is)f(the)f(simplest)i(w)o(a)o(y)d(to)h (create)g(new)h(pro)q(cesses,)g(and)g(is)g(used)g(when)g(one)f(w)o(an)o (ts)74 1635 y(simply)h(to)e(expand)h(the)f(n)o(um)o(b)q(er)h(of)f(pro)q (cesses)h(in)g(a)f(comm)o(unicator.)20 b(It)14 b(mak)o(es)g(sev)o(eral) h(simplifying)74 1692 y(assumptions:)k(that)13 b(the)h(existing)g(pro)q (cesses)g(are)f(all)i(executing)f(the)g(same)f(executable)i(on)e(mac)o (hines)74 1748 y(of)g(the)g(same)g(arc)o(hitecture,)g(and)g(that)g(the) g(new)g(pro)q(cesses)h(do)f(not)f(need)i(an)o(y)f(new)g(argumen)o(ts.) 19 b(That)74 1804 y(is,)c(this)g(call)g(is)g(for)f(expanding)h(the)g(n) o(um)o(b)q(er)f(of)g(pro)q(cesses)h(in)g(an)f(SPMD)g(computation.)20 b(It)14 b(assumes)74 1861 y(that)19 b(the)g(job)h(sc)o(heduler)g(can)g (iden)o(tify)h(pro)q(cessors)e(to)f(run)i(the)g(new)f(pro)q(cesses)h (on)f(without)h(help)74 1917 y(from)15 b(the)g(user)g(program.)74 2027 y Fb(MPI)p 159 2027 V 16 w(COMM)p 317 2027 V 16 w(MODIFY\(oldcomm,)e(count,)j(new)o(comm\))116 2121 y Fk(IN)156 b Fb(oldcomm)412 b Fk(comm)o(unicator)11 b(of)j(spa)o(wning)f (group)116 2178 y(IN)156 b Fb(count)481 b Fk(n)o(um)o(b)q(er)17 b(of)g(pro)q(cesses)j(to)d(c)o(hange)h(b)o(y)g(\(p)q(ositiv)o(e)f(or)g (nega-)904 2234 y(tiv)o(e\))116 2291 y(OUT)109 b Fb(new)o(comm)396 b Fk(new)19 b(in)o(tra-comm)o(unicator)d(with)i(larger)g(n)o(um)o(b)q (er)g(of)g(pro-)904 2347 y(cesses)145 2494 y Fi(This)g(op)q(eration)h (is)f(collectiv)o(e)i(o)o(v)o(er)d Fb(oldcomm)p Fi(.)27 b(If)18 b Fb(count)h Fi(is)g(p)q(ositiv)o(e,)g(the)g(spa)o(wned)f(pro)q (cesses)74 2551 y(come)f(in)o(to)f(existence)i(with)f(an)g Fb(MPI)p 735 2551 V 16 w(COMM)p 893 2551 V 16 w(W)o(ORLD)g Fi(that)f(includes)j(b)q(oth)e(the)f(pro)q(cesses)h(in)h Fb(old-)74 2607 y(comm)c Fi(and)i(the)g Fb(count)i Fi(pro)q(cesses)e(b) q(eing)h(spa)o(wned)g(b)o(y)e(this)i(call.)23 b(In)17 b Fb(new)o(comm)d Fi(the)i(new)h(pro)q(cesses)74 2664 y(are)h(added)h(with)g(ranks)f(higher)h(than)f(those)g(of)g(the)h (original)g(pro)q(cesses,)g(or)f(subtracted)g(from)g(the)963 2790 y(8)p eop %%Page: 9 9 9 8 bop 74 7 a Fi(pro)q(cesses)20 b(with)f(the)g(highest)h(ranks.)31 b(This)19 b(call)h(do)q(es)g(not)e(require)i(an)o(y)f(understanding)h (of)f(in)o(ter-)74 64 y(comm)o(unicators.)145 150 y(A)k(pro)q(cess)h (can)g(determine)g(if)g(it)g(w)o(as)f(created)h(b)o(y)f(testing)h Fb(MPI)p 1363 150 14 2 v 15 w(COMM)p 1520 150 V 17 w(P)l(ARENT)g Fi(against)74 207 y Fb(MPI)p 159 207 V 16 w(COMM)p 317 207 V 16 w(NULL)p Fi(.)17 b(The)h(created)g(pro)q(cesses)g(b)q(egin)h (execution)g(at)e(the)g(b)q(eginning)j(of)d(their)i(main)74 263 y(program,)14 b(just)h(as)g(if)g(they)h(had)f(b)q(een)h(started)f (b)o(y)g(the)g(user.)215 364 y Fl(Discussion:)e Fk(W)m(e)c(could)g(ha)o (v)o(e)h(t)o(w)o(o)e(functions)i Fa(MPI)p 1052 364 13 2 v 14 w(COMM)p 1196 364 V 15 w(EXP)m(AND)d Fk(and)j Fa(MPI)p 1534 364 V 14 w(COMM)p 1678 364 V 15 w(CONTRA)o(CT)p Fk(,)145 413 y(but)k(it)f(seems)h(simpler)f(just)h(to)g(com)o(bine)e (these.)215 535 y Fl(Discussion:)i Fk(An)c(alternativ)o(e)h(uses)g(an)f Fa(INOUT)g Fk(comm)o(unicator)d(and)k(a)f(coun)o(t)g(that)h(is)f (signi\014can)o(t)145 585 y(only)j(at)h(the)g(spa)o(wners.)145 692 y Fi(The)h(follo)o(wing)h(function)g(is)f(the)g(next-easiest)h(w)o (a)o(y)e(to)g(create)h(new)g(pro)q(cesses,)g(and)g(is)h(most)e(lik)o(e) 74 748 y(the)d(PVM)g(routine)h Fc(pvm_spawn)p Fi(.)17 b(It)12 b(starts)d(the)j(same)f(executable)h(with)g(the)f(same)g (argumen)o(t)f(list)i(on)f(a)74 805 y(set)i(of)g(pro)q(cessors.)19 b(It)13 b(is)h(a)f(collectiv)o(e)i(op)q(eration)e(o)o(v)o(er)g(the)g (pro)q(cesses)h(in)g(the)f(group)g(asso)q(ciated)h(with)74 861 y Fb(oldcomm)p Fi(,)d(and)i(returns)g(an)g(in)o(ter-comm)o (unicator)g Fb(new)o(comm)p Fi(,)e(whic)o(h)j(has)e(the)h(new)g(pro)q (cesses)h(as)e(the)74 918 y(remote)18 b(group.)30 b(If)19 b(an)g(expanded)g(comm)o(unicator)g(is)g(desired,)h(an)f(ordinary)f(in) o(tra-comm)o(unicator)74 974 y(con)o(taining)12 b(all)f(pro)q(cesses)h (can)e(then)i(b)q(e)f(constructed)g(using)g Fb(MPI)p 1217 974 14 2 v 16 w(INTERCOMM)p 1507 974 V 17 w(MERGE)p Fi(.)g(Only)h(one)74 1030 y(of)18 b(the)g(pro)q(cesses)g(in)h Fb(oldcomm)d Fi(m)o(ust)h(supply)i(the)f(argumen)o(ts)f(that)g(describ) q(e)j(the)e(new)g(pro)q(cesses;)74 1087 y(that)d(pro)q(cess)g(is)h (designated)g(b)o(y)f Fb(ro)q(ot)p Fi(.)74 1197 y Fb(MPI)p 159 1197 V 16 w(SP)l(A)-5 b(WN\(oldcomm,)24 b(ro)q(ot,)g(a)o(rch)p 750 1197 V 16 w(t)o(yp)q(e,)i(count,)g(a)o(rra)o(y)p 1124 1197 V 15 w(of)p 1176 1197 V 16 w(names,)f(executable,)h(a)o (rgvecto)o(r,)e(\015ag,)74 1253 y(new)o(comm\))116 1340 y Fk(IN)156 b Fb(oldcomm)412 b Fk(comm)o(unicator)11 b(of)j(spa)o(wning)f(group)116 1396 y(IN)156 b Fb(ro)q(ot)507 b Fk(rank)14 b(of)f(pro)q(cess)j(supplying)d(follo)o(wing)f(argumen)o (ts)116 1453 y(IN)156 b Fb(a)o(rch)p 401 1453 V 16 w(t)o(yp)q(e)410 b Fk(arc)o(hitecture)16 b(t)o(yp)q(e)e(of)g(mac)o(hine)e(to)i(spa)o(wn) g(on)116 1509 y(IN)156 b Fb(count)481 b Fk(n)o(um)o(b)q(er)14 b(of)f(pro)q(cesses)j(to)e(spa)o(wn)g(\(in)o(t\))116 1566 y(IN)156 b Fb(a)o(rra)o(y)p 416 1566 V 14 w(of)p 467 1566 V 16 w(names)305 b Fk(arra)o(y)14 b(of)f(hostnames)h(\(arra)o (y)f(of)h(strings\))116 1622 y(IN)156 b Fb(executable)390 b Fk(executable)15 b(\014le)f(for)g(new)g(pro)q(cesses)j(to)c(execute)j (\(string\))116 1679 y(IN)156 b Fb(a)o(rgvecto)o(r)411 b Fk(argumen)o(ts)20 b(to)h(b)q(e)g(passed)h(to)f(new)g(pro)q(cesses)i (\(arra)o(y)e(of)904 1735 y(strings\))116 1792 y(IN)156 b Fb(\015ag)517 b Fk(options)14 b(\(in)o(t\))116 1848 y(OUT)109 b Fb(new)o(comm)396 b Fk(new)21 b(in)o(ter-comm)o(unicator)d (including)i(new)h(pro)q(cesses)i(as)904 1905 y(the)15 b(remote)e(group)145 2045 y Fi(The)21 b(spa)o(wned)h(pro)q(cesses)g (are)f(created)h(with)g Fb(MPI)p 1080 2045 V 15 w(COMM)p 1237 2045 V 17 w(W)o(ORLD)f Fi(consisting)i(of)e(the)g(pro-)74 2101 y(cesses)d(spa)o(wned)f(with)h(this)f(call,)i(and)e(in)h(addition) g(they)g(ha)o(v)o(e)e(a)h(prede\014ned)i(in)o(ter-comm)o(unicator)74 2158 y Fb(MPI)p 159 2158 V 16 w(COMM)p 317 2158 V 16 w(P)l(ARENT)p Fi(,)e(in)g(whic)o(h)g(the)f(remote)f(group)h(consists)g (of)g(the)g(spa)o(wning)g(pro)q(cesses.)23 b(The)74 2214 y(call)17 b(blo)q(c)o(ks)f(un)o(til)h(all)g(pro)q(cesses)f(in)g Fb(oldcomm)e Fi(ha)o(v)o(e)i(called)h Fb(MPI)p 1226 2214 V 16 w(Spa)o(wn)f Fi(and)g(all)h(the)e(spa)o(wned)h(pro-)74 2270 y(cesses)f(ha)o(v)o(e)f(called)i Fb(MPI)p 520 2270 V 16 w(INIT)p Fi(,)d(after)h(whic)o(h)h(comm)o(unication)h(is)f(p)q (ossible.)21 b(\(W)l(e)14 b(treat)g(spa)o(wning)h(of)74 2327 y(non-MPI)h(pro)q(cesses)g(b)q(elo)o(w.\))22 b(Note)15 b(that)g(this)h(means)g(that)e Fb(MPI)p 1251 2327 V 16 w(COMM)p 1409 2327 V 17 w(W)o(ORLD)h Fi(is)h(di\013eren)o(t)g(on)74 2383 y(the)c(spa)o(wners)g(and)h(spa)o(wnees.)19 b(Also)12 b(note)g(that)g(b)o(y)g(using)h Fb(MPI)p 1193 2383 V 16 w(COMM)p 1351 2383 V 16 w(SELF)p Fi(,)f(a)g(single)h(pro)q(cess)g (can)74 2440 y(create)h(man)o(y)g(others;)g(this)h(is)f(useful)i(b)q (oth)e(for)g(master-sla)o(v)o(e)f(programs)h(and)g(for)g(dynamic)h (sizing)h(of)74 2496 y(parallel)e(jobs)e(from)g(a)f(single)j(initial)g (pro)q(cess.)20 b(This)12 b(function)i(is)e(similar)i(to)d(the)i(one)f (describ)q(ed)i(in)g([8)o(],)74 2553 y(but)j(here)g(the)f(argumen)o(ts) g(are)g(made)g(explicit)j(instead)e(of)f(b)q(eing)h(com)o(bined)h(in)f (a)f(single)i(string.)23 b(In)74 2609 y(the)16 b(case)h(where)f(job-sc) o(heduler-dep)q(ende)q(n)o(t)j(information)d(m)o(ust)g(b)q(e)g (supplied,)j(w)o(e)d(do)g(use)h(a)f(string,)74 2666 y(in)k(the)g(more)f (\015exible)j Fb(MPI)p 577 2666 V 15 w(IALLOCA)l(TE)e Fi(describ)q(ed)h(in)g(Section)f(2.1.)32 b(The)19 b Fb(a)o(rra)o(y)p 1573 2666 V 15 w(of)p 1625 2666 V 16 w(names)g Fi(is)h(an)963 2790 y(9)p eop %%Page: 10 10 10 9 bop 74 7 a Fi(implemen)o(tation-de\014ned)20 b(string)e(that)f (de\014nes)h(a)f(particular)h(pro)q(cessor.)27 b(The)17 b(v)m(alue)i Fc(*)e Fi(sp)q(eci\014es)i(a)74 64 y(pro)q(cessor)c(of)g (the)g(same)g(arc)o(hitecture)h(t)o(yp)q(e)f(as)g(giv)o(en)g(b)o(y)h (the)f Fb(a)o(rch)p 1270 64 14 2 v 16 w(t)o(yp)q(e)h Fi(argumen)o(t.)145 150 y(The)d Fb(a)o(rch)p 319 150 V 16 w(t)o(yp)q(e)i Fi(is)e(an)g(implemen)o(tation-de\014ned)j(string)e (that)e(de\014nes)i(a)f(particular)h(arc)o(hitecture.)74 207 y(Tw)o(o)f(pro)q(cessors)g(ha)o(v)o(e)g(the)h(same)f(arc)o (hitecture)h(if)h(the)e(can)h(run)g(the)g(same)f(executable.)21 b(The)13 b(v)m(alue)i Fc(*)74 263 y Fi(for)g(arc)o(hitecture)i(allo)o (ws)f(an)o(y)f(arc)o(hitecture)i(\(this)f(requires)g(executables)i (that)d(can)h(execute)h(on)e(an)o(y)74 319 y(arc)o(hitecture,)h(or)g (executable)h(names)f(that)f(can)h(b)q(e)h(parameterized)f(b)o(y)g(arc) o(hitecture\).)23 b(This)16 b(string)74 376 y(is)g(signi\014can)o(t)g (only)g(if)f Fb(a)o(rra)o(y)p 575 376 V 15 w(of)p 627 376 V 16 w(names)g Fi(con)o(tains)g(one)h(or)e(more)h(en)o(tries)h (with)f(the)h(v)m(alue)g Fc(*)p Fi(.)215 476 y Fl(Discussion:)i Fk(The)d(use)h(of)e(implemen)o(tation-)o(dep)q(enden)o(t)g(strings)h (ma)o(y)e(w)o(orry)i(those)h(in)o(terested)145 526 y(in)c(p)q (ortabilit)o(y)m(.)j(W)m(e)d(b)q(eliev)o(e)h(that,)f(since)h(these)h (strings)f(are)g(used)g(to)f(in)o(terface)h(to)f(the)h(outside)g(w)o (orld,)145 576 y(they)g(can)g(not)f(b)q(e)h(completely)f(de\014ned.)19 b(Filenames)11 b(pro)o(vide)h(an)h(example)e(of)h(a)g(non-p)q(ortable)h (string;)145 626 y(users)20 b(are)f(used)h(to)f(ha)o(ving)f(to)g(c)o (hange)i(\014lename)d(when)j(mo)o(ving)c(from)h(system)i(to)f(system;)j (these)145 676 y(strings)14 b(are)g(really)g(no)f(di\013eren)o(t.)145 783 y Fi(F)l(or)h(clien)o(t-serv)o(er)j(applications,)f(w)o(e)f(prop)q (ose)g(the)h(follo)o(wing)f(as)g(the)h(simplest,)f(blo)q(c)o(king)i (calls.)74 893 y Fb(MPI)p 159 893 V 16 w(CLIENT)p 332 893 V 16 w(CONNECT\(mycomm,)c(name,)h(new)o(comm\))116 982 y Fk(IN)156 b Fb(mycomm)411 b Fk(comm)o(unicator)10 b(of)h(the)i(clien)o(t,)f(o)o(v)o(er)h(whic)o(h)f(this)g(call)f(is)i (col-)904 1039 y(lectiv)o(e)116 1095 y(IN)156 b Fb(name)484 b Fk(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)h(whic)o(h)f(the)i(serv)o(er)g (can)f(b)q(e)h(con)o(tacted)904 1152 y(\(string\))116 1208 y(OUT)109 b Fb(new)o(comm)396 b Fk(new)10 b(in)o(ter-comm)o (unicator,)d(whic)o(h)i(includes)h(the)g(serv)o(er)h(pro-)904 1265 y(cesses)17 b(as)d(the)g(remote)g(group)74 1431 y Fb(MPI)p 159 1431 V 16 w(SERVER)p 342 1431 V 18 w(CONNECT\(mycomm,)f (name,)h(new)o(comm\))116 1520 y Fk(IN)156 b Fb(mycomm)411 b Fk(comm)o(unicator)18 b(of)i(the)h(serv)o(er,)j(o)o(v)o(er)c(whic)o (h)h(this)f(call)g(is)904 1577 y(collectiv)o(e)116 1633 y(IN)156 b Fb(name)484 b Fk(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)h(whic)o (h)f(the)i(serv)o(er)g(can)f(b)q(e)h(con)o(tacted)904 1690 y(\(string\))116 1746 y(OUT)109 b Fb(new)o(comm)396 b Fk(new)10 b(in)o(ter-comm)o(unicator,)d(whic)o(h)i(includes)h(the)g (clien)o(t)f(pro-)904 1803 y(cesses)17 b(as)d(the)g(remote)g(group)145 1945 y Fi(Disconnection)f(o)q(ccurs)f(when)h(pro)q(cesses)f(call)h Fb(MPI)p 1045 1945 V 16 w(COMM)p 1203 1945 V 16 w(FREE)g Fi(on)e(the)h(in)o(ter-comm)o(unicator.)145 2032 y(Then)17 b(an)o(y)g(pro)q(cess)h(in)g(the)f(clien)o(t)i(group)d(can)i(comm)o (unicate)f(with)h(an)o(y)f(pro)q(cess)g(in)h(the)f(serv)o(er)74 2088 y(group)e(and)g(vice)i(v)o(ersa,)d(using)i(the)f(in)o(ter-comm)o (unicator.)145 2175 y(The)j(form)g(of)g(the)h Fb(name)f Fi(argumen)o(t)f(has)i(sev)o(eral)f(p)q(ossibilitie)q(s.)32 b(The)19 b(most)e(ob)o(vious)i(is)g(to)f(use)74 2231 y(the)13 b Fc(net-address:port-number)d Fi(format)h(that)h(curren)o(t)h (systems)g(will)h(\014nd)g(most)e(straigh)o(tforw)o(ard.)74 2288 y(Ho)o(w)o(ev)o(er,)i(in)i(the)f(long)h(run,)f(name)g(serv)o(ers)g (of)g(v)m(arious)g(kinds)h(ma)o(y)f(require)h(more)f(\015exibilit)o(y)l (.)74 2437 y Fd(3.2.3)52 b(New)17 b(T)o(yp)q(es)g(of)g Fc(MPI)p 643 2437 15 2 v 17 w(Request)p Fd(s)74 2553 y Fi(In)i(the)f(follo)o(wing)g(sections,)h(w)o(e)e(will)j(in)o(tro)q (duce)f(sev)o(eral)f(non-blo)q(c)o(king)h(op)q(erations.)28 b(Rather)18 b(than)74 2609 y(in)o(tro)q(ducing)c(new)e(v)o(ersions)g (of)g Fb(MPI)p 707 2609 14 2 v 15 w(W)l(AIT)p Fi(,)g Fb(MPI)p 942 2609 V 16 w(TEST)p Fi(,)g(etc.,)g(w)o(e)f(prop)q(ose)h(to) g(use)g(the)g(existing,)i(ric)o(h)74 2666 y(set)k(of)g(MPI)g (functions.)29 b(In)18 b(order)g(to)g(allo)o(w)g(this,)h(w)o(e)f(m)o (ust)f(allo)o(w)h(an)g Fb(MPI)p 1453 2666 V 16 w(Request)j Fi(to)c(represen)o(t)951 2790 y(10)p eop %%Page: 11 11 11 10 bop 74 7 a Fi(some)16 b(new)h(kinds)g(of)f(requests.)23 b(A)16 b(general)h(mec)o(hanism)g(for)e(user)i(de\014nition)h(of)d (request)i(t)o(yp)q(es)f(w)o(as)74 64 y(describ)q(ed)22 b(in)g([3)o(],)f(and)f(our)h(prop)q(osal)f(here)h(is)g(quite)g(similar) h(although)e(not)g(iden)o(tical.)37 b(Because)74 120 y(an)19 b(implemen)o(tation)h(of)e(extensions)h(to)f Fb(MPI)p 890 120 14 2 v 16 w(Request)j Fi(pro)o(vides)e(a)f(consisten)o (t)h(w)o(a)o(y)f(to)g(implemen)o(t)74 177 y(the)d(functions)g(that)e(w) o(e)h(prop)q(ose,)h(it)f(is)h(useful)g(to)f(pro)o(vide)h(a)f(standard)g (extension)h(to)f(MPI)g(for)g(user-)74 233 y(de\014ned)g(request)e(t)o (yp)q(es.)19 b(Ho)o(w)o(ev)o(er,)12 b(since)i(w)o(e)e(do)g(not)g(prop)q (ose)h(here)g(to)f(use)h(this)g(general)f(mec)o(hanism)74 289 y(for)h(creating)h(and)g(starting)f(the)h(requests)g(used)g(here,)h (w)o(e)e(relegate)h(this)g(prop)q(osal)g(to)f(the)h(App)q(endix.)74 437 y Fd(3.2.4)52 b(In)o(terfacing)18 b(to)g(the)f(Job)h(Sc)o(heduler) 74 553 y Fi(There)g(are)g(at)f(least)h(t)o(w)o(o)f(reasons)g(wh)o(y)h (the)g(high-lev)o(el,)j(simple)e(in)o(terface)f(is)h(not)e(enough.)28 b(In)19 b(the)74 610 y(\014rst)c(place,)h(these)g(op)q(erations)g(are)f (liable)i(to)e(b)q(e)h(exp)q(ensiv)o(e,)h(and)e(so)g(it)h(w)o(ould)g(b) q(e)g(nice)h(not)e(to)g(ha)o(v)o(e)74 666 y(to)e(blo)q(c)o(k)h(w)o (aiting)g(for)f(their)h(completion)h(while)g(other)e(useful)i(w)o(ork)d (could)j(b)q(e)f(done.)20 b(Secondly)l(,)15 b(they)74 722 y(encapsulate)e(to)q(o)d(m)o(uc)o(h,)i(com)o(bining)h (unnecessarily)h(in)o(teractions)d(with)h(b)q(oth)g(the)f(sc)o(heduler) i(and)f(the)74 779 y(pro)q(cess)17 b(manager.)22 b(In)17 b(the)g(next)f(few)h(sections)f(w)o(e)h(break)f(these)g(higher-lev)o (el)j(functions)e(in)o(to)g(their)74 835 y(comp)q(onen)o(t)c(parts)f (for)g(greater)g(con)o(trol,)g(and)h(mak)o(e)f(the)h(comp)q(onen)o(ts)g (non-blo)q(c)o(king.)21 b(Non-blo)q(c)o(king)74 892 y(op)q(erations)15 b(in)i(general)e(return)g Fb(MPI)p 729 892 V 16 w(Request)j Fi(ob)s(jects.)145 978 y(Here)13 b(w)o(e)f(giv)o(e)h(a)g(set)f(of)g (functions)i(for)e(in)o(terfacing)i(directly)g(to)e(the)h(job)f(sc)o (heduler.)21 b(The)13 b(\014rst)f(one)74 1035 y(can)k(b)q(e)g(used)h (to)e(obtain)h(the)f(hostnames)h(used)g(in)g(a)g(call)g(to)g Fb(MPI)p 1246 1035 V 15 w(SP)l(A)-5 b(WN)p Fi(.)16 b(Since)i(in)o (teraction)e(with)74 1091 y(the)j(job)g(sc)o(heduler)i(can)e(b)q(e)g (time-consuming,)i(w)o(e)e(mak)o(e)g(this)g(a)g(non-blo)q(c)o(king)h (op)q(eration,)g(whic)o(h)74 1148 y(returns)14 b(an)g(arra)o(y)f(of)g (requests)h(to)f(b)q(e)i(w)o(aited)f(on)g(later.)19 b(In)c(k)o(eeping)g (with)f(other)f(MPI)h(non-blo)q(c)o(king)74 1204 y(op)q(erations,)h(w)o (e)g(call)h(it)g Fb(IALLOCA)l(TE)f Fi(instead)h(of)f Fb(IALLOCA)l(TE)p Fi(.)215 1305 y Fl(Discussion:)f Fk(In)c(general,)h (this)f(function)g(do)q(es)h(not)g(start)f(an)o(y)g(new)h(pro)q (cesses;)j(rather,)d(it)f(obtains)145 1354 y(resources)17 b(from)c(the)i(job)f(sc)o(heduler)j(for)d(use)i(b)o(y)e(other)i (functions.)k(Ho)o(w)o(ev)o(er,)15 b(w)o(e)g(need)h(to)e(tak)o(e)h(in)o (to)145 1404 y(accoun)o(t)h(those)h(job)f(sc)o(hedulers)i(that)e (cannot)g(pro)o(vide)g(this)h(function)e(without)h(starting)g(pro)q (cesses,)145 1454 y(suc)o(h)c(as)g(LoadLev)o(eler)g(or)g(DQS.)g(In)f (those)i(cases,)g(the)g(executables)g(ma)o(y)d(not)i(b)q(e)g(the)h (application)d(exe-)145 1504 y(cutables,)i(but)h(rather)g(in)o(terface) f(pro)q(cesses)j(that)d(will)f(create)i(the)g(application)d(pro)q (cesses)15 b(in)d(resp)q(onse)145 1554 y(to)h(one)i(of)e(the)h(pro)q (cess-creation)i(functions)e(describ)q(ed)i(here)f(\(lik)o(e)e Fa(MPI)p 1316 1554 13 2 v 15 w(Spa)o(wn)p Fk(\).)145 1661 y Fi(The)i(follo)o(wing)h(function)g(is)g(for)e(resource)i(allo)q (cation)g(only;)f(it)h(do)q(es)f(not)g(start)f(an)o(y)h(pro)q(cesses.) 74 1771 y Fb(MPI)p 159 1771 14 2 v 16 w(IALLOCA)l(TE\(num)p 515 1771 V 16 w(requested,)20 b(global)p 855 1771 V 16 w(js)p 900 1771 V 17 w(dep)p 983 1771 V 17 w(string,)e(a)o(rra)o(y)p 1232 1771 V 15 w(of)p 1284 1771 V 16 w(lo)q(cal)p 1388 1771 V 16 w(js)p 1433 1771 V 17 w(dep)p 1516 1771 V 17 w(strings,)g(a)o(rch)p 1767 1771 V 16 w(t)o(yp)q(e,)74 1827 y(a)o(rra)o(y)p 172 1827 V 15 w(of)p 224 1827 V 16 w(no)q(denames,)e(ha)o(rdness,)g(a)o(rra)o(y)p 762 1827 V 15 w(of)p 814 1827 V 16 w(requests\))116 1909 y Fk(IN)156 b Fb(num)p 404 1909 V 16 w(requested)309 b Fk(n)o(um)o(b)q(er)14 b(of)f(hosts)h(requested)116 1966 y(IN)156 b Fb(global)p 434 1966 V 15 w(js)p 478 1966 V 17 w(dep)p 561 1966 V 18 w(string)222 b Fk(sp)q(ecial)15 b(information)10 b(parsed)15 b(b)o(y)f(job)f(sc)o(heduler)116 2022 y(IN)156 b Fb(a)o(rra)o(y)p 416 2022 V 14 w(of)p 467 2022 V 16 w(lo)q(cal)p 571 2022 V 16 w(js)p 616 2022 V 17 w(dep)p 699 2022 V 18 w(strings)67 b Fk(sp)q(ecial)13 b(information)d(parsed)k(b)o(y)f(job)f(sc)o(heduler,)i(on)f(a)f(p)q (er-)904 2079 y(pro)q(cess)k(basis)116 2135 y(IN)156 b Fb(a)o(rch)p 401 2135 V 16 w(t)o(yp)q(e)410 b Fk(arc)o(hitecture)16 b(t)o(yp)q(e)e(of)g(mac)o(hine)e(to)i(spa)o(wn)g(on)116 2192 y(IN)156 b Fb(a)o(rra)o(y)p 416 2192 V 14 w(of)p 467 2192 V 16 w(no)q(denames)215 b Fk(arra)o(y)14 b(of)f(hostnames)h (\(arra)o(y)f(of)h(strings\))116 2248 y(IN)156 b Fb(ha)o(rdness)426 b Fk(whether)16 b(the)e(request)i(is)d(hard)h(or)g(soft)g(\(in)o (teger\))116 2304 y(OUT)109 b Fb(a)o(rra)o(y)p 416 2304 V 14 w(of)p 467 2304 V 16 w(requests)272 b Fk(set)15 b(of)e(requests)145 2440 y Fi(Authen)o(tication)21 b(information,)h(if) f(required)g(b)o(y)g(the)g(job)f(sc)o(heduler,)j(can)e(b)q(e)g (supplied)i(in)e(the)74 2496 y(job-sc)o(heduler-dep)q(ende)q(n)o(t)e (string.)26 b(The)18 b(arra)o(y)e(of)g(no)q(denames)i(ma)o(y)f(con)o (tain)g(wildcard)h(indicators)74 2553 y(\()p Fb(MPI)p 177 2553 V 16 w(ANYWHERE)p Fi(\))h(to)f(allo)o(w)h(the)f(job)h(sc)o (heduler)g(to)f(pic)o(k)h(the)g(pro)q(cessors)f(to)g(b)q(e)h(used.)30 b(A)19 b Fe(har)n(d)74 2609 y Fi(allo)q(cation)13 b(request)e(is)h (required)h(to)e(ev)o(en)o(tually)i(return)e(the)h(en)o(tire)g(n)o(um)o (b)q(er)g(of)f(pro)q(cessors)g(requested,)74 2666 y(whereas)16 b(a)f Fe(soft)20 b Fi(allo)q(cation)d(request)f(ma)o(y)f(complete)i (when)f(it)g(can)g(allo)q(cate)h(some)e(pro)q(cessors)h(ev)o(en)951 2790 y(11)p eop %%Page: 12 12 12 11 bop 74 7 a Fi(if)15 b(it)f(kno)o(ws)f(that)h(it)g(will)h(not)f(b) q(e)h(able)f(to)g(satisfy)f(the)h(en)o(tire)h(request.)k(The)c (executable)g(\014les)g(are)f(not)74 64 y(sp)q(eci\014ed)19 b(on)d(this)h(call,)h(since)f(this)g(is)g(just)g(for)e(resource)i(allo) q(cation.)25 b(W)l(e)17 b(can)f(attac)o(h)g(them)g(to)g(the)74 120 y(requests)f(with)h(the)f(function)h(describ)q(ed)h(in)f(the)g (next)f(section.)145 207 y(The)g(strings)h(con)o(taining)g(job)f(sc)o (heduler)i(dep)q(enden)o(t)g(strings)f(are)f(implemen)o(tation)i (de\014ned)g(\(b)o(y)74 263 y(the)d(job)h(sc)o(heduler,)g(not)f (MPI!\).)19 b(These)14 b(ma)o(y)g(con)o(tain)g(information)g(to)g(b)q (e)h(applied)h(to)e(all)h(pro)q(cesses)74 319 y(\(in)c Fb(global)p 256 319 14 2 v 16 w(js)p 301 319 V 17 w(dep)p 384 319 V 17 w(string)p Fi(\))f(and)h(information)f(for)f(a)h (particular)h(pro)q(cess)g(\(in)f Fb(a)o(rra)o(y)p 1476 319 V 15 w(of)p 1528 319 V 16 w(lo)q(cal)p 1632 319 V 16 w(js)p 1677 319 V 17 w(dep)p 1760 319 V 17 w(strings)p Fi(\).)74 376 y(An)h(example)g(of)e Fb(global)p 479 376 V 16 w(js)p 524 376 V 17 w(dep)p 607 376 V 18 w(string)h Fi(is)h Fc(gang_schedule)p Fi(;)e(and)i(example)g(of)e Fb(a)o(rra)o(y)p 1514 376 V 15 w(of)p 1566 376 V 16 w(lo)q(cal)p 1670 376 V 16 w(js)p 1715 376 V 17 w(dep)p 1798 376 V 18 w(strings)74 432 y Fi(is)146 556 y Fc({)23 b("large_memory",)193 613 y("local_hippi")g(})145 736 y Fi(The)10 b(argumen)o(ts)f Fb(a)o(rch)p 532 736 V 17 w(t)o(yp)q(e)i Fi(and)f Fb(a)o(rra)o(y)p 818 736 V 15 w(of)p 870 736 V 16 w(hostnames)h Fi(ha)o(v)o(e)f(the)g (same)g(meaning)g(as)g(for)g Fb(MPI)p 1751 736 V 15 w(Spa)o(wn)p Fi(.)74 887 y Fd(3.2.5)52 b(In)o(terfacing)18 b(to)g(the)f(Pro)q(cess)g (Manager)74 1002 y Fi(While)h Fb(MPI)p 293 1002 V 15 w(SP)l(A)-5 b(WN)17 b Fi(con)o(v)o(enien)o(tly)h(captures)e(man)o(y)f (asp)q(ects)i(of)e(resource)h(allo)q(cation)i(and)e(pro)q(cess)74 1059 y(startup)c(in)h(one)f(call,)i(w)o(e)e(need)h(more)f(detailed)i (con)o(trol)e(of)g(these)h(steps.)18 b(Our)13 b(mo)q(del)g(will)h(b)q (e)f(that)f(one)74 1115 y(calls)17 b Fb(MPI)p 261 1115 V 15 w(IALLOCA)l(TE)f Fi(to)f(reserv)o(e)g(pro)q(cessors,)g(getting)g (bac)o(k)g(a)g(set)g(of)g(requests.)20 b(These)c(requests)74 1172 y(can)g(b)q(e)h(further)f(mo)q(di\014ed)i(with)e(functions)h(w)o (e)f(presen)o(t)g(in)h(this)g(section,)g(and)f(then)h(actual)f(pro)q (cess)74 1228 y(startup)f(can)g(b)q(e)h(accomplished)h(with)e(the)h (existing)g Fb(MPI)p 1092 1228 V 16 w(ST)l(ART)g Fi(or)e Fb(MPI)p 1403 1228 V 16 w(ST)l(ART)l(ALL)i Fi(calls.)145 1315 y(First,)h(there)h(are)g(routines)g(for)f(setting)h(attributes)f (of)h(requests)g(that)f(ma)o(y)g(not)g(ha)o(v)o(e)g(b)q(een)i(set)74 1371 y(with)d Fb(MPI)p 263 1371 V 15 w(IALLOCA)l(TE)p Fi(.)74 1481 y Fb(MPI)p 159 1481 V 16 w(SET)p 258 1481 V 17 w(EXEC\(request,)g(executable\))116 1575 y Fk(IN)156 b Fb(request)451 b Fk(request)16 b(\(handle\))116 1631 y(IN)156 b Fb(executable)390 b Fk(name)13 b(of)g(executable)i(\014le)74 1779 y Fi(sets)g(the)g(name)h(of)e(the)i(\014le)g(to)e(b)q(e)i (executed,)g(and)74 1889 y Fb(MPI)p 159 1889 V 16 w(SET)p 258 1889 V 17 w(ARGS\(request,)h(a)o(rgs\))116 1983 y Fk(IN)156 b Fb(request)451 b Fk(request)16 b(\(handle\))116 2039 y(IN)156 b Fb(a)o(rgs)509 b Fk(arra)o(y)14 b(of)f(strings)74 2187 y Fi(sets)i(the)g(command-line)i(argumen)o(ts)e(that)f(pro)q(cess) i(will)g(receiv)o(e.)215 2287 y Fl(Discussion:)d Fk(These)e(same)e(op)q (erations)g(p)q(erhaps)i(should)e(also)f(b)q(e)i(de\014ned)h(for)e (arra)o(ys)g(of)g(requests.)145 2337 y(On)14 b(the)g(other)h(hand,)e (they)i(are)f(not)g(exp)q(ensiv)o(e)h(op)q(erations.)215 2458 y Fl(Discussion:)j Fk(F)m(ollo)o(wing)13 b(the)i(reasoning)g(for)g Fa(MPI)p 1055 2458 13 2 v 14 w(PUT)p 1153 2458 V 15 w(A)m(TTR)p Fk(,)f(the)h Fa(request)i Fk(is)e(considered)i(an)145 2508 y(IN)d(argumen)o(t)e(instead)j(of)e(an)h(INOUT)g(argumen)o(t.)145 2615 y Fi(W)l(e)h(ma)o(y)f(also)i(need)g(functions)g(to)e(extract)h (attributes)g(of)f(requests,)h(lik)o(e)951 2790 y(12)p eop %%Page: 13 13 13 12 bop 74 7 a Fb(MPI)p 159 7 14 2 v 16 w(GET)p 263 7 V 17 w(NODENAME\(request,)17 b(hostname\))116 101 y Fk(IN)156 b Fb(request)451 b Fk(request)16 b(\(handle\))116 157 y(OUT)109 b Fb(hostname)405 b Fk(the)15 b(name)e(of)g(the)h(host)h (asso)q(ciated)f(with)g(the)g(request)74 305 y Fi(to)f(retriev)o(e)g (the)g(hostname)g(that)g(w)o(as)f(\014lled)j(in)f(b)o(y)f(the)h(job)f (sc)o(heduler)h(for)f(a)g(particular)h(request,)f(and)74 415 y Fb(MPI)p 159 415 V 16 w(GET)p 263 415 V 17 w(JSINF)o(O\(request,) k(js)p 641 415 V 16 w(dep)p 723 415 V 18 w(info\))116 509 y Fk(IN)156 b Fb(request)451 b Fk(request)16 b(\(handle\))116 565 y(OUT)109 b Fb(js)p 350 565 V 17 w(dep)p 433 565 V 17 w(info)386 b Fk(job-sc)o(heduler-dep)q(enden)o(t)17 b(information)74 713 y Fi(to)e(retriev)o(e)g(sp)q(ecial)i(information)e (dep)q(enden)o(t)i(on)e(the)h(job)f(sc)o(heduler)h(b)q(eing)h(used.)145 799 y(Once)d(the)g(requests)f(ha)o(v)o(e)h(b)q(een)g(set)g(up,)g(they)f (can)h(b)q(e)g(initiated)h(with)f(the)g(existing)g Fb(MPI)p 1718 799 V 16 w(ST)l(ART)74 856 y Fi(or)h Fb(MPI)p 215 856 V 16 w(ST)l(ART)l(ALL)h Fi(routines,)145 942 y(Pro)q(cess)i (requests)h(ha)o(v)o(e)f Fe(two)k Fi(stages,)d(and)f(w)o(e)h(w)o(ait)f (on)h(b)q(oth.)30 b(The)19 b(\014rst)f(stage)g(is)h(completed)74 998 y(when)13 b(the)g(pro)q(cess)g(has)f(b)q(een)i(started.)k(The)13 b(second)h(stage)d(is)j(completed)f(when)g(the)g(pro)q(cesses)g(exits.) 74 1055 y(\(W)l(e)i(can)g(think)h(of)e(the)h(second)h(stage)e(as)g(the) h(MPI)g(in)o(terface)h(to)e(the)h(Pro)q(cess)g(Manager's)e(handling)74 1111 y(of)f(the)g(signal)g Fc(SIGCHILD)p Fi(\).)e(The)i(status)f (returned)i(b)o(y)e(the)h(usual)h(w)o(ait)e(and)h(test)g(routines)g (can)g(b)q(e)h(used)74 1168 y(to)i(determine)i(whic)o(h)f(state)f(has)g (b)q(een)i(completed.)22 b(Note)15 b(that)g(this)h(lev)o(el)h(of)e(pro) q(cess)h(managemen)o(t)74 1224 y(allo)o(ws)j(us)f(to)f(manage)h (non-MPI)h(pro)q(cesses,)g(since)g(comm)o(unicators)f(are)g(not)g(in)o (v)o(olv)o(ed.)30 b(In)18 b(order)74 1281 y(to)f(get)f(the)i(status,)e (w)o(e)h(need)h(a)f(routine)g(to)g(extract)f(the)h(v)m(alue)i(from)d (the)h Fb(status)i Fi(argumen)o(t)e(to)f(the)74 1337 y Fb(MPI)p 159 1337 V 16 w(W)l(AIT)f Fi(call.)74 1447 y Fb(MPI)p 159 1447 V 16 w(GET)p 263 1447 V 17 w(RETURN)p 459 1447 V 17 w(CODE\(request,)i(co)q(de\))116 1541 y Fk(IN)156 b Fb(request)451 b Fk(request)16 b(\(handle\))116 1598 y(OUT)109 b Fb(co)q(de)499 b Fk(exit)14 b(status)h(of)e(pro)q (cess)74 1745 y Fi(to)i(retriev)o(e)g(the)g(exit)h(status)e(\(from)h(a) f Fc(return)23 b(n)15 b Fi(or)g Fc(exit\(n\))f Fi(in)i(C)f(or)g Fc(STOP)23 b(n)15 b Fi(in)h(F)l(ortran\).)145 1831 y(If)d(the)g (started)f(pro)q(cesses)i(are)e(MPI)h(pro)q(cesses)g(\(that)f(is,)i(if) f(they)g(call)h Fb(MPI)p 1456 1831 V 16 w(INIT)p Fi(\),)e(then)h(the)g (com-)74 1888 y(m)o(unicator)j(that)f(includes)j(them)d(can)h(b)q(e)h (constructed)e(with)h(the)g Fb(MPI)p 1341 1888 V 16 w(COMM)p 1499 1888 V 16 w(P)l(ARENT)p 1689 1888 V 18 w(CREA)l(TE)74 1944 y Fi(function)22 b(describ)q(ed)h(b)q(elo)o(w.)37 b(In)21 b(either)h(case,)g(at)e(this)h(stage)f(w)o(e)h(ha)o(v)o(e)f (allo)q(cated)i(resources)f(and)74 2001 y(the)c(pro)q(cesses)h(ha)o(v)o (e)f(b)q(een)h(started.)25 b(The)18 b(third)g(comp)q(onen)o(t)f(of)g (the)g(run)o(time)h(en)o(vironmen)o(t)f(is)h(the)74 2057 y(message-passing)f(library)l(.)25 b(In)17 b(order)f(to)g(comm)o (unicate)h(with)g(these)g(pro)q(cesses,)g(w)o(e)f(m)o(ust)g(in)o (terface)74 2114 y(with)g(the)f(message-passing)g(library)l(.)21 b(In)16 b(MPI)f(terms,)f(this)i(means)f(setting)g(up)h(comm)o (unicators.)145 2200 y(Once)h(the)f(pro)q(cesses)h(ha)o(v)o(e)f(b)q (een)i(started,)d(w)o(e)h(ma)o(y)g(or)g(ma)o(y)f(not)h(wish)h(to)f (establish)h(comm)o(uni-)74 2257 y(cation)d(with)g(them.)19 b(Note)13 b(that)g(the)h Fb(MPI)p 811 2257 V 16 w(IALLOCA)l(TE)p Fi(/)p Fb(MPI)p 1171 2257 V 16 w(ST)l(ART)p Fi(/)p Fb(MPI)p 1434 2257 V 16 w(W)l(AIT)g Fi(mec)o(hanism)g(can)74 2313 y(b)q(e)g(used)g(b)o(y)f(a)g(pro)q(cess)h(manager)e(written)i(in)g(MPI) f(to)g(manage)g(non-MPI)g(pro)q(cesses)h(as)f(w)o(ell)h(as)f(MPI)74 2370 y(pro)q(cesses.)28 b(If)18 b(the)g(application)i(program)d(is)h (creating)g(new)g(pro)q(cesses,)h(ho)o(w)o(ev)o(er,)e(it)h(is)g(lik)o (ely)i(that)74 2426 y(it)f(will)g(w)o(an)o(t)e(to)h(comm)o(unicate)g (with)h(them,)f(via)h(an)f(in)o(ter-comm)o(unicator.)29 b(The)18 b(complication)i(is)74 2482 y(that)12 b(creation)g(of)g(this)h (in)o(ter-comm)o(unicator)g(is)f(a)g(collectiv)o(e)j(op)q(eration)d(o)o (v)o(er)g(all)h(pro)q(cesses)g(in)o(v)o(olv)o(ed,)74 2539 y(y)o(et)i(w)o(e)g(don't)f(w)o(an)o(t)g(the)h(spa)o(wning)h(pro)q (cess\(es\))f(to)f(blo)q(c)o(k)i(if)f(the)h(spa)o(wned)f(pro)q(cesses)g (are)g(not)g(ev)o(en)74 2595 y(going)j(to)f(call)i Fb(MPI)p 427 2595 V 16 w(INIT)p Fi(.)e(Our)h(solution)h(is)g(to)e(create)h(a)f (\\stub")h(in)o(ter-comm)o(unicator)g(whic)o(h)g(will)74 2652 y(b)q(ecome)e(v)m(alid)h(if)f(and)f(when)h(the)f(spa)o(wned)g(pro) q(cesses)h(call)g Fb(MPI)p 1222 2652 V 16 w(INIT)p Fi(.)951 2790 y(13)p eop %%Page: 14 14 14 13 bop 74 7 a Fb(MPI)p 159 7 14 2 v 16 w(COMM)p 317 7 V 16 w(P)l(ARENT)p 507 7 V 18 w(CREA)l(TE\(lo)q(calcomm,)10 b(num)p 1023 7 V 17 w(requests,)j(a)o(rra)o(y)p 1313 7 V 15 w(of)p 1365 7 V 16 w(requests,)g(intercomm\))116 99 y Fk(IN)156 b Fb(lo)q(calcomm)381 b Fk(comm)o(unicator)11 b(of)j(the)g(spa)o(wning)g(pro)q(cesses)116 156 y(IN)156 b Fb(num)p 404 156 V 16 w(requests)335 b Fk(n)o(um)o(b)q(er)14 b(of)f(requests)j(in)d(arra)o(y)116 212 y(IN)156 b Fb(a)o(rra)o(y)p 416 212 V 14 w(of)p 467 212 V 16 w(requests)272 b Fk(requests)16 b(represen)o(ting)g(pro)q(cesses)g(to)e(b)q(e)h(created)116 269 y(OUT)109 b Fb(intercomm)383 b Fk(new)10 b(in)o(ter-comm)o (unicator,)d(whic)o(h)i(ma)o(y)f(b)q(ecome)h Fa(MPI)p 1783 269 13 2 v 14 w(COMM)p 1927 269 V 15 w(P)m(ARENT)904 325 y Fi(in)16 b(the)g(created)f(pro)q(cesses.)145 471 y(After)9 b Fb(MPI)p 343 471 14 2 v 16 w(W)l(AIT)l(ALL)h Fi(indicates)i(that)d(all)i(of)f(these)g(pro)q(cesses)h(are)f(running,) i(the)e(in)o(ter-comm)o(unicator)74 527 y(is)24 b(v)m(alid)g(on)f(the)h (spa)o(wning)f(side)h(and)f(the)g(spa)o(wned)h(pro)q(cesses)f(ha)o(v)o (e)g(the)g(in)o(ter-comm)o(unicator)74 584 y Fb(MPI)p 159 584 V 16 w(COMM)p 317 584 V 16 w(P)l(ARENT)17 b Fi(de\014ned,)f (along)f(with)h(the)f(usual)h Fb(MPI)p 1197 584 V 16 w(COMM)p 1355 584 V 16 w(W)o(ORLD)p Fi(.)145 670 y(It)h(is)g(erroneous) g(to)f(use)h Fb(MPI)p 672 670 V 16 w(COMM)p 830 670 V 16 w(P)l(ARENT)p 1020 670 V 18 w(CREA)l(TE)h Fi(with)f(pro)q(cesses)g (that)f(are)h(not)f(MPI)74 726 y(jobs.)215 827 y Fl(Discussion:)25 b Fk(An)19 b(alternativ)o(e)g(w)o(ould)f(b)q(e)h(to)g(ha)o(v)o(e)f(the) i(in)o(ter-comm)o(unicator)c(returned)k(as)f(a)145 877 y(\014eld)13 b(in)f(the)i(status)g(ob)r(ject)g(when)f(the)h(requests)h (ha)o(v)o(e)e(reac)o(hed)h(the)f(state)h(where)g(the)g(pro)q(cesses)i (ha)o(v)o(e)145 927 y(b)q(een)j(started.)34 b(This)18 b(eliminates)f(the)j(need)f(for)f Fa(MPI)p 1042 927 13 2 v 15 w(COMM)p 1187 927 V 14 w(P)m(ARENT)p 1362 927 V 14 w(CREA)m(TE)f Fk(at)i(the)g(cost)g(of)145 976 y(complexifying)11 b(the)k(status)g(ob)r(ject,)f(whic)o(h)g(is)g(a)g(user-visible)g (structure.)21 b(It)14 b(also)f(do)q(es)i(not)f(solv)o(e)g(the)145 1026 y(problem)h(of)g(sp)q(ecifying)h(the)h(in)o(ten)o(t)f(of)f(the)i (spa)o(wned)g(pro)q(cesses)i(|)c(are)i(they)f(MPI)h(pro)q(cesses)h (that)145 1076 y(will)c(b)q(e)i(comm)o(unicated)e(with)h(using)h(MPI)g (calls)f(or)h(are)g(they)g(stand-alone)g(programs)f(\(either)h(MPI)145 1126 y(programs)c(or)i(non-MPI)g(programs\).)145 1233 y Fi(W)l(e)h(will)j(also)d(need)i(to)e(ask)g(the)h(pro)q(cess)g (manager)f(to)g(deliv)o(er)i(signals)f(to)f(pro)q(cesses,)h(whic)o(h)g (are)74 1290 y(represen)o(ted)g(b)o(y)f(requests:)74 1400 y Fb(MPI)p 159 1400 14 2 v 16 w(SIGNAL\(signal,)g(num)p 566 1400 V 16 w(requests,)i(a)o(rra)o(y)p 859 1400 V 15 w(of)p 911 1400 V 16 w(requests\))116 1492 y Fk(IN)156 b Fb(signal)479 b Fk(signal)13 b(t)o(yp)q(e)i(\(in)o(t\))116 1548 y(IN)156 b Fb(num)p 404 1548 V 16 w(requests)335 b Fk(n)o(um)o(b)q(er)14 b(of)f(requests)j(in)d(arra)o(y)116 1605 y(IN)156 b Fb(a)o(rra)o(y)p 416 1605 V 14 w(of)p 467 1605 V 16 w(requests)272 b Fk(requests)16 b(represen)o(ting)g(pro)q (cesses)g(to)e(b)q(e)h(signalled)145 1750 y Fi(It)d(is)g(the)g(resp)q (onsibilit)o(y)i(of)d(an)h(implemen)o(tation)h(to)e(translate)h(b)q(et) o(w)o(een)g(signals;)h(in)g(other)e(w)o(ords,)74 1807 y(a)j Fc(SIGINT)g Fi(that)g(has)h(v)m(alue)g Fc(3)g Fi(on)f(system)h(A) f(m)o(ust)g(b)q(e)i(deliv)o(ered)g(as)e(a)h Fc(SIGINT)e Fi(on)i(system)f(b,)h(ev)o(en)g(if)74 1863 y Fc(SIGINT)f Fi(on)h(system)f(b)h(uses)g(the)g(v)m(alue)h Fc(5)e Fi(for)g Fc(SIGINT)p Fi(.)g(If)h(the)f(signal)i(can)f(not)f(b)q(e)i(deliv)o (ered)g(b)q(ecause)74 1920 y(there)f(is)h(no)f(corresp)q(onding)h (signal,)g(the)f(error)g(co)q(de)h(is)f Fb(MPI)p 1158 1920 V 16 w(ERR)p 1259 1920 V 17 w(INV)l(ALID)p 1448 1920 V 17 w(SIGNAL)p Fi(.)215 2020 y Fl(Discussion:)39 b Fk(Should)15 b(there)h(b)q(e)g(a)f(single-request)h(v)o(ersion)g(of)e (this,)h(and)g(the)h(ab)q(o)o(v)o(e)f(renamed)145 2070 y Fa(MPI)p 222 2070 13 2 v 14 w(SIGNAL)p 378 2070 V 15 w(ALL)p Fk(?)f(Is)g(the)h(notion)e(of)g(signals)g(to)q(o)h(Unix-cen)o (tric?)215 2191 y Fl(Discussion:)45 b Fk(Should)17 b(there)h(b)q(e)f(a) f(w)o(a)o(y)g(to)h(deliv)o(er)g(a)f(signal)g(to)g(a)g(pro)q(cesses)k (that)d(has)f(b)q(een)145 2241 y(spa)o(wned,)e(so)g(that)g(there)h(is)f (no)g(request)i(ob)r(ject?)j(It)14 b(w)o(ould)f(b)q(e)i(a)f(collectiv)o (e)g(op)q(eration)g(on)g(an)f(in)o(ter-)145 2291 y(comm)o(unicator.)145 2398 y Fi(Note)f(that)g Fb(MPI)p 433 2398 14 2 v 16 w(SP)l(A)-5 b(WN)14 b Fi(can)f(b)q(e)h(written)e(in)i(terms)e(of)h(the)g(lo)o(w)o (er-lev)o(el)h(routines.)19 b(F)l(or)13 b(example,)122 2532 y Fc(MPI_SPAWN\(comm,arch\\_type)o(,num,hos)o(tnames,)o(executa)o (ble,argv)o(ecs,fla)o(g,newco)o(mm\))74 2666 y Fi(can)i(b)q(e)h (written)g(as)e(\(without)h(error)g(handling\))951 2790 y(14)p eop %%Page: 15 15 15 14 bop 122 7 a Fc(MPI_IALLOCATE\(num,advice,)o(\(char)21 b(**\)0,arch_type,hostnames)o(,MPI_HA)o(RD,requ)o(ests\))122 64 y(for)i(\(i=0;i To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann), itf@mcs.anl.gov Subject: mcast.txt Date: Fri, 10 Mar 1995 16:31:29 -0600 From: Rusty Lusk Multicast proposal #1 (MPI Forum 1.5 Proposal) Anthony Skjellum Nathan Doss Mississippi State University February 26, 1995 Synopsis -------- A multicast that acts as a multi send has been requested by several users. We propose a new chapter of the standard on semi-collective operations, to include MPI_Mcast() variants described below. Discussion ---------- MPI-F's rationale for not including this feature in the original standard was briefly as follows: * all receives would have to test an extra envelope bit to see if the message was "regular" or "multicast" * the cost of implementing this feature by the user, with persistent objects, would not be prohibitive * point-to-point performance was not to be compromised unduly for collective performance (mantra of MPI) * what would the recipients posit for rank and tag for this operation? Extensive net discussion ensued over the last year. Furthermore, L. Kale of UIUC has brought this issue up at at least two meetings (Supercomputing '94, and SIAM Seventh Conf. on Parallel Proc. for Scientific Computing), and called this as the most serious omission in MPI. Others have complained as well. It is consistently argued that for certain systems, it is exactly this tradeoff that is wanted by users. So, how do we do this, and still avoid impacting send and receive? We ask that the MPI-F should consider again the need for MPI_Mcast, given real user and developer demand. This constructive proposal seeks to do this with minimal impact on communication that does not make use of this "semi-collective operation." We seek options that lead to high performance implementations. Possible features of implementation proposed I) The accept that all communicators deal with overhead of mcasts a) use sender's true rank, and special tag, ordering guarantees, can be overridden by tag = MPI_SRC_ANY b) user manages all tags, no ordering guarantees OR II) The MPI_Mcast should only impact communicators that utilize it. That is, send and/or receives might be slower to support the overhead needed for this mcast to work. a) A "protocol" synchronization might be required to turn this feature on/off for a communicator. AND/OR b) A version of communicator constructors that has a flag field for special features could be added. The existing constructors would assume inheritance (as appropriate) of the mcast capability. Later, other flags could be specified that help with performance relevant issues. For instance, we already know that we want a MPI_Comm_dup() with NO INHERITANCE OF ATTRIBUTES, and now we have to resort to MPI_Comm_split() [or even MPI_Comm_create] to get it. AND c) Send/receive semantics are optimized with special tags and srcs, as noted above in Scenario Ia. OR d) Same as Ib. We favor II. Furthermore, we consider options between a), b), and c) below. *) In either I) or II): The MPI_Mcast must be implementable with an algorithm relevant to number of recipients. For many participants, spanning trees must be possible. For few participants, linear sends must be possible. In either event, the behavior must be consistently described to the user, and in no event will the context concept be violated. a) The entire group of the communicator b) a subset of these. We will propose two major call variants. The second will subsume the first, by including a group to specify recipients. This will be analogous to use of MPI_Comm_create where a communicator and subset group appear in the construction of a new communicator. It will be less convenient to use, if one intends to broadcast to whole group, since a separate group object must be created and maintained. *) A natural extension to intercommunicators must be possible *) Deadlock does not arise for non-erroneous programs -- loosely, Mcast is not allowed to be a synchronization. Conceptually, it should behave a lot like a loop with multiple sends. Syntax /Semantics ----------------- Scenario #I *) The overhead on all receives to provide this feature is accepted, presumably by supporting a bit in the message envelope that tells the recipient processor to store and forward, if a logarithmic algorithm is used. MPI_Mcast(buffer, count, datatype, [tag1,] comm, ierr) [sender only] MPI_Recv(buffer, count, datatype, rank, tag2, comm, status, ierr) [Recipients only] MPI_Mcast_group(buffer, count, datatytpe, [tag1], comm, grp, ierr) [sender only, grp is a subset of group of comm] *) All intra and intercommunicators support this operation with no change to other MPI calls. comm is either an intra- or inter-comm in the above calls. What are the typical values of tag1, tag2 and rank? sub-scenario #a tag1 = argument not used (like true collective ops) tag2 = MPI_MCAST_TAG rank = sender's true rank Implementations are responsible for using tag2 and rank as source identification for the mcast. Two mcast's from the same root are received in the same order sent, providing partial ordering extension to mcast. If rank = MPI_SRC_ANY, then the guarantees of ordering are relaxed, as in sub-scenario #b. The use of MPI_MCAST_TAG allows implementations to pose receives that know they have to do store-and-forward broadcast or other additional work. Furthermore, by using MPI_IRecv in that way, one could hope for higher performance of the MPI_Mcast. In combination with persistent objects, other optimizations might be possible. If a message is sent to a process from the root before the MPI_Mcast, does the recipient get it before or after the MPI_Mcast data? For this sub-scenario, where the recipient is Mcast aware, the answer should be that the partial ordering be maintained, except when src = MPI_SRC_ANY is used. status indicates tag= MPI_MCAST; src = effective source (root) sub-scenario #b (recipient only vaguely mcast aware) tag1 = user-specified tag2 = user-specified rank = MPI_SRC_ANY Implementation promises nothing about ordering. User manages the tags used for multicasts, and has to be sure that, within a communicator, no two multicasts with the same tag are active at the same time. Otherwise, unpredictable results occur. No guarantees of ordering are provided. The user must use appropriate tags (and possibly multiple communicators) to get appropriate behavior. For the operation MPI_Mcast_group(), the message is sent to all members listed in the group. MPI_Mcast() does not send a copy of the message to the sender, so the calls have legitimately different applicability. Furthermore, the MPI_Mcast_group() call allows substructure to be posed by an application on a communicator without making more communication contexts. Since groups might be more in flux than communicators, this offers a lighter weight mechanism for describing broadcast boundaries. We favor sub-scenario #1. Scenario #II IIa. No communicator that does not have the semi-collective attribute set may use the semi-collective operations (currently MCASTs). That is, a program is erroneous if it violates this condition. MPI_Comm_set_semi_collective(comm, ierr) MPI_Comm_unset_semi_collective(comm, ierr) This call is a synchronization over all the members of group of comm. IIb. Same rule as for IIa. MPI_Comm_dup_selective(comm, newcomm, flags, ierr) MPI_Comm_split_selective(comm, color, key, newcomm, flags, ierr) Two orable flags are suggested: MPI_COMM_SEMI_COLLECTIVE MPI_COMM_NO_ATTRIBUTES (see keys and user attributes) IIa and IIb are not mutually exclusive. IIc&d. Semantic choices are same as for Scenario Ia&b. Hopefully, implementations can avoid overhead for communicators that do not demand semi-collective capability. Summary ------- This concludes the first version of our proposal. We are asking for the establishment of two variants of MCAST in MPI, with choices on whether or not communicators must activtate this feature. The choices posed are aimed at reducing overhead for communicators that do not exploit MCASTs, and should be judged on their ability to do so. We invite further discussion, and hope to finalize this proposal for MPI meeting in March. From owner-mpi-core@CS.UTK.EDU Fri Mar 10 17:37:55 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA19372; Fri, 10 Mar 1995 17:37:55 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA21851; Fri, 10 Mar 1995 17:38:27 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 10 Mar 1995 17:38:24 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id RAA21831; Fri, 10 Mar 1995 17:38:21 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA29655; Fri, 10 Mar 1995 16:38:11 -0600 Message-Id: <199503102238.QAA29655@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann), itf@mcs.anl.gov Subject: the next few messages Date: Fri, 10 Mar 1995 16:38:10 -0600 From: Rusty Lusk I am going to distribute a number of things to read over the weekend, for possible discussion next week. You have already received some things to think about from Marc Snir. In the next few messages, I am going to send you: The errata from the Standard, prepared by Steve Otto, which contains some discussion items that we need to resolve. (postscript) A Dynamic Process creation proposal from Tony Skjellum. (postscript) A somewhat more elaborate process-management and client-server proposal from Bill Gropp and me. (postscript) A multicast proposal from Tony Skjellum. (ascii) If anyone else wants me to distribute anything else before the meeting, let me know soon. Rusty From owner-mpi-core@CS.UTK.EDU Fri Mar 10 18:16:54 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA20165; Fri, 10 Mar 1995 18:16:54 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id SAA23884; Fri, 10 Mar 1995 18:17:00 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 10 Mar 1995 18:16:58 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id SAA23841; Fri, 10 Mar 1995 18:16:31 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA00614; Fri, 10 Mar 1995 17:16:14 -0600 Message-Id: <199503102316.RAA00614@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann), itf@mcs.anl.gov Subject: progress.txt Date: Fri, 10 Mar 1995 17:16:14 -0600 From: Rusty Lusk Progress Rule proposal #1 Greg Burns Raja Daoud Ohio Supercomputer Center March 10, 1995 We propose adding a "Guaranteed Resources" section to complement the "Progress Rule" and provide users with a strong but limited progress guarantee. This would tighten the currently ``vacuous'' combination of "Progress Rule" and "Resource Limitations". ==================================================================== Progress: Within the limitation of Guaranteed Resources (see below), ... [same as before] ... Guaranteed Resources: Per-process system memory resources, limited by nature, are consumed whenever messages are not received in the order they arrive. Progress, as defined above, is guaranteed as long as these resources are not depleted. An MPI implementation will advertise a Guaranteed Resources (GR) figure of merit and will guarantee the progress of an application so long as the disorderliness of message receives by any process does not exceed the GR limit. [??? The minimum value of GR for an MPI compliant implementation is 40. Rationale: Moses spent 40 years wandering the Sinai desert, and if it's good enough for Moses, it's certainly good enough for me! ???] Advice to users: An upper-bound on per-process system memory resources needed to handle the disorderliness of any application is based on the total number of processes (P), the number of communication contexts (C), and the maximum displacement in out-of-order tag receives (T). An application that satisfies the equation: P * C * T <= GR is guaranteed progress on the implementation advertising that GR limit. If this equation is not satisfied an implementation may either deadlock, return an error to the application, abort, or may progress since the equation relates to an upper-bound. Advice to implementors: The GR of an implementation is the number of per-process envelope buffers. An implementation cannot expect to know the C * T factor of the equation prior to starting an application. However it can know P and might use this knowledge to warn the user when P > GR. [??? An implementation is encouraged to make its GR limit larger than the MPI minimum required value. ???] Advice to software providers: The (C * T) factor of an application or library should be advertised to the customer. From owner-mpi-core@CS.UTK.EDU Sat Mar 11 00:38:49 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA24577; Sat, 11 Mar 1995 00:38:49 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id AAA04216; Sat, 11 Mar 1995 00:38:50 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 11 Mar 1995 00:38:48 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id AAA04201; Sat, 11 Mar 1995 00:38:45 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA03253; Fri, 10 Mar 1995 22:17:39 -0600 Message-Id: <199503110417.WAA03253@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann), itf@mcs.anl.gov Subject: [wcs@nas.nasa.gov (William C. Saphir): late agenda item] Date: Fri, 10 Mar 1995 22:17:38 -0600 From: Rusty Lusk ------- Forwarded Message Date: Fri, 10 Mar 1995 18:37:50 -0800 From: wcs@nas.nasa.gov (William C. Saphir) To: lusk@mcs.anl.gov Subject: late agenda item This is a bit late, but if it's possible to add to the agenda I'd appreciate it. I would be interested in adding to the agenda a clarification of the status of Fortran 90 (and possibly C++ although I don't personally care about it and it's not yet a standard). While a careful treatment might be best suited for MPI II, I think we need something minimal in the meantime. I'm looking for a statement along the lines of "Insofar as Fortran 77 is a subset of Fortran 90, the Fortran 77 interfaces described in the MPI standard apply to Fortran 90 as well." Also well within the range of the current standard, would be to modify the first sentence of section A.2 to read "These are required defined constants, to be defined in the files mpi.h (for C) and mpif.h (for Fortran 77) and in the module "mpi" (for Fortran 90). My goal is that Fortran 90 users be guaranteed access to MPI, and that F90 codes be portable. The current standard, "It is expected that any Fortran 90 and C++ implementations use the Fortran and C bindings, respectively", does not require that MPI work with Fortran 90. Adding an mpi F90 module is trivial for implementors and gives F90 users a portable interface to MPI that doesn't rely on deprecated features (the include statement) or specific source form (free or fixed). In any case, I think the above additions are trivial and would make the job of promoting Fortran 90 easier. Fortran 90 already works with MPI implementations - this just makes it official and frees the user from messing with mpif.h (which I've had to do a few times). Bill ------- End of Forwarded Message From owner-mpi-core@CS.UTK.EDU Sat Mar 11 00:38:54 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA24582; Sat, 11 Mar 1995 00:38:53 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id AAA04217; Sat, 11 Mar 1995 00:38:50 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 11 Mar 1995 00:38:46 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id AAA04196; Sat, 11 Mar 1995 00:38:43 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA03233; Fri, 10 Mar 1995 22:16:17 -0600 Message-Id: <199503110416.WAA03233@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov (William C. Saphir), rbarrett@wrangler.LANL.GOV (Richard Barrett), gdburns@tbag.osc.edu (Greg Burns), Hans-Christian Hoppe , elster@austin.asc.slb.com (Anne Elster), shuler@cs.sandia.gov (Lance Shuler), raja@alex.osc.edu (Raja Daoud), "Eric Salo" , golan@convex.com (Dan Golan), sfleisch@convex.com (Steve Fleischmann), itf@mcs.anl.gov Subject: [Clive Baillie: MPI-RGL proposal ] Date: Fri, 10 Mar 1995 22:16:16 -0600 From: Rusty Lusk ------- Forwarded Message From: Clive Baillie To: lusk@mcs.anl.gov Subject: MPI-RGL proposal Date: Fri, 10 Mar 95 18:39:11 -0700 Dear All We would like to submit a proposal to the MPI Forum next week. One of us (McBryan) will be attending to discuss it. We are sorry for the short notice but we have only just completed the first VERY PRELIMINARY draft of our "MPI-RGL" library and would like to see how it is received before producing a more polished copy. Basically, we are proposing an extension to MPI which provides a usability layer for the application specialist who wishes to parallelize a code whose data structures are regular grids, without having to know any low-level message passing. To get the postscript file, please look at either http://www.cs.colorado.edu/~clive/public_html/mpi_rgl.ps or http://www.cs.colorado.edu/~broker/mpi_rgl/main.html Note that we do not plan to go off and implement this on our own, we would like to have a collaborative effort - assuming anybody is interested. Clive Baillie ------- End of Forwarded Message From owner-mpi-core@CS.UTK.EDU Sat Mar 11 15:33:48 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA18421; Sat, 11 Mar 1995 15:33:47 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA16868; Sat, 11 Mar 1995 15:34:09 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 11 Mar 1995 15:34:06 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA16861; Sat, 11 Mar 1995 15:34:05 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA13802; Sat, 11 Mar 95 14:33:58 CST Date: Sat, 11 Mar 95 14:33:58 CST From: Tony Skjellum Message-Id: <9503112033.AA13802@Aurora.CS.MsState.Edu> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov, rbarrett@wrangler.LANL.GOV, gdburns@tbag.osc.edu, hch@pallas.de, elster@austin.asc.slb.com, shuler@cs.sandia.gov, raja@alex.osc.edu, salo@mrjones.engr.sgi.com, golan@convex.com, sfleisch@convex.com, itf@mcs.anl.gov, lusk@mcs.anl.gov Subject: Re: [Clive Baillie: MPI-RGL proposal ] I would like to see what Clive has done, but if it can be layered, then we need to ask what performance limitation is implied by the layering, and look if any MPI primitives at all are needed to remove such limitations. -Tony From owner-mpi-core@CS.UTK.EDU Sun Mar 12 09:11:35 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA29946; Sun, 12 Mar 1995 09:11:34 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA06792; Sun, 12 Mar 1995 09:11:38 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sun, 12 Mar 1995 09:11:36 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA06782; Sun, 12 Mar 1995 09:11:34 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA14139; Sun, 12 Mar 95 08:11:11 CST Date: Sun, 12 Mar 95 08:11:11 CST From: Tony Skjellum Message-Id: <9503121411.AA14139@Aurora.CS.MsState.Edu> To: elster@austin.asc.slb.com, gdburns@tbag.osc.edu, golan@convex.com, hch@pallas.de, itf@mcs.anl.gov, lusk@mcs.anl.gov, mpi-core@CS.UTK.EDU, raja@alex.osc.edu, rbarrett@wrangler.LANL.GOV, salo@mrjones.engr.sgi.com, sfleisch@convex.com, shuler@cs.sandia.gov, wcs@nas.nasa.gov Subject: Re: mcast.txt Name Service proposal #1 (MPI Forum 1.5 Proposal) Anthony Skjellum Nathan Doss Mississippi State University February 26, 1995 Synopsis -------- Originally, MPI intercommunicators included a name service. This was a controversial aspect of the [Chapter 5] of the standard. In the last revision, we omitted name service, and provided for loosely synchronous constructors for intercommunicators. This was done for several reasons: * a correct model for intercommunicator that was minimal seemed necessary for passing that part of the standard by MPI-F, * there were objections to name service from the committee (especially TMC) * No established practice existed, from which to extrapolate the large-scale benefits of a name service * We were not attempting inter-vendor support at that time. Now there is strong motivation, and a much better understanding of how intercommunicators create an environment in which to build parallel client server computations. The opportunity to create a set of related services in MPI II, including a name service, presents itself. Name service would be a set of inter-communicator constructors/destructors that allow groups to rendezvous for the purpose of connecting applications. Both loosely synchronous, and asynchronous versions seem necessary. Discussion ---------- What has changed? 1) Experience with augmenting intercommunicators for collective communicators, as implemented in MPIX library. This is a separate proposal we will make for MPI 1.5...2. In this mode, we are able to describe parallel client server model very effectively. (Please see ftp://aurora.cs.msstate.edu/pub/reports/Message-Passing/mpix-rev1.ps.Z) [The MS thesis based on this work was completed last week, and will be available by ftp soon. The software will be made part of MPICH release in April. Port to LAM MPI is also planned, subsequently.] 2) Reasonable descriptions of asychronous collective operations for MPI and MPIX collective operations are possible, based on the encapsulation of a multi-threaded model. (Please see ftp://aurora.cs.msstate.edu/pub/reports/Message-Passing/extendmpi.ps.Z) [The multi-threaded version of MPICH is in progress.] 3) It is difficult to create inter-vendor (or vendor neutral) MPI without a name service means to connect groups that did not have antecedents in any one MPI_COMM_WORLD, nor should the synchronization of the entire "world" be necessary to grow an application. 4) Spawn proposals, MPIX intercommunicator collective ops, and inter-vendor capabilities all seem to work best with Intercommunicators, and a name service could abstract many of these issues. If the service can be used by MPI and by applications, we can keep extensions to a minimum. 5) Low level standards activities may provide configuration help * eg, standard for gang scheduling * eg, low-level message-passing extensions to TCP/IP and MPI should be able to exploit but yet encapsulate these features for applications. Desired Initial Action ---------------------- * The name services proposed in the original MPI drafts will be brought up to date, and made compatible with the final MPI specification. * Examples of name service use will be provided. * Loosely synchronous name services will be specified [We will revive this info ASAP.] Secondary Actions ----------------- * Further examples of name service use will be provided. * If MPI accepts asynchronous collective communication and MPIX-style collective operations for inter-communicators, then asychronous name service will be specified. Editorial --------- If accepted, this would add to Ch 5 of the standard. From owner-mpi-core@CS.UTK.EDU Sun Mar 12 09:21:54 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA00129; Sun, 12 Mar 1995 09:21:54 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA07159; Sun, 12 Mar 1995 09:22:28 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sun, 12 Mar 1995 09:22:26 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA07152; Sun, 12 Mar 1995 09:22:25 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA14163; Sun, 12 Mar 95 08:22:11 CST Date: Sun, 12 Mar 95 08:22:11 CST From: Tony Skjellum Message-Id: <9503121422.AA14163@Aurora.CS.MsState.Edu> To: elster@austin.asc.slb.com, gdburns@tbag.osc.edu, golan@convex.com, hch@pallas.de, itf@mcs.anl.gov, lusk@mcs.anl.gov, mpi-core@CS.UTK.EDU, raja@alex.osc.edu, rbarrett@wrangler.LANL.GOV, salo@mrjones.engr.sgi.com, sfleisch@convex.com, shuler@cs.sandia.gov, wcs@nas.nasa.gov Subject: C++ Binding for MPI II C++ Binding for MPI proposal #1 (MPI Forum 1.5 Proposal) Anthony Skjellum Nathan Doss Mississippi State University March 12, 1995 Synopsis -------- Initial experience with C++ MPI programming suggests that a C++ binding would be much more useful than the C binding currently available. For instance, the handling of temporaries is not optimal at present, and a stance on overloading of operators should be taken. This was clearly beyond the personpower scope of the original committee, but is extremely timely now. Discussion ---------- 1) A simple add-on binding has been done by us, but it is far from definitive. (Please see ftp://aurora.cs.msstate.edu/pub/reports/Message-Passing/oon-ski94.ps.Z) [MPI++ binding for MPICH is being cleaned up for re-release at present.] 2) Sotelino et al of Purdue have suggested an alternative strategy. [Reference to follow.] We foresee vast uses of C++ based MPI, and find it unfortunate that non- standard interfaces will start to be created. Though one or both of the C++ strategies above MIGHT become widely used, an official binding would be preferable. We recognize that for some C++ applications in parallel, MPI calls will be totally encapsulated in other classes. Yet, it would be nice for MPI to have a binding. We must look for cases where PERFORMANCE IS INCREASED THROUGH THE BINDING. MPI fits nicely into a class structure, with communicators as objects, groups as objects, etc. Editorial --------- If accepted, this would add to Language Bindings section of the standard. From owner-mpi-core@CS.UTK.EDU Sun Mar 12 09:37:39 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA00350; Sun, 12 Mar 1995 09:37:39 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA07623; Sun, 12 Mar 1995 09:38:13 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sun, 12 Mar 1995 09:38:10 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA07616; Sun, 12 Mar 1995 09:38:07 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA14181; Sun, 12 Mar 95 08:37:48 CST Date: Sun, 12 Mar 95 08:37:48 CST From: Tony Skjellum Message-Id: <9503121437.AA14181@Aurora.CS.MsState.Edu> To: elster@austin.asc.slb.com, gdburns@tbag.osc.edu, golan@convex.com, hch@pallas.de, itf@mcs.anl.gov, lusk@mcs.anl.gov, mpi-core@CS.UTK.EDU, raja@alex.osc.edu, rbarrett@wrangler.LANL.GOV, salo@mrjones.engr.sgi.com, sfleisch@convex.com, shuler@cs.sandia.gov, wcs@nas.nasa.gov Subject: MPI Topology Extensions Topology Extensions proposal #1 (MPI Forum 1.5 Proposal) Anthony Skjellum Nathan Doss Mississippi State University March 12, 1995 Synopsis -------- Topology constructors are a form of communicator constructor. The topology constructors applied to MPI are too limited, in several areas: * they apply only to intra-communicators * they cannot be changed after created, even by loose synchronization over the communicator This proposal would take the MPIX intercommunicator topologies, as specified, as a starting proposal for MPI II, but also propose extending intra-communicator topologies to allow them to be added without creating a new communicator, and to be modified through a synchronization. Both of these features are demonstrated in MPIX for the inter-communicator case. Topologies, especially cartesian topologies, find extremely wide application use, and are one of the most important benefits of MPI. Logically extending them to intercommunicators, and relaxing the arbitrary restrictions on constructors will benefit the standard with little additional effort on the part of implementors (the code already exists in MPIX as a demonstration of same). Omission in the original standard was based on lack of acceptance of intercommunicators as general-purpose communication mechanisms, and because of lack of time we did not argue about more general semantics for intra-communicator constructors with topologies. The binding of topologies to communicators instead of groups came late in the standard, which contributed to this as well. All aspects of this proposal thus far would be upward compatible with existing features. We should reinvestigate runtime links between topology and architecture as well. Some users will want to have more control, and a heterogeneous environment begs this issue as well. Discussion ---------- 1) The syntax and semantics for inter-communicator collective operations follows (yet extends) the intra-communicator strategy: (Please see ftp://aurora.cs.msstate.edu/pub/reports/Message-Passing/mpix-rev1.ps.Z) [The MS thesis based on this work was completed last week, and will be available by ftp soon. The software will be made part of MPICH release in April. Port to LAM MPI is also planned, subsequently.] 2) MPICH implements topologies with the cached attributes mechanism; that makes for a natural layering on top of any MPI implementation. It should be discussed whether/if the user should have more control over the topology mechanisms, such as a mechanism for defining his/her own topologies that is somehow supported by MPI features. In certain cases, hidden contexts of communication are inaccessible, but would be appropriate for certain constructors. 3) Stronger links to mapping onto architecture may be desirable. At present, this is tenuous. Particularly, it would be nice to be able to ask for sub-meshes, or sub-cubes, etc, and be sure that the topologies built on such communicators conformed appropriately. This will be of interest to a small but not insignificant audience. Editorial --------- If accepted, this would add to Ch 6 of the standard. From owner-mpi-core@CS.UTK.EDU Wed Mar 15 16:31:52 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA17110; Wed, 15 Mar 1995 16:31:52 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA25547; Wed, 15 Mar 1995 16:31:36 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 15 Mar 1995 16:31:33 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id QAA25533; Wed, 15 Mar 1995 16:31:31 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA12570 for ; Wed, 15 Mar 1995 15:31:24 -0600 Message-Id: <199503152131.PAA12570@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: Minutes of meeting Date: Wed, 15 Mar 1995 15:31:21 -0600 From: William Gropp The minutes are now available at http://www.mcs.anl.gov/mpi/mpi2/mar13.dvi . I'll have pure HTML versions soon. Bill From owner-mpi-core@CS.UTK.EDU Thu Mar 16 11:06:38 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA10218; Thu, 16 Mar 1995 11:06:37 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA17041; Thu, 16 Mar 1995 11:06:46 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 16 Mar 1995 11:06:45 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA17034; Thu, 16 Mar 1995 11:06:43 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 8935; Thu, 16 Mar 95 11:06:42 EST Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 1087; Thu, 16 Mar 1995 11:06:40 EST Received: from bubba.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Thu, 16 Mar 95 11:06:39 EST Received: from localhost by bubba.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA16421; Thu, 16 Mar 1995 11:06:38 -0500 Message-Id: <9503161606.AA16421@bubba.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: William Gropp Cc: mpi-impl@antares.mcs.anl.gov, mpi-core@CS.UTK.EDU Subject: Re: Standardized error classes In-Reply-To: Your message of Thu, 16 Mar 1995 09:33:03 CST Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Thu, 16 Mar 1995 11:06:38 -0500 From: "Hubertus Franke" It could be useful to shift the all MPI error codes (but MPI_SUCCESS) to a higher levels in order to allow system errors to be pushed to higher levels above error levels defined in /usr/include/errno.h on UNIX systems. From what I have seen this is limited to 128 is the limit but we should leave some space for future system extension. 256 or 1K are possible discusion negotiation points as well. e.g.: #define MPI_SUCCESS 0 /* Successful return code */ #define MPI_ERR_EXHAUSTED 257 /* Memory exhausted */ #define MPI_ERR_TAG 258 /* Invalid tag argument */ : : Hubertus Franke IBM T.J. Watson Research Center From owner-mpi-core@CS.UTK.EDU Thu Mar 16 11:06:39 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA10223; Thu, 16 Mar 1995 11:06:38 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA17090; Thu, 16 Mar 1995 11:07:23 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 16 Mar 1995 11:07:21 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA17068; Thu, 16 Mar 1995 11:07:17 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA29244; Thu, 16 Mar 1995 09:33:04 -0600 Message-Id: <199503161533.JAA29244@antares.mcs.anl.gov> To: mpi-impl@antares.mcs.anl.gov cc: mpi-core@CS.UTK.EDU Subject: Standardized error classes Date: Thu, 16 Mar 1995 09:33:03 -0600 From: William Gropp The MPI Forum would like to specify the actual values for error classes, so that an error class on one MPI process could be guarenteed to have the same value (not just symbolic name). The issue comes up for interoperability of different MPI implementations; for instance, if there was a way to connect two MPP MPI versions by IP/TCP. The MPI Forum understands (I hope!) that this is getting into constraints on the implementation that we normally try to avoid; at the same time, if it is a trivial matter for the existing implementations, then we could make life a little easier for the users. To start this, I'm going to list the error classes and values currently used by MPICH. Note that there are a few error classes not mentioned in the MPI standard; this is permitted (and is one of the areas of difficulty in this task). Please send me your comments. Bill #define MPI_SUCCESS 0 /* Successful return code */ #define MPI_ERR_EXHAUSTED 1 /* Memory exhausted */ #define MPI_ERR_TAG 2 /* Invalid tag argument */ #define MPI_ERR_COMM_NULL 3 /* NULL communicator argument passed to function */ #define MPI_ERR_COMM_INTER 4 /* Intercommunicator is not allowed in function */ #define MPI_ERR_COMM_INTRA 5 /* Intracommunicator is not allowed in function */ #define MPI_ERR_ARG 6 /* Invalid argument */ #define MPI_ERR_BUFFER 7 /* Invalid buffer pointer */ #define MPI_ERR_COUNT 8 /* Invalid count argument */ #define MPI_ERR_TYPE 9 /* Invalid datatype argument */ #define MPI_ERR_ROOT 10 /* Invalid root */ #define MPI_ERR_OP 11 /* Invalid operation */ #define MPI_ERR_ERRORCODE 12 /* Invalid error code */ #define MPI_ERR_GROUP 13 /* Null group passed to function */ #define MPI_ERR_RANK 14 /* Invalid rank */ #define MPI_ERR_TOPOLOGY 15 /* Invalid topology */ #define MPI_ERR_DIMS 16 /* Illegal dimension argument */ #define MPI_ERR_NULL 17 /* Null parameter */ #define MPI_ERR_UNKNOWN 18 /* Unknown error */ #define MPI_ERR_REQUEST 19 /* illegal mpi_request handle */ #define MPI_ERR_LIMIT 20 /* limit reached */ #define MPI_ERR_INTERN 21 /* internal error code */ #define MPI_ERR_NOMATCH 22 /* no recv posted for ready send */ #define MPI_ERR_TRUNCATE 23 /* message truncated on receive */ #define MPI_ERR_BAD_ARGS 24 #define MPI_ERR_INIT 25 /* MPI_INIT already called */ #define MPI_ERR_PERM_KEY 26 /* Can't free a perm key */ #define MPI_ERR_BUFFER_EXISTS 27 #define MPI_ERR_COMM 28 /* Invalid communicator */ #define MPI_ERR_PERM_TYPE 29 /* Can't free a perm type */ #define MPI_ERR_OTHER 30 /* Other error; use Error_string */ #define MPI_ERR_LASTCODE 31 /* Last error code -- always at end */ From owner-mpi-core@CS.UTK.EDU Thu Mar 16 11:09:11 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA10289; Thu, 16 Mar 1995 11:09:11 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA17369; Thu, 16 Mar 1995 11:10:00 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 16 Mar 1995 11:09:58 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA17359; Thu, 16 Mar 1995 11:09:55 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA00282 for ; Thu, 16 Mar 1995 10:09:53 -0600 Message-Id: <199503161609.KAA00282@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: Text about MPI_BOTTOM Date: Thu, 16 Mar 1995 10:09:49 -0600 From: William Gropp Here is my proposal for clarification on MPI_BOTTOM in Fortran In Fortran, MPI_BOTTOM is not required to be a constant as defined by the Fortran 77 standard (section 4.2). Users are cautioned that in Fortran, MPI_BOTTOM may not be assignable to another variable. Bill From owner-mpi-core@cs.utk.edu Sun Mar 19 06:20:21 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA02623; Sun, 19 Mar 1995 06:20:21 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA07724; Sat, 18 Mar 1995 19:08:45 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 18 Mar 1995 19:08:43 EST Errors-to: owner-mpi-core@cs.utk.edu Received: from infomeister.osc.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id TAA07716; Sat, 18 Mar 1995 19:08:41 -0500 Received: from alex.osc.edu for raja@alex.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id TAA04182; Sat, 18 Mar 1995 19:08:34 -0500 Received: by alex.osc.edu (4.1/OSC-SUB-2.0) id AA03886; Sat, 18 Mar 95 19:05:18 EST Date: Sat, 18 Mar 95 19:05:18 EST From: raja@alex.osc.edu (Raja Daoud) Message-Id: <9503190005.AA03886@alex.osc.edu> To: mpi-impl@antares.mcs.anl.gov Subject: Re: Standardized error classes Cc: mpi-core@cs.utk.edu _If_ some form of mandate is to be imposed on error class numbers, I would suggest: 1- Let MPI provide 2 class regions, one in which numbers are assigned by the MPI Forum (i.e. is rigidly structured) and one left open for implementations to define their own new classes (at least until the Forum decides to make these new classes part of MPI). There is precedence for an implementation needing to define its own error class, and I'd like to keep providing this flexibility. 2- Let MPI choose the class regions along 2^N boundaries, allowing implementations to do bit-slicing if needed. Example: 0 MPI_SUCCESS 1 <= x <= MPI_ERR_LASTMANDATEDCODE <= 127 mandated classes 128 <= x <= MPI_ERR_LASTIMPLCODE <= 255 implementation classes An application running on a multi-implementation system _and_ not wanting to send error strings around (what's the probability of this scenario?), would be able to send error classes as integers and print remotely the error message if x < 128 or print "warning: impl. dependent error class" and provide the x >= 128 number. Applications running on a single-impl system _or_ willing to send strings wouldn't need to rely on the limited mandated classes and can provide better error reporting, using the impl-dependent classes as well as the rest of the error code. --Raja From owner-mpi-core@CS.UTK.EDU Tue Mar 21 08:35:20 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA19784; Tue, 21 Mar 1995 08:35:19 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id IAA04570; Tue, 21 Mar 1995 08:33:50 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 21 Mar 1995 08:33:48 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id IAA04563; Tue, 21 Mar 1995 08:33:45 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA28629 (5.65c/IDA-1.4.4 for frankeh@watson.ibm.com); Mon, 20 Mar 1995 16:31:08 GMT Received: by deneb.co.uk (5.0/SMI-SVR4) id AA13625; Mon, 20 Mar 1995 16:30:46 +0000 Date: Mon, 20 Mar 1995 16:30:46 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9503201630.AA13625@deneb.co.uk> To: frankeh@watson.ibm.com Cc: mpi-impl@antares.mcs.anl.gov, mpi-core@CS.UTK.EDU In-Reply-To: <9503161606.AA16421@bubba.watson.ibm.com> (frankeh@watson.ibm.com) Subject: Re: Standardized error classes Reply-To: jim@meiko.com > It could be useful to shift the all MPI error codes (but MPI_SUCCESS) > to a higher levels in order to allow system errors to be > pushed to higher levels above error levels defined in > /usr/include/errno.h on UNIX systems. > > From what I have seen this is limited to 128 is the limit > but we should leave some space for future system extension. > 256 or 1K are possible discusion negotiation points as well. Be careful... Solaris errno.h has already got up to /* SUN Network File System */ #define ESTALE 151 /* Stale NFS file handle */ (and there appear to be gaps, too). -- Jim From owner-mpi-core@CS.UTK.EDU Wed Mar 22 20:40:33 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA04958; Wed, 22 Mar 1995 20:40:32 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id UAA19212; Wed, 22 Mar 1995 20:39:31 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 22 Mar 1995 20:39:30 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id UAA19205; Wed, 22 Mar 1995 20:39:21 -0500 Received: from ark.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA19661; Wed, 22 Mar 95 17:38:47 PST Received: from eel.ssd.intel.com by ark.SSD.intel.com (4.1/SMI-4.1) id AA11166; Wed, 22 Mar 95 17:38:44 PST Date: Wed, 22 Mar 95 17:38:44 PST Message-Id: <9503230138.AA11166@ark.SSD.intel.com> X-Sender: prp@eudora.ssd.intel.com X-Mailer: Windows Eudora Version 2.0.3 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: mpi-core@CS.UTK.EDU From: prp@ark.ssd.intel.com (Paul Pierce) Subject: MPI Constants I promised to look into what to do about MPI constants. The basic problem is that users want to be able to use them as initializers while implementers want more flexibility. This message contains everything interesting that I found about constants in the standard, for reference and to help identify the text that might have to change if we adopt a specific proposal. Discussion and maybe even an actual proposal will follow in a later message. Here is what the standard has to say about constants. "2.4.4 Named constants MPI procedures sometimes assign a special meaning to a special value of a basic type argument; e.g. tag is an integer-valued argument of point-to-point communication operations, with a special wild-card value, MPI_ANY_TAG. Such arguments will have a range of regular values, which is a proper subrange of the range of values of the corresponding basic type; special values (such as MPI_ANY_TAG) will be outside the regular range. The range of regular values can be queried using environmental inquiry functions (Section 7)." "2.5.1 Fortran 77 Binding Issues ... Although not required, it is strongly suggested that named MPI constants (PARAMETERs) be provided in an include file, called mpif.h. On systems that do not support include files, the implementation should specify the values of named constants." "2.5.2 C Binding Issues ... defined constants are in all capital letters ..." For purposes of discussion, I'm using this last phrase as the definitive indication of what is a "constant", referring to the capitalization of names in the C binding. This is consistent with the complaint about constants, that in C names in all caps usually indicate a #define macro that you can use in an initializer, while in some cases existing implementations do not appear to follow that convention. In the standard most text which introduces a new constant calls it (at most) a value. Some text does distinguish different kinds of constants. For instance, in 5.2.1 (Groups) and again in 5.2.4 with respect to communicators appears: "There is a special pre-defined group: MPI_GROUP_EMPTY, which is a group with no members. The predefined constant MPI_GROUP_NULL is the value used for invalid group handles." As it seeks to distinguish the empty object from the null handle this text seems to indicate that different "constants" might be implemented differently. Here are all the C constants by datatype. The ones with an asterisk are #define'd in the MPICH and LAM implementations. The rest aren't, although in some instances they could still be valid initializers. The only different constants between C and Fortran are the datatypes and the Fortran constants for status, MPI_STATUS_SIZE, MPI_SOURCE and MPI_TAG. In MPICH most are either PARAMETERs or variables in a named common block. The constants: MPI_Comm MPI_COMM_NULL* MPI_Comm MPI_COMM_SELF MPI_Comm MPI_COMM_WORLD MPI_Datatype MPI_2INT MPI_Datatype MPI_BYTE MPI_Datatype MPI_CHAR MPI_Datatype MPI_DATATYPE_NULL* MPI_Datatype MPI_DOUBLE MPI_Datatype MPI_DOUBLE_INT MPI_Datatype MPI_FLOAT MPI_Datatype MPI_FLOAT_INT MPI_Datatype MPI_INT MPI_Datatype MPI_LB MPI_Datatype MPI_LONG MPI_Datatype MPI_LONG_DOUBLE MPI_Datatype MPI_LONG_INT MPI_Datatype MPI_LONG_LONG_INT MPI_Datatype MPI_PACKED MPI_Datatype MPI_SHORT MPI_Datatype MPI_UB MPI_Datatype MPI_UNSIGNED MPI_Datatype MPI_UNSIGNED_CHAR MPI_Datatype MPI_UNSIGNED_LONG MPI_Datatype MPI_UNSIGNED_SHORT MPI_Errhandler MPI_ERRHANDLER_NULL* MPI_Errhandler MPI_ERRORS_ARE_FATAL MPI_Errhandler MPI_ERRORS_RETURN MPI_Group MPI_GROUP_EMPTY MPI_Group MPI_GROUP_NULL* MPI_Op MPI_BAND MPI_Op MPI_BOR MPI_Op MPI_BXOR MPI_Op MPI_LAND MPI_Op MPI_LOR MPI_Op MPI_LXOR MPI_Op MPI_MAX MPI_Op MPI_MAXLOC MPI_Op MPI_MIN MPI_Op MPI_MINLOC MPI_Op MPI_OP_NULL* MPI_Op MPI_PROD MPI_Op MPI_SUM MPI_Request MPI_REQUEST_NULL* int MPI_ANY_SOURCE* int MPI_ANY_TAG* int MPI_CART* int MPI_CONGRUENT* int MPI_ERR_ARG* int MPI_ERR_BUFFER* int MPI_ERR_COMM* int MPI_ERR_COUNT* int MPI_ERR_DIMS* int MPI_ERR_GROUP* int MPI_ERR_INTERN* int MPI_ERR_LASTCODE* int MPI_ERR_OP* int MPI_ERR_OTHER* int MPI_ERR_RANK* int MPI_ERR_REQUEST* int MPI_ERR_ROOT* int MPI_ERR_TAG* int MPI_ERR_TOPOLOGY* int MPI_ERR_TRUNCATE* int MPI_ERR_TYPE* int MPI_ERR_UNKNOWN* int MPI_GRAPH* int MPI_HOST int MPI_IDENT* int MPI_IO int MPI_KEYVAL_INVALID* int MPI_MAX_ERROR_STRING* int MPI_MAX_PROCESSOR_NAME* int MPI_PROC_NULL* int MPI_SIMILAR* int MPI_SUCCESS* int MPI_TAG_UB int MPI_UNDEFINED* int MPI_UNEQUAL* void * MPI_BOTTOM* (int(*)()) MPI_NULL_FN* -------------------------------------------------------------------------------- Paul Pierce prp@ssd.intel.com From owner-mpi-core@CS.UTK.EDU Thu Mar 23 14:26:11 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA23737; Thu, 23 Mar 1995 14:26:10 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA04835; Thu, 23 Mar 1995 14:25:45 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 23 Mar 1995 14:25:44 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA04818; Thu, 23 Mar 1995 14:25:42 -0500 Received: from ark.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA12430; Thu, 23 Mar 95 11:25:02 PST Received: from eel.ssd.intel.com by ark.SSD.intel.com (4.1/SMI-4.1) id AA16226; Thu, 23 Mar 95 11:25:00 PST Date: Thu, 23 Mar 95 11:25:00 PST Message-Id: <9503231925.AA16226@ark.SSD.intel.com> X-Sender: prp@eudora.ssd.intel.com X-Mailer: Windows Eudora Version 2.0.3 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: mpi-core@CS.UTK.EDU From: prp@ark.ssd.intel.com (Paul Pierce) Subject: MPI Constants The literal reading of the standard as it exists would seem to be that all "constants", that is, names in Annex A and which in the C binding are all caps, must be implemented such that they can be used in static initializers. Its reasonable to infer this from the terms "defined constant" (as opposed to "macro" or something else weaker) in the description of the C binding and "PARAMETER" in the description of the Fortran binding. This was not the original intent of the MPI Forum and is not consistent with current implementations. In particular, we intended some flexibility in implementing opaque objects so we specified unique types for their handles. It was expected (and has in fact turned out to be the case) that the handles for opaque data types would be pointers into dynamically allocated memory and that those "constants" which refer to usable objects might be allocated in MPI_INIT. On the other hand, it is convenient to be able to use many of these constants in static initializers. In C by convention names in all caps are assumed to be usable in this way. One could implement all handles as integers which index into a table of pointers to arbitrary objects, so it would be possible to statically define the indices of "constant" objects. This technique also has the advantage of making it easier to parameter check a handle argument, but disadvantages in the extra indirection required and in dynamically supporting arbitrary numbers of objects. In C one could define handles to pointers to statically allocated objects so that the handles could be used in static initializers but in Fortran this is not generally possible. Since we are discussing "constants" which aren't really constant in some sense, I will use the term "static" to mean that a constant is usable in any declaration consistent with its data type. "Not static" or "dynamic" means you should never use the constant in a declaration. One might ask which constants we most want to be static. The most important is a constant used to determine the size of an array, otherwise such an array would have to be allocated dynamically. That isn't possible in Fortran 77, so its a serious consideration. The constants directly implicated here are MPI_ERR_LASTCODE, MPI_MAX_ERROR_STRING, MPI_MAX_PROCESSOR_NAME and MPI_STATUS_SIZE. Other uses are to statically fill in values of data structures. For these, its not so bad to do it dynamically. An exception is that for safety you might like to be able to statically initialize handles to null values and other values to MPI_UNDEFINED so that you can detect coding errors. The standard explicitly allows (in advice to implementors) that MPI_NULL_FN need not be a null pointer, although it could be static. The standard also explicitly distinguishes between null handles and empty objects. There is one "constant" which cannot reasonably be static, that is, MPI_BOTTOM in Fortran. There is a separate proposal about that. We have been debating whether the values of error classes and/or codes should be defined in the standard. If so, they would certainly be static. I would be glad to hear of any other arguments. In the mean time, based on the above considerations I suggest the following groups of constants in priority order. 1. MPI_ERR_LASTCODE, MPI_MAX_ERROR_STRING, MPI_MAX_PROCESSOR_NAME and MPI_STATUS_SIZE. 2. Error codes: MPI_SUCCESS, MPI_ERR_ARG etc. 3. MPI_UNDEFINED. 4. Null handles: MPI_COMM_NULL, MPI_DATATYPE_NULL, MPI_ERRHANDLER_NULL, MPI_GROUP_NULL, MPI_OP_NULL, MPI_REQUEST_NULL, MPI_NULL_FN. 5. All constants which are "int" in C plus the Fortran status object constants. This includes groups 1-3 above. 6. Handles to opaque objects which are probably data structures. Constants of type MPI_Comm, MPI_Datatype, MPI_Group, and MPI_Request. 7. Ops, which may be procedure pointers: MPI_SUM etc. 8. Error handlers, which are probably procedure pointers: MPI_ERRORS_ARE_FATAL, MPI_ERRORS_RETURN. 9. MPI_BOTTOM. As a reasonable compromise and something that is fairly easy to specify and remember, I propose groups 1-5 above. This corresponds pretty well to current practice and makes things easy for implementors and programmers alike. --- Proposal: That the following text be added to section 2.4.4 "Named Constants". Some named constants must be statically defined so that they can be used in declarations. These are constants which are of type int in the C binding, or which have NULL in their name. For the Fortran binding, this additionally includes the constants related to the status object. All other constants may be defined dynamically by the implementation and must not be used in declarations. They may be used in code only after MPI_INIT and before MPI_FINALIZE. Between these calls all constants may be used and implementations must not change their values. Annex A lists all constants and whether they are statically or dynamically defined. That Annex A be updated to indicate which constants are static, according to the definition above. --- Here are all the C constants, grouped as defined in the proposal. Static MPI_ANY_SOURCE MPI_ANY_TAG MPI_CART MPI_CONGRUENT MPI_ERR_ARG MPI_ERR_BUFFER MPI_ERR_COMM MPI_ERR_COUNT MPI_ERR_DIMS MPI_ERR_GROUP MPI_ERR_INTERN MPI_ERR_LASTCODE MPI_ERR_OP MPI_ERR_OTHER MPI_ERR_RANK MPI_ERR_REQUEST MPI_ERR_ROOT MPI_ERR_TAG MPI_ERR_TOPOLOGY MPI_ERR_TRUNCATE MPI_ERR_TYPE MPI_ERR_UNKNOWN MPI_GRAPH MPI_HOST MPI_IDENT MPI_IO MPI_KEYVAL_INVALID MPI_MAX_ERROR_STRING MPI_MAX_PROCESSOR_NAME MPI_SIMILAR MPI_SUCCESS MPI_TAG_UB MPI_UNDEFINED MPI_UNEQUAL MPI_COMM_NULL MPI_DATATYPE_NULL MPI_ERRHANDLER_NULL MPI_GROUP_NULL MPI_OP_NULL MPI_REQUEST_NULL MPI_PROC_NULL MPI_NULL_FN Dynamic MPI_COMM_SELF MPI_COMM_WORLD MPI_2INT MPI_BYTE MPI_CHAR MPI_DOUBLE MPI_DOUBLE_INT MPI_FLOAT MPI_FLOAT_INT MPI_INT MPI_LB MPI_LONG MPI_LONG_DOUBLE MPI_LONG_INT MPI_LONG_LONG_INT MPI_PACKED MPI_SHORT MPI_UB MPI_UNSIGNED MPI_UNSIGNED_CHAR MPI_UNSIGNED_LONG MPI_UNSIGNED_SHORT MPI_ERRORS_ARE_FATAL MPI_ERRORS_RETURN MPI_GROUP_EMPTY MPI_BAND MPI_BOR MPI_BXOR MPI_LAND MPI_LOR MPI_LXOR MPI_MAX MPI_MAXLOC MPI_MIN MPI_MINLOC MPI_PROD MPI_SUM MPI_BOTTOM Here are the additional Fortran constants. Static MPI_STATUS_SIZE MPI_SOURCE MPI_TAG Dynamic MPI_INTEGER MPI_REAL MPI_DOUBLE_PRECISION MPI_COMPLEX MPI_DOUBLE_COMPLEX MPI_LOGICAL MPI_CHARACTER MPI_BYTE MPI_PACKED MPI_2REAL MPI_2DOUBLE_PRECISION MPI_2INTEGER MPI_2COMPLEX MPI_INTEGER1 MPI_INTEGER2 MPI_INTEGER4 MPI_REAL2 MPI_REAL4 MPI_REAL8 -------------------------------------------------------------------------------- Paul Pierce prp@ssd.intel.com From owner-mpi-core@CS.UTK.EDU Fri Mar 24 09:27:35 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA16234; Fri, 24 Mar 1995 09:27:35 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA26090; Fri, 24 Mar 1995 09:25:30 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 24 Mar 1995 09:25:27 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from dasher.cs.utk.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id JAA26071; Fri, 24 Mar 1995 09:25:25 -0500 From: Jack Dongarra Received: by dasher.cs.utk.edu (cf v2.11c-UTK) id JAA07459; Fri, 24 Mar 1995 09:25:23 -0500 Date: Fri, 24 Mar 1995 09:25:23 -0500 Message-Id: <199503241425.JAA07459@dasher.cs.utk.edu> To: mpi-core@CS.UTK.EDU Subject: ack for support of mpi If you received travel money to attend the MPI meetings and would like to give credit to ARPA and NSF for that support the following acknowledgment line could be added to publication you prepare discussing MPI. Work on MPI was supported in part by the Advanced Research Projects Agency under contract number NSF-ASC-9310330, administered by the National Science Foundation's Division of Advanced Scientific Computing. Thanks, Jack From owner-mpi-core@CS.UTK.EDU Fri Mar 24 10:24:01 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA16930; Fri, 24 Mar 1995 10:23:49 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA00695; Fri, 24 Mar 1995 10:23:33 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 24 Mar 1995 10:23:19 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from timbuk.cray.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id KAA00654; Fri, 24 Mar 1995 10:23:13 -0500 Received: from sdiv.cray.com (ironwood.cray.com [128.162.21.36]) by timbuk.cray.com (8.6.9/CRI-fence-1.4) with SMTP id JAA09768; Fri, 24 Mar 1995 09:23:01 -0600 Received: from [128.162.158.2] (poplar) by sdiv.cray.com (5.0/CRI-5.15.b.orgabbr Sdiv) id AA27394; Fri, 24 Mar 1995 09:22:59 -0600 Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Fri, 24 Mar 1995 09:24:17 -0600 To: jim@meiko.com From: par@ironwood.cray.com (Peter Rigsbee) Subject: Re: Standardized error classes Cc: frankeh@watson.ibm.com, mpi-impl@antares.mcs.anl.gov, mpi-core@CS.UTK.EDU, hlg@ironwood.cray.com At 10:30 AM 3/20/95, James Cownie wrote: >> It could be useful to shift the all MPI error codes (but MPI_SUCCESS) >> to a higher levels in order to allow system errors to be >> pushed to higher levels above error levels defined in >> /usr/include/errno.h on UNIX systems. >> >> From what I have seen this is limited to 128 is the limit >> but we should leave some space for future system extension. >> 256 or 1K are possible discusion negotiation points as well. > >Be careful... Solaris errno.h has already got up to > >/* SUN Network File System */ >#define ESTALE 151 /* Stale NFS file handle */ > >(and there appear to be gaps, too). CRAY UNICOS systems have that beaten pretty badly. We've defined a ton of errno constants, and have also reserved big sections for different subsystems: 1-127 -- System V errno's 128-199 -- TCP/IP, sockets 200-249 -- tapes and so on, up to 809. Our philosophy is that it is good to generate specific error numbers when possible, rather than reusing the same error number for dozens of unrelated problems and hoping the user can figure out which problem actually occurred. This also has an advantage that different subsystems can assign error numbers without having to coordinate with the rest of the division. And then our errno list reserves some large sets of error numbers for the use of the Fortran library: 1000-1999 -- Fortran library 5000-5999 -- ffio (a Fortran library layering capability) The bottom line is if MPI wants to define specific error numbers, it will inevitably conflict with "errno" values on some systems. We could pick some range that does not currently conflict with errno values for any systems familiar to MPI members, but we can't keep conflicts from occuring on new systems that might appear. I personally think that starting at "1" is just fine... - Peter From owner-mpi-core@CS.UTK.EDU Fri Mar 24 11:03:26 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA18149; Fri, 24 Mar 1995 11:03:25 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA04371; Fri, 24 Mar 1995 11:03:24 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 24 Mar 1995 11:03:21 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA04349; Fri, 24 Mar 1995 11:03:14 -0500 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) id LAA15281; Fri, 24 Mar 1995 11:02:39 -0500 From: David Walker Message-Id: <199503241602.LAA15281@rios2.EPM.ORNL.GOV> To: mpi-core@CS.UTK.EDU Subject: multicast proposal from David Walker Date: Fri, 24 Mar 95 11:02:38 -0500 The idea here is to come up with a way of letting a receive operation receive messages from both point-to-point sends and collective broadcasts. A simple approach is to add 2 new routines to MPI called mpi_msend and mpi_mrecv. These are essentially new communication modes, so we can also have nonblocking variants, mpi_imsend and mpi_imrecv. The arguments to mpi_msend and mpi_imsend are the same as for the corresponding standard sends, except there is no destination. The arguments to mpi_mrecv and mpi_imrecv are the same as for mpi_recv and mpi_irecv. Upon receipt the message tag and the rank of the original sender (i.e. whoever called mpi_msend or mpi_imsend) can be recovered from the status argument, and the actual number of items received from mpi_get_count, just as for the current receive routines. It is erroneous for an application to receive a multicast message with mpi_recv or mpi_irecv. Thus in any situation in which a multicast may be received, mpi_mrecv or mpi_imrecv must be used. Issues: 1) Should mpi_mrecv and mpi_imrecv be able to receive from mpi_bcast? If so, are mpi_msend and mpi_imsend needed? 2) Should mpi_mrecv and mpi_imrecv be able to receive from any pt-2-pt send operation. Probably yes, though the cost is having to put an extra flag an every envelope. Let me know what you think. David From owner-mpi-core@CS.UTK.EDU Fri Mar 24 11:36:21 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA18663; Fri, 24 Mar 1995 11:36:20 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA06777; Fri, 24 Mar 1995 11:36:15 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 24 Mar 1995 11:36:13 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA06769; Fri, 24 Mar 1995 11:36:11 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.10/8.6.10.1) with ESMTP id LAA22704 for ; Fri, 24 Mar 1995 11:32:32 -0500 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.10/8.6.10.1) with SMTP id LAA06695 for ; Fri, 24 Mar 1995 11:36:00 -0500 Date: Fri, 24 Mar 1995 11:36:00 -0500 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199503241636.LAA06695@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA12864; Fri, 24 Mar 95 11:36:00 EST To: mpi-core@CS.UTK.EDU Subject: Hello from the Editor This is a note to describe how I have started to set up things as the MPI-2 editor. All items will be archived and available via anonymous ftp from SRC. WEB user can access ftp://ftp.super.org/pub/mpi/mpi2. The following subdirectories exist: Editor: This contains information for those writing MPIF documents. It has the LaTeX macros, etc. mpif-0395: This has an archive of proposals, agenda, etc from the first meeting. I will try and create this for each meeting we have. It is not an archive of the mail messages, etc. of the MPIF. It is just the documents that may have been used at the meetings. Attached to this note is a description of the LaTeX macros the MPIF has been using. Many thanks to Steve Otto for doing these (and the HPF before that). If you have problems or questions please contact me. If you think they need to be modified then let me know. If you don't use official marcos your documents may not work correctly when assembled with everyone elses. Now for the informal deadlines. I would like to have all materals 4 working days before the meeting. For example, I would like everything for the next meeting by Tuesday, April 18. I know we all work at the last minute but if you want to be sure something is incorporated into what is distributed then you need to meet this deadline. For the next meeting this is not a big deal since most will be individual proposals. I will try to send everyone a postscript version to everyone 2 business days in advance. For the next meeting this will be late Thursday, April 20. This should get it to people in Europe by Friday morning and give them time to print it out and read it. I will also place the drafts in the mpif-"date" directory. Any thoughts or suggestions are always welcome. Steve P.S. - Please note that MPI-1 changes are still officially handled by Steve Otto. ---------------------------------------------------------------------- LaTeX Macros ------------ I've started to do some work on defining LaTeX macros for mpi constants, mpi functions, mpi function definitions, C function definitions, etc. We are separating out the macros into several different LaTeX macros so as to facilitate the creation of indices. For example, when it comes time to create an index to all the C function bindings that appear in the draft, we can modify the \cfuncdef macro so as to help us create the index. For this reason, it is important for all of us to start using the macros consistently. \MPI or \mpi Gives you MPI with the correct font. It is used to refer to the original MPI or MPI in general. Example usage: It is highly desirable that \MPI/ not use \MPI2 or \mpi2 Use like \MPI but when you want to specifically refer to MPI2. Not yet completely clear what MPI2 will be called. \discuss To highlight points that need further discussion. This text is removed in final versions if it gets left. Example of usage: \discuss{ We might want to modify the above function to make it more general } \missing Notes items that need to be added. This should be gone by the final draft. Example of usage: \missing{ We need to add a function to deal with issue foo } \change Used to put annonation in margin that a change has occured. Used in later drafts to make changes easier to see. Example of usage: \change This text is different \func or \mpifunc These are synonyms for the same macro. They are used to refer to mpi functions in normal text, when the abstract, language independent function is meant. \func takes one argument: the name of the function. Example usage from coll.tex: \func{MPI\_GATHERC} behaves like \func{MPI\_GATHER} restricted to contiguous buffers, and with the additional ... \mpiarg Use this to refer to (language independent) function arguments in normal text. Eg: \mpiarg{array\_of\_handles} \type Use this to refer to the type of (language independent) function arguments in normal text. Eg: \type{MPI_INTEGER} \ctype Just like \type, but for C types. Eg: \ctype{double} \ftype Just like \type, but for Fortran types. Eg: \ftype{INTEGER} \const Use this to refer to mpi-defined constants, in a language-indep way. Takes one arg, the name of the constant. Eg: \const{MPI_MINLOC} \cconst like \const, but for the C name of the constant Eg: \const{mpi_int} \fconst like \const, but for the Fortran name of the constant Eg: \const{MPI_INTEGER} \IN, \OUT, \INOUT Macros for our "IN", "OUT", "INOUT" types. \begin{funcdef} \end{funcdef} \funcarg The function definition environment. Language-indep version. \begin{funcdef} takes one additional argument, like this: \begin{funcdef}{MPI\_BCAST( bd\_handle, root, comm )} The argument is the function and arg list that we are defining. \end{funcdef} terminates the environment and takes no arguments. funcdef is basically a list environment, but instead of \item's inside, one says \funcarg. \funcarg takes three arguments, like this: \funcarg{\IN}{root}{rank of broadcast root} The three arguments are: datatype of arg, name of arg, and an English description of what the arg is about. \funcarg typesets these three quantities in different ways. So here is a complete funcdef taken from coll.tex: \begin{funcdef}{MPI\_BCAST( bd\_handle, root, comm )} \funcarg{\INOUT}{bd\_handle}{Handle to buffer descriptor. On root, this is the send buffer descriptor, elsewhere, this is the receive buffer descriptor.} \funcarg{\IN}{root}{rank of broadcast root} \funcarg{\IN}{comm}{communicator handle} \end{funcdef} \mpibind Give C binding of a function Example of usage from inquiry chapter: \mpibind{MPI\_Get\_processor\_name(char~*name, int~*resultlen)} \mpifbind Give the Fortran binding of a function \fargs Give Fortran arguments to a function Example of usage from inquiry chapter: \mpifbind{MPI\_GET\_PROCESSOR\_NAME( NAME, RESULTLEN, IERROR)\fargs CHARACTER*(*) NAME\\INTEGER RESULTLEN,IERROR} \begin{funcdefna} \end{funcdefna} Used to define functions without arguments. Example usage from inquiry chapter: \begin{funcdefna}{MPI\_WTIME()} \end{funcdefna} \mpibindnotint Used to give C definition of a function that does not return an int. Example usage from inquiry chapter: \mpibindnotint{double MPI\_Wtime(void)} \begin{constlist} \end{constlist} \constitem An environment for displaying lists of mpi constants. Usage: \begin{constlist} \constitem{MPI\_MAX}{ maximum} \constitem{MPI\_MIN}{ minimum} \constitem{MPI\_SUM}{ sum} \constitem{MPI\_PROD}{ product} \constitem{MPI\_AND}{ and (logical or bit-wise integer)} \constitem{MPI\_OR}{ or (logical or bit-wise integer)} \constitem{MPI\_XOR}{ xor (logical or bit-wise integer)} \constitem{MPI\_MAXLOC}{ maximum value and rank of process with maximum value (rank of first process with maximum value, in case of ties)} \constitem{MPI\_MINLOC}{ minimum value and rank of process with minimum value (rank of first process with minimum value, in case of ties)} \end{constlist} Note that \begin{constlist} takes no arguments, and that \constitem takes two arguments. I guess I don't have the C and Fortran versions of these yet. \begin{implementors} \end{implementors} Used for notes to implementors example of usage from pt2pt: \begin{implementors} Message context and other communicator information can be implemented as an additional tag field. It differs from the regular message tag in that wild card matching is not allowed on this field, and that value setting for this field is controlled by communicator manipulation functions. \end{implementors} \begin{rationale} \end{rationale} Used to describe why something was done the way it was. Example of usage from inquiry chapter: \begin{rationale} This function allows \MPI/ implementations that do process migration to return the current processor. Note that nothing in \MPI/ {\em requires} or defines process migration; this definition of \mpifunc{MPI\_GET\_PROCESSOR\_NAME} simply allows such an implementation. \end{rationale} \begin{users} \end{users} Used to tell advise to users. Example of usage from inquiry chapter: \begin{users} The user must provide at least \const{MPI\_MAX\_PROCESSOR\_NAME} space to write the processor name --- processor names can be this long. The user should examine the ouput argument, \mpiarg{resultlen}, to determine the actual length of the name. \end{users} \chapter Begin a new chapter. Example of usage from inquiry chapter. Note that you should define a label for the chapter and section. These should not be changed since they are used by others to make references. \chapter{MPI Environmental Management} \label{sec:environment} \label{chap:environment} \section Begin a new section. Example of usage from inquiry chapter: \section{Error handling} \label{sec:inquiry-error} \subsection Begin a new subsection. Example of usage from inquiry chapter: \subsection{Environmental Inquiries} \label{subsec:inquiry-inquiry} \subsubsection Begin a new subsubsection. Oh my! Example of usage from inquiry chapter: \subsubsection{Tag values} \paragraph Seems to be used to bold face beginning of paragraph. Example of usage from context chapter: \paragraph{Caching.} Communicators (see below) provide a ``caching'' mechanism that allows one to associate new attributes with communicators, on a par with \MPI/ built-in features. This can be used by advanced users to adorn communicators further, and by \MPI/ to implement some communicator functions. For example, the virtual-topology functions described in Chapter~\ref{chap:topol} are likely to be supported this way. \commentOut Allows one to remove large sections of text. \code ???? From owner-mpi-core@CS.UTK.EDU Mon Mar 27 22:09:25 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA04807; Mon, 27 Mar 1995 22:09:24 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id WAA05300; Mon, 27 Mar 1995 22:08:44 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 27 Mar 1995 22:08:42 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id WAA05293; Mon, 27 Mar 1995 22:08:41 -0500 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) id WAA15330; Mon, 27 Mar 1995 22:08:06 -0500 From: David Walker Message-Id: <199503280308.WAA15330@rios2.EPM.ORNL.GOV> To: tony@cs.msstate.edu cc: mpi-core@CS.UTK.EDU Subject: Re: multicast proposal from David Walker Date: Mon, 27 Mar 95 22:08:05 -0500 > this seems equivalent to me to ordaining that a communicator must > activate msend... > > It seems to violate the concept that the receiver did not need to > know that a sender might be a broadcaster?!? > > - -Tony My proposal was intended to be simple and requires only a small addition to the existing MPI specification. All that's really required is the addition of mpi_mrecv and mpi_imrecv. How do you violate a concept? David From owner-mpi-core@CS.UTK.EDU Fri Mar 31 06:14:40 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA07922; Fri, 31 Mar 1995 06:14:39 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id GAA14867; Fri, 31 Mar 1995 06:14:01 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 31 Mar 1995 06:13:56 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id GAA14775; Fri, 31 Mar 1995 06:13:47 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA02111 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Fri, 31 Mar 1995 12:13:35 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA19124; Fri, 31 Mar 1995 12:13:31 +0000 Date: Fri, 31 Mar 1995 12:13:31 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9503311113.AA19124@deneb.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: <199503241602.LAA15281@rios2.EPM.ORNL.GOV> (message from David Walker on Fri, 24 Mar 95 11:02:38 -0500) Subject: Re: multicast proposal from David Walker Reply-To: jim@meiko.com There are a number of extremely unpleasant issues in specifying/implementing a non co-operative multicast operation. (As opposed to the co-operative broadcast we already have). Objectives ========== As I see understand it the objectives are something like this 1) We want to be able to receive a multicast message with a "normal" receive. (In David's proposal this is a variant of a normal receive, "mpi_mrecv" which can presumably also be used to receive a message sent with mpi_Xsend (X != 'm')). 2) We want the multicast to be more efficient than linear, so we are expecting the implementation to use a tree or some other logarithmic structure. 3) We want the multicast operation to be "just like a set of send operations". In particular this implies that we expect ordering of messages sent with send and msend. Implementation ============== Now consider the implementation. Imagine (since this seems the most likely scheme) that we construct a tree rooted in the source node, and tag these messages with an extra flag in the envelope to say that they are multi-casts, and an additional "true source" field. Problems ======== Then 1) every receive has to check the flag (this violates one of the design aims of MPI, that simple things go fast, and you shouldn't have to pay for something you don't use). (David's proposal side-steps this by saying that only an mrecv has to do this check, however it is still necessary for every envelope to contain the flag, so you still pay something). 2) the ordering properties will be violated unless we collect acks back up the tree, and forbid more sends until all the receives have occurred. 3) I haven't managed to convince myself that once can guarantee deadlock freedom in the case of multiple simultaneous multicasts with different sources. (but neither have I created an example, so take this as general nervousness !) Violations of message order --------------------------- Consider a simple case with a two level tree 0 1 (more here but not relevant to discussion) 2 (ditto) with the following sequence of operations 0 1 2 recv mcast send 2 -------> 1st recv gets 2nd message recv recv forward 2nd recv gets first (multicast) message Because the intermediate node in the tree (1) doesn't forward until he chooses to receive (or perhaps just call an MPI function, if you want to add more cost to LOTS of things), we have lost the ordering of messages between 0 and 2. Conclusions =========== 1) I'd like to quote Paul Pierce "I've implemented this three times, and it's always horrible". 2) If the relaxed ordering is sufficient and it is acceptable to use different send/recv operations, then it should be possible to implement these functions entirely on top of the existing MPI functionality. I'd therefore propose that someone do this and make it available. With any luck the MPI forum can then forget about this particular can of worms. -- Jim From owner-mpi-core@CS.UTK.EDU Fri Mar 31 09:09:35 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA10261; Fri, 31 Mar 1995 09:09:34 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA25635; Fri, 31 Mar 1995 09:09:31 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 31 Mar 1995 09:09:25 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id JAA25608; Fri, 31 Mar 1995 09:09:22 -0500 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) id JAA10831; Fri, 31 Mar 1995 09:08:09 -0500 From: David Walker Message-Id: <199503311408.JAA10831@rios2.EPM.ORNL.GOV> To: jim@meiko.com Cc: mpi-core@CS.UTK.EDU Subject: Re: multicast proposal from David Walker In-reply-to: (Your message of Fri, 31 Mar 95 12:13:31 GMT.) <9503311113.AA19124@deneb.co.uk> Date: Fri, 31 Mar 95 09:08:09 -0500 Jim wrote: > As I see understand it the objectives are something like this > > 1) We want to be able to receive a multicast message with a "normal" > receive. (In David's proposal this is a variant of a normal > receive, "mpi_mrecv" which can presumably also be used to receive a > message sent with mpi_Xsend (X != 'm')). > > 2) We want the multicast to be more efficient than linear, so we are > expecting the implementation to use a tree or some other > logarithmic structure. > > 3) We want the multicast operation to be "just like a set of send > operations". > In particular this implies that we expect ordering of messages sent > with send and msend. Do we really want to require (3)? In particular is a separate mcast routine needed? It seems to me we just need mpi_bcast and mpi_Xsend (as in current MPI spec), to which we add mpi_mrecv and mpi_imrecv, both of which can receive from mpi_bcast and mpi_Xsend. The example of violation of message order that Jim gave is a variation of Example 4.26 in the MPI spec. If we do away with mpi_mcast (i.e. make Jim's example the following: 0 1 2 recv bcast send 2----------------------->1st recv gets 2nd message mrecv mrecv where the mrecv's receive the bcast data then there's no problem. David From owner-mpi-core@CS.UTK.EDU Fri Mar 31 09:39:09 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA10851; Fri, 31 Mar 1995 09:39:09 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA28646; Fri, 31 Mar 1995 09:39:33 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 31 Mar 1995 09:39:31 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA28636; Fri, 31 Mar 1995 09:39:23 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA06773 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Fri, 31 Mar 1995 15:39:14 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA19263; Fri, 31 Mar 1995 15:39:11 +0000 Date: Fri, 31 Mar 1995 15:39:11 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9503311439.AA19263@deneb.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: <199503311408.JAA10831@rios2.EPM.ORNL.GOV> (message from David Walker on Fri, 31 Mar 95 09:08:09 -0500) Subject: Re: multicast proposal from David Walker Reply-To: jim@meiko.com > Do we really want to require (3)? In particular is a separate mcast > routine needed? It seems to me we just need mpi_bcast and mpi_Xsend (as > in current MPI spec), to which we add mpi_mrecv and mpi_imrecv, both of > which can receive from mpi_bcast and mpi_Xsend. If we're going to do this at all, we absolutely require a separate multi-send operation. The current broadcast is either a send or a receive, depending on whether you are the source process or not. It is amenable to good implementations, because you KNOW what is happening, and can implement it with barrier semantics. (For instance we can use the broadcast in our network to achieve a broadcast time which is less than the time to send a single point to point tagged message). Mixing the collective broadcast with the point to point receive would remove this possibility, and hopelessly entangle the implementation of collective operations with that of point to point messages. This would be awful. Also remember that the current broadcast does not have a tag. Therefore it is hard to see how to integrate it into the point to point, where a tag is required on all sends. > The example of violation of message order that Jim gave is a variation > of Example 4.26 in the MPI spec. > If we do away with mpi_mcast (i.e. make Jim's example the following): > > 0 1 2 > recv > bcast > > send 2----------------------->1st recv gets 2nd message > > mrecv > > mrecv > > where the mrecv's receive the bcast data then there's no problem. All you seem to be doing is saying that multicast messages are unordered with respect to point to point ones. That's a possible position to take, but seems likely to lead to obscure bugs and portability problems, and be hard to explain to users. -- Jim From owner-mpi-core@CS.UTK.EDU Fri Mar 31 11:43:45 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA14466; Fri, 31 Mar 1995 11:43:45 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA10836; Fri, 31 Mar 1995 11:43:32 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 31 Mar 1995 11:43:30 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from rios2.EPM.ORNL.GOV by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA10823; Fri, 31 Mar 1995 11:43:29 -0500 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) id LAA05263; Fri, 31 Mar 1995 11:42:17 -0500 From: David Walker Message-Id: <199503311642.LAA05263@rios2.EPM.ORNL.GOV> To: jim@meiko.com Cc: mpi-core@CS.UTK.EDU Subject: Re: multicast proposal from David Walker In-reply-to: (Your message of Fri, 31 Mar 95 15:39:11 GMT.) <9503311439.AA19263@deneb.co.uk> Date: Fri, 31 Mar 95 11:42:17 -0500 JIm wrote: > we absolutely require a separate > multi-send operation. The current broadcast is either a send or a > receive, depending on whether you are the source process or not. It is > amenable to good implementations, because you KNOW what is happening, > and can implement it with barrier semantics. (For instance we can use > the broadcast in our network to achieve a broadcast time which is less > than the time to send a single point to point tagged message). > > Mixing the collective broadcast with the point to point receive would > remove this possibility, and hopelessly entangle the implementation of > collective operations with that of point to point messages. This would > be awful. > >Also remember that the current broadcast does not have a tag. Therefore >it is hard to see how to integrate it into the point to point, where a >tag is required on all sends. OK, then accepting that mcasts are necessary for the reasons outlined by Jim we have: 0 1 2 recv mcast send 2-----------------------1st recv gets 2nd message mrecv mrecv > All you seem to be doing is saying that multicast messages are > unordered with respect to point to point ones. That's a possible > position to take, but seems likely to lead to obscure bugs and > portability problems, and be hard to explain to users. But we already impose the same potential for obscure bugs and portability problems in the interaction of broadcasts with point-to-point messages. It seems not too unreasonable to impose the same constraints on mcast. From owner-mpi-core@CS.UTK.EDU Fri Mar 31 17:25:55 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA29793; Fri, 31 Mar 1995 17:25:54 -0500 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA13232; Fri, 31 Mar 1995 17:25:18 -0500 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 31 Mar 1995 17:25:16 EST Errors-to: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA13219; Fri, 31 Mar 1995 17:25:14 -0500 Received: from ark.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA19297; Fri, 31 Mar 95 14:24:31 PST Received: by ark.SSD.intel.com (4.1/SMI-4.1) id AA25067; Fri, 31 Mar 95 14:20:33 PST Date: Fri, 31 Mar 95 14:20:33 PST From: Paul Pierce Message-Id: <9503312220.AA25067@ark.SSD.intel.com> To: jim@meiko.com, mpi-core@CS.UTK.EDU Subject: Re: multicast proposal from David Walker Thanks, Jim. I would like to note that the NX "global send" implementations (using our terminology) did not solve the ordering problem. This does on occaision trip up an unsuspecting user, but since its not practical to fix it within the given NX API with decent performance the user must always work around it. This is one of the areas where MPI is a nice step forward from NX, since it makes possible a cleaner, more consistent implementation. The way I pitch MPI within Intel SSD is: MPI is the new NX that no NX ever could be, because MPI is not constrained by backward compatibility. Paul From owner-mpi-core@CS.UTK.EDU Tue Apr 4 11:16:08 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA23544; Tue, 4 Apr 1995 11:16:08 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA03968; Tue, 4 Apr 1995 11:15:33 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 4 Apr 1995 11:15:32 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from convex.convex.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA03954; Tue, 4 Apr 1995 11:15:29 -0400 Received: from hydra.convex.com by convex.convex.com (8.6.4.2/1.35) id KAA05782; Tue, 4 Apr 1995 10:14:54 -0500 Received: by hydra.convex.com (5.64/1.28) id AA11828; Tue, 4 Apr 95 10:14:53 -0500 From: sfleisch@hydra.convex.com (Steve Fleischman) Message-Id: <9504041514.AA11828@hydra.convex.com> Subject: allgatherv and reduce_scatter To: mpi-core@CS.UTK.EDU Date: Tue, 4 Apr 95 10:14:53 CDT X-Mailer: ELM [version 2.3 PL11] Here is a proposal for having a single in/out buffer for collective communication routines. I will just discuss Reduce_scatter and Allgatherv. The idea is that the input buffer and the output buffer are the same and that the final result goes into this buffer. In many cases this is the desired behavior since you are updating or accumulating into an array and want a completed version for subsequent use. This avoids the additional copy. We can support this in the standard without adding new routines or changing the argument list of old routines. int MPI_Reduce_scatter ( sendbuf, recvbuf, recvcnts, datatype, op, comm ) Modify the standard to permit sendbuf to be equal to recvbuf in which case the piece defined by recvcnts[my_rank] and datatype will have the fully combined values and that the values of the elements of the rest of the buffer are undefined. int MPI_Allgatherv ( sendbuf, sendcount, sendtype, recvbuf, recvcounts, displs, recvtype, comm ) Similarly for Allgatherv, modify the standard to permit sendbuf=recvbuf. In this case sendcount and sendtype are ignored and on input each process, j, has its data to be distributed in the jth block of the buffer. Stephen Fleischman From owner-mpi-core@CS.UTK.EDU Tue Apr 4 13:28:49 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA26780; Tue, 4 Apr 1995 13:28:49 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA13453; Tue, 4 Apr 1995 13:28:57 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 4 Apr 1995 13:28:54 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hades.osc.epsilon.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA13434; Tue, 4 Apr 1995 13:28:52 -0400 Received: by hades.osc.epsilon.com (4.1/3.1.-Epsilon Data Management) id AA23860; Tue, 4 Apr 95 13:00:40 EDT Received: from kosmos.quantum.epsilon.com(159.127.20.130) by hades.osc.epsilon.com via smap (V1.3) id sma023836; Tue Apr 4 13:00:22 1995 Received: from pluto.epsilon.com by kosmos.epsilon.com (4.1/SMI-4.1) id AA09075; Tue, 4 Apr 95 12:58:46 EDT Date: Tue, 4 Apr 95 12:58:46 EDT From: moose@epsilon.com (Adam Greenberg) Message-Id: <9504041658.AA09075@kosmos.epsilon.com> Received: by pluto.epsilon.com (4.1/SMI-4.1) id AA14750; Tue, 4 Apr 95 12:58:45 EDT To: sfleisch@hydra.convex.com Cc: mpi-core@CS.UTK.EDU In-Reply-To: <9504041514.AA11828@hydra.convex.com> (sfleisch@hydra.convex.com) Subject: Re: allgatherv and reduce_scatter Errors-To: owner-mpi-core@CS.UTK.EDU X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 4 Apr 1995 11:15:32 EDT Errors-To: owner-mpi-core@CS.UTK.EDU From: sfleisch@hydra.convex.com (Steve Fleischman) Date: Tue, 4 Apr 95 10:14:53 CDT X-Mailer: ELM [version 2.3 PL11] Here is a proposal for having a single in/out buffer for collective communication routines. I will just discuss Reduce_scatter and Allgatherv. The idea is that the input buffer and the output buffer are the same and that the final result goes into this buffer. In many cases this is the desired behavior since you are updating or accumulating into an array and want a completed version for subsequent use. This avoids the additional copy. Does this avoid the additional copy? Methinks it merely hides it. I imagine the implementation will still need a work space and/or landing zone that is/are independent from the source buffer. If this is the case, then the result will probably still be copied back into the source or receive buffer under the hood. moose We can support this in the standard without adding new routines or changing the argument list of old routines. int MPI_Reduce_scatter ( sendbuf, recvbuf, recvcnts, datatype, op, comm ) Modify the standard to permit sendbuf to be equal to recvbuf in which case the piece defined by recvcnts[my_rank] and datatype will have the fully combined values and that the values of the elements of the rest of the buffer are undefined. int MPI_Allgatherv ( sendbuf, sendcount, sendtype, recvbuf, recvcounts, displs, recvtype, comm ) Similarly for Allgatherv, modify the standard to permit sendbuf=recvbuf. In this case sendcount and sendtype are ignored and on input each process, j, has its data to be distributed in the jth block of the buffer. Stephen Fleischman -- *-----------------------------------------------------------------------------* The opinions expressed herein are my own. They do not represent Epsilon in any way. Adam Greenberg moose@epsilon.com Quadruped Systems, Inc. 617 273 2630 x6455 22 Laurel St. Somerville, MA. 02143-2848 From owner-mpi-core@CS.UTK.EDU Tue Apr 4 14:18:23 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA27714; Tue, 4 Apr 1995 14:18:23 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA16305; Tue, 4 Apr 1995 14:18:44 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 4 Apr 1995 14:18:39 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from sgi.sgi.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id OAA16290; Tue, 4 Apr 1995 14:18:31 -0400 Received: from cthulhu.engr.sgi.com by sgi.sgi.com via ESMTP (950221.405.SGI.8.6.10/910110.SGI) for <@sgi.sgi.com:mpi-core@CS.UTK.EDU> id LAA14497; Tue, 4 Apr 1995 11:18:28 -0700 Received: from mrjones.engr.sgi.com by cthulhu.engr.sgi.com via ESMTP (941129.SGI.8.6.9/911001.SGI) for <@cthulhu.engr.sgi.com:mpi-core@CS.UTK.EDU> id LAA00076; Tue, 4 Apr 1995 11:18:14 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) for mpi-core@CS.UTK.EDU id LAA23664; Tue, 4 Apr 1995 11:18:05 -0700 From: "Eric Salo" Message-Id: <9504041117.ZM23662@mrjones.engr.sgi.com> Date: Tue, 4 Apr 1995 11:17:55 -0700 In-Reply-To: sfleisch@hydra.convex.com (Steve Fleischman) "allgatherv and reduce_scatter" (Apr 4, 10:14am) References: <9504041514.AA11828@hydra.convex.com> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: mpi-core@CS.UTK.EDU Subject: Re: allgatherv and reduce_scatter Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii > Here is a proposal for having a single in/out buffer for collective > communication routines. I will just discuss Reduce_scatter and > Allgatherv. The idea is that the input buffer and the output buffer > are the same and that the final result goes into this buffer. In many > cases this is the desired behavior since you are updating or accumulating > into an array and want a completed version for subsequent use. > This avoids the additional copy. Please supply an algorithm that demonstrates how this can work without requiring a hidden temporary buffer. If that can be done, I would strongly support such a proposal. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From owner-mpi-core@CS.UTK.EDU Tue Apr 4 14:24:33 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA27818; Tue, 4 Apr 1995 14:24:32 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA16681; Tue, 4 Apr 1995 14:25:10 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 4 Apr 1995 14:25:07 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from convex.convex.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id OAA16671; Tue, 4 Apr 1995 14:25:05 -0400 Received: from hydra.convex.com by convex.convex.com (8.6.4.2/1.35) id NAA16390; Tue, 4 Apr 1995 13:24:33 -0500 Received: by hydra.convex.com (5.64/1.28) id AA19059; Tue, 4 Apr 95 13:24:32 -0500 From: sfleisch@hydra.convex.com (Steve Fleischman) Message-Id: <9504041824.AA19059@hydra.convex.com> Subject: Re: allgatherv and reduce_scatter (fwd) To: mpi-core@CS.UTK.EDU Date: Tue, 4 Apr 95 13:24:31 CDT X-Mailer: ELM [version 2.3 PL11] Forwarded message: > From owner-mpi-core@CS.UTK.EDU Tue Apr 4 12:38:40 1995 > X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 4 Apr 1995 13:28:54 EDT > Errors-To: owner-mpi-core@CS.UTK.EDU > Date: Tue, 4 Apr 95 12:58:46 EDT > From: moose@epsilon.com (Adam Greenberg) > Message-Id: <9504041658.AA09075@kosmos.epsilon.com> > To: sfleisch@hydra.convex.com > Cc: mpi-core@CS.UTK.EDU > In-Reply-To: <9504041514.AA11828@hydra.convex.com> (sfleisch@hydra.convex.com) > Subject: Re: allgatherv and reduce_scatter > > Errors-To: owner-mpi-core@CS.UTK.EDU > X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 4 Apr 1995 11:15:32 EDT > Errors-To: owner-mpi-core@CS.UTK.EDU > From: sfleisch@hydra.convex.com (Steve Fleischman) > Date: Tue, 4 Apr 95 10:14:53 CDT > X-Mailer: ELM [version 2.3 PL11] > > Here is a proposal for having a single in/out buffer for collective > communication routines. I will just discuss Reduce_scatter and > Allgatherv. The idea is that the input buffer and the output buffer > are the same and that the final result goes into this buffer. In many > cases this is the desired behavior since you are updating or accumulating > into an array and want a completed version for subsequent use. > This avoids the additional copy. > > Does this avoid the additional copy? Methinks it merely hides it. I imagine > the implementation will still need a work space and/or landing zone that is/are > independent from the source buffer. If this is the case, then the result will > probably still be copied back into the source or receive buffer under the hood. > > moose > In the current standard, the sendbuf is not usable as a source buffer since it must be returned unchanged. Allgatherv, can be implemented without a work space even with point to point communication. Reduce_scatter would require a work space with point to point communication. However, you still need this work space anyway with the current standard. Moreover, if reduce_scatter is implemented internally with shared memory, on machines that support it, the intermediate work space is not required. The additonal copy I am referring to is the the copy of the recvbuf back to the original array that sendbuf is a part of. So to summarize: there is no hiding of an additional copy. The work space requirement is the same regardless due to sendbuf being untouchable in the current standard. Best regards, Stephen Fleischman From owner-mpi-core@CS.UTK.EDU Tue Apr 4 15:11:28 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA29481; Tue, 4 Apr 1995 15:11:27 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA21174; Tue, 4 Apr 1995 15:11:57 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 4 Apr 1995 15:11:56 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from convex.convex.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA21163; Tue, 4 Apr 1995 15:11:53 -0400 Received: from hydra.convex.com by convex.convex.com (8.6.4.2/1.35) id OAA19156; Tue, 4 Apr 1995 14:11:15 -0500 Received: by hydra.convex.com (5.64/1.28) id AA20501; Tue, 4 Apr 95 14:11:13 -0500 From: sfleisch@hydra.convex.com (Steve Fleischman) Message-Id: <9504041911.AA20501@hydra.convex.com> Subject: Re: allgatherv and reduce_scatter To: mpi-core@CS.UTK.EDU Date: Tue, 4 Apr 95 14:11:12 CDT X-Mailer: ELM [version 2.3 PL11] > > > Here is a proposal for having a single in/out buffer for collective > > communication routines. I will just discuss Reduce_scatter and > > Allgatherv. The idea is that the input buffer and the output buffer > > are the same and that the final result goes into this buffer. In many > > cases this is the desired behavior since you are updating or accumulating > > into an array and want a completed version for subsequent use. > > This avoids the additional copy. > > Please supply an algorithm that demonstrates how this can work without > requiring a hidden temporary buffer. If that can be done, I would strongly > support such a proposal. > > -- > Eric Salo Silicon Graphics Inc. "Do you know what the > (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just > salo@sgi.com Mountain View, CA 94043-1389 before he died?" > Whatever you say: The following functions are for illustrative purposes. They only do power of 2 number of processors. I have routines that do non-power of 2 written in C. The function mps_gdsum corresponds in function to reduce_scatter and mps_dbcast to allgatherv. The gdsum routine does indeed require a work array. However, as I've stated in a reply to a previous response, you would still need this array with the current standard due to the requirement that sendbuf remain the same upon return. Again, the avoided copy I'm referring to is when you want the results in sendbuf since you are updating that array. Note that in dbcast there is no temp array. Furthermore, I am working on a set of routines that use shared memory internally. These routines will not require the temporary array even for reduce_scatter. Best regards, Stephen Fleischman C mpi based distributed collective communication routines. subroutine mps_gdsum(node,nproc,msgtype,x,temp,iparpt,err) implicit none include 'mpif.h' integer ncube,node,nproc,req(2),status_array(2) integer msgtype real*8 x(0:*),temp(0:*) integer iparpt(0:*) logical err C--- integer other,ncycles,k,bit,cs,cr,i integer ns,nr,status integer iceil err=.false. if (nproc.le.1) return ncycles = iceil(log(dble(nproc))/log(2.0D0)) ncube = 2**ncycles if(ncube.ne.nproc) then write(7,*) 'number of processors is not a power of 2' call mpi_abort(MPI_COMM_WORLD,MPI_ERR_ARG,status) endif bit=ncube/2 do k = 0,ncycles-1 other=ieor(node,bit) C send chunk cs = ishft(other,-((ncycles-1)-k))*bit ns = iparpt(cs+bit)-iparpt(cs) call mpi_isend(x(iparpt(cs)),ns, @ MPI_DOUBLE_PRECISION,other,msgtype,MPI_COMM_WORLD, @ req(1),status) C recv chunk cr = ishft(node,-((ncycles-1)-k))*bit nr = iparpt(cr+bit)-iparpt(cr) call mpi_irecv(temp(iparpt(cr)),nr, @ MPI_DOUBLE_PRECISION,other,msgtype,MPI_COMM_WORLD, @ req(2),status) C now wait for all messages to complete before we can use the buffers. call mpi_waitall(2,req,status_array,status) do i=iparpt(cr),iparpt(cr+bit)-1 x(i)=x(i)+temp(i) enddo bit = ishft(bit,-1) enddo return end subroutine mps_dbcast(node,nproc,msgtype,x,iparpt,err) implicit none include 'mpif.h' integer node,nproc,ncube,req(3),status_array(3) integer msgtype real*8 x(0:*) integer iparpt(0:*) logical err C--- integer other,ncycles,k,bit,cs,cr integer ns,nr,status integer iceil err=.false. if (nproc.le.1) return ncycles = iceil(log(dble(nproc))/log(2.0D0)) ncube = 2**ncycles if(ncube.ne.nproc) then write(7,*) 'number of processors is not a power of 2' call mpi_abort(MPI_COMM_WORLD,MPI_ERR_ARG,status) endif bit=1 do k = 0,ncycles-1 other=ieor(node,bit) C send chunk cs = ishft(node,-k)*bit C pack send buffer ns = iparpt(cs+bit)-iparpt(cs) call mpi_isend(x(iparpt(cs)),ns, @ MPI_DOUBLE_PRECISION,other,msgtype,MPI_COMM_WORLD, @ req(1),status) C recv chunk cr = ishft(other,-k)*bit nr = iparpt(cr+bit)-iparpt(cr) call mpi_irecv(x(iparpt(cr)),nr, @ MPI_DOUBLE_PRECISION,other,msgtype,MPI_COMM_WORLD, @ req(2),status) call mpi_waitall(2,req,status_array,status) bit = ishft(bit,1) enddo return 9999 err=.true. return end integer function iceil(x) c--- CEILING FUNCTION implicit none real*8 x integer ix ix = int(x) if(x.gt.dble(ix)) ix=ix+1 iceil = ix return end From owner-mpi-core@CS.UTK.EDU Mon Apr 10 19:09:37 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA22539; Mon, 10 Apr 1995 19:09:37 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA05743; Mon, 10 Apr 1995 19:09:18 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 10 Apr 1995 19:09:16 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA05735; Mon, 10 Apr 1995 19:09:11 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 7635; Mon, 10 Apr 95 19:09:05 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 3289; Mon, 10 Apr 1995 19:09:05 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 10 Apr 95 19:09:03 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA27235; Mon, 10 Apr 1995 19:09:02 -0400 Message-Id: <9504102309.AA27235@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Cc: frankeh@watson.ibm.com Subject: MPI corrections Reply-To: snir@watson.ibm.com Date: Mon, 10 Apr 1995 19:09:01 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: There are two issues that were raised in the MPI implementor forum which need our attention. 1. Collective reduce/scan operations with user-defined datatype. Currently, the user-defined operations are of the form foo(invec, inoutvec, len, datatype). Datatype is arbitrary. The MPI system invokes the function by passing in invec and inoutvec the addresses of two communication buffers with a layout as defined by len and datatype. Problem: many implementations may want to use internally allocated buffers to do the reduction (e.g., to allow a fan-in of more than two, or for receiving incoming messages in arbitrary order). However, one cannot figure out how much space to allocate, in general: this, since the datatype may have an "unnatural" extent, and have data below the LB or above the UB. Solution: disallow unnatural extents in this instance. I.e., disallow datatypes with user-provided MPI_LB or MPI_UB. Alternative solution: add functions that return the "true" lower bound and upper bound of a datatype. 2. Keys in MPI_COMM_SPLIT. Problem: we allow a key to have value MPI_UNDEFINED, but we have not restricted the range of "defined" keys. Solution: restrict legal values of key to be nonnegative. Comments? From owner-mpi-core@CS.UTK.EDU Mon Apr 10 23:47:23 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA26451; Mon, 10 Apr 1995 23:47:23 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id XAA23761; Mon, 10 Apr 1995 23:48:07 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 10 Apr 1995 23:48:05 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id XAA23745; Mon, 10 Apr 1995 23:48:03 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA17895 for ; Mon, 10 Apr 1995 22:47:56 -0500 Message-Id: <199504110347.WAA17895@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: MPI forum meeting reservations. Date: Mon, 10 Apr 1995 22:47:56 -0500 From: Rusty Lusk The next MPI Forum meeting is April 24-26 at the O'Hare Hilton. The announcement below is identical to the last one, except that the date has changed. PLEASE MAKE YOUR RESERVATIONS WITH THE HOTEL IMMEDIATELY TO GET THE MEETING RATE. THE DEADLINE IS APRIL 12. The O'Hare Hilton is the hotel *inside* the airport. You just walk across the street from the baggage-claim level. A block of rooms has been reserved, which will be held until April 12, so please make your reservations with the Hotel before then. The phone number for the hotel is (312) 686-8000 (FAX (312) 601-2876). Mention the *MPI Forum* to get the special rate, which is $90.50 + tax per night. We will charge the usual $75 registration to take care of the meeting room, lunch on Tuesday, and coffee, etc. The meeting will start at 1:30 on Monday to allow people to fly in that morning, and end at noon on Wednesday. Please be certain to mention the MPI FORUM, or have your agent do it, so that we get credit for the reservation. This brings down the cost of the meeting rooms. Rusty From owner-mpi-core@CS.UTK.EDU Tue Apr 11 09:25:43 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA10748; Tue, 11 Apr 1995 09:25:43 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA03545; Tue, 11 Apr 1995 09:26:25 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 11 Apr 1995 09:26:24 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id JAA03536; Tue, 11 Apr 1995 09:26:23 -0400 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA21329; Tue, 11 Apr 1995 08:23:33 -0500 Message-Id: <199504111323.IAA21329@antares.mcs.anl.gov> To: snir@watson.ibm.com cc: mpi-core@CS.UTK.EDU, frankeh@watson.ibm.com Subject: Re: MPI corrections In-Reply-to: Message from Marc Snir of "Mon, 10 Apr 1995 19:09:01 -0400." <9504102309.AA27235@snir.watson.ibm.com> Date: Tue, 11 Apr 1995 08:23:32 -0500 From: William Gropp | :-) :-) :-) *** (-: (-: (-: | | There are two issues that were raised in the MPI implementor forum which need | our attention. | | 1. Collective reduce/scan operations with user-defined datatype. | | Currently, the user-defined operations are of the form foo(invec, inoutvec, | len, datatype). Datatype is arbitrary. The MPI system invokes the function by | passing in invec and inoutvec the addresses of two communication buffers with | a layout as defined by len and datatype. | | Problem: many implementations may want to use internally allocated buffers to | do the reduction (e.g., to allow a fan-in of more than two, or for receiving | incoming messages in arbitrary order). However, one cannot figure out how | much space to allocate, in general: this, since the datatype may have an | "unnatural" extent, and have data below the LB or above the UB. | | Solution: disallow unnatural extents in this instance. I.e., disallow | datatypes with user-provided MPI_LB or MPI_UB. | This would prevent the use of MPI_UB to create a "vector" data type with fixed stride but a number of elements given by the count (in the argument to the MPI_Reduce/Send/ect.). | Alternative solution: add functions that return the "true" lower bound and | upper bound of a datatype. | I believe that using the extent is the wrong approach, since the extent may be arbitrarily greater than the size. One solution that is consistent with the predefined operations is to say that (a) the operations will be called with contiguous items and/or (b) the operation will be called once for each contiguous part. This has its own problems, of course... | | 2. Keys in MPI_COMM_SPLIT. | | Problem: we allow a key to have value MPI_UNDEFINED, but we have not | restricted the range of "defined" keys. | | Solution: restrict legal values of key to be nonnegative. I agree with this. Bill From owner-mpi-core@CS.UTK.EDU Tue Apr 11 09:42:38 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA11618; Tue, 11 Apr 1995 09:42:38 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA04882; Tue, 11 Apr 1995 09:43:43 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 11 Apr 1995 09:43:42 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA04869; Tue, 11 Apr 1995 09:43:38 -0400 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA02262 (5.65c8/IDA-1.4.4 for ); Tue, 11 Apr 1995 15:43:33 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA18127; Tue, 11 Apr 1995 15:43:35 +0200 Date: Tue, 11 Apr 1995 15:43:35 +0200 From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9504111343.AA18127@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: Proposal for pre-defined attribute Cc: gmap10@f1neuman.gmd.de At the last meeting I promised to send a proposal for a function by which a process can determine if another process runs on a processor with the same data representation or not. Here is my Proposal: Add a pre-defined attribute MPI_DATA_REPRESENTATION to the MPI_COMM_WORLD communicator which each process can look up (analogous to MPI_HOST or MPI_TAG_UB). In two processes this attribute gives the same (integer) value if and only if the data representations for all MPI data types match. The attribute value does not carry any other information. With this added functionality it is easy to create a complete data representation table by applying a gather operation to the attributes looked up in the single processes. Rationale: MPI provides automatic conversion of data representations for messages between different processors. For messages containing more than one basic data type, there are two protocols: derived datatypes or the pack/unpack mechanism. Both protocols are likely to make the send/receive operation considerably slower as compared to the transmission of a byte string without conversion. This is true even in the case that no conversion is necessary, because either the derived datatype has to be created, or the message has to be copied more often. If the application process can find out at run time whether the data representation in the receiver process matches or not, it can choose the optimal strategy accordingly. The complicated protocols are only used if necessary, and can be avoided in most cases. Example: The requirement for this functionality came up when we developed an automatic tool which translates PARMACS or PVM application programs into MPI. When this tool encounters a send operation, it does not know if it can reuse an old derived data type, so it has to create a new one for every single send. With the new MPI_DATA_REPRESENTATION attribute, the tool can insert code for both cases (create a derived datatype / send the message as a simple byte string), and branch into the appropriate code at run time. - Rolf Hempel From owner-mpi-core@CS.UTK.EDU Tue Apr 11 10:09:46 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA12201; Tue, 11 Apr 1995 10:09:46 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA06809; Tue, 11 Apr 1995 10:10:34 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 11 Apr 1995 10:10:29 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA06764; Tue, 11 Apr 1995 10:10:14 -0400 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA12025 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Tue, 11 Apr 1995 15:10:06 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA22217; Tue, 11 Apr 1995 15:10:04 +0000 Date: Tue, 11 Apr 1995 15:10:04 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9504111410.AA22217@deneb.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: <9504111343.AA18127@f1neuman.gmd.de> (Rolf.Hempel@gmd.de) Subject: Re: Proposal for pre-defined attribute Rolf's proposal is OK provided that the MPI environment within which it is implemented is provided by a single MPI implementation, i.e. the MPI implementer has control of the values returned at each node. This is the current state of the world, where there is no "internetting" of different MPI implementations. It is not clear that this will remain the case, however. As proposed this attribute may cause a problem when (or if) mixed MPI implementations are produced. (e.g. consider IBM's MPI implementation on an SP/n mixed with MPIch on a workstation network). When this happens, then these "data representation numbers" need to be issued coherently across the whole user visible set of processes. It is unclear how to do this. (Though it's also unclear how to implement such a system anyway !) Note, also, that the target architecture on which an implementation is running is (in general) insufficient information to allow Rolf to achieve his objectives. In particular compiler flags may also have an effect (e.g. flags which treat int as 32 or 64 bits will certainly affect the issue). As a separate issue, this attribute (however it is defined) should be an attribute of MPI_COMM_SELF, not MPI_COMM_WORLD, since it is a property only of the local process (and may well be different in different processes). -- Jim From owner-mpi-core@CS.UTK.EDU Tue Apr 11 10:47:06 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA13013; Tue, 11 Apr 1995 10:47:06 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA09781; Tue, 11 Apr 1995 10:47:38 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 11 Apr 1995 10:47:36 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA09773; Tue, 11 Apr 1995 10:47:33 -0400 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA05145 (5.65c8/IDA-1.4.4 for ); Tue, 11 Apr 1995 16:47:28 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA18527; Tue, 11 Apr 1995 16:47:29 +0200 Date: Tue, 11 Apr 1995 16:47:29 +0200 From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9504111447.AA18527@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: Re: Proposal for pre-defined attribute I agree with Jim Cownie that the new attribute should be attached to MPI_COMM_SELF, not MPI_COMM_WORLD. If different MPI implementations are to be "internetted", the definition of consistent attribute values will, of course, be a problem. However, I think that the brave implementers who join their libraries will face much harder problems than agreeing on consistent attribute values for MPI_DATA_REPRESENTATION. So, this should not be too big an obstacle. - Rolf Hempel From owner-mpi-core@CS.UTK.EDU Tue Apr 11 17:19:19 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA21903; Tue, 11 Apr 1995 17:19:18 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA13293; Tue, 11 Apr 1995 17:20:02 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 11 Apr 1995 17:20:00 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA13281; Tue, 11 Apr 1995 17:19:50 -0400 Received: from eel.ssd.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA00918; Tue, 11 Apr 95 14:19:02 PDT Date: Tue, 11 Apr 95 14:19:01 PDT Message-Id: <9504112119.AA00918@SSD.intel.com> X-Sender: prp@eudora.ssd.intel.com X-Mailer: Windows Eudora Version 2.0.3 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: Rolf.Hempel@gmd.de (Rolf Hempel), mpi-core@CS.UTK.EDU From: prp@ssd.intel.com (Paul Pierce) Subject: Re: Proposal for pre-defined attribute Cc: gmap10@f1neuman.gmd.de At 03:43 PM 4/11/95 +0200, Rolf Hempel wrote: >Proposal: > >Add a pre-defined attribute MPI_DATA_REPRESENTATION to the >MPI_COMM_[SELF] communicator which each process can look up >(analogous to MPI_HOST or MPI_TAG_UB). In two processes this attribute >gives the same (integer) value if and only if the data representations >for all MPI data types match. ... such that contiguous messages can be transferred correctly using the MPI_BYTE datatype regardless of original datatype. Right? Is this the only ramification of same data representation at the MPI interface level? If so, it should be called out specifically, especially since its the reason for making the proposal. I'm not too worried about whether this can be implemented easily in all cases. In homogeneous implementations it is a constant, all other implementations can return distinct values in all processes (say, rank in MPI_COMM_WORLD) with only a potential performance impact. If the realized performance impact is too great one can go back and figure out how to return useful values. Interoperating implementations which otherwise support it can encode an implementation unique version number in the value to be safe. Someone could maintain a registry of comprehensive data representation schemes and associated values, to foster better interoperability. In case it comes up, I recommend that the value be strictly non-negative. MPI_UNDEFINED is not a valid value, because in the cases where MPI_UNDEFINED could apply (attribute not implemented or unknown) you want all values to be different. -------------------------------------------------------------------------------- Paul Pierce prp@ssd.intel.com From owner-mpi-core@CS.UTK.EDU Tue Apr 11 19:13:50 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA23970; Tue, 11 Apr 1995 19:13:49 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA20843; Tue, 11 Apr 1995 19:14:13 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 11 Apr 1995 19:14:12 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from win233.nas.nasa.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id TAA20836; Tue, 11 Apr 1995 19:14:10 -0400 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.8.2/NAS.6.1) id QAA14816; Tue, 11 Apr 1995 16:14:08 -0700 Date: Tue, 11 Apr 1995 16:14:08 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199504112314.QAA14816@win233.nas.nasa.gov> To: Rolf.Hempel@gmd.de (Rolf Hempel) Subject: Re: Proposal for pre-defined attribute Cc: wcs@win233.nas.nasa.gov, mpi-core@CS.UTK.EDU I think I must be missing something about this proposal. Neither the rationale nor the example seem very strong to me. In particular, the purpose of the proposed mechanism seems to be to "outwit" an MPI implementation by allowing/encouraging an application to bypass standard MPI mechanisms. My understanding is that MPI was carefully designed to allow very efficient implementations - if there is a problem with the current implementations or some structural problem with MPI, I'd like to address those problems directly. I don't think the MPI standard should encourage people to use backdoors. > Rationale: > > MPI provides automatic conversion of data representations for messages > between different processors. For messages containing more than one > basic data type, there are two protocols: derived datatypes or the > pack/unpack mechanism. Both protocols are likely to make the > send/receive operation considerably slower as compared to the > transmission of a byte string without conversion. This is true even > in the case that no conversion is necessary, because either the > derived datatype has to be created, or the message has to be copied > more often. > Example: > > The requirement for this functionality came up when we developed an > automatic tool which translates PARMACS or PVM application programs > into MPI. When this tool encounters a send operation, it does not know > if it can reuse an old derived data type, so it has to create a new > one for every single send. With the new MPI_DATA_REPRESENTATION > attribute, the tool can insert code for both cases (create a derived > datatype / send the message as a simple byte string), and branch into > the appropriate code at run time. The party line, as I understand it is 1. derived datatypes may have some creation overhead, but should be fast once they are created. 2. pack/unpack has inherent performance problems. It is only intended for ease of use (e.g. PVM conversion) or when incremental unpacking is necessary. 3. message data should always be typed. MPI is designed to allow fast transmission of typed data between machines with common data representations. If I understand the proposal, the goal is to allow a program to do its own packing and use MPI_BYTE when it knows data representations are the same. Without the example above, the standard response would be party line #3. So what's special about the example? The crucial issue is: "When this tool encounters a send operation, it does not know if it can reuse an old derived data type, so it has to create a new one for every single send." So it has to create a new derived type for every send, and pay the overhead for creating a derived type for every message. My first reaction is that this is an application problem, not an MPI problem. The tool should remember what data types it has created and reuse them when it can. There is potentially a real issue here. Are there real applications that send complex data which is different every time (but known by both sides, so that incremental unpacking isn't necessary), so that complex data types can't be used? Alternatively, if you were to rewrite the PVM application in MPI by hand, would you have the same problem? Supposing that such applications do exist, I suggest that the right solution is not to encourage bypassing typed data mechanisms in MPI. Presumably, in order to "send the message as a simple byte string" the application has to copy non-contiguous data into its own buffer (unless we're talking about a contiguous data structure, in which case it probably corresponds to a C structure and certainly a complex data type can be used). Copying non-contiguous data into a buffer is exactly what mpi_pack would do. Is there some reason not to rely on mpi supplied pack/unpack (which preserve type information)? There is going to be an extra copy anyway. To summarize: 1. The proposed addition exists soley so allow users to bypass the typed data mechanism in MPI. Do we want to encourage/sanction this? 2. I don't see a compelling reason for adding the mechanism. As far as I can tell, it is either unnecessary or can be just as efficiently handled using mpi_pack/unpack. Bill From owner-mpi-core@CS.UTK.EDU Wed Apr 12 20:32:17 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA26204; Wed, 12 Apr 1995 20:32:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id UAA13528; Wed, 12 Apr 1995 20:31:44 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 12 Apr 1995 20:31:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from kanga.cse.nd.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id UAA13521; Wed, 12 Apr 1995 20:31:42 -0400 From: Received: from owl.cse.nd.edu (hooper.cse.nd.edu [129.74.25.119]) by kanga.cse.nd.edu (8.6.12/8.6.12) with ESMTP id TAA10495 for ; Wed, 12 Apr 1995 19:31:41 -0500 Received: (lums@localhost) by owl.cse.nd.edu (8.6.10/8.6.9) id TAA27503; Wed, 12 Apr 1995 19:31:39 -0500 Date: Wed, 12 Apr 1995 19:31:39 -0500 Message-Id: <199504130031.TAA27503@owl.cse.nd.edu> To: mpi-core@CS.UTK.EDU Subject: MPI Developers Conference -- Preliminary Announcement and CFP Reply-to: Andrew.Lumsdaine@nd.edu *** PRELIMINARY ANNOUNCEMENT AND CALL FOR PAPERS *** MPI Developers Conference June 22-23, 1995 University of Notre Dame Notre Dame, IN 46556 The MPI Developers Conference is a conference for developers of applications which use the Message Passing Interface (MPI) standard and is intended to support the continued development and use of MPI and its extensions. The conference will provide a forum for developers from national laboratories, industry, and academia who are using MPI to present their ideas about, and experiences with, MPI. In addition, a full-day pre-conference tutorial on MPI will be offered on June 21, 1995. Since this conference is intended to be cutting-edge (and, if possible, controversial), we welcome abstracts for presented papers, oral presentations (i.e., talks with no full paper), panel discussions, and software demonstrations. Abstracts should be approximately 500 words (+- 250) in length and sent by email (in ASCII, LaTeX, PostScript, or HTML format), if possible. The deadline for submitting abstracts is May 29, 1995. Abstracts will be reviewed and authors notified of their acceptance by June 5, 1995. Final versions of accepted presented papers will be due June 21, 1995 and should be provided in a form suitable for reading by WWW browsers (HTML preferred). In keeping with the cutting-edge nature of the conference, the conference proceedings will be made available on the World Wide Web. Please submit abstracts (in ASCII, LaTeX, PostScript, or HTML format) to mpidc95@lsc.cse.nd.edu Topics include (but are not limited to): o Applications using MPI o Implementations of MPI o Extensions to MPI o Why MPI is better/worse than other message passing systems o Message passing versus shared memory o The future of message passing o Software issues Tutorial: An MPI tutorial will be offered on June 21, 1995. This tutorial will be unique -- it will not be just an extended lecture about MPI. The tutorial will be conducted in Notre Dame's SPARC-based instructional cluster where Each tutorial attendee will have use of a SPARC-10 workstation throughout the tutorial. The tutorial lectures will be interspersed with a series of exercises requiring each tutorial attendee to write, debug, and run MPI-based programs using C or Fortran. Because of the fixed number of SPARCs available in the cluster, tutorial attendance is limited to 20. Instructors: Anthony Skjellum, William Gropp, Nathan Doss, Andrew Lumsdaine Software Demonstrations: Conference attendees who wish to provide software demonstrations instead of, in addition to, or in conjunction with a paper are encouraged to do so. A SPARC-based instructional cluster will be reserved for this purpose. Please indicate on your abstract submission if you are interested in doing a demo and what your computational requirements will be. Social: The following social events will be held in the Center for Continuing Education at Notre Dame: Reception, evening of June 21 Lunch, June 22 & 23 Banquet, evening of June 22 Exact times will be announced with the preliminary program for the conference. Registration: A registration form is available at http://www.cse.nd.edu/mpidc95/Registration.ps Early registration (before June 2, 1995) is $150 for all attendees. After June 2, the registration fee will be $200. On-site registration will be accepted. The conference fees include an evening reception on June 21, lunch on June 22 & 23, and a banquet on June 22. The tutorial fee is $150 for conference attendees, $225 for non-conference attendees. Tutorial attendance is limited to the first 20 respondents. Mail your completed registration form with payment to: MPI Developers Conference Center for Continuing Education P.O. Box 1008 University of Notre Dame Notre Dame, IN 46556 All fees are payable by credit card (M/C or Visa) or check to the University of Notre Dame. Credit card registrations may be faxed to (219) 631-8083 (Web registration is not supported at this time because of concerns about the present state of secure transactions). Transportation: Notre Dame is served by the South Bend airport (officially known as the Michiana Regional Transportation Center). The airport is about 20 minutes from Notre Dame. Taxi and bus service are available. Most major airlines fly into South Bend. Delta, USAir, and Northwest offer jet service (DC-9 and 737) while United and American provide commuter service. Notre Dame is about a two hour drive from Chicago O'Hare airport. A map and directions to Notre Dame from the South Bend airport and from O'Hare airport will be made available on the conference Web page (http://www.cse.nd.edu/mpidc95/). Hotel Accomodations: A block of rooms has been reserved at the Holiday Inn University Area (219) 272-6600. Shuttle service will be available at specified times during the conference. If you would like us make a reservation for you, please indicate so on your registration form. Organizing Committee: Andrew Lumsdaine, University of Notre Dame, Department of Computer Science and Engineering Greg Burns, Ohio Supercomputer Center William Gropp, Argonne National Laboratory Ewing Lusk, Argonne National Laboratory Anthony Skjellum, Mississippi State University, NSF Engineering Research Center for Computational Field Simulation + Computer Science Department For further information, see: http://www.cse.nd.edu/mpidc95/ or contact Andrew Lumsdaine Department of Computer Science and Engineering University of Notre Dame Notre Dame, IN 46556 Phone: (219) 631-8716 Fax: (219) 631-9260 Internet: lumsdaine.1@nd.edu From owner-mpi-core@CS.UTK.EDU Mon Apr 17 13:27:34 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA25675; Mon, 17 Apr 1995 13:27:34 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA06141; Mon, 17 Apr 1995 13:24:58 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 17 Apr 1995 13:24:57 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from igate1.hac.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA06105; Mon, 17 Apr 1995 13:24:53 -0400 Received: from msmail4.hac.com ([192.27.11.35]) by igate1.hac.com (4.1/SMI-4.1) id AA02612; Mon, 17 Apr 95 10:22:27 PDT Message-Id: Date: 17 Apr 1995 09:54:00 -0800 From: "Lewins, Lloyd J" Subject: RE: Proposal for pre-defined attribute To: "mpi-core" X-Mailer: Mail*Link SMTP-MS 3.0.2 As I understand the "need" for this proposal, ALL that is required is a way to determine if a remote MPI process uses the same data representation as the local MPI process. I suggest an alternative abstraction, which provides exactly such information, e.g., MPI_Representation_equal (int remote_rank, MPI_Comm comm, int &is_equal); A simple homogeneous system always returns true. A "low quality" heterogeneous implementation can always return false, forcing the application to use MPI Datatypes or Pack/Unpack. A more complex, high quality, heterogeneous implemenation presumably must know the above information, so that it can internally optimize the sending of data between these nodes. This should be true irrespective of other details, such as compiler switches. One could even add an additional "type" argument to the above routine to determine if a particular (simple or complex) type has the same represenation in both processes. Thus two machines with the same int representation, but differing float represenatations could use the optimized case when only sending int data. A major advantage of the above proposal is that it eliminates the need to track (at the user/MPI standard level) the allocation of enumerations of MPI_DATA_REPRESENTATION. Of course, such an enumeration may still be required, but it will be visible only to the MPI implementor, and or any MPI implementation interoperability standard. P.S., I would also agree with Bill Saphir that the whole proposal is basically undesirable. It will inevitably further incourage users to send structures as sequences of bytes in an attempt to maximize performance, or to minimize the effort neccessary to build the appropriate datatypes (I allready see this trend amongst my users). This in turn will comprimise the portability/scalability of their applications. Lloyd Lewins llewins@msmail4.hac.com Hughes Aircraft Co. (310) 616-0826 From owner-mpi-core@CS.UTK.EDU Mon Apr 17 15:35:30 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA28145; Mon, 17 Apr 1995 15:35:30 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA18186; Mon, 17 Apr 1995 15:35:00 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 17 Apr 1995 15:34:58 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from igate1.hac.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA18179; Mon, 17 Apr 1995 15:34:56 -0400 Received: from msmail4.hac.com ([192.27.11.35]) by igate1.hac.com (4.1/SMI-4.1) id AA08855; Mon, 17 Apr 95 12:32:33 PDT Message-Id: Date: 17 Apr 1995 12:31:00 -0800 From: "Lewins, Lloyd J" Subject: Proposal for Error Reporting for Non-blocking Operations To: "mpi-core" X-Mailer: Mail*Link SMTP-MS 3.0.2 Proposal for Error Reporting for Non-blocking Operations Problem A problem exists in the current definition of MPI when returning errors codes from the following functions: MPI_Wait_all, MPI_Test_all, MPI_Wait_some, MPI_Test_some. These functions return a single error code, but can complete multiple-requests. I.e., there is only one place to put multiple error codes. For example, if MPI_Wait_all is called with a list of two receive requests, one of which completes normally (MPI_SUCCESS), and the other of which experiences the error MPI_ERR_TRUNCATE, it is not clear which error code should be returned by MPI_Wait_all! Proposal Extend the returned status (MPI_Status), to contain an error code for the corresponding request. In C, this could be implemented as an additional user accessible field called MPI_ERROR in the MPI_status structure. Thus, status.MPI_ERROR contains the error code for the completed request. In Fortran, the status array could contain an additional element, indexed by the constant MPI_ERROR. Thus status(MPI_ERROR) contains the error code for the completed request. {Discussion: An alternative would be to implement the error code as an invisible field in the status variable, and provide a function to access this information. We see no advantage to such an approach, as no "decoding" is neccessary to determine this information.} Specify the semantics of the error code returned in the function result, versus the error code returned in the status variable as follows: 1) If all requests complete normally, the function result should be MPI_SUCCESS, and the function should set the value of status.MPI_ERROR for each status variable to MPI_SUCCESS. 2) If a function fails because of any error in its calling arguments, or for any reason which would prevent any of the requests from being completed, the function result should indicate the corresponding error, and the function should set the value of status.MPI_ERROR for each status variable to the same error code. 3) If one or more requests fail, the function result should be MPI_ERR_OTHER, and the value of status.MPI_ERROR for the corresponding status variable(s) should be set to the appropriate error code. The value of status.MPI_ERROR for any (other) requests that complete normally should be MPI_SUCCESS. {Discussion: Another option is that the MPI_ERROR field in the status variables is not valid unless the function returns with an error code other than MPI_SUCCESS. This may optimize the implementation, but may complicate the user code.} For consistency, the above semantics should be applied to all functions, not just functions which can return multiple error code. For example, MPI_Recv should also set the error code in its status argument as well as returning a function result. In this case, however, the value of the error code returned by the function can be the same as that returned in status.MPI_ERROR. {Discussion: This seems rather ugly, particularly for the Fortran bindings, in which the returned error code is already written to the seperate argument IERROR. This also changes the semantics of many existing functions slightly, in that under this new proposal they must return the error code MPI_ERR_OTHER, instead on a specific error code} Rationale For many embedded systems, it is unacceptable that a minor error condition should cause a program to fail or quit. Thus embedded system programers generally handle error conditions in-line, to attempt to provide clean recovery from such conditions. While MPI currently allows such in-line error handling, the existing syntax of MPI_Wait_any, MPI_Test_any, MPI_Wait_some and MPI_Test_some do not allow multiple error codes to be passed back to the application, even though multiple distinct error codes may be generated. Because every completed request already has a distinct status variable associated with it, this is a natural place to put any returned error code. The status variables can be extended with little impact on any existing application code, beyond requiring recompilation. The only application code which may be affected by the proposed change, is code which currently explicitly handles returned error codes from functions. This code may need to be altered to check for specific error conditions in the status variable. Lloyd Lewins Hughes Aircraft Co. llewins@msmail4.hac.com (310) 616-0826 From owner-mpi-core@CS.UTK.EDU Mon Apr 17 16:01:44 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA29272; Mon, 17 Apr 1995 16:01:43 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA21124; Mon, 17 Apr 1995 16:01:16 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 17 Apr 1995 16:01:13 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from sgi.sgi.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id QAA21114; Mon, 17 Apr 1995 16:01:08 -0400 Received: from cthulhu.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) id NAA18744; Mon, 17 Apr 1995 13:00:35 -0700 Received: from mrjones.engr.sgi.com by cthulhu.engr.sgi.com via ESMTP (941129.SGI.8.6.9/911001.SGI) id NAA26513; Mon, 17 Apr 1995 13:00:33 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) id NAA03247; Mon, 17 Apr 1995 13:00:31 -0700 From: "Eric Salo" Message-Id: <9504171300.ZM3245@mrjones.engr.sgi.com> Date: Mon, 17 Apr 1995 13:00:27 -0700 In-Reply-To: "Lewins, Lloyd J" "Proposal for Error Reporting for Non-blocking Operations" (Apr 17, 12:31pm) References: X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: "Lewins, Lloyd J" , "mpi-core" Subject: Re: Proposal for Error Reporting for Non-blocking Operations Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii As it turns out, our implementation of MPI has recently been changed to include just such an error code in the internal data structures. I was planning to make a very similar proposal myself as soon as I found the time. Thanks, Lloyd! -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From owner-mpi-core@CS.UTK.EDU Tue Apr 18 07:19:25 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA16924; Tue, 18 Apr 1995 07:19:24 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id HAA29730; Tue, 18 Apr 1995 07:18:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 18 Apr 1995 07:18:07 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id HAA29723; Tue, 18 Apr 1995 07:18:03 -0400 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA05278 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Tue, 18 Apr 1995 12:17:42 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA23685; Tue, 18 Apr 1995 12:17:40 +0000 Date: Tue, 18 Apr 1995 12:17:40 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9504181117.AA23685@deneb.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: (llewins@msmail4.hac.com) Subject: RE: Proposal for pre-defined attribute Lloyd proposes a (presumably local) function MPI_Representation_equal (int remote_rank, MPI_Comm comm, int &is_equal); to handle Rolf's requirement. As he says this avoids any need to have a centralised allocation of data represntation enumeration values. However it has the possible down side that it may require communicator creation to be more expensive than would otherwise have been the case, even if the user never calls this function. This can arise because we now need to have the data representation constant for each member of the communicator's group. If the system were previously using a "receiver converts" protocol, we would not (previously) have needed this information. We can implement (using receiver converts, a la DCE) without this information like this. We always send in our representation, with an envelope which says how the contents should be interpreted. Thus we only ever need to know 1) a data representation enumeration value for ourself 2) how to convert from all possible incoming data representations to our representation. Therefore in this implementation we do not (yet) have the information Lloyd wants us to give to the user. So we will have to pass it around at some point (comunicator creation time, or in a non-dynamic process world, at startup). Rolf's proposal leaves this communication in the user model, where it's cost is visible. Up to now we have tried to be neutral in the issue of where data conversion takes place. This would be a (very slight) bias towards sender converts... -- Jim From owner-mpi-core@CS.UTK.EDU Tue Apr 18 12:43:47 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA28775; Tue, 18 Apr 1995 12:43:47 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA25315; Tue, 18 Apr 1995 12:43:12 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 18 Apr 1995 12:43:11 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA25308; Tue, 18 Apr 1995 12:43:10 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 3959; Tue, 18 Apr 95 12:43:09 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 8115; Tue, 18 Apr 1995 12:43:09 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Tue, 18 Apr 95 12:43:08 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA21116; Tue, 18 Apr 1995 12:43:07 -0400 Message-Id: <9504181643.AA21116@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: MPI_CART_SHIFT -- clarification needed in MPI document Reply-To: snir@watson.ibm.com Date: Tue, 18 Apr 1995 12:43:06 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: MPI_CART_SHIFT(comm,direction,disp,rank_source, rank_dest) The direction argument indicates the coordinate dimension of the shift. Suppose one has a k-dimensions grid; what are valid values for direction? Possible choices a) 1 <= direction <= k b) 0 <= direction <= k-1 c) 0 <= direction <= k-1 in C and 1 <= direction <= k in Fortran. First choice is consistent with common mathematical usage. Second choice is consistent with MPICH. Third choice is consistent with the way Cartesian topologies are created and manipulated by other functions; dimension will be the index of the corresponding coordinate in the DIMS array used to create or map the grid. As far as I can see there are no ambiguities with the other cartesian functions in the topology chapter. From owner-mpi-core@CS.UTK.EDU Tue Apr 18 16:58:50 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA04463; Tue, 18 Apr 1995 16:58:49 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA13993; Tue, 18 Apr 1995 16:55:57 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 18 Apr 1995 16:55:54 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA13978; Tue, 18 Apr 1995 16:55:51 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9557; Tue, 18 Apr 95 16:55:45 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 9195; Tue, 18 Apr 1995 16:55:45 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Tue, 18 Apr 95 16:55:42 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA21119; Tue, 18 Apr 1995 16:55:41 -0400 Message-Id: <9504182055.AA21119@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: proposals for MPI-1 Reply-To: snir@watson.ibm.com Date: Tue, 18 Apr 1995 16:55:40 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: 1. Proposal of Hempel on MPI_DATA_REPRESENTATION. I see no harm with this proposal, and no much additional implementation effort: a) Homogeneous implementations -- will define this attribute as a constant. b) Heterogeneous implementations -- will need to provide a different value on each type of processor. This could be a constant in the source code, since, anyhow, one need to compile separately the MPI library for each type of node. c) interoperable distinct implementations -- when we get to this, we shall need to agree on a unique naming scheme for various data representations. My suspicion is that such scheme can already be found if we dig into OSI standards. But, in any case, a unique, mutually agreed numbering scheme for different systems will still be compatible with the proposal of Rolf. 2. Proposal of Paul on constants. My preference is to include MPI predefined datatypes in the set of static constants. This will allow to use them in initialization expressions. The loss is that, if these are pointers to data structures, then these data structures need to be statically allocated in static memory. I see no harm in this; I don't see why it is important to allocate both predefined and user defined datatype objects in the same memory area -- I would like to hear the argument here. (Or, is anybody thinking of implementing datatypes as pointers to functions?). From owner-mpi-core@CS.UTK.EDU Tue Apr 18 17:13:42 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA04750; Tue, 18 Apr 1995 17:13:41 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA15398; Tue, 18 Apr 1995 17:13:07 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 18 Apr 1995 17:13:01 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id RAA15381; Tue, 18 Apr 1995 17:12:56 -0400 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id RAA25221 for ; Tue, 18 Apr 1995 17:09:03 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id RAA09361 for ; Tue, 18 Apr 1995 17:12:53 -0400 Date: Tue, 18 Apr 1995 17:12:53 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199504182112.RAA09361@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA19079; Tue, 18 Apr 95 17:12:52 EDT To: mpi-core@CS.UTK.EDU Subject: requests extension proposal Howdy, Attached is the PostScript for a document to begin the discussion of how to extend MPI requests. There are other topics that the external interface subcommittee will discuss at the upcoming meeting. However, these topics are less clearly defined and less urgent so they will not have a document from me. If people have other topics to discuss please let me know. Steve P.S. - This document is also available from ftp://ftp.super.org/pub/mpi/mpi2/mpif-0495/interface.ps ---------------------------------------------------------------------- %!PS-Adobe-2.0 %%Creator: dvips 5.54 Copyright 1986, 1994 Radical Eye Software %%Title: interface.dvi %%CreationDate: Tue Apr 18 17:05:09 1995 %%Pages: 5 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -p=3 -o interface.ps interface.dvi %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1995.04.18:1704 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/net/u01/src/lederman/mpi2-std/interface/interface.dvi) @start /Fa 3 107 df<003C00E001C00180038003800380038003800380038003800380 0380038003800380030007001C00F0001C00070003000380038003800380038003800380 038003800380038003800380018001C000E0003C0E297D9E15>102 DI106 D E /Fb 33 122 df<007000F001E003C007800F001E001C003800380070007000700070 00E000E000E000E000E000E000E000E0007000700070007000380038001C001E000F0007 8003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001 C000E000E000E000E00070007000700070007000700070007000E000E000E000E001C001 C0038007800F001E003C007800F00060000C247C9F18>I<01C00001C00001C00001C000 C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C180 01C00001C00001C00001C00011147D9718>I<1C3E7E7F3F1F070E1E7CF860080C788518> 44 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>59 D<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E0700000E00000E000 00E00000E00000E00000E00000E00000E000007000007000E07000E03800E03C00E01E01 C00F07C007FF8003FE0000F800131C7E9B18>67 D<7FFF00FFFF807FFF0001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D 9B18>73 D77 D80 D<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C0 1C03C01C03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C0380 1C03801C039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>82 D<03F3801FFF803FFF807C0F80700780E00380E00380E00380E000007000007800003F00 001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E000E0E001E0F001 C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFF00FFFF80FFFF807FFF0011047D7F 18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07 00780700E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E00000E000 00E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>99 D<001F80003F80001F8000038000038000038000038000038003E3800FFB801FFF803C1F 80380F80700780700380E00380E00380E00380E00380E00380E00380700780700780380F 803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F0038078070 0380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C000 01C0007FFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1 F007FFF80FFFF81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC 001FF80039E0003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E000 38E000387800F07E03F01FFFC00FFF8001FC00151F7F9318>I<03800007C00007C00007 C0000380000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FF FF80FFFF00111D7C9C18>105 D<7FE000FFE0007FE00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>108 D<7CE0E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F 00FFBFBF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7 FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E000E0E0 00E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001F00013 147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00380E0038 0E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00000E0000 0E00000E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<01E38007FB 801FFF803E1F80380F80700780700780E00380E00380E00380E00380E00380E003807007 80700780380F803C1F801FFF800FFB8003E3800003800003800003800003800003800003 80000380003FF8003FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003 E00003C00003C0000380000380000380000380000380000380000380000380000380007F FE00FFFF007FFE0015147F9318>I<07F7003FFF007FFF00780F00E00700E00700E00700 7C00007FE0001FFC0003FE00001F00600780E00380E00380F00380F80F00FFFF00FFFC00 E7F00011147D9318>I<0180000380000380000380000380007FFFC0FFFFC0FFFFC00380 000380000380000380000380000380000380000380000380000380400380E00380E00380 E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007 FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380 070700070700070700038E00038E00038E00038E0001DC0001DC0001DC0000F80000F800 00700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F80000F80000700000 F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF87F8FF015147F9318>I< 7F8FF0FF8FF87F8FF00E01C00E03800E0380070380070700070700038700038600038E00 01CE0001CE0000CC0000CC0000DC0000780000780000780000700000700000700000F000 00E00079E0007BC0007F80003F00001E0000151E7F9318>I E /Fc 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C060404060C031801F 000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C000C000C 000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006000C00 102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F000080004000 60C060C060804060801F000B107F8F0F>I<0300030007000F000B001300330023004300 C300FFE003000300030003001FE00B107F8F0F>I<20803F002C002000200020002F0030 802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C06000C000 CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC0808080 8001000200040004000C0008000800180018001800180018000B117E900F>I<1F002080 40404040404070803F000F00338061C0C060C060C060404060801F000B107F8F0F>I<1F 00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B107F8F0F >I E /Fd 13 122 df46 D<00C001C007C0FFC0FFC0FBC003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C0FFFFFFFFFFFF10227CA118>49 D<03F0000FFC001FFE003C 1F003007807007C06003C0E003E0C001E04001E04001E00001E00001E00001E00003C000 03C0000780000780000F00001E00003C0000780000F00001E00001C0000380000700000E 00001C0000380000700000FFFFE0FFFFE0FFFFE013227EA118>I70 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF00FF0 0FF83F7FFF3FEF1F8F10167E9517>97 D<01FC0007FF000FFF801F03803C018078000078 0000700000F00000F00000F00000F00000F00000F000007800007800007800003C00401F 03C00FFFC007FF8001FC0012167E9516>99 D105 D108 D110 D<01FC0007FF000FFF801F07C03C01E07800F0 7800F0700070F00078F00078F00078F00078F00078F000787800F07800F07C01F03E03E0 1F07C00FFF8007FF0001FC0015167F9518>I<0F000F000F000F000F000F00FFF8FFF8FF F80F000F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07 F803E00E1C7F9B12>116 DI121 D E /Fe 21 91 df<3078F8787005057C840D>46 D<000C001C00FC0F3800380038003800380038 00700070007000700070007000E000E000E000E000E000E001C001C001C001C001C001C0 038003C0FFFE0F1E7C9D17>49 D<003F8000C1E00100F00200780400780400780F007C0F 807C0F807C0F00780600780000F80000F00001E00001C0000380000700000E00001C0000 380000600000C0000180000300200600200800401000403FFFC07FFF80FFFF80161E7E9D 17>I<0000100000001800000038000000380000007800000078000000FC000001BC0000 013C0000033C0000023C0000063C0000043E0000081E0000081E0000101E0000101E0000 201E0000200F0000400F0000400F0000FFFF0000800F0001000F80010007800200078002 00078004000780040007800C0007C03E0007C0FF807FFC1E207E9F22>65 D<0001F808000E061800380138007000F801E0007803C0007007800030078000300F0000 301F0000301E0000303E0000203C0000007C0000007C0000007C0000007C000000F80000 00F8000000F8000000F8000000F80000007800004078000080780000803C0000803C0001 001C0002000E00020006000C000300100001C0E000003F00001D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E007800 1E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E01E000 3E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C003 C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038 003C001800780018007800080078000800780008007800080078080800F0100000F01000 00F0100000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001E00008 01E0001003C0001003C0001003C0002003C0002003C0006003C000C0078001C0078007C0 FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C0018007800180078000800 78000800780008007800080078000800F0100000F0100000F0100000F0300000F0700000 FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0000003C0000003 C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F7E9E1E>I<0001 FC04000F030C003C009C0070007C00E0003C01C0003803800018078000180F0000181F00 00181E0000183E0000103C0000007C0000007C0000007C0000007C000000F8000000F800 0000F8007FFCF80003E0780001E0780001E0780003C0780003C03C0003C03C0003C01C00 03C00E0007C007000B800380118001E06080003F80001E217B9F24>I<07FFC7FFC0007C 00F800003C007800003C007800007800F000007800F000007800F000007800F000007800 F000007800F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000F001E0 0001E003C00001E003C00001E003C00001E003C00001E003C00001E003C00003C0078000 03C007800003C007800003C007800003C007800003C007800007800F000007C00F8000FF F8FFF800221F7E9E22>I<07FFE0007C00003C00003C0000780000780000780000780000 780000780000F00000F00000F00000F00000F00000F00001E00001E00001E00001E00001 E00001E00003C00003C00003C00003C00003C00003C00007800007C000FFFC00131F7F9E 10>I<07FFF000007E0000003C0000003C00000078000000780000007800000078000000 7800000078000000F0000000F0000000F0000000F0000000F0000000F0000001E0000001 E0000001E0000001E0000001E0008001E0010003C0010003C0010003C0030003C0020003 C0060003C0060007801E0007807C00FFFFFC00191F7E9E1C>76 D<07FC01FFC0003E003E 00003E001800003E001800004F001000004F001000004780100000478010000043C01000 0043C010000083C020000081E020000081E020000080F020000080F02000008078200001 0078400001007C400001003C400001003C400001001E400001001E400002000F80000200 0F800002000F800002000780000200078000060003800006000300000F00010000FFE001 0000221F7E9E22>78 D<07FFFF00007C03C0003C01E0003C00F0007800F0007800F80078 00F8007800F8007800F8007800F000F001F000F001E000F003C000F0078000F00F0000FF F80001E0000001E0000001E0000001E0000001E0000001E0000003C0000003C0000003C0 000003C0000003C0000003C000000780000007C00000FFFC00001D1F7E9E1F>80 D<0003F800001E0E000038070000F0038001E001C003C001E0078001E00F0000F00F0000 F01F0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8F80001 F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0780007C0781E07803C210F 003C409E001E409C000E80B8000740F00003C1C04000FEC0400000C0400000C0800000E1 800000FF800000FF000000FF0000007E0000003C001D297B9F23>I<07FFFC00007C0700 003C03C0003C01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E0 00F003C000F0078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C00 01E01C0001E01E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E04 07C01F04FFFC0F18000003E01F207E9E21>I<003F040060CC01803C03801C03001C0700 180600080E00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FFC0007F E00007E00001E00000E00000F00000F04000E04000E04000E04000E06000C0600180E003 80F80300C60C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030400F00 10400F0010C00F0010800F0010800F0010800F0010001E0000001E0000001E0000001E00 00001E0000001E0000003C0000003C0000003C0000003C0000003C0000003C0000007800 00007800000078000000780000007800000078000000F0000001F800007FFFE0001C1F7A 9E21>II<03FFC0FFC0 007F007E00003E003800001E003000001E002000000F004000000F008000000F81000000 078200000007C600000003C400000003E800000001F000000001F000000000F000000000 F800000000F8000000017C000000023C000000043C0000000C1E000000081E000000101F 000000200F000000400F800000C0078000008007C000010003C000070003E0001F8007E0 00FFE01FFE00221F7F9E22>88 D<03FFFFC003E007800380078007000F0006001E000400 3C000C003C00080078000800F0000801F0000001E0000003C0000007800000078000000F 0000001E0000003C0000003C00000078000000F0010001E0010001E0020003C002000780 02000F8006000F0004001E000C003C001C007C003C007800F800FFFFF8001A1F7D9E1C> 90 D E /Ff 3 107 df<000F0038006000E001C001C001C001C001C001C001C001C001C0 01C001C001C001C001C001C0038007001E00F8001E000700038001C001C001C001C001C0 01C001C001C001C001C001C001C001C001C001C000E000600038000F102D7DA117>102 DI106 D E /Fg 19 87 df45 D<07C01FF03878701C60 1EE00EC00F400F400700070007000F000E001E001C003C007800F001E001C0038007000E 001C0038007000FFFFFFFF101C7E9B15>50 D<001C0000003E0000003E0000002E000000 6700000067000000E7800000C7800000C3800001C3C0000183C0000181C0000381E00003 81E0000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018 003C0038001E0038001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F 00001E00003C00003C0000780000780000780000F00000F00000F00000F00000F00000F0 0000F00000F00000F000007800007800007800003C00003C00001E00000F000807801803 C07800FFF0003F80151F7D9D1B>I II73 D76 DII<003F000001FFE00003FFF00007C0F8000F 807C001E001E003E001F003C000F00780007807800078078000780F00003C0F00003C0F0 0003C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C078000780780007807C 000F803C000F003E001F001F003E000F807C0007C0F80003FFF00001FFE000003F00001A 1F7E9D1F>II<003F000001FFE000 03FFF00007C0F8000F807C001F003E003E001F003C000F00780007807800078078000780 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 7800078078000780780E07803C0F0F003E079F001E03DE000F83FC0007C1F80003FFF000 01FFF800003F780000003C0000003E0000001F0000000F801A237E9D1F>II<03F8000FFE001C0F00380700700300600000E0 0000E00000E00000E00000F000007800007F00003FE0001FFC0007FE0001FF00001F8000 07800003C00003C00001C00001C00001C00001C0C00180E00380F007007C0E001FFC0007 F000121F7E9D17>IIII E /Fh 64 123 df<007E0001C1800301800703C00E03C00E01800E00000E00000E00000E 00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F8151D809C17>12 D<60F0F8680808081010204080050C7C9C0C>39 D<004000800100020006000C000C0018 001800300030007000600060006000E000E000E000E000E000E000E000E000E000E000E0 00E000600060006000700030003000180018000C000C00060002000100008000400A2A7D 9E10>I<800040002000100018000C000C000600060003000300038001800180018001C0 01C001C001C001C001C001C001C001C001C001C001C00180018001800380030003000600 06000C000C00180010002000400080000A2A7E9E10>I<60F0F070101010102020408004 0C7C830C>44 DI<60F0F06004047C830C>I<0003000300070006 0006000E000C001C0018001800380030003000700060006000E000C000C001C001800380 030003000700060006000E000C000C001C001800180038003000700060006000E000C000 C00010297E9E15>I<03C00C301818300C300C700E60066006E007E007E007E007E007E0 07E007E007E007E007E007E007E00760066006700E300C300C18180C3007E0101D7E9B15 >I<030007003F00C7000700070007000700070007000700070007000700070007000700 0700070007000700070007000700070007000F80FFF80D1C7C9B15>I<07C01830201C40 0C400EF00FF80FF807F8077007000F000E000E001C001C00380070006000C00180030006 010C01180110023FFE7FFEFFFE101C7E9B15>I<07E01830201C201C781E780E781E381E 001C001C00180030006007E00030001C001C000E000F000F700FF80FF80FF80FF00E401C 201C183007E0101D7E9B15>I<300C3FF83FF03FC020002000200020002000200023E024 302818301C200E000E000F000F000F600FF00FF00FF00F800E401E401C2038187007C010 1D7E9B15>53 D<00F0030C06040C0E181E301E300C700070006000E3E0E430E818F00CF0 0EE006E007E007E007E007E007600760077006300E300C18180C3003E0101D7E9B15>I< 03C00C301818300C700C600EE006E006E007E007E007E007E0076007700F300F18170C27 07C700060006000E300C780C78187010203030C00F80101D7E9B15>57 D<7FFFFFC0FFFFFFE0000000000000000000000000000000000000000000000000000000 0000000000FFFFFFE07FFFFFC01B0C7E8F20>61 D<0FE03038401CE00EF00EF00EF00E00 0C001C0030006000C0008001800100010001000100010001000000000000000000000003 000780078003000F1D7E9C14>63 D<000600000006000000060000000F0000000F000000 0F00000017800000178000001780000023C0000023C0000023C0000041E0000041E00000 41E0000080F0000080F0000180F8000100780001FFF80003007C0002003C0002003C0006 003E0004001E0004001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E061800180198007000780 0E0003801C0003801C00018038000180780000807800008070000080F0000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000700000807800008078000080 380000801C0001001C0001000E000200070004000180080000E03000001FC000191E7E9C 1E>IIII<001F808000E0618001801980070007800E000380 1C0003801C00018038000180780000807800008070000080F0000000F0000000F0000000 F0000000F0000000F0000000F000FFF0F0000F8070000780780007807800078038000780 1C0007801C0007800E00078007000B800180118000E06080001F80001C1E7E9C21>II I76 DII<00 3F800000E0E0000380380007001C000E000E001C0007003C00078038000380780003C078 0003C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F0 0001E0700001C0780003C0780003C0380003803C0007801C0007000E000E0007001C0003 80380000E0E000003F80001B1E7E9C20>I I<003F800000E0E0000380380007001C000E000E001C0007003C00078038000380780003 C0780003C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001 E0F00001E0700001C0780003C0780003C0380003803C0E07801C1107000E208E0007205C 0003A0780000F0E020003FE0200000602000003060000038E000003FC000003FC000001F 8000000F001B257E9C20>II<07E0801C1980300580 700380600180E00180E00080E00080E00080F00000F800007C00007FC0003FF8001FFE00 07FF0000FF80000F800007C00003C00001C08001C08001C08001C0C00180C00180E00300 D00200CC0C0083F800121E7E9C17>I<7FFFFFC0700F01C0600F00C0400F0040400F0040 C00F0020800F0020800F0020800F0020000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000001F800003FFFC001B1C7F9B1E>II87 D89 D<1FC000307000783800 781C00301C00001C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40 603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E00060007004300418080C 3007C00E127E9112>I<003F000007000007000007000007000007000007000007000007 0000070000070003E7000C1700180F00300700700700600700E00700E00700E00700E007 00E00700E00700600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C30 1818300C700E6006E006FFFEE000E000E000E00060007002300218040C1803E00F127F91 12>I<00F8018C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003 C4C00C38C01C3880181800381C00381C00381C00381C001818001C38000C300013C00010 00003000001800001FF8001FFF001FFF803003806001C0C000C0C000C0C000C060018030 03001C0E0007F800121C7F9215>I I<18003C003C0018000000000000000000000000000000FC001C001C001C001C001C001C 001C001C001C001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E0 00C000000000000000000000000000000FE000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C 0D>IIIII<03F0000E 1C00180600300300700380600180E001C0E001C0E001C0E001C0E001C0E001C060018070 03803003001806000E1C0003F00012127F9115>II<03C1 000C3300180B00300F00700700700700E00700E00700E00700E00700E00700E007006007 00700700300F00180F000C370007C7000007000007000007000007000007000007000007 00003FE0131A7E9116>II<1F9030704030C010C010E010F8007F803FE0 0FF000F880388018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C 001C003C00FFE01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C 100E2003C00C1A7F9910>I III<7F8FF0 0F03800F030007020003840001C80001D80000F00000700000780000F800009C00010E00 020E000607000403801E07C0FF0FF81512809116>II<7F FC70386038407040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E 127F9112>I E /Fi 8 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D >58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000 F80000F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E 9115>99 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F 001F001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E> 105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018E018 F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Fj 43 122 df<007000E001C00380078007000E001E001E003C003C003C00780078 00780078007000F000F000F000F000F000F000F000F000F000F000F000F0007000780078 00780078003C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DI<07F0000FFC001FFE00383F0070 0F00600780E00780E003C04003C04003C00003C00003C00003C0000780000780000F0000 0E00001C00003C0000780000E00001C0000380000700000E00001C0000380000700000FF FFC0FFFFC0FFFFC0121F7E9E17>50 D<001F0000001F0000003F8000003B8000003B8000 007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0F000 03C0780003C078000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE00 1E000F001E000F003C000F803C0007803C000780780007C0780003C0780003C0F00003E0 1B207F9F1E>65 D<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C 0000780000780000780000F00000F00000F00000F00000F00000F00000F00000F0000078 00007800007800007C00003C00003E00001E00000F000207800E03E03E01FFFC00FFF000 1FC017227DA01D>67 DII< FFFFC0FFFFC0FFFFC0F00000F00000F00000F00000F00000F00000F00000F00000F00000 F00000F00000FFFF00FFFF00FFFF00F00000F00000F00000F00000F00000F00000F00000 F00000F00000F00000F00000F00000F00000F00000F0000012207B9F1A>I<001FE000FF F801FFFE03E03E07800E0F00001E00003E00003C00007C0000780000780000780000F000 00F00000F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C00 1E3E001E1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>I73 D76 DII<003F000000FFC00003FFF00007 E1F8000F807C001F003E001E001E003C000F003C000F00780007807800078078000780F0 0003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C07800078078000780780007803C000F003C000F001E001E001F003E000F807C0007 E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC00003FFF00007E1F8000F807C 001F003E001E001E003C000F003C000F00780007807800078078000780F00003C0F00003 C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0780007 8078000780781E07803C0F0F003C0F8F001E079E001F03FE000F83FC0007E1F80003FFF0 0000FFF800003F780000007C0000003E0000001E0000001F0000000F801A277DA021>I< FFF800FFFF00FFFF80F007C0F003E0F001E0F000F0F000F0F000F0F000F0F000F0F001E0 F003E0F007C0FFFF80FFFF00FFF800F03C00F01E00F01E00F00F00F00F00F00780F00780 F003C0F001C0F001E0F000F0F000F0F00078F00078F0003C16207B9F1D>I<01FC0007FF 800FFFC01F03C03C00C03C00007800007800007800007800007800007C00003C00003F00 001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001E00001E00001 E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019>III87 D89 D<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7F FF3FEF1F8F10147E9316>97 D<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F0 00F000780078003C013E0F1FFF0FFE03F010147E9314>99 D<0007800007800007800007 8000078000078000078000078000078000078000078000078007C7800FF7801FFF803E1F 807C0780780780F80780F00780F00780F00780F00780F00780F00780F00780780780780F 803E1F801FFF800FF78007C78011207E9F17>I<03F0000FFC001FFE003E1F003C070078 0700700380FFFF80FFFF80FFFF80F00000F00000F000007000007800003C01003E07001F FF0007FE0001F80011147F9314>I<007E01FE03FE078007000F000F000F000F000F000F 000F00FFF0FFF0FFF00F000F000F000F000F000F000F000F000F000F000F000F000F000F 000F000F000F000F20809F0E>I105 D109 DI<01F80007FE001FFF803F0FC03C03C07801E07801E0F000F0 F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F800 14147F9317>II<03C7800FF7801F FF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F8078078 07807C0F803E1F801FFF800FF78007C78000078000078000078000078000078000078000 0780000780000780111D7E9317>II<07F01FFC3FFC780C7800 780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311> I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E001E 001E001E001E001E001E201FF00FF007C00C1A7F9910>IIII<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F800 00F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F0141480 9315>II E /Fk 20 123 df46 D<00180000380000F80007F800FFF800FF F800F8F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF07FFFF014287BA7 1E>49 D<00FE0003FFC007FFE00FFFF01F03F83C00FC38007E78003E70003EF0001FF000 1F60001F20001F00001F00001F00001F00003E00003E00007C00007C0000F80001F00001 E00003C0000780000F00001E00003C0000780000F00001E00003C0000780000F00001E00 003C00007FFFFF7FFFFF7FFFFF7FFFFF18287EA71E>I<0001FF00000FFFE0003FFFFC00 7FFFFE00FF01FE01F8003E03F0000C07C000000FC000001F8000001F0000003F0000003E 0000007E0000007C0000007C0000007C000000F8000000F8000000F8000000F8000000F8 000000F8000000F8000000F8001FFEF8001FFEF8001FFE7C001FFE7C00003E7C00003E7E 00003E3E00003E3F00003E1F00003E1F80003E0FC0003E07C0003E03F0003E01F8003E00 FF00FE007FFFFE003FFFFC000FFFE00001FF001F2C7CAA28>71 D73 D82 D<01FE000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F80000F80000F80000 F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F800F8F800F8FC01F87E07F87FFF F83FFFF81FFCF80FE0F8151B7E9A1D>97 D<007FC001FFF007FFFC0FFFFC1FC07C1F0008 3E00007C00007C00007C0000F80000F80000F80000F80000F80000F80000F800007C0000 7C00007E00003E00001F000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>99 D<00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E0000 3E00003E00003E00003E00FC3E03FF3E07FFFE0FFFFE1FC1FE3F007E3E003E7C003E7C00 3EFC003EF8003EF8003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E7C003E3E00 7E3F00FE1FC1FE0FFFFE07FFBE03FF3E00FC3E172A7EA91F>I<007E0003FF8007FFC00F FFE01F83F03F00F03E00787C00787C003878003CFFFFFCFFFFFCFFFFFCFFFFFCF80000F8 0000F800007800007C00007C00003E00003F000C1FC07C0FFFFC07FFFC01FFF0007F8016 1B7E9A1B>I105 D108 D110 D<007F000001FFC00007FFF0000FFFF8001F C1FC003F007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001F C1FC000FFFF80007FFF00001FFC000007F0000191B7E9A1E>I<00FC3E03FF3E07FFBE0F FFFE1FC1FE3F80FE3E007E7E003E7C003E7C003EF8003EF8003EF8003EF8003EF8003EF8 003EF8003EFC003E7C003E7E003E3E007E3F00FE1FC1FE1FFFFE0FFFBE03FF3E01F83E00 003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E17 277E9A1F>113 DI<03FC00 1FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8003FFE00 1FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFFC07FFF80 1FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007C000FFFF C0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003FFE003FF E001FF8000FC0013227FA116>II122 D E /Fl 12 121 df69 D73 D<0007FFFC000000007FFFFFC0000001FFFFFFF8000003FF FFFFFE000007FE001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000FFF0000FFF80 007FF0000FFF80007FF8000FFF80007FF80007FF00003FFC0007FF00003FFC0003FE0000 3FFC0000F800003FFC00000000003FFC00000000003FFC00000000003FFC00000000003F FC00000007FFFFFC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC 0003FFF0003FFC0007FFC0003FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC00 7FF800003FFC007FF800003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FF F000003FFC00FFF000003FFC00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC 0003EFFC003FFF0007CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFF F001FFFC0003FF80007FF8362E7DAD3A>97 D<00001FFFC0000000FFFFF8000007FFFFFE 00001FFFFFFF80007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FFE007FF0003FF E00FFE0003FFE00FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF803FFC00003E 007FF8000000007FF8000000007FF800000000FFF800000000FFF800000000FFF8000000 00FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF8000000 00FFF8000000007FF8000000007FF8000000007FFC000000003FFC000000003FFC000000 001FFC000000F81FFE000000F80FFE000000F80FFF000001F007FF800003F003FFC00007 E001FFE0000FC000FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8000000FFFFE0 0000001FFE00002D2E7CAD35>99 D<00001FFE00000001FFFFE0000007FFFFF800001FFF FFFE00007FFC07FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00003FF00FFE00 001FF01FFE00000FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC000007FC7FF800 0007FC7FF8000007FE7FF8000007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFFFFFF FFFFFEFFFFFFFFFFFEFFFFFFFFFFFCFFF800000000FFF800000000FFF800000000FFF800 0000007FF8000000007FF8000000007FFC000000003FFC000000003FFC000000003FFC00 00001C1FFE0000003E0FFE0000003E07FF0000007E07FF000000FC03FF800001F801FFC0 0003F0007FF0001FE0003FFE00FFC0001FFFFFFF800007FFFFFE000000FFFFF80000000F FF80002F2E7DAD36>101 D<000000FFC000000007FFF80000003FFFFC000000FFFFFF00 0001FFC1FF000007FF03FF80000FFC03FF80000FF807FFC0001FF807FFC0003FF007FFC0 003FF007FFC0003FE003FF80007FE003FF80007FE001FF00007FE000FE00007FE0003800 007FE0000000007FE0000000007FE0000000007FE0000000007FE0000000007FE0000000 007FE0000000007FE0000000007FE0000000007FE0000000FFFFFFFE0000FFFFFFFE0000 FFFFFFFE0000FFFFFFFE0000FFFFFFFE0000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF00000003FFFFFF800003FFFFFF800003FFFFFF80000 3FFFFFF800003FFFFFF800002A487DC724>I<007FC000FFFFC000FFFFC000FFFFC000FF FFC000FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FF FFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<007FC001FFC00000FFFFC00FFFF80000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1 FC07FF8000FFFFC3E003FFC00003FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE0 0001FFDC0000FFE00001FFFC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF0 0000FFF00001FFF00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFF E0FFFFFFC07FFFFFE03B2E7CAD42>110 D<00FF803F8000FFFF80FFF000FFFF83FFFC00 FFFF87FFFE00FFFF8FC3FF00FFFF8F07FF0003FF9E0FFF8001FFBC0FFF8001FFB80FFF80 01FFF80FFF8001FFF00FFF8001FFF007FF0001FFF007FF0001FFE003FE0001FFE000F800 01FFE000000001FFE000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC0000000FFFFFFE00000FFFFFFE00000FFFFFFE00000FFFFFFE00000 FFFFFFE00000292E7CAD31>114 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00F F800FFE01FC0001FE03F80000FE03F000007E07F000003E07F000003E0FF000003E0FF00 0003E0FF800003E0FFC0000000FFF0000000FFFE000000FFFFF800007FFFFFC0007FFFFF F0003FFFFFFC001FFFFFFF000FFFFFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFF F00003FFFFF800001FFFF8000000FFFC0000001FFC7800000FFCF8000007FCF8000003FC FC000003FCFC000003FCFE000003F8FE000003F8FF000003F8FF800007F0FFC0000FF0FF F0001FE0FFFC00FFC0FFFFFFFF80FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>I< 0001F000000001F000000001F000000001F000000001F000000001F000000003F0000000 03F000000003F000000007F000000007F000000007F00000000FF00000000FF00000001F F00000003FF00000003FF00000007FF0000001FFF0000003FFF000000FFFFFFFC0FFFFFF FFC0FFFFFFFFC0FFFFFFFFC0FFFFFFFFC000FFF0000000FFF0000000FFF0000000FFF000 0000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF00000 00FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000 FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF001F000FFF001F000FF F001F000FFF001F000FFF001F000FFF001F000FFF001F000FFF001F000FFF001F0007FF0 01E0007FF803E0003FF803E0003FFC07C0001FFE0F80000FFFFF800007FFFE000001FFFC 0000001FF00024427EC12E>I<7FFFFF801FFFFF007FFFFF801FFFFF007FFFFF801FFFFF 007FFFFF801FFFFF007FFFFF801FFFFF00007FF80001FF0000007FFC0001FC0000003FFE 0003F80000001FFF0007F00000000FFF0007E000000007FF800FC000000003FFC01F8000 000003FFE03F8000000001FFF07F0000000000FFF8FE00000000007FF9FC00000000003F FFF800000000003FFFF000000000001FFFE000000000000FFFC0000000000007FFC00000 00000003FFC0000000000001FFE0000000000001FFF0000000000001FFF8000000000003 FFFC000000000003FFFE000000000007FFFE00000000000FEFFF00000000001FCFFF8000 0000003F87FFC0000000007F03FFE000000000FE01FFE000000001FC00FFF000000001F8 007FF800000003F0007FFC00000007F0003FFE0000000FE0001FFF0000001FC0000FFF00 00003F800007FF800000FF800007FFC000FFFFF8003FFFFFC0FFFFF8003FFFFFC0FFFFF8 003FFFFFC0FFFFF8003FFFFFC0FFFFF8003FFFFFC03A2E7EAD3F>120 D E /Fm 8 117 df<00001E000000003E00000000FE00000003FE0000003FFE0000FFFF FE0000FFFFFE0000FFFFFE0000FFCFFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE00 00000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000 000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE000000 0FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000F FE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE00007FFFFFFFC07FFFFFFFC07FFFFFFFC07FFFFFFF C0223879B731>49 D<0000001FFF000070000001FFFFE000F000000FFFFFFC01F000007F FFFFFF03F00001FFFE007F87F00007FFE0000FEFF0000FFF000003FFF0003FFE000001FF F0007FF8000000FFF000FFF00000007FF001FFE00000003FF003FFC00000001FF003FF80 0000001FF007FF800000000FF00FFF0000000007F00FFF0000000007F01FFE0000000007 F01FFE0000000003F03FFC0000000003F03FFC0000000003F07FFC0000000001F07FFC00 00000001F07FFC0000000001F07FF8000000000000FFF8000000000000FFF80000000000 00FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800 0000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF80000000000 00FFF8000000000000FFF80000000000007FF80000000000007FFC0000000000007FFC00 00000000F07FFC0000000000F03FFC0000000000F03FFC0000000000F01FFE0000000000 F01FFE0000000001F00FFF0000000001E00FFF0000000003E007FF8000000003E003FF80 00000007C003FFC000000007C001FFE00000000F8000FFF00000001F00007FF80000003E 00003FFE0000007C00000FFF000001F8000007FFE00007F0000001FFFE003FC00000007F FFFFFF800000000FFFFFFC0000000001FFFFF000000000001FFF0000003C3D7BBB47>67 D<001FFF00000001FFFFF0000007FFFFFC00000FF807FF00001FF801FF80001FFC00FFC0 001FFC007FE0001FFC007FE0001FFC007FF0001FFC003FF0000FF8003FF00007F0003FF0 0001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000000FFFFFF0 00007FF83FF00003FF803FF00007FE003FF0001FFC003FF0003FF8003FF0007FF0003FF0 007FE0003FF000FFE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000FFC0007FF0 00FFC0007FF000FFE000FFF0007FF001DFF0003FF803DFF8001FFC0F8FFFF00FFFFE0FFF F001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFFFE00 00FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03FF00007F83F F00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFFFFFFFCFFFF FFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE00000007FE000 00007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF800007807FC0000 F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE00026267DA52D >101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF0000000007FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF007FC00003FF 01FFF80003FF07FFFE0003FF1F03FF0003FF3C01FF0003FF7801FF8003FF7000FF8003FF E000FFC003FFC000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC3FFFFFFFFF FC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00FFFFFE03FF 00FFFFF001FFC003FFE0007FE003FF80007FF003FF80003FF803FF00001FF803FF00001F FC03FF00000FFC03FF00000FFE03FF00000FFE03FF00000FFE03FF000007FF03FF000007 FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007 FF03FF000007FF03FF000007FF03FF00000FFE03FF00000FFE03FF00000FFE03FF00001F FC03FF00001FFC03FF00001FF803FF80003FF003FFC0007FF003FFE000FFE003FFF001FF 8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000FFFFFC0000 00FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE3FFF80FFFE3C7FC0FFFE707FC007FEF0FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF80FFE003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC0F8007FE1F0001FFFE00007FFC00001FF001E377EB626>116 D E /Fn 46 123 df<003F0000E0C001C0C00381E00701E00701E0070000070000070000 070000070000070000FFFFE00700E00700E00700E00700E00700E00700E00700E00700E0 0700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FC3FE172080 9F19>12 D<70F8FCFC74040404080810102040060E7C840D>44 DI<70F8F8F87005057C840D>I<018003800F80F380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038003800380038007C0FF FE0F1E7C9D17>49 D<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0 F803C02007C00007C0000780000780000F00000E00001C0000380000700000600000C000 0180000300000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0 000C1C00100E00200F00780F80780780780780380F80000F80000F00000F00000E00001C 0000380003F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807 C0F00780400780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E0000 0E00001E00002E00002E00004E00008E00008E00010E00020E00020E00040E00080E0008 0E00100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E0000 0E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE000100000100000 10000010000010000010000011F000161C00180E001007001007800003800003800003C0 0003C00003C07003C0F003C0F003C0E00380400380400700200600100E000C380003E000 121F7E9D17>I<03F0000C0C001006003003002001806001806001806001807001807803 003E03003F06001FC8000FF00003F80007FC000C7E00103F00300F806003804001C0C001 C0C000C0C000C0C000C0C000806001802001001002000C0C0003F000121F7E9D17>56 D<03F0000E18001C0C00380600380700700700700380F00380F00380F003C0F003C0F003 C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C00003800003800003800007 00300700780600780E00700C002018001070000FC000121F7E9D17>I<70F8F8F8700000 000000000000000070F8F8F87005147C930D>I<001F800000E070000100080006000600 0800010008000100100F00802030C0402060404040C0202041C01C2041C01C2081801C10 83801C1083801C1083801C1083801C1083801C1083801C1081801C1041C01C1041C01C10 40C03C2020605C202030CC40100F0780080000000800000006000070010001C000E01F00 001FF0001C207D9F23>64 D<000100000003800000038000000380000007C0000007C000 0007C0000009E0000009E0000009E0000010F0000010F0000010F0000020780000207800 0020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F00 01000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F 22>I<000FC040007030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C 0000C07C0000407C00004078000040F8000000F8000000F8000000F8000000F8000000F8 000000F8000000F8000000F8000000780000007C0000407C0000403C0000401C0000401E 0000800E000080070001000380020001C0040000703800000FC0001A217D9F21>67 D70 D72 DI76 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F00000 F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C0 8001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19> 83 D<7FFFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F001080 0F0010800F0010000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000001F800007FFFE001C1F7E9E21>I<1FE00030300078180078 1C00300E00000E00000E00000E0000FE00078E001E0E00380E00780E00F00E10F00E10F0 0E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E3E000EC3800F01C00F00E00E00 E00E00700E00700E00780E00780E00780E00780E00780E00780E00700E00700E00E00F00 E00D01C00CC300083E0015207F9F19>I<03F80E0C1C1E381E380C70007000F000F000F0 00F000F000F00070007000380138011C020E0C03F010147E9314>I<000380003F800003 8000038000038000038000038000038000038000038000038000038003E380061B801C07 80380380380380700380700380F00380F00380F00380F00380F00380F003807003807003 803803803807801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E0038070038 0700700700700380F00380F00380FFFF80F00000F00000F0000070000070000038008018 00800C010007060001F80011147F9314>I<007C00C6018F038F07060700070007000700 070007000700FFF007000700070007000700070007000700070007000700070007000700 07000700070007007FF01020809F0E>I<0000E003E3300E3C301C1C30380E00780F0078 0F00780F00780F00780F00380E001C1C001E380033E0002000002000003000003000003F FE001FFF800FFFC03001E0600070C00030C00030C00030C000306000603000C01C038003 FC00141F7F9417>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E3E000E43000E81800F01C00F01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F 9F19>I<1C001E003E001E001C000000000000000000000000000E007E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A1F809E0C> I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C 000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>107 D<0E00FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I< 0E1F01F000FE618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E 3E00FE43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E00 1C03803801C03801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E0 7000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00 E00E00E00E00F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E00 E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E00 00FFE000151D7F9319>I<03E0800619801C05803C0780380380780380700380F00380F0 0380F00380F00380F00380F003807003807803803803803807801C0B800E138003E38000 0380000380000380000380000380000380000380000380003FF8151D7E9318>I<0E78FE 8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FF E00F147F9312>I<1F9030704030C010C010C010E00078007F803FE00FF0007080388018 8018C018C018E030D0608F800D147E9312>I<020002000200060006000E000E003E00FF F80E000E000E000E000E000E000E000E000E000E000E000E080E080E080E080E08061003 1001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC1614 7F9319>II< FF9FE1FC3C0780701C0300601C0380200E0380400E0380400E03C0400707C0800704C080 0704E080038861000388710003C8730001D0320001D03A0000F03C0000E01C0000E01C00 00601800004008001E147F9321>I<7FC3FC0F01E00701C007018003810001C20000E400 00EC00007800003800003C00007C00004E000087000107000303800201C00601E01E01E0 FF07FE1714809318>II<3FFF380E 200E201C40384078407000E001E001C00380078007010E011E011C0338027006700EFFFE 10147F9314>I E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 75 356 a Fm(Chapter)34 b(1)75 564 y Fl(External)41 b(In)m(terfaces)760 786 y Fn(Stev)o(e)16 b(Huss-Lederman)775 843 y(lederman@sup)q(er.org)834 899 y(April)h(18,)d(1995)75 974 y Fk(1.1)59 b(Intro)r(duction)75 1076 y Fn(This)16 b(c)o(hapter)f(deals)h(with)f(ho)o(w)g(users)g(and)h(external)f(pac)o (k)m(ages)g(can)h(in)o(terface)f(with)h Fj(MPI-2)p Fn(.)75 1219 y Fk(1.2)59 b(Generalized)19 b(Requests)75 1403 y Fi(Discussion:)34 b Fh(This)14 b(prop)q(osal)g(is)g(based)g(on)g(pap) q(er)h(of)e(Marc)i(Snir)f(\(Prop)q(osals)g(for)g Fg(MPI-2)p Fh(,)e(Marc)o(h)i(10,)f(1995,)75 1459 y(section)20 b(3.3\))e(and)h (Bill)f(Gropp)h(and)g(Rust)o(y)g(Lusk)h(\(Dynamic)d(Pro)q(cess)k (managemen)o(t)c(in)i(an)g Fg(MPI)f Fh(setting,)75 1516 y(F)m(ebruary)f(26,)g(1995,)g(App)q(endix)g(A\).)g(If)g(I)g(ha)o(v)o(e) g(misrepresen)o(ted)h(or)f(misundersto)q(o)q(d)g(these)i(prop)q(osal)e (I)g(am)75 1572 y(sorry)m(.)k(This)15 b(curren)o(t)h(do)q(cumen)o(t)e (has)h(not)g(b)q(een)h(discussed)h(b)o(y)d(the)i(MPIF)f(and)g(is)f (just)h(m)o(y)e(\014rst)j(attempt)e(to)75 1629 y(pull)f(this)h (information)d(together.)166 1768 y Fn(The)21 b(goal)f(of)g(this)h Fj(MPI-2)f Fn(extension)i(is)f(to)f(allo)o(w)h(users)g(to)f(create)g (requests)g(for)h(additional)75 1824 y(non-blo)q(c)o(king)g(op)q (erations.)32 b(These)20 b(requests)f(could)i(b)q(e)e(used)h(to)f(p)q (erform)g(additional)i(op)q(erations)75 1881 y(not)14 b(supp)q(orted)g(b)o(y)g Fj(MPI)g Fn(suc)o(h)g(as)g(in)o(tricate)g (non-blo)q(c)o(king)i(collectiv)o(e)g(op)q(erations.)k(They)14 b(could)h(also)75 1937 y(b)q(e)h(used)g(to)e(create)h(a)g(non-blo)q(c)o (king)i(send-receiv)o(e)g(t)o(yp)q(e)e(function.)166 1994 y(The)20 b(metho)q(d)g(tak)o(en)g(in)g(de\014ning)i(this)e Fj(MPI-2)f Fn(extension)i(is)f(to)f(try)h(and)g(generalize)h(curren)o (t)75 2050 y Fj(MPI)e Fn(functions.)33 b(F)l(or)19 b(example,)i(the)e Fj(MPI)g Fn(functions)h(of)f Fj(MPI)p 1208 2050 14 2 v 16 w Ff(f)p Fj(TEST)p Ff(j)p Fj(W)l(AIT)p Ff(gf)p Fj(ANY)p Ff(j)p Fj(SOME)p Ff(j)p Fj(ALL)p Ff(g)75 2107 y Fn(are)c(generalized.) 21 b(Also,)15 b(whenev)o(er)h(additional)g(functions)g(are)e(needed)j (they)e(are)f(made)h(to)g(lo)q(ok)g(lik)o(e)75 2163 y(similar)j (functions)g(curren)o(tly)f(a)o(v)m(ailable)i(in)e Fj(MPI)p Fn(.)f(Th)o(us,)h(these)g(extensions)h(are)f(based)g(on)g(the)g Fj(MPI)75 2220 y Fn(p)q(ersisten)o(t)f(comm)o(unication)g(requests)f (and)g(the)g(cac)o(hing)h(features.)166 2276 y(As)10 b(with)h(the)f(cac)o(hing)h(feature,)f(a)g(new)g(t)o(yp)q(e)h(of)e (request)h(is)h(\014rst)f(created)g(using)h Fj(MPI)p 1620 2276 V 16 w(REQUEST)p 1839 2276 V 18 w(TYPE)p 1974 2276 V 16 w(CREA)l(TE)75 2332 y Fn(and)21 b(freed)g(with)g Fj(MPI)p 481 2332 V 16 w(REQUEST)p 700 2332 V 18 w(TYPE)p 835 2332 V 16 w(FREE)p Fn(.)g(The)g(rest)f(of)g(the)h(prop)q(osal)g (mimics)h(the)f(mec)o(h-)75 2389 y(anisms)h(in)g(p)q(ersisten)o(t)g (comm)o(unication)h(requests.)39 b(The)22 b(user)g(creates)f(a)g(sp)q (eci\014c)j(instance)e(of)f(a)75 2445 y(request)16 b(via)h Fj(MPI)p 395 2445 V 15 w(REQUEST)p 613 2445 V 18 w(INIT)p Fn(.)e(The)i(op)q(eration)f(asso)q(ciated)g(with)h(the)f(request)g(is)h (b)q(egun)g(with)75 2502 y Fj(MPI)p 160 2502 V 16 w(ST)l(ART)p Ff(f)p Fj(ALL)p Ff(g)p Fn(.)h(Completion)11 b(of)f(the)g(request)g(is)h (done)f(with)h Fj(MPI)p 1313 2502 V 15 w Ff(f)p Fj(TEST)p Ff(j)p Fj(W)l(AIT)p Ff(gf)p Fj(ANY)p Ff(j)p Fj(SOME)p Ff(j)p Fj(ALL)p Ff(g)p Fn(.)75 2558 y(The)k(sp)q(eci\014c)j(request)d (can)g(b)q(e)h(freed)f(with)h Fj(MPI)p 931 2558 V 16 w(REQUEST)p 1150 2558 V 18 w(FREE)p Fn(.)964 2828 y(1)p eop %%Page: 2 2 2 1 bop 75 -100 a Fn(2)894 b Fe(CHAPTER)15 b(1.)35 b(EXTERNAL)16 b(INTERF)-5 b(A)o(CES)75 45 y Fd(1.2.1)49 b(F)o(unctionalit)o(y)75 214 y Fi(Discussion:)34 b Fh(Only)13 b(the)i(language)e(neutral)h (description)g(of)g(functions)g(are)g(giv)o(en.)j(The)e(rest)g(will)d (b)q(e)j(added)75 270 y(once)g(the)f(main)e(discussion)i(items)f(are)h (settled.)166 409 y Fn(A)h(new)h(t)o(yp)q(e)f(of)g(request)g(is)h (created)f(with)75 560 y Fj(MPI)p 160 560 14 2 v 16 w(REQUEST)p 379 560 V 18 w(TYPE)p 514 560 V 17 w(CREA)l(TE\(create)p 832 560 V 17 w(fn,)20 b(sta)o(rt)p 1005 560 V 17 w(fn,)g(test)p 1161 560 V 18 w(fn,)f(w)o(ait)p 1327 560 V 17 w(fn,)h(free)p 1484 560 V 16 w(fn,)g(count,)h(t)o(yp)q(e)p 1789 560 V 17 w(req,)75 617 y(extra)p 173 617 V 16 w(state\))117 694 y Fh(IN)155 b Fj(create)p 435 694 V 17 w(fn)419 b Fh(Creation)14 b(callbac)o(k)f(function)h(for)f(t)o(yp)q(e)p 1542 694 13 2 v 16 w(req)117 766 y(IN)155 b Fj(sta)o(rt)p 407 766 14 2 v 17 w(fn)447 b Fh(Request)15 b(start)g(callbac)o(k)e (function)g(for)h(t)o(yp)q(e)p 1630 766 13 2 v 15 w(req)117 838 y(IN)155 b Fj(test)p 390 838 14 2 v 18 w(fn)463 b Fh(Request)15 b(test)g(callbac)o(k)e(function)h(for)f(t)o(yp)q(e)p 1611 838 13 2 v 16 w(req)117 910 y(IN)155 b Fj(w)o(ait)p 400 910 14 2 v 17 w(fn)454 b Fh(Request)15 b(w)o(ait)e(callbac)o(k)g (function)h(for)f(t)o(yp)q(e)p 1621 910 13 2 v 16 w(req)117 982 y(IN)155 b Fj(free)p 391 982 14 2 v 17 w(fn)463 b Fh(Request)15 b(free)g(callbac)o(k)e(function)g(for)h(t)o(yp)q(e)p 1610 982 13 2 v 15 w(req)117 1054 y(IN)155 b Fj(count)482 b Fh(Maxim)o(um)10 b(n)o(um)o(b)q(er)k(of)f(requests)j(of)d(this)h(t)o (yp)q(e)p 1671 1054 V 15 w(req)117 1126 y(OUT)108 b Fj(t)o(yp)q(e)p 401 1126 14 2 v 18 w(req)430 b Fg(MPI)13 b Fh(created)j(request)f(t)o (yp)q(e)g(for)e(future)i(reference)117 1198 y(IN)155 b Fj(extra)p 416 1198 V 17 w(state)384 b Fh(Extra)14 b(state)h(passed)g(to)f(all)e(callbac)o(k)i(functions)166 1322 y Fn(The)19 b(callbac)o(k)g(functions)h(are)e(describ)q(ed)j(b)q (elo)o(w)e(with)g(the)g(routines)g(that)f(cause)h(them)f(to)g(b)q(e)75 1379 y(called.)166 1518 y Fi(Discussion:)32 b Fh(Marc's)12 b(prop)q(osal)f(had)h(the)h(coun)o(t)f(argumen)o(t.)k(It)c(is)f(not)h (clear)g(whether)i(w)o(e)e(w)o(an)o(t)f(this)h(or)75 1574 y(not.)18 b(The)13 b(functions)g(in)f Fg(MPI)g Fh(to)h(do)g(cac)o (hing)g(do)f(not)h(require)h(an)o(y)e(statemen)o(t)h(to)g(ho)o(w)f(man) o(y)f(y)o(ou)i(will)e(create.)75 1631 y(W)m(e)k(need)h(to)g(discuss)g (what)f(optimization)e(the)j Fg(MPI)f Fh(implemen)o(tatio)o(n)e(can)i (do)g(b)o(y)h(kno)o(wing)e(coun)o(t)i(and)f(if)f(it)75 1687 y(is)i(w)o(orth)g(doing)f(this.)25 b(I)16 b(w)o(ould)f(assume)h (that)g(if)f(y)o(ou)h(exceed)i(coun)o(t)e(then)h(this)f(w)o(ould)f (cause)i(an)f(error.)26 b(By)75 1744 y(default,)15 b(this)h(w)o(ould)e (cause)j(an)e Fg(MPI)p 682 1744 13 2 v 14 w(ABORT)p Fh(.)f(T)m(o)h(mak) o(e)f(this)h(w)o(ork)g(b)q(et)o(w)o(een)i(mo)q(dules,)e(it)g(seems)g (that)h(one)75 1800 y(needs)f(functions)f(to)f(see)i(ho)o(w)f(man)o(y)d (requests)16 b(are)e(already)g(there)h(so)e(y)o(ou)h(kno)o(w)f(if)g(it) g(is)h(safe)f(to)h(add)g(another)75 1857 y(one.)166 1996 y Fn(T)l(o)h(get)g(an)g(instance)h(of)f(t)o(yp)q(e)p 691 1996 14 2 v 16 w(req,)g(one)g(calls:)75 2147 y Fj(MPI)p 160 2147 V 16 w(REQUEST)p 379 2147 V 18 w(INIT\(t)o(yp)q(e)p 584 2147 V 16 w(req,)g(p)o(rivate)p 815 2147 V 17 w(data,)g(comm,)e (request\))117 2224 y Fh(IN)155 b Fj(t)o(yp)q(e)p 401 2224 V 18 w(req)430 b Fh(T)o(yp)q(e)10 b(of)f(request)h(desired)h (\(from)c(MPI)p 1519 2224 13 2 v 16 w(REQUEST)p 1738 2224 V 15 w(TYPE)p 1870 2224 V 16 w(CREA)m(TE\))117 2296 y(IN)155 b Fj(p)o(rivate)p 449 2296 14 2 v 17 w(data)359 b Fh(Data)13 b(needed)j(b)o(y)d(request)117 2368 y(IN)155 b Fj(comm)470 b Fh(Comm)o(unicator)11 b(to)j(asso)q(ciate)g(request)i (with)117 2440 y(OUT)108 b Fj(request)452 b Fh(Handle)14 b(to)g(request)166 2647 y Fi(Discussion:)53 b Fh(Marc's)19 b(prop)q(osal)g(has)f(the)i(comm)c(as)i(an)h(argumen)o(t.)31 b(Since)20 b(one)e(is)h(creating)g(a)f(re-)75 2704 y(quest)h(to)e(p)q (erform)g(a)h(comm)o(unicatio)o(n,)e(it)h(mak)o(es)g(some)g(sense)i(to) f(asso)q(ciate)h(it)e(with)h(a)f(comm.)27 b(Also,)18 b(the)1967 46 y Fc(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 3 3 2 bop 75 -100 a Fe(1.2.)34 b(GENERALIZED)16 b(REQUESTS)1071 b Fn(3)75 45 y Fg(MPI)p 152 45 13 2 v 14 w(ST)m(ART)15 b Fh(function)f(only)g(tak)o(es)h(a)f(request)i(so)f(in)f(its)h(curren) o(t)h(form)d(y)o(ou)h(need)i(to)e(giv)o(e)h(the)g(comm)d(at)i(the)75 102 y Fg(MPI)p 152 102 V 14 w(REQUEST)p 354 102 V 14 w(INIT)p Fh(.)g(The)h(request)i(y)o(ou)d(get)h(will)e(need)j(to)f (remem)o(b)q(er)f(the)h(comm.)j(Is)d(there)i(an)d(argumen)o(t)75 158 y(not)g(use)h(the)f(comm)d(here??)166 380 y Fi(Discussion:)40 b Fh(Bill)14 b(and)h(Rust)o(y's)g(prop)q(osal)g(has)g(priv)n(ate)p 1123 380 V 15 w(data.)22 b(This)15 b(is)h(a)f(place)g(to)g(put)h (information)75 436 y(needed)11 b(to)f(pro)q(cess)i(the)e(request.)19 b(One)10 b(test)h(to)f(the)h(utilit)o(y)d(of)h(this)h(is)g(to)g(see)h (ho)o(w)e(one)i(w)o(ould)e(create)i(the)f(curren)o(t)75 493 y Fg(MPI)17 b Fh(p)q(ersisten)o(t)j(comm)o(unication)15 b(requests)k(with)f(the)h(new)f(mec)o(hanism.)28 b(F)m(or)18 b(example,)f Fg(MPI)p 1660 493 V 14 w(SEND)p 1781 493 V 15 w(INIT)75 549 y Fh(needs)g(a)e(buf,)g(coun)o(t,)h(dest)g(and)f (tag.)23 b(The)16 b(priv)n(ate)p 919 549 V 14 w(data)g(w)o(ould)e(b)q (e)i(the)g(place)g(to)f(put)h(this)g(information.)j(If)75 606 y(y)o(ou)c(ha)o(v)o(e)f(it)h(here)h(then)g(it)f(gets)g(passed)h(to) f(all)f(the)i(callbac)o(k)e(functions.)21 b(On)16 b(the)f(other)h (hand,)f(if)f(the)i(comm)75 662 y(is)f(asso)q(ciated)g(with)f(the)i (request,)g(one)f(could)f(use)i(the)f(cac)o(hing)g(mec)o(hanism)d(to)j (attac)o(h)f(this)h(information.)j(It)75 719 y(a)o(v)o(oids)13 b(another)h(argumen)o(t)f(and)h(is)f(more)g(general)h(but)h(mak)o(e)d (the)i(connection)h(to)f(the)g(request)i(less)e(direct.)166 858 y Fn(This)i(function)g(uses)f(the)g(callbac)o(k)i(function)f Fj(create)p 1092 858 14 2 v 17 w(fn)f Fn(de\014ned)i(as:)75 964 y Fb(typedef)23 b(int)g(MPI_Request_create_function\(M)o(PI_Comm)d (*comm,)j(MPI_Request)552 1020 y(*request,)g(void)g(*private_data,)f (void)h(*extra_state\);)166 1127 y Fn(This)16 b(function)g(returns)f (the)g(request.)166 1266 y Fi(Discussion:)54 b Fh(There)20 b(is)f(the)h(issue)g(that)f(the)g(request)i(returned)g(ma)o(y)c(use)j (\(ha)o(v)o(e)f(asso)q(ciated)g(with)75 1322 y(it\))g(sev)o(eral)i (pt-2-pt)e(requests.)37 b(F)m(or)20 b(example,)f(the)h(de\014ned)h (collectiv)o(e)f(op)q(eration)f(ma)o(y)f(need)i(to)g(do)f(sev-)75 1379 y(eral)h(sends)g(and/or)f(receiv)o(es)j(on)d(eac)o(h)h(pro)q(cess) i(to)d(b)q(e)h(implemen)o(ted.)33 b(Since)20 b(y)o(ou)f(kno)o(w)g(the)h (comm)d(and)75 1435 y(request)p 206 1435 13 2 v 17 w(t)o(yp)q(e)d(at)g (init,)f(it)h(mak)o(es)e(sense)k(to)e(allo)q(cate)g(what)f(y)o(ou)h (need)h(at)f(the)h(init.)i(The)e(question)f(is)g(where)h(to)75 1492 y(put)i(them.)25 b(They)17 b(could)f(b)q(e)i(added)f(to)f(the)h (priv)n(ate)p 947 1492 V 15 w(data.)26 b(They)17 b(could)f(also)g(b)q (e)h(cac)o(hed)h(on)o(to)e(the)h(comm.)75 1548 y(Since)e Fg(MPI)p 261 1548 V 14 w(REQUEST)p 463 1548 V 14 w(INIT)f Fh(do)q(es)h(not)f(return)h(other)g(argumen)o(ts,)e(these)j(seem)e(the) h(only)e(t)o(w)o(o)h(places)h(to)f(put)75 1605 y(the)g(information.)g (In)f(Marc's)h(prop)q(osal,)e(he)i(has)f(an)g(arra)o(y)p 1032 1605 V 14 w(of)p 1080 1605 V 15 w(requests)i(and)e(a)f(coun)o(t)i (returned)g(b)o(y)f(create)p 1813 1605 V 17 w(fn.)75 1661 y(What)h(is)h(not)g(clear)g(to)f(me)g(is)h(what)f Fg(MPI)p 753 1661 V 15 w(REQUEST)p 956 1661 V 14 w(INIT)g Fh(do)q(es)h(with)g(these)h(v)n(alues.)k(This)15 b(issue)h(applies)e (to)75 1717 y(all)f(the)h(callbac)o(k)f(functions.)166 1857 y Fn(A)k(request)f(is)h(b)q(egun)h(with)e(the)h(curren)o(t)g Fj(MPI)e Fn(functions)j Fj(MPI)p 1284 1857 14 2 v 16 w(ST)l(ART)f Fn(and)g Fj(MPI)p 1631 1857 V 16 w(ST)l(ART)l(ALL)p Fn(.)75 1913 y(Calling)g(these)e(functions)h(cause)f(the)h(start)p 842 1913 V 15 w(fn)f(callbac)o(k)h(function)g(to)f(b)q(e)h(in)o(v)o(ok) o(ed.)k(It)15 b(is)h(de\014ned)g(as:)75 2019 y Fb(typedef)23 b(int)g(MPI_Request_start_function\(MP)o(I_Comm)d(*comm,)j(MPI_Request) g(*request,)982 2076 y(void)g(*extra_state\);)166 2182 y Fn(Completion)c(of)g(the)f(request)h(is)g(done)h(with)f(the)f(curren) o(t)h(test)f(and)h(w)o(ait)f(routines.)31 b(T)l(o)19 b(do)f(a)75 2238 y(non-blo)q(c)o(king)f(test,)d(one)h(uses)h Fj(MPI)p 710 2238 V 16 w(TEST)p Ff(f)p Fj(ANY)p Ff(j)p Fj(SOME)p Ff(j)p Fj(ALL)p Ff(g)f Fn(whic)o(h)h(uses)f(the)g(callbac)o (k)h(function:)75 2345 y Fb(typedef)23 b(int)g (MPI_Request_test_function\(MPI)o(_Comm)e(*comm,)i(MPI_Request)f (*request,)481 2401 y(void)h(*extra_state,)f(MPI_Status)h(*status,)481 2458 y(int)g(*flag,)g(int)h(count,)f(int)g(*index\);)166 2647 y Fi(Discussion:)34 b Fh(There)16 b(is)e(the)g(question)g(of)g(ho) o(w)f(man)o(y)g(args)h(this)g(function)g(handles.)k(It)c(ma)o(y)f(b)q (e)h(called)75 2703 y(due)f(to)g(a)f(call)g(to)h Fg(MPI)p 437 2703 13 2 v 14 w(TEST)g Fh(and)g(only)f(need)h(to)g(c)o(hec)o(k)h (one)e(request.)20 b(On)13 b(the)g(other)g(hand,)g(it)f(ma)o(y)f(b)q(e) i(called)-32 46 y Fc(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 4 4 3 bop 75 -100 a Fn(4)894 b Fe(CHAPTER)15 b(1.)35 b(EXTERNAL)16 b(INTERF)-5 b(A)o(CES)75 45 y Fh(b)o(y)12 b Fg(MPI)p 208 45 13 2 v 14 w(TEST)p 326 45 V 15 w(ALL)h Fh(and)f(need)h(to)f (deal)f(with)h(lots)g(of)f(requests.)19 b(The)13 b(later)f(has)g(coun)o (t)g(and)g(index)g(argumen)o(ts.)75 102 y(T)m(o)g(only)g(mak)o(e)f(the) j(user)f(pro)o(vide)g(one)g(function,)f(this)h(prop)q(osal)f(has)h(the) h(more)d(general)i(one)g(and)g(the)g(simpler)75 158 y(one)h(is)g(a)f (subset)j(of)d(this.)18 b(Th)o(us,)c(coun)o(t)g(=)g(1)g(and)f(index)h (=)g(0)g(\(in)g(C\))f(when)i(there)g(is)f(a)f(single)h(request.)166 297 y Fn(T)l(o)h(do)f(a)h(non-blo)q(c)o(king)i(w)o(ait,)d(one)h(uses)g Fj(MPI)p 978 297 14 2 v 16 w(W)l(AIT)p Ff(f)p Fj(ANY)p Ff(j)p Fj(SOME)p Ff(j)p Fj(ALL)p Ff(g)g Fn(whic)o(h)g(uses)h(the)f (call-)75 354 y(bac)o(k)g(function:)75 438 y Fb(typedef)23 b(int)g(MPI_Request_wait_function\(MPI)o(_Comm)e(*comm,)i(MPI_Request)f (*request,)481 494 y(void)h(*extra_state,)f(MPI_Status)h(*status,)481 550 y(int)g(count,)g(int)h(*index\);)166 717 y Fi(Discussion:)34 b Fh(Marc's)15 b(prop)q(osal)e(only)g(has)h(one)h(completion)d (function.)18 b(I)c(think)f(it)h(is)g(more)f(consisten)o(t)75 773 y(with)h(the)g(rest)h(of)e Fg(MPI)h Fh(to)f(ha)o(v)o(e)h(b)q(oth)g (v)o(ersions.)19 b(The)14 b(user)h(can)f(alw)o(a)o(ys)f(giv)o(e)g(the)i (same)e(function)g(for)h(eac)o(h.)166 913 y Fn(A)22 b(request)h(is)g (freed)g(using)g(the)g(curren)o(t)f Fj(MPI)p 1023 913 V 16 w(REQUEST)p 1242 913 V 18 w(FREE)h Fn(whic)o(h)g(uses)g(the)g (callbac)o(k)75 969 y(function:)75 1053 y Fb(typedef)g(int)g (MPI_Request_free_function\(MPI)o(_Comm)e(*comm,)i(MPI_Request)f (*request,)481 1109 y(void)h(*extra_state\);)166 1193 y Fn(Finally)l(,)i(since)e(allo)q(cation)g(of)e(the)h(t)o(yp)q(e)p 904 1193 V 17 w(req)g(ma)o(y)f(use)h(some)g(system)f(resources,)i(it)f (can)g(b)q(e)75 1250 y(returned)16 b(to)e(the)h(system)g(with:)75 1401 y Fj(MPI)p 160 1401 V 16 w(TYPE)p 293 1401 V 17 w(REQUEST)p 513 1401 V 18 w(FREE\(t)o(yp)q(e)p 738 1401 V 17 w(req\))117 1478 y Fh(INOUT)62 b Fj(t)o(yp)q(e)p 401 1478 V 18 w(req)430 b Fh(T)o(yp)q(e)14 b(of)g(request)h(to)f(b)q(e) g(freed)166 1602 y Fn(There)19 b(are)g(sev)o(eral)g(other)g(utilit)o(y) h(functions)g(that)f(should)h(b)q(e)f(around.)32 b(T)l(o)19 b(tell)h(the)f(t)o(yp)q(e)g(of)75 1659 y(request)c(one)h(can)f(use:)75 1810 y Fj(MPI)p 160 1810 V 16 w(REQUEST)p 379 1810 V 18 w(TYPE\(request,)h(t)o(yp)q(e)p 776 1810 V 18 w(req\))117 1887 y Fh(IN)155 b Fj(request)452 b Fh(request)117 1959 y(OUT)108 b Fj(t)o(yp)q(e)p 401 1959 V 18 w(req)430 b Fh(t)o(yp)q(e)15 b(of)e(request)166 2083 y Fn(If)i(w)o(e)g(accept)h (the)f Fj(p)o(rivate)p 629 2083 V 16 w(data)h Fn(then)g(w)o(e)f(need)h (a)f(w)o(a)o(y)f(to)h(lo)q(ok)g(at)g(this)g(data.)20 b(This)15 b(allo)o(ws)h(one)75 2140 y(to)f(access)g(this)h (information.)75 2291 y Fj(MPI)p 160 2291 V 16 w(REQUEST)p 379 2291 V 18 w(D)o(A)l(T)l(A)p 512 2291 V 17 w(GET\(request,)g(t)o(yp) q(e)p 879 2291 V 18 w(req\))117 2368 y Fh(IN)155 b Fj(request)452 b Fh(request)117 2440 y(OUT)108 b Fj(p)o(rivate)p 449 2440 V 17 w(data)359 b Fh(data)14 b(asso)q(ciated)g(with)g(request)166 2647 y Fi(Discussion:)31 b Fh(One)12 b(can)f(completely)f(in)o(tegrate) h(this)g(new)g(requests)i(mec)o(hanism)c(with)i(the)g(old)f(w)o(a)o(y)m (.)16 b(W)m(e)75 2704 y(could)f(de\014ne)h Fg(MPI)f Fh(functions)g(and) g(constan)o(ts)h(to)f(do)g(exactly)h(what)f(the)h(curren)o(t)g Fg(MPI)f Fh(functions)g(do.)22 b(Th)o(us,)1967 46 y Fc(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 5 5 4 bop 75 -100 a Fe(1.2.)34 b(GENERALIZED)16 b(REQUESTS)1071 b Fn(5)75 45 y Fg(MPI)p 152 45 13 2 v 14 w Fa(f)p Fg(B)p Fa(j)p Fg(S)p Fa(j)p Fg(R)p Fa(g)p Fg(SEND)p 417 45 V 14 w(INIT)p Fh(,)18 b Fg(MPI)p 617 45 V 15 w(RECV)p 739 45 V 14 w(INIT)h Fh(could)g(b)q(e)h(b)q(e)h(in)o(tegrated.)35 b(One)21 b(can)f(also)f(ha)o(v)o(e)g(prede\014ned)75 102 y(t)o(yp)q(e)p 156 102 V 16 w(req)14 b(for)f(these.)-32 46 y Fc(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Tue Apr 18 17:42:30 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA05236; Tue, 18 Apr 1995 17:42:29 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA17467; Tue, 18 Apr 1995 17:41:46 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 18 Apr 1995 17:41:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id RAA17452; Tue, 18 Apr 1995 17:41:42 -0400 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id RAA25330; Tue, 18 Apr 1995 17:37:49 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id RAA09526; Tue, 18 Apr 1995 17:41:39 -0400 Date: Tue, 18 Apr 1995 17:41:39 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199504182141.RAA09526@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA19102; Tue, 18 Apr 95 17:41:38 EDT To: snir@watson.ibm.com Cc: mpi-core@CS.UTK.EDU In-Reply-To: <9504181643.AA21116@snir.watson.ibm.com> (message from Marc Snir on Tue, 18 Apr 1995 12:43:06 -0400) Subject: Re: MPI_CART_SHIFT -- clarification needed in MPI document Marc raises the point of whether to number from zero or 1 in MPI_CART_SHIRT. In MPI we generally use zero up numbering. We only use 1 as the first element in a Fortran array. For example, ranks are always 0 to p-1. Also, the corrdinates of cartesian topology are defined to use 0 up numbering and are row major. Given this, I think that zero up numbering should be used for both Fortran and C. Steve From owner-mpi-core@CS.UTK.EDU Tue Apr 18 20:57:45 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA07906; Tue, 18 Apr 1995 20:57:45 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id UAA02791; Tue, 18 Apr 1995 20:55:48 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 18 Apr 1995 20:55:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from win233.nas.nasa.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id UAA02779; Tue, 18 Apr 1995 20:55:41 -0400 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id RAA25740; Tue, 18 Apr 1995 17:55:27 -0700 Date: Tue, 18 Apr 1995 17:55:27 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199504190055.RAA25740@win233.nas.nasa.gov> To: mpi-core@CS.UTK.EDU Subject: process management comments and proposal Here are some comments on Bill and Rusty's dynamic process management proposal. I haven't seen any discussion yet, so this can hopefully get the ball rolling. The ideas here are rough (and unfortunately, not entirely proofread), but I thought it would be better to get something out for people to think about before the meeting. I'll try to TeXify and polish it in the next few days (Steve: I'll take care of making copies, since it's on the late side). Bill ---------------------------- Process Management Comments and Proposal Bill Saphir NAS Scientific Computing Branch NASA Ames Research Center wcs@nas.nasa.gov This note serves two purposes. First, I make some comments on the proposal of Bill and Rusty. That proposal is a good start in the MPI process management discussion. Second, I suggest some changes to their proposal that address some of the problems. There are enough changes that I am calling these changes a proposal in their own right, but they is really based on and should be seen in the context of the original proposal. Comments Three of the problems frequently cited when discussing MPI and dynamic process management are: 0. Maintaining implementation efficiency with dynamic groups. 1. Avoiding race conditions when adding/deleting processes (determinism) 2. Interaction of MPI with an external resource manager 3. Recovering from faults - unexpected/noncooperative deletion/failure of processes. (there are new and different race conditions). The key contribution of the Gropp/Lusk proposal is that it focuses attention on the runtime environment and the interaction between MPI and an external resource manager. In my opinion, avoiding race conditions is relatively easy. There are several ways to avoid them: spawning/deleting collectively, attaching collectively, enlarging communicators collectively. Recovering from faults is extremely difficult, or perhaps I should say that I've thought about it a lot and have come up with nothing at all. PVM, surprisingly, addresses both fairly well, but pays a huge penalty in functionality and performance that would probably not be acceptable to MPI. [I'm not entirely convinced, by the way, that MPI won't have to do a lot of PVM-like things in order to get the functionality that people want]. Interaction with an external environment is in a sense completely separate from the other issues. The key observation is that there are two interrelated but very different things going on: resource management task management [Bill and Rusty use the words "job scheduling" and "process management". I mean the same thing but am a little more comfortable with the new terminology]. The fundamental difference between resource and task management is that resource management resides logically "outside" the MPI application, whereas task management resides "inside." Details of resource management will necessarily vary greatly between machines and environments - the key task of the MPI Forum is to provide a robust way for MPI to interact with external allocators. Specific implementations (e.g., one based on PVM) may provide their own resource "management", but logically it is still external to the application. Task management is largely an internal issue. An application should be able to kill any one of its processes without "approval" from an outside agent. Once given access to resources, an application should be able to start up any task that is consistent with the resource limitations. The distinction between resource and task management appears in the Gropp/Lusk proposal in the form of two separate subroutines: MPI_Iallocate and MPI_Start_all. One must explicitly request resources before using them. While this is a step in the right direction, my feeling is that the division between the two should be even more well defined. The clue that something is wrong is the existence of too many MPI_Request objects as arguments to the new routines. Requests go through sort of a lifecycle, like eggs to caterpillars to moths, where at each stage they are really something different. I think it is more natural to call them caterpillars and moths instead of requests. The way I see it, you allocate resources, you start tasks (on a set of resources), you signal a set of tasks. When a task exists, you may want to start a new task on the same resource (node) as the first task. What you need is two new data types - a TASK, and a place to put a task, generally a set of resources, which I'll call a NODE. The requests still have an important role, but I think they should stick to being requests. I'll explain this in detail below. There are a few other issues as well. 1. resource reusability. Once an application requests and obtains resources, it should be able to use them in any way it chooses. In particular, it should be able to start a new task when an old one dies. Thus the "task farm" example of Gropp/Lusk should only need to allocate resources once. [Note added later: I've now read the task farm sample code in the proposal, and indeed "requests" (== resources) are being reused. IMHO it is nonintuitive to recycle them like this. ] 2. virtual machine model vs. process model for resources. In PVM, an application runs on a "virtual machine" consisting of a number of "hosts". The application can explicitly manipulate individual hosts separately from processes. PVM hosts and tasks roughly correspond to "resources" and tasks as I've described them for MPI. If a user doesn't specify a host when spawning, tasks are spawned in a round-robin fashion across the hosts. The distinction between resources and tasks isn't clean in PVM, though. The PVM interface basically assumes that the program has complete control over the machine and processes on the machine. You can now set things up so that a 3rd party filters (and fulfills) all requests, but in principle it can filter both requests to add hosts and requests to spawn tasks. So there is no longer any resource/task split - both are subject to external control. On the other hand, there is no mechanism to deal gracefully with that external control. For instance, there is no way to make an asynchronous request for resources (ala iallocate). Finally, there is a lot more to "resources" than just a host - PVM doesn't provide enough control. Before proposing some specific changes, I have some comments on other parts of the Gropp/Lusk proposal. MPI_COMM_MODIFY It's not clear to me how this function works and when it is used. If I understand its use in example 1, it is run in both spawner and spawnee and has a different meaning in each. In spawner, it allocates, spawns, and then attaches the children. (It is presumably collective over the original MPI_COMM_WORLD). In the spawnee, it identifies the parent and attaches to the original MPI_COMM_WORLD. For this to work, a a process should know how to spawn (or make copies) of itself. I don't think this is feasible in practice. What about arguments? When would this be useful in practice? I think what the proposal addresses is the common practice in PVM of starting one process and having that process start identical copies of itself. This is a silly way to start a SPMD program (forced by what I might consider deficiencies in PVM) and I don't think we need to try to deal with it. MPI already does very well (and more naturally) with SPMD programs. In general, I'm uncomfortable with the idea of changing existing communicators. What if there is pending communication on the communicator being changed? I don't think there is any way to guarantee that there isn't any. A library may have started some. I suggest that whenever adding or deleting processes, we create a new communicator. MPI_COMM_WORLD always refers to a process and its siblings, that is other processes spawned/created at the same time. It can never be enlarged. MPI_CLIENT_CONNECT MPI_SERVER_CONNECT Note there is no communicator enlarging here. - good. Note that in client/server apps, there are no resource allocation issues. If the client exists, the resources have already been allocated (outside MPI). MPI_IALLOCATE The main problem here, is that you make a request, you wait for the request to complete, and you end up with ... a request! I'll suggest how to fix this later. A second problem is that it's biased to the "host" model of resources. An application needs to be able to request specific hosts, but my feeling is that should be more on the resource manager side (inside the "array_of_js_dep_strings". We're going to have to think quite a bit about how to specify resources. I'm happy to put this off for a while. A third problem is that resources are explicitly not reusable. A "request" goes through a lifecycle of (resource_request -> resource -> resource + task_request -> resource+task). There is no possibility of reusing a resource if a task dies (intentionally or unintentionally). I'd like to arrange things so that you only call iallocate when you actually need new resources. Incidentally, in "static" MPI implementations, I assume that resources will actually be allocated before an application starts. Iallocate() will simply look up these resources in a predetermined list. MPI_SET_EXEC(request, executable) MPI_SET_ARGS(request, args) Very strange to set the executable for a "request". In my view, this is a symptom that something is wrong. MPI_GET_NODENAME(request, hostname) Same comment. I'm not sure that a request need have a "nodename" by the way. After resources are allocated, allocated, but before a task is spawned, a request may simply be a promise that a resource is a available. The hostname may not be determined until the job actually runs (based on, say, the lowest loadave at the time it runs). Hostname may also not be constant if we ever have job migration. (ha ha). MPI_GET_RETURN_CODE(request, code) Same comment about the many uses being made of "request". Incidentally, For all of these functions taking a "request" as an argument, what happens if you call them on a different type of request? MPI_COMM_PARENT_CREATE I'm not sure I see how this is used. Why not use MPI_REMOTE_ATTACH? MPI_SIGNAL(signal, num, array_of_requests) Still strange to send a "signal" to a "request". I'd prefer to send a signal to a task. ------ Final note on collectivism and "request" objects. In MPI so far, request objects are local. MPI_Wait, MPI_Test, etc. are local. Implicit in the Gropp/Lusk proposal, I think, is the idea of collective requests, which I think we should avoid. The SPAWN, and IALLOCATE, and START_ALL routines do not have a communicator argument. In that case, a single node does the resource request and spawning. I think this is the right thing. Other proposals (e.g. Skjellum/Doss) have collective spawning as an integral part. However, MPI_COMM_PARENT_CREATE, MPI_IACCEPT, and MPI_REMOTE_ATTACH all take a communicator argument. Must that communicator always contain only a single node? I suspect this issues hasn't been thought through, but it's something we need to address. It is my feeling that all allocation/spawning should occur on a single node. This feels more "natural" and it makes it easier to deal with "ownership" or resources. ================================================== Proposal/Modifications Interface: Two new MPI data types. MPI_Task A TASK is what we commonly think of as an MPI task. It is an program executing on a processor. It may or may not be "registered" with MPI. I haven't thought all of this through, but I think it might make sense (for completeness and robustness) to see an MPI_Group as being made up as a bunch of MPI_Task objects. I believe this concept could be added to the original standard with no change. You can't communicate directly with a task. But you send signals to task. MPI_NODE A NODE is an object on which a TASK executes - a container for the task. A NODE is requested from and allocated by a resource manager. It is not necessarily a physical processor. [I don't like the name "NODE" because it is too restrictive. I toyed with the idea RESOURCE but that is confusing. Suggestions welcome] A big issue is how many tasks can fit in a NODE. I propose that NODES come in two flavors. Nodes which can hold one task and nodes that can hold an unlimited number of tasks. The one-task model is correct for almost all applications. You almost never want to timeshare a parallel task against itself (i.e. with two processes on the same processors) and one task/node enforces this idea. However, there are some times when you want a PVM-type model in which you "own" a node and can do anything you want with it. In the rest of this I assume one task/node. I believe that a NODE has to have an owner. It is not possible to pass NODES around. On the other hand, any process can get a TASK object associated with any other process. How does this work? 1. request resources 2. get back an array of nodes from resource request 3. request for a certain executable to be spawned on the nodes. 4. get back an array of tasks from spawn request More specifically: MPI_Iallocate(resource_description, array_of_NODES, request) IN resource_description OUT array_of_NODES (type MPI_Node) OUT request array_of_NODES is set asynchronously by MPI (as with other MPI_I* routines). You can't use them until you've waited for the request to complete. You wait for the request to complete with MPI_Wait, test with MPI_Test, etc. [Possibly should call it "MPI_allocate_nodes", but I wanted to make it recognizable]. Possibly MPI_Iallocate is NOT a collective operation. The NODES returned are "owned" by the task that called MPI_Iallocate. Making it collective has serious problems, unless all future operations on the returned NODES are collective. This adds unnecessary complexity. I'm being deliberately vague about the resource description. In the Gropp/Lusk paper it is basically number of nodes+hardness, js dependent info, arch type, nodename list. I think this is probably too restrictive, but I'm not sure what the right way to specify resources is. Resource description probably has to have some architecture-dependent component but I think it should be robust enough so that you can have a portable code that works almost everywhere. The types of things that might have to go into this are: a number of nodes requested. This may be a range to allow softness. amount of time needed (possibly soft and hard limits). whether job requires dedicated or gang-scheduled nodes, or can dynamically load-balance specific node names specific node types (architecture) how soon you need the nodes (can you wait 20 minutes?) other resources (HiPPI connection, large memory, etc) whether application is willing to give up the resources if the resource manager wants them back. While I'm not sure how to implement the resource description, I think it can be done. Note: a resource description could be something as simple as "N", and this simple case should be easy to specify. In any case, you get out of this thing a set of nodes. A set of N nodes may not actually be N specific nodes. All it is is a promise from the resource manager that you can start N tasks and that those tasks will have the resources you requested. Depending on how you requested the resources (and the resource allocator) you may end up with N tasks on a single physical processor! Perhaps it is best to think of a "node" as a reserved slot in the process table! Of course if you specified by node-name (just another resource) you'll get the nodes you asked for. In some implementations (Condor on a large NOW) the RM may choose the nodes according to smallest load average when you actually spawn the tasks. I don't want to specify in this description, and don't think it's necessary, to say that there is necessarily interaction with a resource manager. For instance, on the SP2 you might launch an N+1-node job for 1 manager and N workers. Only the manager actually starts up. It requests N nodes, which it gets immediately, because MPI knows that it has a bunch internally in reserve. If one were trying to mimic plain pvm, you'd get whatever resources you asked for. I said earlier that you would do an MPI_Wait or an MPI_Test to check for completion. These are not specifically addressed in the paper. Presumably you do MPI_Wait(request, status) and internally it calls a potentially user-defined wait function specified in MPI_New_request_type. I'm not sure of the precise interface, but I'd want something like MPI_Get_count(status) to tell you how many nodes you actually got (0, N, or a number in between for soft requests). Now for tasks. You have just gotten N nodes and you want to spawn N tasks on those nodes: MPI_Exec(executable, arguments, n, array_of_nodes, array_of_tasks) IN executable IN arguments IN n IN array_of_nodes OUT array_of_tasks As before, the array_of_tasks is set asynchronously, and you can't look at it until the request completes. Note that the resources have already been allocated. Unless you have an error in arguments (executable does not exist, or something) MPI_Exec normally works. You don't have to worry about waiting for the resources from the resource manager. [You may have to contact the resource manager, but the resources have been promised and should be available immediately]. I specifically have one executable name and one set of arguments. If you want multiple executables or different arguments, you call this twice. There is not a huge amount of overhead because you don't have to go back to the resource manager. As I mentioned earlier, this is logically and "internal" operation. [BTW, this is a drawback of the original proposal, I think. You should be able to specify different executables but going to the resource manager with several separate requests, one for each executable, makes it much harder for the resource manager. [Note: in this implementation, all tasks that you exec at the same time have the same executable and same arguments. I don't think this is a problem. Just use multiple Exec calls to spawn multiple executables. If you want to attach to all of them, you can wait until you've exec'd all of them before you ATTACH to them. ] [Note: there is no "request" argument. There is an obvious Iexec that could have this argument. Since starting a task is "internal" to MPI and resources have already been allocated, I'm assuming MPI_Exec would normally return quickly] It would be possible to have a single routine - mpi_spawn(), that could do both the resource allocation and the task starting. My initial impression is that this isn't necessary. Additional "utility" routines that would be nice (I'm sure there would be more): MPI_Task_node(task, node) IN task OUT node gets the node on which a task is running MPI_Node_task(node, task) IN node OUT task gets the task running on a given node. Returns MPI_TASK_NULL if no task (node is then free for a new task) MPI_Free_nodes(nodes, n) IN nodes IN n releases resources to resource manager. Tasks running on those nodes are lost (killed?). (dealing with this case is difficult and should should be addressed). MPI_Task(comm, rank, task) IN comm (could be a group?) IN rank OUT task gets the task corresponding to a given comm/rank. Tasks are in a way funny objects. You can't communicate with them directly. They have to be associated with communicators. They allow out-of-band communication, such as signals, and, I suspect, can be used to handle failures in a moderately graceful way (I haven't thought it through). You need to have some way of finding out when a task finishes. In MPI we have no mechanism for asynchronous notification. I therefore propose: MPI_Notify_when_done(array_of_tasks, count, request) IN array_of_tasks IN count (number if tasks) OUT request To test to see if a task has completed, use MPI_Test(request) To wait until it has completed, use MPI_Wait(request) The way I've specified it above, you can set things up so that a single request completes only when some number (possibly one) of TASKS have all completed. I'm not sure this is the right way to do it. If you want to independently monitor a large number of tasks, you would do MPI_Notify_when_done(&task[0], 1, &request[0]); MPI_Notify_when_done(&task[1], 1, &request[1]); MPI_Notify_when_done(&task[2], 1, &request[2]); MPI_Testany(n, request, index, flag); Again, I haven't thought it through, but this type of mechanism may be able to address problems of tasks dying unexpectedly. (I don't know how to handle communicators, though). Note that issues of tasks dying, expectedly or unexpectedly, exist within MPI-1. The issues are not new - just more important. ---------- MPI_Attach(comm, root, array_of_tasks, numtasks, intercomm) IN comm IN root IN/OUT? array_of_tasks IN/OUT? numtasks OUT intercomm The goal is to establish some method of communicating with the new tasks. You can do this by creating an intercommunicator or creating a containing communicator or resizing an existing communicator. I'm philosophically opposed to resizing communicators. You can create a containing communicator from an intercommunicator very easily by using MPI_Intercomm_merge. So I opt for creating an intercommunicator. MPI_Attach is collective over comm. array_of_tasks and numtasks are valid only in the root process and are possibly (?) set in the other processes. Why make it a collective operation? Two reasons: 1. if it is a single-node operation, you still need a comm argument, which would be MPI_COMM_SELF or some duplicate. 2. it makes it easier for the spawned processes to get in touch with the whole world. On the spawned task side, all this happens automatically when you call MPI_Init. MPI_Init creates an MPI_COMM_WORLD containing all your siblings. It is also the "other side" of the MPI_Attach operation. You get access to the intercommunicator by calling MPI_Parent. Siblings defined as all the new tasks that participate in the MPI_Attach operation, not all tasks that were spawned on one operation. MPI_Parent(intercomm) OUT intercomm I don't think its wise to have the children call a special routine to get initialized. MPI_Init should connect you to the MPI universe. This MPI_Parent/MPI_Attach scenario allows you to manage spawned processes that become part of MPI as well as spawned processes that never call MPI_Init. You can always SIGNAL them with MPI_SIGNAL. --------------------------------------------------- Examples (Note: these are off the top of my head (possibly not thought out as well as they should be) and undoubtedly contain mistakes. Please excuse syntax problems). Dynamic sizing example: I don't think this is too useful as specified in the original paper. I'll replace it by a manager/worker example. Manager/Worker example /* manager */ #include "mpi.h" main(int argc, char *argv[]) { int n; MPI_Node nodes[MAXPROC]; MPI_Task tasks[MAXPROC]; MPI_Request request; MPI_Comm my_children, my_family; Resource_Description descrip; MPI_Init(&argc, &argv); ... /* set up resource description here. It includes a hard or soft number of nodes */ MPI_Iallocate(descrip, nodes, &request); MPI_Wait(request); n = MPI_Get_count(request); /* first n elements of "nodes" are valid */ MPI_Exec("worker", NULL, nodes, tasks); /* could check for errors here. Assume all n succeed */ MPI_Attach(MPI_COMM_SELF, 0, tasks, n, &my_children); /* * parallel code here. the intercommunicator "my_children" * can be used to communicate with the spawned processes */ MPI_Finalize(); } /* worker */ main(int argc, char *argv[]) { MPI_Init(&argc, &argv); if (MPI_COMM_PARENT == MPI_COMM_NULL) { error("Where's my mommy?"); abort(); } /* parallel code here. MPI_COMM_PARENT is an intercommunicator * that allows all nodes to talk to their parent (in this case, * the single manager process that spawned them). */ MPI_Finalize(); } Task farm example. The nice thing here is that you need allocate resources only once. Then you can reuse them. It also shows that you can interact with processes without using MPI to communicate. /* manager */ #include "mpi.h" main(int argc, char *argv[]) { int i, n, nthings_done; MPI_Node nodes[20]; MPI_Task tasks[20]; MPI_Request request[20]; Thingtodo tbd[NTHINGSTODO]; MPI_Status status, statuses[20]; char **args; Resource_Description descrip; MPI_Init(&argc, &argv); ... /* set up resource description here. For this example, assume it is a hard request, so we will get 20 or 0 nodes */ MPI_Iallocate(descrip, nodes, &request); MPI_Wait(request); /* set up tbd array with everything that needs to be done */ /* assume NTHINGSTODO>20 */ for (i = 0; i < 20; i++) { my_set_args(tbd[i], &args); MPI_Exec("cow", args, &node[i], &tasks[i]); MPI_Notify_when_done(&task[i], 1, &request[i]); } nthings_done = 20; /* at this point we have 20 processes running. wait for a process to finish and start a new one in its place until we've exhausted the list of things to do */ while (nthings_done < NTHINGSTODO) { my_set_args(tbd[nthings_done], &args); MPI_Waitany(20, request, &n, status); MPI_Exec("worker", args, &node[n], &tasks[n]); MPI_Notify_when_done(&task[n], 1, &request[n]); nthings_done++; } /* no more things to do, but we've got 20 processes still running */ MPI_Waitall(20, request, statuses); MPI_Finalize(); } Client/Server. I don't have any changes to make to Bill and Rusty's. Client/Server stuff doesn't really involve a resource manager. From owner-mpi-core@CS.UTK.EDU Tue Apr 18 21:15:37 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA08113; Tue, 18 Apr 1995 21:15:36 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id VAA04054; Tue, 18 Apr 1995 21:14:50 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 18 Apr 1995 21:14:49 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id VAA04046; Tue, 18 Apr 1995 21:14:47 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA16815; Tue, 18 Apr 95 20:14:37 CDT Date: Tue, 18 Apr 95 20:14:37 CDT From: Tony Skjellum Message-Id: <9504190114.AA16815@Aurora.CS.MsState.Edu> To: mpi-core@CS.UTK.EDU, wcs@nas.nasa.gov Subject: Re: process management comments and proposal Let me make a general statement about modifying groups and communicators. A lot of the good features of MPI revolve around the fact that it is object-oriented: groups and communicators do not get modified after creation, and good implementations may exploit this in handling such objects, as well as providing clean semantics. Features like MPI_COMM_MODIFY are a bad idea. They change this principle. I am strongly against such functionality; It provides little that an MPI_COMM_REPLACE wouldn't do, but it breaks the beneficial semantics of these objects. In fact, optimizations that work on the principle of being done when communicators are instantiated would be lost: For instance, a good implementation that exploits special features (homogeneity, powers of two, location on mesh, shared memory) may define methods that implement MPI functionality with communicator scope. MPI_COMM_MODIFY would wreak havoc with such optimizations. So, while I am not against having convenient functions for users, as I believe Bill/Rusty intend, I think that we should formally avoid routines that break the object-nature, and static nature of existing objects. This does not prohibit dynamicism; it just structures it as transformations from object to object. I can rarify and clarify this argument further, if desired :-) -Tony From owner-mpi-core@CS.UTK.EDU Wed Apr 19 04:19:22 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA13911; Wed, 19 Apr 1995 04:19:22 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA29095; Wed, 19 Apr 1995 04:18:06 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 19 Apr 1995 04:18:05 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA29087; Wed, 19 Apr 1995 04:17:54 -0400 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA09979 (5.65c8/IDA-1.4.4 for ); Wed, 19 Apr 1995 10:17:27 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA17271; Wed, 19 Apr 1995 10:17:28 +0200 Date: Wed, 19 Apr 1995 10:17:28 +0200 From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9504190817.AA17271@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: dimension in MPI_CART_SHIFT Cc: falk.zimmermann@gmd.de, gmap10@f1neuman.gmd.de I would like to comment on the MPI_CART_SHIFT issue which Marc has raised. It's true that the standard is not as precise at this point as it should be. I agree with Steve Huss-Lederman who sais: > In MPI we generally use zero up numbering. We only > use 1 as the first element in a Fortran array. So, the coordinate direction would be numbered zero up in both Fortran and C. As Marc points out, this choice is consistent with the MPICH implementation. If we don't want to change the standard document, but rather clarify the situation, we have one choice only. In example 6.4 (IJSA, page 368), a Fortran code shows a shift in column direction. Remember that this is the column direction in a 2D process array which has row-major numbering, independent of the programming language. Thus, the columns are along the second coordinate direction. In the example, direction is set to 1, which means that it is counted zero up. I agree that this is not explicit enough. So, I would recommend to add an advice to users like follows: Note that "direction" denotes the coordinate direction in the process array. It is defined in row-major order, with the coordinate directions being numbered zero up, independent of the programming language. -- Rolf Hempel From owner-mpi-core@CS.UTK.EDU Wed Apr 19 06:43:46 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA20580; Wed, 19 Apr 1995 06:43:46 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id GAA13081; Wed, 19 Apr 1995 06:41:30 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 19 Apr 1995 06:41:25 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id GAA13055; Wed, 19 Apr 1995 06:41:03 -0400 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA03450 (5.65c/IDA-1.4.4 for mpi-core@cs.utk.edu); Wed, 19 Apr 1995 11:40:54 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA24133; Wed, 19 Apr 1995 11:40:52 +0000 Date: Wed, 19 Apr 1995 11:40:52 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9504191040.AA24133@deneb.co.uk> To: mpi-core@CS.UTK.EDU Subject: External interfaces Steve has produced an interesting document, however I'm nervous that we're getting into syntax before we understand the semantics. This is particularly easy to do in cases like this where we are invoking user defined functions from inside the MPI implementation, so we specify in some detail how the user registers her functions, but are very sparse in describing what the user functions have to do ! I am particularly concerned about what the model is which is being used to describe the implementation of MPI_waitany, since I think this is the hardest to get correct. In particular Steve describes that "MPI_WAIT{ANY|...} uses the callback function MPI_request_wait_function ...", but there is no specification for what the blocking semantics of this user function are (though since it is also used for the simple wait, I assume it is intended to block until the request is complete). There are two possible models for how the MPI_WAITANY code works 1) It is a busy loop, polling each of the requests for readiness. 2) It uses something analogous to BSD's select (or Sys V's poll) functions to suspend itself non-busily. Given that the set of requests which are being WAIT_ANY'ed upon will include "normal" MPI requests and "user" requests, there is a problem. If we are using a polling implementation, then WAIT_ANY actually needs to call the Request_test function, NOT the Request_wait function (in contradiction to the statement in Steve's document), otherwise it will block on the "user" request, and never test the system ones. However if we want to implement a non-busy WAIT_ANY (clearly a desirable objective), then the user Request_wait function is still insufficient. (Since we block in it, but somewhow we need to be able to register interest in its completion, and then cancel that interest. The user functions will also need callbacks into MPI to asynchronously inform MPI that the status has changed). To bring this discussion down to earth a bit, I'd like to see proposals in this area demonstrate how one could use the facilities proposed to create an MPI_REQUEST which could be used for input from a file descriptor (or the arrival of an X event if you prefer), since in conjunction with MPI_wait_any this is actually a useful thing to do, providing the ability to "Wait for a message, or input from the keyboard", or "Wait for a message or an X event". -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From owner-mpi-core@CS.UTK.EDU Wed Apr 19 06:47:48 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA20625; Wed, 19 Apr 1995 06:47:48 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id GAA13350; Wed, 19 Apr 1995 06:47:46 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 19 Apr 1995 06:47:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from gmdzi.gmd.de by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id GAA13337; Wed, 19 Apr 1995 06:47:38 -0400 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA19975 (5.65c8/IDA-1.4.4 for ); Wed, 19 Apr 1995 12:47:26 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA17879; Wed, 19 Apr 1995 12:47:28 +0200 Date: Wed, 19 Apr 1995 12:47:28 +0200 From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9504191047.AA17879@f1neuman.gmd.de> To: mpi-core@CS.UTK.EDU Subject: Data representations Cc: falk.zimmermann@gmd.de, gmap10@f1neuman.gmd.de, springstubbe@gmd.de There has been some discussion of my proposal of a data representation attribute lately. I would like to comment on some of the arguments. I completely agree that MPI users should use the types message formalism of MPI whenever they write new programs. In many applications the same data structures are sent frequently, and in those cases the programmer should use the derived datatypes. In other cases he can directly use the pack/unpack functions to build the message buffer. The problem I want to solve with my proposal is about the conversion of legacy codes into MPI. In most popular message passing interfaces messages have been defined as byte strings with a start address and length in bytes. If you want to send data of more than one type in a single message, there are several ways to build the buffer. It may be done by equivalencing vectors of different types, by passing parts of a real array into a subroutine and fill it with integers there, or by any other funny trick Fortran programmers use all the times. If you now want to rewrite such a program into MPI, the solution is straightforward only in the case that the data representations match between sender and receiver. If that cannot be guaranteed, you have to rebuild the buffer supplied by the user program in some way or another. Either you have to set up derived datatypes or you have to use pack/unpack. I don't think it would be practical to keep a list of derived datatypes and, for each send, try to find a matching one before a new datatype is created. This operation does not seem cheap to me. To go back in the user code and find the place where the buffer has been packed and insert the MPI_PACK routines there is difficult, to say the least. Remember there are so many tricks in Fortran. Now, it would be okay if you had to pay for rebuilding the buffer only in cases where the data representations don't match. That is why I would like to know about the match/mismatch beforehand, so that I could execute the corresponding branch. To summarize, the question is whether or not we want to make it easy for programmers to port their old codes to MPI without sacrificing too much performance, and not whether we want to let people to bypass the message typing in MPI in their applications. Several people already pointed out that my proposal does not cost much implementation work, and that it does not cause any performance overhead. I agree with Jim that the function MPI_Representation_equal proposed by Lloyd does not have these properties. I hope that the issue is clear enough now. Unfortunately, I will not be present at the meeting next week, so I asked Hans-Christian Hoppe of PALLAS to defend my proposal. Please, don't kill him instead of me! - Rolf From owner-mpi-core@CS.UTK.EDU Wed Apr 19 09:52:14 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA23302; Wed, 19 Apr 1995 09:52:14 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA23598; Wed, 19 Apr 1995 09:49:20 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 19 Apr 1995 09:49:19 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id JAA23590; Wed, 19 Apr 1995 09:49:17 -0400 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id JAA28174 for ; Wed, 19 Apr 1995 09:45:20 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id JAA17383 for ; Wed, 19 Apr 1995 09:49:11 -0400 Date: Wed, 19 Apr 1995 09:49:11 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199504191349.JAA17383@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA20173; Wed, 19 Apr 95 09:49:10 EDT To: mpi-core@CS.UTK.EDU In-Reply-To: <9504191040.AA24133@deneb.co.uk> (jim@meiko.co.uk) Subject: Re: External interfaces Hello again, I want to thank Jim for reading what I wrote. He is absolutely correct that we should not get hung up on syntax. What Jim is discussing is exactly what I would like to see. I was hoping that a concrete proposal would start a discussion. To be honest, I have been busy until yesterday. I really wanted to start with two examples. The first is to implement one of the current persistant requests and the other is a user defined one. I did not have the time and wanted to get out something yesterday. I still hope to do the examples for discussion at the meeting next week. I have already remembered one other topic I forgot. Bill and Rusty proposed a FREEALL function. This raises the topic of whether we want a mechanism to create, start and free multiple requests. MPI currently has a mechanism for multiple test/waits but the persistant requests are created, started and freed one at a time. Finally, I wanted to comment about what the interface subcommittee is doing. I was told at the last meeting that we needed to move quickly on the handles extension since other subcommittees need it soon. This is why I pushed with a proposal to make sure a serious discussion began. There are other topics that need to be discussed. I plan to have a more general discussions at the meeting next week to start to define these areas. Input and suggestions are welcome. Steve From owner-mpi-core@CS.UTK.EDU Wed Apr 19 10:07:47 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA23481; Wed, 19 Apr 1995 10:07:47 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA25245; Wed, 19 Apr 1995 10:07:06 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 19 Apr 1995 10:07:05 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id KAA25238; Wed, 19 Apr 1995 10:07:03 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA03985 for ; Wed, 19 Apr 1995 09:07:00 -0500 Message-Id: <199504191407.JAA03985@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: MPI Forum meeting, April 24-26 Date: Wed, 19 Apr 1995 09:06:58 -0500 From: Rusty Lusk Dear group, Could you please let me know whether or not you are coming to the meeting next week? I need to check with the hotel to make sure everyone's reservation is in order and to get a count for the food. Thanks. Rusty From owner-mpi-core@CS.UTK.EDU Wed Apr 19 14:04:21 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA28498; Wed, 19 Apr 1995 14:04:20 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA14446; Wed, 19 Apr 1995 14:03:45 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 19 Apr 1995 14:03:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA14438; Wed, 19 Apr 1995 14:03:41 -0400 Received: from eel.ssd.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA28415; Wed, 19 Apr 95 11:02:52 PDT Date: Wed, 19 Apr 95 11:02:52 PDT Message-Id: <9504191802.AA28415@SSD.intel.com> X-Sender: prp@eudora.ssd.intel.com X-Mailer: Windows Eudora Version 2.0.3 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: wcs@nas.nasa.gov (William C. Saphir), mpi-core@CS.UTK.EDU From: prp@ssd.intel.com (Paul Pierce) Subject: Re: process management comments and proposal Thanks, Bill, for some good work on process management. There is a lot to discuss, but here I would like to introduce just one idea. It struck me that - 1. There isn't much difference between the new TASK and existing process concepts. 2. I'm uncomfortable adding to MPI explicit ability to manage non-MPI things. and so I propose to merge TASK into the existing process abstraction. We already identify a process as a rank in a communicator. We could merge MPI_Exec from Bill's proposal with a special case of MPI_Attach: MPI_Exec(executable, arguments, n, array_of_nodes, comm) IN executable IN arguments IN n IN array_of_nodes OUT comm where the output communicator is an inter-communicator between MPI_COMM_SELF from the calling process and MPI_COMM_WORLD of the new process group. That same inter-communicator is MPI_COMM_PARENT in the new group. This immediately leaves us with the common host-node model all set up and ready to go. To get an inter-communicator as from MPI_Attach, simply use MPI_INTERCOMM_CREATE (but see below.) Then MPI_INTERCOMM_MERGE yields the new world. This avoids adding the complexity of a new object and eliminates the need for the translation operation MPI_Task. MPI_Node_task and MPI_Task_node most reasonably reduce to something like the original MPI_Task: MPI_Node(comm, rank, node) IN comm IN rank OUT node or you might want them all at once: MPI_Node_list(comm, node_array) IN comm OUT node_array Similarly for process termination (was MPI_Notify_when_done): MPI_Notify_termination(comm, rank, request) MPI_Notify_termination_list(comm, request_array) By the way, there appears to be a small hole in MPI_INTERCOMM_CREATE. When used as above the peer communicator wants to be an inter-communicator, but the spec says it must be intra. I don't see any reason not to eliminate that requirement since its just used for communication in a normal way. We should consider fixing this, it might come up somewhere else. Paul Pierce From owner-mpi-core@CS.UTK.EDU Wed Apr 19 19:46:00 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA04244; Wed, 19 Apr 1995 19:45:59 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA08495; Wed, 19 Apr 1995 19:44:49 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 19 Apr 1995 19:44:48 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from mobydick.usfca.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id TAA08488; Wed, 19 Apr 1995 19:44:44 -0400 Received: (from peter@localhost) by mobydick.usfca.edu (8.6.9/8.6.9) id QAA00196 for mpi-core@cs.utk.edu; Wed, 19 Apr 1995 16:46:55 -0700 Date: Wed, 19 Apr 1995 16:46:55 -0700 From: Peter Pacheco Message-Id: <199504192346.QAA00196@mobydick.usfca.edu> To: mpi-core@CS.UTK.EDU Subject: Access to stdio Currently MPI imposes no restrictions on the I/O capabilities of an implementation. In fact, there are implementations that provide no access to stdin and allow access to stdout and stderr by only one process. I think that this is going to be a big obstacle to the widespread use of MPI. In spite of the fact that much work has been done on the development of tools for parallel program development, the fact remains that the vast majority of programmers rely heavily on access to stdout and stderr during program development. Furthermore, many programmers will be reluctant to write programs for a system that forces them to recompile their code each time they wish to test it with different input. PROPOSAL: Any MPI compliant system must allow for the possibility that programs will be run interactively by 1. Allowing access to stdin by at least one process in MPI_COMM_WORLD (scanf, READ, etc.), and 2. Allowing access to stdout and stderr by all processes in MPI_COMM_WORLD (printf, fprintf(stderr,...), fflush, PRINT, etc.). DISCUSSION: I realize that we voted at the last meeting to not address the issue of _parallel_ I/O. However, I don't think that this proposal violates that decision, since it essentially specifies that MPI programs be capable of carrying out conventional I/O. This capability does not have to be automatically provided to all program executions. Rather, if the implementor so desires, it can be an option that is requested by the user at execution time. Existence of this capability does not imply that output will be "intelligently delivered." That is, if two processes simultaneously attempt to print data, there is no guarantee that the output of one process will be completed before the output of the other begins. It would seem that a good implementation would guarantee that the output from distinct processes would not be jumbled. Should this be required? From owner-mpi-core@CS.UTK.EDU Thu Apr 20 04:37:20 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA14720; Thu, 20 Apr 1995 04:37:20 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA16407; Thu, 20 Apr 1995 04:36:36 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 04:36:35 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA16398; Thu, 20 Apr 1995 04:36:27 -0400 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA27691 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Thu, 20 Apr 1995 09:36:13 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA24463; Thu, 20 Apr 1995 09:36:10 +0000 Date: Thu, 20 Apr 1995 09:36:10 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9504200836.AA24463@deneb.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: <199504192346.QAA00196@mobydick.usfca.edu> (message from Peter Pacheco on Wed, 19 Apr 1995 16:46:55 -0700) Subject: Re: Access to stdio Peter says > Currently MPI imposes no restrictions on the I/O capabilities of > an implementation. In fact, there are implementations that > provide no access to stdin and allow access to stdout and > stderr by only one process. I think that this is going to be > a big obstacle to the widespread use of MPI. In spite of > the fact that much work has been done on the development of tools > for parallel program development, the fact remains that the vast > majority of programmers rely heavily on access to stdout and stderr > during program development. Furthermore, many programmers will > be reluctant to write programs for a system that forces them to > recompile their code each time they wish to test it with different input. He then goes on to propose that we REQUIRE acces to stdin and stdout. I think that this is a BAD idea for the following reasons 1) If I/O is available, then stdin and stdout are also necessarily available if the language supports them. ("language standard I/O facilities" are available). 2) stdin and stdout are properties of an operating system and environment, which not all OSes support. We should not be trying to enforce a Unix-centric view of the world in MPI. 3) we already see MPI being used in embedded style applications (go talk to Lloyd !) These environments may have no concept of stdin and stdout. (Where should stdin come from on a lunar orbiter ?) We don't want to prevent such implementations being MPI conformant. 4) It is not MPI that fails to provide access to stdin and stdout, it is poor implementations (sorry if I offended anyone there )-:). I would expect implementations in which it makes sense (i.e. at least those in a Unix environment) to provide access to stdin and stdout somehow. Have a good meeting next week. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From owner-mpi-core@CS.UTK.EDU Thu Apr 20 09:51:37 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA22397; Thu, 20 Apr 1995 09:51:37 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA12251; Thu, 20 Apr 1995 09:51:16 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 09:51:14 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id JAA12178; Thu, 20 Apr 1995 09:51:06 -0400 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id JAA04622 for ; Thu, 20 Apr 1995 09:47:10 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id JAA00670 for ; Thu, 20 Apr 1995 09:51:01 -0400 Date: Thu, 20 Apr 1995 09:51:01 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199504201351.JAA00670@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA24547; Thu, 20 Apr 95 09:51:01 EDT To: mpi-core@CS.UTK.EDU Subject: comments on Bill Saphir's proposal Thanks to Bill for making a coherant proposal and discussion. There are many things to discuss, but two things hit me on my first pass. 1) In MPI_Exec you specify an executable. I agree that if you want two different programs on different sets of nodes you can call this twice. What I am trying to figure out how to do is launch the same program on heterogeneous nodes where you need different executables. Since array_of_nodes came from MPI_Iallocate and this really came from a resource manager it may well be heterogeneous. It seems painful to have to check each node to see its arch and then do lots of MPI_Exec calls that get combined later. I may have missed something here. 2) There is MPI_Notify_when_done (sorry Bill, but that isn't even close to the longest name proposed which I believe was 37 characters :-) to allow the application to do test/wait to see when processes are done. As Bill notes, there is not asynchonous notificiation in MPI. This is true but we are considering such things. Active type messages, interupt driven receives, etc. If we buy into this stuff I think it is worth considering if using interrupts here is justified and desirable. I'm not an OS person so I don't know if these interrupts are the same type of thing. This is just a thought. Steve From owner-mpi-core@CS.UTK.EDU Thu Apr 20 12:13:15 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA25806; Thu, 20 Apr 1995 12:13:15 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA24039; Thu, 20 Apr 1995 12:12:35 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 12:12:33 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id MAA24031; Thu, 20 Apr 1995 12:12:32 -0400 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA00329 for ; Thu, 20 Apr 1995 11:12:29 -0500 Message-Id: <199504201612.LAA00329@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: Another predefined attribute Date: Thu, 20 Apr 1995 11:12:27 -0500 From: William Gropp In developing an MPI-based library to instrument programs, it is desirable to have access to a synchronized clock. Since a true synchronized clock requires some hardware assistance, MPI correctly does not require MPI_Wtime to be synchronized. However, it is also impossible to determine from MPI whether or not MPI_Wtime IS synchronized. I propose a new predefined attribute MPI_WTIME_IS_GLOBAL which has value 1 if the WTIME clocks are synchronized and 0 otherwise. This attribute would apply to the communicator in which it was extracted, permitting heterogeneous systems to report true for subsets (for example, an communicator containing only processes in the same SMP hardware could provide a synchronized time, and would so indicate it). Bill From owner-mpi-core@CS.UTK.EDU Thu Apr 20 13:34:31 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA27415; Thu, 20 Apr 1995 13:34:31 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA00779; Thu, 20 Apr 1995 13:34:03 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 13:34:01 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from infomeister.osc.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id NAA00763; Thu, 20 Apr 1995 13:33:58 -0400 Received: from alex.osc.edu for raja@alex.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id NAA17641; Thu, 20 Apr 1995 13:33:54 -0400 Received: by alex.osc.edu (4.1/OSC-SUB-2.0) id AA07074; Thu, 20 Apr 95 13:30:21 EDT Date: Thu, 20 Apr 95 13:30:21 EDT From: raja@alex.osc.edu (Raja Daoud) Message-Id: <9504201730.AA07074@alex.osc.edu> To: mpi-core@CS.UTK.EDU Subject: Re: Another predefined attribute > MPI_WTIME_IS_GLOBAL > which has value 1 if the WTIME clocks are synchronized and 0 otherwise. > This attribute would apply to the communicator in which it was extracted, Would a communicator be allowed not to define it at all? If it is defined, its 0/1 value gives the relevant info, otherwise a 0 is assumed. --Raja From owner-mpi-core@CS.UTK.EDU Thu Apr 20 13:36:04 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA27466; Thu, 20 Apr 1995 13:36:03 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA01065; Thu, 20 Apr 1995 13:36:12 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 13:36:11 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id NAA01056; Thu, 20 Apr 1995 13:36:09 -0400 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA01801 for ; Thu, 20 Apr 1995 12:36:07 -0500 Message-Id: <199504201736.MAA01801@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: MPI_Attr_get Date: Thu, 20 Apr 1995 12:36:05 -0500 From: William Gropp Attribute changes Background: Marc Snir raised the issue of the C binding for MPI_ATTR_GET: int MPI_Attr_get(MPI_Comm comm, int keyval, void **attribute_val, int *flag) particularly the "void **". The comment was made that this is not ANSI C (however, I have been unable to confirm this and the declaration makes perfect sense, and in fact, making this "void *" makes it imposible to return the untyped pointer). In addition, the Fortran binding is inconsistant with the C binding: MPI_ATTR_GET(COMM, KEYVAL, ATTRIBUTE_VAL, FLAG, IERROR) INTEGER COMM, KEYVAL, ATTRIBUTE_VAL, IERROR LOGICAL FLAG (the 'C' form of this is MPI_ATTR_GET( int *comm, int *keyval, int *attribute_val, int *flag )). Thus, in the Fortran case, the attribute value is a Fortran integer; in the C case, it is a pointer. Note that sizeof(Fortran integer) need not equal sizeof(void *). The problem with this is that it makes it impossible to use attributes freely in a mixed C and Fortran environment, and complicates the development of a single implementation for both environments. (While MPI does not mandate multilingual programming, it should take the same stance as it does with threads - do nothing to prevent or discourage). There are several possible actions: 1) Make the void ** a void * (this is probably wrong) Pro: none unless ANSI C requires this Con: Unclear and hard to program with 2) Make the void ** an int * Pro: Fortran and C store objects of the same general type Con: sizeof(integer) need not be sizeof(int), so there are still problems in a multilingual setting; C can't store pointer attributes 3) Make the void ** an MPI_Fint *, where MPI_Fint is the C integer type (if any) that corresponds to a Fortran integer. Pro: Fortran and C store objects of the same type; Fortran and C can access all attributes Con: Cannot store pointers in attributes 4) Leave bindings the same; add an MPI_ATTR_KIND( COMM, KEYVAL, FLAG ) call that returns whether the attribute is an integer or pointer. Pro: No changes to existing programs; enables some multilingual programming Con: Adds a routine; possible problems as additional bindings are made. 5) Make no changes. Pro: No work. Con: Does not address problems with different attribute types. (I've left off the option of letting MPI_ATTR_PUT specify the format of the attribute.) A related issue is the meaning of the predefined attributes MPI_TAG_UB, MPI_IO, and MPI_HOST. The environment chapter specifies the values of these as if the attribute value is an integer. In the case of C, there are two interpretations to void *attribute_val; int flag; MPI_Attr_get( MPI_COMM_WORLD, MPI_TAG_UB, &attribute_val, &flag ); To check the value of the MPI_TAG_UB attribute, does the user use 1) *(int *)attribute_val or 2) (int)attribute_val Whichever choice will need clarification in the environment chapter. Bill From owner-mpi-core@CS.UTK.EDU Thu Apr 20 13:43:49 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA27564; Thu, 20 Apr 1995 13:43:48 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA01653; Thu, 20 Apr 1995 13:43:49 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 13:43:45 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id NAA01641; Thu, 20 Apr 1995 13:43:42 -0400 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA01976; Thu, 20 Apr 1995 12:43:39 -0500 Message-Id: <199504201743.MAA01976@antares.mcs.anl.gov> To: raja@alex.osc.edu (Raja Daoud) cc: mpi-core@CS.UTK.EDU Subject: Re: Another predefined attribute In-Reply-to: Message from raja@alex.osc.edu (Raja Daoud) of "Thu, 20 Apr 1995 13:30:21 -0400." <9504201730.AA07074@alex.osc.edu> Date: Thu, 20 Apr 1995 12:43:38 -0500 From: William Gropp | > MPI_WTIME_IS_GLOBAL | > which has value 1 if the WTIME clocks are synchronized and 0 otherwise. | > This attribute would apply to the communicator in which it was extracted, | | Would a communicator be allowed not to define it at all? If it is defined, | its 0/1 value gives the relevant info, otherwise a 0 is assumed. | Certainly. Bill From owner-mpi-core@CS.UTK.EDU Thu Apr 20 15:58:03 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA00225; Thu, 20 Apr 1995 15:58:03 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA11780; Thu, 20 Apr 1995 15:57:31 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 15:57:30 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from austin.asc.slb.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA11763; Thu, 20 Apr 1995 15:57:26 -0400 From: Received: from cedrick (CEDRICK.AUSTIN.ASC.SLB.COM) by austin.asc.slb.com (4.1/relay.950113a) id AA11147; Thu, 20 Apr 95 14:57:19 CDT Received: by cedrick (4.1/client.nfs.930922a) id AA14516; Thu, 20 Apr 95 14:57:15 CDT Date: Thu, 20 Apr 95 14:57:15 CDT Message-Id: <9504201957.AA14516@cedrick> To: lusk@mcs.anl.gov Subject: Re: MPI Forum meeting next week & POSC Cc: mpi-core@CS.UTK.EDU I'm sorry -- due to an upcoming product deadline, I will not be able to attend next week's meeting. (The curse of working in industry...) I have, however, been talking to people involved on the POSC (Petrotechnical Open Software Corporation) standard committee to make them aware of the MPI effort since have also started looking at standards for networks of workstations. They may also be sending a representative, but given that their next meeting in tomorrow in Houston, so they may not be able to make it this time either. I'll keep you posted on their activities. I may be giving a talk on MPI at the next POSC meeting (after tomorrow). They are holding their meetings about every 3-4 weeks!). Hopefully I'll be able to make it to the next MPI meeting, although it's cutting it veru close to my brother's wedding in Sweden June 10 ... Cheers, Anne -------------------------------------------------------------- Anne C. Elster, PhD (512) 331-3267 (Voice) Schlumberger Austin System Center (512) 331-3059 (Fax) P.O. Box 200015 Austin, TX 78720-0015 elster@slb.com -------------------------------------------------------------- From owner-mpi-core@CS.UTK.EDU Thu Apr 20 16:38:09 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA01359; Thu, 20 Apr 1995 16:38:08 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA15312; Thu, 20 Apr 1995 16:38:01 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 16:37:59 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from nipmuc.fsl.noaa.gov by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA15295; Thu, 20 Apr 1995 16:37:56 -0400 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA11564; Thu, 20 Apr 95 14:37:49 MDT Message-Id: <9504202037.AA11564@nipmuc.fsl.noaa.gov> To: mpi-core@CS.UTK.EDU Cc: hart@nipmuc.fsl.noaa.gov Subject: Re: Access to stdio In-Reply-To: Your message of "Thu, 20 Apr 95 09:36:10 -0000." <9504200836.AA24463@deneb.co.uk> Date: Thu, 20 Apr 95 14:37:49 BST From: Leslie Hart (NOAA/FSL) Peter and Jim, I think that the MPI-1 standard deals with this adequately (and I was one of the four who voted FOR parallel I/O). The predefined attribute MPI_IO returns either MPI_PROC_NULL if NO nodes can perform standard language I/O or a rank (or MPI_ANY_SOURCE) if some (or all) node(s) can perform standard language I/O. Presumably a correct MPI implementation will always return MPI_PROC_NULL if you can't access standard in, out, and error. I can ask if there is I/O and if there isn't I can exit and print an error message :-) See page 189-190 of the May 5 document for this. Regards, Leslie Hart PS I assume that if MPI_PROC_NULL is the value of this attribute for a given implementation, then any portable attempt at a parallel I/O system using that implementation is DOA. From owner-mpi-core@CS.UTK.EDU Thu Apr 20 19:27:28 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA04172; Thu, 20 Apr 1995 19:27:27 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA26978; Thu, 20 Apr 1995 19:26:57 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 20 Apr 1995 19:26:55 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from infomeister.osc.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id TAA26971; Thu, 20 Apr 1995 19:26:53 -0400 Received: from tbag.osc.edu for gdburns@tbag.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id TAA24078; Thu, 20 Apr 1995 19:26:50 -0400 Received: by tbag.osc.edu (4.1/OSC-SUB-2.0) id AA16245; Thu, 20 Apr 95 19:26:49 EDT Date: Thu, 20 Apr 95 19:26:49 EDT From: gdburns@tbag.osc.edu (Greg Burns) Message-Id: <9504202326.AA16245@tbag.osc.edu> To: mpi-core@CS.UTK.EDU Subject: spawning My general motivation in extending MPI is to find the few remaining bits of common practice that are inextricably linked with MPI, define them in MPI terms and possibly improve them slightly. Anything not currently common practice should not be considered, regardless of the projected usefulness. I'm interested in dynamic processes and they are certainly common practice so here is my view. I prefer the Skjellum/Doss proposal. I would offer the amendment described below. I think we should take an "outside-in" view of system resources. Some may see this as an impossible restriction but we already have exactly this case in MPI-1. A rather important resource allocation takes place when an application starts. After main(), all processes are using some resources. Implementations handle this in different ways. I suggest we continue in the same way. Specifying resources in a portable manner is problematic as many have pointed out. I see no reason why the Skjellum/Doss MPI_Comm_spawn() could say nothing about resources. The sys-dependent string becomes the executable pathname alone, which then seems to be fairly sys-independent. Inside this call, the implementation's resouce management strategy is invoked. An application could include an external, impl-dependent configuration file that gives resource requirements/preferences for the various executables that constitute the application. In general, I think the vast majority of applications can have resource parameters submitted from the outside. The application would only give the executable pathname and the number of new processes. If this turns out to be a wild mistake, this design does not shut the door on adding "inside-out" or runtime resource specification in the future. I'd lose the root parameter, though this is not a big issue. So the modified call would be: MPI_COMM_SPAWN(path, count, comm, intercomm) We could standardize a taste of resource specification, without losing any portability by adding to or modifying the above to include the following two options: a) Specify a rank and spawn on the same node as the process with that rank. b) Spawn on a node not currently running an MPI process - fails if no node is available. In the Gropp/Lusk proposal, I find the blocking client/server calls to be compelling, because client/server is the common practice of the real world. I hope to try to implement them soon. -- Greg From owner-mpi-core@CS.UTK.EDU Fri Apr 21 00:11:37 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA07204; Fri, 21 Apr 1995 00:11:36 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id AAA16095; Fri, 21 Apr 1995 00:10:51 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 00:10:47 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id AAA16086; Fri, 21 Apr 1995 00:10:43 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA13836; Thu, 20 Apr 1995 23:10:38 -0500 Message-Id: <199504210410.XAA13836@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: agenda for MPI Forum meeting next week Cc: lusk@antares.mcs.anl.gov Date: Thu, 20 Apr 1995 23:10:38 -0500 From: Rusty Lusk Dear Group, The meeting next week will be a little smaller than last time. Several of the heavy brains will be missing and we will miss them, but we will carry on. Here is a rough draft of what I think we should discuss. Please let me know of anything you would like to place explicitly on the agenda that I have left out. As you can see, there is still plenty of flexibility. Regards, Rusty ______________________________________________________________________ Outline of Agenda for MPI Forum meeting, Apr 24-26 Monday afternoon ------ --------- Discussion of agenda - all Corrections and clarifications discussed last time - Marc Snir Break Inventory of "small" new topics. Several topics arose in the mail since the last meeting. For each of these, we need to do at least one of: assess whether it is part of MPI-1 or MPI-2. assign it to an existing subcommittee. create a new subcommittee on clarifications and assign it to that one. assign responsibility for a concrete proposal to a person, and schedule readings and votes. decide to drop it. Some of these topics are: MPI Constants - Paul Pierce Multicast - David Walker, Tony Skjellum Size of MPI_COMM_NULL, and related questions Single IN/OUT buffer for collective Colors in MPI_Comm_split New predefined attributes Data representation - Rolf Hempel (Hans-Christian Hoppe) clock synchronized - Bill Gropp others? Failure behavior of buffered sends (block or fail) Use of void** in bindings - Bill Gropp Error codes from collections of non-blocking operations Portable error codes Clarification of usage of UB Directions in MPI_Cart_shift I/O - Peter Pacheco Shortcomings in existing datatypes - Ian Foster Others? (I believe these are going to continue to trickle in during the year.) Dinner Meetings of subcommittees and small groups to address above questions Tuesday morning ------- ------- Dynamic Process Creation - discussion of goals and scope, followed by discussion of functionality (We need general consensus before detailed agreement.) - Rusty Lusk separation of resource and process management? non-blocking as well as blocking calls? client/server included? wait on mixed types of operations? scalability? vendor concerns Break Dynamic Process Creation - continued discussion objects to be used: persistent requests, tasks, intercommunicators, overall design if agreement can be reached straw votes if necessary Lunch Interfaces and Generalized Requests - Steve Huss-Lederman Reports on progress and plans from other subcommittees collective one-sided communication bindings real-time? Break Proposals and discussion of "small" topics identified Monday. Those groups with proposals present them. Readings and votes scheduled. Dinner downtown Wednesday morning --------- ------- Creation of outline for Dynamic Process Management agreement on scope, sections, some concrete proposals Break Conclusion of business on as many "small" items as possible Planning for future meetings. Vienna has volunteered. From owner-mpi-core@CS.UTK.EDU Fri Apr 21 04:18:01 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA11112; Fri, 21 Apr 1995 04:18:00 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA02580; Fri, 21 Apr 1995 04:17:22 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 04:17:19 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from win233.nas.nasa.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id EAA02560; Fri, 21 Apr 1995 04:17:17 -0400 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id BAA06328; Fri, 21 Apr 1995 01:17:07 -0700 Date: Fri, 21 Apr 1995 01:17:07 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199504210817.BAA06328@win233.nas.nasa.gov> To: prp@ssd.intel.com (Paul Pierce) Subject: Re: process management comments and proposal Cc: wcs@win233.nas.nasa.gov, mpi-core@CS.UTK.EDU > From: prp@ssd.intel.com (Paul Pierce) > Subject: Re: process management comments and proposal Thanks for the comments. I was amazed to see that someone actually read through my note! > 1. There isn't much difference between the new TASK and existing process > concepts. > > 2. I'm uncomfortable adding to MPI explicit ability to manage non-MPI things. > > and so I propose to merge TASK into the existing process abstraction. We > already identify a process as a rank in a communicator. We could merge > MPI_Exec from Bill's proposal with a special case of MPI_Attach: > [...] > This avoids adding the complexity of a new object and eliminates the need > for the translation operation MPI_Task. I have strong sympathy for these arguments, but I'm still forming an opinion. Let me play devil's advocate for the moment. The current "process abstraction" is really more of a communication port abstraction. A single task may appear in several different communicators, with several different ranks. Within the current MPI, the fact that they are the same is completely hidden (I believe if you forget how you created your communicators, there is no way to tell who is who). The reason we need a TASK object is to get directly at the task itself. For instance, you can send a signal to a (comm, rank) task, and this magically affects all the other communicators associated with that task. One can make the argument that sending a signal to a TASK task also affects several communicators, but at least in this case there is explicit out-of-band action on a unsafe channel. Furthermore, the picture is this /-comm /--comm task---comm \--comm \-comm instead of interconnected communicators: /--------\ //--comm-comm comm | \/ | \ \ | /\ | \ \-comm-comm \-------/ which begs an underlying task concept. When you spawn a new task, what happens if it doesn't check in with MPI through MPI_Init? If your only connection to it is through a communicator, you have no control. You have no way to find out what has happened to it. You have no way to kill if it you want to try again. In general, having an explicit TASK object feels comfortable because 1. it's an abstraction of something that's really there, whether we try to hide it or not and 2. it seems to me that it will make the whole area of signalling, handling faults (unexpected dead processes), etc, cleaner. For instance suppose you want to be notified when a task dies. Do you do MPI_Notify_termination(comm1, rank1, request) MPI_Notify_termination(comm2, rank2, request) MPI_Notify_termination(comm3, rank3, request) for each (comm,rank) pair that refers to that task? I imagine these would all be equivalent, but they seem different. You may not even know that (comm1,rank1) is the same as (comm2,rank2). What if you do MPI_Send(..., rank, comm, ...) MPI_Kill(rank, comm) Normally we expect communicators to be safe and preserve order, etc. Of course in this case who knows what happens. I'd be more comfortable with something that is explicitly not linked to a communication channel. What about MPI_Notify_termination(comm1, rank1, request) MPI_Kill(rank2, comm2) where again, (comm1,rank1)=(comm2, rank2) Of course one affects the other, but that's against communicator intuition. We can only understand it with the concept of a task, so why not make it explicit? Another use of tasks might be to bootstrap a new communicator when a task dies. I'm venturing out into uncharted and perilous territory here, but let's see what happens. Suppose a task dies and you want to recover. Suppose your only communicator is MPI_COMM_WORLD. How about: /* A task has died. Try to recover. Hope other living * tasks notice and try to rendezvous */ int nalive, i; MPI_Task survivors[N], t; MPI_Comm newcomm; nalive = 0 for (i = 0; i < N; i++) { MPI_proc_task(MPI_COMM_WORLD, i, &t); if (MPI_Task_status(t) == MPI_RUNNING) { survivors[nalive] = t; nalive++; } } MPI_Bootstrap_comm(survivors, nalive+1, &t, TIMEOUT); /* +1 = me! */ Of course this has numerous problems, especially race conditions. Actually, it's pretty horrible to think about. But I don't think I could even *think* about it without a TASK concept. I believe it should be possible to hide the task abstraction from most users, say those who want a simple pvm style interface, using the type of functions that Bill and Rusty propose. For instance, MPI_Spawn(executable, arguments, n, array_of_nodes, comm) could do the exec-and-attach. Or you could leave out the nodes argument and have it do allocate-exec-attach. > MPI_Node_task and MPI_Task_node most > reasonably reduce to something like the original MPI_Task: > > MPI_Node(comm, rank, node) > IN comm > IN rank > OUT node There is a minor problem with this and my original proposal. I think a resource has to be owned by one and only one process, so MPI_Node returns MPI_NODE_NULL in all tasks but the one that requested the allocation of the NODE. There remains the question of who (if anyone) owns the resources for the original MPI_COMM_WORLD. Bill From owner-mpi-core@CS.UTK.EDU Fri Apr 21 04:39:05 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA13339; Fri, 21 Apr 1995 04:39:04 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id EAA03678; Fri, 21 Apr 1995 04:39:05 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 04:39:04 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from win233.nas.nasa.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id EAA03671; Fri, 21 Apr 1995 04:39:03 -0400 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id BAA06541; Fri, 21 Apr 1995 01:38:44 -0700 Date: Fri, 21 Apr 1995 01:38:44 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199504210838.BAA06541@win233.nas.nasa.gov> To: lederman@super.org (Steve Huss-Lederman) Subject: Re: comments on Bill Saphir's proposal Cc: wcs@win233.nas.nasa.gov, mpi-core@CS.UTK.EDU > Since array_of_nodes came from MPI_Iallocate and this really came from > a resource manager it may well be heterogeneous. It seems painful to > have to check each node to see its arch and then do lots of MPI_Exec > calls that get combined later. I may have missed something here. A definite problem that we'll probably have to face in any proposal. PVM solves it with the $(PVM_ROOT)/pvm3/bin/ARCH stuff. I think it's fair to say that MPI doesn't want to go that route. I'd be content to let the user handle this one by setting $PATH differently in heterogeneous environments with shared filesystems (or some equivalent on non-Unix systems). > 2) There is MPI_Notify_when_done (sorry Bill, but that isn't even > close to the longest name proposed which I believe was 37 characters :-) > to allow the application to do test/wait to see when processes are > done. As Bill notes, there is not asynchonous notificiation in MPI. > This is true but we are considering such things. Active type > messages, interupt driven receives, etc. If we buy into this stuff I > think it is worth considering if using interrupts here is justified and > desirable. I'm not an OS person so I don't know if these interrupts > are the same type of thing. This is just a thought. A good point. I wonder if MPI_Notify_asynchronously_when_task_done(task, handler_function) would be feasible? Ha! 40 characters! Bill From owner-mpi-core@CS.UTK.EDU Fri Apr 21 05:56:22 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA17713; Fri, 21 Apr 1995 05:56:22 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id FAA13692; Fri, 21 Apr 1995 05:55:44 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 05:55:43 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from win233.nas.nasa.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id FAA13681; Fri, 21 Apr 1995 05:55:41 -0400 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id CAA07399; Fri, 21 Apr 1995 02:55:37 -0700 Date: Fri, 21 Apr 1995 02:55:37 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199504210955.CAA07399@win233.nas.nasa.gov> To: gdburns@tbag.osc.edu (Greg Burns) Subject: Re: spawning Cc: wcs@win233.nas.nasa.gov, mpi-core@CS.UTK.EDU > Date: Thu, 20 Apr 95 19:26:49 EDT > From: gdburns@tbag.osc.edu (Greg Burns) > Subject: spawning > > I think we should take an "outside-in" view of system resources. Some > may see this as an impossible restriction but we already have exactly > this case in MPI-1. A rather important resource allocation takes place > when an application starts. After main(), all processes are using some > resources. Implementations handle this in different ways. I suggest > we continue in the same way. > > Specifying resources in a portable manner is problematic as many have > pointed out. I see no reason why the Skjellum/Doss MPI_Comm_spawn() > could say nothing about resources. The sys-dependent string becomes > the executable pathname alone, which then seems to be fairly > sys-independent. > > Inside this call, the implementation's resouce management strategy is > invoked. An application could include an external, impl-dependent > configuration file that gives resource requirements/preferences for the > various executables that constitute the application. In general, I > think the vast majority of applications can have resource parameters > submitted from the outside. The application would only give the > executable pathname and the number of new processes. If this turns out > to be a wild mistake, this design does not shut the door on adding > "inside-out" or runtime resource specification in the future. I had some specific scenarios I wanted to be able to handle: 1. I can use more resources any time they're available. I want to put out a standing request for more resources and use them when (if ever) the request is fulfilled. Example: a task farm can often handle arbitrarily many tasks. By using any resources that become available, I can make very efficient use of resources (the tiling problem for parallel jobs is a significant source of underutilization). 2. My need for resources is soft, not firm. I can make use of 4 nodes if 10 aren't available. The allocation request may return 4. I might want to spawn different tasks depending on what resources I actually get. [in the specific example I'm thinking of, the guy might actually want to *compile* (!) and run a different task depending on how many nodes he gets - F77 static memory allocation] 3. I want to reuse resources (as in the original task farm example). There should be some way to hold on to them. In the single-spawn scenario, if a task dies, how does the external resource manager know whether the application will want to reclaim the resource or let it go? 4. I want to be able to emulate PVM. In particular, I want to be able to get hosts sleepy, grumpy and dopey, and do whatever I want with them. 5. I'd like to be able to find out what resources I actually got. For instance, will they go away in 20 minutes. I suppose we could add resource inquiry functions on processes but that seems like avoiding the issue. If being object oriented is a good thing, then MPI should have clean abstractions for all objects, even if it means having more objects, IMHO. This may sound strange coming from a lets-cut-mpi crusader. 6. While I can't think of any more now (it's late) I'd like to reserve the option of coming up with more later... :-) I'm not sure how I could handle any of these with a simple spawn call that combined resource allocation, and task creation. I'm not sure whether I buy the statement that the vast majority of applications can have resource parameters submitted from the outside. Certainly this applies to static applications. And it applies to applications where a manager spawns N workers and nothing else happens. I would hope that dynamic process creation in MPI is not being designed to handle this second case, which can be handled statically. What about for true dynamic applications? In saying "if this turns out to be a wild mistake..." are you saying that we could always revert to seeing MPI_Spawn as a concatenation of allocate-exec? This is true. I am willing to believe at this point that the extra functionality in allocate-exec might turn out not be necessary (depite what I said above). I would prefer, though, to have a complete description so that everything is thought out. Then if it turns out we don't need the extra babbage, drop it. As for sticking with current practice, I think we're out on a limb no matter what we do. PVM is the only model I know of for dynamic process creation and there are several reasons why it isn't adequate. It does separate resource and task management, although not adequately, as I've argued. OK, I take that back. Unix process creation is another model and there is clearly no resource allocation component. On the other hand, there are rarely resource allocation issues. Except in unusual circumstances, you always get what you want, and you get it immediately. One saving grace is that the proposal will come out in November and there will be a year for people to use it before it becomes an actual part of the standard. Bill From owner-mpi-core@CS.UTK.EDU Fri Apr 21 12:32:58 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA24305; Fri, 21 Apr 1995 12:32:58 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA14574; Fri, 21 Apr 1995 12:32:40 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 12:32:39 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub.meiko.co.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA14546; Fri, 21 Apr 1995 12:32:27 -0400 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA15813 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Fri, 21 Apr 1995 17:32:18 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA25007; Fri, 21 Apr 1995 17:32:16 +0000 Date: Fri, 21 Apr 1995 17:32:16 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9504211632.AA25007@deneb.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: <199504210955.CAA07399@win233.nas.nasa.gov> (wcs@nas.nasa.gov) Subject: Re: spawning Although I think handling the allocation of "external" resources is a ***MUCH*** harder problem than simply assuming that a parallel job starts with all the resources it will ever need, I believe that we have to take on the harder problem. I believe this because 1) It's simply not reasonable to assume that you can allocate all of the resources a job may require to it a priori. 2) We're moving towards being able to solve some of the complex multi-disciplinary type applications (inter-communicators help a lot here), and these are bound to have time varying resource requirements. (e.g. pre-process the weather data to make a consistent input data set leaving the result distributed, then invoke the atmospheric and ocean models) [This is really the same argument as 1, but rhetoric requires three points, not two :-)] 3) Programs want to be able to soak up all of the "spare" resources either in a parallel machine, or on a network. The classic examples here are task farms, which (if carefully coded) can also handle the removal of their resources while they run. Therefore I strongly agree with general approach that separates resource allocation from process startup. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From owner-mpi-core@CS.UTK.EDU Fri Apr 21 14:40:05 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA26501; Fri, 21 Apr 1995 14:40:05 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA22796; Fri, 21 Apr 1995 14:39:47 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 14:39:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from infomeister.osc.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id OAA22782; Fri, 21 Apr 1995 14:39:42 -0400 Received: from tbag.osc.edu for gdburns@tbag.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id OAA07861; Fri, 21 Apr 1995 14:39:38 -0400 Received: by tbag.osc.edu (4.1/OSC-SUB-2.0) id AA17802; Fri, 21 Apr 95 14:39:36 EDT Date: Fri, 21 Apr 95 14:39:36 EDT From: gdburns@tbag.osc.edu (Greg Burns) Message-Id: <9504211839.AA17802@tbag.osc.edu> To: mpi-core@CS.UTK.EDU Subject: Re: spawning Let me offer another rationale and then respond to Bill's points. We believe that >>50% of parallel apps don't need runtime process creation. Gropp/Lusk admit this: "The existing MPI specification is adequate for most parallel applications." I contend that most of the remaining apps can fully characterize their initial and future resource requirements at launch-time. They don't allocate the resources at launch-time; they just describe them. Examples: "As processes may be spawning at run-time, allocate them to nodes from this list, in a round-robin manner." "My program viz1 is a graphics program that wants to run on an SGI. When the app says spawn viz1, that's where I want it run." "My program slave want to run by itself on a free node. When my app says spawn slave, allocate a node first. Hold any nodes whose processes terminate and reuse them first before allocating new ones." So what is left is the thin sliver of the problem space whose apps somehow determine their resource requirements from runtime data. However you _can_ key some database of resource requirements based on your own program name and the program name you specify to MPI_Comm_spawn(). You can at least chose a program name based on runtime data. So what is left is the thinner sliver of the problem space whose apps somehow determine their resource requirements from runtime data and cannot identify those requirments by simply stating (or overloading?) the program name. Example: "I converged after only 20 iterations, so next time get a node with a bigger memory and double the mesh size (hell, I don't know. :-)" If we need to cover this thinnest sliver, then we need the inside-out full blown resource management system that some have proposed. If we want to say our job is done when we have reached 90%, we can do something simple, leave most things to be decided by implementors, stand back, watch what happens and standardize emerging common practice much later, if necessary. -=- William Saphir writes: >I had some specific scenarios I wanted to be able to handle: Great, that's how I evaluate things. >1. I can use more resources any time they're available. I want >to put out a standing request for more resources and use them >when (if ever) the request is fulfilled. Example: a task farm >can often handle arbitrarily many tasks. By using any >resources that become available, I can make very efficient >use of resources (the tiling problem for parallel jobs is >a significant source of underutilization). This is also #1 in my books. I guess we need an non-blocking allocator that completes when a resource becomes available and that we can periodically test inside the application. Nah. I actually see the optimal task farm as using a client/server connect mechanism proposed by Gropp/Lusk. I place most of the burden on our friends in the resource management business (LSF, LL, Condor, etc). I imagine you tell the RM, in a batch script, for example, that you want to go up to a max of N nodes and that everytime one becomes allocatable, it is allocated, the RM starts the slave program(s) and the slave(s) connects to the master. MPI is not going to be the only thing hanging around the computing center will resource demands. I see the RM as the boss and applications that are as independent of it as possible. You might have to write a batch script in the RM language (JCL :-) to set up your MPI app. >2. My need for resources is soft, not firm. I can make use >of 4 nodes if 10 aren't available. The allocation request >may return 4. I might want to spawn different tasks >depending on what resources I actually get. [in the >specific example I'm thinking of, the guy might actually >want to *compile* (!) and run a different task depending >on how many nodes he gets - F77 static memory allocation] ouch, nice example The number of processes passed to the spawn function could be soft or firm. As I said in the prior message, a few useful resource hints are totally portable and could be brought within MPI. You got me on the recompilation bit - using runtime resource data. Er, I need to be doing another project right now. Catch you later. >3. I want to reuse resources (as in the original task farm >example). There should be some way to hold on to >them. In the single-spawn scenario, if a task dies, how >does the external resource manager know whether the >application will want to reclaim the resource or let it go? This is launch-time information, but also a portable concept. >4. I want to be able to emulate PVM. In particular, >I want to be able to get hosts sleepy, grumpy and dopey, and >do whatever I want with them. We don't want to code grumpy in a .c file. Put them in the launch-time resource information file, if the implementation allows it. >5. I'd like to be able to find out what resources I >actually got. For instance, will they go away in 20 minutes. >I suppose we could add resource inquiry functions >on processes but that seems like avoiding the issue. >If being object oriented is a good thing, then MPI >should have clean abstractions for all objects, even >if it means having more objects, IMHO. This may sound >strange coming from a lets-cut-mpi crusader. Processes can already get some resource information from MPI-1. I hope you are satisfied with this. >6. While I can't think of any more now (it's late) I'd >like to reserve the option of coming up with more later... :-) I reserve the option of coming up with better answers later... :-). -- Greg From owner-mpi-core@CS.UTK.EDU Fri Apr 21 14:56:26 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA26891; Fri, 21 Apr 1995 14:56:25 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA23908; Fri, 21 Apr 1995 14:56:25 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 14:56:21 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA23898; Fri, 21 Apr 1995 14:56:15 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2037; Fri, 21 Apr 95 14:56:13 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 4348; Fri, 21 Apr 1995 14:56:07 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 21 Apr 95 14:56:00 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA26297; Fri, 21 Apr 1995 14:55:55 -0400 Message-Id: <9504211855.AA26297@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: signle-sided accesses. (Better late than later) Reply-To: snir@watson.ibm.com Mime-Version: 1.0 Content-Type: application/postscript Content-Description: mpi2v1.ps Date: Fri, 21 Apr 1995 14:55:54 -0400 From: Marc Snir %!PS-Adobe-2.0 %%Creator: dvips 5.47 (RS/6000 1.0) Copyright 1986-91 Radical Eye Software %%Title: /u/snir/mpi/mpi2/mpi2v1.dvi %%Pages: 15 1 %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N /def load def /B{bind def}N /S /exch load def /X{S N}B /TR /translate load N /isls false N /vsize 10 N /@rigin{ isls{[0 1 -1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale Resolution VResolution vsize neg mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@letter{/vsize 10 N}B /@landscape{/isls true N /vsize -1 N}B /@a4{/vsize 10.6929133858 N}B /@a3{ /vsize 15.5531 N}B /@ledger{/vsize 16 N}B /@legal{/vsize 13 N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder} N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail} B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get} B /ch-xoff{128 ch-data dup length 3 sub get sub} B /ch-yoff{ ch-data dup length 2 sub get 127 sub} B /ch-dx{ch-data dup length 1 sub get} B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image} imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto}N /eop{clear SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put} for}N /p /show load N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval(Display)eq}{pop false}ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /a{ moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{ S p tail}B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w }B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{clear SS restore}B end %%EndProcSet TeXDict begin 1000 300 300 @start /Fa 2 104 df<001F003F00F800E001C001C001C001 C001C001C001C001C001C001C001C001C001C001C001C003800700FE00F800FE000700038001C0 01C001C001C001C001C001C001C001C001C001C001C001C001C001C000E000F8003F001F102D7D A117>102 DI E /Fb 10 116 df<00E001E007C007000F001E003C0038007800700070007000F000E000E000E000E000E000E0 00E000F000700070007000780038003C001E000F00070007C001E000E00B217A9C16>40 DI<387C7E7E 3E0E1E3C7CF860070B798416>44 D<1FE0007FF8007FFC00783C00301E00000E00003E0007FE00 3FFE007FCE00F80E00E00E00E00E00F01E00F83E007FFFE03FF7E00FC3E013127E9116>97 D<003F00007F00003F0000070000070000070000070003E7000FFF003FFF003C1F00780F00700F 00F00700E00700E00700E00700E00700F00F00F00F00781F007C3F003FFFE01FF7F007C7E01419 7F9816>100 D<07E00FF81FFC3C3E780E700FF007FFFFFFFFFFFFE000F000700778073E1F1FFE 0FFC03F010127D9116>I<7E0000FE00007E00000E00000E00000E00000E00000E7FE00E7FE00E 7FE00E0F000E1E000E3C000E78000EF0000FF8000FF8000FBC000F1E000E0E000E0F000E07807F 87F0FFCFF07F87F01419809816>107 D<7E7C00FFFF007FFF800F87800F03800F03800E03800E 03800E03800E03800E03800E03800E03800E03800E03807FC7F0FFE7F87FC7F01512809116> 110 D114 D<0FEC3FFC7FFCF03CE01CE01C F0007F801FF007FC003EE00EE00EF00EF81EFFFCFFF8C7E00F127D9116>I E /Fc 14 118 df<00F18003FDC0078F800E07801C07803C07803C0700780700780700780700F0 0E00F00E00F00E00F00E30F01C60F03C60707C6078FCC03FCFC00F078014147C9317>97 D<0000780003F80003F00000700000700000700000E00000E00000E00000E00001C00001C000F1 C003FDC0078F800E07801C07803C07803C0700780700780700780700F00E00F00E00F00E00F00E 30F01C60F03C60707C6078FCC03FCFC00F078015207C9F17>100 D<007C0001FF000783000F01 801E01803C01803C0300780E007FFC007FE000F00000F00000F00000F000007000007002007807 003C1E001FF80007E00011147C9315>I<003C6000FF7001E3E00381E00701E00F01E00F01C01E 01C01E01C01E01C03C03803C03803C03803C03803C07003C0F001C1F001E3F000FFE0003CE0000 0E00000E00001C00001C00301C00783800F0F0007FE0003F8000141D7E9315>103 D<006000F000F000E000000000000000000000000000000F001F80318031C063806380C3800700 070007000E000E000E001C301C601C6038C018C01F800F000C1F7D9E0E>105 D<03C01FC01F8003800380038007000700070007000E000E000E000E001C001C001C001C003800 3800380038007000700070007180E300E300E300E6007E003C000A207C9F0C>108 D<1E07C0F8003F1FE1FC0033B8730E0063E076070063C03C07006380380700C780780E00070070 0E000700700E000700700E000E00E01C000E00E01C000E00E01C000E00E038601C01C038C01C01 C038C01C01C071801C01C031803803803F001801801E0023147D9325>I<1E07C03F1FE033B870 63E07063C038638038C780700700700700700700700E00E00E00E00E00E00E01C31C01C61C01C6 1C038C1C018C3801F81800F018147D931A>I<007C0001FF000383800F01C01E01C01C01E03C01 E07801E07801E07801E0F003C0F003C0F003C0F00780F00700700F00701E003838001FF00007C0 0013147C9317>I<03C1E007E7F8067E3C0C7C1C0C781E0C701E18E01E00E01E00E01E00E01E01 C03C01C03C01C03C01C07803C07803C07003C0E003E3C0077F80071E000700000700000E00000E 00000E00000E00001C0000FFC000FFC000171D809317>I<1E0F003F3F8033F1C063C1C063C3C0 6383C0C783800700000700000700000E00000E00000E00000E00001C00001C00001C00001C0000 38000018000012147D9313>114 D<00FC03FE07070E0F0E0F0E0E1E000F800FF007F803FC003E 001E701EF01CF01CE03860703FE01F8010147D9313>I<018001C0038003800380038007000700 FFF0FFF00E000E000E000E001C001C001C001C0038003800380038307060706070C071803F001E 000C1C7C9B0F>I<0F00601F80703180E031C0E06380E06380E0C381C00701C00701C00701C00E 03800E03800E03800E038C0E07180E07180E0F180E1F3007F3F003E1E016147D9318>I E /Fd 16 122 df69 D73 D77 D<003F000001FFE00003 FFF00007C0F8000F807C001E001E003E001F003C000F00780007807800078078000780F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C078000780780007 807C000F803C000F003E001F001F003E000F807C0007C0F80003FFF00001FFE000003F00001A1F 7E9D1F>79 DI82 D84 D89 D<0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01C E01CE01CF07C7FFC7FDC3F1C0E127E9114>97 D<000E000E000E000E000E000E000E000E000E00 0E000E0F8E1FEE3FFE7C3E700E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E 0F1D7E9C15>100 D<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C 3FFC1FF807E00E127E9112>I<03C3C00FFFC01FFFC01C3800381C00381C00381C00381C00381C 001C38001FF8001FF0003BC0003800003800001FFC001FFF003FFF80700780E001C0E001C0E001 C0F003C07C0F803FFF001FFE0007F800121B7F9115>103 D112 D114 D<1C001C001C001C001C001C00FFE0FFE0FFE01C001C001C001C 001C001C001C001C001C001C001C001C201FF00FF007C00C187F970F>116 D121 D E /Fe 30 120 df<001F000000007FC0 000000F0C0000001F0E0000001E060000003E060000003E060000003E0E0000003E0C0000003E1 C0000003E380000003E700000003FE007FE001FC007FE001F8000E0001F8000E0001FC001C0003 FC003800077E0030000E7E0070001E3F00E0007C3F80C0007C1FC1C000FC0FE38000FC07F70000 FC03FE0000FC01FC0000FE00FC00607E00FF00E03F07DF81C00FFF07FF8003F800FE0023207E9F 29>38 D<387CFEFEFE7C38000000000000387CFEFEFE7C3807147C930F>58 D<0000E000000000E000000001F000000001F000000001F000000003F800000003F800000007FC 00000007FC0000000FFE0000000CFE0000000CFE000000187F000000187F000000307F80000030 3F800000703FC00000601FC00000601FC00000C01FE00000C00FE00001FFFFF00001FFFFF00001 8007F000030003F800030003F800060003FC00060001FC000E0001FE00FFE01FFFE0FFE01FFFE0 231F7E9E28>65 D<0007FC02003FFF0E00FE03DE03F000FE07E0003E0FC0001E1F80001E3F0000 0E3F00000E7F0000067E0000067E000006FE000000FE000000FE000000FE000000FE000000FE00 0000FE0000007E0000007E0000067F0000063F0000063F00000C1F80000C0FC0001807E0003803 F0007000FE01C0003FFF800007FC001F1F7D9E26>67 DIII 73 D 77 DI<001FF80000FFFF0001F81F8007E007E00FC003F01F8001F81F0000F83F00 00FC7F0000FE7E00007E7E00007EFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE 00007FFE00007FFE00007F7E00007E7F0000FE7F0000FE3F0000FC3F8001FC1F8001F80FC003F0 07E007E001F81F8000FFFF00001FF800201F7D9E27>II<001F F80000FFFF0001F81F8007E007E00FC003F01F8001F81F8001F83F0000FC7F0000FE7F0000FE7E 00007EFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007F 7E00007E7E00007E7F0000FE3F0000FC3F87C1FC1F8FF1F80FDC3BF007F83FE001FC3F8000FFFF 00001FFE0300001F0300001F8700000FFF00000FFF00000FFE000007FE000007FC000003FC0000 00F020287D9E27>II<03FC080FFF381E03F83C00F8780078780038F80038F80018 FC0018FC0000FF0000FFF8007FFF007FFFC03FFFE01FFFF00FFFF803FFF8001FFC0001FC0000FC 0000FCC0007CC0007CC0007CE00078E00078F800F0FE01E0E7FFC081FF00161F7D9E1D>I<7FFF FFFC7FFFFFFC7C0FE07C700FE01C600FE00C600FE00CE00FE00EC00FE006C00FE006C00FE006C0 0FE006000FE000000FE000000FE000000FE000000FE000000FE000000FE000000FE000000FE000 000FE000000FE000000FE000000FE000000FE000000FE000000FE000000FE00007FFFFC007FFFF C01F1E7E9D24>II89 D<07FC001FFF003F0F803F07C03F03E03F03E0 0C03E00003E001FFE00FFFE03F83E07E03E07C03E0F803E0F803E0F803E0FC07E07E0DE03FF9FE 07E07E17147F9319>97 D<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC 0000FC0000FC0000FC00007C00007E00003E00603F00C01F81C007FF0001FC0013147E9317>99 D<0007F80007F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F801F8 F80FFEF81F83F83E01F87E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00 F87C00F87E00F83E01F81F07F80FFEFF03F8FF18207E9F1D>I<01FE0007FF801F83E03F01F07E 00F07E00F8FC00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007E00003E00183F00380F 807007FFE000FF8015147F9318>I 104 D109 D<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC007EFC00 7E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>111 DI114 D<0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC007C007E0 07F006F81EFFFCC7F010147E9315>I<01800180018003800380038007800F803F80FFFCFFFC0F 800F800F800F800F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C 14>I119 D E /Ff 15 118 df<387CFEFEFE7C3800000000387CFEFEFE7C3807127D910D>58 D68 D73 D<0FF8001FFE003E1F803E 07803E07C01C07C00007C003FFC01FFFC03F87C07E07C0FC07C0FC07C0FC07C0FC0FC07E1FC03F FBF80FE1F815127F9117>97 D<03FC000FFE001F1F003E1F007C1F007C0E00FC0000FC0000FC00 00FC0000FC0000FC00007C00007E01803E03801F07000FFE0003F80011127E9115>99 D<01FC000FFF001F0F803E07C07C03C07C03E0FC03E0FFFFE0FFFFE0FC0000FC0000FC00007C00 007E00603E00C01F81C00FFF0001FC0013127F9116>101 D<1E003F003F007F003F003F001E00 00000000000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F 001F00FFE0FFE00B1E7F9D0E>105 D108 DII<01FC000FFF801F07C03E03E07C01F07C01F0FC01F8FC01F8FC01F8FC01F8FC01 F8FC01F87C01F07C01F03E03E01F07C00FFF8001FC0015127F9118>II<1FD83F F87038E018E018F000FF807FE07FF01FF807FC007CC01CC01CE01CF038FFF0CFC00E127E9113> 115 D<030003000300070007000F000F003F00FFFCFFFC1F001F001F001F001F001F001F001F00 1F001F0C1F0C1F0C1F0C0F9807F003E00E1A7F9913>II E /Fg 62 125 df<03C00007E0000FF0001E78001C38001C38001C38001C38001C 7BF01CF3F01EE3F00FE7800FC7000F87000F0F001F0E003F8E007B9E0073DC00F1DC00E1FC00E0 F800E07870F0F87071FE707FFFF03FCFE01F03C0141C7F9B18>38 D<007000F001F003C007800F 001E001C003C003800780070007000F000E000E000E000E000E000E000E000E000F00070007000 780038003C001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E 000F000780038003C001C001E000E000E000F00070007000700070007000700070007000F000E0 00E001E001C003C0038007800F001E003C007800F00060000C247C9F18>I<01C00001C00001C0 0001C000C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C1 8001C00001C00001C00001C00011147D9718>I<00600000F00000F00000F00000F00000F00000 F00000F0007FFFC0FFFFE0FFFFE07FFFC000F00000F00000F00000F00000F00000F00000F00000 600013147E9718>I<3C7E7F7F7F3F0F0E1E7CF870080C788518>I<78FCFCFCFC780606778518> 46 D<01800380078007800F803F80FF80FB806380038003800380038003800380038003800380 03800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>49 D<07F8001FFE003FFF007C0F 807003C0F001E0F000E0F000E0F000E00000E00000E00001E00001C00003C0000780000F00001E 00003C0000780000F00003E00007C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18 >I<78FCFCFCFC78000000000000000078FCFCFCFC780614779318>58 D<3C7E7E7E7E3C000000 0000000000387C7E7E7E3E0E1E3C78F060071A789318>I<000300000F80001F80003F0000FE00 01FC0003F00007E0001FC0003F80007E0000FC0000FC00007E00003F80001FC00007E00003F000 01FC0000FE00003F00001F80000F8000030011187D9918>I<7FFFC0FFFFE0FFFFE0FFFFE00000 00000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C7E9318>I<600000F80000FC00007E 00003F80001FC00007E00003F00001FC0000FE00003F00001F80001F80003F0000FE0001FC0003 F00007E0001FC0003F80007E0000FC0000F8000060000011187D9918>I<0FF8003FFE007FFF00 F80F80F00380F00380F00780001F00003E0000FC0001F00001E00003C00003C00003C00003C000 03C00003800000000000000000000000000000000003800007C00007C00007C000038000111C7D 9B18>I<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E00038E 00038E000306000707000707000707000707000FFF800FFF800FFF800E03800E03801C01C01C01 C07F07F0FF8FF87F07F0151C7F9B18>65 D<01FCE003FEE007FFE00F07E01E03E03C01E07800E0 7000E07000E0F00000E00000E00000E00000E00000E00000E00000E00000E00000F000007000E0 7000E07800E03C01E01E01C00F07C007FF8003FF0001FC00131C7E9B18>67 D<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C00701C00701C00 701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF 00FFFE007FF800141C7F9B18>II I<01F9C007FFC00FFFC01F0FC01C03C03C03C07801C07001C07001C0F00000E00000E00000E000 00E00000E00000E00FF0E01FF0E00FF0F001C07001C07003C07803C03C03C01C07C01F0FC00FFF C007FDC001F9C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C01C01C 01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C01C01C01C 01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18> I<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00700E00700E00700E00700E00707FFF F0FFFFF07FFFF0141C7F9B18>76 DI<7E07F0FF0FF87F07F01D81 C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39 C01C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18 >I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780700700780F007F FF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF00780F 00700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00001C00001E 00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF001C0F801C03801C03C01C 01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F001C07801C03801C03801C 03801C03801C03801C039C1C039C1C039C7F03FCFF81F87F00F0161C7F9B18>I<07F3801FFF80 3FFF807C1F80700780F00380E00380E00380E00000F000007800003F00001FF0000FFE0001FF00 001F800003C00001E00000E00000E06000E0E000E0E001E0F001C0FC07C0FFFF80FFFF00E7FC00 131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E070380070000070000070000070 000070000070000070000070000070000070000070000070000070000070000070000070000070 0000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E0007 8E00039C0003DC0001F80001F80000F00000F00000700000F00000F80001F80001DC00039E0003 8E00070F000707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00783E00 300F0000070000070001FF000FFF003FFF007F0700780700F00700E00700E00700F00F00783F00 7FFFF03FFBF00FE1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E 00000E3E000EFF800FFFC00FE3E00F80F00F00700F00780E00380E00380E00380E00380E00380F 00380F00780F00700F80F00FC3E00FFFC00EFF80067E00151C809B18>I<01FE0007FF001FFF80 3F07803C0300780000700000F00000E00000E00000E00000E00000F000007000007801C03C01C0 3F07C01FFF8007FF0001FC0012147D9318>I<001F80003F80001F800003800003800003800003 8000038003F3800FFB801FFF803E1F80780F80700780F00780E00380E00380E00380E00380E003 80E00780F00780700780780F803E3F801FFFF00FFBF803E3F0151C7E9B18>I<03F0000FFC001F FE003E1F00780780700380F003C0E001C0E001C0FFFFC0FFFFC0FFFFC0F000007000007801C03C 01C03F07C01FFF8007FF0001FC0012147D9318>I<001FC0007FE000FFE001F1E001C0C001C000 01C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<03F1F007FF F80FFFF81E1F303C0F003807003807003807003807003807003C0F001E1E001FFC003FF8003BF0 003800003C00001FFF001FFFC03FFFE07801F0F00078E00038E00038E00038F000787800F07E03 F03FFFE00FFF8003FE00151F7F9318>I<7E0000FE00007E00000E00000E00000E00000E00000E 00000E3F000EFF800FFFC00FE1E00F80E00F00E00F00E00E00E00E00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C000 07C0000380000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FFFF00 111D7C9C18>I<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E0007FFFC0FFFFE07FFFC0131C7E9B18>108 D<7DF1F000FFFBF8007FFFFC001F1F1C001E 1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F001914819318>I<7E3F00FEFF807FFF C00FE1E00F80E00F00E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C070 01C0E000E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001 F00013147E9318>I<7E3E00FEFF807FFFC00FE3E00F80F00F00700F00780E00380E00380E0038 0E00380E00380F00380F00780F00700F80F00FC3E00FFFC00EFF800E7E000E00000E00000E0000 0E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<03F3800FFB801FFF803E1F 80780F80700780F00780E00380E00380E00380E00380E00380E00380F00780700780780F803E1F 801FFF800FFB8003F380000380000380000380000380000380000380000380003FF8003FF8003F F8151E7E9318>I<7F87E0FF9FF87FBFF803FC7803F03003E00003C00003C00003C00003800003 80000380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318>I<0FF700 3FFF007FFF00F81F00E00700E00700F007007C00007FF0001FFC0007FE00001F00600780E00380 F00380F00780FC0F00FFFF00FFFE00E7F80011147D9318>I<0180000380000380000380000380 007FFFC0FFFFC0FFFFC00380000380000380000380000380000380000380000380000380000380 400380E00380E00381E003C3C001FFC000FF80007E0013197F9818>I<7E07E0FE0FE07E07E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F 07E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380 070700070700070700038E00038E00038E00038E0001DC0001DC0001DC0000F80000F800007000 15147F9318>II<7F8FF07F 9FF07F8FF0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E0003 8E000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E0380 0E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000DC00007800 00780000780000700000700000700000F00000E00079E0007BC0007F80003F00001E0000151E7F 9318>I<60F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0 600424769F18>124 D E /Fh 57 123 df<003F0F0000FFBF8003C3F3C00703E3C00703C1800E 01C0000E01C0000E01C0000E01C0000E01C0000E01C000FFFFFC00FFFFFC000E01C0000E01C000 0E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0007F87FC007F87FC001A1D809C18>11 D<003F0000FF8003C1C00703C00703 C00E01800E00000E00000E00000E00000E0000FFFFC0FFFFC00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F87F87F8151D809C17 >I<003FC000FFC003C3C00703C00701C00E01C00E01C00E01C00E01C00E01C00E01C0FFFFC0FF FFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C07FCFF87FCFF8151D809C17>I<7070F8F8FCFCFCFC7C7C0C0C0C0C0C0C1818181830 30606040400E0D7F9C15>34 D<00F0000001F8000003980000030C0000070C0000070C0000070C 0000070C0000070C000007180000073800000730000007607FE003C07FE003801F0003800C0007 C00C000DC0180019E0180038E0300070F06000F0706000F038C000F03CC000F01F8000F00F0060 78070060780F80C03C39E1C01FF0FF8007C03E001B1F7E9D20>38 D<00C00180030006000E000C 001C0018003800300030007000700060006000E000E000E000E000E000E000E000E000E000E000 E000E000600060007000700030003000380018001C000C000E0006000300018000C00A2A7D9E10 >40 DI<70F0F8F8781818183030706040050D7D840C>44 DI<70F8F8F87005057D840C>I<00030003000700060006000E000C 001C0018001800380030003000700060006000E000C000C001C001800380030003000700060006 000E000C000C001C001800180038003000700060006000E000C000C00010297E9E15>I<030007 003F00FF00C7000700070007000700070007000700070007000700070007000700070007000700 07000700070007000700FFF8FFF80D1C7C9B15>49 D<07C01FF03878603C601EF01EF80FF80FF8 0F700F000F000E001E001C003C0078007000E001C0038007000E030C03180330067FFEFFFEFFFE 101C7E9B15>I<70F8F8F870000000000000000070F8F8F87005127D910C>58 D<70F8F8F870000000000000000070F0F8F8781818183030706040051A7D910C>I<0FE03FF870 3C601CF01EF01EF01E001E003C007800E001C00180038003000300030003000300020000000000 0000000007000F800F800F8007000F1D7E9C14>63 D<00060000000F0000000F0000000F000000 1F8000001F8000001F8000001F80000033C0000033C0000033C0000061E0000061E0000061E000 00C0F00000C0F00000C0F000018078000180780001FFF80003FFFC0003003C0003003C0006001E 0006001E0006001E001F001F00FFC0FFF0FFC0FFF01C1D7F9C1F>65 D<001F808000FFE18003F0 338007801B800F000F801E0007801C0003803C000380780003807800018070000180F0000180F0 000000F0000000F0000000F0000000F0000000F0000000F0000000700001807800018078000180 3C0001801C0003001E0003000F00060007800C0003F0380000FFF000001F8000191E7E9C1E>67 DII 73 D77 DI<003F800000FFE00003E0F80007803C000E000E001E000F003C00078038000380780003C0 780003C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001 E0780003C0780003C0780003C03C0007803C0007801E000F000F001E0007803C0003E0F80000FF E000003F80001B1E7E9C20>II82 D<07E0801FF9803C1F80700780700380E00380E00180E00180E00180F00000F000007C00007FC0 003FF8001FFE0007FF0000FF80000F800003C00003C00001C0C001C0C001C0C001C0E00180E003 80F00300FC0E00CFFC0083F800121E7E9C17>I<7FFFFFC07FFFFFC0780F03C0700F01C0600F00 C0E00F00E0C00F0060C00F0060C00F0060C00F0060000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F000003FFFC0003FFFC001B1C7F9B1E>II87 D<08081818303060606060C0C0C0C0 C0C0F8F8FCFCFCFC7C7C38380E0D7B9C15>92 D<0FE0001FF8003C3C003C1E00180E00000E0000 1E0007FE001FFE003E0E00780E00F00E00F00E60F00E60F01E60783E603FFFC01F878013127F91 15>97 DI<03F00FF81E3C383C78187000F000F000F000F0 00F000F000780078063C061E0C0FF803E00F127F9112>I<001F80001F80000380000380000380 00038000038000038000038000038000038003E3800FFB801E0F80380780780380700380F00380 F00380F00380F00380F00380F003807003807803803807801E1F800FFBF007E3F0141D7F9C17> I<03E00FF01C38381C781E700EFFFEFFFEF000F000F000F000700078063C061E0C0FF803E00F12 7F9112>I<007801FC039E071E0E0C0E000E000E000E000E000E00FFE0FFE00E000E000E000E00 0E000E000E000E000E000E000E000E000E000E007FE07FE00F1D809C0D>I<00038007E7C00FFD C03C3DC0381C00781E00781E00781E00781E00381C003C3C003FF00037E0007000007000003000 003FFC001FFF003FFF80700780E001C0E001C0E001C0E001C07003803C0F001FFE0007F800121C 7F9215>II<18003C007C003C0018000000000000000000 00000000FC00FC001C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80FF 80091D7F9C0C>I<01C003E003E003E001C00000000000000000000000000FE00FE000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F1C0F1C0 7F803E000B25839C0D>IIIII<03F0000FFC001E1E00380700780780700380F003C0F003 C0F003C0F003C0F003C0F003C07003807807803807001E1E000FFC0003F00012127F9115>II<03E1800FF9801E1F803C0780780780780380F00380F00380F00380F00380F00380 F003807803807807803C07801E1F800FFB8007E380000380000380000380000380000380000380 001FF0001FF0141A7F9116>II<1F903FF07070E030E030E030F8007F803FE00FF000F8 C038C038E038E038F070DFE08FC00D127F9110>I<0C000C000C000C000C001C001C003C00FFE0 FFE01C001C001C001C001C001C001C001C001C301C301C301C301C301E600FC007800C1A7F9910 >IIII<7F8FF07F8FF00F0780070600038E0001DC0001D80000F00000700000780000 F80001DC00038E00030E000607000F0380FF8FF8FF8FF81512809116>II<7FFC 7FFC7838707060F060E061C063C00380070C0F0C0E0C1C1C3C1838187078FFF8FFF80E127F9112 >I E /Fi 48 122 df<007C000000FE000001FF000003C78000038780000383C0000783C00007 03C0000703C0000783C00007878000078F8000078F0000079E000003BC008003F800E003F001E0 03E001C007E001C00FF003C01EF003803CF8078078780700787C0F00F03E0E00F01F1E00F01F3C 00F00FF800F807F0007803E0007C0FF8303FFFFFF01FFE3FF007F00FC01C227EA022>38 D<007000E001C00380078007000E001E001E003C003C003C0078007800780078007000F000F000 F000F000F000F000F000F000F000F000F000F000700078007800780078003C003C003C001E001E 000E0007000780038001C000E000700C2E7EA112>40 DI<787878781830306060E0050A7D830D>44 DII<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17>49 D<03F0000FFC001FFE003C1F00780F00300780200780000780000780000780000F00000F00003E 0003FC0003F80003FC00001E00000F000007800003800003C00003C00003C00003C08003C0C003 C0C00780700F807C1F003FFE000FFC0003F00012207E9E17>51 D<001F0000001F0000003F8000 003B8000003B8000007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F0 0001C0F00003C0780003C078000380780007803C0007803C0007003C000FFFFE000FFFFE000FFF FE001E000F001E000F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B 207F9F1E>65 DI<001FC000FFF801 FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000780000F00000F00000F0 0000F00000F00000F00000F00000F000007800007800007800007C00003C00003E00001E00000F 000207800E03E03E01FFFC00FFF0001FC017227DA01D>I69 DI<001FE0 00FFF801FFFE03E03E07800E0F00001E00003E00003C00007C0000780000780000780000F00000 F00000F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E 1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>I73 D76 DII<003F000000FFC00003FFF00007E1F8000F807C 001F003E001E001E003C000F003C000F00780007807800078078000780F00003C0F00003C0F000 03C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C0780007807800078078 0007803C000F003C000F001E001E001F003E000F807C0007E1F80003FFF00000FFC000003F0000 1A227DA021>II<003F000000FFC0 0003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F0078000780780007807800 0780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F0 0003C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE000F83FC0007E1F800 03FFF00000FFF800003F780000007C0000003E0000001E0000001F0000000F801A277DA021>I< FFF800FFFF00FFFF80F007C0F003E0F001E0F000F0F000F0F000F0F000F0F000F0F001E0F003E0 F007C0FFFF80FFFF00FFF800F03C00F01E00F01E00F00F00F00F00F00780F00780F003C0F001C0 F001E0F000F0F000F0F00078F00078F0003C16207B9F1D>I<01FC0007FF800FFFC01F03C03C00 C03C00007800007800007800007800007800007C00003C00003F00001FE0000FFC0007FE0001FF 00003F800007C00003C00003E00001E00001E00001E00001E00001E00001C0C003C0F007C0FC0F 807FFF001FFE0003F80013227EA019>III87 D89 D<07E03FF87FFC701E401F000F000F000F003F07 FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000 F000780078003C013E0F1FFF0FFE03F010147E9314>I<00078000078000078000078000078000 078000078000078000078000078000078000078007C7800FF7801FFF803E1F807C0780780780F8 0780F00780F00780F00780F00780F00780F00780F00780780780780F803E1F801FFF800FF78007 C78011207E9F17>I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80 F00000F00000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<03E0 F00FFFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003FF8 0033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F801F07E07 E03FFFC00FFF0003FC00141E7F9317>103 D105 D107 DIII<01F80007FE001FFF803F0FC03C 03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01F FF8007FE0001F80014147F9317>II<03C7800FF7801FFF 803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F807807807807C0F 803E1F801FFF800FF78007C7800007800007800007800007800007800007800007800007800007 80111D7E9317>II<07F01FFC3FFC780C7800780078007C003FC01FF00FF803 F8007C003C003CC03CF07CFFF87FF00FC00E147F9311>I<1E001E001E001E001E001E00FFF0FF F0FFF01E001E001E001E001E001E001E001E001E001E001E001E001E001E201FF00FF007C00C1A 7F9910>IIII121 D E /Fj 42 122 df<000078007C7800FC7801FC7803C4000380000780000780000780000780000780 00078000078000078000FFFC78FFFC78FFFC780780780780780780780780780780780780780780 780780780780780780780780780780780780780780780780780780780780780780780780781524 80A31A>12 D45 DI<0001800001800003 80000300000300000700000600000600000E00000C00000C00001C000018000018000018000038 0000300000300000700000600000600000E00000C00000C00001C0000180000180000380000300 000300000700000600000600000E00000C00000C00000C00001C00001800001800003800003000 00300000700000600000600000E00000C00000C0000011317DA418>I<00C001C007C0FFC0FFC0 FBC003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C0FFFFFFFFFFFF10227CA118>49 D<03F0000FFC001FFE003C1F00 3807807007C07003C0F003E0E001E06001E02001E00001E00001E00001E00003C00003C0000780 000780000F00001E00003C0000780000F00001E00001C0000380000700000E00001C0000380000 700000FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC001FFF003E0F003807807003C02003 C02003C00003C00003C00003C0000780000780000F00001E0003FC0003F80003FE00000F000007 800003C00003C00001E00001E00001E00001E00001E08001E0C003C0E003C07007803C0F801FFF 000FFC0003F00013237EA118>I<001F00001F00002F00002F00006F0000EF0000CF0001CF0001 CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F00780F00F00F00FF FFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F00000F00000F0015217FA0 18>I<3FFF803FFF803FFF803C00003C00003C00003C00003C00003C00003C00003C00003C0000 3CF8003FFE003FFF003F0F803E07803C03C03803C00001E00001E00001E00001E00001E00001E0 0001E04003C04003C0E003C07007807C1F003FFE000FFC0003F00013227EA018>I<007E0001FF 0003FF0007C1000F00001E00001E00003C00003C000078000078000078F800F3FE00F7FF00FF0F 80FC0780F803C0F803C0F801C0F001E0F001E0F001E0F001E0F001E07001E07001E07801E07803 C03803C03C03801C07801F0F000FFE0007FC0001F00013237EA118>I66 D<000FF000007FFC0000FFFF0001F0 1F0003C00700078002000F0000001E0000003E0000003C0000003C000000780000007800000078 000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000 7800000078000000780000003C0000003C0000003E0000001E0000000F0000800780018003C003 8001F00F8000FFFF00007FFC00000FF00019257DA31F>III<000FF000003FFE0000FFFF8001F81F8003E00780078001000F0000001E 0000001E0000003C0000003C000000780000007800000078000000F0000000F0000000F0000000 F0000000F0000000F0000000F000FFC0F000FFC0F000FFC0780003C0780003C0780003C03C0003 C03C0003C01E0003C01E0003C00F0003C0078003C003E003C001F80FC000FFFFC0003FFF00000F F8001A257DA321>71 D78 D 80 D82 D<00FF0003FFC007FFF00F83F01E00F03C00203C00007800007800007800007800007800007C00 003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F00000F00000F80000780000 78000078000078000078400078E000F0F000F0F801E07F07C03FFF800FFF0001FC0015257EA31B >I<07E01FF83FFC3C1E301E200F000F000F000F01FF0FFF3FFF7F0F780FF00FF00FF00FF80F7C 3F7FFF3FFF1F8F10167E9517>97 DI<01FC0007FF000FFF801F07803C0180780000780000700000F00000F000 00F00000F00000F00000F000007800007800007800403C00C01F07C00FFFC007FF8001FC001216 7E9516>I<0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C000 03C00003C003F3C00FFFC01FFFC03F0FC03C07C07803C07803C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C07803C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I< 03F00007FC001FFE003E0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000 F000007000007800007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01 FF03C1038007800780078007800780078007800780FFF8FFF8FFF8078007800780078007800780 0780078007800780078007800780078007800780078007800780102380A20F>I<01F0F807FFF8 0FFFF81F1F801E0F003C07803C07803C07803C07803C07801E0F001F1F001FFE001FFC0039F000 3800003800003C00003FFE001FFFC01FFFE03FFFF07801F0F800F8F00078F00078F00078F00078 7800F03E03E01FFFC00FFF8001FC0015217F9518>III<00F000F000F000F000000000000000 000000000000000000000000F000F000F000F000F000F000F000F000F000F000F000F000F000F0 00F000F000F000F000F000F000F000F000F000F000F000F000F080F0E3E0FFE07FC01F000C2D83 A20D>IIIII<01FC0007FF000FFF 801F07C03C01E07800F07800F0700070F00078F00078F00078F00078F00078F000787800F07800 F07C01F03E03E01F07C00FFF8007FF0001FC0015167F9518>II<03F3C007FBC00FFFC01F0FC03C07C07C03C07803C0F803C0F003C0 F003C0F003C0F003C0F003C0F003C0F803C07803C07803C03C07C03F0FC01FFFC00FFBC003E3C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C012207E9519>II<07F01FFE3FFF3C0F7803780078007C003F003FF01FF80FFC01FE001F00 0F000F400FE00FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FF F80F000F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E0 0E1C7F9B12>II<7801F07C01E03E03C01E07C00F0780078F0007DE 0003FC0001FC0000F80000700000F80001FC0003DC00039E00078F000F07801E07801E03C03C01 E07800F0F800F81516809516>120 DI E /Fk 69 124 df<000FC3E0007FEFF000E07C7801C0F8780380F830070070000700700007 00700007007000070070000700700007007000FFFFFF80FFFFFF80070070000700700007007000 070070000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070007FE3FF007FE3FF001D20809F1B>11 D<001F80007FC001E0E003 81E00381E00701E0070000070000070000070000070000070000FFFFE0FFFFE00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E07FC3FE7FC3FE1720809F19>I<70F8F8F8F8F8F8F8F8707070707070707070707070707000 0000000070F8F8F87005217CA00D>33 D<7038F87CFC7EFC7E7C3E0C060C060C06180C180C381C 3018603040200F0E7E9F17>I<0078000000FC0000018600000386000003030000070300000703 000007030000070300000706000007060000070C00000718000003B81FFC03F01FFC03E003E003 C0018003C0030007C003000EE006001CE0060038700C0078780C0070381800F01C3000F01E3000 F00E6000F007C000F003C0187801C0187807E0303E1E78701FF83FE007E00F801E227EA023>38 D<006000C001800300070006000C001C00180038003800300070007000700060006000E000E000 E000E000E000E000E000E000E000E000E000E0006000600070007000700030003800380018001C 000C00060007000300018000C000600B2E7DA112>40 DI<70F8FCFC7C0C0C0C181838306040060E7C840D>44 DI< 70F8F8F87005057C840D>I<00030003000700060006000E000C000C001C001800180038003000 3000700060006000E000C000C001C00180018001800380030003000700060006000E000C000C00 1C0018001800380030003000700060006000E000C000C000102D7DA117>I<03F0000FFC001E1E 001C0E00380700780780700380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003 C0F003C0F003C0F003C0F003C0F003C0F003C07003807003807003807807803807001C0E001E1E 000FFC0003F000121F7E9D17>I<018003801F80FF80E380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038003800380FFFEFFFE0F1E7C9D 17>I<03F0000FFC00183E00300F00600F80C00780F007C0F807C0F803C0F803C02003C00007C0 000780000780000F00000E00001C0000380000700000E00001C0000380000300000600C00C00C0 1800C03001807FFF80FFFF80FFFF80121E7E9D17>I<03F0000FFC001C1E00300F00780F80780F 80780780380F80000F80000F00000F00001E00003C0003F80003F000001C00000E00000F000007 800007800007C02007C0F807C0F807C0F807C0F00780600F80700F003C1E001FFC0003F000121F 7E9D17>I<000E00000E00001E00003E00003E00006E0000EE0000CE00018E00038E00030E0006 0E000E0E000C0E00180E00380E00300E00600E00E00E00FFFFF0FFFFF0000E00000E00000E0000 0E00000E00000E00000E0000FFE000FFE0141E7F9D17>I<3803003FFF003FFE003FF8003FE000 30000030000030000030000030000030000031F00037FC003E0E003C0700380780300380000380 0003C00003C00003C06003C0F003C0F003C0F00380C00780600700700E003C1E000FF80007E000 121F7E9D17>I<007C0001FE000783000E03800C07801C07803803003800007800007000007000 00F3F800F7FE00FC0E00F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003 C07003803803803807001C07000E1E0007FC0001F000121F7E9D17>I<6000007FFFC07FFFC07F FF80600180C00300C00600C00600000C0000180000180000300000600000600000E00000C00001 C00001C00001C00003800003800003800003800007800007800007800007800007800007800007 8000030000121F7D9D17>I<03F0000FFC001E1E00380700300700700380700380700380700380 7807003E07003F0E001FDC000FF80003F80007FC001E7E00383F00700F80700780E003C0E001C0 E001C0E001C0E001C0F001807003803807001E0E000FFC0003F000121F7E9D17>I<03F00007F8 001E1C00380E00380700700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007 C07007C0380FC01C0FC01FFBC007F3C0000380000380000780000700300700780E00780E00701C 003078001FF0000FC000121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F8700514 7C930D>I<70F8F8F8700000000000000000000070F8F8F878181818183030606040051D7C930D> I<0FC03FF070786038E03CF03CF03C603C003C007800F000E001C0018003800300030003000300 0300030003000000000000000000000007000F800F800F8007000E207D9F15>63 D<0003800000038000000380000007C0000007C0000007C000000DE000000DE000000DE0000018 F0000018F0000018F00000307800003078000030780000603C0000603C0000603C0000E01E0000 C01E0000FFFE0001FFFF0001800F0001800F0003800F800300078003000780070007C0070003C0 0F8003C0FFE03FFEFFE03FFE1F207F9F22>65 DI<001FC04000 7FF0C001F839C003C00DC0078007C00F0003C01E0003C03E0001C03C0001C07C0001C07C0000C0 780000C0F80000C0F8000000F8000000F8000000F8000000F8000000F8000000F8000000F80000 00780000C07C0000C07C0000C03C0000C03E0001801E0001800F0003000780030003C00E0001F8 1C00007FF000001FC0001A217D9F21>IIII73 D76 DII<001F800000FFF00001E0780007C03E000F80 1F000F000F001E0007803C0003C03C0003C07C0003E07C0003E0780001E0F80001F0F80001F0F8 0001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C0003E0 3C0003C03E0007C01E0007800F000F000F801F0007C03E0001F0F80000FFF000001F80001C217D 9F23>II82 D<07E0800FF9801C1F80300F80700780600380E00380E00180E00180E00180F00000F000007800 007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C0C001C0C001C0C001 C0C001C0E00180F00380F80700FE0E00CFFC0081F80012217D9F19>I<7FFFFFE07FFFFFE0780F 01E0700F00E0600F0060600F0060E00F0070C00F0030C00F0030C00F0030C00F0030000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F000007FFFE0007FFFE001C1F7E 9E21>I87 D<7FF83FF87FF83FF807C00F8003C0060001E00E0001F00C0000F018000078 3800007C3000003C7000003E6000001EC000000FC000000F8000000780000007C0000007E00000 0DE000001DF0000018F8000038780000307C0000603C0000E01E0000C01F0001800F0003800780 030007C00F8007C0FFE01FFEFFE01FFE1F1F7F9E22>I<0804180C3018703860306030C060C060 C060F87CFC7EFC7E7C3E381C0F0E7B9F17>92 D<1FE0003FF000783800781C00300E00000E0000 0E00000E0003FE001FFE003E0E00780E00F80E00F00E30F00E30F00E30F01E30783F703FEFE00F 878014147E9317>97 D<0E0000FE0000FE00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E3F000EFF800FC3C00F00E00E00F00E00700E00780E00780E00780E00780E00 780E00780E00780E00700E00F00F00F00F01E00FC3C00CFF800C3E0015207F9F19>I<03F80FFC 1E1E3C1E380C78007000F000F000F000F000F000F0007000780038033C031F0E0FFC03F010147E 9314>I<000380003F80003F800003800003800003800003800003800003800003800003800003 8003E3800FFB801E0F803C0780780380780380700380F00380F00380F00380F00380F00380F003 807003807003807807803807801E1F800FFBF803E3F815207E9F19>I<03F0000FFC001E1E003C 0F00380700780700700380F00380FFFF80FFFF80F00000F00000F000007000007800003801801C 03800F070007FE0001F80011147F9314>I<003C00FE01CF038F03060700070007000700070007 000700FFF0FFF00700070007000700070007000700070007000700070007000700070007000700 7FF07FF01020809F0E>I<0001E003E3F00FFF701C1C703C1E00380E00780F00780F00780F0078 0F00380E003C1E001C1C003FF80033E0003000003000003800003FFE001FFF803FFFC07003E070 00F0E00070E00070E00070F000F07801E03E07C00FFF0003FC00141F7F9417>I<0E0000FE0000 FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000EFF000FC380 0F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C0FFE7FCFFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000000000 00000E007E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FF C0FFC00A1F809E0C>I<00E001F001F001F000E0000000000000000000000000007007F007F000 F00070007000700070007000700070007000700070007000700070007000700070007000700070 00706070F0E0F1C07F803E000C28829E0E>I<0E0000FE0000FE00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E1FF00E1FF00E0F800E0E000E0C000E18000E30000E70 000EF0000FF8000F38000E1C000E1E000E0E000E07000E07800E03800E03C0FFCFF8FFCFF81520 7F9F18>I<0E00FE00FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E00FFE0FFE00B20809F0C>I<0E3F 03F000FEFF8FF800FFC1DC1C000F80F80E000F00F00E000E00E00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E00FFE7FE7FE0FFE7FE7FE023147F9326>I<0E3E00FEFF00FFC3800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C0FFE7FCFFE7FC16147F9319>I<01F80007FE001E07803C03C03801C07000E07000E0F000 F0F000F0F000F0F000F0F000F0F000F07000E07801E03801C03C03C01E078007FE0001F8001414 7F9317>I<0E3F00FEFF80FFC3C00F01E00E00F00E00F00E00780E00780E00780E00780E00780E 00780E00780E00700E00F00F00F00F01E00FC3C00EFF800E3E000E00000E00000E00000E00000E 00000E00000E0000FFE000FFE000151D7F9319>I<03E1800FF9801E1F803C0780780780780380 700380F00380F00380F00380F00380F00380F003807003807803807807803C0F801E1F800FF380 03E380000380000380000380000380000380000380000380003FF8003FF8151D7E9318>I<0E78 FEFCFF9E0F1E0F0C0F000E000E000E000E000E000E000E000E000E000E000E000E00FFE0FFE00F 147F9312>I<1F903FF07070E030E030E030F00078007F803FE00FF000F0C078C038C038E038E0 38F070DFE08F800D147E9312>I<06000600060006000E000E001E003E00FFF8FFF80E000E000E 000E000E000E000E000E000E000E000E180E180E180E180E18073007E003C00D1C7F9B12>I<0E 01C0FE1FC0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E03C00E03C0070FC007FDFC01F1FC16147F9319>III<7FC7FC7FC7FC0703E00703800383 0001C70000EE0000EC00007800003800003C00007C0000EE0001C7000187000303800701C00F01 E0FF87FEFF87FE1714809318>II<3FFF3FFF380E301C70 3C6038607060F001E001C0038007830F030E031C073C063806701EFFFEFFFE10147F9314>II E /Fl 38 121 df<000FC00000001FE00000003FF00000007FF800 000078FC000000F07C000000F03E000000F03E000001E03E000001E03E000001E03E000001E03E 000001E07C000001F07C000001F0F8000001F1F8000001F3F0000001F7E0000000FFC0018000FF 8001E000FF0003E000FE0003C000FC0003C001FE0003C003FE0007800FFF0007801FBF000F803F 1F800F003E0F801F007C0FC01E007C07E03E00F807F03C00F803F07C00F801F8F800F800FDF000 F800FFE000FC007FC0007C003F80007E003FC0003F81FFF00C1FFFFFFFFC0FFFF1FFFC07FFC0FF FC00FE001FF0262C7EAA2C>38 D45 D<00180000380000F800 07F800FFF800FFF800FFF800F8F80000F80000F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF07FFFF07FFFF014287CA71D> 49 D<00FE0003FF800FFFC01FFFE01F07F03C01F87800FC78007C70007CF0003EF0003E60003E 20003E00003E00003E00003E00007C00007C0000780000F80001F00001E00003E00007C0000F80 001F00003E00007C0000F80001F00003E00003C0000780000F00001E00003C00007FFFFE7FFFFE 7FFFFE7FFFFE17287EA71D>I<007E0003FF8007FFE00FFFF01F81F83E00F87C007C78007C3000 7C10007C00007C00007C0000780000F80000F80001F00007E000FFC000FF8000FFC000FFF00001 F80000FC00007C00003E00003E00001F00001F00001F00001F00001F80001FC0003EE0003EF000 7E7C00FC3F01F81FFFF00FFFE003FFC000FE0018297EA71D>I<0007E0000FE0000BE0001BE000 1BE0003BE0003BE00073E00073E000F3E000F3E001E3E001E3E003E3E003C3E007C3E00F83E00F 83E01F03E01F03E03E03E03E03E07C03E07C03E0F803E0FFFFFFFFFFFFFFFFFFFFFFFF0003E000 03E00003E00003E00003E00003E00003E00003E00003E00003E018277EA61D>I<3FFFF83FFFF8 3FFFF83FFFF83E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E3E00 3EFF803FFFE03FFFF03FC1F03F80F83F007C3E007C00007C00003E00003E00003E00003E00003E 00003E00003E20003E60007C70007CF800F87C01F87F07F03FFFE00FFFC007FF0001FC0017287E A61D>I<000FE0003FF800FFF801FFF803F81807E0000F80000F80001F00003E00003E00003C00 007C00007C02007C3FC0F8FFE0F9FFF0FBFFF8FF80FCFE007CFE003EFC003EFC003FFC001FF800 1FF8001FF8001FF8001F78001F7C001F7C001F7C001E3C003E3E003E1E007C1F00FC0FC1F807FF F003FFE001FF80007E0018297EA71D>I66 D<0003FE00000FFFC0003FFFF000FFFFF001FE03F003F8006007E0002007C000000F8000001F80 00001F0000003E0000003E0000007C0000007C0000007C0000007C000000F8000000F8000000F8 000000F8000000F8000000F8000000F8000000F8000000F8000000F80000007C0000007C000000 7C0000007C0000003E0000003E0000001F0000001F8000000F80000007C0000007E0000803F800 3801FE01F800FFFFF8003FFFF0000FFFC00003FE001D2C7CAA25>I69 DI72 DI77 DI80 D82 D<007FC00001FFF80007FFFE000FFFFF00 1FC07F003F000F007E0006007C0000007C000000F8000000F8000000F8000000F8000000F80000 00FC0000007E0000007F0000003F8000001FF800000FFF000007FFE00003FFF80000FFFC00000F FE000000FF0000003F0000001F8000000F8000000FC0000007C0000007C0000007C0000007C000 0007C0000007C000000F8060000F80F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE000 00FF80001A2C7DAA21>I87 D<01FC000FFF003FFF803FFFC03C07E03003E00001F00001F00001F00001F00001F00003F001FF F007FFF01FFFF03FC1F07E01F0FC01F0F801F0F801F0F801F0FC03F07E0FF07FFFF03FFFF01FF9 F00FE1F0141B7E9A1C>97 D<007F8001FFE007FFF80FFFF81FC0783F00103E00007C00007C0000 7C0000F80000F80000F80000F80000F80000F80000F800007C00007C00007C00003E00003F0018 1FC0780FFFF807FFF801FFE0007F80151B7E9A1A>99 D<00007C00007C00007C00007C00007C00 007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00FC7C03FF7C07FFFC0F FFFC1FC1FC3F00FC3E007C7C007C7C007CFC007CF8007CF8007CF8007CF8007CF8007CF8007CF8 007CF8007C7C007C7C007C7E00FC3F00FC1F83FC1FFFFC0FFF7C03FE7C01F87C162A7EA91E>I< 00FC0003FF0007FF800FFFC01F87E03F01E03E00F07C00F07C0070F80078FFFFF8FFFFF8FFFFF8 FFFFF8F80000F80000F800007800007C00007C00003E00003F00181FC0F80FFFF807FFF801FFE0 007F00151B7E9A1A>I<00F80F0003FE7F800FFFFF801FFFFF801F07C0003E03E0003E03E0007C 01F0007C01F0007C01F0007C01F0007C01F0003E03E0003E03E0001F07C0001FFFC0003FFF8000 3BFE000038F8000078000000780000007C0000003FFF80003FFFF0001FFFF8001FFFFC003FFFFE 007C00FE007C003F00F8001F00F8001F00F8001F00F8001F00FC003F007E007E003F81FC001FFF F8000FFFF00007FFE00000FF000019287E9A1D>103 D105 D107 DIII<007E0001FF8007FFE00FFFF01F81 F83F00FC3E007C7C003E7C003E78001EF8001FF8001FF8001FF8001FF8001FF8001FF8001F7C00 3E7C003E7C003E3E007C3F00FC1FC3F80FFFF007FFE003FFC0007E00181B7E9A1D>II114 D<07F8001FFF003FFF807FFF80 7C0780F80100F80000F80000F80000FC00007F00007FF0003FFC001FFE0007FF0000FF80001FC0 000FC00007C00007C04007C0E007C0FC1F80FFFF807FFF001FFE0007F800121B7E9A16>I<07C0 0007C00007C00007C00007C00007C00007C000FFFF80FFFF80FFFF80FFFF8007C00007C00007C0 0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C08007E38003FFC003FFC001FF0000FC0012227FA115>II< F8001FF8001F7C003E7C003E7C003E3E007C3E007C3E007C1F00F81F00F81F00F80F81F00F81F0 0F81F007C3E007C3E007C3E003C3C003E7C001E78001E78001E78000E70000F70000FF00007E00 007E00181B7F9A1B>I<7C001F803E003F003F003E001F807C000F80FC0007C0F80007E1F00003 F3E00001F7E00000FFC000007F8000007F0000003E0000003E0000003F0000007F800000FFC000 01F3E00003E1E00003E1F00007C0F8000F807C001F807C001F003E003E001F007C001F80FC000F C01A1B809A1B>120 D E /Fm 15 115 df<70F8FCFC7C0C0C0C0C181830306040060F7C840E> 44 D<018003800F80FF80F3800380038003800380038003800380038003800380038003800380 0380038003800380038003800380038003800380038003800380FFFEFFFE0F217CA018>49 D<03F0000FFC001C1F00300F806007806003C0C003C0F003E0F801E0F801E0F801E02003E00003 E00003C00003C0000780000780000F00001E00001C0000380000700000E00001C0000380000700 600E00600C00601800E03000C07FFFC0FFFFC0FFFFC013217EA018>I<1000801C07801FFF001F FE001FFC001FF00018000018000018000018000018000018000018F8001BFE001F0F001C078018 03801803C00001C00001E00001E00001E00001E06001E0F001E0F001E0F001C0C003C06003C060 03803007001C1E000FFC0003F00013227EA018>53 D<01F00007FC000E0E001C07003803807803 80700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003E07803E03807 E01C0DE00FF9E007F1E00081C00001C00003C0000380000380300700780700780E00701C003838 001FF0000FC00013227EA018>57 D<000180000003C0000003C0000003C0000007E0000007E000 0007E000000FF000000CF000000CF000001CF800001878000018780000383C0000303C0000303C 0000601E0000601E0000601E0000C00F0000C00F0000C00F0001FFFF8001FFFF80018007800300 03C0030003C0030003C0060001E0060001E0060001E00E0000F01F0001F0FFC00FFFFFC00FFF20 237EA225>65 D77 D<03F0200FFC601C0EE038 03E07001E07001E0E000E0E000E0E00060E00060E00060F00000F000007800007F00003FF0001F FE000FFF0003FF80003FC00007E00001E00000F00000F0000070C00070C00070C00070C00070E0 0060E000E0F000C0F801C0EF0380C7FF0081FC0014247DA21B>83 D<0FE0001FF8003C1C003C0E 00180700000700000700000F0003FF000FFF003F07007C0700780700F00700F00718F00718F00F 18780F187C3FB83FF3F00FC3C015157E9418>97 D<01FE0007FF000F07801C0780380300780000 700000F00000F00000F00000F00000F00000F00000F000007800007800C03C00C01E01800F0300 07FE0001F80012157E9416>99 D<1C001E003E001E001C00000000000000000000000000000000 000E00FE00FE001E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 FFC0FFC00A227FA10E>105 D<0E00FE00FE001E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 00FFE0FFE00B237FA20E>108 D<0E1F80FE7FC0FFC1E01F80F00F00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FFFFE7FF18 157F941B>110 D<0E1F80FE7FE0FFC1F00F00780E00780E003C0E003C0E001E0E001E0E001E0E 001E0E001E0E001E0E001E0E003C0E003C0F00780F80700FC1F00E7FC00E1F000E00000E00000E 00000E00000E00000E00000E00000E0000FFE000FFE000171F7F941B>112 D<0E3CFEFEFFCF1F8F0F060F000E000E000E000E000E000E000E000E000E000E000E000E000E00 FFF0FFF010157F9413>114 D E /Fn 12 116 df45 D<007F800001FFE0000781F8000E007C0018003E0018001F0030000F8060000FC0600007C06000 07C0F00007E0F80007E0FC0003E0FC0003E0FC0003E0780007E0000007E0000007C0000007C000 000FC000000F8000000F8000001F0000003E0000003C00000078000000F8000000F0000001E000 0003C00000070000000E0000001C000000180000003000000060000000C0006001800060038000 60070000C00E0000C00C0000C0180001C03FFFFFC07FFFFF80FFFFFF80FFFFFF801B2F7DAE21> 50 D 73 D77 D80 D<01FE00000FFFC0001C03E0003C00F0003E 0078003E003C003E001C0008001E0000001E0000001E0000001E0000001E000001FE00003FFE00 01FF1E0007F01E000F801E001F001E003E001E007C001E007C001E00F8001E0CF8001E0CF8001E 0CF8003E0CF8003E0C7C007E0C3E00DF1C1F038FB80FFF07F003F803C01E1F7D9E21>97 D<0003E0001FF8003C3800707C00F07C01E07C01E03803C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007FFF0016327FB114 >102 D<0780FF80FF800F80078007800780078007800780078007800780078007800780078007 800780078007800780078007800780078007800780078007800780078007800780078007800780 07800780078007800780078007800780078007800FC0FFFCFFFC0E327EB112>108 D<001FC00000FFF80001E03C0007800F000F0007801E0003C01E0003C03C0001E03C0001E07800 00F0780000F0780000F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F8 0000F8780000F07C0001F03C0001E03C0001E01E0003C01E0003C00F00078007C01F0001F07C00 00FFF800001FC0001D1F7E9E21>111 D<0780FC00FF87FF80FF9E07C00FB803E007E000F007C0 00F807C0007C0780007C0780003E0780003E0780003E0780001F0780001F0780001F0780001F07 80001F0780001F0780001F0780001F0780001F0780003E0780003E0780003E0780007C07C00078 07C000F807E001F007B803E0079E0FC00787FF000781FC00078000000780000007800000078000 00078000000780000007800000078000000780000007800000078000000FC00000FFFC0000FFFC 0000202D7E9E25>I<0783E0FF8FF8FF9C7C0FB07C07F07C07E03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100FFF301E03F03800F0700070E00070E00030E00030E00030F00030F800007E00003FF0 001FFF000FFF8003FFE0003FF00003F8000078C0003CC0003CC0001CE0001CE0001CF0001CF000 18F80038FC0070EF01E0C3FFC080FE00161F7E9E1A>I E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300 TeXDict begin %%EndSetup %%Page: 1 1 bop 684 231 a Fn(Prop)r(osals)20 b(for)i(MPI-2)869 358 y Fm(Marc)16 b(Snir)824 460 y(April)f(21,)i(1995)75 648 y Fl(1)59 b(Single)19 b(sided)g(communication)75 749 y Fk(Rationale:)75 806 y(Man)o(y)e (applications)j(could)f(b)q(ene\014t)g(from)e(supp)q(ort)h(to)g(\\put")f(or)h (\\get")f(functions,)i(where)f(data)f(is)75 862 y(mo)o(v)o(ed)e(from)g(the)g (memory)g(of)h(one)f(pro)q(cess)h(to)f(the)h(memory)f(of)g(another)g(pro)q (cess,)g(follo)o(wing)i(a)e(call)75 919 y(executed)h(b)o(y)f(only)h(one)f(of) g(the)g(t)o(w)o(o)f(pro)q(cesses.)20 b(Suc)o(h)c(function)g(ma)o(y)e(b)o(y)h (implemen)o(ted)i(on)e(top)g(of)f(a)75 975 y(receiv)o(e)i(and)g(call)g(or)f (activ)o(e)g(message)g(in)o(terface)g(and)g(ma)o(y)g(get)g(h/w)g(supp)q(ort)g (on)g(some)g(platforms.)75 1096 y Fj(1.1)49 b(Creation)17 b(of)f(comm)n (unication)f(requests)75 1277 y Fi(MPI)p 160 1277 14 2 v 16 w(T)l(ARGET)p 350 1277 V 17 w(INIT\(request\))117 1354 y Fh(OUT)108 b Fi(request)452 b Fh(request)16 b(\(handle\))75 1478 y Fg(int)23 b(MPI)p 245 1478 15 2 v 17 w(Target)p 406 1478 V 17 w(init\(MPI)p 615 1478 V 16 w(Request)g(*request\))75 1565 y(MPI)p 150 1565 V 17 w(TARGET)p 311 1565 V 16 w(INIT\(REQUEST,)f(IERROR\))170 1621 y(INTEGER)h(REQUEST,)g(IERROR)166 1708 y Fk(Allo)q(cates)14 b(a)f(request)g(ob)s(ject)g(that)g(can)g(b)q(e)h(used)g(for)f(testing)g(on)g (a)g(target)f(pro)q(cess)i(the)f(status)f(of)75 1764 y(put/get)h(op)q (erations)h(that)f(w)o(ere)h(initiated)h(b)o(y)f(a)g(\(p)q(ossibly\))g (di\013eren)o(t)g(origin)h(pro)q(cess.)20 b(This)14 b(request)75 1821 y(can)k(b)q(e)g(used)g(in)g(MPI)g(T)l(est)f(and)g(W)l(ait)h(calls.)28 b(The)17 b(request)h(is)g(p)q(ersisten)o(t:)25 b(it)17 b(is)h(\\set")f(b)o(y) h(remote)75 1877 y(put/get)d(op)q(erations)h(that)e(use)i(it)g(as)f(their)h Fi(ta)o(rget)p 967 1877 14 2 v 17 w(request)h Fk(argumen)o(t.)j(It)15 b(is)i(\\unset")e(b)o(y)g(lo)q(cal)i(calls)75 1934 y(to)e(W)l(ait)g(or)f(T)l (est.)20 b(It)15 b(is)h(freed)f(b)o(y)h(a)e(call)j(to)d Fi(MPI)p 947 1934 V 16 w(REQUEST)p 1166 1934 V 18 w(FREE)p Fk(.)166 2073 y Ff(Discussion:)44 b Fh(If)16 b(MPI-2)g(incorp)q(orates)i(the)f(prop)q (osal)f(for)g(de\014ning)g(new)h(request)h(classes,)g(then)f(this)75 2129 y(function)d(ma)o(y)e(not)h(b)q(e)i(required.)75 2333 y Fj(1.2)49 b(Extensions)16 b(to)h(the)e(set)g(of)i(p)o(re-de\014ned)e(data)h (t)o(yp)q(es)75 2419 y Fk(The)f(set)f(of)g(basic)h(datat)o(yp)q(es)f(that)g (can)h(b)q(e)g(used)g(in)h(message)e(passing)h(calls)g(is)g(extended)h(to)e (include)75 2510 y Fe(MPI)p 184 2510 16 2 v 18 w(AINT)22 b Fk(Datat)o(yp)q(e)14 b(for)h(an)g(in)o(teger)h(the)f(size)h(of)f(a)g(p)q(oin) o(ter.)75 2602 y Fe(MPI)p 184 2602 V 18 w(REQUEST)24 b Fk(Request)16 b(handle)75 2695 y Fe(MPI)p 184 2695 V 18 w(D)o(A)l(T)l(A)l(TYPE)21 b Fk(Datat)o(yp)q(e)14 b(handle)964 2828 y(1)p eop %%Page: 2 2 bop 75 49 a Fj(1.3)49 b(Remote)14 b(Datat)o(yp)q(e)75 135 y Fk(The)h(set)g(of)g(datat)o(yp)q(e)g(constructors)f(is)i(extended)g(to)f (include)i(the)e(follo)o(wing.)75 286 y Fi(MPI)p 160 286 14 2 v 16 w(TYPE)p 293 286 V 17 w(REMOTE\(remotet)o(yp)q(e,)e(newt)o(yp)q(e\)) 117 363 y Fh(IN)155 b Fi(remotet)o(yp)q(e)373 b Fh(remote)14 b(datat)o(yp)q(e)g(\(handle\))117 438 y(OUT)108 b Fi(newt)o(yp)q(e)433 b Fh(new)15 b(lo)q(cal)e(datat)o(yp)q(e)h(\(handle\))75 563 y Fg(int)23 b(MPI)p 245 563 15 2 v 17 w(Type)p 358 563 V 17 w(remote\(MPI)p 615 563 V 16 w(Datatype)f(remotetype,)h(MPI)p 1204 563 V 17 w(Datatype)f(newtype\))75 649 y(MPI)p 150 649 V 17 w(TYPE)p 263 649 V 16 w(REMOTE\(REMOTETYPE,)g(NEWTYPE,)h(IERR\))170 706 y(INTEGER)g(REMOTETYPE,)g(NEWTYPE,)f(IERROR)166 792 y Fk(Encapsulates)e (a)f(remote)g(t)o(yp)q(e)h(handle)h(in)o(to)f(a)f(lo)q(cal)i(datat)o(yp)q(e)e (ob)s(ject.)32 b Fi(remotet)o(yp)q(e)18 b Fk(should)75 849 y(b)q(e)g(a)g(handle)h(to)e(a)g(t)o(yp)q(e)h(ob)s(ject,)f(either)h(lo)q(cal)h (or)e(remote)g(\(in)h(the)g(later)g(case,)g(it)g(m)o(ust)f(ha)o(v)o(e)g(b)q (een)75 905 y(receiv)o(ed)e(in)g(a)f(previous)g(comm)o(unication,)h(using)g (datat)o(yp)q(e)e Fg(MPI)p 1227 905 V 17 w(DATATYPE)p Fk(\).)f(A)i(handle)h (to)e(a)h(newly)75 962 y(created)h(lo)q(cal)i(datat)o(yp)q(e)d(is)i(returned) g(in)g Fi(newt)o(yp)q(e)p Fk(.)166 1018 y(This)i(newly)g(created)g(datat)o (yp)q(e)f(cannot)g(b)q(e)i(used)f(recursiv)o(ely)h(in)f(datat)o(yp)q(e)f (constructors.)26 b(It)75 1074 y(can)12 b(only)g(b)q(e)g(used)g(in)g(put/get) f(comm)o(unication)i(calls,)f(in)h(order)e(to)g(sp)q(ecify)i(a)e(target)f (comm)o(unication)75 1131 y(bu\013er)16 b(at)f(the)h(pro)q(cess)h(where)f Fi(remotet)o(yp)q(e)e Fk(where)i(de\014ned.)24 b(Note)16 b(that)f Fi(remotet)o(yp)q(e)f Fk(ma)o(y)i(b)q(e)g(lo)q(cal,)75 1187 y(when)g(the)f(put/get)g(call)h(is)g(used)f(b)o(y)h(a)e(pro)q(cess)i(to)e (comm)o(unicate)i(with)g(itself.)166 1320 y Ff(Discussion:)50 b Fh(It)18 b(is)g(in)o(triguing)f(to)g(relax)h(the)h(last)f(constrain)o(t.)30 b(One)19 b(could)f(extend)h(the)f(de\014nition)75 1370 y(of)c(remotet)o(yp)q (e)h(so)f(that)h(it)f(also)g(con)o(tains)h(the)g(rank)g(of)f(the)h(remote)f (pro)q(cess,)i(and)e(allo)o(w)f(to)i(apply)f(datat)o(yp)q(e)75 1419 y(constructors)e(on)d(it.)16 b(Then,)11 b(one)f(put/get)g(comm)o (unicati)o(on)d(call)i(could)g(scatter/gather)i(data)f(from/to)d(m)o(ultiple) 75 1469 y(pro)q(cesses.)21 b(Implemen)o(tation)10 b(b)q(ecomes)15 b(somew)o(ah)o(t)d(more)h(complex.)166 1526 y(Do)g(w)o(e)h(w)o(an)o(t)g(to)g (mandate)e(an)i(MPI)p 757 1526 13 2 v 15 w(COMMIT)g(call)f(here)i(to)q(o?)964 2828 y Fk(2)p eop %%Page: 3 3 bop 75 49 a Fi(1.3.1)44 b(Non-blo)q(cking)16 b(and)g(Blo)q(cking)g(puts)75 230 y(MPI)p 160 230 14 2 v 16 w(IPUT\(o)o(rigin)p 404 230 V 15 w(addr,)23 b(count,)h(o)o(rigin)p 787 230 V 16 w(datat)o(yp)q(e,)g(o)o (rigin)p 1109 230 V 15 w(request,)h(dest,)g(comm)m(,)20 b(ta)o(rget)p 1674 230 V 16 w(addr,)j(ta)o(r-)75 286 y(get)p 137 286 V 17 w(datat)o(yp)q(e,)16 b(ta)o(rget)p 458 286 V 16 w(request\))117 363 y Fh(IN)155 b Fi(o)o(rigin)p 427 363 V 16 w(addr)381 b Fh(address)15 b(of)f(origin)e(bu\013er)j(\(c)o(hoice\))117 438 y(IN)155 b Fi(count)482 b Fh(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(sen) o(t)i(\(in)o(teger\))117 513 y(IN)155 b Fi(o)o(rigin)p 427 513 V 16 w(datat)o(yp)q(e)302 b Fh(datat)o(yp)q(e)14 b(of)g(elemen)o(ts)f (sen)o(t)i(\(handle\))117 589 y(OUT)108 b Fi(o)o(rigin)p 427 589 V 16 w(request)330 b Fh(request)16 b(at)d(origing)g(pro)q(cess)j (\(handle\))117 664 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(destination)h (\(in)o(teger\))117 739 y(IN)155 b Fi(com)505 b Fh(comm)o(unicator)11 b(used)k(for)f(comm)o(unicati)o(on)d(\(handle\))117 814 y(OUT)108 b Fi(ta)o(rget)p 433 814 V 17 w(addr)374 b Fh(address)15 b(of)f(destination)f (bu\013er)i(\(c)o(hoice\))117 889 y(IN)155 b Fi(ta)o(rget)p 433 889 V 17 w(datat)o(yp)q(e)295 b Fh(datat)o(yp)q(e)14 b(used)h(for)f (receiving)g(\(handle\))117 964 y(OUT)108 b Fi(ta)o(rget)p 433 964 V 17 w(request)323 b Fh(request)16 b(at)d(target)i(pro)q(cess)g (\(handle\))75 1089 y Fg(int)47 b(int)24 b(MPI)p 365 1089 15 2 v 17 w(iput\(void)e(*origin)p 788 1089 V 16 w(addr,)i(int)f(count,)g(MPI)p 1282 1089 V 17 w(Datatype)393 1145 y(origin)p 540 1145 V 17 w(datatype,)f(MPI)p 867 1145 V 17 w(request)h(*origin)p 1243 1145 V 16 w(request,)g(int)g(dest,)393 1202 y(MPI)p 468 1202 V 17 w(Comm)g(comm,)g(void)h(*target)p 1035 1202 V 16 w(addr,)f(MPI)p 1266 1202 V 17 w(Datatype)g(target)p 1642 1202 V 16 w(datatype,)393 1258 y(MPI)p 468 1258 V 17 w(request)g(*target)p 844 1258 V 16 w(request\))75 1345 y(MPI)p 150 1345 V 17 w(IPUT\(ORIGIN)p 431 1345 V 15 w(ADDR,)h(COUNT,)f(ORIGIN)p 901 1345 V 16 w(DATATYPE,)g(ORIGIN) p 1300 1345 V 16 w(REQUEST,)g(DEST,)g(COMM,)393 1401 y(TARGET)p 540 1401 V 17 w(ADDR,)g(TARGET)p 844 1401 V 16 w(DATATYPE,)g(TARGET)p 1243 1401 V 16 w(REQUEST,)g(IERROR\))170 1457 y()g(ORIGIN)p 484 1457 V 17 w(ADDR\(*\),)g(TARGET)p 860 1457 V 16 w(ADDR\(*\))170 1514 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 1514 V 17 w(DATATYPE,)f(ORIGIN)p 1074 1514 V 17 w(REQUEST,)h(DEST,)g(COMM,)170 1570 y(TARGET)p 317 1570 V 17 w(DATATYPE,)f(TARGET)p 716 1570 V 17 w(REQUEST,)h(IERROR\))166 1657 y Fk(T)l(ransfers)12 b Fi(count)h Fk(successiv)o(e)h(elemen)o(ts)f(of)e (the)i(t)o(yp)q(e)f(sp)q(eci\014ed)j(b)o(y)d(the)g Fi(o)o(rigin)p 1518 1657 14 2 v 15 w(datat)o(yp)q(e)p Fk(,)j(starting)75 1713 y(at)g(address)i Fi(o)o(rigin)p 404 1713 V 15 w(addr)f Fk(on)g(the)g(origin)h (to)f(the)g(target)f(pro)q(cess)h(sp)q(eci\014ed)i(b)o(y)e(the)g Fi(comm)n Fk(,)c Fi(dest)18 b Fk(pair.)75 1770 y(The)d(data)g(are)g(written)g (in)o(to)g(the)h(receiv)o(e)g(bu\013er)f(sp)q(eci\014ed)i(b)o(y)e Fi(ta)o(rget)p 1318 1770 V 17 w(addr)g Fk(and)g Fi(ta)o(rget)p 1634 1770 V 17 w(datat)o(yp)q(e)p Fk(.)166 1826 y(The)i Fi(ta)o(rget)p 376 1826 V 16 w(datat)o(yp)q(e)h Fk(argumen)o(t)d(is)i(a)f(handle)i(to)e(a)g (lo)q(cal)h(datat)o(yp)q(e)f(ob)s(ject.)23 b(If)17 b(this)f(datat)o(yp)q(e)75 1883 y(ob)s(ject)d(w)o(as)g(created)h(b)o(y)f(a)g(call)i(to)e Fi(MPI)p 773 1883 V 16 w(TYPE)p 906 1883 V 17 w(REMOTE)h Fk(then)g(it)g (should)h(encapsulate)f(a)g(handle)h(to)75 1939 y(a)i(datat)o(yp)q(e)f(ob)s (ject)h(that)f(w)o(as)g(de\014ned)j(at)d(the)h(target)f(pro)q(cess.)26 b(Otherwise,)18 b(it)f(should)h(b)q(e)g(a)e(basic)75 1996 y(datat)o(yp)q(e,)e (or)h(a)g(lo)q(cally)i(de\014ned)f(datat)o(yp)q(e)f(that)f(do)q(es)i(not)f (use)g(absolute)h(addresses.)166 2052 y(The)25 b Fi(ta)o(rget)p 384 2052 V 16 w(request)h Fk(parameter)e(m)o(ust)g(ha)o(v)o(e)h(b)q(een)g (created)g(at)f(the)h(target)e(pro)q(cess)i(using)75 2108 y Fi(MPI)p 160 2108 V 16 w(T)l(ARGET)p 350 2108 V 17 w(INIT)p Fk(,)20 b(and)h(comm)o(unicated)g(to)f(the)g(origin)h(pro)q(cess)g(using)g (some)g(earlier)g(comm)o(u-)75 2165 y(nication)c(op)q(eration)e(\(with)h (datat)o(yp)q(e)f Fi(MPI)p 851 2165 V 16 w(REQUEST)p Fk(\).)h(The)g(origin)g (pro)q(cess)g(is)g(not)f(p)q(ermitted)i(to)75 2221 y(c)o(hange)f(the)g Fi(ta)o(rget)p 420 2221 V 17 w(request)h Fk(parameter)f(or)f(use)i(it)f(an)o (y)g(MPI)g(call)h(other)f(than)g(put/get)f(calls.)24 b(\(The)75 2278 y(origin)16 b(request)f(need)h(no)f(b)q(e)h(preallo)q(cated.\))166 2334 y(Let)22 b Fi(remt)o(yp)q(e)d Fk(b)q(e)k(equal)f(to)f(the)h(remote)f (datat)o(yp)q(e)g(encapsulated)i(in)f Fg(target)p 1608 2334 15 2 v 17 w(datatype)p Fk(,)f(if)75 2391 y(this)e(w)o(as)f(created)g(b)o(y)h (a)f(call)i(to)d Fi(MPI)p 758 2391 14 2 v 16 w(TYPE)p 891 2391 V 17 w(REMOTE)p Fk(,)i(or)f(to)g Fi(ta)o(rget)p 1357 2391 V 16 w(datat)o(yp)q(e)p Fk(,)j(otherwise.)30 b(The)75 2447 y(outcome)24 b(is)g(as)g(if)h(the)f(origin)h(pro)q(cess)f(executed)h(a)f(non)o(blo)q(c)o (king)i(send)e(op)q(eration)h(with)f(argu-)75 2504 y(men)o(ts)g Fg(origin)p 364 2504 15 2 v 16 w(addr,)f(origin)p 667 2504 V 17 w(datatype,)f(count,)h(dest,)g(tag,)h(comm,)f(request)p Fk(,)i(for)e(some)75 2560 y Fg(tag)17 b Fk(v)m(alue,)i(and)f(the)g(receiving) h(pro)q(cess)f(executed)g(a)f(non)o(blo)q(c)o(king)i(receiv)o(e)g(op)q (eration)e(with)h(argu-)75 2617 y(men)o(ts)13 b Fg(target)p 353 2617 V 16 w(address,)23 b(maxint,)g(remtype,)f(source,)h(tag,)h(comm,)f (request)p Fk(,)12 b(for)g(the)h(same)75 2673 y Fg(tag)i Fk(v)m(alue,)h(and)f Fg(source)g Fk(set)f(to)h(the)g(rank)g(of)g(the)g(origin)h(pro)q(cess.)964 2828 y(3)p eop %%Page: 4 4 bop 166 49 a Fk(The)16 b(origin)g(and/or)f(the)g(target)f(pro)q(cesss)i(ma)o (y)f(c)o(hec)o(k)g(or)g(w)o(ait)g(for)g(the)g(comm)o(unication)i(op)q(er-)75 106 y(ation)g(to)g(complete)h(using)g(MPI)f(test)g(and)g(w)o(ait)g (constructs,)g(the)h(origin)g(using)g(the)f(origin)h(request)75 162 y(and)h(the)f(target)f(using)j(the)e(target)f(request.)30 b(The)19 b(op)q(eration)f(completes)i(at)d(the)i(origin)g(after)f(the)75 219 y(data)f(ha)o(v)o(e)f(b)q(een)j(copied)f(out)f(of)g(the)g(origin)h (bu\013er)f(and)h(completes)g(at)e(the)i(target)e(after)g(the)i(data)75 275 y(ha)o(v)o(e)d(b)q(een)h(copied)h(in)o(to)e(the)g(target)f(bu\013er.)166 414 y Ff(Discussion:)j Fh(Is)e(the)g(user)h(resp)q(onsible)f(for)g(pro)o (viding)e(a)h(\\clean")h(comm)o(unicator)d(for)i(put/get)h(op)q(era-)75 471 y(tions?)166 686 y Ff(Implemen)o(tati)o(on)c(note:)34 b Fh(The)13 b(design)g(en)o(tails)f(no)g(c)o(hange)h(in)f(curren)o(t)i(MPI)f (and)f(curren)o(t)i(datat)o(yp)q(es,)75 736 y(except)i(p)q(ossibly)e(a)h(new) g(\015ag)f(in)g(datat)o(yp)q(es)h(ob)r(jects.)22 b(It)15 b(requires)g(an)g (extension)g(to)g(the)g(datat)o(yp)q(e)g(ob)r(ject)g(to)75 785 y(accomdate)e(remote)h(datat)o(yp)q(e)g(handles.)166 842 y(Consider)g(the)h(follo)o(wing)c(t)o(w)o(o)i(cases:)75 898 y(\(1\))19 b Fd(ta)o(rget)p 251 898 13 2 v 15 w(datat)o(yp)q(e)i Fh(is)e(a)f(basic)h(datat)o(yp)q(e)h(of)e(a)h(lo)q(cally)f(de\014ned)i(datat) o(yp)q(e.)34 b(Then,)20 b(the)g(lo)q(cal)e(co)q(de)i(that)75 955 y(implemen)o(ts)15 b(Put)j(has)f(to)h(create)h(a)e(\\\015attened")h (represen)o(tation)h(of)e(the)h(datat)o(yp)q(e)f(and)h(transfer)g(it)f(to)g (the)75 1011 y(target)h(pro)q(cess.)32 b(The)19 b(put)f(handler)g(at)g(the)g (target)h(pro)q(cess)g(uses)h(this)e(\015attened)h(datat)o(yp)q(e)f(to)g (access)h(the)75 1068 y(remote)12 b(memory)m(.)j(On)e(man)o(y)e(implemen)o (tations)f(the)k(in)o(ternal)e(represen)o(tation)i(of)f(committed)d(datat)o (yp)q(es)k(can)75 1124 y(b)q(e)e(used)h(as)f(the)h(\\\015attened")f(t)o(yp)q (e,)g(and)g(can)g(b)q(e)g(directly)h(used)f(at)g(the)g(remote)g(no)q(de)g(to) g(access)i(the)e(data.)17 b(I.e.,)75 1181 y(if)c(the)h(Put)g(call)f(is)g(la)o (y)o(ered)h(on)g(top)f(of)g(curren)o(t)i(MPI,)f(then)g(additional)e(datat)o (yp)q(e)i(\\\015atten")f(and)h(\\un\015atten")75 1237 y(calls)f(are)i (needed;)f(if)f(it)h(is)f(la)o(y)o(ered)h(on)g(top)f(of)h(the)g(in)o(ternals) g(of)f(MPI,)g(then)i(no)e(suc)o(h)i(calls)e(are)i(lik)o(ely)d(needed.)75 1294 y(The)18 b(lo)q(cal)e(call)h(can)g(recognize)i(the)f(simple)e(datat)o (yp)q(es)i(that)f(map)f(directly)h(on)g(put/get)h(h/w,)g(on)f(systems)75 1350 y(that)d(ha)o(v)o(e)g(suc)o(h.)75 1406 y(\(2\))g Fd(ta)o(rget)p 246 1406 V 16 w(datat)o(yp)q(e)h Fh(w)o(as)g(created)g(b)o(y)g(a)f(call)f(to) i Fd(MPI)p 949 1406 V 14 w(TYPE)p 1071 1406 V 14 w(REMOTE)p Fh(.)f(Then,)g(the)h(lo)q(cal)f(co)q(de)h(will)e(extract)75 1463 y(the)18 b(remote)f(handle)g(and)g(pass)g(it)g(to)g(the)h(target)g(pro)q (cess.)30 b(The)17 b(put)h(handler)f(at)g(the)h(target)g(pro)q(cess)h(uses)75 1519 y(the)f(datat)o(yp)q(e)f(ob)r(ject)g(indicated)g(b)o(y)g(this)g(handle)g (to)g(access)h(the)g(data)e(in)h(its)g(memory)m(.)24 b(The)17 b(call)g(will)e(not)75 1576 y(normally)e(map)i(in)o(to)g(put/get)i(hardw)o (are)f(\(since)i(it)d(uses)j(a)d(data)h(la)o(y)o(out)f(description)i(that)f (is)g(stored)h(at)f(the)75 1632 y(target)e(pro)q(cess\).)20 b(One)15 b(could,)e(or)h(course,)h(think)e(of)h(optimizations)d(to)j(do)g (so.)75 1866 y Fi(MPI)p 160 1866 14 2 v 16 w(PUT\(o)o(rigin)p 391 1866 V 15 w(addr,)20 b(count,)h(o)o(rigin)p 768 1866 V 16 w(datat)o(yp)q(e,)g(dest,)g(comm)m(,)c(ta)o(rget)p 1351 1866 V 16 w(addr,)k(ta)o(rget)p 1597 1866 V 16 w(datat)o(yp)q(e,)g(ta)o(r-)75 1922 y(get)p 137 1922 V 17 w(request\))117 2000 y Fh(IN)155 b Fi(o)o(rigin)p 427 2000 V 16 w(addr)381 b Fh(address)15 b(of)f(origin)e (bu\013er)j(\(c)o(hoice\))117 2074 y(IN)155 b Fi(count)482 b Fh(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(sen)o(t)i(\(in)o(teger\))117 2149 y(IN)155 b Fi(o)o(rigin)p 427 2149 V 16 w(datat)o(yp)q(e)302 b Fh(datat)o(yp)q(e)14 b(of)g(elemen)o(ts)f(sen)o(t)i(\(handle\))117 2224 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(destination)h(\(in)o(teger\)) 117 2298 y(IN)155 b Fi(com)505 b Fh(comm)o(unicator)11 b(used)k(for)f(comm)o (unicati)o(on)d(\(handle\))117 2373 y(OUT)108 b Fi(ta)o(rget)p 433 2373 V 17 w(addr)374 b Fh(address)15 b(of)f(destination)f(bu\013er)i(\(c) o(hoice\))117 2448 y(IN)155 b Fi(ta)o(rget)p 433 2448 V 17 w(datat)o(yp)q(e)295 b Fh(datat)o(yp)q(e)14 b(used)h(for)f(receiving)g (\(handle\))117 2523 y(OUT)108 b Fi(ta)o(rget)p 433 2523 V 17 w(request)323 b Fh(request)16 b(at)d(target)i(pro)q(cess)g(\(handle\))75 2647 y Fg(int)47 b(int)24 b(MPI)p 365 2647 15 2 v 17 w(put\(void)e(*origin)p 764 2647 V 17 w(addr,)h(int)g(count,)g(MPI)p 1258 2647 V 17 w(Datatype)g(origin)p 1634 2647 V 16 w(datatype,)393 2704 y(int)h(dest,)f (MPI)p 707 2704 V 17 w(Comm)g(comm,)g(void)g(*target)p 1273 2704 V 17 w(addr,)g(MPI)p 1505 2704 V 17 w(Datatype)964 2828 y Fk(4)p eop %%Page: 5 5 bop 393 49 a Fg(target)p 540 49 15 2 v 17 w(datatype,)22 b(MPI)p 867 49 V 17 w(request)h(*target)p 1243 49 V 16 w(request\))75 136 y(MPI)p 150 136 V 17 w(PUT\(ORIGIN)p 407 136 V 16 w(ADDR,)g(COUNT,)g (ORIGIN)p 877 136 V 16 w(DATATYPE,)g(DEST,)g(COMM,)g(TARGET)p 1562 136 V 17 w(ADDR,)393 192 y(TARGET)p 540 192 V 17 w(DATATYPE,)f(TARGET)p 939 192 V 17 w(REQUEST,)g(IERROR\))170 249 y()h(ORIGIN)p 484 249 V 17 w(ADDR\(*\),)g(TARGET)p 860 249 V 16 w(ADDR\(*\))170 305 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 305 V 17 w(DATATYPE,)f(DEST,)i(COMM,)f (TARGET)p 1361 305 V 16 w(DATATYPE,)170 362 y(TARGET)p 317 362 V 17 w(REQUEST,)g(IERROR\))166 448 y Fk(This)14 b(is)f(the)g(blo)q(c)o (king)i(v)o(ersion)e(of)g Fg(MPI)p 843 448 V 17 w(IPUT)p Fk(.)f(Con)o(trol)g (do)q(es)i(not)f(return)g(to)f(the)h(caller)i(un)o(til)f(the)75 504 y(data)h(ha)o(v)o(e)f(b)q(een)j(copied)f(out)f(of)g(the)g(origin)h (bu\013er.)75 626 y Fj(1.4)49 b(Non-blo)q(cking)19 b(and)d(Blo)q(cking)i (Synchronous)f(puts)75 806 y Fi(MPI)p 160 806 14 2 v 16 w(ISPUT\(o)o(rigin)p 429 806 V 15 w(addr,)j(count,)g(o)o(rigin)p 805 806 V 16 w(datat)o(yp)q(e,)h (o)o(rigin)p 1124 806 V 15 w(request,)g(dest,)g(comm)m(,)16 b(ta)o(rget)p 1677 806 V 17 w(addr,)j(ta)o(r-)75 863 y(get)p 137 863 V 17 w(datat)o(yp)q(e,)d(ta)o(rget)p 458 863 V 16 w(request\))117 940 y Fh(IN)155 b Fi(o)o(rigin)p 427 940 V 16 w(addr)381 b Fh(address)15 b(of)f(origin)e(bu\013er)j(\(c)o(hoice\))117 1015 y(IN)155 b Fi(count)482 b Fh(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f (sen)o(t)i(\(in)o(teger\))117 1090 y(IN)155 b Fi(o)o(rigin)p 427 1090 V 16 w(datat)o(yp)q(e)302 b Fh(datat)o(yp)q(e)14 b(of)g(elemen)o(ts) f(sen)o(t)i(\(handle\))117 1166 y(OUT)108 b Fi(o)o(rigin)p 427 1166 V 16 w(request)330 b Fh(request)16 b(at)d(origin)g(pro)q(cess)j (\(handle\))117 1241 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(destination)h (\(in)o(teger\))117 1316 y(IN)155 b Fi(com)505 b Fh(comm)o(unicator)11 b(used)k(for)f(comm)o(unicati)o(on)d(\(handle\))117 1391 y(OUT)108 b Fi(ta)o(rget)p 433 1391 V 17 w(addr)374 b Fh(address)15 b(of)f(destination) f(bu\013er)i(\(c)o(hoice\))117 1466 y(IN)155 b Fi(ta)o(rget)p 433 1466 V 17 w(datat)o(yp)q(e)295 b Fh(datat)o(yp)q(e)14 b(used)h(for)f (receiving)g(\(handle\))117 1541 y(OUT)108 b Fi(ta)o(rget)p 433 1541 V 17 w(request)323 b Fh(request)16 b(at)d(target)i(pro)q(cess)g (\(handle\))75 1666 y Fg(int)47 b(int)24 b(MPI)p 365 1666 15 2 v 17 w(isput\(void)e(*origin)p 812 1666 V 16 w(addr,)h(int)h(count,)f(MPI)p 1306 1666 V 17 w(Datatype)393 1722 y(origin)p 540 1722 V 17 w(datatype,)f(MPI)p 867 1722 V 17 w(request)h(*origin)p 1243 1722 V 16 w(request,)g(int)g(dest,)393 1779 y(MPI)p 468 1779 V 17 w(Comm)g(comm,)g(void)h(*target)p 1035 1779 V 16 w(addr,)f(MPI)p 1266 1779 V 17 w(Datatype)g(target)p 1642 1779 V 16 w(datatype,)393 1835 y(MPI)p 468 1835 V 17 w(request)g(*target)p 844 1835 V 16 w(request\))75 1921 y(MPI)p 150 1921 V 17 w(ISPUT\(ORIGIN)p 455 1921 V 15 w(ADDR,)g(COUNT,)h(ORIGIN)p 925 1921 V 16 w(DATATYPE,)f(ORIGIN) p 1324 1921 V 16 w(REQUEST,)g(DEST,)g(COMM,)393 1978 y(TARGET)p 540 1978 V 17 w(ADDR,)g(TARGET)p 844 1978 V 16 w(DATATYPE,)g(TARGET)p 1243 1978 V 16 w(REQUEST,)g(IERROR\))170 2034 y()g(ORIGIN)p 484 2034 V 17 w(ADDR\(*\),)g(TARGET)p 860 2034 V 16 w(ADDR\(*\))170 2091 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 2091 V 17 w(DATATYPE,)f(ORIGIN)p 1074 2091 V 17 w(REQUEST,)h(DEST,)g(COMM,)170 2147 y(TARGET)p 317 2147 V 17 w(DATATYPE,)f(TARGET)p 716 2147 V 17 w(REQUEST,)h(IERROR\))166 2234 y Fk(Similar)13 b(to)f Fg(MPI)p 447 2234 V 17 w(IPUT)p Fk(,)f(except)h(that)g(the)g(seman)o(tics)g(of)g(completion)h(at)e(the)i (origin)f(are)g(di\013eren)o(t.)75 2290 y(Completion)18 b(of)f(an)g Fg(MPI)p 516 2290 V 17 w(ISPUT)f Fk(means)h(that)g(the)g(data)g(ha)o(v)o(e)g (arriv)o(ed)g(at)f(the)i(target)e(pro)q(cess,)h(i.e.,)75 2347 y(the)e(transfer)g(has)g(completed)h(b)q(oth)f(in)h(origin)g(and)g(target)e (memory)l(.)964 2828 y(5)p eop %%Page: 6 6 bop 75 49 a Fi(MPI)p 160 49 14 2 v 16 w(SPUT\(o)o(rigin)p 416 49 V 15 w(addr,)17 b(count,)g(o)o(rigin)p 786 49 V 16 w(datat)o(yp)q(e,)g (dest,)h(comm)m(,)13 b(ta)o(rget)p 1358 49 V 17 w(addr,)j(ta)o(rget)p 1600 49 V 16 w(datat)o(yp)q(e,)i(ta)o(r-)75 106 y(get)p 137 106 V 17 w(request\))117 183 y Fh(IN)155 b Fi(o)o(rigin)p 427 183 V 16 w(addr)381 b Fh(address)15 b(of)f(origin)e(bu\013er)j(\(c)o(hoice\)) 117 258 y(IN)155 b Fi(count)482 b Fh(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f (sen)o(t)i(\(in)o(teger\))117 333 y(IN)155 b Fi(o)o(rigin)p 427 333 V 16 w(datat)o(yp)q(e)302 b Fh(datat)o(yp)q(e)14 b(of)g(elemen)o(ts)f (sen)o(t)i(\(handle\))117 408 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(destination)h(\(in)o(teger\))117 483 y(IN)155 b Fi(com)505 b Fh(comm)o(unicator)11 b(used)k(for)f(comm)o(unicati)o(on)d(\(handle\))117 559 y(OUT)108 b Fi(ta)o(rget)p 433 559 V 17 w(addr)374 b Fh(address)15 b(of)f(destination)f(bu\013er)i(\(c)o(hoice\))117 634 y(IN)155 b Fi(ta)o(rget)p 433 634 V 17 w(datat)o(yp)q(e)295 b Fh(datat)o(yp)q(e)14 b(used)h(for)f(receiving)g(\(handle\))117 709 y(OUT)108 b Fi(ta)o(rget)p 433 709 V 17 w(request)323 b Fh(request)16 b(at)d(target)i(pro)q(cess)g (\(handle\))75 833 y Fg(int)47 b(int)24 b(MPI)p 365 833 15 2 v 17 w(sput\(void)e(*origin)p 788 833 V 16 w(addr,)i(int)f(count,)g(MPI)p 1282 833 V 17 w(Datatype)393 890 y(origin)p 540 890 V 17 w(datatype,)f(int)i (dest,)f(MPI)p 1106 890 V 17 w(Comm)g(comm,)g(void)g(*target)p 1672 890 V 17 w(addr,)393 946 y(MPI)p 468 946 V 17 w(Datatype)g(target)p 844 946 V 16 w(datatype,)g(MPI)p 1171 946 V 16 w(request)g(*target)p 1546 946 V 17 w(request\))75 1033 y(MPI)p 150 1033 V 17 w(SPUT\(ORIGIN)p 431 1033 V 15 w(ADDR,)h(COUNT,)f(ORIGIN)p 901 1033 V 16 w(DATATYPE,)g(DEST,)g (COMM,)g(TARGET)p 1586 1033 V 16 w(ADDR,)393 1089 y(TARGET)p 540 1089 V 17 w(DATATYPE,)f(TARGET)p 939 1089 V 17 w(REQUEST,)g(IERROR\))170 1146 y()h(ORIGIN)p 484 1146 V 17 w(ADDR\(*\),)g(TARGET)p 860 1146 V 16 w(ADDR\(*\))170 1202 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 1202 V 17 w(DATATYPE,)f(DEST,)i(COMM,)f(TARGET)p 1361 1202 V 16 w(DATATYPE,)170 1259 y(TARGET)p 317 1259 V 17 w(REQUEST,)g(IERROR\))166 1345 y Fk(This)16 b(is)f(the)h(blo)q(c)o(king)g(v)o(ersion)g(of)e Fg(MPI)p 855 1345 V 17 w(ISPUT)p Fk(.)75 1467 y Fj(1.5)49 b(Non-blo)q(cking) 19 b(and)d(Blo)q(cking)i(Gets)75 1647 y Fi(MPI)p 160 1647 14 2 v 16 w(IGET\(o)o(rigin)p 401 1647 V 15 w(addr,)24 b(count,)g(o)o(rigin)p 785 1647 V 15 w(datat)o(yp)q(e,)i(o)o(rigin)p 1108 1647 V 15 w(request,)f(dest,)g(comm)m(,)20 b(ta)o(rget)p 1673 1647 V 17 w(addr,)j(ta)o(r-)75 1703 y(get)p 137 1703 V 17 w(datat)o(yp)q(e,)16 b(ta)o(rget)p 458 1703 V 16 w(request\))117 1781 y Fh(IN)155 b Fi(o)o(rigin)p 427 1781 V 16 w(addr)381 b Fh(address)15 b(of)f(origin)e (bu\013er)j(\(c)o(hoice\))117 1856 y(IN)155 b Fi(count)482 b Fh(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(sen)o(t)i(\(in)o(teger\))117 1931 y(IN)155 b Fi(o)o(rigin)p 427 1931 V 16 w(datat)o(yp)q(e)302 b Fh(datat)o(yp)q(e)14 b(of)g(elemen)o(ts)f(sen)o(t)i(\(handle\))117 2006 y(OUT)108 b Fi(o)o(rigin)p 427 2006 V 16 w(request)330 b Fh(request)16 b(at)d(origin)g(pro)q(cess)j(\(handle\))117 2081 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(destination)h(\(in)o(teger\)) 117 2156 y(IN)155 b Fi(com)505 b Fh(comm)o(unicator)11 b(used)k(for)f(comm)o (unicati)o(on)d(\(handle\))117 2231 y(OUT)108 b Fi(ta)o(rget)p 433 2231 V 17 w(addr)374 b Fh(address)15 b(of)f(destination)f(bu\013er)i(\(c) o(hoice\))117 2307 y(IN)155 b Fi(ta)o(rget)p 433 2307 V 17 w(datat)o(yp)q(e)295 b Fh(datat)o(yp)q(e)14 b(used)h(for)f(receiving)g (\(handle\))117 2382 y(OUT)108 b Fi(ta)o(rget)p 433 2382 V 17 w(request)323 b Fh(request)16 b(at)d(target)i(pro)q(cess)g(\(handle\))75 2506 y Fg(int)47 b(int)24 b(MPI)p 365 2506 15 2 v 17 w(iget\(void)e(*origin)p 788 2506 V 16 w(addr,)i(int)f(count,)g(MPI)p 1282 2506 V 17 w(Datatype)393 2563 y(origin)p 540 2563 V 17 w(datatype,)f(MPI)p 867 2563 V 17 w(request)h(*origin)p 1243 2563 V 16 w(request,)g(int)g(dest,) 393 2619 y(MPI)p 468 2619 V 17 w(Comm)g(comm,)g(void)h(*target)p 1035 2619 V 16 w(addr,)f(MPI)p 1266 2619 V 17 w(Datatype)g(target)p 1642 2619 V 16 w(datatype,)393 2676 y(MPI)p 468 2676 V 17 w(request)g (*target)p 844 2676 V 16 w(request\))964 2828 y Fk(6)p eop %%Page: 7 7 bop 75 49 a Fg(MPI)p 150 49 15 2 v 17 w(IGET\(ORIGIN)p 431 49 V 15 w(ADDR,)24 b(COUNT,)f(ORIGIN)p 901 49 V 16 w(DATATYPE,)g(ORIGIN)p 1300 49 V 16 w(REQUEST,)g(DEST,)g(COMM,)393 106 y(TARGET)p 540 106 V 17 w(ADDR,)g(TARGET)p 844 106 V 16 w(DATATYPE,)g(TARGET)p 1243 106 V 16 w(REQUEST,)g(IERROR\))170 162 y()g(ORIGIN)p 484 162 V 17 w(ADDR\(*\),)g(TARGET)p 860 162 V 16 w(ADDR\(*\))170 219 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 219 V 17 w(DATATYPE,)f(ORIGIN)p 1074 219 V 17 w(REQUEST,)h(DEST,)g(COMM,)170 275 y(TARGET)p 317 275 V 17 w(DATATYPE,)f(TARGET)p 716 275 V 17 w(REQUEST,)h(IERROR\))166 362 y Fk(Similar)18 b(to)d Fg(MPI)p 455 362 V 17 w(ISPUT)p Fk(,)g(except)h(that)g(the)g(direction)h(of)f(data)g(transfer)f(is)i(rev)o (ersed.)22 b(Data)15 b(are)75 418 y(copied)g(from)e(the)g(target)g(pro)q (cess)g(to)g(the)h(origin.)20 b(The)14 b Fg(target)p 1202 418 V 16 w(request)f Fk(completes)h(after)f(the)h(data)75 474 y(ha)o(v)o(e)g(b)q (een)h(copied)g(out)f(of)f(the)h(memory)g(at)f(the)h(target)f(pro)q(cess.)20 b(The)14 b Fg(origin)p 1486 474 V 16 w(request)f Fk(completes)75 531 y(when)j(the)f(data)g(ha)o(v)o(e)f(arriv)o(ed)i(at)e(the)h(origin.)75 682 y Fi(MPI)p 160 682 14 2 v 16 w(GET\(o)o(rigin)p 388 682 V 15 w(addr,)21 b(count,)g(o)o(rigin)p 766 682 V 16 w(datat)o(yp)q(e,)g (dest,)h(comm)m(,)17 b(ta)o(rget)p 1350 682 V 17 w(addr,)j(ta)o(rget)p 1596 682 V 16 w(datat)o(yp)q(e,)i(ta)o(r-)75 738 y(get)p 137 738 V 17 w(request\))117 816 y Fh(IN)155 b Fi(o)o(rigin)p 427 816 V 16 w(addr)381 b Fh(address)15 b(of)f(origin)e(bu\013er)j(\(c)o(hoice\)) 117 891 y(IN)155 b Fi(count)482 b Fh(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f (sen)o(t)i(\(in)o(teger\))117 966 y(IN)155 b Fi(o)o(rigin)p 427 966 V 16 w(datat)o(yp)q(e)302 b Fh(datat)o(yp)q(e)14 b(of)g(elemen)o(ts)f (sen)o(t)i(\(handle\))117 1041 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(destination)h(\(in)o(teger\))117 1116 y(IN)155 b Fi(com)505 b Fh(comm)o(unicator)11 b(used)k(for)f(comm)o(unicati)o(on)d(\(handle\))117 1191 y(OUT)108 b Fi(ta)o(rget)p 433 1191 V 17 w(addr)374 b Fh(address)15 b(of)f(destination)f(bu\013er)i(\(c)o(hoice\))117 1266 y(IN)155 b Fi(ta)o(rget)p 433 1266 V 17 w(datat)o(yp)q(e)295 b Fh(datat)o(yp)q(e)14 b(used)h(for)f(receiving)g(\(handle\))117 1341 y(OUT)108 b Fi(ta)o(rget)p 433 1341 V 17 w(request)323 b Fh(request)16 b(at)d(target)i(pro)q(cess)g(\(handle\))75 1466 y Fg(int)47 b(int)24 b(MPI)p 365 1466 15 2 v 17 w(get\(void)e(*origin)p 764 1466 V 17 w(addr,)h(int)g(count,)g(MPI)p 1258 1466 V 17 w(Datatype)g(origin)p 1634 1466 V 16 w(datatype,)393 1522 y(int)h(dest,)f (MPI)p 707 1522 V 17 w(Comm)g(comm,)g(void)g(*target)p 1273 1522 V 17 w(addr,)g(MPI)p 1505 1522 V 17 w(Datatype)393 1579 y(target)p 540 1579 V 17 w(datatype,)f(MPI)p 867 1579 V 17 w(request)h(*target)p 1243 1579 V 16 w(request\))75 1665 y(MPI)p 150 1665 V 17 w(GET\(ORIGIN)p 407 1665 V 16 w(ADDR,)g(COUNT,)g(ORIGIN)p 877 1665 V 16 w(DATATYPE,)g(DEST,)g(COMM,)g(TARGET)p 1562 1665 V 17 w(ADDR,)393 1722 y(TARGET)p 540 1722 V 17 w(DATATYPE,)f(TARGET)p 939 1722 V 17 w(REQUEST,)g(IERROR\))170 1778 y()h(ORIGIN)p 484 1778 V 17 w(ADDR\(*\),)g(TARGET)p 860 1778 V 16 w(ADDR\(*\))170 1835 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 1835 V 17 w(DATATYPE,)f(DEST,)i(COMM,) f(TARGET)p 1361 1835 V 16 w(DATATYPE,)170 1891 y(TARGET)p 317 1891 V 17 w(REQUEST,)g(IERROR\))166 1978 y Fk(This)16 b(is)f(the)h(blo)q(c)o (king)g(v)o(ersion)g(of)e Fg(MPI)p 855 1978 V 17 w(ISGET)p Fk(.)964 2828 y(7)p eop %%Page: 8 8 bop 75 49 a Fj(1.6)49 b(P)o(ersistent)14 b(put/get)i(objects)75 230 y Fi(MPI)p 160 230 14 2 v 16 w(IPUT)p 280 230 V 16 w(CREA)l(TE\(o)o (rigin)p 589 230 V 16 w(addr,)11 b(count,)h(o)o(rigin)p 949 230 V 16 w(datat)o(yp)q(e,)g(o)o(rigin)p 1259 230 V 15 w(request,)h(dest,)f (comm)m(,)c(ta)o(rget)p 1787 230 V 16 w(addr,)75 286 y(ta)o(rget)p 190 286 V 16 w(datat)o(yp)q(e,)17 b(ta)o(rget)p 511 286 V 16 w(request\))117 363 y Fh(IN)155 b Fi(o)o(rigin)p 427 363 V 16 w(addr)381 b Fh(address)15 b(of)f(origin)e(bu\013er)j(\(c)o(hoice\))117 438 y(IN)155 b Fi(count)482 b Fh(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(sen) o(t)i(\(in)o(teger\))117 513 y(IN)155 b Fi(o)o(rigin)p 427 513 V 16 w(datat)o(yp)q(e)302 b Fh(datat)o(yp)q(e)14 b(of)g(elemen)o(ts)f (sen)o(t)i(\(handle\))117 589 y(OUT)108 b Fi(o)o(rigin)p 427 589 V 16 w(request)330 b Fh(request)16 b(at)d(origing)g(pro)q(cess)j (\(handle\))117 664 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(destination)h (\(in)o(teger\))117 739 y(IN)155 b Fi(com)505 b Fh(comm)o(unicator)11 b(used)k(for)f(comm)o(unicati)o(on)d(\(handle\))117 814 y(OUT)108 b Fi(ta)o(rget)p 433 814 V 17 w(addr)374 b Fh(address)15 b(of)f(destination)f (bu\013er)i(\(c)o(hoice\))117 889 y(IN)155 b Fi(ta)o(rget)p 433 889 V 17 w(datat)o(yp)q(e)295 b Fh(datat)o(yp)q(e)14 b(used)h(for)f (receiving)g(\(handle\))117 964 y(OUT)108 b Fi(ta)o(rget)p 433 964 V 17 w(request)323 b Fh(request)16 b(at)d(target)i(pro)q(cess)g (\(handle\))75 1089 y Fg(int)47 b(int)24 b(MPI)p 365 1089 15 2 v 17 w(Iput)p 478 1089 V 16 w(create\(void)f(*origin)p 949 1089 V 16 w(addr,)g(int)g(count,)g(MPI)p 1442 1089 V 17 w(Datatype)393 1145 y(origin)p 540 1145 V 17 w(datatype,)f(MPI)p 867 1145 V 17 w(request)h(*origin)p 1243 1145 V 16 w(request,)g(int)g(dest,)393 1202 y(MPI)p 468 1202 V 17 w(Comm)g(comm,)g(void)h(*target)p 1035 1202 V 16 w(addr,)f(MPI)p 1266 1202 V 17 w(Datatype)g(target)p 1642 1202 V 16 w(datatype,)393 1258 y(MPI)p 468 1258 V 17 w(request)g (*target)p 844 1258 V 16 w(request\))75 1345 y(MPI)p 150 1345 V 17 w(IPUT)p 263 1345 V 16 w(CREATE\(ORIGIN)p 591 1345 V 16 w(ADDR,)g(COUNT,)g(ORIGIN)p 1061 1345 V 16 w(DATATYPE,)g(ORIGIN)p 1460 1345 V 16 w(REQUEST,)g(DEST,)393 1401 y(COMM,)g(TARGET)p 683 1401 V 17 w(ADDR,)g(TARGET)p 987 1401 V 16 w(DATATYPE,)g(TARGET)p 1386 1401 V 16 w(REQUEST,)g(IERROR\))170 1457 y()g(ORIGIN)p 484 1457 V 17 w(ADDR\(*\),)g(TARGET)p 860 1457 V 16 w(ADDR\(*\))170 1514 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 1514 V 17 w(DATATYPE,)f(ORIGIN)p 1074 1514 V 17 w(REQUEST,)h(DEST,)g(COMM,)170 1570 y(TARGET)p 317 1570 V 17 w(DATATYPE,)f(TARGET)p 716 1570 V 17 w(REQUEST,)h(IERROR\))166 1657 y Fk(Returns)18 b(in)g Fi(o)o(rigin)p 505 1657 14 2 v 16 w(request)h Fk(a)e(p)q(ersisten)o(t)h(handle)h(for)e(Put)h(op)q(erations.) 27 b(The)18 b(parameters)e(are)75 1713 y(as)f(for)g(the)h Fi(MPI)p 365 1713 V 16 w(IPUT)g Fk(call.)22 b(Once)17 b(created,)f(the)f(handle)i(can) f(b)q(e)h(used)f(to)f(initiate)i(Put)f(op)q(erations)75 1770 y(b)o(y)i(a)g(call)i(to)d Fi(MPI)p 412 1770 V 16 w(ST)l(ART\(o)o(rigin)p 694 1770 V 16 w(request\))j Fk(\(or)d Fi(MPI)p 1041 1770 V 16 w(ST)l(ART)l(ALL)p Fk(\),)h(as)g(for)g(usual)h(MPI)f(p)q(ersisten)o(t)75 1826 y(requests.)i(The)15 b(request)g(is)h(freed)g(with)f(a)g(call)h(to)f Fi(MPI)p 1044 1826 V 16 w(REQUEST)p 1263 1826 V 18 w(FREE)p Fk(.)75 1977 y Fi(MPI)p 160 1977 V 16 w(ISPUT)p 305 1977 V 16 w(CREA)l(TE\(o)o(rigin)p 614 1977 V 16 w(addr,)c(count,)i(o)o(rigin)p 975 1977 V 15 w(datat)o(yp)q(e,)f(o)o(rigin)p 1284 1977 V 15 w(request,)h(dest,)f(comm)m(,)c(ta)o(rget)p 1812 1977 V 16 w(addr,)75 2034 y(ta)o(rget)p 190 2034 V 16 w(datat)o(yp)q(e,)17 b(ta)o(rget)p 511 2034 V 16 w(request\))75 2137 y Fg(int)47 b(int)24 b(MPI)p 365 2137 15 2 v 17 w(Isput)p 502 2137 V 16 w(create\(void)e(*origin)p 972 2137 V 17 w(addr,)h(int)g(count,)g(MPI)p 1466 2137 V 17 w(Datatype)393 2194 y(origin)p 540 2194 V 17 w(datatype,)f(MPI)p 867 2194 V 17 w(request)h(*origin)p 1243 2194 V 16 w(request,)g(int)g(dest,)393 2250 y(MPI)p 468 2250 V 17 w(Comm)g(comm,)g(void)h(*target)p 1035 2250 V 16 w(addr,)f(MPI)p 1266 2250 V 17 w(Datatype)g(target)p 1642 2250 V 16 w(datatype,)393 2307 y(MPI)p 468 2307 V 17 w(request)g(*target)p 844 2307 V 16 w(request\))75 2393 y(MPI)p 150 2393 V 17 w(ISPUT)p 287 2393 V 16 w(CREATE\(ORIGIN)p 615 2393 V 16 w(ADDR,)g(COUNT,)g(ORIGIN)p 1085 2393 V 16 w(DATATYPE,)g(ORIGIN)p 1484 2393 V 16 w(REQUEST,)g(DEST,)393 2450 y(COMM,)g(TARGET)p 683 2450 V 17 w(ADDR,)g(TARGET)p 987 2450 V 16 w(DATATYPE,)g(TARGET)p 1386 2450 V 16 w(REQUEST,)g(IERROR\))170 2506 y()g(ORIGIN)p 484 2506 V 17 w(ADDR\(*\),)g(TARGET)p 860 2506 V 16 w(ADDR\(*\))170 2562 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 2562 V 17 w(DATATYPE,)f(ORIGIN)p 1074 2562 V 17 w(REQUEST,)h(DEST,)g (COMM,)170 2619 y(TARGET)p 317 2619 V 17 w(DATATYPE,)f(TARGET)p 716 2619 V 17 w(REQUEST,)h(IERROR\))964 2828 y Fk(8)p eop %%Page: 9 9 bop 75 49 a Fi(MPI)p 160 49 14 2 v 16 w(IGET)p 277 49 V 16 w(CREA)l(TE\(o)o(rigin)p 586 49 V 16 w(addr,)11 b(count,)h(o)o(rigin)p 946 49 V 16 w(datat)o(yp)q(e,)g(o)o(rigin)p 1256 49 V 15 w(request,)h(dest,)f (comm)m(,)c(ta)o(rget)p 1784 49 V 16 w(addr,)75 106 y(ta)o(rget)p 190 106 V 16 w(datat)o(yp)q(e,)17 b(ta)o(rget)p 511 106 V 16 w(request\))75 209 y Fg(int)47 b(int)24 b(MPI)p 365 209 15 2 v 17 w(Iget)p 478 209 V 16 w(create\(void)f(*origin)p 949 209 V 16 w(addr,)g(int)g(count,)g(MPI)p 1442 209 V 17 w(Datatype)393 266 y(origin)p 540 266 V 17 w(datatype,)f(MPI)p 867 266 V 17 w(request)h(*origin)p 1243 266 V 16 w(request,)g(int)g(dest,)393 322 y(MPI)p 468 322 V 17 w(Comm)g(comm,)g(void)h(*target)p 1035 322 V 16 w(addr,)f(MPI)p 1266 322 V 17 w(Datatype)g(target)p 1642 322 V 16 w(datatype,)393 379 y(MPI)p 468 379 V 17 w(request)g(*target)p 844 379 V 16 w(request\))75 465 y(MPI)p 150 465 V 17 w(IGET)p 263 465 V 16 w(CREATE\(ORIGIN)p 591 465 V 16 w(ADDR,)g(COUNT,)g(ORIGIN)p 1061 465 V 16 w(DATATYPE,)g(ORIGIN)p 1460 465 V 16 w(REQUEST,)g(DEST,)393 522 y(COMM,)g(TARGET)p 683 522 V 17 w(ADDR,)g(TARGET)p 987 522 V 16 w(DATATYPE,)g(TARGET)p 1386 522 V 16 w(REQUEST,)g(IERROR\))170 578 y()g(ORIGIN)p 484 578 V 17 w(ADDR\(*\),)g(TARGET)p 860 578 V 16 w(ADDR\(*\))170 635 y(INTEGER)g(COUNT,)g(ORIGIN)p 675 635 V 17 w(DATATYPE,)f(ORIGIN)p 1074 635 V 17 w(REQUEST,)h(DEST,)g(COMM,) 170 691 y(TARGET)p 317 691 V 17 w(DATATYPE,)f(TARGET)p 716 691 V 17 w(REQUEST,)h(IERROR\))166 778 y Fk(Create)15 b(p)q(ersisten)o(t)g (request)g(ob)s(jects)g(for)g(Isput)g(and)h(Iget)f(op)q(erations.)166 917 y Ff(Discussion:)49 b Fh(P)o(ersisten)o(t)19 b(rquests)g(in)e(MPI)h(are)g (\\half-c)o(hannels":)24 b(they)18 b(bind)f(the)i(send)f(or)g(receiv)o(e)75 973 y(param)o(ters,)11 b(but)i(they)f(do)g(not)g(bind)g(a)g(send)h(to)e(a)h (receiv)o(e.)19 b(Since)13 b(put/get)f(op)q(erations)g(are)h(single-sided,)f (there)75 1030 y(is)i(no)f(di\013erence)j(here)f(b)q(et)o(w)o(een)g(a)f (\\half-c)o(hannel")f(and)g(a)h(\\full-c)o(hannel".)166 1251 y Ff(Implemen)o(tati)o(on)f(note:)37 b Fh(An)14 b(ob)o(vious)g(optimization)d (for)j(p)q(ersisten)o(t)i(put/get)f(requests)h(is)e(to)g(cac)o(he)75 1308 y(the)i(remote)f(datat)o(yp)q(e)g(at)g(the)h(remote)f(no)q(de.)22 b(The)16 b(\015attened)g(datat)o(yp)q(e)f(is)g(transfered)i(to)e(the)h (remote)f(no)q(de)75 1364 y(when)g(the)h(\014rst)f(use)h(of)e(the)i(p)q (ersisten)o(t)g(request)h(o)q(ccurs,)f(and)e(cac)o(hed)i(there.)22 b(Afterw)o(ards,)16 b(the)f(lo)q(cal)f(cac)o(hed)75 1421 y(cop)o(y)g(is)g (used.)19 b(Ob)o(vious)13 b(extensions)i(can)f(b)q(e)h(suggested)g(to)f(deal) f(with)h(cac)o(he)h(o)o(v)o(er\015o)o(w.)75 1644 y Fl(2)59 b(RMW)19 b(op)r(erations)75 1746 y Fk(Remote)12 b(Read)p 343 1746 14 2 v 17 w(Mo)q(dify)p 501 1746 V 16 w(W)l(rite)g(op)q(erations)g(pro)o (vide)g(atomic)f(test)g(and)h(up)q(date)g(of)f(a)g(remote)g(v)m(ariable.)75 1802 y(They)19 b(can)g(b)q(e)h(used)g(to)e(implemen)o(t)i(sync)o(hronization) g(op)q(erations)f(suc)o(h)h(as)e(semaphors,)i(coun)o(ting)75 1859 y(semaphors,)14 b(etc.)75 2010 y Fi(MPI)p 160 2010 V 16 w(RMW\(op,)h(invalue,)g(ta)o(rgetva)o(r,)f(datat)o(yp)q(e,)i(dest,)g(comm)n (,)11 b(returnval\))117 2087 y Fh(IN)155 b Fi(op)541 b Fh(read-mo)q (dify-write)12 b(op)q(eration)i(\(handle\))117 2159 y(IN)155 b Fi(invalue)456 b Fh(lo)q(cal)13 b(input)h(\(c)o(hoice\))117 2232 y(INOUT)62 b Fi(ta)o(rgetva)o(r)417 b Fh(target)15 b(v)n(ariable)d(\(c)o (hoice\))117 2305 y(IN)155 b Fi(datat)o(yp)q(e)424 b Fh(datat)o(yp)q(e)14 b(of)g(v)n(ariables)117 2377 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(remote)h(pro)q(cess)i(\(in)o(teger\))117 2450 y(IN)155 b Fi(comm)466 b Fh(comm)o(unicator)11 b(\(handle\))117 2523 y(OUT)108 b Fi(returnval)419 b Fh(lo)q(cal)13 b(output)h(\(c)o(hoice\))75 2647 y Fg(int)23 b(MPI)p 245 2647 15 2 v 17 w(Rmw\(MPI)p 430 2647 V 16 w(Rmw)p 518 2647 V 17 w(op)h(op,)f(void)h(*invalue,)e(void)i (*targetvar,)e(MPI)p 1538 2647 V 17 w(Datatype)393 2704 y(datatype,)h(int)g (dest,)g(MPI)p 945 2704 V 17 w(Comm)h(comm,)f(void)g(*returnval\))964 2828 y Fk(9)p eop %%Page: 10 10 bop 75 49 a Fg(MPI)p 150 49 15 2 v 17 w(RMW\(OP,)23 b(INVALUE,)f(TARGETVAR,)h (DATATYPE,)g(DEST,)g(COMM,)g(RETURNVAL,)f(IERROR\))170 106 y()h(INVALUE,)g(TARGETVAR,)g(RETURNVAL)170 162 y(INTEGER)g(OP,)h (DATATYPE,)e(DEST,)i(COMM,)f(IERROR)166 249 y Fk(Executes)16 b(an)g(atomic)g(read)p 661 249 14 2 v 16 w(mo)q(dify)p 814 249 V 17 w(write)g(op)q(eration)g(on)g(a)g(remote)f(v)m(ariable.)24 b Fg(inputval)15 b Fk(and)75 305 y Fg(returnval)j Fk(are)h(addresses)g(in)h (lo)q(cal)g(memory)l(,)g(and)f Fg(targetvar)f Fk(is)i(the)f(address)g(in)h (the)g(memory)75 362 y(of)e(the)h(remote)f(pro)q(cess)g(sp)q(eci\014ed)j(b)o (y)d Fg(dest)g Fk(and)h Fg(comm)p Fk(.)29 b(Eac)o(h)18 b(of)g(these)h(three)g (lo)q(cations)g(con)o(tain)75 418 y(a)j(v)m(ariable)i(of)d(the)i(t)o(yp)q(e)f (sp)q(eci\014ed)i(b)o(y)f(the)f(basic)h(datat)o(yp)q(e)f Fg(datatype)p Fk(.)39 b(The)23 b(remote)f(v)m(ariable)75 474 y Fg(targetvar)17 b Fk(is)h(up)q(dated)h(as)f(a)f(function)i(of)f(its)g(previous)h(v)m(alue)g (and)f(the)g(v)m(alue)h(of)f Fg(inputval)p Fk(;)g(the)75 531 y(old)e(v)m(alue)g(of)f Fg(targetvar)f Fk(is)i(returned)g(in)g Fg(returnval)p Fk(.)j(The)d(outcome)e(corresp)q(onds)i(to)f(the)g(atomic)75 587 y(execution)h(of)f(the)g(follo)o(wing)h(co)q(de.)75 694 y Fg(retrunval)23 b(=)g(targetvar;)75 750 y(targetvar)g(=)g(targetvar)g(op)h (inputval;)166 856 y Fk(Prede\014ned)16 b(op)q(erations)g(include:)75 963 y Fe(T)l(est&Set:)22 b Fg(a)i(op)f(b)h(=)g(a||b)p Fk(.)41 b(Th)o(us,)24 b(if)f Fg(sendvar)f Fk(is)h(1,)g(then)g Fg(destvar)f Fk(is)h(set)f(to)g(1)g(and)h(the)189 1019 y(previous)14 b(v)m(alue)h(of)e Fg(destvar)g Fk(is)h(returned)g(in)g Fg(sendvar)p Fk(.)19 b(Supp)q(orted)14 b(datat)o(yp)q(es)f(are)g(C)h(in)o(teger)189 1076 y(and)h(Logical)h(\(see)f (Section)h(4.9.2)e(of)h(MPI)g(do)q(cumen)o(t\).)75 1169 y Fe(T)l(est&Or:)22 b Fg(a)h(op)h(b)g(=)f(a|b)p Fk(.)f(Bit)o(wise)17 b(OR,)f(rather)g(than)f(Bo)q (olean)i(OR,)f(th)o(us)g(executing)h(m)o(ultiple)189 1226 y(test&set)11 b(op)q(erations)h(sim)o(ultaneously)l(.)20 b(Datat)o(yp)q(e)10 b(is)i(C)g(in)o(teger,)g(F)l(ortran)e(in)o(teger,)i(Byte.)19 b(Can)189 1282 y(b)q(e)d(extended)g(to)e Fg(Test&And,)23 b(Test&Xor)p Fk(.)75 1376 y Fe(F)l(etc)o(h&Add:)e Fg(a)j(op)f(b)h(=)g(a+b)p Fk(.)19 b(C)c(in)o(teger,)g(F)l(ortran)f(in)o(teger,)h(Floating)h(p)q(oin)o (t,)f(Complex.)75 1470 y Fe(Compare&Sw)o(ap:)21 b Fg(a)j(op)f(\(b1,b2\))g(=)h (\(a==b1\)?)47 b(b2:a)p Fk(.)18 b(C)13 b(in)o(teger,)g(F)l(ortran)e(in)o (teger)i(\(includ-)189 1526 y(ing)19 b Fg(MPI)p 343 1526 15 2 v 17 w(Aint)p Fk(\).)30 b(Note)19 b(that,)f(for)h(this)g(op)q(eration,)h (one)f(pro)o(vides)g(t)o(w)o(o)f(lo)q(cal)i(v)m(alues,)g(stored)189 1583 y(consecutiv)o(ely)l(.)26 b(Also,)17 b(it)g(migh)o(t)f(b)q(e)h(more)g (con)o(v)o(enien)o(t)g(to)f(return)g(T)l(rue)h(if)g(sw)o(ap)f(succeeeded,)189 1639 y(F)l(alse,)f(if)g(it)h(failed,)g(rather)f(than)g(the)g(remote)g(v)m (alue.)166 1746 y(The)h(calls)h(is)f(blo)q(c)o(king,)h(i.e.,)f(con)o(trol)g (do)q(es)g(not)f(return)h(to)g(the)g(caller)h(till)g(the)f(op)q(eration)g (com-)75 1802 y(pletes.)j(Note)11 b(that)f(unlik)o(e)j(the)e(data)g(transfer) f(calls,)j(the)e(RMW)g(calls)h(do)f(not)g(pro)o(vide)g(an)o(y)g(indication)75 1858 y(of)k(completion)h(on)f(the)h(target)d(pro)q(cess.)166 1915 y(It)e(is)h(erroneous)g(to)f(directly)h(access)g(v)m(ariables)h(that)e (ma)o(y)f(b)q(e)j(accessed)f(using)g(RMW)f(op)q(erations.)75 1971 y(These)16 b(v)m(ariable)h(should)g(only)f(b)q(e)g(accessed)g(using)h (RMW)e(calls,)i(ev)o(en)f(if)g(the)g(v)m(ariables)g(are)g(lo)q(cal)h(to)75 2028 y(a)i(pro)q(cess.)33 b(In)20 b(other)f(w)o(ords,)g(the)g(target)g(pro)q (cess)g(ma)o(y)g(not)g(directly)i(load)e(or)g(store)g(in)o(to)g(RMW)75 2084 y(v)m(ariables.)i(Also,)14 b Fc(get)f Fk(or)g Fc(put)h Fk(calls)h(ma)o(y)e(not)g(access)h(the)g(RMW)f(v)m(ariables.)21 b(If)13 b(these)h(restrictions)g(are)75 2141 y(violated,)i(the)f(RMW)g(op)q (erations)g(are)g(not)g(guaran)o(teed)g(to)f(b)q(e)i(atomic.)166 2280 y Ff(Discussion:)35 b Fh(W)m(e)13 b(ma)o(y)f(w)o(an)o(t)i(to)g(allo)o(w) e(user-de\014ned)17 b(sync)o(hronization)d(op)q(erations,)f(similar)f(to)i (user-)75 2336 y(de\014ned)h(collectiv)o(e)f(reduce)i(op)q(erations.)j(E.g.,) 13 b(w)o(e)i(ma)o(y)d(w)o(an)o(t)i(to)g(de\014ne)h(a)f(Compare&Sw)o(ap)f(op)q (eration)h(that)75 2393 y(tak)o(es)f(a)e(pair)h(of)g(op)q(erands,)h Fb(\(rank,)20 b(address\))p Fh(,)11 b(and)h(use)h(this)f(op)q(eration)g(to)g (manipulate)e(distributed)j(link)o(ed)75 2449 y(lists.)952 2828 y Fk(10)p eop %%Page: 11 11 bop 75 49 a Fi(MPI)p 160 49 14 2 v 16 w(IRMW\(op,)14 b(invalue,)i(ta)o (rgetva)o(r,)d(datat)o(yp)q(e,)k(dest,)f(comm)m(,)c(returnval,)j(request\)) 117 126 y Fh(IN)155 b Fi(op)541 b Fh(read-mo)q(dify-write)12 b(op)q(eration)i(\(handle\))117 202 y(IN)155 b Fi(invalue)456 b Fh(lo)q(cal)13 b(input)h(\(c)o(hoice\))117 277 y(INOUT)62 b Fi(ta)o(rgetva)o(r)417 b Fh(target)15 b(v)n(ariable)d(\(c)o(hoice\))117 352 y(IN)155 b Fi(datat)o(yp)q(e)424 b Fh(datat)o(yp)q(e)14 b(of)g(v)n(ariables)117 427 y(IN)155 b Fi(dest)511 b Fh(rank)14 b(of)f(remote)h(pro)q(cess)i(\(in)o(teger\))117 502 y(IN)155 b Fi(comm)466 b Fh(comm)o(unicator)11 b(\(handle\))117 577 y(OUT)108 b Fi(returnval)419 b Fh(lo)q(cal)13 b(output)h(\(c)o(hoice\))117 652 y(OUT)108 b Fi(request)452 b Fh(RMW)13 b(comm)o(unication)e(request)k (\(handle\))75 777 y Fg(int)23 b(MPI)p 245 777 15 2 v 17 w(Rmw\(MPI)p 430 777 V 16 w(Rmw)p 518 777 V 17 w(op)h(op,)f(void)h(*invalue,)e(void)i (*targetvar,)e(MPI)p 1538 777 V 17 w(Datatype)393 833 y(datatype,)h(int)g (dest,)g(MPI)p 945 833 V 17 w(Comm)h(comm,)f(void)g(*returnval,)393 890 y(MPI)p 468 890 V 17 w(Request)g(*request\))75 976 y(MPI)p 150 976 V 17 w(RMW\(OP,)g(INVALUE,)f(TARGETVAR,)h(DATATYPE,)g(DEST,)g(COMM,)g (RETURNVAL,)f(REQUEST,)393 1033 y(IERROR\))170 1089 y()h(INVALUE,)g (TARGETVAR,)g(RETURNVAL)170 1146 y(INTEGER)g(OP,)h(DATATYPE,)e(DEST,)i(COMM,) f(REQUEST,)g(IERROR)166 1232 y Fk(The)16 b(corresp)q(onding)g(non-blo)q(c)o (king)h(Read-Mo)q(dify-W)l(rite)g(call.)22 b(It)15 b(has)h(an)f(additional)i (request)75 1289 y(parameter)d(that)h(is)h(used)f(for)g(testing)g(for)g(the)g (completion)h(of)f(the)g(op)q(eration.)75 1432 y Fl(3)59 b(F)n(ence)75 1628 y Fi(MPI)p 160 1628 14 2 v 16 w(FENCE\(comm)m(\))117 1705 y Fh(IN)155 b Fi(comm)466 b Fh(comm)o(unicator)11 b(\(handle\))75 1829 y Fg(int)23 b(MPI)p 245 1829 15 2 v 17 w(fence\(MPI)p 478 1829 V 16 w(Comm)g(comm\))75 1916 y(MPI)p 150 1916 V 17 w(FENCE\(COMM,)f(IERROR\))170 1972 y(INTEGER)h(COMM,)h(IERROR)166 2059 y Fk(F)l(ence)17 b(is)g(a)g(mec)o(hanism)g(to)f(enforce)h(ordering)g(b)q (et)o(w)o(een)g(comm)o(unications.)25 b(All)18 b(put/get)e(calls)75 2115 y(issued)11 b(with)g(the)g(same)f(comm)o(unicator)g(b)q(efore)g(the)h (command)f Fi(MPI)p 1270 2115 14 2 v 16 w(FENCE)g Fk(is)h(issued)h(will)g (complete,)75 2172 y(b)q(oth)22 b(with)f(resp)q(ect)h(to)f(origin)h(and)g (target)e(bu\013ers,)i(b)q(efore)g(an)o(y)f(put/get)g(call)i(issued)f(after)f (the)75 2228 y(fence)15 b(will)h(execute.)k(The)14 b Fi(MPI)p 631 2228 V 16 w(FENCE)g Fk(call)i(itself)f(ma)o(y)e(return)h(immediatly)l(,)i (b)q(efore)e(the)g(preceding)75 2285 y(comm)o(unications)i(ha)o(v)o(e)f (completed.)166 2424 y Ff(Implemen)o(tati)o(on)h(note:)48 b Fh(A)17 b(v)n(alid)e(implemen)o(tation)f(is)i(that)h(the)h(call)e(blo)q(c)o (ks)h(un)o(til)f(all)g(preceding)75 2480 y(put/get)e(calls)g(ha)o(v)o(e)f (completed.)952 2828 y Fk(11)p eop %%Page: 12 12 bop 75 49 a Fl(4)59 b(Receive&Call)19 b(and)g(active)h(messages)75 151 y Fk(Rationale:)75 207 y(In)15 b(man)o(y)g(situations)g(it)g(is)g (desirable)h(to)e(in)o(v)o(ok)o(e,)h(up)q(on)g(message)g(arriv)m(al,)g(a)f (message)h(handler.)20 b(This)75 264 y(can)11 b(b)q(e)g(done)g(in)h(t)o(w)o (o)d(di\013eren)o(t)i(st)o(yles.)18 b(Similarly)13 b(to)d(NX)h Fg(Hreceive)p Fk(,)f(where)h(message)f(is)h(receiv)o(ed)h(and)75 320 y(handler)17 b(is)g(in)o(v)o(ok)o(ed)f(after)g(data)f(is)i(in)g(the)f (receiv)o(e)h(bu\013er,)f(or)g(similar)h(to)f(CMMD)f(activ)o(e)h(message,)75 377 y(where)f(incoming)i(data)d(is)i(passed)g(as)e(argumen)o(t)h(to)f(the)i (handler.)21 b(W)l(e)15 b(presen)o(t)g(b)q(elo)o(w)h(b)q(oth)f(st)o(yles.)75 496 y Fi(Receive&call)75 586 y Fg(MPI_HRECV\()22 b(buf,)i(count,)f(datatype,) f(origin,)h(tag,)h(comm,)f(request,)g(handler\))166 677 y Fk(Beha)o(v)o(es)16 b(iden)o(tically)j(to)c Fg(MPI)p 694 677 15 2 v 17 w(IRECV)p Fk(,)g(except)h(that,)g(after)f(message)h(reception)h(completed)g(the)75 733 y(function)f(indicated)h(b)o(y)e Fg(handler)f Fk(is)i(in)o(v)o(ok)o(ed.)k Fg(handler)14 b Fk(is)i(a)f(p)q(oin)o(ter)h(to)e(a)h(function)h(of)f(t)o(yp)q (e)75 834 y Fg(typedef)23 b(int)g(MPI_recv_handler\()f(void)h(*buf,)g(int)h (count,)f(MPI_Datatype)f(datatype,)242 891 y(int)h(origin,)g(int)h(tag,)f (MPI_Comm,)g(comm,)g(MPI_request)f(*request\))166 992 y Fk(When)15 b(in)o(v)o(ok)o(ed)f(b)o(y)h(MPI,)f(the)g(receiv)o(e)i(handler)f(is)g(passed) g(the)g(argumen)o(ts)e(of)h(the)h(receiv)o(e)g(call.)166 1049 y(Discussion:)75 1105 y(In)h(a)f(single)i(threaded)f(en)o(vironmen)o(t,)f (the)h(handler)g(in)o(v)o(o)q(cation)g(will)h(o)q(ccur)f(as)f(a)h(callbac)o (k)g(executed)75 1162 y(b)o(y)e(the)h(main)g(pro)q(cess.)20 b(In)15 b(a)f(m)o(ultithreaded)h(en)o(vironmen)o(t,)g(or)f(a)g(mac)o(hine)h (with)g(a)f(comm)o(unication)75 1218 y(copro)q(cessor,)f(the)g(handler)h(ma)o (y)f(b)q(e)h(executed)g(b)o(y)f(a)g(separate)f(thread.)19 b(It)14 b(is)f(lik)o(ely)i(that)e(the)g(handler)75 1274 y(has)21 b(to)f(access)h (data)f(structures)g(that)g(are)g(shared)h(with)g(the)g(computing)g(thread.) 37 b(Therefore,)21 b(a)75 1331 y(mec)o(hanism)h(has)f(to)f(b)q(e)i(pro)o (vided)g(for)e(m)o(utual)h(exclusion)i(b)q(et)o(w)o(een)f(the)f(handler)h (and)f(the)h(main)75 1387 y(compute)14 b(thread.)20 b(The)14 b(mec)o(hanisms)h(used)g(for)e(m)o(utual)i(exclusion)h(will)f(b)q(e)g(quite)g (di\013eren)o(t)g(in)g(these)75 1444 y(t)o(w)o(o)9 b(en)o(vironmen)o(ts.)19 b(In)11 b(a)f(single-threaded)i(implemen)o(tation)g(one)f(can)g(use)f(a)h (single)h(lo)q(c)o(k)f(that)e(disable)75 1500 y(handler)19 b(in)o(v)o(o)q(cations)f(in)h(order)e(to)g(protect)g(critical)j(user)e(co)q (de)g(sections,)g(or)g(protect)f(one)h(handler)75 1557 y(from)13 b(another.)19 b(In)c(a)e(m)o(ultithreaded)i(en)o(vironmen)o(t)f(one)g(ma)o(y) f(w)o(an)o(t)g(to)g(use)h(a)g(blo)q(c)o(king)h(m)o(utexes,)f(to)75 1613 y(ha)o(v)o(e)h(lo)q(c)o(ks)i(with)f(\014ner)g(gran)o(ularit)o(y)l(.)21 b(In)c(an)o(y)e(case,)h(I)g(fail)h(to)e(see)h(ho)o(w)f(one)h(can)g(hide)h (from)e(the)h(user)75 1670 y(the)g(di\013erence)h(b)q(et)o(w)o(een)f(these)g (t)o(w)o(o)e(t)o(yp)q(es)i(of)f(implemen)o(tations,)h(without)g(seriously)h (crippling)h(the)75 1726 y(functionalit)o(y)e(of)f(this)h(construct.)j(One)d (c)o(hoice)g(is)g(to)e(decree)i(that)f(the)g(handler)h(is)g(alw)o(a)o(ys)e (executed)75 1783 y(b)o(y)e(the)g(thread)g(that)g(in)o(v)o(ok)o(ed)g(the)g (receiv)o(e&call)j(function,)e(and)f(pro)o(vide)h(a)f(mec)o(hanism)g(for)g (disabling)75 1839 y(handler)i(in)o(v)o(o)q(cations)f(b)o(y)g(that)g(thread)f (during)i(critical)h(sections.)k(Another)13 b(c)o(hoice)h(is)g(to)e(decree)i (that)75 1895 y(the)h(handler)i(alw)o(a)o(ys)d(execute)i(in)g(a)f(separate)f (thread.)20 b(An)o(y)15 b(other)g(c)o(hoice?)75 1952 y(One)f(ma)o(y)e(w)o(an) o(t)f(to)i(restrict)f(the)h(abilit)o(y)h(of)f(handlers)h(to)e(in)o(v)o(ok)o (e)h(MPI)f(calls,)i(so)f(as)f(to)h(a)o(v)o(oid)f(the)h(need)75 2008 y(for)i(m)o(utual)g(exclusion)i(proto)q(cols)e(in)h(a)f(single-threaded) i(MPI)e(implemen)o(tation.)166 2065 y(TBD:)75 2121 y(Do)e(w)o(e)h(need)g(a)f (blo)q(c)o(king)j Fg(MPI)p 607 2121 V 16 w(HRECV)p Fk(?)e(Do)f(w)o(e)g(need)i (a)e(p)q(ersisten)o(t)h Fg(HRECV)f Fk(request?)21 b(Do)13 b(w)o(e)g(need)i (\(as)75 2178 y(in)h(NX\))f(also)g Fg(hsend)p Fk(,)f(and)i Fg(hsendrecv)p Fk(?)75 2234 y(Is)c(an)f(exception)i(during)g(the)e(execution) i(of)e(the)h(handler)g(treated)g(as)f(an)g(MPI)h(comm)o(unication)g(error?)75 2354 y Fi(Active)k(message)75 2444 y Fg(MPI_ARECV\()22 b(origin,)h(tag,)h (comm,)f(datatype,)g(handler_func\))166 2534 y Fk(Beha)o(v)o(es)17 b(iden)o(tically)i(to)d Fg(MPI)p 696 2534 V 17 w(IRECV)p Fk(,)g(except)h (that,)f(the)h(incoming)h(message)e(is)i(passed)f(as)f(ar-)75 2591 y(gumen)o(t)f(to)f(the)i(handler.)21 b Fg(handler)p 725 2591 V 16 w(func)14 b Fk(is)i(a)f(p)q(oin)o(ter)h(to)e(a)h(function)h(of)f(t) o(yp)q(e)g(of)g(the)g(form)166 2647 y Fg(typedef)23 b(int)g(MPI)p 527 2647 V 17 w(recv)p 640 2647 V 17 w(handler\()f(int)i(origin,)f(int)g (tag,)h(MPI)p 1445 2647 V 16 w(Comm)g(comm,)75 2704 y Fc(additional)p 274 2704 14 2 v 17 w(p)n(ar)n(ameter)p 488 2704 V 17 w(list)f Fg(\))952 2828 y Fk(12)p eop %%Page: 13 13 bop 166 49 a Fk(Eac)o(h)17 b(parameter)g(in)i(the)f(additional)h(parameter)e (list)h(should)h(ha)o(v)o(e)f(a)f(basic)h(t)o(yp)q(e.)28 b(The)18 b(t)o(yp)q(e)75 106 y(signature)g(of)g(this)h(list)g(of)f(parameters)f (should)j(matc)o(h)d(the)i(t)o(yp)q(e)f(signature)h(of)f Fg(datatype)p Fk(.)28 b(When)75 162 y(in)o(v)o(ok)o(ed)14 b(b)o(y)g(MPI,)f(the)h(receiv)o (e)h(handler)g(is)g(passed)f(the)g Fg(origin,)23 b(tag)13 b Fk(and)h Fg(comm)f Fk(argumen)o(ts)g(of)h(the)75 219 y(receiv)o(e)j(call.)23 b(In)17 b(addition,)g(the)f(v)m(alue)h(of)e(the)h(argumen)o(ts)g(in)g(the)g (additional)i(parameter)d(list)i(is)f(set)75 275 y(to)d(con)o(tain)g(the)g (message)g(data.)18 b(The)c(t)o(yp)q(e)f(signature)g(of)g(the)g(incoming)h (message)f(should)h(matc)o(h)f(the)75 332 y(t)o(yp)q(e)i(signature)h(of)e Fg(datatype)p Fk(.)166 388 y(Alternativ)o(e:)75 444 y(The)h(handler)i(is)e (of)g(t)o(yp)q(e)75 536 y Fg(typedef)23 b(int)g(MPI_recv_handler\()f(int)h (origin,)g(int)h(tag,)f(MPI_Comm)g(comm,)g(...\))166 627 y Fk(i.e.,)c(the)g(handler)h(has)f(a)f(v)m(ariable)i(parameter)e(list.)32 b(This)19 b(allo)o(ws)g(one)g(to)g(ha)o(v)o(e)f(a)g(\014xed)i(t)o(yp)q(e)75 683 y(de\014nition)j(for)e(the)g(handler,)j(and)d(to)g(c)o(hange)g(the)g(n)o (um)o(b)q(er)h(of)f(parameters)f(from)h(in)o(v)o(o)q(cation)h(to)75 740 y(in)o(v)o(o)q(cation.)166 796 y(Discussion:)75 853 y(The)15 b(alternativ)o(e)h(design)g(do)q(es)g(not)e(w)o(ork)h(in)h(F)l(ortran.)75 909 y(All)g(the)g(commen)o(ts)e(on)i(handlers)g(in)g(the)f(previous)h (paragraph)e(apply)i(here,)g(to)q(o.)166 966 y(TBD:)75 1022 y(Do)f(w)o(e)g(need)h(a)e(blo)q(c)o(king)j(v)o(ersion?)75 1125 y Fg(MPI_ARECV_INIT\()22 b(origin,)h(tag,)g(comm,)g(datatype,)g (handler_func,)f(handle\))166 1228 y Fk(Creates)16 b(a)g(p)q(ersisten)o(t)h (activ)o(e)g(receiv)o(e.)25 b(While,)18 b(a)f(call)g(to)f Fg(MPI)p 1287 1228 15 2 v 17 w(ARECV)g Fk(p)q(osts)g(one)h(receiv)o(e)h(that)75 1284 y(will)13 b(consume)e(one)h(incoming)g(message,)f Fg(MPI)p 866 1284 V 17 w(ARECV)p 1003 1284 V 16 w(INIT)g Fk(p)q(osts)g(an)g(activ)o(e) g(receiv)o(e)h(that)f(will)h(p)q(ersist)75 1340 y(consuming)k(messages)f(un)o (til)h(it)f(is)h(freed)g(b)o(y)f(a)g(call)h(to)f Fg(MPI)p 1119 1340 V 16 w(ARECV)p 1255 1340 V 17 w(FREE\(handle\))p Fk(.)166 1397 y(Note:)24 b(in)19 b(case)f(an)f(incoming)i(message)e(is)i(matc)o(hed)e (b)q(oth)h(b)o(y)g(a)f(regular)h(receiv)o(e)g(call)h(and)f(b)o(y)75 1453 y(a)e(p)q(ersisten)o(t)i(activ)o(e)f(receiv)o(e)g(then)h(the)e(standard) h(do)q(es)g(not)f(sp)q(ecify)j(ho)o(w)d(the)h(race)f(is)i(resolv)o(ed.)25 b(A)75 1510 y(p)q(ossible)14 b(implemen)o(tation)g(of)p 616 1510 V 29 w Fg(ARECV)d Fk(is)i(that)f(a)p 939 1510 V 29 w Fg(HRECV)f Fk(is)i(p)q(osted)g(at)e(the)p 1401 1510 V 30 w Fg(INIT)h Fk(call,)h(and)g (is)g(p)q(osted)75 1566 y(anew)i(after)g(the)g(previously)h(p)q(osted)p 743 1566 V 33 w Fg(HRECV)f Fk(completes.)75 1709 y Fl(5)59 b(MPI)19 b(Handle)h(Extension)75 1811 y Fk(Rationale:)75 1867 y(It)h(is)h(desirable)h(to)d(pro)o(vide)i(a)f(mec)o(hanism)h(for)e(users)i (to)e(extend)i(MPI)f(with)h(new)f(non-blo)q(c)o(king)75 1923 y(op)q(erations.)e(Suc)o(h)14 b(extension)h(mec)o(hanism)e(could)i(b)q(e)f (used)g(to)e(de\014ne)j(non-blo)q(c)o(king)g(collectiv)o(e)g(com-)75 1980 y(m)o(unications,)i(or)e(non-blo)q(c)o(king)j(in)o(terpartition)f(op)q (erations,)f(or)f(...)22 b(The)16 b(k)o(ey)g(issue)h(is)f(to)g(allo)o(w)g (the)75 2036 y(use)f(of)g(the)g(regular)g Fg(MPI)p 512 2036 V 17 w Fa(f)p Fg(WAIT|TEST)p Fa(gf)21 b Fg(|ANY|SOME|ALL)p Fa(g)13 b Fk(calls)j(for)e(w)o(aiting)i(for)e(the)h(completion)75 2093 y(of)k(user-de\014ned)h(non-blo)q(c)o(king)h(op)q(erations.)32 b(This,)20 b(in)g(order)f(to)f(a)o(v)o(oid)h(the)g(need)h(to)e(in)o(tro)q (duce)i(a)75 2149 y(new)d(op)q(eration)h(completion)g(call)g(for)e(eac)o(h)h (t)o(yp)q(e)h(of)e(non-blo)q(c)o(king)j(op)q(eration)e(initiation)i(call;)g (and)75 2206 y(to)c(allo)o(w)g(to)g(w)o(ait)f(for)h(the)g(completion)h(of)f Fa(f)p Fg(ANY|SOME|ALL)p Fa(g)e Fk(op)q(erations)i(in)h(a)f(set)g(of)g(op)q (erations)g(of)75 2262 y(di\013eren)o(t)g(t)o(yp)q(es.)166 2319 y(Prop)q(osal:)75 2375 y(Allo)o(w)h(the)f(creation)g(of)g(requests)g (with)h(user-de\014ned)h(b)q(eha)o(vior.)166 2432 y(The)e(follo)o(wing)h (call)h(de\014nes)f(a)f(new)g(t)o(yp)q(e)g(of)g(\(p)q(ersisten)o(t\))g (request.)75 2534 y Fg(MPI_Create_class)22 b(\(MPI_Class)g(*class,)505 2591 y(MPI_Request_create_func)e(create_func,)505 2647 y (MPI_Request_free_func)h(free_func,)505 2704 y(MPI_request_start_func)44 b(start_func,)952 2828 y Fk(13)p eop %%Page: 14 14 bop 505 49 a Fg(MPI_Request_complete_fu)o(nc)21 b(complete_func)505 106 y(int)i(count,)505 162 y(void)g(*extra_state)f(\))166 250 y Fk(The)16 b(call)i(returns)e(in)h Fg(class)f Fk(a)g(handle)h(to)f(an)g(ob)s (ject)g(that)g(de\014nes)h(this)g(new)f(class.)24 b(The)16 b(four)75 307 y(function)c(argumen)o(ts)e(are)h(callbac)o(k)h(functions)g (that)e(are)h(in)o(v)o(ok)o(ed)g(b)o(y)g(v)m(arious)h(op)q(erations)f(on)g (requests)75 363 y(of)16 b(this)h(class.)24 b(The)17 b Fg(count)e Fk(parameter)h(is)h(an)f(upp)q(er)i(b)q(ound)f(on)g(the)f(n)o(um)o(b)q(er)h (of)f(p)q(ending)i(comm)o(u-)75 419 y(nications)d(that)f(ma)o(y)g(b)q(e)h (asso)q(ciated)f(with)h(suc)o(h)g(request)f(\(the)g(request)h(ob)s(ject)f (will)i(store)d(in)o(ternally)75 476 y(an)k(arra)o(y)e(of)i(requests)f(and)h (an)g(arra)o(y)e(of)i(statuses)f(for)g(these)h(op)q(erations\).)24 b(The)17 b Fg(extra)p 1662 476 15 2 v 17 w(state)f Fk(pa-)75 532 y(rameter)e(pro)o(vides)h(additional)h(information)e(that)g(will)i(b)q(e) f(passed)g(to)f(the)h(callbac)o(k)g(functions)g(when)75 589 y(in)o(v)o(ok)o(ed.)166 645 y(A)g(new)h(request)f(instance)h(of)f(this)g (class)h(is)f(created)h(b)o(y)f(a)g(call)h(to)75 733 y Fg (MPI_Request_init\(class,)21 b(comm,)i(request\))166 821 y Fk(The)17 b(call)h(returns)f(in)h Fg(request)e Fk(a)h(handle)h(to)f(this)g(p) q(ersisten)o(t)h(request.)25 b(The)18 b(call)g(in)o(v)o(ok)o(es)f(the)75 878 y(callbac)o(k)f(function)g Fg(create)p 574 878 V 17 w(func)e Fk(to)h(initialize)j(the)d(request.)20 b(This)c(is)g(a)f(function)h(of)e(the) i(form)75 966 y Fg(typedef)23 b(int)g(MPI_request_create_func\()e(MPI_Comm)i (comm,)242 1022 y(MPI_request)f(*request,)h(int)g(*count,)242 1078 y(MPI_Request)f(*array_of_requests,)g(void)h(*extra_state\);)166 1166 y Fk(This)16 b(callbac)o(k)h(function)f(ma)o(y)f(create)g(p)q(ersisten)o (t)h(comm)o(unication)g(handles)h(and)f(return)f(them)75 1223 y(in)h Fg(array)p 251 1223 V 17 w(of)p 316 1223 V 16 w(requests)e Fk(\(whic)o(h)i(has)f(length)h Fg(count)p Fk(\).)166 1279 y(An)24 b(non-blo)q(c)o(king)h(op)q(eration)f(of)g(the)g(class)g(sp)q(eci\014ed)i(b)o (y)d Fg(class)g Fk(is)i(started)e(b)o(y)g(a)h(call)h(to)75 1336 y Fg(MPI)p 150 1336 V 17 w(START\(request\))8 b Fk(\(or)i Fg(MPI)p 652 1336 V 16 w(STARTALL)p Fk(\).)f(This)i(call)g(in)o(v)o(ok)o(es)f (the)h(callbac)o(k)g(function)g Fg(start)p 1766 1336 V 17 w(func)75 1392 y Fk(to)k(start)f(the)h(request)g(op)q(eration.)20 b(This)c(is)g(a)e (function)j(of)d(t)o(yp)q(e)75 1480 y Fg(typedef)23 b(int)g (MPI_request_start_func\()e(MPI_Comm)i(comm,)290 1537 y(MPI_request)f (*request,)h(int)g(*count,)290 1593 y(MPI_Request)f(*array_of_requests,)f (void)j(*extra_state\);)166 1681 y Fk(The)e(callbac)o(k)h(function)f(ma)o(y)g (start)e(one)i(or)f(more)h(comm)o(unications,)i(and)e(use)g Fg(count)f Fk(and)75 1737 y Fg(array)p 198 1737 V 16 w(or)p 262 1737 V 17 w(requests)g Fk(to)g(return)h(the)g(n)o(um)o(b)q(er)g(of)g (comm)o(unications)g(that)f(w)o(ere)h(started)f(and)h(an)75 1794 y(arra)o(y)14 b(of)h(requests)g(for)f(these)i(comm)o(unications.)166 1850 y(MPI)10 b(in)o(v)o(ok)o(es)g(the)g(callbac)o(k)h(function)g Fg(complete)p 1028 1850 V 17 w(func)e Fk(whenev)o(er)i(a)e(comm)o(unication)i (in)g Fg(array)p 1857 1850 V 17 w(of)p 1922 1850 V 17 w(requests)75 1907 y Fk(has)k(completed.)21 b(This)16 b(callbac)o(k)g(function)g(is)g(of)e (t)o(yp)q(e)75 1995 y Fg(typedef)23 b(int)g(MPI_request_complete_func\(MPI)o (_Comm)e(*comm,)218 2051 y(MPI_request)i(*request,)f(int)i(*count,)218 2108 y(MPI_Request)f(*array_of_requests,)e(MPI_Status)h(*array_of_statuses,) 218 2164 y(int)i(index,)f(void)g(*extra_state,)f(int)i(*flag\);)166 2252 y Fk(The)12 b Fg(index)f Fk(argumen)o(t)g(is)h(the)g(index)h(of)f(a)f (comm)o(unication)h(that)g(has)f(completed.)20 b(The)12 b(callbac)o(k)75 2308 y(function)19 b(ma)o(y)e(start)f(new)i(comm)o(unications.)29 b(It)18 b(returns)f(zero)h(in)h Fg(flag)e Fk(if)h(the)g(class)g(op)q(eration) g(is)75 2365 y(incomplete,)d(and)e(one)g(if)g(it)g(is)h(complete.)20 b(MPI)12 b(will)j(not)e(in)o(v)o(ok)o(e)f(this)i(callbac)o(k)g(function)g(an) o(y)e(further)75 2421 y(once)k(it)f(has)g(returned)h(one)f(in)h Fg(flag)p Fk(.)166 2478 y(The)22 b(class)h(op)q(eration)f(is)g(completed)h(b) o(y)f(a)g(call)h(to)f Fg(MPI)p 1209 2478 V 17 w Fa(f)p Fg(TEST|WAIT)p Fa(gf)f Fg(|ANY|SOME|ALL)p Fa(g)p Fk(.)75 2534 y(This)14 b(call)h(will)g (succeed)f(if)g(the)g(callbac)o(k)g(function)g Fg(request)p 1141 2534 V 17 w(complete)e Fk(has)h(already)h(returned)g Fg(flag)75 2591 y(=)24 b(1)p Fk(.)166 2647 y(The)15 b(class)h(request)f(is)h(deallo)q (cated)g(b)o(y)g(a)e(call)j(to)d Fg(MPI)p 1123 2647 V 17 w(REQUEST)p 1308 2647 V 16 w(FREE)p Fk(.)g(The)i(callbac)o(k)g(function)75 2704 y Fg(free)p 174 2704 V 17 w(func)e Fk(is)i(in)o(v)o(ok)o(ed.)k(This)c (is)g(a)e(function)i(of)f(t)o(yp)q(e)952 2828 y(14)p eop %%Page: 15 15 bop 75 49 a Fg(typedef)23 b(int)g(MPI_request_free_func\()e(MPI_request)i (*request,)815 106 y(void)g(*extra_state\);)166 212 y Fk(The)16 b(callbac)o(k)g(functions)g(should)h(not)e(in)o(v)o(ok)o(e)g(blo)q(c)o(king)i (calls)g(\(blo)q(c)o(king)f(MPI)g(calls)g(or)f(system)75 268 y(calls\).)21 b(They)15 b(should)h(not)f(ha)o(v)o(e)g(side)h(e\013ects.)75 412 y Fl(6)59 b(Non-Blo)r(cking)19 b(Collective)h(Communications)75 513 y Fk(It)j(is)g(prop)q(osed)h(to)e(extend)h(all)h(collectiv)o(e)h(comm)o (unication)e(calls)h(\(except)f Fg(MPI)p 1557 513 15 2 v 17 w(OP)p 1622 513 V 17 w(CREATE)f Fk(and)75 570 y Fg(MPI)p 150 570 V 17 w(OP)p 215 570 V 17 w(FREE)p Fk(\))c(to)g(b)q(e)h(a)o(v)m(ailable)h (in)g(a)e(non-blo)q(c)o(king)j(form.)30 b(F)l(or)18 b(eac)o(h)h(collectiv)o (e)h(comm)o(unication)75 626 y(call)f(with)g(name)f Fg(MPI)p 467 626 V 17 w(xxx)g Fk(there)g(is)h(a)f(corresp)q(onding)h(non-blo)q(c)o (king)h(one,)f(with)g(name)f Fg(MPI)p 1753 626 V 17 w(Ixxx)p Fk(.)75 682 y(Non-blo)q(c)o(king)h(collectiv)o(e)g(comm)o(unication)g(calls)f (ha)o(v)o(e)f(the)h(same)f(syn)o(tax)g(as)g(blo)q(c)o(king)i(ones,)f(with)75 739 y(the)e(addition)g(of)f(a)h Fg(request)e Fk(argumen)o(t)h(in)h(the)g (last)f(p)q(osition.)22 b(Non)o(blo)q(c)o(king)17 b(collectiv)o(e)g(comm)o (uni-)75 795 y(cation)e(calls)h(return)f(a)f(request)h(handle.)21 b(A)15 b(non-blo)q(c)o(king)i(collectiv)o(e)g(comm)o(unication)e(op)q (eration)g(is)75 852 y(completed)g(via)f(calls)h(to)e Fg(MPI)p 593 852 V 17 w Fa(f)p Fg(WAIT|TEST)p Fa(gf)22 b Fg(|ANY|SOME|ALL)p Fa(g)p Fk(,)12 b(as)h(for)g(a)h(non-blo)q(c)o(king)i(p)q(oin)o(t-to-)75 908 y(p)q(oin)o(t)f(comm)o(unication,)f(using)h(as)f(argumen)o(t)f(the)h (request)g(handle)i(returned)e(b)o(y)g(the)g(collectiv)o(e)i(call.)75 965 y(As)11 b(with)h(p)q(oin)o(t-to-p)q(oin)o(t)g(comm)o(unication,)h(blo)q (c)o(king)f(and)g(non-blo)q(c)o(king)h(collectiv)o(e)g(comm)o(unication)75 1021 y(calls)20 b(can)f(b)q(e)h(mixed)f(and)h(matc)o(hed.)30 b(I.e.,)20 b(some)e(of)h(the)g(participan)o(ts)g(in)h(a)f(collectiv)o(e)h (comm)o(uni-)75 1078 y(cation)e(ma)o(y)f(execute)h(a)g(blo)q(c)o(king)h (call,)g(while)g(others)f(execute)g(a)f(non-blo)q(c)o(king)j(call.)29 b(The)18 b(user)f(is)75 1134 y(resp)q(onsible)g(for)d(initiating)i(collectiv) o(e)h(comm)o(unication)e(calls)h(in)g(a)e(consisten)o(t)h(order,)f(follo)o (wing)i(the)75 1191 y(same)f(rules)h(as)f(for)f(initiation)j(of)e(blo)q(c)o (king)i(collectiv)o(e)f(comm)o(unication)g(calls.)166 1247 y(Rationale:)75 1303 y(Collectiv)o(e)24 b(comm)o(unication)f(calls)h(can)f(b) q(e)g(quite)g(exp)q(ensiv)o(e.)44 b(Most)21 b(of)h(the)h(calls)g(require)h (eac)o(h)75 1360 y(pro)q(cess)e(to)f(blo)q(c)o(k)h(un)o(til)h(all)f(pro)q (cesses)g(ha)o(v)o(e)g(reac)o(hed)g(the)f(call)i({)e(resulting)i(in)f(idle)i (time)d(when)75 1416 y(pro)q(cesses)c(are)g(not)f(tigh)o(tly)i(sync)o (hronized.)26 b(The)17 b(use)g(of)g(non-blo)q(c)o(king)i(collectiv)o(e)f (comm)o(unication)75 1473 y(alleviate)j(this)e(problem.)33 b(The)20 b(prop)q(osed)f(de\014nition)i(in)o(tro)q(duces)g(no)e(particular)h (implemen)o(tation)75 1529 y(problems,)d(except)g(that)f(a)g(request)g(ob)s (ject)g(ma)o(y)g(no)o(w)g(b)q(e)h(asso)q(ciated)g(with)g(sev)o(eral)f(p)q (oin)o(t-to-p)q(oin)o(t)75 1586 y(comm)o(unications)j(\(since)h(the)f (completion)h(of)e(a)h(collectiv)o(e)h(comm)o(unication)g(ma)o(y)e(require)i (sev)o(eral)75 1642 y(p)q(oin)o(t-to-p)q(oin)o(t)c(comm)o(unications)g(to)e (complete\).)166 1699 y(Alternativ)o(e:)75 1755 y(Allo)o(w)i(tags)f(in)i (non-blo)q(c)o(king)h(collectiv)o(e)f(comm)o(unications.)23 b(This)17 b(allo)o(ws)f(to)f(restrict)h(the)g(ordering)75 1812 y(rules)g(for)f(initiating)j(collectiv)o(e)f(comm)o(unications.)22 b(On)16 b(the)f(other)h(hand,)f(the)h(non-blo)q(c)o(king)h(collec-)75 1868 y(tiv)o(e)12 b(comm)o(unication)h(calls)g(b)q(ecome)g(more)e(di\013eren) o(t)h(than)g(the)g(blo)q(c)o(king)i(ones)e(and)g(it)g(is)h(not)e(p)q(ossible) 75 1924 y(an)o(ymore)j(to)h(matc)o(h)g(blo)q(c)o(king)h(and)g(non)o(blo)q(c)o (king)g(calls.)166 1981 y(TBD:)75 2037 y(Do)f(w)o(e)g(w)o(an)o(t)f(p)q (ersisten)o(t)h(collectiv)o(e)i(comm)o(unication)f(requests?)75 2094 y(Do)21 b(w)o(e)g(w)o(an)o(t)f(to)h(b)q(e)h(able)g(to)f(cancel)i(a)e (non)o(blo)q(c)o(king)i(collectiv)o(e)g(comm)o(unication?)40 b(\(non-trivial)75 2150 y(endea)o(v)o(or!\))75 2207 y(Do)17 b(w)o(e)h(w)o(an)o(t)e(to)h(b)q(e)i(able)f(to)f(prob)q(e)i(for)e(a)g (collectiv)o(e)j(comm)o(unication?)29 b(This)18 b(will)i(require)e(a)g(new)75 2263 y(t)o(yp)q(e)d(of)g(prob)q(e)h(function.)952 2828 y(15)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From owner-mpi-core@CS.UTK.EDU Fri Apr 21 16:54:00 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA28963; Fri, 21 Apr 1995 16:54:00 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA03473; Fri, 21 Apr 1995 16:53:33 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 16:53:32 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from fslg8.fsl.noaa.gov by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA03466; Fri, 21 Apr 1995 16:53:29 -0400 Received: by fslg8.fsl.noaa.gov (5.57/Ultrix3.0-C) id AA24912; Fri, 21 Apr 95 20:53:08 GMT Received: from macaw.fsl.noaa.gov by bora.fsl.noaa.gov (4.1/SMI-4.1) id AA15978; Fri, 21 Apr 95 20:53:11 GMT Received: by macaw.fsl.noaa.gov (4.1/SMI-4.1) id AA14506; Fri, 21 Apr 95 14:53:22 MDT Date: Fri, 21 Apr 95 14:53:22 MDT From: hender@macaw.fsl.noaa.gov (Tom Henderson) Message-Id: <9504212053.AA14506@macaw.fsl.noaa.gov> To: gdburns%tbag.osc.edu@bora-gw.fsl.noaa.gov Subject: Re: spawning: soft resource requirements Cc: mpi-core%cs.utk.edu@bora-gw.fsl.noaa.gov Bill Saphir's "soft resource requirements" is something we actually do here. > >2. My need for resources is soft, not firm. I can make use > >of 4 nodes if 10 aren't available. The allocation request > >may return 4. I might want to spawn different tasks > >depending on what resources I actually get. [in the > >specific example I'm thinking of, the guy might actually > >want to *compile* (!) and run a different task depending > >on how many nodes he gets - F77 static memory allocation] > > ouch, nice example > > The number of processes passed to the spawn function could be > soft or firm. As I said in the prior message, a few useful > resource hints are totally portable and could be brought within MPI. > > You got me on the recompilation bit - using runtime resource data. > Er, I need to be doing another project right now. Catch you later. However, we avoid the recompilation issues. I'll go into a bit more detail below for those who care to wade through it. We are running a national-scale weather forecast model using "real-time" input data every three hours (primary target machine in this case is Paragon). We must run on 11 or more processors to produce a timely forecast. We will run on more processors if they are available. The code is Fortran. Since we require that the code be portable (so we can run on an SGI Challenge when the Paragon is unavailable), we have to stick closely to the Fortran77 standard (i.e. no extensions for dynamic memory allocation). We have observed that the code runs best when static memory usage is minimized (due to cache effects-- most notable on T3D). However, performance is only reduced slightly if static memory usage increases by 10-20%. So, we don't need a different executable for every conceivable processor topology. Our simple-minded solution at the moment is to maintain several executables for different static sizes. At run time we attempt to run the smallest executable with the largest number of processors. If this fails (due to lack of resources), then we can attempt to run a larger executable with a smaller number of processors. This can continue until we either succeed or decide to run on another machine. Clearly, there's more sophistication to it then this, but the main point is that we don't see any need for compilation at run time for this kind of application. Can MPI help us here? We have not actually used "spawning" to make this work. The "dynamic" behavior for resource allocation and task selection is all handled in scripts, not in source code. The ability to select tasks based on available resources in MPI would allow us to simplify the scripts by moving this stuff into the source code (this would include running on a backup machine for a multi-platform MPI). Is this a win? It is not clear to me at this point. I think Bill's "task farm" example makes a much stronger case for providing explicit control of both "task" and "resource" management in MPI. Fire away! :-) Tom From owner-mpi-core@CS.UTK.EDU Fri Apr 21 17:17:50 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA29525; Fri, 21 Apr 1995 17:17:50 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA05855; Fri, 21 Apr 1995 17:17:46 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 17:17:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from wrangler.lanl.gov by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA05848; Fri, 21 Apr 1995 17:17:38 -0400 Received: from sutra.lanl.gov by wrangler.lanl.gov (4.1/SMI-4.1) id AA24353; Fri, 21 Apr 95 15:17:28 MDT Date: Fri, 21 Apr 95 15:17:28 MDT From: suresh@lanl.gov (Suresh Damodaran-Kamal) Message-Id: <9504212117.AA24353@wrangler.lanl.gov> To: gropp@mcs.anl.gov Subject: Re: Dynamic Process Management Cc: lusk@mcs.anl.gov, mpi-core@CS.UTK.EDU Bill: > | However, I did not see how the equivalents of the following PVM > | functions can be implemented using the proposed scheme. > | > | pvm_reg_rm > | pvm_reg_hoster > | pvm_reg_tasker > | > | These functions are really useful for interfacing debuggers > | and load levellers. > | > These PVM functions are very closely tied to PVM's model which requires a PVM > demon. We are trying to avoid REQUIRING a demon in the MPI model. > > The equivalent functionallity is (roughly) provided by the choice of job > starting and resource management software that the MPI-2 application is linked > with. However, I agree that we don't have a simple interface for this. May be so, because I didn't quite see how this can be done. Note, > however, that this capability may be hard to provide on many platforms (try > doing the equivalent of pvm_reg_tasker on a T3D node or SP2 node with the > HPS/user-space switch). Any suggestions will be welcome. Well, I understand that the equivalent of these functions will be hard on a SINGLE node of a T3D. Doesn't MPI_SPAWN and related functions face the same problems? How do you plan to implement spawn functions without an MPI daemon? (Or, something equivalent to it)? One way to go about defining the equivalents of the previously mentioned PVM functions on an MPP is to assume that that MPI_REG_TASKER simply calls the existing job scheduler. In other words, defining such a function does not require an implementation that will COMPLETELY override the existing job scheduler. (Assumption is that MPI_REG_TASKER will be called from an MPI process running in the front end.) Does this sound like a feasible approach? Thanks, -Suresh From owner-mpi-core@CS.UTK.EDU Fri Apr 21 19:14:43 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA01098; Fri, 21 Apr 1995 19:14:42 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA13294; Fri, 21 Apr 1995 19:14:25 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 19:14:24 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from fermi.pnl.gov by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA13287; Fri, 21 Apr 1995 19:14:22 -0400 Received: by fermi.pnl.gov (4.1/SMI-4.1) id AA03672; Fri, 21 Apr 95 16:13:35 PDT Message-Id: <9504212313.AA03672@fermi.pnl.gov> From: j_nieplocha@pnl.gov To: mpi-core@CS.UTK.EDU Cc: j_nieplocha@CS.UTK.EDU Subject: Re: signle-sided accesses. (Better late than later) Date: Fri, 21 Apr 1995 16:13:35 -0700 Sender: d3h325@fermi.pnl.gov I do not understand the motivation for target_request argument in Marc's proposal for MPI_IPUT, MPI_PUT, MPI_ISPUT and their GET counterparts. This argument is supposed to be used as a flag by the target process to verify if a put/get operation was performed into/from the target process's address space. I wonder if the need for such a flag was established based on the real application requirements, some existing and relevant communication model, or implementation issues. Presumably, the goal for one-sided communication with put/get in MPI is to go beyond the message-passing model by providing a mechanism that decouples (removes implied synchronization between) sender and receiver of the data. The flag to be tested for completion of put/get operation at the target still implies some coupling between sender and receiver and appears to be inherited from the message-passing model. If an application needs to do testing of completion flags both at source and target, it seems to me that the application should be using message-passing communication instead of the put/get model. Could we look at some examples that illustrate the usefulness of the target_request flag ? BTW, naming of target and origin arguments is a bit confusing with respect to direction of data transfer in description of the MPI_GET set of operations. For example, in description of MPI_IGET we find on page 7: "Data are copied from the target process to the origin." Could we change it so that the data is copied from the origin (source) to target (destination) ? Another comment that I have deals with the proposed MPI_HRECV function and the suggestion that ability of handlers to invoke MPI calls could be restricted. Specificaly, disallowing point-to point communication calls the handler (for example to respond to the process that sent the receive&call message that invoked the handler) will seriously limit value of this operation for some applications. On the other hand, I am unaware of applications that invoke collective communication operations in the handler. --Jarek From owner-mpi-core@CS.UTK.EDU Fri Apr 21 20:15:21 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA01468; Fri, 21 Apr 1995 20:15:21 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id UAA15835; Fri, 21 Apr 1995 20:14:56 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 21 Apr 1995 20:14:54 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from sgi.sgi.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id UAA15828; Fri, 21 Apr 1995 20:14:52 -0400 Received: from cthulhu.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) for <@sgi.sgi.com:mpi-core@CS.UTK.EDU> id RAA14324; Fri, 21 Apr 1995 17:14:49 -0700 Received: from mrjones.engr.sgi.com by cthulhu.engr.sgi.com via ESMTP (950413.SGI.8.6.12/911001.SGI) for <@cthulhu.engr.sgi.com:mpi-core@CS.UTK.EDU> id RAA02061; Fri, 21 Apr 1995 17:14:43 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) for mpi-core@CS.UTK.EDU id RAA15221; Fri, 21 Apr 1995 17:14:32 -0700 From: "Eric Salo" Message-Id: <9504211714.ZM15219@mrjones.engr.sgi.com> Date: Fri, 21 Apr 1995 17:14:28 -0700 In-Reply-To: j_nieplocha@pnl.gov "Re: signle-sided accesses. (Better late than later)" (Apr 21, 4:13pm) References: <9504212313.AA03672@fermi.pnl.gov> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: mpi-core@CS.UTK.EDU Subject: Re: single-sided accesses. (Better late than later) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii >> I do not understand the motivation for target_request argument in Marc's >> proposal for MPI_IPUT, MPI_PUT, MPI_ISPUT and their GET counterparts. This bothers me, too. We need to decide whether the single-sided operations can intelligently fit into a more general request framework, or if they are a special kind of beast. I'm inclined to view them as the latter. I think that the T3D model, where sender and receiver use some other mechanism (such as a barriers) to synchronize gets and puts, might be the better way to go here. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From owner-mpi-core@CS.UTK.EDU Sat Apr 22 10:34:29 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA12758; Sat, 22 Apr 1995 10:34:29 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA11608; Sat, 22 Apr 1995 10:33:11 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 22 Apr 1995 10:33:09 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id KAA11578; Sat, 22 Apr 1995 10:33:08 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA06610 for ; Sat, 22 Apr 1995 09:33:01 -0500 Message-Id: <199504221433.JAA06610@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: connecting to a running process Date: Sat, 22 Apr 1995 09:33:00 -0500 From: Rusty Lusk Did everyone get this, since it was sent to mpi-comments? I will forward his original note as well. Rusty ------- Forwarded Message From: Gary Oberbrunner To: mpi-maint@antares.mcs.anl.gov Cc: mpi-comments@cs.utk.edu, mpi-maint Subject: Re: [MPI #411] Re: connecting p4 and/or mpich to a running process Hi, MPI folks. I was just wondering (since I haven't heard anything at all) if my suggestion for a rendezvous protocol for MPI processes made it out to this list, and if so are there any comments on it -- does it make sense, is there an alternative plan I should know about, etc.? I won't re-enclose my entire original mail, but here's an extract of the significant parts: [background: I want to take some already-running non-MPI processes with which I can already communicate by some other means and incorporate them into an MPI group. -- gary o] ... I think what you want to end up with is the master process somehow knows a more generic triple for each to-be slave: (hostname, rendezvous-info, rendezvous-type) where rendezvous-info could be a port number, a Unix-domain socket filename, a PID, a PID/thread-id combo, or any well-known name by which the other process can be found, and rendezvous-type identifies the type of rendezvous information in rendezvous-info.... ... Perhaps this [user-supplied] do_rendezvous function could be genericised in the following way: it takes a buffer of "stuff" which it knows nothing about, passes it to the remote process (via whatever mechanism it wants, *not* MPI), which will pass that buffer (locally) to an MPI slave init function. That remote MPI slave init function gives another similar buffer back to the remote process, which is responsible for getting that buffer back to the local process (again using whatever comm mechanism it has). At that point the two processes now know about each other, and MPI can take over. Presumably these buffers can be kept relatively short, in case the alternative comm channel is low bandwidth. ... =========== thanks, - -- Gary Oberbrunner garyo@avs.com Advanced Visual Systems, Inc. http://www.avs.com/~garyo 300 Fifth Avenue (617)890-8192 x2133 TEL Waltham, MA 02154 (617)890-2887 FAX ------- End of Forwarded Message From owner-mpi-core@CS.UTK.EDU Sat Apr 22 10:43:10 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA12772; Sat, 22 Apr 1995 10:43:10 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA12299; Sat, 22 Apr 1995 10:43:21 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 22 Apr 1995 10:43:20 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id KAA12292; Sat, 22 Apr 1995 10:43:17 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA06667 for ; Sat, 22 Apr 1995 09:43:15 -0500 Message-Id: <199504221443.JAA06667@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: original note from Gary O. Date: Sat, 22 Apr 1995 09:43:13 -0500 From: Rusty Lusk To: Rusty Lusk Cc: mpi-maint@antares.mcs.anl.gov, mpi-maint Subject: [MPI #411] Re: connecting p4 and/or mpich to a running process [Feel free to forward this to an appropriate list if you'd like, or if you think it might be interesting to a wider audience -- I'm new to MPI and don't know all the right places yet.] I have the "Dynamic Process Management in and MPI Setting" document, which I think describes the functions you're talking about. Having read that, I am still not quite sure that what I need is in there. There's still no way for an *existing* process to declare itself to be an MPI process (or group), right? In other words, I don't need an interface to the job scheduler *nor* an interface to the process manager (at least not for process startup; just for managing it later). Maybe what I need is a split-phase MPI_ATTACH, where I just want to be able to call explicitly the part that's invoked by the new process after it's running. I.e. I don't want/need to specify an executable, nor cmd-line args (though I *do* have access to the other processes via sockets and if MPI is passing rank info or socket port numbers etc. through command line args I could get that info another way.) Where can I look for more info on how MPI_INIT works, i.e. how the processes rendezvous and so on? I was thinking that I could just call MPI_Init independently in each of my processes (creating in each a separate group) and then create an intercommunicator by merging pairwise combinations of those processes until they're all interwoven into the same intra-communicator. But that doesn't work because you can't create an intercommunicator without having the two groups connected by MPI in the first place. Hmm. PS: thanks for subscribing me to the mpich mailing list. That should be useful. I intend to become a heavy user of MPI; I will be writing libraries and basing commercial code on it in the next couple of years. Gary Oberbrunner garyo@avs.com Advanced Visual Systems, Inc. http://www.avs.com/~garyo 300 Fifth Avenue (617)890-8192 x2133 TEL Waltham, MA 02154 (617)890-2887 FAX ______________________________________________________________________ [background: I want to take some already-running non-MPI processes with which I can already communicate by some other means and incorporate them into an MPI group. -- gary o] > Date: Wed, 29 Mar 1995 10:17:46 -0600 > From: Rusty Lusk > > The incorporation of non-MPI processes into an existing MPI > process's communicator is supposed to be handled by the > MPI_Remote_attach, but you have made me want to go back and think > about it some more, since you have indeed brought up a situation we > weren't thinking about explicitly. If I might be so bold as to make a suggestion: what you need is a more generic rendezvous request. Your current process request (is there going to be a defined constant returned by MPI_REQUEST_TYPE for this type of request so I can call it something?) format is basically a triple of (node, exec-name, arg-array). These are the only "non-MPI" data elements you currently deal with. I think what you want to end up with is the master process somehow knows a more generic triple for each to-be slave: (hostname, rendezvous-info, rendezvous-type) where rendezvous-info could be a port number, a Unix-domain socket filename, a PID, a PID/thread-id combo, or any well-known name by which the other process can be found, and rendezvous-type identifies the type of rendezvous information in rendezvous-info. Note that if rendezvous-type is "start new process" then rendezvous-info can be an array of (name, arg, ...), and this subsumes the current MPI_REMOTE_ATTACH interface. Then once the master process has created a bunch of requests and filled in these slots (perhaps completely manually, via local accessors, no interaction with anything MPI-related, especially MPI_IALLOCATE) it calls a version of MPI_REMOTE_ATTACH which, in an implementation-defined way depending on the rendezvous-type, attempts to rendezvous with the running remote process(es) (e.g. by starting the handshake using the given rendezvous port). One implementation of rendezvous-type could just be a function pointer: rendezvous-type->do_rendezvous(hostname, rdv-info) would actually start the handshake using the hostname and rdv-info. Then once the two processes know enough basic MPI (or p4, whatever) info about each other this function returns, passing control to the MPI handshake start code. Perhaps this do_rendezvous function could be genericised in the following way: it takes a buffer of "stuff" which it knows nothing about, passes it to the remote process (via whatever mechanism it wants, *not* MPI), which will pass that buffer (locally) to an MPI slave init function. That remote MPI slave init function gives another similar buffer back to the remote process, which is responsible for getting that buffer back to the local process (again using whatver comm mechanism it has). At that point the two processes now know about each other, and MPI can take over. Presumably these buffers can be kept relatively short, in case the alternative comm channel is low bandwidth. There might want to be another rendezvous_done function that could be used to close down the non-MPI communication channels once MPI is in control. Not sure whether that would be necessary. Does this make sense? Anyway, the gist of my proposal is just this: allow a generic rendezvous between a group and one or more other "alien" processes which are described by (hostname, rendezvous-info, rendezvous-type). Don't assume that MPI can be the one to start those processes. Somehow expose to the user as much of the handshake mechanism as is necessary to start MPI communications (but no more). Thanks very much, and feel free to ignore these suggestions from an MPI novice (though after five years at TMC, I did pick up a *bit* of parallel programming intuition...) -- Gary Oberbrunner garyo@avs.com Advanced Visual Systems, Inc. http://www.avs.com/~garyo 300 Fifth Avenue (617)890-8192 x2133 TEL Waltham, MA 02154 (617)890-2887 FAX =========================================================================== From owner-mpi-core@CS.UTK.EDU Sat Apr 22 12:07:49 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA13511; Sat, 22 Apr 1995 12:07:49 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA18159; Sat, 22 Apr 1995 12:07:52 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 22 Apr 1995 12:07:50 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from mobydick.usfca.edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id MAA18152; Sat, 22 Apr 1995 12:07:49 -0400 Received: (from peter@localhost) by mobydick.usfca.edu (8.6.9/8.6.9) id JAA02166; Sat, 22 Apr 1995 09:09:51 -0700 Date: Sat, 22 Apr 1995 09:09:51 -0700 From: Peter Pacheco Message-Id: <199504221609.JAA02166@mobydick.usfca.edu> To: jim@meiko.co.uk, mpi-core@CS.UTK.EDU Subject: Re: Access to stdio Jim writes > He [meaning me] then goes on to propose that we REQUIRE acces to stdin > and stdout. > > I think that this is a BAD idea for the following reasons > 1) If I/O is available, then stdin and stdout are also necessarily > available if the language supports them. ("language standard I/O > facilities" are available). Leslie also referred to this -- the predefined attribute MPI_IO. What I'm objecting to is implementations that return MPI_PROC_NULL for this. My proposal was intended to get these implementations to provide limited access to I/O without insisting that they provide full language standard I/O. > 2) stdin and stdout are properties of an operating system and > environment, which not all OSes support. We should not be trying > to enforce a Unix-centric view of the world in MPI. If "stdin" and "stdout" are Unix-centric, the proposal can be rewritten so that they aren't mentioned by insisting that "printf", "scanf", etc. be implemented. > 3) we already see MPI being used in embedded style applications (go > talk to Lloyd !) These environments may have no concept of stdin > and stdout. (Where should stdin come from on a lunar orbiter ?) > We don't want to prevent such implementations being MPI conformant. I think this is a red herring. If an implementation will never access stdin and stdout, then they can be defined in any way the implementor wants -- e.g., stdin = 'EOF', stdout = /dev/null. > 4) It is not MPI that fails to provide access to stdin and stdout, it > is poor implementations (sorry if I offended anyone there )-:). I > would expect implementations in which it makes sense (i.e. at least > those in a Unix environment) to provide access to stdin and stdout > somehow. Exactly. The intent of the proposal is to increase the "useability" of MPI by insisting that implementations provide some access. Thanks to Jim and Leslie for their comments! Peter Pacheco From owner-mpi-core@CS.UTK.EDU Mon Apr 24 10:03:16 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA24838; Mon, 24 Apr 1995 10:03:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA29341; Mon, 24 Apr 1995 10:02:42 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 24 Apr 1995 10:02:41 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id KAA29334; Mon, 24 Apr 1995 10:02:39 -0400 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA22342; Mon, 24 Apr 1995 09:02:32 -0500 Message-Id: <199504241402.JAA22342@antares.mcs.anl.gov> To: suresh@lanl.gov (Suresh Damodaran-Kamal) cc: lusk@mcs.anl.gov, mpi-core@CS.UTK.EDU Subject: Re: Dynamic Process Management In-Reply-to: Message from suresh@lanl.gov (Suresh Damodaran-Kamal) of "Fri, 21 Apr 1995 15:17:28 -0600." <9504212117.AA24353@wrangler.lanl.gov> Date: Mon, 24 Apr 1995 09:02:30 -0500 From: William Gropp | Note, | > however, that this capability may be hard to provide on many platforms (try | > doing the equivalent of pvm_reg_tasker on a T3D node or SP2 node with the | > HPS/user-space switch). Any suggestions will be welcome. | | Well, I understand that the equivalent of these functions will be hard | on a SINGLE node of a T3D. Doesn't MPI_SPAWN and related functions face the | same problems? How do you plan to implement spawn functions without an MPI | daemon? (Or, something equivalent to it)? | By not specify a particular mechanism (in particular, by not providing a user-mode hook), we allow a vendor to implement this in any way, including the use of privaleged (e.g. uid 0) calls. Insisting on a fork/exec** interface may make it impossible to provide. This does not mean that more work does not need to be done on this interface, just that the assumptions that the processes are started by fork/exec** must be re-evaluated. | One way to go about defining the equivalents of the previously mentioned | PVM functions on an MPP is to assume that that MPI_REG_TASKER simply | calls the existing job scheduler. In other words, defining such a function | does not require an implementation that will COMPLETELY override the | existing job scheduler. (Assumption is that MPI_REG_TASKER will be called from | an MPI process running in the front end.) | | Does this sound like a feasible approach? I still think that we are talking at cross purposes here. A pvm_reg_tasker tells the PVM demon running on a node that the process that called pvm_reg_tasker should be used to start jobs. MPI has no requirement that there even BE a demon, so a direct analogy won't work. If we back up and say that the "mpi_reg_tasker" is the process that starts a "local" process, then we still need to define what this means (unless we require the fork/exec** form, which implies a process-child relationship to the starter). I see pvm_reg_tasker as a PVM-style solution to some problems, not as the appropriate general-purpose abstraction. I would like to identify the functionality that the pvm_reg_* are being used to implement, and try to find an abstraction that makes sense on a wider variety of platforms. Bill From owner-mpi-core@CS.UTK.EDU Tue Apr 25 12:28:18 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA24697; Tue, 25 Apr 1995 12:28:17 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA01008; Tue, 25 Apr 1995 12:27:51 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 25 Apr 1995 12:27:49 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from epcc.ed.ac.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA00984; Tue, 25 Apr 1995 12:27:32 -0400 Date: Tue, 25 Apr 95 17:27:09 BST Message-Id: <21085.9504251627@subnode.epcc.ed.ac.uk> From: L J Clarke Subject: MPI Amusement To: mpi-core@CS.UTK.EDU Reply-To: lyndon@epcc.ed.ac.uk Hi y'all Hope the current meeting is going well, and I'm sorry that I can;t be there with you. You may recall I took some photos at one of the meetings back in Dallas. At long last (some of) these are now on-line, for your amusement. URL http://www.epcc.ed.ac.uk/~lyndon/image/mpi/ Best Regards Lyndon -----------------------------------------------------------------------------| e||) Lyndon J Clarke Edinburgh Parallel Computing Centre| "You know there | c||c Tel: 0131 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | aint no devil, |- -----| URL: http://www.epcc.ed.ac.uk/~lyndon/ | that's just God ------------------------------------------------------| when he's drunk." -------------------- From owner-mpi-core@CS.UTK.EDU Wed Apr 26 15:09:06 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA25911; Wed, 26 Apr 1995 15:09:06 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA03585; Wed, 26 Apr 1995 15:08:03 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 26 Apr 1995 15:08:02 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA03574; Wed, 26 Apr 1995 15:08:00 -0400 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA16250 for ; Wed, 26 Apr 1995 14:07:53 -0500 Message-Id: <199504261907.OAA16250@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: Minutes of April 24-26 meeting Date: Wed, 26 Apr 1995 14:07:52 -0500 From: William Gropp are at http://www.mcs.anl.gov/Projects/mpi/mpi2/mpi2 or by ftp from ftp://info.mcs.anl.gov/pub/mpi/mpi2/apr24.{dvi,ps} . Bill From owner-mpi-core@CS.UTK.EDU Wed Apr 26 15:33:26 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA26410; Wed, 26 Apr 1995 15:33:26 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA05723; Wed, 26 Apr 1995 15:33:32 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 26 Apr 1995 15:33:30 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA05716; Wed, 26 Apr 1995 15:33:26 -0400 Received: from dalek (dalek.mcs.anl.gov [140.221.1.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA16981 for ; Wed, 26 Apr 1995 14:33:23 -0500 From: Ian Foster Date: Wed, 26 Apr 1995 14:33:22 -0500 Message-Id: <199504261933.OAA03878@dalek> To: mpi-core@CS.UTK.EDU Subject: Nexus Info The Nexus system represents a set of solutions to some issues being discussed in MPI-2, and so might be of interest to some of you. Nexus supports multiple threads of control, dynamic processor acquisition, dynamic address space creation, a global memory model via interprocessor references, and asynchronous events. It is being used as a compiler target by several parallel language projects. Information is available at: http://www.mcs.anl.gov/nexus/ The paper Nexus: Runtime Support for Task-Parallel Programming Languages provides an overview, and the report Nexus: An Interoperability Layer for Parallel and Distributed Computer Systems describes the interface. Ian. From owner-mpi-core@CS.UTK.EDU Wed Apr 26 16:08:04 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA27015; Wed, 26 Apr 1995 16:08:03 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id QAA08693; Wed, 26 Apr 1995 16:07:45 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 26 Apr 1995 16:07:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id QAA08686; Wed, 26 Apr 1995 16:07:42 -0400 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA18103; Wed, 26 Apr 1995 15:07:34 -0500 Message-Id: <199504262007.PAA18103@antares.mcs.anl.gov> To: knighten@ssd.intel.com (Bob Knighten) cc: mpi-core@CS.UTK.EDU Subject: Re: Minutes of April 24-26 meeting In-Reply-to: Message from Bob Knighten of "Wed, 26 Apr 1995 12:32:58 -0700." <9504261932.AA00357@tualatin.SSD.intel.com> Date: Wed, 26 Apr 1995 15:07:33 -0500 From: William Gropp | William Gropp writes: | > are at http://www.mcs.anl.gov/Projects/mpi/mpi2/mpi2 or by ftp from | > ftp://info.mcs.anl.gov/pub/mpi/mpi2/apr24.{dvi,ps} . | > Bill | Trying that URL produced: | | The requested URL /Projects/mpi/mpi2/mpi2 was not found on this server. Sorry, the URL is http://www.mcs.anl.gov/Projects/mpi/mpi2/mpi2.html Bill From owner-mpi-core@CS.UTK.EDU Thu Apr 27 13:50:45 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA23629; Thu, 27 Apr 1995 13:50:45 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA19568; Thu, 27 Apr 1995 13:49:47 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 27 Apr 1995 13:49:45 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA19560; Thu, 27 Apr 1995 13:49:43 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 0505; Thu, 27 Apr 95 13:49:41 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 0172; Thu, 27 Apr 1995 13:49:41 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Thu, 27 Apr 95 13:49:40 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA23568; Thu, 27 Apr 1995 13:49:38 -0400 Message-Id: <9504271749.AA23568@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: Proposal of Lloyd on multiple error handling Reply-To: snir@watson.ibm.com Date: Thu, 27 Apr 1995 13:49:37 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: One problem we have not covered: Suppose that an error occurred in one of the communications that a WAITALL waits for. It may not be feasible or desireable at this point to wait for the other communications to complete. One will probably want to signal an error as soon as possible, since the communication subsystem may be now inconsistent, or the error may corrupt more state, or... Thus, I take it that when the call returns with an error message some of the requests have not completed successfully, nor have failed, but are just still pending. I suggest that, in such case, the error field of the request should be set to a special value, say MPI_ERR_PENDING (how about MPI_ERR_NOERR?). I can imagine an error handler that tries to execute anew a Wait on these requests, after it cleaned the mess. In fact, MPI_ERR_PENDING, should signify exactly that: the request is "clean", and a new Wait can be posted. Comments? From owner-mpi-core@CS.UTK.EDU Thu Apr 27 14:07:37 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA24104; Thu, 27 Apr 1995 14:07:37 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA20684; Thu, 27 Apr 1995 14:06:58 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 27 Apr 1995 14:06:56 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA20677; Thu, 27 Apr 1995 14:06:54 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 0923; Thu, 27 Apr 95 14:06:53 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 1157; Thu, 27 Apr 1995 14:06:53 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Thu, 27 Apr 95 14:06:52 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA16964; Thu, 27 Apr 1995 14:06:51 -0400 Message-Id: <9504271806.AA16964@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: P.S. multiple errors Reply-To: snir@watson.ibm.com Date: Thu, 27 Apr 1995 14:06:51 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: My previous message applies only to MPI_WAIT_ALL or MPI_TEST_ALL. In the case of MPI_WAIT_SOME or MPI_TEST_SOME one should assume that, when a communication fails, the call will signal completion and return information only about those requests that failed or succeeded. From owner-mpi-core@CS.UTK.EDU Thu Apr 27 17:06:00 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA27552; Thu, 27 Apr 1995 17:06:00 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA06385; Thu, 27 Apr 1995 17:05:24 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 27 Apr 1995 17:05:21 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA06358; Thu, 27 Apr 1995 17:05:19 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 4535; Thu, 27 Apr 95 17:05:14 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 4980; Thu, 27 Apr 1995 17:05:13 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Thu, 27 Apr 95 17:05:13 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA18764; Thu, 27 Apr 1995 17:05:11 -0400 Message-Id: <9504272105.AA18764@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: error classes Reply-To: snir@watson.ibm.com Date: Thu, 27 Apr 1995 17:05:11 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: Current text still says that vendors can add new error classes (not just new error codes). My recolection from previous meeting is that we agreed that the error classes are a fixed, closed sets, and vendors can add error codes. Am I right? From owner-mpi-core@CS.UTK.EDU Thu Apr 27 19:11:51 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA29783; Thu, 27 Apr 1995 19:11:50 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA15713; Thu, 27 Apr 1995 19:11:05 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 27 Apr 1995 19:11:04 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from igate1.hac.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA15702; Thu, 27 Apr 1995 19:10:59 -0400 Received: from msmail4.hac.com ([192.27.11.35]) by igate1.hac.com (4.1/SMI-4.1) id AA13701; Thu, 27 Apr 95 16:08:32 PDT Message-Id: Date: 27 Apr 1995 16:09:03 -0800 From: "Lewins, Lloyd J" Subject: RE: Proposal of Lloyd on multiple error handling To: "mpi-core" , snir@watson.ibm.com X-Mailer: Mail*Link SMTP-MS 3.0.2 Marc says: One problem we have not covered: Suppose that an error occurred in one of the communications that a WAITALL waits for. It may not be feasible or desireable at this point to wait for the other communications to complete. One will probably want to signal an error as soon as possible, since the communication subsystem may be now inconsistent, or the error may corrupt more state, or... Thus, I take it that when the call returns with an error message some of the requests have not completed successfully, nor have failed, but are just still pending. I suggest that, in such case, the error field of the request should be set to a special value, say MPI_ERR_PENDING (how about MPI_ERR_NOERR?). I can imagine an error handler that tries to execute anew a Wait on these requests, after it cleaned the mess. In fact, MPI_ERR_PENDING, should signify exactly that: the request is "clean", and a new Wait can be posted. Lloyd Replys: This is a case that I noticed late in Wednesdays meeting, and I didn't get time to raise. The simplest approach is to specify that MPI_WAITALL wait until all requests complete. This has the most consistent semantics, for example it doesn't require the addition of a new "not-an-error" error code! I am not too worried about the possibility of further corrupted state - I am sure all our good implementations will guard against such problems. However, I suppose a deadlock could occur if the wait in question was blocked on a send/receive pair. If the send failed, and the message expected by the receive is only generated by a remote process after it receives the send, then the wait will never return. The case of MPI_TESTALL is slightly different. After all, the function will always return, and thus an error code could be signaled. However, I am assuming that MPI_TESTALL only returns a flag value of true when all the requests complete. Application code might wait for this condition before testing for errors. The question becomes: does MPI_TESTALL only return an error code, which is raised by a request, when it returns a flag value of true. Obviously, MPI_TESTALL and MPI_WAIT_ALL should return error codes raised by erroneous arguments immediately (with flag false), so therefore we would have to distinguish between two different types of error, those returned immediately, and those which block. This is something I would like to avoid. Thus I lean towards Marcs previous suggestion. Lloyd Lewins Hughes Aircraft Co, llewins@msmail4.hac.com From owner-mpi-core@CS.UTK.EDU Thu Apr 27 19:16:36 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA29820; Thu, 27 Apr 1995 19:16:36 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id TAA16007; Thu, 27 Apr 1995 19:16:34 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 27 Apr 1995 19:16:32 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id TAA15999; Thu, 27 Apr 1995 19:16:31 -0400 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id TAA24298 for ; Thu, 27 Apr 1995 19:16:29 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id TAA19240 for ; Thu, 27 Apr 1995 19:16:29 -0400 Date: Thu, 27 Apr 1995 19:16:29 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199504272316.TAA19240@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA29493; Thu, 27 Apr 95 19:16:28 EDT To: mpi-core@CS.UTK.EDU Subject: preparing MPI-2 document I have created the files for the MPI-2 document. I envision the process working as follows: - The chapter chair gets a copy of the files from ftp://ftp.super.org/pub/mpi/mpi2/Editor/mpi2.tar.Z (anonymous ftp to ftp.super.org, cd pub/mpi/mpi2/Editor, binary, get mpi2.tar.Z). The individual files are also in this directory. - The chapter chair coordinates writing/updating the chapter. He/She takes care of making sure that edits are done in a coordinated way to avoid multiple versions of the same chapter. - By one week before the meeting, the chapter chair should mail the updated tex file and any other modified files (such as new figures) to me (lederman@super.org). Missing this deadline may mean that your updated chapter is not included in the material that is distributed. Once it is mailed to me no other changes should be made since I may need to make some changes to integrate all the chapters. If you find that changes are necessary, then you must contact me first to get the latest copy and permission. - I will put the new version into the working copy of the document. Once it is all assembled I will send a PostScript copy to mpi-core@cs.utk.edu and update the version on the ftp server. - Rusty plans to get the whole document reproduced so that copies are available at the start of each meeting. The scripts have been tested and will hopefully work for everyone. As always, let me know any problems you have. Steve From owner-mpi-core@CS.UTK.EDU Thu Apr 27 21:44:53 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA01135; Thu, 27 Apr 1995 21:44:53 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id VAA25039; Thu, 27 Apr 1995 21:44:19 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 27 Apr 1995 21:44:15 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Phoenix.ERC.MsState.Edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id VAA25030; Thu, 27 Apr 1995 21:44:07 -0400 Received: from python.ERC.MsState.Edu (doss@Python.ERC.MsState.Edu [192.208.142.42]); by Phoenix.ERC.MsState.Edu using ESMTP (8.6.8.1/7.0m-FWP-MsState); id UAA23734; Thu, 27 Apr 1995 20:44:14 -0500 From: "Nathan E. Doss" Received: by python.ERC.MsState.Edu (8.6.8.1/6.0c-FWP); id UAA00540; Thu, 27 Apr 1995 20:44:01 -0500 Message-Id: <199504280144.UAA00540@python.ERC.MsState.Edu> Subject: Re: Proposal of Lloyd on multiple error handling To: llewins@msmail4.hac.com (Lewins Lloyd J) Date: Thu, 27 Apr 1995 20:44:00 -0500 (CDT) Cc: mpi-core@CS.UTK.EDU In-Reply-To: from "Lewins, Lloyd J" at Apr 27, 95 04:09:03 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit The proposal for status error codes that we agreed on is worded something like: "If the MPI function returns an error code other than MPI_ERR_IN_STATUS, it is guaranteed that the function does not modify the value of the error code field in the status argument." I don't believe that we want the user to directly manipulate the error code in status. "Guaranteeing" that it is not touched is an open invitation to the user to assign values to it. I would prefer: "If the MPI function returns an error code other than MPI_ERR_IN_STATUS, the value of the error code field in the status argument is undefined." -- Nathan Doss doss@ERC.MsState.Edu From owner-mpi-core@CS.UTK.EDU Thu Apr 27 22:00:35 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA01229; Thu, 27 Apr 1995 22:00:34 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id WAA26214; Thu, 27 Apr 1995 22:00:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 27 Apr 1995 22:00:05 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from sgi.sgi.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id VAA26179; Thu, 27 Apr 1995 22:00:00 -0400 Received: from cthulhu.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) for <@sgi.sgi.com:mpi-core@CS.UTK.EDU> id SAA20606; Thu, 27 Apr 1995 18:59:57 -0700 Received: from mrjones.engr.sgi.com by cthulhu.engr.sgi.com via ESMTP (950413.SGI.8.6.12/911001.SGI) for <@cthulhu.engr.sgi.com:mpi-core@CS.UTK.EDU> id SAA25530; Thu, 27 Apr 1995 18:59:55 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) for mpi-core@CS.UTK.EDU id SAA23912; Thu, 27 Apr 1995 18:58:47 -0700 From: "Eric Salo" Message-Id: <9504271858.ZM23910@mrjones.engr.sgi.com> Date: Thu, 27 Apr 1995 18:58:43 -0700 In-Reply-To: "Nathan E. Doss" "Re: Proposal of Lloyd on multiple error handling" (Apr 27, 8:44pm) References: <199504280144.UAA00540@python.ERC.MsState.Edu> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: mpi-core@CS.UTK.EDU Subject: Re: Proposal of Lloyd on multiple error handling Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii > I don't believe that we want the user to directly manipulate the > error code in status. "Guaranteeing" that it is not touched is an > open invitation to the user to assign values to it. I would prefer: > > "If the MPI function returns an error code other than MPI_ERR_IN_STATUS, > the value of the error code field in the status argument is undefined." This is a good point. We might also consider simply adding a sentence stating that it is illegal for a user to modify the error code field. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From owner-mpi-core@CS.UTK.EDU Fri Apr 28 00:18:22 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA03394; Fri, 28 Apr 1995 00:18:22 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id AAA06226; Fri, 28 Apr 1995 00:17:45 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 00:17:44 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from win233.nas.nasa.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id AAA06218; Fri, 28 Apr 1995 00:17:41 -0400 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id VAA23837; Thu, 27 Apr 1995 21:17:38 -0700 Date: Thu, 27 Apr 1995 21:17:38 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199504280417.VAA23837@win233.nas.nasa.gov> To: "Eric Salo" Subject: Re: Proposal of Lloyd on multiple error handling Cc: mpi-core@CS.UTK.EDU References: <199504280144.UAA00540@python.ERC.MsState.Edu> Nathan writes: > > I don't believe that we want the user to directly manipulate the > > error code in status. "Guaranteeing" that it is not touched is an > > open invitation to the user to assign values to it. I would prefer: > > > > "If the MPI function returns an error code other than MPI_ERR_IN_STATUS, > > the value of the error code field in the status argument is undefined." > Eric responds: > This is a good point. We might also consider simply adding a sentence stating > that it is illegal for a user to modify the error code field. At the meeting I think it was explicitly discussed the user would set the error code. There are several reasons MPI should explicitly not modify the error code (and the user should be able to set it). 1. if MPI isn't going to set the error code, at least it should behave like something known, like errno - which is set on error and unmodified otherwise (or so I was told at the meeting). 2. There are three choices for what to do if there is no error: a. set the error code to MPI_SUCCESS. This was rejected at the meeting. b. not touch the error code at all. This is Lloyd's proposal. c. set the error code to some random value. This is essentially what the latest proposal is. Seen in this light, it's a little bizarre. d. Of course there's always the choice of setting the error code to MPI_NOT_A_TTY no matter what happens :-) 3. The user may want to process the requests using the error code. I can think of a few places it might make sense to do it this way. For instance: ... set error codes to MPI_SUCCESS ... err = MPI_Waitall(requests, statuses) if (err == MPI_SUCCESS || err == MPI_ERR_IN_STATUS) { for (i = 0; i < nrequests; i++) process_status(statuses[i]); } else { ... there was an error in the arguments to MPI_Waitall ... } 4. We've already told the user that MPI calls may not return a self-consistent status object. Now we would be saying we're not going to let the user fix it him/herself. Of course I'm biased in all this, having argued that MPI should always set the error value. :-) It's probably too late for MPI-1, but I think the right solution to the performance problem with setting the error is to allow the user to pass NULL for the status address (I'm not sure about fortran) so that he/she can bypass *all* the extra work in the usual case where the return status is ignored entirely. Bill From owner-mpi-core@CS.UTK.EDU Fri Apr 28 09:42:02 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA17637; Fri, 28 Apr 1995 09:42:01 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id JAA22067; Fri, 28 Apr 1995 09:41:18 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 09:41:15 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id JAA22056; Fri, 28 Apr 1995 09:41:13 -0400 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id JAA27607; Fri, 28 Apr 1995 09:41:03 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id JAA28013; Fri, 28 Apr 1995 09:41:03 -0400 Date: Fri, 28 Apr 1995 09:41:03 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199504281341.JAA28013@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA10149; Fri, 28 Apr 95 09:41:02 EDT To: doss@erc.msstate.edu Cc: llewins@msmail4.hac.com, mpi-core@CS.UTK.EDU In-Reply-To: <199504280144.UAA00540@python.ERC.MsState.Edu> (doss@erc.msstate.edu) Subject: Re: Proposal of Lloyd on multiple error handling My recollection is that we said the value would not be changed so that the user could set it to MPI_SUCCESS at the start and it would stay that way unless an error occured. This would allow the user to be able to look at this field and get the right answer even if MPI didn't set it. I don't feel strongly about this but this point of view does seem to have some merit. Steve From owner-mpi-core@CS.UTK.EDU Fri Apr 28 10:55:39 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA18889; Fri, 28 Apr 1995 10:55:39 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA28955; Fri, 28 Apr 1995 10:55:05 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 10:55:01 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Phoenix.ERC.MsState.Edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id KAA28941; Fri, 28 Apr 1995 10:54:58 -0400 Received: from python.ERC.MsState.Edu (doss@Python.ERC.MsState.Edu [192.208.142.42]); by Phoenix.ERC.MsState.Edu using ESMTP (8.6.8.1/7.0m-FWP-MsState); id JAA08175; Fri, 28 Apr 1995 09:55:06 -0500 From: "Nathan E. Doss" Received: by python.ERC.MsState.Edu (8.6.8.1/6.0c-FWP); id JAA00883; Fri, 28 Apr 1995 09:54:52 -0500 Message-Id: <199504281454.JAA00883@python.ERC.MsState.Edu> Subject: Re: Proposal of Lloyd on multiple error codes To: mpi-core@CS.UTK.EDU Date: Fri, 28 Apr 1995 09:54:50 -0500 (CDT) X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Bill: > > Nathan writes: > > > I would prefer: > > > > > > "If the MPI function returns an error code other than MPI_ERR_IN_STATUS, > > > the value of the error code field in the status argument is undefined." > > > Eric responds: > > This is a good point. We might also consider simply adding a sentence > > stating that it is illegal for a user to modify the error code field. > > At the meeting I think it was explicitly discussed the user would > set the error code. Originally, the proposal was for MPI to set an error code in a status only if there was an error in the corresponding request. Under this proposal, the user had to initialize the error codes in each status to MPI_SUCCESS and be guaranteed that MPI would not touch it unless there was an error. The proposal now says: "The error code field in the status argument is only valid if the MPI function returns the special error code MPI_ERR_IN_STATUS." I read this to say that the user does not have to initialize the error code field as was proposed early on since all status error fields will be set if MPI_ERR_IN_STATUS is returned by the function. > There are several reasons MPI should explicitly not modify the error > code (and the user should be able to set it). > > 1. if MPI isn't going to set the error code, at least it should behave > like something known, like errno - which is set on error and > unmodified otherwise (or so I was told at the meeting). > > 2. There are three choices for what to do if there is no error: > a. set the error code to MPI_SUCCESS. This was rejected at the meeting. > b. not touch the error code at all. This is Lloyd's proposal. > c. set the error code to some random value. This is essentially > what the latest proposal is. Seen in this light, it's a little > bizarre. No, this is not the latest proposal. I have proposed that we simply don't say what an implementation does. > d. Of course there's always the choice of setting the > error code to MPI_NOT_A_TTY no matter what happens :-) > > > 3. The user may want to process the requests using the error > code. I can think of a few places it might make sense to do it > this way. For instance: > > ... set error codes to MPI_SUCCESS ... > err = MPI_Waitall(requests, statuses) > if (err == MPI_SUCCESS || err == MPI_ERR_IN_STATUS) { > for (i = 0; i < nrequests; i++) process_status(statuses[i]); > } else { > ... there was an error in the arguments to MPI_Waitall ... > } > /* not needed -- ... set error codes to MPI_SUCCESS ... */ err = MPI_Waitall(requests, statuses) if (err == MPI_SUCCESS) { for (i=0;i 4. We've already told the user that MPI calls may not return a > self-consistent status object. Now we would be saying we're not > going to let the user fix it him/herself. Not at all. See previous remarks. -- Nathan Doss doss@ERC.MsState.Edu From owner-mpi-core@CS.UTK.EDU Fri Apr 28 12:56:39 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA20804; Fri, 28 Apr 1995 12:56:39 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA10043; Fri, 28 Apr 1995 12:56:25 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 12:56:21 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA10014; Fri, 28 Apr 1995 12:56:18 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 8117; Fri, 28 Apr 95 12:56:10 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 6335; Fri, 28 Apr 1995 12:56:10 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 28 Apr 95 12:56:09 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA21233; Fri, 28 Apr 1995 12:56:08 -0400 Message-Id: <9504281656.AA21233@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: errors Reply-To: snir@watson.ibm.com Date: Fri, 28 Apr 1995 12:56:08 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: The current proposal is: 1. Error field of status only set by {Wait|Test}{Some|All} calls that return MPI_ERR_IN_STATUS. 2. Such call will set the error fields of all statuses returned. 3. In any other situation MPI calls do not change the value of the error field of status. The assumed program logic is: - programmer tests the error code returned by function; - if error code is MPI_ERR_IN_STATUS, then programmer does additional testing on all status error fields. From that perspective, the guarantee that the status error field is not touched by MPI is not essential: the status error field does not report errors that occurred in normal wait or test calls, so that the programmer cannot check this field, instead of checking the return call of the function. A programmer would not check this field unless it was set by a call that returned MPI_ERR_IN_STATUS, and in such case the call would have updated the error codes in statuses. Therefore, if there is an argument that (3) can harm, I am willing to drop (3). While I have no good argument it helps, I have not seen an argument that it harms. From owner-mpi-core@CS.UTK.EDU Fri Apr 28 13:20:23 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA21049; Fri, 28 Apr 1995 13:20:23 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA11800; Fri, 28 Apr 1995 13:19:59 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 13:19:57 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id NAA11793; Fri, 28 Apr 1995 13:19:56 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA01898; Fri, 28 Apr 1995 12:19:53 -0500 Message-Id: <199504281719.MAA01898@antares.mcs.anl.gov> To: snir@watson.ibm.com cc: mpi-core@CS.UTK.EDU Subject: Re: errors In-reply-to: Your message of "Fri, 28 Apr 1995 12:56:08 EDT." <9504281656.AA21233@snir.watson.ibm.com> Date: Fri, 28 Apr 1995 12:19:51 -0500 From: Rusty Lusk | | The current proposal is: | | 1. Error field of status only set by {Wait|Test}{Some|All} calls that return | MPI_ERR_IN_STATUS. | 2. Such call will set the error fields of all statuses returned. | 3. In any other situation MPI calls do not change the value of the error field | of status. | This states it very clearly. | Therefore, if there is an argument that (3) can harm, I am willing to drop | (3). While I have no good argument it helps, I have not seen an argument that | it harms. | I would prefer that (3) be dropped. I am thinking of the logic of the {Wait|Test}{Some|All} calls, which in the presence of (3) would not be allowed to set the error field of *any* of the statuses until it knew for sure that it was going to return MPI_ERR_IN_STATUS. Also, I cannot see any benefit to it. Rusty From owner-mpi-core@CS.UTK.EDU Fri Apr 28 13:34:06 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA21413; Fri, 28 Apr 1995 13:34:06 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA12904; Fri, 28 Apr 1995 13:33:46 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 13:33:45 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA12897; Fri, 28 Apr 1995 13:33:43 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 8783; Fri, 28 Apr 95 13:33:42 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 8962; Fri, 28 Apr 1995 13:33:41 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 28 Apr 95 13:33:32 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA27013; Fri, 28 Apr 1995 13:33:31 -0400 Message-Id: <9504281733.AA27013@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Cc: frankeh@watson.ibm.com Subject: Binding for user defined attribure copy and delete functions Reply-To: snir@watson.ibm.com Date: Fri, 28 Apr 1995 13:33:31 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: The current Fortran binding for these functions (pg 166,167) have them as functions that return an error code, rather than the usual fortran binding which is a procedure with an ierr argument. The "historical" reason for this binding is that, at some point, it was hoped that the same error handler interface could be used both by Fortran and C, and Bill tried to make their parameter lists identical. This is also the reason that all parameters are passed by reference in the C binding. We realized later that one cannot use the same interface on most systems because of different parameter passing conventions. The rule now is that one need to provide Fortran copy and delete functions for Fortran and C copy and delete functions for C. Yet, we have not updated the bindings to reflect this divergence. If we wanted to be consistent with other bindings in MPI we should have typedef int MPI_Copy_function(MPI_Comm oldcomm, int keyval, void *extra_state, void *attribute_val_in, void *attribute_val_out, int *flag) [* deleted in first two arguments] and PROCEDURE COPY_FUNCTION(OLDCOMM, KEYVAL, EXTRA_STATE, ATTRIBUTE_VAL_IN. ATTRIBUTE_VAL_OUT, FLAG, IERR) typedef int MPI_Delete_function(MPI_Comm comm, int keyval, void *attribute_val, void *extra_state) [* deleted for first two arguments] and PROCEDURE DELETE_FUNCTION(COMM, KEYVAL, ATTRIBUTE_VAL, EXTRA_STATE, IERR) I know its late but: 1. We discussed this informally and forgot to bring it to vote 2. It does not break any code (??) Comments? From owner-mpi-core@CS.UTK.EDU Fri Apr 28 13:42:30 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA21590; Fri, 28 Apr 1995 13:42:29 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA13416; Fri, 28 Apr 1995 13:42:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 13:42:07 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from igate1.hac.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA13408; Fri, 28 Apr 1995 13:42:04 -0400 Received: from msmail4.hac.com ([192.27.11.35]) by igate1.hac.com (4.1/SMI-4.1) id AA19633; Fri, 28 Apr 95 10:39:37 PDT Message-Id: Date: 28 Apr 1995 10:39:52 -0800 From: "Lewins, Lloyd J" Subject: RE: errors To: "mpi-core" X-Mailer: Mail*Link SMTP-MS 3.0.2 Given the input from Marc and Rusty, I am happy to drop the requirement that the implementation only touch the error field in the status block if it sets MPI_ERR_IN_STATUS. Lloyd Lewins Hughes Aircraft Co. llewins@msmail4.hac.com From owner-mpi-core@CS.UTK.EDU Fri Apr 28 14:22:14 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA22175; Fri, 28 Apr 1995 14:22:13 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA16601; Fri, 28 Apr 1995 14:21:10 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 14:21:06 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id OAA16583; Fri, 28 Apr 1995 14:21:02 -0400 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id OAA29016 for ; Fri, 28 Apr 1995 14:20:59 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id OAA01024 for ; Fri, 28 Apr 1995 14:21:00 -0400 Date: Fri, 28 Apr 1995 14:21:00 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199504281821.OAA01024@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA13593; Fri, 28 Apr 95 14:20:59 EDT To: mpi-core@CS.UTK.EDU In-Reply-To: <199504281719.MAA01898@antares.mcs.anl.gov> (message from Rusty Lusk on Fri, 28 Apr 1995 12:19:51 -0500) Subject: Re: errors ---------------------------------------------------------------------- Rusty says: I would prefer that (3) be dropped. I am thinking of the logic of the {Wait|Test}{Some|All} calls, which in the presence of (3) would not be allowed to set the error field of *any* of the statuses until it knew for sure that it was going to return MPI_ERR_IN_STATUS. Also, I cannot see any benefit to it. ---------------------------------------------------------------------- I would like to support this point of view. I said before that not touching it would allow the programmer who initialized the value to MPI_SUCCESS to just look at it and ignore the return code. However, Rusty makes the point that this could make it hard on implementations that just set the values as it handles each request. I think this is the correct way to go. Therefore, a programmer must check the return code before looking at this status field. Steve From owner-mpi-core@CS.UTK.EDU Fri Apr 28 15:22:38 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA23447; Fri, 28 Apr 1995 15:22:38 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA21550; Fri, 28 Apr 1995 15:21:38 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 15:21:37 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from win233.nas.nasa.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA21542; Fri, 28 Apr 1995 15:21:35 -0400 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id MAA03586; Fri, 28 Apr 1995 12:21:32 -0700 Date: Fri, 28 Apr 1995 12:21:32 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199504281921.MAA03586@win233.nas.nasa.gov> To: lederman@super.org (Steve Huss-Lederman) Subject: Re: errors Cc: mpi-core@CS.UTK.EDU > ---------------------------------------------------------------------- > Rusty says: > > I would prefer that (3) be dropped. I am thinking of the logic of the > {Wait|Test}{Some|All} calls, which in the presence of (3) would not be allowed > to set the error field of *any* of the statuses until it knew for sure that it > was going to return MPI_ERR_IN_STATUS. Also, I cannot see any benefit to it. > ---------------------------------------------------------------------- > I would like to support this point of view. I said before that not > touching it would allow the programmer who initialized the value to > MPI_SUCCESS to just look at it and ignore the return code. However, > Rusty makes the point that this could make it hard on implementations > that just set the values as it handles each request. I think this is > the correct way to go. Therefore, a programmer must check the return > code before looking at this status field. > > Steve > I think we've lost track of the original justification here, which was that there would be extra overhead (as much as 1 cache miss and a couple of instructions per request) if an implementation were forced to set status.error every single time (whether or not there was an error). The point was that an implementation will ignore this field entirely except in the exceptional case when there's an error. Why are we now worried about implementations that *do* set it every time? According to our earlier reasoning, these are poor implementations. To reiterate the "benefits" of not touching status.error: 1. status would behave like something known - errno. I am unaware of error codes in any other library that are not either a) always set or b) left alone on success. Examples? 2. a valid implementation of the "status.error is undefined on success" would be to set status.error to a random value. 3. if we declare status.error to be undefined on success a status object will almost always be internally inconsistent with no way to make it consistent. I believe Nathan's example of how to get around this is inconsistent with object abstraction. I suspect we are precluding a reasonable C++ binding. Any support for a proposal (possibly mpi-2) to a. set status.error every time (or only on error, but always in a well defined way) b. allow the user to pass MPI_STATUS_NULL for the status to bypass *all* overhead in the usual case (where status is ignored entirely)? Seems to me this allows greater consistency and higher performance than anything currently under discussion. It also makes programming easier. If mpi-2, it can also be backward compatible if we choose to make status.error well defined. Bill From owner-mpi-core@CS.UTK.EDU Fri Apr 28 15:55:53 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA24202; Fri, 28 Apr 1995 15:55:52 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA24279; Fri, 28 Apr 1995 15:55:13 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 15:55:10 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Phoenix.ERC.MsState.Edu by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA24267; Fri, 28 Apr 1995 15:55:09 -0400 Received: from python.ERC.MsState.Edu (doss@Python.ERC.MsState.Edu [192.208.142.42]); by Phoenix.ERC.MsState.Edu using ESMTP (8.6.8.1/7.0m-FWP-MsState); id OAA16235; Fri, 28 Apr 1995 14:55:17 -0500 From: "Nathan E. Doss" Received: by python.ERC.MsState.Edu (8.6.8.1/6.0c-FWP); id OAA01264; Fri, 28 Apr 1995 14:55:04 -0500 Message-Id: <199504281955.OAA01264@python.ERC.MsState.Edu> Subject: Re: errors To: wcs@nas.nasa.gov (William C. Saphir) Date: Fri, 28 Apr 1995 14:55:03 -0500 (CDT) Cc: mpi-core@CS.UTK.EDU In-Reply-To: <199504281921.MAA03586@win233.nas.nasa.gov> from "William C. Saphir" at Apr 28, 95 12:21:32 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit > ---------------------------------------------------------------------- > Rusty says: > > I would prefer that (3) be dropped. I am thinking of the logic of the > {Wait|Test}{Some|All} calls, which in the presence of (3) would not be > allowed to set the error field of *any* of the statuses until it knew > for sure that it was going to return MPI_ERR_IN_STATUS. Also, I > cannot see any benefit to it. > ---------------------------------------------------------------------- > I would like to support this point of view. I said before that not > touching it would allow the programmer who initialized the value to > MPI_SUCCESS to just look at it and ignore the return code. However, > Rusty makes the point that this could make it hard on implementations > that just set the values as it handles each request. I think this is > the correct way to go. Therefore, a programmer must check the return > code before looking at this status field. > > Steve To clarify my point of view. My understanding of what we wanted: a) We don't want the user to set the error code field of status. b) We don't want implementations to set the error code field on success. c) We want to be able to determine which status has an error if MPI_ERR_IN_STATUS is returned. d) We don't want users examining the error field unless MPI_ERR_IN_STATUS is returned. Dropping (3): Strengthens (a) while not huring (b), (c), or (d). Allows simpler implementations of multiple request operations. In no way mandates that the error code field must be changed. Bills example looked something like: > ... set error codes to MPI_SUCCESS ... > err = MPI_Waitall(requests, statuses) > if (err == MPI_SUCCESS || err == MPI_ERR_IN_STATUS) { > for (i = 0; i < nrequests; i++) process_status(statuses[i]); > } else { > ... there was an error in the arguments to MPI_Waitall ... > } It was my understanding that we want to prevent: > ... set error codes to MPI_SUCCESS ... -- Nathan Doss doss@ERC.MsState.Edu From owner-mpi-core@CS.UTK.EDU Fri Apr 28 17:39:17 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA25932; Fri, 28 Apr 1995 17:39:16 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA02329; Fri, 28 Apr 1995 17:37:44 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Fri, 28 Apr 1995 17:37:42 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from SSD.intel.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id RAA02322; Fri, 28 Apr 1995 17:37:37 -0400 Received: from eel.ssd.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA15336; Fri, 28 Apr 95 14:36:19 PDT Date: Fri, 28 Apr 95 14:36:19 PDT Message-Id: <9504282136.AA15336@SSD.intel.com> X-Sender: prp@eudora.ssd.intel.com X-Mailer: Windows Eudora Version 2.0.3 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: wcs@nas.nasa.gov (William C. Saphir), lederman@super.org (Steve Huss-Lederman) From: prp@ssd.intel.com (Paul Pierce) Subject: Re: errors Cc: mpi-core@CS.UTK.EDU At 12:21 PM 4/28/95 -0700, William C. Saphir wrote: >b. allow the user to pass MPI_STATUS_NULL for the status to bypass >*all* overhead in the usual case (where status is ignored entirely)? This is in our experience a very beneficial option. Paul From owner-mpi-core@CS.UTK.EDU Mon May 1 08:40:07 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA15861; Mon, 1 May 1995 08:40:06 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id IAA02241; Mon, 1 May 1995 08:38:46 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 1 May 1995 08:38:45 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id IAA02233; Mon, 1 May 1995 08:38:32 -0400 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA11735 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Mon, 1 May 1995 13:38:28 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA26170; Mon, 1 May 1995 13:38:26 +0000 Date: Mon, 1 May 1995 13:38:26 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9505011238.AA26170@deneb.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: <9504211714.ZM15219@mrjones.engr.sgi.com> (salo@mrjones.engr.sgi.com) Subject: Re: single-sided accesses. (Better late than later) Reply-To: jim@meiko.com > >> I do not understand the motivation for target_request argument in Marc's > >> proposal for MPI_IPUT, MPI_PUT, MPI_ISPUT and their GET counterparts. > > This bothers me, too. We need to decide whether the single-sided operations can > intelligently fit into a more general request framework, or if they are a > special kind of beast. I'm inclined to view them as the latter. > > I think that the T3D model, where sender and receiver use some other mechanism > (such as a barriers) to synchronize gets and puts, might be the better way to > go here. There are many examples where the added possibility of knowing that the data has been accessed/deposited is useful. In particular this style is that encouraged by Split-C (I'm sure that you can find out all you need from Berkeley). The problem with requiring barrier synchronisation is that there are circumstances under which you may not know the set of processes with which to barrier synchronise, but you do know how much data you expect. Since the lowest level hardware primitive in the CS-2 is a remote DMA operation which has the option of setting a completion event at both ends, we have some experience with this model, and find it useful. Basically it allows you to be slightly less tightly coupled than if you have to barrier synchronise. This can lead to performance improvements as a result. However I would recommend that the remote status update be made optional (i.e. don't do it unless the user asks for it), rather than mandatory. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From owner-mpi-core@CS.UTK.EDU Mon May 1 12:54:24 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA19930; Mon, 1 May 1995 12:54:23 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA24470; Mon, 1 May 1995 12:54:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 1 May 1995 12:54:06 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from sgi.sgi.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id MAA24448; Mon, 1 May 1995 12:53:59 -0400 Received: from cthulhu.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) for <@sgi.sgi.com:mpi-core@CS.UTK.EDU> id JAA00590; Mon, 1 May 1995 09:53:57 -0700 Received: from mrjones.engr.sgi.com by cthulhu.engr.sgi.com via ESMTP (950413.SGI.8.6.12/911001.SGI) for <@cthulhu.engr.sgi.com:mpi-core@CS.UTK.EDU> id JAA12142; Mon, 1 May 1995 09:53:56 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) for mpi-core@CS.UTK.EDU id JAA29151; Mon, 1 May 1995 09:53:54 -0700 From: "Eric Salo" Message-Id: <9505010953.ZM29149@mrjones.engr.sgi.com> Date: Mon, 1 May 1995 09:53:50 -0700 In-Reply-To: jim@meiko.co.uk (James Cownie) "Re: single-sided accesses. (Better late than later)" (May 1, 1:38pm) References: <9505011238.AA26170@deneb.co.uk> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: mpi-core@CS.UTK.EDU Subject: Re: single-sided accesses. (Better late than later) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii > However I would recommend that the remote status update be made > optional (i.e. don't do it unless the user asks for it), rather than > mandatory. This was pretty much the consensus that we came to in Chicago. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From owner-mpi-core@CS.UTK.EDU Thu May 4 15:26:00 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA11189; Thu, 4 May 1995 15:26:00 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA09053; Thu, 4 May 1995 15:25:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 4 May 1995 15:25:06 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA09044; Thu, 4 May 1995 15:25:04 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA22877; Thu, 4 May 1995 14:25:01 -0500 Message-Id: <199505041925.OAA22877@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: latest MPI-1 Cc: doss@erc.msstate.edu Date: Thu, 04 May 1995 14:25:00 -0500 From: Rusty Lusk The changes we voted on at the last meeting have been incorporated into a new draft and are available for anonymous ftp on ftp.mcs.anl.gov in the directory pub/mpi/mpi-1.may95. Both the whole thing, in both postscript and as a uuencoded postsript file, and the individual chapters in postscript, are there. Changes from the May 5, 1994 version are marked in the margins. You can also get to this directory through the Web page http://www.mcs.anl.gov/mpi I hope that the chapter authors can read their chapters very carefully soon, and that everyone else can read as much as they would like to, so that we can make any necessary corrections in time to have this printed and voted on for the last time at the next meeting, which is June 5-7. Regards, Rusty From owner-mpi-core@CS.UTK.EDU Mon May 8 10:52:09 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA19464; Mon, 8 May 1995 10:52:09 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA06941; Mon, 8 May 1995 10:44:47 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 8 May 1995 10:44:42 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id KAA06919; Mon, 8 May 1995 10:44:32 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 4653; Mon, 08 May 95 10:43:52 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 2339; Mon, 8 May 1995 10:43:51 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 08 May 95 10:43:49 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA21569; Mon, 8 May 1995 10:43:47 -0400 Message-Id: <9505081443.AA21569@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Cc: frankeh@watson.ibm.com, treumann@kgnvmc.watson.ibm.com Subject: C binding for MPI_Cancel Reply-To: snir@watson.ibm.com Date: Mon, 08 May 1995 10:43:47 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: Currently is MPI_Cancel(MPI_Request *request). This seems to imply that the call may change the request handle. However, this is not allowed, according to the latest version of the document (handles change only when objects are allocated or deallocated, or by MPI_Type_commit). Nor is there a good reason for the handle to change value: the request is deallocated by the subsequent call to MPI_Wait (test,...), not by the call to MPI_Cancel itself. This seems to suggest the the correct binding should be MPI_Cancel(MPI_Request request) Comments? From owner-mpi-core@CS.UTK.EDU Mon May 8 14:57:09 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA08879; Mon, 8 May 1995 14:57:09 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA27068; Mon, 8 May 1995 14:56:31 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 8 May 1995 14:56:30 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA27061; Mon, 8 May 1995 14:56:28 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9761; Mon, 08 May 95 14:54:54 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 5188; Mon, 8 May 1995 14:54:54 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 08 May 95 14:54:53 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA29041; Mon, 8 May 1995 14:54:52 -0400 Message-Id: <9505081854.AA29041@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: C binding of MPI_Start Reply-To: snir@watson.ibm.com Date: Mon, 08 May 1995 14:54:51 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: Currently is MPI_Start(MPI_request request). This function, too is not supposed to modify the request handle, and should logically be MPI_Start(MPI_Request request) From owner-mpi-core@CS.UTK.EDU Mon May 8 15:28:29 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA10315; Mon, 8 May 1995 15:28:29 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA00398; Mon, 8 May 1995 15:28:40 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 8 May 1995 15:28:35 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from super.super.org by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id PAA00391; Mon, 8 May 1995 15:28:33 -0400 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id PAA11059 for ; Mon, 8 May 1995 15:28:21 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id PAA20904 for ; Mon, 8 May 1995 15:28:31 -0400 Date: Mon, 8 May 1995 15:28:31 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199505081928.PAA20904@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA16528; Mon, 8 May 95 15:28:30 EDT To: mpi-core@CS.UTK.EDU Subject: proposed posting for input Attached is a first shot at a posting for input to the external interface subcommittee. It would go to comp.parallel.mpi, comp.parallel, comp.sys.super. Ideas on the content or places to post is welcome. Steve ---------------------------------------------------------------------- The MPIF (Message Passing Interface Forum) has begun to meet again to discuss a second round of the MPI standard. Part of this effort will involve defining external interfaces so that important functionality can be layered on top of MPI. For example, allowing portable parallel debuggers and profilers on top of MPI is a desirable goal. The MPIF is interested in supporting general functions and not functionality that will only be used by a very small group. One type of functionality that needs defining is what opaque objects need to be exposed. For example, looking at internal parts of the MPI communicator. The MPIF is seeking input to help define this functionality. Responses should be sent to the MPIF mailing list for external interfaces (???address??). The MPIF would like to thank you for your time and efforts. Input now is the best and easiest way to have what you want in the next version of the MPI standard. From owner-mpi-core@CS.UTK.EDU Tue May 9 13:32:42 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA04847; Tue, 9 May 1995 13:32:42 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA16054; Tue, 9 May 1995 13:32:08 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 9 May 1995 13:32:06 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id NAA16042; Tue, 9 May 1995 13:32:05 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA18310; Tue, 9 May 1995 12:31:49 -0500 Message-Id: <199505091731.MAA18310@mcs.anl.gov> To: snir@watson.ibm.com cc: mpi-core@CS.UTK.EDU, frankeh@watson.ibm.com, treumann@kgnvmc.watson.ibm.com Subject: Re: C binding for MPI_Cancel In-reply-to: Your message of "Mon, 08 May 1995 10:43:47 EDT." <9505081443.AA21569@snir.watson.ibm.com> Date: Tue, 09 May 1995 12:31:48 -0500 From: Rusty Lusk | :-) :-) :-) *** (-: (-: (-: | | Currently is MPI_Cancel(MPI_Request *request). | | This seems to imply that the call may change the request handle. However, | this is not allowed, according to the latest version of the document (handles | change only when objects are allocated or deallocated, or by MPI_Type_commit). | Nor is there a good reason for the handle to change value: the request is | deallocated by the subsequent call to MPI_Wait (test,...), not by the call to | MPI_Cancel itself. This seems to suggest the the correct binding should be | MPI_Cancel(MPI_Request request) | | Comments? | | I think that your comments are all correct, but I would advise (and vote) against changing it. It is aesthetically displeasing, and I sort of wish that we had done it as MPI_Cancel(MPI_Request request) originally, but it isn't broken, the standard has been out for a year now, and I think that we should leave it the way it is. Rusty From owner-mpi-core@cs.utk.edu Tue May 9 14:39:53 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA05939; Tue, 9 May 1995 14:39:52 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA16724; Tue, 9 May 1995 13:41:22 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 9 May 1995 13:41:21 EDT Errors-to: owner-mpi-core@cs.utk.edu Received: from mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id NAA16713; Tue, 9 May 1995 13:41:18 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA18490; Tue, 9 May 1995 12:41:15 -0500 Message-Id: <199505091741.MAA18490@mcs.anl.gov> To: snir@watson.ibm.com cc: mpi-core@cs.utk.edu Subject: Re: C binding of MPI_Start In-reply-to: Your message of "Mon, 08 May 1995 14:54:51 EDT." <9505081854.AA29041@snir.watson.ibm.com> Date: Tue, 09 May 1995 12:41:14 -0500 From: Rusty Lusk | Currently is MPI_Start(MPI_request request). | This function, too is not supposed to modify the request handle, and should | logically be | | MPI_Start(MPI_Request request) | My comments to the previous note apply: I don't think we should fix it if it isn't actually broken in some way. And while changing MPI_Cancel might not have a major impact on existing codes, changing MPI_Start would have a huge impact on existing codes, documentation, and tutorial materials. Rusty From owner-mpi-core@CS.UTK.EDU Tue May 9 14:56:41 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA06153; Tue, 9 May 1995 14:56:40 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id OAA22541; Tue, 9 May 1995 14:57:10 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 9 May 1995 14:57:09 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from sgi.sgi.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id OAA22534; Tue, 9 May 1995 14:57:03 -0400 Received: from cthulhu.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) for <@sgi.sgi.com:mpi-core@CS.UTK.EDU> id LAA12603; Tue, 9 May 1995 11:57:00 -0700 Received: from mrjones.engr.sgi.com by cthulhu.engr.sgi.com via ESMTP (950413.SGI.8.6.12/911001.SGI) for <@cthulhu.engr.sgi.com:mpi-core@CS.UTK.EDU> id LAA23166; Tue, 9 May 1995 11:56:44 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) for mpi-core@CS.UTK.EDU id LAA17457; Tue, 9 May 1995 11:56:41 -0700 From: "Eric Salo" Message-Id: <9505091156.ZM17455@mrjones.engr.sgi.com> Date: Tue, 9 May 1995 11:56:36 -0700 In-Reply-To: Rusty Lusk "Re: C binding for MPI_Cancel" (May 9, 12:31pm) References: <199505091731.MAA18310@mcs.anl.gov> X-Mailer: Z-Mail (3.2.1 6apr95 MediaMail) To: mpi-core@CS.UTK.EDU Subject: Re: C binding for MPI_Cancel Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii > I think that your comments are all correct, but I would advise (and vote) > against changing it. It is aesthetically displeasing, and I sort of wish that > we had done it as MPI_Cancel(MPI_Request request) originally, but it isn't > broken, the standard has been out for a year now, and I think that we should > leave it the way it is. Are there any MPI routines that actually do take a MPI_Request as an argument instead of a pointer? I was unable to find one in a very cursory search thru the spec. If not, one might argue that at least the current usage is consistent. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From owner-mpi-core@CS.UTK.EDU Wed May 10 08:08:53 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA23864; Wed, 10 May 1995 08:08:53 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id IAA01183; Wed, 10 May 1995 08:08:47 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Wed, 10 May 1995 08:08:46 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from hub by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id IAA01174; Wed, 10 May 1995 08:08:38 -0400 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA08953 (5.65c/IDA-1.4.4 for mpi-core@CS.UTK.EDU); Wed, 10 May 1995 13:08:31 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA03554; Wed, 10 May 1995 13:08:29 +0000 Date: Wed, 10 May 1995 13:08:29 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9505101208.AA03554@deneb.co.uk> To: mpi-core@CS.UTK.EDU In-Reply-To: <199505091731.MAA18310@mcs.anl.gov> (message from Rusty Lusk on Tue, 09 May 1995 12:31:48 -0500) Subject: C bindings I'm with Rusty on this. The arguments for change are technically sensible, but its too late. We simply can't change these now, too many people already have code written the other way, and we just can't afford to break it. This is now a political issue, not a technical one, and I believe it would be a BAD mistake to "tidy" these things now. -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From owner-mpi-core@CS.UTK.EDU Sat May 13 12:14:02 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA08780; Sat, 13 May 1995 12:14:02 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA08516; Sat, 13 May 1995 12:13:19 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Sat, 13 May 1995 12:13:18 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from Aurora.CS.MsState.Edu by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA08509; Sat, 13 May 1995 12:13:17 -0400 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA22113; Sat, 13 May 95 11:13:05 CDT Date: Sat, 13 May 95 11:13:05 CDT From: Tony Skjellum Message-Id: <9505131613.AA22113@Aurora.CS.MsState.Edu> To: hpff-core@cs.rice.edu, mpi-core@CS.UTK.EDU Subject: Gordon Conference Please note the upcoming deadline for registration ****************************************************************************** GORDON RESEARCH CONFERENCE ADVANCE PROGRAM and REGISTRATION CALL FOR PARTICIPATION **** NOTE: YOU MUST REGISTER BY MAY 31 TO ATTEND **** ****************************************************************************** 1995 Gordon Research Conference High-Performance Computing and National Information Infrastructure Dates: July 16-21, 1995 Place: Plymouth State College Plymouth, New Hampshire Additional Information: http://www-pablo.cs.uiuc.edu/Gordon.html The World Wide Web (WWW) site named above will have continuously updated information on the conference. Check it regularly for new announcements and details. ============================================================================== OVERVIEW The Gordon conferences are a group of meetings intended to bring together a group of leading researchers in a specific scientific discipline to discuss cutting edge issues. As such, they are a bit different than the traditional conference. In particular, there are no published proceedings, and the goal is free-wheeling, ``blue sky'' discussion of open problems. Our goal is to discuss the state of High-Performance Computing and Communications (HPCC) and the National Information Infrastructure (NII) --- the technical issues, the politics, and the funding. In short, this is *NOT* your typical conference; it's closer in spirit to a retreat. In keeping with the spirit of blue sky discussion and open-ended debate, the conferences are held on a college campus, with attendee lodging in a college dormitory; this year's meeting is at Plymouth State College. There are morning and evening sessions, with the afternoons free for open-ended discussion. ============================================================================== REGISTRATION All attendees MUST preregister SIX WEEKS prior to the conference. We realize that this is a bit unusual, but we don't make the rules in this case --- the Gordon conference folks do. The registration form is available electronically from the World Wide Web (WWW) at http://www-pablo.cs.uiuc.edu/Gordon/application.ps.Z The completed registration form must be sent directly to the Gordon conference management, not to the conference organizers. ============================================================================== POSTER SESSION On Thursday afternoon of the conference agenda a poster session is scheduled from 4:30 to 6:00pm. If you would like to submit a poster, please contact either of the conference organizers. This is an excellent opportunity to present an early work in progress. ============================================================================== QUESTIONS AND INFORMATION For more information, for general questions, or if you are unable to retrieve the postscript copy of the registration form, contact the conference organizers. Dan Reed (conference chair) Department of Computer Science University of Illinois 1304 West Springfield Avenue Urbana, Illinois 61801 Email: reed@cs.uiuc.edu Telephone: (217) 333-3807 FAX: (217) 333-3501 Tony Skjellum (conference vice-chair) Department of Computer Science Bulter Hall, Room 300 Mississippi State University MS State, Mississippi 39762 Email: tony@Aurora.CS.MsState.Edu Telephone: (601) 325-8435 FAX: (601) 325-8997 ============================================================================== DRAFT PROGRAM An updated version of this program can always be found at http://www-pablo.cs.uiuc.edu/Gordon/agenda.ps.Z Network and Processor Technology: Evolution or Revolution? Monday, July 17, 9:00am to 12:15pm Discussion Leader: Andrew Chien, University of Illinois David Patterson, UC-Berkeley ``Networks of Workstations'' Anant Agarwal, MIT ``Scalable Workstations, or Why Distance Makes the Data Wander'' Chuck Seitz, Myricom ``MsgWay: Internetting MPPs and Clusters'' Exploiting the Information Superhighway Monday, July 17, 7:30pm to 10:00pm Discussion Leader: Rick Stevens, Argonne National Laboratory Charlie Catlett, NCSA ``Metacomputers, Cyberspace, and Other Hackneyed Terms'' Udi Manber, University of Arizona ``Finding Useful Information Through the Internet'' High-Performance Input/Output Tuesday, July 18, 9:00am to 12:15pm Discussion Leader: Joel Saltz, University of Maryland Bill Gropp, Argonne National Laboratory ``Connecting Parallel Files to the Outside World'' David Kotz, Dartmouth College ``A New Structure for Parallel File-System Design'' Garth Gibson, Carnegie-Mellon University ``Network Attached Disks'' (tentative title) Virtual Reality and High-Performance Computing Tuesday, July 18, 7:30pm to 10:00pm Discussion Leader: Dan Reed, University of Illinois Rick Stevens, Argonne National Laboratory ``Integration of High-End Virtual Environments with MPP Systems'' Henry Sowizral, Boeing Computer Services ``VR and High-Performance Computing'' Poster Session Tuesday, July 18, 4:30pm to 6:00pm The Future of Languages and Compilers for High-Performance Computing Wednesday, July 19, 9:00am to 12:15pm Discussion Leader: Carl Kesselman, Caltech Ian Foster, Argonne National Laboratory ``Integrating HPC into NII Environments: How Can Languages, Compilers, and Libraries Help?'' Dennis Gannon, Indiana University ``Programming Tools for Integrating Supercomputing Applications with the NII'' Marina Chen, Boston University ``World Wide Computer: Hybrid Models of Computation and Programming Language Issues'' The Future of Libraries and Reusable Software Wednesday, July 19, 7:30pm to 10:00pm Discussion Leader: Bill Gropp, Argonne National Laboratory TBD, ``TBD'' Tony Skjellum, Mississippi State University ``The Future of Reusable Libraries: Poly-Algorithms, Scalability, Classes and Meta-Code Representations'' Performance/Algorithm Visualization Tools Thursday, July 20, 9:00am to 12:15pm Discussion Leader: Mike Heath, University of Illinois Dan Reed, University of Illinois ``Learning and Real-time Control'' Bart Miller, University of Wisconsin ``Performance Tuning as Experiment Management'' Rusty Lusk, Argonne National Laboratory ``Performance Analysis of MPI Programs on Multiple Architectures'' Gordon Research Conference Business Meeting Thursday, July 20, 7:30pm to 9:00pm (times approximate after banquet) Election of vice-chair for 1996 conference 1995 vice-chair becomes 1996 chair Milestones, Economics, and Technology Friday, July 21, 9:00am to 11:45am Discussion Leader: Al Geist, Oak Ridge National Laboratory Jim McGraw, Lawrence Livermore National Laboratory ``Measuring Programing in HPC and NII Research'' Joel Saltz, University of Maryland ``The Application Programmers Dilemma: Great GUIs on Pentiums Versus Dumb Terminals on Teraflop Machines'' Jill Mesirov, Boston University (formerly with Thinking Machines) ``TBD'' From owner-mpi-core@CS.UTK.EDU Tue May 16 12:21:24 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA29963; Tue, 16 May 1995 12:21:24 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id MAA09725; Tue, 16 May 1995 12:19:07 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Tue, 16 May 1995 12:19:06 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id MAA09718; Tue, 16 May 1995 12:19:04 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA27944 for ; Tue, 16 May 1995 11:19:01 -0500 Message-Id: <199505161619.LAA27944@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: Final version of MPI-1 Date: Tue, 16 May 1995 11:18:59 -0500 From: Rusty Lusk At the last meeting we took the first vote on various minor "cleanup" modifications to the MPI (1) document. The plan is to have a second and final vote on these at the upcoming meeting. Marc has prepared an updated version with change bars in the margins to indicate changes from the original final (:-)) draft of May 5, 1994. All the chapters separately and the whole report itself are available in .tex, .dvi and .ps form by anonymous ftp to ftp.mcs.anl.gov in the directory pub/mpi/mpi-1.jun95. There is also a tar file of this whole directory in pub/mpi/mpi-1.jun95.tar.Z. Please check out the changes so that we can finish with MPI-1 once and for all at the next meeting. Let me know if you encounter any problems. Rusty From owner-mpi-core@CS.UTK.EDU Thu May 18 11:14:14 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA16062; Thu, 18 May 1995 11:14:14 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id LAA28206; Thu, 18 May 1995 11:12:15 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Thu, 18 May 1995 11:12:14 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from antares.mcs.anl.gov by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id LAA28198; Thu, 18 May 1995 11:12:12 -0400 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA12853 for ; Thu, 18 May 1995 10:12:03 -0500 Message-Id: <199505181512.KAA12853@antares.mcs.anl.gov> To: mpi-core@CS.UTK.EDU Subject: mailing lists Date: Thu, 18 May 1995 10:12:02 -0500 From: Rusty Lusk Dear mpi-core, David and I are moving the mailing lists, in order to place them under the control of the majordomo list manager, which will let people subsucribe and unsubscribe themselves from lists, find out who is on them, etc. You should soon get a message welcoming you to the "new" mpi-core mailing list, since we are automatically subscribing everyone on mpi-core. The full address of the mpi-core mailing list will now be mpi-core@mcs.anl.gov instead of mpi-core@cs.utk.edu. After that, you should get (on the new list) a set of subcommittee mailing lists that you can join. Let us know if there are any questions, problems, or confusions. David Walker and Rusty Lusk From owner-mpi-core@mcs.anl.gov Thu May 18 11:15:00 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA16067; Thu, 18 May 1995 11:15:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA12905 for mpi-core-archive@netlib2.cs.utk.edu; Thu, 18 May 1995 10:15:08 -0500 Date: Thu, 18 May 1995 10:15:08 -0500 Message-Id: <199505181515.KAA12905@antares.mcs.anl.gov> To: mpi-core-archive@netlib2.cs.utk.edu From: Majordomo@mcs.anl.gov Subject: Welcome to mpi-core Reply-To: Majordomo@mcs.anl.gov -- Welcome to the mpi-core mailing list! If you ever want to remove yourself from this mailing list, send the following command in email to "mpi-core-request@mcs.anl.gov": unsubscribe Or you can send mail to "Majordomo@mcs.anl.gov" with the following command in the body of your email message: unsubscribe mpi-core mpi-core-archive@netlib2.cs.utk.edu Here's the general information for the list you've subscribed to, in case you don't already have it: [Last updated on: Wed May 17 21:02:13 1995] This is the core MPI Forum list. It is intended primarily for those who are attending meetings MPI Forum meetings. To unsubscribe, send unsubscribe mpi-core to majordomo@mcs.anl.gov. From mpi-core-human@mcs.anl.gov Thu May 18 14:32:42 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA17836; Thu, 18 May 1995 14:32:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA19835 for mpi-core-out; Thu, 18 May 1995 13:32:33 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA19788; Thu, 18 May 1995 13:30:56 -0500 Message-Id: <199505181830.NAA19788@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov, mpi-comm@antares.mcs.anl.gov Subject: Subcommittee discussion lists Date: Thu, 18 May 1995 13:30:55 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk You are getting this note because you are subscribed to either of the following lists mpi-comm@mcs.anl.gov MPI Forum discussion list, all comments welcome mpi-core@mcs.anl.gov MPI forum meeting attendees list The MPI-2 Forum has tentatively formed subcommittees to work on various projects within MPI-2. We are not automatically subscribing anyone to these lists. If you want to participate in the discussion on a particular topic, please send mail to majordomo.mcs.anl.gov containing subscribe The lists and their topics are: mpi-dynamic MPI Forum discussion list for dynamic process management mpi-1sided MPI Forum discussion list for one-sided communication mpi-coll MPI Forum discussion list for collective operations mpi-external MPI Forum discussion list for external interfaces mpi-bind MPI Forum discussion list for C++ and Fortran-90 bindings Other subcommittees might appear. These are the ones agreed on at the first MPI-2 meeting. In MPI-1, the most specific discussions took place on the the subcommittee lists, where draft proposals were put together. We apologize for the barrage of mail today on organizing the mailing lists. This should be the last message on this topic. Regards, Rusty Lusk and David Walker From mpi-core-human@mcs.anl.gov Thu May 18 14:50:20 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA17955; Thu, 18 May 1995 14:50:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA20639 for mpi-core-out; Thu, 18 May 1995 13:50:08 -0500 Received: from igate1.hac.com (igate1.hac.com [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id NAA20630 for ; Thu, 18 May 1995 13:50:01 -0500 Received: from msmail4.hac.com ([192.27.11.35]) by igate1.hac.com (4.1/SMI-4.1) id AA29165; Thu, 18 May 95 11:47:37 PDT Message-Id: Date: 18 May 1995 11:45:21 -0800 From: "Lewins, Lloyd J" Subject: Definitions of Extent versus Stride To: "mpi-core" X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The current definition of Extent, LB and UB leads to certain problems. I believe this is caused by the confusion between two different concepts: 1) The "extent" of the underlying data. I.e., the difference between the byte address of the highest byte in the datatype, and the byte address of the lowest byte in the datatype. 2) The "stride" of the datatype. I.e., when replicating instances of the datatype, the byte spacing between these instances. As currently defined, Extent refers to the later quantity, i.e., it is actually the stride. However, the rules for modifying the extent using MPI_UB and MPI_LB, prevent the stride from being reduced, if a subcomponent of the datatype already contains an UB or LB marker. This is an unneccessary restriction for strides. The second concept is not addressed in the current MPI specification. The lack of this abstraction, prevents libraries from creating dynamically allocating memory which corresponds to a given MPI datatype. In particular, this makes it very difficult for the collective function MPI_REDUCE to be efficiently implemented by layering over the MPI point-to-point functions. Consider, a tree implementation of MPI_REDUCE. The internal nodes of the tree must receive a message from the outer nodes, and pass the received data into the reduction function. Because the function may be user defined, and may use any datatype, the buffer passed into the reduction function cannot be packed, but must represent the actual layout of the data as specified by the datatype. Further, the internal node, unless it is the root, cannot use the recv or send buffer specified by the user for this buffer, but must instead create a temporary buffer of the appropriate size. Unfortunately, no MPI function exists which returns this size. MPI_TYPE_SIZE returns the "packed" size of the datatype, and MPI_TYPE_EXTENT returns the stride between replications of this type. For example, consider a datatype which represents a column vector, the stride for this vector will be one element. However, the size of the temporary buffer needs to be much larger. I propose that we modify the MPI-1 standard to seperate the issue of data extent from stride. I suggest that we leave the functions MPI_TYPE_EXTENT, MPI_TYPE_UB and MPI_TYPE_LB alone, but that we change the definition of ub, lb and extent. In addition, I suggest that we add the concept of a datatype stride, along with two functions, one to set the stride for a datatype, and one to find the stride from a datatype. Thus in section 3.12, lb(Typemap) = min disp j j and ub(Typemap) = max (disp + sizeof(type )) j j j extent(Typemap) = ub(Typemap) - lb(Typemap) NOTE: no rounding for alignment purposes. Assuming that we define sizeof(UB) = sizeof(LB) = 0, we no longer need the additional rules defined in section 3.12.3. I.e., ub and lb are sticky over all datatypes. Then MPI_LB and MPI_UB can only be used to set lower or higher values for lb and ub respectively, they cannot be used to increase lb or decrease ub. We need to add a definition for the stride of a datatype as follows: For basic datatypes: stride(type) = sizeof(type) For UB and LB: stride(type) = 0 stride(Typemap) = extent(Typemap) + e Where: If type requires alignment to a byte address that is a multiple of k , i i then e is the least nonnegative increment needed to round stride(Typemap) to the next multiple of max k i i This default value of stride can be overridden using the function: MPI_TYPE_SET_STRIDE (type, stride) INOUT type in stride NOTE: This only sets the stride for this particular datatype. It does not affect the stride of datatype created from this datatype!! The accessor function MPI_TYPE_STRIDE finds the current stride of a given datatype, thus: MPI_TYPE_STRIDE (type, stride) IN type OUT stride Finally, we need to adjust the definitions of the datatype creator functions, so that they use stride instead of extent when specifying the type map for the created type. NOTES: 1) Generally the definition of stride is equivelent to the old definition of extent. The differences are only apparent when either LB or UB are used (old style), or MPI_TYPE_SET_STRIDE is used (new style). 2) The new definition of extent doesn't include the rounding neccessary for alignment purposes, nor is it ever decreased below the actual size of the data. Extent can thus be safely used to malloc a memory buffer for a given type. 3) MPI_LB and MPI_UB can be used to increase the extent of a datatype, although for what purpose, I am not clear. EXAMPLES: 1) Creating a column vector type for a 10 * 10 int array: MPI_Type_vector (10, 1, 10, MPI_INT, &column_type); MPI_Type_set_stride (&column_type, sizeof (int)); This vector can then be used in calls to access the array in column major order. 2) Creating a new memory buffer, to be used with a datatype: MPI_Type_extent (type, &extent); MPI_Type_lb (type, &lb); buffer = ((char *)malloc (extent)) + lb; Note: the addition of the type lb to the buffer is neccessary because some types may include elements with negative displacements!! I believe the above changes are urgently required, and should be included in any changes made to the MPI-1 standard. The lack of the ability to determine the correct size with which to malloc a buffer causes problems in the MPICH implementation, which prevents it from correctly implementing the reduce functions. In addition, I expect other library writers will or have triped over this problem (probably without knowing it!!) Lloyd Lewins Hughes Aircraft Co. llewins@msmail4.hac.com (310) 616-0826 From mpi-core-human@mcs.anl.gov Sun May 21 17:09:42 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA00978; Sun, 21 May 1995 17:09:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA29078 for mpi-core-out; Sun, 21 May 1995 16:09:25 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA29072 for ; Sun, 21 May 1995 16:09:19 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id RAA04203 for ; Sun, 21 May 1995 17:08:58 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id RAA21940 for ; Sun, 21 May 1995 17:09:18 -0400 Date: Sun, 21 May 1995 17:09:18 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199505212109.RAA21940@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA06827; Sun, 21 May 95 17:09:17 EDT To: mpi-core@mcs.anl.gov Subject: small MPI-1 change to MPI_GRAPH_CREATE Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy, I noticed a small inconsistancy in the topology chapter that I suspect we should correct. MPI_GRAPH_CREATE says that comm_old is an input communicator without topology. On the other hand, MPI_CART_CREATE says comm_old is an input communicator. I think the fact that MPI_GRAPH_CREATE says its input comm_old has no topology is just meant to say that it is going to add one. I don't think we indended to exclude taking a comm with one topology and using it to create a new topology on a new comm. I propose we remove the "without topology" from MPI_GRAPH_CREATE. Does anyone agree or disagree? Steve From mpi-core-human@mcs.anl.gov Sun May 21 17:19:55 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA01030; Sun, 21 May 1995 17:19:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA29166 for mpi-core-out; Sun, 21 May 1995 16:19:56 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA29155 for ; Sun, 21 May 1995 16:19:50 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA02943; Sun, 21 May 95 16:19:49 CDT Date: Sun, 21 May 95 16:19:49 CDT From: Tony Skjellum Message-Id: <9505212119.AA02943@Aurora.CS.MsState.Edu> To: mpi-core@mcs.anl.gov, lederman@super.org Subject: Re: small MPI-1 change to MPI_GRAPH_CREATE Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There is no reason I can think of for the asymmetry, or for the restriction. -Tony ----- Begin Included Message ----- From mpi-core-human@mcs.anl.gov Sun May 21 16:17:51 1995 Date: Sun, 21 May 1995 17:09:18 -0400 From: lederman@super.org (Steve Huss-Lederman) To: mpi-core@mcs.anl.gov Subject: small MPI-1 change to MPI_GRAPH_CREATE Sender: owner-mpi-core@mcs.anl.gov Content-Length: 613 Howdy, I noticed a small inconsistancy in the topology chapter that I suspect we should correct. MPI_GRAPH_CREATE says that comm_old is an input communicator without topology. On the other hand, MPI_CART_CREATE says comm_old is an input communicator. I think the fact that MPI_GRAPH_CREATE says its input comm_old has no topology is just meant to say that it is going to add one. I don't think we indended to exclude taking a comm with one topology and using it to create a new topology on a new comm. I propose we remove the "without topology" from MPI_GRAPH_CREATE. Does anyone agree or disagree? Steve ----- End Included Message ----- From mpi-core-human@mcs.anl.gov Sun May 21 19:04:13 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA02355; Sun, 21 May 1995 19:04:12 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA29609 for mpi-core-out; Sun, 21 May 1995 18:04:18 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA29603 for ; Sun, 21 May 1995 18:04:14 -0500 Message-Id: <199505212304.SAA29603@antares.mcs.anl.gov> To: mpi-core@mcs.anl.gov Subject: updates to MPI-1 Date: Sun, 21 May 1995 18:04:13 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A set of changes (in almost all chapters) has been applied (today) to the MPI-1 in pub/mpi/mpi-1.jun95 on our ftp server ftp.mcs.anl.gov. These are followup changes, by Steve Huss-Lederman, to those made by Marc Snir last week, to bring everything into line with what we voted on at the last meeting. Please check any changes that you have been following, especially chapter authors. Both latex and postscript are available for each chapter and for the whole report, and the whole directory is in a tar file mpi-1.jun95.tar.Z in the parent directory pub/mpi. Rusty From mpi-core-human@mcs.anl.gov Mon May 22 02:46:09 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA05069; Mon, 22 May 1995 02:46:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA01526 for mpi-core-out; Mon, 22 May 1995 01:46:05 -0500 Received: from gmdzi.gmd.de (gmdzi.gmd.de [129.26.8.90]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id BAA01520 for ; Mon, 22 May 1995 01:45:59 -0500 Received: from f1neuman.gmd.de (f1neuman) by gmdzi.gmd.de with SMTP id AA09308 (5.65c8/IDA-1.4.4 for ); Mon, 22 May 1995 08:45:54 +0200 Received: by f1neuman.gmd.de (AIX 3.2/UCB 5.64/4.03) id AA18599; Mon, 22 May 1995 08:45:51 +0200 Date: Mon, 22 May 1995 08:45:51 +0200 From: Rolf.Hempel@gmd.de (Rolf Hempel) Message-Id: <9505220645.AA18599@f1neuman.gmd.de> To: mpi-core@mcs.anl.gov Subject: Topologies Cc: gmap10@f1neuman.gmd.de Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear All, I completely agree with Steve and Tony that there is no reason for the asymmetry, or for the restriction for the input communicator in MPI_CART_CREATE/MPI_GRAPH_CREATE. Thus, I think the proposed change to remove the words "without topology" is the best we can do. Thanks! Rolf ----- Begin Included Message ----- >From mpi-core-human@mcs.anl.gov Sun May 21 16:17:51 1995 Date: Sun, 21 May 1995 17:09:18 -0400 From: lederman@super.org (Steve Huss-Lederman) To: mpi-core@mcs.anl.gov Subject: small MPI-1 change to MPI_GRAPH_CREATE Sender: owner-mpi-core@mcs.anl.gov Content-Length: 613 Howdy, I noticed a small inconsistancy in the topology chapter that I suspect we should correct. MPI_GRAPH_CREATE says that comm_old is an input communicator without topology. On the other hand, MPI_CART_CREATE says comm_old is an input communicator. I think the fact that MPI_GRAPH_CREATE says its input comm_old has no topology is just meant to say that it is going to add one. I don't think we indended to exclude taking a comm with one topology and using it to create a new topology on a new comm. I propose we remove the "without topology" from MPI_GRAPH_CREATE. Does anyone agree or disagree? Steve ----- End Included Message ----- From mpi-core-human@mcs.anl.gov Mon May 22 08:19:53 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA11843; Mon, 22 May 1995 08:19:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA02776 for mpi-core-out; Mon, 22 May 1995 07:19:52 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA02769 for ; Mon, 22 May 1995 07:19:47 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id IAA05375 for ; Mon, 22 May 1995 08:19:24 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id IAA27181 for ; Mon, 22 May 1995 08:19:45 -0400 Date: Mon, 22 May 1995 08:19:45 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199505221219.IAA27181@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA08787; Mon, 22 May 95 08:19:44 EDT To: mpi-core@mcs.anl.gov Subject: input for June meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy, This is a reminder from the MPI-2 editor that chapters to be distributed at the MPI meeting on June 5-7 are due in one week. Everyone (including me :-) should send their input to lederman@super.org by COB Monday, May 29. I have talked with Rusty and the plan is as follows. Each person at the next meeting will receive a copy of the revised (and near final) MPI-1 document. They will also receive a copy of the latest draft of the MPI-2 document based on the input I receive. I will also post when an electronic version of the MPI-2 document is available. Steve From owner-mpi-core@CS.UTK.EDU Mon May 22 13:39:28 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA15985; Mon, 22 May 1995 13:39:27 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA03828; Mon, 22 May 1995 13:38:09 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 22 May 1995 13:38:07 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id NAA03817; Mon, 22 May 1995 13:38:05 -0400 From: Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 7645; Mon, 22 May 95 13:37:32 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 0589; Mon, 22 May 1995 13:37:32 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 22 May 95 13:37:31 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA22673; Mon, 22 May 1995 13:37:30 -0400 Message-Id: <9505221737.AA22673@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: jim@meiko.co.uk (James Cownie) Cc: mpi-core@CS.UTK.EDU Subject: Re: MPI (derived datatypes) In-Reply-To: (Your message of Mon, 22 May 1995 16:23:16 -0000.) Date: Mon, 22 May 95 13:37:31 EDT : -) :-) :-) *** (-: (-: (-: Date: Mon, 22 May 1995 13:37:29 -0400 From: Marc Snir > > Marc, > > There's a statement on page 75 of the latest MPI-1 report, which > should probably be clarified. It says > > "A datatype may specify overlapping entries. If such a datatype is > used in a receive operation, that is, if some part of the buffer is > written more than once by the receive operation, then the call is > erroneous." > > This would appear to allow a datatype which specified an overlapping > area to be used in a receive, provided that the message which was > received was short enough not to overwrite the same field twice. > > Is this really what we want ? > > Surely we just want to outlaw the use of such datatypes in receives > once and for all, in which case we could simply say. > > "A datatype may specify overlapping entries. The use of such a > datatype in a receive operation is erroneous. (This is erroneous even > if the actual message received is short enough not to write any entry > more than once.)" > > Or can you see a use for this ? > > -- Jim I agree with Jim, and suggest to change the text to say that it is erroneous to use a datatype with overlaps in a receive call. From owner-mpi-core@CS.UTK.EDU Mon May 22 15:31:49 1995 Received: from CS.UTK.EDU by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA17047; Mon, 22 May 1995 15:31:48 -0400 Received: from localhost by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA14289; Mon, 22 May 1995 15:31:45 -0400 X-Resent-To: mpi-core@CS.UTK.EDU ; Mon, 22 May 1995 15:31:43 EDT Errors-to: owner-mpi-core@CS.UTK.EDU Received: from watson.ibm.com by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id PAA14282; Mon, 22 May 1995 15:31:41 -0400 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9869; Mon, 22 May 95 15:31:09 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 0448; Mon, 22 May 1995 15:31:09 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 22 May 95 15:31:08 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA20161; Mon, 22 May 1995 15:31:07 -0400 Message-Id: <9505221931.AA20161@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@CS.UTK.EDU Subject: another minor MPI correction Reply-To: snir@watson.ibm.com Date: Mon, 22 May 1995 15:31:07 -0400 From: Marc Snir :-) :-) :-) *** (-: (-: (-: pg 119 l1 (section 4.9.4). The text describing reduce operations says: "If commute = false, then the order of operations is fixed and is defined to be in ascending, process rank order, beginning with process zero". This can be interpreted as disallowing a change in the order of operations, using associativity. The text on pg 112 line 20 and the implementation note later in the text makes clear that MPI wants to allow a balanced tree to be used for reduction. Suggested change: "If commute = false, then the order of operands is fixed and is defined to be in ascending, process rank order, beginning with process zero. The order of evaluation can be changed, talking advantage of the associativity of the operation. If commute = true then the order of evaluation can be changed, taking advantage of commutativity and associativity." From mpi-core-human@mcs.anl.gov Thu May 25 17:31:36 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA08685; Thu, 25 May 1995 17:31:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA08728 for mpi-core-out; Thu, 25 May 1995 16:29:43 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA08722 for ; Thu, 25 May 1995 16:29:33 -0500 Message-Id: <199505252129.QAA08722@antares.mcs.anl.gov> To: mpi-core@mcs.anl.gov Subject: MPI meeting Date: Thu, 25 May 1995 16:29:29 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I need a head count for the hotel for the meeting on June 5-7. This will be an important meeting since we will take final votes on the changes to the MPI-1 document and we will have the first set of MPI-2 proposals to begin serious work on. Please make your reservations immediately since the Hotel is sold out except for the block they are holding for us until tomorrow. Please let me know right away whether you are coming or not. If you have any questions, don't hesitate to send me mail at lusk@mcs.anl.gov or call me at (708) 252-7852. Regards, Rusty The usual blurb: The O'Hare Hilton is the hotel *inside* the airport. You just walk across the street from the baggage-claim level. A block of rooms has been reserved, which will be held until March 1, so please make your reservations with the Hotel before then. The phone number for the hotel is (312) 686-8000 (FAX (312) 601-2876). Mention the *MPI Forum* to get the special rate, which is $90.50 + tax per night. We will charge the usual $75 registration to take care of the meeting room, lunch on Tuesday, and coffee, etc. The meeting will start at 1:30 on Monday to allow people to fly in that morning, and end at noon on Wednesday. From mpi-core-human@mcs.anl.gov Thu May 25 18:34:21 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA09035; Thu, 25 May 1995 18:34:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA10278 for mpi-core-out; Thu, 25 May 1995 17:34:25 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA10254 for ; Thu, 25 May 1995 17:34:07 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id SAA22974 for ; Thu, 25 May 1995 18:33:41 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id SAA06667 for ; Thu, 25 May 1995 18:34:05 -0400 Date: Thu, 25 May 1995 18:34:05 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199505252234.SAA06667@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA24125; Thu, 25 May 95 18:34:04 EDT To: mpi-core@mcs.anl.gov Subject: input due on Tuesday Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy, It was mentioned that Monday, May 29 is a national holiday in the U.S. As a result, the deadline for inputs to be distributed at the next meeting will be Tuesday, May 30. If you wait until Tuesday, please have the input in a form that will not be too hard for me to include. Thanks, Steve From mpi-core-human@mcs.anl.gov Fri May 26 11:19:11 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA24001; Fri, 26 May 1995 11:19:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA19741 for mpi-core-out; Fri, 26 May 1995 10:19:10 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA19735 for ; Fri, 26 May 1995 10:19:01 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 4977; Fri, 26 May 95 11:18:01 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 4777; Fri, 26 May 1995 11:18:01 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 26 May 95 11:16:32 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA19721; Fri, 26 May 1995 11:16:31 -0400 Message-Id: <9505261516.AA19721@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Cc: frankeh@watson.ibm.com Subject: MPI1 pt2pt sec 3.12.6 correct use of addresses Reply-To: snir@watson.ibm.com Date: Fri, 26 May 1995 11:16:30 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: This section reflects an attempt to protect MPI from problems that may arise in segmented address spaces, and from the fact the a Fortran INTEGER may not be large enough to hold an address. I think the attempt has failed. Since an address, at least in Fortran, is a 32 bit INTEGER, one cannot cannot associate much hidden information with this address without going through unseemely contorsions. Therefore, there no much scope for doing something smarter then done in current implementations: namely MPI_BOTTOM is some program address, and absolute addresses in datatypes are displacements wrt to that address. This means that absolute addresses are restricted to a 4 GB range and Intel 16 bit implementations have to use normalized addresses (huge pointers, or whatever they are called). (The "real" solution is to use a special datatype for addresses and special methods for address arithmetics. But that's C++/MPI2). I suggest to delete this section and replace it with some warning, at the end of 3.12.2, that implementations may restrict the range of absolute address to a only part of the application address space. Relative displacements within contiguous storage should always work OK. I shall write down something more detailed. This will get rid of a fairly obscure and badly written part of the standard that seems not to impact any current implementation. Note that I am not adding/deleting functions, nor am I changing their behavior. Just weakening or "fuzzying" a correctness requirement. Comments? From mpi-core-human@mcs.anl.gov Tue May 30 12:37:27 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA20861; Tue, 30 May 1995 12:37:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA29617 for mpi-core-out; Tue, 30 May 1995 11:37:37 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA29606; Tue, 30 May 1995 11:37:29 -0500 Message-Id: <199505301637.LAA29606@antares.mcs.anl.gov> To: mpi-core@mcs.anl.gov Subject: Final version of MPI-1 Cc: lusk@antares.mcs.anl.gov Date: Tue, 30 May 1995 11:37:28 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I have now made what I believe to be the final round of changes to the MPI-1 document, and marked them with change bars. If I don't hear from anyone in the next 24 hours, I will print this for us to vote on at next week's meeting. Chapter authors, please check that this is how you want your chapter to look for posterity. Everything, including both tex and ps files, is in pub/mpi/mpi-1.jun95 on ftp.mcs.anl.gov, and is availabie by anonymous ftp. Please take a look. Regards, Rusty From mpi-core-human@mcs.anl.gov Tue May 30 17:03:01 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA22265; Tue, 30 May 1995 17:03:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA06240 for mpi-core-out; Tue, 30 May 1995 16:03:22 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA06232 for ; Tue, 30 May 1995 16:03:14 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 0463; Tue, 30 May 95 17:02:10 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 9193; Tue, 30 May 1995 17:02:09 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Tue, 30 May 95 17:02:08 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA23735; Tue, 30 May 1995 17:02:07 -0400 Message-Id: <9505302102.AA23735@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Cc: frankeh@watson.ibm.com Subject: Is the following code correct? (pack/unpack) Reply-To: snir@watson.ibm.com Date: Tue, 30 May 1995 17:02:06 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: It may be sometimes convenient to unpack the same information twice. Suppose, e.g., that we are receiving a packed array of structures. We may want to unpack the first field of the structure, in order to determine where the entire structure should be copied to. In such case, it will be more convenient to first unpack the first field, then unpack the entire structure, including the first field again. I wrote below an artificial, simplified code for illustration. Is such code legal? Our standard document could be interpreted either way. int a[1000][2]; char buffer[10000]; int pos0, pos1, i, myrank; MPI_comm_rank(MPI_COMM_WORLD, myrank); if (myrank == 0) { for (i=0; i < 1000; i++) { a[i][1] = permute(i); /* permute() computes some permutation of 1..1000 */ a[i][2] = foo(i); /* foo() computes some useful stuff */ MPI_Send(a, 2000, MPI_INT, 1, 0, MPI_COMM_WORLD); } else if (myrank == 1) { MPI_Recv(buffer, 10000, MPI_PACKED, 0, 0, MPI_COMM_WORLD, status); pos0 = pos1 = 0; for (i=0; i< 1000; i++) { MPI_Unpack(buffer, 10000, &pos0, &i, 1, MPI_INT, MPI_COMM_WORLD); MPI_Unpack(buffer, 10000, &pos1, &a[i], 2, MPI_INT, MPI_COMM_WORLD); pos0 = pos1; } } From mpi-core-human@mcs.anl.gov Wed May 31 11:10:09 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA04191; Wed, 31 May 1995 11:10:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA17411 for mpi-core-out; Wed, 31 May 1995 10:08:20 -0500 Received: from hub (root@hub.meiko.co.uk [192.131.108.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA17397 for ; Wed, 31 May 1995 10:08:06 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA08048 (5.65c/IDA-1.4.4 for mpi-core@mcs.anl.gov); Wed, 31 May 1995 16:07:51 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA19241; Wed, 31 May 1995 16:07:48 +0000 Date: Wed, 31 May 1995 16:07:48 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9505311507.AA19241@deneb.co.uk> To: mpi-core@mcs.anl.gov In-Reply-To: (llewins@msmail4.hac.com) Subject: Re: Definitions of Extent versus Stride Reply-To: jim@meiko.com Content-Length: 1635 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Lloyd points out that we are currently trying to express two different properties of a datatype, the stride (i.e. repeat spacing when you array the datatype), and the extent (i.e. difference between the lowest and highest addresses referenced by the datatype). in the single function MPI_TYPE_EXTENT. The problem that he attempts to solve is that there is no reasonable way to find out how much space one needs to allocate to hold an instance of a particular datatype when it is in its "natural" state. (I.e. with the alignment padding it needs on the local machine). This should be completely divorced from the way that the ub and lb may have been manipulated to place the datatype at a particular address, or manipulate the stride such that one can send rows/columns of an array. I think Lloyd's objective is a good one, BUT there has been NO DISCUSSION OF IT (unless my mailer died at some point). Unfortunately, while I think that his solution is clean and workable, I also think it may be too late to change things as much as this solution requires, so I've a feeling such an amendment would be lost on compatibility grounds. However I'd like to see some discussion of the issues Lloyd has raised. Maybe there's an alternative proposal which would also fix the problems, but without the compatibility hit... -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From mpi-core-human@mcs.anl.gov Wed May 31 12:05:46 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA04585; Wed, 31 May 1995 12:05:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA19233 for mpi-core-out; Wed, 31 May 1995 11:05:55 -0500 Received: from igate1.hac.com (igate1.hac.com [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA19227 for ; Wed, 31 May 1995 11:05:48 -0500 Received: from msmail4.hac.com ([192.27.11.35]) by igate1.hac.com (4.1/SMI-4.1) id AA02430; Wed, 31 May 95 09:03:23 PDT Message-Id: Date: 31 May 1995 09:05:22 -0800 From: "Lewins, Lloyd J" Subject: RE: Definitions of Extent versus Stride To: "mpi-core" X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Jim, thanks for your support!! >The problem that he attempts to solve is that there is no reasonable >way to find out how much space one needs to allocate to hold an >instance of a particular datatype when it is in its "natural" >state. (I.e. with the alignment padding it needs on the local >machine). This should be completely divorced from the way that the ub >and lb may have been manipulated to place the datatype at a particular >address, or manipulate the stride such that one can send rows/columns >of an array. I am not sure what you mean by "natural", but I don't think it is the same as my use of "extent". I think you may mean the padding required on the local machine between concatenated objects, assuming one was "packing" the message into a contiguous sequence. My definition of extent is simply the diference between the displacement of the last byte referenced by the type, and the displacement of the first byte referenced by the type. Such a type may include internal strides neccessary to walk the column of a 2-d array. In such a case, the extent would be quite large (elementsize * (1 + rowlength * (columnlength - 1))). This would allow one to malloc a region of memory to hold such a (sparse) vector. >Unfortunately, while I think that his solution is clean and workable, >I also think it may be too late to change things as much as this >solution requires, so I've a feeling such an amendment would be lost >on compatibility grounds. While I agree that this change is late, I also think that it is urgently needed. The current definition of UB and LB cause confusion even for MPI Forum members. It is currently complex to create datatypes needed to perform simple operations like array transposition. Libraries, in particular layered implementations of MPI have no way to determine the actual size of memory required for an MPI datatype, and therefore cannot create instances of same. >However I'd like to see some discussion of the issues Lloyd has >raised. Maybe there's an alternative proposal which would also fix the >problems, but without the compatibility hit... While I agree that my proposal causes potential compatibility problems, I expect that they are still few. UB and LB are esoteric enough that the community still apprears to be grappeling with how to use them!! A expect that simplifying the creation of strided datatypes would actually be appreciated. Lloyd Lewins Hughes Aircraft Co, llewins@msmail4.hac.com From mpi-core-human@mcs.anl.gov Wed May 31 12:15:21 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA04687; Wed, 31 May 1995 12:15:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA19517 for mpi-core-out; Wed, 31 May 1995 11:15:56 -0500 Received: from igate1.hac.com (igate1.hac.com [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA19511 for ; Wed, 31 May 1995 11:15:49 -0500 Received: from msmail4.hac.com ([192.27.11.35]) by igate1.hac.com (4.1/SMI-4.1) id AA02830; Wed, 31 May 95 09:13:26 PDT Message-Id: Date: 31 May 1995 09:15:34 -0800 From: "Lewins, Lloyd J" Subject: RE: Is the following code correct? (pack/unpack) To: "mpi-core" X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >It may be sometimes convenient to unpack the same information twice.... >Is such code legal? Our standard document could be >interpreted either way. Interesting question!! While persuing the answer, I noticed the following error in the standard: Section 3.13, page 84, line 39: The input value of position is the first location in the OUTPUT buffer occupied by the packed message. ...so that the output value of position is the first location in the OUTPUT buffer after the locations occupied by the message that was unpacked. should read: The input value of position is the first location in the INPUT buffer occupied by the packed message. ...so that the output value of position is the first location in the INPUT buffer after the locations occupied by the message that was unpacked. Lloyd Lewins Hughes Aircraft Co llewins@msmail4.hac.com From mpi-core-human@mcs.anl.gov Wed May 31 12:27:47 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA04745; Wed, 31 May 1995 12:27:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA19771 for mpi-core-out; Wed, 31 May 1995 11:28:11 -0500 Received: from taurus.cs.nps.navy.mil (cs.nps.navy.mil [131.120.1.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA19765 for ; Wed, 31 May 1995 11:28:04 -0500 Received: from holmes.cs.nps.navy.mil by taurus.cs.nps.navy.mil (4.1/SMI-4.1) id AA12019; Wed, 31 May 95 09:27:43 PDT Received: by holmes.cs.nps.navy.mil (950215.SGI.8.6.10/940406.SGI) id JAA08226; Wed, 31 May 1995 09:27:37 -0700 From: "Robert George" Message-Id: <9505310927.ZM8224@holmes.cs.nps.navy.mil> Date: Wed, 31 May 1995 09:27:36 -0700 In-Reply-To: jim@meiko.co.uk (James Cownie) "Re: Definitions of Extent versus Stride" (May 31, 4:07pm) References: <9505311507.AA19241@deneb.co.uk> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: llewins@msmail4.hac.com, jim@meiko.com, mpi-core@mcs.anl.gov Subject: Re: Definitions of Extent versus Stride Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On May 31, 4:07pm, James Cownie wrote: > Subject: Re: Definitions of Extent versus Stride > > Lloyd points out that we are currently trying to express two different > properties of a datatype, > > the stride (i.e. repeat spacing when you array the datatype), > and the extent (i.e. difference between the lowest and highest > addresses referenced by the datatype). > in the single function MPI_TYPE_EXTENT. > > The problem that he attempts to solve is that there is no reasonable > way to find out how much space one needs to allocate to hold an > instance of a particular datatype when it is in its "natural" > state. (I.e. with the alignment padding it needs on the local > machine). While the local architecture will determine the aligment of primitive data types such as ints and floats, the compiler determines how the data will be packed. Some compilers group data elements by type for efficient packing. Ada even allows user-defined packing. Shouldn't the alignment/packing problem be handled as a heterogeneous interoperability issue (aka XDR), while handling the stride/extent problem as Lloyd suggested? Robert George Army Research Laboratory From mpi-core-human@mcs.anl.gov Wed May 31 12:50:10 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA04877; Wed, 31 May 1995 12:50:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA20126 for mpi-core-out; Wed, 31 May 1995 11:49:35 -0500 Received: from hub (root@hub.meiko.co.uk [192.131.108.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA20120 for ; Wed, 31 May 1995 11:49:25 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA11352 (5.65c/IDA-1.4.4 for mpi-core@mcs.anl.gov); Wed, 31 May 1995 17:49:08 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA19439; Wed, 31 May 1995 17:49:06 +0000 Date: Wed, 31 May 1995 17:49:06 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9505311649.AA19439@deneb.co.uk> To: llewins@msmail4.hac.com Cc: mpi-core@mcs.anl.gov In-Reply-To: (llewins@msmail4.hac.com) Subject: RE: Definitions of Extent versus Stride Content-Length: 2634 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Jim, thanks for your support!! No problem. > I am not sure what you mean by "natural", but I don't think it is the same > as my use of "extent". I think you may mean the padding required on the local > machine between concatenated objects, assuming one was "packing" the message > into a contiguous sequence. I agree, they're different. Your thought is about right. The concept I'm trying to capture in "natural" is this : Suppose I take the type signature of the datatype (i.e. the datatype without any of the information which says how it is layed out in store), and lay this out in store so that each type is on the correct alignment, then this is the natural layout on the machine. The size of this "natural" layout is what I need to malloc to hold a useful local instance of the data type, say in the collective routines you mention. Actually this definition is still slightly wrong, although it gets me the right answers for C arrays of struct { double; char; }; and struct {char; double;}; it fails for struct {char; double; char;}; unless I know to treat this struct as "the datatype". The problem here is the same one that you were pointing out, which is that ub is used for two conflicting purposes : 1) to alter the stride, so that you can play the row/column tricks 2) to alter the alignment/extent. These two uses conflict, because what I want to say is that the natural layout is the layout you get if you ignore any ub and lb markers (because I'm NOT interested in the stride changing effects), however this definition would lose me useful padding/alignment information which I do still need (as examples like the third can be constructed to show). On a related, but slightly tangential issue, it would be really nice if we could somehow also capture the ability to send the whole of a structure with gaps, rather than having to treat it as non-contiguous, because we're not allowed to update the padding fields. This can be worth a LOT of performance, and currently provides an incentive for people to code their programs using MPI_BYTE, rather than the real datatype. I think what we'd have to do to make this work would be to change the type matching rules a bit, so that in this circumstance we match one level above the bottom of the hierarchy... (Marc, are you there ? We need you !) -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From mpi-core-human@mcs.anl.gov Wed May 31 12:56:10 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA04916; Wed, 31 May 1995 12:56:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA20326 for mpi-core-out; Wed, 31 May 1995 11:56:51 -0500 Received: from hub (root@hub.meiko.co.uk [192.131.108.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA20319 for ; Wed, 31 May 1995 11:56:42 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA11520 (5.65c/IDA-1.4.4 for mpi-core@mcs.anl.gov); Wed, 31 May 1995 17:56:38 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA19454; Wed, 31 May 1995 17:56:36 +0000 Date: Wed, 31 May 1995 17:56:36 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9505311656.AA19454@deneb.co.uk> To: george@holmes.cs.nps.navy.mil Cc: mpi-core@mcs.anl.gov In-Reply-To: <9505310927.ZM8224@holmes.cs.nps.navy.mil> (george@holmes.cs.nps.navy.mil) Subject: Re: Definitions of Extent versus Stride Content-Length: 1119 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > While the local architecture will determine the aligment of primitive data > types such as ints and floats, the compiler determines how the data will be > packed. Some compilers group data elements by type for efficient packing. Ada > even allows user-defined packing. Fine. MPI provides a type definition "language" to allow the user to express however it is the compiler/user between them choose to layout her data. > Shouldn't the alignment/packing problem be handled as a heterogeneous > interoperability issue (aka XDR), while handling the stride/extent problem as > Lloyd suggested? This issue actually has nothing to do with heterogeneity. It is an issue about the definition of the type language that MPI provides, and what inquiry functions are useful on such types. The type language is used to specify the actual data types (the type signature, which is needed for heterogeneous implementations), the placement of the types in store, and how to move between one instance of a type and the next when arraying them. The problem is resolving the conflicting requirements of these three uses. -- Jim From mpi-core-human@mcs.anl.gov Wed May 31 16:21:54 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA06726; Wed, 31 May 1995 16:21:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA02918 for mpi-core-out; Wed, 31 May 1995 15:21:25 -0500 Received: from pnl.gov (gate.pnl.gov [130.20.64.36]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA02912 for ; Wed, 31 May 1995 15:21:18 -0500 Received: from snacker.pnl.gov by pnl.gov (PMDF V4.3-13 #6012) id <01HR5KGL4T3K94EFNH@pnl.gov>; Wed, 31 May 1995 13:21:14 -0700 (PDT) Received: by snacker.pnl.gov (4.1/SMI-4.1) id AA17772; Wed, 31 May 95 13:21:12 PDT Date: Wed, 31 May 1995 13:21:12 -0700 From: j_nieplocha@pnl.gov Subject: proposal for consistency of Fortran and C interfaces To: mpi-core@mcs.anl.gov Message-id: <9505312021.AA17772@snacker.pnl.gov> Content-transfer-encoding: 7BIT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk When writing libraries in C language for Fortran programs, usually there are some implicit assumptions made about consistency of the C and Fortran interfaces to a message-passing library. As a minimum, MPI should guarantee consistency of: 1. startup and cleanup operations: MPI_INIT and MPI_FINALIZE MPI_INIT when called from either language should initialize both Fortran and C interfaces to MPI. Similarly, MPI_FINALIZE called from either language should be able to clean and shut up both C and Fortran interfaces. 2. groups and communicators This requires that routines like MPI_COMM_RANK, MPI_GROUP_RANK, MPI_COMM_SIZE, MPI_GROUP_SIZE returned the same values for the specified communicator/group when called by the same process from either language. Of course, we still need a mechanism for passing MPI handles between Fortran and C, for example like the one proposed by Leslie Hart and Tom Henderson. --Jarek Nieplocha From mpi-core-human@mcs.anl.gov Wed May 31 16:27:59 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA06739; Wed, 31 May 1995 16:27:57 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA03115 for mpi-core-out; Wed, 31 May 1995 15:28:36 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA03108; Wed, 31 May 1995 15:28:29 -0500 Message-Id: <199505312028.PAA03108@antares.mcs.anl.gov> To: j_nieplocha@pnl.gov cc: mpi-core@mcs.anl.gov Subject: Re: proposal for consistency of Fortran and C interfaces In-Reply-to: Message from j_nieplocha@pnl.gov of "Wed, 31 May 1995 13:21:12 -0700." <9505312021.AA17772@snacker.pnl.gov> Date: Wed, 31 May 1995 15:28:28 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | | | When writing libraries in C language for Fortran programs, usually | there are some implicit assumptions made about consistency of | the C and Fortran interfaces to a message-passing library. | | As a minimum, MPI should guarantee consistency of: | | 1. startup and cleanup operations: MPI_INIT and MPI_FINALIZE | | MPI_INIT when called from either language should initialize both Fortran | and C interfaces to MPI. | | Similarly, MPI_FINALIZE called from either language should be able to clean | and shut up both C and Fortran interfaces. | One requirement that our implementation requires is that MPI_INIT be called from the same language as the "main" program (main if C, program foo if Fortran). This is because of the dependency on command-line arguments for some startup information. With that restriction, all is as well as it is for mixed language programming on the target platform. Bill From mpi-core-human@mcs.anl.gov Wed May 31 17:56:10 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA07380; Wed, 31 May 1995 17:56:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA05323 for mpi-core-out; Wed, 31 May 1995 16:56:38 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA05315 for ; Wed, 31 May 1995 16:56:29 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id RAA13858 for ; Wed, 31 May 1995 17:55:59 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id RAA02394 for ; Wed, 31 May 1995 17:56:28 -0400 Date: Wed, 31 May 1995 17:56:28 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199505312156.RAA02394@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA16289; Wed, 31 May 95 17:56:27 EDT To: mpi-core@mcs.anl.gov Subject: MPI-2 document available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The MPI-2 document to be handed out at the meeting is now available. It is missing the collective chapter which should be done soon. Once it is done the electronic version will be updated. A hardcopy of both the current version and the collective chapter will be available at the meeting next week. The report is in: ftp://ftp.super.org/pub/mpi/mpi2/mpif-0695/mpi2-report.ps As usual, the ASCII text is available in: ftp://ftp.super.org/pub/mpi/mpi2/Editor See most of you soon. Steve From mpi-core-human@mcs.anl.gov Thu Jun 1 11:47:52 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA20223; Thu, 1 Jun 1995 11:47:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA03030 for mpi-core-out; Thu, 1 Jun 1995 10:47:47 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA03023 for ; Thu, 1 Jun 1995 10:47:41 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id LAA17537 for ; Thu, 1 Jun 1995 11:47:10 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id LAA15384 for ; Thu, 1 Jun 1995 11:47:39 -0400 Date: Thu, 1 Jun 1995 11:47:39 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199506011547.LAA15384@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA28059; Thu, 1 Jun 95 11:47:38 EDT To: mpi-core@mcs.anl.gov Subject: add advice to MPI-1 document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I know it is late to be suggesting a change to the document, but what I am proposing is only advice to users and not any change in the standard. I welcome comments. If there are not serious objections I plan to propose it at the meeting next week. Potential coding mistake: You have a legacy code that had the length of msgs in bytes. You know these are actually doubles and convert the code to use MPI_DOUBLE. You know you have to fix up the length so you do: len_elements = len_bytes / sizeof(MPI_DOUBLE) This will likely give you the wrong result. The reason is that MPI_DOUBLE is actually an opaque object and what you get back is this size (likely to be the sizeof a pointer), not the size of the datatype. What makes this more confusing is that you can, and need to, do sizeof(MPI_Status) and sizeof(MPI_Request) to allocate arrays to hold status and request. These return exactly what you expect. This mistake can lead to real bugs in code (I just found such a mistake yesterday). To make this clear to the user, I propose the following addition to the MPI-1 document: p. 19, line 13 of May 30, 1995 draft (end of section 3.2.2): \begin{advice} The basic datatyes in \MPI/ are stored in opaque objects. As a result, the size of the datatype in C should be determined from the size of the C datatype and not the \MPI/ datatype. For example, sizeof(\type{MPI\_DOUBLE}) will likely return the size of the pointer to the opaque object and not the desired result which is the number of bytes of storage used by an \type{MPI\_DOUBLE}. The user should use sizof(double) to get this information. Thus, if you have a legacy code which used bytes as message lengths (len\_bytes) and want to convert to the number of elements of double (len\_elements) you should do: $${\rm len\_elements = len\_bytes / sizeof(double)}$$ On the other hand, one can do a sizeof for MPI\_Request and MPI\_Status to allocate arrays of requests and status as discussed later in this chapter. \end{advice} See you next week. Steve From mpi-core-human@mcs.anl.gov Thu Jun 1 18:19:08 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA24608; Thu, 1 Jun 1995 18:19:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA13391 for mpi-core-out; Thu, 1 Jun 1995 17:18:49 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA13379 for ; Thu, 1 Jun 1995 17:18:39 -0500 From: snir@watson.ibm.com Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 9909; Thu, 01 Jun 95 18:17:31 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 8009; Thu, 1 Jun 1995 18:17:30 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Thu, 01 Jun 95 18:17:30 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA23552; Thu, 1 Jun 1995 18:17:28 -0400 Message-Id: <9506012217.AA23552@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: "Lewins, Lloyd J" Cc: mpi-core@mcs.anl.gov, pratap@watson.ibm.com, frankeh@watson.ibm.com Subject: Re: Final version of MPI-1 -- error handling In-Reply-To: (Your message of 31 May 1995 08:24:01 -0800.) Date: Thu, 01 Jun 95 18:17:30 EDT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk : -) :-) :-) *** (-: (-: (-: Date: Thu, 01 Jun 1995 18:17:28 -0400 From: Marc Snir There are three choices for error handling in MPI_WAITALL: 1. after an error has been detected on some communication, MPI_WAITALL should just wait for all other communications to complete, for as long as it takes to wait for normal completion. Problem: An error may have occurred that will prevent that from happening forever. E.g., the process may have posted a send and a receive to a process that has died. The send fails, but MPI_WAITALL does not return, ever, because the receive is still posted. (The receive may have source=dontcare, so that MPI cannot fail the receive.) 2. after an error has occurred, MPI_WAITALL has the option of cancelling the remaining pending communications, so as to be able to return, and return in a mode where all pending communications completed. The error code returned will indicate one of three things: 1. Successful completion 2. Real error 3. Successful cancelation. We need a new error code to indicate the 3rd option 3. after an error has occurred, MPI_WAITALL has the option of returning immediatly, without canceling any of the still active communications. The error code returned will indicate one of 3 things: 1. successful completion 2. real error 3. communication pending (MPI_ERR_PENDING) The difference, from the user viewpoint, between 2 and 3, is that with 2 the user has to post again the Send or Recv operation, whereas with 3 it just has to execute again the Wait or Test. The difference, from the implementation viewpoint, is that error handling in 2 requires more cleanup (cancelling operations). I believe that 1 is not acceptable, since it creates situations where, by definition, there can be no error recovery. 2 is questionable: cancelling communications is expensive; the cancel itself may fail (e.g., in the scenario I outlined before), which brings us back to were we started. 3 has the advantage that, once an error has been detected, then the WAITALL operation can be completed with only very simple local processing (stuffing information in statuses), and the error handler can be invoked asap. I think this is an important goal: namely that, once an error is detected, error recovery can be initiated asap, without requiring any further communication. The current version of the MPI doc is written assuming option 3. But this is an issue that was not dicussed at last meeting. Therefore, I want to flag it, and make sure it is discussed. Btw, the text should make clear that "Pending" does not imply that the operation has not completed (It may complete by the time we check the status). It just means it was pending when the status was updated. Note that there is no race condition: the MPI_ERR_PENDING flag is in the status; when the operation completes, then the request is updated by MPI. Next time the user posts a Wait, it will get the right information in the (new) status. From mpi-core-human@mcs.anl.gov Fri Jun 2 13:42:13 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA09023; Fri, 2 Jun 1995 13:42:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA26925 for mpi-core-out; Fri, 2 Jun 1995 12:38:34 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id MAA26919 for ; Fri, 2 Jun 1995 12:38:26 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 1167; Fri, 02 Jun 95 13:37:20 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 2585; Fri, 2 Jun 1995 13:37:20 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 02 Jun 95 13:37:19 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA30061; Fri, 2 Jun 1995 13:37:18 -0400 Message-Id: <9506021737.AA30061@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Subject: proposal of Llyods on extend and stride Reply-To: snir@watson.ibm.com Date: Fri, 02 Jun 1995 13:37:17 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The proposal comes to solve two problems: 1. We do not have now in MPI a mechanism for inquiring about the "true" lower bound and upper bound of a datatype (location of first and last byte of data, irrepective of MPI_UB or MPI_LB markers). As a result, for example, we cannot support reduce/scan collective functions with user defined datatypes, on top of MPI point-to-point, since we do not know how much space to allocate for temp buffer. Currrent implementations have to use "internal" knowledge about the true extent of a datatype. This is a real problem and a real gap in the current implementation(s) that needs fixing. 2. The current way of setting the "virtual" upper bound and lower bound of a datatype (the bounds that are used for replication in type constructors) is inconvenient: a) The MPI_UB and MPI_LB bounds are sticky, so that one cannot lower the virtual upper bound once it was set by an MPI_UB marker. b) There is no real need for MPI_LB -- it is sufficient (and natural) to manipulate only UB. We need to fix (1) since it is a bug in MPI1, either (preferably) by updating the definition of the functions MPI_TYPE_EXTENT, MPI_TYPE_LB and MPI_TYPE_UB, or (less desireably) by adding 2-3 new functions. We can fix (2) either by changing the way fake upper bounds and fake lower bounds are defined, or by adding a new MPI_SET_VUB(oldtype, ub, newtype) constructor that "erases" all exisitng MPI_UB markers and set a new one (this is the same functionality as the MPI_TYPE_STRIDE proposal of Lloyds) Incidentally, I see no reason not to keep rounding in the definition of extent, so that extent and "stride" (virtual extent) be the same unless the user explicitly redefine "stride" (virtual extent). ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Fri Jun 2 13:42:20 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA09028; Fri, 2 Jun 1995 13:42:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA26955 for mpi-core-out; Fri, 2 Jun 1995 12:40:08 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id MAA26949 for ; Fri, 2 Jun 1995 12:40:02 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 1183; Fri, 02 Jun 95 13:38:23 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 2663; Fri, 2 Jun 1995 13:38:23 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 02 Jun 95 13:38:22 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA18040; Fri, 2 Jun 1995 13:38:20 -0400 Message-Id: <9506021738.AA18040@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Subject: MPI2 proposal Reply-To: snir@watson.ibm.com Date: Fri, 02 Jun 1995 13:38:20 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk -:) -:) -:) -:) -:) Since I shall not be at next meeting, I would like to comment on some of the proposals. 3.3.1 MPI_TYPE_CONIGUOUS_STRUCT. Addresses a real performance issue. Without such construct one cannot send efficiently structures, unless one uses MPI_BYTE, defeating much of the purpose of datatypes. I would like (in addition, if not instead) to the function, as defined, a function that does not take an array_of_displacement argument. After all, the successive components can be assumed to be contiguous, up to rounding. Thus, the array_of_displacement is only for structures with nondefault alignment. 3.3.2 MPI_COMM_CONTEXT Presumes that a communicator is associated with a context. Nowhere in MPI it is said that each communicator is associated with a system-wide unique id, and that each process in the communicator group knows this id. Indeed, there are many optimizations possible when this condition is broken: one can allocate "contexts" (i.e., local communicator id's) more densely, or one can even generate new communicators with no communication. Thus, "context" can only be guaranteed to be locally unique, which will not help much profiling. MPI_GLOBAL_RANK Is superfluous: MPI_GROUP_TRANSLATE_RANK can be used to achieve the same purpose. 3.4 C to Fortran handle translation. A Fortran communicator is attached with Fortran attribute copy and delete functions. C attributes (of type void *) may be 64 bits while Fortran attributes (of type INTEGER) will be 32 bits. It is not clear one can convert C attributes to Fortran attributes without loosing information and it is quite clear one cannot convert the copy and delete functions. Needs more thinking. 3.6 Generalized requests I think it solves the problem of Steve (3.2), but I had not time to write detailed code. 4.1 Alternative proposal for 1-sided communication by Llyod. The proposal makes some inaccurate claims: a) One does not need global, cache coherent shared memory in order to implement put/get. In fact one can implement quite efficiently put/get on a Meiko or a Paragon today. Therefore, put/get does not require "complex hardware, in which all memory is shared", nor does it restrict the total amount of memory accessible by put/get to a total of 4GB on a 32 bit machine. Each processor can have 4GB, and the number of processors is not limited. b) The creation of shared objects, as proposed, implies that put/get can be used only to access data on the heap. I do not think this is an acceptable restriction (certainly not with Fortran 77). c) A shared object always resides in the memory of one process. Thus, in order to allow put/get communication to a distributed array, one needs p collective calls for p processes. d) The declaration of the shared object also specifies its "datatype". Thus one cannot scatter into/ gather from a shared object. This occurs frequently in "real-life" code. Indeed, a main reason to use put/get is to be able to support well data transfers where the layout of the data is known only to the remote site. The idea of being able to restrict put/get to a certain memory area is good. One does not need a new "shared" object for this purpose -- the information can be cached with a communicator. But I would strongly prefer a collective operation where each process exposes an area (or areas) in its memory that can be accessed using a specific communicator. ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Fri Jun 2 19:25:15 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA11656; Fri, 2 Jun 1995 19:25:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA04477 for mpi-core-out; Fri, 2 Jun 1995 18:25:32 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA04471 for ; Fri, 2 Jun 1995 18:25:26 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 7493; Fri, 02 Jun 95 19:23:35 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 4453; Fri, 2 Jun 1995 19:23:34 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 02 Jun 95 19:23:34 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA12379; Fri, 2 Jun 1995 19:23:32 -0400 Message-Id: <9506022323.AA12379@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Subject: Lloyd's proposal on ub/lb/extent Reply-To: snir@watson.ibm.com Date: Fri, 02 Jun 1995 19:23:31 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: I rewrote a close approximation of it in a more formal style, in case people want a real text to vote on... Postscript in next message. From mpi-core-human@mcs.anl.gov Fri Jun 2 19:25:58 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA11661; Fri, 2 Jun 1995 19:25:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA04502 for mpi-core-out; Fri, 2 Jun 1995 18:26:43 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA04496 for ; Fri, 2 Jun 1995 18:26:34 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 7531; Fri, 02 Jun 95 19:25:28 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.0" id 4487; Fri, 2 Jun 1995 19:25:24 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 02 Jun 95 19:25:18 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA19560; Fri, 2 Jun 1995 19:25:14 -0400 Message-Id: <9506022325.AA19560@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Reply-To: snir@watson.ibm.com Mime-Version: 1.0 Content-Type: application/postscript Content-Description: extent.ps Date: Fri, 02 Jun 1995 19:25:13 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk %!PS-Adobe-2.0 %%Creator: dvips 5.47 (RS/6000 1.0) Copyright 1986-91 Radical Eye Software %%Title: temp.dvi %%Pages: 7 1 %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N /def load def /B{bind def}N /S /exch load def /X{S N}B /TR /translate load N /isls false N /vsize 10 N /@rigin{ isls{[0 1 -1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale Resolution VResolution vsize neg mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@letter{/vsize 10 N}B /@landscape{/isls true N /vsize -1 N}B /@a4{/vsize 10.6929133858 N}B /@a3{ /vsize 15.5531 N}B /@ledger{/vsize 16 N}B /@legal{/vsize 13 N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder} N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail} B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get} B /ch-xoff{128 ch-data dup length 3 sub get sub} B /ch-yoff{ ch-data dup length 2 sub get 127 sub} B /ch-dx{ch-data dup length 1 sub get} B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image} imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto}N /eop{clear SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put} for}N /p /show load N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval(Display)eq}{pop false}ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /a{ moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{ S p tail}B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w }B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{clear SS restore}B end %%EndProcSet TeXDict begin 1000 300 300 @start /Fa 27 122 df<000180000300000600000C00001800 00380000700000600000C00001C0000180000380000300000700000600000E00000E00001C0000 1C0000180000380000380000380000300000700000700000700000700000600000600000E00000 E00000E00000E00000E00000E00000600000600000600000700000700000300000300000180000 1800000C0000112E7AA113>40 D<001800000C0000060000060000070000030000030000038000 038000038000018000018000018000018000038000038000038000038000038000038000030000 0700000700000700000600000E00000E00000C00001C00001C0000180000380000300000700000 600000E00000C00001C0000380000300000600000C00001C0000380000600000C00000112E80A1 13>I<001F80007FE000E0E001807003003806603806703806303806607007C0700380600000E0 0001C000078000FE0000FE00000700000700000780000780000780300780780780780780F00F00 C00F00601E00603C007078003FE0001F8000151F7C9D17>51 D<01FFFF8001FFFFE0001E00F000 1E0078001E003C003C001C003C001E003C000E003C000E0078000E0078000E0078000E0078000E 00F0001E00F0001E00F0001E00F0001E01E0003C01E0003C01E0003C01E0007803C0007003C000 F003C000E003C001C0078003C00780078007801E0007807C00FFFFF000FFFFC0001F1F7D9E22> 68 D<01FFF3FFE001FFF3FFE0001E003C00001E003C00001E003C00003C007800003C00780000 3C007800003C007800007800F000007800F000007800F000007800F00000F001E00000FFFFE000 00FFFFE00000F001E00001E003C00001E003C00001E003C00001E003C00003C007800003C00780 0003C007800003C007800007800F000007800F000007800F000007800F0000FFF9FFF000FFF1FF E000231F7D9E22>72 D<0FFFFFF01FFFFFE01E0781E0180780E038078060300F0060300F006060 0F0060600F00C0C01E00C0C01E00C0001E0000001E0000003C0000003C0000003C0000003C0000 0078000000780000007800000078000000F0000000F0000000F0000000F0000001E0000001E000 0001E0000003E00000FFFF0000FFFF00001C1F789E21>84 D<00F18003FDC0078F800E07801C07 803C07803C0700780700780700780700F00E00F00E00F00E00F00E30F01C60F03C60707C6078FC C03FCFC00F078014147C9317>97 D<07803F803F000700070007000E000E000E000E001C001C00 1CF01FFC3F1E3E0E3C0F380F700F700F700F700FE01EE01EE01EE03CE03CE038607071E03FC01F 0010207B9F15>I<007E0001FF000383800F07801E07801C07003C0200780000780000780000F0 0000F00000F00000F00000F00000700200700700381E001FF80007E00011147C9315>I<000078 0003F80003F00000700000700000700000E00000E00000E00000E00001C00001C000F1C003FDC0 078F800E07801C07803C07803C0700780700780700780700F00E00F00E00F00E00F00E30F01C60 F03C60707C6078FCC03FCFC00F078015207C9F17>I<007C0001FF000783000F01801E01803C01 803C0300780E007FFC007FE000F00000F00000F00000F000007000007002007807003C1E001FF8 0007E00011147C9315>I<0000F80001FC0003BC00033C000718000700000700000E00000E0000 0E00000E00000E0001FFE001FFE0001C00001C00001C0000380000380000380000380000380000 700000700000700000700000700000700000E00000E00000E00000E00001C00001C00001C00001 80003380007B8000F300007E00003C00001629829F0E>I<01E0000FE0000FC00001C00001C000 01C000038000038000038000038000070000070000073E00077F000EC3800F81C00F01C00E01C0 1E03801C03801C03801C0380380700380700380700380E18700E30700E30701C60700C60E00FC0 60078015207D9F17>104 D<006000F000F000E000000000000000000000000000000F001F8031 8031C063806380C3800700070007000E000E000E001C301C601C6038C018C01F800F000C1F7D9E 0E>I<03C01FC01F8003800380038007000700070007000E000E000E000E001C001C001C001C00 38003800380038007000700070007180E300E300E300E6007E003C000A207C9F0C>108 D<1E07C0F8003F1FE1FC0033B8730E0063E076070063C03C07006380380700C780780E00070070 0E000700700E000700700E000E00E01C000E00E01C000E00E01C000E00E038601C01C038C01C01 C038C01C01C071801C01C031803803803F001801801E0023147D9325>I<1E07C03F1FE033B870 63E07063C038638038C780700700700700700700700E00E00E00E00E00E00E01C31C01C61C01C6 1C038C1C018C3801F81800F018147D931A>I<007C0001FF000383800F01C01E01C01C01E03C01 E07801E07801E07801E0F003C0F003C0F003C0F00780F00700700F00701E003838001FF00007C0 0013147C9317>I<03C1E007E7F8067E3C0C7C1C0C781E0C701E18E01E00E01E00E01E00E01E01 C03C01C03C01C03C01C07803C07803C07003C0E003E3C0077F80071E000700000700000E00000E 00000E00000E00001C0000FFC000FFC000171D809317>I<1E0F003F3F8033F1C063C1C063C3C0 6383C0C783800700000700000700000E00000E00000E00000E00001C00001C00001C00001C0000 38000018000012147D9313>114 D<00FC03FE07070E0F0E0F0E0E1E000F800FF007F803FC003E 001E701EF01CF01CE03860703FE01F8010147D9313>I<018001C0038003800380038007000700 FFF0FFF00E000E000E000E001C001C001C001C0038003800380038307060706070C071803F001E 000C1C7C9B0F>I<0F00601F80703180E031C0E06380E06380E0C381C00701C00701C00701C00E 03800E03800E03800E038C0E07180E07180E0F180E1F3007F3F003E1E016147D9318>I<0F01C0 1F83C03183E031C1E06380E06380E0C380C00700C00700C00700C00E01800E01800E01800E0300 0E03000E06000E06000F0C0007F80001E00013147D9315>I<0F0060E01F8071E03180E1F031C0 E0F06380E0706380E070C381C0600701C0600701C0600701C0600E0380C00E0380C00E0380C00E 0381800E0381800E0781000E078300070F860007F9FC0001F0F8001C147D931E>I<0787800FCF C018F8E03070E06071E06071E0C0E1C000E00000E00000E00001C00001C00001C00071C060F380 C0F380C0E38180C7C3007CFE00387C0013147D9315>I<0F00601F80703180E031C0E06380E063 80E0C381C00701C00701C00701C00E03800E03800E03800E03800E07000E07000E0F000E1F0007 FE0003EE00000E00000E00001C00781C0078380070700060E0003FC0001F0000141D7D9316>I E /Fb 14 121 df<000078007C7800FC7801FC7803C40003800007800007800007800007800007 8000078000078000078000FFFC78FFFC78FFFC7807807807807807807807807807807807807807 807807807807807807807807807807807807807807807807807807807807807807807807807815 2480A31A>12 D46 D<00C001C007C0FFC0FFC0FBC003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C0FFFFFFFFFFFF10227CA118>49 D<001F00001F00002F00002F00006F0000EF0000CF0001 CF0001CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F00780F00F0 0F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F00000F00000F0015 217FA018>52 D<001F0000001F0000003F8000003F8000003B8000007BC0000073C0000071C000 00F1E00000F1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C 0007803C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF001C000F003C0007803C00 078038000780780003C0780003C0700003C0F00001E0F00001E0E00001E01B237EA220>65 D<07E01FF83FFC3C1E301E200F000F000F000F01FF0FFF3FFF7F0F780FF00FF00FF00FF80F7C3F 7FFF3FFF1F8F10167E9517>97 D<01FC0007FF000FFF801F07803C0180780000780000700000F0 0000F00000F00000F00000F00000F000007800007800007800403C00C01F07C00FFFC007FF8001 FC0012167E9516>99 D<03F00007FC001FFE003E0F003C0780780380780380F001C0FFFFC0FFFF C0FFFFC0F00000F00000F000007000007800007800003C00801F07800FFF8007FF0001F8001216 7E9516>101 D105 D108 D110 D114 D<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F00 0F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12> 116 D<7801F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F80000700000F800 01FC0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F81516809516>120 D E /Fc 20 90 df<7FE07FE0FFC00B037E8A0F>45 D<3078F8787005057C840D>I<000C001C00 FC0FF80F38003800380038003800700070007000700070007000E000E000E000E000E000E001C0 01C001C001C001C001C003C07FFEFFFE0F1E7C9D17>49 D<0001C00003C00003C0000780000F80 001B80003B8000738000638000C7000187000307000707000E07000C0700180E00300E00600E00 E00E00FFFFF0FFFFF0001C00001C00001C00001C00001C00001C00003C0003FFC003FFC0141E7D 9D17>52 D<0000180000003800000038000000780000007C000000FC000000FC000001BC000001 BC0000033C0000033E0000061E0000061E00000C1E00000C1E0000181E0000181F0000300F0000 300F0000600F00007FFF0000FFFF0000C00F000180078001800780030007800300078006000780 060007801F0007C0FFC07FFCFFC07FFC1E207E9F22>65 D<0003F808000FFE18003F0738007801 F801F000F803C000F007C00070078000700F0000701F0000701E0000703E0000603C0000607C00 00007C0000007C0000007C000000F8000000F8000000F8000000F8000000F80000C0780000C078 0001807C0001803C0003003C0003001E0006001E000C000F80180007E0700001FFC000007F0000 1D217B9F21>67 D<07FFFF0007FFFFE0003C01F0003C00F80078007C0078003C0078001E007800 1E0078001E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E01E0 003E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C003C003 C00F8007C03F007FFFFC00FFFFE000201F7E9E23>I<07FFFFF807FFFFF8003C00F8003C007800 7800380078003800780038007800380078003800780C3000F0183000F0180000F0180000F03800 00FFF80000FFF80001E0700001E0300001E0300001E0301801E0303001E0003003C0003003C000 6003C0006003C000E003C001C003C003C007C00FC07FFFFF80FFFFFF801D1F7E9E1F>I<07FFC7 FFC007FFC7FFC0003C007800003C007800007800F000007800F000007800F000007800F0000078 00F000007800F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000FFFFE00001 E003C00001E003C00001E003C00001E003C00001E003C00001E003C00003C007800003C0078000 03C007800003C007800003C007800003C007800007C00F80007FFCFFF800FFF8FFF800221F7E9E 22>72 D<07FFE007FFE0003C00003C0000780000780000780000780000780000780000F00000F0 0000F00000F00000F00000F00001E00001E00001E00001E00001E00001E00003C00003C00003C0 0003C00003C00003C00007C000FFFC00FFFC00131F7F9E10>I<07FC0000FFC007FC0001FFC000 3E0001F800003E00037800006E0003F000006E0006F000006E0006F000006E000CF0000067000C F00000670018F00000C70019E00000C70031E00000C70031E00000C70061E00000C38061E00000 C380C1E000018380C3C00001838183C00001838183C0000181C303C0000181C303C0000181C603 C0000301C60780000301CC0780000301CC0780000300F80780000300F80780000700F00780000F 80F00F80007FF0E0FFF800FFF0E1FFF8002A1F7E9E2A>77 D<07FC03FFC007FC03FFC0003E007C 00003E003800006F003000006F003000006F003000006780300000678030000063C0300000C3C0 600000C3C0600000C1E0600000C1E0600000C0F0600000C0F060000180F0C000018078C0000180 78C00001803CC00001803CC00001803CC00003001F800003001F800003000F800003000F800003 000F800007000780000F800700007FF0030000FFF0030000221F7E9E22>I<0003F800001FFE00 003C1F0000F0078001E003C003C001E0078001E00F8000F00F0000F01F0000F01E0000F83E0000 F83C0000F87C0000F87C0000F87C0000F87C0000F8F80001F0F80001F0F80001F0F80001F0F800 03E0780003E0780007C07C0007C07C000F803C000F003E001E001E003C000F00780007C1F00003 FFC00000FE00001D217B9F23>I<07FFFF0007FFFFC0003C03E0003C01F0007800F0007800F800 7800F8007800F8007800F8007800F800F001F000F001F000F001E000F003C000F00F8000FFFE00 01FFF80001E0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C000 0003C0000003C0000007C000007FFC0000FFFC00001D1F7E9E1F>I<07FFFC0007FFFF00003C07 C0003C03E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F0 078000F01F0000FFFC0000FFF00001E0380001E03C0001E01C0001E01C0001E01E0001E01E0003 C03E0003C03E0003C03E0003C03E0003C03E0603C03E0607C03E0C7FFC1F18FFFC0FF8000003E0 1F207E9E21>82 D<003F04007FCC01E0FC03807C03003C0700380600180E00180E00180E00180E 00000F00000F00000FE00007FE0003FF8001FFC0007FE00007E00001E00000E00000F00000F060 00E06000E06000E06000E07001C0700180F80380FE0F00C7FC0081F80016217D9F19>I<1FFFFF F81FFFFFF81E03C0F83803C038380780383007803870078018600780186007803860078030C00F 0030000F0000000F0000000F0000000F0000000F0000001E0000001E0000001E0000001E000000 1E0000001E0000003C0000003C0000003C0000003C0000003C0000003C0000007C00001FFFF000 3FFFF0001D1F7B9E21>III<7FF803FF80FFF803FF0007C000F80007C000E00003C000C00003E001C00001 E003800001F003000000F006000000F80E000000F80C00000078180000007C300000003C700000 003E600000001EC00000001F800000000F800000000F000000000F000000000F000000000E0000 00001E000000001E000000001E000000001E000000001E000000001C000000003C00000003FFE0 000007FFE00000211F7B9E22>89 D E /Fd 10 58 df<1F003F8060C04040C060C060C060C060 C060C060C060C06060C060C03F801F000B107F8F0F>48 D<18007800F800980018001800180018 00180018001800180018001800FF80FF8009107E8F0F>I<3F007F80F1C0F0E06060006000E000 C00180030006001C0038606060FFC0FFC00B107F8F0F>I<1F003F8071C071C031C001800F800F 0001C000E060E0F0E0F0E0F1C07F801F000B107F8F0F>I<070007000F001F001B003B00330063 00E300FFE0FFE00300030003001FE01FE00B107F8F0F>I<60807F807F007C00600060006F007F 8070C060E000E060E0E0E0E1C07F803F000B107F8F0F>I<07801FC039C061C06000C000DF80FF C0E060C060C060C060606060C03F801F000B107F8F0F>I<60007FE07FE0C0C0C1800180030006 000E000C000C001C001C001C001C001C0008000B117E900F>I<1F003F8061C060C060C079C03F 801F803FC063E0C0E0C060C06060C03F801F000B107F8F0F>I<1F003F8060C0C0C0C060C060C0 60C0E07FE03F60006000C070C071807F003E000B107F8F0F>I E /Fe 49 123 df<007000F001F003C007800F001E001C003C003800780070007000F000E000E000E000E0 00E000E000E000E000F00070007000780038003C001C001E000F00078003C001F000F000700C24 799F18>40 D<6000F00078003C001E000F000780038003C001C001E000E000E000F00070007000 700070007000700070007000F000E000E001E001C003C0038007800F001E003C007800F0006000 0C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007F00007F0 001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D9718>I<3C7E7F7F 7F3F0F0E1E7CF870080C788518>44 D<01F00007FC000FFE001F1F001C07003803807803C07001 C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C07001 C07803C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>48 D<01800380078007800F 803F80FF80FB806380038003800380038003800380038003800380038003800380038003800380 03807FFCFFFE7FFC0F1C7B9B18>I<07F8001FFE003FFF007C0F807003C0F001E0F000E0F000E0 F000E00000E00000E00001E00001C00003C0000780000F00001E00003C0000780000F00003E000 07C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007C0F 807803C07801C03001C00001C00003C0000780000F8003FF0003FE0003FF00000F800003C00001 C00001E00000E00000E0F000E0F001E0F001C0F003C07C0F803FFF001FFE0007F800131C7E9B18 >I<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E00038E0003 8E000306000707000707000707000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F 07F0FF8FF87F07F0151C7F9B18>65 DI<01FCE003FEE007FFE00F 07E01E03E03C01E07800E07000E07000E0F00000E00000E00000E00000E00000E00000E00000E0 0000E00000F000007000E07000E07800E03C01E01E01C00F07C007FF8003FF0001FC00131C7E9B 18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C00701C0070 1C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F80 7FFF00FFFE007FF800141C7F9B18>III<01F9C007FFC00FFFC01F0FC01C03C03C03C07801C07001C07001C0F00000E00000E00000 E00000E00000E00000E00FF0E01FF0E00FF0F001C07001C07003C07803C03C03C01C07C01F0FC0 0FFFC007FDC001F9C0141C7E9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>73 D<7FE000FFE0007FE000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F 9B18>76 DI<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C0 1CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C0 1C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF 00780F00700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C 7D9B18>II<7FF800FFFE007FFF001C0F801C03801C03C01C01C0 1C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F001C07801C03801C03801C0380 1C03801C03801C039C1C039C1C039C7F03FCFF81F87F00F0161C7F9B18>82 D<07F3801FFF803FFF807C1F80700780F00380E00380E00380E00000F000007800003F00001FF0 000FFE0001FF00001F800003C00001E00000E00000E06000E0E000E0E001E0F001C0FC07C0FFFF 80FFFF00E7FC00131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E0703800700000 700000700000700000700000700000700000700000700000700000700000700000700000700000 700000700000700000700007FF0007FF0007FF00151C7F9B18>I II<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003 DC0001F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F0007 07000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>88 DI<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00783E00300F000007 0000070001FF000FFF003FFF007F0700780700F00700E00700E00700F00F00783F007FFFF03FFB F00FE1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E00 0EFF800FFFC00FE3E00F80F00F00700F00780E00380E00380E00380E00380E00380F00380F0078 0F00700F80F00FC3E00FFFC00EFF80067E00151C809B18>I<01FE0007FF001FFF803F07803C03 00780000700000F00000E00000E00000E00000E00000F000007000007801C03C01C03F07C01FFF 8007FF0001FC0012147D9318>I<001F80003F80001F8000038000038000038000038000038003 F3800FFB801FFF803E1F80780F80700780F00780E00380E00380E00380E00380E00380E00780F0 0780700780780F803E3F801FFFF00FFBF803E3F0151C7E9B18>I<03F0000FFC001FFE003E1F00 780780700380F003C0E001C0E001C0FFFFC0FFFFC0FFFFC0F000007000007801C03C01C03F07C0 1FFF8007FF0001FC0012147D9318>I<001FC0007FE000FFE001F1E001C0C001C00001C00001C0 007FFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<03800007C00007C00007 C0000380000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FFFF0011 1D7C9C18>105 D<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>108 D<7DF1F000FFFBF8007FFFFC001F1F1C00 1E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F001914819318>I<7E3F00FEFF807F FFC00FE1E00F80E00F00E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E 00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C0 7001C0E000E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE00 01F00013147E9318>I<7E3E00FEFF807FFFC00FE3E00F80F00F00700F00780E00380E00380E00 380E00380E00380F00380F00780F00700F80F00FC3E00FFFC00EFF800E7E000E00000E00000E00 000E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<7F87E0FF9FF87FBFF803 FC7803F03003E00003C00003C00003C00003800003800003800003800003800003800003800003 80007FFE00FFFF007FFE0015147F9318>114 D<0FF7003FFF007FFF00F81F00E00700E00700F0 07007C00007FF0001FFC0007FE00001F00600780E00380F00380F00780FC0F00FFFF00FFFE00E7 F80011147D9318>I<0180000380000380000380000380007FFFC0FFFFC0FFFFC0038000038000 0380000380000380000380000380000380000380000380400380E00380E00381E003C3C001FFC0 00FF80007E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F07E007FFFC03FFFE01FCFC1714809318 >I119 D<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC 00039E00038E000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E 01C00E03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000 DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F80003F00001E 0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00007C0000F800 01F00003E0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I E /Ff 30 122 df<00C00180030006000E000C001C0018003800300030007000700060006000E0 00E000E000E000E000E000E000E000E000E000E000E00060006000700070003000300038001800 1C000C000E0006000300018000C00A2A7D9E10>40 DI<70F0F8F8 781818183030706040050D7D840C>44 D73 D78 D<003F800000FFE00003E0F80007803C000E000E001E000F003C0007803800038078 0003C0780003C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0 F00001E0780003C0780003C0780003C03C0007803C0007801E000F000F001E0007803C0003E0F8 0000FFE000003F80001B1E7E9C20>I<7FFFFFC07FFFFFC0780F03C0700F01C0600F00C0E00F00 E0C00F0060C00F0060C00F0060C00F0060000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F000003FFFC0003FFFC001B1C7F9B1E>84 DI<0FE0001FF8003C3C003C1E00180E 00000E00001E0007FE001FFE003E0E00780E00F00E00F00E60F00E60F01E60783E603FFFC01F87 8013127F9115>97 DI<03F00FF81E3C383C78187000F000 F000F000F000F000F000780078063C061E0C0FF803E00F127F9112>I<001F80001F8000038000 038000038000038000038000038000038000038000038003E3800FFB801E0F8038078078038070 0380F00380F00380F00380F00380F00380F003807003807803803807801E1F800FFBF007E3F014 1D7F9C17>I<03E00FF01C38381C781E700EFFFEFFFEF000F000F000F000700078063C061E0C0F F803E00F127F9112>I<007801FC039E071E0E0C0E000E000E000E000E000E00FFE0FFE00E000E 000E000E000E000E000E000E000E000E000E000E000E000E007FE07FE00F1D809C0D>I<000380 07E7C00FFDC03C3DC0381C00781E00781E00781E00781E00381C003C3C003FF00037E000700000 7000003000003FFC001FFF003FFF80700780E001C0E001C0E001C0E001C07003803C0F001FFE00 07F800121C7F9215>II<18003C007C003C001800000000 000000000000000000FC00FC001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C00FF80FF80091D7F9C0C>I 108 DII<03F0000FFC001E1E00380700780780700380F003C0F003C0F003C0F003C0F003C0F0 03C07003807807803807001E1E000FFC0003F00012127F9115>II114 D<1F903FF07070E030E030E030F8007F803FE00FF000F8C038C038E038E038F070DFE08FC00D12 7F9110>I<0C000C000C000C000C001C001C003C00FFE0FFE01C001C001C001C001C001C001C00 1C001C301C301C301C301C301E600FC007800C1A7F9910>IIII<7F8FF07F8FF00F07 80070600038E0001DC0001D80000F00000700000780000F80001DC00038E00030E000607000F03 80FF8FF8FF8FF81512809116>II E /Fg 39 122 df<007000E001C003800780 07000E001E001E003C003C003C0078007800780078007000F000F000F000F000F000F000F000F0 00F000F000F000F000700078007800780078003C003C003C001E001E000E0007000780038001C0 00E000700C2E7EA112>40 DI<787878781830 306060E0050A7D830D>44 D66 D<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C00007800007800007800 00F00000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00003C00 003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>I69 D<001FE000FFF801FFFE03E03E07800E0F00001E00 003E00003C00007C0000780000780000780000F00000F00000F00000F00000F00000F00000F003 FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07801E03E03E01FFFE00FF F8001FC017227DA01E>71 DII76 DII<003F000000FFC00003FFF0 0007E1F8000F807C001F003E001E001E003C000F003C000F00780007807800078078000780F000 03C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C078 00078078000780780007803C000F003C000F001E001E001F003E000F807C0007E1F80003FFF000 00FFC000003F00001A227DA021>I I82 D<01FC0007FF800FFFC01F03C0 3C00C03C00007800007800007800007800007800007C00003C00003F00001FE0000FFC0007FE00 01FF00003F800007C00003C00003E00001E00001E00001E00001E00001E00001C0C003C0F007C0 FC0F807FFF001FFE0003F80013227EA019>IIII<780007807C000F003E001F001E001E000F003C000F807C000780780003C0F00003E1 F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000003F0000007F800000 7FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C000F003E001F001E001E000F00 3C000F807C0007C0780003C0F00003E01B207F9F1E>88 DI<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF 3FEF1F8F10147E9316>97 DI<03F0 0FFC1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F010 147E9314>I<000780000780000780000780000780000780000780000780000780000780000780 00078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F00780F00780F00780 F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17>I<03F0000FFC001FFE 003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F00000F000007000007800003C01 003E07001FFF0007FE0001F80011147F9314>I104 DI108 DII<01F80007FE 001FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000F07801E07801 E03C03C03F0FC01FFF8007FE0001F80014147F9317>II< F0E0F3E0F7E0FF00FC00FC00F800F800F000F000F000F000F000F000F000F000F000F000F000F0 000B147D9310>114 D<07F01FFC3FFC780C7800780078007C003FC01FF00FF803F8007C003C00 3CC03CF07CFFF87FF00FC00E147F9311>I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E 001E001E001E001E001E001E001E001E001E001E001E001E201FF00FF007C00C1A7F9910>II<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F80000F00001F0 0001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414809315>120 DI E /Fh 1 1 df0 D E /Fi 3 111 df<03000380030000000000000000001C003E006600C600CC000C000C001800 1980318033003F001C00091480930C>105 D<0060007000600000000000000000038007C00CE0 18C018C000C000C0018001800180018003000300030003006600EE00FC0078000C1A81930E>I< 387C007CFE006F8600CF0600CE06000E06000C06001C0C00180CC01818C0181980301F80300E00 120D808C15>110 D E /Fj 2 50 df<0FC01FE0387070386018E01CE01CE01CE01CE01CE01CE0 1CE01CE01C6018703838701FE00FC00E137F9211>48 D<06001E00FE00EE000E000E000E000E00 0E000E000E000E000E000E000E000E000E00FFE0FFE00B137D9211>I E /Fk 3 104 df0 D<001F003F00F800E001C001C001C001C001 C001C001C001C001C001C001C001C001C001C001C003800700FE00F800FE000700038001C001C0 01C001C001C001C001C001C001C001C001C001C001C001C001C000E000F8003F001F102D7DA117 >102 DI E /Fl 19 121 df<387CFEFEFE7C3807 077C860F>46 D<00700000F0000FF000FFF000F3F00003F00003F00003F00003F00003F00003F0 0003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F00003F0 0003F00003F00003F000FFFF80FFFF80111D7C9C1A>49 D<07F0001FFE00383F007C1F80FE0FC0 FE0FC0FE0FE0FE07E07C07E03807E0000FE0000FC0000FC0001F80001F00003E0000780000F000 00E00001C0600380600700600C00E01FFFE01FFFC03FFFC07FFFC0FFFFC0FFFFC0131D7D9C1A> I69 D<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003 E001FFE00FFFE03F83E07E03E07C03E0F803E0F803E0F803E0FC07E07E0DE03FF9FE07E07E1714 7F9319>97 D<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000 FC0000FC00007C00007E00003E00603F00C01F81C007FF0001FC0013147E9317>99 D<01FE0007FF801F83E03F01F07E00F07E00F8FC00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00 007C00007E00003E00183F00380F807007FFE000FF8015147F9318>101 D<1C003F007F007F007F003F001C00000000000000000000000000FF00FF001F001F001F001F00 1F001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B217EA00E>105 D108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C00 7CFC007EFC007EFC007EFC007EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF 0017147F931A>II114 D<0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE 001FC007C007E007F006F81EFFFCC7F010147E9315>I<01800180018003800380038007800F80 3F80FFFCFFFC0F800F800F800F800F800F800F800F800F800F800F860F860F860F860F8607CC03 F801F00F1D7F9C14>III120 D E /Fm 14 122 df46 D<00180000380000F80007F800FFF800FFF800FFF800F8F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF07FFF F07FFFF014287CA71D>49 D<0007E0000FE0000BE0001BE0001BE0003BE0003BE00073E00073E0 00F3E000F3E001E3E001E3E003E3E003C3E007C3E00F83E00F83E01F03E01F03E03E03E03E03E0 7C03E07C03E0F803E0FFFFFFFFFFFFFFFFFFFFFFFF0003E00003E00003E00003E00003E00003E0 0003E00003E00003E00003E018277EA61D>52 D68 D<01FC000FFF003FFF803FFFC03C07E03003E00001F00001F00001F00001F00001F00003 F001FFF007FFF01FFFF03FC1F07E01F0FC01F0F801F0F801F0F801F0FC03F07E0FF07FFFF03FFF F01FF9F00FE1F0141B7E9A1C>97 D<00007C00007C00007C00007C00007C00007C00007C00007C 00007C00007C00007C00007C00007C00007C00007C00FC7C03FF7C07FFFC0FFFFC1FC1FC3F00FC 3E007C7C007C7C007CFC007CF8007CF8007CF8007CF8007CF8007CF8007CF8007CF8007C7C007C 7C007C7E00FC3F00FC1F83FC1FFFFC0FFF7C03FE7C01F87C162A7EA91E>100 D<00FC0003FF0007FF800FFFC01F87E03F01E03E00F07C00F07C0070F80078FFFFF8FFFFF8FFFF F8FFFFF8F80000F80000F800007800007C00007C00003E00003F00181FC0F80FFFF807FFF801FF E0007F00151B7E9A1A>I105 D112 D114 D<07F8001FFF003FFF807FFF807C07 80F80100F80000F80000F80000FC00007F00007FF0003FFC001FFE0007FF0000FF80001FC0000F C00007C00007C04007C0E007C0FC1F80FFFF807FFF001FFE0007F800121B7E9A16>I<07C00007 C00007C00007C00007C00007C00007C000FFFF80FFFF80FFFF80FFFF8007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C08007E38003FFC003FFC001FF0000FC0012227FA115>I118 D121 D E /Fn 22 123 df<007E01FE07800E001E003C003C0078007FF87FF8F0 00F000F00070007000700038001C180FF807E00F147E9312>15 D<70F8F8F87005057C840D>58 D<70F8FCFC7C0C0C0C181838306040060E7C840D>I<0FFFFFFC1FFFFFF81E03C0781803C03838 03C01830078018300780186007801860078030C00F0030C00F0030000F0000000F0000001E0000 001E0000001E0000001E0000003C0000003C0000003C0000003C00000078000000780000007800 000078000000F0000000F0000000F0000001F000007FFFC0007FFFC0001E1F7F9E1B>84 D<00F18003FDC0078F800E07801C07803C07803C0700780700780700780700F00E00F00E00F00E 00F00E18F01C30F03C30707C3078FC603FCFC00F078015147E9318>97 D<07803F803F00070007 0007000E000E000E000E001C001C001CF01FFC3F1E3E0E3C0F380F700F700F700F700FE01EE01E E01EE03CE03CE038607071E03FC01F0010207E9F14>I<0000780003F80003F000007000007000 00700000E00000E00000E00000E00001C00001C000F1C003FDC0078F800E07801C07803C07803C 0700780700780700780700F00E00F00E00F00E00F00E18F01C30F03C30707C3078FC603FCFC00F 078015207E9F18>100 D<007C01FE07870E031E033C033C06781C7FF87FC0F000F000F000F000 700070027007381E1FF807E010147E9315>I<00007C0000FE00019E00039E00030C0007000007 00000700000700000E00000E00000E0000FFF001FFF0000E00001C00001C00001C00001C00001C 0000380000380000380000380000380000700000700000700000700000700000E00000E00000E0 0000E00000C00001C00031C000798000F300007E00003C000017297E9F16>I<007000F000F000 60000000000000000000000000000007001F8039C031C061C061C0C38003800380070007000E00 0E000E181C301C301C601CE00FC007000D1F7F9E10>105 D<01E0000FE0000FC00001C00001C0 0001C0000380000380000380000380000700000700000701C00707E00E0C300E18700E30F00E60 F01CC0E01F80001F80001FE00038F000383800383800381C307038607038607038407018C0E01F 80600F0014207E9F18>107 D<03C01FC01F8003800380038007000700070007000E000E000E00 0E001C001C001C001C0038003800380038007000700070007180E300E300E300E6007E003C000A 207E9F0E>I<0F03E03E001F8FF0FF0031DC39C38061F03B038061E01E01C061C01C01C0C3C03C 03800380380380038038038003803803800700700700070070070007007007000700700E0C0E00 E00E180E00E01C180E00E01C300E00E00C701C01C00FE00C00C0078026147F9328>I<0F07C000 1F8FE00031D8700061F0380061E0380061C03800C3C070000380700003807000038070000700E0 000700E0000700E0000701C1800E01C3000E0383000E0386000E018E001C01FC000C00F0001914 7F931B>I<007C0001FF000383800F01C01E01C01C01E03C01E07801E07801E07801E0F003C0F0 03C0F003C0F00780F00700700F00701E003838001FF00007C00013147E9316>I<03C1E007E7F8 0C7E3C187C1C18781E18701E30E01E00E01E00E01E00E01E01C03C01C03C01C03C01C07803C078 03C07003C0E003E3C0077F80071E000700000700000E00000E00000E00000E00001C0000FFC000 FFC000171D819317>I<007C01FE0387070F070F070E0F00078007F803FC01FE001E000F700FF0 0EF00EE01C70383FF01FC010147E9315>115 D<00C000E001C001C001C001C003800380FFF8FF F807000700070007000E000E000E000E001C001C001C001C1838303830386038E01FC00F000D1C 7F9B10>I<0780301FC03838C07030E07060E07061C070C1C0E00380E00380E00380E00701C007 01C00701C00701C3070386070386070786070F8C03F9F801F0F018147F931A>I<07C3C00FE7E0 187C703038706038F06038F0C070E000700000700000700000E00000E00000E00070E030F1C060 F1C060E1C0C0E3E1807E7F003C3E0014147E931A>120 D<0780301FC03838C07030E07060E070 61C070C1C0E00380E00380E00380E00701C00701C00701C00701C0070380070380070780070F80 03FF0001F700000700000700000E003C0C003C1C0038380030F0001FE0000F8000151D7F9316> I<01E06003F0C007FFC00E1F800C0300000600000C0000180000300000600000C0000180000300 000600C00C01801803803F87007FFE0061FC00C0700013147E9315>I E /Fo 19 122 df45 D66 D<000000007FFC00000E0000000FFFFFC0001E0000007FFFFFF8003E000003FFFFFFFE007E0000 0FFFFFFFFF80FE00003FFFF8007FC1FE0000FFFF80000FF3FE0003FFFC000003FFFE0007FFF000 0001FFFE000FFFC00000007FFE001FFF800000003FFE003FFF000000001FFE007FFC000000000F FE00FFFC000000000FFE01FFF80000000007FE03FFF00000000003FE03FFE00000000003FE07FF E00000000001FE0FFFC00000000001FE0FFFC00000000000FE1FFF800000000000FE1FFF800000 0000007E3FFF8000000000007E3FFF0000000000007E3FFF0000000000003E7FFF000000000000 3E7FFF0000000000003E7FFF0000000000003E7FFE000000000000007FFE00000000000000FFFE 00000000000000FFFE00000000000000FFFE00000000000000FFFE00000000000000FFFE000000 00000000FFFE00000000000000FFFE00000000000000FFFE00000000000000FFFE000000000000 00FFFE00000000000000FFFE00000000000000FFFE00000000000000FFFE000000000000007FFE 000000000000007FFE000000000000007FFF000000000000007FFF0000000000003E7FFF000000 0000003E3FFF0000000000003E3FFF0000000000003E3FFF8000000000003E1FFF800000000000 7E1FFF8000000000007C0FFFC000000000007C0FFFC00000000000FC07FFE00000000000FC03FF E00000000000F803FFF00000000001F801FFF80000000003F000FFFC0000000003F0007FFE0000 000007E0003FFF000000000FC0001FFF800000001F80000FFFC00000003F000007FFF0000000FE 000003FFFC000001FC000000FFFF80000FF80000003FFFF8007FE00000000FFFFFFFFFC0000000 03FFFFFFFF00000000007FFFFFFC00000000000FFFFFE00000000000007FFE00000047497AC754 >I80 D<0003FFF0000000003FFFFF000000 00FFFFFFE0000001FFFFFFF8000003FF003FFC000007FF800FFE000007FF8007FF000007FF8003 FF800007FF8003FFC00007FF8001FFC00007FF8001FFC00003FF0000FFE00001FE0000FFE00000 FC0000FFE00000000000FFE00000000000FFE00000000000FFE00000000000FFE00000000000FF E000000000FFFFE00000007FFFFFE0000007FFFFFFE000003FFFFCFFE00000FFFE00FFE00003FF F000FFE00007FFC000FFE0000FFF0000FFE0001FFE0000FFE0003FFC0000FFE0007FFC0000FFE0 007FF80000FFE000FFF80000FFE000FFF00000FFE000FFF00000FFE000FFF00000FFE000FFF000 01FFE000FFF00001FFE000FFF80003FFE0007FFC0007FFE0007FFC000FFFE0003FFE001F7FF800 1FFF80FE7FFFF007FFFFFC3FFFF003FFFFF01FFFF0007FFFC00FFFF00007FE0003FFF0342E7DAD 38>97 D<007FC000000000FFFFC000000000FFFFC000000000FFFFC000000000FFFFC000000000 FFFFC00000000003FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001 FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000 000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00FFE000001FF C0FFFFC00001FFC3FFFFF00001FFCFFFFFFC0001FFFFE01FFF0001FFFF0007FF8001FFFC0001FF C001FFF80000FFE001FFF00000FFF001FFE000007FF001FFC000007FF801FFC000003FFC01FFC0 00003FFC01FFC000003FFC01FFC000001FFE01FFC000001FFE01FFC000001FFE01FFC000001FFF 01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000 001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01 FFC000001FFE01FFC000001FFE01FFC000003FFE01FFC000003FFC01FFC000003FFC01FFC00000 3FF801FFC000007FF801FFE000007FF001FFF00000FFE001FFF80001FFE001FFFC0003FFC001FF 7F0007FF0001FE3FC03FFE0001FC0FFFFFF80001FC03FFFFF00001F800FFFF800001F0001FF800 0038487CC741>I<00001FFF80000001FFFFF8000007FFFFFE00001FFFFFFF00007FF801FF8000 FFE003FFC003FFC003FFC007FF8003FFC007FF0003FFC00FFE0003FFC01FFE0003FFC01FFC0001 FF803FFC0000FF003FFC00007E007FFC000000007FF8000000007FF800000000FFF800000000FF F800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF80000 0000FFF800000000FFF800000000FFF800000000FFF8000000007FFC000000007FFC000000007F FC000000003FFC000000003FFE000000001FFE000001F01FFF000001F00FFF000003F007FF8000 07E007FFC00007C003FFE0000FC000FFF8003F80007FFF00FF00001FFFFFFC000007FFFFF00000 01FFFFC00000001FFC00002C2E7CAD34>I<00000000003F000000FFF000FFC00007FFFE03FFE0 003FFFFFCFFFF0007FFFFFFFCFF001FFE07FFE0FF003FF801FFC1FF007FF000FFE0FF00FFE0007 FF0FF00FFE0007FF07E01FFC0003FF81801FFC0003FF80003FFC0003FFC0003FFC0003FFC0003F FC0003FFC0003FFC0003FFC0003FFC0003FFC0003FFC0003FFC0003FFC0003FFC0003FFC0003FF C0003FFC0003FFC0001FFC0003FF80001FFC0003FF80000FFE0007FF00000FFE0007FF000007FF 000FFE000003FF801FFC000001FFE07FF8000003FFFFFFE0000003FFFFFFC000000787FFFE0000 000780FFF0000000078000000000000F8000000000000F8000000000000FC000000000000FC000 000000000FE000000000000FF8000000000007FFFFFFF0000007FFFFFFFF000007FFFFFFFFE000 03FFFFFFFFF80003FFFFFFFFFC0001FFFFFFFFFE0000FFFFFFFFFF0003FFFFFFFFFF800FFFFFFF FFFF801FF000001FFFC03FC0000001FFC07F800000007FE07F800000003FE0FF000000003FE0FF 000000001FE0FF000000001FE0FF000000001FE0FF000000001FE0FF800000003FE07F80000000 3FC07FC00000007FC03FE0000000FF801FF8000003FF000FFE00000FFE0007FFE000FFFC0001FF FFFFFFF000003FFFFFFF8000000FFFFFFE000000007FFFC0000034447DAE3A>103 D<007FC000000000FFFFC000000000FFFFC000000000FFFFC000000000FFFFC000000000FFFFC0 0000000003FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000000000 01FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001 FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC007FE000001FFC03FFF C00001FFC0FFFFF00001FFC1FFFFFC0001FFC7F01FFC0001FFCFC00FFE0001FFDF0007FF0001FF DE0007FF0001FFFC0007FF0001FFF80003FF8001FFF80003FF8001FFF00003FF8001FFE00003FF 8001FFE00003FF8001FFE00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF80 01FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC000 03FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001 FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003 FF80FFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFF3848 7BC741>I<00FC0001FF0003FF8007FF800FFFC00FFFC00FFFE00FFFE00FFFE00FFFC00FFFC007 FF8003FF8001FF0000FC0000000000000000000000000000000000000000000000000000000000 0000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF18497CC820>I<007FC0 00FFFFC000FFFFC000FFFFC000FFFFC000FFFFC00003FFC00001FFC00001FFC00001FFC00001FF C00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FF C00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FFFFFF80FFFFFF80 FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<00FF8003FF000003FF000000FFFF801FFFE0 001FFFE00000FFFF807FFFF8007FFFF80000FFFF81FFFFFC01FFFFFC0000FFFF83F01FFE03F01F FE0000FFFF87C007FF07C007FF000003FF8F0007FF8F0007FF800001FF9E0003FF9E0003FF8000 01FFBC0003FFBC0003FF800001FFF80001FFF80001FFC00001FFF80001FFF80001FFC00001FFF0 0001FFF00001FFC00001FFE00001FFE00001FFC00001FFE00001FFE00001FFC00001FFE00001FF E00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FF C00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FF C00001FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FF FFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80592E7BAD62>I<00FF8007FE 0000FFFF803FFFC000FFFF80FFFFF000FFFF81FFFFFC00FFFF87F01FFC00FFFF8FC00FFE0003FF 9F0007FF0001FF9E0007FF0001FFBC0007FF0001FFF80003FF8001FFF80003FF8001FFF00003FF 8001FFE00003FF8001FFE00003FF8001FFE00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF80 01FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC000 03FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001 FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003 FF8001FFC00003FF80FFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFFFFFF FF81FFFFFF382E7BAD41>I<00000FFE0000000001FFFFF000000007FFFFFC0000001FFFFFFF00 00007FF803FFC00000FFE000FFE00001FF80003FF00003FF00001FF80007FE00000FFC000FFE00 000FFE000FFC000007FE001FFC000007FF003FFC000007FF803FF8000003FF803FF8000003FF80 7FF8000003FFC07FF8000003FFC07FF8000003FFC0FFF8000003FFE0FFF8000003FFE0FFF80000 03FFE0FFF8000003FFE0FFF8000003FFE0FFF8000003FFE0FFF8000003FFE0FFF8000003FFE0FF F8000003FFE0FFF8000003FFE07FF8000003FFC07FF8000003FFC07FF8000003FFC07FF8000003 FFC03FFC000007FF803FFC000007FF801FFC000007FF001FFC000007FF000FFE00000FFE0007FF 00001FFC0003FF00001FF80001FFC0007FF00000FFE000FFE000007FFC07FFC000001FFFFFFF00 000007FFFFFC00000001FFFFF0000000001FFF000000332E7DAD3A>I<007FC00FFE0000FFFFC0 FFFFC000FFFFC3FFFFF000FFFFCFFFFFFC00FFFFFFE03FFF00FFFFFF000FFF8003FFFC0007FFC0 01FFF80003FFE001FFF00001FFF001FFE00000FFF001FFC00000FFF801FFC000007FFC01FFC000 007FFC01FFC000003FFC01FFC000003FFE01FFC000003FFE01FFC000003FFE01FFC000001FFF01 FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC00000 1FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FFC000001FFF01FF C000003FFE01FFC000003FFE01FFC000003FFE01FFC000007FFC01FFC000007FFC01FFC000007F F801FFC00000FFF801FFE00000FFF001FFF00001FFE001FFF80003FFE001FFFC0007FFC001FFFF 000FFF0001FFFFC07FFE0001FFCFFFFFF80001FFC3FFFFF00001FFC0FFFF800001FFC01FF80000 01FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001 FFC00000000001FFC00000000001FFC00000000001FFC000000000FFFFFF80000000FFFFFF8000 0000FFFFFF80000000FFFFFF80000000FFFFFF8000000038427CAD41>I<00FF807F00FFFF81FF C0FFFF87FFF0FFFF8FFFF8FFFF9F8FFCFFFF9F1FFE03FFBE1FFE01FFBC1FFE01FFF81FFE01FFF8 1FFE01FFF01FFE01FFF00FFC01FFE007F801FFE003F001FFE0000001FFE0000001FFE0000001FF C0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001 FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC00000 01FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC000 00FFFFFFE000FFFFFFE000FFFFFFE000FFFFFFE000FFFFFFE000272E7CAD2F>114 D<0001F000000001F000000001F000000001F000000001F000000001F000000003F000000003F0 00000003F000000007F000000007F000000007F00000000FF00000000FF00000001FF00000003F F00000003FF00000007FF0000001FFF0000003FFF000000FFFFFFFC0FFFFFFFFC0FFFFFFFFC0FF FFFFFFC0FFFFFFFFC0007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000 007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF000 00007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF0 0000007FF00000007FF001F0007FF001F0007FF001F0007FF001F0007FF001F0007FF001F0007F F001F0007FF001F0007FF001F0007FF003F0003FF803E0003FF807E0001FFC07C0001FFE0F8000 0FFFFF800003FFFE000000FFFC0000001FE00024427EC12D>116 D<007FC00000FF80FFFFC001 FFFF80FFFFC001FFFF80FFFFC001FFFF80FFFFC001FFFF80FFFFC001FFFF8003FFC00007FF8001 FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003 FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FF C00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF 8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00007FF8001FFC00007FF80 01FFC00007FF8001FFC0000FFF8001FFC0001FFF8000FFC0003FFF8000FFE0007FFF80007FE000 FBFFC0007FF803F3FFFF003FFFFFE3FFFF000FFFFF83FFFF0003FFFE03FFFF00007FF003FFFF38 2E7BAD41>I121 D E /Fp 8 117 df<00001E000000003E00000000FE00000003FE0000003FFE 0000FFFFFE0000FFFFFE0000FFFFFE0000FFCFFE0000000FFE0000000FFE0000000FFE0000000F FE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE000000 0FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000 000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE00 00000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000F FE00007FFFFFFF807FFFFFFF807FFFFFFF807FFFFFFF80213879B730>49 D<0000001FFE0000E0000003FFFFE001E000001FFFFFF803E000007FFFFFFE07E00001FFFC00FF 0FE00007FFC0001FDFE0000FFF000007FFE0003FFC000001FFE0007FF0000000FFE000FFE00000 007FE001FFC00000003FE003FF800000001FE007FF800000001FE007FF000000000FE00FFE0000 000007E00FFE0000000007E01FFC0000000007E01FFC0000000003E03FFC0000000003E03FF800 00000003E07FF80000000001E07FF80000000001E07FF80000000001E07FF0000000000000FFF0 000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FF F0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000 FFF0000000000000FFF0000000000000FFF00000000000007FF00000000000007FF80000000000 007FF80000000001E07FF80000000001E03FF80000000001E03FFC0000000001E01FFC00000000 01E01FFC0000000003E00FFE0000000003C00FFE0000000007C007FF0000000007C007FF800000 000F8003FF800000000F8001FFC00000001F0000FFE00000003E00007FF00000007C00003FFC00 0000F800000FFF000003F0000007FFC0000FE0000001FFFC007FC00000007FFFFFFF000000001F FFFFFC0000000003FFFFE000000000001FFE0000003B3D7BBB46>67 D<003FFF00000001FFFFE0 000007FFFFFC00000FF007FE00001FF801FF80001FF800FFC0001FF8007FE0001FF8007FE0001F F8007FF0000FF0003FF00007E0003FF00003C0003FF0000000003FF0000000003FF0000000003F F0000000003FF0000000FFFFF000000FFFFFF000007FF83FF00003FF803FF00007FE003FF0001F FC003FF0003FF8003FF0007FF0003FF0007FE0003FF000FFE0003FF000FFC0003FF000FFC0003F F000FFC0003FF000FFC0007FF000FFC0007FF000FFE000FFF0007FE001DFF0003FF003DFFC001F FC0F9FFFE00FFFFE0FFFE001FFF807FFE0003FE001FFE02B267DA52F>97 D<0003FF8000001FFFF000007FFFFC0000FF83FF0003FE00FF8007FC003F800FF8003FC01FF800 1FE01FF0001FE03FF0001FF03FF0000FF07FE0000FF07FE0000FF87FE0000FF8FFE0000FF8FFE0 000FF8FFFFFFFFF8FFFFFFFFF8FFFFFFFFF8FFE0000000FFE0000000FFE0000000FFE0000000FF E00000007FE00000007FE00000007FF00000003FF00000783FF00000781FF80000F80FF80000F0 07FC0001F003FE0003E001FF000FC000FFC07F80003FFFFE00000FFFFC000001FFC00025267DA5 2C>101 D<00FF0000000000FFFF0000000000FFFF0000000000FFFF0000000000FFFF00000000 0007FF000000000003FF000000000003FF000000000003FF000000000003FF000000000003FF00 0000000003FF000000000003FF000000000003FF000000000003FF000000000003FF0000000000 03FF000000000003FF000000000003FF000000000003FF000000000003FF000000000003FF0000 00000003FF003FE0000003FF01FFFC000003FF07FFFE000003FF0F81FF800003FF3C00FF800003 FF3800FFC00003FF7000FFC00003FFE0007FE00003FFC0007FE00003FFC0007FE00003FF80007F E00003FF80007FE00003FF00007FE00003FF00007FE00003FF00007FE00003FF00007FE00003FF 00007FE00003FF00007FE00003FF00007FE00003FF00007FE00003FF00007FE00003FF00007FE0 0003FF00007FE00003FF00007FE00003FF00007FE00003FF00007FE00003FF00007FE00003FF00 007FE00003FF00007FE00003FF00007FE00003FF00007FE00003FF00007FE00003FF00007FE000 03FF00007FE000FFFFFC1FFFFF80FFFFFC1FFFFF80FFFFFC1FFFFF80FFFFFC1FFFFF80313C7DBB 36>104 D<00FF00FF8000FFFF0FFFF800FFFF3FFFFE00FFFFFE03FF00FFFFF000FFC007FFE000 7FE003FFC0003FF003FF80003FF803FF00001FF803FF00001FFC03FF00000FFC03FF00000FFE03 FF00000FFE03FF00000FFE03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF0000 07FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF00000FFE03 FF00000FFE03FF00000FFE03FF00000FFC03FF00001FFC03FF00001FF803FF80003FF003FFC000 7FF003FFE000FFE003FFF001FF8003FFFE07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003 FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000 000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000FF FFFC000000FFFFFC000000FFFFFC000000FFFFFC00000030377EA536>112 D<00FE01F800FFFE07FF00FFFE1FFF80FFFE3E3FC0FFFE787FE007FE707FE003FEE07FE003FEE0 7FE003FFC07FE003FFC03FC003FF801F8003FF800F0003FF80000003FF80000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF000000 03FF00000003FF00000003FF000000FFFFFE0000FFFFFE0000FFFFFE0000FFFFFE000023267EA5 28>114 D<000F0000000F0000000F0000000F0000000F0000001F0000001F0000001F0000001F 0000003F0000003F0000007F0000007F000000FF000001FF000003FF000007FF00001FFFFFF0FF FFFFF0FFFFFFF0FFFFFFF003FF000003FF000003FF000003FF000003FF000003FF000003FF0000 03FF000003FF000003FF000003FF000003FF000003FF000003FF000003FF000003FF000003FF00 0003FF000003FF000003FF003C03FF003C03FF003C03FF003C03FF003C03FF003C03FF003C03FF 003C01FF807801FF807800FFC0F000FFE1F0003FFFE0000FFF800001FE001E377EB626>116 D E /Fq 58 123 df<000FC3E0007FEFF000E07C7801C0F8780380F83007007000070070000700 700007007000070070000700700007007000FFFFFF80FFFFFF8007007000070070000700700007 007000070070000700700007007000070070000700700007007000070070000700700007007000 0700700007007000070070007FE3FF007FE3FF001D20809F1B>11 D<001F80007FC001E0E00381 E00381E00701E0070000070000070000070000070000070000FFFFE0FFFFE00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E07FC3FE7FC3FE1720809F19>I<7038F87CFC7EFC7E7C3E0C060C060C06180C180C381C301860 3040200F0E7E9F17>34 D<006000C001800300070006000C001C00180038003800300070007000 700060006000E000E000E000E000E000E000E000E000E000E000E000E000600060007000700070 0030003800380018001C000C00060007000300018000C000600B2E7DA112>40 DI<0006000000060000000600000006000000 060000000600000006000000060000000600000006000000060000000600000006000000060000 00060000FFFFFFF0FFFFFFF0000600000006000000060000000600000006000000060000000600 0000060000000600000006000000060000000600000006000000060000000600001C207D9A23> 43 D<70F8FCFC7C0C0C0C181838306040060E7C840D>II<70F8F8 F87005057C840D>I<03F0000FFC001E1E001C0E00380700780780700380700380700380F003C0 F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0700380 7003807003807807803807001C0E001E1E000FFC0003F000121F7E9D17>48 D<018003801F80FF80E38003800380038003800380038003800380038003800380038003800380 038003800380038003800380038003800380FFFEFFFE0F1E7C9D17>I<03F0000FFC00183E0030 0F00600F80C00780F007C0F807C0F803C0F803C02003C00007C0000780000780000F00000E0000 1C0000380000700000E00001C0000380000300000600C00C00C01800C03001807FFF80FFFF80FF FF80121E7E9D17>I<03F0000FFC001C1E00300F00780F80780F80780780380F80000F80000F00 000F00001E00003C0003F80003F000001C00000E00000F000007800007800007C02007C0F807C0 F807C0F807C0F00780600F80700F003C1E001FFC0003F000121F7E9D17>I<000E00000E00001E 00003E00003E00006E0000EE0000CE00018E00038E00030E00060E000E0E000C0E00180E00380E 00300E00600E00E00E00FFFFF0FFFFF0000E00000E00000E00000E00000E00000E00000E0000FF E000FFE0141E7F9D17>I<007C0001FE000783000E03800C07801C078038030038000078000070 0000700000F3F800F7FE00FC0E00F80700F80380F80380F003C0F003C0F003C0F003C0F003C070 03C07003C07003803803803807001C07000E1E0007FC0001F000121F7E9D17>54 D<03F0000FFC001E1E003807003007007003807003807003807003807807003E07003F0E001FDC 000FF80003F80007FC001E7E00383F00700F80700780E003C0E001C0E001C0E001C0E001C0F001 807003803807001E0E000FFC0003F000121F7E9D17>56 D<03F00007F8001E1C00380E00380700 700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007C0380FC01C0FC0 1FFBC007F3C0000380000380000780000700300700780E00780E00701C003078001FF0000FC000 121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<7FFFFFE0FFFF FFF00000000000000000000000000000000000000000000000000000000000000000FFFFFFF07F FFFFE01C0C7D9023>61 D<0003800000038000000380000007C0000007C0000007C000000DE000 000DE000000DE0000018F0000018F0000018F00000307800003078000030780000603C0000603C 0000603C0000E01E0000C01E0000FFFE0001FFFF0001800F0001800F0003800F80030007800300 0780070007C0070003C00F8003C0FFE03FFEFFE03FFE1F207F9F22>65 D<001FC040007FF0C001 F839C003C00DC0078007C00F0003C01E0003C03E0001C03C0001C07C0001C07C0000C0780000C0 F80000C0F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000780000 C07C0000C07C0000C03C0000C03E0001801E0001800F0003000780030003C00E0001F81C00007F F000001FC0001A217D9F21>67 D69 DI72 DI76 DII80 D82 D<7FFFFFE07FFFFFE0780F01E0700F00E0600F00 60600F0060E00F0070C00F0030C00F0030C00F0030C00F0030000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F000007FFFE0007FFFE001C1F7E9E21>84 D87 D<7FF83FF87FF83FF807C00F8003C0060001E00E0001F00C0000F018000078380000 7C3000003C7000003E6000001EC000000FC000000F8000000780000007C0000007E000000DE000 001DF0000018F8000038780000307C0000603C0000E01E0000C01F0001800F0003800780030007 C00F8007C0FFE01FFEFFE01FFE1F1F7F9E22>I<0804180C3018703860306030C060C060C060F8 7CFC7EFC7E7C3E381C0F0E7B9F17>92 D<1FE0003FF000783800781C00300E00000E00000E0000 0E0003FE001FFE003E0E00780E00F80E00F00E30F00E30F00E30F01E30783F703FEFE00F878014 147E9317>97 D<0E0000FE0000FE00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E3F000EFF800FC3C00F00E00E00F00E00700E00780E00780E00780E00780E00780E00 780E00780E00700E00F00F00F00F01E00FC3C00CFF800C3E0015207F9F19>I<03F80FFC1E1E3C 1E380C78007000F000F000F000F000F000F0007000780038033C031F0E0FFC03F010147E9314> I<000380003F80003F8000038000038000038000038000038000038000038000038000038003E3 800FFB801E0F803C0780780380780380700380F00380F00380F00380F00380F00380F003807003 807003807807803807801E1F800FFBF803E3F815207E9F19>I<03F0000FFC001E1E003C0F0038 0700780700700380F00380FFFF80FFFF80F00000F00000F000007000007800003801801C03800F 070007FE0001F80011147F9314>I<003C00FE01CF038F03060700070007000700070007000700 FFF0FFF007000700070007000700070007000700070007000700070007000700070007007FF07F F01020809F0E>I<0001E003E3F00FFF701C1C703C1E00380E00780F00780F00780F00780F0038 0E003C1E001C1C003FF80033E0003000003000003800003FFE001FFF803FFFC07003E07000F0E0 0070E00070E00070F000F07801E03E07C00FFF0003FC00141F7F9417>I<0E0000FE0000FE0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000EFF000FC3800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C0FFE7FCFFE7FC16207F9F19>I<1C001E003E001E001C000000000000000000000000000E 007E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC0FFC0 0A1F809E0C>I<0E00FE00FE000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE0FFE00B20809F0C> 108 D<0E3F03F000FEFF8FF800FFC1DC1C000F80F80E000F00F00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E00FFE7FE7FE0FFE7FE7FE023147F9326>I<0E3E00FEFF00FF C3800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C0FFE7FCFFE7FC16147F9319>I<01F80007FE001E07803C03C03801C07000E0 7000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07801E03801C03C03C01E078007FE00 01F80014147F9317>I<0E3F00FEFF80FFC3C00F01E00E00F00E00F00E00780E00780E00780E00 780E00780E00780E00780E00700E00F00F00F00F01E00FC3C00EFF800E3E000E00000E00000E00 000E00000E00000E00000E0000FFE000FFE000151D7F9319>I<03E1800FF9801E1F803C078078 0780780380700380F00380F00380F00380F00380F00380F003807003807803807807803C0F801E 1F800FF38003E380000380000380000380000380000380000380000380003FF8003FF8151D7E93 18>I<0E78FEFCFF9E0F1E0F0C0F000E000E000E000E000E000E000E000E000E000E000E000E00 FFE0FFE00F147F9312>I<1F903FF07070E030E030E030F00078007F803FE00FF000F0C078C038 C038E038E038F070DFE08F800D147E9312>I<06000600060006000E000E001E003E00FFF8FFF8 0E000E000E000E000E000E000E000E000E000E000E180E180E180E180E18073007E003C00D1C7F 9B12>I<0E01C0FE1FC0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E03C00E03C0070FC007FDFC01F1FC16147F9319>III<7FC7FC7FC7FC0703E0 07038003830001C70000EE0000EC00007800003800003C00007C0000EE0001C700018700030380 0701C00F01E0FF87FEFF87FE1714809318>II<3FFF3FFF 380E301C703C6038607060F001E001C0038007830F030E031C073C063806701EFFFEFFFE10147F 9314>I E /Fr 47 122 df<70F8FCFC7C0C0C0C0C181830306040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F00007FC000E0E001C07003803 803803807803C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001 E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C03803803803801C07 000E0E0007FC0001F00013227EA018>48 D<018003800F80FF80F3800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003800380 0380FFFEFFFE0F217CA018>I<03F0000FFC001C1F00300F806007806003C0C003C0F003E0F801 E0F801E0F801E02003E00003E00003C00003C0000780000780000F00001E00001C000038000070 0000E00001C0000380000700600E00600C00601800E03000C07FFFC0FFFFC0FFFFC013217EA018 >I<03F0000FFC001C1F003007807007807807C07807C07803C03807C00007C000078000078000 0F00000E00003C0003F80003F800001E00000F000007800007C00003C00003E02003E07003E0F8 03E0F803E0F003C06003C06007803007801C1F000FFC0003F00013227EA018>I<000E00000E00 001E00001E00003E00003E00006E0000EE0000CE0001CE00018E00030E00070E00060E000E0E00 0C0E00180E00180E00300E00700E00600E00E00E00FFFFF8FFFFF8000E00000E00000E00000E00 000E00000E00000E0001FFF001FFF015217FA018>I<1000801C07801FFF001FFE001FFC001FF0 0018000018000018000018000018000018000018F8001BFE001F0F001C07801803801803C00001 C00001E00001E00001E00001E06001E0F001E0F001E0F001C0C003C06003C06003803007001C1E 000FFC0003F00013227EA018>I<007E0001FF0003C1800701C00E03C01C03C01C018038000038 0000780000700000702000F1FC00F3FE00F60700FC0380F803C0F801C0F001C0F001E0F001E0F0 01E0F001E0F001E07001E07001E07001E03801C03803C01803801C07000E0E0007FC0001F00013 227EA018>I<01F00007FC000E0F001807803803803001C07001C07001C07001C07801C0780380 3E03803F07001FCE000FFC0003F80003FC000F7F001C3F80380F807007C07003C0E001E0E000E0 E000E0E000E0E000E0E000E07001C07001C03803801E0F000FFE0003F00013227EA018>56 D<01F00007FC000E0E001C0700380380780380700380F001C0F001C0F001C0F001E0F001E0F001 E0F001E0F001E07001E07003E07803E03807E01C0DE00FF9E007F1E00081C00001C00003C00003 80000380300700780700780E00701C003838001FF0000FC00013227EA018>I<000180000003C0 000003C0000003C0000007E0000007E0000007E000000FF000000CF000000CF000001CF8000018 78000018780000383C0000303C0000303C0000601E0000601E0000601E0000C00F0000C00F0000 C00F0001FFFF8001FFFF8001800780030003C0030003C0030003C0060001E0060001E0060001E0 0E0000F01F0001F0FFC00FFFFFC00FFF20237EA225>65 D<000FE010003FF83000F81C7001E006 7003C003F0078001F00F0000F01E0000F03E0000703C0000707C0000707C000030780000307800 0030F8000030F8000000F8000000F8000000F8000000F8000000F8000000F80000007800003078 0000307C0000307C0000303C0000603E0000601E0000600F0000C0078000C003C0018001E00300 00F80E00003FF800000FE0001C247DA223>67 D69 DI73 D<03FFF003FFF0000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F 00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F 00700F00F80F00F80F00F80F00F01E00601C003878001FF00007C00014237EA119>I77 DI80 D82 D<03F0200FFC601C0EE03803 E07001E07001E0E000E0E000E0E00060E00060E00060F00000F000007800007F00003FF0001FFE 000FFF0003FF80003FC00007E00001E00000F00000F0000070C00070C00070C00070C00070E000 60E000E0F000C0F801C0EF0380C7FF0081FC0014247DA21B>I<7FFFFFF87FFFFFF87C0780F870 0780386007801860078018E007801CC007800CC007800CC007800CC007800CC007800C00078000 000780000007800000078000000780000007800000078000000780000007800000078000000780 0000078000000780000007800000078000000780000007800000078000000780000007800003FF FF0003FFFF001E227EA123>I<0FE0001FF8003C1C003C0E00180700000700000700000F0003FF 000FFF003F07007C0700780700F00700F00718F00718F00F18780F187C3FB83FF3F00FC3C01515 7E9418>97 D<0E0000FE0000FE00001E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E1F800E7FE00FC0F00F00780E00380E003C0E001C0E001E0E001E0E001E 0E001E0E001E0E001E0E001E0E001C0E003C0F00380F80700FC1F00C7FC00C1F0017237FA21B> I<01FE0007FF000F07801C0780380300780000700000F00000F00000F00000F00000F00000F000 00F000007800007800C03C00C01E01800F030007FE0001F80012157E9416>I<0000E0000FE000 0FE00001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E003F0E007 FEE01F07E03C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E070 00E07800E03801E03C03E01E0EF00FFCFE03F0FE17237EA21B>I<01FC0007FF000F07801C03C0 3801C07801E07000E0FFFFE0FFFFE0F00000F00000F00000F00000F000007800007800603C0060 1E00C00F838007FF0000FC0013157F9416>I<003C00FE01CF038F038F07000700070007000700 0700070007000700FFF8FFF8070007000700070007000700070007000700070007000700070007 000700070007007FF87FF8102380A20F>I<0000F001F1F807FFB80F1F381E0F001C07003C0780 3C07803C07803C07803C07801C07001E0F000F1E001FFC0019F0001800001800001C00001FFF00 0FFFC01FFFE03801F0700070E00038E00038E00038E000387000707800F01E03C00FFF8001FC00 15217F9518>I<0E0000FE0000FE00001E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E1F800E7FC00FC1E00F80F00F00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FFFFE7FF18237FA21B >I<1C001E003E001E001C00000000000000000000000000000000000E00FE00FE001E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC0FFC00A227FA10E>I<01 C003E003E003E001C00000000000000000000000000000000001E00FE00FE001E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060 C0F1C0F1807F003E000B2C82A10F>I<0E0000FE0000FE00001E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E0FFC0E0FFC0E07E00E03800E07000E0E000E18 000E30000E78000EF8000F9C000F1E000E0E000E07000E07800E03C00E01C00E01E00E01F0FFE3 FEFFE3FE17237FA21A>I<0E00FE00FE001E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FF E0FFE00B237FA20E>I<0E1FC07F00FE7FE1FF80FEC0F303C01F807E01E00F003C00E00E003800 E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFEFFE3FF8FFE2715 7F942A>I<0E1F80FE7FC0FFC1E01F80F00F00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E0070FFE7FFFFE7FF18157F941B>I<01FC00 07FF000F07801C01C03800E07800F0700070F00078F00078F00078F00078F00078F00078F00078 7000707800F03800E01C01C00F078007FF0001FC0015157F9418>I<0E1F80FE7FE0FFC1F00F00 780E00780E003C0E003C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0F00 780F80700FC1F00E7FC00E1F000E00000E00000E00000E00000E00000E00000E00000E0000FFE0 00FFE000171F7F941B>I<0E3CFEFEFFCF1F8F0F060F000E000E000E000E000E000E000E000E00 0E000E000E000E000E00FFF0FFF010157F9413>114 D<0F883FF87078E038E018E018E018F000 7F003FE01FF001F8003CC01CC01CE01CE01CF018F878DFF08FC00E157E9413>I<060006000600 060006000E000E000E001E003E00FFF8FFF80E000E000E000E000E000E000E000E000E000E000E 0C0E0C0E0C0E0C0E0C0E08071803F001E00E1F7F9E13>I<0E0070FE07F0FE07F01E00F00E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E01F0 07037803FE7F01F87F18157F941B>III121 D E /Fs 20 118 df45 D68 D73 D77 D80 D<007F802001FFE06007C0F8600F001CE01E000FE03C0007E0380003E07800 01E0700001E0700000E0F00000E0F00000E0F0000060F0000060F0000060F8000060F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF00000FFF80 0000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F8C0000078C00000 78C0000078C0000078C0000078E0000078E0000070F00000F0F00000E0F80000E0FC0001C0FE00 0380E7800700C1F01E00C07FFC00800FF0001D337CB125>83 D<01FE00000FFFC0001C03E0003C 00F0003E0078003E003C003E001C0008001E0000001E0000001E0000001E0000001E000001FE00 003FFE0001FF1E0007F01E000F801E001F001E003E001E007C001E007C001E00F8001E0CF8001E 0CF8001E0CF8003E0CF8003E0C7C007E0C3E00DF1C1F038FB80FFF07F003F803C01E1F7D9E21> 97 D<003FC000FFF803E01C07801E0F003E1E003E1E003E3C00087C00007C0000780000F80000 F80000F80000F80000F80000F80000F80000F80000F800007C00007C00007C00003E00031E0003 1F00060F800E07C01C03F03800FFE0003F80181F7D9E1D>99 D<000001E000003FE000003FE000 0003E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0 000001E0000001E0000001E0000001E0000001E0000001E0003F81E000FFE1E003E079E007800D E00F0007E01E0003E01E0001E03C0001E07C0001E07C0001E0780001E0F80001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07C0001E07C0001E03C 0001E03E0003E01E0007E00F000FE007801DE003E071F001FFE1FF003F01FF20327DB125>I<00 3F800000FFE00003E0F80007803C000F001E001E001E001E000F003C000F007C000F007C000F80 78000780F8000780FFFFFF80FFFFFF80F8000000F8000000F8000000F8000000F8000000F80000 00780000007C0000003C0000003C0001801E0001800F0003000F80060007C00E0001F03C0000FF F000001FC000191F7E9E1D>I<0003E0001FF8003C3800707C00F07C01E07C01E03803C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0 007FFF007FFF0016327FB114>I<000000F0007F03F801FFCF1C03C1FC1C0780F8180F0078001F 007C001E003C003E003E003E003E003E003E003E003E003E003E003E003E003E003E001E003C00 1F007C000F0078000780F0000FC1E0000DFFC0001C7F000018000000180000001C0000001C0000 001E0000000FFFF8000FFFFF0007FFFF800FFFFFC01E0007E0380001F07000007070000078E000 0038E0000038E0000038E0000038E000003870000070780000F03C0001E01E0003C007C01F0001 FFFC00003FE0001E2F7E9F21>I<07000F801F801F800F80070000000000000000000000000000 0000000000000000000780FF80FF800F8007800780078007800780078007800780078007800780 07800780078007800780078007800780078007800780078007800FC0FFF8FFF80D307EAF12> 105 D<0781FE003FC000FF87FF80FFF000FF8E07C1C0F8000F9803E3007C0007B001E6003C0007 E000FC001E0007C000F8001E0007C000F8001E0007C000F8001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E000780 00F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0781FE0000FF87FF8000FF8E07C0000F9803E00007B001E00007E000F00007C000F00007C000 F00007C000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF80 211F7E9E25>I<001FC00000FFF80001E03C0007800F000F0007801E0003C01E0003C03C0001E0 3C0001E0780000F0780000F0780000F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000 F8F80000F8F80000F8780000F07C0001F03C0001E03C0001E01E0003C01E0003C00F00078007C0 1F0001F07C0000FFF800001FC0001D1F7E9E21>I<0783E0FF8FF8FF9C7C0FB07C07F07C07E038 07C00007C00007C000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000780000780000780000FC000FFFE00FFFE00161F7E 9E19>114 D<01FC100FFF301E03F03800F0700070E00070E00030E00030E00030F00030F80000 7E00003FF0001FFF000FFF8003FFE0003FF00003F8000078C0003CC0003CC0001CE0001CE0001C F0001CF00018F80038FC0070EF01E0C3FFC080FE00161F7E9E1A>I<00C00000C00000C00000C0 0000C00001C00001C00001C00003C00003C00007C0000FC0001FC000FFFFE0FFFFE003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C03003C03003C03003C03003C03003C03003C03003C03001E06001E06000F0C0007F 80001F00142C7FAB19>I<078000F000FF801FF000FF801FF0000F8001F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078001F000078001F000078003F000038007F00003C00EF00001F03CF800 00FFF0FF80003FC0FF80211F7E9E25>I E /Ft 5 85 df<00000000600000000000E000000000 00E00000000001E00000000001F00000000003F00000000003F00000000007F0000000000FF000 0000000FF0000000001BF0000000001BF00000000033F00000000033F00000000063F000000000 63F000000000C3F000000001C3F00000000183F00000000303F00000000303F00000000603F800 00000603F80000000C01F80000000C01F80000001801F80000003801F80000003001F800000060 01F80000006001F8000000C001F8000000C001F80000018001F8000001FFFFF8000003FFFFF800 00070001F80000060001F800000C0001F800000C0001F80000180001FC0000180001FC00003000 00FC0000300000FC0000600000FC0000E00000FC0000C00000FC0001C00000FC0003C00000FC00 0FE00003FC00FFFC003FFFE0FFF8003FFFE02B337CB234>65 D<000FFFFFFC00000FFFFFFF8000 007F000FC000007E0003E000007E0001F000007E0000F800007E00007C0000FC00007C0000FC00 003E0000FC00003E0000FC00003E0001F800001E0001F800001F0001F800001F0001F800001F00 03F000001F0003F000001F0003F000001F0003F000001F0007E000003F0007E000003F0007E000 003F0007E000003F000FC000003E000FC000007E000FC000007E000FC000007E001F8000007C00 1F800000FC001F800000FC001F800000F8003F000001F8003F000001F0003F000003F0003F0000 03E0007E000007E0007E000007C0007E00000F80007E00000F0000FC00001F0000FC00003E0000 FC00007C0000FC0000F80001F80001F00001F80007C00001F8000F800003F8007E0000FFFFFFF8 0000FFFFFFC0000030317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007F0003F800007E0000 F800007E00007800007E00007800007E0000300000FC0000300000FC0000300000FC0000300000 FC0000300001F80000300001F80000300001F80000300001F80000600003F00180600003F00180 000003F00180000003F00180000007E00300000007E00300000007E00700000007E01F0000000F FFFE0000000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00180000003F00000000003F00000000007E00000000007E 00000000007E00000000007E0000000000FC0000000000FC0000000000FC0000000000FC000000 0001F80000000001F80000000001F80000000003F800000000FFFFF0000000FFFFF00000002D31 7BB02E>70 D<000FFFFFE000000FFFFFFC0000007F003F0000007E000F8000007E0007C000007E 0003E000007E0003F00000FC0001F00000FC0001F00000FC0001F00000FC0001F00001F80003F0 0001F80003F00001F80003F00001F80003E00003F00007E00003F00007C00003F0000FC00003F0 001F800007E0003F000007E0007C000007E000F8000007E007E000000FFFFF8000000FFFFF0000 000FC00FC000000FC007E000001F8003F000001F8001F000001F8001F800001F8001F800003F00 01F800003F0001F800003F0001F800003F0001F800007E0003F000007E0003F000007E0003F000 007E0003F00000FC0007E00000FC0007E00000FC0007E00000FC0007E01801F80007E03001F800 07E03001F80007E06003F80003E060FFFFE001F1C0FFFFC000FF80000000003E002D327BB033> 82 D<07FFFFFFFFF007FFFFFFFFF00FE007F007F00F8007E001F00E0007E000E00C0007E000E0 1C0007E000E018000FC000E018000FC000E030000FC000C030000FC000C030001F8000C060001F 8000C060001F8000C060001F8000C0C0003F00018000003F00000000003F00000000003F000000 00007E00000000007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC 0000000000FC0000000001F80000000001F80000000001F80000000001F80000000003F0000000 0003F00000000003F00000000003F00000000007E00000000007E00000000007E00000000007E0 000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000000001F80000000 001F80000000003F800000007FFFFF8000007FFFFF8000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300 TeXDict begin %%EndSetup %%Page: 0 1 bop 799 911 a Ft(D)25 b(R)g(A)g(F)g(T)225 1002 y Fs(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1196 y Fr(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)841 1323 y(June)h(2,)g(1995)77 1381 y(This)h(w)o(ork)f(w)o(as)h(supp)q(orted)g (in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h(ASC-9310330,)i (the)192 1439 y(National)d(Science)f(F)l(oundation)i(Science)e(and)i(T)l(ec)o (hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1497 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h(Commission)e(of)j(the)f(Europ)q(ean)i (Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 bop 166 49 a Fq(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g(run)g(of)g (a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF)f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 bop 75 361 a Fp(Chapter)31 b(1)75 568 y Fo(P)m(oin)m(t-to-P)m(oin)m(t)39 b(Comm)m(u)o(nication)75 791 y Fn(:)8 b(:)g(:)75 933 y Fm(1.4)59 b(Derived)19 b(datat)n(yp)r(es)75 1035 y Fn(:)8 b(:)g(:)166 1091 y Fq(The)15 b Fl(exten)o(t)f Fq(of)g(a)g(datat)o(yp)q(e)g(is)h (de\014ned)g(to)f(b)q(e)h(the)g(span)f(from)g(the)g(\014rst)g(b)o(yte)h(to)e (the)i(last)f(b)o(yte)75 1147 y(o)q(ccupied)19 b(b)o(y)d(en)o(tries)h(in)h (this)f(datat)o(yp)q(e,)f(rounded)h(up)g(to)f(satisfy)h(alignmen)o(t)g (requiremen)o(ts.)25 b(That)75 1204 y(is,)15 b(if)189 1294 y Fn(T)6 b(y)r(pemap)12 b Fq(=)h Fk(f)p Fq(\()p Fn(ty)r(pe)562 1301 y Fj(0)580 1294 y Fn(;)8 b(disp)685 1301 y Fj(0)703 1294 y Fq(\))p Fn(;)g(:::;)g Fq(\()o Fn(ty)r(pe)903 1301 y Fi(n)p Fh(\000)p Fj(1)966 1294 y Fn(;)g(disp)1071 1301 y Fi(n)p Fh(\000)p Fj(1)1134 1294 y Fq(\))p Fk(g)p Fn(;)75 1384 y Fq(then)283 1473 y Fn(l)q(b)p Fq(\()p Fn(T)e(y)r(pemap)p Fq(\))40 b(=)h(min)689 1500 y Fi(j)742 1473 y Fn(disp)826 1480 y Fi(j)843 1473 y Fn(;)272 1560 y(ub)p Fq(\()p Fn(T)6 b(y)r(pemap)p Fq(\))40 b(=)h(max)693 1586 y Fi(j)743 1560 y Fq(\()p Fn(disp)845 1567 y Fi(j)872 1560 y Fq(+)10 b Fn(siz)r(eof)5 b Fq(\()p Fn(ty)r(pe)1149 1567 y Fi(j)1167 1560 y Fq(\)\))10 b(+)g Fn(\017;)23 b Fq(and)189 1647 y Fn(extent)p Fq(\()p Fn(T)6 b(y)r(pemap)p Fq(\))42 b(=)f Fn(ub)p Fq(\()p Fn(T)6 b(y)r(pemap)p Fq(\))j Fk(\000)h Fn(l)q(b)p Fq(\()p Fn(T)c(y)r(pemap)p Fq(\))p Fn(:)528 b Fq(\(1.1\))75 1737 y(If)17 b Fn(ty)r(pe)206 1744 y Fi(i)237 1737 y Fq(requires)g(alignmen)o (t)g(to)f(a)h(b)o(yte)f(address)h(that)f(is)h(is)g(a)f(m)o(ultiple)j(of)d Fn(k)1474 1744 y Fi(i)1487 1737 y Fq(,)h(then)g Fn(\017)g Fq(is)g(the)g (least)75 1793 y(nonnegativ)o(e)f(incremen)o(t)g(needed)g(to)f(round)g Fn(extent)p Fq(\()p Fn(T)6 b(y)r(pemap)p Fq(\))16 b(to)e(the)h(next)h(m)o (ultiple)h(of)d(max)1794 1800 y Fi(i)1815 1793 y Fn(k)1839 1800 y Fi(i)1853 1793 y Fq(.)75 1894 y Fl(Example)k(1.1)k Fq(Assume)17 b(that)e Fn(T)6 b(y)r(pe)14 b Fq(=)g Fk(f)p Fq(\()p Fg(double)q Fn(;)8 b Fq(0\))p Fn(;)g Fq(\()p Fg(cha)o(r)n Fn(;)g Fq(8\))p Fk(g)14 b Fq(\(a)h Fg(double)j Fq(at)d(displacemen)o(t)i(zero,)75 1950 y(follo)o(w)o(ed)h(b)o(y)g(a)f Fg(cha)o(r)g Fq(at)g(displacemen)o(t)j (eigh)o(t\).)27 b(Assume,)18 b(furthermore,)f(that)g(doubles)i(ha)o(v)o(e)e (to)g(b)q(e)75 2007 y(strictly)d(aligned)g(at)e(addresses)i(that)e(are)h(m)o (ultiples)h(of)f(eigh)o(t.)19 b(Then,)14 b(the)f(exten)o(t)g(of)f(this)i (datat)o(yp)q(e)e(is)75 2063 y(16)e(\(9)g(rounded)h(to)f(the)h(next)g(m)o (ultiple)h(of)f(8\).)17 b(A)11 b(datat)o(yp)q(e)f(that)g(consists)h(of)f(a)g (c)o(haracter)g(immediately)75 2120 y(follo)o(w)o(ed)16 b(b)o(y)f(a)g(double) h(will)h(also)e(ha)o(v)o(e)g(an)g(exten)o(t)g(of)g(16.)166 2220 y Fn(:)8 b(:)g(:)166 2277 y Fq(The)15 b(follo)o(wing)h(auxiliary)h (functions)f(pro)o(vide)f(useful)i(information)e(on)g(deriv)o(ed)h(datat)o (yp)q(es.)75 2428 y Fg(MPI)p 160 2428 14 2 v 16 w(TYPE)p 293 2428 V 17 w(EXTENT\(datat)o(yp)q(e,)g(extent\))117 2505 y Ff(IN)155 b Fg(datat)o(yp)q(e)424 b Ff(datat)o(yp)q(e)14 b(\(handle\))117 2579 y(OUT)108 b Fg(extent)471 b Ff(datat)o(yp)q(e)14 b(exten)o(t)h(\(in)o (teger\))75 2704 y Fe(int)23 b(MPI)p 245 2704 15 2 v 17 w(Type)p 358 2704 V 17 w(extent\(MPI)p 615 2704 V 16 w(Datatype)f(datatype,)h(MPI)p 1156 2704 V 17 w(Aint)g(*extent\))-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 2 4 bop 75 -100 a Fq(2)631 b Fc(CHAPTER)15 b(1.)35 b(POINT-TO-POINT)17 b(COMMUNICA)l(TION)75 49 y Fe(MPI)p 150 49 15 2 v 17 w(TYPE)p 263 49 V 16 w(EXTENT\(DATATYPE,)22 b(EXTENT,)h(IERROR\))170 106 y(INTEGER)g(DATATYPE,)g(EXTENT,)g(IERROR)75 287 y Fg(MPI)p 160 287 14 2 v 16 w(TYPE)p 293 287 V 17 w(LB\()14 b(datat)o(yp)q(e,)j (displacement\))117 364 y Ff(IN)155 b Fg(datat)o(yp)q(e)424 b Ff(datat)o(yp)q(e)14 b(\(handle\))117 439 y(OUT)108 b Fg(displacement)342 b Ff(displacemen)o(t)14 b(of)f(lo)o(w)o(er)h(b)q(ound)g(from)e(datat)o(yp)q (e)j(origin,)d(in)905 495 y(b)o(ytes)j(\(in)o(teger\))75 620 y Fe(int)23 b(MPI)p 245 620 15 2 v 17 w(Type)p 358 620 V 17 w(lb\(MPI)p 519 620 V 16 w(Datatype)g(datatype,)g(MPI)p 1061 620 V 16 w(Aint*)h(displacement\))75 706 y(MPI)p 150 706 V 17 w(TYPE)p 263 706 V 16 w(LB\()g(DATATYPE,)e(DISPLACEMENT,)h(IERROR\))170 763 y(INTEGER)g(DATATYPE,)g(DISPLACEMENT,)f(IERROR)75 944 y Fg(MPI)p 160 944 14 2 v 16 w(TYPE)p 293 944 V 17 w(UB\()15 b(datat)o(yp)q(e,)h(displacement\))117 1021 y Ff(IN)155 b Fg(datat)o(yp)q(e) 424 b Ff(datat)o(yp)q(e)14 b(\(handle\))117 1096 y(OUT)108 b Fg(displacement)342 b Ff(displacemen)o(t)12 b(of)g(upp)q(er)i(b)q(ound)e (from)f(datat)o(yp)q(e)i(origin,)e(in)905 1153 y(b)o(ytes)k(\(in)o(teger\))75 1277 y Fe(int)23 b(MPI)p 245 1277 15 2 v 17 w(Type)p 358 1277 V 17 w(ub\(MPI)p 519 1277 V 16 w(Datatype)g(datatype,)g(MPI)p 1061 1277 V 16 w(Aint*)h(displacement\))75 1363 y(MPI)p 150 1363 V 17 w(TYPE)p 263 1363 V 16 w(UB\()g(DATATYPE,)e(DISPLACEMENT,)h (IERROR\))170 1420 y(INTEGER)g(DATATYPE,)g(DISPLACEMENT,)f(IERROR)166 1506 y Fn(:)8 b(:)g(:)75 1628 y Fb(1.4.1)49 b(Arti\014cial)17 b(extent)75 1714 y Fq(It)i(is)h(sometimes)f(con)o(v)o(enien)o(t)h(to)e(asso)q (ciate)i(an)f(\\arti\014cial")h(exten)o(t)f(to)f(a)h(datat)o(yp)q(e)g(whic)o (h)h(di\013ers)75 1770 y(from)d(the)i(one)f(giv)o(en)h(b)o(y)f(Equation)g (1.1)f(on)h(page)h(1.)28 b(This,)19 b(in)g(order)f(to)g(mo)q(dify)g(the)h(b)q (eha)o(vior)f(of)75 1827 y(datat)o(yp)q(e)g(constructors.)29 b(F)l(or)18 b(example,)i(a)e(C)h(compiler)h(ma)o(y)d(allo)o(w)i(the)g(user)g (to)f(o)o(v)o(eride)g(default)75 1883 y(alignmen)o(t)h(rules)g(for)f(some)g (of)g(the)h(structures)f(within)h(a)f(program.)29 b(In)19 b(suc)o(h)f(a)h (case,)f(a)g(datat)o(yp)q(e)75 1940 y(built)c(to)f(describ)q(e)h(this)g (strucure)f(will)i(not)d(ha)o(v)o(e)h(its)g(upp)q(er)h(b)q(ound)g(de\014ned)g (in)g(a)f(useful)h(manner:)19 b(the)75 1996 y(concatenation)g(of)g(suc)o(h)g (structures)g(built)i(using)e Fg(MPI)p 1068 1996 14 2 v 16 w(TYPE)p 1201 1996 V 17 w(CONTIGUOUS)i Fq(will)f(not)f(yield)i(the)75 2053 y(righ)o(t)c(la)o(y)o(out)g(for)g(an)h(arra)o(y)e(of)h(these)h (structures.)26 b(T)l(o)18 b(a)o(v)o(oid)f(this)h(problem,)g(the)g(user)f(w)o (ould)h(ha)o(v)o(e)75 2109 y(to)13 b(use)i(the)f Fg(MPI)p 369 2109 V 16 w(TYPE)p 502 2109 V 17 w(HVECTOR)h Fq(datat)o(yp)q(e)f (constructor.)k(Alternativ)o(ely)l(,)e(one)e(can)g(asso)q(ciate)g(an)75 2166 y(arti\014cial)h(exten)o(t)e(with)h(the)g(datat)o(yp)q(e.)19 b(W)l(e)14 b(shall)h(use)f(the)f(follo)o(wing)i(notation:)k(the)13 b Fa(natur)n(al)h Fq(b)q(ounds)75 2222 y(and)k(exten)o(t)g(of)g(a)f(datat)o (yp)q(e)h(are)g(as)f(de\014ned)j(in)f(Equation)f(1.1.)27 b(The)19 b Fl(constructiv)o(e)f Fq(b)q(ounds)h(and)75 2278 y(exten)o(t)d(of)h(a)f (datatat)o(yp)q(e)f(are)i(the)f(v)m(alues)i(used)f(b)o(y)g(datat)o(yp)q(e)f (constructors.)23 b(Normally)l(,)17 b(these)g(t)o(w)o(o)75 2335 y(coincide.)j(Ho)o(w)o(ev)o(er,)11 b(the)f(user)h(can)g(explicitly)j (reset)c(them,)h(using)h(the)e(function)i Fg(MPI)p 1571 2335 V 16 w(TYPE)p 1704 2335 V 16 w(SETUB)p Fq(,)75 2391 y(b)q(elo)o(w.)21 b(This)16 b(will)h(c)o(hange)e(the)h(constructiv)o(e)f(upp)q(er)i(b)q(ound)f (of)f(the)g(datat)o(yp)q(e)g(and)g(accordingly)l(,)i(its)75 2448 y(constructiv)o(e)e(exten)o(t.)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 bop 75 -100 a Fc(1.4.)29 b(DERIVED)15 b(D)o(A)l(T)l(A)l(TYPES)1169 b Fq(3)75 49 y Fg(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(SET)p 393 49 V 16 w(CUB\(t)o(yp)q(e,)16 b(ub\))117 126 y Ff(INOUT)62 b Fg(datat)o(yp)q(e)424 b Ff(datat)o(yp)q(e)14 b(\(handle\))117 202 y(IN)155 b Fg(ub)541 b Ff(new)18 b(constructiv)o(e)h (upp)q(er)f(b)q(ound)f(for)g(datat)o(yp)q(e)g(in)g(b)o(ytes)905 258 y(\(in)o(teger\))75 383 y Fe(int)23 b(MPI)p 245 383 15 2 v 17 w(Type)p 358 383 V 17 w(cub\(MPI)p 543 383 V 16 w(Datatype)g(*type,)g (MPI)p 1013 383 V 17 w(Aint)g(ub\))75 469 y(MPI)p 150 469 V 17 w(TYPE)p 263 469 V 16 w(CUB\()h(TYPE,)f(UB,)g(IERROR\))170 525 y(INTEGER)g(TYPE,)h(UB,)f(IERROR)166 612 y Fq(The)11 b(constructiv)o(e)g (upp)q(er)g(b)q(ound)h(and)f(the)f(constructiv)o(e)h(exten)o(t)f(of)h(a)f (datat)o(yp)q(e)g(can)h(b)q(e)g(querried)75 668 y(using)16 b(the)f(functions)75 819 y Fg(MPI)p 160 819 14 2 v 16 w(TYPE)p 293 819 V 17 w(CUB\()g(datat)o(yp)q(e,)h(displacement\))117 896 y Ff(IN)155 b Fg(datat)o(yp)q(e)424 b Ff(datat)o(yp)q(e)14 b(\(handle\))117 972 y(OUT)108 b Fg(displacement)342 b Ff(displacemen)o(t)9 b(of)f(constructiv)o(e)j(upp)q(er)f(b)q(ound)g(from)d(datat)o(yp)q(e)905 1028 y(origin,)12 b(in)i(b)o(ytes)h(\(in)o(teger\))75 1153 y Fe(int)23 b(MPI)p 245 1153 15 2 v 17 w(Type)p 358 1153 V 17 w(cub\(MPI)p 543 1153 V 16 w(Datatype)g(datatype,)f(MPI)p 1084 1153 V 17 w(Aint*)h(displacement\))75 1239 y(MPI)p 150 1239 V 17 w(TYPE)p 263 1239 V 16 w(CUB\()h(DATATYPE,)e(DISPLACEMENT,)g (IERROR\))170 1295 y(INTEGER)h(DATATYPE,)g(DISPLACEMENT,)f(IERROR)75 1476 y Fg(MPI)p 160 1476 14 2 v 16 w(TYPE)p 293 1476 V 17 w(CEXTENT\(datat)o (yp)q(e,)16 b(extent\))117 1554 y Ff(IN)155 b Fg(datat)o(yp)q(e)424 b Ff(datat)o(yp)q(e)14 b(\(handle\))117 1629 y(OUT)108 b Fg(extent)471 b Ff(datat)o(yp)q(e)14 b(constructiv)o(e)i(exten)o(t)e(\(in)o(teger\))75 1753 y Fe(int)23 b(MPI)p 245 1753 15 2 v 17 w(Type)p 358 1753 V 17 w(cextent\(MPI)p 639 1753 V 16 w(Datatype)f(datatype,)h(MPI)p 1180 1753 V 17 w(Aint)g(*extent\))75 1840 y(MPI)p 150 1840 V 17 w(TYPE)p 263 1840 V 16 w(CEXTENT\(DATATYPE,)f(EXTENT,)h(IERROR\))170 1896 y(INTEGER)g(DATATYPE,)g(EXTENT,)g(IERROR)166 1983 y Fq(Note)14 b(that)g(the)g(constructiv)o(e)h(lo)o(w)o(er)f(b)q(ound)h(alw)o(a)o(ys)f (coincide)j(with)d(the)h(natural)f(lo)o(w)o(er)g(b)q(ound.)166 2039 y(All)19 b(datat)o(yp)q(e)e(constructors)f(are)i(using)g(the)f (constructiv)o(e)h(b)q(ounds)h(and)e(exten)o(ts)g(of)g(the)h(com-)75 2095 y(p)q(onen)o(t)e(datat)o(yp)q(es.)75 2202 y Fl(Example)i(1.2)k Fa(T)m(r)n(ansp)n(ose)15 b(of)h(3D)h(matrix)g(\(c)n(ould)f(b)n(e)f(e)n(asily) h(done)g(with)h(Hve)n(ctor\))75 2296 y Fe(REAL)23 b(a\(n1,n2,n3\),)g (B\(n1,n2,n3\))75 2352 y(INTEGER)g(sizeofreal,)f(ierr,)h(onerow,)g(oneplane,) g(xpose)75 2465 y(CALL)g(MPI_TYPE_EXTENT\(MPI_REAL,)e(sizeofreal,)h(ierr\))75 2521 y(CALL)h(MPI_TYPE_VECTOR\(n3,)f(1,)h(n1*n2,)g(MPI_REAL,)g(onerow,)g (ierr\))75 2578 y(CALL)g(MPI_TYPE_SET_CUB\(onerow,)e(sizeofreal,)h(ierr\))75 2634 y(CALL)h(MPI_TYPE_VECTOR\(n2,)f(1,)h(n1,)h(onerow,)f(oneplane,)f(ierr\)) 75 2691 y(CALL)h(MPI_TYPE_SET_CUB\(oneplane,)e(sizeofreal,)h(ierr\))-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 bop 75 -100 a Fq(4)631 b Fc(CHAPTER)15 b(1.)35 b(POINT-TO-POINT)17 b(COMMUNICA)l(TION)75 49 y Fe(CALL)23 b(MPI_TYPE_VECTOR\(n1,)f(1,)h(1,)h (oneplane,)e(xpose,)h(ierr\))75 106 y(CALL)g(MPI_TYPE_COMMIT\(xpose,)e (ierr\))75 162 y(CALL)i(MPI_SENDRECV_REPLACE\(a,)e(1,)j(xpose,)f(0,)g(0,)h (0,)f(0,)h(MPI_COMM_SELF,)695 219 y(status,)f(ierr\))1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 bop 75 381 a Fo(Bibliograph)m(y)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Thu Jun 8 16:17:23 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA06369; Thu, 8 Jun 1995 16:17:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA21419 for mpi-core-out; Thu, 8 Jun 1995 15:16:00 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA21413 for ; Thu, 8 Jun 1995 15:15:55 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id QAA21956 for ; Thu, 8 Jun 1995 16:15:11 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id QAA13141 for ; Thu, 8 Jun 1995 16:15:47 -0400 Date: Thu, 8 Jun 1995 16:15:47 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199506082015.QAA13141@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA12979; Thu, 8 Jun 95 16:15:46 EDT To: mpi-core@mcs.anl.gov Subject: latest version of document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The latest version of the MPI-2 document is available for the chapter authors. It is in ftp://ftp.super.org/pub/mpi/mpi2/Editor. The changes are: - has all text handed out at meeting - scripts have been updated. Now you can produce a single chapter without any refs. - running a single chapter gives the correct chapter numbers. - the function index creation now works. Please start from this version with any changes you want to make in your chapters. Let me know of any problems. Steve From mpi-core-human@mcs.anl.gov Wed Jun 21 09:05:23 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA21220; Wed, 21 Jun 1995 09:05:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA17466 for mpi-core-out; Wed, 21 Jun 1995 08:03:44 -0500 Received: from rios2.EPM.ORNL.GOV (rios2.epm.ornl.gov [128.219.8.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA17460 for ; Wed, 21 Jun 1995 08:03:38 -0500 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) id JAA19837; Wed, 21 Jun 1995 09:03:07 -0400 From: David Walker Message-Id: <199506211303.JAA19837@rios2.EPM.ORNL.GOV> To: mpi-core@mcs.anl.gov Subject: All-gather Date: Wed, 21 Jun 95 09:03:07 -0500 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk ------- Forwarded Message Received: from msr.epm.ornl.gov (root@msr.epm.ornl.gov [128.219.8.1]) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) with ESMTP id HAA17449 for ; Wed, 21 Jun 1995 07:31:52 -0400 Received: from CS.UTK.EDU (CS.UTK.EDU [128.169.94.1]) by msr.epm.ornl.gov (8.6.10/8.6.10) with ESMTP id HAA20303 for ; Wed, 21 Jun 1995 07:31:45 -0400 Received: from elegabalus.cs.qub.ac.uk by CS.UTK.EDU with SMTP (cf v2.9s-UTK) id HAA07533; Wed, 21 Jun 1995 07:28:04 -0400 Received: from domitian.cs.qub.ac.uk by elegabalus.cs.qub.ac.uk (NeXT-1.0 (From Sendmail 5.52)/NeXT-2.0) id AA14283; Wed, 21 Jun 95 12:27:58 BST From: sfitzp@cs.qub.ac.uk (Stephen Fitzpatrick) Message-Id: <9506211127.AA14283@ elegabalus.cs.qub.ac.uk > Received: by domitian.cs.qub.ac.uk (NX5.67c/NX3.0X) id AA01875; Wed, 21 Jun 95 12:26:44 +0100 Date: Wed, 21 Jun 95 12:26:44 +0100 Received: by NeXT.Mailer (1.87.1) Received: by NeXT Mailer (1.87.1) To: mpi-comments@cs.utk.edu Subject: All-gather description Reply-To: S.Fitzpatrick@cs.qub.ac.uk A small suggestion... The description of mpi_allgather is, perhaps, misleading: (5 May 1994 edition of definition, page 107, lines 47,48) The jth block of data sent from each process is received by every process and placed in the jth block of the buffer recvbuf. This implies that each processor sends multiple blocks of data. Assuming I understand the operation correctly, I believe "The block of data sent from processor j" would be more accurate? A similar comment applies to the description of mpi_allgatherv. - -- Stephen Fitzpatrick, Department of Computer Science, Queen's University of Belfast, Belfast BT7 1NN, N. Ireland. S.Fitzpatrick@cs.qub.ac.uk (ASCII, NeXT, maybe MIME) PGP key via WWW: ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Fri Jun 23 09:30:29 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA01213; Fri, 23 Jun 1995 09:30:28 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA29567 for mpi-core-out; Fri, 23 Jun 1995 08:29:07 -0500 Received: from rios2.EPM.ORNL.GOV (rios2.epm.ornl.gov [128.219.8.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA29559 for ; Fri, 23 Jun 1995 08:29:00 -0500 Received: (from walker@localhost) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) id JAA19954; Fri, 23 Jun 1995 09:28:28 -0400 From: David Walker Message-Id: <199506231328.JAA19954@rios2.EPM.ORNL.GOV> To: mpi-core@mcs.anl.gov Subject: selling MPI Date: Fri, 23 Jun 95 09:28:28 -0500 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Officially I think the university of tennesse holds the copyright. I don't think we should let this guy repackage or sell the MPI specification. What do the rest of you think? David ------- Forwarded Message Received: from msr.epm.ornl.gov (root@msr.epm.ornl.gov [128.219.8.1]) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) with ESMTP id OAA19345 for ; Thu, 22 Jun 1995 14:41:33 -0400 Received: from CS.UTK.EDU (CS.UTK.EDU [128.169.94.1]) by msr.epm.ornl.gov (8.6.10/8.6.10) with ESMTP id OAA04216 for ; Thu, 22 Jun 1995 14:41:29 -0400 Received: from convex.convex.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) id OAA23180; Thu, 22 Jun 1995 14:40:46 -0400 Received: from imagine.convex.com by convex.convex.com (8.6.4.2/1.35) id NAA29577; Thu, 22 Jun 1995 13:40:13 -0500 Received: by imagine.convex.com (8.6.12/1.28) id NAA20337; Thu, 22 Jun 1995 13:34:25 -0500 Date: Thu, 22 Jun 1995 13:34:25 -0500 From: jgreat@imagine.convex.com (J L Greathouse) Message-Id: <199506221834.NAA20337@imagine.convex.com> To: mpi-comments@cs.utk.edu Subject: Rights to MPI standard Cc: jgreat@imagine.convex.com I'm interested in copying, repackaging, and perhaps selling the MPI Standard available on the WWW at http://www.mcs.anl.gov/mpi/mpi-report/mpi-report.html. According to the 5th paragraph on that WWW page, permission is given to copy without fee as long as certain requriements are met, but there is no mention of repackaging or selling. I'd like to talk with someone about this. Can any of you give me the name of someone of whom I could ask these and related questions? Janet Greathouse Convex Computer Corp. (214) 497-4309 ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Fri Jun 23 09:44:04 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA01257; Fri, 23 Jun 1995 09:44:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA29793 for mpi-core-out; Fri, 23 Jun 1995 08:44:26 -0500 Received: from hub.meiko.co.uk (root@hub.meiko.co.uk [192.131.108.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA29785 for ; Fri, 23 Jun 1995 08:44:17 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub.meiko.co.uk with SMTP id AA17478 (5.65c/IDA-1.4.4 for mpi-core@mcs.anl.gov); Fri, 23 Jun 1995 14:44:12 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA16666; Fri, 23 Jun 1995 14:44:10 +0000 Date: Fri, 23 Jun 1995 14:44:10 +0000 From: jim@meiko.co.uk (James Cownie) Message-Id: <9506231344.AA16666@deneb.co.uk> To: mpi-core@mcs.anl.gov In-Reply-To: <199506231328.JAA19954@rios2.EPM.ORNL.GOV> (message from David Walker on Fri, 23 Jun 95 09:28:28 -0500) Subject: Re: selling MPI Reply-To: jim@meiko.com Content-Length: 1507 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Officially I think the university of tennesse holds the copyright. I don't > think we should let this guy repackage or sell the MPI specification. What > do the rest of you think? > > David My understanding is that the reason we asserted copyright was to be able to control distribution in the sense that we don't want someone to put out an "MPI standard" which omitted bits that their implementation found hard ! I see no objection to letting vendors include a copy of the MPI standard in their documentation sets (and therefore probably charge for it) provided that it is a VERBATIM copy of the standard. After all this just gets it distributed further (which we surely want), and doesn't stop those who want it from ftp'ing it the same as they always could. We could (perhaps) insist that such "vendor copies" included directions on where to get the official, free, copy on the web (if we're confident that such a reference will remain valid for a worthwhile length of time). I don't think it's up to us to stop people from paying money for things they could have got for free. (I don't even think we're capable of stopping this happening !). Are there other reasons for objecting ? -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From mpi-core-human@mcs.anl.gov Fri Jun 23 09:57:44 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA01323; Fri, 23 Jun 1995 09:57:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA00299 for mpi-core-out; Fri, 23 Jun 1995 08:58:05 -0500 Received: from epcc.ed.ac.uk (root@daedalus.epcc.ed.ac.uk [129.215.56.21]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA00287 for ; Fri, 23 Jun 1995 08:57:53 -0500 Date: Fri, 23 Jun 95 14:57:31 BST Message-Id: <619.9506231357@subnode.epcc.ed.ac.uk> From: L J Clarke Subject: Re: selling MPI To: mpi-core@mcs.anl.gov In-Reply-To: James Cownie's message of Fri, 23 Jun 1995 14:44:10 +0000 Reply-To: lyndon@epcc.ed.ac.uk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I basically agree with Jim. Surely it is in the interest of MPI to allow vendors to use a copy of the standard in their documentation sets. -----------------------------------------------------------------------------| e||) Lyndon J Clarke Edinburgh Parallel Computing Centre| | c||c Tel: 0131 650 5021 Email: lyndon@epcc.edinburgh.ac.uk | |- -----| URL: http://www.epcc.ed.ac.uk/~lyndon/ | ------------------------------------------------------| -------------------- From mpi-core-human@mcs.anl.gov Fri Jun 23 09:58:17 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA01341; Fri, 23 Jun 1995 09:58:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA00341 for mpi-core-out; Fri, 23 Jun 1995 08:58:52 -0500 Received: from pobox.cscs.ch (pobox.cscs.ch [148.187.10.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA00323 for ; Fri, 23 Jun 1995 08:58:32 -0500 Received: from agno-fddi.cscs.ch by pobox.cscs.ch with SMTP inbound id <29642-0@pobox.cscs.ch>; Fri, 23 Jun 1995 15:57:19 +0200 Received: from thun (thun [148.187.170.11]) by agno-ether.cscs.ch (8.6.10/8.6.10) with SMTP id PAA11325; Fri, 23 Jun 1995 15:57:14 +0200 From: Jack Dongarra (UNI of Tennessee) Received: by thun (5.0) id AA19227; Fri, 23 Jun 1995 15:57:39 +0200 Message-Id: <9506231357.AA19227@thun> Subject: Re: selling MPI To: walker@rios2.epm.ornl.gov (David Walker) Date: Fri, 23 Jun 1995 15:57:37 +0200 (MET DST) Cc: mpi-core@mcs.anl.gov In-Reply-To: <199506231328.JAA19954@rios2.EPM.ORNL.GOV> from "David Walker" at Jun 23, 95 09:28:28 am X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 2010 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with David. UT hold the copyrights on the standard. Since he comes from Convex he may just want to print copies and distribute them to Convex MPI users and charge for printing costs. I'll ask him. Jack > > Officially I think the university of tennesse holds the copyright. I don't > think we should let this guy repackage or sell the MPI specification. What > do the rest of you think? > > David > > ------- Forwarded Message > > Received: from msr.epm.ornl.gov (root@msr.epm.ornl.gov [128.219.8.1]) by rios2.EPM.ORNL.GOV (8.6.10/8.6.10) with ESMTP id OAA19345 for ; Thu, 22 Jun 1995 14:41:33 -0400 > Received: from CS.UTK.EDU (CS.UTK.EDU [128.169.94.1]) by msr.epm.ornl.gov (8.6.10/8.6.10) with ESMTP id OAA04216 for ; Thu, 22 Jun 1995 14:41:29 -0400 > Received: from convex.convex.com by CS.UTK.EDU with ESMTP (cf v2.9s-UTK) > id OAA23180; Thu, 22 Jun 1995 14:40:46 -0400 > Received: from imagine.convex.com by convex.convex.com (8.6.4.2/1.35) > id NAA29577; Thu, 22 Jun 1995 13:40:13 -0500 > Received: by imagine.convex.com (8.6.12/1.28) > id NAA20337; Thu, 22 Jun 1995 13:34:25 -0500 > Date: Thu, 22 Jun 1995 13:34:25 -0500 > From: jgreat@imagine.convex.com (J L Greathouse) > Message-Id: <199506221834.NAA20337@imagine.convex.com> > To: mpi-comments@cs.utk.edu > Subject: Rights to MPI standard > Cc: jgreat@imagine.convex.com > > > I'm interested in copying, repackaging, and perhaps selling the MPI Standard > available on the WWW at http://www.mcs.anl.gov/mpi/mpi-report/mpi-report.html. > > According to the 5th paragraph on that WWW page, permission is given to copy > without fee as long as certain requriements are met, but there is no mention > of repackaging or selling. > > I'd like to talk with someone about this. Can any of you give me the > name of someone of whom I could ask these and related questions? > > Janet Greathouse > Convex Computer Corp. > (214) 497-4309 > > > ------- End of Forwarded Message > > From mpi-core-human@mcs.anl.gov Tue Jun 27 16:25:23 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA03449; Tue, 27 Jun 1995 16:25:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA25712 for mpi-core-out; Tue, 27 Jun 1995 15:24:30 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA25702 for ; Tue, 27 Jun 1995 15:24:21 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.11/8.6.10.1) with ESMTP id QAA08216 for ; Tue, 27 Jun 1995 16:22:58 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.11/8.6.10.1) with SMTP id QAA15338 for ; Tue, 27 Jun 1995 16:23:49 -0400 Date: Tue, 27 Jun 1995 16:23:49 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199506272023.QAA15338@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA19152; Tue, 27 Jun 95 16:23:47 EDT To: mpi-core@mcs.anl.gov Subject: new (temporary) archive for MPI2 docs Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy, Due to system difficulities, people cannot reach my local anonymous ftp server. As a result, MPIF members cannot get at the latest copies of the documents. Since my site may be down for a while, I have copied the files to Argonne. For now, you can get them from: ftp://www.mcs.anl.gov/pub/p4/mpi2/Editor Another point is that a number of people will be on travel the week before the next MPI meeting. If it is possible to get input to me by 5:00 on Friday, July 14 that would be nice. After this date, please send me and Rusty (lusk@mcs.anl.gov) any input to make sure it is included in the MPI document for distribution. All input should be in by COB Friday, June 21. Thanks, Steve From mpi-core-human@mcs.anl.gov Wed Jul 5 12:10:37 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA15907; Wed, 5 Jul 1995 12:10:36 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA24082 for mpi-core-out; Wed, 5 Jul 1995 11:09:51 -0500 Received: from mail.think.com (Mail.Think.COM [131.239.16.33]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA24072 for ; Wed, 5 Jul 1995 11:09:39 -0500 Received: from Icarus.Think.COM by mail.think.com; Wed, 5 Jul 95 12:09:36 -0400 From: Eric Sharakan Received: by icarus.think.com (4.1/Think-1.2) id AA03009; Wed, 5 Jul 95 12:09:35 EDT Date: Wed, 5 Jul 95 12:09:35 EDT Message-Id: <9507051609.AA03009@icarus.think.com> To: mpi-core@mcs.anl.gov Subject: meeting info Reply-To: sharakan@think.com Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi, folks. I'm looking for info concerning the upcoming forum meetings. First, I want to verify that the next meeting is being held on 7/26-7/28 at the O'hare Hilton. Are there any discounted rates available at the Hilton for the meeting? I'm also interested in geting hold of the operating rules of the forum. I had found something on the web last week, but it looked old, and now I can't remember where I found it, anyway. Also, any other pointers you can give to aide a couple people about to attend their first MPI forum meeting would be most appreciated. Thanks in advance. --- Eric Sharakan I can explain it to you, but sharakan@think.com I can't understand it for you. {uunet,harvard}!think!sharakan - Anon. (617) 234-2710 From mpi-core-human@mcs.anl.gov Wed Jul 5 14:03:14 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA16749; Wed, 5 Jul 1995 14:03:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA26393 for mpi-core-out; Wed, 5 Jul 1995 13:04:00 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id NAA26384 for ; Wed, 5 Jul 1995 13:03:52 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA01210; Wed, 5 Jul 95 13:03:43 CDT Date: Wed, 5 Jul 95 13:03:43 CDT From: Tony Skjellum Message-Id: <9507051803.AA01210@Aurora.CS.MsState.Edu> To: sharakan@think.com Subject: Re: meeting info Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk 1 person can vote per institution must attend 2 of 3 meetings to vote, per institution lots of straw polls... everyone votes Meeting on July 26-28, and September 6-8. -Tony From mpi-core-human@mcs.anl.gov Wed Jul 5 23:29:59 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA21749; Wed, 5 Jul 1995 23:29:59 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA05432 for mpi-core-out; Wed, 5 Jul 1995 22:27:58 -0500 Received: from HARVARDA.HARVARD.EDU (harvarda.harvard.edu [128.103.60.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id WAA05231; Wed, 5 Jul 1995 22:24:57 -0500 Message-Id: <199507060324.WAA05231@antares.mcs.anl.gov> Received: from physics1.byu.edu by HARVARDA.HARVARD.EDU (IBM VM SMTP V2R2) with TCP; Wed, 05 Jul 95 23:22:04 EDT Date: Wed, 5 Jul 1995 21:21:57 -0600 To: mpi-coll@mcs.anl.gov, mpi-comm@mcs.anl.gov, mpi-core@mcs.anl.gov, mpi-dynamic@mcs.anl.gov, mpi-external@mcs.anl.gov From: David@wishes.to.remain.anonymous.UA Subject: your romance ad in USSR pn R A #L O (Unverified) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk ## meet women of the former USSR through romance ads ## Months ago, Olga Kosmina placed my personal ad in several papers of the former Soviet Union. Since that time I have received over 40 responses for the $50 I mailed Olga. (I believe she paid the newspapers something around $35 and kept the rest for her efforts.) I have found greater success and savings by placing my own personal romance advertisement rather than purchasing addresses through Russian "bride" catalog companies. If you are interested in placing a personal romance ad as I did, contact Olga. She has built up a list of most every newspaper and magazine in the former Soviet Union and could help direct your ad to certain areas if you wish. She writes, "please say that I place all ad _throughout_ Russia and other countries of former Soviet Union, not only Western Russia." Olga is 23 years old, has a bachelors in biology and works full-time as a florist in Kiev. She speaks, reads and writes English as well as her native languages of Russian and Ukrainian. I realize that it is a very trusting person who would put $ into an envelope and mail to a foreign country. If you would rather send a letter of inquiry first, Olga will respond to your questions. It takes about 16 days for a letter to travel from the USA to Kiev. Olga Kozmina Dekabristov Street 5 - 178 Kiev 253121 Ukraine I have found that by placing a single bill between two pieces of newsprint inside an envelope, the Ukrainian post cannot see through and does not bother to tamper. I have yet to lose a letter sent to Kiev. I am sorry that Olga does not have e-mail because it would make contact with her much easier. I am posting anonymously because of the inordinate amount of e-mail which I would receive -- inquiries as well as flames. Best Wishes, David and Olga Although Olga has never seen a newsgroup nor heard of "net-etiquette," she believes that helping others exceeds the cost of angering those who feel the net should not be used in this fashion. IHA (I humbly ask) that you not flame the postmaster of this site. peace. . . From mpi-core-human@mcs.anl.gov Fri Jul 7 14:38:12 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA18138; Fri, 7 Jul 1995 14:38:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA08196 for mpi-core-out; Fri, 7 Jul 1995 13:38:49 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA08178 for ; Fri, 7 Jul 1995 13:38:39 -0500 Message-Id: <199507071838.NAA08178@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: MPI Forum meeting, July 26-28 Date: Fri, 07 Jul 1995 13:38:37 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, I need a head count for the hotel for the MPI Forum meeting on July 26-28. You should make your reservations before July 17th in order to get the good room rate. Note that the meeting this time is Wednesday-Thursday-Friday instead of Monday-Tuesday-Wednesday. Please let me know as soon as possible whether you are planning to come or not. If you have any questions, don't hesitate to send me mail at lusk@mcs.anl.gov or call me at (708) 252-7852. Regards, Rusty The usual blurb: The O'Hare Hilton is the hotel *inside* the airport. You just walk across the street from the baggage-claim level. A block of rooms has been reserved, which will be held until July 17, so please make your reservations with the Hotel before then. The phone number for the hotel is (312) 686-8000 (FAX (312) 601-2876). Mention the *MPI Forum* to get the special rate, which is $90.50 + tax per night. We will charge the usual $75 registration to take care of the meeting room, lunch on Thursday, and coffee, etc. The meeting will start at 1:30 on Wednesday to allow people to fly in that morning, and end at noon on Friday. From mpi-core-human@mcs.anl.gov Fri Jul 7 16:59:39 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA18826; Fri, 7 Jul 1995 16:59:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA11245 for mpi-core-out; Fri, 7 Jul 1995 15:59:20 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA11237 for ; Fri, 7 Jul 1995 15:59:12 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id QAA18763 for ; Fri, 7 Jul 1995 16:57:42 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with SMTP id QAA21972 for ; Fri, 7 Jul 1995 16:58:40 -0400 Date: Fri, 7 Jul 1995 16:58:40 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199507072058.QAA21972@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA01757; Fri, 7 Jul 95 16:58:38 EDT To: mpi-core@mcs.anl.gov Subject: non-blocking ops vs. threads Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Since this topic effects several aspects of MPI-2, I'm posting to everyone. As I'm rewriting the external interfaces chapter, my notes from the last meeting indicate that some people were in favor of using threads instead of new non-blocking operations. This decision will effect several areas of MPI-2 including non-blocking collective and external interfaces. At the meeting there was a discussion in which it was noted that threads are more general and not on all systems. Others felt that MPI-2 was starting to reinvent mechanisms done by others. Other points were made as well. Since the mail has been slow lately, I thought I would try and start a discussion of this issue. I think the time has come for us to decide which path we think is best. Steve From mpi-core-human@mcs.anl.gov Fri Jul 7 17:33:56 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA19759; Fri, 7 Jul 1995 17:33:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA11982 for mpi-core-out; Fri, 7 Jul 1995 16:35:13 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA11973 for ; Fri, 7 Jul 1995 16:35:03 -0500 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) for <@sgi.sgi.com:mpi-core@mcs.anl.gov> id OAA17020; Fri, 7 Jul 1995 14:35:00 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id OAA17590; Fri, 7 Jul 1995 14:34:57 -0700 From: "Eric Salo" Message-Id: <9507071434.ZM17588@mrjones.engr.sgi.com> Date: Fri, 7 Jul 1995 14:34:55 -0700 In-Reply-To: lederman@super.org (Steve Huss-Lederman) "non-blocking ops vs. threads" (Jul 7, 4:58pm) References: <199507072058.QAA21972@gotham.super.org> X-Mailer: Z-Mail (3.2.1 6apr95 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: non-blocking ops vs. threads Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Okay, since Steve has brought this up, here's an attempt to keep the ball rolling: MPI was supposedly designed to allow for multi-threaded implementations; that's supposed to be one if its advantages, right? Since we already have this, we get non-blocking collective ops for free once threads are actually implemented. On the other hand, it is not obvious to me that even if we do decide to implement non-blocking collective ops, we won't still have to deal with threads at some later time. I therefore would argue that threads are the way to go and that special non-blocking collective ops are pretty much a waste of time. Now let's hear from someone on the other side... -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Jul 7 17:44:26 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA19860; Fri, 7 Jul 1995 17:44:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA12186 for mpi-core-out; Fri, 7 Jul 1995 16:45:33 -0500 Received: from hades.osc.epsilon.com (hades.osc.epsilon.com [159.127.66.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA12174 for ; Fri, 7 Jul 1995 16:45:25 -0500 Received: by hades.osc.epsilon.com (4.1/3.1.-Epsilon Data Management) id AA29436; Fri, 7 Jul 95 17:47:21 EDT Received: from kosmos.quantum.epsilon.com(159.127.20.130) by hades.osc.epsilon.com via smap (V1.3) id sma029422; Fri Jul 7 17:46:52 1995 Received: from pluto.epsilon.com by kosmos.epsilon.com (4.1/SMI-4.1) id AA12163; Fri, 7 Jul 95 17:44:32 EDT Date: Fri, 7 Jul 95 17:44:32 EDT From: moose@epsilon.com (Adam Greenberg) Message-Id: <9507072144.AA12163@kosmos.epsilon.com> Received: by pluto.epsilon.com (4.1/SMI-4.1) id AA02895; Fri, 7 Jul 95 17:44:31 EDT To: salo@mrjones.engr.sgi.com Cc: mpi-core@mcs.anl.gov In-Reply-To: <9507071434.ZM17588@mrjones.engr.sgi.com> (salo@mrjones.engr.sgi.com) Subject: Re: non-blocking ops vs. threads Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk From: "Eric Salo" Date: Fri, 7 Jul 1995 14:34:55 -0700 References: <199507072058.QAA21972@gotham.super.org> X-Mailer: Z-Mail (3.2.1 6apr95 MediaMail) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Okay, since Steve has brought this up, here's an attempt to keep the ball rolling: MPI was supposedly designed to allow for multi-threaded implementations; that's supposed to be one if its advantages, right? Since we already have this, we get non-blocking collective ops for free once threads are actually implemented. On the other hand, it is not obvious to me that even if we do decide to implement non-blocking collective ops, we won't still have to deal with threads at some later time. I therefore would argue that threads are the way to go and that special non-blocking collective ops are pretty much a waste of time. Actually, MPI was designed to be `thread safe'. It was widely acknowledged that designing integrated thread support was beyond the means of the committee. Given that, my own view is that thread support would be great if it were possible to make sufficiently generic to allow many (all) platforms to use it. Is it the case that threads packages (either vendor supplied, or third party) are sufficiently similar to allow this? Is the POSIX threads specification stable and widely represented on platforms of interest? moose Now let's hear from someone on the other side... -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" *-----------------------------------------------------------------------------* The opinions expressed herein are my own. They do not represent Epsilon in any way. Adam Greenberg Quadruped Systems, Inc. Epsilon Data Management 68 Putnam Ave. High Performance Computing, 4th Floor Cambridge, MA. 02139-2952 50 Cambridge St. Burlington MA 01803-4692 617 273 2630 x6455 moose@epsilon.com From mpi-core-human@mcs.anl.gov Fri Jul 7 18:05:22 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA20003; Fri, 7 Jul 1995 18:05:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA12560 for mpi-core-out; Fri, 7 Jul 1995 17:06:07 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA12551; Fri, 7 Jul 1995 17:05:59 -0500 Message-Id: <199507072205.RAA12551@antares.mcs.anl.gov> To: moose@epsilon.com (Adam Greenberg) cc: salo@mrjones.engr.sgi.com, mpi-core@mcs.anl.gov Subject: Re: non-blocking ops vs. threads In-reply-to: Your message of "Fri, 07 Jul 1995 17:44:32 EDT." <9507072144.AA12163@kosmos.epsilon.com> Date: Fri, 07 Jul 1995 17:05:58 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | Given that, my own view is that thread support would be great if it were | possible to make sufficiently generic to allow many (all) platforms to use | it. I agree. | Is it the case that threads packages (either vendor supplied, or third | party) are sufficiently similar to allow this? Not yet. | Is the POSIX threads | specification stable and widely represented on platforms of interest? Not yet. - Rusty From mpi-core-human@mcs.anl.gov Fri Jul 7 19:45:57 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA20314; Fri, 7 Jul 1995 19:45:57 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA13908 for mpi-core-out; Fri, 7 Jul 1995 18:46:50 -0500 Received: from wk65.nas.nasa.gov (wk65.nas.nasa.gov [129.99.64.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA13899 for ; Fri, 7 Jul 1995 18:46:43 -0500 Received: (from barszcz@localhost) by wk65.nas.nasa.gov (8.6.12/NAS.6.1) id QAA18723; Fri, 7 Jul 1995 16:46:40 -0700 Date: Fri, 7 Jul 1995 16:46:40 -0700 From: barszcz@nas.nasa.gov (Eric Barszcz) Message-Id: <199507072346.QAA18723@wk65.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: Re: non-blocking ops vs. threads Cc: barszcz@wk65.nas.nasa.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Sorry for not following the arguments too closely. However, I would like to see a non-blocking barrier in MPI-2. It is not clear to me whether this is being considered separately from other collective reduction operations. If it is, please ignore the rest of this message. The reason for a "portble" non-blocking barrier is I have an application dealing with multiple groups where the pt-to-pt communication pattern between the groups is not known until execution time. The problem is each processor may act as both client and server. However, the servers do not know how many clients they will serve. They remain "active" until all clients are satisfied. Clients are satisfied when they have all of the data they need or have determined that the data does not exist. Each client has a list of servers that may contain the data. To keep the number of messages under control, a client goes down the list of potential servers and requests data. If the server does not have the requested data, the client tries the next on the list. In the meantime, other processes may be requesting service from the client processor. Which clients and which servers will be "active" changes from iteration to iteration. To solve this problem, a non-blocking barrier is implemented using a spanning tree and non-blocking probes for messages. The application was originally developed on the iPSC/860 using the intercube library. This approach has been portable across 3 different message passing machines and currently uses MPI. Other methods exist to solve the problem. However, this one appears to minimize the number of messages and global synchronization. This looks like there should be a thread for the client and one for the server. Unfortunately, as it has been pointed out, there is no portable common thread mechanism. So, I would like to see a non-blocking barrier in MPI. Eric Barszcz barszcz@nas.nasa.gov From mpi-core-human@mcs.anl.gov Fri Jul 7 19:49:56 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA20326; Fri, 7 Jul 1995 19:49:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA13998 for mpi-core-out; Fri, 7 Jul 1995 18:51:17 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA13990; Fri, 7 Jul 1995 18:51:11 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA03747; Fri, 7 Jul 95 18:50:56 CDT Date: Fri, 7 Jul 95 18:50:56 CDT From: Tony Skjellum Message-Id: <9507072350.AA03747@Aurora.CS.MsState.Edu> To: lusk@mcs.anl.gov, moose@epsilon.com Subject: Re: non-blocking ops vs. threads Cc: mpi-core@mcs.anl.gov, salo@mrjones.engr.sgi.com Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My two cents on this one is that we should identify properties in addition to POSIX requirements that would make MPI happy with threads, and then push for such a "High Performance POSIX Threads" option to be supported by vendors... -Tony From mpi-core-human@mcs.anl.gov Fri Jul 7 19:56:44 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA20345; Fri, 7 Jul 1995 19:56:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA14121 for mpi-core-out; Fri, 7 Jul 1995 18:57:58 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA14113 for ; Fri, 7 Jul 1995 18:57:47 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA03780; Fri, 7 Jul 95 18:57:36 CDT Date: Fri, 7 Jul 95 18:57:36 CDT From: Tony Skjellum Message-Id: <9507072357.AA03780@Aurora.CS.MsState.Edu> To: lederman@super.org, mpi-core@mcs.anl.gov Subject: Re: non-blocking ops vs. threads Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve, it is my intention to propose collective extensions chapter that basically mirror collective blocking ops in MPI-1, including constructors. Explaining how all these work will take some time :-) My main difference from Marc is that I think of waiting on collective ops as a potentially more difficult [non-local] issue than he does. IBM has already, as we know, added non-blocking collectives to their MPI release, and it would be interesting to see these, though I would not feel restricted by them, for the initial chapter proposal. The IBM speaker at MPIDC said we could get this info, but we will probably have to ask again. -Tony From mpi-core-human@mcs.anl.gov Fri Jul 7 20:17:34 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA20439; Fri, 7 Jul 1995 20:17:33 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA14277 for mpi-core-out; Fri, 7 Jul 1995 19:18:52 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id TAA14269 for ; Fri, 7 Jul 1995 19:18:45 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA03825; Fri, 7 Jul 95 19:18:32 CDT Date: Fri, 7 Jul 95 19:18:32 CDT From: Tony Skjellum Message-Id: <9507080018.AA03825@Aurora.CS.MsState.Edu> To: barszcz@nas.nasa.gov Subject: Re: non-blocking ops vs. threads Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Eric, Yes, I envision proposing: MPI_IBARRIER(comm, request, ierror) MPI_COMM comm [IN] MPI_REQUEST request [OUT] integer ierror [OUT] A correct program later calls MPI_WAIT(request, status, ierror) or MPI_TEST(request,status, ierror) in the entire group of comm. [See comments below] Allowing... 0) point to point on the comm does not impact non-blocking collective ops. A single blocking collective op is also allowed, and is defined to have the same semantics as MPI_IBARRIER followed immediately by MPI_BARRIER, thus recovering MPI-1 rules. 1) Arbitrary number of outstanding non-blocking barriers per communicator. Implementations will clearly pose restrictions, and this will be a so-called "quality of implementation" issue. Like contexts, it should be possible to provide a large number of outstanding requests, but not if each truly implies a unique thread per process in a real implementation. Hence, some careful language and discussion will be needed here. 2) possible need for different handling of requests for collective ops as compared to point-to-point, to be discussed in draft. Specifically, a) same structure of requests as for point-to-point b) separate test/wait calls for collective Assuming 2a) One waits on the completion with MPI_WAIT() or MPI_TEST() or etc, else 2b) contemplates adding a set of calls MPI_WAIT_COLL() MPI_TEST_COLL() MPI_WAITALL_COLL() etc. The problem with 2b) is that i) lots of new calls ii) can these be applied with different communicators involved? [if over one comm, then are these local or collective over group of comm] iii) Implementations can do some of the work in the wait/test phase, that might be collective More implementations might handle 2b) than 2a), tbd. 3) We would want persistent collective operations. a) same mechanism as what start point to point b) analogous mechanism for collective, subject to caveats outlined under 2b) above. -Tony >From barszcz@nas.nasa.gov Fri Jul 7 19:03:50 1995 >Received: from wk65.nas.nasa.gov by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); > id AA03796; Fri, 7 Jul 95 19:03:49 CDT >Received: (from barszcz@localhost) > by wk65.nas.nasa.gov (8.6.12/NAS.6.1) id RAA18810; Fri, 7 Jul 1995 17:03:49 -0700 >Date: Fri, 7 Jul 1995 17:03:49 -0700 >From: barszcz@nas.nasa.gov (Eric Barszcz) >Message-Id: <199507080003.RAA18810@wk65.nas.nasa.gov> >To: Tony Skjellum >Subject: Re: non-blocking ops vs. threads >Cc: barszcz@wk65.nas.nasa.gov >Status: R > >Again, I'm sorry that I haven't been keeping up with the >discussions. > >Basically what I envision is something based on MPI_BARRIER() >where it returns a handle immediately and then you can test >for completion with MPI_WAIT() or MPI_TEST(). > >Is this what you had in mind? > >Also, would one want multiple outstanding non-blocking barriers? > > > >Eric Barszcz >barszcz@nas.nasa.gov > From mpi-core-human@mcs.anl.gov Fri Jul 7 21:04:17 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA20482; Fri, 7 Jul 1995 21:04:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA14573 for mpi-core-out; Fri, 7 Jul 1995 20:05:18 -0500 Received: from infomeister.osc.edu (infomeister.osc.edu [128.146.36.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA14565 for ; Fri, 7 Jul 1995 20:05:13 -0500 Received: from tbag.osc.edu for gdburns@tbag.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id VAA11773; Fri, 7 Jul 1995 21:05:11 -0400 Received: by tbag.osc.edu (4.1/OSC-SUB-2.0) id AA14991; Fri, 7 Jul 95 21:05:10 EDT Date: Fri, 7 Jul 95 21:05:10 EDT From: gdburns@tbag.osc.edu (Greg Burns) Message-Id: <9507080105.AA14991@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: Re: non-blocking ops vs. threads Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >This decision will effect >several areas of MPI-2 including non-blocking collective and external >interfaces. and one-sided communication Without the shelter of the blocking semantic, I*send() and Bsend() forced implementors either to put polling code into MPI routines (at a minimum, any MPI routine that could block) to check if "anything pending can move along" or to use interrupts. With some of the MPI-2 proposals (user requests, more I*stuff()) we'll have to put a lot more polling code in. I am assuming that the MPI_Finalize() requirement gives us cover from the progress thingy. I think that we can actually legally implement the user requests and more non-blocking stuff without forcing threads (prettier interrupts) upon the implementation. It will not be enjoyable, however. The user request stuff reminds me of window programming. We should just mandate an MPI_Main_loop() call... It is the devil one-sided communication that fully demands an asynchronous software solution for machines without a hardware solution. Now we must rely on interrupts. In some cases we must use threads to get at protected address spaces. In LAM we already have the big interrupt-demultiplexor-daemon, so not much sweat. I'd like to hear from Bill about how he plans on doing this in the leaner MPICH. It used to be a virtue of MPI that a pure polling implementation, a pure library implementation straight on hardware was possible. So we message-passing people have to have a shared address space interrupt handler lying around so that we can emulate more complex hardware (a shared memory or one-sided comm. computer) so we can give the shared memory programmers more portability because thread standards are messed up and it's better to put that non-portable dirt under our rug instead of theirs. Notice Eric and kin don't have any trouble doing nice MPI-1 implementations on more complex hardware. I suggest a separate SMI Forum or a 1SI Forum. They'll need some semaphores and we can build an MPI implementation on top of it. Alternatively, they can do what some thread programmers looking for portability do today - write their own thread wrappers and port them around. I am for not building into MPI things that could be done with threads. -- Greg From mpi-core-human@mcs.anl.gov Fri Jul 7 22:35:08 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA20537; Fri, 7 Jul 1995 22:35:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA14919 for mpi-core-out; Fri, 7 Jul 1995 21:36:22 -0500 Received: from msr.EPM.ORNL.GOV (msr.epm.ornl.gov [128.219.8.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA14911 for ; Fri, 7 Jul 1995 21:36:15 -0500 Received: (from geist@localhost) by msr.EPM.ORNL.GOV (8.6.10/8.6.10) id WAA14371 for mpi-core@mcs.anl.gov; Fri, 7 Jul 1995 22:35:44 -0400 Date: Fri, 7 Jul 1995 22:35:44 -0400 From: Al Geist Message-Id: <199507080235.WAA14371@msr.EPM.ORNL.GOV> To: mpi-core@mcs.anl.gov Subject: Re: non-blocking ops vs. threads Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >From: Tony Skjellum >we should identify properties >in addition to POSIX requirements that would make MPI happy >with threads, and then push for such a "High Performance POSIX >Threads" option to be supported by vendors. While this would be nice, my concern is that this may delay the release of MPI-2 by vendors if they have to create a High Performance POSIX Threads package for their machine first. My feeling is that threads is the way to go in the long run, but the present environment does not lend itself to portability. We just got through testing a multithreaded version of PVM. Porting to just a few machines was an awful mess. And what about portability to non-UNIX boxes like microkernel compute nodes or (gag me) Windows NT? My e cents worth, Al Geist From mpi-core-human@mcs.anl.gov Sat Jul 8 17:25:25 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA29253; Sat, 8 Jul 1995 17:25:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA19592 for mpi-core-out; Sat, 8 Jul 1995 16:33:00 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA19579 for ; Sat, 8 Jul 1995 16:32:52 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 5047; Sat, 08 Jul 95 17:32:05 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 0583; Sat, 8 Jul 1995 17:32:05 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Sat, 08 Jul 95 17:32:05 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA34202; Sat, 8 Jul 1995 17:31:45 -0400 Message-Id: <9507082131.AA34202@snir.watson.ibm.com> To: mpi-core@mcs.anl.gov Cc: pratap@watson.ibm.com Subject: nonblocking collective -- why we need them Reply-To: snir@watson.ibm.com Date: Sat, 08 Jul 1995 17:31:44 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: 0. We already have nonblocking point-to-point communications, even though the argument could be made, there too, that one could a blocking call done by a separate thread. The take was that it is important to have nonblocking communications, for performance. 1. Not all systems that run MPI have efficient light-weight thread packages. 2. The integration of MPI communication with a thread package (making MPI calls reentrant, having communication events affect the thread scheduler, etc.) will slow down MPI. Some implementers may prefer to have MPI nonthread safe or not integratyed with the thread scheduler. 3. An MPI specific solution (nonblocking communication) is likely to be more efficient than a general solution. 4. By extending into nonblocking collective (or nonblocking what have you), we are not adding new mechanisms to MPI, we are building on top of mechanisms that MPI implementations already have in support of nonblockign communication. We are not introducing new ideas, we are extending existing MPI mechanisms in a fairly orthogonal manner. 5. Nonblocking collective communication answers a clear need from applications. Collective communications are used quite frequently; skews in the execution time at different processes may cause large performance losses is a blocking, polling implementation of collective communication is used (we have measured up to 40% perofmrnace loss, due to < 3% background system activity on a 128 node system). From mpi-core-human@mcs.anl.gov Mon Jul 10 08:49:10 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA19352; Mon, 10 Jul 1995 08:49:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA08558 for mpi-core-out; Mon, 10 Jul 1995 07:56:27 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA08550 for ; Mon, 10 Jul 1995 07:56:22 -0500 Received: from gotham.super.org (gotham-e [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id IAA24125; Mon, 10 Jul 1995 08:54:47 -0400 Received: from b125.super.org (b125.super.org [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with SMTP id IAA22342; Mon, 10 Jul 1995 08:55:48 -0400 Date: Mon, 10 Jul 1995 08:55:48 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199507101255.IAA22342@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA27426; Mon, 10 Jul 95 08:55:44 EDT To: tony@Aurora.CS.MsState.Edu Cc: barszcz@nas.nasa.gov, mpi-core@mcs.anl.gov In-Reply-To: <9507080018.AA03825@Aurora.CS.MsState.Edu> (message from Tony Skjellum on Fri, 7 Jul 95 19:18:32 CDT) Subject: Re: non-blocking ops vs. threads Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In external interfaces we are also dealing with user created non-blocking operations. Most of the proposals use the current WAIT/TEST mechanism. The appeal is not creating new functions. The drawback is that these routines currently return a status argument which gets SOURCE, TAG and ERROR. Using MPI_GET_COUNT you can get the number of entries received. This information does not seem appropriate to many other non-blocking operations. For example, what is source or count on a broadcast? Worst yet, what about user defined ops where it may be necessary to return unique information to that call. Options seem to be that you either need to: 1) specify that some or all the current fields in STATUS are set to "ignore" values and add new ones 2) state which values are valid for which calls 3) add new TEST/WAIT calls. Using 2) seems almost as bad as new calls. Each call need unique text and this seems ugly. 1) is not too bad but I don't like overloading functions so much. It shows the wart of reusing functions that were intended for another purpose. My current leanings is for either: a) create new TEST/WAIT functions that use a pointer in the "STATUS" to allow it to return what is necessary for the user created operation. (choice #1) or b) set all the fields in the current STATUS to ignore and add a new value which is a pointer. We can do this because the STATUS is opaque. I would like to stay consistant with non-blocking collective and others if this is possible and makes sense. What do others think? Steve From mpi-core-human@mcs.anl.gov Mon Jul 10 09:12:51 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA19431; Mon, 10 Jul 1995 09:12:50 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA08845 for mpi-core-out; Mon, 10 Jul 1995 08:20:49 -0500 Received: from hub (root@hub.meiko.co.uk [192.131.108.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA08836 for ; Mon, 10 Jul 1995 08:20:38 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA01004 (5.65c/IDA-1.4.4 for mpi-core@mcs.anl.gov); Mon, 10 Jul 1995 14:20:21 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA02626; Mon, 10 Jul 1995 14:20:28 +0000 Date: Mon, 10 Jul 1995 14:20:28 +0000 Message-Id: <9507101320.AA02626@deneb.co.uk> From: James Cownie To: mpi-core@mcs.anl.gov In-Reply-To: <9507080018.AA03825@Aurora.CS.MsState.Edu> (message from Tony Skjellum on Fri, 7 Jul 95 19:18:32 CDT) Subject: Re: non-blocking ops vs. threads Reply-To: jim@meiko.com Content-Length: 3880 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Tony discussed various options for non-blocking collective operations. These seem to come down to 1) Is a communicator restricted to only one non-blocking collective op "at a time", or can there be many simultaneous collective operations. 2) How is completion of non-blocking collective ops tested for a) as current non-blocking ops ? b) with a new set of calls ? Here's my tuppence worth... 3) Do we allow a non-blocking collective to match with a blocking one ? Consistency with pt2pt says we should. From the user viewpoint it seems reasonable (if non-blocking collective ops exist at all) to allow an arbitrary number in each communicator simultaneously. Reasons for taking this position include 1) by analogy with pt2pt, where there is no limitation, other than resources, which restricts the number of currently active non-blocking sends/recvs in the same communicator. 2) it's potentially useful to have many collective ops going in the same communicator at the same time. This raises further questions, though 1) Do we allow many of the same collective operation in the same communicator at the same time ? (We could allow multiple collective ops, but only one of each type. Though this would be a bit odd...) 2) Assuming we allow many concurrent "identical" collective ops in a communicator, how do we match them ? There seem to be two options here 1) They match in order. (Since they are collective in the communicator there's a well defined order which must be the same at all participants). 2) We add a tag argument, and enforce macthing by tag. (Which will explicitly allow them to complete out of order). If we do this, we'd have to either a) specify the tag used by existing blocking collective ops, so that they can be matched with non-blocking ones or b) forbid mixing of blocking and non-blocking collective ops. I favour the simplicity of answer 1, though this does leave at least one further question. a) What exactly does "matching in order" mean ? A strict "in-order" match would assert that the sequence of collective ops (both blocking and non-blocking) in a communicator must be the same at all processes with the communicator. This is the cleanest definition, but may not achieve everything that people require. One of the reasons some people want non-blocking collective is probably so that they can issue the requests in different orders. I can't yet see why this is needed, but I can believe there are algorithms where it would be useful. A slightly laxer definition would group collective ops into classes, at a first stab barrier, broadcast, scatters, gathers, all-to-all, reduction ops and insist only that the order within each class be the same. What do we want ? Do we need the laxer definition ? If we go for the laxer definition, how do we handle things inside the library, where we may need to implement an operation in one class using an operation in a different one. (e.g. implement barrier as zero length broadcast, or all-to-all as a series of broadcasts). Completion tests I'd like to be able to use current tests, I certainly want to be able to do a WAITANY on a collective op, or a message arriving. Steve points out various problems with the Status object, and usggests adding a pointer into it. Unfortunately this is unlikely to be very pleasant (if doable at all) in Fortran... -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From mpi-core-human@mcs.anl.gov Mon Jul 10 15:07:50 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA21287; Mon, 10 Jul 1995 15:07:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA17272 for mpi-core-out; Mon, 10 Jul 1995 14:14:21 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA17264 for ; Mon, 10 Jul 1995 14:14:14 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2219; Mon, 10 Jul 95 15:13:21 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 0361; Mon, 10 Jul 1995 15:13:21 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 10 Jul 95 15:13:20 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA21995; Mon, 10 Jul 1995 15:13:01 -0400 Message-Id: <9507101913.AA21995@snir.watson.ibm.com> To: mpi-core@mcs.anl.gov Subject: IBM implementation of nonblocking collective communication Reply-To: snir@watson.ibm.com Date: Mon, 10 Jul 1995 15:13:00 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: Quite simply, the IBM implementation of MPI has a nonblocking collective communication call matching each collective communication call. The syntax is identical, except that there is an additional request argument. The call is completed by a call to TEST, WAIT, etc., as for nonblocking point to point. Blocking and nonblocking calls can be mixed: some processes may participate in a collective call with a blocking call, whereas others in the group participate with a nonblocking call. This did no seem to raise any major implementation problems. From mpi-core-human@mcs.anl.gov Mon Jul 10 15:23:33 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA21369; Mon, 10 Jul 1995 15:23:31 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA17806 for mpi-core-out; Mon, 10 Jul 1995 14:30:58 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA17798 for ; Mon, 10 Jul 1995 14:30:51 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2609; Mon, 10 Jul 95 15:30:05 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 1887; Mon, 10 Jul 1995 15:30:04 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 10 Jul 95 15:30:03 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA18785; Mon, 10 Jul 1995 15:29:43 -0400 Message-Id: <9507101929.AA18785@snir.watson.ibm.com> To: mpi-core@mcs.anl.gov Subject: nonblocking collective Reply-To: snir@watson.ibm.com From: Marc Snir Date: Mon, 10 Jul 1995 15:29:43 -0400 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: As Cownie makes clear, there is a choice of either adding tags, and have matching by tag, or not using tags, and requiring that collective operations are issued in the "right order". The most conservative interpretation of "right order" is that, on any communicator, collective operations are issued in the same order at all processes. The IBM nonblocking collective do not use tags. We made this choice, in order to be consistent with blocking collective. Insisting that nonblocking collective are issued in the same order at all processes of a group makes implementation simpler. This also allows to mix nonblocking and blocking calls (as for point to point communication) without risks, and implies that the same rules should be obeyed by programs for collective calls, whether these are blocking or nonblocking -- simpler to explain. At the limit, one could use the same implementation for nonblocking collective as for blocking. Then, the option of a more lenient implementation, that allows some processes to complete ahead of other in a collective call, becomes an performance issue, not a correctness issue. The number of pending nonblocking collective operations per communicator need not be limited, since the implementation may always "limit" this number by completing all communication that pertains to a collective communication (or a set of collective communicaqtions) before starting a new one. Users that want "out of order invocation" of collective ops will need to use multiple communicators. Unless ones bring a convincing example of an application that requires such out of order invocation, I would stick with the simpler and more restrictive usage. ------- End of Unsent Draft From mpi-core-human@mcs.anl.gov Tue Jul 11 21:49:28 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA08162; Tue, 11 Jul 1995 21:49:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA16263 for mpi-core-out; Tue, 11 Jul 1995 20:57:09 -0500 Received: from pnl.gov (gate.pnl.gov [130.20.64.36]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA16255 for ; Tue, 11 Jul 1995 20:57:03 -0500 Received: from maddog.pnl.gov by pnl.gov (PMDF V4.3-13 #6012) id <01HSR64PYAMO8Y5ZQF@pnl.gov>; Tue, 11 Jul 1995 18:56:45 -0700 (PDT) Received: by maddog.pnl.gov (4.1/SMI-4.1) id AA25373; Tue, 11 Jul 95 18:56:43 PDT Date: Tue, 11 Jul 1995 18:56:43 -0700 (PDT) From: Rik Littlefield Subject: RE: non-blocking ops vs. threads To: gdburns@tbag.osc.edu, mpi-core@mcs.anl.gov Cc: j_nieplocha@pnl.gov, m_rosing@pnl.gov, rj_littlefield@pnl.gov Message-id: <9507120156.AA25373@maddog.pnl.gov> Content-transfer-encoding: 7BIT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Greg Burns writes: > It is the devil one-sided communication that fully demands an > asynchronous software solution for machines without a hardware solution. > Now we must rely on interrupts. In some cases we must use threads > to get at protected address spaces. In LAM we already have the > big interrupt-demultiplexor-daemon, so not much sweat. I'd like > to hear from Bill about how he plans on doing this in the leaner MPICH. > > It used to be a virtue of MPI that a pure polling implementation, > a pure library implementation straight on hardware was possible. > > So we message-passing people have to have a shared address space > interrupt handler lying around so that we can emulate more > complex hardware (a shared memory or one-sided comm. computer) > so we can give the shared memory programmers more portability because > thread standards are messed up and it's better to put that non-portable > dirt under our rug instead of theirs. Notice Eric and kin don't > have any trouble doing nice MPI-1 implementations on more complex hardware. Separating content from diatribe, it seems to me that there are a couple of critical questions that need examining: 1. What "one-sided communication" functionality is needed? 2. What is the cost of providing that functionality, both in a. complexity of implementation (labor cost), and b. performance impact to the rest of MPI (i.e., cooperative message passing) ? I suspect that the costs in question 2 are very much different for active messages and interrupt receive than for remote data access. I also suspect that the performance impact for remote data access can be made pretty small, and that the complexity of implementation for any particular platform is also not forbidding. I quite frankly admit that these opinions are based on limited information -- basically just our own experiences and feedback from a few others. I would welcome more data and reasoned discussion. We at PNL need remote data access. We can use other mechanisms like interrupt receive, active messages, and good thread interfaces (assuming proper definitions in each case) as ways to implement remote data access, but they're not really what we want. So how about it, developers -- what is the cost of including remote data access, and if it's high, then why, and how might it be reduced? --Rik ---------------------------------------------------------------------- rj_littlefield@pnl.gov (alias 'd39135') Rik Littlefield Tel: 509-375-3927 Pacific Northwest Lab, MS K7-15 Fax: 509-375-3641 P.O.Box 999, Richland, WA 99352 From mpi-core-human@mcs.anl.gov Tue Jul 11 22:56:07 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA17768; Tue, 11 Jul 1995 22:56:05 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA16790 for mpi-core-out; Tue, 11 Jul 1995 22:04:08 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA16782 for ; Tue, 11 Jul 1995 22:04:02 -0500 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) id UAA28003; Tue, 11 Jul 1995 20:03:59 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) id UAA24395; Tue, 11 Jul 1995 20:03:55 -0700 From: "Eric Salo" Message-Id: <9507112003.ZM24393@mrjones.engr.sgi.com> Date: Tue, 11 Jul 1995 20:03:53 -0700 In-Reply-To: Rik Littlefield "RE: non-blocking ops vs. threads" (Jul 11, 6:56pm) References: <9507120156.AA25373@maddog.pnl.gov> X-Mailer: Z-Mail (3.2.1 6apr95 MediaMail) To: Rik Littlefield Subject: Re: non-blocking ops vs. threads Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > We at PNL need remote data access. We can use other mechanisms > like interrupt receive, active messages, and good thread > interfaces (assuming proper definitions in each case) as ways to > implement remote data access, but they're not really what we > want. > > So how about it, developers -- what is the cost of including > remote data access, and if it's high, then why, and how might it > be reduced? Okay, this is starting to diverge from the original topic but let's take it a bit further anyway...how exactly do you define "remote access?" At the last MPI Forum meeting, I made the suggestion that since "puts" are often so much messier than "gets", perhaps we ought to consider only adding the "gets" to the standard and just hope that would be sufficient for most real user needs. This was immediately blasted by everyone and their dog as a Bad Idea. What are your specific needs as a MPI programmer? P.S. Remote data access on our machines is dirt cheap. :-) -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Tue Jul 11 23:04:03 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA21669; Tue, 11 Jul 1995 23:04:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA16896 for mpi-core-out; Tue, 11 Jul 1995 22:12:26 -0500 Received: from mcs.anl.gov (dalek.mcs.anl.gov [140.221.1.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA16887; Tue, 11 Jul 1995 22:12:19 -0500 Message-Id: <199507120312.WAA16887@antares.mcs.anl.gov> To: Rik Littlefield cc: gdburns@tbag.osc.edu, mpi-core@mcs.anl.gov, j_nieplocha@pnl.gov, m_rosing@pnl.gov Subject: Re: non-blocking ops vs. threads In-reply-to: Your message of "Tue, 11 Jul 1995 18:56:43 PDT." <9507120156.AA25373@maddog.pnl.gov> Date: Tue, 11 Jul 1995 22:12:18 -0500 From: Ian Foster Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Rik observes: > I suspect that the costs in question 2 are very much different > for active messages and interrupt receive than for remote data > access. I think it is more precise to say that different functionalities have different costs, depending on whether the remote logic associated with the function can be executed in hardware; in an ISR; on the same stack as the current thread; in a special thread; etc (and many variants of these mechanisms). The cost and choice of mechanism required for a particular functionality are _very_ system dependent. In particular, on many systems, the mechanism and cost associated with a traditional "active message" will not be very different from that of a remote data access, because (almost by definition), an active message is restricted in form to those functions that admit the same sort of very lightweight implementation. Given the range of implementation alternatives and associated functionality/cost/complexity tradeoffs, it is certainly reasonable to conclude that we should only support one or two things that we know we can almost always do using the very simplest mechanism: e.g., simple forms of put and get. However, I do have the feeling that with a little more thought we could define a much more general mechanism that would support a much wider range of functionalities. As on many machines a remote get or put will involve execution of remote code, one way of addressing the issue is to ask the question "what can this code reasonably do without involving the use of more expensive mechanisms?" It can certainly do more than parse an MPI datatype. Can it allocate memory? Can it call functions? Can it make system calls? Ian. From mpi-core-human@mcs.anl.gov Tue Jul 11 23:31:08 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA25051; Tue, 11 Jul 1995 23:31:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA17184 for mpi-core-out; Tue, 11 Jul 1995 22:39:23 -0500 Received: from taurus.cs.nps.navy.mil (cs.nps.navy.mil [131.120.1.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id WAA17176 for ; Tue, 11 Jul 1995 22:39:17 -0500 Received: from holmes.cs.nps.navy.mil by taurus.cs.nps.navy.mil (4.1/SMI-4.1) id AA25853; Tue, 11 Jul 95 20:38:36 PDT Received: by holmes.cs.nps.navy.mil (950215.SGI.8.6.10/940406.SGI) for mpi-core@mcs.anl.gov id UAA01589; Tue, 11 Jul 1995 20:38:34 -0700 From: "Robert George" Message-Id: <9507112038.ZM1587@holmes.cs.nps.navy.mil> Date: Tue, 11 Jul 1995 20:38:32 -0700 In-Reply-To: "Eric Salo" "Re: non-blocking ops vs. threads" (Jul 11, 8:03pm) References: <9507120156.AA25373@maddog.pnl.gov> <9507112003.ZM24393@mrjones.engr.sgi.com> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: non-blocking ops vs. threads Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Jul 11, 8:03pm, Eric Salo wrote: > Subject: Re: non-blocking ops vs. threads > > We at PNL need remote data access. > > > > So how about it, developers -- what is the cost of including > > remote data access, and if it's high, then why, and how might it > > be reduced? One issue that can be difficult is maintaining the coherence of the cache on a processing node. In other words, when data gets "put" in your local memory, you will have to flush any associated cache lines. This also implies that you will have to implement a memory consistency model across the members of the communicator, with some sort of barrier to keep everyone in sync. > At the last MPI Forum meeting, I made the suggestion that since "puts" are > often so much messier than "gets", perhaps we ought to consider only adding > the "gets" to the standard and just hope that would be sufficient for most > real user needs. Puts are required for active messages, in which many people in this group (myself included) are very interested. > P.S. Remote data access on our machines is dirt cheap. :-) So how exactly do you build a bus-based share-memory machine with more than 24 processors? :) Robert George Army Research Laboratory From mpi-core-human@mcs.anl.gov Tue Jul 11 23:48:40 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA00391; Tue, 11 Jul 1995 23:48:39 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA17354 for mpi-core-out; Tue, 11 Jul 1995 22:56:48 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA17342 for ; Tue, 11 Jul 1995 22:56:42 -0500 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) id UAA02940; Tue, 11 Jul 1995 20:56:40 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) id UAA24726; Tue, 11 Jul 1995 20:56:37 -0700 From: "Eric Salo" Message-Id: <9507112056.ZM24724@mrjones.engr.sgi.com> Date: Tue, 11 Jul 1995 20:56:35 -0700 In-Reply-To: "Robert George" "Re: non-blocking ops vs. threads" (Jul 11, 8:38pm) References: <9507120156.AA25373@maddog.pnl.gov> <9507112003.ZM24393@mrjones.engr.sgi.com> <9507112038.ZM1587@holmes.cs.nps.navy.mil> X-Mailer: Z-Mail (3.2.1 6apr95 MediaMail) To: "Robert George" , mpi-core@mcs.anl.gov Subject: Re: non-blocking ops vs. threads Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > One issue that can be difficult is maintaining the coherence of the cache on a > processing node. In other words, when data gets "put" in your local memory, > you will have to flush any associated cache lines. ...which is exactly why I suggested ignoring the puts entirely. When a process does a get, the get function can then immediately invalidate its own cache if the hardware is too stupid to do it for you. Problem solved. With a put, you now need some sort of portable sync mechanism so that the destination process knows to invalidate its cache. Yuck. > Puts are required for active messages, in which many people in this group > (myself included) are very interested. Well, if active messages are really what people want/need, then that's what we should be talking about standardizing. They don't have to be implemented with puts, that's just one possible (and perhaps optimal) mechanism. I would argue that specifying a decent, portable active message interface would, at worst, still be easier than specifying a decent, portable put interface. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Wed Jul 12 06:18:55 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA07300; Wed, 12 Jul 1995 06:18:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA20475 for mpi-core-out; Wed, 12 Jul 1995 05:27:00 -0500 Received: from hub (root@hub.meiko.co.uk [192.131.108.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id FAA20467 for ; Wed, 12 Jul 1995 05:26:49 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA26384 (5.65c/IDA-1.4.4 for mpi-core@mcs.anl.gov); Wed, 12 Jul 1995 11:26:14 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA14844; Wed, 12 Jul 1995 11:26:21 +0000 Date: Wed, 12 Jul 1995 11:26:21 +0000 Message-Id: <9507121026.AA14844@deneb.co.uk> From: James Cownie To: rj_littlefield@pnl.gov Cc: gdburns@tbag.osc.edu, mpi-core@mcs.anl.gov, j_nieplocha@pnl.gov, m_rosing@pnl.gov In-Reply-To: <9507120156.AA25373@maddog.pnl.gov> (message from Rik Littlefield on Tue, 11 Jul 1995 18:56:43 -0700 (PDT)) Subject: Re: non-blocking ops vs. threads Reply-To: jim@meiko.com Content-Length: 3935 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Rik writes > We at PNL need remote data access. We can use other mechanisms > like interrupt receive, active messages, and good thread > interfaces (assuming proper definitions in each case) as ways to > implement remote data access, but they're not really what we > want. Since remote data access is a *much* more constrained problem than the other alternatives Rik mentions (interrupt receive, active messages etc), I'm strongly in favour of giving him what he wants (which is easy to specify), rather than something he doesn't want which is hardto do. If you don't think specifying the other mechanisms is hard, simply look at Ian's list of questions. Ian... > However, I do have the feeling that with a > little more thought we could define a much more general mechanism that > would support a much wider range of functionalities. As on many > machines a remote get or put will involve execution of remote code, > one way of addressing the issue is to ask the question "what can this > code reasonably do without involving the use of more expensive > mechanisms?" It can certainly do more than parse an MPI datatype. > Can it allocate memory? Can it call functions? Can it make system > calls? Back to Rik > So how about it, developers -- what is the cost of including > remote data access, and if it's high, then why, and how might it > be reduced? On the Meiko CS-2 remote store access is easy (checked in the right places and cache coherent) it does *not* require code to be executed remotely. Note also that remote write (which some people want to do away with) is inherently faster than remote read, because the write requires one network transit time, while the read requires two. Even if we're going to add other more complex mechanisms, I strongly believe that we should specify a simple, direct, interface to both remote read and remote write. Another issue which needs to be addressed in specifying the more complex interface is where the remotely invoked code runs ? In the CS-2 we have a small processor in the communications interface. This is mapped into user space, and the user can write code for it. (This code is loaded along with her executable, and has no more privilege than any other code in the process). However this code needs to be compiled with a different compiler, since this "thread" processor is small (e.g. it has no floating point instructions, and has extra instructions targetted at doing comms). Threads of code running in this processor can be woken by comms events *without* having to take an interrupt on the main processor(s). Thus the impact of protocol handling can be removed completely from the application, and overlapped with computation. This is very useful for building communication libraries, and run-time libraries for parallel languages. However I woudn't suggest that an average Fortran programmer go near it ! The point I'm making here is that executing "random user subroutine" will never be as good as executing something that we knew a priori was for the comms processor, and that was compiled that way. Viewing the issue another way, I think what I'm trying to ask is who we expect to use these facilities, (random Fortran programmers, or people writing the run time for a parallel language) ? If it is the latter, then the simplicity of the interface is less important than the performance which can be achieved, and it may be that we simply can't specify anything which is useful, because the portability comes at too high a performance cost (Lloyd's strongly argued point that portability without performance is no portability at all.) -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From mpi-core-human@mcs.anl.gov Wed Jul 12 17:31:39 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA07194; Wed, 12 Jul 1995 17:31:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA04983 for mpi-core-out; Wed, 12 Jul 1995 16:37:35 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA04975 for ; Wed, 12 Jul 1995 16:37:27 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA08927; Wed, 12 Jul 95 16:37:18 CDT Date: Wed, 12 Jul 95 16:37:18 CDT From: Tony Skjellum Message-Id: <9507122137.AA08927@Aurora.CS.MsState.Edu> To: mpi-core@mcs.anl.gov Subject: Comm. Pattern for MPI-2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk We are now seeing more than one application that has the following communication pattern. David O'Hallaron at CMU wants this for compilers. Folks at PNL want this for their chemistry codes. Who else? -Tony Starting conditions. A page-oriented communicator has been initialized, so that the puts shown below make sense for the datatypes involved. Here it is [informal C]: MPI_Allput(MPI_Comm comm, /* data-page-oriented communicator */ int local_send_count, void *local_addr[], MPI_Type local_type[], void *remote_addr[], MPI_Type remote_type[], int dest_rank[], volatile int *recv_counts[]) { int size, myrank; int *local_send_counts, global_send_counts; MPI_Comm_size(comm, &size); MPI_Comm_rank(comm, &myrank); *recv_counts[myrank] = 0; /* this is a local address */ local_send_counts = new(int,size); global_send_counts = new(int,size); for(i =0; i < size; i++) local_counts[i] = 0; MPI_Barrier(comm); for(i = 0; i < local_send_count; i++) { MPI_Put(local_data[i], local_type[i], remote_data[i], remote_type[i], dest_rank[i], recv_counts[i], comm); ++local_counts[dest_rank[i]]; } /* find out how many messages have been sent to everyone: */ MPI_Allreduce(local_counts, global_counts, MPI_INT, size, MPI_SUM, comm); /* check arg order here :-) */ /* let incomplete puts complete after allreduce, which is effective barrier as well, so we know all puts have been started, at least */ while(*recv_counts[myrank] < global_counts[myrank]) ; /* spin until puts complete; assumes comparison is atomic */ /* clean up */ free(local_counts); free(global_counts); } Comments: 1. Assumes that MPI_Put increments recv_counts[i] when put completes. 2. No other signalling, synchronization or remote execution. 3. Do we need to implement MPI_Allput() in the standard, since it can be layered [as above, assuming MPI_Put() is nonblocking]? Whether or not we should add a call for this, it is a useful pattern. Furthermore, it shows that MPI_Put is useful inasmuch as produces a separation of synchronization and transfer. 4. This all assumes that MPI_Put() is flow controlled, so that one does not swamp a network by blasting puts. This seems to put that burden on implementations. [Should I have used MPI_Iput() instead with requests and later waits. I really want a non-blocking call that does not make me wait later on each one... ] 5. Is my use of MPI_Put OK based on MPI-2 draft? AM I really able to get non-blocking Put this way, or with MPI_Iput? If I have to wait on every Put, if this better than message passing? From mpi-core-human@mcs.anl.gov Wed Jul 12 17:47:48 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA07244; Wed, 12 Jul 1995 17:47:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA05334 for mpi-core-out; Wed, 12 Jul 1995 16:54:40 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA05326 for ; Wed, 12 Jul 1995 16:54:32 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2093; Wed, 12 Jul 95 17:53:45 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 9439; Wed, 12 Jul 1995 17:53:45 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Wed, 12 Jul 95 17:53:44 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA16779; Wed, 12 Jul 1995 17:53:24 -0400 Message-Id: <9507122153.AA16779@snir.watson.ibm.com> To: mpi-core@mcs.anl.gov Subject: cost of single sided communication Reply-To: snir@watson.ibm.com Date: Wed, 12 Jul 1995 17:53:24 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: Contrary to what Greg Burns writes, it is not the case that one can correctly implement MPI with a pure polling based implementation. Such implementation will violate the progress rule -- this is the reason why Greg does not like this rule. In order to guarantee progress, it is necessary to "poll" at the receiving end even if no MPI calls are executed. We do achieve this in our implementation by having the low-level polling layer of MPI activated periodically by a clock interrupt. Paragon or Meilo may achieve the same effect by having their communication processor "poll" autonomously. It is not necessary to do so frequently, as polling executed "on the side" as result of other MPI calls is usually sufficient. Thus, a correct implementation of MPI already requires "single-sided" communication. Information on a posted send has to be promoted and moved to the receiving process even if the receiving process executes no MPI calls. The receiving process has to clear this information from the network, even if it executes no MPI call. Under the user visible layer of communication effected by calls on both ends there must be an underlying layer where communication occurs as result of the initiative of one process only. Single-sided communication does not require any change in this strategy and will not add any overhead to such MPI implementation. It will be a problem for MPI implementations that cheat on the MPI semantics now. From mpi-core-human@mcs.anl.gov Wed Jul 12 18:31:30 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA07565; Wed, 12 Jul 1995 18:31:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA05980 for mpi-core-out; Wed, 12 Jul 1995 17:38:19 -0500 Received: from igate1.hac.com (igate1.hac.com [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA05972 for ; Wed, 12 Jul 1995 17:38:13 -0500 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (4.1/SMI-4.1) id AA14056; Wed, 12 Jul 95 15:35:39 PDT Message-Id: Date: 12 Jul 1995 15:33:54 -0800 From: "Lewins, Lloyd J" Subject: RE: cost of single sided communication To: "mpi-core" , snir@watson.ibm.com X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc wrote: >Contrary to what Greg Burns writes, it is not the case that one can correctly >implement MPI with a pure polling based implementation. Such implementation >will violate the progress rule... I don't think this is strictly correct. In a shared memory implementation of MPI, where the sender can directly manipulate the receive queues, a pure polling implementation won't violate the progress rule (at least not because it doesn't "promote" sends). Lloyd Lewins Hughes Aircraft Co, llewins@msmail4.hac.com From mpi-core-human@mcs.anl.gov Wed Jul 12 19:03:04 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA08089; Wed, 12 Jul 1995 19:03:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA06891 for mpi-core-out; Wed, 12 Jul 1995 18:09:25 -0500 Received: from infomeister.osc.edu (infomeister.osc.edu [128.146.36.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA06883 for ; Wed, 12 Jul 1995 18:09:18 -0500 Received: from tbag.osc.edu for gdburns@tbag.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id TAA13075; Wed, 12 Jul 1995 19:09:15 -0400 Received: by tbag.osc.edu (4.1/OSC-SUB-2.0) id AA25381; Wed, 12 Jul 95 19:09:14 EDT Date: Wed, 12 Jul 95 19:09:14 EDT From: gdburns@tbag.osc.edu (Greg Burns) Message-Id: <9507122309.AA25381@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: Re: cost of single sided communication Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >Such implementation will violate the progress rule -- this is the reason >why Greg does not like this rule. As we stated in the paper, the reason we don't like the progress rule is that it adds little and confuses a lot. The polling point would be the second reason we don't like it if we agreed with Marc's interpretation. I don't, but it does clarify an earlier point that Marc made, which he reiterates: >Single-sided communication does not require any change in this strategy >and will not add any overhead to such MPI implementation. Marc states that the mandatory strategy is: >In order to guarantee progress, it is necessary to "poll" at the >receiving end even if no MPI calls are executed. Well, implementors, is everybody doing this? The progress rule states that matching communication will complete "independently of other actions in the system". This would be your only leg to stand on, I think. My interpretation of this ambiguous line is "actions by other processes". It can't be action within the implementation, because Marc is saying that timeout action is required to get it done. At the least there is enough doubt here that the interpretation of us "cheating" implementations has be considered valid. I have heard it said by MPI-1 people that a correct program must call MPI_Finalize(). I can't find this in the doc, but it is the truth that I live by. The doc does say that all pending communication must be completed by the program before calling MPI_Finalize(). Therefore I conclude that either this mandatory action (to avoid being an erroneous program - TILT) or some other intervening call will yield to the implementation and allow source buffered messages to move along. Pure polling library-only implementations are fully valid. Thankfully. -- Greg From mpi-core-human@mcs.anl.gov Wed Jul 12 20:11:18 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA08555; Wed, 12 Jul 1995 20:11:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA07527 for mpi-core-out; Wed, 12 Jul 1995 19:17:35 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id TAA07519 for ; Wed, 12 Jul 1995 19:17:28 -0500 From: snir@watson.ibm.com Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 4017; Wed, 12 Jul 95 20:16:25 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 3049; Wed, 12 Jul 1995 20:16:25 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Wed, 12 Jul 95 20:16:24 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA25755; Wed, 12 Jul 1995 20:16:04 -0400 Message-Id: <9507130016.AA25755@snir.watson.ibm.com> To: "Robert George" Subject: Re: non-blocking ops vs. threads Cc: mpi-core@mcs.anl.gov In-Reply-To: (Your message of Tue, 11 Jul 1995 20:38:32 PDT.) Date: Wed, 12 Jul 95 20:16:24 EDT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk : -) :-) :-) *** (-: (-: (-: Date: Wed, 12 Jul 1995 20:16:04 -0400 From: Marc Snir >> To: mpi-core@mcs.anl.gov >> Subject: Re: non-blocking ops vs. threads >> Mime-Version: 1.0 >> Content-Type: text/plain; charset=us-ascii >> Sender: owner-mpi-core@mcs.anl.gov >> Precedence: bulk >> >> On Jul 11, 8:03pm, Eric Salo wrote: >> > Subject: Re: non-blocking ops vs. threads >> > > We at PNL need remote data access. >> > > >> > > So how about it, developers -- what is the cost of including >> > > remote data access, and if it's high, then why, and how might it >> > > be reduced? >> >> One issue that can be difficult is maintaining the coherence of the cache on a >> processing node. In other words, when data gets "put" in your local memory, >> you will have to flush any associated cache lines. >> This is true only if the remote "put agent" is not cache coherent with the local processor(s). While this may be the case for a Cray T3D, it is unlikely to be the case on most present/future systems. If put is implemented by software executed by the computation processor, the problem does not occur. If put is implemented by software in a cache coherent communication coprocessor (Meiko, Paragon) the problem does not occur. If put is implemented by a local controller with a cache-coherent DMA engine, the problem does not occur. >> This also implies that you will have to implement a memory consistency model >> across the members of the communicator, with some sort of barrier to keep >> everyone in sync. No, one does not need global memory coherence across all members of the group. One only needs the local agent that implements the put/get (main processor, communication coprocessor, DMA engine,...) to be coherent with the local computation processor. >> >> > At the last MPI Forum meeting, I made the suggestion that since "puts" are >> > often so much messier than "gets", perhaps we ought to consider only adding >> > the "gets" to the standard and just hope that would be sufficient for most >> > real user needs. In a machine where the remote put/get engine is not cache coherent get is also a problem, since one needs the remote processor to flush its cache before the get is executed. But the remote processor is not supposed to be aware that a get is executed! >> >> Puts are required for active messages, in which many people in this group >> (myself included) are very interested. I don't understand this statement. I know a fair number of systems that support active messages but do not support puts, and vice-versa. In any case, the issue is not what are the basic communication mechanisms on the underlying machine, the issue is what is made visible by MPI to the user. We can decide to provide an active message interface, we can decide to provide a put/get interface, we can decide to provide both. Implementers may, at their convenience, use one to implemente the other. >> >> > P.S. Remote data access on our machines is dirt cheap. :-) >> >> So how exactly do you build a bus-based share-memory machine with more than 24 >> processors? :) >> >> Robert George >> Army Research Laboratory >> >> ------- End of Unsent Draft From mpi-core-human@mcs.anl.gov Wed Jul 12 20:48:06 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA08645; Wed, 12 Jul 1995 20:48:05 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA07842 for mpi-core-out; Wed, 12 Jul 1995 19:56:27 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id TAA07834 for ; Wed, 12 Jul 1995 19:56:21 -0500 From: snir@watson.ibm.com Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 4387; Wed, 12 Jul 95 20:55:38 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 4883; Wed, 12 Jul 1995 20:55:37 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Wed, 12 Jul 95 20:55:37 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA16941; Wed, 12 Jul 1995 20:55:17 -0400 Message-Id: <9507130055.AA16941@snir.watson.ibm.com> To: gdburns@tbag.osc.edu (Greg Burns) Cc: mpi-core@mcs.anl.gov Subject: why "offline polling" is needed -- at least on parallel machines In-Reply-To: (Your message of Wed, 12 Jul 1995 19:09:14 EDT.) Date: Wed, 12 Jul 95 20:55:37 EDT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk : -) :-) :-) *** (-: (-: (-: Date: Wed, 12 Jul 1995 20:55:17 -0400 From: Marc Snir In a distributed memory system (with no hw support for put) when a send is posted, the source has to move at least the message header to the destination. This header will be sent, whether a matching receive is posted or not at the destination. Since very few systems can tolerate that data sits on the wire forever, the data will be accepted, at some level of the communication protocol stack, at the receiving node. If communication uses IP, then an interrupt occurs because the LAN adapter signals incoming data. If communication uses some user-space driver, then this driver has to poll periodically the wire. In either cases it is not too expensive to invoke the next layer of the communication protocol when data has arrived. I doubt very much that any hardware vendor will accept a design where a badly written program can clog the network. Some form of flow control, and therefore some form of "offline polling" or of interrupt driven receive will exist in the communication subsystem. I suspect, in fact, that such "offline" polling does occur in the system that Greg implemented, even if incoming data is not processed, at the next level of the communication stack, until an MPI call occurs at the receiving node. The bottom layer of IP either buffers the data at the receive node or instruct the sender to resend latter. From mpi-core-human@mcs.anl.gov Thu Jul 13 00:52:57 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA10035; Thu, 13 Jul 1995 00:52:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id AAA10129 for mpi-core-out; Thu, 13 Jul 1995 00:00:31 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id AAA10111 for ; Thu, 13 Jul 1995 00:00:17 -0500 Received: by Aurora.CS.MsState.Edu (4.1/6.0s-FWP); id AA09372; Thu, 13 Jul 95 00:00:15 CDT Date: Thu, 13 Jul 95 00:00:15 CDT From: Tony Skjellum Message-Id: <9507130500.AA09372@Aurora.CS.MsState.Edu> To: mpi-core@mcs.anl.gov Subject: Contributions Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk If there are proposals, contributions, or suggestions for realtime and collective chapters, I need them by Monday. -Tony From mpi-core-human@mcs.anl.gov Thu Jul 13 06:22:49 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA18214; Thu, 13 Jul 1995 06:22:48 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA15750 for mpi-core-out; Thu, 13 Jul 1995 05:30:55 -0500 Received: from hub (root@hub.meiko.co.uk [192.131.108.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id FAA15739 for ; Thu, 13 Jul 1995 05:30:36 -0500 Received: from deneb.co.uk (deneb.meiko.co.uk) by hub with SMTP id AA07382 (5.65c/IDA-1.4.4 for mpi-core@mcs.anl.gov); Thu, 13 Jul 1995 11:30:07 +0100 Received: by deneb.co.uk (5.0/SMI-SVR4) id AA18903; Thu, 13 Jul 1995 10:31:57 +0000 Date: Thu, 13 Jul 1995 10:31:57 +0000 Message-Id: <9507130931.AA18903@deneb.co.uk> From: James Cownie To: gdburns@tbag.osc.edu Cc: mpi-core@mcs.anl.gov In-Reply-To: <9507122309.AA25381@tbag.osc.edu> (gdburns@tbag.osc.edu) Subject: Re: cost of single sided communication Reply-To: jim@meiko.com Content-Length: 0 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > >In order to guarantee progress, it is necessary to "poll" at the > >receiving end even if no MPI calls are executed. > > Well, implementors, is everybody doing this? On the Meiko CS-2 we don't have any polling, but that's because we work in an event driven manner. There's a thread in the comms processor which is woken when anyone else wants to send to a tagged message port within a process. It receives the envelope and does a match against outstanding receive requests, then either allocates buffer space for the message (if there isn't an outstanding receive), or tells the sender where to put the message (direct to the user buffer) if there was a matching receive. None of this involves the main processor, which carries on executing code throughout, and doesn't get any interrupts. So 1) No polling 2) Data from the network is ALWAYS put into store without needing any main processor intervention (otherwise you are potentially blocking other unrelated communications by holding a route through the switch open). 3) Tag matching is handled by the comms processor. (In user level code, though). the effect of this is that messages can arrive without requiring any intervention from the main processor (it could have process switched to another process, or be executing in the kernel for all we care). The only limitation will be if you run out of space for envelope information, but since this is allocated from the virtual address space of the process, all reasonable cases (and some unreasonable ones !) should be OK (if not increase the amount of swap space, and expect things to run slowly while the envelopes churn on and off disk. Not a recommended thing to do...) -- Jim James Cownie Meiko Limited Meiko Inc. 650 Aztec West 130C Baker Avenue Ext. Bristol BS12 4SD Concord England MA 01742 Phone : +44 1454 616171 +1 508 371 0088 FAX : +44 1454 618188 +1 508 371 7516 E-Mail: jim@meiko.co.uk or jim@meiko.com WWW : http://www.meiko.com/welcome.html From mpi-core-human@mcs.anl.gov Thu Jul 13 06:37:23 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA18225; Thu, 13 Jul 1995 06:37:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA15881 for mpi-core-out; Thu, 13 Jul 1995 05:45:37 -0500 Received: from infomeister.osc.edu (infomeister.osc.edu [128.146.36.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id FAA15873 for ; Thu, 13 Jul 1995 05:45:31 -0500 Received: from alex.osc.edu for raja@alex.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id GAA20965; Thu, 13 Jul 1995 06:45:29 -0400 Received: by alex.osc.edu (4.1/OSC-SUB-2.0) id AA07688; Thu, 13 Jul 95 06:41:14 EDT Date: Thu, 13 Jul 95 06:41:14 EDT From: raja@alex.osc.edu (Raja Daoud) Message-Id: <9507131041.AA07688@alex.osc.edu> To: mpi-core@mcs.anl.gov Subject: Re: why "offline polling" is needed -- at least on parallel machines Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Since very few systems can tolerate that data sits on the wire forever, > the data will be accepted, at some level of the communication protocol > stack, at the receiving node. This unnecessarily narrows down the scope of supported machines. MPI should be able to run on systems where this is desirable and certainly run on systems where no interrupt support is provided. These systems should not be dismissed just because they don't resemble the current vendors architectures. > If communication uses IP, then an interrupt occurs because the LAN > adapter signals incoming data. True, but a restrictive view of the system. An interrupt may happen deep in the bowels of the system but that doesn't mean it is necessarily visible to the MPI implementor. In such a case, implementors dealing with a single machine may modify the system to support whatever the MPI implementation requires. This is not the case for those providing portable implementations. So I feel it would be unwise for MPI to place a must-have-interrupts restriction on systems. > If communication uses some user-space driver, then this driver has to > poll periodically the wire. This is one way of implementing it, but not the only way. An implementation may decide to only poll when the user calls an MPI function. After all, in MPI-1, if users don't call recv/test/wait/... they can't find out if a message was received or not. So, in some systems, it is a good optimization to only poll/read the wire (and buffer the received envelope if necessary) when the user asks for an "update", otherwise letting the process keep computing with no interruptions [which is also one way to make the computation segments run for communication-independent time periods]. I don't see why we should remove this potential optimization/feature. > I doubt very much that any hardware vendor will accept a design where > a badly written program can clog the network. This depends on the network, the amount of buffering it supports, if it's store-and-forward or not, what protocols it uses, whether a 2nd "system" network is there for control purposes, etc... I'd like to add that implementations not tied to a specific vendor are part of the solution space. :-) Portability can be achieved by lots of #ifdefs or by using the lowest common denominator, and I want to keep that option open. > Some form of flow control, and therefore some form of "offline polling" > or of interrupt driven receive will exist in the communication subsystem. Flow control doesn't imply off-line-polling, just like immediately removing the message from the wire isn't the only way of doing things. It's clear that we have different views on how an implementation should be done, but I think we can agree that MPI shouldn't dictate this. > I suspect, in fact, that such "offline" > polling does occur in the system that Greg implemented, even if incoming [LAM isn't the issue here - its daemon makes most of the proposed extensions easy enough to implement.] The main issue is the introduction of unnecessarily restrictive assumptions about the machine's architecture. A system that doesn't support implementor-visible interrupts should be able to have a fully compliant MPI implementation. Likewise, a system that has no problem keeping messages on some of its wires until the next update call is issued, should be considered. Diversity is good! :-) --Raja From mpi-core-human@mcs.anl.gov Thu Jul 13 13:08:49 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA20035; Thu, 13 Jul 1995 13:08:05 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA22723 for mpi-core-out; Thu, 13 Jul 1995 12:15:08 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA22659; Thu, 13 Jul 1995 12:13:54 -0500 Message-Id: <199507131713.MAA22659@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov, mpi-dynamic@mcs.anl.gov Subject: Dynamic Process Management Date: Thu, 13 Jul 1995 12:13:51 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Enclosed is a greatly revised the "Dynamic Processes" Chapter based on discussions both at the last meeting and since then. (Bill Saphir has done most of the work.) It is still rough in parts (the examples haven't been updated yet), but it is in good enough shape to start some discussions. Please take a look and comment. Perhaps the discussion should move from mpi-core to mpi-dynamic. To post mail there, send to mpi-dynamic@mcs.anl.gov To subscribe to that discussion group if you are not already subscribed, send the message subscribe mpi-dynamic to majordomo@mcs.anl.gov Rusty %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Thu Jul 13 09:11:17 1995 %%Pages: 32 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1995.07.13:0910 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet %%BeginProcSet: special.pro TeXDict begin /SDict 200 dict N SDict begin /@SpecialDefaults{/hs 612 N /vs 792 N /ho 0 N /vo 0 N /hsc 1 N /vsc 1 N /ang 0 N /CLIP 0 N /rwiSeen false N /rhiSeen false N /letter{}N /note{}N /a4{}N /legal{}N}B /@scaleunit 100 N /@hscale{@scaleunit div /hsc X}B /@vscale{@scaleunit div /vsc X}B /@hsize{/hs X /CLIP 1 N}B /@vsize{/vs X /CLIP 1 N}B /@clip{ /CLIP 2 N}B /@hoffset{/ho X}B /@voffset{/vo X}B /@angle{/ang X}B /@rwi{ 10 div /rwi X /rwiSeen true N}B /@rhi{10 div /rhi X /rhiSeen true N}B /@llx{/llx X}B /@lly{/lly X}B /@urx{/urx X}B /@ury{/ury X}B /magscale true def end /@MacSetUp{userdict /md known{userdict /md get type /dicttype eq{userdict begin md length 10 add md maxlength ge{/md md dup length 20 add dict copy def}if end md begin /letter{}N /note{}N /legal{} N /od{txpose 1 0 mtx defaultmatrix dtransform S atan/pa X newpath clippath mark{transform{itransform moveto}}{transform{itransform lineto} }{6 -2 roll transform 6 -2 roll transform 6 -2 roll transform{ itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll curveto}}{{ closepath}}pathforall newpath counttomark array astore /gc xdf pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{PaintBlack}if}N /txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 -1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop 90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr 2 get ppr 0 get neg sub neg 0 S TR}if}ifelse scaleby96{ppr aload pop 4 -1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S TR}if}N /cp {pop pop showpage pm restore}N end}if}if}N /normalscale{Resolution 72 div VResolution 72 div neg scale magscale{DVImag dup scale}if 0 setgray} N /psfts{S 65781.76 div N}N /startTexFig{/psf$SavedState save N userdict maxlength dict begin /magscale false def normalscale currentpoint TR /psf$ury psfts /psf$urx psfts /psf$lly psfts /psf$llx psfts /psf$y psfts /psf$x psfts currentpoint /psf$cy X /psf$cx X /psf$sx psf$x psf$urx psf$llx sub div N /psf$sy psf$y psf$ury psf$lly sub div N psf$sx psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub TR /showpage{}N /erasepage{}N /copypage{}N /p 3 def @MacSetUp}N /doclip{ psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto closepath clip newpath moveto}N /endTexFig{end psf$SavedState restore}N /@beginspecial{SDict begin /SpecialSave save N gsave normalscale currentpoint TR @SpecialDefaults count /ocount X /dcount countdictstack N}N /@setspecial {CLIP 1 eq{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs neg 0 rlineto closepath clip}if ho vo TR hsc vsc scale ang rotate rwiSeen{rwi urx llx sub div rhiSeen{rhi ury lly sub div}{dup}ifelse scale llx neg lly neg TR }{rhiSeen{rhi ury lly sub div dup scale llx neg lly neg TR}if}ifelse CLIP 2 eq{newpath llx lly moveto urx lly lineto urx ury lineto llx ury lineto closepath clip}if /showpage{}N /erasepage{}N /copypage{}N newpath }N /@endspecial{count ocount sub{pop}repeat countdictstack dcount sub{ end}repeat grestore SpecialSave restore end}N /@defspecial{SDict begin} N /@fedspecial{end}B /li{lineto}B /rl{rlineto}B /rc{rcurveto}B /np{ /SaveX currentpoint /SaveY X N 1 setlinecap newpath}N /st{stroke SaveX SaveY moveto}N /fil{fill SaveX SaveY moveto}N /ellipse{/endangle X /startangle X /yrad X /xrad X /savematrix matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix}N end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 1 111 df<1E07802318C023A06043C0704380704380708700E00700E00700 E00700E00E01C00E01C00E01C00E03821C03841C07041C07081C03083803101801E01714 7E931B>110 D E /Fb 2 104 df<003C00E001C001800380038003800380038003800380 038003800380038003800380030007001C00F0001C000700030003800380038003800380 03800380038003800380038003800380018001C000E0003C0E297D9E15>102 DI E /Fc 12 117 df45 D<03CC063C0C3C181C3838303870387038E070E070E070E070E0E2C0E2C0E261E4 62643C380F127B9115>97 D<01F007080C08181C3838300070007000E000E000E000E000 E000E008E010602030C01F000E127B9113>99 D<001F8000038000038000070000070000 0700000700000E00000E00000E00000E0003DC00063C000C3C00181C0038380030380070 3800703800E07000E07000E07000E07000E0E200C0E200C0E20061E4006264003C380011 1D7B9C15>I<0003C0000670000C70001C60001C00001C00003800003800003800003800 00380003FF8000700000700000700000700000700000E00000E00000E00000E00000E000 01C00001C00001C00001C00001C000038000038000038000030000030000070000C60000 E60000CC00007800001425819C0D>102 D<0FC00001C00001C000038000038000038000 0380000700000700000700000700000E78000E8C000F0E000E0E001C0E001C0E001C0E00 1C0E00381C00381C00381C00383800703880703880707080707100E03200601C00111D7D 9C15>104 D<3C1E0780266318C04683A0E04703C0E08E0380E08E0380E00E0380E00E03 80E01C0701C01C0701C01C0701C01C070380380E0388380E0388380E0708380E0710701C 0320300C01C01D127C9122>109 D<3C3C002646004687004707008E07008E07000E0700 0E07001C0E001C0E001C0E001C1C00381C40381C40383840383880701900300E0012127C 9117>I<01E007180C0C180C380C300E700E700EE01CE01CE01CE018E038E030E06060C0 31801E000F127B9115>I<3C3C26C2468747078E068E000E000E001C001C001C001C0038 003800380038007000300010127C9112>114 D<01F006080C080C1C18181C001F001FC0 0FF007F0007800386030E030C030806060C01F000E127D9111>I<00C001C001C001C003 80038003800380FFE00700070007000E000E000E000E001C001C001C001C003840384038 40388019000E000B1A7D990E>I E /Fd 27 122 df<0000600000E00000E00001C00001 C0000380000380000380000700000700000700000E00000E00001C00001C00001C000038 0000380000380000700000700000E00000E00000E00001C00001C0000380000380000380 000700000700000700000E00000E00001C00001C00001C00003800003800003800007000 00700000E00000E00000C00000132D7DA11A>47 D<0000E000000000E000000001F00000 0001F000000001F000000003F800000003F800000006FC00000006FC0000000EFE000000 0C7E0000000C7E000000183F000000183F000000303F800000301F800000701FC0000060 0FC00000600FC00000C007E00000FFFFE00001FFFFF000018003F000018003F000030001 F800030001F800060001FC00060000FC000E0000FE00FFE00FFFE0FFE00FFFE0231F7E9E 28>65 D69 D73 D77 D80 D<03FC080FFF381E03F83800F8700078700038F00038F00018F00018F80000FC0000 7FC0007FFE003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC0003C C0003CC0003CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>83 D<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03 E07C03E0F803E0F803E0F803E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007 FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000FC0000FC00007C 00007E00003E00603F00C01F81C007FF0001FC0013147E9317>I<0007F80007F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F801F8F80FFEF81F83F8 3E01F87E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F8 7E00F83E01F81F07F80FFEFF03F8FF18207E9F1D>I<01FE0007FF800F83C01E01E03E00 F07C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E00 180F807007FFE000FF8015147F9318>I104 D<1C003E007F007F007F003E001C000000000000000000 00000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F001F00 1F001F00FFE0FFE00B217EA00E>I107 DIII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC 007EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>I< FF1FC0FF7FF01FE1F81F80FC1F007E1F007E1F003E1F003F1F003F1F003F1F003F1F003F 1F003F1F003E1F007E1F007C1F80FC1FC1F81F7FE01F1F801F00001F00001F00001F0000 1F00001F00001F0000FFE000FFE000181D7E931D>I114 D<0FE63FFE701E600EE006E006F800FFC07F F83FFC1FFE03FE001FC007C007E007F006F81EFFFCC7F010147E9315>I<018001800180 03800380038007800F803F80FFFCFFFC0F800F800F800F800F800F800F800F800F800F80 0F860F860F860F860F8607CC03F801F00F1D7F9C14>IIII121 D E /Fe 42 123 df34 D<00E001E0038007000E001C001C0038003800700070 007000E000E000E000E000E000E000E000E000E000700070007000380038001C001C000E 000700038001E000E00B217A9C16>40 DI<387C7E7E3E0E1E1C78F060070B798416>44 DI<01800380038007800F807F80FF807380038003 800380038003800380038003800380038003800380038003807FF87FFC7FF80E197C9816 >49 D<01F18007FB800FFF801F0F803C0780380380700380700380F00000E00000E00000 E00000E00000E00000E00000E00000F000007003807003803803803C07001F0F000FFE00 07FC0001F00011197E9816>67 D70 D73 D77 D<7FF800FFFE007FFF001C0F801C03 801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FF8001C00001C00 001C00001C00001C00001C00001C00007F0000FF80007F000012197F9816>80 D<7FE000FFF8007FFC001C1E001C0F001C07001C07001C07001C07001C0F001C1E001FFC 001FF8001FFC001C1C001C0E001C0E001C0E001C0E001C0E201C0E701C0E707F07E0FF87 E07F03C014197F9816>82 D<07E3001FFF003FFF00781F00F00700E00700E00700E00000 F000007800003F80001FF00007FC0000FE00000F00000700000380000380600380E00380 E00700F80F00FFFE00FFFC00C7F00011197E9816>I87 D91 D93 D95 D<1FE0003FF0007FF800783C00300E00000E00 000E0003FE001FFE003E0E00700E00E00E00E00E00E00E00783E007FFFE03FE7E00F83E0 13127E9116>97 D<7E0000FE00007E00000E00000E00000E00000E00000E3E000EFF000F FF800F83C00F00E00E00E00E00700E00700E00700E00700E00700E00700E00E00F01E00F 83C00FFF800EFF00063C001419809816>I<03F80FFC1FFE3C1E780C7000E000E000E000 E000E000F000700778073E0E1FFC0FF803F010127D9116>I<003F00007F00003F000007 0000070000070000070003C7000FF7001FFF003C1F00780F00700700E00700E00700E007 00E00700E00700E00700700F00700F003C1F001FFFE00FE7F007C7E014197F9816>I<03 E00FF81FFC3C1E780E7007E007FFFFFFFFFFFFE000E000700778073C0F1FFE0FFC03F010 127D9116>I<001F00007F8000FF8001E78001C30001C00001C0007FFF00FFFF00FFFF00 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 3FFE007FFF003FFE0011197F9816>I<03E3C007F7E00FFFE01C1CC0380E00380E00380E 00380E00380E001C1C000FF8001FF0001BE0003800001800001FFC001FFF003FFF807803 C0E000E0E000E0E000E0E000E07001C07C07C03FFF800FFE0003F800131C7F9116>I<7E 0000FE00007E00000E00000E00000E00000E00000E3C000EFE000FFF000F87800F03800E 03800E03800E03800E03800E03800E03800E03800E03800E03800E03807FC7F0FFE7F87F C7F01519809816>I<018003C003C0018000000000000000007FC07FC07FC001C001C001 C001C001C001C001C001C001C001C001C001C07FFFFFFF7FFF101A7D9916>I<7E0000FE 00007E00000E00000E00000E00000E00000E7FE00E7FE00E7FE00E0F000E1E000E3C000E 78000EF0000FF0000FF8000FBC000F1E000E0E000E07000E07807F87F0FFCFF07F87F014 19809816>107 DII<7E3C00FEFE007FFF000F87800F03800E03800E03800E03800E03800E03 800E03800E03800E03800E03800E03807FC7F0FFE7F87FC7F01512809116>I<03E0000F F8001FFC003C1E00780F00700700E00380E00380E00380E00380E00380F0078070070078 0F003C1E001FFC000FF80003E00011127E9116>I<7E3E00FEFF007FFF800F83C00F00E0 0E00E00E00700E00700E00700E00700E00700E00700E00E00F01E00F83C00FFF800EFF00 0E3C000E00000E00000E00000E00000E00000E00007FC000FFE0007FC000141B809116> I<07C7000FE7001FF7003C1F00700F00700F00E00700E00700E00700E00700E00700E007 00700F00700F003C3F003FF7001FE70007C7000007000007000007000007000007000007 00003FE0007FF0003FE0141B7E9116>II<0FEC3FFC7FFCF03CE01CE01C70007F801FF007F8003C600EE00EF00EF81E FFFCFFF8C7E00F127D9116>I<0300000700000700000700000700007FFF00FFFF00FFFF 000700000700000700000700000700000700000700000701000703800703800703800787 0003FE0001FC0000F80011177F9616>I<7E1F80FE3F807E1F800E03800E03800E03800E 03800E03800E03800E03800E03800E03800E03800E03800E0F800FFFF007FBF803E3F015 12809116>I<7F1FC0FF1FE07F1FC01C07001E0F000E0E000E0E000E0E00071C00071C00 071C00071C0003B80003B80003B80001F00001F00000E00013127F9116>II<7F1FC07F3FC07F1FC00F1C00073C0003 B80003F00001F00000E00001E00001F00003B800073C00071C000E0E007F1FC0FF3FE07F 1FC013127F9116>I<7F1FC0FF9FE07F1FC01C07000E07000E0E000E0E00070E00071C00 071C00039C00039C0003980001B80001B80000F00000F00000F00000E00000E00000E000 01C00079C0007BC0007F80003F00003C0000131B7F9116>I<3FFFC07FFFC07FFFC07007 80700F00701E00003C0000780001F00003E0000780000F00001E01C03C01C07801C0FFFF C0FFFFC0FFFFC012127F9116>I E /Ff 45 122 df<00E001C0038007000E000E001C00 1C003800380038007000700070007000E000E000E000E000E000E000E000E000E000E000 E000E00070007000700070003800380038001C001C000E000E000700038001C000E00B2A 7E9E10>40 DI<001C0000003E0000003E00 00002E0000006700000067000000E7800000C7800000C3800001C3C0000183C0000181C0 000381E0000381E0000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C003C 001C003C0018003C0038001E0038001E0070001F0070000F0070000F00E0000780191D7F 9C1C>65 D<003FC000FFF003C0F00780300F00001E00003C00003C000078000078000078 0000F00000F00000F00000F00000F00000F00000F00000F00000F0000078000078000078 00003C00003C00001E00000F000807801803C07800FFF0003F80151F7D9D1B>67 DIII<003F8001FF F003C0F80780380F00181E00003C00003C0000780000780000780000F00000F00000F000 00F00000F00000F00000F007F8F007F8F000387800387800387800383C00383C00381E00 380F003807803803C0F801FFF0003F80151F7D9D1C>III76 DII<003F000001FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F00 780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F80007C078000780780007807C000F803C000F003E001F001F003E00 0F807C0007C0F80003FFF00001FFE000003F00001A1F7E9D1F>II82 D<03F8000FFE001C0F00380700700300600000E00000E00000E00000E00000F000007800 007F00003FE0001FFC0007FE0001FF00001F800007800003C00003C00001C00001C00001 C00001C0C00180E00380F007007C0E001FFC0007F000121F7E9D17>II86 DI<78000E00 7C001E003C003C001E0038000F0070000F00F0000781E00003C1C00001C3C00001E78000 00F70000007E0000003E0000003C0000003C0000007E00000077000000E7800001E38000 03C1C0000381E0000700F0000F00F8000E0078001C003C003C003E0078001F0070000F00 F0000F80191D7F9C1C>II<0FC03FF07FF87038401C 001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC3F1C0E127E9114>97 DI<07E00FF81FFC 3C1C70047000E000E000E000E000E000E000700070043C1C1FFC0FF807E00E127E9112> I<000E000E000E000E000E000E000E000E000E000E000E0F8E1FEE3FFE7C3E700E700EE0 0EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E0F1D7E9C15>I<07C01FE03FF0 78787018601CFFFCFFFCFFFCE000E000E000700070043C1C3FFC1FF807E00E127E9112> I<00FC01FC03FC07000E000E000E000E000E000E000E00FFE0FFE00E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E1D809C0D>I<03C3C00FFFC0 1FFFC01C3800381C00381C00381C00381C00381C001C38001FF8001FF0003BC000380000 3800001FFC001FFF003FFF80700780E001C0E001C0E001C0F003C07C0F803FFF001FFE00 07F800121B7F9115>III<03C003C003C003C0000000000000000000000000000001C001C001C001C001C001 C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C0C3C0FF 80FF003E000A25839C0B>I108 DII<03F0000FFC001FFE 003C0F00780780700380E001C0E001C0E001C0E001C0E001C0F003C07003807807803C0F 001FFE000FFC0003F00012127F9115>II<078E1FEE3FFE7C3E781E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE 1FEE0F8E000E000E000E000E000E000E000E000E0F1A7E9115>II<1FC0 3FF07FF0F030E000E000F0007F003FC01FE000F0003800388038F078FFF07FE01FC00D12 7F9110>I<1C001C001C001C001C001C00FFE0FFE01C001C001C001C001C001C001C001C 001C001C001C001C001C201FF00FF007C00C187F970F>IIII121 D E /Fg 68 124 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0 000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703 C00E03C00E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C07F87F8151D809C17>I<003F07E00001C09C18000380F018000701F03C000E01E03C00 0E00E018000E00E000000E00E000000E00E000000E00E000000E00E00000FFFFFFFC000E 00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00 E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E0 1C000E00E01C007FC7FCFF80211D809C23>14 D<60F0F0F0F0F0F0F06060606060606060 6060606060000000000060F0F060041E7C9D0C>33 D<6060F0F0F8F86868080808080808 101010102020404080800D0C7F9C15>I<60F0F8680808081010204080050C7C9C0C>39 D<004000800100020006000C000C0018001800300030007000600060006000E000E000E0 00E000E000E000E000E000E000E000E000E000600060006000700030003000180018000C 000C00060002000100008000400A2A7D9E10>I<800040002000100018000C000C000600 060003000300038001800180018001C001C001C001C001C001C001C001C001C001C001C0 01C0018001800180038003000300060006000C000C00180010002000400080000A2A7E9E 10>I<60F0F0701010101020204080040C7C830C>44 DI<60F0F0 6004047C830C>I<00010003000600060006000C000C000C001800180018003000300030 0060006000C000C000C0018001800180030003000300060006000C000C000C0018001800 1800300030003000600060006000C000C00010297E9E15>I<03C00C301818300C300C70 0E60066006E007E007E007E007E007E007E007E007E007E007E007E007E0076006600670 0E300C300C18180C3007E0101D7E9B15>I<030007003F00C70007000700070007000700 070007000700070007000700070007000700070007000700070007000700070007000F80 FFF80D1C7C9B15>I<07C01830201C400C400EF00FF80FF807F8077007000F000E000E00 1C001C00380070006000C00180030006010C01180110023FFE7FFEFFFE101C7E9B15>I< 07E01830201C201C781E780E781E381E001C001C00180030006007E00030001C001C000E 000F000F700FF80FF80FF80FF00E401C201C183007E0101D7E9B15>I<000C00000C0000 1C00003C00003C00005C0000DC00009C00011C00031C00021C00041C000C1C00081C0010 1C00301C00201C00401C00C01C00FFFFC0001C00001C00001C00001C00001C00001C0000 1C0001FFC0121C7F9B15>I<60F0F0600000000000000000000060F0F06004127C910C> 58 D<60F0F0600000000000000000000060F0F0701010101020204080041A7C910C>I<0F E03038401CE00EF00EF00EF00E000C001C0030006000C000800180010001000100010001 0001000000000000000000000003000780078003000F1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F00000017800000178000001780 000023C0000023C0000023C0000041E0000041E0000041E0000080F0000080F0000180F8 000100780001FFF80003007C0002003C0002003C0006003E0004001E0004001E000C001F 001E001F00FF80FFF01C1D7F9C1F>65 DI< 001F808000E0618001801980070007800E0003801C0003801C0001803800018078000080 7800008070000080F0000000F0000000F0000000F0000000F0000000F0000000F0000000 F0000000700000807800008078000080380000801C0001001C0001000E00020007000400 0180080000E03000001FC000191E7E9C1E>IIII<001F8080 00E0618001801980070007800E0003801C0003801C000180380001807800008078000080 70000080F0000000F0000000F0000000F0000000F0000000F0000000F000FFF0F0000F80 700007807800078078000780380007801C0007801C0007800E00078007000B8001801180 00E06080001F80001C1E7E9C21>III76 DII<003F800000E0E0000380380007001C000E000E001C 0007003C00078038000380780003C0780003C0700001C0F00001E0F00001E0F00001E0F0 0001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C0780003C0380003803C 0007801C0007000E000E0007001C000380380000E0E000003F80001B1E7E9C20>II82 D<07E0801C1980300580700380600180E00180E00080E00080E00080F00000F800007C00 007FC0003FF8001FFE0007FF0000FF80000F800007C00003C00001C08001C08001C08001 C0C00180C00180E00300D00200CC0C0083F800121E7E9C17>I<7FFFFFC0700F01C0600F 00C0400F0040400F0040C00F0020800F0020800F0020800F0020000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000001F800003FFFC001B1C7F9B1E> IIII<08081010202040404040808080808080B0B0F8F8787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C0001FC000F1C00381C00701C00601C 00E01C40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E0 0060007004300418080C3007C00E127E9112>I<003F0000070000070000070000070000 070000070000070000070000070000070003E7000C1700180F00300700700700600700E0 0700E00700E00700E00700E00700E00700600700700700300700180F000C370007C7E013 1D7E9C17>I<03E00C301818300C700E6006E006FFFEE000E000E000E000600070023002 18040C1803E00F127F9112>I<00F8018C071E061E0E0C0E000E000E000E000E000E00FF E00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E007FE00F 1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C00381C00381C00181800 1C38000C300013C0001000003000001800001FF8001FFF001FFF803003806001C0C000C0 C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C0018000000000000000000000000000000FC001C 001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80091D7F 9C0C>I<00C001E001E000C000000000000000000000000000000FE000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0 F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180E001C0E001C0E001C0E001C0 E001C0E001C06001807003803003001806000E1C0003F00012127F9115>II<03C1000C3300180B00300F00700700700700E00700E00700E00700E0 0700E00700E00700600700700700300F00180F000C370007C70000070000070000070000 0700000700000700000700003FE0131A7E9116>II<1F9030704030C010 C010E010F8007F803FE00FF000F880388018C018C018E010D0608FC00D127F9110>I<04 000400040004000C000C001C003C00FFE01C001C001C001C001C001C001C001C001C001C 101C101C101C101C100C100E2003C00C1A7F9910>III< FF3FCFE03C0F03801C0701801C0701001C0B01000E0B82000E0B82000E1182000711C400 0711C4000720C40003A0E80003A0E80003C0680001C0700001C070000180300000802000 1B127F911E>I<7F8FF00F03800F030007020003840001C80001D80000F0000070000078 0000F800009C00010E00020E000607000403801E07C0FF0FF81512809116>II<7FFC70386038407040F040E041C003C0038007000F040E041C043C 0C380870087038FFF80E127F9112>II E /Fh 8 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000F800 00F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115 >99 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F 001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018E018 F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Fi 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C060404060 C031801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C 000C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C001800300 06000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F0000 8000400060C060C060804060801F000B107F8F0F>I<0300030007000F000B0013003300 23004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C002000200020 002F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C0 6000C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07F C08080808001000200040004000C0008000800180018001800180018000B117E900F>I< 1F00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F8F 0F>I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B 107F8F0F>I E /Fj 80 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F87000000000 0070F8F8F870051C779B18>33 D<4010E038F078E038E038E038E038E038E038E038E038 E038E03860300D0E7B9C18>I<030600078F00078F00078F00078F00078F00078F007FFF C0FFFFE0FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFF E07FFFC01E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>I<01C00007 E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3800FC7000F 87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E0787070FC707F FFE03FCFE00F03C0141C7F9B18>38 D<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18> I<007000F001E003C007800F001E001C00380038007000700070007000E000E000E000E0 00E000E000E000E0007000700070007000380038001C001E000F00078003C001F000F000 700C24799F18>I<6000F00078003C001E000F000780038001C001C000E000E000E000E0 0070007000700070007000700070007000E000E000E000E001C001C0038007800F001E00 3C007800F00060000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF80 7FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C000 01C00011147D9718>I<00600000F00000F00000F00000F00000F00000F00000F0007FFF C0FFFFE0FFFFE07FFFC000F00000F00000F00000F00000F00000F00000F0000060001314 7E9718>I<1C3E7E7F3F1F070E1E7CF860080C788518>I<7FFF00FFFF80FFFF807FFF0011 047D8F18>I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00 001E00001E00003E00003C00007C0000780000780000F80000F00001F00001E00003E000 03C00003C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C0000 7C0000780000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F 001C07003803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000 E0E000E0E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F0 00131C7E9B18>I<01800380038007800F803F80FF80FB80438003800380038003800380 038003800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03 F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C000 01C00003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C 00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C07801C0 3001C00001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C00000E0 0000E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B18> I<001F00003F0000770000770000E70001E70001C7000387000787000707000E07001E07 003C0700380700780700F00700FFFFF8FFFFF8FFFFF80007000007000007000007000007 00000700007FF000FFF8007FF0151C7F9B18>I<1FFF803FFF803FFF8038000038000038 00003800003800003800003800003800003BF8003FFE003FFF003C07801803C00001C000 00E00000E06000E0F000E0F000E0E001C07003C07C0F803FFF001FFC0003F000131C7E9B 18>I<3078FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<0003 00000780001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC00 007E00003F00001FC00007E00003F00001FC00007E00003F00001F800007800003001118 7D9918>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FF FFE07FFFC0130C7E9318>I<600000F00000FC00007E00003F00001FC00007E00003F000 01FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F0000 7E0000FC0000F0000060000011187D9918>I<0FF0003FFC007FFF00700F00F00380F003 80600780000F00003E00007C0001F00001E00003C00003C00003C00003C00003C0000380 0000000000000000000000000000000003800007C00007C00007C000038000111C7D9B18 >I<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E0003 8E00038E000306000707000707000707000707000FFF800FFF800FFF800E03800E03801C 01C01C01C07F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E0700000E0 0000E00000E00000E00000E00000E00000E00000E000007000007000E07000E03800E03C 00E01E01C00F07C007FF8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F80 1C03C01C03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C00701C0070 1C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF800 141C7F9B18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001C0 700000E00000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001C07003C0 3803C03803C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07 F01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01FFF C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF8F F87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7FE000FFE000 7FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00700E00700E00700E00700E00707FFFF0 FFFFF07FFFF0141C7F9B18>76 DI<7E07F0 FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C0 1C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC0 7F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F00700700F00780E003 80E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF00780F00700700F00780 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00001C00001E00 000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF001C0F801C03801C03 C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F001C07001C03 801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18 >I<03F3801FFF803FFF807C0F80700780E00380E00380E00380E000007000007800003F 00001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E000E0E001E0F0 01C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038 E07038E07038007000007000007000007000007000007000007000007000007000007000 00700000700000700000700000700000700000700000700007FF0007FF0007FF00151C7F 9B18>IIII<7F8FE07F9FE07F8FE00E07 000F0700070E00078E00039C0003DC0001F80001F80000F00000F00000700000F00000F8 0001F80001DC00039E00038E00070F000707000E07800E03801E03C07F07F0FF8FF87F07 F0151C7F9B18>II91 D<600000F00000F00000F800007800007C00003C00003C00003E00001E00001F00000F00 000F00000F800007800007C00003C00003C00003E00001E00001F00000F00000F8000078 00007800007C00003C00003E00001E00001E00001F00000F00000F800007800007800003 0011247D9F18>II<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I<0038007C007C007C003800000000000000000FFC1FFC0FFC001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C6038F078FFF07FE03F800E277E9C18>II<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF800 1F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F0019148193 18>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0 000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001 E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807F FFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F 00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E 00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780 700780E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF80 0FFB8003E380000380000380000380000380000380000380000380003FF8003FF8003FF8 151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C0000380000380 000380000380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318 >I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE0000 1F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<018000 0380000380000380000380007FFFC0FFFFC0FFFFC0038000038000038000038000038000 0380000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E00 13197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318 >I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E0003 8E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8F F0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E 000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E 03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000 DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F80003F 00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00 003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF0 14147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000007800007F E0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003FE0003FE000 7FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00003C000FFC000FF00007C000013247E9F18>125 D E /Fk 53 122 df<007000E001C00380078007000E001E001E003C003C003C007800780078007800 7000F000F000F000F000F000F000F000F000F000F000F000F00070007800780078007800 3C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DII<000100 030003000600060006000C000C000C00180018001800300030003000600060006000C000 C000C00180018001800300030003000600060006000C000C000C00180018001800300030 003000600060006000C000C000C000102D7DA117>I<003E00003E00005E00005E0000DE 0001DE00019E00039E00039E00079E00071E000F1E000E1E001E1E003C1E003C1E00781E 00781E00F01E00FFFFF0FFFFF0FFFFF0001E00001E00001E00001E00001E00001E00001E 00001E00141E7F9D17>52 D<001F0000001F0000003F8000003B8000003B8000007BC000 0073C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0F00003C07800 03C078000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F00 1E000F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B207F9F 1E>65 D<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C00007800 00780000780000F00000F00000F00000F00000F00000F00000F00000F000007800007800 007800007C00003C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC01722 7DA01D>67 D III<001FE000FFF801FFFE 03E03E07800E0F00001E00003E00003C00007C0000780000780000780000F00000F00000 F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E 1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>III76 DII<003F000000FFC00003FFF00007E1F8000F807C001F003E001E001E00 3C000F003C000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F80007C0780007807800078078000780 3C000F003C000F001E001E001F003E000F807C0007E1F80003FFF00000FFC000003F0000 1A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C00 003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001 E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019 >IIIII<780007807C000F003E001F001E001E000F003C000F807C000780780003C0F0 0003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000003F00 00007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C000F003E 001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>II<07E03FF87FFC701E401F000F00 0F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III<01E001E001E001E00000000000 000000000000000000000001E001E001E001E001E001E001E001E001E001E001E001E001 E001E001E001E001E001E001E001E001E001E001E001E001E0C3C0FFC0FF803F000B2983 9F0C>IIIII<01F80007FE001FFF 803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000F07801E07801 E03C03C03F0FC01FFF8007FE0001F80014147F9317>II<03C7800FF7801FFF803E1F807C0F80780780780780F00780F00780 F00780F00780F00780F00780F807807807807C0F803E1F801FFF800FF78007C780000780 000780000780000780000780000780000780000780000780111D7E9317>II<07F01FFC3FFC780C7800780078007C003FC01FF00FF803F8007C003C003C C03CF07CFFF87FF00FC00E147F9311>I<1E001E001E001E001E001E00FFF0FFF0FFF01E 001E001E001E001E001E001E001E001E001E001E001E001E001E201FF00FF007C00C1A7F 9910>IIII<7801E07C03C03E07801E0F 000F0F00079E0003FC0003F80001F80000F00001F00001F80003FC00079E000F0F000E0F 001E07803C03C07801E0F801F01414809315>II E /Fl 36 125 df<0E1F3F3F1D0102020404081020C0080E779F0E>39 D<7FF0FFE07FE00C037D8A10>45 D<070F1F1F0E0000000000000000000070F8F8F0E008 147B930E>58 D<00000200000006000000060000000E0000001E0000001E0000003F0000 002F0000004F0000004F0000008F0000010F0000010F0000020F0000020F0000040F0000 0C0F0000080F0000100F0000100F0000200F80003FFF800040078000C007800080078001 000780010007800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<0000FE0200078186001C004C0038003C0060003C00C0001C01C0001803800018070000 180F0000181E0000101E0000103C0000003C000000780000007800000078000000780000 00F0000000F0000000F0000000F0000000F0000080700000807000008070000100380001 0038000200180004000C001800060020000381C00000FE00001F217A9F21>67 D<0000FC040007030C001C00980030007800E0007801C000380380003003800030070000 300E0000301E0000201E0000203C0000003C000000780000007800000078000000780000 00F0000000F000FFF0F0000780F0000780F0000F0070000F0070000F0070000F0070001E 0038001E0018003E001C002E000E00CC000383040000FC00001E217A9F23>71 D<01FE00007FC0001E0000FC00001E0000F80000170001780000170001780000270002F0 0000270004F00000270004F00000270008F00000470009E00000470011E00000470021E0 0000470021E00000870043C00000838043C00000838083C00000838083C0000103810780 000103820780000103820780000103840780000203840F00000203880F00000203900F00 000203900F00000401E01E00000401E01E00000401C01E00000C01801E00001C01803E00 00FF8103FFC0002A1F7D9E29>77 D<01FF007FE0001F000F00001F000400001780040000 1780040000278008000023C008000023C008000023C008000041E010000041E010000041 F010000040F010000080F0200000807820000080782000008078200001003C400001003C 400001003C400001001E400002001E800002001E800002000F800002000F800004000F00 00040007000004000700000C000700001C00020000FF80020000231F7D9E22>I<01FFFF 80001E00E0001E0070001E0038001E003C003C003C003C003C003C003C003C003C007800 7800780078007800F0007800E000F003C000F00F0000FFFC0000F0000001E0000001E000 0001E0000001E0000003C0000003C0000003C0000003C000000780000007800000078000 00078000000F800000FFF000001E1F7D9E1F>80 D<0FFFFFF01E0780E018078020100780 2020078020200F0020600F0020400F0020400F0020801E0040001E0000001E0000001E00 00003C0000003C0000003C0000003C00000078000000780000007800000078000000F000 0000F0000000F0000000F0000001E0000001E0000001E0000001E0000003E00000FFFF00 001C1F789E21>84 D<7FFC1FF807C003C00780010007800100078001000F0002000F0002 000F0002000F0002001E0004001E0004001E0004001E0004003C0008003C0008003C0008 003C00080078001000780010007800100078001000F0002000F0002000F0002000F00040 00F0004000700080007001000030020000380400000C18000007E000001D20779E22>I< FFF007FC0F8000E00F0000C00F0000800F0001000F0001000F0002000F0004000F000400 0F8008000780080007801000078020000780200007804000078040000780800007810000 0781000007C2000003C2000003C4000003C8000003C8000003D0000003D0000003E00000 03C0000003C000000380000001800000010000001E20779E22>I<00F1800389C0070780 0E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40 F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E 000E000E000E001C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE0 1EE01CE03CE038607060E031C01F0010207B9F15>I<007E0001C1000300800E07801E07 801C07003C0200780000780000780000F00000F00000F00000F00000F000007001007002 0030040018380007C00011147C9315>I<0000780003F800007000007000007000007000 00E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C03803C038038 0700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C4030 8C800F070015207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000 F000F0007000700170023804183807C010147C9315>I<00007800019C00033C00033C00 0718000700000700000E00000E00000E00000E00000E0001FFE0001C00001C00001C0000 1C0000380000380000380000380000380000700000700000700000700000700000700000 E00000E00000E00000E00000C00001C00001C0000180003180007B0000F300006600003C 00001629829F0E>I<003C6000E27001C1E00380E00700E00F00E00E01C01E01C01E01C0 1E01C03C03803C03803C03803C03803C07003C07001C0F001C17000C2E0003CE00000E00 000E00001C00001C00301C00783800F0700060E0003F8000141D7E9315>I<01E0000FE0 0001C00001C00001C00001C000038000038000038000038000070000070000071E000763 000E81800F01C00E01C00E01C01C03801C03801C03801C0380380700380700380700380E 10700E20700C20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C00000 0000000000000000000000000E003300230043804300470087000E000E000E001C001C00 1C003840388030807080310033001C000B1F7C9E0E>I<01E0000FE00001C00001C00001 C00001C0000380000380000380000380000700000700000703C00704200E08E00E11E00E 21E00E40C01C80001D00001E00001FC00038E00038700038700038384070708070708070 7080703100E03100601E0013207D9F15>107 D<03C01FC0038003800380038007000700 070007000E000E000E000E001C001C001C001C0038003800380038007000700070007100 E200E200E200E200640038000A207C9F0C>I<1C0F80F0002630C318004740640C004780 680E004700700E004700700E008E00E01C000E00E01C000E00E01C000E00E01C001C01C0 38001C01C038001C01C038001C01C0708038038071003803806100380380E10038038062 007007006600300300380021147C9325>I<1C0F802630C0474060478060470070470070 8E00E00E00E00E00E00E00E01C01C01C01C01C01C01C0384380388380308380708380310 7003303001C016147C931A>I<007C0001C3000301800E01C01E01C01C01E03C01E07801 E07801E07801E0F003C0F003C0F003C0F00780F00700700F00700E0030180018700007C0 0013147C9317>I<01C1E002621804741C04781C04701E04701E08E01E00E01E00E01E00 E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0072380071E0007000007 00000E00000E00000E00000E00001C00001C0000FFC000171D809317>I<00F0400388C0 0705800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E00 F01C00F01C00703C00705C0030B8000F3800003800003800007000007000007000007000 00E00000E0000FFE00121D7C9315>I<1C1E002661004783804787804707804703008E00 000E00000E00000E00001C00001C00001C00001C00003800003800003800003800007000 0030000011147C9313>I<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00800E00800E 00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C00012147C93 15>I<0E00C1C03300E3C02301C3E04381C1E04301C0E04701C060870380400E0380400E 0380400E0380401C0700801C0700801C0700801C0701001C0701001C0602001C0F02000C 0F04000E13080003E1F0001B147C931E>I<0E00C03300E02301C04381C04301C04701C0 8703800E03800E03800E03801C07001C07001C07001C07001C0E00180E00180E001C1E00 0C3C0007DC00001C00001C00003800F03800F07000E06000C0C0004380003E0000131D7C 9316>121 D124 D E /Fm 42 122 df<787878781830306060E0050A7C830E>44 D46 D<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFFFFFFFFFF10227C A118>49 D<03F0000FFC001FFE003C1F003007807007C06003C0E003E0C001E04001E040 01E00001E00001E00001E00003C00003C0000780000780000F00001E00003C0000780000 F00001E00001C0000380000700000E00001C0000380000700000FFFFE0FFFFE0FFFFE013 227EA118>I<01F00007FC001FFF003E0F003807807003C02003C02003C00003C00003C0 0003C0000780000780000F00001E0003FC0003F80003FE00000F000007800003C00003C0 0001E00001E00001E00001E00001E08001E0C003C0E003C07007803C0F801FFF000FFC00 03F00013237EA118>I<001F00001F00002F00002F00006F0000EF0000CF0001CF0001CF 00038F00038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F00780F00F00F 00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F00000F00000F 0015217FA018>I<001F0000001F0000003F8000003F8000003B8000007BC0000073C000 0071C00000F1E00000F1E00000E0E00001E0F00001E0F00001C0F00003C0780003C07800 0380780007803C0007803C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF00 1C000F003C0007803C00078038000780780003C0780003C0700003C0F00001E0F00001E0 E00001E01B237EA220>65 D<000FF000007FFC0000FFFF0001F01F0003C0070007800000 0F0000001E0000003E0000003C0000003C000000780000007800000078000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000078000000 78000000780000003C0000003C0000003E0000001E0000000F0000000780008003C00380 01F00F8000FFFF00007FFC00000FF00019257DA31F>67 DII 73 D77 D<001FC000007FF00001FFFC0003F07E0007C01F000F800F801F0007C01E 0003C03C0001E03C0001E0780000F0780000F0780000F070000070F0000078F0000078F0 000078F0000078F0000078F0000078F0000078F0000078F0000078780000F0780000F078 0000F07C0001F03C0001E03E0003E01E0003C01F0007C00F800F8007C01F0003F07E0001 FFFC00007FF000001FC0001D257DA324>79 DI82 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800007800007800007800007800 007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F00000F00000 F8000078000078000078000078000078000078C000F0E000F0F801E07E07C03FFF800FFF 0001FC0015257EA31B>III<07E01FF83FFC381E201E000F000F00 0F000F00FF07FF1FFF3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180780000780000700000F00000F00000F0 0000F00000F00000F000007800007800007800003C00401F03C00FFFC007FF8001FC0012 167E9516>I<0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C003E3C00FFBC01FFFC03F0FC03C07C07803C07803C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C07803C07803C03C07C03E0FC01FFFC00FFBC0 03E3C012237EA219>I<03F00007FC001FFE003E0F003C0780780380780380F001C0FFFF C0FFFFC0FFFFC0F00000F00000F000007000007800007800003C00801F07800FFF8007FF 0001F80012167E9516>I<003F00FF01FF03C00380078007800780078007800780078007 80FFF8FFF8FFF80780078007800780078007800780078007800780078007800780078007 800780078007800780102380A20F>I<01F07807FFF80FFFF81F1F001E0F003C07803C07 803C07803C07803C07801E0F001F1F000FFE001FFC0019F0003800003800003C00001FFE 001FFFC01FFFE03FFFF07801F07800F8F00078F00078F00078F000787800F03E03E01FFF C00FFF8001FC0015217F9518>III<00F000F000F000F0000000 00000000000000000000000000000000F000F000F000F000F000F000F000F000F000F000 F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F080F0E3 E0FFE07FC01F000C2D83A20D>I108 DII<01FC0007FF000FFF801F07C0 3C01E07800F07800F0700070F00078F00078F00078F00078F00078F000787800F07800F0 7C01F03E03E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07C07C 03C07803C0F803C0F003C0F003C0F003C0F003C0F003C0F003C0F803C07803C07803C03C 07C03F0FC01FFFC00FFBC003E3C00003C00003C00003C00003C00003C00003C00003C000 03C00003C00003C012207E9519>II<07F01FFC3FFE 3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE3FFC 0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12>I< F00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF01F F83F7FFF7FCF1F0F10167C9519>III<7801F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F800007000 00F80001FC0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F8151680 9516>II E /Fn 27 90 df<3078F8787005057C840D>46 D<000C001C00FC0F3800380038003800 38003800700070007000700070007000E000E000E000E000E000E001C001C001C001C001 C001C0038003C0FFFE0F1E7C9D17>49 D<003F8000C1E00100F00200780400780400780F 007C0F807C0F807C0F00780600780000F80000F00001E00001C0000380000700000E0000 1C0000380000600000C0000180000300200600200800401000403FFFC07FFF80FFFF8016 1E7E9D17>I<007F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E0 0003C0000380000700000E0000F800000E000007000007800007C00003C00007C03007C0 7807C0F807C0F807C0F00780800F00400E00201C0018780007E000141F7D9D17>I<0000 600000600000E00001C00003C00005C0000DC00009C00011C000238000438000C3800083 80010380020380040700080700180700100700200700400700FFFFF0000E00000E00000E 00000E00000E00001C00001E0001FFE0141E7E9D17>I<01803001FFE003FFC003FF0003 FC00020000020000020000040000040000040000047C000587000603800C01800801C000 01C00001E00001E00001E00001E07003C0F803C0F003C0E00380800780400700400E0020 1C0018700007C000141F7D9D17>I<000010000000180000003800000038000000780000 0078000000FC000001BC0000013C0000033C0000023C0000063C0000043E0000081E0000 081E0000101E0000101E0000201E0000200F0000400F0000400F0000FFFF0000800F0001 000F8001000780020007800200078004000780040007800C0007C03E0007C0FF807FFC1E 207E9F22>65 D<07FFFF00007C01C0003C01E0003C00F0007800F8007800F8007800F800 7800F8007800F8007800F000F001F000F001E000F003C000F00F8000FFFE0000F00F0001 E007C001E003C001E003E001E001E001E001E001E001E003C001E003C003E003C003E003 C003C003C007C003C00F8007800F0007803E00FFFFF0001D1F7E9E20>I<0001F808000E 061800380138007000F801E0007803C0007007800030078000300F0000301F0000301E00 00303E0000203C0000007C0000007C0000007C0000007C000000F8000000F8000000F800 0000F8000000F80000007800004078000080780000803C0000803C0001001C0002000E00 020006000C000300100001C0E000003F00001D217B9F21>I<07FFFF00007C01E0003C00 F0003C00780078003C0078003C0078001E0078001E0078001E0078001F00F0001F00F000 1F00F0001F00F0001F00F0001F00F0001F01E0001E01E0003E01E0003E01E0003E01E000 3C01E0007C03C0007803C000F003C000F003C001E003C003C003C0078007800F0007803C 00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038003C00180078001800780008 0078000800780008007800080078080800F0100000F0100000F0100000F0300000FFF000 00F0700001E0200001E0200001E0200001E0200001E0000801E0001003C0001003C00010 03C0002003C0002003C0006003C000C0078001C0078007C0FFFFFF801D1F7E9E1F>I<07 FFFFF8007C0078003C0038003C0018007800180078000800780008007800080078000800 78000800F0100000F0100000F0100000F0300000F0700000FFF00001E0600001E0200001 E0200001E0200001E0200001E0000003C0000003C0000003C0000003C0000003C0000003 C000000780000007C00000FFFE00001D1F7E9E1E>I<0001FC04000F030C003C009C0070 007C00E0003C01C0003803800018078000180F0000181F0000181E0000183E0000103C00 00007C0000007C0000007C0000007C000000F8000000F8000000F8007FFCF80003E07800 01E0780001E0780003C0780003C03C0003C03C0003C01C0003C00E0007C007000B800380 118001E06080003F80001E217B9F24>I<07FFC7FFC0007C00F800003C007800003C0078 00007800F000007800F000007800F000007800F000007800F000007800F00000F001E000 00F001E00000F001E00000F001E00000FFFFE00000F001E00001E003C00001E003C00001 E003C00001E003C00001E003C00001E003C00003C007800003C007800003C007800003C0 07800003C007800003C007800007800F000007C00F8000FFF8FFF800221F7E9E22>I<07 FFE0007C00003C00003C0000780000780000780000780000780000780000F00000F00000 F00000F00000F00000F00001E00001E00001E00001E00001E00001E00003C00003C00003 C00003C00003C00003C00007800007C000FFFC00131F7F9E10>I<00FFFC0007C00003C0 0003C0000780000780000780000780000780000780000F00000F00000F00000F00000F00 000F00001E00001E00001E00001E00001E00001E00003C00303C00783C00F83C00F83800 F0780080700040E00021C0001F000016207D9E17>I<07FFF000007E0000003C0000003C 000000780000007800000078000000780000007800000078000000F0000000F0000000F0 000000F0000000F0000000F0000001E0000001E0000001E0000001E0000001E0008001E0 010003C0010003C0010003C0030003C0020003C0060003C0060007801E0007807C00FFFF FC00191F7E9E1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800004E 0002F000004E0002F000004E0004F000004E0004F000004E0008F000004E0008F0000087 0011E00000870011E00000870021E00000870021E00000870041E00000838041E0000103 8083C00001038083C00001038103C00001038203C0000101C203C0000101C403C0000201 C40780000201C80780000201C80780000201D00780000200F00780000600E00780000600 E00F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E00 1800003E001800004F001000004F001000004780100000478010000043C010000043C010 000083C020000081E020000081E020000080F020000080F0200000807820000100784000 01007C400001003C400001003C400001001E400001001E400002000F800002000F800002 000F800002000780000200078000060003800006000300000F00010000FFE0010000221F 7E9E22>I<0003F800001E0E000038070000E0038001C001C003C001E0078000E00F0000 F00F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000 F8F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C0007 803C000F003C001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23>I< 07FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F8007800F8007800F8 007800F000F001F000F001E000F003C000F0078000F00F0000FFF80001E0000001E00000 01E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C00000 03C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C000 3C01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000 F0078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001 E01E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FF FC0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C070018060008 0E00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E0 0001E00000E00000F00000F04000E04000E04000E04000E06000C0600180E00380F80300 C60C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F00 10C00F0010800F0010800F0010800F0010001E0000001E0000001E0000001E0000001E00 00001E0000003C0000003C0000003C0000003C0000003C0000003C000000780000007800 000078000000780000007800000078000000F0000001F800007FFFE0001C1F7A9E21>I< FFF07FF81FF01F000FC007C01F00078003000F00078003000F00078002000F000F800200 0F000F8004000F001F8004000F0017C008000F8013C00800078023C01000078023C01000 078043C02000078043C02000078083C04000078083E0400007C101E0800003C101E08000 03C201E1000003C201E1000003C401E2000003C401E2000003C801F4000003E800F40000 01F000FC000001F000F8000001E000F8000001E000F0000001C000F0000001C000600000 00C0006000000080004000002C207A9E2F>87 D<03FFC0FFC0007F007E00003E00380000 1E003000001E002000000F004000000F008000000F81000000078200000007C600000003 C400000003E800000001F000000001F000000000F000000000F800000000F8000000017C 000000023C000000043C0000000C1E000000081E000000101F000000200F000000400F80 0000C0078000008007C000010003C000070003E0001F8007E000FFE01FFE00221F7F9E22 >II E /Fo 1 16 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF8 0FF003C010127D9317>15 D E /Fp 37 122 df46 D<00180000380000F80007F800FFF800FFF800F8F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F8007FFFF07FFFF07FFFF014287BA71E>49 D<00FE0003FFC007FFE00FFFF01F03F8 3C00FC38007E78003E70003EF0001FF0001F60001F20001F00001F00001F00001F00003E 00003E00007C00007C0000F80001F00001E00003C0000780000F00001E00003C00007800 00F00001E00003C0000780000F00001E00003C00007FFFFF7FFFFF7FFFFF7FFFFF18287E A71E>I<007F000001FFC00007FFF0000FFFF8001FC1F8003E007C003C003E0078003E00 38003E0010003E0000003E0000003E0000003C0000007C000000FC000001F8000007F000 00FFE00000FFC00000FFE00000FFF0000001FC0000007C0000003E0000001F0000001F00 00000F8000000F8000000F8000000F8000000F8040000F8060001F00F0001F00F8003F00 7E007E003F81FC001FFFF8000FFFF00003FFE000007F000019297EA71E>I<0003F00000 07F0000005F000000DF000000DF000001DF0000039F0000039F0000079F0000079F00000 F1F00000F1F00001E1F00003E1F00003E1F00007C1F00007C1F0000F81F0000F81F0001F 01F0001F01F0003E01F0007C01F0007C01F000F801F000FFFFFF80FFFFFF80FFFFFF80FF FFFF800001F0000001F0000001F0000001F0000001F0000001F0000001F0000001F00000 01F0000001F00019277EA61E>I<3FFFFC3FFFFC3FFFFC3FFFFC3E00003E00003E00003E 00003E00003E00003E00003E00003E00003E00003E3F003EFFC03FFFE03FFFF03FE1F83F 807C3F003E3E003E00003E00001F00001F00001F00001F00001F00001F00001F20001F60 003E70003EF8007C7C00FC3F03F81FFFF00FFFE007FF8000FE0018287EA61E>I68 DI73 D77 DI<0001FC0000000FFF8000003FFFE000007FFFF00001FE03 FC0003F800FE0007E0003F0007C0001F000F80000F801F000007C01F000007C03E000003 E03E000003E07C000001F07C000001F07C000001F078000000F0F8000000F8F8000000F8 F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8 000000F87C000001F07C000001F07C000001F07E000003F03E000003E03F000007E01F00 0007C01F80000FC00FC0001F8007E0003F0007F0007F0003F800FE0001FE03FC0000FFFF F800003FFFE000000FFF80000001FC0000252C7DAA2C>II<007FC00001FFF80007FFFE000FFFFF001F C07F003F000F007E0006007C0000007C000000F8000000F8000000F8000000F8000000F8 000000FC0000007E0000007F0000003F8000001FF800000FFF800007FFE00003FFF80000 FFFC00000FFE000000FF0000003F0000001F8000000F8000000FC0000007C0000007C000 0007C0000007C0000007C0000007C000000F8060000F80F0001F00FC003F00FF80FE007F FFFC001FFFF80007FFE00000FF80001A2C7DAA21>83 DI<01FE000FFF803FFFC03FFFE0 3C03F03001F00001F80000F80000F80000F80000F80000F8007FF807FFF81FFFF83FE0F8 7F00F8FC00F8F800F8F800F8F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E 9A1D>97 DI<007FC001FFF0 07FFFC0FFFFC1FC07C1F00083E00007C00007C00007C0000F80000F80000F80000F80000 F80000F80000F800007C00007C00007E00003E00001F000C1FC07C0FFFFC07FFFC01FFF0 007F80161B7E9A1B>I<00003E00003E00003E00003E00003E00003E00003E00003E0000 3E00003E00003E00003E00003E00003E00003E00FC3E03FF3E07FFFE0FFFFE1FC1FE3F00 7E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003EF8003EF8003EFC00 3E7C003E7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03FF3E00FC3E172A7EA91F>I<00 7E0003FF8007FFC00FFFE01F83F03F00F03E00787C00787C003878003CFFFFFCFFFFFCFF FFFCFFFFFCF80000F80000F800007800007C00007C00003E00003F000C1FC07C0FFFFC07 FFFC01FFF0007F80161B7E9A1B>I<001FC0007FC000FFC001FFC003F00003E00007C000 07C00007C00007C00007C00007C00007C00007C00007C000FFFE00FFFE00FFFE0007C000 07C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C000 07C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C000122A7F A912>I<00F8078003FE7FC00FFFFFC01FFFFFC01F07C0003E03E0003E03E0007C01F000 7C01F0007C01F0007C01F0007C01F0007C01F0003E03E0003E03E0001F07C0001FFFC000 3FFF80003BFE000038F8000078000000780000003C0000003FFFC0003FFFF8001FFFFC00 1FFFFE003FFFFF007C007F00F8001F80F8000F80F8000F80F8000F80FC001F807E003F00 3F80FE003FFFFE000FFFF80007FFF00000FF80001A287E9A1E>III<007C007C007C00 7C007C0000000000000000000000000000000000000000007C007C007C007C007C007C00 7C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C00 7C007C007C007C007C007C007C007C007C40FCF1F8FFF8FFF07FE00F800E3683A910>I< F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8 F8F8F8F8F8F8052A7CA90E>108 DII<007F000001FFC0 0007FFF0000FFFF8001FC1FC003F007E003E003E007C001F007C001F0078000F00F8000F 80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F807C001F007C001F007E003F 003E003E003F007E001FC1FC000FFFF80007FFF00001FFC000007F0000191B7E9A1E>I< F83F00F9FFC0FBFFE0FFFFF0FF07F0FC01F8F800FCF800FCF8007CF8007EF8003EF8003E F8003EF8003EF8003EF8003EF8003EF8007CF8007CF800FCFC00F8FC03F8FF07F0FFFFE0 FBFFC0F9FF80F87E00F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F8000017277B9A1F>I114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00 007F80007FF8003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007 E0FC0FC0FFFFC07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007 C00007C00007C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C04007E1C003FFE003FFE001FF8000FC0013227FA116>II119 D<7C000FC03E001F803F001F001F803E000F807C0007C0FC0003E0F80001F1F000 01FBE00000FFC000007FC000003F8000001F0000001F0000003F8000007FC00000FBC000 00F3E00001F1F00003E0F80007C07C000F807C000F803E001F001F003E000F807E000FC0 FC0007E01B1B809A1C>II E /Fq 18 122 df66 D68 D80 D<0007FFFC000000007FFFFFC0000001FFFFFFF8000003FFFFFFFE000007FE001F FF000007FF0003FFC0000FFF8001FFE0000FFF8000FFF0000FFF80007FF0000FFF80007F F8000FFF80007FF80007FF00003FFC0007FF00003FFC0003FE00003FFC0000F800003FFC 00000000003FFC00000000003FFC00000000003FFC00000000003FFC00000007FFFFFC00 0000FFFFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC0003FFF0003FFC0007 FFC0003FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC007FF800003FFC007FF8 00003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000 003FFC00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC0003EFFC003FFF0007 CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFFF001FFFC0003FF8000 7FF8362E7DAD3A>97 D<007FC00000000000FFFFC00000000000FFFFC00000000000FFFF C00000000000FFFFC00000000000FFFFC0000000000003FFC0000000000001FFC0000000 000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FF C0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000 000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FF C0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000 000001FFC00FFC00000001FFC07FFFC0000001FFC3FFFFF0000001FFCFFFFFFC000001FF DFF00FFF000001FFFF8003FF800001FFFE0001FFC00001FFF800007FE00001FFF000007F F00001FFE000003FF80001FFE000001FFC0001FFE000001FFC0001FFE000001FFE0001FF E000001FFE0001FFE000000FFF0001FFE000000FFF0001FFE000000FFF0001FFE000000F FF0001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FF E000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000F FF8001FFE000000FFF8001FFE000000FFF0001FFE000000FFF0001FFE000000FFF0001FF E000001FFE0001FFE000001FFE0001FFE000001FFC0001FFE000001FFC0001FFE000003F F80001FFF000003FF80001FFF800007FF00001FFFC0000FFE00001FFFE0001FFC00001FF BF0007FF800001FF1FE01FFE000001FE0FFFFFFC000001FC03FFFFF0000001F800FFFF80 000001F0001FF800000039487CC742>I<00001FFFC0000000FFFFF8000007FFFFFE0000 1FFFFFFF80007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FFE007FF0003FFE00F FE0003FFE00FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF803FFC00003E007F F8000000007FF8000000007FF800000000FFF800000000FFF800000000FFF800000000FF F800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FF F8000000007FF8000000007FF8000000007FFC000000003FFC000000003FFC000000001F FC000000F81FFE000000F80FFE000000F80FFF000001F007FF800003F003FFC00007E001 FFE0000FC000FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8000000FFFFE00000 001FFE00002D2E7CAD35>I<00001FFE00000001FFFFE0000007FFFFF800001FFFFFFE00 007FFC07FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00003FF00FFE00001FF0 1FFE00000FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC000007FC7FF8000007FC 7FF8000007FE7FF8000007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFFFFFFFFFFFE FFFFFFFFFFFEFFFFFFFFFFFCFFF800000000FFF800000000FFF800000000FFF800000000 7FF8000000007FF8000000007FFC000000003FFC000000003FFC000000003FFC0000001C 1FFE0000003E0FFE0000003E07FF0000007E07FF000000FC03FF800001F801FFC00003F0 007FF0001FE0003FFE00FFC0001FFFFFFF800007FFFFFE000000FFFFF80000000FFF8000 2F2E7DAD36>101 D<00000000001F8000007FF000FFE00007FFFF03FFF0001FFFFFC7FF F0007FFFFFFFC7F800FFE03FFE0FF801FF800FFC0FF803FF0007FE0FF807FE0003FF07F0 07FE0003FF07F00FFC0001FF81C00FFC0001FF80000FFC0001FF80001FFC0001FFC0001F FC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0000FFC 0001FF80000FFC0001FF80000FFC0001FF800007FE0003FF000007FE0003FF000003FF00 07FE000001FF800FFC000000FFE03FF8000001FFFFFFF0000001DFFFFFC0000003C7FFFF 00000003C07FF000000007C0000000000007C0000000000007C0000000000007C0000000 000007E0000000000007F0000000000007F8000000000007FFFFFFF0000007FFFFFFFF00 0003FFFFFFFFE00003FFFFFFFFF80001FFFFFFFFFE0001FFFFFFFFFF0000FFFFFFFFFF80 007FFFFFFFFF8003FFFFFFFFFFC00FFFFFFFFFFFC01FF800001FFFE03FE0000001FFE07F C00000007FF07FC00000003FF0FF800000001FF0FF800000001FF0FF800000001FF0FF80 0000001FF0FF800000001FF07FC00000003FE07FC00000003FE03FE00000007FC03FF000 0000FFC01FFC000003FF800FFF00000FFF0003FFF000FFFC0000FFFFFFFFF000003FFFFF FFC0000007FFFFFE000000003FFFC0000035447DAE3B>103 D<007FC00000000000FFFF C00000000000FFFFC00000000000FFFFC00000000000FFFFC00000000000FFFFC0000000 000003FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FF C0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000 000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FF C0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000 000001FFC0000000000001FFC0000000000001FFC001FFC0000001FFC00FFFF8000001FF C03FFFFE000001FFC0FFFFFF000001FFC1FC07FF800001FFC3E003FFC00001FFC7C001FF C00001FFCF0001FFE00001FFDE0000FFE00001FFDC0000FFE00001FFFC0000FFF00001FF F80000FFF00001FFF00000FFF00001FFF00000FFF00001FFF00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF000FFFFFFC07FFFFFE0FFFFFFC07FFF FFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B487CC742>I<00FC00 01FE0003FF0007FF800FFFC01FFFE01FFFE01FFFE01FFFE01FFFE01FFFE00FFFC007FF80 03FF0001FE0000FC00000000000000000000000000000000000000000000000000000000 000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001FFC001FFC001FFC0 01FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0 01FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0 01FFC001FFC001FFC001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF 18497CC820>I<007FC000FFFFC000FFFFC000FFFFC000FFFFC000FFFFC00003FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001 FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FFFFFF80FFFFFF80FFFFFF80FF FFFF80FFFFFF8019487CC720>108 D<007FC001FFC00000FFE00000FFFFC00FFFF80007 FFFC0000FFFFC03FFFFE001FFFFF0000FFFFC0FFFFFF007FFFFF8000FFFFC1FC07FF80FE 03FFC000FFFFC3E003FFC1F001FFE00003FFC7C001FFC3E000FFE00001FFCF0001FFE780 00FFF00001FFDE0000FFEF00007FF00001FFDC0000FFEE00007FF00001FFFC0000FFFE00 007FF80001FFF80000FFFC00007FF80001FFF00000FFF800007FF80001FFF00000FFF800 007FF80001FFF00000FFF800007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000 007FF800FFFFFFC07FFFFFE03FFFFFF0FFFFFFC07FFFFFE03FFFFFF0FFFFFFC07FFFFFE0 3FFFFFF0FFFFFFC07FFFFFE03FFFFFF0FFFFFFC07FFFFFE03FFFFFF05C2E7CAD63>I<00 7FC001FFC00000FFFFC00FFFF80000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1FC07 FF8000FFFFC3E003FFC00003FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE00001 FFDC0000FFE00001FFFC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF00000 FFF00001FFF00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001 FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000 FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001 FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000 FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001 FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000 FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FF FFFFC07FFFFFE03B2E7CAD42>I<00000FFF0000000000FFFFF000000007FFFFFE000000 1FFFFFFF8000003FFC03FFC00000FFE0007FF00001FF80001FF80003FF00000FFC0007FE 000007FE000FFE000007FF000FFC000003FF001FFC000003FF803FFC000003FFC03FF800 0001FFC03FF8000001FFC07FF8000001FFE07FF8000001FFE07FF8000001FFE0FFF80000 01FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001 FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF07FF8000001FF E07FF8000001FFE07FF8000001FFE07FF8000001FFE03FFC000003FFC03FFC000003FFC0 1FFC000003FF801FFE000007FF800FFE000007FF0007FF00000FFE0003FF80001FFC0001 FFC0003FF80000FFE0007FF000007FFC03FFE000001FFFFFFF80000007FFFFFE00000000 FFFFF0000000000FFF000000342E7DAD3B>I<007FC00FFC000000FFFFC07FFFC00000FF FFC3FFFFF00000FFFFCFFFFFFC0000FFFFDFF01FFF0000FFFFFF8007FF800003FFFE0001 FFC00001FFF80000FFE00001FFF00000FFF00001FFE000007FF80001FFE000003FFC0001 FFE000003FFC0001FFE000003FFE0001FFE000001FFE0001FFE000001FFF0001FFE00000 1FFF0001FFE000001FFF0001FFE000000FFF0001FFE000000FFF8001FFE000000FFF8001 FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE00000 0FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF0001 FFE000001FFF0001FFE000001FFF0001FFE000001FFE0001FFE000001FFE0001FFE00000 3FFC0001FFE000003FFC0001FFE000007FF80001FFF000007FF80001FFF80000FFF00001 FFFC0001FFE00001FFFE0003FFC00001FFFF0007FF800001FFFFE03FFE000001FFEFFFFF FC000001FFE3FFFFF0000001FFE0FFFF80000001FFE01FF800000001FFE0000000000001 FFE0000000000001FFE0000000000001FFE0000000000001FFE0000000000001FFE00000 00000001FFE0000000000001FFE0000000000001FFE0000000000001FFE0000000000001 FFE0000000000001FFE0000000000001FFE0000000000001FFE0000000000001FFE00000 000000FFFFFFC000000000FFFFFFC000000000FFFFFFC000000000FFFFFFC000000000FF FFFFC00000000039427CAD42>I<00FF803F8000FFFF80FFF000FFFF83FFFC00FFFF87FF FE00FFFF8FC3FF00FFFF8F07FF0003FF9E0FFF8001FFBC0FFF8001FFB80FFF8001FFF80F FF8001FFF00FFF8001FFF007FF0001FFF007FF0001FFE003FE0001FFE000F80001FFE000 000001FFE000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC0000000FFFFFFE00000FFFFFFE00000FFFFFFE00000FFFFFFE00000FFFFFFE0 0000292E7CAD31>114 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FFE0 1FC0001FE03F80000FE03F000007E07F000003E07F000003E0FF000003E0FF000003E0FF 800003E0FFC0000000FFF0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFF FFFC001FFFFFFF000FFFFFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FF FFF800001FFFF8000000FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003 FCFC000003FCFE000003F8FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0 FFFC00FFC0FFFFFFFF80FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>I<7FFFFFC0 00FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF80 00FFF000000FE00000FFF800000FC00000FFF800000FC000007FFC00000F8000007FFC00 001F8000003FFC00001F0000003FFE00003F0000001FFE00003E0000001FFF00007E0000 000FFF00007C0000000FFF8000FC00000007FF8000F800000007FFC001F800000003FFC0 01F000000003FFE003F000000003FFE003F000000001FFF003E000000001FFF007E00000 0000FFF007C000000000FFF80FC0000000007FF80F80000000007FFC1F80000000003FFC 1F00000000003FFE3F00000000001FFE3E00000000001FFF7E00000000000FFF7C000000 00000FFFFC00000000000FFFFC000000000007FFF8000000000007FFF8000000000003FF F0000000000003FFF0000000000001FFE0000000000001FFE0000000000000FFC0000000 000000FFC00000000000007F800000000000007F800000000000003F000000000000003F 000000000000003F000000000000003E000000000000007E000000000000007C00000000 000000FC000000001F8000F8000000003FC001F8000000007FE001F000000000FFF003F0 00000000FFF003E000000000FFF007E000000000FFF00FC000000000FFF01F8000000000 FFF03F80000000007FE07F00000000007F43FE00000000003FFFF800000000001FFFF000 0000000007FFC0000000000001FE00000000000039427EAD3F>121 D E /Fr 8 117 df<00001E000000003E00000000FE00000003FE0000003FFE0000FFFF FE0000FFFFFE0000FFFFFE0000FFCFFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE00 00000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000 000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE000000 0FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000F FE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE00007FFFFFFFC07FFFFFFFC07FFFFFFFC07FFFFFFF C0223879B731>49 D<0000001FFF000030000001FFFFE000F000000FFFFFFC01F000007F FFFFFE03F00001FFFE007F87F00003FFE0000FCFF0000FFF000003FFF0001FFC000001FF F0003FF80000007FF0007FF00000003FF000FFC00000003FF001FFC00000001FF003FF80 0000000FF007FF000000000FF00FFF0000000007F00FFE0000000007F01FFE0000000003 F01FFE0000000003F03FFC0000000003F03FFC0000000001F03FFC0000000001F07FFC00 00000001F07FF80000000001F07FF80000000000007FF8000000000000FFF80000000000 00FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800 0000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF80000000000 00FFF80000000000007FF80000000000007FF80000000000007FF80000000000007FFC00 00000000F03FFC0000000000F03FFC0000000000F03FFC0000000000F01FFE0000000000 F01FFE0000000001E00FFE0000000001E00FFF0000000001E007FF0000000003C003FF80 00000003C001FFC0000000078000FFE00000000F00007FF00000001F00003FF80000003E 00001FFC0000007C00000FFF000001F8000003FFE00007F0000001FFFE003FC00000007F FFFFFF000000000FFFFFFC0000000001FFFFF000000000001FFF0000003C3D7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF00001FFC00FF80 001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF00007F0003FF0 0001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000000FFFFFF0 00007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF0003FE0003FF0 007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000FFC0003FF0 00FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF007FFFE0FFF F001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFFFE00 00FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03FF00007F83F F00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFFFFFFFCFFFF FFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE00000007FE000 00007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF800007807FC0000 F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE00026267DA52D >101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF0000000007FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF007FC00003FF 01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF6000FF8003FF E000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC3FFFFFFFFF FC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00FFFFFE03FF 00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF803FF00001F FC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF03FF000007 FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007 FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC03FF00000F FC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC003FFF001FF 8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000FFFFFC0000 00FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fs 82 124 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C000C00180018003000300030007000600060006000E0 00E000E000E000E000E000E000E000E000E000E000E00060006000600070003000300030 00180018000C000C000600020001000080004000200B2E7DA112>I<8000400020001000 08000C00060006000300030001800180018001C000C000C000C000E000E000E000E000E0 00E000E000E000E000E000E000E000C000C000C001C00180018001800300030006000600 0C00080010002000400080000B2E7DA112>I<01800180018001800180C183F18F399C0F F003C003C00FF0399CF18FC1830180018001800180018010147DA117>I<70F8FCFC7404 0404080810102040060E7C840D>44 DI<70F8F8F87005057C84 0D>I<000100030003000600060006000C000C000C001800180018003000300030006000 60006000C000C000C00180018001800300030003000600060006000C000C000C00180018 001800300030003000600060006000C000C000C000102D7DA117>I<03F0000E1C001C0E 00180600380700700380700380700380700380F003C0F003C0F003C0F003C0F003C0F003 C0F003C0F003C0F003C0F003C0F003C0F003C0F003C07003807003807003807807803807 001806001C0E000E1C0003F000121F7E9D17>I<018003800F80F3800380038003800380 038003800380038003800380038003800380038003800380038003800380038003800380 0380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0 F803C0F803C0F803C02007C00007C0000780000780000F00000E00001C00003800007000 00600000C0000180000300000600400C00401800401000803FFF807FFF80FFFF80121E7E 9D17>I<03F0000C1C00100E00200F00780F80780780780780380F80000F80000F00000F 00000E00001C0000380003F000003C00000E00000F000007800007800007C02007C0F807 C0F807C0F807C0F00780400780400F00200E001C3C0003F000121F7E9D17>I<00060000 0600000E00000E00001E00002E00002E00004E00008E00008E00010E00020E00020E0004 0E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E0000 0E00000E00000E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE000 10000010000010000010000010000010000011F000161C00180E00100700100780000380 0003800003C00003C00003C07003C0F003C0F003C0E00380400380400700200600100E00 0C380003E000121F7E9D17>I<007C000182000701000E03800C07801C07803803003800 00780000700000700000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003 C0F003C0F003C07003C07003C07003803803803807001807000C0E00061C0001F000121F 7E9D17>I<4000007FFFC07FFF807FFF8040010080020080020080040000080000080000 100000200000200000400000400000C00000C00001C00001800003800003800003800003 8000078000078000078000078000078000078000078000030000121F7D9D17>I<03F000 0C0C001006003003002001806001806001806001807001807803003E03003F06001FC800 0FF00003F80007FC000C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0 C000806001802001001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C003806 00380700700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007 C03807C0180BC00E13C003E3C0000380000380000380000700300700780600780E00700C 002018001070000FC000121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F8 7005147C930D>I<70F8F8F8700000000000000000000070F0F8F8780808081010102020 40051D7C930D>I<7FFFFFE0FFFFFFF00000000000000000000000000000000000000000 000000000000000000000000FFFFFFF07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000C00180018001000300020002 0002000200020002000000000000000000000007000F800F800F8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C0000007C0000009E0000009E0 000009E0000010F0000010F0000010F00000207800002078000020780000403C0000403C 0000403C0000801E0000801E0000FFFE0001000F0001000F0001000F0002000780020007 8002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0 038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C00004078000040 F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 780000007C0000407C0000403C0000401C0000401E0000800E0000800700010003800200 01C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0070001E0 0F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E07C0001E0 3C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E0046000781820 000FE0001E217D9F24>III<0FFFC0007C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00203C00F83C00F83C 00F83C00F0380040780040700030E0000F800012207E9E17>IIIII<001F800000F0F00001C0380007801E000F000F000E0007001E00 07803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F80001F0F800 01F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C0003E03C00 03C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F000001F80001C21 7D9F23>II<001F800000F0F000 01C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E07C0003E0 780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0 F80001F0780001E0780001E07C0003E03C0003C03C0F03C01E1087800E2047000F204F00 07A03E0001E0380000F0F010001FB01000003010000038300000387000003FF000001FE0 00001FE000000FC0000007801C297D9F23>II<07E0800C1980100780300380600180600180E00180E00080E000 80E00080F00000F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003 C00003C00001C08001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F8 0012217D9F19>I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010 800F0010800F0010800F0010000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF8 0FE00FC007C0070003C0020001E0040001F00C0000F0080000781000007C1000003C2000 003E4000001E4000000F8000000F8000000780000003C0000007E0000005E0000009F000 0018F8000010780000207C0000603C0000401E0000801F0001800F0001000780020007C0 070003C01F8007E0FFE01FFE1F1F7F9E22>II91 D<08041008201020104020402080 4080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E001E0E0038 0E00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>II E /Ft 46 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000071C000C060018030038 03803803807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C038 03803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038007C0FFFE0F217CA018>I<03F8000C1E0010070020 07804007C07807C07803C07807C03807C0000780000780000700000F00000E0000380003 F000001C00000F000007800007800003C00003C00003E02003E07003E0F803E0F803E0F0 03C04003C0400780200780100F000C1C0003F00013227EA018>51 D<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>I<10 00801E07001FFF001FFE001FF80013E00010000010000010000010000010000010000010 F800130E001407001803801003800001C00001C00001E00001E00001E00001E07001E0F0 01E0F001E0E001C08001C04003C04003802007001006000C1C0003F00013227EA018>I< 007E0001C1000300800601C00E03C01C03C0180180380000380000780000700000700000 F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013227EA018> I<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000 E0C00060C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018 >56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C0 0001C00001C0000380000380300300780700780600700C002018001030000FC00013227E A018>I<0001800000018000000180000003C0000003C0000003C0000005E0000005E000 000DF0000008F0000008F0000010F800001078000010780000203C0000203C0000203C00 00401E0000401E0000401E0000800F0000800F0000FFFF000100078001000780030007C0 020003C0020003C0040003E0040001E0040001E00C0000F00C0000F03E0001F8FF800FFF 20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E000070 1E0000701C0000303C0000303C0000307C0000107800001078000010F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F800000078000000780000107C000010 3C0000103C0000101C0000201E0000200E000040070000400380008001C0010000E00200 00381C000007E0001C247DA223>67 D69 DI73 D<03FFF0001F00000F00000F00000F00000F 00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F 00000F00000F00000F00000F00000F00000F00000F00000F00700F00F80F00F80F00F80E 00F01E00401C0020380018700007C00014237EA119>I77 DI80 D82 D<03F0200C0C601802603001E070 00E0600060E00060E00060E00020E00020E00020F00000F000007800007F00003FF0001F FE000FFF0003FF80003FC00007E00001E00000F00000F000007080007080007080007080 0070C00060C00060E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF87807 807860078018400780084007800840078008C007800C8007800480078004800780048007 800400078000000780000007800000078000000780000007800000078000000780000007 800000078000000780000007800000078000000780000007800000078000000780000007 8000000780000007800000078000000FC00003FFFF001E227EA123>I<0FE0001838003C 0C003C0E0018070000070000070000070000FF0007C7001E07003C0700780700700700F0 0708F00708F00708F00F087817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E1F000E61C00E80600F00300E00380E003C0E001C0E001E0E001E0E00 1E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C80600C41C0083F001723 7FA21B>I<01FE000703000C07801C0780380300780000700000F00000F00000F00000F0 0000F00000F00000F000007000007800403800401C00800C010007060001F80012157E94 16>I<0000E0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E001F8E00704E00C02E01C01E03800E07800E07000E0F000E0F000E0 F000E0F000E0F000E0F000E0F000E07000E07800E03800E01801E00C02E0070CF001F0FE 17237EA21B>I<01FC000707000C03801C01C03801C07801E07000E0F000E0FFFFE0F000 00F00000F00000F00000F000007000007800203800201C00400E008007030000FC001315 7F9416>I<003C00C6018F038F030F070007000700070007000700070007000700FFF807 000700070007000700070007000700070007000700070007000700070007000700070007 807FF8102380A20F>I<00007001F198071E180E0E181C07001C07003C07803C07803C07 803C07801C07001C07000E0E000F1C0019F0001000001000001800001800001FFE000FFF C00FFFE03800F0600030400018C00018C00018C000186000306000303800E00E038003FE 0015217F9518>I<0E0000FE00001E00000E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E1F800E60C00E80E00F00700F00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E 0070FFE7FF18237FA21B>I<1C001E003E001E001C000000000000000000000000000000 00000E00FE001E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E00FFC00A227FA10E>I<01C003E003E003E001C00000000000000000000000 000000000001E00FE001E000E000E000E000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E060E0F0C0F18061803E000B2C82A10F >I<0E0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E00000E 00000E00000E00000E03FC0E01F00E01C00E01800E02000E04000E08000E10000E38000E F8000F1C000E1E000E0E000E07000E07800E03C00E01C00E01E00E00F00E00F8FFE3FE17 237FA21A>I<0E00FE001E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E00FFE00B237FA20E>I<0E1FC07F00FE60E183801E807201C00F003C00E00F003C00E0 0E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E0FFE3FF8FFE27157F942A>I<0E1F80FE60C01E80E00F00700F00700E00700E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E0070FFE7FF18157F941B>I<01FC000707000C01801800C03800E0700070700070F000 78F00078F00078F00078F00078F00078F000787000707800F03800E01C01C00E03800707 0001FC0015157F9418>I<0E1F00FE61C00E80600F00700E00380E003C0E001C0E001E0E 001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E00380F00700E80E00E41C00E 3F000E00000E00000E00000E00000E00000E00000E00000E00000E0000FFE000171F7F94 1B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E000E000E000E000E000E 000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F80 3FE00FF001F8003C801C800C800CC00CC008E018D0308FC00E157E9413>I<0200020002 0002000600060006000E001E003E00FFF80E000E000E000E000E000E000E000E000E000E 000E000E040E040E040E040E040E040708030801F00E1F7F9E13>I<0E0070FE07F01E00 F00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00F00E00F006017003827800FC7F18157F941B>III121 D E /Fu 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E078 0000E0700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F8 0000007C0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000 FFFF00000FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F800 0000F88000007880000078800000788000007880000078C0000078C0000070E00000F0E0 0000E0F00000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125 >83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E 001E0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F 001E003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478 003E047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E060038018 0700040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000 F80000F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F0002 07000403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E0 000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0 000001E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E0 078005E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0 F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E0 780001E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF 003F81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C 000F003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F8 000000F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E 0000800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007 E0001C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF 007FFF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E 003C001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E 003C000F007800070070000780E00009C1C000087F000018000000180000001800000018 000000180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E030 0000F070000070E0000038E0000038E0000038E0000038E0000038700000707000007038 0000E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F80 0700000000000000000000000000000000000000000000000780FF80FF800F8007800780 078007800780078007800780078007800780078007800780078007800780078007800780 07800780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F0 00FF8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E00 07C000F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E000780 00F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fv 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 908 a Fv(D)26 b(R)g(A)f(F)h(T)225 999 y Fu(Do)r(cumen)n(t) 20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1194 y Ft(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)833 1320 y(July)h(13,)g(1995)77 1378 y(This)h(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1436 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1494 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1552 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 45 a Fs(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fr(Chapter)34 b(1)75 564 y Fq(Dynamic)40 b(Pro)s(cesses)75 805 y Fp(1.1)59 b(Intro)r(duction)75 906 y Fs(The)15 b(MPI-1)g(message)g(passing)g(library)h(allo)o(ws)f (pro)q(cesses)h(in)g(a)f(parallel)h(program)e(to)h(comm)o(unicate)75 963 y(with)21 b(one)f(another.)34 b(MPI-1)20 b(sp)q(eci\014es)i (neither)g(ho)o(w)d(the)i(pro)q(cesses)f(are)g(created,)h(nor)f(ho)o(w) g(they)75 1019 y(establish)f(comm)o(unication.)30 b(Moreo)o(v)o(er,)17 b(an)i(MPI-1)f(application)i(is)e(static,)h(that)e(is,)i(no)g(pro)q (cesses)75 1076 y(can)c(b)q(e)h(added)g(to)f(or)f(deleted)j(from)d(an)h (application)i(while)g(it)e(is)h(running.)166 1132 y(MPI)22 b(users)g(ha)o(v)o(e)g(ask)o(ed)f(that)h(the)g(MPI-1)g(mo)q(del)h(b)q (e)f(relaxed)h(to)f(allo)o(w)g(dynamic)h(pro)q(cess)75 1188 y(managemen)o(t.)17 b(The)11 b(main)f(imp)q(etus)i(comes)e(from)f (the)h(PVM)g([1])f(researc)o(h)h(e\013ort,)g(whic)o(h)h(has)f(pro)o (vided)75 1245 y(a)16 b(w)o(ealth)h(of)f(exp)q(erience)j(with)e (dynamic)h(pro)q(cess)f(managemen)o(t)f(that)g(illustrates)h(its)g(b)q (ene\014ts)h(and)75 1301 y(p)q(oten)o(tial)e(pitfalls.)22 b(The)15 b(reasons)g(for)g(adding)h(dynamic)g(pro)q(cess)g(managemen)o (t)e(are)h(b)q(oth)g(tec)o(hnical)75 1358 y(and)g(practical.)143 1464 y Fo(\017)23 b Fs(W)l(orkstation)d(net)o(w)o(ork)f(users)i (migrating)g(from)g(PVM)f(to)g(MPI)h(are)g(accustomed)g(to)f(using)189 1521 y(PVM's)14 b(capabilities)j(for)e(pro)q(cess)g(and)g(resource)h (managemen)o(t.)j(While)d(relativ)o(ely)g(few)f(PVM)189 1577 y(applications)d(are)e(truly)h(dynamic)h(or)e(require)h(features)g (not)f(in)h(MPI,)g(the)f(lac)o(k)h(of)f(these)h(features)189 1633 y(is)k(a)g(practical)h(stum)o(bling)g(blo)q(c)o(k)g(to)f (migration.)143 1727 y Fo(\017)23 b Fs(Imp)q(ortan)o(t)16 b(classes)h(of)g(message)f(passing)h(applications,)i(suc)o(h)e(as)f (clien)o(t-serv)o(er)i(systems)f(and)189 1784 y(task)d(farming)h(jobs,) g(require)h(dynamic)g(pro)q(cess)f(con)o(trol.)143 1878 y Fo(\017)23 b Fs(With)16 b(dynamic)h(resource)g(and)f(pro)q(cess)h (managemen)o(t)e(extensions,)i(it)g(w)o(ould)g(b)q(e)g(p)q(ossible)h (to)189 1934 y(write)d(ma)s(jor)f(parts)g(of)h(the)g(parallel)i (programming)e(en)o(vironmen)o(t)g(in)h(MPI)f(itself.)143 2028 y Fo(\017)23 b Fs(The)14 b(abilit)o(y)i(to)d(write)h(fault)h (toleran)o(t)e(applications)j(is)f(imp)q(ortan)o(t)f(in)h(unstable)g (en)o(vironmen)o(ts)189 2084 y(and)23 b(for)f(commercial)i (applications.)45 b(MPI-1)23 b(do)q(es)g(not)f(pro)o(vide)i(mec)o (hanisms)f(for)g(build-)189 2141 y(ing)18 b(fault-toleran)o(t)f (applications.)30 b(The)18 b(mec)o(hanisms)g(required)h(to)e(supp)q (ort)h(fault)g(tolerance)189 2197 y(largely)d(o)o(v)o(erlap)g(with)h (those)f(needed)h(to)f(supp)q(ort)g(dynamic)h(pro)q(cess)g(managemen)o (t.)166 2303 y(While)e(dynamic)g(pro)q(cess)f(managemen)o(t)f(is)i (essen)o(tial,)g(it)f(is)g(imp)q(ortan)o(t)g(not)f(to)h(compromise)g (the)75 2360 y(p)q(ortabilit)o(y)j(or)f(p)q(erformance)g(of)g(MPI.)g (In)h(particular:)143 2454 y Fo(\017)23 b Fs(The)f(MPI-2)f(dynamic)i (pro)q(cess)e(managemen)o(t)g(mo)q(del)i(m)o(ust)e(apply)h(to)f(the)h (v)m(ast)f(ma)s(jorit)o(y)189 2510 y(of)e(curren)o(t)h(parallel)h(en)o (vironmen)o(ts.)34 b(These)20 b(include)j(ev)o(erything)d(from)f(tigh)o (tly)h(in)o(tegrated)189 2567 y(MPPs)d(suc)o(h)i(as)f(the)g(In)o(tel)h (P)o(aragon)e(and)h(the)g(Meik)o(o)g(CS-2)h(to)e(heterogeneous)h(net)o (w)o(orks)f(of)189 2623 y(w)o(orkstations.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fs(2)951 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)143 45 y Fo(\017)23 b Fs(MPI)16 b(m)o(ust)g(not)g(tak)o (e)g(o)o(v)o(er)f(op)q(erating)i(system)f(resp)q(onsibiliti)q(es.)26 b(It)17 b(should)g(instead)g(pro)o(vide)189 102 y(a)e(clean)h(in)o (terface)f(b)q(et)o(w)o(een)h(an)f(application)i(and)e(system)g(soft)o (w)o(are.)143 195 y Fo(\017)23 b Fs(MPI)17 b(m)o(ust)g(con)o(tin)o(ue)h (to)f(guaran)o(tee)g(comm)o(unication)h(determinism,)h(i.e.,)e(dynamic) i(pro)q(cess)189 252 y(managemen)o(t)14 b(m)o(ust)h(not)f(in)o(tro)q (duce)j(una)o(v)o(oidable)f(race)f(conditions.)143 346 y Fo(\017)23 b Fs(MPI)15 b(m)o(ust)f(not)h(con)o(tain)h(features)e (that)h(compromise)g(p)q(erformance.)143 439 y Fo(\017)23 b Fs(MPI-1)16 b(programs)f(m)o(ust)h(w)o(ork)g(under)h(MPI-2,)f(i.e.,)g (the)h(MPI-1)f(static)g(pro)q(cess)h(mo)q(del)h(m)o(ust)189 496 y(b)q(e)e(a)e(sp)q(ecial)j(case)f(of)e(the)i(MPI-2)f(dynamic)h(mo)q (del.)166 590 y(The)e(MPI)g(dynamic)h(pro)q(cess)g(managemen)o(t)e(mo)q (del)i(address)f(these)h(issues)g(in)f(t)o(w)o(o)f(w)o(a)o(ys.)19 b(First,)75 646 y(it)13 b(separates)f(the)g(run)o(time)h(en)o(vironmen) o(t)g(of)f(a)h(parallel)h(program)d(in)o(to)i(three)f(logical)i(parts:) k(resource)75 703 y(managemen)o(t,)c(pro)q(cess)i(managemen)o(t,)e(and) h(comm)o(unication.)21 b(MPI-1)15 b(addresses)h(the)f(comm)o(unica-)75 759 y(tion)20 b(comp)q(onen)o(t.)33 b(MPI-2)20 b(pro)o(vides)g(an)g(in) o(terface)g(b)q(et)o(w)o(een)g(an)f(MPI)h(application)h(and)f (logically)75 816 y(external)c(resource)f(and)g(pro)q(cess)g(managers.) k(The)d(resource)f(and)g(pro)q(cess)g(managers)g(ma)o(y)f(b)q(e)i (sepa-)75 872 y(rate)g(programs,)f(the)i(same)g(program,)e(part)h(of)g (the)h(op)q(erating)g(system,)f(or)g(ev)o(en)h(con)o(tained)h(within)75 928 y(the)i(MPI)f(implemen)o(tation)i(itself,)h(but)d(their)h (functionalit)o(y)h(is)f(w)o(ell-de\014ned)j(and)c(exists)h(in)h(most) 75 985 y(curren)o(t)15 b(message)g(passing)g(en)o(vironmen)o(ts,)g (includin)q(g)i(PVM.)166 1041 y(Second,)g(MPI-2)g(do)q(es)g(not)f(c)o (hange)h(the)g(concept)g(of)f(comm)o(unicator.)24 b(Once)18 b(a)e(comm)o(unicator)75 1098 y(is)h(built,)g(it)g(b)q(eha)o(v)o(es)f (as)g(sp)q(eci\014ed)j(in)e(MPI-1.)23 b(A)16 b(comm)o(unicator)g(is)h (nev)o(er)f(c)o(hanged)h(once)f(created,)75 1154 y(and)f(it)h(is)g(alw) o(a)o(ys)e(created)h(using)h(deterministic)h(collectiv)o(e)g(seman)o (tics.)75 1297 y Fp(1.2)59 b(The)19 b(MPI)h(Dynamic)f(Pro)r(cess)g(Mo)r (del)75 1401 y Fm(1.2.1)49 b(Comp)q(onents)75 1486 y Fs(The)13 b(MPI)g(dynamic)g(pro)q(cess)g(mo)q(del)h(separates)e(the)h (functions)g(of)g Fl(r)n(esour)n(c)n(e)g(manager)p Fs(,)g Fl(pr)n(o)n(c)n(ess)g(man-)75 1543 y(ager)i Fs(and)h Fl(message-p)n(assing)e(libr)n(ary)p Fs(.)75 1663 y Fk(Resource)i (Manager)44 b Fs(The)15 b Fl(r)n(esour)n(c)n(e)g(manager)20 b Fs(is)15 b(the)f(part)g(of)g(the)h(system)f(that)g(con)o(trols)g (resources)75 1719 y(and)i(allo)q(cates)g(them)g(to)f(an)h (application.)23 b(It)16 b(decides)h(when)f(a)g(job)f(will)j(run)e(and) g(whic)o(h)g(pro)q(cessors)75 1776 y(will)21 b(b)q(e)f(allo)q(cated)g (to)f(it)g(when)h(it)g(do)q(es)f(run.)33 b(In)20 b(some)f(en)o (vironmen)o(ts)g(the)h(resource)f(manager)f(is)75 1832 y(a)f(sophisticated)h(batc)o(h)f(queueing)h(system;)f(in)h(others)f(it) g(is)g(the)h(user)f(him/herself,)h(who)f(can)g(start)75 1889 y(jobs)f(on)g(a)f(net)o(w)o(ork)g(whenev)o(er)i(and)f(wherev)o(er) g(he/she)g(lik)o(es.)23 b(Logically)l(,)18 b(the)e(resource)g(manager)f (is)75 1945 y(external)h(to)e(an)h(application)i(ev)o(en)f(if)f(it)h (is)g(implemen)o(ted)g(in)o(ternally)l(.)166 2002 y(In)g(con)o(trast,)f (the)h(PVM)f(mo)q(del)i(of)e(a)h(resource)f(manager)g(is)i(in)o(ternal) f(to)g(an)f(application,)j(ev)o(en)75 2058 y(though)g(it)g(ma)o(y)g(b)q (e)h(implemen)o(ted)g(externally)h(through)d(the)h(hoster)g(and)g(task) o(er)g(in)o(terfaces.)28 b(This)75 2115 y(mak)o(es)15 b(it)g(di\016cult)i(for)d(PVM)h(applications)i(to)d(in)o(teract)i(w)o (ell)g(with)f(external)h(resource)f(managers.)75 2235 y Fk(Pro)q(cess)d(Manager)45 b Fs(Once)12 b(pro)q(cessors)f(ha)o(v)o(e) f(b)q(een)i(allo)q(cated)g(to)f(a)g(program,)f(user)h(pro)q(cesses)h(m) o(ust)e(b)q(e)75 2291 y(started)g(on)g(those)g(pro)q(cessors,)h(and)f (managed)h(after)e(startup.)18 b(By)10 b(\\managed")g(w)o(e)g(mean)g (that)g(signals)75 2348 y(m)o(ust)15 b(b)q(e)h(deliv)o(erable,)h(that)e Fj(stdin)p Fs(,)f Fj(stdout)p Fs(,)g(and)h Fj(stderr)g Fs(m)o(ust)g(b)q(e)h(handled)g(in)h(some)e(reasonable)75 2404 y(w)o(a)o(y)l(,)j(and)g(that)g(orderly)h(termination)f(can)h(b)q (e)g(guaran)o(teed.)29 b(A)18 b(minimal)i(example)f(is)g Fj(rsh)p Fs(,)f(whic)o(h)75 2460 y(starts)11 b(pro)q(cesses)i(and)g (reroutes)f Fj(stdin)p Fs(,)g Fj(stdout)p Fs(,)g(and)h Fj(stderr)e Fs(bac)o(k)i(to)f(the)g(originating)i(pro)q(cess.)19 b(A)75 2517 y(more)14 b(complex)i(example)g(is)f(giv)o(en)g(b)o(y)g Fj(poe)f Fs(on)h(the)g(IBM)g(SP2)g(or)f Fj(prun)g Fs(on)h(the)g(Meik)o (o)g(CS-2,)f(whic)o(h)75 2573 y(start)j(pro)q(cesses)i(on)f(pro)q (cessors)g(giv)o(en)h(to)f(them)g(b)o(y)g(the)g(job)h(sc)o(heduler)g (and)g(manage)f(them)g(un)o(til)75 2630 y(they)g(are)f(\014nished.)28 b(In)19 b(a)e(tigh)o(tly)h(in)o(tegrated)f(parallel)i(computer,)f(pro)q (cess)g(managemen)o(t)e(ma)o(y)h(b)q(e)75 2686 y(done)f(en)o(tirely)g (b)o(y)f(the)g(op)q(erating)h(system.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fn(1.2.)34 b(THE)15 b(MPI)g(D)o(YNAMIC)g(PR)o(OCESS)h (MODEL)813 b Fs(3)166 45 y(In)21 b(some)f(cases)g(the)h(situation)g(is) g(m)o(uddied)h(b)q(ecause)f(a)f(single)i(piece)g(of)e(soft)o(w)o(are)e (com)o(bines)75 102 y(the)i(functions)g(of)f(resource)g(manager)g(and)h (pro)q(cess)f(manager.)32 b(Examples)20 b(of)f(this)h(approac)o(h)f (are)75 158 y(the)i(batc)o(h)f(queueing)j(systems)d(suc)o(h)h(as)g (Condor,)g(DQS,)g(and)g(LoadLev)o(eler.)38 b(Nonetheless,)22 b(it)f(is)75 214 y(con)o(v)o(enien)o(t)f(to)e(consider)i(resource)f (and)g(pro)q(cess)g(managemen)o(t)g(separately)l(,)h(since)g(although)f (they)75 271 y(in)o(teract,)c(they)g(are)g(separate)g(functions)g(that) g(can)g(b)q(e)h(indep)q(enden)o(tly)i(mo)q(di\014ed.)75 391 y Fk(Message-P)o(assing)g(Lib)o(ra)o(ry)44 b Fs(By)17 b(the)g Fl(message-p)n(assing)f(libr)n(ary)k Fs(w)o(e)d(mean)f(the)h (library)h(used)f(b)o(y)g(the)75 447 y(application)f(program)e(for)g (its)h(in)o(terpro)q(cess)h(comm)o(unication.)k(Programs)14 b(con)o(taining)h(only)h(calls)g(to)75 504 y(a)i(message-passing)h (library)g(can)f(b)q(e)h(extremely)g(p)q(ortable,)g(since)h(they)e (\014t)g(cleanly)i(in)o(to)f(a)f(v)m(ariet)o(y)75 560 y(of)13 b(job)h(sc)o(heduler{pro)q(cess)h(manager)e(en)o(vironmen)o (ts.)20 b(MPI-1)14 b(de\014nes)g(a)g(standard)f(message)h(passing)75 617 y(library)l(.)75 739 y Fm(1.2.2)49 b(Interaction)16 b(of)h(User,)d(Application)k(and)f(Runtime)f(Environment)375 1576 y @beginspecial 71 @llx 464 @lly 494 @urx 728 @ury 2880 @rwi @setspecial %%BeginDocument: fig1.eps 1 setlinejoin /M { moveto } bind def /S { show } bind def /R { rmoveto } bind def /L { lineto } bind def /B { newpath 0 0 M 0 1 L 1 1 L 1 0 L closepath } bind def /CS { closepath stroke } bind def /S { /fixwidth exch def dup length /nchars exch def dup stringwidth pop fixwidth exch sub nchars div exch 0 exch ashow } def /bwproc { rgbproc dup length 3 idiv string 0 3 0 5 -1 roll { add 2 1 roll 1 sub dup 0 eq { pop 3 idiv 3 -1 roll dup 4 -1 roll dup 3 1 roll 5 -1 roll put 1 add 3 0 } { 2 1 roll } ifelse } forall pop pop pop } def systemdict /colorimage known not { /colorimage { pop pop /rgbproc exch def { bwproc } image } def } if 1 1 scale 0 setlinewidth /drawtri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def 0 setgray newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath stroke } bind def /filltri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath fill } bind def /cliptri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def 0 setgray newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath clip } bind def /imgscanrgb { gsave translate /scandy exch def /scandx exch def /istr scandx 3 mul string def scandx scandy scale scandx scandy 8 [scandx 0 0 scandy neg 0 scandy] {currentfile istr readhexstring pop} false 3 colorimage grestore } bind def /imgscanbw { gsave translate /scandy exch def /scandx exch def /istr scandx string def scandx scandy scale scandx scandy 8 [scandx 0 0 scandy neg 0 scandy] {currentfile istr readhexstring pop} image grestore } bind def /showcaseisoencoding [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /space /exclam /quotedbl /numbersign /dollar /percent /ampersand /quoteright /parenleft /parenright /asterisk /plus /comma /minus /period /slash /zero /one /two /three /four /five /six /seven /eight /nine /colon /semicolon /less /equal /greater /question /at /A /B /C /D /E /F /G /H /I /J /K /L /M /N /O /P /Q /R /S /T /U /V /W /X /Y /Z /bracketleft /backslash /bracketright /asciicircum /underscore /quoteleft /a /b /c /d /e /f /g /h /i /j /k /l /m /n /o /p /q /r /s /t /u /v /w /x /y /z /braceleft /bar /braceright /asciitilde /guilsinglright /fraction /florin /quotesingle /quotedblleft /guilsinglleft /fi /fl /endash /dagger /daggerdbl /bullet /quotesinglbase /quotedblbase /quotedblright /ellipsis /trademark /dotlessi /grave /acute /circumflex /tilde /macron /breve /dotaccent /dieresis /perthousand /ring /cedilla /Ydieresis /hungarumlaut /ogonek /caron /emdash /exclamdown /cent /sterling /currency /yen /brokenbar /section /dieresis /copyright /ordfeminine /guillemotleft /logicalnot /hyphen /registered /macron /degree /plusminus /twosuperior /threesuperior /acute /mu /paragraph /periodcentered /cedilla /onesuperior /ordmasculine /guillemotright /onequarter /onehalf /threequarters /questiondown /Agrave /Aacute /Acircumflex /Atilde /Adieresis /Aring /AE /Ccedilla /Egrave /Eacute /Ecircumflex /Edieresis /Igrave /Iacute /Icircumflex /Idieresis /Eth /Ntilde /Ograve /Oacute /Ocircumflex /Otilde /Odieresis /multiply /Oslash /Ugrave /Uacute /Ucircumflex /Udieresis /Yacute /Thorn /germandbls /agrave /aacute /acircumflex /atilde /adieresis /aring /ae /ccedilla /egrave /eacute /ecircumflex /edieresis /igrave /iacute /icircumflex /idieresis /eth /ntilde /ograve /oacute /ocircumflex /otilde /odieresis /divide /oslash /ugrave /uacute /ucircumflex /udieresis /yacute /thorn /ydieresis ] def /showcasedingbatencoding [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /a1 /a2 /a202 /a3 /a4 /a5 /a119 /a118 /a117 /a11 /a12 /a13 /a14 /a15 /a16 /a105 /a17 /a18 /a19 /a20 /a21 /a22 /a23 /a24 /a25 /a26 /a27 /a28 /a6 /a7 /a8 /a9 /a10 /a29 /a30 /a31 /a32 /a33 /a34 /a35 /a36 /a37 /a38 /a39 /a40 /a41 /a42 /a43 /a44 /a45 /a46 /a47 /a48 /a49 /a50 /a51 /a52 /a53 /a54 /a55 /a56 /a57 /a58 /a59 /a60 /a61 /a62 /a63 /a64 /a65 /a66 /a67 /a68 /a69 /a70 /a71 /a72 /a73 /a74 /a203 /a75 /a204 /a76 /a77 /a78 /a79 /a81 /a82 /a83 /a84 /a97 /a98 /a99 /a100 /.notdef /a205 /a85 /a206 /a86 /a87 /a88 /a89 /a90 /a91 /a92 /a93 /a94 /a95 /a96 /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /a101 /a102 /a103 /a104 /a106 /a107 /a108 /a112 /a111 /a110 /a109 /a120 /a121 /a122 /a123 /a124 /a125 /a126 /a127 /a128 /a129 /a130 /a131 /a132 /a133 /a134 /a135 /a136 /a137 /a138 /a139 /a140 /a141 /a142 /a143 /a144 /a145 /a146 /a147 /a148 /a149 /a150 /a151 /a152 /a153 /a154 /a155 /a156 /a157 /a158 /a159 /a160 /a161 /a163 /a164 /a196 /a165 /a192 /a166 /a167 /a168 /a169 /a170 /a171 /a172 /a173 /a162 /a174 /a175 /a176 /a177 /a178 /a179 /a193 /a180 /a199 /a181 /a200 /a182 /.notdef /a201 /a183 /a184 /a197 /a185 /a194 /a198 /a186 /a195 /a187 /a188 /a189 /a190 /a191 /.notdef ] def /Courier-Bold findfont dup length dict begin {1 index /FID ne {def} {pop pop} ifelse} forall /Encoding showcaseisoencoding def currentdict end /Courier-Bold-SHOWISO exch definefont pop /newfont 10 dict def newfont begin /FontType 3 def /FontMatrix [1 0 0 1 0 0] def /FontBBox [0 0 1 1] def /Encoding 256 array def 0 1 255 {Encoding exch /.notdef put} for /CharProcs 1 dict def CharProcs begin /.notdef {} def end /BuildChar { 1 0 0 0 1 1 setcachedevice exch begin Encoding exch get CharProcs exch get end exec } def end /PatternFont newfont definefont pop /#copies 1 def gsave gsave 0 0 0 setrgbcolor 1.000000 setlinewidth newpath 214 599.5 M 208.75 600.25 L 205 602.5 L 202.75 606.25 L 202 611.5 L 202 611.5 L 202 612.373 L 202 614.496 L 202 617.692 L 202 621.781 L 202 626.585 L 202 631.926 L 202 637.624 L 202 643.5 L 202 649.376 L 202 655.074 L 202 660.415 L 202 665.219 L 202 669.308 L 202 672.504 L 202 674.627 L 202 675.5 L 202 675.5 L 202.75 680.75 L 205 684.5 L 208.75 686.75 L 214 687.5 L 214 687.5 L 215.535 687.5 L 219.527 687.5 L 225.624 687.5 L 233.469 687.5 L 242.708 687.5 L 252.988 687.5 L 263.954 687.5 L 275.25 687.5 L 286.523 687.5 L 297.418 687.5 L 307.581 687.5 L 316.656 687.5 L 324.291 687.5 L 330.129 687.5 L 333.817 687.5 L 335 687.5 L 335 687.5 L 340.25 686.75 L 344 684.5 L 346.25 680.75 L 347 675.5 L 347 675.5 L 347 674.627 L 347 672.504 L 347 669.308 L 347 665.219 L 347 660.415 L 347 655.074 L 347 649.376 L 347 643.5 L 347 637.624 L 347 631.926 L 347 626.585 L 347 621.781 L 347 617.692 L 347 614.496 L 347 612.373 L 347 611.5 L 347 611.5 L 346.25 606.25 L 344 602.5 L 340.25 600.25 L 335 599.5 L 335 599.5 L 333.487 599.5 L 329.555 599.5 L 323.548 599.5 L 315.812 599.5 L 306.694 599.5 L 296.539 599.5 L 285.692 599.5 L 274.5 599.5 L 263.308 599.5 L 252.461 599.5 L 242.306 599.5 L 233.187 599.5 L 225.452 599.5 L 219.445 599.5 L 215.513 599.5 L 214 599.5 L closepath stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 239.9 629.5] concat newpath 0 0 M 0 28 L 69.2 28 L 69.2 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 237.9 627.5 M 237.9 659.5 L 311.1 659.5 L 311.1 627.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 239.9 629.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (Resource) 67.2 S 0 3.5 M (Manager) 58.8 S setmatrix grestore gsave 0 0 0 setrgbcolor 1.000000 setlinewidth newpath 83.5 464.5 M 78.25 465.25 L 74.5 467.5 L 72.25 471.25 L 71.5 476.5 L 71.5 476.5 L 71.5 477.373 L 71.5 479.496 L 71.5 482.692 L 71.5 486.781 L 71.5 491.585 L 71.5 496.926 L 71.5 502.624 L 71.5 508.5 L 71.5 514.376 L 71.5 520.074 L 71.5 525.415 L 71.5 530.219 L 71.5 534.308 L 71.5 537.504 L 71.5 539.627 L 71.5 540.5 L 71.5 540.5 L 72.25 545.75 L 74.5 549.5 L 78.25 551.75 L 83.5 552.5 L 83.5 552.5 L 85.0347 552.5 L 89.0273 552.5 L 95.1235 552.5 L 102.969 552.5 L 112.208 552.5 L 122.488 552.5 L 133.454 552.5 L 144.75 552.5 L 156.023 552.5 L 166.918 552.5 L 177.081 552.5 L 186.156 552.5 L 193.791 552.5 L 199.629 552.5 L 203.317 552.5 L 204.5 552.5 L 204.5 552.5 L 209.75 551.75 L 213.5 549.5 L 215.75 545.75 L 216.5 540.5 L 216.5 540.5 L 216.5 539.627 L 216.5 537.504 L 216.5 534.308 L 216.5 530.219 L 216.5 525.415 L 216.5 520.074 L 216.5 514.376 L 216.5 508.5 L 216.5 502.624 L 216.5 496.926 L 216.5 491.585 L 216.5 486.781 L 216.5 482.692 L 216.5 479.496 L 216.5 477.373 L 216.5 476.5 L 216.5 476.5 L 215.75 471.25 L 213.5 467.5 L 209.75 465.25 L 204.5 464.5 L 204.5 464.5 L 202.987 464.5 L 199.055 464.5 L 193.048 464.5 L 185.312 464.5 L 176.194 464.5 L 166.039 464.5 L 155.192 464.5 L 144 464.5 L 132.808 464.5 L 121.961 464.5 L 111.806 464.5 L 102.688 464.5 L 94.9521 464.5 L 88.9453 464.5 L 85.0127 464.5 L 83.5 464.5 L closepath stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 109.4 494.5] concat newpath 0 0 M 0 28 L 69.2 28 L 69.2 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 107.4 492.5 M 107.4 524.5 L 180.6 524.5 L 180.6 492.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 109.4 494.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (Process ) 67.2 S 0 3.5 M (Manager) 58.8 S setmatrix grestore gsave 0 0 0 setrgbcolor 1.000000 setlinewidth newpath 340 464.5 M 334.75 465.25 L 331 467.5 L 328.75 471.25 L 328 476.5 L 328 476.5 L 328 477.373 L 328 479.496 L 328 482.692 L 328 486.781 L 328 491.585 L 328 496.926 L 328 502.624 L 328 508.5 L 328 514.376 L 328 520.074 L 328 525.415 L 328 530.219 L 328 534.308 L 328 537.504 L 328 539.627 L 328 540.5 L 328 540.5 L 328.75 545.75 L 331 549.5 L 334.75 551.75 L 340 552.5 L 340 552.5 L 341.535 552.5 L 345.527 552.5 L 351.624 552.5 L 359.469 552.5 L 368.708 552.5 L 378.988 552.5 L 389.954 552.5 L 401.25 552.5 L 412.523 552.5 L 423.418 552.5 L 433.581 552.5 L 442.656 552.5 L 450.291 552.5 L 456.129 552.5 L 459.817 552.5 L 461 552.5 L 461 552.5 L 466.25 551.75 L 470 549.5 L 472.25 545.75 L 473 540.5 L 473 540.5 L 473 539.627 L 473 537.504 L 473 534.308 L 473 530.219 L 473 525.415 L 473 520.074 L 473 514.376 L 473 508.5 L 473 502.624 L 473 496.926 L 473 491.585 L 473 486.781 L 473 482.692 L 473 479.496 L 473 477.373 L 473 476.5 L 473 476.5 L 472.25 471.25 L 470 467.5 L 466.25 465.25 L 461 464.5 L 461 464.5 L 459.487 464.5 L 455.555 464.5 L 449.548 464.5 L 441.812 464.5 L 432.694 464.5 L 422.539 464.5 L 411.692 464.5 L 400.5 464.5 L 389.308 464.5 L 378.461 464.5 L 368.306 464.5 L 359.187 464.5 L 351.452 464.5 L 345.445 464.5 L 341.513 464.5 L 340 464.5 L closepath stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 352.3 486.5] concat newpath 0 0 M 0 42 L 94.4 42 L 94.4 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 350.3 484.5 M 350.3 530.5 L 448.7 530.5 L 448.7 484.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 352.3 486.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 31.5 M (Application) 92.4 S 0 17.5 M (\(Message) 67.2 S 0 3.5 M ( Passing\)) 75.6 S setmatrix grestore gsave 1.000000 setlinewidth matrix currentmatrix [92.7699 0 0 92.7699 135 576] concat 0 0 0 setrgbcolor newpath 0 0 1 116.565 116.565 arc setmatrix stroke grestore gsave 1.000000 setlinewidth matrix currentmatrix [62.516 0 0 62.516 135 630] concat 0 0 0 setrgbcolor newpath 0 0 1 331.699 331.699 arc setmatrix stroke grestore gsave 0 0 0 setrgbcolor 1.000000 setlinewidth newpath 488.024 707.855 M 486.739 709.928 L 485.304 711.944 L 483.721 713.889 L 481.997 715.747 L 480.138 717.503 L 478.154 719.143 L 476.055 720.654 L 473.855 722.024 L 471.569 723.244 L 469.211 724.306 L 466.799 725.205 L 464.348 725.937 L 461.877 726.501 L 459.401 726.899 L 456.937 727.134 L 454.5 727.21 L 454.5 727.21 L 452.063 727.134 L 449.599 726.899 L 447.123 726.501 L 444.652 725.937 L 442.201 725.205 L 439.789 724.306 L 437.431 723.244 L 435.145 722.024 L 432.945 720.654 L 430.846 719.143 L 428.862 717.503 L 427.003 715.747 L 425.279 713.889 L 423.696 711.944 L 422.261 709.928 L 420.976 707.855 L 420.976 707.855 L 419.824 705.706 L 418.795 703.455 L 417.902 701.112 L 417.155 698.69 L 416.563 696.202 L 416.135 693.663 L 415.876 691.09 L 415.79 688.5 L 415.876 685.91 L 416.135 683.337 L 416.563 680.798 L 417.155 678.31 L 417.902 675.888 L 418.795 673.545 L 419.824 671.294 L 420.976 669.145 L 420.976 669.145 L 422.261 667.072 L 423.696 665.056 L 425.279 663.111 L 427.003 661.253 L 428.862 659.497 L 430.846 657.857 L 432.945 656.346 L 435.145 654.976 L 437.431 653.756 L 439.789 652.694 L 442.201 651.795 L 444.652 651.063 L 447.123 650.499 L 449.599 650.101 L 452.063 649.866 L 454.5 649.79 L 454.5 649.79 L 456.937 649.866 L 459.401 650.101 L 461.877 650.499 L 464.348 651.063 L 466.799 651.795 L 469.211 652.694 L 471.569 653.756 L 473.855 654.976 L 476.055 656.346 L 478.154 657.857 L 480.138 659.497 L 481.997 661.253 L 483.721 663.111 L 485.304 665.056 L 486.739 667.072 L 488.024 669.145 L 488.024 669.145 L 489.176 671.294 L 490.205 673.545 L 491.098 675.888 L 491.845 678.311 L 492.437 680.798 L 492.865 683.337 L 493.124 685.91 L 493.21 688.5 L 493.124 691.09 L 492.865 693.663 L 492.437 696.202 L 491.845 698.69 L 491.098 701.112 L 490.205 703.455 L 489.176 705.706 L 488.024 707.855 L closepath stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 436.7 681.5] concat newpath 0 0 M 0 14 L 35.6 14 L 35.6 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 434.7 679.5 M 434.7 697.5 L 474.3 697.5 L 474.3 679.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 436.7 681.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 3.5 M (User) 33.6 S setmatrix grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 350.058 661.164 translate -160.346 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 414 684 M 367.951 667.554 L stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 219.5 517.5 translate 180 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore gsave 0 setlinejoin 325 517.5 translate 0 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 238.5 517.5 M 306 517.5 L stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 138.567 553.141 translate -158.962 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore gsave 0 setlinejoin 257.433 598.859 translate 21.0375 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 156.3 559.962 M 239.7 592.038 L stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 396.923 553.115 translate -22.6199 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore gsave 0 setlinejoin 287.077 598.885 translate 157.38 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 379.385 560.423 M 304.615 591.577 L stroke grestore grestore showpage %%EndDocument @endspecial 470 1674 a Fs(Figure)f(1.1:)k(Structure)d(of)e(the)i(Run)o (time)g(En)o(vironmen)o(t)75 1780 y(The)f(starting)f(p)q(oin)o(t)h(for) f(dynamic)h(resource)g(and)f(pro)q(cess)h(managemen)o(t)f(is)h(the)f (resource)h(manager.)75 1836 y(It)k(is)h(the)g(resource)f(manager)g (that)g(allo)q(cates,)h(implicitly)j(or)c(explicitly)l(,)k(resources)c (necessary)h(for)75 1893 y(running)f(pro)q(cesses.)29 b(The)18 b(\\resource)g(manager")f(ma)o(y)h(b)q(e)h(an)o(ything)f(from) f(a)h(sophisticated)h(batc)o(h)75 1949 y(sc)o(heduling)f(system,)d(to)g (a)h(\014le)h(con)o(taining)g(a)e(list)i(of)e(mac)o(hines)i(on)f(whic)o (h)h(an)e(application)j(can)e(run,)75 2006 y(to)f(the)g(user)g (him/herself.)75 2126 y Fk(Acquiring)j(Resources)47 b Fs(In)18 b(a)f(generic)h(application,)h(there)f(are)f(t)o(w)o(o)f (distinct)i(phases)g(of)e(in)o(teraction)75 2182 y(with)k(the)f (resource)g(manager.)32 b(In)20 b(the)f(\014rst)g(phase,)i(the)e(user)h (requests)f(the)g(initial)j(resources)d(on)75 2238 y(whic)o(h)g(an)f (application)i(will)g(run.)29 b(This)19 b(phase)f(is)h(implicit)h(but)f (not)e(sp)q(eci\014ed)k(in)e(the)f(static)g(MPI)75 2295 y(mo)q(del.)30 b(In)19 b(the)f(second)h(phase,)g(the)g(application)h (itself)f(ma)o(y)e(request)i(additional)g(resources)g(from)75 2351 y(the)c(resource)h(manager.)166 2408 y(While)21 b(it)f(seems)g(at)g(\014rst)f(that)g(MPI)h(need)h(only)f(b)q(e)h (concerned)g(with)f(the)g(second)h(phase,)g(it)75 2464 y(m)o(ust)16 b(b)q(e)i(a)o(w)o(are)d(of)i(the)g(\014rst)f(as)h(w)o (ell.)25 b(The)18 b(reason)e(is)h(that)g(resources)f(ma)o(y)h(b)q(e)g (\\preallo)q(cated")h(to)75 2521 y(an)d(application.)166 2577 y(W)l(e)g(exp)q(ect)h(the)f(most)f(common)h(t)o(yp)q(e)g(of)g (\\dynamic")g(application)i(to)d(b)q(e)i(one)f(in)h(whic)o(h)g(all)g (re-)75 2634 y(sources)e(are)h(actually)g(allo)q(cated)g(b)q(efore)g (the)f(application)i(starts.)j(The)14 b(application)j(will)f(b)q(e)f (started)75 2690 y(on)10 b(a)g(subset)g(of)g(the)g(resources,)h(will)h Fl(disc)n(over)d Fs(the)i(additional)g(resources)f(using)h Fk(MPI)p 1545 2690 14 2 v 16 w(Resource)p 1729 2690 V 18 w(discover)p Fs(,)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fs(4)951 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fs(and)i(will)h(start)d(the)i(rest)f(of)g(the)g (application)i(on)f(those)f(additonal)h(resources.)24 b(W)l(e)16 b(term)g(these)h(ap-)75 102 y(plications)i Fl(quasi-static)e Fs(b)q(ecause)i(they)e(use)h(static)f(resources)h (but)f(ma)o(y)g(ha)o(v)o(e)g(dynamic)h(pro)q(cesses.)75 158 y(Applications)h(whic)o(h)f(disco)o(v)o(er)f(preallo)q(cated)h (resources)f(but)g(do)g(not)g(allo)q(cate)h(new)f(ones)g(will)i(gen-)75 214 y(erally)e(b)q(e)g(more)f(p)q(ortable)h(than)f(those)g(whic)o(h)h (allo)q(cate)g(new)g(resources.)23 b(Resource)17 b(disco)o(v)o(ery)g (ma)o(y)75 271 y(or)g(ma)o(y)h(not)f(require)i(comm)o(unication)g(with) f(an)g(external)g(resource)g(manager,)g(dep)q(ending)i(on)e(the)75 327 y(MPI)d(implemen)o(tation.)22 b(Resource)16 b(disco)o(v)o(ery)g (can)f(b)q(e)i(an)e(extremely)h(simple)h(op)q(eration,)e(so)g(that)g (it)75 384 y(can)k(b)q(e)g(used)g(b)o(y)f(a)g(naiv)o(e)h(user)g(in)g (conjunction)g(with)g Fk(MPI)p 1160 384 14 2 v 16 w(Spa)o(wn)g Fs(to)f(simply)h(mimic)h(PVM-st)o(yle)75 440 y(b)q(eha)o(vior.)166 497 y(T)l(ruly)e(dynamic)h(applications)g(will)g(allo)q(cate)f(new)g (resources)g(while)h(running,)g(in)o(teracting)f(di-)75 553 y(rectly)c(with)g(the)g(resource)g(manager)f(through)h Fk(MPI)p 995 553 V 16 w(Resource)p 1179 553 V 17 w(iallo)q(cate)p Fs(.)20 b(Because)14 b(this)g(op)q(eration)g(is)75 610 y(p)q(oten)o(tially)f(\(and)f(usually\))h(exp)q(ensiv)o(e,)h(it)e(is)h (non)o(blo)q(c)o(king.)20 b(A)12 b(resource)g(request)g(ma)o(y)g(b)q(e) g(arbitrarily)75 666 y(complicated,)21 b(but)e(the)g(details)h(of)e (the)h(request)g(are)f(in)o(terpreted)i(b)o(y)f(the)g(resource)g (manager,)f(not)75 723 y(MPI.)d(The)g(request)g(itself)i(it)e (therefore)g(a)g(string)g(whic)o(h)h(has)f(no)g(meaning)h(to)f(MPI.)166 779 y(Regardless)f(of)f(ho)o(w)g(an)h(application)h(acquires)f (resources,)g(it)f(ma)o(y)g(release)h(them)g(to)f(the)g(system)75 835 y(through)i Fk(MPI)p 332 835 V 16 w(Resource)p 516 835 V 17 w(free)p Fs(.)166 968 y Fh(Discussion:)61 b Fg(The)21 b(description)h(ab)q(o)o(v)o(e)e(is)h(sligh)o(tly)e (di\013eren)o(t)j(from)d(earlier)i(descriptions)h(in)e(that)75 1018 y(it)f(con)o(tains)g(the)h(concept)h(of)e(resource)i(disco)o(v)o (ery)m(.)34 b(There)21 b(are)e(t)o(w)o(o)g(reasons)i(for)e(this.)34 b(First,)21 b(almost)c(all)75 1068 y(applications)11 b(\(including)h(most)f(new)i(ones)g(w)o(e)g(target)g(in)f(MPI-2\))g (need)i(only)e(the)h(\014rst)g(phase)g(-)f(making)e(use)k(of)75 1117 y(pre-allo)q(cated)f(resources.)19 b(An)13 b(application)e (usually)g(do)q(es)j(not)e(actually)f(request)j(resources)h(nor)d(do)q (es)h(it)f(kno)o(w)75 1167 y(what)f(it)g(w)o(an)o(ts)g(\(nor)g(should)g (it!\).)16 b(It)c(only)e(w)o(an)o(ts)h(to)g(\014nd)g(out)g(what)g(the)h (user)g(has)f(already)g(allo)q(cated.)17 b(Second,)75 1217 y(it)12 b(w)o(as)g(di\016cult)f(to)h(come)f(up)i(with)e(a)h (uni\014ed)g(seman)o(tics)g(that)g(w)o(ould)f(w)o(ork)h(w)o(ell)g(for)f (b)q(oth)i(resource)h(disco)o(v)o(ery)75 1267 y(and)h(resource)j(allo)q (cation.)j(They)16 b(are)g(therefore)h(separate,)f(although)f(they)h(b) q(oth)g(pro)q(duce)g(the)h(same)d(result)75 1317 y(-)i(an)h Ff(MPI)p 243 1317 13 2 v 14 w(Resource)h Fg(ob)r(ject)g(\(or)e(ob)r (jects\).)28 b(This)16 b(distinction)h(actually)e(exists)j(in)e(PVM,)g (sort)i(of.)25 b(It)17 b(is)f(the)75 1366 y(di\013erence)g(b)q(et)o(w)o (een)f Fe(pvm)p 490 1366 14 2 v 15 w(config)e Fg(and)g Fe(pvm)p 796 1366 V 15 w(addhosts)p Fg(.)166 1416 y(It)22 b(w)o(ould)f(b)q(e)i(p)q(ossible)f(to)f(treat)i(b)q(oth)f(resource)i (allo)q(cation)c(and)i(resource)i(disco)o(v)o(ery)e(through)g(an)75 1466 y(allo)q(cation-based)16 b(in)o(terface.)29 b(An)18 b(application)e(w)o(ould)g(\\allo)q(cate")h(the)h(resources)h(it)e (knew)h(to)f(b)q(e)h(a)o(v)n(ailable)75 1516 y(and)e(the)h(allo)q (cation)d(routine)i(w)o(ould)f(return)j(quic)o(kly)m(.)23 b(This)16 b(approac)o(h)g(is)g(undesirable)g(for)g(sev)o(eral)g (reasons.)75 1566 y(First,)e(in)g(the)h(allo)q(cation-based)e(approac)o (h,)h(an)g(application)f(m)o(ust)g(\014nd)h(out)g(through)h(some)e (external)i(means)75 1616 y(\(suc)o(h)h(as)f(a)g(con\014guration)g (\014le\))g(what)g(resources)j(it)d(should)f(ask)i(for.)21 b(This)15 b(is)g(cum)o(b)q(ersome)f(and)h(redundan)o(t)75 1665 y(with)e(the)i(initial)d(resource)j(request.)20 b(Second,)14 b(it)f(requires)i(ev)o(en)g(simple)d(applications)h(to)g (form)o(ulate)f(a)i(p)q(oten-)75 1715 y(tially)e(complicated)h (resource)i(request.)20 b(Resource)15 b(sp)q(eci\014cation)f(will)f(v)n (ary)g(widely)g(with)g(resource)j(managers)75 1765 y(and)g(should)g (remain)e(outside)i(an)g(application)f(as)h(m)o(uc)o(h)e(as)i(p)q (ossible,)h(for)e(p)q(ortabilit)o(y)m(.)23 b(Third,)16 b(it)f(w)o(ould)g(b)q(e)75 1815 y(a)h(lie,)h(as)g(resource)h(ha)o(v)o (e)f(already)f(b)q(een)i(allo)q(cated.)26 b(This)16 b(is)h(alw)o(a)o (ys)f(a)g(dangerous)h(thing)f(and)h(can)g(mislead)75 1865 y(programmers.)166 1914 y(I)h(exp)q(ect)i(that)f(true)g(dynamic)e (applications)g(are)i(rare.)32 b(A)18 b(lot)g(of)g(the)h(true)g (dynamicism)c(ma)o(y)i(come)75 1964 y(in)e(parallel)g(en)o(vironmen)o (ts)g(whic)o(h)h(are)g(written)g(in)g(MPI.)f(F)m(or)h(instance,)g(with) g(the)g(dynamic)e(in)o(terface,)j(it)e(is)75 2014 y(p)q(ossible)e(to)g (imagine)d(writing)j(an)f(MPI)h(console)h(similar)c(to)j(the)g(PVM)h (console)f(that)g(puts)g(together)h(a)f(virtual)75 2064 y(mac)o(hine)g(and)g(spa)o(wns)i(MPI)f(applications)f(on)g(that)h (virtual)f(mac)o(hine.)166 2114 y(I)i(also)g(exp)q(ect)i(that)f (resource)i(disco)o(v)o(ery)e(will)e(b)q(e)i(quite)f(p)q(ortable,)h (while)f(resource)j(allo)q(cation)c(ma)o(y)f(b)q(e)75 2163 y(tailored)g(to)h(sp)q(eci\014c)h(en)o(vironmen)o(ts.)166 2213 y(-)f(w)o(cs)166 2352 y Fs(MPI)h(do)q(es)g(not)g(address)g(the)g (issue)h(of)f(\014nding)h(out)f(what)f(additional)j(resources)e(ma)o(y) f(b)q(e)i(a)o(v)m(ail-)75 2409 y(able.)26 b(F)l(or)16 b(instance,)i(an)f(application)i(migh)o(t)d(w)o(an)o(t)g(to)h(kno)o(w)f (what)g(is)i(p)q(ossible)h(b)q(efore)e(it)g(requests)75 2465 y(resources.)40 b(This)22 b(issue)h(is)f(quite)h(complex)f(and)g (not)g(directly)h(necessary)f(for)f(MPI,)h(in)g(con)o(trast)75 2522 y(to)d(resource)h(allo)q(cation)h(itself,)h(in)f(whic)o(h)f(an)g (application)i(obtains)e(simple)h(MPI-de\014ned)h(ob)s(jects)75 2578 y(necessary)g(for)f(starting)h(pro)q(cesses.)40 b(It)22 b(is)g(exp)q(ected)h(that)e(an)h(application)i(will)f(use)f(a)g (resource)75 2635 y(manager-sp)q(eci\014c)17 b(API)e(to)g(inquire)i(ab) q(out)e(resource)g(a)o(v)m(ailabilit)o(y)l(.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fn(1.2.)34 b(THE)15 b(MPI)g(D)o(YNAMIC)g(PR)o(OCESS)h (MODEL)813 b Fs(5)75 45 y Fk(Sta)o(rting)21 b(and)g(Managing)f(Pro)q (cesses)47 b Fs(As)20 b(with)h(resource)g(managemen)o(t,)f(there)h(are) f(t)o(w)o(o)f(t)o(yp)q(es)h(of)75 102 y(pro)q(cess)e(creation.)27 b(The)18 b(\014rst)f(is)h(the)f(creation)h(of)f(the)h(original)h(MPI)e (application.)29 b(The)17 b(second)h(is)75 158 y(creation)g(of)g(pro)q (cesses)g(from)f(within)i(an)f(MPI)g(application.)29 b(F)l(ortunately)18 b(MPI)g(need)h(only)f(w)o(orry)75 214 y(ab)q(out)d(the)g(second)h(t)o(yp)q(e.)166 271 y(MPI)h (applications)h(ma)o(y)e(start)f(new)i(pro)q(cesses)h(\(including)h (non-MPI)e(pro)q(cesses\),)g(send)g(them)75 327 y(signals,)24 b(and)e(\014nd)h(out)e(when)i(they)f(die)h(or)e(b)q(ecome)i(unreac)o (hable.)41 b(They)22 b(do)g(this)g(through)g(an)75 384 y(in)o(terface)16 b(to)f(the)h(pro)q(cess)g(manager,)f(whic)o(h)i (range)e(from)g(a)h(parallel)h(op)q(erating)f(system)g(\(CMOST\))75 440 y(to)f(la)o(y)o(ered)g(soft)o(w)o(are)e(\(POE\))i(to)g(an)g Fj(rsh)g Fs(command)g(\(P4\).)166 497 y(There)h(are)f(t)o(w)o(o)g(w)o (a)o(ys)g(to)g(start)f(new)i(pro)q(cesses.)22 b Fk(MPI)p 1132 497 14 2 v 16 w(Spa)o(wn)17 b Fs(is)f(a)g(simple)h(in)o(terface)f (adequate)75 553 y(or)e(most)g(applications.)21 b Fk(MPI)p 596 553 V 16 w(Pro)q(cess)p 755 553 V 17 w(sta)o(rt)15 b Fs(pro)o(vides)g(more)f(\015exibilit)o(y)j(and)e(the)g(abilit)o(y)h (to)d(manage)75 610 y(non-MPI)f(pro)q(cesses.)19 b(Both)11 b(tak)o(e)g(as)g(input)i(an)f Fk(MPI)p 996 610 V 15 w(Resource)h Fs(ob)s(ject,)e(represen)o(ting)i(computational)75 666 y(resources)j(on)f(whic)o(h)i(a)e(job)g(can)h(run.)22 b(F)l(or)15 b(simple)i(applications,)g(there)f(is)g(a)f(simple)i(w)o(a) o(y)e(to)g(obtain)75 723 y(this)h Fk(MPI)p 249 723 V 15 w(Resource)h Fs(ob)s(ject.)166 779 y Fk(MPI)p 251 779 V 16 w(Spa)o(wn)e Fs(starts)f(MPI)h(pro)q(cesses)g(and)g (establishes)h(comm)o(unication)g(with)f(them,)f(returning)75 835 y(a)h(comm)o(unicator.)166 892 y Fk(MPI)p 251 892 V 16 w(Pro)q(cess)p 410 892 V 17 w(sta)o(rt)i Fs(merely)g(starts)e(new) i(pro)q(cesses.)24 b(An)17 b(application)h(ma)o(y)e(establish)h(comm)o (u-)75 948 y(nication)f(with)g(these)f(new)h(pro)q(cesses)f(through)g Fk(MPI)p 1016 948 V 16 w(Pro)q(cess)p 1175 948 V 17 w(attach)p Fs(.)75 1070 y Fm(1.2.3)49 b(Examples)16 b(of)h(Runtime)f(Environments) 75 1156 y Fs(T)l(o)11 b(illustrate)h(ho)o(w)f(the)h(ab)q(o)o(v)o(e)e (framew)o(ork)g(allo)o(ws)i(us)f(to)g(describ)q(e)i(a)e(wide)h(v)m (ariet)o(y)g(of)e(actual)i(systems,)75 1212 y(w)o(e)j(giv)o(e)g(here)h (some)f(examples.)75 1332 y Fk(Environments)22 b(with)f(explicit)h (Resource)g(Managers)45 b Fs(The)22 b(SP2)e(computers)h(at)g(Argonne)g (National)75 1389 y(Lab)q(oratory)d(and)g(NASA)h(Ames)g(Lab)q(oratory)f (use)g(\(di\013eren)o(t\))h(lo)q(cally)h(written)e(job)h(sc)o(hedulers) h(to)75 1445 y(manage)d(the)g(SP2.)25 b(The)18 b(sc)o(hedulers)g (ensure)g(that)f(only)g(one)h(user)f(has)g(access)g(to)g(an)o(y)f(SP)i (no)q(de)g(at)75 1502 y(a)e(time)g(and)g(manage)f(resources)h(in)h(a)e (\\fair")h(w)o(a)o(y)f(to)g(ensure)h(that)f(all)i(users)f(can)g(get)g (access)g(to)f(the)75 1558 y(mac)o(hines.)20 b(They)15 b(require)g(users)f(to)f(pro)o(vide)i(times)f(limits)i(for)d(their)i (jobs)f(so)f(that)h(the)g(mac)o(hine)h(can)75 1615 y(b)q(e)i(tigh)o (tly)g(sc)o(heduled.)26 b(Users)16 b(submit)h(scripts)g(to)f(the)h(sc)o (heduler,)h(whic)o(h)f(allo)q(cates)g(resources)g(and)75 1671 y(runs)f(jobs)f(using)i(IBM's)e(P)o(arallel)i(En)o(vironmen)o(t)f (soft)o(w)o(are.)k(The)15 b(systems)h(in)o(teract)f(with)h(a)g(v)m (ariet)o(y)75 1728 y(of)f(message-passing)g(libraries,)h(includin)q(g)h (t)o(w)o(o)d(based)i(on)f(MPI.)166 1784 y(The)j(corresp)q(ondence)h (with)f(the)f(mo)q(del)i(ab)q(o)o(v)o(e)e(is)h(straigh)o(tforw)o(ard.) 25 b(The)18 b(resource)g(manager)75 1841 y(is)h(the)g(lo)q(cally)i (written)e(job)g(sc)o(heduler.)32 b(The)20 b(pro)q(cess)f(manager)f(is) h(con)o(tained)h(in)g(IBM's)e(parallel)75 1897 y(en)o(vironmen)o(t)d (soft)o(w)o(are.)166 1953 y(There)23 b(are)g(n)o(umerous)f(examples)i (of)e(job)h(managemen)o(t)f(systems)h(that)f(allo)q(cate)h(resources.) 75 2010 y(These)d(include)i(PBS)e(\(from)f(NASA)h(Ames\),)g(EASY)g (\(from)e(ANL\),)i(LSF)g(\(from)f(Platform)f(Com-)75 2066 y(puting\),)f(LoadLev)o(eler)g(\(from)e(IBM\),)h(DQS)g([2])f (\(from)h(Florida)g(State)g(Univ)o(ersit)o(y\),)g(Condor)g(\(from)75 2123 y(the)f(Univ)o(ersit)o(y)h(of)f(Wisconsin\),)h(and)f(NQS.)166 2179 y(Eac)o(h)20 b(one)h(of)f(these)h(resource)g(managers)e(can)i(b)q (e)g(used)h(in)f(conjunction)h(with)f(one)f(or)g(more)75 2236 y(pro)q(cess)c(managers.)22 b(Pro)q(cess)16 b(managers)g(ma)o(y)f (b)q(e)i(programs)e(external)h(to)g(an)g(application)i(\()p Fj(poe)d Fs(on)75 2292 y(the)i(IBM)h(SP2\),)f(part)g(of)g(the)g(op)q (erating)h(system)f(\(usually)h(on)f(tigh)o(tly)h(in)o(tegrated)f(MPPs) g(suc)o(h)g(as)75 2349 y(the)d(In)o(tel)g(P)o(aragon,)e(Meik)o(o)i (CS-2)f(and)h(TMC)f(CM-5\),)g(part)g(of)g(the)h(message)f(passing)h (library)g(\()p Fj(p4)f Fs(or)75 2405 y Fj(pvm)p Fs(\))h(or)h(in)o (tegrated)g(with)h(the)f(resource)g(manager)g(\(Condor)f(with)i Fj(pvm)p Fs(\).)75 2525 y Fk(Net)o(w)o(o)o(rk)c(of)f(W)o(o)o (rkstations)i(with)f(PVM)h(o)o(r)e(P4)45 b Fs(One)13 b(reason)e(for)h(PVM's)f(p)q(opularit)o(y)i(is)f(that)g(it)g(can)g(b)q (e)75 2582 y(view)o(ed)h(as)f(a)g(completely)i(self-con)o(tained)f (system)f(that)g(supplies)i(its)f(o)o(wn)f(pro)q(cess)g(managemen)o(t)g (and)75 2638 y(can)g(b)q(e)h(used)g(to)e(implemen)o(t)i(a)f(resource)g (manager)f(as)h(w)o(ell.)20 b(On)13 b(systems)e(that)g(ha)o(v)o(e)h (neither)h(of)f(these)75 2694 y(functions)20 b(pre-installed,)i(PVM)c (can)h(pro)o(vide)h(a)e(complete)i(solution.)32 b(A)19 b(user)g(creates)g(a)g(\\virtual)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fs(6)951 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fs(mac)o(hine")h(b)o(y)f(starting)g(\\daemons")g (on)g(an)g(assortmen)o(t)f(of)h(mac)o(hines)h(and)g(then)g(sc)o (hedules)g(jobs)g(to)75 102 y(run)i(on)g(it)h(and)f(manages)f(his)i (pro)q(cesses)f(with)h(the)f(help)h(of)f(the)g(daemons.)28 b(The)18 b(virtual)h(mac)o(hine)75 158 y(itself)h(can)g(b)q(e)g (recon\014gured)g(from)f(inside)i(the)f(user)g(program.)31 b(A)20 b(di\016cult)o(y)h(with)e(this)h(approac)o(h)75 214 y(is)g(that)e(the)h(user)g(is)h(assumed)f(to)f(ha)o(v)o(e)h(the)g (necessary)h(p)q(ermissions)g(to)e(execute)i(suc)o(h)g(functions.)75 271 y(This)15 b(ma)o(y)f(b)q(e)h(the)f(case)h(on)f(a)g(w)o(orkstation)f (net)o(w)o(ork,)g(but)i(seldom)f(on)h(an)f(MPP)l(.)g(Con\015icts)h(b)q (et)o(w)o(een)75 327 y(existing)e(pro)q(cess)g(managers)e(and)h(PVM)g (can)h(inhibit)h(the)e(p)q(ortabilit)o(y)i(\(to)d(MPP's\))g(of)h (self-con)o(tained)75 384 y(programs)i(that)g(assume)i(all)g (functionalit)o(y)g(will)h(b)q(e)f(pro)o(vided)g(b)o(y)f(PVM.)166 440 y(Equiv)m(alen)o(t)h(MPI)f(functionalit)o(y)h(requires)g(an)f(MPI)f (implemen)o(tation)i(in)g(whic)o(h)g(b)q(oth)f(resource)75 497 y(managemen)o(t)22 b(and)i(pro)q(cess)f(managemen)o(t)g(are)f(pro)o (vided)i(b)o(y)g(the)f(MPI)g(implemen)o(tation)h(itself.)75 553 y(While)f(implemen)o(ted)g(within)f(MPI,)f(these)g(functions)h(w)o (ould)g(b)q(e)g(logically)h(external)e(to)g(an)g(MPI)75 610 y(application,)f(allo)o(wing)f(the)e(application)j(to)d(run)h (virtually)h(unc)o(hanged)g(in)g(the)f(presence)h(of)e(a)h(real)75 666 y(resource)c(and)g(pro)q(cess)g(managers.)19 b(In)14 b(suc)o(h)g(an)g(implemen)o(tation,)h(requests)f(to)f(the)h(\\resource) g(man-)75 723 y(ager")k(w)o(ould)h(alw)o(a)o(ys)g(succeed,)h(assuming)g (the)f(user)g(had)g(appropriate)g(p)q(ermission)h(to)e(start)g(pro-)75 779 y(cesses.)75 899 y Fk(PVM)11 b(p)o(ro)q(cess)h(management)e(with)i (MPI)e(message-passing.)47 b Fs(One)11 b(can)h(conceiv)o(e)g(of)e(a)h (system)f(in)i(whic)o(h)75 956 y(resource)j(allo)q(cation)h(and)f(pro)q (cess)g(managemen)o(t)f(functions)i(w)o(ere)f(pro)o(vided)h(b)o(y)e (the)h(existing)h(PVM)75 1012 y(daemon)d(structure,)g(while)h(MPI)f(w)o (as)f(used)h(b)o(y)g(the)g(application)i(for)d(message)h(passing.)19 b(PVM)13 b(w)o(ould)75 1068 y(b)q(e)j(used)g(as)f(the)h(implemen)o (tation)g(la)o(y)o(er)f(for)g(the)h(functions)g(describ)q(ed)h(in)f (this)g(c)o(hapter,)f(but)h(w)o(ould)75 1125 y(not)f(b)q(e)h(visible)h (to)e(the)g(application)i(program.)75 1247 y Fm(1.2.4)49 b(Applications)18 b(Requiring)f(Direct)e(Communication)i(with)g(the)e (Runtime)h(System)75 1332 y Fs(The)i(existing)h(MPI)f(sp)q (eci\014cation)h(is)g(adequate)e(for)g(most)g(parallel)j(applications.) 29 b(In)18 b(these)g(appli-)75 1389 y(cations,)e(the)h(resource)f (manager)g(and)g(pro)q(cess)h(manager,)f(whether)g(simple)i(or)e(elab)q (orate,)g(allo)q(cate)75 1445 y(resources)f(and)h(manage)e(user)i(pro)q (cesses)g(without)f(in)o(teracting)h(with)f(the)g(application)i (program.)i(In)75 1502 y(other)e(applications,)j(ho)o(w)o(ev)o(er,)c (it)i(is)g(necessary)g(that)f(the)h Fl(user)g(level)j Fs(of)c(the)h(application)h(comm)o(u-)75 1558 y(nicate)f(with)g(the)g (job)g(sc)o(heduler)h(and/or)e(the)h(pro)q(cess)g(manager.)27 b(Here)18 b(w)o(e)f(describ)q(e)i(three)f(broad)75 1615 y(classes)e(of)g(suc)o(h)g(applications.)24 b(In)17 b(Section)f(1.5)f (w)o(e)h(will)i(giv)o(e)e(concrete)g(examples)h(of)e(eac)o(h)h(of)g (these)75 1671 y(classes.)75 1791 y Fk(T)l(ask)c(F)o(a)o(rming)43 b Fs(By)11 b(a)g(\\task)f(farm")g(application)i(w)o(e)f(mean)g(a)f (program)g(that)g(manages)h(the)g(execution)75 1848 y(of)j(a)g(set)g (of)g(other,)g(p)q(ossibly)i(sequen)o(tial,)g(programs.)i(This)d (situation)g(often)f(arises)h(when)g(one)f(w)o(an)o(ts)75 1904 y(to)19 b(run)h(the)g(same)f(sequen)o(tial)i(program)e(man)o(y)g (times)h(with)g(v)m(arying)h(input)f(data.)33 b(W)l(e)20 b(call)h(eac)o(h)75 1961 y(in)o(v)o(o)q(cation)14 b(of)e(the)i(sequen)o (tial)g(program)e(a)h Fl(task)p Fs(.)19 b(It)13 b(is)h(often)f (simplest)h(to)f(\\parallelize")i(the)e(existing)75 2017 y(sequen)o(tial)h(program)d(b)o(y)i(writing)g(a)f(parallel)j (\\harness")d(program)f(that)h(in)i(turn)e(dev)o(otes)h(a)f(separate,) 75 2073 y(transien)o(t)j(pro)q(cess)g(to)g(eac)o(h)g(task.)k(When)d (one)f(task)f(\014nishes,)i(a)f(new)g(pro)q(cess)h(is)f(started)g(to)f (execute)75 2130 y(the)19 b(next)f(one.)30 b(Ev)o(en)19 b(if)g(the)f(resources)h(allo)q(cated)g(to)f(the)h(job)f(are)g (\014xed,)i(the)f(\\harness")f(pro)q(cess)75 2186 y(m)o(ust)13 b(in)o(teract)g(frequen)o(tly)h(with)g(the)f(pro)q(cess)h(manager)f (\(ev)o(en)g(if)h(this)g(is)g(just)f Fj(rsh)p Fs(,)g(to)g(start)f(the)i (new)75 2243 y(pro)q(cesses)i(with)f(the)h(new)f(input)h(data\).)j(In)d (man)o(y)f(cases)g(this)h(harness)f(can)g(b)q(e)h(written)g(in)g(a)e (simple)75 2299 y(scripting)k(language)f(lik)o(e)g Fj(csh)g Fs(or)f Fj(perl)p Fs(,)g(but)h(some)f(users)h(prefer)g(to)f(use)h(F)l (ortran)e(or)h(C.)g(Note)h(that)75 2356 y(it)f(is)g(an)g(explicit)i (goal)d(of)h(the)f(MPI)h(dynamic)h(pro)q(cess)f(arc)o(hitecture)g(to)f (allo)o(w)h(the)f(managemen)o(t)g(of)75 2412 y(non-MPI)h(pro)q(cesses.) 75 2532 y Fk(Dynamic)f(numb)q(er)i(of)e(p)o(ro)q(cesses)j(in)e(pa)o (rallel)f(job)46 b Fs(The)16 b(program)f(wishes)i(to)e(decide)j Fl(inside)g Fs(the)e(pro-)75 2589 y(gram)e(to)g(adjust)g(the)g(n)o(um)o (b)q(er)h(of)f(pro)q(cesses)h(to)f(\014t)h(the)f(size)i(of)e(the)h (problem.)20 b(F)l(urthermore,)14 b(it)h(ma)o(y)75 2645 y(con)o(tin)o(ue)k(to)g(add)g(and)g(subtract)f(pro)q(cesses)h(during)h (the)f(computation)g(to)f(\014t)h(separate)f(phases)h(of)75 2702 y(the)g(computation,)h(some)f(of)g(whic)o(h)h(ma)o(y)f(b)q(e)h (more)f(parallel)i(than)e(others.)32 b(In)20 b(order)f(to)g(do)g(this,) 1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fn(1.3.)33 b(NEW)15 b(OBJECTS)g(F)o(OR)g (IMPLEMENTING)g(D)o(YNAMIC)f(PR)o(OCESS)h(MANA)o(GEMENT)p Fs(7)75 45 y(the)g(application)i(program)d(will)j(ha)o(v)o(e)d(to)h(in) o(teract)g(with)g(the)g(resource)g(manager)g(\(ho)o(w)o(ev)o(er)e(it)j (is)f(im-)75 102 y(plemen)o(ted\))20 b(to)f(request)g(and)g(acquire)h (or)f(return)g(computation)g(resources.)32 b(It)19 b(will)i(also)e(ha)o (v)o(e)g(to)75 158 y(in)o(teract)c(with)h(the)f(pro)q(cess)h(manager)e (to)h(request)g(that)f(pro)q(cess)i(b)q(e)g(started)e(and)i(in)g(order) f(to)f(mak)o(e)75 214 y(the)e(new)f(pro)q(cesses)h(kno)o(wn)f(to)g(the) h(message-passing)g(library)g(so)f(that)g(the)g(larger)h(\(or)e (smaller\))i(group)75 271 y(of)j(pro)q(cesses)g(can)h(comm)o(unicate.) 166 327 y(An)21 b(imp)q(ortan)o(t)f(t)o(yp)q(e)h(of)f(dynamic)h (application)i(is)e(a)f(sca)o(v)o(enger)g(application.)38 b(A)20 b(sca)o(v)o(enger)75 384 y(application)j(is)g(\\em)o (barassingly)f(parallel")h(in)f(the)g(sense)g(that)f(it)h(p)q(erforms)g (a)f(large)h(n)o(um)o(b)q(er)g(of)75 440 y(completely)17 b(indep)q(enden)o(t)h(tasks.)j(It)16 b(the)g(n)o(um)o(b)q(er)g(of)f (tasks)g(is)i(large)f(enough,)f(suc)o(h)i(an)e(application)75 497 y(can)21 b(mak)o(e)f(use)h(of)f(an)o(y)h(resources)g(as)f(they)h(b) q(ecome)g(a)o(v)m(ailable.)38 b(Con)o(v)o(ersely)l(,)22 b(it)f(can)g(easily)h(giv)o(e)75 553 y(up)17 b(resources)h(to)e (another)h(application.)27 b(Sca)o(v)o(enger)16 b(applications)j(are)e (excellen)o(t)h(for)f(\014lling)i(in)f(the)75 610 y(\\holes")d(on)g(a)g (space-shared)h(parallel)h(mac)o(hine,)f(allo)o(wing)g(it)f(to)g(ac)o (hiev)o(e)g(v)o(ery)g(high)h(utilization.)75 730 y Fk(Client/Server)46 b Fs(This)15 b(situation)h(is)f(the)g(opp)q(osite)h(of)e(the)h (situations)g(ab)q(o)o(v)o(e,)g(where)g(pro)q(cesses)g(come)75 786 y(and)21 b(go)f(up)q(on)h(request.)37 b(In)21 b(the)g(clien)o (t/serv)o(er)g(mo)q(del,)i(one)e(set)f(of)g(pro)q(cesses)h(is)g (relativ)o(ely)h(p)q(er-)75 843 y(manen)o(t)17 b(\(the)g(serv)o(er,)g (whic)o(h)h(w)o(e)f(assume)g(here)h(ma)o(y)e(b)q(e)i(a)f(parallel)i (program\).)24 b(A)o(t)17 b(unpredictable)75 899 y(times,)e(another)f (\(p)q(ossibly)h(parallel\))h(program)d(\(the)i(clien)o(t\))g(b)q (egins)h(execution)f(and)g(m)o(ust)f(establish)75 956 y(comm)o(unication)k(with)f(the)g(serv)o(er.)26 b(In)17 b(this)h(case)f(the)g(pro)q(cess)g(manager)g(m)o(ust)f(pro)o(vide)i(a)f (w)o(a)o(y)f(for)75 1012 y(the)e(clien)o(t)h(to)e(lo)q(cate)h(the)g (serv)o(er)f(and)h(comm)o(unicate)g(to)f(the)g(message-passing)h (library)h(that)e(it)h(m)o(ust)75 1068 y(no)o(w)h(supp)q(ort)g(comm)o (unications)h(with)f(a)g(new)h(collection)h(of)d(pro)q(cesses.)166 1125 y(It)19 b(is)g(curren)o(tly)h(p)q(ossible)g(to)f(write)g(the)g (parallel)h(clien)o(ts)g(and)f(serv)o(ers)g(in)g(MPI,)g(but)g(b)q (ecause)75 1181 y(MPI)g(do)q(es)g(not)g(pro)o(vide)h(the)f(necessary)g (in)o(terfaces)g(b)q(et)o(w)o(een)h(the)f(application)i(program)d(and)h (the)75 1238 y(resource)f(manager)g(or)g(pro)q(cess)h(manager,)f(other) g(nonp)q(ortable,)h(mac)o(hine)g(sp)q(eci\014c)i(libraries)f(m)o(ust)75 1294 y(b)q(e)15 b(called)h(in)f(order)f(for)g(the)g(clien)o(t)h(and)g (serv)o(er)f(to)f(comm)o(unicate)i(with)g(one)f(another.)19 b(On)c(the)f(other)75 1351 y(hand,)g(MPI)g(do)q(es)g(con)o(tain)g(sev)o (eral)f(features)h(that)f(mak)o(e)g(it)h(relativ)o(ely)h(easy)e(to)g (add)h(suc)o(h)g(in)o(terfaces,)75 1407 y(and)h(w)o(e)g(prop)q(ose)h(b) q(oth)f(a)g(simple)h(in)o(terface)g(and)f(a)g(more)g(complex)h(but)f (\015exible)j(one.)75 1550 y Fp(1.3)59 b(New)20 b(Objects)e(fo)n(r)j (Implementing)c(Dynamic)i(Pro)r(cess)g(Management)75 1652 y Fs(Using)d(dynamic)g(pro)q(cesses)f(in)h(MPI)g(is)f(a)g(three)h (step)f(pro)q(cess)g(sho)o(wn)g(in)h(Figure)g(1.2.)131 1746 y(1.)22 b Fd(Allo)q(cate)14 b(or)e(disco)o(v)o(er)f(resources)p Fs(,)f(obtaining)h(one)g(or)g(more)f Fk(MPI)p 1403 1746 14 2 v 16 w(Resource)h Fs(ob)s(jects,)g(whic)o(h)189 1802 y(represen)o(t)k(p)q(ermission)h(to)f(start)f(additional)j(pro)q (cesses.)131 1896 y(2.)22 b Fd(Start)12 b(tasks)f(on)h(allo)q(cated/di) q(sco)o(v)o(ered)g(resources)p Fs(,)e(obtaining)h(one)f(or)g(more)f Fk(MPI)p 1742 1896 V 16 w(Pro)q(cess)189 1952 y Fs(ob)s(jects,)14 b(whic)o(h)i(represen)o(t)f(running)h(pro)q(cesses)g(but)f(don't)g (allo)o(w)g(comm)o(unication.)131 2046 y(3.)22 b Fd(Establish)16 b(MPI)e(comm)o(unication)i(with)f(or)g(b)q(et)o(w)o(een)f(newly)h (created)g(pro)q(cesses)d Fs(b)o(y)189 2103 y(obtaining)k(a)f(comm)o (unicator)f(whose)h(group)g(con)o(tains)h(the)f(new)g(pro)q(cesses.)75 2262 y Fm(1.3.1)49 b(The)16 b(MPI)p 409 2262 15 2 v 18 w(Resource)g(Object)75 2348 y Fs(A)23 b(request)g(to)f(the)g(resource)h (manager)f(returns)h(an)g Fk(MPI)p 1144 2348 14 2 v 15 w(Resource)h Fs(ob)s(ject.)42 b(An)23 b Fk(MPI)p 1693 2348 V 16 w(Resource)75 2404 y Fs(represen)o(ts)18 b(a)g(set)g(of)g (computational)h(resources)f(on)g(whic)o(h)h(an)f(application)i(ma)o(y) e(start)f(pro)q(cesses.)75 2460 y(Dep)q(ending)23 b(on)e(the)g(request) g(that)f(generated)h(it,)i(an)e Fk(MPI)p 1163 2460 V 15 w(Resource)i Fs(ma)o(y)d(represen)o(t)h(a)g(ph)o(ysical)75 2517 y(CPU,)14 b(a)f(collection)j(of)e(CPUs,)g(or)f(ev)o(en)i(a)f (promise)g(from)f(a)h(resource)g(manager)g(that)f(an)h(application)75 2573 y(is)22 b(allo)o(w)o(ed)g(to)e(spa)o(wn)i(one)f(or)g(more)g(pro)q (cesses)h(\(whose)f(ph)o(ysical)h(lo)q(cation)h(will)g(b)q(e)f (determined)75 2630 y(dynamically\).)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fs(8)951 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)375 691 y @beginspecial 90 @llx 451 @lly 518 @urx 696 @ury 2880 @rwi @setspecial %%BeginDocument: fig2.eps 1 setlinejoin /M { moveto } bind def /S { show } bind def /R { rmoveto } bind def /L { lineto } bind def /B { newpath 0 0 M 0 1 L 1 1 L 1 0 L closepath } bind def /CS { closepath stroke } bind def /S { /fixwidth exch def dup length /nchars exch def dup stringwidth pop fixwidth exch sub nchars div exch 0 exch ashow } def /bwproc { rgbproc dup length 3 idiv string 0 3 0 5 -1 roll { add 2 1 roll 1 sub dup 0 eq { pop 3 idiv 3 -1 roll dup 4 -1 roll dup 3 1 roll 5 -1 roll put 1 add 3 0 } { 2 1 roll } ifelse } forall pop pop pop } def systemdict /colorimage known not { /colorimage { pop pop /rgbproc exch def { bwproc } image } def } if 1 1 scale 0 setlinewidth /drawtri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def 0 setgray newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath stroke } bind def /filltri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath fill } bind def /cliptri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def 0 setgray newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath clip } bind def /imgscanrgb { gsave translate /scandy exch def /scandx exch def /istr scandx 3 mul string def scandx scandy scale scandx scandy 8 [scandx 0 0 scandy neg 0 scandy] {currentfile istr readhexstring pop} false 3 colorimage grestore } bind def /imgscanbw { gsave translate /scandy exch def /scandx exch def /istr scandx string def scandx scandy scale scandx scandy 8 [scandx 0 0 scandy neg 0 scandy] {currentfile istr readhexstring pop} image grestore } bind def /showcaseisoencoding [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /space /exclam /quotedbl /numbersign /dollar /percent /ampersand /quoteright /parenleft /parenright /asterisk /plus /comma /minus /period /slash /zero /one /two /three /four /five /six /seven /eight /nine /colon /semicolon /less /equal /greater /question /at /A /B /C /D /E /F /G /H /I /J /K /L /M /N /O /P /Q /R /S /T /U /V /W /X /Y /Z /bracketleft /backslash /bracketright /asciicircum /underscore /quoteleft /a /b /c /d /e /f /g /h /i /j /k /l /m /n /o /p /q /r /s /t /u /v /w /x /y /z /braceleft /bar /braceright /asciitilde /guilsinglright /fraction /florin /quotesingle /quotedblleft /guilsinglleft /fi /fl /endash /dagger /daggerdbl /bullet /quotesinglbase /quotedblbase /quotedblright /ellipsis /trademark /dotlessi /grave /acute /circumflex /tilde /macron /breve /dotaccent /dieresis /perthousand /ring /cedilla /Ydieresis /hungarumlaut /ogonek /caron /emdash /exclamdown /cent /sterling /currency /yen /brokenbar /section /dieresis /copyright /ordfeminine /guillemotleft /logicalnot /hyphen /registered /macron /degree /plusminus /twosuperior /threesuperior /acute /mu /paragraph /periodcentered /cedilla /onesuperior /ordmasculine /guillemotright /onequarter /onehalf /threequarters /questiondown /Agrave /Aacute /Acircumflex /Atilde /Adieresis /Aring /AE /Ccedilla /Egrave /Eacute /Ecircumflex /Edieresis /Igrave /Iacute /Icircumflex /Idieresis /Eth /Ntilde /Ograve /Oacute /Ocircumflex /Otilde /Odieresis /multiply /Oslash /Ugrave /Uacute /Ucircumflex /Udieresis /Yacute /Thorn /germandbls /agrave /aacute /acircumflex /atilde /adieresis /aring /ae /ccedilla /egrave /eacute /ecircumflex /edieresis /igrave /iacute /icircumflex /idieresis /eth /ntilde /ograve /oacute /ocircumflex /otilde /odieresis /divide /oslash /ugrave /uacute /ucircumflex /udieresis /yacute /thorn /ydieresis ] def /showcasedingbatencoding [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /a1 /a2 /a202 /a3 /a4 /a5 /a119 /a118 /a117 /a11 /a12 /a13 /a14 /a15 /a16 /a105 /a17 /a18 /a19 /a20 /a21 /a22 /a23 /a24 /a25 /a26 /a27 /a28 /a6 /a7 /a8 /a9 /a10 /a29 /a30 /a31 /a32 /a33 /a34 /a35 /a36 /a37 /a38 /a39 /a40 /a41 /a42 /a43 /a44 /a45 /a46 /a47 /a48 /a49 /a50 /a51 /a52 /a53 /a54 /a55 /a56 /a57 /a58 /a59 /a60 /a61 /a62 /a63 /a64 /a65 /a66 /a67 /a68 /a69 /a70 /a71 /a72 /a73 /a74 /a203 /a75 /a204 /a76 /a77 /a78 /a79 /a81 /a82 /a83 /a84 /a97 /a98 /a99 /a100 /.notdef /a205 /a85 /a206 /a86 /a87 /a88 /a89 /a90 /a91 /a92 /a93 /a94 /a95 /a96 /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /a101 /a102 /a103 /a104 /a106 /a107 /a108 /a112 /a111 /a110 /a109 /a120 /a121 /a122 /a123 /a124 /a125 /a126 /a127 /a128 /a129 /a130 /a131 /a132 /a133 /a134 /a135 /a136 /a137 /a138 /a139 /a140 /a141 /a142 /a143 /a144 /a145 /a146 /a147 /a148 /a149 /a150 /a151 /a152 /a153 /a154 /a155 /a156 /a157 /a158 /a159 /a160 /a161 /a163 /a164 /a196 /a165 /a192 /a166 /a167 /a168 /a169 /a170 /a171 /a172 /a173 /a162 /a174 /a175 /a176 /a177 /a178 /a179 /a193 /a180 /a199 /a181 /a200 /a182 /.notdef /a201 /a183 /a184 /a197 /a185 /a194 /a198 /a186 /a195 /a187 /a188 /a189 /a190 /a191 /.notdef ] def /Courier-Bold findfont dup length dict begin {1 index /FID ne {def} {pop pop} ifelse} forall /Encoding showcaseisoencoding def currentdict end /Courier-Bold-SHOWISO exch definefont pop /pmask10 { <18182424424281811818242442428181 18182424424281811818242442428181> } def /newfont 10 dict def newfont begin /FontType 3 def /FontMatrix [1 0 0 1 0 0] def /FontBBox [0 0 1 1] def /Encoding 256 array def 0 1 255 {Encoding exch /.notdef put} for Encoding 42 /patchar10 put /CharProcs 2 dict def CharProcs begin /.notdef {} def /patchar10 { 16 16 true [16.08 0 0 -16.08 0 16.08] pmask10 imagemask } def end /BuildChar { 1 0 0 0 1 1 setcachedevice exch begin Encoding exch get CharProcs exch get end exec } def end /PatternFont newfont definefont pop /#copies 1 def gsave gsave matrix currentmatrix [117 0 0 -126 90 648] concat B setmatrix 0 0 0 setrgbcolor 2.000000 setlinewidth gsave stroke grestore grestore gsave gsave matrix currentmatrix [1 0 0 1 91.7 654] concat newpath 0 0 M 0 42 L 77.6 42 L 77.6 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 89.2 651.5 M 89.2 698.5 L 171.8 698.5 L 171.8 651.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 91.7 654] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 31.5 M (Allocate/) 75.6 S 0 17.5 M (Discover) 67.2 S 0 3.5 M (Resources) 75.6 S setmatrix grestore gsave matrix currentmatrix [117 0 0 -126 247.5 648] concat B setmatrix 0 0 0 setrgbcolor 2.000000 setlinewidth gsave stroke grestore grestore gsave gsave matrix currentmatrix [1 0 0 1 249.2 654.5] concat newpath 0 0 M 0 28 L 77.6 28 L 77.6 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 246.7 652 M 246.7 685 L 329.3 685 L 329.3 652 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 249.2 654.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (Start) 42 S 0 3.5 M (Processes) 75.6 S setmatrix grestore gsave matrix currentmatrix [117 0 0 -126 400.5 648] concat B setmatrix 0 0 0 setrgbcolor 2.000000 setlinewidth gsave stroke grestore grestore gsave gsave matrix currentmatrix [1 0 0 1 402.2 654.5] concat newpath 0 0 M 0 28 L 111.2 28 L 111.2 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 399.7 652 M 399.7 685 L 515.9 685 L 515.9 652 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 402.2 654.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (Establish ) 84 S 0 3.5 M (Communication) 109.2 S setmatrix grestore gsave 0 0 0 setrgbcolor 2.000000 setlinewidth newpath stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 274.5 523 translate 90 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 180 522 M 180 468 L 274.5 468 L 274.5 504 L stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 441 523 translate 90 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 337.5 522 M 337.5 495 L 337.5 468 L 441 468 L 441 504 L stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 179 451] concat newpath 0 0 M 0 14 L 102.8 14 L 102.8 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 176.5 448.5 M 176.5 467.5 L 284.3 467.5 L 284.3 448.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 179 451] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 3.5 M (MPI_Resource) 100.8 S setmatrix grestore gsave gsave matrix currentmatrix [1 0 0 1 341 451] concat newpath 0 0 M 0 14 L 94.4 14 L 94.4 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 338.5 448.5 M 338.5 467.5 L 437.9 467.5 L 437.9 448.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 341 451] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 3.5 M (MPI_Process) 92.4 S setmatrix grestore gsave gsave matrix currentmatrix [1 0 0 1 98 585] concat newpath 0 0 M 0 42 L 102.8 42 L 102.8 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 95.5 582.5 M 95.5 629.5 L 203.3 629.5 L 203.3 582.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 98 585] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 31.5 M (MPI_Resource) 100.8 S 0 17.5 M ( _Discover) 92.4 S 0 3.5 M ( _Allocate) 92.4 S setmatrix grestore gsave gsave matrix currentmatrix [1 0 0 1 255.5 599] concat newpath 0 0 M 0 28 L 94.4 28 L 94.4 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 253 596.5 M 253 629.5 L 352.4 629.5 L 352.4 596.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 255.5 599] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (MPI_Process) 92.4 S 0 3.5 M ( _Start) 67.2 S setmatrix grestore gsave gsave matrix currentmatrix [1 0 0 1 413 613] concat newpath 0 0 M 0 14 L 86 14 L 86 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 410.5 610.5 M 410.5 629.5 L 501.5 629.5 L 501.5 610.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 413 613] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 3.5 M (MPI_Attach) 84 S setmatrix grestore grestore showpage %%EndDocument @endspecial 552 788 a Fs(Figure)g(1.2:)k(Dynamic)d(Pro)q(cess)f (Managemen)o(t)166 922 y(Within)g(MPI,)e(an)g Fk(MPI)p 581 922 14 2 v 16 w(Resource)i Fs(is)f(simply)h(a)e(con)o(tainer)h(of)f (pro)q(cess)h(slots.)20 b(Pro)q(cess)13 b(slots)h(ma)o(y)75 978 y(either)i(b)q(e)f(\014lled)i(\(b)o(y)d(a)h(single)h(running)g(pro) q(cess\))e(or)h(empt)o(y)l(.)k(The)c(con)o(tainer)g(should)h(b)q(e)f (though)o(t)f(of)75 1035 y(as)g(a)f(bag)h(b)q(ecause)h(there)f(is)g(no) g(in)o(ternal)h(structure)f(visible)i(to)d(an)h(application)i(\(i.e.,)d (if)i(it)f(represen)o(ts)75 1091 y(m)o(ultiple)i(CPUs,)e(there)g(is)h (no)f(w)o(a)o(y)f(for)h(an)g(application)i(to)d(tell)j(on)e(whic)o(h)h (of)f(those)g(CPUs)g(a)g(pro)q(cess)75 1148 y(is)k(running\).)29 b(An)18 b Fk(MPI)p 498 1148 V 16 w(Resource)h Fs(is)f(generally)h (reusable,)g(as)e(it)h(represen)o(ts)g(a)g(committmen)o(t)f(from)75 1204 y(an)e(external)h(resource)f(manager)g(that)f(resources)h(are)g(a) o(v)m(ailable.)166 1337 y Fh(Discussion:)61 b Fg(One)21 b(big)f(hole)h(in)f(the)h(curren)o(t)i Ff(MPI)p 1088 1337 13 2 v 14 w(Resource)f Fg(description)f(is)g(that)g(o)o(wnership)g (of)75 1386 y(resources)16 b(is)e(v)n(aguely)f(de\014ned.)20 b(I)14 b(think)f(collectiv)o(e)h(o)o(wnership)h(is)f(unnatural)f(and)h (single-pro)q(cess)i(o)o(wnership)75 1436 y(is)g(m)o(uc)o(h)e(easier)j (to)e(understand.)25 b(Collectiv)o(e)15 b(o)o(wnership)h(do)q(esn't)h (mak)o(e)d(things)h(an)o(y)h(easier.)24 b(It)16 b(just)g(means)75 1486 y(that)e(all)f(op)q(erations)h(are)g(collectiv)o(e)g(-)g(not)f (that)h(an)o(y)g(pro)q(cess)h(can)g(start)f(c)o(hildren.)166 1536 y(This)g(lea)o(v)o(es)g(t)o(w)o(o)f(problems)134 1611 y(1.)22 b(Can)13 b(w)o(e)i(transfer)g(resource)h(o)o(wnership?)j (It)14 b(ma)o(y)e(b)q(e)j(clean)f(to)g(transfer)h(it)f(to)f(c)o (hildren)i(but)f(otherwise)189 1660 y(I)f(don't)h(see)h(a)f(nice)g(in)o (terface.)134 1735 y(2.)22 b(Who)15 b(o)o(wns)h(the)h(resources)h(used) f(b)o(y)f(initial)e(pro)q(cesses)19 b(and)d(preallo)q(cated)g (resources?)27 b(My)16 b(ten)o(tativ)o(e)189 1785 y(prop)q(osals)11 b(are)h(that)f(a)g(pro)q(cess)i(o)o(wns)e(its)g(o)o(wn)g(resources)j (\(un)o(til)c(transferred)j(someho)o(w\))d(and)h(that)h(no)q(de)189 1835 y(0)h(o)o(wns)h(all)f(the)h(preallo)q(cated)g(resources)j(\(yuc)o (k!\).)166 1909 y(Another)h(issue)f(is)g(what)g(happ)q(ens)h(to)f (resources)i(that)e(expire.)28 b(I)17 b(don't)g(think)f(this)h(is)g(to) q(o)g(m)o(uc)o(h)f(of)g(a)75 1959 y(problem,)c(esp.)19 b(if)13 b(resources)j(aren't)e(collectiv)o(e.)k(-w)o(cs)166 2009 y(The)f(uses)h(of)e(o)o(wnership)h(\(eg.)27 b(y)o(ou)17 b(can)g(only)f(free)h(resources)i(y)o(ou)e(o)o(wn\))f(needs)i(to)f(b)q (e)g(sp)q(elled)g(out.)27 b(I)75 2059 y(opp)q(ose)14 b(an)o(ything)f(fancy)h(with)f(resp)q(ect)k(to)c(o)o(wnership)h(\(e.g.) k(transfer)d(of)e(it)h(or)g(collectiv)o(e)f(o)o(wnership\))i(on)e(the) 75 2109 y(grounds)h(that)g(it)f(will)f(b)q(e)j(to)q(o)e(dep)q(enden)o (t)j(on)d(the)i(capabilities)d(of)i(the)g(\(unkno)o(wn\))g(resource)h (manager.)i(-)c(RL)166 2248 y Fs(The)19 b(notion)h(of)f Fl(r)n(esour)n(c)n(e)f Fs(is)i(delib)q(erately)h(\015exible.)35 b(Most)18 b(applications)j(do)e(not)g(care)g(where)75 2304 y(pro)q(cesses)h(are)f(spa)o(wned,)i(as)e(long)h(as)g(there)f(is)i (one)e(pro)q(cess)h(p)q(er)h(CPU.)e(F)l(or)g(these)h(applications,)75 2361 y(a)e(single)i(resource)e(ob)s(ject)g(con)o(taining)h(sev)o(eral)f (CPUs)g(ma)o(y)g(b)q(e)g(appropriate.)30 b(Other)18 b(appications)75 2417 y(will)g(w)o(an)o(t)c(explicit)19 b(con)o(trol)c(o)o(v)o(er)g (whic)o(h)i(pro)q(cesses)f(go)g(on)g(whic)o(h)g(mac)o(hine.)23 b(In)17 b(this)f(case,)g(a)g(single)75 2474 y(resource)i(ob)s(ject)e (ma)o(y)h(corresp)q(ond)h(to)f(a)g(single)i(mac)o(hine.)27 b(Similarly)l(,)20 b(it)e(migh)o(t)f(b)q(e)i(con)o(v)o(enien)o(t)f(to) 75 2530 y(think)e(of)f(an)g(SMP)g(as)g(a)g(single)h(\\no)q(de")f(or)g (as)g(a)g(collection)i(of)e(\(virtual\))g(pro)q(cessors.)166 2587 y(Within)c(MPI,)f(ho)o(w)o(ev)o(er,)g(a)g(resource)h(is)g(nothing) f(more)g(than)h(a)f(bag)g(of)f(pro)q(cess)i(slots.)18 b(Additional)75 2643 y(information)13 b(ab)q(out)f(what)g(ph)o(ysical)i (resources)f(the)g(pro)q(cesses)g(will)h(use)f(ma)o(y)f(b)q(e)h(kno)o (wn)g(to)f(and)g(used)75 2699 y(b)o(y)j(the)g(application,)i(but)e(is)h (not)f(imp)q(ortan)o(t)g(to)f(MPI.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fn(1.3.)33 b(NEW)15 b(OBJECTS)g(F)o(OR)g (IMPLEMENTING)g(D)o(YNAMIC)f(PR)o(OCESS)h(MANA)o(GEMENT)p Fs(9)166 45 y(Three)g(n)o(um)o(b)q(ers)h(c)o(haracterize)f(an)g Fk(MPI)p 880 45 14 2 v 16 w(Resource)p Fs(.)131 139 y(1.)22 b(Num)o(b)q(er)15 b(of)g(running)h(pro)q(cesses)131 233 y(2.)22 b(Num)o(b)q(er)15 b(of)g(pro)q(cess)g(slots)h(\(\014lled)g(and) g(empt)o(y\))e(recommended)131 327 y(3.)22 b(Num)o(b)q(er)15 b(of)g(pro)q(cess)g(slots)h(\(\014led)g(and)f(empt)o(y\))g(total)166 420 y(It)e(is)g(necessary)g(to)g(distinguish)i(b)q(et)o(w)o(een)e(the)g (total)f(n)o(um)o(b)q(er)h(of)g(pro)q(cess)g(slots)g(and)g(the)g(n)o (um)o(b)q(er)75 477 y(of)j(pro)q(cesses)i(recommended)f(for)f(a)h(giv)o (en)g(resource.)25 b(F)l(or)16 b(instance,)i(if)f(a)g(resource)f (corresp)q(onds)i(to)75 533 y(a)f(Unix)i(w)o(orkstation,)e(it)h(ma)o(y) f(theoreticaly)h(run)g(as)g(man)o(y)f(pro)q(cesses)h(as)f(Unix)i(can)f (supp)q(ort.)27 b(On)75 590 y(the)17 b(other)g(hand,)g(it)g(should)h (probably)g(run)f(at)f(most)g(one)h(pro)q(cess)g(p)q(er)h(pro)q (cessor.)25 b(W)l(e)17 b(could)h(giv)o(e)75 646 y(it)e(one)g(pro)q (cess)h(slot,)e(but)i(some)e(applications)j(will)g(need)e(to)g(b)q(e)g (able)h(to)f(do)g(more.)21 b(W)l(e)c(don't)e(w)o(an)o(t)75 703 y(to)j(tell)h(the)g(user,)g(ho)o(w)o(ev)o(er,)f(that)g(there)h(are) f(100)g(slots)g(a)o(v)m(ailable)i(b)q(ecause)g(there)e(will)i(b)q(e)g (no)e(w)o(a)o(y)75 759 y(to)d(distinguish)i(this)f(resource)f(from)g (one)g(with)h(100)e(CPUs.)20 b(W)l(e)c(ma)o(y)e(ha)o(v)o(e)h(an)h(SMP)l (,)e(but)i(our)f(MPI)75 816 y(pro)q(cesses)g(ma)o(y)f(b)q(e)h(m)o (ultithreaded)g(pro)q(cesses,)g(so)f(that)g(w)o(e)g(still)i(w)o(an)o(t) d(only)i(one)g(pro)q(cess)g(p)q(er)g(no)q(de.)75 872 y(Th)o(us)c(\\recommended)i(pro)q(cesses")e(do)q(esn't)h(alw)o(a)o(ys)e (mean)i(\\n)o(um)o(b)q(er)g(of)f(CPUs.")18 b(Most)10 b(applications)75 928 y(can)19 b(lo)q(ok)g(at)f(recommended)i(pro)q (cesses)f(and)h(ignore)f(max)f(pro)q(cesses.)31 b(Finally)l(,)22 b(for)c(debugging,)i(a)75 985 y(user)d(migh)o(t)f(w)o(an)o(t)f(to)h (\\fo)q(ol")g(an)g(application)i(b)o(y)e(suggesting)h(that)e(it)i (start)e(m)o(ultiple)j(pro)q(cesses)f(on)75 1041 y(a)e(single)h(w)o (orkstation.)166 1180 y Fh(Discussion:)32 b Fg(Suggestions)13 b(for)f(an)h(alternativ)o(e)f(name)f(to)i(\\recommended")e(are)i(w)o (elcome.)j(Thanks)d(to)75 1237 y(Ian)f(F)m(oster)g(for)g(making)d (clear)j(the)h(need)f(to)g(distinguish)f(b)q(et)o(w)o(een)j(max/recomm) o(ended)c(\(hop)q(e)i(I)g(got)f(it)h(righ)o(t!\))75 1293 y(-)i(w)o(cs)166 1433 y Fs(The)h(follo)o(wing)h(routines)g(can)f(b)q(e) h(used)g(to)e(access)i(information)f(ab)q(out)g(an)g Fk(MPI)p 1589 1433 V 16 w(Resource)p Fs(.)75 1583 y Fk(MPI)p 160 1583 V 16 w(RESOURCE)p 406 1583 V 18 w(NSLOTS\(resource,)h (nslots\))117 1661 y Fg(IN)155 b Fk(resource)432 b Fg(an)14 b Ff(MPI)p 1040 1661 13 2 v 14 w(Resource)h Fg(ob)r(ject)117 1736 y(OUT)108 b Fk(nslots)480 b Fg(n)o(um)o(b)q(er)13 b(of)h(slots)166 1860 y Fk(nslots)21 b Fs(giv)o(es)f(the)h(n)o(um)o(b)q (er)f(of)g(recommended)h(pro)q(cesses)f(in)h(a)f(resource.)34 b(This)21 b(n)o(um)o(b)q(er)f(will)75 1917 y(alw)o(a)o(ys)14 b(b)q(e)i(a)f(\014nite)h(non-negativ)o(e)g(in)o(teger)f(\(it)h(ma)o(y)e (b)q(e)i(zero,)f(if)g(resources)g(ha)o(v)o(e)g(expired\).)75 2068 y Fk(MPI)p 160 2068 14 2 v 16 w(RESOURCE)p 406 2068 V 18 w(NSLOTS)p 595 2068 V 17 w(TOT)l(AL\(resource,)g(nslots\))117 2145 y Fg(IN)155 b Fk(resource)432 b Fg(an)14 b Ff(MPI)p 1040 2145 13 2 v 14 w(Resource)h Fg(ob)r(ject)117 2220 y(OUT)108 b Fk(nslots)480 b Fg(n)o(um)o(b)q(er)13 b(of)h(slots)166 2344 y Fk(nslots)i Fs(giv)o(es)g(the)f(total)g(n)o(um)o(b)q(er)g(of)g (pro)q(cess)g(slots)g(in)h(a)f(resource.)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fs(10)928 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fk(MPI)p 160 45 14 2 v 16 w(RESOURCE)p 406 45 V 18 w(NPROCS)p 601 45 V 18 w(RUNNING\(resource,)h(np)o(ro)q(cs\))117 122 y Fg(IN)155 b Fk(resource)432 b Fg(an)14 b Ff(MPI)p 1040 122 13 2 v 14 w(Resource)h Fg(ob)r(ject)117 197 y(OUT)108 b Fk(np)o(ro)q(cs)465 b Fg(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i (curren)o(tly)f(running)75 416 y Fk(MPI)p 160 416 14 2 v 16 w(RESOURCE)p 406 416 V 18 w(NAME\(resource,)g(name\))117 494 y Fg(IN)155 b Fk(resource)432 b Fg(an)14 b Ff(MPI)p 1040 494 13 2 v 14 w(Resource)h Fg(ob)r(ject)117 569 y(OUT)108 b Fk(name)485 b Fg(a)14 b(string)g(iden)o(tifying)e(the)j (resource)166 693 y Fs(The)f(v)m(alue)h(of)e Fk(name)h Fs(only)g(suggested)g(b)o(y)f(the)h(MPI)g(standard.)19 b(If)14 b(the)g(resource)g(corresp)q(onds)g(to)75 750 y(a)g(particular)h(host,)e Fk(name)h Fs(should)h(b)q(e)g(the)f(name)g (of)g(that)g(host.)19 b(If)14 b(the)g(resource)h(w)o(as)e(obtained)i (from)75 806 y(a)h(request)g(for)f(m)o(ultiple)j(hosts)d(of)h(a)f(giv)o (en)i(arc)o(hitecture,)f Fk(name)f Fs(should)i(con)o(tain)f(the)g(name) g(of)g(that)75 863 y(arc)o(hitecture.)28 b(If)18 b(the)g(resource)g(w)o (as)f(obtained)h(from)f(a)h(complicated)h(resource)f(description)h (string)75 919 y(\(opaque)14 b(to)g(MPI)h(but)g(sen)o(t)f(to)g(an)g (external)h(resource)g(manager\))f Fk(name)g Fs(ma)o(y)g(b)q(e)h(that)f (string.)19 b Fk(name)75 975 y Fs(is)d(primarily)g(useful)g(when)g (disco)o(v)o(ering)g(pre-allo)q(cated)h(resources.)75 1126 y Fk(MPI)p 160 1126 14 2 v 16 w(RESOURCE)p 406 1126 V 18 w(DESCRIPTION\(resource,)e(description\))117 1204 y Fg(IN)155 b Fk(resource)432 b Fg(an)14 b Ff(MPI)p 1040 1204 13 2 v 14 w(Resource)h Fg(ob)r(ject)117 1279 y(OUT)108 b Fk(description)384 b Fg(a)14 b(string)g(describing)g(the)h(resource) 166 1403 y Fs(The)c(v)m(alue)h(of)e Fk(description)i Fs(is)f(only)h(suggested)e(b)o(y)h(the)g(MPI)f(standard.)18 b(It)11 b(describ)q(es)h(the)f(resource)75 1460 y(in)17 b(a)e(w)o(a)o(y)g(that)g(is)h(opaque)g(to)f(MPI)h(itself,)g(but)g(ma)o (y)f(b)q(e)i(understo)q(o)q(d)f(b)o(y)g(the)g(resource)f(manager)g(or) 75 1516 y(application.)21 b(Its)15 b(primary)f(use)h(is)g(in)g (conjunction)g(with)g Fk(MPI)p 1174 1516 14 2 v 16 w(Resource)p 1358 1516 V 17 w(discover)g Fs(if)g(a)f(user)g(needs)i(to)75 1573 y(get)f(more)g(detailed)h(information)g(ab)q(out)f(a)g(resource.) 75 1724 y Fk(MPI)p 160 1724 V 16 w(RESOURCE)p 406 1724 V 18 w(PROCESSES\(resource,)i(np)o(ro)q(cs,)f(a)o(rra)o(y)p 1122 1724 V 14 w(of)p 1173 1724 V 16 w(p)o(ro)q(cesses\))117 1801 y Fg(IN)155 b Fk(resource)432 b Fg(an)14 b Ff(MPI)p 1040 1801 13 2 v 14 w(Resource)h Fg(ob)r(ject)117 1876 y(OUT)108 b Fk(np)o(ro)q(cs)465 b Fg(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(returned)g(in)d(arra)o(y)117 1951 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1951 14 2 v 15 w(of)p 468 1951 V 16 w(p)o(ro)q(cesses)251 b Fg(an)14 b(arra)o(y)f(of)h Ff(MPI)p 1196 1951 13 2 v 14 w(Pro)q(cess)g Fg(ob)r(jects)h(\(see)h(b)q (elo)o(w\))166 2158 y Fh(Discussion:)34 b Fg(Should)13 b Ff(np)o(ro)q(cs)i Fg(b)q(e)g(an)e(IN)h(v)n(ariable)f(as)h(w)o(ell,)f (to)h(sp)q(ecify)g(the)g(length)g(of)g(the)g(arra)o(y?)166 2380 y Fh(Discussion:)50 b Fg(Should)17 b(sev)o(eral)i(of)e(these)i (routines)g(b)q(e)f(com)o(bined)f(in)o(to)g(one,)i(whic)o(h)e(returns)j (lots)e(of)75 2436 y(information)11 b(ab)q(out)j(a)f(resource)j(in)e(a) f(single)h(call?)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fn(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(11)75 45 y Fm(1.3.2)49 b(The)16 b(MPI)p 409 45 15 2 v 18 w(Pro)q(cess)g(Object)75 207 y Fh(Discussion:)38 b Fg(In)15 b(MPI-1,)g(pro)q(cesses)j(can)d(b)q(e)h(iden)o(ti\014ed)f(b) o(y)g(their)g(rank)g(in)g(a)f(group)h(or)g(comm)o(unicator.)k(It)75 257 y(migh)o(t)12 b(b)q(e)k(p)q(ossible)f(to)f(con)o(tin)o(ue)h(to)f (use)i(a)e(\(group,rank\))g(pair)g(to)h(iden)o(tify)f(a)g(pro)q(cess)i (as)f(required)h(for)e(MPI-)75 307 y(2.)29 b(While)17 b(con)o(v)o(enien)o(t)h(for)g(some)f(op)q(erations)h(\(e.g.,)f(spa)o (wning)h(a)f(group)h(of)f(pro)q(cesses\),)k(the)d(\(group,rank\))75 356 y(iden)o(ti\014cation)d(w)o(ould)g(b)q(e)h(incon)o(v)o(enien)o(t)f (or)h(unnatural)f(for)g(others)h(\(e.g.,)f(sending)h(a)f(signal\).)22 b(Moreo)o(v)o(er,)16 b(an)75 406 y Ff(MPI)p 152 406 13 2 v 14 w(Group)c Fg(ob)r(ject)h(pro)o(vides)e(an)h(abstraction)f(for)g (a)h(collection)f(of)g(pro)q(cesses)j(whic)o(h)d(is)h(an)f(unnatural)g (starting)75 456 y(p)q(oin)o(t)h(to)g(de\014ne)i(a)e(single)g(pro)q (cess.)19 b(Therefore,)14 b(in)e(the)h(follo)o(wing)d(w)o(e)j(in)o(tro) q(duce)g(a)f(new)h(ob)r(ject,)g Ff(MPI)p 1720 456 V 14 w(Pro)q(cess)p Fg(.)75 506 y(When)f(the)g(b)q(eha)o(vior)f(of)f(this)i (ob)r(ject)g(is)f(agreed)h(up)q(on)g(and)f(understo)q(o)q(d,)i(w)o(e)e (can)h(decide)g(among)e(three)i(options)134 581 y(1.)22 b(Absorb)14 b Ff(MPI)p 410 581 V 14 w(Pro)q(cess)h Fg(in)o(to)e Ff(MPI)p 729 581 V 14 w(Group)134 655 y Fg(2.)22 b(Ha)o(v)o(e)13 b(t)o(w)o(o)h(separate)h(ob)r(jects)g(\(with)f(p)q(erhaps)h(additional) d(accessor/creation)k(functions\))134 730 y(3.)22 b(Go)13 b(with)g Ff(MPI)p 427 730 V 15 w(Pro)q(cess)h Fg(and)g(deprecate)h Ff(MPI)p 929 730 V 15 w(Group)166 894 y Fs(An)g Fk(MPI)p 325 894 14 2 v 16 w(Pro)q(cess)h Fs(represen)o(ts)f(a)g(pro)q(cess)h({) f(a)g(program)f(executing)i(on)f(a)g(pro)q(cessor.)20 b(It)15 b(ma)o(y)f(or)75 950 y(ma)o(y)d(not)f(b)q(e)i(\\registered")f (with)h(MPI;)f(it)g(ma)o(y)g(or)g(ma)o(y)f(not)h(call)h Fk(MPI)p 1289 950 V 16 w(INIT)p Fs(.)e(Y)l(ou)i(can't)f(comm)o(unicate) 75 1007 y(directly)18 b(with)e(an)h Fk(MPI)p 496 1007 V 15 w(Pro)q(cess)h Fs(\(that)d(requires)i(a)f(comm)o(unicator\),)g (but)g(y)o(ou)g(can)h(send)f(signals)i(to)75 1063 y(one.)28 b(If)18 b(it)g(is)g(an)f(MPI)h(pro)q(cess)g(in)h(the)f(sense)g(that)f (it)h(has)f(a)h(rank)f(in)i(a)e(comm)o(unicator,)h(then)g(the)75 1120 y Fk(MPI)p 160 1120 V 16 w(Pro)q(cess)13 b Fs(ob)s(ject)e (represen)o(ting)i(it)g(can)f(b)q(e)h(retriev)o(ed)g(from)e(its)h(rank) g(in)h(that)f(comm)o(unicator)g(\(See)75 1176 y Fk(MPI)p 160 1176 V 16 w(COMM)p 318 1176 V 16 w(PROCESS)i Fs(b)q(elo)o(w\).)19 b(MPI)13 b(pro)o(vides)f(no)h(guaran)o(tees)e(on)i(the)f(order)g(of)g (op)q(erations)h(with)75 1233 y(resp)q(ect)e(to)f(an)h Fk(MPI)p 421 1233 V 15 w(PROCESS)i Fs(and)d(pro)o(vides)h(none)g(of)g (the)f(safet)o(y)g(asso)q(ciated)h(with)g(comm)o(unicators.)75 1289 y(In)16 b(particular,)f(there)h(is)f(no)g(guaran)o(tee)g(of)g (order)g(b)q(et)o(w)o(een)g(messages)g(and)g(signals.)166 1428 y Fh(Discussion:)34 b Fg(A)14 b(bad)g(programming)d(practice)k (that)f(has)g(b)q(een)h(seen)g(b)o(y)f(naiv)o(e)g(users)h(of)f(systems) g(that)75 1485 y(pro)o(vide)f(b)q(oth)g(messages)h(and)f(signals)f(is)i (to)f(send)h(a)f(pro)q(cess)i(a)e(message)g(and)g(then)h(signal)e(it)h (to)g(w)o(ak)o(e)g(up)g(and)75 1541 y(receiv)o(e)i(the)g(message.)i (This)d(needs)h(to)f(b)q(e)h(plainly)d(forbidden.)18 b(-)13 b(A)o(G.)75 1767 y Fp(1.4)59 b(Simple)20 b(Interface)75 1945 y Fh(Discussion:)37 b Fg(\(w)o(cs\))15 b(I'v)o(e)g(done)g(some)e (rethinking)i(of)f Ff(MPI)p 1067 1945 13 2 v 14 w(Spa)o(wn)p Fg(.)21 b(In)15 b(particular,)f(as)h(presen)o(ted)i(earlier)d(it)75 1994 y(b)q(oth)f(allo)q(cates)f(resources)j(and)e(spa)o(wns)g(pro)q (cesses.)20 b(This)13 b(is)f(to)q(o)g(m)o(uc)o(h.)17 b(Generally)12 b(sp)q(eaking,)g(resources)j(will)75 2044 y(already)g(b)q(e)h(allo)q(cated.)23 b(This)15 b(is)g(true)i(in)e(PVM.) g Fe(pvm)p 951 2044 14 2 v 15 w(spawn)g Fg(do)q(es)h(not)f(add)h(an)o (y)f(hosts.)23 b(It)16 b(only)e(spa)o(wns)i(on)75 2094 y(the)h(existing)f(virtual)f(mac)o(hine.)24 b(I)16 b(think)g(what)g(w)o (e)g(need)h(is)f(to)g(mak)o(e)f(the)i(\\simple")d(in)o(terface)j(a)f(t) o(w)o(o)g(stage)75 2144 y(pro)q(cess:)134 2219 y(1.)22 b(Disco)o(v)o(er)12 b(existing)g(resources)j(with)d Ff(MPI)p 853 2219 13 2 v 15 w(Resource)p 1021 2219 V 16 w(discover)i Fg(\(using)e(a)h(simple)d(form)h(of)h(the)h(subroutine)189 2268 y(to)g(get)i(a)e(single)h(resource\))134 2343 y(2.)22 b(Spa)o(wn)c(on)o(to)g(this)h(resource)i(with)d Ff(MPI)p 853 2343 V 15 w(Spa)o(wn)p Fg(.)33 b(This)19 b(will)e(b)q(e)j(a)e (single)g(call)g(that)h(will)e(b)q(oth)i(start)189 2393 y(pro)q(cesses)d(and)e(attac)o(h)g(to)g(them.)166 2468 y(This)9 b(approac)o(h)g(also)g(has)g(an)g(adv)n(an)o(tage)g(of)f (eliminating)f(the)j(\\other)f(args")g(a)o(wkw)o(ardness)h(from)e Ff(PVM)p 1804 2468 V 14 w(Spa)o(wn)p Fg(.)75 2517 y(Normally)h(these)14 b(w)o(ouldn't)d(b)q(e)h(used.)19 b(If)11 b(the)i(user)g(w)o(an)o(ts)f (the)h(extra)f(functionalit)o(y)f(\(i.e.,)g(to)h(spa)o(wn)g(on)f(sp)q (eci\014c)75 2567 y(hosts)k(or)e(arc)o(hitectures\))k(he/she)e(can)f (use)g(a)g(sligh)o(tly)f(more)f(complicated)h(form)f(of)h Ff(MPI)p 1513 2567 V 15 w(Resource)p 1681 2567 V 16 w(discover)p Fg(.)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fs(12)928 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 45 y Fs(In)23 b(this)f(section)h(w)o (e)f(pro)o(vide)h(high-lev)o(el)h(functions)f(b)q(oth)g(for)e(resource) i(disco)o(v)o(ery)l(,)h(pro)q(cess)75 102 y(creation)d(and)f(for)g (clien)o(t-serv)o(er)h(requiremen)o(ts)g(that)f(are)g(the)g(easiest)h (to)f(use.)36 b(These)20 b(pro)o(vide)h(a)75 158 y(useful)16 b(subset)e(of)g(op)q(erations;)h(routines)g(that)f(pro)o(vide)h(more)f (con)o(trol)g(and)h(\015exibilit)o(y)i(are)d(describ)q(ed)75 214 y(later.)27 b(Indeed,)19 b(the)f(functions)g(in)h(this)f(section)g (can)g(b)q(e)g(la)o(y)o(ered)g(on)f(those)g(routines,)i(describ)q(ed)g (in)75 271 y(Sections)d(1.4.1,)d(1.4.1,)g(and)j(1.4.1.)75 388 y Fk(Spa)o(wning)h(p)o(ro)q(cesses)47 b Fs(In)16 b(the)g(most)e(common)i(case,)f(an)g(application)j(will)f(simply)g(spa) o(wn)e(new)h(pro-)75 444 y(cesses)f(on)f(preallo)q(cated)i(resources.)k (This)15 b(is)g(a)f(t)o(w)o(o)f(step)i(pro)q(cess.)20 b(First)14 b(the)h(application)h(disco)o(v)o(ers)75 500 y(its)f(resources,)g(then)h(it)f(spa)o(wns)g(a)g(program.)166 557 y(Detailed)i(mec)o(hanisms)g(for)f(obtaining)h(resources)g(are)f (describ)q(ed)i(in)g(Section)f(1.4.1.)22 b(F)l(or)16 b(most)75 613 y(applications,)g(the)g(follo)o(wing)g(is)f(su\016cien)o (t:)147 694 y Fj(nresource=1)147 750 y(MPI_RESOURCE_DISCOVER\(M)o (PI_RESO)o(URCE_DEF)o(AULT,)21 b(resource,)h(nresource\))147 807 y(MPI_RESOURCE_NSLOTS\(res)o(ource,)e(nslots\))166 888 y Fs(This)15 b(pro)q(duces)h(a)f(resource)g(ob)s(ject)f(on)h(whic)o (h)g(to)g(spa)o(wn)f(pro)q(cesses)h(and)g(tells)h(the)f(application)75 944 y(ho)o(w)g(man)o(y)f(pro)q(cesses)i(it)f(ma)o(y)g(spa)o(wn.)166 1000 y(The)g(follo)o(wing)h(function)g(is)g(the)g(easiest)f(w)o(a)o(y)f (to)h(create)g(new)h(pro)q(cesses,)f(and)g(is)h(most)e(lik)o(e)j(the)75 1057 y(PVM)g(routine)g Fj(pvm)p 432 1057 15 2 v 17 w(spawn)p Fs(.)24 b(It)17 b(starts)f(the)h(same)f(executable)j(with)e(the)g(same) g(argumen)o(t)f(list)h(on)g(a)75 1113 y(set)c(of)g(pro)q(cessors.)19 b(It)13 b(is)h(a)f(collectiv)o(e)i(op)q(eration)e(o)o(v)o(er)g(the)g (pro)q(cesses)h(in)g(the)f(group)g(asso)q(ciated)h(with)75 1170 y Fk(oldcomm)p Fs(,)d(and)i(returns)g(an)g(in)o(ter-comm)o (unicator)f Fk(new)o(comm)p Fs(,)g(whic)o(h)i(has)e(the)h(new)g(pro)q (cesses)g(as)g(the)75 1226 y(remote)18 b(group.)30 b(If)19 b(an)f(expanded)i(comm)o(unicator)e(is)i(desired,)g(an)f(ordinary)f(in) o(tra-comm)o(unicator)75 1283 y(con)o(taining)12 b(all)f(pro)q(cesses)g (can)g(then)g(b)q(e)h(constructed)f(using)g Fk(MPI)p 1218 1283 14 2 v 16 w(INTERCOMM)p 1508 1283 V 17 w(MERGE)p Fs(.)g(Only)h(one)75 1339 y(of)17 b(the)h(pro)q(cesses)h(in)f Fk(oldcomm)f Fs(m)o(ust)g(supply)i(the)f(argumen)o(ts)f(that)g(describ) q(e)j(the)e(new)g(pro)q(cesses;)75 1396 y(that)c(pro)q(cess)i(is)g (designated)g(b)o(y)f Fk(ro)q(ot)p Fs(.)75 1547 y Fk(MPI)p 160 1547 V 16 w(SP)l(A)-5 b(WN\(oldcomm,)13 b(resource,)i(count,)i (executable,)f(other)p 1185 1547 V 17 w(a)o(rgs,)e(new)o(comm\))117 1624 y Fg(IN)155 b Fk(oldcomm)413 b Fg(comm)o(unicator)11 b(of)j(spa)o(wning)f(group)117 1695 y(IN)155 b Fk(resource)432 b Fg(resource)16 b(to)e(spa)o(wn)g(on)o(to)117 1767 y(IN)155 b Fk(count)482 b Fg(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(to)e(spa)o (wn)g(\(in)o(t\))117 1838 y(IN)155 b Fk(executable)391 b Fg(executable)15 b(\014le)f(for)g(new)g(pro)q(cesses)j(to)c(execute)j (\(string\))117 1910 y(OUT)108 b Fk(new)o(comm)397 b Fg(new)21 b(in)o(ter-comm)o(unicator)d(including)i(new)h(pro)q(cesses)i (as)905 1966 y(the)15 b(remote)e(group)166 2173 y Fh(Discussion:)34 b Fg(Arg.)18 b(There)d(should)f(b)q(e)g(a)g(ro)q(ot)g(argumen)o(t)e(to) i(sp)q(ecify)h(the)f(o)o(wner)g(of)g(the)g(resource)166 2395 y Fh(Discussion:)h Fg(F)m(or)f(ease)g(of)f(use,)h(w)o(e)g(ma)o(y)e (w)o(an)o(t)h(to)h(create)h(a)e(regular)h(comm)o(unicator,)c(con)o (taining)j(b)q(oth)75 2452 y(new)j(and)f(old)g(pro)q(cesses,)j(so)e (that)g(casual)f(users)i(don't)e(ha)o(v)o(e)g(to)h(w)o(orry)f(ab)q(out) h(in)o(tercomm)o(unicators,)d(whic)o(h)75 2508 y(can)h(b)q(e)h (confusing.)166 2647 y Fs(The)20 b(spa)o(wned)f(pro)q(cesses)h(are)f (created)h(with)g Fk(MPI)p 1089 2647 V 15 w(COMM)p 1246 2647 V 17 w(W)o(ORLD)f Fs(consisting)h(of)g(the)f(pro-)75 2704 y(cesses)f(spa)o(wned)f(with)g(this)h(call,)g(and)g(in)g(addition) g(they)f(ha)o(v)o(e)g(a)g(prede\014ned)i(in)o(ter-comm)o(unicator)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fn(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(13)75 45 y Fk(MPI)p 160 45 14 2 v 16 w(COMM)p 318 45 V 16 w(P)l(ARENT)p Fs(,)17 b(in)g(whic)o(h)g(the)f (remote)f(group)h(consists)g(of)g(the)g(spa)o(wning)g(pro)q(cesses.)23 b(The)75 102 y(call)17 b(blo)q(c)o(ks)f(un)o(til)h(all)g(pro)q(cesses)f (in)g Fk(oldcomm)e Fs(ha)o(v)o(e)h(called)j Fk(MPI)p 1227 102 V 15 w(Spa)o(wn)f Fs(and)f(all)g(the)g(spa)o(wned)g(pro-)75 158 y(cesses)f(ha)o(v)o(e)f(called)i Fk(MPI)p 521 158 V 16 w(INIT)p Fs(,)d(after)h(whic)o(h)h(comm)o(unication)g(is)g(p)q (ossible.)22 b(\(W)l(e)14 b(treat)g(spa)o(wning)g(of)75 214 y(non-MPI)i(pro)q(cesses)g(b)q(elo)o(w.\))21 b(Note)16 b(that)f(this)h(means)f(that)g Fk(MPI)p 1252 214 V 16 w(COMM)p 1410 214 V 16 w(W)o(ORLD)h Fs(is)g(di\013eren)o(t)g(on)75 271 y(the)c(spa)o(wners)g(and)h(spa)o(wnees.)19 b(Also)12 b(note)g(that)g(b)o(y)g(using)h Fk(MPI)p 1194 271 V 16 w(COMM)p 1352 271 V 16 w(SELF)p Fs(,)f(a)g(single)h(pro)q(cess)g(can)75 327 y(create)h(man)o(y)g(others;)g(this)g(is)h(useful)h(b)q(oth)e(for)g (master-sla)o(v)o(e)f(programs)g(and)i(for)f(dynamic)h(sizing)g(of)75 384 y(parallel)f(jobs)e(from)f(a)h(single)i(initial)g(pro)q(cess.)19 b(This)13 b(function)g(is)g(similar)h(to)d(the)i(one)f(describ)q(ed)i (in)f([3],)75 440 y(but)k(here)f(the)h(argumen)o(ts)e(are)i(made)f (explicit)j(instead)e(of)f(b)q(eing)h(com)o(bined)h(in)f(a)f(single)i (string.)23 b(In)75 497 y(the)16 b(case)h(where)f(job-sc)o(heduler-dep) q(ende)q(n)o(t)i(information)f(m)o(ust)e(b)q(e)i(supplied,)i(w)o(e)d (do)g(use)g(a)g(string,)75 553 y(in)g(the)f(more)g(\015exible)i Fk(MPI)p 560 553 V 16 w(RESOURCE)p 806 553 V 19 w(IALLOCA)l(TE)e Fs(describ)q(ed)i(in)f(Section)g(1.2.1.)166 692 y Fh(Discussion:)33 b Fg(In)13 b(order)h(to)g(mak)o(e)d Ff(MPI)p 827 692 13 2 v 15 w(SP)m(A)-5 b(WN)13 b Fg(easy)g(to)g(use,)h(it)f(seems)h (easiest)g(to)f(ha)o(v)o(e)g Ff(MPI)p 1726 692 V 14 w(INIT)g Fg(b)q(e)75 749 y(the)h(corresp)q(onding)f(collectiv)o(e)g(op)q (eration)g(in)g(the)g(c)o(hildren.)18 b(The)c(usage)f(of)f Ff(MPI)p 1387 749 V 15 w(A)m(TT)m(A)o(CH)f Fg(\(Section)j(1.4.1\))d(is) 75 805 y(no)o(w)i(consisten)o(t)j(with)d(this.)18 b(-w)o(cs)166 1027 y Fh(Discussion:)45 b Fg(I)17 b(think)f(that)h(although)f(this)h (is)f(p)q(ossible,)i(it)e(is)h(not)f(desireable.)28 b(I)17 b(w)o(ould)e(prefer)j(the)75 1084 y(separate)c(call)e(to)h Ff(MPI)p 440 1084 V 15 w(COMM)p 585 1084 V 14 w(P)m(ARENT)p Fg(.)f(That)h(w)o(a)o(y)f(is)h(is)g(easy)g(to)g(read)g(the)h(co)q(de)g (for)e(the)i(spa)o(wnee)g(and)f(see)75 1140 y(easily)g(that)h(this)g (co)q(de)h(is)f(mean)o(t)e(to)i(b)q(e)h(spa)o(wned.)j({)c(RL)75 1343 y Fk(Client-Server)d(Applications)48 b Fs(The)11 b(ab)q(o)o(v)o(e)f(paragraphs)g(co)o(v)o(ered)g(the)h(case)g(of)f (creating)h(new)g(pro)q(cesses.)75 1399 y(F)l(or)17 b(clien)o(t-serv)o (er)i(applications,)h(the)e(situation)g(is)h(di\013eren)o(t,)f(b)q (ecause)h(the)f(pro)q(cesses)h(in)f(question)75 1456 y(already)h(exist,)g(and)f(what)g(w)o(e)g(need)h(is)f(a)g(comm)o (unicator)g(to)g(b)q(e)h(used)g(b)o(y)f(them)g(to)g(comm)o(unicate)75 1512 y(with)e(one)f(another.)166 1569 y(The)f(follo)o(wing)g(are)f(the) h(simplest,)h(blo)q(c)o(king)f(calls.)21 b(The)14 b(non-blo)q(c)o(king) h(v)o(ersions)f(are)f(describ)q(ed)75 1625 y(in)j(Section)g(1.4.2.)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fs(14)928 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fk(MPI)p 160 45 14 2 v 16 w(CLIENT)p 333 45 V 16 w(CONNECT\(mycomm,)d(name,)j(new)o (comm\))117 122 y Fg(IN)155 b Fk(mycomm)412 b Fg(comm)o(unicator)10 b(of)h(the)i(clien)o(t,)f(o)o(v)o(er)g(whic)o(h)h(this)f(call)f(is)h (col-)905 179 y(lectiv)o(e)117 254 y(IN)155 b Fk(name)485 b Fg(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)g(whic)o(h)h(the)h(serv)o(er)g (can)f(b)q(e)h(con)o(tacted)905 310 y(\(string\))117 385 y(OUT)108 b Fk(new)o(comm)397 b Fg(new)10 b(in)o(ter-comm)o (unicator,)d(whic)o(h)i(includes)h(the)g(serv)o(er)h(pro-)905 442 y(cesses)17 b(as)c(the)i(remote)e(group)75 661 y Fk(MPI)p 160 661 V 16 w(SERVER)p 343 661 V 18 w(CONNECT\(mycomm,)f (name,)j(new)o(comm\))117 738 y Fg(IN)155 b Fk(mycomm)412 b Fg(comm)o(unicator)18 b(of)i(the)h(serv)o(er,)j(o)o(v)o(er)c(whic)o (h)h(this)f(call)g(is)905 795 y(collectiv)o(e)117 870 y(IN)155 b Fk(name)485 b Fg(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)g(whic)o (h)h(the)h(serv)o(er)g(can)f(b)q(e)h(con)o(tacted)905 926 y(\(string\))117 1001 y(OUT)108 b Fk(new)o(comm)397 b Fg(new)10 b(in)o(ter-comm)o(unicator,)d(whic)o(h)i(includes)h(the)g (clien)o(t)f(pro-)905 1058 y(cesses)17 b(as)c(the)i(remote)e(group)166 1182 y Fs(Then)j(an)o(y)g(pro)q(cess)g(in)g(the)g(clien)o(t)h(group)e (can)h(comm)o(unicate)g(with)h(an)o(y)e(pro)q(cess)h(in)h(the)e(serv)o (er)75 1239 y(group)g(and)g(vice)h(v)o(ersa,)f(using)h(the)f(in)o (ter-comm)o(unicator.)166 1295 y(Disconnection)c(o)q(ccurs)g(when)g (pro)q(cesses)f(call)h Fk(MPI)p 1057 1295 V 16 w(COMM)p 1215 1295 V 17 w(FREE)f Fs(on)g(the)h(in)o(ter-comm)o(unicator.)166 1352 y(The)17 b(form)g(of)f(the)i Fk(name)f Fs(argumen)o(t)f(has)h(sev) o(eral)g(p)q(ossibilitie)q(s.)28 b(The)18 b(most)e(ob)o(vious)h(is)h (to)f(use)75 1408 y(the)c Fj(net-address:port-number)d Fs(format)h(that)h(curren)o(t)h(systems)f(will)j(\014nd)e(most)f (straigh)o(tforw)o(ard.)75 1464 y(Ho)o(w)o(ev)o(er,)i(in)i(the)f(long)h (run,)f(name)g(serv)o(ers)g(of)f(v)m(arious)i(kinds)g(ma)o(y)f(require) h(more)f(\015exibilit)o(y)l(.)75 1586 y Fm(1.4.1)49 b(Interfacing)16 b(to)h(the)e(Resource)g(Manager)75 1672 y Fs(This)h(section)g(describ)q (es)g(an)g(in)o(terface)f(for)g(obtaining)h(resources)f(from)f(a)h (resource)g(manager.)166 1728 y(There)e(are)f(t)o(w)o(o)f(w)o(a)o(ys)h (that)g(applications)i(t)o(ypically)g(use)f(new)g(resources.)19 b(The)13 b(\014rst)f(is)h(that)f(they)75 1785 y(mak)o(e)k(use)g(of)g (existing,)h(pre-allo)q(cated)g(resources.)23 b(F)l(or)16 b(instance,)h(in)g(PVM,)e(a)h(user)g(ma)o(y)g(con\014gure)75 1841 y(a)21 b(virtual)g(mac)o(hine,)i(and)e(start)e(up)j(a)e(master)g (pro)q(cess.)37 b(The)21 b(master)g(pro)q(cess)g(determines)g(ho)o(w)75 1898 y(man)o(y)c(hosts)g(are)g(a)o(v)m(ailable)i(and)f(spa)o(wns)f(sla) o(v)o(e)h(pro)q(cesses)g(on)f(the)h(un)o(used)g(hosts.)26 b(New)18 b(resources)75 1954 y(are)c(disco)o(v)o(ered,)g(not)g(allo)q (cated.)20 b(The)15 b(second)f(w)o(a)o(y)f(is)i(for)e(an)h(application) i(to)d(obtain)i(new)f(resources)75 2011 y(explicitly)l(.)23 b(In)16 b(PVM,)e(this)i(is)f(done)h(through)f(the)g Fj(pvm)p 1045 2011 15 2 v 17 w(addhosts)f Fs(routine.)166 2067 y(It)h(is)h(p)q(ossible)i(to)c(treat)h(these)h(in)g(the)f(same)g(w)o(a) o(y)l(,)g(using)h(a)f(single)i Fk(ALLOCA)l(TE)f Fs(function.)21 b(This)75 2124 y(requires)e(in)h(the)e(case)h(of)f(pre-allo)q(cated)i (resources)e(that)g(an)h(application)h(kno)o(w)e(in)h(adv)m(ance)h (what)75 2180 y(resources)14 b(are)h(a)o(v)m(ailable)h(so)e(that)g(it)g (kno)o(ws)g(what)g(to)g(allo)q(cate.)20 b(W)l(e)15 b(therefore)f (distinguish)j(b)q(et)o(w)o(een)75 2237 y(allo)q(cation)e(and)f(disco)o (v)o(ery)g(of)g(resources.)19 b(Resource)c(disco)o(v)o(ery)f(is)h(exp)q (ected)g(to)e(b)q(e)i(a)f(fast)f(op)q(eration)75 2293 y(that)k(ma)o(y)f(b)q(e)i(p)q(ossible)i(to)c(implemen)o(t)j(through)e (in)o(ternal)h(lo)q(okup)g(rather)f(than)g(external)h(comm)o(u-)75 2349 y(nication)e(\(dep)q(ending)h(on)e(the)g(MPI)g(implemen)o (tation\).)21 b(Resource)16 b(allo)q(cation,)g(on)f(the)g(other)g (hand,)75 2406 y(can)h(b)q(e)h(time-consuming,)g(and)f(is)h(therefore)f (a)f(non-blo)q(c)o(king)j(op)q(eration)e(whic)o(h)h(returns)f(a)g (request)75 2462 y(to)f(b)q(e)g(w)o(aited)h(on)f(later.)166 2519 y(Neither)h(function)f(starts)f(an)o(y)h(application-visibl)q(e)j (pro)q(cesses;)d(rather,)f(they)h(obtain)g(resources)75 2575 y(from)10 b(the)h(resource)g(manager)g(for)f(use)i(b)o(y)f(other)f (functions.)19 b(Ho)o(w)o(ev)o(er,)11 b(w)o(e)g(need)h(to)e(tak)o(e)g (in)o(to)h(accoun)o(t)75 2632 y(those)j(resource)g(managers)g(that)f (cannot)i(pro)o(vide)f(this)h(function)g(without)f(starting)g(pro)q (cesses,)h(suc)o(h)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fn(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(15)75 45 y(as)12 b(LoadLev)o(eler)i(or)f(DQS.)f(In)i (those)e(cases,)h(the)g(executables)h(ma)o(y)e(not)h(b)q(e)g(the)g (application)i(executa-)75 102 y(bles,)g(but)g(rather)f(in)o(terface)g (pro)q(cesses)h(that)f(will)i(create)e(the)h(application)h(pro)q (cesses)e(in)i(resp)q(onse)f(to)75 158 y(one)g(of)g(the)g(pro)q (cess-creation)h(functions)g(describ)q(ed)h(here)f(\(lik)o(e)g Fk(MPI)p 1303 158 14 2 v 16 w(Spa)o(wn)p Fs(\).)166 214 y(The)f(in)o(terface)g(to)f(an)h(external)g(resource)g(manager)f(could) i(b)q(e)f(arbitrarily)g(complex.)21 b(The)15 b(MPI)75 271 y(in)o(terface)20 b(sp)q(eci\014es)i(the)e(absolute)g(minim)o(um)i (amoun)o(t)d(of)g(information)h(that)g(m)o(ust)f(b)q(e)i(kno)o(wn)f(to) 75 327 y(MPI)15 b(and)h(tries)f(to)f(mak)o(e)h(common)g(cases)g(v)o (ery)g(simple)i(to)d(sp)q(ecify)l(.)75 445 y Fk(Resource)i(Discovery)75 625 y(MPI)p 160 625 V 16 w(RESOURCE)p 406 625 V 18 w(DISCO)o (VER\(resource)p 815 625 V 17 w(t)o(yp)q(e,)g(nresources,)g(a)o(rra)o (y)p 1260 625 V 15 w(of)p 1312 625 V 16 w(resources\))117 702 y Fg(IN)155 b Fk(resource)p 476 702 V 17 w(t)o(yp)q(e)335 b Fg(explained)14 b(b)q(elo)o(w)117 774 y(OUT)108 b Fk(nresources)392 b Fg(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(returned)j(in)d(arra)o(y) 117 847 y(OUT)108 b Fk(a)o(rra)o(y)p 416 847 V 15 w(of)p 468 847 V 16 w(resources)252 b Fg(arra)o(y)14 b(of)f Ff(MPI)p 1138 847 13 2 v 14 w(Resource)j Fg(ob)r(jects)166 1054 y Fh(Discussion:)37 b Fg(There's)15 b(the)h(usual)e(question)h(of) f(whether)i(nresources)h(should)d(b)q(e)i(an)e(IN/OUT)h(arg)f(to)75 1110 y(sp)q(ecify)g(the)h(total)e(length)h(of)f(the)i(arra)o(y)m(.)166 1249 y Fk(resource)p 324 1249 14 2 v 17 w(t)o(yp)q(e)20 b Fs(determines)h(ho)o(w)e(resources)h(are)g(split)g(up)h(in)o(to)e (one)h(or)g(more)f(MPI)p 1684 1249 V 16 w(Resource)75 1306 y(ob)s(jects.)g(It)14 b(is)g(a)g(suggestion,)g(not)f(a)h (requiremen)o(t,)g(as)g(certain)g(concepts)g(ma)o(y)g(b)q(e)g(foreign)g (to)g(certain)75 1362 y(parallel)j(en)o(vironmen)o(ts.)j(It)15 b(can)g(tak)o(e)g(the)g(follo)o(wing)h(prede\014ned)h(v)m(alues.)143 1449 y Fo(\017)23 b Fk(MPI)p 274 1449 V 15 w(RESOURCE)p 519 1449 V 19 w(DEF)l(A)o(UL)l(T)p Fs(:)d(A)g(single)h(resource)g(ob)s (ject)e(is)i(returned,)h(with)e(all)h(a)o(v)m(ailable)189 1505 y(resources)15 b(in)h(that)e(ob)s(ject.)143 1593 y Fo(\017)23 b Fk(MPI)p 274 1593 V 15 w(RESOURCE)p 519 1593 V 19 w(HOST)p Fs(:)g(Eac)o(h)f(returned)g(resource)h(corresp)q (onds)f(to)g(one)g(host.)41 b(A)22 b(host)189 1650 y(can)16 b(b)q(e)h(an)g(SMP)l(.)f(The)h(name)f(of)g(the)h(host)f(ma)o(y)g(b)q(e) h(disco)o(v)o(ered)g(using)g Fk(MPI)p 1562 1650 V 16 w(Resource)p 1746 1650 V 18 w(name)p Fs(.)189 1706 y(If)k(the)g (pre-allo)q(cated)h(resources)f(cannot)f(b)q(e)i(describ)q(ed)h(as)d(a) h(host,)g(\(e.g.,)g(it)g(is)g(a)g(promise)189 1762 y(from)j(a)g (resource)h(manager,)h(not)e(an)h(explicit)h(ph)o(ysical)g(resource\))f (this)g(is)g(the)g(same)f(as)189 1819 y Fk(MPI)p 274 1819 V 15 w(RESOURCE)p 519 1819 V 19 w(DEF)l(A)o(UL)l(T)p Fs(.)143 1907 y Fo(\017)f Fk(MPI)p 274 1907 V 15 w(RESOURCE)p 519 1907 V 19 w(ARCH)p Fs(:)15 b(Eac)o(h)g(returned)g(resource)g (corresp)q(onds)h(to)e(a)h(single)h(arc)o(hitecture.)189 1963 y(An)e(arc)o(hitecture)h(is)g(a)f(class)h(of)e(binary-compatible)k (computers.)i(The)c(name)f(of)g(the)g(arc)o(hitec-)189 2020 y(ture)h(ma)o(y)f(b)q(e)i(disco)o(v)o(ered)g(using)g Fk(MPI)p 868 2020 V 16 w(Resource)p 1052 2020 V 17 w(name)p Fs(.)143 2108 y Fo(\017)23 b Fk(MPI)p 274 2108 V 15 w(RESOURCE)p 519 2108 V 19 w(PROCESSOR)p Fs(:)d(Eac)o(h)e(returned)h(resource)g (corresp)q(onds)f(to)g(a)g(single)i(pro-)189 2164 y(cessor.)38 b(These)22 b(ma)o(y)e(b)q(e)i(virtual)g(pro)q(cessors.)39 b(F)l(or)20 b(instance)j(in)f(an)f(SMP)l(,)g(the)h(op)q(erating)189 2221 y(system)14 b(ma)o(y)h(not)g(ha)o(v)o(e)f(con)o(trol)h(o)o(v)o(er) g(whic)o(h)h(ph)o(ysical)g(pro)q(cessor)f(is)h(used)g(b)o(y)f(a)g(pro)q (cess.)143 2309 y Fo(\017)23 b Fk(MPI)p 274 2309 V 15 w(RESOURCE)p 519 2309 V 19 w(SPECIAL)p Fs(:)14 b(Resource)h(splitting)h (done)f(in)g(some)g(sp)q(ecial)h(w)o(a)o(y)l(.)j(Usually)d(this)189 2365 y(will)g(b)q(e)f(sp)q(eci\014ed)i(externally)l(,)f(b)o(y)e(the)h (user.)20 b(The)15 b(order)f(of)h(resources)f(can)h(b)q(e)g(coun)o(ted) g(on)g(b)o(y)189 2422 y(the)g(application.)24 b(F)l(or)15 b(instance,)h(the)g(user)g(ma)o(y)f(ha)o(v)o(e)g(requested)h(a)f(HP)h (no)q(de)h(with)f(a)f(HiPPI)189 2478 y(connection,)h(an)f(HP)g(no)q(de) h(with)f(a)g(FDDI)g(connection,)h(8)f(pro)q(cessors)f(of)h(an)g(SP2,)g (etc.)166 2647 y Fh(Discussion:)36 b Fg(Suggestions)14 b(for)g(additional)f(allo)q(cation)f(strategies)k(are)f(encouraged.)20 b(Are)15 b(there)h Fc(c)n(om-)75 2704 y(mon)e Fg(things)g(y)o(ou)g (migh)o(t)d(w)o(an)o(t)j(to)f(do)h(that)g(y)o(ou)f(can't)h(do)g(with)f (the)i(ab)q(o)o(v)o(e?)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Fs(16)928 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fk(Resource)h(Allo)q(cation)75 225 y(MPI)p 160 225 14 2 v 16 w(RESOURCE)p 406 225 V 18 w(IALLOCA)l(TE\(resource)p 836 225 V 17 w(t)o(yp)q(e,)22 b(np)o(ro)q(cs,)f(resource)p 1279 225 V 17 w(description,)h(resource,)g (request\))117 359 y Fg(IN)155 b Fk(resource)p 476 359 V 17 w(t)o(yp)q(e)335 b Fg(See)15 b(description)g(b)q(elo)o(w)117 434 y(IN)155 b Fk(np)o(ro)q(cs)465 b Fg(The)15 b(minim)n(um)9 b(n)o(um)o(b)q(er)k(of)h(pro)q(cess)h(slots)f(requested)117 508 y(IN)155 b Fk(resource)p 476 508 V 17 w(description)212 b Fg(String)14 b(con)o(taining)f(description)h(of)f(resources)117 583 y(OUT)108 b Fk(resource)432 b Ff(MPI)p 982 583 13 2 v 15 w(Resource)15 b Fg(allo)q(cated)117 657 y(OUT)108 b Fk(request)452 b Fg(an)14 b Ff(MPI)p 1040 657 V 14 w(Request)166 782 y Fk(resource)h Fs(is)h(set)f(async)o(hronously)h(b)o (y)f(MPI)g(\(as)g(with)g(other)g(MPI)p 1326 782 14 2 v 17 w(I*)g(routines\).)20 b(It)c(is)f(not)g(v)m(alid)75 838 y(un)o(til)23 b(the)g(request)f(has)g(b)q(een)h(completed.)42 b(One)23 b(can)f(w)o(ait)g(for)g(the)g(request)g(to)g(complete)h(with) 75 895 y Fk(MPI)p 160 895 V 16 w(W)o(ait)p Fs(,)14 b(test)h(with)h Fk(MPI)p 569 895 V 15 w(T)l(est)p Fs(,)g(etc.)166 951 y Fk(resource)p 324 951 V 17 w(t)o(yp)q(e)j Fs(is)f(a)g(general)g (category)f(of)g(resource)h(request,)h(used)f(to)f(mak)o(e)h(it)g(easy) g(to)f(sp)q(ec-)75 1008 y(ify)22 b(simple)h(requests.)40 b(The)22 b(categories)f(are)h(the)g(same)f(as)g(for)g(the)h Fk(resource)p 1494 1008 V 17 w(t)o(yp)q(e)h Fs(argumen)o(t)e(for)75 1064 y Fk(MPI)p 160 1064 V 16 w(RESOURCE)p 406 1064 V 18 w(DISCO)o(VER)p Fs(.)16 b(In)g(this)f(con)o(text,)f(they)i(mean:)143 1166 y Fo(\017)23 b Fk(MPI)p 274 1166 V 15 w(RESOURCE)p 519 1166 V 19 w(DEF)l(A)o(UL)l(T)p Fs(:)15 b(Unde\014ned)i(for)d (resource)h(allo)q(cation.)143 1259 y Fo(\017)23 b Fk(MPI)p 274 1259 V 15 w(RESOURCE)p 519 1259 V 19 w(HOST)p Fs(:)15 b(The)h(string)f(in)h Fk(resource)p 1118 1259 V 17 w(description)h Fs(is)e(a)g(hostname.)143 1351 y Fo(\017)23 b Fk(MPI)p 274 1351 V 15 w(RESOURCE)p 519 1351 V 19 w(ARCH)p Fs(:)e(The)f(string)h (in)g Fk(resource)p 1138 1351 V 16 w(description)h Fs(is)f(an)g(arc)o (hitecture.)35 b(Note)189 1408 y(that)14 b(arc)o(hitecture)i(names)f (are)g(not)f(sp)q(eci\014ed)k(b)o(y)d(MPI)g(itself.)143 1501 y Fo(\017)23 b Fk(MPI)p 274 1501 V 15 w(RESOURCE)p 519 1501 V 19 w(PROCESSOR)p Fs(:)g(The)e(string)g(in)h Fk(resource)p 1281 1501 V 17 w(description)g Fs(describ)q(es)h(a)e (single)189 1557 y(pro)q(cessor.)k(The)17 b(recommended)h(approac)o(h)f (is)g(that)g(the)g(name)g(should)h(b)q(e)g(the)f(name)g(of)g(the)189 1613 y(host)d(in)i(whic)o(h)g(the)g(pro)q(cessor)f(resides.)143 1706 y Fo(\017)23 b Fk(MPI)p 274 1706 V 15 w(RESOURCE)p 519 1706 V 19 w(SPECIAL)p Fs(:)c(The)h(string)g(in)g Fk(resource)p 1192 1706 V 17 w(description)i Fs(describ)q(es)f(an)f (arbitrary)189 1763 y(set)15 b(of)f(resources.)166 1947 y Fh(Discussion:)31 b Fg(Need)11 b(a)g(w)o(a)o(y)e(to)i(deal)f(with)g (hard/soft)h(requests.)19 b(A)10 b Fc(har)n(d)15 b Fg(allo)q(cation)9 b(request)k(is)d(required)75 2004 y(to)h(ev)o(en)o(tually)f(return)i (the)f(en)o(tire)g(n)o(um)o(b)q(er)g(of)f(pro)q(cessors)j(requested,)g (whereas)f(a)e Fc(soft)k Fg(allo)q(cation)c(request)i(ma)o(y)75 2060 y(complete)g(when)h(it)g(can)g(allo)q(cate)f(some)g(pro)q(cessors) j(ev)o(en)e(if)f(it)h(kno)o(ws)f(that)h(it)f(will)g(not)g(b)q(e)i(able) e(to)h(satisfy)f(the)75 2117 y(en)o(tire)j(request.)166 2339 y Fh(Discussion:)39 b Fg(Note)16 b(that)f Ff(MPI)p 709 2339 13 2 v 15 w(Resource)p 877 2339 V 16 w(Iallo)q(cate)g Fg(do)q(es)h(not)f(tak)o(e)h(a)f(comm)o(uni)o(cator)e(argumen)o(t)h (and)75 2395 y(is)g(called)f(in)g(a)g(single)h(pro)q(cess.)19 b(This)14 b(means)f(that)h(an)f Ff(MPI)p 1038 2395 V 14 w(Resource)i Fg(ob)r(ject)g(has)f(a)f(single)g(o)o(wner.)18 b(T)m(rying)13 b(to)75 2452 y(mak)o(e)f(this)h(op)q(eration)g (collectiv)o(e)g(w)o(ould)f(force)i(us)f(to)g(in)o(tro)q(duce)h (collectiv)o(e)f(ob)r(jects)h(\(collectiv)o(e)f(requests)i(and)75 2508 y(collectiv)o(e)f Ff(MPI)p 332 2508 V 14 w(No)q(de)h Fg(ob)r(jects\).)166 2647 y Fk(MPI)p 251 2647 14 2 v 16 w(W)o(ait)h Fs(or)f Fk(MPI)p 512 2647 V 16 w(T)l(est)i Fs(can)g(b)q(e)g(used)f(to)g(c)o(hec)o(k)g(for)g(completion.)24 b(The)16 b Fk(MPI)p 1557 2647 V 16 w(ST)l(A)l(TUS)i Fs(ob)s(ject)75 2704 y(can)d(b)q(e)h(mo)q(di\014ed)h(to)d(con)o(tain)i(information)f (relating)h(to)e(the)i(completion)g(of)f(the)g(op)q(eration.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fn(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(17)166 45 y Fh(Discussion:)32 b Fg(This)13 b(is)f(left)h(o)o(v)o(er)f(from)f(a)h(previous)h(v)o(ersion,)f(but)h (will)e(b)q(ecome)i(relev)n(an)o(t)f(again)g(once)h(w)o(e)75 95 y(get)h(deep)q(er)i(in)o(to)d(the)i(discussion)f(of)f(describing)i (resources.)20 b({)14 b(RL)166 145 y(Authen)o(tication)i(information,)d (if)i(required)i(b)o(y)f(the)g(resource)i(manager,)d(can)h(b)q(e)h (supplied)f(in)f(the)i(job-)75 195 y(sc)o(heduler-dep)q(enden)o(t)h (resource)f(description)e(string.)21 b(The)16 b(executable)g(\014les)f (are)g(not)g(sp)q(eci\014ed)i(on)d(this)h(call,)75 244 y(since)k(this)g(is)f(just)h(for)f(resource)i(allo)q(cation.)30 b(W)m(e)18 b(can)h(attac)o(h)g(them)e(to)i(the)g(requests)h(with)e(the) h(function)75 294 y(describ)q(ed)d(in)d(the)i(next)f(section.)166 344 y(The)g(string)g(con)o(taining)f(resource-manager-dep)q(enden)o(t)i (strings)g(are)f(implem)o(en)o(tation)d(de\014ned)k(\(b)o(y)e(the)75 394 y(resource)22 b(manager,)e(not)g(MPI\).)g(These)h(ma)o(y)e(con)o (tain)g(information)e(to)j(b)q(e)h(applied)f(to)g(all)e(pro)q(cesses)23 b(\(in)75 444 y Ff(global)p 180 444 13 2 v 15 w(js)p 222 444 V 15 w(dep)p 297 444 V 16 w(string)p Fg(\))14 b(and)g(information)d(for)i(a)h(particular)g(pro)q(cess)h(\(in)75 493 y Ff(a)o(rra)o(y)p 162 493 V 15 w(of)p 211 493 V 14 w(lo)q(cal)p 305 493 V 16 w(js)p 348 493 V 14 w(dep)p 422 493 V 17 w(strings)p Fg(\).)27 b(An)17 b(example)e(of)h Ff(global)p 996 493 V 15 w(js)p 1038 493 V 15 w(dep)p 1113 493 V 16 w(string)h Fg(is)g Fe(gang)p 1377 493 14 2 v 14 w(schedule)p Fg(;)f(and)h(example)f(of)75 543 y Ff(a)o(rra)o(y)p 162 543 13 2 v 15 w(of)p 211 543 V 14 w(lo)q(cal)p 305 543 V 16 w(js)p 348 543 V 14 w(dep)p 422 543 V 17 w(strings)e Fg(is)f Fb(f)22 b Fe("large)p 783 543 14 2 v 14 w(memory",)f("local)p 1126 543 V 14 w(hippi")p Fb(g)p Fg(.)166 600 y(A)15 b(no)q(de)g(ma)o(y)e(ha)o(v)o(e)h (m)o(ultiple)f(comm)o(unicati)o(on)f(in)o(terfaces)k(\(Ethernet,)g (HiPPI,)e(A)m(TM\).)h(These)h(ha)o(v)o(e)e(to)75 656 y(sp)q(eci\014ed)h(so)f(that)g(the)h(desired)g(comm)o(unicatio)o(n)c (infrastructure)16 b(can)e(b)q(e)g(used.)75 859 y Fk(Interfacing)h(to)h (the)g(Pro)q(cess)g(Manager)75 945 y Fs(While)h Fk(MPI)p 293 945 V 16 w(SP)l(A)-5 b(WN)17 b Fs(con)o(v)o(enien)o(tly)g(captures) g(man)o(y)e(asp)q(ects)h(of)g(resource)g(allo)q(cation)h(and)g(pro)q (cess)75 1001 y(startup)g(in)h(one)f(call,)h(w)o(e)f(need)h(more)f (detailed)i(con)o(trol)e(of)g(these)g(steps.)26 b(Our)18 b(mo)q(del)g(will)h(b)q(e)f(that)75 1058 y(one)e(calls)g Fk(MPI)p 345 1058 V 16 w(IALLOCA)l(TE)g Fs(to)f(reserv)o(e)g(pro)q (cessors,)g(getting)h(bac)o(k)f(a)g(set)h(of)f(resources.)21 b(Pro)q(cesses)75 1114 y(can)16 b(then)g(b)q(e)g(started)f(on)g(these)h (resources)g(\(i.e.)21 b(the)15 b(pro)q(cess)h(manager)f(can)h(b)q(e)g (requested)g(to)f(start)75 1171 y(pro)q(cesses)h(on)f(these)g (resources\))g(with)h(the)f(routines)g(presen)o(ted)h(in)g(this)g (section.)75 1321 y Fk(MPI)p 160 1321 V 16 w(EXEC\(executable,)h(a)o (rguments,)d(n,)i(resource,)f(a)o(rra)o(y)p 1086 1321 V 14 w(of)p 1137 1321 V 16 w(p)o(ro)q(cesses\))117 1399 y Fg(IN)155 b Fk(executable)391 b Fg(executable)15 b(\014le)f(con)o (taining)f(program)f(to)i(b)q(e)g(run)117 1474 y(IN)155 b Fk(a)o(rguments)391 b Fg(argumen)o(ts)13 b(for)h(the)g(program)117 1549 y(IN)155 b Fk(n)564 b Fg(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i (to)e(start)117 1624 y(IN)155 b Fk(resource)432 b Ff(MPI)p 982 1624 13 2 v 15 w(Resource)15 b Fg(to)f(run)g(pro)q(cesses)i(on)117 1699 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1699 14 2 v 15 w(of)p 468 1699 V 16 w(p)o(ro)q(cesses)251 b Ff(MPI)p 982 1699 13 2 v 15 w(Pro)q(cess)14 b Fg(ob)r(jects)h(returned)166 1824 y Fs(This)g(is)g(a)g(blo)q(c)o(king)h(call;)g(when)f(it)g(returns) f(the)h(pro)q(cesses)g(ha)o(v)o(e)g(b)q(een)h(started.)i(Note)d(that)f (the)75 1880 y(resources)k(ha)o(v)o(e)g(already)g(b)q(een)i(allo)q (cated.)29 b(Unless)19 b(there)g(is)f(an)g(error)g(in)h(argumen)o(ts)e (\(executable)75 1937 y(do)q(es)i(not)g(exist,)h(for)f(example\))h Fk(MPI)p 750 1937 14 2 v 15 w(Exec)g Fs(normally)g(should)g(w)o(ork.)31 b(\(The)19 b(system)f(ma)o(y)h(ha)o(v)o(e)g(to)75 1993 y(con)o(tact)11 b(the)h(resource)g(manager,)f(but)h(the)g(resources)g (ha)o(v)o(e)f(b)q(een)i(promised)g(and)f(should)h(b)q(e)f(a)o(v)m (ailable)75 2049 y(immediately\).)23 b(There)16 b(is)g(one)g (executable)h(name)f(and)g(one)g(set)f(of)h(argumen)o(ts.)21 b(T)l(o)15 b(start)g(pro)q(cesses)75 2106 y(with)h(m)o(ultiple)h (executables)f(or)f(di\013eren)o(t)g(argumen)o(ts,)f(call)i(this)g(t)o (wice.)166 2162 y(Note:)25 b(in)19 b(this)f(v)o(ersion)g(of)g Fk(MPI)p 743 2162 V 16 w(Exec)p Fs(,)h(all)f(tasks)g(initiated)h(with)g (one)f(call)h(to)e Fk(MPI)p 1666 2162 V 16 w(Exec)h Fs(ha)o(v)o(e)75 2219 y(the)e(same)f(executable)h(and)g(same)f(argumen)o(ts,)f(and)i (execute)g(on)g(the)f(same)g(resource.)21 b(One)16 b(can)g(use)75 2275 y(m)o(ultiple)j Fk(MPI)p 340 2275 V 16 w(Exec)e Fs(calls)h(to)f(spa)o(wn)f(m)o(ultiple)j(executables,)f(to)f(pass)f (di\013eren)o(t)i(argumen)o(t)e(lists)i(to)75 2332 y(eac)o(h)d (executatable,)h(or)e(to)h(run)g(di\013eren)o(t)h(pro)q(cesses)f(on)h (di\013eren)o(t)f(resources.)166 2471 y Fh(Discussion:)50 b Fg(Note)19 b(that)f(a)g(single)f(resource)j(migh)o(t)c(b)q(e)j(\\a)f (P)o(aragon",)f(so)h(it)g(mak)o(es)f(sense)j(to)e(run)75 2527 y(m)o(ultiple)12 b(pro)q(cesses)18 b(on)c(a)h(single)f(resource.) 23 b(If)14 b(w)o(e)h(allo)o(w)f(construction)h(of)g(a)f(\\virtual)g (mac)o(hine")f(as)i(a)f(single)75 2584 y(resource,)f(then)d(it)g(again) g(mak)o(es)f(sense)j(and)e(is)g(con)o(v)o(enien)o(t)h(to)f(use)h(a)f (single)g(call)f(to)h(start)h(m)o(ultiple)d(pro)q(cesses)13 b(on)75 2640 y(a)g(single)h(resource.)20 b(On)14 b(the)g(other)h(hand,) e(one)h(migh)o(t)d(conceiv)o(e)k(of)e(situation)g(where)i(it)e(w)o (ould)g(b)q(e)i(con)o(v)o(enien)o(t)75 2697 y(to)f(replace)h(the)f Ff(resource)i Fg(argumem)o(t)11 b(ab)q(o)o(v)o(e)j(with)f(an)h(arra)o (y)f(of)h(resources.)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Fs(18)928 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 45 y Fs(The)g(non)o(blo)q(c)o(king)i (v)o(ersion)e(returns)g(a)g(request)g(that)g(can)g(b)q(e)h(tested)f (and)h(w)o(aited)f(on.)75 196 y Fk(MPI)p 160 196 14 2 v 16 w(IEXEC\(executable,)h(a)o(rguments,)f(n,)g(resource,)g(a)o(rra)o (y)p 1098 196 V 15 w(of)p 1150 196 V 16 w(p)o(ro)q(cesses,request\))117 273 y Fg(IN)155 b Fk(executable)391 b Fg(executable)15 b(\014le)f(con)o(taining)f(program)f(to)i(b)q(e)g(run)117 348 y(IN)155 b Fk(a)o(rguments)391 b Fg(argumen)o(ts)13 b(for)h(the)g(program)117 424 y(IN)155 b Fk(n)564 b Fg(n)o(um)o(b)q(er) 13 b(of)h(pro)q(cesses)117 499 y(IN)155 b Fk(resource)432 b Ff(MPI)p 982 499 13 2 v 15 w(Resourc)p Fg(s)15 b(to)e(run)i(pro)q (cesses)h(on)117 574 y(OUT)108 b Fk(a)o(rra)o(y)p 416 574 14 2 v 15 w(of)p 468 574 V 16 w(p)o(ro)q(cesses)251 b Ff(MPI)p 982 574 13 2 v 15 w(Pro)q(cess)14 b Fg(ob)r(jects)h (returned)117 649 y(OUT)108 b Fk(request)452 b Ff(MPI)p 982 649 V 15 w(Request)15 b Fg(ob)r(jects)g(returned,)g(to)f(w)o(ait)f (on)166 856 y Fh(Discussion:)32 b Fg(A)12 b(more)g(detailed)g(v)o (ersion)g(w)o(ould)g(return)h(an)f(arra)o(y)g(of)g(requests,)i(so)e (that)g(startup)h(could)75 913 y(b)q(e)j(w)o(aited)f(on)g(one)g(pro)q (cess)i(at)e(a)g(time.)20 b(This)15 b(functionalit)o(y)f(could)h(b)q(e) h(pro)o(vided)f(with)g(the)g(ab)q(o)o(v)o(e)g(prop)q(osal)75 969 y(b)o(y)f(using)f Ff(MPI)p 318 969 V 15 w(IEXEC)f Fg(on)h(one)h(pro)q(cess)i(at)e(a)g(time.)166 1191 y Fh(Discussion:)22 b Fg(W)m(e)16 b(ma)o(y)g(w)o(an)o(t)g(to)h(consider)h (routines)g(that)f(build)f(a)h(resource,)i(corresp)q(onding)f(to)f(the) 75 1247 y(w)o(a)o(y)d(PVM)h(users)h(build)d(a)i(virtual)e(mac)o(hine.) 19 b(In)c(this)f(w)o(a)o(y)g(m)o(ultiple)e(resources)17 b(can)e(b)q(e)g(com)o(bined)e(in)o(to)h(one,)75 1304 y(handy)i(for)f(constructing)i(a)f(sp)q(eci\014c)h(set)g(of)e(w)o (orkstations,)h(for)g(example.)22 b(This)16 b(w)o(ould)f(mak)o(e)g(it)g (ev)o(en)i(more)75 1360 y(unnecessary)f(to)e(mak)o(e)e(m)o(ultiple)g Ff(MPI)p 704 1360 V 14 w(EXEC)h Fg(calls.)75 1563 y Fk(Utilit)o(y)j (routines)75 1649 y Fs(In)d(this)g(section)g(w)o(e)f(collect)h (routines)g(that)f(pro)o(vide)h(access)f(to)g(the)g(data)g(con)o (tained)h(in)g(the)g(datat)o(yp)q(es)75 1705 y Fk(MPI)p 160 1705 14 2 v 16 w(Pro)q(cess)j Fs(and)f Fk(MPI)p 505 1705 V 16 w(Resource)p Fs(.)75 1856 y Fk(MPI)p 160 1856 V 16 w(PROCESS)p 373 1856 V 18 w(RESOURCE\(p)o(ro)q(cess,)i(resource\)) 117 1933 y Fg(IN)155 b Fk(p)o(ro)q(cess)451 b Fg(an)14 b Ff(MPI)p 1040 1933 13 2 v 14 w(Pro)q(cess)117 2008 y Fg(OUT)108 b Fk(resource)432 b Fg(the)15 b Ff(MPI)p 1054 2008 V 14 w(Resource)g Fg(on)f(whic)o(h)g(the)g(pro)q(cess)i(is)e (running)75 2133 y Fs(Gets)k(the)g(resource)g(on)h(whic)o(h)g(a)f(pro)q (cess)g(is)h(running.)30 b(This)19 b(routine)g(can)f(only)h(b)q(e)g (called)h(on)e(the)75 2189 y(pro)q(cess)d(whic)o(h)i(o)o(wns)d(the)h (resource.)20 b(On)c(all)g(other)f(pro)q(cess)h(it)f(returns)g Fk(MPI)p 1453 2189 14 2 v 16 w(RESOURCE)p 1699 2189 V 18 w(NULL)p Fs(.)166 2328 y Fh(Discussion:)36 b Fg(Allo)o(wing)13 b(other)i(pro)q(cesses)i(to)d(retriev)o(e)i(the)f(corresp)q(onding)g (resource)i(causes)f(all)d(sorts)75 2385 y(of)h(problems.)19 b(The)c(curren)o(t)h(de\014nition)e(is)h(probably)e(\014ne)i(un)o(til)f (w)o(e)h(start)g(talking)e(ab)q(out)i(fault)e(tolerance,)i(in)75 2441 y(whic)o(h)f(case)h(w)o(e)f(ha)o(v)o(e)f(another)i(problem.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fn(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(19)75 45 y Fk(MPI)p 160 45 14 2 v 16 w(RESOURCE)p 406 45 V 18 w(PROCESSES\(resource,)17 b(n,)e(a)o(rra)o(y)p 1022 45 V 15 w(of)p 1074 45 V 16 w(p)o(ro)q(cesses\))117 122 y Fg(IN)155 b Fk(resource)432 b Fg(an)14 b Ff(MPI)p 1040 122 13 2 v 14 w(Resource)117 196 y Fg(OUT)108 b Fk(n)564 b Fg(n)o(um)o(b)q(er)13 b(of)h(pro)q (cesses)i(running)e(on)f(resource)117 270 y(OUT)108 b Fk(a)o(rra)o(y)p 416 270 14 2 v 15 w(of)p 468 270 V 16 w(p)o(ro)q(cesses)251 b Fg(arra)o(y)14 b(of)f Ff(MPI)p 1138 270 13 2 v 14 w(Pro)q(cess)p Fg(es)75 395 y Fs(Gets)18 b(the)g(pro)q(cesses)g(running)i(on)e(a)f(giv)o(en)i(resource.)29 b(Returns)18 b Fa(n)g Fs(=)g(0)g(if)h(none)f(\(resource)g(is)h(then)75 451 y(free)c(for)g(a)g(new)g(task\))166 590 y Fh(Discussion:)30 b Fg(Do)9 b(w)o(e)i(need)g(to)f(sp)q(ecify)g(the)h(size)g(of)e(the)i (arra)o(y)f(or)g(do)g(w)o(e)g(let)g(the)h(user)g(hang)e(him/herself)s (?)75 880 y Fk(MPI)p 160 880 14 2 v 16 w(FREE)p 285 880 V 17 w(RESOURCE\(resources\))117 957 y Fg(IN)155 b Fk(resources)415 b Fg(resource)16 b(to)e(b)q(e)g(returned)i(to)e(resource)i(manager)75 1082 y Fs(releases)24 b(resources)g(to)f(resource)g(manager.)45 b(Pro)q(cesses)24 b(running)g(on)g(those)f(resources)h(are)f(lost)75 1138 y(\(killed?\).)f(\(dealing)15 b(with)g(this)g(case)f(is)h (di\016cult)h(and)f(should)g(should)g(b)q(e)g(addressed\).)20 b(The)15 b(resource)75 1195 y(m)o(ust)g(b)q(e)g(o)o(wned)h(b)o(y)f(the) g(caller.)75 1346 y Fk(MPI)p 160 1346 V 16 w(PROCESS)p 373 1346 V 18 w(IN)p 436 1346 V 16 w(GROUP\(group,)h(rank,)f(p)o(ro)q (cess\))117 1423 y Fg(IN)155 b Fk(comm)470 b Fg(group)117 1497 y(IN)155 b Fk(rank)504 b Fg(rank)14 b(in)f(group)117 1571 y(OUT)108 b Fk(p)o(ro)q(cess)451 b Ff(MPI)p 982 1571 13 2 v 15 w(Pro)q(cess)14 b Fg(corresp)q(onding)h(to)e(rank)h(in)g (group)75 1695 y Fs(gets)h(the)g Fk(MPI)p 332 1695 14 2 v 16 w(Pro)q(cess)h Fs(corresp)q(onding)g(to)e(a)h(giv)o(en)h(group)f (and)g(rank.)166 1752 y(A)g(pro)q(cess)f(represen)o(ted)i(b)o(y)e(an)h Fk(MPI)p 824 1752 V 16 w(Pro)q(cess)g Fs(cannot)f(b)q(e)i(comm)o (unicated)f(with)g(directly)l(,)h(un)o(til)75 1808 y(a)i(comm)o (unicator)h(is)g(constructed)f(con)o(taining)i(it)f(\(see)f(next)h (section\).)30 b(It)19 b(need)g(not)f(b)q(e)i(an)e(\\MPI)75 1864 y(pro)q(cess")h(in)g(the)g(sense)g(that)f(it)i(migh)o(t)e(not)h (call)g Fk(MPI)p 1059 1864 V 16 w(INIT)p Fs(.)f(On)h(the)g(other)g (hand,)g(it)g(allo)o(ws)g(out-)75 1921 y(of-band)h(comm)o(unication,)h (suc)o(h)e(as)g(signals,)i(and)f(migh)o(t)f(b)q(e)h(a)f(useful)i (concept)f(for)f(dealing)i(with)75 1977 y(failures.)166 2034 y(T)l(o)15 b(ask)g(the)g(pro)q(cess)g(manager)g(to)f(deliv)o(er)j (signals)f(to)f(pro)q(cesses,)g(w)o(e)g(use)75 2185 y Fk(MPI)p 160 2185 V 16 w(SIGNAL\(signal,)g(num)p 567 2185 V 16 w(p)o(ro)q(cesses,)i(a)o(rra)o(y)p 881 2185 V 14 w(of)p 932 2185 V 16 w(p)o(ro)q(cesses\))117 2262 y Fg(IN)155 b Fk(signal)480 b Fg(signal)13 b(t)o(yp)q(e)h(\(in)o(t\)) 117 2336 y(IN)155 b Fk(num)p 404 2336 V 17 w(p)o(ro)q(cesses)314 b Fg(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(in)e(arra)o(y)117 2410 y(IN)155 b Fk(a)o(rra)o(y)p 416 2410 V 15 w(of)p 468 2410 V 16 w(p)o(ro)q(cesses)251 b Ff(MPI)p 982 2410 13 2 v 15 w(Pro)q(cess)p Fg(es)15 b(to)e(b)q(e)i(signalled)166 2534 y Fs(It)22 b(is)g(the)f(resp)q(onsibilit)o(y)j(of)d(an)h(implemen) o(tation)h(to)e(translate)g(b)q(et)o(w)o(een)h(signals;)j(in)d(other)75 2591 y(w)o(ords,)13 b(a)h Fj(SIGINT)g Fs(that)f(has)i(v)m(alue)g Fj(3)f Fs(on)h(system)e(A)i(m)o(ust)f(b)q(e)h(deliv)o(ered)h(as)e(a)g Fj(SIGINT)f Fs(on)h(system)g(B,)75 2647 y(ev)o(en)i(if)g Fj(SIGINT)f Fs(on)g(system)h(B)f(uses)h(the)g(v)m(alue)h Fj(5)e Fs(for)g Fj(SIGINT)p Fs(.)g(If)h(the)f(signal)i(can)f(not)f(b)q (e)h(deliv)o(ered)75 2704 y(b)q(ecause)g(there)f(is)h(no)f(corresp)q (onding)h(signal,)g(the)f(error)g(co)q(de)h(is)f Fk(MPI)p 1327 2704 14 2 v 16 w(ERR)p 1428 2704 V 17 w(INV)l(ALID)p 1617 2704 V 17 w(SIGNAL)p Fs(.)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Fs(20)928 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 45 y Fh(Discussion:)29 b Fg(Should)9 b(there)i(b)q(e)e(a)g(single-request)i(v)o(ersion)e(of)g (this,)h(and)f(the)h(ab)q(o)o(v)o(e)f(renamed)g Ff(MPI)p 1767 45 13 2 v 14 w(SIGNAL)p 1923 45 V 15 w(ALL)p Fg(?)75 102 y(Is)14 b(the)h(notion)e(of)g(signals)g(to)q(o)h(Unix-cen)o(tric?) 166 241 y Fs(There)j(needs)h(to)f(b)q(e)h(some)f(w)o(a)o(y)f(of)h (\014nding)h(out)f(when)h(a)f(pro)q(cess)g(\014nishes.)27 b(In)18 b(MPI)f(w)o(e)g(ha)o(v)o(e)75 297 y(no)f(mec)o(hanism)h(for)e (async)o(hronous)h(noti\014cation.)23 b(Therefore)16 b(the)h(b)q(est)f(w)o(e)g(can)g(do)g(is)h(to)e(construct)75 354 y(a)f(request)g(that)f(can)h(b)q(e)h(tested)f(and)g(w)o(aited)g (on.)19 b(If)14 b(pro)q(cesses)h(w)o(ere)e(represen)o(ted)i(b)o(y)f (requests,)g(then)75 410 y(w)o(e)k(could)g(w)o(ait)g(on)g(them)f (directly)l(.)30 b(Since)19 b(w)o(e)f(ha)o(v)o(e)f(in)o(tro)q(duced)i (the)f(notion)g(of)g Fk(MPI)p 1636 410 14 2 v 15 w(Pro)q(cess)p Fs(,)h(w)o(e)75 467 y(explicitly)f(attac)o(h)c(a)h(request)g(to)g(it)g (in)h(order)f(to)g(w)o(ait)f(on)i(it.)75 617 y Fk(MPI)p 160 617 V 16 w(NOTIFY)p 341 617 V 17 w(WHEN)p 492 617 V 16 w(DONE\(n,)g(a)o(rra)o(y)p 798 617 V 15 w(of)p 850 617 V 16 w(p)o(ro)q(cesses,)g(request\))117 695 y Fg(IN)155 b Fk(n)564 b Fg(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(in)e(follo)o (wing)d(arra)o(y)117 770 y(IN)155 b Fk(a)o(rra)o(y)p 416 770 V 15 w(of)p 468 770 V 16 w(p)o(ro)q(cesses)251 b Fg(arra)o(y)14 b(of)f Ff(MPI)p 1138 770 13 2 v 14 w(Pro)q(cess)i Fg(ob)r(jects)117 845 y(OUT)108 b Fk(request)452 b Fg(mpifuncMPI)p 1136 845 V 14 w(Request)15 b(to)e(b)q(e)i(tested)g(and/or)f(w)o(aited)f (on)166 1052 y Fh(Discussion:)40 b Fg(Should)15 b Ff(MPI)p 657 1052 V 15 w(NOTIFY)p 824 1052 V 14 w(WHEN)p 960 1052 V 15 w(DONE)g Fg(return)i(an)e(arra)o(y)h(of)f(requests)i(instead)f(of) f(just)75 1109 y(one?)166 1248 y Fs(W)l(e)g(can)g(think)g(of)g(this)g (as)g(MPI)g(in)o(terface)g(to)f(the)h(Pro)q(cess)g(Manager's)e (handling)k(of)d(the)h(signal)75 1304 y Fj(SIGCHILD)p Fs(\).)g(The)i(exit)g(co)q(de)h(from)e(the)h(pro)q(cess)g(\(from)f(a)g Fj(return)23 b(n)17 b Fs(or)f Fj(exit\(n\))g Fs(in)i(C)e(or)h Fj(STOP)23 b(n)75 1361 y Fs(in)16 b(F)l(ortran\)\))d(can)j(b)q(e)g (retriev)o(ed)f(from)g(the)g Fk(MPI)p 934 1361 14 2 v 16 w(status)i Fs(\014lled)g(in)f(the)f Fk(MPI)p 1402 1361 V 16 w(W)l(AIT)p Fs(.)166 1417 y(Note)i(that)g(this)h(lev)o(el)h (of)e(pro)q(cess)h(managemen)o(t)f(allo)o(ws)h(us)g(to)f(manage)g (non-MPI)h(pro)q(cesses,)75 1473 y(since)e(comm)o(unicators)f(are)g (not)g(in)o(v)o(olv)o(ed.)166 1530 y(If)d(the)f(started)g(pro)q(cesses) h(are)f(MPI)h(pro)q(cesses)g(\(that)f(is,)h(if)g(they)g(call)g Fk(MPI)p 1461 1530 V 16 w(INIT)p Fs(\),)e(then)i(the)g(com-)75 1586 y(m)o(unicator)17 b(that)f(includes)j(them)e(can)g(b)q(e)g (constructed)g(with)g(the)g Fk(MPI)p 1352 1586 V 16 w(A)o(ttach)h Fs(function)g(describ)q(ed)75 1643 y(b)q(elo)o(w.)i(In)14 b(either)h(case,)e(at)g(this)h(stage)f(w)o(e)g(ha)o(v)o(e)h(allo)q (cated)g(resources)g(and)g(the)g(pro)q(cesses)g(ha)o(v)o(e)f(b)q(een)75 1699 y(started.)25 b(The)18 b(third)f(comp)q(onen)o(t)h(of)f(the)g(run) o(time)g(en)o(vironmen)o(t)h(is)g(the)f(message-passing)g(library)l(.) 75 1756 y(In)f(order)e(to)h(comm)o(unicate)g(with)h(these)f(pro)q (cesses,)g(w)o(e)g(m)o(ust)f(in)o(terface)h(with)h(the)f (message-passing)75 1812 y(library)l(.)21 b(In)16 b(MPI)f(terms,)f (this)i(means)f(setting)g(up)h(comm)o(unicators.)75 1932 y Fk(Communicating)f(with)g(Dynamically)f(Created)i(MPI)f(Pro)q(cesses) 75 2018 y Fs(Once)g(the)f(pro)q(cesses)g(ha)o(v)o(e)f(b)q(een)i (started,)e(w)o(e)h(ma)o(y)f(or)g(ma)o(y)g(not)h(wish)g(to)f(establish) i(comm)o(unication)75 2075 y(with)d(them.)19 b(Note)11 b(that)h(the)g Fk(MPI)p 670 2075 V 15 w(IALLOCA)l(TE)p Fs(/)p Fk(MPI)p 1029 2075 V 16 w(EXEC)h Fs(mec)o(hanism)f(can)g(b)q(e)h (used)f(b)o(y)g(a)g(pro)q(cess)75 2131 y(manager)18 b(written)h(in)g (MPI)g(to)f(manage)g(non-MPI)i(pro)q(cesses)f(as)f(w)o(ell)i(as)e(MPI)h (pro)q(cesses.)31 b(If)19 b(the)75 2187 y(application)j(program)e(is)h (creating)g(new)g(pro)q(cesses,)i(ho)o(w)o(ev)o(er,)e(it)g(is)g(lik)o (ely)h(that)e(it)h(will)i(w)o(an)o(t)d(to)75 2244 y(comm)o(unicate)d (with)f(them,)h(via)f(an)h(in)o(ter-comm)o(unicator.)23 b(The)17 b(complication)g(is)g(that)f(creation)g(of)75 2300 y(this)i(in)o(ter-comm)o(unicator)g(is)g(a)f(collectiv)o(e)i(op)q (eration)f(o)o(v)o(er)f(all)h(pro)q(cesses)g(in)o(v)o(olv)o(ed,)h(y)o (et)e(w)o(e)g(don't)75 2357 y(w)o(an)o(t)j(the)h(spa)o(wning)h(pro)q (cess\(es\))f(to)f(blo)q(c)o(k)i(if)f(the)h(spa)o(wned)f(pro)q(cesses)g (are)g(not)g(ev)o(en)g(going)h(to)75 2413 y(call)e Fk(MPI)p 247 2413 V 16 w(INIT)p Fs(.)e(Our)i(solution)f(is)h(to)f(create)f(this) i(comm)o(unicator)f(through)f(a)h(collectiv)o(e)i(function)75 2470 y(separate)15 b(from)f Fk(MPI)p 447 2470 V 16 w(Exec)p Fs(.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fn(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(21)75 45 y Fk(MPI)p 160 45 14 2 v 16 w(A)l(TT)l(A)o(CH\(comm,)13 b(ro)q(ot,)i(n,)g(a)o(rra)o(y)p 764 45 V 15 w(of)p 816 45 V 16 w(p)o(ro)q(cesses,)h(intercomm\))117 122 y Fg(IN)155 b Fk(comm)470 b Fg(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o(h)g(this)g(op)q(eration)g(is)f(collectiv)o(e)117 197 y(IN)155 b Fk(ro)q(ot)508 b Fg(pro)q(cess)16 b(where)f Ff(n)f Fg(and)g Ff(a)o(rra)o(y)p 1373 197 13 2 v 15 w(of)p 1422 197 V 14 w(p)o(ro)q(cesses)i Fg(are)e(v)n(alid)117 273 y(IN)155 b Fk(n)564 b Fg(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i (in)e(follo)o(wing)d(arra)o(y)117 348 y(IN)155 b Fk(a)o(rra)o(y)p 416 348 14 2 v 15 w(of)p 468 348 V 16 w(p)o(ro)q(cesses)251 b Fg(the)15 b(spa)o(wned)f(pro)q(cesses)j(to)c(b)q(e)i(connected)h(to) 117 423 y(OUT)108 b Fk(intercomm)384 b Fg(in)o(tercomm)o(unicator)7 b(in)i(whic)o(h)g(spa)o(wned)h(pro)q(cesses)i(are)e(the)905 479 y(remote)k(group)75 604 y Fk(MPI)p 160 604 V 16 w(A)l(TT)l(A)o(CH)i Fs(is)f(collectiv)o(e)i(among)e(the)g(pro)q(cesses)g(in)h Fk(comm)d Fs(and)j(with)f(the)g(routine)h Fk(MPI)p 1721 604 V 16 w(INIT)e Fs(in)75 660 y(the)h(pro)q(cesses)h(b)q(eing)g(attac) o(hed)f(to)g(the)g(MPI)g(univ)o(erse.)166 717 y Fk(MPI)p 251 717 V 16 w(INIT)e Fs(creates)h(an)g Fk(MPI)p 664 717 V 16 w(COMM)p 822 717 V 17 w(W)o(ORLD)g Fs(con)o(taining)h(all)g (one's)f(sibling)i(pro)q(cesses,)f(where)75 773 y(siblings)g(are)d (de\014ned)j(as)d(all)i(the)f(new)g(pro)q(cesses)g(that)f(participate)i (in)g(the)f Fk(MPI)p 1474 773 V 15 w(A)l(TT)l(A)o(CH)h Fs(op)q(eration,)75 830 y(not)h(all)h(tasks)e(that)h(w)o(ere)g(spa)o (wned)g(in)h(one)f(op)q(eration.)166 886 y(The)j(\\other)f(side")h(of)g Fk(intercomm)e Fs(is)i Fk(MPI)p 929 886 V 16 w(COMM)p 1087 886 V 16 w(P)l(ARENT)h Fs(in)g(the)f(c)o(hildren.)29 b(In)18 b(MPI)g(pro-)75 942 y(cesses)e(whic)o(h)g(w)o(ere)f(not)f (created)i(b)o(y)f Fk(MPI)p 828 942 V 15 w(Exec)p Fs(,)h Fk(MPI)p 1042 942 V 16 w(COMM)p 1200 942 V 16 w(P)l(ARENT)g Fs(is)g Fk(MPI)p 1534 942 V 16 w(COMM)p 1692 942 V 16 w(NULL)p Fs(.)166 1075 y Fh(Discussion:)49 b Fg(In)18 b(the)g(curren)o(t)i(prop)q(osal,)e(c)o(hildren)g(attac)o(h)f(to)h (their)g(paren)o(ts)h(automatically)14 b(when)75 1125 y(they)g(call)f Ff(MPI)p 321 1125 13 2 v 15 w(INIT)p Fg(.)g(This)g(means)g(that)h(pro)q(cesses)i(cannot)e(call)f Ff(MPI)p 1219 1125 V 15 w(INIT)g Fg(unless)i(they)f(comm)o(unicate)d (with)75 1175 y(their)j(paren)o(ts)h(\(and)f(their)g(paren)o(ts)h(call) e Ff(MPI)p 817 1175 V 14 w(A)o(ttach)p Fg(\).)166 1224 y(An)h(alternativ)o(e)g(w)o(ould)g(b)q(e)h(to)f(allo)o(w)f(c)o(hildren) h(to)g(call)g Ff(MPI)p 1143 1224 V 14 w(INIT)g Fg(without)g(comm)o (unicating)d(with)j(their)75 1274 y(paren)o(ts,)g(and)g(ha)o(v)o(e)g (another)g(function)g(to)f(obtain)h(the)g(in)o(tercomm)o(unicator:)75 1419 y Ff(MPI)p 152 1419 V 14 w(P)o(a)o(rent\(intercomm\))117 1489 y Fg(OUT)107 b Ff(intercomm)403 b Fg(in)o(tercomm)o(unicator)15 b(in)i(whic)o(h)g(the)h(lo)q(cal)f(group)g(consists)905 1539 y(of)d(one's)g(siblings)g(and)g(the)h(remote)f(group)g(consists)i (of)d(the)905 1589 y(pro)q(cess)j(that)e(did)f(the)i Ff(MPI)p 1360 1589 V 14 w(A)o(ttach)166 1707 y Fg(This)10 b(has)g(the)h(disadv)n(an)o(tage)e(of)g(requiring)h(an)g(extra)g(sync)o (hronization)g(in)g(the)g(c)o(hildren)h(and)f(less)g(straigh)o(t-)75 1756 y(forw)o(ard)h(usage.)17 b(On)12 b(the)f(other)h(hand,)f(similar)e (functionalit)o(y)h(migh)o(t)f(b)q(e)i(necessary)j(when)d(b)q(o)q (otstrapping)h(new)75 1806 y(comm)o(unicators)f(for)j(fault)f(reco)o(v) o(ery)m(.)166 1856 y(Are)g(there)h(cases)g(where)g(c)o(hildren)f(w)o (ould)e(w)o(an)o(t)h(to)h(comm)o(unicate)d(using)i(MPI)h(but)g(not)f(w) o(an)o(t)h(to)f(b)q(e)h(able)75 1906 y(to)h(talk)f(to)h(their)g(paren)o (ts?)166 1956 y(De\014nitely)m(.)20 b(One)c(example)e(is)h(the)g(case)h (where)g(a)f(pro)q(cess)i(manager)c(written)j(in)e(MPI)h(starts)h(pro)q (cesses)75 2005 y(\(with)g Ff(MPI)p 265 2005 V 15 w(EXEC)p Fg(\))f(that)i(comprise)f(an)g(MPI)h(application.)25 b(There)18 b(is)e(no)g(reason)i(for)e(them)g(to)g(con)o(tact)h(the)75 2055 y(pro)q(cess)f(manager.)166 2105 y(I)e(prop)q(ose)g(to)g(mak)o(e)f (this)h(discussion)g(section)h(the)f(main)e(prop)q(osal)h({)h(RL.)166 2327 y Fh(Discussion:)33 b Fg(The)13 b(goal)e(is)i(to)f(establish)h (some)f(metho)q(d)g(of)g(comm)o(unicating)d(with)j(the)i(new)f(pro)q (cesses.)75 2383 y(The)g(alternativ)o(es)g(are)g(to)f(create)i(an)e(in) o(tercomm)o(unicator,)e(to)i(create)i(a)f(con)o(taining)e(comm)o (unicator,)f(or)i(to)g(re-)75 2440 y(size)g(an)e(existing)h(comm)o (unicator.)j(Resizing)d(an)g(existing)f(comm)o(unicator)e(seems)k(a)e (bad)h(idea.)17 b(Giv)o(en)10 b(an)h(in)o(ter-)75 2496 y(comm)o(unicator,)f(it)i(is)h(easy)g(to)g(create)i(a)d(con)o(taining)g (comm)o(unicator)e(b)o(y)j(using)f Ff(MPI)p 1445 2496 V 15 w(INTERCOMM)p 1711 2496 V 14 w(MERGE)p Fg(.)75 2553 y(Therefore)j(w)o(e)f(c)o(ho)q(ose)h(to)f(create)h(an)f(in)o(tercomm)o (unicator.)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 -100 a Fs(22)928 b Fn(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 45 y Fs(The)21 b Fk(MPI)p 350 45 14 2 v 16 w(P)o(a)o(rent)p Fs(/)p Fk(MPI)p 595 45 V 15 w(A)o(ttach)i Fs(scenario)e(allo)o(ws)g(one)g(to)g(manage)f (spa)o(wned)h(pro)q(cesses)h(that)75 102 y(b)q(ecome)f(part)f(of)g(MPI) g(\(b)o(y)g(calling)i Fk(MPI)p 833 102 V 16 w(INIT)p Fs(,)d(as)h(w)o(ell)h(as)f(spa)o(wned)h(pro)q(cesses)g(that)e(nev)o(er) i(call)75 158 y Fk(MPI)p 160 158 V 16 w(INIT)p Fs(.)14 b(The)h(latter)g(can)h(only)f(b)q(e)h(\\comm)o(unicated")f(with)h(via)f Fk(MPI)p 1361 158 V 16 w(SIGNAL)p Fs(.)166 214 y(Note)c(that)f Fk(MPI)p 451 214 V 16 w(SP)l(A)-5 b(WN)12 b Fs(can)g(b)q(e)f(written)h (in)g(terms)e(of)h(the)h(lo)o(w)o(er-lev)o(el)g(routines.)19 b(F)l(or)10 b(example,)123 321 y Fj(MPI\\_SPAWN\(oldcomm,)21 b(resource,)i(num,)g(executable,)f(arguments,)h(newcomm\))75 427 y Fs(can)15 b(b)q(e)h(written)f(as)g(\(without)g(error)g (handling\))123 521 y Fj(MPI_RESOURCE_IALLOCATE\(M)o(PI_RESO)o (URCE_DEF)o(AULT,nu)o(m,"",re)o(source,r)o(equest\))123 577 y(MPI_WAIT\(request,status\))123 634 y(MPI_EXEC\(executable,argu)o (ments,n)o(um,resou)o(rce,arr)o(ay_of_p)o(rocesses)o(\))123 690 y(MPI_ATTACH\(comm,myrank,n)o(um,arra)o(y_of_pro)o(cesses,)o (newcomm)o(\))75 812 y Fm(1.4.2)49 b(Clients)16 b(and)h(Servers)75 898 y Fs(In)22 b(a)e(previous)i(section)g(w)o(e)f(describ)q(ed)i (simpli\014ed,)i(blo)q(c)o(king)d(calls)g(for)f(a)f(clien)o(t)j(and)e (serv)o(er.)37 b(In)75 954 y(general,)15 b(just)f(as)g(MPI)g(needs)i (non-blo)q(c)o(king)g(message-passing)e(op)q(erations,)h(w)o(e)f(need)h (non-blo)q(c)o(king)75 1011 y(connection)h(op)q(erations.)166 1067 y(The)g(new)g(functions)g(needed)h(for)e(clien)o(t-serv)o(er)i (applications)g(b)q(egin)g(with)f(routines)g(needed)h(so)75 1124 y(that)11 b(the)i(clien)o(ts)g(can)g(\014nd)f(the)h(serv)o(er.)18 b(The)13 b(\014rst)e(one)i(is)g(called)g(b)o(y)f(the)h(serv)o(er)f(in)h (order)f(to)f(announce)75 1180 y(that)j(it)i(is)f(ready)g(to)g(accept)g (connections.)21 b(It)15 b(pro)o(vides)g(an)g(arra)o(y)f(of)h(requests) g(that)f(it)h(can)g(test)g(and)75 1236 y(w)o(ait)j(on)g(to)f(tell)i (whether)f(a)g(clien)o(t)h(wishes)g(to)f(connect.)28 b(W)l(e)19 b(assume)e(here)i(that)e(b)q(oth)h(clien)o(t)i(and)75 1293 y(serv)o(er)f(ma)o(y)f(b)q(e)i(parallel)g(programs,)f(whose)g(pro) q(cesses)g(are)g(already)g(in)h(in)o(ternal)g(comm)o(unication)75 1349 y(via)15 b(a)g(comm)o(unicator.)166 1406 y(When)22 b(a)e(connection)j(is)e(established,)j(it)e(will)h(b)q(e)f(returned)f (in)h(a)f(new)h(MPI)f(datat)o(yp)q(e,)h(the)75 1462 y Fk(MPI)p 160 1462 V 16 w(Connection)p Fs(.)28 b(Connections)18 b(are)f(the)g(clien)o(t-serv)o(er)i(analogue)e(of)g(pro)q(cesses)h (whic)o(h)g(ha)o(v)o(e)f(b)q(een)75 1519 y(created)d(b)o(y)f(MPI)h(but) f(with)h(whom)f(comm)o(unication)i(has)e(not)g(y)o(et)g(b)q(een)i (establish.)20 b(After)14 b(establish-)75 1575 y(ing)d(the)g (connection,)i(a)e(separate)f(call)i(actually)g(builds)h(the)e(comm)o (unicator)f(so)h(that)f(message-passing)75 1632 y(can)15 b(tak)o(e)g(place.)166 1688 y(Both)g(of)g(the)g(follo)o(wing)h(op)q (erations)f(are)g(collectiv)o(e)i(o)o(v)o(er)d(the)i(comm)o(unicator)f Fk(comm)p Fs(.)75 1839 y Fk(MPI)p 160 1839 V 16 w(IA)o(CCEPT\(comm,)d (name,)j(num)p 736 1839 V 16 w(connections,)i(a)o(rra)o(y)p 1096 1839 V 15 w(of)p 1148 1839 V 16 w(connections,a)o(rra)o(y)p 1491 1839 V 16 w(of)p 1544 1839 V 16 w(requests\))117 1916 y Fg(IN)155 b Fk(comm)470 b Fg(serv)o(er's)16 b(comm)o(uni)o (cator)117 1991 y(IN)155 b Fk(name)485 b Fg(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)g(whic)o(h)h(the)h(serv)o(er)g(can)f(b)q(e)h(con)o (tacted)905 2048 y(\(string\))117 2123 y(IN)155 b Fk(num)p 404 2123 V 17 w(connections)268 b Fg(n)o(um)o(b)q(er)13 b(of)h(connections)h(in)e(arra)o(y)h(\(in)o(t\))117 2198 y(OUT)108 b Fk(a)o(rra)o(y)p 416 2198 V 15 w(of)p 468 2198 V 16 w(connections)205 b Ff(MPI)p 982 2198 13 2 v 15 w(Connections)11 b Fg(represen)o(ting)h(\\p)q(orts")e(on)g(whic)o (h)h(clien)o(ts)905 2255 y(can)j(connect)117 2330 y(OUT)108 b Fk(a)o(rra)o(y)p 416 2330 14 2 v 15 w(of)p 468 2330 V 16 w(requests)272 b Ff(MPI)p 982 2330 13 2 v 15 w(Request)p Fg(s)15 b(for)f(w)o(aiting/testing)166 2454 y Fs(The)h(next)h(routine)f (is)h(called)h(b)o(y)e(the)g(clien)o(t)i(in)f(order)f(to)f(mak)o(e)h (con)o(tact)f(with)i(the)f(serv)o(er.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 23 24 bop 75 -100 a Fn(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(23)75 45 y Fk(MPI)p 160 45 14 2 v 16 w(ICONT)l(A)o(CT\(comm,)13 b(name,)h(connection,)j(request\))117 122 y Fg(IN)155 b Fk(comm)470 b Fg(clien)o(t's)14 b(comm)o(unicator)117 197 y(IN)155 b Fk(name)485 b Fg(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)g (whic)o(h)h(the)h(serv)o(er)g(can)f(b)q(e)h(con)o(tacted)905 254 y(\(string\))117 329 y(OUT)108 b Fk(connection)385 b Ff(MPI)p 982 329 13 2 v 15 w(Connection)15 b Fg(for)e(the)i(serv)o (er)117 404 y(OUT)108 b Fk(request)452 b Fg(request)13 b(that)f(is)f(satis\014ed)h(when)g(a)f(serv)o(er)i(accepts)g(the)f (con-)905 461 y(nection)166 585 y Fs(The)f(ab)q(o)o(v)o(e)f(are)h(b)q (oth)g(collectiv)o(e,)i(non-blo)q(c)o(king)g(calls,)f(to)e(allo)o(w)h (eac)o(h)g(collection)i(of)d(pro)q(cesses)i(to)75 642 y(o)o(v)o(erlap)i(computation)g(with)h(the)g(p)q(ossibly)g (time-consuming)h(task)e(of)g(establishing)i(the)e(connection.)166 781 y Fh(Discussion:)42 b Fg(Building)14 b(in)i(a)f(certain)i(lev)o(el) e(of)g(securit)o(y)i(in)o(to)e(the)h(clien)o(t-serv)o(er)i(connection)e (pro)q(cess)75 837 y(could)d(b)q(e)g(done)g(in)f(the)i(follo)o(wing)c (w)o(a)o(y)m(.)16 b(An)d(extra)g(argumen)o(t)f(to)h Ff(MPI)p 1211 837 V 14 w(IA)o(CCEPT)d Fg(w)o(ould)i(consist)i(of)e(the)h(name)75 894 y(of)j(a)h(function)f(to)h(b)q(e)g(called)g(to)g(pro)o(vide)f(v)n (alidation)f(for)h(the)i(clien)o(t)f(that)f(is)h(attempting)f(to)g (connect.)29 b(The)75 950 y(clien)o(t,)13 b(in)g(an)g(extra)g(argumen)o (t)f(on)h(the)h(call)f(to)g Ff(MPI)p 925 950 V 14 w(ICONT)m(A)o(CT)p Fg(,)e(w)o(ould)h(pro)o(vide)h(a)g(k)o(ey)h(that)f(w)o(as)g(v)n (alidated)75 1006 y(b)o(y)k(the)g(routine)h(passed)g(to)f Ff(MPI)p 623 1006 V 14 w(IA)o(CCEPT)p Fg(.)d(If)j(the)h(k)o(ey)f(w)o (as)g(not)g(v)n(alidated,)f(the)h(request)i(w)o(ould)d(ha)o(v)o(e)h(no) 75 1063 y(e\013ect)f(on)d(the)i(requests)h(supplied)e(to)f Ff(MPI)p 766 1063 V 15 w(IA)o(CCEPT)p Fg(.)166 1202 y Fs(It)18 b(is)g(useful)g(if)g(it)g(is)g(not)f(required)i(that)e(the)h Fk(name)f Fs(argumen)o(t)g(to)g(the)g(ab)q(o)o(v)o(e)g(calls)i(b)q(e)f (kno)o(wn)75 1259 y(un)o(til)e(execution)g(time.)k(The)c(follo)o(wing)f (routine)h(pro)o(vides)f(a)g(w)o(a)o(y)f(for)g(the)h(application)i(to)d (request)h(a)75 1315 y Fk(name)p Fs(,)f(but)i(b)q(e)g(giv)o(en)f(a)g (di\013eren)o(t)h(one)f(if)h(the)f(system)g(prefers)g(it)g(that)g(w)o (a)o(y)l(.)75 1466 y Fk(MPI)p 160 1466 14 2 v 16 w(GET)p 264 1466 V 17 w(SERVER)p 448 1466 V 17 w(NAME\(requested)p 790 1466 V 19 w(name,)g(given)p 1037 1466 V 16 w(name\))117 1543 y Fg(IN)155 b Fk(requested)p 499 1543 V 19 w(name)288 b Fg(name)13 b(that)h(the)g(serv)o(er)i(w)o(ould)d(lik)o(e)g(to)h(b)q (e)g(kno)o(wn)g(b)o(y)117 1618 y(OUT)108 b Fk(given)p 419 1618 V 17 w(name)370 b Fg(name)13 b(supplied)h(b)o(y)g(the)g (system)166 1743 y Fs(In)23 b(order)f(for)f(comm)o(unication)i(to)e (tak)o(e)h(place,)i(an)e(in)o(ter-comm)o(unicator)h(m)o(ust)e(b)q(e)i (created)75 1799 y(connecting)15 b(the)f(clien)o(t)h(with)f(the)f(serv) o(er.)19 b(The)14 b(follo)o(wing)h(routine)f(serv)o(es)f(this)h(purp)q (ose.)20 b(It)14 b(is)g(called)75 1856 y(b)o(y)h(b)q(oth)g(the)h(clien) o(t)g(and)g(the)f(serv)o(er.)75 2007 y Fk(MPI)p 160 2007 V 16 w(REMOTE)p 363 2007 V 17 w(A)l(TT)l(A)o(CH\(oldcomm,)f(num)p 856 2007 V 16 w(connections,)j(a)o(rra)o(y)p 1216 2007 V 15 w(of)p 1268 2007 V 15 w(connections,)g(new)o(comm\))117 2084 y Fg(IN)155 b Fk(oldcomm)413 b Fg(comm)o(unicator)117 2159 y(IN)155 b Fk(num)p 404 2159 V 17 w(connections)268 b Fg(n)o(um)o(b)q(er)13 b(of)h(connections)h(in)e(arra)o(y)h(\(in)o (t\))117 2234 y(IN)155 b Fk(a)o(rra)o(y)p 416 2234 V 15 w(of)p 468 2234 V 16 w(connections)205 b Fg(the)16 b Ff(MPI)p 1055 2234 13 2 v 14 w(Connection)p Fg(s)g(represen)o(ting)h (the)e(pro)q(cesses)j(to)c(b)q(e)905 2291 y(attac)o(hed)h(to)117 2366 y(IN)155 b Fk(new)o(comm)397 b Fg(new)15 b(in)o(ter-comm)o (unicator)c(for)i(the)i(new)f(pro)q(cesses)75 2490 y Fs(This)20 b(op)q(eration)g(is)g(collectiv)o(e)h(o)o(v)o(er)e Fk(oldcomm)p Fs(,)g(and)g(returns)h(an)f(in)o(ter-comm)o(unicator.)33 b(One)21 b(can)75 2547 y(think)f(of)f(this)g(op)q(eration)h(as)e(v)o (ery)h(m)o(uc)o(h)g(lik)o(e)i Fk(MPI)p 1016 2547 14 2 v 15 w(INTERCOMM)p 1305 2547 V 17 w(CREA)l(TE)p Fs(,)f(where)f(the)h (remote)75 2603 y(pro)q(cesses)c(are)e(represen)o(ted)i(b)o(y)f(an)g (arra)o(y)f(of)h(requests)g(rather)g(than)g(b)o(y)g(another)g(group.) -32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 24 25 bop 75 -100 a Fs(24)933 b Fn(CHAPTER)16 b(1.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 45 y Fh(Discussion:)g Fg(One)f(option)e(w)o(ould)g(b)q(e)h(to)g(com)o(bine)f(this)h(call)f (with)h(the)g(previous)g(ones,)h(but)f(that)g(w)o(ould)75 102 y(require)19 b(that)g(the)g(in)o(ter-comm)o(unicator)d(b)q(eing)i (constructed)j(b)q(e)e(in)f(an)g(unde\014ned)i(state)f(b)q(et)o(w)o (een)h(the)f(re-)75 158 y(quest)h(and)f(the)h(time)f(that)g(the)h (requests)h(w)o(ere)g(satis\014ed.)35 b(Still)18 b(another)i(option)e (w)o(ould)h(b)q(e)h(to)f(ha)o(v)o(e)g(the)75 214 y(in)o(ter-comm)o (unicator)e(returned)k(as)e(part)g(of)g(the)h Ff(status)f Fg(ob)r(ject)i(on)e(the)h Ff(MPI)p 1376 214 13 2 v 14 w(W)m(AIT)p Fg(.)e(W)m(e)h(ha)o(v)o(e)g(c)o(hosen)h(to)75 271 y(mak)o(e)13 b(separate)k(the)e(pro)q(cess)i(of)d(constructing)i (the)g(in)o(ter-comm)o(unicator)c(after)k(the)f Ff(MPI)p 1547 271 V 15 w(W)m(AIT)p Fg(s)f(ha)o(v)o(e)h(b)q(een)75 327 y(completed.)166 460 y Fs(The)j(sequence)g(of)f(ev)o(en)o(ts)g(for) g(a)g(sequen)o(tial)i(clien)o(t)g(con)o(tacting)e(a)g(parallel)i(serv)o (er)e(migh)o(t)h(lo)q(ok)75 510 y(lik)o(e)e(this:)337 594 y Fe(Client)783 b(Server)337 644 y(------)g(------)75 694 y(MPI_Icontact\(serv)o(er_na)o(me,co)o(nn,re)o(quest)o(\))84 b(MPI_Iaccept\(myname,)o(num,c)o(onns,)o(requ)o(ests\))75 743 y(MPI_Wait\(request,)o(statu)o(s\))389 b(MPI_Waitsome\(num,re)o (quest)o(s,num)o(read)o(y,)1208 793 y(which,statuses\))75 843 y(MPI_Remote_attach)o(\(comm)o(,1,co)o(nn,ne)o(wcomm)o(\))84 b(MPI_Remote_attach\(c)o(omm,1)o(,conn)o(s[wh)o(ich],)1448 893 y(newcomm\))75 943 y(\()22 b(MPI)f(communication)e(in)i(newcomm)g (\))217 b(\()22 b(MPI)f(communication)e(in)j(newcomm)e(\))75 992 y(MPI_Comm_free\(new)o(comm\))432 b(MPI_Comm_free\(newco)o(mm\))75 1042 y(MPI_Finalize\(\))607 b(\(process)20 b(other)h(requests,)f(loop)h (back)1056 1092 y(to)g(accept)g(again\))75 1142 y(\(exit\))166 1233 y Fs(Just)g(as)f(MPI)g(allo)o(ws)h(a)f(non)o(blo)q(c)o(king)i (send)f(to)f(b)q(e)i(matc)o(hed)e(with)h(a)f(blo)q(c)o(king)i(receiv)o (e)g(and)75 1289 y(vice)16 b(v)o(ersa,)e(the)h(ab)q(o)o(v)o(e)g (non-blo)q(c)o(king)i(op)q(erations)e(ma)o(y)f(b)q(e)i(matc)o(hed)f (with)g(their)h(simpli\014ed)h(forms,)75 1346 y Fk(MPI)p 160 1346 14 2 v 16 w(CLIENT)p 333 1346 V 16 w(CONNECT)f Fs(and)f Fk(MPI)p 748 1346 V 16 w(SERVER)p 931 1346 V 18 w(CONNECT)p Fs(.)75 1488 y Fp(1.5)59 b(Examples)75 1590 y Fs(These)17 b(Examples)h(are)e(no)h(longer)g(v)m(alid.)27 b(I)17 b(am)g(w)o(orking)f(on)h(con)o(v)o(erting)g(them)g(to)f(matc)o (h)h(the)g(new)75 1646 y(v)o(ersion)e(of)g(the)g(prop)q(osal.)21 b({)14 b(RL)75 1766 y Fk(Manager-w)o(o)o(rk)o(er)f(example.)75 1857 y Fj(/*)24 b(manager)e(*/)75 1913 y(#include)h("mpi.h")75 2026 y(main\(int)g(argc,)g(char)g(*argv[]\))75 2083 y({)123 2139 y(int)g(n;)123 2195 y(MPI_Resource)f(resources[MAXPROC];)123 2252 y(MPI_Process)g(processes[MAXPROC];)123 2308 y(MPI_Comm)g (my_children,)h(my_family;)123 2365 y(Resource_Description)e(descrip;) 123 2421 y(MPI_Status)h(status;)123 2478 y(MPI_Request)g(*request;)123 2591 y(MPI_Init\(&argc,)g(&argv\);)123 2647 y(...)h(/*)h(set)f(up)h (resource)f(description)f(here.)h(It)h(includes)290 2704 y(a)f(hard)h(or)f(soft)h(number)f(of)g(resources)47 b(*/)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 25 26 bop 75 -100 a Fn(1.5.)29 b(EXAMPLES)1398 b Fs(25)123 45 y Fj(MPI_Iallocate\(descrip,)21 b(resources,)h(request\);)123 102 y(MPI_Wait\(request,&status)o(\);)123 158 y(n)h(=)h()123 214 y (MPI_Exec\("worker",)d(NULL,)i(resources,)g(processes\);)123 271 y(/*)g(could)g(check)h(for)f(errors)g(here.)g(Assume)g(all)h(n)g (succeed)e(*/)123 384 y(MPI_Attach\(MPI_COMM_SELF)o(,)f(0,)j (processes,)e(n,)i(&my_children\);)123 440 y(/*)147 497 y(*)f(parallel)g(code)g(here.)h(the)f(intercommunicator)f ("my_children")147 553 y(*)h(can)h(be)f(used)h(to)f(communicate)g(with) g(the)g(spawned)g(processes)147 610 y(*/)123 723 y(MPI_Finalize\(\);)75 779 y(})75 892 y(/*)h(worker)f(*/)75 948 y(main\(int)g(argc,)g(char)g (*argv[]\))75 1005 y({)123 1118 y(MPI_Init\(&argc,)f(&argv\);)123 1174 y(if)h(\(MPI_COMM_PARENT)f(==)i(MPI_COMM_NULL\))e({)194 1231 y(error\("Where's)g(my)i(mommy?"\);)194 1287 y(MPI_Abort\(\);)123 1344 y(})123 1400 y(/*)f(parallel)g(code)g(here.)h(MPI_COMM_PARENT)d (is)j(an)g(intercommunicator)147 1456 y(*)f(that)h(allows)f(all)g (resources)g(to)g(talk)h(to)f(their)g(parent)g(\(in)h(this)f(case,)147 1513 y(*)g(the)h(single)f(manager)g(process)g(that)g(spawned)g(them\).) 147 1569 y(*/)123 1682 y(MPI_Finalize\(\);)75 1739 y(})75 1859 y Fk(T)l(ask)12 b(fa)o(rm)e(example.)45 b Fs(The)12 b(nice)h(thing)g(here)f(is)g(that)f(y)o(ou)h(need)h(allo)q(cate)f (resources)g(only)g(once.)19 b(Then)75 1915 y(y)o(ou)c(can)g(reuse)g (them.)20 b(It)15 b(also)g(sho)o(ws)f(that)g(y)o(ou)h(can)g(in)o (teract)g(with)g(pro)q(cesses)g(without)g(using)h(MPI)75 1972 y(to)f(comm)o(unicate.)75 2078 y Fj(/*)24 b(manager)e(*/)75 2134 y(#include)h("mpi.h")75 2247 y(main\(int)g(argc,)g(char)g (*argv[]\))75 2304 y({)123 2360 y(int)g(i,)h(n,)f(nthings_done;)123 2417 y(MPI_Resource)f(resources[20];)123 2473 y(MPI_Process)g (processes[20];)123 2530 y(MPI_Request)g(*request[20];)123 2586 y(Thingtodo)g(tbd[NTHINGSTODO];)123 2643 y(MPI_Status)g(status,)h (statuses[20];)123 2699 y(char)g(**args;)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 26 27 bop 75 -100 a Fs(26)933 b Fn(CHAPTER)16 b(1.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)123 45 y Fj(Resource_Description)21 b(descrip;)123 158 y(MPI_Init\(&argc,)h(&argv\);)123 214 y(...)h(/*)h(set)f(up)h(resource)f(description)f(here.)h(For)h (this)f(example,)290 271 y(assume)g(it)g(is)h(a)g(hard)f(request,)g(so) g(we)h(will)f(get)h(20)f(or)h(0)g(resources)e(*/)123 327 y(MPI_Iallocate\(descrip,)f(resources,)h(&request\);)123 384 y(MPI_Wait\(request\);)123 497 y(/*)h(set)h(up)f(tbd)h(array)f (with)g(everything)g(that)g(needs)g(to)h(be)f(done)h(*/)123 553 y(/*)f(assume)g(NTHINGSTODO>20)f(*/)123 666 y(for)h(\(i)h(=)f(0;)h (i)g(<)g(20;)f(i++\))g({)170 723 y(my_set_args\(tbd[i],)f(&args\);)170 779 y(MPI_Exec\("cow",)g(args,)h(&resource[i],)g(&processes[i]\);)170 835 y(MPI_Notify_when_done\(&process)o([i],)e(1,)i(&request[i]\);)123 892 y(})123 948 y(nthings_done)f(=)i(20;)123 1061 y(/*)f(at)h(this)f (point)g(we)h(have)f(20)h(processes)f(running.)f(wait)i(for)194 1118 y(a)g(process)f(to)h(finish)f(and)g(start)g(a)h(new)f(one)h(in)f (its)h(place)194 1174 y(until)f(we've)h(exhausted)e(the)i(list)f(of)h (things)f(to)g(do)147 1231 y(*/)123 1344 y(while)g(\(nthings_done)f(<)i (NTHINGSTODO\))e({)170 1400 y(my_set_args\(tbd[nthings_done])o(,)f (&args\);)170 1456 y(MPI_Waitany\(20,)h(request,)h(&n,)g(status\);)170 1513 y(MPI_Exec\("worker",)f(args,)h(&resource[n],)f(&processes[n]\);) 170 1569 y(MPI_Notify_when_done\(&process)o([n],)f(1,)i(&request[n]\);) 170 1626 y(nthings_done++;)123 1682 y(})123 1795 y(/*)g(no)h(more)f (things)g(to)h(do,)f(but)h(we've)f(got)g(20)h(processes)f(still)g (running)g(*/)123 1852 y(MPI_Waitall\(20,)f(request,)g(statuses\);)123 1908 y(MPI_Finalize\(\);)75 1965 y(})75 2085 y Fk(Another)c(task)f(fa)o (rm)e(example,)h(in)h(old)g(syntax.)46 b Fs(It)16 b(alw)o(a)o(ys)g(k)o (eeps)h(a)f(request)h(for)f(ten)h(resources)f(out-)75 2141 y(standing,)k(but)g(starts)e(jobs)h(as)g(so)q(on)h(as)f(p)q (ossible.)34 b(T)l(o)19 b(a)o(v)o(oid)g(spin-w)o(aits)h(on)g(the)f (allo)q(cation)i(and)75 2198 y(running)h(of)d(jobs,)j(it)e(uses)h Fk(MPI)p 658 2198 14 2 v 16 w(W)l(AITSOME)g Fs(on)f(an)g(arra)o(y)f(of) h(requests)h(that)e(includes)k(b)q(oth)d(al-)75 2254 y(lo)q(cation)g(requests)f(and)h(started)e(jobs.)32 b(The)19 b(index)i Fj(alloc)p 1155 2254 15 2 v 16 w(top)e Fs(giv)o(es)g(the)g(n) o(um)o(b)q(er)h(of)f(allo)q(cation)75 2310 y(requests)c(curren)o(tly)h (activ)o(e;)e Fj(r)p 616 2310 V 17 w(top)h Fs(giv)o(es)g(the)g(total)g (n)o(um)o(b)q(er)g(of)g(activ)o(e)g(requests)g(\(b)q(oth)g(allo)q (cations)75 2367 y(and)g(started)g(pro)q(cesses\).)166 2423 y(The)k(programs)f(in)i(this)f(example)h(are)f Fl(not)f Fs(MPI)h(jobs;)h(MPI)f(is)h(simply)g(b)q(eing)g(used)g(to)e(start)75 2480 y(and)f(manage)g(the)g(programs.)23 b(F)l(or)17 b(simplicit)o(y)l(,)i(w)o(e)e(ha)o(v)o(e)f(not)h(included)i(an)o(y)e (co)q(de)h(to)e(decide)i(when)75 2536 y(the)g(program)f(is)i(done)f(or) g(to)g(describ)q(e)h(the)g(program)e(to)g(b)q(e)i(run)f(and)h(its)f (argumen)o(ts.)28 b(Note)18 b(that)75 2593 y Fk(MPI)p 160 2593 14 2 v 16 w(CANCEL)d Fs(can)h(b)q(e)g(used)f(to)g(cancel)h(an) o(y)f(unneeded)i(allo)q(cation)f(requests.)75 2699 y Fj(#include)23 b("mpi.h")1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 27 28 bop 75 -100 a Fn(1.5.)34 b(EXAMPLES)1393 b Fs(27)75 45 y Fj(main\()23 b(int)h(argc,)f(char)g(**argv)g(\))75 102 y({)75 158 y(MPI_Request)f(r[20];)75 214 y(MPI_Status)46 b(s[20];)75 271 y(int)214 b(idx[20],)23 b(nout;)75 327 y(int)214 b(alloc_top,)23 b(r_top;)75 384 y(int)214 b(rc;)75 553 y(MPI_Iallocate\()22 b(10,)h(\(char)h(*\)0,)f(\(char)g(**\)0,)g ("*",)h(\(char)f(**\)0,)g(MPI_HARD,)g(r)g(\);)75 610 y(alloc_top)g(=)g(10;)75 666 y(r_top)119 b(=)23 b(10;)75 723 y(while)g(\(!done\))g({)170 779 y(MPI_Waitsome\()g(r_top,)g(r,)g (&nout,)g(idx,)h(s)f(\);)170 835 y(for)h(\(i=0;)f(i; Thu, 13 Jul 1995 12:21:34 -0500 Received: from snacker.pnl.gov by pnl.gov (PMDF V4.3-13 #6012) id <01HSTGP8L4PS8Y617U@pnl.gov>; Thu, 13 Jul 1995 10:21:12 -0700 (PDT) Received: by snacker.pnl.gov (4.1/SMI-4.1) id AA19208; Thu, 13 Jul 95 10:21:09 PDT Date: Thu, 13 Jul 1995 10:21:09 -0700 From: j_nieplocha@pnl.gov Subject: Re: non-blocking ops vs. threads In-reply-to: Your message of "Wed, 12 Jul 1995 20:16:24 EDT." <9507130016.AA25755@snir.watson.ibm.com> To: snir@watson.ibm.com Cc: mpi-core@mcs.anl.gov Message-id: <9507131721.AA19208@snacker.pnl.gov> Content-transfer-encoding: 7BIT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk From: Robert George : > >> > >> Puts are required for active messages, in which many people in this group > >> (myself included) are very interested. From: Marc Snir : > I don't understand this statement. I know a fair number of systems that sup > port active messages but do not support puts, and vice-versa. In any case, > the issue is not what are the basic communication mechanisms on the underlying > machine, the issue is what is made visible by MPI to the user. We can decide > to provide an active message interface, we can decide to provide a put/get > interface, we can decide to provide both. Implementers may, at their > convenience, use one to implemente the other. Marc, The Active Message package from Berkeley, that provides run-time support for the Split C compiler, now comes with put and get functionality in addition to the regular active messages. I also have a few comments to your statement that one mechanism can be used to implement the other. Neglecting performance issues, active messages as a mechanism more general than put/get could be certainly used to implement get/put interface. On the other hand, an attempt to implement active messages with put/get alone could be quite entertaining. I believe that the need for active messages in addition to put/get arises from the fact that put and get are not the only form of remote access that people need. For example, in case of chemistry codes that we support at PNL, we have scatter/gather, atomic daxpy-like operation, atomic read-and-increment in additional to regular gets/puts. Although, all of these operations could be implemented with put/get in addition to some atomic operation (like defined in your proposal), on certain platforms the hrecv/AM implementation could be faster. This might sound controversial, concerning how much performance of interrupt- receive deteriorated on some recent parallel machines ;-) however, I can provide specific examples. One might argue that if we have hrecv/AM as powerful mechanisms to implement other forms of remote access, gets/puts are not needed. The reason why get/put in addition to RPC are still useful is that on some shared/global memory platforms (like SGI or T3D) get/put could be implemented with extremely low overhead as comparing to cost of hrecv/AM. Thus, for the performance purpose, we should have interfaces to both. --Jarek Nieplocha From mpi-core-human@mcs.anl.gov Fri Jul 14 01:59:07 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id BAA14884; Fri, 14 Jul 1995 01:59:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA07060 for mpi-core-out; Fri, 14 Jul 1995 01:07:25 -0500 Received: from hitiij.hitachi.co.jp (hitiij.hitachi.co.jp [133.145.224.3]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA07052 for ; Fri, 14 Jul 1995 01:07:17 -0500 Received: from softgw ([158.214.44.2]) by hitiij.hitachi.co.jp (8.6.11+2.4W/3.3W9-hitiij) with SMTP id PAA01191 for ; Fri, 14 Jul 1995 15:07:11 +0900 Received: from [158.214.40.7] by softgw (5.61/1.01) id AA05626; Fri, 14 Jul 95 15:06:48 +0900 Received: by hisoft.soft.hitachi.co.jp (5.61/5.18) id AA21209; Fri, 14 Jul 95 15:04:29 +0900 Date: Fri, 14 Jul 95 15:04:29 +0900 From: haradake@hisoft.soft.hitachi.co.jp (Kei_Harada) Message-Id: <9507140604.AA21209@hisoft.soft.hitachi.co.jp > To: mpi-core%mcs.anl.gov@hitiij.hitachi.co.jp Subject: Would you do me a favoe? Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear MPI participants, My name is Kei Harada from Hitachi SDC, and is planning to attend next MPIF Meeting at Chicago with Mr. Sagawa, researcher of our Central Research Lab. However both Mr. Sagawa and I have to leave Japan this week to accomplish other overseas business. therefore we will miss email access over a week. Would you make us a favor, that if you will post a new proposal on the mailing list, could you bring hardcopy all the way to Chicago? I don't care about individual discussion, new proposal only. I am looking forward to meeting you at Chicago. Best Regards, Kei Harada Software Development Center, Hitachi, Ltd. Tel : +81-45-862-8722 Fax : +81-45-865-9011 email: haradake@soft.hitachi.co.jp From mpi-core-human@mcs.anl.gov Fri Jul 14 02:10:22 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA20387; Fri, 14 Jul 1995 02:10:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA07148 for mpi-core-out; Fri, 14 Jul 1995 01:18:51 -0500 Received: from hitiij.hitachi.co.jp (hitiij.hitachi.co.jp [133.145.224.3]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA07140 for ; Fri, 14 Jul 1995 01:18:44 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by hitiij.hitachi.co.jp (8.6.11+2.4W/3.3W9-hitiij) with ESMTP id PAA02473 for ; Fri, 14 Jul 1995 15:18:40 +0900 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) id XAA26780; Thu, 13 Jul 1995 23:18:35 -0700 Received: by mrjones.engr.sgi.com (940816.SGI.8.6.9/940406.SGI.AUTO) id XAA02387; Thu, 13 Jul 1995 23:18:32 -0700 From: "Eric Salo" Message-Id: <9507132318.ZM2385@mrjones.engr.sgi.com> Date: Thu, 13 Jul 1995 23:18:29 -0700 In-Reply-To: haradake@hisoft.soft.hitachi.co.jp (Kei_Harada) "Would you do me a favoe?" (Jul 14, 3:04pm) References: <9507140604.AA21209@hisoft.soft.hitachi.co.jp > X-Mailer: Z-Mail (3.2.1 6apr95 MediaMail) To: haradake@hisoft.soft.hitachi.co.jp (Kei_Harada), mpi-core%mcs.anl.gov@hitiij.hitachi.co.jp Subject: Re: Would you do me a favor? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk It is common practice for proposals to be distributed as hardcopy at each meeting, so you're safe. Looking forward to seeing you in Chicago. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Jul 14 10:45:20 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA05384; Fri, 14 Jul 1995 10:45:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA12767 for mpi-core-out; Fri, 14 Jul 1995 09:53:03 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA12748; Fri, 14 Jul 1995 09:52:47 -0500 Message-Id: <199507141452.JAA12748@antares.mcs.anl.gov> To: raja@alex.osc.edu (Raja Daoud) cc: mpi-core@mcs.anl.gov Subject: Re: why "offline polling" is needed -- at least on parallel machines In-Reply-to: Message from raja@alex.osc.edu (Raja Daoud) of "Thu, 13 Jul 1995 06:41:14 -0400." <9507131041.AA07688@alex.osc.edu> Date: Fri, 14 Jul 1995 09:52:44 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I'd like to add that I agree with Raja and Greg. While Marc is correct that someone has to pull data off the network (though relying on SIGALRM is a perscription for disaster, since the user may replace SIGALRM for any number of good reasons), FORCING MPI to be integrated with the system at this level can be difficult or impossible, because of the degree of integration that is required with parts of the system (OS, device drivers, etc) that a vendor may wish to keep "pure". I am particularly concerned about multiplatform implementations; these currently are implemented by third parties (e.g., ANL, OSC, EPCC) and often can NOT be tightly integrated with the lowest levels of the network. In our case, we can not (and we have tried!) get access to those parts of the system for our main MPP. In addition, the cost of the interrupts should not be underestimated; the vendor's proprietary message passing for our MPP has an "hrecv" with very high negative performance impact on the system (this is not meant as a criticism; it IS expensive). At the same time, I believe that it is important MPI not preclude access to these features where available, particularly since I believe that the message-passing (two-sided) model is a good way to establish the parameters (e.g., addresses) for one-sided operations. Bill From mpi-core-human@mcs.anl.gov Fri Jul 14 12:52:59 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA06363; Fri, 14 Jul 1995 12:52:59 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA16106 for mpi-core-out; Fri, 14 Jul 1995 12:00:35 -0500 Received: from pnl.gov (gate.pnl.gov [130.20.64.36]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA16098; Fri, 14 Jul 1995 12:00:27 -0500 Received: from maddog.pnl.gov by pnl.gov (PMDF V4.3-13 #6012) id <01HSUU9E8PV48Y6GUO@pnl.gov>; Fri, 14 Jul 1995 10:00:05 -0700 (PDT) Received: by maddog.pnl.gov (4.1/SMI-4.1) id AA11320; Fri, 14 Jul 95 10:00:03 PDT Date: Fri, 14 Jul 1995 10:00:03 -0700 (PDT) From: Rik Littlefield Subject: RE: why "offline polling" is needed -- at least on parallel machines To: gropp@mcs.anl.gov, raja@alex.osc.edu Cc: mpi-core@mcs.anl.gov, rj_littlefield@pnl.gov Message-id: <9507141700.AA11320@maddog.pnl.gov> Content-transfer-encoding: 7BIT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Bill Gropp writes: > ... In addition, the cost of the interrupts should not be > underestimated; the vendor's proprietary message passing for our MPP has an > "hrecv" with very high negative performance impact on the system (this is not > meant as a criticism; it IS expensive). Can Bill and/or others please clarify what is meant by the "cost" of interrupts? E.g., can you address the following possibilities (and feel free to add others based on your insight): 1. Interrupt-receive is expensive when invoked, but otherwise does not affect the performance of conventional message passing. 2. The mere possibility of interrupt-receive hurts the performance of conventional message passing. Why? How much? Under what assumptions? But again, the main need of my group is for remote data access, meaning get/put/accumulate with some usable mutex/synchronization mechanism. So by analogy, I would like to see the following possibilities addressed: 3. Remote data access costs something when invoked, but otherwise does not affect the performance of conventional message passing. 4. The mere possibility of remote data access hurts the performance of conventional message passing. Why? How much? Under what assumptions? --Rik ---------------------------------------------------------------------- rj_littlefield@pnl.gov (alias 'd39135') Rik Littlefield Tel: 509-375-3927 Pacific Northwest Lab, MS K7-15 Fax: 509-375-3641 P.O.Box 999, Richland, WA 99352 From mpi-core-human@mcs.anl.gov Fri Jul 14 13:33:52 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA06639; Fri, 14 Jul 1995 13:33:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA16780 for mpi-core-out; Fri, 14 Jul 1995 12:39:19 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA16772; Fri, 14 Jul 1995 12:39:05 -0500 Message-Id: <199507141739.MAA16772@antares.mcs.anl.gov> To: Rik Littlefield cc: raja@alex.osc.edu, mpi-core@mcs.anl.gov Subject: Re: why "offline polling" is needed -- at least on parallel machines In-Reply-to: Message from Rik Littlefield of "Fri, 14 Jul 1995 10:00:03 -0700." <9507141700.AA11320@maddog.pnl.gov> Date: Fri, 14 Jul 1995 12:39:04 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | Bill Gropp writes: | | > ... In addition, the cost of the interrupts should not be | > underestimated; the vendor's proprietary message passing for our MPP has an | > "hrecv" with very high negative performance impact on the system (this is not | > meant as a criticism; it IS expensive). | | Can Bill and/or others please clarify what is meant by the "cost" | of interrupts? | | E.g., can you address the following possibilities (and feel free to | add others based on your insight): | | 1. Interrupt-receive is expensive when invoked, but otherwise | does not affect the performance of conventional message passing. | | 2. The mere possibility of interrupt-receive hurts the performance | of conventional message passing. | | Why? How much? Under what assumptions? For example, If the implementation is otherwise single-threaded (and for single-threaded use), then, in the absence of hrecv, it may be able to avoid taking special precautions for mutual exclusion/locks/etc. On a number of systems, adding the locks can be expensive, particularly when you're trying to get latency down. Bill From mpi-core-human@mcs.anl.gov Fri Jul 14 14:12:26 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA06881; Fri, 14 Jul 1995 14:12:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA17587 for mpi-core-out; Fri, 14 Jul 1995 13:17:25 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA17574; Fri, 14 Jul 1995 13:17:14 -0500 Message-Id: <199507141817.NAA17574@antares.mcs.anl.gov> To: haradake@hisoft.soft.hitachi.co.jp (Kei_Harada) Cc: mpi-core@antares.mcs.anl.gov Subject: Re: Would you do me a favoe? In-reply-to: Your message of "Fri, 14 Jul 1995 15:04:29 +0900." <9507140604.AA21209@hisoft.soft.hitachi.co.jp > Date: Fri, 14 Jul 1995 13:17:12 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk We always try to have hardcopy of everything available at the meetings if we get it in time. Rusty From mpi-core-human@mcs.anl.gov Sat Jul 15 21:02:39 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA18667; Sat, 15 Jul 1995 21:02:39 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA05587 for mpi-core-out; Sat, 15 Jul 1995 20:09:51 -0500 Received: from N2.SP.CS.CMU.EDU (N2.SP.CS.CMU.EDU [128.2.250.82]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id UAA05575 for ; Sat, 15 Jul 1995 20:09:44 -0500 From: Dave.OHallaron@N2.SP.CS.CMU.EDU Date: Sat, 15 Jul 1995 20:42-EDT To: Tony Skjellum Cc: mpi-core@mcs.anl.gov Subject: Re: MPI_Allput() initial proposal + discussion Message-Id: <805855353/droh@N2.SP.CS.CMU.EDU> In-Reply-To: Tony Skjellum's mail message of Fri, 14 Jul 95 17:07:24 CDT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Tony, Thanks for introducing the Allput pattern to the discusssion. The Allput is the key communication pattern in every program generated by our Fx compiler (Fx is a dialect of HPF that integrates task and data parallelism. See http://www.cs.cmu.edu/~fx for more info and example codes). The Allput is the technique we use for compiling array assignment statements, and thus could be important for any HPF compiler that targets MPI. We have measured big performance improvements over sends and receives using the Allput pattern (we call it direct deposit message passing). The fundamental reason for the performance improvement is that the Allput separates synchronization and data transfer, which allows us then to replace p^2 potential synchronizations (e.g. one synch for each message in a complete exchange induced by assigning a BLOCK distributed HPF array to a CYCLIC distributed array) with a single logn barrier. The idea is discussed in more detail (along with measured T3D numbers) in T. Stricker, J. Stichnoth, D. O'Hallaron, S. Hinrichs, and T. Gross. Decoupling Synchronization and Data Transfer in Message Passing Systems of Parallel Computers. In Proc. of the 9th International Conference on Supercomputing, ACM, Barcelona, Spain, July, 1995. The postscript is available from the Fx Web page at http://www.cs.cmu.edu/~fx Dave O'Hallaron From mpi-core-human@mcs.anl.gov Mon Jul 17 15:02:12 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA13209; Mon, 17 Jul 1995 15:02:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA01230 for mpi-core-out; Mon, 17 Jul 1995 14:08:51 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA01209 for ; Mon, 17 Jul 1995 14:08:33 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2361; Mon, 17 Jul 95 15:07:31 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 0043; Mon, 17 Jul 1995 15:07:30 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 17 Jul 95 15:07:30 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA27546; Mon, 17 Jul 1995 15:07:09 -0400 Message-Id: <9507171907.AA27546@snir.watson.ibm.com> To: mpi-core@mcs.anl.gov Cc: lederman@super.org, pratap@watson.ibm.com Subject: proposal of S-H lederman for external interfaces (MPI_REQUEST_REGISTER) Reply-To: snir@watson.ibm.com Date: Mon, 17 Jul 1995 15:07:08 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: It is not clear to me, in this proposal, how MPI figures out which callback function it should invoke when a registered request completes. I would suggest to provide this information with the registration. This may also make registration id's superfluous. Thus MPI_REQUEST_REGISTER(request, extra_state, comm, status, callback_fn). When the communication request ``request'' completes, MPI sets ``status'' and invokes ``callback_func'' with arguments request, extra_state, comm. No changes in the definitions of MPI_REQUEST_TYPE_CREATE MPI_REQUEST_INIT MPI_Request_start_fn MPI_Request_free_fn The continue function is of type typedef int MPI_Request_continue_fn(MPI_Comm comm, MPI_Request request, void *extra_state, MPI_Status *status, int *flag); ``comm'' is the communicator used for the user defined operation (the argument in the call to MPI_Request_init), and ``request'' is the request returned by the call to MPI_Request_init; ``extra_state'' is whatever was returned by the previous call to Request_continue (or by the call to Request_start, if none). ``status'' is the status for the communication that just completed. (If people believe they need an additional identification for the request that completed, we can add back the id in Request_register, and have it as an input parameter to Request_continue). ``flag'' returns 0 if the user-defined operation has not completed yet, 1 if it has completed. I.e., it is the user's responsibility (in its Request_continue code), to figure out if the user-defined operation has completed, not MPI's. This avoids the need for a reverse index (one needs to hold for each registered request a pointer to the callback function, but one does not need for each callback function to hold the set of active registered communications). My take is that it should be pretty easy for the user to keep track of the calls that were initiated as part of the execution of user-defined request; the logic of the program will require hinm/her to do so, anyhow. Therefore, there is no reason to have MPI keep track of this information. From mpi-core-human@mcs.anl.gov Thu Jul 20 23:01:43 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA12548; Thu, 20 Jul 1995 23:01:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA04926 for mpi-core-out; Thu, 20 Jul 1995 22:06:53 -0500 Received: from VM1.NoDak.EDU (SMTP@vm1.NoDak.edu [134.129.111.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id WAA04877; Thu, 20 Jul 1995 22:06:11 -0500 Date: Thu, 20 Jul 1995 22:06:11 -0500 Message-Id: <199507210306.WAA04877@antares.mcs.anl.gov> Received: from physics1.byu.edu by VM1.NoDak.EDU (IBM VM SMTP V2R2) with TCP; Thu, 20 Jul 95 22:05:05 CDT To: mpi-bind@mcs.anl.gov, mpi-coll@mcs.anl.gov, mpi-comm@mcs.anl.gov, mpi-core@mcs.anl.gov, mpi-dynamic@mcs.anl.gov From: romance@in.the.former.USSR.ua Subject: letter from Olga! =) (Unverified) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Not to long ago, I posted a message re: meeting women of the former soviet union through romance ads. In August, Olga will travel to Moscow from her home in Kiev, Ukraine. In Moscow, Olga will have a much easier and cost efficient means to place your personal romance ad throughout Russia. Last week I received the following from Olga: "I have already sent your ad to the papers in such towns: Moscow, St. Petersburg, Vladimir, Kazan. At nearest future I will send your ad to the paper in some more 12 towns of Russia, where papers are published. Some times (in winter, spring & now) I placed your ad in other papers, but they are not most popular paper in Moscow and some large cities of Russia. Besides, I am continuing to place your ad in papers of Ukraine. I promise to place your ad in some other papers when I will come to Moscow in August. I am glad that you have received fairly many letters from Russian & Ukrainian girls and I think you will received some more ones and will find your ideal in my country soon. I thank you very much ones more for your kindness & your help. My best wishes, Olga" This isn't a scam - call it panhandling if you want. . . I sent her $40 or $50 and I've received over 45 responses. Unlike placing romance ads in the U.S., women from the former USSR respond. Although one would guess the are doing so in the hopes of American citizenship, I haven't found it so. Olga lives in Kiev, Ukraine (population 3 million) and will travel to Moscow in August to visit her father. If you were to send a letter this week, she would receive it in time. The population of Moscow is 10 million -- (3 times the size of Los Angeles.) Feel free to send a letter and ask her your questions. She will be happy to respond. Olga's address: Ukraine Kiev 253121 Dekabristov Street 5 - 178 Olga Kozmina I am posting anonymously because of the flames and volume of inquiries that would result otherwise. I think those who are truely interested will take the time to write. _____________________________________________________________________________ To: probable flamer Subject: polite note Although Olga has never seen a newsgroup nor heard of "net-etiquette," she believes that offering lonely singles the possibility of romance exceeds the cost of angering those who feel the net shouldn't be used in this fashion. IHA (I humbly ask) that you not flame the postmaster of this site. peace. . . From mpi-core-human@mcs.anl.gov Fri Jul 21 11:18:42 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA24053; Fri, 21 Jul 1995 11:18:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA13290 for mpi-core-out; Fri, 21 Jul 1995 10:26:22 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA13271 for ; Fri, 21 Jul 1995 10:26:06 -0500 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id LAA12159 for ; Fri, 21 Jul 1995 11:24:17 -0400 Received: from hume.super.org (hume.super.org [192.239.79.11]) by gotham.super.org (8.6.12/8.6.12.1) with SMTP id LAA05013 for ; Fri, 21 Jul 1995 11:25:28 -0400 Date: Fri, 21 Jul 1995 11:25:28 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199507211525.LAA05013@gotham.super.org> Received: by hume.super.org (4.1/SMI-4.1) id AA05959; Fri, 21 Jul 95 11:25:27 EDT To: mpi-core@mcs.anl.gov Subject: input due Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy, This is a reminder that all input for the document to be distributed at the meeting next week is due by COB today (Friday, July 21). Please e-mail to me (lederman@super.org) and Rusty (lusk@mcs.anl.gov). If you posted the proposal to one of the mail reflectors then you do not need to send it to us. However, if you only posted postscript then please send us the ASCII version. Thanks and see you next week, Steve From mpi-core-human@mcs.anl.gov Fri Jul 21 18:04:39 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA26748; Fri, 21 Jul 1995 18:04:39 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA24018 for mpi-core-out; Fri, 21 Jul 1995 17:11:32 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA24000 for ; Fri, 21 Jul 1995 17:11:18 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 7561; Fri, 21 Jul 95 18:09:55 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 3457; Fri, 21 Jul 1995 18:09:54 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Fri, 21 Jul 95 18:09:54 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA35765; Fri, 21 Jul 1995 18:09:34 -0400 Message-Id: <9507212209.AA35765@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Cc: pratap@watson.ibm.com Subject: 2 comments Reply-To: snir@watson.ibm.com Date: Fri, 21 Jul 1995 18:09:33 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: 1. Lst comment (I promise!) on the issue of "why offline polling is needed to implement MPI". Consider the following code: if (myrank == 0) MPI_Send(&buff, count, MPI_BYTE, 1, 0, comm); else if (myrank == 1) MPI_Irecv(&buff, count, MPI_BYTE, 0, 0, comm, &status); This is a correct (albeit somewhat useless) MPI program: there is no requirement to test or wait for each nonblocking call. The MPI standard says quite explicitly that the blocking send call must complete, even if the receiver never executes a wait or test call. Consider the case where the send started after the matching receive, and where the message sent is too large to be buffered. The only way the send can complete is for the receiving node to start receiving the sent message, even if no MPI call executes on the receiving node. Therefore, a correct implementation of MPI cannot postpone reception of pending messages until an MPI call occurs at the receiving processor. The mechanism for initiating the reception can vary, of course: polling by an independent coprocessor, periodic polling by the main processor, interrupt at messafge arrival, etc. 2. ALLPUT/ALLGET. The current proposal for put/get does not require a synchronization to occur for each individual put/get. The put/get can be issued without specifying a target request, in which case, there is no synchronization for that individual operation. A global fence operation can be used to wait for the completion of all pending put/get operations. Thus, for algorithms that work in phases, the code will be: repeat generate data to be communicated initiate all put/get needed for communication global fence use data from communication. Some local computation is interleaved in this loop, so as to hide latency. One can, of course, use double buffering for better latency tolerance. If one thinks in terms of Valiant's BSP model, the global fence is the global clock tick for moving from one phase to the next. I.e., the computation (within a group) proceeds in phases: at each phase one can use data communicated via put/get commands issued at the previous phase. The question is: do we envisage algorithms that will use put/get, where a global, loosely synchronous model of the type described above is not suitable, but where one still wants to avoid the need to synchronize for each individual put/get communication? - ------------------- Marc Snir IBM T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 email: snir@watson.ibm.com phone: 914-945-3204 fax: 914-945-4425 ------- End of Forwarded Message ------------------- Marc Snir IBM T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 email: snir@watson.ibm.com phone: 914-945-3204 fax: 914-945-4425 From mpi-core-human@mcs.anl.gov Sat Jul 22 19:27:15 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA08187; Sat, 22 Jul 1995 19:27:15 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA04405 for mpi-core-out; Sat, 22 Jul 1995 18:33:12 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA04395 for ; Sat, 22 Jul 1995 18:33:04 -0500 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 3917; Sat, 22 Jul 95 19:32:05 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 1447; Sat, 22 Jul 1995 19:32:05 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Sat, 22 Jul 95 19:32:05 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA23475; Sat, 22 Jul 1995 19:31:45 -0400 Message-Id: <9507222331.AA23475@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Cc: mpi-impl@mcs.anl.gov Subject: MPI1 clarification -- error handling Reply-To: snir@watson.ibm.com Date: Sat, 22 Jul 1995 19:31:44 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: the default error handler MPI_ERRORS_ARE_FATAL is said to "cause the program to abort on all executing processes. This has the same effect as if MPI_ABORT was called by the process that invoked the handler." MPI_ABORT(comm, errorcode) is said to "make a best attempt to abort all tasks in the group of comm." (It may also abort all processes in MPI_COMM_WORLD). These two statement are contradictory. If we want MPI_ERRORS_ARE_FATAL to behave as a call to MPI_ABORT we should allow for the possiblity that this error handler only aborts the processes in the group of the relevant communicator. If we want MPI_ERRORS_ARE_FATAL to abort the entire computation, then we should say that it behaves as if MPI_ABORT(MPI_COMM_WORLD, somecodenumber) was called. I would suggest the second interpretation. ------------------- Marc Snir IBM T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 email: snir@watson.ibm.com phone: 914-945-3204 fax: 914-945-4425 From mpi-core-human@mcs.anl.gov Sat Jul 22 21:09:35 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA08457; Sat, 22 Jul 1995 21:09:34 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA04808 for mpi-core-out; Sat, 22 Jul 1995 20:18:23 -0500 Received: from msr.EPM.ORNL.GOV (msr.epm.ornl.gov [128.219.8.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA04564 for ; Sat, 22 Jul 1995 19:07:39 -0500 Received: (from geist@localhost) by msr.EPM.ORNL.GOV (8.6.10/8.6.10) id UAA16854 for mpi-core@mcs.anl.gov; Sat, 22 Jul 1995 20:07:08 -0400 Date: Sat, 22 Jul 1995 20:07:08 -0400 From: Al Geist Message-Id: <199507230007.UAA16854@msr.EPM.ORNL.GOV> To: mpi-core@mcs.anl.gov Subject: re: MPI1 clarification -- error handling Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >If we want MPI_ERRORS_ARE_FATAL to abort the entire >computation, then we should say that it behaves as if >MPI_ABORT(MPI_COMM_WORLD, somecodenumber) was called. I would suggest the >second interpretation. Been there with PVM, and we found aborting the "world" to be a bad thing. I suggest "at most" just processes in COMM are aborted. For MPI-1 picking either interpretation is OK but looking ahead to MPI-2 with client servers I wouldn't want a failed client to cause the server to be killed. (this is what we ran into and had to fix in PVM) Looking even further ahead (MPI-3?) if we incorporate some fault tolerance then we don't want choices made in MPI-1 to hinder this work. Al Geist From mpi-core-human@mcs.anl.gov Sun Jul 23 11:39:33 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA18346; Sun, 23 Jul 1995 11:39:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA08812 for mpi-core-out; Sun, 23 Jul 1995 10:46:27 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA08804 for ; Sun, 23 Jul 1995 10:46:20 -0500 From: snir@watson.ibm.com Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 6147; Sun, 23 Jul 95 11:45:31 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 3327; Sun, 23 Jul 1995 11:45:31 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Sun, 23 Jul 95 11:45:31 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA16902; Sun, 23 Jul 1995 11:45:11 -0400 Message-Id: <9507231545.AA16902@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: Al Geist Cc: mpi-core@mcs.anl.gov Subject: Re: MPI1 clarification -- error handling In-Reply-To: (Your message of Sat, 22 Jul 1995 20:07:08 EDT.) Date: Sun, 23 Jul 95 11:45:31 EDT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk : -) :-) :-) *** (-: (-: (-: Date: Sun, 23 Jul 1995 11:45:10 -0400 From: Marc Snir > > >If we want MPI_ERRORS_ARE_FATAL to abort the entire > >computation, then we should say that it behaves as if > >MPI_ABORT(MPI_COMM_WORLD, somecodenumber) was called. I would suggest the > >second interpretation. > > Been there with PVM, and we found aborting the "world" to be a bad thing. > I suggest "at most" just processes in COMM are aborted. > For MPI-1 picking either interpretation is OK > but looking ahead to MPI-2 with client servers > I wouldn't want a failed client to cause the server to be killed. > (this is what we ran into and had to fix in PVM) > > Looking even further ahead (MPI-3?) if we incorporate some fault tolerance > then we don't want choices made in MPI-1 to hinder this work. > > Al Geist > This would require a (modest) change in the current MPI1 text. The current text allows implementations of MPI_ABORT(comm,...) that kill only processes in comm or all processes in MPI_COMM_WORLD. (The current text seems to imply that it is even OK to have an implementation where only comm=MPI_COMM_WORLD is accepted -- which seems strange). The bhavior which I think you want is that MPI_ABORT(comm,....) always kill only the processes in comm -- I.e. restrict the set of allowable implementations. A separate discussion is what should be the behavior of the default error handler MPI_ERRORS_ARE_FATAL. I would argue that the default error handler should keep killing the entire computation, otherwise naive users would be faced with the strange behavior where program error causes some but not all the processes to disapear. Users can always define new error handlers that kill only processes in the communicator group. Or, if we wish, we could add such new errror handler (MPI_ERRORS_ARE_GROUP_FATAL, MPI_ERRORS_ARE_PROCESS_FATAL?). ------------------- Marc Snir IBM T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 email: snir@watson.ibm.com phone: 914-945-3204 fax: 914-945-4425 From mpi-core-human@mcs.anl.gov Mon Jul 24 11:14:11 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA00688; Mon, 24 Jul 1995 11:14:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA18535 for mpi-core-out; Mon, 24 Jul 1995 10:19:28 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA18522; Mon, 24 Jul 1995 10:19:14 -0500 Message-Id: <199507241519.KAA18522@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Cc: geist@msr.epm.ornl.gov Subject: MPI meeting this week, final head count Date: Mon, 24 Jul 1995 10:19:13 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here are the people that I think are coming to the meeting this week. If you know of anyone is is coming who is not on here, or if you are on here and not coming, please let me know immediately. Thanks. Rusty Coming to MPI Forum meeting in Chicago, July 26-28: Paul Pierce Joel Clark Tony Skjellum Nathan Doss Eric Salo Lloyd Lewins Steve Fleishman Arindam Saha Susan Kraus Pratap Pattniak Al Geist Steve Huss-Lederman Bill Saphir Eric Sharakan David Taylor Greg Burns Gordon Smith Tom Henderson Kei Harada Sagawa Pratap Pattnaik Joefon Jann Heidi Poxon Lance Schuler Hans-Christian Hoppe Jim Pryune Robert George Rusty Lusk Bill Gropp From mpi-core-human@mcs.anl.gov Mon Jul 24 16:59:02 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA04341; Mon, 24 Jul 1995 16:59:01 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA26899 for mpi-core-out; Mon, 24 Jul 1995 16:07:08 -0500 Received: from infomeister.osc.edu (infomeister.osc.edu [128.146.36.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA26887 for ; Mon, 24 Jul 1995 16:06:58 -0500 Received: from tbag.osc.edu for gdburns@tbag.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id RAA04344; Mon, 24 Jul 1995 17:06:55 -0400 Received: by tbag.osc.edu (4.1/OSC-SUB-2.0) id AA13427; Mon, 24 Jul 95 17:06:51 EDT Date: Mon, 24 Jul 95 17:06:51 EDT From: gdburns@tbag.osc.edu (Greg Burns) Message-Id: <9507242106.AA13427@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: Re: 2 comments Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >if (myrank == 0) > MPI_Send(&buff, count, MPI_BYTE, 1, 0, comm); >else if (myrank == 1) > MPI_Irecv(&buff, count, MPI_BYTE, 0, 0, comm, &status); > >This is a correct (albeit somewhat useless) MPI program: there is no >requirement to test or wait for each nonblocking call. The MPI standard says >quite explicitly that the blocking send call must complete, even if the >receiver never executes a wait or test call. Where does it say that? In the progress guarantee? As I said earlier, this is anything but explicit and we interpret it differently. Somewhere else? The blocking send call could complete when the receiver calls any other MPI routine. The only crack I see is whether or not you believe a correct program must eventually call MPI_Finalize(). If you do not believe so, then the only danger for the above sender to not complete is for the the receiver to effectively code while(1); after MPI_Irecv(). This seems like a useless example to force "off-line polling" on implementations. -- Greg From mpi-core-human@mcs.anl.gov Mon Jul 24 19:33:19 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA06499; Mon, 24 Jul 1995 19:33:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA00367 for mpi-core-out; Mon, 24 Jul 1995 18:41:05 -0500 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA00359 for ; Mon, 24 Jul 1995 18:40:55 -0500 From: snir@watson.ibm.com Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 4681; Mon, 24 Jul 95 19:39:42 EDT Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.01 on VAGENT2" id 8023; Mon, 24 Jul 1995 19:39:42 EDT Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2R3) with TCP; Mon, 24 Jul 95 19:39:41 EDT Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/930311) id AA26527; Mon, 24 Jul 1995 19:39:21 -0400 Message-Id: <9507242339.AA26527@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: gdburns@tbag.osc.edu (Greg Burns) Cc: mpi-core@mcs.anl.gov Subject: Re: 2 comments In-Reply-To: (Your message of Mon, 24 Jul 1995 17:06:51 EDT.) Date: Mon, 24 Jul 95 19:39:42 EDT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk : -) :-) :-) *** (-: (-: (-: Date: Mon, 24 Jul 1995 19:39:21 -0400 From: Marc Snir > > > >if (myrank == 0) > > MPI_Send(&buff, count, MPI_BYTE, 1, 0, comm); > >else if (myrank == 1) > > MPI_Irecv(&buff, count, MPI_BYTE, 0, 0, comm, &status); > > > >This is a correct (albeit somewhat useless) MPI program: there is no > >requirement to test or wait for each nonblocking call. The MPI standard says > >quite explicitly that the blocking send call must complete, even if the > >receiver never executes a wait or test call. > > Where does it say that? In the progress guarantee? As I said earlier, > this is anything but explicit and we interpret it differently. Somewhere else? > pg 42, l37. "... Similarly, a call to MPI_WAIT that completes a send will eventually return if a matching receive has been started, unless the receive is satisfied by another send, AND EVEN IF NO CALL IS EXECUTED TO COMPLETE THE RECEIVE." Replace "MPI_Send" by "MPI_ISend; MPI_Wait" in my example. > The blocking send call could complete when the receiver calls any other > MPI routine. > > The only crack I see is whether or not you believe a correct program > must eventually call MPI_Finalize(). If you do not believe so, then > the only danger for the above sender to not complete is for the > the receiver to effectively code while(1); after MPI_Irecv(). > This seems like a useless example to force "off-line polling" on > implementations. > > -- > Greg ------------------- Marc Snir IBM T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 email: snir@watson.ibm.com phone: 914-945-3204 fax: 914-945-4425 From mpi-core-human@mcs.anl.gov Mon Jul 24 21:17:15 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA06951; Mon, 24 Jul 1995 21:17:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA01102 for mpi-core-out; Mon, 24 Jul 1995 20:25:21 -0500 Received: from SSD.intel.com (ssd.intel.com [137.46.3.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id UAA01094 for ; Mon, 24 Jul 1995 20:25:06 -0500 Received: from ark.SSD.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA24890; Mon, 24 Jul 95 18:24:33 PDT Received: by ark.SSD.intel.com (4.1/SMI-4.1) id AA25167; Mon, 24 Jul 95 18:24:30 PDT Date: Mon, 24 Jul 95 18:24:30 PDT From: Paul Pierce Message-Id: <9507250124.AA25167@ark.SSD.intel.com> To: mpi-core@mcs.anl.gov Subject: Get/Put Proposal Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is a get/put proposal based on our discussion at the last meeting. Paul %!PS-Adobe-2.0 %%Creator: dvipsk 5.55a Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%Pages: 7 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=600, compressed, comments removed %DVIPSSource: TeX output 1995.07.24:1813 %%BeginProcSet: texc.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]/id ch-image N /rw ch-width 7 add 8 idiv string N /rc 0 N /gp 0 N /cp 0 N{rc 0 ne{rc 1 sub /rc X rw}{G}ifelse}imagemask restore}B /G{{id gp get /gp gp 1 add N dup 18 mod S 18 idiv pl S get exec}loop}B /adv{cp add /cp X}B /chg{rw cp id gp 4 index getinterval putinterval dup gp add /gp X adv}B /nd{/cp 0 N rw exit}B /lsh{rw cp 2 copy get dup 0 eq{pop 1}{ dup 255 eq{pop 254}{dup dup add 255 and S 1 and or}ifelse}ifelse put 1 adv}B /rsh{rw cp 2 copy get dup 0 eq{pop 128}{dup 255 eq{pop 127}{dup 2 idiv S 128 and or}ifelse}ifelse put 1 adv}B /clr{rw cp 2 index string putinterval adv}B /set{rw cp fillstr 0 4 index getinterval putinterval adv}B /fillstr 18 string 0 1 17{2 copy 255 put pop}for N /pl[{adv 1 chg} {adv 1 chg nd}{1 add chg}{1 add chg nd}{adv lsh}{adv lsh nd}{adv rsh}{ adv rsh nd}{1 add adv}{/rc X nd}{1 add set}{1 add clr}{adv 2 chg}{adv 2 chg nd}{pop nd}]dup{bind pop}forall N /D{/cc X dup type /stringtype ne{] }if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 600 600 (temp.dvi) @start /Fa 8 118 df58 D68 D<903801FFC0010F13FC017F13FFD9FF8013802603FE0013C048485AEA0FF812 1F13F0123F6E13804848EB7F00151C92C7FC12FFA9127FA27F123FED01E06C7E15036C6C EB07C06C6C14806C6C131FC69038C07E006DB45A010F13F00101138023257DA42A>99 D105 D<01FEEB7FC000FF903803FFF8020F13FE91381F03FFDA3C0113 80000713780003497E6D4814C05CA25CA291C7FCB3A3B5D8FC3F13FFA430257DA435> 110 D<903801FFC0010F13F8017F13FFD9FF807F3A03FE003FE048486D7E48486D7E4848 6D7EA2003F81491303007F81A300FF1680A9007F1600A3003F5D6D1307001F5DA26C6C49 5A6C6C495A6C6C495A6C6C6CB45A6C6CB5C7FC011F13FC010113C029257DA430>I<9038 3FF0383903FFFEF8000F13FF381FC00F383F0003007E1301007C130012FC15787E7E6D13 0013FCEBFFE06C13FCECFF806C14C06C14F06C14F81203C614FC131F9038007FFE140700 F0130114007E157E7E157C6C14FC6C14F8EB80019038F007F090B512C000F8140038E01F F81F257DA426>115 D<01FFEC3FC0B5EB3FFFA4000714016C80B3A35DA25DA26C5C6E48 13E06CD9C03E13FF90387FFFFC011F13F00103138030257DA435>117 D E /Fb 10 58 df<13FF000313C0380781E0380F00F0001E137848133CA248131EA400 F8131FAD0078131EA2007C133E003C133CA26C13786C13F0380781E03803FFC0C6130018 227DA01E>48 D<13E01201120712FF12F91201B3A7487EB512C0A212217AA01E>II<13FF000313C0380F03E0381C00F014F8003E13FC 147CA2001E13FC120CC712F8A2EB01F0EB03E0EB0FC03801FF00A2380003E0EB00F01478 147C143E143F1230127812FCA2143E48137E0060137C003813F8381E03F0380FFFC00001 130018227DA01E>I<14E01301A213031307A2130D131D13391331136113E113C1EA0181 1203EA07011206120C121C12181230127012E0B6FCA2380001E0A6EB03F0EB3FFFA21822 7DA11E>I<00101330381E01F0381FFFE014C01480EBFE00EA1BF00018C7FCA513FE381B FF80381F03C0381C01E0381800F014F8C71278A2147CA21230127812F8A214784813F800 6013F0387001E01238381E07803807FF00EA01F816227CA01E>II<1230123C003FB5FCA24813FE14FC3860001C 143814704813E014C0EA0001EB0380EB07001306130E5BA25BA21378A35BA41201A76C5A 18237CA11E>I<137F3803FFC0380781E0380E00704813380018131C1238A3123C003F13 38381FC078EBE0F0380FF9E03807FF80120114C0000713F0380F0FF8381C03FC383801FE 3870007E141F48130F1407A314060070130E0078130C6C1338001F13F03807FFC0C61300 18227DA01E>I<13FE3803FFC0380781E0380E0070481378003C133848133CA200F8131E A3141FA40078133FA26C137F121C380F01DF3807FF9F3803FE1EC7FCA2143E143C001C13 38003E13781470003C13E0381801C0381C0780380FFE00EA03F818227DA01E>I E /Fc 2 60 df<121EEA7F80A2EAFFC0A4EA7F80A2EA1E000A0A798919>58 D<121EEA7F8012FF13C0A213E0A3127FEA1E601200A413E013C0A312011380120313005A 120E5A1218123812300B1C798919>I E /Fd 22 90 df<121EEA3F80EA7FC012FFA41380 EA7F00123C0A0A788919>46 D<157015F014011407143F903803FFE0137FEBFFCFEBF80F 1300141F15C0A5143F1580A5147F1500A55C5CA513015CA513035CA513075CA5130F5CA3 131F497EB612F8A31D3D78BC2D>49 D<161C163C167CA216FCED01F815031507150FA215 1DED3BF0157315E315C31401EC038391380707E0140E141CA2143814709138E00FC0EB01 C014801303EB0700130E49EB1F805B133013705B485A4848EB3F0090C7FC5A120E5A5A48 147E1260B8FCA3C73801FE00A25DA41403A25DA314074A7E0107B512F8A3283E7BBD2D> 52 D<17E016011603831607A2160FA2161F83163FA2167F167716F7EEE7FCED01E316C3 150316831507EE03FEED0F01150E151E151C153C03387FED7800157015F05D4A4880177F 4A5AA24AC7FCA2020E81173F5C021FB6FC5CA20270C7EA3FE0171F5CA2495AA249488117 0F49C8FCA2130EA24982013C1507A2137CD801FE4B7E2607FF80EC3FFEB500F00107B512 FC19F85E3E417DC044>65 D<013FB7FC18E018FC903B007FE00007FE6E48903801FF8094 38007FC05DF03FE0F01FF0A3027F16F892C8FCA54A16F04A153F19E0187F19C0F0FF8001 014B13004A4A5A4D5AEF1FF04D5ADC03FFC7FC49B612F8EFFF8002F8C7EA3FE0EF0FF0EF 07FC717E010715014A81711380A319C0130F5CA5011F4B13805C19005F601707013F4B5A 4A4A5A4D5A4D5A017F913801FF8001FF020F90C7FCB812FC17F094C8FC3D3E7DBD40>I< DCFFC01338030F01F01378037F01FC13F0913A01FF803F01913A07FC000781DA1FE0EB03 C3DA7FC0EB01E74AC812FF4948ED7FE0D907FC153F495A4948151F495A4948150F494816 C018074890C9FC485AA2485A000F1880491603121FA248481607A295C7FC485AA412FF5B A75BA2181C183C1838A27F007F1778187018F0003F5F6D150160001F16036C6C4B5A95C7 FC6C6C5D6C6C151E6C6C5D6C6C15F86D6C495A6D6CEB07C0D91FF0EB1F80D907FE01FEC8 FC0101B512F86D6C13E0DA07FEC9FC3D4276BF42>I<013FB7FC18E018F8903B007FF000 0FFE6E48EB01FF9438007FC04B6E7E180F85727E727E147F4B6E7EA2727EA302FF178092 C9FCA54918C05CA41A8013034A5DA41A0013074A5DA261A24E5A130F4A5E180F61181F61 011F4C5A5C4E5A4EC7FC4D5A4D5A013F4B5A4A4A5AEF3FE0EF7F80017F4A48C8FC01FFEC 1FFCB812F0178004FCC9FC423E7DBD45>I<013FB812F8A39026007FF0C7127F6E48140F 18034B14011800A31978147F4B1570A502FF143892C7FCA3190017784915704A14F01601 1603160F91B6FC495DA29138FC001F16071603160101075D5CA2197019F019E0010F4A5A 4A90C7120119C0A218031980011F16075CF00F00A260181E013F163E4A157E4D5A170301 7F150F01FFEDFFF8B9FCA2603D3E7DBD3E>I<013FB812E0A3903A007FF000016E48EB00 3F180F4B14071803A31801147F4B15C0A514FF92C71270A395C7FC17F0495D5C16011603 1607161F49B65AA39138FC003F160F160701075D4A1303A5010F4AC8FC5C93C9FCA4131F 5CA5133F5CA3137FEBFFF0B612F8A33B3E7DBD3B>I<4BB46C1370031F01F013F0037F90 38FC01E0913A03FF807E03913A0FF8000F83DA1FE0EB07C7DA7F80EB01EF4AC812FFD903 FE16C04948157F4948153F495A4948151F495A4948168091C9120F5A485AA2485A000F18 004982121FA248485EA295C7FC485AA412FF5BA6043FB512E05BA29339001FFC00715AA2 607F127FA2171F123F6D5EA2121F7F000F163F6C7E6C6C4B5A7F6C6C15FF6C6DEB01EFD9 3FC0EB07C7D91FF0EB1F87D907FE9038FE03800101B5EAF8016D6C01E0C8FCDA07FEC9FC 3C4276BF47>I<013FB5D8F807B6FC04F015FEA29026007FF0C7380FFE006E486E5AA24B 5DA4180F147F4B5DA4181F14FF92C85BA4183F5B4A5EA491B8FC5B6102FCC8127FA318FF 13074A93C7FCA45F130F4A5DA41703131F4A5DA41707133F4A5DA3017F150F496C4A7EB6 D8E01FB512FC6115C0483E7DBD44>I<011FB512FC5BA29039003FF8006E5AA25DA5143F 5DA5147F5DA514FF92C7FCA55B5CA513035CA513075CA5130F5CA5131F5CA3133F497E00 7FB512F0A2B6FC263E7EBD21>I<013FB512FEA25E9026007FF8C8FCEC3FE0A25DA5147F 5DA514FF92C9FCA55B5CA513035CA513075CA21838A21870130F5CA218E0A3011F15014A 15C01703A21707EF0F80013F151F4A143F177FEFFF00017F140301FF143FB9FC5FA2353E 7DBD39>76 D<90263FFFF0933807FFFE5013FC629026007FF8EFFC00023F4D5AA2023BEF 77F0A2DA39FC16E7A2F101CF0279EE038FDA70FE5FF1070FA2190E1A1FDAF07F151C02E0 60193819706F7EF1E03F130102C0DB01C05BA26F6CEB0380A2953807007F0103160E4A6C 6C93C7FC60A2606201076D6C5B02005F60A26F6C485A94380380015B010EDB07005BA292 3801FC0EA24D1303131E011C6D6C485C5FA25F1907013CEC7FC0013860013C5D137C01FE 6EC7120F2607FF80013E4A7EB500FC031FB512F8043C5E4A131C573E7DBD53>I<90263F FFE0023FB5FC6F16FEA29026003FF8020313C0021F030013004A6C157C023B163C6F1538 1439810238167802787FDA707F157082153F82031F15F002F07FDAE00F5D821507820303 1401010180DAC0015D82811780047F1303010315C04A013F5C17E0161F17F0040F130701 0715F891C7000791C7FC17FC160317FE04015B4915FF010E6E130E188E177F18CEEF3FDE 011E16FE011C6F5AA2170FA21707133C01386F5A133C017C150113FE2607FF801400B512 FC18705C483E7DBD44>I<923803FF80031F13F09238FE01FE913903F0003FDA0FC0EB1F C0DA3F80EB07E0027EC76C7E49486E7E49488149486E7E4948157F495A013F17804948ED 3FC049C9FCA24848EE1FE012035B000718F05B120FA2485A19F8123F5BA2127FA219F048 48163FA5F07FE0A35BF0FFC0A219805F19007F4D5A127F4D5A60003F160F6D5E001F4C5A 4D5A6C6C4B5A95C7FC6C6C15FE00034B5A6C6C4A5A6C6C4A5A017FEC1FC06D6C495AD90F E001FEC8FC903903F807F80100B512C0DA0FFCC9FC3D4276BF47>I<013FB612FEEFFFE0 18F8903B007FF0000FFC6E48EB01FF7113804BEC7FC0183F19E0F01FF0A2147F5D19F8A4 02FFED3FF092C8FCA219E0A2F07FC05B4AEDFF8019004D5A4D5AEF0FF80103ED3FE04A90 3801FF8091B648C7FC17F002FCCAFCA213075CA5130F5CA5131F5CA5133F5CA3137F497E B612E0A25D3D3E7DBD3E>I<013FB612F017FF18E0903B007FF0003FF86E48EB07FCEF01 FE4B6D7EF07F8019C0183F19E0147F4B15F0A502FFED7FE092C8FCA219C0F0FF80A2494B 13004A5D4D5AEF0FF04D5AEF7F800103DA07FEC7FC91B612F017809139FC0007E0EE03F8 EE00FC0107814A147F717EA284A2130F5CA484011F157F5CA41902013F17075CA2F0F00F 017F170E496C143FB600E0011F131C94380FF83C4B01071378CA3801FFE09438003F8040 407DBD43>82 D<9239FF8003800207EBF007021F9038FC0F0091387F00FE02FCEB1F1FD9 03F0EB07BF49486DB4FC49487F4A6D5A49C8FC49157E133E137E173E49153CA57F1738A2 6D92C7FC808080EB7FFEECFFE06D13FEEDFFC06D14F06D14FC010380010080143F020380 DA003F7F15031500707E163F161FA2160F121CA31607160F003C5EA35F003E151F94C7FC 007E5D007F153E6D5C16FC01E0495AD87DF0495AD8FCFCEB0FC03AF87F803F8027F01FFF FEC8FCD8E00713F839C0007FC031427BBF33>I<0007B912F0A33C0FFE000FF8003F01F0 160F01C04A13034848160190C7FC121EF000E048141F5E1238A212781270153F5E5AA3C8 1600157F5EA515FF93C9FCA55C5DA514035DA514075DA5140F5DA3141FEC7FFC0003B7FC A33C3D76BC42>II89 D E /Fe 33 122 df<141E147F14FF5BEB03FEEB 07FCEB0FF0EB1FE0EB3FC0EB7F80EBFF00485A5B12035B485A120F5BA2485AA2123F5BA2 127F90C7FCA412FEAD127FA47F123FA27F121FA26C7EA27F12076C7E7F12017F6C7EEB7F 80EB3FC0EB1FE0EB0FF0EB07FCEB03FEEB01FF7F147F141E184771BE30>40 D<127812FE7E7F6C7E6C7EEA0FF06C7E6C7E6C7E6C7EEB7F80133F14C0131FEB0FE014F0 1307A2EB03F8A214FC1301A214FE1300A4147FAD14FEA4130114FCA2130314F8A2EB07F0 A2130F14E0EB1FC0133F1480137FEBFF00485A485A485A485AEA3FE0485A485A90C7FC5A 1278184778BE30>I<14E0497E497EA60038EC0380007EEC0FC0D8FF83EB3FE001C3137F 9038F3F9FF267FFBFB13C06CB61280000FECFE00000314F86C5C6C6C13C0011F90C7FC01 7F13C048B512F04880000F14FE003FECFF80267FFBFB13C026FFF3F913E09038C3F87F01 83133FD87E03EB0FC00038EC0380000091C7FCA66D5A6D5A23277AAE30>I44 D<91387F803C903903FFF03E49EBFC7E 011F13FE49EBFFFE5B9038FFE07F48EB801F3903FE000F484813075B48481303A2484813 015B123F491300A2127F90C8FC167C16005A5AAC7E7EA2167C6D14FE123FA27F121F6D13 016C6C14FCA26C6CEB03F86D13076C6CEB0FF03901FF801F6C9038E07FE06DB512C06D14 806D1400010713FC6D13F09038007FC0273A7CB830>67 D<003FB512E04814FCB67E6F7E 6C816C813A03F8007FF0ED1FF8150F6F7E6F7E15016F7EA2EE7F80A2163F17C0161FA4EE 0FE0AC161F17C0A3163F1780A2167F17005E4B5A15034B5A150F4B5AED7FF0003FB65A48 5DB75A93C7FC6C14FC6C14E02B387FB730>I<02FF13F00103EBC0F8010F13F1013F13FD 4913FF90B6FC4813C1EC007F4848133F4848131F49130F485A491307121F5B123F491303 A2127F90C7FC6F5A92C8FC5A5AA892B5FC4A14805CA26C7F6C6D1400ED03F8A27F003F14 07A27F121F6D130F120F7F6C6C131FA2D803FE133F6C6C137FECC1FF6C90B5FC7F6D13FB 010F13F30103EBC1F0010090C8FC293A7DB830>71 D<007FB6FCB71280A46C1500260007 F0C7FCB3B3A8007FB6FCB71280A46C1500213879B730>73 D77 D<007FB512F0B612FE6F7E16E0826C813903F8003FED0FFCED03 FE15016F7EA2821780163FA6167F17005EA24B5A1503ED0FFCED3FF890B6FC5E5E16804B C7FC15F001F8C9FCB0387FFFC0B57EA46C5B29387EB730>80 D<003FB57E4814F0B612FC 15FF6C816C812603F8017F9138003FF0151F6F7E15071503821501A515035E1507150F4B 5A153F4AB45A90B65A5E93C7FC5D8182D9F8007FED3FE0151F150F821507A817F8EEF1FC A53A3FFF8003FB4801C0EBFFF8B56C7E17F06C496C13E06C49EB7FC0C9EA1F002E397FB7 30>82 D<90390FF803C0D97FFF13E048B512C74814F74814FF5A381FF80F383FE001497E 4848137F90C7123F5A48141FA2150FA37EED07C06C91C7FC7F7FEA3FF0EA1FFEEBFFF06C 13FF6C14E0000114F86C80011F13FF01031480D9003F13C014019138007FE0151FED0FF0 A2ED07F8A2007C140312FEA56C140716F07F6DEB0FE06D131F01F8EB3FC001FF13FF91B5 1280160000FD5CD8FC7F13F8D8F81F5BD878011380253A7BB830>I<003FB712C04816E0 B8FCA43AFE003F800FA8007CED07C0C791C7FCB3B1011FB5FC4980A46D91C7FC2B387EB7 30>I97 DI< ECFFE0010713FC011F7F017F7F90B612804815C048EB807F3907FC003F485A485A49EB1F 804848EB0F004990C7FC127F90C9FCA25A5AA87E7EA27F003FEC07C06DEB0FE06C7E6D13 1F6C6C14C0D807FE133F9039FFC0FF806C90B5FCC615006D5B011F13F801075B01011380 232A7AA830>I<913801FFE04A7F5CA28080EC0007AAEB03FE90381FFF874913E790B6FC 5A5A481303380FFC00D81FF0133F49131F485A150F4848130790C7FCA25AA25AA87E6C14 0FA27F003F141F6D133F6C7E6D137F390FF801FF2607FE07EBFFC06CB712E06C16F06C14 F76D01C713E0011F010313C0D907FCC8FC2C397DB730>I<49B4FC010713E0011F13F801 7F7F90B57E488048018113803A07FC007FC04848133FD81FE0EB1FE0150F484814F04913 07127F90C7FCED03F85A5AB7FCA516F048C9FC7E7EA27F003FEC01F06DEB03F86C7E6C7E 6D1307D807FEEB1FF03A03FFC07FE06C90B5FC6C15C0013F14806DEBFE00010713F80100 13C0252A7CA830>III<14E0EB03F8A2497E A36D5AA2EB00E091C8FCA9381FFFF8487F5AA27E7EEA0001B3A9003FB612C04815E0B7FC A27E6C15C023397AB830>105 D<02FC137E3B7FC3FF01FF80D8FFEF01877F90B500CF7F 15DF92B57E6C010F13872607FE07EB03F801FC13FE9039F803FC01A201F013F8A301E013 F0B3A23C7FFE0FFF07FF80B548018F13C0A46C486C01071380322881A730>109 DI<49B4FC010F13E0013F13F8497F90B57E0003ECFF 8014013A07FC007FC04848EB3FE0D81FE0EB0FF0A24848EB07F8491303007F15FC90C712 01A300FEEC00FEA86C14016C15FCA26D1303003F15F86D13076D130F6C6CEB1FF06C6CEB 3FE06D137F3A07FF01FFC06C90B512806C15006C6C13FC6D5B010F13E0010190C7FC272A 7CA830>II<49B413F8010F EBC1FC013F13F14913FD48B6FC5A481381390FFC007F49131F4848130F491307485A4913 03127F90C7FC15015A5AA77E7E15037FA26C6C1307150F6C6C131F6C6C133F01FC137F39 07FF01FF6C90B5FC6C14FD6C14F9013F13F1010F13C1903803FE0190C7FCAD92B512F84A 14FCA46E14F82E3C7DA730>II<90381FFC1E48B5129F000714FF5A5A5A38 7FF007EB800100FEC7FC4880A46C143E007F91C7FC13E06CB4FC6C13FC6CEBFF806C14E0 000114F86C6C7F01037F9038000FFF02001380007C147F00FEEC1FC0A2150F7EA27F151F 6DEB3F806D137F9039FC03FF0090B6FC5D5D00FC14F0D8F83F13C026780FFEC7FC222A79 A830>III<3B3FFFC07FFF80486DB512C0B515E0A26C16C06C496C 13803B01F80003F000A26D130700005DA26D130F017E5CA2017F131F6D5CA2EC803F011F 91C7FCA26E5A010F137EA2ECE0FE01075BA214F101035BA3903801FBF0A314FF6D5BA36E 5A6E5A2B277EA630>I<3A3FFF81FFFC4801C37FB580A26C5D6C01815BC648C66CC7FC13 7FEC80FE90383F81FC90381FC3F8EB0FE3ECE7F06DB45A6D5B7F6D5B92C8FC147E147F5C 497F81903803F7E0EB07E790380FE3F0ECC1F890381F81FC90383F80FE90387F007E017E 137F01FE6D7E48486D7E267FFF80B5FCB500C1148014E3A214C16C0180140029277DA630 >120 D<3B3FFFC07FFF80486DB512C0B515E0A26C16C06C496C13803B01FC0003F000A2 000014076D5C137E150F017F5C7F151FD91F805BA214C0010F49C7FCA214E00107137EA2 EB03F0157C15FCEB01F85DA2EB00F9ECFDF0147D147FA26E5AA36E5AA35DA2143F92C8FC A25C147EA2000F13FE486C5AEA3FC1EBC3F81387EB8FF0EBFFE06C5B5C6C90C9FC6C5AEA 01F02B3C7EA630>I E /Ff 39 122 df11 D<146014E0EB01C0EB0380EB0700130E131E5B5BA25B485AA2485AA212075B120F90 C7FCA25A121EA2123EA35AA65AB2127CA67EA3121EA2121F7EA27F12077F1203A26C7EA2 6C7E1378A27F7F130E7FEB0380EB01C0EB00E01460135278BD20>40 D<12C07E12707E7E7E120F6C7E6C7EA26C7E6C7EA21378A2137C133C133E131EA2131F7F A21480A3EB07C0A6EB03E0B2EB07C0A6EB0F80A31400A25B131EA2133E133C137C1378A2 5BA2485A485AA2485A48C7FC120E5A5A5A5A5A13527CBD20>I 45 D<121C127FEAFF80A5EA7F00121C0909798817>I68 DI73 D77 DIII83 D<003FB812E0A3D9C003EB001F273E0001FE130348EE01F00078160000701770A3006017 30A400E01738481718A4C71600B3B0913807FF80011FB612E0A335397DB83C>II97 DIII I<147E903803FF8090380FC1E0EB1F8790383F0FF0137EA213FCA23901F803C091C7FCAD B512FCA3D801F8C7FCB3AB487E387FFFF8A31C3B7FBA19>IIII107 DI<27 03F00FF0EB1FE000FFD93FFCEB7FF8913AF03F01E07E903BF1C01F83803F3D0FF3800FC7 001F802603F70013CE01FE14DC49D907F8EB0FC0A2495CA3495CB3A3486C496CEB1FE0B5 00C1B50083B5FCA340257EA445>I<3903F00FF000FFEB3FFCECF03F9039F1C01F803A0F F3800FC03803F70013FE496D7EA25BA35BB3A3486C497EB500C1B51280A329257EA42E> II<3903F01FE000FFEB7FF89038F1E07E9039F3801F803A07F7000FC0 D803FEEB07E049EB03F04914F849130116FC150016FEA3167FAA16FEA3ED01FCA26DEB03 F816F06D13076DEB0FE001F614C09039F7803F009038F1E07E9038F0FFF8EC1FC091C8FC AB487EB512C0A328357EA42E>II<3807E01F00FFEB7FC09038E1E3E09038E387F038 0FE707EA03E613EE9038EC03E09038FC0080491300A45BB3A2487EB512F0A31C257EA421 >II<1318 A51338A31378A313F8120112031207001FB5FCB6FCA2D801F8C7FCB215C0A93800FC0115 80EB7C03017E13006D5AEB0FFEEB01F81A347FB220>IIIIII E /Fg 37 122 df<14FCEB01F8EB03F0EB07E0EB0FC0131F1480EB3F005B137E5B12015B12 03A2485AA25B120FA25B121FA25B123FA448C7FCA712FEB3A2127FA76C7EA4121F7FA212 0F7FA212077FA26C7EA212017F1200137E137F7FEB1F8014C0130FEB07E0EB03F0EB01F8 EB00FC165A79C323>40 D<12FC127E7E6C7E6C7E7F12076C7E7F12016C7E7F137E137FA2 EB3F80A2131F14C0A2130F14E0A2130714F0A4EB03F8A7EB01FCB3A2EB03F8A7EB07F0A4 14E0130FA214C0131FA21480133FA2EB7F00A2137E13FE5B485A12035B485A120F5B485A 48C7FC127E5A165A7BC323>I44 D65 D67 DIIII<12FFB3B3B3A9083F78BE19>73 D77 DI80 D82 DIIII97 D<12FEB3A414FF010713E0011F7F017F7FB67E819038F80FFFEBE001 D98000138090C7EA7FC0153F48141F16E0150FA3ED07F0AAED0FE0A3151FED3FC07E6DEB 7F8015FFD9E00313009038F81FFE90B55A485C6D5B6D5B010F1380260001FEC7FC244079 BE2F>I<49B47E010F13F0013F13FC4913FF90B612805A481300D807FCEB1F00D80FF013 0748487F4990C7FC123F5B127F90C9FCA312FEAA127FA36C7EA26C6C14406DEB01C06C6C 13036C6C131F01FF13FF6C90B5FC7E6C6C14806DEBFE00010F13F001011380222B7DA928 >II< EB03F8EB1FFF017F13C090B57E488048803807FE07390FF801FC9038E000FE4848137E00 3F143E49133F90C77E5A127EED0F80B7FCA600FCC9FCA37E127EA2127FA26C7EA26C7E6D 14806C6C1303D807FC131F01FF13FF6C90B5FC7E6C6C14006D13FC010F13E0010190C7FC 212B7DA928>III<12FFA81200AF127FB3B3A4083F7ABE16>105 D<26FC01FFECFF800107D9C003 13E0011FD9F00F13F8017FD9F83F7F90B56C487F00FD92B5FC3CFFF80FFFFC07FFD9E003 EBF001496C497E496C49EB7F80A290C76C48133FA34892C7FCB3A9392979A848>109 D<38FC01FF010713C0011F13F0017F13F890B512FC12FD39FFF80FFEEBE003EBC0019038 8000FFA290C7127FA35AB3A9202979A82F>II<14FFD8FE0713E0011F7F017F7FB67E819038F80FFFEBE003D98000138090C7 EA7FC0153F5AED1FE0A2150FA216F01507A8150F16E0A2151FA2ED3FC06C147F6DEBFF80 5CD9E00313009038F81FFE90B55A485C6D5B6D5B010F1380D901FEC7FC90C9FCB1243B79 A82F>I<903903F807F0EB1FFF017F13C790B512E74814F74814FF481381390FFC007F48 48133F49131F4848130F5B48481307A290C7FCA25A5AA87E7EA27FA26C7E150F6C6C131F 6D133FD80FFC137F9038FF01FF6C90B5FC6C14F76C14E76C148790383FFE07EB0FF090C7 FCB1243B7DA82F>I<00FC137CEB03FC130F131F133F137FEBFFC038FDFE00EAFFF85B5B 5BA25BA290C7FCA25AB3A6162979A81F>III<00FE147FB3AC15FFA25C6C5B6C130FEBC03F90B6FC6CEBFE7F6C13FC6C13E000 0390C7FC202979A72F>I120 DI E /Fh 20 119 df<120FEA3FC0127FA212FFA31380EA7F00123C0A0A7789 1C>46 D<171C173C177CA217FCA216011603A21607A24C7EA2161DA216391679167116E1 A2ED01C1A2ED038115071601150EA2031C7FA24B7EA25D15F05D4A5AA24A5AA24AC7FC5C 140E5C021FB6FC4A81A20270C7127FA25C13015C495AA249C8FCA2130E131E131C133C5B 01F882487ED807FEEC01FFB500E0017FEBFF80A25C39417BC044>65 D<49B812F8A390260003FEC7121F18074B14031801F000F014075DA3140F5D19E0A2141F 4B1338A2EF7801023F027013C04B91C7FCA217F0027F5CED80011603160F91B65AA3ED00 1F49EC07805CA3010392C8FC5CF003804C13070107020E14005C93C75A180E010F161E4A 151C183CA2011F5E5C60A2013F15014A4A5A1707017F150F4D5A4A147F01FF913807FF80 B9FCA295C7FC3D3E7BBD3E>69 D<49B612FCEFFF8018F0903B0003FE000FF8EF03FE4BEB 00FF8419800207ED3FC05DA219E0140F5DA3021FED7FC05DA2F0FF80143F4B15004D5A60 027F4A5A4B495A4D5AEF3F8002FF02FEC7FC92380007F892B512E01780499038000FE04A 6D7E707E707E0103814A130083A213075CA25E130F5C5F1603131F5CA3013F020714404A 16E05F017F160119C04A01031303496C1680B6D8800113079438FE0F009338007E1ECAEA 3FFCEF07F03B407BBD42>82 D<147E49B47E903907C1C38090391F80EFC090383F00FF01 7E137F4914804848133F485AA248481400120F5B001F5C157E485AA215FE007F5C90C7FC A21401485C5AA21403EDF0385AA21407EDE078020F1370127C021F13F0007E013F13E000 3E137FECF3E1261F01E313C03A0F8781E3803A03FF00FF00D800FC133E252977A72E>97 D 99 DII<167C4BB4 FC923807C78092380F83C0ED1F87161FED3F3FA2157EA21780EE0E004BC7FCA414015DA4 14035DA30103B512F8A390260007E0C7FCA3140F5DA5141F5DA4143F92C8FCA45C147EA4 14FE5CA413015CA4495AA4495AA4495A121E127F5C12FF49C9FCA2EAFE1EEAF83C1270EA 7878EA3FE0EA0F802A5383BF1C>I<1478EB01FCA21303A314F8EB00E01400AD137C48B4 FC38038F80EA0707000E13C0121E121CEA3C0F1238A2EA781F00701380A2EAF03F140012 005B137E13FE5BA212015BA212035B1438120713E0000F1378EBC070A214F0EB80E0A2EB 81C01383148038078700EA03FEEA00F8163E79BC1C>105 D108 DIII<903903E001F890390FF807FE903A1E7C1E0F80903A1C3E3C07C001 3C137801389038E003E0EB783F017001C013F0ED80019038F07F0001E015F8147E160300 0113FEA2C75AA20101140717F05CA20103140F17E05CA20107EC1FC0A24A1480163F010F 15005E167E5E131F4B5A6E485A4B5A90393FB80F80DA9C1FC7FCEC0FFCEC03E049C9FCA2 137EA213FEA25BA21201A25BA21203A2387FFFE0B5FCA22D3A80A72E>I114 DII<137C48B4141C26038F80137EEA0707000E7F001E15FE 121CD83C0F5C12381501EA781F007001805BA2D8F03F1303140000005D5B017E1307A201 FE5C5B150F1201495CA2151F0003EDC1C0491481A2153F1683EE0380A2ED7F07000102FF 13005C01F8EBDF0F00009038079F0E90397C0F0F1C90391FFC07F8903907F001F02A2979 A731>I<017CEB01C048B4EB07F038038F80EA0707000E01C013F8121E001C1403EA3C0F 0038EC01F0A2D8781F130000705BA2EAF03F91C712E012005B017E130116C013FE5B1503 000115805BA2ED07001203495B150EA25DA25D1578000114706D5B0000495A6D485AD97E 0FC7FCEB1FFEEB03F0252979A72A>I E /Fi 35 122 df<12FFA8080877871B>46 D<1418147814F81303130F137FB5FCA413F71387EA0007B3B3AD007FB61280A6214378C2 31>49 DI<49B4FC010F13E001 3F13F890B512FE48800007158048010113C03A1FF8007FE0D83FE0133F49EB1FF048C712 0F123E16F8001C140712181208C8FC150FA216F0A2151F16E0153F16C0157FEDFF800203 1300EC1FFE90380FFFFC5D15E08115FC15FFD9000113809138007FC0ED1FE0ED0FF016F8 ED07FCA2ED03FEA3ED01FFA80040EC03FE1260A200F0EC07FC7E00FEEC0FF8127FD83FC0 EB1FF0D81FF0EB7FE0390FFE01FF6CB612806C15006C5C6C6C13F8011F13E0010190C7FC 28457CC231>II<000FB612F8A601F0C8 FCB1EC3FC09038F1FFF001F313FC90B57EEDFF8016C002E013E0EC003F49EB1FF049EB0F F84913074914FC15034914FEC8FCA2150116FFAAED03FE12101230003815FC007C140700 7E15F800FE140FD87F80EB1FF06DEB3FE0D83FF0137F3A1FFE03FFC06CB6128000031500 6C14FC6C6C5B011F13C0D903FEC7FC28437DC031>III<49B47E010F13F0013F13FC497F48B612804815C048 010013E0D80FFCEB3FF001F0130F4848EB07F84913034848EB01FCA2007F15FE90C8FCA7 6C6CEB01FCA26C6CEB03F8A26C6CEB07F06C6CEB0FE0D803FCEB3FC03A01FF81FF806C90 B51200013F13FC010F13F0A2013F13FC90B6FC4801001380D803FCEB3FC04848EB1FE048 48EB0FF04848EB07F84848EB03FCA24848EB01FEA290C8FC4815FFA86C15FE6D1301A26D 1303003F15FC6D13076C6CEB0FF86C6CEB1FF06D133F3A07FF81FFE06C90B512C06C1580 6C6CEBFE006D5B010F13F0010090C7FC28457CC231>I 66 D71 D73 D78 D80 D83 DI87 D97 DI III103 DII107 D<12FFB3B3B3AF084579C417>I 110 DI<141F00FE13FF13 035B131F5B5BEBFFF0148038FFFE005B5B5B5B5BA25BA390C7FCB3A8182D79AC21>114 DIII119 D121 D E /Fj 24 122 df46 D<14075C5C147F5C1307133F0007B5FCB6FCA413F913C1EAF8011200B3B3B3A4B712FEA7 275075CF3B>49 D52 D67 D80 D<913803FFC0023F13FE49B612C0010715F04915FC013F15FF491680 90B8FC489038FC007F4801E001071300480180130191C87ED80FFC814848811707484815 024992C7FCA3485AA67FA2123F7F7FA26C7E13FF6C7F6C13E014FC6CEBFFC06C14FC6CEC FFC06D14F86D14FE010F6E7E6D81010115F0D9003F80020780DA007F7F03077FED007F04 1F1380040713C082040013E0A2EF7FF0173FA2171F18F8170FA8EF1FF0A212200070EE3F E0127C007E167F007FEEFFC001C05C01F04A1380D8FFFC020F1300D9FF80EB3FFE9139F8 01FFFC6C90B65A001F5E00075E00015E6C6C92C7FC010F14FC010114F0D9000F90C8FC35 577BD441>83 D<903801FFC0011F13F8017F13FE0003B67E000F814881829138007FF801 F0EB1FFCD80FC0130F90C76C7E000E1403000C81C87EA26F1380AA0207B5FC91B6FC1307 133F90B7FC5A000714004813E04848C7FCEA3FF813E0485A5B48C8FCA55D7F6C6C5B6D13 0F6D5B263FFE01B5FC90B7FC7E6C5C6C14F86C14E06C140026007FE090C7FC29387BB638 >97 DI< EC0FFF027F13F049B512FC010714FF4915C0013F15E05B9038FFF801489039E0003FC048 0180130F91C71203484814014848EC0040491500485AA2485AA3485AA4485AAB6C7EA46C 7EA26C7EA26D15206C6C15600007ED01E001FF14036C6DEB0FF06C01E0133F6C9038F803 FF6DB6FC6D15E06D15800107ECFE00010114F86D6C13E0DA0FFEC7FC2C387CB634>IIIII105 D108 DIIII114 D<903803FFC0013F13FC90B6FC4815E04815F05A5A3A1FFC007FE001F0130F4848130349 1300162048481400A57FA26C7E7F13FC381FFFE06C13FEECFFE06C14F86C14FE6C806C6C 1480011F14C0010714E0D9003F13F014019138007FF8151F150FED07FCA21503A5127000 78EC07F8127C007F140F01E0EB3FF0D8FFFC13FF90B612E016C06C1580001F1500000314 FCC66C13F00107138026387DB62D>II< B46CECFF80B3B3A35DA25DA25D6D5B007F5C6D137FD9FC01B5FC6CB55A6C5C6C14F06C14 C06C49C6FC26007FE090C7FC293677B43C>I121 D E /Fk 14 118 df[<0803B500C0EE01F00703B600FEEE03F8077FDBFFE0150706 07B800FC150F063F05FF151F4DBA00E0143F050F07F8147F053F07FE14FF94BC5B040393 26F8000FECC003040F4BC86CEBF007043F03C0030F6D5A93B648C900036D5A4B03F09339 007FFF3F030703C0051F90B5FC4B92CB7E033F02FC18034B02F08492B648844A0380193F 4A92CD7E4A4A864A4A864A02F0864A4A864A8991B65A494B874992CF7E4C885B494A885E 498B494A88A2495C8D90B65A8D5A5E48217FA24892D1FC223FA25A5DA248211FA3485CFA 0FF09FC7FCA25AA45DA3B6FCB27EA381A47EA46C80FA07F0FA0FF87EA2817EA36C6F1D1F 23F07E827E223F6D6E1EE0A26D6E1D7F23C06D6E1DFF7F705213806D806D55130070646D 6F646D6F515A6E6E1B1F6E6E515A6E6E515A6E6E1BFF6E6E505B6E6E505B6E6F4F5B6E03 E04F90C7FC6F6EF13FFE6F02FC4F5A030F02FF4E485A6F03C005075B030103F0051F5B6F 03FE057F1380043FDAFFE00303B5C8FC040F03FE033F13FC0403DBFFF80107B55A040093 B812E0053F1A80050F4FC9FC050119F8DD003F18C0060795CAFCDE007F16F0070393CBFC DF000314C0>141 146 115 271 168 67 D[<97B512F0077FECFFE00607B712FE067FEE FFE00503B912FC051FF0FF80057F19E00403BB12FC040F9226E0007F14FF043F02FCC700 0315C04C02E0DA007F804BB60080031F14F8030702FCC9000314FE4B4A70804B02E0706C 80037F0280051F14E092B6CB6C804A4A72804A4A72804A02F00600804A4A737F4A4A7380 4A8B4A4A738091B6CD6C80494A7480A2494A7480494A7480498C4C86498D4C87498D494A 7580A290B68B4C87488EA24892CF6C80A3488E4B88A2488EA3484A761580A34823C0A548 4A7615E0A7B621F0B36C23E0A26F64A56C23C0A46F646C2380A36C23006F64A26C6AA270 636C6AA26C6A70636C6A70636D69A26D6E98B65AA26D6E505DA26D6E5092C7FC6D687062 6D6E505C6D686D6F4F5C6E6E4F5C6E6E4F5CA26E6E96B65A6E6E4E92C8FC6E6E4E5C0201 02FF060F14F86E6F4D5C6F6E4D5C6F02F094B65A030F6E4C92C9FC6F02FE04075C03016E 6C031F14F86F03F092B65A043F02FE020715C0040FDAFFF090B7CAFC040392B812FC0400 1AF0051F198005074ECBFCDD007F17E0060F94CCFCDE007F15E0070002F0CDFC>148 146 115 271 175 79 D[<93260FFFF8163E4BB600E0153F031F03FE5D037FDBFFC05C02 03B800F05B020F05FC5B4A05FF5B027FF0C00F91B526FE000FECF01F010302C0D9007F6D 5A4991C800076D5A4901FC030090B6FC4901F0163F4949160F4901808290B5170192CBFC 4849844849181F87484984A2484984874886A248498588A24887A388A2B58680A36E85A2 80A26E8580A2818103F0725A6C6E96C7FC15FE8116E06C15FEEEFFE017FF6C17F0F0FF80 6C18F8F1FFC06C19FCF2FF806C1AE01BF86C1AFE6C747E6D1AE0886D866D866D1AFF6D87 6D87010087806E86020F86020386020086153F030F851501DB001F19801601DC000F18C0 EF007F060717E0F0003F070316F0F1003F1A0F080315F81A00871B1F877514FCA287007F 86486C85A288A388A36D86A31EF87FA37F1EF0A26D626D1CE0A27F6D5013C0A26E1B806E 96B5FC6E1B0002F8606E4E5B6E626E6C5F03E04D5B03F84D5B03FE057F5BDBFFC093B55A 04F803035C496CD9FF80021F91C7FCD9FC1F02FF49B55AD9F80792B75A496C19F049C661 49011F18804901074DC8FC90C817F848031F16C048030003FCC9FC007C04011480>102 146 115 271 129 83 D<93B512FC037FECFFF00207B8FC023F17E091B912F84918FE01 07727E499126C0007F14E04901E0C7000F80496D020380496D020014FE6F6F7F90B57080 6F6F8085486E6F807380A27380A28885886C5CA26D4982886D5B6D5B010713C0010190CA FC90CCFCA90603B7FC050FB8FC0403B9FC167F0307BAFC153F4AB7EA807F020FEDE00002 3F02FCC7FC91B612E0010392C8FC4914FC011F14F04914C0495C90B548C9FC485C485C48 5C485C5A5D485CA24891CAFCA3B6FC5CA397B6FCA461806C60F107EF6C6E150F6F16CF6C 183F6FDB7F8F806C6EDBFF0F14E06C02FCDA03FE15FE6C6E91260FFC0791B5FC6C6E6CD9 3FF817806C923AF803FFF003013F91B6487E010FEF8000010394C77E010004FC141F021F 03F0140702010380DA007F1400DA000701F8CDFC695F79DD71>97 D<94387FFFF0041FB612E093B712FE0307707E031F17F092B97E4A18FE020784021F9126 F8000F14804A0280010014C04A49C74814E049B500F85C494A17F0494A5C495C494A4A14 F84991C8FC5D495B90B5FC5D5A485C7314F05A4B6F14E05A7314C0487214804B93383FFE 00F20FF84896C8FCA4485CA5B6FCB07EA281A37EA36C80A37E6F18FE6CF201FFA26C6E5F 1CFE6C801B076C6EEF0FFC6D7F70EE1FF86DF13FF06D6E167F6D6EEEFFE06D02F84B13C0 6D6E5D6D02FF030F13806D03C0023F1300023F02F0903801FFFC6E9126FF801F5B020792 B65A6E18C0020060033F4CC7FC030716F8030016C0041F4AC8FCDC007F13C0585F78DD67 >99 D[113 144 120 270 129 I<94387FFFC0040FB6FC93B712E0030716FC031F16FF037F17C04AB912F0 0207DAF80380021F912680003F13FE4A49C7000F7F4A01F802038049B5486E804902C06E 6C7F494A6F7F4991C9FC49727F4949707F4B84498490B548707F5A4B198048855D481CC0 86481CE05D5A871DF05AA25D5AA21DF887A2B6FCA392BBFCA51DF00380CDFCA77EA4817E A37EA2817EA26CF307F06FF00FF87E816C1B1F6F19F06C1B3F6D6DF07FE06D7FF4FFC06D 6E4C13806D6E5E6D02F04C13006D6EEE1FFE6D6E4C5A6D6C01FFEEFFF86E02E002035B6E 02FC021F5B02079126FFC003B55A6E92B7C7FC020060033F17F8030F17E003011780DB00 3F03FCC8FC040315C0DC000F01F8C9FC5D5F7ADD6A>I[49 144 119 271 65 105 D109 DI<94381FFFF00407B612C0 047F15FC0303B87E030F17E0037F17FC4ABAFC4A9126FC007F80020F02C0010714E04A49 C880027F01F8033F13FC91B5486F7F4902C003077F494A6F804991C96C80494970804949 717F49874949717FA290B548717F48884B83481D80A2481DC04B83481DE0A2481DF0A348 4A7114F8A4481DFCA5B61BFEAF6C1DFCA56C6E4D14F8A36C1DF0A36C1DE06F5F6C1DC0A2 6C6E4D1480A26C1D006F5F6C646D6D4D5B6F94B5FC6D636D6D4C5C6D6E4B5C6D6E4B5C6D 02F0031F5C6D6E4B91C7FC6D6C01FE92B512FC6ED9FFC001075C6E02FC017F5C020791B8 12C0020196C8FC6E6C17FC031F17F003031780DB007F03FCC9FC040715C0DC001F01F0CA FC675F7ADD74>I<92261FFFF814F80203B638C001FC023FEDFC0791B8121F010317FF13 0F013F9038F8001F4990C8FCD9FFF8153F4801E0150F484915034849814890CAFC197F48 48173F191F485AA2007F180FA31907487EA27FA28002E0705A6E93C8FC14FC14FF15F06C ECFF8016FCEEFFF06CEEFF8018F06C17FE727E6C18E0856C18FC6C846C727E6C856D8401 1F846D841303010084023F83140F020183EC001FDB007F16801603DC000F15C01700183F 060F14E0007F1703486C82727E857F85857FA2857F1BC07FA27F1B806D5F7F1B006E5E6E 5F6E163F6E4C5A02FC4C5A6E03035B6E6C4A5B03F0023F5B03FF0107B55A01F991B7C7FC D9F07F16FCD9E01F16F0D9800716C0D9000193C8FC48D9003F14F8007C020349C9FC4B5F 78DD5C>115 D[72 132 124 258 90 II E /Fl 8 117 df52 D<96267FFFE01670063FB6ED01F80503B700F01403053F04FC14074CB96C130F040706E0 131F043F72133F93BA00FC137F0303DC00076D13FF030F03C09039003FFF814B02FCC800 0713C3037F02E0030113F792B600806F6CB5FC02034ACA121F4A02F8834A02E0834A4A17 01027F4A8391B548CC7E494A85495C4C854988494A85494A85495C8A4991CDFC90B54886 A2484A1B7FA2481E3F5D481E1F5D5A1F0FA2485CA3481E075DA2F703F0489BC7FCA45DA2 B6FCB27EA281A47EA2F703F06FF307F87EA36C80A21F0F7E6F1CF07E6F1B1F7E20E06C6E 1B3F816DF57FC06D80F7FF806D806D6E4F13006D6E616D525A826D6E4F5A6D6E4F5A6E6D 6C4E5A021F6EF0FFE06E6E4D5B6E02F84D5B6E02FE050F90C7FC02006E6CEE3FFE6F02F0 EEFFFC031F02FE03035B6FDAFFC0021F13E0030303FF0103B55A030093B7C8FC043F18FC 040718F0040118C0DC003F94C9FC050316F8DD003F1580DE007F01F0CAFC757A75F78C> 67 D<92383FFFF80207B612E0027F15FC49B87E010717E0011F83499026F0007F13FC49 48C7000F7F90B502036D7E486E6D806F6D80727F486E6E7F8486727FA28684A26C5C7280 6C5C6D90C8FC6D5AEB0FF8EB03E090CAFCA70507B6FC041FB7FC0303B8FC157F0203B9FC 021FECFE0391B612800103ECF800010F14C04991C7FC017F13FC90B512F04814C0485C48 91C8FC485B5A485B5C5A5CA2B5FC5CA360A36E5DA26C5F6E5D187E6C6D846E4A48806C6D 4A4814FC6C6ED90FF0ECFFFC6C02E090263FE07F14FE00019139FC03FFC06C91B6487E01 3F4B487E010F4B1307010303F01301D9003F0280D9003F13FC020101F8CBFC57507ACE5E >97 D<93387FFF80030FB512FC037FECFF804AB712E0020716F8021F16FE027FD9F8077F 49B5D8C000804991C7003F13E04901FC020F7F49496E7F49498049496E7F49496E7F90B5 5A48727E92C914804884485B1BC048841BE0485BA27313F05AA25C5AA21BF885A2B5FCA3 91BAFCA41BF002F8CCFCA67EA3807EA47E806CF103F0F207F86C7F1A0F6C6E17F06C191F 6F17E06C6E163F6D6DEE7FC06D6D16FF6D6D4B13806D6D4B13006D6D6CEC0FFE6D02E0EC 3FFC6D02F8ECFFF86D9126FFC00F5B023F91B65A020F178002034CC7FC020016F8031F15 E0030392C8FCDB000F13E04D507BCE58>101 D<903801FFFCB6FCA8C67E131F7FB3AD95 380FFFE095B512FE05036E7E050F15E0053F15F84D81932701FFF01F7F4CD900077FDC07 FC6D80DC0FF06D80DC1FC07F4C48824CC8FC047E6F7F5EEDFDF85E03FF707F5EA25EA25E A293C9FCA45DB3B3A6B8D8E003B81280A8617879F76C>104 D<902601FFFCEC7FFEB602 0FB512F0057F14FE4CB712C0040716F0041F82047F16FE93B5C66C7F92B500F0010F14C0 C66C0380010380011F4AC76C806D4A6E8004F06F7F4C6F7F4C6F7F4C8193C915804B7014 C0861DE0A27414F0A27414F8A47513FCA57513FEAF5113FCA598B512F8A31DF0621DE062 1DC0621D806F5E701800704B5B505B704B5B7092B55A04FC4A5C704A5C706C010F5C05E0 013F49C7FC9227FE7FFC01B55A70B712F0040F16C0040393C8FC040015F8053F14C00503 01F0C9FC94CCFCB3A6B812E0A85F6F7ACD6C>112 D<902601FFF8EB07FEB691383FFFC0 94B512F00403804C14FE4C8093261FFC3F138093263FE07F13C0DC7F80B5FCC66C5D011F DAFE0114E06DEBF9FC16F815FB16F016E015FF16C07114C05E72138095381FFE0093C76C 5AF001E095C8FCA25DA65DB3B3A2B812F8A8434E7ACD4F>114 D<15FFA75CA55CA45CA2 5CA25CA25CA25C91B5FCA25B5B5B131F5B90B9FC120FBAFCA6D8000791C9FCB3B3A3F01F E0AE183F7014C07F187F7014806D16FF826D4B13006E6D485AEEFE0F6E90B55A020F5D6E 5D020115C06E6C5C031F49C7FC030113F03B6E7CEC4B>116 D E /Fm 57 123 df<4AB4EB0FE0021F9038E03FFC913A7F00F8FC1ED901FC90383FF03FD907 F090397FE07F80494801FF13FF4948485BD93F805C137F0200ED7F00EF003E01FE6D91C7 FC82ADB97EA3C648C76CC8FCB3AE486C4A7E007FD9FC3FEBFF80A339407FBF35>11 D<4AB4FC021F13C091387F01F0903901FC0078D907F0131C4948133E494813FF49485A13 7F1400A213FE6F5A163893C7FCAA167FB8FCA33900FE00018182B3AC486CECFF80007FD9 FC3F13FEA32F407FBF33>I<1430147014E0EB01C0EB03801307EB0F00131E133E133C5B 13F85B12015B1203A2485AA2120F5BA2121F90C7FCA25AA3123E127EA6127C12FCB2127C 127EA6123E123FA37EA27F120FA27F1207A26C7EA212017F12007F13787F133E131E7FEB 07801303EB01C0EB00E014701430145A77C323>40 D<12C07E12707E7E121E7E6C7E7F12 036C7E7F12007F1378137CA27FA2133F7FA21480130FA214C0A3130714E0A6130314F0B2 14E01307A614C0130FA31480A2131F1400A25B133EA25BA2137813F85B12015B485A1207 5B48C7FC121E121C5A5A5A5A145A7BC323>I<1506150FB3A9007FB912E0BA12F0A26C18 E0C8000FC9FCB3A915063C3C7BB447>43 D<121EEA7F8012FF13C0A213E0A3127FEA1E60 1200A413E013C0A312011380120313005A120E5A1218123812300B1C798919>I<121EEA 7F80A2EAFFC0A4EA7F80A2EA1E000A0A798919>46 DI49 DII<150E151E153EA2 157EA215FE1401A21403EC077E1406140E141CA214381470A214E0EB01C0A2EB0380EB07 00A2130E5BA25B5BA25B5B1201485A90C7FC5A120E120C121C5AA25A5AB8FCA3C8EAFE00 AC4A7E49B6FCA3283E7EBD2D>I<00061403D80780131F01F813FE90B5FC5D5D5D15C092 C7FC14FCEB3FE090C9FCACEB01FE90380FFF8090383E03E090387001F8496C7E49137E49 7F90C713800006141FC813C0A216E0150FA316F0A3120C127F7F12FFA416E090C7121F12 FC007015C012780038EC3F80123C6CEC7F00001F14FE6C6C485A6C6C485A3903F80FE0C6 B55A013F90C7FCEB07F8243F7CBC2D>I<15074B7EA34B7EA34B7EA34B7EA34B7E15E7A2 913801C7FC15C3A291380381FEA34AC67EA3020E6D7EA34A6D7EA34A6D7EA34A6D7EA34A 6D7EA349486D7E91B6FCA249819138800001A249C87EA24982010E157FA2011E82011C15 3FA2013C820138151FA2017882170F13FC00034C7ED80FFF4B7EB500F0010FB512F8A33D 417DC044>65 DI68 DIII73 D<011FB512FCA3D9000713006E5A1401B3B3A6123F EA7F80EAFFC0A44A5A1380D87F005B007C130700385C003C495A6C495A6C495A2603E07E C7FC3800FFF8EB3FC026407CBD2F>I76 DIIII82 DI<003FB91280A3903AF0007FE001018090393FC0003F48C7ED1FC0007E17 07127C00781703A300701701A548EF00E0A5C81600B3B14B7E4B7E0107B612FEA33B3D7D BC42>I87 D<007FB5D8C003B512E0A3C649C7EBFC00D93FF8EC3FE06D48EC1F806D6C 92C7FC171E6D6C141C6D6C143C5F6D6C14706D6D13F04C5ADA7FC05B023F13036F485ADA 1FF090C8FC020F5BEDF81E913807FC1C163C6E6C5A913801FF7016F06E5B6F5AA26F7E6F 7EA28282153FED3BFEED71FF15F103E07F913801C07F0203804B6C7EEC07004A6D7E020E 6D7E5C023C6D7E02386D7E14784A6D7E4A6D7F130149486E7E4A6E7E130749C86C7E496F 7E497ED9FFC04A7E00076DEC7FFFB500FC0103B512FEA33F3E7EBD44>I97 DI<49B4FC010F13 E090383F00F8017C131E4848131F4848137F0007ECFF80485A5B121FA24848EB7F00151C 007F91C7FCA290C9FC5AAB6C7EA3003FEC01C07F001F140316806C6C13076C6C14000003 140E6C6C131E6C6C137890383F01F090380FFFC0D901FEC7FC222A7DA828>IIII<167C903903F801FF903A1FFF078F8090397E0FDE1F9038F803F83803F0 01A23B07E000FC0600000F6EC7FC49137E001F147FA8000F147E6D13FE00075C6C6C485A A23901F803E03903FE0FC026071FFFC8FCEB03F80006CAFC120EA3120FA27F7F6CB512E0 15FE6C6E7E6C15E06C810003813A0FC0001FFC48C7EA01FE003E140048157E825A82A46C 5D007C153E007E157E6C5D6C6C495A6C6C495AD803F0EB0FC0D800FE017FC7FC90383FFF FC010313C0293D7EA82D>III<1478EB01FEA2EB03FFA4EB01FE A2EB00781400AC147FEB7FFFA313017F147FB3B3A5123E127F38FF807E14FEA214FCEB81 F8EA7F01387C03F0381E07C0380FFF803801FC00185185BD1C>III<2701F801FE14FF00FF902707FFC00313E0913B1E 07E00F03F0913B7803F03C01F80007903BE001F87000FC2603F9C06D487F000101805C01 FBD900FF147F91C75B13FF4992C7FCA2495CB3A6486C496CECFF80B5D8F87FD9FC3F13FE A347287DA74C>I<3901F801FE00FF903807FFC091381E07E091387803F000079038E001 F82603F9C07F0001138001FB6D7E91C7FC13FF5BA25BB3A6486C497EB5D8F87F13FCA32E 287DA733>I<14FF010713E090381F81F890387E007E01F8131F4848EB0F804848EB07C0 4848EB03E0000F15F04848EB01F8A2003F15FCA248C812FEA44815FFA96C15FEA36C6CEB 01FCA3001F15F86C6CEB03F0A26C6CEB07E06C6CEB0FC06C6CEB1F80D8007EEB7E009038 3F81FC90380FFFF0010090C7FC282A7EA82D>I<3901FC03FC00FF90381FFF8091387C0F E09039FDE003F03A03FFC001FC6C496C7E91C7127F49EC3F805BEE1FC017E0A2EE0FF0A3 EE07F8AAEE0FF0A4EE1FE0A2EE3FC06D1580EE7F007F6E13FE9138C001F89039FDE007F0 9039FC780FC0DA3FFFC7FCEC07F891C9FCAD487EB512F8A32D3A7EA733>I<02FF131C01 07EBC03C90381F80F090397F00387C01FC131CD803F8130E4848EB0FFC15074848130312 1F485A1501485AA448C7FCAA6C7EA36C7EA2001F14036C7E15076C6C130F6C7E6C6C133D D8007E137990383F81F190380FFFC1903801FE0190C7FCAD4B7E92B512F8A32D3A7DA730 >I<3901F807E000FFEB1FF8EC787CECE1FE3807F9C100031381EA01FB1401EC00FC01FF 1330491300A35BB3A5487EB512FEA31F287EA724>I<90383FC0603901FFF8E03807C03F 381F000F003E1307003C1303127C0078130112F81400A27E7E7E6D1300EA7FF8EBFFC06C 13F86C13FE6C7F6C1480000114C0D8003F13E0010313F0EB001FEC0FF800E01303A21401 7E1400A27E15F07E14016C14E06CEB03C0903880078039F3E01F0038E0FFFC38C01FE01D 2A7DA824>I<131CA6133CA4137CA213FCA2120112031207001FB512C0B6FCA2D801FCC7 FCB3A215E0A912009038FE01C0A2EB7F03013F138090381F8700EB07FEEB01F81B397EB7 23>IIIIII<001FB61280A2EBE000018014 0049485A001E495A121C4A5A003C495A141F00385C4A5A147F5D4AC7FCC6485AA2495A49 5A130F5C495A90393FC00380A2EB7F80EBFF005A5B484813071207491400485A48485BA2 48485B4848137F00FF495A90B6FCA221277EA628>I E /Fn 47 122 df<121EEA7F8012FF13C0A213E0A3127FEA1E601200A413E013C0A31201138012031300 5A1206120E5A5A5A12600B1D78891B>44 DI<121EEA7F80A2EA FFC0A4EA7F80A2EA1E000A0A78891B>I<14FF010713E090381F81F890383E007C01FC13 3F4848EB1F8049130F4848EB07C04848EB03E0A2000F15F0491301001F15F8A2003F15FC A390C8FC4815FEA54815FFB3A46C15FEA56D1301003F15FCA3001F15F8A26C6CEB03F0A3 6C6CEB07E0000315C06D130F6C6CEB1F806C6CEB3F00013E137C90381F81F8903807FFE0 010090C7FC28447CC131>48 D<143014F013011303131F13FFB5FC13E713071200B3B3B0 497E497E007FB6FCA3204278C131>I I<49B4FC010F13E0013F13FC9038FE01FE3A01F0007F80D803C0EB3FC048C7EA1FE0120E ED0FF0EA0FE0486C14F8A215077F5BA26C48130FEA03C0C813F0A3ED1FE0A2ED3FC01680 ED7F0015FE4A5AEC03F0EC1FC0D90FFFC7FC15F090380001FCEC007FED3F80ED1FC0ED0F E016F0ED07F816FC150316FEA2150116FFA3121EEA7F80487EA416FE491303A2007EC713 FC00701407003015F80038140F6C15F06CEC1FE06C6CEB3FC0D803E0EB7F803A01FE01FE 0039007FFFF8010F13E0010190C7FC28447CC131>II<000615C0D807C01307 01FCEB7F8090B612005D5D5D15E0158026063FFCC7FC90C9FCAE14FF010713C090381F01 F090383800FC01F0137ED807C07F49EB1F8016C090C7120F000615E0C8EA07F0A316F815 03A216FCA5123E127F487EA416F890C712075A006015F0A20070140F003015E00038EC1F C07E001EEC3F806CEC7F006C6C13FE6C6C485A3901F807F039007FFFE0011F90C7FCEB07 F826447BC131>II<14FF010713E0011F13F890387F 00FE01FC133FD801F0EB1F804848EB0FC049EB07E00007EC03F048481301A290C713F848 1400A47FA26D130116F07F6C6CEB03E013FC6C6CEB07C09039FF800F806C9038C01F006C EBF03EECF87839007FFEF090383FFFC07F01077F6D13F8497F90381E7FFFD97C1F138049 6C13C02601E00313E048486C13F000079038007FF84848EB3FFC48C7120F003EEC07FE15 0148140016FF167F48153FA2161FA56C151E007C153EA2007E153C003E157C6C15F86DEB 01F06C6CEB03E06C6CEB07C0D803F8EB1F80C6B4EBFF0090383FFFFC010F13F001011380 28447CC131>56 D<14FF010713E0011F13F890387F80FC9038FC007E48487F4848EB1F80 4848EB0FC0000FEC07E0485AED03F0485A16F8007F140190C713FCA25AA216FE1500A516 FFA46C5CA36C7E5D121F7F000F5C6C6C1306150E6C6C5B6C6C5BD8007C5B90383F01E090 390FFF80FE903801FE0090C8FC150116FCA4ED03F8A216F0D80F801307486C14E0486C13 0F16C0ED1F80A249EB3F0049137E001EC75A001C495A000F495A3907E01FE06CB51280C6 49C7FCEB1FF028447CC131>I<16C04B7EA34B7EA34B7EA34B7EA3ED19FEA3ED30FFA203 707FED607FA203E07FEDC03FA2020180ED801FA2DA03007F160FA20206801607A24A6D7E A34A6D7EA34A6D7EA20270810260147FA202E08191B7FCA249820280C7121FA249C87F17 0FA20106821707A2496F7EA3496F7EA3496F7EA201788313F8486C83D80FFF03037FB500 E0027FEBFFC0A342477DC649>65 D 67 D69 DI73 D<010FB512FEA3D90003 13806E130080B3B3AB123F487E487EA44A5A13801300006C495A00705C6C13076C5C6C49 5A6CEB1F802603E07FC7FC3800FFFCEB1FE027467BC332>I77 DI80 D82 D<49B41303010FEBE007 013F13F89039FE00FE0FD801F8131FD807E0EB079F49EB03DF48486DB4FC48C8FC488100 3E81127E82127C00FC81A282A37E82A27EA26C6C91C7FC7F7FEA3FF813FE381FFFE06C13 FE6CEBFFE06C14FC6C14FF6C15C0013F14F0010F80010180D9001F7F14019138001FFF03 031380816F13C0167F163F161F17E000C0150FA31607A37EA36C16C0160F7E17806C151F 6C16006C5D6D147ED8FBC05CD8F9F0495AD8F07C495A90393FC00FE0D8E00FB512800101 49C7FC39C0003FF02B487BC536>I<003FB912F8A3903BF0001FF8001F01806D48130300 3EC7150048187C0078183CA20070181CA30060180CA5481806A5C81600B3B3A54B7EED7F FE49B77EA33F447DC346>I97 DII<167FED3FFFA315018182B3EC7F809038 03FFF090380FC07C90383F000E017E1307496D5AD803F87F48487F5B000F81485AA2485A A2127FA290C8FC5AAB7E7FA2123FA26C7EA2000F5D7F6C6C5B00035C6C6C9038077F806C 6C010E13C0013F011C13FE90380FC0F8903803FFE09026007F0013002F467DC436>IIIIII<143C14FFA2491380A46D1300A2143C 91C7FCADEC7F80EB3FFFA31300147F143FB3B3AA123E127F39FF807F00A2147EA25C6C48 5A383C01F06C485A3807FF80D801FEC7FC195785C21E>IIII<3901FC01FE00FF903807FFC091381E07F091 383801F8000701707F0003EBE0002601FDC07F5C01FF147F91C7FCA25BA35BB3A8486CEC FF80B5D8F83F13FEA32F2C7DAB36>II<39 01FC03FC00FF90380FFF8091383C07E091387001F83A07FDE000FE00010180137F01FFEC 3F8091C7EA1FC04915E049140F17F0160717F8160317FCA3EE01FEABEE03FCA3EE07F8A2 17F0160F6D15E0EE1FC06D143F17806EEB7E00D9FDC05B9039FCF003F891383C0FE09138 1FFF80DA03FCC7FC91C9FCAE487EB512F8A32F3F7DAB36>I<3903F803F000FFEB1FFCEC 3C3EEC707F0007EBE0FF3803F9C000015B13FBEC007E153C01FF13005BA45BB3A748B4FC B512FEA3202C7DAB26>114 D<90383FE0183901FFFC383907E01F78390F0003F8001E13 01481300007C1478127800F81438A21518A27EA27E6C6C13006C7E13FC383FFFE06C13FC 6C13FF6C14C06C14E0C614F0011F13F81300EC0FFC140300C0EB01FE1400157E7E153EA2 7EA36C143C6C147C15786C14F86CEB01F039F38003E039F1F00F8039E07FFE0038C00FF0 1F2E7DAC26>I<1306A5130EA4131EA3133E137EA213FE12011207001FB512F0B6FCA2C6 48C7FCB3A4150CAA017E131C017F1318A26D133890381F8030ECC070903807E0E0903801 FFC09038007F001E3E7EBC26>IIII 121 D E /Fo 20 118 df45 D68 D73 D77 D80 D83 D97 D<4AB47E020F13F8023F13FE9139FF007F80D903FCEB07E0D907F0EB01F0D91FE0EB0078 49488049488049C87E48485D4915FF00034B138048485CA2485AA2485AA2003F6F130049 EC007C94C7FC127FA35B12FFAD127F7FA4123F7FA2001FEE01C07F000F16036D16801207 6C6C15076D160000015E6C6C151E6D6C5C6D6C5C6D6C5CD90FF8495AD903FCEB07C0903A 00FF803F8091263FFFFEC7FC020F13F80201138032417CBF3A>99 D<181EEF3FFEEE07FFA4EE000F1703A21701B3AAEDFF80020F13F8023F13FE9139FF803F 81903A03FC0007C14948EB01E1D91FE0EB00F94948147D4948143D49C8121F4848150F49 1507120348481503491501120F121F5BA2123F5B127FA45B12FFAD127F7FA3123FA27F12 1FA26C6C1503A26C6C150712036D150F6C6C151F0000163D137F6D6CECF9FF6D6CEB01F1 D90FF0D903C113C06D6CD90F81EBFF80D901FFEB7F019039007FFFFC021F13E002010100 91C7FC41657CE349>IIII<133C13FF487F487FA66C5B6C90C7FC133C90C8FCB3A2EB03C0EA07FF12 7FA41201EA007FA2133FB3B3AC497E497EB612E0A41B5F7DDE23>105 D109 D II<90 39078003F8D807FFEB0FFFB5013F13C092387C0FE0913881F01F9238E03FF00001EB8380 39007F8700148FEB3F8E029CEB1FE0EE0FC00298EB030002B890C7FCA214B014F0A25CA5 5CB3B0497EEBFFF8B612FCA42C3F7CBE33>114 D<9139FFE00180010FEBFC03017FEBFF 073A01FF001FCFD803F8EB03EFD807E0EB01FF48487F4848147F48C8123F003E151F007E 150F127CA200FC1507A316037EA27E7F6C7E6D91C7FC13F8EA3FFE381FFFF06CEBFF806C 14F86C14FF6C15C06C6C14F0011F80010714FED9007F7F02031480DA003F13C015030300 13E0167F00E0ED1FF0160F17F86C15071603A36C1501A37EA26C16F016037E17E06D1407 6DEC0FC06D1580D8FDF0141FD8F8F8EC7F00013E14FC3AF01FC00FF80107B512E0D8E001 148027C0003FF8C7FC2D417DBF34>I<1438A71478A414F8A31301A31303A21307130F13 1FA2137F13FF1203000F90B6FCB8FCA3260007F8C8FCB3AE17E0AE6D6CEB01C0A316036D 6C148016076D6C14006E6C5A91383FC01E91381FF07C6EB45A020313E09138007F802B59 7FD733>II E /Fp 5 85 864 600 dfs[61 71 123 198 104 65 D[<91B712F818FF19C00201903980003F F06E90C7EA0FF84AED03FCF000FE4B157FA2F13F800203EE1FC05DF10FE0A214074B16F0 1907A2140F5D1AF8A2141F5DA2190F143F5D1AF0A2147F4B151FA302FF17E092C9123FA3 4918C04A167F1A80A2010317FF4A1700A24E5A13074A4B5A611807010F5F4A4B5A181F61 011F4C5A4A4BC7FC18FE4D5A013F4B5A4A4A5A4D5A017FED3FC005FFC8FC4AEB03FE01FF EC1FF8B812E094C9FC16F8>69 68 122 195 107 68 D[<91B91280A302019026800007 13006E90C8FC4A163FA24B81A30203160E5DA314074B151E191CA2140F5D17075F021F02 0E90C7FC5DA2171E023F141C4B133CA2177C027F5CED800392B5FCA291B65AED00071601 A2496E5A5CA2160101035D5CA2160301075D4A90CAFCA3130F5CA3131F5CA3133F5CA213 7FA313FFB612E0A3>65 68 122 195 92 70 D[<91B77E18F818FE020190398001FF806E 90C7EA3FC04AED1FE0F00FF04BEC07F8180319FC14034B15FEA314075DA3020FED07FC5D A2F00FF8141F4B15F0F01FE0F03FC0023F16804BEC7F0018FEEF03F8027F4A5A4BEB1FC0 4CB4C7FC92B512F891B612E092380003F8EE00FE177F496F7E4A6E7EA28413034A140FA2 171F13075CA2173F130F5CA24D5A131F5CA3013F170E5CA2017FEE801E191C4A163C496C 1638B66C90383FC070051F13F094380FE1E0CA3803FF80943800FE00>63 70 122 195 102 82 D[<48B912F85AA2913B0007FC001FF0D807F84A130701E0010F14 0349160148485C90C71500A2001E021F15E05E121C123C0038143F4C1301007818C01270 00F0147F485DA3C800FF91C7FC93C9FCA35C5DA314035DA314075DA3140F5DA3141F5DA3 143F5DA3147F5DA314FF92CAFCA35B5CA21303A21307497E007FB612C0A25E>61 68 111 195 101 84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 600dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 1597 1821 a Fp(D)49 b(R)i(A)f(F)g(T)449 2004 y Fo(Do)t(cumen)l(t)44 b(for)f(a)g(Standard)h(Message-P)l(assing)f(In)l (terface)1242 2393 y Fn(Message)34 b(P)m(assing)f(In)m(terface)h(F)-8 b(orum)1666 2646 y(July)33 b(24,)f(1995)155 2762 y(This)h(w)m(ork)g(w)m (as)g(supp)s(orted)h(in)e(part)g(b)m(y)i(ARP)-8 b(A)32 b(and)h(NSF)g(under)g(gran)m(t)f(ASC-9310330,)g(the)384 2878 y(National)f(Science)i(F)-8 b(oundation)31 b(Science)j(and)e(T)-8 b(ec)m(hnology)34 b(Cen)m(ter)g(Co)s(op)s(erativ)m(e)152 2994 y(Agreemen)m(t)f(No.)43 b(CCR-8809615,)31 b(and)i(b)m(y)h(the)f (Commission)d(of)i(the)h(Europ)s(ean)g(Comm)m(unit)m(y)1308 3110 y(through)f(Esprit)h(pro)5 b(ject)33 b(P6643.)p eop %%Page: 1 2 1 1 bop 332 99 a Fm(This)38 b(is)g(the)i(result)f(of)h(a)g(LaT)-8 b(eX)40 b(run)f(of)g(a)i(draft)e(of)h(a)g(single)e(c)m(hapter)i(of)g (the)g(MPIF)g(Final)150 211 y(Rep)s(ort)30 b(do)s(cumen)m(t.)1937 5656 y(i)p eop %%Page: 1 3 1 2 bop 150 721 a Fl(Chapter)65 b(4)150 1136 y Fk(One)77 b(Sided)g(Comm)-6 b(unications)150 1618 y Fj(4.1)116 b(Prop)s(osal)38 b(fo)m(r)g(Single)f(Sided)i(Communication)c(b)m(y)j(P) m(aul)h(Pierce)150 1824 y Fi(4.1.1)97 b(Intro)s(duction)150 1996 y Fm(This)36 b(is)h(a)h(prop)s(osal)e(based)h(on)h(the)g(get/put)h (discussion)c(at)j(the)g(June)f(meeting.)63 b(It)37 b(includes)f(the) 150 2109 y(concept)31 b(of)g(op)s(ening)e(a)i(target)g(windo)m(w)e(b)s (efore)h(initiating)e(gets)k(or)e(puts.)377 2321 y Fh(R)-5 b(ationale.)81 b Fm(There)28 b(are)g(t)m(w)m(o)h(distinct)d(adv)-5 b(an)m(tages)29 b(in)e(sp)s(ecifying)e(a)k(target)g(area)f(for)g(gets)h (and)377 2434 y(puts.)67 b(First,)41 b(it)e(allo)m(ws)g(heterogenous)h (op)s(eration)f(b)m(y)g(sp)s(ecifying)e(enough)i(information)f(for)377 2547 y(automatic)h(address)d(and)h(datat)m(yp)s(e)h(translation)e(b)s (et)m(w)m(een)i(pro)s(cesses)g(running)c(on)j(di\013eren)m(t)377 2660 y(arc)m(hitectures.)51 b(Second,)35 b(it)e(giv)m(es)h(the)g (programmer)f(more)h(explicit)e(con)m(trol)i(of)g(the)g(memory)377 2773 y(sub)5 b(ject)28 b(to)h(remote)g(access.)41 b(This)27 b(mak)m(es)i(it)f(p)s(ossible)d(to)k(use)f(gets)h(and)f(puts)f(b)s(et)m (w)m(een)i(lo)s(osely)377 2886 y(coupled)42 b(pro)s(cesses)h(\(suc)m(h) f(as)h(clien)m(t/serv)m(er\))h(without)e(a)h(high)e(degree)j(of)f (trust.)77 b(\()p Fh(End)45 b(of)377 2999 y(r)-5 b(ationale.)p Fm(\))150 3242 y Fi(4.1.2)97 b(T)-8 b(a)m(rget)32 b(Windo)m(w)h (Instantiation)150 3603 y Fg(MPI)p 318 3603 28 4 v 33 w(T)-8 b(ARGET)29 b(\(tbuf,)i(tcount,)g(tt)m(yp)s(e,)f(comm,)i(tcomm\)) 233 3757 y Ff(IN)312 b Fg(tbuf)1018 b Ff(initial)28 b(address)f(of)g (target)g(windo)n(w)g(\(c)n(hoice\))233 3907 y(IN)312 b Fg(tcount)928 b Ff(n)n(um)n(b)r(er)37 b(of)g(elemen)n(ts)g(in)g (target)g(windo)n(w)f(\(non)h(negativ)n(e)1810 4020 y(in)n(teger\))233 4170 y(IN)312 b Fg(tt)m(yp)s(e)978 b Ff(data)28 b(t)n(yp)r(e)f(of)h (eac)n(h)f(target)f(windo)n(w)h(elemen)n(t)h(\(handle\))233 4321 y(IN)312 b Fg(comm)944 b Ff(comm)n(unicator)27 b(\(handle\))233 4471 y(OUT)217 b Fg(tcomm)911 b Ff(comm)n(unicator)27 b(with)h(target)e(windo)n(w)i(\(handle\))150 4720 y Fe(int)47 b(MPI)p 491 4720 29 4 v 33 w(Target\(void*)e(tbuf,)h(int)h(tcount,)f (MPI)p 2148 4720 V 34 w(Datatype)f(ttype,)h(MPI)p 3089 4720 V 34 w(Comm)g(comm,)786 4833 y(MPI)p 936 4833 V 34 w(Comm)h(tcomm\))332 5006 y Fm(De\014ne)36 b(a)g(target)h(windo)m(w) d(in)g(eac)m(h)j(pro)s(cess)e(in)f Fg(comm)j Fm(for)e(use)h(with)e (gets)j(and)e(puts.)55 b(This)34 b(is)150 5119 y(a)j(blo)s(c)m(king)f (collectiv)m(e)h(call)f(on)h Fg(comm)p Fm(.)61 b(If)37 b(an)m(y)g(participating)e(pro)s(cess)h(do)s(es)h(not)g(wish)e(to)i(op) s(en)g(a)150 5232 y(windo)m(w,)29 b(it)h(can)h(mak)m(e)g(this)e(call)h (with)f Fg(tcount)i Fm(equal)f(to)h(zero.)1927 5656 y(1)p eop %%Page: 2 4 2 3 bop 150 -200 a Fm(2)1493 b Fd(CHAPTER)30 b(4.)71 b(ONE)30 b(SIDED)h(COMMUNICA)-8 b(TIONS)332 99 y Fm(The)28 b(target)j(windo)m(w)c(is)h(de\014ned)g(b)m(y)g Fg(tbuf)p Fm(,)h Fg(tcount)g Fm(and)g Fg(tt)m(yp)s(e)p Fm(.)40 b(Gets)30 b(and)e(puts)g(ma)m(y)h(access)i(an)m(y)150 211 y(part)g(of)g(the)g(windo)m(w.)41 b(Eac)m(h)32 b(get)g(or)f(put)f (reads)h(or)g(writes)f(a)h(blo)s(c)m(k)f(of)h(one)h(or)f(more)g Fg(tt)m(yp)s(e)g Fm(elemen)m(ts)150 324 y(somewhere)f(in)f(the)i(windo) m(w.)332 437 y(The)39 b(target)h(windo)m(w)e(is)g(v)-5 b(alid)38 b(in)g(the)h(output)g(comm)m(unicator)h Fg(tcomm)g Fm(and)f(comm)m(unicators)150 550 y(deriv)m(ed)29 b(from)h(it.)377 756 y Fh(A)-5 b(dvic)g(e)26 b(to)g(implementors.)77 b Fm(Note)24 b(that)g(v)-5 b(arious)21 b(pro)s(cesses)i(in)e Fg(comm)j Fm(ma)m(y)f(sp)s(ecify)e(completely)377 869 y(di\013eren)m(t)33 b(target)i(windo)m(ws,)e(in)f(lo)s(cation,)i(size)g (and)f(datat)m(yp)s(e.)51 b(As)33 b(long)g(as)h(all)e(the)i(gets)h(and) 377 982 y(puts)42 b(to)i(a)f(particular)f(pro)s(cess)g(matc)m(h)i(its)e (sp)s(eci\014c)g(target)i(windo)m(w)e(this)g(should)e(p)s(ose)j(no)377 1095 y(problem.)c(\()p Fh(End)34 b(of)e(advic)-5 b(e)34 b(to)f(implementors.)p Fm(\))377 1302 y Fh(A)-5 b(dvic)g(e)36 b(to)g(users.)99 b Fm(If)33 b(y)m(ou)h(w)m(an)m(t)h(to)f(pro)m(vide)e (m)m(ultiple)f(views)i(of)h(the)f(same)h(target)h(windo)m(w,)377 1414 y(sa)m(y)c(with)d(di\013eren)m(t)i(datat)m(yp)s(es)g(or)g(la)m(y)m (out,)h(y)m(ou)g(can)f(call)f Fg(MPI)p 2598 1414 28 4 v 32 w(T)-8 b(ARGET)30 b Fm(m)m(ultiple)d(times)j(and)377 1527 y(use)j(the)g(resulting)e(comm)m(unicators)j(to)f(distinguish)d(b) s(et)m(w)m(een)j(the)h(di\013eren)m(t)e(views.)48 b(\()p Fh(End)36 b(of)377 1640 y(advic)-5 b(e)33 b(to)h(users.)p Fm(\))150 1883 y Fi(4.1.3)97 b(Blo)s(cking)32 b(Put)150 2243 y Fg(MPI)p 318 2243 V 33 w(PUT)d(\(buf,)i(count,)g(datat)m(yp)s (e,)f(dest,)h(ta)m(rget)p 1863 2243 V 33 w(index,)g(tcomm\))233 2398 y Ff(IN)312 b Fg(buf)1051 b Ff(initial)28 b(address)f(of)g(put)h (bu\013er)g(\(c)n(hoice\))233 2547 y(IN)312 b Fg(count)961 b Ff(n)n(um)n(b)r(er)32 b(of)f(elemen)n(ts)h(in)f(put)i(bu\013er)e (\(non)h(negativ)n(e)e(in)n(te-)1810 2660 y(ger\))233 2809 y(IN)312 b Fg(datat)m(yp)s(e)843 b Ff(data)28 b(t)n(yp)r(e)f(of)h (eac)n(h)f(put)h(bu\013er)g(elemen)n(t)f(\(handle\))233 2958 y(IN)312 b Fg(dest)1018 b Ff(rank)27 b(of)h(destination)f(pro)r (cess)f(in)i(tcomm)g(\(in)n(teger\))233 3108 y(IN)312 b Fg(ta)m(rget)p 865 3108 V 33 w(index)719 b Ff(index)40 b(in)n(to)f(target)g(bu\013er)h(in)f(destination)h(pro)r(cess)e(\(non) 1810 3220 y(negativ)n(e)27 b(in)n(teger\))233 3370 y(IN)312 b Fg(tcomm)911 b Ff(comm)n(unicator)27 b(\(handle\))150 3619 y Fe(int)47 b(MPI)p 491 3619 29 4 v 33 w(Put\(void*)f(buf,)g(int)h (count,)f(MPI)p 1909 3619 V 34 w(Datatype)f(datatype,)h(int)h(dest,)786 3732 y(int)g(target)p 1271 3732 V 33 w(index,)f(MPI)p 1782 3732 V 34 w(Comm)g(tcomm\))332 3904 y Fm(W)-8 b(rite)34 b(the)g(con)m(ten)m(ts)i(of)e Fg(buf)f Fm(to)i(a)f(lo)s(cation)f (within)f(the)i(target)h(windo)m(w)d(de\014ned)h(b)m(y)h Fg(dest)g Fm(and)150 4017 y Fg(tcomm)d Fm(at)h(o\013set)f Fg(ta)m(rget)p 1023 4017 28 4 v 33 w(index)p Fm(.)41 b(Blo)s(c)m(k)31 b(un)m(til)d(the)j(put)f(bu\013er)f(can)i(b)s(e)e (safely)i(written.)332 4130 y(If)e Fg(datat)m(yp)s(e)h Fm(and)f Fg(tt)m(yp)s(e)h Fm(are)g(the)h(same,)f(the)h(call)e (transfers)g(data)i(in)e(the)h(same)h(w)m(a)m(y)g(that)f(w)m(ould)150 4243 y(o)s(ccur)g(if)g(the)g(source)h(executed)g(a)g(call)e(to)377 4449 y Fe(MPI)p 527 4449 29 4 v 34 w(Send)p Fm(\()p Fe(buf)p Fc(;)15 b Fe(count)p Fc(;)g Fe(datatype)p Fc(;)g Fe(dest)p Fc(;)g(:::;)g Fe(tcomm)p Fm(\))150 4656 y(and)30 b(the)g(destination)g (executed)h(a)g(call)e(to)377 4862 y Fe(MPI)p 527 4862 V 34 w(Recv)p Fm(\()p Fe(tbuf)18 b Fm(+)i Fe(target)p 1377 4862 V 33 w(index)p Fc(;)15 b Fe(count)p Fc(;)g Fe(ttype)p Fc(;)g(:::;)g Fe(tcomm)p Fc(;)g(:::)p Fm(\))332 5068 y(where)31 b Fg(tbuf)g Fm(and)g Fg(tt)m(yp)s(e)h Fm(refer)f(to)i(the)e(parameters)h(in)f(the)h(call)f(to)h Fg(MPI)p 2854 5068 28 4 v 33 w(T)-8 b(ARGET)31 b Fm(that)h(created)150 5181 y Fg(tcomm)p Fm(.)42 b(It)30 b(returns)f(when)h(the)g(send)g(w)m (ould)f(return.)332 5294 y(The)42 b(t)m(yp)s(e)h(signature)f(implied)d (b)m(y)k Fg(count)g Fm(and)f Fg(datat)m(yp)s(e)g Fm(on)h(the)g(source)f (m)m(ust)h(b)s(e)f(equal)g(to)150 5407 y(the)g(t)m(yp)s(e)g(signature)g (resulting)e(from)i(an)g(appropriate)f(n)m(um)m(b)s(er)f(of)j (replications)d(of)i Fg(tt)m(yp)s(e)g Fm(at)h(the)3934 92 y Fb(1)3934 205 y(2)3934 318 y(3)3934 431 y(4)3934 544 y(5)3934 657 y(6)3934 770 y(7)3934 882 y(8)3934 995 y(9)3919 1108 y(10)3919 1221 y(11)3919 1334 y(12)3919 1447 y(13)3919 1560 y(14)3919 1673 y(15)3919 1786 y(16)3919 1899 y(17)3919 2012 y(18)3919 2124 y(19)3919 2237 y(20)3919 2350 y(21)3919 2463 y(22)3919 2576 y(23)3919 2689 y(24)3919 2802 y(25)3919 2915 y(26)3919 3028 y(27)3919 3141 y(28)3919 3254 y(29)3919 3366 y(30)3919 3479 y(31)3919 3592 y(32)3919 3705 y(33)3919 3818 y(34)3919 3931 y(35)3919 4044 y(36)3919 4157 y(37)3919 4270 y(38)3919 4383 y(39)3919 4496 y(40)3919 4609 y(41)3919 4721 y(42)3919 4834 y(43)3919 4947 y(44)3919 5060 y(45)3919 5173 y(46)3919 5286 y(47)3919 5399 y(48)p eop %%Page: 3 5 3 4 bop 150 -200 a Fd(4.1.)72 b(PR)m(OPOSAL)29 b(F)m(OR)i(SINGLE)f (SIDED)g(COMMUNICA)-8 b(TION)31 b(BY)g(P)-8 b(A)m(UL)31 b(PIER)m(CE)243 b Fm(3)150 99 y(destination.)40 b(The)30 b(put)f(data)j(m)m(ust)e(\014t)g(within)e(the)j(sp)s(eci\014ed)d(area)k (in)d(the)h(target)i(windo)m(w)d(or)i(su\013er)150 211 y(the)h(same)f(consequences)h(as)g(sending)d(to)s(o)j(large)g(a)g (message)g(or)f(receiving)g(in)m(to)g(to)s(o)h(small)e(a)i(bu\013er.) 150 324 y(In)h(particular,)g(if)f Fg(datat)m(yp)s(e)h Fm(is)g(the)g(same)h(as)g Fg(tt)m(yp)s(e)g Fm(then)f Fg(ta)m(rget)p 2432 324 28 4 v 33 w(index)h Fm(+)f Fg(count)h Fm(m)m(ust)f(b)s(e)g(less)g(than)150 437 y(or)d(equal)g(to)h Fg(tcount)p Fm(.)332 715 y Fa(Discussion:)110 b Ff(Note)38 b(that)f(the)h(argumen)n(ts)e(are)g(mostly)h(similar)f(to)h(and)h(in)f (the)h(same)e(order)g(as)h(in)150 828 y(MPI)p 318 828 25 4 v 29 w(SEND.)150 1237 y Fi(4.1.4)97 b(Blo)s(cking)32 b(Get)150 1598 y Fg(MPI)p 318 1598 28 4 v 33 w(GET)e(\(buf,)g(count,)h (datat)m(yp)s(e,)f(dest,)h(ta)m(rget)p 1857 1598 V 33 w(index,)g(tcomm\))233 1752 y Ff(IN)312 b Fg(buf)1051 b Ff(initial)28 b(address)f(of)g(get)h(bu\013er)f(\(c)n(hoice\))233 1902 y(IN)312 b Fg(count)961 b Ff(n)n(um)n(b)r(er)19 b(of)g(elemen)n(ts)f(in)i(get)e(bu\013er)h(\(non)g(negativ)n(e)f(in)n (teger\))233 2053 y(IN)312 b Fg(datat)m(yp)s(e)843 b Ff(data)28 b(t)n(yp)r(e)f(of)h(eac)n(h)f(get)g(bu\013er)h(elemen)n(t)g (\(handle\))233 2203 y(IN)312 b Fg(dest)1018 b Ff(rank)27 b(of)h(destination)f(pro)r(cess)f(in)i(tcomm)g(\(in)n(teger\))233 2353 y(IN)312 b Fg(ta)m(rget)p 865 2353 V 33 w(index)719 b Ff(index)40 b(in)n(to)f(target)g(bu\013er)h(in)f(destination)h(pro)r (cess)e(\(non)1810 2466 y(negativ)n(e)27 b(in)n(teger\))233 2616 y(IN)312 b Fg(tcomm)911 b Ff(comm)n(unicator)27 b(\(handle\))150 2865 y Fe(int)47 b(MPI)p 491 2865 29 4 v 33 w(Get\(void*)f(buf,)g(int)h(count,)f(MPI)p 1909 2865 V 34 w(Datatype)f(datatype,)h(int)h(dest,)786 2978 y(int)g(target)p 1271 2978 V 33 w(index,)f(MPI)p 1782 2978 V 34 w(Comm)g(tcomm\))332 3151 y Fm(Read)33 b(the)h(con)m(ten)m (ts)h(of)e(a)h(lo)s(cation)f(within)e(the)i(target)i(windo)m(w)d (de\014ned)g(b)m(y)h Fg(dest)g Fm(and)g Fg(tcomm)150 3264 y Fm(at)e(o\013set)h Fg(ta)m(rget)p 730 3264 28 4 v 33 w(index)e Fm(and)g(return)f(the)i(data)g(in)e Fg(buf)p Fm(.)40 b(Blo)s(c)m(k)31 b(un)m(til)e(the)h(data)h(is)f (presen)m(t.)332 3377 y(This)h(is)g(the)i(rev)m(erse)g(of)g Fg(MPI)p 1374 3377 V 32 w(PUT)f Fm(and)g(all)g(the)h(argumen)m(ts)f (and)g(rules)f(are)i(the)g(same,)h(except)150 3490 y(that)d(data)g(go)s (es)g(in)e(the)i(opp)s(osite)e(direction.)-64 92 y Fb(1)-64 205 y(2)-64 318 y(3)-64 431 y(4)-64 544 y(5)-64 657 y(6)-64 770 y(7)-64 882 y(8)-64 995 y(9)-80 1108 y(10)-80 1221 y(11)-80 1334 y(12)-80 1447 y(13)-80 1560 y(14)-80 1673 y(15)-80 1786 y(16)-80 1899 y(17)-80 2012 y(18)-80 2124 y(19)-80 2237 y(20)-80 2350 y(21)-80 2463 y(22)-80 2576 y(23)-80 2689 y(24)-80 2802 y(25)-80 2915 y(26)-80 3028 y(27)-80 3141 y(28)-80 3254 y(29)-80 3366 y(30)-80 3479 y(31)-80 3592 y(32)-80 3705 y(33)-80 3818 y(34)-80 3931 y(35)-80 4044 y(36)-80 4157 y(37)-80 4270 y(38)-80 4383 y(39)-80 4496 y(40)-80 4609 y(41)-80 4721 y(42)-80 4834 y(43)-80 4947 y(44)-80 5060 y(45)-80 5173 y(46)-80 5286 y(47)-80 5399 y(48)p eop %%Page: 4 6 4 5 bop 150 -200 a Fm(4)1493 b Fd(CHAPTER)30 b(4.)71 b(ONE)30 b(SIDED)h(COMMUNICA)-8 b(TIONS)150 99 y Fi(4.1.5)97 b(Synchronous)33 b(Put)150 459 y Fg(MPI)p 318 459 28 4 v 33 w(SPUT)c(\(buf,)h(count,)i(datat)m(yp)s(e,)e(dest,)g(ta)m(rget)p 1913 459 V 34 w(index,)g(tcomm\))233 614 y Ff(IN)312 b Fg(buf)1051 b Ff(initial)28 b(address)f(of)g(put)h(bu\013er)g(\(c)n (hoice\))233 764 y(IN)312 b Fg(count)961 b Ff(n)n(um)n(b)r(er)32 b(of)f(elemen)n(ts)h(in)f(put)i(bu\013er)e(\(non)h(negativ)n(e)e(in)n (te-)1810 877 y(ger\))233 1027 y(IN)312 b Fg(datat)m(yp)s(e)843 b Ff(data)28 b(t)n(yp)r(e)f(of)h(eac)n(h)f(put)h(bu\013er)g(elemen)n(t) f(\(handle\))233 1177 y(IN)312 b Fg(dest)1018 b Ff(rank)27 b(of)h(destination)f(pro)r(cess)f(in)i(tcomm)g(\(in)n(teger\))233 1328 y(IN)312 b Fg(ta)m(rget)p 865 1328 V 33 w(index)719 b Ff(index)40 b(in)n(to)f(target)g(bu\013er)h(in)f(destination)h(pro)r (cess)e(\(non)1810 1440 y(negativ)n(e)27 b(in)n(teger\))233 1591 y(IN)312 b Fg(tcomm)911 b Ff(comm)n(unicator)27 b(\(handle\))150 1840 y Fe(int)47 b(MPI)p 491 1840 29 4 v 33 w(Iput\(void*)e(buf,)i(int)g(count,)f(MPI)p 1957 1840 V 34 w(Datatype)f(datatype,)g(int)i(dest,)786 1953 y(int)g(target)p 1271 1953 V 33 w(index,)f(MPI)p 1782 1953 V 34 w(Comm)g(tcomm\))332 2125 y Fm(W)-8 b(rite)34 b(the)g(con)m(ten)m(ts)i(of)e Fg(buf)f Fm(to)i(a)f(lo)s(cation)f (within)f(the)i(target)h(windo)m(w)d(de\014ned)h(b)m(y)h Fg(dest)g Fm(and)150 2238 y Fg(tcomm)d Fm(at)h(o\013set)f Fg(ta)m(rget)p 1023 2238 28 4 v 33 w(index)p Fm(.)332 2351 y(Blo)s(c)m(k)26 b(un)m(til)e(the)h(data)i(has)e(b)s(een)g (written)f(at)j(the)e(destination.)38 b(Other)25 b(than)h(blo)s(c)m (king)e(b)s(eha)m(vior)150 2464 y(this)29 b(call)h(is)f(iden)m(tical)h (to)h Fg(MPI)p 1231 2464 V 32 w(PUT)p Fm(.)150 2708 y Fi(4.1.6)97 b(Nonblo)s(cking)33 b(Put)150 3068 y Fg(MPI)p 318 3068 V 33 w(IPUT)d(\(buf,)g(count,)h(datat)m(yp)s(e,)f(dest,)h(ta)m (rget)p 1888 3068 V 33 w(index,)g(tcomm,)h(request\))233 3223 y Ff(IN)312 b Fg(buf)1051 b Ff(initial)28 b(address)f(of)g(put)h (bu\013er)g(\(c)n(hoice\))233 3373 y(IN)312 b Fg(count)961 b Ff(n)n(um)n(b)r(er)32 b(of)f(elemen)n(ts)h(in)f(put)i(bu\013er)e (\(non)h(negativ)n(e)e(in)n(te-)1810 3486 y(ger\))233 3636 y(IN)312 b Fg(datat)m(yp)s(e)843 b Ff(data)28 b(t)n(yp)r(e)f(of)h (eac)n(h)f(put)h(bu\013er)g(elemen)n(t)f(\(handle\))233 3786 y(IN)312 b Fg(dest)1018 b Ff(rank)27 b(of)h(destination)f(pro)r (cess)f(in)i(tcomm)g(\(in)n(teger\))233 3937 y(IN)312 b Fg(ta)m(rget)p 865 3937 V 33 w(index)719 b Ff(index)40 b(in)n(to)f(target)g(bu\013er)h(in)f(destination)h(pro)r(cess)e(\(non) 1810 4050 y(negativ)n(e)27 b(in)n(teger\))233 4200 y(IN)312 b Fg(tcomm)911 b Ff(comm)n(unicator)27 b(\(handle\))233 4350 y(OUT)217 b Fg(request)900 b Ff(comm)n(unication)27 b(request)g(\(handle\))150 4599 y Fe(int)47 b(MPI)p 491 4599 29 4 v 33 w(Iput\(void*)e(buf,)i(int)g(count,)f(MPI)p 1957 4599 V 34 w(Datatype)f(datatype,)g(int)i(dest,)786 4712 y(int)g(target)p 1271 4712 V 33 w(index,)f(MPI)p 1782 4712 V 34 w(Comm)g(tcomm,)g(MPI)p 2532 4712 V 34 w(Request)g(request\))332 4885 y Fm(Initiate)34 b(a)g(write)g(of)h(the) f(con)m(ten)m(ts)i(of)f Fg(buf)f Fm(to)h(a)g(lo)s(cation)f(within)e (the)i(target)i(windo)m(w)d(de\014ned)150 4998 y(b)m(y)d Fg(dest)h Fm(and)e Fg(tcomm)j Fm(at)f(o\013set)g Fg(ta)m(rget)p 1511 4998 28 4 v 34 w(index)p Fm(.)332 5111 y(The)37 b(request)g(completes)h(when)e(the)h(put)g(bu\013er)f(can)i(b)s(e)e (safely)h(written.)61 b(The)37 b(normal)f(MPI)150 5224 y(w)m(ait)27 b(and)f(test)h(calls)f(can)h(b)s(e)f(used)g(with)f Fg(request)i Fm(to)h(\014nd)d(out)i(when)e(it)i(completes.)39 b(The)27 b(status)f(\014elds)150 5336 y(are)31 b(\014lled)d(in)h(as)i (b)m(y)f Fg(MPI)p 1036 5336 V 33 w(ISEND)p Fm(,)g(except)i(that)f(the)f Fg(MPI)p 2186 5336 V 33 w(T)-8 b(A)m(G)30 b Fm(\014eld)f(will)f(b)s(e)h Fg(MPI)p 3077 5336 V 33 w(UNDEFINED)p Fm(.)3934 92 y Fb(1)3934 205 y(2)3934 318 y(3)3934 431 y(4)3934 544 y(5)3934 657 y(6)3934 770 y(7)3934 882 y(8)3934 995 y(9)3919 1108 y(10)3919 1221 y(11)3919 1334 y(12)3919 1447 y(13)3919 1560 y(14)3919 1673 y(15)3919 1786 y(16)3919 1899 y(17)3919 2012 y(18)3919 2124 y(19)3919 2237 y(20)3919 2350 y(21)3919 2463 y(22)3919 2576 y(23)3919 2689 y(24)3919 2802 y(25)3919 2915 y(26)3919 3028 y(27)3919 3141 y(28)3919 3254 y(29)3919 3366 y(30)3919 3479 y(31)3919 3592 y(32)3919 3705 y(33)3919 3818 y(34)3919 3931 y(35)3919 4044 y(36)3919 4157 y(37)3919 4270 y(38)3919 4383 y(39)3919 4496 y(40)3919 4609 y(41)3919 4721 y(42)3919 4834 y(43)3919 4947 y(44)3919 5060 y(45)3919 5173 y(46)3919 5286 y(47)3919 5399 y(48)p eop %%Page: 5 7 5 6 bop 150 -200 a Fd(4.1.)72 b(PR)m(OPOSAL)29 b(F)m(OR)i(SINGLE)f (SIDED)g(COMMUNICA)-8 b(TION)31 b(BY)g(P)-8 b(A)m(UL)31 b(PIER)m(CE)243 b Fm(5)150 99 y Fi(4.1.7)97 b(Synchronous)33 b(Nonblo)s(cking)g(Put)150 459 y Fg(MPI)p 318 459 28 4 v 33 w(ISPUT)c(\(buf,)i(count,)g(datat)m(yp)s(e,)f(dest,)g(ta)m(rget) p 1938 459 V 34 w(index,)g(tcomm,)i(request\))233 614 y Ff(IN)312 b Fg(buf)1051 b Ff(initial)28 b(address)f(of)g(put)h (bu\013er)g(\(c)n(hoice\))233 764 y(IN)312 b Fg(count)961 b Ff(n)n(um)n(b)r(er)32 b(of)f(elemen)n(ts)h(in)f(put)i(bu\013er)e (\(non)h(negativ)n(e)e(in)n(te-)1810 877 y(ger\))233 1027 y(IN)312 b Fg(datat)m(yp)s(e)843 b Ff(data)28 b(t)n(yp)r(e)f(of)h (eac)n(h)f(put)h(bu\013er)g(elemen)n(t)f(\(handle\))233 1177 y(IN)312 b Fg(dest)1018 b Ff(rank)27 b(of)h(destination)f(pro)r (cess)f(in)i(tcomm)g(\(in)n(teger\))233 1328 y(IN)312 b Fg(ta)m(rget)p 865 1328 V 33 w(index)719 b Ff(index)40 b(in)n(to)f(target)g(bu\013er)h(in)f(destination)h(pro)r(cess)e(\(non) 1810 1440 y(negativ)n(e)27 b(in)n(teger\))233 1591 y(IN)312 b Fg(tcomm)911 b Ff(comm)n(unicator)27 b(\(handle\))233 1741 y(OUT)217 b Fg(request)900 b Ff(comm)n(unication)27 b(request)g(\(handle\))150 1990 y Fe(int)47 b(MPI)p 491 1990 29 4 v 33 w(Iput\(void*)e(buf,)i(int)g(count,)f(MPI)p 1957 1990 V 34 w(Datatype)f(datatype,)g(int)i(dest,)786 2103 y(int)g(target)p 1271 2103 V 33 w(index,)f(MPI)p 1782 2103 V 34 w(Comm)g(tcomm,)g(MPI)p 2532 2103 V 34 w(Request)g(request\))332 2276 y Fm(Initiate)34 b(a)g(write)g(of)h(the) f(con)m(ten)m(ts)i(of)f Fg(buf)f Fm(to)h(a)g(lo)s(cation)f(within)e (the)i(target)i(windo)m(w)d(de\014ned)150 2389 y(b)m(y)d Fg(dest)h Fm(and)e Fg(tcomm)j Fm(at)f(o\013set)g Fg(ta)m(rget)p 1511 2389 28 4 v 34 w(index)p Fm(.)332 2502 y(The)d(request)h (completes)g(when)f(the)h(data)h(has)e(b)s(een)g(written)g(at)i(the)f (destination.)39 b(Other)28 b(than)150 2614 y(completion)i(b)s(eha)m (vior)f(this)g(call)h(is)f(iden)m(tical)h(to)h Fg(MPI)p 2067 2614 V 32 w(IPUT)p Fm(.)150 2858 y Fi(4.1.8)97 b(Nonblo)s(cking)33 b(Get)150 3218 y Fg(MPI)p 318 3218 V 33 w(IGET)d(\(buf,)h(count,)g (datat)m(yp)s(e,)f(dest,)g(ta)m(rget)p 1882 3218 V 34 w(index,)g(tcomm,)i(request\))233 3373 y Ff(IN)312 b Fg(buf)1051 b Ff(initial)28 b(address)f(of)g(get)h(bu\013er)f(\(c)n (hoice\))233 3523 y(IN)312 b Fg(count)961 b Ff(n)n(um)n(b)r(er)19 b(of)g(elemen)n(ts)f(in)i(get)e(bu\013er)h(\(non)g(negativ)n(e)f(in)n (teger\))233 3673 y(IN)312 b Fg(datat)m(yp)s(e)843 b Ff(data)28 b(t)n(yp)r(e)f(of)h(eac)n(h)f(get)g(bu\013er)h(elemen)n(t)g (\(handle\))233 3824 y(IN)312 b Fg(dest)1018 b Ff(rank)27 b(of)h(destination)f(pro)r(cess)f(in)i(tcomm)g(\(in)n(teger\))233 3974 y(IN)312 b Fg(ta)m(rget)p 865 3974 V 33 w(index)719 b Ff(index)40 b(in)n(to)f(target)g(bu\013er)h(in)f(destination)h(pro)r (cess)e(\(non)1810 4087 y(negativ)n(e)27 b(in)n(teger\))233 4237 y(IN)312 b Fg(tcomm)911 b Ff(comm)n(unicator)27 b(\(handle\))233 4387 y(OUT)217 b Fg(request)900 b Ff(comm)n(unication) 27 b(request)g(\(handle\))150 4636 y Fe(int)47 b(MPI)p 491 4636 29 4 v 33 w(Iget\(void*)e(buf,)i(int)g(count,)f(MPI)p 1957 4636 V 34 w(Datatype)f(datatype,)g(int)i(dest,)786 4749 y(int)g(target)p 1271 4749 V 33 w(index,)f(MPI)p 1782 4749 V 34 w(Comm)g(tcomm,)g(MPI)p 2532 4749 V 34 w(Request)g(request\))332 4922 y Fm(Initiate)32 b(a)i(read)f(of)g(the)h (con)m(ten)m(ts)h(of)e(a)h(lo)s(cation)e(within)f(the)j(target)g(windo) m(w)e(de\014ned)g(b)m(y)h Fg(dest)150 5035 y Fm(and)d Fg(tcomm)h Fm(at)g(o\013set)h Fg(ta)m(rget)p 1200 5035 28 4 v 33 w(index)e Fm(and)g(return)g(the)g(data)h(in)e Fg(buf)p Fm(.)332 5148 y(This)g(is)h(the)h(rev)m(erse)h(of)f Fg(MPI)p 1366 5148 V 32 w(IPUT)g Fm(and)f(all)g(the)h(argumen)m(ts)g (and)f(rules)g(are)h(the)g(same,)h(except)150 5261 y(that)27 b(data)g(go)s(es)g(in)e(the)h(opp)s(osite)f(direction)g(and)h(the)g (status)h(ob)5 b(ject)27 b(\(except)h(for)e Fg(MPI)p 3209 5261 V 32 w(T)-8 b(A)m(G)p Fm(\))27 b(is)e(\014lled)150 5374 y(in)k(as)i(b)m(y)f Fg(MPI)p 662 5374 V 33 w(RECV)p Fm(.)-64 92 y Fb(1)-64 205 y(2)-64 318 y(3)-64 431 y(4)-64 544 y(5)-64 657 y(6)-64 770 y(7)-64 882 y(8)-64 995 y(9)-80 1108 y(10)-80 1221 y(11)-80 1334 y(12)-80 1447 y(13)-80 1560 y(14)-80 1673 y(15)-80 1786 y(16)-80 1899 y(17)-80 2012 y(18)-80 2124 y(19)-80 2237 y(20)-80 2350 y(21)-80 2463 y(22)-80 2576 y(23)-80 2689 y(24)-80 2802 y(25)-80 2915 y(26)-80 3028 y(27)-80 3141 y(28)-80 3254 y(29)-80 3366 y(30)-80 3479 y(31)-80 3592 y(32)-80 3705 y(33)-80 3818 y(34)-80 3931 y(35)-80 4044 y(36)-80 4157 y(37)-80 4270 y(38)-80 4383 y(39)-80 4496 y(40)-80 4609 y(41)-80 4721 y(42)-80 4834 y(43)-80 4947 y(44)-80 5060 y(45)-80 5173 y(46)-80 5286 y(47)-80 5399 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Jul 24 23:13:40 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA07432; Mon, 24 Jul 1995 23:13:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA02512 for mpi-core-out; Mon, 24 Jul 1995 22:21:57 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA02475; Mon, 24 Jul 1995 22:20:47 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id UAA28773; Mon, 24 Jul 1995 20:20:46 -0700 Date: Mon, 24 Jul 1995 20:20:46 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199507250320.UAA28773@win233.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: dynamic processes chapter Cc: mpi-dynamic@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is a new dynamic processes chapter. Changes since the last one are: 1. integration of the changes I suggested last week (with Jim's comments flagged for discussion) 2. several minor inconsistencies corrected 3. examples brought up to date 4. an attempt at improving the client/server section that brought up some issues for discussion but didn't resolve much Bill %!PS-Adobe-2.0 %%Creator: dvips, version 5.4 (C) 1986-90 Radical Eye Software %%Title: temp.dvi %%Pages: 32 1 %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 200 dict def TeXDict begin /N /def load def /B{bind def}N /S /exch load def /X{S N}B /TR /translate load N /isls false N /vsize 10 N /@rigin{ isls{[0 1 -1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale Resolution VResolution vsize neg mul TR}B /@letter{/vsize 10 N}B /@landscape{ /isls true N /vsize -1 N}B /@a4{/vsize 10.6929133858 N}B /@a3{/vsize 15.5531 N }B /@ledger{/vsize 16 N}B /@legal{/vsize 13 N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0 ]N /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0 ]N df-tail}B /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[ }B /E{pop nn dup definefont setfont}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ctr 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image}imagemask restore}B /D{ /cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto}B /eop{clear SI restore showpage userdict /eop-hook known{eop-hook}if}B /@start{userdict /start-hook known{start-hook}if /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for}B /p /show load N /RMat[1 0 0 -1 0 0]N /BDot 8 string N /v{/ruley X /rulex X V}B /V{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}B /a{moveto}B /delta 0 N /tail{ dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M}B /d{ -3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{4 M}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t {p 4 w}B /w{0 rmoveto}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{clear SS restore}B end %%EndProcSet %%BeginProcSet: special.pro TeXDict begin /SDict 200 dict N SDict begin /@SpecialDefaults{/hs 612 N /vs 792 N /ho 0 N /vo 0 N /hsc 1 N /vsc 1 N /ang 0 N /CLIP false N /BBcalc false N /p 3 def}B /@scaleunit 100 N /@hscale{@scaleunit div /hsc X}B /@vscale{ @scaleunit div /vsc X}B /@hsize{/hs X /CLIP true N}B /@vsize{/vs X /CLIP true N}B /@hoffset{/ho X}B /@voffset{/vo X}B /@angle{/ang X}B /@rwi{10 div /rwi X} B /@llx{/llx X}B /@lly{/lly X}B /@urx{/urx X}B /@ury{/ury X /BBcalc true N}B /magscale true def end /@MacSetUp{userdict /md known{userdict /md get type /dicttype eq{md begin /letter{}N /note{}N /legal{}N /od{txpose 1 0 mtx defaultmatrix dtransform S atan/pa X newpath clippath mark{transform{ itransform moveto}}{transform{itransform lineto}}{6 -2 roll transform 6 -2 roll transform 6 -2 roll transform{itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll curveto}}{{closepath}}pathforall newpath counttomark array astore /gc xdf pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{ PaintBlack}if}N /txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 -1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop 90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr 2 get ppr 0 get neg sub neg 0 S TR}if} ifelse scaleby96{ppr aload pop 4 -1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S TR}if}N /cp{pop pop showpage pm restore}N end}if} if}N /normalscale{Resolution 72 div VResolution 72 div neg scale magscale{ DVImag dup scale}if}N /psfts{S 65536 div N}N /startTexFig{/psf$SavedState save N userdict maxlength dict begin /magscale false def normalscale currentpoint TR /psf$ury psfts /psf$urx psfts /psf$lly psfts /psf$llx psfts /psf$y psfts /psf$x psfts currentpoint /psf$cy X /psf$cx X /psf$sx psf$x psf$urx psf$llx sub div N /psf$sy psf$y psf$ury psf$lly sub div N psf$sx psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub TR /showpage{}N /erasepage{}N /copypage{}N @MacSetUp}N /doclip{psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto closepath clip newpath moveto}N /endTexFig{end psf$SavedState restore}N /@beginspecial{SDict begin /SpecialSave save N gsave normalscale currentpoint TR @SpecialDefaults}B /@setspecial{CLIP{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs neg 0 rlineto closepath clip}{initclip}ifelse ho vo TR hsc vsc scale ang rotate BBcalc{rwi urx llx sub div dup scale llx neg lly neg TR}if /showpage{}N /erasepage{}N /copypage{}N newpath}B /@endspecial{grestore clear SpecialSave restore end}B /@defspecial{SDict begin}B /@fedspecial{end}B /li{lineto}B /rl{rlineto}B /rc{rcurveto}B /np{/SaveX currentpoint /SaveY X N 1 setlinecap newpath}B /st{stroke SaveX SaveY moveto}B /fil{fill SaveX SaveY moveto}B /ellipse{/endangle X /startangle X /yrad X /xrad X /savematrix matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix}B end %%EndProcSet TeXDict begin 1000 300 300 @start /Fa 1 111 df<1E07802318C023A06043C070438070 4380708700E00700E00700E00700E00E01C00E01C00E01C00E03821C03841C07041C07081C0308 3803101801E017147E931B>110 D E /Fb 7 112 df45 D<03CC063C0C3C181C3838303870387038E070E070E070E070E0E2C0E2C0E261E462643C380F12 7B9115>97 D<01F007080C08181C3838300070007000E000E000E000E000E000E008E010602030 C01F000E127B9113>99 D<1F800380038007000700070007000E000E000E000E001C001C001C00 1C0038003800380038007000700070007000E400E400E400E40068003800091D7C9C0B>108 D<3C1E0780266318C04683A0E04703C0E08E0380E08E0380E00E0380E00E0380E01C0701C01C07 01C01C0701C01C070380380E0388380E0388380E0708380E0710701C0320300C01C01D127C9122 >I<3C3C002646004687004707008E07008E07000E07000E07001C0E001C0E001C0E001C1C0038 1C40381C40383840383880701900300E0012127C9117>I<01E007180C0C180C380C300E700E70 0EE01CE01CE01CE018E038E030E06060C031801E000F127B9115>I E /Fc 1 16 df<07E01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF807E010107E 9115>15 D E /Fd 18 119 df<70F8F8F8700505788416>46 D<1FE0007FF8007FFC00783C0030 1E00000E00000E0003FE001FFE007E0E00F00E00E00E00E00E00F01E00F83E007FFFE03FE7E00F 83E013127E9116>97 D<03F80FFE1FFE3C1E780C7000F000E000E000E000E000F000700778073E 0F1FFE0FFC03F010127D9116>99 D<003F00007F00003F0000070000070000070000070003C700 0FF7003FFF003C1F00780F00F00700F00700E00700E00700E00700E00700F00700F00F00781F00 7C3F003FFFE01FF7F007C7E014197F9816>I<03E00FF81FFC3C1E780E7007E007FFFFFFFFFFFF E000E000700778073C0F1FFE0FFC03F010127D9116>I<001F00007F8000FF8001E78001C30001 C00001C0007FFF00FFFF00FFFF0001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C0003FFE007FFF003FFE0011197F9816>I<03E3C00FFFE01FFFE01E3CC0 3C1E00380E00380E00380E003C1E001E3C001FFC001FF8003BE0003800003800001FFC001FFF00 3FFFC07803C0F001E0E000E0E000E0E000E0F001E07C07C03FFF800FFE0003F800131C7F9116> I<7E0000FE00007E00000E00000E00000E00000E00000E3C000EFF000FFF800F87800F03800F03 800E03800E03800E03800E03800E03800E03800E03800E03800E03807FC7F0FFE7F87FC7F01519 809816>I<018003C003C0018000000000000000007FC07FC07FC001C001C001C001C001C001C0 01C001C001C001C001C001C07FFFFFFF7FFF101A7D9916>I109 D<7E3C00FEFF007FFF800F87800F03800F03800E03800E03800E03800E03 800E03800E03800E03800E03800E03807FC7F0FFE7F87FC7F01512809116>I<03E0000FF8001F FC003C1E00780F00700700E00380E00380E00380E00380E00380F00780700700780F003C1E001F FC000FF80003E00011127E9116>I<7E3E00FEFF807FFFC00F83E00F01E00E00F00E00F00E0070 0E00700E00700E00700E00F00F00E00F01E00F83C00FFFC00EFF000E3C000E00000E00000E0000 0E00000E00000E00007FC000FFE0007FC000141B809116>I114 D<0FEC3FFC7FFCF03CE01CE01CF0007F801FF007FC003EE00EE00EF00EF8 1EFFFCFFF8C7E00F127D9116>I<0300000700000700000700000700007FFF00FFFF00FFFF0007 000007000007000007000007000007000007000007010007038007038007078007878003FF0003 FE0000F80011177F9616>I<7E1F80FE3F807E1F800E03800E03800E03800E03800E03800E0380 0E03800E03800E03800E03800E07800F0F800FFFF007FFF803E3F01512809116>I<7F1FC0FF1F E07F1FC01C07001E0F000E0E000E0E000E0E00071C00071C00071C00071C0003B80003B80003B8 0001F00001F00000E00013127F9116>I E /Fe 35 122 df<001C0000003E0000003E0000002E 0000006700000067000000E7800000C7800000C3800001C3C0000183C0000181C0000381E00003 81E0000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C00 38001E0038001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 D<003FC000FFF003C0F00780300F00001E00003C00003C0000780000780000780000F00000F000 00F00000F00000F00000F00000F00000F00000F000007800007800007800003C00003C00001E00 000F000807801803C07800FFF0003F80151F7D9D1B>67 D III<003F8001FFF0 03C0F80780380F00181E00003C00003C0000780000780000780000F00000F00000F00000F00000 F00000F00000F007F8F007F8F000387800387800387800383C00383C00381E00380F0038078038 03C0F801FFF0003F80151F7D9D1C>III76 DII< 003F000001FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F0078000780780007 8078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F800 07C078000780780007807C000F803C000F003E001F001F003E000F807C0007C0F80003FFF00001 FFE000003F00001A1F7E9D1F>II82 D<03F8000FFE001C0F00380700700300600000E00000E00000E00000E00000F0 00007800007F00003FE0001FFC0007FE0001FF00001F800007800003C00003C00001C00001C000 01C00001C0C00180E00380F007007C0E001FFC0007F000121F7E9D17>IIIII89 D<0FC03FF07FF87038401C001C001C 00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC3F1C0E127E9114>97 D<07E00FF81FFC3C1C70 047000E000E000E000E000E000E000700070043C1C1FFC0FF807E00E127E9112>99 D<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C3FFC1FF807E00E12 7E9112>101 D<00FC01FC03FC07000E000E000E000E000E000E000E00FFE0FFE00E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E1D809C0D>I105 D109 DI<03F0000FFC001FFE003C0F00780780700380E001C0E001C0E001C0E001C0E0 01C0F003C07003807807803C0F001FFE000FFC0003F00012127F9115>II<078E1FEE3FFE7C3E781E700EE00EE00EE00EE00EE00EE00EF00E701E 7C3E3FFE1FEE0F8E000E000E000E000E000E000E000E000E0F1A7E9115>II<1FC03FF0 7FF0F030E000E000F0007F003FC01FE000F0003800388038F078FFF07FE01FC00D127F9110>I< 1C001C001C001C001C001C00FFE0FFE01C001C001C001C001C001C001C001C001C001C001C001C 001C201FF00FF007C00C187F970F>II 121 D E /Ff 67 124 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C000 0E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01 C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C00E01800E0000 0E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F8151D809C17>I<007FC001C1 C00303C00703C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFFFC00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C07FCFF8151D809C17>I<003F07E00001C09C18000380F018000701F03C000E01E03C000E00E0 18000E00E000000E00E000000E00E000000E00E000000E00E00000FFFFFFFC000E00E01C000E00 E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E 00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C007FC7FCFF80 211D809C23>I<60F0F0F0F0F0F0F060606060606060606060606060000000000060F0F060041E 7C9D0C>33 D<6060F0F0F8F86868080808080808101010102020404080800D0C7F9C15>I<60F0 F8680808081010204080050C7C9C0C>39 D<004000800100020006000C000C0018001800300030 007000600060006000E000E000E000E000E000E000E000E000E000E000E000E000600060006000 700030003000180018000C000C00060002000100008000400A2A7D9E10>I<8000400020001000 18000C000C000600060003000300038001800180018001C001C001C001C001C001C001C001C001 C001C001C001C0018001800180038003000300060006000C000C00180010002000400080000A2A 7E9E10>I<01800180018001804182F18F399C0FF003C003C00FF0399CF18F4182018001800180 018010127E9E15>I<60F0F0701010101020204080040C7C830C>44 DI< 60F0F06004047C830C>I<00030003000700060006000E000C001C001800180038003000300070 0060006000E000C000C001C001800380030003000700060006000E000C000C001C001800180038 003000700060006000E000C000C00010297E9E15>I<030007003F00C700070007000700070007 00070007000700070007000700070007000700070007000700070007000700070007000F80FFF8 0D1C7C9B15>49 D<07C01830201C400C400EF00FF80FF807F8077007000F000E000E001C001C00 380070006000C00180030006010C01180110023FFE7FFEFFFE101C7E9B15>I<07E01830201C20 1C781E780E781E381E001C001C00180030006007E00030001C001C000E000F000F700FF80FF80F F80FF00E401C201C183007E0101D7E9B15>I<60F0F0600000000000000000000060F0F0600412 7C910C>58 D<0FE03038401CE00EF00EF00EF00E000C001C0030006000C0008001800100010001 000100010001000000000000000000000003000780078003000F1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F00000017800000178000001780000023 C0000023C0000023C0000041E0000041E0000041E0000080F0000080F0000180F8000100780001 FFF80003007C0002003C0002003C0006003E0004001E0004001E000C001F001E001F00FF80FFF0 1C1D7F9C1F>65 DI<001F808000E0618001801980070007800E00 03801C0003801C00018038000180780000807800008070000080F0000000F0000000F0000000F0 000000F0000000F0000000F0000000F0000000700000807800008078000080380000801C000100 1C0001000E000200070004000180080000E03000001FC000191E7E9C1E>IIII<001F808000E0618001801980070007800E0003 801C0003801C00018038000180780000807800008070000080F0000000F0000000F0000000F000 0000F0000000F0000000F000FFF0F0000F80700007807800078078000780380007801C0007801C 0007800E00078007000B800180118000E06080001F80001C1E7E9C21>III76 DII<003F800000E0 E0000380380007001C000E000E001C0007003C00078038000380780003C0780003C0700001C0F0 0001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C0 780003C0380003803C0007801C0007000E000E0007001C000380380000E0E000003F80001B1E7E 9C20>II82 D<07E0801C1980300580700380600180 E00180E00080E00080E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F80 0007C00003C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E7E 9C17>I<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F0020800F00 20000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F800003FFFC001B 1C7F9B1E>IIII<08081010202040404040808080808080B0B0F8F8787830300D 0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C0001FC000F1C00381C00701C 00601C00E01C40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E0006000700430041808 0C3007C00E127E9112>I<003F0000070000070000070000070000070000070000070000070000 070000070003E7000C1700180F00300700700700600700E00700E00700E00700E00700E00700E0 0700600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006 E006FFFEE000E000E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E061E 0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C00 381C00381C001818001C38000C300013C0001000003000001800001FF8001FFF001FFF80300380 6001C0C000C0C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C0018000000000000000000000000000000FC001C001C001C001C 001C001C001C001C001C001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001 E000C000000000000000000000000000000FE000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>II III<03F0000E1C00180600300300700380600180E001C0E001C0E001C0E001C0E001C0E001C0 6001807003803003001806000E1C0003F00012127F9115>II<03C1000C330018 0B00300F00700700700700E00700E00700E00700E00700E00700E00700600700700700300F0018 0F000C370007C700000700000700000700000700000700000700000700003FE0131A7E9116>I< FCE01D301E781E781C301C001C001C001C001C001C001C001C001C001C001C001C00FFC00D127F 9110>I<1F9030704030C010C010E010F8007F803FE00FF000F880388018C018C018E010D0608F C00D127F9110>I<04000400040004000C000C001C003C00FFE01C001C001C001C001C001C001C 001C001C001C101C101C101C101C100C100E2003C00C1A7F9910>IIII<7F8FF00F03 800F030007020003840001C80001D80000F00000700000780000F800009C00010E00020E000607 000403801E07C0FF0FF81512809116>II<7FFC70386038407040F040E041C003 C0038007000F040E041C043C0C380870087038FFF80E127F9112>II E /Fg 8 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000F80000F80000F80000F80000F800 007800007801803C01801C03000E0E0003F80011127E9115>99 D<1E003F003F003F003F001E00 000000000000000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F 001F001F00FFE0FFE00B1E7F9D0E>105 D 110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800F8 7800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018E018F000FF80 7FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Fh 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C060404060C031801F000B107F8F0F> 48 D<0C003C00CC000C000C000C000C000C000C000C000C000C000C000C000C00FF8009107E8F 0F>I<1F00618040C08060C0600060006000C00180030006000C00102020207FC0FFC00B107F8F 0F>I<1F00218060C060C000C0008001800F00008000400060C060C060804060801F000B107F8F 0F>I<0300030007000F000B001300330023004300C300FFE003000300030003001FE00B107F8F 0F>I<20803F002C002000200020002F0030802040006000600060C06080C061801F000B107F8F 0F>I<0780184030C060C06000C000CF00F080E040C060C060C060406060C030801F000B107F8F 0F>I<40007FE07FC08080808001000200040004000C0008000800180018001800180018000B11 7E900F>I<1F00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B10 7F8F0F>I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B10 7F8F0F>I E /Fi 78 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8 F870051C779B18>33 D<4010E038F078E038E038E038E038E038E038E038E038E038E03860300D 0E7B9C18>I<030600078F00078F00078F00078F00078F00078F007FFFC0FFFFE0FFFFE07FFFC0 0F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFFC01E3C001E3C001E3C00 1E3C001E3C001E3C000C1800131C7E9B18>I<01C00007E0000FF0000E70001C38001C38001C38 001C38001C73F01C73F01CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC 00E0F800E0F800E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18>I<007000F001E003C007800F001E001C0038 0038007000700070007000E000E000E000E000E000E000E000E000700070007000700038003800 1C001E000F00078003C001F000F000700C24799F18>I<6000F00078003C001E000F0007800380 01C001C000E000E000E000E00070007000700070007000700070007000E000E000E000E001C001 C0038007800F001E003C007800F00060000C247C9F18>I<01C00001C00001C00001C000C1C180 F1C780F9CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C000 01C00001C00011147D9718>I<00600000F00000F00000F00000F00000F00000F00000F0007FFF C0FFFFE0FFFFE07FFFC000F00000F00000F00000F00000F00000F00000F00000600013147E9718 >I<1C3E7E7F3F1F070E1E7CF860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D8F18>I< 3078FCFC78300606778518>I<000300000780000780000F80000F00001F00001E00001E00003E 00003C00007C0000780000780000F80000F00001F00001E00003E00003C00003C00007C0000780 000F80000F00000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F000 0060000011247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001C07001C0E0 00E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C07001C07803C038 03801C07001F1F000FFE0007FC0001F000131C7E9B18>I<01800380038007800F803F80FF80FB 80438003800380038003800380038003800380038003800380038003800380038003807FFCFFFE 7FFC0F1C7B9B18>I<03F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E0 0000E00001C00001C00003C0000780000F00001E00003C0000780000F00001E00007C0000F8000 1E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C07801 C03001C00001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C00000E00000 E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<001F0000 3F0000770000770000E70001E70001C7000387000787000707000E07001E07003C070038070078 0700F00700FFFFF8FFFFF8FFFFF8000700000700000700000700000700000700007FF000FFF800 7FF0151C7F9B18>I<1FFF803FFF803FFF80380000380000380000380000380000380000380000 3800003BF8003FFE003FFF003C07801803C00001C00000E00000E06000E0F000E0F000E0E001C0 7003C07C0F803FFF001FFC0003F000131C7E9B18>I<03F8000FFE001FFF003E0F803803807001 C07001C07001C07001C03803803C07801FFF0007FC000FFE001F1F003C07807001C0F001E0E000 E0E000E0E000E0E000E07001C07803C03E0F801FFF000FFE0003F800131C7E9B18>56 D<3078FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C1800000000 00000000183C7E7E3E1E0E1C3C78F060071A789318>I<000300000780001F80003F00007E0001 FC0003F00007E0001FC0003F00007E0000FC0000FC00007E00003F00001FC00007E00003F00001 FC00007E00003F00001F8000078000030011187D9918>I<7FFFC0FFFFE0FFFFE0FFFFE0000000 000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C7E9318>I<600000F00000FC00007E00 003F00001FC00007E00003F00001FC00007E00003F00001F80001F80003F00007E0001FC0003F0 0007E0001FC0003F00007E0000FC0000F0000060000011187D9918>I<0FF0003FFC007FFF0070 0F00F00380F00380600780000F00003E00007C0001F00001E00003C00003C00003C00003C00003 C00003800000000000000000000000000000000003800007C00007C00007C000038000111C7D9B 18>I<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E00038E00 038E000306000707000707000707000707000FFF800FFF800FFF800E03800E03801C01C01C01C0 7F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE0 0F07E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E00000 E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F800131C7E 9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C00701C00 701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F 807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E000 00E00000E00000E00000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0F C00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C 01C01C01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C01C 01C01C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF00 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D 9B18>I<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00700E00700E00700E00700E00 707FFFF0FFFFF07FFFF0141C7F9B18>76 DI<7E07F0FF0FF87F07 F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31 C01C39C01C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C 7F9B18>I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F0078070070078 0F007FFF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF 00780F00700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00001C 00001E00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF001C0F801C03801C 03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F001C07001C03801C 03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>I<03F380 1FFF803FFF807C0F80700780E00380E00380E00380E000007000007800003F00001FF00007FE00 00FF00000F800003C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00 E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E070380070000070000070 000070000070000070000070000070000070000070000070000070000070000070000070000070 0000700000700007FF0007FF0007FF00151C7F9B18>II87 D<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001 F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F000707000E 07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II91 D93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000 070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F00F00781F003FFFF01F FBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E 000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00 700F00700F80E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038 0300700000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380000380 03E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380E00380E00380 700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F 00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03 C01FFF8007FF0001FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001 C0007FFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFFF8 1E1E301C0E003807003807003807003807003807001C0E001E1E001FFC001FF80039E000380000 1C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E000387800F07E03F01FFFC0 0FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E 000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FFFF00111D7C9C 18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF800 7FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F001914819318> I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E 0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E 0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E0070 0E00380E00380E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E00 0E00000E00000E00000E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<01E3 8007FB801FFF803E1F80380F80700780700780E00380E00380E00380E00380E00380E003807007 80700780380F803C1F801FFF800FFB8003E3800003800003800003800003800003800003800003 80003FF8003FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003 C0000380000380000380000380000380000380000380000380000380007FFE00FFFF007FFE0015 147F9318>I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE00 001F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<0180000380 000380000380000380007FFFC0FFFFC0FFFFC00380000380000380000380000380000380000380 000380000380000380400380E00380E00380E001C1C001FFC000FF80003E0013197F9818>I<7E 07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E 00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C0 0E03800E03800E0380070700070700070700038E00038E00038E00038E0001DC0001DC0001DC00 00F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F80000F80000700000F00000 F80001DC00039E00038E000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF8 7F8FF00E01C00E03800E0380070380070700070700038700038600038E0001CE0001CE0000CC00 00CC0000DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F8000 3F00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00003C 0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF014147F9318 >I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E000007800007FE0001FE00007E013247E9F18>I< 7C0000FF0000FFC00003C00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000F000007FC0003FE0003FE0007FC000F00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00003C000FFC000FF00007C000013247E9F18>125 D E /Fj 52 122 df<007000E001C00380078007000E001E001E003C003C003C00780078007800 78007000F000F000F000F000F000F000F000F000F000F000F000F000700078007800780078003C 003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<787878781830306060E0050A7D830D>44 DII<00030003000700060006000E000C00 0C001C0018001800380030003000700060006000E000C000C001C0018001800180038003000300 0700060006000E000C000C001C0018001800380030003000700060006000E000C000C000102D7D A117>I<003E00003E00005E00005E0000DE0001DE00019E00039E00039E00079E00071E000F1E 000E1E001E1E003C1E003C1E00781E00781E00F01E00FFFFF0FFFFF0FFFFF0001E00001E00001E 00001E00001E00001E00001E00001E00141E7F9D17>52 D<001F0000001F0000003F8000003B80 00003B8000007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0 F00003C0780003C078000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E 000F001E000F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B207F9F 1E>65 D<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C00007800007800 00780000F00000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00 003C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>67 DIII<001F E000FFF801FFFE03E03E07800E0F00001E00003E00003C00007C0000780000780000780000F000 00F00000F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E00 1E1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>III 76 DII<003F000000FFC00003FFF00007E1F8000F807C001F003E001E001E003C000F00 3C000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003 C0F00003C0F00003C0F00003C0F80007C07800078078000780780007803C000F003C000F001E00 1E001F003E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C0000780000780000780000780000780000 7C00003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001E0 0001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019>IIIII89 D<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF8 0FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000F00078007800 3C013E0F1FFF0FFE03F010147E9314>I<00078000078000078000078000078000078000078000 078000078000078000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F0 0780F00780F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F 17>I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F00000 F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01FE03FE0780 07000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F000F000F000F000F 000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00 780F00780F00780F00780F00780F003C1E003E3E001FFC003FF80033E0003000003800003FFE00 3FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00141E7F 9317>III<01E001 E001E001E00000000000000000000000000000000001E001E001E001E001E001E001E001E001E0 01E001E001E001E001E001E001E001E001E001E001E001E001E001E001E001E0C3C0FFC0FF803F 000B29839F0C>IIIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000F0F000F0F0 00F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F80014147F9317>I< F1F000F7FC00FFFE00FC3E00F81F00F00F00F00F80F00780F00780F00780F00780F00780F00780 F00F00F00F00F81F00FC3E00FFFC00F7F800F1E000F00000F00000F00000F00000F00000F00000 F00000F00000F00000111D7D9317>I<03C7800FF7801FFF803E1F807C0F80780780780780F007 80F00780F00780F00780F00780F00780F807807807807C0F803E1F801FFF800FF78007C7800007 80000780000780000780000780000780000780000780000780111D7E9317>I I<07F01FFC3FFC780C7800780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF0 0FC00E147F9311>I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E00 1E001E001E001E001E001E001E001E201FF00FF007C00C1A7F9910>IIII<7801E07C 03C03E07801E0F000F0F00079E0003FC0003F80001F80000F00001F00001F80003FC00079E000F 0F000E0F001E07803C03C07801E0F801F01414809315>I I E /Fk 21 122 df<7FF0FFE07FE00C037D8A10>45 D<00F1800389C00707800E03801C03803C 0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C4030 8C800F070013147C9317>97 D<07803F8007000700070007000E000E000E000E001C001C001CF0 1D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010 207B9F15>I<007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000 F00000F00000F00000F0000070010070020030040018380007C00011147C9315>I<0000780003 F80000700000700000700000700000E00000E00000E00000E00001C00001C000F1C00389C00707 800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C 40703C40705C40308C800F070015207C9F17>I<007C01C207010E011C013C013802780C7BF07C 00F000F000F000F0007000700170023804183807C010147C9315>I<003C6000E27001C1E00380 E00700E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C03803C07003C07001C0F 001C17000C2E0003CE00000E00000E00001C00001C00301C00783800F0700060E0003F8000141D 7E9315>103 D<00C001E001E001C000000000000000000000000000000E003300230043804300 470087000E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E>105 D<01E0000FE00001C00001C00001C00001C0000380000380000380000380000700000700000703 C00704200E08E00E11E00E21E00E40C01C80001D00001E00001FC00038E0003870003870003838 40707080707080707080703100E03100601E0013207D9F15>107 D<03C01FC003800380038003 8007000700070007000E000E000E000E001C001C001C001C003800380038003800700070007000 7100E200E200E200E200640038000A207C9F0C>I<1C0F80F0002630C318004740640C00478068 0E004700700E004700700E008E00E01C000E00E01C000E00E01C000E00E01C001C01C038001C01 C038001C01C038001C01C0708038038071003803806100380380E1003803806200700700660030 0300380021147C9325>I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E 00E01C01C01C01C01C01C01C03843803883803083807083803107003303001C016147C931A>I< 007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E002621804741C0478 1C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380780380700380 E003C1C0072380071E000700000700000E00000E00000E00000E00001C00001C0000FFC000171D 809317>I<00F0400388C00705800E03801C03803C0380380700780700780700780700F00E00F0 0E00F00E00F00E00F01C00F01C00703C00705C0030B8000F380000380000380000700000700000 700000700000E00000E0000FFE00121D7C9315>I<1C1E00266100478380478780470780470300 8E00000E00000E00000E00001C00001C00001C00001C0000380000380000380000380000700000 30000011147C9313>I<00FC030206010C030C070C060C000F800FF007F803FC003E000E700EF0 0CF00CE008401020601F8010147D9313>I<018001C0038003800380038007000700FFF007000E 000E000E000E001C001C001C001C003800380038003820704070407080708031001E000C1C7C9B 0F>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E03801C07001C0700 1C07001C07101C0E20180E20180E201C1E200C264007C38014147C9318>I<0E03803307802307 C04383C04301C04700C08700800E00800E00800E00801C01001C01001C01001C02001C02001C04 001C04001C08000E300003C00012147C9315>I<0E00C03300E02301C04381C04301C04701C087 03800E03800E03800E03801C07001C07001C07001C07001C0E00180E00180E001C1E000C3C0007 DC00001C00001C00003800F03800F07000E06000C0C0004380003E0000131D7C9316>121 D E /Fl 42 122 df<787878781830306060E0050A7C830E>44 D46 D<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C0FFFFFFFFFFFF10227CA118>49 D<03F0000FFC001FFE003C1F003007807007C06003C0E003E0C001E04001E04001E00001E00001 E00001E00003C00003C0000780000780000F00001E00003C0000780000F00001E00001C0000380 000700000E00001C0000380000700000FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC001F FF003E0F003807807003C02003C02003C00003C00003C00003C0000780000780000F00001E0003 FC0003F80003FE00000F000007800003C00003C00001E00001E00001E00001E00001E08001E0C0 03C0E003C07007803C0F801FFF000FFC0003F00013237EA118>I<001F00001F00002F00002F00 006F0000EF0000CF0001CF0001CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F00 3C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00 000F00000F00000F0015217FA018>I<001F0000001F0000003F8000003F8000003B8000007BC0 000073C0000071C00000F1E00000F1E00000E0E00001E0F00001E0F00001C0F00003C0780003C0 78000380780007803C0007803C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF001C 000F003C0007803C00078038000780780003C0780003C0700003C0F00001E0F00001E0E00001E0 1C237EA220>65 D<000FF000007FFC0000FFFF0001F01F0003C00700078000000F0000001E0000 003E0000003C0000003C000000780000007800000078000000F0000000F0000000F0000000F000 0000F0000000F0000000F0000000F0000000F00000007800000078000000780000003C0000003C 0000003E0000001E0000000F0000000780008003C0038001F00F8000FFFF00007FFC00000FF000 19257DA31F>67 DII73 D77 D<001FC000007FF00001FFFC0003F07E0007C01F000F800F801F0007C01E0003C03C0001E03C00 01E0780000F0780000F0780000F070000070F0000078F0000078F0000078F0000078F0000078F0 000078F0000078F0000078F0000078780000F0780000F0780000F07C0001F03C0001E03E0003E0 1E0003C01F0007C00F800F8007C01F0003F07E0001FFFC00007FF000001FC0001D257DA324>79 DI82 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800007800007800007800007800007C00 003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F00000F00000F80000780000 78000078000078000078000078C000F0E000F0F801E07E07C03FFF800FFF0001FC0015257EA31B >III<07E01FF83FFC381E20 1E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E95 17>97 DI<01 FC0007FF000FFF801F03803C0180780000780000700000F00000F00000F00000F00000F00000F0 00007800007800007800003C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC0 1FFFC03F0FC03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0 7803C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC001FFE003E0F 003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F000007000007800007800 003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01FF03C00380078007800780 07800780078007800780FFF8FFF8FFF80780078007800780078007800780078007800780078007 800780078007800780078007800780102380A20F>I<01F07807FFF80FFFF81F1F001E0F003C07 803C07803C07803C07803C07801E0F001F1F000FFE001FFC0019F0003800003800003C00001FFE 001FFFC01FFFE03FFFF07801F07800F8F00078F00078F00078F000787800F03E03E01FFFC00FFF 8001FC0015217F9518>III<00F000F000F000F0000000000000000000000000000000000000 00F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000 F000F000F000F000F000F000F000F080F0E3E0FFE07FC01F000C2D83A20D>I108 DII<01FC0007FF000FFF80 1F07C03C01E07800F07800F0700070F00078F00078F00078F00078F00078F000787800F07800F0 7C01F03E03E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07C07C03C07803C0F803C0F003C0F0 03C0F003C0F003C0F003C0F003C0F803C07803C07803C03C07C03F0FC01FFFC00FFBC003E3C000 03C00003C00003C00003C00003C00003C00003C00003C00003C00003C012207E9519>II<07F01FFC3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F 000F000FC00FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF8 0F000F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E00E 1C7F9B12>IIII<7801F07C01E03E03C0 1E07C00F0780078F0007DE0003FC0001FC0000F80000700000F80001FC0003DC00039E00078F00 0F07801E07801E03C03C01E07800F0F800F81516809516>II E /Fm 27 90 df<3078F8787005057C840D>46 D<000C001C00FC0F380038003800380038003800700070007000700070007000E000E000E000E0 00E000E001C001C001C001C001C001C0038003C0FFFE0F1E7C9D17>49 D<003F8000C1E00100F0 0200780400780400780F007C0F807C0F807C0F00780600780000F80000F00001E00001C0000380 000700000E00001C0000380000600000C0000180000300200600200800401000403FFFC07FFF80 FFFF80161E7E9D17>I<007F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001 E00003C0000380000700000E0000F800000E000007000007800007C00003C00007C03007C07807 C0F807C0F807C0F00780800F00400E00201C0018780007E000141F7D9D17>I<00006000006000 00E00001C00003C00005C0000DC00009C00011C000238000438000C38000838001038002038004 0700080700180700100700200700400700FFFFF0000E00000E00000E00000E00000E00001C0000 1E0001FFE0141E7E9D17>I<01803001FFE003FFC003FF0003FC00020000020000020000040000 040000040000047C000587000603800C01800801C00001C00001E00001E00001E00001E07003C0 F803C0F003C0E00380800780400700400E00201C0018700007C000141F7D9D17>I<0000100000 001800000038000000380000007800000078000000FC000001BC0000013C0000033C0000023C00 00063C0000043E0000081E0000081E0000101E0000101E0000201E0000200F0000400F0000400F 0000FFFF0000800F0001000F8001000780020007800200078004000780040007800C0007C03E00 07C0FF807FFC1E207E9F22>65 D<07FFFF00007C01C0003C01E0003C00F0007800F8007800F800 7800F8007800F8007800F8007800F000F001F000F001E000F003C000F00F8000FFFE0000F00F00 01E007C001E003C001E003E001E001E001E001E001E001E003C001E003C003E003C003E003C003 C003C007C003C00F8007800F0007803E00FFFFF0001D1F7E9E20>I<0001F808000E0618003801 38007000F801E0007803C0007007800030078000300F0000301F0000301E0000303E0000203C00 00007C0000007C0000007C0000007C000000F8000000F8000000F8000000F8000000F800000078 00004078000080780000803C0000803C0001001C0002000E00020006000C000300100001C0E000 003F00001D217B9F21>I<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E 0078001E0078001E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E000 1E01E0003E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C0 03C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038003C 001800780018007800080078000800780008007800080078080800F0100000F0100000F0100000 F0300000FFF00000F0700001E0200001E0200001E0200001E0200001E0000801E0001003C00010 03C0001003C0002003C0002003C0006003C000C0078001C0078007C0FFFFFF801D1F7E9E1F>I< 07FFFFF8007C0078003C0038003C00180078001800780008007800080078000800780008007800 0800F0100000F0100000F0100000F0300000F0700000FFF00001E0600001E0200001E0200001E0 200001E0200001E0000003C0000003C0000003C0000003C0000003C0000003C000000780000007 C00000FFFE00001D1F7E9E1E>I<0001FC04000F030C003C009C0070007C00E0003C01C0003803 800018078000180F0000181F0000181E0000183E0000103C0000007C0000007C0000007C000000 7C000000F8000000F8000000F8007FFCF80003E0780001E0780001E0780003C0780003C03C0003 C03C0003C01C0003C00E0007C007000B800380118001E06080003F80001E217B9F24>I<07FFC7 FFC0007C00F800003C007800003C007800007800F000007800F000007800F000007800F0000078 00F000007800F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000F001E00001 E003C00001E003C00001E003C00001E003C00001E003C00001E003C00003C007800003C0078000 03C007800003C007800003C007800003C007800007800F000007C00F8000FFF8FFF800221F7E9E 22>I<07FFE0007C00003C00003C0000780000780000780000780000780000780000F00000F000 00F00000F00000F00000F00001E00001E00001E00001E00001E00001E00003C00003C00003C000 03C00003C00003C00007800007C000FFFC00131F7F9E10>I<00FFFC0007C00003C00003C00007 80000780000780000780000780000780000F00000F00000F00000F00000F00000F00001E00001E 00001E00001E00001E00001E00003C00303C00783C00F83C00F83800F0780080700040E00021C0 001F000016207D9E17>I<07FFF000007E0000003C0000003C0000007800000078000000780000 00780000007800000078000000F0000000F0000000F0000000F0000000F0000000F0000001E000 0001E0000001E0000001E0000001E0008001E0010003C0010003C0010003C0030003C0020003C0 060003C0060007801E0007807C00FFFFFC00191F7E9E1C>76 D<07FC0000FFC0007C0000F80000 3C00017800003C00017800004E0002F000004E0002F000004E0004F000004E0004F000004E0008 F000004E0008F00000870011E00000870011E00000870021E00000870021E00000870041E00000 838041E00001038083C00001038083C00001038103C00001038203C0000101C203C0000101C403 C0000201C40780000201C80780000201C80780000201D00780000200F00780000600E007800006 00E00F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E001800 003E001800004F001000004F001000004780100000478010000043C010000043C010000083C020 000081E020000081E020000080F020000080F020000080782000010078400001007C400001003C 400001003C400001001E400001001E400002000F800002000F800002000F800002000780000200 078000060003800006000300000F00010000FFE0010000221F7E9E22>I<0003F800001E0E0000 38070000E0038001C001C003C001E0078000E00F0000F00F0000F01E0000F01E0000F83E0000F8 3C0000F87C0000F87C0000F87C0000F87C0000F8F80001F0F80001F0F80001F0F80001F0F80003 E0780003E0780003C0780007C07C0007803C000F003C001E001E001C000E0038000700F00003C3 C00000FE00001D217B9F23>I<07FFFF00007C03C0003C01E0003C00F0007800F0007800F80078 00F8007800F8007800F8007800F000F001F000F001E000F003C000F0078000F00F0000FFF80001 E0000001E0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C00000 03C0000003C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0 003C01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F007 8000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E01E0003C0 3E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC0F18000003E01F 207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E00080E00080E00 000F00000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E00000F00000F04000 E04000E04000E04000E06000C0600180E00380F80300C60C0081F80016217D9F19>I<3FFFFFF0 3C0780F03007803060078030400F0010400F0010C00F0010800F0010800F0010800F0010001E00 00001E0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C 0000003C000000780000007800000078000000780000007800000078000000F0000001F800007F FFE0001C1F7A9E21>I87 D<03FFC0FFC0007F007E00003E003800001E003000001E0020 00000F004000000F008000000F81000000078200000007C600000003C400000003E800000001F0 00000001F000000000F000000000F800000000F8000000017C000000023C000000043C0000000C 1E000000081E000000101F000000200F000000400F800000C0078000008007C000010003C00007 0003E0001F8007E000FFE01FFE00221F7F9E22>II E /Fn 1 16 df<07E01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC 1FF807E010127D9317>15 D E /Fo 28 122 df<0000600000E00000E00001C00001C000038000 0380000380000700000700000700000E00000E00001C00001C00001C0000380000380000380000 700000700000E00000E00000E00001C00001C0000380000380000380000700000700000700000E 00000E00001C00001C00001C0000380000380000380000700000700000E00000E00000C0000013 2D7DA11A>47 D<07F8001FFE00381F80780F80FC0FC0FC0FC0FC0FC0780FC0301F80001F00003E 00007C0000700000E00000E00000C00000C00000C00000C00000C00000C0000000000000000000 0000000001C00007F00007F00007F00007F00007F00001C00012207D9F19>63 D<0000E000000000E000000001F000000001F000000001F000000003F800000003F800000006FC 00000006FC0000000EFE0000000C7E0000000C7E000000183F000000183F000000303F80000030 1F800000701FC00000600FC00000600FC00000C007E00000FFFFE00001FFFFF000018003F00001 8003F000030001F800030001F800060001FC00060000FC000E0000FE00FFE00FFFE0FFE00FFFE0 231F7E9E28>65 D69 D73 D77 D80 D<03FC080FFF381E03F838 00F8700078700038F00038F00018F00018F80000FC00007FC0007FFE003FFF801FFFC00FFFF007 FFF000FFF80007F80000FC00007C00003CC0003CC0003CC0003CE00038E00078F80070FE01E0E7 FFC081FF00161F7D9E1D>83 D<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003E0007F E007FBE01F03E03C03E07C03E0F803E0F803E0F803E0FC05E07E0DE03FF9FE0FE07E17147F9319 >97 DI<01FE0007FF801F0FC03E0F C03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000FC0000FC00007C00007E00003E00603F00 C01F81C007FF0001FC0013147E9317>I<0007F80007F80000F80000F80000F80000F80000F800 00F80000F80000F80000F80000F801F8F80FFEF81F83F83E01F87E00F87C00F87C00F8FC00F8FC 00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F87E00F83E01F81F07F80FFEFF03F8FF18207E9F 1D>I<01FE0007FF801F83E03F01F07E00F07E00F8FC00F8FC00F8FFFFF8FFFFF8FC0000FC0000 FC00007C00007E00003E00183F00380F807007FFE000FF8015147F9318>I104 D<1C003F007F007F007F003F001C00000000000000 000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F001F001F 001F00FFE0FFE00B217EA00E>I 107 DIII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC 007EFC007EFC007EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F93 1A>II114 D<0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC007C0 07E007F006F81EFFFCC7F010147E9315>I<01800180018003800380038007800F803F80FFFCFF FC0F800F800F800F800F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D 7F9C14>IIII121 D E /Fp 37 122 df46 D<00180000380000F80007F800FFF800FFF8 00F8F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F8007FFFF07FFFF07FFFF014287BA71E>49 D<00FE0003FFC007FFE00FFFF01F03F83C00FC38007E78003E70003EF0001FF0001F60001F2000 1F00001F00001F00001F00003E00003E00007C00007C0000F80001F00001E00003C0000780000F 00001E00003C0000780000F00001E00003C0000780000F00001E00003C00007FFFFF7FFFFF7FFF FF7FFFFF18287EA71E>I<007F000001FFC00007FFF0000FFFF8001FC1F8003E007C003C003E00 78003E0038003E0010003E0000003E0000003E0000003C0000007C000000FC000001F8000007F0 0000FFE00000FFC00000FFE00000FFF0000001FC0000007C0000003E0000001F0000001F000000 0F8000000F8000000F8000000F8000000F8040000F8060001F00F0001F00F8003F007E007E003F 81FC001FFFF8000FFFF00003FFE000007F000019297EA71E>I<0003F0000007F0000005F00000 0DF000000DF000001DF0000039F0000039F0000079F0000079F00000F1F00000F1F00001E1F000 03E1F00003E1F00007C1F00007C1F0000F81F0000F81F0001F01F0001F01F0003E01F0007C01F0 007C01F000F801F000FFFFFF80FFFFFF80FFFFFF80FFFFFF800001F0000001F0000001F0000001 F0000001F0000001F0000001F0000001F0000001F0000001F00019277EA61E>I<3FFFFC3FFFFC 3FFFFC3FFFFC3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E3F00 3EFFC03FFFE03FFFF03FE1F83F807C3F003E3E003E00003E00001F00001F00001F00001F00001F 00001F00001F20001F60003E70003EF8007C7C00FC3F03F81FFFF00FFFE007FF8000FE0018287E A61E>I68 DI73 D77 DI<0001FC0000000FFF8000003FFFE000 007FFFF00001FE03FC0003F800FE0007E0003F0007C0001F000F80000F801F000007C01F000007 C03E000003E03E000003E07C000001F07C000001F07C000001F078000000F0F8000000F8F80000 00F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F800 0000F87C000001F07C000001F07C000001F07E000003F03E000003E03F000007E01F000007C01F 80000FC00FC0001F8007E0003F0007F0007F0003F800FE0001FE03FC0000FFFFF800003FFFE000 000FFF80000001FC0000252C7DAA2C>II<007FC0 0001FFF80007FFFE000FFFFF001FC07F003F000F007E0006007C0000007C000000F8000000F800 0000F8000000F8000000F8000000FC0000007E0000007F0000003F8000001FF800000FFF800007 FFE00003FFF80000FFFC00000FFE000000FF0000003F0000001F8000000F8000000FC0000007C0 000007C0000007C0000007C0000007C0000007C000000F8060000F80F0001F00FC003F00FF80FE 007FFFFC001FFFF80007FFE00000FF80001A2C7DAA21>83 DI<01FE000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F80000F800 00F80000F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F800F8F800F8FC01F87E07F87F FFF83FFFF81FFCF80FE0F8151B7E9A1D>97 DI<007FC001 FFF007FFFC0FFFFC1FC07C1F00083E00007C00007C00007C0000F80000F80000F80000F80000F8 0000F80000F800007C00007C00007E00003E00001F000C1FC07C0FFFFC07FFFC01FFF0007F8016 1B7E9A1B>I<00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00FC3E03FF3E07FFFE0FFFFE1FC1FE3F007E3E003E7C003E7C003E FC003EF8003EF8003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E7C003E3E007E3F00FE 1FC1FE0FFFFE07FFBE03FF3E00FC3E172A7EA91F>I<007E0003FF8007FFC00FFFE01F83F03F00 F03E00787C00787C003878003CFFFFFCFFFFFCFFFFFCFFFFFCF80000F80000F800007800007C00 007C00003E00003F000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<001FC0007FC000 FFC001FFC003F00003E00007C00007C00007C00007C00007C00007C00007C00007C00007C000FF FE00FFFE00FFFE0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C000122A7FA912>I<00F8078003FE7FC00FFFFFC01FFFFFC01F07C0003E03E0003E03E0007C01 F0007C01F0007C01F0007C01F0007C01F0007C01F0003E03E0003E03E0001F07C0001FFFC0003F FF80003BFE000038F8000078000000780000003C0000003FFFC0003FFFF8001FFFFC001FFFFE00 3FFFFF007C007F00F8001F80F8000F80F8000F80F8000F80FC001F807E003F003F80FE003FFFFE 000FFFF80007FFF00000FF80001A287E9A1E>III<007C007C007C007C007C0000000000000000000000000000000000000000007C 007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C00 7C007C007C007C007C007C007C007C007C007C007C007C007C40FCF1F8FFF8FFF07FE00F800E36 83A910>I108 DII<007F000001FFC00007FFF0000FFFF8001FC1FC003F007E003E00 3E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8 000F807C001F007C001F007E003F003E003E003F007E001FC1FC000FFFF80007FFF00001FFC000 007F0000191B7E9A1E>II 114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF800 3FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFFC07FFF80 1FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007C000FFFFC0FFFF C0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C00007C04007E1C003FFE003FFE001FF8000FC001322 7FA116>II 119 D<7C000FC03E001F803F001F001F803E000F807C0007C0FC0003E0F80001F1F00001FBE000 00FFC000007FC000003F8000001F0000001F0000003F8000007FC00000FBC00000F3E00001F1F0 0003E0F80007C07C000F807C000F803E001F001F003E000F807E000FC0FC0007E01B1B809A1C> II E /Fq 12 122 df68 D80 D<0007FFFC000000007FFFFFC0000001 FFFFFFF8000003FFFFFFFE000007FE001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000FF F0000FFF80007FF0000FFF80007FF8000FFF80007FF80007FF00003FFC0007FF00003FFC0003FE 00003FFC0000F800003FFC00000000003FFC00000000003FFC00000000003FFC00000000003FFC 00000007FFFFFC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC0003FFF0 003FFC0007FFC0003FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC007FF800003FFC00 7FF800003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF00000 3FFC00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC0003EFFC003FFF0007CFFF000F FFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFFF001FFFC0003FF80007FF8362E7DAD3A >97 D<00001FFFC0000000FFFFF8000007FFFFFE00001FFFFFFF80007FFC00FFC000FFE001FFC0 01FFC003FFE003FF8003FFE007FF0003FFE00FFE0003FFE00FFE0003FFE01FFC0001FFC01FFC00 01FFC03FFC0000FF803FFC00003E007FF8000000007FF8000000007FF800000000FFF800000000 FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800 000000FFF800000000FFF8000000007FF8000000007FF8000000007FFC000000003FFC00000000 3FFC000000001FFC000000F81FFE000000F80FFE000000F80FFF000001F007FF800003F003FFC0 0007E001FFE0000FC000FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8000000FFFFE000 00001FFE00002D2E7CAD35>99 D<00001FFE00000001FFFFE0000007FFFFF800001FFFFFFE0000 7FFC07FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00003FF00FFE00001FF01FFE0000 0FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC000007FC7FF8000007FC7FF8000007FE7F F8000007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFF FFFCFFF800000000FFF800000000FFF800000000FFF8000000007FF8000000007FF8000000007F FC000000003FFC000000003FFC000000003FFC0000001C1FFE0000003E0FFE0000003E07FF0000 007E07FF000000FC03FF800001F801FFC00003F0007FF0001FE0003FFE00FFC0001FFFFFFF8000 07FFFFFE000000FFFFF80000000FFF80002F2E7DAD36>101 D<00FC0001FF0003FF8007FFC00F FFC01FFFE01FFFE01FFFE01FFFE01FFFE01FFFE00FFFC007FFC003FF8001FF0000FC0000000000 0000000000000000000000000000000000000000000000000000000000007FC0FFFFC0FFFFC0FF FFC0FFFFC0FFFFC003FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0FFFFFFFF FFFFFFFFFFFFFFFFFFFFFF18497CC820>105 D<007FC001FFC00000FFE00000FFFFC00FFFF800 07FFFC0000FFFFC03FFFFE001FFFFF0000FFFFC0FFFFFF007FFFFF8000FFFFC1FC07FF80FE03FF C000FFFFC3E003FFC1F001FFE00003FFC7C001FFC3E000FFE00001FFCF0001FFE78000FFF00001 FFDE0000FFEF00007FF00001FFDC0000FFEE00007FF00001FFFC0000FFFE00007FF80001FFF800 00FFFC00007FF80001FFF00000FFF800007FF80001FFF00000FFF800007FF80001FFF00000FFF8 00007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000007F F80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001 FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE000 00FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF0 00007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000007F F80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001 FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE000 00FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF000007FF80001FFE00000FFF0 00007FF800FFFFFFC07FFFFFE03FFFFFF0FFFFFFC07FFFFFE03FFFFFF0FFFFFFC07FFFFFE03FFF FFF0FFFFFFC07FFFFFE03FFFFFF0FFFFFFC07FFFFFE03FFFFFF05C2E7CAD65>109 D<007FC001FFC00000FFFFC00FFFF80000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1FC07FF 8000FFFFC3E003FFC00003FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE00001FFDC0000 FFE00001FFFC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF00000FFF00001FFF000 00FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001 FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF000 01FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFF FFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B2E7CAD42>I<00000FFF0000000000FFFFF00000 0007FFFFFE0000001FFFFFFF8000003FFC03FFC00000FFE0007FF00001FF80001FF80003FF0000 0FFC0007FE000007FE000FFE000007FF000FFC000003FF001FFC000003FF803FFC000003FFC03F F8000001FFC03FF8000001FFC07FF8000001FFE07FF8000001FFE07FF8000001FFE0FFF8000001 FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8 000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF07FF8000001FFE07FF8000001FF E07FF8000001FFE07FF8000001FFE03FFC000003FFC03FFC000003FFC01FFC000003FF801FFE00 0007FF800FFE000007FF0007FF00000FFE0003FF80001FFC0001FFC0003FF80000FFE0007FF000 007FFC03FFE000001FFFFFFF80000007FFFFFE00000000FFFFF0000000000FFF000000342E7DAD 3B>I<00FF803F8000FFFF80FFF000FFFF83FFFC00FFFF87FFFE00FFFF8FC3FF00FFFF8F07FF00 03FF9E0FFF8001FFBC0FFF8001FFB80FFF8001FFF80FFF8001FFF00FFF8001FFF007FF0001FFF0 07FF0001FFE003FE0001FFE000F80001FFE000000001FFE000000001FFC000000001FFC0000000 01FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0 00000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0 00000001FFC000000001FFC0000000FFFFFFE00000FFFFFFE00000FFFFFFE00000FFFFFFE00000 FFFFFFE00000292E7CAD31>114 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FF E01FC0001FE03F80000FE03F000007E07F000003E07F000003E0FF000003E0FF000003E0FF8000 03E0FFC0000000FFF0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFFFFFC001FFF FFFF000FFFFFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FFFFF800001FFFF800 0000FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003FCFC000003FCFE000003F8 FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0FFFC00FFC0FFFFFFFF80FC7FFFFE 00F81FFFF800E003FF8000262E7CAD2F>I<7FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000 FFFF807FFFFFC000FFFF807FFFFFC000FFFF8000FFF000000FE00000FFF800000FC00000FFF800 000FC000007FFC00000F8000007FFC00001F8000003FFC00001F0000003FFE00003F0000001FFE 00003E0000001FFF00007E0000000FFF00007C0000000FFF8000FC00000007FF8000F800000007 FFC001F800000003FFC001F000000003FFE003F000000003FFE003F000000001FFF003E0000000 01FFF007E000000000FFF007C000000000FFF80FC0000000007FF80F80000000007FFC1F800000 00003FFC1F00000000003FFE3F00000000001FFE3E00000000001FFF7E00000000000FFF7C0000 0000000FFFFC00000000000FFFFC000000000007FFF8000000000007FFF8000000000003FFF000 0000000003FFF0000000000001FFE0000000000001FFE0000000000000FFC0000000000000FFC0 0000000000007F800000000000007F800000000000003F000000000000003F000000000000003F 000000000000003E000000000000007E000000000000007C00000000000000FC000000001F8000 F8000000003FC001F8000000007FE001F000000000FFF003F000000000FFF003E000000000FFF0 07E000000000FFF00FC000000000FFF01F8000000000FFF03F80000000007FE07F00000000007F 43FE00000000003FFFF800000000001FFFF0000000000007FFC0000000000001FE000000000000 39427EAD3F>121 D E /Fr 8 117 df<00000F000000001F000000007F00000003FF0000001FFF 0000FFFFFF0000FFFFFF0000FFFFFF0000FFE7FF00000007FF00000007FF00000007FF00000007 FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF000000 07FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF0000 0007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00 000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF 00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007FF00000007 FF00007FFFFFFFE07FFFFFFFE07FFFFFFFE07FFFFFFFE0233879B732>49 D<0000001FFE0000E0000003FFFFC001E000001FFFFFF803E000007FFFFFFE07E00001FFFC00FF 0FE00007FFC0001FDFE0000FFF000007FFE0003FFC000003FFE0007FF8000001FFE000FFF00000 00FFE001FFE00000007FE003FFC00000003FE007FF800000001FE007FF800000001FE00FFF0000 00000FE00FFF000000000FE01FFE0000000007E01FFE0000000007E03FFC0000000007E03FFC00 00000007E07FFC0000000003E07FFC0000000003E07FFC0000000003E07FF8000000000000FFF8 000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FF F8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000 FFF8000000000000FFF8000000000000FFF80000000000007FF80000000000007FFC0000000000 007FFC0000000001E07FFC0000000001E03FFC0000000001E03FFC0000000001E01FFE00000000 01E01FFE0000000003E00FFF0000000003C00FFF0000000007C007FF8000000007C007FF800000 000F8003FFC00000000F8001FFE00000001F0000FFF00000003E00007FF80000007C00003FFC00 0000F800000FFF000003F0000007FFE0000FE0000001FFFC007FC00000007FFFFFFF000000001F FFFFFC0000000003FFFFE000000000001FFE0000003B3D7ABB48>67 D<001FFF00000001FFFFF0 000007FFFFFC00000FF807FF00001FF801FF80001FFC00FFC0001FFC007FE0001FFC007FE0001F FC007FF0001FFC003FF0000FF8003FF00007F0003FF00001C0003FF0000000003FF0000000003F F0000000003FF0000000FFFFF000000FFFFFF000007FF83FF00003FF803FF00007FE003FF0001F FC003FF0003FF8003FF0007FF0003FF0007FE0003FF000FFE0003FF000FFC0003FF000FFC0003F F000FFC0003FF000FFC0007FF000FFC0007FF000FFE000FFF0007FF001DFF0003FF803DFF8001F FC0F8FFFF00FFFFE0FFFF001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFFFE0000FF80FF0001FE003F8007FC001FC00FF8000FE00FF800 0FF01FF00007F03FF00007F83FF00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE0 0003FCFFFFFFFFFCFFFFFFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007F E00000007FE00000007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF8000078 07FC0000F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE00026267DA5 2D>101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF0000000007FF0000 000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003 FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000 000003FF0000000003FF0000000003FF0000000003FF007FC00003FF01FFF80003FF07FFFE0003 FF1F03FF0003FF3C01FF0003FF7801FF8003FF7000FF8003FFE000FFC003FFC000FFC003FFC000 FFC003FF8000FFC003FF8000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003 FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000 FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003 FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3F FFFFFFFFFC3FFFFFFFFFFC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFF FC00FFFFFE03FF00FFFFF001FFC003FFE0007FE003FF80007FF003FF80003FF803FF00001FF803 FF00001FFC03FF00000FFC03FF00000FFE03FF00000FFE03FF00000FFE03FF000007FF03FF0000 07FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03 FF000007FF03FF000007FF03FF00000FFE03FF00000FFE03FF00000FFE03FF00001FFC03FF0000 1FFC03FF00001FF803FF80003FF003FFC0007FF003FFE000FFE003FFF001FF8003FFFC07FF0003 FF3FFFFC0003FF0FFFF00003FF01FF000003FF0000000003FF0000000003FF0000000003FF0000 000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003 FF0000000003FF0000000003FF00000000FFFFFC000000FFFFFC000000FFFFFC000000FFFFFC00 000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE3FFF80FFFE3C7FC0FFFE707FC007FEF0 FFE003FEE0FFE003FEC0FFE003FFC0FFE003FF80FFE003FF807FC003FF803F8003FF800E0003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF000000 03FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE0000FFFFFE00 00FFFFFE000023267DA529>114 D<0007800000078000000780000007800000078000000F8000 000F8000000F8000000F8000001F8000001F8000003F8000003F8000007F800000FF800001FF80 0007FF80001FFFFFF8FFFFFFF8FFFFFFF8FFFFFFF801FF800001FF800001FF800001FF800001FF 800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001 FF800001FF800001FF800001FF800001FF800001FF803C01FF803C01FF803C01FF803C01FF803C 01FF803C01FF803C01FF803C00FF807800FFC078007FC0F8007FE1F0001FFFE00007FFC00001FF 001E377EB627>116 D E /Fs 82 124 df<07FFF800003F0000001E0000001E0000001E000000 1E0000007F800003DEF0000F1E3C001E1E1E003C1E0F007C1E0F80781E0780F81E07C0F81E07C0 F81E07C0F81E07C0F81E07C0781E07807C1E0F803C1E0F001E1E1E000F1E3C0003DEF000007F80 00001E0000001E0000001E0000001E0000003F000007FFF8001A1F7D9E21>8 D<001F83E000F06E3001C078780380F8780300F030070070000700700007007000070070000700 70000700700007007000FFFFFF8007007000070070000700700007007000070070000700700007 007000070070000700700007007000070070000700700007007000070070000700700007007000 07007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007 0000070000070000070000070000070000FFFFE00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FC3FE17 20809F19>I<003FE000E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E0 0700E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0 0700E00700E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F 040001C07C06000380F80F000300F00F000700F00F000700700000070070000007007000000700 70000007007000000700700000FFFFFFFF00070070070007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700700700 0700700700070070070007007007000700700700070070070007007007007FE3FE3FF02420809F 26>I<7038F87CFC7EFC7E743A0402040204020804080410081008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C000C00 180018003000300030007000600060006000E000E000E000E000E000E000E000E000E000E000E0 00E0006000600060007000300030003000180018000C000C000600020001000080004000200B2E 7DA112>I<800040002000100008000C00060006000300030001800180018001C000C000C000C0 00E000E000E000E000E000E000E000E000E000E000E000E000C000C000C001C001800180018003 000300060006000C00080010002000400080000B2E7DA112>I<01800180018001800180C183F1 8F399C0FF003C003C00FF0399CF18FC1830180018001800180018010147DA117>I<70F8FCFC74 040404080810102040060E7C840D>44 DI<70F8F8F87005057C840D>I< 00030003000700060006000E000C000C001C0018001800380030003000700060006000E000C000 C001C00180018001800380030003000700060006000E000C000C001C0018001800380030003000 700060006000E000C000C000102D7DA117>I<03F0000E1C001C0E001806003807007003807003 80700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003 C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003F000121F7E9D17 >I<018003800F80F3800380038003800380038003800380038003800380038003800380038003 8003800380038003800380038003800380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E00 200700400780800780F007C0F803C0F803C0F803C02007C00007C0000780000780000F00000E00 001C0000380000700000600000C0000180000300000600400C00401800401000803FFF807FFF80 FFFF80121E7E9D17>I<03F0000C1C00100E00200F00780F80780780780780380F80000F80000F 00000F00000E00001C0000380003F000003C00000E00000F000007800007800007C02007C0F807 C0F807C0F807C0F00780400780400F00200E001C3C0003F000121F7E9D17>I<00060000060000 0E00000E00001E00002E00002E00004E00008E00008E00010E00020E00020E00040E00080E0008 0E00100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E0000 0E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE000100000100000100000100000 10000010000011F000161C00180E001007001007800003800003800003C00003C00003C07003C0 F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17>I<007C000182 000701000E03800C07801C0780380300380000780000700000700000F1F000F21C00F40600F807 00F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003C07003803803803807001807 000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF807FFF8040010080020080020080 040000080000080000100000200000200000400000400000C00000C00001C00001800003800003 8000038000038000078000078000078000078000078000078000078000030000121F7D9D17>I< 03F0000C0C001006003003002001806001806001806001807001807803003E03003F06001FC800 0FF00003F80007FC000C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0C00080 6001802001001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C003806003807007007 00700380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13 C003E3C0000380000380000380000700300700780600780E00700C002018001070000FC000121F 7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F870000000 0000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0FFFFFFF0000000 0000000000000000000000000000000000000000000000000000000000FFFFFFF07FFFFFE01C0C 7D9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000C0018001800100030002 00020002000200020002000000000000000000000007000F800F800F8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C0000007C0000009E0000009E0000009 E0000010F0000010F0000010F00000207800002078000020780000403C0000403C0000403C0000 801E0000801E0000FFFE0001000F0001000F0001000F00020007800200078002000780040003C0 0E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC04000 7030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C000040 78000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F80000 00780000007C0000407C0000403C0000401C0000401E0000800E000080070001000380020001C0 040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0070001 E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000F8000000F800 0000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E07C0001E03C0001E03C 0001E01E0001E01E0001E00F0001E0070001E0038002E000E0046000781820000FE0001E217D9F 24>III<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C0000 3C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0020 3C00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>I76 DII<001F 800000F0F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E078 0001E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0 F80001F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E 0001C0380000F0F000001F80001C217D9F23>II<001F800000 F0F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E07C0003E0 780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001 F0780001E0780001E07C0003E03C0003C03C0F03C01E1087800E2047000F204F0007A03E0001E0 380000F0F010001FB01000003010000038300000387000003FF000001FE000001FE000000FC000 0007801C297D9F23>II<07E0800C19801007803003 80600180600180E00180E00080E00080E00080F00000F000007800007F00003FF0001FFC000FFE 0003FF00001F800007800003C00003C00001C08001C08001C08001C08001C0C00180C00380E003 00F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F 0030800F0010800F0010800F0010800F0010000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF81FF80FE00F C007C0070003C0020001E0040001F00C0000F0080000781000007C1000003C2000003E4000001E 4000000F8000000F8000000780000003C0000007E0000005E0000009F0000018F8000010780000 207C0000603C0000401E0000801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE 1F1F7F9E22>II91 D<080410082010201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E 001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E 000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00780E00780E00 700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C1C1E381E380C700070 00F000F000F000F000F000F00070007000380138011C020E0C03F010147E9314>I<000380003F 8000038000038000038000038000038000038000038000038000038000038003E380061B801C07 80380380380380700380700380F00380F00380F00380F00380F00380F003807003807003803803 803807801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E0038070038070070070070 0380F00380F00380FFFF80F00000F00000F000007000007000003800801800800C010007060001 F80011147F9314>I<007C00C6018F038F07060700070007000700070007000700FFF007000700 07000700070007000700070007000700070007000700070007000700070007007FF01020809F0E >I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E 380033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C00F01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 FFE7FC16207F9F19>I<1C001E003E001E001C000000000000000000000000000E007E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A1F809E0C> I<00E001F001F001F000E0000000000000000000000000007007F000F000700070007000700070 00700070007000700070007000700070007000700070007000700070007000706070F060F0C061 803F000C28829E0E>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38 000E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E 000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C00F01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC 16147F9319>I<01F800070E001C03803801C03801C07000E07000E0F000F0F000F0F000F0F000 F0F000F0F000F07000E07000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00FE C3800F01C00F00E00E00E00E00F00E00700E00780E00780E00780E00780E00780E00780E00700E 00F00E00E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E 0000FFE000151D7F9319>I<03E0800619801C05803C0780380380780380700380F00380F00380 F00380F00380F00380F003807003807803803803803807801C0B800E138003E380000380000380 000380000380000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E 000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F800D14 7E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E000E000E000E00 0E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC 00007800003800003C00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714 809318>II<3FFF380E200E201C40384078407000E001E0 01C00380078007010E011E011C0338027006700EFFFE10147F9314>II E /Ft 47 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000071C000C060018030038038038 03807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C03803803803801C07000C 0600071C0001F00013227EA018>48 D<008003800F80F380038003800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003800380 07C0FFFE0F217CA018>I<03F0000C1C001007002007804003C04003C08003E0F003E0F801E0F8 01E0F801E02003E00003E00003C00003C0000780000700000E00001C0000180000300000600000 C0000180000100000200200400200800201800603000403FFFC07FFFC0FFFFC013217EA018>I< 03F8000C1E001007002007804007C07807C07803C07807C03807C0000780000780000700000F00 000E0000380003F000001C00000F000007800007800003C00003C00003E02003E07003E0F803E0 F803E0F003C04003C0400780200780100F000C1C0003F00013227EA018>I<000200000600000E 00000E00001E00001E00002E00004E00004E00008E00008E00010E00020E00020E00040E00040E 00080E00100E00100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E00000E 00000E00000E00001F0001FFF015217FA018>I<1000801E07001FFF001FFE001FF80013E00010 000010000010000010000010000010000010F800130E001407001803801003800001C00001C000 01E00001E00001E00001E07001E0F001E0F001E0E001C08001C04003C04003802007001006000C 1C0003F00013227EA018>I<007E0001C1000300800601C00E03C01C03C0180180380000380000 780000700000700000F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0 F001E0F001E07001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013227E A018>I<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000E0C000 60C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018>56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0F001E0F001 E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C00001C00001C00003 80000380300300780700780600700C002018001030000FC00013227EA018>I<00018000000180 00000180000003C0000003C0000003C0000005E0000005E000000DF0000008F0000008F0000010 F800001078000010780000203C0000203C0000203C0000401E0000401E0000401E0000800F0000 800F0000FFFF000100078001000780030007C0020003C0020003C0040003E0040001E0040001E0 0C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C001 70038000F0070000F00E0000701E0000701C0000303C0000303C0000307C000010780000107800 0010F8000000F8000000F8000000F8000000F8000000F8000000F8000000F80000007800000078 0000107C0000103C0000103C0000101C0000201E0000200E000040070000400380008001C00100 00E0020000381C000007E0001C247DA223>67 D69 DI73 D<03FFF0001F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F 00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F 00700F00F80F00F80F00F80E00F01E00401C0020380018700007C00014237EA119>I77 DI80 D82 D<03F0200C0C601802603001 E07000E0600060E00060E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE 000FFF0003FF80003FC00007E00001E00000F00000F0000070800070800070800070800070C000 60C00060E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF8780780786007801840 0780084007800840078008C007800C800780048007800480078004800780040007800000078000 000780000007800000078000000780000007800000078000000780000007800000078000000780 00000780000007800000078000000780000007800000078000000780000007800000078000000F C00003FFFF001E227EA123>I<0FE0001838003C0C003C0E0018070000070000070000070000FF 0007C7001E07003C0700780700700700F00708F00708F00708F00F087817083C23900FC1E01515 7E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E1F000E61C00E80600F00300E00380E003C0E001C0E001E0E001E0E001E 0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C80600C41C0083F0017237FA21B> I<01FE000703000C07801C0780380300780000700000F00000F00000F00000F00000F00000F000 00F000007000007800403800401C00800C010007060001F80012157E9416>I<0000E0000FE000 01E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E001F8E007 04E00C02E01C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E070 00E07800E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C0 3801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000700000780020380020 1C00400E008007030000FC0013157F9416>I<003C00C6018F038F030F07000700070007000700 0700070007000700FFF80700070007000700070007000700070007000700070007000700070007 0007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C07001C07003C0780 3C07803C07803C07801C07001C07000E0E000F1C0019F0001000001000001800001800001FFE00 0FFFC00FFFE03800F0600030400018C00018C00018C000186000306000303800E00E038003FE00 15217F9518>I<0E0000FE00001E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E1F800E60C00E80E00F00700F00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B >I<1C001E003E001E001C00000000000000000000000000000000000E00FE001E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01 C003E003E003E001C00000000000000000000000000000000001E00FE001E000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060 E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E02000E04000E08 000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C00E01E00E00F00E00 F8FFE3FE17237FA21A>I<0E00FE001E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 00FFE00B237FA20E>I<0E1FC07F00FE60E183801E807201C00F003C00E00F003C00E00E003800 E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFE2715 7F942A>I<0E1F80FE60C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC00 0707000C01801800C03800E0700070700070F00078F00078F00078F00078F00078F00078F00078 7000707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00 700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E00 380F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E00000E00 00FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E000E000E00 0E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F000 7F803FE00FF001F8003C801C800C800CC00CC008E018D0308FC00E157E9413>I<020002000200 02000600060006000E001E003E00FFF80E000E000E000E000E000E000E000E000E000E000E000E 040E040E040E040E040E040708030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F0 06017003827800FC7F18157F941B>III121 D E /Fu 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E07800 00E0700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF00000FFF80 0000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880000078800000 78800000788000007880000078C0000078C0000070E00000F0E00000E0F00000E0F80001C0EC00 0380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010 007000100038003C003C003E001C003E001E003E001E0008001E0000001E0000001E0000001E00 000FFE0000FC1E0003E01E000F801E001F001E003E001E003C001E007C001E00F8001E04F8001E 04F8001E04F8003E04F8003E0478003E047C005E043E008F080F0307F003FC03E01E1F7D9E21> 97 D<003F8000E0600380180700040F00041E001E1C003E3C003E7C003E7C0008780000F80000 F80000F80000F80000F80000F80000F80000F80000F800007800007C00007C00003C00011E0001 1E00020F000207000403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE000 0003E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0 000001E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E0078005 E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E0780001E03C0001E03C 0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F81FF20327DB125>I<00 3F800000E0E0000380380007003C000E001E001E001E001C000F003C000F007C000F0078000F80 78000780F8000780F8000780FFFFFF80F8000000F8000000F8000000F8000000F8000000F80000 00780000007C0000003C0000003C0000801E0000800E0001000F0002000780020001C00C0000F0 3000001FC000191F7E9E1D>I<0007E0001C1000383800707C00E07C01E07C01C03803C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0 007FFF007FFF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E 003C001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C00 0F007800070070000780E00009C1C000087F000018000000180000001800000018000000180000 001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000F070000070E000 0038E0000038E0000038E0000038E00000387000007070000070380000E01C0001C00700070001 C01C00003FE0001E2F7E9F21>I<07000F801F801F800F80070000000000000000000000000000 0000000000000000000780FF80FF800F8007800780078007800780078007800780078007800780 07800780078007800780078007800780078007800780078007800FC0FFF8FFF80D307EAF12> 105 D<0780FE001FC000FF83078060F000FF8C03C18078000F9001E2003C0007A001E4003C0007 A000F4001E0007C000F8001E0007C000F8001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E000780 00F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F00007C000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF80 211F7E9E25>I<001FC00000F0780001C01C00070007000F0007801E0003C01C0001C03C0001E0 3C0001E0780000F0780000F0780000F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000 F8F80000F8F80000F8780000F07C0001F03C0001E03C0001E01E0003C01E0003C00F0007800780 0F0001C01C0000F07800001FC0001D1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C038 07C00007C00007C000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000780000780000780000FC000FFFE00FFFE00161F7E 9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F80000 7E00003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC0001C E0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<0040000040000040000040 0000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FFFFE003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C01003C01003C01003C01003C01003C01003C01003C01001C02001E02000E0400078 C0001F00142C7FAB19>I<078000F000FF801FF000FF801FF0000F8001F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078001F000078001F000078001F000038002F00003C004F00001C008F800 007030FF80001FC0FF80211F7E9E25>I E /Fv 5 85 df<000000006000000000007000000000 00F00000000001F00000000001F00000000003F00000000003F00000000007F00000000007F000 0000000FF0000000000FF0000000001BF00000000033F00000000033F00000000063F000000000 63F800000000C1F800000000C1F80000000181F80000000381F80000000301F80000000601F800 00000601F80000000C01F80000000C01F80000001801F80000001801F80000003001F800000060 01F80000006001F8000000C001F8000000C001FC000001FFFFFC000001FFFFFC0000030000FC00 00070000FC0000060000FC00000C0000FC00000C0000FC0000180000FC0000180000FC00003000 00FC0000700000FC0000600000FC0000E00000FC0001E00000FC0003E00000FE000FF00001FE00 FFFE003FFFF0FFFE003FFFF02C327CB135>65 D<000FFFFFFF0000000FFFFFFFC00000003F8007 F00000003F8001F80000003F00007C0000003F00007E0000007F00003E0000007F00001F000000 7E00001F0000007E00001F800000FE00000F800000FE00000F800000FC00000FC00000FC00000F C00001FC00000FC00001FC00000FC00001F800000FC00001F800000FC00003F800000FC00003F8 00001FC00003F000001FC00003F000001FC00007F000001FC00007F000001F800007E000003F80 0007E000003F80000FE000003F80000FE000003F00000FC000007F00000FC000007F00001FC000 007E00001FC00000FE00001F800000FC00001F800000FC00003F800001F800003F800001F00000 3F000003F000003F000007E000007F000007C000007F00000FC000007E00001F8000007E00003F 000000FE00007E000000FE0000F8000000FC0001F0000000FC0007E0000001FC003F800000FFFF FFFE000000FFFFFFF000000032317CB036>68 D<000FFFFFFFFE000FFFFFFFFE00003F8000FE00 003F80003E00003F00001E00003F00001E00007F00000C00007F00000C00007E00000C00007E00 000C0000FE00000C0000FE00000C0000FC00000C0000FC00000C0001FC00001C0001FC00C01800 01F800C0000001F800C0000003F801C0000003F801C0000003F00180000003F00380000007F00F 80000007FFFF80000007FFFF00000007E00F0000000FE0070000000FE0070000000FC006000000 0FC0060000001FC00E0000001FC00E0000001F800C0000001F80000000003F80000000003F8000 0000003F00000000003F00000000007F00000000007F00000000007E00000000007E0000000000 FE0000000000FE0000000000FC0000000000FC0000000001FC00000000FFFFFC000000FFFFFC00 00002F317CB02F>70 D<000FFFFFF800000FFFFFFF0000003F801FC000003F8007E000003F0003 F000003F0001F800007F0000FC00007F0000FC00007E0000FC00007E0000FC0000FE0000FC0000 FE0001FC0000FC0001FC0000FC0001FC0001FC0001F80001FC0003F80001F80003F00001F80007 E00003F80007E00003F8000F800003F0003F000003F0007E000007F003F8000007FFFFE0000007 FFFF80000007E007C000000FE003F000000FE001F000000FC000F800000FC000F800001FC000FC 00001FC000FC00001F8000FC00001F8000FC00003F8001FC00003F8001FC00003F0001FC00003F 0001FC00007F0003F800007F0003F800007E0003F800007E0003F80600FE0003F80E00FE0003F8 0C00FC0003F80C00FC0003F81C01FC0001F838FFFFF000FC70FFFFF0007FE0000000001F802F32 7CB034>82 D<07FFFFFFFFF807FFFFFFFFF80FE007F001F80F8007F000F80E0007E000701E0007 E000701C000FE0007018000FE0007038000FC0007038000FC0007030001FC0006070001FC00060 60001F80006060001F80006060003F8000E0E0003F8000C000003F00000000003F00000000007F 00000000007F00000000007E00000000007E0000000000FE0000000000FE0000000000FC000000 0000FC0000000001FC0000000001FC0000000001F80000000001F80000000003F80000000003F8 0000000003F00000000003F00000000007F00000000007F00000000007E00000000007E0000000 000FE0000000000FE0000000000FC0000000000FC0000000001FC0000000001FC0000000001F80 000000003F80000000007FC00000007FFFFFC000007FFFFFC000002D3174B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300 TeXDict begin @letter /letter where {pop letter} if %%EndSetup %%Page: 0 1 bop 795 908 a Fv(D)26 b(R)g(A)f(F)h(T)225 999 y Fu(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1194 y Ft(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)833 1320 y(July)h(24,)g(1995)77 1378 y(This)h(w)o(ork)f(w)o(as)h(supp)q(orted)g (in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h(ASC-9310330,)i (the)192 1436 y(National)d(Science)f(F)l(oundation)i(Science)e(and)i(T)l(ec)o (hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1494 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h(Commission)e(of)j(the)f(Europ)q(ean)i (Comm)o(unit)n(y)654 1552 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 bop 166 45 a Fs(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g(run)g(of)g (a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF)f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 bop 75 356 a Fr(Chapter)32 b(1)75 564 y Fq(Dynamic)38 b(Pro)s(cesses)75 805 y Fp(1.1)59 b(Intro)r(duction)75 906 y Fs(The)15 b(MPI-1)g(message)g (passing)g(library)h(allo)o(ws)f(pro)q(cesses)h(in)g(a)f(parallel)h(program)e (to)h(comm)o(unicate)75 963 y(with)21 b(one)f(another.)34 b(MPI-1)20 b(sp)q(eci\014es)i(neither)g(ho)o(w)d(the)i(pro)q(cesses)f(are)g(created,)h (nor)f(ho)o(w)g(they)75 1019 y(establish)f(comm)o(unication.)30 b(Moreo)o(v)o(er,)17 b(an)i(MPI-1)f(application)i(is)e(static,)h(that)e(is,)i (no)g(pro)q(cesses)75 1076 y(can)c(b)q(e)h(added)g(to)f(or)f(deleted)j(from)d (an)h(application)i(while)g(it)e(is)h(running.)166 1132 y(MPI)d(users)g(ha)o (v)o(e)g(ask)o(ed)g(that)g(the)g(MPI-1)g(mo)q(del)h(b)q(e)g(relaxed)g(to)f (allo)o(w)g(dynamic)h(pro)q(cess)g(man-)75 1188 y(agemen)o(t.)22 b(The)17 b(main)g(imp)q(etus)g(comes)f(from)f(the)i(PVM)f([)p Fo(?)p Fs(])22 b(researc)o(h)16 b(e\013ort,)f(whic)o(h)i(has)f(pro)o(vided)75 1245 y(a)g(w)o(ealth)h(of)f(exp)q(erience)j(with)e(dynamic)h(pro)q(cess)f (managemen)o(t)f(that)g(illustrates)h(its)g(b)q(ene\014ts)h(and)75 1301 y(p)q(oten)o(tial)e(pitfalls.)22 b(The)15 b(reasons)g(for)g(adding)h (dynamic)g(pro)q(cess)g(managemen)o(t)e(are)h(b)q(oth)g(tec)o(hnical)75 1358 y(and)g(practical.)143 1464 y Fn(\017)23 b Fs(W)l(orkstation)d(net)o(w)o (ork)f(users)i(migrating)g(from)g(PVM)f(to)g(MPI)h(are)g(accustomed)g(to)f (using)189 1521 y(PVM's)14 b(capabilities)j(for)e(pro)q(cess)g(and)g (resource)h(managemen)o(t.)j(While)d(relativ)o(ely)g(few)f(PVM)189 1577 y(applications)d(are)e(truly)h(dynamic)h(or)e(require)h(features)g(not)f (in)h(MPI,)g(the)f(lac)o(k)h(of)f(these)h(features)189 1633 y(is)k(a)g(practical)h(stum)o(bling)g(blo)q(c)o(k)g(to)f(migration.)143 1727 y Fn(\017)23 b Fs(Imp)q(ortan)o(t)16 b(classes)h(of)g(message)f(passing) h(applications,)i(suc)o(h)e(as)f(clien)o(t-serv)o(er)i(systems)f(and)189 1784 y(task)d(farming)h(jobs,)g(require)h(dynamic)g(pro)q(cess)f(con)o(trol.) 143 1878 y Fn(\017)23 b Fs(With)16 b(dynamic)h(resource)g(and)f(pro)q(cess)h (managemen)o(t)e(extensions,)i(it)g(w)o(ould)g(b)q(e)g(p)q(ossible)h(to)189 1934 y(write)d(ma)s(jor)f(parts)g(of)h(the)g(parallel)i(programming)e(en)o (vironmen)o(t)g(in)h(MPI)f(itself.)143 2028 y Fn(\017)23 b Fs(The)14 b(abilit)o(y)i(to)d(write)h(fault)h(toleran)o(t)e(applications)j (is)f(imp)q(ortan)o(t)f(in)h(unstable)g(en)o(vironmen)o(ts)189 2084 y(and)23 b(for)f(commercial)i(applications.)45 b(MPI-1)23 b(do)q(es)g(not)f(pro)o(vide)i(mec)o(hanisms)f(for)g(build-)189 2141 y(ing)18 b(fault-toleran)o(t)f(applications.)30 b(The)18 b(mec)o(hanisms)g(required)h(to)e(supp)q(ort)h(fault)g(tolerance)189 2197 y(largely)d(o)o(v)o(erlap)g(with)h(those)f(needed)h(to)f(supp)q(ort)g (dynamic)h(pro)q(cess)g(managemen)o(t.)166 2303 y(While)e(dynamic)g(pro)q (cess)f(managemen)o(t)f(is)i(essen)o(tial,)g(it)f(is)g(imp)q(ortan)o(t)g(not) f(to)h(compromise)g(the)75 2360 y(p)q(ortabilit)o(y)j(or)f(p)q(erformance)g (of)g(MPI.)g(In)h(particular:)143 2454 y Fn(\017)23 b Fs(The)f(MPI-2)f (dynamic)i(pro)q(cess)e(managemen)o(t)g(mo)q(del)i(m)o(ust)e(apply)h(to)f (the)h(v)m(ast)f(ma)s(jorit)o(y)189 2510 y(of)e(curren)o(t)h(parallel)h(en)o (vironmen)o(ts.)34 b(These)20 b(include)j(ev)o(erything)d(from)f(tigh)o(tly)h (in)o(tegrated)189 2567 y(MPPs)d(suc)o(h)i(as)f(the)g(In)o(tel)h(P)o(aragon)e (and)h(the)g(Meik)o(o)g(CS-2)h(to)e(heterogeneous)h(net)o(w)o(orks)f(of)189 2623 y(w)o(orkstations.)964 2828 y(1)p eop %%Page: 2 4 bop 75 -100 a Fs(2)951 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)143 45 y Fn(\017)23 b Fs(MPI)16 b(m)o(ust)g(not)g(tak)o(e)g(o) o(v)o(er)f(op)q(erating)i(system)f(resp)q(onsibiliti)q(es.)26 b(It)17 b(should)g(instead)g(pro)o(vide)189 102 y(a)e(clean)h(in)o(terface)f (b)q(et)o(w)o(een)h(an)f(application)i(and)e(system)g(soft)o(w)o(are.)143 195 y Fn(\017)23 b Fs(MPI)17 b(m)o(ust)g(con)o(tin)o(ue)h(to)f(guaran)o(tee)g (comm)o(unication)h(determinism,)h(i.e.,)e(dynamic)i(pro)q(cess)189 252 y(managemen)o(t)14 b(m)o(ust)h(not)f(in)o(tro)q(duce)j(una)o(v)o(oidable) f(race)f(conditions.)143 346 y Fn(\017)23 b Fs(MPI)15 b(m)o(ust)f(not)h(con)o (tain)h(features)e(that)h(compromise)g(p)q(erformance.)143 439 y Fn(\017)23 b Fs(MPI-1)16 b(programs)f(m)o(ust)h(w)o(ork)g(under)h (MPI-2,)f(i.e.,)g(the)h(MPI-1)f(static)g(pro)q(cess)h(mo)q(del)h(m)o(ust)189 496 y(b)q(e)e(a)e(sp)q(ecial)j(case)f(of)e(the)i(MPI-2)f(dynamic)h(mo)q(del.) 166 590 y(The)e(MPI)g(dynamic)h(pro)q(cess)g(managemen)o(t)e(mo)q(del)i (address)f(these)h(issues)g(in)f(t)o(w)o(o)f(w)o(a)o(ys.)19 b(First,)75 646 y(it)13 b(separates)f(the)g(run)o(time)h(en)o(vironmen)o(t)g (of)f(a)h(parallel)h(program)d(in)o(to)i(three)f(logical)i(parts:)k(resource) 75 703 y(managemen)o(t,)c(pro)q(cess)i(managemen)o(t,)e(and)h(comm)o (unication.)21 b(MPI-1)15 b(addresses)h(the)f(comm)o(unica-)75 759 y(tion)20 b(comp)q(onen)o(t.)33 b(MPI-2)20 b(pro)o(vides)g(an)g(in)o (terface)g(b)q(et)o(w)o(een)g(an)f(MPI)h(application)h(and)f(logically)75 816 y(external)c(resource)f(and)g(pro)q(cess)g(managers.)k(The)d(resource)f (and)g(pro)q(cess)g(managers)g(ma)o(y)f(b)q(e)i(sepa-)75 872 y(rate)g(programs,)f(the)i(same)g(program,)e(part)h(of)g(the)h(op)q(erating)g (system,)f(or)g(ev)o(en)h(con)o(tained)h(within)75 928 y(the)i(MPI)f (implemen)o(tation)i(itself,)h(but)d(their)h(functionalit)o(y)h(is)f(w)o (ell-de\014ned)j(and)c(exists)h(in)h(most)75 985 y(curren)o(t)15 b(message)g(passing)g(en)o(vironmen)o(ts,)g(includin)q(g)i(PVM.)166 1041 y(Second,)g(MPI-2)g(do)q(es)g(not)f(c)o(hange)h(the)g(concept)g(of)f (comm)o(unicator.)24 b(Once)18 b(a)e(comm)o(unicator)75 1098 y(is)h(built,)g(it)g(b)q(eha)o(v)o(es)f(as)g(sp)q(eci\014ed)j(in)e(MPI-1.)23 b(A)16 b(comm)o(unicator)g(is)h(nev)o(er)f(c)o(hanged)h(once)f(created,)75 1154 y(and)f(it)h(is)g(alw)o(a)o(ys)e(created)h(using)h(deterministic)h (collectiv)o(e)g(seman)o(tics.)75 1297 y Fp(1.2)59 b(The)19 b(MPI)h(Dynamic)f(Pro)r(cess)g(Mo)r(del)75 1401 y Fl(1.2.1)49 b(Comp)q(onents)75 1486 y Fs(The)13 b(MPI)g(dynamic)g(pro)q(cess)g(mo)q(del)h (separates)e(the)h(functions)g(of)g Fk(r)n(esour)n(c)n(e)g(manager)p Fs(,)g Fk(pr)n(o)n(c)n(ess)g(man-)75 1543 y(ager)i Fs(and)h Fk(message-p)n(assing)e(libr)n(ary)p Fs(.)75 1663 y Fj(Resource)i(Manager)44 b Fs(The)15 b Fk(r)n(esour)n(c)n(e)g(manager)20 b Fs(is)15 b(the)f(part)g(of)g(the)h(system)f(that)g(con)o(trols)g(resources)75 1719 y(and)i(allo)q(cates)g(them)g(to)f(an)h(application.)23 b(It)16 b(decides)h(when)f(a)g(job)f(will)j(run)e(and)g(whic)o(h)g(pro)q (cessors)75 1776 y(will)21 b(b)q(e)f(allo)q(cated)g(to)f(it)g(when)h(it)g(do) q(es)f(run.)33 b(In)20 b(some)f(en)o(vironmen)o(ts)g(the)h(resource)f (manager)f(is)75 1832 y(a)f(sophisticated)h(batc)o(h)f(queueing)h(system;)f (in)h(others)f(it)g(is)g(the)h(user)f(him/herself,)h(who)f(can)g(start)75 1889 y(jobs)f(on)g(a)f(net)o(w)o(ork)g(whenev)o(er)i(and)f(wherev)o(er)g (he/she)g(lik)o(es.)23 b(Logically)l(,)18 b(the)e(resource)g(manager)f(is)75 1945 y(external)h(to)e(an)h(application)i(ev)o(en)f(if)f(it)h(is)g(implemen)o (ted)g(in)o(ternally)l(.)166 2002 y(In)g(con)o(trast,)f(the)h(PVM)f(mo)q(del) i(of)e(a)h(resource)f(manager)g(is)i(in)o(ternal)f(to)g(an)f(application,)j (ev)o(en)75 2058 y(though)g(it)g(ma)o(y)g(b)q(e)h(implemen)o(ted)g (externally)h(through)d(the)h(hoster)g(and)g(task)o(er)g(in)o(terfaces.)28 b(This)75 2115 y(mak)o(es)15 b(it)g(di\016cult)i(for)d(PVM)h(applications)i (to)d(in)o(teract)i(w)o(ell)g(with)f(external)h(resource)f(managers.)75 2235 y Fj(Pro)q(cess)d(Manager)45 b Fs(Once)12 b(pro)q(cessors)f(ha)o(v)o(e)f (b)q(een)i(allo)q(cated)g(to)f(a)g(program,)f(user)h(pro)q(cesses)h(m)o(ust)e (b)q(e)75 2291 y(started)g(on)g(those)g(pro)q(cessors,)h(and)f(managed)h (after)e(startup.)18 b(By)10 b(\\managed")g(w)o(e)g(mean)g(that)g(signals)75 2348 y(m)o(ust)15 b(b)q(e)h(deliv)o(erable,)h(that)e Fi(stdin)p Fs(,)f Fi(stdout)p Fs(,)g(and)h Fi(stderr)g Fs(m)o(ust)g(b)q(e)h(handled)g (in)h(some)e(reasonable)75 2404 y(w)o(a)o(y)l(,)j(and)g(that)g(orderly)h (termination)f(can)h(b)q(e)g(guaran)o(teed.)29 b(A)18 b(minimal)i(example)f (is)g Fi(rsh)p Fs(,)f(whic)o(h)75 2460 y(starts)11 b(pro)q(cesses)i(and)g (reroutes)f Fi(stdin)p Fs(,)g Fi(stdout)p Fs(,)g(and)h Fi(stderr)e Fs(bac)o(k)i(to)f(the)g(originating)i(pro)q(cess.)19 b(A)75 2517 y(more)14 b(complex)i(example)g(is)f(giv)o(en)g(b)o(y)g Fi(poe)f Fs(on)h(the)g(IBM)g(SP2)g(or)f Fi(prun)g Fs(on)h(the)g(Meik)o(o)g (CS-2,)f(whic)o(h)75 2573 y(start)j(pro)q(cesses)i(on)f(pro)q(cessors)g(giv)o (en)h(to)f(them)g(b)o(y)g(the)g(job)h(sc)o(heduler)g(and)g(manage)f(them)g (un)o(til)75 2630 y(they)g(are)f(\014nished.)28 b(In)19 b(a)e(tigh)o(tly)h (in)o(tegrated)f(parallel)i(computer,)f(pro)q(cess)g(managemen)o(t)e(ma)o(y)h (b)q(e)75 2686 y(done)f(en)o(tirely)g(b)o(y)f(the)g(op)q(erating)h(system.) 1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 bop 75 -100 a Fm(1.2.)34 b(THE)15 b(MPI)g(D)o(YNAMIC)g(PR)o(OCESS)h(MODEL)813 b Fs(3)166 45 y(In)21 b(some)f(cases)g(the)h(situation)g(is)g(m)o(uddied)h(b) q(ecause)f(a)f(single)i(piece)g(of)e(soft)o(w)o(are)e(com)o(bines)75 102 y(the)i(functions)g(of)f(resource)g(manager)g(and)h(pro)q(cess)f (manager.)32 b(Examples)20 b(of)f(this)h(approac)o(h)f(are)75 158 y(the)i(batc)o(h)f(queueing)j(systems)d(suc)o(h)h(as)g(Condor,)g(DQS,)g (and)g(LoadLev)o(eler.)38 b(Nonetheless,)22 b(it)f(is)75 214 y(con)o(v)o(enien)o(t)f(to)e(consider)i(resource)f(and)g(pro)q(cess)g (managemen)o(t)g(separately)l(,)h(since)g(although)f(they)75 271 y(in)o(teract,)c(they)g(are)g(separate)g(functions)g(that)g(can)g(b)q(e)h (indep)q(enden)o(tly)i(mo)q(di\014ed.)75 391 y Fj(Message-P)o(assing)g(Lib)o (ra)o(ry)44 b Fs(By)17 b(the)g Fk(message-p)n(assing)f(libr)n(ary)k Fs(w)o(e)d(mean)f(the)h(library)h(used)f(b)o(y)g(the)75 447 y(application)f(program)e(for)g(its)h(in)o(terpro)q(cess)h(comm)o(unication.) k(Programs)14 b(con)o(taining)h(only)h(calls)g(to)75 504 y(a)i (message-passing)h(library)g(can)f(b)q(e)h(extremely)g(p)q(ortable,)g(since)h (they)e(\014t)g(cleanly)i(in)o(to)f(a)f(v)m(ariet)o(y)75 560 y(of)13 b(job)h(sc)o(heduler{pro)q(cess)h(manager)e(en)o(vironmen)o(ts.)20 b(MPI-1)14 b(de\014nes)g(a)g(standard)f(message)h(passing)75 617 y(library)l(.)75 739 y Fl(1.2.2)49 b(Interaction)16 b(of)h(User,)d (Application)k(and)f(Runtime)f(Environment)375 1576 y @beginspecial 71 @llx 464 @lly 494 @urx 728 @ury 2880 @rwi @setspecial %%BeginDocument: fig1.eps 1 setlinejoin /M { moveto } bind def /S { show } bind def /R { rmoveto } bind def /L { lineto } bind def /B { newpath 0 0 M 0 1 L 1 1 L 1 0 L closepath } bind def /CS { closepath stroke } bind def /S { /fixwidth exch def dup length /nchars exch def dup stringwidth pop fixwidth exch sub nchars div exch 0 exch ashow } def /bwproc { rgbproc dup length 3 idiv string 0 3 0 5 -1 roll { add 2 1 roll 1 sub dup 0 eq { pop 3 idiv 3 -1 roll dup 4 -1 roll dup 3 1 roll 5 -1 roll put 1 add 3 0 } { 2 1 roll } ifelse } forall pop pop pop } def systemdict /colorimage known not { /colorimage { pop pop /rgbproc exch def { bwproc } image } def } if 1 1 scale 0 setlinewidth /drawtri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def 0 setgray newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath stroke } bind def /filltri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath fill } bind def /cliptri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def 0 setgray newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath clip } bind def /imgscanrgb { gsave translate /scandy exch def /scandx exch def /istr scandx 3 mul string def scandx scandy scale scandx scandy 8 [scandx 0 0 scandy neg 0 scandy] {currentfile istr readhexstring pop} false 3 colorimage grestore } bind def /imgscanbw { gsave translate /scandy exch def /scandx exch def /istr scandx string def scandx scandy scale scandx scandy 8 [scandx 0 0 scandy neg 0 scandy] {currentfile istr readhexstring pop} image grestore } bind def /showcaseisoencoding [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /space /exclam /quotedbl /numbersign /dollar /percent /ampersand /quoteright /parenleft /parenright /asterisk /plus /comma /minus /period /slash /zero /one /two /three /four /five /six /seven /eight /nine /colon /semicolon /less /equal /greater /question /at /A /B /C /D /E /F /G /H /I /J /K /L /M /N /O /P /Q /R /S /T /U /V /W /X /Y /Z /bracketleft /backslash /bracketright /asciicircum /underscore /quoteleft /a /b /c /d /e /f /g /h /i /j /k /l /m /n /o /p /q /r /s /t /u /v /w /x /y /z /braceleft /bar /braceright /asciitilde /guilsinglright /fraction /florin /quotesingle /quotedblleft /guilsinglleft /fi /fl /endash /dagger /daggerdbl /bullet /quotesinglbase /quotedblbase /quotedblright /ellipsis /trademark /dotlessi /grave /acute /circumflex /tilde /macron /breve /dotaccent /dieresis /perthousand /ring /cedilla /Ydieresis /hungarumlaut /ogonek /caron /emdash /exclamdown /cent /sterling /currency /yen /brokenbar /section /dieresis /copyright /ordfeminine /guillemotleft /logicalnot /hyphen /registered /macron /degree /plusminus /twosuperior /threesuperior /acute /mu /paragraph /periodcentered /cedilla /onesuperior /ordmasculine /guillemotright /onequarter /onehalf /threequarters /questiondown /Agrave /Aacute /Acircumflex /Atilde /Adieresis /Aring /AE /Ccedilla /Egrave /Eacute /Ecircumflex /Edieresis /Igrave /Iacute /Icircumflex /Idieresis /Eth /Ntilde /Ograve /Oacute /Ocircumflex /Otilde /Odieresis /multiply /Oslash /Ugrave /Uacute /Ucircumflex /Udieresis /Yacute /Thorn /germandbls /agrave /aacute /acircumflex /atilde /adieresis /aring /ae /ccedilla /egrave /eacute /ecircumflex /edieresis /igrave /iacute /icircumflex /idieresis /eth /ntilde /ograve /oacute /ocircumflex /otilde /odieresis /divide /oslash /ugrave /uacute /ucircumflex /udieresis /yacute /thorn /ydieresis ] def /showcasedingbatencoding [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /a1 /a2 /a202 /a3 /a4 /a5 /a119 /a118 /a117 /a11 /a12 /a13 /a14 /a15 /a16 /a105 /a17 /a18 /a19 /a20 /a21 /a22 /a23 /a24 /a25 /a26 /a27 /a28 /a6 /a7 /a8 /a9 /a10 /a29 /a30 /a31 /a32 /a33 /a34 /a35 /a36 /a37 /a38 /a39 /a40 /a41 /a42 /a43 /a44 /a45 /a46 /a47 /a48 /a49 /a50 /a51 /a52 /a53 /a54 /a55 /a56 /a57 /a58 /a59 /a60 /a61 /a62 /a63 /a64 /a65 /a66 /a67 /a68 /a69 /a70 /a71 /a72 /a73 /a74 /a203 /a75 /a204 /a76 /a77 /a78 /a79 /a81 /a82 /a83 /a84 /a97 /a98 /a99 /a100 /.notdef /a205 /a85 /a206 /a86 /a87 /a88 /a89 /a90 /a91 /a92 /a93 /a94 /a95 /a96 /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /a101 /a102 /a103 /a104 /a106 /a107 /a108 /a112 /a111 /a110 /a109 /a120 /a121 /a122 /a123 /a124 /a125 /a126 /a127 /a128 /a129 /a130 /a131 /a132 /a133 /a134 /a135 /a136 /a137 /a138 /a139 /a140 /a141 /a142 /a143 /a144 /a145 /a146 /a147 /a148 /a149 /a150 /a151 /a152 /a153 /a154 /a155 /a156 /a157 /a158 /a159 /a160 /a161 /a163 /a164 /a196 /a165 /a192 /a166 /a167 /a168 /a169 /a170 /a171 /a172 /a173 /a162 /a174 /a175 /a176 /a177 /a178 /a179 /a193 /a180 /a199 /a181 /a200 /a182 /.notdef /a201 /a183 /a184 /a197 /a185 /a194 /a198 /a186 /a195 /a187 /a188 /a189 /a190 /a191 /.notdef ] def /Courier-Bold findfont dup length dict begin {1 index /FID ne {def} {pop pop} ifelse} forall /Encoding showcaseisoencoding def currentdict end /Courier-Bold-SHOWISO exch definefont pop /newfont 10 dict def newfont begin /FontType 3 def /FontMatrix [1 0 0 1 0 0] def /FontBBox [0 0 1 1] def /Encoding 256 array def 0 1 255 {Encoding exch /.notdef put} for /CharProcs 1 dict def CharProcs begin /.notdef {} def end /BuildChar { 1 0 0 0 1 1 setcachedevice exch begin Encoding exch get CharProcs exch get end exec } def end /PatternFont newfont definefont pop /#copies 1 def gsave gsave 0 0 0 setrgbcolor 1.000000 setlinewidth newpath 214 599.5 M 208.75 600.25 L 205 602.5 L 202.75 606.25 L 202 611.5 L 202 611.5 L 202 612.373 L 202 614.496 L 202 617.692 L 202 621.781 L 202 626.585 L 202 631.926 L 202 637.624 L 202 643.5 L 202 649.376 L 202 655.074 L 202 660.415 L 202 665.219 L 202 669.308 L 202 672.504 L 202 674.627 L 202 675.5 L 202 675.5 L 202.75 680.75 L 205 684.5 L 208.75 686.75 L 214 687.5 L 214 687.5 L 215.535 687.5 L 219.527 687.5 L 225.624 687.5 L 233.469 687.5 L 242.708 687.5 L 252.988 687.5 L 263.954 687.5 L 275.25 687.5 L 286.523 687.5 L 297.418 687.5 L 307.581 687.5 L 316.656 687.5 L 324.291 687.5 L 330.129 687.5 L 333.817 687.5 L 335 687.5 L 335 687.5 L 340.25 686.75 L 344 684.5 L 346.25 680.75 L 347 675.5 L 347 675.5 L 347 674.627 L 347 672.504 L 347 669.308 L 347 665.219 L 347 660.415 L 347 655.074 L 347 649.376 L 347 643.5 L 347 637.624 L 347 631.926 L 347 626.585 L 347 621.781 L 347 617.692 L 347 614.496 L 347 612.373 L 347 611.5 L 347 611.5 L 346.25 606.25 L 344 602.5 L 340.25 600.25 L 335 599.5 L 335 599.5 L 333.487 599.5 L 329.555 599.5 L 323.548 599.5 L 315.812 599.5 L 306.694 599.5 L 296.539 599.5 L 285.692 599.5 L 274.5 599.5 L 263.308 599.5 L 252.461 599.5 L 242.306 599.5 L 233.187 599.5 L 225.452 599.5 L 219.445 599.5 L 215.513 599.5 L 214 599.5 L closepath stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 239.9 629.5] concat newpath 0 0 M 0 28 L 69.2 28 L 69.2 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 237.9 627.5 M 237.9 659.5 L 311.1 659.5 L 311.1 627.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 239.9 629.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (Resource) 67.2 S 0 3.5 M (Manager) 58.8 S setmatrix grestore gsave 0 0 0 setrgbcolor 1.000000 setlinewidth newpath 83.5 464.5 M 78.25 465.25 L 74.5 467.5 L 72.25 471.25 L 71.5 476.5 L 71.5 476.5 L 71.5 477.373 L 71.5 479.496 L 71.5 482.692 L 71.5 486.781 L 71.5 491.585 L 71.5 496.926 L 71.5 502.624 L 71.5 508.5 L 71.5 514.376 L 71.5 520.074 L 71.5 525.415 L 71.5 530.219 L 71.5 534.308 L 71.5 537.504 L 71.5 539.627 L 71.5 540.5 L 71.5 540.5 L 72.25 545.75 L 74.5 549.5 L 78.25 551.75 L 83.5 552.5 L 83.5 552.5 L 85.0347 552.5 L 89.0273 552.5 L 95.1235 552.5 L 102.969 552.5 L 112.208 552.5 L 122.488 552.5 L 133.454 552.5 L 144.75 552.5 L 156.023 552.5 L 166.918 552.5 L 177.081 552.5 L 186.156 552.5 L 193.791 552.5 L 199.629 552.5 L 203.317 552.5 L 204.5 552.5 L 204.5 552.5 L 209.75 551.75 L 213.5 549.5 L 215.75 545.75 L 216.5 540.5 L 216.5 540.5 L 216.5 539.627 L 216.5 537.504 L 216.5 534.308 L 216.5 530.219 L 216.5 525.415 L 216.5 520.074 L 216.5 514.376 L 216.5 508.5 L 216.5 502.624 L 216.5 496.926 L 216.5 491.585 L 216.5 486.781 L 216.5 482.692 L 216.5 479.496 L 216.5 477.373 L 216.5 476.5 L 216.5 476.5 L 215.75 471.25 L 213.5 467.5 L 209.75 465.25 L 204.5 464.5 L 204.5 464.5 L 202.987 464.5 L 199.055 464.5 L 193.048 464.5 L 185.312 464.5 L 176.194 464.5 L 166.039 464.5 L 155.192 464.5 L 144 464.5 L 132.808 464.5 L 121.961 464.5 L 111.806 464.5 L 102.688 464.5 L 94.9521 464.5 L 88.9453 464.5 L 85.0127 464.5 L 83.5 464.5 L closepath stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 109.4 494.5] concat newpath 0 0 M 0 28 L 69.2 28 L 69.2 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 107.4 492.5 M 107.4 524.5 L 180.6 524.5 L 180.6 492.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 109.4 494.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (Process ) 67.2 S 0 3.5 M (Manager) 58.8 S setmatrix grestore gsave 0 0 0 setrgbcolor 1.000000 setlinewidth newpath 340 464.5 M 334.75 465.25 L 331 467.5 L 328.75 471.25 L 328 476.5 L 328 476.5 L 328 477.373 L 328 479.496 L 328 482.692 L 328 486.781 L 328 491.585 L 328 496.926 L 328 502.624 L 328 508.5 L 328 514.376 L 328 520.074 L 328 525.415 L 328 530.219 L 328 534.308 L 328 537.504 L 328 539.627 L 328 540.5 L 328 540.5 L 328.75 545.75 L 331 549.5 L 334.75 551.75 L 340 552.5 L 340 552.5 L 341.535 552.5 L 345.527 552.5 L 351.624 552.5 L 359.469 552.5 L 368.708 552.5 L 378.988 552.5 L 389.954 552.5 L 401.25 552.5 L 412.523 552.5 L 423.418 552.5 L 433.581 552.5 L 442.656 552.5 L 450.291 552.5 L 456.129 552.5 L 459.817 552.5 L 461 552.5 L 461 552.5 L 466.25 551.75 L 470 549.5 L 472.25 545.75 L 473 540.5 L 473 540.5 L 473 539.627 L 473 537.504 L 473 534.308 L 473 530.219 L 473 525.415 L 473 520.074 L 473 514.376 L 473 508.5 L 473 502.624 L 473 496.926 L 473 491.585 L 473 486.781 L 473 482.692 L 473 479.496 L 473 477.373 L 473 476.5 L 473 476.5 L 472.25 471.25 L 470 467.5 L 466.25 465.25 L 461 464.5 L 461 464.5 L 459.487 464.5 L 455.555 464.5 L 449.548 464.5 L 441.812 464.5 L 432.694 464.5 L 422.539 464.5 L 411.692 464.5 L 400.5 464.5 L 389.308 464.5 L 378.461 464.5 L 368.306 464.5 L 359.187 464.5 L 351.452 464.5 L 345.445 464.5 L 341.513 464.5 L 340 464.5 L closepath stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 352.3 486.5] concat newpath 0 0 M 0 42 L 94.4 42 L 94.4 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 350.3 484.5 M 350.3 530.5 L 448.7 530.5 L 448.7 484.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 352.3 486.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 31.5 M (Application) 92.4 S 0 17.5 M (\(Message) 67.2 S 0 3.5 M ( Passing\)) 75.6 S setmatrix grestore gsave 1.000000 setlinewidth matrix currentmatrix [92.7699 0 0 92.7699 135 576] concat 0 0 0 setrgbcolor newpath 0 0 1 116.565 116.565 arc setmatrix stroke grestore gsave 1.000000 setlinewidth matrix currentmatrix [62.516 0 0 62.516 135 630] concat 0 0 0 setrgbcolor newpath 0 0 1 331.699 331.699 arc setmatrix stroke grestore gsave 0 0 0 setrgbcolor 1.000000 setlinewidth newpath 488.024 707.855 M 486.739 709.928 L 485.304 711.944 L 483.721 713.889 L 481.997 715.747 L 480.138 717.503 L 478.154 719.143 L 476.055 720.654 L 473.855 722.024 L 471.569 723.244 L 469.211 724.306 L 466.799 725.205 L 464.348 725.937 L 461.877 726.501 L 459.401 726.899 L 456.937 727.134 L 454.5 727.21 L 454.5 727.21 L 452.063 727.134 L 449.599 726.899 L 447.123 726.501 L 444.652 725.937 L 442.201 725.205 L 439.789 724.306 L 437.431 723.244 L 435.145 722.024 L 432.945 720.654 L 430.846 719.143 L 428.862 717.503 L 427.003 715.747 L 425.279 713.889 L 423.696 711.944 L 422.261 709.928 L 420.976 707.855 L 420.976 707.855 L 419.824 705.706 L 418.795 703.455 L 417.902 701.112 L 417.155 698.69 L 416.563 696.202 L 416.135 693.663 L 415.876 691.09 L 415.79 688.5 L 415.876 685.91 L 416.135 683.337 L 416.563 680.798 L 417.155 678.31 L 417.902 675.888 L 418.795 673.545 L 419.824 671.294 L 420.976 669.145 L 420.976 669.145 L 422.261 667.072 L 423.696 665.056 L 425.279 663.111 L 427.003 661.253 L 428.862 659.497 L 430.846 657.857 L 432.945 656.346 L 435.145 654.976 L 437.431 653.756 L 439.789 652.694 L 442.201 651.795 L 444.652 651.063 L 447.123 650.499 L 449.599 650.101 L 452.063 649.866 L 454.5 649.79 L 454.5 649.79 L 456.937 649.866 L 459.401 650.101 L 461.877 650.499 L 464.348 651.063 L 466.799 651.795 L 469.211 652.694 L 471.569 653.756 L 473.855 654.976 L 476.055 656.346 L 478.154 657.857 L 480.138 659.497 L 481.997 661.253 L 483.721 663.111 L 485.304 665.056 L 486.739 667.072 L 488.024 669.145 L 488.024 669.145 L 489.176 671.294 L 490.205 673.545 L 491.098 675.888 L 491.845 678.311 L 492.437 680.798 L 492.865 683.337 L 493.124 685.91 L 493.21 688.5 L 493.124 691.09 L 492.865 693.663 L 492.437 696.202 L 491.845 698.69 L 491.098 701.112 L 490.205 703.455 L 489.176 705.706 L 488.024 707.855 L closepath stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 436.7 681.5] concat newpath 0 0 M 0 14 L 35.6 14 L 35.6 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 434.7 679.5 M 434.7 697.5 L 474.3 697.5 L 474.3 679.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 436.7 681.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 3.5 M (User) 33.6 S setmatrix grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 350.058 661.164 translate -160.346 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 414 684 M 367.951 667.554 L stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 219.5 517.5 translate 180 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore gsave 0 setlinejoin 325 517.5 translate 0 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 238.5 517.5 M 306 517.5 L stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 138.567 553.141 translate -158.962 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore gsave 0 setlinejoin 257.433 598.859 translate 21.0375 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 156.3 559.962 M 239.7 592.038 L stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 396.923 553.115 translate -22.6199 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore gsave 0 setlinejoin 287.077 598.885 translate 157.38 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 379.385 560.423 M 304.615 591.577 L stroke grestore grestore showpage %%EndDocument @endspecial 470 1674 a Fs(Figure)f(1.1:)k(Structure)d(of)e(the)i(Run)o(time) g(En)o(vironmen)o(t)75 1780 y(The)f(starting)f(p)q(oin)o(t)h(for)f(dynamic)h (resource)g(and)f(pro)q(cess)h(managemen)o(t)f(is)h(the)f(resource)h (manager.)75 1836 y(It)k(is)h(the)g(resource)f(manager)g(that)g(allo)q (cates,)h(implicitly)j(or)c(explicitly)l(,)k(resources)c(necessary)h(for)75 1893 y(running)f(pro)q(cesses.)29 b(The)18 b(\\resource)g(manager")f(ma)o(y)h (b)q(e)h(an)o(ything)f(from)f(a)h(sophisticated)h(batc)o(h)75 1949 y(sc)o(heduling)f(system,)d(to)g(a)h(\014le)h(con)o(taining)g(a)e(list)i (of)e(mac)o(hines)i(on)f(whic)o(h)h(an)e(application)j(can)e(run,)75 2006 y(to)f(the)g(user)g(him/herself.)75 2126 y Fj(Acquiring)j(Resources)47 b Fs(In)18 b(a)f(generic)h(application,)h(there)f(are)f(t)o(w)o(o)f(distinct) i(phases)g(of)e(in)o(teraction)75 2182 y(with)k(the)f(resource)g(manager.)32 b(In)20 b(the)f(\014rst)g(phase,)i(the)e(user)h(requests)f(the)g(initial)j (resources)d(on)75 2238 y(whic)o(h)g(an)f(application)i(will)g(run.)29 b(This)19 b(phase)f(is)h(implicit)h(but)f(not)e(sp)q(eci\014ed)k(in)e(the)f (static)g(MPI)75 2295 y(mo)q(del.)30 b(In)19 b(the)f(second)h(phase,)g(the)g (application)h(itself)f(ma)o(y)e(request)i(additional)g(resources)g(from)75 2351 y(the)c(resource)h(manager.)166 2408 y(While)21 b(it)f(seems)g(at)g (\014rst)f(that)g(MPI)h(need)h(only)f(b)q(e)h(concerned)g(with)f(the)g (second)h(phase,)g(it)75 2464 y(m)o(ust)16 b(b)q(e)i(a)o(w)o(are)d(of)i(the)g (\014rst)f(as)h(w)o(ell.)25 b(The)18 b(reason)e(is)h(that)g(resources)f(ma)o (y)h(b)q(e)g(\\preallo)q(cated")h(to)75 2521 y(an)d(application.)166 2577 y(W)l(e)g(exp)q(ect)h(the)f(most)f(common)h(t)o(yp)q(e)g(of)g (\\dynamic")g(application)i(to)d(b)q(e)i(one)f(in)h(whic)o(h)g(all)g(re-)75 2634 y(sources)e(are)h(actually)g(allo)q(cated)g(b)q(efore)g(the)f (application)i(starts.)j(The)14 b(application)j(will)f(b)q(e)f(started)75 2690 y(on)10 b(a)g(subset)g(of)g(the)g(resources,)h(will)h Fk(disc)n(over)d Fs(the)i(additional)g(resources)f(using)h Fj(MPI)p 1545 2690 14 2 v 16 w(Resource)p 1729 2690 V 18 w(discover)p Fs(,)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 bop 75 -100 a Fs(4)951 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fs(and)i(will)h(start)d(the)i(rest)f(of)g(the)g (application)i(on)f(those)f(additonal)h(resources.)24 b(W)l(e)16 b(term)g(these)h(ap-)75 102 y(plications)i Fk(quasi-static)e Fs(b)q(ecause)i(they)e(use)h(static)f(resources)h(but)f(ma)o(y)g(ha)o(v)o(e)g (dynamic)h(pro)q(cesses.)75 158 y(Applications)h(whic)o(h)f(disco)o(v)o(er)f (preallo)q(cated)h(resources)f(but)g(do)g(not)g(allo)q(cate)h(new)f(ones)g (will)i(gen-)75 214 y(erally)e(b)q(e)g(more)f(p)q(ortable)h(than)f(those)g (whic)o(h)h(allo)q(cate)g(new)g(resources.)23 b(Resource)17 b(disco)o(v)o(ery)g(ma)o(y)75 271 y(or)g(ma)o(y)h(not)f(require)i(comm)o (unication)g(with)f(an)g(external)g(resource)g(manager,)g(dep)q(ending)i(on)e (the)75 327 y(MPI)d(implemen)o(tation.)22 b(Resource)16 b(disco)o(v)o(ery)g (can)f(b)q(e)i(an)e(extremely)h(simple)h(op)q(eration,)e(so)g(that)g(it)75 384 y(can)i(b)q(e)g(used)g(b)o(y)f(a)g(naiv)o(e)h(user)g(in)g(conjunction)h (with)f Fj(MPI)p 1141 384 14 2 v 15 w(SP)l(A)-5 b(WN)18 b Fs(to)e(simply)h (mimic)h(PVM-st)o(yle)75 440 y(b)q(eha)o(vior.)166 497 y(T)l(ruly)g(dynamic)h (applications)g(will)g(allo)q(cate)f(new)g(resources)g(while)h(running,)g(in) o(teracting)f(di-)75 553 y(rectly)g(with)f(the)h(resource)f(manager)f (through)h Fj(MPI)p 1015 553 V 16 w(RESOURCE)p 1261 553 V 18 w(IALLOCA)l(TE)p Fs(.)g(Because)h(this)g(op-)75 610 y(eration)g(is)h(p)q (oten)o(tially)h(\(and)e(usually\))i(exp)q(ensiv)o(e,)g(it)f(is)g(non)o(blo)q (c)o(king.)30 b(A)19 b(resource)f(request)h(ma)o(y)75 666 y(b)q(e)g (arbitrarily)h(complicated,)g(but)f(the)g(details)h(of)e(the)h(request)g(are) f(in)o(terpreted)h(b)o(y)g(the)g(resource)75 723 y(manager,)14 b(not)h(MPI.)g(The)g(request)g(itself)h(it)g(therefore)f(a)g(string)g(whic)o (h)h(has)f(no)g(meaning)h(to)e(MPI.)166 779 y(Regardless)g(of)f(ho)o(w)g(an)h (application)h(acquires)f(resources,)g(it)f(ma)o(y)g(release)h(them)g(to)f (the)g(system)75 835 y(through)i Fj(MPI)p 332 835 V 16 w(Resource)p 516 835 V 17 w(free)p Fs(.)166 968 y Fg(Discussion:)61 b Ff(The)21 b(description)h(ab)q(o)o(v)o(e)e(is)h(sligh)o(tly)e(di\013eren)o(t)j(from)d (earlier)i(descriptions)h(in)e(that)75 1018 y(it)f(con)o(tains)g(the)h (concept)h(of)e(resource)i(disco)o(v)o(ery)m(.)34 b(There)21 b(are)e(t)o(w)o(o)g(reasons)i(for)e(this.)34 b(First,)21 b(almost)c(all)75 1068 y(applications)11 b(\(including)h(most)f(new)i(ones)g(w)o(e)g(target)g (in)f(MPI-2\))g(need)i(only)e(the)h(\014rst)g(phase)g(-)f(making)e(use)k(of) 75 1117 y(pre-allo)q(cated)f(resources.)19 b(An)13 b(application)e(usually)g (do)q(es)j(not)e(actually)f(request)j(resources)h(nor)d(do)q(es)h(it)f(kno)o (w)75 1167 y(what)f(it)g(w)o(an)o(ts)g(\(nor)g(should)g(it!\).)16 b(It)c(only)e(w)o(an)o(ts)h(to)g(\014nd)g(out)g(what)g(the)h(user)g(has)f (already)g(allo)q(cated.)17 b(Second,)75 1217 y(it)12 b(w)o(as)g(di\016cult)f (to)h(come)f(up)i(with)e(a)h(uni\014ed)g(seman)o(tics)g(that)g(w)o(ould)f(w)o (ork)h(w)o(ell)g(for)f(b)q(oth)i(resource)h(disco)o(v)o(ery)75 1267 y(and)h(resource)j(allo)q(cation.)j(They)16 b(are)g(therefore)h (separate,)f(although)f(they)h(b)q(oth)g(pro)q(duce)g(the)h(same)d(result)75 1317 y(-)i(an)h Fe(MPI)p 243 1317 13 2 v 14 w(Resource)h Ff(ob)r(ject)g(\(or) e(ob)r(jects\).)28 b(This)16 b(distinction)h(actually)e(exists)j(in)e(PVM,)g (sort)i(of.)25 b(It)17 b(is)f(the)75 1366 y(di\013erence)g(b)q(et)o(w)o(een)f Fd(pvm)p 490 1366 14 2 v 15 w(config)e Ff(and)g Fd(pvm)p 796 1366 V 15 w(addhosts)p Ff(.)166 1416 y(It)22 b(w)o(ould)f(b)q(e)i(p)q (ossible)f(to)f(treat)i(b)q(oth)f(resource)i(allo)q(cation)c(and)i(resource)i (disco)o(v)o(ery)e(through)g(an)75 1466 y(allo)q(cation-based)16 b(in)o(terface.)29 b(An)18 b(application)e(w)o(ould)g(\\allo)q(cate")h(the)h (resources)h(it)e(knew)h(to)f(b)q(e)h(a)o(v)n(ailable)75 1516 y(and)e(the)h(allo)q(cation)d(routine)i(w)o(ould)f(return)j(quic)o(kly)m(.)23 b(This)16 b(approac)o(h)g(is)g(undesirable)g(for)g(sev)o(eral)g(reasons.)75 1566 y(First,)e(in)g(the)h(allo)q(cation-based)e(approac)o(h,)h(an)g (application)f(m)o(ust)g(\014nd)h(out)g(through)h(some)e(external)i(means)75 1616 y(\(suc)o(h)h(as)f(a)g(con\014guration)g(\014le\))g(what)g(resources)j (it)d(should)f(ask)i(for.)21 b(This)15 b(is)g(cum)o(b)q(ersome)f(and)h (redundan)o(t)75 1665 y(with)e(the)i(initial)d(resource)j(request.)20 b(Second,)14 b(it)f(requires)i(ev)o(en)g(simple)d(applications)h(to)g(form)o (ulate)f(a)i(p)q(oten-)75 1715 y(tially)e(complicated)h(resource)i(request.) 20 b(Resource)15 b(sp)q(eci\014cation)f(will)f(v)n(ary)g(widely)g(with)g (resource)j(managers)75 1765 y(and)g(should)g(remain)e(outside)i(an)g (application)f(as)h(m)o(uc)o(h)e(as)i(p)q(ossible,)h(for)e(p)q(ortabilit)o(y) m(.)23 b(Third,)16 b(it)f(w)o(ould)g(b)q(e)75 1815 y(a)g(lie,)h(as)g (resources)i(ha)o(v)o(e)d(already)h(b)q(een)h(allo)q(cated.)23 b(This)16 b(is)f(alw)o(a)o(ys)g(a)h(dangerous)g(thing)f(and)h(can)g(mislead) 75 1865 y(programmers.)166 1914 y(I)i(exp)q(ect)i(that)f(true)g(dynamic)e (applications)g(are)i(rare.)32 b(A)18 b(lot)g(of)g(the)h(true)g(dynamicism)c (ma)o(y)i(come)75 1964 y(in)e(parallel)g(en)o(vironmen)o(ts)g(whic)o(h)h(are) g(written)g(in)g(MPI.)f(F)m(or)h(instance,)g(with)g(the)g(dynamic)e(in)o (terface,)j(it)e(is)75 2014 y(p)q(ossible)e(to)g(imagine)d(writing)j(an)f (MPI)h(console)h(similar)c(to)j(the)g(PVM)h(console)f(that)g(puts)g(together) h(a)f(virtual)75 2064 y(mac)o(hine)g(and)g(spa)o(wns)i(MPI)f(applications)f (on)g(that)h(virtual)f(mac)o(hine.)166 2114 y(I)i(also)g(exp)q(ect)i(that)f (resource)i(disco)o(v)o(ery)e(will)e(b)q(e)i(quite)f(p)q(ortable,)h(while)f (resource)j(allo)q(cation)c(ma)o(y)f(b)q(e)75 2163 y(tailored)g(to)h(sp)q (eci\014c)h(en)o(vironmen)o(ts.)166 2213 y(-)f(w)o(cs)166 2352 y Fs(MPI)22 b(do)q(es)g(not)g(address)g(the)g(issue)h(of)f(\014nding)h(out)f (what)g(additional)h(\(not)f(y)o(et)f(allo)q(cated\))75 2409 y(resources)11 b(ma)o(y)f(b)q(e)h(a)o(v)m(ailable.)20 b(F)l(or)10 b(instance,)i(an)e(application)j(migh)o(t)d(w)o(an)o(t)g(to)g(kno)o(w)g(what) g(is)h(p)q(ossible)75 2465 y(to)f(request)h(b)q(efore)g(it)h(requests)e (resources.)19 b(This)11 b(issue)h(is)g(quite)f(complex)h(and)f(not)g (directly)h(necessary)75 2522 y(for)19 b(MPI,)g(in)h(con)o(trast)e(to)g (resource)i(allo)q(cation)g(itself,)h(in)f(whic)o(h)g(an)f(application)i (obtains)e(simple)75 2578 y(MPI-de\014ned)c(ob)s(jects)d(necessary)i(for)e (starting)h(pro)q(cesses.)19 b(It)14 b(is)f(exp)q(ected)i(that)d(an)h (application)i(will)75 2635 y(use)h(a)e(resource)i(manager-sp)q(eci\014c)g (API)g(to)e(inquire)j(ab)q(out)e(resource)h(a)o(v)m(ailabilit)o(y)l(.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 bop 75 -100 a Fm(1.2.)34 b(THE)15 b(MPI)g(D)o(YNAMIC)g(PR)o(OCESS)h(MODEL)813 b Fs(5)75 45 y Fj(Sta)o(rting)21 b(and)g(Managing)f(Pro)q(cesses)47 b Fs(As)20 b(with)h(resource)g(managemen)o(t,)f(there)h(are)f(t)o(w)o(o)f(t)o (yp)q(es)h(of)75 102 y(pro)q(cess)e(creation.)27 b(The)18 b(\014rst)f(is)h (the)f(creation)h(of)f(the)h(original)h(MPI)e(application.)29 b(The)17 b(second)h(is)75 158 y(creation)g(of)g(pro)q(cesses)g(from)f(within) i(an)f(MPI)g(application.)29 b(F)l(ortunately)18 b(MPI)g(need)h(only)f(w)o (orry)75 214 y(ab)q(out)d(the)g(second)h(t)o(yp)q(e.)166 271 y(MPI)h(applications)h(ma)o(y)e(start)f(new)i(pro)q(cesses)h(\(including)h (non-MPI)e(pro)q(cesses\),)g(send)g(them)75 327 y(signals,)24 b(and)e(\014nd)h(out)e(when)i(they)f(die)h(or)e(b)q(ecome)i(unreac)o(hable.) 41 b(They)22 b(do)g(this)g(through)g(an)75 384 y(in)o(terface)16 b(to)f(the)h(pro)q(cess)g(manager,)f(whic)o(h)i(range)e(from)g(a)h(parallel)h (op)q(erating)f(system)g(\(CMOST\))75 440 y(to)f(la)o(y)o(ered)g(soft)o(w)o (are)e(\(POE\))i(to)g(an)g Fi(rsh)g Fs(command)g(\(P4\).)166 497 y(There)f(are)f(t)o(w)o(o)g(w)o(a)o(ys)f(to)h(start)g(new)h(pro)q (cesses.)20 b Fj(MPI)p 1116 497 14 2 v 16 w(SP)l(A)-5 b(WN)14 b Fs(is)g(a)g(simple)h(in)o(terface)f(adequate)75 553 y(or)22 b(most)f(applications.)43 b Fj(MPI)p 633 553 V 15 w(PROCESS)p 845 553 V 18 w(CREA)l(TE)24 b Fs(pro)o(vides)e(more)g(\015exibilit)o(y)j(and) d(the)h(abilit)o(y)75 610 y(to)18 b(manage)g(non-MPI)h(pro)q(cesses.)31 b(Both)18 b(tak)o(e)g(as)g(input)i(an)e Fj(MPI)p 1278 610 V 16 w(Resource)i Fs(ob)s(ject,)e(represen)o(ting)75 666 y(computational)d (resources)g(on)g(whic)o(h)h(a)f(job)f(can)h(run.)20 b(F)l(or)15 b(simple)h(applications,)g(there)f(is)h(a)e(simple)75 723 y(w)o(a)o(y)g(to)h (obtain)g(this)h Fj(MPI)p 538 723 V 16 w(Resource)g Fs(ob)s(ject.)166 779 y Fj(MPI)p 251 779 V 16 w(SP)l(A)-5 b(WN)12 b Fs(starts)f(MPI)h(pro)q (cesses)g(and)g(establishes)h(comm)o(unication)f(with)g(them,)g(returning)75 835 y(a)j(comm)o(unicator.)166 892 y Fj(MPI)p 251 892 V 16 w(PROCESS)p 464 892 V 18 w(CREA)l(TE)21 b Fs(merely)f(starts)f(new)i(pro)q (cesses.)34 b(An)21 b(application)g(ma)o(y)f(establish)75 948 y(comm)o(unication)c(with)f(these)h(new)f(pro)q(cesses)h(through)f Fj(MPI)p 1159 948 V 16 w(PROCESS)p 1372 948 V 17 w(A)l(TT)l(A)o(CH)p Fs(.)75 1070 y Fl(1.2.3)49 b(Examples)16 b(of)h(Runtime)f(Environments)75 1156 y Fs(T)l(o)11 b(illustrate)h(ho)o(w)f(the)h(ab)q(o)o(v)o(e)e(framew)o (ork)g(allo)o(ws)i(us)f(to)g(describ)q(e)i(a)e(wide)h(v)m(ariet)o(y)g(of)e (actual)i(systems,)75 1212 y(w)o(e)j(giv)o(e)g(here)h(some)f(examples.)75 1332 y Fj(Environments)22 b(with)f(explicit)h(Resource)g(Managers)45 b Fs(The)22 b(SP2)e(computers)h(at)g(Argonne)g(National)75 1389 y(Lab)q(oratory)e(and)g(the)h(NAS)g(facilit)o(y)g(at)f(NASA)h(Ames)f (Researc)o(h)h(Cen)o(ter)f(use)h(\(di\013eren)o(t\))f(lo)q(cally)75 1445 y(written)f(job)f(sc)o(hedulers)i(to)e(manage)g(the)h(SP2.)27 b(The)18 b(sc)o(hedulers)h(ensure)f(that)f(only)h(one)g(user)g(has)75 1502 y(access)g(to)e(an)o(y)i(SP)f(no)q(de)h(at)f(a)g(time)h(and)g(manage)f (resources)g(in)i(a)e(\\fair")g(w)o(a)o(y)f(to)h(ensure)h(that)f(all)75 1558 y(users)g(can)g(get)g(access)g(to)g(the)g(mac)o(hines.)26 b(They)17 b(require)h(users)f(to)g(pro)o(vide)g(times)h(limits)g(for)e(their) 75 1615 y(jobs)h(so)f(that)h(the)g(mac)o(hine)h(can)f(b)q(e)h(tigh)o(tly)f (sc)o(heduled.)27 b(Users)17 b(submit)h(scripts)f(to)g(the)g(sc)o(heduler,)75 1671 y(whic)o(h)i(allo)q(cates)g(resources)f(and)g(runs)g(jobs)g(using)g (IBM's)g(P)o(arallel)h(En)o(vironmen)o(t)f(soft)o(w)o(are.)27 b(The)75 1728 y(systems)15 b(in)o(teract)g(with)g(a)g(v)m(ariet)o(y)h(of)e (message-passing)i(libraries,)g(including)i(t)o(w)o(o)c(based)h(on)g(MPI.)166 1784 y(The)j(corresp)q(ondence)h(with)f(the)f(mo)q(del)i(ab)q(o)o(v)o(e)e(is) h(straigh)o(tforw)o(ard.)25 b(The)18 b(resource)g(manager)75 1841 y(is)h(the)g(lo)q(cally)i(written)e(job)g(sc)o(heduler.)32 b(The)20 b(pro)q(cess)f(manager)f(is)h(con)o(tained)h(in)g(IBM's)e(parallel) 75 1897 y(en)o(vironmen)o(t)d(soft)o(w)o(are.)166 1953 y(There)23 b(are)g(n)o(umerous)f(examples)i(of)e(job)h(managemen)o(t)f(systems)h(that)f (allo)q(cate)h(resources.)75 2010 y(These)f(include)i(PBS)e(\(from)e(NAS\),)i (EASY)f(\(from)g(ANL\),)g(LSF)h(\(from)f(Platform)f(Computing\),)75 2066 y(LoadLev)o(eler)d(\(from)d(IBM\),)h(DQS)h([)p Fo(?)p Fs(])21 b(\(from)14 b(the)i(Sup)q(ercomputer)g(Researc)o(h)g(Institute)h(at)e (Florida)75 2123 y(State)g(Univ)o(ersit)o(y\),)g(Condor)g(\(from)f(the)h (Univ)o(ersit)o(y)h(of)f(Wisconsin\),)g(and)h(NQS.)166 2179 y(Eac)o(h)k(one)h(of)f(these)h(resource)g(managers)e(can)i(b)q(e)g(used)h(in) f(conjunction)h(with)f(one)f(or)g(more)75 2236 y(pro)q(cess)c(managers.)22 b(Pro)q(cess)16 b(managers)g(ma)o(y)f(b)q(e)i(programs)e(external)h(to)g(an)g (application)i(\()p Fi(poe)d Fs(on)75 2292 y(the)k(IBM)g(SP2\),)g(part)f(of)g (the)h(op)q(erating)g(system)f(\(the)h(usual)g(case)g(on)g(tigh)o(tly)g(in)o (tegrated)g(MPPs)75 2349 y(suc)o(h)12 b(as)g(the)g(In)o(tel)h(P)o(aragon,)e (Meik)o(o)h(CS-2)g(and)h(TMC)e(CM-5\),)h(part)f(of)h(the)g(message)g(passing) g(library)75 2405 y(\()p Fi(p4)j Fs(or)f Fi(pvm)p Fs(\))h(or)f(in)o(tegrated) h(with)h(the)f(resource)g(manager)g(\(Condor)f(with)i Fi(pvm)p Fs(\).)75 2525 y Fj(Net)o(w)o(o)o(rk)c(of)f(W)o(o)o(rkstations)i(with)f(PVM)h (o)o(r)e(P4)45 b Fs(One)13 b(reason)e(for)h(PVM's)f(p)q(opularit)o(y)i(is)f (that)g(it)g(can)g(b)q(e)75 2582 y(view)o(ed)h(as)f(a)g(completely)i (self-con)o(tained)f(system)f(that)g(supplies)i(its)f(o)o(wn)f(pro)q(cess)g (managemen)o(t)g(and)75 2638 y(can)g(b)q(e)h(used)g(to)e(implemen)o(t)i(a)f (resource)g(manager)f(as)h(w)o(ell.)20 b(On)13 b(systems)e(that)g(ha)o(v)o(e) h(neither)h(of)f(these)75 2694 y(functions)20 b(pre-installed,)i(PVM)c(can)h (pro)o(vide)h(a)e(complete)i(solution.)32 b(A)19 b(user)g(creates)g(a)g (\\virtual)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 bop 75 -100 a Fs(6)951 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fs(mac)o(hine")h(b)o(y)f(starting)g(\\daemons")g(on)g (an)g(assortmen)o(t)f(of)h(mac)o(hines)h(and)g(then)g(sc)o(hedules)g(jobs)g (to)75 102 y(run)i(on)g(it)h(and)f(manages)f(his)i(pro)q(cesses)f(with)h(the) f(help)h(of)f(the)g(daemons.)28 b(The)18 b(virtual)h(mac)o(hine)75 158 y(itself)h(can)g(b)q(e)g(recon\014gured)g(from)f(inside)i(the)f(user)g (program.)31 b(A)20 b(di\016cult)o(y)h(with)e(this)h(approac)o(h)75 214 y(is)g(that)e(the)h(user)g(is)h(assumed)f(to)f(ha)o(v)o(e)h(the)g (necessary)h(p)q(ermissions)g(to)e(execute)i(suc)o(h)g(functions.)75 271 y(This)15 b(ma)o(y)f(b)q(e)h(the)f(case)h(on)f(a)g(w)o(orkstation)f(net)o (w)o(ork,)g(but)i(seldom)f(on)h(an)f(MPP)l(.)g(Con\015icts)h(b)q(et)o(w)o (een)75 327 y(existing)e(pro)q(cess)g(managers)e(and)h(PVM)g(can)h(inhibit)h (the)e(p)q(ortabilit)o(y)i(\(to)d(MPP's\))g(of)h(self-con)o(tained)75 384 y(programs)i(that)g(assume)i(all)g(functionalit)o(y)g(will)h(b)q(e)f(pro) o(vided)g(b)o(y)f(PVM.)166 440 y(Equiv)m(alen)o(t)h(MPI)f(functionalit)o(y)h (requires)g(an)f(MPI)f(implemen)o(tation)i(in)g(whic)o(h)g(b)q(oth)f (resource)75 497 y(managemen)o(t)22 b(and)i(pro)q(cess)f(managemen)o(t)g(are) f(pro)o(vided)i(b)o(y)g(the)f(MPI)g(implemen)o(tation)h(itself.)75 553 y(While)12 b(implemen)o(ted)h(within)f(MPI,)e(these)h(functions)h(w)o (ould)f(b)q(e)h(logically)g(external)g(to)e(an)h(MPI)f(appli-)75 610 y(cation,)15 b(allo)o(wing)g(the)g(application)i(to)d(run)h(virtually)h (unc)o(hanged)f(in)h(the)f(presence)g(of)g(real)g(resource)75 666 y(and)21 b(pro)q(cess)g(managers.)35 b(In)21 b(suc)o(h)g(an)g(implemen)o (tation,)h(requests)f(to)f(the)h(\\resource)f(manager")75 723 y(w)o(ould)c(alw)o(a)o(ys)e(succeed,)i(assuming)g(the)f(user)g(had)h (appropriate)f(p)q(ermission)h(to)f(start)f(pro)q(cesses.)75 843 y Fj(PVM)d(p)o(ro)q(cess)h(management)e(with)i(MPI)e(message-passing.)47 b Fs(One)11 b(can)h(conceiv)o(e)g(of)e(a)h(system)f(in)i(whic)o(h)75 899 y(resource)j(allo)q(cation)h(and)f(pro)q(cess)g(managemen)o(t)f (functions)i(w)o(ere)f(pro)o(vided)h(b)o(y)e(the)h(existing)h(PVM)75 956 y(daemon)d(structure,)g(while)h(MPI)f(w)o(as)f(used)h(b)o(y)g(the)g (application)i(for)d(message)h(passing.)19 b(PVM)13 b(w)o(ould)75 1012 y(b)q(e)j(used)g(as)f(the)h(implemen)o(tation)g(la)o(y)o(er)f(for)g(the) h(functions)g(describ)q(ed)h(in)f(this)g(c)o(hapter,)f(but)h(w)o(ould)75 1068 y(not)f(b)q(e)h(visible)h(to)e(the)g(application)i(program.)75 1190 y Fl(1.2.4)49 b(Applications)18 b(Requiring)f(Direct)e(Communication)i (with)g(the)e(Runtime)h(System)75 1276 y Fs(The)i(existing)h(MPI)f(sp)q (eci\014cation)h(is)g(adequate)e(for)g(most)g(parallel)j(applications.)29 b(In)18 b(these)g(appli-)75 1332 y(cations,)e(the)h(resource)f(manager)g(and) g(pro)q(cess)h(manager,)f(whether)g(simple)i(or)e(elab)q(orate,)g(allo)q (cate)75 1389 y(resources)f(and)h(manage)e(user)i(pro)q(cesses)g(without)f (in)o(teracting)h(with)f(the)g(application)i(program.)i(In)75 1445 y(other)e(applications,)j(ho)o(w)o(ev)o(er,)c(it)i(is)g(necessary)g (that)f(the)h Fk(user)g(level)j Fs(of)c(the)h(application)h(comm)o(u-)75 1502 y(nicate)f(with)g(the)g(job)g(sc)o(heduler)h(and/or)e(the)h(pro)q(cess)g (manager.)27 b(Here)18 b(w)o(e)f(describ)q(e)i(three)f(broad)75 1558 y(classes)f(of)e(suc)o(h)i(applications.)24 b(In)16 b(Section)h Fo(??)22 b Fs(w)o(e)16 b(will)i(giv)o(e)e(concrete)h(examples)f(of)g(eac)o(h) g(of)g(these)75 1615 y(classes.)75 1735 y Fj(T)l(ask)c(F)o(a)o(rming)43 b Fs(By)11 b(a)g(\\task)f(farm")g(application)i(w)o(e)f(mean)g(a)f(program)g (that)g(manages)h(the)g(execution)75 1791 y(of)j(a)g(set)g(of)g(other,)g(p)q (ossibly)i(sequen)o(tial,)g(programs.)i(This)d(situation)g(often)f(arises)h (when)g(one)f(w)o(an)o(ts)75 1848 y(to)19 b(run)h(the)g(same)f(sequen)o(tial) i(program)e(man)o(y)g(times)h(with)g(v)m(arying)h(input)f(data.)33 b(W)l(e)20 b(call)h(eac)o(h)75 1904 y(in)o(v)o(o)q(cation)14 b(of)e(the)i(sequen)o(tial)g(program)e(a)h Fk(task)p Fs(.)19 b(It)13 b(is)h(often)f(simplest)h(to)f(\\parallelize")i(the)e(existing)75 1961 y(sequen)o(tial)h(program)d(b)o(y)i(writing)g(a)f(parallel)j(\\harness") d(program)f(that)h(in)i(turn)e(dev)o(otes)h(a)f(separate,)75 2017 y(transien)o(t)j(pro)q(cess)g(to)g(eac)o(h)g(task.)k(When)d(one)f(task)f (\014nishes,)i(a)f(new)g(pro)q(cess)h(is)f(started)g(to)f(execute)75 2073 y(the)19 b(next)f(one.)30 b(Ev)o(en)19 b(if)g(the)f(resources)h(allo)q (cated)g(to)f(the)h(job)f(are)g(\014xed,)i(the)f(\\harness")f(pro)q(cess)75 2130 y(m)o(ust)13 b(in)o(teract)g(frequen)o(tly)h(with)g(the)f(pro)q(cess)h (manager)f(\(ev)o(en)g(if)h(this)g(is)g(just)f Fi(rsh)p Fs(,)g(to)g(start)f (the)i(new)75 2186 y(pro)q(cesses)i(with)f(the)h(new)f(input)h(data\).)j(In)d (man)o(y)f(cases)g(this)h(harness)f(can)g(b)q(e)h(written)g(in)g(a)e(simple) 75 2243 y(scripting)k(language)f(lik)o(e)g Fi(csh)g Fs(or)f Fi(perl)p Fs(,)g(but)h(some)f(users)h(prefer)g(to)f(use)h(F)l(ortran)e(or)h (C.)g(Note)h(that)75 2299 y(it)f(is)g(an)g(explicit)i(goal)d(of)h(the)f(MPI)h (dynamic)h(pro)q(cess)f(arc)o(hitecture)g(to)f(allo)o(w)h(the)f(managemen)o (t)g(of)75 2356 y(non-MPI)h(pro)q(cesses.)75 2476 y Fj(Dynamic)f(numb)q(er)i (of)e(p)o(ro)q(cesses)j(in)e(pa)o(rallel)f(job)46 b Fs(The)16 b(program)f(wishes)i(to)e(decide)j Fk(inside)g Fs(the)e(pro-)75 2532 y(gram)e(to)g(adjust)g(the)g(n)o(um)o(b)q(er)h(of)f(pro)q(cesses)h(to)f (\014t)h(the)f(size)i(of)e(the)h(problem.)20 b(F)l(urthermore,)14 b(it)h(ma)o(y)75 2589 y(con)o(tin)o(ue)k(to)g(add)g(and)g(subtract)f(pro)q (cesses)h(during)h(the)f(computation)g(to)f(\014t)h(separate)f(phases)h(of)75 2645 y(the)g(computation,)h(some)f(of)g(whic)o(h)h(ma)o(y)f(b)q(e)h(more)f (parallel)i(than)e(others.)32 b(In)20 b(order)f(to)g(do)g(this,)75 2702 y(the)i(application)i(program)d(will)j(ha)o(v)o(e)e(to)g(in)o(teract)g (with)h(the)f(resource)g(manager)g(\(ho)o(w)o(ev)o(er)f(it)h(is)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 bop 75 -100 a Fm(1.3.)33 b(NEW)15 b(OBJECTS)g(F)o(OR)g(IMPLEMENTING)g(D)o (YNAMIC)f(PR)o(OCESS)h(MANA)o(GEMENT)p Fs(7)75 45 y(implemen)o(ted\))k(to)d (request)i(and)f(acquire)h(or)f(return)g(computational)h(resources.)26 b(It)17 b(will)i(also)e(ha)o(v)o(e)75 102 y(to)f(in)o(teract)h(with)g(the)g (pro)q(cess)g(manager)g(to)f(request)h(that)f(pro)q(cesses)h(b)q(e)h(started) e(and)h(in)h(order)f(to)75 158 y(mak)o(e)12 b(the)g(new)h(pro)q(cesses)f(kno) o(wn)g(to)g(the)g(message-passing)g(library)h(so)f(that)g(the)g(larger)g (\(or)g(smaller\))75 214 y(group)j(of)g(pro)q(cesses)g(can)h(comm)o(unicate.) 166 271 y(An)21 b(imp)q(ortan)o(t)f(t)o(yp)q(e)h(of)f(dynamic)h(application)i (is)e(a)f(sca)o(v)o(enger)g(application.)38 b(A)20 b(sca)o(v)o(enger)75 327 y(application)j(is)g(\\em)o(barassingly)f(parallel")h(in)f(the)g(sense)g (that)f(it)h(p)q(erforms)g(a)f(large)h(n)o(um)o(b)q(er)g(of)75 384 y(completely)17 b(indep)q(enden)o(t)i(tasks.)i(If)c(the)f(n)o(um)o(b)q (er)g(of)g(tasks)f(is)i(large)f(enough,)g(suc)o(h)g(an)g(application)75 440 y(can)k(mak)o(e)g(use)h(of)f(an)o(y)g(resources)g(that)g(b)q(ecome)h(a)o (v)m(ailable.)37 b(Con)o(v)o(ersely)l(,)21 b(it)g(can)g(easily)g(giv)o(e)f (up)75 497 y(resources)i(to)g(another)g(application.)42 b(Sca)o(v)o(enger)22 b(applications)i(are)e(excellen)o(t)i(for)e(\014lling)i(in)f(the)75 553 y(\\holes")15 b(on)g(a)g(space-shared)h(parallel)h(mac)o(hine,)f(allo)o (wing)g(it)f(to)g(ac)o(hiev)o(e)g(v)o(ery)g(high)h(utilization.)75 673 y Fj(Client/Server)46 b Fs(This)15 b(situation)h(is)f(the)g(opp)q(osite)h (of)e(the)h(situations)g(ab)q(o)o(v)o(e,)g(where)g(pro)q(cesses)g(come)75 730 y(and)21 b(go)f(up)q(on)h(request.)37 b(In)21 b(the)g(clien)o(t/serv)o (er)g(mo)q(del,)i(one)e(set)f(of)g(pro)q(cesses)h(is)g(relativ)o(ely)h(p)q (er-)75 786 y(manen)o(t)17 b(\(the)g(serv)o(er,)g(whic)o(h)h(w)o(e)f(assume)g (here)h(ma)o(y)e(b)q(e)i(a)f(parallel)i(program\).)24 b(A)o(t)17 b(unpredictable)75 843 y(times,)e(another)f(\(p)q(ossibly)h(parallel\))h (program)d(\(the)i(clien)o(t\))g(b)q(egins)h(execution)f(and)g(m)o(ust)f (establish)75 899 y(comm)o(unication)k(with)f(the)g(serv)o(er.)26 b(In)17 b(this)h(case)f(the)g(pro)q(cess)g(manager)g(m)o(ust)f(pro)o(vide)i (a)f(w)o(a)o(y)f(for)75 956 y(the)e(clien)o(t)h(to)e(lo)q(cate)h(the)g(serv)o (er)f(and)h(comm)o(unicate)g(to)f(the)g(message-passing)h(library)h(that)e (it)h(m)o(ust)75 1012 y(no)o(w)h(supp)q(ort)g(comm)o(unications)h(with)f(a)g (new)h(collection)h(of)d(pro)q(cesses.)166 1068 y(It)19 b(is)g(curren)o(tly)h (p)q(ossible)g(to)f(write)g(the)g(parallel)h(clien)o(ts)g(and)f(serv)o(ers)g (in)g(MPI,)g(but)g(b)q(ecause)75 1125 y(MPI)g(do)q(es)g(not)g(pro)o(vide)h (the)f(necessary)g(in)o(terfaces)g(b)q(et)o(w)o(een)h(the)f(application)i (program)d(and)h(the)75 1181 y(resource)f(manager)g(or)g(pro)q(cess)h (manager,)f(other)g(nonp)q(ortable,)h(mac)o(hine)g(sp)q(eci\014c)i(libraries) f(m)o(ust)75 1238 y(b)q(e)15 b(called)h(in)f(order)f(for)g(the)g(clien)o(t)h (and)g(serv)o(er)f(to)f(comm)o(unicate)i(with)g(one)f(another.)19 b(On)c(the)f(other)75 1294 y(hand,)g(MPI)g(do)q(es)g(con)o(tain)g(sev)o(eral) f(features)h(that)f(mak)o(e)g(it)h(relativ)o(ely)h(easy)e(to)g(add)h(suc)o(h) g(in)o(terfaces,)75 1351 y(and)h(w)o(e)g(prop)q(ose)h(b)q(oth)f(a)g(simple)h (in)o(terface)g(and)f(a)g(more)g(complex)h(but)f(\015exible)j(one.)75 1494 y Fp(1.3)59 b(New)20 b(Objects)e(fo)n(r)j(Implementing)c(Dynamic)i(Pro)r (cess)g(Management)75 1595 y Fs(Using)d(dynamic)g(pro)q(cesses)f(in)h(MPI)g (is)f(a)g(three)h(step)f(pro)q(cess)g(sho)o(wn)g(in)h(Figure)g Fo(??)p Fs(.)131 1689 y(1.)22 b Fo(Allo)q(cate)14 b(or)e(disco)o(v)o(er)f (resources)p Fs(,)f(obtaining)h(one)g(or)g(more)f Fj(MPI)p 1403 1689 14 2 v 16 w(Resource)h Fs(ob)s(jects,)g(whic)o(h)189 1746 y(represen)o(t)k(p)q(ermission)h(to)f(start)f(additional)j(pro)q (cesses.)131 1839 y(2.)22 b Fo(Start)12 b(tasks)f(on)h(allo)q(cated/di)q(sco) o(v)o(ered)g(resources)p Fs(,)e(obtaining)h(one)f(or)g(more)f Fj(MPI)p 1742 1839 V 16 w(Pro)q(cess)189 1896 y Fs(ob)s(jects,)14 b(whic)o(h)i(represen)o(t)f(running)h(pro)q(cesses)g(but)f(don't)g(allo)o(w)g (comm)o(unication.)131 1990 y(3.)22 b Fo(Establish)16 b(MPI)e(comm)m (unication)g(with)h(or)g(b)q(et)o(w)o(een)f(newly)h(created)g(pro)q(cesses)d Fs(b)o(y)189 2046 y(obtaining)k(a)f(comm)o(unicator)f(whose)h(group)g(con)o (tains)h(the)f(new)g(pro)q(cesses.)75 2205 y Fl(1.3.1)49 b(The)16 b(MPI)p 409 2205 15 2 v 18 w(Resource)g(Object)75 2291 y Fs(A)23 b(request)g(to)f(the)g(resource)h(manager)f(returns)h(an)g Fj(MPI)p 1144 2291 14 2 v 15 w(Resource)h Fs(ob)s(ject.)42 b(An)23 b Fj(MPI)p 1693 2291 V 16 w(Resource)75 2348 y Fs(represen)o(ts)18 b(a)g(set)g(of)g(computational)h(resources)f(on)g(whic)o(h)h(an)f (application)i(ma)o(y)e(start)f(pro)q(cesses.)75 2404 y(Dep)q(ending)23 b(on)e(the)g(request)g(that)f(generated)h(it,)i(an)e Fj(MPI)p 1163 2404 V 15 w(Resource)i Fs(ma)o(y)d(represen)o(t)h(a)g(ph)o(ysical)75 2460 y(CPU,)14 b(a)f(collection)j(of)e(CPUs,)g(or)f(ev)o(en)i(a)f(promise)g (from)f(a)h(resource)g(manager)g(that)f(an)h(application)75 2517 y(is)22 b(allo)o(w)o(ed)g(to)e(spa)o(wn)i(one)f(or)g(more)g(pro)q (cesses)h(\(whose)f(ph)o(ysical)h(lo)q(cation)h(will)g(b)q(e)f(determined)75 2573 y(dynamically\).)166 2630 y(Within)15 b(MPI,)e(an)g Fj(MPI)p 581 2630 V 16 w(Resource)i Fs(is)f(simply)h(a)e(con)o(tainer)h(of)f(pro)q (cess)h(slots.)20 b(Pro)q(cess)13 b(slots)h(ma)o(y)75 2686 y(either)i(b)q(e)f(\014lled)i(\(b)o(y)d(a)h(single)h(running)g(pro)q(cess\))e (or)h(empt)o(y)l(.)k(The)c(con)o(tainer)g(should)h(b)q(e)f(though)o(t)f(of) -32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 bop 75 -100 a Fs(8)951 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)375 691 y @beginspecial 90 @llx 451 @lly 518 @urx 696 @ury 2880 @rwi @setspecial %%BeginDocument: fig2.eps 1 setlinejoin /M { moveto } bind def /S { show } bind def /R { rmoveto } bind def /L { lineto } bind def /B { newpath 0 0 M 0 1 L 1 1 L 1 0 L closepath } bind def /CS { closepath stroke } bind def /S { /fixwidth exch def dup length /nchars exch def dup stringwidth pop fixwidth exch sub nchars div exch 0 exch ashow } def /bwproc { rgbproc dup length 3 idiv string 0 3 0 5 -1 roll { add 2 1 roll 1 sub dup 0 eq { pop 3 idiv 3 -1 roll dup 4 -1 roll dup 3 1 roll 5 -1 roll put 1 add 3 0 } { 2 1 roll } ifelse } forall pop pop pop } def systemdict /colorimage known not { /colorimage { pop pop /rgbproc exch def { bwproc } image } def } if 1 1 scale 0 setlinewidth /drawtri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def 0 setgray newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath stroke } bind def /filltri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath fill } bind def /cliptri { /y3 exch def /x3 exch def /y2 exch def /x2 exch def /y1 exch def /x1 exch def 0 setgray newpath x1 y1 moveto x2 y2 lineto x3 y3 lineto closepath clip } bind def /imgscanrgb { gsave translate /scandy exch def /scandx exch def /istr scandx 3 mul string def scandx scandy scale scandx scandy 8 [scandx 0 0 scandy neg 0 scandy] {currentfile istr readhexstring pop} false 3 colorimage grestore } bind def /imgscanbw { gsave translate /scandy exch def /scandx exch def /istr scandx string def scandx scandy scale scandx scandy 8 [scandx 0 0 scandy neg 0 scandy] {currentfile istr readhexstring pop} image grestore } bind def /showcaseisoencoding [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /space /exclam /quotedbl /numbersign /dollar /percent /ampersand /quoteright /parenleft /parenright /asterisk /plus /comma /minus /period /slash /zero /one /two /three /four /five /six /seven /eight /nine /colon /semicolon /less /equal /greater /question /at /A /B /C /D /E /F /G /H /I /J /K /L /M /N /O /P /Q /R /S /T /U /V /W /X /Y /Z /bracketleft /backslash /bracketright /asciicircum /underscore /quoteleft /a /b /c /d /e /f /g /h /i /j /k /l /m /n /o /p /q /r /s /t /u /v /w /x /y /z /braceleft /bar /braceright /asciitilde /guilsinglright /fraction /florin /quotesingle /quotedblleft /guilsinglleft /fi /fl /endash /dagger /daggerdbl /bullet /quotesinglbase /quotedblbase /quotedblright /ellipsis /trademark /dotlessi /grave /acute /circumflex /tilde /macron /breve /dotaccent /dieresis /perthousand /ring /cedilla /Ydieresis /hungarumlaut /ogonek /caron /emdash /exclamdown /cent /sterling /currency /yen /brokenbar /section /dieresis /copyright /ordfeminine /guillemotleft /logicalnot /hyphen /registered /macron /degree /plusminus /twosuperior /threesuperior /acute /mu /paragraph /periodcentered /cedilla /onesuperior /ordmasculine /guillemotright /onequarter /onehalf /threequarters /questiondown /Agrave /Aacute /Acircumflex /Atilde /Adieresis /Aring /AE /Ccedilla /Egrave /Eacute /Ecircumflex /Edieresis /Igrave /Iacute /Icircumflex /Idieresis /Eth /Ntilde /Ograve /Oacute /Ocircumflex /Otilde /Odieresis /multiply /Oslash /Ugrave /Uacute /Ucircumflex /Udieresis /Yacute /Thorn /germandbls /agrave /aacute /acircumflex /atilde /adieresis /aring /ae /ccedilla /egrave /eacute /ecircumflex /edieresis /igrave /iacute /icircumflex /idieresis /eth /ntilde /ograve /oacute /ocircumflex /otilde /odieresis /divide /oslash /ugrave /uacute /ucircumflex /udieresis /yacute /thorn /ydieresis ] def /showcasedingbatencoding [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /a1 /a2 /a202 /a3 /a4 /a5 /a119 /a118 /a117 /a11 /a12 /a13 /a14 /a15 /a16 /a105 /a17 /a18 /a19 /a20 /a21 /a22 /a23 /a24 /a25 /a26 /a27 /a28 /a6 /a7 /a8 /a9 /a10 /a29 /a30 /a31 /a32 /a33 /a34 /a35 /a36 /a37 /a38 /a39 /a40 /a41 /a42 /a43 /a44 /a45 /a46 /a47 /a48 /a49 /a50 /a51 /a52 /a53 /a54 /a55 /a56 /a57 /a58 /a59 /a60 /a61 /a62 /a63 /a64 /a65 /a66 /a67 /a68 /a69 /a70 /a71 /a72 /a73 /a74 /a203 /a75 /a204 /a76 /a77 /a78 /a79 /a81 /a82 /a83 /a84 /a97 /a98 /a99 /a100 /.notdef /a205 /a85 /a206 /a86 /a87 /a88 /a89 /a90 /a91 /a92 /a93 /a94 /a95 /a96 /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /a101 /a102 /a103 /a104 /a106 /a107 /a108 /a112 /a111 /a110 /a109 /a120 /a121 /a122 /a123 /a124 /a125 /a126 /a127 /a128 /a129 /a130 /a131 /a132 /a133 /a134 /a135 /a136 /a137 /a138 /a139 /a140 /a141 /a142 /a143 /a144 /a145 /a146 /a147 /a148 /a149 /a150 /a151 /a152 /a153 /a154 /a155 /a156 /a157 /a158 /a159 /a160 /a161 /a163 /a164 /a196 /a165 /a192 /a166 /a167 /a168 /a169 /a170 /a171 /a172 /a173 /a162 /a174 /a175 /a176 /a177 /a178 /a179 /a193 /a180 /a199 /a181 /a200 /a182 /.notdef /a201 /a183 /a184 /a197 /a185 /a194 /a198 /a186 /a195 /a187 /a188 /a189 /a190 /a191 /.notdef ] def /Courier-Bold findfont dup length dict begin {1 index /FID ne {def} {pop pop} ifelse} forall /Encoding showcaseisoencoding def currentdict end /Courier-Bold-SHOWISO exch definefont pop /pmask10 { <18182424424281811818242442428181 18182424424281811818242442428181> } def /newfont 10 dict def newfont begin /FontType 3 def /FontMatrix [1 0 0 1 0 0] def /FontBBox [0 0 1 1] def /Encoding 256 array def 0 1 255 {Encoding exch /.notdef put} for Encoding 42 /patchar10 put /CharProcs 2 dict def CharProcs begin /.notdef {} def /patchar10 { 16 16 true [16.08 0 0 -16.08 0 16.08] pmask10 imagemask } def end /BuildChar { 1 0 0 0 1 1 setcachedevice exch begin Encoding exch get CharProcs exch get end exec } def end /PatternFont newfont definefont pop /#copies 1 def gsave gsave matrix currentmatrix [117 0 0 -126 90 648] concat B setmatrix 0 0 0 setrgbcolor 2.000000 setlinewidth gsave stroke grestore grestore gsave gsave matrix currentmatrix [1 0 0 1 91.7 654] concat newpath 0 0 M 0 42 L 77.6 42 L 77.6 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 89.2 651.5 M 89.2 698.5 L 171.8 698.5 L 171.8 651.5 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 91.7 654] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 31.5 M (Allocate/) 75.6 S 0 17.5 M (Discover) 67.2 S 0 3.5 M (Resources) 75.6 S setmatrix grestore gsave matrix currentmatrix [117 0 0 -126 247.5 648] concat B setmatrix 0 0 0 setrgbcolor 2.000000 setlinewidth gsave stroke grestore grestore gsave gsave matrix currentmatrix [1 0 0 1 249.2 654.5] concat newpath 0 0 M 0 28 L 77.6 28 L 77.6 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 246.7 652 M 246.7 685 L 329.3 685 L 329.3 652 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 249.2 654.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (Start) 42 S 0 3.5 M (Processes) 75.6 S setmatrix grestore gsave matrix currentmatrix [117 0 0 -126 400.5 648] concat B setmatrix 0 0 0 setrgbcolor 2.000000 setlinewidth gsave stroke grestore grestore gsave gsave matrix currentmatrix [1 0 0 1 402.2 654.5] concat newpath 0 0 M 0 28 L 111.2 28 L 111.2 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 399.7 652 M 399.7 685 L 515.9 685 L 515.9 652 L closepath clip newpath 0 0 0 setrgbcolor matrix currentmatrix [1 0 0 1 402.2 654.5] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (Establish ) 84 S 0 3.5 M (Communication) 109.2 S setmatrix grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 274.5 523 translate 90 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 180 522 M 180 468 L 274.5 468 L 274.5 504 L stroke grestore gsave 0 0 0 setrgbcolor gsave 0 setlinejoin 441 523 translate 90 rotate 1.2 1.2 scale newpath 0 0 M -20 -5 L -20 5 L closepath fill grestore 2.000000 setlinewidth newpath 337.5 522 M 337.5 495 L 337.5 468 L 441 468 L 441 504 L stroke grestore gsave gsave matrix currentmatrix [1 0 0 1 179 451] concat newpath 0 0 M 0 14 L 102.8 14 L 102.8 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 176.5 448.5 M 176.5 467.5 L 284.3 467.5 L 284.3 448.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 179 451] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 3.5 M (MPI_Resource) 100.8 S setmatrix grestore gsave gsave matrix currentmatrix [1 0 0 1 341 451] concat newpath 0 0 M 0 14 L 94.4 14 L 94.4 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 338.5 448.5 M 338.5 467.5 L 437.9 467.5 L 437.9 448.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 341 451] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 3.5 M (MPI_Process) 92.4 S setmatrix grestore gsave gsave matrix currentmatrix [1 0 0 1 98 585] concat newpath 0 0 M 0 42 L 102.8 42 L 102.8 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 95.5 582.5 M 95.5 629.5 L 203.3 629.5 L 203.3 582.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 98 585] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 31.5 M (MPI_Resource) 100.8 S 0 17.5 M ( _Discover) 92.4 S 0 3.5 M ( _Allocate) 92.4 S setmatrix grestore gsave gsave matrix currentmatrix [1 0 0 1 255.5 599] concat newpath 0 0 M 0 28 L 94.4 28 L 94.4 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 253 596.5 M 253 629.5 L 352.4 629.5 L 352.4 596.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 255.5 599] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (MPI_Process) 92.4 S 0 3.5 M ( _Create) 75.6 S setmatrix grestore gsave gsave matrix currentmatrix [1 0 0 1 413 599] concat newpath 0 0 M 0 28 L 94.4 28 L 94.4 0 L closepath setmatrix 0 0 0 setrgbcolor grestore newpath 410.5 596.5 M 410.5 629.5 L 509.9 629.5 L 509.9 596.5 L closepath clip newpath 0.560784 0.560784 0.87451 setrgbcolor matrix currentmatrix [1 0 0 1 413 599] concat /Courier-Bold-SHOWISO findfont 14 scalefont setfont 0 0 0 setrgbcolor 0 17.5 M (MPI_Process) 92.4 S 0 3.5 M ( _Attach) 67.2 S setmatrix grestore grestore showpage %%EndDocument @endspecial 552 788 a Fs(Figure)g(1.2:)k(Dynamic)d(Pro)q(cess)f(Managemen)o (t)75 922 y(as)f(a)f(bag)h(b)q(ecause)h(there)f(is)g(no)g(in)o(ternal)h (structure)f(visible)i(to)d(an)h(application)i(\(i.e.,)d(if)i(it)f(represen)o (ts)75 978 y(m)o(ultiple)i(CPUs,)e(there)g(is)h(no)f(w)o(a)o(y)f(for)h(an)g (application)i(to)d(tell)j(on)e(whic)o(h)h(of)f(those)g(CPUs)g(a)g(pro)q (cess)75 1035 y(is)k(running\).)29 b(An)18 b Fj(MPI)p 498 1035 14 2 v 16 w(Resource)h Fs(is)f(generally)h(reusable,)g(as)e(it)h(represen)o (ts)g(a)g(committmen)o(t)f(from)75 1091 y(an)e(external)h(resource)f(manager) g(that)f(resources)h(are)g(a)o(v)m(ailable.)166 1148 y(The)k(notion)h(of)f Fk(r)n(esour)n(c)n(e)f Fs(is)i(delib)q(erately)h(\015exible.)35 b(Most)18 b(applications)j(do)e(not)g(care)g(where)75 1204 y(pro)q(cesses)h(are)f(spa)o(wned,)i(as)e(long)h(as)g(there)f(is)i(one)e(pro) q(cess)h(p)q(er)h(CPU.)e(F)l(or)g(these)h(applications,)75 1261 y(a)e(single)i(resource)e(ob)s(ject)g(con)o(taining)h(sev)o(eral)f(CPUs) g(ma)o(y)g(b)q(e)g(appropriate.)30 b(Other)18 b(appications)75 1317 y(will)g(w)o(an)o(t)c(explicit)19 b(con)o(trol)c(o)o(v)o(er)g(whic)o(h)i (pro)q(cesses)f(go)g(on)g(whic)o(h)g(mac)o(hine.)23 b(In)17 b(this)f(case,)g(a)g(single)75 1373 y(resource)i(ob)s(ject)e(ma)o(y)h (corresp)q(ond)h(to)f(a)g(single)i(mac)o(hine.)27 b(Similarly)l(,)20 b(it)e(migh)o(t)f(b)q(e)i(con)o(v)o(enien)o(t)f(to)75 1430 y(think)e(of)f(an)g(SMP)g(as)g(a)g(single)h(\\no)q(de")f(or)g(as)g(a)g (collection)i(of)e(\(virtual\))g(pro)q(cessors.)166 1486 y(Within)c(MPI,)f (ho)o(w)o(ev)o(er,)g(a)g(resource)h(is)g(nothing)f(more)g(than)h(a)f(bag)g (of)f(pro)q(cess)i(slots.)18 b(Additional)75 1543 y(information)13 b(ab)q(out)f(what)g(ph)o(ysical)i(resources)f(the)g(pro)q(cesses)g(will)h (use)f(ma)o(y)f(b)q(e)h(kno)o(wn)g(to)f(and)g(used)75 1599 y(b)o(y)j(the)g(application,)i(but)e(is)h(not)f(imp)q(ortan)o(t)g(to)f(MPI.) 166 1656 y(Three)h(n)o(um)o(b)q(ers)h(c)o(haracterize)f(an)g Fj(MPI)p 880 1656 V 16 w(Resource)p Fs(.)131 1750 y(1.)22 b(Num)o(b)q(er)15 b(of)g(running)h(pro)q(cesses)131 1843 y(2.)22 b(Num)o(b)q(er)15 b(of)g(pro)q(cess)g(slots)h(\(\014lled)g(and)g(empt)o(y\))e(recommended)131 1937 y(3.)22 b(Num)o(b)q(er)15 b(of)g(pro)q(cess)g(slots)h(\(\014lled)g(and)g (empt)o(y\))e(total)166 2031 y(It)f(is)g(necessary)g(to)g(distinguish)i(b)q (et)o(w)o(een)e(the)g(total)f(n)o(um)o(b)q(er)h(of)g(pro)q(cess)g(slots)g (and)g(the)g(n)o(um)o(b)q(er)75 2087 y(of)j(pro)q(cesses)i(recommended)f(for) f(a)h(giv)o(en)g(resource.)25 b(F)l(or)16 b(instance,)i(if)f(a)g(resource)f (corresp)q(onds)i(to)75 2144 y(a)f(Unix)i(w)o(orkstation,)e(it)h(ma)o(y)f (theoreticaly)h(run)g(as)g(man)o(y)f(pro)q(cesses)h(as)f(Unix)i(can)f(supp)q (ort.)27 b(On)75 2200 y(the)17 b(other)g(hand,)g(it)g(should)h(probably)g (run)f(at)f(most)g(one)h(pro)q(cess)g(p)q(er)h(pro)q(cessor.)25 b(W)l(e)17 b(could)h(giv)o(e)75 2257 y(it)e(one)g(pro)q(cess)h(slot,)e(but)i (some)e(applications)j(will)g(need)e(to)g(b)q(e)g(able)h(to)f(do)g(more.)21 b(W)l(e)c(don't)e(w)o(an)o(t)75 2313 y(to)j(tell)h(the)g(user,)g(ho)o(w)o(ev) o(er,)f(that)g(there)h(are)f(100)g(slots)g(a)o(v)m(ailable)i(b)q(ecause)g (there)e(will)i(b)q(e)g(no)e(w)o(a)o(y)75 2370 y(to)d(distinguish)i(this)f (resource)f(from)g(one)g(with)h(100)e(CPUs.)20 b(W)l(e)c(ma)o(y)e(ha)o(v)o(e) h(an)h(SMP)l(,)e(but)i(our)f(MPI)75 2426 y(pro)q(cesses)g(ma)o(y)f(b)q(e)h(m) o(ultithreaded)g(pro)q(cesses,)g(so)f(that)g(w)o(e)g(still)i(w)o(an)o(t)d (only)i(one)g(pro)q(cess)g(p)q(er)g(no)q(de.)75 2483 y(Th)o(us)c (\\recommended)i(pro)q(cesses")e(do)q(esn't)h(alw)o(a)o(ys)e(mean)i(\\n)o(um) o(b)q(er)g(of)f(CPUs.")18 b(Most)10 b(applications)75 2539 y(can)19 b(lo)q(ok)g(at)f(recommended)i(pro)q(cesses)f(and)h(ignore)f(max)f (pro)q(cesses.)31 b(Finally)l(,)22 b(for)c(debugging,)i(a)75 2596 y(user)d(migh)o(t)f(w)o(an)o(t)f(to)h(\\fo)q(ol")g(an)g(application)i(b) o(y)e(suggesting)h(that)e(it)i(start)e(m)o(ultiple)j(pro)q(cesses)f(on)75 2652 y(a)e(single)h(w)o(orkstation.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 bop 75 -100 a Fm(1.3.)33 b(NEW)15 b(OBJECTS)g(F)o(OR)g(IMPLEMENTING)g(D)o (YNAMIC)f(PR)o(OCESS)h(MANA)o(GEMENT)p Fs(9)166 45 y Fg(Discussion:)32 b Ff(Suggestions)13 b(for)f(an)h(alternativ)o(e)f(name)f(to)i(\\recommended") e(are)i(w)o(elcome.)j(Thanks)d(to)75 102 y(Ian)f(F)m(oster)g(for)g(making)d (clear)j(the)h(need)f(to)g(distinguish)f(b)q(et)o(w)o(een)j(max/recomm)o (ended)c(\(hop)q(e)i(I)g(got)f(it)h(righ)o(t!\))75 158 y(-)i(w)o(cs)166 297 y Fs(MPI)i(resources)h(are)f(o)o(wned)h(b)o(y)f(a)g(single)i(MPI)e(pro)q (cess.)24 b(Ownership)18 b(cannot)f(b)q(e)g(transferred.)75 354 y(The)22 b(resources)f(used)h(b)o(y)f(pro)q(cesses)h(constituting)g Fj(MPI)p 1102 354 14 2 v 15 w(COMM)p 1259 354 V 17 w(W)o(ORLD)f Fs(cannot)g(b)q(e)h(reused)g(b)o(y)75 410 y(those)d(pro)q(cesses)g(\(but)g (ma)o(y)g(b)q(e)h(reused)f(b)o(y)g(their)h(paren)o(ts\).)31 b(Preallo)q(cated)20 b(resources)f(are)g(o)o(wned)75 467 y(b)o(y)14 b(pro)q(cess)h(0)f(of)h Fj(MPI)p 469 467 V 15 w(COMM)p 626 467 V 17 w(W)o(ORLD)f Fs(unless)i(sp)q(eci\014ed)h(otherwise)d(\(externally)l (,)h(when)g(the)g(job)f(is)75 523 y(launc)o(hed\).)166 655 y Fg(Discussion:)53 b Ff(Resource)19 b(o)o(wnership)g(can)g(b)q(ecome)f(a)h (thorn)o(y)f(issue.)33 b(The)19 b(ab)q(o)o(v)o(e)f(approac)o(h)h(is)f(the)75 705 y(simplest)13 b(but)h(also)f(the)i(least)f(\015exible.)k(W)m(e)13 b(ma)o(y)f(w)o(an)o(t)i(to)f(in)o(tro)q(duce)i(mec)o(hanisms)d(to)i(allo)o(w) 145 779 y Fc(\017)23 b Ff(T)m(ransfer)14 b(of)f(resource)j(o)o(wnership,)e (esp)q(ecially)g(to)g(c)o(hildren.)145 852 y Fc(\017)23 b Ff(Reuse)14 b(of)g(resources)i(used)f(b)o(y)e Fe(MPI)p 764 852 13 2 v 15 w(COMM)p 909 852 V 14 w(W)o(ORLD)p Ff(.)166 926 y(Collectiv)o(e)g(o)o (wnership)i(app)q(ears)f(to)g(b)q(e)g(a)g(mess)g(without)f(an)o(y)h (redeeming)f(features.)166 982 y(Another)f(issue)g(is)f(what)h(happ)q(ens)g (to)f(resources)j(that)e(expire.)17 b(Needs)c(to)e(b)q(e)h(nailed)f(do)o(wn,) g(but)h(hop)q(efully)75 1039 y(isn't)h(to)q(o)h(m)o(uc)o(h)f(of)g(a)h (problem,)e(esp.)19 b(if)13 b(resources)j(aren't)e(collectiv)o(e.)75 1243 y Fl(1.3.2)49 b(The)16 b(MPI)p 409 1243 15 2 v 18 w(Pro)q(cess)g(Object) 75 1404 y Fg(Discussion:)38 b Ff(In)15 b(MPI-1,)g(pro)q(cesses)j(can)d(b)q(e) h(iden)o(ti\014ed)f(b)o(y)g(their)g(rank)g(in)g(a)f(group)h(or)g(comm)o (unicator.)k(It)75 1454 y(migh)o(t)12 b(b)q(e)k(p)q(ossible)f(to)f(con)o(tin) o(ue)h(to)f(use)i(a)e(\(group,rank\))g(pair)g(to)h(iden)o(tify)f(a)g(pro)q (cess)i(as)f(required)h(for)e(MPI-)75 1504 y(2.)29 b(While)17 b(con)o(v)o(enien)o(t)h(for)g(some)f(op)q(erations)h(\(e.g.,)f(spa)o(wning)h (a)f(group)h(of)f(pro)q(cesses\),)k(the)d(\(group,rank\))75 1554 y(iden)o(ti\014cation)d(w)o(ould)g(b)q(e)h(incon)o(v)o(enien)o(t)f(or)h (unnatural)f(for)g(others)h(\(e.g.,)f(sending)h(a)f(signal\).)22 b(Moreo)o(v)o(er,)16 b(an)75 1604 y Fe(MPI)p 152 1604 13 2 v 14 w(Group)c Ff(ob)r(ject)h(pro)o(vides)e(an)h(abstraction)f(for)g(a)h (collection)f(of)g(pro)q(cesses)j(whic)o(h)d(is)h(an)f(unnatural)g(starting) 75 1653 y(p)q(oin)o(t)h(to)g(de\014ne)i(a)e(single)g(pro)q(cess.)19 b(Therefore,)14 b(in)e(the)h(follo)o(wing)d(w)o(e)j(in)o(tro)q(duce)g(a)f (new)h(ob)r(ject,)g Fe(MPI)p 1720 1653 V 14 w(Pro)q(cess)p Ff(.)75 1703 y(When)f(the)g(b)q(eha)o(vior)f(of)f(this)i(ob)r(ject)g(is)f (agreed)h(up)q(on)g(and)f(understo)q(o)q(d,)i(w)o(e)e(can)h(decide)g(among)e (three)i(options)134 1777 y(1.)22 b(Absorb)14 b Fe(MPI)p 410 1777 V 14 w(Pro)q(cess)h Ff(in)o(to)e Fe(MPI)p 729 1777 V 14 w(Group)134 1850 y Ff(2.)22 b(Ha)o(v)o(e)13 b(t)o(w)o(o)h(separate)h(ob)r (jects)g(\(with)f(p)q(erhaps)h(additional)d(accessor/creation)k(functions\)) 134 1924 y(3.)22 b(Go)13 b(with)g Fe(MPI)p 427 1924 V 15 w(Pro)q(cess)h Ff(and)g(deprecate)h Fe(MPI)p 929 1924 V 15 w(Group)166 2087 y Fs(An)k Fj(MPI)p 329 2087 14 2 v 16 w(Pro)q(cess)g Fs(represen)o(ts)g(a)g (pro)q(cess)g({)f(a)h(program)f(executing)i(on)e(a)h(pro)q(cessor.)31 b(An)19 b(An)75 2143 y Fj(MPI)p 160 2143 V 16 w(Pro)q(cess)c Fs(is)g(a)g(\\pro)q(cess",)f(not)g(a)g(\\thread")g(in)i(that)e(pro)q(cesses)h (ha)o(v)o(e)f(their)h(o)o(wn)f(address)h(spaces)75 2200 y(and)h(don't)g (share)g(date)f(unless)j(they)e(tak)o(e)f(explicit)j(action)e(to)g(do)g(so)f (\(outside)i(of)e(MPI\).)h(A)g(pro)q(cess)75 2256 y(ma)o(y)e(or)h(ma)o(y)g (not)f(b)q(e)i(\\registered")f(with)h(MPI;)f(it)g(ma)o(y)g(or)f(ma)o(y)h(not) g(call)h Fj(MPI)p 1491 2256 V 16 w(INIT)p Fs(.)166 2395 y Fg(Discussion:)44 b Ff(P)o(eople)17 b(ha)o(v)o(e)f(b)q(een)h(confused)h(b)o(y)e(the)h(pro)q (cess)h(vs.)26 b(thread)17 b(distinction)f(and)g(the)h(fact)75 2452 y(that)d(an)g Fe(MPI)p 300 2452 13 2 v 14 w(Pro)q(cess)g Ff(ma)o(y)e(not)i(b)q(e)g(an)g(MPI)g(Pro)q(cess.)166 2591 y Fs(Y)l(ou)h(can't)g(comm)o(unicate)g(directly)i(with)e(an)g Fj(MPI)p 1066 2591 14 2 v 16 w(Pro)q(cess)h Fs(\(that)e(requires)i(a)f(comm)o (unicator\),)75 2647 y(but)g(y)o(ou)f(can)h(send)g(signals)g(to)f(one.)20 b(If)15 b(it)f(is)h(an)g(MPI)f(pro)q(cess)h(in)h(the)e(sense)h(that)f(it)h (has)f(a)h(rank)f(in)h(a)75 2704 y(comm)o(unicator,)d(then)h(the)f Fj(MPI)p 643 2704 V 16 w(Pro)q(cess)h Fs(ob)s(ject)f(represen)o(ting)h(it)g (can)f(b)q(e)h(retriev)o(ed)g(from)f(its)g(rank)g(in)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 bop 75 -100 a Fs(10)933 b Fm(CHAPTER)16 b(1.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fs(that)d(comm)o(unicator)f(\(See)i Fj(MPI)p 645 45 14 2 v 16 w(COMM)p 803 45 V 16 w(PROCESS)h Fs(b)q(elo)o(w\).)19 b(MPI)12 b(pro)o(vides)h(no)f(guaran)o(tees)g(on)g(the) 75 102 y(order)e(of)g(op)q(erations)h(with)g(resp)q(ect)g(to)f(an)g Fj(MPI)p 897 102 V 16 w(Pro)q(cess)i Fs(and)e(pro)o(vides)h(none)g(of)f(the)h (safet)o(y)f(asso)q(ciated)75 158 y(with)18 b(comm)o(unicators.)28 b(In)18 b(particular,)h(there)f(is)h(no)f(guaran)o(tee)f(of)g(order)h(b)q(et) o(w)o(een)g(messages)g(and)75 214 y(signals.)166 354 y Fg(Discussion:)34 b Ff(A)14 b(bad)g(programming)d(practice)k(that)f(has)g(b)q(een)h(seen)g(b)o (y)f(naiv)o(e)g(users)h(of)f(systems)g(that)75 410 y(pro)o(vide)f(b)q(oth)g (messages)h(and)f(signals)f(is)i(to)f(send)h(a)f(pro)q(cess)i(a)e(message)g (and)g(then)h(signal)e(it)h(to)g(w)o(ak)o(e)g(up)g(and)75 467 y(receiv)o(e)i(the)g(message.)i(This)d(needs)h(to)f(b)q(e)h(plainly)d (forbidden.)18 b(-)13 b(A)o(G.)75 692 y Fp(1.4)59 b(Simple)20 b(Interface)75 870 y Fg(Discussion:)39 b Ff(\(w)o(cs\))17 b Fe(MPI)p 537 870 13 2 v 14 w(SP)m(A)-5 b(WN)p Ff(,)14 b(as)i(presen)o(ted)h (earlier,)e(b)q(oth)h(allo)q(cated)f(resources)i(and)e(spa)o(wned)h(pro-)75 920 y(cesses.)33 b(This)18 b(w)o(as)g(problematic)e(b)q(ecause)k(\(1\))e (almost)e(alw)o(a)o(ys,)h(y)o(ou)h(w)o(an)o(t)g(to)f(spa)o(wn)h(on)g(preallo) q(cated)g(re-)75 970 y(sources,)j(not)d(allo)q(cate)g(new)g(ones)h(and)f (\(2\))h(the)g(syn)o(tax)f(required)h(y)o(ou)f(to)g(kno)o(w)g(ho)o(w)g(man)o (y)e(pro)q(cesses)21 b(to)75 1019 y(spa)o(wn,)13 b(whic)o(h)h(y)o(ou)f(had)h (no)f(w)o(a)o(y)g(of)h(\014nding)f(out.)18 b(\(These)d(are)f(the)h(same)d (issues)j(that)f(motiv)n(ated)e(separating)75 1069 y(resource)k(disco)o(v)o (ery)e(and)g(resource)i(allo)q(cation.)g(\))166 1119 y(The)j(new)g Fe(MPI)p 423 1119 V 14 w(SP)m(A)-5 b(WN)19 b Ff(tak)o(es)g(a)f(resource)j (argumen)o(t)c(to)i(a)o(v)o(oid)e(these)j(problems.)31 b(This)19 b(mak)o(es)e(the)75 1169 y(\\simple")12 b(in)o(terface)i(a)g(t)o(w)o(o)f (stage)i(pro)q(cess:)134 1244 y(1.)22 b(Disco)o(v)o(er)12 b(existing)g (resources)i(with)e Fe(MPI)p 852 1244 V 15 w(RESOURCE)p 1081 1244 V 12 w(DISCO)o(VER)e Ff(\(using)i(a)g(simple)f(form)f(of)i(the)h(sub-) 189 1293 y(routine)h(to)f(get)i(a)e(single)h(resource\))134 1368 y(2.)22 b(Spa)o(wn)17 b(on)o(to)h(this)g(resource)i(with)e Fe(MPI)p 850 1368 V 14 w(SP)m(A)-5 b(WN)p Ff(.)18 b(This)f(will)g(b)q(e)i(a)e (single)h(call)f(that)i(will)d(b)q(oth)i(start)189 1418 y(pro)q(cesses)e(and) e(attac)o(h)g(to)g(them.)166 1493 y(The)g(alternativ)o(e)g(is)g(an)f Fe(MPI)p 635 1493 V 15 w(SP)m(A)-5 b(WN)13 b Ff(with)h(the)g(follo)o(wing)e (prop)q(erties:)145 1567 y Fc(\017)23 b Ff(Automatically)15 b(starts)k(pro)q(cesses)i(on)d(preallo)q(cated)g(resources,)j(in)d(a)f (roundrobin)h(fasion)g(similar)d(to)189 1617 y(PVM)145 1692 y Fc(\017)23 b Ff(The)13 b(user)g(m)o(ust)e(\014nd)i(out)f(through)h(some)e (external)i(mec)o(hanism)d(ho)o(w)i(man)o(y)f(pro)q(cesses)k(to)d(spa)o(wn)h (\(this)189 1742 y(will)g(probably)h(b)q(e)i(hardco)q(ded)g(or)e(in)h(a)f (con\014g)h(\014le,)g(and)g(is)f(redundan)o(t)i(with)e(information)e(that)j (could)189 1791 y(b)q(e)f(obtained)g(from)e Fe(MPI)p 591 1791 V 14 w(RESOURCE)p 819 1791 V 13 w(DISCO)o(VER)p Ff(.)75 1866 y(\).)166 1916 y(The)17 b(new)h(approac)o(h)f(also)g(has)g(an)g(adv)n(an)o (tage)f(of)h(eliminating)d(the)j(\\other)h(args")f(a)o(wkw)o(ardness)h(from) 75 1966 y Fe(MPI)p 152 1966 V 14 w(SP)m(A)-5 b(WN)p Ff(.)19 b(Normally)f(these)j(w)o(ouldn't)e(b)q(e)h(used.)37 b(If)19 b(the)i(user)f(w)o(an)o(ts)g(the)h(extra)f(functionalit)o(y)e(\(e.g.,)75 2016 y(to)h(spa)o(wn)f(on)h(sp)q(eci\014c)h(hosts)g(or)f(arc)o(hitectures\))i (he/she)f(can)f(use)g(a)g(sligh)o(tly)e(more)h(complicated)g(form)f(of)75 2065 y Fe(MPI)p 152 2065 V 14 w(RESOURCE)p 380 2065 V 13 w(DISCO)o(VER)p Ff(.)166 2115 y(Another)23 b(problem)e(with)h Fe(MPI)p 687 2115 V 14 w(SP)m(A)-5 b(WN)22 b Ff(is)g(that)h(to)f(b)q(e)h(simple)e(it)g (should)i(return)g(a)f(regular)g(com-)75 2165 y(m)o(unicator,)d(not)h(an)g (in)o(tercomm)o(unicator.)33 b(With)20 b(the)g(curren)o(t)h(b)q(eha)o(vior)f (of)f Fe(MPI)p 1471 2165 V 15 w(COMM)p 1616 2165 V 14 w(P)m(ARENT)g Ff(and)75 2215 y Fe(MPI)p 152 2215 V 14 w(PROCESS)p 349 2215 V 14 w(A)m(TT)m(A)o(CH)10 b Ff(this)h(w)o(ould)g(require)i(the)f(c)o(hild)f (to)g(do)h(an)f Fe(MPI)p 1270 2215 V 14 w(INTERCOMM)p 1535 2215 V 15 w(MERGE)p Ff(.)f(Since)i(the)75 2265 y(user)18 b(has)g(to)f(deal)f (with)h Fe(MPI)p 563 2265 V 15 w(INTERCOMM)p 829 2265 V 14 w(MERGE)f Ff(in)h(an)o(y)g(case,)h Fe(MPI)p 1312 2265 V 15 w(SP)m(A)-5 b(WN)17 b Ff(no)o(w)f(returns)j(an)e(in)o(ter-)75 2314 y(comm)o(unicator.)166 2371 y(P)o(erhaps)e(w)o(e)f(should)g(consider)g (eliminating)d Fe(MPI)p 973 2371 V 14 w(SP)m(A)-5 b(WN)14 b Ff(altogether?)75 2560 y Fi(MPI_Spawn\(oldcomm,)21 b(root,)j(count,)f (executable,)f(args,)h(resource,)g(newcomm\);)166 2666 y Fs(can)15 b(b)q(e)h(implemen)o(ted)h(as:)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 bop 75 -100 a Fm(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(11)75 45 y Fi(MPI_Process)22 b(*processes)h(=)h(NULL;)75 102 y(if)g(\(myid)f(==)g(0\))h({)123 158 y(processes)e(=)i (malloc\(sizeof\(MPI_Process\)*)o(count\);)123 214 y (MPI_Process_create\(execu)o(table,)c(args,)k(count,)f (MPI_PROCESS_TYPE_CHILD,)576 271 y(resource,)g(processes\);)75 327 y(})75 384 y(MPI_Process_attach\(comm,)e(0,)i(count,)g(processes,)g (newcomm\);)75 440 y(if)h(\(myid)f(==)g(0\))h(free\(processes\);)166 603 y Fs(In)f(this)f(section)h(w)o(e)f(pro)o(vide)h(high-lev)o(el)h (functions)f(b)q(oth)g(for)e(resource)i(disco)o(v)o(ery)l(,)h(pro)q(cess)75 659 y(creation)d(and)f(for)g(clien)o(t-serv)o(er)h(requiremen)o(ts)g(that)f (are)g(the)g(easiest)h(to)f(use.)36 b(These)20 b(pro)o(vide)h(a)75 716 y(useful)16 b(subset)e(of)g(op)q(erations;)h(routines)g(that)f(pro)o (vide)h(more)f(con)o(trol)g(and)h(\015exibilit)o(y)i(are)d(describ)q(ed)75 772 y(later.)27 b(Indeed,)19 b(the)f(functions)g(in)h(this)f(section)g(can)g (b)q(e)g(la)o(y)o(ered)g(on)f(those)g(routines,)i(describ)q(ed)g(in)75 829 y(Sections)d Fo(??)p Fs(,)e Fo(??)p Fs(,)g(and)i Fo(??)p Fs(.)75 949 y Fj(Spa)o(wning)h(p)o(ro)q(cesses)47 b Fs(In)16 b(the)g(most)e(common)i(case,)f(an)g(application)j(will)f(simply)g(spa)o(wn)e (new)h(pro-)75 1005 y(cesses)f(on)f(preallo)q(cated)i(resources.)k(This)15 b(is)g(a)f(t)o(w)o(o)f(step)i(pro)q(cess.)20 b(First)14 b(the)h(application)h (disco)o(v)o(ers)75 1062 y(its)f(resources,)g(then)h(it)f(spa)o(wns)g(a)g (program.)166 1118 y(Detailed)21 b(mec)o(hanisms)f(for)f(obtaining)i (resources)f(are)f(describ)q(ed)j(in)f(Section)f Fo(??)p Fs(.)34 b(F)l(or)19 b(most)75 1175 y(applications,)d(the)g(follo)o(wing)g(is)f (su\016cien)o(t:)147 1281 y Fi(MPI_Resource_discover\(M)o(PI_RESO)o(URCE_TYP) o(E_DEFAU)o(LT,)21 b(NULL,)i(&resource\))147 1337 y(MPI_Resource_nslots\(res) o(ource,)d(&nslots\))166 1444 y Fs(This)15 b(pro)q(duces)h(a)f(resource)g(ob) s(ject)f(on)h(whic)o(h)g(to)g(spa)o(wn)f(pro)q(cesses)h(and)g(tells)h(the)f (application)75 1500 y(ho)o(w)g(man)o(y)f(pro)q(cesses)i(it)f(ma)o(y)g(spa)o (wn.)166 1557 y(The)g(follo)o(wing)h(function)g(is)g(the)g(easiest)f(w)o(a)o (y)f(to)h(create)g(new)h(pro)q(cesses,)f(and)g(is)h(most)e(lik)o(e)j(the)75 1613 y(PVM)g(routine)g Fi(pvm)p 432 1613 15 2 v 17 w(spawn)p Fs(.)24 b(It)17 b(starts)f(the)h(same)f(executable)j(with)e(the)g(same)g (argumen)o(t)f(list)h(on)g(a)75 1670 y(set)c(of)g(pro)q(cessors.)19 b(It)13 b(is)h(a)f(collectiv)o(e)i(op)q(eration)e(o)o(v)o(er)g(the)g(pro)q (cesses)h(in)g(the)f(group)g(asso)q(ciated)h(with)75 1726 y Fj(oldcomm)p Fs(,)d(and)i(returns)g(an)g(in)o(ter-comm)o(unicator)f Fj(new)o(comm)p Fs(,)g(whic)o(h)i(has)e(the)h(new)g(pro)q(cesses)g(as)g(the) 75 1782 y(remote)18 b(group.)30 b(If)19 b(an)f(expanded)i(comm)o(unicator)e (is)i(desired,)g(an)f(ordinary)f(in)o(tra-comm)o(unicator)75 1839 y(con)o(taining)12 b(all)f(pro)q(cesses)g(can)g(then)g(b)q(e)h (constructed)f(using)g Fj(MPI)p 1218 1839 14 2 v 16 w(INTERCOMM)p 1508 1839 V 17 w(MERGE)p Fs(.)g(Only)h(one)75 1895 y(of)17 b(the)h(pro)q(cesses)h(in)f Fj(oldcomm)f Fs(m)o(ust)g(supply)i(the)f(argumen) o(ts)f(that)g(describ)q(e)j(the)e(new)g(pro)q(cesses;)75 1952 y(that)c(pro)q(cess)i(is)g(designated)g(b)o(y)f Fj(ro)q(ot)p Fs(.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 bop 75 -100 a Fs(12)928 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(SP)l(A)-5 b(WN\(oldcomm,)13 b(ro)q(ot,)i(count,)h(executable,)h(a)o(rgs,)d(resource,)h (new)o(comm\))117 122 y Ff(IN)155 b Fj(oldcomm)413 b Ff(comm)o(unicator)11 b(of)j(spa)o(wning)f(group)117 197 y(IN)155 b Fj(ro)q(ot)508 b Ff(o)o(wner)14 b(of)g(resource)117 273 y(IN)155 b Fj(count)482 b Ff(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(to)e(spa)o(wn)g(\(in)o(t\))117 348 y(IN)155 b Fj(executable)391 b Ff(executable)15 b(\014le)f(for)g(new)g (pro)q(cesses)j(to)c(execute)j(\(string\))117 423 y(IN)155 b Fj(a)o(rgs)510 b Ff(argumen)o(ts)13 b(for)h(the)g(spa)o(wned)h(pro)q (cesses)117 498 y(IN)155 b Fj(resource)432 b Ff(resource)16 b(to)e(spa)o(wn)g(on)o(to.)j(Signi\014can)o(t)c(only)g(at)h Fe(ro)q(ot)117 573 y Ff(OUT)108 b Fj(new)o(comm)397 b Ff(new)21 b(in)o(ter-comm)o(unicator)d(including)i(new)h(pro)q(cesses)i(as)905 630 y(the)15 b(remote)e(group)166 830 y Fg(Discussion:)i Ff(F)m(or)f(ease)g (of)f(use,)h(w)o(e)g(ma)o(y)e(w)o(an)o(t)h(to)h(create)h(a)e(regular)h(comm)o (unicator,)c(con)o(taining)j(b)q(oth)75 880 y(new)j(and)f(old)g(pro)q (cesses,)j(so)e(that)g(casual)f(users)i(don't)e(ha)o(v)o(e)g(to)h(w)o(orry)f (ab)q(out)h(in)o(tercomm)o(unicators,)d(whic)o(h)75 930 y(can)h(b)q(e)h (confusing.)166 986 y(This)h(ma)o(y)d(not)j(w)o(ork)f(b)q(ecause)j(there)f (is)f(no)f(ob)o(vious)g(w)o(a)o(y)g(to)h(create)h(a)e(regular)h(comm)o (unicator)d(in)i(the)75 1043 y(c)o(hildren.)166 1182 y Fs(The)20 b(spa)o(wned)f(pro)q(cesses)h(are)f(created)h(with)g Fj(MPI)p 1089 1182 V 15 w(COMM)p 1246 1182 V 17 w(W)o(ORLD)f Fs(consisting)h(of)g(the) f(pro-)75 1238 y(cesses)g(spa)o(wned)g(with)f(this)h(call.)31 b(The)19 b(call)h(blo)q(c)o(ks)f(un)o(til)h(all)f(pro)q(cesses)g(in)h Fj(oldcomm)c Fs(ha)o(v)o(e)i(called)75 1295 y Fj(MPI)p 160 1295 V 16 w(SP)l(A)-5 b(WN)12 b Fs(and)f(all)h(the)f(spa)o(wned)h(pro)q (cesses)f(ha)o(v)o(e)g(called)h Fj(MPI)p 1234 1295 V 16 w(INIT)f Fs(and)g Fj(MPI)p 1516 1295 V 16 w(COMM)p 1674 1295 V 16 w(P)l(ARENT)p Fs(,)75 1351 y(after)k(whic)o(h)i(comm)o(unication)g(is)g(p)q(ossible.)25 b(Note)15 b(that)h(this)h(means)f(that)f Fj(MPI)p 1495 1351 V 16 w(COMM)p 1653 1351 V 16 w(W)o(ORLD)h Fs(is)75 1408 y(di\013eren)o(t)c (on)h(the)f(spa)o(wners)g(and)g(spa)o(wnees.)19 b(Also)12 b(note)g(that)g(b)o (y)g(using)h Fj(MPI)p 1430 1408 V 16 w(COMM)p 1588 1408 V 16 w(SELF)p Fs(,)f(a)g(single)75 1464 y(pro)q(cess)17 b(can)h(create)f(man)o(y)f (others;)h(this)h(is)g(useful)g(b)q(oth)f(for)g(master-sla)o(v)o(e)f (programs)g(and)h(for)g(dy-)75 1520 y(namic)d(sizing)h(of)e(parallel)i(jobs)e (from)g(a)g(single)i(initial)g(pro)q(cess.)20 b(This)14 b(function)g(is)g (similar)h(to)d(the)i(one)75 1577 y(describ)q(ed)k(in)g([)p Fo(?)p Fs(],)d(but)i(here)f(the)h(argumen)o(ts)e(are)i(made)f(explicit)j (instead)e(of)f(b)q(eing)h(com)o(bined)h(in)f(a)75 1633 y(single)g(string.)k (In)16 b(the)f(case)h(where)f(job-sc)o(heduler-dep)q(end)q(en)o(t)j (information)e(m)o(ust)f(b)q(e)h(supplied,)h(w)o(e)75 1690 y(do)e(use)h(a)f(string,)f(in)i(the)g(more)e(\015exible)k Fj(MPI)p 882 1690 V 16 w(RESOURCE)p 1128 1690 V 18 w(IALLOCA)l(TE)d Fs(describ)q(ed)i(in)f(Section)h Fo(??)p Fs(.)75 1810 y Fj(Client-Server)11 b(Applications)48 b Fs(The)11 b(ab)q(o)o(v)o(e)f(paragraphs)g(co)o(v)o(ered)g (the)h(case)g(of)f(creating)h(new)g(pro)q(cesses.)75 1866 y(F)l(or)17 b(clien)o(t-serv)o(er)i(applications,)h(the)e(situation)g(is)h(di\013eren)o (t,)f(b)q(ecause)h(the)f(pro)q(cesses)h(in)f(question)75 1923 y(already)h(exist,)g(and)f(what)g(w)o(e)g(need)h(is)f(a)g(comm)o(unicator)g (to)g(b)q(e)h(used)g(b)o(y)f(them)g(to)g(comm)o(unicate)75 1979 y(with)e(one)f(another.)166 2036 y(The)f(follo)o(wing)g(are)f(the)h (simplest,)h(blo)q(c)o(king)f(calls.)21 b(The)14 b(non-blo)q(c)o(king)h(v)o (ersions)f(are)f(describ)q(ed)75 2092 y(in)j(Section)g Fo(??)p Fs(.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 bop 75 -100 a Fm(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(13)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(CLIENT)p 333 45 V 16 w(CONNECT\(mycomm,)12 b(name,)j(new)o(comm\))117 122 y Ff(IN)155 b Fj(mycomm)412 b Ff(comm)o(unicator)10 b(of)h(the)i(clien)o(t,)f (o)o(v)o(er)g(whic)o(h)h(this)f(call)f(is)h(col-)905 179 y(lectiv)o(e)117 254 y(IN)155 b Fj(name)485 b Ff(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)g(whic)o(h) h(the)h(serv)o(er)g(can)f(b)q(e)h(con)o(tacted)905 310 y(\(string\))117 385 y(OUT)108 b Fj(new)o(comm)397 b Ff(new)10 b(in)o(ter-comm)o(unicator,)d (whic)o(h)i(includes)h(the)g(serv)o(er)h(pro-)905 442 y(cesses)17 b(as)c(the)i(remote)e(group)75 661 y Fj(MPI)p 160 661 V 16 w(SERVER)p 343 661 V 18 w(CONNECT\(mycomm,)f(name,)j(new)o(comm\))117 738 y Ff(IN)155 b Fj(mycomm)412 b Ff(comm)o(unicator)18 b(of)i(the)h(serv)o (er,)j(o)o(v)o(er)c(whic)o(h)h(this)f(call)g(is)905 795 y(collectiv)o(e)117 870 y(IN)155 b Fj(name)485 b Ff(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)g(whic)o(h) h(the)h(serv)o(er)g(can)f(b)q(e)h(con)o(tacted)905 926 y(\(string\))117 1001 y(OUT)108 b Fj(new)o(comm)397 b Ff(new)10 b(in)o(ter-comm)o(unicator,)d (whic)o(h)i(includes)h(the)g(clien)o(t)f(pro-)905 1058 y(cesses)17 b(as)c(the)i(remote)e(group)166 1182 y Fs(Then)j(an)o(y)g(pro)q(cess)g(in)g (the)g(clien)o(t)h(group)e(can)h(comm)o(unicate)g(with)h(an)o(y)e(pro)q(cess) h(in)h(the)e(serv)o(er)75 1239 y(group)g(and)g(vice)h(v)o(ersa,)f(using)h (the)f(in)o(ter-comm)o(unicator.)166 1295 y(Disconnection)c(o)q(ccurs)g(when) g(pro)q(cesses)f(call)h Fj(MPI)p 1057 1295 V 16 w(COMM)p 1215 1295 V 17 w(FREE)f Fs(on)g(the)h(in)o(ter-comm)o(unicator.)166 1352 y(The)17 b(form)g(of)f(the)i Fj(name)f Fs(argumen)o(t)f(has)h(sev)o (eral)g(p)q(ossibilitie)q(s.)28 b(The)18 b(most)e(ob)o(vious)h(is)h(to)f(use) 75 1408 y(the)c Fi(net-address:port-number)d Fs(format)h(that)h(curren)o(t)h (systems)f(will)j(\014nd)e(most)f(straigh)o(tforw)o(ard.)75 1464 y(Ho)o(w)o(ev)o(er,)i(in)i(the)f(long)h(run,)f(name)g(serv)o(ers)g(of)f (v)m(arious)i(kinds)g(ma)o(y)f(require)h(more)f(\015exibilit)o(y)l(.)75 1586 y Fl(1.4.1)49 b(Interfacing)16 b(to)h(the)e(Resource)g(Manager)75 1672 y Fs(This)h(section)g(describ)q(es)g(an)g(in)o(terface)f(for)g (obtaining)h(resources)f(from)f(a)h(resource)g(manager.)166 1728 y(There)c(are)f(t)o(w)o(o)f(w)o(a)o(ys)g(that)h(applications)i(t)o (ypically)f(use)g(new)g(resources.)18 b(In)11 b(the)f(\014rst,)h(they)f(mak)o (e)75 1785 y(use)16 b(of)f(existing,)i(pre-allo)q(cated)g(resources.)k(F)l (or)15 b(instance,)i(in)f(PVM,)f(a)h(user)f(ma)o(y)g(create)h(a)f(virtual)75 1841 y(mac)o(hine,)g(and)f(than)g(start)f(a)h(master)f(pro)q(cess)h(on)g (that)g(mac)o(hine.)20 b(The)14 b(master)g(pro)q(cess)g(determines)75 1898 y(ho)o(w)i(man)o(y)h(hosts)f(are)h(a)o(v)m(ailable)i(and)e(spa)o(wns)g (sla)o(v)o(e)f(pro)q(cesses)i(on)f(the)g(un)o(used)h(hosts.)24 b(Resources)75 1954 y(are)15 b(disco)o(v)o(ered,)i(not)e(allo)q(cated,)h(b)o (y)g(the)g(master)f(pro)q(cess.)21 b(The)16 b(second)h(w)o(a)o(y)d(is)j(for)e (an)g(application)75 2011 y(to)d(obtain)h(new)g(resources)g(explicitly)l(.)22 b(In)13 b(PVM,)f(this)i(is)f(done)g(through)g(the)f Fi(pvm)p 1503 2011 15 2 v 17 w(addhosts)g Fs(routine.)166 2067 y(It)e(is)h(p)q (ossible)h(to)d(treat)h(disco)o(v)o(ery)g(and)g(allo)q(cation)i(in)f(the)f (same)g(w)o(a)o(y)l(,)g(using)h(a)f(single)h Fj(ALLOCA)l(TE)75 2124 y Fs(function.)27 b(This)18 b(requires)g(in)g(the)f(case)g(of)g (pre-allo)q(cated)i(resources)e(that)f(an)i(application)g(kno)o(w)f(in)75 2180 y(adv)m(ance)j(what)f(resources)h(are)f(a)o(v)m(ailable)i(so)e(that)g (it)h(kno)o(ws)f(what)g(to)g(\\allo)q(cate".)33 b(W)l(e)19 b(therefore)75 2237 y(distinguish)g(b)q(et)o(w)o(een)f(allo)q(cation)g(and)g (disco)o(v)o(ery)f(of)g(resources.)26 b(Resource)18 b(disco)o(v)o(ery)g(is)g (exp)q(ected)75 2293 y(to)f(b)q(e)h(a)g(fast)f(op)q(eration)g(that)g(ma)o(y)g (b)q(e)i(p)q(ossible)g(to)e(implemen)o(t)i(through)f(in)o(ternal)g(lo)q(okup) h(rather)75 2349 y(than)10 b(external)h(comm)o(unication)h(\(dep)q(ending)g (on)f(the)f(MPI)h(implemen)o(tation\).)19 b(Resource)11 b(allo)q(cation,)75 2406 y(on)i(the)h(other)f(hand,)h(can)f(b)q(e)h(time-consuming,)h(and)e(is)h (therefore)f(a)g(non-blo)q(c)o(king)j(op)q(eration)d(whic)o(h)75 2462 y(returns)i(a)g(request)g(to)g(b)q(e)h(w)o(aited)f(on)g(later.)166 2519 y(Neither)h(function)f(starts)f(an)o(y)h(application-visibl)q(e)j(pro)q (cesses;)d(rather,)f(they)h(obtain)g(resources)75 2575 y(for)f(use)h(b)o(y)f (other)g(functions.)20 b(Ho)o(w)o(ev)o(er,)14 b(w)o(e)g(need)h(to)f(tak)o(e)g (in)o(to)g(accoun)o(t)g(those)g(resource)h(managers)75 2632 y(that)g(cannot)g(allo)q(cate)i(resources)f(without)f(starting)g(pro)q (cesses,)h(suc)o(h)g(as)g(LoadLev)o(eler)h(or)e(DQS.)g(In)75 2688 y(those)j(cases,)g(the)h(executables)g(ma)o(y)e(not)h(b)q(e)h(the)f (application)i(executables,)g(but)e(rather)g(in)o(terface)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 bop 75 -100 a Fs(14)928 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fs(pro)q(cesses)d(that)f(will)i(create)e(the)h (application)h(pro)q(cesses)f(in)g(resp)q(onse)g(to)f(one)h(of)f(the)g(pro)q (cess-creation)75 102 y(functions)16 b(describ)q(ed)h(here)f(\(lik)o(e)g Fj(MPI)p 757 102 14 2 v 16 w(SP)l(A)-5 b(WN)p Fs(\).)166 158 y(The)15 b(in)o(terface)g(to)f(an)h(external)g(resource)g(manager)f(could)i (b)q(e)f(arbitrarily)g(complex.)21 b(The)15 b(MPI)75 214 y(in)o(terface)20 b(sp)q(eci\014es)i(the)e(absolute)g(minim)o(um)i(amoun)o(t)d(of)g (information)h(that)g(m)o(ust)f(b)q(e)i(kno)o(wn)f(to)75 271 y(MPI)15 b(and)h(tries)f(to)f(mak)o(e)h(common)g(cases)g(v)o(ery)g(simple)i (to)d(sp)q(ecify)l(.)75 391 y Fj(Resource)i(Discovery)75 571 y(MPI)p 160 571 V 16 w(RESOURCE)p 406 571 V 18 w(DISCO)o(VER\(resource)p 815 571 V 17 w(t)o(yp)q(e,)g(nresources,)g(a)o(rra)o(y)p 1260 571 V 15 w(of)p 1312 571 V 16 w(resources\))117 648 y Ff(IN)155 b Fj(resource)p 476 648 V 17 w(t)o(yp)q(e)335 b Ff(explained)14 b(b)q(elo)o(w)117 724 y(INOUT)62 b Fj(nresources)392 b Ff(On)19 b(en)o(trance,)h(size)f(of)f(arra)o(y)m(.)31 b(On)18 b(exit,)h(n)o(um)o(b)q (er)f(of)g(ele-)905 780 y(men)o(ts)c(returned)h(in)f(arra)o(y)117 855 y(OUT)108 b Fj(a)o(rra)o(y)p 416 855 V 15 w(of)p 468 855 V 16 w(resources)252 b Ff(arra)o(y)14 b(of)f Fe(MPI)p 1138 855 13 2 v 14 w(Resource)j Ff(ob)r(jects)166 980 y Fj(resource)p 324 980 14 2 v 17 w(t)o(yp)q(e)d Fs(determines)h(ho)o(w)f(resources)g(are)f (split)i(up)f(in)o(to)g(one)g(or)f(more)h(MPI)p 1608 980 V 16 w(Resource)h(ob-)75 1036 y(jects.)19 b(It)14 b(is)g(a)f(suggestion,)h(not) f(a)g(requiremen)o(t)h(\(exept)g(for)f Fj(TYPE)p 1250 1036 V 16 w(DEF)l(A)o(UL)l(T)p Fs(\),)g(as)h(certain)g(concepts)75 1093 y(ma)o(y)e(b)q(e)h(foreign)g(to)f(certain)h(parallel)i(en)o(vironmen)o (ts.)k(It)13 b(can)g(tak)o(e)f(the)g(follo)o(wing)i(prede\014ned)g(v)m (alues.)143 1199 y Fn(\017)23 b Fj(MPI)p 274 1199 V 15 w(RESOURCE)p 519 1199 V 19 w(TYPE)p 655 1199 V 17 w(DEF)l(A)o(UL)l(T)p Fs(:)12 b(A)h(single)h(resource)f(ob)s(ject)f(is)h(returned,)g(with)g(all)h(a)o(v)m (ail-)189 1255 y(able)f(resources)g(in)h(that)f(ob)s(ject.)18 b(The)c Fj(nresources)g Fs(argumen)o(t)e(is)h(ignored,)h(and)f(ma)o(y)g(b)q (e)g(NULL.)143 1349 y Fn(\017)23 b Fj(MPI)p 274 1349 V 15 w(RESOURCE)p 519 1349 V 19 w(TYPE)p 655 1349 V 17 w(HOST)p Fs(:)d(Eac)o(h)f(returned)h (resource)f(corresp)q(onds)h(to)f(one)h(host.)32 b(A)189 1406 y(host)9 b(can)i(b)q(e)f(an)h(SMP)l(.)e(The)i(name)f(of)g(the)g(host)f(ma)o (y)h(b)q(e)h(disco)o(v)o(ered)f(using)h Fj(MPI)p 1573 1406 V 16 w(RESOURCE)p 1819 1406 V 19 w(NAME)p Fs(.)189 1462 y(If)21 b(the)g(pre-allo)q(cated)h(resources)f(cannot)f(b)q(e)i(describ)q(ed)h(as)d (a)h(host,)g(\(e.g.,)g(it)g(is)g(a)g(promise)189 1518 y(from)j(a)g(resource)h (manager,)h(not)e(an)h(explicit)h(ph)o(ysical)g(resource\))f(this)g(is)g(the) g(same)f(as)189 1575 y Fj(MPI)p 274 1575 V 15 w(RESOURCE)p 519 1575 V 19 w(TYPE)p 655 1575 V 17 w(DEF)l(A)o(UL)l(T)p Fs(.)143 1669 y Fn(\017)f Fj(MPI)p 274 1669 V 15 w(RESOURCE)p 519 1669 V 19 w(TYPE)p 655 1669 V 17 w(ARCH)p Fs(:)15 b(Eac)o(h)h(returned)f(resource) h(corresp)q(onds)f(to)g(a)g(single)i(arc)o(hi-)189 1725 y(tecture.)j(An)c (arc)o(hitecture)g(is)g(a)f(class)h(of)f(binary-compatible)i(computers.)k (The)15 b(name)h(of)f(the)189 1782 y(arc)o(hitecture)g(ma)o(y)g(b)q(e)h (disco)o(v)o(ered)f(using)h Fj(MPI)p 1023 1782 V 16 w(RESOURCE)p 1269 1782 V 18 w(NAME)p Fs(.)143 1875 y Fn(\017)23 b Fj(MPI)p 274 1875 V 15 w(RESOURCE)p 519 1875 V 19 w(TYPE)p 655 1875 V 17 w(PROCESSOR)p Fs(:)15 b(Eac)o(h)e(returned)h(resource)g(corresp)q(onds)g (to)e(a)i(single)189 1932 y(pro)q(cessor.)25 b(These)18 b(ma)o(y)e(b)q(e)i (virtual)g(pro)q(cessors.)26 b(F)l(or)16 b(instance)i(in)g(an)g(SMP)l(,)e (the)i(op)q(erating)189 1988 y(system)c(ma)o(y)h(not)g(ha)o(v)o(e)f(con)o (trol)h(o)o(v)o(er)g(whic)o(h)h(ph)o(ysical)g(pro)q(cessor)f(is)h(used)g(b)o (y)f(a)g(pro)q(cess.)143 2082 y Fn(\017)23 b Fj(MPI)p 274 2082 V 15 w(RESOURCE)p 519 2082 V 19 w(TYPE)p 655 2082 V 17 w(SPECIAL)p Fs(:)15 b(Resource)i(splitting)g(done)f(in)h(some)e(sp)q(ecial)j(w)o(a)o(y)l (.)j(Usu-)189 2139 y(ally)16 b(this)g(will)h(b)q(e)g(sp)q(eci\014ed)g (externally)l(,)g(b)o(y)e(the)h(user,)g(when)g(he/she)g(starts)e(the)i (application.)189 2195 y(The)h(order)f(of)g(resources)h(can)g(b)q(e)g(coun)o (ted)g(on)f(b)o(y)h(the)g(application.)26 b(F)l(or)16 b(instance,)h(the)g (user)189 2252 y(ma)o(y)d(ha)o(v)o(e)g(requested)i(an)e(HP)h(no)q(de)h(with)f (a)f(HiPPI)i(connection,)g(an)e(HP)h(no)q(de)h(with)f(a)f(FDDI)189 2308 y(connection,)i(8)e(pro)q(cessors)h(of)g(an)g(SP2,)g(etc.)166 2497 y Fg(Discussion:)36 b Ff(Suggestions)14 b(for)g(additional)f(allo)q (cation)f(strategies)k(are)f(encouraged.)20 b(Are)15 b(there)h Fb(c)n(om-)75 2553 y(mon)e Ff(things)g(y)o(ou)g(migh)o(t)d(w)o(an)o(t)j(to)f (do)h(that)g(y)o(ou)f(can't)h(do)g(with)f(the)i(ab)q(o)o(v)o(e?)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 bop 75 -100 a Fm(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(15)75 45 y Fj(Resource)16 b(Allo)q(cation)75 225 y(MPI)p 160 225 14 2 v 16 w(RESOURCE)p 406 225 V 18 w(IALLOCA)l(TE\(resource)p 836 225 V 17 w(t)o(yp)q(e,)22 b(np)o(ro)q(cs,)f(resource)p 1279 225 V 17 w(description,)h(resource,)g(request\))117 359 y Ff(IN)155 b Fj(resource)p 476 359 V 17 w(t)o(yp)q(e)335 b Ff(See)15 b(description)g(b)q(elo)o(w)117 431 y(IN)155 b Fj(np)o(ro)q(cs)465 b Ff(The)15 b(minim)n(um)9 b(n)o(um)o(b)q(er)k(of)h(pro)q(cess)h(slots)f (requested)117 504 y(IN)155 b Fj(resource)p 476 504 V 17 w(description)212 b Ff(String)14 b(con)o(taining)f(description)h(of)f(resources)117 576 y(OUT)108 b Fj(resource)432 b Fe(MPI)p 982 576 13 2 v 15 w(Resource)15 b Ff(allo)q(cated)117 649 y(OUT)108 b Fj(request)452 b Ff(an)14 b Fe(MPI)p 1040 649 V 14 w(Request)166 773 y Fj(resource)h Fs(is)h(set)f(async)o(hronously)h(b)o(y)f(MPI)g(\(as)g(with)g(other)g(MPI)p 1326 773 14 2 v 17 w(I*)g(routines\).)20 b(It)c(is)f(not)g(v)m(alid)75 829 y(un)o(til)23 b(the)g(request)f(has)g(b)q(een)h(completed.)42 b(One)23 b(can)f(w)o(ait)g(for)g(the)g(request)g(to)g(complete)h(with)75 886 y Fj(MPI)p 160 886 V 16 w(W)l(AIT)p Fs(,)15 b(test)f(with)i Fj(MPI)p 591 886 V 16 w(TEST)p Fs(,)f(etc.)166 942 y Fj(resource)p 324 942 V 17 w(t)o(yp)q(e)k Fs(is)f(a)g(general)g(category)f(of)g(resource)h (request,)h(used)f(to)f(mak)o(e)h(it)g(easy)g(to)f(sp)q(ec-)75 999 y(ify)22 b(simple)h(requests.)40 b(The)22 b(categories)f(are)h(the)g (same)f(as)g(for)g(the)h Fj(resource)p 1494 999 V 17 w(t)o(yp)q(e)h Fs(argumen)o(t)e(for)75 1055 y Fj(MPI)p 160 1055 V 16 w(RESOURCE)p 406 1055 V 18 w(DISCO)o(VER)p Fs(.)16 b(In)g(this)f(con)o(text,)f(they)i (mean:)143 1142 y Fn(\017)23 b Fj(MPI)p 274 1142 V 15 w(RESOURCE)p 519 1142 V 19 w(TYPE)p 655 1142 V 17 w(DEF)l(A)o(UL)l(T)p Fs(:)15 b(The)h(string)g(in)h Fj(resource)p 1327 1142 V 17 w(description)g Fs(describ)q(es)h(an)d(ar-)189 1199 y(bitrary)g(set)g(of)f(resources.)20 b(String)c(ma)o(y)e(b)q(e)i(empt)o(y)l(.)143 1287 y Fn(\017)23 b Fj(MPI)p 274 1287 V 15 w(RESOURCE)p 519 1287 V 19 w(TYPE)p 655 1287 V 17 w(HOST)p Fs(:)15 b(The)h(string)f(in)h Fj(resource)p 1252 1287 V 17 w(description)g Fs(is)g(a)f(hostname.)143 1375 y Fn(\017)23 b Fj(MPI)p 274 1375 V 15 w(RESOURCE)p 519 1375 V 19 w(TYPE)p 655 1375 V 17 w(ARCH)p Fs(:)d(The)g(string)g(in)h Fj(resource)p 1270 1375 V 16 w(description)h Fs(is)e(an)g(arc)o(hitecture.) 189 1432 y(Note)14 b(that)h(arc)o(hitecture)g(names)h(are)f(not)f(sp)q (eci\014ed)k(b)o(y)d(MPI)g(itself.)143 1520 y Fn(\017)23 b Fj(MPI)p 274 1520 V 15 w(RESOURCE)p 519 1520 V 19 w(TYPE)p 655 1520 V 17 w(PROCESSOR)p Fs(:)f(The)f(string)g(in)h Fj(resource)p 1414 1520 V 16 w(description)h Fs(describ)q(es)f(a)189 1577 y(single)15 b(pro)q(cessor.)k(The)13 b(recommended)i(approac)o(h)e(is)h(that) f(the)h(name)f(should)i(b)q(e)f(the)g(name)f(of)189 1633 y(the)i(host)g(in)h (whic)o(h)g(the)f(pro)q(cessor)g(resides.)143 1721 y Fn(\017)23 b Fj(MPI)p 274 1721 V 15 w(RESOURCE)p 519 1721 V 19 w(TYPE)p 655 1721 V 17 w(SPECIAL)p Fs(:)17 b(The)i(string)f(in)h Fj(resource)p 1320 1721 V 17 w(description)g Fs(describ)q(es)h(an)e(ar-)189 1778 y(bitrary)d(set)g(of)f(resources.)166 1947 y Fg(Discussion:)30 b Ff(Note)11 b(that)f Fe(MPI)p 690 1947 13 2 v 15 w(RESOURCE)p 919 1947 V 12 w(IALLOCA)m(TE)f Ff(do)q(es)i(not)f(tak)o(e)h(a)f(comm)o(uni)o (cator)e(argumen)o(t)75 2004 y(and)k(is)f(called)h(in)f(a)g(single)h(pro)q (cess.)19 b(This)11 b(means)g(that)h(an)g Fe(MPI)p 1099 2004 V 14 w(Resource)h Ff(ob)r(ject)g(has)f(a)f(single)g(o)o(wner.)18 b(T)m(rying)75 2060 y(to)d(mak)o(e)f(this)h(op)q(eration)h(collectiv)o(e)f(w) o(ould)g(force)h(us)f(to)h(in)o(tro)q(duce)g(collectiv)o(e)f(ob)r(jects)h (\(collectiv)o(e)g(requests)75 2117 y(and)e(collectiv)o(e)g Fe(MPI)p 413 2117 V 14 w(Resource)h Ff(ob)r(jects\))g(and)f(pro)o(vide)g(no)f (b)q(ene\014t.)166 2256 y Fj(MPI)p 251 2256 14 2 v 16 w(W)l(AIT)i Fs(or)g Fj(MPI)p 533 2256 V 15 w(TEST)h Fs(can)f(b)q(e)h(used)g(to)e(c)o(hec) o(k)i(for)e(completion.)166 2395 y Fg(Discussion:)45 b Ff(This)16 b(is)h(the)g(\014rst)g(case)h(of)e(man)o(y)f(where)i(w)o(e)g(m)o(ust)f (address)i(the)f(question)g(of)f(what)g(is)75 2452 y(in)e(the)i(status)f(ob)r (ject.)22 b(Do)14 b(w)o(e)h(assine)g(meaning)e(to)i Fd(status.tag)e Ff(and)h Fd(status.source)e Ff(Do)j(w)o(e)g(replace)g(the)75 2508 y(status)g(argumen)o(t)d(with)i(a)g(\(v)o(oid)f(*\))g(and)h(do)g (di\013eren)o(t)h(things)e(dep)q(ending)i(on)f(the)g(t)o(yp)q(e)g(of)g (request?)166 2647 y Fs(Authen)o(tication)19 b(information,)h(if)f(required)g (b)o(y)g(the)f(resource)h(manager,)f(can)h(b)q(e)g(supplied)i(in)75 2704 y(the)15 b Fj(resource)p 311 2704 V 17 w(description)i Fs(string.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 bop 75 -100 a Fs(16)928 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 45 y Fs(A)h(no)q(de)h(ma)o(y)e(ha)o(v)o(e)h(m)o(ultiple)i (comm)o(unication)f(in)o(terfaces)f(\(Ethernet,)g(HiPPI,)g(A)l(TM\).)g(These) 75 102 y(ha)o(v)o(e)e(to)f(sp)q(eci\014ed)j(so)e(that)g(the)g(desired)h(comm) o(unication)g(infrastructure)f(can)g(b)q(e)h(used,)g(and)f(can)g(b)q(e)75 158 y(included)k(in)e(the)f Fj(resource)p 546 158 14 2 v 17 w(description)i Fs(string.)166 214 y(When)e(m)o(ultiple)h(resources)e(are)g (required)i(at)e(the)g(same)g(time,)h(it)f(is)h(b)q(est)g(to)f(ask)g(for)f (them)i(with)75 271 y(a)g(single)i(request)f(-)f(for)g(e\016ciency)l(,)i(and) f(b)q(ecause)g(the)g(user)f(ma)o(y)g(need)i(to)d(sp)q(ecify)j(global)f (resources,)75 327 y(suc)o(h)g(as)e(gang)h(sc)o(heduling,)i(whic)o(h)f (extend)g(b)q(ey)o(ond)f(a)g(single)i(resource)e(ob)s(ject.)166 384 y(The)g(follo)o(wing)h(routine)g(pro)o(vides)g(this)f(functionalit)o(y)l (.)75 535 y Fj(MPI)p 160 535 V 16 w(RESOURCES)p 431 535 V 18 w(IALLOCA)l(TE\(global)p 819 535 V 16 w(resource)p 990 535 V 17 w(description,)d(a)o(rra)o(y)p 1330 535 V 15 w(of)p 1382 535 V 16 w(resource)p 1553 535 V 17 w(t)o(yp)q(es,)g(a)o(rra)o(y)p 1787 535 V 14 w(of)p 1838 535 V 16 w(np)o(ro)q(cs,)75 591 y(a)o(rra)o(y)p 173 591 V 15 w(of)p 225 591 V 16 w(resource)p 396 591 V 16 w(descriptions,)17 b(a)o(rra)o(y)p 757 591 V 15 w(of)p 809 591 V 16 w(resources,)e(request\))117 668 y Ff(IN)155 b Fj(global)p 434 668 V 16 w(resource)p 605 668 V 17 w(description)83 b Ff(string)14 b(describing)h(resources)h(common)11 b(to)j(all)e(requests)117 744 y(IN)155 b Fj(a)o(rra)o(y)p 416 744 V 15 w(of)p 468 744 V 16 w(resource)p 639 744 V 17 w(t)o(yp)q(e)172 b Ff(Arra)o(y)15 b(of)f(in)o(tegers)h(eac)o(h)g(of)f(whic)o(h)g(categorizes)i(the)f(corre-)905 800 y(sp)q(onding)i(resource)h(description)g(as)e(describ)q(ed)j(ab)q(o)o(v)o (e)d(for)905 856 y Fe(MPI)p 982 856 13 2 v 15 w(RESOURCE)p 1211 856 V 12 w(IALLOCA)m(TE)117 932 y Ff(IN)155 b Fj(a)o(rra)o(y)p 416 932 14 2 v 15 w(of)p 468 932 V 16 w(np)o(ro)q(cs)302 b Ff(The)18 b(minim)n(um)13 b(n)o(um)o(b)q(er)k(of)g(pro)q(cess)i(slots)f (requested)h(for)905 988 y(eac)o(h)c(resource)h(ob)r(ject.)117 1063 y(IN)155 b Fj(a)o(rra)o(y)p 416 1063 V 15 w(of)p 468 1063 V 16 w(resource)p 639 1063 V 17 w(descriptions)43 b Ff(Strings)14 b(con)o(taining)f(description)i(of)e(resources)117 1138 y(OUT)108 b Fj(a)o(rra)o(y)p 416 1138 V 15 w(of)p 468 1138 V 16 w(resources)252 b Fe(MPI)p 982 1138 13 2 v 15 w(Resource)15 b Ff(ob)r(jects)g(allo)q(cated) 117 1213 y(OUT)108 b Fj(request)452 b Ff(an)14 b Fe(MPI)p 1040 1213 V 14 w(Request)166 1421 y Fg(Discussion:)34 b Ff(Suggestions)14 b(for)g(a)f(b)q(etter)j(name)c(are)j(encouraged.)75 1623 y Fj(Interfacing)g(to)h(the)g(Pro)q(cess)g(Manager)75 1709 y Fs(Once)k(an)e(application)j(has)d(obtained)i(resources)e(through)h(allo)q (cation)g(or)g(disco)o(v)o(ery)l(,)g(it)g(ma)o(y)f(start)75 1766 y(pro)q(cesses)f(on)f(those)g(resources)g(using)h(the)f(routines)h (presen)o(ted)g(in)g(this)f(section.)24 b(More)16 b(abstractly)l(,)75 1822 y(it)f(can)h(request)f(that)f(the)i(pro)q(cess)f(manager)g(start)f(pro)q (cesses)h(on)g(those)g(resources.)75 1973 y Fj(MPI)p 160 1973 14 2 v 16 w(PROCESS)p 373 1973 V 18 w(CREA)l(TE\(executable,)i(a)o(rguments,) d(n,)h(p)o(ro)q(cess)p 1214 1973 V 18 w(t)o(yp)q(e,)g(resource,)g(a)o(rra)o (y)p 1618 1973 V 15 w(of)p 1670 1973 V 16 w(p)o(ro)q(cesses\))117 2107 y Ff(IN)155 b Fj(executable)391 b Ff(executable)15 b(\014le)f(con)o (taining)f(program)f(to)i(b)q(e)g(run)117 2182 y(IN)155 b Fj(a)o(rguments)391 b Ff(argumen)o(ts)13 b(for)h(the)g(program)117 2257 y(IN)155 b Fj(n)564 b Ff(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(to)e(start)117 2332 y(IN)155 b Fj(p)o(ro)q(cess)p 457 2332 V 18 w(t)o(yp)q(e)353 b Ff(see)15 b(b)q(elo)o(w)117 2407 y(IN)155 b Fj(resource)432 b Fe(MPI)p 982 2407 13 2 v 15 w(Resource)15 b Ff(to)f(run)g(pro)q(cesses)i (on)117 2482 y(OUT)108 b Fj(a)o(rra)o(y)p 416 2482 14 2 v 15 w(of)p 468 2482 V 16 w(p)o(ro)q(cesses)251 b Fe(MPI)p 982 2482 13 2 v 15 w(Pro)q(cess)14 b Ff(ob)r(jects)h(returned)166 2607 y Fs(This)g(is)g(a)g(blo)q(c)o(king)h(call;)g(when)f(it)g(returns)f(the)h (pro)q(cesses)g(ha)o(v)o(e)g(b)q(een)h(started.)i(Note)d(that)f(the)75 2663 y(resources)k(ha)o(v)o(e)g(already)g(b)q(een)i(allo)q(cated.)29 b(Unless)19 b(there)g(is)f(an)g(error)g(in)h(argumen)o(ts)e(\(executable)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 bop 75 -100 a Fm(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(17)75 45 y(do)q(es)17 b(not)g(exist,)h(for)e(example\))h Fj(MPI)p 738 45 14 2 v 16 w(PROCESS)p 951 45 V 18 w(CREA)l(TE)h Fs(normally)g(should)g(w)o(ork.)24 b(\(The)17 b(system)75 102 y(ma)o(y)11 b(ha)o(v)o(e)g(to)f(con)o(tact)h(the)g(resource)h(manager,)f(but) g(the)h(resources)f(ha)o(v)o(e)g(b)q(een)h(promised)g(and)g(should)75 158 y(b)q(e)h(a)o(v)m(ailable)g(immediately\).)20 b(There)12 b(is)h(one)f(executable)h(name)f(and)g(one)g(set)f(of)h(argumen)o(ts.)18 b(T)l(o)11 b(start)75 214 y(pro)q(cesses)16 b(with)f(m)o(ultiple)i (executables)g(or)d(di\013eren)o(t)i(argumen)o(ts,)e(call)i(this)g(t)o(wice.) 166 271 y Fj(p)o(ro)q(cess)p 305 271 V 17 w(t)o(yp)q(e)g Fs(can)g(tak)o(e)e (one)i(of)e(the)i(follo)o(wing)g(v)m(alues:)143 365 y Fn(\017)23 b Fj(MPI)p 274 365 V 15 w(PROCESS)p 486 365 V 18 w(TYPE)p 621 365 V 17 w(MPI)p 720 365 V 16 w(CHILD)p Fs(:)17 b(The)h(new)g(pro)q(cesses)h (are)e(MPI)h(pro)q(cesses)g(whic)o(h)h(will)189 421 y(call)d Fj(MPI)p 357 421 V 16 w(Comm)p 499 421 V 14 w(pa)o(rent)p Fs(.)143 515 y Fn(\017)23 b Fj(MPI)p 274 515 V 15 w(PROCESS)p 486 515 V 18 w(TYPE)p 621 515 V 17 w(MPI)p 720 515 V 16 w(ORPHAN)p Fs(:)12 b(The)f(new)h(pro)q(cesses)f(are)g(MPI)g(pro)q(cesses)g(whic)o(h)h (will)189 571 y(not)i(call)j Fj(MPI)p 438 571 V 16 w(Comm)p 580 571 V 14 w(pa)o(rent)p Fs(.)143 665 y Fn(\017)23 b Fj(MPI)p 274 665 V 15 w(PROCESS)p 486 665 V 18 w(TYPE)p 621 665 V 17 w(NONMPI)p Fs(:)15 b(The)h(new)f(pro)q(cesses)h(are)f(not)f(MPI)i(pro)q (cesses.)166 835 y Fg(Discussion:)53 b Ff(It)19 b(is)g(not)f(completely)g (clear)h(whether)h(w)o(e)f(need)h(the)f Fe(p)o(ro)q(cess)p 1480 835 13 2 v 16 w(t)o(yp)q(e)h Ff(argumen)o(t.)32 b(The)75 885 y(argumen)o(ts)13 b(in)h(fa)o(v)o(or)e(are:)145 960 y Fc(\017)23 b Ff(W)m(e)11 b(need)j(to)e(distinguish)f(b)q(et)o(w)o(een)j(orphans)f(and)f (c)o(hildren)g(to)g(allo)o(w)f Fe(MPI)p 1393 960 V 14 w(COMM)p 1537 960 V 15 w(P)m(ARENT)g Ff(to)h(either)189 1009 y(blo)q(c)o(k)h(or)h (return)h Fe(MPI)p 553 1009 V 14 w(COMM)p 697 1009 V 15 w(NULL)p Ff(.)145 1084 y Fc(\017)23 b Ff(W)m(e)9 b(need)h(to)f(distinguish)g(b)q(et)o (w)o(een)i(MPI)e(and)g(non-MPI)h(b)q(ecause)h(MPI)e(pro)q(cesses)j(ma)o(y)c (need)i(command-)189 1134 y(line)j(argumen)o(ts.)75 1209 y(The)h(argumen)o (ts)f(against)h(are:)145 1283 y Fc(\017)23 b Ff(Children)9 b(should)g(only)f(call)h Fe(MPI)p 717 1283 V 14 w(COMM)p 861 1283 V 15 w(P)m(ARENT)f Ff(if)g(they)i(kno)o(w)f(the)h(paren)o(t)g(will)e (call)g Fe(MPI)p 1684 1283 V 14 w(PROCESS)p 1881 1283 V 14 w(A)m(TT)m(A)o(CH)p Ff(.)145 1358 y Fc(\017)23 b Ff(There)17 b(are)g(other)g(w)o(a)o(ys)f(to)g(pass)h(information)c(to)j(MPI)h(pro)q (cesses,)i(suc)o(h)e(as)f(en)o(vironmen)o(t)f(v)n(ariables,)189 1408 y(whic)o(h)e(can)i(b)q(e)f(ignored)g(if)f(a)g(pro)q(cess)j(do)q(esn't)e (need)h(them.)166 1572 y Fs(All)i(tasks)e(initiated)j(with)e(one)g(call)h(to) f Fj(MPI)p 954 1572 14 2 v 15 w(PROCESS)p 1166 1572 V 18 w(CREA)l(TE)h Fs(ha)o(v)o(e)f(the)g(same)g(executable)75 1628 y(and)10 b(same)g(argumen)o (ts,)g(and)g(execute)h(on)f(the)h(same)e(resource.)19 b(One)11 b(can)f(use)g(m)o(ultiple)i Fj(MPI)p 1672 1628 V 16 w(PROCESS)p 1885 1628 V 18 w(CREA)l(TE)75 1685 y Fs(calls)18 b(to)f(spa)o(wn)f(m)o (ultiple)j(executables,)g(to)d(pass)h(di\013eren)o(t)g(argumen)o(t)f(lists)i (to)f(eac)o(h)g(executatable,)75 1741 y(or)c(to)h(run)g(di\013eren)o(t)g(pro) q(cesses)h(on)f(di\013eren)o(t)g(resources.)19 b(It)14 b(will)i(generally)f (b)q(e)g(more)f(e\016cien)o(t)g(to)g(use)75 1798 y(the)h(non-blo)q(c)o(king)i (v)o(ersion)f(\(b)q(elo)o(w\))f(in)h(this)f(case.)166 1930 y Fg(Discussion:)50 b Ff(Note)19 b(that)f(a)g(single)f(resource)j(migh)o(t)c (b)q(e)j(\\a)f(P)o(aragon",)f(so)h(it)g(mak)o(es)f(sense)j(to)e(run)75 1980 y(m)o(ultiple)12 b(pro)q(cesses)18 b(on)c(a)h(single)f(resource.)23 b(If)14 b(w)o(e)h(allo)o(w)f(construction)h(of)g(a)f(\\virtual)g(mac)o(hine") f(as)i(a)f(single)75 2030 y(resource,)f(then)d(it)g(again)g(mak)o(es)f(sense) j(and)e(is)g(con)o(v)o(enien)o(t)h(to)f(use)h(a)f(single)g(call)f(to)h(start) h(m)o(ultiple)d(pro)q(cesses)13 b(on)75 2080 y(a)g(single)h(resource.)20 b(On)14 b(the)g(other)h(hand,)e(one)h(migh)o(t)d(conceiv)o(e)k(of)e (situation)g(where)i(it)e(w)o(ould)g(b)q(e)i(con)o(v)o(enien)o(t)75 2130 y(to)f(replace)h(the)f Fe(resource)i Ff(argumen)o(t)c(ab)q(o)o(v)o(e)i (with)g(an)f(arra)o(y)h(of)f(resources.)166 2186 y(Is)18 b(it)f(su\016cien)o (t)h(to)f(call)f(the)i(non-blo)q(c)o(king)f(v)o(ersion)g(m)o(ultiple)e(times) i(or)g(do)g(w)o(e)h(need)g(a)f(single)g(call)g(to)75 2242 y(spa)o(wn)d(man)o (y)e(things)i(at)f(once?)166 2382 y Fs(The)i(non)o(blo)q(c)o(king)i(v)o (ersion)e(returns)g(a)g(request)g(that)g(can)g(b)q(e)h(tested)f(and)h(w)o (aited)f(on.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 bop 75 -100 a Fs(18)928 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(PROCESS)p 373 45 V 18 w(ICREA)l(TE\(executable,)e(a)o(rguments,)d(n,)i(p)o(ro)q(cess)p 1216 45 V 17 w(t)o(yp)q(e,)g(resource,)f(a)o(rra)o(y)p 1612 45 V 14 w(of)p 1663 45 V 16 w(p)o(ro)q(cesses,request\))117 179 y Ff(IN)155 b Fj(executable)391 b Ff(executable)15 b(\014le)f(con)o (taining)f(program)f(to)i(b)q(e)g(run)117 254 y(IN)155 b Fj(a)o(rguments)391 b Ff(argumen)o(ts)13 b(for)h(the)g(program)117 329 y(IN)155 b Fj(n)564 b Ff(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)117 404 y(IN)155 b Fj(p)o(ro)q(cess)p 457 404 V 18 w(t)o(yp)q(e)353 b Ff(as)14 b(describ)q(ed)i(ab)q(o)o(v)o(e)117 479 y(IN)155 b Fj(resource)432 b Fe(MPI)p 982 479 13 2 v 15 w(Resource)p Ff(s)15 b(to)f(run)g(pro)q(cesses)j(on)117 554 y(OUT)108 b Fj(a)o(rra)o(y)p 416 554 14 2 v 15 w(of)p 468 554 V 16 w(p)o(ro)q(cesses)251 b Fe(MPI)p 982 554 13 2 v 15 w(Pro)q(cess)14 b Ff(ob)r(jects)h(returned)117 630 y(OUT)108 b Fj(request)452 b Fe(MPI)p 982 630 V 15 w(Request)15 b Ff(ob)r(ject)g(returned,)g(to)e(w)o(ait)h(on)166 837 y Fg(Discussion:)32 b Ff(A)12 b(more)g(detailed)g(v)o(ersion)g(w)o(ould)g(return)h(an)f(arra)o(y) g(of)g(requests,)i(so)e(that)g(startup)h(could)75 893 y(b)q(e)j(w)o(aited)f (on)g(one)g(pro)q(cess)i(at)e(a)g(time.)20 b(This)15 b(functionalit)o(y)f (could)h(b)q(e)h(pro)o(vided)f(with)g(the)g(ab)q(o)o(v)o(e)g(prop)q(osal)75 950 y(b)o(y)f(using)f Fe(MPI)p 318 950 V 15 w(PROCESS)p 516 950 V 13 w(ICREA)m(TE)f Ff(on)h(one)h(pro)q(cess)i(at)e(a)f(time.)166 1171 y Fg(Discussion:)22 b Ff(W)m(e)16 b(ma)o(y)g(w)o(an)o(t)g(to)h(consider) h(routines)g(that)f(build)f(a)h(resource,)i(corresp)q(onding)f(to)f(the)75 1228 y(w)o(a)o(y)d(PVM)h(users)h(build)d(a)i(virtual)e(mac)o(hine.)19 b(In)c(this)f(w)o(a)o(y)g(m)o(ultiple)e(resources)17 b(can)e(b)q(e)g(com)o (bined)e(in)o(to)h(one,)75 1284 y(handy)i(for)f(constructing)i(a)f(sp)q (eci\014c)h(set)g(of)e(w)o(orkstations,)h(for)g(example.)22 b(This)16 b(w)o(ould)f(mak)o(e)g(it)g(ev)o(en)i(more)75 1341 y(unnecessary)f(to)e(mak)o(e)e(m)o(ultiple)g Fe(MPI)p 704 1341 V 14 w(PROCESS)p 901 1341 V 13 w(CREA)m(TE)h Ff(calls.)75 1544 y Fj(Communicating)i(with)g(Dynamically)f(Created)i(MPI)f(Pro)q(cesses)75 1629 y Fs(Once)g(the)f(pro)q(cesses)g(ha)o(v)o(e)f(b)q(een)i(started,)e(w)o (e)h(ma)o(y)f(or)g(ma)o(y)g(not)h(wish)g(to)f(establish)i(comm)o(unication)75 1686 y(with)k(them.)31 b(Note)19 b(that)f(the)h Fj(MPI)p 710 1686 14 2 v 16 w(RESOURCE)p 956 1686 V 18 w(IALLOCA)l(TE)p Fs(/)p Fj(MPI)p 1318 1686 V 16 w(PROCESS)p 1531 1686 V 18 w(CREA)l(TE)h Fs(mec)o(ha-)75 1742 y(nism)e(can)f(b)q(e)h(used)g(b)o(y)f(a)g(pro)q(cess)g (manager)g(written)g(in)h(MPI)f(to)g(manage)f(non-MPI)i(pro)q(cesses)f(as)75 1799 y(w)o(ell)g(as)e(MPI)g(pro)q(cesses.)22 b(If)16 b(the)g(application)h (program)d(is)j(creating)e(new)h(pro)q(cesses,)g(ho)o(w)o(ev)o(er,)f(it)g(is) 75 1855 y(lik)o(ely)g(that)d(it)i(will)h(w)o(an)o(t)d(to)g(comm)o(unicate)i (with)f(them,)g(via)h(an)f(in)o(ter-comm)o(unicator.)19 b(The)14 b(compli-)75 1912 y(cation)f(is)h(that)f(creation)g(of)g(this)h(in)o (ter-comm)o(unicator)f(is)h(a)f(collectiv)o(e)i(op)q(eration)e(o)o(v)o(er)f (all)j(pro)q(cesses)75 1968 y(in)o(v)o(olv)o(ed,)g(y)o(et)f(w)o(e)h(don't)f (w)o(an)o(t)f(the)i(spa)o(wning)g(pro)q(cess\(es\))f(to)g(blo)q(c)o(k)i(if)f (the)f(spa)o(wned)h(pro)q(cesses)g(are)75 2025 y(not)k(ev)o(en)h(going)g(to)e (call)j Fj(MPI)p 626 2025 V 16 w(INIT)p Fs(.)d(Our)i(solution)h(is)f(to)f (create)g(this)h(comm)o(unicator)f(through)g(a)75 2081 y(collectiv)o(e)e (function)f(separate)f(from)f Fj(MPI)p 822 2081 V 16 w(PROCESS)p 1035 2081 V 18 w(CREA)l(TE)p Fs(.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 bop 75 -100 a Fm(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(19)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(PROCESS)p 373 45 V 18 w(A)l(TT)l(A)o(CH\(comm,)13 b(ro)q(ot,)i(n,)g(a)o(rra)o(y)p 979 45 V 14 w(of)p 1030 45 V 16 w(p)o(ro)q(cesses,)i(intercomm\))117 122 y Ff(IN)155 b Fj(comm)470 b Ff(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o (h)g(this)g(op)q(eration)g(is)f(collectiv)o(e)117 197 y(IN)155 b Fj(ro)q(ot)508 b Ff(pro)q(cess)16 b(where)f Fe(n)f Ff(and)g Fe(a)o(rra)o(y)p 1373 197 13 2 v 15 w(of)p 1422 197 V 14 w(p)o(ro)q(cesses)i Ff(are)e(v)n(alid)117 273 y(IN)155 b Fj(n)564 b Ff(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(in)e(follo)o(wing)d(arra)o(y)117 348 y(IN)155 b Fj(a)o(rra)o(y)p 416 348 14 2 v 15 w(of)p 468 348 V 16 w(p)o(ro)q(cesses)251 b Ff(the)15 b(spa)o(wned)f(pro)q(cesses)j(to)c(b)q (e)i(connected)h(to)117 423 y(OUT)108 b Fj(intercomm)384 b Ff(in)o(tercomm)o(unicator)7 b(in)i(whic)o(h)g(spa)o(wned)h(pro)q(cesses)i (are)e(the)905 479 y(remote)k(group)166 604 y Fj(MPI)p 251 604 V 16 w(PROCESS)p 464 604 V 18 w(A)l(TT)l(A)o(CH)j Fs(is)g(collectiv)o(e)h (among)d(the)i(pro)q(cesses)f(in)h Fj(comm)e Fs(and)h(the)h(pro)q(cesses)75 660 y(b)q(eing)11 b(\\attac)o(hed".)17 b(In)11 b(the)f(case)h(of)e(a)h(paren) o(t)g(comm)o(unicating)h(with)f(its)h(c)o(hildren,)h Fj(MPI)p 1608 660 V 16 w(PROCESS)p 1821 660 V 18 w(A)l(TT)l(A)o(CH)75 717 y Fs(is)20 b(collectiv)o(e)h(with)e(with)g(the)h(routine)f Fj(MPI)p 868 717 V 16 w(COMM)p 1026 717 V 16 w(P)l(ARENT)i Fs(in)f(the)f(c)o(hildren.)33 b(More)19 b(generally)75 773 y(\(see)g(the)g(clien)o(t/serv)o(er)h(section\))f Fj(MPI)p 776 773 V 16 w(PROCESS)p 989 773 V 18 w(A)l(TT)l(A)o(CH)h Fs(is)g(collectiv)o (e)h(with)e(a)g(corresp)q(onding)75 830 y Fj(MPI)p 160 830 V 16 w(PROCESS)p 373 830 V 18 w(A)l(TT)l(A)o(CH)d Fs(in)g(the)f(pro)q(cesses) h(b)q(eing)g(attac)o(hed.)166 886 y Fj(MPI)p 251 886 V 16 w(INIT)10 b Fs(creates)i(an)f Fj(MPI)p 656 886 V 16 w(COMM)p 814 886 V 16 w(W)o(ORLD)g Fs(con)o(taining)i(all)f(one's)f(sibling)i(pro)q(cesses,)g (de\014ned)75 942 y(as)i(all)h(the)f(other)g(pro)q(cesses)h(spa)o(wned)f(in)h (the)f(same)g Fj(MPI)p 1101 942 V 16 w(PROCESS)p 1314 942 V 18 w(CREA)l(TE)h Fs(op)q(eration.)166 1082 y Fg(Discussion:)45 b Ff(Note)17 b(that)g Fe(MPI)p 718 1082 13 2 v 14 w(COMM)p 862 1082 V 15 w(W)o(ORLD)f Ff(ma)o(y)f(not)h(b)q(e)i(as)e(big)g(as)h(w)o(e)g (w)o(an)o(t)f(it)g(to)h(b)q(e)g(if,)f(for)75 1138 y(some)d(reason,)h(w)o(e)g (ha)o(v)o(e)g(to)g(use)g(m)o(ultiple)e Fe(MPI)p 839 1138 V 14 w(PROCESS)p 1036 1138 V 13 w(CREA)m(TE)h Ff(calls)g(for)h(related)g(pro)q (cesses.)166 1277 y Fs(The)k(\\other)f(side")h(of)g Fj(intercomm)e Fs(is)i Fj(MPI)p 929 1277 14 2 v 16 w(COMM)p 1087 1277 V 16 w(P)l(ARENT)h Fs(in)g(the)f(c)o(hildren.)29 b(In)18 b(MPI)g(pro-)75 1334 y(cesses)i(whic)o(h)g(w)o(ere)f(not)g(created)g(b)o(y)h Fj(MPI)p 853 1334 V 15 w(PROCESS)p 1065 1334 V 18 w(CREA)l(TE)p Fs(,)g Fj(MPI)p 1367 1334 V 16 w(COMM)p 1525 1334 V 17 w(P)l(ARENT)g Fs(returns)75 1390 y Fj(MPI)p 160 1390 V 16 w(COMM)p 318 1390 V 16 w(NULL)p Fs(.)75 1541 y Fj(MPI)p 160 1541 V 16 w(COMM)p 318 1541 V 16 w(P)l(ARENT\(intercomm\))117 1618 y Ff(OUT)108 b Fj(intercomm)384 b Ff(in)o(tercomm)o(unicator)8 b(in)h(whic)o(h)h(the)h(lo) q(cal)e(group)h(consists)i(of)905 1675 y(one's)f(siblings)f(\(MPI)p 1250 1675 13 2 v 15 w(COMM)p 1403 1675 V 15 w(W)o(ORLD\))g(and)g(the)h (remote)905 1731 y(group)e(consists)i(of)d(the)i(pro)q(cess)h(that)f(did)e (the)i Fe(MPI)p 1712 1731 V 15 w(PROCESS)p 1910 1731 V 13 w(A)m(TT)m(A)o(CH) 166 1938 y Fg(Discussion:)33 b Ff(The)13 b(goal)e(is)i(to)f(establish)h(some) f(metho)q(d)g(of)g(comm)o(unicating)d(with)j(the)i(new)f(pro)q(cesses.)75 1995 y(The)g(alternativ)o(es)g(are)g(to)f(create)i(an)e(in)o(tercomm)o (unicator,)e(to)i(create)i(a)f(con)o(taining)e(comm)o(unicator,)f(or)i(to)g (re-)75 2051 y(size)g(an)e(existing)h(comm)o(unicator.)j(Resizing)d(an)g (existing)f(comm)o(unicator)e(seems)k(a)e(bad)h(idea.)17 b(Giv)o(en)10 b(an)h(in)o(ter-)75 2108 y(comm)o(unicator,)f(it)i(is)h(easy)g(to)g(create)i (a)d(con)o(taining)g(comm)o(unicator)e(b)o(y)j(using)f Fe(MPI)p 1445 2108 V 15 w(INTERCOMM)p 1711 2108 V 14 w(MERGE)p Ff(.)75 2164 y(Therefore)j(w)o(e)f(c)o(ho)q(ose)h(to)f(create)h(an)f(in)o(tercomm)o (unicator.)166 2303 y Fs(The)c Fj(MPI)p 339 2303 14 2 v 16 w(COMM)p 497 2303 V 17 w(P)l(ARENT)p Fs(/)p Fj(MPI)p 793 2303 V 16 w(PROCESS)p 1006 2303 V 18 w(A)l(TT)l(A)o(CH)h Fs(scenario)g(allo)o(ws)f (one)g(to)g(manage)f(spa)o(wned)75 2360 y(pro)q(cesses)18 b(that)e(comm)o (unicate)i(with)g(their)f(paren)o(ts)g(\(b)o(y)g(calling)i Fj(MPI)p 1346 2360 V 16 w(COMM)p 1504 2360 V 16 w(P)l(ARENT)p Fs(\),)f(as)f(w)o(ell)75 2416 y(as)22 b(spa)o(wned)g(MPI)g(pro)q(cesses)g (that)f(nev)o(er)i(call)g Fj(MPI)p 1051 2416 V 16 w(COMM)p 1209 2416 V 16 w(P)l(ARENT)g Fs(or)f(non-MPI)g(pro)q(cesses.)75 2473 y(The)e(last)f(t)o(w)o(o)f(t)o(yp)q(es)i(cannot)f(b)q(e)i(comm)o (unicated)f(with,)g(although)g(they)g(can)f(b)q(e)i(signalled)g(using)75 2529 y Fj(MPI)p 160 2529 V 16 w(PROCESS)p 373 2529 V 18 w(SIGNAL)p Fs(.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 bop 75 -100 a Fs(20)928 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 45 y Fg(Discussion:)44 b Ff(A)o(t)16 b(this)h(p)q(oin)o (t,)f Fe(MPI)p 790 45 13 2 v 14 w(PROCESS)p 987 45 V 13 w(A)m(TT)m(A)o(CH)f Ff(do)q(es)i(not)g(lo)q(ok)e(to)q(o)h(useful.)26 b(The)16 b(curren)o(t)75 102 y(v)o(ersion)11 b(of)f Fe(MPI)p 334 102 V 14 w(PROCESS)p 531 102 V 14 w(CREA)m(TE)f Ff(already)h(sp)q(eci\014es)j(whether)f(w)o(e)f (are)g(spa)o(wning)f(an)g(MPI)h(pro)q(cess)i(or)e(not,)75 158 y(and)g(whether)i(w)o(e)f(will)e(comm)o(unicate)f(with)j(it.)k(I)c(b)q(eliev) o(e)g Fe(MPI)p 1083 158 V 14 w(PROCESS)p 1280 158 V 13 w(A)m(TT)m(A)o(CH)f Ff(will)f(sho)o(w)h(its)h(v)o(ersatilit)o(y)75 214 y(in)h(clien)o(t/serv)o (er)j(connection)e(\(and)g(p)q(ossibly)g(later)g(on)f(for)h(fault)f (tolerance\).)19 b(-)13 b(w)o(cs)75 419 y Fl(1.4.2)49 b(Clients)16 b(and)h(Servers)75 505 y Fs(In)22 b(a)e(previous)i(section)g(w)o(e)f(describ) q(ed)i(simpli\014ed,)i(blo)q(c)o(king)d(calls)g(for)f(a)f(clien)o(t)j(and)e (serv)o(er.)37 b(In)75 561 y(general,)15 b(just)f(as)g(MPI)g(needs)i(non-blo) q(c)o(king)g(message-passing)e(op)q(erations,)h(w)o(e)f(need)h(non-blo)q(c)o (king)75 618 y(connection)h(op)q(erations.)166 674 y(The)g(new)g(functions)g (needed)h(for)e(clien)o(t-serv)o(er)i(applications)g(b)q(egin)g(with)f (routines)g(needed)h(so)75 730 y(that)11 b(the)i(clien)o(ts)g(can)g(\014nd)f (the)h(serv)o(er.)18 b(The)13 b(\014rst)e(one)i(is)g(called)g(b)o(y)f(the)h (serv)o(er)f(in)h(order)f(to)f(announce)75 787 y(that)j(it)i(is)f(ready)g(to) g(accept)g(connections.)21 b(It)15 b(pro)o(vides)g(an)g(arra)o(y)f(of)h (requests)g(that)f(it)h(can)g(test)g(and)75 843 y(w)o(ait)j(on)g(to)f(tell)i (whether)f(a)g(clien)o(t)h(wishes)g(to)f(connect.)28 b(W)l(e)19 b(assume)e(here)i(that)e(b)q(oth)h(clien)o(t)i(and)75 900 y(serv)o(er)h(are)g (MPI)g(programs,)g(whose)g(pro)q(cesses)h(are)e(already)i(in)g(in)o(ternal)g (comm)o(unication)g(via)f(a)75 956 y(comm)o(unicator.)166 1095 y Fg(Discussion:)33 b Ff(In)12 b(the)h(follo)o(wing,)d(I)j(ha)o(v)o(e)f(tak)o (en)h(the)g(lib)q(ert)o(y)f(of)g(replacing)h(the)g Fe(MPI)p 1544 1095 V 14 w(Connection)h Ff(ob)r(ject)75 1152 y(with)k(an)h Fe(MPI)p 314 1152 V 14 w(Pro)q(cess)g Ff(argumen)o(t.)32 b(I)19 b(ha)o(v)o(en't)f(heard)i(an)o(y)e(ob)r(jections,)i(but)f(of)f(course)i(it's) f(dangerous)g(to)75 1208 y(remo)o(v)o(e)13 b(something)g(y)o(ou)g(don't)g (understand)i(-)f(w)o(cs)166 1347 y Fs(The)21 b(follo)o(wing)g(non)o(blo)q(c) o(king)g(functions)g(are)g(called)g(in)h(a)e(single)h(pro)q(cess.)36 b(They)21 b(return)f(an)75 1404 y Fj(MPI)p 160 1404 14 2 v 16 w(Pro)q(cess)c Fs(ob)s(ject)e(whic)o(h)i(can)g(b)q(e)g(used)f(to)g (establish)h(comm)o(unication.)166 1460 y(The)f(\014rst)g(routine)h(is)g (called)g(b)o(y)f(the)h(serv)o(er)f(to)f(get)h(accept)g(a)g(connection)h (from)f(a)g(clien)o(t.)75 1611 y Fj(MPI)p 160 1611 V 16 w(IA)o(CCEPT\(name,)f (p)o(ro)q(cesses,)i(requests\))117 1689 y Ff(IN)155 b Fj(name)485 b Ff(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)g(whic)o(h)h(the)h(serv)o(er)g(can)f (b)q(e)h(con)o(tacted)905 1745 y(\(string\))117 1820 y(OUT)108 b Fj(p)o(ro)q(cess)451 b Ff(arra)o(y)19 b(of)f Fe(MPI)p 1148 1820 13 2 v 15 w(Pro)q(cess)h Ff(ob)r(ject)h(represen)o(ting)g(connecting)905 1877 y(clien)o(ts)117 1952 y(OUT)108 b Fj(requests)435 b Ff(one)12 b(of)g(these)h(requests)h(is)d(satis\014ed)i(when)f(a)g(clien)o(t)g(success-) 905 2008 y(fully)h(establishes)i(con)o(tact)166 2215 y Fg(Discussion:)g Ff(The)e(original)e(syn)o(tax)i(included)f(an)h(arra)o(y)f(of)h Fe(MPI)p 1232 2215 V 14 w(Connection)h Ff(ob)r(jects.)19 b(I)12 b(didn't)g(under-)75 2272 y(stand)e(at)f(\014rst)h(wh)o(y)f(an)g(arra)o(y)g (w)o(as)g(required)h(\(esp.)18 b(when)10 b(compared)e(to)h(the)h(syn)o(tax)f (for)g Fe(MPI)p 1552 2272 V 15 w(REMOTE)p 1739 2272 V 14 w(A)m(TT)m(A)o(CH)p Ff(\),)75 2328 y(but)15 b(left)g(it)g(in)g(with)g(the)h(follo)o(wing)c (rationalization:)19 b(One)d(can)f(imagine)e(ha)o(ving)h(only)g(a)h(single)g (pro)q(cess)i(and)75 2385 y(request)d(argumen)o(t)d(\(rather)j(than)e(arra)o (ys\).)18 b(After)13 b(a)f(request)i(w)o(ere)f(satis\014ed,)g(ho)o(w)o(ev)o (er,)f(there)i(w)o(ould)e(b)q(e)h(none)75 2441 y(p)q(ending,)g(and)g(sim)o (ultaneous)e(connection)j(requests)h(w)o(ould)d(fail.)k(This)d(allo)o(ws)f(a) h(serv)o(er)h(to)f(k)o(eep)h(a)e(n)o(um)o(b)q(er)h(of)75 2498 y(\\p)q(orts")h(op)q(en)g(at)g(all)f(times.)k(-)d(w)o(cs)166 2637 y Fs(The)h(next)h(routine)f(is)h(called)h(b)o(y)e(the)g(clien)o(t)i(in)f (order)f(to)f(mak)o(e)h(con)o(tact)f(with)i(the)f(serv)o(er.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 bop 75 -100 a Fm(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(21)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(ICONT)l(A)o(CT\(name,)14 b(p)o(ro)q(cess,)i(request\))117 122 y Ff(IN)155 b Fj(name)485 b Ff(w)o(ell-kno)o(wn)9 b(name)g(b)o(y)g(whic)o(h)h(the)h(serv)o(er)g(can)f (b)q(e)h(con)o(tacted)905 179 y(\(string\))117 253 y(OUT)108 b Fj(p)o(ro)q(cess)451 b Fe(MPI)p 982 253 13 2 v 15 w(Pro)q(cess)14 b Ff(represen)o(ting)h(the)g(serv)o(er)117 327 y(OUT)108 b Fj(request)452 b Ff(request)13 b(that)f(is)f(satis\014ed)h(when)g(a)f(serv)o (er)i(accepts)g(the)f(con-)905 384 y(nection)166 584 y Fg(Discussion:)31 b Ff(Building)10 b(a)h(certain)h(lev)o(el)e(of)h(securit)o(y)h(in)o(to)e(the) i(clien)o(t-serv)o(er)g(connection)g(pro)q(cess)h(could)75 634 y(b)q(e)i(done)g(in)f(the)h(follo)o(wing)d(w)o(a)o(y)m(.)19 b(An)14 b(extra)h(argumen)o(t)f(to)g Fe(MPI)p 1119 634 V 14 w(IA)o(CCEPT)e Ff(w)o(ould)i(consist)h(of)f(the)h(name)e(of)h(a)75 684 y(function)g(to)h(b)q(e)g(called)g(to)f(pro)o(vide)h(v)n(alidation)d(for) i(the)i(clien)o(t)e(that)h(is)g(attempting)e(to)i(connect.)21 b(The)15 b(clien)o(t,)75 734 y(in)d(an)h(extra)h(argumen)o(t)d(on)i(the)h (call)e(to)h Fe(MPI)p 800 734 V 14 w(ICONT)m(A)o(CT)p Ff(,)d(w)o(ould)j(pro)o (vide)f(a)h(k)o(ey)g(that)g(w)o(as)g(v)n(alidated)f(b)o(y)g(the)75 784 y(routine)j(passed)h(to)e Fe(MPI)p 480 784 V 15 w(IA)o(CCEPT)p Ff(.)d(If)k(the)g(k)o(ey)g(w)o(as)f(not)h(v)n(alidated,)e(the)j(request)g(w)o (ould)e(ha)o(v)o(e)g(no)h(e\013ect)h(on)75 833 y(the)e(requests)i(supplied)e (to)g Fe(MPI)p 599 833 V 14 w(IA)o(CCEPT)p Ff(.)166 883 y(Note)d(that)g(pro)o (viding)e(a)i Fe(MPI)p 640 883 V 14 w(Pro)q(cess)g Ff(ob)r(ject)h(to)e(the)i (clien)o(t)f(is)f(not)h(a)f(securit)o(y)i(problem)e(in)g(itself)g(b)q(ecause) 75 933 y(a)i(pro)q(cess)j(ob)r(ject)e(itself)f(do)q(es)h(not)g(confer)g(the)g (authorit)o(y)f(to)g(send)i(signals)d(-)i(only)e(the)i(mec)o(hanism.)j(If)c (a)g(clien)o(t)75 983 y(do)q(es)j(not)e(ha)o(v)o(e)h(the)h(correct)g (priviliges,)e(it)g(has)h(no)g(con)o(trol)f(o)o(v)o(er)h(the)h(serv)o(er.)166 1122 y Fs(It)j(is)g(useful)g(if)g(it)g(is)g(not)f(required)i(that)e(the)h Fj(name)f Fs(argumen)o(t)g(to)g(the)g(ab)q(o)o(v)o(e)g(calls)i(b)q(e)f(kno)o (wn)75 1178 y(un)o(til)e(execution)g(time.)k(The)c(follo)o(wing)f(routine)h (pro)o(vides)f(a)g(w)o(a)o(y)f(for)g(the)h(application)i(to)d(request)h(a)75 1235 y Fj(name)p Fs(,)f(but)i(b)q(e)g(giv)o(en)f(a)g(di\013eren)o(t)h(one)f (if)h(the)f(system)g(prefers)g(it)g(that)g(w)o(a)o(y)l(.)75 1386 y Fj(MPI)p 160 1386 14 2 v 16 w(GET)p 264 1386 V 17 w(SERVER)p 448 1386 V 17 w(NAME\(requested)p 790 1386 V 19 w(name,)g(given)p 1037 1386 V 16 w(name\))117 1463 y Ff(IN)155 b Fj(requested)p 499 1463 V 19 w(name)288 b Ff(name)13 b(that)h(the)g(serv)o(er)i(w)o(ould)d (lik)o(e)g(to)h(b)q(e)g(kno)o(wn)g(b)o(y)117 1537 y(OUT)108 b Fj(given)p 419 1537 V 17 w(name)370 b Ff(name)13 b(supplied)h(b)o(y)g(the)g (system)166 1745 y Fg(Discussion:)33 b Ff(I'm)11 b(not)i(sure)g(I)g (understand)h(the)g(p)q(oin)o(t)e(of)g(this)h(function.)18 b(If)12 b(the)i(system)e(can)i(translate)75 1801 y(b)q(et)o(w)o(een)22 b(names)d(at)h(run)h(time,)f(wh)o(y)g(not)g(just)g(alw)o(a)o(ys)g(use)h(the)g (\\w)o(ell-kno)o(wn")d(name?)36 b(I'm)18 b(not)j(sure)g(I)75 1857 y(understand)14 b(the)f(di\013erence)i(b)q(et)o(w)o(een)f(the)f (requested)h(name)e(and)g(the)i(giv)o(en)e(name)f(in)h(the)h(\014rst)h (place.)k(-)12 b(w)o(cs)166 1997 y Fs(In)23 b(order)f(for)f(comm)o(unication) i(to)e(tak)o(e)h(place,)i(an)e(in)o(ter-comm)o(unicator)h(m)o(ust)e(b)q(e)i (created)75 2053 y(connecting)17 b(the)g(clien)o(t)h(and)e(serv)o(er.)23 b(This)17 b(is)g(done)g(using)g Fj(MPI)p 1216 2053 V 16 w(PROCESS)p 1429 2053 V 18 w(A)l(TT)l(A)o(CH)g Fs(describ)q(ed)h(in)75 2109 y(the)f(previous)g(section.)24 b Fj(MPI)p 595 2109 V 16 w(PROCESS)p 808 2109 V 18 w(A)l(TT)l(A)o(CH)17 b Fs(is)g(collectiv)o(e)i(at)c (b)q(oth)i(ends)g(of)f(the)h(connection)75 2166 y(and)f(allo)o(ws)f(clien)o (t)i(and)e(serv)o(er)g(to)g(form)f(an)i(arbitrary)e(in)o(tercomm)o(unicator)i (from)e(a)h(p)q(oin)o(t-to-p)q(oin)o(t)75 2222 y(connection.)166 2355 y Fg(Discussion:)54 b Ff(This)19 b(brings)g(up)g(an)g(in)o(teresting)g (subtle)h(p)q(oin)o(t.)32 b(In)19 b(paren)o(t/c)o(hild)g(in)o(teractions,)h (all)75 2405 y(remote)10 b(pro)q(cesses)j(are)e(giv)o(en)f(to)g Fe(MPI)p 681 2405 13 2 v 15 w(PROCESS)p 879 2405 V 13 w(A)m(TT)m(A)o(CH)p Ff(.)f(In)h(clien)o(t/serv)o(er)i(in)o(teractions,)f(only)f(one)g(remote)75 2455 y(pro)q(cess)20 b(is)f(giv)o(en)f(to)g Fe(MPI)p 516 2455 V 15 w(PROCESS)p 714 2455 V 13 w(A)m(TT)m(A)o(CH)p Ff(.)f(This)h(isn't)g (consisten)o(t,)j(but)d(can)h(b)q(e)g(resolv)o(ed)h(if)d(w)o(e)i(sa)o(y)75 2504 y(that)e(in)f(general)g Fe(MPI)p 441 2504 V 15 w(PROCESS)p 639 2504 V 13 w(A)m(TT)m(A)o(CH)f Ff(ma)o(y)g(sp)q(ecify)i(an)o(y)f (arbitrary)h(collection)f(of)g(remote)g(pro)q(cesses.)75 2554 y(It)i(will)e(succeed)k(as)e(long)f(as)h(all)f(of)g(those)h(remote)g(pro)q (cesses)i(are)f(participating)d(in)i(a)f(matc)o(hing)f Fe(A)m(TT)m(A)o(CH)75 2604 y Ff(op)q(eration)f(\(either)h Fe(MPI)p 474 2604 V 14 w(PROCESS)p 671 2604 V 13 w(A)m(TT)m(A)o(CH)e Ff(or)h Fe(MPI)p 986 2604 V 14 w(COMM)p 1130 2604 V 15 w(P)m(ARENT)p Ff(\).)f(The)h(only)f (requiremen)o(t)h(is)g(that)75 2654 y(the)c(pro)q(cesses)i(\\matc)o(h)c(up".) 17 b(In)11 b(order)g(for)f(this)h(to)f(w)o(ork,)h(w)o(e)f(need)i(to)e (\014ddle)h(sligh)o(tly)e(\(?\))17 b(with)11 b(the)g(argumen)o(ts)75 2704 y(to)j Fe(MPI)p 203 2704 V 14 w(PROCESS)p 400 2704 V 13 w(A)m(TT)m(A)o(CH)p Ff(.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 bop 75 -100 a Fs(22)928 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 45 y Fg(Discussion:)32 b Ff(After)13 b(going)e(through)h (the)h(exercise)h(of)e(replacing)g(connections)h(with)f(pro)q(cesses,)j(I)d (think)75 95 y(I)i(understand)i(wh)o(y)e(connections)h(w)o(ere)g(prop)q(osed) h(in)d(the)i(\014rst)g(place.)20 b(The)15 b(problem)e(is)h(that)g(y)o(ou)g (ma)o(y)e(w)o(an)o(t)75 145 y(to)k(prev)o(en)o(t)h(the)f(serv)o(er)i(from)c (blo)q(c)o(king)h(no)g(matter)h(what)f(the)i(clien)o(t)f(do)q(es.)25 b(As)16 b(prop)q(osed)h(here,)g(the)f(serv)o(er)75 195 y(will)h(blo)q(c)o(k)g (while)h(A)m(TT)m(A)o(CHing)f(if)g(the)i(clien)o(t)f(do)q(esn't)h(do)e(a)h (matc)o(hing)f(A)m(TT)m(A)o(CH.)g(A)h(\\connection")g(w)o(as)75 244 y(more)h(than)h(a)g(simple)e(pro)q(cess)k(in)e(that)g(establishing)f(an)h (in)o(tercomm)o(unicator)d(from)i(an)h Fe(MPI)p 1669 244 13 2 v 14 w(Connection)75 294 y Ff(\(using)c(REMOTE)p 391 294 V 15 w(A)m(TT)m(A)o(CH\))g(w)o(as)g(a)f Fb(lo)n(c)n(al)h Ff(op)q(eration,)g (not)g(requiring)g(sync)o(hronization)g(with)g(the)g(remote)75 344 y(pro)q(cesses.)166 394 y(I)d(main)o(tain)d(that)j(this)g(only)f(sw)o (ept)i(the)g(problem)d(under)j(the)g(rug.)k(In)13 b(e\013ect,)h(establishing) f(a)f(connection)75 444 y(with)g(ICONNECT/IA)o(CCEPT)h(w)o(as)g(a)f(a)g(non)o (blo)q(c)o(king)f(collectiv)o(e)i(op)q(eration)f(that)h(established)g(comm)o (unica-)75 493 y(tion.)19 b(REMOTE)p 367 493 V 15 w(A)m(TT)m(A)o(CH)14 b(w)o(as)g(really)g(only)g(a)g(lo)q(cal)f(lo)q(okup)h(of)g(the)h (already-established)f(comm)o(unicator.)75 543 y(A)k(more)f(direct)i (solution)e(is)g(\(IMHO\))i(to)f(pro)o(vide)g(a)f(non-blo)q(c)o(king)g(v)o (ersion)h(of)f Fe(MPI)p 1495 543 V 14 w(PROCESS)p 1692 543 V 14 w(A)m(TT)m(A)o(CH)p Ff(.)75 593 y(This)d(has)g(all)e(the)j(same)e(asso)q (ciated)i(issues)g(of)e(non)o(blo)q(c)o(king)f(collectiv)o(e)i(op)q (erations,)g(but)g(seems)g(cleaner.)166 643 y(More)21 b(generally)m(,)g(a)f (serv)o(er)i(m)o(ust)e(alw)o(a)o(ys)f(a)o(v)o(oid)g(collectiv)o(e)i(comm)o (unicatio)o(n)d(with)i(a)g(clien)o(t)h(\(unless)75 693 y(non)o(blo)q(c)o (king)11 b(collectiv)o(e)h(is)g(a)o(v)n(ailable\))f(and)h(m)o(ust)f(alw)o(a)o (ys)g(use)i(non-blo)q(c)o(king)e(p)q(oin)o(t-to-p)q(oin)o(t)g(comm)o (unication)75 742 y(if)f(it)h(wishes)h(to)g(a)o(v)o(oid)e(deadlo)q(c)o(k)h (due)h(to)f(an)g(unco)q(op)q(erativ)o(e/incorrect)i(clien)o(t.)k(Because)c (there)g(are)f(\(curren)o(tly\))75 792 y(no)f(non-blo)q(c)o(king)e(collectiv) o(e)i(op)q(erations,)g(and)f(b)q(ecause)j(these)f(w)o(ould)e(probably)g(not)h (b)q(e)g(to)q(o)g(useful)f(to)h(a)f(serv)o(er)75 842 y(in)16 b(an)o(y)f(case)i(\(?\))25 b(W)m(e)15 b(ma)o(y)f(w)o(an)o(t)i(to)f(a)o(v)o (oid)g(the)i(issue)f(en)o(tirely)g(and)g(pro)o(vide)g(only)f(a)h(lo)q(cal)f (\(non-collectiv)o(e\))75 892 y(non-blo)q(c)o(king)e(v)o(ersion)h(of)f Fe(MPI)p 586 892 V 14 w(PROCESS)p 783 892 V 13 w(IA)m(TT)m(A)o(CH)g Ff(\(i.e,)g(without)g(the)i Fe(comm)d Ff(and)h Fe(ro)q(ot)h Ff(argumen)o(ts.)j(\).)166 942 y(If)12 b(w)o(e)h(do)f(ha)o(v)o(e)g(a)g(non)o (blo)q(c)o(king)f(collectiv)o(e)h(op)q(eration,)g(it)g(will)f(need)j(to)e (function)g(correctly)i(when)e(sev)o(eral)75 992 y(are)i(o)o(v)o(erlapp)q (ed.)k(Is)d(this)f(p)q(ossible?)75 1048 y(w)o(cs)166 1187 y Fs(Just)21 b(as)f(MPI)g(allo)o(ws)h(a)f(non)o(blo)q(c)o(king)i(send)f(to)f(b) q(e)i(matc)o(hed)e(with)h(a)f(blo)q(c)o(king)i(receiv)o(e)g(and)75 1244 y(vice)16 b(v)o(ersa,)e(the)h(ab)q(o)o(v)o(e)g(non-blo)q(c)o(king)i(op)q (erations)e(ma)o(y)f(b)q(e)i(matc)o(hed)f(with)g(their)h(simpli\014ed)h (forms,)75 1300 y Fj(MPI)p 160 1300 14 2 v 16 w(CLIENT)p 333 1300 V 16 w(CONNECT)f Fs(and)f Fj(MPI)p 748 1300 V 16 w(SERVER)p 931 1300 V 18 w(CONNECT)p Fs(.)75 1420 y Fj(Utilit)o(y)h(routines)75 1506 y Fs(In)d(this)g(section)g(w)o(e)f(collect)h(routines)g(that)f(pro)o (vide)h(access)f(to)g(the)g(data)g(con)o(tained)h(in)g(the)g(datat)o(yp)q(es) 75 1562 y Fj(MPI)p 160 1562 V 16 w(Pro)q(cess)j Fs(and)f Fj(MPI)p 505 1562 V 16 w(Resource)p Fs(.)75 1713 y Fj(MPI)p 160 1713 V 16 w(RESOURCE)p 406 1713 V 18 w(NSLOTS\(resource,)h(nslots\))117 1791 y Ff(IN)155 b Fj(resource)432 b Ff(an)14 b Fe(MPI)p 1040 1791 13 2 v 14 w(Resource)h Ff(ob)r(ject)117 1866 y(OUT)108 b Fj(nslots)480 b Ff(n)o(um)o(b)q(er)13 b(of)h(slots)75 1990 y Fj(nslots)j Fs(giv)o(es)g(the)f(n)o(um)o(b)q(er)g(of)g(recommended)h(pro)q (cesses)f(in)h(a)f(resource.)23 b(This)16 b(n)o(um)o(b)q(er)h(will)h(alw)o(a) o(ys)75 2047 y(b)q(e)e(a)f(\014nite)h(non-negativ)o(e)g(in)o(teger)f(\(it)g (ma)o(y)g(b)q(e)g(zero,)g(for)g(instance,)g(if)h(resources)f(ha)o(v)o(e)g (expired\).)75 2198 y Fj(MPI)p 160 2198 14 2 v 16 w(RESOURCE)p 406 2198 V 18 w(NSLOTS)p 595 2198 V 17 w(TOT)l(AL\(resource,)g(nslots\))117 2275 y Ff(IN)155 b Fj(resource)432 b Ff(an)14 b Fe(MPI)p 1040 2275 13 2 v 14 w(Resource)h Ff(ob)r(ject)117 2350 y(OUT)108 b Fj(nslots)480 b Ff(n)o(um)o(b)q(er)13 b(of)h(slots)75 2474 y Fj(nslots)i Fs(giv)o(es)g(the)f(total)g(n)o(um)o(b)q(er)g(of)g(pro)q(cess)g (slots)h(in)g(a)e(resource.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 bop 75 -100 a Fm(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(23)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(RESOURCE)p 406 45 V 18 w(NPROCS)p 601 45 V 18 w(RUNNING\(resource,)16 b(np)o(ro)q(cs\))117 122 y Ff(IN)155 b Fj(resource)432 b Ff(an)14 b Fe(MPI)p 1040 122 13 2 v 14 w(Resource)h Ff(ob)r(ject)117 197 y(OUT)108 b Fj(np)o(ro)q(cs)465 b Ff(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(curren)o (tly)f(running)75 416 y Fj(MPI)p 160 416 14 2 v 16 w(RESOURCE)p 406 416 V 18 w(NAME\(resource,)g(name\))117 494 y Ff(IN)155 b Fj(resource)432 b Ff(an)14 b Fe(MPI)p 1040 494 13 2 v 14 w(Resource)h Ff(ob)r(ject)117 569 y(OUT)108 b Fj(name)485 b Ff(a)14 b(string)g(iden)o(tifying)e(the)j(resource)75 693 y Fs(The)g(v)m(alue)g(of)f Fj(name)g Fs(is)h(only)g(suggested)g(b)o(y)f(the)g (MPI)h(standard.)k(If)c(the)f(resource)h(corresp)q(onds)f(to)g(a)75 750 y(particular)j(host,)f Fj(name)g Fs(should)i(b)q(e)f(the)f(name)h(of)f (that)g(host.)23 b(If)17 b(the)f(resource)h(w)o(as)f(obtained)h(from)75 806 y(a)f(request)g(for)f(m)o(ultiple)j(hosts)d(of)h(a)f(giv)o(en)i(arc)o (hitecture,)f Fj(name)f Fs(should)i(con)o(tain)f(the)g(name)g(of)g(that)75 863 y(arc)o(hitecture.)28 b(If)18 b(the)g(resource)g(w)o(as)f(obtained)h (from)f(a)h(complicated)h(resource)f(description)h(string)75 919 y(\(opaque)14 b(to)g(MPI)h(but)g(sen)o(t)f(to)g(an)g(external)h(resource) g(manager\))f Fj(name)g Fs(ma)o(y)g(b)q(e)h(that)f(string.)19 b Fj(name)75 975 y Fs(is)d(primarily)g(useful)g(when)g(disco)o(v)o(ering)g (pre-allo)q(cated)h(resources.)75 1126 y Fj(MPI)p 160 1126 14 2 v 16 w(RESOURCE)p 406 1126 V 18 w(DESCRIPTION\(resource,)e (description\))117 1204 y Ff(IN)155 b Fj(resource)432 b Ff(an)14 b Fe(MPI)p 1040 1204 13 2 v 14 w(Resource)h Ff(ob)r(ject)117 1279 y(OUT)108 b Fj(description)384 b Ff(a)14 b(string)g(describing)g(the)h (resource)75 1403 y Fs(The)i(v)m(alue)i(of)d Fj(description)j Fs(is)f(only)f(suggested)g(b)o(y)g(the)h(MPI)f(standard.)25 b(It)17 b(describ)q(es)i(the)e(resource)75 1460 y(in)g(a)e(w)o(a)o(y)g(that)g (is)h(opaque)g(to)f(MPI)h(itself,)g(but)g(ma)o(y)f(b)q(e)i(understo)q(o)q(d)f (b)o(y)g(the)g(resource)f(manager)g(or)75 1516 y(application.)21 b(Its)15 b(primary)f(use)h(is)g(in)g(conjunction)g(with)g Fj(MPI)p 1174 1516 14 2 v 16 w(Resource)p 1358 1516 V 17 w(discover)g Fs(if)g(a)f(user)g(needs)i(to)75 1573 y(get)f(more)g(detailed)h(information)g (ab)q(out)f(a)g(resource.)75 1724 y Fj(MPI)p 160 1724 V 16 w(RESOURCE)p 406 1724 V 18 w(PROCESSES\(resource,)i(n,)e(a)o(rra)o(y)p 1022 1724 V 15 w(of)p 1074 1724 V 16 w(p)o(ro)q(cesses\))117 1801 y Ff(IN)155 b Fj(resource)432 b Ff(an)14 b Fe(MPI)p 1040 1801 13 2 v 14 w(Resource)117 1876 y Ff(OUT)108 b Fj(n)564 b Ff(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(running)e(on)f(resource)117 1951 y(OUT)108 b Fj(a)o(rra)o(y)p 416 1951 14 2 v 15 w(of)p 468 1951 V 16 w(p)o(ro)q(cesses)251 b Ff(arra)o(y)14 b(of)f Fe(MPI)p 1138 1951 13 2 v 14 w(Pro)q(cess)p Ff(es)75 2075 y Fs(Gets)18 b(the)g(pro)q(cesses)g(running)i(on)e(a)f(giv)o(en)i(resource.)29 b(Returns)18 b Fa(n)g Fs(=)g(0)g(if)h(none)f(\(resource)g(is)h(then)75 2132 y(free)c(for)g(a)g(new)g(task\))166 2271 y Fg(Discussion:)30 b Ff(Do)9 b(w)o(e)i(need)g(to)f(sp)q(ecify)g(the)h(size)g(of)e(the)i(arra)o (y)f(or)g(do)g(w)o(e)g(let)g(the)h(user)g(hang)e(him/herself)s(?)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 bop 75 -100 a Fs(24)928 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(RESOURCE)p 406 45 V 18 w(FREE\(resources\))117 122 y Ff(IN)155 b Fj(resources)415 b Ff(resource)16 b(to)e(b)q(e)g(returned)i(to)e(resource)i(manager)75 247 y Fs(Releases)24 b(resources)e(to)g(resource)g(manager.)41 b(Pro)q(cesses)23 b(running)g(on)f(those)h(resources)f(are)g(lost)75 303 y(\(killed?\).)k(\(dealing)17 b(with)g(this)g(case)g(is)g(di\016cult)h (and)e(should)i(b)q(e)f(addressed\).)24 b(The)17 b(resource)g(m)o(ust)75 360 y(b)q(e)f(o)o(wned)f(b)o(y)g(the)g(caller.)166 499 y Fg(Discussion:)50 b Ff(Should)17 b(sev)o(eral)i(of)e(these)i(routines)g(b)q(e)f(com)o(bined)f (in)o(to)g(one,)i(whic)o(h)e(returns)j(lots)e(of)75 555 y(information)11 b(ab)q(out)j(a)f(resource)j(in)e(a)f(single)h(call?)75 789 y Fj(MPI)p 160 789 V 16 w(PROCESS)p 373 789 V 18 w(RESOURCE\(p)o(ro)q(cess,)j (resource\))117 866 y Ff(IN)155 b Fj(p)o(ro)q(cess)451 b Ff(an)14 b Fe(MPI)p 1040 866 13 2 v 14 w(Pro)q(cess)117 941 y Ff(OUT)108 b Fj(resource)432 b Ff(the)15 b Fe(MPI)p 1054 941 V 14 w(Resource)g Ff(on)f(whic)o(h)g(the)g(pro)q(cess)i(is)e(running)75 1066 y Fs(Gets)k(the)g(resource)g(on)h(whic)o(h)g(a)f(pro)q(cess)g(is)h(running.) 30 b(This)19 b(routine)g(can)f(only)h(b)q(e)g(called)h(on)e(the)75 1122 y(pro)q(cess)d(whic)o(h)i(o)o(wns)d(the)h(resource.)20 b(On)c(all)g(other)f(pro)q(cess)h(it)f(returns)g Fj(MPI)p 1453 1122 14 2 v 16 w(RESOURCE)p 1699 1122 V 18 w(NULL)p Fs(.)166 1261 y Fg(Discussion:)36 b Ff(Allo)o(wing)13 b(other)i(pro)q(cesses)i(to)d (retriev)o(e)i(the)f(corresp)q(onding)g(resource)i(causes)f(all)d(sorts)75 1318 y(of)h(problems.)19 b(The)c(curren)o(t)h(de\014nition)e(is)h(probably)e (\014ne)i(un)o(til)f(w)o(e)h(start)g(talking)e(ab)q(out)i(fault)e(tolerance,) i(in)75 1374 y(whic)o(h)f(case)h(w)o(e)f(ha)o(v)o(e)f(another)i(problem.)75 1608 y Fj(MPI)p 160 1608 V 16 w(PROCESS)p 373 1608 V 18 w(IN)p 436 1608 V 16 w(GROUP\(group,)h(rank,)f(p)o(ro)q(cess\))117 1685 y Ff(IN)155 b Fj(comm)470 b Ff(group)117 1760 y(IN)155 b Fj(rank)504 b Ff(rank)14 b(in)f(group)117 1835 y(OUT)108 b Fj(p)o(ro)q(cess)451 b Fe(MPI)p 982 1835 13 2 v 15 w(Pro)q(cess)14 b Ff(corresp)q(onding)h(to)e(rank)h(in)g(group)75 1960 y Fs(gets)h(the)g Fj(MPI)p 332 1960 14 2 v 16 w(Pro)q(cess)h Fs(corresp)q(onding)g(to)e(a)h (giv)o(en)h(group)f(and)g(rank.)166 2016 y(A)g(pro)q(cess)f(represen)o(ted)i (b)o(y)e(an)h Fj(MPI)p 824 2016 V 16 w(Pro)q(cess)g Fs(cannot)f(b)q(e)i(comm) o(unicated)f(with)g(directly)l(,)h(un)o(til)75 2073 y(a)i(comm)o(unicator)h (is)g(constructed)f(con)o(taining)i(it)f(\(see)f(next)h(section\).)30 b(It)19 b(need)g(not)f(b)q(e)i(an)e(\\MPI)75 2129 y(pro)q(cess")h(in)g(the)g (sense)g(that)f(it)i(migh)o(t)e(not)h(call)g Fj(MPI)p 1059 2129 V 16 w(INIT)p Fs(.)f(On)h(the)g(other)g(hand,)g(it)g(allo)o(ws)g(out-)75 2186 y(of-band)h(comm)o(unication,)h(suc)o(h)e(as)g(signals,)i(and)f(migh)o (t)f(b)q(e)h(a)f(useful)i(concept)f(for)f(dealing)i(with)75 2242 y(failures.)166 2298 y(T)l(o)15 b(ask)g(the)g(pro)q(cess)g(manager)g(to) f(deliv)o(er)j(signals)f(to)f(pro)q(cesses,)g(w)o(e)g(use)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 bop 75 -100 a Fm(1.4.)34 b(SIMPLE)16 b(INTERF)-5 b(A)o(CE)1191 b Fs(25)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(PROCESS)p 373 45 V 18 w(SIGNAL\(signal,)15 b(num)p 782 45 V 16 w(p)o(ro)q(cesses,)i(a)o(rra)o (y)p 1096 45 V 14 w(of)p 1147 45 V 16 w(p)o(ro)q(cesses\))117 122 y Ff(IN)155 b Fj(signal)480 b Ff(signal)13 b(t)o(yp)q(e)h(\(in)o(t\))117 197 y(IN)155 b Fj(num)p 404 197 V 17 w(p)o(ro)q(cesses)314 b Ff(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(in)e(arra)o(y)117 273 y(IN)155 b Fj(a)o(rra)o(y)p 416 273 V 15 w(of)p 468 273 V 16 w(p)o(ro)q(cesses)251 b Fe(MPI)p 982 273 13 2 v 15 w(Pro)q(cess)p Ff(es)15 b(to)e(b)q(e)i(signalled)166 397 y Fs(It)22 b(is)g(the)f(resp)q (onsibilit)o(y)j(of)d(an)h(implemen)o(tation)h(to)e(translate)g(b)q(et)o(w)o (een)h(signals;)j(in)d(other)75 453 y(w)o(ords,)17 b(a)h Fi(SIGINT)f Fs(that)g(has)g(v)m(alue)i Fi(3)f Fs(on)g(system)f(A)h(m)o(ust)f(b)q(e)h (deliv)o(ered)i(as)d(a)h Fi(SIGINT)f Fs(on)g(system)75 510 y(B,)e(ev)o(en)h(if)f(system)g(B)h(uses)f(the)g(v)m(alue)i Fi(5)e Fs(for)g Fi(SIGINT)p Fs(.)f(If)h(the)h(signal)g(can)f(not)g(b)q(e)h (deliv)o(ered)h(b)q(ecause)75 566 y(there)e(is)h(no)f(corresp)q(onding)h (signal,)g(the)f(error)g(co)q(de)h(is)f Fj(MPI)p 1159 566 14 2 v 16 w(ERR)p 1260 566 V 17 w(INV)l(ALID)p 1449 566 V 16 w(SIGNAL)p Fs(.)166 699 y Fg(Discussion:)29 b Ff(Should)9 b(there)i(b)q(e)e(a)g (single-request)i(v)o(ersion)e(of)g(this,)h(and)f(the)h(ab)q(o)o(v)o(e)f (renamed)g Fe(MPI)p 1767 699 13 2 v 14 w(PROCESS)p 1964 699 V 13 w(SIGNAL)p 2119 699 V 15 w(ALL)p Ff(?)75 749 y(Is)14 b(the)h(notion)e (of)g(signals)g(to)q(o)h(Unix-cen)o(tric?)166 799 y(Nah.)21 b(The)15 b(notion)f(of)g(signals)g(is)h(not)f(Unix-cen)o(tric)i({)e(ANSI)h(C) g(requires)h(the)f(C)g(library)f(to)h(implem)o(en)o(t)75 848 y(\\signal\(\)")10 b(and)h(\\raise\(\)")h(\(presumably)m(,)e(to)h(the)h(b)q (est)g(of)f(the)h(underlying)f(system's)g(abilit)o(y\))f(and)h(to)g(deal)g (with)75 898 y(\(at)j(least\))g(a)g(small)d(set)k(of)e(signals,)g(one)h(of)g (whic)o(h)f(is)h(SIGINT)166 1037 y Fs(There)j(needs)h(to)f(b)q(e)h(some)f(w)o (a)o(y)f(of)h(\014nding)h(out)f(when)h(a)f(pro)q(cess)g(\014nishes.)27 b(In)18 b(MPI)f(w)o(e)g(ha)o(v)o(e)75 1094 y(no)f(mec)o(hanism)h(for)e(async) o(hronous)h(noti\014cation.)23 b(Therefore)16 b(the)h(b)q(est)f(w)o(e)g(can)g (do)g(is)h(to)e(construct)75 1150 y(a)f(request)g(that)f(can)h(b)q(e)h (tested)f(and)g(w)o(aited)g(on.)19 b(If)14 b(pro)q(cesses)h(w)o(ere)e (represen)o(ted)i(b)o(y)f(requests,)g(then)75 1207 y(w)o(e)k(could)g(w)o(ait) g(on)g(them)f(directly)l(.)30 b(Since)19 b(w)o(e)f(ha)o(v)o(e)f(in)o(tro)q (duced)i(the)f(notion)g(of)g Fj(MPI)p 1636 1207 14 2 v 15 w(Pro)q(cess)p Fs(,)h(w)o(e)75 1263 y(explicitly)f(attac)o(h)c(a)h(request)g(to)g(it)g(in)h (order)f(to)g(w)o(ait)f(on)i(it.)75 1414 y Fj(MPI)p 160 1414 V 16 w(NOTIFY)p 341 1414 V 17 w(WHEN)p 492 1414 V 16 w(DONE\(n,)g(a)o(rra)o (y)p 798 1414 V 15 w(of)p 850 1414 V 16 w(p)o(ro)q(cesses,)g(request\))117 1491 y Ff(IN)155 b Fj(n)564 b Ff(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(in) e(follo)o(wing)d(arra)o(y)117 1566 y(IN)155 b Fj(a)o(rra)o(y)p 416 1566 V 15 w(of)p 468 1566 V 16 w(p)o(ro)q(cesses)251 b Ff(arra)o(y)14 b(of)f Fe(MPI)p 1138 1566 13 2 v 14 w(Pro)q(cess)i Ff(ob)r(jects)117 1642 y(OUT)108 b Fj(request)452 b Ff(mpifuncMPI)p 1136 1642 V 14 w(Request)15 b(to)e(b)q(e)i(tested)g(and/or)f(w)o(aited)f(on) 166 1849 y Fg(Discussion:)40 b Ff(Should)15 b Fe(MPI)p 657 1849 V 15 w(NOTIFY)p 824 1849 V 14 w(WHEN)p 960 1849 V 15 w(DONE)g Ff(return)i(an)e(arra)o(y)h(of)f(requests)i(instead)f(of)f(just)75 1905 y(one?)166 2044 y Fs(W)l(e)g(can)g(think)g(of)g(this)g(as)g(MPI)g(in)o (terface)g(to)f(the)h(Pro)q(cess)g(Manager's)e(handling)k(of)d(the)h(signal) 75 2101 y Fi(SIGCHILD)p Fs(.)e(The)i(exit)g(co)q(de)g(from)f(the)g(pro)q (cess)h(\(from)f(a)g Fi(return)23 b(n)14 b Fs(or)g Fi(exit\(n\))g Fs(in)h(C)f(or)g Fi(STOP)24 b(n)14 b Fs(in)75 2157 y(F)l(ortran\))g(can)h(b)q (e)h(retriev)o(ed)g(from)e(the)h Fj(MPI)p 863 2157 14 2 v 16 w(status)i Fs(\014lled)g(in)f(the)g Fj(MPI)p 1332 2157 V 15 w(W)l(AIT)p Fs(.)166 2214 y(Note)h(that)g(this)h(lev)o(el)h(of)e(pro)q(cess)h (managemen)o(t)f(allo)o(ws)h(us)g(to)f(manage)g(non-MPI)h(pro)q(cesses,)75 2270 y(since)e(comm)o(unicators)f(are)g(not)g(in)o(v)o(olv)o(ed.)166 2327 y(If)22 b(the)g(started)g(pro)q(cesses)g(are)g(MPI)g(pro)q(cesses)g (\(that)f(is,)j(if)f(they)f(call)h Fj(MPI)p 1588 2327 V 16 w(INIT)p Fs(\),)e(then)h(a)75 2383 y(comm)o(unicator)f(that)f(includes)k (them)d(can)g(b)q(e)h(constructed)g(with)f(the)g Fj(MPI)p 1472 2383 V 16 w(PROCESS)p 1685 2383 V 18 w(A)l(TT)l(A)o(CH)75 2439 y Fs(function)h(describ)q(ed)g(b)q(elo)o(w.)38 b(In)21 b(either)h(case,)g(at) e(this)h(stage)f(w)o(e)h(ha)o(v)o(e)f(allo)q(cated)i(resources)f(and)75 2496 y(the)c(pro)q(cesses)h(ha)o(v)o(e)f(b)q(een)h(started.)25 b(The)18 b(third)g(comp)q(onen)o(t)f(of)g(the)g(run)o(time)h(en)o(vironmen)o (t)f(is)h(the)75 2552 y(message-passing)f(library)l(.)25 b(In)17 b(order)f(to)g(comm)o(unicate)h(with)g(these)g(pro)q(cesses,)f(w)o(e)h(m)o (ust)f(in)o(terface)75 2609 y(with)g(the)f(message-passing)g(library)l(.)21 b(In)16 b(MPI)f(terms,)f(this)i(means)f(setting)g(up)h(comm)o(unicators.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 bop 75 -100 a Fs(26)933 b Fm(CHAPTER)16 b(1.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 45 y Fp(1.5)59 b(Examples)75 147 y Fj(Manager-w)o(o)o(rk)o (er)13 b(example.)75 240 y Fi(/*)24 b(manager)e(*/)75 297 y(#include)h ()75 353 y(#define)g(MAXPROC)g(128)75 410 y(main\(int,)g(argc,)g(char) g(*arg[]\))75 466 y({)147 523 y(MPI_Resource)f(resource;)147 579 y(MPI_Status)g(status;)147 636 y(int)h(count,)g(world_size;)147 692 y(MPI_Comm)f(everyone;)75 748 y(#ifndef)h(SPAWNEM)147 805 y(MPI_Process)f(processes[MAXPROC];)147 861 y(MPI_Comm)g(my_children;)75 918 y(#endif)147 1087 y(MPI_Init\(&argc,)f(&argv\);)147 1144 y(MPI_Comm_size\(MPI_COMM_)o(WORLD,)f(&world_size\);)147 1200 y(if)j(\(world_size)g(!=)g(1\))h(error\("Top)e(heavy)h(with)h(management"\);) 147 1257 y(MPI_Resource_discover\(M)o(PI_RESO)o(URCE_TYP)o(E_DEFAU)o(LT,)d (NULL,)i(&resource\);)147 1369 y(MPI_Resource_nslots\(res)o(ource,)d (&count\);)147 1426 y(if)j(\(count)g(<)h(1\))g(error\("No)e(resources"\);)147 1482 y(if)h(\(count)g(>)h(MAXPROC\))f({)g(warning\("Too)g(many)g (resources"\);)f(count)h(=)h(MAXPROC)f(};)75 1595 y(#ifdef)g(SPAWNEM)147 1652 y(MPI_Spawn\(MPI_COMM_SELF)o(,)e(0,)j(count,)f("worker",)f(NULL,)i (resource,)e(&everyone\);)75 1708 y(#else)147 1765 y (MPI_Process_create\("wor)o(ker",)f(NULL,)i(count,)g(resource,)f (processes\);)147 1821 y(MPI_Process_attach\(MPI_)o(COMM_SE)o(LF,)f(0,)j (count,)f(processes,)f(&my_children\);)147 1878 y(MPI_Intercomm_merge\(my_)o (childre)o(n,)f(FALSE,)i(&everyone\);)75 1934 y(#endif)147 2047 y(/*)170 2103 y(*)h(Parallel)f(code)g(here.)g(The)h(communicator)e ("everyone")h(can)g(be)h(used)170 2160 y(*)g(to)g(communicate)e(with)h(the)h (spawned)f(processes)170 2216 y(*/)147 2329 y(MPI_Finalize\(\);)75 2386 y(})75 2499 y(/*)h(worker)f(*/)75 2611 y(#include)g()75 2668 y(main\(int,)g(argc,)g(char)g(*argv[]\))1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 bop 75 -100 a Fm(1.5.)29 b(EXAMPLES)1398 b Fs(27)75 45 y Fi({)147 102 y(MPI_Comm)22 b(parent;)95 b(/*)23 b(intercommunicator)f(*/)147 158 y(MPI_Comm)g(everyone;)47 b(/*)23 b(intracommunicator)f(*/)147 214 y(MPI_Init\(&argc,)f(&argv\);)147 271 y(MPI_Parent\(&parent\);)147 327 y(MPI_Intercomm_merge\(par)o(ent,)g(TRUE,)i(&everyone\))147 440 y(/*)170 497 y(*)h(Parallel)f(code)g(here.)170 553 y(*/)147 610 y(MPI_Finalize\(\);)75 666 y(})75 899 y Fj(T)l(ask)12 b(fa)o(rm)e (example.)45 b Fs(The)12 b(nice)h(thing)g(here)f(is)g(that)f(y)o(ou)h(need)h (allo)q(cate)f(resources)g(only)g(once.)19 b(Then)75 956 y(y)o(ou)c(can)g (reuse)g(them.)20 b(It)15 b(also)g(sho)o(ws)f(that)g(y)o(ou)h(can)g(in)o (teract)g(with)g(pro)q(cesses)g(without)g(using)h(MPI)75 1012 y(to)f(comm)o(unicate.)75 1118 y Fi(#include)23 b()75 1175 y(#define)g(MAXPROC)g(100)75 1231 y(main\(int)g(argc,)g(char)g (*argv[]\))75 1288 y({)147 1344 y(MPI_Resource)f(resource;)147 1401 y(MPI_Process)g(processes[MAXPROC];)147 1457 y(MPI_Request)g (obituaries[MAXPROC];)147 1513 y(MPI_Status)g(statuses[MAXPROC];)45 b(/*)24 b(see)f(comment)g(below)g(*/)147 1570 y(int)g(nslots,)g(i,)h (deadone;)147 1626 y(char)f(**args;)147 1739 y(ThingToDo)f(*thingstodo;)147 1796 y(int)h(nthingstodo,)f(nthingsdone;)147 1909 y(MPI_Init\(&argc,)f (&argv\);)147 1965 y(MPI_Resource_discover\(M)o(PI_RESO)o(URCE_TYP)o(E_DEFAU) o(LT,)g(NULL,)i(&resource\);)147 2022 y(MPI_Resource_nslots\(res)o(ource,)d (&nslots\);)147 2134 y(GetThingsToDo\(&thingsto)o(do,)h(nthingstodo\);)147 2247 y(/*)i(need)g(to)h(initialize)f(these)g(for)g(Waitany/all\(\))f(below)h (to)h(work)f(*/)147 2304 y(for)g(\(i)h(=)f(0;)h(i)g(<)f(MAXPROC;)g(i++\))g (obituaries[i])g(=)g(MPI_REQUEST_NULL;)147 2417 y(/*)g(start)g(up)h(tasks)f (on)h(initial)f(resources)f(*/)147 2473 y(for)h(\(nthingsdone)f(=)i(0;)g (nthingsdone)e(<)i(nslots)f(&&)g(nthingsdone)g(<)g(nthingstodo;)g (nthingsdone++\))f({)194 2530 y(SetArgs\(thingstodo[nthingsdo)o(ne],)f (&args\);)194 2586 y(MPI_Process_create\("cow",)g(args,)i(1,)h (MPI_PROCESS_TYPE_NONMPI)o(,)648 2643 y(resource,)e (&processes[nthingsdone]\);)194 2699 y(MPI_Process_notify_when_done)o (\(nthings)o(done,)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 28 30 bop 75 -100 a Fs(28)933 b Fm(CHAPTER)16 b(1.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)648 45 y Fi(&processes[nthingsdone],)648 102 y(&obnthingsdonetuaries[nt)o(hingsdo)o(ne]\);)147 158 y(})147 271 y(/*)23 b(spawn)g(new)h(ones)f(as)h(old)f(ones)g(finish)h(*/)147 327 y(while\(nthingsdone)d(<)j(nthingstodo\))e({)194 384 y(/*)i(perhaps)f (could)g(have)g(some)h(element)f(of)g(status)g(=)h(return)218 440 y(*)g(code)f(so)h(we)f(could)h(check)f(for)g(error)g(?)h(*/)194 497 y(MPI_Waitany\(nslots,)e(obituaries,)g(&deadone,)h(&status[0]\);)194 553 y(SetArgs\(thingstodo[nthingsdo)o(ne],)e(&args\);)194 610 y(/*)j(it)f(would)h(probably)e(be)i(more)f(efficient)g(to)h(use)218 666 y(*)g(MPI_Process_icreate)d(here)i(*/)194 723 y (MPI_Process_create\("cow",)e(args,)i(1,)h(MPI_PROCESS_TYPE_NONMPI)o(,)648 779 y(resource,)e(&processes[nthingsdone]\);)194 835 y (MPI_Process_notify_when_done)o(\(nthings)o(done,)886 892 y (&processes[nthingsdone],)886 948 y(&obituaries[nthingsdone]\);)147 1061 y(})147 1118 y(/*)h(this)g(is)h(a)g(good)f(reason)g(why)h(MPI)f(should)g (allow)g(you)h(to)f(pass)170 1174 y(*)h(statusptr)f(=)h(NULL)170 1231 y(*/)147 1287 y(MPI_Waitall\(nslots,)d(obituaries,)h(statuses\);)147 1400 y(MPI_Finalize\(\);)75 1456 y(})75 1689 y Fj(PVM-st)o(yle)16 b(SPMD)f(example)45 b Fs(This)16 b(is)g(ho)o(w)f(man)o(y)f(PVM)h(programs)f (are)h(t)o(ypically)i(written.)j(There)75 1746 y(is)h(no)g(reason)g(they)g (can't)f(b)q(e)i(done)f(with)g(MPI-1,)h(but)f(in)h(case)f(users)g(w)o(an)o(t) f(the)h(app)q(earance)g(of)75 1802 y(minimal)c(c)o(hange,)e(here)g(is)h(is.)k (\(V)l(ery)15 b(similar)i(to)d(the)i(manager-w)o(ork)o(er)d(example)j(ab)q(o) o(v)o(e.\))75 1909 y Fi(#include)23 b()75 1965 y(#define)g(MAXPROC)g (128)75 2022 y(main\(int,)g(argc,)g(char)g(*argv[]\))75 2078 y({)147 2134 y(MPI_Resource)f(resource;)147 2191 y(MPI_Status)g(status;)147 2247 y(int)h(count,)g(world_size;)147 2304 y(MPI_Comm)f(everyone;)147 2360 y(MPI_Process)g(processes[MAXPROC];)147 2417 y(MPI_Comm)g(my_children,)h (my_parent;)147 2586 y(MPI_Init\(&argc,)e(&argv\);)147 2643 y(MPI_Comm_parent\(&my_par)o(ent\);)147 2699 y(if)i(\(my_parent)g(==)g (MPI_COMM_NULL\))f({)i(/*)g(I'm)f(the)g(parent)g(*/)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 29 31 bop 75 -100 a Fm(1.5.)29 b(EXAMPLES)1398 b Fs(29)218 45 y Fi (MPI_Resource_discover\(MPI_R)o(ESOURCE_)o(TYPE_DE)o(FAULT,)20 b(NULL,)k(&resource\);)218 102 y(MPI_Resource_nslots\(resourc)o(e,)d (&count\);)218 158 y(if)j(\(count)f(<)h(1\))f(error\("No)g(resources"\);)218 214 y(if)h(\(count)f(>)h(MAXPROC\))e({)i(warning\("Too)e(many)i (resources"\);)e(nprocs)h(=)h(MAXPROC)e(};)218 271 y (MPI_Process_create\(&argv[0])o(,)f(NULL,)i(count,)g(resource,)g (processes\);)218 327 y(MPI_Process_attach\(MPI_COMM)o(_SELF,)e(0,)i(count,)g (processes,)g(&my_children\);)218 384 y(MPI_Intercomm_merge\(my_chil)o(dren,) e(FALSE,)i(&everyone\);)170 440 y(})h(else)f({)218 497 y (MPI_Intercomm_merge\(my_pare)o(nt,)e(TRUE,)i(&everyone\);)170 553 y(})170 666 y(/*)h(SPMD)f(parallel)g(code)g(here,)h(using)f(everyone)g (instead)f(of)i(MPI_COMM_WORLD)e(*/)170 779 y(MPI_Finalize\(\);)75 835 y(})75 1012 y Fj(Client-server)14 b(example.)44 b Fs(This)14 b(is)f(a)f(simple)j(example;)f(the)f(serv)o(er)f(accepts)h(only)g(a)g(single) h(connection)75 1068 y(at)h(a)f(time)i(and)f(serv)o(es)g(that)g(connection)h (un)o(til)g(the)g(clien)o(t)g(requests)f(to)g(b)q(e)h(disconnected.)166 1125 y(Here)21 b(is)h(the)g(serv)o(er.)38 b(It)21 b(accepts)h(a)f(single)h (connection)h(and)e(then)h(pro)q(cesses)g(data)e(un)o(til)j(it)75 1181 y(receiv)o(es)16 b(a)f(message)g(with)g(tag)f Fi(1)p Fs(.)20 b(A)15 b(message)g(with)h(tag)e Fi(0)h Fs(tells)h(the)g(serv)o(er)e(to)h (exit.)75 1275 y Fi(#include)23 b("mpi.h")75 1332 y(main\()g(int)h(argc,)f (char)g(**argv)g(\))75 1388 y({)75 1445 y(MPI_Comm)g(client;)75 1501 y(MPI_Status)f(status;)75 1557 y(double)h(buf[MAX_DATA];)75 1614 y(int)95 b(again;)75 1727 y(MPI_Init\()23 b(&argc,)g(&argv)g(\);)75 1783 y(while)g(\(1\))h({)170 1840 y(MPI_Server_connect\()e(MPI_COMM_WORLD,)g ("cave:1234",)g(&client)h(\);)170 1896 y(again)h(=)f(1;)170 1953 y(while)h(\(again\))f({)266 2009 y(MPI_Recv\()g(buf,)g(MAX_DATA,)f (MPI_DOUBLE,)h(0,)g(MPI_ANY_TAG,)505 2066 y(client,)f(&status)h(\);)266 2122 y(switch)g(\(status.tag\))f({)361 2178 y(case)i(0:)f(MPI_Comm_free\()f (&client)h(\);)552 2235 y(MPI_Finalize\(\);)552 2291 y(return)g(0;)361 2348 y(case)h(1:)f(MPI_Comm_free\()f(&client)h(\);)552 2404 y(again)g(=)h(0;)552 2461 y(break;)361 2517 y(case)g(2:)f(/*)h(do)f (something)g(*/)361 2574 y(...)361 2630 y(default:)552 2687 y(MPI_Abort\()g(MPI_COMM_WORLD,)f("Unexpected)g(message)h(type")g(\);)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 30 32 bop 75 -100 a Fs(30)928 b Fm(CHAPTER)16 b(1.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)361 45 y Fi(})266 102 y(})170 158 y(})75 214 y(})166 308 y Fs(Here)g(is)h(the)f(clien)o(t.)75 415 y Fi(#include)23 b("mpi.h")75 471 y(main\()g(int)h(argc,)f(char)g(**argv)g(\))75 527 y({)75 584 y(MPI_Comm)g(server;)75 640 y(double)g(buf[MAX_DATA];)75 753 y(MPI_Init\()g(&argc,)g(&argv)g(\);)75 810 y(MPI_Client_connect\()e (MPI_COMM_WORLD,)h("cave:1234",)g(&server)h(\);)75 866 y(while)g(\(!done\))g ({)170 923 y(tag)h(=)g(2;)f(/*)h(Action)f(to)g(perform)g(*/)170 979 y(MPI_Send\()g(buf,)g(n,)h(MPI_DOUBLE,)e(0,)i(tag,)f(server)g(\);)170 1036 y(/*)h(etc)f(*/)170 1092 y(})75 1148 y(MPI_Send\()g(buf,)g(0,)h (MPI_DOUBLE,)e(0,)h(1,)h(server)f(\);)75 1205 y(MPI_Comm_free\()f(&client)h (\);)75 1261 y(MPI_Finalize\(\);)75 1318 y(})166 1424 y Fs(If)17 b(the)g(serv)o(er)f(needs)i(to)e(manage)g(m)o(ultiple)i(connections)g(at)e (once,)h(it)g(can)g(use)g Fj(MPI)p 1675 1424 14 2 v 16 w(IA)o(CCEPT)75 1481 y Fs(instead.)j(The)c(clien)o(t)g(need)g(not)f(b)q(e)h(c)o(hanged.)166 1537 y(\010ibliograph)o(yst)o(yleplain)i(\010ibliograph)o(yrefs)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Tue Jul 25 16:27:27 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA22049; Tue, 25 Jul 1995 16:27:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA11041 for mpi-core-out; Tue, 25 Jul 1995 15:34:51 -0500 Received: from infomeister.osc.edu (infomeister.osc.edu [128.146.36.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA11018 for ; Tue, 25 Jul 1995 15:34:38 -0500 Received: from tbag.osc.edu for gdburns@tbag.osc.edu by infomeister.osc.edu (8.6.10/930601.1506) id QAA07005; Tue, 25 Jul 1995 16:34:35 -0400 Received: by tbag.osc.edu (4.1/OSC-SUB-2.0) id AA23745; Tue, 25 Jul 95 16:34:32 EDT Date: Tue, 25 Jul 95 16:34:32 EDT From: gdburns@tbag.osc.edu (Greg Burns) Message-Id: <9507252034.AA23745@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: Re: 2 comments Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > pg 42, l37. actually p. 42, ll. 44-46 in the 1.1 version of the document > "... Similarly, a call to MPI_WAIT that completes a send will eventually > return if a matching receive has been started, unless the receive is satisfied > by another send, AND EVEN IF NO CALL IS EXECUTED TO COMPLETE THE RECEIVE." > Replace "MPI_Send" by "MPI_ISend; MPI_Wait" in my example. Yes, the non-blocking section says a lot more about completion than the blocking section. Let us also recognize the following expression expansion. An MPI_Ssend is equivalent to an MPI_Send implemented with a short/long message protocol and the short message size is zero. First, from the blocking section, p. 38, ll. 10-12: "However, the [synchronous] send will complete successfully only if a matching receive is posted, and the receive operation has started to receive the message sent by the synchronous send." This implies that two actions are needed to complete the send: 1- The MPI_Irecv() posts the operation which means information has been supplied about the operation. 2- The receive operation started to received the message. The implementation may require something else to happen (e.g. any other MPI call, not just completion calls) before actually "starting to receive the message". Then, from the non-blocking section, p. 36, ll. 47-48: "A separate receive complete call is needed to complete the receive operation and verify that the data has been received into the receive buffer." X (completion) requires Y (complete call). So your receive is incomplete. Let's say I have no system buffers. Therefore the message may still reside in the sender's buffer. Therefore your send is incomplete. and p. 37, ll. 16-17: "If the send mode is synchronous, then the send can complete only if a matching receive has started." A matching receive is one prerequiste, not the full set of conditions. This does not say it must complete on this one condition. and p. 37, ll. 26-27: "On the other hand, the [standard] send-complete may not complete until a matching receive occurred, and the message was copied into the receive buffer." Again, there are two actions here. You have supplied the first action but we have flexibility in taking the second (system) action. Therefore the send-complete is allowed not to complete until the second action is taken. There are other useful quotes, but let's skip to yours, from the "extended" definition of the progress semantic in 3.7.4. > "... Similarly, a call to MPI_WAIT that completes a send will eventually > return if a matching receive has been started, unless the receive is satisfied > by another send, AND EVEN IF NO CALL IS EXECUTED TO COMPLETE THE RECEIVE." I take this to mean one of the request completion calls, MPI_Wait*, MPI_Test* and possibly MPI_Request_free. Other calls may be required to allow the system to progress. -=- With all cards on the table, I'll allow that the standard suggests that a while(1); after the MPI_Irecv will guarantee that the MPI_Send will complete, which I'm sure is your position and desire and intent if you in fact wrote the relevant sections. My desire is to allow for simpler, peak efficient implementations on the widest possible range of parallel hardware. This means no reliance on interrupts to make MPI work. I am trying to demonstrate that there is enough leeway in the standard to create fully compliant implementations that rely exclusively on "in-line polling" (to extend your terminology). We think this is a fine quality of MPI. Now this all started with your position that MPI-2 extensions requiring interrupts poses no serious complexity delta because proper implementations already have such infrastructure. And I say no way. -- Greg From mpi-core-human@mcs.anl.gov Mon Aug 7 12:13:49 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA26687; Mon, 7 Aug 1995 12:13:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA20477 for mpi-core-out; Mon, 7 Aug 1995 11:22:40 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA20469 for ; Mon, 7 Aug 1995 11:22:33 -0500 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id MAA23191 for ; Mon, 7 Aug 1995 12:20:36 -0400 Received: from b125.super.org (b125 [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with ESMTP id MAA04809 for ; Mon, 7 Aug 1995 12:22:00 -0400 From: lederman@super.org (Steve Huss-Lederman) Received: (lederman@localhost) by b125.super.org (8.6.12/8.6.12) id MAA17434; Mon, 7 Aug 1995 12:22:00 -0400 Date: Mon, 7 Aug 1995 12:22:00 -0400 Message-Id: <199508071622.MAA17434@b125.super.org> To: mpi-core@mcs.anl.gov Subject: latest document available for authors Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy, I have placed the latest MPI-2 document at ftp://ftp.super.org/pub/mpi/mpi2/Editor. Please use this version from which to start edits for the next meeting. Authors please note the following items that were added to the README. Following these will make document integration easier. - if you add new references to ref.bib please add them at the top and let the editor know. That way they can easily be incorporated into the master copy. - to include postscript figures, add the desired figure to the subdirectory figures/ with the name chap-name.eps. For example, you could have figures/ei-fig1.eps. Please use epsf to include figures. A sample is: \begin{figure}[htpb] \centerline{\epsfxsize=4.0in\epsfbox{figures/ei-fig1.eps}} \caption[\ ]{Figure for external interfaces chapter} \label{fig:ei-fig1} \end{figure} epsf is included in the chapter and report templates so you do not need to add it to each chapter. - if you add a new label, please preface it with the chapter name so we don't get name conflicts later. Thanks, Steve From mpi-core-human@mcs.anl.gov Fri Aug 18 16:30:33 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA09356; Fri, 18 Aug 1995 16:30:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA19010 for mpi-core-out; Fri, 18 Aug 1995 15:29:04 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA19002 for ; Fri, 18 Aug 1995 15:28:56 -0500 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id QAA18424 for ; Fri, 18 Aug 1995 16:26:51 -0400 Received: from hume.super.org (hume [192.239.79.11]) by gotham.super.org (8.6.12/8.6.12.1) with SMTP id QAA11706 for ; Fri, 18 Aug 1995 16:28:24 -0400 Date: Fri, 18 Aug 1995 16:28:24 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199508182028.QAA11706@gotham.super.org> Received: by hume.super.org (4.1/SMI-4.1) id AA04895; Fri, 18 Aug 95 16:28:24 EDT To: mpi-core@mcs.anl.gov Subject: deadline for meeting input Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy, This is a note to let everyone know that the deadline for input to the document to be distributed at the next MPI meeting on Sept. 6-8 in Chicago is THURSDAY, AUGUST 31. The final document needs to go to reproduction on Friday morning since Monday is a holiday. See you in Chicago! Steve From mpi-core-human@mcs.anl.gov Tue Aug 22 18:41:13 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA20344; Tue, 22 Aug 1995 18:41:12 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA21193 for mpi-core-out; Tue, 22 Aug 1995 17:39:57 -0500 Received: from SSD.intel.com (ssd.intel.com [137.46.3.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA21180 for ; Tue, 22 Aug 1995 17:39:48 -0500 Received: from sv004.scic.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA08431; Tue, 22 Aug 95 15:39:13 PDT Received: from rs021.scic.intel.com by sv004.scic.intel.com (4.1/sun41.1); Tue, 22 Aug 95 15:39:16 PDT Received: from pc032.scic.intel.com by rs021.scic.intel.com (AIX 3.2/UCB 5.64/rs32.1); Tue, 22 Aug 1995 15:39:12 -0700 Date: Tue, 22 Aug 1995 15:39:12 -0700 Message-Id: <9508222239.AA14300@rs021.scic.intel.com> X-Sender: prp@prp-mail X-Mailer: Windows Eudora Version 2.0.3 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: mpi-core@mcs.anl.gov From: prp@ssd.intel.com (Paul Pierce) Subject: One-sided communications Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk For the next version of my get/put proposal, look in http://www.teleport.com/~prp/papers/ This contains all the get/put stuff we talked about at the last two meetings, fleshed out with variations, plus accumulate, read/modify/write and scatter/gather possibilities. There is extreme bloat due to explosion of different varieties of the same calls. The basic initiator call families are: put synchronous put get accumulate read/modify/write put-scatter get-gather while the variations are: blocking/non-blocking plain/counter-incrementing resulting in 28 calls, but synchronous and scatter/gather could be done as variations yielding about 48 calls. While I'm not averse to neat orthogonal sets of calls, I don't think all the variations really add much in this case. In particular, I think we should drop synchronous put; and the counter-incrementing variations should be the only option on the fancier calls if we decide to keep them. By the way, this copy doesn't contain Marc's old proposal, which is still active. It also doesn't have HRECV etc; as far as I'm concerned the "Communication Handlers" section in Marc's proposal is a fine starting point. If it is not convenient to get this over the Web, let me know and I will email a copy. Paul Pierce From mpi-core-human@mcs.anl.gov Wed Aug 23 18:11:28 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA06137; Wed, 23 Aug 1995 18:11:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA13058 for mpi-core-out; Wed, 23 Aug 1995 17:10:52 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA13049 for ; Wed, 23 Aug 1995 17:10:42 -0500 Message-Id: <199508232210.RAA13049@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: MPI Forum meeting, September 6-8 Date: Wed, 23 Aug 1995 17:10:40 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, I need a head count for the hotel for the MPI Forum meeting on September 6-8. You should make your reservations before August 28th in order to get the good room rate. Note that this means that you probably want to make your reservations *tomorrow*. Please let me know as soon as possible whether you are planning to come or not. If you have any questions, don't hesitate to send me mail at lusk@mcs.anl.gov or call me at (708) 252-7852. Regards, Rusty The usual blurb: The O'Hare Hilton is the hotel *inside* the airport. You just walk across the street from the baggage-claim level. A block of rooms has been reserved, which will be held until August 28, so please make your reservations with the Hotel before then. The phone number for the hotel is (312) 686-8000 (FAX (312) 601-2876). Mention the *MPI Forum* to get the special rate, which is $90.50 + tax per night. We will charge the usual $75 registration to take care of the meeting room, lunch on Thursday, and coffee, etc. The meeting will start at 1:30 on Wednesday to allow people to fly in that morning, and end at noon on Friday. From mpi-core-human@mcs.anl.gov Tue Aug 29 13:09:59 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA03465; Tue, 29 Aug 1995 13:09:58 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA17152 for mpi-core-out; Tue, 29 Aug 1995 12:09:04 -0500 Received: from mailhost.lanl.gov (mailhost.lanl.gov [128.165.3.12]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA17144 for ; Tue, 29 Aug 1995 12:08:56 -0500 Received: from wrangler.lanl.gov by mailhost.lanl.gov (8.6.11/1.2) id LAA21626; Tue, 29 Aug 1995 11:08:55 -0600 Received: from sutra.lanl.gov (sutra.lanl.gov [128.165.64.18]) by wrangler.lanl.gov (8.6.12/8.6.12) with SMTP id LAA04647 for ; Tue, 29 Aug 1995 11:08:54 -0600 Date: Tue, 29 Aug 1995 11:08:54 -0600 From: Suresh Damodaran-Kamal Message-Id: <199508291708.LAA04647@wrangler.lanl.gov> To: mpi-core@mcs.anl.gov Subject: Explicit Job Scheduler Interface Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Chapter 5 in MPI-2 (on dynamic process management) does not seem to have an example on the interface with external (to MPI) job scedulers ( load balancers and debuggers). Is is possible to do such interfacing using the current form of MPI-2? (It didn't look like such provisions exist after reading the chapter- correct me if I am wrong). Thanks, -Suresh ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Suresh K. Damodaran-Kamal Ph:(505)-665-9984 (Office) CIC-8, MS K766, Fx:(505)-665-8442 Los Alamos National Laboratory, Email: suresh@lanl.gov Los Alamos, NM 87545 http://www-c8.lanl.gov/dist_comp2/cic8-distcomp2.html ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From mpi-core-human@mcs.anl.gov Fri Sep 1 13:02:46 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA20368; Fri, 1 Sep 1995 13:02:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA27255 for mpi-core-out; Fri, 1 Sep 1995 12:01:41 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA27242 for ; Fri, 1 Sep 1995 12:01:31 -0500 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id MAA29421 for ; Fri, 1 Sep 1995 12:59:15 -0400 Received: from b125.super.org (b125 [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with SMTP id NAA20239 for ; Fri, 1 Sep 1995 13:00:59 -0400 Date: Fri, 1 Sep 1995 13:00:59 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199509011700.NAA20239@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA10400; Fri, 1 Sep 95 13:00:58 EDT To: mpi-core@mcs.anl.gov Subject: document available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The document to be distributed at the next MPI meeting is now available. It can be gotten from ftp://ftp.super.org/pub/mpi/mpi2/Editor/mpi2-report.ps.Z. The sources are also in this directory. The collective chapter has not been updated but a new version should be handed out at the meeting. As usual, Rusty will bring copies of the report to the meeting so you only need to bring it if you want it before the meeting. Steve From mpi-core-human@mcs.anl.gov Mon Sep 4 02:05:16 1995 Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA20472; Mon, 4 Sep 1995 02:05:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA23447 for mpi-core-out; Mon, 4 Sep 1995 01:03:29 -0500 Received: from hitiij.hitachi.co.jp (hitiij.hitachi.co.jp [133.145.224.3]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA23439; Mon, 4 Sep 1995 01:03:08 -0500 Received: from softgw ([158.214.44.2]) by hitiij.hitachi.co.jp (8.6.11+2.4W/3.3W9-hitiij) with SMTP id PAA10668; Mon, 4 Sep 1995 15:02:14 +0900 Received: from [158.214.40.7] by softgw (5.61/1.01) id AA01549; Mon, 4 Sep 95 15:01:04 +0900 Received: by hisoft.soft.hitachi.co.jp (5.61/5.18) id AA13726; Mon, 4 Sep 95 14:58:55 +0900 Date: Mon, 4 Sep 95 14:58:55 +0900 From: haradake@hisoft.soft.hitachi.co.jp (Kei_Harada) Message-Id: <9509040558.AA13726@hisoft.soft.hitachi.co.jp > To: lusk@mcs.anl.gov Subject: No attendee from Hitachi Cc: mpi-core@mcs.anl.gov, ooyamako@hisoft.soft.hitachi.co.jp, sagawa@crl.hitachi.co.jp, sato_y@hisoft.soft.hitachi.co.jp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Mr. Lusk and the Group, Unfortunately, Hitachi can not send a representative to the MPIF meeting at Chicago, Sept. 6-8, due to various reasons. I think Hitachi will send someone to the Denver MPIF meeting in October. Sorry for late notice. Best Regards, Kei Harada Software Development Center, Hitachi, Ltd. Tel : +81-45-862-8875 Fax : +81-45-865-9060 email: haradake@soft.hitachi.co.jp From mpi-core-human@mcs.anl.gov Tue Sep 12 15:28:37 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA09695; Tue, 12 Sep 1995 15:28:36 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA26411 for mpi-core-out; Tue, 12 Sep 1995 14:22:44 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA26403; Tue, 12 Sep 1995 14:22:32 -0500 Message-Id: <199509121922.OAA26403@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Partial Reimbursement for Meeting expenses from NSF Cc: lusk@antares.mcs.anl.gov, rafferty@cs.utk.edu Date: Tue, 12 Sep 1995 14:22:31 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, Jack Dongarra has negotiated a grant from ARPA and NSF to provide partial travel support for attending MPI meetings. The grant is through the University of Tennessee. The constraints are: U.S. academics only. $500 max per person per meeting. The grant started Aug 1, so the July meeting is not covered, but the the recent September meeting is covered. These are the only constraints I know of. Please fill out the forms that I distributed at the last meeting, headed "The University of Tennessee Travel Expense Report". Where it says "Object of trip", say "Message Passing Interface Forum meeting". Fill out the dates and expenses and enclose original receipts. **Please sign the forms.** The grant name and number and other stuff you don't know will be filled in at the U. of T. Send the form and your receipts to: Tracy Rafferty University of Tennessee 104 Ayres Hall Knoxville, TN 37996-1301 If you have detailed questions or problems, please contact the excellent and helpful Tracy Rafferty herself at rafferty@cs.utk.edu. Regards, Rusty From mpi-core-human@mcs.anl.gov Tue Sep 12 16:03:14 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA11430; Tue, 12 Sep 1995 16:03:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA27250 for mpi-core-out; Tue, 12 Sep 1995 14:57:59 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA27048 for ; Tue, 12 Sep 1995 14:51:47 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA03711; Tue, 12 Sep 95 13:51:44 MDT Message-Id: <9509121951.AA03711@nipmuc.fsl.noaa.gov> To: mpi-core@antares.mcs.anl.gov Cc: hart@nipmuc.fsl.noaa.gov, lange@fsl.noaa.gov Subject: Meeting In Denver Date: Tue, 12 Sep 95 13:51:43 BST From: Leslie Hart (NOAA/FSL) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi All, Here are the arrangements for the Denver meeting: Hotel: Executive Tower Inn 1405 Curtis St Denver, CO 80202 1 (800) 525-6651 1 (303) 571-0300 Dates: October 23-25 Rooms are held under MPI Forum, the rate is $68 (+tax I think). The cutoff date for the MPI rate is 9 October (we'll send out another email in early October). Transportation: The registration fee will include a round trip on the Denver Airport Shuttle (DASH), more details about the mechanics will come later. The shuttle runs every 1/2 hour and takes about 1 Hour to get to/from the airport. If you have any questions, you can contact me (hart@fsl.noaa.gov) or Tom Henderson (hender@fsl.noaa.gov). Leslie Hart From mpi-core-human@mcs.anl.gov Fri Sep 15 13:01:43 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA17567; Fri, 15 Sep 1995 13:01:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA01673 for mpi-core-out; Fri, 15 Sep 1995 12:00:56 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA01664 for ; Fri, 15 Sep 1995 12:00:49 -0500 Message-Id: <199509151700.MAA01664@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: [dongarra@cs.utk.edu: Re: mpi travel money] Date: Fri, 15 Sep 1995 12:00:47 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I got a clarification from Jack about using the NSF grant (also contributed to by ARPA) for travel to the Vienna meeting: ------- Forwarded Message From: dongarra@cs.utk.edu Subject: Re: mpi travel money To: lusk@mcs.anl.gov (Rusty Lusk) Date: Fri, 15 Sep 1995 04:56:32 -0400 (EDT) > Is it OK spending this money on travel to the European meeting in January? > It seems to me that this should be fine, but I thought I would check with > you. The meeting will be in Vienna, and people will ask about this at the > next meeting. > No we can't use the money for foreign travel. Arpa has this restriction. Jack ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Wed Sep 20 14:55:00 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA06201; Wed, 20 Sep 1995 14:54:59 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA19182 for mpi-core-out; Wed, 20 Sep 1995 13:52:57 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id NAA18640 for ; Wed, 20 Sep 1995 13:30:56 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA14584; Wed, 20 Sep 95 12:30:48 MDT Message-Id: <9509201830.AA14584@nipmuc.fsl.noaa.gov> To: mpi-core@antares.mcs.anl.gov Subject: Denver Meeting Date: Wed, 20 Sep 95 12:30:47 BST From: Leslie Hart (NOAA/FSL) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi, It was pointed out that some of you may need some sort of registration form to make your travel department happy. You can either generate your own or there is one available from the following web page: http://www-ad.fsl.noaa.gov/MPI.Den.html Notes: 1) Please DO NOT make checks for fees payable to NOAA, make them payable to MPI Forum 2) We don't need the registration forms to make anything happen on this end, they are there if your organization demands a piece of paper. Regards, Leslie Hart (hart@fsl.noaa.gov) From mpi-core-human@mcs.anl.gov Mon Sep 25 16:06:20 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA29849; Mon, 25 Sep 1995 16:06:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA09347 for mpi-core-out; Mon, 25 Sep 1995 15:01:26 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA09335 for ; Mon, 25 Sep 1995 15:01:18 -0500 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id PAA11955 for ; Mon, 25 Sep 1995 15:57:42 -0400 Received: from b125.super.org (b125 [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with SMTP id PAA15836 for ; Mon, 25 Sep 1995 15:59:44 -0400 Date: Mon, 25 Sep 1995 15:59:44 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199509251959.PAA15836@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA28607; Mon, 25 Sep 95 15:59:44 EDT To: mpi-core@mcs.anl.gov Subject: BOF at SC'95 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The MPI BOF at Supercomputing '95 is scheduled for Wednesday, December 6 from 3:30 to 5:00 in Room 11A. Information is available from the SC'95 home page at http://www.sdsc.edu/SC95 (then go through: technical program -> Birds of a Feather -> MPI Forum). More information will come out as we get closer to the meeting. Steve From mpi-core-human@mcs.anl.gov Fri Oct 6 21:49:45 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA08909; Fri, 6 Oct 1995 21:49:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA25957 for mpi-core-out; Fri, 6 Oct 1995 20:49:09 -0500 Received: from firewall.mc.com (firewall.mc.com [192.148.197.15]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id UAA25941 for ; Fri, 6 Oct 1995 20:48:52 -0500 Received: by firewall.mc.com id AA17879 (5.65c/IDA-1.4.4 for ); Fri, 6 Oct 1995 21:48:47 -0400 Received: from jericho.mc.com(192.233.16.4) by firewall via smap (V1.3) id sma017877; Fri Oct 6 21:47:58 1995 Received: from [192.233.16.190] (macip190 [192.233.16.190]) by jericho (8.6.11/8.6.11) with SMTP id VAA09127; Fri, 6 Oct 1995 21:47:51 -0400 X-Sender: clund@192.233.16.4 (Unverified) Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Fri, 6 Oct 1995 21:49:44 -0400 To: mpi-core@antares.mcs.anl.gov From: clund@mc.com (Craig Lund) Subject: MsgWay after MPI Mtg Cc: cohen@myri.com Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There appears to be significant common interest between the MPI community and the MessageWay community. Therefore, we have arranged to have a meeting of the MessageWay working group occur right after the Denver MPI meeting. The MessageWay group will meet in the same hotel from 1 PM until around 7 PM on Wednesday. The MessageWay meeting is open to anyone. There is no need to register. For those who don't know, MessageWay is a working group of the Internet Engineering Task Force (IETF). MessageWay is a proposed low latency protocol for connecting System Area Networks (also known as MPP interconnects; i. e. directly connecting a Paragon mesh to a T3D mesh). Danny Cohen of Myricom is the chairman of the group (cohen@myri.com). MessageWay will be one of several potential transports within heterogenous MPI systems. Craig Lund Email clund@mc.com Mercury Computer Systems Tel 508 256 1300 x264 199 Riverneck Road Fax 508 256 3599 Chelmsford, MA 01824-2820 From mpi-core-human@mcs.anl.gov Tue Oct 10 13:04:27 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA22805; Tue, 10 Oct 1995 13:04:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA25381 for mpi-core-out; Tue, 10 Oct 1995 12:03:24 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA25361 for ; Tue, 10 Oct 1995 12:03:12 -0500 Message-Id: <199510101703.MAA25361@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: make your reservations now Date: Tue, 10 Oct 1995 12:03:10 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Monday October 9 is that last day to get the good rate at the Executive Tower Inn in Denver for the MPI meeting. Rusty From mpi-core-human@mcs.anl.gov Wed Oct 11 05:33:22 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA05886; Wed, 11 Oct 1995 05:33:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA13380 for mpi-core-out; Wed, 11 Oct 1995 04:32:55 -0500 Received: from hitiij.hitachi.co.jp (hitiij.hitachi.co.jp [133.145.224.3]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA13370 for ; Wed, 11 Oct 1995 04:32:40 -0500 Received: from softgw ([158.214.44.2]) by hitiij.hitachi.co.jp (8.6.12+2.4W/3.3W9-hitiij) with SMTP id SAA04218 for ; Wed, 11 Oct 1995 18:30:10 +0900 Received: from [158.214.40.7] by softgw (5.61/1.01) id AB03051; Wed, 11 Oct 95 18:31:29 +0900 Received: by hisoft.soft.hitachi.co.jp (5.61/5.18) id AA11683; Wed, 11 Oct 95 18:29:24 +0900 Date: Wed, 11 Oct 95 18:29:24 +0900 From: haradake@hisoft.soft.hitachi.co.jp (Kei_Harada) Message-Id: <9510110929.AA11683@hisoft.soft.hitachi.co.jp > To: lusk@mcs.anl.gov Subject: Re: make your reservation now Cc: mpi-core%mcs.anl.gov@hitiij.hitachi.co.jp, ooyamako@hisoft.soft.hitachi.co.jp, sagawa@crl.hitachi.co.jp, sato_y@hisoft.soft.hitachi.co.jp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Rusty >Monday October 9 is that last day to get the good rate at the Executive Tower >Inn in Denver for the MPI meeting. > >Rusty I will attend coming Denver meeting, and already made a reservation at the hotel. Unfortunately, I can not attend Message Way meeting since I have to fly to San Jose Wednesday afternoon. Best regards, Kei Harada Hitachi, Ltd. From mpi-core-human@mcs.anl.gov Wed Oct 11 18:29:03 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA14771; Wed, 11 Oct 1995 18:29:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA01623 for mpi-core-out; Wed, 11 Oct 1995 17:27:15 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA01615 for ; Wed, 11 Oct 1995 17:27:03 -0500 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id SAA06593 for ; Wed, 11 Oct 1995 18:24:14 -0400 Received: from b125.super.org (b125 [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with SMTP id SAA08567 for ; Wed, 11 Oct 1995 18:26:31 -0400 Date: Wed, 11 Oct 1995 18:26:31 -0400 From: lederman@super.org (Steve Huss-Lederman) Message-Id: <199510112226.SAA08567@gotham.super.org> Received: by b125.super.org (4.1/SMI-4.1) id AA21089; Wed, 11 Oct 95 18:26:30 EDT To: mpi-core@mcs.anl.gov Subject: Input deadline for next meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy Everyone, I will need the input for the next MPI meeting by COB (Close Of Business) on Tuesday, October 17. Time to start writing! Steve From mpi-core-human@mcs.anl.gov Wed Oct 18 10:00:11 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA03795; Wed, 18 Oct 1995 10:00:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA15419 for mpi-core-out; Wed, 18 Oct 1995 08:58:29 -0500 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA15411 for ; Wed, 18 Oct 1995 08:58:19 -0500 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.6.12/8.6.12.1) with ESMTP id JAA17678 for ; Wed, 18 Oct 1995 09:55:24 -0400 Received: from b125.super.org (b125 [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with ESMTP id JAA28784 for ; Wed, 18 Oct 1995 09:57:47 -0400 From: lederman@super.org (Steve Huss-Lederman) Received: (lederman@localhost) by b125.super.org (8.6.12/8.6.12.client) id JAA29338; Wed, 18 Oct 1995 09:57:46 -0400 Date: Wed, 18 Oct 1995 09:57:46 -0400 Message-Id: <199510181357.JAA29338@b125.super.org> To: mpi-core@mcs.anl.gov Subject: document for meeting available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The MPI-2 document to be handed out at the next meeting is now available in electronic form if you wish to have it now. It can be gotten from: ftp://ftp.super.org/pub/mpi/mpi2/Editor/mpi2-report.ps.Z The sources are also available in this directory. The collective, binding, and real time chapters are not yet complete. They should be available as hardcopy at the meeting and may be sent in electronic format by the authors later. See you next week in Denver. Steve From mpi-core-human@mcs.anl.gov Wed Oct 18 14:38:07 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA06530; Wed, 18 Oct 1995 14:38:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA22784 for mpi-core-out; Wed, 18 Oct 1995 13:35:22 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id NAA22776 for ; Wed, 18 Oct 1995 13:35:10 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA04089; Wed, 18 Oct 95 12:35:07 MDT Message-Id: <9510181835.AA04089@nipmuc.fsl.noaa.gov> To: mpi-core@mcs.anl.gov Cc: hart@nipmuc.fsl.noaa.gov Subject: Meeting in Denver Date: Wed, 18 Oct 95 12:35:06 BST From: Leslie Hart (NOAA/FSL) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi All, I've had a couple of questions regarding transportation. We will be using DASH (Denver Airport SHuttle). When you get to DIA, go to the DASH counter in the ground transportation section of the Airport. Tell them your name and they should be able to give you a ticket to get you to Executive Tower Inn. Also, I need a complete list of names of who is attending (for obvious reasons) so if you are attending please send me (hart@fsl.noaa.gov) email. I have a list from the hotel, but some organizations have several rooms under a single name. Also, the registration fee is $75.00. Please have any checks payable to MPI Forum, NOT NOAA. Thanks, Leslie From mpi-core-human@mcs.anl.gov Fri Oct 20 12:34:32 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA09732; Fri, 20 Oct 1995 12:34:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA04334 for mpi-core-out; Fri, 20 Oct 1995 11:34:03 -0500 Received: from sun1.pallas.de (sun1.pallas.de [194.45.33.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA04318 for ; Fri, 20 Oct 1995 11:33:48 -0500 Received: by sun1.pallas.de (4.1/pallasUK-2.2) id AA26174; Fri, 20 Oct 95 17:32:01 +0100 Date: Fri, 20 Oct 1995 17:32:01 +0100 (MET) From: Hans-Christian Hoppe Subject: Re: Meeting in Denver To: Leslie Hart Cc: mpi-core@mcs.anl.gov, hart@nipmuc.fsl.noaa.gov In-Reply-To: <9510181835.AA04089@nipmuc.fsl.noaa.gov> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Wed, 18 Oct 1995, Leslie Hart wrote: > > Hi All, > > I've had a couple of questions regarding transportation. > We will be using DASH (Denver Airport SHuttle). When you > get to DIA, go to the DASH counter in the ground transportation > section of the Airport. Tell them your name and they should > be able to give you a ticket to get you to Executive Tower > Inn. Also, I need a complete list of names of who is attending > (for obvious reasons) so if you are attending please send me > (hart@fsl.noaa.gov) email. I have a list from the hotel, but some > organizations have several rooms under a single name. > > Also, the registration fee is $75.00. Please have any checks > payable to MPI Forum, NOT NOAA. > > Thanks, > Leslie > Leslie, I'm attending the meeting. Cheers Hans-Christian ------------------------------------------------------------- ---/--- Hans-Christian Hoppe phone : +49-2232-1896-0 / / PALLAS GmbH direct line: +49-2232-1896-11 / / / Hermuelheimer Strasse 10 fax : +49-2232-1896-29 / / / / D-50321 Bruehl email : hch@pallas.de / / / Germany URL : http://www.pallas.de / / PALLAS ------------------------------------------------------------- ---/--- From mpi-core-human@mcs.anl.gov Fri Oct 20 17:06:05 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA12095; Fri, 20 Oct 1995 17:06:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA10538 for mpi-core-out; Fri, 20 Oct 1995 16:05:30 -0500 Received: from mailhost.lanl.gov (mailhost.lanl.gov [128.165.3.12]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA10529 for ; Fri, 20 Oct 1995 16:05:20 -0500 Received: from wrangler.lanl.gov by mailhost.lanl.gov (8.6.12/1.2) id PAA02725; Fri, 20 Oct 1995 15:05:19 -0600 Received: from sutra.lanl.gov (sutra.lanl.gov [128.165.64.18]) by wrangler.lanl.gov (8.6.12/8.6.12) with SMTP id PAA26569 for ; Fri, 20 Oct 1995 15:05:19 -0600 Date: Fri, 20 Oct 1995 15:05:19 -0600 From: Suresh Damodaran-Kamal Message-Id: <199510202105.PAA26569@wrangler.lanl.gov> To: mpi-core@mcs.anl.gov Subject: MPI_SPAWN Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The new dynamic spawn document does away with interfaces to resource managers altogether. However, the document says that "complex interaction of an MPI application with its runtime environment should be done through an environment specific API." The following is a sample scenario: Assume that the runtime environment is PVM. Then, it appears to me that somehow MPI_SPAWN() must result in a call to pvm_spawn(). Otherwise, how will the PVM runtime know about the status of resources and processes running under it? Does this mean there will be different versions of MPI_SPAWN() for different runtimes (operating systems + PVM + ...)? -Suresh From mpi-core-human@mcs.anl.gov Fri Oct 20 17:06:05 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA12094; Fri, 20 Oct 1995 17:06:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA10521 for mpi-core-out; Fri, 20 Oct 1995 16:05:08 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA10506 for ; Fri, 20 Oct 1995 16:04:58 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA09320; Fri, 20 Oct 95 15:04:57 MDT Message-Id: <9510202104.AA09320@nipmuc.fsl.noaa.gov> To: mpi-core@mcs.anl.gov Subject: Weather in Denver 10/22-10/25 Date: Fri, 20 Oct 95 15:04:56 BST From: Leslie Hart (NOAA/FSL) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi All, Here are the local meteorologists best (scientific?) guess at the weather for early next week: Sunday High in the 30s Low in the 20s (Fahrenheit) (0 to -5 C) Chance of 2-4 inches of snow Monday High in the 40s Low in the 20s (Fahrenheit) (5 to -5 C) Clearing Tuesday High in the 50s Low in the 30s (Fahrenheit) (10 to 0 C) Still clear Wednesday (Probably a bit warmer, but that is anybody's guess) Leslie (hart@fsl.noaa.gov) From mpi-core-human@mcs.anl.gov Fri Oct 20 17:35:23 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA12310; Fri, 20 Oct 1995 17:35:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA11275 for mpi-core-out; Fri, 20 Oct 1995 16:34:50 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA11248; Fri, 20 Oct 1995 16:33:37 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id OAA11989; Fri, 20 Oct 1995 14:33:20 -0700 From: "William C. Saphir" Message-Id: <9510201433.ZM11987@win233.nas.nasa.gov> Date: Fri, 20 Oct 1995 14:33:19 -0700 In-Reply-To: Suresh Damodaran-Kamal "MPI_SPAWN" (Oct 20, 3:03pm) References: <199510202103.PAA26535@wrangler.lanl.gov> X-Mailer: Z-Mail (3.1.0 22feb94 MediaMail) To: Suresh Damodaran-Kamal , mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: MPI_SPAWN Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Oct 20, 3:03pm, Suresh Damodaran-Kamal wrote: > Subject: MPI_SPAWN > > The new dynamic spawn document does away with interfaces to > resource managers altogether. However, the document > says that "complex interaction of an MPI application with its runtime > environment should be done through an environment specific API." > > The following is a sample scenario: > > Assume that the runtime environment is PVM. > Then, it appears to me that somehow MPI_SPAWN() must result in a call to > pvm_spawn(). Otherwise, how will the PVM runtime know about > the status of resources and processes running under it? > > Does this mean there will be different versions of MPI_SPAWN() for > different runtimes (operating systems + PVM + ...)? >-- End of excerpt from Suresh Damodaran-Kamal Absolutely. An MPI implementation will have to work with the runtime system at a level below what the user sees. This isn't new in the current proposal, though. I agree it is a barrier to portability. One solution to the portability problem is to specify the interface explicitly. I personally suspect this is impossible to do in a way that works well with *all* (or even most) runtime environments. If it's possible, then what we need is a specific proposal to think about. I haven't written such a proposal because I don't see how to do it, but someone with a clear vision could write one. The solution I'm hoping for is that a public domain implementation (MPICH or LAM or even the PVM folks) will come up with a nice interface that works in a bunch of cases but may not be completely flexible. Vendors can implement their own system and *also* provide an interface compatible with this pseudo-standard interface. Folks needing a portable interface (debuggers, for instance) could write for this portable one. Of course it wouldn't provide full flexibility and wouldn't work everywhere, but I think it would help a lot. I see this also as a solution for interoperability between different MPI implementations. For instance, implementations could standardize on MPICH/P{3,4} socket protocol for heterogeneous communication. However, the MPI Forum would never consider requiring the protocol for all MPI implementations. So I see this as primarily an implementors issue. Bill From mpi-core-human@mcs.anl.gov Fri Oct 20 17:48:27 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA12397; Fri, 20 Oct 1995 17:48:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA11511 for mpi-core-out; Fri, 20 Oct 1995 16:48:28 -0500 Received: from kanga.cse.nd.edu (kanga.cse.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA11474; Fri, 20 Oct 1995 16:47:59 -0500 From: Andrew.Lumsdaine@nd.edu Received: from owl.cse.nd.edu (owl.cse.nd.edu [129.74.25.119]) by kanga.cse.nd.edu (8.6.12/8.6.12) with ESMTP id QAA23507; Fri, 20 Oct 1995 16:47:58 -0500 Received: (lums@localhost) by owl.cse.nd.edu (8.6.12/8.6.12) id QAA01459; Fri, 20 Oct 1995 16:47:58 -0500 Date: Fri, 20 Oct 1995 16:47:58 -0500 Message-Id: <199510202147.QAA01459@owl.cse.nd.edu> To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: bindings chapter Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk We are working furiously on the new MPI bindings chapter. A rough draft of it should be ready by midnight tonight. We will bring copies of the very latest draft with us to distribute at the meeting in Denver. For those of you who want to take it along for reading on the plane, please check your email tomorrow morning. Sorry for the delay . . . . Cheers, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Fri Oct 20 18:55:30 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA12796; Fri, 20 Oct 1995 18:55:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA12570 for mpi-core-out; Fri, 20 Oct 1995 17:55:08 -0500 Received: from mailhost.lanl.gov (mailhost.lanl.gov [128.165.3.12]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA12547; Fri, 20 Oct 1995 17:54:44 -0500 Received: from wrangler.lanl.gov by mailhost.lanl.gov (8.6.12/1.2) id QAA28665; Fri, 20 Oct 1995 16:54:43 -0600 Received: from sutra.lanl.gov (sutra.lanl.gov [128.165.64.18]) by wrangler.lanl.gov (8.6.12/8.6.12) with SMTP id QAA27748; Fri, 20 Oct 1995 16:54:42 -0600 Date: Fri, 20 Oct 1995 16:54:42 -0600 From: Suresh Damodaran-Kamal Message-Id: <199510202254.QAA27748@wrangler.lanl.gov> To: wcs@nas.nasa.gov Subject: Re: MPI_SPAWN Cc: suresh@lanl.gov, mpi-core@mcs.anl.gov, mpi-dynamic@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > > > The new dynamic spawn document does away with interfaces to > > resource managers altogether. However, the document > > says that "complex interaction of an MPI application with its runtime > > environment should be done through an environment specific API." > > > I agree it is a barrier to portability. One solution > to the portability problem is to specify the interface > explicitly. > I personally suspect this is impossible to do in a way that > works well with *all* (or even most) runtime environments. > If it's possible, then what we need is a specific proposal > to think about. OK, how about the following proposal: MPI_SPAWN() has to call a runtime interface function to physically spawn processes. How about standardizing at this level for flexibility sake? In this scenario, MPI_SPAWN() will take two additional arguments which are a function pointer, and a null terminated list containing arguments to this function. This function, lets call it spawn_fn, will return a null terminated list of IDs corresponding to each new process spawned. spawn_fn will also take the following arguments by default, in addition to the arguments passed specifically for spawn_fn, thru the MPI_SPAWN() interface: "executable, arguments,n" spawn_fn will in turn call the resource manager for each environment, and has to be implemented by each environment. If spawn_fn is nonNULL, MPI_SPAWN() will call spawn_fn() to do the physical spawning, and use the returned IDs for book keeping. The use of spawn_fn() frees the resource manager interface from doing the MPI specific chores such as keeping track of groups etc. -Suresh From mpi-core-human@mcs.anl.gov Sat Oct 21 03:05:30 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA15330; Sat, 21 Oct 1995 03:05:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA16432 for mpi-core-out; Sat, 21 Oct 1995 02:05:10 -0500 Received: from kanga.cse.nd.edu (kanga.cse.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA16390; Sat, 21 Oct 1995 02:04:20 -0500 Received: (from jsquyres@localhost) by kanga.cse.nd.edu (8.6.12/8.6.12) id CAA27938; Sat, 21 Oct 1995 02:04:19 -0500 From: Jeff Squyres Message-Id: <199510210704.CAA27938@kanga.cse.nd.edu> Subject: Object Oriented MPI (OOMPI) To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Date: Sat, 21 Oct 1995 02:04:18 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 241707 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I think that one of our goals was to have the longest proposal of this meeting. ;-) Here is a late draft of the proposal from myself, Andy Lumsdaine, and Brian McCandless (another graduate student here at Notre Dame) for OOMPI: Object Oriented MPI. It is a proposal for the C++ bindings as well as a C++ MPI class library (the difference is explained in the document). We will bring copies of the "final" version of this document to the meeting on Monday. %!PS-Adobe-2.0 %%Creator: dvipsk 5.58e Copyright 1986, 1994 Radical Eye Software %%Title: mpi-plus-plus.dvi %%Pages: 44 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o mpi-plus-plus.ps mpi-plus-plus %DVIPSParameters: dpi=300, compressed, comments removed %DVIPSSource: TeX output 1995.10.21:0157 %%BeginProcSet: texc.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]/id ch-image N /rw ch-width 7 add 8 idiv string N /rc 0 N /gp 0 N /cp 0 N{rc 0 ne{rc 1 sub /rc X rw}{G}ifelse}imagemask restore}B /G{{id gp get /gp gp 1 add N dup 18 mod S 18 idiv pl S get exec}loop}B /adv{cp add /cp X}B /chg{rw cp id gp 4 index getinterval putinterval dup gp add /gp X adv}B /nd{/cp 0 N rw exit}B /lsh{rw cp 2 copy get dup 0 eq{pop 1}{ dup 255 eq{pop 254}{dup dup add 255 and S 1 and or}ifelse}ifelse put 1 adv}B /rsh{rw cp 2 copy get dup 0 eq{pop 128}{dup 255 eq{pop 127}{dup 2 idiv S 128 and or}ifelse}ifelse put 1 adv}B /clr{rw cp 2 index string putinterval adv}B /set{rw cp fillstr 0 4 index getinterval putinterval adv}B /fillstr 18 string 0 1 17{2 copy 255 put pop}for N /pl[{adv 1 chg} {adv 1 chg nd}{1 add chg}{1 add chg nd}{adv lsh}{adv lsh nd}{adv rsh}{ adv rsh nd}{1 add adv}{/rc X nd}{1 add set}{1 add clr}{adv 2 chg}{adv 2 chg nd}{pop nd}]dup{bind pop}forall N /D{/cc X dup type /stringtype ne{] }if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (mpi-plus-plus.dvi) @start /Fa 5 115 df<123F1207A2120EA45AA4EA39C0EA3E60EA3830A2EA7038A4EAE0 70A3136013E0EAC0C012C1EA6180EA6300123C0D1D7B9C13>98 D 101 DI111 D114 D E /Fb 2 117 df<121F1206A45AA4EA18F0EA1B18EA1C081218EA38181230A2133012 60133113611362EAC022133C10177E9614>104 D<1203A21206A4EAFFC0EA0C00A35AA4 5A1380A2EA3100A2121E0A147F930D>116 D E /Fc 27 123 df<13E01201EA0380EA07 00120E5AA25AA25AA35AA91270A37EA27EA27E7EEA0380EA01E012000B217A9C16>40 D<12C07E12707E7E7EA27EA2EA0380A3EA01C0A9EA0380A3EA0700A2120EA25A5A5A5A5A 0A217B9C16>II<13E0487EA213B0A2EA03B8A31318EA071CA5 EA0E0EA2EA0FFEA2487EEA1C07A3387F1FC000FF13E0007F13C013197F9816>65 D<3801F180EA07FBEA0FFFEA1F0FEA3C07EA38031270A200F0C7FC5AA77E38700380A212 38383C0700EA1F0FEA0FFE6C5AEA01F011197E9816>67 D73 D<38FC07E0EAFE0FA2383A0B80EA3B1BA513BBEA39B3A413F3EA38E3A2 1303A538FE0FE0A313197F9816>77 D79 DI< EA07E3EA1FFF5AEA781FEAF00712E0A290C7FC7E1278EA3F80EA1FF0EA07FCC67E130F7F EB0380A2126012E0EB0700EAF80FEAFFFE5BEAC7F011197E9816>83 D97 D99 D<133F5B7F1307A4EA03C7EA 0FF748B4FCEA3C1F487EEA700712E0A6EA700FA2EA3C1F381FFFE0380FE7F03807C7E014 197F9816>II<3803E3C03807F7E0EA0FFF381C1CC0 38380E00A56C5AEA0FF8485AEA1BE00038C7FC1218EA1FFC13FF481380387803C038E000 E0A4387001C0EA7C07383FFF80380FFE00EA03F8131C7F9116>103 D105 D<127E12FE127E120EA4EB7FE0A3EB0F00131E5B5B5B120F7F13BC131EEA0E0E7F148038 7F87F0EAFFCFEA7F871419809816>107 D<38F9C38038FFEFC0EBFFE0EA3C78A2EA3870 AA38FE7CF8A2EB3C781512809116>109 DII<38FF0FC0 EB3FE0137F3807F040EBC0005BA290C7FCA8EAFFFCA313127F9116>114 DI<12035AA4EA7FFFB5FCA20007C7FCA75B EB0380A3EB8700EA03FE6C5A6C5A11177F9616>I<387E1F80EAFE3FEA7E1FEA0E03AB13 0F380FFFF03807FBF83803E3F01512809116>I<387F1FC000FF13E0007F13C0381C0700 EA1E0FEA0E0EA36C5AA4EA03B8A3EA01F0A26C5A13127F9116>I<387F1FC038FF9FE038 7F1FC0381C0700120E130EA212075BA2EA039CA21398EA01B8A2EA00F0A35BA3485A1279 127BEA7F806CC7FC123C131B7F9116>121 D<383FFFC05AA238700780EB0F00131EC65A 5B485A485AEA078048C7FC381E01C0123C1278B5FCA312127F9116>I E /Fd 2 63 df60 D<12E01278121EEA0780EA01E0EA0078131C1307EB03C0EB00F014 3C140FEC03C0A2EC0F00143C14F0EB03C049C7FC131C1378EA01E0EA0780001EC8FC1278 12E01A1A7C9723>62 D E /Fe 47 122 df<1478137C13FC12013803C0005B1207A838FF FC78A3EA0780B3152480A31A>12 D<1278A412181230A21260A212E0050A7C830E>44 D<12F0A404047B830E>46 D48 D<13C01201120712FFA212FB1203B3A7B5FCA310227CA118>III<131FA2132FA2136F13EF13CF1201A2EA038FA21207130F120F120E12 1E123CA21278A212F0B512F8A338000F00A915217FA018>I<383FFF80A3003CC7FCA913 F8EA3FFE7FEB0F80EA3E07383C03C01238380001E0A7384003C0A212E038700780387C1F 00EA3FFEEA0FFCEA03F013227EA018>I<137E48B4FC5AEA07C148C7FC121EA25AA25AA2 13F8EAF1FEEAF7FF38FF0F80EAFC0738F803C0A2130100F013E0A51270A21278EB03C012 38003C1380EA1C07381F0F00EA0FFE6C5AEA01F013237EA118>IIII<131FA2497E A2133BEB7BC013731371EBF1E0A213E000017FA213C000031378A2138000077FA2130048 7FA2EBFFFE487FA2381C000F003CEB0780A212380078EB03C0A2127000F0EB01E0A25A1B 237EA220>65 D67 DI<12F0 B3B104237CA20D>73 D<00FC147E6C14FEA300F7EB01DEA3EB800300F3149EA2EBC007A2 00F1141EA2EBE00F00F0130EA2EBF01EA2EB701CEB783CA2EB3838EB3C78A2EB1C70A2EB 1EF0EB0EE0A2130FEB07C0A390C7FC1F237BA22A>77 D79 DI< EAFFFCEBFF8014C038F003E0EB00F014781438143CA51438147814F0EB03E0B512C01480 EBFE00EAF01E7F7F1480130314C0EB01E0A2EB00F0A214781438143C141EA2140F18237B A21F>82 D<13FE3803FFC04813E0EA0F81381E0060481300A25AA5127C123C123FEA1FC0 EA0FFC6CB4FC0001138038003FC0EB07E0EB01F0130014F81478A600C013F07E38F801E0 387E07C0383FFF80000F1300EA01FC15257EA31B>II<00F0133CB3A900781378A36C13F0381E01E0EA1F87380FFFC000031300EA00FC 16247BA221>I<00F01478A2007814F0A36CEB01E0A36CEB03C0A2121F6CEB0780A21380 0007EB0F00A23803C00E141EA23801E01C143CA23800F0381478A2EB787014F0A2EB38E0 133DEB1DC0131FA26D5AA21D237FA220>I97 D<12F0ADEAF1F0EAF7FCB47EEAFC1F487E38F00780A2EB03C0A8EB0780A238F80F00EAFC 3EEAFFFEEAF7F8EAF1F012237CA219>IIII<133F 13FF5AEA03C013801207A8EAFFF8A3EA0780B3102380A20F>I<3801F0783807FFF85A38 1F1F00EA1E0F383C0780A5381E0F00EA1F1FEA0FFE485AEA19F00038C7FCA2123CEA1FFE EBFFC014E04813F0EA7801EB00F8481378A4007813F0383E03E0381FFFC06C13803801FC 0015217F9518>I<12F0A41200A912F0B3A404237DA20B>105 D<13F0A41300A913F0B3A9 1280EAE3E012FFEA7FC0EA1F000C2D83A20D>I<12F0ADEB1F80EB3F00133E5B5BEAF1F0 EAF3E0EAF7C012FFA27FA2EAFDF0EAF8F8EAF078137C7F131E131F7F1480EB07C012237C A218>I<12F0B3B104237DA20B>I<39F0F807C039F3FE1FF039F7FF3FF839FE0F707800FC EBE07C39F807C03CA200F01380AF1E167C9527>IIII114 DI<120FA6EAFFF8A3EA0F00AE1308131CEA07FC13F8EA03E00E1C7F9B12>II<39F007803CEB0FC0A2D8780D1378131D14 E0131CD83C1813F0133814F0001C14E0001E13711330EB7079000E14C01439380F603B00 071480141BEBC01FA2000314001E167F9521>119 D<387801F0007C13E0383E03C0EA1E 07000F138038078F0013DEEA03FC12016C5A137013F8487EEA03DC139EEA078F380F0780 121EEB03C0383C01E0387800F000F813F81516809516>I<38F001E0A2387803C0A2127C 383C0780A2121E14005B120F130EEA071E139EEA039CA3EA019813D8EA00F85BA35BA212 015BA2485A127F90C7FC127E13207F9516>I E /Ff 2 25 df15 D24 D E /Fg 12 122 df97 D<1478EB03F8EB0070A414E0A4EB01C0A213F1EA038938070780EA0E03121C 123C383807001278A3EAF00EA31420EB1C40A2EA703C135C38308C80380F070015207C9F 17>100 D<13C0EA01E0A213C0C7FCA7120E12331223EA4380130012471287120EA35AA3 EA3840138012301270EA31001233121C0B1F7C9E0E>105 D108 D<391C0F80F0392630C318394740640C903880680EEB0070A2008E495A120EA34848485A A3ED70803A3803807100156115E115623970070066D83003133821147C9325>I<381C0F 80382630C0384740601380EB0070A2008E13E0120EA3381C01C0A3EB0384003813881408 1307EB031000701330383001C016147C931A>I<137CEA01C338030180000E13C0121E00 1C13E0123C1278A338F003C0A3EB07801400EA700F130EEA3018EA1870EA07C013147C93 17>I<3801C1E0380262183804741C1378EB701EA2EA08E01200A33801C03CA314383803 8078147014E0EBC1C038072380EB1E0090C7FCA2120EA45AA2EAFFC0171D809317>I 114 D<13FCEA0302EA0601EA0C03130713061300EA0F8013F0EA07F8EA03FCEA003E130E 1270EAF00CA2EAE008EA4010EA2060EA1F8010147D9313>II< 000E13C0003313E0382301C0EA43811301124738870380120EA3381C0700A4130E1218A2 EA1C1EEA0C3CEA07DCEA001CA25B12F05BEAE060485AEA4380003EC7FC131D7C9316> 121 D E /Fh 44 123 df<1278A412181230A21260A212E0050A7D830D>44 D<12F0A404047C830D>46 D<13011303A21306A3130CA31318A31330A31360A313C0A3EA 0180A3EA0300A31206A35AA35AA35AA35AA35AA3102D7DA117>I<131FA2497E133BA2EB 7BC013731371EBF1E013E113E000017FA213C000031378A2138000077FA21300380FFFFE A3381E000FA24814801407A24814C01403A24814E01B207F9F1E>65 DIIIIII<00F01378AEB512F8A338F00078AF15207B9F20>I<12F0B3AE04207C9F 0D>I<00F8EB01F86C1303A200F4147800F61307A300F3130EA3EB801EA200F1131CEBC0 3CA200F01338A2EBE078A2EB6070EB70F0A2EB30E01339A2EB19C0A3EB0F80A390C7FC1D 207B9F28>77 D<00FC13787EA212F612F7A2EAF380A313C012F113E0A212F013F01370A2 13781338133CA2131C131E130EA31307A2130314F8A2130115207B9F20>I<133FEBFFC0 000313F03807E1F8380F807C48487E001E131E487FA248EB0780A348EB03C0A96C130700 781480A36CEB0F00A26C131E001F133E6C6C5A3807E1F86CB45AC613C0013FC7FC1A227D A021>II82 DII<00F013F0B3A7387801E0A2383C03C0A2381F0F80380FFF006C5AEA01F814217B9F 1F>I97 D<12F0ACEAF1F0EAF7FCB47EEAFC3EEA F80F12F0EB0780A7EB0F00A26C5AEAFC3EEAFFFCEAF7F8EAF1E011207D9F17>III I<137EEA01FE1203EA078013005AA7EAFFF0A3EA0F00B10F20809F0E>I<3803E0F0EA0F FF5A383E3E00EA3C1E487EA5EA3C1EEA3E3EEA1FFC485AEA33E00030C7FC1238EA3FFEEB FF806C13C04813E0387803F0EAF000A3EAF801387E07E0383FFFC0000F1300EA03FC141E 7F9317>I<12F0ACEAF1F8EAF3FCEAF7FEEAFC1FEAF80FA212F0AE10207D9F17>I<12F0A4 1200A812F0B3A204207D9F0B>II<12F0B3AE04207D9F0B>108 D<39F0FC07E039F3FE1FF039F7FF3FF839FE 0FF07C39F807C03CA200F01380AE1E147D9325>IIII<3803C780EA0FF7EA1FFFEA3E1FEA7C0FEA7807A212F0A612F812 78EA7C0FEA3E1FEA1FFFEA0FF7EA07C7EA0007A9111D7E9317>III<121EA6 EAFFF0A3EA1E00AD1320EA1FF0120FEA07C00C1A7F9910>II<38F003C0A2007813801307A2383C0F00A3EA1E0E131EA2EA 0F1C133CA2EA0738A213B8EA03F0A26C5A12147F9315>I<387801E0387C03C0383E0780 381E0F00120FEA079EEA03FC5B12016C5A12017F487EEA079EEA0F0F120E381E0780383C 03C0387801E000F813F01414809315>120 D<38F003C0A238780780A2127C383C0F00A2 121E131E120EEA0F1CA2EA073C1338EA03B8A213B0EA01F0A26C5AA2485AA3485A120700 7FC7FC127E127C121D7F9315>II E /Fi 52 123 df11 D<137E3801C180EA0301380703C0120EEB018090C7FCA5 B512C0EA0E01B0387F87F8151D809C17>I<90383F07E03901C09C18380380F0D8070113 3C000E13E00100131892C7FCA5B612FC390E00E01CB03A7FC7FCFF80211D809C23>14 D 34 D<13401380EA0100120212065AA25AA25AA212701260A312E0AC1260A312701230A2 7EA27EA27E12027EEA008013400A2A7D9E10>40 D<7E12407E7E12187EA27EA27EA21380 1201A313C0AC1380A312031300A21206A25AA25A12105A5A5A0A2A7E9E10>I<1306ADB6 12E0A2D80006C7FCAD1B1C7E9720>43 D<126012F0A212701210A41220A212401280040C 7C830C>II<126012F0A2126004047C830C>I<12035A123F12C7 1207B3A4EA0F80EAFFF80D1C7C9B15>49 DI<126012F0A212601200AA126012F0A212600412 7C910C>58 D<1306A3130FA3EB1780A3EB23C0A3EB41E0A3EB80F0A200017FEB0078EBFF F83803007C0002133CA20006133E0004131EA2000C131F121E39FF80FFF01C1D7F9C1F> 65 D<90381F8080EBE0613801801938070007000E13035A14015A00781300A2127000F0 1400A8007014801278A212386CEB0100A26C13026C5B380180083800E030EB1FC0191E7E 9C1E>67 D70 D<39FFF3FFC0390F003C00ACEBFFFCEB003CAD39FF F3FFC01A1C7E9B1F>72 DI77 D79 DI<3807E080EA1C19EA3005EA7003EA600112E01300A36C13007E127CEA7FC0 EA3FF8EA1FFEEA07FFC61380130FEB07C0130313011280A300C01380A238E00300EAD002 EACC0CEA83F8121E7E9C17>83 D<007FB512C038700F010060130000401440A200C01420 1280A300001400B1497E3803FFFC1B1C7F9B1E>I<39FFF07FC0390F000E001404B3A26C 5B138000035B12016C6C5AEB70C0011FC7FC1A1D7E9B1F>I<3AFFE0FFE0FF3A1F001F00 3C001E011E13186C011F1310A3D807801420EC2780A2D803C01440EC43C0A213E0000190 3881E080A33A00F100F100A3017913FA017A137AA2013E137C013C133CA301181318A328 1D7F9B2B>87 D92 D97 D<12FC121CAA137CEA1D86EA1E03 381C018014C0130014E0A614C013011480381E0300EA1906EA10F8131D7F9C17>II<133F1307AAEA03E7EA0C17EA180F487E1270126012E0A61260127012306C5A EA0C373807C7E0131D7E9C17>II<13F8EA018CEA071E 1206EA0E0C1300A6EAFFE0EA0E00B0EA7FE00F1D809C0D>II<12FC12 1CAA137C1387EA1D03001E1380121CAD38FF9FF0141D7F9C17>I<1218123CA21218C7FC A712FC121CB0EAFF80091D7F9C0C>I<13C0EA01E0A2EA00C01300A7EA0FE01200B3A212 60EAF0C012F1EA6180EA3E000B25839C0D>I<12FC121CAAEB3FC0EB0F00130C13085B5B 5B13E0121DEA1E70EA1C781338133C131C7F130F148038FF9FE0131D7F9C16>I<12FC12 1CB3A9EAFF80091D7F9C0C>I<39FC7E07E0391C838838391D019018001EEBE01C001C13 C0AD3AFF8FF8FF8021127F9124>IIIIIII<1204A4120CA2121C123CEAFFE0EA1C00A91310A5 120CEA0E20EA03C00C1A7F9910>I<38FC1F80EA1C03AD1307120CEA0E1B3803E3F01412 7F9117>I<38FF07E0383C0380381C0100A2EA0E02A26C5AA3EA0388A213D8EA01D0A2EA 00E0A3134013127F9116>I<39FF3FCFE0393C0F0380381C07011500130B000E1382A213 11000713C4A213203803A0E8A2EBC06800011370A2EB8030000013201B127F911E>I<38 7F8FF0380F03801400EA0702EA0384EA01C813D8EA00F01370137813F8139CEA010E1202 EA060738040380381E07C038FF0FF81512809116>I<38FF07E0383C0380381C0100A2EA 0E02A26C5AA3EA0388A213D8EA01D0A2EA00E0A31340A25BA212F000F1C7FC12F3126612 3C131A7F9116>II E /Fj 8 118 df<127812FCA412781200A6127812FCA4127806127D910D>58 D68 D99 D<121E123FA4121EC7FCA6B4FCA2121FAEEAFFE0A20B1E7F9D0E>105 D<38FF0FC0EB31E0381F40F0EB80F8A21300AB38FFE7FFA218127F911B>110 DI115 D<38FF07F8A2EA1F00AC1301120F380786FFEA01F818127F911B>117 D E /Fk 7 112 df67 D73 D<38FE0FE0A2003E1380EA3B1BA413BBA2EA39B3A313F3EA38E3A2 1303A638FE0FE0A21317809614>77 D79 DI109 D111 D E /Fl 34 122 df34 D<1380EA010012025A120C120812185A A35AA412E0AA1260A47EA37E1208120C12047E7EEA008009267D9B0F>40 D<7E12407E7E12181208120C7EA37EA41380AA1300A41206A35A1208121812105A5A5A09 267E9B0F>I<126012F0A212701210A31220A212401280040B7D830B>44 DI<126012F0A2126004047D830B>I73 D77 D<137F3801C1C038070070000E7F487F003C131E0038130E0078130F00707F00F01480A8 0078EB0F00A20038130E003C131E001C131C6C5B6C5B3801C1C0D8007FC7FC191A7E991E >79 DI83 D<007FB5FC38701C0700401301A200C0148000801300A300001400B13803FFE019 1A7F991C>I 92 D97 D<12FC121CA913F8EA1F0EEA1E07381C0380130114C0A6EB03 801400EA1E07EA1B0CEA10F0121A7F9915>II<137E130EA9EA03CEEA0C3EEA380E1230 127012E0A612601270EA381EEA1C2E3807CFC0121A7F9915>IIII<12FC121CA913F8EA1D0CEA1E0EA2121CAB38 FF9FC0121A7F9915>I<1218123CA21218C7FCA612FC121CAEEAFF80091A80990A>I<12FC 121CA9EB3F80EB1E00131813105B5BEA1DC0EA1FE0121C1370137813387F131E131F38FF 3FC0121A7F9914>107 D<12FC121CB3A6EAFF80091A80990A>I<38FC7C1F391D8E638039 1E0781C0A2001C1301AB39FF9FE7F81D107F8F20>IIII114 DI<1204A3120CA2121C123C EAFFC0EA1C00A81320A5EA0E40EA03800B177F960F>II<39FF3F9F80393C0E070000381306381C16041317001E130C380E 23081488000F13983807419014D03803C1E01380A200015BEB004019107F8F1C>119 D<38FF1F80383C0600EA1C04A2EA1E0CEA0E08A26C5AA21390EA03A0A2EA01C0A36C5AA2 48C7FCA212E112E212E4127811177F8F14>121 D E /Fm 10 58 df<121FEA3180EA60C0EA4040EAC060A8EA4040EA60C0EA3180EA1F000B107F8F0F>48 D<120C123C12CC120CACEAFF8009107E8F0F>I<121FEA6180EA40C0EA806012C01200A2 13C0EA0180EA030012065AEA10201220EA7FC012FF0B107F8F0F>I<121FEA2180EA60C0 A2120013801201EA0F00EA00801340136012C0A2EA8040EA6080EA1F000B107F8F0F>I< 1203A25A5A120B121312331223124312C3EAFFE0EA0300A4EA1FE00B107F8F0F>III<1240EA7FE013C0EA8080A2EA010012025AA2120C1208A21218 A50B117E900F>I<121FEA3180EA60C0A3EA7180EA3F00120FEA3380EA61C0EAC060A3EA 4040EA6080EA1F000B107F8F0F>I<121FEA3180EA60C0EAC0401360A3EA40E01221EA1E 6012001340EA60C01380EA4300123E0B107F8F0F>I E /Fn 1 50 df<12035AB4FC1207B1EA7FF00C157E9412>49 D E /Fo 81 127 df<127012F8B012701200A5127012F8A31270051C779B18>33 DII37 DI<137013F0EA01E0EA03C0EA0780EA0F00121E121C5AA25AA45AA81270A47EA27E12 1E7EEA0780EA03C0EA01F0120013700C24799F18>40 D<126012F012787E7E7EEA078012 03EA01C0A2EA00E0A41370A813E0A4EA01C0A2EA03801207EA0F00121E5A5A5A12600C24 7C9F18>II<136013F0A7387FFFC0B512E0A26C 13C03800F000A7136013147E9718>I<121C123E127E127F123F121F1207120E121E127C 12F81260080C788518>II<1230127812FCA21278 12300606778518>I<1303EB0780A2130F14005B131EA2133E133C137C1378A213F85B12 015B12035BA212075B120F90C7FCA25A121E123E123CA2127C127812F85AA2126011247D 9F18>II< EA01801203A21207120F123F12FF12FB12431203B0EA7FFCEAFFFEEA7FFC0F1C7B9B18> I II<131F5B1377A213E7120113C7EA038712071307120E121E123C1238127812F0B512 F8A338000700A6EB7FF0EBFFF8EB7FF0151C7F9B18>I<381FFF805AA20038C7FCA8EA3B F8EA3FFE7F383C0780381803C0EA0001EB00E0A2126012F0A238E001C0EA7003387C0F80 383FFF00EA1FFCEA03F0131C7E9B18>I<1230127812FCA2127812301200A81230127812 FCA2127812300614779318>58 D<1218123C127EA2123C12181200A81218123C127EA212 3E121E120E121C123C127812F01260071A789318>I<1303EB0780131FEB3F00137E485A EA03F0485AEA1FC048C7FC127E5AA2127E7EEA1FC0EA07E06C7EEA01FCEA007E7FEB1F80 1307EB030011187D9918>I<387FFFC0B512E0A3C8FCA4B512E0A36C13C0130C7E9318>I< 126012F012FC127E7EEA1FC0EA07E06C7EEA01FCEA007E7FEB1F80A2EB3F00137E485AEA 03F0485AEA1FC048C7FC127E5A12F0126011187D9918>I<137CEA01FEEA07FF380F8780 381E03C0EA3C1DEA387F3870FFE0EA71E313C112E1EAE380A638E1C1C0127113E33870FF 8038387F00EA3C1C381E00E0EA0F833807FFC00001138038007E00131C7E9B18>64 D<137013F8A213D8A2EA01DCA3138CEA038EA41306EA0707A4380FFF80A3EA0E03A2381C 01C0A2387F07F038FF8FF8387F07F0151C7F9B18>II IIII<3801F1C0EA03FDEA 0FFFEA1F0FEA1C03123813011270A290C7FC5AA5EB0FF0131F130F387001C0A213031238 A2EA1C07EA1F0FEA0FFFEA03FDEA01F1141C7E9B18>I<387F07F038FF8FF8387F07F038 1C01C0A9EA1FFFA3EA1C01AA387F07F038FF8FF8387F07F0151C7F9B18>II<387F07F038FF87F8387F07 F0381C03C0EB07801400130E131E5B13385B13F0121DA2EA1FB8A2131C121EEA1C0EA27F A2EB0380A2EB01C0387F03F038FF87F8387F03F0151C7F9B18>75 DI<38FC01F8EAFE 03A2383B06E0A4138EA2EA398CA213DCA3EA38D8A213F81370A21300A638FE03F8A3151C 7F9B18>I<387E07F038FF0FF8387F07F0381D81C0A313C1121CA213E1A313611371A213 311339A31319A2131D130DA3EA7F07EAFF87EA7F03151C7F9B18>IIIII<3803F380EA1F FF5AEA7C0FEA7007EAE003A390C7FC12701278123FEA1FF0EA07FEC67EEB0F80EB03C013 01EB00E0A2126012E0130100F013C038F80780B5FCEBFE00EAE7F8131C7E9B18>I<387F FFF8B5FCA238E07038A400001300B2EA07FFA3151C7F9B18>I<38FF83FEA3381C0070B3 6C13E0EA0F01380783C03803FF806C1300EA007C171C809B18>I<38FF07F8A3381C01C0 A4380E0380A4EA0F0700071300A4EA038EA4EA018C13DCA3EA00D813F8A21370151C7F9B 18>I<38FE03F8A338700070A36C13E0A513F8A2EA39DCA2001913C0A3138CEA1D8DA400 0D13801305EA0F07A2EA0E03151C7F9B18>I<38FF07F8A3381C01C0EA1E03000E1380EA 0F0700071300A2EA038EA2EA01DCA3EA00F8A21370A9EA01FC487E6C5A151C7F9B18>89 D91 D93 D95 D97 D<127E12FE127E120EA5133EEBFF80000F13C0EBC1E01380EB0070120E14 38A6000F1370A2EB80E013C1EBFFC0000E138038063E00151C809B18>IIIII<3801E1F03807FFF8 5A381E1E30381C0E00487EA5EA1C0EEA1E1EEA1FFC5BEA39E00038C7FC7EEA1FFEEBFFC0 4813E0387801F038700070481338A4007813F0EA7E03381FFFC06C13803801FC00151F7F 9318>I<127E12FE127E120EA5133EEBFF80000F13C013C1EB80E01300120EAB387FC3FC 38FFE7FE387FC3FC171C809B18>II<1338137CA313381300A4EA0FFC121F120FEA001CB3A4EA 6038EAF078EAFFF0EA7FE0EA3F800E277E9C18>I<12FEA3120EA5EB3FF0137F133FEB07 80EB0F00131E5B5B5BEA0FF87F139C130EEA0E0F7FEB038014C038FFC7F8A3151C7F9B18 >II<387CE0E038FFFB F8EA7FFF381F1F1CEA1E1EA2EA1C1CAB387F1F1F39FFBFBF80397F1F1F001914819318> IIII<3801E380EA07FBEA1FFFEA3E1FEA380FEA 7007A2EAE003A6EA7007A2EA380FEA3C1FEA1FFFEA0FFBEA03E3EA0003A7EB3FF8A3151E 7E9318>I<387F87E038FF9FF0387FBFF83803F878EBF030EBE0005BA25BA9EA7FFEB5FC 6C5A15147F9318>II<487E1203A4 387FFFC0B5FCA238038000A9144014E0A33801C1C013FF6C1380EB3E0013197F9818>I< 387E07E0EAFE0FEA7E07EA0E00AC1301EA0F033807FFFC6C13FE3801FCFC1714809318> I<387F8FF000FF13F8007F13F0381E03C0000E1380A338070700A3EA038EA4EA01DCA3EA 00F8A2137015147F9318>I<38FF8FF8A3383800E0A3381C01C0A2137113F9A213D9A238 0DDD80A3138DEA0F8FA23807070015147F9318>I<387F8FF0139F138F38070700138EEA 039EEA01DC13F81200137013F07FEA01DCEA039E138EEA0707000F1380387F8FF000FF13 F8007F13F015147F9318>I<387F8FF000FF13F8007F13F0380E01C0EB0380A21207EB07 00A2EA03871386138EEA01CEA2EA00CCA213DC1378A31370A313F05B1279EA7BC0EA7F80 6CC7FC121E151E7F9318>I<383FFFF05AA2387001E0EB03C0EB078038000F00131E5B13 F8485AEA03C0485A380F0070121E5A5AB512F0A314147F9318>II< 126012F0B3B012600424769F18>I<127CB4FC13C01203C67EAB7FEB7FC0EB3FE0A2EB7F C0EBF0005BABEA03C012FF90C7FC127C13247E9F18>II E /Fp 75 125 df<90381F83E09038F06E303901C0787838 0380F8903800F03048EB7000A7B612803907007000B2387FE3FF1D20809F1B>11 D<133FEBE0C0EA01C0380381E0EA0701A290C7FCA6B512E0EA0700B2387FC3FE1720809F 19>II<9038 1F81F89038F04F043901C07C06390380F80FEB00F05A0270C7FCA6B7FC3907007007B23A 7FE3FE3FF02420809F26>I34 D<137813841201EA03021207A45BA25BA26C5A9038A0 0FFC9038C003E090388001C03901C00080EC0100EA03E000041302EA08F0381870043830 7808EA70386D5AEAF01EEB0E206D5A903803C0081270D8780113103938067030391C1838 603907E00F801E227EA023>38 D<127012F812FCA212741204A31208A21210A212201240 060E7C9F0D>I<132013401380EA0100120212065AA25AA25AA312701260A312E0AC1260 A312701230A37EA27EA27E12027EEA0080134013200B2E7DA112>I<7E12407E7E7E120C 7EA27EA2EA0180A313C01200A313E0AC13C0A312011380A3EA0300A21206A25A12085A5A 5A5A0B2E7DA112>I<1306AFB612F0A2D80006C7FCAF1C207D9A23>43 D<127012F812FCA212741204A31208A21210A212201240060E7C840D>II<127012F8A3127005057C840D>I<13011303A21306A3130CA31318A31330A313 60A313C0A3EA0180A3EA0300A31206A35AA35AA35AA35AA35AA3102D7DA117>IIIII<1306A2130EA2131E132EA2134E138EA2EA010E1202A212041208 A212101220A2124012C0B512F038000E00A7EBFFE0141E7F9D17>II<137CEA0182EA0701380E03 80EA0C07121C3838030090C7FC12781270A2EAF1F0EAF21CEAF406EAF807EB0380A200F0 13C0A51270A214801238EB07001218EA0C0E6C5AEA01F0121F7E9D17>I<1240387FFFC0 1480A238400100EA8002A25B485AA25B5BA25BA213C0A212015B1203A41207A76CC7FC12 1F7D9D17>III<127012F8A312701200AA127012F8 A3127005147C930D>I<127012F8A312701200AA127012F012F8A212781208A31210A312 20A21240051D7C930D>I<5B497EA3497EA3EB09E0A3EB10F0A3EB2078A3497EA3497EA2 EBFFFE3801000FA30002EB0780A348EB03C0120E001FEB07E039FFC03FFE1F207F9F22> 65 DI<90380FC04090387030C0 3801C0093803800538070003000E1301001E1300121C123C007C1440A2127800F81400A9 1278007C1440A2123C121C001E1480120E6CEB0100380380026C6C5A38007038EB0FC01A 217D9F21>IIII<90380FE020903878186090 38E004E03803800238070001481300001E1460A25A1520127C127800F81400A7EC7FFCEC 03E000781301127C123CA27EA27E7E380380023900E00460903878182090380FE0001E21 7D9F24>I<39FFF8FFF8390F800F8000071400AC90B5FCEB800FAE000F148039FFF8FFF8 1D1F7E9E22>II<380FFFC03800 7C00133CB3A3122012F8A3EAF038EA40781370EA30E0EA0F8012207E9E17>I76 DI<39FF803FF83907C007C0EC03803905E00100A2EA04F01378A2133CA213 1E130FA2EB0781A2EB03C1EB01E1A2EB00F1A21479143DA2141FA28080A2000E7F121F38 FFE0011D1F7E9E22>II III<3807E080EA0C19EA1007EA3003EA6001A212E01300A36C1300A21278127FEA3F F0EA1FFC6C7EEA03FF38001F801307EB03C0A213011280A400C01380130300E01300EAF0 06EACE0CEA81F812217D9F19>I<007FB512E038780F010060EB006000401420A200C014 3000801410A400001400B3497E3807FFFE1C1F7E9E21>I<39FFFC3FF8390FC007C03907 800380EC0100B3A300031302A2EA01C05C6C6C5AEB7018EB3820EB0FC01D207E9E22>I< 3BFFF07FF81FF03B1F800FC007C03B0F0007800380EE010015C0D807801402A33A03C009 E004A39039E010F00C00011508A29038F0207800005DA2EC403C01785CA2ECC03E90393C 801E40A390391F000F80A3011E1307010E91C7FCA2010C7F010413022C207F9E2F>87 D92 D97 D<120E12FE120EAA133EEBC380380F 01C0EB00E0120E1470A21478A61470A214E0120F380D01C0380CC300EA083E15207F9F19 >IIII<137C13C6EA018F1203EA07061300A7EAFFF0EA0700B2EA7FF01020809F0E>I<14E038 03E330EA0E3CEA1C1C38380E00EA780FA5EA380E6C5AEA1E38EA33E00020C7FCA21230A2 EA3FFE381FFF806C13C0383001E038600070481330A4006013606C13C0381C03803803FC 00141F7F9417>I<120E12FE120EAA133E1343EB8180380F01C0A2120EAE38FFE7FC1620 7F9F19>I<121C121E123E121E121CC7FCA6120E127E120EB1EAFFC00A1F809E0C>I<13E0 EA01F0A3EA00E01300A61370EA07F012001370B3A31260EAF06013C0EA6180EA3F000C28 829E0E>I<120E12FE120EAAEB0FF0EB03C0140013025B5B5B1330137013F8EA0F38EA0E 1C131E130E7F1480130314C014E038FFCFF815207F9F18>I<120E12FE120EB3ABEAFFE0 0B20809F0C>I<390E1F01F039FE618618390E81C81C390F00F00EA2000E13E0AE3AFFE7 FE7FE023147F9326>IIII<3803E080EA0619EA1C05EA3C07EA38031278127012F0A61270127812381307EA1C0B EA0E13EA03E3EA0003A8EB3FF8151D7E9318>III<1202A31206A212 0EA2123EEAFFF8EA0E00AB1308A5EA06101203EA01E00D1C7F9B12>I<380E01C0EAFE1F EA0E01AE13031206EA030D3801F1FC16147F9319>I<38FF83F8381E01E0381C00C06C13 80A338070100A2EA0382A3EA01C4A213ECEA00E8A21370A3132015147F9318>I<39FF9F E1FC393C078070391C030060EC8020000E1440A214C0D807071380130414E03903886100 1471EBC8733801D032143A3800F03CEBE01CA2EB6018EB40081E147F9321>I<387FC3FC 380F01E0000713C0148038038100EA01C2EA00E413EC13781338133C137C134E1387EA01 0738030380380201C0000613E0121E38FF07FE1714809318>I<38FF83F8381E01E0381C 00C06C1380A338070100A2EA0382A3EA01C4A213ECEA00E8A21370A31320A25BA3EAF080 A200F1C7FC1262123C151D7F9318>II< B812F02C01808C2D>124 D E /Fq 37 122 df<1438B2B712FEA3C70038C7FCB227277D A12E>43 D<12F8A505057B8410>46 D<137E3801FF80000713E04813F0381FC3F8130000 3E137CA2003C133C007C133EA20078131EA200F8131FAF0078131E007C133EA46C137C00 3F13FC6C13F813C3380FFFF06C13E00001138038007E0018297EA71D>48 D<13301370EA01F0120F12FFA212F11201B3ACB512E0A313287BA71D>I<13FE3803FF80 000F13C04813E0EB07F0383C01F8387800FC147C127000F0133EA212601220C7FCA3147C A2147814F8EB01F014E01303EB07C0EB0F80EB1F00133E5B5B485A485A5B485A48C7FC12 1E5A387FFFFEA417287EA71D>I<137E3803FF804813E0001F13F0383F83F8EA3E004813 7C123812301210C7FCA214F8A21301EB03F0EB0FE03801FFC0148014C014E0380001F8EB 00FC147C143EA2141FA57E6C133E7E6C137E007C13FC383F01F8381FFFF06C13E0000313 C03800FE0018297EA71D>II< 137E3801FF80000713C04813E0381F83F0EB00F8003E137848137C143C0078133E12F8A2 141E141FA4143F7E127C147F7E383F01FF381FFFDF6C139F6C131F3803FC1E3800403E13 00A2147CA214F8A2EB01F0381803E0381E0FC0383FFF806C13006C5AEA03F818297EA71D >57 D65 DIIIII<12F8B3B3A6052A7BA9 10>73 D78 DI82 D84 D97 D99 D<143EAFEA01FCEA03FF00 0F13BE4813FE13C1383F007E003E133E5AA212FC5AA77E127CA2007E137E6C13FEEA1FC1 13FF6C13BE3803FE3EEA01F8172A7EA91E>I<13FCEA03FF4813804813C0381F87E0EA3F 01383E00F05A1470481378B512F8A400F8C7FCA31278127CA27E003F1318381FC0F8EA0F FF7E000113E038007F00151B7E9A1A>I<90387C07803901FF3FC0000713FF5A903883E0 00381F01F0A2383E00F8A56C485AA2380F83E013FF485B001D90C7FCEA1C7C003CC8FCA2 123E381FFFE014F86C13FE487F481480387E003F007CEB0FC0481307A46C130F007EEB1F 80393F807F00381FFFFE6C5B000313F038007F801A287F9A1D>103 D<12F8A51200AA12F8B3A9052A7CA90E>105 D<12F8B3B3A6052A7CA90E>108 DII<137E3801FF80000713E04813F0381F81F8383F00FC003E137C4813 3EA20078131E00F8131FA7007C133EA36C137C003F13FC381FC3F8380FFFF06C13E06C13 C038007E00181B7E9A1D>IIIIII<00F813 F8B3A213011303EAFC07B5FCEA7FFEEA3FF8EA1FC0151B7C9A1E>I<007CEB1F80007EEB 3F006C133E6C5B380F80FC6C6C5AEBE1F03803E3E0EA01F76CB45A6D5A91C7FC133EA213 3F497E497E3801F3E0EA03E1803807C0F8380F807C121F497E003E7F007E148048EB0FC0 1A1B809A1B>120 D<00F8131F7E007C133EA27E147CA27E14F81380000F13F01381EA07 C114E013C31203EBE3C0120113E71480EA00F714001377A2137E133EA2133CA2137C1378 A25BA21201EA7FE05B5B90C7FC18277F9A1B>I E /Fr 27 122 df<15E01401EC038014 07EC1F00143E5C5C13015C495A1307495A495AA249C7FC5B137E13FEA2485AA212035B12 07A2485AA3121F5BA2123FA35B127FA7485AB06C7EA7123F7FA3121FA27F120FA36C7EA2 12037F1201A26C7EA2137E137F7F6D7EA26D7E6D7E13036D7E801300147C8080EC078014 03EC01E014001B6675CC2D>40 D<12C07E12701278123E7E6C7E6C7E7F12036C7E7F6C7E 137EA27F80131F80A26D7EA280130780A26D7EA3801301A280A37F1580A7EC7FC0B0ECFF 80A715005BA35CA213035CA3495AA25C130F5CA2495AA25C133F91C7FC137EA25B485A5B 485A12075B485A48C8FC123E127812705A5A1A6679CC2D>I43 D58 D66 DI73 D76 DI<923803FFF0037FEBFF800203B612F0020F90 38003FFCDA3FF8EB07FFDAFFE0010113C0010301809038007FF04990C86C7E49486F7E49 486F7E49486F7E49486F7F01FF8448496F7F48854A8148854A167F4885A24890CA6C7EA2 4885A3481A804983A400FF1AC0AE007F1A806D5FA36C1A00A36C6D4C5AA36C6D4C5AA26C 616C6D4B5BA26C6D4B5B6C6D4B5B017F606E5D6D6C4B90C7FCD90FFFED3FFC6D01C0ECFF F86D6D495B010001F8010713C091277FFF807F5B020F90B500FCC8FC020315F0DA007F14 80030301F0C9FC4A497AC757>79 DI<9038 0FFFF090B6FC000315C03A07F8007FF0486CEB1FFCED07FE486C6D7E838183816C48816C 5A6C5AC9FCA5157F023FB5FC0103B6FC011F13F090387FFE003801FFE0481380D80FFEC7 FC485A5B123F485AA2485AA45DA26C6C5BED077F6C6C130F6C6C013E13F83C0FFF80F83F FFE000039038FFF01FC6ECC00F90390FFE0003332E7CAD38>97 DI<91381FFFC049B512FC010714FF903A1FF8007F80D9 7FE0EBFFC0EBFF8048494813E0485A1207485AA248486D13C0003FED7F80EE3F00007F92 C7FC5BA312FFAA127FA27FA2123FA26C6C15F0120F6DEC01E012076C6CEC03C06C6DEB07 806C6DEB0F00D97FF0133E90391FFC01FC0107B512F0010114C09026001FFCC7FC2C2E7C AD34>III<171FDA7FF0EBFFC00107B5000313E0 011FECC7E7903B7FE03FFF0FF09039FF800FFC48EB00074848EB03FE00079238FF07E049 6DEB03C0000FEE8000A2001F82A8000F5EA2000793C7FC6D5B00035D6C6C495A6C6D485A 9138E03FF0D801DFB512C0D803C791C8FC9038C07FF04848CAFCA37FA27FA213F890B612 F06C15FF17E06C8217FC6C826D8148B812801207D80FF0C7001F13C0D81FC014014848EC 007F007FEE3FE048C9FC171FA56C6CED3FC0A26C6CED7F806C6CEDFF00D80FF8EC03FED8 03FEEC0FF82601FFE0EBFFF06C6CB612C0010F4AC7FCD9007F13C034447DAE3A>103 D<137C48B4FC4813804813C0A24813E0A56C13C0A26C13806C1300EA007C90C7FCACEB7F C0B5FCA412037EB3B2B6FCA418497CC820>105 DI108 D<9039FF8007FEB590383FFFC04B13F0913981F81FF8913983C00FFC00039039870007FE 6C138E029C8002B87F188014F0A25CA35CB3A9B60081B6FCA4382E7BAD41>110 D<90397F803F80B5EBFFE0028113F8913883C3FC91388707FE0003138E6C90389C0FFF14 B8A214F0A2ED07FE9138E003FCED01F892C7FCA25CB3A8B612E0A4282E7DAD2F>114 D<90390FFE01C090B512C7000314FF3807F801390FC0007F48C7121F48140F007E1407A2 150312FEA27E7F01E090C7FC13F8387FFFC014FF6C14E015F86C806C14FF6C1580000115 C06C6C14E0131F010014F014039138007FF80070141F00F0140F15077E1503A26C15F0A2 7E6CEC07E07F6DEB0FC001F0EB1F80D8FEFCEBFF0039F87FFFFCD8F01F13F0D8E0031380 252E7CAD2E>III120 DI E /Fs 11 121 df57 D65 D67 D97 D<49B47E010F13F0017F13FC9038FF81FE3A03FE007F80D807F8133F 4848EB1FC0ED0FE0485A003F15F01507485A16F8A212FFA290B6FCA301C0C8FCA4127FA3 6C7E1678121F7F000F15F06C6C13016C6CEB03E06C6CEB0FC03A00FFC07F8090393FFFFE 00010F13F8010013C025267DA52C>101 D<13FE12FFA412071203B0EDFF80020313F002 0F7F91381E03FC91383801FE02607F4A7E01FF15805C91C7FCA35BB3A4B5D8F83F13FEA4 2F3C7CBB36>104 D110 D<9038FE01FF00FF011F13F0027F13FC9138FC07FE9039FFF001FF00079039C0007F806C 90C7EA3FC04915E0EE1FF017F8160F17FCA2160717FEA917FC160FA317F8EE1FF0A26DEC 3FE06EEB7FC06EEBFF806E4813009039FEFC07FE91387FFFF8021F13E0DA03FEC7FC91C9 FCADB512F8A42F377DA536>112 D<3901FC03F000FFEB0FFC4AB4FC91383C3F80EC707F 00079038E0FFC000035BEBFD80A201FFEB7F809138003F00151E92C7FC5BB3A3B512FCA4 22267DA528>114 D<130FA55BA45BA25BA25B5A5A5A001FEBFFF0B6FCA3000190C7FCB3 153CA86C14781480017F13F090383FC1E090381FFFC06D1380903801FE001E377EB626> 116 D120 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 75 361 a Fs(Chapter)31 b(9)75 568 y Fr(Language)39 b(Binding)75 809 y Fq(9.1)59 b(Intro)r(duction)75 910 y Fp(This)14 b(c)o(hapter)f(discusses)i(the)e(C++)h(bindings)h(for)e (MPI)g(and)g(describ)q(es)i(an)f(MPI)f(C++)h(class)f(library)l(.)75 1054 y Fq(9.2)59 b(C++)19 b(Binding)75 1155 y Fp(Prop)q(osal)c(of)g (Andrew)g(Lumsdaine,)i(Je\013)e(Squyres,)g(and)h(Brian)f(McCandless.) 166 1261 y(W)l(e)e(w)o(ould)h(lik)o(e)g(to)e(prop)q(ose)h(that)g(the)g (ANSI)h(C)f(bindings)i(for)d(MPI-1)h(b)q(e)h(the)f(C++)g(bindings)i(as) 75 1318 y(w)o(ell.)22 b(Our)16 b(reasoning)g(for)f(this)h(prop)q(osal)g (is)g(that)f(a)g(language)h(binding)i(should)e(b)q(e)h(the)e(most)g (direct)75 1374 y(and)e(unam)o(biguous)h(statemen)o(t)e(of)g(the)h(MPI) g(function)h(in)g(the)f(target)f(language.)19 b(This)13 b(w)o(ould)h(clearly)75 1431 y(b)q(e)k(the)f(existing)i(C)e(bindings,)i (with)f(p)q(erhaps)g(some)f(c)o(hanges)g(to)f(function)j(calls)f(to)f (pass)g(v)m(ariables)75 1487 y(b)o(y)h(reference)h(instead)f(of)g(b)o (y)g(p)q(oin)o(ter)g(to)f(pro)o(vide)i(a)e(clearer)i(seman)o(tic)f (statemen)o(t.)27 b(F)l(or)17 b(example,)75 1544 y(the)e(C)g(binding)i (declaration)75 1637 y Fo(int)23 b(MPI_Recv\(void*)f(buf,)i(int)f (count,)g(MPI_Datatype)f(datatype,)h(int)h(source,)385 1694 y(int)g(tag,)f(MPI_Comm)g(comm,)g(MPI_Status)f(*status\);)75 1788 y Fp(is)16 b(more)e(clearly)j(expressed)f(as)75 1881 y Fo(int)23 b(MPI_Recv\(void*)f(buf,)i(int)f(count,)g (MPI_Datatype)f(datatype,)h(int)h(source,)385 1938 y(int)g(tag,)f (MPI_Comm)g(comm,)g(MPI_Status&)f(status\);)75 2032 y Fp(This)f(is)g(a)f(minor)h(distinction)h(but)f(it)g(is)g(probably)g (one)g(w)o(orth)e(making)i(should)g(the)g(C)f(bindings)75 2088 y(b)q(ecome)c(the)f(C++)h(bindings.)166 2145 y(There)k(has)f(b)q (een)i(quite)f(a)f(bit)i(of)e(discussion)i(in)f(the)g Fo(mpi-binding)e Fp(mailing)j(list)f(ab)q(out)g(the)75 2201 y(desirabilit)o(y)h(of)d(ha)o(ving)h(an)g(MPI)g(pac)o(k)m(age)g (that)f(uses)h(features)f(of)h(C++.)31 b(W)l(e)19 b(agree)f(that)g (this)h(is)75 2258 y(a)f(w)o(orth)o(y)g(goal.)30 b(W)l(e)19 b(claim,)h(ho)o(w)o(ev)o(er,)e(that)g(a)h(C++)g(binding)i(de\014nition) f(that)e(incorp)q(orates)h(the)75 2314 y(notion)12 b(of)f(ob)s(jects)f (is)i(no)g(longer)f(a)g(functional)i(binding,)h(but)d(rather)g(the)g (de\014nition)j(of)d(a)g(class)g(library)l(.)75 2370 y(W)l(e)17 b(question)h(whether)f(a)g(class)g(library)h(de\014nition)h (is)f(appropriate)f(as)g(a)f(language)i(binding,)h(since)75 2427 y(the)c(MPI-1)g(standard)g(is,)g(for)g(the)g(most)g(part,)f(a)h (functional)h(de\014nition.)166 2483 y(That)f(b)q(eing)h(said,)f(in)h (the)g(attac)o(hed)e(annex)i(w)o(e)f(prop)q(ose)g(one)g(p)q(ossible)i (C++)f(class)g(library)g(for)75 2540 y(MPI)f(|)h(Ob)s(ject)f(Orien)o (ted)h(MPI,)f(or)g(OOMPI.)923 2523 y Fn(1)p 75 2579 720 2 v 127 2606 a Fm(1)144 2622 y Fl(OOMPI)e(is)i(pronounced)g(\\o)q (om-pie")h(although)f(\\o)q(om-p)q(ee")g(is)f(acceptable)h(when)f(part) g(of)g(a)f(longer)i(name)f(\(suc)o(h)75 2668 y(as)f(an)g(OOMPI)g (class\),)h(e.g.,)e Fk(OOMPI)p 622 2668 12 2 v 12 w(Comm)p Fl(.)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 2 2 2 1 bop 166 49 a Fj(Discussion:)33 b Fi(Although)12 b(it)g(ma)o(y)f(b)q (e)i(v)n(aluable)f(to)g(ha)o(v)o(e)h(di\013eren)o(t)g(groups)g(create)i (di\013eren)o(t)e(C++)h(class)75 106 y(libraries)d(for)g(MPI,)h(ha)o (ving)e(a)h(v)o(ersion)h(endorsed)h(b)o(y)e(the)h(MPI)g(F)m(orum)e(is)h (ob)o(viously)g(imp)q(ortan)o(t)e(as)j(w)o(ell.)k(Since)75 162 y(the)f(de\014nition)f(of)g(a)g(class)h(library)f(is)g(not)h(a)f (language)f(binding)h(\(as)h(discussed)h(in)e(the)h(previous)g (section\),)g(w)o(e)75 219 y(prop)q(ose)g(that)f(this)g(class)g (library)f(sp)q(eci\014cation)i(should)e(b)q(ecome)h(an)g(annex)g(to)g (the)g(MPI-1)g(do)q(cumen)o(t)f(\(or)h(to)75 275 y(the)h(MPI-2)f(do)q (cumen)o(t\))g(instead)g(of)g(a)g(c)o(hapter.)20 b(As)14 b(suc)o(h,)h(OOMPI)g(will)e(b)q(e)h(the)h(class)g(de\014nitions)f (appro)o(v)o(ed)75 332 y(b)o(y)g(the)g(MPI)g(F)m(orum,)e(but)i(other)h (groups)f(are)g(free)h(to)e(create)j(their)e(o)o(wn)f(as)h(w)o(ell.)75 557 y Fq(9.3)59 b(F)n(ORTRAN)20 b(90)g(Binding)75 659 y Fp(W)l(ork)15 b(is)g(b)q(eginning.)22 b(Really)l(.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 3 3 2 bop 75 361 a Fs(Annex)32 b(A)75 568 y Fr(C++)39 b(Class)g(Library:) 54 b(Ob)6 b(ject)75 693 y(Orien)m(ted)39 b(MPI)g(\(OOMPI\))75 920 y Fp(Prop)q(osal)15 b(of)g(Andrew)g(Lumsdaine)i(&)f(Je\013)f (Squyres.)75 1063 y Fq(A.1)59 b(Requirements)75 1165 y Fp(With)15 b(the)f(sp)q(eci\014cation)j(of)d(a)g(C++)h(class)f (library)l(,)i(w)o(e)e(will)i(necessarily)g(b)q(e)f(mo)o(ving)f(a)o(w)o (a)o(y)f(from)h(the)75 1221 y(simple)k(one-to-one)e(mapping)h(of)e(MPI) i(function)g(to)e(language)h(binding)j(\(as)c(with)i(C)e(and)i(F)l (ortran\).)75 1278 y(W)l(e)f(therefore)g(also)g(run)g(the)h(risk)f(of)g (adding,)h(losing,)f(or)g(c)o(hanging)h(MPI-1)e(sp)q(eci\014ed)k (functionalit)o(y)75 1334 y(with)d(the)f(library)h(sp)q(eci\014cation.) 166 1391 y(In)22 b(order)e(to)h(prop)q(erly)h(delimit)h(the)e(scop)q(e) g(of)g(the)g(MPI)g(C++)h(class)f(library)l(,)i(w)o(e)e(ha)o(v)o(e)g (the)75 1447 y(follo)o(wing)16 b(guidelines:)75 1567 y Fh(Semantics.)46 b Fp(The)18 b(MPI)g(C++)g(class)h(library)f(m)o(ust) g(pro)o(vide)g(a)g(seman)o(tically)h(correct)e(in)o(terface)i(to)75 1624 y(MPI.)75 1744 y Fh(F)o(unctionalit)o(y)l(.)46 b Fp(The)16 b(MPI)f(C++)h(class)g(library)g(m)o(ust)f(pro)o(vide)h(all)g (functionalit)o(y)h(de\014ned)g(b)o(y)e(MPI-)75 1800 y(1.)25 b(T)l(o)17 b(the)g(greatest)f(exten)o(t)g(p)q(ossible,)j(this)f (functionalit)o(y)g(should)g(b)q(e)f(pro)o(vided)h(through)f(mem)o(b)q (er)75 1857 y(functions)f(of)f(ob)s(jects,)f(although)h(some)g (globally)i(scop)q(ed)f(functions)g(ma)o(y)e(b)q(e)i(p)q(ermitted.)75 1977 y Fh(Objects.)47 b Fp(It)17 b(is)g(only)g(natural)g(to)f(think)h (of)f(comm)o(unicating)h(ob)s(jects)f(in)i(a)e(message-passing)g(C++)75 2033 y(program.)26 b(The)18 b(MPI-1)f(sp)q(eci\014cation,)j(ho)o(w)o (ev)o(er,)d(do)q(es)h(not)f(deal)i(with)f(ob)s(jects.)26 b(It)18 b(only)g(sp)q(eci\014es)75 2090 y(ho)o(w)11 b Fg(data)h Fp(ma)o(y)f(b)q(e)i(comm)o(unicated.)19 b(Th)o(us,)12 b(w)o(e)f(require)i(that)d(the)i(MPI)g(C++)g(class)g(library)g (similarly)75 2146 y(pro)o(vide)k(the)f(capabilit)o(y)i(for)d(sending)j (the)e(data)f(that)h(is)h(con)o(tained)f(within)i(ob)s(jects.)166 2203 y(Moreo)o(v)o(er,)12 b(since)k(the)e(data)f(con)o(tained)i(within) g(an)g(ob)s(ject)e(is)i(essen)o(tially)g(a)f(user-de\014ned)i(struc-)75 2259 y(ture,)h(w)o(e)g(require)h(that)f(mec)o(hanisms)h(b)q(e)g(pro)o (vided)g(to)e(build)k(MPI-1)d(user-de\014ned)i(data)d(t)o(yp)q(es)i (for)75 2315 y(ob)s(ject)12 b(data)f(and)i(for)f(comm)o(unicating)h (that)e(data)h(in)h(a)f(manner)g(iden)o(tical)j(to)c(comm)o(unicating)i (prim-)75 2372 y(itiv)o(e)18 b(data)e(t)o(yp)q(es.)26 b(Ob)s(jects)17 b(that)g(ha)o(v)o(e)f(complex)i(data)f(to)f(b)q(e)i (comm)o(unicated)g(m)o(ust)e(b)q(e)i(explicitly)75 2428 y(constructed)d(to)g(do)g(so.)75 2548 y Fh(Implementation.)45 b Fp(The)16 b(MPI)f(C++)i(class)f(library)g(m)o(ust)f(b)q(e)i(a)e(la)o (y)o(er)g(on)h(top)f(of)g(the)h(C++)g(\(i.e.,)f(C\))75 2605 y(bindings.)28 b(In)19 b(conjunction)f(with)g(the)f(guideline)k (for)16 b(functionalit)o(y)l(,)j(this)f(implies)i(that)d(the)g(MPI-1)75 2661 y(functionalit)o(y)i(will)g(essen)o(tially)g(b)q(e)f(pro)o(vided)h (with)f(calls)g(to)f(C)g(functions.)28 b(That)17 b(is,)h(there)g(will)h (b)q(e)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 4 4 3 bop 75 49 a Fp(no)14 b(attempts)g(for)f(the)i(C++)g(class)f (library)i(itself)f(to)e(pro)o(vide)i(an)o(y)f(MPI-1)h(functionalit)o (y)g(apart)f(from)75 106 y(that)g(pro)o(vided)i(b)o(y)g(the)f(C)g (bindings.)166 162 y(F)l(urther)g(implemen)o(tation)h(stipulations)h (are)e(that:)143 246 y Ff(\017)23 b Fp(The)15 b(class)h(library)g(m)o (ust)e(in)o(tro)q(duce)i(as)f(little)i(o)o(v)o(erhead)e(as)f(p)q (ossible)j(to)e(the)g(C)g(bindings.)143 336 y Ff(\017)23 b Fp(The)16 b(class)g(library)h(ma)o(y)e(not)h(mak)o(e)f(use)h(of)g(in) o(ternal)h(details)g(of)e(particular)i(implemen)o(tations)189 393 y(of)d(C)h(bindings.)143 482 y Ff(\017)23 b Fp(Except)16 b(where)g(the)h(C++)f(language)h(o\013ers)e(a)h(simpler)h(in)o (terface,)g(preserv)o(e)f(similar)h(function)189 539 y(names)e(from)f(the)i(ANSI)g(C)e(MPI)i(bindings)h(as)e(w)o(ell)h(as)f (necessary)g(argumen)o(ts.)75 680 y Fq(A.2)59 b(Analysis)75 783 y Fe(A.2.1)49 b(Syntax)75 869 y Fp(A)15 b(t)o(ypical)h(MPI)f (function)h(call)h(\(in)e(C\))g(is)h(of)f(the)g(follo)o(wing)h(form:)75 953 y Fo(MPI_Comm)23 b(comm;)75 1010 y(int)g(i,)h(dest,)f(tag;)75 1066 y(.)h(.)f(.)75 1179 y(MPI_Send\(&i,)f(1,)i(MPI_INT,)f(dest,)g (tag,)g(comm\);)75 1263 y Fp(Here,)17 b Fo(i)p Fp(,)g Fo(1)p Fp(,)f Fo(MPI)p 379 1263 15 2 v 17 w(INT)p Fp(,)g(and)h Fo(tag)g Fp(sp)q(ecify)h(the)f(con)o(ten)o(t)f(and)h(t)o(yp)q(e)g(of)f (the)h(message)f(to)h(b)q(e)g(sen)o(t,)g(and)75 1319 y Fo(comm)e Fp(and)g Fo(dest)f Fp(sp)q(ecify)j(the)e(destination)h(of)f (the)g(message.)20 b(A)15 b(more)g(natural)g(syn)o(tax)f(results)i (from)75 1376 y(encapsulating)d(the)e(pieces)i(of)e(information)g(that) g(mak)o(e)g(up)h(the)f(message)g(and)h(the)f(destination.)20 b(That)75 1432 y(is,)15 b(w)o(e)f(could)i(p)q(erhaps)f(encapsulate)h Fo(i)p Fp(,)e Fo(1)p Fp(,)h Fo(MPI)p 913 1432 V 17 w(INT)p Fp(,)e(and)i Fo(tag)g Fp(as)f(a)g(message)h(ob)s(ject)f(and)h Fo(comm)f Fp(and)75 1489 y Fo(dest)h Fp(as)f(a)h(destination)h(\(or)f (source\))g(ob)s(ject.)166 1545 y(Before)e(committing)h(to)e(an)o(y)h (ob)s(jects,)g(let's)g(examine)h(the)g(sort)e(of)h(expressiv)o(e)h(syn) o(tax)f(w)o(e)g(w)o(ould)75 1602 y(lik)o(e)j(for)f Fo(OOMPI)p Fp(.)f(The)h(function)h(call)h(ab)q(o)o(v)o(e)e(w)o(ould)g(b)q(e)h(v)o (ery)f(naturally)h(expressed)g(as)75 1686 y Fo(int)23 b(i;)75 1742 y(.)h(.)f(.)75 1855 y(Send\(i\);)75 1939 y Fp(But)14 b(this)h(is)g(incomplete)h(|)e(w)o(e)g(still)i(require)f (some)f(sort)f(of)h(destination)h(ob)s(ject.)k(In)c(fact,)f(w)o(e)g(w)o (ould)75 1996 y(lik)o(e)19 b(an)e(ob)s(ject)h(that)f(can)g(serv)o(e)h (as)f(b)q(oth)h(a)f(source)h(and)g(a)g(destination)g(of)f(a)h(message.) 27 b(In)18 b Fo(OOMPI)p Fp(,)75 2052 y(this)e(ob)s(ject)e(is)i(called)h (a)e Fg(p)n(ort)592 2035 y Fn(1)612 2052 y Fp(.)75 2172 y Fe(A.2.2)49 b(P)o(o)o(rts)15 b(and)h(Communicato)o(rs)75 2258 y Fp(Using)g(an)f Fo(OOMPI)p 389 2258 V 16 w(Port)p Fp(,)g(w)o(e)g(can)g(send)h(and)f(receiv)o(e)h(ob)s(jects)f(with)g (statemen)o(ts)f(lik)o(e:)75 2342 y Fo(int)23 b(i,)h(j;)75 2398 y(OOMPI_Port)e(Port;)75 2455 y(...)75 2567 y(Port.Send\(i\);)75 2624 y(Port.Receive\(j\);)p 75 2661 720 2 v 127 2688 a Fm(1)144 2704 y Fl(The)13 b(monik)o(er)h(\\p)q(ort")g(w)o(as)f (suggested)h(b)o(y)f(Marc)h(Snir.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 5 5 4 bop 75 49 a Fp(The)17 b(OOMPI)p 332 49 14 2 v 17 w(P)o(ort)e(ob)s(ject)h(con)o(tains)h(information)f(ab)q(out)h(its)f (MPI)h(comm)o(unicator)f(and)h(the)f(rank)75 106 y(of)22 b(the)h(pro)q(cess)g(to)f(whom)h(the)g(message)f(is)h(to)g(b)q(e)g(sen) o(t.)42 b(Note,)24 b(ho)o(w)o(ev)o(er,)g(that)e(although)h(the)75 162 y(expression)12 b Fo(Port.Send\(i\))d Fp(is)i(a)g(v)o(ery)f(clear)i (statemen)o(t)d(of)i(what)f(w)o(e)h(w)o(an)o(t)e(to)h(do,)i(there)f(is) g(no)g(explicit)75 219 y(construction)22 b(of)f(a)g(message)h(ob)s (ject.)38 b(Rather,)23 b(the)f(message)f(ob)s(ject)g(is)h(implicitly)j (constructed)75 275 y(\(see)15 b(A.2.3)f(b)q(elo)o(w\).)166 332 y(P)o(ort)20 b(ob)s(jects)i(are)f(v)o(ery)g(closely)i(related)g(to) e(comm)o(unicator)g(ob)s(jects)g(|)h(a)g(p)q(ort)f(is)h(said)h(to)75 388 y(b)q(e)e(comm)o(unicator's)e(view)h(of)f(a)h(pro)q(cess.)34 b(Th)o(us,)21 b(a)e(comm)o(unicator)h(con)o(tains)g(of)f(a)h (collection)h(of)75 444 y(p)q(orts,)13 b(one)g(for)f(eac)o(h)h (participating)h(pro)q(cess.)19 b Fo(OOMPI)12 b Fp(pro)o(vides)h(an)g (abstract)f(base)h(class)g Fo(OOMPI)p 1766 444 15 2 v 17 w(Comm)75 501 y Fp(to)20 b(represen)o(t)h(comm)o(unicators.)37 b(Deriv)o(ed)22 b(classes)g(pro)o(vided)f(b)o(y)g Fo(OOMPI)g Fp(include)i Fo(OOMPI)p 1701 501 V 17 w(Intra)p 1838 501 V 16 w(-)75 557 y(comm)p Fp(,)12 b Fo(OOMPI)p 319 557 V 16 w(Inter)p 455 557 V 17 w(comm)p Fp(,)g Fo(OOMPI)p 713 557 V 16 w(Cart)p 825 557 V 17 w(comm)p Fp(,)f(and)i Fo(OOMPI)p 1168 557 V 16 w(Graph)p 1304 557 V 17 w(comm)p Fp(,)f(corresp)q(onding)h(to)e(in)o(tra)75 614 y(comm)o(unicator,)i(in) o(ter)g(comm)o(unicator,)f(in)o(tra)h(comm)o(unicator)f(with)h (Cartesian)f(top)q(ology)l(,)h(and)g(in)o(tra)75 670 y(comm)o(unicator)i(with)g(graph)g(top)q(ology)l(,)g(resp)q(ectiv)o (ely)l(.)166 727 y(Individual)i(p)q(orts)e(within)h(a)e(comm)o (unicator)g(are)g(accessed)i(with)e Fo(operator[]\(\))p Fp(,)f(i.e.,)i(the)f Fo(i)p Fp(th)75 783 y(p)q(ort)e(of)g(an)g Fo(OOMPI)p 404 783 V 17 w(Comm)23 b(c)12 b Fp(is)h Fo(c[i])p Fp(.)18 b(The)13 b(follo)o(wing)g(co)q(de)g(fragmen)o(t)e(sho)o(ws)g (an)i(example)g(of)f(sending)75 840 y(and)j(receiving:)75 933 y Fo(int)23 b(i,)h(j,)g(m,)f(n;)75 990 y(OOMPI_Intra_comm)f(Comm;) 75 1046 y(...)75 1159 y(Comm[m].Send\(i\);)75 1216 y (Comm[n].Receive\(j\);)75 1310 y Fp(Here,)14 b(the)g(in)o(teger)h Fo(i)f Fp(is)g(sen)o(t)g(to)g(p)q(ort)f Fo(m)h Fp(in)h(the)f(comm)o (unicator)g(and)g(the)h(in)o(teger)f Fo(j)g Fp(is)g(receiv)o(ed)i(from) 75 1366 y(p)q(ort)f Fo(n)p Fp(.)75 1488 y Fe(A.2.3)49 b(Messages)75 1574 y Fp(W)l(e)19 b(de\014ne)i(an)e Fo(OOMPI)p 483 1574 V 16 w(Message)g Fp(ob)s(ject)f(with)i(a)f(set)g(of)g (constructors,)g(one)g(for)g(eac)o(h)g(of)g(the)g(MPI)75 1630 y(base)d(data)e(t)o(yp)q(es.)21 b(Then,)16 b(w)o(e)f(de\014ne)i (all)f(of)f(the)h(comm)o(unication)g(op)q(erations)f(in)i(terms)e(of)g Fo(OOMPI)p 1838 1630 V 16 w(-)75 1686 y(Message)i Fp(ob)s(jects.)30 b(The)18 b(need)i(to)e(construct)g Fo(OOMPI)p 1035 1686 V 16 w(Message)g Fp(ob)s(jects)g(explicitly)j(is)e(ob)o(viated)g(|)75 1743 y(since)c(constructors)d(for)h(eac)o(h)g(of)g(the)h(base)f(data)g (t)o(yp)q(es)h(is)g(declared,)g(an)g Fo(OOMPI)p 1475 1743 V 16 w(Message)e Fp(ob)s(ject)h(will)75 1799 y(b)q(e)20 b(constructed)g(automatically)g(\(and)f(transparen)o(tly\))g(whenev)o (er)h(a)f(comm)o(unication)h(function)g(is)75 1856 y(called)d(with)e (one)h(of)e(the)i(base)f(data)g(t)o(yp)q(es.)166 1995 y Fj(Discussion:)37 b Fi(Message)16 b(ob)r(jects)g(could)f(b)q(e)g (eliminated)e(en)o(tirely)i(b)o(y)g(declaring)f(eac)o(h)h(comm)o (unication)75 2051 y(function)k(in)g(terms)g(of)f(ev)o(ery)i(base)g (data)f(t)o(yp)q(e.)35 b(Ho)o(w)o(ev)o(er,)21 b(this)e(w)o(ould)f (result)i(in)f(an)g(enormous)g(n)o(um)o(b)q(er)75 2108 y(of)g(almost)e(iden)o(tical)i(mem)o(b)q(er)e(functions.)34 b(The)20 b(use)g(of)f(message)g(ob)r(jects)h(seems)f(b)q(etter)i(for)e (the)h(sak)o(e)g(of)75 2164 y(main)o(tainabil)o(it)o(y)l(.)14 b(There)g(is)e(some)f(function)g(o)o(v)o(erhead)i(b)q(ecause)h(of)d (the)i(need)g(to)e(construct)j(a)e(message)g(ob)r(ject,)75 2221 y(but)i(the)h(constructors)h(can)e(b)q(e)g(made)f(v)o(ery)h(ligh)o (t)o(w)o(eigh)o(t)e(so)i(that)g(the)h(o)o(v)o(erhead)f(is)g (negligible.)166 2360 y Fp(The)h(base)h(t)o(yp)q(es)f(supp)q(orted)h(b) o(y)f Fo(OOMPI)f Fp(are:)328 2450 y Fo(char)264 b(short)f(int)169 b(long)328 2506 y(unsigned)22 b(char)50 b(unsigned)22 b(short)49 b(unsigned)g(unsigned)23 b(long)328 2563 y(float)240 b(double)75 2655 y Fp(On)16 b(arc)o(hitectures/compilers)g(that)f(allo) o(w)g(for)g(them,)f(the)i(follo)o(wing)g(t)o(yp)q(es)f(are)g(also)g (supp)q(orted:)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 6 6 5 bop 282 43 a Fo(long)23 b(long)g(int)97 b(unsigned)22 b(long)i(long)49 b(long)23 b(double)282 100 y(Complex)p Fd(<)p Fo(float)p Fd(>)49 b Fo(Complex)p Fd(<)p Fo(double)p Fd(>)97 b Fo(Complex)p Fd(<)p Fo(long)23 b(double)p Fd(>)166 190 y Fp(In)12 b(addition)g(to)e(messages)h(comp)q(osed)g(of)g(single)h (elemen)o(ts)g(of)e(these)i(t)o(yp)q(es,)f(it)g(is)h(also)f(desirable)i (to)75 246 y(send)j(messages)f(comp)q(osed)h(of)f(arra)o(ys)f(of)h (these)h(t)o(yp)q(es.)k(By)c(in)o(tro)q(duing)h(an)e Fo(OOMPI)p 1558 246 15 2 v 16 w(Array)p 1694 246 V 17 w(message)75 302 y Fp(ob)s(ject,)22 b(w)o(e)f(can)g(also)g(pro)o(vide)h (automatic)f(promotion)g(of)g(arra)o(ys.)36 b(Th)o(us,)23 b(to)d(send)i(an)f(arra)o(y)f(of)75 359 y(in)o(tegers,)15 b(w)o(e)g(can)g(use)h(a)f(statemen)o(t)f(lik)o(e:)75 453 y Fo(int)23 b(a[10];)75 509 y(OOMPI_Port)f(Port;)75 566 y(...)75 679 y(Port.Send\(a,)g(10\);)75 772 y Fp(Again,)15 b(no)g(explicit)j(message)c(is)i(constructed.)166 829 y(Note)i(that)g(in)h(the)g(ab)q(o)o(v)o(e)f(examples)h(w)o(e)f(ha)o(v)o (e)g(not)g(explicitly)j(giv)o(en)e(a)g(tag)e(to)h(the)g(messages)75 885 y(that)d(are)h(sen)o(t.)21 b(If)16 b(no)g(tag)f(is)h(giv)o(en,)g(a) g(default)g(tag)f(is)h(assigned)h(b)o(y)f Fo(OOMPI)p Fp(,)e(but)i(a)g(user)g(can)g(supply)75 942 y(a)f(tag)f(as)h(w)o(ell:) 75 1036 y Fo(int)23 b(a[10];)75 1092 y(OOMPI_Port)f(Port;)75 1148 y(...)75 1261 y(Port.Send\(a,)g(10,)i(201\);)75 1355 y Fp(The)15 b(declaration)h(of)f Fo(OOMPI)p 578 1355 V 17 w(Port::Send\(\))e Fp(is)75 1449 y Fo(void)23 b(OOMPI_Port::Send\(OOMPI_Messa)o(ge)e(buf,)i(int)h(tag)f(=)h (OOMPI_NO_TAG\);)75 1505 y(void)f(OOMPI_Port::Send\(OOMPI_Array)o (_message)d(buf,)k(int)f(count)g(=)h(1,)g(int)f(tag)g(=)h (OOMPI_NO_TAG\);)75 1599 y Fp(Here,)c(the)g(default)g(v)m(alue)h(of)e Fo(OOMPI)p 741 1599 V 17 w(NO)p 806 1599 V 16 w(TAG)g Fp(is)i(not)e(the)g(tag)g(used)h(on)f(the)h(message.)32 b(Rather,)21 b(it)75 1656 y(is)f(a)g(dumm)o(y)g(v)m(alue)h(that)e (indicates)j(that)d(no)h(tag)f(w)o(as)g(explicitly)j(giv)o(en,)g(so)d (inside)j(the)e(b)q(o)q(dy)g(of)75 1712 y Fo(OOMPI)p 198 1712 V 16 w(Send\(\))p Fp(,)14 b(a)h(default)h(tag)e(is)i(used,)g (dep)q(ending)h(on)e(the)g(t)o(yp)q(e)h(of)e(the)i(data.)75 1834 y Fe(A.2.4)49 b(User)15 b(De\014ned)h(Data)h(T)l(yp)q(es)75 1920 y Fp(Although)d(it)f(is)h(con)o(v)o(enien)o(t)f(to)g(b)q(e)g(able) h(to)f(pass)f(messages)h(of)g(arra)o(ys)e(or)i(of)g(single)h(elemen)o (ts)g(of)e(basic)75 1976 y(data)17 b(t)o(yp)q(es,)h(signi\014can)o(tly) h(more)f(expressiv)o(e)g(p)q(o)o(w)o(er)g(is)g(a)o(v)m(ailable)h(b)o(y) f(accomo)q(dating)g(user)g(ob)s(jects)75 2033 y(\(i.e.,)12 b(user-de\014ned)i(data)e(t)o(yp)q(es\).)18 b(That)12 b(is,)h Fo(OOMPI)e Fp(should)i(pro)o(vide)g(the)f(abilit)o(y)i(to)d (mak)o(e)h(statemen)o(ts)75 2089 y(of)j(the)g(form)75 2183 y Fo(MyClass)23 b(a[10];)75 2239 y(OOMPI_Port)f(Port;)75 2296 y(...)75 2409 y(Port.Send\(a,)g(10,)i(201\);)166 2503 y Fp(T)l(o)10 b(accomplish)j(this,)e Fo(OOMPI)f Fp(pro)o(vides)i(a)e(base)h(class)g Fo(OOMPI)p 1215 2503 V 16 w(User)p 1327 2503 V 17 w(type)f Fp(from)g(whic)o(h)i(all)g(ob)s (jects)75 2559 y(that)e(will)i(b)q(e)f(comm)o(unicated)g(should)h(b)q (e)f(deriv)o(ed.)19 b(This)11 b(class)g(pro)o(vides)g(an)f(in)o (terface)h(to)f(the)h Fo(OOMPI)p 1838 2559 V 16 w(-)75 2615 y(Message)k Fp(and)h Fo(OOMPI)p 470 2615 V 17 w(Array)p 607 2615 V 16 w(message)g Fp(classes)g(so)g(that)f(ob)s(jects)h(deriv)o (ed)h(from)e Fo(OOMPI)p 1653 2615 V 17 w(User)p 1766 2615 V 17 w(type)75 2672 y Fp(can)g(b)q(e)h(sen)o(t)f(using)h(the)f (syn)o(tax)g(ab)q(o)o(v)o(e.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 7 7 6 bop 166 49 a Fp(Besides)17 b(inheriting)h(from)e Fo(OOMPI)p 768 49 15 2 v 16 w(User)p 880 49 V 17 w(type)p Fp(,)f(the)i(user)f(m)o(ust)f(also)i(construct)e(ob)s(jects)h(in)h(the) 75 106 y(deriv)o(ed)g(class)f(in)h(a)f(particular)g(w)o(a)o(y)f(so)h (that)f(an)h(underlying)i Fo(MPI)p 1260 106 V 17 w(Datatype)c Fp(can)j(b)q(e)f(built.)24 b Fo(OOMPI)75 162 y Fp(pro)o(vides)14 b(a)g(streams)f(based)h(in)o(terface)g(to)f(mak)o(e)g(this)i(pro)q (cess)f(easier.)19 b(The)c(follo)o(wing)f(is)g(an)g(example)75 219 y(of)h(a)g(user)g(de\014ned)i Fo(struct)d Fp(ob)s(ject.)75 312 y Fo(class)23 b(foo)h(:)f(public)g(OOMPI_User_type)f({)75 369 y(public:)123 425 y(foo\(\))h(:)h(OOMPI_User_type\(type,)d(this\))i ({)170 538 y(//)h(Build)f(the)h(data)f(type)g(if)h(it)f(is)h(not)g (built)f(already)170 651 y(if)h(\(!type.Built\(\)\))e({)218 708 y(type.Struct_start\(this\);)218 764 y(type[2])h(<<)h(a;)218 821 y(type[2])f(<<)h(c;)218 877 y(type)f(<<)h(e;)218 933 y(type.Struct_end\(\);)170 990 y(})123 1046 y(};)75 1159 y(private:)123 1272 y(//)f(The)h(data)f(for)h(this)f(class)123 1329 y(int)g(a,)h(b;)123 1385 y(double)f(c,)g(d;)123 1442 y(char)g(e;)123 1554 y(//)g(Static)g(variable)g(to)h(hold)f(the)g (newly)h(constructed)e(MPI_Datatype)123 1667 y(static)h(OOMPI_Datatype) f(type;)75 1724 y(};)166 1818 y Fp(The)15 b(steps)g(in)i(making)e(an)g (ob)s(ject)g(suitable)h(for)f(use)g(in)h Fo(OOMPI)f Fp(as)f(an)i Fo(OOMPI)p 1535 1818 V 16 w(Message)e Fp(are)131 1911 y(1.)22 b(Deriv)o(e)15 b(the)g(class)h(from)e Fo(OOMPI)p 747 1911 V 17 w(User)p 860 1911 V 17 w(type)p Fp(.)131 2005 y(2.)22 b(The)15 b(ob)s(ject)g(m)o(ust)f(con)o(tain)i(a)f(static)g Fo(OOMPI)p 978 2005 V 16 w(Datatype)f Fp(mem)o(b)q(er.)131 2099 y(3.)22 b(The)15 b(constructor)f(for)h(the)g(class)h(m)o(ust)f (initialize)j Fo(OOMPI)p 1197 2099 V 16 w(User)p 1309 2099 V 17 w(type)d Fp(according)g(to)380 2212 y Fo (OOMPI_User_type\(type,)21 b(this\))189 2324 y Fp(where)15 b Fo(type)g Fp(is)g(the)h(name)f(of)g(the)g(static)g Fo(OOMPI)p 1053 2324 V 16 w(Datatype)f Fp(mem)o(b)q(er.)131 2418 y(4.)22 b(Iden)o(tify)16 b(the)f(in)o(ternal)h(data)f(to)f(b)q(e)i (comm)o(unicated.)131 2512 y(5.)22 b(The)c(constructor)g(for)f(the)h (class)h(m)o(ust)f(c)o(hec)o(k)g(if)h(the)f(static)g Fo(OOMPI)p 1409 2512 V 17 w(Datatype)f Fp(mem)o(b)q(er)h(has)189 2568 y(b)q(een)e(built)h(\(b)o(y)e(calling)i(its)e Fo(Built\(\))g Fp(mem)o(b)q(er)g(function\).)21 b(If)16 b(it)f(has)g(not)g(b)q(een)i (built,)f(appro-)189 2625 y(priate)f(calls)h(to)f(the)g(datat)o(yp)q(e) g(m)o(ust)f(b)q(e)i(made)g(so)e(that)h(it)g(can)h(b)q(e)f(built.)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 8 8 7 bop 75 49 a Fe(A.2.5)49 b(Return)15 b(V)o(alues)75 135 y Fp(All)k(functions)f(in)g(MPI-1)g(return)f(an)g(error)g (condition;)j(a)d(pro)o(vision)h(for)f(installing)i(error)e(handlers)75 192 y(allo)o(ws)g(for)e(errors)h(to)g(b)q(e)h(trapp)q(ed)g(in)g(v)m (arious)g(sp)q(eci\014ed)i(w)o(a)o(ys.)j(Again,)17 b(C++)g(allo)o(ws)g Fo(OOMPI)e Fp(to)h(b)q(e)75 248 y(more)f(expressiv)o(e.)23 b(Rather)15 b(than)h(returning)g(error)f(co)q(des,)h Fo(OOMPI)f Fp(functions)i(ha)o(v)o(e)e(sp)q(eci\014ed)j(return)75 304 y(v)m(alues)k(\(t)o(ypically)g(corresp)q(onding)g(to)f(MPI-1)g (\\out")f(parameters\).)36 b(Error)21 b(conditions)h(are)f(then)75 361 y(handled)c(using)f(exceptions.)21 b(In)16 b(addition,)g(a)f (global)h(v)m(ariable)h Fo(OOMPI)p 1320 361 15 2 v 16 w(errno)e Fp(is)g(set)g(to)g(indicate)i(the)75 417 y(cause)e(of)g(the)h (error.)75 539 y Fe(A.2.6)49 b(A)16 b(Stream)g(Interface)f(fo)o(r)g (Message)h(P)o(assing)75 625 y Fp(Streams)d(are)h(a)g(standard)f(mec)o (hanism)i(used)g(in)f(C++)h(for)e(p)q(erforming)i(I/O.)f(The)g(syn)o (tax)f(of)h(stream)75 681 y(based)j(I/O)f(is)h(appropriate)g(for)e (message)h(passing.)24 b(That)16 b(is,)h(messages)f(can)g(b)q(e)h(sen)o (t)f(and)h(receiv)o(ed)75 738 y(in)f Fo(OOMPI)e Fp(with)i(the)f (statemen)o(t:)75 832 y Fo(int)23 b(i,)h(j;)75 888 y(OOMPI_Port)e (Port;)75 945 y(...)75 1057 y(Port)h(<<)h(i;)75 1114 y(Port)f(>>)h(j;)75 1208 y Fp(Since)13 b(a)e(second)h(argumen)o(t)f(to) g Fo(operator>>)f Fp(and)i Fo(operator<<)e Fp(is)i(not)g(p)q(ossible,)h (default)f(tags)f(\(based)75 1264 y(up)q(on)16 b(the)f(message)g(t)o (yp)q(e\))g(are)g(used.)75 1386 y Fe(A.2.7)49 b(P)o(ack)o(ed)15 b(Data)75 1472 y Fp(MPI-1)h(pro)o(vides)h(the)f(capabilit)o(y)h(for)f (users)g(to)f(pac)o(k)h(their)h(o)o(wn)f(messages.)22 b(A)16 b(similar)h(mec)o(hanism)75 1528 y(is)e(pro)o(vided)g(in)g Fo(OOMPI)p Fp(,)e(along)h(with)h(a)f(stream)f(in)o(terface.)20 b(In)15 b(the)f(follo)o(wing)h(example,)g(a)f(message)f(of)75 1585 y(200)h(in)o(tegers)i(is)f(constructed:)123 1678 y Fo(OOMPI_Packed)22 b(msg\(200)h(*)h(sizeof\(int\),)e (OOMPI_COMM_WORLD\);)123 1735 y(msg.Start\(\);)123 1791 y(for)h(\(int)g(i)h(=)g(0;)g(i)f(<)h(100;)f(i++\))170 1848 y(msg)h(<<)f(i)h(<<)g(rank;)123 1904 y(msg.End\(\);)75 2026 y Fe(A.2.8)49 b(Objects)75 2112 y Fp(Listed)18 b(b)q(elo)o(w)f (are)g(all)h(the)f(ob)s(jects)f(that)g(are)h(men)o(tioned)h(brie\015y)g (ab)q(o)o(v)o(e.)24 b(Eac)o(h)17 b(ob)s(ject)f(con)o(tains)h(a)75 2168 y(brief)f(description)h(and)e(list)h(of)f(functional)h(requiremen) o(ts.)166 2225 y(Eac)o(h)e(ob)s(ject)h(is)g(pre\014xed)h(with)f Fo(OOMPI)p 860 2225 V 31 w Fp(so)f(that)g(no)h(name)g(clashes)h(will)g (o)q(ccur)f(with)g(the)g(ANSI)75 2281 y(C)h(bindings)h(of)e(functions,) i(datat)o(yp)q(es,)d(and)i(constan)o(ts.)21 b(All)c(OOMPI)f(names)g (\(mem)o(b)q(er)g(functions,)75 2338 y(ob)s(jects,)j(and)f(constan)o (ts\))g(follo)o(w)h(the)g(same)f(capitalization)i(sc)o(heme)f(as)g (MPI-1)f(names.)31 b(In)19 b(addi-)75 2394 y(tion)h(to)e(the)i Fo(MPI)p 390 2394 V 36 w Fp(pre\014x,)h(man)o(y)e(MPI-1)g(functions)h (also)f(con)o(tained)h(a)g(second)f(pre\014x)h(to)f(classify)75 2450 y(functionalit)o(y)f(\(e.g.)d Fo(MPI)p 520 2450 V 16 w(Type)p 632 2450 V 17 w(*)p Fp(\).)23 b(In)17 b(suc)o(h)f(cases,) g(the)h(second)f(pre\014x)h(w)o(as)f(made)g(part)g(of)f(the)i(ob-)75 2507 y(ject)f(name)g(and)g(the)g(mem)o(b)q(er)g(functions)g(w)o(ere)g (named)g(from)f(the)h(remaining)h(su\016x.)22 b(F)l(or)15 b(example,)75 2563 y Fo(MPI)p 150 2563 V 17 w(Type)p 263 2563 V 16 w(Vector\(\))e Fp(b)q(ecame)h(the)g Fo(Vector\(\))e Fp(mem)o(b)q(er)i(function)g(of)f(the)h Fo(OOMPI)p 1522 2563 V 16 w(Datatype)f Fp(ob)s(ject.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 9 9 8 bop 75 49 a Fe(A.2.9)49 b(Communicato)o(r)16 b(Objects)75 135 y Fp(The)f(ob)s(jects)g(asso)q(ciated)g(with)h(comm)o(unicators)f (are:)429 273 y Fo(OOMPI)p 552 273 15 2 v 16 w(Comm)186 b(OOMPI)p 970 273 V 17 w(Comm)p 1083 273 V 16 w(world)49 b(OOMPI)p 1388 273 V 17 w(Group)429 329 y(OOMPI)p 552 329 V 16 w(Intra)p 688 329 V 17 w(comm)g(OOMPI)p 970 329 V 17 w(Cart)p 1083 329 V 16 w(comm)73 b(OOMPI)p 1388 329 V 17 w(Port)429 385 y(OOMPI)p 552 385 V 16 w(Inter)p 688 385 V 17 w(comm)49 b(OOMPI)p 970 385 V 17 w(Graph)p 1107 385 V 16 w(comm)166 475 y Fp(These)18 b(ob)s(jects)e(encapsulate)i (the)g(functionalit)o(y)g(of)f(MPI)g(comm)o(unicators)g(and)g(are)g (the)h(basis)75 532 y(for)12 b(all)h(comm)o(unication)g(\(p)q(oin)o (t-to-p)q(oin)o(t)f(and)h(collectiv)o(e\).)20 b(The)13 b(comm)o(unictor)f(ob)s(jects)f(con)o(tain)i(the)75 588 y(algebraic)j(group)e(ob)s(ject)g(used)h(to)f(create)h(the)f(comm)o (unicator,)g(a)h(p)q(ort)f(ob)s(ject)g(for)g(eac)o(h)h(rank)f(in)i(the) 75 645 y(comm)o(unicator,)e(and)i(an)f(error)f(handler)j(\(if)e(there)g (is)h(one\).)166 701 y(The)e Fo(OOMPI)p 381 701 V 16 w(Comm)f Fp(ob)s(ject)g(is)h(an)f(abstract)g(base)h(class)f(from)g (whic)o(h)i(the)e(classes)h Fo(OOMPI)p 1701 701 V 17 w(Intra)p 1838 701 V 16 w(-)75 758 y(comm)p Fp(,)h Fo(OOMPI)p 322 758 V 17 w(Inter)p 459 758 V 16 w(comm)p Fp(,)h(and)g Fo(OOMPI)p 809 758 V 16 w(Comm)p 921 758 V 17 w(world)f Fp(are)h(deriv)o(ed.)24 b(These)16 b(classes)h(represen)o(t)f(and)75 814 y(pro)o(vide)j(the)h(functionalit)o(y)g(asso)q(ciated)f(with)g(in)o (tra-comm)o(unicators,)g(in)o(ter-comm)o(unicators,)h(and)75 871 y Fo(MPI)p 150 871 V 17 w(Comm)p 263 871 V 16 w(world)p Fp(,)13 b(resp)q(ectiv)o(ely)l(.)22 b(Note)13 b(that)g(the)h(class)g Fo(OOMPI)p 1194 871 V 17 w(Comm)p 1307 871 V 16 w(world)f Fp(has)h(only)g(one)g(instance)75 927 y(of)h(an)g(ob)s(ject,)f(the)h (global)h(v)m(ariable)h Fo(OOMPI)p 846 927 V 16 w(Comm)p 958 927 V 17 w(world)p Fp(.)166 1066 y Fj(Discussion:)e Fi(F)m(or)f(MPI-2,)f(it)h(migh)o(t)d(b)q(e)k(desirable)f(to)g(create)h (a)f(one-w)o(a)o(y)f(comm)o(unicator)e(class)j(that)g(is)75 1123 y(also)f(deriv)o(ed)i(from)d(the)i Fc(OOMPI)p 586 1123 14 2 v 15 w(Comm)f Fi(ob)r(ject.)166 1262 y Fp(The)h Fo(OOMPI)p 381 1262 15 2 v 16 w(Group)g Fp(ob)s(ject)f(will)i (encapsulate)g(all)g(the)f(op)q(eration)g(on)g(groups.)19 b(A)14 b(group)f(in)i(MPI)75 1318 y(is)j(an)g(ordered)g(set)g(of)g(pro) q(cess)g(iden)o(ti\014ers)h(and)f(is)h(represen)o(ted)f(b)o(y)g(a)g (group)g(ob)s(ject.)27 b(In)19 b(OOMPI,)75 1375 y(the)c Fo(OOMPI)p 276 1375 V 17 w(Group)f Fp(is)i(used)g(b)o(y)f(the)g Fo(OOMPI)p 838 1375 V 17 w(Communicator)e Fp(ob)s(ject.)166 1431 y(An)20 b Fo(OOMPI)p 368 1431 V 17 w(Port)f Fp(ob)s(ject)g(is)i (created)e(for)h(eac)o(h)g(rank)f(in)i(a)f(comm)o(unicator.)33 b(It)20 b(encapsulates)75 1488 y(all)i(the)f(p)q(oin)o(t-to-p)q(oin)o (t)h(and)f(ro)q(oted)g(collectiv)o(e)i(comm)o(unication)f(functionalit) o(y)l(.)39 b(P)o(oin)o(t-to-p)q(oin)o(t)75 1544 y(comm)o(unication)22 b(routines)f(\(e.g.)f Fo(Send\(\))g Fp(and)h Fo(Recv\(\))p Fp(\))f(in)o(v)o(ok)o(ed)h(on)g(an)g Fo(OOMPI)p 1557 1544 V 16 w(Port)g Fp(implicitly)75 1601 y(sp)q(ecify)15 b(the)f(destination)g(\(or)f(source\))h(rank.)19 b(Ro)q(oted)14 b(collectiv)o(e)h(comm)o(unication)g(routines)f(in)o(v)o(ok)o(ed)75 1657 y(on)h(an)g Fo(OOMPI)p 324 1657 V 17 w(Port)f Fp(implicitl)q(y)k (sp)q(ecify)e(the)f(ro)q(ot)g(of)g(the)g(op)q(eration.)75 1779 y Fe(A.2.10)49 b(Message)16 b(Objects)75 1865 y Fp(The)f Fo(OOMPI)g Fp(ob)s(jects)f(asso)q(ciated)i(with)f(messages)g (are:)381 1954 y Fo(OOMPI)p 504 1954 V 17 w(Message)185 b(OOMPI)p 994 1954 V 16 w(User)p 1106 1954 V 17 w(type)49 b(OOMPI)p 1388 1954 V 17 w(Request)381 2011 y(OOMPI)p 504 2011 V 17 w(Array)p 641 2011 V 16 w(Message)g(OOMPI)p 994 2011 V 16 w(Packed)114 b(OOMPI)p 1388 2011 V 17 w(Status)381 2067 y(OOMPI)p 504 2067 V 17 w(Datatype)161 b(OOMPI)p 994 2067 V 16 w(Op)166 2160 y Fp(The)12 b Fo(OOMPI)p 379 2160 V 17 w(Message)f Fp(ob)s(ject)g(is)i(a)f(base)g(class)h(that)e (is)i(used)f(to)g(unify)h(all)g(non-arra)o(y)e(t)o(yp)q(es)h(\(base)75 2216 y(t)o(yp)q(es)20 b(and)g(user-de\014ned)i(t)o(yp)q(es\))d(in)o(to) h(one)g(ob)s(ject)g(t)o(yp)q(e.)34 b(That)19 b(is,)i(ev)o(ery)f(MPI-1)g (function)h(that)75 2273 y(includes)15 b(a)e Fo(\(void)23 b(*\))13 b Fp(data)f(bu\013er)h(argumen)o(t)f(is)i(replaced)g(with)f(a) g Fo(\(OOMPI)p 1422 2273 V 16 w(Message)23 b(&\))13 b Fp(argumen)o(t)75 2329 y(\(or)i Fo(OOMPI)p 272 2329 V 16 w(Array)p 408 2329 V 17 w(message)f Fp(argumen)o(t,)h(see)h(b)q(elo) o(w\).)22 b(Since)17 b(the)e Fo(OOMPI)p 1381 2329 V 17 w(Message)g Fp(ob)s(ject)g(includes)75 2386 y(the)f(MPI)g(datat)o(yp)q (e)g(and)g(a)g(p)q(oin)o(ter)g(to)f(the)i(top)e(of)h(the)g(data,)f (functions)i(who)f(ha)o(v)o(e)f Fo(OOMPI)p 1694 2386 V 17 w(Message)75 2442 y Fp(argumen)o(ts)f(inheren)o(tly)i(kno)o(w)e (the)h(data's)f(t)o(yp)q(e)h(and)g(where)g(it)g(resides)g(in)h(memory)l (.)k Fo(OOMPI)p 1694 2442 V 17 w(Message)75 2499 y Fp(is)e Fg(only)e Fp(used)i(as)f(an)g(in)o(ternal)h(ob)s(ject;)e(it)i(is)g(not) e(considered)j(to)d(b)q(e)i(part)f(of)g(the)g(user)g(in)o(terface.)166 2638 y Fj(Discussion:)j Fi(The)e(MPI-1)f(C)h(bindings)e(of)h(MPI-1)g (sp)q(ecify)h(that)g(all)e(data)h(bu\013ers)i(are)e(of)g(t)o(yp)q(e)h Fc(\(void)75 2694 y(*\))p Fi(.)h(Since)c(the)g(t)o(yp)q(e)g(of)f(the)h (data)g(is)f(not)h(inheren)o(t)g(in)f(the)h(argumen)o(t,)e(a)i(second)g (argumen)o(t)f(m)o(ust)f(b)q(e)i(sp)q(eci\014ed)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 10 10 9 bop 75 49 a Fi(that)17 b(pro)o(vides)g(the)g(t)o(yp)q(e.)26 b(In)17 b(C++,)g(functions)g(can)g(b)q(e)g(o)o(v)o(erloaded)f(to)h (inheren)o(tly)g(kno)o(w)f(the)h(t)o(yp)q(e)g(of)f(the)75 106 y(argumen)o(t,)11 b(but)i(there)h(are)f(t)o(w)o(o)f(problems)g (with)g(this)h(approac)o(h:)k(it)c(leads)f(to)h(a)f(function)h (explosion)f(and)g(user-)75 162 y(de\014ned)h(t)o(yp)q(es)g(are)g(not)f (included)g(in)g(this)g(sc)o(heme.)17 b(Using)12 b Fc(OOMPI)p 1139 162 14 2 v 15 w(Message)f Fi(as)h(a)g(base)g(class)h(with)f(ligh)o (t)o(w)o(eigh)o(t)75 219 y(default)19 b(promotions)f(for)i(all)f(base)h (t)o(yp)q(es)h(pro)o(vides)f(a)f(clean,)i(e\016cien)o(t,)g(and)f (useful)g(w)o(a)o(y)f(to)h(not)f(ha)o(v)o(e)h(to)75 275 y(o)o(v)o(erload)13 b(functions)h(for)g(eac)o(h)g(t)o(yp)q(e.)166 414 y Fp(The)22 b Fo(OOMPI)p 389 414 15 2 v 16 w(Array)p 525 414 V 17 w(message)e Fp(ob)s(ject)h(is)h(v)o(ery)f(similar)h(to)f (the)g Fo(OOMPI)p 1453 414 V 17 w(Message)f Fp(ob)s(ject)h(\(see)75 471 y(b)q(elo)o(w\))c(except)h(that)f(it)g(promotes)f(arra)o(ys.)25 b(Ho)o(w)o(ev)o(er,)16 b(it)i(do)q(es)f Fg(not)g Fp(tak)o(e)g(an)g (argumen)o(t)f(indicating)75 527 y(ho)o(w)h(man)o(y)f(elemen)o(ts)i (exist)f(in)h(the)g(arra)o(y;)e Fo(OOMPI)p 987 527 V 16 w(Array)p 1123 527 V 17 w(message)g Fp(is)i(only)f(used)h(as)f(a)f Fg(pr)n(omotion)75 584 y Fp(mec)o(hanism,)f(and)h(can)f(therefore)g (only)h(tak)o(e)e(one)i(argumen)o(t.)166 723 y Fj(Discussion:)g Fi(One)f(of)e(the)i(main)d(reasons)j(for)f(splitting)f(arra)o(ys)i(in)o (to)e(their)i(o)o(wn)e(promotion)f(class)j(is)f(to)75 779 y(a)o(v)o(oid)d(an)h(am)o(biguit)o(y)d(where)k(coun)o(t)g(argumen)o (ts)e(are)i(required.)18 b(Since)13 b(it)f(is)g(often)g(desirable)h(to) f(send)h(messages)75 836 y(con)o(taining)j(di\013eren)o(t)h(p)q (ortions)g(of)f(an)g(arra)o(y)m(,)h(it)f(is)h(not)f(practical)h(to)f (wrap)h(the)g Fc(count)f Fi(argumen)o(t)f(in)o(to)h(the)75 892 y Fc(OOMPI)p 188 892 14 2 v 15 w(Array)p 313 892 V 14 w(message)j Fi(ob)r(ject,)j(b)q(ecause)f(this)f(w)o(ould)g(imply)d (that)j(a)g(new)g(datat)o(yp)q(e)h(w)o(ould)e(ha)o(v)o(e)g(to)h(b)q(e) 75 949 y(created)15 b(for)e(eac)o(h)g(di\013eren)o(t)i(size)f(arra)o(y) f(that)g(will)f(b)q(e)h(sen)o(t.)19 b(A)13 b(b)q(etter)i(solution)e(is) g(to)g(use)h(a)f(second)h(argumen)o(t)75 1005 y(to)d(explicitly)g (state)h(the)g Fc(count)e Fi(argumen)o(t)g(for)h(functions)h(suc)o(h)g (as)f Fc(MPI)p 1218 1005 V 16 w(Send\(\))p Fi(.)k(The)d(OOMPI)h(equiv)n (alen)o(ts)e(of)75 1061 y(the)i Fc(MPI)p 214 1061 V 16 w(Send\(\))e Fi(function)i(are)g(declared)h(b)q(elo)o(w.)j(In)c(the)h (second)g(function,)e(it)g(is)h(necessary)i(to)e(ha)o(v)o(e)f(a)h Fc(count)75 1118 y Fi(argumen)o(t)i(to)i(sp)q(ecify)g(ho)o(w)f(man)o(y) f(elemen)o(ts)h(are)h(in)f(the)h(arra)o(y)m(.)25 b(A)o(t)17 b(\014rst)g(glance,)g(it)f(w)o(ould)f(seem)i(that)f(the)75 1174 y Fc(count)d Fi(argumen)o(t)g(could)i(ha)o(v)o(e)f(a)g(default)g (v)n(alue)g(of)f(1)h(for)g(those)h(argumen)o(ts)f(that)g(are)h(not)f (arra)o(ys.)20 b(Ho)o(w)o(ev)o(er,)75 1231 y(this)14 b(w)o(ould)f(in)o(tro)q(duce)i(an)e(am)o(biguit)o(y)e(if)i(a)h(user)h (w)o(an)o(ted)f(to)f(use)i(a)f(default)f Fc(count)g Fi(of)g(1)h(and)g (sp)q(eci\014c)h(tag.)170 1377 y Fo(void)24 b(Send\(OOMPI_Message)d (buf,)i(int)h(tag)f(=)h(OOMPI_NO_TAG\);)170 1434 y(void)g (Send\(OOMPI_Message_array)c(buf,)k(int)f(count,)g(int)h(tag)f(=)h (OOMPI_NO_TAG\);)166 1528 y(OOMPI)p 289 1528 15 2 v 16 w(Array)p 425 1528 V 17 w(message)12 b Fp(is)i Fg(only)f Fp(used)h(as)f(an)g(in)o(ternal)h(ob)s(ject;)f(it)g(is)h(not)f (considered)i(to)e(b)q(e)h(part)75 1584 y(of)h(the)g(user)g(in)o (terface.)166 1640 y(The)f Fo(OOMPI)p 381 1640 V 16 w(Datatype)f Fp(ob)s(ject)g(is)i(used)f(to)f(describ)q(e)j(the)d(datat)o(yp)q(e)h (of)f(a)g(message.)19 b(In)c(addition)75 1697 y(to)22 b(pro)o(viding)h(access)g(to)f(functions)h(that)f(build)j(the)d(less)h (complicated)h(user-de\014ned)h(datat)o(yp)q(es)75 1753 y(suc)o(h)15 b(as)f Fo(MPI)p 307 1753 V 17 w(Type)p 420 1753 V 17 w(contiguous\(\))f Fp(and)h Fo(MPI)p 897 1753 V 17 w(Type)p 1010 1753 V 17 w(vector\(\))p Fp(,)f(the)i Fo(OOMPI)p 1443 1753 V 16 w(Datatype)f Fp(ob)s(ject)g(also)75 1810 y(pro)o(vides)21 b(a)g(simple,)i(streams-based)d(in)o(terface)h (to)f(build)j(more)d(complex)h(datat)o(yp)q(es)f(with)h Fo(MPI)p 1837 1810 V 17 w(-)75 1866 y(Type)p 174 1866 V 17 w(struct\(\))p Fp(.)e(The)d Fo(OOMPI)p 629 1866 V 17 w(Datatype)e Fp(ob)s(ject)h(can)h(build)h(and)f(commit)f(an)o(y)h (v)m(alid)h(user)f(de\014ned)75 1923 y(MPI)f(datat)o(yp)q(e.)166 1979 y(The)10 b Fo(OOMPI)p 377 1979 V 17 w(User)p 490 1979 V 17 w(type)f Fp(ob)s(ject)h(is)h(the)f(heart)g(of)g (user-de\014ned)i(datat)o(yp)q(es.)17 b(It)11 b(m)o(ust)e(b)q(e)i (inherited)75 2036 y(and)i(initialized)k(b)o(y)c(all)h(ob)s(jects)e (that)g(will)j(b)q(e)f(sen)o(t)f(and/or)f(receiv)o(ed)i(in)g(message)f (passing)g(calls.)20 b(It)13 b(is)75 2092 y(v)o(ery)f(similar)i(to)e Fo(OOMPI)p 495 2092 V 16 w(Message)g Fp(in)h(that)f(it)h(is)g(used)g (to)e(unify)j(all)f(datat)o(yp)q(es)f(\(through)g(inheritence\))75 2149 y(in)o(to)j(a)g(single)i(t)o(yp)q(e)e(that)f(can)i(b)q(e)g(used)f (to)g(access)g(the)g(ob)s(ject's)g(t)o(yp)q(e)g(and)g(data.)166 2205 y(The)i Fo(OOMPI)p 384 2205 V 17 w(Packed)f Fp(ob)s(ject)h(pro)o (vides)h(a)f(simple,)i(streams)d(based)i(in)o(terface)f(for)g(pac)o (king)h(and)75 2261 y(unpac)o(king)23 b(messages.)39 b(The)22 b(bu\013er)g(that)f(is)i(used)f(for)f(pac)o(king)i(and)f (unpac)o(king)h(can)f(either)g(b)q(e)75 2318 y(sp)q(eci\014ed)17 b(b)o(y)e(the)h(user)f(or)g(allo)q(cated)h(b)o(y)f(the)g Fo(OOMPI)p 1009 2318 V 17 w(Packed)f Fp(ob)s(ject.)166 2374 y(The)d Fo(OOMPI)p 378 2374 V 16 w(Op)f Fp(ob)s(ject)g(is)h(a)g (simple)h(wrapp)q(er)e(to)g(the)h Fo(MPI)p 1167 2374 V 16 w(Op)p 1231 2374 V 17 w(create\(\))f Fp(and)g Fo(OOMPI)p 1653 2374 V 17 w(Op)p 1718 2374 V 17 w(free\(\))75 2431 y Fp(functions.)166 2487 y Fo(OOMPI)p 289 2487 V 16 w(Request)h Fp(ob)s(jects)g(are)h(used)g(for)f(non-blo)q(c)o(king)i(comm)o (unications)g(to)e(iden)o(tify)i(a)e(comm)o(u-)75 2544 y(nication)k(op)q(eration)g(and)g(matc)o(h)e(the)i(op)q(eration)f(that) g(initiates)i(the)e(op)q(eration)h(with)f(the)h(op)q(eration)75 2600 y(that)20 b(terminates)g(it.)36 b(A)21 b(request)f(ob)s(ject)g (iden)o(ti\014es)i(prop)q(erties)f(of)f(a)h(comm)o(unication)g(op)q (eration)75 2657 y(suc)o(h)g(as)f(send)h(mo)q(de,)h(the)f(comm)o (unication)g(bu\013er,)h(its)e(con)o(text,)h(and)g(the)g(tag)e(and)i (destination)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 11 11 10 bop 75 49 a Fp(argumen)o(ts)18 b(to)f(b)q(e)j(used)f(for)e(a)h (send)i(\(or)d(receiv)o(e\).)30 b(In)19 b(addition,)h(this)f(ob)s(ject) f(stores)g(information)75 106 y(ab)q(out)d(the)g(status)g(of)f(the)i(p) q(ending)h(comm)o(unication)f(op)q(eration.)166 162 y(The)e Fo(OOMPI)p 381 162 15 2 v 17 w(Status)f Fp(ob)s(ject)g(encapsulates)i (all)g(the)f(op)q(erations)g(that)f(can)h(b)q(e)h(p)q(erformed)f(on)g (an)75 219 y Fo(MPI)p 150 219 V 17 w(Status)i Fp(handle.)25 b(These)17 b(op)q(erations)g(include)i(the)d(MPI)h(functions)g Fo(MPI)p 1463 219 V 17 w(Get)p 1552 219 V 17 w(count\(\))p Fp(,)e Fo(MPI)p 1837 219 V 17 w(-)75 275 y(Get)p 150 275 V 17 w(elements\(\))p Fp(,)e(and)i(functions)g(for)f(determining)i (the)f(source)f(and)h(t)o(yp)q(e)g(of)f(incoming)i(messages.)75 418 y Fq(A.3)59 b(Examples)75 520 y Fp(Here)15 b(w)o(e)g(presen)o(t)h (some)e(small)i(examples)g(of)f(parallel)i(programs)d(written)h(using)h Fo(OOMPI)p Fp(.)75 641 y Fe(A.3.1)49 b(Ring,)17 b(V)o(ersion)f(1)75 727 y Fp(The)f(\014rst)g(example)h(is)g(of)f(the)g(ubiquitous)i(ring)e (program.)75 834 y Fo(//)75 890 y(//)24 b(OOMPI)f(Demo)g(program)75 946 y(//)h(Message)e(ring)75 1003 y(//)75 1116 y(#include)h ()75 1172 y(#include)g("oompi.h")75 1285 y(int)75 1342 y(main\(int)g(argc,)g(char)g(*argv[]\))75 1398 y({)123 1455 y(int)g(msg;)123 1511 y(OOMPI_COMM_WORLD.Init\(ar)o (gc,)e(argv\);)123 1624 y(int)i(rank)g(=)h(OOMPI_COMM_WORLD.Rank\(\);) 123 1680 y(int)f(size)g(=)h(OOMPI_COMM_WORLD.Size\(\);)123 1737 y(int)f(to)71 b(=)24 b(\(rank)f(+)h(1\))g(\045)f(size;)123 1793 y(int)g(from)g(=)h(\(size)f(+)h(rank)f(-)h(1\))g(\045)f(size;)123 1906 y(cout)g(<<)h("I)f(am)h(node)f(")h(<<)f(rank)h(<<)f(")h(of)g(")f (<<)h(size)f(<<)h(endl;)123 1963 y(cout)f(<<)h("Sending)e(to)i(")g(<<)f (to)h(<<)f(")h(and)g(receiving)e(from)i(")f(<<)h(from)f(<<)h(endl;)123 2076 y(if)f(\(rank)g(==)h(0\))170 2132 y(OOMPI_COMM_WORLD[to])e(<<)h (i;)123 2245 y(for)g(\(int)g(i)h(=)g(0;)g(i)f(<)h(5;)g(i++\))f({)170 2301 y(OOMPI_COMM_WORLD[from])e(>>)j(msg;)170 2358 y(cout)g(<<)f("Node) h(")f(<<)h(rank)f(<<)h("received)e(")i(<<)g(msg)f(<<)h(endl;)170 2471 y(if)g(\(rank)f(!=)h(0)g(||)f(i)h(<)g(4\))218 2527 y(OOMPI_COMM_WORLD[to])d(<<)j(i;)123 2584 y(})123 2697 y(cout)f(<<)h("All)f(done!")g(<<)h(endl;)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 12 12 11 bop 123 49 a Fo(OOMPI_COMM_WORLD.Finaliz)o(e\(\);)123 162 y(return)23 b(0;)75 219 y(})75 397 y Fe(A.3.2)49 b(Ring,)17 b(V)o(ersion)f(2)75 483 y Fp(The)f(follo)o(wing)h(program)e (is)i(essen)o(tially)h(the)e(same)g(as)f(the)i(previous)g(one,)e(but)i (some)f(more)f(features)75 539 y(of)j Fo(OOMPI)g Fp(are)g(used.)28 b(A)18 b(cop)o(y)f(is)h(made)g(of)f Fo(OOMPI)p 982 539 15 2 v 16 w(COMM)p 1094 539 V 17 w(WORLD)g Fp(and)h(p)q(orts)f(are)g (created)h(and)f(used)75 596 y(for)e(the)g(stream-based)g(comm)o (unication.)75 702 y Fo(//)75 758 y(//)24 b(OOMPI)f(Demo)g(program)75 815 y(//)h(Message)e(ring)i(with)f(copied)g(communicator)f(and)i(ports) 75 871 y(//)75 984 y(#include)f()75 1041 y(#include)g("oompi.h")75 1153 y(int)75 1210 y(main\(int)g(argc,)g (char)g(*argv[]\))75 1266 y({)123 1323 y(int)g(msg;)123 1379 y(OOMPI_COMM_WORLD.Init\(ar)o(gc,)e(argv\);)123 1436 y(OOMPI_Intra_comm)g(a\(OOMPI_COMM_WORLD\);)123 1549 y(int)i(rank)g(=)h(a.Rank\(\);)123 1605 y(int)f(size)g(=)h (a.Size\(\);)123 1662 y(int)f(to)71 b(=)24 b(\(rank)f(+)h(1\))g(\045)f (size;)123 1718 y(int)g(from)g(=)h(\(size)f(+)h(rank)f(-)h(1\))g(\045)f (size;)123 1774 y(OOMPI_Port)f(To\(a[to]\);)123 1831 y(OOMPI_Port)g(From\(a[from]\);)123 1944 y(cout)h(<<)h("I)f(am)h(node)f (")h(<<)f(rank)h(<<)f(")h(of)g(")f(<<)h(size)f(<<)h(endl;)123 2000 y(cout)f(<<)h("Sending)e(to)i(")g(<<)f(to)h(<<)f(")h(and)g (receiving)e(from)i(")f(<<)h(from)f(<<)h(endl;)123 2113 y(if)f(\(rank)g(==)h(0\))170 2170 y(To)g(<<)g(i;)123 2283 y(for)f(\(int)g(i)h(=)g(0;)g(i)f(<)h(5;)g(i++\))f({)170 2339 y(From)h(>>)f(msg;)170 2395 y(cout)h(<<)f("Node)h(")f(<<)h(rank)f (<<)h(")f(received)g(")h(<<)g(msg)f(<<)h(endl;)170 2508 y(if)g(\(rank)f(!=)h(0)g(||)f(i)h(<)g(4\))218 2565 y(To)g(<<)f(i;)123 2621 y(})1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 13 13 12 bop 123 49 a Fo(cout)23 b(<<)h("All)f(done!")g(<<)h(endl;)123 106 y(OOMPI_COMM_WORLD.Finaliz)o(e\(\);)123 219 y(return)f(0;)75 275 y(})75 418 y Fq(A.4)59 b(Design)75 520 y Fp(In)17 b(this)g(section,)g(the)g(functional)g(in)o(terface)g(to)f(eac)o(h)g (OOMPI)h(ob)s(ject)f(will)i(b)q(e)g(describ)q(ed)g(in)f(detail;)75 576 y(all)f(public)i(mem)o(b)q(er)e(functions)g(will)h(describ)q(e)h (what)d(they)g(do,)h(what)f(MPI-1)g(functions)h(are)f(used)i(to)75 633 y(implemen)o(t)d(it,)e(and)h(what)f(kinds)h(of)f(copies)h(are)f(p)q (erformed)h(\(if)f(appropriate\).)19 b(An)13 b(index)g(is)g(pro)o (vided)75 689 y(at)h(the)h(end)h(of)e(this)h(annex)h(to)e(cross)g (reference)i(MPI-1)e(functions)i(to)e(their)h(corresp)q(onding)h(OOMPI) 75 746 y(ob)s(jects.)166 802 y(Since)21 b(most)d(MPI-1)h(ob)s(jects)g (\(e.g.,)g Fo(MPI)p 904 802 15 2 v 16 w(Comm)p Fp(\))g(are)g(simply)h (handles)h(to)d(opaque)i(data,)f(they)75 858 y(cannot)e(b)q(e)i (directly)g(copied.)28 b(Therefore,)18 b(an)o(y)f(OOMPI)h(ob)s(ject)f (that)g(con)o(tains)h(an)f(MPI-1)h(handle)75 915 y(has)h(t)o(w)o(o)e(c) o(hoices)j(when)f(making)g(a)f(cop)o(y)h(of)f(itself:)28 b(in)o(v)o(ok)o(e)19 b(the)g(appropriate)f(MPI-1)h(function)h(to)75 971 y(cop)o(y)e(the)h(opaque)f(data,)g(or)g(use)h(a)f(reference)h(coun) o(ting)f(sc)o(heme)h(that)f(will)i(pro)o(vide)f(references)g(to)75 1028 y(the)12 b(handle)i(\(\014nally)f(in)o(v)o(oking)g(the)g (appropriate)f Fo(MPI)p 1018 1028 V 17 w(*)p 1059 1028 V 17 w(free\(\))f Fp(function)i(when)g(all)g(references)g(ha)o(v)o(e)75 1084 y(b)q(een)19 b(deleted\).)29 b(The)18 b(former)f(sc)o(heme)h(is)g (referred)g(to)f(as)h(a)f(\\deep")h(cop)o(y;)h(for)e(example,)i(when)f (an)75 1141 y Fo(OOMPI)p 198 1141 V 16 w(Comm)11 b Fp(ob)s(ject)f(is)i (copied,)h(it)e(in)o(v)o(ok)o(es)g Fo(MPI)p 914 1141 V 17 w(Comm)p 1027 1141 V 17 w(dup\(\))p Fp(,)f(and)i(is)f(therefore)g (a)g(\\deep")g(cop)o(y)l(.)19 b(The)75 1197 y(latter)f(sc)o(heme)g(is)g (a)g(referred)g(to)f(as)h(a)f(\\shallo)o(w")h(sc)o(heme;)h(when)f(an)g Fo(OOMPI)p 1468 1197 V 17 w(Op)f Fp(ob)s(ject)h(is)g(copied,)75 1254 y(it)d(creates)f(another)g(reference)i(to)e(the)g(in)o(ternal)i Fo(MPI)p 1003 1254 V 17 w(Op)e Fp(handle.)21 b(When)15 b(all)g(references)h(are)e(deleted,)75 1310 y Fo(MPI)p 150 1310 V 17 w(Op)p 215 1310 V 17 w(free\(\))g Fp(is)i(in)o(v)o(ok)o (ed.)166 1367 y(A)11 b(b)q(eta)g(v)o(ersion)g(of)g(OOMPI)g(will)i(so)q (on)e(b)q(e)g(a)o(v)m(ailable)i(\(p)q(ending)g(c)o(hanges)e(from)f(the) h(MPI)g(F)l(orum\))75 1423 y(from:)409 1517 y Fo (http://www.cse.nd.edu/~lsc/)o(researc)o(h/oompi)o(/)75 1611 y Fp(Questions,)16 b(commen)o(ts,)e(bugs,)h(etc.,)f(can)i(b)q(e)g (sen)o(t)f(to)f Fo(oompi@lsc.cse.nd.edu)p Fp(.)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 14 14 13 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Cart)p 693 49 V 17 w(comm)100 156 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 213 y(class)h(OOMPI_Cart_comm)100 320 y Fh(Description)145 b Fp(A)17 b(class)g(for)g(MPI)g(comm)o (unicators)g(with)g(Cartesian)g(top)q(ology)l(.)25 b(This)18 b(class)f(is)458 376 y(deriv)o(ed)f(from)e Fo(OOMPI)p 847 376 V 17 w(Intra)p 984 376 V 16 w(comm)p Fp(.)100 483 y Fh(Constructo)o(rs/Destructo)o(rs)458 575 y Fo(OOMPI)p 581 575 V 16 w(Cart)p 693 575 V 17 w(comm\(\).)19 b Fp(Default)c (constructor.)k(The)c(in)o(ternal)h(MPI)f(comm)o(uni-)458 631 y(cator)f(is)i(set)f(to)f Fo(MPI)p 821 631 V 17 w(COMM)p 934 631 V 17 w(NULL)p Fp(.)458 723 y Fo(OOMPI)p 581 723 V 16 w(Cart)p 693 723 V 17 w(comm\(const)22 b(OOMPI)p 1092 723 V 17 w(Intra)p 1229 723 V 16 w(comm&)i(intra)p 1509 723 V 16 w(comm,)f(int)h(ndims,)458 780 y(int)f(dims[],)g(int)h (periods[],)e(int)h(reorder)g(=)h(0\).)18 b Fp(Constructor.)f(Uses)458 836 y Fo(MPI)p 533 836 V 16 w(Cart)p 645 836 V 17 w(create\(\))12 b Fp(to)g(construct)g(an)h(MPI)f(comm)o(unicator)g(ha)o(ving)h(a)g (Carte-)458 893 y(sian)i(top)q(ology)g(from)g(the)g(in)o(tra-comm)o (unicator)g Fo(intra)p 1454 893 V 16 w(comm)p Fp(.)458 985 y Fo(OOMPI)p 581 985 V 16 w(Cart)p 693 985 V 17 w(comm\(const)22 b(OOMPI)p 1092 985 V 17 w(Cart)p 1205 985 V 16 w(comm&)i(a\).)18 b Fp(Cop)o(y)9 b(constructor.)18 b(Cre-)458 1041 y(ate)h(a)h(new)h Fo(OOMPI)p 803 1041 V 16 w(Cart)p 915 1041 V 17 w(comm)e Fp(and)i(p)q(erform)f(a)f(shallo)o(w)i(cop)o(y)f(of)g Fo(a)p Fp(.)34 b Fo(MPI)p 1837 1041 V 17 w(-)458 1098 y(Comm)p 557 1098 V 16 w(dup\(\))19 b Fp(is)h(in)o(v)o(ok)o(ed)g(on)g (the)g(in)o(ternal)g Fo(MPI)p 1327 1098 V 17 w(Comm)f Fp(of)g Fo(a)h Fp(to)f(create)g(a)h(new)458 1154 y(MPI)15 b(comm)o(unicator.)458 1246 y Fo(const)23 b(OOMPI)p 724 1246 V 16 w(Cart)p 836 1246 V 17 w(comm&)g(operator=\(const)f(OOMPI)p 1498 1246 V 17 w(Cart)p 1611 1246 V 16 w(comm&)h(a\).)18 b Fp(As-)458 1303 y(signmen)o(t)e(op)q(erator.)k(P)o(erforms)14 b(a)h(shallo)o(w)h(cop)o(y;)f(the)h(existing)h(MPI)e(comm)o(u-)458 1359 y(nicator)j(is)g(freed)g(with)g Fo(MPI)p 959 1359 V 17 w(Comm)p 1072 1359 V 17 w(free\(\))f Fp(and)h Fo(MPI)p 1413 1359 V 17 w(Comm)p 1526 1359 V 16 w(dup\(\))f Fp(is)i(in)o(v)o(ok) o(ed)458 1416 y(on)14 b(the)h(in)o(ternal)h Fo(MPI)p 841 1416 V 17 w(Comm)e Fp(of)g Fo(a)h Fp(to)f(create)h(the)g(in)o (ternal)h(MPI)e(comm)o(unicator)458 1472 y(for)g(this.)100 1579 y Fh(Access)j(and)f(Info)o(rmation)458 1671 y Fo(OOMPI)p 581 1671 V 16 w(Cart)p 693 1671 V 17 w(comm&)23 b(Sub\(int)g(remain)p 1188 1671 V 16 w(dims[]\).)40 b Fp(Calls)23 b Fo(MPI)p 1629 1671 V 17 w(Cart)p 1742 1671 V 17 w(sub\(\))458 1728 y Fp(and)15 b(returns)g(the)g(new)h(comm)o(unicator.)458 1820 y Fo(int)23 b(Dim)p 628 1820 V 17 w(get\(\).)30 b Fp(Calls)20 b Fo(MPI)p 1009 1820 V 16 w(Cartdim)p 1193 1820 V 17 w(get\(\))e Fp(and)h(returns)f(n)o(um)o(b)q(er)i(of)e(di-)458 1876 y(mensions.)458 1968 y Fo(int)23 b(*Get\(int)g(maxdims,)g(int)g (dims[],)g(int)h(period[],)e(int)i(coords[]\).)458 2025 y Fp(Calls)16 b Fo(MPI)p 647 2025 V 16 w(Cart)p 759 2025 V 17 w(get\(\))e Fp(and)i(returns)f(dimension)i(arra)o(y)l(.)458 2117 y Fo(int)23 b(Rank\(int)g(coords[]\).)c Fp(Calls)d Fo(MPI)p 1216 2117 V 16 w(Cart)p 1328 2117 V 17 w(rank\(\))e Fp(and)i(returns)f(rank.)458 2209 y Fo(int)23 b(*Coords\(int)g(rank,)g (int)g(maxdims,)g(int)g(coords[]\).)d Fp(Calls)c Fo(MPI)p 1837 2209 V 17 w(-)458 2265 y(Cart)p 557 2265 V 16 w(coords\(\))e Fp(and)i(returns)f(co)q(ordinates.)458 2357 y Fo(int)23 b(Shift\(int)g(direction,)f(int&)i(rank)p 1273 2357 V 16 w(source,)f(int)h(disp)f(=)h(1\).)18 b Fp(Calls)458 2414 y Fo(MPI)p 533 2414 V 16 w(Cart)p 645 2414 V 17 w(shift\(\))c Fp(and)i(returns)f(rank)g(of)f(destination.)458 2506 y Fo(int)23 b(Map\(int)g(ndims,)g(int)h(dims[],)e(int)i (periods[]\).)17 b Fp(Calls)11 b Fo(MPI)p 1758 2506 V 17 w(Cart)p 1871 2506 V 16 w(-)458 2562 y(map\(\))j Fp(and)i(returns)f (rank)g(of)f(destination.)100 2669 y Fh(See)i(Also)196 b Fo(OOMPI)p 581 2669 V 16 w(Comm)p Fp(,)14 b Fo(OOMPI)p 840 2669 V 17 w(Intra)p 977 2669 V 16 w(comm)p Fp(,)h Fo(OOMPI)p 1237 2669 V 16 w(Graph)p 1373 2669 V 17 w(comm)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 15 15 14 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Comm)100 168 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 224 y(class)h(OOMPI_Comm)100 343 y Fh(Description)145 b Fp(An)13 b(abstract)e(base)i(class)g(for)f(MPI)h(comm)o(unicators.)18 b(This)c(class)f(is)g(resp)q(onsible)458 400 y(for)j(main)o(taining)h (an)g(in)o(ternal)h(MPI)e(comm)o(unicator,)h(information)f(ab)q(out)h (the)458 456 y(group)f(asso)q(ciated)g(with)g(the)h(comm)o(unicator,)e (and)i(an)f(arra)o(y)e(of)i Fo(OOMPI)p 1748 456 V 17 w(Port)p Fp(s)458 513 y(\(one)j(p)q(ort)h(for)f(eac)o(h)g(MPI)h(pro)q (cess)g(ha)o(ving)g(a)f(rank)h(in)g(the)g(comm)o(unicator\).)458 569 y(Cac)o(hed)15 b(attributes)g(are)g(also)g(asso)q(ciated)h(with)f (this)h(class.)100 688 y Fh(Constructo)o(rs/Destructo)o(rs)458 786 y Fo(OOMPI)p 581 786 V 16 w(Comm\(\).)27 b Fp(Default)18 b(constructor.)27 b(The)18 b(in)o(ternal)h(MPI)e(comm)o(unicator)458 842 y(is)e(set)g(to)g Fo(MPI)p 705 842 V 17 w(COMM)p 818 842 V 16 w(NULL)p Fp(.)458 940 y Fo(OOMPI)p 581 940 V 16 w(Comm\(const)23 b(OOMPI)p 980 940 V 16 w(Comm&)g(a\).)d Fp(Calls)c Fo(MPI)p 1417 940 V 17 w(Comm)p 1530 940 V 17 w(dup\(\))e Fp(to)h(mak)o(e)f(a)458 997 y(cop)o(y)h(of)g Fo(a)p Fp(.)458 1095 y Fo(const)23 b(OOMPI)p 724 1095 V 16 w(Comm&)h(operator=\(const)d(OOMPI)p 1385 1095 V 17 w(Comm&)i(a\).)31 b Fp(Calls)19 b Fo(MPI)p 1837 1095 V 17 w(-)458 1151 y(Comm)p 557 1151 V 16 w(dup\(\))c Fp(to)f(mak)o(e)h(a)g(cop)o(y)g(of)g Fo(a)p Fp(.)458 1249 y Fo(virtual)23 b Ff(\030)p Fo(OOMPI)p 807 1249 V 16 w(Comm\(\).)18 b Fp(The)11 b(destructor)g(deletes)h(all)g(of)e (its)h Fo(OOMPI)p 1742 1249 V 17 w(Ports)458 1305 y Fp(then)k(call)i Fo(MPI)p 720 1305 V 16 w(Comm)p 832 1305 V 17 w(free\(\))d Fp(to)h(free)g(the)g(comm)o(unicator.)100 1424 y Fh(MPI)f(A)o (ttributes)75 b Fo(void)23 b(Attr)p 676 1424 V 17 w(delete\(int)f (keyval\).)d Fp(Calls)d Fo(MPI)p 1352 1424 V 17 w(Attr)p 1465 1424 V 17 w(delete\(\))p Fp(.)458 1522 y Fo(int)23 b(Attr)p 652 1522 V 17 w(get\(int)g(keyval,)g(void)g(*attribute)p 1410 1522 V 16 w(val\).)18 b Fp(Calls)10 b Fo(MPI)p 1744 1522 V 17 w(Attr)p 1857 1522 V 17 w(-)458 1579 y(get\(\))p Fp(.)19 b(It)c(returns)g(the)g Fo(flag)g Fp(out)g(v)m(alue.)458 1676 y Fo(void)23 b(Attr)p 676 1676 V 17 w(put\(int)g(keyval,)f(void)i (*attribute)p 1434 1676 V 16 w(val\).)17 b Fp(Calls)11 b Fo(MPI)p 1768 1676 V 17 w(Attr)p 1881 1676 V 17 w(-)458 1733 y(put\(\))p Fp(.)100 1852 y Fh(Communicato)o(r)i(Management)458 1950 y Fo(int)23 b(Compare\(OOMPI)p 868 1950 V 16 w(Comm&)g(a\).)18 b Fp(Calls)11 b Fo(MPI)p 1298 1950 V 17 w(Comm)p 1411 1950 V 16 w(compare\(\))f Fp(and)g(returns)458 2006 y(the)h(result,)i (whic)o(h)f(can)g(b)q(e)h(either)f Fo(OOMPI)p 1182 2006 V 17 w(IDENT)p Fp(,)e Fo(OOMPI)p 1462 2006 V 17 w(CONGRUENT)p Fp(,)g Fo(OOMPI)p 1838 2006 V 16 w(-)458 2063 y(SIMILAR)p Fp(,)16 b(or)h Fo(OOMPI)p 836 2063 V 17 w(UNEQUAL)p Fp(.)f(These)j (constan)o(ts)e(ha)o(v)o(e)g(the)h(same)f(meanings)458 2119 y(as)e(their)g(MPI)g(coun)o(terparts.)458 2217 y Fo(friend)23 b(int)g(operator==\(OOMPI)p 1107 2217 V 15 w(Comm&)g(a,)h(OOMPI)p 1457 2217 V 16 w(Comm&)g(b\).)18 b Fp(Calls)11 b Fo(MPI)p 1888 2217 V 16 w(-)458 2273 y(Comm)p 557 2273 V 16 w(compare\(\))k Fp(and)h(returns)f(a)h(logical)h Fo(true)e Fp(if)h(the)g(result)g(is)g(not)g Fo(OOMPI)p 1838 2273 V 16 w(-)458 2330 y(UNEQUAL)p Fp(.)458 2428 y Fo(friend)23 b(int)g(operator!=\(OOMPI)p 1107 2428 V 15 w(Comm&)g(a,)h(OOMPI)p 1457 2428 V 16 w(Comm&)g(b\).)18 b Fp(Calls)11 b Fo(MPI)p 1888 2428 V 16 w(-)458 2484 y(Comm)p 557 2484 V 16 w(compare\(\))23 b Fp(and)h(returns)f(a)h (logical)g Fo(true)f Fp(if)i(the)e(result)h Fg(is)g Fo(OOMPI)p 1838 2484 V 16 w(-)458 2541 y(UNEQUAL)p Fp(.)458 2639 y Fo(OOMPI)p 581 2639 V 16 w(Group&)f(Group\(\).)h Fp(Returns)17 b(the)g Fo(OOMPI)p 1354 2639 V 16 w(Group)f Fp(of)h(the)f(comm)o (unica-)458 2695 y(tor.)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 16 16 15 bop 458 49 a Fo(int)23 b(Rank\(\).)e Fp(Calls)16 b Fo(MPI)p 931 49 15 2 v 17 w(Comm)p 1044 49 V 17 w(rank\(\))e Fp(and)i(returns)g(the)f(rank)h(in)g(the)g(com-)458 106 y(m)o(unicator.)458 204 y Fo(int)23 b(Size\(\).)j Fp(Calls)18 b Fo(MPI)p 938 204 V 17 w(Comm)p 1051 204 V 17 w(size\(\))e Fp(and)i(returns)f(the)h(size)g(of)f(the)h(com-)458 260 y(m)o(unicator.)100 379 y Fh(Info)o(rmation)138 b Fo(virtual)23 b(int)g(Test)p 843 379 V 17 w(inter\(\))g(=)g(0.)c Fp(This)13 b(is)f(a)g(pure)h(virtual)f(function)h(whic)o(h)458 435 y(is)e(de\014ned)h(in)f(the)f(classes)h(whic)o(h)h(inherit)f(from)f Fo(class)23 b(OOMPI)p 1548 435 V 17 w(Comm)p Fp(.)18 b(It)10 b(returns)458 492 y Fo(true)k Fp(if)i(the)f(comm)o(unicator)g (is)h(an)f(in)o(ter-comm)o(unicator,)g(otherwise)g Fo(false)458 590 y(int)23 b(Initialized\(\).)42 b Fp(Calls)24 b Fo(MPI)p 1128 590 V 17 w(Initialized\(\))p Fp(.)41 b(It)24 b(returns)e(true)h (or)458 646 y(false.)458 744 y Fo(int)g(Pack)p 652 744 V 17 w(size\(OOMPI)p 909 744 V 16 w(Message&)f(data\).)37 b Fp(Calls)21 b Fo(MPI)p 1511 744 V 17 w(Pack)p 1624 744 V 17 w(size\(\))f Fp(and)458 801 y(returns)15 b(the)g(size.)458 899 y Fo(int)23 b(Pack)p 652 899 V 17 w(size\(OOMPI)p 909 899 V 16 w(Array)p 1045 899 V 16 w(message&)g(data,)g(int)h(count)f (=)g(1\).)c Fp(Calls)458 955 y Fo(MPI)p 533 955 V 16 w(Pack)p 645 955 V 17 w(size\(\))14 b Fp(and)i(returns)f(the)g(size.) 100 1074 y Fh(Erro)o(r)e(Handling)80 b Fo(MPI)p 533 1074 V 16 w(Errhandler)23 b(Errhandler)p 1052 1074 V 16 w(get\(\).)18 b Fp(Calls)10 b Fo(MPI)p 1410 1074 V 17 w(Errhandler)p 1667 1074 V 16 w(get\(\))f Fp(and)458 1130 y(returns)15 b(the)g Fo(errhandler)f Fp(curren)o(tly)i(asso)q(ciated)f(with)h(the)f (comm)o(unicator.)458 1228 y Fo(void)23 b(Errhandler)p 820 1228 V 16 w(set\(MPI)p 1004 1228 V 16 w(Errhandler)g(handler\).)17 b Fp(Calls)11 b Fo(MPI)p 1697 1228 V 16 w(Errhandler)p 1953 1228 V 16 w(-)458 1285 y(set\(\))j Fp(and)i(returns)f(nothing.)458 1383 y Fo(void)23 b(Abort\(int)g(errorcode\).)17 b Fp(Calls)12 b Fo(MPI)p 1282 1383 V 17 w(Abort\(\))e Fp(and)h(returns)g(nothing.)100 1501 y Fh(P)o(o)o(rt)j(Access)138 b Fo(OOMPI)p 581 1501 V 16 w(Port&)23 b(operator[]\(int)f(i\).)32 b Fp(Returns)20 b(the)f Fo(i)1482 1485 y Fb(th)1536 1501 y Fp(p)q(ort)g(in)h(the)f (com-)458 1558 y(m)o(unicator,)14 b(where)i Fo(i)f Fp(is)h(the)f(rank)g (of)g(that)f(p)q(ort)h(in)h(the)f(comm)o(unicator.)458 1656 y Fo(OOMPI)p 581 1656 V 16 w(Any)p 669 1656 V 17 w(port&)23 b(Any\(void\).)18 b Fp(Returns)13 b(the)g(\\an)o(y")f(p)q (ort)g(p)q(ort)g(in)h(the)g(com-)458 1712 y(m)o(unicator.)100 1831 y Fh(See)j(Also)196 b Fo(OOMPI)p 581 1831 V 16 w(Intra)p 717 1831 V 17 w(comm)p Fp(,)10 b Fo(OOMPI)p 973 1831 V 16 w(Inter)p 1109 1831 V 17 w(comm)p Fp(,)g Fo(OOMPI)p 1365 1831 V 17 w(Cart)p 1478 1831 V 16 w(comm)p Fp(,)g Fo(OOMPI)p 1733 1831 V 17 w(Graph)p 1870 1831 V 16 w(-)458 1887 y(comm)p Fp(,)k Fo(OOMPI)p 704 1887 V 16 w(Comm)p 816 1887 V 17 w(world)p Fp(,)g Fo(OOMPI)p 1100 1887 V 17 w(Port)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 17 17 16 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Comm)p 693 49 V 17 w(world)100 168 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 224 y(class)h(OOMPI_Comm_world)100 343 y Fh(Description)145 b Fp(Deriv)o(ed)18 b(from)f Fo(OOMPI)p 862 343 V 17 w(Intra)p 999 343 V 16 w(Comm)p Fp(.)28 b(The)18 b Fo(OOMPI)p 1368 343 V 17 w(Comm)p 1481 343 V 16 w(World)g Fp(has)g(t)o(w)o(o)e(pri-)458 400 y(mary)c(functions)i(whic)o(h)g(distinguish)i(it)d(from)g(its)g (base)g(class:)20 b Fo(MPI)p 1632 400 V 16 w(Init\(\))13 b Fp(and)458 456 y Fo(MPI)p 533 456 V 16 w(Finalize)p Fp(.)19 b(There)d(is)g(only)g(one)f(instance)h(of)f Fo(OOMPI)p 1483 456 V 16 w(Comm)p 1595 456 V 17 w(World)g Fp(named)458 513 y Fo(OOMPI)p 581 513 V 16 w(COMM)p 693 513 V 17 w(WORLD)f Fp(whic)o(h)i(has)f Fo(MPI)p 1127 513 V 17 w(COMM)p 1240 513 V 17 w(WORLD)f Fp(as)h(its)g(in)o(ternal)h(MPI)f(com-)458 569 y(m)o(unicator.)100 688 y Fh(Constructo)o(rs/Destructo)o(rs)458 786 y Fo(OOMPI)p 581 786 V 16 w(Comm)p 693 786 V 17 w(world\(void\).)i Fp(This)10 b(creates)g(an)g(empt)o(y)g(comm)o(unicator.)18 b Fo(OOMPI)p 1885 786 V 17 w(-)458 842 y(COMM)p 557 842 V 16 w(WORLD)d Fp(do)q(es)g(not)g(con)o(tain)h Fo(MPI)p 1124 842 V 16 w(COMM)p 1236 842 V 17 w(WORLD)e Fp(un)o(til)j Fo(Init\(\))d Fp(is)i(called.)458 940 y Fo(OOMPI)p 581 940 V 16 w(Comm)p 693 940 V 17 w(world\(const)22 b(OOMPI)p 1116 940 V 17 w(Comm)p 1229 940 V 16 w(world&)h(a\).)c Fp(This)11 b(creates)f(a)g(com-)458 997 y(m)o(unicator)23 b(that)g(con)o(tain)g Fo(MPI)p 1032 997 V 17 w(COMM)p 1145 997 V 16 w(NULL)p Fp(.)g Fo(OOMPI)p 1413 997 V 16 w(COMM)p 1525 997 V 17 w(WORLD)g Fp(is)h(not)e(al-)458 1053 y(lo)o(w)o(ed)15 b(to)g(b)q(e)g(copied.)458 1151 y Fo(OOMPI)p 581 1151 V 16 w(Comm)p 693 1151 V 17 w(world&)23 b(operator=\(const)f(OOMPI)p 1379 1151 V 16 w(Comm)p 1491 1151 V 17 w(world&)h(a\).)45 b Fp(Cre-)458 1207 y(ates)21 b(a)g(comm)o(unicator)g(that)f(con)o(tain)i Fo(MPI)p 1251 1207 V 17 w(COMM)p 1364 1207 V 16 w(NULL)p Fp(.)f Fo(OOMPI)p 1630 1207 V 16 w(COMM)p 1742 1207 V 17 w(WORLD)458 1264 y Fp(is)15 b(not)g(allo)o(w)o(ed)h(to)e(b)q(e)i (copied.)458 1362 y Ff(\030)p Fo(OOMPI)p 616 1362 V 17 w(Comm)p 729 1362 V 16 w(world\(void\).)j Fp(Destructor.)g(Calls)c Fo(MPI)p 1482 1362 V 17 w(Finalize\(\))p Fp(.)100 1481 y Fh(Init/Finalize)128 b Fo(void)23 b(Init\(int&)g(argc,)g(char)g (*argv[]\).)g Fp(Calls)18 b Fo(MPI)p 1508 1481 V 16 w(Init\(\))p Fp(.)24 b(The)16 b(\014rst)458 1537 y(time)k(this)g(function)h(is)g (called,)h(it)e(adds)g Fo(MPI)p 1276 1537 V 17 w(COMM)p 1389 1537 V 17 w(WORLD)f Fp(to)g Fo(OOMPI)p 1725 1537 V 17 w(COMM)p 1838 1537 V 16 w(-)458 1593 y(WORLD)13 b Fp(and)h(creates)f(all)i(the)f(p)q(orts.)19 b(Nothing)14 b(will)i(happ)q(en)f(if)f Fo(Init\(\))f Fp(is)h(called)458 1650 y(more)g(than)i(once.)458 1748 y Fo(void)23 b(Finalize\(void\).)h Fp(Calls)17 b Fo(MPI)p 1151 1748 V 17 w(Finalize\(\))p Fp(.)24 b(If)18 b(the)f(program)f(termi-)458 1804 y(nates)j(b)q(efore)h Fo(Finalize\(\))e Fp(is)h(called,)j(then)e Fo(MPI)p 1359 1804 V 17 w(Finalize\(\))e Fp(is)i(automati-)458 1861 y(cally)c(b)o(y)f(virtue)h(of)f(the)g(destructor)g(in)o(v)o(o)q (cation.)100 1980 y Fh(See)h(Also)196 b Fo(OOMPI)p 581 1980 V 16 w(Comm)p Fp(,)14 b Fo(OOMPI)p 840 1980 V 17 w(Intra)p 977 1980 V 16 w(comm)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 18 18 17 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Datatype)100 168 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 224 y(class)h(OOMPI_Datatype)100 343 y Fh(Description)145 b Fp(An)12 b Fo(OOMPI)p 652 343 V 17 w(Datatype)f Fp(ob)s(ject)g(is)i(used)g(to)e(create)h (user-de\014ned)i(datat)o(yp)q(es.)19 b(Ac-)458 399 y(cess)d(is)i(pro)o (vided)f(to)f(all)h(the)g(MPI-1)f(functions)h(that)f(are)g(used)i(to)d (create)i(and)458 456 y(commit)j(datat)o(yp)q(es.)34 b(The)20 b Fo(MPI)p 1041 456 V 17 w(Datatype)e Fp(handle)k(that)d(is)i (generated)f(from)458 512 y(the)15 b Fo(MPI)p 611 512 V 17 w(Type)p 724 512 V 16 w(*\(\))g Fp(calls)h(is)g(con)o(tained)g (within)g(the)g Fo(OOMPI)p 1520 512 V 16 w(Datatype)e Fp(ob)s(ject.)458 610 y(The)h Fo(OOMPI)p 674 610 V 16 w(Datatype)f Fp(ob)s(ject)g(is)h(mean)o(t)f(to)g(b)q(e)h(a)f(static)h (attribute)f(of)g(all)i(user)458 667 y(ob)s(jects)h(that)h(will)h(b)q (e)g(passed)f(in)h(messages.)28 b(Therefore,)19 b(the)f(datat)o(yp)q(e) f(only)458 723 y(needs)g(to)g(b)q(e)g(created)g(once)g(for)g(the)g(en)o (tire)g(class)g(\(see)g(the)g Fo(Built\(\))f Fp(mem)o(b)q(er)458 780 y(function)h(b)q(elo)o(w\).)23 b(The)16 b(constructors)g(for)f(the) h(user)h(ob)s(ject)e(should)i(c)o(hec)o(k)g(the)458 836 y Fo(Built\(\))d Fp(function)i(to)f(see)g(if)h(the)f(datat)o(yp)q(e)g (has)g(b)q(een)i(built)f(y)o(et.)k(If)15 b(not,)g(they)458 893 y(should)h(call)g(the)g(prop)q(er)f(mem)o(b)q(er)g(functions)h(to)f (built)h(the)g(datat)o(yp)q(e.)100 1011 y Fh(Constructo)o(rs/Destructo) o(rs)458 1109 y Fo(OOMPI)p 581 1109 V 16 w(Datatype\(MPI)p 885 1109 V 16 w(Datatype)23 b(a)g(=)h(MPI)p 1283 1109 V 17 w(DATATYPE)p 1492 1109 V 16 w(NULL\).)9 b Fp(Default)h(con-)458 1166 y(structor.)27 b(Will)19 b(create)f(a)f(reference)i(to)e Fo(MPI)p 1262 1166 V 17 w(DATATYPE)p 1471 1166 V 16 w(NULL)g Fp(if)h(not)g(in)o(v)o(ok)o(ed)458 1222 y(with)10 b(a)g(v)m(alid)i Fo(MPI)p 769 1222 V 17 w(Datatype)p Fp(.)17 b(The)10 b(usual)h(useage)f(is)h(to)e(instan)o(tiate)h(an)h Fo(OOMPI)p 1840 1222 V 16 w(-)458 1278 y(Datatype)j Fp(with)h(no)g(argumen)o(ts)g (and)g(then)h(pro)q(ceede)g(to)e(create)h(the)g(datat)o(yp)q(e)458 1335 y(with)g(the)h(functions)g(listed)g(b)q(elo)o(w.)458 1433 y Fo(OOMPI)p 581 1433 V 16 w(Datatype\(const)22 b(OOMPI)p 1075 1433 V 17 w(Datatype)g(&a\).)e Fp(P)o(erform)13 b(a)h(shallo)o(w)h(\(ref-)458 1489 y(erence)h(coun)o(ted\))f(cop)o(y)g (of)g(the)g Fo(OOMPI)p 1144 1489 V 16 w(Datatype)g Fp(ob)s(ject.)458 1587 y Fo(OOMPI)p 581 1587 V 16 w(Datatype)23 b(&operator=\(const)f (OOMPI)p 1338 1587 V 16 w(Datatype)h(&a\).)18 b Fp(P)o(erform)11 b(a)458 1644 y(shallo)o(w)k(\(reference)h(coun)o(ted\))f(assignmen)o (t.)458 1742 y Fo(virtual)23 b Ff(\030)p Fo(OOMPI)p 807 1742 V 16 w(Datatype\(void\).)18 b Fp(Delete)c(the)f(curren)o(t)g (reference)h(to)f(the)458 1798 y(in)o(ternal)j Fo(MPI)p 701 1798 V 17 w(Datatype)e Fp(\(ma)o(y)g(trigger)g(a)h(call)i(to)d Fo(MPI)p 1435 1798 V 17 w(Type)p 1548 1798 V 17 w(free\(\))p Fp(\).)100 1917 y Fh(Accesso)o(rs)179 b Fo(MPI)p 533 1917 V 16 w(Datatype)23 b(*Get)p 860 1917 V 17 w(type\(void\).)17 b Fp(Return)11 b(a)e(p)q(oin)o(ter)i(to)f(the)g(in)o(ternal)h Fo(MPI)p 1851 1917 V 17 w(-)458 1973 y(Datatype)p Fp(.)17 b(This)11 b(is)g(necessary)f(for)g(functions)h(that)f(require)h(the)g (MPI)f(datat)o(yp)q(e,)458 2030 y(suc)o(h)15 b(as)g Fo(MPI)p 691 2030 V 17 w(Send\(\))p Fp(,)f(etc.)458 2128 y Fo(int)23 b(Built\(\).)18 b Fp(Indicates)13 b(whether)f(the)g(datat)o(yp)q(e)f (has)g(b)q(een)i(built)g(y)o(et)f(or)f(not.)458 2184 y(This)j(function)h(should)h(b)q(e)e(c)o(hec)o(k)o(ed)h(in)g(the)f (constructors)g(for)f(the)h(user)h(ob)s(jects)458 2240 y(that)f(will)j(ha)o(v)o(e)e(corresp)q(onding)h(MPI)f(datat)o(yp)q(es)g (built.)458 2338 y Fo(MPI)p 533 2338 V 16 w(Aint)24 b(Extent\(void\).) 18 b Fp(Calls)e Fo(MPI)p 1185 2338 V 17 w(Type)p 1298 2338 V 17 w(extent\(\))e Fp(with)h(the)g(datat)o(yp)q(e)458 2395 y(of)f(the)i(instance)g(of)f(this)g(ob)s(ject)g(and)g(returns)g (the)h(exten)o(t.)458 2493 y Fo(int)23 b(Size\(void\).)f Fp(Calls)17 b Fo(MPI)p 1029 2493 V 17 w(Type)p 1142 2493 V 16 w(size\(\))e Fp(with)i(the)f(datat)o(yp)q(e)g(of)g(the)g(in-)458 2549 y(stance)f(of)g(this)g(ob)s(ject)g(and)g(returns)g(the)h(size.)458 2647 y Fo(MPI)p 533 2647 V 16 w(Aint)24 b(Lb\(void\).)39 b Fp(Calls)23 b Fo(MPI)p 1117 2647 V 17 w(Type)p 1230 2647 V 16 w(lb\(\))f Fp(with)g(the)g(datat)o(yp)q(e)f(of)h(the)458 2704 y(instance)16 b(of)f(this)g(ob)s(ject)g(and)g(returns)g(the)h(lo)o (w)o(er)e(b)q(ound.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 19 19 18 bop 458 49 a Fo(MPI)p 533 49 15 2 v 16 w(Aint)24 b(Ub\(void\).)39 b Fp(Calls)23 b Fo(MPI)p 1117 49 V 17 w(Type)p 1230 49 V 16 w(ub\(\))f Fp(with)g(the)g(datat)o(yp)q(e)f(of)h (the)458 106 y(instance)16 b(of)f(this)g(ob)s(ject)g(and)g(returns)g (the)h(upp)q(er)g(b)q(ound.)100 222 y Fh(Datat)o(yp)q(e)g(Building)458 318 y Fo(void)23 b(Contiguous\(OOMPI)p 964 318 V 15 w(Array)p 1099 318 V 17 w(message)f(type,)i(int)f(count)g(=)h(1\).)18 b Fp(Calls)458 374 y Fo(MPI)p 533 374 V 16 w(Type)p 645 374 V 17 w(contiguous\(\))p Fp(.)24 b(The)18 b(argumen)o(t)e(is)i(a)f (p)q(oin)o(ter)h(to)e(an)o(y)h(v)m(alid)i(base)458 431 y(MPI)13 b(datat)o(yp)q(e,)g(whic)o(h)i(will)g(automatically)g(b)q(e)f (promoted)f(to)g Fo(OOMPI)p 1701 431 V 17 w(Array)p 1838 431 V 16 w(-)458 487 y(message)p Fp(.)21 b(The)16 b(user)g(is)g(not)g (required)h(to)e(cast)g(the)h(p)q(oin)o(ter.)23 b(After)15 b(the)h Fo(MPI)p 1837 487 V 17 w(-)458 544 y(Type)p 557 544 V 16 w(contiguous\(\))g Fp(function)i(is)g(called,)h Fo(MPI)p 1321 544 V 17 w(Type)p 1434 544 V 16 w(commit\(\))e Fp(is)g(called)i(so)458 600 y(that)14 b(this)i(instance)g(of)f Fo(OOMPI)p 997 600 V 16 w(Datatype)f Fp(is)i(ready)f(for)g(use.)458 697 y Fo(void)23 b(Vector\(int)g(count,)g(int)g(blocklength,)f(MPI)p 1487 697 V 17 w(Aint)h(stride,)g(OOMPI)p 1934 697 V 17 w(-)458 753 y(Array)p 581 753 V 16 w(message)g(type\).)c Fp(Calls)c Fo(MPI)p 1136 753 V 16 w(Type)p 1248 753 V 17 w(vector\(\))p Fp(.)j(The)c(argumen)o(t)g Fo(type)458 810 y Fp(is)f(a)f(p)q(oin)o(ter)h(to)f(an)o(y)g(v)m(alid)i(base)e(MPI)h (datat)o(yp)q(e,)f(whic)o(h)h(will)h(automatically)f(b)q(e)458 866 y(promoted)f(to)g Fo(OOMPI)p 836 866 V 17 w(Array)p 973 866 V 16 w(message)p Fp(.)18 b(The)13 b(user)g(is)g(not)g(required) h(to)e(cast)g(the)458 923 y(p)q(oin)o(ter.)36 b(After)20 b(the)g Fo(MPI)p 930 923 V 17 w(Type)p 1043 923 V 17 w(vector\(\))f Fp(function)i(is)g(called,)i Fo(MPI)p 1725 923 V 17 w(Type)p 1838 923 V 16 w(-)458 979 y(commit\(\))14 b Fp(is)h(called)i(so)d(that)g(this)i(instance)f(of)g Fo(OOMPI)p 1431 979 V 16 w(Datatype)f Fp(is)i(ready)f(for)458 1036 y(use.)458 1132 y Fo(void)23 b(Hvector\(int)f(count,)h(int)h (blocklength,)e(int)i(stride,)e(OOMPI)p 1845 1132 V 17 w(-)458 1188 y(Array)p 581 1188 V 16 w(message)h(type\).)18 b Fp(Calls)11 b Fo(MPI)p 1131 1188 V 16 w(Type)p 1243 1188 V 17 w(hvector\(\))p Fp(.)17 b(The)10 b(argumen)o(t)g Fo(type)458 1245 y Fp(is)j(a)f(p)q(oin)o(ter)h(to)f(an)o(y)g(v)m(alid)i (base)e(MPI)h(datat)o(yp)q(e,)f(whic)o(h)h(will)h(automatically)f(b)q (e)458 1301 y(promoted)f(to)g Fo(OOMPI)p 836 1301 V 17 w(Array)p 973 1301 V 16 w(message)p Fp(.)18 b(The)13 b(user)g(is)g(not)g(required)h(to)e(cast)g(the)458 1358 y(p)q(oin)o(ter.)28 b(After)17 b(the)h Fo(MPI)p 917 1358 V 17 w(Type)p 1030 1358 V 17 w(hvector\(\))e Fp(function)j(is)f (called,)i Fo(MPI)p 1725 1358 V 17 w(Type)p 1838 1358 V 16 w(-)458 1414 y(commit\(\))14 b Fp(is)h(called)i(so)d(that)g(this)i (instance)f(of)g Fo(OOMPI)p 1431 1414 V 16 w(Datatype)f Fp(is)i(ready)f(for)458 1471 y(use.)458 1567 y Fo(void)23 b(Indexed\(int)f(count,)h(int)h(*blocklengths,)e(int)h(*disps,)g(OOMPI) p 1893 1567 V 17 w(-)458 1624 y(Array)p 581 1624 V 16 w(message)g(type\).)18 b Fp(Calls)11 b Fo(MPI)p 1131 1624 V 16 w(Type)p 1243 1624 V 17 w(indexed\(\))p Fp(.)17 b(The)10 b(argumen)o(t)g Fo(type)458 1680 y Fp(is)j(a)f(p)q(oin)o(ter)h (to)f(an)o(y)g(v)m(alid)i(base)e(MPI)h(datat)o(yp)q(e,)f(whic)o(h)h (will)h(automatically)f(b)q(e)458 1737 y(promoted)f(to)g Fo(OOMPI)p 836 1737 V 17 w(Array)p 973 1737 V 16 w(message)p Fp(.)18 b(The)13 b(user)g(is)g(not)g(required)h(to)e(cast)g(the)458 1793 y(p)q(oin)o(ter.)28 b(After)17 b(the)h Fo(MPI)p 917 1793 V 17 w(Type)p 1030 1793 V 17 w(indexed\(\))e Fp(function)j(is)f(called,)i Fo(MPI)p 1725 1793 V 17 w(Type)p 1838 1793 V 16 w(-)458 1850 y(commit\(\))14 b Fp(is)h(called)i(so)d(that)g(this)i(instance)f(of)g Fo(OOMPI)p 1431 1850 V 16 w(Datatype)f Fp(is)i(ready)f(for)458 1906 y(use.)458 2002 y Fo(void)23 b(Hindexed\(int)f(count,)h(int)h (*blocklengths,)e(MPI)p 1583 2002 V 16 w(Aint)i(*disps,)458 2059 y(OOMPI)p 581 2059 V 16 w(Array)p 717 2059 V 17 w(message)f(type\).)e Fp(Calls)c Fo(MPI)p 1277 2059 V 17 w(Type)p 1390 2059 V 16 w(hindexed\(\))p Fp(.)k(The)16 b(argu-)458 2115 y(men)o(t)22 b Fo(type)g Fp(is)h(a)f(p)q(oin)o(ter)i (to)e(an)o(y)g(v)m(alid)i(base)f(MPI)f(datat)o(yp)q(e,)i(whic)o(h)f (will)458 2172 y(automatically)c(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1203 2172 V 17 w(Array)p 1340 2172 V 16 w(message)p Fp(.)29 b(The)19 b(user)g(is)g(not)458 2228 y(required)f(to)f(cast)g (the)g(p)q(oin)o(ter.)27 b(After)17 b(the)h Fo(MPI)p 1330 2228 V 17 w(Type)p 1443 2228 V 16 w(hindexed\(\))e Fp(function)458 2285 y(is)h(called,)h Fo(MPI)p 723 2285 V 17 w(Type)p 836 2285 V 17 w(commit\(\))d Fp(is)j(called)g(so)e(that)g (this)i(instance)f(of)g Fo(OOMPI)p 1838 2285 V 16 w(-)458 2341 y(Datatype)d Fp(is)h(ready)h(for)e(use.)458 2438 y Fo(OOMPI)p 581 2438 V 16 w(Datatype)23 b(&operator[]\(int)f(count\).) c Fp(Used)c(to)f(pass)g(a)g(coun)o(t)g(in)o(to)458 2494 y(the)i(streams)f(in)o(terface)i(for)e(de\014ning)j(complex)f(data)f (structures.)458 2591 y Fo(void)23 b(Struct)p 724 2591 V 16 w(start\(void)g(*t\).)34 b Fp(Start)19 b(de\014ning)i(a)f(complex) h(data)e(struc-)458 2647 y(ture.)g(The)c Fo(this)f Fp(p)q(oin)o(ter)h (m)o(ust)e(b)q(e)i(passed)g(in)g(from)f(the)g(ob)s(ject's)g (constructor)458 2704 y(so)20 b(that)g(displacemen)o(ts)j(can)e(b)q(e)g (calculated.)38 b(After)21 b(this)g(function,)i(sev)o(eral)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 20 20 19 bop 458 49 a Fp(calls)14 b(to)f Fo(operator<<\(\))f Fp(are)h(made)g(to)g(sp)q(ecify)i(the)f(t)o(yp)q(e,)f(coun)o(t,)g(and)h (o\013set)e(of)458 106 y(the)j(individual)j(mem)o(b)q(ers)e(in)g(the)f (ob)s(ject.)458 204 y Fo(OOMPI)p 581 204 15 2 v 16 w(Datatype)23 b(&operator<<\(OOMPI)p 1220 204 V 15 w(Message)g(d\).)458 302 y(OOMPI)p 581 302 V 16 w(Datatype)g(&operator<<\(OOMPI)p 1220 302 V 15 w(Array)p 1355 302 V 16 w(message)g(d\).)36 b Fp(These)21 b(t)o(w)o(o)458 358 y(functions)i(promote)f(a)h(giv)o(en) g(v)m(ariable/arra)o(y)g(and)g(obtain)g(its)g(t)o(yp)q(e,)h(coun)o(t) 458 415 y(\(from)17 b(an)i(optional)g(call)g(to)f Fo(operator[])p Fp(,)f(and)i(o\013set.)29 b(This)19 b(information)f(is)458 471 y(stored)c(for)h(the)g(\014nal)h(call)h(to)d Fo(Struct)p 1129 471 V 17 w(end\(\))p Fp(.)458 569 y Fo(void)23 b(Struct)p 724 569 V 16 w(end\(void\).)c Fp(Assem)o(ble)14 b(all)h(the)f (information)f(from)g(the)h(previ-)458 625 y(ous)f(calls)h(to)f Fo(operator<<\(\))f Fp(and)h(call)i Fo(MPI)p 1233 625 V 16 w(Type)p 1345 625 V 17 w(struct\(\))d Fp(and)i Fo(MPI)p 1725 625 V 17 w(Type)p 1838 625 V 16 w(-)458 682 y(commit\(\))p Fp(.)100 801 y Fh(See)i(Also)196 b Fo(OOMPI)p 581 801 V 16 w(Message)p Fp(,)14 b Fo(OOMPI)p 912 801 V 17 w(Array)p 1049 801 V 16 w(message)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 21 21 20 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Graph)p 717 49 V 17 w(comm)100 161 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 218 y(class)h(OOMPI_Graph_comm)100 330 y Fh(Description)145 b Fp(A)17 b(class)g(for)f(MPI)h(comm)o (unicators)g(with)g(graph)g(top)q(ology)l(.)25 b(This)18 b(class)f(is)g(de-)458 386 y(riv)o(ed)e(from)g Fo(OOMPI)p 802 386 V 16 w(Intra)p 938 386 V 17 w(comm)p Fp(.)100 498 y Fh(Constructo)o(rs/Destructo)o(rs)458 593 y Fo(OOMPI)p 581 593 V 16 w(Graph)p 717 593 V 17 w(comm\(\).)23 b Fp(Default)17 b(constructor.)23 b(The)16 b(in)o(ternal)i(MPI)e(comm)o (u-)458 649 y(nicator)f(is)h(set)f(to)f Fo(MPI)p 859 649 V 17 w(COMM)p 972 649 V 17 w(NULL)p Fp(.)458 744 y Fo(OOMPI)p 581 744 V 16 w(Graph)p 717 744 V 17 w(comm\(const)22 b(OOMPI)p 1116 744 V 17 w(Graph)p 1253 744 V 16 w(comm&)h(a\).)33 b Fp(Cop)o(y)19 b(constructor.)458 800 y(Mak)o(e)14 b(a)h(shallo)o(w)h (cop)o(y)f(of)g Fo(a)p Fp(.)458 895 y Fo(OOMPI)p 581 895 V 16 w(Graph)p 717 895 V 17 w(comm&)23 b(operator=\(const)f(OOMPI)p 1379 895 V 16 w(Graph)p 1515 895 V 17 w(comm&)h(a\).)k Fp(Cop)o(y)458 952 y(constructor.)19 b(Mak)o(e)14 b(a)h(shallo)o(w)h (assignmen)o(t)f(of)g Fo(a)p Fp(.)458 1046 y Fo(OOMPI)p 581 1046 V 16 w(Graph)p 717 1046 V 17 w(comm\(const)22 b(OOMPI)p 1116 1046 V 17 w(Intra)p 1253 1046 V 16 w(comm&)h(intra)p 1532 1046 V 17 w(comm,)g(int)h(nnodes,)458 1103 y(int)f(index[],)g(int) g(edges[],)g(int)h(reorder)f(=)g(0\).)j Fp(Calls)18 b Fo(MPI)p 1701 1103 V 17 w(Graph)p 1838 1103 V 16 w(-)458 1159 y(create\(\))c Fp(to)g(create)h(a)g(new)h(comm)o(unicator)e(with)i (a)f(graph)g(top)q(ology)l(.)100 1271 y Fh(Graph)h(T)l(op)q(ology)e(F)o (unctions)458 1366 y Fo(int)23 b(Dims)p 652 1366 V 17 w(get\(int&)g(nedges\).)c Fp(Calls)d Fo(MPI)p 1281 1366 V 17 w(Graphdims)p 1514 1366 V 16 w(get\(\))p Fp(.)j(The)d(n)o(um-)458 1422 y(b)q(er)f(of)f(edges)h(is)g(returned)g(in)g Fo(nedges)f Fp(and)h(the)f(function)i(returns)e(the)h(n)o(um)o(b)q(er)458 1479 y(of)f(no)q(des.)458 1573 y Fo(int)23 b(*Get\(int)g(maxindex,)g (int)g(maxedges,)g(int)g(index[],)g(int)g(edges[]\).)458 1630 y Fp(Calls)f Fo(MPI)p 653 1630 V 17 w(Graph)p 790 1630 V 17 w(get\(\))f Fp(to)g(obtain)h(information)g(ab)q(out)g(the)g (graph.)40 b(The)458 1686 y(v)m(alue)16 b(of)f Fo(index)f Fp(is)i(the)f(returned.)458 1781 y Fo(int)23 b(*Neighbors\(int)f (maxneighbors,)g(int)i(neighbors[],)e(int)h(rank)h(=)458 1837 y(Rank\(\)\).)18 b Fp(Calls)13 b Fo(MPI)p 854 1837 V 17 w(Graph)p 991 1837 V 16 w(neighbors\(\))p Fp(.)k(The)c(v)m(alue)g (of)f Fo(neighbors)f Fp(is)i(re-)458 1894 y(turned.)458 1988 y Fo(int)23 b(*Neighbors\(int)f(maxneighbors,)g(int)i (neighbors[],)e(const)h(Port&)458 2045 y(port\);)c Fp(Calls)g Fo(MPI)p 813 2045 V 17 w(Graph)p 950 2045 V 16 w(neighbors\(\))p Fp(.)28 b(The)18 b(v)m(alue)i(of)e Fo(neighbors)f Fp(is)i(re-)458 2101 y(turned.)458 2196 y Fo(int)k(Neighbors)p 772 2196 V 16 w(count\(int)g(rank)g(=)h(Rank\(\)\).)17 b Fp(Calls)11 b Fo(MPI)p 1584 2196 V 17 w(Graph)p 1721 2196 V 16 w(neighbors)p 1953 2196 V 16 w(-)458 2252 y(count\(\))j Fp(and)h(returns)g(the)h(n)o (um)o(b)q(er)f(of)g(neigh)o(b)q(ors.)458 2347 y Fo(int)23 b(Neighbors)p 772 2347 V 16 w(count\(const)f(Port&)i(port\).)17 b Fp(Calls)11 b Fo(MPI)p 1560 2347 V 17 w(Graph)p 1697 2347 V 16 w(neighbors)p 1929 2347 V 16 w(-)458 2403 y(count\(\))j Fp(and)h(returns)g(the)h(n)o(um)o(b)q(er)f(of)g(neigh)o(b)q(ors.)458 2498 y Fo(int)23 b(Map\(int)g(nnodes,)g(int)g(index[],)g(int)h (edges[]\))9 b Fp(Calls)i Fo(MPI)p 1726 2498 V 16 w(Graph)p 1862 2498 V 17 w(-)458 2555 y(map\(\))j Fp(and)i(returns)f(the)g(new)g (rank.)100 2667 y Fh(See)h(Also)196 b Fo(OOMPI)p 581 2667 V 16 w(Comm)p Fp(,)14 b Fo(OOMPI)p 840 2667 V 17 w(Cart)p 953 2667 V 17 w(comm)g(OOMPI)p 1200 2667 V 17 w(Intra)p 1337 2667 V 16 w(comm)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 22 22 21 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Group)100 167 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 223 y(class)h(OOMPI_Group)100 341 y Fh(Description)145 b Fp(A)15 b(class)h(for)e(MPI)h(group)g(managemen)o(t.)100 458 y Fh(Constructo)o(rs/Destructo)o(rs)458 556 y Fo(OOMPI)p 581 556 V 16 w(Group\(\).)20 b Fp(Default)15 b(constructor.)20 b(The)c(in)o(ternal)g(MPI)f(comm)o(unicator)458 612 y(is)g(set)g(to)g Fo(MPI)p 705 612 V 17 w(GROUP)p 842 612 V 16 w(NULL)p Fp(.)458 709 y Fo(OOMPI)p 581 709 V 16 w(Group\(OOMPI)p 861 709 V 16 w(Group&)23 b(a\).)44 b Fp(Returns)24 b(the)g(union)g(of)f Fo(OOMPI)p 1742 709 V 17 w(Group)458 766 y Fp(and)15 b Fo(MPI)p 621 766 V 17 w(GROUP)p 758 766 V 16 w(EMPTY)p Fp(.)458 863 y Fo(OOMPI)p 581 863 V 16 w(Group&)23 b(operator=\(OOMPI)p 1124 863 V 15 w(Group&)g(a\).)44 b Fp(Returns)24 b(the)f(unions)h(of) 458 920 y Fo(OOMPI)p 581 920 V 16 w(Group)15 b Fp(and)g Fo(MPI)p 892 920 V 17 w(GROUP)p 1029 920 V 16 w(EMPTY)p Fp(.)458 1017 y Ff(\030)p Fo(OOMPI)p 616 1017 V 17 w(Group\(\).)k Fp(The)c(destructor)g(deletes)h(a)f(single)h(wrapp)q(er)g(reference)g (to)458 1073 y(the)j(group;)h(if)f(the)g(wrapp)q(er)g(reference)g(p)q (oin)o(ts)g(to)f(the)h(\014nal)h(instance)g(of)e(the)458 1130 y Fo(OOMPI)p 581 1130 V 16 w(Group)p Fp(,)c(then)i(the)f(complete) h Fo(OOMPI)p 1238 1130 V 17 w(Group)e Fp(will)j(b)q(e)f(de-allo)q (cated.)100 1247 y Fh(Op)q(erato)o(rs)172 b Fo(int)23 b(Compare\(OOMPI)p 868 1247 V 16 w(Group&)g(g\).)38 b Fp(P)o(erforms)20 b(a)h(comparison)g(with)h Fo(g)f Fp(and)458 1304 y(returns,)14 b Fo(OOMPI)p 750 1304 V 17 w(SIMILAR)p Fp(,)g Fo(OOMPI)p 1082 1304 V 16 w(EQUAL)p Fp(,)g(or)h Fo(OOMPI)p 1421 1304 V 17 w(UNEQUAL)p Fp(.)458 1401 y Fo(friend)23 b(int)g(operator==\(OOMPI)p 1107 1401 V 15 w(Group&)g(g1,)h(OOMPI)p 1505 1401 V 16 w(Group&)f(g2\).)18 b Fp(This)458 1458 y(op)q(erator)c(returns)h(a)g Fo(TRUE)g Fp(if)g Fo(g1)g Fp(and)h Fo(g2)e Fp(are)h(not)g Fo(MPI)p 1434 1458 V 17 w(UNEQUAL)p Fp(.)458 1555 y Fo(friend)23 b(int)g(operator!=\(OOMPI)p 1107 1555 V 15 w(Group&)g(g1,)h(OOMPI)p 1505 1555 V 16 w(Group&)f(g2\).)i Fp(Re-)458 1611 y(turns)15 b Fo(TRUE)f Fp(if)i Fo(g1)f Fp(and)g Fo(g2)g Fp(are)g Fo(MPI)p 1094 1611 V 17 w(UNEQUAL)p Fp(.)458 1709 y Fo(friend)23 b(OOMPI)p 748 1709 V 16 w(Group&)g(operator|\(OOMPI)p 1291 1709 V 15 w(Group&)g(g1,)h(OOMPI)p 1689 1709 V 16 w(Group)g(&)f(g2\).)458 1765 y Fp(Returns)18 b(a)g(new)h Fo(OOMPI)p 894 1765 V 16 w(Group)f Fp(whose)g(in)o(ternal)h Fo(MPI)p 1428 1765 V 17 w(Group)e Fp(is)i(the)f(union)h(of)458 1822 y(the)c(in)o(ternal)h(MPI)f(groups)g(of)g Fo(g1)g Fp(and)g Fo(g2)p Fp(.)458 1919 y Fo(friend)23 b(OOMPI)p 748 1919 V 16 w(Group&)g(operator&\(OOMPI)p 1291 1919 V 15 w(Group&)g(g1,)h(OOMPI)p 1689 1919 V 16 w(Group)g(&)f(g2\).)458 1975 y Fp(Returns)11 b(a)g(new)h Fo(OOMPI)p 873 1975 V 16 w(Group)f Fp(whose)g(in)o(ternal)h Fo(MPI)p 1386 1975 V 17 w(Group)e Fp(is)i(the)f(in)o(tersection)458 2032 y(of)j(the)i(in)o(ternal)g(MPI)f(groups)g(of)g Fo(g1)f Fp(and)i Fo(g2)p Fp(.)458 2129 y Fo(friend)23 b(OOMPI)p 748 2129 V 16 w(Group&)g(operator-\(OOMPI)p 1291 2129 V 15 w(Group&)g(g1,)h(OOMPI)p 1689 2129 V 16 w(Group)g(&)f(g2\).)458 2186 y Fp(Returns)16 b(a)f(new)h Fo(OOMPI)p 886 2186 V 17 w(Group)f Fp(whose)g(in)o(ternal)i Fo(MPI)p 1413 2186 V 17 w(Group)e Fp(is)h(the)g(di\013erence)458 2242 y(of)e(the)i(in)o(ternal)g(MPI)f(groups)g(of)g Fo(g1)f Fp(and)i Fo(g2)p Fp(.)458 2340 y Fo(OOMPI)p 581 2340 V 16 w(Group&)23 b(operator|=\(OOMPI)p 1148 2340 V 15 w(Group&)g(g\))10 b Fp(Mo)q(di\014es)h(the)f(in)o(ternal)h Fo(MPI)p 1873 2340 V 17 w(Group)458 2396 y Fp(to)j(b)q(e)i(the)f(union) i(of)d(the)i(in)o(ternal)g(MPI)f(groups)g(of)g(this)g(ob)s(ject)g(and)g Fo(g)p Fp(.)458 2493 y Fo(OOMPI)p 581 2493 V 16 w(Group&)23 b(operator&=\(OOMPI)p 1148 2493 V 15 w(Group&)g(g\))10 b Fp(Mo)q(di\014es)h(the)f(in)o(ternal)h Fo(MPI)p 1873 2493 V 17 w(Group)458 2550 y Fp(to)j(b)q(e)i(the)f(in)o(tersection)h (of)f(the)h(in)o(ternal)g(MPI)f(groups)g(of)f(this)i(ob)s(ject)f(and)g Fo(g)p Fp(.)458 2647 y Fo(OOMPI)p 581 2647 V 16 w(Group&)23 b(operator-=\(OOMPI)p 1148 2647 V 15 w(Group&)g(g\))10 b Fp(Mo)q(di\014es)h(the)f(in)o(ternal)h Fo(MPI)p 1873 2647 V 17 w(Group)458 2704 y Fp(to)j(b)q(e)i(the)f(di\013erence)i(of)e (the)g(in)o(ternal)h(MPI)f(groups)g(of)g(this)g(ob)s(ject)g(and)g Fo(g)p Fp(.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 23 23 22 bop 100 49 a Fh(Inquiry)15 b(functions)458 147 y Fo(int)23 b(Size\(void\).)c Fp(This)14 b(function)h(returns)g(the)f (n)o(um)o(b)q(er)g(of)g(pro)q(cesses)h(in)g(the)458 204 y(group.)458 302 y Fo(int)23 b(Rank\(void\).)c Fp(This)d(function)g (returns)f(the)g(rank)g(of)g(the)g(calling)i(pro)q(cess)458 358 y(in)f(the)f(group.)458 456 y Fo(const)23 b(MPI)p 676 456 15 2 v 17 w(Group&)g(Get)p 932 456 V 16 w(mpi\(\).)18 b Fp(This)11 b(function)g(returns)f(the)g(in)o(ternal)h Fo(MPI)p 1842 456 V 17 w(-)458 513 y(Group)p Fp(.)458 610 y Fo(int)23 b(*Translate)p 796 610 V 16 w(rank\(int)g(n,)g(int)h (ranks1[],)e(OOMPI)p 1552 610 V 17 w(Group&)h(g2\).)18 b Fp(This)458 667 y(function)j(is)g(used)g(to)e(determine)j(the)e (relativ)o(e)h(n)o(umering)g(of)f(the)g(same)g(pro-)458 723 y(cesses)f(in)i(t)o(w)o(o)d(di\013eren)o(t)h(groups,)h(as)f(it)g (returns)g(an)h(arra)o(y)e(ranks)h(in)h Fo(group2)458 780 y Fp(corresp)q(onding)c(to)e(ranks)h(in)h Fo(group1)p Fp(.)100 899 y Fh(Inclusion/Exclusion)458 997 y Fo(OOMPI)p 581 997 V 16 w(Group&)23 b(Incl\(int)g(n,)h(int)f(ranks[]\).)d Fp(This)c(function)g(calls)g Fo(MPI)p 1837 997 V 17 w(-)458 1053 y(Group)p 581 1053 V 16 w(incl\(\))f Fp(and)g(returns)g(a)g(new)g Fo(OOMPI)p 1252 1053 V 17 w(Group)p Fp(.)458 1151 y Fo(OOMPI)p 581 1151 V 16 w(Group&)23 b(Excl\(int)g(n,)h(int)f(ranks[]\).)d Fp(This)c(function)g(calls)g Fo(MPI)p 1837 1151 V 17 w(-)458 1207 y(Group)p 581 1207 V 16 w(excl\(\))f Fp(and)g(returns)g(a) g(new)g Fo(OOMPI)p 1252 1207 V 17 w(Group)p Fp(.)458 1305 y Fo(OOMPI)p 581 1305 V 16 w(Group&)23 b(Incl\(int)g(n,)h(int)f (ranges[][3]\).)39 b Fp(This)22 b(function)h(calls)458 1362 y Fo(MPI)p 533 1362 V 16 w(Group)p 669 1362 V 17 w(ranges)p 830 1362 V 16 w(incl\(\))15 b Fp(and)g(returns)g(a)g(new)g Fo(OOMPI)p 1501 1362 V 17 w(Group)p Fp(.)458 1460 y Fo(OOMPI)p 581 1460 V 16 w(Group&)23 b(Excl\(int)g(n,)h(int)f(ranges[][3]\).)39 b Fp(This)22 b(function)h(calls)458 1516 y Fo(MPI)p 533 1516 V 16 w(Group)p 669 1516 V 17 w(ranges)p 830 1516 V 16 w(excl\(\))15 b Fp(and)g(returns)g(a)g(new)g Fo(OOMPI)p 1501 1516 V 17 w(Group)p Fp(.)100 1635 y Fh(See)h(Also)196 b Fo(OOMPI)p 581 1635 V 16 w(Comm)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 24 24 23 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Inter)p 717 49 V 17 w(comm)100 168 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 224 y(class)h(OOMPI_Inter_comm})100 343 y Fh(Description)145 b Fp(A)11 b(class)h(for)f(MPI)g(in)o(ter-comm) o(unicators.)19 b(This)12 b(class)g(is)g(deriv)o(ed)g(from)f Fo(OOMPI)p 1838 343 V 16 w(-)458 400 y(comm)p Fp(.)100 518 y Fh(Constructo)o(rs/Destructo)o(rs)458 616 y Fo(OOMPI)p 581 616 V 16 w(Inter)p 717 616 V 17 w(comm\(\).)23 b Fp(Default)17 b(constructor.)23 b(The)16 b(in)o(ternal)i(MPI)e(comm)o (u-)458 673 y(nicator)f(is)h(set)f(to)f Fo(MPI)p 859 673 V 17 w(COMM)p 972 673 V 17 w(NULL)p Fp(.)458 771 y Fo(OOMPI)p 581 771 V 16 w(Inter)p 717 771 V 17 w(comm\(OOMPI)p 974 771 V 16 w(Intra)p 1110 771 V 16 w(comm&)23 b(local)p 1389 771 V 17 w(comm,)g(int)g(local)p 1764 771 V 17 w(leader,)458 827 y(OOMPI)p 581 827 V 16 w(Intra)p 717 827 V 17 w(comm&)g(peer)p 973 827 V 17 w(comm,)g(int)g(remote)p 1372 827 V 17 w(leader,)f(int)i (tag\).)18 b Fp(Con-)458 884 y(structor.)40 b(Uses)23 b Fo(MPI)p 857 884 V 17 w(Intercomm)p 1090 884 V 16 w(create\(\))e Fp(to)h(construct)g(an)g(MPI)h(in)o(ter-)458 940 y(comm)o(unicator)15 b(b)q(et)o(w)o(een)g Fo(local)p 1053 940 V 17 w(comm)f Fp(and)i Fo(remote)p 1413 940 V 16 w(comm)p Fp(.)458 1038 y Fo(OOMPI)p 581 1038 V 16 w(Inter)p 717 1038 V 17 w(comm\(const)22 b(OOMPI)p 1116 1038 V 17 w(Inter)p 1253 1038 V 16 w(comm&)h(a\).)33 b Fp(Cop)o(y)19 b(constructor.)458 1095 y(Create)14 b(a)h(new)h Fo(OOMPI)p 859 1095 V 16 w(Inter)p 995 1095 V 17 w(comm)e Fp(and)i(p)q(erform)f(a)f(shallo)o(w)i (cop)o(y)f(of)g Fo(a)p Fp(.)458 1192 y Fo(OOMPI)p 581 1192 V 16 w(Inter)p 717 1192 V 17 w(comm&)23 b(operator=\(const)f (OOMPI)p 1379 1192 V 16 w(Inter)p 1515 1192 V 17 w(comm&)h(a\).)18 b Fp(Assign-)458 1249 y(men)o(t)d(op)q(erator.)k(P)o(erforms)14 b(a)h(shallo)o(w)g(cop)o(y)l(.)458 1347 y Fo(virtual)23 b Ff(\030)p Fo(OOMPI)p 807 1347 V 16 w(Inter)p 943 1347 V 17 w(comm\(\).)35 b Fp(Destructor.)f(The)20 b(destructor)g(deletes) 458 1403 y(all)f(of)f(its)g Fo(OOMPI)p 768 1403 V 16 w(Ports)g Fp(then)g(calls)h Fo(MPI)p 1204 1403 V 17 w(Comm)p 1317 1403 V 17 w(free\(\))e Fp(to)h(free)g(the)g(in)o(ternal)458 1460 y(MPI)d(comm)o(unicator.)100 1579 y Fh(Intercommunicato)o(r)f(F)o (unctions)458 1676 y Fo(OOMPI)p 581 1676 V 16 w(Group&)23 b(Remote)p 908 1676 V 17 w(group\(\))11 b Fp(Returns)i(an)g Fo(OOMPI)p 1455 1676 V 16 w(Group)f Fp(that)g(holds)h(the)458 1733 y Fo(MPI)p 533 1733 V 16 w(Group)i Fp(obtained)h(b)o(y)f(calling)i Fo(MPI)p 1150 1733 V 17 w(Comm)p 1263 1733 V 16 w(remote)p 1423 1733 V 17 w(group\(\))p Fp(.)458 1831 y Fo(Remote)p 605 1831 V 16 w(size\(\).)k Fp(Returns)16 b(the)g(result)g(of)g Fo(MPI)p 1314 1831 V 17 w(Comm)p 1427 1831 V 16 w(remote)p 1587 1831 V 17 w(size\(\))f Fp(of)g(the)458 1887 y(remote)f(comm)o (unicator.)458 1985 y Fo(OOMPI)p 581 1985 V 16 w(Intra)p 717 1985 V 17 w(comm&)23 b(Merge\(int)g(high)g(=)h(0\).)18 b Fp(Calls)11 b Fo(MPI)p 1554 1985 V 16 w(Intercomm)p 1786 1985 V 16 w(merge\(\))458 2042 y Fp(to)18 b(create)h(a)g(new)h (MPI)f(comm)o(unicator)g(and)h(returns)f(a)g(reference)h(to)f(a)g(new) 458 2098 y Fo(OOMPI)p 581 2098 V 16 w(Intra)p 717 2098 V 17 w(comm)14 b Fp(constructed)i(from)e(that)h(MPI)g(comm)o(unicator.) 100 2217 y Fh(See)h(Also)196 b Fo(OOMPI)p 581 2217 V 16 w(Intra)p 717 2217 V 17 w(comm)p Fp(,)14 b Fo(OOMPI)p 977 2217 V 16 w(Comm)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 25 25 24 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Intra)p 717 49 V 17 w(comm)100 166 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 222 y(class)h(OOMPI_Intra_comm)100 338 y Fh(Description)145 b Fp(A)11 b(class)g(for)g(MPI)g(in)o(tra-comm) o(unicators.)18 b(This)12 b(class)f(is)h(deriv)o(ed)g(from)f Fo(OOMPI)p 1838 338 V 16 w(-)458 395 y(comm)p Fp(.)100 511 y Fh(Constructo)o(rs/Destructo)o(rs)458 608 y Fo(OOMPI)p 581 608 V 16 w(Intra)p 717 608 V 17 w(comm\(\).)23 b Fp(Default)17 b(constructor.)23 b(The)16 b(in)o(ternal)i(MPI)e(comm)o (u-)458 664 y(nicator)f(is)h(set)f(to)f Fo(MPI)p 859 664 V 17 w(COMM)p 972 664 V 17 w(NULL)p Fp(.)458 761 y Fo(OOMPI)p 581 761 V 16 w(Intra)p 717 761 V 17 w(comm\(const)22 b(OOMPI)p 1116 761 V 17 w(Intra)p 1253 761 V 16 w(comm)i(&a\).)32 b Fp(Cop)o(y)19 b(constructor.)458 818 y(Create)14 b(a)h(new)h Fo(OOMPI)p 859 818 V 16 w(Intra)p 995 818 V 17 w(comm)e Fp(and)i(p)q(erform)f(a)f(shallo)o(w)i(cop)o(y)f(of)g Fo(a)p Fp(.)458 914 y Fo(OOMPI)p 581 914 V 16 w(Intra)p 717 914 V 17 w(comm)23 b(&operator=\(const)f(OOMPI)p 1379 914 V 16 w(Intra)p 1515 914 V 17 w(comm)h(&a\).)18 b Fp(Assign-)458 971 y(men)o(t)d(op)q(erator.)k(P)o(erforms)14 b(a)h(shallo)o(w)g(assignmen)o(t.)458 1068 y Fo(virtual)46 b(OOMPI)p 795 1068 V 17 w(Intra)p 932 1068 V 16 w(comm\(\).)19 b Fp(Destructor.)f(The)12 b(destructor)g(deletes)h(all)458 1124 y(of)g(its)i Fo(OOMPI)p 694 1124 V 16 w(Ports)e Fp(then)i(calls)g Fo(MPI)p 1118 1124 V 17 w(Comm)p 1231 1124 V 16 w(free\(\))e Fp(to)h(free)g(the)g(in)o(ternal)h(MPI)458 1181 y(comm)o(unicator.)100 1297 y Fh(Groups,)h(Contexts,)g(and)g (Communicato)o(rs)458 1394 y Fo(OOMPI)p 581 1394 V 16 w(Intra)p 717 1394 V 17 w(comm&)23 b(Create\(OOMPI)p 1165 1394 V 15 w(Group&)g(group\).)g Fp(Calls)17 b Fo(MPI)p 1725 1394 V 17 w(Comm)p 1838 1394 V 16 w(-)458 1450 y(create\(\))10 b Fp(to)h(obtain)g(a)g(new)g(MPI)h(comm)o(unicator)f(based)g(on)g(the)h (in)o(ternal)g(MPI)458 1507 y(group)19 b(of)f Fo(group)p Fp(.)31 b(A)20 b(reference)f(to)g(a)g(new)g Fo(OOMPI)p 1380 1507 V 17 w(Intra)p 1517 1507 V 16 w(comm)g Fp(constructed)458 1563 y(from)14 b(this)i(MPI)f(comm)o(unicator)g(is)h(returned.)458 1660 y Fo(OOMPI)p 581 1660 V 16 w(Intra)p 717 1660 V 17 w(comm&)23 b(Split\(int)g(color,)g(int)g(key)g(=)h(0\).)18 b Fp(Calls)11 b Fo(MPI)p 1792 1660 V 17 w(Comm)p 1905 1660 V 17 w(-)458 1716 y(split\(\))h Fp(to)h(obtain)h(a)f(new)g(MPI)h (comm)o(unicator)f(based)h(on)f(the)h(in)o(ternal)g(MPI)458 1773 y(group)19 b(of)f Fo(group)p Fp(.)31 b(A)20 b(reference)f(to)g(a)g (new)g Fo(OOMPI)p 1380 1773 V 17 w(Intra)p 1517 1773 V 16 w(comm)g Fp(constructed)458 1829 y(from)14 b(this)i(MPI)f(comm)o (unicator)g(is)h(returned.)100 1945 y Fh(Collective)f(Communication)458 2042 y Fo(void)23 b(Allgather\(OOMPI)p 940 2042 V 15 w(Message&)g(sendbuf,)g(OOMPI)p 1505 2042 V 16 w(Array)p 1641 2042 V 17 w(message&)f(recvbuf,)458 2099 y(int)h(recvcount=1\).)28 b Fp(Calls)20 b Fo(MPI)p 1086 2099 V 16 w(Allgather\(\))p Fp(.)29 b(The)18 b(argumen)o(t)g Fo(sendbuf)458 2155 y Fp(is)j(of)g(an)o(y)f(v)m(alid)j(base)e(MPI)g(datat)o(yp)q(e)f(or)g (an)o(y)h(prop)q(erly)h(constructed)f(user-)458 2212 y(de\014ned)c Fo(OOMPI)f Fp(ob)s(ject.)23 b(The)17 b(argumen)o(t)e Fo(recvbuf)h Fp(is)h(a)f(p)q(oin)o(ter)h(to)e(an)o(y)h(v)m(alid)458 2268 y(base)11 b(MPI)h(datat)o(yp)q(e)f(or)g(a)h(p)q(oin)o(ter)g(to)f (an)o(y)g(prop)q(erly)h(constructed)g(user-de\014ned)458 2325 y Fo(OOMPI)i Fp(ob)s(ject.)21 b(The)15 b(argumen)o(ts)g Fo(sendbuf)f Fp(and)i Fo(recvbuf)e Fp(will)k(automatically)458 2381 y(b)q(e)12 b(promoted)f(to)g Fo(OOMPI)p 892 2381 V 16 w(Message)g Fp(and)h Fo(OOMPI)p 1292 2381 V 16 w(Array)p 1428 2381 V 17 w(message)p Fp(,)e(resp)q(ectiv)o(ely)l(.)458 2437 y(The)15 b(user)g(is)h(not)f(required)h(to)f(p)q(erform)g(a)g (cast.)458 2534 y Fo(void)23 b(Allgather\(OOMPI)p 940 2534 V 15 w(Array)p 1075 2534 V 17 w(message&)f(sendbuf,)h(int)h (sendcount,)e(OOMPI)p 1999 2534 V 17 w(-)458 2591 y(Array)p 581 2591 V 16 w(message&)h(recvbuf,)g(int)g(recvcount=1\).)17 b Fp(Calls)11 b Fo(MPI)p 1632 2591 V 17 w(Allgather\(\))p Fp(.)458 2647 y(The)21 b(argumen)o(ts)g Fo(sendbuf)f Fp(and)i Fo(recvbuf)e Fp(are)h(p)q(oin)o(ters)h(to)f(an)o(y)g(v)m(alid) i(base)458 2704 y(MPI)17 b(datat)o(yp)q(e)f(or)h(prop)q(erly)g (constructed)g(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)h(eac)o(h)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 26 26 25 bop 458 49 a Fp(of)13 b(whic)o(h)i(will)g(automatically)f(b)q(e)h (promoted)e(to)g Fo(OOMPI)p 1447 49 15 2 v 17 w(Array)p 1584 49 V 16 w(message)p Fp(.)19 b(The)458 106 y(user)c(is)h(not)f (required)h(to)e(cast)h(the)g(p)q(oin)o(ters.)458 200 y Fo(void)23 b(Allgatherv\(OOMPI)p 964 200 V 15 w(Message&)g(sendbuf,)g (OOMPI)p 1529 200 V 16 w(Array)p 1665 200 V 17 w(message&)f(recvbuf,) 458 256 y(int)h(recvcount[],)f(int)i(displs[]\).)43 b Fp(Calls)24 b Fo(MPI)p 1439 256 V 17 w(Allgatherv\(\))p Fp(.)43 b(The)458 313 y(argumen)o(t)23 b Fo(sendbuf)g Fp(is)h(of)f(an)o(y)h(v)m(alid)h(base)f(MPI)g(datat)o(yp)q(e)f(or)g(an) o(y)h(prop-)458 369 y(erly)18 b(constructed)h(user-de\014ned)h Fo(OOMPI)d Fp(ob)s(ject.)28 b(The)19 b(argumen)o(t)e Fo(recvbuf)g Fp(is)458 426 y(a)f(p)q(oin)o(ter)h(to)f(an)o(y)g(v)m (alid)i(base)f(MPI)f(datat)o(yp)q(e)g(or)g(a)g(p)q(oin)o(ter)h(to)f(an) o(y)g(prop)q(erly)458 482 y(constructed)22 b(user-de\014ned)h Fo(OOMPI)e Fp(ob)s(ject.)39 b(The)22 b(argumen)o(ts)f Fo(sendbuf)g Fp(and)458 539 y Fo(recvbuf)11 b Fp(will)i(automatically)f (b)q(e)h(promoted)e(to)g Fo(OOMPI)p 1437 539 V 17 w(Message)g Fp(and)h Fo(OOMPI)p 1838 539 V 16 w(-)458 595 y(Array)p 581 595 V 16 w(message)p Fp(,)g(resp)q(ectiv)o(ely)l(.)20 b(The)13 b(user)f(is)h(not)f(required)h(to)f(p)q(erform)g(a)g(cast.)458 689 y Fo(void)23 b(Allgatherv\(OOMPI)p 964 689 V 15 w(Array)p 1099 689 V 17 w(message&)f(sendbuf,)h(int)h(sendcount,)458 746 y(OOMPI)p 581 746 V 16 w(Array)p 717 746 V 17 w(message&)e (recvbuf,)h(int)h(recvcount[],)e(int)h(displs[]\).)458 802 y Fp(Calls)g Fo(MPI)p 654 802 V 17 w(Allgatherv\(\))p Fp(.)40 b(The)23 b(argumen)o(ts)f Fo(sendbuf)g Fp(and)h Fo(recvbuf)e Fp(are)458 859 y(p)q(oin)o(ters)c(to)f(an)o(y)h(v)m(alid)h (base)f(MPI)g(datat)o(yp)q(e)f(or)g(prop)q(erly)i(constructed)f(user-) 458 915 y(de\014ned)h Fo(OOMPI)d Fp(ob)s(ject,)h(eac)o(h)g(of)g(whic)o (h)i(will)g(automatically)f(b)q(e)g(promoted)f(to)458 972 y Fo(OOMPI)p 581 972 V 16 w(Array)p 717 972 V 17 w(message)p Fp(.)j(The)c(user)g(is)h(not)f(required)h(to)f(cast)f(the)i (p)q(oin)o(ters.)458 1066 y Fo(void)23 b(Allreduce\(OOMPI)p 940 1066 V 15 w(Message)g(sendbuf,)g(OOMPI)p 1481 1066 V 16 w(Message)g(recvbuf,)458 1122 y(OOMPI)p 581 1122 V 16 w(Op)h(op\).)35 b Fp(Calls)21 b Fo(MPI)p 991 1122 V 16 w(Allreduce\(\))p Fp(,)f(using)h(the)g(in)o(ternal)g Fo(MPI)p 1757 1122 V 16 w(Op)f Fp(of)458 1179 y Fo(op)p Fp(.)28 b(The)19 b(argumen)o(t)e Fo(sendbuf)g Fp(is)i(of)f(an)o(y)g(v)m (alid)i(base)e(MPI)g(datat)o(yp)q(e)g(or)f(an)o(y)458 1235 y(prop)q(erly)c(constructed)g(user-de\014ned)h Fo(OOMPI)e Fp(ob)s(ject.)18 b(The)13 b(argumen)o(t)f Fo(recvbuf)458 1292 y Fp(is)18 b(a)f(p)q(oin)o(ter)h(to)f(an)o(y)g(v)m(alid)i(base)e (MPI)h(datat)o(yp)q(e)f(or)g(a)g(p)q(oin)o(ter)h(to)f(an)o(y)g(prop-) 458 1348 y(erly)11 b(constructed)g(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject.)18 b(The)12 b(argumen)o(ts)e Fo(sendbuf)g Fp(and)458 1405 y Fo(recvbuf)h Fp(will)i(automatically)f(b)q(e)h (promoted)e(to)g Fo(OOMPI)p 1437 1405 V 17 w(Message)g Fp(and)h Fo(OOMPI)p 1838 1405 V 16 w(-)458 1461 y(Array)p 581 1461 V 16 w(message)p Fp(,)g(resp)q(ectiv)o(ely)l(.)20 b(The)13 b(user)f(is)h(not)f(required)h(to)f(p)q(erform)g(a)g(cast.)458 1555 y Fo(void)23 b(Allreduce\(OOMPI)p 940 1555 V 15 w(Array)p 1075 1555 V 17 w(message)g(sendbuf,)f(OOMPI)p 1617 1555 V 17 w(Array)p 1754 1555 V 16 w(message)458 1612 y(recvbuf,)g(OOMPI)p 795 1612 V 17 w(Op)i(op,)f(int)g(sendcount)g (=)h(1\).)18 b Fp(Calls)11 b Fo(MPI)p 1632 1612 V 17 w(Allreduce\(\))p Fp(,)458 1668 y(using)17 b(the)g(in)o(ternal)g Fo(MPI)p 902 1668 V 17 w(Op)f Fp(of)g Fo(op)p Fp(.)23 b(The)17 b(argumen)o(ts)f Fo(sendbuf)f Fp(and)i Fo(recvbuf)458 1725 y Fp(are)j(p)q(oin)o(ters)h(to)e(an)o(y)h(v)m(alid)i(base)f(MPI)f (datat)o(yp)q(e)g(or)g(prop)q(erly)h(constructed)458 1781 y(user-de\014ned)13 b Fo(OOMPI)f Fp(ob)s(ject,)f(eac)o(h)h(of)g (whic)o(h)h(will)g(automatically)g(b)q(e)f(promoted)458 1838 y(to)i Fo(OOMPI)p 636 1838 V 16 w(Array)p 772 1838 V 17 w(message)p Fp(.)k(The)d(user)g(is)g(not)f(required)i(to)e(cast)g (the)h(p)q(oin)o(ters.)458 1932 y Fo(void)23 b(Alltoall\(OOMPI)p 916 1932 V 15 w(Message&)g(sendbuf,)g(OOMPI)p 1481 1932 V 16 w(Array)p 1617 1932 V 17 w(message&)g(recvbuf,)458 1988 y(int)g(recvcount=1\).)j Fp(Calls)18 b Fo(MPI)p 1082 1988 V 17 w(Alltoall\(\))p Fp(,)f(using)h(the)g(in)o(ternal)h Fo(MPI)p 1814 1988 V 16 w(Op)458 2045 y Fp(of)14 b Fo(op)p Fp(.)19 b(The)c(argumen)o(t)e Fo(sendbuf)h Fp(is)h(of)f(an)o(y)g(v)m (alid)i(base)e(MPI)h(datat)o(yp)q(e)e(or)h(an)o(y)458 2101 y(prop)q(erly)f(constructed)g(user-de\014ned)h Fo(OOMPI)e Fp(ob)s(ject.)18 b(The)13 b(argumen)o(t)f Fo(recvbuf)458 2158 y Fp(is)18 b(a)f(p)q(oin)o(ter)h(to)f(an)o(y)g(v)m(alid)i(base)e (MPI)h(datat)o(yp)q(e)f(or)g(a)g(p)q(oin)o(ter)h(to)f(an)o(y)g(prop-) 458 2214 y(erly)11 b(constructed)g(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject.)18 b(The)12 b(argumen)o(ts)e Fo(sendbuf)g Fp(and)458 2271 y Fo(recvbuf)h Fp(will)i(automatically)f(b)q(e)h (promoted)e(to)g Fo(OOMPI)p 1437 2271 V 17 w(Message)g Fp(and)h Fo(OOMPI)p 1838 2271 V 16 w(-)458 2327 y(Array)p 581 2327 V 16 w(message)p Fp(,)g(resp)q(ectiv)o(ely)l(.)20 b(The)13 b(user)f(is)h(not)f(required)h(to)f(p)q(erform)g(a)g(cast.)458 2421 y Fo(void)23 b(Alltoall\(OOMPI)p 916 2421 V 15 w(Array)p 1051 2421 V 17 w(message&)g(sendbuf,)f(int)i(sendcount,)e(OOMPI)p 1975 2421 V 17 w(-)458 2478 y(Array)p 581 2478 V 16 w(message&)h (recvbuf,)g(int)g(recvcount=1\).)17 b Fp(Calls)11 b Fo(MPI)p 1632 2478 V 17 w(Alltoall\(\))p Fp(.)458 2534 y(The)21 b(argumen)o(ts)g Fo(sendbuf)f Fp(and)i Fo(recvbuf)e Fp(are)h(p)q(oin)o (ters)h(to)f(an)o(y)g(v)m(alid)i(base)458 2591 y(MPI)17 b(datat)o(yp)q(e)f(or)h(prop)q(erly)g(constructed)g(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)h(eac)o(h)458 2647 y(of)c(whic)o(h)i(will)g (automatically)f(b)q(e)h(promoted)e(to)g Fo(OOMPI)p 1447 2647 V 17 w(Array)p 1584 2647 V 16 w(message)p Fp(.)19 b(The)458 2704 y(user)c(is)h(not)f(required)h(to)e(cast)h(the)g(p)q (oin)o(ters.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 27 27 26 bop 458 49 a Fo(void)23 b(Alltoallv\(OOMPI)p 940 49 15 2 v 15 w(Array)p 1075 49 V 17 w(message&)f(sendbuf,)h(int)h (sendcounts[],)458 106 y(int)f(sdispls[],)g(OOMPI)p 939 106 V 16 w(Array)p 1075 106 V 17 w(message&)f(recvbuf,)h(int)h (recvcounts[],)458 162 y(int)f(rdispls[]\).)39 b Fp(Calls)22 b Fo(MPI)p 1051 162 V 17 w(Alltoallv\(\))p Fp(.)38 b(The)22 b(argumen)o(ts)f Fo(sendbuf)458 219 y Fp(and)c Fo(recvbuf)g Fp(are)g(p)q(oin)o(ters)h(to)f(an)o(y)h(v)m(alid)h(base)e(MPI)h(datat)o (yp)q(e)f(or)g(prop)q(erly)458 275 y(constructed)d(user-de\014ned)i Fo(OOMPI)e Fp(ob)s(ject,)f(eac)o(h)i(of)e(whic)o(h)j(will)g (automatically)458 332 y(b)q(e)f(promoted)f(to)g Fo(OOMPI)p 901 332 V 16 w(Array)p 1037 332 V 17 w(message)p Fp(.)k(The)d(user)g (is)g(not)f(required)i(to)d(cast)458 388 y(the)i(p)q(oin)o(ters.)458 486 y Fo(void)23 b(Barrier\(void\).)18 b Fp(Calls)e Fo(MPI)p 1120 486 V 17 w(Barrier\(\))p Fp(.)458 584 y Fo(void)23 b(Reduce)p 724 584 V 16 w(scatter\(OOMPI)p 1052 584 V 16 w(Message)g(sendbuf,)f(OOMPI)p 1593 584 V 17 w(Array)p 1730 584 V 16 w(message)458 640 y(recvbuf,)g(int)i(recvcounts[],)e (OOMPI)p 1225 640 V 17 w(Op)h(op\).)18 b Fp(Calls)11 b Fo(MPI)p 1608 640 V 17 w(Reduce)p 1769 640 V 16 w(scatter\(\))p Fp(.)458 697 y(The)21 b(argumen)o(ts)g Fo(sendbuf)f Fp(and)i Fo(recvbuf)e Fp(are)h(p)q(oin)o(ters)h(to)f(an)o(y)g(v)m(alid)i(base) 458 753 y(MPI)17 b(datat)o(yp)q(e)f(or)h(prop)q(erly)g(constructed)g (user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)h(eac)o(h)458 810 y(of)c(whic)o(h)i(will)g(automatically)f(b)q(e)h(promoted)e(to)g Fo(OOMPI)p 1447 810 V 17 w(Array)p 1584 810 V 16 w(message)p Fp(.)19 b(The)458 866 y(user)c(is)h(not)f(required)h(to)e(cast)h(the)g (p)q(oin)o(ters.)458 964 y Fo(void)23 b(Scan\(OOMPI)p 820 964 V 16 w(Message)g(sendbuf,)g(OOMPI)p 1362 964 V 16 w(Array)p 1498 964 V 17 w(message)f(recvbuf,)458 1021 y(OOMPI)p 581 1021 V 16 w(Op)i(op\).)36 b Fp(Calls)22 b Fo(MPI)p 993 1021 V 17 w(Scan\(\))p Fp(.)36 b(The)21 b(argumen)o(t)f Fo(sendbuf)g Fp(is)h(of)f(an)o(y)458 1077 y(v)m(alid)13 b(base)f(MPI)g(datat)o(yp)q(e)f(or)h(an)o(y)f(prop)q (erly)i(constructed)f(user-de\014ned)i Fo(OOMPI)458 1134 y Fp(ob)s(ject.)42 b(The)24 b(argumen)o(t)e Fo(recvbuf)g Fp(is)h(a)g(p)q(oin)o(ter)g(to)g(an)o(y)f(v)m(alid)j(base)e(MPI)458 1190 y(datat)o(yp)q(e)18 b(or)h(a)g(p)q(oin)o(ter)g(to)g(an)o(y)g(prop) q(erly)h(constructed)f(user-de\014ned)i Fo(OOMPI)458 1246 y Fp(ob)s(ject.)d(The)13 b(argumen)o(ts)f Fo(sendbuf)g Fp(and)h Fo(recvbuf)e Fp(will)k(automatically)e(b)q(e)g(pro-)458 1303 y(moted)j(to)f Fo(OOMPI)p 777 1303 V 17 w(Message)g Fp(and)h Fo(OOMPI)p 1186 1303 V 17 w(Array)p 1323 1303 V 16 w(message)p Fp(,)f(resp)q(ectiv)o(ely)l(.)25 b(The)458 1359 y(user)15 b(is)h(not)f(required)h(to)e(p)q(erform)h(a)g(cast.)458 1457 y Fo(void)23 b(Scan\(OOMPI)p 820 1457 V 16 w(Array)p 956 1457 V 16 w(message)g(sendbuf,)g(OOMPI)p 1498 1457 V 17 w(Array)p 1635 1457 V 16 w(message)g(recvbuf,)458 1514 y(OOMPI)p 581 1514 V 16 w(Op)h(op,)f(int)h(count,)f(=)g(1\).)33 b Fp(Calls)20 b Fo(MPI)p 1369 1514 V 17 w(Scan\(\))p Fp(.)31 b(The)20 b(argumen)o(ts)458 1570 y Fo(sendbuf)i Fp(and)i Fo(recvbuf)e Fp(are)h(p)q(oin)o(ters)h(to)f(an)o(y)g(v)m(alid) i(base)e(MPI)g(datat)o(yp)q(e)458 1627 y(or)c(prop)q(erly)i (constructed)e(user-de\014ned)j Fo(OOMPI)d Fp(ob)s(ject,)h(eac)o(h)f (of)h(whic)o(h)g(will)458 1683 y(automatically)f(b)q(e)g(promoted)f(to) g Fo(OOMPI)p 1203 1683 V 17 w(Array)p 1340 1683 V 16 w(message)p Fp(.)29 b(The)19 b(user)g(is)g(not)458 1740 y(required)d(to)f(cast)f(the)i(p)q(oin)o(ters.)100 1858 y Fh(General)f(T)l(op)q(ology)g(F)o(unction)458 1956 y Fo(int)23 b(Topo)p 652 1956 V 17 w(test\(\).)c Fp(Returns)d(the)f (result)h(of)e Fo(MPI)p 1357 1956 V 17 w(Topo)p 1470 1956 V 17 w(test\(\).)458 2054 y(int)23 b(*Dims)p 676 2054 V 17 w(create\(int)f(ndims,)h(int)h(dims[],)f(int)g(nnodes)g(=)h (0\).)18 b Fp(Call)458 2111 y Fo(MPI)p 533 2111 V 16 w(Dims)p 645 2111 V 17 w(create\(\))p Fp(.)h(Returns)d(the)f (dimensions)i(arra)o(y)l(.)100 2229 y Fh(See)f(Also)196 b Fo(OOMPI)p 581 2229 V 16 w(Comm)p Fp(,)22 b Fo(OOMPI)p 848 2229 V 16 w(Inter)p 984 2229 V 17 w(Comm)p Fp(,)f Fo(OOMPI)p 1251 2229 V 17 w(Cart)p 1364 2229 V 16 w(comm)g(OOMPI)p 1617 2229 V 16 w(Graph)p 1753 2229 V 17 w(comm)p Fp(,)458 2286 y Fo(OOMPI)p 581 2286 V 16 w(Message)p Fp(,)14 b Fo(OOMPI)p 912 2286 V 17 w(Array)p 1049 2286 V 16 w(message)p Fp(,)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 28 28 28 27 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Op)100 168 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 224 y(class)h(OOMPI_Op)100 343 y Fh(Description)145 b Fp(An)14 b Fo(OOMPI)p 654 343 V 16 w(Op)f Fp(ob)s(ject)g(has)h(four)f (main)h(functions:)20 b(construction,)14 b(cop)o(ying,)g(use)458 400 y(as)h(an)g(argumen)o(t)f(to)h Fo(MPI)p 911 400 V 16 w(Reduce\(\))p Fp(,)f(and)h(destruction.)100 518 y Fh(Constructo)o(rs/Destructo)o(rs)458 616 y Fo(OOMPI)p 581 616 V 16 w(Op\(MPI)p 741 616 V 17 w(User)p 854 616 V 16 w(function)23 b(*function,)f(int)i(commutative)e(=)i(TRUE\).)458 673 y Fp(Calls)16 b Fo(MPI)p 647 673 V 16 w(Op)p 711 673 V 17 w(create\(\))e Fp(to)h(create)g(an)g Fo(MPI)p 1259 673 V 17 w(Op)g Fp(handle.)458 771 y Fo(OOMPI)p 581 771 V 16 w(Op\(const)23 b(OOMPI)p 932 771 V 16 w(Op)h(&a\).)f Fp(The)17 b(cop)o(y)f(constructor)g(will)i(create)e(and)458 827 y Fo(OOMPI)p 581 827 V 16 w(Op)g Fp(ob)s(ject)f(that)g(p)q(oin)o (ts)i(to)e(the)h(same)f(instance)i(of)e(the)h Fo(MPI)p 1666 827 V 17 w(Op)g Fp(handle)458 884 y(that)e(the)h Fo(a)h Fp(refers)e(to.)458 982 y Fo(operator=\(const)22 b(OOMPI)p 963 982 V 16 w(Op)i(&a\).)37 b Fp(The)21 b(assigmen)o(t)g(op) q(erator)f(mak)o(es)h(an)458 1038 y(additional)16 b(reference)g(to)f Fo(a)p Fp('s)f Fo(OOMPI)p 1111 1038 V 17 w(Op)h Fp(handle.)458 1136 y Ff(\030)p Fo(OOMPI)p 616 1136 V 17 w(Op\(\).)20 b Fp(The)15 b(destructor)g(deletes)i(the)e(reference)h(to)f(the)h Fo(MPI)p 1710 1136 V 17 w(Op)f Fp(han-)458 1192 y(dle.)24 b(This)17 b(will)g(in)o(v)o(ok)o(e)g Fo(MPI)p 960 1192 V 16 w(Op)p 1024 1192 V 17 w(free\(\))f Fp(only)h(if)f(it)h(w)o(as)e (the)h(last)h(reference)g(to)458 1249 y(the)e(handle.)100 1368 y Fh(See)h(Also)196 b Fo(OOMPI)p 581 1368 V 16 w(Comm)p Fp(,)14 b Fo(OOMPI)p 840 1368 V 17 w(Intra)p 977 1368 V 16 w(Comm)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 29 29 29 28 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Packed)100 159 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 216 y(class)h(OOMPI_Packed)100 326 y Fh(Description)145 b Fp(The)12 b Fo(OOMPI)p 671 326 V 16 w(Packed)g Fp(ob)s(ject)f(is)i(used)f(to)f(pro)o(vide)i(a)f (streams-based)f(in)o(terface)i(to)458 382 y(the)i(MPI-1)g(pac)o(king)h (and)f(unpac)o(king)h(functions.)21 b(Note)15 b(that)f(access)i(to)e Fo(MPI)p 1837 382 V 17 w(-)458 439 y(Pack)p 557 439 V 16 w(size\(\))f Fp(is)h Fg(not)f Fp(pro)o(vided)h(through)f(this)g(ob)s (ject.)19 b(Instead,)14 b(it)f(is)h(pro)o(vided)458 495 y(through)h(the)g Fo(OOMPI)p 831 495 V 16 w(Comm)g Fp(ob)s(ject.)458 671 y Fj(Discussion)o(:)20 b Fi(Since)d Fc(MPI)p 891 671 14 2 v 15 w(Pack)p 994 671 V 15 w(size\(\))d Fi(is)i(used)h(to)e (determine)h(ho)o(w)g(large)f(a)h(bu\013er)458 728 y(is)f(necessary)i (to)e(creat)h(in)e(order)i(to)f(pac)o(k)g(or)g(unpac)o(k)h(an)e (message,)h(it)g(is)g(p)q(oin)o(tless)g(to)458 784 y(create)e(an)f Fc(OOMPI)p 747 784 V 14 w(Packed)f Fi(ob)r(ject)h(with)g(a)f(bu\013er)i (only)e(to)h(determine)f(what)h(the)g(\\real")458 841 y(size)18 b(of)f(the)h(bu\013er)h(should)e(b)q(e.)30 b(Therefore,)19 b Fc(MPI)p 1281 841 V 15 w(Pack)p 1384 841 V 15 w(size\(\))d Fi(is)i(encapsulated)g(in)458 897 y(all)c(comm)o(unicators)f(so)i(that)h(the)g(prop)q(er)g(bu\013er)g (size)g(can)g(b)q(e)g(determined)f Fa(b)n(efor)n(e)g Fi(an)458 954 y Fc(OOMPI)p 571 954 V 14 w(Packed)e Fi(ob)r(ject)i(is)e (created.)100 1146 y Fh(Constructo)o(rs/Destructo)o(rs)458 1240 y Fo(OOMPI)p 581 1240 15 2 v 16 w(Packed\(int)23 b(size,)g(OOMPI)p 1123 1240 V 16 w(Comm)h(&c,)f(int)h(tag)f(=)h(OOMPI)p 1713 1240 V 16 w(PACKED)p 1873 1240 V 17 w(-)458 1296 y(TAG\).)14 b Fp(Constructor,)g(in)i(whic)o(h)g(a)f(bu\013er)g(is)h (allo)q(cated)g(in)o(ternally)l(.)458 1390 y Fo(OOMPI)p 581 1390 V 16 w(Packed\(void)23 b(*ptr,)g(int)g(size,)g(OOMPI)p 1385 1390 V 17 w(Comm)g(&c,)h(int)f(tag)h(=)f(OOMPI)p 1975 1390 V 17 w(-)458 1447 y(PACKED)p 605 1447 V 16 w(TAG\).)15 b Fp(Constructor,)e(in)j(whic)o(h)g(a)f(bu\013er)g(is)h (pro)o(vided)g(b)o(y)f(the)g(caller.)458 1540 y Fo(OOMPI)p 581 1540 V 16 w(Packed\(const)22 b(OOMPI)p 1027 1540 V 17 w(Packed)h(&a\).)36 b Fp(The)20 b(cop)o(y)h(constructor)e(p)q(er-) 458 1597 y(forms)14 b(a)h(deep)h(cop)o(y)f(of)g(the)g Fo(a)g Fp(ob)s(ject.)458 1690 y Fo(OOMPI)p 581 1690 V 16 w(Packed)23 b(&operator=\(const)f(OOMPI)p 1290 1690 V 16 w(Packed)h(&a\).)18 b Fp(The)11 b(assignmen)o(t)458 1747 y(op)q(erator)18 b(p)q(erforms)g(a)h(deep)h(cop)o(y)e(of)h(the)g Fo(a)f Fp(ob)s(ject.)31 b(The)19 b(original)h(bu\013er)e(is)458 1803 y(only)f(deleted)i(if)e(the)h Fo(OOMPI)p 965 1803 V 16 w(Packed)e Fp(ob)s(ject)h(initially)j(created)d(it;)h(if)g(the)f (user)458 1860 y(sp)q(eci\014ed)g(the)e(bu\013er)h(in)g(the)f Fo(OOMPI)p 1104 1860 V 16 w(Packed)g Fp(constructor,)f(it)h(is)h(not)f (deleted.)458 1953 y Ff(\030)p Fo(OOMPI)p 616 1953 V 17 w(Packed\(\).)i Fp(The)11 b(destructor)f(deletes)i(the)e(bu\013er)h (only)g(if)g(the)g Fo(OOMPI)p 1838 1953 V 16 w(-)458 2010 y(Packed)k Fp(ob)s(ject)h(created)h(the)g(bu\013er;)f(if)h(the)g (user)g(sp)q(eci\014ed)h(the)f(bu\013er)f(in)i(the)458 2066 y Fo(OOMPI)p 581 2066 V 16 w(Packed)d Fp(constructor,)f(it)h(is)h (not)f(deleted.)458 2160 y Fo(int)23 b(Get)p 628 2160 V 17 w(size\(void\).)18 b Fp(Returns)d(the)f(size)h(of)f(the)g (bu\013er)g(a)o(v)m(ailable)i(for)d(pac)o(k-)458 2216 y(ing)i(and)h(unpac)o(king.)458 2310 y Fo(void)23 b(Set)p 652 2310 V 17 w(size\(int)g(size\).)31 b Fp(Allo)o(ws)19 b(the)h(user)f(to)g(expand)g(or)g(shrink)h(the)458 2366 y(bu\013er)15 b(used)h(for)e(pac)o(king)i(and)f(unpac)o(king.)458 2460 y Fo(int)23 b(Get)p 628 2460 V 17 w(position\(void\).)16 b Fp(Returns)11 b(the)f(curren)o(t)g(p)q(osition)i(of)d(the)h(pac)o (k/unpac)o(k.)458 2554 y Fo(void)23 b(Set)p 652 2554 V 17 w(position\(int)f(size\).)c Fp(Sets)10 b(the)g(curren)o(t)g(p)q (osition)h(of)f(the)g(pac)o(k/unpac)o(k.)458 2647 y Fo(void)23 b(Reset\(void\).)k Fp(Reset)18 b(the)g(state)g(of)f(the)h(curren)o(t)g (pac)o(k/unpac)o(k)g(bac)o(k)458 2704 y(to)c(the)i(b)q(eginning)h(of)e (the)g(bu\013er)g(and)h(zeros)f(out)f(the)i(coun)o(t.)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 30 30 30 29 bop 458 49 a Fo(void)23 b(Pack)p 676 49 15 2 v 17 w(start\(int)f(position)h(=)h(0\).)e Fp(Resets)16 b(the)g(state)f(and)h(prepares)458 106 y(for)9 b(rep)q(eated)i(calls)g (to)f Fo(operator<<\(\))e Fp(to)i(pac)o(k)g(in)o(to)g(the)g(bu\013er)g (or)g Fo(operator>>\(\))458 162 y Fp(to)k(unpac)o(k)i(from)e(the)i (bu\013er.)458 260 y Fo(OOMPI)p 581 260 V 16 w(Packed)23 b(&operator[]\(int)f(incount\).)38 b Fp(Sp)q(ecify)23 b(a)e(coun)o(t)g(for)g(the)458 317 y(pac)o(k)13 b(or)h(unpac)o(k)g(op)q (eration.)20 b(The)14 b(coun)o(t)g(only)g(applies)h(to)f(the)g Fo(operator>>\(\))458 373 y Fp(or)e Fo(operator<<\(\))f Fp(that)h(immediately)j(follo)o(ws)e(the)g(call)h(to)e Fo(operator[])f Fp(when)458 429 y(applied)17 b(to)d(an)i Fo(OOMPI)p 859 429 V 16 w(Array)p 995 429 V 17 w(message)e Fp(ob)s(ject.)458 527 y Fo(OOMPI)p 581 527 V 16 w(Packed)23 b(&operator<<\(OOMPI)p 1172 527 V 15 w(Message)g(&data\).)458 625 y(OOMPI)p 581 625 V 16 w(Packed)g(&operator<<\(OOMPI)p 1172 625 V 15 w(Array)p 1307 625 V 17 w(message)g(&data\).)e Fp(P)o(ac)o(k)15 b(the)458 682 y(sp)q(eci\014ed)h(ob)s(ject)e(in)o(to)h (the)f(bu\013er)h(using)g Fo(MPI)p 1269 682 V 17 w(Pack\(\))p Fp(.)j(A)d(coun)o(t)f(can)h(b)q(e)g(sp)q(ec-)458 738 y(i\014ed)23 b(with)g(an)f(optional)g(call)h(to)f Fo(operator[])p Fp(,)g(otherwise)g(a)g(coun)o(t)g(of)g(1)f(is)458 795 y(assumed.)f(The)15 b(coun)o(t)g(only)h(applies)h(to)d Fo(OOMPI)p 1306 795 V 17 w(Array)p 1443 795 V 16 w(message)g Fp(ob)s(jects.)458 893 y Fo(OOMPI)p 581 893 V 16 w(Packed)23 b(&operator>>\(OOMPI)p 1172 893 V 15 w(Message)g(&data\).)458 991 y(OOMPI)p 581 991 V 16 w(Packed)g(&operator>>\(OOMPI)p 1172 991 V 15 w(Array)p 1307 991 V 17 w(message)g(&data\).)45 b Fp(Unpac)o(k)458 1047 y(the)17 b(sp)q(eci\014ed)j(ob)s(ject)d(from)g (the)g(bu\013er)h(using)g Fo(MPI)p 1383 1047 V 17 w(Unpack\(\))p Fp(.)26 b(A)17 b(coun)o(t)h(can)458 1104 y(b)q(e)c(sp)q(eci\014ed)i (with)e(an)g(optional)g(call)h(to)f Fo(operator[])p Fp(,)e(otherwise)i (a)f(coun)o(t)h(of)f(1)458 1160 y(is)i(assumed.)20 b(The)c(coun)o(t)f (only)h(applies)g(to)f Fo(OOMPI)p 1352 1160 V 16 w(Array)p 1488 1160 V 17 w(message)f Fp(ob)s(jects.)458 1258 y Fo(void)23 b(Pack)p 676 1258 V 17 w(end\(void\).)f Fp(P)o(erform)15 b(an)o(y)h(necessary)g(clean)o(up)i(functions.)24 b(Not)458 1315 y(curren)o(tly)e(necessary)l(,)i(but)f(useful)g(in)g(terms)f(of)g (symmetry)f(and)h(future)h(ex-)458 1371 y(pansion.)33 b Fo(Pack)p 754 1371 V 17 w(end\(\))19 b Fp(should)h(alw)o(a)o(ys)f(b)q (e)h(called)i(when)e(a)f(pac)o(k/unpac)o(k)g(is)458 1427 y(completed.)100 1546 y Fh(See)d(Also)196 b Fo(OOMPI)p 581 1546 V 16 w(Comm)p Fp(,)14 b Fo(OOMPI)p 840 1546 V 17 w(Intra)p 977 1546 V 16 w(Comm)p Fp(,)h Fo(OOMPI)p 1237 1546 V 16 w(Port)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 31 31 31 30 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Port)100 166 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 222 y(class)h(OOMPI_Port)100 338 y Fh(Description)145 b Fp(A)15 b(class)h(for)e(managing)h(p)q(oin)o(t)h(to)e(p)q(oin)o(t)i (and)f(ro)q(oted)g(collectiv)o(e)i(op)q(erations.)100 455 y Fh(Constructo)o(rs/Destructo)o(rs)458 551 y Fo(OOMPI)p 581 551 V 16 w(Port\(\).)i Fp(Default)d(constructor.)458 648 y Fo(OOMPI)p 581 648 V 16 w(Port\(const)23 b(OOMPI)p 980 648 V 16 w(Port&)g(a\).)41 b Fp(Cop)o(y)21 b(constructor.)39 b(P)o(erforms)21 b(a)458 705 y(shallo)o(w)15 b(cop)o(y)g(of)g(the)g(in) o(ternal)h(state)e(of)h(the)g(p)q(ort)g(\(the)g(asso)q(ciated)g(MPI)g (com-)458 761 y(m)o(unicator\).)458 858 y Fo(OOMPI)p 581 858 V 16 w(Port&)23 b(operator=\(const)f(OOMPI)p 1242 858 V 17 w(Port&)h(a\).)30 b Fp(Assignmen)o(t)19 b(op)q(era-)458 914 y(tor.)g(P)o(erforms)13 b(a)i(shallo)o(w)g(cop)o(y) g(of)g(the)g(in)o(ternal)h(state)e(of)g(the)h(p)q(ort)g(\(the)g(asso-) 458 971 y(ciated)h(MPI)f(comm)o(unicator\).)458 1068 y Ff(\030)p Fo(OOMPI)p 616 1068 V 17 w(Port\(\).)k Fp(Destructor.)100 1184 y Fh(Utilit)o(y)242 b Fo(int)23 b(Rank\(void\).)c Fp(Returns)d(the)f(rank)g(of)g(the)g(p)q(ort)g(in)h(its)f(comm)o (unicator.)100 1300 y Fh(Ro)q(oted)h(Collective)g(Op)q(erations)458 1397 y Fo(void)23 b(Bcast\(OOMPI)p 844 1397 V 16 w(Message)g(buf\).)e Fp(Calls)16 b Fo(MPI)p 1378 1397 V 17 w(Bcast\(\))p Fp(.)k(The)c (argumen)o(t)458 1454 y Fo(buf)c Fp(is)i(of)f(an)o(y)g(v)m(alid)i(base) e(MPI)g(datat)o(yp)q(e)g(or)f(an)o(y)h(prop)q(erly)h(constructed)g (user-)458 1510 y(de\014ned)h Fo(OOMPI)f Fp(ob)s(ject,)f(whic)o(h)i (will)h(automatically)f(b)q(e)f(promoted)g(to)g Fo(OOMPI)p 1838 1510 V 16 w(-)458 1566 y(Message)p Fp(.)19 b(The)c(user)g(is)h (not)f(required)h(to)f(p)q(erform)g(a)g(cast.)458 1663 y Fo(void)23 b(Bcast\(OOMPI)p 844 1663 V 16 w(Array)p 980 1663 V 16 w(message)g(buf,)g(int)h(count=)f(1\).)35 b Fp(The)20 b(argu-)458 1720 y(men)o(t)c Fo(buf)g Fp(is)i(a)e(p)q(oin)o (ter)h(to)f(an)o(y)h(v)m(alid)h(base)f(MPI)f(datat)o(yp)q(e)h(or)f(an)o (y)g(prop)q(erly)458 1776 y(constructed)e(user-de\014ned)h Fo(OOMPI)e Fp(ob)s(ject,)g(whic)o(h)i(will)g(automatically)g(b)q(e)f (pro-)458 1833 y(moted)h(to)f Fo(OOMPI)p 775 1833 V 17 w(Message)p Fp(.)19 b(The)c(user)g(is)h(not)f(required)h(to)f(p)q (erform)g(a)g(cast.)458 1929 y Fo(void)23 b(Gather\(OOMPI)p 868 1929 V 16 w(Message)f(sendbuf,)h(OOMPI)p 1409 1929 V 17 w(Array)p 1546 1929 V 16 w(message)g(recvbuf,)458 1986 y(int)g(recvcount=)g(1\).)29 b Fp(Calls)19 b Fo(MPI)p 1109 1986 V 17 w(Gather\(\))p Fp(.)28 b(The)19 b(argumen)o(t)e Fo(sendbuf)g Fp(is)458 2042 y(of)11 b(an)o(y)h(v)m(alid)h(base)f(MPI)g (datat)o(yp)q(e)f(or)h(an)o(y)f(prop)q(erly)i(constructed)f (user-de\014ned)458 2099 y Fo(OOMPI)h Fp(ob)s(ject.)19 b(The)c(argumen)o(t)f Fo(rcvbuf)f Fp(is)i(a)f(p)q(oin)o(ter)h(to)f(an)o (y)g(v)m(alid)i(base)e(MPI)458 2155 y(datat)o(yp)q(e)k(or)h(a)g(p)q (oin)o(ter)g(to)g(an)o(y)g(prop)q(erly)h(constructed)f(user-de\014ned)i Fo(OOMPI)458 2212 y Fp(ob)s(ject.)d(The)13 b(argumen)o(ts)f Fo(sendbuf)g Fp(and)h Fo(recvbuf)e Fp(will)k(automatically)e(b)q(e)g (pro-)458 2268 y(moted)j(to)f Fo(OOMPI)p 777 2268 V 17 w(Message)g Fp(and)h Fo(OOMPI)p 1186 2268 V 17 w(Array)p 1323 2268 V 16 w(message)p Fp(,)f(resp)q(ectiv)o(ely)l(.)25 b(The)458 2325 y(user)15 b(is)h(not)f(required)h(to)e(p)q(erform)h(a)g (cast.)458 2421 y Fo(void)23 b(Gather\(OOMPI)p 868 2421 V 16 w(Array)p 1004 2421 V 16 w(message)g(sendbuf,)g(int)g(sendcount,)g (OOMPI)p 1904 2421 V 16 w(-)458 2478 y(Array)p 581 2478 V 16 w(message)g(recvbuf,)g(int)g(recvcount=)g(1\).)34 b Fp(Calls)21 b Fo(MPI)p 1658 2478 V 16 w(Gather\(\))p Fp(.)458 2534 y(The)13 b(argumen)o(ts)g Fo(sendbuf)g Fp(and)g Fo(rcvbuf)g Fp(are)g(p)q(oin)o(ters)h(to)f(an)o(y)g(v)m(alid)i (base)f(MPI)458 2591 y(datat)o(yp)q(e)23 b(or)f(prop)q(erly)j (constructed)e(user-de\014ned)i Fo(OOMPI)e Fp(ob)s(ject,)h(eac)o(h)f (of)458 2647 y(whic)o(h)d(will)i(automatically)e(b)q(e)h(promoted)e(to) g Fo(OOMPI)p 1433 2647 V 17 w(Array)p 1570 2647 V 16 w(message)p Fp(.)33 b(The)458 2704 y(user)15 b(is)h(not)f(required)h (to)e(cast)h(the)g(p)q(oin)o(ters.)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 32 32 32 31 bop 458 49 a Fo(void)23 b(Gatherv\(OOMPI)p 892 49 15 2 v 15 w(Message)g(sendbuf,)g(OOMPI)p 1433 49 V 17 w(Array)p 1570 49 V 16 w(message)g(recvbuf,)458 106 y(int)g(recvcounts[],)f(int)i(displs[]\).)31 b Fp(Calls)20 b Fo(MPI)p 1447 106 V 17 w(Gatherv\(\))p Fp(.)32 b(The)19 b(ar-)458 162 y(gumen)o(t)13 b Fo(sendbuf)f Fp(is)i(of)e(an)o(y)h(v)m (alid)i(base)e(MPI)g(datat)o(yp)q(e)g(or)g(an)o(y)g(prop)q(erly)h(con-) 458 219 y(structed)i(user-de\014ned)j Fo(OOMPI)d Fp(ob)s(ject.)24 b(The)17 b(argumen)o(t)f Fo(rcvbuf)g Fp(is)h(a)f(p)q(oin)o(ter)458 275 y(to)11 b(an)o(y)h(v)m(alid)i(base)f(MPI)f(datat)o(yp)q(e)g(or)g(a) g(p)q(oin)o(ter)h(to)f(an)o(y)g(prop)q(erly)h(constructed)458 332 y(user-de\014ned)f Fo(OOMPI)d Fp(ob)s(ject.)18 b(The)10 b(argumen)o(ts)f Fo(sendbuf)h Fp(and)g Fo(recvbuf)f Fp(will)j(au-)458 388 y(tomatically)j(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1139 388 V 17 w(Message)f Fp(and)i Fo(OOMPI)p 1545 388 V 17 w(Array)p 1682 388 V 16 w(message)p Fp(,)458 444 y(resp)q(ectiv)o(ely)l(.)21 b(The)16 b(user)f(is)h(not)f(required)h(to) f(p)q(erform)g(a)f(cast.)458 539 y Fo(void)23 b(Gatherv\(OOMPI)p 892 539 V 15 w(Array)p 1027 539 V 17 w(message)g(sendbuf,)g(int)g (sendcount,)g(OOMPI)p 1928 539 V 16 w(-)458 595 y(Array)p 581 595 V 16 w(message)g(recvbuf,)g(int)g(recvcounts[],)f(int)i (displs[]\).)17 b Fp(Calls)458 652 y Fo(MPI)p 533 652 V 16 w(Gatherv\(\))p Fp(.)33 b(The)20 b(argumen)o(ts)f Fo(sendbuf)g Fp(and)h Fo(rcvbuf)f Fp(are)g(p)q(oin)o(ters)i(to)458 708 y(an)o(y)11 b(v)m(alid)j(base)e(MPI)g(datat)o(yp)q(e)f(or)g(prop)q (erly)i(constructed)f(user-de\014ned)i Fo(OOMPI)458 765 y Fp(ob)s(ject,)f(eac)o(h)i(of)e(whic)o(h)i(will)h(automatically)f(b)q (e)g(promoted)f(to)f Fo(OOMPI)p 1701 765 V 17 w(Array)p 1838 765 V 16 w(-)458 821 y(message)p Fp(.)19 b(The)c(user)g(is)h(not)f (required)h(to)f(cast)f(the)i(p)q(oin)o(ters.)458 915 y Fo(void)23 b(Reduce\(OOMPI)p 868 915 V 16 w(Message)f(sendbuf,)h (OOMPI)p 1409 915 V 17 w(Array)p 1546 915 V 16 w(message)g(recvbuf,)458 972 y(OOMPI)p 581 972 V 16 w(Op&)h(op\).)h Fp(Calls)18 b Fo(MPI)p 1002 972 V 17 w(Reduce\(\))p Fp(,)e(using)h(the)h(in)o (ternal)f Fo(MPI)p 1682 972 V 17 w(Op)g Fp(of)g Fo(op)p Fp(.)458 1028 y(The)e(argumen)o(t)g Fo(sendbuf)f Fp(is)i(of)f(an)o(y)g (v)m(alid)i(base)e(MPI)g(datat)o(yp)q(e)g(or)g(an)o(y)g(prop-)458 1085 y(erly)i(constructed)g(user-de\014ned)h Fo(OOMPI)e Fp(ob)s(ject.)24 b(The)16 b(argumen)o(t)g Fo(rcvbuf)g Fp(is)h(a)458 1141 y(p)q(oin)o(ter)j(to)f(an)o(y)g(v)m(alid)j(base)e (MPI)f(datat)o(yp)q(e)h(or)f(a)g(p)q(oin)o(ter)i(to)e(an)o(y)g(prop)q (erly)458 1197 y(constructed)j(user-de\014ned)h Fo(OOMPI)e Fp(ob)s(ject.)39 b(The)22 b(argumen)o(ts)f Fo(sendbuf)g Fp(and)458 1254 y Fo(recvbuf)11 b Fp(will)i(automatically)f(b)q(e)h (promoted)e(to)g Fo(OOMPI)p 1437 1254 V 17 w(Message)g Fp(and)h Fo(OOMPI)p 1838 1254 V 16 w(-)458 1310 y(Array)p 581 1310 V 16 w(message)p Fp(,)g(resp)q(ectiv)o(ely)l(.)20 b(The)13 b(user)f(is)h(not)f(required)h(to)f(p)q(erform)g(a)g(cast.)458 1405 y Fo(void)23 b(Reduce\(OOMPI)p 868 1405 V 16 w(Array)p 1004 1405 V 16 w(message)g(sendbuf,)g(OOMPI)p 1546 1405 V 16 w(Array)p 1682 1405 V 17 w(message)g(recvbuf,)458 1461 y(OOMPI)p 581 1461 V 16 w(Op&)h(op,)f(int)h(sendcount=)e(1\).)29 b Fp(Calls)19 b Fo(MPI)p 1436 1461 V 16 w(Reduce\(\))e Fp(,)i(using)g(the)458 1518 y(in)o(ternal)14 b Fo(MPI)p 699 1518 V 17 w(Op)f Fp(of)h Fo(op)p Fp(.)19 b(The)14 b(argumen)o(ts)f Fo(sendbuf)f Fp(and)i Fo(rcvbuf)f Fp(are)h(p)q(oin)o (ters)458 1574 y(to)20 b(an)o(y)g(v)m(alid)i(base)f(MPI)g(datat)o(yp)q (e)f(or)g(prop)q(erly)h(constructed)g(user-de\014ned)458 1630 y Fo(OOMPI)14 b Fp(ob)s(ject,)f(eac)o(h)i(of)f(whic)o(h)h(will)i (automatically)e(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1838 1630 V 16 w(-)458 1687 y(Array)p 581 1687 V 16 w(message)p Fp(.)19 b(The)d(user)f(is)h(not)e(required)j(to)d(cast)h(the)g(p)q(oin) o(ters.)458 1781 y Fo(void)23 b(Scatter\(OOMPI)p 892 1781 V 15 w(Message)g(recvbuf,)g(OOMPI)p 1433 1781 V 17 w(Array)p 1570 1781 V 16 w(message)g(sendbuf,)458 1838 y(int)g(sendcount=)g(1\).)f Fp(Calls)16 b Fo(MPI)p 1099 1838 V 17 w(Scatter\(\))p Fp(.)21 b(The)16 b(argumen)o(t)f Fo(sendbuf)g Fp(is)458 1894 y(of)c(an)o(y)h(v)m(alid)h(base)f(MPI)g (datat)o(yp)q(e)f(or)h(an)o(y)f(prop)q(erly)i(constructed)f (user-de\014ned)458 1951 y Fo(OOMPI)h Fp(ob)s(ject.)19 b(The)c(argumen)o(t)f Fo(rcvbuf)f Fp(is)i(a)f(p)q(oin)o(ter)h(to)f(an)o (y)g(v)m(alid)i(base)e(MPI)458 2007 y(datat)o(yp)q(e)k(or)h(a)g(p)q (oin)o(ter)g(to)g(an)o(y)g(prop)q(erly)h(constructed)f(user-de\014ned)i Fo(OOMPI)458 2063 y Fp(ob)s(ject.)d(The)13 b(argumen)o(ts)f Fo(sendbuf)g Fp(and)h Fo(recvbuf)e Fp(will)k(automatically)e(b)q(e)g (pro-)458 2120 y(moted)j(to)f Fo(OOMPI)p 777 2120 V 17 w(Message)g Fp(and)h Fo(OOMPI)p 1186 2120 V 17 w(Array)p 1323 2120 V 16 w(message)p Fp(,)f(resp)q(ectiv)o(ely)l(.)25 b(The)458 2176 y(user)15 b(is)h(not)f(required)h(to)e(p)q(erform)h(a)g (cast.)458 2271 y Fo(void)23 b(Scatter\(OOMPI)p 892 2271 V 15 w(Array)p 1027 2271 V 17 w(message)g(recvbuf,)g(int)g(recvcount,)g (OOMPI)p 1928 2271 V 16 w(-)458 2327 y(Array)p 581 2327 V 16 w(message)g(sendbuf,)g(int)g(sendcount=)g(1\).)18 b Fp(Calls)13 b Fo(MPI)p 1634 2327 V 17 w(Scatter\(\))p Fp(.)458 2384 y(The)g(argumen)o(ts)g Fo(sendbuf)g Fp(and)g Fo(rcvbuf)g Fp(are)g(p)q(oin)o(ters)h(to)f(an)o(y)g(v)m(alid)i(base)f (MPI)458 2440 y(datat)o(yp)q(e)23 b(or)f(prop)q(erly)j(constructed)e (user-de\014ned)i Fo(OOMPI)e Fp(ob)s(ject,)h(eac)o(h)f(of)458 2496 y(whic)o(h)d(will)i(automatically)e(b)q(e)h(promoted)e(to)g Fo(OOMPI)p 1433 2496 V 17 w(Array)p 1570 2496 V 16 w(message)p Fp(.)33 b(The)458 2553 y(user)15 b(is)h(not)f(required)h(to)e(cast)h (the)g(p)q(oin)o(ters.)458 2647 y Fo(void)23 b(Scatterv\(OOMPI)p 916 2647 V 15 w(Message)g(recvbuf,)g(OOMPI)p 1457 2647 V 16 w(Array)p 1593 2647 V 17 w(message)g(sendbuf,)458 2704 y(int)g(sendcounts[],)f(int)i(displs[]\).)e Fp(Calls)c Fo(MPI)p 1436 2704 V 16 w(Scatterv\(\))p Fp(.)23 b(The)16 b(ar-)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 33 33 33 32 bop 458 49 a Fp(gumen)o(t)13 b Fo(sendbuf)f Fp(is)i(of)e(an)o(y)h (v)m(alid)i(base)e(MPI)g(datat)o(yp)q(e)g(or)g(an)o(y)g(prop)q(erly)h (con-)458 106 y(structed)i(user-de\014ned)j Fo(OOMPI)d Fp(ob)s(ject.)24 b(The)17 b(argumen)o(t)f Fo(rcvbuf)g Fp(is)h(a)f(p)q(oin)o(ter)458 162 y(to)11 b(an)o(y)h(v)m(alid)i(base)f (MPI)f(datat)o(yp)q(e)g(or)g(a)g(p)q(oin)o(ter)h(to)f(an)o(y)g(prop)q (erly)h(constructed)458 219 y(user-de\014ned)f Fo(OOMPI)d Fp(ob)s(ject.)18 b(The)10 b(argumen)o(ts)f Fo(sendbuf)h Fp(and)g Fo(recvbuf)f Fp(will)j(au-)458 275 y(tomatically)j(b)q(e)g (promoted)f(to)g Fo(OOMPI)p 1139 275 15 2 v 17 w(Message)f Fp(and)i Fo(OOMPI)p 1545 275 V 17 w(Array)p 1682 275 V 16 w(message)p Fp(,)458 332 y(resp)q(ectiv)o(ely)l(.)21 b(The)16 b(user)f(is)h(not)f(required)h(to)f(p)q(erform)g(a)f(cast.)458 429 y Fo(void)23 b(Scatterv\(OOMPI)p 916 429 V 15 w(Array)p 1051 429 V 17 w(message)g(recvbuf,)f(int)i(recvcount,)e(OOMPI)p 1951 429 V 17 w(-)458 486 y(Array)p 581 486 V 16 w(message)h(sendbuf,)g (int)g(sendcounts[],)f(int)i(displs[]\).)17 b Fp(Calls)458 542 y Fo(MPI)p 533 542 V 16 w(Scatterv\(\))p Fp(.)26 b(The)17 b(argumen)o(ts)g Fo(sendbuf)f Fp(and)i Fo(rcvbuf)f Fp(are)g(p)q(oin)o(ters)h(to)458 599 y(an)o(y)11 b(v)m(alid)j(base)e (MPI)g(datat)o(yp)q(e)f(or)g(prop)q(erly)i(constructed)f (user-de\014ned)i Fo(OOMPI)458 655 y Fp(ob)s(ject,)f(eac)o(h)i(of)e (whic)o(h)i(will)h(automatically)f(b)q(e)g(promoted)f(to)f Fo(OOMPI)p 1701 655 V 17 w(Array)p 1838 655 V 16 w(-)458 712 y(message)p Fp(.)19 b(The)c(user)g(is)h(not)f(required)h(to)f(cast) f(the)i(p)q(oin)o(ters.)458 810 y Fo(OOMPI)p 581 810 V 16 w(Port&)23 b(operator[]\(int)f(i\).)29 b Fp(Sets)18 b(the)g(coun)o(t)g(for)f(subsequen)o(t)i(op-)458 866 y(erations)c(with)g Fo(OOMPI)p 856 866 V 17 w(Array)p 993 866 V 16 w(message)g Fp(ob)s(jects.)100 985 y Fh(Sends)250 b Fo(OOMPI)p 581 985 V 16 w(Port&)23 b(operator<<\(OOMPI)p 1124 985 V 15 w(Message)g(buf\).)36 b Fp(Stream)20 b(in)o(terface)h(to) 458 1041 y Fo(MPI)p 533 1041 V 16 w(Send\(\))p Fp(.)29 b(The)18 b(argumen)o(t)f Fo(buf)h Fp(is)h(of)f(an)o(y)g(v)m(alid)h (base)g(MPI)f(datat)o(yp)q(e)f(or)458 1098 y(an)o(y)j(prop)q(erly)i (constructed)f(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)h(whic)o(h)h (will)g(auto-)458 1154 y(matically)c(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1110 1154 V 16 w(Message)p Fp(.)25 b(The)18 b(user)f(is)h(not)f(required)i(to)458 1211 y(p)q(erform)c(a)g(cast.)458 1309 y Fo(OOMPI)p 581 1309 V 16 w(Port&)23 b(operator<<\(OOMPI)p 1124 1309 V 15 w(Array)p 1259 1309 V 17 w(message)g(buf\).)i Fp(Stream)17 b(in)o(ter-)458 1365 y(face)d(to)h Fo(MPI)p 680 1365 V 16 w(Send\(\))p Fp(.)k(The)c(argumen)o(t)f Fo(buf)g Fp(is)i(a)e(p)q(oin)o(ter)h(an)o(y)g(v)m(alid)h(base)f(MPI)458 1422 y(datat)o(yp)q(e)g(or)g(an)o(y)h(prop)q(erly)g(constructed)g (user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)g(whic)o(h)458 1478 y(will)j(automatically)f(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1282 1478 V 17 w(Message)p Fp(.)23 b(The)17 b(user)g(is)g(not)f(re-)458 1535 y(quired)g(to)f(p)q(erform)g(a)f (cast.)458 1632 y Fo(void)23 b(Bsend\(OOMPI)p 844 1632 V 16 w(Message)g(buf,)g(int)g(tag)h(=)g(OOMPI)p 1529 1632 V 16 w(NO)p 1593 1632 V 17 w(TAG\).)9 b Fp(Calls)i Fo(MPI)p 1920 1632 V 17 w(-)458 1689 y(Bsend\(\))p Fp(.)22 b(The)17 b(argumen)o(t)f Fo(buf)g Fp(is)h(of)f(an)o(y)g(v)m(alid)i (base)f(MPI)f(datat)o(yp)q(e)g(or)g(an)o(y)458 1745 y(prop)q(erly)22 b(constructed)f(user-de\014ned)h Fo(OOMPI)f Fp(ob)s(ject,)g(whic)o(h)h (will)h(automati-)458 1802 y(cally)18 b(b)q(e)f(promoted)g(to)f Fo(OOMPI)p 1017 1802 V 16 w(Message)p Fp(.)24 b(The)17 b(user)g(is)h(not)e(required)i(to)e(p)q(er-)458 1858 y(form)e(a)h(cast.)458 1956 y Fo(void)23 b(Bsend\(OOMPI)p 844 1956 V 16 w(Array)p 980 1956 V 16 w(message)g(buf,)g(int)h(count,)f (int)g(tag)h(=)g(OOMPI)p 1928 1956 V 16 w(-)458 2013 y(NO)p 509 2013 V 17 w(TAG\).)15 b Fp(Calls)i Fo(MPI)p 848 2013 V 17 w(Bsend\(\))p Fp(.)k(The)c(argumen)o(t)e Fo(buf)h Fp(is)g(a)g(p)q(oin)o(ter)h(an)o(y)e(v)m(alid)458 2069 y(base)g(MPI)h(datat)o(yp)q(e)e(or)h(an)o(y)g(prop)q(erly)i (constructed)e(user-de\014ned)i Fo(OOMPI)e Fp(ob-)458 2126 y(ject,)e(whic)o(h)h(will)g(automatically)g(b)q(e)f(promoted)g(to) f Fo(OOMPI)p 1490 2126 V 16 w(Message)p Fp(.)19 b(The)13 b(user)458 2182 y(is)i(not)g(required)i(to)d(p)q(erform)h(a)g(cast.)458 2280 y Fo(void)23 b(Rsend\(OOMPI)p 844 2280 V 16 w(Message)g(buf,)g (int)g(tag)h(=)g(OOMPI)p 1529 2280 V 16 w(NO)p 1593 2280 V 17 w(TAG\).)9 b Fp(Calls)i Fo(MPI)p 1920 2280 V 17 w(-)458 2337 y(Rsend\(\))p Fp(.)22 b(The)17 b(argumen)o(t)f Fo(buf)g Fp(is)h(of)f(an)o(y)g(v)m(alid)i(base)f(MPI)f(datat)o(yp)q(e)g (or)g(an)o(y)458 2393 y(prop)q(erly)22 b(constructed)f(user-de\014ned)h Fo(OOMPI)f Fp(ob)s(ject,)g(whic)o(h)h(will)h(automati-)458 2449 y(cally)18 b(b)q(e)f(promoted)g(to)f Fo(OOMPI)p 1017 2449 V 16 w(Message)p Fp(.)24 b(The)17 b(user)g(is)h(not)e (required)i(to)e(p)q(er-)458 2506 y(form)e(a)h(cast.)458 2604 y Fo(void)23 b(Rsend\(OOMPI)p 844 2604 V 16 w(Array)p 980 2604 V 16 w(message)g(buf,)g(int)h(count,)f(int)g(tag)h(=)g(OOMPI)p 1928 2604 V 16 w(-)458 2660 y(NO)p 509 2660 V 17 w(TAG\).)15 b Fp(Calls)i Fo(MPI)p 848 2660 V 17 w(Rsend\(\))p Fp(.)k(The)c(argumen) o(t)e Fo(buf)h Fp(is)g(a)g(p)q(oin)o(ter)h(an)o(y)e(v)m(alid)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 34 34 34 33 bop 458 49 a Fp(base)15 b(MPI)h(datat)o(yp)q(e)e(or)h(an)o(y)g (prop)q(erly)i(constructed)e(user-de\014ned)i Fo(OOMPI)e Fp(ob-)458 106 y(ject,)e(whic)o(h)h(will)g(automatically)g(b)q(e)f (promoted)g(to)f Fo(OOMPI)p 1490 106 15 2 v 16 w(Message)p Fp(.)19 b(The)13 b(user)458 162 y(is)i(not)g(required)i(to)d(p)q (erform)h(a)g(cast.)458 260 y Fo(void)23 b(Send\(OOMPI)p 820 260 V 16 w(Message)g(buf,)g(int)h(tag)f(=)h(OOMPI)p 1505 260 V 16 w(NO)p 1569 260 V 17 w(TAG\).)10 b Fp(Calls)g Fo(MPI)p 1896 260 V 17 w(-)458 317 y(Send\(\))p Fp(.)28 b(The)18 b(argumen)o(t)g Fo(buf)f Fp(is)i(of)f(an)o(y)g(v)m(alid)i (base)e(MPI)g(datat)o(yp)q(e)g(or)f(an)o(y)458 373 y(prop)q(erly)22 b(constructed)f(user-de\014ned)h Fo(OOMPI)f Fp(ob)s(ject,)g(whic)o(h)h (will)h(automati-)458 429 y(cally)18 b(b)q(e)f(promoted)g(to)f Fo(OOMPI)p 1017 429 V 16 w(Message)p Fp(.)24 b(The)17 b(user)g(is)h(not)e(required)i(to)e(p)q(er-)458 486 y(form)e(a)h(cast.) 458 584 y Fo(void)23 b(Send\(OOMPI)p 820 584 V 16 w(Array)p 956 584 V 16 w(message)g(buf,)h(int)f(count,)g(int)h(tag)f(=)h(OOMPI)p 1904 584 V 16 w(-)458 640 y(NO)p 509 640 V 17 w(TAG\).)g Fp(Stream)g(in)o(terface)h(to)f Fo(MPI)p 1167 640 V 17 w(Send\(\))p Fp(.)47 b(The)25 b(argumen)o(t)f Fo(buf)g Fp(is)h(a)458 697 y(p)q(oin)o(ter)16 b(an)o(y)f(v)m(alid)i(base)f(MPI)f (datat)o(yp)q(e)g(or)g(an)o(y)h(prop)q(erly)g(constructed)g(user-)458 753 y(de\014ned)f Fo(OOMPI)f Fp(ob)s(ject,)f(whic)o(h)i(will)h (automatically)f(b)q(e)f(promoted)g(to)g Fo(OOMPI)p 1838 753 V 16 w(-)458 810 y(Message)p Fp(.)19 b(The)c(user)g(is)h(not)f (required)h(to)f(p)q(erform)g(a)g(cast.)458 908 y Fo(void)23 b(Ssend\(OOMPI)p 844 908 V 16 w(Message)g(buf,)g(int)g(tag)h(=)g(OOMPI) p 1529 908 V 16 w(NO)p 1593 908 V 17 w(TAG\).)9 b Fp(Calls)i Fo(MPI)p 1920 908 V 17 w(-)458 964 y(Ssend\(\))p Fp(.)22 b(The)17 b(argumen)o(t)f Fo(buf)g Fp(is)h(of)f(an)o(y)g(v)m(alid)i (base)f(MPI)f(datat)o(yp)q(e)g(or)g(an)o(y)458 1021 y(prop)q(erly)22 b(constructed)f(user-de\014ned)h Fo(OOMPI)f Fp(ob)s(ject,)g(whic)o(h)h (will)h(automati-)458 1077 y(cally)18 b(b)q(e)f(promoted)g(to)f Fo(OOMPI)p 1017 1077 V 16 w(Message)p Fp(.)24 b(The)17 b(user)g(is)h(not)e(required)i(to)e(p)q(er-)458 1134 y(form)e(a)h(cast.)458 1231 y Fo(void)23 b(Ssend\(OOMPI)p 844 1231 V 16 w(Array)p 980 1231 V 16 w(message)g(buf,)g(int)h(count,)f (int)g(tag)h(=)g(OOMPI)p 1928 1231 V 16 w(-)458 1288 y(NO)p 509 1288 V 17 w(TAG\).)15 b Fp(Calls)i Fo(MPI)p 848 1288 V 17 w(Bsend\(\))p Fp(.)k(The)c(argumen)o(t)e Fo(buf)h Fp(is)g(a)g(p)q(oin)o(ter)h(an)o(y)e(v)m(alid)458 1344 y(base)g(MPI)h(datat)o(yp)q(e)e(or)h(an)o(y)g(prop)q(erly)i (constructed)e(user-de\014ned)i Fo(OOMPI)e Fp(ob-)458 1401 y(ject,)e(whic)o(h)h(will)g(automatically)g(b)q(e)f(promoted)g(to) f Fo(OOMPI)p 1490 1401 V 16 w(Message)p Fp(.)19 b(The)13 b(user)458 1457 y(is)i(not)g(required)i(to)d(p)q(erform)h(a)g(cast.)458 1555 y Fo(OOMPI)p 581 1555 V 16 w(Request&)23 b(Bsend)p 932 1555 V 16 w(init\(OOMPI)p 1188 1555 V 16 w(Message)g(buf,)h(int)f (tag)g(=)h(OOMPI)p 1873 1555 V 17 w(-)458 1612 y(NO)p 509 1612 V 17 w(TAG\).)18 b Fp(Calls)h Fo(MPI)p 853 1612 V 17 w(Bsend)p 990 1612 V 17 w(init\(\))p Fp(.)30 b(The)19 b(argumen)o(t)f Fo(buf)h Fp(is)g(of)g(an)o(y)f(v)m(alid)458 1668 y(base)d(MPI)h(datat)o(yp)q(e)e(or)h(an)o(y)g(prop)q(erly)i (constructed)e(user-de\014ned)i Fo(OOMPI)e Fp(ob-)458 1725 y(ject,)e(whic)o(h)h(will)g(automatically)g(b)q(e)f(promoted)g(to) f Fo(OOMPI)p 1490 1725 V 16 w(Message)p Fp(.)19 b(The)13 b(user)458 1781 y(is)j(not)f(required)i(to)e(p)q(erform)g(a)g(cast.)21 b(An)16 b Fo(OOMPI)p 1341 1781 V 16 w(Request)f Fp(constructed)h(from) 458 1838 y(the)f(MPI)g(request)g(generated)h(b)o(y)f Fo(MPI)p 1144 1838 V 16 w(Bsend)p 1280 1838 V 17 w(init\(\))f Fp(is)i(returned.)458 1936 y Fo(OOMPI)p 581 1936 V 16 w(Request&)23 b(Bsend)p 932 1936 V 16 w(init\(OOMPI)p 1188 1936 V 16 w(Array)p 1324 1936 V 17 w(message)g(buf,)g(int)g (count,)458 1992 y(int)g(tag)h(=)f(OOMPI)p 819 1992 V 17 w(NO)p 884 1992 V 17 w(TAG\).)f Fp(Calls)h Fo(MPI)p 1236 1992 V 17 w(Bsend)p 1373 1992 V 17 w(init\(\))p Fp(.)42 b(The)23 b(argumen)o(t)458 2048 y Fo(buf)f Fp(is)h(a)g(p)q(oin) o(ter)g(an)o(y)g(v)m(alid)h(base)f(MPI)g(datat)o(yp)q(e)f(or)g(an)o(y)h (prop)q(erly)h(con-)458 2105 y(structed)e(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)i(whic)o(h)g(will)h(automatically)e(b)q(e)h (pro-)458 2161 y(moted)10 b(to)h Fo(OOMPI)p 767 2161 V 16 w(Message)p Fp(.)18 b(The)11 b(user)g(is)h(not)f(required)h(to)e (p)q(erform)h(a)g(cast.)18 b(An)458 2218 y Fo(OOMPI)p 581 2218 V 16 w(Request)f Fp(constructed)h(from)g(the)g(MPI)g(request)f (generated)h(b)o(y)g Fo(MPI)p 1837 2218 V 17 w(-)458 2274 y(Bsend)p 581 2274 V 16 w(init\(\))d Fp(is)g(returned.)458 2372 y Fo(OOMPI)p 581 2372 V 16 w(Request&)23 b(Rsend)p 932 2372 V 16 w(init\(OOMPI)p 1188 2372 V 16 w(Message)g(buf,)h(int)f (tag)g(=)h(OOMPI)p 1873 2372 V 17 w(-)458 2429 y(NO)p 509 2429 V 17 w(TAG\).)18 b Fp(Calls)h Fo(MPI)p 853 2429 V 17 w(Rsend)p 990 2429 V 17 w(init\(\))p Fp(.)30 b(The)19 b(argumen)o(t)f Fo(buf)h Fp(is)g(of)g(an)o(y)f(v)m(alid)458 2485 y(base)d(MPI)h(datat)o(yp)q(e)e(or)h(an)o(y)g(prop)q(erly)i (constructed)e(user-de\014ned)i Fo(OOMPI)e Fp(ob-)458 2542 y(ject,)e(whic)o(h)h(will)g(automatically)g(b)q(e)f(promoted)g(to) f Fo(OOMPI)p 1490 2542 V 16 w(Message)p Fp(.)19 b(The)13 b(user)458 2598 y(is)j(not)f(required)i(to)e(p)q(erform)g(a)g(cast.)21 b(An)16 b Fo(OOMPI)p 1341 2598 V 16 w(Request)f Fp(constructed)h(from) 458 2654 y(the)f(MPI)g(request)g(generated)h(b)o(y)f Fo(MPI)p 1144 2654 V 16 w(Rsend)p 1280 2654 V 17 w(init\(\))f Fp(is)i(returned.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 35 35 35 34 bop 458 49 a Fo(OOMPI)p 581 49 15 2 v 16 w(Request&)23 b(Rsend)p 932 49 V 16 w(init\(OOMPI)p 1188 49 V 16 w(Array)p 1324 49 V 17 w(message)g(buf,)g(int)g(count,)458 106 y(int)g(tag)h(=)f(OOMPI)p 819 106 V 17 w(NO)p 884 106 V 17 w(TAG\).)f Fp(Calls)h Fo(MPI)p 1236 106 V 17 w(Rsend)p 1373 106 V 17 w(init\(\))p Fp(.)42 b(The)23 b(argumen)o(t)458 162 y Fo(buf)f Fp(is)h(a)g(p)q(oin)o(ter)g(an)o(y)g(v)m(alid)h(base)f (MPI)g(datat)o(yp)q(e)f(or)g(an)o(y)h(prop)q(erly)h(con-)458 219 y(structed)e(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)i(whic)o(h)g (will)h(automatically)e(b)q(e)h(pro-)458 275 y(moted)10 b(to)h Fo(OOMPI)p 767 275 V 16 w(Message)p Fp(.)18 b(The)11 b(user)g(is)h(not)f(required)h(to)e(p)q(erform)h(a)g(cast.)18 b(An)458 332 y Fo(OOMPI)p 581 332 V 16 w(Request)f Fp(constructed)h (from)g(the)g(MPI)g(request)f(generated)h(b)o(y)g Fo(MPI)p 1837 332 V 17 w(-)458 388 y(Rsend)p 581 388 V 16 w(init\(\))d Fp(is)g(returned.)458 482 y Fo(OOMPI)p 581 482 V 16 w(Request&)23 b(Send)p 908 482 V 17 w(init\(OOMPI)p 1165 482 V 15 w(Message)g(buf,)h (int)f(tag)h(=)f(OOMPI)p 1849 482 V 17 w(-)458 539 y(NO)p 509 539 V 17 w(TAG\).)10 b Fp(Calls)i Fo(MPI)p 838 539 V 17 w(Send)p 951 539 V 17 w(init\(\))p Fp(.)17 b(The)12 b(argumen)o(t)e Fo(buf)h Fp(is)h(of)f(an)o(y)g(v)m(alid)i(base)458 595 y(MPI)19 b(datat)o(yp)q(e)f(or)h(an)o(y)g(prop)q(erly)h (constructed)f(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)458 652 y(whic)o(h)h(will)h(automatically)f(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1425 652 V 16 w(Message)p Fp(.)29 b(The)19 b(user)f(is)458 708 y(not)12 b(required)h(to)e(p)q(erform)h(a)g(cast.) 19 b(An)12 b Fo(OOMPI)p 1272 708 V 17 w(Request)f Fp(constructed)h (from)g(the)458 765 y(MPI)j(request)g(generated)g(b)o(y)g Fo(MPI)p 1065 765 V 17 w(Send)p 1178 765 V 17 w(init\(\))f Fp(is)i(returned.)458 859 y Fo(OOMPI)p 581 859 V 16 w(Request&)23 b(Send)p 908 859 V 17 w(init\(OOMPI)p 1165 859 V 15 w(Array)p 1300 859 V 17 w(message)g(buf,)g(int)h(count,)f(int)458 915 y(tag)g(=)h(OOMPI)p 724 915 V 16 w(NO)p 788 915 V 17 w(TAG\).)14 b Fp(Calls)h Fo(MPI)p 1124 915 V 17 w(Send)p 1237 915 V 16 w(init\(\))p Fp(.)k(The)c(argumen)o(t)e Fo(buf)h Fp(is)h(a)458 972 y(p)q(oin)o(ter)h(an)o(y)f(v)m(alid)i(base)f (MPI)f(datat)o(yp)q(e)g(or)g(an)o(y)h(prop)q(erly)g(constructed)g (user-)458 1028 y(de\014ned)f Fo(OOMPI)f Fp(ob)s(ject,)f(whic)o(h)i (will)h(automatically)f(b)q(e)f(promoted)g(to)g Fo(OOMPI)p 1838 1028 V 16 w(-)458 1085 y(Message)p Fp(.)j(The)c(user)f(is)g(not)g (required)h(to)e(p)q(erform)g(a)h(cast.)18 b(An)12 b Fo(OOMPI)p 1694 1085 V 17 w(Request)458 1141 y Fp(constructed)i(from)g (the)h(MPI)g(request)f(generated)h(b)o(y)g Fo(MPI)p 1493 1141 V 16 w(Send)p 1605 1141 V 17 w(init\(\))f Fp(is)h(re-)458 1197 y(turned.)458 1292 y Fo(OOMPI)p 581 1292 V 16 w(Request&)23 b(Ssend)p 932 1292 V 16 w(init\(OOMPI)p 1188 1292 V 16 w(Message)g(buf,)h(int)f(tag)g(=)h(OOMPI)p 1873 1292 V 17 w(-)458 1348 y(NO)p 509 1348 V 17 w(TAG\).)18 b Fp(Calls)h Fo(MPI)p 853 1348 V 17 w(Ssend)p 990 1348 V 17 w(init\(\))p Fp(.)30 b(The)19 b(argumen)o(t)f Fo(buf)h Fp(is)g(of)g(an)o(y)f(v)m(alid)458 1405 y(base)d(MPI)h(datat)o(yp)q(e)e (or)h(an)o(y)g(prop)q(erly)i(constructed)e(user-de\014ned)i Fo(OOMPI)e Fp(ob-)458 1461 y(ject,)e(whic)o(h)h(will)g(automatically)g (b)q(e)f(promoted)g(to)f Fo(OOMPI)p 1490 1461 V 16 w(Message)p Fp(.)19 b(The)13 b(user)458 1518 y(is)j(not)f(required)i(to)e(p)q (erform)g(a)g(cast.)21 b(An)16 b Fo(OOMPI)p 1341 1518 V 16 w(Request)f Fp(constructed)h(from)458 1574 y(the)f(MPI)g(request)g (generated)h(b)o(y)f Fo(MPI)p 1144 1574 V 16 w(Ssend)p 1280 1574 V 17 w(init\(\))f Fp(is)i(returned.)458 1668 y Fo(OOMPI)p 581 1668 V 16 w(Request&)23 b(Ssend)p 932 1668 V 16 w(init\(OOMPI)p 1188 1668 V 16 w(Array)p 1324 1668 V 17 w(message)g(buf,)g(int)g(count,)458 1725 y(int)g(tag)h(=)f (OOMPI)p 819 1725 V 17 w(NO)p 884 1725 V 17 w(TAG\).)f Fp(Calls)h Fo(MPI)p 1236 1725 V 17 w(Ssend)p 1373 1725 V 17 w(init\(\))p Fp(.)42 b(The)23 b(argumen)o(t)458 1781 y Fo(buf)f Fp(is)h(a)g(p)q(oin)o(ter)g(an)o(y)g(v)m(alid)h(base)f (MPI)g(datat)o(yp)q(e)f(or)g(an)o(y)h(prop)q(erly)h(con-)458 1838 y(structed)e(user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)i(whic)o(h) g(will)h(automatically)e(b)q(e)h(pro-)458 1894 y(moted)10 b(to)h Fo(OOMPI)p 767 1894 V 16 w(Message)p Fp(.)18 b(The)11 b(user)g(is)h(not)f(required)h(to)e(p)q(erform)h(a)g(cast.)18 b(An)458 1951 y Fo(OOMPI)p 581 1951 V 16 w(Request)f Fp(constructed)h(from)g(the)g(MPI)g(request)f(generated)h(b)o(y)g Fo(MPI)p 1837 1951 V 17 w(-)458 2007 y(Ssend)p 581 2007 V 16 w(init\(\))d Fp(is)g(returned.)458 2101 y Fo(OOMPI)p 581 2101 V 16 w(Request&)23 b(Ibsend\(OOMPI)p 1100 2101 V 16 w(Message)f(buf,)i(int)f(tag)h(=)f(OOMPI)p 1784 2101 V 17 w(NO)p 1849 2101 V 17 w(-)458 2158 y(TAG\).)13 b Fp(Calls)i Fo(MPI)p 779 2158 V 17 w(Ibsend\(\))p Fp(.)j(The)d (argumen)o(t)e Fo(buf)h Fp(is)g(of)g(an)o(y)g(v)m(alid)i(base)e(MPI)458 2214 y(datat)o(yp)q(e)h(or)g(an)o(y)h(prop)q(erly)g(constructed)g (user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)g(whic)o(h)458 2271 y(will)j(automatically)f(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1282 2271 V 17 w(Message)p Fp(.)23 b(The)17 b(user)g(is)g(not)f(re-)458 2327 y(quired)f(to)e(p)q(erform)g(a)h (cast.)19 b(An)14 b Fo(OOMPI)p 1165 2327 V 16 w(Request)f Fp(constructed)h(from)f(the)h(MPI)458 2384 y(request)h(generated)g(b)o (y)g Fo(MPI)p 961 2384 V 17 w(Ibsend\(\))f Fp(is)i(returned.)458 2478 y Fo(OOMPI)p 581 2478 V 16 w(Request&)23 b(Ibsend\(OOMPI)p 1100 2478 V 16 w(Array)p 1236 2478 V 16 w(message)g(buf,)g(int)h (count,)f(int)458 2534 y(tag)g(=)h(OOMPI)p 724 2534 V 16 w(NO)p 788 2534 V 17 w(TAG\).)d Fp(Calls)i Fo(MPI)p 1139 2534 V 17 w(Ibsend\(\))p Fp(.)38 b(The)22 b(argumen)o(t)g Fo(buf)f Fp(is)h(a)458 2591 y(p)q(oin)o(ter)16 b(an)o(y)f(v)m(alid)i (base)f(MPI)f(datat)o(yp)q(e)g(or)g(an)o(y)h(prop)q(erly)g(constructed) g(user-)458 2647 y(de\014ned)f Fo(OOMPI)f Fp(ob)s(ject,)f(whic)o(h)i (will)h(automatically)f(b)q(e)f(promoted)g(to)g Fo(OOMPI)p 1838 2647 V 16 w(-)458 2704 y(Message)p Fp(.)41 b(The)23 b(user)f(is)i(not)e(required)h(to)f(p)q(erform)h(a)f(cast.)42 b(An)23 b Fo(OOMPI)p 1838 2704 V 16 w(-)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 36 36 36 35 bop 458 49 a Fo(Request)13 b Fp(constructed)h(from)g(the)g(MPI)g (request)g(generated)g(b)o(y)g Fo(MPI)p 1670 49 15 2 v 17 w(Ibsend\(\))458 106 y Fp(is)h(returned.)458 204 y Fo(OOMPI)p 581 204 V 16 w(Request&)23 b(Irsend\(OOMPI)p 1100 204 V 16 w(Message)f(buf,)i(int)f(tag)h(=)f(OOMPI)p 1784 204 V 17 w(NO)p 1849 204 V 17 w(-)458 260 y(TAG\).)13 b Fp(Calls)i Fo(MPI)p 779 260 V 17 w(Irsend\(\))p Fp(.)j(The)d(argumen) o(t)e Fo(buf)h Fp(is)g(of)g(an)o(y)g(v)m(alid)i(base)e(MPI)458 317 y(datat)o(yp)q(e)h(or)g(an)o(y)h(prop)q(erly)g(constructed)g (user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)g(whic)o(h)458 373 y(will)j(automatically)f(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1282 373 V 17 w(Message)p Fp(.)23 b(The)17 b(user)g(is)g(not)f(re-)458 429 y(quired)f(to)e(p)q(erform)g(a)h(cast.) 19 b(An)14 b Fo(OOMPI)p 1165 429 V 16 w(Request)f Fp(constructed)h (from)f(the)h(MPI)458 486 y(request)h(generated)g(b)o(y)g Fo(MPI)p 961 486 V 17 w(Irsend\(\))f Fp(is)i(returned.)458 584 y Fo(OOMPI)p 581 584 V 16 w(Request&)23 b(Irsend\(OOMPI)p 1100 584 V 16 w(Array)p 1236 584 V 16 w(message)g(buf,)g(int)h(count,)f (int)458 640 y(tag)g(=)h(OOMPI)p 724 640 V 16 w(NO)p 788 640 V 17 w(TAG\).)d Fp(Calls)i Fo(MPI)p 1139 640 V 17 w(Irsend\(\))p Fp(.)38 b(The)22 b(argumen)o(t)g Fo(buf)f Fp(is)h(a)458 697 y(p)q(oin)o(ter)16 b(an)o(y)f(v)m(alid)i (base)f(MPI)f(datat)o(yp)q(e)g(or)g(an)o(y)h(prop)q(erly)g(constructed) g(user-)458 753 y(de\014ned)f Fo(OOMPI)f Fp(ob)s(ject,)f(whic)o(h)i (will)h(automatically)f(b)q(e)f(promoted)g(to)g Fo(OOMPI)p 1838 753 V 16 w(-)458 810 y(Message)p Fp(.)41 b(The)23 b(user)f(is)i(not)e(required)h(to)f(p)q(erform)h(a)f(cast.)42 b(An)23 b Fo(OOMPI)p 1838 810 V 16 w(-)458 866 y(Request)13 b Fp(constructed)h(from)g(the)g(MPI)g(request)g(generated)g(b)o(y)g Fo(MPI)p 1670 866 V 17 w(Irsend\(\))458 923 y Fp(is)h(returned.)458 1021 y Fo(OOMPI)p 581 1021 V 16 w(Request&)23 b(Isend\(OOMPI)p 1076 1021 V 16 w(Message)g(buf,)g(int)g(tag)h(=)g(OOMPI)p 1761 1021 V 16 w(NO)p 1825 1021 V 17 w(TAG\).)458 1077 y Fp(Calls)11 b Fo(MPI)p 642 1077 V 17 w(Isend\(\))p Fp(.)18 b(The)11 b(argumen)o(t)f Fo(buf)h Fp(is)g(of)g(an)o(y)g(v)m (alid)h(base)f(MPI)g(datat)o(yp)q(e)458 1134 y(or)i(an)o(y)h(prop)q (erly)h(constructed)g(user-de\014ned)h Fo(OOMPI)d Fp(ob)s(ject,)h(whic) o(h)h(will)g(auto-)458 1190 y(matically)j(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1110 1190 V 16 w(Message)p Fp(.)25 b(The)18 b(user)f(is)h(not)f(required)i(to)458 1246 y(p)q(erform)e(a)g(cast.)26 b(An)18 b Fo(OOMPI)p 990 1246 V 16 w(Request)f Fp(constructed)g(from)g (the)h(MPI)f(request)458 1303 y(generated)e(b)o(y)g Fo(MPI)p 803 1303 V 17 w(Isend\(\))f Fp(is)i(returned.)458 1401 y Fo(OOMPI)p 581 1401 V 16 w(Request&)23 b(Isend\(OOMPI)p 1076 1401 V 16 w(Array)p 1212 1401 V 16 w(message)g(buf,)g(int)h (count,)f(int)g(tag)458 1457 y(=)g(OOMPI)p 628 1457 V 17 w(NO)p 693 1457 V 17 w(TAG\).)17 b Fp(Calls)h Fo(MPI)p 1035 1457 V 16 w(Isend\(\))p Fp(.)26 b(The)18 b(argumen)o(t)e Fo(buf)h Fp(is)h(a)f(p)q(oin)o(ter)458 1514 y(an)o(y)g(v)m(alid)i(base) e(MPI)g(datat)o(yp)q(e)g(or)g(an)o(y)g(prop)q(erly)h(constructed)f (user-de\014ned)458 1570 y Fo(OOMPI)c Fp(ob)s(ject,)h(whic)o(h)h(will)g (automatically)g(b)q(e)g(promoted)e(to)h Fo(OOMPI)p 1682 1570 V 16 w(Message)p Fp(.)458 1627 y(The)21 b(user)f(is)h(not)f (required)i(to)e(p)q(erform)g(a)g(cast.)36 b(An)21 b Fo(OOMPI)p 1591 1627 V 16 w(Request)f Fp(con-)458 1683 y(structed)15 b(from)f(the)i(MPI)f(request)g(generated)g(b)o(y)g Fo(MPI)p 1428 1683 V 17 w(Isend\(\))f Fp(is)i(returned.)458 1781 y Fo(OOMPI)p 581 1781 V 16 w(Request&)23 b(Issend\(OOMPI)p 1100 1781 V 16 w(Message)f(buf,)i(int)f(tag)h(=)f(OOMPI)p 1784 1781 V 17 w(NO)p 1849 1781 V 17 w(-)458 1838 y(TAG\).)13 b Fp(Calls)i Fo(MPI)p 779 1838 V 17 w(Issend\(\))p Fp(.)j(The)d (argumen)o(t)e Fo(buf)h Fp(is)g(of)g(an)o(y)g(v)m(alid)i(base)e(MPI)458 1894 y(datat)o(yp)q(e)h(or)g(an)o(y)h(prop)q(erly)g(constructed)g (user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)g(whic)o(h)458 1950 y(will)j(automatically)f(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1282 1950 V 17 w(Message)p Fp(.)23 b(The)17 b(user)g(is)g(not)f(re-)458 2007 y(quired)f(to)e(p)q(erform)g(a)h (cast.)19 b(An)14 b Fo(OOMPI)p 1165 2007 V 16 w(Request)f Fp(constructed)h(from)f(the)h(MPI)458 2063 y(request)h(generated)g(b)o (y)g Fo(MPI)p 961 2063 V 17 w(Issend\(\))f Fp(is)i(returned.)458 2161 y Fo(OOMPI)p 581 2161 V 16 w(Request&)23 b(Issend\(OOMPI)p 1100 2161 V 16 w(Array)p 1236 2161 V 16 w(message)g(buf,)g(int)h (count,)f(int)458 2218 y(tag)g(=)h(OOMPI)p 724 2218 V 16 w(NO)p 788 2218 V 17 w(TAG\).)d Fp(Calls)i Fo(MPI)p 1139 2218 V 17 w(Issend\(\))p Fp(.)38 b(The)22 b(argumen)o(t)g Fo(buf)f Fp(is)h(a)458 2274 y(p)q(oin)o(ter)16 b(an)o(y)f(v)m(alid)i (base)f(MPI)f(datat)o(yp)q(e)g(or)g(an)o(y)h(prop)q(erly)g(constructed) g(user-)458 2331 y(de\014ned)f Fo(OOMPI)f Fp(ob)s(ject,)f(whic)o(h)i (will)h(automatically)f(b)q(e)f(promoted)g(to)g Fo(OOMPI)p 1838 2331 V 16 w(-)458 2387 y(Message)p Fp(.)41 b(The)23 b(user)f(is)i(not)e(required)h(to)f(p)q(erform)h(a)f(cast.)42 b(An)23 b Fo(OOMPI)p 1838 2387 V 16 w(-)458 2444 y(Request)13 b Fp(constructed)h(from)g(the)g(MPI)g(request)g(generated)g(b)o(y)g Fo(MPI)p 1670 2444 V 17 w(Issend\(\))458 2500 y Fp(is)h(returned.)100 2619 y Fh(Receives)200 b Fo(OOMPI)p 581 2619 V 16 w(Port&)23 b(operator>>\(OOMPI)p 1124 2619 V 15 w(Message)g(buf\).)36 b Fp(Stream)20 b(in)o(terface)h(to)458 2675 y Fo(MPI)p 533 2675 V 16 w(Recv\(\))p Fp(.)29 b(The)18 b(argumen)o(t)f Fo(buf)h Fp(is)h(of)f(an)o(y)g(v)m(alid)h(base)g(MPI)f(datat)o(yp)q(e)f (or)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 37 37 37 36 bop 458 49 a Fp(an)o(y)20 b(prop)q(erly)i(constructed)f (user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)h(whic)o(h)h(will)g(auto-) 458 106 y(matically)c(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1110 106 15 2 v 16 w(Message)p Fp(.)25 b(The)18 b(user)f(is)h(not)f (required)i(to)458 162 y(p)q(erform)c(a)g(cast.)458 260 y Fo(OOMPI)p 581 260 V 16 w(Port&)23 b(operator>>\(OOMPI)p 1124 260 V 15 w(Array)p 1259 260 V 17 w(message)g(buf\).)i Fp(Stream)17 b(in)o(ter-)458 317 y(face)d(to)h Fo(MPI)p 680 317 V 16 w(Recv\(\))p Fp(.)k(The)c(argumen)o(t)f Fo(buf)g Fp(is)i(a)e(p)q(oin)o(ter)h(an)o(y)g(v)m(alid)h(base)f(MPI)458 373 y(datat)o(yp)q(e)g(or)g(an)o(y)h(prop)q(erly)g(constructed)g (user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)g(whic)o(h)458 429 y(will)j(automatically)f(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1282 429 V 17 w(Message)p Fp(.)23 b(The)17 b(user)g(is)g(not)f(re-)458 486 y(quired)g(to)f(p)q(erform)g(a)f(cast.) 458 584 y Fo(OOMPI)p 581 584 V 16 w(Status&)23 b(Recv\(OOMPI)p 1028 584 V 16 w(Message)g(buf,)g(int)h(tag)f(=)h(OOMPI)p 1713 584 V 16 w(NO)p 1777 584 V 17 w(TAG\).)458 640 y Fp(Calls)14 b Fo(MPI)p 645 640 V 17 w(Recv\(\))p Fp(.)k(The)c(argumen)o (t)e Fo(buf)h Fp(is)h(of)f(an)o(y)g(v)m(alid)i(base)e(MPI)g(datat)o(yp) q(e)458 697 y(or)18 b(an)o(y)g(prop)q(erly)i(constructed)e (user-de\014ned)j Fo(OOMPI)d Fp(ob)s(ject,)g(whic)o(h)i(will)g(au-)458 753 y(tomatically)f(b)q(e)h(promoted)e(to)g Fo(OOMPI)p 1156 753 V 17 w(Message)p Fp(.)30 b(The)19 b(user)g(is)g(not)g (required)458 810 y(to)d(p)q(erform)g(a)g(cast.)24 b(An)17 b Fo(OOMPI)p 1042 810 V 16 w(Status)f Fp(constructed)h(from)f(the)g (MPI)h(status)458 866 y(generated)e(b)o(y)g Fo(MPI)p 803 866 V 17 w(Recv\(\))f Fp(is)i(returned.)458 964 y Fo(OOMPI)p 581 964 V 16 w(Status&)23 b(Recv\(OOMPI)p 1028 964 V 16 w(Array)p 1164 964 V 16 w(message)g(buf,)h(int)f(count,)g (int)h(tag)458 1021 y(=)f(OOMPI)p 628 1021 V 17 w(NO)p 693 1021 V 17 w(TAG\).)15 b Fp(Stream)g(in)o(terface)h(to)f Fo(MPI)p 1315 1021 V 16 w(Send\(\))p Fp(.)21 b(The)16 b(argumen)o(t)e Fo(buf)458 1077 y Fp(is)k(a)g(p)q(oin)o(ter)h(an)o(y)e (v)m(alid)j(base)e(MPI)g(datat)o(yp)q(e)g(or)f(an)o(y)h(prop)q(erly)h (constructed)458 1134 y(user-de\014ned)25 b Fo(OOMPI)e Fp(ob)s(ject,)i(whic)o(h)g(will)g(automatically)f(b)q(e)g(promoted)g (to)458 1190 y Fo(OOMPI)p 581 1190 V 16 w(Message)p Fp(.)18 b(The)13 b(user)g(is)g(not)f(required)i(to)e(p)q(erform)g(a)g(cast.)19 b(An)13 b Fo(OOMPI)p 1838 1190 V 16 w(-)458 1246 y(Status)19 b Fp(constructed)h(from)f(the)h(MPI)f(status)g(generated)h(b)o(y)g Fo(MPI)p 1668 1246 V 17 w(Recv\(\))e Fp(is)458 1303 y(returned.)458 1401 y Fo(OOMPI)p 581 1401 V 16 w(Request&)23 b(Irecv\(OOMPI)p 1076 1401 V 16 w(Message)g(buf,)g(int)g(tag)h(=)g(OOMPI)p 1761 1401 V 16 w(NO)p 1825 1401 V 17 w(TAG\).)458 1457 y Fp(Calls)11 b Fo(MPI)p 642 1457 V 17 w(Irecv\(\))p Fp(.)18 b(The)11 b(argumen)o(t)f Fo(buf)h Fp(is)g(of)g(an)o(y)g(v)m (alid)h(base)f(MPI)g(datat)o(yp)q(e)458 1514 y(or)i(an)o(y)h(prop)q (erly)h(constructed)g(user-de\014ned)h Fo(OOMPI)d Fp(ob)s(ject,)h(whic) o(h)h(will)g(auto-)458 1570 y(matically)j(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1110 1570 V 16 w(Message)p Fp(.)25 b(The)18 b(user)f(is)h(not)f(required)i(to)458 1627 y(p)q(erform)e(a)g(cast.)26 b(An)18 b Fo(OOMPI)p 990 1627 V 16 w(Request)f Fp(constructed)g(from)g (the)h(MPI)f(request)458 1683 y(generated)e(b)o(y)g Fo(MPI)p 803 1683 V 17 w(Recv)p 916 1683 V 16 w(init\(\))g Fp(is)h(returned.)458 1781 y Fo(OOMPI)p 581 1781 V 16 w(Request&)23 b(Irecv\(OOMPI)p 1076 1781 V 16 w(Array)p 1212 1781 V 16 w(message)g(buf,)g(int)h (count,)f(int)g(tag)458 1838 y(=)g(OOMPI)p 628 1838 V 17 w(NO)p 693 1838 V 17 w(TAG\).)17 b Fp(Calls)h Fo(MPI)p 1035 1838 V 16 w(Irecv\(\))p Fp(.)26 b(The)18 b(argumen)o(t)e Fo(buf)h Fp(is)h(a)f(p)q(oin)o(ter)458 1894 y(an)o(y)g(v)m(alid)i(base) e(MPI)g(datat)o(yp)q(e)g(or)g(an)o(y)g(prop)q(erly)h(constructed)f (user-de\014ned)458 1950 y Fo(OOMPI)c Fp(ob)s(ject,)h(whic)o(h)h(will)g (automatically)g(b)q(e)g(promoted)e(to)h Fo(OOMPI)p 1682 1950 V 16 w(Message)p Fp(.)458 2007 y(The)21 b(user)f(is)h(not)f (required)i(to)e(p)q(erform)g(a)g(cast.)36 b(An)21 b Fo(OOMPI)p 1591 2007 V 16 w(Request)f Fp(con-)458 2063 y(structed)i(from)f(the)i(MPI)f(request)g(generated)g(b)o(y)g Fo(MPI)p 1477 2063 V 17 w(Recv)p 1590 2063 V 17 w(init\(\))f Fp(is)i(re-)458 2120 y(turned.)458 2218 y Fo(OOMPI)p 581 2218 V 16 w(Request&)g(Recv)p 908 2218 V 17 w(init\(OOMPI)p 1165 2218 V 15 w(Message)g(buf,)h(int)f(tag)h(=)f(OOMPI)p 1849 2218 V 17 w(-)458 2274 y(NO)p 509 2274 V 17 w(TAG\).)10 b Fp(Calls)i Fo(MPI)p 838 2274 V 17 w(Recv)p 951 2274 V 17 w(init\(\))p Fp(.)17 b(The)12 b(argumen)o(t)e Fo(buf)h Fp(is)h(of)f(an)o(y)g(v)m(alid)i(base)458 2331 y(MPI)19 b(datat)o(yp)q(e)f(or)h(an)o(y)g(prop)q(erly)h(constructed)f (user-de\014ned)i Fo(OOMPI)d Fp(ob)s(ject,)458 2387 y(whic)o(h)h(will)h (automatically)f(b)q(e)g(promoted)f(to)g Fo(OOMPI)p 1425 2387 V 16 w(Message)p Fp(.)29 b(The)19 b(user)f(is)458 2444 y(not)12 b(required)h(to)e(p)q(erform)h(a)g(cast.)19 b(An)12 b Fo(OOMPI)p 1272 2444 V 17 w(Request)f Fp(constructed)h(from)g (the)458 2500 y(MPI)j(request)g(generated)g(b)o(y)g Fo(MPI)p 1065 2500 V 17 w(Recv)p 1178 2500 V 17 w(init\(\))f Fp(is)i(returned.) 458 2598 y Fo(OOMPI)p 581 2598 V 16 w(Request&)23 b(Recv)p 908 2598 V 17 w(init\(OOMPI)p 1165 2598 V 15 w(Array)p 1300 2598 V 17 w(message)g(buf,)g(int)h(count,)f(int)458 2654 y(tag)g(=)h(OOMPI)p 724 2654 V 16 w(NO)p 788 2654 V 17 w(TAG\).)14 b Fp(Calls)h Fo(MPI)p 1124 2654 V 17 w(Recv)p 1237 2654 V 16 w(init\(\))p Fp(.)k(The)c(argumen)o(t)e Fo(buf)h Fp(is)h(a)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 38 38 38 37 bop 458 49 a Fp(p)q(oin)o(ter)16 b(an)o(y)f(v)m(alid)i(base)f (MPI)f(datat)o(yp)q(e)g(or)g(an)o(y)h(prop)q(erly)g(constructed)g (user-)458 106 y(de\014ned)f Fo(OOMPI)f Fp(ob)s(ject,)f(whic)o(h)i (will)h(automatically)f(b)q(e)f(promoted)g(to)g Fo(OOMPI)p 1838 106 15 2 v 16 w(-)458 162 y(Message)p Fp(.)j(The)c(user)f(is)g (not)g(required)h(to)e(p)q(erform)g(a)h(cast.)18 b(An)12 b Fo(OOMPI)p 1694 162 V 17 w(Request)458 219 y Fp(constructed)i(from)g (the)h(MPI)g(request)f(generated)h(b)o(y)g Fo(MPI)p 1493 219 V 16 w(Recv)p 1605 219 V 17 w(init\(\))f Fp(is)h(re-)458 275 y(turned.)100 394 y Fh(See)h(Also)196 b Fo(OOMPI)p 581 394 V 16 w(Comm)p Fp(,)10 b Fo(OOMPI)p 836 394 V 17 w(Intra)p 973 394 V 16 w(comm)p Fp(,)h Fo(OOMPI)p 1229 394 V 16 w(Inter)p 1365 394 V 17 w(comm)p Fp(,)f Fo(OOMPI)p 1621 394 V 16 w(Message)p Fp(,)g Fo(OOMPI)p 1948 394 V 17 w(-)458 450 y(Array)p 581 450 V 16 w(message)p Fp(,)k Fo(OOMPI)p 912 450 V 17 w(Request)p Fp(,)f Fo(OOMPI)p 1243 450 V 17 w(Status)p Fp(,)h Fo(OOMPI)p 1551 450 V 16 w(User)p 1663 450 V 17 w(type)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 39 39 39 38 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Request)100 168 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 224 y(class)h(OOMPI_Request)100 343 y Fh(Description)145 b Fp(A)14 b(class)h(for)f(encapsulating)i(the)f Fo(MPI)p 1119 343 V 17 w(Request)e Fp(handles)j(and)f(their)g(asso)q (ciated)458 400 y(functionalit)o(y)l(.)458 580 y Fj(Discussion)o(:)h Fi(W)m(e)e(will)e(expand)i(on)g(this.)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 40 40 40 39 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(Status)100 168 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 224 y(class)h(OOMPI_Status)100 343 y Fh(Description)145 b Fp(A)17 b(class)h(for)e(encapsulating)j(the)e Fo(MPI)p 1132 343 V 17 w(Status)f Fp(handles)j(and)e(their)h(asso)q (ciated)458 400 y(functionalit)o(y)l(.)458 580 y Fj(Discussion)o(:)e Fi(W)m(e)e(will)e(expand)i(on)g(this.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 41 41 41 40 bop 100 49 a Fh(Name)247 b Fo(OOMPI)p 581 49 15 2 v 16 w(User)p 693 49 V 17 w(type)100 168 y Fh(Decla)o(ration)142 b Fo(#include)22 b("oompi.h")458 224 y(class)h(OOMPI_User_type)100 343 y Fh(Description)145 b Fp(A)18 b(base)g(class)g(for)g(creating)g (user-de\014ned)i Fo(OOMPI)d Fp(data)g(ob)s(jects.)28 b(Classes)18 b(de-)458 400 y(riv)o(ed)g(from)e Fo(OOMPI)p 806 400 V 17 w(User)p 919 400 V 16 w(type)h Fp(can)h(immediately)g(use) g(existing)g Fo(OOMPI)f Fp(com-)458 456 y(m)o(unication)k(functions,)h (pro)o(vided)f(the)f(user-de\014ned)i(t)o(yp)q(e)e(is)h(prop)q(erly)g (con-)458 513 y(structed.)458 693 y Fj(Discussion)o(:)16 b Fi(W)m(e)e(will)e(expand)i(on)g(this.)-32 46 y Fm(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 42 42 42 41 bop 75 49 a Fe(A.4.1)49 b(Ackno)o(wledgements)75 135 y Fp(John)11 b(J.)g(T)l(ran)f(\(Univ)o(ersit)o(y)h(of)g(Notre)f (Dame,)h(Departmen)o(t)f(of)g(Computer)g(Science)j(and)e(Engineering\)) 75 192 y(assisted)k(in)h(the)g(OOMPI)g(implemen)o(tation)g(and)f (design.)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 43 43 43 42 bop 75 283 a Fr(Index)75 495 y Fp(MPI)p 167 495 14 2 v 16 w(Ab)q(ort,)15 b(16)75 551 y(MPI)p 167 551 V 16 w(Allgather,)h(25)75 607 y(MPI)p 167 607 V 16 w(Allgatherv,)g(26) 75 664 y(MPI)p 167 664 V 16 w(Allreduce,)h(26)75 720 y(MPI)p 167 720 V 16 w(Alltoall,)g(26)75 777 y(MPI)p 167 777 V 16 w(Alltoallv,)g(27)75 833 y(MPI)p 167 833 V 16 w(A)o(ttr)p 270 833 V 16 w(delete,)f(15)75 890 y(MPI)p 167 890 V 16 w(A)o(ttr)p 270 890 V 16 w(get,)e(15)75 946 y(MPI)p 167 946 V 16 w(A)o(ttr)p 270 946 V 16 w(put,)h(15)75 1003 y(MPI)p 167 1003 V 16 w(Barrier,)g(27)75 1059 y(MPI)p 167 1059 V 16 w(Bcast,)g(31)75 1116 y(MPI)p 167 1116 V 16 w(Bsend,)h(33,)e(34)75 1172 y(MPI)p 167 1172 V 16 w(Bsend)p 303 1172 V 18 w(init,)i(34)75 1228 y(MPI)p 167 1228 V 16 w(Cart)p 275 1228 V 16 w(co)q(ords,)e(14)75 1285 y(MPI)p 167 1285 V 16 w(Cart)p 275 1285 V 16 w(create,)g(14)75 1341 y(MPI)p 167 1341 V 16 w(Cart)p 275 1341 V 16 w(get,)g(14)75 1398 y(MPI)p 167 1398 V 16 w(Cart)p 275 1398 V 16 w(map,)g(14)75 1454 y(MPI)p 167 1454 V 16 w(Cart)p 275 1454 V 16 w(rank,)g(14)75 1511 y(MPI)p 167 1511 V 16 w(Cart)p 275 1511 V 16 w(shift,)h(14)75 1567 y(MPI)p 167 1567 V 16 w(Cart)p 275 1567 V 16 w(sub,)g(14)75 1624 y(MPI)p 167 1624 V 16 w(Cartdim)p 350 1624 V 17 w(get,)f(14)75 1680 y(MPI)p 167 1680 V 16 w(Comm)p 315 1680 V 16 w(compare,)h(15)75 1737 y(MPI)p 167 1737 V 16 w(Comm)p 315 1737 V 16 w(create,)g(25)75 1793 y(MPI)p 167 1793 V 16 w(Comm)p 315 1793 V 16 w(dup,)h(14,)e(15)75 1849 y(MPI)p 167 1849 V 16 w(Comm)p 315 1849 V 16 w(free,)h(14,)f(15,)g (24,)h(25)75 1906 y(MPI)p 167 1906 V 16 w(Comm)p 315 1906 V 16 w(rank,)g(16)75 1962 y(MPI)p 167 1962 V 16 w(Comm)p 315 1962 V 16 w(remote)p 468 1962 V 16 w(group,)f(24)75 2019 y(MPI)p 167 2019 V 16 w(Comm)p 315 2019 V 16 w(remote)p 468 2019 V 16 w(size,)i(24)75 2075 y(MPI)p 167 2075 V 16 w(Comm)p 315 2075 V 16 w(size,)g(16)75 2132 y(MPI)p 167 2132 V 16 w(Comm)p 315 2132 V 16 w(split,)g(25)75 2188 y(MPI)p 167 2188 V 16 w(Dims)p 286 2188 V 17 w(create,)e(27)75 2245 y(MPI)p 167 2245 V 16 w(Errhandler)p 398 2245 V 17 w(get,)h(16)75 2301 y(MPI)p 167 2301 V 16 w(Errhandler)p 398 2301 V 17 w(set,)g(16)75 2358 y(MPI)p 167 2358 V 16 w(Finalize,)i(17)75 2414 y(MPI)p 167 2414 V 16 w(Finalize\(\),)f(17) 75 2470 y(MPI)p 167 2470 V 16 w(Gather,)e(31)75 2527 y(MPI)p 167 2527 V 16 w(Gatherv,)g(32)75 2583 y(MPI)p 167 2583 V 16 w(Graph)p 310 2583 V 16 w(create,)h(21)75 2640 y(MPI)p 167 2640 V 16 w(Graph)p 310 2640 V 16 w(get,)g(21)75 2696 y(MPI)p 167 2696 V 16 w(Graph)p 310 2696 V 16 w(map,)g(21)1048 495 y(MPI)p 1140 495 V 16 w(Graph)p 1283 495 V 16 w(neigh)o(b)q(ors,)h (21)1048 551 y(MPI)p 1140 551 V 16 w(Graph)p 1283 551 V 16 w(neigh)o(b)q(ors)p 1488 551 V 17 w(coun)o(t,)f(21)1048 607 y(MPI)p 1140 607 V 16 w(Graphdims)p 1376 607 V 17 w(get,)f(21)1048 664 y(MPI)p 1140 664 V 16 w(Ibsend,)i(35,)e(36)1048 720 y(MPI)p 1140 720 V 16 w(Init,)i(17)1048 777 y(MPI)p 1140 777 V 16 w(Initialized,)i(16)1048 833 y(MPI)p 1140 833 V 16 w(In)o(tercomm)p 1371 833 V 16 w(create,)d(24)1048 890 y(MPI)p 1140 890 V 16 w(In)o(tercomm)p 1371 890 V 16 w(merge,)g(24)1048 946 y(MPI)p 1140 946 V 16 w(Irecv,)g(37)1048 1003 y(MPI)p 1140 1003 V 16 w(Irsend,)h(36)1048 1059 y(MPI)p 1140 1059 V 16 w(Isend,)g(36)1048 1116 y(MPI)p 1140 1116 V 16 w(Issend,)g(36)1048 1172 y(MPI)p 1140 1172 V 16 w(Op,)f(32)1048 1228 y(MPI)p 1140 1228 V 16 w(Op)p 1216 1228 V 17 w(create,)g(28)1048 1285 y(MPI)p 1140 1285 V 16 w(Op)p 1216 1285 V 17 w(free,)g(28)1048 1341 y(MPI)p 1140 1341 V 16 w(P)o(ac)o(k,)f(30)1048 1398 y(MPI)p 1140 1398 V 16 w(P)o(ac)o(k)p 1252 1398 V 16 w(size,)h(16,)g(29)1048 1454 y(MPI)p 1140 1454 V 16 w(Recv,)h(36,)e(37) 1048 1511 y(MPI)p 1140 1511 V 16 w(Recv)p 1253 1511 V 17 w(init,)i(37,)e(38)1048 1567 y(MPI)p 1140 1567 V 16 w(Reduce,)i(28,)f(32)1048 1624 y(MPI)p 1140 1624 V 16 w(Reduce)p 1299 1624 V 18 w(scatter,)f(27)1048 1680 y(MPI)p 1140 1680 V 16 w(Rsend,)i(33)1048 1737 y(MPI)p 1140 1737 V 16 w(Rsend)p 1277 1737 V 17 w(init,)g(34,)f(35)1048 1793 y(MPI)p 1140 1793 V 16 w(Scan,)g(27)1048 1849 y(MPI)p 1140 1849 V 16 w(Scatter,)f(32)1048 1906 y(MPI)p 1140 1906 V 16 w(Scatterv,)g(32,)h(33)1048 1962 y(MPI)p 1140 1962 V 16 w(Send,)h(18,)e(33,)g(34,)g(37)1048 2019 y(MPI)p 1140 2019 V 16 w(Send)p 1251 2019 V 17 w(init,)i(35)1048 2075 y(MPI)p 1140 2075 V 16 w(Ssend,)g(34)1048 2132 y(MPI)p 1140 2132 V 16 w(Ssend)p 1269 2132 V 17 w(init,)g(35)1048 2188 y(MPI)p 1140 2188 V 16 w(T)l(op)q(o)p 1257 2188 V 16 w(test,)f(27)1048 2245 y(MPI)p 1140 2245 V 16 w(T)o(yp)q(e)p 1258 2245 V 17 w(comit,)g(19)1048 2301 y(MPI)p 1140 2301 V 16 w(T)o(yp)q(e)p 1258 2301 V 17 w(commit,)f(19,)h(20)1048 2358 y(MPI)p 1140 2358 V 16 w(T)o(yp)q(e)p 1258 2358 V 17 w(con)o(tiguous,)g(19)1048 2414 y(MPI)p 1140 2414 V 16 w(T)o(yp)q(e)p 1258 2414 V 17 w(exten)o(t,)f(18)1048 2470 y(MPI)p 1140 2470 V 16 w(T)o(yp)q(e)p 1258 2470 V 17 w(free,)h(18)1048 2527 y(MPI)p 1140 2527 V 16 w(T)o(yp)q(e)p 1258 2527 V 17 w(hindexed,)h(19)1048 2583 y(MPI)p 1140 2583 V 16 w(T)o(yp)q(e)p 1258 2583 V 17 w(h)o(v)o(ector,)e(19)1048 2640 y(MPI)p 1140 2640 V 16 w(T)o(yp)q(e)p 1258 2640 V 17 w(index,)i(19)1048 2696 y(MPI)p 1140 2696 V 16 w(T)o(yp)q(e)p 1258 2696 V 17 w(indexed,)g(19)952 2828 y(43)p eop %%Page: 44 44 44 43 bop 75 49 a Fp(MPI)p 167 49 14 2 v 16 w(T)o(yp)q(e)p 285 49 V 17 w(lb,)16 b(18)75 106 y(MPI)p 167 106 V 16 w(T)o(yp)q(e)p 285 106 V 17 w(size,)g(18)75 162 y(MPI)p 167 162 V 16 w(T)o(yp)q(e)p 285 162 V 17 w(struct,)e(20)75 219 y(MPI)p 167 219 V 16 w(T)o(yp)q(e)p 285 219 V 17 w(ub,)h(19)75 275 y(MPI)p 167 275 V 16 w(T)o(yp)q(e)p 285 275 V 17 w(v)o(ector,)f(19)75 332 y(MPI)p 167 332 V 16 w(Unpac)o(k,)h(30)75 429 y(OOMPI)p 237 429 V 17 w(Cart)p 346 429 V 15 w(comm,)g(14)75 486 y(OOMPI)p 237 486 V 17 w(Comm,)f(15)75 542 y(OOMPI)p 237 542 V 17 w(Comm)p 386 542 V 16 w(w)o(orld,)h(17)75 599 y(OOMPI)p 237 599 V 17 w(Datat)o(yp)q(e,)f(18)75 655 y(OOMPI)p 237 655 V 17 w(Graph)p 381 655 V 16 w(comm,)h(21)75 712 y(OOMPI)p 237 712 V 17 w(Group,)g(22)75 768 y(OOMPI)p 237 768 V 17 w(In)o(ter)p 350 768 V 17 w(comm,)f(24)75 825 y(OOMPI)p 237 825 V 17 w(In)o(tra)p 353 825 V 16 w(comm,)h(25)75 881 y(OOMPI)p 237 881 V 17 w(Op,)g(28)75 938 y(OOMPI)p 237 938 V 17 w(P)o(ac)o(k)o(ed,)f(29)75 994 y(OOMPI)p 237 994 V 17 w(P)o(ort,)g(31)75 1050 y(OOMPI)p 237 1050 V 17 w(Request,)i(39)75 1107 y(OOMPI)p 237 1107 V 17 w(Status,)e(40)75 1163 y(OOMPI)p 237 1163 V 17 w(User)p 344 1163 V 16 w(t)o(yp)q(e,)h(41)1967 46 y Fm(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Wed Oct 25 22:40:17 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA11558; Wed, 25 Oct 1995 22:40:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA01348 for mpi-core-out; Wed, 25 Oct 1995 21:40:27 -0500 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA10791; Mon, 23 Oct 1995 12:32:47 -0500 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.18.106]) by franklin.sdsc.edu (8.6.11/SDSCserver-7) with SMTP id KAA07134; Mon, 23 Oct 1995 10:32:41 -0700 Received: by rogue.sdsc.edu (940406.SGI/1.11-client) id AA17525; Mon, 23 Oct 95 10:32:41 -0700 Date: Mon, 23 Oct 1995 10:32:40 +0059 (PDT) From: Richard Frost Reply-To: Richard Frost Subject: mpi-io, mpi-dynamic generalizations To: mpi-io@nas.nasa.gov Cc: mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov, Richard Frost Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi folks, I'm working on a project for ARPA and IBM involving the integration of applications, web-browsers, databases (DB2 & Illustra), and archival storage (Unitree & HPSS). MPI will play a key role and I hope to leverage from MPI-IO as well. In particular, we are driven by the use of MPI derived data types. The following 2 components are necessary to complete a prototype system using MPI. I'm interested in learning about similar efforts before defining an API. 1. An on-the-wire protocol for transmiting derived data types, and a mechanism for run-time parsing of these types on the receiving end. For example, consider a parallel DBMS responding to a request from a parallel computation. We would like the DBMS to store the derived data type as metadata for reuse at a later run time instead of linking the DBMS to an intractable number of hardcoded data types. (Yes, this is an MPI/SQL interface.) In particular, DB2 would need an MPI-IO-like module (MDIO) for interfacing with computational applications and browsers; and another module for interfacing with archival storage. SQL transactions would only occur between the DBMS modules and the DBMS. 2. A generalization of files; i.e., we need to overload MPI-IO's concept of files so that we can perform I/O transfers between agents (databases, web-browsers, archival storage systems) along with actual file systems. For example, a file might sometimes be designated by a Unix file descriptor, and at other times an external communicator containing an arbitrary number of processes. We need to facilitate M-to-N I/O transactions between arbitrary information sources and consumers in a way that gives applications a simplified view of open, read, write, close. See the "Example system architecture" on http://www.sdsc.edu/EnablingTech/MassDataAnal/MassDataAnal.html for a diagram. Thanks, Richard Frost SDSC From mpi-core-human@mcs.anl.gov Wed Oct 25 22:45:41 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA11608; Wed, 25 Oct 1995 22:45:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA01439 for mpi-core-out; Wed, 25 Oct 1995 21:46:04 -0500 Received: from caffeine.avs.com (caffeine.avs.com [192.138.253.107]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA13405 for ; Mon, 23 Oct 1995 14:26:07 -0500 Received: from darkstar.avs.com by caffeine.avs.com (5.x/SMI-SVR4) id AA26915; Mon, 23 Oct 1995 15:23:19 -0400 Received: by darkstar.avs.com (5.x/SMI-SVR4) id AA08229; Mon, 23 Oct 1995 15:21:22 -0400 Date: Mon, 23 Oct 1995 15:21:22 -0400 Message-Id: <9510231921.AA08229@darkstar.avs.com> From: Gary Oberbrunner To: mpi-core@mcs.anl.gov Subject: MPI_WAIT() and select()? Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Has anyone got any advice on waiting for events with select() and MPI_WAIT() simultaneously (where either may return with data at any time)? Is there anything better than using a select() with a small timeout and calling MPI_TEST() at each timeout? Does anyone care about that kind of functionality? (Yes, I know it's not portable to anything that doesn't support select()). thanks, -- Gary Oberbrunner garyo@avs.com Advanced Visual Systems, Inc. http://www.avs.com/~garyo 300 Fifth Avenue (617)890-8192 x2133 TEL Waltham, MA 02154 (617)890-2887 FAX From mpi-core-human@mcs.anl.gov Wed Oct 25 22:55:05 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA11627; Wed, 25 Oct 1995 22:55:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA01560 for mpi-core-out; Wed, 25 Oct 1995 21:55:24 -0500 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA19013; Mon, 23 Oct 1995 19:15:50 -0500 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.18.106]) by franklin.sdsc.edu (8.6.11/SDSCserver-7) with SMTP id RAA03065; Mon, 23 Oct 1995 17:09:46 -0700 Received: by rogue.sdsc.edu (940406.SGI/1.11-client) id AA17831; Mon, 23 Oct 95 17:09:46 -0700 Date: Mon, 23 Oct 1995 17:09:45 +0059 (PDT) From: Richard Frost Subject: Re: mpi-io, mpi-dynamic generalizations To: mpi-io@nas.nasa.gov, mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Thanks for the quick responses and discussion. Here's some detail, and an MPI_2 proposal. Perhaps there are other approaches? client ... MPI_Type_commit( &utype ); ... MDIO_write( server, utype ); ... server MDIO_read( client, sometype ); When "server" has no previous knowledge of "utype" (the client is a new application), then "MDIO_write" need to send a facimile of the utype struct (not the data) so that "MDIO_read" can instantiate one. In MPI_1, MPI_Datatype_"whatever"() stores no information about the data types, just relative binary addressing. This in effect assumes that the code defining a type is compiled for all participants. Not so in dynamic information services. If not already proposed in MPI_2, I propose (and will implement) the following: ~MPI_Type_Attribute( MPI_Datatype utype, MPI_DataAttr description ) where description is an array of structures, one for each element of utype. For example elem.type -- a previously defined MPI type elem.label -- a descriptive string, perhaps "Content/Type keyword(s) ..." elem.label_length The data generated here would be either inserted in the existing (uncommitted) type, or (barf) placed in a separate structure. The former affords more transparency to the user. In either case, the type can be transmitted down the wire. A receiver could rebuild and instantiate the type without knowing the application source code. All of the MPI and MPI-IO functionality can then be brought into play. This is an extremely useful capability when performing 3rd party transfers. I'm interested in knowing what other kinds of attributes are useful to users and library writers. Richard Frost SDSC From mpi-core-human@mcs.anl.gov Wed Oct 25 23:01:22 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA11636; Wed, 25 Oct 1995 23:01:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA01620 for mpi-core-out; Wed, 25 Oct 1995 22:00:52 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA01607; Wed, 25 Oct 1995 22:00:42 -0500 Message-Id: <199510260300.WAA01607@antares.mcs.anl.gov> To: Gary Oberbrunner cc: mpi-core@mcs.anl.gov Subject: Re: MPI_WAIT() and select()? In-Reply-to: Message from Gary Oberbrunner of "Mon, 23 Oct 1995 15:21:22 -0400." <9510231921.AA08229@darkstar.avs.com> Date: Wed, 25 Oct 1995 22:00:40 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | Has anyone got any advice on waiting for events with select() and | MPI_WAIT() simultaneously (where either may return with data at any | time)? Is there anything better than using a select() with a small | timeout and calling MPI_TEST() at each timeout? Does anyone care | about that kind of functionality? (Yes, I know it's not portable to | anything that doesn't support select()). There is certainly nothing general that is any better than what you suggest, but I've been thinking for a long time that an optional interface to the Unix file interface (since all devices are files in Unix) would make sense, just as it did for X11. One possibility would be to register an fd and user-callback with MPI; another would be to do the reverse (getting an fdmask from MPI and an MPI service routine to call for one of those). Getting the list of restrictions right would require care. Of course, the thread crowd would say to just do a blocking wait in separate threads :) Bill From mpi-core-human@mcs.anl.gov Thu Nov 2 18:09:56 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA00591; Thu, 2 Nov 1995 18:09:54 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA04324 for mpi-core-out; Thu, 2 Nov 1995 17:07:43 -0600 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA04308 for ; Thu, 2 Nov 1995 17:06:58 -0600 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 5681; Thu, 02 Nov 95 18:06:13 EST Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.02 on VAGENT2" id 3415; Thu, 2 Nov 1995 18:06:13 EST Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2Rx) with TCP; Thu, 02 Nov 95 18:06:12 EST Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/950830) id AA33100; Thu, 2 Nov 1995 18:05:31 -0500 Message-Id: <9511022305.AA33100@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Subject: pointers in Fortran Reply-To: snir@watson.ibm.com Date: Thu, 02 Nov 1995 18:05:31 -0500 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: I would like to understand how "F77 pointers" are supported by various Fortran compilers. With the IBM XLF compiler, I could have code that looks like that REAL A POINTER (P, A(100,100)) ! no memory is allocated; P is really an INTEGER CALL MPI_RMC_MALLOC(P, 4*100*100,...) ! allocates 40,000 bytes and return the address of the allocated buffer in P. ! at that point, memory was allocated for array A A(2,3) = 57.5 .... The MPI_RMC_MALLOC can, under the cover, calls a C malloc, or shmget, or what have you -- it just has to return in P the address of the allocated buffer. How portable would be such code? How is similar code written with other Fortran compilers? Note that Fortran 90 pointers will not do here. ------------------- Marc Snir IBM T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 email: snir@watson.ibm.com phone: 914-945-3204 fax: 914-945-4425 From mpi-core-human@mcs.anl.gov Mon Nov 6 18:20:00 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA07171; Mon, 6 Nov 1995 18:19:59 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA12088 for mpi-core-out; Mon, 6 Nov 1995 17:18:16 -0600 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA12076 for ; Mon, 6 Nov 1995 17:18:03 -0600 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.7.1/8.6.12.1) with ESMTP id SAA29676 for ; Mon, 6 Nov 1995 18:14:52 -0500 (EST) Received: from b125.super.org (b125 [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with ESMTP id SAA00467 for ; Mon, 6 Nov 1995 18:17:31 -0500 From: lederman@super.org (Steve Huss-Lederman) Received: (lederman@localhost) by b125.super.org (8.6.12/8.6.12.client) id SAA10321; Mon, 6 Nov 1995 18:17:30 -0500 Date: Mon, 6 Nov 1995 18:17:30 -0500 Message-Id: <199511062317.SAA10321@b125.super.org> To: mpi-core@mcs.anl.gov Subject: document due for SC95 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hello All! I would like to have the input for the final pass of the MPI-2 draft document by Monday, November 20. This is a few days earlier than previously announced, but due to my vacation plans over Thanksgiving, I would like to do this. If someone cannot make this deadline, PLEASE let me know now. I would prefer to get input that the whole group has had a chance to read since this is going to be publically announced. To give people time to read things, it would be best to get the chapters distributed in the next week or so. Thanks, Steve From mpi-core-human@mcs.anl.gov Sun Nov 12 19:44:56 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA16169; Sun, 12 Nov 1995 19:44:56 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA12859 for mpi-core-out; Sun, 12 Nov 1995 18:40:23 -0600 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA12851 for ; Sun, 12 Nov 1995 18:40:14 -0600 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 2783; Sun, 12 Nov 95 19:39:28 EST Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.02 on VAGENT2" id 2663; Sun, 12 Nov 1995 19:39:28 EST Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2Rx) with TCP; Sun, 12 Nov 95 19:39:28 EST Received: by snir.watson.ibm.com (AIX 3.2/UCB 5.64/950830) id AA20519; Sun, 12 Nov 1995 19:38:12 -0500 Date: Sun, 12 Nov 1995 19:38:12 -0500 From: snir@watson.ibm.com (Marc Snir) Message-Id: <9511130038.AA20519@snir.watson.ibm.com> To: mpi-core@mcs.anl.gov Subject: a new datatype constructor for structures Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk struct.list From mpi-core-human@mcs.anl.gov Mon Nov 13 09:42:15 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA29132; Mon, 13 Nov 1995 09:42:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA18550 for mpi-core-out; Mon, 13 Nov 1995 08:39:34 -0600 Received: from watson.ibm.com (watson.ibm.com [129.34.139.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA18537 for ; Mon, 13 Nov 1995 08:39:24 -0600 Received: from WATSON by watson.ibm.com (IBM VM SMTP V2R3) with BSMTP id 8079; Mon, 13 Nov 95 09:38:37 EST Received: from YKTVMV by watson.vnet.ibm.com with "VAGENT.V1.02 on VAGENT2" id 2069; Mon, 13 Nov 1995 09:38:36 EST Received: from snir.watson.ibm.com by yktvmv.watson.ibm.com (IBM VM SMTP V2Rx) with TCP; Mon, 13 Nov 95 09:38:36 EST Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/950830) id AA27774; Mon, 13 Nov 1995 09:37:18 -0500 Message-Id: <9511131437.AA27774@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Subject: My apology for too succint previous message (mailers...) Reply-To: snir@watson.ibm.com Date: Mon, 13 Nov 1995 09:37:17 -0500 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk :-) :-) :-) *** (-: (-: (-: Portable datatypes In the current MPI definition of datatypes we have convertible datatypes: Those are the basic datatypes, and the derived datatypes defined with MPI_TYPE_CONTIGUOUS, MPI_TYPE_VECTOR and MPI_TYPE_INDEXED. One can give a portable, architecture independent, description of such datatypes. This, because all displacements in such datatypes are defined as multiples of the extent of a basic MPI datatype. Given such a datatype, defined on one architecture, then the corresponding datatype for another architecture is obtained by scaling all displacements by the ratio between the extent of the basic datatype on these two architectures. On the other hand, datatypes defined with MPI_TYPE_HVECTOR, MPI_TYPE_HINDEXED and MPI_TYPE_STRUCT are not convertible. They incorporate in their definition byte displacements, and there are no obvious rules for mapping byte displacements on one architecture into byte displacements on another architecture. We cannot expect that strange data layout definitions be convertible, in the general case. However, it would be nice that data layouts that correspond to variable types that can be defined in C and Fortran be convertible -- after all the language types are convertible. The main gap is due to the lack of a convertible datatype that corresponds to C structures: MPI_TYPE_STRUCT uses byte displacements. I propose to add a new datatype constructor that can be used to construct a convertible datatype definition for C structures. MPI_TYPE_RECORD(count, blocklengths, datatypes, newtype) creates a newtype that is obtained by concatenating blocklengths[0] copies of datatypes[0], followed by blocklengths[1] copies of datatypes[1], ...., followed by blocklengths[count-1] copies of datatypes[count-1], with padding added for alignment. What do we solve with this proposal? 1. A convertible definition of datatypes for structures: the definition need not speak about byte displacements. This may solve the problem of 3rd party datatypes that was debated of late. In fact, any derived datatype constructed by MPI_TYPE_CONTIGUOUS, MPI_TYPE_VECTOR, MPI_TYPE_INDEXED and MPI_TYPE_RECORD is convertible. 2. Ease of use. If we have a C structure defined, say as struct { char[5] c; float[3] f; int[7] i; } then, the corresponding datatype is defined by the code int blen[3] = {5,3,7}; MPI_Datatype types[3] = {MPI_CHAR, MPI_FLOAT, MPI_INT}; MPI_Datatype newtype; MPI_Type_record(3, blen, types, newtype); Much simpler and more natural then the current way that uses MPI_TYPE_STRUCT. 3. The problem of holes in structure (can we overwrite them?) If we receive data into a buffer defined with MPI_TYPE_RECORD, then holes can be overwritten. I.e., if the user define a data layout using MPI_TYPE_RECORD, then padding spaces in between "contiguous" fields are assumed not to contain significant data. Suppose that one has a reasonable compiler that always aligns basic variables at natural boundaries. Suppose one send down the wire data with padding, so that variables are aligned at natural boundaries. Then, if data is received into a receive buffer defined by MPI_TYPE_RECORD, data can be written contiguously in memory, form the wire, padding included (on a homogeneous system). What is not solved by this proposal? The problem of compilers where different alignment rules can be used within one source: effectively, such compiler allow to switch "architecture" midstream, and there is little we can do. The problem of compilers that do not align using one single alignment rule for each type of variable. E.g., a compiler that align doubles at 8 byte boundaries if they appear alone, but only at 4 byte boundaries if they are mixed with floats. In such case, the alignment rules are context sensitive, and one cannot derive the right alignment for a subsegment of a structure just from a description of the content of this subsegment. ------------------- Marc Snir IBM T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 email: snir@watson.ibm.com phone: 914-945-3204 fax: 914-945-4425 From mpi-core-human@mcs.anl.gov Mon Nov 13 12:05:30 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA01840; Mon, 13 Nov 1995 12:05:26 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA22759 for mpi-core-out; Mon, 13 Nov 1995 11:05:46 -0600 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA22741 for ; Mon, 13 Nov 1995 11:05:21 -0600 Received: by felix.dircon.co.uk id AA02115 (5.67b/IDA-1.5 for ); Mon, 13 Nov 1995 17:05:14 GMT Message-Id: <199511131705.AA02115@felix.dircon.co.uk> Received: from al037.du.pipex.com(193.130.251.37) by amnesiac via smap (V1.3) id sma002107; Mon Nov 13 17:05:06 1995 Received: by jim (5.x) id AA01205; Mon, 13 Nov 1995 16:03:39 GMT To: mpi-core@mcs.anl.gov Subject: Re: My apology for too succint previous message (mailers...) In-Reply-To: Your message of "Mon, 13 Nov 1995 09:37:17 EST." <9511131437.AA27774@snir.watson.ibm.com> Date: Mon, 13 Nov 1995 16:03:38 +0000 From: "James Cownie " Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc, I found this proposal much more intersting than your last (empty !) one :-). I actually like it. However I think that the comments I made about contiguous_struct (see below) also apply to type_record *if* you intend to send the padding. (They only went to mpi-misc, so many people may not have seen them). -- Jim > Date: Fri, 20 Oct 1995 10:05:44 +0100 > From: James Cownie > To: mpi-misc@mcs.anl.gov > Subject: Contigous struct > Reply-To: jcownie@bbn.com > Sender: owner-mpi-core@mcs.anl.gov > Precedence: bulk > Status: > > > While I admire the objective of this proposal (to achieve efficiency), > I'm nervous about the details, because it seems to me that to make > this proposal safe we need to modify the type matching rules. > > In the current draft MPI_TYPE_CONTIGUOUS_STRUCT is another type > constructor which says that any padding in the given structure can be > trampled upon in a receive. > > The objective being that the implementation can then send and receive a > contiguous chunk of data including the padding within the structure > when sending such structures, and simply trample on the whole store > area allocated to the structure when receiving. This may send a few > more bytes of data, but will avoid copying and packing/unpacking, so > will almost certainly be faster. > > The problem is that the way MPI is specified at the moment all the > information one needs to convert a byte-stream arriving at a processor > is contained in the lowest level type signature of the receive > operation. (So you don't care about anything more than the fact that > the message contains char[2], double, int16, char[20], etc.) > > Once you introduce CONTIGUOUS_STRUCT this important property cease to > be true, since the bytes packed into a message for a CONTIGUOUS_STRUCT > { char[2], double, int16, char[20] } will likely be different from > those packed by a STRUCT *with the same low level type > signature*. (because the CONTIGUOUS_STRUCT will include the padding > bytes, while the struct won't). > > In particular, at present, the layout of data in store at the receiver > is completely unknown to the sender, and no matching is required. (So > once can receive something sent as a STRUCT and fragment it into a set > of arrays as it is received, for instance). This ceases to be true > when you introduce CONTIGUOUS_STRUCT. > > So, I believe that to make this work, you need to introduce a new > concept into the type signature matching mechanism, which is analogous > to a typedef, in that a CONTIGUOUS_STRUCT creates a new fundamental > type which must type match only with itself. > > Thus > contiguous_struct { char[2], double, int16 } > can be sent (or received) *only* from another > contiguous_struct { char[2], double, int16 } > you can't match it with > struct { char[2], double, int16 }, > or a user defined datatype which happens to have the type signature > char[2], double, int16 > > -- Jim > > James Cownie > BBN UK Ltd > Phone : +44 117 9071438 > E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Thu Nov 16 16:35:51 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA03886; Thu, 16 Nov 1995 16:35:48 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA08532 for mpi-core-out; Thu, 16 Nov 1995 15:34:06 -0600 Received: from molly (molly.vcpc.univie.ac.at [193.171.58.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA08233; Thu, 16 Nov 1995 15:26:13 -0600 Received: from beavis.vcpc.univie.ac.at (beavis [193.171.58.38]) by molly (8.7.1/8.7.1) with ESMTP id WAA24600; Thu, 16 Nov 1995 22:25:50 +0100 (MET) From: Ian Received: (from ian@localhost) by beavis.vcpc.univie.ac.at (8.7.1/8.7.1) id WAA09257; Thu, 16 Nov 1995 22:26:04 +0100 (MET) Date: Thu, 16 Nov 1995 22:26:04 +0100 (MET) Message-Id: <199511162126.WAA09257@beavis.vcpc.univie.ac.at> To: mpi-users@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: 2nd European MPI workshop, VCPC, Vienna, Austria X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Announcement 2nd European MPI Workshop January 24-26, 1996, Vienna, Austria MPI (Message Passing Interface) is rapidly gaining wide acceptance as the future standard for message-passing programming. All major HPC hardware vendors support MPI or have announced that they will do so. Public domain implementations are already available. The MPI Forum reconvened during 1995 to address new functionality and to develop an MPI-2 standard. MPI-2 is a superset of MPI and is expected to be finalized by the end of 1996. A first set of extensions (for dynamic process creation) will be publically presented at Supercomputing '95 in San Diego. The 2nd European MPI Workshop is the successor of the highly successful 1st European MPI Workshop held in January 1994 at INRIA (organized by PALLAS and GMD). This workshop is organized by the European Centre for Parallel Computing at Vienna (VCPC) and PALLAS and sponsored by the ESPRIT project PPPE. There will be several additional events during the week. They are announced separately. Who should attend MPI-2 Open Meeting: European software developers with experience in message passing programming or tool development. Participants from universities, research organizations, and industry are welcome to attend. Participants of ESPRIT projects, such as EUROPORT and PPPE, are expected to take part. The number of participants is limited. Copies of the MPI-2 draft will be available from December 1995 on. Write to the email address below. MPI-2 Forum meeting: Although the MPI-2 is open to anybody who wants to come, it is recommended that only those with a strong and continuous interest in MPI attend. Venue: Hotel Regina, Rooseveltplatz 15, Vienna Fees MPI Open Meeting and Forum: US $ 85 (to be paid cash at registration). Fees include refreshments and lunch. Additional events There are several other events at the same location during the week of the MPI Forum. These events are supported by the PPPE project, funded by the ESPRIT research programme of the European Union. This support is gratefully acknowledged. The complete agenda of events follows. January 22/23: MPI Training Course January 22: Tutorial Introduction to Parallel Programming January 23: Workshop on Languages and Models for Future Computing Systems January 24: MPI-2 Open Meeting: MPI Forum Presentation of MPI-2 and open discussion with European MPI application developers and all other attendees January 25/26: MPI-2 Forum Meeting The Training Course will take place at the VCPC. All other events take place at the Regina Hotel, which is centrally located in Vienna and is just a few minutes walk from the VCPC. Participants may register for one or more of the above events. There are separate announcements for the training course, the tutorial and the workshop. Registration To register for one or more of the above events, please use the attached registration form or the form on the World Wide Web (http://www.vcpc.univie.ac.at/news/mpi/). Mail or fax it to the VCPC at the following address: Workshops and Courses Attn. Helga Pfeifer VCPC Liechtensteinstr. 22 A - 1090 Vienna Phone +43-1-310-9396 AUSTRIA FAX +43-1-310-560888 or +43-1-310-939613 Accommodation If you need help with accommodation, please fill out the arrival and departure information on the registration form. We will attempt to place you in a hotel close to the location of the events. The cost of accomodation is not included in the registration fee. Information For further information, please check the information on the WWW, send in the registration form, or email to: course@vcpc.univie.ac.at Registration form European Centre for Parallel Computing at Vienna (VCPC) 2nd European MPI Workshop MPI Training Course Tutorial: An Introduction to Parallel Programming Workshop on Languages and Models for Future Computing Systems Name: _________________________________________________________________________ Affiliation: __________________________________________________________________ Address: ______________________________________________________________________ ______________________________________ Telephone: _____________________________ Fax: _________________________________ E-mail: ________________________________ I wish to attend: o the tutorial on January 22 or o the training course on January 22/23 o the workshop on January 23 o the MPI meeting on January 24-26 Please send me further information on: o the tutorial on January 22 o the training course on January 22/23 o the workshop on January 23 o the MPI meeting on January 24-26 o Please send me information on future workshops and tutorials o Please send me hotel information o Please book a room for me at the Hotel Regina or a nearby hotel. Type of room: o Single o Double Arrival date: ____________________ Departure date: ______________________ Date: _____________________________ Signature: ______________________________ Fees Payment should be enclosed if you register for the tutorial or the workshop. Please make cheques payable to VCPC. All payments must be in Austrian Schillings. Tutorial: Before Jan 10 After Jan 10 Academic, ESPRIT/ACTS projects: 1950 ATS 2450 ATS Industry: 2850 ATS 3450 ATS Workshop: Before Jan 10 After Jan 10 Academic, ESPRIT/ACTS projects: 1150 ATS 1450 ATS Industry: 1850 ATS 2350 ATS I qualify for the o Academic, project fee o Industry fee Name of project: ______________________________________________________________ Method of payment: o Enclosed Cheque o American Express o Eurocard/Mastercard o Visa o Diners Club Total amount of payment: ______________________________________________________ Credit Card Number: ___________________________ Exp. date: __________________ Cardholder Name: ______________________________________________________________ Date: ______________________________ Signature: _____________________________ From mpi-core-human@mcs.anl.gov Fri Nov 17 19:04:58 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA24901; Fri, 17 Nov 1995 19:04:57 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA03604 for mpi-core-out; Fri, 17 Nov 1995 18:04:08 -0600 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA03591 for ; Fri, 17 Nov 1995 18:04:00 -0600 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id QAA21862 for mpi-core@mcs.anl.gov; Fri, 17 Nov 1995 16:03:59 -0800 Date: Fri, 17 Nov 1995 16:03:59 -0800 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199511180003.QAA21862@win233.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: hypermail archives of MPI mailing lists Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I have created hypermail archives of the MPI-2 mailing lists. They are available at http://parallel.nas.nasa.gov/MPI-2/ Bill From mpi-core-human@mcs.anl.gov Mon Nov 20 11:42:10 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA11613; Mon, 20 Nov 1995 11:42:10 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA29336 for mpi-core-out; Mon, 20 Nov 1995 10:41:56 -0600 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA29327 for ; Mon, 20 Nov 1995 10:41:48 -0600 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.7.1/8.6.12.1) with ESMTP id LAA08670 for ; Mon, 20 Nov 1995 11:38:25 -0500 (EST) Received: from b125.super.org (b125 [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with ESMTP id LAA03030 for ; Mon, 20 Nov 1995 11:41:17 -0500 From: lederman@super.org (Steve Huss-Lederman) Received: (lederman@localhost) by b125.super.org (8.6.12/8.6.12.client) id LAA14474; Mon, 20 Nov 1995 11:41:16 -0500 Date: Mon, 20 Nov 1995 11:41:16 -0500 Message-Id: <199511201641.LAA14474@b125.super.org> To: mpi-core@mcs.anl.gov Subject: schedule for SC95 document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the schedule for the SC95 document: - 12:00 (noon) on Tuesday 11/21: all input should be to me. - Wednesday morning 11/22: document will be released to MPI Forum. - Monday morning 11/27: document will be released to public. PLEASE take the time to give the document the once over. We are publically releasing it to give people and idea of our thoughts - we will make it clear that the proposals are still in flux. For authors: We would like to keep changes to a minimum after the MPI Forum release on Wednesday. However, changes can be made until the public release on Monday 11/27. Because I will be away on vacation, please carbon Rusty (lusk@mcs.anl.gov) on mail for input to the document that would normally only come to me. Steve From mpi-core-human@mcs.anl.gov Wed Nov 22 17:59:58 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA18035; Wed, 22 Nov 1995 17:59:57 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA21847 for mpi-core-out; Wed, 22 Nov 1995 16:59:19 -0600 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA21839 for ; Wed, 22 Nov 1995 16:59:11 -0600 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.7.1/8.6.12.1) with ESMTP id RAA16800 for ; Wed, 22 Nov 1995 17:55:46 -0500 (EST) Received: from b125.super.org (b125 [192.239.79.73]) by gotham.super.org (8.6.12/8.6.12.1) with ESMTP id RAA08679 for ; Wed, 22 Nov 1995 17:58:39 -0500 From: lederman@super.org (Steve Huss-Lederman) Received: (lederman@localhost) by b125.super.org (8.6.12/8.6.12.client) id RAA15158; Wed, 22 Nov 1995 17:58:39 -0500 Date: Wed, 22 Nov 1995 17:58:39 -0500 Message-Id: <199511222258.RAA15158@b125.super.org> To: mpi-core@mcs.anl.gov Subject: draft available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The latest version of the MPI-2 draft that will be released Monday morning is now available. See ftp://ftp.super.org/pub/mpi/mpi2/Editor. The main report is in mpi2-report.ps.Z. PLEASE TAKE THE TIME TO READ THIS. I HAVE BEEN UNDER GREAT PRESSURE AND HAVE NOT CAREFULLY CHECKED THE DOCUMENT. MAKE SURE IT IS THE TEXT YOU WANTED. UNLESS WE HEAR OTHERWISE THIS IS THE TEXT THAT WILL BE RELEASED TO THE PUBLIC FOR SC95. The collective and real time chapters are missing and will be release soon by Tony. Send all corrections to me and Rusty since I leave on vacation right now. I hope everyone has a great Thanksgiving. Steve From mpi-core-human@mcs.anl.gov Mon Nov 27 09:24:08 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA14146; Mon, 27 Nov 1995 09:24:05 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA11029 for mpi-core-out; Mon, 27 Nov 1995 08:22:21 -0600 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA10498 for ; Mon, 27 Nov 1995 07:40:39 -0600 Received: (tony@localhost); by Aurora.CS.MsState.Edu (SMI-8.6/7.0m-FWP-MsState); id HAA03236; Mon, 27 Nov 1995 07:40:34 -0600 Date: Mon, 27 Nov 1995 07:40:34 -0600 From: Tony Skjellum Message-Id: <199511271340.HAA03236@Aurora.CS.MsState.Edu> To: mpi-core@mcs.anl.gov Subject: MPI RT chapter (SC'95 version) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the postscript of the SC'95 version of the MPI RT chapter. Also, this is the first version of the chapter to appear in a cohesive form: --- cut here --- %!PS-Adobe-2.0 %%Creator: dvipsk 5.55a Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%Pages: 10 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, compressed, comments removed %DVIPSSource: TeX output 1995.11.27:0737 %%BeginProcSet: texc.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]/id ch-image N /rw ch-width 7 add 8 idiv string N /rc 0 N /gp 0 N /cp 0 N{rc 0 ne{rc 1 sub /rc X rw}{G}ifelse}imagemask restore}B /G{{id gp get /gp gp 1 add N dup 18 mod S 18 idiv pl S get exec}loop}B /adv{cp add /cp X}B /chg{rw cp id gp 4 index getinterval putinterval dup gp add /gp X adv}B /nd{/cp 0 N rw exit}B /lsh{rw cp 2 copy get dup 0 eq{pop 1}{ dup 255 eq{pop 254}{dup dup add 255 and S 1 and or}ifelse}ifelse put 1 adv}B /rsh{rw cp 2 copy get dup 0 eq{pop 128}{dup 255 eq{pop 127}{dup 2 idiv S 128 and or}ifelse}ifelse put 1 adv}B /clr{rw cp 2 index string putinterval adv}B /set{rw cp fillstr 0 4 index getinterval putinterval adv}B /fillstr 18 string 0 1 17{2 copy 255 put pop}for N /pl[{adv 1 chg} {adv 1 chg nd}{1 add chg}{1 add chg nd}{adv lsh}{adv lsh nd}{adv rsh}{ adv rsh nd}{1 add adv}{/rc X nd}{1 add set}{1 add clr}{adv 2 chg}{adv 2 chg nd}{pop nd}]dup{bind pop}forall N /D{/cc X dup type /stringtype ne{] }if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (temp.dvi) @start /Fa 60 126 df<137013F0EA01E0EA03C0EA0780EA0F00121E121C5AA25AA45A A81270A47EA27E121E7EEA0780EA03C0EA01F0120013700C24799F18>40 D<126012F012787E7E7EEA07801203EA01C0A2EA00E0A41370A813E0A4EA01C0A2EA0380 1207EA0F00121E5A5A5A12600C247C9F18>I<121C123E127E127F123F121F1207120E12 1E127C12F81260080C788518>44 D<387FFFC0B512E0A26C13C013047E8F18>I<123012 7812FCA2127812300606778518>I48 DI51 D<1230127812FCA2127812301200A81230127812FC A2127812300614779318>58 D<1218123C127EA2123C12181200A81218123C127EA2123E 121E120E121C123C127812F01260071A789318>I<387FFFC0B512E0A3C8FCA4B512E0A3 6C13C0130C7E9318>61 D<126012F87E127F6C7EEA0FE06C7EEA01FC6C7EEB3F80EB1FC0 EB07E0A2EB1FC0EB3F80EBFE00485AEA07F0485AEA3F8048C7FC12FC5A126013187E9918 >I<137013F8A213D8A2EA01DCA3138CEA038EA4EA0707A5380FFF80A3EA0E03381C01C0 A3387F07F000FF13F8007F13F0151C7F9B18>65 D67 DII< B512F8A3381C0038A41400A21307A3EA1FFFA3EA1C07A390C7FCA7EAFFC0A3151C7F9B18 >I<3801F1C0EA03FDEA0FFFEA1F0FEA1C03123813011270A290C7FC5AA5EB0FF0131F13 0F387001C0A213031238A2EA1C07EA1F0FEA0FFFEA03FDEA01F1141C7E9B18>I<387F07 F038FF8FF8387F07F0381C01C0A9EA1FFFA3EA1C01AA387F07F038FF8FF8387F07F0151C 7F9B18>II<387F 07F038FF87F8387F07F0381C03C0EB07801400130E131E5B13385B13F0121DA2EA1FB8A2 131C121EEA1C0EA27FA2EB0380A2EB01C0387F03F038FF87F8387F03F0151C7F9B18>75 DI<38FC01F8EAFE03A2383B06E0A4 138EA2EA398CA213DCA3EA38D8A213F81370A21300A638FE03F8A3151C7F9B18>I<387E 07F038FF0FF8387F07F0381D81C0A313C1121CA213E1A313611371A213311339A31319A2 131D130DA3EA7F07EAFF87EA7F03151C7F9B18>II< EAFFFEEBFF8014C0EA1C03EB01E013001470A514E01301EB03C0EA1FFF1480EBFE00001C C7FCA8B47EA3141C7F9B18>I 82 D<3803F1C0EA1FFF5AEA7C0FEA7003EAE001A390C7FC12701278123FEA1FF0EA07FE C67EEB0F80EB03C01301EB00E0A2126012E0130100F013C038F80780B5FCEBFE00EAE7F8 131C7E9B18>I<387FFFF8B5FCA238E07038A400001300B2EA07FFA3151C7F9B18>I<38FF 83FEA3381C0070B36C13E0EA0F01380783C03803FF806C1300EA007C171C809B18>I<38 FE03F8A338700070A36C13E0A513F8EA39FC13DCA2001913C0A3138CA2EA1D8DA3130500 0D1380EA0F07A2EA0E03151C7F9B18>87 D<387F0FE0139F130F380E0700120FEA070E13 8EEA039C13DCEA01F8A212005B137013F07F487E13DCEA039E138EEA070F7F000E138013 03001E13C0387F07F000FF13F8007F13F0151C7F9B18>I<38FE03F8EAFF07EAFE03381C 01C0EA1E03000E1380EA0F0700071300A2EA038EA2EA01DCA3EA00F8A21370A9EA01FC48 7E6C5A151C7F9B18>I91 D<126012F0A27E1278127C123CA2123E121E121F7EA27F12077F1203A27F12017F12007F 1378A2137C133C133E131EA2131F7F14801307A2EB030011247D9F18>II<387FFFC0B512E0A26C13C013047E7F18>95 D97 D 99 DIII<3801E1F03807FFF85A381E1E3038 1C0E00487EA5EA1C0EEA1E1EEA1FFC5BEA39E00038C7FC7EEA1FFEEBFFC04813E0387801 F038700070481338A4007813F0EA7E03381FFFC06C13803801FC00151F7F9318>I<127E 12FE127E120EA5133EEBFF80000F13C013C1EB80E01300120EAB387FC7FC38FFE7FE387F C7FC171C809B18>II108 D<38F9C1C038FFF7F013FF 383E3E38EA3C3CA2EA3838AB38FE3E3EEB7E7EEB3E3E1714809318>IIII<38FF0FC0EB3FE0EB7FF0EA07F0EBE060EBC0005BA290C7FCA9 EAFFFC7F5B14147E9318>114 DI< 487E1203A4387FFFC0B5FCA238038000A9144014E0A33801C1C013FF6C1380EB3E001319 7F9818>I<387E07E0EAFE0FEA7E07EA0E00AC1301EA0F033807FFFC6C13FE3801FCFC17 14809318>I<387F8FF000FF13F8007F13F0381C01C0380E0380A338070700A3138FEA03 8EA3EA01DCA3EA00F8A2137015147F9318>I<38FF07F8138F1307383800E0A4381C01C0 137113F9A213D9EA1DDD000D1380A3138DEA0F8FA23807070015147F9318>I<387F8FF0 139F138F380F0700EA078EEA039EEA01DC13F81200137013F07FEA01DCEA039E138EEA07 07000E1380387F8FF000FF13F8007F13F015147F9318>I<387F8FF000FF13F8007F13F0 380E01C0EB0380A21207EB0700A2EA0387A2138EEA01CEA213CC120013DC1378A31370A3 13F05B1279EA7BC0EA7F806CC7FC121E151E7F9318>I123 D<127CB4FC13C01203C67EAB7FEB7FC0EB3FE0A2EB7FC0EBF0005BABEA03C012FF90C7FC 127C13247E9F18>125 D E /Fb 1 78 df<00FCEB07E0A300EE130DA300E71319A3EB80 3900E31331EBC071A200E11361A2EBE0E1A200E013C113F1EB7181A3EB3B01A3131EA313 001B1D7C9C24>77 D E /Fc 10 58 df<121FEA3180EA60C0EA4040EAC060A8EA4040EA 60C0EA3180EA1F000B107F8F0F>48 D<120C123C12CC120CACEAFF8009107E8F0F>I<12 1FEA6180EA40C0EA806012C01200A213C0EA0180EA030012065AEA10201220EA7FC012FF 0B107F8F0F>I<121FEA2180EA60C0A2120013801201EA0F00EA00801340136012C0A2EA 8040EA6080EA1F000B107F8F0F>I<1203A25A5A120B121312331223124312C3EAFFE0EA 0300A4EA1FE00B107F8F0F>III<1240EA7FE013C0EA8080 A2EA010012025AA2120C1208A21218A50B117E900F>I<121FEA3180EA60C0A3EA7180EA 3F00120FEA3380EA61C0EAC060A3EA4040EA6080EA1F000B107F8F0F>I<121FEA3180EA 60C0EAC0401360A3EA40E01221EA1E6012001340EA60C01380EA4300123E0B107F8F0F> I E /Fd 45 123 df<137013E0EA01C01203EA078013005A120E121E121C123CA2123812 78A4127012F0AE12701278A41238123CA2121C121E120E120F7E1380EA03C01201EA00E0 13700C327DA413>40 D<12E012707E123C7E120E120F7E1380120313C0A2120113E0A412 0013F0AE13E01201A413C01203A21380120713005A120E121E5A12385A5A0C327DA413> I<12F0A404047B830E>46 D<13C01201120712FFA212FB1203B3A7B5FCA310227CA118> 49 DII<131FA2132FA2136F13EF13CF1201A2 EA038FA21207130F120F120E121E123CA21278A212F0B512F8A338000F00A915217FA018 >I<383FFF80A3003CC7FCA913F8EA3FFE7FEB0F80EA3E07383C03C01238380001E0A738 4003C0A212E038700780387C1F00EA3FFEEA0FFCEA03F013227EA018>I<137E48B4FC5A EA07C148C7FC121EA25AA25AA213F8EAF1FEEAF7FF38FF0F80EAFC0738F803C0A2130100 F013E0A51270A21278EB03C01238003C1380EA1C07381F0F00EA0FFE6C5AEA01F013237E A118>II67 DI70 D<00F0131EAFB512FEA338F0001EB117237BA222>72 D<12F0B3B104237CA20D>I<12F0 B3AEB512C0A312237BA21A>76 D<00FC147E6C14FEA300F7EB01DEA3EB800300F3149EA2 EBC007A200F1141EA2EBE00F00F0130EA2EBF01EA2EB701CEB783CA2EB3838EB3C78A2EB 1C70A2EB1EF0EB0EE0A2130FEB07C0A390C7FC1F237BA22A>I<00FC131E7EA312F7A2EA F380A213C012F1A213E012F013F0A21378A21338133CA2131EA2130E130F1307A2149E13 03A2EB01DEA2EB00FEA3147E17237BA222>III<13FE3803FF C04813E0EA0F81381E0060481300A25AA5127C123C123FEA1FC0EA0FFC6CB4FC00011380 38003FC0EB07E0EB01F0130014F81478A600C013F07E38F801E0387E07C0383FFF80000F 1300EA01FC15257EA31B>83 DI<00F01478A2 007814F0A36CEB01E0A36CEB03C0A2121F6CEB0780A213800007EB0F00A23803C00E141E A23801E01C143CA23800F0381478A2EB787014F0A2EB38E0133DEB1DC0131FA26D5AA21D 237FA220>86 D97 D99 DII<133F13FF5AEA03C013801207A8EAFFF8A3EA0780B3102380A20F>I<3801F078 3807FFF85A381F1F00EA1E0F383C0780A5381E0F00EA1F1FEA0FFE485AEA19F00038C7FC A2123CEA1FFEEBFFC014E04813F0EA7801EB00F8481378A4007813F0383E03E0381FFFC0 6C13803801FC0015217F9518>I<12F0ADEAF1F8EAF3FCEAF7FEEAFE1EEAF80FA212F0B0 10237CA219>I<12F0A41200A912F0B3A404237DA20B>I<12F0ADEB1F80EB3F00133E5B5B EAF1F0EAF3E0EAF7C012FFA27FA2EAFDF0EAF8F8EAF078137C7F131E131F7F1480EB07C0 12237CA218>107 D<12F0B3B104237DA20B>I<39F0F807C039F3FE1FF039F7FF3FF839FE 0F707800FCEBE07C39F807C03CA200F01380AF1E167C9527>IIII<3803E3C0EA07 FBEA0FFFEA1F0FEA3C07EA7C03127812F812F0A612F81278A2EA3C07EA3F0FEA1FFFEA0F FBEA03E3EA0003AA12207E9519>I II<120FA6EAFFF8A3EA0F00AE 1308131CEA07FC13F8EA03E00E1C7F9B12>II<39F007803CEB0FC0A2D8780D1378131D14E0131CD83C1813F0133814F0 001C14E0001E13711330EB7079000E14C01439380F603B00071480141BEBC01FA2000314 001E167F9521>119 D<38F001E0A2387803C0A2127C383C0780A2121E14005B120F130E EA071E139EEA039CA3EA019813D8EA00F85BA35BA212015BA2485A127F90C7FC127E1320 7F9516>121 D<387FFFC0A338000F801400131E133E5B13785B1201485A5B485A120F48 C7FC121E5A127CB512C0A312167F9515>I E /Fe 26 90 df<1218123C127CA2123C1204 A21208A21210A2122012401280060E7D840D>44 DI<12301278 12F81278127005057C840D>I<13181338EA01F8EA0E701200A513E0A6EA01C0A6EA0380 A6EA07001380EAFFFC0E1E7B9D17>49 DI<380180303803FFE014C014 0013FC0002C7FCA25AA4137CEA0587380A0380EA0C01000813C0120014E0A4387003C012 F812F000E01380EA800700401300130E6C5AEA1070EA0FC0141F7D9D17>53 D<1408140C141C143CA2147C147E149EA2EB011EA21302801304A21308A2011013801407 1320A2EB7FFF90384007C0EB8003A2EA0100A21202EC01E01206001F130339FF801FFE1F 207F9F22>65 D<0007B5FC3900F803C090387801E0EC00F04913F8A515F03801E001EC03 E015C0EC0F809038FFFE009038E00F803903C003C0EC01E015F0A21400A2485A1401A215 E01403EC07C0390F000F80EC3E00B512F01D1F7E9E20>II<0007B57E3900F801E0903878007081497F151E150E150FA348481480A6484814 005DA3151E153E4848133C5DA25D4A5A4A5A260F000FC7FC143CB512F0211F7E9E23>I< 0007B512FC3900F8007C0178131C150C5B1504A414043901E00800A31438EBFFF8EBE038 3803C010A4EC00081510485AA21520A2156015C0380F00011407B612801E1F7E9E1F>I< 0007B512F83900F800780178133815185B1508A53901E00800A314181438EBFFF83803C0 301410A491C7FC485AA648C8FC7FEAFFFC1D1F7E9E1E>II<3A07FF83FFC03A00F8007C000178133CA2495BA648485BA4 90B5FCEBE0004848485AA64848485AA64848485A01807F39FFF07FF8221F7E9E22>I<38 07FF803800F8001378A25BA6485AA6485AA6485AA648C7FC7FEAFFF0111F7E9E10>I<38 07FFE0D800FCC7FC1378A25BA6485AA6485AA41580EC0100EA0780A25C14021406140E38 0F001E147CB512FC191F7E9E1C>76 DIII<0007B5FC3900F803C090387800F015785B157CA41578484813F815F0EC01 E0EC03C0EC0F00EBFFFCD803C0C7FCA6485AA648C8FC7FEAFFF81E1F7E9E1F>I<3807FF FE3900F8078090387801E0EC00F05B15F8A415F03801E00115E0EC03C0EC0780EC1E00EB FFF03803C03880141E140EA2140F48485AA51502D80F001304EB800F39FFF00788C7EA01 F01F207E9E21>82 DI<003FB512F0383C 078000301430126039400F0010A212C01280A3D8001E1300A65BA65BA65B7F383FFFE01C 1F7A9E21>I<39FFF00FF8391F0003E06CEB01801400001EEB0100A6481302A6485BA600 705BA25CA200785B1238001813C06C48C7FCEA0706EA01F81D20799E22>I<3A03FFC0FF C03A007F003E00013C1318013E1310011E5B011F5B6D5B0281C7FCEB078314C2EB03C414 E8EB01F0A2130080A2EB017CEB023CEB043EEB0C1EEB081F497E132001407FEB80070001 80EB0003000780391F8007F039FFC01FFE221F7F9E22>88 D<39FFF001FF391F8000786C C712606D134000071480EBC00100031400EBE0025C12016D5A00005B6D5A1378EB7C40EB 3C80013FC7FC7F131EA3131C133CA513381378137C3807FF80201F7A9E22>I E /Ff 24 124 df<903807FC0290383FFF0E9038FE03DE3903F000FE4848133E4848131E 485A48C7120EA2481406127EA200FE1400A7127E1506127F7E150C6C7E6C6C13186C6C13 386C6C13703900FE01C090383FFF80903807FC001F1F7D9E26>67 DI 73 D76 D78 D80 D97 D99 D101 DI104 D<121C123E127FA3123E121CC7FCA6B4FCA2121FB0EAFFE0A20B217E A00E>I108 D<3AFE0FE03F8090391FF07FC03A 1E70F9C3E09039407D01F0EB807E121FEB007CAC3AFFE3FF8FFEA227147D932C>I<38FE 0FC0EB3FE0381E61F0EBC0F81380EA1F00AD38FFE7FFA218147D931D>I<48B4FC000713 C0381F83F0383E00F8A248137CA200FC137EA6007C137CA26C13F8A2381F83F03807FFC0 0001130017147F931A>I114 DI<12 03A45AA25AA2123FEAFFFCA2EA1F00AA1306A5EA0F8CEA07F8EA03F00F1D7F9C14>I<38 FF07F8A2EA1F00AD1301A2EA0F063807FCFF6C5A18147D931D>I<39FFE07F80A2391F00 1C00380F8018A26C6C5AA26C6C5AA2EBF0E000015B13F900005B13FF6DC7FCA2133EA213 1CA219147F931C>I<3AFFC7FE1FE0A23A1F00F0030014F8D80F801306A29038C1BC0E00 07140CEBC3BE3903E31E18A29038F60F380001143001FE13B03900FC07E0A2EBF8030178 5BA2903830018023147F9326>I<39FFE07F80A2391F001C00380F8018A26C6C5AA26C6C 5AA2EBF0E000015B13F900005B13FF6DC7FCA2133EA2131CA21318A2EA783012FC5BEAC0 E0EAE1C0EA7F80001EC8FC191D7F931C>121 D123 D E /Fg 1 16 df15 D E /Fh 5 81 df45 D50 D<12F0B3AE04207C9F0D>73 D<00F8EB01F86C1303A200F4147800F61307A300F3130EA3 EB801EA200F1131CEBC03CA200F01338A2EBE078A2EB6070EB70F0A2EB30E01339A2EB19 C0A3EB0F80A390C7FC1D207B9F28>77 D80 D E /Fi 37 122 df45 D<12F8A505057A8411>I<1318133813 F8120712FFA212F81200B3AC387FFFF0A314287BA71E>49 D<13FE3803FFC04813E04813 F0381F03F8383C00FC0038137E0078133E127000F0131FA212601220C7FCA3143EA2147C A214F8EB01F014E0EB03C0EB0780EB0F00131E5B5B5B485A485A485A48C7FC121E5A007F B5FCA418287EA71E>I<137F3801FFC0000713F0487FEA1FC1383E007C487F1238A21210 C7FCA2143C147C14FC495AEB07F0EBFFE05C8080EB01F8EB007C8080A2EC0F80A57E6CEB 1F007E6C5B007C137E383F81FC6CB45A6C5B00035BC66CC7FC19297EA71E>II<383FFFFCA4003EC7FCAA133FEB FFC0003F13E014F0EBC1F8EB807CEB007E003E133EC7FC141FA712200060133E127000F8 137C007C13FC383F03F8381FFFF06C13E06C13803800FE0018287EA61E>II65 DI<49B4FC010F13E0013F13F85B3801FE013903F8003001 E0131048481300485A121F90C8FC123EA2127E127CA35AAA127CA3127E123EA27E7F120F 6C7E6C6C130401F8131CD801FE13FC38007FFF6D13F8010F13E0010113001E2C7CAA26> III<12F8B3B3A6 052A7AA911>73 D<00FEED7F80A36C15FF00FB15EF6D1301A3D8F9C0EB03CFA3D8F8E0EB 078FA26D130FA20170140F0178131FA20138131E013C133EA26D137CA2010E1378010F13 F8A26D13F01481A2010313E014C3A2010113C014E701001380A3EC7F00A3143E91C7FC29 2A7BA934>77 D80 D82 DII97 D<12F8AF133F38F9FF8000FB13E0B512F0130738FC01F838F800FC147CA2147E143EA714 7CA214FC6C13F8130138FF07F0EBFFE000FB13C000F9138038F87E00172A7BA91F>II<143EAF13FCEA03FF4813FE5AEA 1FC1383F007E003E133E5AA212FC5AA77E127CA26C137E003F13FEEA1FC1EA0FFF6C13BE 6C133EEA00FC172A7EA91F>I<137E3803FF804813C04813E0381F83F0EA3F00003E1378 5A14380078133CB512FCA400F8C7FCA31278127CA27E003F130C381FC07C380FFFFC7E00 0113F038007F80161B7E9A1B>I<9038FC07803903FF3FC04813FF5A391F87E000383E01 F0A2486C7EA66C485AA2381F87E0EBFFC0485B003B90C7FCEA38FC0078C8FCA27E383FFF C014F86C7F80487F387C007F48EB1F80140FA36C131F007EEB3F00383F80FE13FF000F13 F86C5BC613801A287E9A1E>103 D<12F8A51200AA12F8B3A9052A7CA90E>105 D<12F8B3B3A6052A7CA90E>108 D<3AF83F801FC09039FFE07FF000F99038F0FFF800FB 01F913FC39FF81FBC03AFE00FF007E6E133E48137EA248137CB2271B7C9A30>I<38F83F 80EBFFE000F913F000FB13F8EAFF8138FE00FC147C5AA25AB2161B7C9A1F>I<137F3801 FFC0000713F0487F381FC1FC383F007E003E133E487FA200787F00F81480A7007CEB1F00 A2007E5B003E133E003F137E381FC1FC6CB45A6C5B000113C06C6CC7FC191B7E9A1E>I< EAF83F38F9FF8000FB13E0B512F0130738FC01F838F800FCA2147C147E143EA7147CA214 FC6C13F8130338FF07F0EBFFE000FB13C000F9138038F87E0090C7FCAC17277B9A1F>I< EAF83C13FC12F912FBEAFFE0138013005A5AA35AB00E1B7C9A14>114 DII<00F8137CB3A214FC1301EAFC07B5FC6C137CEA3FFCEA1FE0161B7C9A1F>I<00 F8EB0F80A2007CEB1F00A2127E003E133EA26C133C147CA2380F807814F8A26C6C5A13C1 A23803E1E013E300015BA213F33800F780A3017FC7FCA3191B7F9A1C>I<00F8EB0F807E 007CEB1F00A26C133EA2123F6C5BA2EA0F805CEA07C05C13C1EA03E15C120113E3EBF3C0 12005C1373137791C7FC133F133E131EA25BA35BA25B1201EA7FE0A25B90C8FC19277F9A 1C>121 D E /Fj 19 121 df<923807FF804AB512F0020F14FC023F809139FFFC00FF49 01C0EB3F80010790C7487E4948EB01FF4948814A5B495AA2495AA4705B705B053EC7FC94 C8FCA693383FFFF0B9FCA526007FF0C7FC177FB3B1003FB5D8E03FB512E0A53B487EC742 >12 D44 DI69 D80 D82 D<903807FFFC017FEBFFC048B612 F84815FE3A07FE001FFF6D010313C0486D6C7F6F7F167F83A26C90C76C7EA26C5AEA00F8 90C8FCA40207B5FC91B6FC1307013FEBE03F9038FFFE00000313F04813C04890C7FC485A 485A485AA2485AA5167F6C6C14FFED01EF6C6C130301FF903807CFFF000F9026C03F8F13 F86C90B5000713FC0001ECFC036C6CEBF0010103903980007FF8362E7DAD3A>97 DI 100 DI<13FC487E487E4813804813C04813E0A66C13C06C13806C13006C5A6C 5A90C7FCABEB7FC0B5FCA512037EB3B0B6FCA518497CC820>105 D108 D<90287FC001FFC0ECFF E0B5010F01F8010713FC033F01FE011F13FF92B6017F809126C1FC07902680FE037F9126 C3E0039026C1F0017F00039026C7C0019038C3E0006CD9CF00DAE7808002DE6D01EFC712 7F02DC15EE02FC03FE814A5D4A5DA34A5DB3A8B6D8C07F9026FFE03FB512F0A55C2E7CAD 63>I<903A7FC001FFC0B5010F13F8033F13FE92B6FC9126C1FC077F9126C3E0037F0003 9038C7C0016CD9CF008002DE7F14DC02FC815C5CA35CB3A8B6D8C07FEBFFE0A53B2E7CAD 42>II<9039 FF803F80B5EBFFF0028313FC02877F91388FC3FF15070003D99E0F13806C13BC14B814F8 14F06F1300A24A6C5AED00F893C7FCA25CB3A6B612E0A5292E7CAD31>114 DII<007FB5D8801FB5FCA528007FF8000190 C7FC6E14FC6D6C495A6D6C495A6D5D6D6D485A6D6D485AEDE03F6D6D48C8FC6DEBF8FE91 387FF9FC6EB45A5E6E5B6E5B80806E7F82824A7F825C91380FEFFFDA1FCF7FDA3F877FDA 7F037FECFE0149486C7F4A6D7E49488001076E7E49486D7E49487F49486D7F01FF82B500 F8013FEBFFC0A53A2E7EAD3F>120 D E /Fk 8 117 df<141E143E14FE1303133FB5FCA3 13CFEA000FB3B3A7007FB612C0A4223879B731>49 D67 D97 D<903801FFC0010F13F801 3F13FE9038FF80FF3A01FE003F80D807FCEB1FC04848EB0FE016F048481307003F15F8A2 485AED03FCA312FF90B6FCA301E0C8FCA4127FA3123F6D143C121FA26C6C14786C6C14F8 6C6CEB01F06C6CEB07E06C9038C03FC0013FB51200010713FC010013E026267DA52D> 101 D<13FFB5FCA412077EB0ED7FC0913801FFF802077F91380F03FE91381C01FF023014 804A7E02E014C05C5CA391C7FCB3A4B5D8FC3F13FFA4303C7CBB37>104 D<9039FF01FF80B5000F13F0023F13FC9138FE03FFDAF0001380000301C0EB7FC04AEB3F E091C713F0EE1FF817FC160F17FEA2160717FFA917FEA2160F17FCA2EE1FF8A2EE3FF06E 14E06EEB7FC09139F001FF80DAFC07130091383FFFFC020F13F0020190C7FC91C9FCADB5 12FCA430377DA537>112 D<9038FE03F000FFEB0FFE4A7E91383C3F809138707FC00007 903860FFE0000313E05C13FF9138807FC0A2ED3F80ED0E0091C8FCB3A3B512FEA423267D A529>114 D116 D E /Fl 81 125 df<90381F83E09038F06E303901C07878380380F89038 00F03048EB7000A7B612803907007000B2383FE3FF1D20809F1B>11 D<133FEBE0C0EA01C0380381E0EA0701A290C7FCA6B512E0EA0700B2383FC3FC1620809F 19>II<9038 1F81F89038F04F043901C07C06390380F80FEB00F05A0270C7FCA6B7FC3907007007B23A 3FE3FE3FE02320809F26>I34 D<127012F812FCA212741204A31208A21210A2122012 40060E7C9F0D>39 D<13401380EA01005A12061204120C5AA212381230A212701260A412 E0AC1260A412701230A212381218A27E120412067E7EEA008013400A2E7BA112>I<7E12 407E12307E1208120C7EA212077EA213801201A413C0AC1380A412031300A25A1206A25A 120812185A12205A5A0A2E7EA112>I<127012F012F8A212781208A31210A31220A21240 050E7C840D>44 DI<127012F8A3127005057C840D>I<144014C0 EB0180A3EB0300A31306A25BA35BA35BA25BA35BA3485AA348C7FCA21206A35AA35AA25A A35AA35AA2122D7EA117>II<13801203120F 12F31203B3A6EA07C0EAFFFE0F1E7C9D17>III<1306A2130EA2131E132E A2134E138EA2EA010E1202A212041208A212101220A2124012C0B512F038000E00A7EBFF E0141E7F9D17>II<137CEA0182EA0701380E0380EA0C0712183838030090C7FC12781270A2EAF1F0 EAF21CEAF406EAF807EB0380A200F013C0A51270A214801238EB07001218EA0C0E6C5AEA 01F0121F7E9D17>I<1240387FFFE014C0A23840008038800100A21302485AA25B5BA25B A21360A213E05B1201A41203A76C5A131F7E9D17>I I I<127012F8A312701200AA127012F8A3127005147C930D>I<127012F8A312701200AA12 7012F012F8A212781208A31210A31220A21240051D7C930D>I<007FB512F8B612FCC9FC A8B612FC6C14F81E0C7E9023>61 D63 D<5B497EA3497EA3EB09E0A3EB10F0A3EB2078A3497EA2EBC03EEB801EA248B5FC EB000FA20002EB0780A348EB03C0A2120C001E14E039FF801FFE1F207F9F22>65 DI<90380FE0109038381C30 9038E002703803C00139078000F048C71270121E15305A1510127C127800F81400A91278 007C1410123CA26C1420A27E6C6C13406C6C13803900E00300EB380CEB0FF01C217E9F21 >IIII<90380FE01090 38381C309038E002703803C00139078000F048C71270121E15305A1510127C127800F814 00A7EC3FFEEC01F000781300127C123CA27EA27E6C7E3903C001703900E002309038380C 1090380FF0001F217E9F24>I<39FFF07FF8390F000780AD90B5FCEB0007AF39FFF07FF8 1D1F7E9E22>II<39FFF007FC390F0003E0EC01 80150014025C5C5C5C5C5C49C7FC5B497E130FEB13C0EB21E01341EB80F0EB0078A28080 A280EC0780A2EC03C015E015F039FFF01FFE1F1F7E9E23>75 DIIIIIII<3803F040380C 0CC0EA1803EA3001EA6000A212E01440A36C13007E127CEA7F80EA3FF86CB4FC00071380 C613C0EB1FE013031301EB00F014707EA46C136014E06C13C038F8018038C60300EA81FC 14217E9F19>I<007FB512E038780F010060EB006000401420A200C0143000801410A400 001400B3497E3803FFFC1C1F7E9E21>I<39FFF00FF8390F0003E0EC0080B3A46CEB0100 1380120314026C6C5A6C6C5AEB3830EB0FC01D207E9E22>I<39FFF003FE391F8000F86C C7126015206C6C1340A36C6C1380A2EBE00100011400A23800F002A213F8EB7804A26D5A A36D5AA2131F6D5AA2EB07C0A36D5AA36DC7FC1F207F9E22>I<3BFFF07FF81FF03B1F00 0FC007C06C903907800180170015C001805C00071502EC09E013C000035DEC19F01410D8 01E05CA2EC2078D800F05CA2EC403C01785CA2EC801E017C1460013C144090383D000F13 3F6D5CA2011E1307010E91C7FCA2010C7F010413022C207F9E2F>I<397FF81FF8390FE0 07C03907C0030000031302EBE0063801F00400005BEBF818EB78106D5AEB3E60EB1E406D 5AA213076D7E497E1305EB08F0EB18F8EB1078EB207CEB603EEB401EEB801F3901000F80 1407000214C000061303001FEB07E039FFC01FFE1F1F7F9E22>I<39FFF001FF391F8000 78000F146012076D1340000314807F3901F001001200EBF802EB7C06EB3C04EB3E08131E EB1F10EB0FB0EB07A014E06D5AACEB3FFC201F7F9E22>I<12FFA212C0B3B3A512FFA208 2D7CA10D>91 DI<12FFA21203B3B3A512FFA2082D80A10D>I97 D<121C12FC121CAA137CEA1D87381E0180EB00C0001C13E01470A21478 A6147014F014E0001E13C0381A018038198700EA107C15207E9F19>IIII< 137CEA01C6EA030F1207EA0E061300A7EAFFF0EA0E00B2EA7FE01020809F0E>I<14E038 03E330EA0E3CEA1C1C38380E00EA780FA5EA380E6C5AEA1E38EA33E00020C7FCA21230A2 EA3FFE381FFF8014C0383001E038600070481330A4006013606C13C0381C03803803FC00 141F7F9417>I<121C12FC121CAA137C1386EA1D03001E1380A2121CAE38FF8FF014207E 9F19>I<1238127CA31238C7FCA6121C12FC121CB1EAFF80091F7F9E0C>I<13E0EA01F0A3 EA00E01300A61370EA07F012001370B3A31260EAF06013C0EA6180EA3F000C28829E0E> I<121C12FC121CAAEB1FE0EB0780EB060013045B5B5B136013E0EA1DF0EA1E70EA1C3813 3C131C7F130F7F148014C038FF9FF014207E9F18>I<121C12FC121CB3ABEAFF8009207F 9F0C>I<391C3E03E039FCC30C30391D039038391E01E01CA2001C13C0AE3AFF8FF8FF80 21147E9326>IIII<38 01F04038070CC0EA0E02EA1C03EA38011278127012F0A6127012781238EA1C03EA0C05EA 0709EA01F1EA0001A8EB0FF8151D7F9318>III<1202A31206A212 0EA2123EEAFFF8EA0E00AB1304A5EA07081203EA01F00E1C7F9B12>I<381C0380EAFC1F EA1C03AE1307120CEA061B3803E3F014147E9319>I<38FF83F8383E00E0001C13C06C13 80A338070100A21383EA0382A2EA01C4A213E4EA00E8A21370A3132015147F9318>I<39 FF9FE1FC393C078070391C030060EC8020000E1440A214C0D80704138014E0A239038861 001471A23801D032143A143E3800E01CA2EB6018EB40081E147F9321>I<38FF87F8381E 03C0380E0180EB0300EA0702EA0384EA01C813D8EA00F01370137813F8139CEA010E1202 EA060738040380000C13C0003C13E038FE07FC16147F9318>I<38FF83F8383E00E0001C 13C06C1380A338070100A21383EA0382A2EA01C4A213E4EA00E8A21370A31320A25BA3EA F080A200F1C7FC1262123C151D7F9318>IIII E /Fm 47 122 df<127012F812FCA212741204A41208A21210A212201240060F7C840E>44 DI<127012F8A3127005057C840E>I48 D<13801203120F12F31203B3A9EA07C0EAFFFE0F217CA018 >III<1303A25BA25B13 17A21327136713471387120113071202120612041208A212101220A2124012C0B512F838 000700A7EB0F80EB7FF015217FA018>I<00101380381E0700EA1FFF5B13F8EA17E00010 C7FCA6EA11F8EA120CEA1C07381803801210380001C0A214E0A4127012F0A200E013C012 80EA4003148038200700EA1006EA0C1CEA03F013227EA018>I<137EEA01C13803008038 0601C0EA0C03121C381801800038C7FCA212781270A2EAF0F8EAF30CEAF4067F00F81380 EB01C012F014E0A51270A3003813C0A238180380001C1300EA0C06EA070CEA01F013227E A018>I<12401260387FFFE014C0A23840008038C0010012801302A2485A5BA25B5BA213 60134013C0A21201A25B1203A41207A76CC7FC13237DA118>III<497EA3497EA3EB05E0A2EB09F01308A2EB10 78A3497EA3497EA2EBC01F497EA248B51280EB0007A20002EB03C0A348EB01E0A348EB00 F0121C003EEB01F839FF800FFF20237EA225>65 D<903807E0109038381830EBE0063901 C0017039038000F048C7FC000E1470121E001C1430123CA2007C14101278A200F81400A8 12781510127C123CA2001C1420121E000E14407E6C6C13803901C001003800E002EB381C EB07E01C247DA223>67 D69 DI73 D77 D<39FF8007FF3907C000 F81570D805E01320EA04F0A21378137C133C7F131F7FEB0780A2EB03C0EB01E0A2EB00F0 14F81478143C143E141E140FA2EC07A0EC03E0A21401A21400000E1460121FD8FFE01320 20227EA125>I80 D82 D<3803F020380C0C60EA1802 383001E0EA70000060136012E0A21420A36C1300A21278127FEA3FF0EA1FFE6C7E000313 8038003FC0EB07E01301EB00F0A214707EA46C1360A26C13C07E38C8018038C60700EA81 FC14247DA21B>I<007FB512F839780780780060141800401408A300C0140C00801404A4 00001400B3A3497E3801FFFE1E227EA123>I97 D<120E12FE121E120EAB131FEB61C0EB8060380F0030000E1338143C141C141EA7 141C143C1438000F1370380C8060EB41C038083F0017237FA21B>II<14E0130F13011300ABEA01F8EA0704EA0C02EA1C01EA3800127812 7012F0A7127012781238EA1801EA0C0238070CF03801F0FE17237EA21B>II<133E13E33801C780EA0387130748C7FCA9 EAFFF80007C7FCB27FEA7FF0112380A20F>I<14703803F198380E1E18EA1C0E38380700 A200781380A400381300A2EA1C0EEA1E1CEA33F00020C7FCA212301238EA3FFE381FFFC0 6C13E0383000F0481330481318A400601330A2003813E0380E03803803FE0015217F9518 >I<120E12FE121E120EABEB1F80EB60C0EB80E0380F0070A2120EAF38FFE7FF18237FA2 1B>I<121C123EA3121CC7FCA8120E127E121E120EB1EAFFC00A227FA10E>I<13E0EA01F0 A3EA00E01300A81370EA07F012001370B3A51260EAF0E013C0EA6180EA3F000C2C83A10F >I<120E12FE121E120EABEB03FCEB01F014C01480EB02005B5B5B133813F8EA0F1CEA0E 1E130E7F1480EB03C0130114E0EB00F014F838FFE3FE17237FA21A>I<120E12FE121E12 0EB3ADEAFFE00B237FA20E>I<390E1FC07F3AFE60E183803A1E807201C03A0F003C00E0 A2000E1338AF3AFFE3FF8FFE27157F942A>I<380E1F8038FE60C0381E80E0380F0070A2 120EAF38FFE7FF18157F941B>III114 DI<1202A41206A3120E121E123EEAFFFCEA0E00AB1304A6EA07081203 EA01F00E1F7F9E13>I<000E137038FE07F0EA1E00000E1370AD14F0A238060170380382 783800FC7F18157F941B>I<38FF80FE381E00781430000E1320A26C1340A2EB80C00003 1380A23801C100A2EA00E2A31374A21338A3131017157F941A>I<39FF8FF87F393E01E0 3C001CEBC01814E0000E1410EB0260147000071420EB04301438D803841340EB8818141C D801C81380EBD00C140E3900F00F00497EA2EB6006EB400220157F9423>I<38FF80FE38 1E00781430000E1320A26C1340A2EB80C000031380A23801C100A2EA00E2A31374A21338 A31310A25BA35B12F05B12F10043C7FC123C171F7F941A>121 D E /Fn 20 118 df45 D68 D73 D 77 D80 D<90387F80203801FFE03907C07860380F001C001EEB06E048130300381301007813 001270156012F0A21520A37E1500127C127E7E13C0EA1FF86CB47E6C13F86C7FC613FF01 0F1380010013C0EC1FE01407EC03F01401140015F8A26C1478A57E15706C14F015E07E6C EB01C000ECEB038000C7EB070038C1F01E38807FFCEB0FF01D337CB125>83 D<13FE380303C0380C00E00010137080003C133C003E131C141EA21208C7FCA3EB0FFEEB FC1EEA03E0EA0F80EA1F00123E123C127C481404A3143EA21278007C135E6CEB8F08390F 0307F03903FC03E01E1F7D9E21>97 D99 DIII<15F090387F03083901C1C41C380380E8390700700848EB7800001E 7FA2003E133EA6001E133CA26C5B6C13706D5A3809C1C0D8087FC7FC0018C8FCA5121C7E 380FFFF86C13FF6C1480390E000FC00018EB01E048EB00F000701470481438A500701470 A26C14E06CEB01C00007EB07003801C01C38003FE01E2F7E9F21>I<1207EA0F80121FA2 120FEA0700C7FCABEA078012FFA2120F1207B3A6EA0FC0EAFFF8A20D307EAF12>105 D<260780FEEB1FC03BFF83078060F0903A8C03C180783B0F9001E2003CD807A013E4DA00 F47F01C013F8A2495BB3A2486C486C133F3CFFFC1FFF83FFF0A2341F7E9E38>109 D<380780FE39FF83078090388C03C0390F9001E0EA07A06E7E13C0A25BB3A2486C487E3A FFFC1FFF80A2211F7E9E25>II<380783E038FF8C18EB 907C120FEA07A0EBC0381400A35BB3487EEAFFFEA2161F7E9E19>114 D<3801FC10380E0330381800F048137048133012E01410A37E6C1300127EEA3FF06CB4FC 6C13C0000313E038003FF0EB01F813006C133CA2141C7EA27E14186C1338143000CC1360 38C301C03880FE00161F7E9E1A>I<1340A513C0A31201A212031207120F381FFFE0B5FC 3803C000B01410A80001132013E000001340EB78C0EB1F00142C7FAB19>II E /Fo 5 85 df<16C08215011503A21507A2150FA2151F153F153715 67A2EDC7F015C3EC0183140315031406A2140CA21418A214301460A202C07FA290380180 01A249B5FC5B90380600015BA25BA25B137013605B8248481300120300071401D81FC013 03D8FFF890383FFFE0A22B327BB135>65 D<010FB512FE707E903A007F000FE0027EEB03 F0707E707E177C4A80A2173F83495A1880A3495AA44948143FA4494815005FA34948147E 17FEA25F49C712015FA24C5A017E5D16074C5A5F494AC7FC5E167E5E4848EB01F04B5AED 0FC00003027FC8FCB612FC15E031317BB036>68 D<010FB612FCA29039007F0001027EEB 007C173CA217185CA4495AA44948EB4010EEC000A34948485AA21503150F49B5C7FCA2EC C00F8190381F8006A449485A150492C8FCA2137EA45BA4485AA31203B512F0A22E317BB0 2F>70 D<010FB512F016FE903A007F003F80027EEB0FC0EE03E017F0EE01F85CA4494813 03A317F04948130717E0EE0FC0A24948EB1F00163E16FCED07F049B512C093C7FC9138C0 0F80ED03C0D91F807F6F7EA282EB3F00A4017E1303A25EA2491307A21708170C48481518 A2030313301203B5D8C00113E0923800FFC0C9EA3F002E327BB034>82 D<0007B712F05A9039C00FE003D81E00EBC000001C16E01218003816600030494813E0A2 4816C0A24AC7FC5AA348017E1480C71500A35CA4495AA4495AA4495AA4495AA4495AA449 C9FCA313FF007F13FFA22C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 911 a Fo(D)26 b(R)g(A)f(F)h(T)225 1002 y Fn(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n (terface)621 1196 y Fm(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum) 772 1323 y(No)o(v)o(em)o(b)q(er)e(27,)k(1995)77 1381 y(This)g(w)o(ork)f(w)o(as)h(supp)q(orted)g(in)f(part)h(b)o(y)e(ARP)l(A) h(and)h(NSF)e(under)h(gran)o(t)h(ASC-9310330,)i(the)192 1439 y(National)d(Science)f(F)l(oundation)i(Science)e(and)i(T)l(ec)o (hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1497 y(Agreemen)o(t)e (No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h(Commission)e(of)j(the)f (Europ)q(ean)i(Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s (ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 49 a Fl(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 361 a Fk(Chapter)34 b(11)75 568 y Fj(Real-time,)42 b(Em)m(b)s(edded)e(Extensions,)75 693 y(and)f(Pro\014les)75 933 y Fi(11.1)59 b(Intro)r(duction)75 1035 y Fl(The)14 b(goal)f(of)h(real-time)g Fh(MPI)f Fl(is)h(to)f(pro)o(vide)h(comm)o (unications)h(with)f(predictabilit)o(y)i(and)d(guaran)o(teed)75 1091 y(timing)k(p)q(erformance.)k(The)16 b(philosoph)o(y)h(for)e(a)h (real-time)h(v)o(ersion)e(of)h(MPI)g(can)f(b)q(e)i(summarized)f(as)75 1148 y(follo)o(ws:)143 1221 y Fg(\017)23 b Fl(Minimal)16 b(c)o(hanges)f(to)g(MPI.)143 1307 y Fg(\017)23 b Fl(Use)15 b(a)g(middlew)o(are)h(approac)o(h)f(\(application,)h(middlew)o(are,)g (MPI,)f(OS\).)143 1393 y Fg(\017)23 b Fl(Three)15 b(di\013eren)o(t)g (real-time)i(paradigms)e(m)o(ust)f(b)q(e)i(considered:)240 1478 y Ff({)23 b Fl(time-driv)o(en)240 1543 y Ff({)g Fl(ev)o(en)o(t-driv)o(en)240 1608 y Ff({)g Fl(priorit)o(y-based)143 1694 y Fg(\017)g Fl(Issues)15 b(of)g(\\em)o(b)q(eddabilit)o(y")i (should)f(b)q(e)g(considered)h(separately)l(.)143 1779 y Fg(\017)23 b Fl(Real-time)16 b(MPI)g(m)o(ust)e(allo)o(w)i(not)e(only) i(p)q(ortabilit)o(y)g(but)g(also)f(p)q(erformance)g(p)q(ortabilit)o(y)l (.)143 1865 y Fg(\017)23 b Fl(W)l(e)17 b(cannot)g(\014x)g(all)h (real-time)f(problems)h(in)g(MPI.)f(F)l(or)f(example)i(real-time)g(MPI) e(cannot)h(b)q(e)189 1922 y(exp)q(ected)f(to)f(\014x)g(a)g (non-real-time)h(OS.)143 2007 y Fg(\017)23 b Fl(The)16 b(results)h(of)f(this)h(e\013ort)e(will)j(b)q(e)f(one)f(or)g(more)g (pro\014les,)h(to)f(b)q(e)h(adopted)f(separately)h(from)189 2064 y(the)j(MPI-2)h(standard,)g(but)g(related)g(to)f(it)h(whereev)o (er)f(p)q(ossible.)38 b(That)20 b(is,)i(not)f(all)g(MPI-2)189 2120 y(implemen)o(tations)14 b(will)h(need)f(to)f(supp)q(ort)g (realtime)h(and)f(em)o(b)q(edded)i(features)e(describ)q(ed)i(here.)143 2206 y Fg(\017)23 b Fl(E\013orts)17 b(will)k(b)q(e)f(made)f(to)f(mak)o (e)g(existing)i(MPI)f(libraries)i(w)o(ork)d(as)h(seamlessly)h(as)e(p)q (ossible)189 2262 y(within)e(these)f(pro\014les.)143 2348 y Fg(\017)23 b Fl(There)15 b(is)h(a)f(need)h(for)e(a)h(p)q (ortable)h(real-time)g(b)q(enc)o(hmark)g(suite.)166 2421 y(Real-time)i Fh(MPI)d Fl(will)j(serv)o(e)e(as)f(an)h(op)q(en)h (standard)f(for)f(a)h(wide)h(v)m(ariet)o(y)f(of)g(high-p)q(erformance,) 75 2478 y(real-time,)22 b(em)o(b)q(edded,)h(heterogeneous,)f(parallel)g (computing)f(systems)f(encompassing)h(a)f(div)o(erse)75 2534 y(mix)13 b(of)g(computational)g(paradigms.)19 b(Real-time)14 b Fh(MPI)f Fl(should)h(mak)o(e)e Fh(MPI)g Fl(relev)m(an)o(t)i(to)e(the) h(real-time)75 2591 y(comm)o(unit)o(y)21 b(as)g(w)o(ell)h(as)f(enable)h (dev)o(elopmen)o(t)g(of)f(v)o(endor-indep)q(enden)o(t)j(real-time)e (applications.)75 2647 y(It)e(should)i(also)e(assist)g(in)h(bridging)g (the)g(gap)f(b)q(et)o(w)o(een)g(theory)g(and)h(implemen)o(tation)g(in)g (parallel)75 2704 y(real-time)16 b(computing)g(and)f(comm)o (unications.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fl(2)187 b Fe(CHAPTER)16 b(11.)34 b(REAL-TIME,)16 b(EMBEDDED)e(EXTENSIONS,)i(AND)f(PR)o(OFILES)75 49 y Fi(11.2)59 b(Issues)19 b(in)h(Real-time)e(Computing)75 151 y Fl(There)f(are)g(man)o(y)f(imp)q(ortan)o(t)h(prop)q(erties)h(of)e (real-time)i(applications)g(that)f(are)f(imp)q(ortan)o(t,)h(as)g(fol-) 75 207 y(lo)o(ws:)143 301 y Fg(\017)23 b Fl(Timing)16 b(correctness)f(as)g(w)o(ell)h(as)f(program)f(correctness.)143 395 y Fg(\017)23 b Fl(Determinism)16 b(\(predictabilit)o(y\))g(|)g(ev)o (en)g(at)e(the)h(cost)g(of)g(p)q(erformance.)143 489 y Fg(\017)23 b Fl(T)l(asks)14 b(ma)o(y)h(ha)o(v)o(e)g(di\013eren)o(t)g (degrees)h(of)e(urgency)i(or)f(relativ)o(e)g(priorities.)143 582 y Fg(\017)23 b Fl(F)l(ast)14 b(resp)q(onse)i(time)f(at)g(least)g (for)g(tasks)f(with)i(higher)g(priorities)g(|)g(built-in)h(unfairness.) 143 676 y Fg(\017)23 b Fl(High)15 b(degree)h(of)f(sc)o(hedulabilit)o(y) j(\(real-time)d(throughput\).)143 770 y Fg(\017)23 b Fl(Dep)q(endabilit)o(y)17 b(|)f(rigid)g(fault)f(tolerance,)h(again)f (sometimes)g(at)f(the)i(cost)e(of)h(p)q(erformance.)143 864 y Fg(\017)23 b Fl(F)l(rom)14 b(the)h(MPI)h(standp)q(oin)o(t)f(|)h (t)o(w)o(o)e(k)o(ey)h(prop)q(erties)g(are)g(p)q(ortabilit)o(y)i(and)e (scalabilit)o(y)l(.)75 986 y Fd(11.2.1)49 b(Timing)18 b(Co)o(rrectness)75 1071 y Fl(Timing)e(correctness)f(has)g(these)h (prop)q(erties:)143 1178 y Fg(\017)23 b Fl(Program)12 b(can)i(pro)q(ceed,)g(in)h(a)e(meaningful)i(w)o(a)o(y)l(,)e(only)h(if)g (all)h(previous)f(steps)g(are)f(time-correct.)143 1271 y Fg(\017)23 b Fl(Most)11 b(basic)h(MPI)g(\(curren)o(t\))f(functions)i (should)g(ha)o(v)o(e)e(analogous)h(functions)h(for)e(real-time)i(that) 189 1328 y(guaran)o(tee)h(message)h(passing)g(in)h(a)f(timely)i (fashion.)166 1434 y(F)l(or)h(practicalit)o(y)l(,)j(the)e(w)o(ork)g(of) f(this)i(c)o(hapter)f(should)h(b)q(e)g(to)e(minimize)j(the)f(n)o(um)o (b)q(er)f(of)g(new)75 1491 y(functions)d(in)o(tro)q(duced)g(in)o(to)g (MPI)f(pro\014les)h(to)e(pro)o(vide)i(these)f(features.)75 1612 y Fd(11.2.2)49 b(Virtual)17 b(Channels)75 1698 y Fl(The)i(idea)h(of)f(a)f(soft)o(w)o(are)f(virtual)j(c)o(hannel)g(\(V)o (C\))e(is)i(not)e(new)h(in)h(a)f(parallel)h(mac)o(hine.)32 b(Thinking)75 1755 y(Mac)o(hines)14 b(has)g(it)g(on)g(the)g(CM-5)f(via) h(CMMD)f(where)h(the)g(motiv)m(ation)g(is)h(to)e(minimize)j (handshaking)75 1811 y(o)o(v)o(erhead.)j(Our)d(motiv)m(ations)f(for)g (ha)o(ving)h(V)o(Cs)e(in)i(MPI/R)l(T)g(are:)143 1905 y Fg(\017)23 b Fl(Abilit)o(y)16 b(to)f(exploit)h(p)q(ersisten)o(t)g (comm)o(unication)143 1999 y Fg(\017)23 b Fl(Deadlo)q(c)o(k)15 b(a)o(v)o(oidance)143 2093 y Fg(\017)23 b Fl(Once)16 b(established,)g(V)o(C)f(should)h(guaran)o(tee)e(prop)q(erties)i (critical)g(for)f(timing)g(correctness)g(lik)o(e:)240 2186 y Ff({)23 b Fl(Bounds)15 b(on)h(end-to-end)g(dela)o(y)240 2259 y Ff({)23 b Fl(Minim)o(um)16 b(bandwidth)240 2332 y Ff({)23 b Fl(Bu\013er)15 b(space)1967 46 y Fc(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fe(11.2.)34 b(ISSUES)16 b(IN)g(REAL-TIME)g(COMPUTING) 837 b Fl(3)75 49 y Fd(11.2.3)49 b(Deadlo)q(cks)75 135 y Fl(It)18 b(is)g(imp)q(ortan)o(t)g(to)f(b)q(e)i(able)f(to)g(prev)o(en) o(t,)g(a)o(v)o(oid)f(or)h(in)h(the)f(w)o(orst)e(case)i(detect)g(and)g (reco)o(v)o(er)g(from)75 192 y(deadlo)q(c)o(ks.)26 b(The)17 b(problem)g(of)g(deadlo)q(c)o(ks)h(is)f(di\016cult)h(in)g(distributed)g (real-time)g(systems)e(and)h(it)g(is)75 248 y(exacerbated)e(with)h (prioritization.)166 304 y(Tw)o(o)e(p)q(ossible)j(side)f(issues)g(are)f (as)g(follo)o(ws:)143 398 y Fg(\017)23 b Fl(Liv)o(elo)q(c)o(k)d(|)e(it) h(happ)q(ens)h(when)f(t)o(w)o(o)e(in)o(teracting)i(tasks)f(miss)g (deadlines)j(due)e(to)f(one)g(doing)189 455 y(some)d(secondary)g (activit)o(y)g(when)h(the)f(other)g(tries)g(to)g(rendezv)o(ous.)143 549 y Fg(\017)23 b Fl(Orphan)d(pro)q(cesses)h(|)f(in)h(the)f(presence)h (of)e(spa)o(wn)h(capabilit)o(y)i(\(a)d(pro)q(cess)h(creating)g(other) 189 605 y(pro)q(cesses\))15 b(and)g(failures.)75 727 y Fd(11.2.4)49 b(Scheduling)75 812 y Fl(Should)19 b(sc)o(heduling)g(b)q (e)f(done)g(within)h(MPI?)f(Once)g(it's)f(outside)h(MPI's)f(domain,)h (it)g(b)q(ecomes)g(OS-)75 869 y(dep)q(enden)o(t)c(and)e(hence)i(w)o(e)e (lo)q(ose)g(p)q(ortabilit)o(y)h(to)f(some)g(exten)o(t.)19 b(One)13 b(p)q(ossiblit)o(y)h(is)e(that)g(middlew)o(are)75 925 y(b)q(et)o(w)o(een)k Fh(MPI)e Fl(and)h(the)h(OS)g(will)g(handle)h (sc)o(heduling.)75 1047 y Fd(11.2.5)49 b(Prio)o(ritization)75 1133 y Fl(P)o(ossible)16 b(candidates)g(for)f(prioritization)h(in)g (MPI)g(include:)22 b(messages,)14 b(pro)q(cesses,)h(and/or)g(comm)o(u-) 75 1189 y(nicators.)20 b(Issues)c(include:)143 1283 y Fg(\017)23 b Fl(Static)15 b(vs)h(Dynamic)g(priorities)h(|)f(It)g(is)g (con)o(v)o(enien)o(t)g(to)f(ha)o(v)o(e)g(static)h(priorities,)g(but)g (without)189 1340 y(the)k(preemption)h(capabilit)o(y)l(,)h(priorities)f (need)g(to)f(b)q(e)g(dynamically)i(c)o(hanged)f(to)e(cop)q(e)i(with)189 1396 y(un)o(b)q(ounded)16 b(priorit)o(y)g(in)o(v)o(ersion.)143 1490 y Fg(\017)23 b Fl(Resource)16 b(Allo)q(cation)h(|)g(It's)e(ob)o (vious)h(that)f(if)h(an)g(en)o(tit)o(y)g(has)f(a)h(priorit)o(y)g(then)g (the)g(resource)189 1546 y(that)h(it)h(o)q(ccupies)i(should)e(inherit)i (that)d(priorit)o(y)l(.)28 b(Then)19 b(w)o(e)e(should)i(outline)h(ho)o (w)d(resources)189 1603 y(are)e(allo)q(cated)h(and)f(freed)h(and)f(ho)o (w)g(con)o(ten)o(tions)g(are)g(resolv)o(ed.)143 1697 y Fg(\017)23 b Fl(E\013ect)10 b(on)i(Progress)e(Rule)j(|)e(In)h(the)g (presence)g(of)f(prioritized)i(comm)o(unication)f(and)f(deadlines,)189 1753 y(w)o(e)18 b(need)i(to)f(establish)h(deterministic)h(progress)d (rules)i(for)f(in)o(tra-comm)o(unicator)f(and)i(in)o(ter-)189 1810 y(comm)o(unicator)14 b(comm)o(unication.)75 1931 y Fd(11.2.6)49 b(F)o(ault)16 b(T)l(olerance)75 2017 y Fl(Issues)g(of)f(fault)g(tolerance)g(are)g(imp)q(ortan)o(t)g(for)g (real-time)h(and)f(em)o(b)q(edded)i(systems:)143 2111 y Fg(\017)23 b Fl(Limit)16 b(failures)143 2205 y Fg(\017)23 b Fl(F)l(ault)15 b(detection)h(and)f(reco)o(v)o(ery)143 2299 y Fg(\017)23 b Fl(Pro)o(vide)15 b(con)o(trol)g(o)o(v)o(er)f(lo)q (cal)j(bu\013ers)143 2392 y Fg(\017)23 b Fl(Replication)143 2486 y Fg(\017)g Fl(History)15 b(preserv)m(ation)-32 46 y Fc(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fl(4)187 b Fe(CHAPTER)16 b(11.)34 b(REAL-TIME,)16 b(EMBEDDED)e(EXTENSIONS,)i(AND)f(PR)o(OFILES)75 49 y Fi(11.3)59 b(Time-Driven)75 151 y Fh(MPI-2)14 b Fl(will)j(address:)143 238 y Fg(\017)23 b Fl(sync)o(hronized)16 b(clo)q(c)o(ks)143 329 y Fg(\017)23 b Fl(k)o(ey)17 b(functionalit)o(y)h(for)f(sc)o (heduling)i(messages,)e(b)o(y)g(adding)h(p)q(ersisten)o(t)g(send)f(and) h(receiv)o(e)g(ca-)189 385 y(pabilit)o(y)e(tied)g(to)f(a)g(sc)o (heduler)h(in)o(terface.)143 476 y Fg(\017)23 b Fl(\\Middlew)o(are")15 b(exemplars)h(will)h(demonstrate)d(in)o(teraction)i(of)f(calendar)h (with)f(MPI.)166 563 y(The)g(sp)q(eci\014c)i(in)o(terfaces)f(for)e(the) i(new)f(functions)h(ha)o(v)o(e)f(not)g(b)q(een)h(completed,)g(as)f(y)o (et.)75 705 y Fi(11.4)59 b(Event-Driven)75 807 y Fl(No)15 b(prop)q(osals)g(ha)o(v)o(e)g(b)q(een)h(made)g(in)g(this)f(area.)75 949 y Fi(11.5)59 b(Prio)n(rit)n(y)21 b(Based)e(Additions)h(to)f(MPI)75 1050 y Fl(Recall)h(that)d(MPI)h(pro)q(cesses)h(ha)o(v)o(e)f(the)g(same) g(priorit)o(y)g(curren)o(tly)l(.)29 b(This)19 b(sub-prop)q(osal)g (addresses)75 1107 y(messaging)c(priorit)o(y)l(,)g(not)g(pro)q(cess)h (priorit)o(y)l(.)166 1163 y(This)g(sub-prop)q(osal)g(do)q(es)f(not)g(y) o(et)g(address)g(in)o(tercomm)o(unicator)g(v)m(arian)o(ts.)75 1283 y Fd(11.5.1)49 b(Prio)o(rit)o(y)75 1369 y Fl(A)22 b(subsystem)h(with)f(its)h(o)o(wn)f Fb(M)p Fl(PI)p 727 1369 14 2 v 17 w(COMM)p 896 1369 V 16 w(W)o(ORLD)g(will)i(ha)o(v)o(e)e (a)g(giv)o(en)h(n)o(um)o(b)q(er)g(of)f(ph)o(ysical)75 1426 y(priorities.)d(F)l(or)10 b(eac)o(h)g(priorit)o(y)l(,)i(w)o(e)e (assume)g(an)g(unlimited)i(n)o(um)o(b)q(er)f(of)f(comm)o(unications)h (are)f(p)q(ossible.)75 1482 y(Additional)16 b(priorities)f(ma)o(y)f(b)q (e)h(a)o(v)m(ailable)g(for)f(subsets)g(of)g(this)h(W)o(ORLD,)f(but)g(w) o(e)g(will)i(assume)e(that)75 1539 y(the)h(w)o(orld)g(is)g(constructed) g(in)h(a)f(w)o(a)o(y)f(to)g(b)q(e)h(inclusiv)o(e)j(of)c(all)i(p)q (ossible)h(priorities)f(of)e(the)h(subsets.)20 b(In)75 1595 y(this)c(w)o(a)o(y)l(,)e(comm)p 387 1595 V 16 w(splits)j(or)e (comm)p 698 1595 V 16 w(creates)g(will)i(rev)o(eal)f(additional)h (priorit)o(y)f(lev)o(els)g(without)g(loss)f(of)75 1652 y(generalit)o(y)l(,)h(or)e(need)i(for)f(ph)o(ysical)h(priorit)o(y)g (ren)o(um)o(b)q(ering.)166 1708 y(W)l(e)h(assume)g(that)f(ev)o(ery)h (group)g(will)i(ha)o(v)o(e)d(at)h(its)g(disp)q(osal)h(one)g(or)e(more)h (ph)o(ysical)h(priorities.)75 1764 y(The)d(lo)o(w)o(est)g(suc)o(h)h (priorit)o(y)f(is)h(alw)o(a)o(ys)e(0,)h(and)g(the)g(set)g(of)g(a)o(v)m (ailabile)j(priorities)e(is)g(as)e(follo)o(ws:)75 1851 y Fa(physical_priorities)21 b(:=)j({priority_i,i=0...K-1})d(\(some)i (set)g(of)h(priority)f(levels\))290 1908 y([of)g(a)h(comm)f(or)h (group])75 1995 y Fl(MPI)15 b(logical)i(priorities)f(ha)o(v)o(e)f(the)g (same)g(form)75 2082 y Fa(logical_priorities)21 b(:=)j({i};)f(i)h(=)g (0,)f(mpi\\_max\\_logical\\_priority-1)75 2169 y Fl(MPI)15 b(sp)q(eci\014es)i(a)e(w)o(a)o(y)f(to)h(map)g(b)q(et)o(w)o(een)g(the) 266 2256 y Fa(logical_priorities)21 b(->)j(physical)f(priorities)75 2343 y Fl(The)15 b(map)g(ma)o(y)g(b)q(e)h(as)f(follo)o(ws:)143 2430 y Fg(\017)23 b Fl(default)15 b(ev)o(erything)h(to)f(priorit)o(y)g (0)75 2517 y(And,)g(if)h(not)f(default,)143 2604 y Fg(\017)23 b Fl(completely)16 b(\014xed)g(b)o(y)f(the)g(en)o(vironmen)o(t)143 2695 y Fg(\017)23 b Fl(c)o(hangeable)16 b(in)g(all)g(or)f(in)h(part)e (b)o(y)h(the)h(user)1967 46 y Fc(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fe(11.5.)29 b(PRIORITY)17 b(BASED)e(ADDITIONS)h(TO)g (MPI)781 b Fl(5)75 49 y Fd(11.5.2)49 b(Inquiry)75 135 y Fl(Appropriate)16 b(inquiry)g(functions)g(will)h(b)q(e)f(needed:)166 192 y(Getting)f(ph)o(ysical)h(priorit)o(y)g(lev)o(els)g(of)f(a)g(GR)o (OUP)g(or)g(COMM:)147 285 y Fa(MPI_GROUP_GET_PHYS_PRIO)o(_LENGTH)o (\(MPI_GRO)o(UP)21 b(GRP,)i(INTEGER)g(LENGTH\))147 342 y(IN)g(GRP)147 398 y(OUT)g(LENGTH)147 455 y(MPI_GROUP_GET_PHYS_PRIO)o (\(MPI_GR)o(OUP)e(GRP,)i(INTEGER)g(MAXLENGTH,)719 511 y(INTEGER)g(PMAP[]\))147 568 y(IN)g(GRP)147 624 y(IN)g(MAXLENGTH)147 681 y(OUT)g(PMAP[LENGTH])75 774 y Fl(The)15 b(ab)q(o)o(v)o(e)g(ph)o (ysical)i(information)e(is)h(pro)o(vided.)k(The)c(COMM)f(v)o(ersions)g (are)g(quite)h(similar:)147 868 y Fa(MPI_COMM_GET_PHYS_PRIO_)o (LENGTH\()o(MPI_COMM)k(COMM,)j(INTEGER)g(LENGTH\))147 925 y(IN)g(COMM)147 981 y(OUT)g(LENGTH)147 1037 y (MPI_COMM_GET_PHYS_PRIO\()o(MPI_COM)o(M)e(COMM,)i(INTEGER)g(MAXLENGTH,) g(INTEGER)g(PMAP[]\))147 1094 y(IN)g(COMM)147 1150 y(IN)g(MAXLENGTH)147 1207 y(OUT)g(PMAP[LENGTH])75 1301 y Fl(The)18 b(ab)q(o)o(v)o (e-de\014ned)g(ph)o(ysical)h(map)e(is)h(pro)o(vided.)27 b(This)18 b(is)g(pro)o(vided)g(so)f(programs)f(can)h(strategize)75 1357 y(ab)q(out)e(ho)o(w)g(to)f(use)i(a)o(v)m(ailable)h(priorit)o(y)e (sc)o(hemes)h(on)f(the)g(\015y)l(.)20 b(These)c(are)f(lo)q(cal)h(op)q (erations.)166 1414 y(Maps)f(apply)h(only)f(to)g(comm)o(unicators:)147 1507 y Fa(MPI_PRIORITY_GET_LENGTH)o(_OF_MAP)o(\(MPI_COM)o(M)21 b(COMM,)i(INTEGER)g(LENGTH\))147 1564 y(IN)g(COMM)147 1620 y(OUT)g(LENGTH)147 1733 y(MPI_PRIORITY_GET_MAP\(MP)o(I_COMM)d (COMM,)k(INTEGER)f(MAXLENGTH,)f(INTEGER)h(MAP[][3]\))147 1790 y(IN)g(COMM)147 1846 y(IN)g(MAXLENGTH)147 1903 y(OUT)g (MAP[LENGTH][3])166 1996 y Fl(These)15 b(are)g(lo)q(cal)i(op)q (erations.)j(MAP)15 b(con)o(tains)g(\(for)f(eac)o(h)h(index)i(i\):)143 2090 y Fg(\017)23 b Fl(ith)15 b(index)i(of)e(map)g(describ)q(es)h(ith)g (logical)h(priorit)o(y)143 2184 y Fg(\017)23 b Fl(map[i][0])14 b(is)h(curren)o(t)g(n)o(um)o(b)q(er)h(of)f(lo)q(cal)h(comms)f(for)f (this)i(priorit)o(y)143 2278 y Fg(\017)23 b Fl(map[i][1])14 b(is)h(ph)o(ysical)i(priorit)o(y)e(this)h(logical)g(priorit)o(y)g(maps) e(to)143 2372 y Fg(\017)23 b Fl(map[i][2])14 b(is)h(the)g(logical)i (\015ag)e(set)g(\(c)o(hangeable)g(or)g(not,)g(etc\))147 2478 y Fa(MPI_PRIORITY_SET_MAP\(MP)o(I_COMM)20 b(COMM,)k(INTEGER)f (root,)147 2534 y(INTEGER)f(LENGTH,)h(INTEGER)g(INMAP[][3],)147 2591 y(LOGICAL)f(MASK[],)h(INTEGER)g(OUTMAP[][3]\))147 2647 y(IN)g(COMM)147 2704 y(IN)g(root)g(\(The)h(process)f(with)g(the)g (map,)h(all)f(get)h(the)f(new)h(map)f(on)h(output\))-32 46 y Fc(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fl(6)187 b Fe(CHAPTER)16 b(11.)34 b(REAL-TIME,)16 b(EMBEDDED)e(EXTENSIONS,)i(AND)f(PR)o(OFILES)147 49 y Fa(IN)23 b(LENGTH)147 106 y(IN)g(INMAP[LENGTH][3])147 162 y(IN)g(MASK[LENGTH])147 219 y(OUT)g(OUTMAP[LENGTH][3])147 275 y(OUT)g(IERROR)166 380 y Fl(In)16 b(this)f(v)m(arian)o(t,)g(ev)o (ery)o(one)g(kno)o(ws)g(the)g(new)g(map)g(on)h(input:)147 472 y Fa(MPI_PRIORITY_ALLSET_MAP)o(\(MPI_CO)o(MM)21 b(COMM,)147 529 y(INTEGER)h(LENGTH,)h(INTEGER)g(INMAP[][3],)147 585 y(LOGICAL)f(MASK[],)h(INTEGER)g(OUTMAP[][3],)g(INTEGER)f(IERROR\))166 678 y Fl(These)17 b(calls)h(will)g(b)q(e)f(collectiv)o(e)i(o)o(v)o(er)d (COMM's)f(group.)24 b(It)17 b(ma)o(y)f(require)h(a)g(condition)h(of)e (qui-)75 734 y(escence)h(o)o(v)o(er)e(all)i(the)f(groups)g(pro)q (cesses)h(and)f(comm)o(unicators)f(to)h(b)q(e)h(appropriate.)22 b(Quiescence)c(is)75 791 y(a)g(condition)i(originally)h(describ)q(ed)f (in)g(the)e(MPI-1)h(do)q(cumen)o(t,)g(but)g(whic)o(h)h(w)o(as)d(a)o(v)o (oided)i(through)75 847 y(restructuring)d(of)g(comm)o(unicator)f(seman) o(tics.)22 b(Essen)o(tially)l(,)17 b(it)f(means)g(a)g(programmatic)f (guaran)o(tee)75 904 y(that)f(no)i(relev)m(an)o(t)f(message)g(passing)h (is)g(on-going)f(while)i(the)e(c)o(hange)g(is)h(made.)166 960 y(MASK's)k(en)o(tries)h(con)o(trol)f(whether)g(a)g(c)o(hange)h(is)f (attempted)g(for)g(eac)o(h)g(en)o(try)l(.)35 b(MASK[i])21 b(=)75 1017 y(.false.)f(implies)d(no)e(c)o(hange)h(attempt,)d(MASK[i])j (=)f(.true.)20 b(implies)d(an)e(attempt)f(to)h(c)o(hange.)166 1073 y(Setting)g(ma)o(y)g(mak)o(e)g(the)g(logical)h(map)f(longer)h(or)f (shorter)f(than)h(it)h(w)o(as)e(previously)l(.)166 1130 y(Logical)k(priorities)h(will)g(use)f(ph)o(ysical)g(priorities)h(in)f (monotonically)g(decreasing)g(order.)27 b(That)75 1186 y(is,)15 b(logical)i(priorit)o(y)e(10)g(cannot)f(use)i(a)f(higher)h(ph) o(ysical)g(priorit)o(y)f(than)g(logical)i(priorit)o(y)e(9,)g(but)g (they)75 1242 y(could)h(use)g(the)f(same)g(ph)o(ysical)h(priorit)o(y)l (.)75 1364 y Fd(11.5.3)49 b(Prop)q(erties)16 b(of)g(Logical)j(Prio)o (rities)75 1450 y Ff(Defaults)24 b Fl(The)15 b(logical)h(priorit)o(y)f (of)g(MPI)p 830 1450 14 2 v 16 w(COMM)p 998 1450 V 16 w(W)o(ORLD's)g(are)g(0.)k(It)c(is)h(the)f(lo)o(w)o(est)f(priorit)o(y)h (of)189 1506 y(messaging)g(among)f(the)i(group)e(of)h(pro)q(cesses)h (constituting)g(the)f(w)o(orld.)75 1600 y Ff(Inheritance)j(of)g (Priorit)o(y)k Fl(A)17 b(comm)o(unicator)g(gets)g(a)g(priorit)o(y)g (when)h(it)f(is)h(created.)26 b(A)17 b(comm)o(u-)189 1656 y(nicator)f(inherits)h(the)f(priorit)o(y)g(of)g(its)g(paren)o(t)g (when)h(dup'd)f(or)g(otherwise)g(created.)22 b(T)l(o)16 b(a)o(v)o(oid)189 1712 y(an)f(un)o(w)o(an)o(ted)g(incompatibilit)o(y)i (with)f(non-prioritied)h(comm)o(unication,)75 1806 y Ff(New)g(Priorities)h(Lev)o(el)f(Comm)o(unicators)22 b Fl(A)13 b(v)m(arian)o(t)g(of)g Fa(MPI)p 1269 1806 15 2 v 16 w(COMM)p 1381 1806 V 17 w(DUP)f Fl(is)i(pro)o(vided)g(to)e (assert)189 1862 y(a)j(comm)o(unicator)f(at)h(a)g(new)g(priorit)o(y:) 236 1973 y Fa(MPI_COMM_DUP_PRIORITY\(MPI_COM)o(M)21 b(OLD,)i(INTEGER)g (PRIO,)h(MPI_COMM)e(NEW\))236 2030 y(IN)i(OLD)236 2086 y(IN)g(PRIO)236 2143 y(OUT)g(NEW)189 2254 y Fl(This)19 b(call)h(collectiv)o(ely)h(attempts)d(to)g(create)h(a)g(dup'd)g(comm)o (unicator)g(at)f(the)h(new)g(priorit)o(y)189 2310 y(lev)o(el.)i(An)15 b(error)g(o)q(ccurs)g(if)h(the)f(priorit)o(y)h(is)f(not)g(a)o(v)m (ailable.)236 2421 y Fa(MPI_COMM_SPLIT_PRIORITY\(MPI_C)o(OMM)21 b(OLD,)i(INTEGER)g(COLOR,)g(INTEGER)g(KEY,)236 2478 y(INTEGER)g(PRIO,)g (MPI_COMM)g(OUT\))236 2534 y(IN)h(OLD)236 2591 y(IN)g(PRIO)236 2647 y(IN)g(COLOR,)f(KEY)236 2704 y(OUT)h(NEW)1967 46 y Fc(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fe(11.5.)34 b(PRIORITY)17 b(BASED)e(ADDITIONS)h(TO)g (MPI)776 b Fl(7)189 49 y(This)23 b(call)h(collectiv)o(ely)h(splits)f (OLD)f(in)o(to)f(disjoin)o(t)i(comm)o(unicators)e(\(a)g(total)g(of)h (COLOR)189 106 y(suc)o(h)g(comm)o(unicators\).)44 b(F)l(or)23 b(a)g(giv)o(en)h(color)g(all)g(callers)g(m)o(ust)f(sp)q(ecify)i(the)f (same)f(prior-)189 162 y(it)o(y)18 b(PRIO,)i(or)e(the)h(call)g(is)h (erroneous.)30 b(COLOR/KEY)20 b(otherwise)f(b)q(eha)o(v)o(es)g(as)f (for)g(regular)189 219 y(MPI)p 281 219 14 2 v 16 w(COMM)p 449 219 V 16 w(SPLIT.)189 293 y(If)13 b(an)o(y)g(of)g(the)g(priorit)o (y)g(lev)o(els)i(is)e(una)o(v)m(ailable,)j(the)d(call)h(fails)g(only)g (for)e(those)h(that)g(are)f(una)o(v)m(ail-)189 349 y(able)k (\(returning)f(MPI)p 594 349 V 16 w(COMM)p 762 349 V 16 w(NULL)h(in)g(OUT\).)f(The)h(others)e(succeed.)75 470 y Fd(11.5.4)49 b(Note)75 556 y Fl(It)14 b(m)o(ust)e(b)q(e)j (considered)f(whether)g(to)f(allo)o(w)h(comm)o(unicators)f(to)f(alter)i (their)g(priorit)o(y)f(after)g(creation.)75 613 y(In)j(man)o(y)f(MPI)g (situations,)h(the)f(ob)s(jects)g(do)g(not)g(c)o(hange)h(their)g(state) e(except)i(in)g(extremely)g(limited)75 669 y(w)o(a)o(ys)d(after)h (creation.)19 b(A)c(c)o(hange)f(of)g(priorit)o(y)g(seems)g(to)g(b)q(e)h (an)f(OK)h(thing)g(to)e(allo)o(w)i(for)e(a)h(comm)o(uni-)75 725 y(cator,)g(but)h(a)g(requiremen)o(t)h(of)e(quiescence)k(on)d(that)f (comm)o(unicator)h(is)h(probably)f(needed.)21 b(In)16 b(order)75 782 y(to)i(a)o(v)o(oid)g(realtime)h(p)q(erformance)f (problems)h(sited)g(at)f(the)g(outset,)g(dynamic)h(c)o(hange)g(of)f (priorities)75 838 y(seems)d(to)g(b)q(e)h(imp)q(ortan)o(t)e(as)h(w)o (ell,)h(though)f(this)h(migh)o(t)f(b)q(e)h(in)o(visible)i(to)c(the)i (application.)166 895 y(W)l(e)f(do)g(not)g(prop)q(ose)g(constructors)g (for)f(top)q(ology)h(here)h(for)e(t)o(w)o(o)g(reasons:)143 984 y Fg(\017)23 b Fl(w)o(e)15 b(are)h(going)g(to)f(prop)q(ose)h (additional)h(top)q(ology)f(ops)g(elsewhere)h(that)e(will)j(mak)o(e)d (it)h(p)q(ossible)189 1041 y(to)e(add)i(top)q(ology)e(to)h(existing)h (comm)o(unicators.)143 1133 y Fg(\017)23 b Fl(it)15 b(is)h(not)f(clear) g(if)h(priorit)o(y)f(based)h(comm)o(unicators)f(will)i(b)q(e)e(used)h (with)g(top)q(ology)f(often.)143 1225 y Fg(\017)23 b Fl(w)o(e)15 b(can)g(\014x)g(this)h(easily)g(later)f(for)g(symmetry)f (considerations,)i(if)g(deemed)g(essen)o(tial.)166 1315 y(Without)d(a)g(sp)q(ecial)i(call)g(to)d(mo)q(dify)i(priorit)o(y)l(,)g (here)g(is)g(ho)o(w)f(logical)h(priorit)o(y)g(could)g(b)q(e)g(c)o (hanged:)75 1404 y Fa(MPI_COMM_DUP_PRIORITY\(old,)20 b(new,)k(newprio\);)75 1461 y(MPI_COMM_FREE\(old\);)75 1551 y Fl(And)16 b(the)g(p)q(ersisten)o(t)g(data)f(ob)s(jects)g(using)h ("old")f(are)h(presumably)g(dropp)q(ed)h(or)e(\014xed)h(up.)21 b(P)o(ossibly)75 1607 y(this)16 b(is)f(painful.)22 b(Hence)16 b(w)o(e)f(prop)q(ose:)75 1697 y Fa(MPI_COMM_CHANGE_PRIORITY\(M)o (PI_COMM)20 b(COMM,)k(INTEGER)e(NEWPRIORITY\))75 1753 y(INOUT)h(COMM)75 1810 y(INOUT)g(NEWPRIORITY)75 1899 y Fl(This)d(is)g(a)g(collectiv)o(e)h(call,)h(that)d(c)o(hanges)g(the)h (logical)h(priorit)o(y)f(of)f(a)h(comm)o(unicator)f Fa(COMM)p Fl(.)g Fa(COMM)75 1956 y Fl(is)h(INOUT)g(here)g(\(un)o(usual)h(for)e (MPI,)g(as)g(noted)h(ab)q(o)o(v)o(e\).)32 b(NEWPRIORITY)21 b(is)f(the)g(new)g(logical)75 2012 y(priorit)o(y)i(prop)q(osed,)h(and)f (the)g(NEWPRIORITY)h(on)f(output)g(is)g(the)g(priorit)o(y)g(ac)o(hiev)o (ed.)40 b(If)22 b(the)75 2068 y(priorit)o(y)15 b(cannot)g(b)q(e)h(c)o (hanged,)f(it)h(will)h(b)q(e)f(detectable)g(b)o(y)f(c)o(hec)o(king)h (NEWPRIORITY.)166 2125 y(F)l(or)9 b(bac)o(kw)o(ard)h(compatibilit)o(y)l (,)i(the)f(priorit)o(y)f(of)g(MPI)p 1095 2125 V 16 w(COMM)p 1263 2125 V 16 w(W)o(ORLDs)g(ma)o(y)g(not)g(b)q(e)g(c)o(hanged.)75 2181 y(It)16 b(will)h(b)q(e)f(necessary)g(to)f(w)o(ork)g(with)h(a)g (DUP)f(of)g(MPI)p 1045 2181 V 17 w(COMM)p 1214 2181 V 16 w(W)o(ORLD,)g(in)i(general,)f(for)f(priorit)o(y)75 2238 y(comm)o(unication.)75 2359 y Fd(11.5.5)49 b(Ho)o(w)17 b(p)o(rio)o(rit)o(y)f(messaging)g(in)h(MPI)g(should)g(w)o(o)o(rk)e(\(a) o(rea)h(of)g(lots)h(of)f(discussion\))75 2445 y Fl(Curren)o(tly)l(,)d (MPI)f(do)q(es)g(not)g(p)q(ose)g(ordering)h(requiremen)o(ts)f(b)q(et)o (w)o(een)g(comm)o(unicators.)19 b(F)l(urthermore,)75 2501 y(all)d(messaging)f(on)g(a)g(comm)o(unicator)g(is)h(at)e(a)h (single)i(lev)o(el)f(of)f(priorit)o(y)l(.)166 2558 y(MPI)g(implemen)o (tations)i(m)o(ust)d(do)h(at)g(least)g(the)g(follo)o(wing:)143 2647 y Fg(\017)23 b Fl(When)16 b(it)h(is)g(p)q(ossible)h(to)e(handle)i (t)o(w)o(o)d(order)h(more)g(receiv)o(e)h(requests)g(at)f(the)g(same)g (time,)h(the)189 2704 y(highest)e(priorit)o(y)h(request)f(is)h(handled) -32 46 y Fc(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fl(8)187 b Fe(CHAPTER)16 b(11.)34 b(REAL-TIME,)16 b(EMBEDDED)e(EXTENSIONS,)i(AND)f(PR)o(OFILES)143 49 y Fg(\017)23 b Fl(When)16 b(t)o(w)o(o)f(or)h(more)f(send)i(requests)f (are)g(in)h(progress,)e(the)i(highest)f(priorit)o(y)h(request)f(should) 189 106 y(b)q(e)g(handled.)143 200 y Fg(\017)23 b Fl(The)18 b(system)g(m)o(ust)g(accept)h(en)o(v)o(elop)q(es)g(remote)f(senders)h (in)g(order)f(to)g(supp)q(ort)g(the)h(progress)189 256 y(rule)143 350 y Fg(\017)k Fl(When)13 b(higher)h(priorit)o(y)f (transmissions)h(are)f(blo)q(c)o(k)o(ed,)h(lo)o(w)o(er)e(priorit)o(y)i (messages)e(m)o(ust)h(b)q(e)h(han-)189 406 y(dled)19 b(\(this)f(means)g(that)g(some)f(priorit)o(y)i(in)o(v)o(ersion)g (e\013ects)e(are)h(inevitable,)j(and)d(w)o(e)g(need)h(to)189 463 y(discuss)d(ho)o(w)f(w)o(e)g(w)o(an)o(t)f(MPI)h(to)f(b)q(eha)o(v)o (e\))75 584 y Fd(11.5.6)49 b(Other)16 b(Comments)143 670 y Fg(\017)23 b Fl(W)l(e)f(m)o(ust)f(consider)i(whether)g(some)e (statemen)o(t)g(ab)q(out)h(fairness)h(is)f(useful)h(in)g(the)f(face)g (of)189 727 y(priorit)o(y)15 b(comm)o(unication.)21 b(F)l(airness)15 b(is)h(not)f(necessarily)h(go)q(o)q(d.)143 821 y Fg(\017)23 b Fl(W)l(e)13 b(need)h(a)e(mec)o(hanism)i(to)e(a)o(v)o(oid)h(deadlo)q (c)o(ks)h(due)f(to)g(priorit)o(y)g(messaging)g({)g(ceiling)i (priorities)189 877 y(/)g(priorit)o(y)g(inheritance)75 983 y(And)g(w)o(e)f(need)i(to)d(en)o(umerate)i(the)f(kinds)i(of)e (problems)h(that)f(can)h(arise)f(b)q(ecause)i(of)e(priorities)i(in)f (the)75 1040 y(lo)o(w)g(lev)o(els)i(of)d(the)i(implemen)o(tation.)75 1161 y Fd(11.5.7)49 b(Other)16 b(situations)75 1247 y Fl(The)f(follo)o(wing)h(conditions)g(will)g(b)q(e)g(addressed)f(in)g (the)g(next)g(v)o(ersion)g(of)g(the)f(priorit)o(y)h(sub-prop)q(osal:) 143 1341 y Fg(\017)23 b Fl(Priorit)o(y)15 b(maps)g(when)g(comms)g (split)143 1435 y Fg(\017)23 b Fl(Priorit)o(y)15 b(maps)g(when)g(w)o (orlds)g(merge)75 1578 y Fi(11.6)59 b(Issues)19 b(on)g(Emb)r(edded)f (Systems)75 1680 y Fl(F)l(or)e(em)o(b)q(edded)j(systems,)d(the)h(follo) o(wing)h(issues)g(b)q(ecome)f(more)g(imp)q(ortan)o(t)f(than)h(in)h (other)e(uses)h(of)75 1736 y Fh(MPI)p Fl(:)143 1830 y Fg(\017)23 b Fl(Small)16 b(amoun)o(ts)e(of)h(program)f(space)i(\(co)q (de)f(bloat)g(unacceptable\),)143 1924 y Fg(\017)23 b Fl(Small)17 b(amoun)o(t)e(of)h(data)g(space)g(\(bu\013ering)h(limited,) h(maxim)o(um)e(message)g(sizes)h(ma)o(y)f(b)q(e)h(lim-)189 1980 y(ited\),)143 2074 y Fg(\017)23 b Fl(Simpli\014ed)18 b(programming)c(en)o(vironmen)o(ts)i(as)f(compared)g(to)f(full-blo)o (wn)j(OS's,)143 2168 y Fg(\017)23 b Fl(Static)15 b(loading)h(en)o (vironmen)o(ts)f(more)g(nearly)h(compatible)g(with)g(MPI-1)f(view)h(of) e(the)i(w)o(orld.)75 2262 y(Suc)o(h)g(systems)f(ma)o(y)f(also)h(ha)o(v) o(e)g(realtime)h(requiremen)o(ts.)166 2318 y(MPI)21 b(is)h(already)f (migrating)g(in)o(to)h(em)o(b)q(edded)g(protot)o(yp)q(e)f(systems;)i (hence)f(it)g(is)f(in)o(teresting)75 2374 y(to)g(o\013er)f (suggestions,)j(and)e(p)q(ossibly)i(additional)g(pro\014le)f(language,) h(to)e(address)g(this)h(particular)75 2431 y(space)16 b(of)f(applications)i(and)f(systems.)21 b(One)16 b(particular)g (approac)o(h)g(will)h(b)q(e)f(to)f(o\013er)g(a)g(set)h(of)f(subset)75 2487 y(implemen)o(tation)g(pro\014les)g(for)e(em)o(b)q(edded)j (systems,)d(so)h(that)f(if)h(subsetting)h(should)g(o)q(ccur,)f(it)g (can)g(b)q(e)75 2544 y(done)i(according)f(to)g(a)g(systematic)g(con)o (v)o(en)o(tion)g(do)q(cumen)o(ted)h(in)g(the)f Fh(MPI-2)g Fl(standard.)1967 46 y Fc(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Nov 27 09:24:51 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA14150; Mon, 27 Nov 1995 09:24:33 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA11019 for mpi-core-out; Mon, 27 Nov 1995 08:22:13 -0600 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA10645 for ; Mon, 27 Nov 1995 07:56:43 -0600 Received: (tony@localhost); by Aurora.CS.MsState.Edu (SMI-8.6/7.0m-FWP-MsState); id HAA03293; Mon, 27 Nov 1995 07:56:37 -0600 Date: Mon, 27 Nov 1995 07:56:37 -0600 From: Tony Skjellum Message-Id: <199511271356.HAA03293@Aurora.CS.MsState.Edu> To: mpi-core@mcs.anl.gov Subject: Collective chapter : SC'95 version Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The following postscript is the promised update and integration of collective chapter for SC'95. Clearly, more work remains to be done for the next MPI meetings, but it is much further along... -Tony Skjellum ---cut here--- %!PS-Adobe-2.0 %%Creator: dvipsk 5.55a Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%Pages: 24 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, compressed, comments removed %DVIPSSource: TeX output 1995.11.27:0753 %%BeginProcSet: texc.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]/id ch-image N /rw ch-width 7 add 8 idiv string N /rc 0 N /gp 0 N /cp 0 N{rc 0 ne{rc 1 sub /rc X rw}{G}ifelse}imagemask restore}B /G{{id gp get /gp gp 1 add N dup 18 mod S 18 idiv pl S get exec}loop}B /adv{cp add /cp X}B /chg{rw cp id gp 4 index getinterval putinterval dup gp add /gp X adv}B /nd{/cp 0 N rw exit}B /lsh{rw cp 2 copy get dup 0 eq{pop 1}{ dup 255 eq{pop 254}{dup dup add 255 and S 1 and or}ifelse}ifelse put 1 adv}B /rsh{rw cp 2 copy get dup 0 eq{pop 128}{dup 255 eq{pop 127}{dup 2 idiv S 128 and or}ifelse}ifelse put 1 adv}B /clr{rw cp 2 index string putinterval adv}B /set{rw cp fillstr 0 4 index getinterval putinterval adv}B /fillstr 18 string 0 1 17{2 copy 255 put pop}for N /pl[{adv 1 chg} {adv 1 chg nd}{1 add chg}{1 add chg nd}{adv lsh}{adv lsh nd}{adv rsh}{ adv rsh nd}{1 add adv}{/rc X nd}{1 add set}{1 add clr}{adv 2 chg}{adv 2 chg nd}{pop nd}]dup{bind pop}forall N /D{/cc X dup type /stringtype ne{] }if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet %%BeginProcSet: special.pro TeXDict begin /SDict 200 dict N SDict begin /@SpecialDefaults{/hs 612 N /vs 792 N /ho 0 N /vo 0 N /hsc 1 N /vsc 1 N /ang 0 N /CLIP 0 N /rwiSeen false N /rhiSeen false N /letter{}N /note{}N /a4{}N /legal{}N}B /@scaleunit 100 N /@hscale{@scaleunit div /hsc X}B /@vscale{@scaleunit div /vsc X}B /@hsize{/hs X /CLIP 1 N}B /@vsize{/vs X /CLIP 1 N}B /@clip{ /CLIP 2 N}B /@hoffset{/ho X}B /@voffset{/vo X}B /@angle{/ang X}B /@rwi{ 10 div /rwi X /rwiSeen true N}B /@rhi{10 div /rhi X /rhiSeen true N}B /@llx{/llx X}B /@lly{/lly X}B /@urx{/urx X}B /@ury{/ury X}B /magscale true def end /@MacSetUp{userdict /md known{userdict /md get type /dicttype eq{userdict begin md length 10 add md maxlength ge{/md md dup length 20 add dict copy def}if end md begin /letter{}N /note{}N /legal{} N /od{txpose 1 0 mtx defaultmatrix dtransform S atan/pa X newpath clippath mark{transform{itransform moveto}}{transform{itransform lineto} }{6 -2 roll transform 6 -2 roll transform 6 -2 roll transform{ itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll curveto}}{{ closepath}}pathforall newpath counttomark array astore /gc xdf pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{PaintBlack}if}N /txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 -1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop 90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr 2 get ppr 0 get neg sub neg 0 S TR}if}ifelse scaleby96{ppr aload pop 4 -1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S TR}if}N /cp {pop pop showpage pm restore}N end}if}if}N /normalscale{Resolution 72 div VResolution 72 div neg scale magscale{DVImag dup scale}if 0 setgray} N /psfts{S 65781.76 div N}N /startTexFig{/psf$SavedState save N userdict maxlength dict begin /magscale false def normalscale currentpoint TR /psf$ury psfts /psf$urx psfts /psf$lly psfts /psf$llx psfts /psf$y psfts /psf$x psfts currentpoint /psf$cy X /psf$cx X /psf$sx psf$x psf$urx psf$llx sub div N /psf$sy psf$y psf$ury psf$lly sub div N psf$sx psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub TR /showpage{}N /erasepage{}N /copypage{}N /p 3 def @MacSetUp}N /doclip{ psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto closepath clip newpath moveto}N /endTexFig{end psf$SavedState restore}N /@beginspecial{SDict begin /SpecialSave save N gsave normalscale currentpoint TR @SpecialDefaults count /ocount X /dcount countdictstack N}N /@setspecial {CLIP 1 eq{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs neg 0 rlineto closepath clip}if ho vo TR hsc vsc scale ang rotate rwiSeen{rwi urx llx sub div rhiSeen{rhi ury lly sub div}{dup}ifelse scale llx neg lly neg TR }{rhiSeen{rhi ury lly sub div dup scale llx neg lly neg TR}if}ifelse CLIP 2 eq{newpath llx lly moveto urx lly lineto urx ury lineto llx ury lineto closepath clip}if /showpage{}N /erasepage{}N /copypage{}N newpath }N /@endspecial{count ocount sub{pop}repeat countdictstack dcount sub{ end}repeat grestore SpecialSave restore end}N /@defspecial{SDict begin} N /@fedspecial{end}B /li{lineto}B /rl{rlineto}B /rc{rcurveto}B /np{ /SaveX currentpoint /SaveY X N 1 setlinecap newpath}N /st{stroke SaveX SaveY moveto}N /fil{fill SaveX SaveY moveto}N /ellipse{/endangle X /startangle X /yrad X /xrad X /savematrix matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix}N end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (temp.dvi) @start /Fa 4 104 df<12181238127812381208A21210A212201240A21280050C7D830D >44 D<1230127812F0126005047C830D>46 D101 D<13F3EA018FEA030FEA0607EA0E0E120C121CA2EA381CA413381230A2EA187813F0EA0F 701200A213E0A2EAC0C012E1EAC300127E101A7D9113>103 D E /Fb 20 117 df45 D<14E0A2497EA3497EA2EB06FCA2EB0EFEEB 0C7EA2497EA201307F141F01707FEB600FA2496C7E90B5FC4880EB8003000380EB0001A2 00066D7EA2000E803AFFE00FFFE0A2231F7E9E28>65 D<903807FC0290383FFF0E9038FE 03DE3903F000FE4848133E4848131E485A48C7120EA2481406127EA200FE1400A7127E15 06127F7E150C6C7E6C6C13186C6C13386C6C13703900FE01C090383FFF80903807FC001F 1F7D9E26>67 DII73 D77 DIII82 D<007FB512FCA2397C07E07C0070141C0060140CA200E0140E00C01406A400001400B100 03B512C0A21F1E7E9D24>84 DI101 D104 D108 D<38FE0FC0EB3FE0381E61F0EBC0F81380EA1F00AD38FFE7FFA218147D931D>110 D<48B4FC000713C0381F83F0383E00F8A248137CA200FC137EA6007C137CA26C13F8A238 1F83F03807FFC00001130017147F931A>I114 D<1203A45AA25AA2123FEAFFFCA2EA1F00AA1306 A5EA0F8CEA07F8EA03F00F1D7F9C14>116 D E /Fc 2 60 df<127012F8A3127005057C 840D>58 D<127012F012F8A212781208A31210A31220A21240050E7C840D>I E /Fd 3 81 df<12F0B3A8041A7D990B>73 D<00FCEB07E0A300EE130DA200EF131D00E7 1319A2EB803900E31331A2EBC071A238E1E0E1A200E013C1A213F1EB7181A2137BEB3B01 A2131EA31B1A7D9922>77 D80 D E /Fe 6 115 df<143014F0EB03C0EB07 00131C1378EA01E0EA0780000EC7FC123812F0A21238120E6C7EEA01E0EA0078131C1307 EB03C0EB00F0143014167D921B>60 D97 D<120313801300C7FCA6121C12241246A25A12 0C5AA31231A21232A2121C09177F960C>105 D<121F1206A45AA4EA181C1366138EEA19 0CEA3200123C123FEA3180EA60C013C4A3EAC0C813700F177E9612>107 D110 D114 D E /Ff 1 81 df80 D E /Fg 39 122 df<126012F0A212701210A31220 A212401280040B7D830B>44 D<126012F0A2126004047D830B>46 D<130CA3131EA2133F1327A2EB4380A3EB81C0A348C67EA213FF38020070A20006137800 041338A2487FA2001C131EB4EBFFC01A1A7F991D>65 D67 DI70 D<38FFE7FF380E0070AB380FFFF0380E0070AC38FFE7FF181A7E991D>72 D<39FFE07F80390E001E00141814105C5C5C49C7FC13025B5B131C132E134E1387380F03 80120E6D7E6D7EA21470A28080143E39FFE0FF80191A7E991E>75 DII<38FE01FF380F00381410EA0B80A2EA09C0EA08E0A213 70A21338131CA2130EA21307EB0390A2EB01D0A2EB00F01470A21430121C38FF8010181A 7E991D>I80 D83 D<38FFE1FF380E00381410B20006132012076C1340EA01803800C180EB3E 00181A7E991D>85 D<39FF801FC0391E000F00001C13066C1304A26C5BA36C6C5AA2EBC0 3000011320A26C6C5AA3EB7080A20139C7FCA3131EA3130CA21A1A7F991D>I97 D<12FC121CA913F8EA1F0EEA1E07381C0380130114C0A6EB03801400EA1E07EA 1B0CEA10F0121A7F9915>II<137E130EA9EA03CEEA0C3EEA380E1230127012E0A61260 1270EA381EEA1C2E3807CFC0121A7F9915>IIII<12FC121CA913F8EA1D0CEA1E0EA2121CAB38FF9FC0121A7F99 15>I<1218123CA21218C7FCA612FC121CAEEAFF80091A80990A>II<12FC121CA9EB3F80EB 1E00131813105B5BEA1DC0EA1FE0121C1370137813387F131E131F38FF3FC0121A7F9914 >I<12FC121CB3A6EAFF80091A80990A>I<38FC7C1F391D8E6380391E0781C0A2001C1301 AB39FF9FE7F81D107F8F20>II< EA07E0EA1C38EA300CEA700EEA6006EAE007A6EA6006EA700EEA381CEA1C38EA07E01010 7F8F13>II 114 DI<1204A3120CA2121C123CEAFFC0EA1C00A81320A5EA 0E40EA03800B177F960F>II<38FF 1F80383C0600EA1C04A2EA1E0CEA0E08A26C5AA21390EA03A0A2EA01C0A36C5A11107F8F 14>I<39FF3F9F80393C0E070000381306381C16041317001E130C380E23081488000F13 983807419014D03803C1E01380A200015BEB004019107F8F1C>I<38FF3F80383C1C00EA 1C18EA0E106C5A13606C5A12017F1203EA0270487E1208EA181CEA381E38FC3FC012107F 8F14>I<38FF1F80383C0600EA1C04A2EA1E0CEA0E08A26C5AA21390EA03A0A2EA01C0A3 6C5AA248C7FCA212E112E212E4127811177F8F14>I E /Fh 7 62 df48 D<1206120E12FE120EB1EAFFE00B157D9412>III<13 30A2137013F012011370120212041208121812101220124012C0EAFFFEEA0070A5EA03FE 0F157F9412>II61 D E /Fi 10 58 df<121FEA3180EA60C0EA4040EAC060A8EA4040EA60C0 EA3180EA1F000B107F8F0F>48 D<120C123C12CC120CACEAFF8009107E8F0F>I<121FEA 6180EA40C0EA806012C01200A213C0EA0180EA030012065AEA10201220EA7FC012FF0B10 7F8F0F>I<121FEA2180EA60C0A2120013801201EA0F00EA00801340136012C0A2EA8040 EA6080EA1F000B107F8F0F>I<1203A25A5A120B121312331223124312C3EAFFE0EA0300 A4EA1FE00B107F8F0F>III<1240EA7FE013C0EA8080A2EA 010012025AA2120C1208A21218A50B117E900F>I<121FEA3180EA60C0A3EA7180EA3F00 120FEA3380EA61C0EAC060A3EA4040EA6080EA1F000B107F8F0F>I<121FEA3180EA60C0 EAC0401360A3EA40E01221EA1E6012001340EA60C01380EA4300123E0B107F8F0F>I E /Fj 67 126 df34 D37 DI<1238127CA2127E123E120EA3121CA2123812F812F012 C0070E789B18>I<137013F0EA01E0EA03C0EA0780EA0F00121E121C5AA25AA45AA81270 A47EA27E121E7EEA0780EA03C0EA01F0120013700C24799F18>I<126012F012787E7E7E EA07801203EA01C0A2EA00E0A41370A813E0A4EA01C0A2EA03801207EA0F00121E5A5A5A 12600C247C9F18>II<121C123E127E127F123F 121F1207120E121E127C12F81260080C788518>44 D<387FFFC0B512E0A26C13C013047E 8F18>I<1230127812FCA2127812300606778518>I<1303EB0780A2130F14005B131EA213 3E133C137C1378A213F85B12015B12035BA212075B120F90C7FCA25A121E123E123CA212 7C127812F85AA2126011247D9F18>IIII<131F5B1377A213E7120113C7EA038712071307120E121E 123C1238127812F0B512F8A338000700A6EB7FF0A3151C7F9B18>52 D<383FFF80A30038C7FCA8EA3BF8EA3FFE7F383C0780383003C0EA0001EB00E0A2126012 F0A238E001C0EA7003387C0F80383FFF00EA1FFCEA03F0131C7E9B18>I<1230127812FC A2127812301200A81230127812FCA2127812300614779318>58 D<1218123C127EA2123C 12181200A81218123C127EA2123E121E120E121C123C127812F01260071A789318>I<38 7FFFC0B512E0A3C8FCA4B512E0A36C13C0130C7E9318>61 D<137013F8A213D8A2EA01DC A3138CEA038EA4EA0707A5380FFF80A3EA0E03381C01C0A3387F07F000FF13F8007F13F0 151C7F9B18>65 DIIIII<3801F1C0EA03FDEA0FFFEA1F0FEA1C0312 3813011270A290C7FC5AA5EB0FF0131F130F387001C0A213031238A2EA1C07EA1F0FEA0F FFEA03FDEA01F1141C7E9B18>I73 D76 D<38FC01F8EAFE03A2383B06E0A4138EA2EA398CA213DCA3EA38D8A213F81370A21300A6 38FE03F8A3151C7F9B18>I<387E07F038FF0FF8387F07F0381D81C0A313C1121CA213E1 A313611371A213311339A31319A2131D130DA3EA7F07EAFF87EA7F03151C7F9B18>III82 D<3803F1C0EA1FFF5AEA7C0FEA7003EAE001A390C7 FC12701278123FEA1FF0EA07FEC67EEB0F80EB03C01301EB00E0A2126012E0130100F013 C038F80780B5FCEBFE00EAE7F8131C7E9B18>I<387FFFF8B5FCA238E07038A400001300 B2EA07FFA3151C7F9B18>I<38FF83FEA3381C0070B36C13E0EA0F01380783C03803FF80 6C1300EA007C171C809B18>I<38FE03F8A338700070A36C13E0A513F8EA39FC13DCA200 1913C0A3138CA2EA1D8DA31305000D1380EA0F07A2EA0E03151C7F9B18>87 D<38FE03F8EAFF07EAFE03381C01C0EA1E03000E1380EA0F0700071300A2EA038EA2EA01 DCA3EA00F8A21370A9EA01FC487E6C5A151C7F9B18>89 D<387FFFC0B512E0A26C13C013 047E7F18>95 D97 D<127E12FE127E120EA5133EEBFF80000F13C0EBC1E01380EB0070120E1438A6000F1370 A2EB80E013C1EBFFC0000E138038063E00151C809B18>IIIII<3801E1F03807FFF85A381E1E3038 1C0E00487EA5EA1C0EEA1E1EEA1FFC5BEA39E00038C7FC7EEA1FFEEBFFC04813E0387801 F038700070481338A4007813F0EA7E03381FFFC06C13803801FC00151F7F9318>I<127E 12FE127E120EA5133EEBFF80000F13C013C1EB80E01300120EAB387FC7FC38FFE7FE387F C7FC171C809B18>II<1338137CA313381300A4EA0FFCA3EA001CB3A4EA6038EAF078EAFFF0EA7F E0EA3F800E277E9C18>I<127E12FE127E120EA5EB3FF0A3EB0780EB0F00131E5B5B5BEA 0FF87F139C130EEA0E0F7FEB038014C0387FC7F812FF127F151C7F9B18>II<38F9C1C038FFF7F013FF383E3E38EA3C3CA2EA3838AB 38FE3E3EEB7E7EEB3E3E1714809318>IIII<38 FF0FC0EB3FE0EB7FF0EA07F0EBE060EBC0005BA290C7FCA9EAFFFC7F5B14147E9318> 114 DI<487E1203A4387FFFC0B5FC A238038000A9144014E0A33801C1C013FF6C1380EB3E0013197F9818>I<387E07E0EAFE 0FEA7E07EA0E00AC1301EA0F033807FFFC6C13FE3801FCFC1714809318>I<387F8FF000 FF13F8007F13F0381C01C0380E0380A338070700A3138FEA038EA3EA01DCA3EA00F8A213 7015147F9318>I<38FF07F8138F1307383800E0A4381C01C0137113F9A213D9EA1DDD00 0D1380A3138DEA0F8FA23807070015147F9318>I<387F8FF0139F138F380F0700EA078E EA039EEA01DC13F81200137013F07FEA01DCEA039E138EEA0707000E1380387F8FF000FF 13F8007F13F015147F9318>I<387F8FF000FF13F8007F13F0380E01C0EB0380A21207EB 0700A2EA0387A2138EEA01CEA213CC120013DC1378A31370A313F05B1279EA7BC0EA7F80 6CC7FC121E151E7F9318>I<383FFFF05AA2387001E0EB03C0EB078038000F00131E5B13 F8485AEA03C0485A380F0070121E5A5AB512F0A314147F9318>II< 126012F0B3B012600424769F18>I<127CB4FC13C01203C67EAB7FEB7FC0EB3FE0A2EB7F C0EBF0005BABEA03C012FF90C7FC127C13247E9F18>I E /Fk 36 121 df45 D<12F0A404047B830E>I<13C01201120712FFA212FB 1203B3A7B5FCA310227CA118>49 DII<131FA2 132FA2136F13EF13CF1201A2EA038FA21207130F120F120E121E123CA21278A212F0B512 F8A338000F00A915217FA018>I<137E48B4FC5AEA07C148C7FC121EA25AA25AA213F8EA F1FEEAF7FF38FF0F80EAFC0738F803C0A2130100F013E0A51270A21278EB03C01238003C 1380EA1C07381F0F00EA0FFE6C5AEA01F013237EA118>54 D66 DI69 D<12F0B3B104237CA20D>73 D<00FC147E6C14FEA300F7EB01DEA3EB800300F3149EA2EB C007A200F1141EA2EBE00F00F0130EA2EBF01EA2EB701CEB783CA2EB3838EB3C78A2EB1C 70A2EB1EF0EB0EE0A2130FEB07C0A390C7FC1F237BA22A>77 D79 DI82 D97 D<12F0ADEAF1F0EAF7FCB47EEA FC1F487E38F00780A2EB03C0A8EB0780A238F80F00EAFC3EEAFFFEEAF7F8EAF1F012237C A219>IIII<133F13FF5AEA03C013801207A8EAFF F8A3EA0780B3102380A20F>I<3801F0783807FFF85A381F1F00EA1E0F383C0780A5381E 0F00EA1F1FEA0FFE485AEA19F00038C7FCA2123CEA1FFEEBFFC014E04813F0EA7801EB00 F8481378A4007813F0383E03E0381FFFC06C13803801FC0015217F9518>I<12F0A41200 A912F0B3A404237DA20B>105 D<12F0ADEB1F80EB3F00133E5B5BEAF1F0EAF3E0EAF7C0 12FFA27FA2EAFDF0EAF8F8EAF078137C7F131E131F7F1480EB07C012237CA218>107 D<12F0B3B104237DA20B>I<39F0F807C039F3FE1FF039F7FF3FF839FE0F707800FCEBE0 7C39F807C03CA200F01380AF1E167C9527>II II114 DI< 120FA6EAFFF8A3EA0F00AE1308131CEA07FC13F8EA03E00E1C7F9B12>II<38F001E0A2387803C0A3383C0780A3001E13 005BA2EA0F0E131EA2EA071C139C13BCEA03B8A213F86C5AA213167F9516>I<39F00780 3CEB0FC0A2D8780D1378131D14E0131CD83C1813F0133814F0001C14E0001E13711330EB 7079000E14C01439380F603B00071480141BEBC01FA2000314001E167F9521>I<387801 F0007C13E0383E03C0EA1E07000F138038078F0013DEEA03FC12016C5A137013F8487EEA 03DC139EEA078F380F0780121EEB03C0383C01E0387800F000F813F81516809516>I E /Fl 25 89 df<1306130C13101330136013C013801201EA0300A21206120E120CA25A A21238A21230A21270A21260A312E0A81260A57EA2121012181208120C12047E0F2E7AA1 12>40 D<138013C0134013201330A21318A3130CAD131CA31318A21338A21330A2137013 6013E013C0A2EA018013005A120612045A12185A12405A0E2E80A112>I<1230127812F8 1278127005057C840D>46 D<137F380183C0380201E0EA0400000713F05A1301A2000013 E0A2EB03C0EB07801400131CEA01F8EA000E7F148014C0130313071230127812F8A200F0 138038800F00EA400E6C5AEA1078EA0FE0141F7D9D17>51 D<1460A214E0EB01C0130313 05130D13091311EB2380134313C31383EA010312023804070012081218121012201240B5 12F038000E00A55B131E3801FFC0141E7E9D17>I<380180303803FFE014C0140013FC00 02C7FCA25AA4137CEA0587380A0380EA0C01000813C0120014E0A4387003C012F812F000 E01380EA800700401300130E6C5AEA1070EA0FC0141F7D9D17>II<1408140C141C143CA2147C147E149EA2EB011EA21302801304A21308A2011013 8014071320A2EB7FFF90384007C0EB8003A2EA0100A21202EC01E01206001F130339FF80 1FFE1F207F9F22>65 D67 D<0007B57E3900F801E0 903878007081497F151E150E150FA348481480A6484814005DA3151E153E4848133C5DA2 5D4A5A4A5A260F000FC7FC143CB512F0211F7E9E23>I<0007B512FC3900F8007C017813 1C150C5B1504A414043901E00800A31438EBFFF8EBE0383803C010A4EC00081510485AA2 1520A2156015C0380F00011407B612801E1F7E9E1F>I71 D<3A07FF83FFC03A00F8007C000178133CA2495BA648485BA4 90B5FCEBE0004848485AA64848485AA64848485A01807F39FFF07FF8221F7E9E22>I<38 07FF803800F8001378A25BA6485AA6485AA6485AA648C7FC7FEAFFF0111F7E9E10>I<38 07FFE0D800FCC7FC1378A25BA6485AA6485AA41580EC0100EA0780A25C14021406140E38 0F001E147CB512FC191F7E9E1C>76 DIII<0007B5FC3900F803C090387800F015785B157CA41578484813F815F0EC01 E0EC03C0EC0F00EBFFFCD803C0C7FCA6485AA648C8FC7FEAFFF81E1F7E9E1F>I<3807FF FE3900F8078090387801E0EC00F05B15F8A415F03801E00115E0EC03C0EC0780EC1E00EB FFF03803C03880141E140EA2140F48485AA51502D80F001304EB800F39FFF00788C7EA01 F01F207E9E21>82 DI<003FB512F0383C 078000301430126039400F0010A212C01280A3D8001E1300A65BA65BA65B7F383FFFE01C 1F7A9E21>I<39FFF00FF8391F0003E06CEB01801400001EEB0100A6481302A6485BA600 705BA25CA200785B1238001813C06C48C7FCEA0706EA01F81D20799E22>I<39FFF001FE 391F8000F86CC712601540A215807F0007EB0100A214021406EBC00400035BA25CA26C6C 5AA25CA26D5A120001F1C7FC13F2A213FC137C1378A21370A213201F207A9E22>I<3A03 FFC0FFC03A007F003E00013C1318013E1310011E5B011F5B6D5B0281C7FCEB078314C2EB 03C414E8EB01F0A2130080A2EB017CEB023CEB043EEB0C1EEB081F497E132001407FEB80 07000180EB0003000780391F8007F039FFC01FFE221F7F9E22>88 D E /Fm 17 119 df45 D50 D<12F0B3AB041D7C9C0C>73 D<00FCEB07E0A300EE130DA300E71319A3EB803900E31331EBC071A200E11361A2EBE0E1 A200E013C113F1EB7181A3EB3B01A3131EA313001B1D7C9C24>77 D<00FC1370A27E12EE12EF12E7A2138012E313C0A2EAE1E0A212E013F013701378A2133C A2131C131E130EA2130F130714F01303A2141D7C9C1D>I80 D<12E0ABEAE3E0EAEFF0EAFFF8EAF87CEAF01CEAE01E130EA6131C12F0EAF87CEAFFF8EA EFF0EAE3C00F1D7D9C15>98 DI<130EABEA0F8EEA1FEEEA3FFEEA7C3EEA70 0EA212E0A612F0EA701EEA7C3EEA3FFEEA1FEEEA0F8E0F1D7E9C15>II<13FC12011203EA0700120EA7EAFFE0A2EA0E00B00E1D809C0D>I<12F0A412 00A71270B2041D7E9C0A>105 D110 D114 DI 117 DI E /Fn 67 123 df11 D<137E3801C180EA0301 380703C0120EEB018090C7FCA5B512C0EA0E01B0387F87F8151D809C17>II<126012F0A71260AD12 00A5126012F0A21260041E7C9D0C>33 D<126012F012F812681208A31210A21220124012 80050C7C9C0C>39 D<13401380EA0100120212065AA25AA25AA212701260A312E0AC1260 A312701230A27EA27EA27E12027EEA008013400A2A7D9E10>I<7E12407E7E12187EA27E A27EA213801201A313C0AC1380A312031300A21206A25AA25A12105A5A5A0A2A7E9E10> I<126012F0A212701210A41220A212401280040C7C830C>44 D I<126012F0A2126004047C830C>I<130113031306A3130CA31318A31330A31360A213C0 A3EA0180A3EA0300A31206A25AA35AA35AA35AA35AA210297E9E15>I<12035A123F12C7 1207B3A4EA0F80EAFFF80D1C7C9B15>49 DII<130CA2131C133CA2135C13DC139CEA011C1203120212 04120C1208121012301220124012C0B512C038001C00A73801FFC0121C7F9B15>II<13F0EA030CEA0604EA0C0E EA181E1230130CEA7000A21260EAE3E0EAE430EAE818EAF00C130EEAE0061307A51260A2 EA7006EA300E130CEA1818EA0C30EA03E0101D7E9B15>I<1240387FFF801400A2EA4002 485AA25B485AA25B1360134013C0A212015BA21203A41207A66CC7FC111D7E9B15>I57 D<126012F0A212601200AA126012F0A2126004127C910C>I<126012F0A212601200AA12 6012F0A212701210A41220A212401280041A7C910C>I63 D<1306A3130FA3EB1780A3EB23C0A3EB41E0A3EB80F0A200017FEB0078EBFFF838 03007C0002133CA20006133E0004131EA2000C131F121E39FF80FFF01C1D7F9C1F>65 D<90381F8080EBE0613801801938070007000E13035A14015A00781300A2127000F01400 A8007014801278A212386CEB0100A26C13026C5B380180083800E030EB1FC0191E7E9C1E >67 DII< B512F8380F007814181408A2140C1404A213021400A3130613FE13061302A490C7FCA77F EAFFF8161C7E9B1B>I<90381F8080EBE0613801801938070007000E13035A14015A0078 1300A2127000F01400A6ECFFF0EC0F80007013071278A212387EA27E6C130B3801801138 00E06090381F80001C1E7E9C21>I73 D77 DIII82 D<3807E080EA1C19EA3005EA7003EA600112E01300A36C13007E127CEA7FC0EA3F F8EA1FFEEA07FFC61380130FEB07C0130313011280A300C01380A238E00300EAD002EACC 0CEA83F8121E7E9C17>I<007FB512C038700F010060130000401440A200C014201280A3 00001400B1497E3803FFFC1B1C7F9B1E>I<39FFF07FC0390F000E001404B3A26C5B1380 00035B12016C6C5AEB70C0011FC7FC1A1D7E9B1F>I<3AFFE0FFE0FF3A1F001F003C001E 011E13186C011F1310A3D807801420EC2780A2D803C01440EC43C0A213E00001903881E0 80A33A00F100F100A3017913FA017A137AA2013E137C013C133CA301181318A3281D7F9B 2B>87 D<397FF0FFC0390FC03E0038078018EA03C0EBE01000015BEBF06000001340EB78 80137D013DC7FC7F131F7F80A2EB13C0EB23E01321EB41F0EBC0F8EB80783801007C4813 3C00027F0006131F001FEB3F8039FFC0FFF01C1C7F9B1F>I<12FEA212C0B3B312FEA207 297C9E0C>91 D<12FEA21206B3B312FEA20729809E0C>93 D97 D<12FC121CAA137CEA1D86EA1E03381C018014C0130014E0A614C013011480381E 0300EA1906EA10F8131D7F9C17>II<133F1307AAEA03E7EA0C17EA180F 487E1270126012E0A61260127012306C5AEA0C373807C7E0131D7E9C17>II<13F8EA018CEA071E1206EA0E0C1300A6EAFFE0EA0E00B0EA7FE00F 1D809C0D>II<12FC121CAA137C1387EA1D03001E1380121CAD38FF9F F0141D7F9C17>I<1218123CA21218C7FCA712FC121CB0EAFF80091D7F9C0C>I<13C0EA01 E0A2EA00C01300A7EA0FE01200B3A21260EAF0C012F1EA6180EA3E000B25839C0D>I<12 FC121CAAEB3FC0EB0F00130C13085B5B5B13E0121DEA1E70EA1C781338133C131C7F130F 148038FF9FE0131D7F9C16>I<12FC121CB3A9EAFF80091D7F9C0C>I<39FC7E07E0391C83 8838391D019018001EEBE01C001C13C0AD3AFF8FF8FF8021127F9124>IIIIIII<1204A412 0CA2121C123CEAFFE0EA1C00A91310A5120CEA0E20EA03C00C1A7F9910>I<38FC1F80EA 1C03AD1307120CEA0E1B3803E3F014127F9117>I<38FF07E0383C0380381C0100A2EA0E 02A26C5AA3EA0388A213D8EA01D0A2EA00E0A3134013127F9116>I<39FF3FCFE0393C0F 0380381C07011500130B000E1382A21311000713C4A213203803A0E8A2EBC06800011370 A2EB8030000013201B127F911E>I<387F8FF0380F03801400EA0702EA0384EA01C813D8 EA00F01370137813F8139CEA010E1202EA060738040380381E07C038FF0FF81512809116 >I<38FF07E0383C0380381C0100A2EA0E02A26C5AA3EA0388A213D8EA01D0A2EA00E0A3 1340A25BA212F000F1C7FC12F31266123C131A7F9116>II E /Fo 9 118 df<127812FCA412781200A6127812FCA4127806127D910D>58 D63 D68 D99 D<121E123FA4121EC7FCA6B4FCA2121FAEEAFFE0A20B 1E7F9D0E>105 D<38FF0FC0EB31E0381F40F0EB80F8A21300AB38FFE7FFA218127F911B> 110 DI 115 D<38FF07F8A2EA1F00AC1301120F380786FFEA01F818127F911B>117 D E /Fp 11 118 df45 D<127012F8A212F012E00505 7B840E>I98 D<1478EB03F8EB0070A414E0A4EB01C0A213F1EA038938070780EA0E03121C123C383807 001278A3EAF00EA31420EB1C40A2EA703C135C38308C80380F070015207C9F17>100 D<137CEA01C2EA0701120E121C123CEA3802EA780CEA7BF0EA7C0012F0A4127013011302 EA3804EA1838EA07C010147C9315>I103 D<381C0F80382630C0384740601380EB0070A2008E 13E0120EA3381C01C0A3EB03840038138814081307EB031000701330383001C016147C93 1A>110 D<137CEA01C338030180000E13C0121E001C13E0123C1278A338F003C0A3EB07 801400EA700F130EEA3018EA1870EA07C013147C9317>I114 D116 D<000E13C0003313E0382301C0EA438113011247388703 80120EA3381C0700A31410EB0E201218A2EA1C1E380C26403807C38014147C9318>I E /Fq 4 104 df<127012F8A3127005057C8D0D>1 D15 D<130F1338136013E0EA01 C0AFEA0380EA0700121E12F8121E1207EA0380EA01C0AFEA00E013601338130F102D7DA1 17>102 D<12F8121E1207EA0380EA01C0AFEA00E013601338130F1338136013E0EA01C0 AFEA0380EA0700121E12F8102D7DA117>I E /Fr 55 123 df<90387F07F03801FF1F48 133F3907807800EB00704813F0A739FFF8FF80A3390F00F000B11C20809F1B>11 D13 D<137013E0EA01C0EA038012071300120E121EA25AA35AA4127012F0AC12701278A47EA3 7EA2120E7E13801203EA01C0EA00E013700C2E7EA112>40 D<12E012707E7E121E120E7E 1380A2EA03C0A3EA01E0A4120013F0AC13E01201A4EA03C0A3EA0780A21300120E121E12 1C5A5A5A0C2E7DA112>I<1306AFB612F0A2D80006C7FCAF1C207D9A23>43 D<1278A412181230A21260A212E0050A7D830D>II50 D<12F0A41200AC12F0A4 04147C930D>58 D<131FA2497E133BA2EB7BC013731371EBF1E013E113E000017FA213C0 00031378A2138000077FA21300380FFFFEA3381E000FA24814801407A24814C01403A248 14E01B207F9F1E>65 DIIIIII<00F01378AEB512F8A338F00078AF15207B9F20> I<12F0B3AE04207C9F0D>I<12F0B3ABB51280A311207B9F19>76 D<00F8EB01F86C1303A200F4147800F61307A300F3130EA3EB801EA200F1131CEBC03CA2 00F01338A2EBE078A2EB6070EB70F0A2EB30E01339A2EB19C0A3EB0F80A390C7FC1D207B 9F28>I<00FC13787EA212F612F7A2EAF380A313C012F113E0A212F013F01370A2137813 38133CA2131C131E130EA31307A2130314F8A2130115207B9F20>I<133FEBFFC0000313 F03807E1F8380F807C48487E001E131E487FA248EB0780A348EB03C0A96C130700781480 A36CEB0F00A26C131E001F133E6C6C5A3807E1F86CB45AC613C0013FC7FC1A227DA021> II82 DII<00 F013F0B3A7387801E0A2383C03C0A2381F0F80380FFF006C5AEA01F814217B9F1F>I<00 F0EB01E0A20078EB03C0A36CEB0780A36CEB0F00A2121F6C131EA23807801C143CA23803 C0381478A23801E07014F0A26C6C5A13F1A2EB71C0137BEB3B80A2133F6DC7FCA21B207F 9F1E>I<00F814F8007CEB01F0003C14E0003E13036C14C06CEB0780EB800FD807C01300 00035BEBE01E6C6C5A0000137CEB78785C133CEB1CE0131F6D5A6D5AAE1D20809F1E>89 D<12FFA312F0B3B3A312FFA3082D7DA10D>91 D<12FFA3120FB3B3A312FFA3082D7FA10D >93 D97 D<12F0ACEAF1F0EAF7FCB47EEAFC3EEA F80F12F0EB0780A7EB0F00A26C5AEAFC3EEAFFFCEAF7F8EAF1E011207D9F17>III I<137EEA01FE1203EA078013005AA7EAFFF0A3EA0F00B10F20809F0E>I<3803E0F0EA0F FF5A383E3E00EA3C1E487EA5EA3C1EEA3E3EEA1FFC485AEA33E00030C7FC1238EA3FFEEB FF806C13C04813E0387803F0EAF000A3EAF801387E07E0383FFFC0000F1300EA03FC141E 7F9317>I<12F0ACEAF1F8EAF3FCEAF7FEEAFC1FEAF80FA212F0AE10207D9F17>I<12F0A4 1200A812F0B3A204207D9F0B>II<12F0AC131F131E5B5B5BEAF1E0EAF3C0EAF780B47EA27FEAF9F012F848 7E137CA27F131E131FEB0F8011207D9F16>I<12F0B3AE04207D9F0B>I<39F0FC07E039F3 FE1FF039F7FF3FF839FE0FF07C39F807C03CA200F01380AE1E147D9325>IIII114 DI<121EA6EAFFF0A3EA 1E00AD1320EA1FF0120FEA07C00C1A7F9910>II<38F003C0A2007813801307A2383C0F00A3EA1E0E131EA2EA0F1C133CA2 EA0738A213B8EA03F0A26C5A12147F9315>I<387801E0387C03C0383E0780381E0F0012 0FEA079EEA03FC5B12016C5A12017F487EEA079EEA0F0F120E381E0780383C03C0387801 E000F813F01414809315>120 D<38F003C0A238780780A2127C383C0F00A2121E131E12 0EEA0F1CA2EA073C1338EA03B8A213B0EA01F0A26C5AA2485AA3485A1207007FC7FC127E 127C121D7F9315>II E /Fs 35 121 df<131E133C137813F0120113 E0EA03C012071380120F13005AA2121E123EA35AA5127812F8AF1278127CA57EA3121E12 1FA27E1380120713C01203EA01E013F012001378133C131E0F3D7CAC17>40 D<12F012787E7E121F7EEA078013C0120313E0120113F0A2120013F8A3137CA5133C133E AF133C137CA513F8A313F01201A213E0120313C012071380EA0F005A121E5A5A5A0F3D7D AC17>I45 D<12F8A505057A8411>I<1318133813F8120712FFA2 12F81200B3AC387FFFF0A314287BA71E>49 D<13FE3803FFC04813E04813F0381F03F838 3C00FC0038137E0078133E127000F0131FA212601220C7FCA3143EA2147CA214F8EB01F0 14E0EB03C0EB0780EB0F00131E5B5B5B485A485A485A48C7FC121E5A007FB5FCA418287E A71E>I<137F3801FFC0000713F0487FEA1FC1383E007C487F1238A21210C7FCA2143C14 7C14FC495AEB07F0EBFFE05C8080EB01F8EB007C8080A2EC0F80A57E6CEB1F007E6C5B00 7C137E383F81FC6CB45A6C5B00035BC66CC7FC19297EA71E>II<383FFFFCA4003EC7FCAA133FEBFFC0003F13E0 14F0EBC1F8EB807CEB007E003E133EC7FC141FA712200060133E127000F8137C007C13FC 383F03F8381FFFF06C13E06C13803800FE0018287EA61E>II66 D<49B4FC010F13E0013F13F85B3801FE013903F8003001E013 1048481300485A121F90C8FC123EA2127E127CA35AAA127CA3127E123EA27E7F120F6C7E 6C6C130401F8131CD801FE13FC38007FFF6D13F8010F13E0010113001E2C7CAA26>I69 D<12F8B3B3A6052A7AA911>73 D78 DI84 D97 D99 D<143EAF13FCEA03FF4813 FE5AEA1FC1383F007E003E133E5AA212FC5AA77E127CA26C137E003F13FEEA1FC1EA0FFF 6C13BE6C133EEA00FC172A7EA91F>I<137E3803FF804813C04813E0381F83F0EA3F0000 3E13785A14380078133CB512FCA400F8C7FCA31278127CA27E003F130C381FC07C380FFF FC7E000113F038007F80161B7E9A1B>I<9038FC07803903FF3FC04813FF5A391F87E000 383E01F0A2486C7EA66C485AA2381F87E0EBFFC0485B003B90C7FCEA38FC0078C8FCA27E 383FFFC014F86C7F80487F387C007F48EB1F80140FA36C131F007EEB3F00383F80FE13FF 000F13F86C5BC613801A287E9A1E>103 D<12F8A51200AA12F8B3A9052A7CA90E>105 D<12F8AF14FCEB01F8EB03F0EB07E0EB0FC0EB1F80EB3F00137E137C5BEAF9F0EAFBF8B4 7EA2137E487E12FCEB1F80EAF80F14C0EB07E0A2EB03F0EB01F8A2EB00FC147E172A7CA9 1D>107 D<12F8B3B3A6052A7CA90E>I<3AF83F801FC09039FFE07FF000F99038F0FFF800 FB01F913FC39FF81FBC03AFE00FF007E6E133E48137EA248137CB2271B7C9A30>I<38F8 3F80EBFFE000F913F000FB13F8EAFF8138FE00FC147C5AA25AB2161B7C9A1F>I<137F38 01FFC0000713F0487F381FC1FC383F007E003E133E487FA200787F00F81480A7007CEB1F 00A2007E5B003E133E003F137E381FC1FC6CB45A6C5B000113C06C6CC7FC191B7E9A1E> I I114 DII<00F8137CB3A214FC1301EAFC07B5FC6C137CEA3FFCEA1FE0161B7C9A1F>I<00 F8EB0F80A2007CEB1F00A2127E003E133EA26C133C147CA2380F807814F8A26C6C5A13C1 A23803E1E013E300015BA213F33800F780A3017FC7FCA3191B7F9A1C>I<007CEB0FC06C EB1F80003F1400381F803E000F5B3807C0FC6C6C5A3801F1F0EBFBE06CB45A137F6D5A6D C7FCA2497E497E13FBEBF3E03801F1F03803E0F83807C07CEA0F808048487E003EEB0F80 007E14C048EB07E01B1B809A1C>120 D E /Ft 22 122 df66 DI69 D<923803FFF0037FEBFF800203B612F0020F15FC 023F9038807FFF9127FFF8000713C0010301E0010113F049496D7F4990C8EA3FFC49486F 7E49486F7E017F8449486F7F48496F7F48854A8148854A814885A24849707EA24885A348 90CA6C1380A5B519C0AE6C1A80A26E5EA26C1A00A36C6D4C5AA36C6D4B5BA26C616C6D4B 5B6E5D6C616C6D4B5B6D6C4B5B6E5D6D6D4A90C7FC010F6DECFFFC6D01F001035B6D01FC 010F5B01009026FF807F13C06E90B65A020F03FCC8FC020315F0DA007F1480030301F0C9 FC4A4979C759>79 D<903807FFFC017FEBFFC048B612F84815FE3A07FE001FFF6D010313 C0486D6C7F6F7F167F83A26C90C76C7EA26C5AEA00F890C8FCA40207B5FC91B6FC130701 3FEBE03F9038FFFE00000313F04813C04890C7FC485A485A485AA2485AA5167F6C6C14FF ED01EF6C6C130301FF903807CFFF000F9026C03F8F13F86C90B5000713FC0001ECFC036C 6CEBF0010103903980007FF8362E7DAD3A>97 DI<91 381FFFC091B512F8010714FE011FECFF8090267FFC0013C09038FFE00148494813E04813 80481300485AA248486D13C0A2003F6E1380EE3E00484891C7FCA312FFAA127FA27F123F A2001F16F87F120F6DEC01F06C6D13036C6DEB07E06C6DEB0FC06C6DEB1F80D97FFEEBFF 00011FB55A010714F8010014E0DA1FFEC7FC2D2E7CAD35>III103 DI<13FC487E487E4813804813C04813E0A66C13C06C13806C13006C5A6C5A90C7 FCABEB7FC0B5FCA512037EB3B0B6FCA518497CC820>I108 D<903A7FC001FFC0B5010F13F8033F13FE92B6FC9126C1FC 077F9126C3E0037F00039038C7C0016CD9CF008002DE7F14DC02FC815C5CA35CB3A8B6D8 C07FEBFFE0A53B2E7CAD42>110 DI<90397FC00FFCB590387FFFC002C3B512F002CF14FC9139DFF01F FF9126FF80077F00039026FE00017F6C01F86D7F4A814A6E7E717EA284171F84A3831980 AA19005FA260A24D5AA24D5A806E4A5A6E495B6E495B6E495B9226E03FFEC7FC02EFB55A 02E314F002E01480DB1FF8C8FC92CAFCAFB612C0A539427CAD42>I<9039FF803F80B5EB FFF0028313FC02877F91388FC3FF15070003D99E0F13806C13BC14B814F814F06F1300A2 4A6C5AED00F893C7FCA25CB3A6B612E0A5292E7CAD31>114 DII118 D<007FB5D8801FB5FCA528007FF800 0190C7FC6E14FC6D6C495A6D6C495A6D5D6D6D485A6D6D485AEDE03F6D6D48C8FC6DEBF8 FE91387FF9FC6EB45A5E6E5B6E5B80806E7F82824A7F825C91380FEFFFDA1FCF7FDA3F87 7FDA7F037FECFE0149486C7F4A6D7E49488001076E7E49486D7E49487F49486D7F01FF82 B500F8013FEBFFC0A53A2E7EAD3F>120 D<007FB500C090B51280A5C601F0C7380FE000 6E5DA26D6C5D171F013F93C7FC6E5C011F153E6E147E6D157C6F13FC6D5DEDC0016D5DED E003A26D01F05B16076D5DEDF80F027F5CEDFC1F023F91C8FC6F5A021F133EEDFF7E6E13 7C16FCA26E5BA26E5BA26E5BA26E5BA26F5AA26FC9FCA3153E157E157C15FCD81F805B38 3FC001D87FE05B38FFF0035D14074A5A4A5A143F6C4848CAFCEB43FE383FFFF86C5B0007 13C0D801FECBFC39427EAD3F>I E /Fu 8 117 df54 D67 D97 D<903801FFC0010F13F8013F13FE9038FF80 FF3A01FE003F80D807FCEB1FC04848EB0FE016F048481307003F15F8A2485AED03FCA312 FF90B6FCA301E0C8FCA4127FA3123F6D143C121FA26C6C14786C6C14F86C6CEB01F06C6C EB07E06C9038C03FC0013FB51200010713FC010013E026267DA52D>101 D<13FFB5FCA412077EB0ED7FC0913801FFF802077F91380F03FE91381C01FF023014804A 7E02E014C05C5CA391C7FCB3A4B5D8FC3F13FFA4303C7CBB37>104 D<9039FF01FF80B5000F13F0023F13FC9138FE03FFDAF0001380000301C0EB7FC04AEB3F E091C713F0EE1FF817FC160F17FEA2160717FFA917FEA2160F17FCA2EE1FF8A2EE3FF06E 14E06EEB7FC09139F001FF80DAFC07130091383FFFFC020F13F0020190C7FC91C9FCADB5 12FCA430377DA537>112 D<9038FE03F000FFEB0FFE4A7E91383C3F809138707FC00007 903860FFE0000313E05C13FF9138807FC0A2ED3F80ED0E0091C8FCB3A3B512FEA423267D A529>114 D116 D E /Fv 81 125 df<90381F83E09038F06E303901C07878380380F89038 00F03048EB7000A7B612803907007000B2383FE3FF1D20809F1B>11 D<133FEBE0C0EA01C0380381E0EA0701A290C7FCA6B512E0EA0700B2383FC3FC1620809F 19>II<9038 1F81F89038F04F043901C07C06390380F80FEB00F05A0270C7FCA6B7FC3907007007B23A 3FE3FE3FE02320809F26>I34 D<127012F812FCA212741204A31208A21210A2122012 40060E7C9F0D>39 D<13401380EA01005A12061204120C5AA212381230A212701260A412 E0AC1260A412701230A212381218A27E120412067E7EEA008013400A2E7BA112>I<7E12 407E12307E1208120C7EA212077EA213801201A413C0AC1380A412031300A25A1206A25A 120812185A12205A5A0A2E7EA112>I<1303AFB612FCA2D80003C7FCAF1E207E9A23>43 D<127012F012F8A212781208A31210A31220A21240050E7C840D>II<127012F8A3127005057C840D>I<144014C0EB0180A3EB0300A31306A25BA35BA35B A25BA35BA3485AA348C7FCA21206A35AA35AA25AA35AA35AA2122D7EA117>II<13801203120F12F31203B3A6EA07C0EAFFFE0F1E7C9D 17>III<1306A2130EA2131E132EA2134E138EA2EA010E1202A212041208 A212101220A2124012C0B512F038000E00A7EBFFE0141E7F9D17>II<137CEA0182EA0701380E0380 EA0C0712183838030090C7FC12781270A2EAF1F0EAF21CEAF406EAF807EB0380A200F013 C0A51270A214801238EB07001218EA0C0E6C5AEA01F0121F7E9D17>I<1240387FFFE014 C0A23840008038800100A21302485AA25B5BA25BA21360A213E05B1201A41203A76C5A13 1F7E9D17>III<127012F8A312701200AA127012F8A3 127005147C930D>I<127012F8A312701200AA127012F012F8A212781208A31210A31220 A21240051D7C930D>I<007FB512F8B612FCC9FCA8B612FC6C14F81E0C7E9023>61 D63 D<5B497EA3497EA3EB09E0 A3EB10F0A3EB2078A3497EA2EBC03EEB801EA248B5FCEB000FA20002EB0780A348EB03C0 A2120C001E14E039FF801FFE1F207F9F22>65 DI<90380FE0109038381C309038E002703803C00139078000F048C712 70121E15305A1510127C127800F81400A91278007C1410123CA26C1420A27E6C6C13406C 6C13803900E00300EB380CEB0FF01C217E9F21>III< B61280380780071401A2140015C01540A4EC2000A3146014E013FF138014601420A391C7 FCA87FEAFFFE1A1F7F9E1E>I<90380FE0109038381C309038E002703803C00139078000 F048C71270121E15305A1510127C127800F81400A7EC3FFEEC01F000781300127C123CA2 7EA27E6C7E3903C001703900E002309038380C1090380FF0001F217E9F24>I<39FFF07F F8390F000780AD90B5FCEB0007AF39FFF07FF81D1F7E9E22>II<3807FFC038003E00131EB3A3122012F8A3EAF01CEA403CEA6038EA10 70EA0FC012207F9E17>I<39FFF007FC390F0003E0EC0180150014025C5C5C5C5C5C49C7 FC5B497E130FEB13C0EB21E01341EB80F0EB0078A28080A280EC0780A2EC03C015E015F0 39FFF01FFE1F1F7E9E23>IIII II 82 D<3803F040380C0CC0EA1803EA3001EA6000A212E01440A36C13007E127CEA7F80EA 3FF86CB4FC00071380C613C0EB1FE013031301EB00F014707EA46C136014E06C13C038F8 018038C60300EA81FC14217E9F19>I<007FB512E038780F010060EB006000401420A200 C0143000801410A400001400B3497E3803FFFC1C1F7E9E21>I<39FFF00FF8390F0003E0 EC0080B3A46CEB01001380120314026C6C5A6C6C5AEB3830EB0FC01D207E9E22>I<39FF F003FE391F8000F86CC7126015206C6C1340A36C6C1380A2EBE00100011400A23800F002 A213F8EB7804A26D5AA36D5AA2131F6D5AA2EB07C0A36D5AA36DC7FC1F207F9E22>I<3B FFF07FF81FF03B1F000FC007C06C903907800180170015C001805C00071502EC09E013C0 00035DEC19F01410D801E05CA2EC2078D800F05CA2EC403C01785CA2EC801E017C146001 3C144090383D000F133F6D5CA2011E1307010E91C7FCA2010C7F010413022C207F9E2F> I<397FF81FF8390FE007C03907C0030000031302EBE0063801F00400005BEBF818EB7810 6D5AEB3E60EB1E406D5AA213076D7E497E1305EB08F0EB18F8EB1078EB207CEB603EEB40 1EEB801F3901000F801407000214C000061303001FEB07E039FFC01FFE1F1F7F9E22>I< 12FFA212C0B3B3A512FFA2082D7CA10D>91 DI<12FFA21203B3B3A512FFA2082D80A1 0D>I<120812101220A21240A21280A312B812FCA2127C1238060E7D9F0D>96 DI<121C12FC121CAA137CEA1D87381E0180EB00 C0001C13E01470A21478A6147014F014E0001E13C0381A018038198700EA107C15207E9F 19>IIII<137CEA01C6EA030F1207EA0E061300A7EAFFF0EA0E00B2EA7FE0 1020809F0E>I<14E03803E330EA0E3CEA1C1C38380E00EA780FA5EA380E6C5AEA1E38EA 33E00020C7FCA21230A2EA3FFE381FFF8014C0383001E038600070481330A4006013606C 13C0381C03803803FC00141F7F9417>I<121C12FC121CAA137C1386EA1D03001E1380A2 121CAE38FF8FF014207E9F19>I<1238127CA31238C7FCA6121C12FC121CB1EAFF80091F 7F9E0C>I<13E0EA01F0A3EA00E01300A61370EA07F012001370B3A31260EAF06013C0EA 6180EA3F000C28829E0E>I<121C12FC121CAAEB1FE0EB0780EB060013045B5B5B136013 E0EA1DF0EA1E70EA1C38133C131C7F130F7F148014C038FF9FF014207E9F18>I<121C12 FC121CB3ABEAFF8009207F9F0C>I<391C3E03E039FCC30C30391D039038391E01E01CA2 001C13C0AE3AFF8FF8FF8021147E9326>IIII<3801F04038070CC0EA0E02EA1C03EA38011278127012F0A612 7012781238EA1C03EA0C05EA0709EA01F1EA0001A8EB0FF8151D7F9318>III<1202A31206A2120EA2123EEAFFF8EA0E00AB1304A5EA07081203EA01F00E1C 7F9B12>I<381C0380EAFC1FEA1C03AE1307120CEA061B3803E3F014147E9319>I<38FF83 F8383E00E0001C13C06C1380A338070100A21383EA0382A2EA01C4A213E4EA00E8A21370 A3132015147F9318>I<39FF9FE1FC393C078070391C030060EC8020000E1440A214C0D8 0704138014E0A239038861001471A23801D032143A143E3800E01CA2EB6018EB40081E14 7F9321>I<38FF87F8381E03C0380E0180EB0300EA0702EA0384EA01C813D8EA00F01370 137813F8139CEA010E1202EA060738040380000C13C0003C13E038FE07FC16147F9318> I<38FF83F8383E00E0001C13C06C1380A338070100A21383EA0382A2EA01C4A213E4EA00 E8A21370A31320A25BA3EAF080A200F1C7FC1262123C151D7F9318>II124 D E /Fw 47 122 df<127012F812FCA212741204A41208A21210A212201240060F7C840E>44 DI<127012F8A3127005057C840E>I48 D<13801203120F12F31203B3A9EA07C0EAFFFE0F217CA018 >III<1303A25BA25B13 17A21327136713471387120113071202120612041208A212101220A2124012C0B512F838 000700A7EB0F80EB7FF015217FA018>I<00101380381E0700EA1FFF5B13F8EA17E00010 C7FCA6EA11F8EA120CEA1C07381803801210380001C0A214E0A4127012F0A200E013C012 80EA4003148038200700EA1006EA0C1CEA03F013227EA018>I<137EEA01C13803008038 0601C0EA0C03121C381801800038C7FCA212781270A2EAF0F8EAF30CEAF4067F00F81380 EB01C012F014E0A51270A3003813C0A238180380001C1300EA0C06EA070CEA01F013227E A018>I<12401260387FFFE014C0A23840008038C0010012801302A2485A5BA25B5BA213 60134013C0A21201A25B1203A41207A76CC7FC13237DA118>III<497EA3497EA3EB05E0A2EB09F01308A2EB10 78A3497EA3497EA2EBC01F497EA248B51280EB0007A20002EB03C0A348EB01E0A348EB00 F0121C003EEB01F839FF800FFF20237EA225>65 D<903807E0109038381830EBE0063901 C0017039038000F048C7FC000E1470121E001C1430123CA2007C14101278A200F81400A8 12781510127C123CA2001C1420121E000E14407E6C6C13803901C001003800E002EB381C EB07E01C247DA223>67 D69 DI73 D77 D<39FF8007FF3907C000 F81570D805E01320EA04F0A21378137C133C7F131F7FEB0780A2EB03C0EB01E0A2EB00F0 14F81478143C143E141E140FA2EC07A0EC03E0A21401A21400000E1460121FD8FFE01320 20227EA125>I80 D82 D<3803F020380C0C60EA1802 383001E0EA70000060136012E0A21420A36C1300A21278127FEA3FF0EA1FFE6C7E000313 8038003FC0EB07E01301EB00F0A214707EA46C1360A26C13C07E38C8018038C60700EA81 FC14247DA21B>I<007FB512F839780780780060141800401408A300C0140C00801404A4 00001400B3A3497E3801FFFE1E227EA123>I97 D<120E12FE121E120EAB131FEB61C0EB8060380F0030000E1338143C141C141EA7 141C143C1438000F1370380C8060EB41C038083F0017237FA21B>II<14E0130F13011300ABEA01F8EA0704EA0C02EA1C01EA3800127812 7012F0A7127012781238EA1801EA0C0238070CF03801F0FE17237EA21B>II<133E13E33801C780EA0387130748C7FCA9 EAFFF80007C7FCB27FEA7FF0112380A20F>I<14703803F198380E1E18EA1C0E38380700 A200781380A400381300A2EA1C0EEA1E1CEA33F00020C7FCA212301238EA3FFE381FFFC0 6C13E0383000F0481330481318A400601330A2003813E0380E03803803FE0015217F9518 >I<120E12FE121E120EABEB1F80EB60C0EB80E0380F0070A2120EAF38FFE7FF18237FA2 1B>I<121C123EA3121CC7FCA8120E127E121E120EB1EAFFC00A227FA10E>I<13E0EA01F0 A3EA00E01300A81370EA07F012001370B3A51260EAF0E013C0EA6180EA3F000C2C83A10F >I<120E12FE121E120EABEB03FCEB01F014C01480EB02005B5B5B133813F8EA0F1CEA0E 1E130E7F1480EB03C0130114E0EB00F014F838FFE3FE17237FA21A>I<120E12FE121E12 0EB3ADEAFFE00B237FA20E>I<390E1FC07F3AFE60E183803A1E807201C03A0F003C00E0 A2000E1338AF3AFFE3FF8FFE27157F942A>I<380E1F8038FE60C0381E80E0380F0070A2 120EAF38FFE7FF18157F941B>III114 DI<1202A41206A3120E121E123EEAFFFCEA0E00AB1304A6EA07081203 EA01F00E1F7F9E13>I<000E137038FE07F0EA1E00000E1370AD14F0A238060170380382 783800FC7F18157F941B>I<38FF80FE381E00781430000E1320A26C1340A2EB80C00003 1380A23801C100A2EA00E2A31374A21338A3131017157F941A>I<39FF8FF87F393E01E0 3C001CEBC01814E0000E1410EB0260147000071420EB04301438D803841340EB8818141C D801C81380EBD00C140E3900F00F00497EA2EB6006EB400220157F9423>I<38FF80FE38 1E00781430000E1320A26C1340A2EB80C000031380A23801C100A2EA00E2A31374A21338 A31310A25BA35B12F05B12F10043C7FC123C171F7F941A>121 D E /Fx 20 118 df45 D68 D73 D 77 D80 D<90387F80203801FFE03907C07860380F001C001EEB06E048130300381301007813 001270156012F0A21520A37E1500127C127E7E13C0EA1FF86CB47E6C13F86C7FC613FF01 0F1380010013C0EC1FE01407EC03F01401140015F8A26C1478A57E15706C14F015E07E6C EB01C000ECEB038000C7EB070038C1F01E38807FFCEB0FF01D337CB125>83 D<13FE380303C0380C00E00010137080003C133C003E131C141EA21208C7FCA3EB0FFEEB FC1EEA03E0EA0F80EA1F00123E123C127C481404A3143EA21278007C135E6CEB8F08390F 0307F03903FC03E01E1F7D9E21>97 D99 DIII<15F090387F03083901C1C41C380380E8390700700848EB7800001E 7FA2003E133EA6001E133CA26C5B6C13706D5A3809C1C0D8087FC7FC0018C8FCA5121C7E 380FFFF86C13FF6C1480390E000FC00018EB01E048EB00F000701470481438A500701470 A26C14E06CEB01C00007EB07003801C01C38003FE01E2F7E9F21>I<1207EA0F80121FA2 120FEA0700C7FCABEA078012FFA2120F1207B3A6EA0FC0EAFFF8A20D307EAF12>105 D<260780FEEB1FC03BFF83078060F0903A8C03C180783B0F9001E2003CD807A013E4DA00 F47F01C013F8A2495BB3A2486C486C133F3CFFFC1FFF83FFF0A2341F7E9E38>109 D<380780FE39FF83078090388C03C0390F9001E0EA07A06E7E13C0A25BB3A2486C487E3A FFFC1FFF80A2211F7E9E25>II<380783E038FF8C18EB 907C120FEA07A0EBC0381400A35BB3487EEAFFFEA2161F7E9E19>114 D<3801FC10380E0330381800F048137048133012E01410A37E6C1300127EEA3FF06CB4FC 6C13C0000313E038003FF0EB01F813006C133CA2141C7EA27E14186C1338143000CC1360 38C301C03880FE00161F7E9E1A>I<1340A513C0A31201A212031207120F381FFFE0B5FC 3803C000B01410A80001132013E000001340EB78C0EB1F00142C7FAB19>II E /Fy 5 85 df<16C08215011503A21507A2150FA2151F153F153715 67A2EDC7F015C3EC0183140315031406A2140CA21418A214301460A202C07FA290380180 01A249B5FC5B90380600015BA25BA25B137013605B8248481300120300071401D81FC013 03D8FFF890383FFFE0A22B327BB135>65 D<010FB512FE707E903A007F000FE0027EEB03 F0707E707E177C4A80A2173F83495A1880A3495AA44948143FA4494815005FA34948147E 17FEA25F49C712015FA24C5A017E5D16074C5A5F494AC7FC5E167E5E4848EB01F04B5AED 0FC00003027FC8FCB612FC15E031317BB036>68 D<010FB612FCA29039007F0001027EEB 007C173CA217185CA4495AA44948EB4010EEC000A34948485AA21503150F49B5C7FCA2EC C00F8190381F8006A449485A150492C8FCA2137EA45BA4485AA31203B512F0A22E317BB0 2F>70 D<010FB512F016FE903A007F003F80027EEB0FC0EE03E017F0EE01F85CA4494813 03A317F04948130717E0EE0FC0A24948EB1F00163E16FCED07F049B512C093C7FC9138C0 0F80ED03C0D91F807F6F7EA282EB3F00A4017E1303A25EA2491307A21708170C48481518 A2030313301203B5D8C00113E0923800FFC0C9EA3F002E327BB034>82 D<0007B712F05A9039C00FE003D81E00EBC000001C16E01218003816600030494813E0A2 4816C0A24AC7FC5AA348017E1480C71500A35CA4495AA4495AA4495AA4495AA4495AA449 C9FCA313FF007F13FFA22C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 911 a Fy(D)26 b(R)g(A)f(F)h(T)225 1002 y Fx(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n (terface)621 1196 y Fw(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum) 772 1323 y(No)o(v)o(em)o(b)q(er)e(27,)k(1995)77 1381 y(This)g(w)o(ork)f(w)o(as)h(supp)q(orted)g(in)f(part)h(b)o(y)e(ARP)l(A) h(and)h(NSF)e(under)h(gran)o(t)h(ASC-9310330,)i(the)192 1439 y(National)d(Science)f(F)l(oundation)i(Science)e(and)i(T)l(ec)o (hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1497 y(Agreemen)o(t)e (No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h(Commission)e(of)j(the)f (Europ)q(ean)i(Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s (ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 49 a Fv(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 361 a Fu(Chapter)34 b(6)75 568 y Ft(Extended)40 b(Collecti)q(v)m(e)i(Op)s(erations)75 809 y Fs(6.1)59 b(Intro)r(duction)75 910 y Fv(Based)15 b(on)f(exp)q(erience)j(with)d Fr(MPI)p Fv(,)g(the)g(follo)o(wing)i(t)o(yp)q(es)e(of)g(additional)i (functionalit)o(y)f(are)g(prop)q(osed)75 967 y(for)g Fr(MPI-2)p Fv(:)143 1061 y Fq(\017)23 b Fv(Non-blo)q(c)o(king)16 b(collectiv)o(e)h(comm)o(unication)f(\(async)o(hronous\),)143 1154 y Fq(\017)23 b Fv(New)15 b(or)g(extended)h(collectiv)o(e)h(op)q (erations)e(\()p Fp(e.g.)p Fv(,)f(in-place,)j(m)o(ulticast,)e(exclusiv) o(e)i(scan,)e(etc\),)143 1248 y Fq(\017)23 b Fv(Extension)15 b(of)g Fr(MPI)g Fv(collectiv)o(e)i(op)q(erations)e(to)g(in)o(tercomm)o (unicators,)143 1342 y Fq(\017)23 b Fv(T)l(op)q(ology)15 b(extensions)h(to)e(in)o(tercomm)o(unicators)h(\(bi-partite)h(comm)o (unication\),)143 1436 y Fq(\017)23 b Fv(New)15 b(top)q(ology)g(tec)o (hnology)l(,)143 1530 y Fq(\017)23 b Fv(Collectiv)o(e)16 b(put/get)f(tec)o(hnology)g(\(to)f(b)q(e)i(cross-referenced)g(with)g (the)f(1-sided)h(c)o(hapter\).)75 1624 y(This)f(c)o(hapter)f(ma)o(y)g (also)g(b)q(e)h(the)g(logical)h(place)f(to)f(describ)q(e)i(other)e (collectiv)o(e)i(op)q(erations)f(that)e(w)o(ere)75 1680 y(implicitly)18 b(housed)e(in)g(other)f(c)o(hapter)g(in)h(the)f Fr(MPI)g Fv(standard,)f(including:)143 1774 y Fq(\017)23 b Fv(Constructors/destructors)12 b(for)j(in)o(tracomm)o(unicators)f (and)h(in)o(tercomm)o(unicators)g(\(including)189 1830 y(extensions)h(and)f(non)o(blo)q(c)o(king)h(forms\),)143 1924 y Fq(\017)23 b Fv(Constructors)14 b(for)g(top)q(ology-orien)o(ted) i(comm)o(unicators.)75 2018 y(F)l(urthermore,)22 b(a)f(general)h (discussion)i(of)d(the)g(prop)q(erties)i(of)e(non)o(blo)q(c)o(king)i (collectiv)o(e)g(op)q(erations)75 2074 y(\(b)q(oth)15 b(sync)o(hronizing)i(and)e(non-sync)o(hronizing\))i(b)q(elongs)f(in)g (this)g(c)o(hapter.)166 2207 y Fo(Discussion:)f Fn(Op)q(en)g(issues)g (for)f(this)g(c)o(hapter:)134 2282 y(1.)22 b(Can)15 b(threads)h(plus)f Fm(MPI)f Fn(substitute)j(for)e(non-blo)q(c)o(king)e(collectiv)o(e?)22 b(If)15 b(not)g(no)o(w,)g(when?)22 b(If)15 b(w)o(e)g(think)189 2331 y(this)g(will)e(b)q(e)j(b)o(y)f(1997)f(\(completion)g(time)g(for)g Fm(MPI-2)p Fn(\),)g(then)i(w)o(e)f(ough)o(t)g(to)g(w)o(ork)g(on)g (thread-orien)o(ted)189 2381 y(compatibilit)o(y)l(,)10 b(not)i(extra)i(calls.)j(If)12 b(w)o(e)i(don't)e(exp)q(ect)i(high)f(p)q (erformance)f(threads)i(b)o(y)f(1997,)e(then)j(this)189 2431 y(section)f(is)f(more)f(acceptable.)18 b(\(Of)13 b(course,)g(not)f(all)f(systems)i(will)d(ha)o(v)o(e)i(threads)i(in)d (an)o(y)h(foreseeable)h(fu-)189 2481 y(ture,)f(but)h(will)d(these)k (mac)o(hines)d(meaningfully)e(run)j Fm(MPI)p Fn(,)f(or)h Fm(MPI)g Fn(non)o(blo)q(c)o(king)f(op)q(erations?)17 b(P)o(ossibly)189 2531 y(y)o(es:)h(for)c(instance,)g(em)o(b)q(edded)g (and)g(realtime)e(systems.\))134 2605 y(2.)22 b(Do)17 b(w)o(e)g(w)o(an)o(t)g(p)q(ersisten)o(t)j(collectiv)o(e)d(comm)o (unication)d(requests?)31 b(There)19 b(is)e(strong)h(justi\014cation)f (for)189 2655 y(these)e(a)o(v)n(ailable)d(in)h(terms)h(of)f(p)q (erformance.)964 2828 y Fv(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fv(2)560 b Fl(CHAPTER)16 b(6.)34 b(EXTENDED)15 b(COLLECTIVE)h(OPERA)l(TIONS)134 49 y Fn(3.)22 b(Do)15 b(w)o(e)i(w)o(an)o(t)f(to)g(b)q(e)h(able)g(to)f(cancel)h(a)f(non-blo)q (c)o(king)f(collectiv)o(e)i(comm)o(uni)o(cation?)23 b(\(non-trivial)15 b(en-)189 99 y(dea)o(v)o(or!\))134 174 y(4.)22 b(Do)15 b(w)o(e)i(w)o(an)o(t)f(to)g(b)q(e)g(able)g(to)g(prob)q(e)h(for)f(a)g (collectiv)o(e)g(comm)o(unication?)22 b(This)16 b(w)o(ould)g(require)h (a)f(new)189 224 y(t)o(yp)q(e)e(of)f(prob)q(e)i(function.)134 298 y(5.)22 b(Should)13 b(there)i(b)q(e)f(a)f(minim)n(um)c(n)o(um)o(b)q (er)k(of)g(outstanding)g(collectiv)o(e)h(op)q(erations)g(supp)q(orted,) g(and)f(ho)o(w)189 348 y(should)g(the)i(standard)f(sp)q(eak)h(to)e (this)h(minim)n(um)c(vis)k(a)f(vis)h(qualit)o(y)e(of)i Fm(MPI)f Fn(implemen)o(tatio)o(n)e(issues?)134 423 y(6.)22 b(Comm)n(unicator)10 b(constructors)15 b(and)e(destructors)i(\(b)q(oth) e(in)o(tra)f(and)h(in)o(ter\))g(are)g(collectiv)o(e,)g(and)f(need)i(to) 189 473 y(b)q(e)g(revisited)h(in)e(this)h(setting)g(\(that)g(is,)g (non-blo)q(c)o(king)e(extensions)j(m)o(ust)e(b)q(e)i(prop)q(osed\).)134 547 y(7.)22 b(Other)14 b(non)o(blo)q(c)o(king)e(collectiv)o(e)h(op)q (erations)h(prop)q(osed)g(elsewhere)i(in)c Fm(MPI-2)h Fn(m)o(ust)f(b)q(e)i(made)e(syn)o(tacti-)189 597 y(cally)h(and)g(seman) o(tically)f(uniform)g(with)h(this)h(section.)75 848 y Fs(6.2)59 b(Non-Blo)r(cking)20 b(Collective)e(Communication)75 949 y Fv(This)45 b(section)f(prop)q(oses)h(extensions)f(to)g(all)h (collectiv)o(e)h(comm)o(unication)f(calls)g(\(except)75 1006 y Fr(M)p Fv(PI)p 165 1006 14 2 v 16 w(OP)p 247 1006 V 17 w(CREA)l(TE)17 b(and)f Fr(M)p Fv(PI)p 648 1006 V 17 w(OP)p 731 1006 V 16 w(FREE\))g(in)h(a)f(non-blo)q(c)o(king)i(form.) 23 b(F)l(or)16 b(eac)o(h)g(collectiv)o(e)i(com-)75 1062 y(m)o(unication)13 b(call)g(with)f(name)h Fr(M)p Fv(PI)p 697 1062 V 16 w(xxx)f(there)g(is)h(to)e(b)q(e)i(a)e(corresp)q(onding)i (non-blo)q(c)o(king)h(one,)f(named)75 1119 y Fr(M)p Fv(PI)p 165 1119 V 16 w(Ixxx.)25 b(Non-blo)q(c)o(king)19 b(collectiv)o(e)f (comm)o(unication)g(calls)g(ha)o(v)o(e)e(the)h(same)f(syn)o(tax)g(as)h (blo)q(c)o(king)75 1175 y(op)q(erations,)i(with)g(the)f(addition)i(of)e (a)g Fr(r)p Fv(equest)h(argumen)o(t)e(in)j(the)e(last)h(p)q(osition)g (of)f(the)h(argumen)o(t)75 1232 y(list.)75 1353 y Fk(6.2.1)49 b(Basic)17 b(p)o(rop)q(erties)e(of)h(non-blo)q(cking)j(collective)e(op) q(erations)75 1439 y Fv(All)f(non-blo)q(c)o(king)h(collectiv)o(e)g(op)q (erations)f(ha)o(v)o(e)f(the)g(follo)o(wing)h(prop)q(erties:)131 1533 y(1.)22 b(Non-blo)q(c)o(king)16 b(collectiv)o(e)h(comm)o (unication)f(calls)h(return)e(a)g(request)g(handle.)189 1608 y(A)34 b(non-blo)q(c)o(king)i(collectiv)o(e)g(comm)o(unication)f (op)q(eration)g(is)f(completed)i(via)e(calls)i(to)189 1665 y Fj(\(MPI)p 288 1665 15 2 v 16 w Fq(f)p Fj(WAIT|TEST)p Fq(gf)22 b Fj(|ANY|SOME|ALL)p Fq(g)p Fj(\))p Fv(,)13 b(as)h(for)h(a)g(non-blo)q(c)o(king)h(p)q(oin)o(t-to-p)q(oin)o(t)g (comm)o(u-)189 1721 y(nication,)g(using)f(as)g(argumen)o(t)g(the)g (request)g(handle)i(returned)e(b)o(y)g(the)h(collectiv)o(e)h(call.)131 1815 y(2.)22 b(Blo)q(c)o(king)15 b(and)f(non-blo)q(c)o(king)i (collectiv)o(e)g(comm)o(unication)f(calls)g(can)g(b)q(e)g(mixed)g(and)f (matc)o(hed)189 1871 y(\(just)g(as)h(with)h(the)f(p)q(oin)o(t-to-p)q (oin)o(t)h(calls\).)189 1947 y(That)10 b(is,)i(some)f(of)g(the)h (participan)o(ts)f(in)h(a)f(collectiv)o(e)i(comm)o(unication)f(ma)o(y)f (execute)h(a)f(blo)q(c)o(king)189 2003 y(call,)16 b(while)g(others)f (execute)h(a)f(non-blo)q(c)o(king)i(call.)189 2154 y Fo(Discussion)o(:)25 b Fn(The)19 b(sub)q(committee)f(is)g(almost)f(ev)o (enly)h(split)g(on)h(this;)h(argumen)o(ts)e(for)g(and)h(against)189 2204 y(p)q(erformance)c(impact)g(on)g(blo)q(c)o(king)g(collectiv)o(e)h (calls)f(will)g(b)q(e)h(levied)g(b)q(efore)h(the)f(feature)h(is)e(v)o (oted)h(on)189 2254 y(formally)l(.)189 2329 y(An)e(argumen)o(t)e(in)i (fa)o(v)o(or)f(of)g(this)h(functionalit)o(y)e(is)i(usabilit)o(y)m(.)131 2505 y Fv(3.)22 b(Collectiv)o(e)17 b(comm)o(unication)h(calls)f(m)o (ust)f(b)q(e)h(initiated)h(in)f(a)f(consisten)o(t)h(order,)f(follo)o (wing)h(the)189 2562 y(same)e(rules)g(as)g(for)g(initiation)i(of)e(blo) q(c)o(king)h(collectiv)o(e)h(comm)o(unication)f(calls.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fl(6.3.)34 b(EXTENDED)14 b(COLLECTIVE)j(OPERA)l (TIONS)g(\(INTRA)o(COMMUNICA)l(TORS\))144 b Fv(3)243 49 y Fq(\017)23 b Fv(All)15 b(non-blo)q(c)o(king)h(\(and)e(blo)q(c)o (king\))h(collectiv)o(e)g(op)q(erations)g(m)o(ust)e(b)q(e)i(called)g (in)g(the)f(same)289 106 y(order)h(within)i(a)e(comm)o(unicator.)21 b(Ordering)c(b)q(et)o(w)o(een)f(t)o(w)o(o)e(or)h(more)g(comm)o (unicators)g(is)289 162 y(not)f(required.)243 235 y Fq(\017)23 b Fv(There)d(ma)o(y)g(b)q(e)h(an)f(arbitrary)g(n)o(um)o(b)q(er)g(of)g (outstanding)h(non-blo)q(c)o(king)h(collectiv)o(e)g(re-)289 292 y(quests.)243 365 y Fq(\017)h Fv(Ho)o(w)14 b(man)o(y)h(progress)f (concurren)o(tly)i(is)g(implemen)o(tation)h(de\014ned)f(\(1)f(or)f (more\).)131 471 y(4.)22 b(Insofar)14 b(as)h(collectiv)o(e)i(calls)f (are)e(extended)i(to)f(in)o(tercomm)o(unicators)f(elsewhere)i(in)g (this)g(c)o(hap-)189 527 y(ter,)f(the)h(discussion)i(here)e(applicable) j(to)c(in)o(tracomm)o(unicators)g(shall)j(b)q(e)e(extended)h(to)f(in)o (ter-)189 584 y(comm)o(unicators.)75 704 y Fr(Rationale:)46 b Fv(Collectiv)o(e)20 b(comm)o(unication)h(calls)f(can)g(b)q(e)g(quite) g(exp)q(ensiv)o(e.)34 b(Most)18 b(of)h(the)g(calls)i(re-)75 760 y(quire)d(eac)o(h)e(pro)q(cess)h(to)f(blo)q(c)o(k)i(un)o(til)f(all) h(pro)q(cesses)f(ha)o(v)o(e)f(reac)o(hed)h(the)g(call,)h(resulting)g (in)f(idle)i(time)75 817 y(when)g(pro)q(cesses)g(are)g(not)f(tigh)o (tly)h(sync)o(hronized.)32 b(The)19 b(use)g(of)f(non-blo)q(c)o(king)j (collectiv)o(e)f(comm)o(u-)75 873 y(nication)d(can)g(help)g(alleviate)h (this)e(problem.)24 b(The)16 b(prop)q(osed)h(de\014nition)h(in)o(tro)q (duces)f(no)f(particular)75 930 y(implemen)o(tation)f(problems,)f (except)h(that)e(a)g(request)h(ob)s(ject)g(ma)o(y)f(no)o(w)g(b)q(e)i (asso)q(ciated)f(with)g(sev)o(eral)75 986 y(p)q(oin)o(t-to-p)q(oin)o(t) 21 b(comm)o(unications)g(\(since)g(the)f(completion)i(of)e(a)g (collectiv)o(e)i(comm)o(unication)f(ma)o(y)75 1043 y(require)f(sev)o (eral)g(p)q(oin)o(t-to-p)q(oin)o(t)g(comm)o(unications)g(to)f (complete\).)33 b(Ho)o(w)o(ev)o(er,)19 b(it)h(is)g(debatable)g(as)75 1099 y(to)f(whether)h(non-blo)q(c)o(king)h(collectiv)o(e)h(op)q (erations)e(ma)o(y)f(b)q(e)h(allo)o(w)o(ed)g(to)f(put)h(an)g(implemen)o (tation)75 1156 y(or)c(p)q(erformance)h(on)o(us)f(on)h Fr(MPI)f Fv(blo)q(c)o(king)i(collectiv)o(e)g(op)q(erations,)f(since)h Fr(MPI-2)e Fv(is)h(to)f(b)q(e)h(strictly)g(a)75 1212 y(sup)q(erset.)75 1355 y Fs(6.3)59 b(Extended)18 b(Collective)g(Op)r (erations)h(\(Intracommunicato)n(rs\))75 1458 y Fk(6.3.1)49 b(In-Place)17 b(Op)q(erations)75 1544 y Fv(Curren)o(tly)267 1528 y Fh(1)297 1544 y Fv(all)11 b(of)f(the)g(collectiv)o(e)i(comm)o (unication)f(routines,)g(except)g(for)e Fr(MPI)p 1449 1544 14 2 v 16 w(Ba)o(rrier)g Fv(and)h Fr(MPI)p 1769 1544 V 16 w(Bcast)p Fv(,)75 1601 y(tak)o(e)15 b(as)g(argumen)o(ts)f(an) h(input)i(bu\013er,)e Fr(sendbuf)p Fv(,)i(and)f(an)f(output)g (bu\013er,)g Fr(recvbuf)h Fv(.)k(This)c(p)q(ermits)g(a)75 1657 y(large)f(degree)g(of)g(\015exibilit)o(y)j(as)c(only)i(the)f(t)o (yp)q(e)g(signatures)g(and)g(not)g(the)g(t)o(yp)q(e)g(map)g(ha)o(v)o(e) g(to)f(matc)o(h.)75 1714 y(Ho)o(w)o(ev)o(er,)f(it)i(is)h(often)e(the)h (case)g(that)f(the)h(user)g(w)o(an)o(ts)e(to)h(use)h(a)g(single)h (bu\013er)e(and)h(ha)o(v)o(e)g(the)g(results)75 1770 y(of)e(the)h(collectiv)o(e)i(comm)o(unication)e(op)q(eration)g(o)o(v)o (erwrite)g(all)g(or)g(part)f(of)g(this)h(bu\013er)g(\(dep)q(ending)i (on)75 1826 y(the)i(routine)h(used\).)29 b(With)18 b(the)g(curren)o(t)g (de\014nitions,)j(the)d(user)g(m)o(ust)g(use)g(extra)g(memory)f(for)h (the)75 1883 y(separate)d(bu\013ers)g(and)g(do)g(an)g(additional)i(cop) o(y)e(to)g(get)f(the)i(results)f(in)o(to)g(the)h(desired)g(bu\013er.) 166 1939 y(T)l(o)10 b(a)o(v)o(oid)h(this)g(means)g(of)f(e\013ecting)h (\\in-place")h(collectiv)o(e)g(comm)o(unication)g(op)q(erations)f(is)g (giv)o(en,)75 1996 y(as)k(outlined)i(in)f(the)f(previous)h(paragraph.)j (Three)c(p)q(ossible)i(approac)o(hes)e(app)q(ear)h(v)m(alid:)131 2090 y(1.)22 b(Alias)16 b(the)f(input)h(and)g(output)f(bu\013ers)g(or)f (parts)h(thereof.)20 b(Disadv)m(an)o(tages:)208 2183 y(\(a\))i(Requires)17 b(displacemen)o(t)g(for)e(routines)h(suc)o(h)g (as)g Fr(MPI)p 1238 2183 V 16 w(Reduce)p 1389 2183 V 18 w(scatter)h Fv(that)e(do)g(not)h(tak)o(e)289 2240 y(them)f(as)g(argumen)o(ts.)205 2313 y(\(b\))23 b(Violates)15 b(the)h(F)l(ortran)e(standard)g(whic)o(h)j(prohibits)f(up)q(dating)g (of)f(aliased)h(arra)o(ys.)210 2386 y(\(c\))23 b(Prev)o(en)o(ts)14 b(use)i(of)f Fr(MPI)p 691 2386 V 15 w(BOTTOM)h Fv(with)g(struct)f(t)o (yp)q(es.)131 2480 y(2.)22 b(Set)11 b(the)h(address)g(of)f Fr(sendbuf)j Fv(or)e Fr(recvbuf)p Fv(,)g(dep)q(ending)i(on)e(the)g (function,)h(to)e(a)g(de\014ned)i(constan)o(t.)131 2574 y(3.)22 b(Add)17 b(a)f(new)h(set)g(of)f(functions.)25 b(Disadv)m(an)o(tage:)d(increases)c(the)f(n)o(um)o(b)q(er)g(of)f (functions)h(in)h(the)189 2630 y Fr(MPI)c Fv(standard.)p 75 2661 720 2 v 127 2688 a Fi(1)144 2704 y Fg(Prop)q(osed)g(b)o(y)g(S.) f(Fleisc)o(hman,)h(Con)o(v)o(ex)g(Computer)f(Corp)q(oration.)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fv(4)560 b Fl(CHAPTER)16 b(6.)34 b(EXTENDED)15 b(COLLECTIVE)h(OPERA)l(TIONS)166 49 y Fv(The)38 b(second)h(approac)o(h) f(is)h(adv)o(o)q(cated.)88 b(De\014ne)39 b(constan)o(ts,)j Fr(MPI)p 1533 49 14 2 v 16 w(NO)p 1614 49 V 17 w(C)p 1660 49 V 17 w(ARG)d Fv(and)75 106 y Fr(MPI)p 160 106 V 16 w(NO)p 241 106 V 17 w(FN)p 316 106 V 16 w(ARG)p Fv(,)22 b(and)f(test)g(the)g(address)g(of)f(the)i(bu\013er)f(to)f (indicate)j(the)e(use)g(of)g(the)g(in-place)75 162 y(op)q(eration.)g(A) 15 b(separate)g(\015ag)g(for)g(C)g(and)g(F)l(ortran)f(are)h(suggested)h (for)e(safet)o(y's)g(sak)o(e)h(giv)o(en)h(that)f(the)75 219 y(in)o(ter-language)d(in)o(terface)g(is)g(not)f(standardized)i(and) f(the)f(means)h(in)g(whic)o(h)h(to)e(share)g(global)i(v)m(ariables)75 275 y(v)m(aries)j(considerably)h(b)q(et)o(w)o(een)e(compilers.)166 332 y(Note)i(that)f(the)h(user)g(giv)o(es)g(up)h(abilit)o(y)g(to)e(c)o (hange)h(the)h(exten)o(t)e(of)h(the)g(initial)i(data)d(from)h(that)75 388 y(of)d(the)h(\014nal)h(when)g(the)f(in-place)h(option)g(is)f(used.) 20 b(Basically)l(,)d(the)e(gather)f(routines)h(pass)g(the)g(NULL)75 444 y(address)f(for)g Fr(sendbuf)i Fv(and)e(the)h(scatter)e(and)h (reduce)h(routines)g(pass)f(the)g(NULL)h(address)f(for)g Fr(recvbuf)p Fv(.)75 501 y(When)21 b(this)g(is)g(done)g(the)f(t)o(yp)q (e)h(and)g(displacemen)o(t)h(information)e(is)h(tak)o(en)f(from)g(the)h (non-NULL)75 557 y(bu\013er.)f(See)c(b)q(elo)o(w.)166 614 y(The)10 b(follo)o(wing)h(list)g(describ)q(es)h(the)e(mo)q (di\014cation)i(for)d(eac)o(h)h(of)g(the)g(a\013ected)g(routines)h (\(su\016cien)o(tly)75 670 y(up)o(w)o(ard)k(compatible?\):)75 761 y Fr(MPI)p 160 761 V 16 w(GA)l(THER)24 b Fv(Ro)q(ot,)14 b(and)i(only)f(ro)q(ot,)f(passes)h(a)f Fr(NULL)h Fv(address)g(for)f Fr(sendbuf)p Fv(.)23 b Fr(sendt)o(yp)q(e)17 b Fv(and)e Fr(send-)189 818 y(count)g Fv(are)f(ignored;)h Fr(recvt)o(yp)q(e)h (recvcount)f Fv(are)f(used)h(for)f(b)q(oth)h(input)g(and)g(output)f(on) g(ro)q(ot)g(The)189 874 y(sp)q(eci\014ed)k(space)f(m)o(ust)f(exist)h (at)f Fr(recvbuf+ro)q(ot)p Fq(\001)p Fr(recvcount)p Fq(\001)p Fr(extent\(recvt)o(yp)q(e)q(\))j Fv(for)d(the)g(segmen)o(t)189 931 y(corresp)q(onding)g(to)e(ro)q(ot.)75 1023 y Fr(MPI)p 160 1023 V 16 w(GA)l(THERV)24 b Fv(Ro)q(ot,)f(and)e(only)h(ro)q(ot,)f (passes)h(a)f Fr(NULL)f Fv(address)i(for)e Fr(sendbuf)p Fv(.)41 b Fr(sendt)o(yp)q(e)24 b Fv(and)189 1080 y Fr(sendcount)19 b Fv(are)c(ignored;)i Fr(recvbuf)p Fv(,)g Fr(recvt)o(yp)q(e[ro)q(ot])g Fv(and)f Fr(recvcounts[ro)q(ot])h Fv(are)f(used)h(for)e(input.)189 1136 y(The)e(sp)q(eci\014ed)j(space)d(m)o(ust)g(exist)h(at)f Fr(recvbuf+displs[ro)q(ot])p Fq(\001)p Fr(extent)q(\(recvt)o(yp)q(e\))j Fv(for)d(the)g(segmen)o(t)189 1193 y(corresp)q(onding)j(to)e(ro)q(ot.) 75 1286 y Fr(MPI)p 160 1286 V 16 w(SCA)l(TTER)24 b Fv(Ro)q(ot,)14 b(and)i(only)f(ro)q(ot,)f(passes)h(a)g Fr(NULL)g Fv(address)g(for)g Fr(recvbuf)p Fv(.)20 b Fr([)p Fv(sendbuf)t(])c(remains)189 1342 y(unc)o(hanged)g(on)f(ro)q(ot.)75 1435 y Fr(MPI)p 160 1435 V 16 w(SCA)l(TTERV)24 b Fv(Ro)q(ot,)12 b(and)g(only)h(ro)q (ot,)f(passes)g(a)f Fr(NULL)h Fv(address)g(for)g Fr(recvbuf)p Fv(.)19 b Fr([)p Fv(sendbuf)t(])13 b(remains)189 1491 y(unc)o(hanged)j(on)f(ro)q(ot.)75 1584 y Fr(MPI)p 160 1584 V 16 w(ALLGA)l(THER)24 b Fv(A)17 b(NULL)i(address)f(is)g(passed)g (for)f Fr(sendbuf)p Fv(.)29 b Fr(sendt)o(yp)q(e)21 b Fv(and)d Fr(sendcount)i Fv(are)d(ig-)189 1640 y(nored)e(and)g Fr(recvbuf[])p Fv(,)g Fr(recvt)o(yp)q(e[rank])h Fv(and)f Fr(recvcounts[rank])i Fv(are)e(used)g(for)f(input.)21 b(The)15 b(input)189 1697 y(for)f(eac)o(h)h(task)g(is)h(at)e Fr(recvbuf+rank)p Fq(\001)p Fr(recvcount)p Fq(\001)p Fr(extent)q(\(recvt)o(yp)q(e\))p Fv(.)75 1789 y Fr(MPI)p 160 1789 V 16 w(ALLGA)l(THERV)24 b Fv(A)14 b(NULL)i(address)f(is)g (passed)g(for)f Fr(sendbuf)p Fv(.)22 b(F)l(or)14 b Fr(MPI)p 1434 1789 V 15 w(ALLGA)l(THER)i(recvt)o(yp)q(e)189 1846 y Fv(and)f Fr(recvcount)h Fv(and)g(for)e Fr(MPI)p 718 1846 V 16 w(ALLGA)l(THERV)i(recvt)o(yp)q(e[rank])h Fv(and)e Fr(recvcount[rank])i Fv(are)d(used)189 1902 y(for)g(input.)21 b(The)16 b(input)g(for)e(eac)o(h)i(task)e(is)i(at)e Fr (recvbuf+displ[rank])p Fq(\001)p Fr(exten)q(t\(recvt)o(yp)r(e\))p Fv(.)75 1995 y Fr(MPI)p 160 1995 V 16 w(REDUCE)23 b Fv(Ro)q(ot,)14 b(and)f(only)i(ro)q(ot,)d(passes)i(a)f(NULL)h(address)g(for)f Fr(recvbuf)p Fv(.)20 b(On)14 b(ro)q(ot)f(the)h(results)189 2052 y(are)h(placed)h(in)g Fr(sendbuf)p Fv(.)75 2144 y Fr(MPI)p 160 2144 V 16 w(ALLREDUCE)23 b Fv(A)13 b(NULL)h(address)f (is)g(passed)h(for)e Fr(recvbuf)p Fv(.)20 b(The)13 b(results)g(are)g (placed)h(in)g Fr(sendbuf)p Fv(.)75 2237 y Fr(MPI)p 160 2237 V 16 w(REDUCE)p 352 2237 V 17 w(SCA)l(TTER)24 b Fv(A)16 b(NULL)h(address)f(is)h(passed)f(for)g Fr(recvbuf)p Fv(.)23 b(The)16 b(results)h(are)f(placed)h(in)189 2293 y Fr(sendbuf+disp)q(l)h Fv(where)d Fr(displ)h Fv(=)751 2261 y Ff(P)795 2275 y Fe(i def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 141.734 356.153 0.000 0.000 30.287 101.571 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 30.287] CP [33.857 0.000] CP [101.571 0.000 ] CP [67.714 30.287 ] CP [67.714 30.287 0.000 30.287 0.000 30.287] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <666666333333999999CCCCCC666666333333999999CCCCCC> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 141.734 423.867 0.000 0.000 90.861 33.857 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 30.287] CP [0.000 90.861] CP [33.857 60.574 ] CP [33.857 0.000 ] CP [33.857 0.000 0.000 30.287 0.000 30.287] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 422.255 212.308 356.959 212.308 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 421.852 191.553 356.556 191.553 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 457.724 162.042 424.673 191.553 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 458.127 180.021 425.076 209.531 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 439.586 156.489 374.290 156.489 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 441.602 217.840 441.602 157.265 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 390.010 232.595 390.010 172.797 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 426.285 140.957 390.010 172.797 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {13.500 RR2} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 67.000 74.000 0.000 0.000 605.000 438.000 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {SR} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 0.050 setcmykcolor end} {0.949 0.949 0.949 setrgbcolor} ifelse } def /PF {gsave fill grestore} def 1.000 0 160.000 142.000 0.000 0.000 72.000 73.000 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <777777BBBBBBDDDDDDEEEEEE777777BBBBBBDDDDDDEEEEEE> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 145.000 142.000 0.000 0.000 15.000 90.000 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 15.000] CP [15.000 0.000] CP [90.000 0.000 ] CP [74.000 15.000 ] CP [74.000 15.000 0.000 15.000 0.000 15.000] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <666666333333999999CCCCCC666666333333999999CCCCCC> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 145.000 215.000 0.000 0.000 88.000 16.000 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 15.000] CP [0.000 88.000] CP [16.000 71.000 ] CP [16.000 0.000 ] CP [16.000 0.000 0.000 15.000 0.000 15.000] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 214.000 208.000 143.000 208.000 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 212.500 184.000 141.500 184.000 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 180.000 229.000 180.000 156.000 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 229.000 170.000 213.000 185.000 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 232.500 191.000 217.500 209.000 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 192.000 144.000 177.000 159.000 Line %@EndLine % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginOval /RT {E} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 545.724 134.749 0.000 0.000 104.108 112.068 rect %@EndOval %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginOval /RT {E} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 303.891 134.749 0.000 0.000 104.108 112.068 rect %@EndOval %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginOval /RT {E} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 424.307 134.749 0.000 0.000 104.108 112.068 rect %@EndOval %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {13.500 RR2} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 300.000 93.500 0.000 0.000 113.000 401.000 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {13.500 RR2} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 419.000 92.500 0.000 0.000 113.000 401.000 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {13.500 RR2} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 539.000 93.500 0.000 0.000 113.000 401.000 rect %@EndRect % %tfBegGrp % % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {SR} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 0.050 setcmykcolor end} {0.949 0.949 0.949 setrgbcolor} ifelse } def /PF {gsave fill grestore} def 1.000 0 352.352 145.437 0.000 0.000 17.647 77.500 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <777777BBBBBBDDDDDDEEEEEE777777BBBBBBDDDDDDEEEEEE> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 345.000 145.437 0.000 0.000 7.352 90.000 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 7.352] CP [12.500 0.000] CP [90.000 0.000 ] CP [77.500 7.352 ] CP [77.500 7.352 0.000 7.352 0.000 7.352] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <666666333333999999CCCCCC666666333333999999CCCCCC> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 345.245 222.937 0.000 0.000 24.754 12.500 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 7.107] CP [0.000 24.754] CP [12.500 17.156 ] CP [12.500 0.000 ] CP [12.500 0.000 0.000 7.107 0.000 7.107] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly % %tfEndGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 185.143 370.000 185.143 352.575 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 196.613 345.000 185.143 352.575 Line %@EndLine % %tfEndGrp % % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginOval /RT {E} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 303.500 337.073 0.000 0.000 104.108 112.068 rect %@EndOval % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {SR} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 0.050 setcmykcolor end} {0.949 0.949 0.949 setrgbcolor} ifelse } def /PF {gsave fill grestore} def 1.000 0 354.114 339.892 0.000 0.000 17.885 68.001 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <777777BBBBBBDDDDDDEEEEEE777777BBBBBBDDDDDDEEEEEE> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 332.000 339.892 0.000 0.000 21.981 101.000 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 21.981] CP [33.377 0.000] CP [101.000 0.000 ] CP [68.055 21.981 ] CP [68.055 21.981 0.000 21.981 0.000 21.981] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <666666333333999999CCCCCC666666333333999999CCCCCC> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 332.360 407.947 0.000 0.000 39.639 32.944 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 21.621] CP [0.000 39.639] CP [32.944 17.657 ] CP [32.944 0.000 ] CP [32.944 0.000 0.000 21.621 0.000 21.621] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 424.853 343.171 357.664 343.171 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 423.552 361.909 423.552 343.531 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 373.269 353.981 406.647 332.360 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 373.269 372.000 373.269 353.981 Line %@EndLine % %tfEndGrp % % %tfEndGrp % % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginOval /RT {E} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 422.981 337.073 0.000 0.000 104.108 112.068 rect %@EndOval % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {SR} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 0.050 setcmykcolor end} {0.949 0.949 0.949 setrgbcolor} ifelse } def /PF {gsave fill grestore} def 1.000 0 473.596 339.892 0.000 0.000 17.885 68.001 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <777777BBBBBBDDDDDDEEEEEE777777BBBBBBDDDDDDEEEEEE> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 451.481 339.892 0.000 0.000 21.981 101.000 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 21.981] CP [33.377 0.000] CP [101.000 0.000 ] CP [68.055 21.981 ] CP [68.055 21.981 0.000 21.981 0.000 21.981] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <666666333333999999CCCCCC666666333333999999CCCCCC> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 451.842 407.947 0.000 0.000 39.639 32.944 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 21.621] CP [0.000 39.639] CP [32.944 17.657 ] CP [32.944 0.000 ] CP [32.944 0.000 0.000 21.621 0.000 21.621] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 424.853 462.653 357.664 462.653 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 423.552 481.391 423.552 463.013 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 373.269 473.463 406.647 451.842 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 373.269 491.481 373.269 473.463 Line %@EndLine % %tfEndGrp % % %tfEndGrp % % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginOval /RT {E} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {} def /PC {} def 1.000 0 542.463 337.073 0.000 0.000 104.108 112.068 rect %@EndOval % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {SR} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 0.050 setcmykcolor end} {0.949 0.949 0.949 setrgbcolor} ifelse } def /PF {gsave fill grestore} def 1.000 0 593.078 339.892 0.000 0.000 17.885 68.001 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <777777BBBBBBDDDDDDEEEEEE777777BBBBBBDDDDDDEEEEEE> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 570.963 339.892 0.000 0.000 21.981 101.000 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 21.981] CP [33.377 0.000] CP [101.000 0.000 ] CP [68.055 21.981 ] CP [68.055 21.981 0.000 21.981 0.000 21.981] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <666666333333999999CCCCCC666666333333999999CCCCCC> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 571.324 407.947 0.000 0.000 39.639 32.944 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 21.621] CP [0.000 39.639] CP [32.944 17.657 ] CP [32.944 0.000 ] CP [32.944 0.000 0.000 21.621 0.000 21.621] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 424.853 582.134 357.664 582.134 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 423.552 600.873 423.552 582.495 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 373.269 592.945 406.647 571.324 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 373.269 610.963 373.269 592.945 Line %@EndLine % %tfEndGrp % % %tfEndGrp % % %tfBegGrp % % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {SR} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 0.050 setcmykcolor end} {0.949 0.949 0.949 setrgbcolor} ifelse } def /PF {gsave fill grestore} def 1.000 0 472.251 145.437 0.000 0.000 17.647 77.500 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <777777BBBBBBDDDDDDEEEEEE777777BBBBBBDDDDDDEEEEEE> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 464.898 145.437 0.000 0.000 7.352 90.000 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 7.352] CP [12.500 0.000] CP [90.000 0.000 ] CP [77.500 7.352 ] CP [77.500 7.352 0.000 7.352 0.000 7.352] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <666666333333999999CCCCCC666666333333999999CCCCCC> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 465.143 222.937 0.000 0.000 24.754 12.500 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 7.107] CP [0.000 24.754] CP [12.500 17.156 ] CP [12.500 0.000 ] CP [12.500 0.000 0.000 7.107 0.000 7.107] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly % %tfEndGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 185.143 489.898 185.143 472.474 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 196.613 464.898 185.143 472.474 Line %@EndLine % %tfEndGrp % % %tfBegGrp % % %tfBegGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginRect /RT {SR} def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 0.050 setcmykcolor end} {0.949 0.949 0.949 setrgbcolor} ifelse } def /PF {gsave fill grestore} def 1.000 0 592.631 145.783 0.000 0.000 17.647 77.500 rect %@EndRect %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <777777BBBBBBDDDDDDEEEEEE777777BBBBBBDDDDDDEEEEEE> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 585.278 145.783 0.000 0.000 7.352 90.000 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 7.352] CP [12.500 0.000] CP [90.000 0.000 ] CP [77.500 7.352 ] CP [77.500 7.352 0.000 7.352 0.000 7.352] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginPoly /saveobj save def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /backfill {/setcmykcolor where {begin gsave 0 0 0 0 setcmykcolor fill grestore end} {gsave 1 1 1 setrgbcolor fill grestore} ifelse} def /fillpat <666666333333999999CCCCCC666666333333999999CCCCCC> def /iSizeX 24 def /iSizeY 24 def /PF { /pat fillpat def DefaultPatternFont backfill 65 PatternFill } def 1.000 0 585.524 223.283 0.000 0.000 24.754 12.500 pbox astore pop 0 792 translate 1 -1 scale exch translate pop CTM currentmatrix pop /objectCord mstore def pbox bbox copy pop /CP {aload pop moveto /CP {aload length 2 eq {lineto}{curveto}ifelse} bind def} bind def [0.000 7.107] CP [0.000 24.754] CP [12.500 17.156 ] CP [12.500 0.000 ] CP [12.500 0.000 0.000 7.107 0.000 7.107] CP closepath /useClip true def PC PF LC SS saveobj restore %@EndPoly % %tfEndGrp % %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 185.489 610.278 185.489 592.854 Line %@EndLine %@BeginRuler %@ 9.0000000000000000000e+0 viewScale %@ 1.1111111111111110490e-1 realScale %@ 8 realDivs %@ 0000 rulerFormat %@EndRuler %@BeginLine /ArrowType{ } def /SD {} def /SS {setlinewidth SD stroke} def /LC { /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse } def /PF {gsave fill grestore} def /SS {setlinewidth SD stroke} def 1.000 196.960 585.278 185.489 592.854 Line %@EndLine % %tfEndGrp % % %tfEndGrp % %@BeginText save [ 39/quotesingle 96/grave 128/Adieresis/Aring/Ccedilla/Eacute/Ntilde/Odieresis /Udieresis/aacute/agrave/acircumflex/adieresis/atilde/aring/ccedilla/eacute /egrave/ecircumflex/edieresis/iacute/igrave/icircumflex/idieresis/ntilde /oacute/ograve/ocircumflex/odieresis/otilde/uacute/ugrave/ucircumflex /udieresis/dagger/degree/cent/sterling/section/bullet/paragraph/germandbls /registered/copyright/trademark/acute/dieresis/.notdef/AE/Oslash /.notdef/plusminus/.notdef/.notdef/yen/mu/.notdef/.notdef /.notdef/.notdef/.notdef/ordfeminine/ordmasculine/.notdef/ae/oslash /questiondown/exclamdown/logicalnot/.notdef/florin/.notdef/.notdef /guillemotleft/guillemotright/ellipsis/.notdef/Agrave/Atilde/Otilde/OE/oe /endash/emdash/quotedblleft/quotedblright/quoteleft/quoteright/divide /.notdef/ydieresis/Ydieresis/fraction/currency/guilsinglleft/guilsinglright /fi/fl/daggerdbl/periodcentered/quotesinglbase/quotedblbase/perthousand /Acircumflex/Ecircumflex/Aacute/Edieresis/Egrave/Iacute/Icircumflex /Idieresis/Igrave/Oacute/Ocircumflex/.notdef/Ograve/Uacute/Ucircumflex /Ugrave/dotlessi/circumflex/tilde/macron/breve/dotaccent/ring/cedilla /hungarumlaut/ogonek/caron] StandardEncoding Re-encode /_NewCenturySchlbk-Roman /NewCenturySchlbk-Roman EncodeFont %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 278.000 110.000 0.000 0.000 21.000 43.154 rect %@EndRect gsave 131.577 288.500 T translate 0.0 rotate 1 1 scale 18.000 (_NewCenturySchlbk-Roman) plain F /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (A) -21.333 6.000 S /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (f) -8.333 6.000 S (t) -2.339 6.000 S (e) 4.665 6.000 S (r) 13.665 6.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 94.000 114.000 0.000 0.000 21.000 53.982 rect %@EndRect gsave 140.991 104.500 T translate 0.0 rotate 1 1 scale 18.000 (_NewCenturySchlbk-Roman) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (B) -26.500 6.000 S /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (e) -13.500 6.000 S (f) -4.500 6.000 S (o) 1.493 6.000 S (r) 10.493 6.000 S (e) 18.482 6.000 S grestore %@EndText % %tfEndGrp % % %tLayer % %@BeginText restore save [ 39/quotesingle 96/grave 128/Adieresis/Aring/Ccedilla/Eacute/Ntilde/Odieresis /Udieresis/aacute/agrave/acircumflex/adieresis/atilde/aring/ccedilla/eacute /egrave/ecircumflex/edieresis/iacute/igrave/icircumflex/idieresis/ntilde /oacute/ograve/ocircumflex/odieresis/otilde/uacute/ugrave/ucircumflex /udieresis/dagger/degree/cent/sterling/section/bullet/paragraph/germandbls /registered/copyright/trademark/acute/dieresis/.notdef/AE/Oslash /.notdef/plusminus/.notdef/.notdef/yen/mu/.notdef/.notdef /.notdef/.notdef/.notdef/ordfeminine/ordmasculine/.notdef/ae/oslash /questiondown/exclamdown/logicalnot/.notdef/florin/.notdef/.notdef /guillemotleft/guillemotright/ellipsis/.notdef/Agrave/Atilde/Otilde/OE/oe /endash/emdash/quotedblleft/quotedblright/quoteleft/quoteright/divide /.notdef/ydieresis/Ydieresis/fraction/currency/guilsinglleft/guilsinglright /fi/fl/daggerdbl/periodcentered/quotesinglbase/quotedblbase/perthousand /Acircumflex/Ecircumflex/Aacute/Edieresis/Egrave/Iacute/Icircumflex /Idieresis/Igrave/Oacute/Ocircumflex/.notdef/Ograve/Uacute/Ucircumflex /Ugrave/dotlessi/circumflex/tilde/macron/breve/dotaccent/ring/cedilla /hungarumlaut/ogonek/caron] StandardEncoding Re-encode /_Times-Bold /Times-Bold EncodeFont %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 73.000 110.664 0.000 0.000 26.000 330.670 rect %@EndRect gsave 276.000 86.000 T translate 0.0 rotate 1 1 scale 24.000 (_Times-Bold) plain F /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (I) -164.835 6.000 S 18.000 (_Times-Bold) fess (N) -155.495 6.000 S (T) -142.496 6.000 S (E) -130.490 6.000 S (R) -118.484 6.000 S 24.000 (_Times-Bold) fess (-) -105.485 6.000 S 18.000 (_Times-Bold) fess (C) -97.493 6.000 S (O) -84.494 6.000 S (M) -70.493 6.000 S (M) -53.504 6.000 S (U) -36.515 6.000 S (N) -23.516 6.000 S (I) -10.517 6.000 S (C) -3.512 6.000 S (A) 9.486 6.000 S (T) 22.485 6.000 S (O) 34.491 6.000 S (R) 48.492 6.000 S 24.000 (_Times-Bold) fess ( ) 61.491 6.000 S (C) 67.491 6.000 S 18.000 (_Times-Bold) fess (A) 84.823 6.000 S (R) 97.822 6.000 S (T) 110.821 6.000 S 24.000 (_Times-Bold) fess (-) 122.827 6.000 S 18.000 (_Times-Bold) fess (S) 130.819 6.000 S (U) 140.830 6.000 S (B) 153.829 6.000 S grestore %@EndText % %tLayer % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 165.000 156.500 0.000 0.000 16.000 7.000 rect %@EndRect gsave 160.000 173.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (0) -3.000 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 165.885 197.615 0.000 0.000 16.000 7.000 rect %@EndRect gsave 201.115 173.885 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (1) -2.884 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 189.310 156.500 0.000 0.000 16.000 7.000 rect %@EndRect gsave 160.000 197.310 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (2) -3.000 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 190.196 197.615 0.000 0.000 16.000 7.000 rect %@EndRect gsave 201.115 198.196 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (3) -2.884 4.000 S grestore %@EndText % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 211.620 156.500 0.000 0.000 16.000 7.000 rect %@EndRect gsave 160.000 219.620 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (4) -3.000 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 212.506 197.615 0.000 0.000 16.000 7.000 rect %@EndRect gsave 201.115 220.506 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (5) -2.884 4.000 S grestore %@EndText % %tfEndGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 356.000 164.000 0.000 0.000 16.000 7.000 rect %@EndRect gsave 167.500 364.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (0) -3.500 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 355.000 205.000 0.000 0.000 16.000 7.000 rect %@EndRect gsave 208.500 363.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (1) -3.500 4.000 S grestore %@EndText % %tfEndGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 475.000 165.000 0.000 0.000 16.000 7.000 rect %@EndRect gsave 168.500 483.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (2) -3.500 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 475.000 205.000 0.000 0.000 16.000 7.000 rect %@EndRect gsave 208.500 483.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (3) -3.500 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 596.000 165.000 0.000 0.000 16.000 7.000 rect %@EndRect gsave 168.500 604.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (4) -3.500 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 595.000 203.000 0.000 0.000 16.000 7.000 rect %@EndRect gsave 206.500 603.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (5) -3.500 4.000 S grestore %@EndText % %tfBegGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 176.091 369.942 0.000 0.000 16.000 7.000 rect %@EndRect gsave 373.442 184.091 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (0) -2.557 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 176.770 406.795 0.000 0.000 16.000 7.000 rect %@EndRect gsave 410.295 184.770 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (1) -2.704 4.000 S grestore %@EndText % %tfEndGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 196.126 369.942 0.000 0.000 16.000 7.000 rect %@EndRect gsave 373.442 204.126 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (2) -2.557 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 196.805 406.795 0.000 0.000 16.000 7.000 rect %@EndRect gsave 410.295 204.805 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (3) -2.704 4.000 S grestore %@EndText % %tfEndGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 214.162 368.942 0.000 0.000 16.000 7.000 rect %@EndRect gsave 372.442 222.162 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (4) -2.557 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 214.841 405.795 0.000 0.000 16.000 7.000 rect %@EndRect gsave 409.295 222.841 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (5) -2.704 4.000 S grestore %@EndText % %tfEndGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 331.197 377.942 0.000 0.000 16.000 7.000 rect %@EndRect gsave 381.442 339.197 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (6) -2.557 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 332.876 414.795 0.000 0.000 16.000 7.000 rect %@EndRect gsave 418.295 340.876 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (7) -2.704 4.000 S grestore %@EndText % %tfEndGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 449.000 377.500 0.000 0.000 16.000 7.000 rect %@EndRect gsave 381.000 457.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (8) -3.000 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 449.000 413.500 0.000 0.000 16.000 7.000 rect %@EndRect gsave 417.000 457.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (9) -3.000 4.000 S grestore %@EndText % %tfEndGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 355.660 354.073 0.000 0.000 16.000 7.000 rect %@EndRect gsave 357.573 363.660 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (0) -3.426 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 356.339 390.926 0.000 0.000 16.000 7.000 rect %@EndRect gsave 394.426 364.339 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (1) -2.573 4.000 S grestore %@EndText % %tfEndGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 476.660 355.073 0.000 0.000 16.000 7.000 rect %@EndRect gsave 358.573 484.660 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (2) -3.426 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 477.339 391.926 0.000 0.000 16.000 7.000 rect %@EndRect gsave 395.426 485.339 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (3) -2.573 4.000 S grestore %@EndText % %tfEndGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 596.660 353.073 0.000 0.000 16.000 7.000 rect %@EndRect gsave 356.573 604.660 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (4) -3.426 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 595.339 389.926 0.000 0.000 16.000 7.000 rect %@EndRect gsave 393.426 603.339 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (5) -2.573 4.000 S grestore %@EndText % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 570.000 376.500 0.000 0.000 16.000 14.000 rect %@EndRect gsave 383.500 578.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (1) -7.000 4.000 S (0) 0.000 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 570.000 412.500 0.000 0.000 16.000 14.000 rect %@EndRect gsave 419.500 578.000 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (1) -7.000 4.000 S (1) 0.000 4.000 S grestore %@EndText % %tfEndGrp % % %tfBegGrp % %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 143.160 396.073 0.000 0.000 16.000 7.000 rect %@EndRect gsave 399.573 151.160 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (6) -3.426 4.000 S grestore %@EndText %@BeginText %@BeginRect /RT {SR} def /LC {} def /SS {pop} def /PF {} def /PC {} def 0.000 0 144.839 432.926 0.000 0.000 16.000 7.000 rect %@EndRect gsave 436.426 152.839 T translate 0.0 rotate 1 1 scale 14.000 (_Times-Bold) fess /setcmykcolor where {begin 0.000 0.000 0.000 1.000 setcmykcolor end} {0.000 0.000 0.000 setrgbcolor} ifelse (7) -2.573 4.000 S grestore %@EndText % %tfEndGrp % % %tfEndGrp % % %tfEndGrp % restore end %%EndDocument @endspecial 456 2049 a(Figure)15 b(6.8:)k(in)o(tercomm)o(unicator)c (cartesian)h(sub)q(division)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 381 a Ft(Biblio)q(graph)m(y)75 604 y Fv([1])22 b(Message)15 b(P)o(assing)i(In)o(terface)f(F)l(orum.)23 b(MPI:)16 b(A)g(Message-Passing)g(In)o(terface)g(Standard.)23 b(T)l(ec)o(h-)146 660 y(nical)f(Rep)q(ort)g(Computer)f(Science)i (Departmen)o(t)e(T)l(ec)o(hnical)i(Rep)q(ort)f(CS-94-230,)f(Univ)o (ersit)o(y)146 717 y(of)d(T)l(ennessee,)j(Kno)o(xville,)g(TN,)d(Ma)o(y) g(5)h(1994.)29 b(T)l(o)19 b(app)q(ear)g(in)g(the)g(In)o(ternational)h (Journal)f(of)146 773 y(Sup)q(ercomputing)d(Applications,)h(V)l(olume)f (8,)f(Num)o(b)q(er)h(3/4,)d(1994.)75 867 y([2])22 b(An)o(thon)o(y)e (Skjellum,)k(Nathan)d(E.)g(Doss,)g(and)g(Kishore)h(Visw)o(anathan.)37 b(In)o(ter-comm)o(unicator)146 923 y(extensions)14 b(to)g(MPI)g(in)g (the)h(MPIX)f(\(MPI)f(eXtension\))i(Library)l(.)j(T)l(ec)o(hnical)e (rep)q(ort,)e(Mississippi)146 980 y(State)g(Univ)o(ersit)o(y)i(|)g (Dept.)e(of)h(Computer)g(Science,)i(April)f(1994.)j(Draft)14 b(v)o(ersion.)952 2828 y(22)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Nov 27 12:48:47 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA24169; Mon, 27 Nov 1995 12:48:46 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA16072 for mpi-core-out; Mon, 27 Nov 1995 11:45:22 -0600 Received: from piper.cs.colorado.edu (root@piper.cs.Colorado.EDU [128.138.236.20]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA16064 for ; Mon, 27 Nov 1995 11:45:15 -0600 Received: from [198.11.16.30] (mac3bryan.cs.colorado.edu [198.11.16.30]) by piper.cs.colorado.edu (8.7.1/8.7.1) with SMTP id KAA03741; Mon, 27 Nov 1995 10:45:03 -0700 (MST) X-Sender: mcbr@piper.cs.colorado.edu Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Mon, 27 Nov 1995 10:46:27 -0700 To: Tony Skjellum , mpi-core@mcs.anl.gov From: mcbryan@cs.colorado.edu (Oliver A. McBryan) Subject: Re: Collective chapter : SC'95 version Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I'm not sure it makes the most sense to email documents like this today, wheras a fewyears ago it was the way to do it, because other alternatives were very awkward. A lot of us read email from powerbooks on the road over slow modems. This morning I received the Collective chapter in the form of 26 mail items, each of length 23Kb, on my powerbook (Eudora breaks large messages into 32K pieces or smaller). The real problem is the long download time during which the mail I was trying to send out was stalled!. Why not use the Web to provide the same access capability, while emailing only a pointer. At 7:56 AM 11/27/95, Tony Skjellum wrote: >The following postscript is the promised update and integration >of collective chapter for SC'95. Clearly, more work remains to be >done for the next MPI meetings, but it is much further along... >-Tony Skjellum Oliver McBryan; mcbryan@cs.colorado.edu Phone: 303-6650544 and 303-4923898; Cell: 303-8097804; Fax: 303-4922844 Dept of Computer Science, Univ. of Colorado, Boulder, CO 80309-0430. WWW: http://www.cs.colorado.edu/~mcbryan/Home.html From mpi-core-human@mcs.anl.gov Mon Nov 27 21:34:50 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA17583; Mon, 27 Nov 1995 21:34:50 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA27050 for mpi-core-out; Mon, 27 Nov 1995 20:34:17 -0600 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA27041 for ; Mon, 27 Nov 1995 20:34:08 -0600 Received: (tony@localhost); by Aurora.CS.MsState.Edu (SMI-8.6/7.0m-FWP-MsState); id UAA06801; Mon, 27 Nov 1995 20:33:54 -0600 Date: Mon, 27 Nov 1995 20:33:54 -0600 From: Tony Skjellum Message-Id: <199511280233.UAA06801@Aurora.CS.MsState.Edu> To: mcbryan@cs.colorado.edu, mpi-core@mcs.anl.gov Subject: Re: Collective chapter : SC'95 version Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Oliver, this is the standard practice on all the MPI reflectors that I have been working on for over 2 years. Please make a proposal at the next meeting if you don't like it. I get e-mail postscript all the time from folks relating to MPI. Steve Lederman specifically asked me to distribute the files when they were done... -Tony SKjellum From mpi-core-human@mcs.anl.gov Tue Nov 28 10:17:50 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04611; Tue, 28 Nov 1995 10:17:49 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA03464 for mpi-core-out; Tue, 28 Nov 1995 09:15:09 -0600 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA03454; Tue, 28 Nov 1995 09:14:54 -0600 Message-Id: <199511281514.JAA03454@antares.mcs.anl.gov> To: Tony Skjellum cc: mcbryan@cs.colorado.edu, mpi-core@mcs.anl.gov Subject: Re: Collective chapter : SC'95 version In-Reply-to: Message from Tony Skjellum of "Mon, 27 Nov 1995 20:33:54 -0600." <199511280233.UAA06801@Aurora.CS.MsState.Edu> Date: Tue, 28 Nov 1995 09:14:53 -0600 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with Tony. The problem is really in inadequate mail software at your end, Oliver. You need a smarter downloader... Bill From mpi-core-human@mcs.anl.gov Tue Nov 28 10:44:41 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04900; Tue, 28 Nov 1995 10:44:40 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA04177 for mpi-core-out; Tue, 28 Nov 1995 09:42:49 -0600 Received: from donner (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA04158; Tue, 28 Nov 1995 09:42:31 -0600 Date: Tue, 28 Nov 1995 09:42:30 -0600 Message-Id: <199511281542.JAA17171@donner> From: lusk@mcs.anl.gov (Rusty Lusk) To: mpi-comm@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: MPI-2 BOF at SC95 [comp.parallel.mpi #1058] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk ------ Forwarded Article ------ From lederman@descartes.super.org (Steve Huss-Lederman) This is to announce that there will be a Birds-of-a-Feather about MPI-2 at Supercomputing '95. The session will be from 3:30-5:00 on Wednesday, December 6 in Room 11A. The first half of the session will be an overview of the current thinking in the MPI-2 standards effort; the second half will be public comments and input. To give people interested in the details of the MPI-2 standards effort a chance to review them, the MPI Forum is making the current DRAFT of the MPI-2 document available. It can be gotten from ftp://ftp.mcs.anl.gov/pub/mpi/mpi2/mpi-draft.ps.Z. To give people a flavor of the MPI-2 standards effort, the abstract for the document is attached below. Hope to see you there. The MPI Forum ---------------------------------------------------------------------- This document describes the MPI-2 effort to add extensions to the original MPI standard. The MPI-2 effort began in March 1995 and the additions to the standard are scheduled to be released for public comment at Supercomputing '96. Topics being explored for possible inclusion in MPI-2 are dynamic processes, one-sided communications, extended collective operations, external interfaces, additional language bindings, real-time communications, and miscellaneous topics. NOTE: This is the current state of some of the chapters being drafted for possible inclusion in the MPI-2 standard document. It represents the ongoing work of the MPI Forum in an incomplete and tentative form, but is being distributed with the intention of desseminating the work of the MPI Forum and to allow people attending the BOF at Supercomputing '95 to understand the topics under consideration. The proposals for MPI-2 are still very much under development. It is very likely that the final MPI-2 standard will differ in important ways from this draft. As a result, this draft should not be taken as a promise of the final form of the MPI-2 standard. ------ End of Forwarded Article From mpi-core-human@mcs.anl.gov Tue Nov 28 10:56:02 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA05055; Tue, 28 Nov 1995 10:56:01 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA04444 for mpi-core-out; Tue, 28 Nov 1995 09:54:31 -0600 Received: from piper.cs.colorado.edu (root@piper.cs.colorado.edu [128.138.236.20]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA04431; Tue, 28 Nov 1995 09:54:20 -0600 Received: from [198.11.16.30] (mac3bryan.cs.colorado.edu [198.11.16.30]) by piper.cs.colorado.edu (8.7.1/8.7.1) with SMTP id IAA07260; Tue, 28 Nov 1995 08:54:09 -0700 (MST) X-Sender: mcbr@piper.cs.colorado.edu Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 28 Nov 1995 08:55:34 -0700 To: William Gropp , Tony Skjellum From: mcbryan@cs.colorado.edu (Oliver A. McBryan) Subject: Re: Collective chapter : SC'95 version Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 9:14 AM 11/28/95, William Gropp wrote: >I agree with Tony. The problem is really in inadequate mail software at your >end, Oliver. You need a smarter downloader... >Bill No degree of smartness will bring in a massive mail file over a slow link. My fast modem failed recently, and I had temporarily restored the 2400b internal modem that came with my Powerbook, while looking for a 28.8k internal for it (no one seems to have one for 170/180 powerbooks). I usually take along a 28.8kb external modem if I'm expecting to download any significant files. However on this occasion only the old modem was in place, which is still fine for regular email traffic. While this may be an extreme case, lots of people are still using 9.6k modems. If you happen to download email on a flight as I sometimes do (because I need to find out where I'm supposed to be, or need to send out something urgent that I could only finish on the plane), the data rate is restricted to 9.6K, and airtime is expensive. So there are good reasons for people to want not to have random giant objects interspersed with their email. You can of course refuse to receive the large mail item. Eudora allows a setting that says "please dont transfer any item over X Kb". However I would then not even discover that I maybe urgently needed that particular large file. If you know of a better email handler for Mac's than Eudora, I'd be interested in knowing about it. In my view its a great and reliable mail program, much better than most workstation mailers, and has only one defficiency - it represents large email items as lots of small ones. On the other hand, one would normally send a large file like this as an attachment, not imbedded text, and Eudora handles attachments beautifully, whether they are postscript or videos, and no matter what size. What it cannot do is speed them up. I'm still convinced that mailing round massive postscript files is not the way to go today. The Web makes it far easier to achieve the same effect without drowning the net (many people on the list may not want to read a particular document) in pbytes (postscript bytes, which are pretty inefficient). Also with a Web pointer, you can access the latest version of something if desirable, not the static version that existed at the time of mailing. Oliver McBryan; mcbryan@cs.colorado.edu Phone: 303-6650544 and 303-4923898; Cell: 303-8097804; Fax: 303-4922844 Dept of Computer Science, Univ. of Colorado, Boulder, CO 80309-0430. WWW: http://www.cs.colorado.edu/~mcbryan/Home.html From mpi-core-human@mcs.anl.gov Tue Nov 28 11:31:17 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA05495; Tue, 28 Nov 1995 11:31:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA05267 for mpi-core-out; Tue, 28 Nov 1995 10:29:49 -0600 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA05253; Tue, 28 Nov 1995 10:29:36 -0600 Message-Id: <199511281629.KAA05253@antares.mcs.anl.gov> To: mcbryan@cs.colorado.edu (Oliver A. McBryan) cc: Tony Skjellum , mpi-core@mcs.anl.gov Subject: Re: Collective chapter : SC'95 version In-Reply-to: Message from mcbryan@cs.colorado.edu (Oliver A. McBryan) of "Tue, 28 Nov 1995 08:55:34 -0700." Date: Tue, 28 Nov 1995 10:29:34 -0600 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | At 9:14 AM 11/28/95, William Gropp wrote: | >I agree with Tony. The problem is really in inadequate mail software at your | >end, Oliver. You need a smarter downloader... | >Bill | | No degree of smartness will bring in a massive mail file over a slow link. I don't disagree, but I do believe that the mailer should be smart enough to (1) handle bi-directional transmission (not let long incoming files block outgoing messages), (2) allow you to preview the headers of "large" files so that you can decide whether you want to transfer the whole thing or not, (3) allow you to safely abort a transfer if it is inconvenient - note that it MUST ALREADY be able to do this; there is always the possibility that the link will fail or local disk will fill up. None of these are particularly difficult to implement. You should demand them of your mailer software. It is also the case that email remains the only (nearly) ubiquitous mechanism for sending data around. You and I are at locations that permit easy internet access, but some many people are not so lucky. Quite a few companies make it very difficult to impossible to access other internet services. I agree that a mechanism could be set up to handle both submissions to and from the "internet challenged", but I do not believe that is the right way to go. Bill From mpi-core-human@mcs.anl.gov Tue Nov 28 12:28:27 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA07092; Tue, 28 Nov 1995 12:28:25 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA06650 for mpi-core-out; Tue, 28 Nov 1995 11:27:08 -0600 Received: from wk65.nas.nasa.gov (wk65.nas.nasa.gov [129.99.64.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA06579 for ; Tue, 28 Nov 1995 11:24:49 -0600 Received: (from barszcz@localhost) by wk65.nas.nasa.gov (8.6.12/NAS.6.1) id JAA08768 for mpi-core@mcs.anl.gov; Tue, 28 Nov 1995 09:24:48 -0800 Date: Tue, 28 Nov 1995 09:24:48 -0800 From: barszcz@nas.nasa.gov (Eric Barszcz) Message-Id: <199511281724.JAA08768@wk65.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: Great Mail Debate Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Would it be possible to put the smarts on the sending side? Typically, the email involving postscript has a header saying what the postscript is for and sometimes a WWW address. Is it possible to do the following? 1) Assume that people can access the postscript over the web. 2) Set up an automatic mailing list for people who want postscript emailed. 3) Send out the header and postscript to 2 different mailing aliases: header with a URL to normal and postscript to the subscribed list. An alternative would be to create both lists with everyone to receive postscript and allow individuals to unsubscribe. Eric Barszcz barszcz@nas.nasa.gov From mpi-core-human@mcs.anl.gov Tue Nov 28 13:08:16 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA07737; Tue, 28 Nov 1995 13:08:15 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA07445 for mpi-core-out; Tue, 28 Nov 1995 12:03:39 -0600 Received: from piper.cs.colorado.edu (root@piper.cs.colorado.edu [128.138.236.20]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA07433 for ; Tue, 28 Nov 1995 12:03:30 -0600 Received: from [198.11.16.30] (mac3bryan.cs.colorado.edu [198.11.16.30]) by piper.cs.colorado.edu (8.7.1/8.7.1) with SMTP id LAA07982; Tue, 28 Nov 1995 11:03:18 -0700 (MST) X-Sender: mcbr@piper.cs.colorado.edu Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 28 Nov 1995 11:04:43 -0700 To: barszcz@nas.nasa.gov (Eric Barszcz), mpi-core@mcs.anl.gov From: mcbryan@cs.colorado.edu (Oliver A. McBryan) Subject: Re: Great Mail Debate Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk After complaining about email overload, i'm now subjecting everyone to a further overload in these discussions. There are various systems - such as netlib at Oak Ridge - that can email any document to someone who has sent in a request. So here is a possibility: Send a mail message with the Web address and the netlib (or equivalent) address. Then most people can access it over the Web (or equivalently ftp) while the few who dont have ftp access can mail a request to netlib (or equivalent). A simplified version of netlib is that those few people simply mail the sender asking for the full postscript. This would probably suffice for our group. While I believe there are lots of people who dont run Web servers, I find it hard to believe there are that many people who dont have a machine running ftp but do have one running sendmail. I believe there are some in this category, but I doubt its more than 5% of this group. At 9:24 AM 11/28/95, Eric Barszcz wrote: >Would it be possible to put the smarts on the sending side? > >Typically, the email involving postscript has a header saying >what the postscript is for and sometimes a WWW address. Is it >possible to do the following? > >1) Assume that people can access the postscript over the web. > >2) Set up an automatic mailing list for people who want postscript emailed. > >3) Send out the header and postscript to 2 different mailing aliases: > header with a URL to normal and postscript to the subscribed list. > >An alternative would be to create both lists with everyone to receive >postscript and allow individuals to unsubscribe. > > >Eric Barszcz >barszcz@nas.nasa.gov Oliver McBryan; mcbryan@cs.colorado.edu Phone: 303-6650544 and 303-4923898; Cell: 303-8097804; Fax: 303-4922844 Dept of Computer Science, Univ. of Colorado, Boulder, CO 80309-0430. WWW: http://www.cs.colorado.edu/~mcbryan/Home.html From mpi-core-human@mcs.anl.gov Tue Nov 28 13:34:37 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA07955; Tue, 28 Nov 1995 13:34:36 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA07943 for mpi-core-out; Tue, 28 Nov 1995 12:32:48 -0600 Received: from piper.cs.colorado.edu (root@piper.cs.colorado.edu [128.138.236.20]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA07932 for ; Tue, 28 Nov 1995 12:32:37 -0600 Received: from [198.11.16.30] (mac3bryan.cs.colorado.edu [198.11.16.30]) by piper.cs.colorado.edu (8.7.1/8.7.1) with SMTP id LAA08090; Tue, 28 Nov 1995 11:32:12 -0700 (MST) X-Sender: mcbr@piper.cs.colorado.edu Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 28 Nov 1995 11:33:51 -0700 To: mcbryan@mroe.cs.colorado.edu (Oliver A. McBryan), barszcz@nas.nasa.gov (Eric Barszcz), mpi-core@mcs.anl.gov From: mcbryan@cs.colorado.edu (Oliver A. McBryan) Subject: Re: Great Mail Debate Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Since I'm very aware that I'm wasting everyone's time reading this discussion I'm going to close it here, and respond only individually to further comments. Oliver McBryan; mcbryan@cs.colorado.edu Phone: 303-6650544 and 303-4923898; Cell: 303-8097804; Fax: 303-4922844 Dept of Computer Science, Univ. of Colorado, Boulder, CO 80309-0430. WWW: http://www.cs.colorado.edu/~mcbryan/Home.html From mpi-core-human@mcs.anl.gov Tue Nov 28 15:11:56 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA10394; Tue, 28 Nov 1995 15:11:55 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA10082 for mpi-core-out; Tue, 28 Nov 1995 14:09:57 -0600 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA10074 for ; Tue, 28 Nov 1995 14:09:47 -0600 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id MAA16759; Tue, 28 Nov 1995 12:09:40 -0800 From: "William C. Saphir" Message-Id: <9511281209.ZM16757@win233.nas.nasa.gov> Date: Tue, 28 Nov 1995 12:09:40 -0800 In-Reply-To: mcbryan@cs.colorado.edu (Oliver A. McBryan) "Re: Great Mail Debate" (Nov 28, 11:33am) References: X-Mailer: Z-Mail (3.1.0 22feb94 MediaMail) To: mcbryan@cs.colorado.edu (Oliver A. McBryan), mcbryan@mroe.cs.colorado.edu (Oliver A. McBryan), barszcz@nas.nasa.gov (Eric Barszcz), mpi-core@mcs.anl.gov Subject: WWW archives (was: Re: Great Mail Debate) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Just want to remind everyone that all the MPI-2 lists are archived (with hypermail) for easy access on the WWW. It is always an option to take yourself off the mailing list and browse the archives occasionally. Everything is indexed by subject, date, and author, and the archives go back as far as Majordomo has been keeping logs. Even Olga is in there. The URL is http://parallel.nas.nasa.gov/MPI-2/ Bill From mpi-core-human@mcs.anl.gov Tue Nov 28 17:03:55 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA11366; Tue, 28 Nov 1995 17:03:52 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA13169 for mpi-core-out; Tue, 28 Nov 1995 16:02:00 -0600 Received: from molly.vcpc.univie.ac.at (molly.vcpc.univie.ac.at [193.171.58.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA13140 for ; Tue, 28 Nov 1995 16:01:42 -0600 Received: from beavis.vcpc.univie.ac.at (beavis [193.171.58.38]) by molly.vcpc.univie.ac.at (8.7.1/8.7.1) with ESMTP id XAA02965 for ; Tue, 28 Nov 1995 23:01:37 +0100 (MET) From: Ian Received: (from ian@localhost) by beavis.vcpc.univie.ac.at (8.7.1/8.7.1) id XAA05622 for mpi-core@mcs.anl.gov; Tue, 28 Nov 1995 23:01:53 +0100 (MET) Date: Tue, 28 Nov 1995 23:01:53 +0100 (MET) Message-Id: <199511282201.XAA05622@beavis.vcpc.univie.ac.at> To: mpi-core@mcs.anl.gov Subject: January MPI meeting - Book soon for reduced room rate X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear all, for the benefit of those of you who will be coming to the January MPI meeting here in Vienna, I'd just like to let you know that we have negotiated a reduced rate with the Hotel Regina, at which the meeting will take place, but that this only applies to a *very* limited number of rooms. As the VCPC are holding some other events earlier in the week (MPI training course, introduction to parallel programming, languages workshop), some of the people attending them may well be staying on for the MPI workshop, but we would like to give you the first opportunity to take advantage of this deal. If you would like to do so, you will have to do so soon though, as the rooms will be allocated on a first come first served basis. In any case, I look forward to seeing those of you who come here in January, and also those of you who make it to SC '95 next week, which I will be attending. It's been a while since the MPI-1 days, and it will be nice to renew old acquaintances. Ian -- Ian Glendinning HPC Centre, Electronics & Computer Science igl@ecs.soton.ac.uk University of Southampton, SO17 1BJ, UK Tel: +44 1703 592897 WWW URL: http://hpcc.soton.ac.uk/staff/igl.html From mpi-core-human@mcs.anl.gov Tue Nov 28 17:27:25 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA11677; Tue, 28 Nov 1995 17:27:24 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA13954 for mpi-core-out; Tue, 28 Nov 1995 16:25:28 -0600 Received: from molly.vcpc.univie.ac.at (molly.vcpc.univie.ac.at [193.171.58.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA13946 for ; Tue, 28 Nov 1995 16:25:20 -0600 Received: from beavis.vcpc.univie.ac.at (beavis [193.171.58.38]) by molly.vcpc.univie.ac.at (8.7.1/8.7.1) with ESMTP id XAA03026 for ; Tue, 28 Nov 1995 23:25:18 +0100 (MET) From: Ian Received: (from ian@localhost) by beavis.vcpc.univie.ac.at (8.7.1/8.7.1) id XAA05632 for mpi-core@mcs.anl.gov; Tue, 28 Nov 1995 23:25:35 +0100 (MET) Date: Tue, 28 Nov 1995 23:25:35 +0100 (MET) Message-Id: <199511282225.XAA05632@beavis.vcpc.univie.ac.at> To: mpi-core@mcs.anl.gov Subject: Address correction... X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hello again, I've just realised I inadvertently managed to attach my old .sig file to the message I just posted about the January MPI meeting in Vienna... How embarrassing - my current address is indeed in Vienna, as below. (What's the smiley for a red face :-) Ian -- Ian Glendinning European Centre for Parallel Computing at Vienna (VCPC) ian@vcpc.univie.ac.at Liechtensteinstr. 22, A-1090 Vienna, Austria Tel: +43 1 310 939612 WWW: http://www.vcpc.univie.ac.at/~ian From mpi-core-human@mcs.anl.gov Mon Dec 11 11:45:55 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA10070; Mon, 11 Dec 1995 11:45:54 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA19558 for mpi-core-out; Mon, 11 Dec 1995 10:43:17 -0600 Received: from super.super.org (super.super.org [192.31.192.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA19550 for ; Mon, 11 Dec 1995 10:43:05 -0600 Received: from gotham.super.org (gotham [192.239.79.2]) by super.super.org (8.7.1/8.6.12.1) with ESMTP id LAA05205 for ; Mon, 11 Dec 1995 11:39:23 -0500 (EST) Received: from descartes.super.org (descartes [192.239.79.5]) by gotham.super.org (8.6.12/8.6.12.1) with ESMTP id LAA19507 for ; Mon, 11 Dec 1995 11:42:30 -0500 From: lederman@super.org (Steve Huss-Lederman) Received: (lederman@localhost) by descartes.super.org (8.6.12/8.6.12.client) id LAA17342; Mon, 11 Dec 1995 11:42:30 -0500 Date: Mon, 11 Dec 1995 11:42:30 -0500 Message-Id: <199512111642.LAA17342@descartes.super.org> To: mpi-core@mcs.anl.gov Subject: deadline for European document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This is just a short note to let everyone know that the deadline for input to the document for the European meeting is Friday, January 6. After that the document will be assembled and publically released. I assume many chapters will stay the same but some may change based on discussion via e-mail and at SC95. I once again urge the subcommittees to finish up their changes in time for others to review the chapters before they are publically released. Hope to see you in Europe. Steve From mpi-core-human@mcs.anl.gov Sat Dec 30 11:46:09 1995 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA02153; Sat, 30 Dec 1995 11:46:09 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA24039 for mpi-core-out; Sat, 30 Dec 1995 10:48:10 -0600 Received: from kanga.cse.nd.edu (kanga.cse.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA24026; Sat, 30 Dec 1995 10:48:01 -0600 From: Andrew.Lumsdaine@nd.edu Received: from owl.cse.nd.edu (owl.cse.nd.edu [129.74.25.119]) by kanga.cse.nd.edu (8.7.1/8.7.1) with ESMTP id LAA11615; Sat, 30 Dec 1995 11:47:59 -0500 (EST) Received: (from lums@localhost) by owl.cse.nd.edu (8.7.1/8.7.1) id LAA06455; Sat, 30 Dec 1995 11:47:58 -0500 (EST) Date: Sat, 30 Dec 1995 11:47:58 -0500 (EST) Message-Id: <199512301647.LAA06455@owl.cse.nd.edu> To: mpi-core@mcs.anl.gov, mpi-users@mcs.anl.gov Subject: CFP: 1996 MPI Developers Conference and Users Group Meeting Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk CALL FOR PARTICIPATION MPI Developers Conference and Users Group Meeting July 1-2, 1996 University of Notre Dame Notre Dame, IN 46556 The MPI Developers Conference and Users Group Meeting is a conference for all users of the Message Passing Interface (MPI) standard and is intended to support the continued development and use of MPI and its extensions. The conference will provide a forum for users from industry, national laboratories, and academia who are using MPI to present their ideas about, and experiences with, MPI. SCOPE: Examples of appropriate topics for this conference include: o) Applications using MPI o) Implementations of MPI o) Experiences with MPI o) Extensions to MPI o) Why MPI is better/worse than other message passing systems o) Message passing versus shared memory o) The future of message passing o) Software issues o) Anything else about MPI, message passing, or parallel computing SUBMISSIONS: Abstracts are solicited for: 1) Papers 2) Posters 3) Panel discussions 4) Software demonstrations TUTORIALS: Two pre-conference tutorials on MPI will be offered on June 29 and 30, 1996. The first tutorial, on June 29, will cover the basics of MPI including point-to-point and collective communication operations, and communicators. The second tutorial, on June 30, will pick up where the first tutorial left off and will cover data types, topologies, and intercommunicators. Attendees may take either or both of the tutorials. Both tutorials will be conducted in one of Notre Dame's SPARC-based instructional clusters where each attendee will have use of a workstation throughout the tutorial. The tutorial lectures will be interspersed with a series of exercises requiring each tutorial attendee to write, debug, and run MPI-based programs using C or Fortran. DEADLINES: Abstracts for papers: April 26, 1996 for posters: May 24, 1996 for panel discussions: May 24, 1996 for software demonstrations: May 24, 1996 Papers for proceedings: May 31, 1996 Lodging reservations: June 10, 1996 Early registration: June 10, 1996 FOR MORE INFORMATION: Complete and up-to-the-minute information about the conference can be found on the Web at http://www.cse.nd.edu/mpidc96/ or contact Andrew Lumsdaine Internet: Lumsdaine.1@nd.edu Computer Science & Engineering Phone: (219) 631-8716 University of Notre Dame Fax: (219) 631-9260 Notre Dame, IN 46556 From mpi-core-human@mcs.anl.gov Wed Jan 3 16:00:45 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA13775; Wed, 3 Jan 1996 16:00:44 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA01878 for mpi-core-out; Wed, 3 Jan 1996 15:02:13 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA01751 for ; Wed, 3 Jan 1996 14:58:20 -0600 Message-Id: <199601032058.OAA01751@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: spring meeting dates Date: Wed, 03 Jan 1996 14:58:19 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear group, At the Denver meeting we discussed and reached tentative agreement on a set of dates for the meetings in Chicago this spring. Unfortunately, I seem to have lost my notes on those dates. Can someone help me out, so I can go ahead and make arrangements with the hotel? Thanks very much. Rusty From mpi-core-human@mcs.anl.gov Thu Jan 4 11:46:09 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA01485; Thu, 4 Jan 1996 11:46:08 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA21335 for mpi-core-out; Thu, 4 Jan 1996 10:47:44 -0600 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id DAA14461 for ; Thu, 4 Jan 1996 03:28:36 -0600 Received: by felix.dircon.co.uk id AA16383 (5.67b/IDA-1.5 for ); Thu, 4 Jan 1996 09:28:25 GMT Message-Id: <199601040928.AA16383@felix.dircon.co.uk> Received: from gw4-198.pool.dircon.co.uk(194.73.168.198) by amnesiac via smap (V1.3) id sma016380; Thu Jan 4 09:28:18 1996 Received: by jim (5.x) id AA00490; Thu, 4 Jan 1996 09:28:30 GMT To: Rusty Lusk Cc: mpi-core@antares.mcs.anl.gov Subject: Re: spring meeting dates In-Reply-To: Your message of "Wed, 03 Jan 1996 14:58:19 CST." <199601032058.OAA01751@antares.mcs.anl.gov> Date: Thu, 04 Jan 1996 09:28:29 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > At the Denver meeting we discussed and reached tentative agreement on a set > of dates for the meetings in Chicago this spring. Unfortunately, I seem to > have lost my notes on those dates. Can someone help me out, so I can go > ahead and make arrangements with the hotel? Thanks very much. Before you re-book the O'Hare Hilton, can we have a quick poll on whether we'd like to try a different hotel. There must be a few more in the immediate airport vicinity which we could also use. I'd like one with a more social bar... -- Jim From mpi-core-human@mcs.anl.gov Thu Jan 4 12:01:45 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA01753; Thu, 4 Jan 1996 12:01:45 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA21768 for mpi-core-out; Thu, 4 Jan 1996 11:04:53 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA21760 for ; Thu, 4 Jan 1996 11:04:46 -0600 Message-Id: <199601041704.LAA21760@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Spring meeting dates Date: Thu, 04 Jan 1996 11:04:45 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Thanks to all who reminded me that the Spring meeting dates are March 6-8, 1996 April 24-26, 1996 plus a tentative one in the first week of June. Jim Cownie has suggested that I survey the bars in all the hotels near the airport in a quest for a better one. I may make some small effort in this direction, since I agree on the weakness of the O'Hare Hilton's bar. Does anyone have any specific recommendations in this area? Rusty From mpi-core-human@mcs.anl.gov Thu Jan 4 13:10:22 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA02606; Thu, 4 Jan 1996 13:10:21 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA23316 for mpi-core-out; Thu, 4 Jan 1996 12:12:55 -0600 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA23308 for ; Thu, 4 Jan 1996 12:12:47 -0600 Received: (tony@localhost); by Aurora.CS.MsState.Edu (SMI-8.6/7.0m-FWP-MsState); id MAA06399; Thu, 4 Jan 1996 12:12:35 -0600 Date: Thu, 4 Jan 1996 12:12:35 -0600 From: Tony Skjellum Message-Id: <199601041812.MAA06399@Aurora.CS.MsState.Edu> To: lusk@mcs.anl.gov, jcownie@bbn.com Subject: Re: spring meeting dates Cc: mpi-core@antares.mcs.anl.gov X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am against changing hotels. -Tony From mpi-core-human@mcs.anl.gov Thu Jan 4 13:52:45 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA03234; Thu, 4 Jan 1996 13:52:44 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA24181 for mpi-core-out; Thu, 4 Jan 1996 12:55:38 -0600 Received: from mailhost.lanl.gov (mailhost.lanl.gov [128.165.3.12]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA24172 for ; Thu, 4 Jan 1996 12:55:30 -0600 Received: from wrangler.lanl.gov by mailhost.lanl.gov (8.6.12/1.2) id LAA02440; Thu, 4 Jan 1996 11:55:28 -0700 Received: from twinkie.lanl.gov.lanl.gov (twinkie.lanl.gov [128.165.113.79]) by wrangler.lanl.gov (8.7.1/8.7.1) with SMTP id LAA24189 for ; Thu, 4 Jan 1996 11:55:28 -0700 (MST) Date: Thu, 4 Jan 1996 11:55:28 -0700 (MST) From: MaryDell Tholburn Message-Id: <199601041855.LAA24189@wrangler.lanl.gov> To: mpi-core@antares.mcs.anl.gov Subject: MPI Spring Meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The O'Hare Hilton is very convenient for those of us who are traveling to Chicago for the MPI Forum, I agree with Tony. MaryDell From mpi-core-human@mcs.anl.gov Thu Jan 4 14:07:18 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA03454; Thu, 4 Jan 1996 14:07:17 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA24513 for mpi-core-out; Thu, 4 Jan 1996 13:10:19 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA24427 for ; Thu, 4 Jan 1996 13:07:10 -0600 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) for <@sgi.engr.sgi.com:mpi-core@antares.mcs.anl.gov> id LAA02431; Thu, 4 Jan 1996 11:07:02 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@antares.mcs.anl.gov id LAA08348; Thu, 4 Jan 1996 11:07:00 -0800 From: "Eric Salo" Message-Id: <9601041107.ZM8346@mrjones.engr.sgi.com> Date: Thu, 4 Jan 1996 11:07:00 -0800 In-Reply-To: Rusty Lusk "Spring meeting dates" (Jan 4, 11:04am) References: <199601041704.LAA21760@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 10apr95 MediaMail) To: mpi-core@antares.mcs.anl.gov Subject: Re: Spring meeting dates Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I'm inclined to favor sticking with the Hilton, in spite of its warts. It's just so darn convenient - taking that shuttle in Denver sure was annoying, especially this time of year. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Thu Jan 4 15:46:43 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA04772; Thu, 4 Jan 1996 15:46:42 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA26738 for mpi-core-out; Thu, 4 Jan 1996 14:48:58 -0600 Received: from kanga.cse.nd.edu (kanga.cse.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA26729 for ; Thu, 4 Jan 1996 14:48:48 -0600 From: Andrew.Lumsdaine@nd.edu Received: from owl.cse.nd.edu (owl.cse.nd.edu [129.74.25.119]) by kanga.cse.nd.edu (8.7.1/8.7.1) with ESMTP id PAA00648; Thu, 4 Jan 1996 15:48:47 -0500 (EST) Received: (from lums@localhost) by owl.cse.nd.edu (8.7.1/8.7.1) id PAA10721; Thu, 4 Jan 1996 15:48:47 -0500 (EST) Date: Thu, 4 Jan 1996 15:48:47 -0500 (EST) Message-Id: <199601042048.PAA10721@owl.cse.nd.edu> To: salo@mrjones.engr.sgi.com CC: mpi-core@antares.mcs.anl.gov In-reply-to: <9601041107.ZM8346@mrjones.engr.sgi.com> (message from Eric Salo on Thu, 4 Jan 1996 11:07:00 -0800) Subject: Re: Spring meeting dates Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode Eric Salo wrote: > I'm inclined to favor sticking with the Hilton, in spite of its > warts. It's just so darn convenient - taking that shuttle in Denver > sure was annoying, especially this time of year. There are many hotels that are very close to O'Hare -- not quite as close as the Hilton, of course, but much closer than the last hotel was to DIA. I stayed at the O'Hare Westin recently -- it would be about a two minute shuttle ride from O'Hare. I don't have any information about the bar at the Westin (I seem to recall it was crowded but I didn't go in). I think the Westin does have a bakery and an espresso bar (is that the generation-x term for a coffee shop?) in addition to a restaurant/lounge/bar/gift-shop (each of which seemed nicer than at the Hilton). The guest rooms are much bigger and their bathrooms have TVs (for whatever that's worth). So, I don't think it would hurt to at least try the Westin once. Best Regards, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Fri Jan 5 05:36:26 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA18588; Fri, 5 Jan 1996 05:36:26 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA05312 for mpi-core-out; Fri, 5 Jan 1996 04:39:09 -0600 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id EAA05304 for ; Fri, 5 Jan 1996 04:38:59 -0600 Received: by felix.dircon.co.uk id AA21024 (5.67b/IDA-1.5 for ); Fri, 5 Jan 1996 10:38:53 GMT Message-Id: <199601051038.AA21024@felix.dircon.co.uk> Received: from gw4-163.pool.dircon.co.uk(194.73.168.163) by amnesiac via smap (V1.3) id sma021009; Fri Jan 5 10:38:37 1996 Received: by jim (5.x) id AA00576; Fri, 5 Jan 1996 09:42:40 GMT To: MaryDell Tholburn Cc: mpi-core@antares.mcs.anl.gov Subject: Re: MPI Spring Meeting In-Reply-To: Your message of "Thu, 04 Jan 1996 11:55:28 MST." <199601041855.LAA24189@wrangler.lanl.gov> Date: Fri, 05 Jan 1996 09:42:40 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > The O'Hare Hilton is very convenient for those of us who are traveling > to Chicago for the MPI Forum, I agree with Tony. I also travel (I think Bristol UK is further from Chicago than Los Alamos !), and I also find the O'Hare Hilton convenient. However be clear that I'm not proposing to move downtown, or to Argonne, I'd just like to try a different airport hotel. (With a free shuttle). While the bar may seem like a small issue, I actually think that the lack of a good place to relax and be able to schmooze the technical issues over a beer in the evening is hurting us. I'm willing to wait ten minutes for a shuttle at the end of a 16 hour journey for that. -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Fri Jan 5 10:31:04 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA20467; Fri, 5 Jan 1996 10:31:03 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA07779 for mpi-core-out; Fri, 5 Jan 1996 09:33:21 -0600 Received: from molly.vcpc.univie.ac.at (molly.vcpc.univie.ac.at [193.171.58.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA04891 for ; Fri, 5 Jan 1996 03:11:14 -0600 Received: from sec.vcpc.univie.ac.at (sec [193.171.58.21]) by molly.vcpc.univie.ac.at (8.7.3/8.7.3) with SMTP id KAA21131 for ; Fri, 5 Jan 1996 10:11:00 +0100 (MET) Received: by sec.vcpc.univie.ac.at (NX5.67e/NX3.0S) id AA01039; Fri, 5 Jan 96 10:16:17 +0100 Message-Id: <9601050916.AA01039@sec.vcpc.univie.ac.at> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3risc v118.3) Received: by NeXT.Mailer (1.118.3.RR) From: Helga Pfeifer - Secretary of VCPC Date: Fri, 5 Jan 96 10:16:17 +0100 To: mpi-core@mcs.anl.gov Subject: events in Vienna Reply-To: helga@vcpc.univie.ac.at Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Colleague, You have registered for one or more of the events from January 22 to 2 6 organised by the VCPC in Vienna, Austria. For those of you with access to the World Wide Web, you will find a number of tips and some useful information for your trip under http://www.vcpc.univie.ac.at/news/NewsAndEvents.html We will be sending out a summary of the most important items and the final programme to all who have registered shortly after January 10. In the meantime, we would like to make a few suggestions for your social programme. On the evening of Tuesday, January 23, there will be a concert in the Brahms Chamber of the famous MUSIKVEREIN (whose main hall you may know from the Vienna Philharmonic New Year's Concerts on TV), which will give you an impression of the rich Viennese musical life. The ALTENBERG TRIO is performing several classic piano works by Haydn, Brahms and Bjelik in a concert beginning at 19.30. Tickets are available for ATS 50, 140, 180, 210 and 290. If you decide to attend, give us your order as soon as possible and we can book seats for you. As an alternative, you may consider an evening at the Vienna State Opera. Credit card holders (AmEx, Diners, Visa, Eurocard, MasterCard, JCB Card) can order tickets directly by phoning +43-1-513-1-513 from Monday to Friday, 10 a.m. to 6 p.m. or Saturday/Sunday from 10 a.m. to 12 noon. The programme during the week you will be here is: 21/1 18.00 Un Ballo in Maschera 22/1 18.30 Manon 23/1 19.30 Manon (ballet) 24/1 18.00 Lohengrin 25/1 19.00 Un Ballo in Maschera 26/1 18.30 Manon 27/1 19.00 Die Zauberfloete (The Magic Flute) We will be organizing dinner with different kinds of Viennese fare for those who do not plan to go out to a concert; if you arrive before your event starts, or are staying on a few days longer you are welcome to join in throughout your stay. Please help us to plan this by indicating whether you will participate as soon as you are able. Monday evening Cellar restaurant in the City Centre Tuesday evening Heuriger (Viennese wine tavern) Wednesday evening Historical Viennese restaurant Thursday evening - to be determined =========================================================== ***** PLEASE RETURN THIS SECTION ONLY ***** Name of participant:_______________________________________ I wish to order _____ tickets for the Altenberg Trio on Tuesday evening at a price of _______. If there are no tickets left at this price, please reserve tickets at a price of ________. Note: These tickets MUST be paid for by you; we cannot return them. ____ persons will attend the dinner on Monday ____ Tuesday ____ Wednesday ____ Thursday evening =========================================================== Please return to course@vcpc.univie.ac.at From mpi-core-human@mcs.anl.gov Fri Jan 5 12:21:32 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA21947; Fri, 5 Jan 1996 12:21:31 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA10669 for mpi-core-out; Fri, 5 Jan 1996 11:23:47 -0600 Received: from molly.vcpc.univie.ac.at (molly.vcpc.univie.ac.at [193.171.58.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA10660 for ; Fri, 5 Jan 1996 11:23:35 -0600 Received: from beavis.vcpc.univie.ac.at (beavis [193.171.58.38]) by molly.vcpc.univie.ac.at (8.7.3/8.7.3) with ESMTP id SAA23520 for ; Fri, 5 Jan 1996 18:23:23 +0100 (MET) From: Ian Received: (from ian@localhost) by beavis.vcpc.univie.ac.at (8.7.1/8.7.1) id SAA23306 for mpi-core@mcs.anl.gov; Fri, 5 Jan 1996 18:23:47 +0100 (MET) Date: Fri, 5 Jan 1996 18:23:47 +0100 (MET) Message-Id: <199601051723.SAA23306@beavis.vcpc.univie.ac.at> To: mpi-core@mcs.anl.gov Subject: Who wants to talk at Vienna MPI open forum? X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear MPI people, as January 24th is rapidly approaching, I think it's about time we sorted out an agenda for the MPI open meeting in Vienna. (The MPIF meeting will be 25th-26th.) I think a reasonable format would be to use the morning for presentations by MPI committee members, giving an overview of the proposed MPI-2 standard, and the afternoon for feedback from end users and implementors. The workshop need not be exclusively devoted to MPI-2 issues, and we could even have a slot for short reports on applications if people feel it would be worthwhile. I suspect we may have some attendees who come just to find out about MPI in general. I guess that if any vendors are planning to attend, that could fit into the implentor slot. So how about: Morning: Presentations by MPI committee Afternoon: Feedback from MPI implementors and users (short) Presentations on implementions (short) Presentations on applications Open discussion Comments? I guess we will need presentations on each of the major sections of the draft for the morning - will the committee chairs all be coming, and if not, who else can stand in to talk about the relevant section? Also, if any of you would like to talk briefly about implementations or applications, could you please let me know, so I can get an idea of how much time there will be for each section. Thanks, Ian -- Ian Glendinning European Centre for Parallel Computing at Vienna (VCPC) ian@vcpc.univie.ac.at Liechtensteinstr. 22, A-1090 Vienna, Austria Tel: +43 1 310 939612 WWW: http://www.vcpc.univie.ac.at/~ian From mpi-core-human@mcs.anl.gov Sun Jan 7 23:50:59 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA29460; Sun, 7 Jan 1996 23:50:59 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA19931 for mpi-core-out; Sun, 7 Jan 1996 22:52:57 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA19923; Sun, 7 Jan 1996 22:52:48 -0600 Message-Id: <199601080452.WAA19923@antares.mcs.anl.gov> To: Ian cc: mpi-core@mcs.anl.gov Subject: Re: Who wants to talk at Vienna MPI open forum? In-reply-to: Your message of "Fri, 05 Jan 1996 18:31:45 +0100." <199601051731.SAA23309@beavis.vcpc.univie.ac.at> Date: Sun, 07 Jan 1996 22:52:48 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | I am currently working out an agenda for the MPI open forum, | to be held in Vienna on January 24th. My current proposal (which may change) | is as follows: | | Morning: Presentations by MPI committee on MPI-2 | | Afternoon: Feedback from MPI implementors and users | (short) Presentations on implementions | (short) Presentations on applications | Open discussion | | If any of you would like to allocate a short slot in the afternoon, to talk | about your requirements/opinions/implementations/applications (the above | categories are flexible and may change) please let me know, so I can get an | idea how much time to allow for each section. Thanks, I will be there that day, and can host the MPI-2 presentation in the morning. I can also cover the dynamic and/or misc chapters if necessary, although others may be available. Can the chapter captains let Ian and me know who all will be there? Either Bill Gropp or I can give a presentation on MPICH as an implementation in the afternoon. This can range anywhere from 5 minutes to 30 minutes, depending on your preferred schedule. You might also want to contact Marc Snir about an MPI-IO presentation in the afternoon. Perhaps Hans-Christian Hoppe would also like to talk briefly about PALLAS's plans for MPICH support. Ian, we all appreciate your efforts in organizing this. Regards, Rusty From mpi-core-human@mcs.anl.gov Mon Jan 8 14:35:01 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA13751; Mon, 8 Jan 1996 14:35:00 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA00270 for mpi-core-out; Mon, 8 Jan 1996 13:37:54 -0600 Received: from reggiano.cs.wisc.edu (reggiano.cs.wisc.edu [128.105.76.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA00262 for ; Mon, 8 Jan 1996 13:37:42 -0600 Received: (from lederman@localhost) by reggiano.cs.wisc.edu (8.6.12/8.6.12) id NAA01844; Mon, 8 Jan 1996 13:37:41 -0600 Date: Mon, 8 Jan 1996 13:37:41 -0600 Message-Id: <199601081937.NAA01844@reggiano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: Europeon MPI draft Cc: ian@vcpc.univie.ac.at Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The draft for the European MPI meeting is now available. It is split into two parts. The first is the main document, the second is the oompi appendix. You can see the release on my home page at http://www.cs.wics.edu/~lederman/lederman.html. There is also a link to the current sources. For those without the WEB you can get the info via anonymous ftp at ftp.mcs.anl.gov in pub/mpi/mpi2. See mpi2-report.ps.Z and mpi2-report-oompi.ps.Z. Please check this document over if you authored a chapter or made input to make sure it is correct. Send corrections to me. See you in Europe. Steve From mpi-core-human@mcs.anl.gov Tue Jan 9 17:30:14 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA19926; Tue, 9 Jan 1996 17:30:13 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA24127 for mpi-core-out; Tue, 9 Jan 1996 16:32:08 -0600 Received: from reggiano.cs.wisc.edu (reggiano.cs.wisc.edu [128.105.76.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA24119 for ; Tue, 9 Jan 1996 16:31:57 -0600 Received: (from lederman@localhost) by reggiano.cs.wisc.edu (8.6.12/8.6.12) id QAA06453; Tue, 9 Jan 1996 16:31:55 -0600 Date: Tue, 9 Jan 1996 16:31:55 -0600 Message-Id: <199601092231.QAA06453@reggiano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: Correction to Europeon MPI draft Cc: ian@vcpc.univie.ac.at Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In the original note I put wics instead of wisc. The text below is corrected. Sorry for the mistake. ---------------------------------------------------------------------- The draft for the European MPI meeting is now available. It is split into two parts. The first is the main document, the second is the oompi appendix. You can see the release on my home page at http://www.cs.wisc.edu/~lederman/lederman.html. There is also a link to the current sources. For those without the WEB you can get the info via anonymous ftp at ftp.mcs.anl.gov in pub/mpi/mpi2. See mpi2-report.ps.Z and mpi2-report-oompi.ps.Z. Please check this document over if you authored a chapter or made input to make sure it is correct. Send corrections to me. See you in Europe. From mpi-core-human@mcs.anl.gov Mon Jan 15 17:35:38 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA10696; Mon, 15 Jan 1996 17:35:37 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA00808 for mpi-core-out; Mon, 15 Jan 1996 16:37:17 -0600 Received: from molly.vcpc.univie.ac.at (molly.vcpc.univie.ac.at [193.171.58.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA00800 for ; Mon, 15 Jan 1996 16:37:04 -0600 Received: from beavis.vcpc.univie.ac.at (beavis [193.171.58.38]) by molly.vcpc.univie.ac.at (8.7.3/8.7.3) with ESMTP id XAA09128 for ; Mon, 15 Jan 1996 23:37:01 +0100 (MET) From: Ian Received: (from ian@localhost) by beavis.vcpc.univie.ac.at (8.7.1/8.7.1) id XAA03397 for mpi-core@mcs.anl.gov; Mon, 15 Jan 1996 23:37:28 +0100 (MET) Date: Mon, 15 Jan 1996 23:37:28 +0100 (MET) Message-Id: <199601152237.XAA03397@beavis.vcpc.univie.ac.at> To: mpi-core@mcs.anl.gov Subject: MPI-2 open meeting agenda X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear all, finally here is the preliminary agenda for the MPI open meeting next week. It's lacking the names of a couple of speakers in the applications section, and we need another name or two for the panel, but apart from that, I think it's pretty much fixed. Does anyone else volunteer to be on the panel? How many people do you think we ought to have on the panel? Should they all be MPI committee members, or should we throw in a few wild cards? Ian MPI-2 Open Meeting January 24, 1996, Vienna, Austria AGENDA Morning session: MPI-2 and Related Activities Chair: Rusty Lusk 09.00 Overview of the proposed MPI-2 standard Introduction - Rusty Lusk Dynamic Processes - Bill Saphir One-sided Communications - Marc Snir Extended Collective Operations - Steven Huss-Lederman Real-time, Embedded Extensions, and Profiles - Robert Babb External Interfaces - Steven Huss-Lederman Language Binding & Miscellaneous - Rusty Lusk 10.30 Coffee 11.00 Related activities MPI-IO - Marc Snir HPF-MPI - Ian Foster 11.45 MPI support and tools PALLAS MPI support and tools - Hans-Christian Hoppe BBN Totalview/MPI - James Cownie 12.30 Lunch Afternoon session: Implementations and Application Developers Chair: Hans-Christian Hoppe 14.00 Implementations LAM 6.0 - Nick Nevin MPICH - Rusty Lusk, Bill Gropp Experiences on porting MPICH on Myrinet and FM - Giulo Iannello 15.15 Feedback from application developers 3rd/4th level trigger farm for HERA-B - Rainer Kowallik HEP Monte-Carlo simulations - Rainer Kowallik PARMACS->MPI conversion of PAM-CRASH - Jan Clinckemaillie (Approximately four others - to be announced) 16.00 Coffee Panel session: Open Discussion Chair: Ian Glendinning 16.30 Panel: James Cownie, Steven Huss-Lederman, Rusty Lusk, ... (Others to be announced) -- Ian Glendinning European Centre for Parallel Computing at Vienna (VCPC) ian@vcpc.univie.ac.at Liechtensteinstr. 22, A-1090 Vienna, Austria Tel: +43 1 310 939612 WWW: http://www.vcpc.univie.ac.at/~ian From mpi-core-human@mcs.anl.gov Tue Jan 16 23:27:53 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA16128; Tue, 16 Jan 1996 23:27:52 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA24471 for mpi-core-out; Tue, 16 Jan 1996 22:29:51 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA24434; Tue, 16 Jan 1996 22:28:13 -0600 Message-Id: <199601170428.WAA24434@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Cc: tuecke@antares.mcs.anl.gov, itf@antares.mcs.anl.gov Subject: attendance at Vienna meeting Date: Tue, 16 Jan 1996 22:28:12 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear group, I have a preliminary list of registrants for the Vienna meeting next week. In order to help me draw up the agenda of the MPI Forum meeting on Thursday and Friday, could you tell me when you are arriving in Vienna and when you are leaving? me: arriving late Tuesday, departing early Saturday Thanks. Rusty From mpi-core-human@mcs.anl.gov Fri Jan 19 04:55:26 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA22882; Fri, 19 Jan 1996 04:55:25 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA09511 for mpi-core-out; Fri, 19 Jan 1996 03:57:46 -0600 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA09486; Fri, 19 Jan 1996 03:57:09 -0600 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id BAA00292; Fri, 19 Jan 1996 01:57:07 -0800 Date: Fri, 19 Jan 1996 01:57:07 -0800 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199601190957.BAA00292@win233.nas.nasa.gov> To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: F90 bindings discussion/proposal Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk After hemming and hawing about Fortran 90 bindings for some time, I've finally sat down and written an F90 bindings proposal/discussion. As you'll see, it's pretty minimal, consisting mostly of a list of things that cannot work correctly in F90 and which users must avoid. I don't see at the moment how MPI can do any better than this. While it doesn't solve a number of F90/MPI problems, the proposal has two positive points, which IMHO are all that we really need: 1. It guarantees that F90 will indeed work with MPI. 2. It places very little burden on implementors. [At various times I have made notes on F90/MPI issues, but I can't seem to find any of them now. So I've probably omitted something important. ] Bill Fortran 90 Bindings Proposal ---------------------------- An MPI implementation may provide a Fortran 90 (F90) interface. This section describes what the F90 interface should look like. \discussion{ There are two possible approaches to an F90 MPI interface. One would be a "full F90" interface which would make as much use as possible of F90 features. Another would be an "F77+" interface, nearly identical to the Fortran 77 interface, but with additional clarifications. We choose the second approach for several reasons: 1. the burden on implementors to provide an entirely separate interface would be large. In most environments where a Fortran 90 compiler is available, that compiler also serves as the Fortran 77 compiler, so Fortran 90 can make immediate use of the Fortran 77 interface without additional work by implementors. 2. While Fortran 90 is usually considered a richer language than Fortran 77, it provides no better access to "low level" machine features than does Fortran 77. Thus, where C can use a pointer, Fortran 77 must use an integer, and Fortran 90 provides nothing more useful than Fortran 77. In particular, Fortran 90 pointers are pointers to descriptors, not addresses of data, and are strongly typed. Therefore Fortran 90 contains few additional features that might help to simplify the necessarily clunky F77 interface. 3. Unfortunately, several "features" of Fortran 90 cause problems for MPI for which there is no simple solution. Foremost among these is the copy-in-copy-out semantics of Fortran, which make the use of MPI's "non-blocking" functions problematic. Unfortunately this is most likely to be a problem when using Fortran 90 array sections. Stricter type checking in Fortran 90 is incompatible with "choice" arguments in MPI. These technically violate the F77 standard as well, but Fortran 90 provides a mechanism for enforcing type checking which MPI must avoid. Fortran 90 derived types, because their actual structure is invisible to the user, may potentially pose a problem for MPI datatypes. For instance, a user has no way of knowing whether the information in a derived type is contiguous or even whether it appears in the same order it is declared. Fortran 90 implementations may provide subroutines with additional hidden arguments containing type or array descriptors. Hidden arguments describing character strings already pose a problem for MPI functions with choice arguments, and the situation only becomes worse in F90. 4. A significant component of a "full F90" interface would probably be the use of F90 derived types where the F77 interface is forced to use integers. The most important practical advantage of this would be automatic type checking. However, as mentioned above, type checking must be disabled for all functions with choice arguments - most of the calls in a typical MPI program - so little advantage is gained (for a lot of effort). } - Basic interface Fortran 90 uses the same procedure interface as F77. - Module MPI provides an F90 module called "mpi". This module is analogous to and replaces the include files mpi.h and mpif.h for the C and Fortran 77 interfaces. The module includes all of the information available in mpif.h plus interface blocks for all MPI functions which do not have choice arguments. - Restrictions on nonblocking operations MPI nonblocking functions cannot possibly work if a compiler makes copies of the subroutine arguments and copies them back when the subroutine returns. Technically, a Fortran 77 or Fortran 90 compiler is allowed to do this with any arguments. Fortunately it is absurdly inefficient in most cases and is not done by any Fortran 77 compiler. However, a Fortran 90 compiler is likely to copy arguments when array sections are passed to a subroutine (the only exception is when the array section is contiguous in memory, e.g. a(:, 2:5) or a(1:10, 7) ). Since it is impossible for MPI nonblocking functions to work if arguments are copied out when a subroutine returns, it is the user's responsibility not to use F90 constructs that may involve data copying when using nonblocking functions. MPI recommends that array sections never be passed to nonblocking MPI functions, and that MPI datatypes be used instead. There are also potential problems with nonblocking operations due to code reordering. For instance (as pointed out by Ian Foster) the following code fails if the compiler moves the third statement before the second: call MPI_Irecv(x, ..., request) ... call MPI_Wait(request) y = x C programs can avoid this problem with a "volatile" keyword in the declaration of x. (?). Fortran provides no portable mechanism to tell the compiler not to reorder. Again, it is the user's responsibility to make sure incorrect reordering does not happen. \discuss{ It would be nice if MPI could say something more helpful here, rather than put all the burden on the user, but I don't see how MPI can solve the problem, short of abolishing nonblocking functions. The best we can do may be to educate users. Note that some of these same issues already exist with F77 and C. For instance, you can't pass an automatic array to MPI_Isend and then return from the subroutine. It's not too much to expect users to understand that this can't work, so perhaps we can trust them to take the F90 restrictions in stride. } - Restrictions on use of other MPI routines that store addresses of user data. Copy in/out for array sections is most obviously a problem for the nonblocking routines. However, some other routines pose the same problem. In particular, the datatype constructors all store the address of user data. Therefore it is incorrect to call datatype constructors with array sections. [ are there other types of routines that store addresses?] - Restrictions on derived datatypes and pointers Information contained in derived datatypes and pointers is not accessible outside of the compiler. Therefore it is not legal to pass an F90 pointer or derived datatype to an MPI routine. It is legal to pass individual elements of a derived type, but not the type itself. [ actually, pointers to arrays may appear as regular arrays (+hidden arg) in the child routine, so perhaps pointers are ok?] - Fortran 90 intrinsic data types Unfortunately Fortran 90 adds a lot of baggage in the area of intrinsic data types. In addition to real, double precision, logical, etc, you can now have stuff like: integer (kind=short) integer (selected_int_kind(5)) real (kind=high) real (selected_real_kind(8, 70) complex (kind=quad) character(kind = cyrillic) character(kind=kanji) The F90 handbook I have doesn't say anything about how many kinds must or may be supported. A simple naming scheme becomes very difficult. Presumably most implementations actually implement few intrinsic types. I suggest the following: A Fortran 90 interface should provide data types corresponding to every intrinsic type supported by the associated compiler. The names of these datatypes should be {INTRINSIC-NAME}_{EXTENSION}, i.e., REAL_SHORT REAL_8 COMPLEX_16 CHARACTER_CYRILLIC etc. The name of the extension is up to the MPI implementation. The implementation must document what MPI datatype name must be used for each type of data. \advice{ A portable MPI program written in F90, that uses datatypes outside of the Fortran 77 set, should not rely directly on MPI datatypes. Instead, it should use its own names for datatypes and initialize those internal datatypes in a module that can be customized for a particular platform. } \discuss{ This situation already exists to some extent with Fortran 77. DOUBLE PRECISION and REAL aren't portable. For instance, on most machine, a double precision number is 64 bits. On a Cray C90, it's 128 bits, but you really only want 64. So you compile with a special compiler flag, -dp which automatically uses 64-bit numbers for DOUBLE PRECISION. Unfortunately you can no longer use MPI_DOUBLE_PRECISION to send this data, because MPI has been precompiled so that DOUBLE_PRECISION means 128 bits. So you fool MPI by using a REAL type instead. The portable way to do this is to use your own internal name, say "dptype", and initialize that to MPI_REAL or MPI_DOUBLE_PRECISION depending on what platform you're on. } From mpi-core-human@mcs.anl.gov Sat Jan 27 07:07:15 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA13043; Sat, 27 Jan 1996 07:07:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id GAA18285 for mpi-core-out; Sat, 27 Jan 1996 06:05:46 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA18280 for ; Sat, 27 Jan 1996 06:05:38 -0600 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov> id EAA04395; Sat, 27 Jan 1996 04:05:35 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id EAA11014; Sat, 27 Jan 1996 04:05:34 -0800 From: "Eric Salo" Message-Id: <9601270405.ZM11012@mrjones.engr.sgi.com> Date: Sat, 27 Jan 1996 04:05:33 -0800 X-Mailer: Z-Mail (3.2.3 10apr95 MediaMail) To: mpi-core@mcs.anl.gov Subject: FORTRAN, constants, deallocation Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk We have this nice new wording in the 6/12/95 draft that says that constants are constant. I even voted for it. However, I now claim that it causes trouble. Take any of the MPI deallocation functions; I'll use MPI_Errhandler_free() as an example, but this also applies to the other opaque types. First, let's look at the following bit of FORTRAN: call MPI_Errhandler_free(MPI_ERRORS_RETURN, ierr) I think one can reasonably argue that the above is erroneous. In fact, it pretty much has to be because MPI_ERRORS_RETURN is now a constant, but the standard says that this function sets the errhandler argument to MPI_ERRHANDLER_NULL. Now consider the next example, which I claim is perfectly valid: call MPI_Errhandler_set(mycomm, MPI_ERRORS_RETURN, ierr) ... call MPI_Errhandler_get(mycomm, my_handler, ierr) call MPI_Errhandler_free(my_handler, ierr) One common optimization when implementing these opaque types is to use reference counts instead of mindless duplication. However, if this is done here then the above two examples become indistinguishable! I think that as a minimum first step, we need to explicitly state that passing MPI constants to the deallocation functions is erroneous. If we don't want to restrict implementations, we should also indicate that it is a form of error that might be fundamentally undetectable by MPI. We do have the escape hatch that says errors have undefined behavior, but that's not a very satisfying answer. One possible way around this for implementors is to just not use reference counts for the permanent types. However, now it becomes impossible for the MPI programmer to tell whether any two given handles refer to the same object. While there's nothing in the standard that says this should be possible, it seems reasonable to me to at least imagine that multiple handles to a common object might be identical! How are other implementations dealing with this issue? -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)390-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Tue Jan 30 11:00:28 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA14098; Tue, 30 Jan 1996 11:00:27 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA28369 for mpi-core-out; Tue, 30 Jan 1996 10:02:31 -0600 Received: from reggiano.cs.wisc.edu (reggiano.cs.wisc.edu [128.105.76.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA28363 for ; Tue, 30 Jan 1996 10:02:16 -0600 Received: (from lederman@localhost) by reggiano.cs.wisc.edu (8.6.12/8.6.12) id KAA01620; Tue, 30 Jan 1996 10:02:14 -0600 Date: Tue, 30 Jan 1996 10:02:14 -0600 Message-Id: <199601301602.KAA01620@reggiano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: status and cancel Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This note relates two topics discussed at the MPI Forum meeting in Vienna: Status and Cancel. Since they relate to several groups, I am sending it to the core group. The first is what to do with status. We have created new requests in several places in MPI-2 and use TEST/WAIT to see if they are done. What is put in status? In some cases (iput/iget, iconnect/iaccept) they should pass no info since the blocking version does not give info. This is easy. Others do need to pass back info. The choices are to reuse the status, create a new object(s), or new functions to use instead of TEST/WAIT. The vote to reuse status was: reuse: 19 new solution: 0 abstain: 2 Thus, we will reuse status. The question is how to do this. It is more difficult since 3 of the fields in status are not obaque. They are status, tag and error. We still want error so it can stay. One choice is to say that status and tag are undefined but an implementation cannot reuse the fields. The second choice was that the user cannot modify the values in status and tag so that the implementation could use this space to store new values. The vote was: implementations can reuse fields: 16 can't reuse: 2 Thus, they can be reused. Each chapter that has a new request must now define what information they wish to receive back. New functions should be proposed to decode this info held in the status. The second topic is canceling non-blocking operations. One tricky issue is that because multiple processes are involved, buffers on some processes may have been modified before the cancel occurs. How do you deal with this? The only hope seemed to be to warn users this could happen unless someone can come up with a better scheme. To be able to cancel, you must clean up the operation. The thinking is that if you issue a cancel on your process, MPI must remove your contribution from the collective operation. Thus, if you reissued the call on that process, the collective operation would complete based on the second call. Doing this is VERY hard. The committee wants each chapter author to write up how cancel would work for their non-blocking collective ops in this model. The feeling was that you can say you cannot cancel them if you can show it cannot be done. For example, Lloyd showed some of us that the hypercube algorithm for bcast cannot even be canceled for 2 processes. This is my recollection of the discussion. Let us know if you remember other details. Steve From mpi-core-human@mcs.anl.gov Tue Jan 30 16:22:08 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA03924; Tue, 30 Jan 1996 16:22:06 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA05960 for mpi-core-out; Tue, 30 Jan 1996 15:22:16 -0600 Received: from reggiano.cs.wisc.edu (reggiano.cs.wisc.edu [128.105.76.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA05953 for ; Tue, 30 Jan 1996 15:21:57 -0600 Received: (from lederman@localhost) by reggiano.cs.wisc.edu (8.6.12/8.6.12) id PAA03075; Tue, 30 Jan 1996 15:21:56 -0600 Date: Tue, 30 Jan 1996 15:21:56 -0600 Message-Id: <199601302121.PAA03075@reggiano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: deadline for March meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy, I hope everyone who went has recovered from the trip to Vienna. I know we just got back, but I wanted to give everyone plenty of notice about the deadline for the next meeting. I need all the input by Monday, March 4 at 6:00 AM. This is a hard deadline since I must get the document processed and then it has to be printed. As usual, it is best to get it in early so people have time to comment and read it before the meeting. Steve From mpi-core-human@mcs.anl.gov Thu Feb 8 11:08:13 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA14971; Thu, 8 Feb 1996 11:08:06 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA21949 for mpi-core-out; Thu, 8 Feb 1996 10:08:13 -0600 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA21937 for ; Thu, 8 Feb 1996 10:07:57 -0600 Received: by felix.dircon.co.uk id AA03585 (5.67b/IDA-1.5 for ); Thu, 8 Feb 1996 16:07:51 GMT Message-Id: <199602081607.AA03585@felix.dircon.co.uk> Received: from gw2-104.pool.dircon.co.uk(194.112.35.104) by amnesiac via smap (V1.3) id sma003577; Thu Feb 8 16:07:35 1996 Received: by jim (5.x) id AA02324; Thu, 8 Feb 1996 16:05:45 GMT To: mpi-io@nas.nasa.gov Cc: mpi-core@mcs.anl.gov Subject: Re: MPI and MPI-IO In-Reply-To: Your message of "Thu, 08 Feb 1996 07:27:26 PST." <199602081525.KAA17664@pogo.tc.cornell.edu> Date: Thu, 08 Feb 1996 16:05:44 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Sorry to re-cap so much, but I've also included mpi-core in the distribution, and not everyone on that list will have seen the context... Jerry Gerner said > Rajeev said,... > > >> From: Rusty Lusk > >> > >> | Has mpi forum adopted mpi-io as its io interface/system etc.??? > >> > >> The MPI Forum has not taken any position whatever on I/O, and is not > >> currently considering one. > >> > > > >I think users will be thankful if there is a well integrated standard > >for both message-passing and I/O. I remember reading the results of a > >survey on comp.parallel.mpi about what new features users would like > >to see in MPI, and an overwhelming majority said I/O. > > > >I think the MPI Forum is in a good position to provide such a thing. > >It would be great if they could incorporate MPI-IO, with whatever > >changes they feel necessary. > > > >In fact, some of the new features being considered in MPI-2, such as > >generalized test and wait, are actually useful for I/O. Then we also > >won't need to duplicate several MPI functions for MPI-IO, such as > >MPIO_INIT, MPIO_FINALIZE, MPIO_VERSION, MPIO_TEST, MPIO_WAIT, .... > > > >What do others feel about this? > >Your opinion is important. > > > > I feel strongly that there should be some coordination/cooperation > (if possible) between the groups on this issue! > > As someone who hears from (CTC) users about I/O being one of the > biggest "show-stoppers" re: large-scale (dare I say "massively) > scalable parallel computing on our SP2 -- along with the usual > "portability" concerns, I'd sure to like to see something which > addresses both kinds of concerns *and* brings to bear the efforts of > both(!) the MPI Forum and the MPI-IO group. > > BTW,... I'm working with MPI, trying to "do" traditional I/O, using > PIOFS, *and* (now) HPSS. Please, PLEASE, *PLEASE*!!! A single API, > my kingdom for a single API! :) Speaking *only* for myself, not BBN, not the MPI forum (though I guess I have attended > 75% of meetings in MPI-1 and MPI-2), my view is 1) The MPI forum doesn't have the expertise in I/O to know what the right answer is. 2) (A corollary of 1) The MPI-Forum can't judge whether "MPI-IO" is the right answer or not, so don't bet on the MPI Forum endorsing it, or any other I/O proposal. (The forum won't be swayed by the choice of name !) 3) If you really want a "standard" at the level of MPI (i.e. an informal standard done by the people who need it), then form yourself into a committee, persuade the right technical people from your machine vendors and the I/O community to attend, go to Dallas or Chicago O'Hare every six weeks, be open to all attendees, vote on proposals and go do it. (Or alternatively, do the same thing, but get the people to come to the MPI-Forum meetings, in which case you'll be able to change the views of the Forum and have it do what you want...) 4) Realise that the MPI-Forum can't change reality. Just because we made a good enough job of MPI-1 to get it used and adopted by vendors doesn't mean that we can apply a magic touch to other things and make them a success. 5) If MPI-IO is the right thing, then it should succeed without needing the endorsement of the MPI-Forum. Sorry if I sound negative, I get slightly perturbed by the attitude of people to MPI which is normally 1) It's too big. 2) I need this extra feature added to it. (Often something which is just physically impossible to mandate, like a high resolution low access time globally synchronised clock. I know it would be nice, but the MPI Forum saying that won't make the hardware to make it implementable appear on your workstation network... ) In summary : These things happen because people are prepared to put the work in to make them happen, so go do it. You don't need the MPI Forum to do it for you, you can do it yourself. -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Thu Feb 8 11:49:13 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA15787; Thu, 8 Feb 1996 11:49:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA23153 for mpi-core-out; Thu, 8 Feb 1996 10:50:07 -0600 Received: from capra.mcs.anl.gov (capra.mcs.anl.gov [140.221.6.98]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA23148; Thu, 8 Feb 1996 10:50:01 -0600 Date: Thu, 8 Feb 1996 10:49:59 -0600 Message-Id: <199602081649.KAA16211@capra.mcs.anl.gov> From: Rajeev Thakur To: jcownie@bbn.com CC: mpi-io@nas.nasa.gov, mpi-core@mcs.anl.gov In-reply-to: <199602081607.AA03585@felix.dircon.co.uk> (message from James Cownie on Thu, 08 Feb 1996 16:05:44 +0000) Subject: Re: MPI and MPI-IO Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Date: Thu, 08 Feb 1996 16:05:44 +0000 > From: James Cownie > > 1) The MPI forum doesn't have the expertise in I/O to know what the > right answer is. I think that the I/O community would be interested in working with the MPI Forum to formulate a standard. I, for one, am certainly interested in it. > > 2) (A corollary of 1) The MPI-Forum can't judge whether "MPI-IO" is > the right answer or not, so don't bet on the MPI Forum endorsing > it, or any other I/O proposal. (The forum won't be swayed by the > choice of name !) The right answer will come out of discussions, just as other things have been discussed and agreed upon. MPI-IO is a good starting point. > 3) If you really want a "standard" at the level of MPI (i.e. an > informal standard done by the people who need it), then form > yourself into a committee, persuade the right technical people from > your machine vendors and the I/O community to attend, go to Dallas > or Chicago O'Hare every six weeks, be open to all attendees, vote > on proposals and go do it. > > (Or alternatively, do the same thing, but get the people to come to > the MPI-Forum meetings, in which case you'll be able to change the > views of the Forum and have it do what you want...) I am in favor of this. (meeting with the MPI-Forum). > 4) Realise that the MPI-Forum can't change reality. Just because we > made a good enough job of MPI-1 to get it used and adopted by > vendors doesn't mean that we can apply a magic touch to other > things and make them a success. I agree. But if the next version of MPI also includes I/O, it might be an even greater success (as many users indicate). > 5) If MPI-IO is the right thing, then it should succeed without > needing the endorsement of the MPI-Forum. Rather than endorsement, it is more a question of working together. We are finding that MPI-IO needs many of the MPI functions (e.g. test, wait, init, finalize), but we currently need to redefine them separately for MPI-IO. My whole position on this issue is that if users have expressed a need for a unified message-passing and I/O interface, we should put our differences aside and provide them with such an interface. I am willing to work on this and I welcome others to do so. Rajeev From mpi-core-human@mcs.anl.gov Tue Feb 13 17:33:47 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA27741; Tue, 13 Feb 1996 17:33:46 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA18533 for mpi-core-out; Tue, 13 Feb 1996 16:34:44 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA18527; Tue, 13 Feb 1996 16:34:32 -0600 Message-Id: <199602132234.QAA18527@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov cc: lusk@antares.mcs.anl.gov Subject: MPI Forum meeting, March 6-8 Date: Tue, 13 Feb 1996 16:34:31 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, We return now to the O'Hare Hilton for the next meeting, which will be on March 6-8. You should make your reservations now. A block of rooms has been reserved, which will be held until February 21, so please make your reservations with the Hotel before then. The phone number for the hotel is (312) 686-8000 (FAX (312) 601-2876). Mention the *MPI Forum* to get the special rate, which is $130.00 + tax per night. We will charge the usual $75 registration to take care of the meeting room, lunch on Thursday, and coffee, etc. The meeting will start at 1:30 on Wednesday to allow people to fly in that morning, and end at noon on Friday. Please let me know as soon as possible whether you are planning to come or not. I need a firm number a few days before the meeting. If you have any questions, don't hesitate to send me mail at lusk@mcs.anl.gov or call me at (708) 252-7852. Unless you read the above too quickly, you will have noticed that the room rate just jumped by about $40. The O'Hare Hilton is no longer willing to allow the government rate during the week. The above rate *is* still lower than the default of $160, and you need to mention the MPI Forum to get it. Regards, Rusty From mpi-core-human@mcs.anl.gov Wed Feb 28 09:42:14 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA27570; Wed, 28 Feb 1996 09:42:13 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA26234 for mpi-core-out; Wed, 28 Feb 1996 08:42:45 -0600 Received: from cs.sandia.gov (cs.sandia.gov [132.175.13.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA17460 for ; Tue, 27 Feb 1996 17:57:16 -0600 Received: from cheetah.cs.sandia.gov.noname by cs.sandia.gov with smtp (Smail3.1.28.1 #5) id m0trZG8-000XT0C; Tue, 27 Feb 96 16:56 MST Received: by cheetah.cs.sandia.gov.noname (4.1/SMI-4.1) id AA02867; Tue, 27 Feb 96 16:56:44 MST From: bright@cs.sandia.gov (Ron Brightwell) Message-Id: <9602272356.AA02867@cheetah.cs.sandia.gov.noname> Subject: feedback on a new MPI idea To: mpi-core@mcs.anl.gov Date: Tue, 27 Feb 96 16:56:43 MST X-Mailer: ELM [version 2.3 PL11] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Greetings: I wanted some feedback on an idea that I might propose to MPI-2. I was telling an application developer at SNL that the easiest way to avoid memcpy's and to get lowest latencies and highest bw (for large msgs under Puma on the Paragon), is to set up a protocol where he pre-posts all the recv's and then sends an "okay to send" msg to the sender. The sender waits for the "okay to send" msg, and then blasts away with ready sends. It occurred to me that if MPI_Rsend() is used with the idea that there are posted recv's on the other side, then the recv's that get posted on the other side are done so with the idea that the send's haven't been done yet. Therefore, there would be no need to search an unexpected msgs list looking for something that hasn't been sent. In both of the MPICH devices that I've done, a good portion of the time spent posting a recv is in looking through the unexpected queue. It seems like it would be good to have a recv function that has the semantics of: post this recv and don't bother looking for matches with unexpected msgs, because I know there aren't any here. This type of recv could still be matched with any type of send (sync,ready, etc.), and if there are matching unexpected msgs, blame it on programmer error (like ready sends w/o posted recvs). I can think of several reasons why the time to search an unexpected queue would be nominal, but I can also think of several reasons why it might not be. The protocol described above may also be used when the time comes to implement MPI-2's get/iget on top of point-to-point operations (post a recv and then send a request message). I don't know if this was ever discussed for MPI-1, but I can't think of a good reason not to have such a function. Any thoughts? -Ron From mpi-core-human@mcs.anl.gov Wed Feb 28 10:14:34 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA27813; Wed, 28 Feb 1996 10:14:33 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA27111 for mpi-core-out; Wed, 28 Feb 1996 09:17:25 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA27102 for ; Wed, 28 Feb 1996 09:17:16 -0600 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) id HAA28674; Wed, 28 Feb 1996 07:17:13 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) id HAA05660; Wed, 28 Feb 1996 07:17:12 -0800 From: "Eric Salo" Message-Id: <9602280717.ZM5658@mrjones.engr.sgi.com> Date: Wed, 28 Feb 1996 07:17:11 -0800 In-Reply-To: bright@cs.sandia.gov (Ron Brightwell) "feedback on a new MPI idea" (Feb 27, 4:56pm) References: <9602272356.AA02867@cheetah.cs.sandia.gov.noname> X-Mailer: Z-Mail (3.2.3 10apr95 MediaMail) To: bright@cs.sandia.gov (Ron Brightwell), mpi-core@mcs.anl.gov Subject: Re: feedback on a new MPI idea Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hmm...this idea is sort of interesting, but I wonder just how much benefit would result from it. In MPI_Rsend(), the potential performance benefits are perhaps a bit more straightforward; one can imagine implementations in which using MPI_Rsend() might prevent the buffering of a large message, or might eliminate a handshake between sender and receiver. Both of these operations could be very expensive, so eliminating them might actually be worth the pain of using MPI_Rsend() in the first place. For an "un-ready" receive, though, it seems to me that the possibilities for optimization are a bit more dubious because all that it saves you is a search through the posted receive queue. On the other hand, unlike MPI_Rsend(), it seems likely that a large number of implementations might be able to implement this new call in a way that actually does some good! Do you have any hard numbers for how much time an application might expect to save by using this call in your MPI implementation(s)? -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Wed Feb 28 10:37:54 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA27984; Wed, 28 Feb 1996 10:37:53 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA27689 for mpi-core-out; Wed, 28 Feb 1996 09:40:39 -0600 Received: from mbunix.mitre.org (mbunix.mitre.org [129.83.20.100]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA27572 for ; Wed, 28 Feb 1996 09:38:16 -0600 Received: from tgate4.mitre.org (tgate4.mitre.org [129.83.20.28]) by mbunix.mitre.org (8.6.10/8.6.9) with SMTP id KAA19631; Wed, 28 Feb 1996 10:38:11 -0500 Received: from [129.83.20.44:4304] by tgate3 with SMTP id A745831 ; Wed, 28 Feb 96 10:34:19 EDT Received: by mail11; (5.65/1.1.8.2/22Jun94-0628PM) id AA10527; Wed, 28 Feb 1996 10:34:32 -0500 Subject: RE: feedback on a new MPI idea From: arkady@mail11.mitre.org (Arkady Kanevsky) To: mpi-core@mcs.anl.gov, bright@cs.sandia.gov (Ron Brightwell) Message-Id: <960228103427.29267@mail11.mitre.org.0> Date: Wed, 28 Feb 96 10:34:32 -0500 X-Mailer: MAILworks 1.7-A-1 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Ron, Under your proposal the sender is still need to receive the message that the receiver is ready and posted the appropriate receive to reduce memory copy. I can not speak for the general MPI users ... but I will speak (with Tony's permission) about one of the real-time extensions. It is based upon the time-driven approach. Both the sender and receiver apriori agree (written in the code) when they will send/receive messages (the same is true for collective operations). The format of the send is slightly different then now. We just add the time when the sent starts. The receiver guarantee that by that time the receive is posted (actually the receive is not really needed, the MPI server provider just need to know the address where to put an incomming message). The receiver application nows at what time it is free to access the received message memory and when this memory is under MPI server control (the same is true for the sender). This removes the need of any handshaking between sender and receiver and between application and MPI server, thus allowing higher application-to-application throughput, and lower latencies. The main advantage for real-time applications is that this approach allows application to "schedule" the communication media to ensure timely message deliveries and reduce message collision (delays) on a backplane. This is just a sketch... The approach may not be for general use but for a specific niche... Arkady > >Greetings: > >I wanted some feedback on an idea that I might >propose to MPI-2. > >I was telling an application developer at SNL >that the easiest way to avoid memcpy's and to >get lowest latencies and highest bw (for large >msgs under Puma on the Paragon), is to set up a >protocol where he pre-posts all the recv's and >then sends an "okay to send" msg to the sender. >The sender waits for the "okay to send" msg, and >then blasts away with ready sends. > >It occurred to me that if MPI_Rsend() is used with >the idea that there are posted recv's on the other >side, then the recv's that get posted on the other >side are done so with the idea that the send's >haven't been done yet. Therefore, there would be >no need to search an unexpected msgs list looking >for something that hasn't been sent. In both of >the MPICH devices that I've done, a good portion >of the time spent posting a recv is in looking >through the unexpected queue. It seems like it >would be good to have a recv function that has >the semantics of: post this recv and don't >bother looking for matches with unexpected msgs, >because I know there aren't any here. This >type of recv could still be matched with any >type of send (sync,ready, etc.), and if there >are matching unexpected msgs, blame it on >programmer error (like ready sends w/o posted >recvs). > >I can think of several reasons why the time to >search an unexpected queue would be nominal, but >I can also think of several reasons why it might >not be. > >The protocol described above may also be used >when the time comes to implement MPI-2's get/iget >on top of point-to-point operations (post a recv >and then send a request message). > >I don't know if this was ever discussed for MPI-1, >but I can't think of a good reason not to have such >a function. > >Any thoughts? > >-Ron > > > From mpi-core-human@mcs.anl.gov Wed Feb 28 12:19:33 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA28920; Wed, 28 Feb 1996 12:19:32 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA00406 for mpi-core-out; Wed, 28 Feb 1996 11:21:52 -0600 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA00394 for ; Wed, 28 Feb 1996 11:21:31 -0600 Received: from jim (gw2-109.pool.dircon.co.uk) by felix.dircon.co.uk with SMTP id AA26191 (5.67b/IDA-1.5 for ); Wed, 28 Feb 1996 17:20:57 GMT Message-Id: <199602281720.AA26191@felix.dircon.co.uk> Received: by jim (5.x) id AA07915; Wed, 28 Feb 1996 17:19:13 GMT To: bright@cs.sandia.gov (Ron Brightwell) Cc: mpi-core@mcs.anl.gov Subject: Re: feedback on a new MPI idea In-Reply-To: Your message of "Tue, 27 Feb 1996 16:56:43 MST." <9602272356.AA02867@cheetah.cs.sandia.gov.noname> Date: Wed, 28 Feb 1996 17:19:13 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I don't know if this was ever discussed for MPI-1, > but I can't think of a good reason not to have such > a function. > > Any thoughts? Well, sorry to be brutal, but I think this is really horrible ! Here are some of the reasons 1) You are inventing a new receive mode so that you don't have to search the receive Q, but at the same time you are saying that the Q ought to be empty. How long can it take to search an empty Q ? (OK, I know, I'm being a bit unfair, the Q may have other non-matching messages in it, but how many are there likely to be ? In any high performance application the answer *must* be very few, since otherwise messages are being buffered ==> low performance !) 2) There are already enough receives, without having two more. 3) I never did like ready send, but it kept getting voted through after a discussion which went "It's worth 80% performance improvement on the delta, and you can ignore it." (or some other large number), followed by a vote something like For : 5 Against : 4 Abstain : 15 Adding another version of something akin to ready send (again because it gets x% performance) seems like a bad thing to me. 4) This could *really* confuse users ! 5) If you're really that concerned about the time to search the pending Q, then you can always hash it. (This adds time inserting entries, but speeds up the search. The balance will depend on the length of the Q and the number of matching/non-matching lookups. Personally I don't believe it's worthwhile, because I believe that in a high performance application the Q is always short/empty anyway [see above]). -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Wed Feb 28 14:56:04 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA01214; Wed, 28 Feb 1996 14:55:52 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA04204 for mpi-core-out; Wed, 28 Feb 1996 13:57:16 -0600 Received: from cs.sandia.gov (cs.sandia.gov [132.175.13.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id NAA04198 for ; Wed, 28 Feb 1996 13:57:06 -0600 Received: from jaguar.cs.sandia.gov by cs.sandia.gov with smtp (Smail3.1.28.1 #5) id m0trrzG-000XSxC; Wed, 28 Feb 96 12:56 MST Received: by jaguar.cs.sandia.gov (Smail3.1.28.1 #3) id m0trrzF-000bLSC; Wed, 28 Feb 96 12:56 MST Message-Id: From: shuler@cs.sandia.gov (Lance Shuler) Subject: Re: feedback on a new MPI idea To: mpi-core@mcs.anl.gov Date: Wed, 28 Feb 1996 12:56:33 -0700 (MST) In-Reply-To: <199602281720.AA26191@felix.dircon.co.uk> from "James Cownie" at Feb 28, 96 05:19:13 pm X-Mailer: ELM [version 2.4 PL23] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 4392 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I think that the simplest calls to implement are the ready-send and the ready-recv. - The first knows the receive is posted and so immediately sends. - The second knows the message hasn't arrived and so immediately posts. Advantages: - No extra buffering, hashing, searching, what if'ing. - Simple (No extra buffering, hashing, searching, what if'ing). - Fast (No extra buffering, hashing, searching, what if'ing). - Predictable (No extra buffering, hashing, searching, what if'ing). These are ideal for Arkady's real-time needs because: - Fast (No extra buffering, hashing, searching, what if'ing.) - Predictable (No extra buffering, hashing, searching, what if'ing.) - He can avoid any handshaking by sender and receiver, since both sender/receiver agree when they are going to send and receive. (The ready-to-send message is not required.) These routines support "smart" protocols that implicitly know something about the state of communication, including one-sided communication and various collective operations: - MPI_Get can be implemented with these routines, since it knows the "reply" hasn't arrived yet. (The one-sided communications measure performance in microseconds, where even the checks of empty or hashed unexpected msg queues add up.) - Implementations of MPI_Barrier, MPI_Allreduce, and MPI_Reducescatter could all benefit from using these routines since they also have some smarts about the state of communications. The discussion above addresses "short message" latency issues, since this is where the costs are noticeable. However, these routines also form the basis for bulk data transfers, where memcpy's can be avoided and where there is a great deal of knowledge about the flow of data. The arguments above support viewing read-send and ready-irecv as simple message passing primitives rather than "extra" unnecessary MPI functions. I would think that MPI would choose to embrace primitives that make themselves apparent, for that is it's task (already, MPI has perhaps reluctently seen the value of including MPI_Rsend). While I agree that adding too many functions should be discrouraged, I think it is even more of a mistake to disregard primitives for this reason. Lance James Cownie wrote: > > Here are some of the reasons > > 1) You are inventing a new receive mode so that you don't have to > search the receive Q, but at the same time you are saying that the Q > ought to be empty. > How long can it take to search an empty Q ? > > (OK, I know, I'm being a bit unfair, the Q may have other > non-matching messages in it, but how many are there likely to be ? > In any high performance application the answer *must* be very few, > since otherwise messages are being buffered ==> low performance !) > > 2) There are already enough receives, without having two more. > > 3) I never did like ready send, but it kept getting voted through > after a discussion which went "It's worth 80% performance > improvement on the delta, and you can ignore it." (or some other > large number), followed by a vote something like > For : 5 > Against : 4 > Abstain : 15 > > Adding another version of something akin to ready send (again > because it gets x% performance) seems like a bad thing to me. > > 4) This could *really* confuse users ! > > 5) If you're really that concerned about the time to search the > pending Q, then you can always hash it. (This adds time inserting > entries, but speeds up the search. The balance will depend on the > length of the Q and the number of matching/non-matching > lookups. Personally I don't believe it's worthwhile, because I > believe that in a high performance application the Q is always > short/empty anyway [see above]). > > -- Jim > > James Cownie > BBN UK Ltd > Phone : +44 117 9071438 > E-Mail: jcownie@bbn.com > > > > =========================================================================== Lance Shuler W: (505) 844-2159 Parallel Computing Sciences Department, 9223 Fax: (505) 845-7442 Sandia National Labs Albuquerque, NM 87185-1109 shuler@cs.sandia.gov =========================================================================== From mpi-core-human@mcs.anl.gov Wed Feb 28 16:58:26 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA03525; Wed, 28 Feb 1996 16:58:25 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA06782 for mpi-core-out; Wed, 28 Feb 1996 16:01:07 -0600 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA06777 for ; Wed, 28 Feb 1996 16:00:58 -0600 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id OAA10880; Wed, 28 Feb 1996 14:00:56 -0800 Date: Wed, 28 Feb 1996 14:00:56 -0800 Message-Id: <199602282200.OAA10880@win233.nas.nasa.gov> From: "William C. Saphir" To: shuler@cs.sandia.gov, bright@cs.sandia.gov Cc: mpi-core@mcs.anl.gov Subject: Re: feedback on a new MPI idea Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Feb 27, 4:56pm, Ron Brightwell wrote: > Subject: feedback on a new MPI idea > > Greetings: > > I wanted some feedback on an idea that I might > propose to MPI-2. I second Jim's comments that this would not be good for MPI-2. > > I was telling an application developer at SNL > that the easiest way to avoid memcpy's and to > get lowest latencies and highest bw (for large > msgs under Puma on the Paragon), is to set up a > protocol where he pre-posts all the recv's and > then sends an "okay to send" msg to the sender. > The sender waits for the "okay to send" msg, and > then blasts away with ready sends. I haven't worked on a paragon in a while, and my experience is not with Puma, so this may be a bit out of date, but... The necessity of sending "ok to send" messages on the paragon is basically due to a, uh, non-optimal design of the underlying semantics and protocols. csend() (and isend()!) has always behaved like a buffered send in MPI -- it buffers its data (on the receiving end) if there is not a matching receive. This means that head-to-head sends work and you don't have to worry about deadlock. This is a terrible protocol for MPI_Send(), which is generally implemented with an eager protocol for short messages and a rendezvous protocol for long messages. The dual-protocol gives you low latency for short messages and unbuffered communication for long messages, with a small fuzzy area in the middle. The paragon protocol almost guarantees buffering unless the application resorts to gross hacks like the "ok-to-send + ready send" above (equivalent to but slower than to the rendezvous protocol which should be done internally by MPI). Buffering is evil because the i860 has such poor memory bandwidth. (memory-to-memory copies used to go at 4.8 MB/s, and sometimes still do!). IMO MPI ready send was a mistake because its main use appears to be to get around deficiencies in the underlying MPI implementation (as above). An MPI implementation that relies on NX can't solve the buffering problem. Incidentally, last I checked, ready send on the paragon had exactly the same latency as a regular send. Perhaps this is different under Puma? > It occurred to me that if MPI_Rsend() is used with > the idea that there are posted recv's on the other > side, then the recv's that get posted on the other > side are done so with the idea that the send's > haven't been done yet. Therefore, there would be > no need to search an unexpected msgs list looking > for something that hasn't been sent. In both of > the MPICH devices that I've done, a good portion > of the time spent posting a recv is in looking > through the unexpected queue. It seems like it > would be good to have a recv function that has > the semantics of: post this recv and don't > bother looking for matches with unexpected msgs, > because I know there aren't any here. This > type of recv could still be matched with any > type of send (sync,ready, etc.), and if there > are matching unexpected msgs, blame it on > programmer error (like ready sends w/o posted > recvs). So the idea, I gather, is that the receiver posts an Rrecv() and then sends an "ok-to-send" message, and then the sender can do an Rsend()? This eliminates a couple of microseconds of searching the unexpected queue. Using Rrecv() without ok-to-send or some other synchronization would be extremely dangerous. Jim argues well that the time saved is likely to be very small. I argue additionally that the only time you want to do this is to avoid buffering when you have long messages (for short messages, the latency penalty from "ok to send" would swamp the latency gain in Rrecv()). And when you have long messages, a couple of microconds isn't important. Presumably the actual message transfer time (determined by bandwidth) is much longer or you wouldn't have resorted to the rendezvous protocol. There is of course a fuzzy area in the middle but the effect on total application performance has got to be minimal. In any case, an implementation of MPI_Send() that uses short and long protocols, and doesn't have to worry about the ADI automatically buffering long messages, is going to go faster than any user-level solution on top of new MPI routines. So I suggest that the solution here is to get an efficient native implementation of MPI_Send() on the paragon, so that application-level workarounds aren't necessary. Lance Shuler (shuler@cs.sandia.gov) writes: > These routines support "smart" protocols that implicitly know > something about the state of communication, including one-sided > communication and various collective operations: > [...] > The arguments above support viewing read-send and ready-irecv > as simple message passing primitives rather than "extra" > unnecessary MPI functions. I agree these could be used to implement some faster ops than you can do with MPI_Send(). However, I don't think they don't belong at the user level - they belong inside MPI, perhaps at the ADI level. They might be useful to implementors, but not to users, and implementors would want something more lightweight. Routines like this can only confuse users. I have yet to see a case where an application really needed to use rsend(), and I've seen several where rsend() was inappropriate or unsafe. We should not give to users the impression that is appropriate to implement their own protocols to do things that MPI should be doing itself - for instance, deciding when to use a rendezvous protocol. That's what MPI is for! Bill From mpi-core-human@mcs.anl.gov Wed Feb 28 17:26:38 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA03707; Wed, 28 Feb 1996 17:26:38 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA07625 for mpi-core-out; Wed, 28 Feb 1996 16:29:16 -0600 Received: from pnl.gov (relay.pnl.gov [130.20.20.36]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA07614 for ; Wed, 28 Feb 1996 16:28:59 -0600 Received: from snacker.emsl.pnl.gov by pnl.gov (PMDF V4.3-13 #6012) id <01I1R0A8KV1SA0UC8W@pnl.gov>; Wed, 28 Feb 1996 14:26:04 -0800 (PST) Received: by snacker.emsl.pnl.gov (4.1/SMI-4.1) id AA24160; Wed, 28 Feb 96 14:25:59 PST Date: Wed, 28 Feb 1996 14:25:59 -0800 From: j_nieplocha@pnl.gov Subject: Re: feedback on a new MPI idea In-reply-to: Your message of "Wed, 28 Feb 1996 14:00:56 PST." <199602282200.OAA10880@win233.nas.nasa.gov> To: "William C. Saphir" Cc: mpi-core@mcs.anl.gov Message-id: <9602282225.AA24160@snacker.emsl.pnl.gov> Content-transfer-encoding: 7BIT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In message <199602282200.OAA10880@win233.nas.nasa.gov>Bill Saphir writes: > ........... Buffering is evil > because the i860 has such poor memory bandwidth. (memory-to-memory > copies used to go at 4.8 MB/s, and sometimes still do!). IMO MPI > ready send was a mistake because its main use appears to be to get > around deficiencies in the underlying MPI implementation (as > above). An MPI implementation that relies on NX can't solve the > buffering problem. > I am not arguing with your point about the NX protocols; however, your estimate of the i860 memory bandwidth is a bit too pessimistic. People get around 90MB/s when copying alligned data on Paragons. Jarek Nieplocha From mpi-core-human@mcs.anl.gov Wed Feb 28 22:42:59 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA05619; Wed, 28 Feb 1996 22:42:59 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA10168 for mpi-core-out; Wed, 28 Feb 1996 19:15:07 -0600 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA10163 for ; Wed, 28 Feb 1996 19:15:01 -0600 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id RAA11321; Wed, 28 Feb 1996 17:14:53 -0800 From: "William C. Saphir" Message-Id: <9602281714.ZM11319@win233.nas.nasa.gov> Date: Wed, 28 Feb 1996 17:14:53 -0800 In-Reply-To: j_nieplocha@pnl.gov "Re: feedback on a new MPI idea" (Feb 28, 2:25pm) References: <9602282225.AA24160@snacker.emsl.pnl.gov> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: j_nieplocha@pnl.gov Subject: Re: feedback on a new MPI idea Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Feb 28, 2:25pm, j_nieplocha@pnl.gov wrote: > Subject: Re: feedback on a new MPI idea > I am not arguing with your point about the NX protocols; however, your estimate > of the i860 memory bandwidth is a bit too pessimistic. People get around 90MB/s > when copying alligned data on Paragons. >-- End of excerpt from j_nieplocha@pnl.gov At this point the discussion isn't really about MPI, but it is an interesting side point. I get about 70 MB/s for aligned and blocked data, which is close enough to your 90 MB/s for government work. However, the memcpy() implementation is so stupid that *any* deviation from 8-byte alignment and 8-byte blocking substantially reduces performance, even if the copy can still be done almost entirely with quad load and stores. For instance, copying 1MB+1byte goes at 5 MB/s even for aligned buffers! In the first few releases of the paragon OS, *all* copies went at 5MB/s. Here is some data from today: 8-byte aligned, 8-byte blocked: 67.36 MB/s src&dest 4-byte aligned: 22.44 MB/s src&dest 2-byte aligned: 8.99 MB/s src&dest 1-byte aligned: 4.85 MB/s src&dest 8-byte aligned, not 8-byte blocked: 4.86 MB/s (this last one is 1MB + 1 byte, aligned). There is no reason why all of these can't run at 67MB/s. Of course there are tradeoffs with latency and analyzing the alignment/blocking of your buffers. However, it is the case that most scientific computing sends around blocks of 64-bit doubles, so in that case everything is aligned and blocked and you get the maximum performance. I agree that is the usual situation. In any case, as someone (the folks at Sandia?) pointed out long ago, the fastest way to copy data on the paragon is to send a message (unbuffered!) to yourself, letting the DMA device, not the processor, do the work. Bill ps. Here's the code that produced the above: #include #include #define BUFSIZE (1024*1024) void init(double *buf, int n); void testit(char *msg, int srcstart, int deststart, int len); char inbuf[BUFSIZE+8], outbuf[BUFSIZE+8]; main() { testit(NULL, 0, 0, BUFSIZE); testit("8-byte aligned, 8-byte blocked", 0, 0, BUFSIZE); testit("src&dest 4-byte aligned", 4, 4, BUFSIZE); testit("src&dest 2-byte aligned", 2, 2, BUFSIZE); testit("src&dest 1-byte aligned", 1, 1, BUFSIZE); testit("src&dest 8-byte aligned, not 8-byte blocked", 0, 0, BUFSIZE+1); exit(0); } void testit(char *msg, int srcstart, int deststart, int len) { int i; double t1, t2, rate; for (i = srcstart; i < srcstart+len; i++) { inbuf[i] = 'a'; } for (i = deststart; i < deststart+len; i++) { outbuf[i] = 'a'; } t1 = dclock(); memcpy(&inbuf[srcstart], &outbuf[deststart], len); t2 = dclock(); rate = 1.0e-6*len/(t2-t1); if (msg != NULL) printf("%s: %.2f MB/s\n", msg, rate); } From mpi-core-human@mcs.anl.gov Fri Mar 1 15:50:48 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA25367; Fri, 1 Mar 1996 15:50:47 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA24695 for mpi-core-out; Fri, 1 Mar 1996 14:50:41 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA24689; Fri, 1 Mar 1996 14:50:29 -0600 Message-Id: <199603012050.OAA24689@antares.mcs.anl.gov> Cc: lusk@antares.mcs.anl.gov To: mpi-core@antares.mcs.anl.gov Subject: Head count for next week's meeting Date: Fri, 01 Mar 1996 14:50:27 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Monday morning I need to give the Hotel the final head count for next week's MPI Forum meeting. So far the following people have told me they are coming: Joel Clarke MaryDell Tholburn Andrew Lumsdaine Jeff Squyres Bill Gropp me Steve Huss-Lederman Marc Snir Rolf Rabenseifner Dick Treumann Terry Dontje Steve Taylor Jerell Watts Bill Saphir Parkson Wong Ron Brightwell Leslie Hart Raja Daoud Koichi Konishi Greg Burns Laurie Costello Please let me know immediately if you are planning on coming but are not on this list. Thanks. Rusty From mpi-core-human@mcs.anl.gov Sun Mar 3 13:39:01 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA24989; Sun, 3 Mar 1996 13:39:00 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA21731 for mpi-core-out; Sun, 3 Mar 1996 12:41:06 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA21725; Sun, 3 Mar 1996 12:40:54 -0600 Message-Id: <199603031840.MAA21725@antares.mcs.anl.gov> Cc: lusk@antares.mcs.anl.gov To: mpi-core@antares.mcs.anl.gov Subject: chapter voting schedule Date: Sun, 03 Mar 1996 12:40:52 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At the Vienna meeting we discussed a schedule of how we might spread out the final votes on various chapters so that we a) get our work done in time for SuperComputing '96 b) don't have to do everything at once at the last meeting. For those who weren't there, here is the schedule we came up with. It tries to reflect roughly how close the various chapters are to convergence. "R" means "Get the chapter ready for voting: try to narrow the focus, use straw votes to choose among major branches, etc." "1" means first formal vote on chapter. "2" means second and final vote on chapter. This approach seemed to work fairly well in MPI-1. Here is the proposed schedule: Chapter: Dyn One-S Coll Ext RealT Bind Misc Meeting ------- Jan R Mar 1 R Apr 2 1 R R Jun 2 R 1 1 July 1 2 R R 2 Sep 2 1 1 Oct 2 2 Nov Supercomputing '96, draft publication, call for comments Feb Final Cleanup and publication ---------------------------------------------------------------------- So please start feeling the heat. The discussions are great, but we now have to begin reaching conclusions. Regards, Rusty From mpi-core-human@mcs.anl.gov Mon Mar 4 12:50:17 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA12696; Mon, 4 Mar 1996 12:50:16 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA04488 for mpi-core-out; Mon, 4 Mar 1996 11:52:03 -0600 Received: from reggiano.cs.wisc.edu (reggiano.cs.wisc.edu [128.105.76.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA04470 for ; Mon, 4 Mar 1996 11:51:36 -0600 Received: (from lederman@localhost) by reggiano.cs.wisc.edu (8.6.12/8.6.12) id LAA16126; Mon, 4 Mar 1996 11:51:35 -0600 Date: Mon, 4 Mar 1996 11:51:35 -0600 Message-Id: <199603041751.LAA16126@reggiano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: MPI-2 document for meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk An electronic versionn of the latest MPI-2 document is now available. It can be gotten from http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z Sources are available in the same directory. Hard copy of this document will be available at the start of the MPI Forum meeting on Wednesday. The collective, binding, and real time chapters have not been updated. New versions may be available before (or at) the meeting. Steve From mpi-core-human@mcs.anl.gov Fri Mar 8 15:28:18 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA01257; Fri, 8 Mar 1996 15:28:17 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA19140 for mpi-core-out; Fri, 8 Mar 1996 14:28:08 -0600 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA19129 for ; Fri, 8 Mar 1996 14:28:00 -0600 Message-Id: <199603082028.OAA19129@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Minutes of Jan/March meetings Date: Fri, 08 Mar 1996 14:27:59 -0600 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk are now available at http://www.mcs.anl.gov/mpi/mpi2/mpi2.html . Bill From mpi-core-human@mcs.anl.gov Wed Mar 13 17:39:44 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA10758; Wed, 13 Mar 1996 17:39:43 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA15198 for mpi-core-out; Wed, 13 Mar 1996 16:41:39 -0600 Received: from atc.boeing.com (atc.boeing.com [130.42.28.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA15123 for ; Wed, 13 Mar 1996 16:39:08 -0600 Received: by atc.boeing.com (5.65/splinter.boeing.com) id AA21287; Wed, 13 Mar 1996 14:41:55 -0800 Received: from fury.sdc.cs.boeing.com by splinter.boeing.com with ESMTP (1.37.109.16/16.2) id AA179936681; Wed, 13 Mar 1996 14:38:01 -0800 Received: from stormy by sdc.cs.boeing.com (SMI-8.6/SMI-SVR4) id OAA08881; Wed, 13 Mar 1996 14:37:34 -0800 From: psgdn01@sdc.cs.boeing.com (Giles Nelson) Received: by stormy (SMI-8.6/Client-1.0) id OAA07680; Wed, 13 Mar 1996 14:37:31 -0800 Date: Wed, 13 Mar 1996 14:37:31 -0800 Message-Id: <199603132237.OAA07680@stormy> To: mpi-core@mcs.anl.gov Subject: IPPS96; MPI Birds of a Feather Cc: psgdn01@sdc.cs.boeing.com X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hello MPI developers, Per a suggestion from Ian, I am forwarding to you a note that I sent to him and Bill Gropp. This concerns having a Birds of a Feather get together at IPPS96 next month. This is something that myself and Boeing could benefit from. Please advise. Giles Nelson (206) 865-2215 psgdn01@sdc.cs.boeing.com ----- Begin Included Message ----- >From psgdn01 Tue Mar 12 16:08 PST 1996 From: psgdn01 (Giles Nelson) Date: Tue, 12 Mar 1996 16:08:47 -0800 To: gropp@mcs.anl.gov, itf@mcs.anl.gov Subject: IPPS96; MPI Birds of a Feather Cc: psgdn01, rfb, qarab01, rdl, suresh@kbct.bems.boeing.com, t$car@kbct.bems.boeing.com Subject; Having a MPI Birds Of a Feather (BOF) meeting at IPPS 96. ------------------------------------------------------------------ Hello Mr Foster and Mr Gropp, I will be attending IPPS96 next month and am interested in the possibility of having a little get together of those attendees and MPI developers who wish to share their MPI experiences for the benefit of all. I am the person doing production support of MPI at Boeing and I could like to learn from other companies and organizations who might be further along that path then we are here at Boeing. Some of the possible topics for this BOF could be; - experiences and suggestions doing PVM to MPI migrations - support issues relating to MPI - experiences doing inhouse teaching of MPI - MPI and Queuing products; i.e. DQS, LSF, etc. - experiences using MPI in a heterogeneous environment - using vendor supplied versions of MPI - experiences mixing supercomputers and workstations in a parallel job - convincing skeptical application programmers and management on the value of parallelism in general and MPI specifically. I regret that I will not be able to take the MPI tutorial being given at IPPS96. It conflicts with the joint session that the Heterogeneous and Scheduling workshops are doing on Tuesday morning. I am also in Cluster Computing support here and feel that I need to attend this discussion. Although I have helped develop and give two workshops here at Boeing on MPI, I would have loved to attend your tutorial also to see your methods and to interact with the other attendees. Please advise me on the possibility of having a MPI BOF. Giles Nelson (206) 865-2215 ph, 541-1702 pg Boeing Information & Support Services psgdn01@sdc.cs.boeing.com P.O. Box 24346, MS 7A-26 Cluster Computing and parallel Seattle, WA 98124-0346 programming support "One must learn to see with their ears, and hear with their eyes" - Old Chinese saying - Think about it. ----- End Included Message ----- From mpi-core-human@mcs.anl.gov Thu Mar 14 13:47:11 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA01222; Thu, 14 Mar 1996 13:47:09 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA10033 for mpi-core-out; Thu, 14 Mar 1996 12:49:01 -0600 Received: from win24.nas.nasa.gov (win24.nas.nasa.gov [129.99.33.39]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA09949; Thu, 14 Mar 1996 12:45:19 -0600 Received: from localhost (nitzberg@localhost) by win24.nas.nasa.gov (8.6.12/NAS.6.1) with SMTP id KAA15979; Thu, 14 Mar 1996 10:45:04 -0800 Message-Id: <199603141845.KAA15979@win24.nas.nasa.gov> X-Authentication-Warning: win24.nas.nasa.gov: Host localhost didn't use HELO protocol To: mpi-comm@mcs.anl.gov, mpi-core@mcs.anl.gov Cc: mpi-io@nas.nasa.gov Subject: Re: MPI-IO Standards Process In-reply-to: Your message of "Thu, 14 Mar 1996 07:40:00 PST." Date: Thu, 14 Mar 1996 10:45:03 -0800 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Rusty, How do we pursue proposing the addition of an I/O chapter to the MPI2 forum? - bill Richard Frost writes: > > It appears from your last statement that the MPI-IO group has been more > interested (and driven by real needs) in developing an implementation than > defining a standard. > > If you still hope to be recognized as a chapter in MPI2, why not join the > standards effort now? There are 30 professionals from industry, academia, > and government labs participating in the MPI Forum. > > Rusty Lusk is maintaining a schedule of chapter discussions for the MPI > Forum and has publicly requested that the MPI-IO group join the Forum on > several occasions. > > Given the outcome of last week's MPI Forum meeting, it would appear that > the schedule could accomodate discussion of an MPI-IO chapter in the June, > July, and August meetings. > > I encourage you to pursue this opportunity. > > Richard Frost > SDSC From mpi-core-human@mcs.anl.gov Fri Mar 15 20:37:24 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA28642; Fri, 15 Mar 1996 20:37:23 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA08485 for mpi-core-out; Fri, 15 Mar 1996 19:39:36 -0600 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA08477 for ; Fri, 15 Mar 1996 19:39:29 -0600 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id RAA12811; Fri, 15 Mar 1996 17:39:26 -0800 From: "William C. Saphir" Message-Id: <9603151739.ZM12809@win233.nas.nasa.gov> Date: Fri, 15 Mar 1996 17:39:25 -0800 In-Reply-To: Bill Nitzberg "Re: MPI-IO Standards Process" (Mar 14, 10:45am) References: <199603141845.KAA15979@win24.nas.nasa.gov> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: mpi-io@nas.nasa.gov, mpi-core@mcs.anl.gov Subject: Re: MPI-IO Standards Process Cc: Bill Nitzberg , frost@sdsc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Mar 14, 10:45am, Bill Nitzberg wrote: > > How do we pursue proposing the addition of an I/O chapter > to the MPI2 forum? > > Richard Frost writes: > > > > It appears from your last statement that the MPI-IO group has been more > > interested (and driven by real needs) in developing an implementation than > > defining a standard. > > > > If you still hope to be recognized as a chapter in MPI2, why not join the > > standards effort now? There are 30 professionals from industry, academia, > > and government labs participating in the MPI Forum. I think what Bill was looking for here was a public statement from someone on the MPI Forum about the relationship between MPI and MPI-IO and a hypothetical description of what the process would be for including MPI-IO in MPI2. [I don't take the question to indicate a desire to include MPI-IO in MPI-2]. The issue of including MPI-IO in the MPI-2 standard keeps coming up every few months. People new to the process aren't aware of the history and people old to the process may want to forget it. There has been recent discussion on the MPI-IO list about this (http://lovelace.nas.nasa.gov/MPI-IO/). There has been recent discussion about this on the mpi-core mailing list as well (http://lovelace.nas.nasa.gov/MPI-2/mpi-core/017{7,8}.html). The Forum has no spokesperson. I speak here for myself, as someone who has attended many MPI-2 meetings. Everything below has been said before. I repeat it just for the purpose of continuing education, and to clarify the apparent contradiction that MPI-2 is an open process but that MPI-IO is unlikely to become part of it. I do not advocate any particular point of view on whether MPI-IO should be included in MPI-2. The mechanics of the process (and for any addition to MPI-2) would be: 0. Find an MPI-IO champion who would do the following 1. show up at the next meeting and convince the forum that the chapter should be included 2. organize/lead subcommittee discussions at the meeting and subsequent meetings 3. write a proposal, in the style of the MPI document, for an MPI-IO chapter in MPI-2. 4. send it to the appropriate MPI mailing lists and the MPI-2 editor 5. make an exceedingly large amount of progress in the next few months This is the process, as I understand it. There are few formal rules. There is no formal barrier to getting MPI-IO included in MPI-2. All of this supposes a willingness to open up MPI-IO for large group discussion, possibly letting it wind up as something very different from the current MPI-IO spec. >From a practical point of view it is very unlikely that MPI-IO could make its way into MPI-2, even if MPI-IO wanted to pursue this approach (this is a personal opinion). 1. The MPI Forum voted a long time ago not to put IO in MPI-2. This was, as I see it, with the understanding and concurrence of MPI-IO (I believe there were at least 2 MPI-IO folks at the meeting, and that there was no protest). The decision of the group would have to be reversed, and there don't appear to be many new facts. I have also seen no interest from people working on MPI-IO to get it recognized as an MPI-2 chapter. 2. The rest of MPI-2 is fairly far along in the process, at least as far as educating the group about all the issues. In the end it is getting everyone comfortable and familiar with all the issues that really takes time. There were many reasons why it was decided that it didn't make sense to include MPI-IO in MPI-2. These include: 1. MPI-IO is a research project. There is little in the way of "common practice" for the type of cooperative I/O MPI-IO does. 2. MPI was designed so that portable libraries could be built on top of it. MPI-IO is an example of such a library. (Though there are a few technical issues, like access to datatypes, that MPI-2 *is* considering). MPI is not in the business of standardizing libraries. 3. Many folks on the MPI Forum aren't I/O experts. (The MPI-2 realtime committee is managing to operate in spite of this obstacle, however) 4. If MPI-IO is a good thing, it doesn't need MPI. Making MPI-IO part of MPI may not help users very much and it has the possibility to hurt them if it isn't the right thing. It is, practically and to some extent formally, extremely difficult (essentially impossible) to change MPI once things are settled upon. etc etc All that having been said, I believe there are plenty of communication channels open between MPI-2 and MPI-IO. Communication is a good thing and can't hurt. MPI-2 already is trying to interact with MPI-IO by providing mechanisms (e.g. exposing datatypes) that are necessary and not part of MPI-1. Certainly attendance and participation at MPI-2 meetings by MPI-IO folks is encouraged, and already happens. Bill From mpi-core-human@mcs.anl.gov Mon Apr 1 11:47:04 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA13700; Mon, 1 Apr 1996 11:47:01 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA12509 for mpi-core-out; Mon, 1 Apr 1996 10:47:45 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA12499 for ; Mon, 1 Apr 1996 10:47:35 -0600 Message-Id: <199604011647.KAA12499@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: April MPI Forum meeting Date: Mon, 01 Apr 1996 10:47:34 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, The next meeting of the MPI Forum will take place on Wed.- Fri., April 24-25. This time we are meeting in a ***NEW HOTEL***, the Ramada Hotel O'Hare, located near the airport. It is time to make your reservations. Register at the hotel by calling (708) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). When you arrive at the airport, make your way to Terminal 4 (follow signs). There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you have any problems, let me know. Regards, Rusty From mpi-core-human@mcs.anl.gov Mon Apr 1 13:13:43 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA14732; Mon, 1 Apr 1996 13:13:38 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA14558 for mpi-core-out; Mon, 1 Apr 1996 12:15:57 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA14553 for ; Mon, 1 Apr 1996 12:15:51 -0600 Message-Id: <199604011815.MAA14553@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: April MPI Forum meeting minor correction Date: Mon, 01 Apr 1996 12:15:51 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As several alert readers have pointed out, I made a small mistake in the dates for the MPI Forum meeting, which is Wed.-Fri. April 24-26. > >The next meeting of the MPI Forum will take place on Wed.- Fri., April 24-25. > ^ | should be a 6 ----------------------------+ Sorry for any confusion. Rusty From mpi-core-human@mcs.anl.gov Thu Apr 4 08:53:31 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA06059; Thu, 4 Apr 1996 08:53:30 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA14149 for mpi-core-out; Thu, 4 Apr 1996 07:54:54 -0600 Received: from reggiano.cs.wisc.edu (reggiano.cs.wisc.edu [128.105.76.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA14144 for ; Thu, 4 Apr 1996 07:54:47 -0600 Received: (from lederman@localhost) by reggiano.cs.wisc.edu (8.6.12/8.6.12) id HAA04673; Thu, 4 Apr 1996 07:54:46 -0600 Date: Thu, 4 Apr 1996 07:54:46 -0600 Message-Id: <199604041354.HAA04673@reggiano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: Deadline for input to next meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I must receive all input for the next draft of the MPI-2 document by 6:00 AM CST on Monday, April 22 (Yes, very early in the morning!). I would prefer to get input by Friday, but by Monday morning is ok if you know that your input will easily LaTeX into the docuemnt. This is a hard deadline since it must go to reproduction Monday morning. Steve From mpi-core-human@mcs.anl.gov Fri Apr 5 03:19:05 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA22781; Fri, 5 Apr 1996 03:18:59 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA01173 for mpi-core-out; Fri, 5 Apr 1996 02:19:44 -0600 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA01146; Fri, 5 Apr 1996 02:16:12 -0600 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id AAA10100; Fri, 5 Apr 1996 00:16:10 -0800 Date: Fri, 5 Apr 1996 00:16:10 -0800 Message-Id: <199604050816.AAA10100@win233.nas.nasa.gov> From: "William C. Saphir" To: mpi-dynamic@mcs.anl.gov Cc: mpi-core@mcs.anl.gov Subject: dynamic process chapter - part I Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, Here is a new version of part I of the dynamic process chapter. It contains the results of discussions at the last meeting, as well as discussions on the mailing list since then. Some of the highlights: - bindings for all functions (with the exception of the ispawn functions, for which the bindings are obvious (based on the blocking versions) and would take up an extra couple of pages). - (char *) instead of (void *) for info. I didn't put in both proposals mostly because having only one will probably generate more discussion. - The quote mechanism is based on comments from Dick and Rolf. - The MPI_SPAWN_SOFT issue has not been discussed much since the last meeting, so I have put in the minprocs/maxprocs version, based on the plurality vote at the last meeting. - For universe size, there are two proposals. The first is static. The second is dynamic. Comments? Thanks for Rolf for the 2nd universe size proposal and some detailed editing of the rest. I hope to get the other parts of this chapter done in the next few days. If you have opinions, please send them. I have not heard any comments on a new idea for MPI_NOTIFY sent out after the last meeting. Bill %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: main.dvi %%Pages: 8 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: /usr/local/bin/dvips -o main.ps main.dvi %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.04.05:0013 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (main.dvi) @start /Fa 6 118 df<07C00C20107020706000C000C000C00080008000C010C02060C0 3F000C0E7E8D0F>99 D<0300038003000000000000000000000000001C00240046004600 8C000C0018001800180031003100320032001C0009177F960C>105 D<383C0044C6004702004602008E06000C06000C06000C0C00180C00180C401818401818 80300880300F00120E7F8D15>110 D<07C00C20101020186018C018C018C01880308030 C060C0C061803E000D0E7E8D11>I<030003000600060006000600FFC00C000C000C0018 00180018001800300030803080310031001E000A147F930D>116 D<1C0200260600460600460600860C000C0C000C0C000C0C001818001818801818801838 800C5900078E00110E7F8D14>I E /Fb 1 81 df80 D E /Fc 4 62 df<03000700FF00070007000700070007000700070007000700070007000700070007 000700070007007FF00C157E9412>49 D<0F8030E040708030C038E03840380038007000 70006000C00180030006000C08080810183FF07FF0FFF00D157E9412>I<0FE030306018 701C701C001C00180038006007E000300018000C000E000EE00EE00EC00C401830300FE0 0F157F9412>I61 D E /Fd 6 111 df<70F8F8F87005057C840D>58 D<70F8FCFC74040404080810102040060E7C840D>I<000001C00000078000001E000000 78000001E00000078000000E00000038000000F0000003C000000F0000003C000000F000 0000F00000003C0000000F00000003C0000000F0000000380000000E0000000780000001 E0000000780000001E0000000780000001C01A1A7C9723>I<00E001E001E000C0000000 00000000000000000000000E00130023804380438043808700070007000E000E001C001C 001C20384038403840388019000E000B1F7E9E10>105 D<1E07C07C00231861860023A0 32030043C034030043803803804380380380870070070007007007000700700700070070 07000E00E00E000E00E00E000E00E00E000E00E01C101C01C01C201C01C038201C01C038 401C01C0184038038018801801800F0024147E9328>109 D<1E07802318C023A06043C0 704380704380708700E00700E00700E00700E00E01C00E01C00E01C00E03821C03841C07 041C07081C03083803101801E017147E931B>I E /Fe 8 118 df<78FCFCFCFC78000000 00000078FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000F80000F80000F80000F80000F800 007800007801803C01801C03000E0E0003F80011127E9115>99 D<1E003F003F003F003F 001E00000000000000000000000000FF00FF001F001F001F001F001F001F001F001F001F 001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8 F800F8F800F8F800F8F800F8F800F87800F07800F03C01E01E03C00F078001FC0015127F 9118>I<1FD830786018E018E018F000FF807FE07FF01FF807FC007CC01CC01CE01CE018 F830CFC00E127E9113>115 D117 D E /Ff 3 21 df0 D<03C00FF01FF83FFC7FFE 7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D<000000C0000003C000000F0000003C000000F0000003C00000070000001C0000007800 0001E00000078000001E00000078000000E0000000780000001E0000000780000001E000 0000780000001C0000000700000003C0000000F00000003C0000000F00000003C0000000 C0000000000000000000000000000000000000000000000000000000007FFFFF80FFFFFF C01A247C9C23>20 D E /Fg 10 58 df<1F00318060C04040C060C060C060C060C060C0 60C060C060404060C031801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C 000C000C000C000C000C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060 006000C00180030006000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000 C0008001800F00008000400060C060C060804060801F000B107F8F0F>I<030003000700 0F000B001300330023004300C300FFE003000300030003001FE00B107F8F0F>I<20803F 002C002000200020002F0030802040006000600060C06080C061801F000B107F8F0F>I< 0780184030C060C06000C000CF00F080E040C060C060C060406060C030801F000B107F8F 0F>I<40007FE07FC08080808001000200040004000C0008000800180018001800180018 000B117E900F>I<1F00318060C060C060C071803F000F00338061C0C060C060C0604040 60801F000B107F8F0F>I<1F00318060C0C040C060C060C06040E021E01E600060004060 C0608043003E000B107F8F0F>I E /Fh 17 119 df<70F8F8F0E005057B840E>46 D<00000200000006000000060000000E0000001E0000001E0000003F0000002F0000004F 0000004F0000008F0000010F0000010F0000020F0000020F0000040F00000C0F0000080F 0000100F0000100F0000200F80003FFF800040078000C007800080078001000780010007 800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C0004003C00 040078080800780800007808000078180000F0300000FFF00000F0300000F0300001E020 0001E0200001E0200001E0001003C0002003C0002003C0004003C0004007800080078001 8007800100078007000F001F00FFFFFE001F1F7D9E1F>69 D<00F1800389C00707800E03 801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C 40703C40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E0780 1C07003C0200780000780000780000F00000F00000F00000F00000F00000700100700200 30040018380007C00011147C9315>99 D<0000780003F800007000007000007000007000 00E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C03803C038038 0700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C4030 8C800F070015207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000 F000F0007000700170023804183807C010147C9315>I<00007800019C00033C00033C00 0718000700000700000E00000E00000E00000E00000E0001FFE0001C00001C00001C0000 1C0000380000380000380000380000380000700000700000700000700000700000700000 E00000E00000E00000E00000C00001C00001C0000180003180007B0000F300006600003C 00001629829F0E>I<00C001E001E001C000000000000000000000000000000E00330023 0043804300470087000E000E000E001C001C001C003840388030807080310033001C000B 1F7C9E0E>105 D<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E 00E01C01C01C01C01C01C01C03843803883803083807083803107003303001C016147C93 1A>110 D<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F0 03C0F003C0F003C0F00780F00700700F00700E0030180018700007C00013147C9317>I< 00F0400388C00705800E03801C03803C0380380700780700780700780700F00E00F00E00 F00E00F00E00F01C00F01C00703C00705C0030B8000F3800003800003800007000007000 00700000700000E00000E0000FFE00121D7C9315>113 D<1C1E00266100478380478780 4707804703008E00000E00000E00000E00001C00001C00001C00001C0000380000380000 38000038000070000030000011147C9313>I<00FC030206010C030C070C060C000F800F F007F803FC003E000E700EF00CF00CE008401020601F8010147D9313>I<018001C00380 03800380038007000700FFF007000E000E000E000E001C001C001C001C00380038003800 3820704070407080708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C0 4701C08703800E03800E03800E03801C07001C07001C07001C07101C0E20180E20180E20 1C1E200C264007C38014147C9318>I<0E03803307802307C04383C04301C04700C08700 800E00800E00800E00801C01001C01001C01001C02001C02001C04001C04001C08000E30 0003C00012147C9315>I E /Fi 63 127 df32 D<00C00001C00001C00001C00003F0000FFC003FFE007DCF0071C700E1C380 E1C780E1C780E1C780F1C00079C0003DC0001FE0000FF80003FC0001DE0001CF0001C700 61C380F1C380F1C380E1C380E1C70071C70079DE003FFE001FF80007E00001C00001C000 01C00000C00011247D9F18>36 D<3803007C07807C0780EE0F80EE0F00EE0F00EE1F00EE 1E00EE1E00EE3E007C3C007C3C00387C0000780000780000F80000F00001F00001E00001 E00003E00003C00003C00007C0000783800787C00F87C00F0EE00F0EE01F0EE01E0EE01E 0EE03E0EE03C07C03C07C018038013247E9F18>I<007000F001E003C007800F001E001C 00380038007000700070007000E000E000E000E000E000E000E000E00070007000700070 00380038001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070007000 70007000700070007000E000E000E000E001C001C0038007800F001E003C007800F00060 000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007 F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011 047D8F18>I<000300000780000780000F80000F00001F00001E00001E00003E00003C00 007C0000780000780000F80000F00001F00001E00003E00003C00003C00007C000078000 0F80000F00000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000 F0000060000011247D9F18>47 D<07F8001FFE003FFF007807807803C07801C03001C000 01C00003C0000380000F0003FF0003FE0003FF000007800003C00001C00000E00000E000 00E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B18>51 D<001F00003F0000770000770000E70001E70001C7000387000787000707000E07001E07 003C0700380700780700F00700FFFFF8FFFFF8FFFFF80007000007000007000007000007 00000700007FF000FFF8007FF0151C7F9B18>I<000300000780001F80003F00007E0001 FC0003F00007E0001FC0003F00007E0000FC0000FC00007E00003F00001FC00007E00003 F00001FC00007E00003F00001F8000078000030011187D9918>60 D<600000F00000FC00007E00003F00001FC00007E00003F00001FC00007E00003F00001F 80001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000F000006000 0011187D9918>62 D<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00 038E00038E00038E00038E000306000707000707000707000707000FFF800FFF800FFF80 0E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE00F07E01E03E03C 01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E00000E00000E0 00007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F800131C7E9B 18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C0070 1C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C01C0 1C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00FFFC01F0FC0 1C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E00000E00FF0 E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0 141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C01C01C01C01C01 C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C01C01C01C01 C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF807F FF00111C7D9B18>I<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0070 0E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>76 DI<7E07F0FF0FF87F07F01D81C01D81C01D 81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C 39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B 18>I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780 700700780F007FFF003FFE000FF800111C7D9B18>II<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C 03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C 039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>82 D<03F3801FFF803FFF807C0F 80700780E00380E00380E00380E000007000007800003F00001FF00007FE0000FF00000F 800003C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F8 00131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E0703800700000700000 700000700000700000700000700000700000700000700000700000700000700000700000 700000700000700000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C00 03DC0001F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E00 070F000707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C070078070070000 0F00001E00001C00003C0000780000700000F00001E00001C00003C0000780000700000F 00001E00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<7E0000FE00007E00000E00000E00000E00000E00000E0000 0E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>104 D<03800007C00007C00007C0000380000000000000000000000000007FC000FFC0007FC0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I<7FE000FFE0007FE00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC013 1C7E9B18>108 D<7CE0E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C007F1F1F00FFBFBF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F 80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E 00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C0 7001C0E000E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF00 0FFE0001F00013147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00 380E00380E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E 000E00000E00000E00000E00000E00000E00000E00007FC000FFE0007FC000151E809318 >I<01E38007FB801FFF803E1F80380F80700780700780E00380E00380E00380E00380E0 0380E00380700780700780380F803C1F801FFF800FFB8003E38000038000038000038000 0380000380000380000380003FF8003FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF8 03F87803F03003E00003C00003C000038000038000038000038000038000038000038000 0380000380007FFE00FFFF007FFE0015147F9318>I<07F7003FFF007FFF00780F00E007 00E00700E007007C00007FE0001FFC0003FE00001F00600780E00380E00380F00380F80F 00FFFF00FFFC00E7F00011147D9318>I<0180000380000380000380000380007FFFC0FF FFC0FFFFC003800003800003800003800003800003800003800003800003800003804003 80E00380E00380E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE07E07E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E01E00F03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03 800E03800E0380070700070700070700038E00038E00038E00038E0001DC0001DC0001DC 0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F800 00F80000700000F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF87F8FF0 15147F9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03800703800707000707000387 00038600038E0001CE0001CE0000CC0000CC0000DC000078000078000078000070000070 0000700000F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<3FFFF07F FFF07FFFF07001E07003C0700780000F00001E00003C0000F80001F00003C0000780000F 00701E00703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I<060C1F1E3FBEFBF8F1 F060C00F067C9B18>126 D E /Fj 41 123 df<007E0001C1800301800703C00E03C00E 01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F815 1D809C17>12 D<6060F0F0F8F86868080808080808101010102020404080800D0C7F9C15 >34 D<004000800100020006000C000C0018001800300030007000600060006000E000E0 00E000E000E000E000E000E000E000E000E000E000600060006000700030003000180018 000C000C00060002000100008000400A2A7D9E10>40 D<800040002000100018000C000C 000600060003000300038001800180018001C001C001C001C001C001C001C001C001C001 C001C001C0018001800180038003000300060006000C000C00180010002000400080000A 2A7E9E10>I<60F0F0701010101020204080040C7C830C>44 DI< 60F0F06004047C830C>I72 DI78 D<003F800000E0E0000380 380007001C000E000E001C0007003C00078038000380780003C0780003C0700001C0F000 01E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C07800 03C0780003C0380003803C0007801C0007000E000E0007001C000380380000E0E000003F 80001B1E7E9C20>I82 D<7FFFFFC0700F01C0600F00 C0400F0040400F0040C00F0020800F0020800F0020800F0020000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000001F800003FFFC001B1C7F9B1E>84 DI<08081010202040404040808080808080B0B0F8F8 787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C0001FC00 0F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E801F870012127E9115> 97 DI<07E00C3018783078703060 00E000E000E000E000E000E00060007004300418080C3007C00E127E9112>I<003F0000 070000070000070000070000070000070000070000070000070000070003E7000C170018 0F00300700700700600700E00700E00700E00700E00700E00700E0070060070070070030 0700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006E006FFFEE000 E000E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E061E0E0C0E 000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C3880181800381C00 381C00381C00381C001818001C38000C300013C0001000003000001800001FF8001FFF00 1FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007F800121C7F9215> II<18003C003C0018000000000000 000000000000000000FC001C001C001C001C001C001C001C001C001C001C001C001C001C 001C001C001C00FF80091D7F9C0C>I<00C001E001E000C0000000000000000000000000 00000FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E060E0F0C0F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180 E001C0E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C0003F000 12127F9115>II<03C1000C3300180B00300F0070070070 0700E00700E00700E00700E00700E00700E00700600700700700300F00180F000C370007 C700000700000700000700000700000700000700000700003FE0131A7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000F880388018C018C018E010 D0608FC00D127F9110>I<04000400040004000C000C001C003C00FFE01C001C001C001C 001C001C001C001C001C001C101C101C101C101C100C100E2003C00C1A7F9910>IIII<7F8FF00F03800F030007020003840001C8 0001D80000F00000700000780000F800009C00010E00020E000607000403801E07C0FF0F F81512809116>II<7FFC70386038407040F040E041C003 C0038007000F040E041C043C0C380870087038FFF80E127F9112>I E /Fk 47 123 df<007000E001C00380078007000E001E001E003C003C003C0078007800 780078007000F000F000F000F000F000F000F000F000F000F000F000F000700078007800 780078003C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DII<001F00 00001F0000003F8000003B8000003B8000007BC0000073C0000071C00000F1E00000E1E0 0000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C0007803C 0007003C000FFFFE000FFFFE000FFFFE001E000F001E000F003C000F803C0007803C0007 80780007C0780003C0780003C0F00003E01B207F9F1E>65 DI<001FC000FFF801FFFC03E03C07800C0F 00001E00003E00003C00007C0000780000780000780000F00000F00000F00000F00000F0 0000F00000F00000F000007800007800007800007C00003C00003E00001E00000F000207 800E03E03E01FFFC00FFF0001FC017227DA01D>III<001FE000FFF801FFFE03E03E07800E0F00001E00003E00003C 00007C0000780000780000780000F00000F00000F00000F00000F00000F00000F003FEF0 03FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07801E03E03E01FFFE00 FFF8001FC017227DA01E>71 D73 D75 DIII<003F000000FF C00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F00780007807800 078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F80007C07800078078000780780007803C000F003C000F001E001E001F00 3E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC00003FFF000 07E1F8000F807C001F003E001E001E003C000F003C000F00780007807800078078000780 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE000F83FC00 07E1F80003FFF00000FFF800003F780000007C0000003E0000001E0000001F0000000F80 1A277DA021>II<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C 00003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E000 01E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA0 19>IIIII89 DI<07E03FF8 7FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F 10147E9316>97 D<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000F0007800 78003C013E0F1FFF0FFE03F010147E9314>99 D<00078000078000078000078000078000 078000078000078000078000078000078000078007C7800FF7801FFF803E1F807C078078 0780F80780F00780F00780F00780F00780F00780F00780F00780780780780F803E1F801F FF800FF78007C78011207E9F17>I<03F0000FFC001FFE003E1F003C0700780700700380 FFFF80FFFF80FFFF80F00000F00000F000007000007800003C01003E07001FFF0007FE00 01F80011147F9314>I<007E01FE03FE078007000F000F000F000F000F000F000F00FFF0 FFF0FFF00F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F00 0F000F20809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00780F00780F00780F00780F00 780F003C1E003E3E001FFC003FF80033E0003000003800003FFE003FFF801FFFC03FFFE0 7803F0F000F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00141E7F9317>I 105 D108 DII<01F8 0007FE001FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000 F07801E07801E03C03C03F0FC01FFF8007FE0001F80014147F9317>II<03C7800FF7801FFF803E1F807C0F80780780780780 F00780F00780F00780F00780F00780F00780F807807807807C0F803E1F801FFF800FF780 07C780000780000780000780000780000780000780000780000780000780111D7E9317> II<07F01FFC3FFC780C7800780078007C003FC01FF00FF803F8 007C003C003CC03CF07CFFF87FF00FC00E147F9311>I<1E001E001E001E001E001E00FF F0FFF0FFF01E001E001E001E001E001E001E001E001E001E001E001E001E001E201FF00F F007C00C1A7F9910>II<7801E07C03C03E07801E0F000F0F00 079E0003FC0003F80001F80000F00001F00001F80003FC00079E000F0F000E0F001E0780 3C03C07801E0F801F01414809315>120 DI<7FFF7FFF7FFF003E003C007800F800F001E003E007C007800F001F001E003C00 7C00FFFFFFFFFFFF10147F9314>I E /Fl 39 122 df45 DI<01F00007FC000FFE001F1F003C07803C07807803C07803C0 7803C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0 F001E0F001E0F001E0F001E0F001E07803C07803C07803C07803C03C07803C07801F1F00 0FFE0007FC0001F00013237EA118>48 D<00C001C007C0FFC0FFC0FBC003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C0FFFFFFFFFFFF10227CA118>I<03F0000FFC001FFE003C1F0030078070 07C06003C0E003E0C001E04001E04001E00001E00001E00001E00003C00003C000078000 0780000F00001E00003C0000780000F00001E00001C0000380000700000E00001C000038 0000700000FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC001FFF003E0F00380780 7003C02003C02003C00003C00003C00003C0000780000780000F00001E0003FC0003F800 03FE00000F000007800003C00003C00001E00001E00001E00001E00001E08001E0C003C0 E003C07007803C0F801FFF000FFC0003F00013237EA118>I<001F00001F00002F00002F 00006F0000EF0000CF0001CF0001CF00038F00038F00078F00070F000F0F000E0F001E0F 003C0F003C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F 00000F00000F00000F00000F00000F0015217FA018>I<3FFF803FFF803FFF803C00003C 00003C00003C00003C00003C00003C00003C00003C00003CF8003FFE003FFF003F0F803E 07803C03C03803C00001E00001E00001E00001E00001E00001E00001E04003C04003C0E0 03C07007807C1F003FFE000FFC0003F00013227EA018>I<000FF000007FFC0000FFFF00 01F01F0003C00700078000000F0000001E0000003E0000003C0000003C00000078000000 7800000078000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000 F0000000F00000007800000078000000780000003C0000003C0000003E0000001E000000 0F0000000780008003C0038001F00F8000FFFF00007FFC00000FF00019257DA31F>67 D69 D77 DI80 D<00FE0003FFC007FFE00F81E01E00603C00 003C00007800007800007800007800007800007C00003C00003F00001FC0000FFC0007FF 0001FF80003FC00007E00001F00000F00000F80000780000780000780000780000780000 78C000F0E000F0F801E07E07C03FFF800FFF0001FC0015257EA31B>83 DI87 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00F F00FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F 03803C0180780000780000700000F00000F00000F00000F00000F00000F0000078000078 00007800003C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC0 1FFFC03F0FC03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0 F003C07803C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC 001FFE003E0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F000 007000007800007800003C00801F07800FFF8007FF0001F80012167E9516>I<01F07807 FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F000FFE001F FC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8F00078F0 0078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>103 DII107 DIII<01FC0007FF000FFF801F 07C03C01E07800F07800F0700070F00078F00078F00078F00078F00078F000787800F078 00F07C01F03E03E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07 C07C03C07803C0F803C0F003C0F003C0F003C0F003C0F003C0F003C0F803C07803C07803 C03C07C03F0FC01FFFC00FFBC003E3C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C012207E9519>II<07F01FFC 3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE 3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12 >IIII<7801F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F80000 700000F80001FC0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F815 16809516>I I E /Fm 69 123 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C000C00180018003000300030007000600060006000E0 00E000E000E000E000E000E000E000E000E000E000E00060006000600070003000300030 00180018000C000C000600020001000080004000200B2E7DA112>I<8000400020001000 08000C00060006000300030001800180018001C000C000C000C000E000E000E000E000E0 00E000E000E000E000E000E000E000C000C000C001C00180018001800300030006000600 0C00080010002000400080000B2E7DA112>I<01800180018001800180C183F18F399C0F F003C003C00FF0399CF18FC1830180018001800180018010147DA117>I<000600000006 000000060000000600000006000000060000000600000006000000060000000600000006 000000060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600000006 000000060000000600000006000000060000000600000006000000060000000600000006 00000006000000060000000600001C207D9A23>I<70F8FCFC7404040408081010204006 0E7C840D>II<70F8F8F87005057C840D>I<0001000300030006 00060006000C000C000C00180018001800300030003000600060006000C000C000C00180 018001800300030003000600060006000C000C000C001800180018003000300030006000 60006000C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070 0380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003 F000121F7E9D17>I<018003800F80F38003800380038003800380038003800380038003 800380038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C 9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007 C00007C0000780000780000F00000E00001C0000380000700000600000C0000180000300 000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C0010 0E00200F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003 F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040 0780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00 002E00002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00 200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E00 00FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000 0010000010000011F000161C00180E001007001007800003800003800003C00003C00003 C07003C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17 >I<007C000182000701000E03800C07801C0780380300380000780000700000700000F1 F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C070 03C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC0 7FFF807FFF80400100800200800200800400000800000800001000002000002000004000 00400000C00000C00001C000018000038000038000038000038000078000078000078000 078000078000078000078000030000121F7D9D17>I<70F8F8F870000000000000000000 0070F8F8F87005147C930D>58 D<7FFFFFE0FFFFFFF00000000000000000000000000000 000000000000000000000000000000000000FFFFFFF07FFFFFE01C0C7D9023>61 D<000100000003800000038000000380000007C0000007C0000007C0000009E0000009E0 000009E0000010F0000010F0000010F00000207800002078000020780000403C0000403C 0000403C0000801E0000801E0000FFFE0001000F0001000F0001000F0002000780020007 8002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0 038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C00004078000040 F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 780000007C0000407C0000403C0000401C0000401E0000800E0000800700010003800200 01C0040000703800000FC0001A217D9F21>I69 DI<000FE0200078 186000E004E0038002E0070001E00F0000E01E0000601E0000603C0000603C0000207C00 002078000020F8000000F8000000F8000000F8000000F8000000F8000000F8000000F800 7FFCF80003E0780001E07C0001E03C0001E03C0001E01E0001E01E0001E00F0001E00700 01E0038002E000E0046000781820000FE0001E217D9F24>III 76 DII<001F800000F0F00001C0380007801E000F000F000E0007001E000780 3C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F80001F0F80001F0 F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C0003E03C0003C0 3C0003C01E0007800E0007000F000F0007801E0001C0380000F0F000001F80001C217D9F 23>II<07E0800C198010078030 0380600180600180E00180E00080E00080E00080F00000F000007800007F00003FF0001F FC000FFE0003FF00001F800007800003C00003C00001C08001C08001C08001C08001C0C0 0180C00380E00300F00600CE0C0081F80012217D9F19>83 D<7FFFFFE0780F01E0600F00 60400F0020400F0020C00F0030800F0010800F0010800F0010800F0010000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F80 0007FFFE001C1F7E9E21>IIII<080410082010201040204020804080408040B85CFC7EFC 7E7C3E381C0F0E7B9F17>92 D<1FE000303000781800781C00300E00000E00000E00000E 0000FE00078E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83 C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E0078 0E00780E00780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F 9F19>I<03F80E0C1C1E381E380C70007000F000F000F000F000F000F000700070003801 38011C020E0C03F010147E9314>I<000380003F80000380000380000380000380000380 00038000038000038000038000038003E380061B801C0780380380380380700380700380 F00380F00380F00380F00380F00380F003807003807003803803803807801C07800E1B80 03E3F815207E9F19>I<03F0000E1C001C0E00380700380700700700700380F00380F003 80FFFF80F00000F00000F000007000007000003800801800800C010007060001F8001114 7F9314>I<007C00C6018F038F07060700070007000700070007000700FFF00700070007 000700070007000700070007000700070007000700070007000700070007007FF0102080 9F0E>I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E 001C1C001E380033E0002000002000003000003000003FFE001FFF800FFFC03001E06000 70C00030C00030C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE 00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E 43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C 000000000000000000000000000E007E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E00000 00000000000000000000007007F000F00070007000700070007000700070007000700070 007000700070007000700070007000700070007000706070F060F0C061803F000C28829E 0E>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F3800 0E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00 FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01 F000FE618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E 000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE 43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C0380 3801C03801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E0 3801C03801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00 E00E00F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01 E00F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE0 00151D7F9319>I<03E0800619801C05803C0780380380780380700380F00380F00380F0 0380F00380F00380F003807003807803803803803807801C0B800E138003E38000038000 0380000380000380000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E 0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F14 7F9312>I<1F9030704030C010C010C010E00078007F803FE00FF00070803880188018C0 18C018E030D0608F800D147E9312>I<020002000200060006000E000E003E00FFF80E00 0E000E000E000E000E000E000E000E000E000E000E080E080E080E080E080610031001E0 0D1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319 >III<7FC3FC0F01E00701C007018003810001C20000E40000EC00 007800003800003C00007C00004E000087000107000303800201C00601E01E01E0FF07FE 1714809318>II<3FFF380E200E20 1C40384078407000E001E001C00380078007010E011E011C0338027006700EFFFE10147F 9314>I E /Fn 25 118 df<70F8FCFC7404040404080810102040060F7C840E>44 D<008003800F80F380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038007C0FFFE0F217CA018 >49 D<1000801E07001FFF001FFE001FF80013E000100000100000100000100000100000 10000010F800130E001407001803801003800001C00001C00001E00001E00001E00001E0 7001E0F001E0F001E0E001C08001C04003C04003802007001006000C1C0003F00013227E A018>53 D<007E0001C1000300800601C00E03C01C03C018018038000038000078000070 0000700000F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0F0 01E0F001E07001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013 227EA018>I<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0 F001E0F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E0 0001C00001C00001C0000380000380300300780700780600700C002018001030000FC000 13227EA018>57 D<0001800000018000000180000003C0000003C0000003C0000005E000 0005E000000DF0000008F0000008F0000010F800001078000010780000203C0000203C00 00203C0000401E0000401E0000401E0000800F0000800F0000FFFF000100078001000780 030007C0020003C0020003C0040003E0040001E0040001E00C0000F00C0000F03E0001F8 FF800FFF20237EA225>65 D70 D73 D77 D80 D<0FE0001838003C0C003C0E0018070000070000070000 070000FF0007C7001E07003C0700780700700700F00708F00708F00708F00F087817083C 23900FC1E015157E9418>97 D<01FE000703000C07801C0780380300780000700000F000 00F00000F00000F00000F00000F00000F000007000007800403800401C00800C01000706 0001F80012157E9416>99 D<01FC000707000C03801C01C03801C07801E07000E0F000E0 FFFFE0F00000F00000F00000F00000F000007000007800203800201C00400E0080070300 00FC0013157F9416>101 D<003C00C6018F038F030F0700070007000700070007000700 07000700FFF8070007000700070007000700070007000700070007000700070007000700 07000700070007807FF8102380A20F>I<00007001F198071E180E0E181C07001C07003C 07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010000018000018 00001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860003060003038 00E00E038003FE0015217F9518>I<1C001E003E001E001C000000000000000000000000 00000000000E00FE001E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E00FFC00A227FA10E>105 D<0E00FE001E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B237FA20E>108 D<0E1FC07F00FE60E183 801E807201C00F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E0 0E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60 C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C 01801800C03800E0700070700070F00078F00078F00078F00078F00078F00078F0007870 00707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E8060 0F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C 0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E0000 0E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E 000E000E000E000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>I E /Fo 21 122 df45 D<007F800001FFF0000701F8000C007E0018001F0010000F0020000F80400007C0400007 C0400003E0B00003E0F80003E0FC0003E0FC0003E0FC0003E0780003E0000003E0000007 C0000007C00000078000000F8000000F0000001F0000001E0000003C0000007800000070 000000E0000001C0000003800000070000000E0000000C00000018000000300000006000 0000C000200180002003000020060000400C00004008000040100000C03FFFFFC07FFFFF 80FFFFFF80FFFFFF801B2F7DAE21>50 D<78FCFCFCFC7800000000000000000000000000 00000000000078FCFCFCFC78061F7A9E12>58 D<00003FE0010001FFF8030007F01E0300 1F800307003E000087007800004F00F000002F01E000001F03C000000F078000000F0F80 0000070F000000071F000000031E000000033E000000033C000000017C000000017C0000 00017C000000017800000000F800000000F800000000F800000000F800000000F8000000 00F800000000F800000000F800000000F800000000F800000000F8000000007800000000 7C000000007C000000017C000000013C000000013E000000011E000000011F000000020F 000000020F80000006078000000403C000000801E000000800F00000100078000020003E 0000C0001F8003800007F00F000001FFFC0000003FE00028337CB130>67 DI73 D77 D80 D<00FE00000303C0000C00E00010007000100038 003C003C003E001C003E001E003E001E0008001E0000001E0000001E0000001E00000FFE 0000FC1E0003E01E000F801E001F001E003E001E003C001E007C001E00F8001E04F8001E 04F8001E04F8003E04F8003E0478003E047C005E043E008F080F0307F003FC03E01E1F7D 9E21>97 D<003F8000E0600380180700040F00041E001E1C003E3C003E7C003E7C000878 0000F80000F80000F80000F80000F80000F80000F80000F80000F800007800007C00007C 00003C00011E00011E00020F000207000403801800E060003F80181F7D9E1D>99 D<003F800000E0E0000380380007003C000E001E001E001E001C000F003C000F007C000F 0078000F8078000780F8000780F8000780FFFFFF80F8000000F8000000F8000000F80000 00F8000000F8000000780000007C0000003C0000003C0000801E0000800E0001000F0002 000780020001C00C0000F03000001FC000191F7E9E1D>101 D<0780000000FF80000000 FF800000000F800000000780000000078000000007800000000780000000078000000007 800000000780000000078000000007800000000780000000078000000007800000000780 000000078000000007800000000780FE00000783078000078C03C000079001E00007A001 E00007A000F00007C000F00007C000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078000F000 078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF8021327EB125>104 D<07000F801F801F800F8007000000000000000000000000000000000000000000000007 80FF80FF800F800780078007800780078007800780078007800780078007800780078007 800780078007800780078007800780078007800FC0FFF8FFF80D307EAF12>I<0780FE00 1FC000FF83078060F000FF8C03C18078000F9001E2003C0007A001E4003C0007A000F400 1E0007C000F8001E0007C000F8001E00078000F0001E00078000F0001E00078000F0001E 00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00 078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E000780 00F0001E00078000F0001E000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E 9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F000 07C000F00007C000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F0000FC001F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00 070007000F0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0 F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F0 7C0001F03C0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800 001FC0001D1F7E9E21>I<0781FC00FF860700FF8803C00F9001E007A000F007C0007807 8000780780003C0780003C0780003E0780001E0780001F0780001F0780001F0780001F07 80001F0780001F0780001F0780001F0780001F0780003E0780003E0780003C0780007C07 80007807C000F007A000F007A001E00798038007860F000781F800078000000780000007 80000007800000078000000780000007800000078000000780000007800000078000000F C00000FFFC0000FFFC0000202D7E9E25>I<0783E0FF8C18FF907C0F907C07A07C07C038 07C00007C00007C000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000780000780000780000780000FC000FFFE00 FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010 E00010F00010F800007E00003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C 80003C80001CC0001CC0001CE0001CE00018F00038F00030CC0060C301C080FE00161F7E 9E1A>I<00400000400000400000400000400000C00000C00000C00001C00001C00003C0 0007C0000FC0001FFFE0FFFFE003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C01003C01003C01003C0 1003C01003C01003C01003C01001C02001E02000E0400078C0001F00142C7FAB19>I121 D E /Fp 5 85 df<00000000600000000000E000 00000000E00000000001E00000000003E00000000003E00000000007E00000000007F000 0000000DF0000000000DF00000000019F00000000039F00000000031F00000000061F000 00000061F000000000C1F000000000C1F00000000181F00000000381F00000000301F000 00000601F00000000601F00000000C01F80000000C00F80000001800F80000003800F800 00003000F80000006000F80000006000F8000000C000F8000000C000F8000001FFFFF800 0001FFFFF80000030000F80000060000F80000060000F800000C0000F800000C0000FC00 001800007C00001800007C00003000007C00007000007C00006000007C0000C000007C00 00C000007C0001C000007C0003C000007C000FE00000FE00FFF8001FFFE0FFF8001FFFE0 2B327CB134>65 D<000FFFFFFC0000000FFFFFFF800000007E000FE00000007C0003F000 00007C0000F80000007C00007C0000007C00007C000000F800003E000000F800001E0000 00F800001F000000F800001F000001F000001F000001F000000F800001F000000F800001 F000000F800003E000000F800003E000000F800003E000000F800003E000001F800007C0 00001F800007C000001F800007C000001F800007C000001F80000F8000001F00000F8000 003F00000F8000003F00000F8000003F00001F0000007E00001F0000007E00001F000000 7E00001F0000007C00003E000000FC00003E000000F800003E000001F800003E000001F0 00007C000003E000007C000003E000007C000007C000007C00000F800000F800000F0000 00F800001E000000F800003C000000F8000078000001F00001F0000001F00003E0000001 F0000F80000003F0007F000000FFFFFFFC000000FFFFFFC000000031317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007E0003F800007C0000F800007C00007800007C0000 7800007C0000300000F80000300000F80000300000F80000300000F80000300001F00000 300001F00000300001F00000300001F00000600003E00180600003E00180000003E00180 000003E00180000007C00300000007C00300000007C00700000007C01F0000000FFFFE00 00000FFFFE0000000F801E0000000F800E0000001F000C0000001F000C0000001F000C00 00001F000C0000003E00180000003E00180000003E00000000003E00000000007C000000 00007C00000000007C00000000007C0000000000F80000000000F80000000000F8000000 0000F80000000001F00000000001F00000000001F00000000003F000000000FFFFE00000 00FFFFE00000002D317BB02E>70 D<000FFFFFE000000FFFFFFC0000007E003F0000007C 000F8000007C0007C000007C0003E000007C0003E00000F80001F00000F80001F00000F8 0001F00000F80003F00001F00003F00001F00003F00001F00003F00001F00003E00003E0 0007E00003E00007C00003E0000FC00003E0000F800007C0001F000007C0003E000007C0 0078000007C001E000000F800F8000000FFFFC0000000F800E0000000F80070000001F00 038000001F0003C000001F0001E000001F0001E000003E0001F000003E0001F000003E00 01F000003E0001F000007C0003F000007C0003F000007C0003E000007C0003E00000F800 07E00000F80007E00000F80007E00000F80007E01801F00007E03001F00007E03001F000 07E02003F00003E060FFFF8001E0C0FFFF8000F180000000003E002D327BB033>82 D<07FFFFFFFFF007FFFFFFFFF00FE007E007F00F8007C001F00E0007C000E00C0007C000 E01C0007C000E018000F8000E018000F8000E030000F8000C030000F8000C030001F0000 C060001F0000C060001F0000C060001F0000C0C0003E00018000003E00000000003E0000 0000003E00000000007C00000000007C00000000007C00000000007C0000000000F80000 000000F80000000000F80000000000F80000000001F00000000001F00000000001F00000 000001F00000000003E00000000003E00000000003E00000000003E00000000007C00000 000007C00000000007C00000000007C0000000000F80000000000F80000000000F800000 00000F80000000001F00000000001F00000000001F00000000003F000000003FFFFF0000 003FFFFF0000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 799 988 a Fp(D)25 b(R)g(A)g(F)g(T)388 1079 y Fo(MPI-2)c(Dynamic)f(Pro)r(cess)i(Chapter:)29 b(P)n(art)23 b(I)621 1274 y Fn(Message)16 b(P)o(assing)h(In)o(terface)e(F)l(orum)836 1400 y(April)g(5,)h(1996)p eop %%Page: 1 2 1 1 bop 1852 -100 a Fm(1)75 49 y Fl(0.0.1)49 b(Sta)o(rting)16 b(Pro)q(cesses)f(and)h(Establishing)i(Communication)f(With)f(Them)75 135 y Fm(The)k(follo)o(wing)g(routine)g(starts)e(a)h(n)o(um)o(b)q(er)g (of)g(MPI)h(pro)q(cesses,)g(and)g(establishes)g(comm)o(unication)75 192 y(with)c(them.)75 342 y Fk(MPI)p 160 342 14 2 v 16 w(SP)l(A)-5 b(WN\(command)p 529 342 V 16 w(line,)11 b(minp)o(ro)q(cs,)f (maxp)o(ro)q(cs,)g(info,)h(ro)q(ot,)g(comm,)d(intercomm,)i(a)o(rra)o(y) p 1686 342 V 14 w(of)p 1737 342 V 16 w(errco)q(des\))117 476 y Fj(IN)155 b Fk(command)p 506 476 V 16 w(line)321 b Fj(executable)18 b(and)f(argumen)o(ts,)g(in)g(a)f(single)h(string)g (\(string,)905 533 y(signi\014can)o(t)d(only)f(at)g(ro)q(ot\))117 608 y(IN)155 b Fk(minp)o(ro)q(cs)417 b Fj(minim)n(um)14 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)c(start)i(\(in)o(teger,)g (sig-)905 664 y(ni\014can)o(t)14 b(only)f(at)h(ro)q(ot\))117 739 y(IN)155 b Fk(maxp)o(ro)q(cs)408 b Fj(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g(\(in)o(teger,)h (sig-)905 796 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\))117 871 y(IN)155 b Fk(info)516 b Fj(a)14 b(string)g(telling)f(the)i(run)o (time)e(system)g(where)j(and)d(ho)o(w)h(to)905 927 y(start)h(the)f(pro) q(cesses)j(\(string,)c(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 1002 y(IN)155 b Fk(ro)q(ot)508 b Fj(rank)13 b(of)f(pro)q(cess)j(in)d (whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 1059 y(\(in)o(teger\))117 1134 y(IN)155 b Fk(comm)470 b Fj(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 1209 y(OUT)108 b Fk(intercomm)384 b Fj(in)o(tercomm)o (unicator)7 b(b)q(et)o(w)o(een)k(original)c(group)i(and)g(the)h(newly) 905 1266 y(spa)o(wned)15 b(group)e(\(handle\))117 1341 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1341 V 15 w(of)p 468 1341 V 16 w(errco)q(des)268 b Fj(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))75 1465 y Fi(int)23 b(MPI)p 245 1465 15 2 v 17 w(Spawn\(char*)g(command)p 717 1465 V 16 w(line,)g(int)g(minprocs,)g(int)h(maxprocs,)e(char*)h (info,)393 1522 y(int)h(root,)f(MPI)p 707 1522 V 17 w(Comm)g(comm,)g (MPI)p 1058 1522 V 17 w(Comm*)g(intercomm,)393 1578 y(int*)g(array)p 635 1578 V 17 w(of)p 700 1578 V 17 w(errcodes\))75 1665 y(MPI)p 150 1665 V 17 w(SPAWN\(COMMAND)p 479 1665 V 15 w(LINE,)g(MINPROCS,)g(MAXPROCS,)g(INFO,)g(ROOT,)g(COMM,)g(INTERCOMM,) 393 1721 y(ARRAY)p 516 1721 V 17 w(OF)p 581 1721 V 17 w(ERRCODES,)f(IERR\))170 1778 y(CHARACTER*\(*\))h(COMMAND)p 676 1778 V 16 w(LINE,)g(INFO)170 1834 y(INTEGER)g(MINPROCS,)g (MAXPROCS,)g(ROOT,)g(COMM,)g(INTERCOMM,)170 1890 y(ARRAY)p 293 1890 V 17 w(OF)p 358 1890 V 17 w(ERRCODES\(*\),)f(IERR)166 1977 y Fk(MPI)p 251 1977 14 2 v 16 w(SP)l(A)-5 b(WN)17 b Fm(tries)g(to)f(start)f Fk(maxp)o(ro)q(cs)h Fm(iden)o(tical)i(copies) g(of)e(the)g(program)g(sp)q(eci\014ed)i(b)o(y)f Fk(com-)75 2033 y(mand)p 183 2033 V 16 w(line)c Fm(and)g(establish)h(comm)o (unication)f(with)g(them.)19 b(It)12 b(is)h(collectiv)o(e)h(o)o(v)o(er) e Fk(comm)f Fm(and)h(the)h(newly)75 2090 y(created)g(pro)q(cesses.)19 b(The)13 b(spa)o(wned)g(pro)q(cesses,)g(kno)o(wn)g(as)f(c)o(hildren,)j (are)d Fh(r)n(e)n(quir)n(e)n(d)g Fm(to)g(call)i Fk(MPI)p 1761 2090 V 16 w(INIT)p Fm(,)75 2146 y(whic)o(h)k(is)g(collectiv)o(e)g (with)g Fk(MPI)p 644 2146 V 16 w(SP)l(A)-5 b(WN)18 b Fm(in)f(the)h(paren)o(t)e(pro)q(cesses.)26 b(An)18 b(in)o(tercomm)o (unicator)f(con-)75 2203 y(taining)h(the)f(paren)o(t)f(pro)q(cesses)h (in)h(one)f(group)f(and)i(the)e(c)o(hild)j(pro)q(cesses)e(in)h(the)f (other)f(is)i(returned)75 2259 y(b)o(y)j Fk(MPI)p 229 2259 V 16 w(SP)l(A)-5 b(WN)22 b Fm(in)g(the)f(paren)o(t)g(and)g(can)g (b)q(e)h(obtained)f(in)h(the)g(c)o(hildren)h(through)d(the)h(routine)75 2316 y Fk(MPI)p 160 2316 V 16 w(P)l(ARENT)p Fm(.)f(The)g(c)o(hildren)h (ha)o(v)o(e)e(their)i(o)o(wn)e Fk(MPI)p 1062 2316 V 15 w(COMM)p 1219 2316 V 17 w(W)o(ORLD)g Fm(whic)o(h)i(is)f(separate)f (from)75 2372 y(that)14 b(of)h(the)h(paren)o(t.)166 2429 y(The)22 b Fk(command)p 454 2429 V 15 w(line)h Fm(argumen)o(t)e(is)h(a) g(string)g(con)o(taining)h(a)e(whitespace-separated)i(list)g(of)e(a)75 2485 y(program)14 b(name)h(and)h(argumen)o(ts)e(whic)o(h)i(are)f (passed)g(to)g(the)g(program.)166 2541 y(The)k(follo)o(wing)h(legalese) h(de\014nes)f(all)h(the)e(sp)q(ecial)i(cases)e(and)h(ensures)g(that)e (it)i(is)g(p)q(ossible)h(to)75 2598 y(sp)q(ecify)d(an)o(y)e(argumen)o (ts.)23 b(It)16 b(is)h(unnecessary)g(for)f(normal)h(use.)24 b(Whitespace)17 b(is)g(de\014ned)h(as)e(for)g(the)75 2654 y(isspace\(\))21 b(routine)g(in)h(the)f(standard)f(C)h(library)h (\(space,)f(form-feed,)h(new)f(line,)j(carriage)c(return,)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 2 3 2 2 bop 75 -100 a Fm(2)75 49 y(tab\).)23 b(All)17 b(other)f(c)o (haracters)g(are)g(in)o(terpreted)h(literally)h(except)f(for)f(a)g (small)h(n)o(um)o(b)q(er)g(of)f(t)o(w)o(o-letter)75 106 y(com)o(binations)g(starting)e(with)i(\\)p Fi(\045)p Fm(")e(that)h(ha)o(v)o(e)g(sp)q(ecial)i(meaning.)143 198 y Ff(\017)23 b Fm(\\)p Fi(\045\045)p Fm(")14 b(is)i(replaced)g(b)o (y)f(\\)p Fi(\045)p Fm(")143 291 y Ff(\017)23 b Fm(\\)p Fi(\045)p Fm(")11 b(where)j(\\)p Fi()p Fm(")e(is)i(a)g(single)h(whitespace)g(c)o(haracter)e(is)i(replaced)189 347 y(b)o(y)g(that)f(whitespace)i(c)o(haracter)f(but)g(do)q(es)h(not)e (separate)h(items)h(in)g(the)f(command)g(line.)143 440 y Ff(\017)23 b Fm(\\)p Fi(\045_)p Fm(")13 b(is)j(discarded.)k(Ho)o(w)o (ev)o(er,)14 b(it)h(is)g(treated)f(as)g(a)h(non-whitespace)h(c)o (haracter)d(in)j(that)e(it)h(can)189 497 y(b)q(egin)h(a)f(new)g(w)o (ord.)20 b(Its)15 b(in)o(tended)h(use)g(is)g(to)e(allo)o(w)i(the)f(sp)q (eci\014cation)i(of)e(empt)o(y)g(strings.)75 589 y(F)l(or)e(example,)i (the)g(command-line)g(\\)p Fi(foo ba\045 r \045_ q\045\045u\045\045_ux) p Fm(")c(\(where)j(\\)p Fi( )p Fm(")f(is)i(a)f(space)g(c)o(harac-)75 645 y(ter\))g(sp)q(eci\014es)i(the)e(command)h(\\)p Fi(foo)p Fm(")e(with)i(argumen)o(ts)e(\\)p Fi(ba r)p Fm(",)g(\\",)h(and)g(\\)p Fi(q\045u\045_ux)p Fm(".)19 b(The)14 b(sp)q(ecial)75 702 y(t)o(w)o(o-letter)g(com)o(binations)h(are)g(translated)g(as)f (they)h(are)g(encoun)o(tered)g(when)g(reading)h(the)f(string,)f(so)75 758 y(that)h(\\)p Fi(\045\045_)p Fm(")g(is)i(translated)f(to)f(\\)p Fi(\045_)p Fm(",)h(not)f(\\)p Fi(\045)p Fm(".)22 b(Note)16 b(that)f(\\)p Fi(\045)p Fm(")h(not)f(follo)o(w)o(ed)i(b)o(y)f(\\)p Fi(\045)p Fm(",)f(\\)p Fi(\045)p Fm(",)g(\\)p Fi(_)p Fm(")g(or)75 815 y(\\)p Fi()p Fm(")e(is)j(not)e(treated)h (sp)q(ecially)l(.)166 954 y Fe(Discussion:)34 b Fj(The)14 b(format)e(ma)o(y)f(lo)q(ok)i(complicated)g(to)g(users)i(but)f(is)g (only)f(messy)g(if)g(they)h(w)o(an)o(t)g(to)f(do)75 1010 y(something)h(strange.)25 b(Ho)o(w)o(ev)o(er,)16 b(in)f(en)o(vironmen)o (ts)g(where)i(\014lenames)e(tend)h(to)f(ha)o(v)o(e)h(em)o(b)q(edded)g (whitespace)75 1067 y(this)e(\(and)g(an)o(y)f(other)i(\\command-l)o (ine")c(in)o(terfaces\))k(could)e(b)q(ecome)h(a)g(nigh)o(tmare.)189 1253 y Fh(A)n(dvic)n(e)j(to)i(users.)53 b Fm(Though)17 b Fk(MPI)p 818 1253 14 2 v 16 w(SP)l(A)-5 b(WN)18 b Fm(tak)o(es)f(a)g (command)p 1353 1253 V 16 w(line)j(as)d(an)g(argumen)o(t,)g(it)g(is)189 1310 y(imp)q(ortan)o(t)h(to)g(remem)o(b)q(er)g(that)g(it)h(is)g(not)f (a)g(shell)i(command)e(that)g(y)o(ou)g(migh)o(t)g(t)o(yp)q(e)h(at)f (the)189 1366 y(terminal.)h(F)l(or)12 b(instance,)h(\\)p Fi(foo)23 b(>)h(bar)p Fm(")12 b(will)i(spa)o(wn)d(a)h(program)f(\\)p Fi(foo)p Fm(")h(with)g(argumen)o(ts)g(\\)p Fi(>)p Fm(")189 1423 y(and)17 b(\\)p Fi(bar)p Fm(",)f(but)i(will)h(not)e(redirect)h (output.)25 b(\\)p Fi(mpirun)e(-np)h(4)f(foo)p Fm(")17 b(will)i(spa)o(wn)e Fk(maxp)o(ro)q(cs)189 1479 y Fm(copies)d(of)e(the)i (program)e(\\)p Fi(mpirun)p Fm(".)18 b(The)13 b(application)i(will)g (exp)q(ect)e(to)g(establish)h(comm)o(unica-)189 1536 y(tion)i(with)h(the)f(\\)p Fi(mpirun)p Fm(")f(program,)f(not)i(with)h (the)f(pro)q(cesses)h(that)e(ma)o(y)h(b)q(e)g(started)g(b)o(y)g(the)189 1592 y Fi(mpirun)e Fm(program.)19 b(\()p Fh(End)c(of)i(advic)n(e)f(to)g (users.)p Fm(\))166 1696 y(MPI)f(tries)g(to)f(start)g Fk(maxp)o(ro)q(cs)g Fm(copies)i(of)f(the)g(program)f(sp)q(eci\014ed)j (in)e(the)h Fk(command)p 1676 1696 V 15 w(line)p Fm(.)k(MPI)75 1753 y(do)q(es)f(not)f(sp)q(ecify)i(ho)o(w)e(to)g(\014nd)i(the)f (executable)h(or)e(ho)o(w)g(the)h(w)o(orking)f(directory)h(is)g (determined.)75 1809 y(These)k(rules)h(will)g(b)q(e)f(implemen)o (tation-dep)q(end)q(en)o(t)i(and)e(will)h(b)q(e)g(appropriate)f(for)f (the)h(run)o(time)75 1866 y(en)o(vironmen)o(t.)d(F)l(or)12 b(instance,)i(a)f(homogeneous)g(system)g(with)h(a)f(global)h(\014le)g (system)f(migh)o(t)g(lo)q(ok)g(\014rst)75 1922 y(in)h(the)f(w)o(orking) f(directory)h(of)g(the)g(spa)o(wning)g(pro)q(cess,)g(or)g(migh)o(t)g (searc)o(h)f(the)h(directories)h(in)g(a)f(P)l(A)l(TH)75 1979 y(en)o(vironmen)o(t)e(v)m(ariable)g(as)f(do)g(Unix)h(shells.)20 b(An)10 b(implemen)o(tation)i(on)e(top)g(of)g(PVM)g(w)o(ould)g(use)h (PVM's)75 2035 y(rules)k(for)f(\014nding)h(executables)h(\(usually)f (in)g Fi(~/pvm3/bin/$PVM_ARCH)p Fm(\).)c(An)j(MPI)h(implemen)o(tation) 75 2091 y(running)h(under)g(IBM's)f(POE)h(will)g(use)g(POE's)f(metho)q (d)g(of)g(\014nding)i(executables.)166 2148 y(While)j(MPI)f(alw)o(a)o (ys)f(tries)i(to)e(spa)o(wn)h Fk(maxp)o(ro)q(cs)f Fm(pro)q(cesses,)i (the)f Fk(minp)o(ro)q(cs)g Fm(argumen)o(t)f(allo)o(ws)75 2204 y(some)d(additional)h(\015exibilit)o(y)l(.)23 b(If)15 b(MPI)g(is)g(unable)h(to)f(spa)o(wn)f Fk(maxp)o(ro)q(cs)g Fm(pro)q(cesses,)h(it)h(ma)o(y)e(spa)o(wn)g(as)75 2261 y(few)d(as)f Fk(minp)o(ro)q(cs)p Fm(.)18 b(The)12 b(n)o(um)o(b)q(er)f (of)f(pro)q(cesses)h(actually)h(spa)o(wned)f(can)g(b)q(e)h(determined)g (b)o(y)f(examining)75 2317 y(the)j(size)h(of)f(the)g(remote)f(group)h (of)g(the)g(returned)g(in)o(tercomm)o(unicator.)20 b(If)14 b(MPI)g(is)h(unable)g(to)e(spa)o(wn)75 2374 y(at)20 b(least)h Fk(minp)o(ro)q(cs)p Fm(,)h Fk(MPI)p 538 2374 V 16 w(SP)l(A)-5 b(WN)22 b Fm(will)g(return)f(an)g(error.)36 b(It)21 b(ma)o(y)f(b)q(e)i (p)q(ossible)h(to)d(additionally)75 2430 y(constrain)15 b(the)h(n)o(um)o(b)q(er)f(of)g(pro)q(cesses)h(spa)o(wned)f(through)g (the)g Fk(info)g Fm(argumen)o(t.)189 2534 y Fh(A)n(dvic)n(e)h(to)i (users.)50 b Fm(An)17 b(MPI)f(implemen)o(tation)i(is)g(allo)o(w)o(ed,)f (but)g(is)g(not)f(required,)i(to)e(spa)o(wn)189 2591 y Fk(minp)o(ro)q(cs)d Fm(pro)q(cesses)h(if)g Fk(maxp)o(ro)q(cs)e Fm(are)i(not)f(a)o(v)m(ailable.)21 b(Th)o(us)13 b(it)h(w)o(ould)g(not)f (b)q(e)h(p)q(ortable)g(to)f(try)189 2647 y(to)i(acquire)i(all)g(a)o(v)m (ailable)h(resources)e(b)o(y)g(sp)q(ecifying)i(a)e(v)o(ery)g(large)g(v) m(alue)h(for)f Fk(maxp)o(ro)q(cs)f Fm(and)h(a)189 2704 y(small)g(v)m(alue)g(for)f Fk(minp)o(ro)q(cs)p Fm(.)k(\()p Fh(End)d(of)g(advic)n(e)g(to)h(users.)p Fm(\))1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 4 3 3 bop 1852 -100 a Fm(3)166 49 y(The)12 b Fk(info)g Fm(argumen)o(t)g(is)h(opaque)f(to)f(MPI)i(and)f(is)h(passed)f(directly) i(to)d(the)h(run)o(time)h(en)o(vironmen)o(t,)75 106 y(whic)o(h)g(uses)f (it)g(to)f(determine)h(where)g(and)g(ho)o(w)f(to)h(spa)o(wn)f(pro)q (cesses.)19 b(In)12 b(C)g(it)g(is)g(a)g(NULL-terminated)75 162 y(string.)24 b(If)17 b(the)g(info)g(argumen)o(t)f(is)h(empt)o(y)f (\(NULL)i(or)e(an)g(empt)o(y)h(string)f(in)i(C,)e(or)g(')p Fi( )p Fm(')g(F)l(ortran\))f(the)75 219 y(run)o(time)j(en)o(vironmen)o (t)g(decides)h(where)f(to)f(spa)o(wn)g(pro)q(cesses.)28 b(P)o(ortable)17 b(programs)f(not)i(requiring)75 275 y(detailed)f(con)o(trol)e(o)o(v)o(er)f(pro)q(cess)h(lo)q(cations)h (should)h(use)e(an)g(empt)o(y)g(argumen)o(t.)166 332 y(MPI)k(do)q(es)h(not)f(sp)q(ecify)i(the)e(format)g(of)g(the)g Fk(info)g Fm(argumen)o(t,)h(whic)o(h)g(is)g(left)g(up)g(to)e(the)i(MPI) 75 388 y(implemen)o(tation.)31 b(The)19 b(format)e(is)i(exp)q(ected)h (to)e(b)q(e)h(tailored)h(to)e(the)g(particular)h(run)o(time)g(system)75 444 y(b)q(eing)d(used.)21 b(Examples)16 b(of)e(an)h(info)h(string)f (migh)o(t)g(b)q(e)h(\\hostname=dop)q(ey")f(to)g(spa)o(wn)g(all)h(tasks) e(on)75 501 y(a)g(host)h(named)g(\\dop)q(ey",)f(or)g(\\2:hippi:sgi")i (to)e(spa)o(wn)g(on)h(t)o(w)o(o)e(hippi-connected)18 b(SGI)d(mac)o(hines,)g(or)75 557 y(ev)o(en)h(\\\014le=info.cfg")g(to)e (indicate)j(that)d(detailed)j(information)e(is)h(sp)q(eci\014ed)h(in)f (the)g(\014le)g(\\info.cfg".)166 614 y(Alternativ)o(ely)l(,)f(it)e(is)h (ev)o(en)g(p)q(ossible)i(to)c(use)i(the)g Fk(info)f Fm(argumen)o(t)g (to)g(sp)q(ecify)h(the)g(executable)h(and)75 670 y(its)f(command-line)h (argumen)o(ts,)d(in)i(whic)o(h)h(case)e(the)g Fk(command)p 1198 670 14 2 v 16 w(line)g Fm(argumen)o(t)g(to)g Fk(MPI)p 1630 670 V 16 w(SP)l(A)-5 b(WN)14 b Fm(can)75 727 y(b)q(e)k(empt)o(y)l (.)26 b(The)17 b(abilit)o(y)i(to)e(do)g(this)g(follo)o(ws)h(from)e(the) i(fact)e(that)h(MPI)g(do)q(es)h(not)f(sp)q(ecify)h(ho)o(w)f(an)75 783 y(executable)k(is)f(found,)h(and)f(the)g Fk(info)g Fm(argumen)o(t)f(can)h(tell)g(the)g(run)o(time)g(system)g(where)g(to)f (\\\014nd")75 840 y(the)h(executable)g(\\".)33 b(Of)20 b(course)f(a)g(program)g(that)g(do)q(es)h(this)g(will)h(not)e(b)q(e)h (p)q(ortable)g(across)f(MPI)75 896 y(implemen)o(tations.)166 953 y(The)f Fk(a)o(rra)o(y)p 360 953 V 14 w(of)p 411 953 V 16 w(errco)q(des)g Fm(is)g(an)f(arra)o(y)g(of)g(length)h Fk(maxp)o(ro)q(cs)e Fm(in)j(whic)o(h)f(MPI)f(rep)q(orts)g(the)h(status) 75 1009 y(of)e(eac)o(h)h(pro)q(cess)h(that)e(MPI)h(w)o(as)f(requested)h (to)f(start.)24 b(If)17 b(all)h Fk(maxp)o(ro)q(cs)e Fm(pro)q(cesses)i (w)o(ere)e(spa)o(wned,)75 1065 y Fk(a)o(rra)o(y)p 173 1065 V 15 w(of)p 225 1065 V 16 w(errco)q(des)22 b Fm(is)g(\014lled)h (in)g(with)f(the)f(v)m(alue)i Fk(MPI)p 1048 1065 V 16 w(SUCCESS)p Fm(.)f(If)g(only)g Fd(m)g Fm(\()p Fk(minp)o(ro)q(cs)h Ff(\024)g Fd(m)h(<)75 1122 y Fk(maxp)o(ro)q(cs)o Fm(\))e(pro)q(cesses)h (are)f(spa)o(wned,)i Fd(m)e Fm(of)g(the)g(en)o(tries)h(will)h(con)o (tain)e Fk(MPI)p 1489 1122 V 16 w(SUCCESS)h Fm(and)g(the)75 1178 y(rest)14 b(will)j(con)o(tain)e(an)g(error)f(co)q(de)i(indicating) g(an)f(implemen)o(tation-de\014ned)j(reason)d(MPI)f(could)i(not)75 1235 y(start)g(the)i(pro)q(cess.)27 b(MPI)18 b(do)q(es)f(not)h(sp)q (ecify)g(whic)o(h)h(en)o(tries)f(corresp)q(ond)g(to)f(failed)h(pro)q (cesses.)28 b(An)75 1291 y(implemen)o(tation)19 b(ma)o(y)l(,)g(for)e (instance,)i(\014ll)h(in)f(error)e(co)q(des)i(in)g(one-to-one)f (corresp)q(ondence)i(with)e(a)75 1348 y(detailed)g(sp)q(eci\014cation)g (in)g(the)e Fk(info)g Fm(argumen)o(t.)23 b(These)17 b(error)f(co)q(des) h(all)g(b)q(elong)h(to)d(the)i(error)f(class)75 1404 y Fk(MPI)p 160 1404 V 16 w(ERR)p 261 1404 V 17 w(SP)l(A)-5 b(WN)16 b Fm(if)h(there)e(w)o(as)g(no)h(error)f(in)h(the)g(argumen)o(t) f(list.)22 b(An)15 b(application)j(ma)o(y)d(pass)g(\(in)o(t)75 1461 y(*\)NULL)h(\(C-binding\))g(for)f Fk(a)o(rra)o(y)p 671 1461 V 14 w(of)p 722 1461 V 16 w(errco)q(des)p Fm(,)g(in)i(whic)o (h)f(case)f(it)g(is)h(ignored)g(b)o(y)f(MPI.)166 1517 y(If)c(no)g(pro)q(cesses)h(are)e(spa)o(wned,)i(MPI)f(returns)g(a)f (\\half-empt)o(y")h(in)o(tercomm)o(unicator,)h(that)e(is,)i(an)75 1574 y(in)o(tercomm)o(unicator)j(for)g(whic)o(h)h Fk(MPI)p 749 1574 V 16 w(COMM)p 907 1574 V 16 w(REMOTE)p 1110 1574 V 17 w(GROUP)h Fm(returns)e Fk(MPI)p 1535 1574 V 16 w(GROUP)p 1703 1574 V 18 w(EMPTY)75 1630 y Fm(and)g Fk(MPI)p 248 1630 V 16 w(COMM)p 406 1630 V 17 w(REMOTE)p 610 1630 V 17 w(SIZE)g Fm(returns)g(0.)166 1686 y(The)10 b(return)g(v)m(alue)i(\(C\))d(and)h Fk(ierr)g Fm(argumen)o(t)f(\(F)l (ortran\))f(ha)o(v)o(e)i(their)h(usual)g(meanings.)18 b Fk(MPI)p 1746 1686 V 16 w(SUCCESS)75 1743 y Fm(means)c(that)f(at)h (least)g Fk(minp)o(ro)q(cs)g Fm(pro)q(cesses)g(w)o(ere)g(spa)o(wned)g (successfully)l(.)22 b(Other)14 b(v)m(alues)h(indicate)h(an)75 1799 y(error.)166 1856 y(All)21 b(argumen)o(ts)f(b)q(efore)g(the)h Fk(ro)q(ot)e Fm(argumen)o(t)h(are)g(sp)q(eci\014ed)i(on)e(the)h(pro)q (cess)f(whose)g(rank)g(in)75 1912 y Fk(comm)13 b Fm(is)j(equal)g(to)f Fk(ro)q(ot)p Fm(.)k(The)c(v)m(alue)i(of)e(these)g(argumen)o(ts)f(on)i (other)e(pro)q(cesses)i(is)g(ignored.)75 2063 y Fk(MPI)p 160 2063 V 16 w(P)l(ARENT\(intercomm\))117 2140 y Fj(OUT)108 b Fk(intercomm)384 b Fj(paren)o(t)15 b(in)o(tercomm)o(unicator)c (\(handle\))75 2265 y Fi(int)23 b(MPI)p 245 2265 15 2 v 17 w(Parent\(MPI)p 502 2265 V 16 w(Comm*)g(intercomm\))75 2351 y(MPI)p 150 2351 V 17 w(PARENT\(INTERCOMM,)e(IERR\))170 2408 y(INTEGER)i(INTERCOMM,)g(IERR)166 2494 y Fk(MPI)p 251 2494 14 2 v 16 w(P)l(ARENT)f Fm(can)g(b)q(e)h(called)g(in)f(an)o(y) f(pro)q(cess.)40 b(If)22 b(there)g(is)g(a)f(paren)o(t,)h(\(i.e.,)h(the) f(pro)q(cess)75 2551 y(w)o(as)c(started)h(with)g Fk(MPI)p 517 2551 V 16 w(SP)l(A)-5 b(WN)p Fm(\))20 b(it)f(returns)g(an)h(in)o (tercomm)o(unicator.)31 b(The)20 b(lo)q(cal)g(group)f(of)g(the)75 2607 y(in)o(tercomm)o(unicator)i(consists)g(of)f(the)h(pro)q(cesses)g (created)g(with)g(the)g(same)g(call)h(to)e Fk(MPI)p 1699 2607 V 15 w(SP)l(A)-5 b(WN)p Fm(.)75 2664 y(The)16 b(remote)g(group)g (consists)g(of)g(the)g(pro)q(cesses)h(that)f(co)q(op)q(erated)g(on)g (the)g(call)i(to)d Fk(MPI)p 1652 2664 V 16 w(SP)l(A)-5 b(WN)p Fm(.)17 b(If)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 5 4 4 bop 75 -100 a Fm(4)75 49 y(there)15 b(is)h(no)f(paren)o(t,)g Fk(MPI)p 541 49 14 2 v 15 w(P)l(ARENT)i Fm(returns)e Fk(MPI)p 986 49 V 16 w(COMM)p 1144 49 V 16 w(NULL)p Fm(.)75 171 y Fl(0.0.2)49 b(Sta)o(rting)16 b(Multiple)h(Executables)f(and)h (Establishing)g(Communication)g(with)g(Them)75 257 y Fm(While)h Fk(MPI)p 294 257 V 16 w(SP)l(A)-5 b(WN)17 b Fm(is)h(su\016cien)o(t)f(for)f(most)g(cases,)h(it)g(do)q(es)g(not)f (allo)o(w)h(the)g(spa)o(wning)g(of)f(m)o(ultiple)75 313 y(binaries,)23 b(or)e(of)f(the)h(same)g(binary)g(with)h(m)o(ultiple)g (sets)f(of)f(argumen)o(ts.)37 b(The)21 b(follo)o(wing)g(routine)75 370 y(spa)o(wns)16 b(m)o(ultiple)j(binaries)f(or)e(the)g(same)h(binary) g(with)g(m)o(ultiple)h(sets)e(of)g(argumen)o(ts,)g(establishing)75 426 y(comm)o(unication)g(with)f(them.)20 b(All)d(spa)o(wned)e(programs) f(ha)o(v)o(e)h(the)g(same)g Fk(MPI)p 1469 426 V 16 w(COMM)p 1627 426 V 16 w(W)o(ORLD)p Fm(.)75 577 y Fk(MPI)p 160 577 V 16 w(SP)l(A)-5 b(WN)p 326 577 V 17 w(MUL)l(TIPLE\(count,)11 b(a)o(rra)o(y)p 802 577 V 15 w(of)p 854 577 V 16 w(command)p 1055 577 V 15 w(lines,)g(a)o(rra)o(y)p 1271 577 V 15 w(of)p 1323 577 V 16 w(minp)o(ro)q(cs,)g(a)o(rra)o(y)p 1628 577 V 14 w(of)p 1679 577 V 16 w(maxp)o(ro)q(cs,)75 634 y(a)o(rra)o(y)p 173 634 V 15 w(of)p 225 634 V 16 w(info,)j(ro)q(ot,)h(comm,)e(intercomm,)g(a)o(rra)o(y)p 913 634 V 15 w(of)p 965 634 V 16 w(errco)q(des\))117 711 y Fj(IN)155 b Fk(count)482 b Fj(Num)o(b)q(er)12 b(of)g(command)e (lines)j(\(in)o(teger,)g(size)g(of)f(eac)o(h)h(of)f(the)905 767 y(follo)o(wing)f(arra)o(ys\))117 842 y(IN)155 b Fk(a)o(rra)o(y)p 416 842 V 15 w(of)p 468 842 V 16 w(command)p 669 842 V 15 w(lines)142 b Fj(executable)12 b(and)f(argumen)o(ts)f(\(arra)o(y)g (of)g(strings,)h(signi\014can)o(t)905 899 y(only)i(at)h(ro)q(ot\))117 974 y(IN)155 b Fk(a)o(rra)o(y)p 416 974 V 15 w(of)p 468 974 V 16 w(minp)o(ro)q(cs)254 b Fj(minim)n(um)9 b(n)o(um)o(b)q(er)k(of) f(pro)q(cesses)k(for)d(eac)o(h)h(command)c(line)905 1030 y(to)k(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can)o(t)h(only) f(at)h(ro)q(ot\))117 1106 y(IN)155 b Fk(a)o(rra)o(y)p 416 1106 V 15 w(of)p 468 1106 V 16 w(maxp)o(ro)q(cs)245 b Fj(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o (h)g(command)d(line)905 1162 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o (teger,)f(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 1237 y(IN)155 b Fk(a)o(rra)o(y)p 416 1237 V 15 w(of)p 468 1237 V 16 w(info)353 b Fj(strings)17 b(telling)e(the)i(run)o(time)e (system)i(where)g(and)f(ho)o(w)g(to)905 1294 y(start)21 b(pro)q(cesses)i(\(arra)o(y)e(of)e(strings,)k(signi\014can)o(t)c(only)h (at)905 1350 y(ro)q(ot)117 1425 y(IN)155 b Fk(ro)q(ot)508 b Fj(rank)13 b(of)f(pro)q(cess)j(in)d(whic)o(h)h(previous)g(argumen)o (ts)f(are)i(v)n(alid)905 1482 y(\(in)o(teger\))117 1557 y(IN)155 b Fk(comm)470 b Fj(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o (wning)g(pro)q(cesses)j(\(handle\))117 1632 y(OUT)108 b Fk(intercomm)384 b Fj(in)o(tercomm)o(unicator)12 b(b)q(et)o(w)o(een)j (original)d(group)i(and)g(newly)905 1688 y(spa)o(wned)h(group)e (\(handle\))117 1763 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1763 V 15 w(of)p 468 1763 V 16 w(errco)q(des)268 b Fj(one)14 b(error)h(co)q(de)g(p)q(er)g(pro)q(cess)g(\(arra)o(y)f(of)f(in)o (teger\))75 1888 y Fi(int)23 b(MPI)p 245 1888 15 2 v 17 w(Spawn\(int*)g(count,)g(char*)g(array)p 955 1888 V 16 w(of)p 1019 1888 V 17 w(command)p 1204 1888 V 17 w(lines[],)393 1944 y(int*)g(array)p 635 1944 V 17 w(of)p 700 1944 V 17 w(minprocs,)g(int*)g(array)p 1195 1944 V 16 w(of)p 1259 1944 V 17 w(maxprocs,)393 2001 y(char*)g(array)p 659 2001 V 17 w(of)p 724 2001 V 17 w(info[],)g(int)g(root,)g(MPI)p 1242 2001 V 17 w(Comm)g(comm,)393 2057 y(MPI)p 468 2057 V 17 w(Comm*)g(intercomm,)g(int*)g(array)p 1130 2057 V 16 w(of)p 1194 2057 V 17 w(errcodes\))75 2144 y(MPI)p 150 2144 V 17 w(SPAWN\(COUNT,)f(ARRAY)p 597 2144 V 16 w(OF)p 661 2144 V 17 w(COMMAND)p 846 2144 V 17 w(LINES,)h(ARRAY)p 1150 2144 V 16 w(OF)p 1214 2144 V 17 w(MINPROCS,)393 2200 y(ARRAY)p 516 2200 V 17 w(OF)p 581 2200 V 17 w(MAXPROCS,)f(ARRAY)p 956 2200 V 17 w(OF)p 1021 2200 V 17 w(INFO,)h(ROOT,)g(COMM,)g (INTERCOMM,)393 2257 y(ARRAY)p 516 2257 V 17 w(OF)p 581 2257 V 17 w(ERRCODES,)f(IERR\))170 2313 y(CHARACTER*\(*\))h(ARRAY)p 628 2313 V 16 w(OF)p 692 2313 V 17 w(COMMAND)p 877 2313 V 16 w(LINES\(*\),)g(ARRAY)p 1252 2313 V 16 w(OF)p 1316 2313 V 17 w(INFO)170 2370 y(INTEGER)g(COUNT,)g(ARRAY)p 651 2370 V 17 w(OF)p 716 2370 V 17 w(MINPROCS,)f(ARRAY)p 1091 2370 V 17 w(OF)p 1156 2370 V 17 w(MAXPROCS,)g(ROOT,)i(COMM,)170 2426 y(INTERCOMM,)f(ARRAY)p 556 2426 V 16 w(OF)p 620 2426 V 17 w(ERRCODES\(*\),)g(IERR)166 2512 y Fk(MPI)p 251 2512 14 2 v 16 w(SP)l(A)-5 b(WN)p 417 2512 V 17 w(MUL)l(TIPLE)12 b Fm(is)h(iden)o(tical)h(to)e Fk(MPI)p 1021 2512 V 16 w(SP)l(A)-5 b(WN)13 b Fm(except)g(that)f(there)h(are)f(m)o(ultiple)i (exe-)75 2569 y(cutable)e(sp)q(eci\014cations.)20 b(The)12 b(\014rst)e(argumen)o(t,)h Fk(count)p Fm(,)i(giv)o(es)e(the)g(n)o(um)o (b)q(er)h(of)e(sp)q(eci\014cations.)21 b(Eac)o(h)10 b(of)75 2625 y(the)k(next)g(four)g(argumen)o(ts)f(are)h(simply)i(arra)o(ys)c (of)i(the)g(corresp)q(onding)h(argumen)o(ts)f(in)h Fk(MPI)p 1699 2625 V 15 w(SP)l(A)-5 b(WN)p Fm(.)1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 6 5 5 bop 1852 -100 a Fm(5)166 49 y(All)25 b(of)f(the)g(spa)o(wned)g(pro) q(cesses)h(ha)o(v)o(e)e(the)h(same)g Fk(MPI)p 1203 49 14 2 v 16 w(COMM)p 1361 49 V 16 w(W)o(ORLD)p Fm(.)g(Their)g(ranks)g(in) 75 106 y Fk(MPI)p 160 106 V 16 w(COMM)p 318 106 V 16 w(W)o(ORLD)13 b Fm(corresp)q(ond)g(directly)h(to)e(the)g(order)h(in)g (whic)o(h)g(the)g(command)g(lines)h(are)e(sp)q(ec-)75 162 y(i\014ed)j(in)g Fk(MPI)p 309 162 V 16 w(SP)l(A)-5 b(WN)p 475 162 V 17 w(MUL)l(TIPLE)p Fm(.)13 b(Assume)i(that)e Fd(m)1037 169 y Fc(1)1071 162 y Fm(pro)q(cesses)h(are)g(generated)g(b)o (y)g(the)g(\014rst)g(com-)75 219 y(mand,)h Fd(m)254 226 y Fc(2)289 219 y Fm(b)o(y)g(the)h(second,)f(etc.)20 b(The)c(pro)q (cesses)g(corresp)q(onding)g(to)f(the)g(\014rst)g(command)g(line)i(ha)o (v)o(e)75 275 y(ranks)11 b(0)p Fd(;)d Fm(1)p Fd(;)g(:)g(:)g(:)t(;)g(m) 402 282 y Fc(1)424 275 y Ff(\000)s Fm(1.)19 b(The)12 b(pro)q(cesses)g(in)h(the)e(second)i(command)e(ha)o(v)o(e)h(ranks)f Fd(m)1532 282 y Fc(1)1552 275 y Fd(;)d(m)1613 282 y Fc(1)1635 275 y Fm(+)s(1)p Fd(;)g(:)g(:)g(:)d(m)1817 282 y Fc(1)1840 275 y Fm(+)75 332 y Fd(m)115 339 y Fc(2)141 332 y Ff(\000)i Fm(1.)19 b(The)14 b(pro)q(cesses)f(in)i(the)e(third)h(ha)o(v)o(e)f (ranks)g Fd(m)1028 339 y Fc(1)1054 332 y Fm(+)7 b Fd(m)1136 339 y Fc(2)1156 332 y Fd(;)h(m)1217 339 y Fc(1)1242 332 y Fm(+)f Fd(m)1324 339 y Fc(2)1350 332 y Fm(+)g(1)p Fd(;)h(:)g(:)g(:)t (;)g(m)1556 339 y Fc(1)1582 332 y Fm(+)f Fd(m)1664 339 y Fc(2)1690 332 y Fm(+)g Fd(m)1772 339 y Fc(3)1798 332 y Ff(\000)g Fm(1,)75 388 y(etc.)189 494 y Fh(A)n(dvic)n(e)17 b(to)i(users.)56 b Fm(If)18 b(y)o(ou)g(need)h(to)e(spa)o(wn)g(m)o (ultiple)j(executables,)f(it)g(is)f(recommended)h(to)189 551 y(use)d Fk(MPI)p 353 551 V 16 w(SP)l(A)-5 b(WN)p 519 551 V 18 w(MUL)l(TIPLE)15 b Fm(instead)i(of)f(calling)j Fk(MPI)p 1208 551 V 15 w(SP)l(A)-5 b(WN)18 b Fm(sev)o(eral)e(times.)25 b(There)16 b(are)189 607 y(t)o(w)o(o)11 b(reasons)h(for)h(this.)19 b(First,)13 b(spa)o(wning)g(sev)o(eral)g(things)g(at)f(once)h(ma)o(y)f (b)q(e)i(faster)d(than)i(spa)o(wn-)189 664 y(ing)22 b(them)f(sequen)o (tially)l(.)41 b(Second,)23 b(in)f(some)g(implemen)o(tations,)h(comm)o (unication)g(b)q(et)o(w)o(een)189 720 y(pro)q(cesses)14 b(spa)o(wned)h(at)f(the)g(same)g(time)h(ma)o(y)e(b)q(e)j(faster)d(than) h(comm)o(unication)h(b)q(et)o(w)o(een)g(pro-)189 777 y(cesses)g(spa)o(wned)h(separately)l(.)k(\()p Fh(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fm(\))166 883 y(The)e Fk(a)o(rra)o(y)p 356 883 V 14 w(of)p 407 883 V 16 w(errco)q(des)g Fm(argumen)o(t)f(is)h(1-dimensional)i(arra)o(y)c(of)h(size)1375 851 y Fb(P)1419 864 y Fa(count)1419 894 y(i)p Fc(=1)1515 883 y Fd(n)1542 890 y Fa(i)1557 883 y Fm(,)g(where)h Fd(n)1740 890 y Fa(i)1768 883 y Fm(is)g(the)75 939 y Fd(i)p Fm(th)19 b(elemen)o(t)i(of)e Fk(a)o(rra)o(y)p 480 939 V 14 w(of)p 531 939 V 16 w(maxp)o(ro)q(cs)p Fm(.)33 b(Command)19 b(line)j(n)o(um)o(b)q(er)e Fd(i)f Fm(corresp)q(onds)h(to)f Fd(n)1630 946 y Fa(i)1664 939 y Fm(con)o(tiguous)75 996 y(slots)c(in)h(this)g(arra)o(y)l(,)d(and)j(error)e(co)q(des)i(are)f (treated)g(as)f(for)h Fk(MPI)p 1214 996 V 16 w(SP)l(A)-5 b(WN)p Fm(.)75 1117 y Fl(0.0.3)49 b(Environmental)17 b(inquiry)g(-)g(Prop)q(osal)g(1)75 1203 y Fm(MPI)e(do)q(es)h(not)e(pro) o(vide)i(mec)o(hanisms)g(for)e(in)o(teracting)i(with)f(an)g(external)h (resource)f(manager.)k(It)c(is)75 1260 y(assumed)i(that)g(all)h(suc)o (h)g(in)o(teraction)f(will)i(o)q(ccur)f(through)f(a)g(run)o(time)g(en)o (vironmen)o(t-sp)q(eci\014c)j(API,)75 1316 y(or,)15 b(indirectly)l(,)j (through)d(the)h(opaque)g(\(to)f(MPI\))g(\\info")h(argumen)o(t)f(to)g (the)h(spa)o(wn)g(calls.)22 b(Ho)o(w)o(ev)o(er,)75 1373 y(the)14 b(large)h(ma)s(jorit)o(y)e(of)h(\\dynamic")h(MPI)f (applications)i(are)e(exp)q(ected)i(to)d(exist)i(in)g(a)f(static)h(run) o(time)75 1429 y(en)o(vironmen)o(t,)h(in)h(whic)o(h)g(resources)f(ha)o (v)o(e)f(b)q(een)i(allo)q(cated)g(b)q(efore)f(the)g(application)i(is)f (run.)22 b(This)16 b(is)75 1486 y(the)j(case,)g(for)f(instance,)h(with) g(most)f(PVM)g(applications,)j(whic)o(h)e(rarely)g(manipulate)h(the)e (virtual)75 1542 y(mac)o(hine)d(within)g(an)f(application.)21 b(When)14 b(a)f(user)h(\(or)f(p)q(ossibly)j(a)d(batc)o(h)h(system\))f (runs)h(one)g(of)g(these)75 1598 y(quasi-static)j(applications,)g (he/she/it)g(will)h(usually)f(sp)q(ecify)h(a)e(n)o(um)o(b)q(er)g(of)g (pro)q(cesses)g(to)g(start)f(and)75 1655 y(a)h(total)h(n)o(um)o(b)q(er) g(of)f(pro)q(cesses)h(that)f(are)h(exp)q(ected.)26 b(An)17 b(application)h(simply)g(needs)g(to)e(kno)o(w)g(ho)o(w)75 1711 y(man)o(y)f(slots)g(there)g(are,)g(i.e.,)f(ho)o(w)h(man)o(y)g(pro) q(cesses)g(it)h(should)g(spa)o(wn.)166 1768 y(MPI)h(pro)o(vides)h(an)g (in)o(teger)f(v)m(ariable,)i Fk(MPI)p 943 1768 V 16 w(UNIVERSE)p 1173 1768 V 18 w(SIZE)p Fm(,)d(that)h(tells)h(an)g(application)h(the)75 1824 y(total)e(n)o(um)o(b)q(er)h(of)f(useful)h(pro)q(cess)g(slots)g(a)o (v)m(ailable)h(to)e(an)g(application.)28 b(An)18 b(application)h(will)h (t)o(ypi-)75 1881 y(cally)d(subtract)f(the)h(size)g(of)f Fk(MPI)p 669 1881 V 15 w(COMM)p 826 1881 V 17 w(W)o(ORLD)g Fm(from)f Fk(MPI)p 1211 1881 V 16 w(UNIVERSE)p 1441 1881 V 18 w(SIZE)h Fm(to)g(\014nd)h(out)e(ho)o(w)75 1937 y(man)o(y)f(pro)q (cesses)g(it)h(should)g(spa)o(wn.)k Fk(MPI)p 821 1937 V 16 w(UNIVERSE)p 1051 1937 V 17 w(SIZE)14 b Fm(is)h(initialized)i(in)e Fk(MPI)p 1562 1937 V 16 w(INIT)e Fm(and)i(is)f(not)75 1994 y(c)o(hanged)i(b)o(y)f(MPI.)166 2050 y Fk(MPI)p 251 2050 V 16 w(UNIVERSE)p 481 2050 V 17 w(SIZE)g Fm(is)h(a)e (recommendation,)h(not)g(necessarily)h(a)f(hard)g(limit.)21 b(F)l(or)15 b(instance,)75 2107 y(some)20 b(implemen)o(tations)h(ma)o (y)f(allo)o(w)g(an)g(application)i(to)e(spa)o(wn)g(50)f(pro)q(cesses)i (p)q(er)g(pro)q(cessor,)f(if)75 2163 y(they)15 b(are)f(requested.)20 b(Ho)o(w)o(ev)o(er,)14 b(it)h(is)g(lik)o(ely)h(that)e(the)h(user)g (only)g(w)o(an)o(ts)e(to)h(spa)o(wn)h(one)g(pro)q(cess)f(p)q(er)75 2219 y(pro)q(cessor.)166 2276 y Fk(MPI)p 251 2276 V 16 w(UNIVERSE)p 481 2276 V 17 w(SIZE)c Fm(is)h(assumed)f(to)g(ha)o(v)o(e)f (b)q(een)j(sp)q(eci\014ed)g(when)f(an)f(application)i(w)o(as)d (started,)75 2332 y(and)k(is)g(in)g(essence)h(a)e(p)q(ortable)h(mec)o (hanism)g(to)f(allo)o(w)h(the)g(user)f(to)g(pass)h(to)f(the)g (application)j(\(through)75 2389 y(the)g(MPI)h(pro)q(cess)f(startup)g (mec)o(hanism\))h(a)f(piece)h(of)f(critical)i(run)o(time)f (information.)k(Note)15 b(that)g(no)75 2445 y(in)o(teraction)h(with)g (the)g(run)o(time)g(en)o(vironmen)o(t)g(is)g(required.)23 b(If)16 b(the)g(run)o(time)g(en)o(vironmen)o(t)g(c)o(hanges)75 2502 y(size)g(while)i(an)d(application)i(is)f(running)h Fk(MPI)p 880 2502 V 16 w(UNIVERSE)p 1110 2502 V 17 w(SIZE)f Fm(is)g(not)f(up)q(dated,)h(and)g(the)f(applica-)75 2558 y(tion)g(m)o(ust)g(\014nd)h(out)f(through)g(direct)h(comm)o(unication)f (with)h(the)f(run)o(time)h(system.)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 7 6 6 bop 75 -100 a Fm(6)75 49 y Fl(0.0.4)49 b(Environmental)17 b(inquiry)g(-)g(Prop)q(osal)g(2)75 135 y Fm(While)d(general)g(resource) f(managemen)o(t)g(is)g(b)q(ey)o(ond)h(the)f(scop)q(e)h(of)e(MPI,)h (dynamic)h(applications)h(ma)o(y)75 192 y(need)c(to)f(kno)o(w)f(ho)o(w) h(man)o(y)f(pro)q(cesses)i(they)f(can)h(start.)16 b(The)11 b(function)g Fk(MPI)p 1387 192 14 2 v 16 w(GET)p 1491 192 V 16 w(SP)l(A)-5 b(WNABLE)p 1769 192 V 18 w(SIZE)75 248 y Fm(queries)13 b(the)f(run)o(time)g(system)f(to)g(allo)o(w)h(the)g (user)g(to)f(\014nd)i(out)e(ho)o(w)g(man)o(y)g(pro)q(cesses)i(ma)o(y)e (b)q(e)h(started.)75 399 y Fk(MPI)p 160 399 V 16 w(GET)p 264 399 V 17 w(SP)l(A)-5 b(WNABLE)p 543 399 V 17 w(SIZE\(info,)14 b(size\))117 476 y Fj(IN)155 b Fk(info)516 b Fj(opaque)14 b(string)g(describing)g(resources)j(required)d(\(string\))117 551 y(OUT)108 b Fk(size)519 b Fj(n)o(um)o(b)q(er)13 b(of)h(pro)q (cesses)i(that)e(ma)o(y)e(b)q(e)j(started)g(\(in)o(teger\))75 676 y Fi(int)23 b(MPI)p 245 676 15 2 v 17 w(Get)p 334 676 V 17 w(spawnable)p 567 676 V 16 w(size\(char*)f(info,)i(int*)f (size\))75 762 y(MPI)p 150 762 V 17 w(GET)p 239 762 V 17 w(SPAWNABLE)p 472 762 V 15 w(SIZE\(info,)g(size\))170 819 y(CHARACTER*\(*\))g(info)170 875 y(INTEGER)g(size)166 962 y Fm(The)f Fk(info)f Fm(argumen)o(t)g(is)i(of)e(the)h(same)f(form)g (as)g(the)h Fk(info)g Fm(argumen)o(t)e(of)i Fk(MPI)p 1591 962 14 2 v 16 w(SP)l(A)-5 b(WN)p Fm(.)22 b(It)f(is)75 1018 y(opaque)c(to)e(MPI)i(and)f(is)h(passed)g(directly)h(to)e(the)g (run)o(time)h(en)o(vironmen)o(t,)g(whic)o(h)g(examines)g(it)g(and)75 1074 y(determines)f(ho)o(w)f(man)o(y)g(pro)q(cesses)g(could)h(b)q(e)g (spa)o(wned)g(on)f(the)g(resources)g(describ)q(ed.)166 1131 y Fk(MPI)p 251 1131 V 16 w(GET)p 355 1131 V 16 w(SP)l(A)-5 b(WNABLE)p 633 1131 V 18 w(SIZE\(\))12 b Fm(returns)g(in)h Fk(size)g Fm(the)f(n)o(um)o(b)q(er)h(of)f(pro)q(cesses)h(that)e(could)j (prob-)75 1187 y(ably)k(b)q(e)f(spa)o(wned)h(succesfully)h(if)e Fk(MPI)p 781 1187 V 16 w(SP)l(A)-5 b(WN\(\))18 b Fm(w)o(ere)e(called)j (immediately)g(follo)o(wing.)26 b(There)17 b(is)75 1244 y(no)g(guaran)o(tee,)g(ho)o(w)o(ev)o(er,)f(that)h(a)g(subsequen)o(t)h Fk(MPI)p 1014 1244 V 16 w(SP)l(A)-5 b(WN\(\))17 b Fm(on)g(the)h(same)f (resources)g(describ)q(ed)75 1300 y(b)o(y)f Fk(info)f Fm(can)h(spa)o(wn)g(that)f(n)o(um)o(b)q(er)h(of)f(pro)q(cesses.)22 b(An)17 b(MPI)e(implemen)o(tation)i(is)g(allo)o(w)o(ed)f(to)f(return)75 1357 y Fk(MPI)p 160 1357 V 16 w(ERR)p 261 1357 V 17 w(SIZE)p 371 1357 V 16 w(UNKNO)o(WN)i Fm(as)e(an)h(error)g(co)q(de)g(\(return)g (v)m(alue)h(\(C\))e(or)h Fk(ierr)f Fm(argumen)o(t)g(\(F)l(ortran\)\))75 1413 y(if)i Fk(info)f Fm(is)g(not)g(empt)o(y)g(and)h(it)f(can)g(not)g (\014nd)h(out)f(the)g(n)o(um)o(b)q(er)h(of)e(pro)q(cesses)i(that)e(ma)o (y)h(b)q(e)h(spa)o(wned)75 1470 y(on)e(the)g(resources)h(describ)q(ed)h (b)o(y)e Fk(info)p Fm(.)166 1526 y(Using)c(an)f(empt)o(y)g Fk(info)g Fm(argumen)o(t,)g Fk(MPI)p 859 1526 V 15 w(GET)p 962 1526 V 17 w(SP)l(A)-5 b(WNABLE)p 1241 1526 V 18 w(SIZE\(\))9 b Fm(and)h Fk(MPI)p 1562 1526 V 16 w(SP)l(A)-5 b(WN\(\))11 b Fm(should)75 1583 y(allo)o(w)18 b(the)g(user)h(to)e(write)h(p)q (ortable)h(co)q(de)g(without)f(calling)i(a)d(run)o(time-en)o(vironmen)o (t)i(API,)f(e.g.)28 b(if)75 1639 y(he/she)16 b(w)o(an)o(ts)e(to)g (split)i(the)f(a)o(v)m(ailable)i(resources)e(to)f(spa)o(wn)h (di\013eren)o(t)h(executables)g(on)f(parts)f(of)h(the)75 1695 y(resources.)75 1817 y Fl(0.0.5)49 b(Nonblo)q(cking)18 b(requests)75 1903 y Fm(Spa)o(wning)e(new)g(pro)q(cesses)g(ma)o(y)f(b)q (e)h(an)g(exp)q(ensiv)o(e)h(op)q(eration.)k(In)16 b(order)f(to)g(allo)o (w)h(pro)q(cesses)g(to)f(do)75 1959 y(useful)i(w)o(ork)e(while)i(pro)q (cesses)g(are)e(b)q(eing)j(spa)o(wned,)d(MPI)h(pro)o(vides)h(non-blo)q (c)o(king)g(v)o(ersions)f(of)g(the)75 2016 y(eac)o(h)f(of)g(the)g(ab)q (o)o(v)o(e)g(routines.)1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 8 7 7 bop 1852 -100 a Fm(7)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN\(...,)14 b(request\))117 126 y Fj(OUT)108 b Fk(request)452 b Fj(Request)15 b(ob)r(ject)g(that)f(can)g (b)q(e)h(used)g(to)f(c)o(hec)o(k)h(for)e(comple-)905 183 y(tion)h(\(handle\))75 402 y Fk(MPI)p 160 402 V 16 w(ISP)l(A)-5 b(WN)p 339 402 V 17 w(MUL)l(TIPLE\(...,)12 b(request\))117 479 y Fj(OUT)108 b Fk(request)452 b Fj(Request)15 b(ob)r(ject)g(that)f(can)g(b)q(e)h(used)g(to)f(c)o(hec)o(k)h(for)e (comple-)905 536 y(tion)h(\(handle\))166 660 y Fm(The)22 b(argumen)o(ts)f(of)g(the)h(non-blo)q(c)o(king)h(v)o(ersions)f(are)g (the)f(same)h(as)f(those)h(of)f(the)h(blo)q(c)o(king)75 716 y(v)o(ersions,)16 b(except)h(that)f(there)h(is)g(an)f(additional)i Fk(request)g Fm(argumen)o(t.)23 b(The)16 b(request)h(can)f(b)q(e)h (used)g(as)75 773 y(input)f(for)f Fk(MPI)p 351 773 V 16 w(W)l(AIT)p Fm(,)f Fk(MPI)p 589 773 V 16 w(TEST)p Fm(,)h(etc.)166 829 y(In)23 b(all)g(cases,)g(the)g(output)f Fk(group)g Fm(or)f Fk(intercomm)p Fm(,)h(and)h(the)f Fk(a)o(rra)o(y)p 1386 829 V 15 w(of)p 1438 829 V 16 w(errco)q(des)g Fm(are)g(up)q(dated)75 886 y(async)o(hronously)16 b(and)f(are)g(not)g (v)m(alid)h(un)o(til)h(the)e(request)g(completes.)166 942 y(The)f(status)f(ob)s(ject)h(returned)g(b)o(y)g Fk(MPI)p 858 942 V 16 w(W)l(AIT)g Fm(con)o(tains)g(an)g(error)f(co)q(de)h (indicating)i(whether)e(or)75 999 y(not)h(the)h(spa)o(wn)f(w)o(as)g (successful.)22 b(The)15 b(failure)i(or)e(success)h(of)f(individual)k (pro)q(cesses)d(is)g(indicated)h(in)75 1055 y Fk(a)o(rra)o(y)p 173 1055 V 15 w(of)p 225 1055 V 16 w(errco)q(des)p Fm(.)k(The)16 b(errorco)q(de)g(can)f(b)q(e)i(retriev)o(ed)f(as)f(the)h Fk(MPI)p 1280 1055 V 15 w(ERROR)i Fm(elemen)o(t)e(of)f(the)h(status)75 1112 y(ob)s(ject)f(returned)g(b)o(y)g Fk(MPI)p 545 1112 V 16 w(W)l(AIT)p Fm(,)g Fk(MPI)p 784 1112 V 16 w(TEST)p Fm(,)g(etc.)166 1168 y(All)k(of)e(the)g(routines)h(in)g(this)g(section) g(are)g(collectiv)o(e,)h(and)f(ob)q(ey)f(the)h(follo)o(wing)g(non-blo)q (c)o(king)75 1225 y(collectiv)o(e)f(seman)o(tics.)143 1318 y Ff(\017)23 b Fm(Eac)o(h)11 b(call)i(is)f(collectiv)o(e)h(o)o(v)o (er)e(the)h(calling)h(comm)o(unicator)e(and)h(the)g(newly)g(spa)o(wned) g(pro)q(cesses.)143 1412 y Ff(\017)23 b Fm(The)17 b(request)g(ma)o(y)g (not)f(b)q(e)i(cancelled.)28 b(An)18 b(attempt)e(to)g(cancel)j(the)e (request)g(will)i(return)e(an)189 1469 y(error)d(of)h(error)f(class)i Fk(MPI)p 655 1469 V 16 w(ERR)p 756 1469 V 17 w(REQUEST)p Fm(.)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Apr 8 17:56:42 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA25502; Mon, 8 Apr 1996 17:56:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA11248 for mpi-core-out; Mon, 8 Apr 1996 16:54:30 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA11233 for ; Mon, 8 Apr 1996 16:54:13 -0500 Message-Id: <199604082154.QAA11233@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: MPI Forum meetings for the rest of the year Date: Mon, 08 Apr 1996 16:54:11 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the schedule we agreed on at the last meeting. I have arranged with the hotel (the Ramada) for these dates, so they can be regarded as fixed, unless there is some major collective reason we need to change any of them. Apr 24-26 Jun 5-7 Jul 17-19 Sep 4-6 Oct 9-11 Supercomputing '96 is Nov 17-22. We have to work fast. Rusty From mpi-core-human@mcs.anl.gov Wed Apr 10 20:22:16 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA16301; Wed, 10 Apr 1996 20:21:12 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA19859 for mpi-core-out; Wed, 10 Apr 1996 19:22:58 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA19809; Wed, 10 Apr 1996 19:19:54 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id RAA00904; Wed, 10 Apr 1996 17:19:52 -0700 From: "William C. Saphir" Message-Id: <9604101719.ZM900@win233.nas.nasa.gov> Date: Wed, 10 Apr 1996 17:19:47 -0700 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: mpi-dynamic@mcs.anl.gov Subject: dynamic process chapter - part II Cc: mpi-core@mcs.anl.gov, wcs@win233.nas.nasa.gov Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="PART-BOUNDARY=.19604101719.ZM900.nas.nasa.gov" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk -- --PART-BOUNDARY=.19604101719.ZM900.nas.nasa.gov Content-Type: text/plain; charset=us-ascii Dear Group, Here is a new version of part II of the dynamic process chapter. It contains all the independent spawn routines, signalling routines and the function formerly called MPI_NOTIFY. It also has a detailed section on the non-blocking routines, intended to replace the section currently in part I. It contains the results of discussions at the last meeting as well as a few arbitrary and capricious changes. Highlights include - bindings for all functions - new selection of INDEPENDENT functions as per detailed votes at the last meeting - A few fleshed out explanations and discussions. - A rearranged and somewhat different process utilities section. The rest of the dynamic process chapter will be out shortly. Bill --PART-BOUNDARY=.19604101719.ZM900.nas.nasa.gov X-Zm-Content-Name: main.ps Content-Type: application/postscript ; name="main.ps" Content-Transfer-Encoding: base64 X-Zm-Decoding-Hint: mimencode -b -u JSFQUy1BZG9iZS0yLjAKJSVDcmVhdG9yOiBkdmlwc2sgNS41OGYgQ29weXJpZ2h0IDE5ODYs IDE5OTQgUmFkaWNhbCBFeWUgU29mdHdhcmUKJSVUaXRsZTogbWFpbi5kdmkKJSVQYWdlczog MTUKJSVQYWdlT3JkZXI6IEFzY2VuZAolJUJvdW5kaW5nQm94OiAwIDAgNjEyIDc5MgolJUVu ZENvbW1lbnRzCiVEVklQU0NvbW1hbmRMaW5lOiAvdXNyL2xvY2FsL2Jpbi9kdmlwcyAtbyBt YWluLnBzIG1haW4uZHZpCiVEVklQU1BhcmFtZXRlcnM6IGRwaT0zMDAsIGNvbW1lbnRzIHJl bW92ZWQKJURWSVBTU291cmNlOiAgVGVYIG91dHB1dCAxOTk2LjA0LjEwOjE3MDkKJSVCZWdp blByb2NTZXQ6IHRleC5wcm8KL1RlWERpY3QgMjUwIGRpY3QgZGVmIFRlWERpY3QgYmVnaW4g L057ZGVmfWRlZiAvQntiaW5kIGRlZn1OIC9Te2V4Y2h9TgovWHtTIE59QiAvVFJ7dHJhbnNs YXRlfU4gL2lzbHMgZmFsc2UgTiAvdnNpemUgMTEgNzIgbXVsIE4gL2hzaXplIDguNSA3Mgpt dWwgTiAvbGFuZHBsdXM5MHtmYWxzZX1kZWYgL0ByaWdpbntpc2xze1swIGxhbmRwbHVzOTB7 MSAtMX17LTEgMX0KaWZlbHNlIDAgMCAwXWNvbmNhdH1pZiA3MiBSZXNvbHV0aW9uIGRpdiA3 MiBWUmVzb2x1dGlvbiBkaXYgbmVnIHNjYWxlCmlzbHN7bGFuZHBsdXM5MHtWUmVzb2x1dGlv biA3MiBkaXYgdnNpemUgbXVsIDAgZXhjaH17UmVzb2x1dGlvbiAtNzIgZGl2CmhzaXplIG11 bCAwfWlmZWxzZSBUUn1pZiBSZXNvbHV0aW9uIFZSZXNvbHV0aW9uIHZzaXplIC03MiBkaXYg MSBhZGQgbXVsClRSW21hdHJpeCBjdXJyZW50bWF0cml4e2R1cCBkdXAgcm91bmQgc3ViIGFi cyAwLjAwMDAxIGx0e3JvdW5kfWlmfQpmb3JhbGwgcm91bmQgZXhjaCByb3VuZCBleGNoXXNl dG1hdHJpeH1OIC9AbGFuZHNjYXBley9pc2xzIHRydWUgTn1CCi9AbWFudWFsZmVlZHtzdGF0 dXNkaWN0IC9tYW51YWxmZWVkIHRydWUgcHV0fUIgL0Bjb3BpZXN7LyNjb3BpZXMgWH1CCi9G TWF0WzEgMCAwIC0xIDAgMF1OIC9GQkJbMCAwIDAgMF1OIC9ubiAwIE4gL0lFIDAgTiAvY3Ry IDAgTiAvZGYtdGFpbHsKL25uIDggZGljdCBOIG5uIGJlZ2luIC9Gb250VHlwZSAzIE4gL0Zv bnRNYXRyaXggZm50cnggTiAvRm9udEJCb3ggRkJCIE4Kc3RyaW5nIC9iYXNlIFggYXJyYXkg L0JpdE1hcHMgWCAvQnVpbGRDaGFye0NoYXJCdWlsZGVyfU4gL0VuY29kaW5nIElFIE4KZW5k IGR1cHsvZm9vIHNldGZvbnR9MiBhcnJheSBjb3B5IGN2eCBOIGxvYWQgMCBubiBwdXQgL2N0 ciAwIE5bfUIgL2Rmewovc2YgMSBOIC9mbnRyeCBGTWF0IE4gZGYtdGFpbH1CIC9kZnN7ZGl2 IC9zZiBYIC9mbnRyeFtzZiAwIDAgc2YgbmVnIDAgMF0KTiBkZi10YWlsfUIgL0V7cG9wIG5u IGR1cCBkZWZpbmVmb250IHNldGZvbnR9QiAvY2gtd2lkdGh7Y2gtZGF0YSBkdXAKbGVuZ3Ro IDUgc3ViIGdldH1CIC9jaC1oZWlnaHR7Y2gtZGF0YSBkdXAgbGVuZ3RoIDQgc3ViIGdldH1C IC9jaC14b2ZmewoxMjggY2gtZGF0YSBkdXAgbGVuZ3RoIDMgc3ViIGdldCBzdWJ9QiAvY2gt eW9mZntjaC1kYXRhIGR1cCBsZW5ndGggMiBzdWIKZ2V0IDEyNyBzdWJ9QiAvY2gtZHh7Y2gt ZGF0YSBkdXAgbGVuZ3RoIDEgc3ViIGdldH1CIC9jaC1pbWFnZXtjaC1kYXRhCmR1cCB0eXBl IC9zdHJpbmd0eXBlIG5le2N0ciBnZXQgL2N0ciBjdHIgMSBhZGQgTn1pZn1CIC9pZCAwIE4g L3J3IDAgTgovcmMgMCBOIC9ncCAwIE4gL2NwIDAgTiAvRyAwIE4gL3NmIDAgTiAvQ2hhckJ1 aWxkZXJ7c2F2ZSAzIDEgcm9sbCBTIGR1cAovYmFzZSBnZXQgMiBpbmRleCBnZXQgUyAvQml0 TWFwcyBnZXQgUyBnZXQgL2NoLWRhdGEgWCBwb3AgL2N0ciAwIE4gY2gtZHgKMCBjaC14b2Zm IGNoLXlvZmYgY2gtaGVpZ2h0IHN1YiBjaC14b2ZmIGNoLXdpZHRoIGFkZCBjaC15b2ZmCnNl dGNhY2hlZGV2aWNlIGNoLXdpZHRoIGNoLWhlaWdodCB0cnVlWzEgMCAwIC0xIC0uMSBjaC14 b2ZmIHN1YiBjaC15b2ZmCi4xIHN1Yl17Y2gtaW1hZ2V9aW1hZ2VtYXNrIHJlc3RvcmV9QiAv RHsvY2MgWCBkdXAgdHlwZSAvc3RyaW5ndHlwZSBuZXtdfQppZiBubiAvYmFzZSBnZXQgY2Mg Y3RyIHB1dCBubiAvQml0TWFwcyBnZXQgUyBjdHIgUyBzZiAxIG5le2R1cCBkdXAKbGVuZ3Ro IDEgc3ViIGR1cCAyIGluZGV4IFMgZ2V0IHNmIGRpdiBwdXR9aWYgcHV0IC9jdHIgY3RyIDEg YWRkIE59QiAvSXsKY2MgMSBhZGQgRH1CIC9ib3B7dXNlcmRpY3QgL2JvcC1ob29rIGtub3du e2JvcC1ob29rfWlmIC9TSSBzYXZlIE4gQHJpZ2luCjAgMCBtb3ZldG8gL1YgbWF0cml4IGN1 cnJlbnRtYXRyaXggZHVwIDEgZ2V0IGR1cCBtdWwgZXhjaCAwIGdldCBkdXAgbXVsCmFkZCAu OTkgbHR7L1FWfXsvUlZ9aWZlbHNlIGxvYWQgZGVmIHBvcCBwb3B9TiAvZW9we1NJIHJlc3Rv cmUgdXNlcmRpY3QKL2VvcC1ob29rIGtub3due2VvcC1ob29rfWlmIHNob3dwYWdlfU4gL0Bz dGFydHt1c2VyZGljdCAvc3RhcnQtaG9vawprbm93bntzdGFydC1ob29rfWlmIHBvcCAvVlJl c29sdXRpb24gWCAvUmVzb2x1dGlvbiBYIDEwMDAgZGl2IC9EVkltYWcgWAovSUUgMjU2IGFy cmF5IE4gMCAxIDI1NXtJRSBTIDEgc3RyaW5nIGR1cCAwIDMgaW5kZXggcHV0IGN2biBwdXR9 Zm9yCjY1NzgxLjc2IGRpdiAvdnNpemUgWCA2NTc4MS43NiBkaXYgL2hzaXplIFh9TiAvcHtz aG93fU4gL1JNYXRbMSAwIDAgLTEgMAowXU4gL0JEb3QgMjYwIHN0cmluZyBOIC9ydWxleCAw IE4gL3J1bGV5IDAgTiAvdnsvcnVsZXkgWCAvcnVsZXggWCBWfUIgL1YKe31CIC9SViBzdGF0 dXNkaWN0IGJlZ2luIC9wcm9kdWN0IHdoZXJle3BvcCBwcm9kdWN0IGR1cCBsZW5ndGggNyBn ZXswIDcKZ2V0aW50ZXJ2YWwgZHVwKERpc3BsYXkpZXEgZXhjaCAwIDQgZ2V0aW50ZXJ2YWwo TmVYVCllcSBvcn17cG9wIGZhbHNlfQppZmVsc2V9e2ZhbHNlfWlmZWxzZSBlbmR7e2dzYXZl IFRSIC0uMSAuMSBUUiAxIDEgc2NhbGUgcnVsZXggcnVsZXkgZmFsc2UKUk1hdHtCRG90fWlt YWdlbWFzayBncmVzdG9yZX19e3tnc2F2ZSBUUiAtLjEgLjEgVFIgcnVsZXggcnVsZXkgc2Nh bGUgMSAxCmZhbHNlIFJNYXR7QkRvdH1pbWFnZW1hc2sgZ3Jlc3RvcmV9fWlmZWxzZSBCIC9R Vntnc2F2ZSBuZXdwYXRoIHRyYW5zZm9ybQpyb3VuZCBleGNoIHJvdW5kIGV4Y2ggaXRyYW5z Zm9ybSBtb3ZldG8gcnVsZXggMCBybGluZXRvIDAgcnVsZXkgbmVnCnJsaW5ldG8gcnVsZXgg bmVnIDAgcmxpbmV0byBmaWxsIGdyZXN0b3JlfUIgL2F7bW92ZXRvfUIgL2RlbHRhIDAgTiAv dGFpbAp7ZHVwIC9kZWx0YSBYIDAgcm1vdmV0b31CIC9Ne1MgcCBkZWx0YSBhZGQgdGFpbH1C IC9ie1MgcCB0YWlsfUIgL2N7LTQgTX0KQiAvZHstMyBNfUIgL2V7LTIgTX1CIC9mey0xIE19 QiAvZ3swIE19QiAvaHsxIE19QiAvaXsyIE19QiAvanszIE19QiAva3sKNCBNfUIgL3d7MCBy bW92ZXRvfUIgL2x7cCAtNCB3fUIgL217cCAtMyB3fUIgL257cCAtMiB3fUIgL297cCAtMSB3 fUIgL3F7CnAgMSB3fUIgL3J7cCAyIHd9QiAvc3twIDMgd31CIC90e3AgNCB3fUIgL3h7MCBT IHJtb3ZldG99QiAveXszIDIgcm9sbCBwCmF9QiAvYm9zey9TUyBzYXZlIE59QiAvZW9ze1NT IHJlc3RvcmV9QiBlbmQKJSVFbmRQcm9jU2V0ClRlWERpY3QgYmVnaW4gNDAyNTg0MzcgNTIw OTkxNTQgMTAwMCAzMDAgMzAwIChtYWluLmR2aSkKQHN0YXJ0IC9GYSAxIDEwNiBkZjwwMEUw MDFFMDAxRTAwMEMwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDBFMDAxMzAwCjIzODA0 MzgwNDM4MDQzODA4NzAwMDcwMDA3MDAwRTAwMEUwMDFDMDAxQzAwMUMyMDM4NDAzODQwMzg0 MDM4ODAxOTAwMEUwMAowQjFGN0U5RTEwPjEwNSBEIEUgL0ZiIDEgMTYgZGY8MDNDMDBGRjAx RkY4M0ZGQzdGRkU3RkZFRkZGRkZGRkZGRkZGRkZGRgpGRkZGRkZGRjdGRkU3RkZFM0ZGQzFG RjgwRkYwMDNDMDEwMTI3RDkzMTc+MTUgRCBFIC9GYwoxIDYxIGRmPDAwMDAwMzgwMDAwMDBG MDAwMDAwM0MwMDAwMDBGMDAwMDAwM0MwMDAwMDBGMDAwMDAwM0MwMDAwMDBGMDAwMDAKMDND MDAwMDAwRjAwMDAwMDNDMDAwMDAwRjAwMDAwMDBGMDAwMDAwMDNDMDAwMDAwMEYwMDAwMDAw M0MwMDAwMDAwRjAwMDAwCjAwM0MwMDAwMDAwRjAwMDAwMDAzQzAwMDAwMDBGMDAwMDAwMDND MDAwMDAwMEYwMDAwMDAwMzgwMTkxODdEOTUyMD42MApEIEUgL0ZkIDM3IDEyMiBkZjwwMUYx QzAwM0YxQzAwN0YxQzAwNzAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAKMEUw MUMwMEUwMUMwRkZGMUMwRkZGMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUw MUMwMEUwMUMwMEUwMUMwCjBFMDFDMDBFMDFDMDBFMDFDMDBFMDFDMDBFMDFDMDBFMDFDMDBF MDFDMDBFMDFDMDEyMUQ4MDlDMTY+MTMKRDwwMDFDMDAwMDAwM0UwMDAwMDAzRTAwMDAwMDJF MDAwMDAwNjcwMDAwMDA2NzAwMDAwMEU3ODAwMDAwQzc4MDAwMDBDMzgwCjAwMDFDM0MwMDAw MTgzQzAwMDAxODFDMDAwMDM4MUUwMDAwMzgxRTAwMDA3MDBGMDAwMDcwMEYwMDAwNjAwRjAw MDBFMDA3OAowMDBGRkZGODAwMEZGRkY4MDAxQzAwM0MwMDFDMDAzQzAwMTgwMDNDMDAzODAw MUUwMDM4MDAxRTAwNzAwMDFGMDA3MDAwMEYKMDA3MDAwMEYwMEUwMDAwNzgwMTkxRDdGOUMx Qz42NSBEPDAwM0ZDMDAwRkZGMDAzQzBGMDA3ODAzMDBGMDAwMDFFMDAwMDNDCjAwMDAzQzAw MDA3ODAwMDA3ODAwMDA3ODAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAw MDBGMDAwMDBGMAowMDAwRjAwMDAwNzgwMDAwNzgwMDAwNzgwMDAwM0MwMDAwM0MwMDAwMUUw MDAwMEYwMDA4MDc4MDE4MDNDMDc4MDBGRkYwMDAKM0Y4MDE1MUY3RDlEMUI+NjcgRDxGRkZD MDBGRkZGMDBGMDBGODBGMDAzRTBGMDAxRjBGMDAwRjBGMDAwNzhGMDAwMzhGMDAwCjNDRjAw MDNDRjAwMDFDRjAwMDFFRjAwMDFFRjAwMDFFRjAwMDFFRjAwMDFFRjAwMDFFRjAwMDFFRjAw MDFFRjAwMDNDRjAwMAozQ0YwMDAzQ0YwMDA3OEYwMDBGMEYwMDBGMEYwMDNFMEYwMEZDMEZG RkYwMEZGRkMwMDE3MUQ3QzlDMUU+STxGRkZGQzBGRgpGRkMwRjAwMDAwRjAwMDAwRjAwMDAw RjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRkYKRkY4 MEZGRkY4MEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAw MEYwMDAwMEYwMDAwMEYwCjAwMDBGMDAwMDBGRkZGQzBGRkZGQzAxMjFEN0M5QzE5Pkk8MDAz RjgwMDFGRkYwMDNDMEY4MDc4MDM4MEYwMDE4MUUwMDAwCjNDMDAwMDNDMDAwMDc4MDAwMDc4 MDAwMDc4MDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDdGOApG MDA3RjhGMDAwMzg3ODAwMzg3ODAwMzg3ODAwMzgzQzAwMzgzQzAwMzgxRTAwMzgwRjAwMzgw NzgwMzgwM0MwRjgwMUZGRjAKMDAzRjgwMTUxRjdEOUQxQz43MSBEPEYwRjBGMEYwRjBGMEYw RjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwCkYwRjBGMEYwMDQxRDdDOUMw Qz43MyBEPEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAw MApGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAw RjAwMEYwMDBGRkZFRkZGRTBGMUQKN0M5QzE2Pjc2IEQ8RkMwMDA3RTBGQzAwMDdFMEZDMDAw N0UwRUUwMDBERTBFRTAwMERFMEVFMDAwREUwRTcwMDE5RTBFNzAwCjE5RTBFNzAwMTlFMEU3 ODAzOUUwRTM4MDMxRTBFM0MwNzFFMEUzQzA3MUUwRTFDMDYxRTBFMUMwNjFFMEUxRTBFMUUw RTFFMApFMUUwRTBFMEMxRTBFMEYxQzFFMEUwNzE4MUUwRTA3MTgxRTBFMDcxODFFMEUwM0Iw MUUwRTAzQjAxRTBFMDNCMDFFMEUwMUUKMDFFMEUwMUUwMUUwRTAxRTAxRTBFMDAwMDFFMDFC MUQ3QzlDMjQ+STxGQzAwNzBGQzAwNzBGRTAwNzBFRTAwNzBFRjAwNzAKRTcwMDcwRTcwMDcw RTc4MDcwRTM4MDcwRTNDMDcwRTNDMDcwRTFFMDcwRTFFMDcwRTBFMDcwRTBGMDcwRTA3MDcw RTA3ODcwCkUwNzg3MEUwM0M3MEUwM0M3MEUwMUM3MEUwMUU3MEUwMEU3MEUwMEU3MEUwMEY3 MEUwMDc3MEUwMDdGMEUwMDNGMEUwMDNGMAoxNDFEN0M5QzFEPkk8MDAzRjAwMDAwMUZGRTAw MDAzRkZGMDAwMDdDMEY4MDAwRjgwN0MwMDFFMDAxRTAwM0UwMDFGMDAzQwowMDBGMDA3ODAw MDc4MDc4MDAwNzgwNzgwMDA3ODBGMDAwMDNDMEYwMDAwM0MwRjAwMDAzQzBGMDAwMDNDMEYw MDAwM0MwRjAKMDAwM0MwRjAwMDAzQzBGMDAwMDNDMEY4MDAwN0MwNzgwMDA3ODA3ODAwMDc4 MDdDMDAwRjgwM0MwMDBGMDAzRTAwMUYwMDFGCjAwM0UwMDBGODA3QzAwMDdDMEY4MDAwM0ZG RjAwMDAxRkZFMDAwMDAzRjAwMDAxQTFGN0U5RDFGPkk8RkZGQzAwRkZGRjAwCkYwMEY4MEYw MDNDMEYwMDFFMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDFFMEYw MDNFMEYwMEZDMApGRkZGODBGRkZFMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBG MDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDAKRjAwMDAwRjAwMDAwRjAwMDAwMTQxRDdD OUMxQj5JPEZGRjgwMEZGRkYwMEYwMEY4MEYwMDNDMEYwMDFFMEYwMDBGMEYwMDAKRjBGMDAw RjBGMDAwRjBGMDAwRjBGMDAxRTBGMDAzRTBGMDBGQzBGRkZGODBGRkZGMDBGRkY4MDBGMDND MDBGMDFDMDBGMDFFCjAwRjAwRjAwRjAwRjAwRjAwNzgwRjAwNzgwRjAwM0MwRjAwM0MwRjAw MUUwRjAwMEYwRjAwMEYwRjAwMDc4MTUxRDdDOUMxQgo+ODIgRDwwM0Y4MDAwRkZFMDAxQzBG MDAzODA3MDA3MDAzMDA2MDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBGMDAwMDAKNzgw MDAwN0YwMDAwM0ZFMDAwMUZGQzAwMDdGRTAwMDFGRjAwMDAxRjgwMDAwNzgwMDAwM0MwMDAw M0MwMDAwMUMwMDAwMUMwCjAwMDFDMDAwMDFDMEMwMDE4MEUwMDM4MEYwMDcwMDdDMEUwMDFG RkMwMDA3RjAwMDEyMUY3RTlEMTc+STxGRkZGRkY4MEZGCkZGRkY4MDAwMUUwMDAwMDAxRTAw MDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwMDAwMAox RTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwMDAw MDFFMDAwMDAwMUUwMDAwMDAKMUUwMDAwMDAxRTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAw MDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwMDAwMDFFMDAwMDAwCjFFMDAwMDE5MUQ3RjlDMUM+ STxGMDAwNzBGMDAwNzBGMDAwNzBGMDAwNzBGMDAwNzBGMDAwNzBGMDAwNzBGMDAwNzBGMDAw CjcwRjAwMDcwRjAwMDcwRjAwMDcwRjAwMDcwRjAwMDcwRjAwMDcwRjAwMDcwRjAwMDcwRjAw MDcwRjAwMDcwRjAwMDcwRjAwMAo3MEYwMDA3MEYwMDA3MDc4MDBFMDc4MDBFMDNDMDFDMDFF MDM4MDBGMDc4MDA3RkUwMDAxRjgwMDE0MUU3QzlDMUQ+STxGMAowMEYwMDA3MEYwMDBGODAw NzBGMDAwRjgwMDcwNzgwMEI4MDA2MDc4MDE5QzAwRTA3ODAxOUMwMEUwNzgwMTlDMDBFMDND MDMKOUMwMUMwM0MwMzFFMDFDMDNDMDMwRTAxQzAxRTAzMEUwMTgwMUUwNzBGMDM4MDFFMDcw RjAzODAxRTA2MDcwMzgwMEYwNjA3CjAzMDAwRjBFMDc4NzAwMEYwQzA3ODcwMDA3MEMwMzg2 MDAwNzBDMDM4NjAwMDc4QzAzOEUwMDA3OTgwM0NFMDAwMzk4MDFDQwowMDAzOTgwMUNDMDAw Mzk4MDFDQzAwMDFEMDAxQzgwMDAxRDAwMEQ4MDAwMUYwMDBGODAwMDFFMDAwRjgwMDAwRTAw MDcwMDAKMjQxRDdGOUMyNz44NyBEPEY4MDAwMUUwN0MwMDAxQzAzRTAwMDM4MDFFMDAwNzgw MUYwMDA3MDAwRjgwMEUwMDA3ODAxRTAwCjA3QzAxQzAwMDNFMDNDMDAwMUUwMzgwMDAxRjA3 MDAwMDBGMEYwMDAwMDc4RTAwMDAwN0RDMDAwMDAzRkMwMDAwMDFGODAwMAowMDFGMDAwMDAw MEYwMDAwMDAwRjAwMDAwMDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAwMDBGMDAwMDAwMEYwMDAw MDAwRjAwMDAKMDAwRjAwMDAwMDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAxQjFEODA5QzFDPjg5 IEQ8MEZDMDNGRjA3RkY4NzAzODQwMUMwMDFDCjAwMUMwMEZDMEZGQzNGRkM3ODFDRTAxQ0Uw MUNFMDFDRjA3QzdGRkM3RkRDM0YxQzBFMTI3RTkxMTQ+OTcKRDwwN0UwMEZGODFGRkMzQzFD NzAwNDcwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDA3MDAwNzAwNDNDMUMxRkZDMEZGODA3 CkUwMEUxMjdFOTExMj45OSBEPDAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAw RTAwMEUwMDBFMEY4RTFGRUUzRgpGRTdDM0U3MDBFNzAwRUUwMEVFMDBFRTAwRUUwMEVFMDBF RTAwRUYwMEU3MDFFN0MzRTNGRkUxRkVFMEY4RTBGMUQ3RTlDMTUKPkk8MDdDMDFGRTAzRkYw Nzg3ODcwMTg2MDFDRkZGQ0ZGRkNGRkZDRTAwMEUwMDBFMDAwNzAwMDcwMDQzQzFDM0ZGQzFG RjgKMDdFMDBFMTI3RTkxMTI+STwwMEZDMDFGQzAzRkMwNzAwMEUwMDBFMDAwRTAwMEUwMDBF MDAwRTAwMEUwMEZGRTBGRkUwMEUKMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUw MDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTFEODA5QzBECj5JPDAzQzNDMDBGRkZD MDFGRkZDMDFDMzgwMDM4MUMwMDM4MUMwMDM4MUMwMDM4MUMwMDM4MUMwMDFDMzgwMDFGRjgw MDFGCkYwMDAzQkMwMDAzODAwMDAzODAwMDAxRkZDMDAxRkZGMDAzRkZGODA3MDA3ODBFMDAx QzBFMDAxQzBFMDAxQzBGMDAzQzA3QwowRjgwM0ZGRjAwMUZGRTAwMDdGODAwMTIxQjdGOTEx NT5JPEYwRjBGMEYwMDAwMDAwMDAwMDAwMDA3MDcwNzA3MDcwNzA3MAo3MDcwNzA3MDcwNzA3 MDcwNzA3MDcwMDQxRDdFOUMwQT4xMDUgRDxFM0YwM0YwMEVGRjhGRjgwRkZGREZGQzBGODFG ODFFMApGMDBGMDBFMEUwMEUwMEUwRTAwRTAwRTBFMDBFMDBFMEUwMEUwMEUwRTAwRTAwRTBF MDBFMDBFMEUwMEUwMEUwRTAwRTAwRTAKRTAwRTAwRTBFMDBFMDBFMEUwMEUwMEUwRTAwRTAw RTBFMDBFMDBFMDFCMTI3RDkxMjI+MTA5CkQ8RTNFMEVGRjBGRkY4RjgzQ0YwMUNFMDFDRTAx Q0UwMUNFMDFDRTAxQ0UwMUNFMDFDRTAxQ0UwMUNFMDFDRTAxQ0UwMUNFMAoxQzBFMTI3RDkx MTU+STwwM0YwMDAwRkZDMDAxRkZFMDAzQzBGMDA3ODA3ODA3MDAzODBFMDAxQzBFMDAxQzBF MDAxQzBFMAowMUMwRTAwMUMwRjAwM0MwNzAwMzgwNzgwNzgwM0MwRjAwMUZGRTAwMEZGQzAw MDNGMDAwMTIxMjdGOTExNT5JPEUzRTBFRgpGMEZGRjhGODdDRjAxQ0UwMUVFMDBFRTAwRUUw MEVFMDBFRTAwRUUwMEVFMDFDRjAzQ0Y4N0NGRkY4RUZGMEUzQzBFMDAwRTAKMDBFMDAwRTAw MEUwMDBFMDAwRTAwMEUwMDAwRjFBN0Q5MTE1Pkk8MDc4RTFGRUUzRkZFN0MzRTc4MUU3MDBF RTAwRUUwMEUKRTAwRUUwMEVFMDBFRTAwRUYwMEU3MDFFN0MzRTNGRkUxRkVFMEY4RTAwMEUw MDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFCjBGMUE3RTkxMTU+STxFMzgwRTc4MEVGODBG QzAwRjgwMEYwMDBGMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwCjAwRTAwMEUw MDBFMDAwMDkxMjdEOTEwRT5JPDFGQzAzRkYwN0ZGMEYwMzBFMDAwRTAwMEYwMDA3RjAwM0ZD MDFGRTAwMEYwCjAwMzgwMDM4ODAzOEYwNzhGRkYwN0ZFMDFGQzAwRDEyN0Y5MTEwPkk8MUMw MDFDMDAxQzAwMUMwMDFDMDAxQzAwRkZFMEZGCkUwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAw MUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAwMUMyMDFGRjAwRkYwMDdDMDBDMTg3Rgo5NzBGPkk8 RTAxQ0UwMUNFMDFDRTAxQ0UwMUNFMDFDRTAxQ0UwMUNFMDFDRTAxQ0UwMUNFMDFDRTAxQ0Uw MUNFMDdDRkZGQwo3RkRDM0YxQzBFMTI3RDkxMTU+STxFMDA3RTAwNzcwMDY3MDBFNzAwRTM4 MUMzODFDMzgxQzFDMzgxQzM4MUMzODBFNzAwRQo3MDA2NjAwNzYwMDdFMDAzQzAwM0MwMTAx MjdGOTExMz5JPDcwMDM4MDc4MDcwMDNDMEUwMDFDMUMwMDBFMUMwMDA3MzgwMAowM0YwMDAw MUUwMDAwMUMwMDAwMUUwMDAwM0YwMDAwNzM4MDAwRTE4MDAwRTFDMDAxQzBFMDAzODA3MDA3 MDAzODBGMDAzQzAKMTIxMjgwOTExMz4xMjAgRDxFMDA3RTAwNzcwMEU3MDBFNzgwRTM4MUMz ODFDMUMxODFDMzgwQzM4MEUzMDBFNzAwNjcwMDcKNjAwMzYwMDNDMDAxQzAwMUMwMDE4MDAx ODAwMzAwMDMwMDA3MDBGRTAwRkMwMEY4MDAxMDFBN0Y5MTEzPkkKRSAvRmUgMTAgNTggZGY8 MUYwMDMxODA2MEMwNDA0MEMwNjBDMDYwQzA2MEMwNjBDMDYwQzA2MEMwNjBDMDYwNDA0MDYw QzAKMzE4MDFGMDAwQjEwN0Y4RjBGPjQ4IEQ8MEMwMDNDMDBDQzAwMEMwMDBDMDAwQzAwMEMw MDBDMDAwQzAwMEMwMDBDMDAwQzAwCjBDMDAwQzAwMEMwMEZGODAwOTEwN0U4RjBGPkk8MUYw MDYxODA0MEMwODA2MEMwNjAwMDYwMDA2MDAwQzAwMTgwMDMwMDA2CjAwMEMwMDEwMjAyMDIw N0ZDMEZGQzAwQjEwN0Y4RjBGPkk8MUYwMDIxODA2MEMwNjBDMDAwQzAwMDgwMDE4MDBGMDAw MDgwCjAwNDAwMDYwQzA2MEMwNjA4MDQwNjA4MDFGMDAwQjEwN0Y4RjBGPkk8MDMwMDAzMDAw NzAwMEYwMDBCMDAxMzAwMzMwMDIzCjAwNDMwMEMzMDBGRkUwMDMwMDAzMDAwMzAwMDMwMDFG RTAwQjEwN0Y4RjBGPkk8MjA4MDNGMDAyQzAwMjAwMDIwMDAyMDAwCjJGMDAzMDgwMjA0MDAw NjAwMDYwMDA2MEMwNjA4MEMwNjE4MDFGMDAwQjEwN0Y4RjBGPkk8MDc4MDE4NDAzMEMwNjBD MDYwCjAwQzAwMENGMDBGMDgwRTA0MEMwNjBDMDYwQzA2MDQwNjA2MEMwMzA4MDFGMDAwQjEw N0Y4RjBGPkk8NDAwMDdGRTA3RkMwCjgwODA4MDgwMDEwMDAyMDAwNDAwMDQwMDBDMDAwODAw MDgwMDE4MDAxODAwMTgwMDE4MDAxODAwMEIxMTdFOTAwRj5JPDFGCjAwMzE4MDYwQzA2MEMw NjBDMDcxODAzRjAwMEYwMDMzODA2MUMwQzA2MEMwNjBDMDYwNDA0MDYwODAxRjAwMEIxMDdG OEYwRgo+STwxRjAwMzE4MDYwQzBDMDQwQzA2MEMwNjBDMDYwNDBFMDIxRTAxRTYwMDA2MDAw NDA2MEMwNjA4MDQzMDAzRTAwMEIxMAo3RjhGMEY+SSBFIC9GZiA1NCAxMjIgZGY8MDA3MDAw RjAwMUUwMDNDMDA3ODAwRjAwMUUwMDFDMDAzODAwMzgwMDcwMDA3MAowMDcwMDA3MDAwRTAw MEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDA3MDAwNzAwMDcwMDA3MDAwMzgwMDM4MDAx QzAwMUUKMDAwRjAwMDc4MDAzQzAwMUYwMDBGMDAwNzAwQzI0Nzk5RjE4PjQwIEQ8NjAwMEYw MDA3ODAwM0MwMDFFMDAwRjAwMDc4MDAzCjgwMDFDMDAxQzAwMEUwMDBFMDAwRTAwMEUwMDA3 MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMEUwMDBFMDAwRTAwMApFMDAxQzAwMUMw MDM4MDA3ODAwRjAwMUUwMDNDMDA3ODAwRjAwMDYwMDAwQzI0N0M5RjE4Pkk8MDFDMDAwMDFD MDAwMDFDMAowMDAxQzAwMEMxQzE4MEYxQzc4MEY5Q0Y4MDdGRkYwMDFGRkMwMDA3RjAwMDA3 RjAwMDFGRkMwMDdGRkYwMEY5Q0Y4MEYxQzcKODBDMUMxODAwMUMwMDAwMUMwMDAwMUMwMDAw MUMwMDAxMTE0N0Q5NzE4Pkk8MUMzRTdFN0YzRjFGMDcwRTFFN0NGODYwMDgKMEM3ODg1MTg+ NDQgRDwwN0Y4MDAxRkZFMDAzRkZGMDA3ODA3ODA3ODAzQzA3ODAxQzAzMDAxQzAwMDAxQzAw MDAzQzAwMDAzCjgwMDAwRjAwMDNGRjAwMDNGRTAwMDNGRjAwMDAwNzgwMDAwM0MwMDAwMUMw MDAwMEUwMDAwMEUwMDAwMEUwRjAwMEUwRjAwMApFMEYwMDFDMEYwMDNDMDdDMDc4MDNGRkYw MDFGRkUwMDAzRjgwMDEzMUM3RTlCMTg+NTEgRDwxRkZGODAzRkZGODAzRkZGODAKMzgwMDAw MzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwM0JGODAwM0ZGRTAw M0ZGRjAwM0MwNzgwCjE4MDNDMDAwMDFDMDAwMDBFMDAwMDBFMDYwMDBFMEYwMDBFMEYwMDBF MEUwMDFDMDcwMDNDMDdDMEY4MDNGRkYwMDFGRkMwMAowM0YwMDAxMzFDN0U5QjE4PjUzIEQ8 MDA3MDAwMDBGODAwMDBGODAwMDBEODAwMDBEODAwMDFEQzAwMDFEQzAwMDFEQzAwMDEKOEMw MDAzOEUwMDAzOEUwMDAzOEUwMDAzOEUwMDAzMDYwMDA3MDcwMDA3MDcwMDA3MDcwMDA3MDcw MDBGRkY4MDBGRkY4MDBGCkZGODAwRTAzODAwRTAzODAxQzAxQzAxQzAxQzA3RjA3RjBGRjhG Rjg3RjA3RjAxNTFDN0Y5QjE4PjY1CkQ8MDBGOEUwMDNGRUUwMDdGRkUwMEYwN0UwMUUwM0Uw M0MwMUUwMzgwMEUwNzAwMEUwNzAwMEUwNzAwMDAwRTAwMDAwRTAwMAowMEUwMDAwMEUwMDAw MEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMDcwMDAwMDcwMDBFMDcwMDBFMDM4MDBFMDNDMDBF MDFFMDEKQzAwRjA3QzAwN0ZGODAwM0ZFMDAwMEY4MDAxMzFDN0U5QjE4PjY3IEQ8N0ZGODAw RkZGRTAwN0ZGRjAwMUMwRjgwMUMwM0MwCjFDMDNDMDFDMDFFMDFDMDBFMDFDMDBFMDFDMDBG MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MAoxQzAwNzAxQzAw RjAxQzAwRTAxQzAwRTAxQzAxRTAxQzAxQzAxQzAzQzAxQzBGODA3RkZGMDBGRkZFMDA3RkY4 MDAxNDFDN0YKOUIxOD5JPEZGRkZGMEZGRkZGMEZGRkZGMDFDMDA3MDFDMDA3MDFDMDA3MDFD MDA3MDFDMDAwMDFDMDAwMDFDMEUwMDFDMEUKMDAxQzBFMDAxRkZFMDAxRkZFMDAxRkZFMDAx QzBFMDAxQzBFMDAxQzBFMDAxQzAwMDAxQzAwMDAxQzAwMzgxQzAwMzgxQzAwCjM4MUMwMDM4 MUMwMDM4RkZGRkY4RkZGRkY4RkZGRkY4MTUxQzdGOUIxOD5JPEZGRkZFMEZGRkZFMEZGRkZF MDFDMDBFMDFDCjAwRTAxQzAwRTAxQzAwRTAxQzAwMDAxQzAwMDAxQzFDMDAxQzFDMDAxQzFD MDAxRkZDMDAxRkZDMDAxRkZDMDAxQzFDMDAxQwoxQzAwMUMxQzAwMUMwMDAwMUMwMDAwMUMw MDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwRkZDMDAwRkZDMDAwRkZDMDAwMTMKMUM3RTlC MTg+STwwMUYxQzAwM0ZEQzAwRkZGQzAxRjBGQzAxQzAzQzAzODAzQzAzODAxQzA3MDAxQzA3 MDAxQzA3MDAwMDAKRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwRkYwRTAxRkYw RTAwRkYwNzAwMUMwNzAwMUMwNzAwM0MwMzgwM0MwCjM4MDNDMDFDMDdDMDFGMEZDMDBGRkZD MDAzRkRDMDAxRjFDMDE0MUM3RTlCMTg+STw3RjA3RjBGRjhGRjg3RjA3RjAxQzAxCkMwMUMw MUMwMUMwMUMwMUMwMUMwMUMwMUMwMUMwMUMwMUMwMUMwMUMwMUMwMUMwMUMwMUZGRkMwMUZG RkMwMUZGRkMwMUMwMQpDMDFDMDFDMDFDMDFDMDFDMDFDMDFDMDFDMDFDMDFDMDFDMDFDMDFD MDFDMDFDMDFDMDFDMDFDMDdGMDdGMEZGOEZGODdGMDcKRjAxNTFDN0Y5QjE4Pkk8N0ZGRjAw RkZGRjgwN0ZGRjAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDEKQzAw MDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAw MDAxQzAwMDAxQzAwMDAxCkMwMDAwMUMwMDAwMUMwMDAwMUMwMDA3RkZGMDBGRkZGODA3RkZG MDAxMTFDN0Q5QjE4Pkk8N0YwN0YwRkY4N0Y4N0YwN0YwCjFDMDNDMDFDMDc4MDFDMDcwMDFD MEUwMDFDMUUwMDFDM0MwMDFDMzgwMDFDNzAwMDFDRjAwMDFERjAwMDFERjAwMDFGQjgwMAox RkI4MDAxRjFDMDAxRTFDMDAxQzBFMDAxQzBFMDAxQzA3MDAxQzA3MDAxQzAzODAxQzAzODAx QzAxQzA3RjAzRjBGRjg3RjgKN0YwM0YwMTUxQzdGOUIxOD43NSBEPDdGRTAwMEZGRTAwMDdG RTAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFCjAwMDAwRTAwMDAwRTAwMDAw RTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAw RQowMDcwMEUwMDcwMEUwMDcwMEUwMDcwMEUwMDcwN0ZGRkYwRkZGRkYwN0ZGRkYwMTQxQzdG OUIxOD5JPEZDMDFGOEZFMDNGOApGRTAzRjgzQjA2RTAzQjA2RTAzQjA2RTAzQjA2RTAzQjhF RTAzQjhFRTAzOThDRTAzOThDRTAzOURDRTAzOURDRTAzOURDRTAKMzhEOEUwMzhEOEUwMzhG OEUwMzg3MEUwMzg3MEUwMzgwMEUwMzgwMEUwMzgwMEUwMzgwMEUwMzgwMEUwMzgwMEUwRkUw M0Y4CkZFMDNGOEZFMDNGODE1MUM3RjlCMTg+STw3RTA3RjBGRjBGRjg3RjA3RjAxRDgxQzAx RDgxQzAxRDgxQzAxREMxQzAxQ0MxCkMwMUNDMUMwMUNFMUMwMUNFMUMwMUNFMUMwMUM2MUMw MUM3MUMwMUM3MUMwMUMzMUMwMUMzOUMwMUMzOUMwMUMzOUMwMUMxOQpDMDFDMTlDMDFDMURD MDFDMERDMDFDMERDMDFDMERDMDdGMDdDMEZGODdDMDdGMDNDMDE1MUM3RjlCMTg+STwwRkY4 MDAzRgpGRTAwN0ZGRjAwNzgwRjAwNzAwNzAwRjAwNzgwRTAwMzgwRTAwMzgwRTAwMzgwRTAw MzgwRTAwMzgwRTAwMzgwRTAwMzgwRTAKMDM4MEUwMDM4MEUwMDM4MEUwMDM4MEUwMDM4MEUw MDM4MEUwMDM4MEUwMDM4MEUwMDM4MEYwMDc4MDcwMDcwMDc4MEYwMDdGCkZGMDAzRkZFMDAw RkY4MDAxMTFDN0Q5QjE4Pkk8RkZGRTAwRkZGRjgwRkZGRkMwMUMwM0MwMUMwMUUwMUMwMEUw MUMwMDcwCjFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDBFMDFDMDFFMDFDMDNDMDFGRkZD MDFGRkY4MDFGRkUwMDFDMDAwMDFDMDAwMAoxQzAwMDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAw MDAxQzAwMDBGRjgwMDBGRjgwMDBGRjgwMDAxNDFDN0Y5QjE4Pkk8MEZGOAowMDNGRkUwMDdG RkYwMDc4MEYwMDcwMDcwMEYwMDc4MEUwMDM4MEUwMDM4MEUwMDM4MEUwMDM4MEUwMDM4MEUw MDM4MEUwMDMKODBFMDAzODBFMDAzODBFMDAzODBFMDAzODBFMDAzODBFMDAzODBFMDAzODBF MUUzODBFMUUzODBGMEU3ODA3MEY3MDA3ODdGCjAwN0ZGRjAwM0ZGRTAwMEZGQzAwMDAxQzAw MDAxRTAwMDAwRTAwMDAwRjAwMDAwNzAwMDAwNzAwMTEyMjdEOUIxOD5JPDdGCkY4MDBGRkZF MDA3RkZGMDAxQzBGODAxQzAzODAxQzAzQzAxQzAxQzAxQzAxQzAxQzAxQzAxQzAzQzAxQzAz ODAxQzBGODAxRgpGRjAwMUZGRTAwMUZGRTAwMUMwRjAwMUMwNzAwMUMwMzgwMUMwMzgwMUMw MzgwMUMwMzgwMUMwMzgwMUMwMzlDMUMwMzlDMUMKMDM5QzdGMDFGOEZGODFGODdGMDBGMDE2 MUM3RjlCMTg+STwwM0YzODAxRkZGODAzRkZGODA3QzBGODA3MDA3ODBFMDAzODAKRTAwMzgw RTAwMzgwRTAwMDAwNzAwMDAwNzgwMDAwM0YwMDAwMUZGMDAwMDdGRTAwMDBGRjAwMDAwRjgw MDAwM0MwMDAwMUMwCjAwMDBFMDAwMDBFMDYwMDBFMEUwMDBFMEUwMDFFMEYwMDFDMEY4MDc4 MEZGRkY4MEZGRkUwMEU3RjgwMDEzMUM3RTlCMTg+Ckk8N0ZGRkY4RkZGRkY4RkZGRkY4RTA3 MDM4RTA3MDM4RTA3MDM4RTA3MDM4MDA3MDAwMDA3MDAwMDA3MDAwMDA3MDAwMDA3MAowMDAw NzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAw NzAwMDAwNzAwMDAwNzAKMDAwMDcwMDAwN0ZGMDAwN0ZGMDAwN0ZGMDAxNTFDN0Y5QjE4Pkk8 RkY4M0ZFRkY4M0ZFRkY4M0ZFMUMwMDcwMUMwMDcwMUMKMDA3MDFDMDA3MDFDMDA3MDFDMDA3 MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDMDA3MDFDCjAw NzAxQzAwNzAxQzAwNzAxQzAwNzAxQzAwNzAwRTAwRTAwRjAxRTAwNzgzQzAwM0ZGODAwMUZG MDAwMDdDMDAxNzFDODA5QgoxOD5JPEZGMDdGOEZGMDdGOEZGMDdGODFDMDFDMDFDMDFDMDFD MDFDMDFDMDFDMDBFMDM4MDBFMDM4MDBFMDM4MDBFMDM4MAowRjA3ODAwNzA3MDAwNzA3MDAw NzA3MDAwNzA3MDAwMzhFMDAwMzhFMDAwMzhFMDAwMzhFMDAwMThDMDAwMURDMDAwMURDMDAK MDFEQzAwMDBEODAwMDBGODAwMDBGODAwMDA3MDAwMTUxQzdGOUIxOD5JPEZFMDNGOEZFMDNG OEZFMDNGODcwMDA3MDcwMDAKNzA3MDAwNzAzODAwRTAzODAwRTAzODAwRTAzODAwRTAzODAw RTAzOEY4RTAzOEY4RTAzOURDRTAzOURDRTAxOURDQzAxOURDCkMwMTlEQ0MwMTk4Q0MwMUQ4 REMwMUQ4REMwMUQ4REMwMUQ4REMwMEQ4RDgwMEQwNTgwMEYwNzgwMEYwNzgwMEUwMzgwMTUx Qwo3RjlCMTg+STw3RjhGRTA3RjlGRTA3RjhGRTAwRTA3MDAwRjA3MDAwNzBFMDAwNzhFMDAw MzlDMDAwM0RDMDAwMUY4MDAwMQpGODAwMDBGMDAwMDBGMDAwMDA3MDAwMDBGMDAwMDBGODAw MDFGODAwMDFEQzAwMDM5RTAwMDM4RTAwMDcwRjAwMDcwNzAwMEUKMDc4MDBFMDM4MDFFMDND MDdGMDdGMEZGOEZGODdGMDdGMDE1MUM3RjlCMTg+STxGRjA3RjhGRjA3RjhGRjA3RjgxQzAx QzAKMUUwM0MwMEUwMzgwMEYwNzgwMDcwNzAwMDcwNzAwMDM4RTAwMDM4RTAwMDFEQzAwMDFE QzAwMDFEQzAwMDBGODAwMDBGODAwCjAwNzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAw NzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAxRkMwMDAzRkUwMDAxRkMwMAoxNTFDN0Y5QjE4Pkk8 RkZGOEZGRjhGRkY4RTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUw MDBFMAowMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAw MEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAKMDBGRkY4RkZGOEZGRjgwRDI0Nzc5RjE4PjkxIEQ8 RkZGOEZGRjhGRkY4MDAzODAwMzgwMDM4MDAzODAwMzgwMDM4MDAzODAwCjM4MDAzODAwMzgw MDM4MDAzODAwMzgwMDM4MDAzODAwMzgwMDM4MDAzODAwMzgwMDM4MDAzODAwMzgwMDM4MDAz ODAwMzgwMAozODAwMzgwMDM4MDAzODAwMzhGRkY4RkZGOEZGRjgwRDI0N0Y5RjE4PjkzIEQ8 MUZFMDAwM0ZGODAwN0ZGQzAwNzgxRTAwMzAKMEUwMDAwMDcwMDAwMDcwMDAwRkYwMDA3RkYw MDFGRkYwMDdGMDcwMDc4MDcwMEUwMDcwMEUwMDcwMEUwMDcwMEYwMEYwMDc4CjFGMDAzRkZG RjAxRkZCRjAwN0UxRjAxNDE0N0Q5MzE4Pjk3IEQ8MDFGRTAwMDdGRjAwMUZGRjgwM0UwNzgw MzgwMzAwNzAwMAowMDcwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAw MDcwMDAwMDcwMDFDMDM4MDFDMDNFMDNDMDFGRkYKODAwN0ZGMDAwMUZDMDAxMjE0N0Q5MzE4 Pjk5IEQ8MDAxRjgwMDAzRjgwMDAxRjgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwCjAwMDM4 MDAzRTM4MDBGRkI4MDFGRkY4MDNDMUY4MDM4MEY4MDcwMDc4MDcwMDM4MEUwMDM4MEUwMDM4 MEUwMDM4MEUwMDM4MApFMDAzODBFMDAzODA3MDA3ODA3MDA3ODAzODBGODAzQzFGODAxRkZG RjAwRkZCRjgwM0UzRjAxNTFDN0U5QjE4Pkk8MDFGMAowMDA3RkMwMDFGRkUwMDNFMEYwMDM4 MDc4MDcwMDM4MDcwMDM4MEUwMDFDMEUwMDFDMEZGRkZDMEZGRkZDMEZGRkZDMEUwMDAKMDA3 MDAwMDA3MDAxQzAzODAxQzAzRTAzQzAxRkZGODAwN0ZGMDAwMUZDMDAxMjE0N0Q5MzE4Pkk8 MDAxRjgwMDA3RkMwMDAKRkZFMDAwRTFFMDAxQzBDMDAxQzAwMDAxQzAwMDAxQzAwMDdGRkZD MEZGRkZDMEZGRkZDMDAxQzAwMDAxQzAwMDAxQzAwMDAxCkMwMDAwMUMwMDAwMUMwMDAwMUMw MDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDA3RkZGMDA3RgpG RjAwN0ZGRjAwMTMxQzdGOUIxOD5JPDAxRTFGMDA3RkZGODBGRkZGODFFMUUzMDFDMEUwMDM4 MDcwMDM4MDcwMDM4MDcwMAozODA3MDAzODA3MDAxQzBFMDAxRTFFMDAxRkZDMDAxRkY4MDAz OUUwMDAzODAwMDAxQzAwMDAxRkZFMDAxRkZGQzAzRkZGRTAKNzgwMUYwNzAwMDcwRTAwMDM4 RTAwMDM4RTAwMDM4RTAwMDM4NzgwMEYwN0UwM0YwMUZGRkMwMEZGRjgwMDFGQzAwMTUxRjdG CjkzMTg+STw3RTAwMDBGRTAwMDA3RTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAw MDAwRTNFMDAwRUZGODAwRkZGCkMwMEZDMUMwMEY4MEUwMEYwMEUwMEUwMEUwMEUwMEUwMEUw MEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUwMApFMDBFMDBFMDBFMDBFMDdG QzNGQ0ZGRTdGRTdGQzNGQzE3MUM4MDlCMTg+STwwMzgwMDAwN0MwMDAwN0MwMDAwN0MwMDAw Mwo4MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwN0ZDMDAwRkZDMDAwN0ZDMDAwMDFDMDAw MDFDMDAwMDFDMDAwMDFDMDAwMDEKQzAwMDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAw MDAxQzAwMDAxQzAwMDAxQzAwMDAxQzAwMEZGRkYwMEZGRkY4MEZGCkZGMDAxMTFEN0M5QzE4 Pkk8RkUwMDAwRkUwMDAwRkUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUz RkYwCjBFN0ZGMDBFM0ZGMDBFMDc4MDBFMEYwMDBFMUUwMDBFM0MwMDBFNzgwMDBFRjAwMDBG RjgwMDBGRkMwMDBGOUMwMDBGMEUwMAowRTBGMDAwRTA3MDAwRTAzODAwRTAzQzBGRkM3RjhG RkM3RjhGRkM3RjgxNTFDN0Y5QjE4PjEwNwpEPDdGRTAwMEZGRTAwMDdGRTAwMDAwRTAwMDAw RTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAKMDAwMEUwMDAw MEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAw MEUwMDAwMEUwCjAwMDBFMDAwN0ZGRkMwRkZGRkUwN0ZGRkMwMTMxQzdFOUIxOD5JPDdDRTBF MDAwRkZGQkY4MDA3RkZGRjgwMDFGMUYxQzAwCjFFMUUxQzAwMUUxRTFDMDAxQzFDMUMwMDFD MUMxQzAwMUMxQzFDMDAxQzFDMUMwMDFDMUMxQzAwMUMxQzFDMDAxQzFDMUMwMAoxQzFDMUMw MDFDMUMxQzAwMUMxQzFDMDAxQzFDMUMwMDdGMUYxRjAwRkZCRkJGODA3RjFGMUYwMDE5MTQ4 MTkzMTg+STw3RQozRTAwRkVGRjgwN0ZGRkMwMEZDMUMwMEY4MEUwMEYwMEUwMEUwMEUwMEUw MEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUKMDBFMDBFMDBFMDBFMDBFMDBFMDBFMDBF MDBFMDdGQzNGQ0ZGRTdGRTdGQzNGQzE3MTQ4MDkzMTg+STwwMUYwMDAwRkZFMDAKMUZGRjAw M0UwRjgwMzgwMzgwNzAwMUMwNzAwMUMwRTAwMEUwRTAwMEUwRTAwMEUwRTAwMEUwRTAwMEUw RjAwMUUwNzAwMUMwCjc4MDNDMDNDMDc4MDNFMEY4MDFGRkYwMDBGRkUwMDAxRjAwMDEzMTQ3 RTkzMTg+STw3RTNFMDBGRUZGODA3RkZGQzAwRkMxCkUwMEY4MEUwMEYwMDcwMEUwMDcwMEUw MDM4MEUwMDM4MEUwMDM4MEUwMDM4MEUwMDM4MEUwMDM4MEYwMDcwMEYwMDcwMEY4MApFMDBG QzFFMDBGRkZDMDBFRkY4MDBFM0UwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBF MDAwMDBFMDAwMDdGQzAKMDBGRkUwMDA3RkMwMDAxNTFFODA5MzE4Pkk8MDFFMzgwMDdGQjgw MUZGRjgwM0UxRjgwMzgwRjgwNzAwNzgwNzAwNzgwRTAKMDM4MEUwMDM4MEUwMDM4MEUwMDM4 MEUwMDM4MEUwMDM4MDcwMDc4MDcwMDc4MDM4MEY4MDNDMUY4MDFGRkY4MDBGRkI4MDAzCkUz ODAwMDAzODAwMDAzODAwMDAzODAwMDAzODAwMDAzODAwMDAzODAwMDAzODAwMDNGRjgwMDNG RjgwMDNGRjgxNTFFN0U5MwoxOD5JPDdGODdFMEZGOUZGMDdGQkZGODAzRjg3ODAzRjAzMDAz RTAwMDAzQzAwMDAzQzAwMDAzODAwMDAzODAwMDAzODAwMAowMzgwMDAwMzgwMDAwMzgwMDAw MzgwMDAwMzgwMDAwMzgwMDA3RkZFMDBGRkZGMDA3RkZFMDAxNTE0N0Y5MzE4Pkk8MDdGNwow MDNGRkYwMDdGRkYwMDc4MEYwMEUwMDcwMEUwMDcwMEUwMDcwMDdDMDAwMDdGRTAwMDFGRkMw MDAzRkUwMDAwMUYwMDYwMDcKODBFMDAzODBFMDAzODBGMDAzODBGODBGMDBGRkZGMDBGRkZD MDBFN0YwMDAxMTE0N0Q5MzE4Pkk8MDE4MDAwMDM4MDAwMDMKODAwMDAzODAwMDAzODAwMDdG RkZDMEZGRkZDMEZGRkZDMDAzODAwMDAzODAwMDAzODAwMDAzODAwMDAzODAwMDAzODAwMDAz CjgwMDAwMzgwMDAwMzgwMDAwMzgwNDAwMzgwRTAwMzgwRTAwMzgwRTAwMUMxQzAwMUZGQzAw MEZGODAwMDNFMDAxMzE5N0Y5OAoxOD5JPDdFMDdFMEZFMEZFMDdFMDdFMDBFMDBFMDBFMDBF MDBFMDBFMDBFMDBFMDBFMDBFMDBFMDBFMDBFMDBFMDBFMDBFMAowRTAwRTAwRTAwRTAwRTAw RTAwRTAwRTAwRTAxRTAwRjAzRTAwN0ZGRkMwM0ZGRkUwMUZDRkMxNzE0ODA5MzE4Pkk8N0Y4 RgpGMEZGOEZGODdGOEZGMDFFMDNDMDBFMDM4MDBFMDM4MDBFMDM4MDA3MDcwMDA3MDcwMDA3 MDcwMDAzOEUwMDAzOEUwMDAzOEUKMDAwMzhFMDAwMURDMDAwMURDMDAwMURDMDAwMEY4MDAw MEY4MDAwMDcwMDAxNTE0N0Y5MzE4Pkk8RkY4RkY4RkY4RkY4RkYKOEZGODM4MDBFMDM4MDBF MDM4MDBFMDFDMDFDMDFDMDFDMDFDNzFDMDFDRjlDMDFDRjlDMDFDRDlDMDFDRDlDMDBEREQ4 MDBECkREODAwREREODAwRDhEODAwRjhGODAwRjhGODAwNzA3MDAxNTE0N0Y5MzE4Pkk8N0Y4 RkYwN0Y5RkYwN0Y4RkYwMDcwNzAwCjA3OEUwMDAzOUUwMDAxREMwMDAxRjgwMDAwRjgwMDAw NzAwMDAwRjAwMDAwRjgwMDAxREMwMDAzOUUwMDAzOEUwMDA3MDcwMAowRjA3ODA3RjhGRjBG RjhGRjg3RjhGRjAxNTE0N0Y5MzE4Pkk8N0Y4RkYwRkY4RkY4N0Y4RkYwMEUwMUMwMEUwMzgw MEUwMwo4MDA3MDM4MDA3MDcwMDA3MDcwMDAzODcwMDAzODYwMDAzOEUwMDAxQ0UwMDAxQ0Uw MDAwQ0MwMDAwQ0MwMDAwREMwMDAwNzgKMDAwMDc4MDAwMDc4MDAwMDcwMDAwMDcwMDAwMDcw MDAwMEYwMDAwMEUwMDA3OUUwMDA3QkMwMDA3RjgwMDAzRjAwMDAxRTAwCjAwMTUxRTdGOTMx OD5JIEUgL0ZnIDUyIDEyMyBkZjwwMDdFMUYwMDAxQzFCMTgwMDMwM0UzQzAwNzAzQzNDMDBF MDNDMTgwCjBFMDFDMDAwMEUwMUMwMDAwRTAxQzAwMDBFMDFDMDAwMEUwMUMwMDAwRTAxQzAw MEZGRkZGQzAwMEUwMUMwMDAwRTAxQzAwMAowRTAxQzAwMDBFMDFDMDAwMEUwMUMwMDAwRTAx QzAwMDBFMDFDMDAwMEUwMUMwMDAwRTAxQzAwMDBFMDFDMDAwMEUwMUMwMDAKMEUwMUMwMDAw RTAxQzAwMDBFMDFDMDAwMEUwMUMwMDAwRTAxQzAwMDdGODdGQzAwMUExRDgwOUMxOD4xMQpE PDAwN0UwMDAxQzE4MDAzMDE4MDA3MDNDMDBFMDNDMDBFMDE4MDBFMDAwMDBFMDAwMDBFMDAw MDBFMDAwMDBFMDAwMEZGRkYKQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAx QzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxCkMwMEUwMUMwMEUwMUMwMEUw MUMwMEUwMUMwN0Y4N0Y4MTUxRDgwOUMxNz5JPDAwN0ZDMDAxQzFDMDAzMDNDMDA3MDNDMDBF CjAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzBGRkZGQzAwRTAxQzAw RTAxQzAwRTAxQzAwRTAxQzAwRQowMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMw MEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwN0YKQ0ZGODE1MUQ4MDlDMTc+ STwwMDNGMDdFMDAwMDFDMDlDMTgwMDAzODBGMDE4MDAwNzAxRjAzQzAwMEUwMUUwM0MwMDBF MDAKRTAxODAwMEUwMEUwMDAwMDBFMDBFMDAwMDAwRTAwRTAwMDAwMEUwMEUwMDAwMDBFMDBF MDAwMDBGRkZGRkZGQzAwMEUwMEUwCjFDMDAwRTAwRTAxQzAwMEUwMEUwMUMwMDBFMDBFMDFD MDAwRTAwRTAxQzAwMEUwMEUwMUMwMDBFMDBFMDFDMDAwRTAwRTAxQwowMDBFMDBFMDFDMDAw RTAwRTAxQzAwMEUwMEUwMUMwMDBFMDBFMDFDMDAwRTAwRTAxQzAwMEUwMEUwMUMwMDBFMDBF MDFDMDAKMEUwMEUwMUMwMDdGQzdGQ0ZGODAyMTFEODA5QzIzPkk8NjA2MEYwRjBGOEY4Njg2 ODA4MDgwODA4MDgwODEwMTAxMDEwMjAKMjA0MDQwODA4MDBEMEM3RjlDMTU+MzQgRDw2MEYw Rjg2ODA4MDgwODEwMTAyMDQwODAwNTBDN0M5QzBDPjM5CkQ8MDA0MDAwODAwMTAwMDIwMDA2 MDAwQzAwMEMwMDE4MDAxODAwMzAwMDMwMDA3MDAwNjAwMDYwMDA2MDAwRTAwMEUwMDBFMAow MEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMDYwMDA2MDAwNjAwMDcwMDAz MDAwMzAwMDE4MDAxODAwMEMKMDAwQzAwMDYwMDAyMDAwMTAwMDA4MDAwNDAwQTJBN0Q5RTEw Pkk8ODAwMDQwMDAyMDAwMTAwMDE4MDAwQzAwMEMwMDA2MDAKMDYwMDAzMDAwMzAwMDM4MDAx ODAwMTgwMDE4MDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAwMUMw CjAxQzAwMTgwMDE4MDAxODAwMzgwMDMwMDAzMDAwNjAwMDYwMDBDMDAwQzAwMTgwMDEwMDAy MDAwNDAwMDgwMDAwQTJBN0U5RQoxMD5JPDYwRjBGMDcwMTAxMDEwMTAyMDIwNDA4MDA0MEM3 QzgzMEM+NDQgRDxGRkUwRkZFMDBCMDI4MDg5MEU+STw2MEYwRjAKNjAwNDA0N0M4MzBDPkk8 MDAwMTAwMDMwMDA2MDAwNjAwMDYwMDBDMDAwQzAwMEMwMDE4MDAxODAwMTgwMDMwMDAzMDAw MzAKMDA2MDAwNjAwMEMwMDBDMDAwQzAwMTgwMDE4MDAxODAwMzAwMDMwMDAzMDAwNjAwMDYw MDBDMDAwQzAwMEMwMDE4MDAxODAwCjE4MDAzMDAwMzAwMDMwMDA2MDAwNjAwMDYwMDBDMDAw QzAwMDEwMjk3RTlFMTU+STwwMzAwMDcwMDNGMDBDNzAwMDcwMDA3CjAwMDcwMDA3MDAwNzAw MDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3 MDAwNwowMDA3MDAwNzAwMEY4MEZGRjgwRDFDN0M5QjE1PjQ5IEQ8MDAwNjAwMDAwMDA2MDAw MDAwMDYwMDAwMDAwRjAwMDAwMDBGMDAKMDAwMDBGMDAwMDAwMTc4MDAwMDAxNzgwMDAwMDE3 ODAwMDAwMjNDMDAwMDAyM0MwMDAwMDIzQzAwMDAwNDFFMDAwMDA0MUUwCjAwMDA0MUUwMDAw MDgwRjAwMDAwODBGMDAwMDE4MEY4MDAwMTAwNzgwMDAxRkZGODAwMDMwMDdDMDAwMjAwM0Mw MDAyMDAzQwowMDA2MDAzRTAwMDQwMDFFMDAwNDAwMUUwMDBDMDAxRjAwMUUwMDFGMDBGRjgw RkZGMDFDMUQ3RjlDMUY+NjUKRDxGRkZGRjgwRjAwNzgwRjAwMTgwRjAwMDgwRjAwMDgwRjAw MEMwRjAwMDQwRjAwMDQwRjAyMDQwRjAyMDAwRjAyMDAwRjAyCjAwMEYwNjAwMEZGRTAwMEYw NjAwMEYwMjAwMEYwMjAwMEYwMjAwMEYwMjAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYw MAowMDBGMDAwMDBGMDAwMDBGODAwMEZGRjgwMDE2MUM3RTlCMUI+NzAgRDxGRkYzRkZDMDBG MDAzQzAwMEYwMDNDMDAwRjAwM0MKMDAwRjAwM0MwMDBGMDAzQzAwMEYwMDNDMDAwRjAwM0Mw MDBGMDAzQzAwMEYwMDNDMDAwRjAwM0MwMDBGMDAzQzAwMEYwMDNDCjAwMEZGRkZDMDAwRjAw M0MwMDBGMDAzQzAwMEYwMDNDMDAwRjAwM0MwMDBGMDAzQzAwMEYwMDNDMDAwRjAwM0MwMDBG MDAzQwowMDBGMDAzQzAwMEYwMDNDMDAwRjAwM0MwMDBGMDAzQzAwMEYwMDNDMDBGRkYzRkZD MDFBMUM3RTlCMUY+NzIKRDxGRkYwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAw RjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGCjAwMEYwMDBGMDAwRjAwMEYwMDBG MDAwRjAwMEYwMDBGMDAwRjAwRkZGMDBDMUM3RjlCMEY+STxGRjgwMDBGRjgwMEY4MDAwCkY4 MDAwRjgwMDBGODAwMEJDMDAxNzgwMDBCQzAwMTc4MDAwQkMwMDE3ODAwMDlFMDAyNzgwMDA5 RTAwMjc4MDAwOEYwMDQ3OAowMDA4RjAwNDc4MDAwOEYwMDQ3ODAwMDg3ODA4NzgwMDA4Nzgw ODc4MDAwODc4MDg3ODAwMDgzQzEwNzgwMDA4M0MxMDc4MDAKMDgzQzEwNzgwMDA4MUUyMDc4 MDAwODFFMjA3ODAwMDgxRTIwNzgwMDA4MEY0MDc4MDAwODBGNDA3ODAwMDgwNzgwNzgwMDA4 CjA3ODA3ODAwMDgwNzgwNzgwMDA4MDMwMDc4MDAxQzAzMDA3ODAwRkY4MzA3RkY4MDIxMUM3 RTlCMjY+NzcKRDxGRjAwN0ZDMDBGODAwRTAwMEY4MDA0MDAwQkMwMDQwMDA5RTAwNDAwMDlF MDA0MDAwOEYwMDQwMDA4RjgwNDAwMDg3ODA0CjAwMDgzQzA0MDAwODNDMDQwMDA4MUUwNDAw MDgwRjA0MDAwODBGMDQwMDA4MDc4NDAwMDgwN0M0MDAwODAzQzQwMDA4MDFFNAowMDA4MDFF NDAwMDgwMEY0MDAwODAwN0MwMDA4MDA3QzAwMDgwMDNDMDAwODAwM0MwMDA4MDAxQzAwMDgw MDBDMDAxQzAwMEMKMDBGRjgwMDQwMDFBMUM3RTlCMUY+STwwMDNGODAwMDAwRTBFMDAwMDM4 MDM4MDAwNzAwMUMwMDBFMDAwRTAwMUMwMDA3MDAKM0MwMDA3ODAzODAwMDM4MDc4MDAwM0Mw NzgwMDAzQzA3MDAwMDFDMEYwMDAwMUUwRjAwMDAxRTBGMDAwMDFFMEYwMDAwMUUwCkYwMDAw MUUwRjAwMDAxRTBGMDAwMDFFMEYwMDAwMUUwNzAwMDAxQzA3ODAwMDNDMDc4MDAwM0MwMzgw MDAzODAzQzAwMDc4MAoxQzAwMDcwMDBFMDAwRTAwMDcwMDFDMDAwMzgwMzgwMDAwRTBFMDAw MDAzRjgwMDAxQjFFN0U5QzIwPkk8RkZGRjgwMEYwMApFMDBGMDA3ODBGMDAzQzBGMDAxQzBG MDAxRTBGMDAxRTBGMDAxRTBGMDAxRTBGMDAxRTBGMDAxQzBGMDAzQzBGMDA3ODBGMDAKRTAw RkZGODAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAw RjAwMDAwRjAwMDAwRjAwCjAwMEYwMDAwRkZGMDAwMTcxQzdFOUIxQz5JPDA3RTA4MDFDMTk4 MDMwMDU4MDcwMDM4MDYwMDE4MEUwMDE4MEUwMDA4MEUwCjAwODBFMDAwODBGMDAwMDBGODAw MDA3QzAwMDA3RkMwMDAzRkY4MDAxRkZFMDAwN0ZGMDAwMEZGODAwMDBGODAwMDA3QzAwMAow M0MwMDAwMUMwODAwMUMwODAwMUMwODAwMUMwQzAwMTgwQzAwMTgwRTAwMzAwRDAwMjAwQ0Mw QzAwODNGODAwMTIxRTdFOUMKMTc+ODMgRDw3RkZGRkZDMDcwMEYwMUMwNjAwRjAwQzA0MDBG MDA0MDQwMEYwMDQwQzAwRjAwMjA4MDBGMDAyMDgwMEYwMDIwCjgwMEYwMDIwMDAwRjAwMDAw MDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAwMDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAwMDBGMDAw MAowMDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAwMDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAwMDBG MDAwMDAwMEYwMDAwMDAwRjAwMDAKMDAxRjgwMDAwM0ZGRkMwMDFCMUM3RjlCMUU+STxGRkYw N0ZDMDBGMDAwRTAwMEYwMDA0MDAwRjAwMDQwMDBGMDAwNDAwMEYKMDAwNDAwMEYwMDA0MDAw RjAwMDQwMDBGMDAwNDAwMEYwMDA0MDAwRjAwMDQwMDBGMDAwNDAwMEYwMDA0MDAwRjAwMDQw MDBGCjAwMDQwMDBGMDAwNDAwMEYwMDA0MDAwRjAwMDQwMDBGMDAwNDAwMEYwMDA0MDAwRjAw MDQwMDBGMDAwNDAwMDcwMDA4MDAwNwo4MDA4MDAwMzgwMTAwMDAxODAxMDAwMDBDMDIwMDAw MDcwQzAwMDAwMUYwMDAwMUExRDdFOUIxRj5JPEZGRTBGRkUwRkYxRgowMDFGMDAzQzFFMDAx RTAwMTgwRjAwMUYwMDEwMEYwMDFGMDAxMDBGMDAxRjAwMTAwNzgwMUYwMDIwMDc4MDI3ODAy MDA3ODAKMjc4MDIwMDNDMDI3ODA0MDAzQzA0M0MwNDAwM0MwNDNDMDQwMDNFMDQzQzA0MDAx RTA4MUUwODAwMUUwODFFMDgwMDFFMDgxCkUwODAwMEYxMDBGMTAwMDBGMTAwRjEwMDAwRjEw MEYxMDAwMDc5MDBGQTAwMDA3QTAwN0EwMDAwN0EwMDdBMDAwMDNFMDA3QwowMDAwM0MwMDND MDAwMDNDMDAzQzAwMDAzQzAwM0MwMDAwMTgwMDE4MDAwMDE4MDAxODAwMDAxODAwMTgwMDI4 MUQ3RjlCMkIKPjg3IEQ8MDgwODEwMTAyMDIwNDA0MDQwNDA4MDgwODA4MDgwODBCMEIwRjhG ODc4NzgzMDMwMEQwQzdBOUMxNT45MgpEPDFGQzAwMDMwNzAwMDc4MzgwMDc4MUMwMDMwMUMw MDAwMUMwMDAwMUMwMDAxRkMwMDBGMUMwMDM4MUMwMDcwMUMwMDYwMUMKMDBFMDFDNDBFMDFD NDBFMDFDNDA2MDNDNDAzMDRFODAxRjg3MDAxMjEyN0U5MTE1Pjk3IEQ8RkMwMDAwMUMwMDAw MUMwMDAwCjFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAw MDFDN0MwMDFEODYwMDFFMDMwMDFDMDE4MAoxQzAxQzAxQzAwQzAxQzAwRTAxQzAwRTAxQzAw RTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwQzAxQzAxQzAxQzAxODAxRTAzMDAKMTkwNjAwMTBG ODAwMTMxRDdGOUMxNz5JPDA3RTAwQzMwMTg3ODMwNzg3MDMwNjAwMEUwMDBFMDAwRTAwMEUw MDBFMDAwRTAKMDA2MDAwNzAwNDMwMDQxODA4MEMzMDA3QzAwRTEyN0U5MTEyPkk8MDAzRjAw MDAwNzAwMDAwNzAwMDAwNzAwMDAwNzAwMDAKMDcwMDAwMDcwMDAwMDcwMDAwMDcwMDAwMDcw MDAwMDcwMDAzRTcwMDBDMTcwMDE4MEYwMDMwMDcwMDcwMDcwMDYwMDcwMEUwCjA3MDBFMDA3 MDBFMDA3MDBFMDA3MDBFMDA3MDBFMDA3MDA2MDA3MDA3MDA3MDAzMDA3MDAxODBGMDAwQzM3 MDAwN0M3RTAxMwoxRDdFOUMxNz5JPDAzRTAwQzMwMTgxODMwMEM3MDBFNjAwNkUwMDZGRkZF RTAwMEUwMDBFMDAwRTAwMDYwMDA3MDAyMzAwMgoxODA0MEMxODAzRTAwRjEyN0Y5MTEyPkk8 MDBGODAxOEMwNzFFMDYxRTBFMEMwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDBGRgpFMDBFMDAw RTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUw MDBFMDA3RkUwMEYKMUQ4MDlDMEQ+STwwMDAzODAwM0M0QzAwQzM4QzAxQzM4ODAxODE4MDAz ODFDMDAzODFDMDAzODFDMDAzODFDMDAxODE4MDAKMUMzODAwMEMzMDAwMTNDMDAwMTAwMDAw MzAwMDAwMTgwMDAwMUZGODAwMUZGRjAwMUZGRjgwMzAwMzgwNjAwMUMwQzAwMEMwCkMwMDBD MEMwMDBDMDYwMDE4MDMwMDMwMDFDMEUwMDA3RjgwMDEyMUM3RjkyMTU+STxGQzAwMDAxQzAw MDAxQzAwMDAxQzAwCjAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMw MDAwMUM3QzAwMUM4NzAwMUQwMzAwMUUwMzgwMUMwMwo4MDFDMDM4MDFDMDM4MDFDMDM4MDFD MDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDMKODBG RjlGRjAxNDFEN0Y5QzE3Pkk8MTgwMDNDMDAzQzAwMTgwMDAwMDAwMDAwMDAwMDAwMDAwMDAw MDAwMDAwMDBGQzAwMUMKMDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAx QzAwMUMwMDFDMDAxQzAwMUMwMDFDMDBGRjgwMDkxRDdGCjlDMEM+STwwMEMwMDFFMDAxRTAw MEMwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDBGRTAwMEUwMDBFMDAwRTAwMEUwCjAw RTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUw MDBFMDAwRTA2MEUwRjBDMApGMUMwNjE4MDNFMDAwQjI1ODM5QzBEPkk8RkMwMDAwMUMwMDAw MUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMAowMDFDMDAwMDFDMDAwMDFDMDAw MDFDM0ZDMDFDMEYwMDFDMEMwMDFDMDgwMDFDMTAwMDFDMjAwMDFDNDAwMDFDRTAwMDFERTAK MDAxRTcwMDAxQzc4MDAxQzM4MDAxQzNDMDAxQzFDMDAxQzBFMDAxQzBGMDAxQzBGODBGRjlG RTAxMzFEN0Y5QzE2Pkk8RkMKMDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFD MDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDCjAwMUMwMDFDMDAxQzAwMUMw MDFDMDAxQzAwMUMwMDFDMDAxQzAwRkY4MDA5MUQ3RjlDMEM+STxGQzdFMDdFMDAwMUM4Mzg4 CjM4MDAxRDAxOTAxODAwMUUwMUUwMUMwMDFDMDFDMDFDMDAxQzAxQzAxQzAwMUMwMUMwMUMw MDFDMDFDMDFDMDAxQzAxQzAxQwowMDFDMDFDMDFDMDAxQzAxQzAxQzAwMUMwMUMwMUMwMDFD MDFDMDFDMDAxQzAxQzAxQzAwMUMwMUMwMUMwMDFDMDFDMDFDMDAKMUMwMUMwMUMwMEZGOEZG OEZGODAyMTEyN0Y5MTI0Pkk8RkM3QzAwMUM4NzAwMUQwMzAwMUUwMzgwMUMwMzgwMUMwMzgw MUMKMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4 MDFDMDM4MDFDMDM4MEZGOUZGMDE0CjEyN0Y5MTE3Pkk8MDNGMDAwMEUxQzAwMTgwNjAwMzAw MzAwNzAwMzgwNjAwMTgwRTAwMUMwRTAwMUMwRTAwMUMwRTAwMUMwCkUwMDFDMEUwMDFDMDYw MDE4MDcwMDM4MDMwMDMwMDE4MDYwMDBFMUMwMDAzRjAwMDEyMTI3RjkxMTU+STxGQzdDMDAx RDg2CjAwMUUwMzAwMUMwMTgwMUMwMUMwMUMwMEMwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUw MUMwMEUwMUMwMEUwMUMwMUMwMUMwMQpDMDFDMDE4MDFFMDMwMDFEMDYwMDFDRjgwMDFDMDAw MDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDFDMDAwMEZGODAKMDAxMzFBN0Y5MTE3 Pkk8MDNDMTAwMEMzMzAwMTgwQjAwMzAwRjAwNzAwNzAwNzAwNzAwRTAwNzAwRTAwNzAwRTAw NzAwRTAKMDcwMEUwMDcwMEUwMDcwMDYwMDcwMDcwMDcwMDMwMEYwMDE4MEYwMDBDMzcwMDA3 QzcwMDAwMDcwMDAwMDcwMDAwMDcwMDAwCjA3MDAwMDA3MDAwMDA3MDAwMDA3MDAwMDNGRTAx MzFBN0U5MTE2Pkk8RkNFMDFEMzAxRTc4MUU3ODFDMzAxQzAwMUMwMDFDCjAwMUMwMDFDMDAx QzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAwRkZDMDBEMTI3RjkxMTA+STwxRjkwMzA3MDQw MzBDMDEwCkMwMTBFMDEwRjgwMDdGODAzRkUwMEZGMDAwRjg4MDM4ODAxOEMwMThDMDE4RTAx MEQwNjA4RkMwMEQxMjdGOTExMD5JPDA0CjAwMDQwMDA0MDAwNDAwMEMwMDBDMDAxQzAwM0Mw MEZGRTAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQzAwMUMwMDFDMDAxQwoxMDFDMTAxQzEw MUMxMDFDMTAwQzEwMEUyMDAzQzAwQzFBN0Y5OTEwPkk8RkMxRjgwMUMwMzgwMUMwMzgwMUMw MzgwMUMwMwo4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFDMDM4MDFD MDM4MDFDMDM4MDFDMDc4MDBDMDc4MDBFMUIKODAwM0UzRjAxNDEyN0Y5MTE3Pkk8RkYwN0Uw M0MwMzgwMUMwMTAwMUMwMTAwMEUwMjAwMEUwMjAwMDcwNDAwMDcwNDAwMDcKMDQwMDAzODgw MDAzODgwMDAzRDgwMDAxRDAwMDAxRDAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwNDAwMDEzMTI3 RjkxMTY+STwKRkYzRkNGRTAzQzBGMDM4MDFDMDcwMTgwMUMwNzAxMDAxQzBCMDEwMDBFMEI4 MjAwMEUwQjgyMDAwRTExODIwMDA3MTFDNDAwCjA3MTFDNDAwMDcyMEM0MDAwM0EwRTgwMDAz QTBFODAwMDNDMDY4MDAwMUMwNzAwMDAxQzA3MDAwMDE4MDMwMDAwMDgwMjAwMAoxQjEyN0Y5 MTFFPkk8N0Y4RkYwMEYwMzgwMEYwMzAwMDcwMjAwMDM4NDAwMDFDODAwMDFEODAwMDBGMDAw MDA3MDAwMDA3OAowMDAwRjgwMDAwOUMwMDAxMEUwMDAyMEUwMDA2MDcwMDA0MDM4MDFFMDdD MEZGMEZGODE1MTI4MDkxMTY+STxGRjA3RTAzQwowMzgwMUMwMTAwMUMwMTAwMEUwMjAwMEUw MjAwMDcwNDAwMDcwNDAwMDcwNDAwMDM4ODAwMDM4ODAwMDNEODAwMDFEMDAwMDEKRDAwMDAw RTAwMDAwRTAwMDAwRTAwMDAwNDAwMDAwNDAwMDAwODAwMDAwODAwMEYwODAwMEYxMDAwMEYz MDAwMDY2MDAwMDNDCjAwMDAxMzFBN0Y5MTE2Pkk8N0ZGQzcwMzg2MDM4NDA3MDQwRjA0MEUw NDFDMDAzQzAwMzgwMDcwMDBGMDQwRTA0MUMwNDNDCjBDMzgwODcwMDg3MDM4RkZGODBFMTI3 RjkxMTI+SSBFIC9GaCA4IDExOCBkZjw3OEZDRkNGQ0ZDNzgwMDAwMDAwMDAwMDA3OApGQ0ZD RkNGQzc4MDYxMjdEOTEwRD41OCBEPEZGRkZGODAwRkZGRkZGMDAwRkMwMUZDMDBGQzAwN0Uw MEZDMDAxRjAwRkMwMDEKRjgwRkMwMDBGODBGQzAwMEZDMEZDMDAwN0MwRkMwMDA3QzBGQzAw MDdFMEZDMDAwN0UwRkMwMDA3RTBGQzAwMDdFMEZDMDAwCjdFMEZDMDAwN0UwRkMwMDA3RTBG QzAwMDdFMEZDMDAwN0MwRkMwMDA3QzBGQzAwMDdDMEZDMDAwRjgwRkMwMDBGODBGQzAwMQpG MDBGQzAwN0UwMEZDMDFGQzBGRkZGRkYwMEZGRkZGODAwMUYxQzdFOUIyNT42OCBEPDAzRkMw MDBFMEUwMDFDMUYwMDNDMUYKMDA3ODFGMDA3ODBFMDBGODAwMDBGODAwMDBGODAwMDBGODAw MDBGODAwMDBGODAwMDA3ODAwMDA3ODAxODAzQzAxODAxQzAzCjAwMEUwRTAwMDNGODAwMTEx MjdFOTExNT45OSBEPDFFMDAzRjAwM0YwMDNGMDAzRjAwMUUwMDAwMDAwMDAwMDAwMDAwMDAw MAowMDAwMDBGRjAwRkYwMDFGMDAxRjAwMUYwMDFGMDAxRjAwMUYwMDFGMDAxRjAwMUYwMDFG MDAxRjAwMUYwMDFGMDAxRjAwRkYKRTBGRkUwMEIxRTdGOUQwRT4xMDUgRDxGRjBGQzBGRjMx RTAxRjQwRjAxRjgwRjgxRjgwRjgxRjAwRjgxRjAwRjgxRjAwRjgKMUYwMEY4MUYwMEY4MUYw MEY4MUYwMEY4MUYwMEY4MUYwMEY4MUYwMEY4MUYwMEY4RkZFN0ZGRkZFN0ZGMTgxMjdGOTEx Qj4KMTEwIEQ8MDFGQzAwMEYwNzgwMUMwMUMwM0MwMUUwNzgwMEYwNzgwMEYwRjgwMEY4Rjgw MEY4RjgwMEY4RjgwMEY4RjgwMEY4CkY4MDBGODc4MDBGMDc4MDBGMDNDMDFFMDFFMDNDMDBG MDc4MDAxRkMwMDE1MTI3RjkxMTg+STwxRkQ4MzA3ODYwMThFMDE4CkUwMThGMDAwRkY4MDdG RTA3RkYwMUZGODA3RkMwMDdDQzAxQ0MwMUNFMDFDRTAxOEY4MzBDRkMwMEUxMjdFOTExMz4x MTUKRDxGRjA3RjhGRjA3RjgxRjAwRjgxRjAwRjgxRjAwRjgxRjAwRjgxRjAwRjgxRjAwRjgx RjAwRjgxRjAwRjgxRjAwRjgxRjAwCkY4MUYwMEY4MUYwMEY4MUYwMUY4MEYwMUY4MDc4NkZG MDFGOEZGMTgxMjdGOTExQj4xMTcKRCBFIC9GaSA0NSAxMjIgZGY8MDA3MDAwRTAwMUMwMDM4 MDA3ODAwNzAwMEUwMDFFMDAxRTAwM0MwMDNDMDAzQzAwNzgwMDc4CjAwNzgwMDc4MDA3MDAw RjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwNzAwMDc4 MDA3OAowMDc4MDA3ODAwM0MwMDNDMDAzQzAwMUUwMDFFMDAwRTAwMDcwMDA3ODAwMzgwMDFD MDAwRTAwMDcwMEMyRTdFQTExMj40MApEPEUwMDA3MDAwMzgwMDFDMDAxRTAwMEUwMDA3MDAw NzgwMDc4MDAzQzAwM0MwMDNDMDAxRTAwMUUwMDFFMDAxRTAwMEUwMDAKRjAwMEYwMDBGMDAw RjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRTAwMUUwMDFFMDAxRTAwMUUw MDNDMDAzCkMwMDNDMDA3ODAwNzgwMDcwMDBFMDAxRTAwMUMwMDM4MDA3MDAwRTAwMDBDMkU3 REExMTI+STw3ODc4Nzg3ODE4MzAzMDYwCjYwRTAwNTBBN0Q4MzBEPjQ0IEQ8RkZDMEZGQzBG RkMwMEEwMzdGOEIwRj5JPDAwMUYwMDAwMDAxRjAwMDAwMDNGODAwMDAwCjNCODAwMDAwM0I4 MDAwMDA3QkMwMDAwMDczQzAwMDAwNzFDMDAwMDBGMUUwMDAwMEUxRTAwMDAwRTBFMDAwMDFF MEYwMDAwMQpFMEYwMDAwMUMwRjAwMDAzQzA3ODAwMDNDMDc4MDAwMzgwNzgwMDA3ODAzQzAw MDc4MDNDMDAwNzAwM0MwMDBGRkZGRTAwMEYKRkZGRTAwMEZGRkZFMDAxRTAwMEYwMDFFMDAw RjAwM0MwMDBGODAzQzAwMDc4MDNDMDAwNzgwNzgwMDA3QzA3ODAwMDNDMDc4CjAwMDNDMEYw MDAwM0UwMUIyMDdGOUYxRT42NSBEPDAwMUZDMDAwRkZGODAxRkZGQzAzRTAzQzA3ODAwQzBG MDAwMDFFMDAwMAozRTAwMDAzQzAwMDA3QzAwMDA3ODAwMDA3ODAwMDA3ODAwMDBGMDAwMDBG MDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDAKRjAwMDAwRjAwMDAwNzgwMDAwNzgwMDAw NzgwMDAwN0MwMDAwM0MwMDAwM0UwMDAwMUUwMDAwMEYwMDAyMDc4MDBFMDNFMDNFCjAxRkZG QzAwRkZGMDAwMUZDMDE3MjI3REEwMUQ+NjcgRDxGRkZDMDBGRkZGODBGRkZGQzBGMDA3RTBG MDAxRjBGMDAwRjhGMAowMDc4RjAwMDNDRjAwMDNDRjAwMDFFRjAwMDFFRjAwMDBFRjAwMDBG RjAwMDBGRjAwMDBGRjAwMDBGRjAwMDBGRjAwMDBGRjAKMDAwRkYwMDAwRkYwMDAwRkYwMDAx RUYwMDAxRUYwMDAxRUYwMDAzQ0YwMDA3Q0YwMDBGOEYwMDFGMEYwMDdFMEZGRkZDMEZGCkZG ODBGRkZDMDAxODIwN0I5RjIxPkk8RkZGRkMwRkZGRkMwRkZGRkMwRjAwMDAwRjAwMDAwRjAw MDAwRjAwMDAwRjAwMDAwCkYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEZG RkY4MEZGRkY4MEZGRkY4MEYwMDAwMEYwMDAwMEYwMDAwMApGMDAwMDBGMDAwMDBGMDAwMDBG MDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGRkZGRTBGRkZGRTBGRkZGRTAK MTMyMDdCOUYxQj5JPEZGRkZDMEZGRkZDMEZGRkZDMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAw MEYwMDAwMEYwMDAwMEYwMDAKMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGRkZGMDBGRkZG MDBGRkZGMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwCjAwRjAwMDAwRjAwMDAwRjAw MDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwMTIyMDdCOUYx QQo+STwwMDFGRTAwMEZGRjgwMUZGRkUwM0UwM0UwNzgwMEUwRjAwMDAxRTAwMDAzRTAwMDAz QzAwMDA3QzAwMDA3ODAwMDA3OAowMDAwNzgwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAw RjAwMDAwRjAwMDAwRjAwM0ZFRjAwM0ZFNzgwM0ZFNzgwMDFFNzgKMDAxRTdDMDAxRTNDMDAx RTNFMDAxRTFFMDAxRTBGMDAxRTA3ODAxRTAzRTAzRTAxRkZGRTAwRkZGODAwMUZDMDE3MjI3 REEwCjFFPkk8RjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBG MEYwRjBGMEYwRjBGMEYwRjBGMDA0CjIwN0M5RjBEPjczIEQ8RjAwMDNFRjAwMDdDRjAwMEY4 RjAwMUYwRjAwM0UwRjAwN0MwRjAwNzgwRjAwRjAwRjAxRjAwRjAzRQowMEYwN0MwMEYwRjgw MEYxRjAwMEYzRjgwMEYzRjgwMEY3RkMwMEZGQkMwMEZGMUUwMEZFMUYwMEZDMEYwMEY4MDc4 MEYwMDcKODBGMDAzQzBGMDAzRTBGMDAxRTBGMDAwRjBGMDAwRjhGMDAwNzhGMDAwM0NGMDAw M0NGMDAwMUVGMDAwMUYxODIwN0I5RjIwCj43NSBEPEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAw MEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMApGMDAwMDBGMDAw MDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAw MDBGMDAwMDAKRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRkZGRjgwRkZG RjgwRkZGRjgwMTEyMDdCOUYxOT5JPEY4MDAKMDFGOEZDMDAwM0Y4RkMwMDAzRjhGNDAwMDM3 OEY2MDAwNzc4RjYwMDA3NzhGNjAwMDc3OEYzMDAwRTc4RjMwMDBFNzhGMzAwCjBFNzhGMzgw MUU3OEYzODAxRTc4RjE4MDFDNzhGMUMwM0M3OEYxQzAzQzc4RjBDMDM4NzhGMEMwMzg3OEYw RTA3ODc4RjBFMAo3ODc4RjA2MDcwNzhGMDcwRjA3OEYwNzBGMDc4RjAzMEUwNzhGMDM5RTA3 OEYwMzlFMDc4RjAxOUMwNzhGMDE5QzA3OEYwMTkKQzA3OEYwMEY4MDc4RjAwRjgwNzhGMDBG ODA3OEYwMDAwMDc4MUQyMDdCOUYyOD5JPEZDMDA3OEZFMDA3OEZFMDA3OEY2MDAKNzhGNzAw NzhGNzAwNzhGMzgwNzhGMzgwNzhGMzgwNzhGM0MwNzhGMUMwNzhGMUUwNzhGMUUwNzhGMEUw NzhGMEYwNzhGMDcwCjc4RjA3MDc4RjA3ODc4RjAzODc4RjAzQzc4RjAzQzc4RjAxQzc4RjAx RTc4RjAwRTc4RjAwRTc4RjAwRTc4RjAwNzc4RjAwNwo3OEYwMDM3OEYwMDNGOEYwMDNGOEYw MDFGODE1MjA3QjlGMjA+STwwMDNGMDAwMDAwRkZDMDAwMDNGRkYwMDAwN0UxRjgwMAowRjgw N0MwMDFGMDAzRTAwMUUwMDFFMDAzQzAwMEYwMDNDMDAwRjAwNzgwMDA3ODA3ODAwMDc4MDc4 MDAwNzgwRjAwMDAzQzAKRjAwMDAzQzBGMDAwMDNDMEYwMDAwM0MwRjAwMDAzQzBGMDAwMDND MEYwMDAwM0MwRjAwMDAzQzBGMDAwMDNDMEY4MDAwN0MwCjc4MDAwNzgwNzgwMDA3ODA3ODAw MDc4MDNDMDAwRjAwM0MwMDBGMDAxRTAwMUUwMDFGMDAzRTAwMEY4MDdDMDAwN0UxRjgwMAow M0ZGRjAwMDAwRkZDMDAwMDAzRjAwMDAxQTIyN0RBMDIxPkk8RkZGODAwRkZGRjAwRkZGRjgw RjAwRkMwRjAwM0UwRjAwMQpFMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBG MEYwMDBGMEYwMDFFMEYwMDNFMEYwMEZDMEZGRkY4MEZGRkYKMDBGRkY4MDBGMDAwMDBGMDAw MDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAw CjAwRjAwMDAwRjAwMDAwMTQyMDdCOUYxRD5JPDAwM0YwMDAwMDBGRkMwMDAwM0ZGRjAwMDA3 RTFGODAwMEY4MDdDMDAxRjAwCjNFMDAxRTAwMUUwMDNDMDAwRjAwM0MwMDBGMDA3ODAwMDc4 MDc4MDAwNzgwNzgwMDA3ODBGMDAwMDNDMEYwMDAwM0MwRjAwMAowM0MwRjAwMDAzQzBGMDAw MDNDMEYwMDAwM0MwRjAwMDAzQzBGMDAwMDNDMEYwMDAwM0MwRjAwMDAzQzA3ODAwMDc4MDc4 MDAKMDc4MDc4MUUwNzgwM0MwRjBGMDAzQzBGOEYwMDFFMDc5RTAwMUYwM0ZFMDAwRjgzRkMw MDA3RTFGODAwMDNGRkYwMDAwMEZGCkY4MDAwMDNGNzgwMDAwMDA3QzAwMDAwMDNFMDAwMDAw MUUwMDAwMDAxRjAwMDAwMDBGODAxQTI3N0RBMDIxPkk8RkZGODAwCkZGRkYwMEZGRkY4MEYw MDdDMEYwMDNFMEYwMDFFMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDFFMEYw MDNFMApGMDA3QzBGRkZGODBGRkZGMDBGRkY4MDBGMDNDMDBGMDFFMDBGMDFFMDBGMDBGMDBG MDBGMDBGMDA3ODBGMDA3ODBGMDAzQzAKRjAwMUMwRjAwMUUwRjAwMEYwRjAwMEYwRjAwMDc4 RjAwMDc4RjAwMDNDMTYyMDdCOUYxRD5JPDAxRkMwMDA3RkY4MDBGRkYKQzAxRjAzQzAzQzAw QzAzQzAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3QzAwMDAzQzAwMDAzRjAw MDAxRkUwCjAwMEZGQzAwMDdGRTAwMDFGRjAwMDAzRjgwMDAwN0MwMDAwM0MwMDAwM0UwMDAw MUUwMDAwMUUwMDAwMUUwMDAwMUUwMDAwMQpFMDAwMDFDMEMwMDNDMEYwMDdDMEZDMEY4MDdG RkYwMDFGRkUwMDAzRjgwMDEzMjI3RUEwMTk+STxGRkZGRkZDMEZGRkZGRgpDMEZGRkZGRkMw MDAxRTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAw MDAwMDFFMDAKMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAx RTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwCjAwMDAxRTAwMDAwMDFFMDAwMDAwMUUwMDAw MDAxRTAwMDAwMDFFMDAwMDAwMUUwMDAwMDAxRTAwMDAwMDFFMDAwMDAwMUUwMAowMDAwMUUw MDAwMDAxRTAwMDAwMDFFMDAwMDFBMjA3RTlGMUY+STxGMDAwRjBGMDAwRjBGMDAwRjBGMDAw RjBGMDAwRjBGMAowMEYwRjAwMEYwRjAwMEYwRjAwMEYwRjAwMEYwRjAwMEYwRjAwMEYwRjAw MEYwRjAwMEYwRjAwMEYwRjAwMEYwRjAwMEYwRjAKMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYw MDBGMEYwMDBGMEYwMDBGMEYwMDBGMDc4MDFFMDc4MDFFMDNDMDNDMDNDMDNDMDFGCjBGODAw RkZGMDAwN0ZFMDAwMUY4MDAxNDIxN0I5RjFGPkk8RjAwMDdDMDAwRkYwMDA3RTAwMEZGMDAw N0UwMDBGNzgwMDZFCjAwMEU3ODAwRUUwMDFFNzgwMEU3MDAxRTc4MDBFNzAwMUUzQzAwRTcw MDNDM0MwMUU3ODAzQzNDMDFDNzgwM0MzQzAxQzM4MAozQzFFMDFDMzgwNzgxRTAzQzNDMDc4 MUUwMzgzQzA3ODBFMDM4MUMwNzAwRjAzODFDMEYwMEYwNzgxRTBGMDBGMDcwMUUwRjAKMDcw NzAwRTBFMDA3ODcwMEUxRTAwNzhGMDBFMUUwMDc4RTAwRjFFMDAzOEUwMDcxQzAwMzhFMDA3 MUMwMDNDRTAwNzNDMDAxCkRDMDA3MzgwMDFEQzAwM0I4MDAxREMwMDNCODAwMURDMDAzQjgw MDBGODAwM0YwMDAwRjgwMDFGMDAwMEY4MDAxRjAwMjgyMAo3RjlGMkI+ODcgRDxGRkZGRjhG RkZGRjhGRkZGRjgwMDAwRjAwMDAxRjAwMDAxRTAwMDAzQzAwMDA3QzAwMDA3ODAwMDBGODAK MDAwRjAwMDAxRTAwMDAzRTAwMDAzQzAwMDA3QzAwMDA3ODAwMDBGMDAwMDFGMDAwMDFFMDAw MDNFMDAwMDNDMDAwMDc4MDAwCjBGODAwMDBGMDAwMDFGMDAwMDFFMDAwMDNDMDAwMDdDMDAw MDc4MDAwMEZGRkZGQ0ZGRkZGQ0ZGRkZGQzE2MjA3RDlGMUM+CjkwIEQ8MDdFMDNGRjg3RkZD NzAxRTQwMUYwMDBGMDAwRjAwMEYwMDNGMDdGRjFGRkY3RTBGRjgwRkYwMEZGMDBGRjAwRkY4 CjNGN0ZGRjNGRUYxRjhGMTAxNDdFOTMxNj45NyBEPDAzRjAwRkZDMUZGRTNFMEUzQzAyNzgw MDc4MDBGMDAwRjAwMEYwMDBGMAowMEYwMDBGMDAwNzgwMDc4MDAzQzAxM0UwRjFGRkYwRkZF MDNGMDEwMTQ3RTkzMTQ+OTkgRDwwMDA3ODAwMDA3ODAwMDA3ODAKMDAwNzgwMDAwNzgwMDAw NzgwMDAwNzgwMDAwNzgwMDAwNzgwMDAwNzgwMDAwNzgwMDAwNzgwMDdDNzgwMEZGNzgwMUZG RjgwCjNFMUY4MDdDMDc4MDc4MDc4MEY4MDc4MEYwMDc4MEYwMDc4MEYwMDc4MEYwMDc4MEYw MDc4MEYwMDc4MEYwMDc4MDc4MDc4MAo3ODBGODAzRTFGODAxRkZGODAwRkY3ODAwN0M3ODAx MTIwN0U5RjE3Pkk8MDNGMDAwMEZGQzAwMUZGRTAwM0UxRjAwM0MwNwowMDc4MDcwMDcwMDM4 MEZGRkY4MEZGRkY4MEZGRkY4MEYwMDAwMEYwMDAwMEYwMDAwMDcwMDAwMDc4MDAwMDNDMDEw MDNFMDcKMDAxRkZGMDAwN0ZFMDAwMUY4MDAxMTE0N0Y5MzE0Pkk8MDA3RTAxRkUwM0ZFMDc4 MDA3MDAwRjAwMEYwMDBGMDAwRjAwMEYKMDAwRjAwMEYwMEZGRjBGRkYwRkZGMDBGMDAwRjAw MEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGCjAwMEYwMDBGMDAw RjAwMEYwMDBGMjA4MDlGMEU+STwwM0UwRjAwRkZGRjAxRkZGRjAzRTNFMDAzQzFFMDA3ODBG MDA3ODBGCjAwNzgwRjAwNzgwRjAwNzgwRjAwM0MxRTAwM0UzRTAwMUZGQzAwM0ZGODAwMzNF MDAwMzAwMDAwMzgwMDAwM0ZGRTAwM0ZGRgo4MDFGRkZDMDNGRkZFMDc4MDNGMEYwMDBGMEYw MDBGMEYwMDBGMEY4MDFGMDdFMDdFMDNGRkZDMDBGRkYwMDAzRkMwMDE0MUUKN0Y5MzE3Pkk8 RjBGMEYwRjAwMDAwMDAwMDAwMDAwMDAwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYw RjBGMEYwRjAKRjAwNDIwN0Q5RjBCPjEwNSBEPEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYw MDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAKMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDFGMDBG MDFFMDBGMDNDMDBGMDc4MDBGMEYwMDBGMUUwMDBGM0MwMDBGNzgwMDBGRkMwCjAwRkZDMDAw RkZFMDAwRjlGMDAwRjhGMDAwRjBGODAwRjA3QzAwRjA3QzAwRjAzRTAwRjAxRTAwRjAxRjAw RjAwRjgwMTEyMAo3RDlGMTY+MTA3IEQ8RjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYw RjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMApGMEYwRjAwNDIwN0Q5RjBCPkk8RjBGQzA3 RTBGM0ZFMUZGMEY3RkYzRkY4RkUwRkYwN0NGODA3QzAzQ0Y4MDdDMDNDRjAwNwo4MDNDRjAw NzgwM0NGMDA3ODAzQ0YwMDc4MDNDRjAwNzgwM0NGMDA3ODAzQ0YwMDc4MDNDRjAwNzgwM0NG MDA3ODAzQ0YwMDcKODAzQ0YwMDc4MDNDRjAwNzgwM0NGMDA3ODAzQ0YwMDc4MDNDMUUxNDdE OTMyNT5JPEYxRjhGM0ZDRjdGRUZDMUZGODBGRjgKMEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYw MEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBGRjAwRjEwMTQ3RDkzMTc+STwKMDFGODAw MDdGRTAwMUZGRjgwM0YwRkMwM0MwM0MwNzgwMUUwNzgwMUUwRjAwMEYwRjAwMEYwRjAwMEYw RjAwMEYwRjAwMEYwCkYwMDBGMDc4MDFFMDc4MDFFMDNDMDNDMDNGMEZDMDFGRkY4MDA3RkUw MDAxRjgwMDE0MTQ3RjkzMTc+STxGMUYwMDBGN0ZDCjAwRkZGRTAwRkMzRTAwRjgxRjAwRjAw RjAwRjAwRjgwRjAwNzgwRjAwNzgwRjAwNzgwRjAwNzgwRjAwNzgwRjAwNzgwRjAwRgowMEYw MEYwMEY4MUYwMEZDM0UwMEZGRkMwMEY3RjgwMEYxRTAwMEYwMDAwMEYwMDAwMEYwMDAwMEYw MDAwMEYwMDAwMEYwMDAKMDBGMDAwMDBGMDAwMDBGMDAwMDAxMTFEN0Q5MzE3Pkk8MDNDNzgw MEZGNzgwMUZGRjgwM0UxRjgwN0MwRjgwNzgwNzgwNzgKMDc4MEYwMDc4MEYwMDc4MEYwMDc4 MEYwMDc4MEYwMDc4MEYwMDc4MEY4MDc4MDc4MDc4MDdDMEY4MDNFMUY4MDFGRkY4MDBGCkY3 ODAwN0M3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3 ODAwMDA3ODAxMTFEN0U5MwoxNz5JPEYwRTBGM0UwRjdFMEZGMDBGQzAwRkMwMEY4MDBGODAw RjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMAowMEYwMDBGMDAwRjAwMDBCMTQ3 RDkzMTA+STwwN0YwMUZGQzNGRkM3ODBDNzgwMDc4MDA3ODAwN0MwMDNGQzAxRkYwMEZGOAow M0Y4MDA3QzAwM0MwMDNDQzAzQ0YwN0NGRkY4N0ZGMDBGQzAwRTE0N0Y5MzExPkk8MUUwMDFF MDAxRTAwMUUwMDFFMDAxRQowMEZGRjBGRkYwRkZGMDFFMDAxRTAwMUUwMDFFMDAxRTAwMUUw MDFFMDAxRTAwMUUwMDFFMDAxRTAwMUUwMDFFMDAxRTIwMUYKRjAwRkYwMDdDMDBDMUE3Rjk5 MTA+STxGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEYK RjAwRkYwMEZGMDBGRjAxRkYwM0ZGRkZGN0ZFRjNGMEYxMDE0N0Q5MzE3Pkk8RjAwM0MwRjAw M0MwNzgwMzgwNzgwNzgwNzgKMDc4MDNDMEYwMDNDMEYwMDNDMEYwMDFFMEUwMDFFMUUwMDFF MUUwMDBGMUMwMDBGM0MwMDBGM0MwMDA3MzgwMDA3MzgwMDA3CkI4MDAwM0YwMDAwM0YwMDAw MUUwMDAxMjE0N0Y5MzE1Pkk8NzgwMUUwN0MwM0MwM0UwNzgwMUUwRjAwMEYwRjAwMDc5RTAw CjAzRkMwMDAzRjgwMDAxRjgwMDAwRjAwMDAxRjAwMDAxRjgwMDAzRkMwMDA3OUUwMDBGMEYw MDBFMEYwMDFFMDc4MDNDMDNDMAo3ODAxRTBGODAxRjAxNDE0ODA5MzE1PjEyMCBEPEYwMDND MEYwMDNDMDc4MDc4MDc4MDc4MDdDMDc4MDNDMEYwMDNDMEYwMAoxRTBGMDAxRTFFMDAwRTFF MDAwRjFDMDAwRjFDMDAwNzNDMDAwNzM4MDAwM0I4MDAwM0I4MDAwM0IwMDAwMUYwMDAwMUYw MDAKMDBFMDAwMDBFMDAwMDFDMDAwMDFDMDAwMDFDMDAwMDM4MDAwMDc4MDAwN0YwMDAwN0Uw MDAwN0MwMDAwMTIxRDdGOTMxNT4KSSBFIC9GaiAxOCAxMTkgZGY8NzBGOEY4RjBFMDA1MDU3 Qjg0MEU+NDYgRDwwMDAwMDIwMDAwMDAwNjAwMDAwMDA2MDAwMDAwCjBFMDAwMDAwMUUwMDAw MDAxRTAwMDAwMDNGMDAwMDAwMkYwMDAwMDA0RjAwMDAwMDRGMDAwMDAwOEYwMDAwMDEwRjAw MDAwMQowRjAwMDAwMjBGMDAwMDAyMEYwMDAwMDQwRjAwMDAwQzBGMDAwMDA4MEYwMDAwMTAw RjAwMDAxMDBGMDAwMDIwMEY4MDAwM0YKRkY4MDAwNDAwNzgwMDBDMDA3ODAwMDgwMDc4MDAx MDAwNzgwMDEwMDA3ODAwMjAwMDc4MDAyMDAwNzgwMDYwMDA3ODAxRTAwCjBGODBGRjgwN0ZG ODFEMjA3RTlGMjI+NjUgRDwwMUZGRkZGRTAwMUUwMDFDMDAxRTAwMEMwMDFFMDAwNDAwMUUw MDA0MDAzQwowMDA0MDAzQzAwMDQwMDNDMDAwNDAwM0MwMDA0MDA3ODA4MDgwMDc4MDgwMDAw NzgwODAwMDA3ODE4MDAwMEYwMzAwMDAwRkYKRjAwMDAwRjAzMDAwMDBGMDMwMDAwMUUwMjAw MDAxRTAyMDAwMDFFMDIwMDAwMUUwMDAxMDAzQzAwMDIwMDNDMDAwMjAwM0MwCjAwNDAwM0Mw MDA0MDA3ODAwMDgwMDc4MDAxODAwNzgwMDEwMDA3ODAwNzAwMEYwMDFGMDBGRkZGRkUwMDFG MUY3RDlFMUY+CjY5IEQ8MDBGMTgwMDM4OUMwMDcwNzgwMEUwMzgwMUMwMzgwM0MwMzgwMzgw NzAwNzgwNzAwNzgwNzAwNzgwNzAwRjAwRTAwCkYwMEUwMEYwMEUwMEYwMEUyMEYwMUM0MEYw MUM0MDcwM0M0MDcwNUM0MDMwOEM4MDBGMDcwMDEzMTQ3QzkzMTc+OTcKRDwwMDdFMDAwMUMx MDAwMzAwODAwRTA3ODAxRTA3ODAxQzA3MDAzQzAyMDA3ODAwMDA3ODAwMDA3ODAwMDBGMDAw MDBGMDAwCjAwRjAwMDAwRjAwMDAwRjAwMDAwNzAwMTAwNzAwMjAwMzAwNDAwMTgzODAwMDdD MDAwMTExNDdDOTMxNT45OQpEPDAwMDA3ODAwMDNGODAwMDA3MDAwMDA3MDAwMDA3MDAwMDA3 MDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDFDMDAwMDEKQzAwMEYxQzAwMzg5QzAwNzA3 ODAwRTAzODAxQzAzODAzQzAzODAzODA3MDA3ODA3MDA3ODA3MDA3ODA3MDBGMDBFMDBGMDBF CjAwRjAwRTAwRjAwRTIwRjAxQzQwRjAxQzQwNzAzQzQwNzA1QzQwMzA4QzgwMEYwNzAwMTUy MDdDOUYxNz5JPDAwN0MwMUMyCjA3MDEwRTAxMUMwMTNDMDEzODAyNzgwQzdCRjA3QzAwRjAw MEYwMDBGMDAwRjAwMDcwMDA3MDAxNzAwMjM4MDQxODM4MDdDMAoxMDE0N0M5MzE1Pkk8MDAw MDc4MDAwMTlDMDAwMzNDMDAwMzNDMDAwNzE4MDAwNzAwMDAwNzAwMDAwRTAwMDAwRTAwMDAw RQowMDAwMEUwMDAwMEUwMDAxRkZFMDAwMUMwMDAwMUMwMDAwMUMwMDAwMUMwMDAwMzgwMDAw MzgwMDAwMzgwMDAwMzgwMDAwMzgKMDAwMDcwMDAwMDcwMDAwMDcwMDAwMDcwMDAwMDcwMDAw MDcwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEUwMDAwMEMwMDAwMUMwCjAwMDFDMDAwMDE4MDAw MzE4MDAwN0IwMDAwRjMwMDAwNjYwMDAwM0MwMDAwMTYyOTgyOUYwRT5JPDAwQzAwMUUwMDFF MDAxCkMwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDBFMDAzMzAwMjMwMDQzODA0MzAw NDcwMDg3MDAwRTAwMEUwMDBFMDAxQwowMDFDMDAxQzAwMzg0MDM4ODAzMDgwNzA4MDMxMDAz MzAwMUMwMDBCMUY3QzlFMEU+MTA1CkQ8MUMwRjgwMjYzMEMwNDc0MDYwNDc4MDYwNDcwMDcw NDcwMDcwOEUwMEUwMEUwMEUwMEUwMEUwMEUwMEUwMUMwMUMwMUMwMQpDMDFDMDFDMDFDMDM4 NDM4MDM4ODM4MDMwODM4MDcwODM4MDMxMDcwMDMzMDMwMDFDMDE2MTQ3QzkzMUE+MTEwCkQ8 MDA3QzAwMDFDMzAwMDMwMTgwMEUwMUMwMUUwMUMwMUMwMUUwM0MwMUUwNzgwMUUwNzgwMUUw NzgwMUUwRjAwM0MwRjAwMwpDMEYwMDNDMEYwMDc4MEYwMDcwMDcwMEYwMDcwMEUwMDMwMTgw MDE4NzAwMDA3QzAwMDEzMTQ3QzkzMTc+STwwMUMxRTAwMgo2MjE4MDQ3NDFDMDQ3ODFDMDQ3 MDFFMDQ3MDFFMDhFMDFFMDBFMDFFMDBFMDFFMDBFMDFFMDFDMDNDMDFDMDNDMDFDMDNDMDEK QzAzODAzODA3ODAzODA3MDAzODBFMDAzQzFDMDA3MjM4MDA3MUUwMDA3MDAwMDA3MDAwMDBF MDAwMDBFMDAwMDBFMDAwMDBFCjAwMDAxQzAwMDAxQzAwMDBGRkMwMDAxNzFEODA5MzE3Pkk8 MDBGMDQwMDM4OEMwMDcwNTgwMEUwMzgwMUMwMzgwM0MwMzgwCjM4MDcwMDc4MDcwMDc4MDcw MDc4MDcwMEYwMEUwMEYwMEUwMEYwMEUwMEYwMEUwMEYwMUMwMEYwMUMwMDcwM0MwMDcwNUMw MAozMEI4MDAwRjM4MDAwMDM4MDAwMDM4MDAwMDcwMDAwMDcwMDAwMDcwMDAwMDcwMDAwMEUw MDAwMEUwMDAwRkZFMDAxMjFEN0MKOTMxNT5JPDFDMUUwMDI2NjEwMDQ3ODM4MDQ3ODc4MDQ3 MDc4MDQ3MDMwMDhFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDFDMDAKMDAxQzAwMDAxQzAwMDAx QzAwMDAzODAwMDAzODAwMDAzODAwMDAzODAwMDA3MDAwMDAzMDAwMDAxMTE0N0M5MzEzPkk8 MDAKRkMwMzAyMDYwMTBDMDMwQzA3MEMwNjBDMDAwRjgwMEZGMDA3RjgwM0ZDMDAzRTAwMEU3 MDBFRjAwQ0YwMENFMDA4NDAxMDIwCjYwMUY4MDEwMTQ3RDkzMTM+STwwMTgwMDFDMDAzODAw MzgwMDM4MDAzODAwNzAwMDcwMEZGRjAwNzAwMEUwMDBFMDAwRTAwCjBFMDAxQzAwMUMwMDFD MDAxQzAwMzgwMDM4MDAzODAwMzgyMDcwNDA3MDQwNzA4MDcwODAzMTAwMUUwMDBDMUM3QzlC MEY+Ckk8MEUwMEMwMzMwMEUwMjMwMUMwNDM4MUMwNDMwMUMwNDcwMUMwODcwMzgwMEUwMzgw MEUwMzgwMEUwMzgwMUMwNzAwMUMwNwowMDFDMDcwMDFDMDcxMDFDMEUyMDE4MEUyMDE4MEUy MDFDMUUyMDBDMjY0MDA3QzM4MDE0MTQ3QzkzMTg+STwwRTAzODAzMwowNzgwMjMwN0MwNDM4 M0MwNDMwMUMwNDcwMEMwODcwMDgwMEUwMDgwMEUwMDgwMEUwMDgwMUMwMTAwMUMwMTAwMUMw MTAwMUMKMDIwMDFDMDIwMDFDMDQwMDFDMDQwMDFDMDgwMDBFMzAwMDAzQzAwMDEyMTQ3Qzkz MTU+SQpFIC9GayAzMCAxMTggZGY8MDAwMDc4MDA3Qzc4MDBGQzc4MDFGQzc4MDNDMDAwMDM4 MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDcKODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMEZG RkM3OEZGRkM3OEZGRkM3ODA3ODA3ODA3ODA3ODA3ODA3ODA3ODA3ODA3CjgwNzgwNzgwNzgw NzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgwNzgw NzgwNzgwNwo4MDc4MDc4MDc4MDc4MDc4MTUyNDgwQTMxQT4xMiBEPEYwRjBGMEYwMDQwNDdC ODMwRT40NgpEPDAwQzAwMUMwMDdDMEZGQzBGRkMwRkJDMDAzQzAwM0MwMDNDMDAzQzAwM0Mw MDNDMDAzQzAwM0MwMDNDMDAzQzAwM0MwMDMKQzAwM0MwMDNDMDAzQzAwM0MwMDNDMDAzQzAw M0MwMDNDMDAzQzAwM0MwMDNDMDAzQzAwM0MwRkZGRkZGRkZGRkZGMTAyMjdDCkExMTg+NDkg RDwwM0YwMDAwRkZDMDAxRkZFMDAzQzFGMDAzMDA3ODA3MDA3QzA2MDAzQzBFMDAzRTBDMDAx RTA0MDAxRTA0MAowMUUwMDAwMUUwMDAwMUUwMDAwMUUwMDAwM0MwMDAwM0MwMDAwNzgwMDAw NzgwMDAwRjAwMDAxRTAwMDAzQzAwMDA3ODAwMDAKRjAwMDAxRTAwMDAxQzAwMDAzODAwMDA3 MDAwMDBFMDAwMDFDMDAwMDM4MDAwMDcwMDAwMEZGRkZFMEZGRkZFMEZGRkZFMDEzCjIyN0VB MTE4Pkk8MDFGMDAwMDdGQzAwMUZGRjAwM0UwRjAwMzgwNzgwNzAwM0MwMjAwM0MwMjAwM0Mw MDAwM0MwMDAwM0MwCjAwMDNDMDAwMDc4MDAwMDc4MDAwMEYwMDAwMUUwMDAzRkMwMDAzRjgw MDAzRkUwMDAwMEYwMDAwMDc4MDAwMDNDMDAwMDNDMAowMDAxRTAwMDAxRTAwMDAxRTAwMDAx RTAwMDAxRTA4MDAxRTBDMDAzQzBFMDAzQzA3MDA3ODAzQzBGODAxRkZGMDAwRkZDMDAKMDNG MDAwMTMyMzdFQTExOD5JPDAwMUYwMDAwMUYwMDAwMkYwMDAwMkYwMDAwNkYwMDAwRUYwMDAw Q0YwMDAxQ0YwMDAxQ0YKMDAwMzhGMDAwMzhGMDAwNzhGMDAwNzBGMDAwRjBGMDAwRTBGMDAx RTBGMDAzQzBGMDAzQzBGMDA3ODBGMDA3ODBGMDBGMDBGCjAwRkZGRkY4RkZGRkY4RkZGRkY4 MDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRgow MDE1MjE3RkEwMTg+STxGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYw RjBGMEYwRjBGMEYwRjBGMApGMEYwRjBGMEYwRjBGMDA0MjM3Q0EyMEQ+NzMgRDxGQzAwMUVG RTAwMUVGRTAwMUVGRTAwMUVGNzAwMUVGNzAwMUVGMzgwMUUKRjM4MDFFRjNDMDFFRjFDMDFF RjFDMDFFRjFFMDFFRjBFMDFFRjBGMDFFRjBGMDFFRjA3ODFFRjA3ODFFRjAzODFFRjAzQzFF CkYwM0MxRUYwMUUxRUYwMUUxRUYwMEUxRUYwMEYxRUYwMDcxRUYwMDcxRUYwMDc5RUYwMDM5 RUYwMDM5RUYwMDFERUYwMDFERQpGMDAwRkVGMDAwRkVGMDAwRkVGMDAwN0UxNzIzN0JBMjIy Pjc4IEQ8RkZGRTAwRkZGRjgwRkZGRkMwRjAwM0UwRjAwMEYwRjAKMDA3OEYwMDA3OEYwMDAz Q0YwMDAzQ0YwMDAzQ0YwMDAzQ0YwMDAzQ0YwMDAzQ0YwMDA3OEYwMDA3OEYwMDBGMEYwMDNF MEZGCkZGQzBGRkZGODBGRkZFMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAw MDBGMDAwMDBGMDAwMDBGMDAwMDBGMAowMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAw MDAwMTYyMzdCQTIxRj44MCBEPDAwRkUwMDAzRkZDMDA3RkZFMDBGODEKRTAxRTAwNjAzQzAw MDAzQzAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3QzAwMDAzQzAwMDAzRjAw MDAxRkMwCjAwMEZGQzAwMDdGRjAwMDFGRjgwMDAzRkMwMDAwN0UwMDAwMUYwMDAwMEYwMDAw MEY4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMAo3ODAwMDA3ODAwMDA3OEMwMDBGMEUwMDBGMEY4 MDFFMDdFMDdDMDNGRkY4MDBGRkYwMDAxRkMwMDE1MjU3RUEzMUI+ODMKRDxGMDAwM0NGMDAw M0NGMDAwM0NGMDAwM0NGMDAwM0NGMDAwM0NGMDAwM0NGMDAwM0NGMDAwM0NGMDAwM0NGMDAw M0NGMDAwCjNDRjAwMDNDRjAwMDNDRjAwMDNDRjAwMDNDRjAwMDNDRjAwMDNDRjAwMDNDRjAw MDNDRjAwMDNDRjAwMDNDRjAwMDNDRjAwMAozQ0YwMDAzQ0YwMDAzQ0YwMDAzQzc4MDA3ODc4 MDA3ODc4MDA3ODNDMDBGMDFFMDFFMDFGODdFMDBGRkZDMDAzRkYwMDAwRkMKMDAxNjI0N0JB MjIxPjg1IEQ8MDdFMDFGRjgzRkZDMzgxRTIwMUUwMDBGMDAwRjAwMEYwMDBGMDBGRjA3RkYx RkZGM0UwRjc4CjBGRjAwRkYwMEZGMDBGRjAwRkY4M0Y3RkZGM0ZFRjFGOEYxMDE2N0U5NTE3 Pjk3IEQ8RjAwMDAwRjAwMDAwRjAwMDAwRjAwMAowMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAw MEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYwMDAwMEYxRjAwMEY3RkMwMEZGRkUKMDBGQzFG MDBGODBGMDBGMDA3ODBGMDA3ODBGMDAzQzBGMDAzQzBGMDAzQzBGMDAzQzBGMDAzQzBGMDAz QzBGMDAzQzBGMDAzCkMwRjAwNzgwRjAwNzgwRjgwRjAwRkMzRTAwRkZGRTAwRjdGODAwRjFG MDAwMTIyMzdDQTIxOT5JPDAxRkMwMDA3RkYwMDBGCkZGODAxRjAzODAzQzAxODA3ODAwMDA3 ODAwMDA3MDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDA3OAowMDAw NzgwMDAwNzgwMDAwM0MwMDQwMUYwM0MwMEZGRkMwMDdGRjgwMDFGQzAwMTIxNjdFOTUxNj5J PDAwMDNDMDAwMDNDMAowMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAz QzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwM0UzQzAKMEZGQkMwMUZGRkMwM0YwRkMwM0Mw N0MwNzgwM0MwNzgwM0MwRjAwM0MwRjAwM0MwRjAwM0MwRjAwM0MwRjAwM0MwRjAwM0MwCkYw MDNDMEYwMDNDMDc4MDNDMDc4MDNDMDNDMDdDMDNFMEZDMDFGRkZDMDBGRkJDMDAzRTNDMDEy MjM3RUEyMTk+STwwM0YwCjAwMDdGQzAwMUZGRTAwM0UwRjAwM0MwNzgwNzgwMzgwNzgwMzgw RjAwMUMwRkZGRkMwRkZGRkMwRkZGRkMwRjAwMDAwRjAwMAowMEYwMDAwMDcwMDAwMDc4MDAw MDc4MDAwMDNDMDA4MDFGMDc4MDBGRkY4MDA3RkYwMDAxRjgwMDEyMTY3RTk1MTY+STwwMAoz RjAwRkYwMUZGMDNDMDAzODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MEZGRjhG RkY4RkZGODA3ODAwNzgwMDcKODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3 ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMTAyMzgwCkEyMEY+STwwMUYwNzgwN0ZG RjgwRkZGRjgxRjFGMDAxRTBGMDAzQzA3ODAzQzA3ODAzQzA3ODAzQzA3ODAzQzA3ODAxRTBG CjAwMUYxRjAwMEZGRTAwMUZGQzAwMTlGMDAwMzgwMDAwMzgwMDAwM0MwMDAwMUZGRTAwMUZG RkMwMUZGRkUwM0ZGRkYwNzgwMQpGMDc4MDBGOEYwMDA3OEYwMDA3OEYwMDA3OEYwMDA3ODc4 MDBGMDNFMDNFMDFGRkZDMDBGRkY4MDAxRkMwMDE1MjE3Rjk1MTgKPkk8RjAwMEYwMDBGMDAw RjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYxRjhGM0ZDRjdGRUZF MUUKRjgwRkY4MEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBGRjAw RkYwMEZGMDBGRjAwRkYwMEZGMDBGCjEwMjM3Q0EyMTk+STxGMEYwRjBGMDAwMDAwMDAwMDAw MDAwMDAwMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwCkYwRjBGMEYwRjBGMDA0 MjM3REEyMEI+STxGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBG MDAwCjAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAxRjgwRjAzRjAwRjAzRTAw RjA3QzAwRjBGODAwRjFGMDAwRjNFMAowMEY3QzAwMEZGQzAwMEZGQzAwMEZGRTAwMEZGRTAw MEZERjAwMEY4RjgwMEYwNzgwMEYwN0MwMEYwM0UwMEYwMUUwMEYwMUYKMDBGMDBGMDBGMDBG ODBGMDA3QzAxMjIzN0NBMjE4PjEwNyBEPEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBG MEYwRjAKRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwRjBGMEYwMDQyMzdEQTIwQj5J PEYxRjhGM0ZDRjdGRUZFMUVGODBGRjgKMEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBG RjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBGMTAxNjdDCjk1MTk+MTEwIEQ8 MDFGQzAwMDdGRjAwMEZGRjgwMUYwN0MwM0MwMUUwNzgwMEYwNzgwMEYwNzAwMDcwRjAwMDc4 RjAwMDc4CkYwMDA3OEYwMDA3OEYwMDA3OEYwMDA3ODc4MDBGMDc4MDBGMDdDMDFGMDNFMDNF MDFGMDdDMDBGRkY4MDA3RkYwMDAxRkMwMAoxNTE2N0Y5NTE4Pkk8RjFGMDAwRjdGQzAwRkZG RTAwRkMzRjAwRjgwRjAwRjAwNzgwRjAwNzgwRjAwN0MwRjAwM0MwRjAwMwpDMEYwMDNDMEYw MDNDMEYwMDNDMEYwMDNDMEYwMDdDMEYwMDc4MEYwMEY4MEY4MEYwMEZDM0UwMEZGRkUwMEY3 RjgwMEYxRjAKMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBGMDAwMDBG MDAwMDBGMDAwMDBGMDAwMDAxMjIwN0M5NTE5Cj5JPDAzRTNDMDA3RkJDMDBGRkZDMDFGMEZD MDNDMDdDMDdDMDNDMDc4MDNDMEY4MDNDMEYwMDNDMEYwMDNDMEYwMDNDMEYwCjAzQzBGMDAz QzBGMDAzQzBGODAzQzA3ODAzQzA3ODAzQzAzQzA3QzAzRjBGQzAxRkZGQzAwRkZCQzAwM0Uz QzAwMDAzQzAwMAowM0MwMDAwM0MwMDAwM0MwMDAwM0MwMDAwM0MwMDAwM0MwMDAwM0MwMDAw M0MwMDAwM0MwMTIyMDdFOTUxOT5JPEYwRTBGMwpFMEY3RTBGRjAwRkUwMEZDMDBGODAwRjgw MEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAKMDBGMDAw RjAwMDBCMTY3Qzk1MTE+STwwN0YwMUZGQzNGRkUzQzBFNzgwNjc4MDA3ODAwN0MwMDNGMDAz RkYwMUZGODBGRkMKMDFGRTAwMUYwMDBGMDAwRjAwMEZDMDBGRjgxRUZGRkUzRkZDMEZGMDEw MTY3Rjk1MTM+STwwRjAwMEYwMDBGMDAwRjAwMEYKMDAwRjAwRkZGOEZGRjhGRkY4MEYwMDBG MDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwRjAwMEYwMDBGCjAwMEYw ODBGMUMwN0ZDMDdGODAzRTAwRTFDN0Y5QjEyPkk8RjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBG RjAwRkYwMEZGMDBGCkYwMEZGMDBGRjAwRkYwMEZGMDBGRjAwRkYwMEZGMDBGRjAxRkY4M0Y3 RkZGN0ZDRjFGMEYxMDE2N0M5NTE5PkkKRSAvRmwgNjUgMTIzIGRmPDAwMUY4M0UwMDBGMDZF MzAwMUMwNzg3ODAzODBGODc4MDMwMEYwMzAwNzAwNzAwMDA3MDA3MDAwCjA3MDA3MDAwMDcw MDcwMDAwNzAwNzAwMDA3MDA3MDAwMDcwMDcwMDBGRkZGRkY4MDA3MDA3MDAwMDcwMDcwMDAw NzAwNzAwMAowNzAwNzAwMDA3MDA3MDAwMDcwMDcwMDAwNzAwNzAwMDA3MDA3MDAwMDcwMDcw MDAwNzAwNzAwMDA3MDA3MDAwMDcwMDcwMDAKMDcwMDcwMDAwNzAwNzAwMDA3MDA3MDAwMDcw MDcwMDAwNzAwNzAwMDA3MDA3MDAwN0ZFM0ZGMDAxRDIwODA5RjFCPjExCkQ8MDAzRjAwMDBF MEMwMDFDMEMwMDM4MUUwMDcwMUUwMDcwMUUwMDcwMDAwMDcwMDAwMDcwMDAwMDcwMDAwMDcw MDAwMDcwMAowMEZGRkZFMDA3MDBFMDA3MDBFMDA3MDBFMDA3MDBFMDA3MDBFMDA3MDBFMDA3 MDBFMDA3MDBFMDA3MDBFMDA3MDBFMDA3MDAKRTAwNzAwRTAwNzAwRTAwNzAwRTAwNzAwRTAw NzAwRTAwNzAwRTAwNzAwRTA3RkMzRkUxNzIwODA5RjE5Pkk8NzAzOEY4N0MKRkM3RUZDN0U3 NDNBMDQwMjA0MDIwNDAyMDgwNDA4MDQxMDA4MTAwODIwMTA0MDIwMEYwRTdFOUYxNz4zNApE PDAwMjAwMDQwMDA4MDAxMDAwMjAwMDYwMDBDMDAwQzAwMTgwMDE4MDAzMDAwMzAwMDMwMDA3 MDAwNjAwMDYwMDA2MDAwRTAKMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUw MDBFMDAwRTAwMDYwMDA2MDAwNjAwMDcwMDAzMDAwMzAwMDMwCjAwMTgwMDE4MDAwQzAwMEMw MDA2MDAwMjAwMDEwMDAwODAwMDQwMDAyMDBCMkU3REExMTI+NDAKRDw4MDAwNDAwMDIwMDAx MDAwMDgwMDBDMDAwNjAwMDYwMDAzMDAwMzAwMDE4MDAxODAwMTgwMDFDMDAwQzAwMEMwMDBD MDAwCkUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEMw MDBDMDAwQzAwMUMwMDE4MDAxODAwMQo4MDAzMDAwMzAwMDYwMDA2MDAwQzAwMDgwMDEwMDAy MDAwNDAwMDgwMDAwQjJFN0RBMTEyPkk8NzBGOEZDRkM3NDA0MDQwNAowODA4MTAxMDIwNDAw NjBFN0M4NDBEPjQ0IEQ8RkZDMEZGQzAwQTAyN0Y4QTBGPkk8NzBGOEY4Rjg3MDA1MDU3Qzg0 MEQ+STwKMDAwMTAwMDMwMDAzMDAwNjAwMDYwMDA2MDAwQzAwMEMwMDBDMDAxODAwMTgwMDE4 MDAzMDAwMzAwMDMwMDA2MDAwNjAwMDYwCjAwQzAwMEMwMDBDMDAxODAwMTgwMDE4MDAzMDAw MzAwMDMwMDA2MDAwNjAwMDYwMDBDMDAwQzAwMEMwMDE4MDAxODAwMTgwMAozMDAwMzAwMDMw MDA2MDAwNjAwMDYwMDBDMDAwQzAwMEMwMDAxMDJEN0RBMTE3Pkk8MDNGMDAwMEUxQzAwMUMw RTAwMTgwNgowMDM4MDcwMDcwMDM4MDcwMDM4MDcwMDM4MDcwMDM4MEYwMDNDMEYwMDNDMEYw MDNDMEYwMDNDMEYwMDNDMEYwMDNDMEYwMDMKQzBGMDAzQzBGMDAzQzBGMDAzQzBGMDAzQzBG MDAzQzBGMDAzQzA3MDAzODA3MDAzODA3MDAzODA3ODA3ODAzODA3MDAxODA2CjAwMUMwRTAw MEUxQzAwMDNGMDAwMTIxRjdFOUQxNz5JPDAxODAwMzgwMEY4MEYzODAwMzgwMDM4MDAzODAw MzgwMDM4MDAzCjgwMDM4MDAzODAwMzgwMDM4MDAzODAwMzgwMDM4MDAzODAwMzgwMDM4MDAz ODAwMzgwMDM4MDAzODAwMzgwMDM4MDAzODAwMwo4MDA3QzBGRkZFMEYxRTdDOUQxNz5JPDAz RjAwMDBDMUMwMDEwMEUwMDIwMDcwMDQwMDc4MDgwMDc4MEYwMDdDMEY4MDNDMApGODAzQzBG ODAzQzAyMDA3QzAwMDA3QzAwMDA3ODAwMDA3ODAwMDBGMDAwMDBFMDAwMDFDMDAwMDM4MDAw MDcwMDAwMDYwMDAKMDBDMDAwMDE4MDAwMDMwMDAwMDYwMDQwMEMwMDQwMTgwMDQwMTAwMDgw M0ZGRjgwN0ZGRjgwRkZGRjgwMTIxRTdFOUQxNz4KSTwwM0YwMDAwQzFDMDAxMDBFMDAyMDBG MDA3ODBGODA3ODA3ODA3ODA3ODAzODBGODAwMDBGODAwMDBGMDAwMDBGMDAwMDBFCjAwMDAx QzAwMDAzODAwMDNGMDAwMDAzQzAwMDAwRTAwMDAwRjAwMDAwNzgwMDAwNzgwMDAwN0MwMjAw N0MwRjgwN0MwRjgwNwpDMEY4MDdDMEYwMDc4MDQwMDc4MDQwMEYwMDIwMEUwMDFDM0MwMDAz RjAwMDEyMUY3RTlEMTc+STwwMDA2MDAwMDA2MDAwMAowRTAwMDAwRTAwMDAxRTAwMDAyRTAw MDAyRTAwMDA0RTAwMDA4RTAwMDA4RTAwMDEwRTAwMDIwRTAwMDIwRTAwMDQwRTAwMDgKMEUw MDA4MEUwMDEwMEUwMDIwMEUwMDIwMEUwMDQwMEUwMEMwMEUwMEZGRkZGMDAwMEUwMDAwMEUw MDAwMEUwMDAwMEUwMDAwCjBFMDAwMDBFMDAwMDBFMDAwMEZGRTAxNDFFN0Y5RDE3Pkk8MTgw MzAwMUZGRTAwMUZGQzAwMUZGODAwMUZFMDAwMTAwMDAwCjEwMDAwMDEwMDAwMDEwMDAwMDEw MDAwMDEwMDAwMDExRjAwMDE2MUMwMDE4MEUwMDEwMDcwMDEwMDc4MDAwMDM4MDAwMDM4MAow MDAzQzAwMDAzQzAwMDAzQzA3MDAzQzBGMDAzQzBGMDAzQzBFMDAzODA0MDAzODA0MDA3MDAy MDA2MDAxMDBFMDAwQzM4MDAKMDNFMDAwMTIxRjdFOUQxNz5JPDAwN0MwMDAxODIwMDA3MDEw MDBFMDM4MDBDMDc4MDFDMDc4MDM4MDMwMDM4MDAwMDc4MDAKMDA3MDAwMDA3MDAwMDBGMUYw MDBGMjFDMDBGNDA2MDBGODA3MDBGODAzODBGODAzODBGMDAzQzBGMDAzQzBGMDAzQzBGMDAz CkMwRjAwM0MwNzAwM0MwNzAwM0MwNzAwMzgwMzgwMzgwMzgwNzAwMTgwNzAwMEMwRTAwMDYx QzAwMDFGMDAwMTIxRjdFOUQxNwo+STw0MDAwMDA3RkZGQzA3RkZGODA3RkZGODA0MDAxMDA4 MDAyMDA4MDAyMDA4MDA0MDAwMDA4MDAwMDA4MDAwMDEwMDAwMAoyMDAwMDAyMDAwMDA0MDAw MDA0MDAwMDBDMDAwMDBDMDAwMDFDMDAwMDE4MDAwMDM4MDAwMDM4MDAwMDM4MDAwMDM4MDAw MDcKODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDAzMDAwMDEyMUY3 RDlEMTc+STwwM0YwMDAwQzBDMDAKMTAwNjAwMzAwMzAwMjAwMTgwNjAwMTgwNjAwMTgwNjAw MTgwNzAwMTgwNzgwMzAwM0UwMzAwM0YwNjAwMUZDODAwMEZGMDAwCjAzRjgwMDA3RkMwMDBD N0UwMDEwM0YwMDMwMEY4MDYwMDM4MDQwMDFDMEMwMDFDMEMwMDBDMEMwMDBDMEMwMDBDMEMw MDA4MAo2MDAxODAyMDAxMDAxMDAyMDAwQzBDMDAwM0YwMDAxMjFGN0U5RDE3Pkk8MDNGMDAw MEUxODAwMUMwQzAwMzgwNjAwMzgwNwowMDcwMDcwMDcwMDM4MEYwMDM4MEYwMDM4MEYwMDND MEYwMDNDMEYwMDNDMEYwMDNDMEYwMDNDMDcwMDdDMDcwMDdDMDM4MDcKQzAxODBCQzAwRTEz QzAwM0UzQzAwMDAzODAwMDAzODAwMDAzODAwMDA3MDAzMDA3MDA3ODA2MDA3ODBFMDA3MDBD MDAyMDE4CjAwMTA3MDAwMEZDMDAwMTIxRjdFOUQxNz5JPDcwRjhGOEY4NzAwMDAwMDAwMDAw MDAwMDAwMDAwMDcwRjhGOEY4NzAwNTE0CjdDOTMwRD5JPDcwRjhGOEY4NzAwMDAwMDAwMDAw MDAwMDAwMDAwMDcwRjBGOEY4NzgwODA4MDgxMDEwMTAyMDIwNDAwNTFECjdDOTMwRD5JPDAw MDEwMDAwMDAwMzgwMDAwMDAzODAwMDAwMDM4MDAwMDAwN0MwMDAwMDA3QzAwMDAwMDdDMDAw MDAwOUUwCjAwMDAwOUUwMDAwMDA5RTAwMDAwMTBGMDAwMDAxMEYwMDAwMDEwRjAwMDAwMjA3 ODAwMDAyMDc4MDAwMDIwNzgwMDAwNDAzQwowMDAwNDAzQzAwMDA0MDNDMDAwMDgwMUUwMDAw ODAxRTAwMDBGRkZFMDAwMTAwMEYwMDAxMDAwRjAwMDEwMDBGMDAwMjAwMDcKODAwMjAwMDc4 MDAyMDAwNzgwMDQwMDAzQzAwRTAwMDNDMDFGMDAwN0UwRkZDMDNGRkUxRjIwN0Y5RjIyPjY1 CkQ8RkZGRkUwMDAwRjgwMzgwMDA3ODAxRTAwMDc4MDFGMDAwNzgwMEYwMDA3ODAwRjgwMDc4 MDBGODAwNzgwMEY4MDA3ODAwRgo4MDA3ODAwRjgwMDc4MDBGMDAwNzgwMUYwMDA3ODAxRTAw MDc4MDNDMDAwN0ZGRjAwMDA3ODAzQzAwMDc4MDFFMDAwNzgwMEYKMDAwNzgwMEY4MDA3ODAw NzgwMDc4MDA3QzAwNzgwMDdDMDA3ODAwN0MwMDc4MDA3QzAwNzgwMDdDMDA3ODAwNzgwMDc4 MDBGCjgwMDc4MDBGMDAwNzgwMUYwMDBGODAzQzAwRkZGRkYwMDAxQTFGN0U5RTIwPkk8RkZG RkZGMDAwRjgwMEYwMDA3ODAwMzAwCjA3ODAwMzAwMDc4MDAxMDAwNzgwMDE4MDA3ODAwMDgw MDc4MDAwODAwNzgwMDA4MDA3ODAwMDgwMDc4MDgwMDAwNzgwODAwMAowNzgwODAwMDA3ODA4 MDAwMDc4MTgwMDAwN0ZGODAwMDA3ODE4MDAwMDc4MDgwMDAwNzgwODAwMDA3ODA4MDAwMDc4 MDgwMDAKMDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAwMDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAw MDc4MDAwMDAwNzgwMDAwMDBGQzAwMDAwCkZGRkUwMDAwMTkxRjdFOUUxRT43MCBEPDAwMEZF MDIwMDA3ODE4NjAwMEUwMDRFMDAzODAwMkUwMDcwMDAxRTAwRjAwMDBFMAoxRTAwMDA2MDFF MDAwMDYwM0MwMDAwNjAzQzAwMDAyMDdDMDAwMDIwNzgwMDAwMjBGODAwMDAwMEY4MDAwMDAw RjgwMDAwMDAKRjgwMDAwMDBGODAwMDAwMEY4MDAwMDAwRjgwMDAwMDBGODAwN0ZGQ0Y4MDAw M0UwNzgwMDAxRTA3QzAwMDFFMDNDMDAwMUUwCjNDMDAwMUUwMUUwMDAxRTAxRTAwMDFFMDBG MDAwMUUwMDcwMDAxRTAwMzgwMDJFMDAwRTAwNDYwMDA3ODE4MjAwMDBGRTAwMAoxRTIxN0Q5 RjI0Pkk8RkZGOEZGRjgwRjgwMEY4MDA3ODAwRjAwMDc4MDBGMDAwNzgwMEYwMDA3ODAwRjAw MDc4MDBGMDAwNwo4MDBGMDAwNzgwMEYwMDA3ODAwRjAwMDc4MDBGMDAwNzgwMEYwMDA3ODAw RjAwMDc4MDBGMDAwN0ZGRkYwMDA3ODAwRjAwMDcKODAwRjAwMDc4MDBGMDAwNzgwMEYwMDA3 ODAwRjAwMDc4MDBGMDAwNzgwMEYwMDA3ODAwRjAwMDc4MDBGMDAwNzgwMEYwMDA3CjgwMEYw MDA3ODAwRjAwMDc4MDBGMDAwNzgwMEYwMDBGODAwRjgwRkZGOEZGRjgxRDFGN0U5RTIyPkk8 RkZGQzBGQzAwNzgwCjA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAw NzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MAowNzgwMDc4MDA3ODAwNzgwMDc4MDA3 ODAwNzgwMDc4MDBGQzBGRkZDMEUxRjdGOUUxMD5JPEZGRkMwRkZDMEZDMDAzRTAwNwo4MDAx ODAwNzgwMDEwMDA3ODAwMjAwMDc4MDA0MDAwNzgwMDgwMDA3ODAxMDAwMDc4MDIwMDAwNzgw NDAwMDA3ODA4MDAwMDcKODEwMDAwMDc4MzAwMDAwNzg3ODAwMDA3OEY4MDAwMDc5M0MwMDAw NzkxRTAwMDA3QTFFMDAwMDdDMEYwMDAwNzgwRjAwMDA3CjgwNzgwMDA3ODAzQzAwMDc4MDND MDAwNzgwMUUwMDA3ODAxRTAwMDc4MDBGMDAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3QzAwRgpD MDA3RTBGRkZDM0ZGQzFFMUY3RTlFMjM+NzUgRDxGRkZFMDAwRkMwMDAwNzgwMDAwNzgwMDAw NzgwMDAwNzgwMDAwNzgwMDAKMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAw MDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwCjA3ODAwMDA3ODAwMjA3ODAw MjA3ODAwMjA3ODAwMjA3ODAwNjA3ODAwNDA3ODAwNDA3ODAwQzA3ODAxQzBGODA3Q0ZGRkZG QwoxNzFGN0U5RTFDPkk8RkY4MDAwMUZGODBGODAwMDFGODAwNzgwMDAxRjAwMDVDMDAwMkYw MDA1QzAwMDJGMDAwNUMwMDAyRgowMDA0RTAwMDRGMDAwNEUwMDA0RjAwMDQ3MDAwOEYwMDA0 NzAwMDhGMDAwNDcwMDA4RjAwMDQzODAxMEYwMDA0MzgwMTBGMDAKMDQzODAxMEYwMDA0MUMw MjBGMDAwNDFDMDIwRjAwMDQxQzAyMEYwMDA0MEUwNDBGMDAwNDBFMDQwRjAwMDQwRTA0MEYw MDA0CjA3MDgwRjAwMDQwNzA4MEYwMDA0MDcwODBGMDAwNDAzOTAwRjAwMDQwMzkwMEYwMDA0 MDFFMDBGMDAwNDAxRTAwRjAwMDQwMQpFMDBGMDAwRTAwQzAwRjAwMUYwMEMwMUY4MEZGRTBD MUZGRjgyNTFGN0U5RTJBPkk8RkY4MDNGRjgwN0MwMDdDMDA3QzAwMwo4MDA1RTAwMTAwMDVF MDAxMDAwNEYwMDEwMDA0NzgwMTAwMDQ3ODAxMDAwNDNDMDEwMDA0M0MwMTAwMDQxRTAxMDAw NDBGMDEKMDAwNDBGMDEwMDA0MDc4MTAwMDQwNzgxMDAwNDAzQzEwMDA0MDFFMTAwMDQwMUUx MDAwNDAwRjEwMDA0MDBGMTAwMDQwMDc5CjAwMDQwMDNEMDAwNDAwM0QwMDA0MDAxRjAwMDQw MDFGMDAwNDAwMEYwMDA0MDAwNzAwMDQwMDA3MDAwRTAwMDMwMDFGMDAwMwowMEZGRTAwMTAw MUQxRjdFOUUyMj5JPDAwMUY4MDAwMDBGMEYwMDAwMUMwMzgwMDA3ODAxRTAwMEYwMDBGMDAw RTAwMDcwMAoxRTAwMDc4MDNDMDAwM0MwM0MwMDAzQzA3QzAwMDNFMDc4MDAwMUUwNzgwMDAx RTBGODAwMDFGMEY4MDAwMUYwRjgwMDAxRjAKRjgwMDAxRjBGODAwMDFGMEY4MDAwMUYwRjgw MDAxRjBGODAwMDFGMEY4MDAwMUYwNzgwMDAxRTA3QzAwMDNFMDdDMDAwM0UwCjNDMDAwM0Mw M0MwMDAzQzAxRTAwMDc4MDBFMDAwNzAwMEYwMDBGMDAwNzgwMUUwMDAxQzAzODAwMDBGMEYw MDAwMDFGODAwMAoxQzIxN0Q5RjIzPkk8RkZGRkUwMDAwRjgwNzgwMDA3ODAxQzAwMDc4MDFF MDAwNzgwMEYwMDA3ODAwRjgwMDc4MDBGODAwNwo4MDBGODAwNzgwMEY4MDA3ODAwRjgwMDc4 MDBGODAwNzgwMEYwMDA3ODAxRTAwMDc4MDFDMDAwNzgwNzgwMDA3RkZFMDAwMDcKODAwMDAw MDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAwMDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAwMDc4MDAw MDAwNzgwMDAwMDA3CjgwMDAwMDA3ODAwMDAwMDc4MDAwMDAwNzgwMDAwMDBGQzAwMDAwRkZG QzAwMDAxOTFGN0U5RTFGPkk8RkZGRjgwMDAwRjgwCkYwMDAwNzgwNzgwMDA3ODAzQzAwMDc4 MDFFMDAwNzgwMUUwMDA3ODAxRjAwMDc4MDFGMDAwNzgwMUYwMDA3ODAxRjAwMDc4MAoxRTAw MDc4MDFFMDAwNzgwM0MwMDA3ODA3ODAwMDc4MEYwMDAwN0ZGODAwMDA3ODFDMDAwMDc4MEUw MDAwNzgwRjAwMDA3ODAKNzAwMDA3ODA3ODAwMDc4MDc4MDAwNzgwNzgwMDA3ODA3QzAwMDc4 MDdDMDAwNzgwN0MwMDA3ODA3QzA0MDc4MDdFMDQwNzgwCjNFMDQwRkMwMUUwOEZGRkMwRjEw MDAwMDAzRTAxRTIwN0U5RTIxPjgyIEQ8MDdFMDgwMEMxOTgwMTAwNzgwMzAwMzgwNjAwMQo4 MDYwMDE4MEUwMDE4MEUwMDA4MEUwMDA4MEUwMDA4MEYwMDAwMEYwMDAwMDc4MDAwMDdGMDAw MDNGRjAwMDFGRkMwMDBGRkUKMDAwM0ZGMDAwMDFGODAwMDA3ODAwMDAzQzAwMDAzQzAwMDAx QzA4MDAxQzA4MDAxQzA4MDAxQzA4MDAxQzBDMDAxODBDMDAzCjgwRTAwMzAwRjAwNjAwQ0Uw QzAwODFGODAwMTIyMTdEOUYxOT5JPDdGRkZGRkUwNzgwRjAxRTA2MDBGMDA2MDQwMEYwMDIw CjQwMEYwMDIwQzAwRjAwMzA4MDBGMDAxMDgwMEYwMDEwODAwRjAwMTA4MDBGMDAxMDAwMEYw MDAwMDAwRjAwMDAwMDBGMDAwMAowMDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAwMDBGMDAwMDAw MEYwMDAwMDAwRjAwMDAwMDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAKMDAwRjAwMDAwMDBGMDAw MDAwMEYwMDAwMDAwRjAwMDAwMDBGMDAwMDAwMEYwMDAwMDAwRjAwMDAwMDFGODAwMDA3RkZG RTAwCjFDMUY3RTlFMjE+STxGRkYwN0ZGODFGRjAxRjgwMEZDMDA3QzAwRjAwMDc4MDAzODAw RjAwMDc4MDAxMDAwRjAwMDdDMDAxCjAwMDc4MDA3QzAwMjAwMDc4MDA3QzAwMjAwMDc4MDA3 QzAwMjAwMDNDMDA5RTAwNDAwMDNDMDA5RTAwNDAwMDNDMDA5RTAwNAowMDAzRTAxMEYwMEMw MDAxRTAxMEYwMDgwMDAxRTAxMEYwMDgwMDAxRjAyMDc4MDgwMDAwRjAyMDc4MTAwMDAwRjAy MDc4MTAKMDAwMEYwNDAzQzEwMDAwMDc4NDAzQzIwMDAwMDc4NDAzQzIwMDAwMDc4QzAzRTIw MDAwMDNDODAxRTQwMDAwMDNDODAxRTQwCjAwMDAzQzgwMUU0MDAwMDAxRjAwMEY4MDAwMDAx RjAwMEY4MDAwMDAxRjAwMEY4MDAwMDAxRTAwMDc4MDAwMDAwRTAwMDcwMAowMDAwMEUwMDA3 MDAwMDAwMEMwMDAzMDAwMDAwMDQwMDAyMDAwMDJDMjA3RjlFMkY+ODcgRDw3RkY4M0ZGODBG RTAwRkMwMDcKQzAwNzAwMDNDMDAyMDAwMUUwMDQwMDAxRjAwQzAwMDBGMDA4MDAwMDc4MTAw MDAwN0MxMDAwMDAzQzIwMDAwMDNFNDAwMDAwCjFFNDAwMDAwMEY4MDAwMDAwRjgwMDAwMDA3 ODAwMDAwMDNDMDAwMDAwN0UwMDAwMDA1RTAwMDAwMDlGMDAwMDAxOEY4MDAwMAoxMDc4MDAw MDIwN0MwMDAwNjAzQzAwMDA0MDFFMDAwMDgwMUYwMDAxODAwRjAwMDEwMDA3ODAwMjAwMDdD MDA3MDAwM0MwMUYKODAwN0UwRkZFMDFGRkUxRjFGN0Y5RTIyPkk8MDgwNDEwMDgyMDEwMjAx MDQwMjA0MDIwODA0MDgwNDA4MDQwQjg1Q0ZDN0UKRkM3RTdDM0UzODFDMEYwRTdCOUYxNz45 MiBEPDFGRTAwMDMwMzAwMDc4MTgwMDc4MUMwMDMwMEUwMDAwMEUwMDAwMEUwMDAwCjBFMDAw MEZFMDAwNzhFMDAxRTBFMDAzODBFMDA3ODBFMDBGMDBFMTBGMDBFMTBGMDBFMTBGMDFFMTA3 ODFFMTAzODY3MjAwRgo4M0MwMTQxNDdFOTMxNz45NyBEPDBFMDAwMEZFMDAwMDBFMDAwMDBF MDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAKMDAwRTAwMDAwRTAwMDAwRTAwMDAw RTNFMDAwRUMzODAwRjAxQzAwRjAwRTAwRTAwRTAwRTAwNzAwRTAwNzAwRTAwNzgwRTAwCjc4 MEUwMDc4MEUwMDc4MEUwMDc4MEUwMDc4MEUwMDcwMEUwMDcwMEUwMEUwMEYwMEUwMEQwMUMw MENDMzAwMDgzRTAwMTUyMAo3RjlGMTk+STwwM0Y4MEUwQzFDMUUzODFFMzgwQzcwMDA3MDAw RjAwMEYwMDBGMDAwRjAwMEYwMDBGMDAwNzAwMDcwMDAzOAowMTM4MDExQzAyMEUwQzAzRjAx MDE0N0U5MzE0Pkk8MDAwMzgwMDAzRjgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMwo4 MDAwMDM4MDAwMDM4MDAwMDM4MDAwMDM4MDAwMDM4MDAzRTM4MDA2MUI4MDFDMDc4MDM4MDM4 MDM4MDM4MDcwMDM4MDcwMDMKODBGMDAzODBGMDAzODBGMDAzODBGMDAzODBGMDAzODBGMDAz ODA3MDAzODA3MDAzODAzODAzODAzODA3ODAxQzA3ODAwRTFCCjgwMDNFM0Y4MTUyMDdFOUYx OT5JPDAzRjAwMDBFMUMwMDFDMEUwMDM4MDcwMDM4MDcwMDcwMDcwMDcwMDM4MEYwMDM4MEYw CjAzODBGRkZGODBGMDAwMDBGMDAwMDBGMDAwMDA3MDAwMDA3MDAwMDAzODAwODAxODAwODAw QzAxMDAwNzA2MDAwMUY4MDAxMQoxNDdGOTMxND5JPDAwN0MwMEM2MDE4RjAzOEYwNzA2MDcw MDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMEZGRjAwNzAwMDcwMAowNzAwMDcwMDA3MDAwNzAw MDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwN0ZGMDEw MjAKODA5RjBFPkk8MDAwMEUwMDNFMzMwMEUzQzMwMUMxQzMwMzgwRTAwNzgwRjAwNzgwRjAw NzgwRjAwNzgwRjAwNzgwRjAwMzgKMEUwMDFDMUMwMDFFMzgwMDMzRTAwMDIwMDAwMDIwMDAw MDMwMDAwMDMwMDAwMDNGRkUwMDFGRkY4MDBGRkZDMDMwMDFFMDYwCjAwNzBDMDAwMzBDMDAw MzBDMDAwMzBDMDAwMzA2MDAwNjAzMDAwQzAxQzAzODAwM0ZDMDAxNDFGN0Y5NDE3Pkk8MEUw MDAwCkZFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBF MDAwMDBFMDAwMDBFMDAwMDBFM0UwMAowRTQzMDAwRTgxODAwRjAxQzAwRjAxQzAwRTAxQzAw RTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAKMEUwMUMwMEUwMUMw MEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwRkZFN0ZDMTYyMDdGOUYxOT5JPDFDMDAxRTAwM0Uw MDFFMDAKMUMwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDBFMDA3RTAwMEUwMDBFMDAwRTAw MEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwCjBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAw RTAwRkZDMDBBMUY4MDlFMEM+STwwMEUwMDFGMDAxRjAwMUYwMDBFMDAwCjAwMDAwMDAwMDAw MDAwMDAwMDAwMDAwMDcwMDdGMDAwRjAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3 MDAwNzAwMAo3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcw NjA3MEYwNjBGMEMwNjE4MDNGMDAwQzI4ODIKOUUwRT5JPDBFMDAwMEZFMDAwMDBFMDAwMDBF MDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAwMDBFMDAKMDAwRTAwMDAw RTBGRjAwRTAzQzAwRTAzMDAwRTAyMDAwRTA0MDAwRTA4MDAwRTEwMDAwRTMwMDAwRTcwMDAw RUY4MDAwRjM4CjAwMEUxQzAwMEUxRTAwMEUwRTAwMEUwNzAwMEUwNzgwMEUwMzgwMEUwM0Mw MEUwM0UwRkZDRkY4MTUyMDdGOUYxOD5JPDBFCjAwRkUwMDBFMDAwRTAwMEUwMDBFMDAwRTAw MEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRQowMDBFMDAw RTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMEZGRTAwQjIwODA5 RjBDPkk8MEUxRgowMUYwMDBGRTYxODYxODAwMEU4MUM4MUMwMDBGMDBGMDBFMDAwRjAwRjAw RTAwMEUwMEUwMEUwMDBFMDBFMDBFMDAwRTAwRTAKMEUwMDBFMDBFMDBFMDAwRTAwRTAwRTAw MEUwMEUwMEUwMDBFMDBFMDBFMDAwRTAwRTAwRTAwMEUwMEUwMEUwMDBFMDBFMDBFCjAwMEUw MEUwMEUwMDBFMDBFMDBFMDAwRTAwRTAwRTAwMEUwMEUwMEUwMEZGRTdGRTdGRTAyMzE0N0Y5 MzI2Pkk8MEUzRTAwCkZFNDMwMDBFODE4MDBGMDFDMDBGMDFDMDBFMDFDMDBFMDFDMDBFMDFD MDBFMDFDMDBFMDFDMDBFMDFDMDBFMDFDMDBFMDFDMAowRTAxQzAwRTAxQzAwRTAxQzAwRTAx QzAwRTAxQzAwRTAxQzBGRkU3RkMxNjE0N0Y5MzE5Pkk8MDFGODAwMDcwRTAwMUMwMwo4MDM4 MDFDMDM4MDFDMDcwMDBFMDcwMDBFMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYwMDBGMEYw MDBGMDcwMDBFMDcwMDAKRTAzODAxQzAzODAxQzAxQzAzODAwNzBFMDAwMUY4MDAxNDE0N0Y5 MzE3Pkk8MEUzRTAwRkVDMzgwMEYwMUMwMEYwMEUwMEUKMDBFMDBFMDBGMDBFMDA3MDBFMDA3 ODBFMDA3ODBFMDA3ODBFMDA3ODBFMDA3ODBFMDA3ODBFMDA3MDBFMDBGMDBFMDBFMDBGCjAx RTAwRjAxQzAwRUMzMDAwRTNFMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAw MDAwRTAwMDAwRTAwMDBGRgpFMDAwMTUxRDdGOTMxOT5JPDAzRTA4MDA2MTk4MDFDMDU4MDND MDc4MDM4MDM4MDc4MDM4MDcwMDM4MEYwMDM4MEYwMDM4MApGMDAzODBGMDAzODBGMDAzODBG MDAzODA3MDAzODA3ODAzODAzODAzODAzODA3ODAxQzBCODAwRTEzODAwM0UzODAwMDAzODAK MDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAwMzgwMDAzRkY4MTUxRDdF OTMxOD5JPDBFNzhGRThDMEYKMUUwRjFFMEYwQzBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBF MDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwRkZFMDBGCjE0N0Y5MzEyPkk8MUY5MDMw NzA0MDMwQzAxMEMwMTBDMDEwRTAwMDc4MDA3RjgwM0ZFMDBGRjAwMDcwODAzODgwMTg4MDE4 CkMwMThDMDE4RTAzMEQwNjA4RjgwMEQxNDdFOTMxMj5JPDAyMDAwMjAwMDIwMDA2MDAwNjAw MEUwMDBFMDAzRTAwRkZGODBFCjAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAw RTAwMEUwMDBFMDgwRTA4MEUwODBFMDgwRTA4MDYxMDAzMTAwMQpFMDBEMUM3RjlCMTI+STww RTAxQzBGRTFGQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAwRTAxQzAw RQowMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwMUMwMEUwM0MwMDYwM0Mw MDMwREMwMDFGMUZDMTYxNDdGOTMKMTk+STxGRjgzRjgxRTAxRTAxQzAwQzAwRTAwODAwRTAw ODAwRTAwODAwNzAxMDAwNzAxMDAwMzgyMDAwMzgyMDAwMzgyMDAKMDFDNDAwMDFDNDAwMDFF QzAwMDBFODAwMDBFODAwMDA3MDAwMDA3MDAwMDA3MDAwMDAyMDAwMTUxNDdGOTMxOD5JPEZG OUYKRTFGQzNDMDc4MDcwMUMwMzAwNjAxQzAzODAyMDBFMDM4MDQwMEUwMzgwNDAwRTAzQzA0 MDA3MDdDMDgwMDcwNEMwODAwNzA0CkUwODAwMzg4NjEwMDAzODg3MTAwMDNDODczMDAwMUQw MzIwMDAxRDAzQTAwMDBGMDNDMDAwMEUwMUMwMDAwRTAxQzAwMDA2MAoxODAwMDA0MDA4MDAx RTE0N0Y5MzIxPkk8N0ZDM0ZDMEYwMUUwMDcwMUMwMDcwMTgwMDM4MTAwMDFDMjAwMDBFNDAw MDBFQwowMDAwNzgwMDAwMzgwMDAwM0MwMDAwN0MwMDAwNEUwMDAwODcwMDAxMDcwMDAzMDM4 MDAyMDFDMDA2MDFFMDFFMDFFMEZGMDcKRkUxNzE0ODA5MzE4Pkk8RkY4M0Y4MUUwMUUwMUMw MEMwMEUwMDgwMEUwMDgwMEUwMDgwMDcwMTAwMDcwMTAwMDM4MjAwMDMKODIwMDAzODIwMDAx QzQwMDAxQzQwMDAxRUMwMDAwRTgwMDAwRTgwMDAwNzAwMDAwNzAwMDAwNzAwMDAwMjAwMDAw MjAwMDAwCjQwMDAwMDQwMDAwMDQwMDBGMDgwMDBGMDgwMDBGMTAwMDA2MjAwMDAzQzAwMDAx NTFEN0Y5MzE4Pkk8M0ZGRjM4MEUyMDBFCjIwMUM0MDM4NDA3ODQwNzAwMEUwMDFFMDAxQzAw MzgwMDc4MDA3MDEwRTAxMUUwMTFDMDMzODAyNzAwNjcwMEVGRkZFMTAxNAo3RjkzMTQ+SSBF IC9GbSAyNSAxMTggZGY8NzBGOEZDRkM3NDA0MDQwNDA0MDgwODEwMTAyMDQwMDYwRjdDODQw RT40NApEPDAxRjAwMDA3MUMwMDBDMDYwMDE4MDMwMDM4MDM4MDM4MDM4MDcwMDFDMDcwMDFD MDcwMDFDMDcwMDFDMEYwMDFFMEYwMDEKRTBGMDAxRTBGMDAxRTBGMDAxRTBGMDAxRTBGMDAx RTBGMDAxRTBGMDAxRTBGMDAxRTBGMDAxRTBGMDAxRTBGMDAxRTBGMDAxCkUwNzAwMUMwNzAw MUMwNzAwMUMwNzgwM0MwMzgwMzgwMzgwMzgwMUMwNzAwMEMwNjAwMDcxQzAwMDFGMDAwMTMy MjdFQTAxOAo+NDggRDwwMDgwMDM4MDBGODBGMzgwMDM4MDAzODAwMzgwMDM4MDAzODAwMzgw MDM4MDAzODAwMzgwMDM4MDAzODAwMzgwMDMKODAwMzgwMDM4MDAzODAwMzgwMDM4MDAzODAw MzgwMDM4MDAzODAwMzgwMDM4MDAzODAwMzgwMDM4MDA3QzBGRkZFMEYyMTdDCkEwMTg+STww MDdFMDAwMUMxMDAwMzAwODAwNjAxQzAwRTAzQzAxQzAzQzAxODAxODAzODAwMDAzODAwMDA3 ODAwMDA3MDAwCjAwNzAwMDAwRjBGODAwRjMwQzAwRjQwNjAwRjQwMzAwRjgwMzgwRjgwMUMw RjAwMUMwRjAwMUUwRjAwMUUwRjAwMUUwRjAwMQpFMEYwMDFFMDcwMDFFMDcwMDFFMDcwMDFF MDM4MDFDMDM4MDFDMDE4MDM4MDFDMDMwMDBDMDYwMDA3MEMwMDAxRjAwMDEzMjIKN0VBMDE4 PjU0IEQ8MDFGMDAwMDYwQzAwMEMwNjAwMTgwNzAwMzgwMzgwNzAwMzgwNzAwMzgwRjAwMUMw RjAwMUMwRjAwMUMwCkYwMDFFMEYwMDFFMEYwMDFFMEYwMDFFMEYwMDFFMDcwMDFFMDcwMDNF MDM4MDNFMDE4MDVFMDBDMDVFMDA2MTlFMDAzRTFFMAowMDAxQzAwMDAxQzAwMDAxQzAwMDAz ODAwMDAzODAzMDAzMDA3ODA3MDA3ODA2MDA3MDBDMDAyMDE4MDAxMDMwMDAwRkMwMDAKMTMy MjdFQTAxOD41NyBEPDAwMDE4MDAwMDAwMTgwMDAwMDAxODAwMDAwMDNDMDAwMDAwM0MwMDAw MDAzQzAwMDAwMDVFMDAwCjAwMDVFMDAwMDAwREYwMDAwMDA4RjAwMDAwMDhGMDAwMDAxMEY4 MDAwMDEwNzgwMDAwMTA3ODAwMDAyMDNDMDAwMDIwM0MwMAowMDIwM0MwMDAwNDAxRTAwMDA0 MDFFMDAwMDQwMUUwMDAwODAwRjAwMDA4MDBGMDAwMEZGRkYwMDAxMDAwNzgwMDEwMDA3ODAK MDMwMDA3QzAwMjAwMDNDMDAyMDAwM0MwMDQwMDAzRTAwNDAwMDFFMDA0MDAwMUUwMEMwMDAw RjAwQzAwMDBGMDNFMDAwMUY4CkZGODAwRkZGMjAyMzdFQTIyNT42NSBEPEZGRkZGRkMwMEY4 MDA3QzAwNzgwMDFDMDA3ODAwMEMwMDc4MDAwNDAwNzgwMDA0MAowNzgwMDA2MDA3ODAwMDIw MDc4MDAwMjAwNzgwMDAyMDA3ODAyMDIwMDc4MDIwMDAwNzgwMjAwMDA3ODAyMDAwMDc4MDYw MDAKMDc4MEUwMDAwN0ZGRTAwMDA3ODBFMDAwMDc4MDYwMDAwNzgwMjAwMDA3ODAyMDAwMDc4 MDIwMDAwNzgwMjAwMDA3ODAwMDAwCjA3ODAwMDAwMDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAw MDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAwMDc4MDAwMDAwRkMwMDAwMApGRkZFMDAwMDFCMjI3 RUExMjA+NzAgRDxGRkZDMEZDMDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4 MDA3ODAKMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgw MDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwCjA3ODAwNzgwMEZDMEZGRkMwRTIyN0VBMTEyPjcz IEQ8RkZDMDAwMDNGRjBGQzAwMDAzRjAwN0MwMDAwM0UwMDVFMDAwMDVFMAowNUUwMDAwNUUw MDRGMDAwMDlFMDA0RjAwMDA5RTAwNEYwMDAwOUUwMDQ3ODAwMTFFMDA0NzgwMDExRTAwNDc4 MDAxMUUwMDQKM0MwMDIxRTAwNDNDMDAyMUUwMDQzQzAwMjFFMDA0MUUwMDQxRTAwNDFFMDA0 MUUwMDQwRjAwODFFMDA0MEYwMDgxRTAwNDBGCjAwODFFMDA0MDc4MTAxRTAwNDA3ODEwMUUw MDQwNzgxMDFFMDA0MDNDMjAxRTAwNDAzQzIwMUUwMDQwMUU0MDFFMDA0MDFFNAowMUUwMDQw MUU0MDFFMDA0MDBGODAxRTAwNDAwRjgwMUUwMDQwMEY4MDFFMDA0MDA3MDAxRTAwRTAwNzAw MUUwMUYwMDcwMDMKRjBGRkUwMjAzRkZGMjgyMjdFQTEyRD43NyBEPEZGRkZGMDAwMEY4MDND MDAwNzgwMEYwMDA3ODAwNzgwMDc4MDA3QzAwNzgwCjAzQzAwNzgwMDNFMDA3ODAwM0UwMDc4 MDAzRTAwNzgwMDNFMDA3ODAwM0UwMDc4MDAzRTAwNzgwMDNDMDA3ODAwN0MwMDc4MAowNzgw MDc4MDBGMDAwNzgwM0MwMDA3RkZGMDAwMDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAwMDc4MDAw MDAwNzgwMDAwMDA3ODAKMDAwMDA3ODAwMDAwMDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAwMDc4 MDAwMDAwNzgwMDAwMDA3ODAwMDAwMDc4MDAwMDAwRkMwCjAwMDBGRkZDMDAwMDFCMjI3RUEx MjE+ODAgRDwwRkUwMDAxODM4MDAzQzBDMDAzQzBFMDAxODA3MDAwMDA3MDAwMDA3MDAwMAow NzAwMDBGRjAwMDdDNzAwMUUwNzAwM0MwNzAwNzgwNzAwNzAwNzAwRjAwNzA4RjAwNzA4RjAw NzA4RjAwRjA4NzgxNzA4M0MKMjM5MDBGQzFFMDE1MTU3RTk0MTg+OTcgRDwwMUZFMDAwNzAz MDAwQzA3ODAxQzA3ODAzODAzMDA3ODAwMDA3MDAwMDBGMDAwCjAwRjAwMDAwRjAwMDAwRjAw MDAwRjAwMDAwRjAwMDAwRjAwMDAwNzAwMDAwNzgwMDQwMzgwMDQwMUMwMDgwMEMwMTAwMDcw NgowMDAxRjgwMDEyMTU3RTk0MTY+OTkgRDwwMUZDMDAwNzA3MDAwQzAzODAxQzAxQzAzODAx QzA3ODAxRTA3MDAwRTBGMDAwRTAKRkZGRkUwRjAwMDAwRjAwMDAwRjAwMDAwRjAwMDAwRjAw MDAwNzAwMDAwNzgwMDIwMzgwMDIwMUMwMDQwMEUwMDgwMDcwMzAwCjAwRkMwMDEzMTU3Rjk0 MTY+MTAxIEQ8MDAzQzAwQzYwMThGMDM4RjAzMEYwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAw NzAwCjA3MDAwNzAwRkZGODA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3MDAwNzAwMDcwMDA3 MDAwNzAwMDcwMDA3MDAwNzAwMDcwMAowNzAwMDcwMDA3MDAwNzgwN0ZGODEwMjM4MEEyMEY+ STwwMDAwNzAwMUYxOTgwNzFFMTgwRTBFMTgxQzA3MDAxQzA3MDAzQwowNzgwM0MwNzgwM0Mw NzgwM0MwNzgwMUMwNzAwMUMwNzAwMEUwRTAwMEYxQzAwMTlGMDAwMTAwMDAwMTAwMDAwMTgw MDAwMTgKMDAwMDFGRkUwMDBGRkZDMDBGRkZFMDM4MDBGMDYwMDAzMDQwMDAxOEMwMDAxOEMw MDAxOEMwMDAxODYwMDAzMDYwMDAzMDM4CjAwRTAwRTAzODAwM0ZFMDAxNTIxN0Y5NTE4Pkk8 MUMwMDFFMDAzRTAwMUUwMDFDMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwCjAwMDAwMDAwMDAw RTAwRkUwMDFFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUw MDBFMDAwRQowMDBFMDAwRTAwMEUwMDBFMDBGRkMwMEEyMjdGQTEwRT4xMDUgRDwwRTAwRkUw MDFFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMAowRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAw MEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUwMDBFMDAKMEUwMDBFMDAw RTAwMEUwMDBFMDAwRTAwMEUwMDBFMDBGRkUwMEIyMzdGQTIwRT4xMDggRDwwRTFGQzA3RjAw RkU2MEUxODMKODAxRTgwNzIwMUMwMEYwMDNDMDBFMDBGMDAzQzAwRTAwRTAwMzgwMEUwMEUw MDM4MDBFMDBFMDAzODAwRTAwRTAwMzgwMEUwCjBFMDAzODAwRTAwRTAwMzgwMEUwMEUwMDM4 MDBFMDBFMDAzODAwRTAwRTAwMzgwMEUwMEUwMDM4MDBFMDBFMDAzODAwRTAwRQowMDM4MDBF MDBFMDAzODAwRTAwRTAwMzgwMEUwMEUwMDM4MDBFMEZGRTNGRjhGRkUyNzE1N0Y5NDJBPkk8 MEUxRjgwRkU2MApDMDFFODBFMDBGMDA3MDBGMDA3MDBFMDA3MDBFMDA3MDBFMDA3MDBFMDA3 MDBFMDA3MDBFMDA3MDBFMDA3MDBFMDA3MDBFMDAKNzAwRTAwNzAwRTAwNzAwRTAwNzAwRTAw NzAwRTAwNzAwRTAwNzBGRkU3RkYxODE1N0Y5NDFCPkk8MDFGQzAwMDcwNzAwMEMKMDE4MDE4 MDBDMDM4MDBFMDcwMDA3MDcwMDA3MEYwMDA3OEYwMDA3OEYwMDA3OEYwMDA3OEYwMDA3OEYw MDA3OEYwMDA3ODcwCjAwNzA3ODAwRjAzODAwRTAxQzAxQzAwRTAzODAwNzA3MDAwMUZDMDAx NTE1N0Y5NDE4Pkk8MEUxRjAwRkU2MUMwMEU4MDYwCjBGMDA3MDBFMDAzODBFMDAzQzBFMDAx QzBFMDAxRTBFMDAxRTBFMDAxRTBFMDAxRTBFMDAxRTBFMDAxRTBFMDAxRTBFMDAzQwowRTAw M0MwRTAwMzgwRjAwNzAwRTgwRTAwRTQxQzAwRTNGMDAwRTAwMDAwRTAwMDAwRTAwMDAwRTAw MDAwRTAwMDAwRTAwMDAKMEUwMDAwMEUwMDAwMEUwMDAwRkZFMDAwMTcxRjdGOTQxQj5JPDBF M0NGRTQ2MUU4RjBGMEYwRjA2MEYwMDBFMDAwRTAwMEUKMDAwRTAwMEUwMDBFMDAwRTAwMEUw MDBFMDAwRTAwMEUwMDBFMDAwRTAwMEYwMEZGRjAxMDE1N0Y5NDEzPjExNApEPDBGODgzMDc4 NjAxOEMwMThDMDA4QzAwOEUwMDhGMDAwN0Y4MDNGRTAwRkYwMDFGODAwM0M4MDFDODAwQzgw MENDMDBDQzAKMDhFMDE4RDAzMDhGQzAwRTE1N0U5NDEzPkk8MDIwMDAyMDAwMjAwMDIwMDA2 MDAwNjAwMDYwMDBFMDAxRTAwM0UwMEZGRjgKMEUwMDBFMDAwRTAwMEUwMDBFMDAwRTAwMEUw MDBFMDAwRTAwMEUwMDBFMDAwRTA0MEUwNDBFMDQwRTA0MEUwNDBFMDQwNzA4CjAzMDgwMUYw MEUxRjdGOUUxMz5JPDBFMDA3MEZFMDdGMDFFMDBGMDBFMDA3MDBFMDA3MDBFMDA3MDBFMDA3 MDBFMDA3MDBFCjAwNzAwRTAwNzAwRTAwNzAwRTAwNzAwRTAwNzAwRTAwNzAwRTAwNzAwRTAw NzAwRTAwRjAwRTAwRjAwNjAxNzAwMzgyNzgwMApGQzdGMTgxNTdGOTQxQj5JIEUgL0ZuIDIx IDEyMiBkZjxGRkZGODBGRkZGODBGRkZGODAxMTAzN0Y5MDE2PjQ1CkQ8MDA3RjgwMDAwMUZG RjAwMDA3MDFGODAwMEMwMDdFMDAxODAwMUYwMDEwMDAwRjAwMjAwMDBGODA0MDAwMDdDMDQw MDAwNwpDMDQwMDAwM0UwQjAwMDAzRTBGODAwMDNFMEZDMDAwM0UwRkMwMDAzRTBGQzAwMDNF MDc4MDAwM0UwMDAwMDAzRTAwMDAwMDcKQzAwMDAwMDdDMDAwMDAwNzgwMDAwMDBGODAwMDAw MEYwMDAwMDAxRjAwMDAwMDFFMDAwMDAwM0MwMDAwMDA3ODAwMDAwMDcwCjAwMDAwMEUwMDAw MDAxQzAwMDAwMDM4MDAwMDAwNzAwMDAwMDBFMDAwMDAwMEMwMDAwMDAxODAwMDAwMDMwMDAw MDAwNjAwMAowMDAwQzAwMDIwMDE4MDAwMjAwMzAwMDAyMDA2MDAwMDQwMEMwMDAwNDAwODAw MDA0MDEwMDAwMEMwM0ZGRkZGQzA3RkZGRkYKODBGRkZGRkY4MEZGRkZGRjgwMUIyRjdEQUUy MT41MCBEPDc4RkNGQ0ZDRkM3ODAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwCjAwMDAwMDAw MDAwMDc4RkNGQ0ZDRkM3ODA2MUY3QTlFMTI+NTggRDwwMDAwM0ZFMDAxMDAwMUZGRjgwMzAw MDdGMDFFMDMwMAoxRjgwMDMwNzAwM0UwMDAwODcwMDc4MDAwMDRGMDBGMDAwMDAyRjAxRTAw MDAwMUYwM0MwMDAwMDBGMDc4MDAwMDAwRjBGODAKMDAwMDA3MEYwMDAwMDAwNzFGMDAwMDAw MDMxRTAwMDAwMDAzM0UwMDAwMDAwMzNDMDAwMDAwMDE3QzAwMDAwMDAxN0MwMDAwCjAwMDE3 QzAwMDAwMDAxNzgwMDAwMDAwMEY4MDAwMDAwMDBGODAwMDAwMDAwRjgwMDAwMDAwMEY4MDAw MDAwMDBGODAwMDAwMAowMEY4MDAwMDAwMDBGODAwMDAwMDAwRjgwMDAwMDAwMEY4MDAwMDAw MDBGODAwMDAwMDAwRjgwMDAwMDAwMDc4MDAwMDAwMDAKN0MwMDAwMDAwMDdDMDAwMDAwMDE3 QzAwMDAwMDAxM0MwMDAwMDAwMTNFMDAwMDAwMDExRTAwMDAwMDAxMUYwMDAwMDAwMjBGCjAw MDAwMDAyMEY4MDAwMDAwNjA3ODAwMDAwMDQwM0MwMDAwMDA4MDFFMDAwMDAwODAwRjAwMDAw MTAwMDc4MDAwMDIwMDAzRQowMDAwQzAwMDFGODAwMzgwMDAwN0YwMEYwMDAwMDFGRkZDMDAw MDAwM0ZFMDAwMjgzMzdDQjEzMD42NwpEPEZGRkZGRkUwMDAwMEZGRkZGRkZDMDAwMDA3RTAw MDdGMDAwMDAzRTAwMDBGODAwMDAzRTAwMDAzQzAwMDAzRTAwMDAxRTAKMDAwM0UwMDAwMEYw MDAwM0UwMDAwMDc4MDAwM0UwMDAwMDNDMDAwM0UwMDAwMDFFMDAwM0UwMDAwMDFFMDAwM0Uw MDAwMDBGCjAwMDNFMDAwMDAwRjAwMDNFMDAwMDAwRjgwMDNFMDAwMDAwNzgwMDNFMDAwMDAw N0MwMDNFMDAwMDAwN0MwMDNFMDAwMDAwMwpDMDAzRTAwMDAwMDNDMDAzRTAwMDAwMDNFMDAz RTAwMDAwMDNFMDAzRTAwMDAwMDNFMDAzRTAwMDAwMDNFMDAzRTAwMDAwMDMKRTAwM0UwMDAw MDAzRTAwM0UwMDAwMDAzRTAwM0UwMDAwMDAzRTAwM0UwMDAwMDAzRTAwM0UwMDAwMDAzRTAw M0UwMDAwMDAzCkUwMDNFMDAwMDAwM0MwMDNFMDAwMDAwM0MwMDNFMDAwMDAwN0MwMDNFMDAw MDAwN0MwMDNFMDAwMDAwNzgwMDNFMDAwMDAwNwo4MDAzRTAwMDAwMEY4MDAzRTAwMDAwMEYw MDAzRTAwMDAwMUYwMDAzRTAwMDAwMUUwMDAzRTAwMDAwM0MwMDAzRTAwMDAwNzgKMDAwM0Uw MDAwMEY4MDAwM0UwMDAwMUYwMDAwM0UwMDAwM0UwMDAwM0UwMDAwRjgwMDAwN0UwMDAzRjAw MDBGRkZGRkZGQzAwCjAwRkZGRkZGRTAwMDAwMkIzMTdDQjAzMz5JPEZGRkY4MEZGRkY4MDA3 RjAwMDAzRTAwMDAzRTAwMDAzRTAwMDAzRTAwMDAzCkUwMDAwM0UwMDAwM0UwMDAwM0UwMDAw M0UwMDAwM0UwMDAwM0UwMDAwM0UwMDAwM0UwMDAwM0UwMDAwM0UwMDAwM0UwMDAwMwpFMDAw MDNFMDAwMDNFMDAwMDNFMDAwMDNFMDAwMDNFMDAwMDNFMDAwMDNFMDAwMDNFMDAwMDNFMDAw MDNFMDAwMDNFMDAwMDMKRTAwMDAzRTAwMDAzRTAwMDAzRTAwMDAzRTAwMDAzRTAwMDAzRTAw MDAzRTAwMDAzRTAwMDAzRTAwMDAzRTAwMDAzRTAwMDAzCkUwMDAwM0UwMDAwM0UwMDAwN0Yw MDBGRkZGODBGRkZGODAxMTMxN0RCMDE3PjczIEQ8RkZGMDAwMDAwMDdGRjhGRkYwMDAwMAow MDdGRjgwN0YwMDAwMDAwN0YwMDAyRjgwMDAwMDBCRTAwMDJGODAwMDAwMEJFMDAwMkY4MDAw MDAwQkUwMDAyN0MwMDAwMDEKM0UwMDAyN0MwMDAwMDEzRTAwMDIzRTAwMDAwMjNFMDAwMjNF MDAwMDAyM0UwMDAyM0UwMDAwMDIzRTAwMDIxRjAwMDAwNDNFCjAwMDIxRjAwMDAwNDNFMDAw MjFGMDAwMDA0M0UwMDAyMEY4MDAwMDgzRTAwMDIwRjgwMDAwODNFMDAwMjBGODAwMDA4M0Uw MAowMjA3QzAwMDEwM0UwMDAyMDdDMDAwMTAzRTAwMDIwN0MwMDAxMDNFMDAwMjAzRTAwMDIw M0UwMDAyMDNFMDAwMjAzRTAwMDIKMDFGMDAwNDAzRTAwMDIwMUYwMDA0MDNFMDAwMjAxRjAw MDQwM0UwMDAyMDBGODAwODAzRTAwMDIwMEY4MDA4MDNFMDAwMjAwCkY4MDA4MDNFMDAwMjAw N0MwMTAwM0UwMDAyMDA3QzAxMDAzRTAwMDIwMDdDMDEwMDNFMDAwMjAwM0UwMjAwM0UwMDAy MDAzRQowMjAwM0UwMDAyMDAzRTAyMDAzRTAwMDIwMDFGMDQwMDNFMDAwMjAwMUYwNDAwM0Uw MDAyMDAwRjg4MDAzRTAwMDIwMDBGODgKMDAzRTAwMDIwMDBGODgwMDNFMDAwMjAwMDdEMDAw M0UwMDAyMDAwN0QwMDAzRTAwMDIwMDA3RDAwMDNFMDAwMjAwMDNFMDAwCjNFMDAwMjAwMDNF MDAwM0UwMDAyMDAwM0UwMDAzRTAwMDcwMDAxQzAwMDNFMDAwRjgwMDFDMDAwN0YwMEZGRjgw MUMwMEZGRgpGOEZGRjgwMDgwMEZGRkY4MzUzMTdDQjAzRD43NyBEPEZGRkZGRkMwMDBGRkZG RkZGODAwMDdFMDAwN0UwMDAzRTAwMDFGMDAKMDNFMDAwMDc4MDAzRTAwMDAzQzAwM0UwMDAw MUUwMDNFMDAwMDFGMDAzRTAwMDAxRjAwM0UwMDAwMEYwMDNFMDAwMDBGODAzCkUwMDAwMEY4 MDNFMDAwMDBGODAzRTAwMDAwRjgwM0UwMDAwMEY4MDNFMDAwMDBGODAzRTAwMDAwRjAwM0Uw MDAwMUYwMDNFMAowMDAxRTAwM0UwMDAwM0UwMDNFMDAwMDNDMDAzRTAwMDA3ODAwM0UwMDAx RjAwMDNFMDAwN0MwMDAzRkZGRkYwMDAwM0UwMDAKMDAwMDAzRTAwMDAwMDAwM0UwMDAwMDAw MDNFMDAwMDAwMDAzRTAwMDAwMDAwM0UwMDAwMDAwMDNFMDAwMDAwMDAzRTAwMDAwCjAwMDNF MDAwMDAwMDAzRTAwMDAwMDAwM0UwMDAwMDAwMDNFMDAwMDAwMDAzRTAwMDAwMDAwM0UwMDAw MDAwMDNFMDAwMDAwMAowM0UwMDAwMDAwMDNFMDAwMDAwMDAzRTAwMDAwMDAwM0UwMDAwMDAw MDNFMDAwMDAwMDAzRTAwMDAwMDAwN0YwMDAwMDAwRkYKRkY4MDAwMDBGRkZGODAwMDAwMjUz MTdDQjAyRD44MCBEPDAwRkUwMDAwMDMwM0MwMDAwQzAwRTAwMDEwMDA3MDAwMTAwMDM4CjAw M0MwMDNDMDAzRTAwMUMwMDNFMDAxRTAwM0UwMDFFMDAwODAwMUUwMDAwMDAxRTAwMDAwMDFF MDAwMDAwMUUwMDAwMEZGRQowMDAwRkMxRTAwMDNFMDFFMDAwRjgwMUUwMDFGMDAxRTAwM0Uw MDFFMDAzQzAwMUUwMDdDMDAxRTAwRjgwMDFFMDRGODAwMUUKMDRGODAwMUUwNEY4MDAzRTA0 RjgwMDNFMDQ3ODAwM0UwNDdDMDA1RTA0M0UwMDhGMDgwRjAzMDdGMDAzRkMwM0UwMUUxRjdE CjlFMjE+OTcgRDwwMDNGODAwMEUwNjAwMzgwMTgwNzAwMDQwRjAwMDQxRTAwMUUxQzAwM0Uz QzAwM0U3QzAwM0U3QzAwMDg3OAowMDAwRjgwMDAwRjgwMDAwRjgwMDAwRjgwMDAwRjgwMDAw RjgwMDAwRjgwMDAwRjgwMDAwRjgwMDAwNzgwMDAwN0MwMDAwN0MKMDAwMDNDMDAwMTFFMDAw MTFFMDAwMjBGMDAwMjA3MDAwNDAzODAxODAwRTA2MDAwM0Y4MDE4MUY3RDlFMUQ+OTkKRDww MDNGODAwMDAwRTBFMDAwMDM4MDM4MDAwNzAwM0MwMDBFMDAxRTAwMUUwMDFFMDAxQzAwMEYw MDNDMDAwRjAwN0MwMDBGCjAwNzgwMDBGODA3ODAwMDc4MEY4MDAwNzgwRjgwMDA3ODBGRkZG RkY4MEY4MDAwMDAwRjgwMDAwMDBGODAwMDAwMEY4MDAwMAowMEY4MDAwMDAwRjgwMDAwMDA3 ODAwMDAwMDdDMDAwMDAwM0MwMDAwMDAzQzAwMDA4MDFFMDAwMDgwMEUwMDAxMDAwRjAwMDIK MDAwNzgwMDIwMDAxQzAwQzAwMDBGMDMwMDAwMDFGQzAwMDE5MUY3RTlFMUQ+MTAxIEQ8MDc4 MDAwMDAwMEZGODAwMDAwMDAKRkY4MDAwMDAwMDBGODAwMDAwMDAwNzgwMDAwMDAwMDc4MDAw MDAwMDA3ODAwMDAwMDAwNzgwMDAwMDAwMDc4MDAwMDAwMDA3CjgwMDAwMDAwMDc4MDAwMDAw MDA3ODAwMDAwMDAwNzgwMDAwMDAwMDc4MDAwMDAwMDA3ODAwMDAwMDAwNzgwMDAwMDAwMDc4 MAowMDAwMDAwNzgwMDAwMDAwMDc4MDAwMDAwMDA3ODBGRTAwMDAwNzgzMDc4MDAwMDc4QzAz QzAwMDA3OTAwMUUwMDAwN0EwMDEKRTAwMDA3QTAwMEYwMDAwN0MwMDBGMDAwMDdDMDAwRjAw MDA3ODAwMEYwMDAwNzgwMDBGMDAwMDc4MDAwRjAwMDA3ODAwMEYwCjAwMDc4MDAwRjAwMDA3 ODAwMEYwMDAwNzgwMDBGMDAwMDc4MDAwRjAwMDA3ODAwMEYwMDAwNzgwMDBGMDAwMDc4MDAw RjAwMAowNzgwMDBGMDAwMDc4MDAwRjAwMDA3ODAwMEYwMDAwNzgwMDBGMDAwMDc4MDAwRjAw MDA3ODAwMEYwMDAwNzgwMDBGMDAwMDcKODAwMEYwMDAwNzgwMDBGMDAwMEZDMDAxRjgwMEZG RkMxRkZGODBGRkZDMUZGRjgwMjEzMjdFQjEyNT4xMDQKRDwwNzAwMEY4MDFGODAxRjgwMEY4 MDA3MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA3Cjgw RkY4MEZGODAwRjgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3 ODAwNzgwMDc4MDA3ODAwNwo4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4MDA3ODAwNzgwMDc4 MDBGQzBGRkY4RkZGODBEMzA3RUFGMTI+STwwNzgwRkUwMAoxRkMwMDBGRjgzMDc4MDYwRjAw MEZGOEMwM0MxODA3ODAwMEY5MDAxRTIwMDNDMDAwN0EwMDFFNDAwM0MwMDA3QTAwMEY0MDAK MUUwMDA3QzAwMEY4MDAxRTAwMDdDMDAwRjgwMDFFMDAwNzgwMDBGMDAwMUUwMDA3ODAwMEYw MDAxRTAwMDc4MDAwRjAwMDFFCjAwMDc4MDAwRjAwMDFFMDAwNzgwMDBGMDAwMUUwMDA3ODAw MEYwMDAxRTAwMDc4MDAwRjAwMDFFMDAwNzgwMDBGMDAwMUUwMAowNzgwMDBGMDAwMUUwMDA3 ODAwMEYwMDAxRTAwMDc4MDAwRjAwMDFFMDAwNzgwMDBGMDAwMUUwMDA3ODAwMEYwMDAxRTAw MDcKODAwMEYwMDAxRTAwMDc4MDAwRjAwMDFFMDAwNzgwMDBGMDAwMUUwMDA3ODAwMEYwMDAx RTAwMDc4MDAwRjAwMDFFMDAwNzgwCjAwRjAwMDFFMDAwNzgwMDBGMDAwMUUwMDBGQzAwMUY4 MDAzRjAwRkZGQzFGRkY4M0ZGRjBGRkZDMUZGRjgzRkZGMDM0MUY3RQo5RTM4PjEwOSBEPDA3 ODBGRTAwMDBGRjgzMDc4MDAwRkY4QzAzQzAwMDBGOTAwMUUwMDAwN0EwMDFFMDAwMDdBMDAw RjAwMAowN0MwMDBGMDAwMDdDMDAwRjAwMDA3ODAwMEYwMDAwNzgwMDBGMDAwMDc4MDAwRjAw MDA3ODAwMEYwMDAwNzgwMDBGMDAwMDcKODAwMEYwMDAwNzgwMDBGMDAwMDc4MDAwRjAwMDA3 ODAwMEYwMDAwNzgwMDBGMDAwMDc4MDAwRjAwMDA3ODAwMEYwMDAwNzgwCjAwRjAwMDA3ODAw MEYwMDAwNzgwMDBGMDAwMDc4MDAwRjAwMDA3ODAwMEYwMDAwNzgwMDBGMDAwMDc4MDAwRjAw MDA3ODAwMApGMDAwMEZDMDAxRjgwMEZGRkMxRkZGODBGRkZDMUZGRjgwMjExRjdFOUUyNT5J PDAwMUZDMDAwMDBGMDc4MDAwMUMwMUMwMAowNzAwMDcwMDBGMDAwNzgwMUUwMDAzQzAxQzAw MDFDMDNDMDAwMUUwM0MwMDAxRTA3ODAwMDBGMDc4MDAwMEYwNzgwMDAwRjAKRjgwMDAwRjhG ODAwMDBGOEY4MDAwMEY4RjgwMDAwRjhGODAwMDBGOEY4MDAwMEY4RjgwMDAwRjhGODAwMDBG ODc4MDAwMEYwCjdDMDAwMUYwM0MwMDAxRTAzQzAwMDFFMDFFMDAwM0MwMUUwMDAzQzAwRjAw MDc4MDA3ODAwRjAwMDFDMDFDMDAwMEYwNzgwMAowMDFGQzAwMDFEMUY3RTlFMjE+STwwNzgx RkMwMEZGODYwNzAwRkY4ODAzQzAwRjkwMDFFMDA3QTAwMEYwMDdDMDAwNzgwNwo4MDAwNzgw NzgwMDAzQzA3ODAwMDNDMDc4MDAwM0UwNzgwMDAxRTA3ODAwMDFGMDc4MDAwMUYwNzgwMDAx RjA3ODAwMDFGMDcKODAwMDFGMDc4MDAwMUYwNzgwMDAxRjA3ODAwMDFGMDc4MDAwMUYwNzgw MDAzRTA3ODAwMDNFMDc4MDAwM0MwNzgwMDA3QzA3CjgwMDA3ODA3QzAwMEYwMDdBMDAwRjAw N0EwMDFFMDA3OTgwMzgwMDc4NjBGMDAwNzgxRjgwMDA3ODAwMDAwMDc4MDAwMDAwNwo4MDAw MDAwNzgwMDAwMDA3ODAwMDAwMDc4MDAwMDAwNzgwMDAwMDA3ODAwMDAwMDc4MDAwMDAwNzgw MDAwMDA3ODAwMDAwMEYKQzAwMDAwRkZGQzAwMDBGRkZDMDAwMDIwMkQ3RTlFMjU+STwwNzgz RTBGRjhDMThGRjkwN0MwRjkwN0MwN0EwN0MwN0MwMzgKMDdDMDAwMDdDMDAwMDdDMDAwMDc4 MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwMDc4MDAwCjA3 ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3ODAwMDA3 ODAwMDBGQzAwMEZGRkUwMApGRkZFMDAxNjFGN0U5RTE5PjExNCBEPDAxRkMxMDBFMDMzMDE4 MDBGMDMwMDA3MDYwMDAzMEUwMDAzMEUwMDAxMEUwMDAxMApFMDAwMTBGMDAwMTBGODAwMDA3 RTAwMDAzRkYwMDAxRkZGMDAwRkZGQzAwM0ZGRTAwMDNGRjAwMDAxRjgwMDAwRjg4MDAwM0MK ODAwMDNDODAwMDFDQzAwMDFDQzAwMDFDRTAwMDFDRTAwMDE4RjAwMDM4RjAwMDMwQ0MwMDYw QzMwMUMwODBGRTAwMTYxRjdFCjlFMUE+STwwMDQwMDAwMDQwMDAwMDQwMDAwMDQwMDAwMDQw MDAwMEMwMDAwMEMwMDAwMEMwMDAwMUMwMDAwMUMwMDAwM0MwCjAwMDdDMDAwMEZDMDAwMUZG RkUwRkZGRkUwMDNDMDAwMDNDMDAwMDNDMDAwMDNDMDAwMDNDMDAwMDNDMDAwMDNDMDAwMDND MAowMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAzQzAwMDAz QzAxMDAzQzAxMDAzQzAxMDAzQzAKMTAwM0MwMTAwM0MwMTAwM0MwMTAwM0MwMTAwMUMwMjAw MUUwMjAwMEUwNDAwMDc4QzAwMDFGMDAxNDJDN0ZBQjE5Pkk8RkYKRjgwMUZGODBGRkY4MDFG RjgwMEZDMDAwN0MwMDA3ODAwMDM4MDAwN0MwMDAzMDAwMDNDMDAwMjAwMDAzQzAwMDIwMDAw MUUwCjAwNDAwMDAxRTAwMDQwMDAwMUYwMDA0MDAwMDBGMDAwODAwMDAwRjAwMDgwMDAwMDc4 MDEwMDAwMDA3ODAxMDAwMDAwN0MwMQowMDAwMDAzQzAyMDAwMDAwM0MwMjAwMDAwMDFFMDQw MDAwMDAxRTA0MDAwMDAwMUYwNDAwMDAwMDBGMDgwMDAwMDAwRjA4MDAKMDAwMDA3OTAwMDAw MDAwNzkwMDAwMDAwMDdEMDAwMDAwMDAzRTAwMDAwMDAwM0UwMDAwMDAwMDFDMDAwMDAwMDAx QzAwMDAwCjAwMDFDMDAwMDAwMDAwODAwMDAwMDAwMDgwMDAwMDAwMDEwMDAwMDAwMDAxMDAw MDAwMDAwMTAwMDAwMDAwMDIwMDAwMDAwMAowMjAwMDAwMDAwMDQwMDAwMDA3MDA0MDAwMDAw RjgwQzAwMDAwMEY4MDgwMDAwMDBGODEwMDAwMDAwNzAzMDAwMDAwMDMwNDAKMDAwMDAwMUY4 MDAwMDAwMDIxMkQ3RjlFMjM+MTIxIEQgRSAvRm8gNSA4NSBkZjwwMDAwMDAwMDYwMDAwMDAw MDAwMEUwMDAKMDAwMDAwMDBFMDAwMDAwMDAwMDFFMDAwMDAwMDAwMDNFMDAwMDAwMDAwMDNF MDAwMDAwMDAwMDdFMDAwMDAwMDAwMDdGMDAwCjAwMDAwMDBERjAwMDAwMDAwMDBERjAwMDAw MDAwMDE5RjAwMDAwMDAwMDM5RjAwMDAwMDAwMDMxRjAwMDAwMDAwMDYxRjAwMAowMDAwMDA2 MUYwMDAwMDAwMDBDMUYwMDAwMDAwMDBDMUYwMDAwMDAwMDE4MUYwMDAwMDAwMDM4MUYwMDAw MDAwMDMwMUYwMDAKMDAwMDA2MDFGMDAwMDAwMDA2MDFGMDAwMDAwMDBDMDFGODAwMDAwMDBD MDBGODAwMDAwMDE4MDBGODAwMDAwMDM4MDBGODAwCjAwMDAzMDAwRjgwMDAwMDA2MDAwRjgw MDAwMDA2MDAwRjgwMDAwMDBDMDAwRjgwMDAwMDBDMDAwRjgwMDAwMDFGRkZGRjgwMAowMDAx RkZGRkY4MDAwMDAzMDAwMEY4MDAwMDA2MDAwMEY4MDAwMDA2MDAwMEY4MDAwMDBDMDAwMEY4 MDAwMDBDMDAwMEZDMDAKMDAxODAwMDA3QzAwMDAxODAwMDA3QzAwMDAzMDAwMDA3QzAwMDA3 MDAwMDA3QzAwMDA2MDAwMDA3QzAwMDBDMDAwMDA3QzAwCjAwQzAwMDAwN0MwMDAxQzAwMDAw N0MwMDAzQzAwMDAwN0MwMDBGRTAwMDAwRkUwMEZGRjgwMDFGRkZFMEZGRjgwMDFGRkZFMAoy QjMyN0NCMTM0PjY1IEQ8MDAwRkZGRkZGQzAwMDAwMDBGRkZGRkZGODAwMDAwMDA3RTAwMEZF MDAwMDAwMDdDMDAwM0YwMDAKMDAwMDdDMDAwMEY4MDAwMDAwN0MwMDAwN0MwMDAwMDA3QzAw MDA3QzAwMDAwMEY4MDAwMDNFMDAwMDAwRjgwMDAwMUUwMDAwCjAwRjgwMDAwMUYwMDAwMDBG ODAwMDAxRjAwMDAwMUYwMDAwMDFGMDAwMDAxRjAwMDAwMEY4MDAwMDFGMDAwMDAwRjgwMDAw MQpGMDAwMDAwRjgwMDAwM0UwMDAwMDBGODAwMDAzRTAwMDAwMEY4MDAwMDNFMDAwMDAwRjgw MDAwM0UwMDAwMDFGODAwMDA3QzAKMDAwMDFGODAwMDA3QzAwMDAwMUY4MDAwMDdDMDAwMDAx RjgwMDAwN0MwMDAwMDFGODAwMDBGODAwMDAwMUYwMDAwMEY4MDAwCjAwM0YwMDAwMEY4MDAw MDAzRjAwMDAwRjgwMDAwMDNGMDAwMDFGMDAwMDAwN0UwMDAwMUYwMDAwMDA3RTAwMDAxRjAw MDAwMAo3RTAwMDAxRjAwMDAwMDdDMDAwMDNFMDAwMDAwRkMwMDAwM0UwMDAwMDBGODAwMDAz RTAwMDAwMUY4MDAwMDNFMDAwMDAxRjAKMDAwMDdDMDAwMDAzRTAwMDAwN0MwMDAwMDNFMDAw MDA3QzAwMDAwN0MwMDAwMDdDMDAwMDBGODAwMDAwRjgwMDAwMEYwMDAwCjAwRjgwMDAwMUUw MDAwMDBGODAwMDAzQzAwMDAwMEY4MDAwMDc4MDAwMDAxRjAwMDAxRjAwMDAwMDFGMDAwMDNF MDAwMDAwMQpGMDAwMEY4MDAwMDAwM0YwMDA3RjAwMDAwMEZGRkZGRkZDMDAwMDAwRkZGRkZG QzAwMDAwMDAzMTMxN0JCMDM1PjY4CkQ8MDAwRkZGRkZGRkY4MDAwRkZGRkZGRkY4MDAwMDdF MDAwM0Y4MDAwMDdDMDAwMEY4MDAwMDdDMDAwMDc4MDAwMDdDMDAwMAo3ODAwMDA3QzAwMDAz MDAwMDBGODAwMDAzMDAwMDBGODAwMDAzMDAwMDBGODAwMDAzMDAwMDBGODAwMDAzMDAwMDFG MDAwMDAKMzAwMDAxRjAwMDAwMzAwMDAxRjAwMDAwMzAwMDAxRjAwMDAwNjAwMDAzRTAwMTgw NjAwMDAzRTAwMTgwMDAwMDAzRTAwMTgwCjAwMDAwM0UwMDE4MDAwMDAwN0MwMDMwMDAwMDAw N0MwMDMwMDAwMDAwN0MwMDcwMDAwMDAwN0MwMUYwMDAwMDAwRkZGRkUwMAowMDAwMEZGRkZF MDAwMDAwMEY4MDFFMDAwMDAwMEY4MDBFMDAwMDAwMUYwMDBDMDAwMDAwMUYwMDBDMDAwMDAw MUYwMDBDMDAKMDAwMDFGMDAwQzAwMDAwMDNFMDAxODAwMDAwMDNFMDAxODAwMDAwMDNFMDAw MDAwMDAwMDNFMDAwMDAwMDAwMDdDMDAwMDAwCjAwMDA3QzAwMDAwMDAwMDA3QzAwMDAwMDAw MDA3QzAwMDAwMDAwMDBGODAwMDAwMDAwMDBGODAwMDAwMDAwMDBGODAwMDAwMAowMDAwRjgw MDAwMDAwMDAxRjAwMDAwMDAwMDAxRjAwMDAwMDAwMDAxRjAwMDAwMDAwMDAzRjAwMDAwMDAw MEZGRkZFMDAwMDAKMDBGRkZGRTAwMDAwMDAyRDMxN0JCMDJFPjcwIEQ8MDAwRkZGRkZFMDAw MDAwRkZGRkZGQzAwMDAwMDdFMDAzRjAwMDAwMDdDCjAwMEY4MDAwMDA3QzAwMDdDMDAwMDA3 QzAwMDNFMDAwMDA3QzAwMDNFMDAwMDBGODAwMDFGMDAwMDBGODAwMDFGMDAwMDBGOAowMDAx RjAwMDAwRjgwMDAzRjAwMDAxRjAwMDAzRjAwMDAxRjAwMDAzRjAwMDAxRjAwMDAzRjAwMDAx RjAwMDAzRTAwMDAzRTAKMDAwN0UwMDAwM0UwMDAwN0MwMDAwM0UwMDAwRkMwMDAwM0UwMDAw RjgwMDAwN0MwMDAxRjAwMDAwN0MwMDAzRTAwMDAwN0MwCjAwNzgwMDAwMDdDMDAxRTAwMDAw MEY4MDBGODAwMDAwMEZGRkZDMDAwMDAwMEY4MDBFMDAwMDAwMEY4MDA3MDAwMDAwMUYwMAow MzgwMDAwMDFGMDAwM0MwMDAwMDFGMDAwMUUwMDAwMDFGMDAwMUUwMDAwMDNFMDAwMUYwMDAw MDNFMDAwMUYwMDAwMDNFMDAKMDFGMDAwMDAzRTAwMDFGMDAwMDA3QzAwMDNGMDAwMDA3QzAw MDNGMDAwMDA3QzAwMDNFMDAwMDA3QzAwMDNFMDAwMDBGODAwCjA3RTAwMDAwRjgwMDA3RTAw MDAwRjgwMDA3RTAwMDAwRjgwMDA3RTAxODAxRjAwMDA3RTAzMDAxRjAwMDA3RTAzMDAxRjAw MAowN0UwMjAwM0YwMDAwM0UwNjBGRkZGODAwMUUwQzBGRkZGODAwMEYxODAwMDAwMDAwMDNF MDAyRDMyN0JCMDMzPjgyCkQ8MDdGRkZGRkZGRkYwMDdGRkZGRkZGRkYwMEZFMDA3RTAwN0Yw MEY4MDA3QzAwMUYwMEUwMDA3QzAwMEUwMEMwMDA3QzAwMApFMDFDMDAwN0MwMDBFMDE4MDAw RjgwMDBFMDE4MDAwRjgwMDBFMDMwMDAwRjgwMDBDMDMwMDAwRjgwMDBDMDMwMDAxRjAwMDAK QzA2MDAwMUYwMDAwQzA2MDAwMUYwMDAwQzA2MDAwMUYwMDAwQzBDMDAwM0UwMDAxODAwMDAw M0UwMDAwMDAwMDAwM0UwMDAwCjAwMDAwMDNFMDAwMDAwMDAwMDdDMDAwMDAwMDAwMDdDMDAw MDAwMDAwMDdDMDAwMDAwMDAwMDdDMDAwMDAwMDAwMEY4MDAwMAowMDAwMDBGODAwMDAwMDAw MDBGODAwMDAwMDAwMDBGODAwMDAwMDAwMDFGMDAwMDAwMDAwMDFGMDAwMDAwMDAwMDFGMDAw MDAKMDAwMDAxRjAwMDAwMDAwMDAzRTAwMDAwMDAwMDAzRTAwMDAwMDAwMDAzRTAwMDAwMDAw MDAzRTAwMDAwMDAwMDA3QzAwMDAwCjAwMDAwN0MwMDAwMDAwMDAwN0MwMDAwMDAwMDAwN0Mw MDAwMDAwMDAwRjgwMDAwMDAwMDAwRjgwMDAwMDAwMDAwRjgwMDAwMAowMDAwMEY4MDAwMDAw MDAwMUYwMDAwMDAwMDAwMUYwMDAwMDAwMDAwMUYwMDAwMDAwMDAwM0YwMDAwMDAwMDNGRkZG RjAwMDAKMDAzRkZGRkYwMDAwMDAyQzMxNzRCMDMyPjg0IEQgRSBlbmQKJSVFbmRQcm9sb2cK JSVCZWdpblNldHVwCiUlRmVhdHVyZTogKlJlc29sdXRpb24gMzAwZHBpClRlWERpY3QgYmVn aW4KCiUlRW5kU2V0dXAKJSVQYWdlOiAwIDEKMCAwIGJvcCA3OTkgOTg4IGEgRm8oRCkyNSBi KFIpZyhBKWcoRilnKFQpMzc1IDEwNzkgeQpGbihNUEktMilkKER5bmFtaWMpZShQcm8pcihj ZXNzKWkoQ2hhcHRlcjopMjkgYihQKW4oYXJ0KTIyCmIoSSlyKEkpNjIxIDEyNzQgeSBGbShN ZXNzYWdlKTE2IGIoUClvKGFzc2luZyloKEluKW8odGVyZmFjZSllKEYpbAoob3J1bSk4MjQg MTQwMCB5KEFwcmlsKWcoMTAsKWkoMTk5NilwIGVvcAolJVBhZ2U6IDEgMgoxIDEgYm9wIDE4 NTIgLTEwMCBhIEZsKDEpNzUgNDkgeSBGaygyLjMuMSk0OSBiKFN0YSlvKHJ0aW5nKTE2CmIo SW5kZXApcShlbmRlbnQpZyhQcm8pcShjZXNzZXMpNzUgMTM1IHkgRmwoVGhlKWoocHJldmlv dXMpZyhzZWN0aW9uKWcKKGRlc2NyaWIpcShlZCloKGhvKW8odyllKGFuKWcoTVBJKWcoYXBw bGljYXRpb24paShjb3VsZClnKHNwYSlvKHduKWUKKG5ldylnKE1QSSloKHBybylxKGNlc3Nl cyk3NSAxOTIgeShhbmQpYyhlc3RhYmxpc2gpaShjb21tKW8odW5pY2F0aW9uKWYKKHdpdGgp Zih0aGVtLikxNjYgMjQ4IHkoVGhpcylpKHNlY3Rpb24pZihkZXNjcmliKXEoZXMpaShobylv KHcpZSh0bylmCihzcGEpbyh3biloIEZqKGluZGVwKW4oZW5kZW50KWYgRmwocHJvKXEoY2Vz c2VzLCloKHRoYXQpZyhpcywpZyhwcm8pcQooY2Vzc2VzKWgodGhhdCllKGRvKTc1IDMwNCB5 KG5vdClnKGNvbW0pbyh1bmljYXRlKWkod2l0aClmKHRoZWlyKWcKKHBhcmVuKW8odC4pMjIg YihUaGVzZSkxNiBiKG1hKW8oeSlmKGIpcShlKWkoTVBJKWYocHJvKXEoY2Vzc2VzKWcKKFwo aS5lLiwpZihwcm8pcShjZXNzZXMpaSh0aGF0KWUoY2FsbCk3NSAzNjEgeSBGaShNUEkpcAox NjAgMzYxIDE0IDIgdiAxNiB3KElOSVQpcCBGbChcKSlmKG9yKWgobm90LikxNjYgNTAwCnkg RmgoRGlzY3Vzc2lvbjopMjcgYiBGZyhUaGlzKTIwIGIoc2VjdGlvbilnKGhhcylnKGEpZyhs YXJnZSlnKG4pbyh1bSlvCihiKXEoZXIpZihvZilnKGhhaXJ5KWcoZnVuY3Rpb25zLClpKGd1 YXJhbilvKHRlZWQpZih0bylnKHNjYXJlKWgoYW5kKTc1CjU1NiB5KGNvbmZ1c2UpMTUgYih1 c2Vycy4pNzUgNzkwIHkgRmkoTVBJKXAgMTYwIDc5MCBWCjE2IHcoU1ApbChBKS01IGIoV04p cCAzMjYgNzkwIFYgMTcgdyhJTkRFUEVOREVOVClwIDY1OQo3OTAgViAxNyB3KE1QSVwoY29t bWFuZClwIDk2MSA3OTAgViAxNCB3KGxpbmUsKTE1IGIobWlucClvKHJvKXEoY3MsKWcKKG1h eHApbyhybylxKGNzLClmKGluZm8sKWgocm8pcShvdCwpZihjb21tLCk3NSA4NDcgeShncm91 cCwpaChhKW8ocnJhKW8KKHkpcCAzMDkgODQ3IFYgMTQgdyhvZilwIDM2MCA4NDcgViAxNiB3 KGVycmNvKXEoZGVzXCkpMTE3CjkyNCB5IEZnKElOKTE1NSBiIEZpKGNvbW1hbmQpcCA1MDYg OTI0IFYgMTYgdyhsaW5lKTMyMQpiIEZnKGV4ZWN1dGFibGUpMTggYihhbmQpZihhcmd1bWVu KW8odHMsKWcoaW4pZyhhKWYoc2luZ2xlKWgoc3RyaW5nKWcKKFwoc3RyaW5nLCk5MDUgOTgw IHkoc2lnbmlcMDE0Y2FuKW8odClkKG9ubHkpZihhdClnKHJvKXEob3RcKSkxMTcKMTA1NSB5 KElOKTE1NSBiIEZpKG1pbnApbyhybylxKGNzKTQxNyBiIEZnKG1pbmltKW4odW0pMTQKYihu KW8odW0pbyhiKXEoZXIpaihvZilnKHBybylxKGNlc3NlcylrKHRvKWMoc3RhcnQpaShcKGlu KW8odGVnZXIsKWcKKHNpZy0pOTA1IDExMTIgeShuaVwwMTRjYW4pbyh0KTE0IGIob25seSlm KGF0KWgocm8pcShvdFwpKTExNwoxMTg3IHkoSU4pMTU1IGIgRmkobWF4cClvKHJvKXEoY3Mp NDA4IGIgRmcobWF4aW0pbih1bSkxMwpiKG4pbyh1bSlvKGIpcShlcilqKG9mKWcocHJvKXEo Y2Vzc2VzKWoodG8pZShzdGFydClnKFwoaW4pbyh0ZWdlciwpaAooc2lnLSk5MDUgMTI0MyB5 KG5pXDAxNGNhbilvKHQpYyhvbmx5KWYoYXQpaChybylxKG90XCkpMTE3CjEzMTkgeShJTikx NTUgYiBGaShpbmZvKTUxNiBiIEZnKGEpMTQgYihzdHJpbmcpZyh0ZWxsaW5nKWYodGhlKWko cnVuKW8KKHRpbWUpZShzeXN0ZW0pZyh3aGVyZSlqKGFuZClkKGhvKW8odyloKHRvKTkwNSAx Mzc1IHkoc3RhcnQpaCh0aGUpZgoocHJvKXEoY2Vzc2VzKWooXChzdHJpbmcsKWMoc2lnbmlc MDE0Y2FuKW8odCloKG9ubHkpZihhdCloKHJvKXEob3RcKSkKMTE3IDE0NTAgeShJTikxNTUg YiBGaShybylxKG90KTUwOCBiIEZnKHJhbmspMTMgYihvZilmKHBybylxKGNlc3MpaihpbikK ZCh3aGljKW8oaCloKHByZXZpb3VzKWcoYXJndW1lbilvKHRzKWYoYXJlKWkodiluKGFsaWQp OTA1CjE1MDcgeShcKGluKW8odGVnZXJcKSkxMTcgMTU4MiB5KElOKTE1NSBiIEZpKGNvbW0p NDcwCmIgRmcoY29tbSlvKHVuaWNhdG9yKTcgYihvZilpKGdyb3VwKWcob2YpZyhzcGEpbyh3 bmluZylnKHBybylxKGNlc3NlcylqCihcKGhhbmRsZVwpKTExNyAxNjU3IHkoT1VUKTEwOCBi IEZpKGdyb3VwKTQ3OSBiIEZnKGdyb3VwKTE0CmIob2YpZihzcGEpbyh3bmVkKWkocHJvKXEo Y2Vzc2VzKWgoXChoYW5kbGVcKSkxMTcgMTczMgp5KE9VVCkxMDggYiBGaShhKW8ocnJhKW8o eSlwIDQxNiAxNzMyIFYgMTUgdyhvZilwIDQ2OAoxNzMyIFYgMTYgdyhlcnJjbylxKGRlcyky NjggYiBGZyhvbmUpMTQgYihjbylxKGRlKWgocClxKGVyKWcocHJvKXEKKGNlc3MpaChcKGFy cmEpbyh5KWQob2YpaChpbilvKHRlZ2VyXCkpNzUgMTg1NiB5IEZmKGludCkyMwpiKE1QSSlw IDI0NSAxODU2IDE1IDIgdiAxNyB3KFNwYXduKXAgMzgyIDE4NTYgViAxNyB3KGluZGVwZW5k ZW50KXAKNjYzIDE4NTYgViAxNSB3KG1waVwoY2hhciopZyhjb21tYW5kKXAgMTA4NSAxODU2 IFYgMTYKdyhsaW5lLClnKGludCloKG1pbnByb2NzLCkzOTMgMTkxMyB5KGludClnKG1heHBy b2NzLCllKGNoYXIqKWgoaW5mbywpaAooaW50KWYocm9vdCwpZyhNUEkpcCAxMzI3IDE5MTMg ViAxNyB3KENvbW0pZyhjb21tLCkzOTMKMTk2OSB5KE1QSSlwIDQ2OCAxOTY5IFYgMTcgdyhH cm91cCopZyhncm91cCwpZyhpbnQqKWcoYXJyYXkpcAoxMDU4IDE5NjkgViAxNyB3KG9mKXAg MTEyMyAxOTY5IFYgMTcgdyhlcnJjb2Rlc1wpKTc1CjIwNTYgeShNUEkpcCAxNTAgMjA1NiBW IDE3IHcoU1BBV04pcCAyODcgMjA1NiBWIDE2IHcoSU5ERVBFTkRFTlQpcAo1NjcgMjA1NiBW IDE2IHcoTVBJXChDT01NQU5EKXAgODQ3IDIwNTYgViAxNiB3KExJTkUsKWcoTUlOUFJPQ1Ms KWYKKE1BWFBST0NTLCloKElORk8sKWcoUk9PVCwpMzkzIDIxMTIgeShDT01NLClnKElOVEVS Q09NTSwpZyhBUlJBWSlwCjkyMiAyMTEyIFYgMTYgdyhPRilwIDk4NiAyMTEyIFYgMTcgdyhF UlJDT0RFUywpZyhJRVJSXCkpMTcwCjIxNjkgeShDSEFSQUNURVIqXCgqXCkpZyhDT01NQU5E KXAgNjc2IDIxNjkgViAxNiB3KExJTkUsKWcoSU5GTykxNzAKMjIyNSB5KElOVEVHRVIpZyhN SU5QUk9DUywpZyhNQVhQUk9DUywpZyhST09ULClnKENPTU0sKWcoR1JPVVAsKWcKKEFSUkFZ KXAgMTQxNSAyMjI1IFYgMTcgdyhPRilwIDE0ODAgMjIyNSBWIDE2IHcoRVJSQ09ERVNcKCpc KSwpMTcwCjIyODIgeShJRVJSKTE2NiAyMzY4IHkgRmwoVGhpcykxOCBiKHJvdXRpbmUpZyhz cGEpbyh3bnMpZyhhbilnKGluZGVwKXEKKGVuZGVuKW8odClpKE1QSSlkKGFwcGxpY2F0aW9u LikzMCBiKEl0KTE3IGIoaXMpaShpZGVuKW8odGljYWwpZyh0byllCkZpKE1QSSlwIDE3MTEg MjM2OCAxNCAyIHYgMTYgdyhTUClsKEEpLTUgYihXTik3NSAyNDI1CnkgRmwoZXhjZXB0KTE5 IGIodGhhdClnKGl0KWcoZG8pcShlcylnKG5vdClnKGVzdGFibGlzaCloKGNvbW0pbwoodW5p Y2F0aW9uKWYod2l0aCloKHRoZSlmKGMpbyhoaWxkcmVuLClpKHNvKWUodGhhdClmKGl0KWgo cmV0dXJucylnKGFuKQo3NSAyNDgxIHkoTVBJKTEyIGIoZ3JvdXApZyhyYXRoZXIpZyh0aGFu KWcoYSlnKGNvbW0pbyh1bmljYXRvci4pMTgKYihJdCkxMiBiKGlzKWgoY29sbGVjdGl2KW8o ZSloKGluKWYodGhlKWYocGFyZW4pbyh0KWcoY29tbSlvKHVuaWNhdG9yKWcKKGJ1dClnKG5v dCk3NSAyNTM3IHkod2l0aCkxOCBiKHRoZSlnKGMpbyhoaWxkcmVuLikyOQpiKFN1Y2Nlc3Nm dWwpMTkgYihjb21wbGV0aW9uKWcoZG8pcShlcylmKG5vdClmKGluZGljYXRlKWkodGhhdCll KHRoZSloCihwcm8pcShjZXNzZXMpZyhoYSlvKHYpbyhlKWYoY2FsbGVkKTc1IDI1OTQgeSBG aShNUEkpcAoxNjAgMjU5NCBWIDE2IHcoSU5JVClkIEZsKHN1Y2Nlc3NmdWxseSlsKCwpaihv bmx5KWYodGhhdClmKHRoZSlnKHBybylxCihjZXNzZXMpZyh3KW8oZXJlKWcoc3RhcnRlZCln KHN1Y2Nlc3NmdWxseSlsKC4pLTMyIDQ2CnkgRmUoMSktMzIgMTAzIHkoMiktMzIgMTU5IHko MyktMzIgMjE1IHkoNCktMzIgMjcyIHkoNSktMzIKMzI4IHkoNiktMzIgMzg1IHkoNyktMzIg NDQxIHkoOCktMzIgNDk4IHkoOSktNDAgNTU0IHkoMTApLTQwCjYxMSB5KDExKS00MCA2Njcg eSgxMiktNDAgNzI0IHkoMTMpLTQwIDc4MCB5KDE0KS00MCA4MzYKeSgxNSktNDAgODkzIHko MTYpLTQwIDk0OSB5KDE3KS00MCAxMDA2IHkoMTgpLTQwIDEwNjIKeSgxOSktNDAgMTExOSB5 KDIwKS00MCAxMTc1IHkoMjEpLTQwIDEyMzIgeSgyMiktNDAgMTI4OAp5KDIzKS00MCAxMzQ1 IHkoMjQpLTQwIDE0MDEgeSgyNSktNDAgMTQ1NyB5KDI2KS00MCAxNTE0CnkoMjcpLTQwIDE1 NzAgeSgyOCktNDAgMTYyNyB5KDI5KS00MCAxNjgzIHkoMzApLTQwIDE3NDAKeSgzMSktNDAg MTc5NiB5KDMyKS00MCAxODUzIHkoMzMpLTQwIDE5MDkgeSgzNCktNDAgMTk2Ngp5KDM1KS00 MCAyMDIyIHkoMzYpLTQwIDIwNzggeSgzNyktNDAgMjEzNSB5KDM4KS00MCAyMTkxCnkoMzkp LTQwIDIyNDggeSg0MCktNDAgMjMwNCB5KDQxKS00MCAyMzYxIHkoNDIpLTQwIDI0MTcKeSg0 MyktNDAgMjQ3NCB5KDQ0KS00MCAyNTMwIHkoNDUpLTQwIDI1ODcgeSg0NiktNDAgMjY0Mwp5 KDQ3KS00MCAyNjk5IHkoNDgpcCBlb3AKJSVQYWdlOiAyIDMKMiAyIGJvcCA3NSAtMTAwIGEg RmwoMikxNjYgNDkgeSBGaChEaXNjdXNzaW9uOikzNCBiIEZnKEFsdGVybmF0aXYpbwooZWx5 KW0oLCkxMiBiKHcpbyhlKWkoY291bGQpZihzYSlvKHkpaCh0aGF0KWYodGhlKWkoY2FsbCll KGRvKXEoZXMpaAoobm90KWYocmV0dXJuKWkodW4pbyh0aWwpZSh0aGUpaChjKW8oaGlsZHJl bilnKGhhKW8odilvKGUpNzUKMTA2IHkoc3VjY2Vzc2Z1bGx5KWooaW5pdGlhbGl6ZWQpZChN UEkuKWkoSW4pZyh0aGlzKWYoY2FzZSwpaSh0aGUpZwooY2FsbCllKHcpbyhvdWxkKWcoYilx KGUpaChlXDAxM2VjdGl2KW8oZWx5KWgoY29sbGVjdGl2KW8oZSllKHdpdGgpaAoodGhlKWco YylvKGhpbGRyZW4uKTc1IDE2MiB5KEFub3RoZXIpZih3KW8oYSlvKHkpZSh0bylnKGhhbmRs ZSloKHRoaXMpZwoodylvKG91bGUpZihiKXEoZSlpKHRvKWUoYnJpbmcpaChiYWMpbyhrKWYo dGhlKWkgRmQoXDAxNWFncyllCkZnKGFyZ3VtZW4pbyh0KWcoYW5kKWgoYWRkKWYgRmQoTVBJ KXAgMTYwMyAxNjIgMTMgMiB2CjE1IHcoU1ApbShBKS01IGIoV04pcCAxNzU2IDE2MiBWIDE0 IHcoU1lOQyk3NSAyMTkgeSBGZyh0bykxNQpiKHRoZSloKGxpc3QpZShvZiloKHN1cHApcShv cnRlZCloKFwwMTVhZ3MsKWYoc28pZyh0aGF0KWcodGhlKWgodXNlcilnCihoYXMpZihhKWco YylvKGhvaWNlKWgoYWIpcShvdXQpZih3aGV0aGVyKWgodGhlKWcoc3BhKW8od24pZih3KW8o YWl0cylnCihmb3IpZih0aGUpNzUgMjc1IHkoYylvKGhpbGRyZW4pZyh0bylnKGIpcShlY29t ZSlnKE1QSSlnKHBybylxKGNlc3Nlcy4pCjE2NiA0MTQgeSBGbChUaGUpZChjKW8oaGlsZClp KHBybylxKGNlc3NlcylmKGFyZSlmIEZqKHIpbihlKW4ocXVpciluKGUpCm4oZClmIEZsKHRv KWgoY2FsbCloIEZpKE1QSSlwIDEwMDYgNDE0IDE0IDIgdiAxNiB3KElOSVQpcApGbCguKWUo QWxsKWkoYylvKGhpbGRyZW4paShzcGEpbyh3bmVkKWQoaW4paChhKWYoc2luZ2xlKWgoY2Fs bCloKHRvKTc1CjQ3MSB5IEZpKE1QSSlwIDE2MCA0NzEgViAxNiB3KFNQKWwoQSktNSBiKFdO KXAgMzI2IDQ3MQpWIDE3IHcoSU5ERVBFTkRFTlQpcCA2NTkgNDcxIFYgMTcgdyhNUEkpMTgg YiBGbChoYSlvKHYpbyhlKWcodGhlKWcKKHNhbWUpZyBGaShNUEkpcCAxMTY0IDQ3MSBWIDE2 IHcoQ09NTSlwIDEzMjIgNDcxIFYgMTYKdyhXKW8oT1JMRClwIEZsKCwpZyh3aGljKW8oaClp KGlzKWUoc2VwYXJhdGUpNzUgNTI3IHkoZnJvbSljKHRoYXQpaChvZikKZyh0aGUpZyhwYXJl bilvKHQuKWsoSW4pZCh0aGUpZyhjKW8oaGlsZHJlbiwpZyBGaShNUEkpcAo5NzcgNTI3IFYg MTYgdyhQKWwoQVJFTlQpZyBGbChyZXR1cm5zKWcgRmkoTVBJKXAgMTQyMwo1MjcgViAxNSB3 KENPTU0pcCAxNTgwIDUyNyBWIDE3IHcoTlVMTClwIEZsKC4pMTg5IDYzMwp5IEZqKEEpbihk dmljKW4oZSlmKHRvKWgodXNlcnMuKTQwIGIgRmwoSXQpMTUgYihpcylnKG5vdClmKGNvcnJl Y3QpaAoodG8pZih1c2UpaCBGaShNUEkpcCAxMDg4IDYzMyBWIDE2IHcoU1ApbChBKS01IGIo V04pcAoxMjU0IDYzMyBWIDE3IHcoSU5ERVBFTkRFTlQpcCAxNTg3IDYzMyBWIDE3IHcoTVBJ KTE0CmIgRmwodG8pZyhzcGEpbyh3bikxODkgNjkwIHkoYSlrKHNoZWxsKWkoc2NyaXB0KWYo dGhhdClmKHN0YXJ0cylmKGFuKWkKKE1QSSlnKHByb2dyYW0uKTI5IGIoVGhlKTE5IGIocHJv Z3JhbSllKHNwKXEoZWNpXDAxNGVkKWsoaW4pZQpGaShjb21tYW5kKXAgMTc5NyA2OTAgViAx NiB3KGxpbmUpMTg5IDc0NiB5IEZsKG0pbyh1c3QpMTQKYihpdHNlbGYpaShiKXEoZSlnKGFu KWYoTVBJKWgocHJvZ3JhbSllKHRoYXQpZyhjYWxscylpCkZpKE1QSSlwIDExMDcgNzQ2IFYg MTYgdyhJTklUKXAgRmwoLillKFwoKXAgRmooRW5kKWkob2YpZyhhZHZpYyluKGUpZwoodG8p aCh1c2Vycy4pcCBGbChcKSk3NSA5NDcgeSBGaShNUEkpcCAxNjAgOTQ3IFYgMTYgdyhTUCls KEEpLTUKYihXTilwIDMyNiA5NDcgViAxNyB3KElOREVQRU5ERU5UKXAgNjU5IDk0NyBWIDE3 IHcoTk9OTVBJXChjb21tYW5kKXAKMTA1OCA5NDcgViAxNSB3KGxpbmUsKTExIGIobWlucClv KHJvKXEoY3MsKWcobWF4cClvKHJvKXEoY3MsKWYoaW5mbywpZwoocm8pcShvdCwpaChjb21t LCk3NSAxMDA0IHkoZ3JvdXAsKWsoYSlvKHJyYSlvKHkpcCAzMDkKMTAwNCBWIDE0IHcob2Yp cCAzNjAgMTAwNCBWIDE2IHcoZXJyY28pcShkZXNcKSkxMTcgMTA4MQp5IEZnKElOKTE1NSBi IEZpKGNvbW1hbmQpcCA1MDYgMTA4MSBWIDE2IHcobGluZSkzMjEgYgpGZyhleGVjdXRhYmxl KTE4IGIoYW5kKWYoYXJndW1lbilvKHRzLClnKGluKWcoYSlmKHNpbmdsZSloKHN0cmluZyln CihcKHN0cmluZywpOTA1IDExMzcgeShzaWduaVwwMTRjYW4pbyh0KWQob25seSlmKGF0KWco cm8pcShvdFwpKTExNwoxMjEyIHkoSU4pMTU1IGIgRmkobWlucClvKHJvKXEoY3MpNDE3IGIg RmcobWluaW0pbih1bSkxNApiKG4pbyh1bSlvKGIpcShlcilqKG9mKWcocHJvKXEoY2Vzc2Vz KWsodG8pYyhzdGFydClpKFwoaW4pbyh0ZWdlciwpZwooc2lnLSk5MDUgMTI2OSB5KG5pXDAx NGNhbilvKHQpMTQgYihvbmx5KWYoYXQpaChybylxKG90XCkpMTE3CjEzNDQgeShJTikxNTUg YiBGaShtYXhwKW8ocm8pcShjcyk0MDggYiBGZyhtYXhpbSluKHVtKTEzCmIobilvKHVtKW8o YilxKGVyKWoob2YpZyhwcm8pcShjZXNzZXMpaih0byllKHN0YXJ0KWcoXChpbilvKHRlZ2Vy LCloCihzaWctKTkwNSAxNDAwIHkobmlcMDE0Y2FuKW8odCljKG9ubHkpZihhdCloKHJvKXEo b3RcKSkxMTcKMTQ3NSB5KElOKTE1NSBiIEZpKGluZm8pNTE2IGIgRmcoYSkxNCBiKHN0cmlu ZylnKHRlbGxpbmcpZih0aGUpaShydW4pbwoodGltZSllKHN5c3RlbSlnKHdoZXJlKWooYW5k KWQoaG8pbyh3KWgodG8pOTA1IDE1MzIgeShzdGFydCloKHRoZSlmCihwcm8pcShjZXNzZXMp aihcKHN0cmluZywpYyhzaWduaVwwMTRjYW4pbyh0KWgob25seSlmKGF0KWgocm8pcShvdFwp KQoxMTcgMTYwNyB5KElOKTE1NSBiIEZpKHJvKXEob3QpNTA4IGIgRmcocmFuaykxMyBiKG9m KWYocHJvKXEoY2VzcylqKGluKQpkKHdoaWMpbyhoKWgocHJldmlvdXMpZyhhcmd1bWVuKW8o dHMpZihhcmUpaSh2KW4oYWxpZCk5MDUKMTY2NCB5KFwoaW4pbyh0ZWdlclwpKTExNyAxNzM5 IHkoSU4pMTU1IGIgRmkoY29tbSk0NzAKYiBGZyhjb21tKW8odW5pY2F0b3IpNyBiKG9mKWko Z3JvdXApZyhvZilnKHNwYSlvKHduaW5nKWcocHJvKXEoY2Vzc2VzKWoKKFwoaGFuZGxlXCkp MTE3IDE4MTQgeShPVVQpMTA4IGIgRmkoZ3JvdXApNDc5IGIgRmcoZ3JvdXApMTQKYihjb24p byh0YWluaW5nKWYoc3BhKW8od25lZCloKHBybylxKGNlc3NlcylqKFwoaGFuZGxlXCkpMTE3 CjE4ODkgeShPVVQpMTA4IGIgRmkoYSlvKHJyYSlvKHkpcCA0MTYgMTg4OSBWIDE1IHcob2Yp cAo0NjggMTg4OSBWIDE2IHcoZXJyY28pcShkZXMpMjY4IGIgRmcob25lKTE0IGIoY28pcShk ZSloKHApcShlcilnKHBybylxCihjZXNzKWgoXChhcnJhKW8oeSlkKG9mKWgoaW4pbyh0ZWdl clwpKTc1IDIwMTMgeSBGZihpbnQpMjMKYihNUEkpcCAyNDUgMjAxMyAxNSAyIHYgMTcgdyhT cGF3bilwIDM4MiAyMDEzIFYgMTcgdyhpbmRlcGVuZGVudClwCjY2MyAyMDEzIFYgMTUgdyhu b25tcGlcKGNoYXIqKWcoY29tbWFuZClwIDExNTcgMjAxMyBWCjE2IHcobGluZSwpZyhpbnQp ZyhtaW5wcm9jcywpMzkzIDIwNzAgeShpbnQpaChtYXhwcm9jcywpZShjaGFyKiloCihpbmZv LCloKGludClmKHJvb3QsKWcoTVBJKXAgMTMyNyAyMDcwIFYgMTcgdyhDb21tKWcoY29tbSwp MzkzCjIxMjYgeShNUEkpcCA0NjggMjEyNiBWIDE3IHcoR3JvdXAqKWcoZ3JvdXAsKWcoaW50 KilnKGFycmF5KXAKMTA1OCAyMTI2IFYgMTcgdyhvZilwIDExMjMgMjEyNiBWIDE3IHcoZXJy Y29kZXNcKSk3NQoyMjEzIHkoTVBJKXAgMTUwIDIyMTMgViAxNyB3KFNQQVdOKXAgMjg3IDIy MTMgViAxNiB3KElOREVQRU5ERU5UKXAKNTY3IDIyMTMgViAxNiB3KE5PTk1QSVwoQ09NTUFO RClwIDkxOSAyMjEzIFYgMTUgdyhMSU5FLClnKE1JTlBST0NTLClnCihNQVhQUk9DUywpZyhJ TkZPLClnKFJPT1QsKTM5MyAyMjY5IHkoQ09NTSwpZyhHUk9VUCwpZyhBUlJBWSlwCjgyNiAy MjY5IFYgMTcgdyhPRilwIDg5MSAyMjY5IFYgMTcgdyhFUlJDT0RFUywpZihJRVJSXCkpMTcw CjIzMjYgeShDSEFSQUNURVIqXCgqXCkpaChDT01NQU5EKXAgNjc2IDIzMjYgViAxNiB3KExJ TkUsKWcoSU5GTykxNzAKMjM4MiB5KElOVEVHRVIpZyhNSU5QUk9DUywpZyhNQVhQUk9DUywp ZyhST09ULClnKENPTU0sKWcoR1JPVVAsKWcKKEFSUkFZKXAgMTQxNSAyMzgyIFYgMTcgdyhP RilwIDE0ODAgMjM4MiBWIDE2IHcoRVJSQ09ERVNcKCpcKSwpMTcwCjI0MzkgeShJRVJSKTE2 NiAyNTI1IHkgRmwoVGhlKTE1IGIocm91dGluZSloIEZpKE1QSSlwCjUwMSAyNTI1IDE0IDIg diAxNiB3KFNQKWwoQSktNSBiKFdOKXAgNjY3IDI1MjUgViAxNyB3KElOREVQRU5ERU5UKXAK MTAwMCAyNTI1IFYgMTcgdyhOT05NUEkpMTUgYiBGbChpcyloKGlkZW4pbyh0aWNhbCloKHRv KTc1CjI1ODEgeSBGaShNUEkpcCAxNjAgMjU4MSBWIDE2IHcoU1ApbChBKS01IGIoV04pcCAz MjYKMjU4MSBWIDE3IHcoSU5ERVBFTkRFTlQpcCA2NTkgMjU4MSBWIDE3IHcoTVBJKTE2IGIg RmwoZXhjZXB0KWgodGhhdClmCih0aGUpaChjKW8oaGlsZCloKHBybylxKGNlc3NlcylnKGFy ZSllKGFzc3VtZWQpaCh0bylmKGIpcShlKWkobm9uLSk3NQoyNjM4IHkoTVBJKTExIGIocHJv KXEoY2Vzc2VzLikyMCBiKE1QSSkxMSBiKGlzKWgobm90KWYocmVxdWlyZWQpaSh0byllCihk bylnKGFuKW8oeSlnKHNwKXEoZWNpYWwpaihzZXR1cClkKHRvKWcoZW5zdXJlKWgodGhhdClm KGEpZyhjYWxsKWkodG8pCmUgRmkoTVBJKXAgMTc3NCAyNjM4IFYgMTYgdyhJTklUKTE5Njcg NDYgeSBGZSgxKTE5NjcKMTAzIHkoMikxOTY3IDE1OSB5KDMpMTk2NyAyMTUgeSg0KTE5Njcg MjcyIHkoNSkxOTY3IDMyOAp5KDYpMTk2NyAzODUgeSg3KTE5NjcgNDQxIHkoOCkxOTY3IDQ5 OCB5KDkpMTk1OSA1NTQgeSgxMCkxOTU5CjYxMSB5KDExKTE5NTkgNjY3IHkoMTIpMTk1OSA3 MjQgeSgxMykxOTU5IDc4MCB5KDE0KTE5NTkKODM2IHkoMTUpMTk1OSA4OTMgeSgxNikxOTU5 IDk0OSB5KDE3KTE5NTkgMTAwNiB5KDE4KTE5NTkKMTA2MiB5KDE5KTE5NTkgMTExOSB5KDIw KTE5NTkgMTE3NSB5KDIxKTE5NTkgMTIzMiB5KDIyKTE5NTkKMTI4OCB5KDIzKTE5NTkgMTM0 NSB5KDI0KTE5NTkgMTQwMSB5KDI1KTE5NTkgMTQ1NyB5KDI2KTE5NTkKMTUxNCB5KDI3KTE5 NTkgMTU3MCB5KDI4KTE5NTkgMTYyNyB5KDI5KTE5NTkgMTY4MyB5KDMwKTE5NTkKMTc0MCB5 KDMxKTE5NTkgMTc5NiB5KDMyKTE5NTkgMTg1MyB5KDMzKTE5NTkgMTkwOSB5KDM0KTE5NTkK MTk2NiB5KDM1KTE5NTkgMjAyMiB5KDM2KTE5NTkgMjA3OCB5KDM3KTE5NTkgMjEzNSB5KDM4 KTE5NTkKMjE5MSB5KDM5KTE5NTkgMjI0OCB5KDQwKTE5NTkgMjMwNCB5KDQxKTE5NTkgMjM2 MSB5KDQyKTE5NTkKMjQxNyB5KDQzKTE5NTkgMjQ3NCB5KDQ0KTE5NTkgMjUzMCB5KDQ1KTE5 NTkgMjU4NyB5KDQ2KTE5NTkKMjY0MyB5KDQ3KTE5NTkgMjY5OSB5KDQ4KXAgZW9wCiUlUGFn ZTogMyA0CjMgMyBib3AgMTg1MiAtMTAwIGEgRmwoMyk3NSA0OSB5KGluKTE4IGIodGhlKWYo YylvKGhpbGRyZW4paCh3aWxsKWgoYilxCihlKWUoYWJsZSloKHRvKWUoZXN0YWJsaXNoKWko Y29tbSlvKHVuaWNhdGlvbilnKGFtb25nKWUodGhlKWgoYylvCihoaWxkcmVuLikyNyBiKFRo ZSkxNyBiKGVcMDEzZWN0KWYob2YpNzUgMTA2IHkoY2FsbGluZyloCkZpKE1QSSlwIDMwNCAx MDYgMTQgMiB2IDE2IHcoSU5JVClkIEZsKGluKWkodGhlKWcoYylvKGhpbGRyZW4paChpcyll Cih1bmRlXDAxNG5lZC4pNzUgMjU3IHkgRmkoTVBJKXAgMTYwIDI1NyBWIDE2IHcoU1ApbChB KS01CmIoV04pcCAzMjYgMjU3IFYgMTcgdyhNVUwpbChUSVBMRSlwIDU2MCAyNTcgViAxNiB3 KElOREVQRU5ERU5UKXAKODkyIDI1NyBWIDE2IHcoTVBJXChjb3VudCwpMTYgYihhKW8ocnJh KW8oeSlwIDEyMzcgMjU3ClYgMTQgdyhvZilwIDEyODggMjU3IFYgMTYgdyhjb21tYW5kKXAg MTQ4OSAyNTcgViAxNSB3KGxpbmVzLCk3NQozMTMgeShhKW8ocnJhKW8oeSlwIDE3MyAzMTMg ViAxNSB3KG9mKXAgMjI1IDMxMyBWIDE2CncobWlucClvKHJvKXEoY3MsKWUoYSlvKHJyYSlv KHkpcCA1MzMgMzEzIFYgMTQgdyhvZilwCjU4NCAzMTMgViAxNiB3KG1heHApbyhybylxKGNz LClnKGEpbyhycmEpbyh5KXAgOTAxIDMxMwpWIDE1IHcob2YpcCA5NTMgMzEzIFYgMTYgdyhp bmZvLClnKHJvKXEob3QsKWgoY29tbSwpZChncm91cCwpaihhKW8ocnJhKQpvKHkpcCAxNTQ3 IDMxMyBWIDE0IHcob2YpcCAxNTk4IDMxMyBWIDE2IHcoZXJyY28pcShkZXNcKSkxMTcKMzkw IHkgRmcoSU4pMTU1IGIgRmkoY291bnQpNDgyIGIgRmcoTnVtKW8oYilxKGVyKTEyIGIob2Yp Zyhjb21tYW5kKWUKKGxpbmVzKWooXChpbilvKHRlZ2VyLClnKHNpemUpZyhvZilmKGVhYylv KGgpaChvZilmKHRoZSk5MDUKNDQ3IHkoZm9sbG8pbyh3aW5nKWYoYXJyYSlvKHlzXCkpMTE3 IDUyMiB5KElOKTE1NSBiIEZpKGEpbyhycmEpbyh5KXAKNDE2IDUyMiBWIDE1IHcob2YpcCA0 NjggNTIyIFYgMTYgdyhjb21tYW5kKXAgNjY5IDUyMgpWIDE1IHcobGluZXMpMTQyIGIgRmco ZXhlY3V0YWJsZSkxMiBiKGFuZClmKGFyZ3VtZW4pbyh0cylmKFwoYXJyYSlvKHkpZwoob2Yp ZyhzdHJpbmdzLCloKHNpZ25pXDAxNGNhbilvKHQpOTA1IDU3OCB5KG9ubHkpaShhdCloKHJv KXEob3RcKSkxMTcKNjUzIHkoSU4pMTU1IGIgRmkoYSlvKHJyYSlvKHkpcCA0MTYgNjUzIFYg MTUgdyhvZilwIDQ2OAo2NTMgViAxNiB3KG1pbnApbyhybylxKGNzKTI1NCBiIEZnKG1pbmlt KW4odW0pOSBiKG4pbyh1bSlvKGIpcShlcilrKG9mKQpmKHBybylxKGNlc3NlcylrKGZvcilk KGVhYylvKGgpaChjb21tYW5kKWMobGluZSk5MDUgNzEwCnkodG8payhzdGFydClnKFwoYXJy YSlvKHkpZyhvZilnKGluKW8odGVnZXIsKWYoc2lnbmlcMDE0Y2FuKW8odCloKG9ubHkpCmYo YXQpaChybylxKG90XCkpMTE3IDc4NSB5KElOKTE1NSBiIEZpKGEpbyhycmEpbyh5KXAKNDE2 IDc4NSBWIDE1IHcob2YpcCA0NjggNzg1IFYgMTYgdyhtYXhwKW8ocm8pcShjcykyNDUKYiBG ZyhtYXhpbSluKHVtKTggYihuKW8odW0pbyhiKXEoZXIpayhvZilmKHBybylxKGNlc3Nlcylr KGZvcilkKGVhYylvCihoKWcoY29tbWFuZClkKGxpbmUpOTA1IDg0MiB5KHRvKTE0IGIoc3Rh cnQpZyhcKGFycmEpbyh5KWcob2YpZyhpbilvCih0ZWdlciwpZihzaWduaVwwMTRjYW4pbyh0 KWgob25seSlmKGF0KWgocm8pcShvdFwpKTExNwo5MTcgeShJTikxNTUgYiBGaShhKW8ocnJh KW8oeSlwIDQxNiA5MTcgViAxNSB3KG9mKXAgNDY4CjkxNyBWIDE2IHcoaW5mbykzNTMgYiBG ZyhzdHJpbmdzKTE3IGIodGVsbGluZyllKHRoZSlpKHJ1bilvKHRpbWUpZQooc3lzdGVtKWko d2hlcmUpZyhhbmQpZihobylvKHcpZyh0byk5MDUgOTczIHkoc3RhcnQpMjEKYihwcm8pcShj ZXNzZXMpaShcKGFycmEpbyh5KWUob2YpZShzdHJpbmdzLClrKHNpZ25pXDAxNGNhbilvKHQp Yyhvbmx5KWgKKGF0KTkwNSAxMDMwIHkocm8pcShvdFwpKTExNyAxMTA1IHkoSU4pMTU1IGIg Rmkocm8pcShvdCk1MDgKYiBGZyhyYW5rKTEzIGIob2YpZihwcm8pcShjZXNzKWooaW4pZCh3 aGljKW8oaCloKHByZXZpb3VzKWcoYXJndW1lbilvCih0cylmKGFyZSlpKHYpbihhbGlkKTkw NSAxMTYxIHkoXChpbilvKHRlZ2VyXCkpMTE3IDEyMzYKeShJTikxNTUgYiBGaShjb21tKTQ3 MCBiIEZnKGNvbW0pbyh1bmljYXRvcik3IGIob2YpaShncm91cClnKG9mKWcoc3BhKW8KKHdu aW5nKWcocHJvKXEoY2Vzc2VzKWooXChoYW5kbGVcKSkxMTcgMTMxMSB5KE9VVCkxMDgKYiBG aShncm91cCk0NzkgYiBGZyhncm91cCkxNCBiKG9mKWYoc3BhKW8od25lZClpKHBybylxKGNl c3NlcyloCihcKGhhbmRsZVwpKTExNyAxMzg3IHkoT1VUKTEwOCBiIEZpKGEpbyhycmEpbyh5 KXAgNDE2CjEzODcgViAxNSB3KG9mKXAgNDY4IDEzODcgViAxNiB3KGVycmNvKXEoZGVzKTI2 OCBiIEZnKG9uZSkxNApiKGVycm9yKWgoY28pcShkZSlnKHApcShlcilnKHBybylxKGNlc3Mp ZyhcKGFycmEpbyh5KWYob2YpZihpbilvCih0ZWdlclwpKTc1IDE1MTEgeSBGZihpbnQpMjMg YihNUEkpcCAyNDUgMTUxMSAxNSAyIHYKMTcgdyhTcGF3bilwIDM4MiAxNTExIFYgMTcgdyht dWx0aXBsZSlwIDU5MSAxNTExIFYgMTYKdyhpbmRlcGVuZGVudClwIDg3MSAxNTExIFYgMTUg dyhtcGlcKGludCopZyhjb3VudCwpMzkzCjE1NjcgeShjaGFyKilnKGFycmF5KXAgNjU5IDE1 NjcgViAxNyB3KG9mKXAgNzI0IDE1NjcKViAxNyB3KGNvbW1hbmQpcCA5MDkgMTU2NyBWIDE2 IHcobGluZXNbXSwpZyhpbnQqKWcoYXJyYXkpcAoxMzc5IDE1NjcgViAxNyB3KG9mKXAgMTQ0 NCAxNTY3IFYgMTYgdyhtaW5wcm9jcywpMzkzCjE2MjQgeShpbnQqKWcoYXJyYXkpcCA2MzUg MTYyNCBWIDE3IHcob2YpcCA3MDAgMTYyNCBWCjE3IHcobWF4cHJvY3MsKWcoY2hhciopZyhh cnJheSlwIDEyMTkgMTYyNCBWIDE2IHcob2YpcAoxMjgzIDE2MjQgViAxNyB3KGluZm9bXSwp ZyhpbnQpZyhyb290LCkzOTMgMTY4MCB5KE1QSSlwCjQ2OCAxNjgwIFYgMTcgdyhDb21tKWco Y29tbSwpZyhNUEkpcCA4MTkgMTY4MCBWIDE3IHcoR3JvdXAqKWcoZ3JvdXAsKWcKKGludCop aChhcnJheSlwIDE0MTAgMTY4MCBWIDE2IHcob2YpcCAxNDc0IDE2ODAgViAxNwp3KGVycmNv ZGVzXCkpNzUgMTc2NyB5KE1QSSlwIDE1MCAxNzY3IFYgMTcgdyhTUEFXTilwCjI4NyAxNzY3 IFYgMTYgdyhNVUxUSVBMRSlwIDQ5NSAxNzY3IFYgMTYgdyhJTkRFUEVOREVOVClwCjc3NSAx NzY3IFYgMTYgdyhNUElcKENPVU5ULClmKEFSUkFZKXAgMTE3NCAxNzY3IFYgMTYKdyhPRilw IDEyMzggMTc2NyBWIDE3IHcoQ09NTUFORClwIDE0MjMgMTc2NyBWIDE2IHcoTElORVMsKTM5 MwoxODIzIHkoQVJSQVkpcCA1MTYgMTgyMyBWIDE3IHcoT0YpcCA1ODEgMTgyMyBWIDE3IHco TUlOUFJPQ1MsKWYoQVJSQVkpcAo5NTYgMTgyMyBWIDE3IHcoT0YpcCAxMDIxIDE4MjMgViAx NyB3KE1BWFBST0NTLClnKEFSUkFZKXAKMTM5NiAxODIzIFYgMTcgdyhPRilwIDE0NjEgMTgy MyBWIDE3IHcoSU5GTywpaChST09ULCkzOTMKMTg4MCB5KENPTU0sKWcoR1JPVVAsKWcoQVJS QVkpcCA4MjYgMTg4MCBWIDE3IHcoT0YpcAo4OTEgMTg4MCBWIDE3IHcoRVJSQ09ERVMsKWYo SUVSUlwpKTE3MCAxOTM2IHkoQ0hBUkFDVEVSKlwoKlwpKWgoQVJSQVkpcAo2MjggMTkzNiBW IDE2IHcoT0YpcCA2OTIgMTkzNiBWIDE3IHcoQ09NTUFORClwIDg3NyAxOTM2ClYgMTYgdyhM SU5FU1woKlwpLClnKEFSUkFZKXAgMTI1MiAxOTM2IFYgMTYgdyhPRilwIDEzMTYKMTkzNiBW IDE3IHcoSU5GTykxNzAgMTk5MyB5KElOVEVHRVIpZyhDT1VOVCwpZyhBUlJBWSlwCjY1MSAx OTkzIFYgMTcgdyhPRilwIDcxNiAxOTkzIFYgMTcgdyhNSU5QUk9DUywpZihBUlJBWSlwCjEw OTEgMTk5MyBWIDE3IHcoT0YpcCAxMTU2IDE5OTMgViAxNyB3KE1BWFBST0NTLClnKFJPT1Qs KWkoQ09NTSwpZgooR1JPVVAsKTE3MCAyMDQ5IHkoQVJSQVkpcCAyOTMgMjA0OSBWIDE3IHco T0YpcCAzNTggMjA0OQpWIDE3IHcoRVJSQ09ERVNcKCpcKSwpZihJRVJSKTE2NiAyMTM2IHkg RmwoVGhpcylnKGZ1bmN0aW9uKWcoaXMpZihpZGVuKQpvKHRpY2FsKWkgRmkoTVBJKXAgNzg1 IDIxMzYgMTQgMiB2IDE2IHcoU1ApbChBKS01IGIoV04pcAo5NTEgMjEzNiBWIDE4IHcoTVVM KWwoVElQTEUpMjAgYiBGbChleGNlcHQpaSh0aGF0KWUoaXQpaChkbylxKGVzKWgobm90KQpm KGVzdGFibGlzaCk3NSAyMTkyIHkoY29tbSlvKHVuaWNhdGlvbikxNSBiKHdpdGgpZih0aGUp aChjKW8oaGlsZHJlbiwpaAooc28pZCh0aGF0KWgoaXQpZyhyZXR1cm5zKWgoYW4pZihNUEkp Zyhncm91cClnKHJhdGhlcilmKHRoYW4paChhKWcKKGNvbW0pbyh1bmktKTc1IDIyNDggeShj YXRvci4payhUaGUpMTIgYihjKW8oaGlsZCloKHBybylxKGNlc3NlcylnKGFyZSkKZSBGaihy KW4oZSluKHF1aXIpbihlKW4oZClnIEZsKHRvKWgoY2FsbCloIEZpKE1QSSlwIDEwNTMKMjI0 OCBWIDE2IHcoSU5JVClwIEZsKC4pZChBbGwpayhjKW8oaGlsZHJlbilmKHNwYSlvKHduZWQp ZihpbiloKGEpZgooc2luZ2xlKWgoY2FsbCk3NSAyMzA1IHkodG8pZSBGaShNUEkpcCAyMTIg MjMwNSBWIDE2CncoU1ApbChBKS01IGIoV04pcCAzNzggMjMwNSBWIDE3IHcoSU5ERVBFTkRF TlQpcCA3MTEKMjMwNSBWIDE3IHcoTVBJKTExIGIgRmwod2lsbClpKGhhKW8odilvKGUpZSh0 aGUpaChzYW1lKWYKRmkoTVBJKXAgMTI3MSAyMzA1IFYgMTYgdyhDT01NKXAgMTQyOSAyMzA1 IFYgMTYgdyhXKW8oT1JMRClwCkZsKCwpZyh3aGljKW8oaCloKGlzKWcoc2VwLSk3NSAyMzYx IHkoYXJhdGUpZyhmcm9tKWcodGhhdClnKG9mKWcodGhlKWgKKHBhcmVuKW8odC4pMTggYihJ bikxMyBiKHRoZSlnKGMpbyhoaWxkcmVuLClpIEZpKE1QSSlwCjEwNzMgMjM2MSBWIDE2IHco UClsKEFSRU5UKWYgRmwod2lsbClnKHJldHVybillIEZpKE1QSSlwCjE1NzggMjM2MSBWIDE2 IHcoQ09NTSlwIDE3MzYgMjM2MSBWIDE3IHcoTlVMTClwIEZsKC4pMTY2CjI1MDEgeSBGaChE aXNjdXNzaW9uOik1MyBiIEZnKElmKTE5IGIodGhlKWcoc3BhKW8od24pZyh3KW8oYXMpZyhz b2Z0KWYKKFwoKXAgRmQobWlucClvKHJvKXEoY3MpaSBGYyg8KWcgRmQobWF4cClvKHJvKXEo Y3MpcApGZyhcKSlmKHRoZXJlKWgoaXMpZihubylmKGRpcmVjdClpKHcpbyhhKW8oeSllKGZv cilnKHRoZSk3NQoyNTU3IHkoYylvKGhpbGRyZW4pMTIgYih0bylmKFwwMTRuZCloKG91dCln KGhvKW8odylmKG1hbilvKHkpZihvZiloCihlYWMpbyhoKWgodClvKHlwKXEoZSloKHcpbyhh cyllKHNwYSlvKHduZWQuKTE4IGIoVGhlKTEyCmIocGFyZW4pbyh0KWcoY2FuKWcoXDAxNGd1 cmUpZyhvdXQpZyh0aGlzKWcoaW5mb3JtYXRpb24pNzUKMjYxMyB5KFwoYilvKHkpaShleGFt aW5pbmcpZiBGZChhKW8ocnJhKW8oeSlwIDQzNyAyNjEzCjEzIDIgdiAxNSB3KG9mKXAgNDg2 IDI2MTMgViAxNCB3KGVycm8pbyhyY28pcShkZXMpcCBGZyhcKSwpaihidXQpZgooc2luY2Up Zyh0aGVyZSloKGlzKWUobm8paChpbilvKHRlcmNvbW0pbyh1bmljYXRvcilkKGl0KWkoaGFz KWgobm8pZgooZGlyZWN0KWgodylvKGEpbyh5KWYodG8pNzUgMjY3MCB5KHRlbGwpZih0aGUp aShjKW8oaGlsZHJlbi4paihUaGlzKWMKKHByb2JsZW0pZihleGlzdHMpaChpbilnKE1QSS0x KWYoYXMpaCh3KW8oZWxsLiktMzIgNDYKeSBGZSgxKS0zMiAxMDMgeSgyKS0zMiAxNTkgeSgz KS0zMiAyMTUgeSg0KS0zMiAyNzIgeSg1KS0zMgozMjggeSg2KS0zMiAzODUgeSg3KS0zMiA0 NDEgeSg4KS0zMiA0OTggeSg5KS00MCA1NTQgeSgxMCktNDAKNjExIHkoMTEpLTQwIDY2NyB5 KDEyKS00MCA3MjQgeSgxMyktNDAgNzgwIHkoMTQpLTQwIDgzNgp5KDE1KS00MCA4OTMgeSgx NiktNDAgOTQ5IHkoMTcpLTQwIDEwMDYgeSgxOCktNDAgMTA2Mgp5KDE5KS00MCAxMTE5IHko MjApLTQwIDExNzUgeSgyMSktNDAgMTIzMiB5KDIyKS00MCAxMjg4CnkoMjMpLTQwIDEzNDUg eSgyNCktNDAgMTQwMSB5KDI1KS00MCAxNDU3IHkoMjYpLTQwIDE1MTQKeSgyNyktNDAgMTU3 MCB5KDI4KS00MCAxNjI3IHkoMjkpLTQwIDE2ODMgeSgzMCktNDAgMTc0MAp5KDMxKS00MCAx Nzk2IHkoMzIpLTQwIDE4NTMgeSgzMyktNDAgMTkwOSB5KDM0KS00MCAxOTY2CnkoMzUpLTQw IDIwMjIgeSgzNiktNDAgMjA3OCB5KDM3KS00MCAyMTM1IHkoMzgpLTQwIDIxOTEKeSgzOSkt NDAgMjI0OCB5KDQwKS00MCAyMzA0IHkoNDEpLTQwIDIzNjEgeSg0MiktNDAgMjQxNwp5KDQz KS00MCAyNDc0IHkoNDQpLTQwIDI1MzAgeSg0NSktNDAgMjU4NyB5KDQ2KS00MCAyNjQzCnko NDcpLTQwIDI2OTkgeSg0OClwIGVvcAolJVBhZ2U6IDQgNQo0IDQgYm9wIDc1IC0xMDAgYSBG bCg0KTc1IDQ5IHkgRmkoTVBJKXAgMTYwIDQ5IDE0IDIgdgoxNiB3KFNQKWwoQSktNSBiKFdO KXAgMzI2IDQ5IFYgMTcgdyhNVUwpbChUSVBMRSlwIDU2MAo0OSBWIDE2IHcoSU5ERVBFTkRF TlQpcCA4OTIgNDkgViAxNiB3KE5PTk1QSVwoY291bnQsKTE2CmIoYSlvKHJyYSlvKHkpcCAx MzM0IDQ5IFYgMTUgdyhvZilwIDEzODYgNDkgViAxNiB3KGNvbW1hbmQpcAoxNTg3IDQ5IFYg MTUgdyhsaW5lcywpNzUgMTA2IHkoYSlvKHJyYSlvKHkpcCAxNzMgMTA2ClYgMTUgdyhvZilw IDIyNSAxMDYgViAxNiB3KG1pbnApbyhybylxKGNzLCllKGEpbyhycmEpbyh5KXAKNTMzIDEw NiBWIDE0IHcob2YpcCA1ODQgMTA2IFYgMTYgdyhtYXhwKW8ocm8pcShjcywpZyhhKW8ocnJh KW8oeSlwCjkwMSAxMDYgViAxNSB3KG9mKXAgOTUzIDEwNiBWIDE2IHcoaW5mbywpZyhybylx KG90LCloKGNvbW0sKWQoZ3JvdXAsKWoKKGEpbyhycmEpbyh5KXAgMTU0NyAxMDYgViAxNCB3 KG9mKXAgMTU5OCAxMDYgViAxNiB3KGVycmNvKXEoZGVzXCkpMTE3CjE4MyB5IEZnKElOKTE1 NSBiIEZpKGNvdW50KTQ4MiBiIEZnKE51bSlvKGIpcShlcikxMiBiKG9mKWcoY29tbWFuZCll CihsaW5lcylqKFwoaW4pbyh0ZWdlciwpZyhzaXplKWcob2YpZihlYWMpbyhoKWgob2YpZih0 aGUpOTA1CjIzOSB5KGZvbGxvKW8od2luZylmKGFycmEpbyh5c1wpKTExNyAzMTUgeShJTikx NTUgYiBGaShhKW8ocnJhKW8oeSlwCjQxNiAzMTUgViAxNSB3KG9mKXAgNDY4IDMxNSBWIDE2 IHcoY29tbWFuZClwIDY2OSAzMTUKViAxNSB3KGxpbmVzKTE0MiBiIEZnKGV4ZWN1dGFibGUp MTIgYihhbmQpZihhcmd1bWVuKW8odHMpZihcKGFycmEpbyh5KWcKKG9mKWcoc3RyaW5ncywp aChzaWduaVwwMTRjYW4pbyh0KTkwNSAzNzEgeShvbmx5KWkoYXQpaChybylxKG90XCkpMTE3 CjQ0NiB5KElOKTE1NSBiIEZpKGEpbyhycmEpbyh5KXAgNDE2IDQ0NiBWIDE1IHcob2YpcCA0 NjgKNDQ2IFYgMTYgdyhtaW5wKW8ocm8pcShjcykyNTQgYiBGZyhtaW5pbSluKHVtKTkgYihu KW8odW0pbyhiKXEoZXIpayhvZikKZihwcm8pcShjZXNzZXMpayhmb3IpZChlYWMpbyhoKWgo Y29tbWFuZCljKGxpbmUpOTA1IDUwMwp5KHRvKWsoc3RhcnQpZyhcKGFycmEpbyh5KWcob2Yp ZyhpbilvKHRlZ2VyLClmKHNpZ25pXDAxNGNhbilvKHQpaChvbmx5KQpmKGF0KWgocm8pcShv dFwpKTExNyA1NzggeShJTikxNTUgYiBGaShhKW8ocnJhKW8oeSlwCjQxNiA1NzggViAxNSB3 KG9mKXAgNDY4IDU3OCBWIDE2IHcobWF4cClvKHJvKXEoY3MpMjQ1CmIgRmcobWF4aW0pbih1 bSk4IGIobilvKHVtKW8oYilxKGVyKWsob2YpZihwcm8pcShjZXNzZXMpayhmb3IpZChlYWMp bwooaClnKGNvbW1hbmQpZChsaW5lKTkwNSA2MzQgeSh0bykxNCBiKHN0YXJ0KWcoXChhcnJh KW8oeSlnKG9mKWcoaW4pbwoodGVnZXIsKWYoc2lnbmlcMDE0Y2FuKW8odCloKG9ubHkpZihh dCloKHJvKXEob3RcKSkxMTcKNzA5IHkoSU4pMTU1IGIgRmkoYSlvKHJyYSlvKHkpcCA0MTYg NzA5IFYgMTUgdyhvZilwIDQ2OAo3MDkgViAxNiB3KGluZm8pMzUzIGIgRmcoc3RyaW5ncykx NyBiKHRlbGxpbmcpZSh0aGUpaShydW4pbyh0aW1lKWUKKHN5c3RlbSlpKHdoZXJlKWcoYW5k KWYoaG8pbyh3KWcodG8pOTA1IDc2NiB5KHN0YXJ0KTIxCmIocHJvKXEoY2Vzc2VzKWkoXChh cnJhKW8oeSllKG9mKWUoc3RyaW5ncywpayhzaWduaVwwMTRjYW4pbyh0KWMob25seSloCihh dCk5MDUgODIyIHkocm8pcShvdFwpKTExNyA4OTcgeShJTikxNTUgYiBGaShybylxKG90KTUw OApiIEZnKHJhbmspMTMgYihvZilmKHBybylxKGNlc3MpaihpbilkKHdoaWMpbyhoKWgocHJl dmlvdXMpZyhhcmd1bWVuKW8KKHRzKWYoYXJlKWkodiluKGFsaWQpOTA1IDk1NCB5KFwoaW4p byh0ZWdlclwpKTExNyAxMDI5CnkoSU4pMTU1IGIgRmkoY29tbSk0NzAgYiBGZyhjb21tKW8o dW5pY2F0b3IpNyBiKG9mKWkoZ3JvdXApZyhvZilnKHNwYSlvCih3bmluZylnKHBybylxKGNl c3NlcylqKFwoaGFuZGxlXCkpMTE3IDExMDQgeShPVVQpMTA4CmIgRmkoZ3JvdXApNDc5IGIg RmcoZ3JvdXApMTQgYihvZilmKHNwYSlvKHduZWQpaShwcm8pcShjZXNzZXMpaAooXChoYW5k bGVcKSkxMTcgMTE3OSB5KE9VVCkxMDggYiBGaShhKW8ocnJhKW8oeSlwIDQxNgoxMTc5IFYg MTUgdyhvZilwIDQ2OCAxMTc5IFYgMTYgdyhlcnJjbylxKGRlcykyNjggYiBGZyhvbmUpMTQK YihlcnJvciloKGNvKXEoZGUpZyhwKXEoZXIpZyhwcm8pcShjZXNzKWcoXChhcnJhKW8oeSlm KG9mKWYoaW4pbwoodGVnZXJcKSk3NSAxMzA0IHkgRmYoaW50KTIzIGIoTVBJKXAgMjQ1IDEz MDQgMTUgMiB2CjE3IHcoU3Bhd24pcCAzODIgMTMwNCBWIDE3IHcobXVsdGlwbGUpcCA1OTEg MTMwNCBWIDE2CncoaW5kZXBlbmRlbnQpcCA4NzEgMTMwNCBWIDE1IHcobm9ubXBpXChpbnQq KWcoY291bnQsKTM5MwoxMzYwIHkoY2hhciopZyhhcnJheSlwIDY1OSAxMzYwIFYgMTcgdyhv ZilwIDcyNCAxMzYwClYgMTcgdyhjb21tYW5kKXAgOTA5IDEzNjAgViAxNiB3KGxpbmVzW10s KWcoaW50KilnKGFycmF5KXAKMTM3OSAxMzYwIFYgMTcgdyhvZilwIDE0NDQgMTM2MCBWIDE2 IHcobWlucHJvY3MsKTM5MwoxNDE3IHkoaW50KilnKGFycmF5KXAgNjM1IDE0MTcgViAxNyB3 KG9mKXAgNzAwIDE0MTcgVgoxNyB3KG1heHByb2NzLClnKGNoYXIqKWcoYXJyYXkpcCAxMjE5 IDE0MTcgViAxNiB3KG9mKXAKMTI4MyAxNDE3IFYgMTcgdyhpbmZvW10sKWcoaW50KWcocm9v dCwpMzkzIDE0NzMgeShNUEkpcAo0NjggMTQ3MyBWIDE3IHcoQ29tbSlnKGNvbW0sKWcoTVBJ KXAgODE5IDE0NzMgViAxNyB3KEdyb3VwKilnKGdyb3VwLClnCihpbnQqKWgoYXJyYXkpcCAx NDEwIDE0NzMgViAxNiB3KG9mKXAgMTQ3NCAxNDczIFYgMTcKdyhlcnJjb2Rlc1wpKTc1IDE1 NTkgeShNUEkpcCAxNTAgMTU1OSBWIDE3IHcoU1BBV04pcAoyODcgMTU1OSBWIDE2IHcoTVVM VElQTEUpcCA0OTUgMTU1OSBWIDE2IHcoSU5ERVBFTkRFTlQpcAo3NzUgMTU1OSBWIDE2IHco Tk9OTVBJXChDT1VOVCwpZShBUlJBWSlwIDEyNDUgMTU1OSBWCjE3IHcoT0YpcCAxMzEwIDE1 NTkgViAxNyB3KENPTU1BTkQpcCAxNDk1IDE1NTkgViAxNiB3KExJTkVTLCkzOTMKMTYxNiB5 KEFSUkFZKXAgNTE2IDE2MTYgViAxNyB3KE9GKXAgNTgxIDE2MTYgViAxNyB3KE1JTlBST0NT LClnKEFSUkFZKXAKOTU2IDE2MTYgViAxNyB3KE9GKXAgMTAyMSAxNjE2IFYgMTcgdyhNQVhQ Uk9DUywpZyhBUlJBWSlwCjEzOTYgMTYxNiBWIDE3IHcoT0YpcCAxNDYxIDE2MTYgViAxNyB3 KElORk8sKWgoUk9PVCwpMzkzCjE2NzIgeShDT01NLClnKEdST1VQLClnKEFSUkFZKXAgODI2 IDE2NzIgViAxNyB3KE9GKXAKODkxIDE2NzIgViAxNyB3KEVSUkNPREVTLClmKElFUlJcKSkx NzAgMTcyOSB5KENIQVJBQ1RFUipcKCpcKSloKEFSUkFZKXAKNjI4IDE3MjkgViAxNiB3KE9G KXAgNjkyIDE3MjkgViAxNyB3KENPTU1BTkQpcCA4NzcgMTcyOQpWIDE2IHcoTElORVNcKCpc KSwpZyhBUlJBWSlwIDEyNTIgMTcyOSBWIDE2IHcoT0YpcCAxMzE2CjE3MjkgViAxNyB3KElO Rk8pMTcwIDE3ODUgeShJTlRFR0VSKWcoQ09VTlQsKWcoQVJSQVkpcAo2NTEgMTc4NSBWIDE3 IHcoT0YpcCA3MTYgMTc4NSBWIDE3IHcoTUlOUFJPQ1MsKWYoQVJSQVkpcAoxMDkxIDE3ODUg ViAxNyB3KE9GKXAgMTE1NiAxNzg1IFYgMTcgdyhNQVhQUk9DUywpZyhST09ULClpKENPTU0s KWYKKEdST1VQLCkxNzAgMTg0MiB5KEFSUkFZKXAgMjkzIDE4NDIgViAxNyB3KE9GKXAgMzU4 IDE4NDIKViAxNyB3KEVSUkNPREVTXCgqXCksKWYoSUVSUikxNjYgMTkyOCB5IEZsKFRoZSkx NSBiKHJvdXRpbmUpaApGaShNUEkpcCA1MDEgMTkyOCAxNCAyIHYgMTYgdyhTUClsKEEpLTUg YihXTilwIDY2NyAxOTI4ClYgMTcgdyhNVUwpbChUSVBMRSlwIDkwMSAxOTI4IFYgMTYgdyhJ TkRFUEVOREVOVClwIDEyMzMKMTkyOCBWIDE2IHcoTk9OTVBJKTE2IGIgRmwoaXMpZihzaW1p bGFyKWkodG8pNzUgMTk4NQp5IEZpKE1QSSlwIDE2MCAxOTg1IFYgMTYgdyhTUClsKEEpLTUg YihXTilwIDMyNiAxOTg1ClYgMTcgdyhNVUwpbChUSVBMRSlwIDU2MCAxOTg1IFYgMTYgdyhJ TkRFUEVOREVOVClwIDg5MgoxOTg1IFYgMTYgdyhNUEkpMTUgYiBGbChleGNlcHQpaSh0aGF0 KWUodGhlKWgoYylvKGhpbGQpaChwcm8pcShjZXNzZXMpZgooYXJlKWcoYXNzdW1lZCk3NSAy MDQxIHkodG8pZShiKXEoZSloKG5vbi1NUEkpZyhwcm8pcShjZXNzZXMuKTIwCmIoTVBJKTE1 IGIoaXMpZyhub3QpZihyZXF1aXJlZClpKHRvKWUoZG8paChhbilvKHkpZihzcClxKGVjaWFs KWkoc2V0dXApCmYodG8pZihlbnN1cmUpaSh0aGF0KWQoYSlpKGNhbGwpNzUgMjA5OCB5KHRv KWogRmkoTVBJKXAKMjE5IDIwOTggViAxNSB3KElOSVQpZyBGbChpbiloKHRoZSlmKGMpbyho aWxkcmVuKWood2lsbClmKGIpcShlKWUoYWJsZSkKaSh0bylkKGVzdGFibGlzaClqKGNvbW0p byh1bmljYXRpb24pZihhbW9uZyllKHRoZSloKGMpbyhoaWxkcmVuLik3NQoyMTU0IHkoVGhl KWQoZVwwMTNlY3QpZyhvZilnKGNhbGxpbmcpaSBGaShNUEkpcCA1NjkgMjE1NApWIDE2IHco SU5JVClkIEZsKGluKWkodGhlKWcoYylvKGhpbGRyZW4paChpcylmKHVuZGVcMDE0bmVkLikx NjYKMjI5MyB5IEZoKERpc2N1c3Npb246KTMzIGIgRmcoVGhpcykxMyBiKHJvdXRpbmUpaChp cylmKGhlcmUpaShmb3IpZQooY29tcGxldGVuZXNzLikxOCBiKEl0KWMoZG8pcShlcylnKG5v dClmKHBybylvKHZpZGUpZyhhbilvKHkpZyhtaXNzaW5nKQplKGZ1bmN0aW9uLSk3NSAyMzUw IHkoYWxpdClvKHkpayhcKGFzKWkoZG8pZih0aGUpaChvdGhlcilwCjQ5MCAyMzUwIDEzIDIg diAzMiB3IEZkKE1VTCltKFRJUExFKWYgRmcocm91dGluZXNcKSloKGIpcShlY2F1c2UpaQoo dGhlcmUpZShpcylnKG5vKWYobmVlZCloKHRvKWcocHV0KWYoYWxsKWcob2YpZyh0aGUpaChj KW8oaGlsZHJlbilmKGluKQo3NSAyNDA2IHkodGhlKWgoc2FtZSllIEZkKE1QSSlwIDMzMiAy NDA2IFYgMTUgdyhDT01NKXAKNDc3IDI0MDYgViAxNCB3KFcpbyhPUkxEKXAgRmcoLiloKFNp bWlsYXJseSlsKCwpZSh0aGUpaihwKXEob3NzaWJpbGl0KW8KKHkpZShvZiloKGluY3JlYXNl ZCloKGNvbW0pbyh1bmljYXRpb24pYyhwKXEoZXJmb3JtYW5jZSk3NQoyNDYzIHkoYilxKGV0 KW8odylvKGVlbilrKHBybylxKGNlc3NlcyloKHNwYSlvKHduZWQpZShhdClnKHRoZSlnKHNh bWUpZQoodGltZSloKGlzKWcodW5pbXApcShvcnRhbilvKHQpZihiKXEoZWNhdXNlKWoodGhl KWYoc3BhKW8od25lZClnKHBybylxCihjZXNzZXMpaShkb24ndCk3NSAyNTE5IHkoY29tbSlv KHVuaWNhdGUuKTI4IGIoVGhlcmUpMTkKYihtaWdoKW8odClkKGIpcShlKWkoYSlnKHNsaWdo KW8odClmKGFkdiluKGFuKW8odGFnZSlnKGluKWgodGhlKWcKKGVcMDE2Y2llbmN5KWgob2Yp ZShzcGEpbyh3bmluZywpaChidXQpZyh0aGF0KWcoY2FuKWcoYilxKGUpNzUKMjU3NSB5KG1p dGlnYXRlZCkxMiBiKHRocm91Z2gpaSh1c2luZylmKHRoZSlpKG5vbi1ibG8pcShjKW8oa2lu ZylkCihyb3V0aW5lcy4pMTk2NyA0NiB5IEZlKDEpMTk2NyAxMDMgeSgyKTE5NjcgMTU5IHko MykxOTY3CjIxNSB5KDQpMTk2NyAyNzIgeSg1KTE5NjcgMzI4IHkoNikxOTY3IDM4NSB5KDcp MTk2NyA0NDEKeSg4KTE5NjcgNDk4IHkoOSkxOTU5IDU1NCB5KDEwKTE5NTkgNjExIHkoMTEp MTk1OSA2NjcKeSgxMikxOTU5IDcyNCB5KDEzKTE5NTkgNzgwIHkoMTQpMTk1OSA4MzYgeSgx NSkxOTU5IDg5Mwp5KDE2KTE5NTkgOTQ5IHkoMTcpMTk1OSAxMDA2IHkoMTgpMTk1OSAxMDYy IHkoMTkpMTk1OQoxMTE5IHkoMjApMTk1OSAxMTc1IHkoMjEpMTk1OSAxMjMyIHkoMjIpMTk1 OSAxMjg4IHkoMjMpMTk1OQoxMzQ1IHkoMjQpMTk1OSAxNDAxIHkoMjUpMTk1OSAxNDU3IHko MjYpMTk1OSAxNTE0IHkoMjcpMTk1OQoxNTcwIHkoMjgpMTk1OSAxNjI3IHkoMjkpMTk1OSAx NjgzIHkoMzApMTk1OSAxNzQwIHkoMzEpMTk1OQoxNzk2IHkoMzIpMTk1OSAxODUzIHkoMzMp MTk1OSAxOTA5IHkoMzQpMTk1OSAxOTY2IHkoMzUpMTk1OQoyMDIyIHkoMzYpMTk1OSAyMDc4 IHkoMzcpMTk1OSAyMTM1IHkoMzgpMTk1OSAyMTkxIHkoMzkpMTk1OQoyMjQ4IHkoNDApMTk1 OSAyMzA0IHkoNDEpMTk1OSAyMzYxIHkoNDIpMTk1OSAyNDE3IHkoNDMpMTk1OQoyNDc0IHko NDQpMTk1OSAyNTMwIHkoNDUpMTk1OSAyNTg3IHkoNDYpMTk1OSAyNjQzIHkoNDcpMTk1OQoy Njk5IHkoNDgpcCBlb3AKJSVQYWdlOiA1IDYKNSA1IGJvcCAxODUyIC0xMDAgYSBGbCg1KTc1 IDQ5IHkgRmsoMi4zLjIpNDkgYihOb25ibG8pcShja2luZykxOApiKHJlcXVlc3RzKTc1IDEz NSB5IEZsKFNwYSlvKHduaW5nKWUobmV3KWcocHJvKXEoY2Vzc2VzKWcobWEpbyh5KWYoYilx CihlKWgoYW4pZyhleHApcShlbnNpdilvKGUpaChvcClxKGVyYXRpb24uKWsoSW4pMTYgYihv cmRlcilmKHRvKWcoYWxsbylvCih3KWgocHJvKXEoY2Vzc2VzKWcodG8pZihkbyk3NSAxOTIg eSh1c2VmdWwpaSh3KW8ob3JrKWUod2hpbGUpaShwcm8pcQooY2Vzc2VzKWcoYXJlKWUoYilx KGVpbmcpaihzcGEpbyh3bmVkLClkKE1QSSloKHBybylvKHZpZGVzKWgobm9uLWJsbylxCihj KW8oa2luZylnKHYpbyhlcnNpb25zKWYob2YpZyh0aGUpNzUgMjQ4IHkoZWFjKW8oaClmKG9m KWcodGhlKWcoc3BhKW8KKHduaW5nKWgocm91dGluZXMpZihkZVwwMTRuZWQpaShhYilxKG8p byh2KW8oZS4pNzUgMzk5CnkgRmkoTVBJKXAgMTYwIDM5OSAxNCAyIHYgMTYgdyhJU1ApbChB KS01IGIoV05cKGNvbW1hbmQpcAo1NDIgMzk5IFYgMTUgdyhsaW5lLCkxNiBiKG1pbnApbyhy bylxKGNzLCllKG1heHApbyhybylxKGNzLClnKGluZm8sKWgKKHJvKXEob3QsKWYoY29tbSwp ZihpbnRlcmNvbW0sKTc1IDQ1NSB5KGEpbyhycmEpbyh5KXAKMTczIDQ1NSBWIDE1IHcob2Yp cCAyMjUgNDU1IFYgMTYgdyhlcnJjbylxKGRlcywpaShyZXF1ZXN0XCkpMTE3CjUzMyB5IEZn KElOKTE1NSBiIEZpKGNvbW1hbmQpcCA1MDYgNTMzIFYgMTYgdyhsaW5lKTMyMQpiIEZnKGV4 ZWN1dGFibGUpMTggYihhbmQpZihhcmd1bWVuKW8odHMsKWcoaW4pZyhhKWYoc2luZ2xlKWgo c3RyaW5nKWcKKFwoc3RyaW5nLCk5MDUgNTg5IHkoc2lnbmlcMDE0Y2FuKW8odClkKG9ubHkp ZihhdClnKHJvKXEob3RcKSkxMTcKNjY0IHkoSU4pMTU1IGIgRmkobWlucClvKHJvKXEoY3Mp NDE3IGIgRmcobWluaW0pbih1bSkxNApiKG4pbyh1bSlvKGIpcShlcilqKG9mKWcocHJvKXEo Y2Vzc2VzKWsodG8pYyhzdGFydClpKFwoaW4pbyh0ZWdlciwpZwooc2lnLSk5MDUgNzIxIHko bmlcMDE0Y2FuKW8odCkxNCBiKG9ubHkpZihhdCloKHJvKXEob3RcKSkxMTcKNzk2IHkoSU4p MTU1IGIgRmkobWF4cClvKHJvKXEoY3MpNDA4IGIgRmcobWF4aW0pbih1bSkxMwpiKG4pbyh1 bSlvKGIpcShlcilqKG9mKWcocHJvKXEoY2Vzc2VzKWoodG8pZShzdGFydClnKFwoaW4pbyh0 ZWdlciwpaAooc2lnLSk5MDUgODUyIHkobmlcMDE0Y2FuKW8odCljKG9ubHkpZihhdCloKHJv KXEob3RcKSkxMTcKOTI3IHkoSU4pMTU1IGIgRmkoaW5mbyk1MTYgYiBGZyhhKTE0IGIoc3Ry aW5nKWcodGVsbGluZylmKHRoZSlpKHJ1bilvCih0aW1lKWUoc3lzdGVtKWcod2hlcmUpaihh bmQpZChobylvKHcpaCh0byk5MDUgOTg0IHkoc3RhcnQpaCh0aGUpZihwcm8pCnEoY2Vzc2Vz KWooXChzdHJpbmcsKWMoc2lnbmlcMDE0Y2FuKW8odCloKG9ubHkpZihhdCloKHJvKXEob3Rc KSkxMTcKMTA1OSB5KElOKTE1NSBiIEZpKHJvKXEob3QpNTA4IGIgRmcocmFuaykxMyBiKG9m KWYocHJvKXEoY2VzcylqKGluKWQKKHdoaWMpbyhoKWgocHJldmlvdXMpZyhhcmd1bWVuKW8o dHMpZihhcmUpaSh2KW4oYWxpZCk5MDUKMTExNSB5KFwoaW4pbyh0ZWdlclwpKTExNyAxMTkx IHkoSU4pMTU1IGIgRmkoY29tbSk0NzAKYiBGZyhjb21tKW8odW5pY2F0b3IpNyBiKG9mKWko Z3JvdXApZyhvZilnKHNwYSlvKHduaW5nKWcocHJvKXEoY2Vzc2VzKWoKKFwoaGFuZGxlXCkp MTE3IDEyNjYgeShPVVQpMTA4IGIgRmkoaW50ZXJjb21tKTM4NCBiIEZnKGluKW8odGVyY29t bSlvCih1bmljYXRvcik3IGIoYilxKGV0KW8odylvKGVlbilrKG9yaWdpbmFsKWMoZ3JvdXAp aShhbmQpZyh0aGUpaChuZXdseSkKOTA1IDEzMjIgeShzcGEpbyh3bmVkKTE1IGIoZ3JvdXAp ZShcKGhhbmRsZVwpKTExNyAxMzk3CnkoT1VUKTEwOCBiIEZpKGEpbyhycmEpbyh5KXAgNDE2 IDEzOTcgViAxNSB3KG9mKXAgNDY4CjEzOTcgViAxNiB3KGVycmNvKXEoZGVzKTI2OCBiIEZn KG9uZSkxNCBiKGNvKXEoZGUpaChwKXEoZXIpZyhwcm8pcQooY2VzcyloKFwoYXJyYSlvKHkp ZChvZiloKGluKW8odGVnZXJcKSkxMTcgMTQ3MiB5KE9VVCkxMDgKYiBGaShyZXF1ZXN0KTQ1 MiBiIEZnKHJlcXVlc3QpMTYgYihvYilyKGplY3QpZShcKGhhbmRsZVwpKTc1CjE1OTcgeSBG ZihpbnQpMjMgYihNUEkpcCAyNDUgMTU5NyAxNSAyIHYgMTcgdyhJc3Bhd25cKGNoYXIqKWYo Y29tbWFuZClwCjc0MCAxNTk3IFYgMTcgdyhsaW5lLCloKGludClnKG1pbnByb2NzLClnKGlu dClnKG1heHByb2NzLClnKGNoYXIqKWcKKGluZm8sKTM5MyAxNjUzIHkoaW50KWgocm9vdCwp ZihNUEkpcCA3MDcgMTY1MyBWIDE3IHcoQ29tbSlnKGNvbW0sKWcKKE1QSSlwIDEwNTggMTY1 MyBWIDE3IHcoQ29tbSopZyhpbnRlcmNvbW0sKTM5MyAxNzEwIHkoaW50KilnKGFycmF5KXAK NjM1IDE3MTAgViAxNyB3KG9mKXAgNzAwIDE3MTAgViAxNyB3KGVycmNvZGVzLClnKE1QSSlw CjEwMjggMTcxMCBWIDE2IHcoUmVxdWVzdCopZyhyZXF1ZXN0XCkpNzUgMTc5NiB5KE1QSSlw CjE1MCAxNzk2IFYgMTcgdyhJU1BBV05cKENPTU1BTkQpcCA1MDMgMTc5NiBWIDE1IHcoTElO RSwpZyhNSU5QUk9DUywpZwooTUFYUFJPQ1MsKWYoSU5GTywpaShST09ULClmKENPTU0sKWco SU5URVJDT01NLCkzOTMgMTg1Mwp5KEFSUkFZKXAgNTE2IDE4NTMgViAxNyB3KE9GKXAgNTgx IDE4NTMgViAxNyB3KEVSUkNPREVTLClmKFJFUVVFU1QsKWgKKElFUlJcKSkxNzAgMTkwOSB5 KENIQVJBQ1RFUipcKCpcKSlnKENPTU1BTkQpcCA2NzYgMTkwOQpWIDE2IHcoTElORSwpZyhJ TkZPKTE3MCAxOTY2IHkoSU5URUdFUilnKE1JTlBST0NTLClnKE1BWFBST0NTLClnKFJPT1Qs KQpnKENPTU0sKWcoSU5URVJDT01NLCkxNzAgMjAyMiB5KEFSUkFZKXAgMjkzIDIwMjIgViAx Nwp3KE9GKXAgMzU4IDIwMjIgViAxNyB3KEVSUkNPREVTXCgqXCksKWYoUkVRVUVTVCwpaChJ RVJSKS0zMgo0NiB5IEZlKDEpLTMyIDEwMyB5KDIpLTMyIDE1OSB5KDMpLTMyIDIxNSB5KDQp LTMyIDI3Mgp5KDUpLTMyIDMyOCB5KDYpLTMyIDM4NSB5KDcpLTMyIDQ0MSB5KDgpLTMyIDQ5 OCB5KDkpLTQwCjU1NCB5KDEwKS00MCA2MTEgeSgxMSktNDAgNjY3IHkoMTIpLTQwIDcyNCB5 KDEzKS00MCA3ODAKeSgxNCktNDAgODM2IHkoMTUpLTQwIDg5MyB5KDE2KS00MCA5NDkgeSgx NyktNDAgMTAwNgp5KDE4KS00MCAxMDYyIHkoMTkpLTQwIDExMTkgeSgyMCktNDAgMTE3NSB5 KDIxKS00MCAxMjMyCnkoMjIpLTQwIDEyODggeSgyMyktNDAgMTM0NSB5KDI0KS00MCAxNDAx IHkoMjUpLTQwIDE0NTcKeSgyNiktNDAgMTUxNCB5KDI3KS00MCAxNTcwIHkoMjgpLTQwIDE2 MjcgeSgyOSktNDAgMTY4Mwp5KDMwKS00MCAxNzQwIHkoMzEpLTQwIDE3OTYgeSgzMiktNDAg MTg1MyB5KDMzKS00MCAxOTA5CnkoMzQpLTQwIDE5NjYgeSgzNSktNDAgMjAyMiB5KDM2KS00 MCAyMDc4IHkoMzcpLTQwIDIxMzUKeSgzOCktNDAgMjE5MSB5KDM5KS00MCAyMjQ4IHkoNDAp LTQwIDIzMDQgeSg0MSktNDAgMjM2MQp5KDQyKS00MCAyNDE3IHkoNDMpLTQwIDI0NzQgeSg0 NCktNDAgMjUzMCB5KDQ1KS00MCAyNTg3CnkoNDYpLTQwIDI2NDMgeSg0NyktNDAgMjY5OSB5 KDQ4KXAgZW9wCiUlUGFnZTogNiA3CjYgNiBib3AgNzUgLTEwMCBhIEZsKDYpNzUgNDkgeSBG aShNUEkpcCAxNjAgNDkgMTQgMiB2CjE2IHcoSVNQKWwoQSktNSBiKFdOKXAgMzM5IDQ5IFYg MTcgdyhNVUwpbChUSVBMRVwoY291bnQsKTE1CmIoYSlvKHJyYSlvKHkpcCA4MTkgNDkgViAx NCB3KG9mKXAgODcwIDQ5IFYgMTYgdyhjb21tYW5kKXAKMTA3MSA0OSBWIDE2IHcobGluZXMs KWcoYSlvKHJyYSlvKHkpcCAxMjkyIDQ5IFYgMTUgdyhvZilwCjEzNDQgNDkgViAxNiB3KG1p bnApbyhybylxKGNzLCk3NSAxMDYgeShhKW8ocnJhKW8oeSlwCjE3MyAxMDYgViAxNSB3KG9m KXAgMjI1IDEwNiBWIDE2IHcobWF4cClvKHJvKXEoY3MsKWYoYSlvKHJyYSlvKHkpcAo1NDIg MTA2IFYgMTQgdyhvZilwIDU5MyAxMDYgViAxNiB3KGluZm8sKWgocm8pcShvdCwpZihjb21t LClmCihpbnRlcmNvbW0sKWgoYSlvKHJyYSlvKHkpcCAxMjgyIDEwNiBWIDE0IHcob2YpcCAx MzMzCjEwNiBWIDE2IHcoZXJyY28pcShkZXMsKWkocmVxdWVzdFwpKTExNyAxODMgeSBGZyhJ TikxNTUKYiBGaShjb3VudCk0ODIgYiBGZyhOdW0pbyhiKXEoZXIpMTIgYihvZilnKGNvbW1h bmQpZShsaW5lcylqKFwoaW4pbwoodGVnZXIsKWcoc2l6ZSlnKG9mKWYoZWFjKW8oaCloKG9m KWYodGhlKTkwNSAyMzkgeShmb2xsbylvKHdpbmcpZihhcnJhKQpvKHlzXCkpMTE3IDMxNSB5 KElOKTE1NSBiIEZpKGEpbyhycmEpbyh5KXAgNDE2IDMxNSBWCjE1IHcob2YpcCA0NjggMzE1 IFYgMTYgdyhjb21tYW5kKXAgNjY5IDMxNSBWIDE1IHcobGluZXMpMTQyCmIgRmcoZXhlY3V0 YWJsZSkxMiBiKGFuZClmKGFyZ3VtZW4pbyh0cylmKFwoYXJyYSlvKHkpZyhvZilnKHN0cmlu Z3MsKWgKKHNpZ25pXDAxNGNhbilvKHQpOTA1IDM3MSB5KG9ubHkpaShhdCloKHJvKXEob3Rc KSkxMTcKNDQ2IHkoSU4pMTU1IGIgRmkoYSlvKHJyYSlvKHkpcCA0MTYgNDQ2IFYgMTUgdyhv ZilwIDQ2OAo0NDYgViAxNiB3KG1pbnApbyhybylxKGNzKTI1NCBiIEZnKG1pbmltKW4odW0p OSBiKG4pbyh1bSlvKGIpcShlcilrKG9mKQpmKHBybylxKGNlc3NlcylrKGZvcilkKGVhYylv KGgpaChjb21tYW5kKWMobGluZSk5MDUgNTAzCnkodG8payhzdGFydClnKFwoYXJyYSlvKHkp ZyhvZilnKGluKW8odGVnZXIsKWYoc2lnbmlcMDE0Y2FuKW8odCloKG9ubHkpCmYoYXQpaChy bylxKG90XCkpMTE3IDU3OCB5KElOKTE1NSBiIEZpKGEpbyhycmEpbyh5KXAKNDE2IDU3OCBW IDE1IHcob2YpcCA0NjggNTc4IFYgMTYgdyhtYXhwKW8ocm8pcShjcykyNDUKYiBGZyhtYXhp bSluKHVtKTggYihuKW8odW0pbyhiKXEoZXIpayhvZilmKHBybylxKGNlc3NlcylrKGZvcilk KGVhYylvCihoKWcoY29tbWFuZClkKGxpbmUpOTA1IDYzNCB5KHRvKTE0IGIoc3RhcnQpZyhc KGFycmEpbyh5KWcob2YpZyhpbilvCih0ZWdlciwpZihzaWduaVwwMTRjYW4pbyh0KWgob25s eSlmKGF0KWgocm8pcShvdFwpKTExNwo3MDkgeShJTikxNTUgYiBGaShhKW8ocnJhKW8oeSlw IDQxNiA3MDkgViAxNSB3KG9mKXAgNDY4CjcwOSBWIDE2IHcoaW5mbykzNTMgYiBGZyhzdHJp bmdzKTE3IGIodGVsbGluZyllKHRoZSlpKHJ1bilvKHRpbWUpZQooc3lzdGVtKWkod2hlcmUp ZyhhbmQpZihobylvKHcpZyh0byk5MDUgNzY2IHkoc3RhcnQpMjEKYihwcm8pcShjZXNzZXMp aShcKGFycmEpbyh5KWUob2YpZShzdHJpbmdzLClrKHNpZ25pXDAxNGNhbilvKHQpYyhvbmx5 KWgKKGF0KTkwNSA4MjIgeShybylxKG90XCkpMTE3IDg5NyB5KElOKTE1NSBiIEZpKHJvKXEo b3QpNTA4CmIgRmcocmFuaykxMyBiKG9mKWYocHJvKXEoY2VzcylqKGluKWQod2hpYylvKGgp aChwcmV2aW91cylnKGFyZ3VtZW4pbwoodHMpZihhcmUpaSh2KW4oYWxpZCk5MDUgOTU0IHko XChpbilvKHRlZ2VyXCkpMTE3IDEwMjkKeShJTikxNTUgYiBGaShjb21tKTQ3MCBiIEZnKGNv bW0pbyh1bmljYXRvcik3IGIob2YpaShncm91cClnKG9mKWcoc3BhKW8KKHduaW5nKWcocHJv KXEoY2Vzc2VzKWooXChoYW5kbGVcKSkxMTcgMTEwNCB5KE9VVCkxMDgKYiBGaShpbnRlcmNv bW0pMzg0IGIgRmcoaW4pbyh0ZXJjb21tKW8odW5pY2F0b3IpMTIgYihiKXEoZXQpbyh3KW8o ZWVuKWoKKG9yaWdpbmFsKWQoZ3JvdXApaShhbmQpZyhuZXdseSk5MDUgMTE2MSB5KHNwYSlv KHduZWQpaChncm91cCllCihcKGhhbmRsZVwpKTExNyAxMjM2IHkoT1VUKTEwOCBiIEZpKGEp byhycmEpbyh5KXAgNDE2CjEyMzYgViAxNSB3KG9mKXAgNDY4IDEyMzYgViAxNiB3KGVycmNv KXEoZGVzKTI2OCBiIEZnKG9uZSkxNApiKGVycm9yKWgoY28pcShkZSlnKHApcShlcilnKHBy bylxKGNlc3MpZyhcKGFycmEpbyh5KWYob2YpZihpbilvCih0ZWdlclwpKTExNyAxMzExIHko T1VUKTEwOCBiIEZpKHJlcXVlc3QpNDUyIGIgRmcocmVxdWVzdCkxNgpiKG9iKXIoamVjdCll KFwoaGFuZGxlXCkpNzUgMTQzNSB5IEZmKGludCkyMyBiKE1QSSlwCjI0NSAxNDM1IDE1IDIg diAxNyB3KElzcGF3bilwIDQwNiAxNDM1IFYgMTcgdyhtdWx0aXBsZVwoaW50KilmKGNvdW50 LCloCihjaGFyKilnKGFycmF5KXAgMTE4NyAxNDM1IFYgMTYgdyhvZilwIDEyNTEgMTQzNSBW IDE3CncoY29tbWFuZClwIDE0MzYgMTQzNSBWIDE3IHcobGluZXNbXSwpMzkzIDE0OTIgeShp bnQqKWcoYXJyYXkpcAo2MzUgMTQ5MiBWIDE3IHcob2YpcCA3MDAgMTQ5MiBWIDE3IHcobWlu cHJvY3MsKWcoaW50KilnKGFycmF5KXAKMTE5NSAxNDkyIFYgMTYgdyhvZilwIDEyNTkgMTQ5 MiBWIDE3IHcobWF4cHJvY3MsKTM5MwoxNTQ4IHkoY2hhciopZyhhcnJheSlwIDY1OSAxNTQ4 IFYgMTcgdyhvZilwIDcyNCAxNTQ4ClYgMTcgdyhpbmZvW10sKWcoaW50KWcocm9vdCwpZyhN UEkpcCAxMjQyIDE1NDggViAxNyB3KENvbW0pZyhjb21tLCkzOTMKMTYwNSB5KE1QSSlwIDQ2 OCAxNjA1IFYgMTcgdyhDb21tKilnKGludGVyY29tbSwpZyhpbnQqKWcoYXJyYXkpcAoxMTMw IDE2MDUgViAxNiB3KG9mKXAgMTE5NCAxNjA1IFYgMTcgdyhlcnJjb2RlcywpMzkzCjE2NjEg eShNUEkpcCA0NjggMTY2MSBWIDE3IHcoUmVxdWVzdCopZyhyZXF1ZXN0XCkpNzUKMTc0OCB5 KE1QSSlwIDE1MCAxNzQ4IFYgMTcgdyhJU1BBV04pcCAzMTEgMTc0OCBWIDE2IHcoTVVMVElQ TEVcKENPVU5ULCkKZihBUlJBWSlwIDgyOSAxNzQ4IFYgMTYgdyhPRilwIDg5MyAxNzQ4IFYg MTcgdyhDT01NQU5EKXAKMTA3OCAxNzQ4IFYgMTcgdyhMSU5FUywpaChBUlJBWSlwIDEzODIg MTc0OCBWIDE2IHcoT0YpcAoxNDQ2IDE3NDggViAxNyB3KE1JTlBST0NTLCkzOTMgMTgwNCB5 KEFSUkFZKXAgNTE2IDE4MDQKViAxNyB3KE9GKXAgNTgxIDE4MDQgViAxNyB3KE1BWFBST0NT LClmKEFSUkFZKXAgOTU2IDE4MDQKViAxNyB3KE9GKXAgMTAyMSAxODA0IFYgMTcgdyhJTkZP LCloKFJPT1QsKWcoQ09NTSwpZyhJTlRFUkNPTU0sKTM5MwoxODYwIHkoQVJSQVkpcCA1MTYg MTg2MCBWIDE3IHcoT0YpcCA1ODEgMTg2MCBWIDE3IHcoRVJSQ09ERVMsKWYKKFJFUVVFU1Qs KWgoSUVSUlwpKTE3MCAxOTE3IHkoQ0hBUkFDVEVSKlwoKlwpKWcoQVJSQVkpcAo2MjggMTkx NyBWIDE2IHcoT0YpcCA2OTIgMTkxNyBWIDE3IHcoQ09NTUFORClwIDg3NyAxOTE3ClYgMTYg dyhMSU5FU1woKlwpLClnKEFSUkFZKXAgMTI1MiAxOTE3IFYgMTYgdyhPRilwIDEzMTYKMTkx NyBWIDE3IHcoSU5GTykxNzAgMTk3MyB5KElOVEVHRVIpZyhDT1VOVCwpZyhBUlJBWSlwCjY1 MSAxOTczIFYgMTcgdyhPRilwIDcxNiAxOTczIFYgMTcgdyhNSU5QUk9DUywpZihBUlJBWSlw CjEwOTEgMTk3MyBWIDE3IHcoT0YpcCAxMTU2IDE5NzMgViAxNyB3KE1BWFBST0NTLClnKFJP T1QsKWkoQ09NTSwpMTcwCjIwMzAgeShJTlRFUkNPTU0sKWYoQVJSQVkpcCA1NTYgMjAzMCBW IDE2IHcoT0YpcCA2MjAKMjAzMCBWIDE3IHcoRVJSQ09ERVNcKCpcKSwpZyhSRVFVRVNULClm KElFUlIpMTk2NyA0Ngp5IEZlKDEpMTk2NyAxMDMgeSgyKTE5NjcgMTU5IHkoMykxOTY3IDIx NSB5KDQpMTk2NyAyNzIKeSg1KTE5NjcgMzI4IHkoNikxOTY3IDM4NSB5KDcpMTk2NyA0NDEg eSg4KTE5NjcgNDk4IHkoOSkxOTU5CjU1NCB5KDEwKTE5NTkgNjExIHkoMTEpMTk1OSA2Njcg eSgxMikxOTU5IDcyNCB5KDEzKTE5NTkKNzgwIHkoMTQpMTk1OSA4MzYgeSgxNSkxOTU5IDg5 MyB5KDE2KTE5NTkgOTQ5IHkoMTcpMTk1OQoxMDA2IHkoMTgpMTk1OSAxMDYyIHkoMTkpMTk1 OSAxMTE5IHkoMjApMTk1OSAxMTc1IHkoMjEpMTk1OQoxMjMyIHkoMjIpMTk1OSAxMjg4IHko MjMpMTk1OSAxMzQ1IHkoMjQpMTk1OSAxNDAxIHkoMjUpMTk1OQoxNDU3IHkoMjYpMTk1OSAx NTE0IHkoMjcpMTk1OSAxNTcwIHkoMjgpMTk1OSAxNjI3IHkoMjkpMTk1OQoxNjgzIHkoMzAp MTk1OSAxNzQwIHkoMzEpMTk1OSAxNzk2IHkoMzIpMTk1OSAxODUzIHkoMzMpMTk1OQoxOTA5 IHkoMzQpMTk1OSAxOTY2IHkoMzUpMTk1OSAyMDIyIHkoMzYpMTk1OSAyMDc4IHkoMzcpMTk1 OQoyMTM1IHkoMzgpMTk1OSAyMTkxIHkoMzkpMTk1OSAyMjQ4IHkoNDApMTk1OSAyMzA0IHko NDEpMTk1OQoyMzYxIHkoNDIpMTk1OSAyNDE3IHkoNDMpMTk1OSAyNDc0IHkoNDQpMTk1OSAy NTMwIHkoNDUpMTk1OQoyNTg3IHkoNDYpMTk1OSAyNjQzIHkoNDcpMTk1OSAyNjk5IHkoNDgp cCBlb3AKJSVQYWdlOiA3IDgKNyA3IGJvcCAxODUyIC0xMDAgYSBGbCg3KTc1IDQ5IHkgRmko TVBJKXAgMTYwIDQ5IDE0IDIKdiAxNiB3KElTUClsKEEpLTUgYihXTilwIDMzOSA0OSBWIDE3 IHcoSU5ERVBFTkRFTlQpcAo2NzIgNDkgViAxNiB3KE1QSVwoY29tbWFuZClwIDk3MyA0OSBW IDE1IHcobGluZSwpMTUgYihtaW5wKW8ocm8pcShjcywpZwoobWF4cClvKHJvKXEoY3MsKWYo aW5mbywpZyhybylxKG90LCloKGNvbW0sKTc1IDEwNiB5KGdyb3VwLClnKGEpbyhycmEpbwoo eSlwIDMwOSAxMDYgViAxNCB3KG9mKXAgMzYwIDEwNiBWIDE2IHcoZXJyY28pcShkZXMsKWgo cmVxdWVzdFwpKTExNwoxODMgeSBGZyhJTikxNTUgYiBGaShjb21tYW5kKXAgNTA2IDE4MyBW IDE2IHcobGluZSkzMjEKYiBGZyhleGVjdXRhYmxlKTE4IGIoYW5kKWYoYXJndW1lbilvKHRz LClnKGluKWcoYSlmKHNpbmdsZSloKHN0cmluZylnCihcKHN0cmluZywpOTA1IDIzOSB5KHNp Z25pXDAxNGNhbilvKHQpZChvbmx5KWYoYXQpZyhybylxKG90XCkpMTE3CjMxNSB5KElOKTE1 NSBiIEZpKG1pbnApbyhybylxKGNzKTQxNyBiIEZnKG1pbmltKW4odW0pMTQKYihuKW8odW0p byhiKXEoZXIpaihvZilnKHBybylxKGNlc3NlcylrKHRvKWMoc3RhcnQpaShcKGluKW8odGVn ZXIsKWcKKHNpZy0pOTA1IDM3MSB5KG5pXDAxNGNhbilvKHQpMTQgYihvbmx5KWYoYXQpaChy bylxKG90XCkpMTE3CjQ0NiB5KElOKTE1NSBiIEZpKG1heHApbyhybylxKGNzKTQwOCBiIEZn KG1heGltKW4odW0pMTMKYihuKW8odW0pbyhiKXEoZXIpaihvZilnKHBybylxKGNlc3Nlcylq KHRvKWUoc3RhcnQpZyhcKGluKW8odGVnZXIsKWgKKHNpZy0pOTA1IDUwMyB5KG5pXDAxNGNh bilvKHQpYyhvbmx5KWYoYXQpaChybylxKG90XCkpMTE3CjU3OCB5KElOKTE1NSBiIEZpKGlu Zm8pNTE2IGIgRmcoYSkxNCBiKHN0cmluZylnKHRlbGxpbmcpZih0aGUpaShydW4pbwoodGlt ZSllKHN5c3RlbSlnKHdoZXJlKWooYW5kKWQoaG8pbyh3KWgodG8pOTA1IDYzNCB5KHN0YXJ0 KWgodGhlKWYocHJvKQpxKGNlc3NlcylqKFwoc3RyaW5nLCljKHNpZ25pXDAxNGNhbilvKHQp aChvbmx5KWYoYXQpaChybylxKG90XCkpMTE3CjcwOSB5KElOKTE1NSBiIEZpKHJvKXEob3Qp NTA4IGIgRmcocmFuaykxMyBiKG9mKWYocHJvKXEoY2VzcylqKGluKWQKKHdoaWMpbyhoKWgo cHJldmlvdXMpZyhhcmd1bWVuKW8odHMpZihhcmUpaSh2KW4oYWxpZCk5MDUKNzY2IHkoXChp bilvKHRlZ2VyXCkpMTE3IDg0MSB5KElOKTE1NSBiIEZpKGNvbW0pNDcwIGIKRmcoY29tbSlv KHVuaWNhdG9yKTcgYihvZilpKGdyb3VwKWcob2YpZyhzcGEpbyh3bmluZylnKHBybylxKGNl c3NlcylqCihcKGhhbmRsZVwpKTExNyA5MTYgeShPVVQpMTA4IGIgRmkoZ3JvdXApNDc5IGIg RmcoZ3JvdXApMTQKYihvZilmKHNwYSlvKHduZWQpaShwcm8pcShjZXNzZXMpaChcKGhhbmRs ZVwpKTExNyA5OTEKeShPVVQpMTA4IGIgRmkoYSlvKHJyYSlvKHkpcCA0MTYgOTkxIFYgMTUg dyhvZilwIDQ2OAo5OTEgViAxNiB3KGVycmNvKXEoZGVzKTI2OCBiIEZnKG9uZSkxNCBiKGNv KXEoZGUpaChwKXEoZXIpZyhwcm8pcShjZXNzKQpoKFwoYXJyYSlvKHkpZChvZiloKGluKW8o dGVnZXJcKSkxMTcgMTA2NiB5KE9VVCkxMDggYgpGaShyZXF1ZXN0KTQ1MiBiIEZnKHJlcXVl c3QpMTYgYihvYilyKGplY3QpZShcKGhhbmRsZVwpKTc1CjExOTEgeSBGZihpbnQpMjMgYihN UEkpcCAyNDUgMTE5MSAxNSAyIHYgMTcgdyhJc3Bhd24pcAo0MDYgMTE5MSBWIDE3IHcoaW5k ZXBlbmRlbnQpcCA2ODcgMTE5MSBWIDE1IHcobXBpXChjaGFyKilnKGNvbW1hbmQpcAoxMTA5 IDExOTEgViAxNiB3KGxpbmUsKWcoaW50KWgobWlucHJvY3MsKTM5MyAxMjQ3IHkoaW50KWco bWF4cHJvY3MsKWUKKGNoYXIqKWgoaW5mbywpaChpbnQpZihyb290LClnKE1QSSlwIDEzMjcg MTI0NyBWIDE3IHcoQ29tbSlnKGNvbW0sKTM5MwoxMzA0IHkoTVBJKXAgNDY4IDEzMDQgViAx NyB3KEdyb3VwKilnKGdyb3VwLClnKGludCopZyhhcnJheSlwCjEwNTggMTMwNCBWIDE3IHco b2YpcCAxMTIzIDEzMDQgViAxNyB3KGVycmNvZGVzLClmKE1QSSlwCjE0NTAgMTMwNCBWIDE3 IHcoUmVxdWVzdCopaChyZXF1ZXN0XCkpNzUgMTM5MCB5KE1QSSlwCjE1MCAxMzkwIFYgMTcg dyhTUEFXTilwIDI4NyAxMzkwIFYgMTYgdyhJTkRFUEVOREVOVClwCjU2NyAxMzkwIFYgMTYg dyhNUElcKENPTU1BTkQpcCA4NDcgMTM5MCBWIDE2IHcoTElORSwpZyhNSU5QUk9DUywpZgoo TUFYUFJPQ1MsKWgoSU5GTywpZyhST09ULCkzOTMgMTQ0NyB5KENPTU0sKWcoSU5URVJDT01N LClnKEFSUkFZKXAKOTIyIDE0NDcgViAxNiB3KE9GKXAgOTg2IDE0NDcgViAxNyB3KEVSUkNP REVTLClnKFJFUVVFU1QsKWcoSUVSUlwpKTE3MAoxNTAzIHkoQ0hBUkFDVEVSKlwoKlwpKWco Q09NTUFORClwIDY3NiAxNTAzIFYgMTYgdyhMSU5FLClnKElORk8pMTcwCjE1NTkgeShJTlRF R0VSKWcoTUlOUFJPQ1MsKWcoTUFYUFJPQ1MsKWcoUk9PVCwpZyhDT01NLClnKEdST1VQLCln CihBUlJBWSlwIDE0MTUgMTU1OSBWIDE3IHcoT0YpcCAxNDgwIDE1NTkgViAxNiB3KEVSUkNP REVTXCgqXCksKTE3MAoxNjE2IHkoUkVRVUVTVCwpZyhJRVJSKS0zMiA0NiB5IEZlKDEpLTMy IDEwMyB5KDIpLTMyCjE1OSB5KDMpLTMyIDIxNSB5KDQpLTMyIDI3MiB5KDUpLTMyIDMyOCB5 KDYpLTMyIDM4NSB5KDcpLTMyCjQ0MSB5KDgpLTMyIDQ5OCB5KDkpLTQwIDU1NCB5KDEwKS00 MCA2MTEgeSgxMSktNDAgNjY3CnkoMTIpLTQwIDcyNCB5KDEzKS00MCA3ODAgeSgxNCktNDAg ODM2IHkoMTUpLTQwIDg5MyB5KDE2KS00MAo5NDkgeSgxNyktNDAgMTAwNiB5KDE4KS00MCAx MDYyIHkoMTkpLTQwIDExMTkgeSgyMCktNDAKMTE3NSB5KDIxKS00MCAxMjMyIHkoMjIpLTQw IDEyODggeSgyMyktNDAgMTM0NSB5KDI0KS00MAoxNDAxIHkoMjUpLTQwIDE0NTcgeSgyNikt NDAgMTUxNCB5KDI3KS00MCAxNTcwIHkoMjgpLTQwCjE2MjcgeSgyOSktNDAgMTY4MyB5KDMw KS00MCAxNzQwIHkoMzEpLTQwIDE3OTYgeSgzMiktNDAKMTg1MyB5KDMzKS00MCAxOTA5IHko MzQpLTQwIDE5NjYgeSgzNSktNDAgMjAyMiB5KDM2KS00MAoyMDc4IHkoMzcpLTQwIDIxMzUg eSgzOCktNDAgMjE5MSB5KDM5KS00MCAyMjQ4IHkoNDApLTQwCjIzMDQgeSg0MSktNDAgMjM2 MSB5KDQyKS00MCAyNDE3IHkoNDMpLTQwIDI0NzQgeSg0NCktNDAKMjUzMCB5KDQ1KS00MCAy NTg3IHkoNDYpLTQwIDI2NDMgeSg0NyktNDAgMjY5OSB5KDQ4KXAKZW9wCiUlUGFnZTogOCA5 CjggOCBib3AgNzUgLTEwMCBhIEZsKDgpNzUgNDkgeSBGaShNUEkpcCAxNjAgNDkgMTQgMiB2 CjE2IHcoSVNQKWwoQSktNSBiKFdOKXAgMzM5IDQ5IFYgMTcgdyhJTkRFUEVOREVOVClwIDY3 Mgo0OSBWIDE2IHcoTk9OTVBJXChjb21tYW5kKXAgMTA3MCA0OSBWIDE2IHcobGluZSwpMTUg YihtaW5wKW8ocm8pcShjcywpZgoobWF4cClvKHJvKXEoY3MsKWcoaW5mbywpaChybylxKG90 LCk3NSAxMDYgeShjb21tLCllKGdyb3VwLClpKGEpbyhycmEpbwooeSlwIDQ1MiAxMDYgViAx NCB3KG9mKXAgNTAzIDEwNiBWIDE2IHcoZXJyY28pcShkZXMsKWgocmVxdWVzdFwpKTExNwox ODMgeSBGZyhJTikxNTUgYiBGaShjb21tYW5kKXAgNTA2IDE4MyBWIDE2IHcobGluZSkzMjEK YiBGZyhleGVjdXRhYmxlKTE4IGIoYW5kKWYoYXJndW1lbilvKHRzLClnKGluKWcoYSlmKHNp bmdsZSloKHN0cmluZylnCihcKHN0cmluZywpOTA1IDIzOSB5KHNpZ25pXDAxNGNhbilvKHQp ZChvbmx5KWYoYXQpZyhybylxKG90XCkpMTE3CjMxNSB5KElOKTE1NSBiIEZpKG1pbnApbyhy bylxKGNzKTQxNyBiIEZnKG1pbmltKW4odW0pMTQKYihuKW8odW0pbyhiKXEoZXIpaihvZiln KHBybylxKGNlc3NlcylrKHRvKWMoc3RhcnQpaShcKGluKW8odGVnZXIsKWcKKHNpZy0pOTA1 IDM3MSB5KG5pXDAxNGNhbilvKHQpMTQgYihvbmx5KWYoYXQpaChybylxKG90XCkpMTE3CjQ0 NiB5KElOKTE1NSBiIEZpKG1heHApbyhybylxKGNzKTQwOCBiIEZnKG1heGltKW4odW0pMTMK YihuKW8odW0pbyhiKXEoZXIpaihvZilnKHBybylxKGNlc3NlcylqKHRvKWUoc3RhcnQpZyhc KGluKW8odGVnZXIsKWgKKHNpZy0pOTA1IDUwMyB5KG5pXDAxNGNhbilvKHQpYyhvbmx5KWYo YXQpaChybylxKG90XCkpMTE3CjU3OCB5KElOKTE1NSBiIEZpKGluZm8pNTE2IGIgRmcoYSkx NCBiKHN0cmluZylnKHRlbGxpbmcpZih0aGUpaShydW4pbwoodGltZSllKHN5c3RlbSlnKHdo ZXJlKWooYW5kKWQoaG8pbyh3KWgodG8pOTA1IDYzNCB5KHN0YXJ0KWgodGhlKWYocHJvKQpx KGNlc3NlcylqKFwoc3RyaW5nLCljKHNpZ25pXDAxNGNhbilvKHQpaChvbmx5KWYoYXQpaChy bylxKG90XCkpMTE3CjcwOSB5KElOKTE1NSBiIEZpKHJvKXEob3QpNTA4IGIgRmcocmFuaykx MyBiKG9mKWYocHJvKXEoY2VzcylqKGluKWQKKHdoaWMpbyhoKWgocHJldmlvdXMpZyhhcmd1 bWVuKW8odHMpZihhcmUpaSh2KW4oYWxpZCk5MDUKNzY2IHkoXChpbilvKHRlZ2VyXCkpMTE3 IDg0MSB5KElOKTE1NSBiIEZpKGNvbW0pNDcwIGIKRmcoY29tbSlvKHVuaWNhdG9yKTcgYihv ZilpKGdyb3VwKWcob2YpZyhzcGEpbyh3bmluZylnKHBybylxKGNlc3NlcylqCihcKGhhbmRs ZVwpKTExNyA5MTYgeShPVVQpMTA4IGIgRmkoZ3JvdXApNDc5IGIgRmcoZ3JvdXApMTQKYihj b24pbyh0YWluaW5nKWYoc3BhKW8od25lZCloKHBybylxKGNlc3NlcylqKFwoaGFuZGxlXCkp MTE3Cjk5MSB5KE9VVCkxMDggYiBGaShhKW8ocnJhKW8oeSlwIDQxNiA5OTEgViAxNSB3KG9m KXAKNDY4IDk5MSBWIDE2IHcoZXJyY28pcShkZXMpMjY4IGIgRmcob25lKTE0IGIoY28pcShk ZSloKHApcShlcilnKHBybylxCihjZXNzKWgoXChhcnJhKW8oeSlkKG9mKWgoaW4pbyh0ZWdl clwpKTExNyAxMDY2IHkoT1VUKTEwOApiIEZpKHJlcXVlc3QpNDUyIGIgRmcocmVxdWVzdCkx NiBiKG9iKXIoamVjdCllKFwoaGFuZGxlXCkpNzUKMTE5MSB5IEZmKGludCkyMyBiKE1QSSlw IDI0NSAxMTkxIDE1IDIgdiAxNyB3KElzcGF3bilwCjQwNiAxMTkxIFYgMTcgdyhpbmRlcGVu ZGVudClwIDY4NyAxMTkxIFYgMTUgdyhub25tcGlcKGNoYXIqKWYoY29tbWFuZClwCjExODAg MTE5MSBWIDE3IHcobGluZSwpaChpbnQpZyhtaW5wcm9jcywpMzkzIDEyNDcgeShpbnQpaCht YXhwcm9jcywpZQooY2hhciopaChpbmZvLCloKGludClmKHJvb3QsKWcoTVBJKXAgMTMyNyAx MjQ3IFYgMTcgdyhDb21tKWcoY29tbSwpMzkzCjEzMDQgeShNUEkpcCA0NjggMTMwNCBWIDE3 IHcoR3JvdXAqKWcoZ3JvdXAsKWcoaW50KilnKGFycmF5KXAKMTA1OCAxMzA0IFYgMTcgdyhv ZilwIDExMjMgMTMwNCBWIDE3IHcoZXJyY29kZXMsKWYoTVBJKXAKMTQ1MCAxMzA0IFYgMTcg dyhSZXF1ZXN0KiloKHJlcXVlc3RcKSk3NSAxMzkwIHkoTVBJKXAKMTUwIDEzOTAgViAxNyB3 KElTUEFXTilwIDMxMSAxMzkwIFYgMTYgdyhJTkRFUEVOREVOVClwCjU5MSAxMzkwIFYgMTYg dyhOT05NUElcKENPTU1BTkQpcCA5NDMgMTM5MCBWIDE1IHcoTElORSwpZyhNSU5QUk9DUywp ZwooTUFYUFJPQ1MsKWcoSU5GTywpZyhST09ULCkzOTMgMTQ0NyB5KENPTU0sKWcoR1JPVVAs KWcoQVJSQVkpcAo4MjYgMTQ0NyBWIDE3IHcoT0YpcCA4OTEgMTQ0NyBWIDE3IHcoRVJSQ09E RVMsKWYoUkVRVUVTVCwpaChJRVJSXCkpMTcwCjE1MDMgeShDSEFSQUNURVIqXCgqXCkpZyhD T01NQU5EKXAgNjc2IDE1MDMgViAxNiB3KExJTkUsKWcoSU5GTykxNzAKMTU1OSB5KElOVEVH RVIpZyhNSU5QUk9DUywpZyhNQVhQUk9DUywpZyhST09ULClnKENPTU0sKWcoR1JPVVAsKWcK KEFSUkFZKXAgMTQxNSAxNTU5IFYgMTcgdyhPRilwIDE0ODAgMTU1OSBWIDE2IHcoRVJSQ09E RVNcKCpcKSwpMTcwCjE2MTYgeShSRVFVRVNULClnKElFUlIpMTk2NyA0NiB5IEZlKDEpMTk2 NyAxMDMgeSgyKTE5NjcKMTU5IHkoMykxOTY3IDIxNSB5KDQpMTk2NyAyNzIgeSg1KTE5Njcg MzI4IHkoNikxOTY3IDM4NQp5KDcpMTk2NyA0NDEgeSg4KTE5NjcgNDk4IHkoOSkxOTU5IDU1 NCB5KDEwKTE5NTkgNjExCnkoMTEpMTk1OSA2NjcgeSgxMikxOTU5IDcyNCB5KDEzKTE5NTkg NzgwIHkoMTQpMTk1OSA4MzYKeSgxNSkxOTU5IDg5MyB5KDE2KTE5NTkgOTQ5IHkoMTcpMTk1 OSAxMDA2IHkoMTgpMTk1OQoxMDYyIHkoMTkpMTk1OSAxMTE5IHkoMjApMTk1OSAxMTc1IHko MjEpMTk1OSAxMjMyIHkoMjIpMTk1OQoxMjg4IHkoMjMpMTk1OSAxMzQ1IHkoMjQpMTk1OSAx NDAxIHkoMjUpMTk1OSAxNDU3IHkoMjYpMTk1OQoxNTE0IHkoMjcpMTk1OSAxNTcwIHkoMjgp MTk1OSAxNjI3IHkoMjkpMTk1OSAxNjgzIHkoMzApMTk1OQoxNzQwIHkoMzEpMTk1OSAxNzk2 IHkoMzIpMTk1OSAxODUzIHkoMzMpMTk1OSAxOTA5IHkoMzQpMTk1OQoxOTY2IHkoMzUpMTk1 OSAyMDIyIHkoMzYpMTk1OSAyMDc4IHkoMzcpMTk1OSAyMTM1IHkoMzgpMTk1OQoyMTkxIHko MzkpMTk1OSAyMjQ4IHkoNDApMTk1OSAyMzA0IHkoNDEpMTk1OSAyMzYxIHkoNDIpMTk1OQoy NDE3IHkoNDMpMTk1OSAyNDc0IHkoNDQpMTk1OSAyNTMwIHkoNDUpMTk1OSAyNTg3IHkoNDYp MTk1OQoyNjQzIHkoNDcpMTk1OSAyNjk5IHkoNDgpcCBlb3AKJSVQYWdlOiA5IDEwCjkgOSBi b3AgMTg1MiAtMTAwIGEgRmwoOSk3NSA0OSB5IEZpKE1QSSlwIDE2MCA0OSAxNCAyCnYgMTYg dyhJU1ApbChBKS01IGIoV04pcCAzMzkgNDkgViAxNyB3KE1VTClsKFRJUExFKXAKNTczIDQ5 IFYgMTUgdyhJTkRFUEVOREVOVClwIDkwNCA0OSBWIDE3IHcoTVBJXChjb3VudCwpMTUKYihh KW8ocnJhKW8oeSlwIDEyNDkgNDkgViAxNSB3KG9mKXAgMTMwMSA0OSBWIDE2IHcoY29tbWFu ZClwCjE1MDIgNDkgViAxNSB3KGxpbmVzLCk3NSAxMDYgeShhKW8ocnJhKW8oeSlwIDE3MyAx MDYKViAxNSB3KG9mKXAgMjI1IDEwNiBWIDE2IHcobWlucClvKHJvKXEoY3MsKWYoYSlvKHJy YSlvKHkpcAo1MzMgMTA2IFYgMTUgdyhvZilwIDU4NSAxMDYgViAxNiB3KG1heHApbyhybylx KGNzLClnKGEpbyhycmEpbyh5KXAKOTAyIDEwNiBWIDE0IHcob2YpcCA5NTMgMTA2IFYgMTYg dyhpbmZvLCloKHJvKXEob3QsKWYoY29tbSwpZihncm91cCwpaQooYSlvKHJyYSlvKHkpcCAx NTQ4IDEwNiBWIDE1IHcob2YpcCAxNjAwIDEwNiBWIDE2IHcoZXJyY28pcShkZXMsKTc1CjE2 MiB5KHJlcXVlc3RcKSkxMTcgMjM5IHkgRmcoSU4pMTU1IGIgRmkoY291bnQpNDgyIGIKRmco TnVtKW8oYilxKGVyKTEyIGIob2YpZyhjb21tYW5kKWUobGluZXMpaihcKGluKW8odGVnZXIs KWcoc2l6ZSlnKG9mKWYKKGVhYylvKGgpaChvZilmKHRoZSk5MDUgMjk2IHkoZm9sbG8pbyh3 aW5nKWYoYXJyYSlvKHlzXCkpMTE3CjM3MSB5KElOKTE1NSBiIEZpKGEpbyhycmEpbyh5KXAg NDE2IDM3MSBWIDE1IHcob2YpcCA0NjgKMzcxIFYgMTYgdyhjb21tYW5kKXAgNjY5IDM3MSBW IDE1IHcobGluZXMpMTQyIGIgRmcoZXhlY3V0YWJsZSkxMgpiKGFuZClmKGFyZ3VtZW4pbyh0 cylmKFwoYXJyYSlvKHkpZyhvZilnKHN0cmluZ3MsKWgoc2lnbmlcMDE0Y2FuKW8odCkKOTA1 IDQyNyB5KG9ubHkpaShhdCloKHJvKXEob3RcKSkxMTcgNTAzIHkoSU4pMTU1IGIgRmkoYSlv KHJyYSlvKHkpcAo0MTYgNTAzIFYgMTUgdyhvZilwIDQ2OCA1MDMgViAxNiB3KG1pbnApbyhy bylxKGNzKTI1NApiIEZnKG1pbmltKW4odW0pOSBiKG4pbyh1bSlvKGIpcShlcilrKG9mKWYo cHJvKXEoY2Vzc2VzKWsoZm9yKWQoZWFjKW8KKGgpaChjb21tYW5kKWMobGluZSk5MDUgNTU5 IHkodG8payhzdGFydClnKFwoYXJyYSlvKHkpZyhvZilnKGluKW8KKHRlZ2VyLClmKHNpZ25p XDAxNGNhbilvKHQpaChvbmx5KWYoYXQpaChybylxKG90XCkpMTE3CjYzNCB5KElOKTE1NSBi IEZpKGEpbyhycmEpbyh5KXAgNDE2IDYzNCBWIDE1IHcob2YpcCA0NjgKNjM0IFYgMTYgdyht YXhwKW8ocm8pcShjcykyNDUgYiBGZyhtYXhpbSluKHVtKTggYihuKW8odW0pbyhiKXEoZXIp ayhvZikKZihwcm8pcShjZXNzZXMpayhmb3IpZChlYWMpbyhoKWcoY29tbWFuZClkKGxpbmUp OTA1IDY5MQp5KHRvKTE0IGIoc3RhcnQpZyhcKGFycmEpbyh5KWcob2YpZyhpbilvKHRlZ2Vy LClmKHNpZ25pXDAxNGNhbilvKHQpaAoob25seSlmKGF0KWgocm8pcShvdFwpKTExNyA3NjYg eShJTikxNTUgYiBGaShhKW8ocnJhKW8oeSlwCjQxNiA3NjYgViAxNSB3KG9mKXAgNDY4IDc2 NiBWIDE2IHcoaW5mbykzNTMgYiBGZyhzdHJpbmdzKTE3CmIodGVsbGluZyllKHRoZSlpKHJ1 bilvKHRpbWUpZShzeXN0ZW0paSh3aGVyZSlnKGFuZClmKGhvKW8odylnKHRvKTkwNQo4MjIg eShzdGFydCkyMSBiKHBybylxKGNlc3NlcylpKFwoYXJyYSlvKHkpZShvZillKHN0cmluZ3Ms KWsKKHNpZ25pXDAxNGNhbilvKHQpYyhvbmx5KWgoYXQpOTA1IDg3OSB5KHJvKXEob3RcKSkx MTcKOTU0IHkoSU4pMTU1IGIgRmkocm8pcShvdCk1MDggYiBGZyhyYW5rKTEzIGIob2YpZihw cm8pcShjZXNzKWooaW4pZAood2hpYylvKGgpaChwcmV2aW91cylnKGFyZ3VtZW4pbyh0cylm KGFyZSlpKHYpbihhbGlkKTkwNQoxMDEwIHkoXChpbilvKHRlZ2VyXCkpMTE3IDEwODUgeShJ TikxNTUgYiBGaShjb21tKTQ3MApiIEZnKGNvbW0pbyh1bmljYXRvcik3IGIob2YpaShncm91 cClnKG9mKWcoc3BhKW8od25pbmcpZyhwcm8pcShjZXNzZXMpagooXChoYW5kbGVcKSkxMTcg MTE2MSB5KE9VVCkxMDggYiBGaShncm91cCk0NzkgYiBGZyhncm91cCkxNApiKG9mKWYoc3Bh KW8od25lZClpKHBybylxKGNlc3NlcyloKFwoaGFuZGxlXCkpMTE3IDEyMzYKeShPVVQpMTA4 IGIgRmkoYSlvKHJyYSlvKHkpcCA0MTYgMTIzNiBWIDE1IHcob2YpcCA0NjgKMTIzNiBWIDE2 IHcoZXJyY28pcShkZXMpMjY4IGIgRmcob25lKTE0IGIoZXJyb3IpaChjbylxKGRlKWcocClx KGVyKWcKKHBybylxKGNlc3MpZyhcKGFycmEpbyh5KWYob2YpZihpbilvKHRlZ2VyXCkpMTE3 IDEzMTEKeShPVVQpMTA4IGIgRmkocmVxdWVzdCk0NTIgYiBGZyhyZXF1ZXN0KTE2IGIob2Ip cihqZWN0KWUoXChoYW5kbGVcKSk3NQoxNDM1IHkgRmYoaW50KTIzIGIoTVBJKXAgMjQ1IDE0 MzUgMTUgMiB2IDE3IHcoSXNwYXduKXAKNDA2IDE0MzUgViAxNyB3KG11bHRpcGxlKXAgNjE1 IDE0MzUgViAxNiB3KGluZGVwZW5kZW50KXAKODk1IDE0MzUgViAxNSB3KG1waVwoaW50Kiln KGNvdW50LCkzOTMgMTQ5MiB5KGNoYXIqKWcoYXJyYXkpcAo2NTkgMTQ5MiBWIDE3IHcob2Yp cCA3MjQgMTQ5MiBWIDE3IHcoY29tbWFuZClwIDkwOSAxNDkyClYgMTYgdyhsaW5lc1tdLCln KGludCopZyhhcnJheSlwIDEzNzkgMTQ5MiBWIDE3IHcob2YpcAoxNDQ0IDE0OTIgViAxNiB3 KG1pbnByb2NzLCkzOTMgMTU0OCB5KGludCopZyhhcnJheSlwCjYzNSAxNTQ4IFYgMTcgdyhv ZilwIDcwMCAxNTQ4IFYgMTcgdyhtYXhwcm9jcywpZyhjaGFyKilnKGFycmF5KXAKMTIxOSAx NTQ4IFYgMTYgdyhvZilwIDEyODMgMTU0OCBWIDE3IHcoaW5mb1tdLClnKGludClnKHJvb3Qs KTM5MwoxNjA1IHkoTVBJKXAgNDY4IDE2MDUgViAxNyB3KENvbW0pZyhjb21tLClnKE1QSSlw IDgxOQoxNjA1IFYgMTcgdyhHcm91cCopZyhncm91cCwpZyhpbnQqKWgoYXJyYXkpcCAxNDEw IDE2MDUKViAxNiB3KG9mKXAgMTQ3NCAxNjA1IFYgMTcgdyhlcnJjb2RlcywpMzkzIDE2NjEg eShNUEkpcAo0NjggMTY2MSBWIDE3IHcoUmVxdWVzdCopZihyZXF1ZXN0XCkpNzUgMTc0OCB5 KE1QSSlwCjE1MCAxNzQ4IFYgMTcgdyhJU1BBV04pcCAzMTEgMTc0OCBWIDE2IHcoTVVMVElQ TEUpcCA1MTkKMTc0OCBWIDE2IHcoSU5ERVBFTkRFTlQpcCA3OTkgMTc0OCBWIDE2IHcoTVBJ XChDT1VOVCwpZihBUlJBWSlwCjExOTcgMTc0OCBWIDE3IHcoT0YpcCAxMjYyIDE3NDggViAx NyB3KENPTU1BTkQpcCAxNDQ3CjE3NDggViAxNiB3KExJTkVTLCkzOTMgMTgwNCB5KEFSUkFZ KXAgNTE2IDE4MDQgViAxNyB3KE9GKXAKNTgxIDE4MDQgViAxNyB3KE1JTlBST0NTLClnKEFS UkFZKXAgOTU2IDE4MDQgViAxNyB3KE9GKXAKMTAyMSAxODA0IFYgMTcgdyhNQVhQUk9DUywp ZyhBUlJBWSlwIDEzOTYgMTgwNCBWIDE3IHcoT0YpcAoxNDYxIDE4MDQgViAxNyB3KElORk8s KWgoUk9PVCwpMzkzIDE4NjAgeShDT01NLClnKEdST1VQLClnKEFSUkFZKXAKODI2IDE4NjAg ViAxNyB3KE9GKXAgODkxIDE4NjAgViAxNyB3KEVSUkNPREVTLClmKFJFUVVFU1QsKWgoSUVS UlwpKTE3MAoxOTE3IHkoQ0hBUkFDVEVSKlwoKlwpKWcoQVJSQVkpcCA2MjggMTkxNyBWIDE2 IHcoT0YpcAo2OTIgMTkxNyBWIDE3IHcoQ09NTUFORClwIDg3NyAxOTE3IFYgMTYgdyhMSU5F U1woKlwpLClnKEFSUkFZKXAKMTI1MiAxOTE3IFYgMTYgdyhPRilwIDEzMTYgMTkxNyBWIDE3 IHcoSU5GTykxNzAgMTk3Mwp5KElOVEVHRVIpZyhDT1VOVCwpZyhBUlJBWSlwIDY1MSAxOTcz IFYgMTcgdyhPRilwIDcxNgoxOTczIFYgMTcgdyhNSU5QUk9DUywpZihBUlJBWSlwIDEwOTEg MTk3MyBWIDE3IHcoT0YpcAoxMTU2IDE5NzMgViAxNyB3KE1BWFBST0NTLClnKFJPT1QsKWko Q09NTSwpZihHUk9VUCwpMTcwCjIwMzAgeShBUlJBWSlwIDI5MyAyMDMwIFYgMTcgdyhPRilw IDM1OCAyMDMwIFYgMTcgdyhFUlJDT0RFU1woKlwpLClmCihSRVFVRVNULCloKElFUlIpLTMy IDQ2IHkgRmUoMSktMzIgMTAzIHkoMiktMzIgMTU5IHkoMyktMzIKMjE1IHkoNCktMzIgMjcy IHkoNSktMzIgMzI4IHkoNiktMzIgMzg1IHkoNyktMzIgNDQxIHkoOCktMzIKNDk4IHkoOSkt NDAgNTU0IHkoMTApLTQwIDYxMSB5KDExKS00MCA2NjcgeSgxMiktNDAgNzI0CnkoMTMpLTQw IDc4MCB5KDE0KS00MCA4MzYgeSgxNSktNDAgODkzIHkoMTYpLTQwIDk0OSB5KDE3KS00MAox MDA2IHkoMTgpLTQwIDEwNjIgeSgxOSktNDAgMTExOSB5KDIwKS00MCAxMTc1IHkoMjEpLTQw CjEyMzIgeSgyMiktNDAgMTI4OCB5KDIzKS00MCAxMzQ1IHkoMjQpLTQwIDE0MDEgeSgyNSkt NDAKMTQ1NyB5KDI2KS00MCAxNTE0IHkoMjcpLTQwIDE1NzAgeSgyOCktNDAgMTYyNyB5KDI5 KS00MAoxNjgzIHkoMzApLTQwIDE3NDAgeSgzMSktNDAgMTc5NiB5KDMyKS00MCAxODUzIHko MzMpLTQwCjE5MDkgeSgzNCktNDAgMTk2NiB5KDM1KS00MCAyMDIyIHkoMzYpLTQwIDIwNzgg eSgzNyktNDAKMjEzNSB5KDM4KS00MCAyMTkxIHkoMzkpLTQwIDIyNDggeSg0MCktNDAgMjMw NCB5KDQxKS00MAoyMzYxIHkoNDIpLTQwIDI0MTcgeSg0MyktNDAgMjQ3NCB5KDQ0KS00MCAy NTMwIHkoNDUpLTQwCjI1ODcgeSg0NiktNDAgMjY0MyB5KDQ3KS00MCAyNjk5IHkoNDgpcCBl b3AKJSVQYWdlOiAxMCAxMQoxMCAxMCBib3AgNzUgLTEwMCBhIEZsKDEwKTc1IDQ5IHkgRmko TVBJKXAgMTYwIDQ5IDE0CjIgdiAxNiB3KElTUClsKEEpLTUgYihXTilwIDMzOSA0OSBWIDE3 IHcoTVVMKWwoVElQTEUpcAo1NzMgNDkgViAxNSB3KElOREVQRU5ERU5UKXAgOTA0IDQ5IFYg MTcgdyhOT05NUElcKGNvdW50LCkxNgpiKGEpbyhycmEpbyh5KXAgMTM0NyA0OSBWIDE1IHco b2YpcCAxMzk5IDQ5IFYgMTYgdyhjb21tYW5kKXAKMTYwMCA0OSBWIDE1IHcobGluZXMsKTc1 IDEwNiB5KGEpbyhycmEpbyh5KXAgMTczIDEwNgpWIDE1IHcob2YpcCAyMjUgMTA2IFYgMTYg dyhtaW5wKW8ocm8pcShjcywpZShhKW8ocnJhKW8oeSlwCjUzMyAxMDYgViAxNSB3KG9mKXAg NTg1IDEwNiBWIDE2IHcobWF4cClvKHJvKXEoY3MsKWcoYSlvKHJyYSlvKHkpcAo5MDIgMTA2 IFYgMTQgdyhvZilwIDk1MyAxMDYgViAxNiB3KGluZm8sKWgocm8pcShvdCwpZihjb21tLClm KGdyb3VwLClpCihhKW8ocnJhKW8oeSlwIDE1NDggMTA2IFYgMTUgdyhvZilwIDE2MDAgMTA2 IFYgMTYgdyhlcnJjbylxKGRlcywpNzUKMTYyIHkocmVxdWVzdFwpKTExNyAyMzkgeSBGZyhJ TikxNTUgYiBGaShjb3VudCk0ODIgYgpGZyhOdW0pbyhiKXEoZXIpMTIgYihvZilnKGNvbW1h bmQpZShsaW5lcylqKFwoaW4pbyh0ZWdlciwpZyhzaXplKWcob2YpZgooZWFjKW8oaCloKG9m KWYodGhlKTkwNSAyOTYgeShmb2xsbylvKHdpbmcpZihhcnJhKW8oeXNcKSkxMTcKMzcxIHko SU4pMTU1IGIgRmkoYSlvKHJyYSlvKHkpcCA0MTYgMzcxIFYgMTUgdyhvZilwIDQ2OAozNzEg ViAxNiB3KGNvbW1hbmQpcCA2NjkgMzcxIFYgMTUgdyhsaW5lcykxNDIgYiBGZyhleGVjdXRh YmxlKTEyCmIoYW5kKWYoYXJndW1lbilvKHRzKWYoXChhcnJhKW8oeSlnKG9mKWcoc3RyaW5n cywpaChzaWduaVwwMTRjYW4pbyh0KQo5MDUgNDI3IHkob25seSlpKGF0KWgocm8pcShvdFwp KTExNyA1MDMgeShJTikxNTUgYiBGaShhKW8ocnJhKW8oeSlwCjQxNiA1MDMgViAxNSB3KG9m KXAgNDY4IDUwMyBWIDE2IHcobWlucClvKHJvKXEoY3MpMjU0CmIgRmcobWluaW0pbih1bSk5 IGIobilvKHVtKW8oYilxKGVyKWsob2YpZihwcm8pcShjZXNzZXMpayhmb3IpZChlYWMpbwoo aCloKGNvbW1hbmQpYyhsaW5lKTkwNSA1NTkgeSh0bylrKHN0YXJ0KWcoXChhcnJhKW8oeSln KG9mKWcoaW4pbwoodGVnZXIsKWYoc2lnbmlcMDE0Y2FuKW8odCloKG9ubHkpZihhdCloKHJv KXEob3RcKSkxMTcKNjM0IHkoSU4pMTU1IGIgRmkoYSlvKHJyYSlvKHkpcCA0MTYgNjM0IFYg MTUgdyhvZilwIDQ2OAo2MzQgViAxNiB3KG1heHApbyhybylxKGNzKTI0NSBiIEZnKG1heGlt KW4odW0pOCBiKG4pbyh1bSlvKGIpcShlcilrKG9mKQpmKHBybylxKGNlc3NlcylrKGZvcilk KGVhYylvKGgpZyhjb21tYW5kKWQobGluZSk5MDUgNjkxCnkodG8pMTQgYihzdGFydClnKFwo YXJyYSlvKHkpZyhvZilnKGluKW8odGVnZXIsKWYoc2lnbmlcMDE0Y2FuKW8odCloCihvbmx5 KWYoYXQpaChybylxKG90XCkpMTE3IDc2NiB5KElOKTE1NSBiIEZpKGEpbyhycmEpbyh5KXAK NDE2IDc2NiBWIDE1IHcob2YpcCA0NjggNzY2IFYgMTYgdyhpbmZvKTM1MyBiIEZnKHN0cmlu Z3MpMTcKYih0ZWxsaW5nKWUodGhlKWkocnVuKW8odGltZSllKHN5c3RlbSlpKHdoZXJlKWco YW5kKWYoaG8pbyh3KWcodG8pOTA1CjgyMiB5KHN0YXJ0KTIxIGIocHJvKXEoY2Vzc2VzKWko XChhcnJhKW8oeSllKG9mKWUoc3RyaW5ncywpawooc2lnbmlcMDE0Y2FuKW8odCljKG9ubHkp aChhdCk5MDUgODc5IHkocm8pcShvdFwpKTExNwo5NTQgeShJTikxNTUgYiBGaShybylxKG90 KTUwOCBiIEZnKHJhbmspMTMgYihvZilmKHBybylxKGNlc3MpaihpbilkCih3aGljKW8oaClo KHByZXZpb3VzKWcoYXJndW1lbilvKHRzKWYoYXJlKWkodiluKGFsaWQpOTA1CjEwMTAgeShc KGluKW8odGVnZXJcKSkxMTcgMTA4NSB5KElOKTE1NSBiIEZpKGNvbW0pNDcwCmIgRmcoY29t bSlvKHVuaWNhdG9yKTcgYihvZilpKGdyb3VwKWcob2YpZyhzcGEpbyh3bmluZylnKHBybylx KGNlc3NlcylqCihcKGhhbmRsZVwpKTExNyAxMTYxIHkoT1VUKTEwOCBiIEZpKGdyb3VwKTQ3 OSBiIEZnKGdyb3VwKTE0CmIob2YpZihzcGEpbyh3bmVkKWkocHJvKXEoY2Vzc2VzKWgoXCho YW5kbGVcKSkxMTcgMTIzNgp5KE9VVCkxMDggYiBGaShhKW8ocnJhKW8oeSlwIDQxNiAxMjM2 IFYgMTUgdyhvZilwIDQ2OAoxMjM2IFYgMTYgdyhlcnJjbylxKGRlcykyNjggYiBGZyhvbmUp MTQgYihlcnJvciloKGNvKXEoZGUpZyhwKXEoZXIpZwoocHJvKXEoY2VzcylnKFwoYXJyYSlv KHkpZihvZilmKGluKW8odGVnZXJcKSkxMTcgMTMxMQp5KE9VVCkxMDggYiBGaShyZXF1ZXN0 KTQ1MiBiIEZnKHJlcXVlc3QpMTYgYihvYilyKGplY3QpZShcKGhhbmRsZVwpKTc1CjE0MzUg eSBGZihpbnQpMjMgYihNUEkpcCAyNDUgMTQzNSAxNSAyIHYgMTcgdyhJc3Bhd24pcAo0MDYg MTQzNSBWIDE3IHcobXVsdGlwbGUpcCA2MTUgMTQzNSBWIDE2IHcoaW5kZXBlbmRlbnQpcAo4 OTUgMTQzNSBWIDE1IHcobm9ubXBpXChpbnQqKWcoY291bnQsKTM5MyAxNDkyIHkoY2hhciop ZyhhcnJheSlwCjY1OSAxNDkyIFYgMTcgdyhvZilwIDcyNCAxNDkyIFYgMTcgdyhjb21tYW5k KXAgOTA5IDE0OTIKViAxNiB3KGxpbmVzW10sKWcoaW50KilnKGFycmF5KXAgMTM3OSAxNDky IFYgMTcgdyhvZilwCjE0NDQgMTQ5MiBWIDE2IHcobWlucHJvY3MsKTM5MyAxNTQ4IHkoaW50 KilnKGFycmF5KXAKNjM1IDE1NDggViAxNyB3KG9mKXAgNzAwIDE1NDggViAxNyB3KG1heHBy b2NzLClnKGNoYXIqKWcoYXJyYXkpcAoxMjE5IDE1NDggViAxNiB3KG9mKXAgMTI4MyAxNTQ4 IFYgMTcgdyhpbmZvW10sKWcoaW50KWcocm9vdCwpMzkzCjE2MDUgeShNUEkpcCA0NjggMTYw NSBWIDE3IHcoQ29tbSlnKGNvbW0sKWcoTVBJKXAgODE5CjE2MDUgViAxNyB3KEdyb3VwKiln KGdyb3VwLClnKGludCopaChhcnJheSlwIDE0MTAgMTYwNQpWIDE2IHcob2YpcCAxNDc0IDE2 MDUgViAxNyB3KGVycmNvZGVzLCkzOTMgMTY2MSB5KE1QSSlwCjQ2OCAxNjYxIFYgMTcgdyhS ZXF1ZXN0KilmKHJlcXVlc3RcKSk3NSAxNzQ4IHkoTVBJKXAKMTUwIDE3NDggViAxNyB3KFNQ QVdOKXAgMjg3IDE3NDggViAxNiB3KE1VTFRJUExFKXAgNDk1CjE3NDggViAxNiB3KElOREVQ RU5ERU5UKXAgNzc1IDE3NDggViAxNiB3KE5PTk1QSVwoQ09VTlQsKWYoQVJSQVkpcAoxMjQ1 IDE3NDggViAxNyB3KE9GKXAgMTMxMCAxNzQ4IFYgMTcgdyhDT01NQU5EKXAgMTQ5NQoxNzQ4 IFYgMTYgdyhMSU5FUywpMzkzIDE4MDQgeShBUlJBWSlwIDUxNiAxODA0IFYgMTcgdyhPRilw CjU4MSAxODA0IFYgMTcgdyhNSU5QUk9DUywpZyhBUlJBWSlwIDk1NiAxODA0IFYgMTcgdyhP RilwCjEwMjEgMTgwNCBWIDE3IHcoTUFYUFJPQ1MsKWcoQVJSQVkpcCAxMzk2IDE4MDQgViAx NyB3KE9GKXAKMTQ2MSAxODA0IFYgMTcgdyhJTkZPLCloKFJPT1QsKTM5MyAxODYwIHkoQ09N TSwpZyhHUk9VUCwpZyhBUlJBWSlwCjgyNiAxODYwIFYgMTcgdyhPRilwIDg5MSAxODYwIFYg MTcgdyhFUlJDT0RFUywpZihSRVFVRVNULCloKElFUlJcKSkxNzAKMTkxNyB5KENIQVJBQ1RF UipcKCpcKSlnKEFSUkFZKXAgNjI4IDE5MTcgViAxNiB3KE9GKXAKNjkyIDE5MTcgViAxNyB3 KENPTU1BTkQpcCA4NzcgMTkxNyBWIDE2IHcoTElORVNcKCpcKSwpZyhBUlJBWSlwCjEyNTIg MTkxNyBWIDE2IHcoT0YpcCAxMzE2IDE5MTcgViAxNyB3KElORk8pMTcwIDE5NzMKeShJTlRF R0VSKWcoQ09VTlQsKWcoQVJSQVkpcCA2NTEgMTk3MyBWIDE3IHcoT0YpcCA3MTYKMTk3MyBW IDE3IHcoTUlOUFJPQ1MsKWYoQVJSQVkpcCAxMDkxIDE5NzMgViAxNyB3KE9GKXAKMTE1NiAx OTczIFYgMTcgdyhNQVhQUk9DUywpZyhST09ULClpKENPTU0sKWYoR1JPVVAsKTE3MAoyMDMw IHkoQVJSQVkpcCAyOTMgMjAzMCBWIDE3IHcoT0YpcCAzNTggMjAzMCBWIDE3IHcoRVJSQ09E RVNcKCpcKSwpZgooUkVRVUVTVCwpaChJRVJSKTE2NiAyMTE2IHkgRmwoRilsKG9yKWMoYWxs KWkob2YpZSh0aGUpaChyb3V0aW5lcylnKGluKQpoKHRoaXMpZihzZWN0aW9uLCloKHRoZSlm KGFyZ3VtZW4pbyh0cylmKGFyZSlnKGlkZW4pbyh0aWNhbClqKHRvKWQKKHRoZWlyKWkoYmxv KXEoYylvKGtpbmcpNzUgMjE3MyB5KGNvdW4pbyh0ZXJwYXJ0cykxNApiKGV4Y2VwdCloKHRo YXQpZyh0aGVyZSlnKGlzKWcoYW4pZyhhZGRpdGlvbmFsKWggRmkocmVxdWVzdCloCkZsKGFy Z3VtZW4pbyh0LilpKFRoZSljKHJlcXVlc3QpZyhjYW4pZyhiKXEoZSlnKHVzZWQpNzUKMjIy OSB5KGFzKWMoaW5wdXQpaShmb3IpZSBGaShNUEkpcCAzOTYgMjIyOSAxNCAyIHYgMTYKdyhX KWwoQUlUKXAgRmwoLClnIEZpKE1QSSlwIDYzMSAyMjI5IFYgMTYgdyhURVNUKXAgRmwoLCln KGV0Yy4pMTkKYihJbikxMiBiKGFsbCloKGNhc2VzLClmKHRoZSlmKG91dHB1dCloIEZpKGdy b3VwKWYgRmwob3IpZwpGaShpbnRlcmNvbW0pcCBGbCgsKWcoYW5kKWgodGhlKTc1IDIyODYg eSBGaShhKW8ocnJhKW8oeSlwCjE3MyAyMjg2IFYgMTUgdyhvZilwIDIyNSAyMjg2IFYgMTYg dyhlcnJjbylxKGRlcylpIEZsKGFyZSlnKHVwKXEoZGF0ZWQpCmgoYXN5bmMpbyhocm9ub3Vz bHkpZihhbmQpaChhcmUpZihub3QpZih2KW0oYWxpZClqKHVuKW8odGlsKWYodGhlKWcKKHJl cXVlc3QpZihjb21wbGV0ZXMuKTE2NiAyMzQyIHkoVGhlKWcoc3RhdHVzKWYob2IpcyhqZWN0 KWgocmV0dXJuZWQpZwooYilvKHkpZyBGaShNUEkpcCA4NTggMjM0MiBWIDE2IHcoVylsKEFJ VClnIEZsKGNvbilvKHRhaW5zKWcoYW4pZwooZXJyb3IpZihjbylxKGRlKWgoaW5kaWNhdGlu ZylpKHdoZXRoZXIpZShvcik3NSAyMzk5Cnkobm90KWgodGhlKWgoc3BhKW8od24pZih3KW8o YXMpZyhzdWNjZXNzZnVsLikyMiBiKFRoZSkxNQpiKGZhaWx1cmUpaShvcillKHN1Y2Nlc3Mp aChvZilmKGluZGl2aWR1YWwpayhwcm8pcShjZXNzZXMpZChpcylnCihpbmRpY2F0ZWQpaChp bik3NSAyNDU1IHkgRmkoYSlvKHJyYSlvKHkpcCAxNzMgMjQ1NSBWCjE1IHcob2YpcCAyMjUg MjQ1NSBWIDE2IHcoZXJyY28pcShkZXMpcCBGbCguKWsoVGhlKTE2CmIoZXJyb3JjbylxKGRl KWcoY2FuKWYoYilxKGUpaShyZXRyaWV2KW8oZWQpZihhcylmKHRoZSloCkZpKE1QSSlwIDEy ODAgMjQ1NSBWIDE1IHcoRVJST1IpaSBGbChlbGVtZW4pbyh0KWUob2YpZih0aGUpaChzdGF0 dXMpNzUKMjUxMSB5KG9iKXMoamVjdClmKHJldHVybmVkKWcoYilvKHkpZyBGaShNUEkpcCA1 NDUgMjUxMQpWIDE2IHcoVylsKEFJVClwIEZsKCwpZyBGaShNUEkpcCA3ODQgMjUxMSBWIDE2 IHcoVEVTVClwCkZsKCwpZyhldGMuKTE2NiAyNTY4IHkoQWxsKTIyIGIob2YpZih0aGUpZyhu b24tYmxvKXEoYylvKGtpbmcpaAoocm91dGluZXMpZyhpbilmKHRoaXMpaChzZWN0aW9uKWYo YXJlKWcoY29sbGVjdGl2KW8oZSloKFwobylvKHYpbyhlcillCih0aGUpaChzYW1lKWcoc2V0 KWYob2YpNzUgMjYyNCB5KHBybylxKGNlc3NlcyljKGFzKWUodGhlaXIpaShibG8pcShjKW8K KGtpbmcpaChjb3VuKW8odGVycGFydHNcKSwpYyhhbmQpaihvYilxKGV5KWYodGhlKWgodGhl KWYoZm9sbG8pbyh3aW5nKWgKKHJ1bGVzOikxOTY3IDQ2IHkgRmUoMSkxOTY3IDEwMyB5KDIp MTk2NyAxNTkgeSgzKTE5NjcKMjE1IHkoNCkxOTY3IDI3MiB5KDUpMTk2NyAzMjggeSg2KTE5 NjcgMzg1IHkoNykxOTY3IDQ0MQp5KDgpMTk2NyA0OTggeSg5KTE5NTkgNTU0IHkoMTApMTk1 OSA2MTEgeSgxMSkxOTU5IDY2Nwp5KDEyKTE5NTkgNzI0IHkoMTMpMTk1OSA3ODAgeSgxNCkx OTU5IDgzNiB5KDE1KTE5NTkgODkzCnkoMTYpMTk1OSA5NDkgeSgxNykxOTU5IDEwMDYgeSgx OCkxOTU5IDEwNjIgeSgxOSkxOTU5CjExMTkgeSgyMCkxOTU5IDExNzUgeSgyMSkxOTU5IDEy MzIgeSgyMikxOTU5IDEyODggeSgyMykxOTU5CjEzNDUgeSgyNCkxOTU5IDE0MDEgeSgyNSkx OTU5IDE0NTcgeSgyNikxOTU5IDE1MTQgeSgyNykxOTU5CjE1NzAgeSgyOCkxOTU5IDE2Mjcg eSgyOSkxOTU5IDE2ODMgeSgzMCkxOTU5IDE3NDAgeSgzMSkxOTU5CjE3OTYgeSgzMikxOTU5 IDE4NTMgeSgzMykxOTU5IDE5MDkgeSgzNCkxOTU5IDE5NjYgeSgzNSkxOTU5CjIwMjIgeSgz NikxOTU5IDIwNzggeSgzNykxOTU5IDIxMzUgeSgzOCkxOTU5IDIxOTEgeSgzOSkxOTU5CjIy NDggeSg0MCkxOTU5IDIzMDQgeSg0MSkxOTU5IDIzNjEgeSg0MikxOTU5IDI0MTcgeSg0Mykx OTU5CjI0NzQgeSg0NCkxOTU5IDI1MzAgeSg0NSkxOTU5IDI1ODcgeSg0NikxOTU5IDI2NDMg eSg0NykxOTU5CjI2OTkgeSg0OClwIGVvcAolJVBhZ2U6IDExIDEyCjExIDExIGJvcCAxODMw IC0xMDAgYSBGbCgxMSkxNDMgNDkgeSBGYihcMDE3KTIzIGIgRmwoQSljKG5vbi1ibG8pcShj KW8KKGtpbmcpaShjb2xsZWN0aXYpbyhlKWcocm91dGluZSlmKG0pbyh1c3QpZShiKXEoZSlp KGNhbGxlZCloKGluKWYoZXYpbwooZXJ5KWYocHJvKXEoY2VzcyloKG9mKWUodGhlKWkoYXNz bylxKGNpYXRlZCkxODkgMTA2CnkoY29tbSlvKHVuaWNhdG9yKTE0IGIoYilxKGVmb3JlKWkg RmkoTVBJKXAgNzA3IDEwNiAxNAoyIHYgMTYgdyhGSU5BTElaRSllIEZsKGlzKWkoY2FsbGVk LikxNDMgMjAwIHkgRmIoXDAxNykyMwpiIEZsKFRoZSkxMyBiKG9wKXEoZXJhdGlvbilnKGlz KWgobG8pcShjYWwuKTIwIGIoQSkxNApiKG5vbi1ibG8pcShjKW8oa2luZyloKGNvbGxlY3Rp dilvKGUpZyhvcClxKGVyYXRpb24pZSh3aWxsKWkocmV0dXJuKWUKKGV2KW8oZW4paChpZilm KGl0KWgoaXMpZihub3QpMTg5IDI1NiB5KGNhbGxlZClqKGltbWVkaWF0ZWx5KWgob24pZQoo YWxsKWkocHJvKXEoY2Vzc2VzLikxNDMgMzUwIHkgRmIoXDAxNykyMyBiIEZsKE11bHRpcGxl KTE3CmIob3V0c3RhbmRpbmcpZihub24tYmxvKXEoYylvKGtpbmcpaChvcClxKGVyYXRpb25z KWUob2YpaCh0aGUpZihzYW1lKWcKKHQpbyh5cClxKGUpaChvbilmKHRoZSloKHNhbWUpZihj b21tKW8odS0pMTg5IDQwNiB5KG5pY2F0b3IpZihhcmUpZwooYWxsbylvKHcpbyhlZCwpaChh bmQpZyhtYXRjKW8oaClmKHVwKWgoaW4pZyh0aGUpZihvcmRlciloKHRoZXkpZih3KW8KKGVy ZSloKGNhbGxlZCloKFwoc2luY2UpZih0aGVyZSlnKGlzKWcobm8pZiBGaSh0YWcpMTg5CjQ2 MyB5IEZsKGFyZ3VtZW4pbyh0XCkuKTIwIGIoT3ApcShlcmF0aW9ucylkKG9mKWUoZGlcMDEz ZXJlbilvKHQpaSh0KW8KKHlwKXEoZXMpZShhbmQvb3IpaChvbilnKGRpXDAxM2VyZW4pbyh0 KWcoY29tbSlvKHVuaWNhdG9ycylmKG1hKW8oeSlnCihiKXEoZSkxODkgNTE5IHkoaW4pbyh0 ZXJsZWEpbyh2KW8oZWQuKTE0MyA2MTMgeSBGYihcMDE3KTIzCmIgRmwoUmVxdWVzdHMpMTcg YihjYW5ub3QpZyhiKXEoZSloKGNhbmNlbGxlZC4pMjggYihBbikxOApiKGF0dGVtcHQpZSh0 byloKGNhbmNlbCloKGEpZihjb2xsZWN0aXYpbyhlKWkocmVxdWVzdCllKHdpbGwpaShyZXR1 cm4pCjE4OSA2NjkgeShhbiljKGVycm9yKWYob2YpaChlcnJvcilnKGNsYXNzKWcgRmkoTVBJ KXAKNzE4IDY2OSBWIDE2IHcoRVJSKXAgODE5IDY2OSBWIDE3IHcoUkVRVUVTVClwIEZsKC4p NzUKNzkxIHkgRmsoMi4zLjMpNDkgYihQcm8pcShjZXNzKTE1IGIoVXRpbGl0aWVzKTc1IDg3 Nwp5IEZsKEl0KWcoaXMpaChub3QpZShwKXEob3NzaWJsZSlqKHRvKWQoY29tbSlvKHVuaWNh dGUsKWgodXNpbmcpaAoobWVzc2FnZXMsKWUod2l0aClpKGEpZShwcm8pcShjZXNzKWkocmVw cmVzZW4pbyh0ZWQpZihiKW8oeSlnKGEpZwooXChncm91cCwpNzUgOTMzIHkocmFua1wpKWco cGFpciwpaChleGNlcHQpaCh0aHJvdWdoKWUoYSlnKGNvbW0pbwoodW5pY2F0b3IpaChhc3Nv KXEoY2lhdGVkKWcod2l0aClnKHRoYXQpZihncm91cC4pMjIgYihGKWwob3IpMTUKYihpbnN0 YW5jZSwpaChpdClnKGlzKTc1IDk5MCB5KG5vdClmKHApcShvc3NpYmxlKWkodG8pZChzZW5k KWkoYSlmCihtZXNzYWdlKWcodG8pZihhbiloKE1QSSloKHBybylxKGNlc3MpZihzcGEpbyh3 bmVkKWcoYilvKHkpNzUKMTA0NiB5IEZpKE1QSSlwIDE2MCAxMDQ2IFYgMTYgdyhTUClsKEEp LTUgYihXTilwIDMyNgoxMDQ2IFYgMTcgdyhJTkRFUEVOREVOVClwIDY1OSAxMDQ2IFYgMTcg dyhNUEkpcCBGbCgsKTE0CmIoc2luY2UpaShubylmKGNvbW0pbyh1bmljYXRvcilnKGlzKWgo YSlvKHYpbShhaWxhYmxlLikxNjYKMTEwMyB5KEhvKW8odylvKGV2KW8oZXIsKWUoTVBJKWgo cHJvKW8odmlkZXMpaChzaW1wbGUpZyhtZWMpbyhoYW5pc21zKWcKKHRvKWYobWFuYWdlKWYo c3VjKW8oaClpKHBybylxKGNlc3Nlcy4pNzUgMTE1OSB5IEZpKE1QSSlwCjE2MCAxMTU5IFYg MTYgdyhQUk9DRVNTKXAgMzczIDExNTkgViAxOCB3KFNJR05BTCljIEZsKHNlbmRzKWcoYSln CihzaWduYWwpaCh0byllKGEpaChwcm8pcShjZXNzLClnKGFuZClnIEZpKE1QSSlwIDEyNTkK MTE1OSBWIDE2IHcoUFJPQ0VTUylwIDE0NzIgMTE1OSBWIDE4IHcoTU9OSVRPUilnIEZsKHBy bylvKHZpZGVzKTc1CjEyMTYgeShhKWoobWVjKW8oaGFuaXNtKWgodG8pZShkZXRlY3QpaSh3 aGVuKWYoYSlnKHBybylxKGNlc3MpaChkaWVzKWcKKG9yKWYoYylvKGhhbmdlcylnKHN0YXRl Lik3NSAxMzY3IHkgRmkoTVBJKXAgMTYwIDEzNjcKViAxNiB3KFBST0NFU1MpcCAzNzMgMTM2 NyBWIDE4IHcoU0lHTkFMXChncm91cCwpZihyYW5rLCloKHNpZ25hbFwpKTExNwoxNDQ0IHkg RmcoSU4pMTU1IGIgRmkoZ3JvdXApNDc5IGIgRmcoZ3JvdXApMTQgYihjb24pbyh0YWluaW5n KWYocHJvKXEKKGNlc3MpaSh0bylmKHNpZ25hbClmKFwoaGFuZGxlXCkpMTE3IDE1MTkgeShJ TikxNTUgYgpGaShyYW5rKTUwNCBiIEZnKHJhbmspMTQgYihvZilmKHBybylxKGNlc3Mpaih0 byllKHNpZ25hbClmKFwoaW4pbwoodGVnZXJcKSkxMTcgMTU5NCB5KElOKTE1NSBiIEZpKHNp Z25hbCk0ODAgYiBGZyhzaWduYWwpMTMKYih0KW8oeXApcShlKWgoXChpbilvKHRlZ2VyXCkp NzUgMTcxOSB5IEZmKGludCkyMyBiKE1QSSlwCjI0NSAxNzE5IDE1IDIgdiAxNyB3KFByb2Nl c3MpcCA0MzAgMTcxOSBWIDE2IHcoc2lnbmFsXChNUEkpcAo2ODYgMTcxOSBWIDE2IHcoR3Jv dXApZyhncm91cCwpZyhpbnQpaChyYW5rLClmKGludCloKHNpZ25hbFwpKTc1CjE4MDUgeShN UEkpcCAxNTAgMTgwNSBWIDE3IHcoUFJPQ0VTUylwIDMzNSAxODA1IFYgMTYKdyhTSUdOQUxc KEdST1VQLCllKFJBTkssKWgoU0lHTkFMLClnKElFUlJcKSkxNzAgMTg2MQp5KElOVEVHRVIp ZyhHUk9VUCwpZyhSQU5LLCloKFNJR05BTCwpZShJRVJSKTE2NiAxOTQ4CnkgRmkoTVBJKXAg MjUxIDE5NDggMTQgMiB2IDE2IHcoUFJPQ0VTUylwIDQ2NCAxOTQ4IFYKMTggdyhTSUdOQUwp ZyBGbChzZW5kcylnKGEpZyhzaWduYWwpaCh0byllKGEpaChwcm8pcShjZXNzKWcocmVwcmVz ZW4pbwoodGVkKWcoYilvKHkpZyhhKWcoXChncm91cCwpaChyYW5rXCkpNzUgMjAwNCB5KHBh aXIuKTMwCmIoV2hlcmUpMTggYihQT1NJWClpKHNpZ25hbHMpZihhcmUpZihzdXBwKXEob3J0 ZWQsKWgKRmkoc2lnbmFsKWcgRmwoaXMpZihhKWcoc2lnbmFsKWkoZGVcMDE0bmVkKWYoYilv KHkpZyhQT1NJWC4pZyhJdClmKGlzKWgKKHRoZSk3NSAyMDYxIHkocmVzcClxKG9uc2liaWxp dClvKHkpZihvZilkKGFuKWgoaW1wbGVtZW4pbyh0YXRpb24pZyh0bykKZih0cmFuc2xhdGUp aChiKXEoZXQpbyh3KW8oZWVuKWYoc2lnbmFsczspaShpbilmKG90aGVyKWYodylvKG9yZHMs KWcoYSkKZyBGZihTSUdJTlQpNzUgMjExNyB5IEZsKHRoYXQpZShoYXMpaCh2KW0oYWx1ZSlp IEZmKDMpZQpGbChvbilnKHN5c3RlbSlnKEEpZyhtKW8odXN0KWYoYilxKGUpaShkZWxpdilv KGVyZWQpaChhcyllKGEpZwpGZihTSUdJTlQpZiBGbChvbiloKHN5c3RlbSlnKEIsKWcoZXYp byhlbilnKGlmKWgoc3lzdGVtKWYoQik3NQoyMTc0IHkodXNlcylqKHRoZSlmKHYpbShhbHVl KWkgRmYoNSlmIEZsKGZvcilmIEZmKFNJR0lOVClwCkZsKC4pZihJZilpKHRoZXJlKWYoaXMp aChubylnKGNvcnJlc3ApcShvbmRpbmcpZyhzaWduYWwsKWgodGhlKWUKKHJlc3VsdHMpaChh cmUpZih1bmRlXDAxNG5lZCwpNzUgMjIzMCB5KHRob3VnaClnKG5vdCloKGVycm9uZW91cywp ZihzbykKZyh0aGF0KWcodGhlKWgoc2lnbmFsKWcoaXMpaChzaWxlbilvKHRseSlnKGNvbilv KHYpbyhlcnRlZCllKHRvKWcoYSlnCihzaWduYWwpaSh3aXRoKWYobWVhbmluZylnKG9uKTc1 IDIyODcgeSh0aGUpZSh0YXJnZXQpZihtYWMpbyhoaW5lLClpCihvcilmKGlzKWcoZHJvcHAp cShlZCloKGNvbXBsZXRlbHkpbCguKTE2NiAyMzQzIHkoQW4pZihNUEkpaChpbXBsZW1lbilv Cih0YXRpb24pZyhtKW8odXN0KWYoYWxzbylnKHN1cHApcShvcnQpZyh0aGUpZyhNUEktZGVc MDE0bmVkKWkoc2lnbmFsKWYKKHQpbyh5cClxKGUpNzUgMjQwMCB5IEZpKE1QSSlwIDE2MCAy NDAwIFYgMTYgdyhTSUdOQUwpcAozMzEgMjQwMCBWIDE2IHcoS0lMTClwIEZsKC4pMjEgYihT ZW5kaW5nKWoodGhpcylmKFxcc2lnbmFsIilmKHdpbGwpaQoocmVsaWFibHkpZyhraWxsKWgo YSlkKHBybylxKGNlc3MpZyhhbmQpaChhdHRlbXB0KWUodG8paChkbyk3NQoyNDU2IHkodGhl KWYobmVjZXNzYXJ5KWgoc3lzdGVtLWxldilvKGVsKWcoY2xlYW4pbyh1cC4pMzkKYihJbiky MiBiKG9yZGVyKWUodG8paChhbGxvKW8odylnKGFwcHJvcHJpYXRlKWcoY2xlYW4pbyh1cClp KHRvKWQobylxCihjY3VyLClqKGFuKTc1IDI1MTIgeShpbXBsZW1lbilvKHRhdGlvbilkKGNv dWxkLClnKGZvcillKGluc3RhbmNlLClpCihzZW5kKWYoYSlmKGNhdGMpbyhoYWJsZSloKHNp Z25hbClnKFwoZS5nLiwpZihTSUdJTlRcKSloKGZvbGxvKW8odylvCihlZClnKGIpbyh5KWYo YSk3NSAyNTY5IHkobm9uY2F0YylvKGhhYmxlKWkoc2lnbmFsKWcoXChTSUdLSUxMXCkpZyhp ZilnCih0aGUpZihwcm8pcShjZXNzKWcoaGFkKWgobm90KWUoZXhpdGVkKWkoYWZ0ZXIpZihh KWYoc2hvcnQpaCh0aW1lKWcoaW4pCm8odGVydiltKGFsLik3NSAyNjI1IHkoTVBJKWcocmVx dWlyZXMpaChvbmx5KWYodGhhdClmKHRoZSloKHBybylxKGNlc3MpCmgoYilxKGUpZihyZWxp YWJseSlpKGtpbGxlZC4pMzMgYihBcHBsaWNhdGlvbnMpMjEgYihzaG91bGQpZihub3QpZgoo cmVseSlnKG9uKWcoYSk3NSAyNjgyIHkoc3ApcShlY2lcMDE0YyllKGltcGxlbWVuKW8odGF0 aW9uLiktMzIKNDYgeSBGZSgxKS0zMiAxMDMgeSgyKS0zMiAxNTkgeSgzKS0zMiAyMTUgeSg0 KS0zMiAyNzIKeSg1KS0zMiAzMjggeSg2KS0zMiAzODUgeSg3KS0zMiA0NDEgeSg4KS0zMiA0 OTggeSg5KS00MAo1NTQgeSgxMCktNDAgNjExIHkoMTEpLTQwIDY2NyB5KDEyKS00MCA3MjQg eSgxMyktNDAgNzgwCnkoMTQpLTQwIDgzNiB5KDE1KS00MCA4OTMgeSgxNiktNDAgOTQ5IHko MTcpLTQwIDEwMDYKeSgxOCktNDAgMTA2MiB5KDE5KS00MCAxMTE5IHkoMjApLTQwIDExNzUg eSgyMSktNDAgMTIzMgp5KDIyKS00MCAxMjg4IHkoMjMpLTQwIDEzNDUgeSgyNCktNDAgMTQw MSB5KDI1KS00MCAxNDU3CnkoMjYpLTQwIDE1MTQgeSgyNyktNDAgMTU3MCB5KDI4KS00MCAx NjI3IHkoMjkpLTQwIDE2ODMKeSgzMCktNDAgMTc0MCB5KDMxKS00MCAxNzk2IHkoMzIpLTQw IDE4NTMgeSgzMyktNDAgMTkwOQp5KDM0KS00MCAxOTY2IHkoMzUpLTQwIDIwMjIgeSgzNikt NDAgMjA3OCB5KDM3KS00MCAyMTM1CnkoMzgpLTQwIDIxOTEgeSgzOSktNDAgMjI0OCB5KDQw KS00MCAyMzA0IHkoNDEpLTQwIDIzNjEKeSg0MiktNDAgMjQxNyB5KDQzKS00MCAyNDc0IHko NDQpLTQwIDI1MzAgeSg0NSktNDAgMjU4Nwp5KDQ2KS00MCAyNjQzIHkoNDcpLTQwIDI2OTkg eSg0OClwIGVvcAolJVBhZ2U6IDEyIDEzCjEyIDEyIGJvcCA3NSAtMTAwIGEgRmwoMTIpMTY2 IDQ5IHkoU2lnbmFscykxNSBiKGFyZSllKGRlbGl2KW8oZXJlZClpCihhc3luYylvKGhyb25v dXNseSlsKC4pMjAgYihUaGF0KTEzIGIoaXMsKWgod2hlbiloKHRoZSllCkZpKE1QSSlwIDEz NjIgNDkgMTQgMiB2IDE2IHcoU0lHTkFMKWggRmwoZnVuY3Rpb24pZyhyZXR1cm5zLCk3NQox MDYgeSh0aGUpayhzaWduYWwpZyhtYSlvKHkpZyhub3QpZih5KW8oZXQpZyhoYSlvKHYpbyhl KWgoYilxKGVlbiloCihkZWxpdilvKGVyZWQuKTI5IGIoSWYpMTggYihhKWcoc2lnbmFsKWco aXMpZyh1bmRlbGl2KW8oZXJhYmxlKWoKKFwoZS5nLiwpYyh0aGUpaChwcm8pcShjZXNzKTc1 IDE2MiB5KGhhcylkKGFscmVhZHkpaChleGl0ZWRcKSwpZih0aGUpZwooc2lnbmFsKWgoaXMp ZyhzaWxlbilvKHRseSloKGRyb3BwKXEoZWQuKTE2NiAzMDEgeSBGaChEaXNjdXNzaW9uOikz NApiIEZnKEluKTE0IGIobWFuKW8oeSllKGNhc2VzKWsod2hlbillKGEpZyhzaWduYWwpZihp cyloKHVuZGVsaXYpbwooZXJhYmxlLClmKE1QSSloKG1hKW8oeSllKGtubylvKHcpaSh3aGVu KWcodGhlKWgocm91dGluZSk3NQozNTggeShpcylrKGNhbGxlZClmKHRoYXQpaChpdClmKGlz KWcodW5kZWxpdilvKGVyYWJsZSloKFwoZS5nLiwpZyh0aGUpZwoocHJvKXEoY2VzcylpKGhh cylkKGFscmVhZHkpaChiKXEoZWVuKWcobWFyaylvKGVkKWYoZGVhZFwpLikzMwpiKEl0KTE4 IGIodylvKG91bGQpZyhiKXEoZSk3NSA0MTQgeShwKXEob3NzaWJsZSllKHRvKWYocmV0dXJu KWkoYW4pZgooZXJyb3IpZyhpbilmKHRoaXMpaChjYXNlLCloKHRob3VnaCllKGVycm9yKWko cmVwKXEob3J0aW5nKWYodylvKG91bGQpZgooYilxKGUpaCh1bnJlbGlhYmxlLClmKGFuZClo KHcpbyhvdWxkKWUoYilxKGUpNzUgNDcxCnkoYW4payhleGNlcHRpb24paSh0byllKHRoZSlo KHVzdWFsKWcoTVBJKWcocnVsZSlmKHRoYXQpaChhbGwpZShlcnJvcnMpCmooYXJlKWYoZmF0 YWwpZihiKW8oeSlnKGRlZmF1bHQuKTMyIGIoQWx0ZXJuYXRpdilvKGVseSltKCwpMTgKYih3 KW8oZSloKGNvdWxkKTc1IDUyNyB5KHJlcXVpcmUpYyh0aGF0KWYoc2lnbmFscylmKGIpcShl KWkoc2VuKW8odClnCihzeW5jKW8oaHJvbm91c2x5KW0oLCllKGFuZCloKHJldHVybiloKGEp ZihzdGF0dXMpaChpbmRpY2F0aW5nKWQKKHdoZXRoZXIpayhpdClkKHcpbyhhcyloKHN1Y2Nl c3NmdWwpNzUgNTg0IHkob3IpZyhub3QpZyhcKHRob3VnaClnKHRoZSkKaChyZXR1cm4paCh2 KW4oYWx1ZSlkKG9mKWgodGhlKWcoZnVuY3Rpb24pZyh3KW8ob3VsZClnKGIpcShlKWcKRmQo TVBJKXAgMTIwMSA1ODQgMTMgMiB2IDE1IHcoU1VDQ0VTUyllIEZnKHVubGVzcylqKHRoZXJl KWgodylvKGVyZSlmCihhbilmKGVycm9yKTc1IDY0MCB5KGluKWYoYXJndW1lbilvKHRzXCku KTc1IDg3NCB5IEZpKE1QSSlwCjE2MCA4NzQgMTQgMiB2IDE2IHcoR1JPVVApcCAzMjggODc0 IFYgMTggdyhTSUdOQUxcKGdyb3VwLCloKHNpZ25hbFwpKQoxMTcgOTUxIHkgRmcoSU4pMTU1 IGIgRmkoZ3JvdXApNDc5IGIgRmcoZ3JvdXApMTQgYihvZilmKHBybylxKGNlc3NlcylrCih0 byljKHNpZ25hbClnKFwoaGFuZGxlXCkpMTE3IDEwMjYgeShJTikxNTUgYiBGaShzaWduYWwp NDgwCmIgRmcoc2lnbmFsKTEzIGIodClvKHlwKXEoZSloKFwoaW4pbyh0ZWdlclwpKTc1IDEx NTAKeSBGZihpbnQpMjMgYihNUEkpcCAyNDUgMTE1MCAxNSAyIHYgMTcgdyhHcm91cClwIDM4 MgoxMTUwIFYgMTcgdyhzaWduYWxcKE1QSSlwIDYzOSAxMTUwIFYgMTYgdyhHcm91cClnKGdy b3VwLClnKGludClnCihzaWduYWxcKSk3NSAxMjM3IHkoTVBJKXAgMTUwIDEyMzcgViAxNyB3 KEdST1VQKXAgMjg3CjEyMzcgViAxNiB3KFNJR05BTFwoR1JPVVAsKWYoU0lHTkFMLCloKElF UlJcKSkxNzAgMTI5Mwp5KElOVEVHRVIpZyhHUk9VUCwpZyhTSUdOQUwsKWcoSUVSUikxNjYg MTM4MCB5IEZsKFRoZSkxNQpiKGNvbilvKHYpbyhlbmllbmNlKWkoZnVuY3Rpb24pZiBGaShN UEkpcCA3NzEgMTM4MCAxNAoyIHYgMTYgdyhHUk9VUClwIDkzOSAxMzgwIFYgMTggdyhTSUdO QUwpZiBGbChpcyloKGVxdWl2KW0oYWxlbilvKHQpaAoodG8pZChjYWxsaW5nKTc1IDE0MzYg eSBGaShNUEkpcCAxNjAgMTQzNiBWIDE2IHcoUFJPQ0VTUylwCjM3MyAxNDM2IFYgMTggdyhT SUdOQUwpaCBGbChvbmNlKWgoZm9yKWUoZWFjKW8oaClpKHJhbmspZShpbilqKHRoZSllCihn cm91cC4pMTg5IDE1NDMgeSBGaihBKW4oZHZpYyluKGUpZSh0bylpKHVzZXJzLikzOSBiCkZs KFNpZ25hbHMpMTUgYihkbyllKG5vdClnKHBybylvKHZpZGUpaChhbilvKHkpZyhvZilmKHRo ZSlnKHJlbGlhYmlsaXQpCm8oeSlqKG9yKWQoZ3VhcmFuKW8odGVlcylnKG9mKWcocmVndWxh cikxODkgMTU5OSB5KE1QSSlqKGNvbW0pbwoodW5pY2F0aW9uLikyMyBiKEEpMTYgYihoaWdo KWgocXVhbGl0KW8oeSlnKGltcGxlbWVuKW8odGF0aW9uKWcod2lsbCloCihkZWxpdilvKGVy KWYoc2lnbmFscylnKHF1aWMpbyhrbHkpaChhbmQpMTg5IDE2NTUgeShyZWxpYWJseSlsKCwp MjMKYihidXQpZChhcHBsaWNhdGlvbnMpaShzaG91bGQpZihuZXYpbyhlcilmKGRlcClxKGVu ZClpKG9uKWUob3JkZXJpbmcuKQozNSBiKFNpbmNlKTIxIGIoc2lnbmFscylnKG1hKW8oeSll KG5vdCkxODkgMTcxMiB5KGIpcShlKWgocXVldWVkLClpCih3aGVuKWUobSlvKHVsdGlwbGUp aShzaWduYWxzKWYoYXJlKWUoc2VuKW8odCwpaShvbmx5KWYob25lKWcobWEpbyh5KWYKKGdl dCloKHRocm91Z2guKTMzIGIoU29tZSkyMCBiKE1QSSkxODkgMTc2OCB5KGltcGxlbWVuKW8o dGF0aW9ucyllKG1hKW8KKHkpZih1c2UpaChzaWduYWxzKWcoaW4pbyh0ZXJuYWxseSlsKC4p MjkgYihJZikxOCBiKGFuKWYoYXBwbGljYXRpb24paQooYXR0ZW1wdHMpZSh0bylnKHVzZSlo KHRoZXNlKTE4OSAxODI1IHkoc2lnbmFscywpZyh1bmV4cClxKGVjdGVkKWkoYilxCihlaGEp byh2aW9yKWUobWEpbyh5KWYocmVzdWx0LikyNyBiKEZpbmFsbHkpbCgsKTIwIGIoaXQpZSht YSlvKHkpZihub3QpCmcoYilxKGUpaChwKXEob3NzaWJsZSlpKGF0KWQodGhlKWcoaW0tKTE4 OSAxODgxIHkocGxlbWVuKW8odGF0aW9uKWcKKGxldilvKGVsKWkodG8pZChyZXN0cmljdClo KHRoZSlnKGVcMDEzZWN0cylnKG9mKWYoYSloKHNpZ25hbCloKHRvKWUoYSkKaChzaW5nbGUp aChNUEkpZihwcm8pcShjZXNzLClnKHNvKWcodGhhdCkxODkgMTkzOCB5KGEpZChzaWRlLWVc MDEzZWN0KWkKKG9mKWYoc2VuZGluZyloKGEpZShzaWduYWwpaSh0byllKGEpaChwcm8pcShj ZXNzKWcobWEpbyh5KWYoYilxKGUpaQoodGhhdCllKG90aGVyKWgocHJvKXEoY2Vzc2VzKWco cmVjZWl2KW8oZSloKHRoYXQpMTg5CjE5OTQgeShzaWduYWwpZyhvcillKGFyZSloKGtpbGxl ZC4pMjIgYihcKClwIEZqKEVuZCkxNgpiKG9mKWcoYWR2aWMpbihlKWcodG8paCh1c2Vycy4p cCBGbChcKSk3NSAyMTE2IHkgRmsoMi4zLjQpNDkKYihOb3RpXDAxNGNhdGlvbikxOCBiKG9m KWUoY2hhbmdlKWcoaW4paChzdGF0ZSllKG9mKWkoYSlnKHApbyhybylxCihjZXNzKTc1IDIy MDIgeSBGbChJbilmKG9yZGVyKWcodG8pZihtYW5hZ2UpZyhwcm8pcShjZXNzZXMpaShvcill KGhhKW8KKHYpbyhlKWcodGhlKWgobWluaW0pbyh1bSloKGluKWcoZmF1bHQpZih0b2xlcmFu Y2UsKWcoYW4pZyhNUEkpZgooYXBwbGljYXRpb24pNzUgMjI1OCB5KG0pbyh1c3QpZyhiKXEo ZSlnKGFibGUpaCh0bylmKGIpcShlKWgKKG5vdGlcMDE0ZWQpZyh3aGVuKWcoYSlmKHBybylx KGNlc3MpZyhleGl0cy4pMTk2NyA0Ngp5IEZlKDEpMTk2NyAxMDMgeSgyKTE5NjcgMTU5IHko MykxOTY3IDIxNSB5KDQpMTk2NyAyNzIKeSg1KTE5NjcgMzI4IHkoNikxOTY3IDM4NSB5KDcp MTk2NyA0NDEgeSg4KTE5NjcgNDk4IHkoOSkxOTU5CjU1NCB5KDEwKTE5NTkgNjExIHkoMTEp MTk1OSA2NjcgeSgxMikxOTU5IDcyNCB5KDEzKTE5NTkKNzgwIHkoMTQpMTk1OSA4MzYgeSgx NSkxOTU5IDg5MyB5KDE2KTE5NTkgOTQ5IHkoMTcpMTk1OQoxMDA2IHkoMTgpMTk1OSAxMDYy IHkoMTkpMTk1OSAxMTE5IHkoMjApMTk1OSAxMTc1IHkoMjEpMTk1OQoxMjMyIHkoMjIpMTk1 OSAxMjg4IHkoMjMpMTk1OSAxMzQ1IHkoMjQpMTk1OSAxNDAxIHkoMjUpMTk1OQoxNDU3IHko MjYpMTk1OSAxNTE0IHkoMjcpMTk1OSAxNTcwIHkoMjgpMTk1OSAxNjI3IHkoMjkpMTk1OQox NjgzIHkoMzApMTk1OSAxNzQwIHkoMzEpMTk1OSAxNzk2IHkoMzIpMTk1OSAxODUzIHkoMzMp MTk1OQoxOTA5IHkoMzQpMTk1OSAxOTY2IHkoMzUpMTk1OSAyMDIyIHkoMzYpMTk1OSAyMDc4 IHkoMzcpMTk1OQoyMTM1IHkoMzgpMTk1OSAyMTkxIHkoMzkpMTk1OSAyMjQ4IHkoNDApMTk1 OSAyMzA0IHkoNDEpMTk1OQoyMzYxIHkoNDIpMTk1OSAyNDE3IHkoNDMpMTk1OSAyNDc0IHko NDQpMTk1OSAyNTMwIHkoNDUpMTk1OQoyNTg3IHkoNDYpMTk1OSAyNjQzIHkoNDcpMTk1OSAy Njk5IHkoNDgpcCBlb3AKJSVQYWdlOiAxMyAxNAoxMyAxMyBib3AgMTgzMCAtMTAwIGEgRmwo MTMpNzUgNDkgeSBGaShNUEkpcCAxNjAgNDkgMTQKMiB2IDE2IHcoUFJPQ0VTUylwIDM3MyA0 OSBWIDE4IHcoTU9OSVRPUlwoZ3JvdXAsKTE1IGIocmFuaywpZyhldmVudCwpaAoocmVxdWVz dFwpKTExNyAxMjYgeSBGZyhJTikxNTUgYiBGaShncm91cCk0NzkgYiBGZyhhKTE0CmIoZ3Jv dXApZyhvZilmKHBybylxKGNlc3NlcylqKFwoaGFuZGxlXCkpMTE3IDIwMiB5KElOKTE1NQpi IEZpKHJhbmspNTA0IGIgRmcocmFuaykxNCBiKG9mKWYocHJvKXEoY2VzcylqKHRvKWUoYilx KGUpZyhtb25pdG9yZWQpZgooXChpbilvKHRlZ2VyXCkpMTE3IDI3NyB5KElOKTE1NSBiIEZp KGV2ZW50KTQ4NyBiIEZnKGEpMTUKYihcMDE1YWcpZihzcClxKGVjaWZ5aW5nKWgod2hhdCln KGV2KW8oZW4pbyh0KWgodG8pZihiKXEoZSlnCihub3RpXDAxNGVkKWgoYWIpcShvdXQpZShc KGluLSk5MDUgMzMzIHkodGVnZXJcKSkxMTcKNDA4IHkoT1VUKTEwOCBiIEZpKHJlcXVlc3Qp NDUyIGIgRmcoYSkxNCBiKHJlcXVlc3QpaChvYilyKGplY3QpZwooXChoYW5kbGVcKSk3NSA1 MzMgeSBGZihpbnQpMjMgYihNUEkpcCAyNDUgNTMzIDE1IDIgdgoxNyB3KFByb2Nlc3MpcCA0 MzAgNTMzIFYgMTYgdyhtb25pdG9yXChNUEkpcCA3MTAgNTMzClYgMTYgdyhHcm91cClnKGdy b3VwLClnKGludCloKHJhbmssKWYoaW50KWcoZXZlbnQsKTM5Mwo1ODkgeShNUEkpcCA0Njgg NTg5IFYgMTcgdyhSZXF1ZXN0KilnKHJlcXVlc3RcKSk3NSA2NzYKeShNUEkpcCAxNTAgNjc2 IFYgMTcgdyhQUk9DRVNTKXAgMzM1IDY3NiBWIDE2IHcoTU9OSVRPUlwoR1JPVVAsKWYKKFJB TkssKWgoRVZFTlQsKWcoUkVRVUVTVCwpZyhJRVJSXCkpMTcwIDczMiB5KElOVEVHRVIpZyhH Uk9VUCwpZyhSQU5LLCkKaChFVkVOVCwpZihSRVFVRVNULClmKElFUlIpMTY2IDgxOSB5IEZs KFRoaXMpMTMgYihmdW5jdGlvbiloKHBybylvCih2aWRlcylmKGEpZihnZW5lcmFsKWkobWV0 aG8pcShkKWYoZm9yKWYoYW4pZyhNUEkpaChwcm8pcShjZXNzKWcodG8pZgooZGV0ZWN0KWgo YSlmKGMpbyhoYW5nZSloKGluKWgoc3RhdGUpNzUgODc1IHkob2YpaChhbm90aGVyKWgocHJv KXEKKGNlc3MuKTIxIGIoVGhlKTE2IGIocmVxdWVzdClnKGNvbXBsZXRlcylnKHdoZW4paCh0 aGUpZihwcm8pcShjZXNzKWcoYykKbyhoYW5nZXMpZihzdGF0ZSlnKGFzKWgoaW5kaWNhdGVk KWgoYilvKHkpNzUgOTMyIHkgRmkoZXZlbnQpcApGbCguKTE2NiA5ODggeSBGaShldmVudClp IEZsKGlzKWYoYW4pZihpbilvKHRlZ2VyKWgoc3ApcShlY2lmeWluZyloKGEpZQooYylvKGhh bmdlKWgob2YpZihzdGF0ZSlmKHRoYXQpaCh0aGUpZyhjYWxsaW5nKWoocHJvKXEoY2Vzcylk KHdpc2hlcylpCih0bylkKGRlLSk3NSAxMDQ0IHkodGVjdC4paihUaGUpZChtb3N0KWUoaW1w KXEob3J0YW4pbyh0KWcoc3RhdGUtYylvCihoYW5nZSlnKGFuZCloKHRoZSlnKG9ubHkpaChv bmUpZihNUEkpZyhjdXJyZW4pbyh0bHkpZyhkZVwwMTRuZXMpaChmb3IpCmUgRmkoZXZlbnQp aSBGbChpcyk3NSAxMTAxIHkgRmkoTVBJKXAgMTYwIDExMDEgMTQgMiB2CjE2IHcoUFJPQ0VT UylwIDM3MyAxMTAxIFYgMTggdyhESUVEKXAgRmwoLilmKEYpbChvciloKHRoaXMpaChjYXNl LClnCkZpKHJlcXVlc3QpaCBGbChjb21wbGV0ZXMpZih3aGVuKWcodGhlKWcocHJvKXEoY2Vz cylnKHRlcm1pbmF0ZXMpZihvcilnCihiKXEoZS0pNzUgMTE1NyB5KGNvbWVzKWQodW5yZWFj KW8oaGFibGUuKTIwIGIoSWYpMTQKYihhKWYocHJvKXEoY2VzcylnKGhhcylnKGFscmVhZHkp Zyh0ZXJtaW5hdGVkKWgob3IpZihiKXEoZWNvbWUpZwoodW5yZWFjKW8oaGFibGUsKWkodGhl KWUocmVxdWVzdCk3NSAxMjE0IHkoY29tcGxldGVzKWooaW1tZWRpYXRlbHkpbCguKQoyMiBi KElmKTE2IGIoYW4pZihhcHBsaWNhdGlvbilpKHcpbyhhbilvKHRzKWQodG8paChiKXEoZSlo KG5vdGlcMDE0ZWQpZwoob2YpZihzZXYpbyhlcmFsKWcoZGlcMDEzZXJlbilvKHQpaChldilv KGVuKW8odHMsKWYoaXQpNzUKMTI3MCB5KG0pbyh1c3QpaChjYWxsKWkgRmkoTVBJKXAgMzU5 IDEyNzAgViAxNiB3KFBST0NFU1MpcAo1NzIgMTI3MCBWIDE4IHcoTU9OSVRPUilmIEZsKG9u Y2UpZyhmb3IpZyhlYWMpbyhoKWYoZXYpbyhlbilvKHQuKTI1CmIoSWYpMTcgYihhKWYobSlv KHVsdGlwbGUpaihyZXF1ZXN0cylkKGFyZSloKGFjdGl2KW8oZSk3NQoxMzI3IHkoZm9yKWUo YSlmKHNpbmdsZSlqKHQpbyh5cClxKGUpZShvZilnKGV2KW8oZW4pbyh0LClnKGFsbCloCihy ZXF1ZXN0cylmKHdpbGwpaShjb21wbGV0ZSlmKHdoZW4pZih0aGUpaChldilvKGVuKW8odClm KG8pcShjY3Vycy4pMTY2CjE0NTkgeSBGaChEaXNjdXNzaW9uOik0MiBiIEZkKE1QSSlwIDUx OSAxNDU5IDEzIDIgdiAxNAp3KFBST0NFU1MpcCA3MTYgMTQ1OSBWIDEzIHcoRElFRCkxNSBi IEZnKGRvKXEoZXMpaShub3QpZihoYSlvKHYpbyhlKWYKKGFuKW8oeSloKHJhY2UpZyhjb25k aXRpb25zKWcoYXNzbylxKGNpYXRlZCloKHdpdGgpZShpdClnKGIpcShlLSk3NQoxNTA5IHko Y2F1c2UpaShpdCllKGlzKWcocClxKG9zc2libGUpaCh0bylnKGRldGVybWluZSlmKHdoZXRo ZXIpaShhKWYKKHBybylxKGNlc3MpaChpcylmKGRlYWQpZyhiKW8oeSlmKGV4YW1pbmluZyll KGl0LikyMwpiKEEpMTYgYihoKW8oeXApcShvdGhldGljYWwpZihldilvKGVuKW8odCk3NSAx NTU5IHkgRmQoTVBJKXAKMTUyIDE1NTkgViAxNCB3KFBST0NFU1MpcCAzNDkgMTU1OSBWIDE0 IHcoTUlHUkEpbShURUQpcApGZygsKWUob24paih0aGUpaChvdGhlcilmKGhhbmQsKWcoaXMp ZyhtKW8odWMpbyhoKWUobW9yZSloKGNvbXBsaWNhdGVkKQpnKHRvKWcoaGFuZGxlLCloKHNp bmNlKWgob25jZSlmKGEpNzUgMTYwOSB5KHBybylxKGNlc3MpZShoYXMpZAoobWlncmF0ZWQs KWYoaXQpaShhcHApcShlYXJzKWcodGhlKWgoc2FtZSlkKGFzKWkoYW4pbyh5KWYob3RoZXIp aShwcm8pcQooY2Vzcy4pMTkgYihUaGlzKTExIGIobWVhbnMpZyh0aGVyZSlpKGlzKWYoYSlm KHJhY2UpaShjb25kaXRpb24pNzUKMTY1OCB5KGIpcShldClvKHcpbyhlZW4paih0aGUpZihj YWxsKWYodG8pZyBGZChNUEkpcAo1MTMgMTY1OCBWIDE0IHcoUFJPQ0VTUylwIDcxMCAxNjU4 IFYgMTMgdyhNT05JVE9SKWcgRmcoYW5kKWcodGhlKWgKKGFjdHVhbClnKG1pZ3JhdGlvbiwp YyhzbylqKHRoYXQpaChpdClmKGlzKWcoaW1wKXEob3NzaWJsZSlmKHRvKTc1CjE3MDggeShn dWFyYW4pbyh0ZWUpaCh0aGF0KWcoZXYpbyhlbilvKHRzKWgodylvKG9uJ3QpZShiKXEoZSlp KGxvc3QuKQoxNjYgMTc1OCB5KE9uZSloKHcpbyhhKW8oeSlmKHRvKWcoaGFuZGxlKWcodGhp cyloKGlzKWYodG8pZyhoYSlvKHYpbyhlKQpoKFxcc3RpYylvKGt5IilmKGV2KW8oZW4pbyh0 cyloKHRoYXQpZihNUEkpaChyZW1lbSlvKGIpcShlcnMpZihhbmQpZwooY2FuKWgoZ2l2KW8o ZSlmKGJhYylvKGspZyh0byk3NSAxODA4IHkodGhlKWsodXNlcilnKGluKWUodGhlKWkob3Jk ZXIpZgoocmVjZWl2KW8oZWQuKTMyIGIoU2luY2UpMTggYih3KW8oZSloKGRvKWUobm90KWgo aW4pZyhnZW5lcmFsKWcodylvKGFuKQpvKHQpZihmb3IpaChldilvKGVyeSloKHBybylxKGNl c3MpaCh0bylkKGIpcShlKWkobm90aVwwMTRlZCk3NQoxODU4IHkoYWIpcShvdXQpMTMgYihl dilvKGVyeSloKGV2KW8oZW4pbyh0KWcoaW4pZihldilvKGVyeSloKG90aGVyKWcKKHBybylx KGNlc3MpaChpdCllKGtubylvKHdzKWgoYWIpcShvdXQsKWYoaXQpZihtKW8odXN0KWgoYilx KGUpaChwKXEKKG9zc2libGUpZih0byloKHR1cm4pZihvbilnKGFuZCloKG9cMDEzKWYodGhp cyk3NSAxOTA4Cnkoc3RpYylvKGtpbmVzcywpaihhbmQpZih0aGVyZSlpKGlzKWUoYSlnKG5l dyloKHJhY2UpZyhjb25kaXRpb24sKWUKKGFkbWl0dGVkbHkpZihzbGlnaClvKHRseSloKG1v cmUpaChkaVwwMTZjdWx0KWYodG8paCh0cmlnZ2VyLCloKGIpcShldCkKbyh3KW8oZWVuKTc1 IDE5NTcgeShcXGVhcmx5IilkKGV2KW8oZW4pbyh0cylpKGFuZClmKHR1cm5pbmcpZyhvbilm CihzdGljKW8oa2luZXNzLikxNjYgMjAwNyB5KEEpaChoKW8oeXApcShvdGhldGljYWwpaChc XGV2KW8oZW4pbyh0IilmCkZkKE1QSSlwIDY3OSAyMDA3IFYgMTUgdyhQUk9DRVNTKXAgODc3 IDIwMDcgViAxMyB3KFNMRUVQSU5HKWcKRmcoZmFsbHMpZihzb21ld2hlcmUpaShpbilmKHRo ZSloKG1pZGRsZS4paihXaGV0aGVyKWQob3IpNzUKMjA1NyB5KG5vdClmKGEpZihwcm8pcShj ZXNzKWooaXMpZShzbGVlcGluZylmKGNhbilpKGIpcShlKWYoZGV0ZXJtaW5lZCkKZyhiKW8o eSlmKGV4YW1pbmluZylmKHRoZSlpKGN1cnJlbilvKHQpaChzdGF0ZSlnKG9mKWUodGhlKWko cHJvKXEKKGNlc3MsKWcoaXQpZShpcyloKHN0aWxsKTc1IDIxMDcgeShwKXEob3NzaWJsZSln KHRvKWcobWlzcylmKGEpZwooc2xlZXAvdylvKGFrKW8oZXVwKWkocGFpci4pMTY2IDIxNTcg eShUaGUpayhyZXF1aXJlbWVuKW8odClnKHRoYXQpZwooYWxsKWYocmVxdWVzdHMpaihjb21w bGV0ZSlkKGZvciloKGEpZihzaW5nbGUpaChldilvKGVuKW8odClnKHJhdGhlciloCih0aGFu KWYob25lKWcoYXQpZyhhKWcodGltZSllKGlzKTc1IDIyMDYgeShtb3RpdiluKGF0ZWQpOQpi KGIpbyh5KWoodGhlKWcob2JzZXJ2KW4oYXRpb24pZih0aGF0KWcoc3ApcShlY2lmeWluZylo KG9uZSlmKHJlcXVlc3QpaQooY29tcGxldGlvbilkKHApcShlcilpKGV2KW8oZW4pbyh0KWco dylvKG91bGQpZihub3QpZyhjb21wbGV0ZWx5KTc1CjIyNTYgeShzb2x2KW8oZSkxNiBiKHRo ZSlpKHJhY2UpZihjb25kaXRpb24pZihpdClnKHRyaWVzKWgodG8pZwooYWRkcmVzcy4pMjcg YihNb3JlbylvKHYpbyhlciwpMTcgYihpdClnKGVuY291cmFnZXMpZyh3cml0aW5nKWYoaGFp cnkpZwooY28pcShkZSlpKHRvKWUoYWx3KW8oYSlvKHlzKTc1IDIzMDYgeShrKW8oZWVwKWko bSlvKHVsdGlwbGUpZAoocmVxdWVzdHMpMjAgYihvdXRzdGFuZGluZy4pMjggYihGaW5hbGx5 KW0oLCkxNiBiKHRoZXJlKWooaXMpZShjdXJyZW4pbwoodGx5KWkobm8pZSBGZChldmVudClp IEZnKGZvcillKHdoaWMpbyhoKWgoaXQpZih3KW8ob3VsZClmKG1haylvKGUpNzUKMjM1NiB5 KHNlbnNlLikxNjYgMjQ5NSB5IEZsKEYpbCh1dHVyZSljKHYpbyhlcnNpb25zKWgob2YpZih0 aGUpaChNUEkpZgooc3RhbmRhcmQpaChtYSlvKHkpZShzcClxKGVjaWZ5KWoobmV3KWYodClv KHlwKXEoZXMpZyhvZilmKGV2KW8oZW4pbwoodHMsKWcoYW5kKWgoaW1wbGVtZW4pbyh0YS0p NzUgMjU1MSB5KHRpb25zKWgobWEpbyh5KWYoZGVcMDE0bmUpaShuZXcpZgoob25lcylnKGFz KWYodylvKGVsbC4pMjEgYihJZikxNCBiKGFuKWcoYXBwbGljYXRpb24paCh3KW8oYW4pbyh0 cyllKHRvKQpnKGIpcShlKWkobm90aVwwMTRlZClmKG9mKWcoc2V2KW8oZXJhbClnKGRpXDAx M2VyZW4pbyh0KTc1CjI2MDggeShldilvKGVuKW8odHMsKWgoaXQpZyhtKW8odXN0KWcoY2Fs bCloIEZpKE1QSSlwCjU1MiAyNjA4IDE0IDIgdiAxNiB3KFBST0NFU1MpcCA3NjUgMjYwOCBW IDE4IHcoTU9OSVRPUilnCkZsKG9uY2UpZihmb3IpZyhlYWMpbyhoKWcoZXYpbyhlbilvKHQu KS0zMiA0NiB5IEZlKDEpLTMyCjEwMyB5KDIpLTMyIDE1OSB5KDMpLTMyIDIxNSB5KDQpLTMy IDI3MiB5KDUpLTMyIDMyOCB5KDYpLTMyCjM4NSB5KDcpLTMyIDQ0MSB5KDgpLTMyIDQ5OCB5 KDkpLTQwIDU1NCB5KDEwKS00MCA2MTEKeSgxMSktNDAgNjY3IHkoMTIpLTQwIDcyNCB5KDEz KS00MCA3ODAgeSgxNCktNDAgODM2IHkoMTUpLTQwCjg5MyB5KDE2KS00MCA5NDkgeSgxNykt NDAgMTAwNiB5KDE4KS00MCAxMDYyIHkoMTkpLTQwCjExMTkgeSgyMCktNDAgMTE3NSB5KDIx KS00MCAxMjMyIHkoMjIpLTQwIDEyODggeSgyMyktNDAKMTM0NSB5KDI0KS00MCAxNDAxIHko MjUpLTQwIDE0NTcgeSgyNiktNDAgMTUxNCB5KDI3KS00MAoxNTcwIHkoMjgpLTQwIDE2Mjcg eSgyOSktNDAgMTY4MyB5KDMwKS00MCAxNzQwIHkoMzEpLTQwCjE3OTYgeSgzMiktNDAgMTg1 MyB5KDMzKS00MCAxOTA5IHkoMzQpLTQwIDE5NjYgeSgzNSktNDAKMjAyMiB5KDM2KS00MCAy MDc4IHkoMzcpLTQwIDIxMzUgeSgzOCktNDAgMjE5MSB5KDM5KS00MAoyMjQ4IHkoNDApLTQw IDIzMDQgeSg0MSktNDAgMjM2MSB5KDQyKS00MCAyNDE3IHkoNDMpLTQwCjI0NzQgeSg0NCkt NDAgMjUzMCB5KDQ1KS00MCAyNTg3IHkoNDYpLTQwIDI2NDMgeSg0NyktNDAKMjY5OSB5KDQ4 KXAgZW9wCiUlUGFnZTogMTQgMTUKMTQgMTQgYm9wIDc1IC0xMDAgYSBGbCgxNCkxNjYgNDkg eShGKWwob3IpMTEgYihhKWgocmVxdWVzdCloKGdlbmVyYXRlZCkKZihiKW8oeSlnIEZpKE1Q SSlwIDc4NCA0OSAxNCAyIHYgMTYgdyhQUk9DRVNTKXAgOTk3IDQ5ClYgMTggdyhNT05JVE9S KXAgRmwoLClnIEZpKE1QSSlwIDEzMzMgNDkgViAxNiB3KFcpbChBSVQpZwpGbChyZXR1cm5z KWcoYSlnKHN0YXR1cylnKHRoYXQpNzUgMTA2IHkoY29uKW8odGFpbnMpaihubylnKHVzZWZ1 bClpCihpbmZvcm1hdGlvbillKFwoaS5lLiwpZyh0aGUpZyhzdGF0dXMpZihpcylpKHVuZGVc MDE0bmVkXCkuKTE2NgoyNDUgeSBGaChEaXNjdXNzaW9uOik0NyBiIEZnKE9uZSkxOCBiKHAp cShvc3NpYmlsaXQpbyh5KWUodylvKG91bGQpaChiKQpxKGUpZyh0byloKGdldClmKHRoZSlo KGV4aXQpZihjbylxKGRlKWgoZnJvbSllKHRoZSloKHN0YXR1cy4pMjkKYihIbylvKHcpbyhl dilvKGVyLCkxOCBiKHRoZSk3NSAzMDEgeShwcm8pcShjZXNzKWYobWEpbyh5KWMobm90KWko aGEpbwoodilvKGUpZihleGl0ZWQpaSgtKWYoaXQpZihtYSlvKHkpZihoYSlvKHYpbyhlKWko YilxKGVjb21lKWcodW5yZWFjKW8KKGhhYmxlLikyMiBiKEl0KTE1IGIobWEpbyh5KWUoYWxz byloKGhhKW8odilvKGUpaChiKXEoZWVuKWgoa2lsbGVkKWUoYikKbyh5KWcoYSk3NSAzNTgg eShzaWduYWwpYyhhbmQpZyh3aGF0KWgodylvKGUnZClnKHcpbyhhbilvKHQpZih0byloKGtu bykKbyh3KWYoaXMpaCh3aGF0KWYoc2lnbmFsKWcoa2lsbGVkKWcoaXQuKTE2IGIoSW4pMTEg Yih0aGUpaChlbmQsKWYodylvCihlKWcobWlnaClvKHQpZShoYSlvKHYpbyhlKWgodG8paChy ZWltcGxlbWVuKW8odCk3NSA0MTQKeSh3KW8oYWl0Lik3NSA2NDggeSBGaShNUEkpcCAxNjAg NjQ4IFYgMTYgdyhHUk9VUClwIDMyOAo2NDggViAxOCB3KE1PTklUT1JcKGdyb3VwLClrKGV2 ZW50LCloKGEpbyhycmEpbyh5KXAgOTM1CjY0OCBWIDE0IHcob2YpcCA5ODYgNjQ4IFYgMTYg dyhyZXF1ZXN0c1wpKTExNyA3MjUgeSBGZyhJTikxNTUKYiBGaShldmVudCk0ODcgYiBGZyhh KTE0IGIoXDAxNWFnKWYoc3ApcShlY2lmeWluZyloKHdoYXQpZyhldilvKGVuKW8odCkKZyh0 bylnKGIpcShlKWgobm90aVwwMTRlZCllKGFiKXEob3V0KTExNyA4MDAgeShJTikxNTUKYiBG aShncm91cCk0NzkgYiBGZyhhKTE0IGIoZ3JvdXApZyhvZilmKHBybylxKGNlc3NlcykxMTcK ODc1IHkoT1VUKTEwOCBiIEZpKGEpbyhycmEpbyh5KXAgNDE2IDg3NSBWIDE1IHcob2YpcAo0 NjggODc1IFYgMTYgdyhyZXF1ZXN0cykyNzIgYiBGZChNUEkpcCA5ODIgODc1IDEzIDIgdgox NSB3KFJlcXVlc3QpcCBGZyhzKTE1IGIodG8pZihiKXEoZSlnKHRlc3RlZClpKGFuZC9vcilk KHcpbyhhaXRlZCloKG9uKQo3NSAxMDAwIHkgRmYoaW50KTIzIGIoTVBJKXAgMjQ1IDEwMDAg MTUgMiB2IDE3IHcoR3JvdXApcAozODIgMTAwMCBWIDE3IHcobW9uaXRvclwoTVBJKXAgNjYz IDEwMDAgViAxNSB3KEdyb3VwKWgoZ3JvdXAsKWYoaW50KWcKKGV2ZW50LCkzOTMgMTA1NiB5 KE1QSSlwIDQ2OCAxMDU2IFYgMTcgdyhSZXF1ZXN0KilnKGFycmF5KXAKODIwIDEwNTYgViAx NiB3KG9mKXAgODg0IDEwNTYgViAxNyB3KHJlcXVlc3RzXCkpNzUgMTE0Mwp5KE1QSSlwIDE1 MCAxMTQzIFYgMTcgdyhHUk9VUClwIDI4NyAxMTQzIFYgMTYgdyhNT05JVE9SXChHUk9VUCwp ZgooRVZFTlQsKWgoQVJSQVkpcCA5NDggMTE0MyBWIDE3IHcoT0YpcCAxMDEzIDExNDMgViAx Nwp3KFJFUVVFU1RTLClmKElFUlJcKSkxNzAgMTE5OSB5KElOVEVHRVIpaChHUk9VUCwpZyhF VkVOVCwpZyhBUlJBWSlwCjgxOCAxMTk5IFYgMTcgdyhPRilwIDg4MyAxMTk5IFYgMTcgdyhS RVFVRVNUU1woKlwpLClmKElFUlIpMTY2CjEyODYgeSBGbChUaGlzKWMoY29uKW8odilvKGVu aWVuY2UpaChmdW5jdGlvbilmKGlzKWcoZXF1aXYpbShhbGVuKW8odCloCih0byllKGNhbGxp bmcpaSBGaShNUEkpcCAxMjU4IDEyODYgMTQgMiB2IDE2IHcoUFJPQ0VTUylwCjE0NzEgMTI4 NiBWIDE4IHcoTU9OSVRPUilmIEZsKG9uY2UpZyhmb3IpNzUgMTM0MiB5KGVhYylvKGgpYyhw cm8pcQooY2VzcyloKGluKWcgRmkoZ3JvdXApcCBGbCguKWsoVGhlKWMgRmEoaSlwIEZsKHRo KWYoZWxlbWVuKW8odCloKFwobilvCih1bSlvKGIpcShlcmVkKWYoZnJvbSlnKDBcKSlmKG9m KWggRmkoYSlvKHJyYSlvKHkpcCAxMzk2CjEzNDIgViAxNSB3KG9mKXAgMTQ0OCAxMzQyIFYg MTYgdyhyZXF1ZXN0cylpIEZsKGlzKWYoYXNzbylxKGNpYXRlZCk3NQoxMzk4IHkod2l0aClo KHJhbmspZSBGYShpKWggRmwoaW4paCh0aGUpZyhncm91cC4pMTY2CjE0NTUgeSBGaShhKW8o cnJhKW8oeS1vZi1yZXF1ZXN0cyljIEZsKG0pbyh1c3QpZShoYSlvKHYpbyhlKWgoYXQpZwoo bGVhc3QpaChhcylmKG1hbilvKHkpZyhlbGVtZW4pbyh0cyloKGFzKWYodGhlKWcoc2l6ZSlp KG9mKWUodGhlKWcKKGdyb3VwLikxOCBiKEEpMTIgYihyZXF1ZXN0KTc1IDE1MTEgeShjb21w bGV0ZXMpMTggYih3aGVuKWYodGhlKWcKKGNvcnJlc3ApcShvbmRpbmcpaChtZW0pbyhiKXEo ZXIpZihvZilnKHRoZSlnKGdyb3VwKWYoYylvKGhhbmdlcyloKHRvKWcKKHRoZSlnKHN0YXRl KWYoc3ApcShlY2lcMDE0ZWQpaihiKW8oeSk3NSAxNTY4IHkgRmkoZXZlbnQpcApGbCguKTE5 NjcgNDYgeSBGZSgxKTE5NjcgMTAzIHkoMikxOTY3IDE1OSB5KDMpMTk2NyAyMTUKeSg0KTE5 NjcgMjcyIHkoNSkxOTY3IDMyOCB5KDYpMTk2NyAzODUgeSg3KTE5NjcgNDQxIHkoOCkxOTY3 CjQ5OCB5KDkpMTk1OSA1NTQgeSgxMCkxOTU5IDYxMSB5KDExKTE5NTkgNjY3IHkoMTIpMTk1 OQo3MjQgeSgxMykxOTU5IDc4MCB5KDE0KTE5NTkgODM2IHkoMTUpMTk1OSA4OTMgeSgxNikx OTU5Cjk0OSB5KDE3KTE5NTkgMTAwNiB5KDE4KTE5NTkgMTA2MiB5KDE5KTE5NTkgMTExOSB5 KDIwKTE5NTkKMTE3NSB5KDIxKTE5NTkgMTIzMiB5KDIyKTE5NTkgMTI4OCB5KDIzKTE5NTkg MTM0NSB5KDI0KTE5NTkKMTQwMSB5KDI1KTE5NTkgMTQ1NyB5KDI2KTE5NTkgMTUxNCB5KDI3 KTE5NTkgMTU3MCB5KDI4KTE5NTkKMTYyNyB5KDI5KTE5NTkgMTY4MyB5KDMwKTE5NTkgMTc0 MCB5KDMxKTE5NTkgMTc5NiB5KDMyKTE5NTkKMTg1MyB5KDMzKTE5NTkgMTkwOSB5KDM0KTE5 NTkgMTk2NiB5KDM1KTE5NTkgMjAyMiB5KDM2KTE5NTkKMjA3OCB5KDM3KTE5NTkgMjEzNSB5 KDM4KTE5NTkgMjE5MSB5KDM5KTE5NTkgMjI0OCB5KDQwKTE5NTkKMjMwNCB5KDQxKTE5NTkg MjM2MSB5KDQyKTE5NTkgMjQxNyB5KDQzKTE5NTkgMjQ3NCB5KDQ0KTE5NTkKMjUzMCB5KDQ1 KTE5NTkgMjU4NyB5KDQ2KTE5NTkgMjY0MyB5KDQ3KTE5NTkgMjY5OSB5KDQ4KXAKZW9wCiUl VHJhaWxlcgplbmQKdXNlcmRpY3QgL2VuZC1ob29rIGtub3due2VuZC1ob29rfWlmCiUlRU9G Cg== --PART-BOUNDARY=.19604101719.ZM900.nas.nasa.gov-- From mpi-core-human@mcs.anl.gov Mon Apr 15 14:43:22 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA27706; Mon, 15 Apr 1996 14:43:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA16262 for mpi-core-out; Mon, 15 Apr 1996 13:45:31 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA16257 for ; Mon, 15 Apr 1996 13:45:24 -0500 Message-Id: <199604151845.NAA16257@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: April MPI Forum meeting, better instructions, and please RSVP Date: Mon, 15 Apr 1996 13:45:23 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, The next meeting of the MPI Forum will take place on Wed.- Fri., April 24-26. This time we are meeting in a ***NEW HOTEL***, the Ramada Hotel O'Hare, located near the airport. Please make your reservations now if you have not already done so. Register at the hotel by calling (708) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. If you have any problems, let me know. (708-252-7852) We will follow the usual schedule, beginning at 1:30 on Wednesday and ending at noon on Friday. We will charge the usual registration fee of $75, which can be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. I seem to have misplaced my list of people who have told me they are coming. All I have on said list are: Rolf Rabenseifner Marydell Tholborn Dennis Cottel Hans-Christian Hoppe If you are not one of these and are coming to the meeting, please let me know (even though you might suppose that I can guess in your case.) I need a final count for the hotel by Friday. Thanks. Regards, Rusty From mpi-core-human@mcs.anl.gov Thu Apr 18 17:26:37 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA05498; Thu, 18 Apr 1996 17:26:36 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA03421 for mpi-core-out; Thu, 18 Apr 1996 16:26:23 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA03415 for ; Thu, 18 Apr 1996 16:26:14 -0500 Message-Id: <199604182126.QAA03415@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: RSVP for MPI Forum meeting Date: Thu, 18 Apr 1996 16:26:13 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk If you are coming to the MPI Forum meeting next week and are not on the list below, please let me know immediately. I need a final count for the hotel tomorrow at noon. Marc? Eric? Tony? Andrew? Shane? Are you coming? Thanks. Rusty MPI-2 Forum Expected Attendees, April 1996 Rusty Lusk Bill Gropp Rolf Rabenseifner Marydell Tholburn Dennis Cottel Hans-Christian Hoppe Bill Saphir Parkson Wong Simon Tsang Nick Nevin Ron Brightwell Raja Daoud Dick Treumann Koichi Konishi Steve Huss-Lederman Arkady Kanevsky Leslie Hart Al Geist Joel Clarke Heidi Poxon Karl Feind Terry Dontje John May Mark Fallon Rolf Hempel James Cownie Ming Xu From mpi-core-human@mcs.anl.gov Fri Apr 19 02:29:19 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA11949; Fri, 19 Apr 1996 02:29:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA09736 for mpi-core-out; Fri, 19 Apr 1996 01:32:15 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA09730; Fri, 19 Apr 1996 01:32:06 -0500 Received: (tony@localhost); by Aurora.CS.MsState.Edu (SMI-8.6/7.0m-FWP-MsState); id BAA21934; Fri, 19 Apr 1996 01:30:56 -0500 Date: Fri, 19 Apr 1996 01:30:56 -0500 From: Tony Skjellum Message-Id: <199604190630.BAA21934@Aurora.CS.MsState.Edu> To: mpi-coll@mcs.anl.gov, mpi-core@mcs.anl.gov, mpi-realtime@mcs.anl.gov Cc: jcownie@bbn.com, snir@watson.ibm.com Subject: Persistent sendrecv operations vs. persistent channels Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Please consider the following a discussion item for next meeting (at least we will talk about it in collective and realtime subcommittees). Since this is a pseudo-collective operation, I will put it as a proposal in Collective extensions, until we agree on a more suitable place, or decide to abandon the discussion... - - - - - - - We have been reinvestigating the issue of channels for MPI-2. We evidently left them out in MPI-1 (as noted below), and I would like to be given the two sides of the rationale, which I believe Mssrs. Snir and Cownie and others will remember, whereas I do not, unfortunately. So, I have tried to reconstruct rationale... MPI_SENDRECV() and MPI_SENDRECV_REPLACE() have no persistent analogues. This is probably because they can be emulated with two persistent operations. Also, as the standard says, SENDRECV's can be matched with RECV's and SEND's, as appropriate. Hence, although a SENDRECV where the processes pair off in an exchange appears to be a channel, it is not very optimizable. SENDRECV_INIT would not be very optimizable either, because there is no guarantee that persistent handles are used elsewhere. A SENDRECV that is a channel-like exchange (and unidirectional as a special case using a count=0), and has a persistent analogue, does appear to have significant opportunities for performance enhancement. The relaxed matching up with plain receives would have to be sacrificed, but more performance could be obtained: * Negotiation of advance reservations on network or PVCs ... * Addressing any special issues of alignment/misalignment * Negotiation for who does any needed data conversion * Change of protocol as function of length of data For instance: MPI_CHANNEL_INIT(sendbuf, sendcount, sendtype, dest, sendtag, recvbuf, recvcount, recvtype, source, recvtag, comm, request) [IN sendbuf] initial address of send buffer (choice) [IN sendcount] number of elements in send buffer (integer) ... [OUT request] request object (persistent request) Which is analogous to MPI_SENDRECV. MPI_START, MPI_STARTALL, and MPI_WAIT/TEST family to be used as with other non-blocking operations to convert request -> status, and mark as complete the operation. The CHANNEL operations would operate logically out of band from normal send/recv, and using the same tags as are used in SEND/RECV-type calls would be erroneous. The MPI_CHANNEL_INIT would reserve the tags mentioned in the call. Use of wildcard tags/srcs in the CHANNEL call would be erroneous. Tags are freed up for other uses when the MPI_REQUEST_FREE() occurs. For completeness, the syntactic sugar of an MPI_CHANNEL(), equivalent to MPI_CHANNEL_INIT()+MPI_START()+MPI_WAIT()+MPI_REQUEST_FREE() would be provided as well. MPI_CHANNEL(sendbuf, sendcount, sendtype, dest, sendtag, recvbuf, recvcount, recvtype, source, recvtag, comm, status) [IN sendbuf] initial address of send buffer (choice) [IN sendcount] number of elements in send buffer (integer) ... [OUT status] status object (Status) - - - - - - For the realtime discussion, further versions of this channel that provide QoS properties appear very interesting, but I will restrict my remarks to the realtime subcommittee reflector on this front. -Tony Skjellum From mpi-core-human@mcs.anl.gov Fri Apr 19 02:39:36 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA12082; Fri, 19 Apr 1996 02:39:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA09825 for mpi-core-out; Fri, 19 Apr 1996 01:42:47 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA09820 for ; Fri, 19 Apr 1996 01:42:41 -0500 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov> id XAA20444; Thu, 18 Apr 1996 23:42:39 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id XAA13628; Thu, 18 Apr 1996 23:42:38 -0700 From: "Eric Salo" Message-Id: <9604182342.ZM13626@mrjones.engr.sgi.com> Date: Thu, 18 Apr 1996 23:42:37 -0700 In-Reply-To: Tony Skjellum "Persistent sendrecv operations vs. persistent channels" (Apr 19, 1:30am) References: <199604190630.BAA21934@Aurora.CS.MsState.Edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: Persistent sendrecv operations vs. persistent channels Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Tony - Your timing could not possibly have been better. I just decided *yesterday* that we really needed full channels in MPI to get optimal performance, so I of course think this is a great idea. I'd be interested in hearing some good reasons to *not* do it... - Eric -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Apr 19 08:05:25 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA22765; Fri, 19 Apr 1996 08:05:24 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA12113 for mpi-core-out; Fri, 19 Apr 1996 07:07:35 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA12107; Fri, 19 Apr 1996 07:07:26 -0500 Received: (tony@localhost); by Aurora.CS.MsState.Edu (SMI-8.6/7.0m-FWP-MsState); id HAA06364; Fri, 19 Apr 1996 07:06:38 -0500 Date: Fri, 19 Apr 1996 07:06:38 -0500 From: Tony Skjellum Message-Id: <199604191206.HAA06364@Aurora.CS.MsState.Edu> To: mpi-core@mcs.anl.gov Cc: mpi-misc@mcs.anl.gov Subject: Infinite asymptotic bandwidth for thread communication + ... Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Good morning! I make the following proposal, probably for the misc chapter. Chuck Seitz and group had the original idea for these semantics, and I know that a group at NASA Ames keeps the flame going with their MPK project and follow ons. Anyway, this is not something MPI can do, and there is not reason why we couldn't try to help a broader group of users, potentially, such as this NASA group. Anyway, the Caltech Reactive Kernel provided fully buffered operations and only had send/receive. Zipcode in turn utilized this for point-to-point, and used user buffers (as MPI does) for collective. RK was its portability layer. Early in MPI-1, we abandoned the idea to have RK-like semantics, and accepted that there be at least one copy of the data, even in a flat shared memory. The exact reasons are probably known, but I have not found them. I recall the discussion itself vaguely. The reflector is huge :-) However, much after that, we have done two key things on the Forum that suggest that control of buffering was possibly important to users: * Invent a Bsend + user buffer that everyone hates (PVM porting?) * Start talking about special memory for put/get in MPI-2 (e.g., support non-cache coherent puts) Given this liberalization of thinking, I suggest: 1) It seems like threads can now get asymptotically infinite bandwidth, just by using MPI_SEND/RECV or MPI_PUT to pass pointers. That problem is solved. Perhaps an MPI_AINT (or MPI_SHARED_PTR) type is indicated, that only works between processes with common address space. This would be a nice lip worship support for local shared memory beyond what we have offered, with little else required. It certainly is nice for threads, but can, admittedly be done, with a hack. A portable mechanism would be preferable. 2) Provide an additive call mechanism to support RK-like semantics, as a separable proposal from item 1). This could be like MPI_RECV_BUFFER_INIT(datatype, count, comm, src, tag, req) {and other variants} If successful, the storage is taken from attached store provided to the process by user program, but not necessarily when the init is done (implementation defined). A subsequent MPI_START(req) causes the operation to proceed, and the data is stored. A transformation: MPI_WAIT(req, status) and new MPI_BUFFER_RECEIVED(IN status, OUT offset); provides the cursor to the data, as an offset from the attached store base, so usable by C and Fortran. MPI_BUFFER_FREE(IN/OUT offset); relinquishes this space at any time thereafter. MPI_SEND_BUFFER_INIT() is similar to MPI_SEND_INIT(), but makes the buffer argument an out argument offset to the buffer storage base. One fills that memory, prior to calling MPI_START() on the send request. After the MPI_START(), the buffer is no longer available to the user program, and never comes back (implicit MPI_BUFFER_FREE()). The protocol for the SEND should be that there is no transfer, until room has been secured in the receive buffer, so that it can be turned over to the user on completion. In a single memory image, this is a clean offset pass, if the data is organized with the same layout. Otherwise, there is no real advantage, per se. It should be permissible to do a normal SEND or normal receive reusing the base_ptr + offset address, once one has that, and it is filled. This would allow one to do a number of sends (or collectives) with the same buffer, and free only when done (or by using a persistent buffer send on the last one). If we offer this functionality, we potentially can steer people away from the Bsend(), which no one seems to like, and we do offer a standard mode that has no copies on flat shared memory systems (threads, or real-memory embedded multi-processor computers and so on), beyond the user putting the data in and out of the buffers. This has a nice complementarity to other modes, and seems to be easy to add to the implementation. It relies on the extant buffer mechanism, and is seen to work with Fortran as well as C. This would also allow libraries to exploit light-weight memory allocation, if available, to avoid expensive mallocs to get messaging space!?! That might be especially helpful with operations that are sensitive to overheads. - - - - - - - - - You may fire away, :-) Tony From mpi-core-human@mcs.anl.gov Sun Apr 21 18:52:44 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA09287; Sun, 21 Apr 1996 18:52:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA16344 for mpi-core-out; Sun, 21 Apr 1996 17:53:53 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA15603; Sun, 21 Apr 1996 16:24:41 -0500 Received: from porthos.lsc.nd.edu (porthos.lsc.nd.edu [129.74.25.91]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id QAA02817; Sun, 21 Apr 1996 16:24:39 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.7.5/8.7.3) with SMTP id QAA03571; Sun, 21 Apr 1996 16:24:38 -0500 (EST) Date: Sun, 21 Apr 1996 16:24:38 -0500 (EST) From: Jeff Squyres To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Object Oriented MPI (OOMPI) Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The Laboratory for Scientific Computing of the Department of Computer Science and Engineering at the University of Notre Dame is proud to [finally] announce Object Oriented MPI (OOMPI): A full-featured class library for MPI. Some of OOMPI's features include: Full MPI-1.1 functionality Implemented as a thin layer on top of the C MPI bindings Offers convienent and intuitive object-oriented abstractions for message passing Uses many of the powerful semantic features of the C++ language, such as data typing, polymorphism, etc. Full information, documentation, and the source code distribution for OOMPI Version 1.0 can be obtained from: http://www.cse.nd.edu/~lsc/research/oompi/ Any questions, comments, and most especially feedback can be directed to oompi@lsc.nd.edu. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness. {+} (*6* years and counting -- 1 more year of eligibility) From mpi-core-human@mcs.anl.gov Sun Apr 21 22:24:47 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA11234; Sun, 21 Apr 1996 22:23:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA18034 for mpi-core-out; Sun, 21 Apr 1996 21:27:03 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA18012; Sun, 21 Apr 1996 21:26:23 -0500 Received: from porthos.lsc.nd.edu (porthos.lsc.nd.edu [129.74.25.91]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id VAA03766; Sun, 21 Apr 1996 21:26:21 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.7.5/8.7.3) with SMTP id VAA04600; Sun, 21 Apr 1996 21:26:20 -0500 (EST) Date: Sun, 21 Apr 1996 21:26:19 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: C++ Language Bindings Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Greetings. Here is our submission for the C++ bindings of Chapter 7 of the document. The issue of whether to flatten the class heirarchy or not (e.g. with respect to communicators) has been resolved -- it will be flat. See the text for more information. The ND and MSState proposals have been merged and we are now presenting this as a single, unified proposal. If possible, we would like to move this chapter up on the voting schedule, and present this as the "first reading" for the entire Forum. If all goes well, we can take votes on it at the following two meetings. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness. {+} (*6* years and counting -- 1 more year of eligibility) --- postscript data follows this line --- %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: chapter-7.dvi %%Pages: 18 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o chapter-7.ps chapter-7 %DVIPSParameters: dpi=300, compressed, comments removed %DVIPSSource: TeX output 1996.04.21:2053 %%BeginProcSet: texc.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]/id ch-image N /rw ch-width 7 add 8 idiv string N /rc 0 N /gp 0 N /cp 0 N{rc 0 ne{rc 1 sub /rc X rw}{G}ifelse}imagemask restore}B /G{{id gp get /gp gp 1 add N dup 18 mod S 18 idiv pl S get exec}loop}B /adv{cp add /cp X}B /chg{rw cp id gp 4 index getinterval putinterval dup gp add /gp X adv}B /nd{/cp 0 N rw exit}B /lsh{rw cp 2 copy get dup 0 eq{pop 1}{ dup 255 eq{pop 254}{dup dup add 255 and S 1 and or}ifelse}ifelse put 1 adv}B /rsh{rw cp 2 copy get dup 0 eq{pop 128}{dup 255 eq{pop 127}{dup 2 idiv S 128 and or}ifelse}ifelse put 1 adv}B /clr{rw cp 2 index string putinterval adv}B /set{rw cp fillstr 0 4 index getinterval putinterval adv}B /fillstr 18 string 0 1 17{2 copy 255 put pop}for N /pl[{adv 1 chg} {adv 1 chg nd}{1 add chg}{1 add chg nd}{adv lsh}{adv lsh nd}{adv rsh}{ adv rsh nd}{1 add adv}{/rc X nd}{1 add set}{1 add clr}{adv 2 chg}{adv 2 chg nd}{pop nd}]dup{bind pop}forall N /D{/cc X dup type /stringtype ne{] }if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (chapter-7.dvi) @start /Fa 1 59 df<127012F8A3127005057C840D>58 D E /Fb 1 25 df<000F1304EA3F80EA7FC03870F00CEAC078EB3C3838800FF8EB07F0EB03C01609 7E8C1B>24 D E /Fc 6 117 df98 D101 D<121C123E127FA3123E121C C7FCA6B4FCA2121FB0EAFFE0A20B217EA00E>105 D108 D<38FE0FC0EB3FE0381E61F0EBC0F81380EA1F00AD38FFE3FFA218147D931D> 110 D116 D E /Fd 17 118 df<126012F0AF12601200A4126012F0A212600419 779816>33 D<13E01201EA0380EA0700120E5AA25AA25AA35AA91270A37EA27EA27E7EEA 0380EA01E012000B217A9C16>40 D<12C07E12707E7E7EA27EA2EA0380A3EA01C0A9EA03 80A3EA0700A2120EA25A5A5A5A5A0A217B9C16>I61 D68 D<387FFFC0B5FC7EEA1C 01A490C7FCA2131CA2EA1FFCA3EA1C1CA290C7FC14E0A5EA7FFFB5FC7E13197F9816>I< EAFFFEA3EA0380B3EAFFFEA30F197D9816>73 D<38FC07E0EAFE0FA2383A0B80EA3B1BA5 13BBEA39B3A413F3EA38E3A21303A538FE0FE0A313197F9816>77 D<387E1FC038FF3FE0387F1FC0381D07001387A313C7A2121CA213E7A31367A21377A213 37A31317EA7F1FEAFF9FEA7F0F13197F9816>I80 D<387FFFE0B5FCA2EAE0E0A400001300AFEA07FC487E6C5A13197F9816>84 D97 D<127E12FE127E120EA4133E13FF000F1380EB83C0EB 00E0120E1470A614E0EA0F01EB83C0EBFF80000E1300EA063C1419809816>I101 D<38FF0FC0EB3FE0137F3807F040EBC0005BA290C7FCA8EA FFFCA313127F9116>114 D<12035AA4EA7FFFB5FCA20007C7FCA75BEB0380A3EB8700EA 03FE6C5A6C5A11177F9616>116 D<387E1F80EAFE3FEA7E1FEA0E03AB130F380FFFF038 07FBF83803E3F01512809116>I E /Fe 1 16 df15 D E /Ff 69 126 df<127012F8B012701200A5127012F8A31270051C779B18>33 D38 D<137013F0EA01E0EA03C0EA0780EA0F00121E121C5AA25AA45AA81270A47EA27E 121E7EEA0780EA03C0EA01F0120013700C24799F18>40 D<126012F012787E7E7EEA0780 1203EA01C0A2EA00E0A41370A813E0A4EA01C0A2EA03801207EA0F00121E5A5A5A12600C 247C9F18>II<136013F0A7387FFFC0B512E0A2 6C13C03800F000A7136013147E9718>I<121C123E127E127F123F121F1207120E121E12 7C12F81260080C788518>II<1230127812FCA212 7812300606778518>I<1303EB0780A2130F14005B131EA2133E133C137C1378A213F85B 12015B12035BA212075B120F90C7FCA25A121E123E123CA2127C127812F85AA212601124 7D9F18>I49 DII<1230127812FCA2127812301200A81230127812FCA2127812300614 779318>58 D<1218123C127EA2123C12181200A81218123C127EA2123E121E120E121C12 3C127812F01260071A789318>I<387FFFC0B512E0A3C8FCA4B512E0A36C13C0130C7E93 18>61 D<137013F8A213D8A2EA01DCA3138CEA038EA41306EA0707A4380FFF80A3EA0E03 A2381C01C0A2387F07F038FF8FF8387F07F0151C7F9B18>65 DIIIII<3801F1C0 EA03FDEA0FFFEA1F0FEA1C03123813011270A290C7FC5AA5EB0FF0131F130F387001C0A2 13031238A2EA1C07EA1F0FEA0FFFEA03FDEA01F1141C7E9B18>I<387F07F038FF8FF838 7F07F0381C01C0A9EA1FFFA3EA1C01AA387F07F038FF8FF8387F07F0151C7F9B18>II<387F07F038FF87 F8387F07F0381C03C0EB07801400130E131E5B13385B13F0121DA2EA1FB8A2131C121EEA 1C0EA27FA2EB0380A2EB01C0387F03F038FF87F8387F03F0151C7F9B18>75 DI<38FC01F8EAFE 03A2383B06E0A4138EA2EA398CA213DCA3EA38D8A213F81370A21300A638FE03F8A3151C 7F9B18>I<387E07F038FF0FF8387F07F0381D81C0A313C1121CA213E1A313611371A213 311339A31319A2131D130DA3EA7F07EAFF87EA7F03151C7F9B18>IIIII<3803F380EA1F FF5AEA7C0FEA7007EAE003A390C7FC12701278123FEA1FF0EA07FEC67EEB0F80EB03C013 01EB00E0A2126012E0130100F013C038F80780B5FCEBFE00EAE7F8131C7E9B18>I<387F FFF8B5FCA238E07038A400001300B2EA07FFA3151C7F9B18>I<38FF83FEA3381C0070B3 6C13E0EA0F01380783C03803FF806C1300EA007C171C809B18>I<38FE03F8A338700070 A36C13E0A513F8A2EA39DCA2001913C0A3138CEA1D8DA4000D13801305EA0F07A2EA0E03 151C7F9B18>87 D<387F8FE0139F138F380E0700120FEA070E138EEA039C13DCEA01F8A2 6C5AA2137013F07F120113DCEA039E138EEA070F7F000E13801303001E13C0387F07F038 FF8FF8387F07F0151C7F9B18>I<38FF07F8A3381C01C0EA1E03000E1380EA0F07000713 00A2EA038EA2EA01DCA3EA00F8A21370A9EA01FC487E6C5A151C7F9B18>I91 D93 D95 D97 D<127E12FE127E120EA5133EEBFF80000F13C0EBC1E01380EB0070120E1438A6000F1370 A2EB80E013C1EBFFC0000E138038063E00151C809B18>IIIII<3801E1F03807FFF85A381E1E3038 1C0E00487EA5EA1C0EEA1E1EEA1FFC5BEA39E00038C7FC7EEA1FFEEBFFC04813E0387801 F038700070481338A4007813F0EA7E03381FFFC06C13803801FC00151F7F9318>I<127E 12FE127E120EA5133EEBFF80000F13C013C1EB80E01300120EAB387FC3FC38FFE7FE387F C3FC171C809B18>II<12FEA3120EA5EB3FF0137F133FEB0780EB0F00131E5B5B5BEA0FF87F13 9C130EEA0E0F7FEB038014C038FFC7F8A3151C7F9B18>107 DI<387CE0E038FFFBF8EA7FFF381F1F1CEA1E1E A2EA1C1CAB387F1F1F39FFBFBF80397F1F1F001914819318>IIII<3801E380EA07FBEA1FFFEA3E1FEA380FEA7007A2EAE003A6EA7007A2 EA380FEA3C1FEA1FFFEA0FFBEA03E3EA0003A7EB3FF8A3151E7E9318>I<387F87E038FF 9FF0387FBFF83803F878EBF030EBE0005BA25BA9EA7FFEB5FC6C5A15147F9318>II<487E1203A4387FFFC0B5FCA238038000 A9144014E0A33801C1C013FF6C1380EB3E0013197F9818>I<387E07E0EAFE0FEA7E07EA 0E00AC1301EA0F033807FFFC6C13FE3801FCFC1714809318>I<387F8FF000FF13F8007F 13F0381E03C0000E1380A338070700A3EA038EA4EA01DCA3EA00F8A2137015147F9318> I<38FF8FF8A3383800E0A3381C01C0A2137113F9A213D9A2380DDD80A3138DEA0F8FA238 07070015147F9318>I<387F8FF0139F138F38070700138EEA039EEA01DC13F812001370 13F07FEA01DCEA039E138EEA0707000F1380387F8FF000FF13F8007F13F015147F9318> I<387F8FF000FF13F8007F13F0380E01C0EB0380A21207EB0700A2EA03871386138EEA01 CEA2EA00CCA213DC1378A31370A313F05B1279EA7BC0EA7F806CC7FC121E151E7F9318> I<383FFFF05AA2387001E0EB03C0EB078038000F00131E5B13F8485AEA03C0485A380F00 70121E5A5AB512F0A314147F9318>II<127CB4FC13C01203C67EAB 7FEB7FC0EB3FE0A2EB7FC0EBF0005BABEA03C012FF90C7FC127C13247E9F18>125 D E /Fg 10 58 df<121FEA3180EA60C0EA4040EAC060A8EA4040EA60C0EA3180EA1F00 0B107F8F0F>48 D<120C123C12CC120CACEAFF8009107E8F0F>I<121FEA6180EA40C0EA 806012C01200A213C0EA0180EA030012065AEA10201220EA7FC012FF0B107F8F0F>I<12 1FEA2180EA60C0A2120013801201EA0F00EA00801340136012C0A2EA8040EA6080EA1F00 0B107F8F0F>I<1203A25A5A120B121312331223124312C3EAFFE0EA0300A4EA1FE00B10 7F8F0F>III<1240EA7FE013C0EA8080A2EA010012025AA2 120C1208A21218A50B117E900F>I<121FEA3180EA60C0A3EA7180EA3F00120FEA3380EA 61C0EAC060A3EA4040EA6080EA1F000B107F8F0F>I<121FEA3180EA60C0EAC0401360A3 EA40E01221EA1E6012001340EA60C01380EA4300123E0B107F8F0F>I E /Fh 36 122 df<12F0A404047B830E>46 D<14801301A2EB0300A31306A35BA35BA35B A35BA35BA3485AA448C7FCA31206A35AA35AA35AA35AA35AA311317DA418>I<13C01201 120712FFA212FB1203B3A7B5FCA310227CA118>49 DII<131FA2132FA2136F13EF13CF1201A2EA038FA21207130F120F120E121E123CA2 1278A212F0B512F8A338000F00A915217FA018>I<383FFF80A3003CC7FCA913F8EA3FFE 7FEB0F80EA3E07383C03C01238380001E0A7384003C0A212E038700780387C1F00EA3FFE EA0FFCEA03F013227EA018>I<137E48B4FC5AEA07C148C7FC121EA25AA25AA213F8EAF1 FEEAF7FF38FF0F80EAFC0738F803C0A2130100F013E0A51270A21278EB03C01238003C13 80EA1C07381F0F00EA0FFE6C5AEA01F013237EA118>II 67 DI<12F0B3AEB512C0A312 237BA21A>76 D<00FC131E7EA312F7A2EAF380A213C012F1A213E012F013F0A21378A213 38133CA2131EA2130E130F1307A2149E1303A2EB01DEA2EB00FEA3147E17237BA222>78 DI< EAFFFCEBFF8014C038F003E0EB00F014781438143CA51438147814F0EB03E0B512C01480 EBFE00EAF01E7F7F1480130314C0EB01E0A2EB00F0A214781438143C141EA2140F18237B A21F>82 D<13FE3803FFC04813E0EA0F81381E0060481300A25AA5127C123C123FEA1FC0 EA0FFC6CB4FC0001138038003FC0EB07E0EB01F0130014F81478A600C013F07E38F801E0 387E07C0383FFF80000F1300EA01FC15257EA31B>I<00F0011FEB01E0EC3F80A2007816 C0023B1303EC7BC014736C0171EB078014F115E014E16C01E0EB0F00130115F014C0000F EC701E1303157800070180131C0183EB383CA20187133C000301001338151C01C7147813 CFD801CEEB1E70150EA201EE14F0D800EC5C01FC1307A301785C2B237FA22E>87 D97 D<12F0ADEAF1F0EAF7FCB47EEAFC1F487E38F007 80A2EB03C0A8EB0780A238F80F00EAFC3EEAFFFEEAF7F8EAF1F012237CA219>II101 D<3801F0783807FFF85A381F1F00EA1E0F383C0780A5381E0F00EA1F1FEA0FFE485AEA19 F00038C7FCA2123CEA1FFEEBFFC014E04813F0EA7801EB00F8481378A4007813F0383E03 E0381FFFC06C13803801FC0015217F9518>103 D<12F0ADEAF1F8EAF3FCEAF7FEEAFE1E EAF80FA212F0B010237CA219>I<12F0A41200A912F0B3A404237DA20B>I<12F0ADEB1F80 EB3F00133E5B5BEAF1F0EAF3E0EAF7C012FFA27FA2EAFDF0EAF8F8EAF078137C7F131E13 1F7F1480EB07C012237CA218>107 D<12F0B3B104237DA20B>I<39F0F807C039F3FE1FF0 39F7FF3FF839FE0F707800FCEBE07C39F807C03CA200F01380AF1E167C9527>III I114 DI<120FA6EAFFF8A3EA0F00AE1308131CEA07FC13F8EA03 E00E1C7F9B12>II<387801F000 7C13E0383E03C0EA1E07000F138038078F0013DEEA03FC12016C5A137013F8487EEA03DC 139EEA078F380F0780121EEB03C0383C01E0387800F000F813F81516809516>120 D<38F001E0A2387803C0A2127C383C0780A2121E14005B120F130EEA071E139EEA039CA3 EA019813D8EA00F85BA35BA212015BA2485A127F90C7FC127E13207F9516>I E /Fi 44 125 df<137E3801C180EA0301380703C0120EEB018090C7FCA5B512C0EA0E01 B0387F87F8151D809C17>12 D<1306ADB612E0A2D80006C7FCAD1B1C7E9720>43 D<126012F0A212701210A41220A212401280040C7C830C>II<12 6012F0A2126004047C830C>I50 D<1240387FFF801400A2EA4002485AA25B485AA25B136013 4013C0A212015BA21203A41207A66CC7FC111D7E9B15>55 D<126012F0A212601200AA12 6012F0A212701210A41220A212401280041A7C910C>59 D<1306A3130FA3EB1780A3EB23 C0A3EB41E0A3EB80F0A200017FEB0078EBFFF83803007C0002133CA20006133E0004131E A2000C131F121E39FF80FFF01C1D7F9C1F>65 D<90381F8080EBE0613801801938070007 000E13035A14015A00781300A2127000F01400A8007014801278A212386CEB0100A26C13 026C5B380180083800E030EB1FC0191E7E9C1E>67 D70 D<39FFF3FFC0390F003C00ACEBFFFCEB003CAD39FFF3FFC01A1C7E9B1F>72 DI77 D79 DI<007FB512C038 700F010060130000401440A200C014201280A300001400B1497E3803FFFC1B1C7F9B1E> 84 D<12FEA212C0B3B312FEA207297C9E0C>91 D<12FEA21206B3B312FEA20729809E0C> 93 D97 D<12FC121CAA137CEA1D86EA1E03381C018014 C0130014E0A614C013011480381E0300EA1906EA10F8131D7F9C17>II< 133F1307AAEA03E7EA0C17EA180F487E1270126012E0A61260127012306C5AEA0C373807 C7E0131D7E9C17>II<13F8EA018CEA071E1206EA0E0C 1300A6EAFFE0EA0E00B0EA7FE00F1D809C0D>II<12FC121CAA137C13 87EA1D03001E1380121CAD38FF9FF0141D7F9C17>I<1218123CA21218C7FCA712FC121C B0EAFF80091D7F9C0C>I<13C0EA01E0A2EA00C01300A7EA0FE01200B3A21260EAF0C012 F1EA6180EA3E000B25839C0D>I<12FC121CB3A9EAFF80091D7F9C0C>108 D<39FC7E07E0391C838838391D019018001EEBE01C001C13C0AD3AFF8FF8FF8021127F91 24>IIIIIII<1204A4120CA2121C123CEAFFE0EA1C00A91310A5120CEA0E20EA03C00C1A7F99 10>I<38FC1F80EA1C03AD1307120CEA0E1B3803E3F014127F9117>I<38FF07E0383C0380 381C0100A2EA0E02A26C5AA3EA0388A213D8EA01D0A2EA00E0A3134013127F9116>I<39 FF3FCFE0393C0F0380381C07011500130B000E1382A21311000713C4A213203803A0E8A2 EBC06800011370A2EB8030000013201B127F911E>I<387F8FF0380F03801400EA0702EA 0384EA01C813D8EA00F01370137813F8139CEA010E1202EA060738040380381E07C038FF 0FF81512809116>I<38FF07E0383C0380381C0100A2EA0E02A26C5AA3EA0388A213D8EA 01D0A2EA00E0A31340A25BA212F000F1C7FC12F31266123C131A7F9116>I124 D E /Fj 8 118 df<127812FCA412781200A6127812FCA4127806127D910D >58 D68 D99 D<121E123FA4121EC7FCA6B4FCA2121FAEEAFFE0A20B1E7F9D0E>105 D<38FF0FC0EB31E0381F40F0EB80F8A21300AB38FFE7FFA218127F911B>110 DI115 D<38FF07F8A2EA1F00AC1301120F380786FFEA01F818127F911B>117 D E /Fk 30 119 df<14C0A3495AA449C7FCA41306A2B612E0A2D8000CC7FCA45BA45BA4 13201B1C7A9823>43 D<127012F8A212F012E005057B840E>46 D<14021406A2140E141E A2143F142F144FA2148FEB010FA21302A21304130C13081310A201201380EB3FFFEB4007 13C01380EA0100A21202A21206001E130F39FF807FF81D207E9F22>65 D<48B512C039001E00F015781538153C5BA4491378A215F0EC01E09038F007809038FFFE 009038F00F80EC03C03801E00115E0A3EA03C0A315C038078003EC0780EC0F00141E380F 0078B512E01E1F7D9E20>II<48B512FE39001E001C150C1504A25BA4 90387808081500A21418495AEBFFF0EBF030A23801E020A3EC001048481320A21540A248 481380140115001407380F001FB512FE1F1F7D9E1F>69 D<3801FFF038001F00131EA35B A45BA45BA4485AA4485AA4485AA4120FEAFFF8141F7D9E12>73 D77 D<48B5128039001E00E015701538153C5BA4 491378A215F015E09038F003C0EC0F00EBFFFC01F0C7FC485AA4485AA4485AA4120FEAFF F01E1F7D9E1F>80 D<903807E04090381C18C09038300580EB600313C000011301018013 001203A391C7FC7FA213F86CB47E14E06C6C7E131FEB01F8EB0078A21438A21220A21430 00601370146014E000705B38E80380D8C606C7FCEA81F81A217D9F1A>83 D<397FFC1FF83907C003C09038800100A3380F0002A4001E5BA4485BA4485BA4485BA35C A200705B49C7FCEA3002EA3804EA0C18EA07E01D20779E22>85 D97 DI<137E EA01C138030080EA0E07121E001C1300EA3C0248C7FCA35AA5EA70011302EA3004EA1838 EA07C011147C9315>I<1478EB03F8EB0070A414E0A4EB01C0A213F1EA038938070780EA 0E03121C123C383807001278A3EAF00EA31420EB1C40A2EA703C135C38308C80380F0700 15207C9F17>I<137CEA01C2EA0701120E121C123CEA3802EA780CEA7BF0EA7C0012F0A4 127013011302EA3804EA1838EA07C010147C9315>I<1478EB019CEB033CA2EB07181400 A2130EA53801FFE038001C00A45BA55BA65BA45B1201A25B1231007BC7FC12F31266123C 1629829F0E>II<13C0EA01E0A213C0C7FCA7120E12331223EA4380130012471287120EA35AA3 EA3840138012301270EA31001233121C0B1F7C9E0E>105 DI108 D<391C0F80F0392630C318394740640C903880680E EB0070A2008E495A120EA34848485AA3ED70803A3803807100156115E115623970070066 D83003133821147C9325>I<381C0F80382630C0384740601380EB0070A2008E13E0120E A3381C01C0A3EB03840038138814081307EB031000701330383001C016147C931A>I<13 7CEA01C338030180000E13C0121E001C13E0123C1278A338F003C0A3EB07801400EA700F 130EEA3018EA1870EA07C013147C9317>I<3801C1E0380262183804741C1378EB701EA2 EA08E01200A33801C03CA3143838038078147014E0EBC1C038072380EB1E0090C7FCA212 0EA45AA2EAFFC0171D809317>I114 D<13FCEA0302EA0601EA0C03130713061300EA 0F8013F0EA07F8EA03FCEA003E130E1270EAF00CA2EAE008EA4010EA2060EA1F8010147D 9313>II<000E13C0003313E0382301C0EA4381130112473887 0380120EA3381C0700A31410EB0E201218A2EA1C1E380C26403807C38014147C9318>I< 380E0380EA3307002313C0EA43831301EA470000871380120EA3381C0100A31302A25BA2 5BEA0E30EA03C012147C9315>I E /Fl 31 90 df<13031304130813101320136013C0EA 018013005A1206A25AA2121C1218A212381230A21270A21260A412E0A61260A57EA21210 12181208120C12047E7E102E7CA112>40 D<138013C01360132013301318A2130CA3130E 1306AB130EA3130CA2131C1318A213381330A21360A213C0A2EA018013005A12065A5A5A 5A12800F2E7FA112>I<801301A549C7FCA61306A3B612E0A2D80006C7FC5BA65BA65B13 101B207B9A23>43 D<121C123E127EA2123A1202A21204A21208A21210122012C0070E7D 840D>II<1230127812F81278127005057C840D>I50 D<137F380183C0380201E0380400F012075A1301A2000013E0A2EB03C01480EB07 00130E13F8130E7F148014C0130313071230127812F8A200F0138038800F00EA400E6C5A EA1878EA07E0141F7D9D17>I<1460A214E0EB01C013031305130D13091311EB23801343 13C31383EA010312023804070012081218121012201240B512F038000E00A55B131E3801 FFE0141E7E9D17>I54 D<1220383FFFE04813C014803840 0100EAC00212805B485A5B5B5BA25B120148C7FCA25A1206120EA2121E121CA2123CA35A A31230131F799D17>I<141014181438A21478A214FCEB01BC143C130313021306EB043E EB081EA21310A21320801340A290B5FCEB800FD80100138014071202A25AA2000C14C012 3E39FF807FFC1E207E9F22>65 D<0007B5FC39007C01C0013C13E0EC00F04913F8A515F0 EBF00115E0EC03C0EC0F809038FFFE00EBF00F3901E007C0140315E01401A3EA03C01403 A215C01407EC0F80D807801300143EB512F01D1F7E9E20>I<903801F80890380E061890 3838013890387000F8484813784848137048481330A248C7FC5A121E003E1420003C1400 127CA45AA5007814401580A27EEC0100001C13027E0006130C6C13103801C0E0D8003FC7 FC1D217B9F21>I<0007B5FC39007C01E090383C00F0157849133CA2151EA3151F5BA648 48131E153EA3153C157C4848137815F0A2EC01E0EC03C0EC07803907800F00143CB512E0 201F7E9E23>I<0007B512F839007C0078013C133815185B1508A414089038F01000A314 30EBFFF0EBF0703801E020A4EC00081510485AA21520A2156015C0380780011407B61280 1D1F7E9E1F>I<0007B512F839007C0078013C133815185B1508A59038F01000A3143014 70EBFFF03801E0601420A491C7FC485AA6485A7FEAFFFE1D1F7E9E1E>I<903801FC0490 380F030C90383C009C0170137C49133C4848133848481318120748C7FC5A121E003E1410 003C1400127CA45AA2EC7FFCEC03E000781301A2EC03C0A27EA2121C6C13076CEB0B8038 0380113801E06039003F80001E217B9F24>I<3A07FFC7FFC03A007C00F800013C1378A2 495BA649485AA490B5FCEBF0014848485AA64848485AA6484848C7FC01C07F39FFF8FFF8 221F7E9E22>I<3807FFE038007C00133CA25BA65BA6485AA6485AA6485A7FEAFFFC131F 7F9E10>I<3807FFF0D8007EC7FC133CA25BA65BA6485AA41580EC0100EA03C0A25C1402 1406A23807801E147CB512FC191F7E9E1C>76 DI<3A07FC01FFC03A003E003E001518A2014F1310A2EB4780A2EB43C0 A201835BEB81E0A2EB80F0A21478D801005B147C143CA2141EA200026D5AA31407A20006 130392C7FC000F7FEAFFE0221F7E9E22>II<0007B5FC39007C03C090383C01E0EC00F05B15F8A415F0EBF00115E0EC03C0EC07 80EC0F00EBFFF8D801E0C7FCA6485AA6485A7FEAFFFC1D1F7E9E1F>I<3807FFFC38007C 0790383C03C0EC01E0137815F0A415E0EBF00315C0EC0780EC0F00143CEBFFF03801E030 143880A3141E3803C03EA51502D807801304EBC01F39FFFC0F18C7EA03E01F207E9E21> 82 DI<003FB512F0383C07800030143012603940 0F0010A212C01280A3D8001E1300A65BA65BA65B487E387FFFE01C1F7A9E21>I<39FFFC 3FF8390F8007C00007EB0300A2380F0002A6001E5BA6485BA600385BA35C003C5B121C5C 6C48C7FCEA0706EA01F81D20799E22>I<3A03FFC0FFC03A007F007E00013E1338011E13 3015206D5B5D0281C7FCEB078214C6EB03C414E8EB01F0A2130080A2EB017CEB023C1304 EB0C1E1308EB101F497E01407FEBC00701807F38010003000780381F800739FFE01FFE22 1F7F9E22>88 D<39FFF003FF391F8000F86CC712606D134000071480EBC00100031400EB E00200015BEBF00C140800005B6D5AEB7860EB7C40EB3C80013FC7FC7F131EA3131C133C A513381378137C380FFFC0201F7A9E22>I E /Fm 27 122 df<1306AFB612F0A2D80006 C7FCAF1C207D9A23>43 D<131FA2497E133BA2EB7BC013731371EBF1E013E113E000017F A213C000031378A2138000077FA21300380FFFFEA3381E000FA24814801407A24814C014 03A24814E01B207F9F1E>65 D67 D<12F0B3AE04207C9F0D>73 D<12F0B3ABB51280A31120 7B9F19>76 D<00F8EB01F86C1303A200F4147800F61307A300F3130EA3EB801EA200F113 1CEBC03CA200F01338A2EBE078A2EB6070EB70F0A2EB30E01339A2EB19C0A3EB0F80A390 C7FC1D207B9F28>I<133FEBFFC0000313F03807E1F8380F807C48487E001E131E487FA2 48EB0780A348EB03C0A96C130700781480A36CEB0F00A26C131E001F133E6C6C5A3807E1 F86CB45AC613C0013FC7FC1A227DA021>79 DI82 D97 D<12F0ACEAF1F0EAF7FCB47EEAFC3EEAF80F 12F0EB0780A7EB0F00A26C5AEAFC3EEAFFFCEAF7F8EAF1E011207D9F17>IIII<38 03E0F0EA0FFF5A383E3E00EA3C1E487EA5EA3C1EEA3E3EEA1FFC485AEA33E00030C7FC12 38EA3FFEEBFF806C13C04813E0387803F0EAF000A3EAF801387E07E0383FFFC0000F1300 EA03FC141E7F9317>103 D<12F0ACEAF1F8EAF3FCEAF7FEEAFC1FEAF80FA212F0AE1020 7D9F17>I<12F0A41200A812F0B3A204207D9F0B>II<12F0B3AE04207D9F0B>108 D<39F0FC07E039F3FE1FF0 39F7FF3FF839FE0FF07C39F807C03CA200F01380AE1E147D9325>II112 D114 DI<121EA6EAFFF0A3EA1E00AD1320EA1FF0120FEA07C00C1A7F9910>I< D8F01F13F01480A239781B81E0133B14C11339393C31C3C0137114E3D81C701380381E60 E7A213E0000EEB770013C0A20007137E143E13801C147F931F>119 D<38F003C0A238780780A2127C383C0F00A2121E131E120EEA0F1CA2EA073C1338EA03B8 A213B0EA01F0A26C5AA2485AA3485A1207007FC7FC127E127C121D7F9315>121 D E /Fn 78 124 df<90381F83E09038F06E303901C07878380380F8903800F03048EB70 00A7B612803907007000B2387FE3FF1D20809F1B>11 D<133FEBE0C0EA01C0380381E0EA 0701A290C7FCA6B512E0EA0700B2387FC3FE1720809F19>II<90381F81F89038F04F043901C07C06 390380F80FEB00F05A0270C7FCA6B7FC3907007007B23A7FE3FE3FF02420809F26>I 34 D<132013401380EA0100120212065AA25AA25AA312701260A312E0AC1260A3127012 30A37EA27EA27E12027EEA0080134013200B2E7DA112>40 D<7E12407E7E7E120C7EA27E A2EA0180A313C01200A313E0AC13C0A312011380A3EA0300A21206A25A12085A5A5A5A0B 2E7DA112>I<1306AFB612F0A2D80006C7FCAF1C207D9A23>43 D<127012F812FCA21274 1204A31208A21210A212201240060E7C840D>II<127012F8A312 7005057C840D>I<13011303A21306A3130CA31318A31330A31360A313C0A3EA0180A3EA 0300A31206A35AA35AA35AA35AA35AA3102D7DA117>IIIII<13 06A2130EA2131E132EA2134E138EA2EA010E1202A212041208A212101220A2124012C0B5 12F038000E00A7EBFFE0141E7F9D17>II<137CEA0182EA0701380E0380EA0C07121C3838030090 C7FC12781270A2EAF1F0EAF21CEAF406EAF807EB0380A200F013C0A51270A214801238EB 07001218EA0C0E6C5AEA01F0121F7E9D17>I<1240387FFFC01480A238400100EA8002A2 5B485AA25B5BA25BA213C0A212015B1203A41207A76CC7FC121F7D9D17>III<127012F8A312701200AA127012F8A3127005147C930D>I<1270 12F8A312701200AA127012F012F8A212781208A31210A31220A21240051D7C930D>I<5B 497EA3497EA3EB09E0A3EB10F0A3EB2078A3497EA3497EA2EBFFFE3801000FA30002EB07 80A348EB03C0120E001FEB07E039FFC03FFE1F207F9F22>65 DI<90380FC04090387030C03801C009380380053807000300 0E1301001E1300121C123C007C1440A2127800F81400A91278007C1440A2123C121C001E 1480120E6CEB0100380380026C6C5A38007038EB0FC01A217D9F21>IIII<90380FE02090387818609038E004E0380380023807000148 1300001E1460A25A1520127C127800F81400A7EC7FFCEC03E000781301127C123CA27EA2 7E7E380380023900E00460903878182090380FE0001E217D9F24>I<39FFF8FFF8390F80 0F8000071400AC90B5FCEB800FAE000F148039FFF8FFF81D1F7E9E22>II<380FFFC038007C00133CB3A3122012F8A3EA F038EA40781370EA30E0EA0F8012207E9E17>I<39FFFC0FFC390FC003E0390780018015 0014025C5C5C5C5C5C0181C7FC1383EB8780138FEB93C0EB91E013A1EBC0F01380147880 A280A280EC0780A215C0D80FC013E039FFFC3FFC1E1F7E9E23>III<39FF803F F83907C007C0EC03803905E00100A2EA04F01378A2133CA2131E130FA2EB0781A2EB03C1 EB01E1A2EB00F1A21479143DA2141FA28080A2000E7F121F38FFE0011D1F7E9E22>III82 D<3807E080EA0C19EA1007EA3003EA 6001A212E01300A36C1300A21278127FEA3FF0EA1FFC6C7EEA03FF38001F801307EB03C0 A213011280A400C01380130300E01300EAF006EACE0CEA81F812217D9F19>I<007FB512 E038780F010060EB006000401420A200C0143000801410A400001400B3497E3807FFFE1C 1F7E9E21>I<39FFFC3FF8390FC007C03907800380EC0100B3A300031302A2EA01C05C6C 6C5AEB7018EB3820EB0FC01D207E9E22>I<39FFF003FE391F8000F86CC712607F000714 40A26C6C1380A33901E00100A213F000001302A2EBF806EB7804A26D5AA36D5AA2EB1F30 EB0F20A2EB07C0A36D5AA36DC7FC1F207F9E22>I<3BFFF07FF81FF03B1F800FC007C03B 0F0007800380EE010015C0D807801402A33A03C009E004A39039E010F00C00011508A290 38F0207800005DA2EC403C01785CA2ECC03E90393C801E40A390391F000F80A3011E1307 010E91C7FCA2010C7F010413022C207F9E2F>I<387FFFF8EA7C00007013F0386001E012 4038C003C0A238800780130F1400EA001EA25BA25B13F85B485AA23803C004A2EA078012 0FEB000C121E1408481318A248133838F801F8B5FC161F7D9E1C>90 D<12FEA212C0B3B3A512FEA2072D7CA10D>II<12FEA21206B3B3A512FEA2072D7FA1 0D>I<1230127812FC1278123006057D9E0D>95 D97 D<120E12FE120EAA133EEBC380380F01C0EB00E0120E1470A21478A61470A214E012 0F380D01C0380CC300EA083E15207F9F19>IIII<137C13C6EA018F1203EA07061300A7EAFF F0EA0700B2EA7FF01020809F0E>I<14E03803E330EA0E3CEA1C1C38380E00EA780FA5EA 380E6C5AEA1E38EA33E00020C7FCA21230A2EA3FFE381FFF806C13C0383001E038600070 481330A4006013606C13C0381C03803803FC00141F7F9417>I<120E12FE120EAA133E13 43EB8180380F01C0A2120EAE38FFE7FC16207F9F19>I<121C121E123E121E121CC7FCA6 120E127E120EB1EAFFC00A1F809E0C>I<13E0EA01F0A3EA00E01300A61370EA07F01200 1370B3A31260EAF06013C0EA6180EA3F000C28829E0E>I<120E12FE120EAAEB0FF0EB03 C0140013025B5B5B1330137013F8EA0F38EA0E1C131E130E7F1480130314C014E038FFCF F815207F9F18>I<120E12FE120EB3ABEAFFE00B20809F0C>I<390E1F01F039FE61861839 0E81C81C390F00F00EA2000E13E0AE3AFFE7FE7FE023147F9326>IIII<3803E080EA0619EA1C05EA3C07EA3803 1278127012F0A61270127812381307EA1C0BEA0E13EA03E3EA0003A8EB3FF8151D7E9318 >III<1202A31206A2120EA2123EEAFFF8EA0E00AB1308A5EA06101203 EA01E00D1C7F9B12>I<380E01C0EAFE1FEA0E01AE13031206EA030D3801F1FC16147F93 19>I<38FF83F8381E01E0381C00C06C1380A338070100A2EA0382A3EA01C4A213ECEA00 E8A21370A3132015147F9318>I<39FF9FE1FC393C078070391C030060EC8020000E1440 A214C0D807071380130414E039038861001471EBC8733801D032143A3800F03CEBE01CA2 EB6018EB40081E147F9321>I<387FC3FC380F01E0000713C0148038038100EA01C2EA00 E413EC13781338133C137C134E1387EA010738030380380201C0000613E0121E38FF07FE 1714809318>I<38FF83F8381E01E0381C00C06C1380A338070100A2EA0382A3EA01C4A2 13ECEA00E8A21370A31320A25BA3EAF080A200F1C7FC1262123C151D7F9318>III E /Fo 45 122 df12 D<1438B2B712FEA3C70038C7FCB227277DA12E>43 D<12F8A5123812301270A21260A212E0050C7B8410>II<12F8A5 05057B8410>I<13301370EA01F0120F12FFA212F11201B3ACB512E0A313287BA71D>49 D<13FE3803FF80000F13C04813E0EB07F0383C01F8387800FC147C127000F0133EA21260 1220C7FCA3147CA2147814F8EB01F014E01303EB07C0EB0F80EB1F00133E5B5B485A485A 5B485A48C7FC121E5A387FFFFEA417287EA71D>I<137E3803FF804813E0001F13F0383F 83F8EA3E0048137C123812301210C7FCA214F8A21301EB03F0EB0FE03801FFC0148014C0 14E0380001F8EB00FC147C143EA2141FA57E6C133E7E6C137E007C13FC383F01F8381FFF F06C13E0000313C03800FE0018297EA71D>II<383FFFF8A4003EC7FCAA133EEBFF80003F13E014F013C1EB00F8147C12 3EC7FC143EA712400060137C12E06C13F8EA7801387F07F0383FFFE0000F13C06C1300EA 01FC17287EA61D>III<137E38 03FFC04813E04813F0381F81F8383F00FC003E137C007E137E007C133EA56C137CA26C13 F8380FC3F03807FFE06C13C0A2000F13F0381F81F8383F00FC003E137C48133EA248131F A6007C133EA2007E137E6C13FC381F81F8380FFFF06C13E06C13C038007E0018297EA71D >I<137E3801FF80000713C04813E0381F83F0EB00F8003E137848137C143C0078133E12 F8A2141E141FA4143F7E127C147F7E383F01FF381FFFDF6C139F6C131F3803FC1E380040 3E1300A2147CA214F8A2EB01F0381803E0381E0FC0383FFF806C13006C5AEA03F818297E A71D>I65 DIIII71 D<12F8B3B3A6052A7BA910>73 D<00FE15FFA36C5C00FB15DF6D1303A3D8F9C0EB079FA3 6D130F00F8151FA26D131F0170131EA20178133E0138133C013C137CA2011C1378011E13 F8A2010E13F0EB0F01A2010713E01483A2010313C014C7A20101138014EFA20100130014 FF147EA391C7FC282A7BA933>77 D 80 D82 D84 D97 D99 D<143EAFEA01FCEA03FF000F13BE4813FE13C1383F007E003E133E5AA212FC5AA77E127C A2007E137E6C13FEEA1FC113FF6C13BE3803FE3EEA01F8172A7EA91E>I<13FCEA03FF48 13804813C0381F87E0EA3F01383E00F05A1470481378B512F8A400F8C7FCA31278127CA2 7E003F1318381FC0F8EA0FFF7E000113E038007F00151B7E9A1A>II<90387C07803901FF3FC0000713 FF5A903883E000381F01F0A2383E00F8A56C485AA2380F83E013FF485B001D90C7FCEA1C 7C003CC8FCA2123E381FFFE014F86C13FE487F481480387E003F007CEB0FC0481307A46C 130F007EEB1F80393F807F00381FFFFE6C5B000313F038007F801A287F9A1D>I<12F8A5 1200AA12F8B3A9052A7CA90E>105 D<12F8B3B3A6052A7CA90E>108 DII<137E3801FF80000713E04813F0381F81F8383F00FC003E137C4813 3EA20078131E00F8131FA7007C133EA36C137C003F13FC381FC3F8380FFFF06C13E06C13 C038007E00181B7E9A1D>IIIIII<00F813 F8B3A213011303EAFC07B5FCEA7FFEEA3FF8EA1FC0151B7C9A1E>I<00F8131FA2007C13 3EA36C137CA36C13F8A3380F81F0A33807C3E0A3000313C013E700011380A30000130013 F713FF137EA2181B7F9A1B>I<007CEB1F80007EEB3F006C133E6C5B380F80FC6C6C5AEB E1F03803E3E0EA01F76CB45A6D5A91C7FC133EA2133F497E497E3801F3E0EA03E1803807 C0F8380F807C121F497E003E7F007E148048EB0FC01A1B809A1B>120 D<00F8131F7E007C133EA27E147CA27E14F81380000F13F01381EA07C114E013C31203EB E3C0120113E71480EA00F714001377A2137E133EA2133CA2137C1378A25BA21201EA7FE0 5B5B90C7FC18277F9A1B>I E /Fp 20 122 df<15E01401EC03801407EC1F00143E5C5C 13015C495A1307495A495AA249C7FC5B137E13FEA2485AA212035B1207A2485AA3121F5B A2123FA35B127FA7485AB06C7EA7123F7FA3121FA27F120FA36C7EA212037F1201A26C7E A2137E137F7F6D7EA26D7E6D7E13036D7E801300147C8080EC07801403EC01E014001B66 75CC2D>40 D<12C07E12701278123E7E6C7E6C7E7F12036C7E7F6C7E137EA27F80131F80 A26D7EA280130780A26D7EA3801301A280A37F1580A7EC7FC0B0ECFF80A715005BA35CA2 13035CA3495AA25C130F5CA2495AA25C133F91C7FC137EA25B485A5B485A12075B485A48 C8FC123E127812705A5A1A6679CC2D>I43 D66 DI76 D<90380FFFF090B6FC00 0315C03A07F8007FF0486CEB1FFCED07FE486C6D7E838183816C48816C5A6C5AC9FCA515 7F023FB5FC0103B6FC011F13F090387FFE003801FFE0481380D80FFEC7FC485A5B123F48 5AA2485AA45DA26C6C5BED077F6C6C130F6C6C013E13F83C0FFF80F83FFFE000039038FF F01FC6ECC00F90390FFE0003332E7CAD38>97 DI100 DI<171FDA7FF0EBFFC001 07B5000313E0011FECC7E7903B7FE03FFF0FF09039FF800FFC48EB00074848EB03FE0007 9238FF07E0496DEB03C0000FEE8000A2001F82A8000F5EA2000793C7FC6D5B00035D6C6C 495A6C6D485A9138E03FF0D801DFB512C0D803C791C8FC9038C07FF04848CAFCA37FA27F A213F890B612F06C15FF17E06C8217FC6C826D8148B812801207D80FF0C7001F13C0D81F C014014848EC007F007FEE3FE048C9FC171FA56C6CED3FC0A26C6CED7F806C6CEDFF00D8 0FF8EC03FED803FEEC0FF82601FFE0EBFFF06C6CB612C0010F4AC7FCD9007F13C034447D AE3A>103 DI<137C48B4FC4813804813C0A24813E0A56C13C0A26C13806C1300EA007C90C7FC ACEB7FC0B5FCA412037EB3B2B6FCA418497CC820>I108 D<9039FF8007FEB590383FFFC04B13F0913981F81FF8913983 C00FFC00039039870007FE6C138E029C8002B87F188014F0A25CA35CB3A9B60081B6FCA4 382E7BAD41>110 DI<90397FC01FFCB590B57E02C314F09139CFE03FFC9139DF0007FE000301FC6D 7E6C496D13804A6D13C04AEC7FE04A15F0EF3FF8A2EF1FFCA218FEA2170FA218FFAA18FE 171FA218FCA2173F18F818F0177F6EECFFE06E15C06E4913806E49130002DFEB0FFE9139 CFC03FF802C3B512E002C01480DB1FF8C7FC92C9FCB0B67EA438427CAD41>I<90397F80 3F80B5EBFFE0028113F8913883C3FC91388707FE0003138E6C90389C0FFF14B8A214F0A2 ED07FE9138E003FCED01F892C7FCA25CB3A8B612E0A4282E7DAD2F>114 D117 D121 D E /Fq 11 121 df<123C123EEA3FE090B71280A41700485D5E5E5EA2007CC7EA03E000 784A5A4B5A93C7FC48141E153E5D5DC7485A5D14034A5A5D140F4AC8FCA25C143E147E14 FE5CA21301A21303A2495AA4130FA5131FA86D5AA2EB03C0293B7BB930>55 D65 D67 D97 D<49B47E010F13F0017F13FC9038FF81FE3A03FE007F80D807F8133F 4848EB1FC0ED0FE0485A003F15F01507485A16F8A212FFA290B6FCA301C0C8FCA4127FA3 6C7E1678121F7F000F15F06C6C13016C6CEB03E06C6CEB0FC03A00FFC07F8090393FFFFE 00010F13F8010013C025267DA52C>101 D<13FE12FFA412071203B0EDFF80020313F002 0F7F91381E03FC91383801FE02607F4A7E01FF15805C91C7FCA35BB3A4B5D8F83F13FEA4 2F3C7CBB36>104 D110 D<9038FE01FF00FF011F13F0027F13FC9138FC07FE9039FFF001FF00079039C0007F806C 90C7EA3FC04915E0EE1FF017F8160F17FCA2160717FEA917FC160FA317F8EE1FF0A26DEC 3FE06EEB7FC06EEBFF806E4813009039FEFC07FE91387FFFF8021F13E0DA03FEC7FC91C9 FCADB512F8A42F377DA536>112 D<3901FC03F000FFEB0FFC4AB4FC91383C3F80EC707F 00079038E0FFC000035BEBFD80A201FFEB7F809138003F00151E92C7FC5BB3A3B512FCA4 22267DA528>114 D<130FA55BA45BA25BA25B5A5A5A001FEBFFF0B6FCA3000190C7FCB3 153CA86C14781480017F13F090383FC1E090381FFFC06D1380903801FE001E377EB626> 116 D120 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 75 361 a Fq(Chapter)31 b(7)75 568 y Fp(Language)39 b(Binding)75 809 y Fo(7.1)59 b(MPI)20 b(Bindings)75 910 y Fn(The)13 b(MPI)g(standard)g(is)h(a)f(functional)h(sp)q (eci\014cation)h(that)d(curren)o(tly)i(only)f(sp)q(eci\014es)i (bindings)g(for)e(the)75 967 y(languages)j(of)f(C)g(and)h(F)l(ortran)e (77.)20 b(As)15 b(C++)h([2,)f(5)o(])g(gains)h(widespread)h(acceptance)f (as)f(a)g(program-)75 1023 y(ming)h(language)g(for)f(high-p)q (erformance)i(computing,)f(the)g(MPI)g(standard)f(should)i(b)q(e)g (expanded)g(to)75 1080 y(include)j(this)f(user)f(comm)o(unit)o(y)l(.)28 b(In)19 b(addition,)g(the)f(C++)h(language)f(includes)i(seman)o(tic)f (and)f(syn-)75 1136 y(tactic)c(impro)o(v)o(emen)o(ts)g(o)o(v)o(er)f (the)i(C)f(language;)g(C++)h(MPI)f(bindings)i(that)e(tak)o(e)f(adv)m (an)o(tage)h(of)g(these)75 1193 y(impro)o(v)o(emen)o(ts)h(are)g (necessary)g(to)g(gain)g(the)g(acceptance)h(in)g(the)f(ob)s(ject-orien) o(ted)h(comm)o(unit)o(y)l(.)75 1336 y Fo(7.2)59 b(Alternatives)75 1437 y Fn(There)15 b(are)g(three)h(principle)i(alternativ)o(es)d(to)f (consider)j(for)d(C++)i(language)f(bindings:)131 1531 y(1.)22 b(Use)15 b(the)g(existing)h(C)f(language)h(bindings,)131 1625 y(2.)22 b(Pro)o(vide)15 b(a)g(ligh)o(t)o(w)o(eigh)o(t)g(set)g(of)g (ob)s(jects,)f(or)131 1719 y(3.)22 b(Pro)o(vide)15 b(a)g(comprehensiv)o (e)h(class)g(library)l(.)75 1813 y(W)l(e)f(discuss)i(eac)o(h)e(of)g (these)g(alternativ)o(es)g(b)q(elo)o(w.)75 1933 y Fm(C++)f(bindings)47 b Fn(The)14 b(C)e(bindings)k(pro)o(vide)d(one)h(a)o(v)m(ailable)g(in)o (terface)g(for)e(writing)i(MPI)f(programs)f(in)75 1989 y(ANSI)k(C++.)22 b(C++)16 b(MPI)g(programs)e(ma)o(y)h(b)q(e)i(written)e (b)o(y)h(simply)h(including)h(the)e(\\mpi.h")g(header)75 2046 y(\014le)h(\(as)e(sp)q(eci\014ed)j(in)f([3)o(]\))f(and)g(calling)h (MPI)f(functions)h(as)f(describ)q(ed)i(b)o(y)d(the)h(C)g(bindings.)24 b(So,)16 b(one)75 2102 y(prop)q(osal)j(for)f(C++)i(bindings)g(for)f (MPI)f(could)i(b)q(e)g(to)e(mak)o(e)g(the)h(existing)h(C)e(bindings)j (to)d(b)q(e)i(the)75 2158 y(C++)c(bindings)h(as)e(w)o(ell.)166 2215 y(There)j(are)f(some)h(dra)o(wbac)o(ks)f(to)g(this)h(alternativ)o (e.)27 b(First,)18 b(it)g(is)g(conceiv)m(able)i(that)d(there)h(will)75 2271 y(b)q(e)23 b(v)o(ersions)f(of)g(MPI)g(written)g(in)h(C++.)42 b(There)22 b(will)i(b)q(e)f(natural)f(C++)g(bindings)j(in)e(suc)o(h)f (an)75 2328 y(implemen)o(tation)e(on)e(top)g(of)g(whic)o(h)h(the)g(C)f (bindings)j(w)o(ould)d(b)q(e)h(la)o(y)o(ered.)30 b(T)l(o)18 b(require)i(that)d(C++)75 2384 y(programs)c(use)h(the)g(C)f(bindings)j (of)d(suc)o(h)h(an)g(implemen)o(tation)h(seems)f(un)o(w)o(arran)o(ted.) 19 b(Moreo)o(v)o(er,)12 b(the)75 2441 y(sp)q(eci\014cation)19 b(of)e(MPI)h(is)g(in)g(itself)g(ob)s(ject-orien)o(ted.)27 b(Th)o(us,)17 b(it)h(seems)f(that)g(certain)h(C++)g(ob)s(jects)75 2497 y(and)e(bindings)j(w)o(ould)d(b)q(e)h(appropriate)f(in)i(suc)o(h)e (a)g(setting.)23 b(Finally)l(,)18 b(C++)f(p)q(oten)o(tially)h(pro)o (vides)e(a)75 2554 y(m)o(uc)o(h)e(higher)g(lev)o(el)h(of)e(expressiv)o (e)i(p)q(o)o(w)o(er)e(than)g(C)h(and)f(this)i(expressiv)o(eness)f (should)h(b)q(e)f(pro)o(vided)h(to)75 2610 y(MPI)g(users)g(\(if)h(p)q (ossible\).)964 2828 y(1)p eop %%Page: 2 2 2 1 bop 75 -100 a Fn(2)977 b Fl(CHAPTER)16 b(7.)34 b(LANGUA)o(GE)15 b(BINDING)75 49 y Fm(Light)o(w)o(eight)f(Objects)48 b Fn(As)14 b(men)o(tioned)g(ab)q(o)o(v)o(e,)f(the)h(design)h(of)e(MPI)h (is)g(ob)s(ject-orien)o(ted)g(in)h(\015a)o(v)o(or.)j(It)75 106 y(has)13 b(b)q(een)h(prop)q(osed)g(within)g(the)f(MPI)g(F)l(orum)g (that)g(the)g(C++)g(in)o(terface)h(to)e(MPI)h(should)i(b)q(e)e(a)g (basic)75 162 y(set)g(of)g(classes)h(corresp)q(onding)g(to)f(the)g (fundamen)o(tal)h(ob)s(ject)f(t)o(yp)q(es)g(in)h(MPI-1.)19 b(The)14 b(functionalit)o(y)g(of)75 219 y(MPI-1)j(w)o(ould)g(b)q(e)g (realized)i(through)d(mem)o(b)q(er)h(functions)h(of)e(the)h(ob)s (jects.)24 b(This)17 b(t)o(yp)q(e)g(of)g(in)o(terface)75 275 y(w)o(ould)i(b)q(e)g(ligh)o(t)o(w)o(eigh)o(t.)31 b(F)l(or)18 b(instance,)i(MPI)e(error)g(co)q(des)h(w)o(ould)g(still)h (b)q(e)g(returned)f(b)o(y)f(function)75 332 y(calls,)c(no)f(new)g(t)o (yp)q(es)h(of)e(ob)s(jects)h(w)o(ould)g(b)q(e)h(in)o(tro)q(duced,)g (only)g(a)f(minimal)h(use)g(of)e(adv)m(anced)i(features)75 388 y(of)h(C++)g(suc)o(h)h(as)f(p)q(olymorphism)h(w)o(ould)g(b)q(e)g (in)o(tro)q(duced,)g(etc.)75 508 y Fm(Class)21 b(Lib)o(ra)o(ry)44 b Fn(There)21 b(ha)o(v)o(e)g(also)g(b)q(een)h(prop)q(osals)f(within)h (the)f(F)l(orum)f(that)h(a)f(class)i(library)f(b)q(e)75 564 y(de\014ned)i(and)e(used)h(for)f(the)g(C++)h(bindings.)41 b(A)21 b(class)h(library)g(should)g(mak)o(e)f(full)i(use)f(of)e(C++)75 621 y(features)15 b(suc)o(h)g(as)g(inheritance,)i(p)q(olymorphism,)f (etc.)166 677 y(Th)o(us,)e(a)f(class)h(library)h(w)o(ould)f(use)g(p)q (olymorphism)h(to)e(ob)o(viate)g(the)h(need)h(to)e(explicitly)j(pro)o (vide)75 734 y(t)o(yp)q(e)i(information)g(to)g(messages.)28 b(By)18 b(w)o(a)o(y)f(of)h(con)o(trast,)f(the)h(C++)g(bindings)i(w)o (ould)f(still)g(require)75 790 y(that)14 b(t)o(yp)q(e)h(information)h (b)q(e)f(pro)o(vided.)21 b(Similarly)l(,)c(a)d(class)i(library)g(w)o (ould)f(pro)o(vide)h(\\out")e(v)m(ariables)75 847 y(as)f(return)g(v)m (alues)i(to)d(allo)o(w)i(c)o(haining)g(of)f(op)q(erations.)20 b(The)13 b(C++)h(bindings)h(w)o(ould)f(still)g(return)g(error)75 903 y(co)q(des.)166 960 y(A)e(class)f(library)i(is)f(not)f(appropriate) h(as)f(a)g(C++)h Fk(binding)p Fn(,)f(ho)o(w)o(ev)o(er.)18 b(While)13 b(a)e(class)h(library)h(ma)o(y)75 1016 y(mak)o(e)k(user)h (programming)f(more)g(elegan)o(t,)h(a)f(binding)j(m)o(ust)d(pro)o(vide) h(a)f(direct)i(and)e(unam)o(biguous)75 1073 y(mapping)23 b(to)e(the)i(sp)q(eci\014ed)h(functionalit)o(y)g(of)d(MPI.)h(Th)o(us,)i (a)e(general)g(class)h(library)g(should)g(b)q(e)75 1129 y(pro)o(vided,)16 b(but)g(it)f(should)i(b)q(e)f(built)h(on)e(top)g(of)g (the)h(C++)g(bindings.)22 b(Ho)o(w)o(ev)o(er,)15 b(the)g(C++)h (bindings)75 1186 y(should)g(b)q(e)g(general)g(enough)f(to)g(allo)o(w)g (the)h(building)h(of)e(application-sp)q(eci\014)q(c)j(class)e(library)g (as)e(w)o(ell.)166 1325 y Fj(Discussion:)k Fi([7)o(])d(pro)o(vides)h (the)f(sp)q(eci\014cation)h(one)g(p)q(ossible)f(C++)h(class)g(library)e (for)h(MPI)h(|)f(Ob)r(ject)75 1381 y(Orien)o(ted)e(MPI,)f(or)h(OOMPI.)f (Although)g(the)h(OOMPI)g(class)g(library)e(is)h(sp)q(eci\014ed)i(in)e (C++,)h(few)f(assumptions)75 1438 y(ab)q(out)j(implemen)o(tati)o(on)d (are)k(made)d(|)i(in)f(some)g(sense)j(the)e(sp)q(eci\014cation)h(is)f (a)f(generic)i(one)f(that)g(uses)h(C++)75 1494 y(as)g(the)h(program)d (description)j(language.)23 b(Th)o(us,)16 b(the)h(OOMPI)g(sp)q (eci\014cation)g(can)f(also)f(b)q(e)i(considered)g(as)f(a)75 1550 y(generic)g(ob)r(ject)g(orien)o(ted)g(class)g(library)e(sp)q (eci\014cation)i(whic)o(h)f(can)h(th)o(us)g(also)e(form)g(the)i(basis)f (for)g(MPI)h(class)75 1607 y(libraries)f(in)f(other)i(ob)r(ject-orien)o (ted)g(languages.)21 b(As)15 b(suc)o(h,)h(its)f(inclusion)f(in)h(MPI-2) g(w)o(ould)f(b)q(e)h(appropriate.)75 1868 y Fh(7.2.1)49 b(Other)15 b(W)o(o)o(rk)75 1954 y Fn(Other)h(C++)h(in)o(terfaces)f(to)f (MPI)h(include)i Fm(MPI++)e Fn([6)o(],)g Fm(mpi++)g Fn([4)o(],)f(and)h Fm(P)l(ARA++)i Fn([1].)j(W)l(e)16 b(com-)75 2010 y(men)o(t)f(on)g(eac)o (h)g(of)g(these)g(brie\015y)l(.)166 2066 y Fm(MPI++)h Fn(is)g(one)g(of)f(the)h(earliest)g(prop)q(osed)g(C++)g(in)o(terfaces)g (to)f(MPI.)h Fm(MPI++)f Fn(had)h(as)g(one)f(of)75 2123 y(its)k(goals)g(to)f(b)q(e)i(seman)o(tically)g(and)f(syn)o(tactically)h (consisten)o(t)f(with)h(the)f(C)f(in)o(terface.)32 b(The)19 b(basic)75 2179 y(design)d(of)f Fm(MPI++)g Fn(w)o(ould)h(th)o(us)f (form)f(a)h(solid)h(basis)g(for)f(a)f(C++)i(binding)h(for)e(MPI.)166 2236 y Fm(mpi++)k Fn(is)h(a)e(more)h(recen)o(tly)h(in)o(tro)q(duced)g (C++)g(in)o(terface)f(to)g(MPI)g(and)g(is)h(presen)o(tly)f(under)75 2292 y(dev)o(elopmen)o(t.)40 b(The)22 b(v)o(ersion)g(describ)q(ed)h(in) g([4)o(])f(includes)i(only)e(p)q(oin)o(t)g(to)f(p)q(oin)o(t)h(comm)o (unication)75 2349 y(op)q(erations.)c(Ho)o(w)o(ev)o(er,)11 b(ev)o(en)g(with)g(these)f(op)q(erations,)i(mpi++)g(do)q(es)f(not)f (app)q(ear)h(to)f(b)q(e)h(seman)o(tically)75 2405 y(or)h(syn)o (tactically)h(consisten)o(t)f(with)h(the)f(C)g(in)o(terface)g(b)q (ecause)i(of)d(its)i(use)f(of)g(templates)g(for)g(datat)o(yp)q(es.)166 2462 y Fm(P)l(ARA++)24 b Fn(pro)o(vides)f(a)f(generic)i(high-lev)o(el)h (in)o(terface)d(for)g(p)q(erforming)h(message)f(passing)h(in)75 2518 y(C++,)15 b(with)h(no)f(attempt)f(\(b)o(y)h(design\))h(to)e(b)q(e) i(consisten)o(t)g(with)f(MPI.)1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 3 3 2 bop 75 -100 a Fl(7.3.)34 b(RECOMMEND)o(A)l(TIONS)1183 b Fn(3)75 49 y Fo(7.3)59 b(Recommendations)75 151 y Fn(The)19 b(bindings)i(sub)q(committee)f(mak)o(es)f(the)g(follo)o(wing)h (recommendations)f(for)g(C++)h(bindings)h(for)75 207 y(MPI:)131 296 y(1.)h(The)13 b(C++)h(language)g(bindings)h(should)f (consist)g(of)f(a)g(small)h(set)f(of)g(classes)h(with)g(a)f(ligh)o(t)o (w)o(eigh)o(t)189 353 y(functional)18 b(in)o(terface)f(to)g(MPI-2.)26 b(The)17 b(set)g(of)g(classes)g(should)i(b)q(e)e(as)g(small)h(as)f(p)q (ossible)i(and)189 409 y(the)c(classes)h(should)g(corresp)q(ond)f(to)g (fundamen)o(tal)g(MPI)g(ob)s(ject)g(t)o(yp)q(es)g(\(e.g.,)f(comm)o (unicator,)189 466 y(group,)g(etc.,)h(as)f(w)o(ell)j(as)d(an)o(y)h(new) h(t)o(yp)q(es)f(that)f(ma)o(y)h(emerge)g(from)g(MPI-2\).)131 557 y(2.)22 b(The)e(MPI)f(C++)h(language)g(bindings)i(m)o(ust)d(pro)o (vide)h(a)g(seman)o(tically)g(correct)g(in)o(terface)g(to)189 614 y(MPI-2.)131 706 y(3.)i(T)l(o)17 b(the)h(greatest)e(exten)o(t)i(p)q (ossible,)h(the)f(C++)g(bindings)i(should)f(b)q(e)f(seman)o(tically)h (and)f(syn-)189 762 y(tactically)d(iden)o(tical)i(to)d(the)h(C)f (bindings,)j(but)d(use)h(C++)h(seman)o(tics)e(where)h(a)g(higher)g(lev) o(el)h(of)189 819 y(expressiv)o(e)g(p)q(o)o(w)o(er)e(is)i(desired.)131 911 y(4.)22 b(A)14 b(class)h(library)g(should)g(also)g(b)q(e)g (designed)h(and)e(pro)o(vided)h(as)g(an)f(annex)h(to)e(the)i(MPI-2)f (do)q(cu-)189 967 y(men)o(t.)21 b(The)16 b(class)g(library)g(should)h (b)q(e)g(built)g(on)e(top)h(of)f(the)h(C++)g(bindings.)23 b(A)16 b(class)g(library)189 1024 y(built)f(on)e(top)g(of)g(the)h(the)g (C)f(bindings)i(has)f(already)g(b)q(een)g(designed)h(and)f(is)g(pro)o (vided)g(in)h([7)o(].)k(A)189 1080 y(class)c(library)g(built)h(on)e (top)h(of)f(the)h(C++)g(bindings)h(will)g(ha)o(v)o(e)f(a)f(somewhat)g (clearer)h(in)o(terface)189 1136 y(b)q(ecause)h(it)f(will)i(b)q(e)f (able)g(to)f(tak)o(e)f(adv)m(an)o(tage)h(of)f Ff(const)h Fn(argumen)o(ts)f(and)i(so)e(forth.)75 1279 y Fo(7.4)59 b(C++)19 b(Bindings)75 1380 y Fn(The)d(complete)g(set)f(of)g(C++)h (language)g(bindings)h(is)f(presen)o(ted)g(in)h(App)q(endix)g(A.)k (There)16 b(is)g(a)f(small)75 1437 y(set)e(of)g(classes)h(de\014ned)h (for)e(these)g(bindings.)21 b(Ho)o(w)o(ev)o(er,)13 b(to)g(main)o(tain)g (consistency)i(with)f(the)f(binding)75 1493 y(de\014nitions)k(w)o(e)e (list)h(the)g(C++)g(bindings)h(in)f(the)f(same)g(order)h(as)f(giv)o(en) g(for)g(the)h(C)f(bindings)i(\(rather)75 1550 y(than)e(b)q(eing)i (group)q(ed)e(b)o(y)g(class;)g(see)h(Section)g(A.1\).)166 1606 y(The)i(bindings)i(are)e(similar)h(to)f(the)g Fm(MPI++)g Fn(prop)q(osal)h(\(see)f(section)g(7.2.1,)f(but)h(with)h(sev)o(eral)75 1663 y(notable)e(di\013erences:)25 b Fm(MPI++)17 b Fn(mak)o(es)f(use)h (of)f(C++)i(features)e(suc)o(h)h(as)g(p)q(olymorphism,)h(while)g(not)75 1719 y(taking)11 b(adv)m(an)o(tage)f(of)h(other)f(C++)h(features)g(suc) o(h)g(as)g(reference)g(and)g Ff(const)f Fn(seman)o(tics.)19 b(App)q(endix)13 b(A)75 1775 y(prop)q(oses)23 b(the)g(exact)f(opp)q (osite:)36 b(do)22 b(not)h(use)g(p)q(olymorphism)h(b)q(ecause)g(suc)o (h)f(a)f(sc)o(heme)h(cannot)75 1832 y(em)o(brace)15 b(a)h(simple)h (one-to-one)e(mapping)h(that)f(is)h(critical)h(for)e(a)g(bindings,)i (and)f(use)g(reference)g(and)75 1888 y Ff(const)e Fn(seman)o(tics)i (since)g(they)f(are)g(an)g(imp)q(ortan)o(t)g(feature)g(of)g(the)g(C++)h (language.)166 1945 y(In)j(order)f(to)f(mak)o(e)h(the)g(follo)o(wing)h (function)h(list)f(less)g(cluttered,)g(the)f(k)o(eyw)o(ord)g Ff(virtual)f Fn(has)75 2001 y(b)q(een)e(left)f(out)f(of)g(eac)o(h)h (declaration)g(with)g(the)g(implicit)i(understanding)f(that)e(all)h (mem)o(b)q(er)g(functions)75 2058 y(are)g Ff(virtual)f Fn(\(except)i(for)e(constructors)h(and)g(the)h(assignmen)o(t)f(op)q (erator\).)k(In)d(particular,)g(pro\014ling)75 2114 y(library)23 b(ob)s(jects)e(can)h(b)q(e)h(easily)g(deriv)o(ed)g(for)e(these)h(ob)s (jects;)i(the)e(pro\014ling)i(mec)o(hanism)e(can)g(b)q(e)75 2171 y(in)o(v)o(ok)o(ed)15 b(follo)o(w)o(ed)h(b)o(y)f(a)g(call)h(to)f (the)g(original)h(MPI)g(function.)75 2291 y Fh(7.4.1)49 b(Naming)17 b(Rules)75 2377 y Fn(The)11 b(bindings)i(presen)o(ted)e(ab) q(o)o(v)o(e)f(w)o(ere)h(generated)f(b)o(y)h(applying)h(a)f(simple)h (set)e(of)h(rules)g(to)f(the)h(existing)75 2434 y(C)18 b(bindings.)31 b(These)19 b(rules)g(can)g(also)f(b)q(e)h(applied)h(to)e (MPI-2)g(C)g(bindings)i(to)e(create)g(MPI-2)g(C++)75 2490 y(bindings:)131 2591 y(1.)k(Remo)o(v)o(e)d(the)g(MPI)g(opaque)h (handle)g(from)f(the)g(argumen)o(t)f(list)i(and)g(mak)o(e)f(it)g(an)g (ob)s(ject.)32 b(If)189 2647 y(there)16 b(is)h(no)f(MPI)g(opaque)g (handle)i(in)f(the)f(argumen)o(t)f(list,)i(the)g(function)g(should)g(b) q(e)g(globally)189 2704 y(scop)q(ed.)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 4 4 3 bop 75 -100 a Fn(4)977 b Fl(CHAPTER)16 b(7.)34 b(LANGUA)o(GE)15 b(BINDING)243 49 y Fe(\017)23 b Fn(If)18 b(the)g(original)h(function)g (con)o(tained)g(an)f(opaque)g(MPI)g(handle,)i(mak)o(e)d(the)h(function) h(a)289 106 y(metho)q(d)c(of)g(the)h(MPI)f(ob)s(ject.)20 b(Strip)c(the)f(\\)p Ff(MPI)p 1135 106 15 2 v 17 w Fn(")g(from)g(the)g (b)q(eginning)j(of)d(the)g(function)289 162 y(name,)e(as)g(w)o(ell)h (as)f(an)o(y)g(part)g(of)g(the)g(name)h(that)e(is)i(duplicated)h(in)g (the)e(ob)s(ject)g(name)g(\(e.g.)289 219 y Ff(MPI)p 364 219 V 16 w(Comm)p 476 219 V 17 w(compare\(\))h Fn(b)q(ecomes)i Ff(MPI)p 976 219 V 17 w(Comm::Compare\(\))p Fn(\).)243 292 y Fe(\017)23 b Fn(If)e(the)g(original)h(function)f(did)h(not)f(con) o(tain)g(a)f(single)j(opaque)e(MPI)f(handle,)k(the)d(full)289 348 y(function)16 b(name)f(should)i(b)q(e)f(retained)g(\(e.g.)j Ff(MPI)p 1153 348 V 17 w(Buffer)p 1314 348 V 16 w(attach\(\))c Fn(b)q(ecomes)h Ff(::MPI)p 1838 348 V 16 w(-)289 405 y(Buffer)p 436 405 V 16 w(attach\(\))p Fn(\).)131 511 y(2.)22 b(F)l(ollo)o(w)15 b(the)g(MPI-1)g(capitalization)i(rules)f(for) f(the)g(metho)q(d)g(names.)131 605 y(3.)22 b(Metho)q(ds)15 b(should)h(b)q(e)g(declared)h Ff(const)e Fn(if)g(they)h(return)f (information)h(to)e(the)i(user)f(and)h(do)f(not)189 661 y(c)o(hange)g(the)g(in)o(ternal)h(state)f(of)f(the)i(calling)h(ob)s (ject.)131 755 y(4.)22 b(An)o(y)16 b(reference)i(or)e(arra)o(y)f (argumen)o(t)h(that)g(will)i(not)e(b)q(e)i(mo)q(di\014ed)g(in)f(the)g (metho)q(d)g(should)g(b)q(e)189 811 y(declared)f Ff(const)p Fn(.)131 905 y(5.)22 b(P)o(oin)o(ter)12 b(argumen)o(ts)g(should)i(b)q (e)f(c)o(hanged)g(to)f(references)h(when)h(a)e(reference)h(is)h(what)e (is)h(seman-)189 962 y(tically)j(implied.)131 1055 y(6.)22 b(Arra)o(y)14 b(argumen)o(ts)g(should)i(b)q(e)g(denoted)g(with)g (square)f(brac)o(k)o(ets)f(\()p Ff([])p Fn(\),)g(not)h(p)q(oin)o(ters.) 166 1162 y(These)h(rules)g(should)h(b)q(e)f(applied)h(to)e(all)i(new)e (functions)i(created)e(in)i(MPI-2)e(so)g(that)g(the)g(C++)75 1218 y(bindings)i(sta)o(y)d(curren)o(t)h(with)h(the)f(latest)g (functionalit)o(y)i(of)d(MPI.)75 1340 y Fh(7.4.2)49 b(Cop)o(y)17 b(Semantics)75 1426 y Fn(The)g(cop)o(y)f(and)g(assignmen)o(t)h(seman)o (tics)f(in)h(C++)g(should)g(b)q(e)h(the)e(same)g(as)g(those)g(sp)q (eci\014ed)j(b)o(y)d(the)75 1482 y(C)e(and)h(F)l(ortran)e(77)g (bindings.)22 b(That)14 b(is,)g(the)h(MPI)f(user)g(lev)o(el)i(ob)s (jects)e(should)h(b)q(eha)o(v)o(e)g(lik)o(e)g(handles.)189 1588 y Fk(A)n(dvic)n(e)g(to)h(implementors.)40 b Fn(Eac)o(h)14 b(MPI)h(user)g(lev)o(el)h(ob)s(ject)f(is)g(lik)o(ely)i(to)d(con)o (tain,)h(b)o(y)f(v)m(alue)i(or)189 1645 y(b)o(y)i(reference,)i(an)f (implemen)o(tation-dep)q(endent)i(state)d(information.)30 b(The)19 b(assignmen)o(t)g(and)189 1701 y(cop)o(ying)g(of)f(MPI)h(ob)s (ject)g(handles)h(ma)o(y)e(simply)i(cop)o(y)f(this)g(v)m(alue)h(\(or)e (reference\).)32 b(\()p Fk(End)19 b(of)189 1758 y(advic)n(e)d(to)g (implementors.)p Fn(\))75 1880 y Fh(7.4.3)49 b(Construction)16 b(/)h(Destruction)e(Semantics)75 1965 y Fn(The)g(construction)h(and)f (destruction)h(seman)o(tics)g(in)g(C++)f(should)i(b)q(e)f(the)f(same)g (as)g(those)g(sp)q(eci\014ed)75 2022 y(b)o(y)h(the)h(C)f(and)h(F)l (ortran)e(77)h(bindings.)26 b(That)15 b(is,)i(the)g(MPI)f(user)h(lev)o (el)h(ob)s(jects)e(should)h(b)q(eha)o(v)o(e)g(lik)o(e)75 2078 y(handles.)189 2185 y Fk(A)n(dvic)n(e)g(to)h(implementors.)50 b Fn(Default)17 b(constructors)f(for)h(all)h(MPI)e(ob)s(jects)h(should) h(allo)o(w)f(the)189 2241 y(comparison)e(with)g(a)f(corresp)q(onding)i Ff(MPI)p 936 2241 V 17 w(*)p 977 2241 V 17 w(NULL)e Fn(ob)s(ject)h(to)f (return)h(a)f(b)q(o)q(olean)i Ff(true)p Fn(,)e Ff(MPI)p 1837 2241 V 17 w(-)189 2297 y(IDENT)p Fn(,)d(or)g(whatev)o(er)h(the)g (corresp)q(onding)h(comparison)f(function)h(returns)f(\(see)g(Section)h (7.4.4\).)189 2373 y(The)19 b(destructor)f(for)g(eac)o(h)h(MPI)g(user)g (lev)o(el)h(ob)s(ject)e(should)i Fk(not)e Fn(in)o(v)o(ok)o(e)h(the)g (corresp)q(onding)189 2429 y Ff(MPI)p 264 2429 V 16 w(*)p 304 2429 V 17 w(free\(\))c Fn(function)h(\(if)f(it)h(exists\))f(for)f (the)i(follo)o(wing)g(reasons:)231 2535 y(1.)22 b(Suc)o(h)15 b(a)f(sc)o(heme)h(w)o(ould)g(not)f(b)q(e)i(consisten)o(t)f(with)f(the)h (C)g(and)f(F)l(ortran)g(77)g(functionalit)o(y)l(.)231 2608 y(2.)22 b(The)d(mo)q(del)h(put)f(forth)f(in)i(MPI-1)e(mak)o(es)h (memory)f(allo)q(cation)i(and)f(deallo)q(cation)i(the)289 2665 y(resp)q(onsibilit)o(y)c(of)e(the)g(user,)g(not)g(the)g(implemen)o (tation.)1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 5 5 4 bop 75 -100 a Fl(7.4.)34 b(C++)16 b(BINDINGS)1324 b Fn(5)231 49 y(3.)22 b(MPI)15 b(ob)s(jects)g(going)h(out)f(of)h(scop)q (e)g(ma)o(y)f(pro)q(duce)i(collectiv)o(e)g(op)q(erations,)f(whic)o(h)g (is)g(not)289 106 y(in)o(tuitiv)o(e,)k(and)f(ma)o(y)f(not)h(b)q(e)g (what)f(the)h(user)g(w)o(an)o(ts)f(\(this)h(also)f(a\013ects)g(the)h (cop)o(y)g(and)289 162 y(construction)i(seman)o(tics\).)37 b(The)21 b(comm)o(unicator)g(that)f(is)h(created)g(at)g(the)g(b)q (eginning)289 219 y(of)c(the)i(function)g(sho)o(wn)f(b)q(elo)o(w)h(is)g (freed)f(when)h(the)f(function)h(exits,)g(whic)o(h)h(triggers)d(a)289 275 y(collectiv)o(e)g(op)q(eration:)289 371 y Ff(void)23 b(foo\(\))289 427 y({)336 484 y(MPI_Comm)g(a)h(=)g (MPI_COMM_WORLD.Dup\(\);)336 540 y(//)g(Rest)f(of)h(the)f(function)289 597 y(})189 703 y Fn(\()p Fk(End)15 b(of)i(advic)n(e)f(to)g (implementors.)p Fn(\))75 825 y Fh(7.4.4)49 b(Compa)o(rison)17 b(Syntax)75 911 y Fn(Sev)o(eral)11 b(MPI)f(ob)s(jects)f(already)i(ha)o (v)o(e)e(MPI)i(functions)f(for)g(comparison)g(\(e.g.,)g Ff(MPI)p 1506 911 15 2 v 17 w(Comm::Compare\(\))p Fn(\).)75 967 y(These)19 b(functions)g(t)o(ypically)h(can)e(return)g(more)g(than) h(t)o(w)o(o)e(v)m(alues;)j(a)e(b)q(o)q(olean)i(is)f(not)f(su\016cien)o (t)h(to)75 1023 y(describ)q(e)d(their)f(relationship.)21 b(Therefore,)14 b(the)h(equalit)o(y)g(and)f(inequalit)o(y)i(op)q (erators)e Ff(operator==\(\))75 1080 y Fn(and)h Ff(operator!=\(\))p Fn(\))f(should)i Fk(not)f Fn(b)q(e)h(o)o(v)o(erridden)f(for)g(these)g (ob)s(jects.)166 1219 y Fj(Discussion:)26 b Fi(Ho)o(w)o(ev)o(er,)21 b(man)o(y)d(C)h(and)g(F)m(ortran)h(implem)o(en)o(tations)d(that)i(use)i (p)q(oin)o(ters)f(or)f(in)o(tegers)75 1276 y(as)d(MPI)h(handles)g (indirectly)f(allo)o(w)f(the)i(use)g(of)f(the)h Fd(==)f Fi(and)g Fd(!=)g Fi(op)q(erators;)i(the)f(only)e(meaningful)f(b)q(o)q (olean)75 1332 y(resp)q(onse)i(is)f Fd(true)p Fi(,)e(whic)o(h)i(is)f (equiv)n(alen)o(t)g(to)h Fd(MPI)p 880 1332 14 2 v 15 w(IDENT)p Fi(.)e(It)h(ma)o(y)f(b)q(e)i(desirable)g(to)g(adopt)f(this)h (functionalit)o(y)75 1388 y(in)o(to)e(the)h(C++)h(bindings.)i(F)m(or)c (example,)f(after)i(an)g(assignmen)o(t)e(suc)o(h)j(as)f Fd(a)21 b(=)h(b)p Fi(,)13 b(the)h(test)h Fd(\(a)21 b(==)h(b\))13 b Fi(should)75 1445 y(return)i Fd(true)p Fi(.)166 1584 y Fn(Ho)o(w)o(ev)o(er,)j(sev)o(eral)g(ob)s(jects)f(do)i(not)e(ha)o(v)o (e)h(metho)q(ds)h(equiv)m(alen)o(t)h(to)d(the)h(C)g Ff(MPI)p 1605 1584 15 2 v 17 w(*)p 1646 1584 V 17 w(compare\(\))75 1640 y Fn(functions)d(\(e.g.,)e Ff(MPI)p 458 1640 V 17 w(Datatype)p Fn(\).)18 b(Since)e(no)f(other)f(form)g(of)g(comparison)h (is)g(a)o(v)m(ailable,)h(and)f(since)g(a)75 1697 y(simply)i(b)q(o)q (olean)g(answ)o(er)f(is)g(all)h(that)e(is)i(required,)f(the)g(equalit)o (y)h(and)f(inequalit)o(y)i(op)q(erators)d(should)75 1753 y(b)q(e)h(o)o(v)o(erridden)h(to)e(pro)o(vide)h(this)g(functionalit)o(y) l(.)23 b(This)16 b(is)g(also)g(consisten)o(t)g(with)g(the)g(C)f(and)h (F)l(ortran)75 1810 y(77)f(functionalit)o(y)l(.)166 1866 y(The)i(follo)o(wing)h(table)f(summarizes)g(whic)o(h)h(ob)s(jects)e (should)i(mak)o(e)f(use)g(of)f(the)h(o)o(v)o(erridden)h(op-)75 1923 y(erators:)p 517 1978 916 2 v 516 2035 2 57 v 542 2018 a(Ov)o(erride)e(op)q(erators)p 939 2035 V 49 w(Use)f(MPI)h (functionalit)o(y)p 1432 2035 V 517 2037 916 2 v 516 2093 2 57 v 589 2076 a Ff(MPI)p 664 2076 15 2 v 17 w(Datatype)p 939 2093 2 57 v 224 w(MPI)p 1169 2076 15 2 v 17 w(Comm)p 1432 2093 2 57 v 516 2149 V 660 2132 a(MPI)p 735 2132 15 2 v 17 w(Op)p 939 2149 2 57 v 285 w(MPI)p 1157 2132 15 2 v 17 w(Group)p 1432 2149 2 57 v 516 2206 V 613 2189 a(MPI)p 688 2189 15 2 v 16 w(Status)p 939 2206 2 57 v 1432 2206 V 516 2262 V 601 2245 a(MPI)p 676 2245 15 2 v 17 w(Request)p 939 2262 2 57 v 1432 2262 V 517 2264 916 2 v 75 2366 a Fh(7.4.5)49 b(Constants)75 2452 y Fn(Constan)o(ts)11 b(should)i(b)q(e)g(singleton)g(ob)s(jects;)f(they)g(should)h(either)g (b)q(e)g(declared)g Ff(const)f Fn(or)g(deriv)o(ed)h(from)75 2509 y(the)i(original)i(MPI)e(ob)s(ject)f(and)i(o)o(v)o(erride)f(all)h (non-const)f(functions)h(to)f(prev)o(en)o(t)g(mo)q(di\014cation.)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 6 6 5 bop 75 -100 a Fn(6)977 b Fl(CHAPTER)16 b(7.)34 b(LANGUA)o(GE)15 b(BINDING)75 49 y Fh(7.4.6)49 b(Class)17 b(Lib)o(ra)o(ries)75 135 y Fn(The)g(C++)h(bindings)h(ha)o(v)o(e)d(b)q(een)i(designed)h(to)d (supp)q(ort)h(class)h(libraries;)h(they)e(include)i Ff(const)e Fn(and)75 192 y(reference)f(seman)o(tics.)1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 7 7 6 bop 75 381 a Fp(Bibliograph)m(y)75 604 y Fn([1])22 b(Olivier)17 b(Coulaud)f(and)f(Eric)g(Dillon.)21 b(P)o(ara++:)f(C++)15 b(bindings)i(for)d(message)h(passing)g(libraries)146 660 y(user)g(guide.)21 b(T)l(ec)o(h.)15 b(rep)q(ort,)g(INRIA,)h(1995.) 75 754 y([2])22 b(C++)16 b(F)l(orum.)j(W)l(orking)d(pap)q(er)g(for)e (draft)h(prop)q(osed)h(in)o(ternational)g(standard)f(for)g(information) 146 811 y(systems)e({)g(programming)g(language)h(c++.)k(T)l(ec)o (hnical)d(rep)q(ort,)f(American)g(National)g(Standards)146 867 y(Institute,)h(1995.)75 961 y([3])22 b(Message)15 b(P)o(assing)g(In)o(terface)g(F)l(orum.)21 b(Do)q(cumen)o(t)15 b(for)g(a)g(standard)g(message-passing)h(in)o(terface.)146 1017 y(T)l(ec)o(hnical)21 b(Rep)q(ort)e(T)l(ec)o(hnical)i(Rep)q(ort)e (No.)g(CS-93-214)f(\(revised\),)i(Univ)o(ersit)o(y)g(of)e(T)l (ennessee,)146 1074 y(April)e(1994.)j(Av)m(ailable)e(on)f Fc(netlib)p Fn(.)75 1168 y([4])22 b(Dennis)40 b(Kafura)g(and)f(Liy)o(a) i(Huang.)92 b(mpi++:)70 b(A)40 b(C++)g(language)g(binding)i(for)146 1224 y(MPI.)66 b(In)31 b Fk(Pr)n(o)n(c)n(e)n(e)n(dings)e(MPI)g(develop) n(ers)h(c)n(onfer)n(enc)n(e)p Fn(,)j(Notre)d(Dame,)k(IN,)d(June)g (1995.)146 1280 y(h)o(ttp://www.cse.nd.edu/mpidc95/pro)q(ceedings/pap)q (ers/h)o(tml/h)o(uang/.)75 1374 y([5])22 b(The)g(Annotated)g(C++)h (Reference)h(Man)o(ual.)41 b Fk(Mar)n(gar)n(et)22 b(A.)h(El)r(lis)e (and)i(Bjarne)g(Str)n(oustrup)p Fn(.)146 1431 y(Addison)16 b(W)l(esley)l(,)g(1990.)75 1525 y([6])22 b(An)o(thon)o(y)17 b(Skjellum,)j(Ziy)o(ang)d(Lu,)i(Purushotham)e(V.)g(Bangalore,)h(and)g (Nathan)f(E.)g(Doss.)27 b(Ex-)146 1581 y(plicit)18 b(parallel)g (programming)d(in)i(C++)g(based)g(on)f(the)g(message-passing)h(in)o (terface)f(\(MPI\).)22 b(In)146 1637 y(Gregory)d(V.)i(Wilson,)h (editor,)g Fk(Par)n(al)r(lel)e(Pr)n(o)n(gr)n(amming)h(Using)f(C++)p Fn(.)f(MIT)i(Press,)g(1996.)35 b(in)146 1694 y(press.)75 1788 y([7])22 b(Je\013rey)11 b(M.)f(Squyres,)h(Brian)h(C)701 1776 y(_)687 1788 y(McCandless,)g(and)f(Andrew)g(Lumsdaine.)j(Ob)s (ject-Orien)o(ted)e(MPI.)146 1844 y(CSE)i(T)l(ec)o(hnical)i(Rep)q(ort)e (TR96-10,)g(Univ)o(ersit)o(y)g(of)g(Notre)g(Dame,)f(Marc)o(h)h(1996.)j (Also)d(a)o(v)m(ailable)146 1901 y(at)g Ff(http://www.cse.nd.edu/)729 1884 y Fb(\030)756 1901 y Ff(lsc/research/oompi/)p Fn(.)964 2828 y(7)p eop %%Page: 8 8 8 7 bop 75 361 a Fq(Annex)32 b(A)75 568 y Fp(Language)39 b(Binding)e(\(C++\))75 809 y Fo(A.1)59 b(C++)19 b(Classes)75 910 y Fn(The)c(follo)o(wing)h(are)f(the)h(classes)f(pro)o(vided)h(with) g(the)f(C++)h(language)f(bindings:)75 1004 y Ff(class)23 b(MPI_Comm;)75 1061 y(class)g(MPI_Datatype;)75 1117 y(class)g (MPI_Group;)75 1174 y(class)g(MPI_Op;)75 1230 y(class)g(MPI_Request;)75 1286 y(class)g(MPI_Status;)166 1380 y Fn(Note)15 b(that)f Ff(MPI)p 449 1380 15 2 v 17 w(Status)g Fn(has)h(public)j(data)c(mem)o (b)q(ers,)h(as)g(sp)q(eci\014ed)i(in)f(the)g(C)f(bindings.)75 1523 y Fo(A.2)59 b(De\014ned)20 b(Constants)75 1625 y Fn(These)g(are)g(required)h(constan)o(ts,)f(de\014ned)i(in)f(the)f (\014le)h Ff(mpi++.h)p Fn(.)33 b(F)l(or)20 b(brevit)o(y)l(,)h(the)f(t)o (yp)q(es)g(of)g(the)75 1681 y(constan)o(ts)c(are)h(de\014ned)i(b)q(elo) o(w)f(are)f(de\014ned)i(in)f(the)g(commen)o(ts.)25 b(All)19 b(constan)o(ts)d(are)i(also)f(assumed)75 1738 y(to)e(b)q(e)g Ff(const)p Fn(.)75 1844 y Ff(/*)24 b(return)f(codes)g(\(C,)g(C++,)h (and)f(Fortran\))g(*/)75 1901 y(/*)h(Type:)f(int)g(\(or)h(enum\))f(*/) 75 1957 y(MPI_SUCCESS)75 2014 y(MPI_ERR_BUFFER)75 2070 y(MPI_ERR_COUNT)75 2126 y(MPI_ERR_TYPE)75 2183 y(MPI_ERR_TAG)75 2239 y(MPI_ERR_COMM)75 2296 y(MPI_ERR_RANK)75 2352 y(MPI_ERR_REQUEST)75 2409 y(MPI_ERR_ROOT)75 2465 y(MPI_ERR_GROUP)75 2522 y(MPI_ERR_OP)75 2578 y(MPI_ERR_TOPOLOGY)75 2635 y(MPI_ERR_DIMS)75 2691 y(MPI_ERR_ARG)964 2828 y Fn(8)p eop %%Page: 9 9 9 8 bop 75 -100 a Fl(A.2.)29 b(DEFINED)15 b(CONST)l(ANTS)1156 b Fn(9)75 49 y Ff(MPI_ERR_UNKNOWN)75 106 y(MPI_ERR_TRUNCATE)75 162 y(MPI_ERR_OTHER)75 219 y(MPI_ERR_INTERN)75 275 y(MPI_ERR_LASTCODE) 75 388 y(/*)24 b(assorted)e(constants)h(\(C,)g(C++,)h(and)f(Fortran\))g (*/)75 444 y(/*)h(Type:)f(long)g(*/)75 501 y(MPI_BOTTOM)75 557 y(/*)h(Type:)f(int)g(\(or)h(enum\))f(*/)75 614 y(MPI_PROC_NULL)75 670 y(MPI_ANY_SOURCE)75 727 y(MPI_ANY_TAG)75 783 y(MPI_UNDEFINED)75 840 y(MPI_UB)75 896 y(MPI_LB)75 1009 y(/*)h(Error-handling)e (specifiers)g(\(C,)h(C++,)h(and)f(Fortran\))g(*/)75 1065 y(/*)h(Type:)f(int)g(\(or)h(enum\))f(*/)75 1122 y(MPI_ERRORS_ARE_FATAL) 75 1178 y(MPI_ERRORS_RETURN)75 1291 y(/*)h(Maximum)e(sizes)i(for)f (strings)g(*/)75 1348 y(/*)h(Type:)f(int)g(*/)75 1404 y(MPI_MAX_PROCESSOR_NAME)75 1461 y(MPI_MAX_ERROR_STRING)75 1574 y(/*)h(elementary)e(datatypes)h(\(C)g(and)h(C++\))f(*/)75 1630 y(/*)h(Type:)f(MPI_Datatype)f(*/)75 1686 y(MPI_CHAR)75 1743 y(MPI_SHORT)75 1799 y(MPI_INT)75 1856 y(MPI_LONG)75 1912 y(MPI_UNSIGNED_CHAR)75 1969 y(MPI_UNSIGNED_SHORT)75 2025 y(MPI_UNSIGNED)75 2082 y(MPI_UNSIGNED_LONG)75 2138 y(MPI_FLOAT)75 2195 y(MPI_DOUBLE)75 2251 y(MPI_LONG_DOUBLE)75 2307 y(MPI_BYTE)75 2364 y(MPI_PACKED)75 2477 y(/*)i(datatypes)e(for)i (reduction)e(functions)h(\(C\))g(*/)75 2533 y(/*)h(Type:)f (MPI_Datatype)f(*/)75 2590 y(MPI_FLOAT_INT)75 2646 y(MPI_DOUBLE_INT)75 2703 y(MPI_LONG_INT)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 10 10 9 bop 75 -100 a Fn(10)856 b Fl(ANNEX)15 b(A.)30 b(LANGUA)o(GE)15 b(BINDING)h(\(C++\))75 49 y Ff(MPI_2INT)75 106 y(MPI_SHORT_INT)75 162 y(MPI_LONG_DOUBLE_INT)75 275 y(/*)24 b(optional)e(datatypes)h (\(C\))g(*/)75 332 y(/*)h(Type:)f(MPI_Datatype)f(*/)75 388 y(MPI_LONG_LONG_INT)75 501 y(/*)i(optional)e(datatypes)h(\(C++\))g (*/)75 557 y(/*)h(Type:)f(MPI_Datatype)f(*/)75 614 y(MPI_BOOL)75 670 y(MPI_LONG_DOUBLE)75 727 y(MPI_COMPLEX)75 783 y(MPI_DOUBLE_COMPLEX) 75 840 y(MPI_LONG_DOUBLE_COMPLEX)75 953 y(/*)i(reserved)e (communicators)g(\(C,)i(C++,)f(and)h(Fortran\))e(*/)75 1009 y(/*)i(Type:)f(MPI_Comm)g(*/)75 1065 y(MPI_COMM_WORLD)75 1122 y(MPI_COMM_SELF)75 1235 y(/*)h(results)e(of)i(communicator)e(and)i (group)f(comparisons)f(*/)75 1291 y(/*)i(Type:)f(int)g(\(or)h(enum\))f (*/)75 1348 y(MPI_IDENT)75 1404 y(MPI_CONGRUENT)75 1461 y(MPI_SIMILAR)75 1517 y(MPI_UNEQUAL)75 1630 y(/*)h(environmental)e (inquiry)h(keys)g(\(C,)g(C++,)h(and)f(Fortran\))g(*/)75 1686 y(/*)h(Type:)f(int)g(\(or)h(enum\))f(*/)75 1743 y(MPI_TAG_UB)75 1799 y(MPI_IO)75 1856 y(MPI_HOST)75 1969 y(/*)h(collective)e(operations)h(\(C,)g(C++,)g(and)h(Fortran\))f(*/)75 2025 y(/*)h(Type:)f(MPI_Op)g(*/)75 2082 y(MPI_MAX)75 2138 y(MPI_MIN)75 2195 y(MPI_SUM)75 2251 y(MPI_PROD)75 2307 y(MPI_MAXLOC)75 2364 y(MPI_MINLOC)75 2420 y(MPI_BAND)75 2477 y(MPI_BOR)75 2533 y(MPI_BXOR)75 2590 y(MPI_LAND)75 2646 y(MPI_LOR)75 2703 y(MPI_LXOR)1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 11 11 10 bop 75 -100 a Fl(A.3.)34 b(TYPEDEFS)1388 b Fn(11)75 106 y Ff(/*)24 b(Null)f(handles)g(*/)75 162 y(/*)h(Type:)f(MPI_Group)f (*/)75 219 y(MPI_GROUP_NULL)75 275 y(/*)i(Type:)f(MPI_Comm)g(*/)75 332 y(MPI_COMM_NULL)75 388 y(/*)h(Type:)f(MPI_Datatype)f(*/)75 444 y(MPI_DATATYPE_NULL)75 501 y(/*)i(Type:)f(MPI_Request)f(*/)75 557 y(MPI_REQUEST_NULL)75 614 y(/*)i(Type:)f(MPI_Op)g(*/)75 670 y(MPI_OP_NULL)75 727 y(/*)h(Type:)f(int)g(\(or)h(enum\))f(*/)75 783 y(MPI_ERRHANDLER_NULL)75 896 y(/*)h(Empty)f(group)g(*/)75 953 y(/*)h(Type:)f(MPI_Group)f(*/)75 1009 y(MPI_GROUP_EMPTY)75 1122 y(/*)i(topologies)e(\(C,)i(C++,)f(and)g(Fortran\))g(*/)75 1178 y(/*)h(Type:)f(int)g(\(or)h(enum\))f(*/)75 1235 y(MPI_GRAPH)75 1291 y(MPI_CART)75 1488 y Fo(A.3)59 b(T)-5 b(yp)r(edefs)75 1589 y Fn(The)15 b(follo)o(wing)h(are)f(de\014ned)i (C++)e(t)o(yp)q(es,)g(also)g(included)j(in)e(the)g(\014le)g Ff(mpi++.h)p Fn(.)75 1677 y Ff(/*)24 b(opaque)f(types)g(\(C)g(and)h (C++\))f(*/)75 1734 y(MPI_Aint)75 1847 y(/*)h(prototypes)e(for)i (user-defined)e(functions)g(\(C++\))i(*/)75 1903 y(typedef)f(int)g (MPI_Copy_function\(MPI_Comm)e(&oldcomm,)791 1960 y(MPI_Comm)i (&newcomm,)791 2016 y(int)g(&keyval,)791 2072 y(void)g(*extra_state\);) 75 2185 y(typedef)g(int)g(MPI_Delete_function\(MPI_Comm)d(&comm,)839 2242 y(int)j(&keyval,)839 2298 y(void)g(*extra_state\);)75 2411 y(typedef)g(void)g(MPI_Handler_function\(MPI_Com)o(m)e(&,)j(int)f (&,)886 2468 y(...\);)75 2581 y(typedef)g(void)g (MPI_User_function\(void)e(invec[],)815 2637 y(void)i(inoutvec[],)f (int)i(&len,)815 2693 y(MPI_Datatype)e(&datatype\);)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 12 12 11 bop 75 -100 a Fn(12)856 b Fl(ANNEX)15 b(A.)30 b(LANGUA)o(GE)15 b(BINDING)h(\(C++\))75 49 y Fo(A.4)59 b(C++)19 b(Bindings)g(fo)n(r)h(P) n(oint-to-P)n(oint)h(Communication)75 151 y Ff(int)i(MPI)p 245 151 15 2 v 17 w(Comm::Send\(const)f(void*)h(buf,)g(int)h(count,)f (const)393 207 y(MPI)p 468 207 V 17 w(Datatype&)g(datatype,)f(int)i (dest,)f(int)g(tag\))75 294 y(int)g(MPI)p 245 294 V 17 w(Comm::Recv\(void*)f(buf,)h(int)h(count,)f(const)g(MPI)p 1265 294 V 17 w(Datatype&)f(datatype,)393 350 y(int)i(source,)f(int)g (tag,)g(MPI)p 969 350 V 17 w(Status&)g(status\))75 437 y(int)g(MPI)p 245 437 V 17 w(Comm::Bsend\(const)f(void*)h(buf,)g(int)h (count,)f(const)393 493 y(MPI)p 468 493 V 17 w(Datatype&)g(datatype,)f (int)i(dest,)f(int)g(tag\))75 579 y(int)g(MPI)p 245 579 V 17 w(Comm::Ssend\(const)f(void*)h(buf,)g(int)h(count,)f(const)393 636 y(MPI)p 468 636 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g (tag\))75 722 y(int)g(MPI)p 245 722 V 17 w(Comm::Rsend\(const)f(void*)h (buf,)g(int)h(count,)f(const)393 779 y(MPI)p 468 779 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag\))75 865 y(int)g(MPI)p 245 865 V 17 w(Comm::Isend\(const)f(void*)h(buf,)g (int)h(count,)f(const)393 922 y(MPI)p 468 922 V 17 w(Datatype&)g (datatype,)f(int)i(dest,)f(int)g(tag,)393 978 y(MPI)p 468 978 V 17 w(Request&)g(request\))75 1065 y(int)g(MPI)p 245 1065 V 17 w(Comm::Ibsend\(const)f(void*)h(buf,)g(int)h(count,)f (const)393 1121 y(MPI)p 468 1121 V 17 w(Datatype&)g(datatype,)f(int)i (dest,)f(int)g(tag,)393 1178 y(MPI)p 468 1178 V 17 w(Request&)g (request\))75 1264 y(int)g(MPI)p 245 1264 V 17 w(Comm::Issend\(const)f (void*)h(buf,)g(int)h(count,)f(const)393 1320 y(MPI)p 468 1320 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag,)393 1377 y(MPI)p 468 1377 V 17 w(Request&)g(request\))75 1463 y(int)g(MPI)p 245 1463 V 17 w(Comm::Irsend\(const)f(void*)h(buf,)g (int)h(count,)f(const)393 1520 y(MPI)p 468 1520 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag,)393 1576 y(MPI)p 468 1576 V 17 w(Request&)g(request\))75 1663 y(int)g(MPI)p 245 1663 V 17 w(Comm::Irecv\(void*)f(buf,)h(int)h (count,)f(const)g(MPI)p 1289 1663 V 17 w(Datatype&)f(datatype,)393 1719 y(int)i(source,)f(int)g(tag,)g(MPI)p 969 1719 V 17 w(Request&)g(request\))75 1806 y(int)g(MPI)p 245 1806 V 17 w(Comm::Send)p 502 1806 V 16 w(init\(const)g(void*)g(buf,)g(int)g (count,)h(const)393 1862 y(MPI)p 468 1862 V 17 w(Datatype&)f(datatype,) f(int)i(dest,)f(int)g(tag,)393 1919 y(MPI)p 468 1919 V 17 w(Request&)g(request\))75 2005 y(int)g(MPI)p 245 2005 V 17 w(Comm::Bsend)p 526 2005 V 16 w(init\(const)f(void*)i(buf,)f (int)g(count,)g(const)393 2061 y(MPI)p 468 2061 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag,)393 2118 y(MPI)p 468 2118 V 17 w(Request&)g(request\))75 2204 y(int)g(MPI)p 245 2204 V 17 w(Comm::Ssend)p 526 2204 V 16 w(init\(const)f(void*)i(buf,)f(int)g(count,)g(const)393 2261 y(MPI)p 468 2261 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int) g(tag,)393 2317 y(MPI)p 468 2317 V 17 w(Request&)g(request\))75 2404 y(int)g(MPI)p 245 2404 V 17 w(Comm::Rsend)p 526 2404 V 16 w(init\(const)f(void*)i(buf,)f(int)g(count,)g(const)393 2460 y(MPI)p 468 2460 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int) g(tag,)393 2517 y(MPI)p 468 2517 V 17 w(Request&)g(request\))75 2603 y(int)g(MPI)p 245 2603 V 17 w(Comm::Recv)p 502 2603 V 16 w(init\(void*)g(buf,)g(int)g(count,)g(const)h(MPI)p 1378 2603 V 16 w(Datatype&)f(datatype,)393 2660 y(int)h(source,)f(int)g (tag,)g(MPI)p 969 2660 V 17 w(Request&)g(request\))1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 13 13 12 bop 75 -100 a Fl(A.4.)34 b(C++)16 b(BINDINGS)g(F)o(OR)f (POINT-TO-POINT)i(COMMUNICA)l(TION)333 b Fn(13)75 49 y Ff(int)23 b(MPI)p 245 49 15 2 v 17 w(Comm::Iprobe\(int)f(source,)h (int)g(tag,)h(int&)f(flag,)g(MPI)p 1408 49 V 17 w(Status&)g(status\))75 136 y(int)g(MPI)p 245 136 V 17 w(Comm::Probe\(int)f(source,)h(int)g (tag,)h(MPI)p 1122 136 V 16 w(Status&)f(status\))75 222 y(int)g(MPI)p 245 222 V 17 w(Comm::Sendrecv\(const)e(void)j(*sendbuf,)e (int)i(sendcount,)e(const)393 279 y(MPI)p 468 279 V 17 w(Datatype&)h(sendtype,)f(int)i(dest,)f(int)g(sendtag,)g(void)g (*recvbuf,)393 335 y(int)h(recvcount,)e(const)h(MPI)p 969 335 V 17 w(Datatype&)g(recvtype,)f(int)i(source,)393 392 y(int)g(recvtag,)e(MPI)p 778 392 V 17 w(Status&)h(status\))75 478 y(int)g(MPI)p 245 478 V 17 w(Comm::Sendrecv)p 598 478 V 15 w(replace\(void*)g(buf,)g(int)g(count,)g(const)393 534 y(MPI)p 468 534 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g (sendtag,)g(int)h(source,)393 591 y(int)g(recvtag,)e(MPI)p 778 591 V 17 w(Status&)h(status\))75 677 y(int)g(MPI)p 245 677 V 17 w(Status::Get)p 526 677 V 16 w(count\(const)f(MPI)p 900 677 V 17 w(Datatype&)h(datatype,)f(int&)i(count\))f(const)75 764 y(int)g(MPI)p 245 764 V 17 w(Status::Test)p 550 764 V 16 w(cancelled\(int&)f(flag\))h(const)75 850 y(int)g(MPI)p 245 850 V 17 w(Request::Wait\(MPI)p 670 850 V 15 w(Status&)g(status\)) 75 937 y(int)g(MPI)p 245 937 V 17 w(Request::Test\(int&)f(flag,)h(MPI)p 931 937 V 17 w(Status&)f(status\))h(const)75 1023 y(int)g(MPI)p 245 1023 V 17 w(Request::Request)p 646 1023 V 15 w(free\(void\))75 1110 y(int)g(MPI)p 245 1110 V 17 w(Request::Cancel\(void\))75 1196 y(int)g(MPI)p 245 1196 V 17 w(Request::Start\(void\))75 1283 y(int)g(::MPI)p 293 1283 V 17 w(Buffer)p 454 1283 V 16 w(attach\()g(void*)g(buffer,)g(int)h(size\))75 1369 y(int)f(::MPI)p 293 1369 V 17 w(Buffer)p 454 1369 V 16 w(detach\()g(void&*)g(buffer,)g(int&)g(size\))75 1455 y(int)g(::MPI)p 293 1455 V 17 w(Waitany\(int)f(count,)h(MPI)p 835 1455 V 17 w(Request)g(array)p 1163 1455 V 17 w(of)p 1228 1455 V 16 w(requests[],)g(int&)g(index,)393 1512 y(MPI)p 468 1512 V 17 w(Status&)g(status\))75 1598 y(int)g(::MPI)p 293 1598 V 17 w(Testany\(int)f(count,)h(MPI)p 835 1598 V 17 w(Request)g(array)p 1163 1598 V 17 w(of)p 1228 1598 V 16 w(requests[],)g(int&)g(index,)393 1655 y(int&)g(flag,)h(MPI)p 731 1655 V 16 w(Status&)f(status\))75 1741 y(int)g(::MPI)p 293 1741 V 17 w(Waitall\(int)f(count,)h(MPI)p 835 1741 V 17 w(Request)g(array)p 1163 1741 V 17 w(of)p 1228 1741 V 16 w(requests[],)393 1798 y(MPI)p 468 1798 V 17 w(Status)g(array)p 772 1798 V 16 w(of)p 836 1798 V 17 w(statuses[]\))75 1884 y(int)g(::MPI)p 293 1884 V 17 w(Testall\(int)f(count,)h(MPI)p 835 1884 V 17 w(Request)g(array)p 1163 1884 V 17 w(of)p 1228 1884 V 16 w(requests[],)g(int&)g(flag,)393 1941 y(MPI)p 468 1941 V 17 w(Status)g(array)p 772 1941 V 16 w(of)p 836 1941 V 17 w(statuses[]\))75 2027 y(int)g(::MPI)p 293 2027 V 17 w(Waitsome\(int)f(incount,)h(MPI)p 907 2027 V 17 w(Request&)f(array)p 1258 2027 V 17 w(of)p 1323 2027 V 17 w(requests,)393 2084 y(int&)h(outcount,)g(int)h(array)p 970 2084 V 16 w(of)p 1034 2084 V 17 w(indices[],)393 2140 y(MPI)p 468 2140 V 17 w(Status)f(array)p 772 2140 V 16 w(of)p 836 2140 V 17 w(statuses[]\))75 2226 y(int)g(::MPI)p 293 2226 V 17 w(Testsome\(int)f(incount,)h(MPI)p 907 2226 V 17 w(Request)g(array)p 1235 2226 V 16 w(of)p 1299 2226 V 17 w(requests[],)393 2283 y(int&)g(outcount,)g(int)h(array)p 970 2283 V 16 w(of)p 1034 2283 V 17 w(indices[],)393 2339 y(MPI)p 468 2339 V 17 w(Status)f(array)p 772 2339 V 16 w(of)p 836 2339 V 17 w(statuses[]\))75 2426 y(int)g(::MPI)p 293 2426 V 17 w(Startall\(int)f(count,)h(MPI)p 859 2426 V 17 w(Request)g(array)p 1187 2426 V 16 w(of)p 1251 2426 V 17 w(requests[]\))75 2512 y(int)g(MPI)p 245 2512 V 17 w(Datatype::Type)p 598 2512 V 15 w(contiguous\(int)f(count,)h(MPI)p 1210 2512 V 17 w(Datatype&)g(newtype\))g(const)75 2599 y(int)g(MPI)p 245 2599 V 17 w(Datatype::Type)p 598 2599 V 15 w(vector\(int)g(count,)g(int)g(blocklength,)g(int)g(stride,)393 2655 y(MPI)p 468 2655 V 17 w(Datatype&)g(newtype\))f(const)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 14 14 13 bop 75 -100 a Fn(14)856 b Fl(ANNEX)15 b(A.)30 b(LANGUA)o(GE)15 b(BINDING)h(\(C++\))75 49 y Ff(int)23 b(MPI)p 245 49 15 2 v 17 w(Datatype::Type)p 598 49 V 15 w(hvector\(int)g(count,)g(int) g(blocklength,)g(MPI)p 1545 49 V 16 w(Aint)h(stride,)393 106 y(MPI)p 468 106 V 17 w(Datatype&)f(newtype\)const)75 192 y(int)g(MPI)p 245 192 V 17 w(Datatype::Type)p 598 192 V 15 w(indexed\(int)g(count,)g(int)g(array)p 1282 192 V 17 w(of)p 1347 192 V 17 w(blocklengths[],)393 249 y(int)h(array)p 612 249 V 16 w(of)p 676 249 V 17 w(displacements[],)e (MPI)p 1171 249 V 16 w(Datatype&)h(newtype\))g(const)75 335 y(int)g(MPI)p 245 335 V 17 w(Datatype::Type)p 598 335 V 15 w(hindexed\(int)g(count,)g(int)g(array)p 1306 335 V 17 w(of)p 1371 335 V 17 w(blocklengths[],)393 392 y(MPI)p 468 392 V 17 w(Aint)g(array)p 724 392 V 17 w(of)p 789 392 V 17 w(displacements[],)e(MPI)p 1283 392 V 17 w(Datatype&)i(newtype\))g(const)75 534 y(int)g(::MPI)p 293 534 V 17 w(Type)p 406 534 V 17 w(struct\(int)f(count,)h(int)h (array)p 1068 534 V 16 w(of)p 1132 534 V 17 w(blocklengths[],)393 591 y(MPI)p 468 591 V 17 w(Aint)f(array)p 724 591 V 17 w(of)p 789 591 V 17 w(displacements[],)393 647 y(MPI)p 468 647 V 17 w(Datatype)g(array)p 820 647 V 16 w(of)p 884 647 V 17 w(types[],)g(MPI)p 1188 647 V 17 w(Datatype&)f(newtype\)) 75 734 y(int)h(::MPI)p 293 734 V 17 w(Address\(void*)f(location,)h(MPI) p 955 734 V 16 w(Aint&)h(address\))75 820 y(int)f(MPI)p 245 820 V 17 w(Datatype::Type)p 598 820 V 15 w(extent\(int&)g(extent\)) g(const)75 907 y(int)g(MPI)p 245 907 V 17 w(Datatype::Type)p 598 907 V 15 w(size\(int&)g(size\))g(const)75 993 y(int)g(MPI)p 245 993 V 17 w(Datatype::Type)p 598 993 V 15 w(count\(int&)g(count\))g (const)75 1080 y(int)g(MPI)p 245 1080 V 17 w(Datatype::Type)p 598 1080 V 15 w(lb\(int&)g(displacement\))f(const)75 1166 y(int)h(MPI)p 245 1166 V 17 w(Datatype::Type)p 598 1166 V 15 w(ub\(int&)g(displacement\))f(const)75 1253 y(int)h(MPI)p 245 1253 V 17 w(Datatype::Type)p 598 1253 V 15 w(commit\(void\))75 1339 y(int)g(MPI)p 245 1339 V 17 w(Datatype::Type)p 598 1339 V 15 w(free\(void\))75 1425 y(int)g(MPI)p 245 1425 V 17 w(Datatype::Get)p 574 1425 V 16 w(elements\(MPI)p 878 1425 V 15 w(Status)g(status,)g(int&)g (count\))g(const)75 1512 y(int)g(MPI)p 245 1512 V 17 w(Comm::Pack\(const)f(void*)h(inbuf,)g(int)h(incount,)e(const)393 1568 y(MPI)p 468 1568 V 17 w(Datatype&)h(datatype,)f(void)i(*outbuf,)e (int)i(outsize,)393 1625 y(int&)f(position\))g(const)75 1711 y(int)g(MPI)p 245 1711 V 17 w(Comm::Unpack\(const)f(void*)h (inbuf,)g(int)g(insize,)g(int&)h(position,)393 1768 y(void)f(*outbuf,)g (int)h(outcount,)e(const)i(MPI)p 1280 1768 V 16 w(Datatype&)f (datatype\))393 1824 y(const)75 1911 y(int)g(MPI)p 245 1911 V 17 w(Comm::Pack)p 502 1911 V 16 w(size\(int)g(incount,)g(const)g (MPI)p 1163 1911 V 17 w(Datatype&)f(datatype,)393 1967 y(int&)h(size\))h(const)75 2140 y Fo(A.5)59 b(C++)19 b(Bindings)g(fo)n(r)h(Collective)g(Communication)75 2242 y Ff(int)j(MPI)p 245 2242 V 17 w(Comm::Barrier\(void\))75 2328 y(int)g(MPI)p 245 2328 V 17 w(Comm::Bcast\(void*)f(buffer,)h(int)g (count,)g(const)g(MPI)p 1360 2328 V 17 w(Datatype&)g(datatype,)393 2385 y(int)h(root\))75 2471 y(int)f(MPI)p 245 2471 V 17 w(Comm::Gather\(const)f(void*)h(sendbuf,)g(int)g(sendcount,)g(const) 393 2528 y(MPI)p 468 2528 V 17 w(Datatype&)g(sendtype,)f(void*)h (recvbuf,)g(int)h(recvcount,)e(const)393 2584 y(MPI)p 468 2584 V 17 w(Datatype&)h(recvtype,)f(int)i(root\))75 2671 y(int)f(MPI)p 245 2671 V 17 w(Comm::Gatherv\(const)e(void*)j (sendbuf,)e(int)i(sendcount,)e(const)1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 15 15 14 bop 75 -100 a Fl(A.6.)34 b(C++)16 b(BINDINGS)g(F)o(OR)f(GR)o (OUPS,)h(CONTEXTS,)f(AND)g(COMMUNICA)l(TORS)112 b Fn(15)393 49 y Ff(MPI)p 468 49 15 2 v 17 w(Datatype&)23 b(sendtype,)f(void*)h (recvbuf,)g(int)h(recvcounts[],)393 106 y(int)g(displs[],)e(const)h (MPI)p 945 106 V 17 w(Datatype&)g(recvtype,)f(int)i(root\))75 192 y(int)f(MPI)p 245 192 V 17 w(Comm::Scatter\(const)e(void*)j (sendbuf,)e(int)i(sendcount,)e(const)393 249 y(MPI)p 468 249 V 17 w(Datatype&)h(sendtype,)f(void*)h(recvbuf,)g(int)h (recvcount,)e(const)393 305 y(MPI)p 468 305 V 17 w(Datatype&)h (recvtype,)f(int)i(root\))75 392 y(int)f(MPI)p 245 392 V 17 w(Comm::Scatterv\(const)e(void*)j(sendbuf,)e(int)i(sendcounts[],)e (int)h(displs[],)393 448 y(const)g(MPI)p 611 448 V 17 w(Datatype&)g(sendtype,)f(void*)i(recvbuf,)e(int)i(recvcount,)393 504 y(const)f(MPI)p 611 504 V 17 w(Datatype&)g(recvtype,)f(int)i (root\))75 591 y(int)f(MPI)p 245 591 V 17 w(Comm::Allgather\(const)e (void*)i(sendbuf,)g(int)h(sendcount,)e(const)393 647 y(MPI)p 468 647 V 17 w(Datatype&)h(sendtype,)f(void*)h(recvbuf,)g(int)h (recvcount,)e(const)393 704 y(MPI)p 468 704 V 17 w(Datatype&)h (recvtype\))75 790 y(int)g(MPI)p 245 790 V 17 w (Comm::Allgatherv\(const)e(void*)i(sendbuf,)g(int)g(sendcount,)g(const) 393 847 y(MPI)p 468 847 V 17 w(Datatype&)g(sendtype,)f(void*)h (recvbuf,)g(int)h(recvcounts[],)393 903 y(int)g(displs[],)e(const)h (MPI)p 945 903 V 17 w(Datatype&)g(recvtype\))75 990 y(int)g(MPI)p 245 990 V 17 w(Comm::Alltoall\(const)e(void*)j(sendbuf,)e(int)i (sendcount,)e(const)393 1046 y(MPI)p 468 1046 V 17 w(Datatype&)h (sendtype,)f(void*)h(recvbuf,)g(int)h(recvcount,)e(const)393 1103 y(MPI)p 468 1103 V 17 w(Datatype&)h(recvtype\))75 1189 y(int)g(MPI)p 245 1189 V 17 w(Comm::Alltoallv\(const)e(void*)i (sendbuf,)g(int)h(sendcounts[],)393 1245 y(int)g(sdispls[],)e(const)h (MPI)p 969 1245 V 17 w(Datatype&)g(sendtype,)f(void*)i(recvbuf,)393 1302 y(int)g(recvcounts[],)e(int)h(rdispls[],)g(const)g(MPI)p 1399 1302 V 17 w(Datatype&)f(recvtype\))75 1388 y(int)h(MPI)p 245 1388 V 17 w(Comm::Reduce\(const)f(void*)h(sendbuf,)g(void*)g (recvbuf,)g(int)g(count,)g(const)393 1445 y(MPI)p 468 1445 V 17 w(Datatype&)g(datatype,)f(const)h(MPI)p 1177 1445 V 17 w(Op&)h(op,)f(int)h(root\))75 1531 y(int)f(MPI)p 245 1531 V 17 w(Op::Op)p 406 1531 V 17 w(create\(const)f(MPI)p 805 1531 V 17 w(User)p 918 1531 V 16 w(function&)h(function,)f(int)i (commute\))75 1618 y(int)f(MPI)p 245 1618 V 17 w(Op::Op)p 406 1618 V 17 w(free\(void\))75 1704 y(int)g(MPI)p 245 1704 V 17 w(Comm::Allreduce\(const)e(void*)i(sendbuf,)g(void*)g (recvbuf,)g(int)h(count,)393 1761 y(const)f(MPI)p 611 1761 V 17 w(Datatype&)g(datatype,)f(const)i(MPI)p 1321 1761 V 16 w(Op&)g(op\))75 1847 y(int)f(MPI)p 245 1847 V 17 w(Comm::Reduce)p 550 1847 V 16 w(scatter\(const)f(void*)h (sendbuf,)g(void*)g(recvbuf,)393 1904 y(int)h(recvcounts[],)e(const)h (MPI)p 1041 1904 V 17 w(Datatype&)f(datatype,)h(const)393 1960 y(MPI)p 468 1960 V 17 w(Op&)g(op\))75 2046 y(int)g(MPI)p 245 2046 V 17 w(Comm::Scan\(const)f(void*)h(sendbuf,)g(void*)g (recvbuf,)g(int)g(count,)g(const)393 2103 y(MPI)p 468 2103 V 17 w(Datatype&)g(datatype,)f(const)h(MPI)p 1177 2103 V 17 w(Op&)h(op\))75 2276 y Fo(A.6)59 b(C++)19 b(Bindings)g(fo)n (r)h(Groups,)h(Contexts,)g(and)e(Communicato)n(rs)75 2378 y Ff(int)k(MPI)p 245 2378 V 17 w(Group::Size\(int&)f(size\))h (const)75 2464 y(int)g(MPI)p 245 2464 V 17 w(Group::Rank\(int&)f (rank\))h(const)75 2551 y(int)g(MPI)p 245 2551 V 17 w(Group::Translate) p 646 2551 V 15 w(ranks)g(\(int)h(n,)f(int)h(ranks1[],)e(const)393 2607 y(MPI)p 468 2607 V 17 w(Group&)h(group2,)g(int)g(ranks2[]\))g (const)75 2693 y(int)g(MPI)p 245 2693 V 17 w(Group::Compare\(const)e (MPI)p 835 2693 V 17 w(Group&)i(group2,)g(int&)g(result\))g(const)-32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 16 16 15 bop 75 -100 a Fn(16)856 b Fl(ANNEX)15 b(A.)30 b(LANGUA)o(GE)15 b(BINDING)h(\(C++\))75 49 y Ff(int)23 b(MPI)p 245 49 15 2 v 17 w(Comm::Group\(MPI)p 622 49 V 15 w(Group&)g(group\))g(const) 75 136 y(int)g(MPI)p 245 136 V 17 w(Group::Union\(const)f(MPI)p 788 136 V 16 w(Group&)h(group2,)g(MPI)p 1234 136 V 17 w(Group&)g(newgroup\))g(const)75 222 y(int)g(MPI)p 245 222 V 17 w(Group::Intersection\(const)e(MPI)p 955 222 V 16 w(Group&)i(group2,)g(MPI)p 1401 222 V 17 w(Group&)g(newgroup\))393 279 y(const)75 365 y(int)g(MPI)p 245 365 V 17 w (Group::Difference\(const)e(MPI)p 907 365 V 17 w(Group&)i(group2,)g (MPI)p 1354 365 V 16 w(Group&)g(newgroup\))393 422 y(const)75 508 y(int)g(MPI)p 245 508 V 17 w(Group::Incl\(int)f(n,)i(int)f (ranks[],)g(MPI)p 1098 508 V 17 w(Group&)g(newgroup\))f(const)75 594 y(int)h(MPI)p 245 594 V 17 w(Group::Excl\(int)f(n,)i(int)f (ranks[],)g(MPI)p 1098 594 V 17 w(Group&)g(newgroup\))f(const)75 681 y(int)h(MPI)p 245 681 V 17 w(Group::Range)p 550 681 V 16 w(incl\(int)g(n,)g(int)h(ranges[][3],)e(MPI)p 1330 681 V 17 w(Group&)h(newgroup\))f(const)75 824 y(int)h(MPI)p 245 824 V 17 w(Group::Range)p 550 824 V 16 w(excl\(int)g(n,)g(int)h (ranges[][3],)e(MPI)p 1330 824 V 17 w(Group&)h(newgroup\))f(const)75 967 y(int)h(MPI)p 245 967 V 17 w(Group::Free\(void\))75 1053 y(int)g(MPI)p 245 1053 V 17 w(Comm::Size\(int&)f(size\))h(const)75 1140 y(int)g(MPI)p 245 1140 V 17 w(Comm::Rank\(int&)f(rank\))h(const)75 1226 y(int)g(MPI)p 245 1226 V 17 w(Comm::Compare\(const)e(MPI)p 811 1226 V 17 w(Comm&)i(comm1,)g(int&)h(result\))f(const)75 1313 y(int)g(MPI)p 245 1313 V 17 w(Comm::Dup\(const)f(MPI)p 716 1313 V 17 w(Comm&)h(comm,)g(MPI)p 1091 1313 V 17 w(Comm&)g(newcomm\))g(const)75 1399 y(int)g(MPI)p 245 1399 V 17 w(Comm::Create\(const)f(MPI)p 788 1399 V 16 w(Group&)h(group,)g(MPI)p 1210 1399 V 17 w(Comm&)g(newcomm\))g(const)75 1485 y(int)g(MPI)p 245 1485 V 17 w(Comm::Split\(int)f(color,)h(int)h (key,)f(MPI)p 1098 1485 V 17 w(Comm&)g(newcomm\))g(const)75 1572 y(int)g(MPI)p 245 1572 V 17 w(Comm::Free\(void\))75 1658 y(int)g(MPI)p 245 1658 V 17 w(Comm::Test)p 502 1658 V 16 w(inter\(int&)g(flag\))g(const)75 1745 y(int)g(MPI)p 245 1745 V 17 w(Comm::Remote)p 550 1745 V 16 w(size\(int&)f(size\))i (const)75 1831 y(int)f(MPI)p 245 1831 V 17 w(Comm::Remote)p 550 1831 V 16 w(group\(MPI)p 782 1831 V 16 w(Group&)g(group\))g(const) 75 1918 y(int)g(MPI)p 245 1918 V 17 w(Comm::Intercomm)p 622 1918 V 15 w(create\(int)g(local)p 1020 1918 V 16 w(leader,)g(const)g(MPI)p 1442 1918 V 17 w(Comm&)g(peer)p 1698 1918 V 17 w(comm,)393 1974 y(int)h(remote)p 636 1974 V 16 w(leader,)f(int)g(tag,)h(MPI)p 1130 1974 V 16 w(Comm&)g(newintercomm\))e(const)75 2061 y(int)h(MPI)p 245 2061 V 17 w(Comm::Intercomm)p 622 2061 V 15 w(merge\(int)g(high,)g (MPI)p 1091 2061 V 17 w(Comm&)g(newintracomm\))f(const)75 2147 y(int)h(::MPI)p 293 2147 V 17 w(Keyval)p 454 2147 V 16 w(create\(const)g(MPI)p 853 2147 V 16 w(Copy)p 965 2147 V 17 w(function)g(*copy)p 1317 2147 V 16 w(fn,)h(const)393 2204 y(MPI)p 468 2204 V 17 w(Delete)p 629 2204 V 16 w(function)f (*delete)p 1028 2204 V 16 w(fn,)h(int&)f(keyval,)g(void*)g(extra)p 1713 2204 V 17 w(state\))75 2290 y(int)g(::MPI)p 293 2290 V 17 w(Keyval)p 454 2290 V 16 w(free\(int&)g(keyval\))75 2376 y(int)g(MPI)p 245 2376 V 17 w(Comm::Attr)p 502 2376 V 16 w(put\(int)g(keyval,)g(void*)g(attribute)p 1259 2376 V 16 w(val\))75 2463 y(int)g(MPI)p 245 2463 V 17 w(Comm::Attr)p 502 2463 V 16 w(get\(int)g(keyval,)g(void&)g(*attribute) p 1283 2463 V 16 w(val,)g(int&)g(flag\))h(const)75 2549 y(int)f(MPI)p 245 2549 V 17 w(Comm::Attr)p 502 2549 V 16 w(delete\(int)g(keyval\))1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 17 17 16 bop 75 -100 a Fl(A.7.)34 b(C++)16 b(BINDINGS)g(F)o(OR)f(PR)o (OCESS)h(TOPOLOGIES)630 b Fn(17)75 49 y Fo(A.7)59 b(C++)19 b(Bindings)g(fo)n(r)h(Pro)r(cess)h(T)-5 b(op)r(ologies)75 151 y Ff(int)23 b(MPI)p 245 151 15 2 v 17 w(Comm::Cart)p 502 151 V 16 w(create\(int)g(ndims,)g(int)g(dims[],)g(int)g(periods[],) 393 207 y(int)h(reorder,)e(MPI)p 778 207 V 17 w(Comm&)h(comm)p 1034 207 V 17 w(cart\))g(const)75 294 y(int)g(::MPI)p 293 294 V 17 w(Dims)p 406 294 V 17 w(create\(int)f(nnodes,)h(int)g (ndims,)g(int)h(dims[]\))f(const)75 380 y(int)g(MPI)p 245 380 V 17 w(Comm::Graph)p 526 380 V 16 w(create\(int)f(nnodes,)h (int)h(index[],)f(int)g(edges[],)393 437 y(int)h(reorder,)e(MPI)p 778 437 V 17 w(Comm&)h(comm)p 1034 437 V 17 w(graph\))g(const)75 523 y(int)g(MPI)p 245 523 V 17 w(Comm::Topo)p 502 523 V 16 w(test\(int&)g(status\))g(const)75 609 y(int)g(MPI)p 245 609 V 17 w(Comm::Graphdims)p 622 609 V 15 w(get\(int)g(nnodes[],)g (int)g(nedges[]\))g(const)75 696 y(int)g(MPI)p 245 696 V 17 w(Comm::Graph)p 526 696 V 16 w(get\(int)g(maxindex,)f(int)i (maxedges,)f(int)g(index[],)393 752 y(int)h(edges[]\))e(const)75 839 y(int)h(MPI)p 245 839 V 17 w(Comm::Cartdim)p 574 839 V 16 w(get\(int&)f(ndims\))h(const)75 925 y(int)g(MPI)p 245 925 V 17 w(Comm::Cart)p 502 925 V 16 w(get\(int)g(maxdims,)g(int)g (dims[],)g(int)h(periods[],)393 982 y(int)g(coords[]\))e(const)75 1068 y(int)h(MPI)p 245 1068 V 17 w(Comm::Cart)p 502 1068 V 16 w(rank\(int)g(coords[],)f(int&)i(rank\))f(const)75 1155 y(int)g(MPI)p 245 1155 V 17 w(Comm::Cart)p 502 1155 V 16 w(coords\(int)g(rank,)g(int)g(maxdims,)g(int)g(coords[]\))g(const) 75 1241 y(int)g(MPI)p 245 1241 V 17 w(Comm::Graph)p 526 1241 V 16 w(neighbors)p 758 1241 V 16 w(count\(int)f(rank,)i(int&)f (nneighbors\))f(const)75 1328 y(int)h(MPI)p 245 1328 V 17 w(Comm::Graph)p 526 1328 V 16 w(neighbors\(int)f(rank,)h(int)h (maxneighbors,)e(int)h(neighbors[]\))393 1384 y(const)75 1470 y(int)g(MPI)p 245 1470 V 17 w(Comm::Cart)p 502 1470 V 16 w(shift\(int)g(direction,)f(int)i(disp,)f(int&)g(rank)p 1473 1470 V 17 w(source,)393 1527 y(int&)g(rank)p 611 1527 V 17 w(dest\))g(const)75 1613 y(int)g(MPI)p 245 1613 V 17 w(Comm::Cart)p 502 1613 V 16 w(sub\(int)g(remain)p 853 1613 V 16 w(dims[],)g(MPI)p 1132 1613 V 17 w(Comm&)g(newcomm\))g (const)75 1700 y(int)g(MPI)p 245 1700 V 17 w(Comm::Cart)p 502 1700 V 16 w(map\(int)g(ndims,)g(int)g(dims[],)g(int)h(periods[],)e (int&)i(newrank\))393 1756 y(const)75 1843 y(int)f(MPI)p 245 1843 V 17 w(Comm::Graph)p 526 1843 V 16 w(map\(int)g(nnodes,)g(int) g(index[],)g(int)g(edges[],)g(int&)g(newrank\))393 1899 y(const)75 2072 y Fo(A.8)59 b(C++)19 b(Bindings)g(fo)n(r)h (Environmental)g(Inquiry)75 2174 y Ff(int)j(::MPI)p 293 2174 V 17 w(Get)p 382 2174 V 17 w(processor)p 615 2174 V 16 w(name\(char)f(*name,)h(int&)h(resultlen\))75 2260 y(int)f(::MPI)p 293 2260 V 17 w(Errhandler)p 550 2260 V 16 w(create\(const)f(MPI)p 948 2260 V 17 w(Handler)p 1133 2260 V 16 w(function)h(*function,)393 2317 y(MPI)p 468 2317 V 17 w(Errhandler&)f(errhandler\))75 2403 y(int)h(MPI)p 245 2403 V 17 w(Comm::Errhandler)p 646 2403 V 15 w(set\(const)g(MPI)p 972 2403 V 17 w(Errhandler&)f(errhandler\))75 2490 y(int)h(MPI)p 245 2490 V 17 w(Comm::Errhandler)p 646 2490 V 15 w(get\(MPI)p 829 2490 V 16 w(Errhandler&)g(errhandler\))f(const)75 2576 y(int)h(::MPI)p 293 2576 V 17 w(Errhandler)p 550 2576 V 16 w(free\(MPI)p 758 2576 V 16 w(Errhandler&)f(errhandler\))75 2663 y(int)h(::MPI)p 293 2663 V 17 w(Error)p 430 2663 V 16 w(string\(int)g(errorcode,)f(char)i(*string,)f(int&)g(resultlen\)) -32 46 y Fg(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 18 18 17 bop 75 -100 a Fn(18)856 b Fl(ANNEX)15 b(A.)30 b(LANGUA)o(GE)15 b(BINDING)h(\(C++\))75 49 y Ff(int)23 b(::MPI)p 293 49 15 2 v 17 w(Error)p 430 49 V 16 w(class\(int)g(errorcode,)g(int&)g (errorclass\))75 136 y(double)g(::MPI)p 365 136 V 17 w(Wtime\(void\))75 222 y(double)g(::MPI)p 365 222 V 17 w(Wtick\(void\))75 309 y(int)g(::MPI)p 293 309 V 17 w(Init\(int&)g (argc,)g(char&)g(*argv[]\))75 395 y(int)g(::MPI)p 293 395 V 17 w(Finalize\(void\))75 482 y(int)g(::MPI)p 293 482 V 17 w(Initialized\(int&)f(flag\)const)75 568 y(int)h(MPI)p 245 568 V 17 w(Comm::Abort\(int)f(errorcode\))75 741 y Fo(A.9)59 b(C++)19 b(Bindings)g(fo)n(r)h(Pro\014ling)75 843 y Ff(int)j(::MPI)p 293 843 V 17 w(Pcontrol\(const)f(int)h(level,)g Fa(:)8 b(:)g(:)e Ff(\))1967 46 y Fg(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Sun Apr 21 23:51:17 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA12674; Sun, 21 Apr 1996 23:50:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA18681 for mpi-core-out; Sun, 21 Apr 1996 22:53:25 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA18676; Sun, 21 Apr 1996 22:53:14 -0500 Message-Id: <199604220353.WAA18676@antares.mcs.anl.gov> To: mpi-misc@antares.mcs.anl.gov, mpi-core@antares.mcs.anl.gov Subject: Miscellany Chapter Date: Sun, 21 Apr 1996 22:53:11 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the latest version of the Miscellany chapter. Changes since the March meeting are: Two new proposals in the datatype section, one for sending Datatypes themselves and another for a simplified type_indexed. New Interlanguage Interoperability section Discussion of MPI_Type_size removed A proposal about threads from Tony Skjellum A discussion of Volatile and Caching Issues from Rolf Rabenseifner, probably really belongs in 1-sided chapter. %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Sun Apr 21 22:45:10 1996 %%Pages: 29 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.04.21:2244 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 3 86 df<7FE000FFF0007FE0000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E001C0E001C0E001C0E001C0E001C7FFFFCFFFFFC7FFFFC161E7F9D1A>76 D78 D85 D E /Fb 1 16 df<03C00FF01FF83FFC7FFE7FFE FFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF80FF003C010107E9115>15 D E /Fc 19 122 df<60F0F0701010101020204080040C7C830C>59 D<0000038000000F0000003C000000F0000003C000000F0000003C000000F0000003C000 000F0000003C000000F0000000F00000003C0000000F00000003C0000000F00000003C00 00000F00000003C0000000F00000003C0000000F000000038019187D9520>I62 D<03FFC0003C0000380000380000380000380000700000700000700000700000E00000E0 0000E00000E00001C00001C00001C00001C0000380000380000380000380000700000700 000700000700000F0000FFF000121C7E9B12>73 D<01FE0000FF003E0000F0002E0001E0 002E0002E0002E0002E0002E0004E0004E0009C0004E0009C000470011C000470011C000 870023800087004380008700438000870083800107010700010701070001038207000103 8207000203840E000203880E000203880E000203900E000403A01C000403A01C000401C0 1C000C01C01C001C01803C00FF8103FF80281C7E9B28>77 D<01FFFF00003C03C0003800 E0003800F00038007000380070007000F0007000F0007000F0007000E000E001E000E003 C000E0078000E01E0001FFF00001C0000001C0000001C000000380000003800000038000 0003800000070000000700000007000000070000000F000000FFE000001C1C7E9B1B>80 D<01E3000717000C0F00180F00380E00300E00700E00700E00E01C00E01C00E01C00E01C 00E03880E03880E038806078803199001E0E0011127E9116>97 D<01F0030C0E0C1C1E38 3C301870007000E000E000E000E000E000E0046008601030601F800F127E9112>99 D<0007E00000E00000E00001C00001C00001C00001C000038000038000038000038001E7 000717000C0F00180F00380E00300E00700E00700E00E01C00E01C00E01C00E01C00E038 80E03880E038806078803199001E0E00131D7E9C16>I<01F007080C0818043808300870 307FC0E000E000E000E000E000E0046008601030600F800E127E9113>I<007180018B80 0307800607800E07000C07001C07001C0700380E00380E00380E00380E00381C00381C00 381C00183C0008F800073800003800003800007000607000F06000F0E000E180007E0000 111A7F9114>103 D<1F800380038007000700070007000E000E000E000E001C001C001C 001C0038003800380038007000700070007000E400E400E400E40064003800091D7E9C0C >108 D<381F81F04E20C6184640E81C4680F01C8F00F01C8E00E01C0E00E01C0E00E01C 1C01C0381C01C0381C01C0381C01C0703803807138038071380380E1380380E270070064 3003003820127E9124>I<381F004E61804681C04701C08F01C08E01C00E01C00E01C01C 03801C03801C03801C0700380710380710380E10380E2070064030038014127E9119>I< 07078009C86008D03008E03011C03011C03801C03801C038038070038070038070038060 0700E00700C00701800783000E86000E78000E00000E00001C00001C00001C00001C0000 3C0000FF8000151A819115>112 D<383C4E424687470F8E1E8E0C0E000E001C001C001C 001C0038003800380038007000300010127E9113>114 D<00C001C001C001C003800380 03800380FFF00700070007000E000E000E000E001C001C001C001C003820382038403840 18800F000C1A80990F>116 D<1C00C02701C04701C04701C08703808703800E03800E03 801C07001C07001C07001C0700180E20180E20180E201C1E200C264007C38013127E9118 >I<1C00C02701C04701C04701C08703808703800E03800E03801C07001C07001C07001C 0700180E00180E00180E001C1E000C3C0007DC00001C00001800603800F03000F06000E0 C0004180003E0000121A7E9114>121 D E /Fd 75 126 df<60F0F0F0F0F0F0F0F0F0F0 F0F0F0F0F0600000000060F0F0600419779816>33 D<0387000387000387000387000387 000387007FFFC0FFFFE0FFFFE0070E00070E00070E000E1C000E1C000E1C000E1C00FFFF E0FFFFE07FFFC01C38001C38001C38001C38001C38001C380013197F9816>35 D<30787C3C1C1C1C1C3878F0E040060D789816>39 D<00E001E0038007000E001C001C00 38003800700070007000E000E000E000E000E000E000E000E000E0007000700070003800 38001C001C000E000700038001E000E00B217A9C16>II<01C00001C00001C00001C000 71C700F9CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF8071C70001C00001C000 01C00001C00011127E9516>I<01C00001C00001C00001C00001C00001C00001C00001C0 00FFFF80FFFF80FFFF8001C00001C00001C00001C00001C00001C00001C00001C0001113 7E9516>I<387C7E7E3E0E1E1C78F060070B798416>II<70F8F8F8700505788416>I<000180000380000380000700000700000E00000E00001C 00001C0000380000380000700000700000E00000E00001C00001C0000380000380000700 000700000E00000E00001C00001C0000380000380000700000700000E00000E00000C000 0011207E9C16>I<03E0000FF8001FFC001E3C00380E00780F00700700700700E00380E0 0380E00380E00380E00380E00380E00380E00380F00780700700700700780F003C1E001E 3C001FFC000FF80003E00011197E9816>I<01800380038007800F807F80FF8073800380 03800380038003800380038003800380038003800380038003807FF87FFC7FF80E197C98 16>I<07E0001FF8003FFC00783E00E00700F00780F00380600380000380000380000700 000700000E00001C0000380000700000E00001C0000380000F00001E03803803807FFF80 FFFF807FFF8011197E9816>I<3FFE003FFE003FFE003800003800003800003800003800 003800003800003BF0003FFC003FFE003C0F00300700000380000380600380F00380F003 80E00700781E003FFC001FF80007E00011197E9816>53 D<07F0001FFC003FFE007C1F00 F00780E00380E00380E003807007007C1F001FFC0007F0001FFC003C1E00700700F00780 E00380E00380E00380F007807007007C1F003FFE001FFC0007F00011197E9816>56 D<70F8F8F870000000000000000070F8F8F8700512789116>58 D<387C7C7C3800000000 0000000038787C7C3C1C1C3870E0400618799116>I<7FFF00FFFF80FFFF800000000000 00000000000000000000FFFF80FFFF807FFF00110B7E9116>61 D<0FE03FF87FFCF01EF0 0EF00E601E003C007800F001C00380038003800380038003000000000000000000030007 80078003000F197D9816>63 D<00E00001F00001F00001B00001B00003B80003B80003B8 00031800071C00071C00071C00071C00071C000E0E000E0E000FFE000FFE001FFF001C07 001C07001C07007F1FC0FF1FE07F1FC013197F9816>65 D<7FF800FFFE007FFF001C0F00 1C07801C03801C03801C03801C07801C07001FFF001FFE001FFE001C1F001C03801C03C0 1C01C01C01C01C01C01C01C01C03C01C07807FFF80FFFF007FFC0012197F9816>I<01F1 8007FB800FFF801F0F803C0780380380700380700380F00000E00000E00000E00000E000 00E00000E00000E00000F000007003807003803803803C07001F0F000FFE0007FC0001F0 0011197E9816>I<7FF800FFFE007FFF001C0F001C07801C03C01C01C01C01C01C01E01C 00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C01C01C01C01C03C01C07801C 0F807FFF00FFFE007FF8001319809816>I<7FFFC0FFFFC07FFFC01C01C01C01C01C01C0 1C01C01C00001C00001C1C001C1C001FFC001FFC001FFC001C1C001C1C001C00001C00E0 1C00E01C00E01C00E01C00E07FFFE0FFFFE07FFFE013197F9816>II<03E30007FF000FFF001E1F003C0F00380700700700700700F00000E00000E00000E0 0000E00000E03F80E07FC0E03F80F00700700700700700380F003C0F001E1F000FFF0007 F70003E70012197E9816>I<7F1FC0FFBFE07F1FC01C07001C07001C07001C07001C0700 1C07001C07001FFF001FFF001FFF001C07001C07001C07001C07001C07001C07001C0700 1C07001C07007F1FC0FFBFE07F1FC013197F9816>II<7F0FE0FF8FF07F0FE01C07801C0F001C0E001C1C001C3C001C78001CF0 001CE0001DF0001FF0001FF8001F38001E1C001C1C001C0E001C0E001C07001C07001C03 807F07E0FF8FF07F07E01419809816>75 DII<7E1FC0FF3FE07F1FC01D07001D87001D87001D87001DC7001DC7001CC7001CC7001C E7001CE7001CE7001C67001C67001C77001C77001C37001C37001C37001C17007F1F00FF 9F007F0F0013197F9816>I<1FFC003FFE007FFF00780F00F00780E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780 F00780780F007FFF003FFE001FFC0011197E9816>I<7FF800FFFE007FFF001C0F801C03 801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FF8001C00001C00 001C00001C00001C00001C00001C00007F0000FF80007F000012197F9816>I<1FFC003F FE007FFF00780F00F00780E00380E00380E00380E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E0E380E1E380F0F780787F007FFF003FFE001FFC0000 1C00001E00000E00000F00000700000700111F7E9816>I<7FE000FFF8007FFC001C1E00 1C0F001C07001C07001C07001C07001C0F001C1E001FFC001FF8001FFC001C1C001C0E00 1C0E001C0E001C0E001C0E201C0E701C0E707F07E0FF87E07F03C014197F9816>I<07E3 001FFF003FFF00781F00F00700E00700E00700E00000F000007800003F80001FF00007FC 0000FE00000F00000700000380000380600380E00380E00700F80F00FFFE00FFFC00C7F0 0011197E9816>I<7FFFE0FFFFE0FFFFE0E0E0E0E0E0E0E0E0E0E0E0E000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00007FC000FFE0007FC0013197F9816>I<7F07F0FF8FF87F07F01C01C01C01C01C01C0 1C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C0 1C01C00E03800E038007070007FF0003FE0000F8001519809816>III89 D<7FFF80FFFF80FFFF80E00700E0 0F00E01E00E01C00003C0000780000700000F00001E00001C00003C0000780000700000F 00001E03801C03803C0380780380700380FFFF80FFFF80FFFF8011197E9816>II93 D95 D<1FE0003FF0007FF800783C00300E00000E00 000E0003FE001FFE003E0E00700E00E00E00E00E00E00E00783E007FFFE03FE7E00F83E0 13127E9116>97 D<7E0000FE00007E00000E00000E00000E00000E00000E3E000EFF000F FF800F83C00F00E00E00E00E00700E00700E00700E00700E00700E00700E00E00F01E00F 83C00FFF800EFF00063C001419809816>I<03F80FFC1FFE3C1E780C7000E000E000E000 E000E000F000700778073E0E1FFC0FF803F010127D9116>I<003F00007F00003F000007 0000070000070000070003C7000FF7001FFF003C1F00780F00700700E00700E00700E007 00E00700E00700E00700700F00700F003C1F001FFFE00FE7F007C7E014197F9816>I<03 E00FF81FFC3C1E780E7007E007FFFFFFFFFFFFE000E000700778073C0F1FFE0FFC03F010 127D9116>I<001F00007F8000FF8001E78001C30001C00001C0007FFF00FFFF00FFFF00 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 3FFE007FFF003FFE0011197F9816>I<03E3C007F7E00FFFE01C1CC0380E00380E00380E 00380E00380E001C1C000FF8001FF0001BE0003800001800001FFC001FFF003FFF807803 C0E000E0E000E0E000E0E000E07001C07C07C03FFF800FFE0003F800131C7F9116>I<7E 0000FE00007E00000E00000E00000E00000E00000E3C000EFE000FFF000F87800F03800E 03800E03800E03800E03800E03800E03800E03800E03800E03800E03807FC7F0FFE7F87F C7F01519809816>I<018003C003C0018000000000000000007FC07FC07FC001C001C001 C001C001C001C001C001C001C001C001C001C07FFFFFFF7FFF101A7D9916>I<00300078 0078003000000000000000001FF81FF81FF8003800380038003800380038003800380038 00380038003800380038003800380038003800386070F0F0FFE07FC03F800D237E9916> I<7E0000FE00007E00000E00000E00000E00000E00000E7FE00E7FE00E7FE00E0F000E1E 000E3C000E78000EF0000FF0000FF8000FBC000F1E000E0E000E07000E07807F87F0FFCF F07F87F01419809816>III<7E3C00FEFE007FFF000F87800F03800E03800E03800E03800E03 800E03800E03800E03800E03800E03800E03807FC7F0FFE7F87FC7F01512809116>I<03 E0000FF8001FFC003C1E00780F00700700E00380E00380E00380E00380E00380F0078070 0700780F003C1E001FFC000FF80003E00011127E9116>I<7E3E00FEFF007FFF800F83C0 0F00E00E00E00E00700E00700E00700E00700E00700E00700E00E00F01E00F83C00FFF80 0EFF000E3C000E00000E00000E00000E00000E00000E00007FC000FFE0007FC000141B80 9116>I<07C7000FE7001FF7003C1F00700F00700F00E00700E00700E00700E00700E007 00E00700700F00700F003C3F003FF7001FE70007C7000007000007000007000007000007 00000700003FE0007FF0003FE0141B7E9116>II<0FEC3FFC7FFCF03CE01CE01C70007F801FF007F8003C600EE00EF0 0EF81EFFFCFFF8C7E00F127D9116>I<0300000700000700000700000700007FFF00FFFF 00FFFF000700000700000700000700000700000700000700000701000703800703800703 8007870003FE0001FC0000F80011177F9616>I<7E1F80FE3F807E1F800E03800E03800E 03800E03800E03800E03800E03800E03800E03800E03800E03800E0F800FFFF007FBF803 E3F01512809116>I<7F1FC0FF1FE07F1FC01C07001E0F000E0E000E0E000E0E00071C00 071C00071C00071C0003B80003B80003B80001F00001F00000E00013127F9116>II<7F1FC07F3FC07F1FC00F1C0007 3C0003B80003F00001F00000E00001E00001F00003B800073C00071C000E0E007F1FC0FF 3FE07F1FC013127F9116>I<7F1FC0FF9FE07F1FC01C07000E07000E0E000E0E00070E00 071C00071C00039C00039C0003980001B80001B80000F00000F00000F00000E00000E000 00E00001C00079C0007BC0007F80003F00003C0000131B7F9116>I<3FFFC07FFFC07FFF C0700780700F00701E00003C0000780001F00003E0000780000F00001E01C03C01C07801 C0FFFFC0FFFFC0FFFFC012127F9116>I<001F80007F8000FF8001E00001C00001C00001 C00001C00001C00001C00001C00001C00001C00003C0007F8000FF0000FF00007F800003 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001E00000FF8000 7F80001F8011207E9C16>I<7C0000FF0000FF800003C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001E00000FF00007F80007F8000FF0001E00001C000 01C00001C00001C00001C00001C00001C00001C00001C00003C000FF8000FF00007C0000 11207E9C16>125 D E /Fe 19 119 df<3078F06005047C830D>46 D<0000180000001800000038000000380000007800000078000000B8000001B800000138 000002380000023C0000041C0000041C0000081C0000181C0000101C0000201C0000201C 00007FFC0000401C0000801C0001801C0001001C0002001C0002001C0004000E000C000E 001C001E00FF00FFC01A1D7E9C1F>65 D<01FFFFE0003C00E00038006000380040003800 40003800400070004000700040007020400070200000E0400000E0400000E0C00000FFC0 0001C0800001C0800001C0800001C0800003810100038001000380020003800200070004 000700040007000C00070018000E007800FFFFF0001B1C7D9B1C>69 D<03CC063C0C3C181C3838303870387038E070E070E070E070E0E2C0E2C0E261E462643C 380F127B9115>97 D<01F007080C08181C3838300070007000E000E000E000E000E000E0 08E010602030C01F000E127B9113>99 D<001F8000038000038000070000070000070000 0700000E00000E00000E00000E0003DC00063C000C3C00181C0038380030380070380070 3800E07000E07000E07000E07000E0E200C0E200C0E20061E4006264003C3800111D7B9C 15>I<01E007100C1018083810701070607F80E000E000E000E000E000E0086010602030 C01F000D127B9113>I<0003C0000670000C70001C60001C00001C000038000038000038 0000380000380003FF8000700000700000700000700000700000E00000E00000E00000E0 0000E00001C00001C00001C00001C00001C0000380000380000380000300000300000700 00C60000E60000CC00007800001425819C0D>I<01800380010000000000000000000000 000000001C002600470047008E008E000E001C001C001C00380038007100710071007200 72003C00091C7C9B0D>105 D<1F800380038007000700070007000E000E000E000E001C 001C001C001C0038003800380038007000700070007000E400E400E400E4006800380009 1D7C9C0B>108 D<3C1E0780266318C04683A0E04703C0E08E0380E08E0380E00E0380E0 0E0380E01C0701C01C0701C01C0701C01C070380380E0388380E0388380E0708380E0710 701C0320300C01C01D127C9122>I<3C3C002646004687004707008E07008E07000E0700 0E07001C0E001C0E001C0E001C1C00381C40381C40383840383880701900300E0012127C 9117>I<01E007180C0C180C380C300E700E700EE01CE01CE01CE018E038E030E06060C0 31801E000F127B9115>I<07870004D98008E0C008E0C011C0E011C0E001C0E001C0E003 81C00381C00381C00381800703800703000707000706000E8C000E70000E00000E00001C 00001C00001C00001C00003C0000FF8000131A7F9115>I<3C3C26C2468747078E068E00 0E000E001C001C001C001C0038003800380038007000300010127C9112>114 D<01F006080C080C1C18181C001F001FC00FF007F0007800386030E030C030806060C01F 000E127D9111>I<00C001C001C001C00380038003800380FFE00700070007000E000E00 0E000E001C001C001C001C00384038403840388019000E000B1A7D990E>I<1E03002707 00470700470700870E00870E000E0E000E0E001C1C001C1C001C1C001C1C003838803838 801838801839001C5900078E0011127C9116>I<1E06270E470E4706870287020E020E02 1C041C041C041C0818083808181018200C4007800F127C9113>I E /Ff 2 52 df<0F8030E040708030C038E0384038003800700070006000C00180030006 000C08080810183FF07FF0FFF00D157E9412>50 D<0FE030306018701C701C001C001800 38006007E000300018000C000E000EE00EE00EC00C401830300FE00F157F9412>I E /Fg 1 22 df21 D E /Fh 1 59 df<70F8F8F87005057C840D>58 D E /Fi 20 119 df<70F8F8F0E005057B840E>46 D<0000020000000600000006000000 0E0000001E0000001E0000003F0000002F0000004F0000004F0000008F0000010F000001 0F0000020F0000020F0000040F00000C0F0000080F0000100F0000100F0000200F80003F FF800040078000C007800080078001000780010007800200078002000780060007801E00 0F80FF807FF81D207E9F22>65 D<01FFFFFE001E001C001E000C001E0004001E0004003C 0004003C0004003C0004003C00040078080800780800007808000078180000F0300000FF F00000F0300000F0300001E0200001E0200001E0200001E0001003C0002003C0002003C0 004003C00040078000800780018007800100078007000F001F00FFFFFE001F1F7D9E1F> 69 D<01FFFF00001E03C0001E00E0001E0070001E0078003C0078003C0078003C007800 3C0078007800F0007800F0007801E0007801C000F0070000F01E0000FFF00000F0380001 E01C0001E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003C01E0007803C0007 803C0807803C0807803C100F801C10FFF00C20000007C01D207D9E21>82 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F000 00F00000F00000F0000070010070020030040018380007C00011147C9315>99 D<0000780003F80000700000700000700000700000E00000E00000E00000E00001C00001 C000F1C00389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C01C2 07010E011C013C013802780C7BF07C00F000F000F000F0007000700170023804183807C0 10147C9315>I<00007800019C00033C00033C000718000700000700000E00000E00000E 00000E00000E0001FFE0001C00001C00001C00001C000038000038000038000038000038 0000700000700000700000700000700000700000E00000E00000E00000E00000C00001C0 0001C0000180003180007B0000F300006600003C00001629829F0E>I<00C001E001E001 C000000000000000000000000000000E003300230043804300470087000E000E000E001C 001C001C003840388030807080310033001C000B1F7C9E0E>105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030 000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00800E00800E 00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C00012147C93 15>I E /Fj 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C0604040 60C031801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C00 0C000C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C0018003 0006000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F00 008000400060C060C060804060801F000B107F8F0F>I<0300030007000F000B00130033 0023004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C0020002000 20002F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060 C06000C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE0 7FC08080808001000200040004000C0008000800180018001800180018000B117E900F> I<1F00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F 8F0F>I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E00 0B107F8F0F>I E /Fk 61 123 df<01F1C003F1C007F1C00701C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C0FFF1C0FFF1C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0121D809C16> 13 D<00E001C0038007000E000E001C001C003800380038007000700070007000E000E0 00E000E000E000E000E000E000E000E000E000E00070007000700070003800380038001C 001C000E000E000700038001C000E00B2A7E9E10>40 DI<018001C0018001806186F99F7DBE1FF807E007E01FF87DBEF99F61860180 018001C0018010127E9E15>I44 DII<00010003000600060006000C000C00 0C0018001800180030003000300060006000C000C000C001800180018003000300030006 0006000C000C000C00180018001800300030003000600060006000C000C00010297E9E15 >I<03C00FF01FF83C3C381C700E700E700EE007E007E007E007E007E007E007E007E007 E007E007E007E007700E700E700E381C3C3C1FF80FF007E0101D7E9B15>I<010007003F 00FF00C70007000700070007000700070007000700070007000700070007000700070007 0007000700070007000700FFF8FFF80D1C7C9B15>I<07C01FF03878701C601EE00EC00F 400F400700070007000F000E001E001C003C007800F001E001C0038007000E001C003800 7000FFFFFFFF101C7E9B15>I<07E01FF03838701CE01E401E401E001E001E001C003C00 7807F007E007F0003C001C001E000F000F000F000F800F800FC01E601C383C1FF007E010 1D7E9B15>I<003C00007C00005C0000DC0001DC00019C00039C00039C00071C00071C00 0E1C000E1C001C1C003C1C00381C00781C00701C00F01C00FFFFC0FFFFC0001C00001C00 001C00001C00001C00001C00001C00121B7F9A15>I58 D<7FFFFFC0FFFFFFE000000000000000000000000000000000 00000000000000000000000000000000FFFFFFE07FFFFFC01B0C7E8F20>61 D<001C0000003E0000003E0000002E0000006700000067000000E7800000C7800000C380 0001C3C0000183C0000181C0000381E0000381E0000700F0000700F0000600F0000E0078 000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038001E0070001F0070000F 0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F00001E00003C00003C0000780000780000780000 F00000F00000F00000F00000F00000F00000F00000F00000F00000780000780000780000 3C00003C00001E00000F000807801803C07800FFF0003F80151F7D9D1B>IIII<003F8001FFF003C0F80780 380F00181E00003C00003C0000780000780000780000F00000F00000F00000F00000F000 00F00000F007F8F007F8F000387800387800387800383C00383C00381E00380F00380780 3803C0F801FFF0003F80151F7D9D1C>III75 DIII<003F000001FFE00003FFF00007C0F800 0F807C001E001E003E001F003C000F00780007807800078078000780F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C07800078078000780 7C000F803C000F003E001F001F003E000F807C0007C0F80003FFF00001FFE000003F0000 1A1F7E9D1F>II<003F000001FFE0 0003FFF00007C0F8000F807C001F003E003E001F003C000F007800078078000780780007 80F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003 C07800078078000780780E07803C0F0F003E079F001E03DE000F83FC0007C1F80003FFF0 0001FFF800003F780000003C0000003E0000001F0000000F801A237E9D1F>II<03F8000FFE001C0F00380700700300600000 E00000E00000E00000E00000F000007800007F00003FE0001FFC0007FE0001FF00001F80 0007800003C00003C00001C00001C00001C00001C0C00180E00380F007007C0E001FFC00 07F000121F7E9D17>IIII I<78000E007C001E003C003C001E0038000F0070000F00F0000781E00003C1C00001C3C0 0001E7800000F70000007E0000003E0000003C0000003C0000007E00000077000000E780 0001E3800003C1C0000381E0000700F0000F00F8000E0078001C003C003C003E0078001F 0070000F00F0000F80191D7F9C1C>II<7FFFF07FFF F00001E00003E00003C00007C0000780000F00001F00001E00003E00003C0000780000F8 0000F00001F00001E00003C00007C0000780000F80000F00001E00003E00003C00007C00 00780000FFFFF0FFFFF0141D7E9C19>I<0FC03FF07FF87038401C001C001C00FC0FFC3F FC781CE01CE01CE01CF07C7FFC7FDC3F1C0E127E9114>97 DI<07E00FF81FFC3C1C70047000E000E000E000 E000E000E000700070043C1C1FFC0FF807E00E127E9112>I<000E000E000E000E000E00 0E000E000E000E000E000E0F8E1FEE3FFE7C3E700E700EE00EE00EE00EE00EE00EE00EF0 0E701E7C3E3FFE1FEE0F8E0F1D7E9C15>I<07C01FE03FF078787018601CFFFCFFFCFFFC E000E000E000700070043C1C3FFC1FF807E00E127E9112>I<03C3C00FFFC01FFFC01C38 00381C00381C00381C00381C00381C001C38001FF8001FF0003BC0003800003800001FFC 001FFF003FFF80700780E001C0E001C0E001C0F003C07C0F803FFF001FFE0007F800121B 7F9115>103 DI< F0F0F0F000000000000000707070707070707070707070707070707070041D7E9C0A>I< E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0E0031D7D9C0A> 108 DII<03F0000FFC001FFE003C0F00780780700380E0 01C0E001C0E001C0E001C0E001C0F003C07003807807803C0F001FFE000FFC0003F00012 127F9115>II<078E1FEE3FFE7C 3E781E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E000E000E000E00 0E000E000E000E000E0F1A7E9115>II<1FC03FF07FF0F030E000E000F0 007F003FC01FE000F0003800388038F078FFF07FE01FC00D127F9110>I<1C001C001C00 1C001C001C00FFE0FFE01C001C001C001C001C001C001C001C001C001C001C001C001C20 1FF00FF007C00C187F970F>III121 D<7FFC7FFC7FFC007800F000E001E003C003800700 0F001E001C003C007800FFFCFFFCFFFC0E127F9112>I E /Fl 24 124 df<78FCFCFCFC7806067D850D>46 D<000180000380000780000F80001F80003F80 006F8000CF80008F80018F80030F80060F800C0F80180F80300F80600F80C00F80FFFFF8 FFFFF8000F80000F80000F80000F80000F80000F8001FFF801FFF8151B7F9A18>52 D<03F8000FFE001E0F003C07807807807803C0F803C0F803C0F803E0F803E0F803E0F803 E07807E03807E03C0BE00E1BE003E3E00003E00003C00003C03807C07C07807C0700780F 00383C001FF8000FE000131B7E9A18>57 D<78FCFCFCFC7800000000000078FCFCFCFC78 06127D910D>I<00038000000380000007C0000007C0000007C000000FE000000FE00000 1FF000001BF000001BF0000031F8000031F8000061FC000060FC0000E0FE0000C07E0000 C07E0001803F0001FFFF0003FFFF8003001F8003001F8006000FC006000FC00E000FE00C 0007E0FFC07FFEFFC07FFE1F1C7E9B24>65 D68 DI<0FF8001C1E003E0F803E07803E07C01C07C00007C0007FC0 07E7C01F07C03C07C07C07C0F807C0F807C0F807C0780BC03E13F80FE1F815127F9117> 97 DI<03FC000E0E001C1F003C1F 00781F00780E00F80000F80000F80000F80000F80000F800007800007801803C01801C03 000E0E0003F80011127E9115>I<01FC000F07001C03803C01C07801C07801E0F801E0F8 01E0FFFFE0F80000F80000F800007800007C00603C00601E00C00F038001FC0013127F91 16>101 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F 001F001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E> 105 D108 DII<01FC000F07801C01C03C01E07800F07800F0F800F8 F800F8F800F8F800F8F800F8F800F87800F07800F03C01E01E03C00F078001FC0015127F 9118>II114 D<1FD830786018E018E018F000FF807FE07FF01FF807FC007CC01CC01C E01CE018F830CFC00E127E9113>I<0300030003000300070007000F000F003FFCFFFC1F 001F001F001F001F001F001F001F001F001F0C1F0C1F0C1F0C0F08079803F00E1A7F9913 >III120 D123 D E /Fm 30 123 df<387CFEFEFE7C3807077C 860F>46 D<00E00001E0000FE000FFE000F3E00003E00003E00003E00003E00003E00003 E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003 E00003E00003E00003E00003E000FFFF80FFFF80111D7C9C1A>49 D<07F0001FFE00383F007C1F80FE0FC0FE0FC0FE0FE0FE07E07C07E03807E0000FE0000F C0000FC0001F80001F00003E0000780000F00000E00001C0000380600700600E00601C00 E01FFFC03FFFC07FFFC0FFFFC0FFFFC0131D7D9C1A>I<01FC0007FF000E0F801E0FC03F 07E03F07E03F07E03F07E01E0FC0000FC0000F80001F0001FC0001FC00000F800007C000 03E00003F00003F83803F87C03F8FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003 FC00151D7E9C1A>I<01FC000FFF001F07803E03C07C03E07C01E0FC01F0FC01F0FC01F0 FC01F8FC01F8FC01F8FC01F87C03F87C03F83E05F81FFDF807F9F80041F80001F03C01F0 7E01F07E03E07E03E07E07C03C0780381F001FFC0007F000151D7E9C1A>57 D<0007FC02003FFF0E00FE03DE03F000FE07E0003E0FC0001E1F80001E3F00000E3F0000 0E7F0000067E0000067E000006FE000000FE000000FE000000FE000000FE000000FE0000 00FE0000007E0000007E0000067F0000063F0000063F00000C1F80000C0FC0001807E000 3803F0007000FE01C0003FFF800007FC001F1F7D9E26>67 D69 D73 D77 D80 D<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003 E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803E0FC05E07E0DE03FF8FE0FE0 7E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC00 00FC0000FC00007C00007E00003E00603F00C01F81C007FF0001FC0013147E9317>I<01 FE0007FF800F83C01E01E03E00F07C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC 00007C00007C00003E00181E00180F807007FFE000FF8015147F9318>101 D<001F8000FFC001F3E003E7E003C7E007C7E007C3C007C00007C00007C00007C00007C0 00FFFC00FFFC0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC3C07 FFFE0F079E1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC0018 00001800001C00001FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78 003C78003C3F01F80FFFE001FF00171E7F931A>I<1C003E007F007F007F003E001C0000 0000000000000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F 001F001F001F001F001F00FFE0FFE00B217EA00E>105 D<0038007C00FE00FE00FE007C 003800000000000000000000000001FE01FE003E003E003E003E003E003E003E003E003E 003E003E003E003E003E003E003E003E003E003E003E303E783EFC3CFC7C78783FF01FC0 0F2A83A010>I108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC 007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>II<01F81807FE381F87783F01F83E01F87E00 F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87E00F87E00F83F01F81F87 F80FFEF803F8F80000F80000F80000F80000F80000F80000F80000F80007FF0007FF181D 7E931C>II< 0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81E FFFCC7F010147E9315>I<01800180018003800380038007800F803F80FFFCFFFC0F800F 800F800F800F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F 9C14>II120 D<3FFFE03FFFE03C07C0380F80701F80603F00603E 00607C0000F80001F80003F00003E06007C0600F80601F80E03F00C03E01C07C03C0FFFF C0FFFFC013147F9317>122 D E /Fn 79 124 df<007E1F0001C1B1800303E3C00703C3 C00E03C1800E01C0000E01C0000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C00E01800E00000E00000E00000E00000E0000FFFF C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C07F87F8151D809C17>I<007FC001C1C00303C00703C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F CFF8151D809C17>I<003F07E00001C09C18000380F018000701F03C000E01E03C000E00 E018000E00E000000E00E000000E00E000000E00E000000E00E00000FFFFFFFC000E00E0 1C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C 000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C00 0E00E01C007FC7FCFF80211D809C23>I<003F07FC0001C0DC1C000381F03C000701F03C 000E01E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C00 FFFFFFFC000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E 00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00 E01C000E00E01C000E00E01C007FC7FCFF80211D809C23>I<60F0F0F0F0F0F0F0606060 60606060606060606060000000000060F0F060041E7C9D0C>33 D<6060F0F0F8F8686808 0808080808101010102020404080800D0C7F9C15>I<00E0000001900000030800000308 0000070800000708000007080000070800000710000007100000072000000740000003C0 3FE003800F00038006000380040005C0040009C0080010E0100030E01000607020006070 2000E0384000E03C4000E01C8000E00F0020E0070020700780403009C0401830E18007C0 3E001B1F7E9D20>38 D<60F0F8680808081010204080050C7C9C0C>I<00400080010002 0006000C000C0018001800300030007000600060006000E000E000E000E000E000E000E0 00E000E000E000E000E000600060006000700030003000180018000C000C000600020001 00008000400A2A7D9E10>I<800040002000100018000C000C0006000600030003000380 01800180018001C001C001C001C001C001C001C001C001C001C001C001C0018001800180 038003000300060006000C000C00180010002000400080000A2A7E9E10>I<60F0F07010 10101020204080040C7C830C>44 DI<60F0F06004047C830C>I< 00010003000600060006000C000C000C0018001800180030003000300060006000C000C0 00C0018001800180030003000300060006000C000C000C00180018001800300030003000 600060006000C000C00010297E9E15>I<03C00C301818300C300C700E60066006E007E0 07E007E007E007E007E007E007E007E007E007E007E00760066006700E300C300C18180C 3007E0101D7E9B15>I<030007003F00C700070007000700070007000700070007000700 07000700070007000700070007000700070007000700070007000F80FFF80D1C7C9B15> I<07C01830201C400C400EF00FF80FF807F8077007000F000E000E001C001C0038007000 6000C00180030006010C01180110023FFE7FFEFFFE101C7E9B15>I<07E01830201C201C 781E780E781E381E001C001C00180030006007E00030001C001C000E000F000F700FF80F F80FF80FF00E401C201C183007E0101D7E9B15>I<000C00000C00001C00003C00003C00 005C0000DC00009C00011C00031C00021C00041C000C1C00081C00101C00301C00201C00 401C00C01C00FFFFC0001C00001C00001C00001C00001C00001C00001C0001FFC0121C7F 9B15>I<00F0030C06040C0E181E301E300C700070006000E3E0E430E818F00CF00EE006 E007E007E007E007E007600760077006300E300C18180C3003E0101D7E9B15>54 D<4000007FFF807FFF007FFF004002008004008004008008000010000010000020000060 0000400000C00000C00001C0000180000180000380000380000380000380000780000780 00078000078000078000078000030000111D7E9B15>I<03E00C301008200C2006600660 0660067006780C3E083FB01FE007F007F818FC307E601E600FC007C003C003C003C00360 026004300C1C1007E0101D7E9B15>I<03C00C301818300C700C600EE006E006E007E007 E007E007E0076007700F300F18170C2707C700060006000E300C780C78187010203030C0 0F80101D7E9B15>I<60F0F0600000000000000000000060F0F06004127C910C>I<60F0F0 600000000000000000000060F0F0701010101020204080041A7C910C>I<7FFFFFC0FFFF FFE00000000000000000000000000000000000000000000000000000000000000000FFFF FFE07FFFFFC01B0C7E8F20>61 D<0FE03038401CE00EF00EF00EF00E000C001C00300060 00C000800180010001000100010001000100000000000000000000000300078007800300 0F1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F000000178000 00178000001780000023C0000023C0000023C0000041E0000041E0000041E0000080F000 0080F0000180F8000100780001FFF80003007C0002003C0002003C0006003E0004001E00 04001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E0618001801980070007800E0003801C0003801C00018038 000180780000807800008070000080F0000000F0000000F0000000F0000000F0000000F0 000000F0000000F0000000700000807800008078000080380000801C0001001C0001000E 000200070004000180080000E03000001FC000191E7E9C1E>I III<001F808000E0618001801980070007800E0003801C0003801C0001803800018078 0000807800008070000080F0000000F0000000F0000000F0000000F0000000F0000000F0 00FFF0F0000F80700007807800078078000780380007801C0007801C0007800E00078007 000B800180118000E06080001F80001C1E7E9C21>III75 DIII<003F 800000E0E0000380380007001C000E000E001C0007003C00078038000380780003C07800 03C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F000 01E0700001C0780003C0780003C0380003803C0007801C0007000E000E0007001C000380 380000E0E000003F80001B1E7E9C20>II< FFFF00000F01E0000F0078000F003C000F001C000F001E000F001E000F001E000F001E00 0F001C000F003C000F0078000F01E0000FFF00000F03C0000F00E0000F00F0000F007800 0F0078000F0078000F0078000F0078000F0078000F0078100F0078100F0038100F003C20 FFF01C20000007C01C1D7E9B1F>82 D<07E0801C1980300580700380600180E00180E000 80E00080E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007 C00003C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E 7E9C17>I<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F00 20800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00001F800003FFFC001B1C7F9B1E>IIII89 D<7FFFF07C01F07001E06003C06003C0400780400F80400F00401E00001E00003C00007C 0000780000F00000F00001E00003E00003C0100780100780100F00101F00301E00203C00 203C00607800E0F803E0FFFFE0141C7E9B19>I<08081010202040404040808080808080 B0B0F8F8787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C 0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E801F87001212 7E9115>97 DI<07E00C3018783078 70306000E000E000E000E000E000E00060007004300418080C3007C00E127E9112>I<00 3F0000070000070000070000070000070000070000070000070000070000070003E7000C 1700180F00300700700700600700E00700E00700E00700E00700E00700E0070060070070 0700300700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006E006FF FEE000E000E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E061E 0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C3880181800 381C00381C00381C00381C001818001C38000C300013C0001000003000001800001FF800 1FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007F800121C7F 9215>II<18003C003C0018000000 000000000000000000000000FC001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C0000000000000000000 00000000000FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380 600180E001C0E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C00 03F00012127F9115>II<03C1000C3300180B00300F0070 0700700700E00700E00700E00700E00700E00700E00700600700700700300F00180F000C 370007C700000700000700000700000700000700000700000700003FE0131A7E9116>I< FCE01D301E781E781C301C001C001C001C001C001C001C001C001C001C001C001C00FFC0 0D127F9110>I<1F9030704030C010C010E010F8007F803FE00FF000F880388018C018C0 18E010D0608FC00D127F9110>I<04000400040004000C000C001C003C00FFE01C001C00 1C001C001C001C001C001C001C001C101C101C101C101C100C100E2003C00C1A7F9910> IIII<7F8FF00F03800F03000702000384 0001C80001D80000F00000700000780000F800009C00010E00020E000607000403801E07 C0FF0FF81512809116>II<7FFC70386038407040F040E0 41C003C0038007000F040E041C043C0C380870087038FFF80E127F9112>II E /Fo 57 123 df<003F83F800FF8FF801FF9FF803E03E0003803800078078 0007807800078078000780780007807800078078000780780007807800FFFC7FC0FFFC7F C0FFFC7FC007807800078078000780780007807800078078000780780007807800078078 000780780007807800078078000780780007807800078078000780780007807800078078 0007807800078078001D2380A21C>11 D<000078007C7800FC7801FC7803C00003800007 8000078000078000078000078000078000078000078000FFFC78FFFC78FFFC7807807807 807807807807807807807807807807807807807807807807807807807807807807807807 8078078078078078078078078078078078152480A31A>I<787878781830306060E0050A 7C830E>44 DII<01F00007FC000F FE001F1F003C07803C07807803C07803C07803C07001C0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07803C078 03C07803C07803C03C07803C07801F1F000FFE0007FC0001F00013237EA118>48 D<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFFFFFFFFFF10227C A118>I<03F0000FFC001FFE003C1F003007807007C06003C0E003E0C001E04001E04001 E00001E00001E00001E00003C00003C0000780000780000F00001E00003C0000780000F0 0001E00001C0000380000700000E00001C0000380000700000FFFFE0FFFFE0FFFFE01322 7EA118>I<01F00007FC001FFF003E0F003807807003C02003C02003C00003C00003C000 03C0000780000780000F00001E0003FC0003F80003FE00000F000007800003C00003C000 01E00001E00001E00001E00001E08001E0C003C0E003C07007803C0F801FFF000FFC0003 F00013237EA118>I<001F00001F00002F00002F00006F0000EF0000CF0001CF0001CF00 038F00038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F00780F00F00F00 FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F00000F00000F00 15217FA018>I<3FFF803FFF803FFF803C00003C00003C00003C00003C00003C00003C00 003C00003C00003CF8003FFE003FFF003F0F803E07803C03C03803C00001E00001E00001 E00001E00001E00001E00001E04003C04003C0E003C07007807C1F003FFE000FFC0003F0 0013227EA018>I<007E0001FF0003FF0007C1000F00001E00001E00003C00003C000078 000078000078F800F1FE00F7FF00FF0F80FC0780F803C0F803C0F801C0F001E0F001E0F0 01E0F001E0F001E07001E07001E07801E07803C03803C03C03801C07801F0F000FFE0007 FC0001F00013237EA118>II<01F00007FC000FFE001E0F003C07803C07807803C07803C07803 C07803C07803C03803803C07801E0F000F1E0007FC0003F8000FFE001E0F003C07807803 C07803C0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C03C07803E0F801FFF 0007FC0001F00013237EA118>I<01F00007FC000FFE001E0F003C070038078078038078 03C0F003C0F001C0F001E0F001E0F001E0F001E0F001E0F001E07003E07803E07803E03C 07E03E1FE01FFDE00FF1E003E3C00003C00003C0000380000780000700000F00101E0018 3C003FF8001FF00007C00013237EA118>I<001F0000001F0000003F8000003F8000003B 8000007BC0000073C0000071C00000F1E00000F1E00000E0E00001E0F00001E0F00001C0 F00003C0780003C078000380780007803C0007803C0007003C000F001E000F001E000FFF FE001FFFFF001FFFFF001C000F003C0007803C00078038000780780003C0780003C07000 03C0F00001E0F00001E0E00001E01B237EA220>65 DI<000FF000007FFC0000FF FF0001F01F0003C00700078000000F0000001E0000003E0000003C0000003C0000007800 00007800000078000000F0000000F0000000F0000000F0000000F0000000F0000000F000 0000F0000000F00000007800000078000000780000003C0000003C0000003E0000001E00 00000F0000000780008003C0038001F00F8000FFFF00007FFC00000FF00019257DA31F> IIII<000FF000003FFE0000FFFF8001F80F8003E0038007800000 0F0000001E0000001E0000003C0000003C000000780000007800000078000000F0000000 F0000000F0000000F0000000F0000000F0000000F000FFC0F000FFC0F000FFC0780003C0 780003C0780003C03C0003C03C0003C01E0003C01E0003C00F0003C0078003C003E003C0 01F807C000FFFFC0003FFF00000FF8001A257DA321>I73 D75 D77 DI<001FC000007FF00001FFFC0003F07E0007C01F000F800F 801F0007C01E0003C03C0001E03C0001E0780000F0780000F0780000F070000070F00000 78F0000078F0000078F0000078F0000078F0000078F0000078F0000078F0000078780000 F0780000F0780000F07C0001F03C0001E03E0003E01E0003C01F0007C00F800F8007C01F 0003F07E0001FFFC00007FF000001FC0001D257DA324>II82 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800007800007800007800007800 007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F00000F00000 F8000078000078000078000078000078000078C000F0E000F0F801E07E07C03FFF800FFF 0001FC0015257EA31B>II86 D<07E01FF83FFC381E201E000F000F000F000F 00FF07FF1FFF3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180780000780000700000F00000F00000F0 0000F00000F00000F000007800007800007800003C00401F03C00FFFC007FF8001FC0012 167E9516>I<0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C003E3C00FFBC01FFFC03F0FC03C07C07803C07803C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C07803C07803C03C07C03E0FC01FFFC00FFBC0 03E3C012237EA219>I<03F00007FC001FFE003E0F003C0780780380780380F001C0FFFF C0FFFFC0FFFFC0F00000F00000F000007000007800007800003C00801F07800FFF8007FF 0001F80012167E9516>I<003F00FF01FF03C00380078007800780078007800780078007 80FFF8FFF8FFF80780078007800780078007800780078007800780078007800780078007 800780078007800780102380A20F>I<01F07807FFF80FFFF81F1F001E0F003C07803C07 803C07803C07803C07801E0F001F1F000FFE001FFC0019F0003800003800003C00001FFE 001FFFC01FFFE03FFFF07801F07800F8F00078F00078F00078F000787800F03E03E01FFF C00FFF8001FC0015217F9518>III<00F000F000F000F0000000 00000000000000000000000000000000F000F000F000F000F000F000F000F000F000F000 F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F080F0E3 E0FFE07FC01F000C2D83A20D>I108 DII<01FC0007FF000FFF801F07C0 3C01E07800F07800F0700070F00078F00078F00078F00078F00078F000787800F07800F0 7C01F03E03E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07C07C 03C07803C0F803C0F003C0F003C0F003C0F003C0F003C0F003C0F803C07803C07803C03C 07C03F0FC01FFFC00FFBC003E3C00003C00003C00003C00003C00003C00003C00003C000 03C00003C00003C012207E9519>II<07F01FFC3FFE 3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE3FFC 0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12>I< F00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF01F F83F7FFF7FCF1F0F10167C9519>III<7801F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F800007000 00F80001FC0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F8151680 9516>II<7F FFC07FFFC07FFFC0000F80000F00001E00003E00007C0000780000F00001F00003E00003 C0000780000F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012167F9515>I E /Fp 29 91 df<3078F8787005057C840D>46 D<007F000183C00201E00400F00700F0 0F00F00F01F00F01F00001E00001E00003C0000380000700000E0000F800000E00000700 0007800007C00003C00007C03007C07807C0F807C0F807C0F00780800F00400E00201C00 18780007E000141F7D9D17>51 D<0000600000600000E00001C00003C00005C0000DC000 09C00011C000238000438000C38000838001038002038004070008070018070010070020 0700400700FFFFF0000E00000E00000E00000E00000E00001C00001E0001FFE0141E7E9D 17>I<01803001FFE003FFC003FF0003FC00020000020000020000040000040000040000 047C000587000603800C01800801C00001C00001E00001E00001E00001E07003C0F803C0 F003C0E00380800780400700400E00201C0018700007C000141F7D9D17>I<000F800070 4000C0200180E00301E00701E00E00C01E00001C00003C000038000078F800790E007A07 007C0300F80380F80380F003C0F003C0F003C0F003C0F00780E00780E00780E00700E00F 00600E00701C0030180018700007C000131F7C9D17>I<003F0000C1C001006002006004 00300C00300C00300C00300C00600E00600F80C00FC18007F60003FC0001FC0001FF0006 3F800C0F801007C03003C06001C06000C0C000C0C000C0C000C0C00080C0010060030030 040018180007E000141F7D9D17>56 D<007E0001C3000301800601C00E01C01C00C03C00 E03C00E03C01E07801E07801E07801E07801E07803E07803E03803C03807C01C0BC00C13 C003E380000780000780000700000E00600E00F01C00F01800E0300080600041C0003F00 00131F7C9D17>I<0000100000001800000038000000380000007800000078000000FC00 0001BC0000013C0000033C0000023C0000063C0000043E0000081E0000081E0000101E00 00101E0000201E0000200F0000400F0000400F0000FFFF0000800F0001000F8001000780 020007800200078004000780040007800C0007C03E0007C0FF807FFC1E207E9F22>65 D<07FFFF00007C01C0003C01E0003C00F0007800F8007800F8007800F8007800F8007800 F8007800F000F001F000F001E000F003C000F00F8000FFFE0000F00F0001E007C001E003 C001E003E001E001E001E001E001E001E003C001E003C003E003C003E003C003C003C007 C003C00F8007800F0007803E00FFFFF0001D1F7E9E20>I<0001F808000E061800380138 007000F801E0007803C0007007800030078000300F0000301F0000301E0000303E000020 3C0000007C0000007C0000007C0000007C000000F8000000F8000000F8000000F8000000 F80000007800004078000080780000803C0000803C0001001C0002000E00020006000C00 0300100001C0E000003F00001D217B9F21>I<07FFFF00007C01E0003C00F0003C007800 78003C0078003C0078001E0078001E0078001E0078001F00F0001F00F0001F00F0001F00 F0001F00F0001F00F0001F01E0001E01E0003E01E0003E01E0003E01E0003C01E0007C03 C0007803C000F003C000F003C001E003C003C003C0078007800F0007803C00FFFFE00020 1F7E9E23>I<07FFFFF8007C0078003C0038003C00180078001800780008007800080078 0008007800080078080800F0100000F0100000F0100000F0300000FFF00000F0700001E0 200001E0200001E0200001E0200001E0000801E0001003C0001003C0001003C0002003C0 002003C0006003C000C0078001C0078007C0FFFFFF801D1F7E9E1F>I<07FFFFF8007C00 78003C0038003C001800780018007800080078000800780008007800080078000800F010 0000F0100000F0100000F0300000F0700000FFF00001E0600001E0200001E0200001E020 0001E0200001E0000003C0000003C0000003C0000003C0000003C0000003C00000078000 0007C00000FFFE00001D1F7E9E1E>I<0001FC04000F030C003C009C0070007C00E0003C 01C0003803800018078000180F0000181F0000181E0000183E0000103C0000007C000000 7C0000007C0000007C000000F8000000F8000000F8007FFCF80003E0780001E0780001E0 780003C0780003C03C0003C03C0003C01C0003C00E0007C007000B800380118001E06080 003F80001E217B9F24>I<07FFC7FFC0007C00F800003C007800003C007800007800F000 007800F000007800F000007800F000007800F000007800F00000F001E00000F001E00000 F001E00000F001E00000FFFFE00000F001E00001E003C00001E003C00001E003C00001E0 03C00001E003C00001E003C00003C007800003C007800003C007800003C007800003C007 800003C007800007800F000007C00F8000FFF8FFF800221F7E9E22>I<07FFE0007C0000 3C00003C0000780000780000780000780000780000780000F00000F00000F00000F00000 F00000F00001E00001E00001E00001E00001E00001E00003C00003C00003C00003C00003 C00003C00007800007C000FFFC00131F7F9E10>I<07FFF000007E0000003C0000003C00 0000780000007800000078000000780000007800000078000000F0000000F0000000F000 0000F0000000F0000000F0000001E0000001E0000001E0000001E0000001E0008001E001 0003C0010003C0010003C0030003C0020003C0060003C0060007801E0007807C00FFFFFC 00191F7E9E1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800004E00 02F000004E0002F000004E0004F000004E0004F000004E0008F000004E0008F000008700 11E00000870011E00000870021E00000870021E00000870041E00000838041E000010380 83C00001038083C00001038103C00001038203C0000101C203C0000101C403C0000201C4 0780000201C80780000201C80780000201D00780000200F00780000600E00780000600E0 0F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E0018 00003E001800004F001000004F001000004780100000478010000043C010000043C01000 0083C020000081E020000081E020000080F020000080F020000080782000010078400001 007C400001003C400001003C400001001E400001001E400002000F800002000F80000200 0F800002000780000200078000060003800006000300000F00010000FFE0010000221F7E 9E22>I<0003F800001E0E000038070000E0038001C001C003C001E0078000E00F0000F0 0F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8 F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C000780 3C000F003C001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07 FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F8007800F8007800F800 7800F000F001F000F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001 E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003 C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C 01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F0 078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E0 1E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC 0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E 00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E000 01E00000E00000F00000F04000E04000E04000E04000E06000C0600180E00380F80300C6 0C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010 C00F0010800F0010800F0010800F0010001E0000001E0000001E0000001E0000001E0000 001E0000003C0000003C0000003C0000003C0000003C0000003C00000078000000780000 0078000000780000007800000078000000F0000001F800007FFFE0001C1F7A9E21>IIII< FFF003FF1F8000F80F0000600F8000400780008007C0018003C0010003E0020001E00400 01F00C0001F0080000F0100000F8200000786000007C4000003C8000003F0000001F0000 001E0000001E0000001E0000001C0000003C0000003C0000003C0000003C0000003C0000 0038000000780000007C00000FFFC000201F7A9E22>89 D<03FFFFC003E0078003800780 07000F0006001E0004003C000C003C00080078000800F0000801F0000001E0000003C000 0007800000078000000F0000001E0000003C0000003C00000078000000F0010001E00100 01E0020003C00200078002000F8006000F0004001E000C003C001C007C003C007800F800 FFFFF8001A1F7D9E1C>I E /Fq 52 122 df<007F07F001FF1FF003FF3FF00780780007 0070000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000FFF8FF80FF F8FF80FFF8FF800F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F 00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F 00F0001C20809F1B>11 D<007000E001C00380078007000E001E001E003C003C003C0078 007800780078007000F000F000F000F000F000F000F000F000F000F000F000F000700078 007800780078003C003C003C001E001E000E0007000780038001C000E000700C2E7EA112 >40 DI<787878781830 306060E0050A7D830D>44 DI<07F0000FFC001FFE00383F 00700F00600780E00780E003C04003C04003C00003C00003C00003C0000780000780000F 00000E00001C00003C0000780000E00001C0000380000700000E00001C00003800007000 00FFFFC0FFFFC0FFFFC0121F7E9E17>50 D58 D61 D<001F0000001F0000003F8000003B8000003B8000007BC0000073C0000071C00000F1E0 0000E1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C 0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F001E000F003C000F803C0007 803C000780780007C0780003C0780003C0F00003E01B207F9F1E>65 DI<001FC000 FFF801FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000780000F0 0000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00003C 00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>IIII<001FE000FFF801FFFE03E03E07800E0F0000 1E00003E00003C00007C0000780000780000780000F00000F00000F00000F00000F00000 F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07801E 03E03E01FFFE00FFF8001FC017227DA01E>III76 DII<00 3F000000FFC00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F0078 0007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F0 0003C0F00003C0F00003C0F80007C07800078078000780780007803C000F003C000F001E 001E001F003E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II82 D<01FC0007FF800FFFC01F03 C03C00C03C00007800007800007800007800007800007C00003C00003F00001FE0000FFC 0007FE0001FF00003F800007C00003C00003E00001E00001E00001E00001E00001E00001 C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019>IIIII<780007807C000F 003E001F001E001E000F003C000F807C000780780003C0F00003E1F00001F1E00000F3C0 00007FC000007F8000003F0000001F0000001E0000003F0000007F8000007FC00000F3C0 0001F1E00001E0F00003C0F80007C0780007803C000F003E001F001E001E000F003C000F 807C0007C0780003C0F00003E01B207F9F1E>II<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF8 0FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F000F000 F000F000F000F000780078003C013E0F1FFF0FFE03F010147E9314>I<00078000078000 078000078000078000078000078000078000078000078000078000078007C7800FF7801F FF803E1F807C0780780780F80780F00780F00780F00780F00780F00780F00780F0078078 0780780F803E1F801FFF800FF78007C78011207E9F17>I<03F0000FFC001FFE003E1F00 3C0700780700700380FFFF80FFFF80FFFF80F00000F00000F000007000007800003C0100 3E07001FFF0007FE0001F80011147F9314>I<007E01FE03FE078007000F000F000F000F 000F000F000F00FFF0FFF0FFF00F000F000F000F000F000F000F000F000F000F000F000F 000F000F000F000F000F000F20809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00780F00 780F00780F00780F00780F003C1E003E3E001FFC003FF80033E0003000003800003FFE00 3FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00 141E7F9317>III107 DIII<01 F80007FE001FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F0 00F07801E07801E03C03C03F0FC01FFF8007FE0001F80014147F9317>II114 D<07F01FFC3FFC780C7800780078007C003FC01FF00FF803F8007C003C003CC03CF07CFF F87FF00FC00E147F9311>I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E00 1E001E001E001E001E001E001E001E001E001E001E201FF00FF007C00C1A7F9910>II119 D<7801E07C03C03E0780 1E0F000F0F00079E0003FC0003F80001F80000F00001F00001F80003FC00079E000F0F00 0E0F001E07803C03C07801E0F801F01414809315>II E /Fr 76 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8700000 00000070F8F8F870051C779B18>33 D<4010E038F078E038E038E038E038E038E038E038 E038E038E03860300D0E7B9C18>I<01C00007E0000FF0000E70001C38001C38001C3800 1C38001C73F01C73F01CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00 E1DC00E0F800E0F800E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F001E001C00380038007000700070007000E000E000E000E0 00E000E000E000E0007000700070007000380038001C001E000F00078003C001F000F000 700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000 E00070007000700070007000700070007000E000E000E000E001C001C0038007800F001E 003C007800F00060000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF 807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C0 0001C00011147D9718>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC78300606778518>I<00030000 0780000780000F80000F00001F00001E00001E00003E00003C00007C0000780000780000 F80000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F00001F 00001E00003E00003C00003C00007C0000780000F80000F00000F0000060000011247D9F 18>I<01F00007FC000FFE001F1F001C07003803807803C07001C07001C0E000E0E000E0 E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C07001C07803C0380380 1C07001F1F000FFE0007FC0001F000131C7E9B18>I<01800380038007800F803F80FF80 FB8043800380038003800380038003800380038003800380038003800380038003800380 7FFCFFFE7FFC0F1C7B9B18>I<03F0000FFE003FFF007C0F807003C0E001C0F000E0F000 E06000E00000E00000E00001C00001C00003C0000780000F00001E00003C0000780000F0 0001E00007C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001F FE003FFF007807807803C07801C03001C00001C00003C0000380000F0003FF0003FE0003 FF000007800003C00001C00000E00000E00000E0F000E0F000E0F001C0F003C07C07803F FF001FFE0003F800131C7E9B18>I<001F00003F0000770000770000E70001E70001C700 0387000787000707000E07001E07003C0700380700780700F00700FFFFF8FFFFF8FFFFF8 000700000700000700000700000700000700007FF000FFF8007FF0151C7F9B18>I<1FFF 803FFF803FFF803800003800003800003800003800003800003800003800003BF8003FFE 003FFF003C07801803C00001C00000E00000E06000E0F000E0F000E0E001C07003C07C0F 803FFF001FFC0003F000131C7E9B18>I<007E0001FF0007FF800F83C01E03C01C03C038 0180380000700000700000E1F800E7FE00FFFF00FE0780F803C0F001C0F000E0E000E0F0 00E07000E07000E07000E03801C03C03C01E07800FFF0007FE0001F800131C7E9B18>I< 3078FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<0003 00000780001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC00 007E00003F00001FC00007E00003F00001FC00007E00003F00001F800007800003001118 7D9918>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FF FFE07FFFC0130C7E9318>I<600000F00000FC00007E00003F00001FC00007E00003F000 01FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F0000 7E0000FC0000F0000060000011187D9918>I<00700000F80000F80000D80000D80001DC 0001DC0001DC00018C00038E00038E00038E00038E000306000707000707000707000707 000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18 >65 DI<00F8E003FEE007FFE00F07E01E03 E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E00000E000 00E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F800131C 7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C 00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C 01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00FFFC01F 0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E00000E0 0FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003FDC001 F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C01C01C01C0 1C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C01C01C0 1C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF 807FFF00111C7D9B18>I<7F07F0FF87F87F07F01C03C01C07801C07001C0E001C1E001C 3C001C38001C70001CF0001DF0001DF0001FB8001FB8001F1C001E1C001C0E001C0E001C 07001C07001C03801C03801C01C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00700E00700E00700E00 700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C0 1CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C0DC0 1C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F007007 00F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C 7D9B18>II<7FF800FFFE007FFF001C0F80 1C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F00 1C07001C03801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F87F00F0 161C7F9B18>82 D<03F3801FFF803FFF807C0F80700780E00380E00380E00380E0000070 00007800003F00001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E0 00E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8 E07038E07038E07038E07038007000007000007000007000007000007000007000007000 00700000700000700000700000700000700000700000700000700000700007FF0007FF00 07FF00151C7F9B18>II87 D<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C 0003DC0001F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E 00070F000707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700 000F00001E00001C00003C0000780000700000F00001E00001C00003C000078000070000 0F00001E00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I<0038007C007C007C003800000000000000000FFC1FFC0FFC001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C6038F078FFF07FE03F800E277E9C18>II<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF800 1F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F0019148193 18>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0 000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001 E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807F FFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F 00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E 00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780 700780E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF80 0FFB8003E380000380000380000380000380000380000380000380003FF8003FF8003FF8 151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C0000380000380 000380000380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318 >I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE0000 1F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<018000 0380000380000380000380007FFFC0FFFFC0FFFFC0038000038000038000038000038000 0380000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E00 13197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318 >I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E0003 8E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8F F0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E 000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E 03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000 DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F80003F 00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00 003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF0 14147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000007800007F E0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003FE0003FE000 7FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00003C000FFC000FF00007C000013247E9F18>125 D E /Fs 44 123 df45 DI<007F000001FFC0 0007FFF0000FFFF8000FC1F8001F007C003F007E003E003E003C001E007C001F007C001F 007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F 80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F8078000F 007C001F007C001F007C001F003E003E003E003E003F007E001F80FC000FC1F8000FFFF8 0007FFF00001FFC000007F000019297EA71E>48 D<00180000380000F80007F800FFF800 FFF800F8F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF07FFFF014287B A71E>I<00FE0003FFC007FFE00FFFF01F03F83C00FC38007E78003E70003EF0001FF000 1F60001F20001F00001F00001F00001F00003E00003E00007C00007C0000F80001F00001 E00003C0000780000F00001E00003C0000780000F00001E00003C0000780000F00001E00 003C00007FFFFF7FFFFF7FFFFF7FFFFF18287EA71E>I<007F000001FFC00007FFF0000F FFF8001FC1F8003E007C003C003E0078003E0038003E0010003E0000003E0000003E0000 003C0000007C000000FC000001F8000007F00000FFE00000FFC00000FFE00000FFF00000 01FC0000007C0000003E0000001F0000001F0000000F8000000F8000000F8000000F8000 000F8040000F8060001F00F0001F00F8003F007E007E003F81FC001FFFF8000FFFF00003 FFE000007F000019297EA71E>I<0003F0000007F0000005F000000DF000000DF000001D F0000039F0000039F0000079F0000079F00000F1F00000F1F00001E1F00003E1F00003E1 F00007C1F00007C1F0000F81F0000F81F0001F01F0001F01F0003E01F0007C01F0007C01 F000F801F000FFFFFF80FFFFFF80FFFFFF80FFFFFF800001F0000001F0000001F0000001 F0000001F0000001F0000001F0000001F0000001F0000001F00019277EA61E>I<3FFFFC 3FFFFC3FFFFC3FFFFC3E00003E00003E00003E00003E00003E00003E00003E00003E0000 3E00003E3F003EFFC03FFFE03FFFF03FE1F83F807C3F003E3E003E00003E00001F00001F 00001F00001F00001F00001F00001F20001F60003E70003EF8007C7C00FC3F03F81FFFF0 0FFFE007FF8000FE0018287EA61E>I<000FF000003FFC0000FFFC0001FFFC0003F80C00 07E000000FC000000F8000001F0000001E0000003E0000003C0000007C0000007C000000 7C3FE000F8FFF000F9FFF800FBFFFC00FF807E00FF003E00FE003F00FC001F00FC001F00 FC000F80F8000F80F8000F80F8000F80F8000F8078000F807C000F807C000F807C000F00 3E001F003E001F001F003E001F807C000FC1FC0007FFF80003FFF00001FFC000007F0000 19297EA71E>II<007F000001FFC00007FF F0000FFFF8001FC1FC003F007E003E003E007E003F007C001F007C001F007C001F007C00 1F007C001F003E003E003E003E001F007C000FC1F80007FFF00003FFE00003FFE0000FFF F8001FC1FC003F007E003E003E007C001F007C001F00F8000F80F8000F80F8000F80F800 0F80F8000F80F8000F807C001F007C001F007E003F003F007E001FC1FC000FFFF80007FF F00003FFE000007F000019297EA71E>I<007F000001FFC00003FFE0000FFFF0000FC1F8 001F007C003E007C007C003E007C001E007C001F00F8001F00F8001F00F8000F00F8000F 80F8000F80F8000F80F8000F80F8001F807C001F807C001F807E003F803E007F803F00FF 801FFFEF800FFFCF8007FF8F8003FE1F0000001F0000001F0000001E0000003E0000003E 0000007C0000007C000000F8001801F0001E07E0003FFFC0001FFF80000FFE000003F800 0019297EA71E>I<0001FF00000FFFE0003FFFF8007FFFF800FE01F801F8003003F00010 07C000000F8000001F8000001F0000003E0000003E0000007E0000007C0000007C000000 7C000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000F80000007C0000007C0000007C0000007E0000003E0000003E0000001F000000 1F8000000F80000007C0000003F0000401F8001C00FE00FC007FFFFC003FFFF8000FFFE0 0001FF001E2C7CAA26>67 DI70 D73 D77 D<0001FC0000000FFF8000003FFFE000007FFFF00001FE03FC0003F800FE0007E0003F00 07C0001F000F80000F801F000007C01F000007C03E000003E03E000003E07C000001F07C 000001F07C000001F078000000F0F8000000F8F8000000F8F8000000F8F8000000F8F800 0000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F87C000001F07C0000 01F07C000001F07E000003F03E000003E03F000007E01F000007C01F80000FC00FC0001F 8007E0003F0007F0007F0003F800FE0001FE03FC0000FFFFF800003FFFE000000FFF8000 0001FC0000252C7DAA2C>79 DI82 D<007FC00001FFF80007FFFE000FFFFF001FC07F003F000F007E0006007C000000 7C000000F8000000F8000000F8000000F8000000F8000000FC0000007E0000007F000000 3F8000001FF800000FFF800007FFE00003FFF80000FFFC00000FFE000000FF0000003F00 00001F8000000F8000000FC0000007C0000007C0000007C0000007C0000007C0000007C0 00000F8060000F80F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE00000FF8000 1A2C7DAA21>II<01FE000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F800 00F80000F80000F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F800F8F800F8FC 01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1D>97 DI<007FC001FFF007FFFC0FFFFC1FC07C1F00083E00007C 00007C00007C0000F80000F80000F80000F80000F80000F80000F800007C00007C00007E 00003E00001F000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<00003E00003E 00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00FC3E03FF3E07FFFE0FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003E F8003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E7C003E3E007E3F00FE1FC1FE 0FFFFE07FFBE03FF3E00FC3E172A7EA91F>I<007E0003FF8007FFC00FFFE01F83F03F00 F03E00787C00787C003878003CFFFFFCFFFFFCFFFFFCFFFFFCF80000F80000F800007800 007C00007C00003E00003F000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<00 1FC0007FC000FFC001FFC003F00003E00007C00007C00007C00007C00007C00007C00007 C00007C00007C000FFFE00FFFE00FFFE0007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C000122A7FA912>I<00F8078003FE7FC00FFFFFC0 1FFFFFC01F07C0003E03E0003E03E0007C01F0007C01F0007C01F0007C01F0007C01F000 7C01F0003E03E0003E03E0001F07C0001FFFC0003FFF80003BFE000038F8000078000000 780000003C0000003FFFC0003FFFF8001FFFFC001FFFFE003FFFFF007C007F00F8001F80 F8000F80F8000F80F8000F80FC001F807E003F003F80FE003FFFFE000FFFF80007FFF000 00FF80001A287E9A1E>II< F8F8F8F8F800000000000000000000F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8 F8F8F8F8F8F8052A7CA90E>I108 DII<007F000001FFC00007FFF0000FFFF8001FC1FC003F 007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001FC1FC000F FFF80007FFF00001FFC000007F0000191B7E9A1E>II114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8 003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFF C07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007 C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003 FFE003FFE001FF8000FC0013227FA116>IIII121 DI E /Ft 9 122 df77 D<0007FFFC000000007FFFFFC0000001FFFFFFF8000003FFFFFFFE000007FE 001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000FFF0000FFF80007FF0000FFF80 007FF8000FFF80007FF80007FF00003FFC0007FF00003FFC0003FE00003FFC0000F80000 3FFC00000000003FFC00000000003FFC00000000003FFC00000000003FFC00000007FFFF FC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC0003FFF0003FFC 0007FFC0003FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC007FF800003FFC00 7FF800003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FF F000003FFC00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC0003EFFC003FFF 0007CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFFF001FFFC0003FF 80007FF8362E7DAD3A>97 D<00001FFFC0000000FFFFF8000007FFFFFE00001FFFFFFF80 007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FFE007FF0003FFE00FFE0003FFE0 0FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF803FFC00003E007FF800000000 7FF8000000007FF800000000FFF800000000FFF800000000FFF800000000FFF800000000 FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000 7FF8000000007FF8000000007FFC000000003FFC000000003FFC000000001FFC000000F8 1FFE000000F80FFE000000F80FFF000001F007FF800003F003FFC00007E001FFE0000FC0 00FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8000000FFFFE00000001FFE0000 2D2E7CAD35>99 D<00001FFE00000001FFFFE0000007FFFFF800001FFFFFFE00007FFC07 FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00003FF00FFE00001FF01FFE0000 0FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC000007FC7FF8000007FC7FF80000 07FE7FF8000007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFF FFFEFFFFFFFFFFFCFFF800000000FFF800000000FFF800000000FFF8000000007FF80000 00007FF8000000007FFC000000003FFC000000003FFC000000003FFC0000001C1FFE0000 003E0FFE0000003E07FF0000007E07FF000000FC03FF800001F801FFC00003F0007FF000 1FE0003FFE00FFC0001FFFFFFF800007FFFFFE000000FFFFF80000000FFF80002F2E7DAD 36>101 D<00FC0001FE0003FF0007FF800FFFC01FFFE01FFFE01FFFE01FFFE01FFFE01F FFE00FFFC007FF8003FF0001FE0000FC0000000000000000000000000000000000000000 0000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFF FFFFFFFFFFFFFFFF18497CC820>105 D<007FC000FFFFC000FFFFC000FFFFC000FFFFC0 00FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FFFFFF 80FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<007FC001FFC00000FFFF C00FFFF80000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1FC07FF8000FFFFC3E003FF C00003FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE00001FFDC0000FFE00001FF FC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF00000FFF00001FFF00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF000FFFFFFC07FFF FFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B2E 7CAD42>110 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FFE01FC0001F E03F80000FE03F000007E07F000003E07F000003E0FF000003E0FF000003E0FF800003E0 FFC0000000FFF0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFFFFFC001F FFFFFF000FFFFFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FFFFF80000 1FFFF8000000FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003FCFC0000 03FCFE000003F8FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0FFFC00FF C0FFFFFFFF80FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>115 D<7FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFF C000FFFF8000FFF000000FE00000FFF800000FC00000FFF800000FC000007FFC00000F80 00007FFC00001F8000003FFC00001F0000003FFE00003F0000001FFE00003E0000001FFF 00007E0000000FFF00007C0000000FFF8000FC00000007FF8000F800000007FFC001F800 000003FFC001F000000003FFE003F000000003FFE003F000000001FFF003E000000001FF F007E000000000FFF007C000000000FFF80FC0000000007FF80F80000000007FFC1F8000 0000003FFC1F00000000003FFE3F00000000001FFE3E00000000001FFF7E00000000000F FF7C00000000000FFFFC00000000000FFFFC000000000007FFF8000000000007FFF80000 00000003FFF0000000000003FFF0000000000001FFE0000000000001FFE0000000000000 FFC0000000000000FFC00000000000007F800000000000007F800000000000003F000000 000000003F000000000000003F000000000000003E000000000000007E00000000000000 7C00000000000000FC000000001F8000F8000000003FC001F8000000007FE001F0000000 00FFF003F000000000FFF003E000000000FFF007E000000000FFF00FC000000000FFF01F 8000000000FFF03F80000000007FE07F00000000007F43FE00000000003FFFF800000000 001FFFF0000000000007FFC0000000000001FE00000000000039427EAD3F>121 D E /Fu 8 117 df<0001FF8000001FFFF000007FFFFC0000FF81FE0001FE007F0007FC 003F800FF8003FC00FF8001FE01FF8001FF03FF0001FF03FF0000FF87FF0000FF87FF000 0FFCFFF0000FFCFFF0000FFCFFF0000FFEFFF0000FFEFFF0000FFEFFF0000FFEFFF0000F FEFFF0000FFFFFF0000FFFFFF0000FFF7FF0000FFF7FF0001FFF7FF0001FFF3FF0001FFF 3FF0001FFF1FF0003FFF0FF8003FFF07F8007FFF03FC006FFF01FE01CFFF00FFEF8FFF00 7FFF0FFF000FFE0FFE0000000FFE0000000FFE0000000FFE0000000FFC0000000FFC03E0 001FFC07F0001FF80FF8001FF81FFC001FF01FFC003FF01FFC003FE01FFC003FC01FF800 7FC00FF000FF800FE001FF0007C003FE0007F01FFC0003FFFFF00001FFFFC000007FFF00 00000FF8000028397CB731>57 D<0000001FFF000030000001FFFFE000F000000FFFFFFC 01F000007FFFFFFE03F00001FFFE007F87F00003FFE0000FCFF0000FFF000003FFF0001F FC000001FFF0003FF80000007FF0007FF00000003FF000FFC00000003FF001FFC0000000 1FF003FF800000000FF007FF000000000FF00FFF0000000007F00FFE0000000007F01FFE 0000000003F01FFE0000000003F03FFC0000000003F03FFC0000000001F03FFC00000000 01F07FFC0000000001F07FF80000000001F07FF80000000000007FF8000000000000FFF8 000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800000000 0000FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF8 000000000000FFF80000000000007FF80000000000007FF80000000000007FF800000000 00007FFC0000000000F03FFC0000000000F03FFC0000000000F03FFC0000000000F01FFE 0000000000F01FFE0000000001E00FFE0000000001E00FFF0000000001E007FF00000000 03C003FF8000000003C001FFC0000000078000FFE00000000F00007FF00000001F00003F F80000003E00001FFC0000007C00000FFF000001F8000003FFE00007F0000001FFFE003F C00000007FFFFFFF000000000FFFFFFC0000000001FFFFF000000000001FFF0000003C3D 7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF0000 1FFC00FF80001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF000 07F0003FF00001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000 000FFFFFF000007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF000 3FE0003FF0007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000 FFC0003FF000FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF0 07FFFE0FFFF001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800 003FFFFE0000FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03F F00007F83FF00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFF FFFFFCFFFFFFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE000 00007FE00000007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF80000 7807FC0000F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE000 26267DA52D>101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF00 00000007FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 00000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 00000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 7FC00003FF01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF60 00FF8003FFE000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC 3FFFFFFFFFFC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00 FFFFFE03FF00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF8 03FF00001FFC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF 03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF 03FF000007FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC 03FF00000FFC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC0 03FFF001FF8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF00000000 03FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000 03FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000 FFFFFC000000FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fv 70 123 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<0020004000800100020006000C000C00180018003000 300030007000600060006000E000E000E000E000E000E000E000E000E000E000E000E000 6000600060007000300030003000180018000C000C000600020001000080004000200B2E 7DA112>40 D<800040002000100008000C00060006000300030001800180018001C000C0 00C000C000E000E000E000E000E000E000E000E000E000E000E000E000C000C000C001C0 01800180018003000300060006000C00080010002000400080000B2E7DA112>I<70F8FC FC74040404080810102040060E7C840D>44 DI<70F8F8F87005 057C840D>I<03F0000E1C001C0E00180600380700700380700380700380700380F003C0 F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0 7003807003807003807807803807001806001C0E000E1C0003F000121F7E9D17>48 D<018003800F80F380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038003800380038007C0FFFE0F1E7C9D17>I<03F0000C 1C00100E00200700400780800780F007C0F803C0F803C0F803C02007C00007C000078000 0780000F00000E00001C0000380000700000600000C0000180000300000600400C004018 00401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E00200F00780F80 780780780780380F80000F80000F00000F00000E00001C0000380003F000003C00000E00 000F000007800007800007C02007C0F807C0F807C0F807C0F00780400780400F00200E00 1C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00002E00002E00004E 00008E00008E00010E00020E00020E00040E00080E00080E00100E00200E00200E00400E 00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0141E7F9D17 >I<1803001FFE001FFC001FF8001FE00010000010000010000010000010000010000011 F000161C00180E001007001007800003800003800003C00003C00003C07003C0F003C0F0 03C0E00380400380400700200600100E000C380003E000121F7E9D17>I<007C00018200 0701000E03800C07801C0780380300380000780000700000700000F1F000F21C00F40600 F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003C0700380380380 3807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF807FFF804001 0080020080020080040000080000080000100000200000200000400000400000C00000C0 0001C0000180000380000380000380000380000780000780000780000780000780000780 00078000030000121F7D9D17>I<03F0000C0C0010060030030020018060018060018060 01807001807803003E03003F06001FC8000FF00003F80007FC000C7E00103F00300F8060 03804001C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C0003F00012 1F7E9D17>I<03F0000E18001C0C00380600380700700700700380F00380F00380F003C0 F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C0000380000380 000380000700300700780600780E00700C002018001070000FC000121F7E9D17>I<70F8 F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F870000000000000 0000000070F0F8F878080808101010202040051D7C930D>I<0FC0307040384038E03CF0 3CF03C603C0038007000E000C00180018001000300020002000200020002000200000000 0000000000000007000F800F800F8007000E207D9F15>63 D<0001000000038000000380 00000380000007C0000007C0000007C0000009E0000009E0000009E0000010F0000010F0 000010F00000207800002078000020780000403C0000403C0000403C0000801E0000801E 0000FFFE0001000F0001000F0001000F00020007800200078002000780040003C00E0003 C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0038005C0070003C00E0001C01E0000C01C0000C0 3C0000C07C0000407C00004078000040F8000000F8000000F8000000F8000000F8000000 F8000000F8000000F8000000F8000000780000007C0000407C0000403C0000401C000040 1E0000800E000080070001000380020001C0040000703800000FC0001A217D9F21>IIII<000FE020 0078186000E004E0038002E0070001E00F0000E01E0000601E0000603C0000603C000020 7C00002078000020F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8007FFCF80003E0780001E07C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0 070001E0038002E000E0046000781820000FE0001E217D9F24>III76 DII<001F800000F0F00001C0380007801E000F000F000E0007001E0007 803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F80001F0F80001 F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C0003E03C0003 C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F000001F80001C217D 9F23>II82 D<07E0800C19801007803003806001806001 80E00180E00080E00080E00080F00000F000007800007F00003FF0001FFC000FFE0003FF 00001F800007800003C00003C00001C08001C08001C08001C08001C0C00180C00380E003 00F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060400F0020400F00 20C00F0030800F0010800F0010800F0010800F0010000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000001F800007FFFE001C1F7E 9E21>II87 D<7FF83FF80FE00FC007C0070003C0020001E0040001 F00C0000F0080000781000007C1000003C2000003E4000001E4000000F8000000F800000 0780000003C0000007E0000005E0000009F0000018F8000010780000207C0000603C0000 401E0000801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22 >I<080410082010201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17> 92 D<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E001E0E00 380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>I E /Fw 46 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000071C000C060018030038 03803803807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C038 03803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038007C0FFFE0F217CA018>I<03F0000C1C0010070020 07804003C04003C08003E0F003E0F801E0F801E0F801E02003E00003E00003C00003C000 0780000700000E00001C0000180000300000600000C00001800001000002002004002008 00201800603000403FFFC07FFFC0FFFFC013217EA018>I<03F8000C1E00100700200780 4007C07807C07803C07807C03807C0000780000780000700000F00000E0000380003F000 001C00000F000007800007800003C00003C00003E02003E07003E0F803E0F803E0F003C0 4003C0400780200780100F000C1C0003F00013227EA018>I<000200000600000E00000E 00001E00001E00002E00004E00004E00008E00008E00010E00020E00020E00040E00040E 00080E00100E00100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E 00000E00000E00000E00001F0001FFF015217FA018>I<1000801E07001FFF001FFE001F F80013E00010000010000010000010000010000010000010F800130E0014070018038010 03800001C00001C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C040 03C04003802007001006000C1C0003F00013227EA018>I<007E0001C1000300800601C0 0E03C01C03C0180180380000380000780000700000700000F0F800F30C00F40600F40300 F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C0 3801C01803801C03000C0600070C0001F00013227EA018>I<01F800060E000803001001 802001802000C06000C06000C06000C07000C07801803E01003F02001FC4000FF80003F8 0003FC00067F00083F80100F803007C06001C06000E0C000E0C00060C00060C00060C000 606000406000C03000801803000E0E0003F00013227EA018>56 D<01F000060C000C0600 180700380380700380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07003E03803E01805E00C05E00619E003E1E00001C00001C00001C0000380000380 300300780700780600700C002018001030000FC00013227EA018>I<0001800000018000 000180000003C0000003C0000003C0000005E0000005E000000DF0000008F0000008F000 0010F800001078000010780000203C0000203C0000203C0000401E0000401E0000401E00 00800F0000800F0000FFFF000100078001000780030007C0020003C0020003C0040003E0 040001E0040001E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E0000701E0000701C0000 303C0000303C0000307C0000107800001078000010F8000000F8000000F8000000F80000 00F8000000F8000000F8000000F800000078000000780000107C0000103C0000103C0000 101C0000201E0000200E000040070000400380008001C0010000E0020000381C000007E0 001C247DA223>67 D69 DI73 D77 DI80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060 E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC0 0007E00001E00000F00000F0000070800070800070800070800070C00060C00060E000C0 F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840 07800840078008C007800C80078004800780048007800480078004000780000007800000 078000000780000007800000078000000780000007800000078000000780000007800000 078000000780000007800000078000000780000007800000078000000780000007800000 078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700 00070000070000FF0007C7001E07003C0700780700700700F00708F00708F00708F00F08 7817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E 00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E 00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300 780000700000F00000F00000F00000F00000F00000F00000F00000700000780040380040 1C00800C010007060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01 E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800 E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C038 01C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F0000070000078002038 00201C00400E008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007 00070007000700070007000700FFF8070007000700070007000700070007000700070007 00070007000700070007000700070007807FF8102380A20F>I<00007001F198071E180E 0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010 00001000001800001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0 00186000306000303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C0 0E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E00 1C00000000000000000000000000000000000E00FE001E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003 E003E001C00000000000000000000000000000000001E00FE001E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000 E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E0180 0E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C0 0E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00FE60E183 801E807201C00F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E0 0E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60 C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C 01801800C03800E0700070700070F00078F00078F00078F00078F00078F00078F0007870 00707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E8060 0F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C 0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E0000 0E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E 000E000E000E000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>III121 D E /Fx 20 118 df45 D68 D73 D77 D 80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E070 0000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF0000 0FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880 00007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0F0 0000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fy 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 908 a Fy(D)26 b(R)g(A)f(F)h(T)225 999 y Fx(Do)r(cumen)n(t) 20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1194 y Fw(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)824 1320 y(April)g(21,)i(1996)77 1378 y(This)g(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1436 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1494 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1552 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 45 a Fv(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fu(Chapter)34 b(9)75 564 y Ft(Miscellan)m(y)75 805 y Fs(9.1)59 b(Intro)r(duction)75 906 y Fv(There)12 b(are)g(a)g(n)o(um)o(b)q(er)g(of)g(topics)g(that)f(do)h(not)g(\014t)g (con)o(v)o(enien)o(tly)h(in)o(to)f(the)g(other)g(c)o(hapters.)19 b(W)l(e)12 b(collect)75 963 y(them)j(here.)75 1104 y Fs(9.2)59 b(Standa)n(rdize)20 b(mpirun)75 1206 y Fv(A)13 b(n)o(um)o(b)q(er)h(of)f(implemen)o(tations)h(of)f(MPI-1)g(pro)o(vide)h (a)e(startup)h(command)g(for)g(MPI)g(programs)f(that)75 1262 y(is)k(of)e(the)i(form)170 1346 y Fr(mpirun)23 b()g()g()75 1430 y Fv(Separating)16 b(the)g(command)g(to)g(start)f(the)h(program)f(from) g(the)h(program)f(itself)i(pro)o(vides)g(\015exibilit)o(y)l(,)75 1487 y(particularly)23 b(for)e(net)o(w)o(ork)g(and)h(heterogeneous)g (implemen)o(tations.)41 b(F)l(or)21 b(example,)j(the)e(startup)75 1543 y(script)16 b(need)h(not)e(run)h(on)g(one)g(of)f(the)h(mac)o (hines)g(that)g(will)h(b)q(e)f(executing)h(the)f(MPI)g(program)e (itself.)75 1600 y(The)20 b(curren)o(t)f(goal)g(of)g(this)h(c)o(hapter) f(is)h(to)f(determine)i(ho)o(w)d(m)o(uc)o(h)i(of)f Fr()18 b Fv(can)75 1656 y(b)q(e)k(made)g(p)q(ortable)g(across)f (all)h(implemen)o(tations,)i(and)e(th)o(us)f(standardized.)40 b(F)l(or)21 b(example,)i(w)o(e)75 1713 y(curren)o(tly)16 b(prop)q(ose)f(that)170 1797 y Fr(mpirun)23 b(-np)h()e ()75 1881 y Fv(b)q(e)c(at)f(least)g(one)g(w)o(a)o(y)g(to)f (start)g Fr()g Fv(with)i(an)f(initial)j Fq(MPI)p 1275 1881 14 2 v 15 w(COMM)p 1432 1881 V 17 w(W)o(ORLD)d Fv(whose)g(group)75 1937 y(con)o(tains)25 b Fr()e Fv(pro)q(cesses.)48 b(Other)24 b(argumen)o(ts)g(to)g Fr(mpirun)g Fv(ma)o(y)f(b)q(e)j(implemen)o(tation-)75 1994 y(dep)q(enden)o(t.)166 2050 y(A)19 b(starting)f(p)q(oin)o(t)h(is)g (to)f(merge)g(the)h(b)q(eha)o(vior)g(of)f(the)h Fr(mpirun)f Fv(commands)g(in)i(use)f(for)f(NAS,)75 2107 y(LAM,)d(and)g(MPICH.)g (All)i(supp)q(ort)e(a)g(v)o(ersion)g(of)g(the)g(ab)q(o)o(v)o(e)g (argumen)o(t.)166 2163 y(NAS)g(supp)q(orts)h(a)f(syn)o(tax)f(for)h (running)h(m)o(ultiple)h(executables:)170 2256 y Fr(mpirun)23 b(-np)h(2)g(foo)f(:)h(-np)f(4)h(bar)f(:)h(-np)g(3)f(bazz)166 2349 y Fv(and)c(MPICH)f(supp)q(orts)h(a)f(mec)o(hanism)i(for)e(\014n)o (ting)h(and)g(running)h(similarly-named)h(binaries)75 2405 y(for)15 b(di\013eren)o(t)g(arc)o(hitectures:)170 2498 y Fr(mpirun)23 b(-arch)h(sun4)f(-np)g(4)h(-arch)f(rs6000)g(-np)h (3)g(myprog)166 2591 y Fv(All)16 b(three)g(ha)o(v)o(e)e(the)i(option)f (of)g(a)g(\\con\014guration)g(\014le")h(for)e(closer)i(con)o(trol.)166 2647 y(All)f(three)e(ha)o(v)o(e)g(man)o(y)g(other)g(options)h(sp)q (ei\014c)h(to)e(their)h(o)o(wn)f(implmenetions)i(or)e(en)o(vironmen)o (ts.)75 2704 y(It)i(seems)h(unnecessary)f(to)g(try)g(to)f(merge)h(all)h (of)f(them.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fv(2)1156 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fs(9.3)59 b(Datat)n(yp)r(e)18 b(Imp)n(rovements)75 148 y Fo(9.3.1)49 b(Simple)17 b(Struct)d(T)l(yp)q(es)75 329 y Fq(MPI)p 160 329 14 2 v 16 w(TYPE)p 293 329 V 17 w(SIMPLE)p 469 329 V 15 w(STRUCT\(count,)j(a)o(rra)o(y)p 908 329 V 14 w(of)p 959 329 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1334 329 V 14 w(of)p 1385 329 V 16 w(t)o(yp)q(es,)e(newt)o(yp)q(e\))117 406 y Fn(IN)155 b Fq(count)482 b Fn(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 481 y(IN)155 b Fq(a)o(rra)o(y)p 416 481 V 15 w(of)p 468 481 V 16 w(blo)q(cklengths) 191 b Fn(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h(eac)o(h)g(blo)q (c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 556 y(IN)155 b Fq(a)o(rra)o(y)p 416 556 V 15 w(of)p 468 556 V 16 w(t)o(yp)q(es)327 b Fn(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h(in)f(eac)o(h)h(blo)q(c)o(k)g (\(arra)o(y)g(of)f(handle\))117 631 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Fn(new)15 b(datat)o(yp)q(e)f(\(handle\))166 756 y Fv(This)22 b(t)o(yp)q(e)g(constructor)f(is)i(similar)f(to)g Fq(MPI)p 988 756 V 15 w(TYPE)p 1120 756 V 17 w(STRUCT)p Fv(,)g(except)h(that)e (the)h(user)g(do)q(es)75 812 y(not)e(supply)h(an)f(arra)o(y)e(of)i (displacemen)o(ts.)35 b(Instead,)22 b(the)e(successiv)o(e)h(blo)q(c)o (ks)f(are)g(assumed)g(to)f(b)q(e)75 869 y(con)o(tiguous.)g(P)o(adding) 11 b(spaces)h(are)e(added,)j(according)e(to)g(the)g(default)h(alignmen) o(t)g(rules)g(for)e(structure)75 925 y(comp)q(onen)o(ts.)21 b(This)c(facilitates)f(the)g(construction)g(of)f(datat)o(yp)q(es)g(for) g(structures,)g(as)h(the)f(user)h(need)75 981 y(not)f(compute)g (displacemen)o(ts.)75 1088 y Fm(Example)j(9.1)k Fv(The)14 b(follo)o(wing)g(co)q(de)f(declares)h(a)f(structure)g(t)o(yp)q(e)g(and) h(creates)f(a)g(datat)o(yp)q(e)f(for)h(that)75 1144 y(structure)i(t)o (yp)q(e.)75 1250 y Fr(struct)23 b(record)g({)147 1307 y(char)g(name;)147 1363 y(double)g(position[3];)147 1420 y(float)47 b(mass;)147 1476 y(})75 1589 y(MPI_Datatype)22 b(record_type;)75 1646 y(MPI_datatype)g(types[3])h(=)h({MPI_CHAR,)e (MPI_DOUBLE,)h(MPI_FLOAT};)75 1702 y(int)238 b(lenghts[3])23 b(=)g({1,)h(3,)f(1};)75 1815 y(MPI_Type_simple_struct\()e(3,)i (lengths,)g(types,)g(record_type\);)166 1997 y Fl(Discussion:)40 b Fn(The)16 b(co)q(de)g(ab)q(o)o(v)o(e)f(is)h(m)o(uc)o(h)e(simpler)g (and)i(more)e(natural)h(than)g(the)i(curren)o(t)f(approac)o(h,)75 2047 y(that)e(uses)h Fk(MPI)p 330 2047 13 2 v 14 w(TYPE)p 452 2047 V 15 w(STRUCT)p Fn(,)d(and)i(requires)h(to)f(compute)f (displacemen)o(ts.)166 2097 y(It)g(also)g(pro)o(vides)g(a)g (de\014nition)f(of)h(the)h(record)g(t)o(yp)q(e)g(that)f(is)g(arc)o (hitecture)i(indep)q(enden)o(t)f(\(assuming)e(that)75 2147 y(compilers)i(don't)i(\014ddle)f(with)g(the)i(order)f(of)f (\014elds,)h(whic)o(h)f(is)g(true)i(in)e(C\))h({)f(this)g(ma)o(y)f (\014t)i(w)o(ell)e(with)i(v)n(arious)75 2197 y(prop)q(osals)e(for)f (third)h(part)o(y)g(transfers)h(and/or)f(one)g(sided)g(comm)o (unications.)166 2246 y(Ho)o(w)o(ev)o(er,)22 b(this)f(simpli\014cation) d(places)j(a)f(signi\014can)o(t)h(burden)g(on)g(the)g(MPI)g(implem)o (en)o(tation.)36 b(The)75 2296 y(implemen)o(tatio)o(n)17 b(will)g(ha)o(v)o(e)i(to)g(\\kno)o(w")f(all)g(the)h(padding)g(rules)g (that)g(the)h(compiler)e(uses)i(for)f(la)o(ying)e(out)75 2346 y(structure)g(t)o(yp)q(es)f(in)f(memory)m(.)k(W)m(orse)d(still,)e (the)i(padding)e(rules)i(often)f(v)n(ary)g(from)e(compiler)h(to)h (compiler)f(or)75 2396 y(v)o(ersion)h(to)g(v)o(ersion.)22 b(They)15 b(ma)o(y)f(also)g(b)q(e)i(mo)q(di\014ed)d(b)o(y)i(user)i(sp)q (eci\014ed)f(pragmas)e(or)h(options.)21 b(F)m(urther,)16 b(this)75 2446 y(function)g(will)e(only)i(b)q(e)g(v)n(alid)f(for)h (languages)f(whic)o(h)h(guaran)o(tee)h(not)f(to)g(reorder)h(\014elds)g (in)f(a)f(structure)k(\(e.g.)75 2495 y(C\),)13 b(but)i(not)e(languages) h(lik)o(e)f(Ada)h(\(or)g(F)m(ortran?\).)166 2552 y(Th)o(us,)f(this)g (function)g(will)f(b)q(e)i(di\016cult)e(\(or)i(imp)q(ossible\))d(to)j (include)f(in)g(a)g(library)f(only)g(implemen)o(tation)75 2608 y(of)h(MPI;)h(instead)g(it)g(will)e(require)j(close)f(in)o (tegration)f(with)h(the)g(compiler.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fp(9.3.)34 b(D)o(A)l(T)l(A)l(TYPE)15 b(IMPR)o(O)o(VEMENTS)1019 b Fv(3)75 45 y Fo(9.3.2)49 b(Contiguous)17 b(Struct)e(T)l(yp)q(es)75 131 y Fv(The)c(de\014nition)i (of)e(a)g(datat)o(yp)q(e)g(exten)o(t)f(ma)o(y)h(sp)q(ecify)h(that)f(a)g (space)g(b)q(e)h(added)g(after)e(the)h(last)g(datat)o(yp)q(e)75 187 y(comp)q(onen)o(t,)16 b(for)g(alignmen)o(t)h(purp)q(oses.)24 b(If)16 b(suc)o(h)h(datat)o(yp)q(e)f(is)g(replicated)i(con)o(tiguously) f(in)g(a)f(blo)q(c)o(k)75 244 y(b)o(y)d(one)h(of)f(the)h(datat)o(yp)q (e)f(constructors,)f(then)i(successiv)o(e,)h(con)o(tiguous)e(comp)q (onen)o(ts)h(of)f(a)g(blo)q(c)o(k)h(will)75 300 y(b)q(e)i(separated)f (b)o(y)g(these)g(padding)i(spaces.)j(Also,)15 b(a)g(datat)o(yp)q(e)f (constructed)i(b)o(y)75 357 y Fq(MPI)p 160 357 14 2 v 16 w(TYPE)p 293 357 V 17 w(STRUCT)g Fv(ma)o(y)e(con)o(tain)h(in)o (ternal)h(holes,)g(b)q(ecause)g(of)f(alignmen)o(t)h(requiremen)o(ts.) 166 413 y(Suc)o(h)f(holes)g(\(padding)g(space)g(b)q(et)o(w)o(een)f (successiv)o(e)i(comp)q(onen)o(ts)e(of)g(a)g(blo)q(c)o(k)h(or)f (padding)i(space)75 470 y(b)q(et)o(w)o(een)c(comp)q(onen)o(ts)f(of)g(a) g(datat)o(yp)q(e)g(constructed)g(with)h Fq(MPI)p 1188 470 V 16 w(TYPE)p 1321 470 V 17 w(STRUCT)p Fv(\))f(generally)i(con)o (tain)75 526 y(no)18 b(signi\014can)o(t)i(information.)29 b(Ho)o(w)o(ev)o(er,)18 b(an)g(MPI)h(implemen)o(tation)g(is)g(not)f (allo)o(w)o(ed)h(to)f(o)o(v)o(erwrite)75 583 y(these)d(holes)g(when)g (data)f(is)h(receiv)o(ed)h(in)o(to)f(a)f(comm)o(unication)i(bu\013er,)e (th)o(us)g(precluding)j(the)e(natural)75 639 y(optimization)h(of)f(blo) q(c)o(k)h(cop)o(ying)g(data)e(in)o(to)h(the)h(bu\013er.)166 695 y(The)23 b(t)o(yp)q(e)h(constructor)e Fq(MPI)p 711 695 V 16 w(TYPE)p 844 695 V 17 w(CONTIGUOUS)p 1149 695 V 18 w(STRUCT)i Fv(marks)f(an)o(y)g(suc)o(h)g(holes)h(as)75 752 y(\\don)o(t-care")14 b(lo)q(cations,)i(p)q(ermiting)g(the)g (implemen)o(tation)g(to)f(o)o(v)o(erwrite)f(them.)75 903 y Fq(MPI)p 160 903 V 16 w(TYPE)p 293 903 V 17 w(CONTIGUOUS)p 598 903 V 18 w(STRUCT\(count,)k(a)o(rra)o(y)p 1041 903 V 14 w(of)p 1092 903 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1468 903 V 14 w(of)p 1519 903 V 16 w(displacements,)g(a)o(r-)75 959 y(ra)o(y)p 136 959 V 15 w(of)p 188 959 V 16 w(t)o(yp)q(es,)e(newt)o (yp)q(e\))117 1037 y Fn(IN)155 b Fq(count)482 b Fn(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 1110 y(IN)155 b Fq(a)o(rra)o(y)p 416 1110 V 15 w(of)p 468 1110 V 16 w(blo)q(cklengths)191 b Fn(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h (eac)o(h)g(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 1184 y(IN)155 b Fq(a)o(rra)o(y)p 416 1184 V 15 w(of)p 468 1184 V 16 w(displacements)164 b Fn(b)o(yte)14 b(displacemen)o(t)g (of)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 1258 y(IN)155 b Fq(a)o(rra)o(y)p 416 1258 V 15 w(of)p 468 1258 V 16 w(t)o(yp)q(es)327 b Fn(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h (in)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f(handle\))117 1332 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Fn(new)15 b(datat)o(yp)q(e)f (\(handle\))166 1457 y Fv(The)21 b(seman)o(tics)h(of)e Fq(MPI)p 620 1457 V 16 w(TYPE)p 753 1457 V 17 w(CONTIGUOUS)p 1058 1457 V 18 w(STRUCT)i Fv(are)f(otherwise)g(iden)o(tical)i(to)e(the) 75 1513 y(function)16 b Fq(MPI)p 338 1513 V 16 w(TYPE)p 471 1513 V 17 w(STRUCT)p Fv(.)189 1611 y Fi(R)n(ationale.)38 b Fv(C)12 b(struct)f(datat)o(yp)q(es)h(ma)o(y)g(con)o(tain)g(holes)h (due)g(to)e(the)i(need)g(to)e(align)i(basic)g(t)o(yp)q(es.)189 1667 y(F)l(or)g(example,)i(the)f(C)g(struct)f(de\014ned)j(in)f(Example) g(9.1)e(ma)o(y)g(con)o(tain)h(a)g(3)g(b)o(yte)g(hole)h(after)e(the)189 1724 y(c)o(har)h(on)h(a)f(32)g(bit)h(mac)o(hine,)g(where)g(doubles)h (are)e(w)o(ord)g(aligned,)i(or)e(a)g(7)h(b)o(yte)f(hole)i(if)f(doubles) 189 1780 y(are)g(doublew)o(ord)g(aligned.)189 1854 y(Although)k(the)f (ab)q(o)o(v)o(e)g(t)o(yp)q(e)h(con)o(tains)g(a)f(hole,)h(for)f (e\016ciency)i(it)f(is)g(desirable)h(that)e(the)h Fq(MPI)189 1911 y Fv(implemen)o(tation)j(treat)e(the)i(ab)q(o)o(v)o(e)e(structure) h(as)g(a)g(sequence)h(of)f(con)o(tiguous)h(b)o(ytes)e(when)189 1967 y(sending)13 b(the)g(t)o(yp)q(e)f(b)q(et)o(w)o(een)h(homogeneous)f (mac)o(hines,)h(and)g(hence)g(mo)o(v)o(e)f(it)h(using)g(an)f(e\016cien) o(t)189 2024 y(memory)21 b(cop)o(y)l(,)j(e.g.)41 b(memp)q(cy\(\).)g (Otherwise,)24 b(the)f(implemen)o(tation,)i(when)d(receiving)i(a)189 2080 y(message)14 b(of)g(the)h(struct)f(datat)o(yp)q(e)g(m)o(ust)g (actually)i(p)q(erform)e(an)h(elemen)o(t)g(b)o(y)g(elemen)o(t)g(cop)o (y)g(to)189 2137 y(ensure)c(that)g(it)g(preserv)o(es)h(the)f(con)o(ten) o(ts)f(of)h(the)g(holes)h(in)g(the)g(destination.)19 b(\()p Fi(End)12 b(of)h(r)n(ationale.)p Fv(\))189 2234 y Fi(A)n(dvic)n(e)18 b(to)i(users.)61 b Fv(If)19 b Fq(MPI)p 706 2234 V 15 w(TYPE)p 838 2234 V 17 w(STRUCT)h Fv(is)f(used)g(to)f (create)g(an)h Fq(MPI)f Fv(datat)o(yp)q(e)g(corre-)189 2291 y(sp)q(onding)h(to)f(the)h(C)f(datat)o(yp)q(e)g(de\014ned)i(in)g (Example)f(9.1,)f(there)h(is)g(no)f(w)o(a)o(y)g(to)g(indicate)i(to)189 2347 y(the)f Fq(MPI)h Fv(implemen)o(tation)h(that)e(the)h(hole)g(is)g (actually)h(a)e(sequence)i(of)f(\\don)o(t-care")f(b)o(ytes.)189 2404 y(Ho)o(w)o(ev)o(er,)f(if)h Fq(MPI)p 519 2404 V 16 w(TYPE)p 652 2404 V 17 w(CONTIGUOUS)p 957 2404 V 18 w(STRUCT)g Fv(is)g(used)h(to)e(create)g(an)h Fq(MPI)f Fv(datat)o(yp)q(e,)189 2460 y(this)d(indicates)i(to)d(the)i(implemen)o(tation)g(that)f(the)g (padding)h(spaces)g(are)f(not)f(signi\014can)o(t.)189 2534 y(Only)j(padding)f(spaces)g(b)q(et)o(w)o(een)g(\014elds)h(of)e (the)h(struct)g(can)f(b)q(e)i(o)o(v)o(erwritten,)d(an)o(y)i(hole)g (added)189 2591 y(implicitly)22 b(b)o(y)d Fq(MPI)g Fv(due)h(to)f(the)g (default)h(exten)o(t)f(will)i(not)e(b)q(e)h(o)o(v)o(erwritten.)32 b(On)20 b(the)f(other)189 2647 y(hand,)13 b(if)g(the)g(user)g (explicitly)j(de\014ned)e(the)f(upp)q(er)h(b)q(ound)g(of)e(a)h(datat)o (yp)q(e,)f(using)i Fk(MPI)p 1693 2647 13 2 v 14 w(UB)p Fv(,)e(then)189 2704 y(holes)i(in)g(the)f(datat)o(yp)q(e)g(that)f (preceded)j(the)e(user)g(de\014ned)i(upp)q(er)f(b)q(ound)g(will)h(b)q (e)f(o)o(v)o(erwritten.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fv(4)1161 b Fp(CHAPTER)15 b(9.)30 b(MISCELLANY)189 45 y Fv(Users)17 b(should)i(arrange)f(the)g(\014elds)h(in)f(a)g(C)g (struct)f(t)o(yp)q(e)h(in)h(descending)h(order)d(of)h(their)g(size.)189 102 y(This)j(arrangemen)o(t)e(has)h(t)o(w)o(o)f(adv)m(an)o(tages:)30 b(\014rstly)20 b(it)h(ma)o(y)e(reduce)i(the)g(size)g(of)f(the)g(struct) 189 158 y(b)o(y)c(eliminating)i(padding)f(b)q(et)o(w)o(een)f(elemen)o (ts)h(with)f(di\013eren)o(t)h(alignmen)o(ts)f(\(this)g(is)h(generally) 189 214 y(true)h(for)f(C)h(programs\).)28 b(Secondly)l(,)20 b(it)f(ma)o(y)e(increase)i(the)g(n)o(um)o(b)q(er)f(of)g(cases)g(for)g (whic)o(h)h(the)189 271 y(implemen)o(tation)i(can)e(p)q(erform)h(blo)q (c)o(k)g(copies)h(b)o(y)f(ensuring)g(that)f(the)h(\014rst)f(elemen)o(t) i(of)e(the)189 327 y(structure)c(has)g(the)g(most)f(stringen)o(t)h (alignmen)o(t)h(requiremen)o(ts.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fv(\))189 433 y Fi(A)n(dvic)n(e)i(to)i (implementors.)62 b Fv(An)19 b(implemen)o(tation)i(can)e(tak)o(e)f(adv) m(an)o(tage)g(of)h(the)g(leew)o(a)o(y)f(of)189 490 y(o)o(v)o(erwriting) 12 b(holes,)i(if)f(data)f(sen)o(t)h(is)g(alw)o(a)o(ys)f(aligned)i (using)g(the)f(same)f(rules)i(that)e(the)h(compiler)189 546 y(uses)18 b(for)f(\014elds)h(within)h(a)f(struct)f(t)o(yp)q(e.)27 b(E.g.,)17 b(if)h(the)f(compiler)i(forces)f(half-w)o(ord)f(en)o(tries)h (b)q(e)189 602 y(half-w)o(ord)c(aligned,)h(and)g(w)o(ord)e(en)o(tries)i (b)q(e)g(w)o(ord)e(aligned,)j(then)e(the)g(implemen)o(tation)i(should) 189 659 y(add)c(\\don)o(t-care")f(b)o(ytes)g(to)g(the)h(message)g(to)f (matc)o(h.)18 b(The)12 b(receiving)i(pro)q(cess)e(can)g(iden)o(tify)h (the)189 715 y(lo)q(cations)18 b(where)f(\\don)o(t-care")f(b)o(ytes)h (where)h(added)g(from)e(the)i(basic)f(t)o(yp)q(es)h(of)e(the)i(receiv)o (ed)189 772 y(en)o(tries.)24 b(It)17 b(can)g(iden)o(tify)h(\(at)e (datat)o(yp)q(e)g(commit)h(time\))f(when)h(suc)o(h)h(\\don)o(t-care")d (b)o(ytes)i(can)189 828 y(b)q(e)f(copied)g(in)o(to)f(the)g(receiv)o(e)h (bu\013er,)f(without)g(o)o(v)o(erwriting)g(signi\014can)o(t)i (information.)189 903 y(NOTE:)g(Compilers)i(generally)g(align)g (structures)e(based)i(on)f(the)f(most)h(stringen)o(t)f(alignmen)o(t)189 960 y(constrain)o(ts)f(of)f(their)i(elemen)o(ts.)24 b(F)l(or)16 b(example,)h(the)g(C)f(struct)g(de\014ned)i(in)f(Example)g(9.1)e(will) 189 1016 y(often)c(b)q(e)i(aligned)g(on)f(an)g(8)g(b)o(yte)g(b)q (oundary)l(,)g(ev)o(en)h(though)f(the)g(\014rst)f(elemen)o(t)i(could)g (b)q(e)g(aligned)189 1073 y(on)g(a)f(1)h(b)o(yte)g(b)q(oundary)l(.)20 b(Th)o(us)13 b(arra)o(ys)f(of)g(this)i(structure)f(will)h(often)f(ha)o (v)o(e)g(4)f(b)o(ytes)h(of)g(padding)189 1129 y(b)q(et)o(w)o(een)g (successiv)o(e)h(replications.)21 b(In)14 b(con)o(trast,)d(the)i (elemen)o(ts)h(pac)o(k)o(ed)f(in)o(to)g(a)g(message)g(m)o(ust)189 1186 y(alw)o(a)o(ys)h(use)i(an)g(alignmen)o(t)g(rule)g(based)g(solely)h (on)e(their)h(o)o(wn)f(t)o(yp)q(e)h(\(and)f(not)g(their)h(con)o (text\).)189 1242 y(Th)o(us,)f(ev)o(en)i(though)e(the)h(\014rst)g(c)o (har)f(is)i(part)e(of)g(a)h(struct)f(t)o(yp)q(e,)h(their)h(should)g(b)q (e)f(no)g(padding)189 1299 y(b)q(et)o(w)o(een)g(the)g(\015oat)f(at)h (the)g(end)h(of)e(one)h(replication,)i(and)e(the)g(c)o(har)g(at)f(the)h (b)q(eginning)i(of)e(the)189 1355 y(next)g(replication.)24 b(This)16 b(is)h(neccessary)f(to)g(meet)g(the)g(full)h Fq(MPI)e Fv(t)o(yp)q(e)h(matc)o(hing)g(rules,)h(whic)o(h)189 1411 y(allo)o(w)d(an)o(y)h(t)o(w)o(o)e(t)o(yp)q(es)i(to)f(matc)o(h)g (so)g(long)h(as)f(their)h(basic)g(elemen)o(ts)h(matc)o(h.)j(This)c (will)h(reduce)189 1468 y(the)d(o)q(ccasions)g(in)h(whic)o(h)f(the)g (implemen)o(tation)h(can)f(optimize)h(the)f(mo)o(v)o(emen)o(t)f(of)h (struct)f(t)o(yp)q(es)189 1524 y(con)o(taining)k(holes.)k(\()p Fi(End)c(of)g(advic)n(e)g(to)h(implementors.)p Fv(\))166 1706 y Fl(Discussion:)52 b Fn(Supp)q(ose)20 b(w)o(e)e(concatenated)i (10)e(copies)h(of)f(a)h(datat)o(yp)q(e.)32 b(The)19 b(sp)q (eci\014cation)g(in)f(this)75 1756 y(section)f(sa)o(ys)g(that)g(the)g (9)f(holes)h(b)q(et)o(w)o(een)h(the)f(copies)g(and)g(the)g(10'th)f (hole)g(after)h(the)g(last)g(cop)o(y)f(can)h(all)e(b)q(e)75 1806 y(o)o(v)o(erwritten)f(\(so)f(long)f(as)h(the)h(user)g(has)f (explicitly)f(set)i(the)g(exten)o(t)g(of)e(the)i(struct)g(t)o(yp)q(e)g (using)f Fk(MPI)p 1685 1806 13 2 v 14 w(UB)p Fn(\).)f(This)75 1856 y(migh)o(t)h(not)i(b)q(e)g(safe)g(if)f(the)i(space)g(allo)q(cated) e(to)h(an)f(arra)o(y)h(is)g(not)f(a)h(m)o(ultiple)e(of)h(the)h(\\exten) o(t")h(of)e(eac)o(h)h(arra)o(y)75 1905 y(comp)q(onen)o(t.)i(An)o(yb)q (o)q(dy)d(kno)o(ws)g(of)f(a)h(compiler)e(that)i(do)q(es)h(not)f (satisfy)f(this)h(rule?)166 1962 y(T)m(o)d(increase)j(the)f(n)o(um)o(b) q(er)f(of)f(o)q(ccasions)i(for)f(whic)o(h)g(the)h Fk(MPI)f Fn(implemen)o(tati)o(on)e(can)i(optimize)f(the)i(mo)o(v)o(e-)75 2018 y(men)o(t)e(of)h(struct)h(t)o(yp)q(es)g(con)o(taining)f(holes)g (\(without)g(relying)g(on)f(the)i(user)h(ordering)e(\014elds)g(in)g (decreasing)i(size\),)75 2075 y(w)o(e)k(migh)o(t)f(add)h(a)g(further)h (restriction)g(to)f(the)h Fk(MPI)e Fn(t)o(yp)q(e)i(matc)o(hing)d (rules.)32 b(This)18 b(restriction)h(w)o(ould)f(only)75 2131 y(allo)o(w)11 b(t)o(yp)q(es)j(created)g(b)o(y)f Fk(MPI)p 568 2131 V 14 w(TYPE)p 690 2131 V 14 w(CONTIGUOUS)p 971 2131 V 13 w(STRUCT)f Fn(to)g(matc)o(h)g(iden)o(tical)g(t)o(yp)q(es) i(also)e(created)i(b)o(y)75 2188 y Fk(MPI)p 152 2188 V 14 w(TYPE)p 274 2188 V 14 w(CONTIGUOUS)p 555 2188 V 13 w(STRUCT)d Fn(\(or)g(with)g Fk(MPI)p 972 2188 V 14 w(P)m(A)o(CKED)p Fn(\).)d(This)j(restriction)h(w)o(ould)f(allo)o(w)e (the)j(imple-)75 2244 y(men)o(tation)f(to)i(add)g(the)g(same)f(padding) g(b)q(et)o(w)o(een)i(replications)f(of)f(the)i(t)o(yp)q(e)f(in)g(a)f (message)h(that)g(the)g(compiler)75 2301 y(uses)i(b)q(et)o(w)o(een)g (replications)f(of)f(the)i(struct)g(t)o(yp)q(e)g(in)e(an)h(arra)o(y)m (.)75 2505 y Fo(9.3.3)49 b(MPI)p 313 2505 15 2 v 18 w(T)l(yp)q(e)p 430 2505 V 18 w(Datat)o(yp)q(e)75 2647 y Fr(Here)23 b(is)h(a)g(draft;)f (it)g(has)h(a)f(number)g(of)h(questions/issues)e(for)h(discussion)g(in) g(it.)75 2704 y(The)g(main)h(problem)f(is)g(the)h(data-dependence)e(on) h(the)h(size)f(of)g(the)h(representation)e(of)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fp(9.3.)29 b(D)o(A)l(T)l(A)l(TYPE)15 b(IMPR)o(O)o(VEMENTS)1024 b Fv(5)75 45 y Fr(the)23 b(MPI_DATATYPE,)g (and)g(the)g(need)h(to)f(avoid)h(constraining)e(the)h(implementation)f (of)75 102 y(MPI_PACK/UNPACK.)75 214 y(Datatype)h(encoding)g(with)g (MPI_Pack/MPI_Unpack)75 327 y(Rationale)75 384 y(An)h(MPI)f(datatype)g (is)g(an)h(object)f(will)g(a)h(well-defined)e(representation)g(\(the)i (type)f(map\))75 440 y(which)g(is)h(representable)e(on)h(all)h (platforms.)46 b(Certain)23 b(applications)f(need)i(to)f(\(a\))75 497 y(send)g(a)h(datatype)f(between)g(processors)f(and)i(\(b\))f (examine)g(and)g(create)h(MPI)f(datatypes)75 553 y(\(e.g.,)g(I/O)g (operations)g(or)h(user-defined)e(operations)g(\(MPI_Op\))h(on)h (complex)75 610 y(datatypes\).)75 723 y(MPI)f(already)g(has)h(routines) f(to)g(convert)g(to)h(and)f(from)g(an)h(opaque)f(external)75 779 y(representation;)f(these)h(are)g(MPI_Pack)g(and)h(MPI_Unpack.)46 b(These)23 b(can)h(be)f(used)g(to)75 835 y(convert)g(an)g(MPI_Datatype) g(into)g(a)h(form)f(that)g(can)h(be)f(sent)h(between)f(processors)f (\(case)75 892 y(\(a\))h(above\).)75 1005 y(Propose:)75 1061 y(A)h(new)f(datatype,)g(MPI_DATATYPE,)f(which)h(may)h(be)f(used)g (in)h(MPI_PACK/MPI_UNPACK.)75 1174 y(Issues:)75 1231 y(Unlike)f(other)g(MPI)h(datatypes,)e(the)i(amount)f(of)g(space)g (needed)g(to)h(pack)f(an)h(MPI_DATATYPE)75 1287 y(depends)f(on)g(the)h (data.)47 b(To)24 b(determine)e(the)i(amount)f(of)g(space,)g(we)h(add) 75 1400 y(MPI_PACK_SIZE_DATA\()d(void)j(*,)f(MPI_Datatype,)f(MPI_Comm,) h(int)g(*)h(\))75 1513 y(where)f(the)h(first)f(argument)g(points)g(to)g (the)h(data)f(and)g(the)h(last)f(is)h(the)f(length)75 1569 y(required.)46 b(Unlike)24 b(MPI_PACK_SIZE,)d(there)j(is)f(no)h (count)f(argument;)75 1626 y(MPI_PACK_SIZE_DATA)e(gives)j(you)f(only)g (the)h(size)f(to)h(pack)f(the)g(one)h(instance)f(of)g(the)75 1682 y(datatype.)75 1795 y(Example)147 1908 y(MPI_PACK_SIZE_DATA\()e (MPI_INT,)i(MPI_DATATYPE,)f(MPI_COMM_WORLD,)g(&len)h(\))75 2021 y(In)h(order)f(to)g(expose)g(the)h(representation)e(used)h(by)h (MPI_PACK,)e(we)i(must)f(choose)g(both)75 2077 y(a)h(representation)e (form)h(and)g(how)h(to)f(access)h(it.)47 b(The)23 b(form)h(in)f(the)h (previous)e(drafts)75 2134 y(is)i(fine.)75 2247 y(Binding)f(issue:)75 2303 y(Nothing)g(in)g(MPI)h(requires)f(an)g(MPI_Datatype)g(to)g(be)h (either)f(a)g(pointer)g(or)h(an)g(integral)75 2360 y(data)f(item;)g (thus)h(using)f("void)g(*")h(may)f(be)h(inappropriate.)46 b(Perhaps)23 b(an)g(MPI_Handle_t)75 2416 y(basic)g(type)g(should)h(be)f (defined)g(that)g(all)h(MPI)f(opaque)g(objects)g(can)h(be)f(cast)h(to.) 75 2529 y(Issue)75 2642 y(MPI_PACK)f(does)g(not)h(say)f(anything)g (about)g(how)g(the)h(data)f(is)h(represented.)46 b(For)75 2698 y(example,)23 b(an)g(implementation)f(may)i(put)f(a)h(header)f(on) h(the)f(data)g(\("I)h(am)f(packed)g(with)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fv(6)1156 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fr(XDR"\))23 b(or)h(\("Characters)e(are)h(in)h(Unicode"\).)46 b(An)24 b(implementation)e(may)h(include)75 102 y(additional)f (information)h(within)g(the)g(data)h(\(for)f(example,)g(the)g(type-map) g(may)g(be)75 158 y(included)g(to)g(give)h(better)f(error)g (detection\).)46 b(To)24 b(use)f(MPI_PACK)g(to)h(create)f(a)75 214 y(canonical)g(representation,)e(we)j(need)f(to)170 327 y(\(a\))h(restrict)f(the)g(formats)g(used)g(by)h(MPI_PACK)f(\(for)g (example,)g(expect)266 384 y(a)h(header)f(of)g(fixed)g (MPI_PACK_HEADER_SIZE)f(length\))170 440 y(\(b\))i(specify)f(a)h (canonical)e(output)h(format)g(as)h(part)f(of)h(the)f(communicator)266 497 y(\(either)g(by)g(a)h(user-setable)e(attribute)h(or)h(special)e (communicator,)266 553 y(including)h(a)g(null)h(communicator\))170 610 y(\(c\))g(specify)f(special)g(behavior)f(when)i(packing)f (MPI_DATATYPE)f(\(e.g.,)h(must)g(use)266 666 y(canonical)g (representation)e(with)j(no)f(header\))170 723 y(\(d\))h(use)f (MPI_PACK_DATATYPE)f(\(or)h(MPI_PACK_CANONICAL\))f(instead)75 835 y(On)i(the)f(MPI_UNPACK)g(side,)g(we)g(need)75 948 y(MPI_GET_COUNT_DATA\()e(void)j(*,)f(MPI_Status)g(*,)g(MPI_Datatype,)f (int)i(*)g(\))75 1061 y(\(in)f(case)h(you)f(sent)h(3)f(MPI_DATATYPES,)f (all)i(with)f(different)g(sizes)g(of)75 1118 y(canonical)g (representations\))75 1231 y(This)g(is)h(similar)f(to)g (MPI_PACK_SIZE_DATA)f(in)h(that)h(the)f(result)g(depends)g(on)h(the)75 1287 y(data)f(that)h(was)f(sent.)75 1465 y Fo(9.3.4)49 b(Convenient)17 b(F)o(o)o(rm)d(of)j(MPI)p 731 1465 15 2 v 18 w(T)l(yp)q(e)p 848 1465 V 18 w(indexed)75 1608 y Fr(Marydell)23 b(Tholburn:)46 b(We)24 b(would)f(like)g(this)h (proposal)e(discussed)75 1664 y(at)i(the)f(April)g(24th)h(meeting.)75 1777 y(This)f(proposal)g(is)h(to)f(allow)g(constant)g(blocksize)g(and)g (arbitrary)g(displacements)75 1833 y(in)h(the)f(interface)g(to)47 b(MPI_TYPE_INDEXED.)22 b(The)h(routine)g(now)g(takes)75 1890 y(array_of_blocklengths)e(and)i(array_of_displacements)e(as)j (arguments.)46 b(There)75 1946 y(are)23 b(many)h(codes)f(using)g (indirect)g(addressing)f(arising)h(from)h(unstructured)75 2003 y(grids)f(where)g(the)h(blocksize)e(is)i(always)f(1)h (\(gather/scatter\).)75 2059 y(We)g(propose)e(this)i(interface:)75 2116 y(MPI_TYPE_INDEXED\(COUNT,)d(BLOCKLENGTH,)h (ARRAY_OF_DISPLACEMENT,)266 2172 y(OLDTYPE,)h(NEWTYPE,)f(IERROR\))147 2229 y(INTEGER)g(COUNT,)i(INTEGER)e(BLOCKLENGTH,)h (ARRAY_OF_DISPLACEMENT\(*\))o(,)147 2285 y(OLDTYPE,)f(NEWTYPE,)h (IERROR\);)75 2341 y(allowing)g(for)g(constant)g(blocksize)g(and)g (arbitrary)g(displacements.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fp(9.4.)34 b(INTERLANGUA)o(GE)16 b(INTER)o (OPERABILITY)773 b Fv(7)75 45 y Fs(9.4)59 b(Interlanguage)18 b(Interop)r(erabilit)n(y)75 148 y Fo(9.4.1)49 b(Intro)q(duction)75 234 y Fv(It)17 b(is)h(not)e(uncommon)h(for)g(library)h(dev)o(elop)q (ers)g(to)e(use)i(one)f(language)g(to)f(dev)o(elop)i(an)f(applications) 75 290 y(library)g(to)f(b)q(e)h(used)g(b)o(y)g(applications)h(dev)o (elop)q(ers)g(using)f(another)f(language.)24 b(MPI)16 b(curren)o(tly)h(sup-)75 347 y(p)q(orts)d(b)q(oth)i(an)e(ANSI)i(C)f (binding)i(and)e(a)f(F)l(ortran)g(77)g(binding.)22 b(It)15 b(should)h(b)q(e)f(p)q(ossible)i(for)d(F)l(ortran)75 403 y(77)h(to)f(call)i(C)f(and)h(vice)g(v)o(ersa)f(that)f(use)i(MPI)f (in)h(b)q(oth)f(languages.)166 460 y(MPI)h(allo)o(ws)h(the)f(dev)o (elopmen)o(t)h(of)f(clien)o(t-serv)o(er)i(co)q(de,)f(with)g(MPI)f(comm) o(unication)h(used)g(b)q(e-)75 516 y(t)o(w)o(een)d(a)h(parallel)h (clien)o(t)g(and)f(a)g(parallel)h(serv)o(er.)k(It)14 b(should)i(b)q(e)g(p)q(ossible)g(to)e(co)q(de)i(the)f(serv)o(er)f(in)i (one)75 573 y(language,)e(and)g(the)f(clien)o(ts)i(in)g(another)e (language.)19 b(T)l(o)14 b(do)f(so,)g(it)h(should)h(b)q(e)f(p)q (ossible)i(for)d(F)l(ortran77)75 629 y(pro)q(cesses)j(to)e(comm)o (unicate)i(with)f(C)g(pro)q(cesses.)166 686 y(There)g(are)g(sev)o(eral) h(issues)g(that)e(need)i(to)f(b)q(e)h(addressed,)f(to)f(ac)o(hiev)o(e)i (in)o(terop)q(erabilit)o(y)l(.)75 779 y Fm(Initializat)q(ion)26 b Fv(W)l(e)13 b(need)h(to)f(sp)q(ecify)h(ho)o(w)f(is)g(the)g(MPI)h(en)o (vironmen)o(t)f(initialized)j(for)d(b)q(oth)g(F)l(ortran)189 836 y(and)i(C.)75 930 y Fm(Comm)o(unication)j(of)g(MPI)e(opaque)i(ob)s (jects)23 b Fv(W)l(e)13 b(need)h(to)f(sp)q(ecify)i(ho)o(w)e(are)g(MPI)g (ob)s(ject)g(han-)189 986 y(dles)i(passed)g(from)e(F)l(ortran)h(to)f (C,)h(and)h(vice)g(v)o(ersa.)k(W)l(e)c(also)f(need)h(to)f(sp)q(ecify)i (what)e(happ)q(ens)189 1043 y(when)g(an)g(MPI)g(ob)s(ject)g(is)g (accessed)h(in)g(one)f(language,)g(to)g(retriev)o(e)g(information)g (set)g(with)h(this)189 1099 y(ob)s(ject)f(in)i(another)f(language.)75 1193 y Fm(In)o(terlanguage)j(comm)o(unication)24 b Fv(W)l(e)18 b(need)g(to)e(sp)q(ecify)i(ho)o(w)f(messages)g(sen)o(t)g(in)g(one)h (language)189 1249 y(can)d(b)q(e)h(receiv)o(ed)g(in)g(another)f (language.)166 1343 y(It)f(is)h(highly)h(desirable)g(that)d(the)i (solution)g(for)f(in)o(terlanguage)h(in)o(terop)q(erabilit)o(y)h(b)q(e) f(extendable)75 1400 y(to)g(new)g(languages,)g(should)h(MPI)f(bindings) i(b)q(e)f(de\014ned)h(for)d(suc)o(h)i(languages.)75 1521 y Fo(9.4.2)49 b(Assumptions)75 1607 y Fv(W)l(e)21 b(assume)f(that)g (con)o(v)o(en)o(tions)g(exist)h(for)f(C)h(programs)e(to)h(call)i(F)l (ortran)d(subroutines,)j(and)f(vice)75 1664 y(v)o(ersa.)e(These)c(con)o (v)o(en)o(tions)f(sp)q(ecify)i(ho)o(w)e(to)g(link)h(C)g(and)f(F)l (ortran)f(co)q(de;)i(ho)o(w)f(to)g(call)h(from)f(F)l(ortran)75 1720 y(a)j(C)g(function,)h(so)f(that)g(the)g(correct)g(C)g(calling)i (in)o(terface)f(b)q(e)g(generated,)g(and)f(vice)h(v)o(ersa;)g(ho)o(w)f (to)75 1777 y(pass)i(from)g(F)l(ortran)g(parameters)g(b)o(y)g(v)m(alue) i(to)e(C)h(functions;)i(and)e(the)g(corresp)q(ondence)h(b)q(et)o(w)o (een)75 1833 y(F)l(ortran)g(basic)h(data)f(t)o(yp)q(es)h(and)g(C)g (basic)g(datat)o(yp)q(es.)39 b(In)22 b(general,)i(these)e(con)o(v)o(en) o(tions)g(will)h(b)q(e)75 1889 y(implemen)o(tation)16 b(dep)q(enden)o(t.)22 b(\(Ho)o(w)o(ev)o(er,)13 b(there)i(is)g(an)g (ongoing)g(e\013ort)f(to)g(standardize)i(a)e(C)h(calling)75 1946 y(in)o(terface)k(in)h(High)f(P)o(erformance)g(F)l(ortran.\))29 b(F)l(urthermore,)19 b(not)f(ev)o(ery)h(F)l(ortran)f(t)o(yp)q(e)h(need) g(ha)o(v)o(e)75 2002 y(a)g(matc)o(hing)g(C)f(t)o(yp)q(e,)i(and)f(vice)h (v)o(ersa:)26 b(often)o(times,)20 b(for)e(example,)i(C)f(c)o(haracter)f (strings)h(are)f(not)75 2059 y(compatible)k(with)g(F)l(ortran)e(CHARA)o (CTER)i(v)m(ariables.)39 b(Ho)o(w)o(ev)o(er,)21 b(w)o(e)g(assume)h (that)e(a)h(F)l(ortran)75 2115 y Fk(INTEGER)c Fv(can)h(b)q(e)g(passed)g (to)f(a)g(C)h(program.)26 b(Also,)18 b(w)o(e)f(assume)h(that)f(b)q(oth) h(F)l(ortran)e(and)i(C)f(ha)o(v)o(e)75 2172 y(address)e(sized)i(in)o (tegers.)75 2293 y Fo(9.4.3)49 b(Initialization)75 2379 y Fv(A)18 b(call)i(to)e Fr(MPI)p 348 2379 15 2 v 16 w(INIT\(\))p Fv(,)g(either)h(from)f(C)g(or)g(F)l(ortran,)f(initialize)q(s)k(MPI)d (for)g(execution,)i(b)q(oth)e(in)h(C)75 2436 y(and)c(F)l(ortran.)166 2575 y Fl(Discussion:)36 b Fn(Certain)14 b(implemen)o(tations)e(use)j (the)g(\(inout\))f(argc,)g(argv)g(argumen)o(ts)g(of)g(the)h(C)f(v)o (ersion)75 2631 y(of)i Fk(MPI)p 202 2631 13 2 v 14 w(INIT)g Fn(in)f(order)i(to)f(propagate)g(the)h(righ)o(t)f(v)n(alue)f(for)h (argc,)g(argv)g(to)g(all)f(executing)i(pro)q(cesses.)27 b(Users)75 2688 y(should)13 b(b)q(e)i(w)o(arned)f(that)g(use)g(of)f (the)h(F)m(ortran)g(v)o(ersion)g(of)f Fk(MPI)p 1102 2688 V 14 w(INIT)h Fn(to)f(initialize)f(MPI)i(ma)o(y)e(result)i(in)g(a)f (loss)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fv(8)1161 b Fp(CHAPTER)15 b(9.)30 b(MISCELLANY)75 45 y Fn(of)16 b(this)g(abilit)o(y)m(.)22 b(This)16 b(should)g(b)q(e)h (acceptable:)24 b(if)15 b(C)h(execution)h(is)f(not)g(started)h(from)d (main,)g(but)j(is)f(started)75 102 y(from)d(the)j(in)o(v)o(o)q(cation)e (of)h(a)g(\(library\))f(function)h(call,)g(then)g(one)h(do)q(es)g(not)f (necessarily)h(exp)q(ect)h(argc,)e(argv)g(to)75 158 y(b)q(e)g (de\014ned.)166 297 y Fv(The)21 b(function)g Fr(MPI)p 523 297 15 2 v 17 w(INITIALIZED)e Fv(returns)h(the)h(same)f(answ)o(er,) h(whether)f(called)i(from)e(C)g(or)75 354 y(F)l(ortran.)166 410 y(The)15 b(function)h Fr(MPI)p 512 410 V 17 w(FINALIZE)e Fv(\014nalizes)j(b)q(oth)e(C)g(and)h(F)l(ortran)e(MPI)h(en)o(vironmen)o (t.)166 467 y(The)i(function)g Fr(MPI)p 515 467 V 17 w(ABORT)e Fv(kill)k(pro)q(cesses,)e(irresp)q(ectiv)o(e)h(of)e(the)g (language)h(used)g(b)o(y)f(the)h(caller)75 523 y(or)e(b)o(y)g(the)g (pro)q(cesses)h(killed.)166 579 y(When)22 b Fq(MPI)p 390 579 14 2 v 16 w(INIT)g Fv(is)g(called,)k(either)c(from)g(F)l (ortran)f(or)h(C,)f(then)i(the)f(MPI)g(en)o(vironmen)o(t)g(is)75 636 y(initialized)17 b(in)e(the)f(same)f(manner,)h(b)q(oth)g(for)f(F)l (ortran)g(and)h(C.)g(I.e.,)f Fk(MPI)p 1350 636 13 2 v 14 w(COMM)p 1494 636 V 15 w(W)o(ORLD)h Fv(carries)g(the)75 692 y(same)i(information,)g(b)q(oth)g(in)h(F)l(ortran)e(and)i(C:)e (same)h(pro)q(cesses,)h(same)f(en)o(vironmen)o(tal)h(attributes,)75 749 y(same)e(error)f(handlers.)75 871 y Fo(9.4.4)49 b(T)l(ransfer)15 b(of)i(handles)75 956 y Fv(The)g(data)g(t)o(yp)q(e)g Fk(MPI)p 457 956 V 14 w(Fint)g Fv(is)h(pro)o(vided)f(in)h(C)f(for)g(an) g(in)o(teger)g(of)f(the)i(size)f(that)g(matc)o(hes)f(a)h(F)l(ortran)75 1013 y Fk(INTEGER)p Fv(;)d(usually)l(,)i Fk(MPI)p 517 1013 V 15 w(Fint)f Fv(will)i(b)q(e)f(equiv)m(alen)o(t)h(to)d Fk(int)p Fv(.)166 1069 y(The)f(follo)o(wing)h(t)o(w)o(o)e(functions)i (are)f(pro)o(vided)h(in)g(C)f(to)g(con)o(v)o(ert)f(from)h(a)g(F)l (ortran)f(handle)i(\(whic)o(h)75 1126 y(is)i(an)f(in)o(teger\))g(to)f (a)h(C)g(handle,)h(and)g(vice)g(v)o(ersa.)75 1182 y Fr(void)23 b(*MPI)p 293 1182 15 2 v 17 w(Int2handle\()f(MPI)p 668 1182 V 17 w(Fint)h(*f)p 852 1182 V 17 w(handle\))75 1269 y(MPI)p 150 1269 V 17 w(Fint)g(MPI)p 358 1269 V 17 w(Handle2int\()f (void)i(*c)p 829 1269 V 16 w(handle\))166 1355 y Fv(If)e Fq(f)p 235 1355 14 2 v 17 w(handle)h Fv(is)g(a)f(v)m(alid)i(F)l(ortran) d(handle)j(to)d(an)h(opaque)h(ob)s(ject,)g(then)f Fq(MPI)p 1584 1355 V 16 w(INT2HANDLE)75 1412 y Fv(returns)e(a)g(v)m(alid)i(C)e (handle)i(to)e(that)f(same)h(ob)s(ject;)i(if)f Fq(f)p 1105 1412 V 17 w(handle)g Fv(is)g(a)f(n)o(ull)i(F)l(ortran)d(handle,)k (then)75 1468 y Fq(MPI)p 160 1468 V 16 w(INT2HANDLE)18 b Fv(returns)g(a)f(n)o(ull)j(C)e(handle;)i(if)f Fq(f)p 1038 1468 V 16 w(handle)g Fv(is)g(an)f(in)o(v)m(alid)i(F)l(ortran)d (handle,)j(then)75 1524 y Fq(MPI)p 160 1524 V 16 w(INT2HANDLE)15 b Fv(returns)g(an)g(in)o(v)m(alid)i(C)e(handle;)h(and)g(similarly)l(,)h (for)d Fq(MPI)p 1477 1524 V 16 w(HANDLE2INT)p Fv(.)75 1631 y Fm(Example)k(9.2)k Fv(The)11 b(example)g(b)q(elo)o(w)f (illustrates)i(ho)o(w)d(the)i(F)l(ortran)d(MPI)j(function)g Fq(MPI)p 1651 1631 V 15 w(TYPE)p 1783 1631 V 17 w(COMMIT)75 1687 y Fv(can)k(b)q(e)h(implemen)o(ted)h(b)o(y)d(wrapping)i(the)f(C)g (MPI)g(function)h Fq(MPI)p 1244 1687 V 15 w(T)l(yp)q(e)p 1351 1687 V 18 w(commit)d Fv(with)i(a)g(C)g(wrapp)q(er)75 1744 y(to)g(do)g(handle)h(con)o(v)o(ersions.)75 1850 y Fr(!)24 b(FORTRAN)f(PROCEDURE)75 1906 y(SUBROUTINE)f (MPI_TYPE_COMMIT\()g(DATATYPE,)h(IERR\))75 1963 y(INTEGER)g(DATATYPE,)g (IERR)75 2019 y(CALL)g(MPI_X_TYPE_COMMIT\(DATATYPE,)d(IERR\))75 2076 y(RETURN)75 2132 y(END)75 2245 y(!)k(C)f(wrapper)75 2358 y(void)g(MPI_X_TYPE_COMMIT\()f(MPI_Fint)g(*f_handle,)h(*ierr\))75 2414 y({)75 2471 y(MPI_Datatype)f(datatype;)75 2584 y(datatype)h(=)g (\(MPI_Datatype\)MPI_int2handle\()d(*f_handle\);)75 2640 y(*ierr)j(=)h(\(MPI_Fint\)MPI_Type_commit)o(\()d(&datatype\);)75 2697 y(*f_handle)i(=)g(MPI_handle2int\()f(datatype\);)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fp(9.4.)34 b(INTERLANGUA)o(GE)16 b(INTER)o (OPERABILITY)773 b Fv(9)75 45 y Fr(return\(\);)75 102 y(})166 203 y Fv(The)10 b(same)g(approac)o(h)g(can)g(b)q(e)h(used)g (for)e(all)i(other)f(MPI)g(functions.)19 b(The)11 b(call)g(to)e Fq(MPI)p 1637 203 14 2 v 16 w(INT2HANDLE)75 260 y Fv(\(resp.)16 b Fq(MPI)p 288 260 V 16 w(HANDLE2INT)p Fv(\))g(can)g(b)q(e)h(omitted)g (when)g(the)f(handle)i(is)f(an)f Fn(OUT)g(\(resp.)f(IN\))h(argumen)o (t,)75 316 y(rather)f(than)f(INOUT.)166 500 y Fl(Discussion:)46 b Fn(The)17 b(design)g(assumes)g(that)g(C)g(handles)g(can)g(b)q(e)h (con)o(v)o(erted)g(to)e(t)o(yp)q(e)i Fk(void*)p Fn(,)f(and)g(vice)75 557 y(v)o(ersa,)e(without)g(loss.)22 b(This)15 b(is)g(a)g(sligh)o(t)f (restriction)i(on)f(MPI1)g(whic)o(h)g(only)f(requires)i(that)g(handles) f(b)q(e)h(of)e(an)75 613 y(in)o(teger)g(or)g(p)q(oin)o(ter)g(t)o(yp)q (e.)189 797 y Fi(R)n(ationale.)62 b Fv(The)19 b(design)h(here)g(pro)o (vides)f(a)g(con)o(v)o(enien)o(t)g(solution)h(for)e(the)h(prev)m(alen)o (t)h(case,)189 854 y(where)f(a)f(C)h(wrapp)q(er)g(is)h(used)f(to)f (allo)o(w)i(F)l(ortran)d(co)q(de)j(to)e(call)i(a)f(C)f(library)l(,)j (or)d(C)h(co)q(de)h(to)189 910 y(call)g(a)f(F)l(ortran)f(library)l(.)32 b(The)19 b(use)h(of)f(C)f(wrapp)q(ers)i(is)f(m)o(uc)o(h)g(more)g(lik)o (ely)i(than)e(the)g(use)g(of)189 967 y(F)l(ortran)12 b(wrapp)q(ers,)i(b)q(ecause)h(it)f(is)g(m)o(uc)o(h)g(more)f(lik)o(ely)j (that)d(a)g(v)m(ariable)j(of)d(t)o(yp)q(e)h Fk(INTEGER)f Fv(can)189 1023 y(b)q(e)j(passed)f(to)g(C,)f(than)h(a)g(C)g(handle)h (can)g(b)q(e)g(passed)f(to)g(F)l(ortran.)189 1098 y(Assuming)e(that)g (the)g(F)l(ortran)f(handle)i(is)g(used)f(as)g(the)g(language)g(indep)q (enden)o(t)j(represen)o(tation)189 1154 y(for)21 b(handles,)j(then)e (this)g(design)g(can)g(b)q(e)g(extended)h(to)e(allo)o(w)h(wrapp)q(ers)f (written)h(in)g(other)189 1211 y(languages)15 b(b)o(y)g(pro)o(viding)h (con)o(v)o(ersion)g(functions)g(in)g(these)f(languages.)189 1285 y(The)21 b(use)h(of)f(con)o(v)o(ersion)g(functions,)j(rather)c (than)i(pro)q(cedures,)h(allo)o(w)f(us)f(to)g(use)g(inlinin)q(g,)189 1341 y(esp)q(ecially)c(in)f(the)f(case)g(where)g(these)g(functions)h (are)e(the)h(iden)o(tit)o(y)l(.)21 b(The)15 b(con)o(v)o(ersion)g (function)189 1398 y(in)f(the)g(wrapp)q(er)g(do)q(es)g(not)f(catc)o(h)h (an)f(in)o(v)m(alid)j(handle)f(argumen)o(t.)k(Instead,)14 b(an)g(in)o(v)m(alid)i(handle)189 1454 y(is)11 b(passed)h(b)q(elo)o(w)g (to)f(the)g(library)h(function,)h(whic)o(h,)f(presumably)l(,)h(c)o(hec) o(ks)e(its)h(input)g(argumen)o(ts.)189 1511 y(\()p Fi(End)j(of)i(r)n (ationale.)p Fv(\))189 1612 y Fi(A)n(dvic)n(e)c(to)h(users.)38 b Fv(The)13 b(user)g(needs)g(to)g(co)q(erce)g(the)g(result)g(returned)g (b)o(y)f Fq(MPI)p 1560 1612 V 16 w(int2handle)j Fv(from)189 1669 y Fk(void*)g Fv(to)f(a)h(suitable)i(handle)f(t)o(yp)q(e.)k(\()p Fi(End)c(of)g(advic)n(e)g(to)h(users.)p Fv(\))75 1790 y Fo(9.4.5)49 b(MPI)17 b(Opaque)f(objects)75 1876 y Fv(In)11 b(general,)g(opaque)g(ob)s(jects)e(are)h(\\the)g(same")g(in)h(F)l (ortran)e(and)h(C:)g(they)g(carry)g(the)g(same)g(information,)75 1932 y(and)17 b(ha)o(v)o(e)f(the)h(same)f(meaning)h(in)h(b)q(oth)f (languages.)24 b(The)17 b(mec)o(hanism)g(describ)q(ed)i(in)e(the)g (previous)75 1989 y(section)c(can)g(b)q(e)g(used)h(to)e(pass)g (references)i(to)e(MPI)g(ob)s(jects)g(from)g(language)h(to)f(language.) 20 b(An)13 b(ob)s(ject)75 2045 y(created)i(in)h(one)g(language)f(can)g (b)q(e)h(accessed,)g(mo)q(di\014ed)g(or)f(freed)g(in)h(another)f (language.)166 2102 y(W)l(e)g(examine)h(b)q(elo)o(w)g(in)g(more)f (detail,)h(issues)g(that)e(arise)i(for)e(eac)o(h)h(t)o(yp)q(e)h(of)f (MPI)g(ob)s(ject.)75 2223 y Fo(9.4.6)49 b(Datat)o(yp)q(es)75 2308 y Fv(Datat)o(yp)q(es)19 b(enco)q(des)h(the)g(same)g(information)f (in)i(F)l(ortran)e(and)h(in)g(C.)f(E.g.,)h(a)f(datat)o(yp)q(e)g (accessor)75 2365 y(lik)o(e)i Fq(MPI)p 248 2365 V 15 w(TYPE)p 380 2365 V 17 w(EXTENT)f Fv(will)i(return)d(the)h(same)f (information,)h(b)q(oth)g(in)g(F)l(ortran)f(and)g(in)i(C.)e(If)75 2421 y(a)g(datat)o(yp)q(e)f(de\014ned)j(in)f(F)l(ortran)e(is)h(used)h (in)g(a)f(C)f(comm)o(unication)i(call,)h(then)e(the)h(message)e(sen)o (t)75 2478 y(will)e(b)q(e)f(iden)o(tical)h(to)e(the)g(message)g(that)g (w)o(ould)h(b)q(e)g(sen)o(t)f(from)f(F)l(ortran:)19 b(the)14 b(same)g(comm)o(unication)75 2534 y(bu\013er)h(is)h(accessed,)f(and)h (the)f(same)g(represen)o(tation)g(con)o(v)o(ersion)g(is)h(p)q (erformed,)f(if)h(needed.)21 b(Both)15 b(C)75 2591 y(and)h(F)l(ortran)e (basic)i(datat)o(yp)q(es)f(can)g(b)q(e)h(used)g(in)h(datat)o(yp)q(e)d (constructors,)h(in)h(either)g(language.)21 b(If)15 b(a)75 2647 y(datat)o(yp)q(e)h(is)i(committed)e(in)i(either)g(language,)f (then)g(it)g(can)g(b)q(e)g(used)h(for)e(comm)o(unication)h(in)h(b)q (oth)75 2704 y(languages.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fv(10)1133 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)166 45 y Fv(The)14 b(function)g Fq(MPI)p 519 45 14 2 v 16 w(ADDRESS)g Fv(returns)f(the)h(same)f(v)m(alue,)i(b)q (oth)e(in)i(F)l(ortran)d(and)i(in)g(C.)f(On)h(the)75 102 y(other)h(hand,)h(w)o(e)g(do)g(not)f(require)i(that)e(the)h (constan)o(t)f Fk(MPI)p 1127 102 13 2 v 14 w(BOTTOM)g Fv(has)h(the)g(same)f(v)m(alue)i(in)g(b)q(oth)75 158 y(languages.)75 264 y Fm(Example)h(9.3)75 358 y Fr(!)24 b(FORTRAN)f(CODE)75 415 y(REAL)g(R\(5\))75 471 y(INTEGER)g(TYPE,)g (ADDR,)g(IERR)75 584 y(!)h(create)f(an)g(absolute)g(datatype)g(for)g (array)h(R)75 640 y(CALL)f(MPI_ADDRESS\()g(R,)g(ADDR,)g(IERR\))75 697 y(CALL)g(MPI_TYPE_STRUCT\(1,)f(5,)h(ADDR,)g(MPI_REAL,)g(TYPE,)g (IERR\))75 753 y(CALL)g(C_ROUTINE\(TYPE\))75 866 y(/*)h(C)f(code)h(*/) 75 979 y(void)f(C_ROUTINE\(MPI_Fint)f(*ftype\))75 1036 y({)75 1092 y(int)h(count)h(=)f(5;)75 1148 y(void)g(*handle;)75 1205 y(int)g(lens[2])g(=)h({1,1};)75 1261 y(MPI_Aint)f(displs[2];)75 1318 y(MPI_Datatype)f(types[2],)h(newtype;)75 1431 y(/*)h(create)f(an)g (absolute)g(datatype)g(for)g(buffer)g(that)h(consists)70 b(*/)75 1487 y(/*)47 b(of)24 b(count,)f(followed)g(by)g(R\(5\))668 b(*/)75 1600 y(MPI_Addr\(&count,)22 b(displs[0]\);)75 1657 y(displs[1])h(=)g(0;)75 1713 y(types[0])g(=)g(MPI_INT;)75 1769 y(types[1])g(=)g(\(MPI_Datatype\)MPI_Int2handle\(f)o(type\);)75 1826 y(MPI_Type_struct\(2,)e(lens,)j(displs,)e(types,)i(&newtype\);)75 1882 y(MPI_Type_commit\(&newtype\);)75 1995 y(MPI_Send\(MPI_BOTTOM,)d (1,)j(newtype,)e(1,)i(0,)g(MPI_COMM_WORLD\);)75 2052 y(/*)g(the)f(message)g(sent)g(contains)g(an)h(int)f(count)g(of)h(5,)f (followed)47 b(*/)75 2108 y(/*)24 b(by)f(the)h(5)f(REAL)h(entries)f(of) g(the)h(Fortran)f(array)g(R.)238 b(*/)75 2165 y(})189 2271 y Fi(R)n(ationale.)58 b Fv(The)18 b(curren)o(t)g(MPI)g(standard)g (sp)q(eci\014es)i(that)d Fk(MPI)p 1363 2271 V 15 w(ADDRESS)g Fv(can)h(b)q(e)h(used)g(in)189 2327 y(initialization)i(expressions)f (in)g(C,)e(but)h(not)f(in)i(F)l(ortran.)29 b(Since)21 b(F)l(ortran)c(do)q(es)i(not)g(supp)q(ort)189 2384 y(normally)11 b(call)h(b)q(e)g(v)m(alue,)g(then)g Fk(MPI)p 815 2384 V 14 w(ADDRESS)e Fv(m)o(ust)g(b)q(e)i(in)g(F)l(ortran)d(the)i(name)g (of)g(a)f(prede\014ned)189 2440 y(static)19 b(v)m(ariable,)k(e.g.,)d(a) f(v)m(ariable)j(in)f(an)f(MPI)g(declared)h Fk(COMMON)f Fv(blo)q(c)o(k.)35 b(On)20 b(the)g(other)189 2497 y(hand,)e(in)h(C,)e (it)h(is)g(natural)f(to)g(tak)o(e)g Fk(MPI)p 930 2497 V 15 w(BOTTOM)f(=)g(0)h Fv(\(Ca)o(v)o(eat:)23 b(De\014ning)c Fk(MPI)p 1681 2497 V 14 w(BOTTOM)189 2553 y(=)e(0)h Fv(implies)i(that)e Fk(NULL)i Fv(p)q(oin)o(ter)e(cannot)h(b)q(e)g(distinguished)i(from)d Fk(MPI)p 1502 2553 V 14 w(BOTTOM)p Fv(;)g(ma)o(y)f(b)q(e)189 2610 y Fk(MPI)p 266 2610 V 14 w(BOTTOM)12 b(=)i(1)f Fv(is)i(b)q(etter)f Fh(:)8 b(:)g(:)e Fv(\))13 b(Requiring)j(that)e(the)g(F)l(ortran)f(and)h (C)g(v)m(alues)h(b)q(e)g(the)f(same)189 2666 y(will)i(complicate)h(the) e(initialization)j(pro)q(cess.)i(\()p Fi(End)c(of)g(r)n(ationale.)p Fv(\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fp(9.4.)34 b(INTERLANGUA)o(GE)16 b(INTER)o(OPERABILITY)751 b Fv(11)189 45 y Fi(A)n(dvic)n(e)14 b(to)h(implementors.)39 b Fv(The)14 b(follo)o(wing)g(implemen)o(tation) h(can)f(b)q(e)h(used:)k(MPI)14 b(addresses,)189 102 y(as)c(returned)h (b)o(y)g Fq(MPI)p 564 102 14 2 v 15 w(ADDRESS)p Fv(,)h(will)g(ha)o(v)o (e)e(the)h(same)f(v)m(alue,)j(b)q(oth)e(in)g(F)l(ortran)f(and)h(in)g (C.)f(One)189 158 y(ob)o(vious)17 b(c)o(hoice)h(is)f(that)f(MPI)h (addresses)g(b)q(e)h(iden)o(tical)h(to)d(regular)h(addresses.)26 b(The)17 b(address)189 214 y(is)i(stored)f(in)i(the)f(datat)o(yp)q(e,)f (when)i(datat)o(yp)q(es)e(with)h(absolute)g(addresses)g(are)f (constructed.)189 271 y(When)f(a)f(send)i(or)e(receiv)o(e)i(op)q (eration)f(is)g(p)q(erformed,)g(then)g(addresses)g(stored)f(in)i(a)e (datat)o(yp)q(e)189 327 y(are)e(in)o(terpreted)h(as)f(displacemen)o(ts) i(that)d(are)i(all)g(augmen)o(ted)f(b)o(y)h(a)f(base)g(address.)20 b(This)15 b(base)189 384 y(address)d(is)h(\(the)f(address)g(of)t(\))f Fq(buf)p Fv(,)j(or)d(zero,)i(if)g Fq(buf)g(=)f(MPI)p 1203 384 V 16 w(BOTTOM)p Fv(.)h(Th)o(us,)f(if)h Fk(MPI)p 1681 384 13 2 v 14 w(BOTTOM)189 440 y Fv(is)j(zero)g(then)h(a)f(send)g (or)g(receiv)o(e)h(call)g(with)g Fq(buf)g(=)f(MPI)p 1189 440 14 2 v 16 w(BOTTOM)h Fv(is)g(implemen)o(ted)h(exactly)189 497 y(as)12 b(a)h(call)i(with)e(a)g(regular)g(bu\013er)g(argumen)o(t:) 18 b(in)c(b)q(oth)g(cases)f(the)g(base)g(address)g(is)h Fq(buf)p Fv(.)20 b(On)14 b(the)189 553 y(other)j(hand,)h(if)f Fk(MPI)p 559 553 13 2 v 15 w(BOTTOM)g Fv(is)h(not)f(zero,)g(then)h(the) f(implemen)o(tation)i(has)e(to)g(b)q(e)h(sligh)o(tly)189 610 y(di\013eren)o(t.)h(A)12 b(test)g(is)h(p)q(erformed)g(to)f(c)o(hec) o(k)g(whether)h Fq(buf)g(=)g(MPI)p 1325 610 14 2 v 16 w(BOTTOM)p Fv(.)g(If)g(true,)f(then)h(the)189 666 y(base)i(address)g (is)h(zero,)f(otherwise)g(it)h(is)g Fq(buf)p Fv(.)21 b(In)15 b(particular,)h(if)g Fk(MPI)p 1393 666 13 2 v 14 w(BOTTOM)f Fv(do)q(es)g(not)g(ha)o(v)o(e)189 723 y(the)f(same)g(v)m (alue)i(in)f(F)l(ortran)f(and)g(C,)g(then)h(an)f(additional)i(test)e (for)g Fq(bu\013)h(=)f(MPI)p 1618 723 14 2 v 16 w(BOTTOM)h Fv(is)189 779 y(needed)h(in)g(at)f(least)g(one)g(of)g(the)g(t)o(w)o(o)f (languages.)189 854 y(It)h(ma)o(y)g(b)q(e)i(desirable)g(to)e(use)h(a)f (v)m(alue)i(other)e(than)h(zero)f(for)g Fk(MPI)p 1350 854 13 2 v 14 w(BOTTOM)h Fv(ev)o(en)g(in)g(C,)f(so)g(as)189 911 y(to)g(distinguish)k(it)e(from)f(a)g(NULL)h(p)q(oin)o(ter.)24 b(If)17 b Fk(MPI)p 1115 911 V 14 w(BOTTOM)e(=)g(c)i Fv(then)g(one)g (can)f(still)i(a)o(v)o(oid)189 967 y(the)h(test)g Fq(bu\013)h(=)g(MPI)p 597 967 14 2 v 16 w(BOTTOM)p Fv(,)f(b)o(y)h(using)g(the)g(displacemen)o (t)h(from)d Fk(MPI)p 1578 967 13 2 v 15 w(BOTTOM)p Fv(,)g(i.e.,)189 1024 y(the)c(regular)g(address)g(-)h(c,)f(as)g(the)g(MPI)g(address)g (returned)h(b)o(y)f Fq(MPI)p 1384 1024 14 2 v 16 w(ADDRESS)h Fv(and)f(stored)g(in)189 1080 y(absolute)h(datat)o(yp)q(es.)k(\()p Fi(End)d(of)g(advic)n(e)h(to)f(implementors.)p Fv(\))75 1200 y Fq(Addresses)75 1286 y Fv(Some)f(of)f(the)h(datat)o(yp)q(e)f (accessors)g(and)h(constructors)f(ha)o(v)o(e)g(argumen)o(ts)g(of)g(t)o (yp)q(e)h Fk(MPI)p 1608 1286 13 2 v 15 w(Aint)p Fv(,)f(in)h(C,)g(to)75 1342 y(hold)g(addresses.)20 b(The)15 b(corresp)q(onding)h(argumen)o (ts,)d(in)j(F)l(ortran,)d(ha)o(v)o(e)h(t)o(yp)q(e)h Fk(INTEGER)p Fv(.)f(This)h(causes)75 1399 y(F)l(ortran)j(and)h(C)g(to)f(b)q(e)h (incompatible,)j(in)e(an)f(en)o(vironmen)o(t)g(where)g(addresses)g(ha)o (v)o(e)g(64)f(bits,)i(but)75 1455 y(F)l(ortran)14 b Fk(INTEGER)p Fv(s)g(ha)o(v)o(e)h(32)g(bits.)166 1512 y(This)k(is)h(a)e(problem,)i (irresp)q(ectiv)o(e)h(of)d(in)o(terlanguage)h(issues.)32 b(Supp)q(ose)20 b(that)e(a)g(F)l(ortran)g(pro-)75 1568 y(cess)h(has)g(an)g(address)g(space)g(of)g Fg(\025)g Fv(4)g(GB.)g(What)f(should)i(b)q(e)g(the)f(v)m(alue)h(returned)f(in)h (F)l(ortran)e(b)o(y)75 1625 y Fq(MPI)p 160 1625 14 2 v 16 w(ADDRESS)p Fv(,)i(for)e(a)h(v)m(ariable)i(with)e(an)h(address)f (ab)q(o)o(v)o(e)g(2)1195 1608 y Ff(32)1232 1625 y Fv(?)33 b(The)19 b(prop)q(osed)h(design)g(aims)g(to)75 1681 y(solv)o(e)15 b(this)h(issue,)g(while)g(main)o(taining)h(compatibilit)o(y)f(with)g (curren)o(t)f(F)l(ortran)f(co)q(des.)166 1737 y(The)h(constan)o(t)g Fk(MPI)p 520 1737 13 2 v 14 w(ADDRESS)p 720 1737 V 14 w(SIZE)g Fv(is)h(de\014ned)g(so)f(that,)f(in)i(F)l(ortran)e(90,)75 1794 y Fk(INTEGER\(KIND=MPI)p 474 1794 V 14 w(ADDRESS)p 674 1794 V 14 w(SIZE\))c Fv(is)h(an)f(address)g(sized)h(in)o(teger)g(t) o(yp)q(e)f(\(t)o(ypically)l(,)i Fk(MPI)p 1687 1794 V 14 w(ADDRESS)p 1887 1794 V 15 w(SIZE)75 1850 y Fv(is)j(4)g(on)g(32)f (bit)h(address)g(mac)o(hines)h(and)f(8)g(on)f(64)h(bit)g(address)g(mac) o(hines\).)20 b(On)15 b(F)l(ortran)f(77)g(systems)75 1907 y(that)9 b(supp)q(ort,)i(instead,)h(declarations)f(of)e(the)i (form)e Fk(INTEGER*n)p Fv(,)h(then)h Fk(INTEGER*MPI)p 1591 1907 V 13 w(ADDRESS)p 1790 1907 V 14 w(SIZE)75 1963 y Fv(is)i(an)f(address)g(sized)i(in)o(teger)e(t)o(yp)q(e.)19 b(Similarly)l(,)c(the)d(constan)o(t)g Fk(MPI)p 1245 1963 V 14 w(INTEGER)p 1433 1963 V 14 w(SIZE)g Fv(is)h(de\014ned)h(so)e(that) 75 2020 y Fk(INTEGER\(KIND=MPI)p 474 2020 V 14 w(INTEGER)p 662 2020 V 14 w(SIZE\))i Fv(\(or)f Fk(INTEGER*MPI)p 1133 2020 V 13 w(INTEGER)p 1320 2020 V 15 w(SIZE)p Fv(\))g(is)i(a)f(default) g(size)h Fk(INTE-)75 2076 y(GER)p Fv(.)166 2126 y(There)g(are)g(sev)o (en)h(functions)g(that)e(ha)o(v)o(e)h(address)g(argumen)o(ts:)k Fq(MPI)p 1380 2126 14 2 v 16 w(TYPE)p 1513 2126 V 17 w(STRUCT,)75 2176 y(MPI)p 160 2176 V 16 w(TYPE)p 293 2176 V 17 w(HVECTOR,)d(MPI)p 632 2176 V 16 w(TYPE)p 765 2176 V 16 w(HINDEXED,)f(MPI)p 1118 2176 V 16 w(TYPE)p 1251 2176 V 17 w(EXTENT,)g(MPI)p 1556 2176 V 16 w(TYPE)p 1689 2176 V 17 w(LB,)75 2226 y(MPI)p 160 2226 V 16 w(TYPE)p 293 2226 V 17 w(UB)j Fv(and)g Fq(MPI)p 562 2226 V 16 w(ADDRESS)h Fv(\(the)f(address)g(is)h(an)f Fn(IN)e(argumen)o(t)g(in)g (the)h(\014rst)g(three)h(and)e(an)75 2275 y(OUT)d(argumen)o(t)f(in)h (the)g(last)g(four\).)k(These)e(functions)e(should)f(accept)j(address)f (argumen)o(ts)e(whic)o(h)h(are)g(either)75 2325 y(address)19 b(sized,)f(or)f(are)h(default)f(in)o(tegers.)29 b(The)17 b(en)o(vironmen)o(tal)f(v)n(ariable)g Fk(MPI)p 1393 2325 13 2 v 14 w(AINT)p 1504 2325 V 15 w(SIZE)h Fn(is)g(pro)o(vided)g(to)75 2375 y(indicate)i(the)g(curren)o(t)h(size)g(for)e(address)i(argumen)o (ts.)32 b(If)18 b Fk(MPI)p 1123 2375 V 14 w(AINT)p 1234 2375 V 15 w(SIZE)g(=)h(MPI)p 1478 2375 V 15 w(ADDRESS)p 1679 2375 V 14 w(SIZE)f Fn(then)75 2425 y(these)c(sev)o(en)f(functions) g(accept)h(\(or)e(return\))i(address)g(sized)f(in)o(teger)g(v)n(alues)f (\(32)h(bits)f(on)g(32)g(bit)h(mac)o(hines)e(and)75 2475 y(64)j(bits)h(on)g(64)g(bit)f(mac)o(hines\).)21 b(If)14 b Fk(MPI)p 728 2475 V 15 w(AINT)p 840 2475 V 15 w(SIZE)g(=)h(MPI)p 1076 2475 V 15 w(INTEGER)p 1265 2475 V 14 w(SIZE)g Fn(then)g(these)i (functions)e(accept)75 2525 y(or)f(return)h(address)g(v)n(alues)f(that) g(ha)o(v)o(e)f Fk(INTEGER)g Fn(t)o(yp)q(e.)166 2574 y Fk(MPI)p 243 2574 V 14 w(AINT)p 354 2574 V 15 w(SIZE)h Fn(has)g(initial)e(v)n(alue)h Fk(MPI)p 844 2574 V 14 w(INTEGER)p 1032 2574 V 14 w(SIZE)p Fn(.)h(Implemen)o(ters)e(ma)o(y)g (sp)q(ecify)j(a)e(mec)o(hanism)75 2624 y(for)j(c)o(hanging)f(the)i (initial)d(v)n(alue,)i(as)h(part)f(of)g(the)g(program)f(start-up)i(pro) q(cedure)h(\(e.g.,)e(with)g(argumen)o(ts)f(to)75 2674 y(the)g(link)o(er)g(or)g(to)f Fk(mpirun)p Fn(\).)22 b(The)15 b(v)n(ariable)f Fk(MPI)p 858 2674 V 14 w(AINT)p 969 2674 V 15 w(SIZE)g Fn(can)h(b)q(e)h(rep)q(eatedly)g(mo)q(di\014ed)d(during)i (program)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fv(12)1133 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fn(execution.)22 b(In)15 b(a)f(m)o(ultithreaded)g (en)o(vironmen)o(t,)g Fk(MPI)p 967 45 13 2 v 14 w(AINT)p 1078 45 V 15 w(SIZE)g Fn(should)h(not)g(b)q(e)g(mo)q(di\014ed)f(while)g (an)h(MPI)75 95 y(function)f(with)f(address)i(argumen)o(t)e(executes.) 166 227 y Fl(Discussion:)34 b Fn(W)m(e)14 b(ma)o(y)e(also)h(w)o(an)o(t) h(to)f(pro)o(vide)h(a)g(deriv)o(ed)g(datat)o(yp)q(e)g Fk(MPI)p 1410 227 V 15 w(AINT)f Fn(to)h(b)q(e)h(equiv)n(alen)o(t)e(to) 75 277 y Fk(INTEGER\(KIND)f(=)g(MPI)p 498 277 V 15 w(ADDRESS)p 699 277 V 14 w(SIZE\))p Fn(,)g(on)g(those)i(F)m(ortran)e(systems)h (that)f(supp)q(ort)i(deriv)o(ed)f(datat)o(yp)q(es.)189 493 y Fe(A)n(dvic)n(e)h(to)g(users.)36 b Fn(Curren)o(t)14 b(F)m(ortran)f(MPI)g(co)q(des)h(will)e(run)h(unmo)q(di\014ed,)f(with)h (the)g(curren)o(t)i(prop)q(osal.)189 543 y(New)j(co)q(des)h(can)f(b)q (e)g(written)g(so)g(that)g(they)g(alw)o(a)o(ys)f(use)h(address)h(sized) g(address)g(argumen)o(ts.)29 b(Suc)o(h)189 593 y(new)20 b(co)q(de)h(declares)h(address)f(v)n(ariables)f(to)g(b)q(e)g(of)g(t)o (yp)q(e)h Fk(INTEGER\(KIND=MPI)p 1551 593 V 13 w(ADDRESS)p 1750 593 V 15 w(SIZE\))p Fn(,)189 642 y(and)14 b(initializes)f Fk(MPI)p 532 642 V 14 w(AINT)p 643 642 V 15 w(SIZE)h(=)g(MPI)p 878 642 V 14 w(ADDRESS)p 1078 642 V 15 w(SIZE)g Fn(\(or)g(starts)h(up)f (the)h(computation)e(so)h(that)189 692 y(the)g(initial)e(v)n(alue)h(of) g Fk(MPI)p 611 692 V 15 w(AINT)p 723 692 V 15 w(SIZE)g Fn(is)h Fk(MPI)p 953 692 V 14 w(ADDRESS)p 1153 692 V 14 w(SIZE)p Fn(\).)g(\\Old")f(co)q(de)i(and)e(\\new")h(co)q(de)h(can) 189 742 y(b)q(e)i(mixed.)26 b(E.g.,)16 b(a)h(new)g(st)o(yle)g(library)f (can)h(co)q(exist)h(with)f(an)f(old)h(st)o(yle)g(user)h(co)q(de,)g(if)e (the)h(v)n(alue)g(of)189 792 y Fk(MPI)p 266 792 V 14 w(AINT)p 377 792 V 15 w(SIZE)e Fn(is)g(mo)q(di\014ed)f(up)q(on)h(en)o (try)h(to)f(the)h(library)e(and)h(restored)i(up)q(on)f(exit.)22 b(This)15 b(assumes)189 842 y(that)e(the)h(user)g(co)q(de)g(do)q(es)g (not)f(pass)h(addresses)h(\(or)e(datat)o(yp)q(es)h(con)o(taining)e (addresses\))k(to)d(the)h(library)m(.)189 891 y(In)g(the)g(later)g (case,)g(then)h(the)f(library)f(has)h(to)g(written)h(so)f(as)f(to)h (function)g(correctly)h(with)e(either)i(t)o(yp)q(es)189 941 y(of)e(addresses.)20 b(\()p Fe(End)c(of)f(advic)n(e)g(to)g(users.)p Fn(\))189 1024 y Fe(A)n(dvic)n(e)d(to)g(implementors.)34 b Fn(In)11 b(F)m(ortran)g(90,)g(o)o(v)o(erloading)e(can)j(b)q(e)f(used) h(so)g(as)f(to)g(pro)o(vide)g(di\013eren)o(t)h(v)o(er-)189 1074 y(sions)18 b(of)g(the)h(functions)f Fk(MPI)p 683 1074 V 15 w(TYPE)p 806 1074 V 14 w(STRUCT,)f(MPI)p 1086 1074 V 14 w(TYPE)p 1208 1074 V 15 w(HVECTOR,)f(MPI)p 1521 1074 V 15 w(TYPE)p 1644 1074 V 14 w(HINDEXED,)189 1124 y(MPI)p 266 1124 V 14 w(TYPE)p 388 1124 V 14 w(EXTENT,)11 b(MPI)p 663 1124 V 14 w(TYPE)p 785 1124 V 14 w(LB)p Fn(,)g(and)g Fk(MPI)p 1025 1124 V 14 w(TYPE)p 1147 1124 V 14 w(UB)p Fn(,)f(one)h(for)g(eac)o(h)g(address)i(size.)k(The)12 b(func-)189 1174 y(tion)h(v)o(ersions)h(can)g(c)o(hec)o(k)h(the)g (curren)o(t)g(v)n(alue)e(of)h Fk(MPI)p 1067 1174 V 14 w(AINT)p 1178 1174 V 15 w(SIZE)p Fn(,)f(to)h(catc)o(h)g(errors.)189 1235 y(This)e(do)q(es)i(not)e(w)o(ork)g(with)h(the)g(function)f Fk(MPI)p 949 1235 V 15 w(ADDRESS)p Fn(,)f(since)j(this)e(tak)o(es)h(a)g (c)o(hoice)g(argumen)o(t,)e(and)189 1285 y(an)k(in)o(terface)h(blo)q(c) o(k)g(cannot)g(b)q(e)g(pro)o(vided)f(for)h(it.)23 b(This)15 b(function)h(will)e(need)j(to)e(b)q(e)h(reco)q(ded)i(so)d(as)h(to)189 1335 y(branc)o(h)f(according)f(to)h(the)g(curren)o(t)h(v)n(alue)e(of)g Fk(MPI)p 1016 1335 V 15 w(AINT)p 1128 1335 V 15 w(SIZE)p Fn(.)f(\(This)i(function)f(returns)j(an)d(address,)189 1385 y(so)g(that)g(error)g(c)o(hec)o(king)h(is)e(not)h(an)g(issue.\)) 189 1446 y(If)e(o)o(v)o(erloading)f(is)h(not)g(a)o(v)n(ailable,)e(then) j(all)e(sev)o(en)j(functions)e(need)i(b)q(e)f(reco)q(ded)h(so)e(as)h (to)f(tak)o(e)g(in)o(to)g(con-)189 1496 y(sideration)f(the)i(curren)o (t)g(v)n(alue)e(of)g Fk(MPI)p 817 1496 V 14 w(AINT)p 928 1496 V 15 w(SIZE)g Fn(when)i(in)o(terpreting)f(an)f(address)i (input)f(parameter,)189 1545 y(or)h(deciding)h(whic)o(h)g(address)h(v)n (alue)e(to)h(return.)19 b(\()p Fe(End)d(of)f(advic)n(e)g(to)g (implementors.)p Fn(\))166 1711 y Fl(Alternativ)o(es)o(:)134 1784 y Fn(1.)22 b(A)17 b(w)o(eak)o(er)h(v)o(ersion)g(of)f(this)h(prop)q (osal)f(is)h(to)f(sp)q(ecify)h(that)g Fk(MPI)p 1242 1784 V 14 w(AINT)p 1353 1784 V 15 w(SIZE)f Fn(cannot)h(b)q(e)g(mo)q(difed)e (b)o(y)189 1834 y(the)f(user,)g(and)g(require)h(implem)o(en)o(tors)d (to)h(pro)o(vide)h(an)f(MPI)h(en)o(vironmen)o(t)f(where)i Fk(MPI)p 1634 1834 V 14 w(AINT)p 1745 1834 V 15 w(SIZE=)189 1884 y(MPI)p 266 1884 V 14 w(ADDRESS)p 466 1884 V 14 w(SIZE)d Fn(and)h(an)f(en)o(vironmen)o(t)f(where)j Fk(MPI)p 1149 1884 V 14 w(AINT)p 1260 1884 V 15 w(SIZE)e(=)g(MPI)p 1493 1884 V 15 w(INTEGER)p 1682 1884 V 14 w(SIZE)p Fn(.)g(The)189 1933 y(selection)h(of)e(the)i(en)o(vironmen)o(t)e(can)h(b)q(e)h(done)g (at)f(link)f(time)g(or)h(with)g(argumen)o(ts)f(to)h Fk(mpirun)p Fn(.)18 b(Old)13 b(co)q(de)189 1983 y(can)j(b)q(e)h(run)g(with)f(the)h (former)e(en)o(vironmen)o(t.)25 b(New)17 b(co)q(de,)g(compatible)e (with)h(C,)g(can)h(b)q(e)g(run)f(in)g(the)189 2033 y(later.)29 b(Ho)o(w)o(ev)o(er,)19 b(w)o(e)f(lo)q(ose)g(the)g(abilit)o(y)e(to)h (mix)f(old)h(and)h(new)g(co)q(des:)27 b(If)17 b(a)h(library)f(that)g (calls)h(one)189 2083 y(of)e(the)h(sev)o(en)h(MPI)f(functions)g(with)g (address)h(argumen)o(ts)e(has)h(to)g(run)g(in)g(b)q(oth)g(en)o (vironmen)o(ts,)f(then)189 2133 y(w)o(e)f(need)h(either)g(t)o(w)o(o)e (di\013eren)o(t)i(v)o(ersions)f(of)g(the)g(library)m(,)f(or)h(a)g (library)f(that)h(can)g(accommo)q(date)e(b)q(oth)189 2183 y(en)o(vironmen)o(ts.)j(\(If)10 b(w)o(e)h(go)f(this)g(w)o(a)o(y)m (,)g(should)g(w)o(e)h(also)f(assume)g(that)g(there)i(are)f(t)o(w)o(o)f (di\013eren)o(t)h Fk(mpi.h)f Fn(\014les,)189 2232 y(with)j(t)o(w)o(o)g (di\013eren)o(t)h(de\014nitions)f(of)g Fk(MPI)p 848 2232 V 14 w(AINT)p 959 2232 V 15 w(SIZE)p Fn(?)g(I.e.,)f(do)i(w)o(e)f(supp)q (ort)h(declarations)g(of)f(the)h(form)189 2282 y Fk(INTEGER\(KIND=MPI)p 588 2282 V 13 w(AINT)p 698 2282 V 15 w(SIZE\))p Fn(?)j(This)h(will)e (allo)o(w)h(to)g(write)h(source)i(co)q(de)e(that)g(is)g(compatible)189 2332 y(with)13 b(either)i(en)o(vironmen)o(ts;)e(but)h(this)g(co)q(de)g (will)f(need)i(to)e(b)q(e)i(compiled)d(with)i(the)g(righ)o(t)g Fk(mpi.h)f Fn(\014le.\))134 2405 y(2.)22 b(An)14 b(alternativ)o(e)h(is) g(to)f(c)o(hange)h(the)g(F)m(ortran)g(90)f(binding)g(so)h(that)g (functions)f(that)h(ha)o(v)o(e)g(argumen)o(ts)f(of)189 2455 y(t)o(yp)q(e)k Fk(MPI)p 363 2455 V 15 w(Aint)g Fn(will)e(accept)j (an)f(additional)e(optional)h(parameter,)h Fk(aint)p 1394 2455 V 16 w(size)p Fn(,)g(with)g(a)g(default)g(v)n(alue)189 2504 y(whic)o(h)d(is)h(the)h(curren)o(t)g(size)f(of)g(an)f Fk(INTEGER)p Fn(.)g(Th)o(us,)h(curren)o(t)h(MPI)f(F)m(ortran)g(co)q(de) h(will)d(run)i(without)189 2554 y(c)o(hanges;)h(co)q(de)g(that)f(w)o (an)o(ts)g(to)g(use)h(an)e(address)j(size)e(that)g(is)g(di\013eren)o(t) h(from)d(the)j(default)f Fk(INTEGER)189 2604 y Fn(will)f(need)k(to)e (pro)o(vide)g(the)g(additional)f(parameter.)27 b(Note)18 b(that)f(the)h(same)e(implemen)o(tation)e(can)j(b)q(e)189 2654 y(used)h(as)f(for)g(the)h(main)e(prop)q(osal.)28 b(The)18 b(di\013erence)h(is)e(whether)i Fk(aint)p 1339 2654 V 15 w(size)f Fn(is)f(a)g(global)f(v)n(ariable)g(or)h(a)189 2704 y(function)c(argumen)o(t.)k(This)d(design)g(is)g(cleaner,)g(but)g (cannot)g(b)q(e)h(supp)q(orted)g(in)e(F)m(ortran)h(77.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fp(9.4.)34 b(INTERLANGUA)o(GE)16 b(INTER)o(OPERABILITY)751 b Fv(13)75 45 y Fo(9.4.7)49 b(Group)75 122 y Fn(A)13 b(group)g(ob)r(ject)g(represen)o(t)j(the)d (same)f(group)h(in)f(b)q(oth)h(F)m(ortran)g(and)g(C:)f(the)i(same)e (pro)q(cesses,)j(with)d(the)i(same)75 172 y(ranks.)k(Group)c(accessors) i(suc)o(h)f(as)f Fk(MPI)p 731 172 13 2 v 14 w(GROUP)p 887 172 V 13 w(SIZE)g Fn(return)h(the)f(same)f(v)n(alue.)75 288 y Fo(9.4.8)49 b(Communicato)o(rs)75 364 y Fn(A)14 b(comm)o(unicator)d(represen)o(ts)16 b(the)e(same)f(group)h(and)f(same) g(comm)o(unication)d(domain,)h(b)q(oth)j(in)g(F)m(ortran)f(and)75 414 y(C.)h(In)g(particular,)g(a)f(message)h(sen)o(t)h(from)e(F)m (ortran)h(using)g(a)g(comm)o(unicator)d(can)k(b)q(e)f(receiv)o(ed)i (from)d(C,)g(using)75 464 y(the)j(matc)o(hing)d(comm)o(unicator)g(on)i (another)g(pro)q(cess;)i(collectiv)o(e)f(comm)o(unicati)o(on)c(can)k (in)o(v)o(olv)o(e)e(b)q(oth)h(C)g(and)75 514 y(F)m(ortran)i(pro)q (cesses,)i(using)e(matc)o(hing)e(comm)o(unicators;)g(accessors)k(suc)o (h)f(as)f Fk(MPI)p 1429 514 V 14 w(COMM)p 1573 514 V 15 w(SIZE)f Fn(return)i(the)75 564 y(same)13 b(v)n(alue;)g(etc.)166 613 y(T)m(op)q(ology)f(information)f(asso)q(ciated)j(to)g(a)f(comm)o (unicator)e(in)j(C)f(can)h(b)q(e)h(accessed)h(in)d(F)m(ortran,)g(and)h (vice)75 663 y(v)o(ersa.)75 778 y Fq(A)o(ttributes)75 854 y Fn(A)o(ttribute)h(k)o(eys)f(allo)q(cated)f(in)h(F)m(ortran)g(can) g(b)q(e)g(used)h(or)f(freed)h(in)e(C,)h(and)f(vice)i(v)o(ersa.)j(A)o (ttributes)d(v)n(alues)f(set)75 904 y(in)e(F)m(ortran)g(can)g(b)q(e)h (accessed)h(in)e(C,)f(and)h(vice)h(v)o(ersa.)18 b(T)m(o)11 b(ac)o(hiev)o(e)h(this,)g(attribute)h(k)o(eys)g(will)d(b)q(e)j(allo)q (cated)f(in)f(a)75 954 y(range)j(that)f(is)h(v)n(alid)e(b)q(oth)i(for)f (C)g Fk(int)h Fn(t)o(yp)q(e)g(and)g(F)m(ortran)f Fk(INTEGER)g Fn(t)o(yp)q(e.)18 b(The)c(same)f(holds)g(true)i(for)e(system)75 1004 y(de\014ned)i(attribute)f(v)n(alues)g(\(suc)o(h)h(as)f Fk(MPI)p 755 1004 V 14 w(T)m(A)o(G)p 849 1004 V 14 w(UB,)f(MPI)p 1020 1004 V 14 w(IO)p Fn(,)f(etc.\))166 1054 y(A)o(ttribute)18 b(k)o(eys)f(declared)h(in)f(C)g(are)g(asso)q(ciated)h(with)f(\\C)f (attributes")i(and)f(attribute)h(k)o(eys)f(declared)75 1103 y(in)e(F)m(ortran)g(are)g(asso)q(ciated)h(with)f(\\F)m(ortran)g (attributes".)23 b(C)15 b(attributes)h(ha)o(v)o(e)f(C)g(cop)o(y)g(and)g (delete)i(callbac)o(k)75 1153 y(functions)g(\(the)g(functions)f(pro)o (vided)h(b)o(y)f(the)h Fk(MPI)p 913 1153 V 14 w(Keyval)p 1042 1153 V 15 w(create)h Fn(call\).)25 b(F)m(ortran)17 b(attributes)g(ha)o(v)o(e)f(F)m(ortran)75 1203 y(delete)j(and)f (callbac)o(k)g(functions.)30 b(When)19 b(a)f(comm)o(unicator)d(is)j (duplicated)g(then,)i(for)d(eac)o(h)i(attribute,)g(the)75 1253 y(corresp)q(onding)13 b(cop)o(y)g(function)f(is)h(called,)f(using) g(the)h(righ)o(t)f(C)g(or)h(F)m(ortran)f(calling)f(con)o(v)o(en)o (tion;)i(and)f(similarly)l(,)75 1303 y(for)i(the)g(delete)h(callbac)o (k)e(function.)189 1390 y Fe(A)n(dvic)n(e)h(to)g(implementors.)36 b Fn(This)13 b(requires)i(that)e(attributes)h(b)q(e)g(tagged)f(either)i (as)e(\\C")g(or)g(\\F)m(ortran",)189 1440 y(and)18 b(that)h(the)h (language)e(tag)g(b)q(e)i(c)o(hec)o(k)o(ed)g(in)e(order)i(to)e(use)i (the)g(righ)o(t)e(calling)f(con)o(v)o(en)o(tion)i(for)f(the)189 1490 y(callbac)o(k)13 b(function.)k(\()p Fe(End)f(of)f(advic)n(e)g(to)g (implementors.)p Fn(\))166 1577 y(C)k(attributes)i(are)e(of)g(t)o(yp)q (e)h Fk(void*)p Fn(,)g(whereas)h(F)m(ortran)e(attributes)h(are)g(of)f (t)o(yp)q(e)h Fk(INTEGER)p Fn(.)e(On)i(some)75 1627 y(systems,)h Fk(INTEGER)p Fn(s)e(will)f(ha)o(v)o(e)i(32)f(bits,)i(while)e(C)h(p)q (oin)o(ters)g(will)e(ha)o(v)o(e)i(64)f(bits.)36 b(This)19 b(is)h(a)f(problem)g(if)75 1676 y(comm)o(unicator)12 b(attributes)j(are)f(used)i(to)e(mo)o(v)o(e)f(information)e(from)i(a)h (F)m(ortran)g(caller)g(to)g(a)g(C)g(callee,)h(or)f(vice-)75 1726 y(v)o(ersa.)166 1776 y(W)m(e)k(prop)q(ose)h(to)g(use)g(the)g(en)o (vironmen)o(t)f(v)n(ariable)f Fk(MPI)p 1089 1776 V 14 w(AINT)p 1200 1776 V 15 w(SIZE)h Fn(to)h(indicate)f(the)h(curren)o(t)h (size)g(of)75 1826 y(attributes)14 b(in)f(F)m(ortran.)k(If)c(an)g (attribute)h(is)f(set)h(in)f(4)g(b)o(yte)h(mo)q(de)e(and)h(is)g (accessed)i(in)e(8)g(b)o(yte)h(mo)q(de,)d(then)j(the)75 1876 y(4)h(b)o(yte)g(v)n(alue)f(is)h(sign)g(extended)h(to)f(8)f(b)o (ytes.)22 b(If)15 b(an)g(attribute)g(is)g(set)h(in)e(8)h(b)o(yte)g(mo)q (de,)f(next)h(read)g(in)g(4)g(b)o(yte)75 1925 y(mo)q(de,)d(then)j(the)f (8)g(b)o(yte)g(v)n(alue)g(is)f(truncated)i(to)f(4)g(b)o(ytes.)166 2058 y Fl(Discussion:)34 b Fn(W)m(e)13 b(ma)o(y)f(w)o(an)o(t)h(to)h (raise)g(an)g(error)h(if)e(the)h(truncation)g(causes)h(loss)f(of)f (information)e(\(i.e.,)75 2108 y(if)i(the)i(64)e(bit)g(in)o(teger)i (cannot)f(b)q(e)g(con)o(v)o(erted)h(to)f(a)g(32)f(bit)h(in)o(teger.)189 2278 y Fe(A)n(dvic)n(e)g(to)g(implementors.)36 b Fn(Comm)n(unicator)11 b(structures)16 b(are)d(set)h(for)f(8)g(b)o(yte)h(attributes,)g(but)f (F)m(ortran)189 2327 y(access)e(functions)e(will)f(access)j(either)f(4) f(or)g(8)g(b)o(ytes,)i(according)e(to)g(the)h(curren)o(t)h(v)n(alue)d (of)h Fk(MPI)p 1666 2327 V 14 w(AINT)p 1777 2327 V 15 w(SIZE)p Fn(.)189 2390 y(Ov)o(erloading)16 b(can)i(b)q(e)h(used)f(in)f (F)m(ortran)h(90)f(to)h(supp)q(ort)g(the)g(t)o(w)o(o)f(v)o(ersions)i (of)e(the)h(attribute)g(access)189 2439 y(functions.)g(\()p Fe(End)d(of)g(advic)n(e)g(to)g(implementors.)p Fn(\))166 2609 y Fl(Alternativ)o(es)o(:)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fv(14)1133 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)134 45 y Fn(1.)22 b(No)13 b(c)o(hange.)19 b(On)14 b(a)g(64)f(bit)g(system,)h(the)g(attributes)h(will)d(b)q(e,)i (in)o(ternally)m(,)e(of)i(size)g(64)g(bits.)189 95 y Fk(MPI)p 266 95 13 2 v 14 w(A)m(TTRIBUTE)p 510 95 V 13 w(GET)p Fn(,)20 b(in)h(F)m(ortran,)h(will)e(truncate)i(the)f(64)g(bit)g (v)n(alue)f(to)h(a)f(32)h(bit)f(v)n(alue,)i(and)189 145 y Fk(MPI)p 266 145 V 14 w(A)m(TTRIBUTE)p 510 145 V 13 w(SET)14 b Fn(will)f(con)o(v)o(ert)h(the)h(32)e(bit)h(argumen)o(t)e(to) i(a)g(64)f(bit)h(in)o(teger.)134 219 y(2.)22 b(Add)14 b(an)f(optional)g Fk(aint)p 567 219 V 15 w(size)h Fn(argumen)o(t)f(to)h (attribute)g(access)i(functions)e(in)f(F)m(ortran)h(90.)166 294 y(The)k(problem)f(w)o(e)h(face)g(in)f(this)h(section)h(is)e(the)i (same)e(as)h(the)g(problem)f(w)o(e)h(faced)g(with)f(addresses)j(in)75 344 y(Section)14 b(9.4.6;)e(w)o(e)i(should)g(adopt)f(here)i(the)g(same) e(solution)g(that)h(w)o(e)g(adopt)g(for)f(addresses)j(there.)75 543 y Fo(9.4.9)49 b(Requests)75 619 y Fn(A)13 b(request)h(created)g(in) e(C)g(can)h(b)q(e)g(accessed)i(and)e(up)q(dated)g(in)f(F)m(ortran)g(b)o (y)h(w)o(ait)e(or)i(test)h(functions)e(and)h(can)f(b)q(e)75 669 y(freed)j(in)e(F)m(ortran;)g(and)h(vice)g(v)o(ersa.)75 785 y Fo(9.4.10)49 b(Erro)o(r)15 b(handlers)75 862 y Fn(Error)i(handlers)g(allo)q(cated)g(in)f(F)m(ortran)g(can)h(b)q(e)g (in)o(v)o(ok)o(ed)f(or)g(deallo)q(cated)h(from)e(C,)h(and)g(vice)h(v)o (ersa.)27 b(When)75 912 y(an)17 b(MPI)h(exception)g(o)q(ccurs,)h(then)g (the)f(last)f(error)h(handler)g(asso)q(ciated)g(with)f(the)h(relev)n (an)o(t)g(comm)o(unicator)75 962 y(is)c(in)o(v)o(ok)o(ed,)f(irresp)q (ectiv)o(e)j(of)e(the)g(language)g(en)o(vironmen)o(t)f(where)i(the)g (error)g(o)q(ccurred)h(and)e(the)h(en)o(vironmen)o(t)75 1012 y(where)g(the)g(handler)f(w)o(as)g(asso)q(ciated)g(with)g(the)g (comm)o(unicator.)i(The)e(handler)g(should)g(b)q(e)h(in)o(v)o(ok)o(ed)e (with)g(the)75 1061 y(righ)o(t)g(parameter)h(passing)g(con)o(v)o(en)o (tion.)189 1149 y Fe(A)n(dvic)n(e)c(to)h(implementors.)33 b Fn(error)11 b(handler)f(ob)r(jects)g(need)h(to)e(ha)o(v)o(e)h(a)f (language)g(tag,)g(lik)o(e)g(attribute)h(k)o(eys,)189 1198 y(so)15 b(as)g(to)g(use)h(the)g(righ)o(t)f(calling)f(con)o(v)o(en) o(tion.)21 b(These)c(handlers)f(ha)o(v)o(e,)e(in)h(C,)g(a)g(\\)p Fd(stdargs)p Fn(")e(argumen)o(t)189 1248 y(list.)24 b(It)17 b(migh)o(t)d(b)q(e)j(useful)f(to)h(pro)o(vide)f(to)g(the)h(handler)f (information)e(on)i(the)h(language)e(en)o(vironmen)o(t)189 1298 y(where)g(the)f(error)h(o)q(ccurred.)20 b(\()p Fe(End)c(of)e (advic)n(e)i(to)e(implementors.)p Fn(\))75 1414 y Fo(9.4.11)49 b(Reduce)16 b(op)q(erations)75 1491 y Fn(Reduce)f(op)q(erations)f (de\014ned)h(in)e(C)h(can)g(b)q(e)h(used)g(or)e(deallo)q(cated)h(in)g (F)m(ortran,)f(and)h(vice)g(v)o(ersa.)189 1578 y Fe(A)n(dvic)n(e)i(to)h (users.)51 b Fn(Reduce)17 b(op)q(erations)f(receiv)o(e,)i(as)e(one)g (of)g(their)g(argumen)o(t,)g(the)g(datat)o(yp)q(e)h(of)e(the)189 1628 y(op)q(erands.)j(Th)o(us,)12 b(one)g(can)h(de\014ne,)g(either)g (in)e(C)h(or)g(F)m(ortran,)g(\\p)q(olymorphic")e(reduce)k(op)q (erations)e(that)189 1678 y(w)o(ork)h(b)q(oth)h(for)g(C)g(and)f(F)m (ortran)h(datat)o(yp)q(es.)19 b(\()p Fe(End)c(of)g(advic)n(e)g(to)g (users.)p Fn(\))189 1765 y Fe(A)n(dvic)n(e)c(to)g(implementors.)34 b Fn(Reduce)11 b(op)q(eration)f(ob)r(jects)h(need)g(to)f(carry)h(a)f (language)f(tag,)h(lik)o(e)f(attribute)189 1815 y(k)o(eys,)k(so)h(as)g (to)g(use)h(the)f(righ)o(t)g(calling)e(con)o(v)o(en)o(tion.)18 b(\()p Fe(End)e(of)e(advic)n(e)i(to)f(implementors.)p Fn(\))75 1931 y Fo(9.4.12)49 b(Status)75 2007 y Fn(The)16 b(MPI)f(status)h(ob)r(ject)g(is)f(not)g(really)f(an)h(opaque)g(ob)r (ject,)h(since)g(its)f(structure)i(is)e(partially)f(visible)g(to)h(the) 75 2057 y(user,)k(and)e(the)h(ob)r(ject)h(is)e(allo)q(cated)g(b)o(y)g (the)h(user.)30 b(A)17 b(C)h(status)g(ob)r(ject)g(is)g(a)f(structure;)k (a)c(F)m(ortran)g(status)75 2107 y(ob)r(ject)e(is)f(an)g(arra)o(y)m(.)k Fk(MPI)13 b Fn(do)q(es)i(not)f(guaran)o(tee)h(that)f(status)h(ob)r (jects)g(can)f(b)q(e)h(passed)g(from)e(one)h(language)f(to)75 2157 y(another.)166 2289 y Fl(Discussion:)34 b Fn(I)14 b(prop)q(ose)g(that)g(w)o(e)g(add)g(to)g(MPI)g(a)g(nondestructiv)o(e)h (request)g(access)h(function,)75 2339 y Fk(MPI)p 152 2339 V 14 w(GET)p 247 2339 V 15 w(ST)m(A)m(TUS\(request,)j(\015ag,)f (status\))p Fn(,)g(that)g(returns)h Fk(\015ag)f(=)g(true)h Fn(if)d(the)j(request)g(is)e(completed)h(and,)75 2389 y(if)f(so,)i(sets)g Fk(status)p Fn(.)30 b(This)18 b(function,)g(unlik)o (e)f Fk(MPI)p 896 2389 V 15 w(TEST)p Fn(,)g(do)q(es)i(not)f(free)h(the) f(request.)32 b(If)17 b(this)h(function)g(is)75 2439 y(a)o(v)n(ailable,)11 b(then)k(one)f(can)g(pass)g(the)h(request)g(ob)r (ject)g(across)g(languages,)e(rather)h(than)g(the)h(status)f(ob)r (ject.)166 2654 y Fl(Alternativ)o(es)o(:)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fp(9.4.)29 b(INTERLANGUA)o(GE)16 b(INTER)o(OPERABILITY)756 b Fv(15)134 45 y Fn(1.)22 b(W)m(e)9 b(pro)o(vide)h(status)h(con)o(v)o(ersion)g(functions,)f(that)h(con)o(v) o(ert)g(a)e(status)i(ob)r(ject)g(to)f(a)g(language)g(indep)q(enden)o(t) 189 95 y(status)k(ob)r(ject,)f(and)g(vice)h(v)o(ersa.)k(The)c(language) e(indep)q(enden)o(t)i(status)g(ob)r(ject)g(is)f(an)g(arra)o(y)g(of)g (in)o(tegers.)134 169 y(2.)22 b(W)m(e)14 b(require)i(that)f(status)h (ob)r(jects)h(ha)o(v)o(e)d(the)i(same)e(memory)f(la)o(y)o(out)h(in)g(C) h(and)g(F)m(ortran)g(so)g(that)g(they)189 219 y(can)f(b)q(e)g(passed)h (across)g(languages.)75 443 y Fo(9.4.13)49 b(Constants)75 519 y Fn(MPI)14 b(constan)o(ts)h(ha)o(v)o(e)f(the)h(same)e(v)n(alue)g (in)h(F)m(ortran)f(and)h(C.)g(This)g(includes,)g(error)g(co)q(des,)h (constan)o(ts)g(suc)o(h)g(as)75 569 y Fd(MPI)p 144 569 14 2 v 15 w(PROC)p 247 569 V 15 w(NULL)8 b Fn(or)i Fd(MPI)p 471 569 V 15 w(ANY)p 552 569 V 15 w(TAG)p Fn(,)e(etc.)17 b(This)9 b(do)q(es)i(not)e(apply)f(to)h(constan)o(t)h(handles)g(\()p Fk(MPI)p 1537 569 13 2 v 14 w(INT,)f(MPI)p 1716 569 V 14 w(COMM)p 1860 569 V 15 w(W)o(ORLD,)75 619 y(MPI)p 152 619 V 14 w(ERRORS)p 326 619 V 14 w(RETURN,)14 b(MPI)p 606 619 V 14 w(SUM)p Fn(,)g(etc.\))22 b(These)16 b(handles)f(need)h(to) f(b)q(e)g(con)o(v)o(erted,)h(as)e(explained)h(in)f(Sec-)75 669 y(tion)f(9.4.4.)k Fk(MPI)p 353 669 V 14 w(BOTTOM)c Fn(ma)o(y)f(ha)o(v)o(e)i(di\013eren)o(t)h(v)n(alues)f(in)f(F)m(ortran)h (and)f(C.)75 785 y Fo(9.4.14)49 b(Interlanguage)17 b(communication)75 862 y Fn(The)12 b(t)o(yp)q(e)f(matc)o(hing)e(rules)j(for)f(comm)o (unicatio)o(ns)e(in)i(MPI)g(are)h(not)f(c)o(hanged:)17 b(The)11 b(datat)o(yp)q(e)g(sp)q(eci\014cation)h(for)75 911 y(eac)o(h)j(item)e(sen)o(t)i(should)g(matc)o(h,)e(textually)m(,)g (the)i(datat)o(yp)q(e)g(sp)q(eci\014cation)g(used)g(to)g(receiv)o(e)g (this)g(item)e(\(unless)75 961 y(one)e(of)f(the)i(t)o(yp)q(es)f(is)g Fk(MPI)p 483 961 V 14 w(P)m(A)o(CKED)p Fn(\).)e(And)i(the)g(t)o(yp)q(e) g(of)g(a)f(message)h(item)e(should)i(matc)o(h)e(the)j(t)o(yp)q(e)f (declaration)75 1011 y(for)g(the)h(corresp)q(onding)g(comm)o(unicatio)o (n)c(bu\013er)13 b(lo)q(cation,)d(unless)i(the)g(t)o(yp)q(e)f(is)g Fk(MPI)p 1434 1011 V 15 w(BYTE)f Fn(or)h Fk(MPI)p 1691 1011 V 15 w(P)m(A)o(CKED)p Fn(.)75 1061 y(In)o(terlanguage)j(comm)o (unicati)o(on)d(is)j(allo)o(w)o(ed,)e(if)h(it)h(complies)e(with)i (these)h(rules.)75 1160 y Fl(Example)g(9.4)23 b Fn(In)14 b(the)g(example)f(b)q(elo)o(w,)g(a)h(F)m(ortran)f(arra)o(y)h(is)g(sen)o (t)g(from)e(F)m(ortran)i(and)g(receiv)o(ed)h(in)f(C.)75 1259 y Fd(!)22 b(FORTRAN)e(CODE)75 1309 y(REAL)h(R\(5\))75 1359 y(INTEGER)f(TYPE,)h(ADDR,)g(IERR,)g(MYRANK)75 1458 y(!)h(create)e(an)i(absolute)e(datatype)g(for)h(array)g(R)75 1508 y(CALL)g(MPI_ADDRESS\()e(R,)j(ADDR\))75 1558 y(CALL)f (MPI_TYPE_STRUCT\(1)o(,)e(5,)i(ADDR,)g(MPI_REAL,)f(TYPE,)h(IERR\))75 1608 y(CALL)g(MPI_TYPE_COMMIT\()d(TYPE\))75 1707 y(CALL)j (MPI_COMM_RANK\()e(MPI_COMM_WORLD,)g(MYRANK,)h(IERR\))75 1757 y(IF)h(\(MYRANK.EQ.0\))e(THEN)140 1807 y(CALL)i(MPI_SEND\()f (MPI_BOTTOM,)g(1,)h(TYPE,)g(1,)g(0,)h(MPI_COMM_WORLD,)c(IERR\))75 1857 y(ELSE)140 1907 y(CALL)j(MPI_HANDLE2INT\(TYPE)o(,)e(CODE\))140 1956 y(CALL)i(C_ROUTINE\(CODE\))75 2006 y(END)g(IF)75 2156 y(/*)g(C)h(code)f(*/)75 2255 y(void)g(C_ROUTINE\(MPI_Fin)o(t)e (*fhandle\))75 2305 y({)75 2355 y(void)i(*handle;)75 2405 y(MPI_Datatype)e(type;)75 2455 y(MPI_Status)h(status;)75 2554 y(type)h(=)h(\(MPI_Datatype\)M)o(PI_In)o(t2han)o(dle\(f)o(handl)o (e\);)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Fv(16)1133 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)75 344 y Fd(MPI_Recv\()20 b(MPI_BOTTOM,)f(1,)j(type,)f(0,) g(0,)g(MPI_COMM_WORLD,)e(status\);)75 394 y(})166 493 y Fn(MPI)13 b(implemen)o(tors)e(ma)o(y)h(w)o(eak)o(en)h(these)i(t)o(yp) q(e)f(matc)o(hing)d(rules,)j(and)f(allo)o(w)f(messages)h(to)g(b)q(e)h (sen)o(t)g(with)75 543 y(F)m(ortran)d(t)o(yp)q(es)h(and)g(receiv)o(ed)g (with)f(C)h(t)o(yp)q(es,)g(and)f(vice)h(v)o(ersa,)g(when)f(those)i(t)o (yp)q(es)f(matc)o(h.)k(I.e.,)11 b(if)f(the)i(F)m(ortran)75 593 y(t)o(yp)q(e)17 b Fk(INTEGER)f Fn(is)h(iden)o(tical)f(to)h(the)g(C) g(t)o(yp)q(e)g Fk(int)p Fn(,)h(then)f(an)g(MPI)g(implemen)o(tatio)o(n)d (ma)o(y)h(allo)o(w)g(data)i(to)f(b)q(e)75 643 y(sen)o(t)g(with)g(datat) o(yp)q(e)g Fk(MPI)p 513 643 13 2 v 14 w(INTEGER)f Fn(and)g(b)q(e)h (receiv)o(ed)h(with)f(datat)o(yp)q(e)g Fk(MPI)p 1365 643 V 14 w(INT)p Fn(.)f(Ho)o(w)o(ev)o(er,)h(suc)o(h)h(co)q(de)f(is)75 693 y(not)e(p)q(ortable.)75 830 y Fs(9.5)59 b(F)n(o)n(rtran)21 b(cha)n(racter)d(strings)i(and)g(MPI)g(bindings)75 1004 y Fl(Discussion:)166 1053 y Fn(This)15 b(is)g(an)f(up)q(dated)i(prop)q (osal)f(to)g(the)g(F)m(ortran)g(c)o(haracter)i(string)e(problem.)20 b(It)15 b(includes)g(parts)h(of)e(the)75 1103 y(original)g(prop)q(osal) h(submitted)g(b)o(y)h(Laurie)f(Costello)g(and)h(Heidi)f(P)o(o)o(xon,)g (and)h(the)g(new)g(prop)q(osal)g(submitted)75 1153 y(b)o(y)e(Bill)e (Gropp.)166 1285 y(This)i(section)g(describ)q(es)i(a)d(problem)g(with)g (the)i(MPI)f(F)m(ortran)f(bindings)g(and)h(c)o(haracter)h(strings)f(in) g(MPI)75 1335 y(functions)g(that)h(allo)o(w)d Fc(<)p Fn(c)o(hoice)p Fc(>)k Fn(datat)o(yp)q(es.)k(MPI)14 b(functions)h(that)f (ha)o(v)o(e)g(the)h(datat)o(yp)q(e)g(parameter)f(can)g(b)q(e)75 1385 y(di\016cult)h(or)h(ev)o(en)h(imp)q(ossible)d(to)i(implem)o(en)o (t)e(in)h(cases)i(where)g(there)g(is)f(no)g(w)o(a)o(y)f(to)h(reliably)f (determine)g(the)75 1435 y(datat)o(yp)q(e.)j(This)c(di\016cult)o(y)e (arises)i(b)q(ecause)i(of)c(the)j(w)o(a)o(y)d(F)m(ortran)i(c)o (haracter)h(descriptors)g(can)e(b)q(e)i(mapp)q(ed)d(b)o(y)75 1485 y(compilers.)75 1601 y Fo(9.5.1)49 b(Detailed)17 b(Problem)f(Description)75 1678 y Fn(The)e(compiler)f(ma)o(y)f(map)g(a) i(F)m(ortran)g(c)o(haracter)h(descriptor)h(as)e(t)o(w)o(o)f(parameters) h(in)g(the)h(parameter)e(list,)g(the)75 1727 y(\014rst)g(b)q(eing)f (the)h(address)h(of)d(the)i(c)o(haracter)h(string,)e(and)g(the)h (second)g(b)q(eing)f(the)h(length.)18 b(There)13 b(is)f(no)g(problem)75 1777 y(if)g(a)h(compiler)e(places)j(the)g(second)g(parameter)e(at)h (the)h(end)f(of)g(the)g(parameter)g(list.)k(Ho)o(w)o(ev)o(er,)d(if)e (the)h(compiler)75 1827 y(places)i(the)g(second)g(parameter)f(after)g (the)h(address)h(of)d(the)i(c)o(haracter)h(string,)e(the)g(rest)i(of)d (the)i(parameters)f(in)75 1877 y(the)g(list)g(are)g(o\013set.)166 1927 y(The)g(follo)o(wing)e(examples)g(sho)o(w)i(2)g(w)o(a)o(ys)f(that) h(compilers)f(map)f(F)m(ortran)i(c)o(haracter)h(strings.)189 2026 y Fc(M)5 b(P)h(I)p 291 2026 V 17 w(dummy)p 444 2026 V 17 w(c)p Fn(\()p Fc(ar)q(g)q Fn(1)p 577 2026 V 15 w Fc(ptr)o(;)h(ar)q(g)q Fn(2)p Fc(;)g(ar)q(g)q Fn(3)p Fc(;)g(ar)q(g)q Fn(1)p 955 2026 V 13 w Fc(l)q(en)p Fn(\))189 2126 y Fc(M)e(P)h(I)p 291 2126 V 17 w(dummy)p 444 2126 V 17 w(c)p Fn(\()p Fc(ar)q(g)q Fn(1)p 577 2126 V 15 w Fc(ptr)o(;)h(ar)q(g)q Fn(1)p 749 2126 V 14 w Fc(l)q(en;)g(ar)q(g)q Fn(2)p Fc(;)g(ar)q(g)q Fn(3\))166 2225 y(The)17 b(argumen)o(ts)f(follo)o(wing)f(arg1)h(in)h (the)g(2nd)g(example)f(ha)o(v)o(e)g(b)q(een)j(shifted)e(b)o(y)f(one)i (to)e(allo)o(w)g(for)g(the)75 2275 y(length)j(of)g(the)g(string.)34 b(The)20 b(datat)o(yp)q(e)f(can't)g(b)q(e)h(examined)e(b)q(ecause)j (its)e(place)g(in)g(the)g(argumen)o(t)f(list)h(is)75 2325 y(dep)q(enden)o(t)d(on)d(the)i(datat)o(yp)q(e.)166 2375 y(V)m(arious)h("parameter)g(coun)o(ting")f(metho)q(ds)h(can)h(b)q (e)g(used)h(for)e(iden)o(tifying)f(c)o(haracter)i(datat)o(yp)q(es)h(if) d(an)75 2425 y(implemen)o(tatio)o(n)h(has)j(access)i(to)e(functions)f (that)h(coun)o(t)g(parameters)g(on)g(argumen)o(t)f(lists.)32 b(Ho)o(w)o(ev)o(er,)20 b(if)e(a)75 2475 y(parameter)9 b(coun)o(ting)g(function)g(do)q(esn't)h(exist,)g(it)f(is)g(imp)q (ossible)f(to)h(reliably)g(determine)g(the)h(F)m(ortran)f(datat)o(yp)q (e.)166 2524 y(T)m(o)j(resolv)o(e)i(this)f(issue,)h(a)f(mec)o(hanism)e (for)h(iden)o(tifying)g(F)m(ortran)h(CHARA)o(CTER)g(strings)g(as)h (argumen)o(ts)75 2574 y(is)g(needed.)166 2624 y(There)h(are)f(3)g(v)n (ariations)e(of)i(this)g(problem)e(in)i(the)g(MPI)g(standard.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fp(9.5.)34 b(F)o(OR)l(TRAN)16 b(CHARA)o(CTER)g (STRINGS)g(AND)f(MPI)g(BINDINGS)425 b Fv(17)75 45 y Fq(No)15 b(CHOICE)h(a)o(rgument)e(fo)o(r)g(datat)o(yp)q(e)75 122 y Fn(The)g(follo)o(wing)c(MPI)k(functions)f(ha)o(v)o(e)g(a)g(bu\013er)h (parameter)f(that)g(can)h(b)q(e)g(of)e(an)o(y)h(datat)o(yp)q(e,)g(but)h (no)f(datat)o(yp)q(e)75 172 y(parameter)h(to)f(sp)q(ecify)i(the)f(t)o (yp)q(e.)145 259 y Fb(\017)23 b Fn(MPI)p 272 259 13 2 v 15 w(ADDRESS\(LOCA)m(TION,)13 b(ADDRESS,)g(IERR)o(OR\))145 333 y Fb(\017)23 b Fn(MPI)p 272 333 V 15 w(BUFFER)p 460 333 V 15 w(A)m(TT)m(A)o(CH\(BUFFER,)14 b(SIZE,)g(IERR)o(OR\))145 408 y Fb(\017)23 b Fn(MPI)p 272 408 V 15 w(BUFFER)p 460 408 V 15 w(DET)m(A)o(CH\(BUFFER,)14 b(SIZE,)g(IERR)o(OR\))75 523 y Fq(CHOICE)i(datat)o(yp)q(e)g(fo)o(r)e(single)i(a)o(rgument)75 599 y Fn(The)11 b(send,)h(receiv)o(e)h(and)d(broadcast)i(MPI)f (functions)g(ha)o(v)o(e)g(bu\013er)h(and)f(datat)o(yp)q(e)g(argumen)o (ts)f(where)i(the)g(bu\013er)75 649 y(is)i(the)g(\014rst)h(parameter)e (in)h(the)g(list,)f(and)h(the)h(datat)o(yp)q(e)f(is)f(in)h(the)g (middle)e(of)i(the)g(parameter)g(list.)75 764 y Fq(CHOICE)i(datat)o(yp) q(e)g(fo)o(r)e(multiple)i(a)o(rguments)75 840 y Fn(A)h(set)i(of)d(MPI)i (functions)f(suc)o(h)i(as)e Fk(MPI)p 752 840 V 14 w(P)m(A)o(CK)p Fn(,)e Fk(MPI)p 975 840 V 14 w(GA)m(THER)p Fn(,)h Fk(MPI)p 1254 840 V 15 w(REDUCE)f Fn(require)k(2)e(bu\013ers)h(in)f(the)75 890 y(parameter)g(list)f(that)h(can)h(b)q(e)f(of)g(an)o(y)g(t)o(yp)q (e.)28 b(The)17 b(datat)o(yp)q(es)h(for)f(these)h(bu\013ers)g(follo)o (w)e(the)h(bu\013ers)i(in)d(the)75 940 y(parameter)e(list.)75 1056 y Fo(9.5.2)49 b(Prop)q(osed)16 b(Solution)i(1)75 1133 y Fn(The)11 b(follo)o(wing)d(prop)q(osal)j(will)e(address)j(all)d (3)i(v)n(ariations)e(to)i(the)g(problem)e(with)i(F)m(ortran)f(c)o (haracter)i(descriptors)75 1183 y(and)i Fc(<)p Fn(c)o(hoice)p Fc(>)h Fn(argumen)o(ts)e(to)h(MPI)g(functions.)166 1232 y(Add)g(the)g(follo)o(wing)e(function)h(to)h(the)h(F)m(ortran)e (binding:)75 1332 y Fd(MPI_CADDRESS\()19 b(BUF,)i(ADDRESS,)f(IERROR)h (\))75 1382 y(CHARACTER*)f(\(*\))86 b(BUF)75 1432 y(INTEGER)238 b(ADDRESS,)20 b(IERROR)166 1531 y Fn(Users)d(w)o(ould)d(call)g(this)h (function)g(to)g(get)g(the)h(address)g(of)f(a)g Fd(CHARACTER)e Fn(bu\013er)j(b)q(efore)g(using)e(it.)22 b(They)75 1581 y(could)12 b(then)h(use)f Fk(MPI)p 423 1581 V 15 w(BOTTOM)f Fn(and)h(a)g(user-de\014ned)i(datat)o(yp)q(e)e(\(built)g(with)f Fk(MPI)p 1425 1581 V 15 w(TYPE)p 1548 1581 V 14 w(STRUCT)p Fn(\))h(to)f(pass)75 1631 y Fd(CHARACTER)h Fn(data)i(to)f(the)i (functions)f(that)g(allo)o(w)e(c)o(hoice)i(datat)o(yp)q(es.)19 b(F)m(or)13 b(example:)249 1730 y Fd(...)249 1780 y(char)21 b(mydata*128)249 1830 y(int)h(types[2])259 b(/*)22 b(MPI_Datatype)d(*/) 249 1880 y(int)j(disp[2])249 1930 y(int)g(block_len[2])249 1980 y(integer)f(chartype)249 2079 y(MPI_Caddress\(buf,)e(disp[1]\))249 2129 y(block_len[1]=50)249 2179 y(types[1])h(=)i(MPI_BYTE)249 2229 y(MPI_Type_struct\(1,)c(block_len,)i(disp,)h(types,)f (MPI_COMM_WORLD\))249 2278 y(MPI_Type_commit\(cha)o(rtype)o(\))249 2328 y(MPI_Send\(MPI_Bottom)o(,)f(length?,)h(chartype,)g(...\))249 2378 y(...)166 2560 y Fl(Discussion:)53 b Fn(The)19 b(addition)e(of)h (this)h(new)g(function)f(is)h(the)g(smallest)e(c)o(hange)i(that)g (addresses)h(the)75 2610 y(functions)13 b(men)o(tioned)f(ab)q(o)o(v)o (e,)g(and)g(that)h(allo)o(ws)f(users)i(in)e(F)m(ortran)h(to)g(reliably) e(send)j(and)f(receiv)o(e)h Fd(CHARACTER)75 2660 y Fn(data.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Fv(18)1133 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)166 45 y Fn(The)14 b(do)o(wn)e(side)i(of)e(this)i(approac) o(h)f(is)g(that)g(it)g(mak)o(es)f(it)h(quite)g(cum)o(b)q(ersome)f(for)h (F)m(ortran)g(programmers)75 95 y(to)i(send)h Fd(CHARACTER)e Fn(data.)22 b(Note)16 b(also)e(that)i Fk(MPI)p 905 95 13 2 v 14 w(CADDRESS)e Fn(is)h(needed)i(only)e(on)g(systems)g(that)h (pass)f(the)75 145 y(address)j(of)d(the)i(dop)q(e)g(v)o(ector)g(\(and)g (hence)h(pro)o(vide)e(no)g(w)o(a)o(y)g(to)g(detect)i(the)f(di\013eren)o (t)g(datat)o(yp)q(e.\))26 b(Systems)75 195 y(that)12 b(do)f(not)g(pass)h(the)g(address)h(of)e(the)h(dop)q(e)g(v)o(ector)g (can)g(use)g(MPI)p 1152 195 V 16 w(ADDRESS.)e(If)h(w)o(e)h(c)o(ho)q (ose)g(not)g(to)f(supp)q(ort)75 244 y(systems)j(that)g(pass)g(the)h (address)g(of)e(the)i(dop)q(e)f(v)o(ector,)g(no)g(c)o(hange)g(to)g(the) g(existing)g(standard)g(is)g(required.)166 294 y(If)9 b(w)o(e)g(add)g(this)h(function,)f(existing)g(MPI)h(applications)e (that)h(send)h(messages)g(in)o(v)o(olving)d(F)m(ortran)i Fd(CHARACTER)75 344 y Fn(strings)14 b(will)f(ha)o(v)o(e)g(to)h(c)o (hange)g(or)g(they)h(will)d(not)i(b)q(e)g(p)q(ortable.)166 394 y(This)9 b(approac)o(h)h(w)o(ould)e(require)j(the)f(addition)e(of)h (text)h(regarding)f(the)h(use)g(of)f(F)m(ortran)g Fd(CHARACTER)f Fn(strings.)75 444 y(An)k(implemen)o(tatio)o(n)d(not)j(supp)q(orting)g Fd(CHARACTER)e Fn(strings)i(on)g(existing)f(calls)h(w)o(ouldn't)e(b)q (e)j(complian)o(t)c(unless)75 493 y(the)k(text)f(required)h(the)g(use)g (of)f(MPI)p 659 493 V 15 w(CADDRESS)f(for)h(passing)g Fd(CHARACTER)e Fn(strings.)18 b(CRI's)11 b(implemen)o(tation)75 543 y(already)j(allo)o(ws)e(this)i(b)o(y)g(supp)q(orting)g(a)f Fd(CHARACTER)f Fn(string)i(datat)o(yp)q(e)g(on)g(the)h(call)e(to)g(MPI) p 1568 543 V 16 w(ADRESS.)166 593 y(It)h(w)o(ould)e(b)q(e)i(b)q(etter)h (to)f(ha)o(v)o(e)f(the)h(parameter)f Fd(ADDRESS)p Fn(,)f(in)h(the)h (MPI)p 1305 593 V 15 w(CADDRESS)f(function,)g(to)g(b)q(e)i(of)75 643 y(t)o(yp)q(e)f(MPI)p 251 643 V 16 w(AINT)g(instead)g(of)f(t)o(yp)q (e)i Fd(INTEGER)p Fn(.)75 842 y Fo(9.5.3)49 b(Prop)q(osed)16 b(Solution)i(2)75 918 y Fn(Change)c(the)g(MPI)h(sp)q(eci\014cation)f (to)g(state:)166 968 y(\\The)20 b(supp)q(ort)h(of)e(F)m(ortran)g Fd(CHARACTER)f Fn(strings)j(in)e(functions)h(that)g(use)g(argumen)o(ts) f(with)h(a)f(c)o(hoice)75 1018 y(datat)o(yp)q(e)d(is)f(optional)f(and)h (up)h(to)f(an)h(implem)o(en)o(tation,)c(since)17 b(the)f(F)m(ortran)f (MPI)h(functions)f(are)h(not)g(ANSI)75 1068 y(complian)o(t.)166 1118 y(F)m(or)f(MPI)g(implemen)o(tati)o(ons)e(that)i(do)f(not)h(supp)q (ort)h(F)m(ortran)f Fd(CHARACTER)e Fn(strings,)i(it)g(ma)o(y)e(b)q(e)j (p)q(ossible)75 1167 y(to)f(use)g Fd(CHARACTER)e Fn(strings)i(b)o(y)g (equiv)n(alencing)f(them)g(to)g(another)i(datat)o(yp)q(e)f(suc)o(h)g (as)g Fd(INTEGER)e Fn(and)i(passing)75 1217 y(them)e(as)h Fk(MPI)p 309 1217 V 14 w(BYTE)p Fn(.")166 1350 y Fl(Discussion:)31 b Fn(This)11 b(solution)g(allo)o(ws)e(v)o(endors)j(to)f(disallo)o(w)f Fd(CHARACTER)f Fn(datat)o(yp)q(e)j(if)e(they)i(ha)o(v)o(e)f(no)g(w)o(a) o(y)75 1400 y(of)g(implemen)o(ting)e(functions)j(with)g("c)o(hoice")g (datat)o(yp)q(es)g(b)q(ecause)i(of)d(the)i(w)o(a)o(y)e(they)i(map)d(F)m (ortran)i Fd(CHARACTER)75 1449 y Fn(descriptors.)19 b(The)11 b(user)i(still)d(has)h(a)g(mec)o(hanism)e(within)i(the)h(F)m(ortran)f (language)f(to)h(pass)h Fd(CHARACTER)p Fn(,)d(ho)o(w)o(ev)o(er)75 1499 y(this)15 b(mec)o(hanism)d(is)j(not)g(necessarily)h(p)q(ortable)f (b)q(et)o(w)o(een)h(di\013eren)o(t)g(arc)o(hitectures.)23 b(The)16 b(main)c(concern)17 b(here)75 1549 y(is)g(a)g(decrease)j(in)d (p)q(ortabilit)o(y)f(in)h(MPI)g(applications)g(that)g(send)h(messages)g (in)o(v)o(olving)d(F)m(ortran)i Fd(CHARACTER)75 1599 y Fn(strings.)166 1649 y(If)12 b(w)o(e)g(c)o(hange)g(the)h(standard)f (to)g(mak)o(e)f(F)m(ortran)g Fd(CHARACTER)f Fn(optional,)h(existing)h (MPI)g(applications)f(that)75 1698 y(send)j(messages)f(in)o(v)o(olving) e(F)m(ortran)h Fd(CHARACTER)g Fn(strings)h(will)f(not)g(ha)o(v)o(e)h (to)g(c)o(hange,)g(but)g(will)f(not)h(necessarily)75 1748 y(b)q(e)i(p)q(ortable.)166 1798 y(CRI's)9 b(implem)o(en)o(tation)d (is)j(no)o(w)g(complian)o(t)e(with)i(the)h(standard.)17 b(W)m(e)9 b(ha)o(v)o(e)g(c)o(hosen)h(to)f(supp)q(ort)h Fd(CHARACTER)75 1848 y Fn(strings)k(with)g(the)g(restriction)h(that)f (calls)g(con)o(taining)e(t)o(w)o(o)i(c)o(hoice)g(parameters)g(m)o(ust)f (b)q(oth)h(b)q(e)g Fd(CHARACTER)p Fn(.)75 2047 y Fo(9.5.4)49 b(Prop)q(osed)16 b(Solution)i(3)75 2123 y Fn(Add)9 b Fd(CHARACTER)f Fn(function)h(calls)g(to)g(MPI-2)g(for)g(all)f(F)m (ortran)h(MPI)g(functions)h(that)f(allo)o(w)f(the)i Fk(MPI)p 1639 2123 V 14 w(CHARA)o(CTER)75 2173 y Fn(datat)o(yp)q(e.)18 b(Do)q(cumen)o(t)13 b(the)i(p)q(oten)o(tial)e(problem)g(in)g(MPI-1.)166 2306 y Fl(Discussion:)30 b Fn(If)10 b(w)o(e)h(add)f(these)h(functions,) g(existing)f(MPI)g(applications)f(that)i(send)g(messages)f(in)o(v)o (olving)75 2355 y(F)m(ortran)16 b Fd(CHARACTER)e Fn(strings)j(will)e (ha)o(v)o(e)h(to)g(c)o(hange)g(or)h(they)f(will)f(not)h(b)q(e)h(p)q (ortable.)25 b(Adding)16 b(a)g(set)h(of)f(new)75 2405 y(functions)c(is)g(not)g(desirable,)g(ho)o(w)o(ev)o(er)h(sending)f (messages)g(that)g(in)o(v)o(olv)o(e)f Fd(CHARACTER)f Fn(strings)i(is)g(more)f(straigh)o(t)75 2455 y(forw)o(ard,)i(and)h(the) g(c)o(hange)g(will)f(create)i(p)q(ortable)f(co)q(de)h(in)e(the)i (future.)166 2505 y(This)c(solution)g(eliminates)f(the)i(c)o(hec)o (king)g(of)e(parameters)i(in)f(the)h(F)m(ortran)f(in)o(terface,)h(whic) o(h)g(will)e(increase)75 2555 y(p)q(erformance.)26 b(It)17 b(also)f(conforms)f(with)i(existing)f(practice,)i(in)e(that)h(F)m (ortran)f(in)o(terfaces)i(allo)o(wing)c(m)o(ultiple)75 2605 y(datat)o(yp)q(es)g(exclude)h Fd(CHARACTER)d Fn(strings.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fp(9.6.)34 b(TREA)l(TMENT)15 b(OF)g Fq(MPI)p 661 -100 14 2 v 16 w(ST)l(A)l(TUS)991 b Fv(19)75 45 y Fs(9.6)59 b(T)-5 b(reatment)18 b(of)i(MPI)p 670 45 18 2 v 21 w(Status)75 136 y Fn(Tw)o(o)13 b(prop)q(osals)h(are)h(b)q (eing)e(made)g(that)h(add)g(to,)f(rather)i(than)f(c)o(hange,)f(the)i (treatmen)o(t)f(of)f Fk(MPI)p 1636 136 13 2 v 14 w(Status)p Fn(.)75 252 y Fo(9.6.1)49 b(P)o(assing)17 b Fa(NULL)d Fo(fo)o(r)i(MPI)p 670 252 15 2 v 18 w(Status)75 329 y Fn(There)11 b(are)g(a)e(n)o(um)o(b)q(er)h(of)f(places)i(in)e(the)i (MPI-1)f(sp)q(eci\014cation)h(where)g(e\016ciency)g(could)f(b)q(e)h (impro)o(v)o(ed)d(if)h(a)h(NULL)75 379 y(or)j(0)g(argumen)o(t)f(could)g (b)q(e)i(passed)g(instead)f(of)g(the)g(curren)o(tly-sp)q(eci\014ed)j (argumen)o(t)c(as)h(an)f(indication)g(that)h(the)75 428 y(argumen)o(t)c(need)j(not)e(b)q(e)h(pro)q(cessed)i(b)o(y)d(the)h(MPI)f (function.)17 b(In)10 b(particular,)h(the)g Fk(MPI)p 1442 428 13 2 v 14 w(Status)g Fn(argumen)o(t)e(passed)75 478 y(to)j(certain)i(MPI)e(functions)h(could)f(b)q(e)i(allo)o(w)o(ed)d (to)h(b)q(e)h(NULL,)g(in)f(whic)o(h)g(case)i(status)f(\014elds)g(need)h (not)e(b)q(e)h(\014lled)75 528 y(in.)166 578 y(In)h(F)m(ortran,)f(one)h (could)g(use)h Fk(MPI)p 716 578 V 14 w(BOTTOM)p Fn(.)166 628 y(In)g(general,)h(this)g(optimization)c(can)k(apply)f(to)g(all)g (functions)g(for)g(whic)o(h)h Fk(MPI)p 1451 628 V 14 w(Status)g Fn(is)f(an)h(argumen)o(t.)75 678 y(These)f(are)f(all)f(the)i (v)n(arious)e(forms)f(of)i Fk(MPI)p 775 678 V 14 w(RECV)p Fn(,)e Fk(MPI)p 995 678 V 14 w(TEST)p Fn(,)i(and)g Fk(MPI)p 1295 678 V 14 w(W)m(AIT)p Fn(.)75 794 y Fo(9.6.2)49 b(Non-destructive) 16 b(T)l(est)f(of)i(MPI)p 808 794 15 2 v 18 w(Status)75 870 y Fn(This)9 b(call)g(is)h(useful)f(in)h(handlers)g(to)f(access)i (the)g(information)6 b(asso)q(ciated)11 b(with)e(a)g(request,)j (without)d(deleting)g(the)75 920 y(request)17 b(\(in)d(case)i(the)f (user)i(is)d(exp)q(ected)j(to)e(access)i(it)d(after)h(the)h(handler\).) 21 b(It)15 b(allo)o(ws)f(one)h(to)g(la)o(y)o(er)f(libraries)75 970 y(more)j(con)o(v)o(enien)o(tly)m(,)i(since)g(m)o(ultiple)d(la)o(y)o (ers)j(of)e(soft)o(w)o(are)i(ma)o(y)d(access)21 b(the)e(same)e (completed)h(request)i(and)75 1020 y(extract)15 b(from)d(it)i(the)h (status)f(information.)i(This)e(will)e(b)q(e)j(also)e(b)q(e)i(imp)q (ortan)o(t)d(for)i(language)f(in)o(terop)q(erabilit)o(y)75 1070 y(if)i(w)o(e)i(decide)g(that)f(status)h(ob)r(jects)h(cannot)e(b)q (e)h(transferred)h(across)f(language)e(b)q(oundaries,)i(since)g(with)f (this)75 1119 y(function)e(the)g(status)h(ob)r(ject)f(can)h(b)q(e)f (transferred)i(instead.)75 1264 y Fk(MPI)p 152 1264 13 2 v 14 w(GET)p 247 1264 V 15 w(ST)m(A)m(TUS\()d(request,)i(\015ag,)e (status\))117 1334 y Fn(IN)154 b Fk(request)465 b Fn(an)14 b Fk(MPI)p 1040 1334 V 14 w(REQUEST)f Fn(ob)r(ject)117 1397 y(OUT)107 b Fk(\015ag)525 b Fn(\015ag,)13 b(same)g(as)h(from)e (MPI)p 1331 1397 V 15 w(TEST)117 1459 y(OUT)107 b Fk(status)485 b(MPI)p 982 1459 V 15 w(ST)m(A)m(TUS)12 b Fn(ob)r(ject)j(if)e(\015ag)g (is)h(true)75 1577 y Fd(int)21 b(MPI)p 231 1577 14 2 v 15 w(Get)p 312 1577 V 15 w(status\()g(MPI)p 568 1577 V 15 w(Request)f(request,)g(int)h(*flag,)g(MPI)p 1259 1577 V 15 w(Status)g(*status\))75 1656 y(MPI)p 144 1656 V 15 w(GET)p 225 1656 V 15 w(STATUS\()f(REQUEST,)h(FLAG,)f(STATUS,)h (IERROR\))162 1706 y(INTEGER)g(REQUEST,)f(FLAG,)g(STATUS\(MPI)p 886 1706 V 14 w(STATUS)p 1032 1706 V 14 w(SIZE\),)h(IERROR)166 1786 y Fn(Sets)d(\015ag=true)g(if)f(the)h(request)h(has)e(completed,)h (and,)f(if)g(so,)h(returns)h(in)d(status)j(the)f(request)h(status.)75 1836 y(Ho)o(w)o(ev)o(er,)d(unlik)o(e)f(test)i(or)f(w)o(ait,)f(it)g(do)q (es)i(not)e(deallo)q(cate)h(or)g(inactiv)n(ate)f(the)h(request;)i(a)e (subsequen)o(t)h(call)e(to)75 1886 y(test,)f(w)o(ait)g(or)f(free)i (should)f(b)q(e)g(executed)i(with)e(that)g(request.)75 2023 y Fs(9.7)59 b(MPI-1.0)19 b(and)h(MPI-1.1)f(Issues)75 2122 y Fo(9.7.1)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 2122 15 2 v 18 w(INTERCOMM)p 936 2122 V 20 w(CREA)l(TE)75 2199 y Fq(The)f(Problem:)41 b Fn(The)14 b(MPI)f(1.1)g(standard)h(sa)o (ys,)f(in)g(the)i(discussion)f(of)f Fk(MPI)p 1333 2199 13 2 v 14 w(INTERCOMM)p 1598 2199 V 14 w(CREA)m(TE)p Fn(,)f(b)q(oth)75 2249 y(that)189 2323 y(The)i(groups)g(m)o(ust)f(b)q (e)h(disjoin)o(t)75 2398 y(and)g(that)189 2473 y(The)g(leaders)h(ma)o (y)d(b)q(e)i(the)h(same)e(pro)q(cess.)75 2548 y(T)m(o)c(further)i(m)o (uddy)e(the)i(w)o(aters,)g(the)f(reason)h(giv)o(en)f(for)f("The)i (groups)f(m)o(ust)f(b)q(e)i(disjoin)o(t")e(is)h(based)g(on)g(concerns) 75 2597 y(ab)q(out)k(the)h(implm)o(en)o(tation)c(of)i Fk(MPI)p 668 2597 V 15 w(INTERCOMM)p 934 2597 V 14 w(CREA)m(TE)f Fn(that)j(are)f(not)g(applicable)f(for)h(the)h(case)g(where)75 2647 y(the)f(leaders)h(are)g(the)f(same)f(pro)q(cess.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Fv(20)1138 b Fp(CHAPTER)15 b(9.)30 b(MISCELLANY)75 45 y Fq(The)16 b(Fix:)41 b Fn(Delete)14 b(the)h(text:)189 120 y(\(the)f(t)o(w)o(o)g(leaders)g(could)g(b)q(e)h (the)f(same)f(pro)q(cess\))75 195 y(from)f(the)j(discussion)f(of)f Fk(MPI)p 563 195 13 2 v 15 w(INTERCOMM)p 829 195 V 14 w(CREA)m(TE)p Fn(.)166 244 y(Replace)h(the)g(text:)189 319 y(All)d(in)o(ter-comm)o(unicator)e(constructors)14 b(are)f(blo)q(c)o(king)e(and)h(require)h(that)f(the)h(lo)q(cal)e(and)h (remote)189 369 y(groups)i(b)q(e)g(disjoin)o(t)f(in)g(order)i(to)f(a)o (v)o(oid)e(deadlo)q(c)o(k.)75 444 y(with)189 518 y(All)f(in)o(ter-comm) o(unicator)e(constructors)14 b(are)f(blo)q(c)o(king)e(and)h(require)h (that)f(the)h(lo)q(cal)e(and)h(remote)189 568 y(groups)i(b)q(e)g (disjoin)o(t.)189 630 y(Advice)g(to)g(users:)19 b(The)14 b(groups)h(m)o(ust)d(b)q(e)j(disjoin)o(t)e(for)g(sev)o(eral)i(reasons.) k(Primarily)m(,)10 b(this)k(is)g(the)189 680 y(in)o(ten)o(t)h(of)f(the) i(in)o(tercomm)o(unicators)d({)i(to)g(pro)o(vide)g(a)g(comm)o(unicator) d(for)j(comm)o(unication)d(b)q(e-)189 730 y(t)o(w)o(een)i(disjoin)o(t)e (groups.)18 b(This)c(is)f(re\015ected)j(in)d(the)h(de\014nition)f(of)g Fk(MPI)p 1332 730 V 14 w(INTERCOMM)p 1597 730 V 14 w(MERGE)p Fn(,)189 780 y(whic)o(h)i(allo)o(ws)f(the)i(user)g(to)f(con)o(trol)g (the)h(ranking)f(of)g(the)h(pro)q(cesses)i(in)c(the)i(created)h(in)o (tracom-)189 830 y(m)o(unicator;)e(this)h(ranking)g(mak)o(es)f(little)h (sense)i(if)d(the)i(groups)g(are)f(not)g(disjoin)o(t.)25 b(In)16 b(addition,)189 879 y(the)e(natural)g(extension)h(of)f (collectiv)o(e)g(op)q(erations)g(to)g(in)o(tercomm)o(unicators)f(\(b)q (eing)h(considered)189 929 y(for)h(MPI-2\))g(mak)o(es)g(the)h(most)e (sense)j(when)f(the)g(groups)g(are)g(disjoin)o(t.)21 b(It)16 b(is)f(the)h(in)o(ten)o(t)g(of)f(the)189 979 y(MPI)f(de\014nition)f(not)h(to)g(preclude)h(future)f(extensions.)75 1095 y Fo(9.7.2)49 b(Erro)o(r)15 b(Class)i(fo)o(r)f(Invalid)i(Keyval)75 1172 y Fn(There)d(is)f(a)f(new)i(MPI)f(error)h(class:)j Fk(MPI)p 744 1172 V 14 w(ERR)p 837 1172 V 15 w(KEYV)m(AL)166 1304 y Fl(Discussion:)43 b Fn(A)o(t)16 b(the)h(momen)o(t)c(the)k(b)q (est)g(w)o(e)g(can)f(do)g(with)g(an)g(in)o(v)n(alid)e(k)o(eyv)n(al)h (argumen)o(t)g(is)h(return)75 1354 y Fk(MPI)p 152 1354 V 14 w(ERR)p 245 1354 V 15 w(ARG)p Fn(,)g(or)h(p)q(erhaps)h Fk(MPI)p 659 1354 V 15 w(ERR)p 753 1354 V 14 w(OTHER)p Fn(.)f(W)m(e're)g(free)h(to)f(de\014ne)h(whatev)o(er)g(extra)g(error)g (co)q(des)h(w)o(e)75 1404 y(w)o(an)o(t)12 b(to)h(in)f(an)h(implemen)o (tatio)o(n,)d(but)j(this)g(one)g(seems)g(ob)o(vious;)f(it's)g(prett)o (y)i(m)o(uc)o(h)d(the)j(only)e(sp)q(eci\014c)i(kind)f(of)75 1454 y(MPI)h(argumen)o(t)f(that)h(do)q(esn't)g(curren)o(tly)h(ha)o(v)o (e)f(its)g(o)o(wn)f(class.)75 1653 y Fo(9.7.3)49 b(V)o(ersion)16 b(Numb)q(er)75 1729 y Fn(V)m(ersions)c(1.0)e(and)h(1.1)f(of)h(the)g (MPI)h(Standard)f(are)h(already)f(sligh)o(tly)f(inconsisten)o(t,)i (necessitating)g(some)e(w)o(a)o(y)h(of)75 1779 y(distinguishing)h(them) g(for)g(library)g(writers.)19 b(This)13 b(ma)o(y)e(happ)q(en)i(again.)k (A)o(t)c(the)g(risk)g(of)g(encouraging)f(c)o(hanges)75 1829 y(to)i(the)g(Standard,)g(it)f(migh)o(t)f(b)q(e)j(helpful)e(if)g (there)i(w)o(as)f(a)g(p)q(ortable)g(w)o(a)o(y)f(of)g(\014nding)g(out)h (whic)o(h)g(v)o(ersion)g(of)f(the)75 1879 y(Standard)h(one)g(w)o(as)g (compiling)d(for,)i(or)h(running)g(on.)166 1929 y(In)g(C,)f(w)o(e)h (could)g(ha)o(v)o(e)162 2016 y Fd(#define)21 b(MPI_VERSION)e(1.2)75 2103 y Fn(and)14 b(in)f(F)m(ortran,)162 2190 y Fd(REAL)21 b(MPI_VERSION)162 2240 y(PARAMETER)f(\(MPI_VERSION)f(=)j(1.2\))166 2327 y Fn(Alternativ)o(e)14 b(Prop)q(osal)g(1:)166 2377 y(It)g(has)g(b)q(een)h(suggested)g(that)f(this)g(b)q(e)h(a)e(c)o (haracter)i(string)f(instead)h(of)e(a)g(REAL.)166 2427 y(Alternativ)o(e)h(Prop)q(osal)g(2:)166 2477 y(Tw)o(o)f(separate)i(in)o (tegers,)g(for)e(the)i(v)o(ersion)f(and)f(sub)o(v)o(ersion:)166 2526 y(In)h(C,)f(w)o(e)h(could)g(ha)o(v)o(e)162 2614 y Fd(#define)21 b(MPI_VERSION)84 b(1)162 2663 y(#define)21 b(MPI_SUBVERSION)d(2)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fp(9.8.)34 b(PR)o(OBLEMS)16 b(WITH)f(REGISTER)h (OPTIMIZA)l(TION)h(AND)e(CA)o(CHING)275 b Fv(21)75 45 y Fn(and)14 b(in)f(F)m(ortran,)162 132 y Fd(INTEGER)21 b(MPI_VERSION,)e(MPI_SUBVERSION)162 182 y(PARAMETER)h(\(MPI_VERSION)85 b(=)21 b(1\))162 232 y(PARAMETER)f(\(MPI_SUBVERSION)f(=)i(2\))166 319 y Fn(F)m(or)12 b(run)o(time)g(determination,)f(w)o(e)i(could)f (either)i(use)f(an)g(attribute)g(on)f Fk(MPI)p 1392 319 13 2 v 15 w(COMM)p 1537 319 V 14 w(W)o(ORLD)p Fn(,)g(or)h(some-)75 369 y(thing)g(lik)o(e)g(the)i(follo)o(wing:)75 513 y Fk(MPI)p 152 513 V 14 w(VERSION\()e(version,)h(subversion)i(\))117 584 y Fn(OUT)107 b Fk(version)469 b Fn(v)o(ersion)14 b(n)o(um)o(b)q(er)117 646 y(OUT)107 b Fk(subversion)411 b Fn(sub)o(v)o(ersion)15 b(n)o(um)o(b)q(er)75 764 y Fd(int)21 b(MPI)p 231 764 14 2 v 15 w(Version\()f(int)i(*version,)d(int)j (*subversion)d(\))75 844 y(MPI)p 144 844 V 15 w(VERSION\()h(VERSION,)g (SUBVERSION,)g(IERROR)g(\))162 893 y(INTEGER)h(VERSION,)f(SUBVERSION,)f (IERROR)75 1061 y Fs(9.8)59 b(Problems)20 b(with)f(Register)g (Optimization)g(and)g(Caching)75 1152 y Fn(MPI)h(pro)o(vides)g(op)q (erations)h(whic)o(h)e(ma)o(y)f(b)q(e)j(hidden)f(from)e(the)j(user)g (co)q(de)g(and)f(run)g(in)f(parallel)g(with)h(it,)75 1201 y(accessing)15 b(the)g(same)e(memory)f(as)i(user)i(co)q(de.)j (Examples)13 b(include)i(the)f(data)g(transfer)h(for)f(an)g Fk(MPI)p 1694 1201 13 2 v 15 w(IRECV)e Fn(or)75 1251 y(for)g(an)h Fk(MPI)p 271 1251 V 14 w(PUT)p Fn(.)f(The)h(optimizer)e (of)h(a)h(compiler)e(will)g(assume)h(that)h(it)f(can)h(recognize)h(p)q (erio)q(ds)f(when)h(a)e(cop)o(y)75 1301 y(of)h(a)g(v)n(ariable)g(can)g (b)q(e)h(k)o(ept)g(in)f(a)h(register)g(without)f(reloading)g(from)f(or) h(storing)h(to)f(memory)m(.)i(When)f(the)g(user)75 1351 y(co)q(de)k(is)f(w)o(orking)g(with)g(a)g(register)h(cop)o(y)f(of)g (some)f(v)n(ariable)h(while)f(the)i(hidden)g(op)q(eration)f(reads)h(or) f(writes)75 1401 y(the)f(memory)c(cop)o(y)m(,)j(problems)f(o)q(ccur.)22 b(This)15 b(section)h(discusses)h(register)f(optimization)d(pitfalls)g (and)i(related)75 1451 y(problems)e(with)g(cac)o(he)i(coherence.)75 1567 y Fo(9.8.1)49 b(A)17 b(F)o(o)o(rtran)d(Problem)i(with)h(V)o (olatile)g(Bu\013ers)75 1643 y Fn(When)d(a)g(v)n(ariable)f(is)h(lo)q (cal)g(to)g(a)f(F)m(ortran)h(subroutine)h(\(i.e.)k(not)14 b(in)g(a)f(COMMON)i(blo)q(c)o(k\),)f(the)h(compiler)d(will)75 1693 y(assume)17 b(that)h(it)g(cannot)g(b)q(e)h(mo)q(di\014ed)d(b)o(y)i (a)g(called)f(subroutine)i(unless)g(it)e(is)h(an)g(actual)f(argumen)o (t)g(of)g(the)75 1743 y(call.)g(In)c(the)g(most)f(common)e(link)n(age)h (con)o(v)o(en)o(tion,)h(the)i(subroutine)f(is)g(exp)q(ected)i(to)d(sa)o (v)o(e)h(and)g(restore)h(certain)75 1793 y(registers.)19 b(Th)o(us,)14 b(the)g(optimizer)e(will)g(assume)h(that)h(a)f(register)i (whic)o(h)e(held)h(a)f(v)n(alid)f(cop)o(y)i(of)f(suc)o(h)h(a)f(v)n (ariable)75 1843 y(b)q(efore)i(the)f(call)f(will)g(still)g(hold)g(a)g (v)n(alid)g(cop)o(y)h(on)f(return.)166 1892 y(Normally)f(users)j(are)g (not)f(a\017icted)g(with)g(this.)19 b(But)c(the)g(user)g(should)f(pa)o (y)g(atten)o(tion)g(to)g(this)g(section)h(if)75 1942 y(in)h(his/her)g(program)e(a)i(bu\013er)h(argumen)o(t)e(to)h(an)g Fk(MPI)p 975 1942 V 14 w(SEND)p Fn(,)g Fk(MPI)p 1199 1942 V 14 w(RECV)f Fn(etc.)25 b(uses)17 b(a)f(name)f(whic)o(h)h(hides) 75 1992 y(the)f(actual)e(v)n(ariables)h(in)o(v)o(olv)o(ed.)j Fk(MPI)p 700 1992 V 14 w(BOTTOM)d Fn(with)g(an)g(MPI)p 1144 1992 V 15 w(Datat)o(yp)q(e)g(con)o(taining)f(absolute)h(addresses) 75 2042 y(is)h(one)f(example.)20 b(Creating)14 b(a)g(datat)o(yp)q(e)h (whic)o(h)g(uses)h(one)f(v)n(ariable)e(as)i(an)f(anc)o(hor)h(and)g (brings)f(along)g(others)75 2092 y(b)o(y)g(using)f Fk(MPI)p 318 2092 V 14 w(ADDRESS)g Fn(to)h(determine)g(their)g(o\013sets)h(from) d(the)i(anc)o(hor)g(is)g(another.)k(The)c(anc)o(hor)g(v)n(ariable)75 2141 y(w)o(ould)19 b(b)q(e)g(the)h(only)f(one)g(men)o(tioned)g(in)g (the)g(call.)34 b(Also)19 b(atten)o(tion)g(m)o(ust)g(b)q(e)g(pa)o(y)o (ed)h(if)e(MPI)i(op)q(erations)75 2191 y(are)d(used)g(that)f(run)h(in)f (parallel)f(with)h(the)h(user's)h(application.)24 b(This)16 b(op)q(erations)g(are)h(mainly)d Fk(MPI)p 1734 2191 V 14 w(IRECV)p Fn(,)75 2241 y Fk(MPI)p 152 2241 V 14 w(PUT)p Fn(,)f Fk(MPI)p 350 2241 V 14 w(GET)p Fn(,)g Fk(MPI)p 545 2241 V 15 w(IPUT)f Fn(and)i Fk(MPI)p 824 2241 V 14 w(IGET)p Fn(.)166 2291 y(The)g(follo)o(wing)e(example)g(sho)o(ws,)i (what)g(F)m(ortran)f(compilers)g(are)h(allo)o(w)o(ed)f(to)h(do:)288 2391 y(Source)g(of)g(Pro)q(cess)i(1)155 b(Source)15 b(of)f(Pro)q(cess)i (2)155 b(Executed)16 b(in)d(Pro)q(cess)j(2)288 2440 y(bbbb)e(=)g(777) 292 b(aaa)14 b(=)g(bu\013)315 b(reg)p 1360 2440 V 16 w(A:=bu\013)1303 2490 y(aaa:=reg)p 1467 2490 V 15 w(A)795 2540 y(call)14 b(MPI)p 955 2540 V 15 w(SEND\(rdy\))128 b(call)13 b(MPI)p 1462 2540 V 16 w(SEND\(rdy\))288 2590 y(call)g(MPI)p 447 2590 V 15 w(RECV\(rdy\))795 2640 y(call)h(MPI)p 955 2640 V 15 w(W)-5 b(AIT)221 b(stop)15 b(appl.)5 b(thread)288 2689 y(call)13 b(MPI)p 447 2689 V 15 w(PUT\(bbbb)646 b(bu\013:=777)14 b(in)f(PUT)h(thread)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 -100 a Fv(22)1133 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)288 45 y Fn(in)o(to)13 b(bu\013)h(of)f(pro)q(cess)j(2\)) 618 b(con)o(tin)o(ue)14 b(appl.)6 b(thread)288 95 y(call)13 b(MPI)p 447 95 13 2 v 15 w(BARRIER\(...\))71 b(call)14 b(MPI)p 955 95 V 15 w(BARRIER\(...\))71 b(MPI)p 1386 95 V 16 w(BARRIER)13 b(guaran)o(tees)1303 145 y(the)i(\014nish)f(of)f (the)i(PUT)f(thread)795 195 y(ccc)i(=)e(bu\013)322 b(ccc:=reg)p 1458 195 V 17 w(A)166 290 y(In)12 b(this)f(example,)f(v)n(ariable)h (bu\013)h(is)f(allo)q(cated)h(in)f(the)h(register)h(reg)p 1234 290 V 15 w(A)f(while)f Fk(MPI)p 1473 290 V 14 w(SEND)h Fn(and)f Fk(MPI)p 1759 290 V 15 w(W)m(AIT)75 339 y Fn(are)j(executed,)i (and)d(therefore)j(ccc)f(will)d(ha)o(v)o(e)i(the)g(old)g(v)n(alue)f(of) g(bu\013)h(and)g(not)g(the)g(new)h(v)n(alue)e(777.)166 389 y(The)h(next)h(example)d(sho)o(ws)i(extreme,)g(but)g(allo)o(w)o(ed) f(p)q(ossibilities!)288 484 y(Source)388 b(compiled)13 b(as)295 b(or)14 b(compiled)e(as)288 534 y(call)h(xxx\(bu\013,id\))216 b(call)14 b(xxx\(bu\013,id\))216 b(call)13 b(xxx\(bu\013,id\))795 584 y(reg)p 852 584 V 16 w(A)h(=)g(bu\013)276 b(b1)14 b(=)g(bu\013)288 634 y(call)f(yyy\(a,id\))265 b(call)14 b(yyy\(a,id\))265 b(call)13 b(yyy\(a,id\))288 683 y(b1)g(=)h(bu\013)334 b(b1)14 b(:=)g(reg)p 968 683 V 15 w(A)166 778 y(If)h(yyy\(a,id\))f(is)h (able)f(to)h(mo)q(dify)e(bu\013,)i(either)h(b)q(ecause)h(a)e(passes)h (the)g(address)g(of)f(bu\013)g(or)g(b)q(ecause)i(xxx)75 828 y(retained)i(the)g(address)g(of)f(bu\013)g(and)g(id)g(is)g(the)h(k) o(ey)f(whic)o(h)g(iden)o(ti\014es)h(bu\013)f(in)g(yyy)m(,)g(the)h (compiler)d(will)h(not)75 878 y(recognize)g(this)f(p)q(ossible)g(mo)q (di\014cation.)21 b(It)16 b(has)g(no)g(reason)g(to)g(a)o(v)o(oid)e (using)i(a)f(register)i(to)f(hold)f(bu\013)h(across)75 928 y(the)g(call)e(to)h(yyy)m(.)21 b(It)15 b(also)g(can)g(reorder)i (the)f(instructions)f(as)h(in)e(the)i(left)f(case.)23 b(This)15 b(example)f(is)h(v)o(ery)g(m)o(uc)o(h)75 978 y(lik)o(e)e(a)h(non)o(blo)q(c)o(king)e(comm)o(unication)f(and)i(a)h(w)o (ait.)166 1028 y(T)m(o)h(prev)o(en)o(t)j(instruction)e(reordering)h(or) f(the)h(allo)q(cation)e(of)g(a)h(v)n(ariable)f(or)h(bu\013er)i(in)d(a)h (register,)i(there)75 1077 y(are)c(three)h(p)q(ossibilities)f(in)f(F)m (ortran:)145 1151 y Fb(\017)23 b Fn(One)14 b(can)g(put)f(the)h(v)n (ariables)f(and)g(bu\013ers)i(in)o(to)e(common)e(blo)q(c)o(ks;)i(then)h (they)g(will)e(b)q(e)i(allo)q(cated)f(in)g(the)189 1201 y(memory)e(while)i(an)o(y)h(subroutine)g(\(e.g.)k(an)c(MPI)g(routine\)) g(is)g(called,)f(or)145 1274 y Fb(\017)23 b Fn(one)17 b(can)g(declare)h(them)f(as)g(V)o(OLA)m(TILE,)g(but)g(this)g(is)g(not)g (part)g(of)g(the)h(standards)g(F)m(ortran)f(77)f(and)189 1324 y(90,)c(and)i(y)o(our)g(co)q(de)g(will)f(not)g(b)q(e)i(p)q (ortable)f(b)q(ecause)h(there)g(are)g(compilers)d(without)i(V)o(OLA)m (TILE,)f(and)189 1374 y(normally)e(it)i(prev)o(en)o(ts)j(an)o(y)d (optimization,)e(or)145 1447 y Fb(\017)23 b Fn(one)15 b(can)g(add)g(the)g(v)n(ariable)f(or)h(bu\013er)h(as)f(a)f Fe(senseless)h Fn(additional)e(actual,)i(but)g(un)o(used,)g(argumen)o (t)f(at)189 1497 y(the)k(end)g(of)g(the)g(argumen)o(t)f(list;)i(this)f (will)e(cause)j(that)f(the)g(v)n(ariable)f(or)h(bu\013er)h(is)f(allo)q (cated)f(in)g(the)189 1547 y(memory)11 b(while)i(that)h(subroutine)h (is)f(executed.)166 1620 y(In)19 b(C,)g(subroutines)h(whic)o(h)f(mo)q (dify)e(v)n(ariables)h(that)h(are)h(not)f(in)g(the)h(parameter)e(list)h (will)f(not)h(cause)75 1670 y(register)e(optimization)c(problems.)23 b(This)16 b(is)g(b)q(ecause)h(taking)e(p)q(oin)o(ters)i(to)f(storage)g (ob)r(jects)h(b)o(y)f(using)f(the)i(&)75 1720 y(op)q(erator)d(and)g (later)f(referencing)j(the)e(ob)r(jects)h(b)o(y)e(w)o(a)o(y)g(of)g(the) h(p)q(oin)o(ter)g(is)g(an)f(in)o(tegral)g(part)h(of)f(the)h(language.) 75 1769 y(A)e(C)g(compiler)f(understands)j(the)e(implications,)e(but)i (with)g(aggressiv)o(e)g(optimization)e(compilers)h(can)h(p)q(ossibly)75 1819 y(generate)j(incorrect)g(co)q(de.)75 1935 y Fo(9.8.2)49 b(Caching)75 2011 y Fn(A)o(t)14 b(least)g(there)h(are)g(three)g (classes)g(of)e(cac)o(he)i(coherency)h(strategies)f(for)e(m)o(ultipro)q (cessor)h(systems:)145 2085 y Fb(\017)23 b Fn(cac)o(he)12 b(coheren)o(t)g({)f(all)f(memory)f(reads)j(and)e(writes)i(whic)o(h)f (access)i(a)e(memory)e(from)g(m)o(ultiple)g(pro)q(cessors)189 2135 y(are)14 b(immedia)o(tely)d(visible)j(to)f(other)i(pro)q(cessors.) 145 2208 y Fb(\017)23 b Fn(write-through)11 b(incoheren)o(t)h(cac)o (hes)g({)e(all)g(memory)e(writes)k(go)e(directly)h(to)g(memory)m(,)d (but)j(memory)d(reads)189 2258 y(are)14 b(cac)o(hed.)145 2331 y Fb(\017)23 b Fn(bac)o(k-write)11 b(or)g(non-write-through)g (incoheren)o(t)i(cac)o(hes)f({)f(memory)e(writes)j(are)f(stored)i(in)d (a)h(pro)q(cessor's)189 2381 y(cac)o(he)j(and)g(\015ushed)h(to)f (memory)d(at)j(a)g(later)g(time.)166 2455 y(Examples)19 b(for)g(systems)h(with)g(write-through)g(cac)o(hes)h(are)f(the)h(Cra)o (y)e(T90)h(and)f(the)i(Cra)o(y)e(T3D,)g(an)75 2504 y(example)13 b(for)g(bac)o(k-write)h(cac)o(hes)h(is)f(the)h(RA)o(CE)e(i860)g(from)f (Mercury)j(Computer)e(Systems.)166 2554 y(There)i(are)f(t)o(w)o(o)f (groups)h(of)f(MPI)h(functions)g(that)f(can)h(cause)h(problems)e(with)g (cac)o(he)i(incoherency)m(.)k(First)75 2604 y(non-blo)q(c)o(king)14 b(send)h(and)g(receiv)o(e)h(op)q(erations)f(can)g(also)f(cause)i (problems)e(with)h(cac)o(he)h(incoherence.)22 b(That)15 b(is)75 2654 y(one)e(of)f(the)h(reasons)h(wh)o(y)e(the)h(standard)g (explicitly)f(disallo)o(ws)f(access)j(to)f(the)g(send)g(and)g(receiv)o (e)h(bu\013er)f(during)75 2704 y(the)j(non-blo)q(c)o(king)e(op)q (eration.)22 b(The)16 b(second)g(group)g(is)f(the)h(the)g(one-sided)g (comm)o(uni)o(cation)c(with)j Fk(MPI)p 1779 2704 V 15 w(PUT)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 23 24 bop 75 -100 a Fp(9.8.)34 b(PR)o(OBLEMS)16 b(WITH)f(REGISTER)h (OPTIMIZA)l(TION)h(AND)e(CA)o(CHING)275 b Fv(23)75 45 y Fn(and)12 b Fk(MPI)p 231 45 13 2 v 14 w(GET)f Fn(called)h(on)f(a)h (origin)e(no)q(de)i(to)g(write)g(or)f(read)i(the)f(memory)d(on)j(the)g (target)g(no)q(de.)18 b(Problems)11 b(can)75 95 y(arise)16 b(if)e(there)j(is)e(no)g(cac)o(he)i(coherency)g(b)q(et)o(w)o(een)f(the) g(memory)d(accesses)18 b(b)o(y)d(the)h(application)e(on)i(the)f(target) 75 145 y(no)q(de)e(and)f(b)o(y)g(the)h(PUT)g(and)f(GET)g(op)q(erations) h(started)g(b)o(y)g(the)g(application)e(on)h(the)h(origin)e(no)q(de.)18 b(There)c(are)75 195 y(three)h(cases:)145 266 y Fb(\017)23 b Fn(Data)9 b(from)g(the)i(application)e(on)h(the)h(target)g(no)q(de)g (is)g Fl(not)e Fn(stored)i(in)o(to)f(the)h(memory)d(as)j(so)q(on)f(as)h (p)q(ossible)189 316 y(and)16 b(subsequen)o(t)i(accesses)g(to)f(that)f (memory)e(b)o(y)i Fk(MPI)p 1092 316 V 14 w(GET)g Fn(issued)h(from)d (the)j(origin)e(no)q(de)i(will)e(giv)o(e)189 365 y(older)f(v)n(alues.) 242 452 y Fl({)23 b Fk(Prop)q(osal)14 b(1.1:)20 b Fn(A)c(call)e(to)h Fk(MPI)p 793 452 V 15 w(RMA)p 899 452 V 14 w(WINDO)o(W)p 1092 452 V 15 w(READ)o(Y)f Fn(guaran)o(tees)i(that)g(all)e(cac)o(hed)i (data)f(is)289 502 y(written)e(to)g(the)h(memory)m(.)h Fk(MPI)p 804 502 V 14 w(RMA)p 909 502 V 15 w(WINDO)o(W)p 1103 502 V 15 w(READ)o(Y)c Fn(m)o(ust)i(b)q(e)g(called)g(on)g(the)h (target)g(no)q(de)289 551 y(after)h(data)g(is)g(written)g(to)g(the)h (memory)c(lo)q(cally)i(on)h(the)g(target)h(no)q(de)f(and)g(b)q(efore)h (the)f(memory)289 601 y(on)f(the)g(target)h(no)q(de)f(is)g(accessed)j (with)d Fk(MPI)p 1020 601 V 14 w(GET)g Fn(from)e(the)j(origin)e(no)q (de.)19 b(Load)14 b(access)h(b)o(y)f(the)289 651 y(application)e(on)i (the)g(target)h(system)e(is)h(allo)o(w)o(ed)f(in)g(parallel)g(to)g(the) i(accesses)h(b)o(y)e Fk(MPI)p 1692 651 V 14 w(GET)p Fn(.)289 707 y Fl(Discussion)o(:)32 b Fn(One)13 b(di\016cult)o(y)d(with)i(this)f (approac)o(h)h(is)f(that)h(the)g(problem)f(of)g(sync)o(hronizing)g(the) 289 757 y(call)i(to)h(MPI)p 499 757 V 15 w(RMA)p 614 757 V 15 w(WINDO)o(W)p 824 757 V 14 w(READ)o(Y)f(on)h(one)g(no)q(de)h (with)e(the)i(MPI)p 1485 757 V 15 w(GET)f(call)f(on)h(another)289 807 y(no)q(de)j(is)g(left)f(to)h(the)g(user)h(unless)g(a)e(matc)o(hing) f(MPI)p 1161 807 V 16 w(RMA)p 1277 807 V 14 w(WINDO)o(W)p 1486 807 V 15 w(BUSY)i(call)f(lo)q(c)o(ks)h(out)289 857 y(\(dela)o(ys\))12 b(an)o(y)g(MPI)p 604 857 V 15 w(GET)g(requests)i (that)f(come)e(in.)17 b(A)c(call)e(to)h(MPI)p 1386 857 V 15 w(RMA)p 1501 857 V 15 w(WINDO)o(W)p 1711 857 V 14 w(READ)o(Y)289 907 y(w)o(ould)h(unlo)q(c)o(k)g(the)i(windo)o(w,)d(allo) o(wing)g(MPI)p 1022 907 V 15 w(GETs)i(to)g(con)o(tin)o(ue.)242 969 y Fl({)23 b Fk(Prop)q(osal)16 b(1.2:)22 b Fn(Disallo)o(w)14 b(lo)q(cal)i(stores)i(to)e(a)g(windo)o(w)g(b)q(et)o(w)o(een)i(sync)o (hronization)e(p)q(oin)o(ts)h(if)e(an)o(y)289 1019 y(RMA)10 b(op)q(eration)h(from)e(a)h(di\013eren)o(t)i(no)q(de)g(is)e(reading)h (data)f(from)f(the)j(windo)o(w.)k(Disallo)o(w)o(ed)9 b(RMA)289 1069 y(op)q(erations)14 b(include)g(GET,)f(RMW,)g(and)g(A)o (CCUMULA)m(TE.)289 1125 y(A)h(\\sync)o(hronization)f(p)q(oin)o(t")g(is) h(de\014ned)h(as)f(an)o(y)f(MPI)i(1-sided)f(or)f(2-sided)h(comm)o (unication)d(call)289 1175 y(on)17 b(an)o(y)g(comm)o(unicator)e(\(for)j (an)o(y)f(windo)o(w\).)29 b(This)17 b(de\014nition)h(is)f(made)g(with)g (a)h(broad)f(brush)289 1225 y(strok)o(e)12 b(to)f(allo)o(w)f(maxim)n (um)d(user)13 b(\015exibilit)o(y)m(.)i(F)m(or)c(example,)f(a)h(windo)o (w)g(migh)o(t)e(b)q(e)j(set)h(up)e(for)g(an)o(y)289 1275 y(no)q(de)j(to)f(access,)i(but)f(during)g(a)f(particular)g(p)q(oin)o(t) h(in)f(the)h(computation)e(just)i(one)g(pro)q(ducer)h(and)289 1325 y(one)h(consumer)g(m)o(ust)f(sync)o(hronize)j(lo)q(cal)d(\(via)g (ordinary)h(stores\))i(and)e(remote)f(\(via)h Fk(MPI)p 1766 1325 V 14 w(GET)p Fn(\))289 1375 y(up)q(dates)f(to)e(the)i(windo)o (w.)289 1431 y(Lo)q(cal)e(loads)g(ma)o(y)f(o)q(ccur)j(sim)o (ultaneously)d(with)i(remote)f Fk(MPI)p 1301 1431 V 14 w(GET)h Fn(op)q(erations)g(on)g(a)f(windo)o(w.)289 1487 y Fl(Discussion)o(:)59 b Fn(Because)21 b(the)g(MPI)f(implemen)o(tation) c(can)21 b(not)e(see,)k(whic)o(h)c(comm)o(unication)289 1537 y(is)f(used)h(as)f(a)g(\\sync)o(hronization)g(p)q(oin)o(t",)g(it)g (alw)o(a)o(ys)f(m)o(ust)h(issue)h(a)f(write-bac)o(k)g(of)g(the)g(cac)o (he.)289 1587 y(Therefore)d(probably)e(a)h(mixture)e(of)i(prop)q(osal)f (1.1)g(and)h(1.2)f(is)g(needed.)242 1650 y Fl({)23 b Fk(Prop)q(osal)14 b(1.3:)k Fn(This)c(case)h(is)f Fl(not)g Fn(addressed)i(b)o(y)e(MPI)h(and)f(the)h(user)g(is)f(resp)q(onsible)i (to)e(prev)o(en)o(t)289 1700 y(this.)25 b(Programs)15 b(using)h Fk(MPI)p 771 1700 V 15 w(GET)g Fn(are)g(therefore)i(not)e(p)q (ortable.)26 b(The)17 b(do)q(cumen)o(tation)e(of)g(the)289 1749 y(MPI)f(implemen)o(tati)o(on)d(m)o(ust)i(include)h(hin)o(ts,)f(ho) o(w)h(the)g(user)h(can)f(solv)o(e)g(this)g(problem.)242 1812 y Fl({)23 b Fk(Prop)q(osal)9 b(1.4:)15 b Fn(Do)9 b(not)h(address)h(this)f(issue)g(in)f(the)i(MPI)f(standard)g(and)g(lea) o(v)o(e)f(the)h(implemen)o(tation)289 1862 y(to)j(prev)o(en)o(t)i(the)g (problem)d(from)g(o)q(ccurring.)289 1918 y Fl(Discussion)o(:)50 b Fn(The)18 b(a\013ected)h(systems)f(w)o(ould)f(su\013er)i(a)e(v)o(ery) h(high)f(p)q(erformance)h(p)q(enalt)o(y)f(in)289 1968 y(the)d(implemen)o(tation)c(of)k(GET,)f(RMW,)f(and)i(A)o(CCUMULA)m(TE)g (op)q(erations.)k(System)c(calls)f(and)289 2018 y(signal)g(handlers)i (lik)o(ely)e(w)o(ould)h(ha)o(v)o(e)g(to)g(b)q(e)h(in)o(v)o(ok)o(ed)f (instead)h(of)f(simply)e(transferring)j(data)f(and)289 2068 y(p)q(erhaps)h(lo)q(c)o(king/unlo)q(c)o(king)c(memory)h(areas.)145 2154 y Fb(\017)23 b Fn(Data)c(from)f(the)j(application)d(on)i(the)h (target)f(no)q(de)h(is)e Fl(not)g Fn(stored)i(in)o(to)f(the)g(memory)e (as)i(so)q(on)g(as)189 2204 y(p)q(ossible)14 b(and)h(subsequen)o(t)h (stores)f(to)g(that)f(memory)e(b)o(y)i Fk(MPI)p 1196 2204 V 15 w(PUT)f Fn(issued)j(from)c(the)j(origin)f(no)q(de)h(are)189 2254 y(done)10 b(b)q(efore)i(the)f(application)e(data)h(is)h(written)g (from)e(the)i(cac)o(he)g(to)g(the)g(memory)m(.)j(The)d(data)f(written)h (b)o(y)189 2304 y Fk(MPI)p 266 2304 V 14 w(PUT)h Fn(is)g(therefore)i(o) o(v)o(erwritten)g(when)f(the)g(lo)q(cal)f(store)h(data)f(is)h(later)g (\015ushed)g(to)g(memory)m(.)h(\(Note)189 2354 y(that)f(previously)g (data)g(written)h(lo)q(cally)e(b)o(y)h Fk(MPI)p 980 2354 V 15 w(PUT)f Fn(can)i(b)q(e)g(incorrectly)g(o)o(v)o(erwritten)g(b)o(y)f (\015ushes)i(to)189 2403 y(memory)c(of)i(cac)o(he)i(lines)f(con)o (taining)f(lo)q(cally)f(stored)j(data)f(to)q(o\).)242 2490 y Fl({)23 b Fk(Prop)q(osal)14 b(2.1:)20 b Fn(A)c(call)e(to)h Fk(MPI)p 793 2490 V 15 w(RMA)p 899 2490 V 14 w(WINDO)o(W)p 1092 2490 V 15 w(READ)o(Y)f Fn(guaran)o(tees)i(that)g(all)e(cac)o(hed)i (data)f(is)289 2540 y(written)e(to)g(the)h(memory)m(.)h Fk(MPI)p 804 2540 V 14 w(RMA)p 909 2540 V 15 w(WINDO)o(W)p 1103 2540 V 15 w(READ)o(Y)c Fn(m)o(ust)i(b)q(e)g(called)g(on)g(the)h (target)g(no)q(de)289 2589 y(after)i(the)h(memory)d(is)i(used)h(lo)q (cally)e(on)h(the)h(target)g(no)q(de)g(and)f(b)q(efore)h Fk(MPI)p 1550 2589 V 14 w(PUT)f Fn(is)g(issued)h(on)289 2639 y(the)d(origin)f(no)q(de)h(to)g(write)g(data)g(in)o(to)f(the)h (memory)e(of)h(the)h(target)h(no)q(de.)289 2696 y Fl(Discussion)o(:)34 b Fn(The)14 b(same)f(di\016culties)g(noted)h(under)h(prop)q(osal)e(1.1) g(apply)g(here)h(to)g Fk(MPI)p 1741 2696 V 14 w(PUT)p Fn(.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 24 25 bop 75 -100 a Fv(24)1133 b Fp(CHAPTER)15 b(9.)35 b(MISCELLANY)242 45 y Fl({)23 b Fk(Prop)q(osal)16 b(2.2:)22 b Fn(Disallo)o(w)14 b(lo)q(cal)i(stores)i(to)e(a)g(windo)o(w)g(b)q(et)o (w)o(een)i(sync)o(hronization)e(p)q(oin)o(ts)h(if)e(an)o(y)289 95 y(RMA)9 b(op)q(eration)g(from)e(a)i(di\013eren)o(t)i(no)q(de)e(is)g (up)q(dating)g(data)g(from)f(the)i(same)e(windo)o(w.)16 b(Disallo)o(w)o(ed)289 145 y(RMA)d(op)q(erations)h(include)g(PUT,)g (RMW,)f(and)g(A)o(CCUMULA)m(TE.)289 203 y Fl(Discussion)o(:)35 b Fn(See)14 b(discussion)h(under)f(prop)q(osal)g(1.2.)242 269 y Fl({)23 b Fk(Prop)q(osal)14 b(2.3:)k Fn(This)c(case)h(is)f Fl(not)g Fn(addressed)i(b)o(y)e(MPI)h(and)f(the)h(user)g(is)f(resp)q (onsible)i(to)e(prev)o(en)o(t)289 318 y(this.)24 b(Programs)16 b(using)f Fk(MPI)p 770 318 13 2 v 15 w(PUT)g Fn(are)i(therefore)g(not)f (p)q(ortable.)25 b(The)17 b(do)q(cumen)o(tation)e(of)g(the)289 368 y(MPI)f(implemen)o(tati)o(on)d(m)o(ust)i(include)h(hin)o(ts,)f(ho)o (w)h(the)g(user)h(can)f(solv)o(e)g(this)g(problem.)242 434 y Fl({)23 b Fk(Prop)q(osal)9 b(2.4:)15 b Fn(Do)9 b(not)h(address)h(this)f(issue)g(in)f(the)i(MPI)f(standard)g(and)g(lea) o(v)o(e)f(the)h(implemen)o(tation)289 484 y(to)j(prev)o(en)o(t)i(the)g (problem)d(from)g(o)q(ccurring.)289 542 y Fl(Discussion)o(:)43 b Fn(The)17 b(same)e(sev)o(ere)i(p)q(erformance)f(p)q(enalt)o(y)g (noted)h(under)g(prop)q(osal)e(1.4)g(applies)289 592 y(here)g(to)e Fk(MPI)p 506 592 V 15 w(PUT)g Fn(as)h(w)o(ell)f(for)h (the)g(systems)g(a\013ected.)145 684 y Fb(\017)23 b Fn(Data)11 b(from)g(the)i(application)e(on)h(the)h(target)f(no)q(de)h(is)f(stored) i(in)o(to)d(the)i(memory)d Fl(as)k(so)q(on)g(as)g(p)q(ossible)189 734 y Fn(and)e(subsequen)o(t)j(stores)e(to)g(that)g(memory)d(b)o(y)i Fk(MPI)p 1027 734 V 15 w(PUT)f Fn(issued)j(from)d(the)i(origin)f(no)q (de)h(are)g(done)g(also)189 784 y Fl(as)f(so)q(on)h(as)f(p)q(ossible)d Fn(but)i(subsequen)o(t)i(loads)d(from)g(that)h(memory)e(b)o(y)h(the)i (application)e(on)h(the)g(target)189 834 y(no)q(de)j(is)f(done)h(b)o(y) g(using)f(the)i(older)e(data)h(in)f(the)h(cac)o(he)h(that)f(w)o(as)f (written)h(when)h(the)f(data)f(w)o(as)h(stored)189 883 y(b)o(y)f(the)i(application)d(on)i(the)g(target)h(no)q(de.)242 976 y Fl({)23 b Fk(Prop)q(osal)12 b(3.1:)k Fn(A)d(call)f(to)g Fk(MPI)p 779 976 V 15 w(DELIVER)f Fn(\015ushes)j(the)g(cac)o(hes)g(and) e(guaran)o(tees)i(that)e(subsequen)o(t)289 1026 y(loads)g(are)h(fetc)o (hed)h(from)d(the)i(actual)g(memory)d(and)j(not)g(from)e(old)h(cac)o (hes.)19 b Fk(MPI)p 1593 1026 V 14 w(DELIVER)12 b Fn(m)o(ust)289 1076 y(b)q(e)17 b(called)f(on)g(the)h(target)g(no)q(de)g(after)g(the)g (memory)d(is)j(up)q(dated)g(b)o(y)f(\(some\))g Fk(MPI)p 1654 1076 V 14 w(PUT)g Fn(issued)289 1125 y(from)h(one)h(or)h(more)f (origin)f(no)q(des.)33 b(The)19 b(application)e(m)o(ust)h(guaran)o (tee,)i(e.g.)32 b(b)o(y)18 b(lo)q(oking)f(to)289 1175 y(the)e(windo)o(w)f(coun)o(ter,)h(that)g(there)h(is)e(no)h(ongoing)e Fk(MPI)p 1205 1175 V 14 w(PUT)h Fn(to)h(that)f(windo)o(w.)20 b(Before)15 b(calling)289 1225 y Fk(MPI)p 366 1225 V 14 w(DELIVER)e Fn(the)h(application)f(on)h(the)g(target)h(no)q(de)f(m)o (ust)f(not)h(access)i(the)e(windo)o(w)f(memory)m(.)289 1275 y(After)j(calling)d Fk(MPI)p 608 1275 V 15 w(DELIVER)h Fn(the)i(application)e(on)h(the)h(target)g(host)f(ma)o(y)e(access)18 b(the)d(memory)289 1325 y(again)d(but)i(the)h(application)d(on)i(the)h (origin)d(no)q(de)j(m)o(ust)d(not)i(issue)h(further)g Fk(MPI)p 1602 1325 V 14 w(PUT)p Fn(s.)289 1383 y Fl(Discussion)o(:)54 b Fn(This)18 b(prop)q(osal)g(m)o(ust)g(b)q(e)h(\015eshed)h(out)f (further)g(to)g(de\014ne)h(whether)g(the)f(user)289 1432 y(m)o(ust)12 b(sync)o(hronize)j(origin)e(and)g(target)i(with)e (separate)i(MPI)f(sync)o(hronization)g(calls)f(or)h(whether)289 1482 y Fk(MPI)p 366 1482 V 14 w(DELIVER)k Fn(w)o(ould)h(set)i(a)e(lo)q (c)o(k)g(whic)o(h)g(w)o(ould)g(blo)q(c)o(k)g(further)h Fk(MPI)p 1483 1482 V 15 w(PUT)p Fn(s)f(un)o(til)f(the)i(next)289 1532 y(sync)o(hronization)13 b(p)q(oin)o(t.)242 1598 y Fl({)23 b Fk(Prop)q(osal)13 b(3.2:)18 b Fn(A)c(call)f(to)h Fk(MPI)p 786 1598 V 14 w(DELIVER)f Fn(\015ushes)j(the)e(cac)o(he)h(and) f(guaran)o(tees)h(that)f(subsequen)o(t)289 1648 y(loads)g(can)i(access) h(all)d(data)h(deliv)o(ered)g(b)o(y)g(RMA)g(requests)j(to)d(that)g (target)h(prior)f(to)g(the)h(call)e(to)289 1698 y Fk(MPI)p 366 1698 V 14 w(DELIVER)p Fn(.)289 1755 y(MPI)p 372 1755 V 15 w(DELIVER)i(m)o(ust)g(b)q(e)h(called)f(on)g(the)h(target)g(no)q (de)g(to)f(ensure)i(complete)e(deliv)o(ery)h(of)e(an)o(y)289 1805 y(data)f(written)i(to)f(the)h(target)g(b)o(y)f(prior)g(RMA)g (requests)i(originated)d(b)o(y)h(other)h(no)q(des.)23 b(Without)289 1855 y(a)13 b(call)g(to)g Fk(MPI)p 526 1855 V 14 w(DELIVER)p Fn(,)g(it)g(is)g(unde\014ned)i(whether)g(an)o(y)e (lo)q(cal)g(loads)g(of)g(data)g(written)h(b)o(y)f(prior)289 1905 y(RMA)g(op)q(erations)h(access)i(the)f(new)f(or)g(the)g(old)f (memory)f(con)o(ten)o(ts.)289 1963 y(An)d Fk(MPI)p 429 1963 V 14 w(DELIVER)g Fn(op)q(eration)g(is)g(implied)e(b)o(y)i(an)o(y)g (MPI)g(sync)o(hronization)h(routine)f({)g Fk(MPI)p 1736 1963 V 14 w(BARRIER)p Fn(,)289 2013 y Fk(MPI)p 366 2013 V 14 w(RECK)p Fn(,)j Fk(RMW)p Fn(,)i(and)g Fk(A)o(CCUMULA)m(TE)d Fn(requests.)289 2070 y(With)i(this)h(prop)q(osal,)f(no)h(sync)o (hronization)g(is)f(really)h(needed)h(b)q(et)o(w)o(een)g(origin)e(and)h (target.)19 b(The)289 2120 y(most)12 b(t)o(ypical)h(use)i(of)e(this)h (w)o(ould)f(b)q(e)i(when)f(the)h(target)f(no)q(de)g(is)g(spinning)f(on) h(memory:)485 2209 y Fd(volatile)20 b(int)h(windowarray[SIZE];)485 2308 y(while)g(\(windowarray[0])d(==)k(0\))f({)659 2358 y(mpi_deliver\(\);)485 2408 y(})289 2546 y Fn(On)c(systems)g(with)g (coheren)o(t)h(cac)o(hes,)h Fk(MPI)p 1007 2546 V 14 w(DELIVER)p Fn(,)d(could)h(b)q(e)h(de\014ned)g(as)f(a)g(n)o(ull)f(macro)f(in)289 2596 y(mpi.h.)g(Therefore)h(there)f(is)f(a)f(negligible)g(p)q (erformance)g(p)q(enalt)o(y)h(for)g(una\013ected)h(systems.)289 2654 y Fl(Discussion)o(:)31 b Fn(This)10 b(prop)q(osal)g(w)o(eak)o(ens) h(the)g(progress)g(rule)g(for)e(1-sided)i(comm)o(unicati)o(on.)j(But)d (if)289 2704 y(that's)f(a)g(problem)g(w)o(ould)f(someone)h(explain)g (to)h(me)e(wh)o(y)h(it's)h(a)f(problem?)15 b(I)c(understand)h(that)e (the)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 25 26 bop 75 -100 a Fp(9.8.)34 b(PR)o(OBLEMS)16 b(WITH)f(REGISTER)h (OPTIMIZA)l(TION)h(AND)e(CA)o(CHING)275 b Fv(25)289 45 y Fn(strength)14 b(of)f(the)h(progress)h(rule)f(has)f(b)q(een)i (established)f(b)o(y)f(MPI-1)g(for)g(2-sided)h(comm)o(unicatio)o(n.)289 95 y(But,)g(nothing)f(has)h(b)q(een)h(established)g(or)f(standardized)h (for)e(1-sided)h(comm)o(unication)d(as)j(of)f(y)o(et.)242 211 y Fl({)23 b Fk(Prop)q(osal)14 b(3.3:)k Fn(This)c(case)h(is)f Fl(not)g Fn(addressed)i(b)o(y)e(MPI)h(and)f(the)h(user)g(is)f(resp)q (onsible)i(to)e(prev)o(en)o(t)289 261 y(this.)24 b(Programs)16 b(using)f Fk(MPI)p 770 261 13 2 v 15 w(PUT)g Fn(are)i(therefore)g(not)f (p)q(ortable.)25 b(The)17 b(do)q(cumen)o(tation)e(of)g(the)289 311 y(MPI)f(implemen)o(tati)o(on)d(m)o(ust)i(include)h(hin)o(ts,)f(ho)o (w)h(the)g(user)h(can)f(solv)o(e)g(this)g(problem.)242 377 y Fl({)23 b Fk(Prop)q(osal)9 b(3.4:)15 b Fn(Do)9 b(not)h(address)h(this)f(issue)g(in)f(the)i(MPI)f(standard)g(and)g(lea) o(v)o(e)f(the)h(implemen)o(tation)289 427 y(to)j(prev)o(en)o(t)i(the)g (problem)d(from)g(o)q(ccurring.)289 485 y Fl(Discussion)o(:)35 b Fn(The)15 b(same)e(sev)o(ere)j(p)q(erformance)e(p)q(enalt)o(y)g (noted)g(under)h(prop)q(osals)f(1.4)f(and)h(2.4)289 535 y(applies)f(here)i(to)f Fk(MPI)p 645 535 V 14 w(PUT)f Fn(for)h(the)h(systems)f(a\013ected.)289 593 y(This)f(prop)q(osal)h (lea)o(v)o(es)g(the)g(progress)i(rule)e(strongly)f(in)o(tact.)145 686 y Fb(\017)23 b Fn(Memory)12 b(corruption)j(at)e(windo)o(w)g(b)q (oundaries.)189 749 y(There)f(are)f(t)o(w)o(o)f(similar)e(issues)k (whic)o(h)f(can)g(result)g(in)g(memory)d(coherency)13 b(problems)c(at)i(the)g(b)q(oundaries)189 799 y(of)i(1-sided)h(comm)o (unicatio)o(n)d(windo)o(ws:)234 879 y(1.)22 b(w)o(ord)13 b(size)234 946 y(2.)22 b(cac)o(he)14 b(line)g(size)189 1027 y(The)k(problem)f(is)h(most)g(easily)f(illustrated)h(with)g(a)g (windo)o(w)g(con)o(taining)f(one)i(b)o(yte)f(whic)o(h)g(is)h(in)e(the) 189 1077 y(middle)12 b(of)h(a)h(w)o(ord.)k(The)c(b)o(ytes)h(on)f(b)q (oth)g(sides)h(are)f(not)g(in)f(the)i(windo)o(w)e(and)h(so)g(ma)o(y)e (b)q(e)j(up)q(dated)f(b)o(y)189 1126 y(a)e(no)q(de)h(while)f(remote)g Fk(MPI)p 642 1126 V 14 w(PUT)g Fn(op)q(erations)h(are)g(up)q(dating)f (this)g(same)g(b)o(yte.)18 b(On)13 b(systems)g(that)f(ha)o(v)o(e)189 1176 y(no)h(b)o(yte)h(store)h(instruction,)e(the)h(w)o(ord)g(m)o(ust)e (b)q(e)i(loaded,)f(and)h(then)g(the)g(b)o(yte)g(inserted)h(in)o(to)e (the)h(w)o(ord)189 1226 y(b)q(efore)g(storing)g(the)g(en)o(tire)h(w)o (ord)f(bac)o(k)g(to)f(memory)m(.)189 1288 y(A)k(similar)d(issue)k (exists)f(for)g(non-cac)o(he-coheren)o(t)i(systems)e(when)g(a)g(windo)o (w)f(b)q(oundary)h(falls)f(in)g(the)189 1338 y(middle)c(of)h(a)h(cac)o (he)g(line.)189 1400 y(This)h(problem)g(o)q(ccors)i(on)f(an)o(y)f (system)h(that)f(do)q(es)i(not)f(ha)o(v)o(e)f(a)h(b)o(yte-store)h (instruction.)24 b(Net)o(w)o(ork)o(ed)189 1450 y(systems)10 b(and)g(shared)h(memory)d(systems)i(alik)o(e)f(are)i(a\013ected.)18 b(Also)10 b(a\013ected)i(are)e(m)o(ultipro)q(cessor)g(shared)189 1500 y(memory)h(systems)j(without)g(full)e(cac)o(he)j(coherency)h (implemen)o(ted)11 b(in)j(hardw)o(are.)242 1593 y Fl({)23 b Fk(Prop)q(osal)18 b(4.1:)25 b Fn(Require)19 b(the)g(implemen)o(tatio) o(n)d(to)i(handle)g(the)h(issue)g(b)o(y)f(recognizing)g(when)h(a)289 1643 y(PUT)14 b(is)g(b)q(eing)f(done)i(at)e(the)i(b)q(oundary)f(of)f(a) h(windo)o(w.)289 1701 y(Although)f(the)h(b)q(oundary-within-a-cac)o (he-line)f(problem)g(is)h(solv)n(able)e(this)i(w)o(a)o(y)m(,)f(the)h(b) q(oundary-)289 1751 y(inside-a-w)o(ord)f(case)i(is)e(not)h(solv)n (able.)242 1818 y Fl({)23 b Fk(Prop)q(osal)10 b(4.2:)15 b Fn(Remo)o(v)o(e)8 b Fk(MPI)p 760 1818 V 15 w(RMA)p 866 1818 V 15 w(INIT)i Fn(from)e(the)j(standard.)17 b(Require)11 b(that)f(all)f(RMA)h(windo)o(ws)289 1867 y(b)q(e)19 b(allo)q(cated)g(b) o(y)g Fk(MPI)p 672 1867 V 14 w(RMA)p 777 1867 V 15 w(MALLOC)p Fn(.)g(The)h(latter)f(routine)g(could)g(adequately)h(ensure)g(an)o(y) 289 1917 y(needed)15 b(b)q(oundary)f(alignmen)o(t)d(for)j(a)g(giv)o(en) f(system.)242 1984 y Fl({)23 b Fk(Prop)q(osal)c(4.3:)30 b Fn(Keep)21 b Fk(MPI)p 744 1984 V 15 w(RMA)p 850 1984 V 15 w(INIT)e Fn(in)h(the)h(standard,)h(but)e(allo)o(w)f(it)h(to)g (return)h(an)f(error)289 2033 y(co)q(de)15 b(if)g(a)f(user-requested)k (windo)o(w)d(do)q(es)g(not)g(meet)g(an)g(implem)o(en)o(tation's)d (windo)o(w)i(alignmen)o(t)289 2083 y(requiremen)o(ts.)20 b(This)14 b(w)o(ould)g(ensure)i(that)e(an)o(y)g(windo)o(w)g (successfully)h(established)h(w)o(ould)d(a)o(v)o(oid)289 2133 y(the)j(problem.)24 b(But)17 b(the)g(collectiv)o(e)f Fk(MPI)p 965 2133 V 14 w(RMA)p 1070 2133 V 15 w(INIT)g Fn(call)g(w)o(ould)f(ha)o(v)o(e)h(to)g(fail)f(on)h Fl(all)f Fn(no)q(des)i(if)289 2183 y(an)o(y)c(of)g(the)i(no)q(des)g(passed)f(in) g(a)f(bu\013er)i(with)f(improp)q(er)f(alignmen)o(t)e(or)j(length.)289 2241 y(W)m(e)9 b(could)g(also)g(pro)o(vide)h(some)e(constan)o(t)i(e.g.) 17 b Fk(MPI)p 1104 2241 V 14 w(WINDO)o(W)p 1297 2241 V 15 w(ALIGN)p 1431 2241 V 14 w(SIZE)10 b Fn(whic)o(h)f(w)o(ould)g (de\014ne)289 2291 y(the)h(alignmen)o(t)d(requiremen)o(ts)j(\(in)f(b)o (ytes\))i(for)e(the)h(starting)g(lo)q(cation)e(and)i(the)g(length)g(of) e(a)i(windo)o(w)289 2341 y(in)j(a)h(giv)o(en)f(implemen)o(tatio)o(n.)p 75 2434 827 2 v 75 2484 a(The)h(follo)o(wing)e(advice)i(m)o(ust)e(b)q (e)j(added)f(in)75 2545 y(-)g(MPI)g(2,)f(c)o(hapter)i(4.3)e Fk(MPI)p 537 2545 13 2 v 14 w(PUT/GET)p Fn(,)75 2595 y(-)h(MPI)g(2,)f(c)o(hapter)i(4.10)d(Handlers,)75 2645 y(-)i Fk(MPI)p 180 2645 V 14 w(ADDRESS)f Fn(\(MPI)h(1.1,)f(page)h(70,)f (line)g(16\),)g(and)h(in)75 2695 y(-)g Fk(MPI)p 180 2695 V 14 w(IRECV)e Fn(\(MPI)i(1.1,)f(\\Non)o(blo)q(c)o(king)f(comm)o (unication",)e(end)k(of)g(page)g(40\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 26 27 bop 75 -100 a Fv(26)1138 b Fp(CHAPTER)15 b(9.)30 b(MISCELLANY)189 45 y Fe(A)n(dvic)n(e)16 b(to)h(users.)47 b Fn(T)m(o)15 b(prev)o(en)o(t)i(troubles)f(with)f(the)h(register)h (optimization)c(of)i(the)i(F)m(ortran)e(compil-)189 95 y(ers)i(or)f(with)g(cac)o(hing)g(please)h(pa)o(y)f(atten)o(tion)g(to)g (the)h(hin)o(ts)f(in)g(section)h(9.8)f(\\Problems)f(with)h(register)189 145 y(optimization)11 b(and)j(cac)o(hing")f(on)h(page)g(21.)j(\()p Fe(End)f(of)f(advic)n(e)g(to)g(users.)p Fn(\))75 261 y Fo(9.8.3)49 b(Send)p 326 261 15 2 v 17 w(Bu\013er)p 467 261 V 18 w(Init,)16 b(etc.)75 338 y Fn(This)e(is)g(email)d(that)j (is)g(a)f(placeholder)i(for)e(a)h(prop)q(osal)f(from)f(T)m(on)o(y)h (Skjellum:)75 487 y Fd(I)22 b(make)f(the)g(following)f(proposal,)g (probably)g(for)h(the)g(misc)g(chapter.)75 537 y(Chuck)g(Seitz)g(and)g (group)f(had)i(the)f(original)f(idea)h(for)g(these)g(semantics,)f(and)h (I)g(know)75 587 y(that)g(a)h(group)e(at)i(NASA)f(Ames)g(keeps)f(the)i (flame)e(going)h(with)g(their)g(MPK)g(project)75 636 y(and)g(follow)g(ons.)42 b(Anyway,)21 b(this)g(is)g(not)g(something)f (MPI)h(can)h(do,)f(and)g(there)75 686 y(is)g(not)h(reason)e(why)h(we)h (couldn't)e(try)h(to)g(help)g(a)h(broader)e(group)h(of)g(users,)75 736 y(potentially,)e(such)i(as)h(this)f(NASA)g(group.)75 836 y(Anyway,)f(the)i(Caltech)e(Reactive)g(Kernel)h(provided)f(fully)g (buffered)h(operations)75 885 y(and)g(only)g(had)g(send/receive.)41 b(Zipcode)21 b(in)g(turn)g(utilized)f(this)h(for)g(point-to-point,)75 935 y(and)g(used)g(user)g(buffers)f(\(as)i(MPI)f(does\))g(for)g (collective.)e(RK)j(was)f(its)75 985 y(portability)f(layer.)75 1085 y(Early)h(in)g(MPI-1,)g(we)g(abandoned)f(the)h(idea)g(to)h(have)e (RK-like)h(semantics,)75 1135 y(and)g(accepted)f(that)h(there)g(be)g (at)h(least)f(one)g(copy)g(of)g(the)g(data,)g(even)g(in)75 1184 y(a)h(flat)f(shared)f(memory.)64 b(The)21 b(exact)g(reasons)f(are) h(probably)g(known,)f(but)h(I)h(have)f(not)75 1234 y(found)g(them.)42 b(I)22 b(recall)e(the)i(discussion)d(itself)i(vaguely.)85 b(The)22 b(reflector)e(is)75 1284 y(huge)h(:-\))75 1384 y(However,)f(much)h(after)g(that,)g(we)g(have)g(done)g(two)g(key)g (things)g(on)g(the)g(Forum)g(that)75 1433 y(suggest)f(that)h(control)g (of)g(buffering)f(was)h(possibly)f(important)g(to)i(users:)249 1533 y(*)g(Invent)e(a)i(Bsend)f(+)g(user)g(buffer)g(that)g(everyone)f (hates)h(\(PVM)g(porting?\))249 1583 y(*)h(Start)f(talking)f(about)h (special)f(memory)h(for)g(put/get)f(in)i(MPI-2)293 1633 y(\(e.g.,)e(support)h(non-cache)f(coherent)g(puts\))75 1683 y(Given)h(this)g(liberalization)e(of)i(thinking,)f(I)i(suggest:)75 1782 y(1\))f(It)h(seems)f(like)f(threads)h(can)g(now)g(get)g (asymptotically)e(infinite)h(bandwidth,)75 1832 y(just)h(by)g(using)g (MPI_SEND/RECV)e(or)j(MPI_PUT)e(to)h(pass)g(pointers.)42 b(That)21 b(problem)75 1882 y(is)g(solved.)43 b(Perhaps)20 b(an)h(MPI_AINT)f(\(or)i(MPI_SHARED_PTR\))c(type)j(is)h(indicated,)75 1932 y(that)f(only)g(works)g(between)f(processes)g(with)h(common)f (address)h(space.)42 b(This)75 1981 y(would)21 b(be)g(a)h(nice)f(lip)g (worship)f(support)h(for)g(local)f(shared)h(memory)g(beyond)f(what)h (we)75 2031 y(have)g(offered,)f(with)h(little)g(else)g(required.)41 b(It)22 b(certainly)e(is)h(nice)g(for)g(threads,)75 2081 y(but)g(can,)g(admittedly)f(be)h(done,)g(with)g(a)h(hack.)42 b(A)22 b(portable)e(mechanism)g(would)75 2131 y(be)h(preferable.)75 2230 y(2\))g(Provide)g(an)g(additive)f(call)h(mechanism)f(to)i(support) e(RK-like)g(semantics,)75 2280 y(as)h(a)h(separable)e(proposal)g(from)h (item)g(1\).)43 b(This)21 b(could)g(be)g(like)140 2380 y(MPI_RECV_BUFFER_INI)o(T\(dat)o(atype)o(,)e(count,)i(comm,)f(src,)h (tag,)g(req\))75 2430 y({and)g(other)g(variants})75 2529 y(If)g(successful,)f(the)h(storage)f(is)i(taken)f(from)g(attached)f (store)g(provided)h(to)g(the)75 2579 y(process)f(by)i(user)f(program,)f (but)h(not)g(necessarily)f(when)h(the)g(init)g(is)g(done)75 2629 y(\(implementation)e(defined\).)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 27 28 bop 75 -100 a Fp(9.8.)34 b(PR)o(OBLEMS)16 b(WITH)f(REGISTER)h (OPTIMIZA)l(TION)h(AND)e(CA)o(CHING)275 b Fv(27)75 45 y Fd(A)22 b(subsequent)d(MPI_START\(req\))g(causes)i(the)g(operation)f (to)h(proceed,)f(and)i(the)75 95 y(data)f(is)g(stored.)42 b(A)22 b(transformation:)140 195 y(MPI_WAIT\(req,)d(status\))75 294 y(and)i(new)140 394 y(MPI_BUFFER_RECEIVED)o(\(IN)e(status,)h(OUT)h (offset\);)75 493 y(provides)f(the)h(cursor)g(to)g(the)g(data,)g(as)h (an)f(offset)f(from)h(the)h(attached)e(store)h(base,)75 543 y(so)g(usable)g(by)g(C)h(and)f(Fortran.)140 643 y (MPI_BUFFER_FREE\(IN/)o(OUT)e(offset\);)75 742 y(relinquishes)g(this)i (space)g(at)g(any)h(time)f(thereafter.)140 842 y(MPI_SEND_BUFFER_INI)o (T\(\))e(is)i(similar)f(to)i(MPI_SEND_INIT\(\),)c(but)j(makes)75 892 y(the)g(buffer)g(argument)f(an)h(out)g(argument)g(offset)42 b(to)21 b(the)g(buffer)g(storage)f(base.)75 942 y(One)h(fills)g(that)g (memory,)f(prior)h(to)g(calling)g(MPI_START\(\))e(on)j(the)f(send)g (request.)75 992 y(After)g(the)g(MPI_START\(\),)e(the)i(buffer)g(is)g (no)h(longer)e(available)g(to)i(the)f(user)75 1041 y(program,)f(and)h (never)g(comes)g(back)g(\(implicit)f(MPI_BUFFER_FREE\()o(\)\).)75 1141 y(The)h(protocol)f(for)h(the)h(SEND)f(should)f(be)i(that)f(there)f (is)i(no)f(transfer,)f(until)h(room)75 1191 y(has)g(been)g(secured)f (in)i(the)f(receive)f(buffer,)h(so)g(that)g(it)g(can)h(be)f(turned)g (over)75 1241 y(to)g(the)h(user)f(on)g(completion.)41 b(In)22 b(a)f(single)g(memory)f(image,)h(this)g(is)g(a)h(clean)75 1290 y(offset)f(pass,)f(if)i(the)f(data)g(is)g(organized)f(with)h(the)g (same)g(layout.)42 b(Otherwise,)75 1340 y(there)21 b(is)g(no)h(real)e (advantage,)g(per)h(se.)75 1440 y(It)g(should)g(be)g(permissible)f(to)h (do)h(a)f(normal)g(SEND)g(or)g(normal)g(receive)f(reusing)75 1490 y(the)h(base_ptr)f(+)i(offset)e(address,)h(once)g(one)g(has)g (that,)g(and)g(it)g(is)h(filled.)75 1539 y(This)f(would)g(allow)f(one)i (to)f(do)g(a)h(number)f(of)g(sends)g(\(or)g(collectives\))e(with)i(the) 75 1589 y(same)g(buffer,)f(and)h(free)g(only)g(when)g(done)g(\(or)h(by) f(using)g(a)g(persistent)f(buffer)75 1639 y(send)h(on)g(the)h(last)e (one\).)75 1739 y(If)h(we)h(offer)f(this)f(functionality,)f(we)j (potentially)d(can)i(steer)g(people)g(away)75 1789 y(from)g(the)g (Bsend\(\),)f(which)h(no)g(one)h(seems)e(to)i(like,)e(and)i(we)f(do)g (offer)g(a)75 1838 y(standard)f(mode)h(that)g(has)g(no)h(copies)e(on)i (flat)f(shared)f(memory)h(systems)75 1888 y(\(threads,)f(or)h (real-memory)f(embedded)g(multi-processor)f(computers)h(and)h(so)g (on\),)75 1938 y(beyond)g(the)g(user)g(putting)f(the)h(data)g(in)h(and) f(out)g(of)g(the)g(buffers.)42 b(This)21 b(has)h(a)75 1988 y(nice)f(complementarity)e(to)i(other)g(modes,)f(and)i(seems)e(to) i(be)f(easy)g(to)g(add)h(to)f(the)75 2038 y(implementation.)40 b(It)22 b(relies)e(on)i(the)f(extant)g(buffer)f(mechanism,)g(and)h(is)g (seen)75 2087 y(to)g(work)g(with)g(Fortran)g(as)g(well)g(as)g(C.)75 2187 y(This)g(would)g(also)g(allow)f(libraries)g(to)i(exploit)e (light-weight)f(memory)i(allocation,)75 2237 y(if)g(available,)f(to)h (avoid)g(expensive)f(mallocs)h(to)g(get)g(messaging)f(space!?!)42 b(That)75 2287 y(might)21 b(be)g(especially)f(helpful)g(with)h (operations)f(that)h(are)g(sensitive)f(to)h(overheads.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Apr 22 03:55:57 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA17706; Mon, 22 Apr 1996 03:55:24 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA20448 for mpi-core-out; Mon, 22 Apr 1996 02:58:10 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA20443; Mon, 22 Apr 1996 02:57:54 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id AAA25744; Mon, 22 Apr 1996 00:57:52 -0700 Date: Mon, 22 Apr 1996 00:57:52 -0700 Message-Id: <199604220757.AAA25744@win233.nas.nasa.gov> From: "William C. Saphir" To: mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: dynamic process chapter Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, Enclosed is a full dynamic process chapter. Almost all of it appeared previously in three separate sections. Changes: - Introduction added back in (essentially unchanged) - Examples added back in (updated, hopefully) - Many minor corrections/additions - Two tentative new sections (3.4.5 and 3.4.6) These two sections are in response to concerns raised in the last few weeks. The first discusses how to disconnect client and server (or parent and child). The second provides some guidance for the format and content of info arguments. A few people have commented that the client/server stuff is complex and that we should punt name publishing altogether. I intended to add an alternative client/server proposal, but ran out of time. I will write up this proposal and bring it to the meeting. It will be a minimalist proposal. Essentially it will be MPI_Port_open(IN info, OUT port_name) MPI_Port_close(IN port_name) MPI_Accept(IN port_name, IN root, IN comm, OUT intercomm) MPI_Connect(IN port_name, IN root, IN comm, OUT intercomm) No name publishing. "port_name" is just a network address. It publishing is desired/required, it is up to the application. Please note that the chapter is 43 pages long, thereby guaranteeing that a large fraction of users will run in terror. I encourage proposals to trim unnecessary functions (except for functions I'm fond of, or course :-) ). Bill %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: main.dvi %%Pages: 45 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: /usr/local/bin/dvips -o main.ps main.dvi %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.04.22:0035 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (main.dvi) @start /Fa 1 61 df<0000038000000F0000003C000000F0000003C000000F0000003C 000000F0000003C000000F0000003C000000F0000000F00000003C0000000F00000003C0 000000F00000003C0000000F00000003C0000000F00000003C0000000F00000003801918 7D9520>60 D E /Fb 40 122 df<01F1C003F1C007F1C00701C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C0FFF1C0FFF1C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0121D809C16>13 D<001C0000003E0000003E0000002E0000006700000067000000E7800000C7800000C380 0001C3C0000183C0000181C0000381E0000381E0000700F0000700F0000600F0000E0078 000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038001E0070001F0070000F 0070000F00E0000780191D7F9C1C>65 D<003FC000FFF003C0F00780300F00001E00003C 00003C0000780000780000780000F00000F00000F00000F00000F00000F00000F00000F0 0000F000007800007800007800003C00003C00001E00000F000807801803C07800FFF000 3F80151F7D9D1B>67 DIII<003F8001FFF003C0F80780380F00181E00003C00003C00007800007800007800 00F00000F00000F00000F00000F00000F00000F007F8F007F8F000387800387800387800 383C00383C00381E00380F003807803803C0F801FFF0003F80151F7D9D1C>I73 D76 DII<003F000001FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F007800 07807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F80007C078000780780007807C000F803C000F003E001F001F003E000F80 7C0007C0F80003FFF00001FFE000003F00001A1F7E9D1F>II82 D<03F8000FFE001C0F00380700700300600000E00000E00000E00000E00000F000007800 007F00003FE0001FFC0007FE0001FF00001F800007800003C00003C00001C00001C00001 C00001C0C00180E00380F007007C0E001FFC0007F000121F7E9D17>IIIII89 D<7FFFF07FFFF00001E00003E00003C00007C0000780000F00001F00001E00003E00003C 0000780000F80000F00001F00001E00003C00007C0000780000F80000F00001E00003E00 003C00007C0000780000FFFFF0FFFFF0141D7E9C19>I<0FC03FF07FF87038401C001C00 1C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC3F1C0E127E9114>97 D<07E00FF81FFC3C1C70047000E000E000E000E000E000E000700070043C1C1FFC0FF807 E00E127E9112>99 D<000E000E000E000E000E000E000E000E000E000E000E0F8E1FEE3F FE7C3E700E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E0F1D7E9C15 >I<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C3FFC1FF8 07E00E127E9112>I<00FC01FC03FC07000E000E000E000E000E000E000E00FFE0FFE00E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E1D809C0D >I<03C3C00FFFC01FFFC01C3800381C00381C00381C00381C00381C001C38001FF8001F F0003BC0003800003800001FFC001FFF003FFF80700780E001C0E001C0E001C0F003C07C 0F803FFF001FFE0007F800121B7F9115>I105 D109 DI<03F0000FFC001FFE003C0F00780780700380E001C0E001C0E001C0E0 01C0E001C0F003C07003807807803C0F001FFE000FFC0003F00012127F9115>II<078E1FEE3FFE7C3E781E700EE00EE00E E00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E000E000E000E000E000E000E000E000E 0F1A7E9115>II<1FC03FF07FF0F030E000E000F0007F003FC01FE000F0 003800388038F078FFF07FE01FC00D127F9110>I<1C001C001C001C001C001C00FFE0FF E01C001C001C001C001C001C001C001C001C001C001C001C001C201FF00FF007C00C187F 970F>III<7003807807003C0E001C1C000E1C00073800 03F00001E00001C00001E00003F0000738000E18000E1C001C0E00380700700380F003C0 1212809113>120 DI E /Fc 6 118 df<07C00C20107020706000C000C000C00080008000C010C02060C03F00 0C0E7E8D0F>99 D<0300038003000000000000000000000000001C002400460046008C00 0C0018001800180031003100320032001C0009177F960C>105 D<383C0044C600470200 4602008E06000C06000C06000C0C00180C00180C40181840181880300880300F00120E7F 8D15>110 D<07C00C20101020186018C018C018C01880308030C060C0C061803E000D0E 7E8D11>I<030003000600060006000600FFC00C000C000C001800180018001800300030 803080310031001E000A147F930D>116 D<1C0200260600460600460600860C000C0C00 0C0C000C0C001818001818801818801838800C5900078E00110E7F8D14>I E /Fd 1 81 df80 D E /Fe 4 62 df<03000700FF000700070007000700070007000700 07000700070007000700070007000700070007007FF00C157E9412>49 D<0F8030E040708030C038E0384038003800700070006000C00180030006000C08080810 183FF07FF0FFF00D157E9412>I<0FE030306018701C701C001C00180038006007E00030 0018000C000E000EE00EE00EC00C401830300FE00F157F9412>I61 D E /Ff 6 111 df<70F8F8F87005057C840D>58 D<70F8FCFC74040404080810102040 060E7C840D>I<000001C00000078000001E00000078000001E00000078000000E000000 38000000F0000003C000000F0000003C000000F0000000F00000003C0000000F00000003 C0000000F0000000380000000E0000000780000001E0000000780000001E000000078000 0001C01A1A7C9723>I<00E001E001E000C000000000000000000000000000000E001300 23804380438043808700070007000E000E001C001C001C20384038403840388019000E00 0B1F7E9E10>105 D<1E07C07C00231861860023A032030043C034030043803803804380 38038087007007000700700700070070070007007007000E00E00E000E00E00E000E00E0 0E000E00E01C101C01C01C201C01C038201C01C038401C01C0184038038018801801800F 0024147E9328>109 D<1E07802318C023A06043C0704380704380708700E00700E00700 E00700E00E01C00E01C00E01C00E03821C03841C07041C07081C03083803101801E01714 7E931B>I E /Fg 15 119 df<1FE0003FF0007FF800783C00300E00000E00000E0003FE 001FFE003E0E00700E00E00E00E00E00E00E00783E007FFFE03FE7E00F83E013127E9116 >97 D<03F80FFC1FFE3C1E780C7000E000E000E000E000E000F000700778073E0E1FFC0F F803F010127D9116>99 D<003F00007F00003F0000070000070000070000070003C7000F F7001FFF003C1F00780F00700700E00700E00700E00700E00700E00700E00700700F0070 0F003C1F001FFFE00FE7F007C7E014197F9816>I<001F00007F8000FF8001E78001C300 01C00001C0007FFF00FFFF00FFFF0001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C0003FFE007FFF003FFE0011197F9816>102 D<03E3C007F7E00FFFE01C1CC0380E00380E00380E00380E00380E001C1C000FF8001FF0 001BE0003800001800001FFC001FFF003FFF807803C0E000E0E000E0E000E0E000E07001 C07C07C03FFF800FFE0003F800131C7F9116>I<7E0000FE00007E00000E00000E00000E 00000E00000E3C000EFE000FFF000F87800F03800E03800E03800E03800E03800E03800E 03800E03800E03800E03800E03807FC7F0FFE7F87FC7F01519809816>I<018003C003C0 018000000000000000007FC07FC07FC001C001C001C001C001C001C001C001C001C001C0 01C001C07FFFFFFF7FFF101A7D9916>I<7E0000FE00007E00000E00000E00000E00000E 00000E7FE00E7FE00E7FE00E0F000E1E000E3C000E78000EF0000FF0000FF8000FBC000F 1E000E0E000E07000E07807F87F0FFCFF07F87F01419809816>107 D109 D<7E3C00FEFE007FFF000F87800F03800E03800E03800E03800E03800E03800E03800E03 800E03800E03800E03807FC7F0FFE7F87FC7F01512809116>I<03E0000FF8001FFC003C 1E00780F00700700E00380E00380E00380E00380E00380F00780700700780F003C1E001F FC000FF80003E00011127E9116>I<7E3E00FEFF007FFF800F83C00F00E00E00E00E0070 0E00700E00700E00700E00700E00700E00E00F01E00F83C00FFF800EFF000E3C000E0000 0E00000E00000E00000E00000E00007FC000FFE0007FC000141B809116>I<0FEC3FFC7F FCF03CE01CE01C70007F801FF007F8003C600EE00EF00EF81EFFFCFFF8C7E00F127D9116 >115 D<0300000700000700000700000700007FFF00FFFF00FFFF000700000700000700 0007000007000007000007000007010007038007038007038007870003FE0001FC0000F8 0011177F9616>I<7F1FC0FF1FE07F1FC01C07001E0F000E0E000E0E000E0E00071C0007 1C00071C00071C0003B80003B80003B80001F00001F00000E00013127F9116>118 D E /Fh 60 124 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0 000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703 C00E03C00E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C07F87F8151D809C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<003F07 E00001C09C18000380F018000701F03C000E01E03C000E00E018000E00E000000E00E000 000E00E000000E00E000000E00E00000FFFFFFFC000E00E01C000E00E01C000E00E01C00 0E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E 00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C007FC7FCFF80211D 809C23>I<6060F0F0F8F86868080808080808101010102020404080800D0C7F9C15>34 D<60F0F8680808081010204080050C7C9C0C>39 D<004000800100020006000C000C0018 001800300030007000600060006000E000E000E000E000E000E000E000E000E000E000E0 00E000600060006000700030003000180018000C000C00060002000100008000400A2A7D 9E10>I<800040002000100018000C000C000600060003000300038001800180018001C0 01C001C001C001C001C001C001C001C001C001C001C00180018001800380030003000600 06000C000C00180010002000400080000A2A7E9E10>I<60F0F070101010102020408004 0C7C830C>44 DI<60F0F06004047C830C>I<0001000300060006 0006000C000C000C0018001800180030003000300060006000C000C000C0018001800180 030003000300060006000C000C000C00180018001800300030003000600060006000C000 C00010297E9E15>I<030007003F00C70007000700070007000700070007000700070007 000700070007000700070007000700070007000700070007000F80FFF80D1C7C9B15>49 D<07C01830201C400C400EF00FF80FF807F8077007000F000E000E001C001C0038007000 6000C00180030006010C01180110023FFE7FFEFFFE101C7E9B15>I<07E01830201C201C 781E780E781E381E001C001C00180030006007E00030001C001C000E000F000F700FF80F F80FF80FF00E401C201C183007E0101D7E9B15>I<60F0F0600000000000000000000060 F0F06004127C910C>58 D<0FE03038401CE00EF00EF00EF00E000C001C0030006000C000 8001800100010001000100010001000000000000000000000003000780078003000F1D7E 9C14>63 D<000600000006000000060000000F0000000F0000000F000000178000001780 00001780000023C0000023C0000023C0000041E0000041E0000041E0000080F0000080F0 000180F8000100780001FFF80003007C0002003C0002003C0006003E0004001E0004001E 000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E0618001801980070007800E0003801C0003801C00018038000180 780000807800008070000080F0000000F0000000F0000000F0000000F0000000F0000000 F0000000F0000000700000807800008078000080380000801C0001001C0001000E000200 070004000180080000E03000001FC000191E7E9C1E>I70 D72 DI77 DI<003F800000E0E000038038 0007001C000E000E001C0007003C00078038000380780003C0780003C0700001C0F00001 E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003 C0780003C0380003803C0007801C0007000E000E0007001C000380380000E0E000003F80 001B1E7E9C20>II<07E0801C1980300580 700380600180E00180E00080E00080E00080F00000F800007C00007FC0003FF8001FFE00 07FF0000FF80000F800007C00003C00001C08001C08001C08001C0C00180C00180E00300 D00200CC0C0083F800121E7E9C17>83 D<7FFFFFC0700F01C0600F00C0400F0040400F00 40C00F0020800F0020800F0020800F0020000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000001F800003FFFC001B1C7F9B1E>IIII<080810102020 40404040808080808080B0B0F8F8787830300D0C7A9C15>92 D<1FC00030700078380078 1C00301C00001C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C4060 3C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E00060007004300418080C30 07C00E127E9112>I<003F00000700000700000700000700000700000700000700000700 00070000070003E7000C1700180F00300700700700600700E00700E00700E00700E00700 E00700E00700600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C3018 18300C700E6006E006FFFEE000E000E000E00060007002300218040C1803E00F127F9112 >I<00F8018C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4 C00C38C01C3880181800381C00381C00381C00381C001818001C38000C300013C0001000 003000001800001FF8001FFF001FFF803003806001C0C000C0C000C0C000C06001803003 001C0E0007F800121C7F9215>II< 18003C003C0018000000000000000000000000000000FC001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000 C000000000000000000000000000000FE000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D >IIIII<03F0000E1C 00180600300300700380600180E001C0E001C0E001C0E001C0E001C0E001C06001807003 803003001806000E1C0003F00012127F9115>II<03C100 0C3300180B00300F00700700700700E00700E00700E00700E00700E00700E00700600700 700700300F00180F000C370007C700000700000700000700000700000700000700000700 003FE0131A7E9116>II<1F9030704030C010C010E010F8007F803FE00F F000F880388018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C00 1C003C00FFE01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C10 0E2003C00C1A7F9910>II< FF07E03C03801C01001C01000E02000E020007040007040007040003880003880003D800 01D00001D00000E00000E00000E00000400013127F9116>II<7F8FF00F 03800F030007020003840001C80001D80000F00000700000780000F800009C00010E0002 0E000607000403801E07C0FF0FF81512809116>II<7FFC 70386038407040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E12 7F9112>II E /Fi 8 118 df<78FCFCFCFC7800000000000078FC FCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00 781F00780E00F80000F80000F80000F80000F80000F800007800007801803C01801C0300 0E0E0003F80011127E9115>99 D<1E003F003F003F003F001E0000000000000000000000 0000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F001F00FFE0 FFE00B1E7F9D0E>105 D 110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8 F800F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018 E018F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Fj 25 120 df<70F8F8F0E005057B840E>46 D<0000020000000600000006000000 0E0000001E0000001E0000003F0000002F0000004F0000004F0000008F0000010F000001 0F0000020F0000020F0000040F00000C0F0000080F0000100F0000100F0000200F80003F FF800040078000C007800080078001000780010007800200078002000780060007801E00 0F80FF807FF81D207E9F22>65 D<01FFFFFE001E001C001E000C001E0004001E0004003C 0004003C0004003C0004003C00040078080800780800007808000078180000F0300000FF F00000F0300000F0300001E0200001E0200001E0200001E0001003C0002003C0002003C0 004003C00040078000800780018007800100078007000F001F00FFFFFE001F1F7D9E1F> 69 D<01FFFF00001E03C0001E00E0001E0070001E0078003C0078003C0078003C007800 3C0078007800F0007800F0007801E0007801C000F0070000F01E0000FFF00000F0380001 E01C0001E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003C01E0007803C0007 803C0807803C0807803C100F801C10FFF00C20000007C01D207D9E21>82 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207B9F15>I< 007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F00000 F00000F00000F0000070010070020030040018380007C00011147C9315>I<0000780003 F80000700000700000700000700000E00000E00000E00000E00001C00001C000F1C00389 C00707800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E 20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C01C207010E011C01 3C013802780C7BF07C00F000F000F000F0007000700170023804183807C010147C9315> I<00007800019C00033C00033C000718000700000700000E00000E00000E00000E00000E 0001FFE0001C00001C00001C00001C000038000038000038000038000038000070000070 0000700000700000700000700000E00000E00000E00000E00000C00001C00001C0000180 003180007B0000F300006600003C00001629829F0E>I<01E0000FE00001C00001C00001 C00001C000038000038000038000038000070000070000071E000763000E81800F01C00E 01C00E01C01C03801C03801C03801C0380380700380700380700380E10700E20700C2070 1C20700C40E00CC060070014207D9F17>104 D<00C001E001E001C00000000000000000 0000000000000E003300230043804300470087000E000E000E001C001C001C0038403880 30807080310033001C000B1F7C9E0E>I<01E0000FE00001C00001C00001C00001C00003 80000380000380000380000700000700000703C00704200E08E00E11E00E21E00E40C01C 80001D00001E00001FC00038E000387000387000383840707080707080707080703100E0 3100601E0013207D9F15>107 D<03C01FC0038003800380038007000700070007000E00 0E000E000E001C001C001C001C0038003800380038007000700070007100E200E200E200 E200640038000A207C9F0C>I<1C0F80F0002630C318004740640C004780680E00470070 0E004700700E008E00E01C000E00E01C000E00E01C000E00E01C001C01C038001C01C038 001C01C038001C01C0708038038071003803806100380380E10038038062007007006600 300300380021147C9325>I<1C0F802630C04740604780604700704700708E00E00E00E0 0E00E00E00E01C01C01C01C01C01C01C03843803883803083807083803107003303001C0 16147C931A>I<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801 E0F003C0F003C0F003C0F00780F00700700F00700E0030180018700007C00013147C9317 >I<01C1E002621804741C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01 C03C01C03C01C0380380780380700380E003C1C0072380071E000700000700000E00000E 00000E00000E00001C00001C0000FFC000171D809317>I<00F0400388C00705800E0380 1C03803C0380380700780700780700780700F00E00F00E00F00E00F00E00F01C00F01C00 703C00705C0030B8000F380000380000380000700000700000700000700000E00000E000 0FFE00121D7C9315>I<1C1E002661004783804787804707804703008E00000E00000E00 000E00001C00001C00001C00001C00003800003800003800003800007000003000001114 7C9313>I<00FC030206010C030C070C060C000F800FF007F803FC003E000E700EF00CF0 0CE008401020601F8010147D9313>I<018001C0038003800380038007000700FFF00700 0E000E000E000E001C001C001C001C003800380038003820704070407080708031001E00 0C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E03 801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C38014147C9318 >I<0E03803307802307C04383C04301C04700C08700800E00800E00800E00801C01001C 01001C01001C02001C02001C04001C04001C08000E300003C00012147C9315>I<0E00C1 C03300E3C02301C3E04381C1E04301C0E04701C060870380400E0380400E0380400E0380 401C0700801C0700801C0700801C0701001C0701001C0602001C0F02000C0F04000E1308 0003E1F0001B147C931E>I E /Fk 10 58 df<1F00318060C04040C060C060C060C060C0 60C060C060C060404060C031801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C 000C000C000C000C000C000C000C000C00FF8009107E8F0F>I<1F00618040C08060C060 0060006000C00180030006000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060 C000C0008001800F00008000400060C060C060804060801F000B107F8F0F>I<03000300 07000F000B001300330023004300C300FFE003000300030003001FE00B107F8F0F>I<20 803F002C002000200020002F0030802040006000600060C06080C061801F000B107F8F0F >I<0780184030C060C06000C000CF00F080E040C060C060C060406060C030801F000B10 7F8F0F>I<40007FE07FC08080808001000200040004000C000800080018001800180018 0018000B117E900F>I<1F00318060C060C060C071803F000F00338061C0C060C060C060 404060801F000B107F8F0F>I<1F00318060C0C040C060C060C06040E021E01E60006000 4060C0608043003E000B107F8F0F>I E /Fl 86 127 df32 D<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8 F8F870051C779B18>I<4010E038F078E038E038E038E038E038E038E038E038E038E038 60300D0E7B9C18>I<030600078F00078F00078F00078F00078F00078F007FFFC0FFFFE0 FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFFC0 1E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>I<00C00001C00001C0 0001C00003F0000FFC003FFE007DCF0071C700E1C380E1C780E1C780E1C780F1C00079C0 003DC0001FE0000FF80003FC0001DE0001CF0001C70061C380F1C380F1C380E1C380E1C7 0071C70079DE003FFE001FF80007E00001C00001C00001C00000C00011247D9F18>I<38 03007C07807C0780EE0F80EE0F00EE0F00EE1F00EE1E00EE1E00EE3E007C3C007C3C0038 7C0000780000780000F80000F00001F00001E00001E00003E00003C00003C00007C00007 83800787C00F87C00F0EE00F0EE01F0EE01E0EE01E0EE03E0EE03C07C03C07C018038013 247E9F18>I<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F0 1CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800 E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>I<387C7C7E3E0E0E0E1C1C38 F8F0C0070E789B18>I<007000F001E003C007800F001E001C0038003800700070007000 7000E000E000E000E000E000E000E000E0007000700070007000380038001C001E000F00 078003C001F000F000700C24799F18>I<6000F00078003C001E000F000780038001C001 C000E000E000E000E00070007000700070007000700070007000E000E000E000E001C001 C0038007800F001E003C007800F00060000C247C9F18>I<01C00001C00001C00001C000 C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C180 01C00001C00001C00001C00011147D9718>I<00600000F00000F00000F00000F00000F0 0000F00000F0007FFFC0FFFFE0FFFFE07FFFC000F00000F00000F00000F00000F00000F0 0000F00000600013147E9718>I<1C3E7E7F3F1F070E1E7CF860080C788518>I<7FFF00FF FF80FFFF807FFF0011047D8F18>I<3078FCFC78300606778518>I<000300000780000780 000F80000F00001F00001E00001E00003E00003C00007C0000780000780000F80000F000 01F00001E00003E00003C00003C00007C0000780000F80000F00000F00001F00001E0000 3E00003C00003C00007C0000780000F80000F00000F0000060000011247D9F18>I<01F0 0007FC000FFE001F1F001C07003803807803C07001C07001C0E000E0E000E0E000E0E000 E0E000E0E000E0E000E0E000E0E000E0F001E07001C07001C07803C03803801C07001F1F 000FFE0007FC0001F000131C7E9B18>I<01800380038007800F803F80FF80FB80438003 800380038003800380038003800380038003800380038003800380038003807FFCFFFE7F FC0F1C7B9B18>I<03F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E000 00E00000E00001C00001C00003C0000780000F00001E00003C0000780000F00001E00007 C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF00 7807807803C07801C03001C00001C00003C0000380000F0003FF0003FE0003FF00000780 0003C00001C00000E00000E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE00 03F800131C7E9B18>I<001F00003F0000770000770000E70001E70001C7000387000787 000707000E07001E07003C0700380700780700F00700FFFFF8FFFFF8FFFFF80007000007 00000700000700000700000700007FF000FFF8007FF0151C7F9B18>I<1FFF803FFF803F FF803800003800003800003800003800003800003800003800003BF8003FFE003FFF003C 07801803C00001C00000E00000E06000E0F000E0F000E0E001C07003C07C0F803FFF001F FC0003F000131C7E9B18>I<007E0001FF0007FF800F83C01E03C01C03C0380180380000 700000700000E1F800E7FE00FFFF00FE0780F803C0F001C0F000E0E000E0F000E07000E0 7000E07000E03801C03C03C01E07800FFF0007FE0001F800131C7E9B18>I<3078FCFC78 3000000000000000003078FCFC78300614779318>58 D<183C7E7E3C1800000000000000 00183C7E7E3E1E0E1C3C78F060071A789318>I<000300000780001F80003F00007E0001 FC0003F00007E0001FC0003F00007E0000FC0000FC00007E00003F00001FC00007E00003 F00001FC00007E00003F00001F8000078000030011187D9918>I<7FFFC0FFFFE0FFFFE0 FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C7E9318>I<6000 00F00000FC00007E00003F00001FC00007E00003F00001FC00007E00003F00001F80001F 80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000F000006000001118 7D9918>I<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E0003 8E00038E00038E000306000707000707000707000707000FFF800FFF800FFF800E03800E 03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E070 0000E00000E00000E00000E00000E00000E00000E00000E000007000007000E07000E038 00E03C00E01E01C00F07C007FF8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF00 1C0F801C03C01C03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C0070 1C00701C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE00 7FF800141C7F9B18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C0 7001C0700000E00000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001C0 7003C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8F F87F07F01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01FFFC01FFF C01FFFC01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C07F07 F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7F07F0 FF87F87F07F01C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0001DF000 1DF0001FB8001FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03801C01C0 7F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E 00000E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>I< FC01F8FE03F8FE03F83B06E03B06E03B06E03B06E03B8EE03B8EE0398CE0398CE039DCE0 39DCE039DCE038D8E038D8E038F8E03870E03870E03800E03800E03800E03800E03800E0 3800E0FE03F8FE03F8FE03F8151C7F9B18>I<7E07F0FF0FF87F07F01D81C01D81C01D81 C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39 C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18 >I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F0078070 0700780F007FFF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380E00380E00380E00380E00380E1E380E1E380F0E7 8070F700787F007FFF003FFE000FFC00001C00001E00000E00000F000007000007001122 7D9B18>I<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C 03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C 039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>I<03F3801FFF803FFF807C0F80 700780E00380E00380E00380E000007000007800003F00001FF00007FE0000FF00000F80 0003C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800 131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E070380070000070000070 000070000070000070000070000070000070000070000070000070000070000070000070 0000700000700000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003 DC0001F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E0007 0F000707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F 00001E00001C00003C0000780000700000F00001E00001C00003C0000780000700000F00 001E00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II<600000F00000F00000F800007800007C00003C00003C00003E00001E00001F0000 0F00000F00000F800007800007C00003C00003C00003E00001E00001F00000F00000F800 007800007800007C00003C00003E00001E00001E00001F00000F00000F80000780000780 00030011247D9F18>II<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<061E3E387070E0E0E0F8FC7C7C38070E789E18>I<1FE0003FF8007FFC00781E00300E 0000070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F00F00781F 003FFFF01FFBF007E1F014147D9318>I<7E0000FE00007E00000E00000E00000E00000E 00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E 00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C809B18>I< 01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E00000E00000 E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80003F 80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80380F807007 80700380E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF F00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F00380780700380700380E0 01C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF8007FF0001 FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001C0007FFFC0 FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFF F81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC001FF80039E0 003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E000387800 F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00000E00000E 00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B 18>I<03800007C00007C00007C0000380000000000000000000000000007FC000FFC000 7FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8 007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F 001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC17148093 18>I<01F0000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0 E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E 00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00 380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00 000E00000E00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F8038 0F80700780700780E00380E00380E00380E00380E00380E00380700780700780380F803C 1F801FFF800FFB8003E380000380000380000380000380000380000380000380003FF800 3FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C000 0380000380000380000380000380000380000380000380000380007FFE00FFFF007FFE00 15147F9318>I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC 0003FE00001F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318 >I<0180000380000380000380000380007FFFC0FFFFC0FFFFC003800003800003800003 80000380000380000380000380000380000380400380E00380E00380E001C1C001FFC000 FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC 1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E03800707000707000707 00038E00038E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318 >II<7F8FF0 7F9FF07F8FF0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00 039E00038E000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8F F00E01C00E03800E0380070380070700070700038700038600038E0001CE0001CE0000CC 0000CC0000DC0000780000780000780000700000700000700000F00000E00079E0007BC0 007F80003F00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C070078000 0F00001E00003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FF FFF0FFFFF014147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC000 01E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 007800007FE0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003F E0003FE0007FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00003C000FFC000FF00007C000013247E9F18>125 D<060C1F1E3FBEFBF8F1F060C00F067C9B18>I E /Fm 52 122 df<000078007C7800FC 7801FC7803C000038000078000078000078000078000078000078000078000078000FFFC 78FFFC78FFFC780780780780780780780780780780780780780780780780780780780780 78078078078078078078078078078078078078078078078078078078152480A31A>12 D<787878781830306060E0050A7C830E>44 DII<000080000180000180000300000300000300000600000600000600000C 00000C00000C0000180000180000180000300000300000300000600000600000600000C0 0000C00000C0000180000180000180000180000300000300000300000600000600000600 000C00000C00000C00001800001800001800003000003000003000006000006000006000 00C00000C00000C0000011317DA418>I<00C001C007C0FFC0FFC0FBC003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C0FFFFFFFFFFFF10227CA118>49 D<03F0000FFC001FFE003C1F00300780 7007C06003C0E003E0C001E04001E04001E00001E00001E00001E00003C00003C0000780 000780000F00001E00003C0000780000F00001E00001C0000380000700000E00001C0000 380000700000FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC001FFF003E0F003807 807003C02003C02003C00003C00003C00003C0000780000780000F00001E0003FC0003F8 0003FE00000F000007800003C00003C00001E00001E00001E00001E00001E08001E0C003 C0E003C07007803C0F801FFF000FFC0003F00013237EA118>I<001F00001F00002F0000 2F00006F0000EF0000CF0001CF0001CF00038F00038F00078F00070F000F0F000E0F001E 0F003C0F003C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F0000 0F00000F00000F00000F00000F00000F0015217FA018>I<3FFF803FFF803FFF803C0000 3C00003C00003C00003C00003C00003C00003C00003C00003CF8003FFE003FFF003F0F80 3E07803C03C03803C00001E00001E00001E00001E00001E00001E00001E04003C04003C0 E003C07007807C1F003FFE000FFC0003F00013227EA018>I<007E0001FF0003FF0007C1 000F00001E00001E00003C00003C000078000078000078F800F1FE00F7FF00FF0F80FC07 80F803C0F803C0F801C0F001E0F001E0F001E0F001E0F001E07001E07001E07801E07803 C03803C03C03801C07801F0F000FFE0007FC0001F00013237EA118>II<01F00007FC000FFE00 1E0F003C07803C07807803C07803C07803C07803C07803C03803803C07801E0F000F1E00 07FC0003F8000FFE001E0F003C07807803C07803C0F001E0F001E0F001E0F001E0F001E0 F001E07803C07803C03C07803E0F801FFF0007FC0001F00013237EA118>I<01F00007FC 000FFE001E0F003C07003807807803807803C0F003C0F001C0F001E0F001E0F001E0F001 E0F001E0F001E07003E07803E07803E03C07E03E1FE01FFDE00FF1E003E3C00003C00003 C0000380000780000700000F00101E00183C003FF8001FF00007C00013237EA118>I<00 1F0000001F0000003F8000003F8000003B8000007BC0000073C0000071C00000F1E00000 F1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C0007 803C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF001C000F003C0007803C 00078038000780780003C0780003C0700003C0F00001E0F00001E0E00001E01B237EA220 >65 D<000FF000007FFC0000FFFF0001F01F0003C00700078000000F0000001E0000003E 0000003C0000003C000000780000007800000078000000F0000000F0000000F0000000F0 000000F0000000F0000000F0000000F0000000F00000007800000078000000780000003C 0000003C0000003E0000001E0000000F0000000780008003C0038001F00F8000FFFF0000 7FFC00000FF00019257DA31F>67 DIII73 D77 DI80 D82 D<00FE0003FFC007FFE00F81E01E00603C00 003C00007800007800007800007800007800007C00003C00003F00001FC0000FFC0007FF 0001FF80003FC00007E00001F00000F00000F80000780000780000780000780000780000 78C000F0E000F0F801E07E07C03FFF800FFF0001FC0015257EA31B>III87 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF0 0FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03 803C0180780000780000700000F00000F00000F00000F00000F00000F000007800007800 007800003C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C000 03C00003C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC01F FFC03F0FC03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C07803C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC00 1FFE003E0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F00000 7000007800007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01 FF03C0038007800780078007800780078007800780FFF8FFF8FFF8078007800780078007 8007800780078007800780078007800780078007800780078007800780102380A20F>I< 01F07807FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F00 0FFE001FFC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8 F00078F00078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>III107 DIII<01FC0007FF000FFF801F07C03C 01E07800F07800F0700070F00078F00078F00078F00078F00078F000787800F07800F07C 01F03E03E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07C07C03 C07803C0F803C0F003C0F003C0F003C0F003C0F003C0F003C0F803C07803C07803C03C07 C03F0FC01FFFC00FFBC003E3C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C012207E9519>II<07F01FFC3FFE3C 0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE3FFC0F F010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F000F00 0F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12>IIII<7801F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F80000700000 F80001FC0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F815168095 16>II E /Fn 27 90 df45 D<3078F8787005057C840D>I<003F8000C1 E00100F00200780400780400780F007C0F807C0F807C0F00780600780000F80000F00001 E00001C0000380000700000E00001C0000380000600000C0000180000300200600200800 401000403FFFC07FFF80FFFF80161E7E9D17>50 D<007F000183C00201E00400F00700F0 0F00F00F01F00F01F00001E00001E00003C0000380000700000E0000F800000E00000700 0007800007C00003C00007C03007C07807C0F807C0F807C0F00780800F00400E00201C00 18780007E000141F7D9D17>I<0000600000600000E00001C00003C00005C0000DC00009 C00011C000238000438000C3800083800103800203800407000807001807001007002007 00400700FFFFF0000E00000E00000E00000E00000E00001C00001E0001FFE0141E7E9D17 >I<01803001FFE003FFC003FF0003FC0002000002000002000004000004000004000004 7C000587000603800C01800801C00001C00001E00001E00001E00001E07003C0F803C0F0 03C0E00380800780400700400E00201C0018700007C000141F7D9D17>I<000010000000 1800000038000000380000007800000078000000FC000001BC0000013C0000033C000002 3C0000063C0000043E0000081E0000081E0000101E0000101E0000201E0000200F000040 0F0000400F0000FFFF0000800F0001000F80010007800200078002000780040007800400 07800C0007C03E0007C0FF807FFC1E207E9F22>65 D<07FFFF00007C01C0003C01E0003C 00F0007800F8007800F8007800F8007800F8007800F8007800F000F001F000F001E000F0 03C000F00F8000FFFE0000F00F0001E007C001E003C001E003E001E001E001E001E001E0 01E003C001E003C003E003C003E003C003C003C007C003C00F8007800F0007803E00FFFF F0001D1F7E9E20>I<0001F808000E061800380138007000F801E0007803C00070078000 30078000300F0000301F0000301E0000303E0000203C0000007C0000007C0000007C0000 007C000000F8000000F8000000F8000000F8000000F80000007800004078000080780000 803C0000803C0001001C0002000E00020006000C000300100001C0E000003F00001D217B 9F21>I<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E 0078001E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E 01E0003E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E0 03C003C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C007800 3C0038003C001800780018007800080078000800780008007800080078080800F0100000 F0100000F0100000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001 E0000801E0001003C0001003C0001003C0002003C0002003C0006003C000C0078001C007 8007C0FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C0018007800180078 00080078000800780008007800080078000800F0100000F0100000F0100000F0300000F0 700000FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0000003C0 000003C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F7E9E1E> I<0001FC04000F030C003C009C0070007C00E0003C01C0003803800018078000180F0000 181F0000181E0000183E0000103C0000007C0000007C0000007C0000007C000000F80000 00F8000000F8007FFCF80003E0780001E0780001E0780003C0780003C03C0003C03C0003 C01C0003C00E0007C007000B800380118001E06080003F80001E217B9F24>I<07FFC7FF C0007C00F800003C007800003C007800007800F000007800F000007800F000007800F000 007800F000007800F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000 F001E00001E003C00001E003C00001E003C00001E003C00001E003C00001E003C00003C0 07800003C007800003C007800003C007800003C007800003C007800007800F000007C00F 8000FFF8FFF800221F7E9E22>I<07FFE0007C00003C00003C0000780000780000780000 780000780000780000F00000F00000F00000F00000F00000F00001E00001E00001E00001 E00001E00001E00003C00003C00003C00003C00003C00003C00007800007C000FFFC0013 1F7F9E10>I<07FFF000007E0000003C0000003C00000078000000780000007800000078 0000007800000078000000F0000000F0000000F0000000F0000000F0000000F0000001E0 000001E0000001E0000001E0000001E0008001E0010003C0010003C0010003C0030003C0 020003C0060003C0060007801E0007807C00FFFFFC00191F7E9E1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800004E0002F000004E0002F0 00004E0004F000004E0004F000004E0008F000004E0008F00000870011E00000870011E0 0000870021E00000870021E00000870041E00000838041E00001038083C00001038083C0 0001038103C00001038203C0000101C203C0000101C403C0000201C40780000201C80780 000201C80780000201D00780000200F00780000600E00780000600E00F00000F00C00F80 00FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E001800003E001800004F 001000004F001000004780100000478010000043C010000043C010000083C020000081E0 20000081E020000080F020000080F020000080782000010078400001007C400001003C40 0001003C400001001E400001001E400002000F800002000F800002000F80000200078000 0200078000060003800006000300000F00010000FFE0010000221F7E9E22>I<0003F800 001E0E000038070000E0038001C001C003C001E0078000E00F0000F00F0000F01E0000F0 1E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8F80001F0F80001F0 F80001F0F80001F0F80003E0780003E0780003C0780007C07C0007803C000F003C001E00 1E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07FFFF00007C03C000 3C01E0003C00F0007800F0007800F8007800F8007800F8007800F8007800F000F001F000 F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001E0000001E0000001 E0000001E0000003C0000003C0000003C0000003C0000003C0000003C000000780000007 C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C01E0007801E00078 01F0007801F0007801F0007801F0007801E000F003E000F003C000F0078000F00F0000F0 3C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E01E0003C03E0003C0 3E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC0F18000003E01F20 7E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E00080E0008 0E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E00000F0 0000F04000E04000E04000E04000E06000C0600180E00380F80300C60C0081F80016217D 9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010C00F0010800F0010 800F0010800F0010001E0000001E0000001E0000001E0000001E0000001E0000003C0000 003C0000003C0000003C0000003C0000003C000000780000007800000078000000780000 007800000078000000F0000001F800007FFFE0001C1F7A9E21>II 87 D<03FFC0FFC0007F007E00003E003800001E003000001E002000000F004000000F00 8000000F81000000078200000007C600000003C400000003E800000001F000000001F000 000000F000000000F800000000F8000000017C000000023C000000043C0000000C1E0000 00081E000000101F000000200F000000400F800000C0078000008007C000010003C00007 0003E0001F8007E000FFE01FFE00221F7F9E22>II E /Fo 3 21 df0 D<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF80FF003 C010127D9317>15 D<000000C0000003C000000F0000003C000000F0000003C000000700 00001C00000078000001E00000078000001E00000078000000E0000000780000001E0000 000780000001E0000000780000001C0000000700000003C0000000F00000003C0000000F 00000003C0000000C0000000000000000000000000000000000000000000000000000000 007FFFFF80FFFFFFC01A247C9C23>20 D E /Fp 56 123 df<007000E001C00380078007 000E001E001E003C003C003C0078007800780078007000F000F000F000F000F000F000F0 00F000F000F000F000F000700078007800780078003C003C003C001E001E000E00070007 80038001C000E000700C2E7EA112>40 DI<018001C001800180C183E187F99F7DBE1FF807E007E01FF87DBEF99F E187C1830180018001C0018010147DA117>I<787878781830306060E0050A7D830D>44 DII<000100030003000600060006 000C000C000C00180018001800300030003000600060006000C000C000C0018001800180 0300030003000600060006000C000C000C00180018001800300030003000600060006000 C000C000C000102D7DA117>I<001F0000001F0000003F8000003B8000003B8000007BC0 000073C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0F00003C078 0003C078000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F 001E000F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B207F 9F1E>65 DI< 001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000 780000F00000F00000F00000F00000F00000F00000F00000F00000780000780000780000 7C00003C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D> IIII<001FE000FFF801FFFE03E03E0780 0E0F00001E00003E00003C00007C0000780000780000780000F00000F00000F00000F000 00F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F00 1E07801E03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F 000000FFC00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F007800 07807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F00003C0F80007C07800078078000780780007803C000F003C000F001E00 1E001F003E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC000 03FFF00007E1F8000F807C001F003E001E001E003C000F003C000F007800078078000780 78000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE00 0F83FC0007E1F80003FFF00000FFF800003F780000007C0000003E0000001E0000001F00 00000F801A277DA021>II<01FC0007FF800FFFC01F03C03C00C03C0000780000780000780000780000 7800007C00003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E0 0001E00001E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F800 13227EA019>IIIII 89 DI<07E0 3FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF 1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C 013E0F1FFF0FFE03F010147E9314>I<0007800007800007800007800007800007800007 8000078000078000078000078000078007C7800FF7801FFF803E1F807C0780780780F807 80F00780F00780F00780F00780F00780F00780F00780780780780F803E1F801FFF800FF7 8007C78011207E9F17>I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FF FF80FFFF80F00000F00000F000007000007800003C01003E07001FFF0007FE0001F80011 147F9314>I<007E01FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF0 0F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F20 809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C 1E003E3E001FFC003FF80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F0 00F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00141E7F9317>I105 D<01E001E001E001E00000000000000000000000000000000001E001E001E001E001E001 E001E001E001E001E001E001E001E001E001E001E001E001E001E001E001E001E001E001 E001E0C3C0FFC0FF803F000B29839F0C>IIIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0 F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F80014147F9317> II<03C7800FF7801FFF803E1F807C 0F80780780780780F00780F00780F00780F00780F00780F00780F807807807807C0F803E 1F801FFF800FF78007C78000078000078000078000078000078000078000078000078000 0780111D7E9317>II<07F01FFC3FFC780C7800780078007C00 3FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311>I<1E001E001E 001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E001E001E001E001E 001E001E201FF00FF007C00C1A7F9910>IIII<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F80000F00001F000 01F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414809315>II<7FFF7FFF7FFF003E003C007800F800F0 01E003E007C007800F001F001E003C007C00FFFFFFFFFFFF10147F9314>I E /Fq 40 122 df<387CFEFEFE7C3807077C860F>46 D<00E00001E0000FE000FFE000F3 E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003 E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000FFFF80FF FF80111D7C9C1A>49 D<07F0001FFE00383F007C1F80FE0FC0FE0FC0FE0FE0FE07E07C07 E03807E0000FE0000FC0000FC0001F80001F00003E0000780000F00000E00001C0000380 600700600E00601C00E01FFFC03FFFC07FFFC0FFFFC0FFFFC0131D7D9C1A>I<01FC0007 FF000E0F801E0FC03F07E03F07E03F07E03F07E01E0FC0000FC0000F80001F0001FC0001 FC00000F800007C00003E00003F00003F83803F87C03F8FE03F8FE03F8FE03F0FC03F078 07E03C0FC01FFF8003FC00151D7E9C1A>I<0001C00003C00007C00007C0000FC0001FC0 003BC00073C00063C000C3C00183C00383C00703C00E03C00C03C01803C03803C07003C0 E003C0FFFFFEFFFFFE0007C00007C00007C00007C00007C00007C000FFFE00FFFE171D7F 9C1A>I<3803803FFF803FFF003FFE003FFC003FF0003F80003000003000003000003000 0033F80037FE003C1F00380F801007C00007C00007E00007E07807E0FC07E0FC07E0FC07 E0FC07C0780FC0600F80381F001FFC0007F000131D7D9C1A>I<003F0001FFC007E0E00F 81E01F03F01E03F03E03F07C03F07C01E07C0000FC1000FCFF00FDFFC0FD03E0FE01F0FE 01F0FC01F8FC01F8FC01F8FC01F87C01F87C01F87C01F83C01F03E01F01E03E00F07C007 FF8001FE00151D7E9C1A>I<6000007FFFF87FFFF87FFFF07FFFE07FFFC0E00180C00300 C00300C00600000C0000180000380000380000780000700000F00000F00001F00001F000 01F00001F00003F00003F00003F00003F00003F00003F00001E00000C000151E7D9D1A> I<07F8001FFE00381F80780F80FC0FC0FC0FC0FC0FC0780FC0301F80001F00003E00007C 0000700000E00000E00000C00000C00000C00000C00000C00000C0000000000000000000 0000000001C00003E00007F00007F00007F00003E00001C00012207D9F19>63 D69 D76 DII80 D<03FC080FFF381E03F83800F8700078700038F00038F00018F00018F80000FC00007FC0 007FFE003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC0003CC000 3CC0003CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>83 D85 DI<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C 03E07C03E0F803E0F803E0F803E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007 FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000FC0000FC00007C 00007E00003E00603F00C01F81C007FF0001FC0013147E9317>I<0007F80007F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F801F8F80FFEF81F83F8 3E01F87E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F8 7E00F83E01F81F07F80FFEFF03F8FF18207E9F1D>I<01FE0007FF800F83C01E01E03E00 F07C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E00 180F807007FFE000FF8015147F9318>I<001F8000FFC001F3E003E7E003C7E007C7E007 C3C007C00007C00007C00007C00007C000FFFC00FFFC0007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0003F FC003FFC0013207F9F10>I<01FC3C07FFFE0F079E1E03DE3E03E03E03E03E03E03E03E0 3E03E01E03C00F07800FFF0009FC001800001800001C00001FFF800FFFF007FFF81FFFFC 3C007C70003EF0001EF0001EF0001E78003C78003C3F01F80FFFE001FF00171E7F931A> II<1C003E00 7F007F007F003E001C00000000000000000000000000FF00FF001F001F001F001F001F00 1F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B217EA00E>I107 DI< FE0FE03F80FE1FF07FC01E70F9C3E01E407D01F01E807E01F01F807E01F01F007C01F01F 007C01F01F007C01F01F007C01F01F007C01F01F007C01F01F007C01F01F007C01F01F00 7C01F01F007C01F01F007C01F01F007C01F0FFE3FF8FFEFFE3FF8FFE27147D932C>II<01FF0007FFC0 1F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC007EFC007E7C007C 7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>II<01F81807FE381F87783F01F83E01F87E00F87C00F8FC00F8FC 00F8FC00F8FC00F8FC00F8FC00F87C00F87E00F87E00F83F01F81F87F80FFEF803F8F800 00F80000F80000F80000F80000F80000F80000F80007FF0007FF181D7E931C>II<0FE63FFE701E600E E006E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81EFFFCC7F010147E93 15>I<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F800F800F 800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>II119 DII E /Fr 36 122 df45 DI<0030 0000700001F0000FF000FFF000FFF000F1F00001F00001F00001F00001F00001F00001F0 0001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F0 0001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F0 00FFFFE0FFFFE0FFFFE013287BA71D>49 D<00FE0003FF800FFFC01FFFE01F07F03C01F8 7800FC78007C70007CF0003EF0003E60003E20003E00003E00003E00003E00007C00007C 0000780000F80001F00001E00003E00007C0000F80001F00003E00007C0000F80001F000 03E00003C0000780000F00001E00003C00007FFFFE7FFFFE7FFFFE7FFFFE17287EA71D> I<007E0003FF8007FFE01FFFF03F83F83E00F87C007C38007C30007C10007C00007C0000 7C0000F80000F80001F80003F0000FE001FFC001FF8001FFC001FFE00001F80000FC0000 7C00003E00003E00001F00001F00001F00001F00001F80001FC0003EE0003EF0007E7C00 FC3F01F81FFFF00FFFE003FFC000FE0018297EA71D>I<0007E0000FE0000BE0001BE000 1BE0003BE0003BE00073E00073E000F3E000F3E001E3E001E3E003E3E003C3E007C3E00F 83E00F83E01F03E01F03E03E03E03E03E07C03E07C03E0F803E0FFFFFFFFFFFFFFFFFFFF FFFF0003E00003E00003E00003E00003E00003E00003E00003E00003E00003E018277EA6 1D>I<3FFFF83FFFF83FFFF83FFFF83E00003E00003E00003E00003E00003E00003E0000 3E00003E00003E00003E3E003EFF803FFFE03FFFF03FC1F03F00F83F007C3E007C00007C 00003E00003E00003E00003E00003E00003E00003E40003E60007CE0007CF000F87801F8 7F07F03FFFE00FFFC007FF0001FC0017287EA61D>I66 D<0003FF00000FFFE0003FFFF800FFFFF801FE 01F803F8007007E0001007C000000F8000001F8000001F0000003E0000003E0000007C00 00007C0000007C0000007C000000F8000000F8000000F8000000F8000000F8000000F800 0000F8000000F8000000F8000000F80000007C0000007C0000007C0000007C0000003E00 00003E0000001F0000001F8000000F80000007C0000007E0000803F8003801FE01F800FF FFF8003FFFF0000FFFC00003FE001D2C7CAA25>III73 D77 D80 D84 D<01FC000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F80000F80000F8 0001F800FFF807FFF81FFFF83FC0F87E00F8FC00F8F800F8F800F8F800F8FC01F87E07F8 7FFFF83FFFF81FFCF80FE0F8151B7E9A1C>97 DI<007F8001FFE007FFF80FFFF81FC0783F00103E00007C00007C0000 7C0000F80000F80000F80000F80000F80000F80000F800007C00007C00007C00003E0000 3F00181FC0780FFFF807FFF801FFE0007F80151B7E9A1A>I<00003E00003E00003E0000 3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E01FC 3E03FF3E0FFFBE1FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF800 3EF8003EF8003EF8003EF8003EFC003E7C003E7C003E7E007E3F00FE1FC1FE1FFFFE0FFF BE03FE3E01F83E172A7EA91E>I<00FC0003FF0007FF800FFFC01F87E03F01E03E00F07C 00F07C0070F80078FFFFF8FFFFF8FFFFF8FFFFF8F80000F80000F800007800007C00007C 00003E00003F00181FC0F80FFFF807FFF801FFE0007F00151B7E9A1A>I<003FC000FFC0 01FFC003FFC007E00007C0000F80000F80000F80000F80000F80000F80000F80000F8000 0F8000FFFE00FFFE00FFFE000F80000F80000F80000F80000F80000F80000F80000F8000 0F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8000 0F80000F80000F80000F8000122A7FA912>I<007C078001FF3FC007FFFFC00FFFFFC00F 83E0001F01F0001F01F0003E00F8003E00F8003E00F8003E00F8003E00F8001F01F0001F 01F0000F83E0000FFFE0001FFFC0001DFF00001C7C00003C0000003C0000003E0000001F FFE0001FFFF8000FFFFE001FFFFF003FFFFF807E003F807C000FC0F80007C0F80007C0F8 0007C0F80007C0FC000FC07E001F803F807F001FFFFE000FFFFC0003FFF000007F80001A 287F9A1D>III108 DI I<007E0001FF8007FFE00FFFF01F81F83F00FC3E007C7C003E7C003E78001EF8001FF800 1FF8001FF8001FF8001FF8001FF8001F7C003E7C003E7C003E3E007C3F00FC1FC3F80FFF F007FFE003FFC0007E00181B7E9A1D>II114 D<07F8001FFF003FFF807FFF80 7C0780F80100F80000F80000F80000FC00007F00007FF0003FFC001FFE0007FF0000FF80 001FC0000FC00007C00007C04007C0E007C0FC1F80FFFF807FFF001FFE0007F800121B7E 9A16>I<0F80000F80000F80000F80000F80000F80000F8000FFFF80FFFF80FFFF800F80 000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80 000F80000F80000F80000F80000F80000F80800FC38007FFC007FFC003FF0001F8001222 7FA115>II119 D<7C001F807E003F003F003E001F007C 000F80FC0007C0F80007E1F00003E3E00001F7E00000FFC000007F8000007F0000003E00 00003E0000003F0000007F800000FFC00001F3E00003E1E00003E1F00007C0F8000F807C 001F807C001F003E003E001F007E001F80FC000FC01A1B809A1B>II E /Fs 12 122 df68 D80 D<000FFFF000000000FFFFFF00000003FFFFFFC0000007F8 007FF000000FFC001FFC00000FFC0007FE00001FFE0003FF00001FFE0003FF80001FFE00 01FF80001FFE0001FFC0001FFE0000FFC0000FFC0000FFE00007F80000FFE00003F00000 FFE00000000000FFE00000000000FFE00000000000FFE00000000000FFE00000000000FF E0000000007FFFE00000003FFFFFE0000003FFFFFFE000001FFFF0FFE000007FFE00FFE0 0001FFE000FFE00003FF8000FFE0000FFE0000FFE0001FFC0000FFE0001FF80000FFE000 3FF80000FFE0007FF00000FFE0007FF00000FFE000FFE00000FFE000FFE00000FFE000FF E00000FFE000FFE00000FFE000FFE00001FFE000FFE00001FFE0007FF00003FFE0007FF0 00077FE0003FF8000F7FE0001FFC003E7FF8000FFF80F83FFFE003FFFFF01FFFE000FFFF C00FFFE0000FFE0003FFE0332E7CAD38>97 D<00001FFFC0000001FFFFFC000007FFFFFF 00001FF8007F80007FE000FFC000FF8000FFC001FF0001FFE003FE0001FFE007FE0001FF E00FFC0001FFE00FFC0001FFE01FF80000FFC03FF800007F803FF800003F007FF8000000 007FF0000000007FF0000000007FF000000000FFF000000000FFF000000000FFF0000000 00FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF0000000 00FFF0000000007FF0000000007FF0000000007FF8000000007FF8000000003FF8000000 003FF8000000001FFC000000F00FFC000000F00FFE000001E007FE000001E003FF000003 C001FF8000078000FFC0000F00007FF0003E00001FFC01FC000007FFFFF0000001FFFFC0 0000001FFC00002C2E7CAD34>99 D<00001FFC00000001FFFFC0000007FFFFF000001FF8 0FFC00007FC001FE0000FF8000FF0001FF00007F8003FE00003FC007FC00001FE00FFC00 001FE01FF800001FF01FF800000FF03FF800000FF83FF800000FF83FF000000FF87FF000 0007F87FF0000007FC7FF0000007FCFFF0000007FCFFF0000007FCFFFFFFFFFFFCFFFFFF FFFFFCFFFFFFFFFFFCFFF000000000FFF000000000FFF000000000FFF000000000FFF000 0000007FF0000000007FF0000000007FF0000000003FF8000000003FF8000000001FF800 00003C1FF80000003C0FFC0000003C0FFC0000007807FE000000F803FF000001F001FF80 0003E000FFC00007C0003FE0001F80001FFC01FF000007FFFFFC000000FFFFE00000000F FF00002E2E7DAD35>101 D<007C0001FF0003FF8007FFC007FFC00FFFE00FFFE00FFFE0 0FFFE00FFFE007FFC007FFC003FF8001FF00007C00000000000000000000000000000000 000000000000000000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0 03FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0 01FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0 01FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0 01FFC0FFFFFFFFFFFFFFFFFFFFFFFF18497CC820>105 D<00FF8003FF000003FF000000 FFFF801FFFE0001FFFE00000FFFF807FFFF8007FFFF80000FFFF81F80FFC01F80FFC0000 FFFF83C007FE03C007FE000003FF878003FF078003FF000001FF8E0003FF0E0003FF0000 01FF9C0003FF9C0003FF800001FFB80001FFB80001FF800001FFB80001FFF80001FFC000 01FFF00001FFF00001FFC00001FFF00001FFF00001FFC00001FFE00001FFE00001FFC000 01FFE00001FFE00001FFC00001FFE00001FFE00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FFFFFF80FFFFFF80FFFFFF80 FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80 592E7BAD62>109 D<00FF8007FE0000FFFF803FFFC000FFFF807FFFF000FFFF81F81FF8 00FFFF83C00FFC0003FF870007FE0001FF8E0007FE0001FF9C0007FF0001FFB80003FF00 01FFB80003FF8001FFF00003FF8001FFF00003FF8001FFE00003FF8001FFE00003FF8001 FFE00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FF C00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC000 03FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003 FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF 8001FFC00003FF8001FFC00003FF80FFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFF FFFFFF81FFFFFF382E7BAD41>I<00000FFE0000000001FFFFF000000007FFFFFC000000 1FF803FF0000007FE000FFC00000FF80003FE00001FF00001FF00003FE00000FF80007FC 000007FC000FFC000007FE000FF8000003FE001FF8000003FF003FF8000003FF803FF000 0001FF803FF0000001FF807FF0000001FFC07FF0000001FFC07FF0000001FFC0FFF00000 01FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001 FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE07FF0000001FF C07FF0000001FFC07FF0000001FFC07FF0000001FFC03FF8000003FF803FF8000003FF80 1FF8000003FF001FFC000007FF000FFC000007FE0007FE00000FFC0003FE00000FF80001 FF00001FF00000FF80003FE000007FE000FFC000001FFC07FF00000007FFFFFC00000001 FFFFF0000000001FFF000000332E7DAD3A>I<007F803F80FFFF80FFE0FFFF81FFF8FFFF 83C3FCFFFF8707FE03FF8E07FE01FF9C0FFF01FFB80FFF01FFB80FFF01FFF00FFF01FFF0 0FFF01FFF007FE01FFE003FC01FFE001F801FFE0000001FFE0000001FFC0000001FFC000 0001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC00000 01FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001 FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FF C0000001FFC0000001FFC00000FFFFFFE000FFFFFFE000FFFFFFE000FFFFFFE000282E7D AD2F>114 D<000FFE01C000FFFFC7C003FFFFFFC007F801FFC00FC0007FC01F00001FC0 3F00000FC07E000007C07E000007C07E000003C0FE000003C0FE000003C0FF000003C0FF 800003C0FFE0000000FFF80000007FFFC000007FFFFF00003FFFFFE0003FFFFFF8001FFF FFFC000FFFFFFF0007FFFFFF8001FFFFFFC0007FFFFFE0001FFFFFE00000FFFFF0000003 FFF00000007FF87000001FF8F000000FF8F0000007F8F8000007F8F8000003F8F8000003 F8FC000003F0FC000003F0FE000003F0FF000007E0FF800007E0FFC0000FC0FFF0001F80 FEFC00FF00F87FFFFC00F01FFFF000E003FF8000252E7CAD2E>I121 D E /Ft 8 117 df<0003FF0000001FFFF000007FFFFC0000FC07FE0001F001FF0003C0 00FF800780007FC00FE0007FE00FF0007FE01FF8007FF01FFC007FF01FFC007FF01FFC00 7FF01FFC007FF00FF8007FF007F0007FE003E0007FE0008000FFE0000000FFC0000000FF 80000001FF00000001FE00000003FC00000007F80000001FF000000FFFC000000FFF8000 000FFFF000000007FE00000000FF000000007FC00000007FE00000003FF00000003FF800 00003FF80000001FFC0000001FFC0000001FFE0000001FFE1F80001FFE3FC0001FFE7FE0 001FFEFFF0001FFEFFF0001FFEFFF0001FFCFFF0001FFCFFF0001FFCFFE0003FF87FC000 3FF87F80007FF03E00007FE01FC000FFC00FF803FF8007FFFFFF0001FFFFFC00007FFFF0 000007FF000027397CB730>51 D<0000001FFE000060000003FFFFE000E000001FFFFFF8 01E000007FFFFFFE03E00001FFFC00FF07E00007FFC0001FCFE0000FFF000007FFE0001F FC000001FFE0003FF0000000FFE0007FE00000007FE000FFC00000003FE001FF80000000 1FE003FF000000000FE007FF000000000FE00FFE0000000007E00FFE0000000007E01FFC 0000000003E01FFC0000000003E03FF80000000003E03FF80000000001E03FF800000000 01E07FF80000000001E07FF80000000001E07FF00000000000007FF0000000000000FFF0 000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF000000000 0000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0 000000000000FFF00000000000007FF00000000000007FF00000000000007FF800000000 00007FF80000000001E03FF80000000001E03FF80000000001E03FF80000000001E01FFC 0000000001E01FFC0000000003C00FFE0000000003C00FFE0000000003C007FF00000000 078003FF00000000078001FF800000000F0000FFC00000001F00007FE00000003E00003F F00000007C00001FFC000000F800000FFF000003F0000007FFC0000FE0000001FFFC007F 800000007FFFFFFF000000001FFFFFFC0000000003FFFFE000000000001FFE0000003B3D 7BBB46>67 D<003FFE00000001FFFFE0000007FFFFF800000FE007FC00000FF001FE0000 1FF800FF00001FF8007F80001FF8007FC0001FF8003FC0000FF0003FE00007E0003FE000 03C0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000FFFFE000 001FFFFFE000007FF83FE00003FF803FE00007FC003FE0000FF0003FE0001FE0003FE000 3FE0003FE0007FC0003FE0007FC0003FE000FF80003FE000FF80003FE000FF80003FE000 FF80003FE000FF80007FE0007FC0007FE0007FC000DFE0003FE0039FF0001FF80F0FFFE0 07FFFE0FFFE001FFFC07FFE0003FE000FFE02B267DA52F>97 D<0001FF8000000FFFF000 007FFFFC0000FF81FE0003FE007F8007F8003F800FF0001FC00FF0000FE01FE0000FE03F E0000FF03FE00007F07FC00007F07FC00007F87FC00007F8FFC00007F8FFC00007F8FFFF FFFFF8FFFFFFFFF8FFFFFFFFF8FFC0000000FFC0000000FFC0000000FFC00000007FC000 00007FC00000007FC00000003FE00000003FE00000781FE00000781FF00000780FF00000 F007F80001F003FC0003E001FE000FC000FFC07F80003FFFFE00000FFFF8000000FFC000 25267DA52C>101 D<00FE00000000FFFE00000000FFFE00000000FFFE00000000FFFE00 00000007FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE00 00000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE00 00000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE00 FF800003FE03FFF00003FE0FFFF80003FE1E03FC0003FE3801FE0003FE6001FF0003FEC0 00FF0003FFC000FF8003FF8000FF8003FF0000FF8003FF0000FF8003FF0000FF8003FE00 00FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE00 00FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE00 00FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE00 00FF8003FE0000FF8003FE0000FF8003FE0000FF80FFFFF83FFFFEFFFFF83FFFFEFFFFF8 3FFFFEFFFFF83FFFFE2F3C7CBB36>104 D<00FE01FF0000FFFE1FFFF000FFFE7FFFFC00 FFFEFC07FE00FFFFF001FF0007FFC0007F8003FF00003FC003FE00003FE003FE00001FF0 03FE00001FF803FE00000FF803FE00000FFC03FE00000FFC03FE000007FC03FE000007FE 03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE 03FE000007FE03FE000007FE03FE000007FC03FE00000FFC03FE00000FFC03FE00000FFC 03FE00000FF803FE00001FF003FE00001FF003FF00003FE003FF80007FC003FFC000FF80 03FFE001FF0003FEFC07FE0003FE7FFFF80003FE1FFFE00003FE03FE000003FE00000000 03FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE00000000 03FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE00000000 FFFFF8000000FFFFF8000000FFFFF8000000FFFFF80000002F377DA536>112 D<01FC03F000FFFC0FFC00FFFC1FFF00FFFC3C3F80FFFC707F8007FCE0FFC003FCC0FFC0 03FD80FFC003FD80FFC003FF807F8003FF003F0003FF001E0003FF00000003FE00000003 FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE 00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00 000003FE00000003FE00000003FE00000003FE00000003FE000000FFFFFC0000FFFFFC00 00FFFFFC0000FFFFFC000022267DA528>114 D<000F0000000F0000000F0000000F0000 000F0000001F0000001F0000001F0000001F0000003F0000003F0000007F0000007F0000 00FF000001FF000003FF000007FF00001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C00FF0078 00FF8078007F80F0003FC1E0001FFFC0000FFF800001FE001E377EB626>116 D E /Fu 81 124 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<70F8F8F8F8F8F8F87070707070707070707020202020 20000000000070F8F8F87005217CA00D>33 D<7038F87CFC7EFC7E743A04020402040208 04080410081008201040200F0E7E9F17>I<70F8FCFC74040404080810102040060E7C9F 0D>39 D<0020004000800100020006000C000C0018001800300030003000700060006000 6000E000E000E000E000E000E000E000E000E000E000E000E00060006000600070003000 30003000180018000C000C000600020001000080004000200B2E7DA112>I<8000400020 00100008000C00060006000300030001800180018001C000C000C000C000E000E000E000 E000E000E000E000E000E000E000E000E000C000C000C001C00180018001800300030006 0006000C00080010002000400080000B2E7DA112>I<01800180018001800180C183F18F 399C0FF003C003C00FF0399CF18FC1830180018001800180018010147DA117>I<000600 000006000000060000000600000006000000060000000600000006000000060000000600 000006000000060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600 000006000000060000000600000006000000060000000600000006000000060000000600 00000600000006000000060000000600001C207D9A23>I<70F8FCFC7404040408081010 2040060E7C840D>II<70F8F8F87005057C840D>I<0001000300 03000600060006000C000C000C00180018001800300030003000600060006000C000C000 C00180018001800300030003000600060006000C000C000C001800180018003000300030 00600060006000C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070 0380700380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C0F003C07003807003807003807807803807001806001C0E000E 1C0003F000121F7E9D17>I<018003800F80F38003800380038003800380038003800380 038003800380038003800380038003800380038003800380038003800380038007C0FFFE 0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803 C02007C00007C0000780000780000F00000E00001C0000380000700000600000C0000180 000300000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C 1C00100E00200F00780F80780780780780380F80000F80000F00000F00000E00001C0000 380003F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0 0780400780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00 001E00002E00002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00 100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00 000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000 0010000010000010000011F000161C00180E001007001007800003800003800003C00003 C00003C07003C0F003C0F003C0E00380400380400700200600100E000C380003E000121F 7E9D17>I<007C000182000701000E03800C07801C078038030038000078000070000070 0000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C070 03C07003C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<400000 7FFFC07FFF807FFF80400100800200800200800400000800000800001000002000002000 00400000400000C00000C00001C000018000038000038000038000038000078000078000 078000078000078000078000078000030000121F7D9D17>I<03F0000C0C001006003003 002001806001806001806001807001807803003E03003F06001FC8000FF00003F80007FC 000C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001 001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C0038060038070070070070 0380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E 13C003E3C0000380000380000380000700300700780600780E00700C002018001070000F C000121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70 F8F8F8700000000000000000000070F0F8F878080808101010202040051D7C930D>I<7F FFFFE0FFFFFFF00000000000000000000000000000000000000000000000000000000000 000000FFFFFFF07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C00 38007000E000C00180018001000300020002000200020002000200000000000000000000 0007000F800F800F8007000E207D9F15>63 D<0001000000038000000380000003800000 07C0000007C0000007C0000009E0000009E0000009E0000010F0000010F0000010F00000 207800002078000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001 000F0001000F0001000F00020007800200078002000780040003C00E0003C01F0007E0FF C03FFE1F207F9F22>65 DI<000F C040007030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C00 00407C00004078000040F8000000F8000000F8000000F8000000F8000000F8000000F800 0000F8000000F8000000780000007C0000407C0000403C0000401C0000401E0000800E00 0080070001000380020001C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E0 04E0038002E0070001E00F0000E01E0000601E0000603C0000603C0000207C0000207800 0020F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF800 03E0780001E07C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E00380 02E000E0046000781820000FE0001E217D9F24>III75 DIII<001F 800000F0F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C00 03E0780001E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F800 01F0F80001F0F80001F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E00 07000F000F0007801E0001C0380000F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E0 0080F00000F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003C000 03C00001C08001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012 217D9F19>I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F 0010800F0010800F0010000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF80FE0 0FC007C0070003C0020001E0040001F00C0000F0080000781000007C1000003C2000003E 4000001E4000000F8000000F8000000780000003C0000007E0000005E0000009F0000018 F8000010780000207C0000603C0000401E0000801F0001800F0001000780020007C00700 03C01F8007E0FFE01FFE1F1F7F9E22>I91 D<080410082010201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>I< FEFE06060606060606060606060606060606060606060606060606060606060606060606 06060606060606FEFE072D7FA10D>I<1FE000303000781800781C00300E00000E00000E 00000E0000FE00078E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867 200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E0078 0E00780E00780E00780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E00 15207F9F19>I<03F80E0C1C1E381E380C70007000F000F000F000F000F000F000700070 00380138011C020E0C03F010147E9314>I<000380003F80000380000380000380000380 00038000038000038000038000038000038003E380061B801C0780380380380380700380 700380F00380F00380F00380F00380F00380F003807003807003803803803807801C0780 0E1B8003E3F815207E9F19>I<03F0000E1C001C0E00380700380700700700700380F003 80F00380FFFF80F00000F00000F000007000007000003800801800800C010007060001F8 0011147F9314>I<007C00C6018F038F07060700070007000700070007000700FFF00700 070007000700070007000700070007000700070007000700070007000700070007007FF0 1020809F0E>I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F 00380E001C1C001E380033E0002000002000003000003000003FFE001FFF800FFFC03001 E0600070C00030C00030C00030C000306000603000C01C038003FC00141F7F9417>I<0E 0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E 3E000E43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E00 1E001C000000000000000000000000000E007E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000 E0000000000000000000000000007007F000F00070007000700070007000700070007000 700070007000700070007000700070007000700070007000706070F060F0C061803F000C 28829E0E>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF800 0F38000E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18> I<0E00FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I< 0E1F01F000FE618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E 3E00FE43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E00 1C03803801C03801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E0 7000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00 E00E00E00E00F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E00 E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E00 00FFE000151D7F9319>I<03E0800619801C05803C0780380380780380700380F00380F0 0380F00380F00380F00380F003807003807803803803803807801C0B800E138003E38000 0380000380000380000380000380000380000380000380003FF8151D7E9318>I<0E78FE 8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FF E00F147F9312>I<1F9030704030C010C010C010E00078007F803FE00FF0007080388018 8018C018C018E030D0608F800D147E9312>I<020002000200060006000E000E003E00FF F80E000E000E000E000E000E000E000E000E000E000E000E080E080E080E080E08061003 1001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC1614 7F9319>II< FF9FE1FC3C0780701C0300601C0380200E0380400E0380400E03C0400707C0800704C080 0704E080038861000388710003C8730001D0320001D03A0000F03C0000E01C0000E01C00 00601800004008001E147F9321>I<7FC3FC0F01E00701C007018003810001C20000E400 00EC00007800003800003C00007C00004E000087000107000303800201C00601E01E01E0 FF07FE1714809318>II<3FFF380E 200E201C40384078407000E001E001C00380078007010E011E011C0338027006700EFFFE 10147F9314>II E /Fv 46 122 df<70F8FCFC7404040404080810 102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000 071C000C06001803003803803803807001C07001C07001C07001C0F001E0F001E0F001E0 F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C0 7001C07001C07803C03803803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038007C0FFFE0F217CA018 >I<03F0000C1C001007002007804003C04003C08003E0F003E0F801E0F801E0F801E020 03E00003E00003C00003C0000780000700000E00001C0000180000300000600000C00001 80000100000200200400200800201800603000403FFFC07FFFC0FFFFC013217EA018>I< 03F8000C1E001007002007804007C07807C07803C07807C03807C0000780000780000700 000F00000E0000380003F000001C00000F000007800007800003C00003C00003E02003E0 7003E0F803E0F803E0F003C04003C0400780200780100F000C1C0003F00013227EA018> I<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>I<10 00801E07001FFF001FFE001FF80013E00010000010000010000010000010000010000010 F800130E001407001803801003800001C00001C00001E00001E00001E00001E07001E0F0 01E0F001E0E001C08001C04003C04003802007001006000C1C0003F00013227EA018>I< 007E0001C1000300800601C00E03C01C03C0180180380000380000780000700000700000 F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013227EA018> I<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000 E0C00060C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018 >56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C0 0001C00001C0000380000380300300780700780600700C002018001030000FC00013227E A018>I<0001800000018000000180000003C0000003C0000003C0000005E0000005E000 000DF0000008F0000008F0000010F800001078000010780000203C0000203C0000203C00 00401E0000401E0000401E0000800F0000800F0000FFFF000100078001000780030007C0 020003C0020003C0040003E0040001E0040001E00C0000F00C0000F03E0001F8FF800FFF 20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E000070 1E0000701C0000303C0000303C0000307C0000107800001078000010F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F800000078000000780000107C000010 3C0000103C0000101C0000201E0000200E000040070000400380008001C0010000E00200 00381C000007E0001C247DA223>67 D69 DI73 D77 DI80 D82 D<03F0200C0C601802603001E07000E06000 60E00060E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF 0003FF80003FC00007E00001E00000F00000F0000070800070800070800070800070C000 60C00060E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF8780780786007 8018400780084007800840078008C007800C800780048007800480078004800780040007 800000078000000780000007800000078000000780000007800000078000000780000007 800000078000000780000007800000078000000780000007800000078000000780000007 80000007800000078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E 0018070000070000070000070000FF0007C7001E07003C0700780700700700F00708F007 08F00708F00F087817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C0 0E80600F00300E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E 0E001C0E003C0E00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07 801C0780380300780000700000F00000F00000F00000F00000F00000F00000F000007000 007800403800401C00800C010007060001F80012157E9416>I<0000E0000FE00001E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E001F8E007 04E00C02E01C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F0 00E07000E07800E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC00070700 0C03801C01C03801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000 7000007800203800201C00400E008007030000FC0013157F9416>I<003C00C6018F038F 030F070007000700070007000700070007000700FFF80700070007000700070007000700 0700070007000700070007000700070007000700070007807FF8102380A20F>I<000070 01F198071E180E0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E00 0F1C0019F0001000001000001800001800001FFE000FFFC00FFFE03800F0600030400018 C00018C00018C000186000306000303800E00E038003FE0015217F9518>I<0E0000FE00 001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E1F800E60C00E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C 001E003E001E001C00000000000000000000000000000000000E00FE001E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E >I<01C003E003E003E001C00000000000000000000000000000000001E00FE001E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01 F00E01C00E01800E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07 000E07800E03C00E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1F C07F00FE60E183801E807201C00F003C00E00F003C00E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A >I<0E1F80FE60C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I< 01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F00078 F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F 00FE61C00E80600F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E00 1E0E001E0E003C0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00 000E00000E00000E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F06 0F000E000E000E000E000E000E000E000E000E000E000E000E000E000F00FFF010157F94 13>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C80 0CC00CC008E018D0308FC00E157E9413>I<02000200020002000600060006000E001E00 3E00FFF80E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E04 0E040708030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F0060170 03827800FC7F18157F941B>III121 D E /Fw 20 122 df45 D<007F800001FFF0000701F8000C00 7E0018001F0010000F0020000F80400007C0400007C0400003E0B00003E0F80003E0FC00 03E0FC0003E0FC0003E0780003E0000003E0000007C0000007C00000078000000F800000 0F0000001F0000001E0000003C0000007800000070000000E0000001C000000380000007 0000000E0000000C000000180000003000000060000000C0002001800020030000200600 00400C00004008000040100000C03FFFFFC07FFFFF80FFFFFF80FFFFFF801B2F7DAE21> 50 D<00003FE0010001FFF8030007F01E03001F800307003E000087007800004F00F000 002F01E000001F03C000000F078000000F0F800000070F000000071F000000031E000000 033E000000033C000000017C000000017C000000017C000000017800000000F800000000 F800000000F800000000F800000000F800000000F800000000F800000000F800000000F8 00000000F800000000F80000000078000000007C000000007C000000017C000000013C00 0000013E000000011E000000011F000000020F000000020F80000006078000000403C000 000801E000000800F00000100078000020003E0000C0001F8003800007F00F000001FFFC 0000003FE00028337CB130>67 DI 73 D77 D 80 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E 001E0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F 001E003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478 003E047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E060038018 0700040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000 F80000F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F0002 07000403801800E060003F80181F7D9E1D>99 D<003F800000E0E0000380380007003C00 0E001E001E001E001C000F003C000F007C000F0078000F8078000780F8000780F8000780 FFFFFF80F8000000F8000000F8000000F8000000F8000000F8000000780000007C000000 3C0000003C0000801E0000800E0001000F0002000780020001C00C0000F03000001FC000 191F7E9E1D>101 D<0780000000FF80000000FF800000000F8000000007800000000780 000000078000000007800000000780000000078000000007800000000780000000078000 00000780000000078000000007800000000780000000078000000007800000000780FE00 000783078000078C03C000079001E00007A001E00007A000F00007C000F00007C000F000 078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F0000FC001F800FFFC1F FF80FFFC1FFF8021327EB125>104 D<07000F801F801F800F8007000000000000000000 00000000000000000000000000000780FF80FF800F800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800780078007800FC0 FFF8FFF80D307EAF12>I<0780FE001FC000FF83078060F000FF8C03C18078000F9001E2 003C0007A001E4003C0007A000F4001E0007C000F8001E0007C000F8001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00 078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E00078000F0001E000FC001F8003F00FFFC 1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C000 0F9001E00007A001E00007A000F00007C000F00007C000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF80211F7E9E 25>I<001FC00000F0780001C01C00070007000F0007801E0003C01C0001C03C0001E03C 0001E0780000F0780000F0780000F0F80000F8F80000F8F80000F8F80000F8F80000F8F8 0000F8F80000F8F80000F8780000F07C0001F03C0001E03C0001E01E0003C01E0003C00F 00078007800F0001C01C0000F07800001FC0001D1F7E9E21>I<0781FC00FF860700FF88 03C00F9001E007A000F007C00078078000780780003C0780003C0780003E0780001E0780 001F0780001F0780001F0780001F0780001F0780001F0780001F0780001F0780001F0780 003E0780003E0780003C0780007C0780007807C000F007A000F007A001E0079803800786 0F000781F800078000000780000007800000078000000780000007800000078000000780 00000780000007800000078000000FC00000FFFC0000FFFC0000202D7E9E25>I<0783E0 FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000078000078000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0 300070600030E00030E00010E00010E00010F00010F800007E00003FF0001FFF000FFFC0 03FFE0003FF00001F80000F880003C80003C80001CC0001CC0001CE0001CE00018F00038 F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000400000400000400000C0 0000C00000C00001C00001C00003C00007C0000FC0001FFFE0FFFFE003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C01003C01003C01003C01003C01003C01003C01003C01001C02001E02000E0 400078C0001F00142C7FAB19>I121 D E /Fx 5 85 df<00000000600000000000E00000000000E00000000001E00000000003 E00000000003E00000000007E00000000007F0000000000DF0000000000DF00000000019 F00000000039F00000000031F00000000061F00000000061F000000000C1F000000000C1 F00000000181F00000000381F00000000301F00000000601F00000000601F00000000C01 F80000000C00F80000001800F80000003800F80000003000F80000006000F80000006000 F8000000C000F8000000C000F8000001FFFFF8000001FFFFF80000030000F80000060000 F80000060000F800000C0000F800000C0000FC00001800007C00001800007C0000300000 7C00007000007C00006000007C0000C000007C0000C000007C0001C000007C0003C00000 7C000FE00000FE00FFF8001FFFE0FFF8001FFFE02B327CB134>65 D<000FFFFFFC0000000FFFFFFF800000007E000FE00000007C0003F00000007C0000F800 00007C00007C0000007C00007C000000F800003E000000F800001E000000F800001F0000 00F800001F000001F000001F000001F000000F800001F000000F800001F000000F800003 E000000F800003E000000F800003E000000F800003E000001F800007C000001F800007C0 00001F800007C000001F800007C000001F80000F8000001F00000F8000003F00000F8000 003F00000F8000003F00001F0000007E00001F0000007E00001F0000007E00001F000000 7C00003E000000FC00003E000000F800003E000001F800003E000001F000007C000003E0 00007C000003E000007C000007C000007C00000F800000F800000F000000F800001E0000 00F800003C000000F8000078000001F00001F0000001F00003E0000001F0000F80000003 F0007F000000FFFFFFFC000000FFFFFFC000000031317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007E0003F800007C0000F800007C00007800007C0000 7800007C0000300000F80000300000F80000300000F80000300000F80000300001F00000 300001F00000300001F00000300001F00000600003E00180600003E00180000003E00180 000003E00180000007C00300000007C00300000007C00700000007C01F0000000FFFFE00 00000FFFFE0000000F801E0000000F800E0000001F000C0000001F000C0000001F000C00 00001F000C0000003E00180000003E00180000003E00000000003E00000000007C000000 00007C00000000007C00000000007C0000000000F80000000000F80000000000F8000000 0000F80000000001F00000000001F00000000001F00000000003F000000000FFFFE00000 00FFFFE00000002D317BB02E>70 D<000FFFFFE000000FFFFFFC0000007E003F0000007C 000F8000007C0007C000007C0003E000007C0003E00000F80001F00000F80001F00000F8 0001F00000F80003F00001F00003F00001F00003F00001F00003F00001F00003E00003E0 0007E00003E00007C00003E0000FC00003E0000F800007C0001F000007C0003E000007C0 0078000007C001E000000F800F8000000FFFFC0000000F800E0000000F80070000001F00 038000001F0003C000001F0001E000001F0001E000003E0001F000003E0001F000003E00 01F000003E0001F000007C0003F000007C0003F000007C0003E000007C0003E00000F800 07E00000F80007E00000F80007E00000F80007E01801F00007E03001F00007E03001F000 07E02003F00003E060FFFF8001E0C0FFFF8000F180000000003E002D327BB033>82 D<07FFFFFFFFF007FFFFFFFFF00FE007E007F00F8007C001F00E0007C000E00C0007C000 E01C0007C000E018000F8000E018000F8000E030000F8000C030000F8000C030001F0000 C060001F0000C060001F0000C060001F0000C0C0003E00018000003E00000000003E0000 0000003E00000000007C00000000007C00000000007C00000000007C0000000000F80000 000000F80000000000F80000000000F80000000001F00000000001F00000000001F00000 000001F00000000003E00000000003E00000000003E00000000003E00000000007C00000 000007C00000000007C00000000007C0000000000F80000000000F80000000000F800000 00000F80000000001F00000000001F00000000001F00000000003F000000003FFFFF0000 003FFFFF0000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 799 911 a Fx(D)25 b(R)g(A)g(F)g(T)497 1002 y Fw(MPI-2)d(Dynamic)e(Pro)r(cess)i(Chapter)621 1196 y Fv(Message)16 b(P)o(assing)h(In)o(terface)e(F)l(orum)824 1323 y(April)g(22,)i(1996)77 1381 y(This)g(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1439 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1497 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 49 a Fu(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 361 a Ft(Chapter)31 b(3)75 568 y Fs(Dynamic)38 b(Pro)s(cesses)75 809 y Fr(3.1)59 b(Intro)r(duction)75 910 y Fu(The)15 b(MPI-1)g(message)g(passing)g(library)h(allo)o(ws)f (pro)q(cesses)h(in)g(a)f(parallel)h(program)e(to)h(comm)o(unicate)75 967 y(with)21 b(one)f(another.)34 b(MPI-1)20 b(sp)q(eci\014es)i (neither)g(ho)o(w)d(the)i(pro)q(cesses)f(are)g(created,)h(nor)f(ho)o(w) g(they)75 1023 y(establish)f(comm)o(unication.)30 b(Moreo)o(v)o(er,)17 b(an)i(MPI-1)f(application)i(is)e(static,)h(that)e(is,)i(no)g(pro)q (cesses)75 1080 y(can)c(b)q(e)h(added)g(to)f(or)f(deleted)j(from)d(an)h (application)i(while)g(it)e(is)h(running.)166 1136 y(MPI)d(users)g(ha)o (v)o(e)g(ask)o(ed)g(that)g(the)g(MPI-1)g(mo)q(del)h(b)q(e)g(relaxed)g (to)f(allo)o(w)g(dynamic)h(pro)q(cess)g(man-)75 1193 y(agemen)o(t.)19 b(The)14 b(main)h(imp)q(etus)g(comes)f(from)f(the)h (PVM)g([)p Fq(?)o Fu(])g(researc)o(h)g(e\013ort,)f(whic)o(h)h(has)g (pro)o(vided)h(a)75 1249 y(w)o(ealth)c(of)g(exp)q(erience)j(with)d (dynamic)h(pro)q(cess)g(managemen)o(t)e(and)i(resource)f(con)o(trol)g (that)g(illustrates)75 1306 y(their)16 b(b)q(ene\014ts)g(and)f(p)q (oten)o(tial)h(pitfalls.)166 1362 y(It)f(w)o(as)f(decided)j(that)e (resource)g(con)o(trol)g(is)g(outside)h(the)f(scop)q(e)g(of)g(MPI-2,)g (b)q(ecause)h(a)e(p)q(ortable)75 1418 y(in)o(terface)21 b(to)f(all)h(the)f(existing)i(and)f(future)f(resource)h(con)o(trollers) f(and)h(the)g(p)q(oten)o(tial)g(con)o(trollers)75 1475 y(w)o(as)c(deemed)i(unfeasible.)31 b(Resource)18 b(con)o(trol)g(can)g (encompass)g(a)g(wide)h(range)f(of)f(abilities)k(includ-)75 1531 y(ing)g(adding)g(and)g(deleting)g(hosts)f(from)g(an)g(en)o (vironmen)o(t,)i(reserving)e(and)h(sc)o(heduling)h(resources,)75 1588 y(managing)13 b(compute)g(partitions)g(of)g(MPP)l(,)f(and)h (returning)h(information)f(ab)q(out)g(a)o(v)m(ailable)h(resources.)166 1644 y(MPI-2)g(assumes)g(that)f(resource)h(con)o(trol)g(is)h(pro)o (vided)g(b)o(y)f(the)g(v)o(endors)g(in)h(the)f(case)g(of)g(MPP)f(or)75 1701 y(b)o(y)i(some)g(other)g(soft)o(w)o(are)e(pac)o(k)m(age)i(when)h (the)f(en)o(vironmen)o(t)h(is)g(a)e(cluster)i(of)f(hosts.)166 1757 y(MPI-2)j(do)q(es)h(pro)o(vide)g(an)f(in)o(teger)h(v)m(ariable,)h Fp(MPI)p 1074 1757 14 2 v 16 w(UNIVERSE)p 1304 1757 V 18 w(SIZE)e Fu(that)g(recommends)g(ho)o(w)75 1814 y(man)o(y)k(pro)q (cesses)h(ma)o(y)e(b)q(e)i(spa)o(wned)g(in)g(the)f(en)o(vironmen)o(t)h (alltogether.)42 b(The)22 b(addition)i(of)e(this)75 1870 y(single)e(en)o(vironmen)o(t-query)g(capabilit)o(y)g(allo)o(ws)f(man)o (y)g(existing)h(programs)e(to)g(b)q(e)i(p)q(orted)f(to)f(MPI)75 1927 y(straigh)o(tforw)o(ardly)l(.)166 1983 y(The)f(reasons)g(for)f (adding)i(dynamic)g(pro)q(cess)f(managemen)o(t)g(to)f(MPI)h(are)g(b)q (oth)g(tec)o(hnical)i(and)75 2039 y(practical.)143 2146 y Fo(\017)k Fu(Since)f(a)f(cluster)h(of)e(w)o(orkstations)g(o\013ers)g (no)h(generic)h(metho)q(d)f(to)f(start)g(up)i(a)e(MIMD)h(ap-)189 2202 y(plication,)d(the)f(inclusion)j(of)c(dynamic)i(pro)q(cess)f (managemen)o(t)f(in)i(MPI)f(pro)o(vides)g(a)f(p)q(ortable)189 2259 y(metho)q(d)f(for)g(applications)h(to)f(run)g(in)h(this)g(en)o (vironmen)o(t.)143 2352 y Fo(\017)23 b Fu(W)l(orkstation)d(net)o(w)o (ork)f(users)i(migrating)g(from)g(PVM)f(to)g(MPI)h(are)g(accustomed)g (to)f(using)189 2409 y(PVM's)15 b(capabilities)j(for)d(pro)q(cess)h (and)g(resource)g(managemen)o(t.)21 b(The)16 b(lac)o(k)g(of)f(these)h (features)189 2465 y(is)f(a)g(practical)h(stum)o(bling)g(blo)q(c)o(k)g (to)f(migration.)143 2559 y Fo(\017)23 b Fu(Imp)q(ortan)o(t)16 b(classes)h(of)g(message)f(passing)h(applications,)i(suc)o(h)e(as)f (clien)o(t-serv)o(er)i(systems)f(and)189 2616 y(task)d(farming)h(jobs,) g(require)h(dynamic)g(pro)q(cess)f(con)o(trol.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fu(2)951 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)143 49 y Fo(\017)23 b Fu(The)14 b(abilit)o(y)i(to)d (write)h(fault)h(toleran)o(t)e(applications)j(is)f(imp)q(ortan)o(t)f (in)h(unstable)g(en)o(vironmen)o(ts)189 106 y(and)k(for)g(commercial)h (applications.)33 b(MPI-1)19 b(do)q(es)h(not)e(pro)o(vide)i(mec)o (hanisms)g(for)e(building)189 162 y(fault-toleran)o(t)c(applications.) 21 b(The)14 b(mec)o(hanisms)h(required)g(to)f(supp)q(ort)g(fault)g (tolerance)h(o)o(v)o(er-)189 219 y(lap)g(with)h(those)f(needed)h(to)f (supp)q(ort)g(dynamic)h(pro)q(cess)g(managemen)o(t.)166 325 y(While)e(dynamic)g(pro)q(cess)f(managemen)o(t)f(is)i(essen)o (tial,)g(it)f(is)g(imp)q(ortan)o(t)g(not)f(to)h(compromise)g(the)75 381 y(p)q(ortabilit)o(y)j(or)f(p)q(erformance)g(of)g(MPI.)g(In)h (particular:)143 475 y Fo(\017)23 b Fu(The)f(MPI-2)f(dynamic)i(pro)q (cess)e(managemen)o(t)g(mo)q(del)i(m)o(ust)e(apply)h(to)f(the)h(v)m (ast)f(ma)s(jorit)o(y)189 532 y(of)e(curren)o(t)h(parallel)h(en)o (vironmen)o(ts.)34 b(These)20 b(include)j(ev)o(erything)d(from)f(tigh)o (tly)h(in)o(tegrated)189 588 y(MPPs)d(suc)o(h)i(as)f(the)g(In)o(tel)h (P)o(aragon)e(and)h(the)g(Meik)o(o)g(CS-2)h(to)e(heterogeneous)h(net)o (w)o(orks)f(of)189 645 y(w)o(orkstations.)143 738 y Fo(\017)23 b Fu(MPI)16 b(m)o(ust)g(not)g(tak)o(e)g(o)o(v)o(er)f(op)q(erating)i (system)f(resp)q(onsibiliti)q(es.)26 b(It)17 b(should)g(instead)g(pro)o (vide)189 795 y(a)e(clean)h(in)o(terface)f(b)q(et)o(w)o(een)h(an)f (application)i(and)e(system)g(soft)o(w)o(are.)143 889 y Fo(\017)23 b Fu(MPI)17 b(m)o(ust)g(con)o(tin)o(ue)h(to)f(guaran)o (tee)g(comm)o(unication)h(determinism,)h(i.e.,)e(dynamic)i(pro)q(cess) 189 945 y(managemen)o(t)14 b(m)o(ust)h(not)f(in)o(tro)q(duce)j(una)o(v) o(oidable)f(race)f(conditions.)143 1039 y Fo(\017)23 b Fu(MPI)15 b(m)o(ust)f(not)h(con)o(tain)h(features)e(that)h (compromise)g(p)q(erformance.)143 1133 y Fo(\017)23 b Fu(MPI-1)16 b(programs)f(m)o(ust)h(w)o(ork)g(under)h(MPI-2,)f(i.e.,)g (the)h(MPI-1)f(static)g(pro)q(cess)h(mo)q(del)h(m)o(ust)189 1189 y(b)q(e)e(a)e(sp)q(ecial)j(case)f(of)e(the)i(MPI-2)f(dynamic)h(mo) q(del.)166 1283 y(The)22 b(MPI)h(dynamic)g(pro)q(cess)f(managemen)o(t)g (mo)q(del)h(addresses)g(these)f(issues)h(in)g(t)o(w)o(o)e(w)o(a)o(ys.) 75 1339 y(First,)f(MPI)g(remains)g(primarily)h(a)e(comm)o(unication)h (library)l(.)35 b(It)19 b(do)q(es)h(not)g(manage)f(the)g(parallel)75 1396 y(en)o(vironmen)o(t)e(in)g(whic)o(h)h(a)e(parallel)i(program)e (executes,)h(though)f(it)h(pro)o(vides)g(a)g(minimal)h(in)o(terface)75 1452 y(b)q(et)o(w)o(een)e(an)f(application)h(and)g(external)g(resource) f(and)g(pro)q(cess)h(managers.)166 1509 y(Second,)h(MPI-2)g(do)q(es)g (not)f(c)o(hange)h(the)g(concept)g(of)f(comm)o(unicator.)24 b(Once)18 b(a)e(comm)o(unicator)75 1565 y(is)h(built,)g(it)g(b)q(eha)o (v)o(es)f(as)g(sp)q(eci\014ed)j(in)e(MPI-1.)23 b(A)16 b(comm)o(unicator)g(is)h(nev)o(er)f(c)o(hanged)h(once)f(created,)75 1622 y(and)f(it)h(is)g(alw)o(a)o(ys)e(created)h(using)h(deterministic)h (collectiv)o(e)g(seman)o(tics.)75 1765 y Fr(3.2)59 b(The)20 b(MPI-2)f(Dynamic)g(Pro)r(cess)i(Mo)r(del)75 1866 y Fu(The)g(MPI-2)g (dynamic)h(pro)q(cess)g(mo)q(del)g(allo)o(ws)f(for)f(the)i(creation)f (and)g(destruction)h(of)f(pro)q(cesses)75 1923 y(after)14 b(an)g(MPI)h(application)h(has)e(started.)19 b(It)14 b(pro)o(vides)h(a)g(mec)o(hanism)g(to)f(establish)h(comm)o(unication)75 1979 y(b)q(et)o(w)o(een)i(the)g(newly)h(created)f(pro)q(cesses)g(and)g (the)g(existing)h(MPI)f(application.)27 b(It)17 b(also)g(pro)o(vides)g (a)75 2036 y(mec)o(hanism)f(to)e(establish)j(comm)o(unication)f(b)q(et) o(w)o(een)f(t)o(w)o(o)f(existing)i(MPI)f(applications,)i(ev)o(en)e (when)75 2092 y(one)g(did)i(not)d(\\start")g(the)h(other.)75 2214 y Fm(3.2.1)49 b(Sta)o(rting)16 b(and)h(Managing)h(Pro)q(cesses)75 2300 y Fu(MPI)10 b(applications)i(ma)o(y)e(start)f(new)h(pro)q(cesses)h (\(including)i(non-MPI)d(pro)q(cesses\),)h(send)g(them)g(signals,)75 2356 y(and)17 b(\014nd)h(out)e(when)i(they)f(die)h(or)e(b)q(ecome)i (unreac)o(hable.)26 b(They)17 b(do)g(this)g(through)g(an)f(in)o (terface)i(to)75 2413 y(an)d(external)h(pro)q(cess)g(manager,)f(whic)o (h)h(can)g(range)f(from)g(a)g(parallel)i(op)q(erating)f(system)f (\(CMOST\))75 2469 y(to)g(la)o(y)o(ered)g(soft)o(w)o(are)e(\(POE\))i (to)g(an)g Fl(rsh)g Fu(command)g(\(p4\).)166 2526 y(There)g(are)g(t)o (w)o(o)f(w)o(a)o(ys)g(to)h(start)f(new)h(pro)q(cesses.)166 2582 y Fp(MPI)p 251 2582 14 2 v 16 w(SP)l(A)-5 b(WN)12 b Fu(starts)f(MPI)h(pro)q(cesses)g(and)g(establishes)h(comm)o (unication)f(with)g(them,)g(returning)75 2638 y(an)j(in)o(tercomm)o (unicator.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fn(3.2.)34 b(THE)15 b(MPI-2)g(D)o(YNAMIC)g(PR)o (OCESS)h(MODEL)775 b Fu(3)166 49 y Fp(MPI)p 251 49 14 2 v 16 w(SP)l(A)-5 b(WN)p 417 49 V 17 w(INDEPENDENT)p 750 49 V 17 w(MPI)9 b Fu(and)h Fp(MPI)p 1023 49 V 16 w(SP)l(A)-5 b(WN)p 1189 49 V 18 w(INDEPENDENT)p 1523 49 V 16 w(NONMPI)11 b Fu(start)d(pro-)75 106 y(cesses)18 b(but)g(do)f(not)g(establish)i(comm)o(unication)g(with)e(them.)28 b(They)17 b(return)h(a)f(group.)27 b(\(In)18 b(the)g(fol-)75 162 y(lo)o(wing,)d Fp(MPI)p 315 162 V 16 w(SP)l(A)-5 b(WN)p 481 162 V 17 w(INDEPENDENT)16 b Fu(is)g(sometimes)f(used)g(to)g (refer)g(to)g(b)q(oth)g(these)g(routines.\))166 219 y(Corresp)q(onding) c(to)e(these)i(routines)f(are)g(three)g(more)g(routines)h(that)e(start) g(sev)o(eral)i(di\013eren)o(t)f(bina-)75 275 y(ries)h(\(or)e(the)h (same)g(binary)h(with)f(di\013eren)o(t)h(argumen)o(ts\))e(at)g(the)h (same)g(time:)18 b Fp(MPI)p 1485 275 V 16 w(SP)l(A)-5 b(WN)p 1651 275 V 17 w(MUL)l(TIPLE)p Fu(,)75 332 y Fp(MPI)p 160 332 V 16 w(SP)l(A)g(WN)p 326 332 V 17 w(MUL)l(TIPLE)p 560 332 V 16 w(INDEPENDENT)p 892 332 V 16 w(MPI)15 b Fu(and)75 388 y Fp(MPI)p 160 388 V 16 w(SP)l(A)-5 b(WN)p 326 388 V 17 w(MUL)l(TIPLE)p 560 388 V 16 w(INDEPENDENT)p 892 388 V 16 w(NONMPI)p Fu(.)166 444 y(MPI)14 b(uses)h(the)g(existing)g (group)g(abstraction)f(to)g(represen)o(t)g(pro)q(cesses.)20 b(A)15 b(pro)q(cess)g(is)g(iden)o(ti\014ed)75 501 y(b)o(y)g(a)g (\(group,)f(rank\))h(pair.)75 623 y Fm(3.2.2)49 b(The)16 b(Runtime)g(Environment)75 708 y Fu(The)h Fp(MPI)p 255 708 V 16 w(SP)l(A)-5 b(WN)18 b Fu(and)f Fp(MPI)p 611 708 V 16 w(SP)l(A)-5 b(WN)p 777 708 V 17 w(INDEPENDENT)17 b Fu(routines)g(pro)o(vide)h(an)e(in)o(terface)i(b)q(et)o(w)o(een)75 765 y(MPI)d(and)h(the)f Fj(runtime)h(envir)n(onment)e Fu(of)h(an)g(MPI)g(application.)166 821 y(The)f(di\016cult)o(y)h(is)f (that)f(there)h(is)g(an)g(enormous)g(range)f(of)g(run)o(time)h(en)o (vironmen)o(ts)g(and)g(applica-)75 878 y(tion)i(requiremen)o(ts,)g(and) g(MPI)g(m)o(ust)f(not)h(b)q(e)g(tailored)h(to)e(an)o(y)g(particular)i (one.)22 b(Examples)16 b(of)g(suc)o(h)75 934 y(en)o(vironmen)o(ts)f (are:)143 1041 y Fo(\017)23 b Fq(MPP)15 b(managed)i(b)o(y)e(a)i(batc)o (h)g(queueing)f(system)p Fu(.)j(Batc)o(h)13 b(queueing)j(systems)e (generally)189 1097 y(allo)q(cate)j(resources)g(b)q(efore)g(an)g (application)i(b)q(egins,)f(enforce)f(limits)h(on)f(resource)g(use)g (\(CPU)189 1153 y(time,)f(memory)g(use,)g(etc\),)g(and)g(do)h(not)e (allo)o(w)i(a)f(c)o(hange)g(in)h(resource)f(allo)q(cation)i(after)d(a)h (job)189 1210 y(b)q(egins.)21 b(Moreo)o(v)o(er,)13 b(man)o(y)i(MPPs)f (ha)o(v)o(e)h(sp)q(ecial)h(limitations)h(or)d(extensions,)i(suc)o(h)f (as)g(a)g(limit)189 1266 y(the)c(n)o(um)o(b)q(er)h(of)f(pro)q(cesses)h (that)f(ma)o(y)g(run)h(on)g(one)f(pro)q(cessor,)h(or)f(the)h(abilit)o (y)h(to)e(gang-sc)o(hedule)189 1323 y(pro)q(cesses)k(of)g(a)g(parallel) i(application.)143 1417 y Fo(\017)23 b Fq(Net)o(w)o(ork)17 b(of)i(w)o(orkstations)g(with)h(PVM)p Fu(.)15 b(PVM)h(\(P)o(arallel)h (Virtual)h(Mac)o(hine\))e(allo)o(ws)h(a)189 1473 y(user)f(to)g(create)g (a)g(\\virtual)h(mac)o(hine")g(out)f(of)g(a)g(net)o(w)o(ork)f(of)h(w)o (orkstations.)23 b(An)16 b(application)189 1530 y(ma)o(y)i(extend)j (the)e(virtual)h(mac)o(hine)h(or)e(manage)g(pro)q(cesses)h(\(create,)f (kill,)j(redirect)f(output,)189 1586 y(etc.\))26 b(through)18 b(the)f(PVM)g(library)l(.)28 b(Requests)19 b(to)e(manage)g(the)g(mac)o (hine)i(or)e(pro)q(cesses)h(ma)o(y)189 1642 y(b)q(e)e(in)o(tercepted)g (and)f(handled)i(b)o(y)e(an)g(external)h(resource)f(manager.)143 1736 y Fo(\017)23 b Fq(Net)o(w)o(ork)10 b(of)i(w)o(orkstations)f (managed)i(b)o(y)e(a)i(load)g(balancing)h(system)p Fu(.)j(A)10 b(load)h(balanc-)189 1793 y(ing)h(system)f(ma)o(y)f(c)o(ho)q(ose)i(the) f(lo)q(cation)h(of)f(spa)o(wned)h(pro)q(cesses)f(based)h(on)f(dynamic)i (quan)o(tities,)189 1849 y(suc)o(h)19 b(as)f(load)h(a)o(v)o(erage.)29 b(It)18 b(ma)o(y)g(transparen)o(tly)g(migrate)h(pro)q(cesses)g(from)e (one)i(mac)o(hine)h(to)189 1906 y(another)14 b(when)i(a)f(resource)g(b) q(ecomes)h(una)o(v)m(ailable.)143 1999 y Fo(\017)23 b Fq(Large)j(SMP)g(with)g(Unix)p Fu(.)43 b(Applications)24 b(are)f(run)g(directly)h(b)o(y)e(the)h(user.)42 b(They)23 b(are)189 2056 y(sc)o(heduled)15 b(at)d(a)h(lo)o(w)g(lev)o(el)h(b)o(y)f (the)g(op)q(erating)h(system.)k(Pro)q(cesses)13 b(ma)o(y)g(ha)o(v)o(e)g (sp)q(ecial)h(sc)o(hedul-)189 2112 y(ing)g(c)o(haracteristics)h (\(gang-sc)o(heduling,)g(pro)q(cessor)g(a\016nit)o(y)l(,)f(deadline)i (sc)o(heduling,)g(pro)q(cessor)189 2169 y(lo)q(c)o(king,)h(etc.\))24 b(and)16 b(b)q(e)i(sub)s(ject)e(to)g(OS)h(resource)f(limits)i(\(n)o(um) o(b)q(er)f(of)f(pro)q(cesses,)h(amoun)o(t)e(of)189 2225 y(memory)l(,)f(etc.\).)166 2332 y(MPI)f(assumes,)g(implicitly)l(,)k (the)c(existence)h(of)f(an)g(en)o(vironmen)o(t)h(in)g(whic)o(h)g(an)f (application)i(runs.)75 2388 y(It)k(do)q(es)g(not)g(pro)o(vide)g(\\op)q (erating)g(system")f(services,)i(suc)o(h)g(as)e(a)h(general)g(abilit)o (y)h(to)e(query)i(what)75 2444 y(pro)q(cesses)i(are)f(running,)j(to)c (kill)k(arbitrary)c(pro)q(cesses,)j(to)e(\014nd)h(out)f(prop)q(erties)h (of)f(the)h(run)o(time)75 2501 y(en)o(vironmen)o(t)d(\(ho)o(w)f(man)o (y)g(pro)q(cessors,)g(ho)o(w)g(m)o(uc)o(h)h(memory)l(,)f(etc\).)30 b(Complex)19 b(in)o(teraction)g(of)f(an)75 2557 y(MPI)i(application)h (with)f(its)g(run)o(time)h(en)o(vironmen)o(t)f(should)h(b)q(e)f(done)g (through)g(an)g(en)o(vironmen)o(t-)75 2614 y(sp)q(eci\014c)d(API.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fu(4)951 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 49 y Fi(Discussion:)52 b Fh(An)19 b(example)e(of)h(suc)o(h)h(an)g(API)g(w)o(ould)e(b)q(e)i(the)h(PVM)f (task)f(and)h(mac)o(hine)e(manage-)75 106 y(men)o(t)e(routines)i({)f Fg(pvm)p 450 106 14 2 v 15 w(addhosts)p Fh(,)f Fg(pvm)p 734 106 V 15 w(config)p Fh(,)g Fg(pvm)p 974 106 V 16 w(tasks)p Fh(,)g(etc.,)i(p)q(ossibly)f(mo)q(di\014ed)f(to)h(return)i (an)e(MPI)75 162 y(\(group,rank\))e(when)g(p)q(ossible.)k(A)c(Condor)g (or)g(PBS)g(API)g(w)o(ould)g(b)q(e)g(another)g(p)q(ossibilit)o(y)m(.) 166 301 y Fu(A)o(t)i(some)h(lo)o(w)f(lev)o(el,)i(ob)o(viously)l(,)g (MPI)f(m)o(ust)f(b)q(e)h(able)h(to)e(in)o(teract)h(with)g(the)g(run)o (time)g(system,)75 358 y(but)h(the)f(in)o(teraction)h(is)g(not)f (visible)j(at)d(the)h(application)h(lev)o(el)g(and)e(the)h(details)g (are)g(not)f(sp)q(eci\014ed)75 414 y(b)o(y)e(the)g(MPI)h(standard.)166 547 y Fi(Discussion:)34 b Fh(It)14 b(has)g(b)q(een)h(requested)h(that)e (the)g(MPI)g(forum)e(sp)q(ecify)j(this)f(in)o(terface)g(explicitly)m(.) j(MPI)75 597 y(has)f(so)g(far)g(shied)h(a)o(w)o(a)o(y)e(from)f(sp)q (ecifying)i(implemen)o(tation)d(details,)j(for)f(go)q(o)q(d)h(reasons.) 26 b(It)16 b(w)o(ould)f(b)q(e)i(nice,)75 646 y(ho)o(w)o(ev)o(er,)d(if)f (implemen)o(tors)e(agreed)k(on)f(a)f(minim)o(al)e(standard)j(in)o (terface)g(to)g(allo)o(w)e(in)o(terop)q(erabilit)o(y)m(.)166 785 y Fu(MPI)h(do)q(es)g(not)f(require)i(the)f(existence)h(of)e(an)h (underlying)h(\\virtual)f(mac)o(hine")h(mo)q(del,)g(in)f(whic)o(h)75 842 y(there)h(is)g(a)g(consisten)o(t)g(global)g(view)h(of)e(an)h(MPI)g (application,)h(and)f(an)g(implicit)i(\\op)q(erating)e(system")75 898 y(managing)k(resources)f(and)h(pro)q(cesses.)28 b(F)l(or)17 b(instance,)i(pro)q(cesses)f(spa)o(wned)g(b)o(y)g(one)f(task)g(ma)o(y)g (not)75 955 y(b)q(e)i(visible)h(to)e(another;)h(additional)h(hosts)e (added)h(to)e(the)i(run)o(time)f(en)o(vironmen)o(t)h(b)o(y)f(one)h(pro) q(cess)75 1011 y(ma)o(y)13 b(not)h(b)q(e)h(visible)i(in)e(another)e (pro)q(cess;)i(tasks)e(spa)o(wned)i(b)o(y)f(di\013eren)o(t)g(pro)q (cesses)h(ma)o(y)e(not)h(b)q(e)h(au-)75 1068 y(tomatically)g (distributed)h(o)o(v)o(er)d(a)o(v)m(ailable)k(resources.)i(MPI)c(do)q (es)f(require,)h(ho)o(w)o(ev)o(er,)f(that)g(a)g(pro)q(cess)75 1124 y(b)q(e)k(a)o(w)o(are)e(of)h(its)g(o)o(wn)g(c)o(hanges)h(to)e(the) i(run)o(time)f(en)o(vironmen)o(t.)27 b(F)l(or)17 b(instance,)h(it)g (can)f(b)q(e)h(noti\014ed)75 1181 y(when)e(its)f(c)o(hildren)i(die;)f (tasks)e(whic)o(h)i(it)g(spa)o(wns)f(will)i(b)q(e)e(distributed)i(in)f (some)f(sane)g(manner,)g(etc.)189 1271 y Fj(A)n(dvic)n(e)22 b(to)h(implementors.)86 b Fu(A)23 b(virtual)g(mac)o(hine)h(abstraction) e(is)h(appropriate)g(in)h(most)189 1328 y(en)o(vironmen)o(ts)15 b(and)g(should)g(b)q(e)h(part)e(of)g(a)h(\\high-qualit)o(y")h(implemen) o(tation.)21 b(In)15 b(other)f(cases,)189 1384 y(pro)o(viding)j(the)f (virtual)h(mac)o(hine)g(abstraction)f(ma)o(y)g(add)h(to)q(o)e(m)o(uc)o (h)i(baggage)e(\(e.g.,)g(daemon)189 1441 y(pro)q(cesses\))g(or)g (inhibit)i(p)q(ortabilit)o(y)l(.)k(\()p Fj(End)16 b(of)g(advic)n(e)g (to)h(implementors.)p Fu(\))166 1531 y(In)o(teraction)12 b(b)q(et)o(w)o(een)g(MPI)g(and)g(the)f(run)o(time)i(en)o(vironmen)o(t)f (is)g(limited)i(to)d(the)h(follo)o(wing)g(areas:)143 1634 y Fo(\017)23 b Fu(A)17 b(pro)q(cess)h(ma)o(y)f(start)g(new)h(pro)q (cesses)g(with)g Fp(MPI)p 1100 1634 V 16 w(SP)l(A)-5 b(WN)p Fu(,)18 b Fp(MPI)p 1379 1634 V 15 w(SP)l(A)-5 b(WN)p 1544 1634 V 18 w(INDEPENDENT)189 1690 y Fu(and)21 b(their)g(m)o(ultiple-executabl)q(e)j(v)m(arian)o(ts.)37 b(It)21 b(ma)o(y)f(kill)j(pro)q(cesses)f(it)f(has)g(spa)o(wned)g(\(and) 189 1747 y(p)q(ossibly)16 b(others\))f(and)g(ma)o(y)g(b)q(e)h (noti\014ed)g(when)f(pro)q(cesses)h(die)g(or)f(b)q(ecome)h(unreac)o (hable.)143 1840 y Fo(\017)23 b Fu(When)c(a)g(pro)q(cess)g(spa)o(wns)f (a)h(c)o(hild)i(pro)q(cess,)e(it)g(optionlly)i(passes)e(a)f(string)h (to)g(the)g(run)o(time)189 1896 y(en)o(vironmen)o(t)14 b(to)g(indicate)h(where)g(the)f(pro)q(cess)g(should)i(b)q(e)e(started.) 19 b(The)c(string)f(is)g(opaque)h(to)189 1952 y(MPI.)143 2045 y Fo(\017)23 b Fu(A)16 b(v)m(ariable)h Fp(MPI)p 495 2045 V 16 w(UNIVERSE)p 725 2045 V 17 w(SIZE)p Fu(,)f(tells)h(a)e (program)g(ho)o(w)h(\\large")f(the)h(initial)i(run)o(time)e(en-)189 2101 y(vironmen)o(t)i(is,)i(namely)f(ho)o(w)f(man)o(y)h(pro)q(cesses)g (can)f(usefully)j(b)q(e)e(started)f(alltogether.)30 b(One)189 2158 y(can)17 b(subtract)g(the)h(size)g(of)f Fp(MPI)p 764 2158 V 16 w(COMM)p 922 2158 V 16 w(W)o(ORLD)h Fu(from)f(this)h(v)m (alue)g(to)f(\014nd)h(out)f(ho)o(w)g(man)o(y)189 2214 y(pro)q(cesses)e(migh)o(t)g(usefully)i(b)q(e)f(started)e(in)j(addition) f(to)e(those)h(already)h(running.)75 2336 y Fm(3.2.3)49 b(Applications)18 b(Requiring)f(Direct)e(Communication)i(with)g(the)e (Runtime)h(System)75 2421 y Fu(The)22 b(existing)g(MPI)f(sp)q (eci\014cation)j(is)e(adequate)f(for)g(most)f(parallel)j(applications.) 40 b(In)22 b(these)g(ap-)75 2478 y(plications,)d(the)e(run)o(time)h(en) o(vironmen)o(t,)g(whether)f(simple)i(or)e(elab)q(orate,)h(allo)q(cates) f(resources)h(and)75 2534 y(manages)e(user)h(pro)q(cesses)g(without)g (in)o(teracting)g(with)g(the)f(application)j(program.)k(In)17 b(other)f(appli-)75 2591 y(cations,)j(ho)o(w)o(ev)o(er,)f(it)g(is)h (necessary)g(that)e(the)i Fj(user)g(level)j Fu(of)c(the)g(application)i (comm)o(unicate)f(with)75 2647 y(the)g(run)o(time)h(en)o(vironmen)o(t.) 32 b(Here)20 b(w)o(e)f(describ)q(e)i(three)e(broad)g(classes)h(of)f (suc)o(h)h(applications.)33 b(In)75 2704 y(Section)16 b(3.5)e(w)o(e)h(will)i(giv)o(e)f(concrete)f(examples)h(of)f(eac)o(h)g (of)g(these)g(classes.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fn(3.2.)34 b(THE)15 b(MPI-2)g(D)o(YNAMIC)g(PR)o (OCESS)h(MODEL)775 b Fu(5)75 49 y Fp(T)l(ask)12 b(F)o(a)o(rming)43 b Fu(By)11 b(a)g(\\task)f(farm")g(application)i(w)o(e)f(mean)g(a)f (program)g(that)g(manages)h(the)g(execution)75 106 y(of)j(a)g(set)g(of) g(other,)g(p)q(ossibly)i(sequen)o(tial,)g(programs.)i(This)d(situation) g(often)f(arises)h(when)g(one)f(w)o(an)o(ts)75 162 y(to)19 b(run)h(the)g(same)f(sequen)o(tial)i(program)e(man)o(y)g(times)h(with)g (v)m(arying)h(input)f(data.)33 b(W)l(e)20 b(call)h(eac)o(h)75 219 y(in)o(v)o(o)q(cation)14 b(of)e(the)i(sequen)o(tial)g(program)e(a)h Fj(task)p Fu(.)19 b(It)13 b(is)h(often)f(simplest)h(to)f (\\parallelize")i(the)e(existing)75 275 y(sequen)o(tial)h(program)d(b)o (y)i(writing)g(a)f(parallel)j(\\harness")d(program)f(that)h(in)i(turn)e (dev)o(otes)h(a)f(separate,)75 332 y(transien)o(t)j(pro)q(cess)g(to)g (eac)o(h)g(task.)k(When)d(one)f(task)f(\014nishes,)i(a)f(new)g(pro)q (cess)h(is)f(started)g(to)f(execute)75 388 y(the)19 b(next)f(one.)30 b(Ev)o(en)19 b(if)g(the)f(resources)h(allo)q(cated)g(to)f(the)h(job)f (are)g(\014xed,)i(the)f(\\harness")f(pro)q(cess)75 444 y(m)o(ust)13 b(in)o(teract)g(frequen)o(tly)h(with)g(the)f(pro)q(cess)h (manager)f(\(ev)o(en)g(if)h(this)g(is)g(just)f Fl(rsh)p Fu(,)g(to)g(start)f(the)i(new)75 501 y(pro)q(cesses)i(with)f(the)h(new) f(input)h(data\).)j(In)d(man)o(y)f(cases)g(this)h(harness)f(can)g(b)q (e)h(written)g(in)g(a)e(simple)75 557 y(scripting)k(language)f(lik)o(e) g Fl(csh)g Fu(or)f Fl(perl)p Fu(,)g(but)h(some)f(users)h(prefer)g(to)f (use)h(F)l(ortran)e(or)h(C.)g(Note)h(that)75 614 y(it)f(is)g(an)g (explicit)i(goal)d(of)h(the)f(MPI)h(dynamic)h(pro)q(cess)f(arc)o (hitecture)g(to)f(allo)o(w)h(the)f(managemen)o(t)g(of)75 670 y(non-MPI)h(pro)q(cesses.)75 790 y Fp(Dynamic)f(numb)q(er)i(of)e(p) o(ro)q(cesses)j(in)e(pa)o(rallel)f(job)46 b Fu(The)16 b(program)f(wishes)i(to)e(decide)j Fj(inside)g Fu(the)e(pro-)75 847 y(gram)e(to)g(adjust)g(the)g(n)o(um)o(b)q(er)h(of)f(pro)q(cesses)h (to)f(\014t)h(the)f(size)i(of)e(the)h(problem.)20 b(F)l(urthermore,)14 b(it)h(ma)o(y)75 903 y(con)o(tin)o(ue)k(to)g(add)g(and)g(subtract)f (pro)q(cesses)h(during)h(the)f(computation)g(to)f(\014t)h(separate)f (phases)h(of)75 960 y(the)g(computation,)h(some)f(of)g(whic)o(h)h(ma)o (y)f(b)q(e)h(more)f(parallel)i(than)e(others.)32 b(In)20 b(order)f(to)g(do)g(this,)75 1016 y(the)i(application)i(program)d(will) j(ha)o(v)o(e)e(to)g(in)o(teract)g(with)h(the)f(resource)g(manager)g (\(ho)o(w)o(ev)o(er)f(it)h(is)75 1073 y(implemen)o(ted\))e(to)d (request)i(and)f(acquire)h(or)f(return)g(computational)h(resources.)26 b(It)17 b(will)i(also)e(ha)o(v)o(e)75 1129 y(to)f(in)o(teract)h(with)g (the)g(pro)q(cess)g(manager)g(to)f(request)h(that)f(pro)q(cesses)h(b)q (e)h(started)e(and)h(in)h(order)f(to)75 1186 y(mak)o(e)12 b(the)g(new)h(pro)q(cesses)f(kno)o(wn)g(to)g(the)g(message-passing)g (library)h(so)f(that)g(the)g(larger)g(\(or)g(smaller\))75 1242 y(group)j(of)g(pro)q(cesses)g(can)h(comm)o(unicate.)166 1298 y(An)21 b(imp)q(ortan)o(t)f(t)o(yp)q(e)h(of)f(dynamic)h (application)i(is)e(a)f(sca)o(v)o(enger)g(application.)38 b(A)20 b(sca)o(v)o(enger)75 1355 y(application)i(is)f(\\em)o (barrassingly)f(parallel")i(in)f(the)g(sense)g(that)e(it)i(p)q(erforms) f(a)g(large)h(n)o(um)o(b)q(er)f(of)75 1411 y(completely)d(indep)q (enden)o(t)i(tasks.)i(If)c(the)f(n)o(um)o(b)q(er)g(of)g(tasks)f(is)i (large)f(enough,)g(suc)o(h)g(an)g(application)75 1468 y(can)k(mak)o(e)g(use)h(of)f(an)o(y)g(resources)g(that)g(b)q(ecome)h(a) o(v)m(ailable.)37 b(Con)o(v)o(ersely)l(,)21 b(it)g(can)g(easily)g(giv)o (e)f(up)75 1524 y(resources)i(to)g(another)g(application.)42 b(Sca)o(v)o(enger)22 b(applications)i(are)e(excellen)o(t)i(for)e (\014lling)i(in)f(the)75 1581 y(\\holes")15 b(on)g(a)g(space-shared)h (parallel)h(mac)o(hine,)f(allo)o(wing)g(it)f(to)g(ac)o(hiev)o(e)g(v)o (ery)g(high)h(utilization.)75 1701 y Fp(Client/Server)46 b Fu(This)15 b(situation)h(is)f(the)g(opp)q(osite)h(of)e(the)h (situations)g(ab)q(o)o(v)o(e,)g(where)g(pro)q(cesses)g(come)75 1757 y(and)21 b(go)f(up)q(on)h(request.)37 b(In)21 b(the)g(clien)o (t/serv)o(er)g(mo)q(del,)i(one)e(set)f(of)g(pro)q(cesses)h(is)g (relativ)o(ely)h(p)q(er-)75 1814 y(manen)o(t)17 b(\(the)g(serv)o(er,)g (whic)o(h)h(w)o(e)f(assume)g(here)h(ma)o(y)e(b)q(e)i(a)f(parallel)i (program\).)24 b(A)o(t)17 b(unpredictable)75 1870 y(times,)e(another)f (\(p)q(ossibly)h(parallel\))h(program)d(\(the)i(clien)o(t\))g(b)q (egins)h(execution)f(and)g(m)o(ust)f(establish)75 1927 y(comm)o(unication)k(with)f(the)g(serv)o(er.)26 b(In)17 b(this)h(case)f(the)g(pro)q(cess)g(manager)g(m)o(ust)f(pro)o(vide)i(a)f (w)o(a)o(y)f(for)75 1983 y(the)e(clien)o(t)h(to)e(lo)q(cate)h(the)g (serv)o(er)f(and)h(comm)o(unicate)g(to)f(the)g(message-passing)h (library)h(that)e(it)h(m)o(ust)75 2039 y(no)o(w)h(supp)q(ort)g(comm)o (unications)h(with)f(a)g(new)h(collection)h(of)d(pro)q(cesses.)166 2096 y(It)19 b(is)g(curren)o(tly)h(p)q(ossible)g(to)f(write)g(the)g (parallel)h(clien)o(ts)g(and)f(serv)o(ers)g(in)g(MPI,)g(but)g(b)q (ecause)75 2152 y(MPI)g(do)q(es)g(not)g(pro)o(vide)h(the)f(necessary)g (in)o(terfaces)g(b)q(et)o(w)o(een)h(the)f(application)i(program)d(and)h (the)75 2209 y(resource)f(manager)g(or)g(pro)q(cess)h(manager,)f(other) g(nonp)q(ortable,)h(mac)o(hine)g(sp)q(eci\014c)i(libraries)f(m)o(ust)75 2265 y(b)q(e)15 b(called)h(in)f(order)f(for)g(the)g(clien)o(t)h(and)g (serv)o(er)f(to)f(comm)o(unicate)i(with)g(one)f(another.)19 b(On)c(the)f(other)75 2322 y(hand,)g(MPI)g(do)q(es)g(con)o(tain)g(sev)o (eral)f(features)h(that)f(mak)o(e)g(it)h(relativ)o(ely)h(easy)e(to)g (add)h(suc)o(h)g(in)o(terfaces,)75 2378 y(and)h(w)o(e)g(prop)q(ose)h(b) q(oth)f(a)g(simple)h(in)o(terface)g(and)f(a)g(more)g(complex)h(but)f (\015exible)j(one.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fu(6)951 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fr(3.3)59 b(Pro)r(cess)21 b(Manager)e(Interface) 75 152 y Fm(3.3.1)49 b(Pro)q(cesses)15 b(in)i(MPI)75 238 y Fu(A)22 b(pro)q(cess)h(is)f(represen)o(ted)h(in)g(MPI)f(b)o(y)g (a)g(\(group,)h(rank\))f(pair.)41 b(A)22 b(pro)q(cess)h(ma)o(y)e(or)h (ma)o(y)f(not)75 295 y(b)q(e)f(an)g(\\MPI)f(pro)q(cess")g(in)i(that)e (it)g(ma)o(y)g(or)g(ma)o(y)g(not)g(call)i Fp(MPI)p 1259 295 14 2 v 16 w(INIT)e Fu(and)g(th)o(us)h(b)q(e)g(assigned)g(an)75 351 y Fp(MPI)p 160 351 V 16 w(COMM)p 318 351 V 16 w(W)o(ORLD)p Fu(.)15 b(MPI)g(can't)g(comm)o(unicate)g(directly)i(with)e(a)g(pro)q (cess)g(unless)i(it)e(has)g(a)g(com-)75 408 y(m)o(unicator,)21 b(but)f(an)g(application)i(can)e(send)h(signals)g(to)f(an)o(y)g(pro)q (cess)g(if)h(can)f(iden)o(tify)h(through)f(a)75 464 y(\(group,)13 b(rank)h(pair\).)19 b(Note)13 b(that)g(a)h(\(group,)f(rank\))g(iden)o (ti\014cation)j(is)e(not)f(unique)j(b)q(ecause)e(a)g(pro)q(cess)75 520 y(ma)o(y)g(b)q(elong)j(to)d(sev)o(eral)i(groups.)75 642 y Fm(3.3.2)49 b(Sta)o(rting)16 b(Pro)q(cesses)f(and)h(Establishing) i(Communication)f(With)f(Them)75 728 y Fu(The)k(follo)o(wing)g(routine) g(starts)e(a)h(n)o(um)o(b)q(er)g(of)g(MPI)h(pro)q(cesses,)g(and)g (establishes)g(comm)o(unication)75 784 y(with)c(them.)75 935 y Fp(MPI)p 160 935 V 16 w(SP)l(A)-5 b(WN\(command)p 529 935 V 16 w(line,)15 b(minp)o(ro)q(cs,)f(maxp)o(ro)q(cs,)h(info,)f (ro)q(ot,)h(comm,)e(intercomm,)75 992 y(a)o(rra)o(y)p 173 992 V 15 w(of)p 225 992 V 16 w(errco)q(des\))117 1069 y Fh(IN)155 b Fp(command)p 506 1069 V 16 w(line)321 b Fh(executable)18 b(and)f(argumen)o(ts,)g(in)g(a)f(single)h(string)g (\(string,)905 1126 y(signi\014can)o(t)d(only)f(at)g(ro)q(ot\))117 1201 y(IN)155 b Fp(minp)o(ro)q(cs)417 b Fh(minim)n(um)14 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)c(start)i(\(in)o(teger,)g (sig-)905 1257 y(ni\014can)o(t)14 b(only)f(at)h(ro)q(ot\))117 1332 y(IN)155 b Fp(maxp)o(ro)q(cs)408 b Fh(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g(\(in)o(teger,)h (sig-)905 1389 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\))117 1464 y(IN)155 b Fp(info)516 b Fh(a)14 b(string)g(telling)f(the)i(run)o (time)e(system)g(where)j(and)d(ho)o(w)h(to)905 1520 y(start)h(the)f (pro)q(cesses)j(\(string,)c(signi\014can)o(t)h(only)f(at)h(ro)q(ot\)) 117 1595 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in) d(whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 1652 y(\(in)o(teger\))117 1727 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 1802 y(OUT)108 b Fp(intercomm)384 b Fh(in)o(tercomm)o (unicator)12 b(b)q(et)o(w)o(een)j(original)d(group)i(and)f(the)905 1859 y(newly)h(spa)o(wned)g(group)g(\(handle\))117 1934 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1934 V 15 w(of)p 468 1934 V 16 w(errco)q(des)268 b Fh(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))75 2058 y Fl(int)23 b(MPI)p 245 2058 15 2 v 17 w(Spawn\(char*)g(command)p 717 2058 V 16 w(line,)g(int)g(minprocs,)g(int)h(maxprocs,)e(char*)h (info,)393 2115 y(int)h(root,)f(MPI)p 707 2115 V 17 w(Comm)g(comm,)g (MPI)p 1058 2115 V 17 w(Comm*)g(intercomm,)393 2171 y(int*)g(array)p 635 2171 V 17 w(of)p 700 2171 V 17 w(errcodes\))75 2258 y(MPI)p 150 2258 V 17 w(SPAWN\(COMMAND)p 479 2258 V 15 w(LINE,)g(MINPROCS,)g(MAXPROCS,)g(INFO,)g(ROOT,)g(COMM,)g(INTERCOMM,) 393 2314 y(ARRAY)p 516 2314 V 17 w(OF)p 581 2314 V 17 w(ERRCODES,)f(IERR\))170 2370 y(CHARACTER*\(*\))h(COMMAND)p 676 2370 V 16 w(LINE,)g(INFO)170 2427 y(INTEGER)g(MINPROCS,)g (MAXPROCS,)g(ROOT,)g(COMM,)g(INTERCOMM,)170 2483 y(ARRAY)p 293 2483 V 17 w(OF)p 358 2483 V 17 w(ERRCODES\(*\),)f(IERR)166 2570 y Fp(MPI)p 251 2570 14 2 v 16 w(SP)l(A)-5 b(WN)17 b Fu(tries)g(to)f(start)f Fp(maxp)o(ro)q(cs)h Fu(iden)o(tical)i(copies) g(of)e(the)g(program)g(sp)q(eci\014ed)i(b)o(y)f Fp(com-)75 2626 y(mand)p 183 2626 V 16 w(line)c Fu(and)g(establish)h(comm)o (unication)f(with)g(them.)19 b(It)12 b(is)h(collectiv)o(e)h(o)o(v)o(er) e Fp(comm)f Fu(and)h(the)h(newly)75 2683 y(created)g(pro)q(cesses.)19 b(The)13 b(spa)o(wned)g(pro)q(cesses,)g(kno)o(wn)g(as)f(c)o(hildren,)j (are)d Fj(r)n(e)n(quir)n(e)n(d)g Fu(to)g(call)i Fp(MPI)p 1761 2683 V 16 w(INIT)p Fu(,)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)916 b Fu(7)75 49 y(whic)o(h)18 b(is)g(collectiv)o(e)g(with)g Fp(MPI)p 644 49 14 2 v 16 w(SP)l(A)-5 b(WN)18 b Fu(in)f(the)h(paren)o (t)e(pro)q(cesses.)26 b(An)18 b(in)o(tercomm)o(unicator)f(con-)75 106 y(taining)h(the)f(paren)o(t)f(pro)q(cesses)h(in)h(one)f(group)f (and)i(the)e(c)o(hild)j(pro)q(cesses)e(in)h(the)f(other)f(is)i (returned)75 162 y(b)o(y)j Fp(MPI)p 229 162 V 16 w(SP)l(A)-5 b(WN)22 b Fu(in)g(the)f(paren)o(t)g(and)g(can)g(b)q(e)h(obtained)f(in)h (the)g(c)o(hildren)h(through)d(the)h(routine)75 219 y Fp(MPI)p 160 219 V 16 w(P)l(ARENT)p Fu(.)f(The)g(c)o(hildren)h(ha)o(v)o (e)e(their)i(o)o(wn)e Fp(MPI)p 1062 219 V 15 w(COMM)p 1219 219 V 17 w(W)o(ORLD)g Fu(whic)o(h)i(is)f(separate)f(from)75 275 y(that)14 b(of)h(the)h(paren)o(t.)166 332 y(The)22 b Fp(command)p 454 332 V 15 w(line)h Fu(argumen)o(t)e(is)h(a)g(string)g (con)o(taining)h(a)e(whitespace-separated)i(list)g(of)e(a)75 388 y(program)14 b(name)h(and)h(argumen)o(ts)e(whic)o(h)i(are)f(passed) g(to)g(the)g(program.)166 444 y(The)k(follo)o(wing)h(legalese)h (de\014nes)f(all)h(the)e(sp)q(ecial)i(cases)e(and)h(ensures)g(that)e (it)i(is)g(p)q(ossible)h(to)75 501 y(sp)q(ecify)d(an)o(y)e(argumen)o (ts.)23 b(It)16 b(is)h(unnecessary)g(for)f(normal)h(use.)24 b(Whitespace)17 b(is)g(de\014ned)h(as)e(for)g(the)75 557 y(isspace\(\))21 b(routine)g(in)h(the)f(standard)f(C)h(library)h (\(space,)f(form-feed,)h(new)f(line,)j(carriage)c(return,)75 614 y(tab\).)j(All)17 b(other)f(c)o(haracters)g(are)g(in)o(terpreted)h (literally)h(except)f(for)f(a)g(small)h(n)o(um)o(b)q(er)g(of)f(t)o(w)o (o-letter)75 670 y(com)o(binations)g(starting)e(with)i(\\)p Fl(\045)p Fu(")e(that)h(ha)o(v)o(e)g(sp)q(ecial)i(meaning.)143 764 y Fo(\017)23 b Fu(\\)p Fl(\045\045)p Fu(")14 b(is)i(replaced)g(b)o (y)f(\\)p Fl(\045)p Fu(")143 858 y Fo(\017)23 b Fu(\\)p Fl(\045)p Fu(")11 b(where)j(\\)p Fl()p Fu(")e(is)i(a)g(single)h(whitespace)g(c)o(haracter)e(is)i(replaced)189 914 y(b)o(y)g(that)f(whitespace)i(c)o(haracter)f(but)g(do)q(es)h(not)e (separate)h(items)h(in)g(the)f(command)g(line.)143 1008 y Fo(\017)23 b Fu(\\)p Fl(\045_)p Fu(")13 b(is)j(discarded.)k(Ho)o(w)o (ev)o(er,)14 b(it)h(is)g(treated)f(as)g(a)h(non-whitespace)h(c)o (haracter)d(in)j(that)e(it)h(can)189 1065 y(b)q(egin)h(a)f(new)g(w)o (ord.)20 b(Its)15 b(in)o(tended)h(use)g(is)g(to)e(allo)o(w)i(the)f(sp)q (eci\014cation)i(of)e(empt)o(y)g(strings.)75 1158 y(F)l(or)e(example,)i (the)g(command-line)g(\\)p Fl(foo ba\045 r \045_ q\045\045u\045\045_ux) p Fu(")c(\(where)j(\\)p Fl( )p Fu(")f(is)i(a)f(space)g(c)o(harac-)75 1215 y(ter\))g(sp)q(eci\014es)i(the)e(command)h(\\)p Fl(foo)p Fu(")e(with)i(argumen)o(ts)e(\\)p Fl(ba r)p Fu(",)g(\\",)h(and)g(\\)p Fl(q\045u\045_ux)p Fu(".)19 b(The)14 b(sp)q(ecial)75 1271 y(t)o(w)o(o-letter)g(com)o(binations)h (are)g(translated)g(as)f(they)h(are)g(encoun)o(tered)g(when)g(reading)h (the)f(string,)f(so)75 1328 y(that)h(\\)p Fl(\045\045_)p Fu(")g(is)i(translated)f(to)f(\\)p Fl(\045_)p Fu(",)h(not)f(\\)p Fl(\045)p Fu(".)22 b(Note)16 b(that)f(\\)p Fl(\045)p Fu(")h(not)f(follo)o(w)o(ed)i(b)o(y)f(\\)p Fl(\045)p Fu(",)f(\\)p Fl(\045)p Fu(",)g(\\)p Fl(_)p Fu(")g(or)75 1384 y(\\)p Fl()p Fu(")e(is)j(not)e(treated)h(sp)q(ecially) l(.)166 1523 y Fi(Discussion:)34 b Fh(The)14 b(format)e(ma)o(y)f(lo)q (ok)i(complicated)g(to)g(users)i(but)f(is)g(only)f(messy)g(if)g(they)h (w)o(an)o(t)g(to)f(do)75 1580 y(something)h(strange.)25 b(Ho)o(w)o(ev)o(er,)16 b(in)f(en)o(vironmen)o(ts)g(where)i(\014lenames) e(tend)h(to)f(ha)o(v)o(e)h(em)o(b)q(edded)g(whitespace)75 1636 y(this)e(\(and)g(an)o(y)f(other)i(\\command-l)o(ine")c(in)o (terfaces\))k(could)e(b)q(ecome)h(a)g(nigh)o(tmare.)189 1825 y Fj(A)n(dvic)n(e)j(to)i(users.)53 b Fu(Though)17 b Fp(MPI)p 818 1825 V 16 w(SP)l(A)-5 b(WN)18 b Fu(tak)o(es)f(a)g (command)p 1353 1825 V 16 w(line)j(as)d(an)g(argumen)o(t,)g(it)g(is)189 1882 y(imp)q(ortan)o(t)h(to)g(remem)o(b)q(er)g(that)g(it)h(is)g(not)f (a)g(shell)i(command)e(that)g(y)o(ou)g(migh)o(t)g(t)o(yp)q(e)h(at)f (the)189 1938 y(terminal.)h(F)l(or)12 b(instance,)h(\\)p Fl(foo)23 b(>)h(bar)p Fu(")12 b(will)i(spa)o(wn)d(a)h(program)f(\\)p Fl(foo)p Fu(")h(with)g(argumen)o(ts)g(\\)p Fl(>)p Fu(")189 1995 y(and)17 b(\\)p Fl(bar)p Fu(",)f(but)i(will)h(not)e(redirect)h (output.)25 b(\\)p Fl(mpirun)e(-np)h(4)f(foo)p Fu(")17 b(will)i(spa)o(wn)e Fp(maxp)o(ro)q(cs)189 2051 y Fu(copies)d(of)e(the)i (program)e(\\)p Fl(mpirun)p Fu(".)18 b(The)13 b(application)i(will)g (exp)q(ect)e(to)g(establish)h(comm)o(unica-)189 2108 y(tion)i(with)h(the)f(\\)p Fl(mpirun)p Fu(")f(program,)f(not)i(with)h (the)f(pro)q(cesses)h(that)e(ma)o(y)h(b)q(e)g(started)g(b)o(y)g(the)189 2164 y Fl(mpirun)e Fu(program.)19 b(\()p Fj(End)c(of)i(advic)n(e)f(to)g (users.)p Fu(\))166 2270 y(MPI)f(tries)g(to)f(start)g Fp(maxp)o(ro)q(cs)g Fu(copies)i(of)f(the)g(program)f(sp)q(eci\014ed)j (in)e(the)h Fp(command)p 1676 2270 V 15 w(line)p Fu(.)k(MPI)75 2327 y(do)q(es)f(not)f(sp)q(ecify)i(ho)o(w)e(to)g(\014nd)i(the)f (executable)h(or)e(ho)o(w)g(the)h(w)o(orking)f(directory)h(is)g (determined.)75 2383 y(These)k(rules)h(will)g(b)q(e)f(implemen)o (tation-dep)q(end)q(en)o(t)i(and)e(will)h(b)q(e)g(appropriate)f(for)f (the)h(run)o(time)75 2440 y(en)o(vironmen)o(t.)d(F)l(or)12 b(instance,)i(a)f(homogeneous)g(system)g(with)h(a)f(global)h(\014le)g (system)f(migh)o(t)g(lo)q(ok)g(\014rst)75 2496 y(in)h(the)f(w)o(orking) f(directory)h(of)g(the)g(spa)o(wning)g(pro)q(cess,)g(or)g(migh)o(t)g (searc)o(h)f(the)h(directories)h(in)g(a)f(P)l(A)l(TH)75 2553 y(en)o(vironmen)o(t)e(v)m(ariable)g(as)f(do)g(Unix)h(shells.)20 b(An)10 b(implemen)o(tation)i(on)e(top)g(of)g(PVM)g(w)o(ould)g(use)h (PVM's)75 2609 y(rules)k(for)f(\014nding)h(executables)h(\(usually)f (in)g Fl(~/pvm3/bin/$PVM_ARCH)p Fu(\).)c(An)j(MPI)h(implemen)o(tation) 75 2665 y(running)h(under)g(IBM's)f(POE)h(will)g(use)g(POE's)f(metho)q (d)g(of)g(\014nding)i(executables.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fu(8)951 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 49 y Fu(While)20 b(MPI)f(alw)o(a)o(ys)f(tries)i(to)e (spa)o(wn)h Fp(maxp)o(ro)q(cs)f Fu(pro)q(cesses,)i(the)f Fp(minp)o(ro)q(cs)g Fu(argumen)o(t)f(allo)o(ws)75 106 y(some)d(additional)h(\015exibilit)o(y)l(.)23 b(If)15 b(MPI)g(is)g(unable)h(to)f(spa)o(wn)f Fp(maxp)o(ro)q(cs)g Fu(pro)q(cesses,)h(it)h(ma)o(y)e(spa)o(wn)g(as)75 162 y(few)d(as)f Fp(minp)o(ro)q(cs)p Fu(.)18 b(The)12 b(n)o(um)o(b)q(er)f (of)f(pro)q(cesses)h(actually)h(spa)o(wned)f(can)g(b)q(e)h(determined)g (b)o(y)f(examining)75 219 y(the)j(size)h(of)f(the)g(remote)f(group)h (of)g(the)g(returned)g(in)o(tercomm)o(unicator.)20 b(If)14 b(MPI)g(is)h(unable)g(to)e(spa)o(wn)75 275 y(at)20 b(least)h Fp(minp)o(ro)q(cs)p Fu(,)h Fp(MPI)p 538 275 14 2 v 16 w(SP)l(A)-5 b(WN)22 b Fu(will)g(return)f(an)g(error.)36 b(It)21 b(ma)o(y)f(b)q(e)i(p)q(ossible)h(to)d(additionally)75 332 y(constrain)15 b(the)h(n)o(um)o(b)q(er)f(of)g(pro)q(cesses)h(spa)o (wned)f(through)g(the)g Fp(info)g Fu(argumen)o(t.)189 438 y Fj(A)n(dvic)n(e)h(to)i(users.)50 b Fu(An)17 b(MPI)f(implemen)o (tation)i(is)g(allo)o(w)o(ed,)f(but)g(is)g(not)f(required,)i(to)e(spa)o (wn)189 494 y Fp(minp)o(ro)q(cs)d Fu(pro)q(cesses)h(if)g Fp(maxp)o(ro)q(cs)e Fu(are)i(not)f(a)o(v)m(ailable.)21 b(Th)o(us)13 b(it)h(w)o(ould)g(not)f(b)q(e)h(p)q(ortable)g(to)f(try)189 551 y(to)i(acquire)i(all)g(a)o(v)m(ailable)h(resources)e(b)o(y)g(sp)q (ecifying)i(a)e(v)o(ery)g(large)g(v)m(alue)h(for)f Fp(maxp)o(ro)q(cs)f Fu(and)h(a)189 607 y(small)g(v)m(alue)g(for)f Fp(minp)o(ro)q(cs)p Fu(.)k(\()p Fj(End)d(of)g(advic)n(e)g(to)h(users.)p Fu(\))166 713 y(The)12 b Fp(info)g Fu(argumen)o(t)g(is)h(opaque)f(to)f(MPI)i(and) f(is)h(passed)f(directly)i(to)d(the)h(run)o(time)h(en)o(vironmen)o(t,) 75 770 y(whic)o(h)g(uses)f(it)g(to)f(determine)h(where)g(and)g(ho)o(w)f (to)h(spa)o(wn)f(pro)q(cesses.)19 b(In)12 b(C)g(it)g(is)g(a)g (NULL-terminated)75 826 y(string.)24 b(If)17 b(the)g(info)g(argumen)o (t)f(is)h(empt)o(y)f(\(NULL)i(or)e(an)g(empt)o(y)h(string)f(in)i(C,)e (or)g(')p Fl( )p Fu(')g(F)l(ortran\))f(the)75 883 y(run)o(time)j(en)o (vironmen)o(t)g(decides)h(where)f(to)f(spa)o(wn)g(pro)q(cesses.)28 b(P)o(ortable)17 b(programs)f(not)i(requiring)75 939 y(detailed)f(con)o(trol)e(o)o(v)o(er)f(pro)q(cess)h(lo)q(cations)h (should)h(use)e(an)g(empt)o(y)g(argumen)o(t.)166 996 y(MPI)k(do)q(es)h(not)f(sp)q(ecify)i(the)e(format)g(of)g(the)g Fp(info)g Fu(argumen)o(t,)h(whic)o(h)g(is)g(left)g(up)g(to)e(the)i(MPI) 75 1052 y(implemen)o(tation.)31 b(The)19 b(format)e(is)i(exp)q(ected)h (to)e(b)q(e)h(tailored)h(to)e(the)g(particular)h(run)o(time)g(system)75 1109 y(b)q(eing)d(used.)21 b(Examples)16 b(of)e(an)h(info)h(string)f (migh)o(t)g(b)q(e)h(\\hostname=dop)q(ey")f(to)g(spa)o(wn)g(all)h(tasks) e(on)75 1165 y(a)g(host)h(named)g(\\dop)q(ey",)f(or)g(\\2:hippi:sgi")i (to)e(spa)o(wn)g(on)h(t)o(w)o(o)e(hippi-connected)18 b(SGI)d(mac)o(hines,)g(or)75 1222 y(ev)o(en)h(\\\014le=info.cfg")g(to)e (indicate)j(that)d(detailed)j(information)e(is)h(sp)q(eci\014ed)h(in)f (the)g(\014le)g(\\info.cfg".)166 1278 y(Alternativ)o(ely)l(,)f(it)e(is) h(ev)o(en)g(p)q(ossible)i(to)c(use)i(the)g Fp(info)f Fu(argumen)o(t)g(to)g(sp)q(ecify)h(the)g(executable)h(and)75 1334 y(its)f(command-line)h(argumen)o(ts,)d(in)i(whic)o(h)h(case)e(the) g Fp(command)p 1198 1334 V 16 w(line)g Fu(argumen)o(t)g(to)g Fp(MPI)p 1630 1334 V 16 w(SP)l(A)-5 b(WN)14 b Fu(can)75 1391 y(b)q(e)k(empt)o(y)l(.)26 b(The)17 b(abilit)o(y)i(to)e(do)g(this)g (follo)o(ws)h(from)e(the)i(fact)e(that)h(MPI)g(do)q(es)h(not)f(sp)q (ecify)h(ho)o(w)f(an)75 1447 y(executable)k(is)f(found,)h(and)f(the)g Fp(info)g Fu(argumen)o(t)f(can)h(tell)g(the)g(run)o(time)g(system)g (where)g(to)f(\\\014nd")75 1504 y(the)h(executable)g(\\".)33 b(Of)20 b(course)f(a)g(program)g(that)g(do)q(es)h(this)g(will)h(not)e (b)q(e)h(p)q(ortable)g(across)f(MPI)75 1560 y(implemen)o(tations.)166 1617 y(The)f Fp(a)o(rra)o(y)p 360 1617 V 14 w(of)p 411 1617 V 16 w(errco)q(des)g Fu(is)g(an)f(arra)o(y)g(of)g(length)h Fp(maxp)o(ro)q(cs)e Fu(in)j(whic)o(h)f(MPI)f(rep)q(orts)g(the)h(status) 75 1673 y(of)e(eac)o(h)h(pro)q(cess)h(that)e(MPI)h(w)o(as)f(requested)h (to)f(start.)24 b(If)17 b(all)h Fp(maxp)o(ro)q(cs)e Fu(pro)q(cesses)i (w)o(ere)e(spa)o(wned,)75 1730 y Fp(a)o(rra)o(y)p 173 1730 V 15 w(of)p 225 1730 V 16 w(errco)q(des)22 b Fu(is)g(\014lled)h (in)g(with)f(the)f(v)m(alue)i Fp(MPI)p 1048 1730 V 16 w(SUCCESS)p Fu(.)f(If)g(only)g Ff(m)g Fu(\()p Fp(minp)o(ro)q(cs)h Fo(\024)g Ff(m)h(<)75 1786 y Fp(maxp)o(ro)q(cs)o Fu(\))e(pro)q(cesses)h (are)f(spa)o(wned,)i Ff(m)e Fu(of)g(the)g(en)o(tries)h(will)h(con)o (tain)e Fp(MPI)p 1489 1786 V 16 w(SUCCESS)h Fu(and)g(the)75 1843 y(rest)14 b(will)j(con)o(tain)e(an)g(error)f(co)q(de)i(indicating) g(an)f(implemen)o(tation-de\014ned)j(reason)d(MPI)f(could)i(not)75 1899 y(start)g(the)i(pro)q(cess.)27 b(MPI)18 b(do)q(es)f(not)h(sp)q (ecify)g(whic)o(h)h(en)o(tries)f(corresp)q(ond)g(to)f(failed)h(pro)q (cesses.)28 b(An)75 1955 y(implemen)o(tation)19 b(ma)o(y)l(,)g(for)e (instance,)i(\014ll)h(in)f(error)e(co)q(des)i(in)g(one-to-one)f (corresp)q(ondence)i(with)e(a)75 2012 y(detailed)g(sp)q(eci\014cation)g (in)g(the)e Fp(info)g Fu(argumen)o(t.)23 b(These)17 b(error)f(co)q(des) h(all)g(b)q(elong)h(to)d(the)i(error)f(class)75 2068 y Fp(MPI)p 160 2068 V 16 w(ERR)p 261 2068 V 17 w(SP)l(A)-5 b(WN)16 b Fu(if)h(there)e(w)o(as)g(no)h(error)f(in)h(the)g(argumen)o(t) f(list.)22 b(An)15 b(application)j(ma)o(y)d(pass)g(\(in)o(t)75 2125 y(*\)NULL)h(\(C-binding\))g(for)f Fp(a)o(rra)o(y)p 671 2125 V 14 w(of)p 722 2125 V 16 w(errco)q(des)p Fu(,)g(in)i(whic)o (h)f(case)f(it)g(is)h(ignored)g(b)o(y)f(MPI.)166 2181 y(If)c(no)g(pro)q(cesses)h(are)e(spa)o(wned,)i(MPI)f(returns)g(a)f (\\half-empt)o(y")h(in)o(tercomm)o(unicator,)h(that)e(is,)i(an)75 2238 y(in)o(tercomm)o(unicator)j(for)g(whic)o(h)h Fp(MPI)p 749 2238 V 16 w(COMM)p 907 2238 V 16 w(REMOTE)p 1110 2238 V 17 w(GROUP)h Fu(returns)e Fp(MPI)p 1535 2238 V 16 w(GROUP)p 1703 2238 V 18 w(EMPTY)75 2294 y Fu(and)g Fp(MPI)p 248 2294 V 16 w(COMM)p 406 2294 V 17 w(REMOTE)p 610 2294 V 17 w(SIZE)g Fu(returns)g(0.)166 2351 y(The)g(return)g(v)m (alue)i(\(C\))d(and)i Fp(ierr)e Fu(argumen)o(t)g(\(F)l(ortran\))g(ha)o (v)o(e)g(their)i(usual)g(meanings.)75 2407 y Fp(MPI)p 160 2407 V 16 w(SUCCESS)21 b Fu(means)e(that)g(at)g(least)g Fp(minp)o(ro)q(cs)h Fu(pro)q(cesses)g(w)o(ere)f(spa)o(wned)h (successfully)l(.)35 b(Other)75 2464 y(v)m(alues)16 b(indicate)h(an)e (error.)166 2520 y(All)21 b(argumen)o(ts)f(b)q(efore)g(the)h Fp(ro)q(ot)e Fu(argumen)o(t)h(are)g(sp)q(eci\014ed)i(on)e(the)h(pro)q (cess)f(whose)g(rank)g(in)75 2576 y Fp(comm)13 b Fu(is)j(equal)g(to)f Fp(ro)q(ot)p Fu(.)k(The)c(v)m(alue)i(of)e(these)g(argumen)o(ts)f(on)i (other)e(pro)q(cesses)i(is)g(ignored.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)916 b Fu(9)75 49 y Fp(MPI)p 160 49 14 2 v 16 w(P)l(ARENT\(intercomm\))117 126 y Fh(OUT)108 b Fp(intercomm)384 b Fh(paren)o(t)15 b(in)o(tercomm)o(unicator)c(\(handle\))75 251 y Fl(int)23 b(MPI)p 245 251 15 2 v 17 w(Parent\(MPI)p 502 251 V 16 w(Comm*)g(intercomm\))75 337 y(MPI)p 150 337 V 17 w(PARENT\(INTERCOMM,)e(IERR\))170 394 y(INTEGER)i(INTERCOMM,)g (IERR)166 480 y Fp(MPI)p 251 480 14 2 v 16 w(P)l(ARENT)f Fu(can)g(b)q(e)h(called)g(in)f(an)o(y)f(pro)q(cess.)40 b(If)22 b(there)g(is)g(a)f(paren)o(t,)h(\(i.e.,)h(the)f(pro)q(cess)75 537 y(w)o(as)c(started)h(with)g Fp(MPI)p 517 537 V 16 w(SP)l(A)-5 b(WN)p Fu(\))20 b(it)f(returns)g(an)h(in)o(tercomm)o (unicator.)31 b(The)20 b(lo)q(cal)g(group)f(of)g(the)75 593 y(in)o(tercomm)o(unicator)i(consists)g(of)f(the)h(pro)q(cesses)g (created)g(with)g(the)g(same)g(call)h(to)e Fp(MPI)p 1699 593 V 15 w(SP)l(A)-5 b(WN)p Fu(.)75 650 y(The)16 b(remote)g(group)g (consists)g(of)g(the)g(pro)q(cesses)h(that)f(co)q(op)q(erated)g(on)g (the)g(call)i(to)d Fp(MPI)p 1652 650 V 16 w(SP)l(A)-5 b(WN)p Fu(.)17 b(If)75 706 y(there)e(is)h(no)f(paren)o(t,)g Fp(MPI)p 541 706 V 15 w(P)l(ARENT)i Fu(returns)e Fp(MPI)p 986 706 V 16 w(COMM)p 1144 706 V 16 w(NULL)p Fu(.)75 827 y Fm(3.3.3)49 b(Sta)o(rting)16 b(Multiple)h(Executables)f(and)h (Establishing)g(Communication)g(with)g(Them)75 913 y Fu(While)h Fp(MPI)p 294 913 V 16 w(SP)l(A)-5 b(WN)17 b Fu(is)h(su\016cien)o(t)f(for)f(most)g(cases,)h(it)g(do)q(es)g(not)f (allo)o(w)h(the)g(spa)o(wning)g(of)f(m)o(ultiple)75 969 y(binaries,)23 b(or)e(of)f(the)h(same)g(binary)g(with)h(m)o(ultiple)g (sets)f(of)f(argumen)o(ts.)37 b(The)21 b(follo)o(wing)g(routine)75 1025 y(spa)o(wns)16 b(m)o(ultiple)j(binaries)f(or)e(the)g(same)h (binary)g(with)g(m)o(ultiple)h(sets)e(of)g(argumen)o(ts,)g (establishing)75 1082 y(comm)o(unication)g(with)f(them.)20 b(All)d(spa)o(wned)e(programs)f(ha)o(v)o(e)h(the)g(same)g Fp(MPI)p 1469 1082 V 16 w(COMM)p 1627 1082 V 16 w(W)o(ORLD)p Fu(.)75 1233 y Fp(MPI)p 160 1233 V 16 w(SP)l(A)-5 b(WN)p 326 1233 V 17 w(MUL)l(TIPLE\(count,)11 b(a)o(rra)o(y)p 802 1233 V 15 w(of)p 854 1233 V 16 w(command)p 1055 1233 V 15 w(lines,)g(a)o(rra)o(y)p 1271 1233 V 15 w(of)p 1323 1233 V 16 w(minp)o(ro)q(cs,)g(a)o(rra)o(y)p 1628 1233 V 14 w(of)p 1679 1233 V 16 w(maxp)o(ro)q(cs,)75 1289 y(a)o(rra)o(y)p 173 1289 V 15 w(of)p 225 1289 V 16 w(info,)j(ro)q(ot,)h (comm,)e(intercomm,)g(a)o(rra)o(y)p 913 1289 V 15 w(of)p 965 1289 V 16 w(errco)q(des\))117 1366 y Fh(IN)155 b Fp(count)482 b Fh(Num)o(b)q(er)12 b(of)g(command)e(lines)j(\(in)o (teger,)g(size)g(of)f(eac)o(h)h(of)f(the)905 1423 y(follo)o(wing)f (arra)o(ys\))117 1497 y(IN)155 b Fp(a)o(rra)o(y)p 416 1497 V 15 w(of)p 468 1497 V 16 w(command)p 669 1497 V 15 w(lines)142 b Fh(executable)12 b(and)f(argumen)o(ts)f(\(arra)o(y)g (of)g(strings,)h(signi\014can)o(t)905 1553 y(only)i(at)h(ro)q(ot\))117 1627 y(IN)155 b Fp(a)o(rra)o(y)p 416 1627 V 15 w(of)p 468 1627 V 16 w(minp)o(ro)q(cs)254 b Fh(minim)n(um)9 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o(h)h(command)c(line) 905 1684 y(to)k(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can)o (t)h(only)f(at)h(ro)q(ot\))117 1758 y(IN)155 b Fp(a)o(rra)o(y)p 416 1758 V 15 w(of)p 468 1758 V 16 w(maxp)o(ro)q(cs)245 b Fh(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o (h)g(command)d(line)905 1814 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o (teger,)f(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 1888 y(IN)155 b Fp(a)o(rra)o(y)p 416 1888 V 15 w(of)p 468 1888 V 16 w(info)353 b Fh(strings)17 b(telling)e(the)i(run)o(time)e (system)i(where)g(and)f(ho)o(w)g(to)905 1945 y(start)21 b(pro)q(cesses)i(\(arra)o(y)e(of)e(strings,)k(signi\014can)o(t)c(only)h (at)905 2001 y(ro)q(ot)117 2075 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d(whic)o(h)h(previous)g(argumen)o (ts)f(are)i(v)n(alid)905 2131 y(\(in)o(teger\))117 2205 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o (wning)g(pro)q(cesses)j(\(handle\))117 2279 y(OUT)108 b Fp(intercomm)384 b Fh(in)o(tercomm)o(unicator)12 b(b)q(et)o(w)o(een)j (original)d(group)i(and)g(newly)905 2336 y(spa)o(wned)h(group)e (\(handle\))117 2410 y(OUT)108 b Fp(a)o(rra)o(y)p 416 2410 V 15 w(of)p 468 2410 V 16 w(errco)q(des)268 b Fh(one)14 b(error)h(co)q(de)g(p)q(er)g(pro)q(cess)g(\(arra)o(y)f(of)f(in)o (teger\))75 2534 y Fl(int)23 b(MPI)p 245 2534 15 2 v 17 w(Spawn)p 382 2534 V 17 w(multiple\(int*)f(count,)h(char*)g(array)p 1163 2534 V 17 w(of)p 1228 2534 V 16 w(command)p 1412 2534 V 17 w(lines[],)393 2591 y(int*)g(array)p 635 2591 V 17 w(of)p 700 2591 V 17 w(minprocs,)g(int*)g(array)p 1195 2591 V 16 w(of)p 1259 2591 V 17 w(maxprocs,)393 2647 y(char*)g(array)p 659 2647 V 17 w(of)p 724 2647 V 17 w(info[],)g(int)g(root,)g(MPI)p 1242 2647 V 17 w(Comm)g(comm,)393 2704 y(MPI)p 468 2704 V 17 w(Comm*)g(intercomm,)g(int*)g(array)p 1130 2704 V 16 w(of)p 1194 2704 V 17 w(errcodes\))-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fu(10)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fl(MPI)p 150 49 15 2 v 17 w(SPAWN)p 287 49 V 16 w(MULTIPLE\(COUNT,)22 b(ARRAY)p 805 49 V 17 w(OF)p 870 49 V 16 w(COMMAND)p 1054 49 V 17 w(LINES,)h(ARRAY)p 1358 49 V 16 w(OF)p 1422 49 V 17 w(MINPROCS,)393 106 y(ARRAY)p 516 106 V 17 w(OF)p 581 106 V 17 w(MAXPROCS,)f(ARRAY)p 956 106 V 17 w(OF)p 1021 106 V 17 w(INFO,)h(ROOT,)g(COMM,)g(INTERCOMM,)393 162 y(ARRAY)p 516 162 V 17 w(OF)p 581 162 V 17 w(ERRCODES,)f(IERR\))170 219 y(CHARACTER*\(*\))h(ARRAY)p 628 219 V 16 w(OF)p 692 219 V 17 w(COMMAND)p 877 219 V 16 w(LINES\(*\),)g(ARRAY)p 1252 219 V 16 w(OF)p 1316 219 V 17 w(INFO)170 275 y(INTEGER)g(COUNT,)g (ARRAY)p 651 275 V 17 w(OF)p 716 275 V 17 w(MINPROCS,)f(ARRAY)p 1091 275 V 17 w(OF)p 1156 275 V 17 w(MAXPROCS,)g(ROOT,)i(COMM,)170 332 y(INTERCOMM,)f(ARRAY)p 556 332 V 16 w(OF)p 620 332 V 17 w(ERRCODES\(*\),)g(IERR)166 418 y Fp(MPI)p 251 418 14 2 v 16 w(SP)l(A)-5 b(WN)p 417 418 V 17 w(MUL)l(TIPLE)12 b Fu(is)h(iden)o(tical)h(to)e Fp(MPI)p 1021 418 V 16 w(SP)l(A)-5 b(WN)13 b Fu(except)g(that)f(there)h(are)f(m)o(ultiple)i (exe-)75 474 y(cutable)e(sp)q(eci\014cations.)20 b(The)12 b(\014rst)e(argumen)o(t,)h Fp(count)p Fu(,)i(giv)o(es)e(the)g(n)o(um)o (b)q(er)h(of)e(sp)q(eci\014cations.)21 b(Eac)o(h)10 b(of)75 531 y(the)k(next)g(four)g(argumen)o(ts)f(are)h(simply)i(arra)o(ys)c(of) i(the)g(corresp)q(onding)h(argumen)o(ts)f(in)h Fp(MPI)p 1699 531 V 15 w(SP)l(A)-5 b(WN)p Fu(.)166 587 y(All)25 b(of)f(the)g(spa)o(wned)g(pro)q(cesses)h(ha)o(v)o(e)e(the)h(same)g Fp(MPI)p 1203 587 V 16 w(COMM)p 1361 587 V 16 w(W)o(ORLD)p Fu(.)g(Their)g(ranks)g(in)75 644 y Fp(MPI)p 160 644 V 16 w(COMM)p 318 644 V 16 w(W)o(ORLD)13 b Fu(corresp)q(ond)g(directly)h (to)e(the)g(order)h(in)g(whic)o(h)g(the)g(command)g(lines)h(are)e(sp)q (ec-)75 700 y(i\014ed)j(in)g Fp(MPI)p 309 700 V 16 w(SP)l(A)-5 b(WN)p 475 700 V 17 w(MUL)l(TIPLE)p Fu(.)13 b(Assume)i(that)e Ff(m)1037 707 y Fe(1)1071 700 y Fu(pro)q(cesses)h(are)g(generated)g(b)o (y)g(the)g(\014rst)g(com-)75 757 y(mand,)h Ff(m)254 764 y Fe(2)289 757 y Fu(b)o(y)g(the)h(second,)f(etc.)20 b(The)c(pro)q (cesses)g(corresp)q(onding)g(to)f(the)g(\014rst)g(command)g(line)i(ha)o (v)o(e)75 813 y(ranks)11 b(0)p Ff(;)d Fu(1)p Ff(;)g(:)g(:)g(:)t(;)g(m) 402 820 y Fe(1)424 813 y Fo(\000)s Fu(1.)19 b(The)12 b(pro)q(cesses)g(in)h(the)e(second)i(command)e(ha)o(v)o(e)h(ranks)f Ff(m)1532 820 y Fe(1)1552 813 y Ff(;)d(m)1613 820 y Fe(1)1635 813 y Fu(+)s(1)p Ff(;)g(:)g(:)g(:)d(m)1817 820 y Fe(1)1840 813 y Fu(+)75 870 y Ff(m)115 877 y Fe(2)141 870 y Fo(\000)i Fu(1.)19 b(The)14 b(pro)q(cesses)f(in)i(the)e(third)h(ha)o(v)o(e)f (ranks)g Ff(m)1028 877 y Fe(1)1054 870 y Fu(+)7 b Ff(m)1136 877 y Fe(2)1156 870 y Ff(;)h(m)1217 877 y Fe(1)1242 870 y Fu(+)f Ff(m)1324 877 y Fe(2)1350 870 y Fu(+)g(1)p Ff(;)h(:)g(:)g(:)t (;)g(m)1556 877 y Fe(1)1582 870 y Fu(+)f Ff(m)1664 877 y Fe(2)1690 870 y Fu(+)g Ff(m)1772 877 y Fe(3)1798 870 y Fo(\000)g Fu(1,)75 926 y(etc.)189 1032 y Fj(A)n(dvic)n(e)15 b(to)h(users.)39 b Fu(Calling)17 b Fp(MPI)p 785 1032 V 15 w(SP)l(A)-5 b(WN)16 b Fu(m)o(ultiple)h(times)e(w)o(ould)g(create)f (man)o(y)h(sets)f(of)g(c)o(hil-)189 1089 y(dren)23 b(with)f(di\013eren) o(t)h Fp(MPI)p 682 1089 V 16 w(COMM)p 840 1089 V 16 w(W)o(ORLD)p Fu(s)f(whereas)h Fp(MPI)p 1320 1089 V 15 w(SP)l(A)-5 b(WN)p 1485 1089 V 18 w(MUL)l(TIPLE)21 b Fu(creates)189 1145 y(c)o(hildren)e(with)f(a)f(single)i Fp(MPI)p 724 1145 V 16 w(COMM)p 882 1145 V 16 w(W)o(ORLD)p Fu(,)e(so)h(the)f(t)o(w)o (o)f(metho)q(ds)i(are)f(not)h(completely)189 1202 y(equiv)m(alen)o(t.)j (There)13 b(are)g(also)h(t)o(w)o(o)e(p)q(erformance-related)i(reasons)f (wh)o(y)l(,)g(if)h(y)o(ou)f(need)i(to)d(spa)o(wn)189 1258 y(m)o(ultiple)20 b(executables,)h(y)o(ou)d(ma)o(y)g(w)o(an)o(t)g (to)g(use)h Fp(MPI)p 1161 1258 V 16 w(SP)l(A)-5 b(WN)p 1327 1258 V 17 w(MUL)l(TIPLE)18 b Fu(instead)h(of)g(call-)189 1315 y(ing)e Fp(MPI)p 351 1315 V 16 w(SP)l(A)-5 b(WN)17 b Fu(sev)o(eral)g(times.)24 b(First,)17 b(spa)o(wning)g(sev)o(eral)f (things)i(at)e(once)h(ma)o(y)e(b)q(e)j(faster)189 1371 y(than)e(spa)o(wning)h(them)f(sequen)o(tially)l(.)25 b(Second,)17 b(in)h(some)e(implemen)o(tations,)h(comm)o(unication)189 1428 y(b)q(et)o(w)o(een)f(pro)q(cesses)g(spa)o(wned)g(at)g(the)g(same)f (time)i(ma)o(y)e(b)q(e)h(faster)g(than)f(comm)o(unication)i(b)q(e-)189 1484 y(t)o(w)o(een)d(pro)q(cesses)i(spa)o(wned)f(separately)l(.)21 b(\()p Fj(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fu(\))166 1590 y(The)e Fp(a)o(rra)o(y)p 356 1590 V 14 w(of)p 407 1590 V 16 w(errco)q(des)g Fu(argumen)o(t)f(is)h(1-dimensional)i(arra)o (y)c(of)h(size)1375 1558 y Fd(P)1419 1571 y Fc(count)1419 1602 y(i)p Fe(=1)1515 1590 y Ff(n)1542 1597 y Fc(i)1557 1590 y Fu(,)g(where)h Ff(n)1740 1597 y Fc(i)1768 1590 y Fu(is)g(the)75 1647 y Ff(i)p Fu(th)19 b(elemen)o(t)i(of)e Fp(a)o(rra)o(y)p 480 1647 V 14 w(of)p 531 1647 V 16 w(maxp)o(ro)q(cs)p Fu(.)33 b(Command)19 b(line)j(n)o(um)o(b)q(er)e Ff(i)f Fu(corresp)q(onds)h(to)f Ff(n)1630 1654 y Fc(i)1664 1647 y Fu(con)o(tiguous)75 1703 y(slots)c(in)h(this)g(arra)o(y)l(,)d(and)j (error)e(co)q(des)i(are)f(treated)g(as)f(for)h Fp(MPI)p 1214 1703 V 16 w(SP)l(A)-5 b(WN)p Fu(.)75 1825 y Fm(3.3.4)49 b(Environmental)17 b(inquiry)g(-)g(Prop)q(osal)g(1)75 1911 y Fu(MPI)e(do)q(es)h(not)e(pro)o(vide)i(mec)o(hanisms)g(for)e(in)o (teracting)i(with)f(an)g(external)h(resource)f(manager.)k(It)c(is)75 1967 y(assumed)i(that)g(all)h(suc)o(h)g(in)o(teraction)f(will)i(o)q (ccur)f(through)f(a)g(run)o(time)g(en)o(vironmen)o(t-sp)q(eci\014c)j (API,)75 2024 y(or,)15 b(indirectly)l(,)j(through)d(the)h(opaque)g (\(to)f(MPI\))g(\\info")h(argumen)o(t)f(to)g(the)h(spa)o(wn)g(calls.)22 b(Ho)o(w)o(ev)o(er,)75 2080 y(the)14 b(large)h(ma)s(jorit)o(y)e(of)h (\\dynamic")h(MPI)f(applications)i(are)e(exp)q(ected)i(to)d(exist)i(in) g(a)f(static)h(run)o(time)75 2137 y(en)o(vironmen)o(t,)h(in)h(whic)o(h) g(resources)f(ha)o(v)o(e)f(b)q(een)i(allo)q(cated)g(b)q(efore)f(the)g (application)i(is)f(run.)22 b(This)16 b(is)75 2193 y(the)j(case,)g(for) f(instance,)h(with)g(most)f(PVM)g(applications,)j(whic)o(h)e(rarely)g (manipulate)h(the)e(virtual)75 2249 y(mac)o(hine)d(within)g(an)f (application.)21 b(When)14 b(a)f(user)h(\(or)f(p)q(ossibly)j(a)d(batc)o (h)h(system\))f(runs)h(one)g(of)g(these)75 2306 y(quasi-static)j (applications,)g(he/she/it)g(will)h(usually)f(sp)q(ecify)h(a)e(n)o(um)o (b)q(er)g(of)g(pro)q(cesses)g(to)g(start)f(and)75 2362 y(a)h(total)h(n)o(um)o(b)q(er)g(of)f(pro)q(cesses)h(that)f(are)h(exp)q (ected.)26 b(An)17 b(application)h(simply)g(needs)g(to)e(kno)o(w)g(ho)o (w)75 2419 y(man)o(y)f(slots)g(there)g(are,)g(i.e.,)f(ho)o(w)h(man)o(y) g(pro)q(cesses)g(it)h(should)g(spa)o(wn.)166 2475 y(MPI)h(pro)o(vides)h (an)g(in)o(teger)f(v)m(ariable,)i Fp(MPI)p 943 2475 V 16 w(UNIVERSE)p 1173 2475 V 18 w(SIZE)p Fu(,)d(that)h(tells)h(an)g (application)h(the)75 2532 y(total)e(n)o(um)o(b)q(er)h(of)f(useful)h (pro)q(cess)g(slots)g(a)o(v)m(ailable)h(to)e(an)g(application.)28 b(An)18 b(application)h(will)h(t)o(ypi-)75 2588 y(cally)d(subtract)f (the)h(size)g(of)f Fp(MPI)p 669 2588 V 15 w(COMM)p 826 2588 V 17 w(W)o(ORLD)g Fu(from)f Fp(MPI)p 1211 2588 V 16 w(UNIVERSE)p 1441 2588 V 18 w(SIZE)h Fu(to)g(\014nd)h(out)e(ho)o(w) 75 2645 y(man)o(y)f(pro)q(cesses)g(it)h(should)g(spa)o(wn.)k Fp(MPI)p 821 2645 V 16 w(UNIVERSE)p 1051 2645 V 17 w(SIZE)14 b Fu(is)h(initialized)i(in)e Fp(MPI)p 1562 2645 V 16 w(INIT)e Fu(and)i(is)f(not)75 2701 y(c)o(hanged)i(b)o(y)f(MPI.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fu(11)166 49 y Fp(MPI)p 251 49 14 2 v 16 w(UNIVERSE)p 481 49 V 17 w(SIZE)15 b Fu(is)h(a)e(recommendation,)h (not)g(necessarily)h(a)f(hard)g(limit.)21 b(F)l(or)15 b(instance,)75 106 y(some)20 b(implemen)o(tations)h(ma)o(y)f(allo)o(w)g (an)g(application)i(to)e(spa)o(wn)g(50)f(pro)q(cesses)i(p)q(er)g(pro)q (cessor,)f(if)75 162 y(they)15 b(are)f(requested.)20 b(Ho)o(w)o(ev)o(er,)14 b(it)h(is)g(lik)o(ely)h(that)e(the)h(user)g (only)g(w)o(an)o(ts)e(to)h(spa)o(wn)h(one)g(pro)q(cess)f(p)q(er)75 219 y(pro)q(cessor.)166 275 y Fp(MPI)p 251 275 V 16 w(UNIVERSE)p 481 275 V 17 w(SIZE)c Fu(is)h(assumed)f(to)g(ha)o(v)o(e)f(b)q(een)j(sp) q(eci\014ed)g(when)f(an)f(application)i(w)o(as)d(started,)75 332 y(and)k(is)g(in)g(essence)h(a)e(p)q(ortable)h(mec)o(hanism)g(to)f (allo)o(w)h(the)g(user)f(to)g(pass)h(to)f(the)g(application)j (\(through)75 388 y(the)g(MPI)h(pro)q(cess)f(startup)g(mec)o(hanism\))h (a)f(piece)h(of)f(critical)i(run)o(time)f(information.)k(Note)15 b(that)g(no)75 444 y(in)o(teraction)h(with)g(the)g(run)o(time)g(en)o (vironmen)o(t)g(is)g(required.)23 b(If)16 b(the)g(run)o(time)g(en)o (vironmen)o(t)g(c)o(hanges)75 501 y(size)g(while)i(an)d(application)i (is)f(running)h Fp(MPI)p 880 501 V 16 w(UNIVERSE)p 1110 501 V 17 w(SIZE)f Fu(is)g(not)f(up)q(dated,)h(and)g(the)f(applica-)75 557 y(tion)g(m)o(ust)g(\014nd)h(out)f(through)g(direct)h(comm)o (unication)f(with)h(the)f(run)o(time)h(system.)75 679 y Fm(3.3.5)49 b(Environmental)17 b(inquiry)g(-)g(Prop)q(osal)g(2)75 765 y Fu(While)d(general)g(resource)f(managemen)o(t)g(is)g(b)q(ey)o (ond)h(the)f(scop)q(e)h(of)e(MPI,)h(dynamic)h(applications)h(ma)o(y)75 821 y(need)c(to)f(kno)o(w)f(ho)o(w)h(man)o(y)f(pro)q(cesses)i(they)f (can)h(start.)16 b(The)11 b(function)g Fp(MPI)p 1387 821 V 16 w(GET)p 1491 821 V 16 w(SP)l(A)-5 b(WNABLE)p 1769 821 V 18 w(SIZE)75 878 y Fu(queries)13 b(the)f(run)o(time)g (system)f(to)g(allo)o(w)h(the)g(user)g(to)f(\014nd)i(out)e(ho)o(w)g (man)o(y)g(pro)q(cesses)i(ma)o(y)e(b)q(e)h(started.)75 1029 y Fp(MPI)p 160 1029 V 16 w(GET)p 264 1029 V 17 w(SP)l(A)-5 b(WNABLE)p 543 1029 V 17 w(SIZE\(info,)14 b(size\))117 1106 y Fh(IN)155 b Fp(info)516 b Fh(opaque)14 b(string)g(describing)g (resources)j(required)d(\(string\))117 1181 y(OUT)108 b Fp(size)519 b Fh(n)o(um)o(b)q(er)13 b(of)h(pro)q(cesses)i(that)e(ma)o (y)e(b)q(e)j(started)g(\(in)o(teger\))75 1306 y Fl(int)23 b(MPI)p 245 1306 15 2 v 17 w(Get)p 334 1306 V 17 w(spawnable)p 567 1306 V 16 w(size\(char*)f(info,)i(int*)f(size\))75 1392 y(MPI)p 150 1392 V 17 w(GET)p 239 1392 V 17 w(SPAWNABLE)p 472 1392 V 15 w(SIZE\(info,)g(size\))170 1448 y(CHARACTER*\(*\))g(info) 170 1505 y(INTEGER)g(size)166 1591 y Fu(The)f Fp(info)f Fu(argumen)o(t)g(is)i(of)e(the)h(same)f(form)g(as)g(the)h Fp(info)g Fu(argumen)o(t)e(of)i Fp(MPI)p 1591 1591 14 2 v 16 w(SP)l(A)-5 b(WN)p Fu(.)22 b(It)f(is)75 1648 y(opaque)c(to)e (MPI)i(and)f(is)h(passed)g(directly)h(to)e(the)g(run)o(time)h(en)o (vironmen)o(t,)g(whic)o(h)g(examines)g(it)g(and)75 1704 y(determines)f(ho)o(w)f(man)o(y)g(pro)q(cesses)g(could)h(b)q(e)g(spa)o (wned)g(on)f(the)g(resources)g(describ)q(ed.)166 1761 y Fp(MPI)p 251 1761 V 16 w(GET)p 355 1761 V 16 w(SP)l(A)-5 b(WNABLE)p 633 1761 V 18 w(SIZE)16 b Fu(returns)g(in)g Fp(size)g Fu(the)h(n)o(um)o(b)q(er)f(of)f(pro)q(cesses)i(that)e(could)i (prob-)75 1817 y(ably)j(b)q(e)h(spa)o(wned)f(succesfully)i(if)e Fp(MPI)p 795 1817 V 16 w(SP)l(A)-5 b(WN)20 b Fu(w)o(ere)g(called)h (immediately)h(follo)o(wing.)34 b(There)20 b(is)75 1874 y(no)15 b(guaran)o(tee,)f(ho)o(w)o(ev)o(er,)f(that)h(a)h(subsequen)o(t) h Fp(MPI)p 999 1874 V 15 w(SP)l(A)-5 b(WN)16 b Fu(on)f(the)g(same)f (resources)h(describ)q(ed)i(b)o(y)75 1930 y Fp(info)j Fu(can)g(spa)o(wn)g(that)f(n)o(um)o(b)q(er)i(of)e(pro)q(cesses.)35 b(An)20 b(MPI)g(implemen)o(tation)i(is)e(allo)o(w)o(ed)h(to)e(return)75 1987 y Fp(MPI)p 160 1987 V 16 w(ERR)p 261 1987 V 17 w(SIZE)p 371 1987 V 16 w(UNKNO)o(WN)e Fu(as)e(an)h(error)g(co)q(de)g(\(return)g (v)m(alue)h(\(C\))e(or)h Fp(ierr)f Fu(argumen)o(t)g(\(F)l(ortran\)\))75 2043 y(if)i Fp(info)f Fu(is)g(not)g(empt)o(y)g(and)h(it)f(can)g(not)g (\014nd)h(out)f(the)g(n)o(um)o(b)q(er)h(of)e(pro)q(cesses)i(that)e(ma)o (y)h(b)q(e)h(spa)o(wned)75 2099 y(on)e(the)g(resources)h(describ)q(ed)h (b)o(y)e Fp(info)p Fu(.)166 2156 y(Using)h(an)g(empt)o(y)g Fp(info)g Fu(argumen)o(t,)f Fp(MPI)p 887 2156 V 15 w(GET)p 990 2156 V 17 w(SP)l(A)-5 b(WNABLE)p 1269 2156 V 18 w(SIZE)15 b Fu(and)h Fp(MPI)p 1566 2156 V 16 w(SP)l(A)-5 b(WN)17 b Fu(should)75 2212 y(allo)o(w)e(the)h(user)f(to)g(write)g(p)q(ortable) h(co)q(de)f(without)h(calling)h(a)d(run)o(time-en)o(vironmen)o(t)i (API.)166 2351 y Fi(Discussion:)40 b Fh(Prop)q(osal)15 b(2)g(with)g(an)g(empt)o(y)f Fb(info)h Fh(argumen)o(t)g(pro)o(vides)g (functionalit)o(y)f(similar)f(to)i(pro-)75 2408 y(p)q(osal)e(1,)h (except)h(that)f(it)f(ma)o(y)f(c)o(hange)i(as)g(resources)j(c)o(hange)d (\(or)g(as)g(pro)q(cesses)i(are)f(spa)o(wned\).)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fu(12)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fm(3.3.6)49 b(Sta)o(rting)16 b(Indep)q(endent)g(Pro)q(cesses)75 135 y Fu(The)j(previous)g(section)g (describ)q(ed)h(ho)o(w)e(an)g(MPI)g(application)i(could)g(spa)o(wn)e (new)g(MPI)h(pro)q(cesses)75 192 y(and)c(establish)i(comm)o(unication)f (with)f(them.)166 248 y(This)i(section)f(describ)q(es)i(ho)o(w)e(to)f (spa)o(wn)h Fj(indep)n(endent)f Fu(pro)q(cesses,)h(that)g(is,)g(pro)q (cesses)h(that)e(do)75 304 y(not)g(comm)o(unicate)i(with)f(their)g (paren)o(t.)22 b(These)16 b(ma)o(y)f(b)q(e)i(MPI)f(pro)q(cesses)g (\(i.e.,)f(pro)q(cesses)i(that)e(call)75 361 y Fp(MPI)p 160 361 14 2 v 16 w(INIT)p Fu(\))f(or)h(not.)166 500 y Fi(Discussion:)27 b Fh(This)20 b(section)g(has)g(a)g(large)g(n)o(um)o (b)q(er)f(of)g(hairy)g(functions,)i(guaran)o(teed)f(to)g(scare)h(and)75 556 y(confuse)15 b(users.)75 790 y Fp(MPI)p 160 790 V 16 w(SP)l(A)-5 b(WN)p 326 790 V 17 w(INDEPENDENT)p 659 790 V 17 w(MPI\(command)p 961 790 V 14 w(line,)15 b(minp)o(ro)q(cs,)g (maxp)o(ro)q(cs,)f(info,)h(ro)q(ot,)f(comm,)75 847 y(group,)h(a)o(rra)o (y)p 309 847 V 14 w(of)p 360 847 V 16 w(errco)q(des\))117 924 y Fh(IN)155 b Fp(command)p 506 924 V 16 w(line)321 b Fh(executable)18 b(and)f(argumen)o(ts,)g(in)g(a)f(single)h(string)g (\(string,)905 980 y(signi\014can)o(t)d(only)f(at)g(ro)q(ot\))117 1055 y(IN)155 b Fp(minp)o(ro)q(cs)417 b Fh(minim)n(um)14 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)c(start)i(\(in)o(teger,)g (sig-)905 1112 y(ni\014can)o(t)14 b(only)f(at)h(ro)q(ot\))117 1187 y(IN)155 b Fp(maxp)o(ro)q(cs)408 b Fh(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g(\(in)o(teger,)h (sig-)905 1243 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\))117 1319 y(IN)155 b Fp(info)516 b Fh(a)14 b(string)g(telling)f(the)i(run)o (time)e(system)g(where)j(and)d(ho)o(w)h(to)905 1375 y(start)h(the)f (pro)q(cesses)j(\(string,)c(signi\014can)o(t)h(only)f(at)h(ro)q(ot\)) 117 1450 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in) d(whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 1507 y(\(in)o(teger\))117 1582 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 1657 y(OUT)108 b Fp(group)479 b Fh(group)14 b(of)f(spa)o(wned)i(pro)q(cesses)h(\(handle\))117 1732 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1732 V 15 w(of)p 468 1732 V 16 w(errco)q(des)268 b Fh(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))75 1856 y Fl(int)23 b(MPI)p 245 1856 15 2 v 17 w(Spawn)p 382 1856 V 17 w(independent)p 663 1856 V 15 w(mpi\(char*)g(command)p 1085 1856 V 16 w(line,)g(int)h(minprocs,)393 1913 y(int)g(maxprocs,)e(char*)h(info,)h (int)f(root,)g(MPI)p 1327 1913 V 17 w(Comm)g(comm,)393 1969 y(MPI)p 468 1969 V 17 w(Group*)g(group,)g(int*)g(array)p 1058 1969 V 17 w(of)p 1123 1969 V 17 w(errcodes\))75 2056 y(MPI)p 150 2056 V 17 w(SPAWN)p 287 2056 V 16 w(INDEPENDENT)p 567 2056 V 16 w(MPI\(COMMAND)p 847 2056 V 16 w(LINE,)g(MINPROCS,)f (MAXPROCS,)h(INFO,)g(ROOT,)393 2112 y(COMM,)g(INTERCOMM,)g(ARRAY)p 922 2112 V 16 w(OF)p 986 2112 V 17 w(ERRCODES,)g(IERR\))170 2169 y(CHARACTER*\(*\))g(COMMAND)p 676 2169 V 16 w(LINE,)g(INFO)170 2225 y(INTEGER)g(MINPROCS,)g(MAXPROCS,)g(ROOT,)g(COMM,)g(GROUP,)g (ARRAY)p 1415 2225 V 17 w(OF)p 1480 2225 V 16 w(ERRCODES\(*\),)170 2282 y(IERR)166 2368 y Fu(This)18 b(routine)g(spa)o(wns)g(an)g(indep)q (enden)o(t)i(MPI)d(application.)30 b(It)17 b(is)i(iden)o(tical)g(to)e Fp(MPI)p 1711 2368 14 2 v 16 w(SP)l(A)-5 b(WN)75 2425 y Fu(except)19 b(that)g(it)g(do)q(es)g(not)g(establish)h(comm)o (unication)f(with)h(the)f(c)o(hildren,)i(so)e(that)f(it)h(returns)g(an) 75 2481 y(MPI)12 b(group)g(rather)g(than)g(a)g(comm)o(unicator.)18 b(It)12 b(is)h(collectiv)o(e)h(in)f(the)f(paren)o(t)g(comm)o(unicator)g (but)g(not)75 2537 y(with)18 b(the)g(c)o(hildren.)29 b(Successful)19 b(completion)g(do)q(es)f(not)f(indicate)i(that)e(the)h (pro)q(cesses)g(ha)o(v)o(e)f(called)75 2594 y Fp(MPI)p 160 2594 V 16 w(INIT)d Fu(successfully)l(,)j(only)f(that)f(the)g(pro)q (cesses)g(w)o(ere)g(started)g(successfully)l(.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fu(13)166 49 y Fi(Discussion:)34 b Fh(Alternativ)o(ely)m (,)12 b(w)o(e)i(could)f(sa)o(y)h(that)f(the)i(call)e(do)q(es)h(not)f (return)i(un)o(til)e(the)h(c)o(hildren)g(ha)o(v)o(e)75 106 y(successfully)j(initialized)d(MPI.)i(In)g(this)f(case,)i(the)g (call)e(w)o(ould)g(b)q(e)h(e\013ectiv)o(ely)h(collectiv)o(e)e(with)h (the)g(c)o(hildren.)75 162 y(Another)f(w)o(a)o(y)e(to)g(handle)h(this)g (w)o(oule)f(b)q(e)i(to)e(bring)h(bac)o(k)f(the)i Fb(\015ags)e Fh(argumen)o(t)g(and)h(add)f Fb(MPI)p 1603 162 13 2 v 15 w(SP)m(A)-5 b(WN)p 1756 162 V 14 w(SYNC)75 219 y Fh(to)15 b(the)h(list)e(of)h(supp)q(orted)h(\015ags,)f(so)g(that)g(the)h(user)g (has)f(a)g(c)o(hoice)h(ab)q(out)f(whether)h(the)g(spa)o(wn)f(w)o(aits)g (for)f(the)75 275 y(c)o(hildren)g(to)g(b)q(ecome)g(MPI)g(pro)q(cesses.) 166 414 y Fu(The)d(c)o(hild)i(pro)q(cesses)f(are)f Fj(r)n(e)n(quir)n(e) n(d)f Fu(to)h(call)h Fp(MPI)p 1006 414 14 2 v 16 w(INIT)p Fu(.)e(All)i(c)o(hildren)i(spa)o(wned)d(in)h(a)f(single)h(call)h(to)75 471 y Fp(MPI)p 160 471 V 16 w(SP)l(A)-5 b(WN)p 326 471 V 17 w(INDEPENDENT)p 659 471 V 17 w(MPI)18 b Fu(ha)o(v)o(e)g(the)g (same)g Fp(MPI)p 1164 471 V 16 w(COMM)p 1322 471 V 16 w(W)o(ORLD)p Fu(,)g(whic)o(h)i(is)e(separate)75 527 y(from)c(that)h(of) g(the)g(paren)o(t.)k(In)d(the)g(c)o(hildren,)g Fp(MPI)p 977 527 V 16 w(P)l(ARENT)g Fu(returns)g Fp(MPI)p 1423 527 V 15 w(COMM)p 1580 527 V 17 w(NULL)p Fu(.)189 633 y Fj(A)n(dvic)n(e)f(to)h(users.)40 b Fu(It)15 b(is)g(not)f(correct)h (to)f(use)h Fp(MPI)p 1088 633 V 16 w(SP)l(A)-5 b(WN)p 1254 633 V 17 w(INDEPENDENT)p 1587 633 V 17 w(MPI)14 b Fu(to)g(spa)o(wn)189 690 y(a)k(shell)i(script)f(that)f(starts)f(an)i (MPI)g(program.)29 b(The)19 b(program)e(sp)q(eci\014ed)k(in)e Fp(command)p 1797 690 V 16 w(line)189 746 y Fu(m)o(ust)14 b(itself)i(b)q(e)g(an)f(MPI)h(program)e(that)g(calls)i Fp(MPI)p 1107 746 V 16 w(INIT)p Fu(.)e(\()p Fj(End)i(of)g(advic)n(e)g (to)h(users.)p Fu(\))75 947 y Fp(MPI)p 160 947 V 16 w(SP)l(A)-5 b(WN)p 326 947 V 17 w(INDEPENDENT)p 659 947 V 17 w(NONMPI\(command)p 1058 947 V 15 w(line,)11 b(minp)o(ro)q(cs,)g(maxp)o(ro)q(cs,)f(info,)g (ro)q(ot,)h(comm,)75 1004 y(group,)k(a)o(rra)o(y)p 309 1004 V 14 w(of)p 360 1004 V 16 w(errco)q(des\))117 1081 y Fh(IN)155 b Fp(command)p 506 1081 V 16 w(line)321 b Fh(executable)18 b(and)f(argumen)o(ts,)g(in)g(a)f(single)h(string)g (\(string,)905 1137 y(signi\014can)o(t)d(only)f(at)g(ro)q(ot\))117 1212 y(IN)155 b Fp(minp)o(ro)q(cs)417 b Fh(minim)n(um)14 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)c(start)i(\(in)o(teger,)g (sig-)905 1269 y(ni\014can)o(t)14 b(only)f(at)h(ro)q(ot\))117 1344 y(IN)155 b Fp(maxp)o(ro)q(cs)408 b Fh(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g(\(in)o(teger,)h (sig-)905 1400 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\))117 1475 y(IN)155 b Fp(info)516 b Fh(a)14 b(string)g(telling)f(the)i(run)o (time)e(system)g(where)j(and)d(ho)o(w)h(to)905 1532 y(start)h(the)f (pro)q(cesses)j(\(string,)c(signi\014can)o(t)h(only)f(at)h(ro)q(ot\)) 117 1607 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in) d(whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 1664 y(\(in)o(teger\))117 1739 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 1814 y(OUT)108 b Fp(group)479 b Fh(group)14 b(con)o(taining)f(spa)o(wned)h(pro)q(cesses)j(\(handle\))117 1889 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1889 V 15 w(of)p 468 1889 V 16 w(errco)q(des)268 b Fh(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))75 2013 y Fl(int)23 b(MPI)p 245 2013 15 2 v 17 w(Spawn)p 382 2013 V 17 w(independent)p 663 2013 V 15 w(nonmpi\(char*)g(command)p 1157 2013 V 16 w(line,)g(int)g(minprocs,)393 2070 y(int)h(maxprocs,)e(char*)h (info,)h(int)f(root,)g(MPI)p 1327 2070 V 17 w(Comm)g(comm,)393 2126 y(MPI)p 468 2126 V 17 w(Group*)g(group,)g(int*)g(array)p 1058 2126 V 17 w(of)p 1123 2126 V 17 w(errcodes\))75 2213 y(MPI)p 150 2213 V 17 w(SPAWN)p 287 2213 V 16 w(INDEPENDENT)p 567 2213 V 16 w(NONMPI\(COMMAND)p 919 2213 V 15 w(LINE,)g(MINPROCS,)g (MAXPROCS,)g(INFO,)g(ROOT,)393 2269 y(COMM,)g(GROUP,)g(ARRAY)p 826 2269 V 17 w(OF)p 891 2269 V 17 w(ERRCODES,)f(IERR\))170 2326 y(CHARACTER*\(*\))h(COMMAND)p 676 2326 V 16 w(LINE,)g(INFO)170 2382 y(INTEGER)g(MINPROCS,)g(MAXPROCS,)g(ROOT,)g(COMM,)g(GROUP,)g (ARRAY)p 1415 2382 V 17 w(OF)p 1480 2382 V 16 w(ERRCODES\(*\),)170 2439 y(IERR)166 2525 y Fu(The)15 b(routine)h Fp(MPI)p 501 2525 14 2 v 16 w(SP)l(A)-5 b(WN)p 667 2525 V 17 w(INDEPENDENT)p 1000 2525 V 17 w(NONMPI)15 b Fu(is)h(iden)o(tical)h(to)75 2581 y Fp(MPI)p 160 2581 V 16 w(SP)l(A)-5 b(WN)p 326 2581 V 17 w(INDEPENDENT)p 659 2581 V 17 w(MPI)16 b Fu(except)h(that)f (the)h(c)o(hild)h(pro)q(cesses)g(are)e(assumed)h(to)f(b)q(e)i(non-)75 2638 y(MPI)11 b(pro)q(cesses.)20 b(MPI)11 b(is)h(not)f(required)i(to)e (do)g(an)o(y)g(sp)q(ecial)j(setup)d(to)g(ensure)h(that)f(a)g(call)i(to) e Fp(MPI)p 1774 2638 V 16 w(INIT)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fu(14)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fu(in)j(the)f(c)o(hildren)h (will)h(b)q(e)e(able)h(to)e(establish)i(comm)o(unication)g(among)e(the) h(c)o(hildren.)27 b(The)17 b(e\013ect)f(of)75 106 y(calling)h Fp(MPI)p 304 106 14 2 v 16 w(INIT)d Fu(in)i(the)g(c)o(hildren)h(is)e (unde\014ned.)75 257 y Fp(MPI)p 160 257 V 16 w(SP)l(A)-5 b(WN)p 326 257 V 17 w(MUL)l(TIPLE)p 560 257 V 16 w(INDEPENDENT)p 892 257 V 16 w(MPI\(count,)16 b(a)o(rra)o(y)p 1237 257 V 14 w(of)p 1288 257 V 16 w(command)p 1489 257 V 15 w(lines,)75 313 y(a)o(rra)o(y)p 173 313 V 15 w(of)p 225 313 V 16 w(minp)o(ro)q(cs,)e(a)o(rra)o(y)p 533 313 V 14 w(of)p 584 313 V 16 w(maxp)o(ro)q(cs,)g(a)o(rra)o(y)p 901 313 V 15 w(of)p 953 313 V 16 w(info,)g(ro)q(ot,)h(comm,)d(group,)j(a)o(rra) o(y)p 1547 313 V 14 w(of)p 1598 313 V 16 w(errco)q(des\))117 390 y Fh(IN)155 b Fp(count)482 b Fh(Num)o(b)q(er)12 b(of)g(command)e (lines)j(\(in)o(teger,)g(size)g(of)f(eac)o(h)h(of)f(the)905 447 y(follo)o(wing)f(arra)o(ys\))117 522 y(IN)155 b Fp(a)o(rra)o(y)p 416 522 V 15 w(of)p 468 522 V 16 w(command)p 669 522 V 15 w(lines)142 b Fh(executable)12 b(and)f(argumen)o(ts)f(\(arra)o(y)g (of)g(strings,)h(signi\014can)o(t)905 578 y(only)i(at)h(ro)q(ot\))117 653 y(IN)155 b Fp(a)o(rra)o(y)p 416 653 V 15 w(of)p 468 653 V 16 w(minp)o(ro)q(cs)254 b Fh(minim)n(um)9 b(n)o(um)o(b)q(er)k(of) f(pro)q(cesses)k(for)d(eac)o(h)h(command)c(line)905 710 y(to)k(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can)o(t)h(only) f(at)h(ro)q(ot\))117 785 y(IN)155 b Fp(a)o(rra)o(y)p 416 785 V 15 w(of)p 468 785 V 16 w(maxp)o(ro)q(cs)245 b Fh(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o (h)g(command)d(line)905 842 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o (teger,)f(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 917 y(IN)155 b Fp(a)o(rra)o(y)p 416 917 V 15 w(of)p 468 917 V 16 w(info)353 b Fh(strings)17 b(telling)e(the)i(run)o(time)e (system)i(where)g(and)f(ho)o(w)g(to)905 973 y(start)21 b(pro)q(cesses)i(\(arra)o(y)e(of)e(strings,)k(signi\014can)o(t)c(only)h (at)905 1030 y(ro)q(ot\))117 1105 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d(whic)o(h)h(previous)g(argumen)o (ts)f(are)i(v)n(alid)905 1161 y(\(in)o(teger\))117 1236 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o (wning)g(pro)q(cesses)j(\(handle\))117 1311 y(OUT)108 b Fp(group)479 b Fh(group)14 b(of)f(spa)o(wned)i(pro)q(cesses)h (\(handle\))117 1387 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1387 V 15 w(of)p 468 1387 V 16 w(errco)q(des)268 b Fh(one)14 b(error)h(co)q(de)g(p)q(er)g(pro)q(cess)g(\(arra)o(y)f(of)f(in)o (teger\))75 1511 y Fl(int)23 b(MPI)p 245 1511 15 2 v 17 w(Spawn)p 382 1511 V 17 w(multiple)p 591 1511 V 16 w(independent)p 871 1511 V 15 w(mpi\(int*)g(count,)393 1567 y(char*)g(array)p 659 1567 V 17 w(of)p 724 1567 V 17 w(command)p 909 1567 V 16 w(lines[],)g(int*)g(array)p 1379 1567 V 17 w(of)p 1444 1567 V 16 w(minprocs,)393 1624 y(int*)g(array)p 635 1624 V 17 w(of)p 700 1624 V 17 w(maxprocs,)g(char*)g(array)p 1219 1624 V 16 w(of)p 1283 1624 V 17 w(info[],)g(int)g(root,)393 1680 y(MPI)p 468 1680 V 17 w(Comm)g(comm,)g(MPI)p 819 1680 V 17 w(Group*)g(group,)g (int*)h(array)p 1410 1680 V 16 w(of)p 1474 1680 V 17 w(errcodes\))75 1767 y(MPI)p 150 1767 V 17 w(SPAWN)p 287 1767 V 16 w(MULTIPLE)p 495 1767 V 16 w(INDEPENDENT)p 775 1767 V 16 w(MPI\(COUNT,)f(ARRAY)p 1174 1767 V 16 w(OF)p 1238 1767 V 17 w(COMMAND)p 1423 1767 V 16 w(LINES,)393 1823 y(ARRAY)p 516 1823 V 17 w(OF)p 581 1823 V 17 w(MINPROCS,)f(ARRAY)p 956 1823 V 17 w(OF)p 1021 1823 V 17 w(MAXPROCS,)g(ARRAY)p 1396 1823 V 17 w(OF)p 1461 1823 V 17 w(INFO,)h(ROOT,)393 1880 y(COMM,)g(GROUP,)g(ARRAY)p 826 1880 V 17 w(OF)p 891 1880 V 17 w(ERRCODES,)f(IERR\))170 1936 y(CHARACTER*\(*\))h(ARRAY)p 628 1936 V 16 w(OF)p 692 1936 V 17 w(COMMAND)p 877 1936 V 16 w(LINES\(*\),)g(ARRAY)p 1252 1936 V 16 w(OF)p 1316 1936 V 17 w(INFO)170 1993 y(INTEGER)g(COUNT,)g(ARRAY)p 651 1993 V 17 w(OF)p 716 1993 V 17 w(MINPROCS,)f(ARRAY)p 1091 1993 V 17 w(OF)p 1156 1993 V 17 w(MAXPROCS,)g(ROOT,)i(COMM,)f (GROUP,)170 2049 y(ARRAY)p 293 2049 V 17 w(OF)p 358 2049 V 17 w(ERRCODES\(*\),)f(IERR)166 2136 y Fu(This)g(function)g(is)f(iden) o(tical)i Fp(MPI)p 785 2136 14 2 v 16 w(SP)l(A)-5 b(WN)p 951 2136 V 18 w(MUL)l(TIPLE)20 b Fu(except)i(that)e(it)h(do)q(es)h(not) f(establish)75 2192 y(comm)o(unication)15 b(with)f(the)h(c)o(hildren,)h (so)d(that)h(it)g(returns)h(an)f(MPI)g(group)g(rather)f(than)h(a)g (comm)o(uni-)75 2248 y(cator.)k(The)12 b(c)o(hild)h(pro)q(cesses)g(are) e Fj(r)n(e)n(quir)n(e)n(d)g Fu(to)h(call)h Fp(MPI)p 1053 2248 V 16 w(INIT)p Fu(.)d(All)k(c)o(hildren)f(spa)o(wned)f(in)h(a)f (single)h(call)75 2305 y(to)e Fp(MPI)p 212 2305 V 16 w(SP)l(A)-5 b(WN)p 378 2305 V 17 w(INDEPENDENT)p 711 2305 V 17 w(MPI)11 b Fu(will)i(ha)o(v)o(e)e(the)h(same)f Fp(MPI)p 1271 2305 V 16 w(COMM)p 1429 2305 V 16 w(W)o(ORLD)p Fu(,)g(whic)o(h)h(is)g(sep-)75 2361 y(arate)g(from)g(that)g(of)g(the)h (paren)o(t.)18 b(In)13 b(the)g(c)o(hildren,)i Fp(MPI)p 1073 2361 V 16 w(P)l(ARENT)f Fu(will)g(return)e Fp(MPI)p 1578 2361 V 16 w(COMM)p 1736 2361 V 17 w(NULL)p Fu(.)166 2501 y Fi(Discussion:)53 b Fh(If)19 b(the)g(spa)o(wn)g(w)o(as)g(soft)f (\()p Fb(minp)o(ro)q(cs)i Fa(<)g Fb(maxp)o(ro)q(cs)p Fh(\))f(there)h(is)f(no)f(direct)i(w)o(a)o(y)e(for)g(the)75 2557 y(c)o(hildren)12 b(to)f(\014nd)h(out)g(ho)o(w)f(man)o(y)f(of)h (eac)o(h)h(t)o(yp)q(e)h(w)o(as)e(spa)o(wned.)18 b(The)12 b(paren)o(t)g(can)g(\014gure)g(out)g(this)g(information)75 2613 y(\(b)o(y)i(examining)f Fb(a)o(rra)o(y)p 437 2613 13 2 v 15 w(of)p 486 2613 V 14 w(erro)o(rco)q(des)p Fh(\),)j(but)f (since)g(there)h(is)e(no)h(in)o(tercomm)o(unicator)d(it)i(has)h(no)f (direct)h(w)o(a)o(y)f(to)75 2670 y(tell)f(the)i(c)o(hildren.)j(This)c (problem)f(exists)h(in)g(MPI-1)f(as)h(w)o(ell.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fu(15)75 49 y Fp(MPI)p 160 49 14 2 v 16 w(SP)l(A)-5 b(WN)p 326 49 V 17 w(MUL)l(TIPLE)p 560 49 V 16 w(INDEPENDENT)p 892 49 V 16 w(NONMPI\(count,)16 b(a)o(rra)o(y)p 1334 49 V 15 w(of)p 1386 49 V 16 w(command)p 1587 49 V 15 w(lines,)75 106 y(a)o(rra)o(y)p 173 106 V 15 w(of)p 225 106 V 16 w(minp)o(ro)q(cs,)e(a)o(rra)o(y)p 533 106 V 14 w(of)p 584 106 V 16 w(maxp)o(ro)q(cs,)g(a)o(rra)o(y)p 901 106 V 15 w(of)p 953 106 V 16 w(info,)g(ro)q(ot,)h(comm,)d(group,)j (a)o(rra)o(y)p 1547 106 V 14 w(of)p 1598 106 V 16 w(errco)q(des\))117 183 y Fh(IN)155 b Fp(count)482 b Fh(Num)o(b)q(er)12 b(of)g(command)e (lines)j(\(in)o(teger,)g(size)g(of)f(eac)o(h)h(of)f(the)905 239 y(follo)o(wing)f(arra)o(ys\))117 315 y(IN)155 b Fp(a)o(rra)o(y)p 416 315 V 15 w(of)p 468 315 V 16 w(command)p 669 315 V 15 w(lines)142 b Fh(executable)12 b(and)f(argumen)o(ts)f(\(arra)o(y)g (of)g(strings,)h(signi\014can)o(t)905 371 y(only)i(at)h(ro)q(ot\))117 446 y(IN)155 b Fp(a)o(rra)o(y)p 416 446 V 15 w(of)p 468 446 V 16 w(minp)o(ro)q(cs)254 b Fh(minim)n(um)9 b(n)o(um)o(b)q(er)k(of) f(pro)q(cesses)k(for)d(eac)o(h)h(command)c(line)905 503 y(to)k(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can)o(t)h(only) f(at)h(ro)q(ot\))117 578 y(IN)155 b Fp(a)o(rra)o(y)p 416 578 V 15 w(of)p 468 578 V 16 w(maxp)o(ro)q(cs)245 b Fh(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o (h)g(command)d(line)905 634 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o (teger,)f(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 709 y(IN)155 b Fp(a)o(rra)o(y)p 416 709 V 15 w(of)p 468 709 V 16 w(info)353 b Fh(strings)17 b(telling)e(the)i(run)o(time)e (system)i(where)g(and)f(ho)o(w)g(to)905 766 y(start)21 b(pro)q(cesses)i(\(arra)o(y)e(of)e(strings,)k(signi\014can)o(t)c(only)h (at)905 822 y(ro)q(ot\))117 897 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d(whic)o(h)h(previous)g(argumen)o (ts)f(are)i(v)n(alid)905 954 y(\(in)o(teger\))117 1029 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o (wning)g(pro)q(cesses)j(\(handle\))117 1104 y(OUT)108 b Fp(group)479 b Fh(group)14 b(of)f(spa)o(wned)i(pro)q(cesses)h (\(handle\))117 1179 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1179 V 15 w(of)p 468 1179 V 16 w(errco)q(des)268 b Fh(one)14 b(error)h(co)q(de)g(p)q(er)g(pro)q(cess)g(\(arra)o(y)f(of)f(in)o (teger\))75 1304 y Fl(int)23 b(MPI)p 245 1304 15 2 v 17 w(Spawn)p 382 1304 V 17 w(multiple)p 591 1304 V 16 w(independent)p 871 1304 V 15 w(nonmpi\(int*)g(count,)393 1360 y(char*)g(array)p 659 1360 V 17 w(of)p 724 1360 V 17 w(command)p 909 1360 V 16 w(lines[],)g(int*)g(array)p 1379 1360 V 17 w(of)p 1444 1360 V 16 w(minprocs,)393 1417 y(int*)g(array)p 635 1417 V 17 w(of)p 700 1417 V 17 w(maxprocs,)g(char*)g(array)p 1219 1417 V 16 w(of)p 1283 1417 V 17 w(info[],)g(int)g(root,)393 1473 y(MPI)p 468 1473 V 17 w(Comm)g(comm,)g(MPI)p 819 1473 V 17 w(Group*)g(group,)g (int*)h(array)p 1410 1473 V 16 w(of)p 1474 1473 V 17 w(errcodes\))75 1559 y(MPI)p 150 1559 V 17 w(SPAWN)p 287 1559 V 16 w(MULTIPLE)p 495 1559 V 16 w(INDEPENDENT)p 775 1559 V 16 w(NONMPI\(COUNT,)e(ARRAY)p 1245 1559 V 17 w(OF)p 1310 1559 V 17 w(COMMAND)p 1495 1559 V 16 w(LINES,)393 1616 y(ARRAY)p 516 1616 V 17 w(OF)p 581 1616 V 17 w(MINPROCS,)g(ARRAY)p 956 1616 V 17 w(OF)p 1021 1616 V 17 w(MAXPROCS,)g(ARRAY)p 1396 1616 V 17 w(OF)p 1461 1616 V 17 w(INFO,)h(ROOT,)393 1672 y(COMM,)g(GROUP,)g(ARRAY)p 826 1672 V 17 w(OF)p 891 1672 V 17 w(ERRCODES,)f(IERR\))170 1729 y(CHARACTER*\(*\))h(ARRAY)p 628 1729 V 16 w(OF)p 692 1729 V 17 w(COMMAND)p 877 1729 V 16 w(LINES\(*\),)g(ARRAY)p 1252 1729 V 16 w(OF)p 1316 1729 V 17 w(INFO)170 1785 y(INTEGER)g(COUNT,)g(ARRAY)p 651 1785 V 17 w(OF)p 716 1785 V 17 w(MINPROCS,)f(ARRAY)p 1091 1785 V 17 w(OF)p 1156 1785 V 17 w(MAXPROCS,)g(ROOT,)i(COMM,)f (GROUP,)170 1842 y(ARRAY)p 293 1842 V 17 w(OF)p 358 1842 V 17 w(ERRCODES\(*\),)f(IERR)166 1928 y Fu(The)15 b(routine)h Fp(MPI)p 501 1928 14 2 v 16 w(SP)l(A)-5 b(WN)p 667 1928 V 17 w(MUL)l(TIPLE)p 901 1928 V 16 w(INDEPENDENT)p 1233 1928 V 16 w(NONMPI)16 b Fu(is)f(similar)i(to)75 1985 y Fp(MPI)p 160 1985 V 16 w(SP)l(A)-5 b(WN)p 326 1985 V 17 w(MUL)l(TIPLE)p 560 1985 V 16 w(INDEPENDENT)p 892 1985 V 16 w(MPI)15 b Fu(except)i(that)e(the)h(c)o(hild)h(pro)q(cesses)f (are)g(assumed)75 2041 y(to)e(b)q(e)h(non-MPI)g(pro)q(cesses.)20 b(MPI)15 b(is)g(not)f(required)i(to)e(do)h(an)o(y)f(sp)q(ecial)i(setup) f(to)f(ensure)i(that)d(a)i(call)75 2098 y(to)j Fp(MPI)p 219 2098 V 15 w(INIT)g Fu(in)h(the)f(c)o(hildren)j(will)f(b)q(e)e(able) i(to)d(establish)j(comm)o(unication)f(among)e(the)h(c)o(hildren.)75 2154 y(The)d(e\013ect)g(of)g(calling)i Fp(MPI)p 569 2154 V 16 w(INIT)d Fu(in)i(the)g(c)o(hildren)h(is)f(unde\014ned.)166 2293 y Fi(Discussion:)33 b Fh(This)13 b(routine)h(is)f(here)i(for)e (completeness.)18 b(It)c(do)q(es)g(not)f(pro)o(vide)g(an)o(y)g(missing) e(function-)75 2350 y(alit)o(y)k(\(as)i(do)f(the)h(other)p 490 2350 13 2 v 32 w Fb(MUL)m(TIPLE)f Fh(routines\))h(b)q(ecause)i (there)e(is)g(no)f(need)h(to)g(put)f(all)g(of)g(the)h(c)o(hildren)f(in) 75 2406 y(the)h(same)e Fb(MPI)p 332 2406 V 15 w(COMM)p 477 2406 V 14 w(W)o(ORLD)p Fh(.)h(Similarly)l(,)e(the)j(p)q(ossibilit)o (y)e(of)h(increased)h(comm)o(unication)c(p)q(erformance)75 2463 y(b)q(et)o(w)o(een)k(pro)q(cesses)h(spa)o(wned)e(at)g(the)g(same)e (time)h(is)g(unimp)q(ortan)o(t)f(b)q(ecause)j(the)f(spa)o(wned)g(pro)q (cesses)i(don't)75 2519 y(comm)o(unicate.)28 b(There)19 b(migh)o(t)d(b)q(e)i(a)g(sligh)o(t)f(adv)n(an)o(tage)g(in)h(the)g (e\016ciency)h(of)e(spa)o(wning,)h(but)g(that)g(can)g(b)q(e)75 2575 y(mitigated)12 b(through)i(using)f(the)i(non-blo)q(c)o(king)d (routines.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Fu(16)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fm(3.3.7)49 b(Nonblo)q(cking)18 b(requests)75 135 y Fu(Spa)o(wning)e(new)g(pro)q(cesses)g(ma)o(y)f(b)q (e)h(an)g(exp)q(ensiv)o(e)h(op)q(eration.)k(In)16 b(order)f(to)g(allo)o (w)h(pro)q(cesses)g(to)f(do)75 192 y(useful)i(w)o(ork)e(while)i(pro)q (cesses)g(are)e(b)q(eing)j(spa)o(wned,)d(MPI)h(pro)o(vides)h(non-blo)q (c)o(king)g(v)o(ersions)f(of)g(the)75 248 y(eac)o(h)f(of)g(the)g(spa)o (wning)h(routines)f(de\014ned)i(ab)q(o)o(v)o(e.)75 399 y Fp(MPI)p 160 399 14 2 v 16 w(ISP)l(A)-5 b(WN\(command)p 542 399 V 15 w(line,)16 b(minp)o(ro)q(cs,)e(maxp)o(ro)q(cs,)g(info,)h (ro)q(ot,)f(comm,)f(intercomm,)75 455 y(a)o(rra)o(y)p 173 455 V 15 w(of)p 225 455 V 16 w(errco)q(des,)i(request\))117 533 y Fh(IN)155 b Fp(command)p 506 533 V 16 w(line)321 b Fh(executable)18 b(and)f(argumen)o(ts,)g(in)g(a)f(single)h(string)g (\(string,)905 589 y(signi\014can)o(t)d(only)f(at)g(ro)q(ot\))117 664 y(IN)155 b Fp(minp)o(ro)q(cs)417 b Fh(minim)n(um)14 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)c(start)i(\(in)o(teger,)g (sig-)905 721 y(ni\014can)o(t)14 b(only)f(at)h(ro)q(ot\))117 796 y(IN)155 b Fp(maxp)o(ro)q(cs)408 b Fh(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g(\(in)o(teger,)h (sig-)905 852 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\))117 927 y(IN)155 b Fp(info)516 b Fh(a)14 b(string)g(telling)f(the)i(run)o (time)e(system)g(where)j(and)d(ho)o(w)h(to)905 984 y(start)h(the)f(pro) q(cesses)j(\(string,)c(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 1059 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d (whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 1115 y(\(in)o(teger\))117 1191 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 1266 y(OUT)108 b Fp(intercomm)384 b Fh(in)o(tercomm)o (unicator)12 b(b)q(et)o(w)o(een)j(original)d(group)i(and)f(the)905 1322 y(newly)h(spa)o(wned)g(group)g(\(handle\))117 1397 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1397 V 15 w(of)p 468 1397 V 16 w(errco)q(des)268 b Fh(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))117 1472 y(OUT)108 b Fp(request)452 b Fh(request)16 b(ob)r(ject)e(\(handle\))75 1597 y Fl(int)23 b(MPI)p 245 1597 15 2 v 17 w(Ispawn\(char*)f(command)p 740 1597 V 17 w(line,)h(int)g(minprocs,)g(int)g(maxprocs,)g(char*)g (info,)393 1653 y(int)h(root,)f(MPI)p 707 1653 V 17 w(Comm)g(comm,)g (MPI)p 1058 1653 V 17 w(Comm*)g(intercomm,)393 1710 y(int*)g(array)p 635 1710 V 17 w(of)p 700 1710 V 17 w(errcodes,)g(MPI)p 1028 1710 V 16 w(Request*)g(request\))75 1796 y(MPI)p 150 1796 V 17 w(ISPAWN\(COMMAND)p 503 1796 V 15 w(LINE,)g(MINPROCS,)g (MAXPROCS,)f(INFO,)i(ROOT,)f(COMM,)g(INTERCOMM,)393 1853 y(ARRAY)p 516 1853 V 17 w(OF)p 581 1853 V 17 w(ERRCODES,)f(REQUEST,)h (IERR\))170 1909 y(CHARACTER*\(*\))g(COMMAND)p 676 1909 V 16 w(LINE,)g(INFO)170 1966 y(INTEGER)g(MINPROCS,)g(MAXPROCS,)g(ROOT,) g(COMM,)g(INTERCOMM,)170 2022 y(ARRAY)p 293 2022 V 17 w(OF)p 358 2022 V 17 w(ERRCODES\(*\),)f(REQUEST,)h(IERR)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fu(17)75 49 y Fp(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN)p 339 49 V 17 w(MUL)l(TIPLE\(count,)15 b(a)o(rra)o(y)p 819 49 V 14 w(of)p 870 49 V 16 w(command)p 1071 49 V 16 w(lines,)g(a)o(rra)o(y)p 1292 49 V 15 w(of)p 1344 49 V 16 w(minp)o(ro)q(cs,)75 106 y(a)o(rra)o(y)p 173 106 V 15 w(of)p 225 106 V 16 w(maxp)o(ro)q(cs,)f(a)o(rra)o(y)p 542 106 V 14 w(of)p 593 106 V 16 w(info,)h(ro)q(ot,)f(comm,)f (intercomm,)h(a)o(rra)o(y)p 1282 106 V 14 w(of)p 1333 106 V 16 w(errco)q(des,)i(request\))117 183 y Fh(IN)155 b Fp(count)482 b Fh(Num)o(b)q(er)12 b(of)g(command)e(lines)j(\(in)o (teger,)g(size)g(of)f(eac)o(h)h(of)f(the)905 239 y(follo)o(wing)f(arra) o(ys\))117 315 y(IN)155 b Fp(a)o(rra)o(y)p 416 315 V 15 w(of)p 468 315 V 16 w(command)p 669 315 V 15 w(lines)142 b Fh(executable)12 b(and)f(argumen)o(ts)f(\(arra)o(y)g(of)g(strings,)h (signi\014can)o(t)905 371 y(only)i(at)h(ro)q(ot\))117 446 y(IN)155 b Fp(a)o(rra)o(y)p 416 446 V 15 w(of)p 468 446 V 16 w(minp)o(ro)q(cs)254 b Fh(minim)n(um)9 b(n)o(um)o(b)q(er)k(of) f(pro)q(cesses)k(for)d(eac)o(h)h(command)c(line)905 503 y(to)k(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can)o(t)h(only) f(at)h(ro)q(ot\))117 578 y(IN)155 b Fp(a)o(rra)o(y)p 416 578 V 15 w(of)p 468 578 V 16 w(maxp)o(ro)q(cs)245 b Fh(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o (h)g(command)d(line)905 634 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o (teger,)f(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 709 y(IN)155 b Fp(a)o(rra)o(y)p 416 709 V 15 w(of)p 468 709 V 16 w(info)353 b Fh(strings)17 b(telling)e(the)i(run)o(time)e (system)i(where)g(and)f(ho)o(w)g(to)905 766 y(start)21 b(pro)q(cesses)i(\(arra)o(y)e(of)e(strings,)k(signi\014can)o(t)c(only)h (at)905 822 y(ro)q(ot\))117 897 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d(whic)o(h)h(previous)g(argumen)o (ts)f(are)i(v)n(alid)905 954 y(\(in)o(teger\))117 1029 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o (wning)g(pro)q(cesses)j(\(handle\))117 1104 y(OUT)108 b Fp(intercomm)384 b Fh(in)o(tercomm)o(unicator)12 b(b)q(et)o(w)o(een)j (original)d(group)i(and)g(newly)905 1161 y(spa)o(wned)h(group)e (\(handle\))117 1236 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1236 V 15 w(of)p 468 1236 V 16 w(errco)q(des)268 b Fh(one)14 b(error)h(co)q(de)g(p)q(er)g(pro)q(cess)g(\(arra)o(y)f(of)f(in)o (teger\))117 1311 y(OUT)108 b Fp(request)452 b Fh(request)16 b(ob)r(ject)e(\(handle\))75 1435 y Fl(int)23 b(MPI)p 245 1435 15 2 v 17 w(Ispawn)p 406 1435 V 17 w(multiple\(int*)f(count,)h (char*)g(array)p 1187 1435 V 16 w(of)p 1251 1435 V 17 w(command)p 1436 1435 V 17 w(lines[],)393 1492 y(int*)g(array)p 635 1492 V 17 w(of)p 700 1492 V 17 w(minprocs,)g(int*)g(array)p 1195 1492 V 16 w(of)p 1259 1492 V 17 w(maxprocs,)393 1548 y(char*)g(array)p 659 1548 V 17 w(of)p 724 1548 V 17 w(info[],)g(int)g(root,)g(MPI)p 1242 1548 V 17 w(Comm)g(comm,)393 1605 y(MPI)p 468 1605 V 17 w(Comm*)g(intercomm,)g(int*)g(array)p 1130 1605 V 16 w(of)p 1194 1605 V 17 w(errcodes,)393 1661 y(MPI)p 468 1661 V 17 w(Request*)g(request\))75 1748 y(MPI)p 150 1748 V 17 w(ISPAWN)p 311 1748 V 16 w(MULTIPLE\(COUNT,) f(ARRAY)p 829 1748 V 16 w(OF)p 893 1748 V 17 w(COMMAND)p 1078 1748 V 17 w(LINES,)h(ARRAY)p 1382 1748 V 16 w(OF)p 1446 1748 V 17 w(MINPROCS,)393 1804 y(ARRAY)p 516 1804 V 17 w(OF)p 581 1804 V 17 w(MAXPROCS,)f(ARRAY)p 956 1804 V 17 w(OF)p 1021 1804 V 17 w(INFO,)h(ROOT,)g(COMM,)g(INTERCOMM,)393 1860 y(ARRAY)p 516 1860 V 17 w(OF)p 581 1860 V 17 w(ERRCODES,)f (REQUEST,)h(IERR\))170 1917 y(CHARACTER*\(*\))g(ARRAY)p 628 1917 V 16 w(OF)p 692 1917 V 17 w(COMMAND)p 877 1917 V 16 w(LINES\(*\),)g(ARRAY)p 1252 1917 V 16 w(OF)p 1316 1917 V 17 w(INFO)170 1973 y(INTEGER)g(COUNT,)g(ARRAY)p 651 1973 V 17 w(OF)p 716 1973 V 17 w(MINPROCS,)f(ARRAY)p 1091 1973 V 17 w(OF)p 1156 1973 V 17 w(MAXPROCS,)g(ROOT,)i(COMM,)170 2030 y(INTERCOMM,)f(ARRAY)p 556 2030 V 16 w(OF)p 620 2030 V 17 w(ERRCODES\(*\),)g(REQUEST,)f(IERR)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Fu(18)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fp(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN)p 339 49 V 17 w(INDEPENDENT)p 672 49 V 16 w(MPI\(command)p 973 49 V 15 w(line,)15 b(minp)o(ro)q(cs,)g (maxp)o(ro)q(cs,)f(info,)g(ro)q(ot,)h(comm,)75 106 y(group,)g(a)o(rra)o (y)p 309 106 V 14 w(of)p 360 106 V 16 w(errco)q(des,)h(request\))117 183 y Fh(IN)155 b Fp(command)p 506 183 V 16 w(line)321 b Fh(executable)18 b(and)f(argumen)o(ts,)g(in)g(a)f(single)h(string)g (\(string,)905 239 y(signi\014can)o(t)d(only)f(at)g(ro)q(ot\))117 315 y(IN)155 b Fp(minp)o(ro)q(cs)417 b Fh(minim)n(um)14 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)c(start)i(\(in)o(teger,)g (sig-)905 371 y(ni\014can)o(t)14 b(only)f(at)h(ro)q(ot\))117 446 y(IN)155 b Fp(maxp)o(ro)q(cs)408 b Fh(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g(\(in)o(teger,)h (sig-)905 503 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\))117 578 y(IN)155 b Fp(info)516 b Fh(a)14 b(string)g(telling)f(the)i(run)o (time)e(system)g(where)j(and)d(ho)o(w)h(to)905 634 y(start)h(the)f(pro) q(cesses)j(\(string,)c(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 709 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d (whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 766 y(\(in)o(teger\))117 841 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 916 y(OUT)108 b Fp(group)479 b Fh(group)14 b(of)f(spa)o(wned)i(pro)q(cesses)h(\(handle\))117 991 y(OUT)108 b Fp(a)o(rra)o(y)p 416 991 V 15 w(of)p 468 991 V 16 w(errco)q(des)268 b Fh(one)14 b(co)q(de)h(p)q(er)g(pro)q(cess) h(\(arra)o(y)d(of)h(in)o(teger\))117 1066 y(OUT)108 b Fp(request)452 b Fh(request)16 b(ob)r(ject)e(\(handle\))75 1191 y Fl(int)23 b(MPI)p 245 1191 15 2 v 17 w(Ispawn)p 406 1191 V 17 w(independent)p 687 1191 V 15 w(mpi\(char*)g(command)p 1109 1191 V 16 w(line,)g(int)h(minprocs,)393 1247 y(int)g(maxprocs,)e (char*)h(info,)h(int)f(root,)g(MPI)p 1327 1247 V 17 w(Comm)g(comm,)393 1304 y(MPI)p 468 1304 V 17 w(Group*)g(group,)g(int*)g(array)p 1058 1304 V 17 w(of)p 1123 1304 V 17 w(errcodes,)f(MPI)p 1450 1304 V 17 w(Request*)h(request\))75 1390 y(MPI)p 150 1390 V 17 w(SPAWN)p 287 1390 V 16 w(INDEPENDENT)p 567 1390 V 16 w(MPI\(COMMAND)p 847 1390 V 16 w(LINE,)g(MINPROCS,)f (MAXPROCS,)h(INFO,)g(ROOT,)393 1447 y(COMM,)g(INTERCOMM,)g(ARRAY)p 922 1447 V 16 w(OF)p 986 1447 V 17 w(ERRCODES,)g(REQUEST,)g(IERR\))170 1503 y(CHARACTER*\(*\))g(COMMAND)p 676 1503 V 16 w(LINE,)g(INFO)170 1559 y(INTEGER)g(MINPROCS,)g(MAXPROCS,)g(ROOT,)g(COMM,)g(GROUP,)g (ARRAY)p 1415 1559 V 17 w(OF)p 1480 1559 V 16 w(ERRCODES\(*\),)170 1616 y(REQUEST,)g(IERR)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fu(19)75 49 y Fp(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN)p 339 49 V 17 w(INDEPENDENT)p 672 49 V 16 w(NONMPI\(command)p 1070 49 V 16 w(line,)15 b(minp)o(ro)q(cs,)f (maxp)o(ro)q(cs,)g(info,)h(ro)q(ot,)75 106 y(comm,)e(group,)i(a)o(rra)o (y)p 452 106 V 14 w(of)p 503 106 V 16 w(errco)q(des,)h(request\))117 183 y Fh(IN)155 b Fp(command)p 506 183 V 16 w(line)321 b Fh(executable)18 b(and)f(argumen)o(ts,)g(in)g(a)f(single)h(string)g (\(string,)905 239 y(signi\014can)o(t)d(only)f(at)g(ro)q(ot\))117 315 y(IN)155 b Fp(minp)o(ro)q(cs)417 b Fh(minim)n(um)14 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)c(start)i(\(in)o(teger,)g (sig-)905 371 y(ni\014can)o(t)14 b(only)f(at)h(ro)q(ot\))117 446 y(IN)155 b Fp(maxp)o(ro)q(cs)408 b Fh(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g(\(in)o(teger,)h (sig-)905 503 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\))117 578 y(IN)155 b Fp(info)516 b Fh(a)14 b(string)g(telling)f(the)i(run)o (time)e(system)g(where)j(and)d(ho)o(w)h(to)905 634 y(start)h(the)f(pro) q(cesses)j(\(string,)c(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 709 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d (whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 766 y(\(in)o(teger\))117 841 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 916 y(OUT)108 b Fp(group)479 b Fh(group)14 b(con)o(taining)f(spa)o(wned)h(pro)q(cesses)j(\(handle\))117 991 y(OUT)108 b Fp(a)o(rra)o(y)p 416 991 V 15 w(of)p 468 991 V 16 w(errco)q(des)268 b Fh(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))117 1066 y(OUT)108 b Fp(request)452 b Fh(request)16 b(ob)r(ject)e(\(handle\))75 1191 y Fl(int)23 b(MPI)p 245 1191 15 2 v 17 w(Ispawn)p 406 1191 V 17 w(independent)p 687 1191 V 15 w(nonmpi\(char*)f(command)p 1180 1191 V 17 w(line,)h(int)g(minprocs,)393 1247 y(int)h(maxprocs,)e (char*)h(info,)h(int)f(root,)g(MPI)p 1327 1247 V 17 w(Comm)g(comm,)393 1304 y(MPI)p 468 1304 V 17 w(Group*)g(group,)g(int*)g(array)p 1058 1304 V 17 w(of)p 1123 1304 V 17 w(errcodes,)f(MPI)p 1450 1304 V 17 w(Request*)h(request\))75 1390 y(MPI)p 150 1390 V 17 w(ISPAWN)p 311 1390 V 16 w(INDEPENDENT)p 591 1390 V 16 w(NONMPI\(COMMAND)p 943 1390 V 15 w(LINE,)g(MINPROCS,)g (MAXPROCS,)g(INFO,)g(ROOT,)393 1447 y(COMM,)g(GROUP,)g(ARRAY)p 826 1447 V 17 w(OF)p 891 1447 V 17 w(ERRCODES,)f(REQUEST,)h(IERR\))170 1503 y(CHARACTER*\(*\))g(COMMAND)p 676 1503 V 16 w(LINE,)g(INFO)170 1559 y(INTEGER)g(MINPROCS,)g(MAXPROCS,)g(ROOT,)g(COMM,)g(GROUP,)g (ARRAY)p 1415 1559 V 17 w(OF)p 1480 1559 V 16 w(ERRCODES\(*\),)170 1616 y(REQUEST,)g(IERR)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Fu(20)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fp(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN)p 339 49 V 17 w(MUL)l(TIPLE)p 573 49 V 15 w(INDEPENDENT)p 904 49 V 17 w(MPI\(count,)15 b(a)o(rra)o(y)p 1249 49 V 15 w(of)p 1301 49 V 16 w(command)p 1502 49 V 15 w(lines,)75 106 y(a)o(rra)o(y)p 173 106 V 15 w(of)p 225 106 V 16 w(minp)o(ro)q(cs,)f(a)o(rra)o(y)p 533 106 V 15 w(of)p 585 106 V 16 w(maxp)o(ro)q(cs,)g(a)o(rra)o(y)p 902 106 V 14 w(of)p 953 106 V 16 w(info,)h(ro)q(ot,)f(comm,)f(group,)i (a)o(rra)o(y)p 1548 106 V 15 w(of)p 1600 106 V 16 w(errco)q(des,)75 162 y(request\))117 239 y Fh(IN)155 b Fp(count)482 b Fh(Num)o(b)q(er)12 b(of)g(command)e(lines)j(\(in)o(teger,)g(size)g(of)f (eac)o(h)h(of)f(the)905 296 y(follo)o(wing)f(arra)o(ys\))117 371 y(IN)155 b Fp(a)o(rra)o(y)p 416 371 V 15 w(of)p 468 371 V 16 w(command)p 669 371 V 15 w(lines)142 b Fh(executable)12 b(and)f(argumen)o(ts)f(\(arra)o(y)g(of)g(strings,)h(signi\014can)o(t) 905 427 y(only)i(at)h(ro)q(ot\))117 503 y(IN)155 b Fp(a)o(rra)o(y)p 416 503 V 15 w(of)p 468 503 V 16 w(minp)o(ro)q(cs)254 b Fh(minim)n(um)9 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o (h)h(command)c(line)905 559 y(to)k(start)g(\(arra)o(y)g(of)g(in)o (teger,)f(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 634 y(IN)155 b Fp(a)o(rra)o(y)p 416 634 V 15 w(of)p 468 634 V 16 w(maxp)o(ro)q(cs)245 b Fh(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of) f(pro)q(cesses)k(for)d(eac)o(h)g(command)d(line)905 691 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can)o(t)h (only)f(at)h(ro)q(ot\))117 766 y(IN)155 b Fp(a)o(rra)o(y)p 416 766 V 15 w(of)p 468 766 V 16 w(info)353 b Fh(strings)17 b(telling)e(the)i(run)o(time)e(system)i(where)g(and)f(ho)o(w)g(to)905 822 y(start)21 b(pro)q(cesses)i(\(arra)o(y)e(of)e(strings,)k (signi\014can)o(t)c(only)h(at)905 879 y(ro)q(ot\))117 954 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d (whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 1010 y(\(in)o(teger\))117 1085 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 1161 y(OUT)108 b Fp(group)479 b Fh(group)14 b(of)f(spa)o(wned)i(pro)q(cesses)h(\(handle\))117 1236 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1236 V 15 w(of)p 468 1236 V 16 w(errco)q(des)268 b Fh(one)14 b(error)h(co)q(de)g(p)q(er)g (pro)q(cess)g(\(arra)o(y)f(of)f(in)o(teger\))117 1311 y(OUT)108 b Fp(request)452 b Fh(request)16 b(ob)r(ject)e(\(handle\))75 1435 y Fl(int)23 b(MPI)p 245 1435 15 2 v 17 w(Ispawn)p 406 1435 V 17 w(multiple)p 615 1435 V 16 w(independent)p 895 1435 V 15 w(mpi\(int*)g(count,)393 1492 y(char*)g(array)p 659 1492 V 17 w(of)p 724 1492 V 17 w(command)p 909 1492 V 16 w(lines[],)g(int*)g(array)p 1379 1492 V 17 w(of)p 1444 1492 V 16 w(minprocs,)393 1548 y(int*)g(array)p 635 1548 V 17 w(of)p 700 1548 V 17 w(maxprocs,)g(char*)g(array)p 1219 1548 V 16 w(of)p 1283 1548 V 17 w(info[],)g(int)g(root,)393 1605 y(MPI)p 468 1605 V 17 w(Comm)g(comm,)g(MPI)p 819 1605 V 17 w(Group*)g(group,)g(int*)h(array)p 1410 1605 V 16 w(of)p 1474 1605 V 17 w(errcodes,)393 1661 y(MPI)p 468 1661 V 17 w(Request*)f(request\))75 1748 y(MPI)p 150 1748 V 17 w(ISPAWN)p 311 1748 V 16 w(MULTIPLE)p 519 1748 V 16 w(INDEPENDENT)p 799 1748 V 16 w(MPI\(COUNT,)f(ARRAY)p 1197 1748 V 17 w(OF)p 1262 1748 V 17 w(COMMAND)p 1447 1748 V 16 w(LINES,)393 1804 y(ARRAY)p 516 1804 V 17 w(OF)p 581 1804 V 17 w(MINPROCS,)g(ARRAY)p 956 1804 V 17 w(OF)p 1021 1804 V 17 w(MAXPROCS,)g(ARRAY)p 1396 1804 V 17 w(OF)p 1461 1804 V 17 w(INFO,)h(ROOT,)393 1860 y(COMM,)g(GROUP,)g(ARRAY)p 826 1860 V 17 w(OF)p 891 1860 V 17 w(ERRCODES,)f(REQUEST,)h(IERR\))170 1917 y(CHARACTER*\(*\))g(ARRAY)p 628 1917 V 16 w(OF)p 692 1917 V 17 w(COMMAND)p 877 1917 V 16 w(LINES\(*\),)g(ARRAY)p 1252 1917 V 16 w(OF)p 1316 1917 V 17 w(INFO)170 1973 y(INTEGER)g(COUNT,)g(ARRAY)p 651 1973 V 17 w(OF)p 716 1973 V 17 w(MINPROCS,)f(ARRAY)p 1091 1973 V 17 w(OF)p 1156 1973 V 17 w(MAXPROCS,)g(ROOT,)i(COMM,)f(GROUP,)170 2030 y(ARRAY)p 293 2030 V 17 w(OF)p 358 2030 V 17 w(ERRCODES\(*\),)f (REQUEST,)h(IERR)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fu(21)75 49 y Fp(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN)p 339 49 V 17 w(MUL)l(TIPLE)p 573 49 V 15 w(INDEPENDENT)p 904 49 V 17 w(NONMPI\(count,)16 b(a)o(rra)o(y)p 1347 49 V 15 w(of)p 1399 49 V 16 w(command)p 1600 49 V 15 w(lines,)75 106 y(a)o(rra)o(y)p 173 106 V 15 w(of)p 225 106 V 16 w(minp)o(ro)q(cs,)e(a)o(rra)o(y)p 533 106 V 15 w(of)p 585 106 V 16 w(maxp)o(ro)q(cs,)g(a)o(rra)o(y)p 902 106 V 14 w(of)p 953 106 V 16 w(info,)h(ro)q(ot,)f(comm,)f(group,)i (a)o(rra)o(y)p 1548 106 V 15 w(of)p 1600 106 V 16 w(errco)q(des,)75 162 y(request\))117 239 y Fh(IN)155 b Fp(count)482 b Fh(Num)o(b)q(er)12 b(of)g(command)e(lines)j(\(in)o(teger,)g(size)g(of)f (eac)o(h)h(of)f(the)905 296 y(follo)o(wing)f(arra)o(ys\))117 371 y(IN)155 b Fp(a)o(rra)o(y)p 416 371 V 15 w(of)p 468 371 V 16 w(command)p 669 371 V 15 w(lines)142 b Fh(executable)12 b(and)f(argumen)o(ts)f(\(arra)o(y)g(of)g(strings,)h(signi\014can)o(t) 905 427 y(only)i(at)h(ro)q(ot\))117 503 y(IN)155 b Fp(a)o(rra)o(y)p 416 503 V 15 w(of)p 468 503 V 16 w(minp)o(ro)q(cs)254 b Fh(minim)n(um)9 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o (h)h(command)c(line)905 559 y(to)k(start)g(\(arra)o(y)g(of)g(in)o (teger,)f(signi\014can)o(t)h(only)f(at)h(ro)q(ot\))117 634 y(IN)155 b Fp(a)o(rra)o(y)p 416 634 V 15 w(of)p 468 634 V 16 w(maxp)o(ro)q(cs)245 b Fh(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of) f(pro)q(cesses)k(for)d(eac)o(h)g(command)d(line)905 691 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can)o(t)h (only)f(at)h(ro)q(ot\))117 766 y(IN)155 b Fp(a)o(rra)o(y)p 416 766 V 15 w(of)p 468 766 V 16 w(info)353 b Fh(strings)17 b(telling)e(the)i(run)o(time)e(system)i(where)g(and)f(ho)o(w)g(to)905 822 y(start)21 b(pro)q(cesses)i(\(arra)o(y)e(of)e(strings,)k (signi\014can)o(t)c(only)h(at)905 879 y(ro)q(ot\))117 954 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)13 b(of)f(pro)q(cess)j(in)d (whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 1010 y(\(in)o(teger\))117 1085 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 1161 y(OUT)108 b Fp(group)479 b Fh(group)14 b(of)f(spa)o(wned)i(pro)q(cesses)h(\(handle\))117 1236 y(OUT)108 b Fp(a)o(rra)o(y)p 416 1236 V 15 w(of)p 468 1236 V 16 w(errco)q(des)268 b Fh(one)14 b(error)h(co)q(de)g(p)q(er)g (pro)q(cess)g(\(arra)o(y)f(of)f(in)o(teger\))117 1311 y(OUT)108 b Fp(request)452 b Fh(request)16 b(ob)r(ject)e(\(handle\))75 1435 y Fl(int)23 b(MPI)p 245 1435 15 2 v 17 w(Ispawn)p 406 1435 V 17 w(multiple)p 615 1435 V 16 w(independent)p 895 1435 V 15 w(nonmpi\(int*)g(count,)393 1492 y(char*)g(array)p 659 1492 V 17 w(of)p 724 1492 V 17 w(command)p 909 1492 V 16 w(lines[],)g(int*)g(array)p 1379 1492 V 17 w(of)p 1444 1492 V 16 w(minprocs,)393 1548 y(int*)g(array)p 635 1548 V 17 w(of)p 700 1548 V 17 w(maxprocs,)g(char*)g(array)p 1219 1548 V 16 w(of)p 1283 1548 V 17 w(info[],)g(int)g(root,)393 1605 y(MPI)p 468 1605 V 17 w(Comm)g(comm,)g(MPI)p 819 1605 V 17 w(Group*)g(group,)g(int*)h(array)p 1410 1605 V 16 w(of)p 1474 1605 V 17 w(errcodes,)393 1661 y(MPI)p 468 1661 V 17 w(Request*)f(request\))75 1748 y(MPI)p 150 1748 V 17 w(SPAWN)p 287 1748 V 16 w(MULTIPLE)p 495 1748 V 16 w(INDEPENDENT)p 775 1748 V 16 w(NONMPI\(COUNT,)f(ARRAY)p 1245 1748 V 17 w(OF)p 1310 1748 V 17 w(COMMAND)p 1495 1748 V 16 w(LINES,)393 1804 y(ARRAY)p 516 1804 V 17 w(OF)p 581 1804 V 17 w(MINPROCS,)g(ARRAY)p 956 1804 V 17 w(OF)p 1021 1804 V 17 w(MAXPROCS,)g(ARRAY)p 1396 1804 V 17 w(OF)p 1461 1804 V 17 w(INFO,)h(ROOT,)393 1860 y(COMM,)g(GROUP,)g(ARRAY)p 826 1860 V 17 w(OF)p 891 1860 V 17 w(ERRCODES,)f(REQUEST,)h(IERR\))170 1917 y(CHARACTER*\(*\))g(ARRAY)p 628 1917 V 16 w(OF)p 692 1917 V 17 w(COMMAND)p 877 1917 V 16 w(LINES\(*\),)g(ARRAY)p 1252 1917 V 16 w(OF)p 1316 1917 V 17 w(INFO)170 1973 y(INTEGER)g(COUNT,)g(ARRAY)p 651 1973 V 17 w(OF)p 716 1973 V 17 w(MINPROCS,)f(ARRAY)p 1091 1973 V 17 w(OF)p 1156 1973 V 17 w(MAXPROCS,)g(ROOT,)i(COMM,)f(GROUP,)170 2030 y(ARRAY)p 293 2030 V 17 w(OF)p 358 2030 V 17 w(ERRCODES\(*\),)f (REQUEST,)h(IERR)166 2116 y Fu(F)l(or)c(all)i(of)e(the)h(routines)g(in) h(this)f(section,)h(the)f(argumen)o(ts)f(are)g(iden)o(tical)j(to)d (their)i(blo)q(c)o(king)75 2173 y(coun)o(terparts)14 b(except)h(that)g(there)g(is)g(an)g(additional)h Fp(request)h Fu(argumen)o(t.)i(The)c(request)g(can)g(b)q(e)g(used)75 2229 y(as)c(input)i(for)e Fp(MPI)p 396 2229 14 2 v 16 w(W)l(AIT)p Fu(,)g Fp(MPI)p 631 2229 V 16 w(TEST)p Fu(,)g(etc.)19 b(In)12 b(all)h(cases,)f(the)f(output)h Fp(group)f Fu(or)g Fp(intercomm)p Fu(,)g(and)h(the)75 2286 y Fp(a)o(rra)o(y)p 173 2286 V 15 w(of)p 225 2286 V 16 w(errco)q(des)i Fu(are)g(up)q(dated) h(async)o(hronously)f(and)h(are)f(not)f(v)m(alid)j(un)o(til)f(the)g (request)f(completes.)166 2342 y(The)g(status)f(ob)s(ject)h(returned)g (b)o(y)g Fp(MPI)p 858 2342 V 16 w(W)l(AIT)g Fu(con)o(tains)g(an)g (error)f(co)q(de)h(indicating)i(whether)e(or)75 2399 y(not)h(the)h(spa)o(wn)f(w)o(as)g(successful.)22 b(The)15 b(failure)i(or)e(success)h(of)f(individual)k(pro)q(cesses)d(is)g (indicated)h(in)75 2455 y Fp(a)o(rra)o(y)p 173 2455 V 15 w(of)p 225 2455 V 16 w(errco)q(des)p Fu(.)k(The)16 b(errorco)q(de)g(can)f(b)q(e)i(retriev)o(ed)f(as)f(the)h Fp(MPI)p 1280 2455 V 15 w(ERROR)i Fu(elemen)o(t)e(of)f(the)h(status)75 2511 y(ob)s(ject)f(returned)g(b)o(y)g Fp(MPI)p 545 2511 V 16 w(W)l(AIT)p Fu(,)g Fp(MPI)p 784 2511 V 16 w(TEST)p Fu(,)g(etc.)166 2568 y(All)22 b(of)f(the)g(non-blo)q(c)o(king)h (routines)g(in)f(this)h(section)f(are)g(collectiv)o(e)h(\(o)o(v)o(er)e (the)h(same)g(set)f(of)75 2624 y(pro)q(cesses)c(as)e(their)i(blo)q(c)o (king)h(coun)o(terparts\),)c(and)j(ob)q(ey)f(the)h(the)f(follo)o(wing)h (rules:)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 -100 a Fu(22)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)143 49 y Fo(\017)23 b Fu(A)c(non-blo)q(c)o(king)i(collectiv)o(e)g(routine)f(m)o(ust)e(b)q(e)i (called)h(in)f(ev)o(ery)f(pro)q(cess)h(of)e(the)i(asso)q(ciated)189 106 y(comm)o(unicator)14 b(b)q(efore)i Fp(MPI)p 707 106 14 2 v 16 w(FINALIZE)e Fu(is)i(called.)143 200 y Fo(\017)23 b Fu(The)13 b(op)q(eration)g(is)h(lo)q(cal.)20 b(A)14 b(non-blo)q(c)o(king)h(collectiv)o(e)g(op)q(eration)e(will)i(return)e (ev)o(en)h(if)f(it)h(is)f(not)189 256 y(called)j(immediately)h(on)e (all)i(pro)q(cesses.)143 350 y Fo(\017)23 b Fu(Multiple)17 b(outstanding)f(non-blo)q(c)o(king)h(op)q(erations)e(of)h(the)f(same)g (t)o(yp)q(e)h(on)f(the)h(same)f(comm)o(u-)189 406 y(nicator)f(are)g (allo)o(w)o(ed,)h(and)g(matc)o(h)f(up)h(in)g(the)f(order)h(they)f(w)o (ere)h(called)h(\(since)f(there)g(is)g(no)f Fp(tag)189 463 y Fu(argumen)o(t\).)20 b(Op)q(erations)d(of)e(di\013eren)o(t)i(t)o (yp)q(es)e(and/or)h(on)g(di\013eren)o(t)g(comm)o(unicators)f(ma)o(y)g (b)q(e)189 519 y(in)o(terlea)o(v)o(ed.)143 613 y Fo(\017)23 b Fu(Requests)17 b(cannot)g(b)q(e)h(cancelled.)28 b(An)18 b(attempt)e(to)h(cancel)h(a)f(collectiv)o(e)i(request)e(will)i(return) 189 669 y(an)c(error)f(of)h(error)g(class)g Fp(MPI)p 718 669 V 16 w(ERR)p 819 669 V 17 w(REQUEST)p Fu(.)189 776 y Fj(R)n(ationale.)39 b Fu(A)15 b(serv)o(er)g(ma)o(y)f(p)q(ost)h (man)o(y)f(accept)h(calls)h(so)f(that)f(clien)o(t)i(connections)g (don't)e(fail)189 832 y(or)e(hang.)19 b(When)13 b(the)g(game)g(is)g(o)o (v)o(er,)f(it)h(ma)o(y)g(ha)o(v)o(e)f(some)h(lying)h(around)f(that)f (it)h(needs)h(to)e(clean)189 889 y(up.)20 b(\()p Fj(End)c(of)g(r)n (ationale.)p Fu(\))75 1010 y Fm(3.3.8)49 b(Pro)q(cess)15 b(Utilities)75 1096 y Fu(It)g(is)h(not)e(p)q(ossible)j(to)d(comm)o (unicate,)h(using)h(messages,)e(with)i(a)e(pro)q(cess)i(represen)o(ted) f(b)o(y)g(a)g(\(group,)75 1153 y(rank\))g(pair,)h(except)h(through)e(a) g(comm)o(unicator)h(asso)q(ciated)g(with)g(that)f(group.)22 b(F)l(or)15 b(instance,)h(it)g(is)75 1209 y(not)f(p)q(ossible)i(to)d (send)i(a)f(message)g(to)f(an)h(MPI)h(pro)q(cess)f(spa)o(wned)g(b)o(y) 75 1266 y Fp(MPI)p 160 1266 V 16 w(SP)l(A)-5 b(WN)p 326 1266 V 17 w(INDEPENDENT)p 659 1266 V 17 w(MPI)p Fu(,)14 b(since)i(no)f(comm)o(unicator)g(is)h(a)o(v)m(ailable.)166 1322 y(Ho)o(w)o(ev)o(er,)e(MPI)h(pro)o(vides)h(simple)g(mec)o(hanisms)g (to)f(manage)f(suc)o(h)i(pro)q(cesses.)75 1378 y Fp(MPI)p 160 1378 V 16 w(PROCESS)p 373 1378 V 18 w(SIGNAL)c Fu(sends)g(a)g (signal)h(to)e(a)h(pro)q(cess,)g(and)g Fp(MPI)p 1259 1378 V 16 w(PROCESS)p 1472 1378 V 18 w(MONITOR)g Fu(pro)o(vides)75 1435 y(a)j(mec)o(hanism)h(to)e(detect)i(when)f(a)g(pro)q(cess)h(dies)g (or)f(c)o(hanges)g(state.)75 1586 y Fp(MPI)p 160 1586 V 16 w(PROCESS)p 373 1586 V 18 w(SIGNAL\(group,)f(rank,)h(signal\))117 1663 y Fh(IN)155 b Fp(group)479 b Fh(group)14 b(con)o(taining)f(pro)q (cess)i(to)f(signal)f(\(handle\))117 1738 y(IN)155 b Fp(rank)504 b Fh(rank)14 b(of)f(pro)q(cess)j(to)e(signal)f(\(in)o (teger\))117 1813 y(IN)155 b Fp(signal)480 b Fh(signal)13 b(t)o(yp)q(e)h(\(in)o(teger\))75 1938 y Fl(int)23 b(MPI)p 245 1938 15 2 v 17 w(Process)p 430 1938 V 16 w(signal\(MPI)p 686 1938 V 16 w(Group)g(group,)g(int)h(rank,)f(int)h(signal\))75 2024 y(MPI)p 150 2024 V 17 w(PROCESS)p 335 2024 V 16 w(SIGNAL\(GROUP,)e(RANK,)h(SIGNAL,)g(IERR\))170 2081 y(INTEGER)g(GROUP,)g(RANK,)h(SIGNAL,)e(IERR)166 2167 y Fp(MPI)p 251 2167 14 2 v 16 w(PROCESS)p 464 2167 V 18 w(SIGNAL)g Fu(sends)g(a)g(signal)h(to)e(a)h(pro)q(cess)g(represen)o (ted)g(b)o(y)g(a)g(\(group,)h(rank\))75 2224 y(pair.)30 b(Where)18 b(POSIX)i(signals)f(are)f(supp)q(orted,)h Fp(signal)g Fu(is)f(a)g(signal)i(de\014ned)f(b)o(y)g(POSIX.)g(It)f(is)h (the)75 2280 y(resp)q(onsibilit)o(y)f(of)d(an)h(implemen)o(tation)g(to) f(translate)h(b)q(et)o(w)o(een)f(signals;)i(in)f(other)f(w)o(ords,)g(a) g Fl(SIGINT)75 2336 y Fu(that)e(has)h(v)m(alue)i Fl(3)e Fu(on)g(system)g(A)g(m)o(ust)f(b)q(e)i(deliv)o(ered)h(as)e(a)g Fl(SIGINT)f Fu(on)h(system)g(B,)g(ev)o(en)g(if)h(system)f(B)75 2393 y(uses)j(the)f(v)m(alue)i Fl(5)f Fu(for)f Fl(SIGINT)p Fu(.)f(If)i(there)f(is)h(no)g(corresp)q(onding)g(signal,)h(the)e (results)h(are)f(unde\014ned,)75 2449 y(though)g(not)h(erroneous,)f(so) g(that)g(the)h(signal)g(is)h(silen)o(tly)g(con)o(v)o(erted)e(to)g(a)g (signal)i(with)f(meaning)g(on)75 2506 y(the)e(target)f(mac)o(hine,)i (or)f(is)g(dropp)q(ed)h(completely)l(.)166 2562 y(An)f(MPI)h(implemen)o (tation)g(m)o(ust)f(also)g(supp)q(ort)g(the)g(MPI-de\014ned)i(signal)f (t)o(yp)q(e)75 2619 y Fp(MPI)p 160 2619 V 16 w(SIGNAL)p 331 2619 V 16 w(KILL)p Fu(.)21 b(Sending)j(this)f(\\signal")f(will)i (reliably)g(kill)h(a)d(pro)q(cess)g(and)h(attempt)e(to)h(do)75 2675 y(the)f(necessary)h(system-lev)o(el)g(clean)o(up.)39 b(In)22 b(order)e(to)h(allo)o(w)g(appropriate)g(clean)o(up)i(to)d(o)q (ccur,)j(an)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 23 24 bop 75 -100 a Fn(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fu(23)75 49 y(implemen)o(tation)20 b(could,)g(for)e (instance,)i(send)f(a)f(catc)o(hable)h(signal)g(\(e.g.,)f(SIGINT\))h (follo)o(w)o(ed)g(b)o(y)f(a)75 106 y(noncatc)o(hable)i(signal)g (\(SIGKILL\))g(if)g(the)f(pro)q(cess)g(had)h(not)e(exited)i(after)f(a)f (short)h(time)g(in)o(terv)m(al.)75 162 y(MPI)g(requires)h(only)f(that)f (the)h(pro)q(cess)h(b)q(e)f(reliably)i(killed.)33 b(Applications)21 b(should)f(not)f(rely)g(on)g(a)75 219 y(sp)q(eci\014c)e(implemen)o (tation.)166 275 y(Signals)e(are)e(deliv)o(ered)i(async)o(hronously)l (.)20 b(That)13 b(is,)h(when)h(the)e Fp(MPI)p 1362 275 14 2 v 16 w(SIGNAL)h Fu(function)g(returns,)75 332 y(the)k(signal)g(ma) o(y)g(not)f(y)o(et)g(ha)o(v)o(e)h(b)q(een)h(deliv)o(ered.)29 b(If)18 b(a)g(signal)g(is)g(undeliv)o(erable)j(\(e.g.,)c(the)h(pro)q (cess)75 388 y(has)d(already)h(exited\),)f(the)g(signal)h(is)g(silen)o (tly)h(dropp)q(ed.)166 527 y Fi(Discussion:)34 b Fh(In)14 b(man)o(y)e(cases)k(when)e(a)g(signal)f(is)h(undeliv)o(erable,)f(MPI)h (ma)o(y)e(kno)o(w)i(when)g(the)h(routine)75 584 y(is)k(called)f(that)h (it)f(is)g(undeliv)o(erable)h(\(e.g.,)g(the)g(pro)q(cess)i(has)d (already)h(b)q(een)g(mark)o(ed)f(dead\).)33 b(It)18 b(w)o(ould)g(b)q(e) 75 640 y(p)q(ossible)e(to)f(return)i(an)f(error)g(in)f(this)h(case,)h (though)e(error)i(rep)q(orting)f(w)o(ould)f(b)q(e)h(unreliable,)f(and)h (w)o(ould)e(b)q(e)75 696 y(an)k(exception)i(to)e(the)h(usual)g(MPI)g (rule)f(that)h(all)e(errors)j(are)f(fatal)f(b)o(y)g(default.)32 b(Alternativ)o(ely)m(,)18 b(w)o(e)h(could)75 753 y(require)c(that)f (signals)f(b)q(e)i(sen)o(t)g(sync)o(hronously)m(,)e(and)h(return)h(a)f (status)h(indicating)d(whether)k(it)d(w)o(as)h(successful)75 809 y(or)g(not)g(\(though)g(the)h(return)h(v)n(alue)d(of)h(the)g (function)g(w)o(ould)g(b)q(e)g Fb(MPI)p 1201 809 13 2 v 15 w(SUCCESS)e Fh(unless)j(there)h(w)o(ere)f(an)f(error)75 866 y(in)f(argumen)o(ts\).)75 1099 y Fp(MPI)p 160 1099 14 2 v 16 w(GROUP)p 328 1099 V 18 w(SIGNAL\(group,)h(signal\))117 1177 y Fh(IN)155 b Fp(group)479 b Fh(group)14 b(of)f(pro)q(cesses)k(to) c(signal)g(\(handle\))117 1252 y(IN)155 b Fp(signal)480 b Fh(signal)13 b(t)o(yp)q(e)h(\(in)o(teger\))75 1376 y Fl(int)23 b(MPI)p 245 1376 15 2 v 17 w(Group)p 382 1376 V 17 w(signal\(MPI)p 639 1376 V 16 w(Group)g(group,)g(int)g (signal\))75 1463 y(MPI)p 150 1463 V 17 w(GROUP)p 287 1463 V 16 w(SIGNAL\(GROUP,)f(SIGNAL,)h(IERR\))170 1519 y(INTEGER)g(GROUP,)g(SIGNAL,)g(IERR)166 1606 y Fu(The)15 b(con)o(v)o(enience)i(function)f Fp(MPI)p 771 1606 14 2 v 16 w(GROUP)p 939 1606 V 18 w(SIGNAL)f Fu(is)h(equiv)m(alen)o(t)h (to)d(calling)75 1662 y Fp(MPI)p 160 1662 V 16 w(PROCESS)p 373 1662 V 18 w(SIGNAL)h Fu(once)h(for)e(eac)o(h)i(rank)e(in)j(the)e (group.)189 1768 y Fj(A)n(dvic)n(e)e(to)i(users.)39 b Fu(Signals)15 b(do)e(not)g(pro)o(vide)h(an)o(y)g(of)f(the)g(reliabilit) o(y)j(or)d(guaran)o(tees)g(of)g(regular)189 1825 y(MPI)j(comm)o (unication.)23 b(A)16 b(high)h(qualit)o(y)g(implemen)o(tation)g(will)h (deliv)o(er)f(signals)g(quic)o(kly)h(and)189 1881 y(reliably)l(,)23 b(but)d(applications)i(should)f(nev)o(er)f(dep)q(end)i(on)e(ordering.) 35 b(Since)21 b(signals)g(ma)o(y)e(not)189 1938 y(b)q(e)h(queued,)i (when)e(m)o(ultiple)i(signals)f(are)e(sen)o(t,)i(only)f(one)g(ma)o(y)f (get)h(through.)33 b(Some)20 b(MPI)189 1994 y(implemen)o(tations)e(ma)o (y)f(use)h(signals)g(in)o(ternally)l(.)29 b(If)18 b(an)f(application)i (attempts)e(to)g(use)h(these)189 2051 y(signals,)g(unexp)q(ected)i(b)q (eha)o(vior)e(ma)o(y)f(result.)27 b(Finally)l(,)20 b(it)e(ma)o(y)f(not) g(b)q(e)h(p)q(ossible)i(at)d(the)g(im-)189 2107 y(plemen)o(tation)g (lev)o(el)i(to)d(restrict)h(the)g(e\013ects)g(of)f(a)h(signal)h(to)e(a) h(single)h(MPI)f(pro)q(cess,)g(so)g(that)189 2164 y(a)d(side-e\013ect)i (of)f(sending)h(a)e(signal)i(to)e(a)h(pro)q(cess)g(ma)o(y)f(b)q(e)i (that)e(other)h(pro)q(cesses)g(receiv)o(e)h(that)189 2220 y(signal)g(or)e(are)h(killed.)22 b(\()p Fj(End)16 b(of)g(advic)n(e)g(to)h(users.)p Fu(\))75 2342 y Fm(3.3.9)49 b(Noti\014cation)18 b(of)e(change)g(in)h(state)e(of)i(a)g(p)o(ro)q (cess)75 2428 y Fu(In)f(order)g(to)f(manage)g(pro)q(cesses)i(or)e(ha)o (v)o(e)g(the)h(minim)o(um)h(in)g(fault)f(tolerance,)g(an)g(MPI)f (application)75 2484 y(m)o(ust)g(b)q(e)g(able)h(to)f(b)q(e)h (noti\014ed)g(when)g(a)f(pro)q(cess)g(exits.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 24 25 bop 75 -100 a Fu(24)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fp(MPI)p 160 49 14 2 v 16 w(PROCESS)p 373 49 V 18 w(MONITOR\(group,)g(rank,)g(event,)h (request\))117 126 y Fh(IN)155 b Fp(group)479 b Fh(a)14 b(group)g(of)f(pro)q(cesses)j(\(handle\))117 202 y(IN)155 b Fp(rank)504 b Fh(rank)14 b(of)f(pro)q(cess)j(to)e(b)q(e)g(monitored)f (\(in)o(teger\))117 277 y(IN)155 b Fp(event)487 b Fh(a)15 b(\015ag)f(sp)q(ecifying)h(what)g(ev)o(en)o(t)h(to)f(b)q(e)g (noti\014ed)h(ab)q(out)e(\(in-)905 333 y(teger\))117 408 y(OUT)108 b Fp(request)452 b Fh(a)14 b(request)h(ob)r(ject)g (\(handle\))75 533 y Fl(int)23 b(MPI)p 245 533 15 2 v 17 w(Process)p 430 533 V 16 w(monitor\(MPI)p 710 533 V 16 w(Group)g(group,)g(int)h(rank,)f(int)g(event,)393 589 y(MPI)p 468 589 V 17 w(Request*)g(request\))75 676 y(MPI)p 150 676 V 17 w(PROCESS)p 335 676 V 16 w(MONITOR\(GROUP,)f (RANK,)h(EVENT,)g(REQUEST,)g(IERR\))170 732 y(INTEGER)g(GROUP,)g(RANK,) h(EVENT,)f(REQUEST,)f(IERR)166 819 y Fu(This)13 b(function)h(pro)o (vides)f(a)f(general)i(metho)q(d)f(for)f(an)g(MPI)h(pro)q(cess)g(to)f (detect)h(a)f(c)o(hange)h(in)h(state)75 875 y(of)h(another)h(pro)q (cess.)21 b(The)16 b(request)g(completes)g(when)h(the)f(pro)q(cess)g(c) o(hanges)f(state)g(as)h(indicated)h(b)o(y)75 932 y Fp(event)p Fu(.)166 988 y Fp(event)i Fu(is)f(an)f(in)o(teger)h(sp)q(ecifying)h(a)e (c)o(hange)h(of)f(state)f(that)h(the)g(calling)j(pro)q(cess)d(wishes)i (to)d(de-)75 1044 y(tect.)j(The)d(most)e(imp)q(ortan)o(t)g(state-c)o (hange)g(and)h(the)g(only)h(one)f(MPI)g(curren)o(tly)g(de\014nes)h(for) e Fp(event)i Fu(is)75 1101 y Fp(MPI)p 160 1101 14 2 v 16 w(PROCESS)p 373 1101 V 18 w(DIED)p Fu(.)f(F)l(or)h(this)h(case,)g Fp(request)h Fu(completes)f(when)g(the)g(pro)q(cess)g(terminates)f(or)g (b)q(e-)75 1157 y(comes)d(unreac)o(hable.)20 b(If)14 b(a)f(pro)q(cess)g(has)g(already)g(terminated)h(or)f(b)q(ecome)g (unreac)o(hable,)i(the)e(request)75 1214 y(completes)j(immediately)l(.) 22 b(If)16 b(an)f(application)i(w)o(an)o(ts)d(to)h(b)q(e)h(noti\014ed)g (of)f(sev)o(eral)g(di\013eren)o(t)h(ev)o(en)o(ts,)f(it)75 1270 y(m)o(ust)h(call)i Fp(MPI)p 359 1270 V 16 w(PROCESS)p 572 1270 V 18 w(MONITOR)f Fu(once)g(for)g(eac)o(h)f(ev)o(en)o(t.)25 b(If)17 b(a)f(m)o(ultiple)j(requests)d(are)h(activ)o(e)75 1327 y(for)e(a)f(single)j(t)o(yp)q(e)e(of)g(ev)o(en)o(t,)g(all)h (requests)f(will)i(complete)f(when)f(the)h(ev)o(en)o(t)f(o)q(ccurs.)166 1459 y Fi(Discussion:)42 b Fb(MPI)p 519 1459 13 2 v 14 w(PROCESS)p 716 1459 V 13 w(DIED)15 b Fh(do)q(es)i(not)f(ha)o(v)o(e)f (an)o(y)h(race)g(conditions)g(asso)q(ciated)h(with)e(it)g(b)q(e-)75 1509 y(cause)i(it)e(is)g(p)q(ossible)h(to)g(determine)f(whether)i(a)f (pro)q(cess)h(is)f(dead)g(b)o(y)f(examining)e(it.)23 b(A)16 b(h)o(yp)q(othetical)f(ev)o(en)o(t)75 1559 y Fb(MPI)p 152 1559 V 14 w(PROCESS)p 349 1559 V 14 w(MIGRA)m(TED)p Fh(,)e(on)j(the)h(other)f(hand,)g(is)g(m)o(uc)o(h)e(more)h(complicated) g(to)g(handle,)h(since)h(once)f(a)75 1609 y(pro)q(cess)e(has)d (migrated,)f(it)i(app)q(ears)g(the)h(same)d(as)i(an)o(y)f(other)i(pro)q (cess.)19 b(This)11 b(means)g(there)i(is)f(a)f(race)i(condition)75 1658 y(b)q(et)o(w)o(een)j(the)f(call)f(to)g Fb(MPI)p 513 1658 V 14 w(PROCESS)p 710 1658 V 13 w(MONITOR)g Fh(and)g(the)h (actual)g(migration,)c(so)j(that)h(it)f(is)g(imp)q(ossible)f(to)75 1708 y(guaran)o(tee)h(that)g(ev)o(en)o(ts)h(w)o(on't)e(b)q(e)i(lost.) 166 1758 y(One)h(w)o(a)o(y)f(to)g(handle)g(this)h(is)f(to)g(ha)o(v)o(e) h(\\stic)o(ky")f(ev)o(en)o(ts)h(that)f(MPI)h(remem)o(b)q(ers)f(and)g (can)h(giv)o(e)f(bac)o(k)g(to)75 1808 y(the)k(user)g(in)e(the)i(order)f (receiv)o(ed.)32 b(Since)18 b(w)o(e)h(do)e(not)h(in)g(general)g(w)o(an) o(t)f(for)h(ev)o(ery)h(pro)q(cess)h(to)d(b)q(e)i(noti\014ed)75 1858 y(ab)q(out)13 b(ev)o(ery)h(ev)o(en)o(t)g(in)f(ev)o(ery)h(other)g (pro)q(cess)h(it)e(kno)o(ws)h(ab)q(out,)f(it)f(m)o(ust)h(b)q(e)h(p)q (ossible)f(to)h(turn)f(on)g(and)h(o\013)f(this)75 1908 y(stic)o(kiness,)j(and)f(there)i(is)e(a)g(new)h(race)g(condition,)e (admittedly)f(sligh)o(tly)h(more)h(di\016cult)f(to)h(trigger,)h(b)q(et) o(w)o(een)75 1957 y(\\early")d(ev)o(en)o(ts)i(and)f(turning)g(on)f (stic)o(kiness.)166 2007 y(A)h(h)o(yp)q(othetical)h(\\ev)o(en)o(t")f Fb(MPI)p 679 2007 V 15 w(PROCESS)p 877 2007 V 13 w(SLEEPING)g Fh(falls)f(somewhere)i(in)f(the)h(middle.)j(Whether)d(or)75 2057 y(not)f(a)f(pro)q(cess)j(is)e(sleeping)f(can)i(b)q(e)f(determined) g(b)o(y)f(examining)f(the)i(curren)o(t)h(state)g(of)e(the)i(pro)q (cess,)g(it)e(is)h(still)75 2107 y(p)q(ossible)g(to)g(miss)f(a)g (sleep/w)o(ak)o(eup)i(pair.)166 2157 y(The)k(requiremen)o(t)g(that)g (all)f(requests)j(complete)d(for)h(a)f(single)h(ev)o(en)o(t)g(rather)h (than)f(one)g(at)g(a)g(time)e(is)75 2206 y(motiv)n(ated)9 b(b)o(y)j(the)g(observ)n(ation)f(that)g(sp)q(ecifying)h(one)f(request)i (completion)d(p)q(er)i(ev)o(en)o(t)g(w)o(ould)f(not)g(completely)75 2256 y(solv)o(e)16 b(the)i(race)f(condition)f(it)g(tries)h(to)g (address.)27 b(Moreo)o(v)o(er,)17 b(it)g(encourages)g(writing)f(hairy)g (co)q(de)i(to)e(alw)o(a)o(ys)75 2306 y(k)o(eep)i(m)o(ultiple)d (requests)20 b(outstanding.)28 b(Finally)m(,)16 b(there)j(is)e(curren)o (tly)i(no)e Fb(event)i Fh(for)e(whic)o(h)h(it)f(w)o(ould)f(mak)o(e)75 2356 y(sense.)166 2495 y Fu(F)l(uture)c(v)o(ersions)h(of)f(the)h(MPI)f (standard)h(ma)o(y)e(sp)q(ecify)j(new)f(t)o(yp)q(es)g(of)f(ev)o(en)o (ts,)g(and)h(implemen)o(ta-)75 2551 y(tions)h(ma)o(y)f(de\014ne)i(new)f (ones)g(as)f(w)o(ell.)21 b(If)14 b(an)g(application)h(w)o(an)o(ts)e(to) g(b)q(e)i(noti\014ed)f(of)g(sev)o(eral)g(di\013eren)o(t)75 2608 y(ev)o(en)o(ts,)h(it)g(m)o(ust)g(call)h Fp(MPI)p 552 2608 14 2 v 16 w(PROCESS)p 765 2608 V 18 w(MONITOR)g Fu(once)f(for)g(eac)o(h)g(ev)o(en)o(t.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 25 26 bop 75 -100 a Fn(3.4.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fu(25)166 49 y(F)l(or)g(a)h(request)h(generated)f(b)o(y)g Fp(MPI)p 784 49 14 2 v 16 w(PROCESS)p 997 49 V 18 w(MONITOR)p Fu(,)g Fp(MPI)p 1333 49 V 16 w(W)l(AIT)g Fu(returns)g(a)g(status)g (that)75 106 y(con)o(tains)j(no)g(useful)i(information)e(\(i.e.,)g(the) g(status)f(is)i(unde\014ned\).)166 245 y Fi(Discussion:)47 b Fh(One)18 b(p)q(ossibilit)o(y)e(w)o(ould)h(b)q(e)g(to)h(get)f(the)h (exit)f(co)q(de)h(from)e(the)h(status.)29 b(Ho)o(w)o(ev)o(er,)18 b(the)75 301 y(pro)q(cess)f(ma)o(y)c(not)i(ha)o(v)o(e)f(exited)i(-)f (it)f(ma)o(y)f(ha)o(v)o(e)i(b)q(ecome)g(unreac)o(hable.)22 b(It)15 b(ma)o(y)e(also)h(ha)o(v)o(e)h(b)q(een)h(killed)e(b)o(y)g(a)75 358 y(signal)c(and)g(what)h(w)o(e'd)g(w)o(an)o(t)f(to)h(kno)o(w)f(is)h (what)f(signal)g(killed)g(it.)16 b(In)11 b(the)h(end,)f(w)o(e)g(migh)o (t)e(ha)o(v)o(e)h(to)h(reimplemen)o(t)75 414 y(w)o(ait.)75 648 y Fp(MPI)p 160 648 V 16 w(GROUP)p 328 648 V 18 w(MONITOR\(group,)k (event,)h(a)o(rra)o(y)p 935 648 V 14 w(of)p 986 648 V 16 w(requests\))117 725 y Fh(IN)155 b Fp(event)487 b Fh(a)14 b(\015ag)f(sp)q(ecifying)h(what)g(ev)o(en)o(t)g(to)g(b)q(e)h (noti\014ed)e(ab)q(out)117 800 y(IN)155 b Fp(group)479 b Fh(a)14 b(group)g(of)f(pro)q(cesses)117 875 y(OUT)108 b Fp(a)o(rra)o(y)p 416 875 V 15 w(of)p 468 875 V 16 w(requests)272 b Fb(MPI)p 982 875 13 2 v 15 w(Request)p Fh(s)15 b(to)f(b)q(e)g(tested) i(and/or)d(w)o(aited)h(on)75 1000 y Fl(int)23 b(MPI)p 245 1000 15 2 v 17 w(Group)p 382 1000 V 17 w(monitor\(MPI)p 663 1000 V 15 w(Group)h(group,)f(int)g(event,)393 1056 y(MPI)p 468 1056 V 17 w(Request*)g(array)p 820 1056 V 16 w(of)p 884 1056 V 17 w(requests\))75 1143 y(MPI)p 150 1143 V 17 w(GROUP)p 287 1143 V 16 w(MONITOR\(GROUP,)f(EVENT,)h (ARRAY)p 948 1143 V 17 w(OF)p 1013 1143 V 17 w(REQUESTS,)f(IERR\))170 1199 y(INTEGER)h(GROUP,)g(EVENT,)g(ARRAY)p 818 1199 V 17 w(OF)p 883 1199 V 17 w(REQUESTS\(*\),)f(IERR)166 1286 y Fu(This)c(con)o(v)o(enience)h(function)f(is)g(equiv)m(alen)o(t)h(to)e (calling)i Fp(MPI)p 1258 1286 14 2 v 16 w(PROCESS)p 1471 1286 V 18 w(MONITOR)f Fu(once)g(for)75 1342 y(eac)o(h)c(pro)q(cess)h (in)g Fp(group)p Fu(.)k(The)c Ff(i)p Fu(th)f(elemen)o(t)h(\(n)o(um)o(b) q(ered)f(from)g(0\))f(of)h Fp(a)o(rra)o(y)p 1396 1342 V 15 w(of)p 1448 1342 V 16 w(requests)i Fu(is)f(asso)q(ciated)75 1398 y(with)h(rank)e Ff(i)h Fu(in)h(the)g(group.)166 1455 y Fp(a)o(rra)o(y-of-requests)c Fu(m)o(ust)e(ha)o(v)o(e)h(at)g (least)h(as)f(man)o(y)g(elemen)o(ts)h(as)f(the)g(size)i(of)e(the)g (group.)18 b(A)12 b(request)75 1511 y(completes)18 b(when)f(the)g (corresp)q(onding)h(mem)o(b)q(er)f(of)g(the)g(group)f(c)o(hanges)h(to)g (the)g(state)f(sp)q(eci\014ed)j(b)o(y)75 1568 y Fp(event)p Fu(.)75 1711 y Fr(3.4)59 b(Establishing)20 b(Communication)h(Bet)n(w)n (een)f(Indep)r(endent)g(Pro)r(cesses)75 1889 y Fi(Discussion:)35 b Fh(This)14 b(prop)q(osal)f(w)o(as)h(inspired)h(b)o(y)f(Marc's)g (commen)o(ts)e(at)i(the)h(Marc)o(h)f(MPI)h(meeting,)d(though)75 1938 y(in)k(the)i(end)f(it)g(di\013ers)g(substan)o(tially)f(in)h(the)g (details.)27 b(It)17 b(is)f(quite)h(general)g(-)g(p)q(ossibly)f(to)q(o) h(general.)27 b(It)17 b(has)75 1988 y(similar)10 b(functionalit)o(y)i (to)h(previous)g(prop)q(osals)g(but)g(the)g(role)g(of)g(v)n(arious)f (names)g(is)h(more)f(clearly)g(\(?\))18 b(de\014ned.)75 2038 y(In)c(particular,)f(a)g Fb(p)q(o)o(rt)p 437 2038 13 2 v 15 w(name)h Fh(is)f(supplied)h(b)o(y)g(the)g(system)f(and)h(a)f Fb(service)p 1258 2038 V 17 w(name)g Fh(is)h(supplied)g(b)o(y)f(the)h (user,)h(but)75 2088 y(ma)o(y)c(b)q(e)i(published)f(b)o(y)g(the)i (system.)j(In)12 b(the)h(more)f(\015exible)g(previous)h(prop)q(osals,)f (a)h(single)f(name)f(pla)o(y)o(ed)h(b)q(oth)75 2138 y(roles,)i(dep)q (ending)g(on)g(the)g(con)o(text)h(and)f(the)g(implemen)o(tation.)166 2194 y(It)i(is)g(p)q(ossible)g(to)g(write)g(a)g(completely)f(p)q (ortable)h(co)q(de,)h(but)f(the)h(price)g(is)f(that)g(suc)o(h)g(a)g(co) q(de)h(ma)o(y)d(not)75 2251 y(rely)g(on)h(an)o(y)e(name)h(publishing)f (pro)o(vided)h(b)o(y)g(the)h(implemen)o(tation.)h(Co)q(des)f(that)f (rely)h(on)f(name)f(publishing)75 2307 y(are)19 b(not)f(completely)f(p) q(ortable)h(to)h(implem)o(en)o(tations)d(without)i(name)f(publishing)g (\(of)h(course\))i(but)e(should)75 2363 y(b)q(e)i(p)q(ortable)g(among)d (publishing)h(implemen)o(tations.)32 b(Moreo)o(v)o(er,)21 b(it)e(is)g(easy)h(to)f(o)o(v)o(erride)h(the)g(publishing)75 2420 y(mec)o(hanism)12 b(if)h(desired,)h(and)g(it)g(is)g(fairly)e(easy) j(to)f(write)g(p)q(ortable)g(co)q(de)h(that)f(requires)h(only)e(a)h (couple)g(of)f(line)75 2476 y(c)o(hanges)i(to)e(tak)o(e)h(adv)n(an)o (tage)f(of)h(a)f(name)g(publishing)g(implem)o(en)o(tation.)166 2616 y Fu(This)22 b(section)f(pro)o(vides)h(the)f(functions)h(needed)g (to)e(solv)o(e)i(the)f(general)g(case)g(of)g(creating)g(an)75 2672 y(in)o(tercomm)o(unicator)c(b)q(et)o(w)o(een)g(t)o(w)o(o)e(sets)h (of)h(MPI)f(pro)q(cesses)h(that)f(do)h(not)f(share)h(an)f(existing)i (com-)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 26 27 bop 75 -100 a Fu(26)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fu(m)o(unicator.)20 b(This)c(situation)f(arises)h(in)g(sev)o(eral)f(con)o(texts:)131 143 y(1.)22 b(Tw)o(o)14 b(parts)g(of)h(an)g(application)i(that)e(are)g (started)f(indep)q(enden)o(tly)k(need)e(to)f(comm)o(unicate.)131 237 y(2.)22 b(A)15 b(visualization)i(to)q(ol)e(w)o(an)o(ts)f(to)h (attac)o(h)f(to)g(a)h(running)i(pro)q(cess.)131 331 y(3.)22 b(A)16 b(serv)o(er)f(ma)o(y)g(w)o(an)o(t)g(to)g(accept)h(connections)h (from)f(a)f(m)o(ultiple)j(clien)o(ts.)23 b(This)16 b(situation)h(can) 189 387 y(b)q(e)f(more)e(complex)i(than)f(traditional)h(clien)o(t-serv) o(er)h(programs.)h(In)e(particular,)243 481 y Fo(\017)23 b Fu(Both)15 b(clien)o(t)h(and)g(serv)o(er)e(ma)o(y)h(b)q(e)h(parallel) g(programs.)243 554 y Fo(\017)23 b Fu(Multiple)18 b(instances)f(of)e (the)i(clien)o(t)g(ma)o(y)f(w)o(an)o(t)f(to)h(in)o(teract)g(with)g(the) g(same)g(instance)h(of)289 610 y(the)e(serv)o(er)g(at)f(the)i(same)e (time,)i(as)e(w)o(ell)j(as)d(serially)l(.)243 684 y Fo(\017)23 b Fu(Since)17 b(this)f(is)g(an)f(application)i(rather)e(than)g(a)g (system)h(service,)g(there)f(ma)o(y)g(b)q(e)h(m)o(ultiple)289 740 y(instances)g(of)e(the)i(serv)o(er)e(running)j(on)e(the)g(same)g (host.)75 834 y(In)i(eac)o(h)f(of)g(these)g(situations,)g(neither)h (set)f(of)g(pro)q(cesses)h(spa)o(wns)e(the)h(other,)g(but)g(they)h (wish)f(to)g(es-)75 890 y(tablish)d(comm)o(unication.)20 b(As)12 b(in)h(the)f(case)g(of)g Fp(MPI)p 971 890 14 2 v 16 w(SP)l(A)-5 b(WN)p Fu(,)13 b(w)o(e)f(will)h(use)g(MPI)f(in)o (tercomm)o(unicators)75 947 y(to)i(express)h(this)g(situation.)20 b(That)14 b(is,)h(w)o(e)g(will)h(de\014ne)g(MPI)f(functions)g(that)f (supp)q(ort)h(creation)g(of)f(an)75 1003 y(in)o(tercomm)o(unicator)h (in)h(whic)o(h)g(the)g(lo)q(cal)g(and)f(remote)g(groups)g(are)g(the)g (original)h(sets)f(of)g(pro)q(cesses.)166 1060 y(Establishing)21 b(con)o(tact)e(b)q(et)o(w)o(een)h(t)o(w)o(o)f(groups)g(of)h(pro)q (cesses)g(that)f(do)h(not)f(share)h(an)g(existing)75 1116 y(comm)o(unicator)14 b(is)i(a)e(collectiv)o(e,)i(but)f(asymmetric) g(pro)q(cess.)20 b(One)c(group)e(of)g(pro)q(cesses)i(indicates)g(its)75 1173 y(willingness)21 b(to)c(accept)h(connections)h(from)e(other)h (groups)f(of)h(pro)q(cesses.)28 b(W)l(e)18 b(will)i(call)f(this)f (group)75 1229 y(the)c(\(parallel\))h Fj(server)p Fu(,)f(ev)o(en)g(if)h (this)f(is)h(not)f(a)f(clien)o(t/serv)o(er)i(t)o(yp)q(e)f(of)g (application.)21 b(The)15 b(other)e(group)75 1285 y(connects)j(to)e (the)h(serv)o(er;)g(w)o(e)g(will)i(call)f(it)f(the)h Fj(client)p Fu(.)166 1342 y(In)g(the)g(rest)f(of)h(this)g(section,)g(w) o(e)g(use)g(the)g(example)g(of)f(a)h(coupled)h(o)q(cean-atmosphere)f (climate)75 1398 y(mo)q(del,)e(in)g(whic)o(h)f(the)g(o)q(cean)h(and)f (atmosphere)f(are)h(separate)f(MPI)h(programs.)18 b Fq(o)q(cean)c Fu(is)f(the)g(serv)o(er)75 1455 y(and)i Fq(atmos)h Fu(is)f(the)h(clien) o(t.)75 1577 y Fm(3.4.1)49 b(Names,)15 b(addresses,)g(p)q(o)o(rts,)g (and)h(all)i(that)75 1662 y Fu(Almost)i(all)i(of)e(the)g(complexit)o(y) h(in)g(MPI)g(clien)o(t/serv)o(er)g(routines)g(addresses)f(the)h (question)g(\\ho)o(w)75 1719 y(do)q(es)16 b(the)g(clien)o(t)i(\014nd)e (out)g(ho)o(w)f(to)h(con)o(tact)f(the)h(serv)o(er?")22 b(The)16 b(di\016cult)o(y)l(,)i(of)d(course,)h(is)h(that)e(there)75 1775 y(is)k(no)g(existing)h(comm)o(unication)f(c)o(hannel)h(b)q(et)o(w) o(een)f(them,)h(y)o(et)e(they)h(m)o(ust)f(someho)o(w)g(agree)h(on)f(a) 75 1832 y(rendezv)o(ous)e(p)q(oin)o(t)f(where)h(they)f(will)i (establish)f(comm)o(unication)g(-)g(Catc)o(h)e(22.)166 1888 y(Agreeing)19 b(on)g(a)f(rendezv)o(ous)h(p)q(oin)o(t)g(alw)o(a)o (ys)f(in)o(v)o(olv)o(es)h(a)f(third)i(part)o(y)l(.)29 b(The)19 b(third)g(part)o(y)f(ma)o(y)75 1945 y(b)q(e,)i(for)f (instance,)h(a)f(higher)g(lev)o(el)i(of)d(managemen)o(t,)h(whic)o(h)h (tells)g(them)f(b)q(oth,)g(\\thou)g(shalt)g(meet)75 2001 y(at)c(p)q(oin)o(t)i(X)f(at)f(high)h(no)q(on.")22 b(The)16 b(third)h(part)o(y)e(migh)o(t)h(b)q(e)g(a)g(go-b)q(et)o(w)o(een,)f (that)h(gets)f(a)g(rendezv)o(ous)75 2058 y(p)q(oin)o(t)i(from)f(the)h (serv)o(er)g(and)f(passes)h(it)g(to)f(the)h(clien)o(t.)26 b(Complicating)18 b(matters)e(migh)o(t)g(b)q(e)i(the)f(fact)75 2114 y(that)h(the)i(clien)o(t)g(do)q(esn't)f(really)h(care)g(what)e (serv)o(er)h(it)g(con)o(tacts,)g(only)h(that)f(it)g(b)q(e)h(able)g(to)e (get)h(in)75 2170 y(touc)o(h)c(with)h(one)f(that)f(can)i(handle)g(its)g (request.)166 2227 y(Ideally)l(,)21 b(MPI)d(can)h(accomo)q(date)f(a)g (wide)h(v)m(ariet)o(y)g(of)f(run-time)h(systems)f(while)i(retaining)f (the)75 2283 y(abilit)o(y)d(to)f(write)g(simple)i(p)q(ortable)f(co)q (de.)k(The)c(follo)o(wing)f(should)i(b)q(e)e(compatible)i(with)e(MPI:) 143 2390 y Fo(\017)23 b Fu(The)15 b(serv)o(er)g(resides)h(at)f(a)f(w)o (ell-kno)o(wn)j(in)o(ternet)e(address)g(host:p)q(ort.)143 2483 y Fo(\017)23 b Fu(The)18 b(serv)o(er)f(prin)o(ts)h(out)g(an)g (address)f(to)h(the)g(terminal,)g(the)g(user)g(giv)o(es)g(this)h (address)f(to)f(the)189 2540 y(clien)o(t)f(program.)143 2634 y Fo(\017)23 b Fu(The)14 b(serv)o(er)h(places)g(the)g(address)f (information)h(on)g(a)f(nameserv)o(er,)g(where)h(it)g(can)f(b)q(e)i (retriev)o(ed)189 2690 y(with)f(an)g(agreed-up)q(on)h(name.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 27 28 bop 75 -100 a Fn(3.4.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fu(27)143 49 y Fo(\017)23 b Fu(The)17 b(serv)o(er)g(to)g(whic)o(h)h(the)f(clien)o(t) i(connects)e(is)h(actually)g(a)f(brok)o(er,)g(acting)g(as)g(a)g (middleman)189 106 y(b)q(et)o(w)o(een)e(the)g(clien)o(t)i(and)e(the)h (real)f(serv)o(er.)166 195 y(MPI)e(is)h(able)g(to)e(supp)q(ort)i(all)g (of)e(these)i(cases)f(b)o(y)g(pro)o(viding)h(t)o(w)o(o)e(complemen)o (tary)h(handles)i(with)75 251 y(whic)o(h)h(a)f(clien)o(t)h(can)g(con)o (tact)e(a)h(serv)o(er.)143 331 y Fo(\017)23 b Fu(A)f Fp(p)q(o)o(rt)g Fu(is)g(an)g(abstraction)g(of)f(a)h(lo)o(w-lev)o(el)h (net)o(w)o(ork)e(address)h(at)f(whic)o(h)i(a)f(serv)o(er)f(can)h(b)q(e) 189 388 y(con)o(tacted.)d(T)o(ypically)d(this)f(will)i(b)q(e)e(an)f(IP) h(address)g(and)g(a)f(p)q(ort)h(n)o(um)o(b)q(er)g(but)f(an)h(implemen-) 189 444 y(tation)e(is)h(free)g(to)f(use)h(an)o(y)g(proto)q(col.)19 b(A)14 b(p)q(ort)g(is)g(represen)o(ted)g(b)o(y)g(a)f(new)h(MPI)g (opaque)g(ob)s(ject,)189 500 y Fp(MPI)p 274 500 14 2 v 15 w(P)o(o)o(rt)p Fu(.)24 b(Ev)o(ery)17 b(p)q(ort)f(has)h(a)f(name)h (that)f(can)h(b)q(e)h(obtained)f(with)g Fp(MPI)p 1511 500 V 16 w(PORT)p 1649 500 V 17 w(NAME)g Fu(\(on)189 557 y(the)e(serv)o(er\))f(and)i(can)f(b)q(e)h(used)g(b)o(y)f(a)g(clien) o(t)h(to)f(con)o(tact)f(the)h(serv)o(er.)143 645 y Fo(\017)23 b Fu(A)c Fp(service)p 370 645 V 17 w(name)g Fu(is)i(a)e(high-lev)o(el)j (name,)f(asso)q(ciated)f(with)g(a)f(p)q(ort,)h(that)f(a)h(clien)o(t)h (ma)o(y)e(use)189 701 y(to)d(con)o(tact)h(a)g(serv)o(er,)h(and)f(to)g (c)o(ho)q(ose)g(from)g(among)g(man)o(y)g(p)q(ossible)i(services)f(that) f(a)g(serv)o(er)189 758 y(migh)o(t)22 b(pro)o(vide.)43 b(There)23 b(ma)o(y)f(b)q(e)h(man)o(y)f(names)h(asso)q(ciated)g(with)g (a)f(single)i(p)q(ort.)42 b(If)23 b(an)189 814 y(MPI)15 b(implemen)o(tation)i(utilizes)h(some)d(sort)g(of)g(name)h(serv)o(er,)f (a)h(clien)o(t)g(ma)o(y)f(b)q(e)i(able)f(to)f(use)h(a)189 871 y Fp(service)p 317 871 V 16 w(name)h Fu(to)f(con)o(tact)g(the)h (serv)o(er)g(without)g(kno)o(wing)g(the)g(p)q(ort)f(name.)25 b(In)18 b(a)e(distributed)189 927 y(ob)s(ject)e(system,)h(a)g Fp(service)p 656 927 V 16 w(name)g Fu(migh)o(t)g(con)o(tain)g(the)h (name)f(of)g(an)g(ob)s(ject)f(metho)q(d.)75 1046 y Fm(3.4.2)49 b(Server)15 b(Routines)75 1132 y Fu(A)j(serv)o(er)f(mak)o(es)g(itself)i (a)o(v)m(ailable)g(with)f(t)o(w)o(o)e(routines.)28 b(First)17 b(it)h(m)o(ust)f(call)i Fp(MPI)p 1543 1132 V 16 w(PORT)p 1681 1132 V 17 w(OPEN)g Fu(to)75 1189 y(establish)14 b(a)e Fp(p)q(o)o(rt)h Fu(at)f(whic)o(h)i(it)f(ma)o(y)f(b)q(e)i(con)o (tacted.)k(Secondly)d(it)e(m)o(ust)f(call)i Fp(MPI)p 1496 1189 V 16 w(A)o(CCEPT)f Fu(to)f(accept)75 1245 y(connections)18 b(from)e(clien)o(ts.)27 b(Additionally)l(,)20 b(it)d(ma)o(y)g(asso)q (ciate)g(one)g(or)g(more)f Fp(service)p 1596 1245 V 17 w(name)p Fu(s)h(with)g(a)75 1302 y Fp(p)q(o)o(rt)p Fu(,)e(through)g Fp(MPI)p 438 1302 V 16 w(PORT)p 576 1302 V 17 w(ADD)p 689 1302 V 16 w(SERVICE)p Fu(.)75 1452 y Fp(MPI)p 160 1452 V 16 w(PORT)p 298 1452 V 17 w(OPEN\(info,)g(p)q(o)o(rt\))117 1530 y Fh(IN)155 b Fp(info)516 b Fh(implemen)o(tation-sp)q(eci\014c)12 b(information)f(on)j(ho)o(w)g(to)f(estab-)905 1586 y(lish)h(an)f (address)i(\(string\))117 1658 y(OUT)108 b Fp(p)q(o)o(rt)509 b Fh(newly)14 b(established)h(p)q(ort)f(\(handle\))75 1783 y Fl(int)23 b(MPI)p 245 1783 15 2 v 17 w(Port)p 358 1783 V 17 w(open\(char*)f(info,)i(MPI)p 853 1783 V 16 w(Port*)f(port\))75 1869 y(MPI)p 150 1869 V 17 w(PORT)p 263 1869 V 16 w(OPEN\(INFO,)g(PORT,)g(IERR\))170 1926 y(CHARACTER*\(*\))g(INFO)170 1982 y(INTEGER)g(PORT,)h(IERR)166 2069 y Fu(This)17 b(function)h(establishes)g(a)e(net)o(w)o(ork)g (address,)g(represen)o(ted)i(b)o(y)e(the)h Fp(p)q(o)o(rt)g Fu(ob)s(ject,)f(at)g(whic)o(h)75 2125 y(the)f(serv)o(er)g(will)i(b)q(e) f(able)g(to)e(accept)i(connections)g(from)e(clien)o(ts.)166 2182 y Fp(info)j Fu(is)g(an)g(implemen)o(tation-de\014ned)j(string)c (that)g(ma)o(y)h(b)q(e)g(used)h(to)e(tell)i(the)f(implemen)o(tation)75 2238 y(ho)o(w)k(to)f(establish)i(the)f(address.)38 b(It)21 b(ma)o(y)f(\(and)h(usually)i(will\))f(b)q(e)g(an)f(empt)o(y)g(string)g (\(NULL)g(or)75 2295 y(empt)o(y)15 b(string)g(in)h(C,)f Fl(' ')g Fu(in)h(F)l(ortran\))d(to)i(get)g(the)g(implemen)o(tation)i (defaults.)j(Section)c(XXX)g(giv)o(es)75 2351 y(examples)g(of)f(its)g (p)q(ossible)i(uses.)75 2502 y Fp(MPI)p 160 2502 14 2 v 16 w(PORT)p 298 2502 V 17 w(CLOSE\(p)q(o)o(rt\))117 2579 y Fh(IN)155 b Fp(p)q(o)o(rt)509 b Fh(a)14 b(p)q(ort)g(\(handle\)) 75 2704 y Fl(int)23 b(MPI)p 245 2704 15 2 v 17 w(Port)p 358 2704 V 17 w(close\(MPI)p 591 2704 V 16 w(Port)g(port\))-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 28 30 28 29 bop 75 -100 a Fu(28)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fl(MPI)p 150 49 15 2 v 17 w(PORT)p 263 49 V 16 w(CLOSE\(PORT,)23 b(IERR\))170 106 y(INTEGER)g(PORT,)h(IERR)166 192 y Fu(This)15 b(function)g (releases)g(the)f(net)o(w)o(ork)g(address)g(represen)o(ted)h(b)o(y)f Fp(p)q(o)o(rt)p Fu(.)20 b(There)15 b(ma)o(y)e(not)h(b)q(e)h(an)o(y)75 249 y(p)q(ending)i Fp(MPI)p 333 249 14 2 v 16 w(IA)o(CCEPT)e Fu(op)q(erations)g(on)g(the)h(p)q(ort.)166 388 y Fi(Discussion:)34 b Fh(Should)13 b(w)o(e)h(set)h(p)q(ort)f(to)g Fb(MPI)p 917 388 13 2 v 14 w(PORT)p 1044 388 V 14 w(NULL)p Fh(?)75 621 y Fp(MPI)p 160 621 14 2 v 16 w(PORT)p 298 621 V 17 w(NAME\(p)q(o)o(rt,)h(p)q(o)o(rt)p 646 621 V 17 w(name,)f(namelen\))117 699 y Fh(IN)155 b Fp(p)q(o)o(rt)509 b Fh(a)14 b(p)q(ort)g(\(handle\)) 117 774 y(OUT)108 b Fp(p)q(o)o(rt)p 399 774 V 17 w(name)390 b Fh(name)13 b(whic)o(h)h(ma)o(y)e(b)q(e)i(used)h(to)f(con)o(tact)g (serv)o(er)h(\(string\))117 849 y(IN)155 b Fp(namelen)431 b Fh(length)14 b(of)f Fb(p)q(o)o(rt)p 1152 849 13 2 v 16 w(name)g Fh(string)h(\(in)o(teger\))75 973 y Fl(int)23 b(MPI)p 245 973 15 2 v 17 w(Port)p 358 973 V 17 w(name\(MPI)p 567 973 V 16 w(Port*)g(port,)g(char*)g(port)p 1108 973 V 17 w(name,)g(int)h(namelen\))75 1060 y(MPI)p 150 1060 V 17 w(PORT)p 263 1060 V 16 w(NAME\(PORT,)f(PORT)p 638 1060 V 17 w(NAME,)g(NAMELEN,)f(IERR\))170 1116 y(INTEGER)h(PORT,)h (NAMELEN,)e(IERR)170 1173 y(CHARACTER*\(*\))h(PORT)p 604 1173 V 16 w(NAME)166 1259 y Fp(p)q(o)o(rt)p 247 1259 14 2 v 16 w(name)12 b Fu(is)g(a)f(string)g(that)g(iden)o(ti\014es)i (the)f(giv)o(en)g(p)q(ort)f(and)h(can)f(b)q(e)h(used)h(b)o(y)e(a)g (clien)o(t)i(to)e(con)o(tact)75 1316 y(the)21 b(serv)o(er.)38 b(It)22 b(is)g(essen)o(tially)g(a)f(net)o(w)o(ork)g(address.)38 b(It)22 b(is)f(unique)i(within)g(the)e(comm)o(unication)75 1372 y(univ)o(erse)15 b(to)f(whic)o(h)i(is)f(b)q(elongs)g(\(determined) h(b)o(y)e(the)h(implemen)o(tation\),)g(and)g(ma)o(y)f(b)q(e)h(used)g(b) o(y)f(an)o(y)75 1428 y(clien)o(t)22 b(within)f(that)e(comm)o(unication) i(univ)o(erse.)36 b(F)l(or)20 b(instance,)i(if)f(it)f(is)h(an)f(in)o (ternet)h(\(host:p)q(ort\))75 1485 y(address,)14 b(it)g(will)i(b)q(e)f (unique)h(on)e(the)g(in)o(ternet.)20 b(If)14 b(it)h(is)g(a)e(lo)o(w)h (lev)o(el)i(switc)o(h)e(address)h(on)f(an)g(IBM)g(SP2)75 1541 y(it)j(will)i(b)q(e)e(unique)i(to)d(that)g(SP2.)25 b(The)17 b(precise)h(form)e(of)g(the)h(address)g(is)g(implemen)o (tation-de\014ned.)75 1598 y(F)l(or)f(instance,)i(an)e(in)o(ternet)h (address)g(ma)o(y)f(b)q(e)h(a)g(host)f(name)g(or)h(ip)g(address,)g(or)f (an)o(ything)h(that)f(the)75 1654 y(implemen)o(tation)g(can)g(deco)q (de)g(in)o(to)f(an)g(ip)h(address.)189 1761 y Fj(A)n(dvic)n(e)f(to)j (implementors.)43 b Fu(Since)17 b(the)f(user)g(ma)o(y)f(t)o(yp)q(e)h (in)h Fp(p)q(o)o(rt)p 1346 1761 V 16 w(name)f Fu(b)o(y)f(hand,)h(it)h (is)f(useful)189 1817 y(to)e(c)o(ho)q(ose)h(a)g(form)g(that)f(is)i (easily)g(readable)g(and)f(do)q(es)h(not)f(ha)o(v)o(e)g(em)o(b)q(edded) h(spaces.)189 1892 y(The)e(only)h(requiremen)o(t)g(on)g Fp(p)q(o)o(rt)p 776 1892 V 16 w(name)f Fu(is)h(that)f(it)g(uniquely)j (iden)o(tify)e(an)g(MPI)f(p)q(ort)g(from)g(the)189 1949 y(user's)i(p)q(oin)o(t)i(of)e(view.)25 b(This)18 b(do)q(es)f(not)f (rule)i(out)f(fancy)f(implemen)o(tations.)27 b(F)l(or)16 b(instance,)h(if)189 2005 y(there)11 b(is)h(a)g(brok)o(er)f (\(middleman\))h(b)q(et)o(w)o(een)g(the)g(clien)o(t)h(and)e(the)h(serv) o(er,)g Fp(p)q(o)o(rt)p 1530 2005 V 16 w(name)f Fu(could)i(actu-)189 2062 y(ally)e(con)o(tain)g(information)f(ab)q(out)h(ho)o(w)f(to)g(con)o (tact)f(the)i(brok)o(er,)f(along)h(with)g(other)f(information)189 2118 y(that)k(uniquely)j(iden)o(ti\014es)g(the)f(serv)o(er)e(to)h(the)g (brok)o(er.)20 b(\()p Fj(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fu(\))75 2319 y Fp(MPI)p 160 2319 V 16 w(PORT)p 298 2319 V 17 w(ADD)p 411 2319 V 16 w(SERVICE\(p)q(o)o(rt,)g(service)p 857 2319 V 17 w(name\))117 2396 y Fh(IN)155 b Fp(p)q(o)o(rt)509 b Fh(a)14 b(p)q(ort)g(\(handle\))117 2471 y(IN)155 b Fp(service)p 446 2471 V 17 w(name)343 b Fh(a)14 b(service)h(name)e(to)h (asso)q(ciate)g(with)g(the)g(p)q(ort)g(\(string\))75 2596 y Fl(int)23 b(MPI)p 245 2596 15 2 v 17 w(Port)p 358 2596 V 17 w(add)p 447 2596 V 17 w(service\(MPI)p 728 2596 V 15 w(Port)h(port,)f(char*)g(service)p 1317 2596 V 16 w(name\))75 2682 y(MPI)p 150 2682 V 17 w(PORT)p 263 2682 V 16 w(ADD)p 351 2682 V 17 w(SERVICE\(PORT,)f(SERVICE)p 870 2682 V 16 w(NAME,)i(IERR\))1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 29 31 29 30 bop 75 -100 a Fn(3.4.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fu(29)170 49 y Fl(INTEGER)23 b(PORT,)h(IERR)170 106 y(CHARACTER*\(*\))f(SERVICE)p 676 106 15 2 v 16 w(NAME)166 192 y Fu(This)14 b(routine)f(asso)q (ciates)g(a)g Fp(service)p 793 192 14 2 v 17 w(name)g Fu(with)g(with)h(the)f(giv)o(en)h Fp(p)q(o)o(rt)p Fu(.)19 b(After)13 b(a)g(name)g(is)h(added,)75 249 y(clien)o(ts)19 b(requesting)g(to)e(connect)i(to)f(the)g(giv)o(en)g(p)q(ort)g(and)g (service)h(name)f(will)i(blo)q(c)o(k)f(un)o(til)g(a)f(corre-)75 305 y(sp)q(onding)e Fp(MPI)p 353 305 V 16 w(A)o(CCEPT)g Fu(\(with)f(the)g(same)g(p)q(ort)g(and)g(service)h(name\))f(is)h (called)h(on)e(the)g(serv)o(er.)166 362 y(The)i(empt)o(y)f(service)h (\(NULL)g(or)f(empt)o(y)h(string)f(in)i(C,)d Fl(' ')i Fu(in)g(F)l(ortran\))e(is)i(alw)o(a)o(ys)f(asso)q(ciated)75 418 y(with)g(a)e(p)q(ort.)166 474 y(An)e(MPI)g(implemen)o(tation)h(is)f (allo)o(w)o(ed,)h(but)f(not)f(required,)j(to)d Fj(publish)h Fp(service)p 1547 474 V 16 w(name)g Fu(b)o(y)g(placing)75 531 y(\(p)q(ort)i(name,)g(service)h(name\))f(in)h(a)f(database)g (\(e.g.,)e(a)i(nameserv)o(er\))g(so)g(that)f(the)i(clien)o(t)g(can)g (retriev)o(e)75 587 y(the)21 b(p)q(ort)g(name)g(from)g(the)g(service)h (name.)38 b(The)21 b(implemen)o(tation)i(m)o(ust)e(de\014ne)h(the)f Fj(sc)n(op)n(e)g Fu(of)f(a)75 644 y(published)i(service)e(name,)g(that) f(is,)i(the)e(domain)h(o)o(v)o(er)f(whic)o(h)h(the)f(service)i(name)e (is)h(unique,)i(and)75 700 y(con)o(v)o(ersely)l(,)e(the)f(domain)g(o)o (v)o(er)f(whic)o(h)i(the)e(\(p)q(ort)h(name,)g(service)h(name\))e(pair) h(ma)o(y)f(b)q(e)i(retriev)o(ed.)75 757 y(F)l(or)14 b(instance,)i(a)e (service)i(name)f(ma)o(y)g(b)q(e)g(unique)i(to)d(a)h(job)f(\(where)h (job)g(is)h(de\014ned)g(b)o(y)f(a)f(distributed)75 813 y(op)q(erating)i(system)f(or)f(batc)o(h)i(sc)o(heduler\),)g(unique)h (to)d(a)h(mac)o(hine,)h(or)f(unique)i(to)e(a)g(k)o(erb)q(eros)g(realm.) 75 870 y(The)e(case)f(where)g(an)h(MPI)f(implemen)o(tation)i(do)q(es)e (not)g(publish)j Fp(service)p 1333 870 V 16 w(name)d Fu(is)h(just)f(the)g(case)h(where)75 926 y Fp(service)p 203 926 V 17 w(name)18 b Fu(is)i(unique)g(only)g(to)e(the)h(MPI)g(pro)q (cess)g(on)g(whic)o(h)h Fp(MPI)p 1339 926 V 16 w(PORT)p 1477 926 V 17 w(ADD)p 1590 926 V 16 w(SERVICE)g Fu(w)o(as)75 983 y(called.)h(See)16 b Fp(MPI)p 387 983 V 16 w(CONNECT)g Fu(and)g(examples.)166 1039 y(MPI)k(pro)o(vides)g(a)f(sym)o(b)q(ol)h Fp(MPI)p 746 1039 V 16 w(NAMES)p 916 1039 V 17 w(ARE)p 1019 1039 V 17 w(PUBLISHED)p Fu(.)g(It)f(has)h(a)f(v)m(alue)i(of)e(0)h (\(zero\))e(if)75 1095 y(names)g(are)g(not)g(published)j(and)d(a)g (nonzero)h(v)m(alue)g(of)f(they)g(are.)29 b(In)19 b(C,)f(it)g(is)h(a)f (macro)f(de\014ned)j(in)75 1152 y Fl(mpi.h)p Fu(.)k(In)18 b(F)l(ortran,)d(it)j(is)f(an)g(in)o(teger)g(parameter)f(de\014ned)i(in) g Fl(mpif.h)p Fu(.)24 b(This)17 b(allo)o(ws)g(applications)75 1208 y(to)e(mak)o(e)f(run-time)i(or)f(compile-time)i(decisions)g(ab)q (out)e(what)g(strategy)f(to)g(use.)166 1265 y(If)d Fp(service)p 335 1265 V 17 w(name)f Fu(has)g(already)h(b)q(een)h(established)g (within)g(the)f(curren)o(t)g(scop)q(e,)g(the)g(default)g(b)q(eha)o(v-) 75 1321 y(ior)i(is)g(that)f Fp(MPI)p 365 1321 V 16 w(ADD)p 477 1321 V 16 w(SERVICE)i Fu(fails)f(and)g(returns)f(error)g(class)i Fp(MPI)p 1315 1321 V 15 w(ERR)p 1415 1321 V 18 w(SERVICE)p Fu(.)f(An)g(MPI)f(im-)75 1378 y(plemen)o(tation)f(ma)o(y)l(,)f(through) g(an)g(o)o(v)o(erride)g(mec)o(hanism)h(in)g(the)f Fp(info)g Fu(argumen)o(t)g(to)f Fp(MPI)p 1605 1378 V 16 w(OPEN)p 1742 1378 V 17 w(PORT)p Fu(,)75 1434 y(pro)o(vide)17 b(a)e(w)o(a)o(y)h(to)f(allo)o(w)h(m)o(ultiple)i(serv)o(ers)e(with)g (the)h(same)e(service)i(in)g(the)g(same)e(name)h(space.)23 b(In)75 1491 y(this)e(case,)h(an)e(implemen)o(tation-de\014ned)k(p)q (olicy)e(will)h(determine)e(to)f(whic)o(h)i(of)e(sev)o(eral)h(serv)o (ers)f(a)75 1547 y(clien)o(t)c(is)g(connected)g(\(see)f(examples\).)166 1604 y(Note)h(that)g(while)i Fp(service)p 625 1604 V 16 w(name)f Fu(has)f(a)g(limited)i(scop)q(e,)f(determined)h(b)o(y)e (the)h(implemen)o(tation,)75 1660 y Fp(p)q(o)o(rt)p 156 1660 V 17 w(name)f Fu(alw)o(a)o(ys)h(has)g(global)g(scop)q(e)h(within)g (the)f(comm)o(unication)h(univ)o(erse)g(used)g(b)o(y)f(the)g(imple-)75 1716 y(men)o(tation)e(\(i.e.,)f(it)i(is)g(globally)g(unique\).)189 1823 y Fj(A)n(dvic)n(e)e(to)h(implementors.)39 b Fu(In)14 b(some)g(cases,)g(an)f(MPI)h(implemen)o(tation)h(ma)o(y)e(use)h(a)g (nameser-)189 1879 y(vice)d(that)f(a)h(user)g(can)f(also)h(access)g (directly)l(.)20 b(The)11 b(user)g(ma)o(y)l(,)f(for)g(instance,)i(use)f (the)g(nameserv)o(er)189 1936 y(to)16 b(publish)j(p)q(ort)d(names)h(in) g(order)g(to)f(ha)o(v)o(e)g(completely)j(p)q(ortable)e(MPI)g(co)q(de.) 25 b(In)17 b(this)g(case,)189 1992 y(a)e(name)g(published)j(b)o(y)d (MPI)g(could)i(easily)f(con\015ict)g(with)g(the)f(same)g(name)h (published)i(b)o(y)d(the)189 2049 y(user.)31 b(In)19 b(order)g(to)f(a)o(v)o(oid)h(suc)o(h)g(con\015icts,)h(MPI)f(implemen)o (tations)h(should)g(mangle)f(service)189 2105 y(names)14 b(so)f(that)h(they)g(are)g(unlik)o(ely)i(to)e(con\015ict)h(with)f(user) g(co)q(de)h(that)e(mak)o(es)h(use)g(of)g(the)g(same)189 2161 y(service.)32 b(Suc)o(h)20 b(name)f(mangling)g(will)i(of)e(course) g(b)q(e)g(completely)i(transparen)o(t)d(to)g(the)h(user.)189 2218 y(\()p Fj(End)c(of)i(advic)n(e)f(to)g(implementors.)p Fu(\))166 2400 y Fi(Discussion:)54 b Fh(The)19 b(follo)o(wing)d (situation)i(is)h(problematic)e(but)i(una)o(v)o(oidable,)f(if)g(w)o(e)i (w)o(an)o(t)e(to)h(allo)o(w)75 2450 y(implemen)o(tatio)o(ns)14 b(to)i(use)h(nameserv)o(ers.)26 b(Supp)q(ose)17 b(there)g(are)g(m)o (ultiple)d(instances)j(of)f(\\o)q(cean")g(running)g(on)75 2500 y(a)g(mac)o(hine.)22 b(If)16 b(the)g(scop)q(e)h(of)f(a)f(service)i (name)e(is)h(con\014ned)h(to)e(a)h(job,)f(then)i(m)o(ultiple)c(o)q (ceans)k(can)f(co)q(eexist.)75 2550 y(If)g(an)h(implemen)o(tatio)o(n)d (pro)o(vides)j(site-wide)g(scop)q(e,)h(ho)o(w)o(ev)o(er,)f(m)o(ultiple) e(instances)j(are)f(not)f(p)q(ossible)h(as)g(all)75 2599 y(calls)g(to)g Fb(MPI)p 302 2599 13 2 v 15 w(ADD)p 405 2599 V 14 w(SERVICE)f Fh(after)i(the)g(\014rst)g(will)e(fail.)27 b(There)19 b(is)e(no)g(go)q(o)q(d)g(solution)g(to)g(this.)29 b(W)m(e)17 b(could)75 2649 y(sp)q(ecify)d(that)f(the)h(second)g(serv)o (er)h(b)q(ecomes)f(kno)o(wn)f(as)g(\\o)q(cean2",)g(the)h(third)f(as)g (\\o)q(cean3",)g(etc.)19 b(In)13 b(this)g(case,)75 2699 y(the)h(information)d(\\2")h(and)i(\\3")e(m)o(ust)h(b)q(e)h(comm)o (unicated)d(\\b)o(y)i(hand")g(from)f(serv)o(er)j(to)e(clien)o(t,)g(and) g(w)o(e)h(migh)o(t)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 30 32 30 31 bop 75 -100 a Fu(30)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fh(as)j(w)o(ell)f(disp)q(ense)i (with)e Fb(service)p 600 49 13 2 v 17 w(name)g Fh(altogether)h(and)f (go)h(with)f(only)g(the)h(p)q(ort)g(name)f(\(net)o(w)o(ork)h(address\)) 75 99 y(instead.)g(If)12 b(there)h(are)g(m)o(ultiple)c(instances)14 b(of)d(a)h(serv)o(er)i(that)e(w)o(ould)f(lik)o(e)h(to)g(pro)o(vide)g (sev)o(eral)g(di\013eren)o(t)i(service)75 149 y(names)h(as)h(con)o (tact)h(p)q(oin)o(ts,)f(w)o(e)g(ha)o(v)o(e)f(a)h(bigger)g(problem)e (since)j(w)o(e)f(can't)g(get)g(rid)g(of)g Fb(service)p 1608 149 V 16 w(name)g Fh(en)o(tirely)m(.)75 199 y(T)m(o)d(handle)h (these)h(situations,)e(a)g(high)h(qualit)o(y)e(implemen)o(tatio)o(n)f (should)j(mak)o(e)e(it)i(p)q(ossible)g(to)f(turn)h(o\013)g(name)75 249 y(publishing)f(or)h(limit)d(the)j(domain)e(o)o(v)o(er)i(whic)o(h)g (names)f(are)h(published.)166 305 y(Also:)k(W)m(ould)12 b(an)i Fb(info)f Fh(argumen)o(t)g(b)q(e)i(useful?)75 539 y Fp(MPI)p 160 539 14 2 v 16 w(PORT)p 298 539 V 17 w(DELETE)p 485 539 V 16 w(SERVICE\(p)q(o)o(rt,)h(service)p 931 539 V 16 w(name\))117 616 y Fh(IN)155 b Fp(p)q(o)o(rt)509 b Fh(a)14 b(p)q(ort)g(\(handle\))117 691 y(IN)155 b Fp(service)p 446 691 V 17 w(name)343 b Fh(a)14 b(service)h(name)e(to)h(asso)q(ciate) g(with)g(the)g(p)q(ort)g(\(string\))75 815 y Fl(int)23 b(MPI)p 245 815 15 2 v 17 w(Port)p 358 815 V 17 w(delete)p 519 815 V 16 w(service\(MPI)p 799 815 V 16 w(Port)g(port,)g(char*)h (service)p 1389 815 V 16 w(name\))75 902 y(MPI)p 150 902 V 17 w(PORT)p 263 902 V 16 w(DELETE)p 423 902 V 17 w(service\(PORT,)e(SERVICE)p 942 902 V 16 w(NAME,)h(IERR\))170 958 y(INTEGER)g(PORT,)h(IERR)170 1015 y(CHARACTER*\(*\))f(SERVICE)p 676 1015 V 16 w(NAME)166 1101 y Fu(This)16 b(routine)f(deletes)i(a)e (service)h(name)f(previous)h(asso)q(ciated)f(with)h(a)f(p)q(ort)f (through)75 1158 y Fp(MPI)p 160 1158 14 2 v 16 w(PORT)p 298 1158 V 17 w(ADD)p 411 1158 V 16 w(SERVICE)p Fu(.)19 b(It)f(is)h(not)f(p)q(ossible)i(to)e(delete)h(the)g(empt)o(y)f (service.)30 b(A)o(ttempting)18 b(to)75 1214 y(delete)h(the)f(empt)o(y) f(service)i(or)e(a)h(service)h(that)e(do)q(es)h(not)f(exist)h(results)h (in)f(an)g(error,)f(indicated)j(b)o(y)75 1271 y(the)15 b(error)g(co)q(de)h Fp(MPI)p 455 1271 V 15 w(ERR)p 555 1271 V 18 w(SERVICE)p Fu(.)166 1327 y(If)i(there)g(is)g(a)g(p)q(ending) h Fp(MPI)p 682 1327 V 16 w(CONNECT)g Fu(w)o(aiting)f(for)f(a)g(corresp) q(onding)i Fp(MPI)p 1582 1327 V 16 w(A)o(CCEPT)f Fu(with)75 1383 y(that)e(service,)i(the)f(connect)h(will)g(fail.)26 b(It)17 b(is)h(erroneous)f(to)f(remo)o(v)o(e)g(a)h(service)h(for)e (whic)o(h)i(there)f(is)g(a)75 1440 y(p)q(ending)e Fp(MPI)p 331 1440 V 16 w(IA)o(CCEPT)p Fu(.)e(It)g(is)h(not)f(necessary)l(,)h(ho) o(w)o(ev)o(er,)e(to)h(delete)i(a)e(service)h(name)f(b)q(efore)h (closing)75 1496 y(a)h(p)q(ort.)75 1647 y Fp(MPI)p 160 1647 V 16 w(A)o(CCEPT\(p)q(o)o(rt,)g(service)p 599 1647 V 16 w(name,)g(info,)g(ro)q(ot,)f(comm,)f(new)o(comm\))117 1725 y Fh(IN)155 b Fp(p)q(o)o(rt)509 b Fh(p)q(ort)14 b(ob)r(ject)h(\(handle,)f(used)g(only)f(on)h Fb(ro)q(ot)p Fh(\))117 1800 y(IN)155 b Fp(service)p 446 1800 V 17 w(name)343 b Fh(user-supplied)22 b(name)d(of)i(service)h(\(string,)g (used)f(only)f(on)905 1856 y Fb(ro)q(ot)p Fh(\))117 1931 y(IN)155 b Fp(info)516 b Fh(implemen)o(tation-dep)q(enden)o(t)19 b(information)e(\(string,)22 b(used)905 1988 y(only)13 b(on)h Fb(ro)q(ot)p Fh(\))117 2063 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)14 b(in)f Fb(comm)f Fh(of)i(ro)q(ot)g(no)q(de)g(\(in)o (teger\))117 2138 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o(h)g(call)f(is)h(collectiv)o(e)g(\(string\))117 2213 y(OUT)108 b Fp(new)o(comm)397 b Fh(in)o(tercomm)o(unicator)12 b(with)i(clien)o(t)h(as)g(remote)f(group)g(\(han-)905 2270 y(dle\))75 2394 y Fl(int)23 b(MPI)p 245 2394 15 2 v 17 w(Accept\(MPI)p 502 2394 V 16 w(Port)g(port,)h(char*)f(service)p 1092 2394 V 16 w(name,)g(int)h(root,)f(MPI)p 1562 2394 V 17 w(Comm)g(comm,)393 2450 y(MPI)p 468 2450 V 17 w(Comm)g(newcomm\)) 75 2537 y(MPI)p 150 2537 V 17 w(ACCEPT\(PORT,)f(SERVICE)p 645 2537 V 16 w(NAME,)h(COMM,)h(NEWCOMM,)e(IERR\))170 2593 y(CHARACTER*\(*\))h(SERVICE)p 676 2593 V 16 w(NAME)170 2650 y(INTEGER)g(PORT,)h(COMM,)f(NEWCOMM,)f(IERR)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 31 33 31 32 bop 75 -100 a Fn(3.4.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fu(31)166 49 y Fp(MPI)p 251 49 14 2 v 16 w(A)o(CCEPT)k Fu(establishes)i(comm)o (unication)f(with)g(a)f(clien)o(t.)21 b(It)15 b(is)h(collectiv)o(e)h(o) o(v)o(er)e(the)g(calling)75 106 y(comm)o(unicator.)k(It)12 b(returns)g(an)h(in)o(tercomm)o(unicator)f(that)g(allo)o(ws)h(comm)o (unication)g(with)g(the)f(clien)o(t.)166 162 y(If)g Fp(service)p 336 162 V 17 w(name)g Fu(is)g(non-NULL,)h(it)g(m)o(ust)e(ha)o(v)o(e)h (b)q(een)h(asso)q(ciated)f(with)h Fp(p)q(o)o(rt)f Fu(through)f(a)h (previous)75 219 y(call)i(to)e Fp(MPI)p 294 219 V 15 w(PORT)p 431 219 V 18 w(ADD)p 545 219 V 16 w(SERVICE)p Fu(.)h(The)g Fp(service)p 983 219 V 16 w(name)g Fu(ma)o(y)e(b)q(e)j (empt)o(y)l(,)e(in)i(whic)o(h)f(case)g(the)f(serv)o(er)75 275 y(accepts)g(requests)g(for)f(an)o(y)h(service)h(name)f(that)f(has)h (b)q(een)h(asso)q(ciated)f(with)g Fp(p)q(o)o(rt)g Fu(\()p Fj(not)g Fu(just)f(the)h(empt)o(y)75 332 y(service\).)166 388 y Fp(info)h Fu(is)h(a)g(string)f(that)g(con)o(tains)g(implemen)o (tation-sp)q(eci\014)q(c)j(information.)k(See)14 b(section)g(XXX)g(for) 75 444 y(details.)75 566 y Fm(3.4.3)49 b(Client)17 b(Routines)75 652 y Fu(There)e(is)h(only)g(one)f(routine)h(on)f(the)g(clien)o(t)i (side.)75 803 y Fp(MPI)p 160 803 V 16 w(CONNECT\(p)q(o)o(rt)p 485 803 V 17 w(name,)d(service)p 756 803 V 17 w(name,)h(info,)f(ro)q (ot,)h(comm,)e(new)o(comm\))117 880 y Fh(IN)155 b Fp(p)q(o)o(rt)p 399 880 V 17 w(name)390 b Fh(net)o(w)o(ork)14 b(address)h(\(string,)f (used)h(only)e(on)h Fb(ro)q(ot)p Fh(\))117 955 y(IN)155 b Fp(service)p 446 955 V 17 w(name)343 b Fh(user-supplied)22 b(name)d(of)i(service)h(\(string,)g(used)f(only)f(on)905 1012 y Fb(ro)q(ot)p Fh(\))117 1087 y(IN)155 b Fp(info)516 b Fh(implemen)o(tation-dep)q(enden)o(t)19 b(information)e(\(string,)22 b(used)905 1143 y(only)13 b(on)h Fb(ro)q(ot)p Fh(\))117 1218 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)14 b(in)f Fb(comm)f Fh(of)i(ro)q(ot)g(no)q(de)g(\(in)o(teger\))117 1294 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o(h)g(call)f (is)h(collectiv)o(e)g(\(string\))117 1369 y(OUT)108 b Fp(new)o(comm)397 b Fh(in)o(tercomm)o(unicator)11 b(with)i(serv)o(er)h (as)f(remote)g(group)g(\(han-)905 1425 y(dle\))75 1550 y Fl(int)23 b(MPI)p 245 1550 15 2 v 17 w(Connect\(char*)f(port)p 692 1550 V 17 w(name,)h(char*)g(service)p 1163 1550 V 17 w(name,)g(int)g(root,)393 1606 y(MPI)p 468 1606 V 17 w(Comm)g(comm,)g(MPI)p 819 1606 V 17 w(Comm)h(newcomm\))75 1693 y(MPI)p 150 1693 V 17 w(CONNECT\(PORT)p 455 1693 V 15 w(NAME,)f(SERVICE)p 781 1693 V 17 w(NAME,)g(COMM,)g(NEWCOMM,)g (IERR\))170 1749 y(CHARACTER*\(*\))g(PORT)p 604 1749 V 16 w(NAME,)g(SERVICE)p 931 1749 V 17 w(NAME)170 1805 y(INTEGER)g(COMM,)h(NEWCOMM,)e(IERR)166 1892 y Fu(This)12 b(routine)f(establishes)i(comm)o(unication)f(with)f(a)g(serv)o(er)g(sp) q(eci\014ed)i(b)o(y)e Fp(p)q(o)o(rt)p 1533 1892 14 2 v 16 w(name)g Fu(and/or)g Fp(ser-)75 1948 y(vice)p 150 1948 V 17 w(name)p Fu(.)19 b(It)13 b(is)i(collectiv)o(e)g(o)o(v)o(er)e (the)h(calling)i(comm)o(unicator)d(and)i(returns)e(an)h(in)o(tercomm)o (unicator)75 2005 y(in)i(whic)o(h)g(the)f(remote)g(group)g (participated)h(in)g(an)f Fp(MPI)p 1073 2005 V 16 w(A)o(CCEPT)p Fu(.)166 2061 y Fp(p)q(o)o(rt)p 247 2061 V 16 w(name)21 b Fu(is)h(the)f(address)g(of)f(the)i(serv)o(er.)37 b(If)21 b(it)g(is)h(non-empt)o(y)l(,)g(it)f(m)o(ust)g(b)q(e)h(the)f(same)f(as) 75 2118 y(the)h(name)h(returned)f(b)o(y)h Fp(MPI)p 632 2118 V 15 w(PORT)p 769 2118 V 18 w(NAME)f Fu(on)g(the)h(serv)o(er.)38 b(A)21 b(small)h(amoun)o(t)e(of)h(freedom)g(is)75 2174 y(allo)o(w)o(ed)e(here.)32 b(If)20 b(there)f(are)g(equiv)m(alen)o(t)h (forms)f(of)f Fp(p)q(o)o(rt)p 1103 2174 V 17 w(name)p Fu(,)h(an)g(implemen)o(tation)h(ma)o(y)f(accept)75 2231 y(them)14 b(as)f(w)o(ell.)21 b(F)l(or)13 b(instance,)i(if)f Fp(p)q(o)o(rt)p 744 2231 V 17 w(name)f Fu(is)h(\(hostname:p)q(ort\),)f (an)g(implemen)o(tation)j(ma)o(y)d(accept)75 2287 y(\(ip)p 133 2287 V 17 w(address:p)q(ort\))h(as)h(w)o(ell.)21 b(The)15 b(meaning)h(of)f(an)g(empt)o(y)g(string)g(is)h(describ)q(ed)h (b)q(elo)o(w.)166 2344 y Fp(service)p 294 2344 V 17 w(name)f Fu(is)i(the)f(name)g(of)g(a)f(service.)27 b(It)17 b(m)o(ust)f(ha)o(v)o (e)h(b)q(een)h(registered)g(b)o(y)f(a)g(serv)o(er)f(using)75 2400 y Fp(MPI)p 160 2400 V 16 w(PORT)p 298 2400 V 17 w(ADD)p 411 2400 V 16 w(SERVICE)p Fu(.)d(if)g Fp(p)q(o)o(rt)p 750 2400 V 16 w(name)f Fu(is)h(non-empt)o(y)l(,)g Fp(service)p 1282 2400 V 17 w(name)f Fu(ma)o(y)f(b)q(e)i(empt)o(y)l(,)g(in)g(whic)o (h)75 2456 y(case)i(it)h(is)f(the)h(\\empt)o(y)e(service")i(asso)q (ciated)g(with)f(an)o(y)g(p)q(ort.)166 2513 y(F)l(or)f(a)g(completely)i (p)q(ortable)g(program,)d Fp(p)q(o)o(rt)p 968 2513 V 17 w(name)h Fu(m)o(ust)g(b)q(e)h(non-empt)o(y)g(\(and)g(the)g(user)g(m) o(ust)75 2569 y(\014nd)g(some)f(means)g(of)g(transferring)f Fp(p)q(o)o(rt)p 793 2569 V 17 w(name)h Fu(from)f(serv)o(er)h(to)f (clien)o(t\).)21 b(Ho)o(w)o(ev)o(er,)13 b(an)h(implemen)o(ta-)75 2626 y(tion)h(in)h(whic)o(h)g Fp(service)p 479 2626 V 17 w(name)p Fu(s)e(are)h(published)i(\(see)e Fp(MPI)p 1083 2626 V 16 w(ADD)p 1195 2626 V 16 w(SERVICE)p Fu(\))h(ma)o(y)e (supp)q(ort)h(an)g(empt)o(y)75 2682 y Fp(p)q(o)o(rt)p 156 2682 V 17 w(name)f Fu(if)i(it)f(can)h(determine)g Fp(p)q(o)o(rt)p 750 2682 V 17 w(name)e Fu(from)h Fp(service)p 1116 2682 V 17 w(name)p Fu(.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 32 34 32 33 bop 75 -100 a Fu(32)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 49 y Fu(The)21 b(p)q(ossible)h(com)o (binations)f(of)g(empt)o(y)f(and)h(non-empt)o(y)f(names)h(\(and)f (their)i(meanings\))e(is)75 106 y(summarized)c(in)g(the)f(follo)o(wing) h(list.)143 212 y Fo(\017)23 b Fu(non-empt)o(y)15 b Fp(p)q(o)o(rt)p 497 212 14 2 v 16 w(name)g Fu(non-empt)o(y)g Fp(service)p 982 212 V 17 w(name)189 268 y Fu(This)k(is)h(completely)h(p)q(ortable.) 32 b(The)20 b(clien)o(t)g(attac)o(hes)e(to)h(the)g(unique)i(serv)o(er)e (iden)o(ti\014ed)i(b)o(y)189 325 y Fp(p)q(o)o(rt)p 270 325 V 16 w(name)p Fu(.)d(The)10 b(serv)o(er)g(m)o(ust)g(ha)o(v)o(e)g (added)g Fp(service)p 1095 325 V 17 w(name)p Fu(.)18 b(No)10 b(name)g(publishing)j(capabilit)o(y)e(is)189 381 y(required)i(of)g(the)g(implemen)o(tation.)20 b(If)13 b(the)g(implemen)o(tation)h(do)q(es)f(publish)i(names,)e Fp(p)q(o)o(rt)p 1760 381 V 16 w(name)189 438 y Fu(o)o(v)o(errides)i (the)g(net)o(w)o(ork)f(address)h(obtained)h(in)o(ternally)l(.)143 532 y Fo(\017)23 b Fu(non-empt)o(y)15 b Fp(p)q(o)o(rt)p 497 532 V 16 w(name)g Fu(empt)o(y)g Fp(service)p 894 532 V 17 w(name)189 588 y Fu(This)k(is)h(completely)h(p)q(ortable.)32 b(The)20 b(clien)o(t)g(attac)o(hes)e(to)h(the)g(unique)i(serv)o(er)e (iden)o(ti\014ed)i(b)o(y)189 645 y Fp(p)q(o)o(rt)p 270 645 V 16 w(name)p Fu(.)36 b(The)21 b(serv)o(er)f(need)i(not)e(ha)o(v)o (e)g(registered)h(an)o(y)g(service)g(names,)h(and)f(m)o(ust)f(call)189 701 y Fp(MPI)p 274 701 V 15 w(A)o(CCEPT)c Fu(with)f(an)h(empt)o(y)e (service)i(name.)143 795 y Fo(\017)23 b Fu(empt)o(y)15 b Fp(p)q(o)o(rt)p 409 795 V 16 w(name)g Fu(non-empt)o(y)g Fp(service)p 894 795 V 17 w(name)p Fu(.)189 851 y(This)g(requires)h (that)f(an)g(implemen)o(tation)h(publish)i(names.)i(The)15 b(ro)q(ot)f(pro)q(cess)i(of)f(the)189 908 y Fp(MPI)p 274 908 V 15 w(CONNECT)i Fu(call)f(m)o(ust)f(b)q(e)g(within)i(the)e (scop)q(e)h(of)f Fp(service)p 1296 908 V 16 w(name)p Fu(.)143 1002 y Fo(\017)23 b Fu(empt)o(y)15 b Fp(p)q(o)o(rt)p 409 1002 V 16 w(name)g Fu(empt)o(y)g Fp(service)p 806 1002 V 16 w(name)189 1058 y Fu(This)g(is)h(not)f(allo)o(w)o(ed.)75 1180 y Fm(3.4.4)49 b(Non-blo)q(cking)19 b(routines)75 1266 y Fu(MPI)c(pro)o(vides)g(a)g(non-blo)q(c)o(king)i(v)o(ersion)e(of) f Fp(MPI)p 955 1266 V 16 w(A)o(CCEPT)h Fu(to)f(allo)o(w)i(a)e(serv)o (er)h(to)f(pro)q(cess)h(m)o(ultiple)75 1322 y(clien)o(ts)h(at)f(the)g (same)g(time.)75 1473 y Fp(MPI)p 160 1473 V 16 w(IA)o(CCEPT\(p)q(o)o (rt,)f(service)p 611 1473 V 17 w(name,)g(info,)h(ro)q(ot,)g(comm,)e (new)o(comm,)g(request\))117 1550 y Fh(IN)155 b Fp(p)q(o)o(rt)509 b Fh(p)q(ort)14 b(ob)r(ject)h(\(handle,)f(used)g(only)f(on)h Fb(ro)q(ot)p Fh(\))117 1625 y(IN)155 b Fp(service)p 446 1625 V 17 w(name)343 b Fh(user-supplied)22 b(name)d(of)i(service)h (\(string,)g(used)f(only)f(on)905 1682 y Fb(ro)q(ot)p Fh(\))117 1757 y(IN)155 b Fp(info)516 b Fh(implemen)o(tation-dep)q (enden)o(t)19 b(information)e(\(string,)22 b(used)905 1813 y(only)13 b(on)h Fb(ro)q(ot)p Fh(\))117 1888 y(IN)155 b Fp(ro)q(ot)508 b Fh(rank)14 b(in)f Fb(comm)f Fh(of)i(ro)q(ot)g(no)q (de)g(\(in)o(teger\))117 1964 y(IN)155 b Fp(comm)470 b Fh(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o(h)g(call)f(is)h (collectiv)o(e)g(\(string\))117 2039 y(OUT)108 b Fp(new)o(comm)397 b Fh(in)o(tercomm)o(unicator)12 b(with)i(clien)o(t)h(as)g(remote)f (group)g(\(han-)905 2095 y(dle\))117 2170 y(OUT)108 b Fp(request)452 b Fh(request)16 b(ob)r(ject)e(\(handle\))75 2295 y Fl(int)23 b(MPI)p 245 2295 15 2 v 17 w(Accept\(MPI)p 502 2295 V 16 w(Port)g(port,)h(char*)f(service)p 1092 2295 V 16 w(name,)g(int)h(root,)f(MPI)p 1562 2295 V 17 w(Comm)g(comm,)393 2351 y(MPI)p 468 2351 V 17 w(Comm)g(newcomm,)g(MPI)p 891 2351 V 17 w(Request*)g(request\))75 2438 y(MPI)p 150 2438 V 17 w(ACCEPT\(PORT,)f(SERVICE)p 645 2438 V 16 w(NAME,)h(COMM,)h(NEWCOMM,)e(REQUEST,)h(IERR\))170 2494 y(CHARACTER*\(*\))g(SERVICE)p 676 2494 V 16 w(NAME)170 2551 y(INTEGER)g(PORT,)h(COMM,)f(NEWCOMM,)f(REQUEST,)h(IERR)166 2637 y Fu(This)d(function)g(is)f(iden)o(tical)j(to)c Fp(MPI)p 837 2637 14 2 v 16 w(A)o(CCEPT)h Fu(except)h(that)e(it)i(do)q (es)f(not)g(blo)q(c)o(k.)33 b(An)19 b(appli-)75 2693 y(cation)d(ma)o(y)f(test)h(or)f(w)o(ait)h(for)f(completion)i(using)g Fp(MPI)p 1066 2693 V 16 w(W)l(AIT)p Fu(,)f Fp(MPI)p 1306 2693 V 15 w(TEST)p Fu(,)g(and)g(related)h(routines.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 33 35 33 34 bop 75 -100 a Fn(3.4.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fu(33)75 49 y(This)16 b(non-blo)q(c)o(king)h(collectiv)o(e)f(call)h(ob)q(eys)e(the) g(same)g(seman)o(tics)g(as)g(the)g Fp(MPI)p 1471 49 14 2 v 16 w(ISP)l(A)-5 b(WN)p 1650 49 V 17 w(*)15 b Fu(calls,)h(ex-)75 106 y(cept)i(that)f(it)h(ma)o(y)f(b)q(e)h(cancelled.)30 b(The)18 b(cancel)h(op)q(eration)f(is)g(collectiv)o(e)i(o)o(v)o(er)d (the)g(original)i(calling)75 162 y(comm)o(unicator.)166 301 y Fi(Discussion:)36 b Fh(Is)14 b(it)g(in)o(ten)o(tional)f(that)i (there)g(is)g(no)f Fb(MPI)p 1106 301 13 2 v 14 w(ICONNECT)p Fh(.)f(W)m(e)h(should)g(add)g(one)h(if)e(w)o(e)i(can)75 358 y(think)f(of)f(a)g(really)h(go)q(o)q(d)f(use.)75 562 y Fm(3.4.5)49 b(Releasing)17 b(Connections)75 648 y Fu(Before)i(a)g(clien)o(t)i(and)f(serv)o(er)f(connect,)h(they)g(are)f (indep)q(enden)o(t)i(MPI)f(applications.)34 b(An)19 b(error)g(in)75 704 y(one)h(do)q(es)f(not)h(a\013ect)e(the)i(other.)32 b(After)20 b(establishing)h(a)e(connection)i(with)e Fp(MPI)p 1555 704 14 2 v 16 w(CONNECT)i Fu(and)75 761 y Fp(MPI)p 160 761 V 16 w(A)o(CCEPT)p Fu(,)14 b(an)g(error)g(in)h(one)g(ma)o(y)f (a\013ect)f(the)i(other.)k(F)l(or)14 b(instance,)h(if)g(a)f(serv)o(er)g (tries)g(to)g(send)h(a)75 817 y(message)h(to)g(a)g(clien)o(t)h(whic)o (h)h(has)e(died,)h(the)g(serv)o(er)f(will)i(get)e(an)g(error.)23 b(It)16 b(is)h(desirable)h(for)e(a)g(clien)o(t)75 873 y(and)f(serv)o(er)f(to)g(b)q(e)h(able)h(to)e(disconnect,)h(so)g(that)e (an)i(error)f(in)h(one)g(will)h(not)f(a\013ect)f(the)g(other.)20 b(These)75 930 y(same)15 b(considerations)h(apply)g(to)f(paren)o(ts)f (and)i(c)o(hildren.)143 1034 y Fo(\017)23 b Fu(Tw)o(o)14 b(pro)q(cesses)i(are)e Fq(connected)j Fu(if)231 1127 y(1.)22 b(They)c(b)q(oth)h(b)q(elong)g(to)e(the)i(same)f(in)o(tercomm)o (unicator)g(or)g(comm)o(unicator)g(\(including)289 1183 y Fp(MPI)p 374 1183 V 15 w(COMM)p 531 1183 V 17 w(W)o(ORLD)p Fu(\))d(or)231 1256 y(2.)22 b(They)15 b(are)g(are)g(b)q(oth)g (connected)h(to)f(a)g(third)g(pro)q(cess.)143 1349 y Fo(\017)23 b Fu(Tw)o(o)14 b(pro)q(cesses)i(are)e Fq(disconnected)j Fu(\(also)e Fq(indep)q(enden)o(t)p Fu(\))g(if)h(they)f(are)g(not)g (connected.)143 1442 y Fo(\017)23 b Fu(By)18 b(the)h(ab)q(o)o(v)o(e)f (de\014nitions,)i(connectivit)o(y)g(is)f(a)f(transitiv)o(e)h(prop)q (ert)o(y)l(,)f(and)h(divides)h(the)f(uni-)189 1498 y(v)o(erse)e(of)g (MPI)g(pro)q(cesses)h(in)o(to)f(disconnected)i(\(indep)q(enden)o(t\))g (sets)e(\(equiv)m(alence)j(classes\))d(of)189 1555 y(pro)q(cesses.)143 1648 y Fo(\017)23 b Fu(Pro)q(cesses)17 b(whic)o(h)i(are)e(connected,)h (but)g(don't)f(share)g(the)h(same)f Fp(MPI)p 1441 1648 V 16 w(COMM)p 1599 1648 V 16 w(W)o(ORLD)h Fu(ma)o(y)189 1704 y(b)q(ecome)e(disconnected)h(\(indep)q(enden)o(t\))g(if)f(the)f (comm)o(unication)h(path)f(b)q(et)o(w)o(een)h(them)f(is)h(bro-)189 1761 y(k)o(en)f(b)o(y)g(using)h Fp(MPI)p 539 1761 V 16 w(COMM)p 697 1761 V 16 w(DISCONNECT)p Fu(.)166 1948 y Fi(Discussion:)58 b Fh(If)19 b(w)o(e)h(retain)g Fb(MPI)p 775 1948 13 2 v 14 w(PROCESS)p 972 1948 V 14 w(MONITOR)f Fh(and)g Fb(MPI)p 1360 1948 V 15 w(PROCESS)p 1558 1948 V 13 w(SIGNAL)h Fh(a)f(group)75 2004 y(handle)c(\(whic)o(h)h(ma)o(y)d (exist)j(after)f(a)g(disconnect\))i(still)e(pro)o(vides)g(a)g(mec)o (hanism)e(to)j(in)o(teract)f(with)h(a)f(pro)q(cess.)75 2061 y(This)f(should)f(b)q(e)i(disallo)o(w)o(ed,)d(but)i(it)g(is)f(not) h(clear)g(ho)o(w.)166 2200 y Fu(The)h(follo)o(wing)h(additional)h (rules)f(apply)g(to)e(MPI-1)h(functions:)143 2292 y Fo(\017)23 b Fp(MPI)p 274 2292 14 2 v 15 w(FINALIZE)15 b Fu(is)g(collectiv)o(e)i (o)o(v)o(er)e(a)g(set)g(of)f(connected)j(pro)q(cesses.)143 2385 y Fo(\017)23 b Fp(MPI)p 274 2385 V 15 w(ABORT)e Fu(do)q(es)e(not)f(ab)q(ort)g(indep)q(enden)o(t)k(pro)q(cesses.)31 b(As)19 b(in)g(MPI-1,)h(it)f(ma)o(y)f(ab)q(ort)g(all)189 2441 y(pro)q(cesses)12 b(in)h Fp(MPI)p 517 2441 V 16 w(COMM)p 675 2441 V 17 w(W)o(ORLD)f Fu(\(ignoring)g(its)h Fp(comm)d Fu(argumen)o(t\).)18 b(Additionally)l(,)d(it)e(ma)o(y)189 2498 y(ab)q(ort)k(connected)i(pro)q(cesses)g(as)f(w)o(ell,)i(though)e (it)g(mak)o(es)g(a)g(\\b)q(est)g(attempt")f(to)h(ab)q(ort)f(only)189 2554 y(the)e(pro)q(cesses)h(in)g Fp(comm)p Fu(.)143 2647 y Fo(\017)23 b Fu(If)10 b(a)h(pro)q(cess)f(exits)h(without)g(calling)h Fp(MPI)p 908 2647 V 16 w(FINALIZE)e Fu(the)g(e\013ect)g(on)h(other)f (connected)i(pro)q(cesses)189 2704 y(is)j(not)g(de\014ned.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 34 36 34 35 bop 75 -100 a Fu(34)928 b Fn(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 49 y Fi(Discussion:)46 b Fh(I)17 b(am)e(confused)j(ab)q(out)f(the)g(in)o(tended)h(b)q(eha)o (vior)e(of)h Fb(MPI)p 1392 49 13 2 v 14 w(FINALIZE)f Fh(in)h(MPI-1.)27 b(The)75 106 y(standard)13 b(do)q(es)g(not)f(state)i (if)d(it)h(is)h(collectiv)o(e)f(o)o(v)o(er)h Fb(MPI)p 972 106 V 14 w(COMM)p 1116 106 V 15 w(W)o(ORLD)f Fh(or)g(if)g(it)g(is)g (required)i(b)q(efore)f(exiting.)75 162 y(It)19 b(w)o(ould)g(b)q(e)h (nice)g(if)e(it)h(w)o(ere)i(b)q(oth.)34 b(Consider)20 b(the)g(follo)o(wing)d(fragmen)o(t.)33 b(If)19 b Fb(MPI)p 1496 162 V 14 w(FINALIZE)g Fh(is)g(a)g(lo)q(cal)75 219 y(op)q(eration,)d(and)g(cleans)h(up)f(all)f(MPI)i(state,)g(then)f (there)i(is)e(a)g(problem)f(if)g(the)i(send)g(in)f(pro)q(cess)i(1)e (has)g(b)q(een)75 275 y(bu\013ered)e(lo)q(cally)d(and)i(the)g(data.)18 b(has)12 b(not)h(y)o(et)g(b)q(een)h(receiv)o(ed.)19 b(So)12 b(I)h(conclude)h(that)e Fb(MPI)p 1529 275 V 15 w(FINALIZE)g Fh(m)o(ust)f(b)q(e)75 332 y(collectiv)o(e)i(or)g(at)g(least)g(non-lo)q (cal.)k(Similarl)o(y)m(,)9 b(the)14 b(follo)o(wing)c(example)i(sho)o (ws)h(that)g(either)h Fb(MPI)p 1636 332 V 15 w(FINALIZE)e Fh(or)75 388 y(some)h(other)i(collectiv)o(e)e(op)q(eration)h(usually)f (m)o(ust)g(b)q(e)i(called)e(b)q(efore)i(a)e(program)g(exits.)75 603 y Fl(process)23 b(1:)716 b(process)22 b(2:)75 659 y(...)883 b(...)75 715 y(MPI_Send\(buf,)22 b(...)i(dest=2,)e(comm\);) 191 b(...)75 772 y(MPI_Finalize\(\);)595 b(...)1030 828 y(MPI_Recv\(buf,)22 b(...)h(src=1,)g(comm\);)1030 885 y(MPI_Finalize\(\);)75 1111 y Fp(MPI)p 160 1111 14 2 v 16 w(COMM)p 318 1111 V 16 w(DISCONNECT\(comm\))117 1188 y Fh(INOUT)62 b Fp(comm)470 b Fh(comm)o(unicator)11 b(\(handle\))75 1313 y Fl(int)23 b(MPI)p 245 1313 15 2 v 17 w(Comm)p 358 1313 V 17 w(disconnect\(MPI)p 711 1313 V 15 w(Comm*)g(comm\))75 1399 y(MPI)p 150 1399 V 17 w(COMM)p 263 1399 V 16 w(DISCONNECT\(COMM,)f(IERR\))170 1456 y(INTEGER)h(COMM,)h(IERR)166 1542 y Fu(This)14 b(function)h(w)o (aits)e(for)g(all)i(p)q(ending)g(comm)o(unication)f(on)g Fp(comm)e Fu(to)h(complete)h(and)g(then)g(sets)75 1599 y(the)i(comm)o(unicator)g(to)f Fp(MPI)p 593 1599 14 2 v 16 w(COMM)p 751 1599 V 16 w(NULL)h Fu(in)g(the)g(same)g(manner)g(as)g Fp(MPI)p 1447 1599 V 15 w(COMM)p 1604 1599 V 17 w(FREE)p Fu(.)g(It)g(is)g(a)75 1655 y(collectiv)o(e)h(op)q(eration.)k(It)16 b(ma)o(y)f(not)g(b)q(e)h(called)h(with)f(the)g(comm)o(unicator)f Fp(MPI)p 1480 1655 V 15 w(COMM)p 1637 1655 V 17 w(W)o(ORLD)p Fu(.)g(It)75 1712 y(is)g(erroneous)f(to)g(call)i Fp(MPI)p 547 1712 V 16 w(COMM)p 705 1712 V 16 w(DISCONNECT)f Fu(if)g(not)f(all)h (comm)o(unication)h(can)e(b)q(e)h(completed,)75 1768 y(for)e(instance,)i(if)f(there)g(has)g(b)q(een)h(an)f(send)h(op)q (eration)f(without)g(a)f(matc)o(hing)h(receiv)o(e,)h(or)f(a)f (collectiv)o(e)75 1825 y(op)q(eration)g(that)f(has)h(not)f(b)q(een)i (called)h(on)d(all)i(pro)q(cesses)f(in)h(the)f(comm)o(unicator.)19 b(It)12 b(is)i(not)e(erroneous)75 1881 y(to)h(ha)o(v)o(e)g(outstanding) g(non-blo)q(c)o(king)i(op)q(erations,)f(as)f(long)h(as)f(those)g(op)q (erations)g(can)h(complete.)20 b(Af-)75 1937 y(ter)c Fp(MPI)p 232 1937 V 15 w(COMM)p 389 1937 V 17 w(DISCONNECT)h Fu(returns,)e(all)i(non-blo)q(c)o(king)h(op)q(erations)e(on)g(that)f (comm)o(unicator)75 1994 y(will)h(b)q(e)g(complete,)f(and)g(all)g(asso) q(ciated)g(handlers)h(will)g(ha)o(v)o(e)e(b)q(een)i(called.)22 b(It)14 b(is)h(not)g(allo)o(w)o(ed)g(to)f(call)75 2050 y Fp(MPI)p 160 2050 V 16 w(COMM)p 318 2050 V 16 w(DISCONNECT)i Fu(on)f Fp(MPI)p 779 2050 V 16 w(COMM)p 937 2050 V 16 w(W)o(ORLD)p Fu(.)166 2107 y(Note)10 b(that)h(it)g(ma)o(y)f(b)q(e)h (necessary)g(to)f(disconnect)j(sev)o(eral)e(comm)o(unicators)f(b)q (efore)h(t)o(w)o(o)f(pro)q(cesses)75 2163 y(are)15 b(completely)h (indep)q(enden)o(t.)189 2247 y Fj(R)n(ationale.)54 b Fu(It)17 b(w)o(ould)h(b)q(e)g(nice)h(to)e(b)q(e)h(able)h(to)d(use)i Fp(MPI)p 1235 2247 V 16 w(COMM)p 1393 2247 V 17 w(FREE)g Fu(instead,)g(but)g(that)189 2304 y(function)i(explicitly)i(do)q(es)d (not)g(w)o(ait)g(for)g(p)q(ending)i(comm)o(unication)f(to)f(complete.) 32 b(\()p Fj(End)20 b(of)189 2360 y(r)n(ationale.)p Fu(\))75 2479 y Fm(3.4.6)49 b(F)o(o)o(rmat)15 b(of)h(info)h(a)o(rguments)75 2647 y Fi(Discussion:)45 b Fh(There)18 b(has)f(b)q(een)h(considerable)f (distress)h(that)f(MPI)g(do)q(es)g(not)g(sp)q(ecify)g(the)g(format)e (of)h Fb(info)75 2704 y Fh(argumen)o(ts.)26 b(The)17 b(concern)i(is)d(that)h(it)g(will)e(b)q(e)j(imp)q(ossible)d(to)i(write) g(a)f(p)q(ortable)h(program.)26 b(On)17 b(the)g(other)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 35 37 35 36 bop 75 -100 a Fn(3.4.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fu(35)75 49 y Fh(hand,)i(the)i(whole)e(p)q(oin)o(t)h(of)f(the)h Fb(info)g Fh(argumen)o(t)f(w)o(as)h(to)f(sp)q(ecify)i(things)f(that)f(w)o(ere)i (inheren)o(tly)f(nonp)q(ortable.)75 106 y(Nev)o(ertheless,)h (di\013eren)o(t)f(implemen)o(tations)c(will)i(often)h(pro)o(vide)g (similar)d(functionalit)o(y)i(and)h(it)g(w)o(ould)f(b)q(e)i(nice)75 162 y(for)g(them)f(to)g(b)q(e)i(as)f(similar)d(as)j(p)q(ossible.)k (This)c(section)h(is)f(an)f(attempt)g(to)h(w)o(alk)f(the)h(thin)g (line.)166 301 y Fu(Most)h(of)g(the)i(routines)f(in)h(this)f(c)o (hapter)g(tak)o(e)g(an)f(argumen)o(t)h Fp(info)p Fu(,)f(whic)o(h)i(is)g (an)f(opaque)g(string)75 358 y(whose)g(con)o(ten)o(ts)f(are)g(implemen) o(tation-dep)q(end)q(en)o(t.)24 b(Ho)o(w)o(ev)o(er,)15 b(MPI)h(sp)q(eci\014es)h(the)f(general)h(syn)o(tax)75 414 y(of)c(an)h Fp(info)g Fu(argumen)o(t)f(and,)h(in)h(a)e(few)h (cases,)g(sp)q(eci\014es)h Fj(how)g Fu(functionalit)o(y)g(m)o(ust)e(b)q (e)i(made)f(a)o(v)m(ailable,)75 471 y(if)i(it)f(is)h(made)f(a)o(v)m (ailable.)166 527 y(In)25 b(all)h(cases,)g(the)f Fp(info)f Fu(argumen)o(t)g(m)o(ust)g(b)q(e)h(a)f(list)i(of)e(items)h(of)f(the)g (form)g(\\)p Fp(k)o(ey)p Fu(=)p Fp(value)p Fu(".)75 584 y(These)19 b(items)g(are)g(separated)f(b)o(y)h(whitespace)h(as)e (describ)q(ed)j(for)d(the)h Fp(command)p 1531 584 14 2 v 15 w(line)h Fu(argumen)o(t)e(to)75 640 y Fp(MPI)p 160 640 V 16 w(SP)l(A)-5 b(WN)p Fu(.)15 b(Whitespace)h(is)g(quoted)f (in)h(the)g(same)f(manner.)166 696 y(MPI)g(do)q(es)h(not)f(require)h (that)f(an)o(y)g(particular)h(functionalit)o(y)h(b)q(e)f(made)g(a)o(v)m (ailable)h(through)e(the)75 753 y Fp(info)i Fu(argumen)o(t.)23 b(Ho)o(w)o(ev)o(er,)16 b(if)h(an)g(implemen)o(tation)h(pro)o(vides)f (an)o(y)f(of)g(the)h(functionalit)o(y)h(describ)q(ed)75 809 y(b)q(elo)o(w,)c(it)g(m)o(ust)f(mak)o(e)g(that)f(functionalit)o(y)j (a)o(v)m(ailable)g(with)f(the)f(\(k)o(ey)l(,v)m(alue\))h(pair)g (indicated.)21 b(It)14 b(ma)o(y)75 866 y(mak)o(e)h(the)g(functionalit)o (y)h(a)o(v)m(ailable)h(through)e(other)g(means)g(as)g(w)o(ell.)75 986 y Fp(info)g(fo)o(r)f(all)h(routines)2 1072 y Fu(\014lename)24 b(If)16 b(it)h(is)g(p)q(ossible)h(to)d(sp)q(ecify)j(information)e (externally)i(in)f(a)f(\014le,)h(the)f(item)h(\\\014lename=[name)189 1128 y(of)d(\014le]")h(tells)g(MPI)g(to)f(get)g(information)g(from)g (that)g(\014le.)21 b(The)15 b(format)e(of)h(this)h(external)g(\014le)g (is)189 1185 y(not)f(sp)q(eci\014ed)k(b)o(y)d(MPI.)1 1278 y(\014leindex)25 b(If)13 b(it)g(is)g(p)q(ossible)i(to)d(lo)q(ok)h (at)f(a)h(sp)q(eci\014c)i(section)e(of)f(a)h(\014le,)h(the)f(item)g (\\\014leindex=[name)i(of)d(lab)q(el)189 1335 y(within)20 b(\014le]")f(tells)h(MPI)f(to)f(get)g(information)i(from)e(that)g (named)h(\(or)f(p)q(ossibly)i(n)o(um)o(b)q(ered\))189 1391 y(section.)g(Ho)o(w)15 b(sectioning)h(is)g(done)f(is)h(not)f(sp)q (eci\014ed)i(b)o(y)e(MPI.)75 1511 y Fp(info)g(fo)o(r)f(MPI)p 312 1511 V 16 w(SP)l(A)-5 b(WN)16 b(and)g(relatives)82 1597 y Fu(host)23 b(If)d(it)g(is)g(p)q(ossible)i(to)d(sp)q(ecify)i (that)e(all)i(pro)q(cesses)f(should)h(b)q(e)g(spa)o(wned)f(on)g(a)f(sp) q(eci\014c)j(host,)189 1654 y(\\host=[hostname]")d(tells)k(MPI)e(to)g (spa)o(wn)g(all)i(pro)q(cesses)f(on)f(that)g(host.)38 b(MPI)22 b(do)q(es)f(not)189 1710 y(sp)q(ecify)16 b(ho)o(w)f(the)g (host)g(is)h(named.)81 1804 y(arc)o(h)23 b(If)c(it)g(is)h(p)q(ossible)h (to)d(sp)q(ecify)j(that)d(all)i(pro)q(cesses)g(should)g(b)q(e)g(spa)o (wned)f(on)g(a)g(particular)g(ar-)189 1860 y(c)o(hitecture,)j(\\arc)o (h=[arc)o(hname]")d(tells)j(MPI)f(to)f(spa)o(wn)g(all)i(pro)q(cesses)f (on)f(that)g(particular)189 1917 y(arc)o(hitecture.)g(It)15 b(is)h(left)f(to)g(the)g(implemen)o(tation)i(to)d(sp)q(ecify)j(arc)o (hitecture)e(names.)108 2011 y(wd)23 b(If)15 b(it)h(is)g(p)q(ossible)h (to)e(sp)q(ecify)i(that)e(all)h(pro)q(cesses)g(should)h(b)q(e)f (started)e(with)i(a)f(certain)h(w)o(orking)189 2067 y(directory)l(,)j (the)f(item)g(\\wd=[directory)p 892 2067 V 17 w(name]")f(tells)i(MPI)f (to)g(spa)o(wn)f(all)i(pro)q(cesses)g(in)g(that)189 2124 y(directory)l(.)31 b(If)19 b(the)g(directory)g(do)q(es)g(not)f(exist)h (on)g(one)g(or)f(more)h(hosts)f(where)h(MPI)g(tries)g(to)189 2180 y(spa)o(wn)i(the)g(pro)q(cesses,)i(the)f(spa)o(wn)f(fails)h (\(whether)f(or)g(not)g(an)h(error)e(o)q(ccurs)i(dep)q(ends)h(on)189 2236 y(whether)15 b(the)g(spa)o(wn)g(w)o(as)g(soft)f(or)h(not\).)75 2357 y Fp(info)g(fo)o(r)f(MPI)p 312 2357 V 16 w(PORT)p 450 2357 V 17 w(OPEN)27 2442 y Fu(ip)p 67 2442 V 17 w(p)q(ort)23 b(If)c(an)g(implemen)o(tation)i(allo)o(ws)e(a)g(serv)o(er)g(to)f(sp)q (ecify)j(an)e(IP)h(p)q(ort)e(n)o(um)o(b)q(er)i(for)f(the)g(address)189 2499 y(of)d(a)g(p)q(ort,)g(\\ip)p 458 2499 V 17 w(p)q(ort=[p)q(ort]")f (tells)i(MPI)f(to)g(use)h(that)e(p)q(ort.)23 b([p)q(ort])16 b(m)o(ust)g(b)q(e)h(either)g(a)f(n)o(um-)189 2555 y(b)q(er)j(or)g(a)g (name)g(that)g(is)g(alw)o(a)o(ys)g(asso)q(ciated)g(with)h(a)f(certain)g (p)q(ort)g(n)o(um)o(b)q(er)h(\(e.g.,)e(through)189 2612 y(/etc/services\).)h(Note)c(that)f(if)i(the)f(so)q(c)o(k)o(et)f(cannot) h(b)q(e)g(b)q(ound)h(to)e(this)i(p)q(ort,)e(an)h(error)f(o)q(ccurs.)189 2668 y(T)l(o)e(mak)o(e)h(this)g(feature)g(useful,)h(an)f(application)h (m)o(ust)f(b)q(e)g(able)h(to)e(\014gure)h(out)g(a)f Fp(p)q(o)o(rt)p 1677 2668 V 17 w(name)g Fu(if)i(it)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 36 38 36 37 bop 75 -100 a Fu(36)933 b Fn(CHAPTER)16 b(3.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)189 49 y Fu(kno)o(ws)g(a)g(host)g(name)h (and)g(a)f(p)q(ort.)21 b(Therefore)16 b(the)f Fp(p)q(o)o(rt)p 1197 49 14 2 v 17 w(name)g Fu(asso)q(ciated)h(with)g(suc)o(h)g(a)f(p)q (ort)189 106 y(m)o(ust)f(b)q(e)i(\\hostname:p)q(ort".)75 227 y Fm(3.4.7)49 b(Client/Server)16 b(Examples)75 313 y Fq(Example)i(3.1)k Fu(Simplest)17 b(example)f(-)f(completely)i(p)q (ortable.)166 418 y(The)i(follo)o(wing)h(example)g(sho)o(ws)f(the)g (simplest)i(w)o(a)o(y)d(to)h(use)g(the)h(clien)o(t/serv)o(er)g(in)o (terface.)32 b(It)75 474 y(do)q(es)16 b(not)e(use)i(service)g(names)f (at)g(all.)166 531 y(On)h(the)f(serv)o(er)g(side:)170 680 y Fl(MPI_Port)23 b(myport;)170 736 y(MPI_Comm)g(intercomm;)170 793 y(char)h(name[256];)170 849 y(/*)g(...)f(*/)170 906 y(MPI_Port_open\(NULL,)f(&port\);)170 962 y(/*)i(if)g(namelen)f(>)g (256)h(this)f(returns)g(an)h(error)f(*/)170 1018 y (MPI_Port_name\(port,)f(name,)h(256\);)170 1075 y(printf\("port)g(name) g(is:)g(",)h(\045s\\n",)f(name\);)170 1188 y(MPI_Accept\(port,)f(NULL,) h(0,)h(MPI_COMM_SELF,)e(&intercomm\);)170 1244 y(/*)i(do)g(something)e (with)i(intercomm)e(*/)75 1337 y Fu(The)d(serv)o(er)g(prin)o(ts)h(out)f (the)g(p)q(ort)g(name)g(to)g(the)g(terminal)h(and)f(the)h(user)f(m)o (ust)g(t)o(yp)q(e)g(it)g(in)h(when)75 1393 y(starting)g(up)h(the)g (clien)o(t)h(\(assuming)f(the)g(MPI)g(implemen)o(tation)h(supp)q(orts)f (stdin)g(suc)o(h)g(that)g(this)75 1450 y(w)o(orks\).)e(On)c(the)h (clien)o(t)g(side:)170 1542 y Fl(MPI_Comm)23 b(intercomm;)170 1599 y(char)h(name[256];)170 1655 y(printf\("enter)f(port)g(name:)g ("\);)g(fflush\(stdout\);)170 1712 y(gets\(name\);)170 1768 y(MPI_Connect\(name,)f(NULL,)h(NULL,)g(0,)h(MPI_COMM_SELF,)e (&intercomm\);)75 1873 y Fq(Example)c(3.2)k Fu(Multiple)17 b(metho)q(d)f(names)f(-)g(completely)i(p)q(ortable.)166 1978 y(This)e(example)g(sho)o(ws)e(ho)o(w)h(a)f(serv)o(er)h(could)h (use)g(m)o(ultiple)h(service)f(names)f(to)f(pro)o(vide)i(di\013eren)o (t)75 2034 y(services)g(with)f(a)f(single)i(p)q(ort.)20 b(Since)15 b(the)f(name)g(is)g(explicitly)i(transferred)e(through)f (prin)o(tf/gets)h(this)75 2091 y(example)i(is)g(completely)g(p)q (ortable,)g(regardless)f(of)g(whether)g(there)g(is)h(a)f(name)g(serv)o (er.)166 2147 y(This)h(and)f(follo)o(wing)h(examples)g(assume)f(the)g (same)g(declarations)h(as)f(the)g(\014rst.)170 2308 y Fl(MPI_Request)23 b(rqs[2];)170 2365 y(MPI_Status)g(status;)170 2421 y(MPI_Comm)g(inters[2];)170 2478 y(int)h(index;)170 2591 y(MPI_Port_open\(NULL,)e(&port\);)170 2647 y (MPI_Port_add_service\(port,)f("method1"\);)170 2704 y(MPI_Port_add_service\(port,)g("method2"\);)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 37 39 37 38 bop 75 -100 a Fn(3.4.)23 b(EST)l(ABLISHING)13 b(COMMUNICA)l(TION) g(BETWEEN)f(INDEPENDENT)g(PR)o(OCESSES)p Fu(37)170 106 y Fl(MPI_Port_name\(port,)22 b(name,)h(256\);)170 162 y(printf\("port)g(name)g(is:)g(",)h(\045s\\n",)f(name\);)170 275 y(MPI_Iaccept\(port,)f("method0",)g(NULL,)i(0,)f(MPI_COMM_SELF,)f (&inters[0],)h(&rqs[0]\);)170 332 y(MPI_Iaccept\(port,)f("method1",)g (NULL,)i(0,)f(MPI_COMM_SELF,)f(&inters[1],)h(&rqs[1]\);)170 388 y(MPI_Waitany\(2,)f(requests,)h(&index,)g(&status\);)170 444 y(/*)h(do)g(something)e(with)i(inters[index])e(*/)75 593 y Fu(On)16 b(the)f(clien)o(t)i(side:)170 676 y Fl(printf\("enter)23 b(port)g(name:)g("\);)g(fflush\(stdout\);)170 732 y(gets\(name\);)170 789 y(MPI_Connect\(name,)f(,)i("method1",)e(0,)i(MPI_COMM_SELF,)e (&intercomm\);)75 881 y Fq(Example)c(3.3)k Fu(Ocean/A)o(tmosphere)16 b(-)f(relies)i(on)e(name)g(publishing)166 973 y(In)24 b(this)g(example,)i(the)e(\\o)q(cean")f(application)i(is)f(the)g (\\serv)o(er")f(side)h(of)f(a)g(coupled)i(o)q(cean-)75 1030 y(atmosphere)15 b(climate)h(mo)q(del.)21 b(It)15 b(assumes)g(that)f(the)i(MPI)f(implemen)o(tation)h(publishes)i(names.) 170 1179 y Fl(MPI_Port_open\(NULL,)k(&port\);)170 1235 y(MPI_Port_add_service\(port,)f("ocean"\);)170 1348 y (MPI_Accept\(port,)h("ocean",)h(0,)g(MPI_COMM_SELF,)f(&intercomm\);)170 1405 y(/*)i(do)g(something)e(with)i(intercomm)e(*/)170 1461 y(MPI_Port_delete_service\(port,)e("ocean"\);)75 1610 y Fu(On)c(the)f(clien)o(t)i(side:)170 1692 y Fl (MPI_Connect\(NULL,)22 b("ocean",)h(0,)g(MPI_COMM_SELF,)f (&intercomm\);)75 1785 y Fq(Example)c(3.4)k Fu(Ocean/A)o(tmosphere)16 b(-)f(uses)h(name)f(publishing)j(if)d(p)q(ossible)166 1877 y(This)j(example)g(is)f(similar)i(to)d(the)h(previous)h(one)f (except)h(that)e(is)i(mak)o(es)f(a)f(run)i(time)f(decision)75 1934 y(\(compile)f(time)g(w)o(ould)f(also)h(b)q(e)f(p)q(ossible\))i (based)f(on)f(whether)g(names)g(are)g(published.)170 2083 y Fl(MPI_Port_open\(NULL,)22 b(&port\);)170 2139 y(MPI_Port_add_service\(port,)f("ocean"\);)170 2252 y (MPI_Accept\(port,)h("ocean",)h(0,)g(MPI_COMM_SELF,)f(&intercomm\);)170 2308 y(if)i(\(!MPI_NAMES_ARE_PUBLISHED\))c({)218 2365 y(if)k(\(batch_mode\))e(error\(FATAL,)g("can't)h(transfer)g(port)g (name"\);)218 2421 y(MPI_Port_name\(port,)e(name,)j(256\);)218 2478 y(printf\("port)e(name)i(is:)f(",)h(\045s\\n",)f(name\);)170 2534 y(})170 2591 y(/*)h(do)g(something)e(with)i(intercomm)e(*/)170 2647 y(MPI_Port_delete_service\(port,)e("ocean"\);)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 38 40 38 39 bop 75 -100 a Fu(38)933 b Fn(CHAPTER)16 b(3.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fu(On)h(the)f(clien)o(t)i(side:) 170 143 y Fl(if)24 b(\(!MPI_NAMES_ARE_PUBLISHED\))c({)218 200 y(if)k(\(batch_mode\))e(error\(FATAL,)g("can't)h(transfer)g(port)g (name"\);)218 256 y(printf\("low)g(quality)f(mpi)i(implementation)e (encountered.)g(\\\\)409 312 y(enter)h(port)h(name:)f("\);)g (fflush\(stdout\);)218 369 y(gets\(name\);)170 425 y(})170 482 y(else)h(name[0])f(=)g('\\0';)170 538 y(MPI_Connect\(name,)f ("ocean",)h(0,)g(MPI_COMM_SELF,)f(&intercomm\);)75 660 y Fm(3.4.8)49 b(Examples)16 b(of)h(p)q(ossible)g(uses)e(of)i(info)g(a)o (rguments)75 746 y Fu(The)12 b Fp(info)f Fu(argumen)o(t)f(in)i(the)g (routines)g Fp(MPI)p 826 746 14 2 v 15 w(PORT)p 963 746 V 18 w(OPEN)p Fu(,)f Fp(MPI)p 1208 746 V 16 w(A)o(CCEPT)h Fu(and)f Fp(MPI)p 1576 746 V 16 w(CONNECT)h Fu(are)75 802 y(in)o(tended)i(to)e(allo)o(w)h(an)f(MPI)h(implemen)o(tation)g(to)f (pro)o(vide)h(extra)f(con)o(trol)h(o)o(v)o(er)e(ho)o(w)h(comm)o (unication)75 859 y(is)17 b(established.)24 b(This)17 b(section)g(con)o(tains)f(sev)o(eral)g(examples)h(things)g(that)e(an)h (MPI)h(implemen)o(tation)75 915 y(migh)o(t)d(supp)q(ort.)19 b(The)c(syn)o(tax)e(of)g(the)h Fp(info)g Fu(string)g(is)h(purely)g(h)o (yp)q(othetical)g({)e(it)i(is)f(not)g(required)h(that)75 972 y(MPI)g(implemen)o(tations)i(supp)q(ort)e(the)g(syn)o(tax)g (implied)i(in)f(these)g(examples.)75 1078 y Fq(Example)i(3.5)k Fu(P)o(ossible)16 b(uses)g(of)f Fp(info)g Fu(in)h Fp(MPI)p 927 1078 V 16 w(PORT)p 1065 1078 V 17 w(OPEN)166 1184 y Fu(A)f(system)f(serv)o(er)g(ma)o(y)g(need)i(to)e(get)g(a)h(reserv)o (ed)f(p)q(ort)h(in)g(order)g(to)f(pro)o(v)o(e)g(to)g(the)h(clien)o(t)h (that)e(it)75 1241 y(is)i(a)f(privileged)i(pro)q(cess)f(and)f(not)g(a)g (user-lev)o(el)h(tro)s(jan)e(horse:)147 1334 y Fl (MPI_Port_open\("type=pri)o(viliged)o(",)21 b(&port\);)166 1428 y Fu(The)j(implemen)o(tation)h(ma)o(y)e(pro)o(vide)h(a)f(w)o(a)o (y)g(for)g(the)h(application)h(to)e(b)q(e)i(able)f(to)f(predict)75 1485 y Fp(p)q(o)o(rt)p 156 1485 V 17 w(name)18 b Fu(so)h(that)g(it)h (do)q(esn't)f(need)h(to)f(b)q(e)h(transferred)f(b)o(y)g(hand)h(\(in)g (a)f(non-publishing)j(imple-)75 1541 y(men)o(tation\).)j(If)17 b(the)h(follo)o(wing)f(example)h(is)g(run)f(on)h(the)f(mac)o(hine)h (\\dop)q(ey",)f(the)g(follo)o(wing)h(migh)o(t)75 1598 y(pro)q(duce)e(a)f(p)q(ort)g(name)g(\\dop)q(ey:2556")f(\(assuming)i (that)e(the)i(p)q(ort)f(is)g(not)g(already)h(in)g(use,)f(whic)o(h)h(is) 75 1654 y(wh)o(y)f(this)h(is)f(dangerous\).)123 1748 y Fl(MPI_Port_open\("ip_port=2)o(556",)21 b(&port\);)166 1842 y Fu(The)15 b(implemen)o(tation)i(ma)o(y)d(allo)o(w)i(automatic)e (k)o(erb)q(eros)h(authen)o(tication.)123 1935 y Fl (MPI_Port_open\("verify=ke)o(rberos,)20 b(acllist=mylist",)i(&port\);) 166 2029 y Fu(If)12 b(a)f(host)g(has)h(m)o(ultiple)i(in)o(terfaces,)e (it)g(ma)o(y)f(w)o(an)o(t)f(to)h(b)q(e)i(kno)o(wn)e(b)o(y)h(the)g(IP)g (address)f(of)h(a)f(sp)q(eci\014c)75 2086 y(in)o(terface:)123 2180 y Fl(MPI_Port_open\("localhost)o(=dopey-)o(hippi.mp)o(i.org",)20 b(&port\);)166 2273 y Fu(The)15 b(application)i(ma)o(y)e(b)q(e)h(able)g (to)e(sp)q(ecify)j(the)e(lo)q(cation)h(of)f(a)f(nameserv)o(er)h(used)h (b)o(y)f(MPI.)123 2367 y Fl(MPI_Port_open\("nameserve)o(r=names)o (erver.mp)o(i.nas.n)o(asa.gov)o(",)21 b(&port\);)166 2461 y Fu(An)16 b(implemen)o(tation)g(ma)o(y)f(allo)o(w)g(m)o(ultiple)j (instances)e(of)f(the)g(same)g(serice)h(name)g(in)g(the)f(same)75 2517 y(namespace.)33 b(In)20 b(this)g(case,)g(an)f(implemen)o (tation-de\014ned)j(p)q(olicy)f(w)o(ould)f(determine)g(to)f(whic)o(h)h (of)75 2574 y(sev)o(eral)15 b(serv)o(ers)g(a)g(clien)o(t)h(is)g (connected.)123 2668 y Fl(MPI_Port_open\("allow_mul)o(tiple=y)o(es",)21 b(&port\);)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 39 41 39 40 bop 75 -100 a Fn(3.5.)29 b(EXAMPLES)1398 b Fu(39)75 49 y Fq(Example)18 b(3.6)k Fu(P)o(ossible)16 b(uses)g(of)f Fp(info)g Fu(in)h Fp(MPI)p 927 49 14 2 v 16 w(A)o(CCEPT)166 152 y Fu(An)f(implemen)o(tation)i(migh)o(t)e(allo)o(w)g(authen)o (tication)h(on)f(a)g(p)q(er-connection)i(basis.)123 242 y Fl(MPI_Accept\(port,)k(name,)j("verify=kerberos,)d(user=wcs",)i(0,)g (MPI_COMM_SELF,)f(newcomm\);)166 333 y Fu(An)15 b(implemen)o(tation)i (migh)o(t)e(pro)o(vide)h(real-time)g(services)g(on)f(a)g(p)q (er-connection)i(basis.)123 424 y Fl(MPI_Accept\(port,)k(name,)j ("realtime_priority)d(=)j(123",)f(0,)g(MPI_COMM_SELF,)f(newcomm\);)75 526 y Fq(Example)c(3.7)k Fu(P)o(ossible)16 b(uses)g(of)f Fp(info)g Fu(in)h Fp(MPI)p 927 526 V 16 w(CONNECT)166 629 y Fu(An)f(implemen)o(tation)i(migh)o(t)e(allo)o(w)g(an)h (application)g(to)f(send)h(a)f(passw)o(ord)f(for)g(v)o(eri\014cation:) 123 720 y Fl(MPI_Connect\(port_name,)21 b(name,)i("password=kzzf&$l",)e (0,)j(MPI_COMM_SELF,)e(newcomm\);)166 810 y Fu(An)15 b(implemen)o(tation)g(migh)o(t)f(allo)o(w)h(an)f(application)i(to)e (use)h(a)f(remote)f(nameserv)o(er)h(for)g(lo)q(oking)75 867 y(up)i(service)g(names.)123 957 y Fl(MPI_Connect\(port_name,)21 b(name,)i("server=mpi.mcs.anl.gov")o(,)e(0,)j(MPI_COMM_SELF,)e (newcomm\);)75 1100 y Fr(3.5)59 b(Examples)75 1202 y Fp(Manager-w)o(o)o(rk)o(er)13 b(example,)i(using)h(SP)l(A)-5 b(WN.)75 1292 y Fl(/*)24 b(manager)e(*/)75 1349 y(#include)h()75 1405 y(main\(int,)g(argc,)g(char)g(*arg[]\))75 1462 y({)147 1518 y(MPI_Status)f(status;)147 1574 y(int)h(world_size;)147 1631 y(MPI_Comm)f(everyone;)262 b(/*)23 b(intercommunicator)f(*/)147 1744 y(MPI_Init\(&argc,)f(&argv\);)147 1800 y(MPI_Comm_size\(MPI_COMM_) o(WORLD,)f(&world_size\);)147 1857 y(if)j(\(world_size)g(!=)g(1\))191 b(error\("Top)22 b(heavy)h(with)h(management"\);)147 1913 y(if)f(\(MPI_UNIVERSE_SIZE)f(==)h(1\))h(error\("No)e(room)i(to)f (start)g(workers"\);)147 1970 y(MPI_Spawn\("worker",)e (MPI_UNIVERSE_SIZE-1,)g(MPI_UNIVERSE_SIZE-1,)385 2026 y(NULL,)i(0,)h(MPI_COMM_SELF,)e(&everyone,)g(NULL\);)147 2083 y(/*)170 2139 y(*)i(Parallel)f(code)g(here.)g(The)h(communicator)e ("everyone")h(can)g(be)h(used)170 2195 y(*)g(to)g(communicate)e(with)h (the)h(spawned)f(processes,)f(which)h(have)h(ranks)f(0,..)170 2252 y(*)h(MPI_UNIVERSE_SIZE-1)d(in)j(the)f(remote)h(group)f(of)g(the)h (intercommunicator)170 2308 y(*)g("everyone".)170 2365 y(*/)147 2478 y(MPI_Finalize\(\);)75 2534 y(})75 2647 y(/*)g(worker)f(*/)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 40 42 40 41 bop 75 -100 a Fu(40)933 b Fn(CHAPTER)16 b(3.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fl(#include)23 b()75 106 y(main\(int)g(argc,)g(char)g(*argv[]\))75 162 y({)147 219 y(MPI_Comm)f(myboss;)147 275 y(MPI_Init\(&argc,)f (&argv\);)147 388 y(MPI_Parent\(&myboss\);)147 501 y(/*)170 557 y(*)j(Parallel)f(code)g(here.)170 614 y(*)h(The)g(manager)f(is)g (represented)f(as)i(the)g(process)e(with)i(rank)f(0)h(in)f(\(the)h (remote)170 670 y(*)g(group)f(of\))h(``myboss''.)46 b(If)24 b(the)f(workers)g(need)g(to)h(communicate)e(among)170 727 y(*)i(themselves,)f(they)g(can)g(either)g(extract)g(the)h(local)f (group)g(of)h(the)170 783 y(*)g(intercommunicator)e(MPI_COMM_PARENT,)f (or)j(use)f(MPI_COMM_WORLD.)46 b(They)170 840 y(*)24 b(should)f(be)h(the)f(same.)170 896 y(*/)147 1009 y(MPI_Finalize\(\);) 75 1065 y(})75 1298 y Fp(T)l(ask)15 b(fa)o(rm)e(example.)44 b Fu(This)16 b(example)f(sho)o(ws)f(ho)o(w)g(to)g(manage)g(a)g(set)g (of)g(non-MPI)h(pro)q(cesses.)20 b(Note)75 1355 y(that)c(all)h(groups)g (con)o(tain)f(a)h(single)g(pro)q(cess)g(b)q(ecause)h(it)f(w)o(ould)g(b) q(e)g(a)f(mess)h(otherwise.)24 b(This)17 b(could)75 1411 y(b)q(e)d(\\impro)o(v)o(ed")f(b)o(y)g(using)i Fp(MPI)p 641 1411 14 2 v 15 w(SP)l(A)-5 b(WN)p 806 1411 V 18 w(MUL)l(TIPLE)p 1041 1411 V 15 w(INDEPENDENT)14 b Fu(but)f(this)h(w)o(ould)g(also)f (mak)o(e)75 1468 y(group)i(managemen)o(t)f(di\016cult.)75 1574 y Fl(#include)23 b()75 1631 y(#define)g(MAXPROC)g(1000)75 1687 y(main\(int)g(argc,)g(char)g(*argv[]\))75 1743 y({)147 1800 y(MPI_Group)f(workers[MAXPROC];)147 1856 y(MPI_Request)g (obituaries[MAXPROC];)147 1913 y(int)h(nslots,)g(i,)h(deadone;)147 1969 y(char)f(*args;)147 2026 y(char)g(command_lines[MAXLEN];)147 2139 y(ThingToDo)f(*thingstodo;)147 2195 y(int)h(nthingstodo,)f (nthingsdone;)147 2308 y(MPI_Init\(&argc,)f(&argv\);)147 2364 y(nslots)i(=)g(MPI_UNIVERSE_SIZE)f(-)i(1;)f(//)h(assume)f(only)g (one)h(copy)f(of)h(me!)147 2421 y(GetThingsToDo\(&thingsto)o(do,)d (nthingstodo\);)147 2534 y(/*)i(need)g(to)h(initialize)f(these)g(for)g (Waitany/all\(\))f(below)h(to)h(work)f(*/)147 2590 y(for)g(\(i)h(=)f (0;)h(i)g(<)f(MAXPROC;)g(i++\))g(obituaries[i])g(=)g(MPI_REQUEST_NULL;) 147 2703 y(/*)g(start)g(up)h(initial)f(set)g(of)h(tasks)f(*/)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 41 43 41 42 bop 75 -100 a Fn(3.5.)29 b(EXAMPLES)1398 b Fu(41)147 49 y Fl(for)23 b(\(i)h(=)f(0;)h(i)g(<)f(nslots)g(&&)h(i)g(<)f (nthingstodo;)g(i++\))147 106 y({)218 162 y(//)h(we)f(want)h(to)f(run)h (the)f(command)g("cow")g(with)h(some)f(special)g(args)218 219 y(SetArgs\(thingstodo[i],)e(&args\);)218 275 y (sprintf\(command_line,)g("cow)i(\045s",)h(args\);)218 332 y(MPI_Spawn_independent_nonmp)o(i\(comman)o(d_line,)c(1,)k(1,)f (NULL,)h(0,)743 388 y(MPI_COMM_SELF,)e(&workers[i],)g(NULL\))218 444 y(MPI_Process_monitor\(workers)o([i],)f(0,)i(MPI_PROCESS_DIED,)f (&obituaries[i]\);)147 501 y(})147 557 y(nthingsdone=i;)147 670 y(/*)h(spawn)g(new)h(ones)f(as)h(old)f(ones)g(finish)h(*/)147 727 y(while\(nthingsdone)d(<)j(nthingstodo\))147 783 y({)218 840 y(MPI_Waitany\(nslots,)d(obituaries,)i(&deadone,)f(NULL\);) 218 896 y(SetArgs\(thingstodo[nthingsd)o(one],)f(&args\);)218 953 y(sprintf\(command_lines[0],)g("cow)i(\045s",)g(args\))218 1009 y(MPI_Spawn_independent_nonmp)o(i\(comman)o(d_line,)d(1,)k(1,)f (NULL,)h(0,)743 1065 y(MPI_COMM_SELF,)e(&workers[deadone],)g(NULL\);) 218 1122 y(MPI_Notify\(MPI_NOTIFY_EXIT,)e(&workers[deadone],)i (&obituaries[nthingsdone])o(\);)147 1178 y(})147 1235 y(MPI_Waitall\(nslots,)f(obituaries,)h(NULL\);)147 1348 y(MPI_Finalize\(\);)75 1404 y(})75 1581 y Fp(PVM-st)o(yle)16 b(SPMD)f(example)45 b Fu(This)16 b(is)g(ho)o(w)f(man)o(y)f(PVM)h (programs)f(are)h(t)o(ypically)i(written.)j(There)75 1637 y(is)f(no)g(reason)f(they)h(can't)f(b)q(e)h(done)g(with)g(MPI-1)g (\(except)f(for)g(p)q(erformance!\),)h(but)g(in)g(case)g(users)75 1694 y(w)o(an)o(t)e(the)i(app)q(earance)f(of)g(minimal)i(c)o(hange,)f (here)g(is)f(is.)30 b(\(V)l(ery)18 b(similar)i(to)d(the)i(manager-w)o (ork)o(er)75 1750 y(example)d(ab)q(o)o(v)o(e.\))75 1856 y Fl(/*)24 b(myprog)f(*/)75 1969 y(#include)g()75 2026 y(main\(int)g(argc,)g(char)g(*argv[]\))75 2082 y({)147 2139 y(MPI_Comm)f(everyone;)238 b(/*)23 b(intercommunicator)f(*/)147 2195 y(MPI_Comm)g(new_world;)214 b(/*)23 b(corresponding)f (intracommunicator)g(*/)147 2252 y(MPI_Comm)g(myparent;)147 2308 y(MPI_Init\(&argc,)f(&argv\);)147 2421 y(MPI_Parent\(myparent\);) 147 2477 y(if)i(\(myparent)g(==)g(MPI_COMM_NULL\))46 b(/*)24 b(I'm)f(the)h(parent)f(*/)147 2534 y({)218 2590 y(MPI_Spawn\("myproc",)e(MPI_UNIVERSE_SIZE-1,)h(MPI_UNIVERSE_SIZE-1,) 433 2647 y(NULL,)h(0,)h(MPI_COMM_SELF,)e(&everyone,)g(NULL\);)218 2703 y(MPI_Intercomm_merge\(everyon)o(e,)f(FALSE,)i(&new_world\);)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 42 44 42 43 bop 75 -100 a Fu(42)933 b Fn(CHAPTER)16 b(3.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)147 49 y Fl(})147 106 y(else)218 162 y(MPI_Intercomm_merge\(myparen)o(t,)21 b(TRUE,)i(&new_world\);)170 275 y(/*)h(SPMD)f(parallel)g(code)g(here,)h (using)f(new_world)f(instead)h(of)h(MPI_COMM_WORLD)e(*/)170 388 y(MPI_Finalize\(\);)75 444 y(})75 621 y Fp(Simple)e(client-server)i (example.)44 b Fu(This)21 b(is)g(a)g(simple)h(example;)h(the)e(serv)o (er)f(accepts)g(only)h(a)g(single)75 677 y(connection)14 b(at)d(a)i(time)g(and)f(serv)o(es)h(that)e(connection)j(un)o(til)g(the) e(clien)o(t)i(requests)f(to)e(b)q(e)j(disconnected.)75 734 y(The)h(serv)o(er)g(is)h(a)f(single)h(pro)q(cess.)166 790 y(Here)21 b(is)h(the)g(serv)o(er.)38 b(It)21 b(accepts)h(a)f (single)h(connection)h(and)e(then)h(pro)q(cesses)g(data)e(un)o(til)j (it)75 847 y(receiv)o(es)16 b(a)f(message)g(with)g(tag)f Fl(1)p Fu(.)20 b(A)15 b(message)g(with)h(tag)e Fl(0)h Fu(tells)h(the)g(serv)o(er)e(to)h(exit.)75 941 y Fl(#include)23 b("mpi.h")75 997 y(main\()g(int)h(argc,)f(char)g(**argv)g(\))75 1053 y({)170 1110 y(MPI_Comm)g(client;)170 1166 y(MPI_Status)g(status;) 170 1223 y(MPI_Port)g(port;)170 1279 y(char)h(port_name[100];)170 1336 y(double)f(buf[MAX_DATA];)170 1392 y(char)71 b (internal_name[MAX_NAME_SIZE],)20 b(published_name[MAX_NAME_SIZE])o(;) 170 1449 y(int)95 b(size,)24 b(again;)170 1562 y(MPI_Init\()f(&argc,)g (&argv)g(\);)170 1618 y(MPI_Comm_size\(MPI_COMM_WORLD,)d(size\);)170 1675 y(if)k(\(size)f(!=)h(0\))f(error\(FATAL,)g("Server)g(too)g (big"\);)170 1731 y(MPI_Port_open\(NULL,)f(&port\);)170 1787 y(MPI_Port_name\(port,)g(port_name,)g(100\);)170 1844 y(printf\("server)g(available)h(at)h(\045s\\n",port_name\);)170 1900 y(while)g(\(1\))f({)266 1957 y(MPI_Accept\()f(port,)h(NULL,)h (NULL,)f(0,)g(MPI_COMM_WORLD,)f(&client)h(\);)266 2013 y(again)g(=)h(1;)266 2070 y(while)f(\(again\))g({)361 2126 y(MPI_Recv\()g(buf,)g(MAX_DATA,)g(MPI_DOUBLE,)f(MPI_ANY_SOURCE,)g (MPI_ANY_TAG,)600 2183 y(client,)h(&status)g(\);)361 2239 y(switch)g(\(status.tag\))g({)457 2296 y(case)g(0:)h (MPI_Comm_free\()e(&client)h(\);)648 2352 y(MPI_Finalize\(\);)648 2408 y(return)g(0;)457 2465 y(case)g(1:)h(MPI_Comm_disconnect\()d (&client)i(\);)648 2521 y(again)g(=)h(0;)648 2578 y(break;)457 2634 y(case)f(2:)h(/*)f(do)h(something)f(*/)457 2691 y(...)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 43 45 43 44 bop 75 -100 a Fn(3.5.)34 b(EXAMPLES)1393 b Fu(43)457 49 y Fl(default:)648 106 y(MPI_Abort\()22 b(MPI_COMM_WORLD,)g ("Unexpected)g(message)h(type")g(\);)457 162 y(})361 219 y(})266 275 y(})75 332 y(})166 425 y Fu(Here)15 b(is)h(the)f(clien) o(t.)75 532 y Fl(#include)23 b("mpi.h")75 588 y(main\()g(int)h(argc,)f (char)g(**argv)g(\))75 645 y({)170 701 y(MPI_Comm)g(server;)170 757 y(double)g(buf[MAX_DATA];)170 814 y(char)h (port_name[MAX_NAME_SIZE])o(;)170 927 y(MPI_Init\()f(&argc,)g(&argv)g (\);)170 983 y(strcpy\(port_name,)f(argv[1])h(\);/*)g(assume)g (server's)g(name)g(is)h(cmd-line)f(arg)g(*/)170 1096 y(MPI_Connect\()g(port_name,)f(NULL,)h(NULL,)h(0,)f(MPI_COMM_WORLD,)f (&server)h(\);)170 1209 y(while)h(\(!done\))f({)266 1266 y(tag)g(=)h(2;)g(/*)f(Action)g(to)h(perform)f(*/)266 1322 y(MPI_Send\()g(buf,)g(n,)g(MPI_DOUBLE,)g(0,)g(tag,)h(server)f(\);) 266 1378 y(/*)g(etc)h(*/)266 1435 y(})170 1491 y(MPI_Send\()f(buf,)g (0,)h(MPI_DOUBLE,)e(0,)i(1,)g(server)f(\);)170 1548 y (MPI_Comm_disconnect\()f(&server)g(\);)170 1604 y(MPI_Finalize\(\);)75 1661 y(})166 1767 y Fu(If)17 b(the)g(serv)o(er)f(needs)i(to)e(manage)g (m)o(ultiple)i(connections)g(at)e(once,)h(it)g(can)g(use)g Fp(MPI)p 1675 1767 14 2 v 16 w(IA)o(CCEPT)75 1823 y Fu(instead.)j(The)c (clien)o(t)g(need)g(not)f(b)q(e)h(c)o(hanged.)75 1943 y Fp(Elab)q(o)o(rate)g(client-server)g(example.)45 b Fu(This)16 b(example)h(should)f(sho)o(w)f(a)g(concurren)o(t)h(parallel) h(serv)o(er,)e(in)75 2000 y(whic)o(h)i(a)f(serv)o(er)g(spa)o(wns)g(a)g (set)h(of)f(pro)q(cesses)g(to)g(clone)i(itself)f(to)f(manage)f(the)i (clien)o(t,)h(while)f(it)g(go)q(es)75 2056 y(bac)o(k)j(in)o(to)g (iaccept)h(mo)q(de.)35 b(The)21 b(complicated)g(part)f(is)g(to)g(set)g (up)g(the)h(in)o(tercomm)o(unicator)f(that)75 2113 y(connects)c(the)f (clien)o(t)h(with)g(the)f(clone)h(of)f(the)g(serv)o(er,)f(while)j (getting)e(the)g(original)i(serv)o(er)d(out)h(of)g(the)75 2169 y(w)o(a)o(y)l(.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Apr 22 06:32:54 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA25068; Mon, 22 Apr 1996 06:32:53 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA21689 for mpi-core-out; Mon, 22 Apr 1996 05:35:18 -0500 Received: from gcsin1.gecm.com (gcsin1.gecm.com [194.128.74.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id FAA21684 for ; Mon, 22 Apr 1996 05:35:08 -0500 Received: from gcsin3.geccs.gecm.com by gcsin1.gecm.com; (5.65/1.1.8.2/13Mar95-0121PM) id AA28034; Mon, 22 Apr 1996 11:32:09 +0100 Received: from gcschm.geccs.gecm.com by gcsin3.geccs.gecm.com; (5.65/1.1.8.2/13Mar95-1139AM) id AA22065; Mon, 22 Apr 1996 11:32:08 +0100 Date: Mon, 22 Apr 1996 11:32:05 +0100 From: Bob Madahar Subject: Re: Persistent sendrecv operations vs. persistent channels To: mpi-core@mcs.anl.gov, salo@mrjones.engr.sgi.com Message-Id: <199604221032.LAA09789@oberon.gmrc.gecm.com> Content-Transfer-Encoding: 7BIT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk So far I have not had time to come on line and provide input to the various discussions. However in view of the forthcoming meeting the following should be included: Channels for optimal perfromance An entity called an EVENT channel that could be time trigger count OR SOME OTHER EVENT TO PROMPT PROCESS(ES) INTO ACTION/INACTION Will add details, particularly for realtime a.s.a.p -- Regards Dr. B. K. Madahar | Tel: +44 1245 242262 Chief Scientist | Fax: +44 1245 475244 or 478639 High Performance Computing Communications and Computing Laboratory GEC-Marconi Research Centre Great Baddow Chelmsford, Essex UK CM2 8HN | Email: bob.madahar@gecm.com From mpi-core-human@mcs.anl.gov Mon Apr 29 10:08:07 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA14108; Mon, 29 Apr 1996 10:08:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA02919 for mpi-core-out; Mon, 29 Apr 1996 09:07:23 -0500 Received: from sun2.nsfnet-relay.ac.uk (sun2.nsfnet-relay.ac.uk [128.86.8.45]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA01258 for ; Mon, 29 Apr 1996 06:41:13 -0500 Via: uk.co.iec; Mon, 29 Apr 1996 12:40:02 +0100 Received: from helford.nag.co.uk by nags2.nag.co.uk (4.1/UK-2.1) id AA19521; Mon, 29 Apr 96 12:44:53 BST From: K Vince Fernando Message-Id: <16241.9604291142@helford.nag.co.uk> Received: by helford.nag.co.uk (920330.SGI/UK-2.1) id AA16241; Mon, 29 Apr 96 12:42:17 +0100 Subject: MPI for the uninitiated ? To: mpi-core@antares.mcs.anl.gov Date: Mon, 29 Apr 1996 12:42:16 +0000 (BST) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 400 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk For the uninitiated, MPI is a language in SIAM ! Srinuan Duanghom. An Mpi dictionary / by Srinuan Duanghom ; edited by Woranoot Pantupong. Bangkok : Indigenous Languages of Thailand Research Project, 1976. Vince -- K. V. Fernando Numerical Algorithms Group Ltd Jordan Hill Oxford OX2 8DR UK phone: + 44 1865 511245 fax : + 44 1865 310139 email: vince@nag.co.uk From mpi-core-human@mcs.anl.gov Mon Apr 29 13:45:09 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA18175; Mon, 29 Apr 1996 13:45:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA08898 for mpi-core-out; Mon, 29 Apr 1996 12:47:40 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA08883 for ; Mon, 29 Apr 1996 12:47:23 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id MAA07024; Mon, 29 Apr 1996 12:47:21 -0500 Date: Mon, 29 Apr 1996 12:47:21 -0500 Message-Id: <199604291747.MAA07024@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: new macro for C++ Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As per our discussion at the last meeting, I have created a new macro: \mpic++bind It works like the \mpibind for C. It is for use in defining the C++ binding in the document. Steve From mpi-core-human@mcs.anl.gov Tue Apr 30 09:51:47 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA24753; Tue, 30 Apr 1996 09:51:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA27287 for mpi-core-out; Tue, 30 Apr 1996 08:53:28 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA25962 for ; Tue, 30 Apr 1996 07:10:38 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id HAA29580; Tue, 30 Apr 1996 07:10:37 -0500 Date: Tue, 30 Apr 1996 07:10:37 -0500 Message-Id: <199604301210.HAA29580@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: change markers - Resend Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I sent this out once before but not everyone seems to have gotten it. Sorry if you get it twice. Several people have commented that it is often difficult to tell what has changed from one version of the document to another. To help out I have added macros (in http://www.cs.wisc.edu/~lederman/mpi2/mpi-macs.tex) that can mark the text. They put a symbol in the left margin. The beginning of a change is marked like: --- | (date) and the end of a change like: | (date) --- The macros have the following names. The future ones are for later meetings.: For marking changes not associated with a meeting: \begchange \endchange For marking changes for the June meeting draft \begchangejune \endchangejune For marking changes for the July meeting draft \begchangejuly \endchangejuly For marking changes for the September meeting draft \begchangesept \endchangesept For marking changes for the October meeting draft \begchangeoct \endchangeoct A sample usage for the next meeting would be: \begchangejune This is new text for the June meeting \endchangejune I note that the macros are not perfect. Here are a few annoyances I have seen in my testing: - They fail if used in the middle of some other macros. For example, you can't mark part of a \funcdef. Just mark the whole thing. - The marker can roll over to a new page if the change ends at the bottom of a page. - The text marked in the middle of a paragraph can have the line moved closer to the next line. It is still readable. Let me know if there are problems. Steve From mpi-core-human@mcs.anl.gov Wed May 1 08:37:10 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA27917; Wed, 1 May 1996 08:37:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA19332 for mpi-core-out; Wed, 1 May 1996 07:40:00 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA19301; Wed, 1 May 1996 07:38:32 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id HAA14478; Wed, 1 May 1996 07:38:32 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id HAA03924; Wed, 1 May 1996 07:38:29 -0500 (EST) Date: Wed, 1 May 1996 07:38:29 -0500 (EST) Message-Id: <199605011238.HAA03924@aramis.lsc.nd.edu> To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: How constant are MPI constants? Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In defining the C++ bindings for MPI we would like to maximize use of the const language feature. However, to do so, we need to know (e.g., from implementors) how constants the so-called constants really are. The constants fall into the following categories which are listed below along with our preliminary assessment of their const-ness. Please comment on any of these, but especially on those marked with "???". Return codes -- these should all be const Assorted constants -- these should all be const except MPI_BOTTOM Error handling specifiers -- these should all be const Maximum sizes for strings -- these should all be const ??? Elementary data types -- these should all be const ??? Data types for reduction -- these should all be const ??? Reserved communicators -- not const Results of group and comm comparisons -- these should all be const Environmental inquiry -- these should all be const Collective operations -- these should all be const Null handles -- these should all be const ??? Empty group -- const Topologies -- these should all be const Cheers, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Wed May 1 08:49:46 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA28036; Wed, 1 May 1996 08:49:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA19477 for mpi-core-out; Wed, 1 May 1996 07:52:50 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA19460; Wed, 1 May 1996 07:52:24 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id HAA14498; Wed, 1 May 1996 07:52:24 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id HAA03934; Wed, 1 May 1996 07:52:21 -0500 (EST) Date: Wed, 1 May 1996 07:52:21 -0500 (EST) Message-Id: <199605011252.HAA03934@aramis.lsc.nd.edu> To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Mixed language inter-operability Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk An issue came up during the bindings discussion at the recent MPI Forum meeting about mixed language inter-operability. In particular, it was suggested that it might be useful to be able to re-use C structures as C++ classes -- implying that the C++ classes defined for the C++ bindings could not have virtual member functions. There are two issues regarding mixed-language programming that need to be addressed, namely the user interface (the MPI API) aspect as well as the implementor aspect. For the API issue, if I understand the discussion from the meeting correctly, what is desired is to be able to call C MPI-based library functions directly from a C++ program. For example, a C++ program might look like ... MPI_Comm a(MPI_COMM_WORLD); CLib_call(a, arg1, arg2, arg3); ... Here, CLib_call is assumed to be a library function, written in C, that uses the C bindings for MPI -- with the MPI_comm that is passed in. It seems to me that we cannot mandate that this must work. The C++ bindings do mandate that MPI_Comm (for instance) be an object, but the C bindings only mandate that MPI_Comm be a handle. If that handle were guaranteed to be a pointer to a structure, then the above could be mandated, but handles, even in C, may be integers -- as is the case with the IBM implementation of MPI. Therefore, we have the following alternative proposal. The C++ MPI classes must have a member function that returns the equivalent C handle. Similarly, it should have a member function that returns the equivalent Fortran handle (I gag as I write this). The example above would thus look like ... MPI_Comm a(MPI_COMM_WORLD); CLib_call(a.C_handle, arg1, arg2, arg3); ... Now, the second issue has to do with implementing the C++ bindings. Here, again it would be nice for structure and classes to be easily translated so that code can be reused and so forth. A better approach, IMHO, would be to perhaps derive the C++ classes from whatever internal C structures may be lurking about. Among other things, the C_handle() member function above would be trivially realizable in many cases. More importantly, the C++ bindings could be realized as a thin layer on top of the C bindings or on top of whatever the C bindings are layered on top of. Cheers, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Wed May 1 08:52:42 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA28070; Wed, 1 May 1996 08:52:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA19511 for mpi-core-out; Wed, 1 May 1996 07:55:47 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA19495; Wed, 1 May 1996 07:55:20 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id HAA14509; Wed, 1 May 1996 07:55:19 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id HAA03939; Wed, 1 May 1996 07:55:16 -0500 (EST) Date: Wed, 1 May 1996 07:55:16 -0500 (EST) Message-Id: <199605011255.HAA03939@aramis.lsc.nd.edu> To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Reduction operations Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk An objection to including MPI_*COMPLEX and MPI_BOOL datatypes was raised at the recent MPI Forum meeting because many reduction operations are not well defined for these types of data. This issue has already been addressed with respect to Fortran, which also included complex and boolean (logical) data types. See page 115 of the MPI-1 spec. I recommend that we similarly specify which reduction operations can be legally used with which C++ data types. Best Regards, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Wed May 1 09:18:55 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA28313; Wed, 1 May 1996 09:18:50 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA19901 for mpi-core-out; Wed, 1 May 1996 08:21:30 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA19896; Wed, 1 May 1996 08:21:23 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id IAA20330; Wed, 1 May 1996 08:21:21 -0500 Date: Wed, 1 May 1996 08:21:21 -0500 Message-Id: <199605011321.IAA20330@romano.cs.wisc.edu> From: Steve Huss-Lederman To: Andrew.Lumsdaine@nd.edu CC: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <199605011255.HAA03939@aramis.lsc.nd.edu> (Andrew.Lumsdaine@nd.edu) Subject: Re: Reduction operations Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Andrew, I believe the concern is that we not define what the MPI_MAX/MIN of a complex variable be. We removed this in the 1.1 spec (June 12, 1995, p. 116, line 23). The statement at the meeting (maybe the one I said) may have been too general for what I was really concerned about. Steve > > > An objection to including MPI_*COMPLEX and MPI_BOOL datatypes was > raised at the recent MPI Forum meeting because many reduction > operations are not well defined for these types of data. > > This issue has already been addressed with respect to Fortran, which > also included complex and boolean (logical) data types. See page 115 > of the MPI-1 spec. I recommend that we similarly specify which > reduction operations can be legally used with which C++ data types. > > Best Regards, > Andrew Lumsdaine > From mpi-core-human@mcs.anl.gov Wed May 1 09:26:55 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA28392; Wed, 1 May 1996 09:26:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA20060 for mpi-core-out; Wed, 1 May 1996 08:30:08 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA20044; Wed, 1 May 1996 08:29:44 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id IAA14622; Wed, 1 May 1996 08:29:43 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id IAA04037; Wed, 1 May 1996 08:29:42 -0500 (EST) Date: Wed, 1 May 1996 08:29:42 -0500 (EST) Message-Id: <199605011329.IAA04037@aramis.lsc.nd.edu> To: lederman@cs.wisc.edu CC: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <199605011321.IAA20330@romano.cs.wisc.edu> (message from Steve Huss-Lederman on Wed, 1 May 1996 08:21:21 -0500) Subject: Re: Reduction operations Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode Steve Huss-Lederman wrote: > I believe the concern is that we not define what the MPI_MAX/MIN of a > complex variable be. We removed this in the 1.1 spec (June 12, 1995, > p. 116, line 23). The statement at the meeting (maybe the one I said) > may have been too general for what I was really concerned about. Right. Since, in the abstract, complex numbers cannot be ordered, it is erroneous to define max/min of them. Thus, we just have to specify how complex numbers are handled in reductions. My previous reference to page 115 is the one you indicate here, I believe (for some reason, the copy of the June 12 doc I have does not seem to be paginated correctly). Are we saying the same thing? Regards, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Wed May 1 09:52:10 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA28847; Wed, 1 May 1996 09:52:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA20529 for mpi-core-out; Wed, 1 May 1996 08:54:02 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA20504; Wed, 1 May 1996 08:53:24 -0500 Message-Id: <199605011353.IAA20504@antares.mcs.anl.gov> To: Andrew.Lumsdaine@nd.edu cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: How constant are MPI constants? In-Reply-to: Message from Andrew.Lumsdaine@nd.edu of "Wed, 01 May 1996 07:38:29 -0500." <199605011238.HAA03924@aramis.lsc.nd.edu> Date: Wed, 01 May 1996 08:53:23 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Ergh. Constants. The original (1.0) definition meant "unchanging between MPI_Init and MPI_Finalize". In the C sense, this did not imply const, since MPI_Init could set the values. In 1.1 this was strengthened to be "usable as compile-time initializers"; this allowed using "address of object" in C, and constrained implementations in Fortran (basically forcing PARAMETER). The language interoperability may further constrain implementations by mandating the most restrictive (Fortran) form. Because of this, it is probably premature to decide the "const" status of the MPI "constants" until after the language interoperability section is through its first real vote. Having said that, It seems likely that everything (including predefined communicators) except for MPI_BOTTOM will be const. Some implementations will have to change, but this is independent of the C++ binding. I'd be interested in if the language interoperability section will impact the C++ implementations more than it already does the C implementations. Bill From mpi-core-human@mcs.anl.gov Wed May 1 12:40:48 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA00306; Wed, 1 May 1996 12:40:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA26466 for mpi-core-out; Wed, 1 May 1996 11:42:06 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA26351; Wed, 1 May 1996 11:38:59 -0500 Received: from jim (gw2-173.pool.dircon.co.uk) by felix.dircon.co.uk with SMTP id AA18148 (5.67b/IDA-1.5); Wed, 1 May 1996 17:38:35 +0100 Message-Id: <199605011638.AA18148@felix.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id RAA00547; Wed, 1 May 1996 17:39:39 +0100 To: William Gropp Cc: Andrew.Lumsdaine@nd.edu, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: How constant are MPI constants? In-Reply-To: Your message of "Wed, 01 May 1996 08:53:23 CDT." <199605011353.IAA20504@antares.mcs.anl.gov> Date: Wed, 01 May 1996 17:39:38 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk If the values are truly constant enumerated integers (and some, at least are surely just that), then the best thing to do may well be to make them into anonymous enums. This is what I would recommend for all the constant numeric values which the C binding defines using #define. The benefits of this are 1) It makes them just as constant as #define did, but without any of the dangers that #defines have. (e.g. #define foo 1 ... struct foo { }; ) 2) Compilers often generate better code for enums than they do for const int, because they know that an enum really is meant to be a literal constant. Often they will generate an initialised store location for a const int in case you ever take the address of it. (Sometimes one such location for each constant in each file...) 3) enums are silently cast to ints even in C++ -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Wed May 1 12:44:19 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA00336; Wed, 1 May 1996 12:44:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA26644 for mpi-core-out; Wed, 1 May 1996 11:47:12 -0500 Received: from ns.ge.com (ns.ge.com [192.35.39.24]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA26614; Wed, 1 May 1996 11:46:39 -0500 Received: from gein.ae.ge.com ([3.34.221.213]) by ns.ge.com (8.7.5/8.7.3) with SMTP id MAA10648; Wed, 1 May 1996 12:46:33 -0400 Received: from ss09.ae.ge.com (ss09.ae.ge.com [3.59.160.2]) by gein.ae.ge.com (8.6.11/8.6.11) with ESMTP id MAA15678; Wed, 1 May 1996 12:45:52 -0400 Received: from h0006.ae.ge.com by ss09.ae.ge.com with ESMTP (1.37.109.16/15.6) id AA145869184; Wed, 1 May 1996 12:46:25 -0400 Received: from h0006 (localhost) by h0006.ae.ge.com with SMTP (1.37.109.16/16.2) id AA087219182; Wed, 1 May 1996 12:46:22 -0400 Message-Id: <3187955D.34D6@ae.ge.com> Date: Wed, 01 May 1996 12:46:21 -0400 From: "Craig J. Fischberg" Organization: GE Aircraft Engines: Info. Technology Services X-Mailer: Mozilla 2.0 (X11; I; HP-UX A.09.05 9000/720) Mime-Version: 1.0 To: gropp@mcs.anl.gov Cc: Andrew.Lumsdaine@nd.edu, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: How constant are MPI constants? References: <199605011353.IAA20504@antares.mcs.anl.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk William Gropp wrote: > > Ergh. Constants. The original (1.0) definition meant "unchanging between > MPI_Init and MPI_Finalize". In the C sense, this did not imply const, since > MPI_Init could set the values. In 1.1 this was strengthened to be "usable as > compile-time initializers"; this allowed using "address of object" in C, and > constrained implementations in Fortran (basically forcing PARAMETER). The > language interoperability may further constrain implementations by mandating > the most restrictive (Fortran) form. Because of this, it is probably > premature to decide the "const" status of the MPI "constants" until after the > language interoperability section is through its first real vote. > > Having said that, It seems likely that everything (including predefined > communicators) except for MPI_BOTTOM will be const. Some implementations will > have to change, but this is independent of the C++ binding. > > I'd be interested in if the language interoperability section will impact the > C++ implementations more than it already does the C implementations. > > Bill As both an imlementor and end user I will make a suggestion that will probably make most of you cringe. I would have gone even further with the definition of constants under MPI-1. I would have liked to see the actual values (i.e. integers) for the constants defined in the standard. And, I would like to say that no MPI function could be defined as a macro. Why? So that the particular mpi library implementation could be loaded at run-time using shared-object libraries without having to re-compile the code. For example, if my job is to be wholly contained within an SGI I would set-up the environment to load the SGI optimized shared-library version of MPI for use within a single box. If my job is to be split between an SGI and several other boxes, I could set up my environment to load shared object versions of MPICH or CHIMP at run time. In this way, I can use the same binary of my application and load the best library implementation for the particular job that I am running. It is too late to consider something like this. However, I figured it was worth having people think about it. -- ==================================================================== Craig J Fischberg E-mail: Craig.Fischberg@ae.ge.com GE Aircraft Engines Phone: 513-552-2716 One Neumann Way M/D T207 Fax: 513-552-3153 Cincinnati, OH 45215-6301 ==================================================================== From mpi-core-human@mcs.anl.gov Wed May 1 16:22:13 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA04733; Wed, 1 May 1996 16:22:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA02639 for mpi-core-out; Wed, 1 May 1996 15:24:26 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA02626; Wed, 1 May 1996 15:24:16 -0500 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) id NAA06371; Wed, 1 May 1996 13:24:13 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) id NAA04434; Wed, 1 May 1996 13:24:11 -0700 From: "Eric Salo" Message-Id: <9605011324.ZM4432@mrjones.engr.sgi.com> Date: Wed, 1 May 1996 13:24:11 -0700 In-Reply-To: Andrew.Lumsdaine@nd.edu "How constant are MPI constants?" (May 1, 7:38am) References: <199605011238.HAA03924@aramis.lsc.nd.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: How constant are MPI constants? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > The constants fall into the following categories which are listed > below along with our preliminary assessment of their const-ness. > Please comment on any of these, but especially on those marked with In our implementation, *all* constants are constant, including MPI_BOTTOM. All opaque types (pre-defined or not) are implemented as indices into a table, even in C, so I don't see how to make them into enums as Jim suggests. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Wed May 1 16:45:44 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA05003; Wed, 1 May 1996 16:45:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA03321 for mpi-core-out; Wed, 1 May 1996 15:47:01 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA03298; Wed, 1 May 1996 15:46:23 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id PAA19293; Wed, 1 May 1996 15:46:22 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id PAA05104; Wed, 1 May 1996 15:46:19 -0500 (EST) Date: Wed, 1 May 1996 15:46:19 -0500 (EST) Message-Id: <199605012046.PAA05104@aramis.lsc.nd.edu> To: salo@mrjones.engr.sgi.com CC: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <9605011324.ZM4432@mrjones.engr.sgi.com> (message from Eric Salo on Wed, 1 May 1996 13:24:11 -0700) Subject: Re: How constant are MPI constants? Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode Eric Salo wrote: Precedence: bulk > In our implementation, *all* constants are constant, including > MPI_BOTTOM. All opaque types (pre-defined or not) are implemented as > indices into a table, even in C, so I don't see how to make them > into enums as Jim suggests. Even anonymous enums can be given explicit values, can they not? In C++, we would want the pre-defined opaque types to be const objects. The question then is: is this okay? Are the pre-defined opaque objects constant enough to be able to be declared const? This is a semantic as well as a practical issue. Best Wishes, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Thu May 2 08:24:24 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA15759; Thu, 2 May 1996 08:24:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA16565 for mpi-core-out; Thu, 2 May 1996 07:27:04 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id HAA16554; Thu, 2 May 1996 07:26:35 -0500 Received: from jim ([192.128.227.172]) by felix.dircon.co.uk with SMTP id AA03542 (5.67b/IDA-1.5); Thu, 2 May 1996 11:25:30 +0100 Message-Id: <199605021025.AA03542@felix.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id JAA00546; Thu, 2 May 1996 09:52:03 +0100 To: "Eric Salo" Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: How constant are MPI constants? In-Reply-To: Your message of "Wed, 01 May 1996 13:24:11 PDT." <9605011324.ZM4432@mrjones.engr.sgi.com> Date: Thu, 02 May 1996 09:52:02 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Eric wrote > In our implementation, *all* constants are constant, including > MPI_BOTTOM. All opaque types (pre-defined or not) are implemented as > indices into a table, even in C, so I don't see how to make them > into enums as Jim suggests. So either they look like enum /* Anonymous since the type has no name */ { MPI_int = 0, MPI_real= 1, ... }; if the user sees the integer values, or they look like #define MPI_int &(__mpi_type_table[0]) #define MPI_real &(__mpi_type_table[1]) etc. If the former case you can use enums (as above), since the constants are integers. (as I said > If the values are truly constant enumerated integers ******** ), otherwise you can't. However for the types, for instance, even if they'r not enumerated integers then they should be const. For other things this may not be true for the semantic reasons that Andrew is pointing out. -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Fri May 3 18:55:54 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA19956; Fri, 3 May 1996 18:50:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA22779 for mpi-core-out; Fri, 3 May 1996 17:51:36 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA22754; Fri, 3 May 1996 17:48:40 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id PAA20972; Fri, 3 May 1996 15:48:38 -0700 Date: Fri, 3 May 1996 15:48:38 -0700 Message-Id: <199605032248.PAA20972@win233.nas.nasa.gov> From: "William C. Saphir" To: mpi-dynamic@mcs.anl.gov Cc: mpi-core@mcs.anl.gov Subject: new proposal for spawning processes Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, Enclosed is a new proposal for spawning processes from MPI. This proposal is nearly identical in functionality to the proposal of the last meeting, but provides a much simpler and cleaner interface. The key idea (separating spawn and attach) was proposed by Eric Salo, and the proposal has since been revised and polished after substantial commentary from Raja, Al, Greg, Nick, Tony, and myself. Here are the core ideas: 0. There are three basic functions: MPI_Spawn, MPI_Child_attach and MPI_Child_detach. 1. MPI_Spawn() returns a group. It spawns one command line at a time. You specify an MPI or NONMPI flag. 2. If you specified MPI, then you subsequently do MPI_Child_attach() (for MPI_Spawn functionality) or MPI_Child_detach() (for MPI_Spawn_independent functionality). 3. To get the _MULTIPLE functionality, you may manipulate (merge, split, etc) groups before attach/detach. We propose that this proposal become the basis for future discussion of SPAWN_* functionality. Please read it and send your comments ASAP to the mpi-dynamic list, even if you concur, so that we can determine if there is enough support to proceed in this direction. It is important to determine this quickly so that we do not delay the already tight schedule. This draft includes a clarification of the issues discussed at the last meeting. In that sense, it is a full proposal. I am aware of a number of potential amendments floating in the wings. We should consider those after we have established whether this can be a working document. Those amendments include 1. a proposal to eliminate minprocs 2. a proposal to replace the info string with an info handle 3. a proposal to eliminate the mpi/nonmpi flag 4. a proposal to retain the mpi/nonmpi flag but specify what happens when a "nonmpi" process calls mpi_init(). 5. possibly a proposal to add nonblocking versions of the attach and detach functions There are a few subtle differences in functionality between this proposal and the previous, but no major ones. This is an attempt to list all of them, so there won't be any surprises. In some rather vigorous discussion over the last couple of days, all of us have agreed that these are not serious problems. - May not get same efficiency when spawning multiple executables. [For the speed of the spawn this is probably not a problem since ispawn() can allow overlap. Communication is still established among all processes, collectively, so in principle they should be able to choose the best communication. ] - Two phase spawn/detach constrains implementations a bit more than the current proposal single routines. - Spawn/detach synchronizes with mpi_init. Previously, successful spawn_independent did not indicate that processes had called MPI_INIT. - Spawn/attach requires intermediate group object where previous spawn did not. This makes the simple case of single app talking to its parent slightly more complicated, though arguably more natural. MPI_Spawn("foo", count, count, NULL, 0, MPI_COMM_SELF, &family, NULL); becomes MPI_Group g; MPI_Spawn("foo", count, count, MPI_MPI, NULL, &g, NULL); MPI_Child_attach(g, 0, MPI_COMM_SELF, &family); - Errors become much more straightforward. The spawn stage is clearly separate from communication establishment phase. Errors in the communication establishment phase are usual MPI errors. Errors in the spawn stage are tolerable. This is potentially good and bad. Before, softness was in the combined spawn/attach cycle. So if we spawned 6, but only 5 participated in communication, the implementation could theoretically recover, but there might be side effects from the extra one that was started. In this new proposal, once they are started, they are started. If 6 processes don't show up for the child_attach, there is a hard error (no softness in attach). Similarly, if we spawn 6 processes and then detach, if only 5 show up it is an error. - There is a non-blocking spawn, but no non-blocking attach/detach. Since the processes have already started, an attach or detach is not substantially different from an MPI-1 collective operation. The non-blocking spawn is necessary because it might be a very expensive operation (e.g., submitting a request to a batch scheduler). Bill ================================================================ %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: main.dvi %%Pages: 15 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: /usr/local/bin/dvips -o main.ps main.dvi %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.05.03:1528 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (main.dvi) @start /Fa 11 121 df46 D<00300000700001F0000FF000 FFF000FFF000F1F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F000FFFFE0FFFFE0FFFFE0 13287BA71D>49 D<007E0003FF8007FFE01FFFF03F83F83E00F87C007C38007C30007C10 007C00007C00007C0000F80000F80001F80003F0000FE001FFC001FF8001FFC001FFE000 01F80000FC00007C00003E00003E00001F00001F00001F00001F00001F80001FC0003EE0 003EF0007E7C00FC3F01F81FFFF00FFFE003FFC000FE0018297EA71D>51 D69 D<01FC000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F80000F80000F80001 F800FFF807FFF81FFFF83FC0F87E00F8FC00F8F800F8F800F8F800F8FC01F87E07F87FFF F83FFFF81FFCF80FE0F8151B7E9A1C>97 D<00FC0003FF0007FF800FFFC01F87E03F01E0 3E00F07C00F07C0070F80078FFFFF8FFFFF8FFFFF8FFFFF8F80000F80000F80000780000 7C00007C00003E00003F00181FC0F80FFFF807FFF801FFE0007F00151B7E9A1A>101 D108 DI112 D<07F8001FFF003FFF807FFF807C0780F80100F80000F80000F80000FC00007F00007FF0 003FFC001FFE0007FF0000FF80001FC0000FC00007C00007C04007C0E007C0FC1F80FFFF 807FFF001FFE0007F800121B7E9A16>115 D<7C001F807E003F003F003E001F007C000F 80FC0007C0F80007E1F00003E3E00001F7E00000FFC000007F8000007F0000003E000000 3E0000003F0000007F800000FFC00001F3E00003E1E00003E1F00007C0F8000F807C001F 807C001F003E003E001F007E001F80FC000FC01A1B809A1B>120 D E /Fb 2 110 df<000001C00000078000001E00000078000001E00000078000000E00 000038000000F0000003C000000F0000003C000000F0000000F00000003C0000000F0000 0003C0000000F0000000380000000E0000000780000001E0000000780000001E00000007 80000001C01A1A7C9723>60 D<1E07C07C00231861860023A032030043C0340300438038 0380438038038087007007000700700700070070070007007007000E00E00E000E00E00E 000E00E00E000E00E01C101C01C01C201C01C038201C01C038401C01C018403803801880 1801800F0024147E9328>109 D E /Fc 2 21 df<03C00FF01FF83FFC7FFE7FFEFFFFFF FFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D<000000C0000003C000000F0000003C000000F0000003C00000070000001C0000007800 0001E00000078000001E00000078000000E0000000780000001E0000000780000001E000 0000780000001C0000000700000003C0000000F00000003C0000000F00000003C0000000 C0000000000000000000000000000000000000000000000000000000007FFFFF80FFFFFF C01A247C9C23>20 D E /Fd 10 58 df<1F00318060C04040C060C060C060C060C060C0 60C060C060404060C031801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C 000C000C000C000C000C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060 006000C00180030006000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000 C0008001800F00008000400060C060C060804060801F000B107F8F0F>I<030003000700 0F000B001300330023004300C300FFE003000300030003001FE00B107F8F0F>I<20803F 002C002000200020002F0030802040006000600060C06080C061801F000B107F8F0F>I< 0780184030C060C06000C000CF00F080E040C060C060C060406060C030801F000B107F8F 0F>I<40007FE07FC08080808001000200040004000C0008000800180018001800180018 000B117E900F>I<1F00318060C060C060C071803F000F00338061C0C060C060C0604040 60801F000B107F8F0F>I<1F00318060C0C040C060C060C06040E021E01E600060004060 C0608043003E000B107F8F0F>I E /Fe 18 119 df<70F8F8F0E005057B840E>46 D<00000200000006000000060000000E0000001E0000001E0000003F0000002F0000004F 0000004F0000008F0000010F0000010F0000020F0000020F0000040F00000C0F0000080F 0000100F0000100F0000200F80003FFF800040078000C007800080078001000780010007 800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C0004003C00 040078080800780800007808000078180000F0300000FFF00000F0300000F0300001E020 0001E0200001E0200001E0001003C0002003C0002003C0004003C0004007800080078001 8007800100078007000F001F00FFFFFE001F1F7D9E1F>69 D<01FFFF00001E03C0001E00 E0001E0070001E0078003C0078003C0078003C0078003C0078007800F0007800F0007801 E0007801C000F0070000F01E0000FFF00000F0380001E01C0001E01E0001E00E0001E00F 0003C01E0003C01E0003C01E0003C01E0007803C0007803C0807803C0807803C100F801C 10FFF00C20000007C01D207D9E21>82 D<00F1800389C00707800E03801C03803C038038 0700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C4030 8C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C07003C02007800 00780000780000F00000F00000F00000F00000F0000070010070020030040018380007C0 0011147C9315>99 D<0000780003F80000700000700000700000700000E00000E00000E0 0000E00001C00001C000F1C00389C00707800E03801C03803C0380380700780700780700 780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F070015207C 9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000F000F00070007001 70023804183807C010147C9315>I<00007800019C00033C00033C000718000700000700 000E00000E00000E00000E00000E0001FFE0001C00001C00001C00001C00003800003800 00380000380000380000700000700000700000700000700000700000E00000E00000E000 00E00000C00001C00001C0000180003180007B0000F300006600003C00001629829F0E> I<00C001E001E001C000000000000000000000000000000E003300230043804300470087 000E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E>105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C0 1C01C01C01C01C03843803883803083807083803107003303001C016147C931A>110 D<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003 C0F003C0F00780F00700700F00700E0030180018700007C00013147C9317>I<1C1E0026 61004783804787804707804703008E00000E00000E00000E00001C00001C00001C00001C 000038000038000038000038000070000030000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E000E700EF00CF00CE00840 1020601F8010147D9313>I<018001C0038003800380038007000700FFF007000E000E00 0E000E001C001C001C001C003800380038003820704070407080708031001E000C1C7C9B 0F>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E03801C0700 1C07001C07001C07101C0E20180E20180E201C1E200C264007C38014147C9318>I<0E03 803307802307C04383C04301C04700C08700800E00800E00800E00801C01001C01001C01 001C02001C02001C04001C04001C08000E300003C00012147C9315>I E /Ff 83 127 df32 D<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F870051C779B18>I< 4010E038F078E038E038E038E038E038E038E038E038E038E03860300D0E7B9C18>I<03 0600078F00078F00078F00078F00078F00078F007FFFC0FFFFE0FFFFE07FFFC00F1E000F 1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFFC01E3C001E3C001E3C001E 3C001E3C001E3C000C1800131C7E9B18>I<00C00001C00001C00001C00003F0000FFC00 3FFE007DCF0071C700E1C380E1C780E1C780E1C780F1C00079C0003DC0001FE0000FF800 03FC0001DE0001CF0001C70061C380F1C380F1C380E1C380E1C70071C70079DE003FFE00 1FF80007E00001C00001C00001C00000C00011247D9F18>I<3803007C07807C0780EE0F 80EE0F00EE0F00EE1F00EE1E00EE1E00EE3E007C3C007C3C00387C0000780000780000F8 0000F00001F00001E00001E00003E00003C00003C00007C0000783800787C00F87C00F0E E00F0EE01F0EE01E0EE01E0EE03E0EE03C07C03C07C018038013247E9F18>I<01C00007 E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3800FC7000F 87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E0787070FC707F FFE03FCFE00F03C0141C7F9B18>I<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18>I<00 7000F001E003C007800F001E001C00380038007000700070007000E000E000E000E000E0 00E000E000E0007000700070007000380038001C001E000F00078003C001F000F000700C 24799F18>I<6000F00078003C001E000F000780038001C001C000E000E000E000E00070 007000700070007000700070007000E000E000E000E001C001C0038007800F001E003C00 7800F00060000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF 001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C0 0011147D9718>I<00600000F00000F00000F00000F00000F00000F00000F0007FFFC0FF FFE0FFFFE07FFFC000F00000F00000F00000F00000F00000F00000F00000600013147E97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D 8F18>I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00001E 00001E00003E00003C00007C0000780000780000F80000F00001F00001E00003E00003C0 0003C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C00007C00 00780000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F001C 07003803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E0 00E0E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F00013 1C7E9B18>I<01800380038007800F803F80FF80FB804380038003800380038003800380 03800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03F000 0FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C00001C0 0003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C00E0 7FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C07801C03001 C00001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C00000E00000 E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<00 1F00003F0000770000770000E70001E70001C7000387000787000707000E07001E07003C 0700380700780700F00700FFFFF8FFFFF8FFFFF800070000070000070000070000070000 0700007FF000FFF8007FF0151C7F9B18>I<3078FCFC783000000000000000003078FCFC 78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060 071A789318>I<000300000780001F80003F00007E0001FC0003F00007E0001FC0003F00 007E0000FC0000FC00007E00003F00001FC00007E00003F00001FC00007E00003F00001F 8000078000030011187D9918>I<7FFFC0FFFFE0FFFFE0FFFFE000000000000000000000 0000FFFFE0FFFFE0FFFFE07FFFC0130C7E9318>I<600000F00000FC00007E00003F0000 1FC00007E00003F00001FC00007E00003F00001F80001F80003F00007E0001FC0003F000 07E0001FC0003F00007E0000FC0000F0000060000011187D9918>I<00700000F80000F8 0000D80000D80001DC0001DC0001DC00018C00038E00038E00038E00038E000306000707 000707000707000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8F F87F07F0151C7F9B18>65 DI<00F8E003FE E007FFE00F07E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E000 00E00000E00000E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF 8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C 00E01C00E01C00F01C00701C00701C00701C00701C00701C00701C00701C00701C00F01C 00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01 F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E00000E0 0000E00000E00000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F 0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C0 1C01C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C0 1C01C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18> I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C0007FFF00FFFF807FFF00111C7D9B18>I<7FE000FFE0007FE0000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B 18>76 DI<7E07F0FF0FF87F07F01D81C01D 81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C 39C01C39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C015 1C7F9B18>I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 F00780700700780F007FFF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E1E380E1 E380F0E78070F700787F007FFF003FFE000FFC00001C00001E00000E00000F0000070000 070011227D9B18>I<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C0 1C03C01C03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C0380 1C03801C039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>I<03F3801FFF803FFF 807C0F80700780E00380E00380E00380E000007000007800003F00001FF00007FE0000FF 00000F800003C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE 00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E0703800700000 700000700000700000700000700000700000700000700000700000700000700000700000 700000700000700000700000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00 039C0003DC0001F80001F80000F00000F00000700000F00000F80001F80001DC00039E00 038E00070F000707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C070078070 0700000F00001E00001C00003C0000780000700000F00001E00001C00003C00007800007 00000F00001E00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>I< FFF8FFF8FFF8E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000FFF8FFF8FFF8 0D24779F18>I<600000F00000F00000F800007800007C00003C00003C00003E00001E00 001F00000F00000F00000F800007800007C00003C00003C00003E00001E00001F00000F0 0000F800007800007800007C00003C00003E00001E00001E00001F00000F00000F800007 8000078000030011247D9F18>II<7FFF00FFFF80FFFF807FFF0011047D 7F18>95 D<061E3E387070E0E0E0F8FC7C7C38070E789E18>I<1FE0003FF8007FFC0078 1E00300E0000070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F0 0F00781F003FFFF01FFBF007E1F014147D9318>I<7E0000FE00007E00000E00000E0000 0E00000E00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E0038 0E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C80 9B18>I<01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E000 00E00000E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<00 1F80003F80001F8000038000038000038000038000038003E3800FFB801FFF803C1F8038 0F80700780700380E00380E00380E00380E00380E00380E00380700780700780380F803C 1F801FFFF00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F00380780700380 700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF80 07FF0001FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001C0 007FFFC0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007 FFF80FFFF81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC001F F80039E0003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E0 00387800F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E0000 0E00000E00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC 171C809B18>I<03800007C00007C00007C0000380000000000000000000000000007FC0 00FFC0007FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E0 00FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF 807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC 1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000 E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318 >I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E 00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E 00000E00000E00000E00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF80 3E1F80380F80700780700780E00380E00380E00380E00380E00380E00380700780700780 380F803C1F801FFF800FFB8003E380000380000380000380000380000380000380000380 003FF8003FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C0 0003C0000380000380000380000380000380000380000380000380000380007FFE00FFFF 007FFE0015147F9318>I<07F7003FFF007FFF00780F00E00700E00700E007007C00007F E0001FFC0003FE00001F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011 147D9318>I<0180000380000380000380000380007FFFC0FFFFC0FFFFC0038000038000 0380000380000380000380000380000380000380000380400380E00380E00380E001C1C0 01FFC000FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FF FE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E038007070007 0700070700038E00038E00038E00038E0001DC0001DC0001DC0000F80000F80000700015 147F9318>I I<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F80000F80000700000F00000F8 0001DC00039E00038E000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF 8FF87F8FF00E01C00E03800E0380070380070700070700038700038600038E0001CE0001 CE0000CC0000CC0000DC0000780000780000780000700000700000700000F00000E00079 E0007BC0007F80003F00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0 700780000F00001E00003C0000F80001F00003C0000780000F00701E00703C0070780070 FFFFF0FFFFF0FFFFF014147F9318>I<0007E0001FE0007FE000780000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00001E0007FC000FF8000FF80 007FC00001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E000007800007FE0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000F00000 7FC0003FE0003FE0007FC000F00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00003C000FFC000FF00007C000013247E9F18>125 D<060C1F1E3FBEFBF8F1F060C00F067C9B18>I E /Fg 49 122 df<00F8F001F8F003F8 F00780F00700F00F00F00F00F00F00F00F00F00F00F00F00F00F00F0FFF8F0FFF8F0FFF8 F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00 F00F00F00F00F00F00F00F00F00F00F01420809F18>13 D<007000E001C0038007800700 0E001E001E003C003C003C0078007800780078007000F000F000F000F000F000F000F000 F000F000F000F000F000700078007800780078003C003C003C001E001E000E0007000780 038001C000E000700C2E7EA112>40 DI<787878781830306060E0050A7D830D>44 D II<001F0000001F0000003F8000003B8000003B8000007BC000 0073C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0F00003C07800 03C078000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F00 1E000F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B207F9F 1E>65 D<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C00007800 00780000780000F00000F00000F00000F00000F00000F00000F00000F000007800007800 007800007C00003C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC01722 7DA01D>67 D III<001FE000FFF801FFFE 03E03E07800E0F00001E00003E00003C00007C0000780000780000780000F00000F00000 F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E 1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>III76 DII<003F000000FFC00003FFF00007E1F8000F807C001F003E001E001E00 3C000F003C000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F80007C0780007807800078078000780 3C000F003C000F001E001E001F003E000F807C0007E1F80003FFF00000FFC000003F0000 1A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C00 003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001 E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019 >IIIII89 D<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7F FF3FEF1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000F0007800 78003C013E0F1FFF0FFE03F010147E9314>I<0007800007800007800007800007800007 8000078000078000078000078000078000078007C7800FF7801FFF803E1F807C07807807 80F80780F00780F00780F00780F00780F00780F00780F00780780780780F803E1F801FFF 800FF78007C78011207E9F17>I<03F0000FFC001FFE003E1F003C0700780700700380FF FF80FFFF80FFFF80F00000F00000F000007000007800003C01003E07001FFF0007FE0001 F80011147F9314>I<007E01FE03FE078007000F000F000F000F000F000F000F00FFF0FF F0FFF00F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F 000F20809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00780F00780F00780F00780F0078 0F003C1E003E3E001FFC003FF80033E0003000003800003FFE003FFF801FFFC03FFFE078 03F0F000F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00141E7F9317>I 105 D107 DIII<01F80007FE00 1FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000F07801E0 7801E03C03C03F0FC01FFF8007FE0001F80014147F9317>II<03C7800FF7801FFF803E1F807C0F80780780780780F00780F0 0780F00780F00780F00780F00780F807807807807C0F803E1F801FFF800FF78007C78000 0780000780000780000780000780000780000780000780000780111D7E9317>II<07F01FFC3FFC780C7800780078007C003FC01FF00FF803F8007C003C 003CC03CF07CFFF87FF00FC00E147F9311>I<1E001E001E001E001E001E00FFF0FFF0FF F01E001E001E001E001E001E001E001E001E001E001E001E001E001E201FF00FF007C00C 1A7F9910>IIII<7801E07C03C03E0780 1E0F000F0F00079E0003FC0003F80001F80000F00001F00001F80003FC00079E000F0F00 0E0F001E07803C03C07801E0F801F01414809315>II E /Fh 19 90 df<3078F8787005057C840D>46 D<000C001C00FC0F380038003800380038003800700070007000700070007000E000E000 E000E000E000E001C001C001C001C001C001C0038003C0FFFE0F1E7C9D17>49 D<007F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E00003C00003 80000700000E0000F800000E000007000007800007C00003C00007C03007C07807C0F807 C0F807C0F00780800F00400E00201C0018780007E000141F7D9D17>51 D<0000100000001800000038000000380000007800000078000000FC000001BC0000013C 0000033C0000023C0000063C0000043E0000081E0000081E0000101E0000101E0000201E 0000200F0000400F0000400F0000FFFF0000800F0001000F800100078002000780020007 8004000780040007800C0007C03E0007C0FF807FFC1E207E9F22>65 D<0001F808000E061800380138007000F801E0007803C0007007800030078000300F0000 301F0000301E0000303E0000203C0000007C0000007C0000007C0000007C000000F80000 00F8000000F8000000F8000000F80000007800004078000080780000803C0000803C0001 001C0002000E00020006000C000300100001C0E000003F00001D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E007800 1E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E01E000 3E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C003 C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038 003C001800780018007800080078000800780008007800080078080800F0100000F01000 00F0100000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001E00008 01E0001003C0001003C0001003C0002003C0002003C0006003C000C0078001C0078007C0 FFFFFF801D1F7E9E1F>I<07FFC7FFC0007C00F800003C007800003C007800007800F000 007800F000007800F000007800F000007800F000007800F00000F001E00000F001E00000 F001E00000F001E00000FFFFE00000F001E00001E003C00001E003C00001E003C00001E0 03C00001E003C00001E003C00003C007800003C007800003C007800003C007800003C007 800003C007800007800F000007C00F8000FFF8FFF800221F7E9E22>72 D<07FFE0007C00003C00003C0000780000780000780000780000780000780000F00000F0 0000F00000F00000F00000F00001E00001E00001E00001E00001E00001E00003C00003C0 0003C00003C00003C00003C00007800007C000FFFC00131F7F9E10>I<07FFF000007E00 00003C0000003C000000780000007800000078000000780000007800000078000000F000 0000F0000000F0000000F0000000F0000000F0000001E0000001E0000001E0000001E000 0001E0008001E0010003C0010003C0010003C0030003C0020003C0060003C0060007801E 0007807C00FFFFFC00191F7E9E1C>76 D<07FC0000FFC0007C0000F800003C0001780000 3C00017800004E0002F000004E0002F000004E0004F000004E0004F000004E0008F00000 4E0008F00000870011E00000870011E00000870021E00000870021E00000870041E00000 838041E00001038083C00001038083C00001038103C00001038203C0000101C203C00001 01C403C0000201C40780000201C80780000201C80780000201D00780000200F007800006 00E00780000600E00F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC000 3E003E00003E001800003E001800004F001000004F001000004780100000478010000043 C010000043C010000083C020000081E020000081E020000080F020000080F02000008078 2000010078400001007C400001003C400001003C400001001E400001001E400002000F80 0002000F800002000F800002000780000200078000060003800006000300000F00010000 FFE0010000221F7E9E22>I<0003F800001E0E000038070000E0038001C001C003C001E0 078000E00F0000F00F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F8 7C0000F87C0000F8F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0 780007C07C0007803C000F003C001E001E001C000E0038000700F00003C3C00000FE0000 1D217B9F23>I<07FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F800 7800F8007800F8007800F000F001F000F001E000F003C000F0078000F00F0000FFF80001 E0000001E0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003 C0000003C0000003C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C 0700003C03C0003C01E0007801E0007801F0007801F0007801F0007801F0007801E000F0 03E000F003C000F0078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E0 1C0001E01C0001E01E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E020780 3E0407C01F04FFFC0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C0300 1C0700180600080E00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FF C0007FE00007E00001E00000E00000F00000F04000E04000E04000E04000E06000C06001 80E00380F80300C60C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030 400F0010400F0010C00F0010800F0010800F0010800F0010001E0000001E0000001E0000 001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003C0000 00780000007800000078000000780000007800000078000000F0000001F800007FFFE000 1C1F7A9E21>I<03FFC0FFC0007F007E00003E003800001E003000001E002000000F0040 00000F008000000F81000000078200000007C600000003C400000003E800000001F00000 0001F000000000F000000000F800000000F8000000017C000000023C000000043C000000 0C1E000000081E000000101F000000200F000000400F800000C0078000008007C0000100 03C000070003E0001F8007E000FFE01FFE00221F7F9E22>88 DI E /Fi 24 118 df<001C0000003E0000003E0000002E00 00006700000067000000E7800000C7800000C3800001C3C0000183C0000181C0000381E0 000381E0000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C 0018003C0038001E0038001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 D<003FC000FFF003C0F00780300F00001E00003C00003C0000780000780000780000F000 00F00000F00000F00000F00000F00000F00000F00000F000007800007800007800003C00 003C00001E00000F000807801803C07800FFF0003F80151F7D9D1B>67 DII72 D I76 DII<003F000001FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F007800 07807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F80007C078000780780007807C000F803C000F003E001F001F003E000F80 7C0007C0F80003FFF00001FFE000003F00001A1F7E9D1F>II82 D<03F8000FFE001C0F00380700700300600000E00000E00000E00000E00000F000007800 007F00003FE0001FFC0007FE0001FF00001F800007800003C00003C00001C00001C00001 C00001C0C00180E00380F007007C0E001FFC0007F000121F7E9D17>III87 D<07E00FF81FFC3C1C70047000E000E000E000E000E000E000700070043C1C 1FFC0FF807E00E127E9112>99 D<03C3C00FFFC01FFFC01C3800381C00381C00381C0038 1C00381C001C38001FF8001FF0003BC0003800003800001FFC001FFF003FFF80700780E0 01C0E001C0E001C0F003C07C0F803FFF001FFE0007F800121B7F9115>103 D109 D<03F0000FFC001FFE003C0F00780780700380E001C0E001C0E001 C0E001C0E001C0F003C07003807807803C0F001FFE000FFC0003F00012127F9115>111 DI114 D<1C001C001C001C001C001C00FFE0FFE01C001C001C001C001C001C001C001C001C001C 001C001C001C201FF00FF007C00C187F970F>116 DI E /Fj 1 16 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF80F F003C010107E9115>15 D E /Fk 51 123 df<007E0001C1800301800703C00E03C00E01 800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F8151D 809C17>12 D<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<6060F0F0F8F86868 080808080808101010102020404080800D0C7F9C15>34 D<004000800100020006000C00 0C0018001800300030007000600060006000E000E000E000E000E000E000E000E000E000 E000E000E000600060006000700030003000180018000C000C0006000200010000800040 0A2A7D9E10>40 D<800040002000100018000C000C000600060003000300038001800180 018001C001C001C001C001C001C001C001C001C001C001C001C001800180018003800300 0300060006000C000C00180010002000400080000A2A7E9E10>I<60F0F0701010101020 204080040C7C830C>44 DI<60F0F06004047C830C>I<00010003 000600060006000C000C000C0018001800180030003000300060006000C000C000C00180 01800180030003000300060006000C000C000C0018001800180030003000300060006000 6000C000C00010297E9E15>I<030007003F00C700070007000700070007000700070007 00070007000700070007000700070007000700070007000700070007000F80FFF80D1C7C 9B15>49 D<07C01830201C400C400EF00FF80FF807F8077007000F000E000E001C001C00 380070006000C00180030006010C01180110023FFE7FFEFFFE101C7E9B15>I<07E01830 201C201C781E780E781E381E001C001C00180030006007E00030001C001C000E000F000F 700FF80FF80FF80FF00E401C201C183007E0101D7E9B15>I<000C00000C00001C00003C 00003C00005C0000DC00009C00011C00031C00021C00041C000C1C00081C00101C00301C 00201C00401C00C01C00FFFFC0001C00001C00001C00001C00001C00001C00001C0001FF C0121C7F9B15>I<60F0F0600000000000000000000060F0F06004127C910C>58 D66 D70 D72 DI77 DI<003F800000E0E0000380380007 001C000E000E001C0007003C00078038000380780003C0780003C0700001C0F00001E0F0 0001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C078 0003C0380003803C0007801C0007000E000E0007001C000380380000E0E000003F80001B 1E7E9C20>II<7FFFFFC0700F01C0600F00 C0400F0040400F0040C00F0020800F0020800F0020800F0020000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000001F800003FFFC001B1C7F9B1E>84 DI<08081010202040404040808080808080B0B0F8F8 787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C0001FC00 0F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E801F870012127E9115> 97 DI<07E00C3018783078703060 00E000E000E000E000E000E00060007004300418080C3007C00E127E9112>I<003F0000 070000070000070000070000070000070000070000070000070000070003E7000C170018 0F00300700700700600700E00700E00700E00700E00700E00700E0070060070070070030 0700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006E006FFFEE000 E000E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E061E0E0C0E 000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C3880181800381C00 381C00381C00381C001818001C38000C300013C0001000003000001800001FF8001FFF00 1FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007F800121C7F9215> II<18003C003C0018000000000000 000000000000000000FC001C001C001C001C001C001C001C001C001C001C001C001C001C 001C001C001C00FF80091D7F9C0C>I<00C001E001E000C0000000000000000000000000 00000FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E060E0F0C0F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180 E001C0E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C0003F000 12127F9115>II<03C1000C3300180B00300F0070070070 0700E00700E00700E00700E00700E00700E00700600700700700300F00180F000C370007 C700000700000700000700000700000700000700000700003FE0131A7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000F880388018C018C018E010 D0608FC00D127F9110>I<04000400040004000C000C001C003C00FFE01C001C001C001C 001C001C001C001C001C001C101C101C101C101C100C100E2003C00C1A7F9910>IIII<7F8FF00F03800F030007020003840001C8 0001D80000F00000700000780000F800009C00010E00020E000607000403801E07C0FF0F F81512809116>II<7FFC70386038407040F040E041C003 C0038007000F040E041C043C0C380870087038FFF80E127F9112>I E /Fl 8 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000F800 00F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115 >99 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F 001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018E018 F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Fm 33 121 df46 D<01F00007FC000FFE001F1F003C0780 3C07807803C07803C07803C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0 F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C07803C07803C0 3C07803C07801F1F000FFE0007FC0001F00013237EA118>48 D<01F00007FC001FFF003E 0F003807807003C02003C02003C00003C00003C00003C0000780000780000F00001E0003 FC0003F80003FE00000F000007800003C00003C00001E00001E00001E00001E00001E080 01E0C003C0E003C07007803C0F801FFF000FFC0003F00013237EA118>51 D<001F00001F00002F00002F00006F0000EF0000CF0001CF0001CF00038F00038F00078F 00070F000F0F000E0F001E0F003C0F003C0F00780F00780F00F00F00FFFFF8FFFFF8FFFF F8000F00000F00000F00000F00000F00000F00000F00000F00000F0015217FA018>I<3F FF803FFF803FFF803C00003C00003C00003C00003C00003C00003C00003C00003C00003C F8003FFE003FFF003F0F803E07803C03C03803C00001E00001E00001E00001E00001E000 01E00001E04003C04003C0E003C07007807C1F003FFE000FFC0003F00013227EA018>I< 007E0001FF0003FF0007C1000F00001E00001E00003C00003C000078000078000078F800 F1FE00F7FF00FF0F80FC0780F803C0F803C0F801C0F001E0F001E0F001E0F001E0F001E0 7001E07001E07801E07803C03803C03C03801C07801F0F000FFE0007FC0001F00013237E A118>I68 DI73 D77 DI80 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800007800007800007800007800 007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F00000F00000 F8000078000078000078000078000078000078C000F0E000F0F801E07E07C03FFF800FFF 0001FC0015257EA31B>83 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF 3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180780000780000700000F00000F00000F0 0000F00000F00000F000007800007800007800003C00401F03C00FFFC007FF8001FC0012 167E9516>I<03F00007FC001FFE003E0F003C0780780380780380F001C0FFFFC0FFFFC0 FFFFC0F00000F00000F000007000007800007800003C00801F07800FFF8007FF0001F800 12167E9516>101 D<01F07807FFF80FFFF81F1F001E0F003C07803C07803C07803C0780 3C07801E0F001F1F000FFE001FFC0019F0003800003800003C00001FFE001FFFC01FFFE0 3FFFF07801F07800F8F00078F00078F00078F000787800F03E03E01FFFC00FFF8001FC00 15217F9518>103 DII107 DIII<01FC 0007FF000FFF801F07C03C01E07800F07800F0700070F00078F00078F00078F00078F000 78F000787800F07800F07C01F03E03E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC0 0FFFC01F0FC03C07C07C03C07803C0F803C0F003C0F003C0F003C0F003C0F003C0F003C0 F803C07803C07803C03C07C03F0FC01FFFC00FFBC003E3C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C012207E9519>II<07F01FFC3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F 000FC00FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FF F80F000F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07 F803E00E1C7F9B12>II119 D<7801F07C01E03E03C01E07C00F0780078F0007DE0003 FC0001FC0000F80000700000F80001FC0003DC00039E00078F000F07801E07801E03C03C 01E07800F0F800F81516809516>I E /Fn 12 122 df68 D80 D<000FFFF000000000FFFFFF00000003FF FFFFC0000007F8007FF000000FFC001FFC00000FFC0007FE00001FFE0003FF00001FFE00 03FF80001FFE0001FF80001FFE0001FFC0001FFE0000FFC0000FFC0000FFE00007F80000 FFE00003F00000FFE00000000000FFE00000000000FFE00000000000FFE00000000000FF E00000000000FFE0000000007FFFE00000003FFFFFE0000003FFFFFFE000001FFFF0FFE0 00007FFE00FFE00001FFE000FFE00003FF8000FFE0000FFE0000FFE0001FFC0000FFE000 1FF80000FFE0003FF80000FFE0007FF00000FFE0007FF00000FFE000FFE00000FFE000FF E00000FFE000FFE00000FFE000FFE00000FFE000FFE00001FFE000FFE00001FFE0007FF0 0003FFE0007FF000077FE0003FF8000F7FE0001FFC003E7FF8000FFF80F83FFFE003FFFF F01FFFE000FFFFC00FFFE0000FFE0003FFE0332E7CAD38>97 D<00001FFFC0000001FFFF FC000007FFFFFF00001FF8007F80007FE000FFC000FF8000FFC001FF0001FFE003FE0001 FFE007FE0001FFE00FFC0001FFE00FFC0001FFE01FF80000FFC03FF800007F803FF80000 3F007FF8000000007FF0000000007FF0000000007FF000000000FFF000000000FFF00000 0000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF00000 0000FFF000000000FFF0000000007FF0000000007FF0000000007FF8000000007FF80000 00003FF8000000003FF8000000001FFC000000F00FFC000000F00FFE000001E007FE0000 01E003FF000003C001FF8000078000FFC0000F00007FF0003E00001FFC01FC000007FFFF F0000001FFFFC00000001FFC00002C2E7CAD34>99 D<00001FFC00000001FFFFC0000007 FFFFF000001FF80FFC00007FC001FE0000FF8000FF0001FF00007F8003FE00003FC007FC 00001FE00FFC00001FE01FF800001FF01FF800000FF03FF800000FF83FF800000FF83FF0 00000FF87FF0000007F87FF0000007FC7FF0000007FCFFF0000007FCFFF0000007FCFFFF FFFFFFFCFFFFFFFFFFFCFFFFFFFFFFFCFFF000000000FFF000000000FFF000000000FFF0 00000000FFF0000000007FF0000000007FF0000000007FF0000000003FF8000000003FF8 000000001FF80000003C1FF80000003C0FFC0000003C0FFC0000007807FE000000F803FF 000001F001FF800003E000FFC00007C0003FE0001F80001FFC01FF000007FFFFFC000000 FFFFE00000000FFF00002E2E7DAD35>101 D<007C0001FF0003FF8007FFC007FFC00FFF E00FFFE00FFFE00FFFE00FFFE007FFC007FFC003FF8001FF00007C000000000000000000 00000000000000000000000000000000000000000000000000000000007FC0FFFFC0FFFF C0FFFFC0FFFFC003FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC0FFFFFFFFFFFFFFFFFFFFFFFF18497CC820>105 D<00FF8003FF000003FF000000FFFF801FFFE0001FFFE00000FFFF807FFFF8007FFFF800 00FFFF81F80FFC01F80FFC0000FFFF83C007FE03C007FE000003FF878003FF078003FF00 0001FF8E0003FF0E0003FF000001FF9C0003FF9C0003FF800001FFB80001FFB80001FF80 0001FFB80001FFF80001FFC00001FFF00001FFF00001FFC00001FFF00001FFF00001FFC0 0001FFE00001FFE00001FFC00001FFE00001FFE00001FFC00001FFE00001FFE00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 00FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF 80FFFFFF80FFFFFF80FFFFFF80592E7BAD62>109 D<00FF8007FE0000FFFF803FFFC000 FFFF807FFFF000FFFF81F81FF800FFFF83C00FFC0003FF870007FE0001FF8E0007FE0001 FF9C0007FF0001FFB80003FF0001FFB80003FF8001FFF00003FF8001FFF00003FF8001FF E00003FF8001FFE00003FF8001FFE00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC000 03FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003 FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF 8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF80 01FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF80FFFFFF81FFFFFFFF FFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFF382E7BAD41>I<00000FFE0000000001 FFFFF000000007FFFFFC0000001FF803FF0000007FE000FFC00000FF80003FE00001FF00 001FF00003FE00000FF80007FC000007FC000FFC000007FE000FF8000003FE001FF80000 03FF003FF8000003FF803FF0000001FF803FF0000001FF807FF0000001FFC07FF0000001 FFC07FF0000001FFC0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FF E0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0 FFF0000001FFE07FF0000001FFC07FF0000001FFC07FF0000001FFC07FF0000001FFC03F F8000003FF803FF8000003FF801FF8000003FF001FFC000007FF000FFC000007FE0007FE 00000FFC0003FE00000FF80001FF00001FF00000FF80003FE000007FE000FFC000001FFC 07FF00000007FFFFFC00000001FFFFF0000000001FFF000000332E7DAD3A>I<007F803F 80FFFF80FFE0FFFF81FFF8FFFF83C3FCFFFF8707FE03FF8E07FE01FF9C0FFF01FFB80FFF 01FFB80FFF01FFF00FFF01FFF00FFF01FFF007FE01FFE003FC01FFE001F801FFE0000001 FFE0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FF C0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0 000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC000 0001FFC0000001FFC0000001FFC0000001FFC0000001FFC00000FFFFFFE000FFFFFFE000 FFFFFFE000FFFFFFE000282E7DAD2F>114 D<000FFE01C000FFFFC7C003FFFFFFC007F8 01FFC00FC0007FC01F00001FC03F00000FC07E000007C07E000007C07E000003C0FE0000 03C0FE000003C0FF000003C0FF800003C0FFE0000000FFF80000007FFFC000007FFFFF00 003FFFFFE0003FFFFFF8001FFFFFFC000FFFFFFF0007FFFFFF8001FFFFFFC0007FFFFFE0 001FFFFFE00000FFFFF0000003FFF00000007FF87000001FF8F000000FF8F0000007F8F8 000007F8F8000003F8F8000003F8FC000003F0FC000003F0FE000003F0FF000007E0FF80 0007E0FFC0000FC0FFF0001F80FEFC00FF00F87FFFFC00F01FFFF000E003FF8000252E7C AD2E>I121 D E /Fo 8 117 df<0003FF0000001FFFF000007FFFFC0000FC07FE0001F0 01FF0003C000FF800780007FC00FE0007FE00FF0007FE01FF8007FF01FFC007FF01FFC00 7FF01FFC007FF01FFC007FF00FF8007FF007F0007FE003E0007FE0008000FFE0000000FF C0000000FF80000001FF00000001FE00000003FC00000007F80000001FF000000FFFC000 000FFF8000000FFFF000000007FE00000000FF000000007FC00000007FE00000003FF000 00003FF80000003FF80000001FFC0000001FFC0000001FFE0000001FFE1F80001FFE3FC0 001FFE7FE0001FFEFFF0001FFEFFF0001FFEFFF0001FFCFFF0001FFCFFF0001FFCFFE000 3FF87FC0003FF87F80007FF03E00007FE01FC000FFC00FF803FF8007FFFFFF0001FFFFFC 00007FFFF0000007FF000027397CB730>51 D<0000001FFE000060000003FFFFE000E000 001FFFFFF801E000007FFFFFFE03E00001FFFC00FF07E00007FFC0001FCFE0000FFF0000 07FFE0001FFC000001FFE0003FF0000000FFE0007FE00000007FE000FFC00000003FE001 FF800000001FE003FF000000000FE007FF000000000FE00FFE0000000007E00FFE000000 0007E01FFC0000000003E01FFC0000000003E03FF80000000003E03FF80000000001E03F F80000000001E07FF80000000001E07FF80000000001E07FF00000000000007FF0000000 000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FF F0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000 000000FFF0000000000000FFF00000000000007FF00000000000007FF00000000000007F F80000000000007FF80000000001E03FF80000000001E03FF80000000001E03FF8000000 0001E01FFC0000000001E01FFC0000000003C00FFE0000000003C00FFE0000000003C007 FF00000000078003FF00000000078001FF800000000F0000FFC00000001F00007FE00000 003E00003FF00000007C00001FFC000000F800000FFF000003F0000007FFC0000FE00000 01FFFC007F800000007FFFFFFF000000001FFFFFFC0000000003FFFFE000000000001FFE 0000003B3D7BBB46>67 D<003FFE00000001FFFFE0000007FFFFF800000FE007FC00000F F001FE00001FF800FF00001FF8007F80001FF8007FC0001FF8003FC0000FF0003FE00007 E0003FE00003C0003FE0000000003FE0000000003FE0000000003FE0000000003FE00000 00FFFFE000001FFFFFE000007FF83FE00003FF803FE00007FC003FE0000FF0003FE0001F E0003FE0003FE0003FE0007FC0003FE0007FC0003FE000FF80003FE000FF80003FE000FF 80003FE000FF80003FE000FF80007FE0007FC0007FE0007FC000DFE0003FE0039FF0001F F80F0FFFE007FFFE0FFFE001FFFC07FFE0003FE000FFE02B267DA52F>97 D<0001FF8000000FFFF000007FFFFC0000FF81FE0003FE007F8007F8003F800FF0001FC0 0FF0000FE01FE0000FE03FE0000FF03FE00007F07FC00007F07FC00007F87FC00007F8FF C00007F8FFC00007F8FFFFFFFFF8FFFFFFFFF8FFFFFFFFF8FFC0000000FFC0000000FFC0 000000FFC00000007FC00000007FC00000007FC00000003FE00000003FE00000781FE000 00781FF00000780FF00000F007F80001F003FC0003E001FE000FC000FFC07F80003FFFFE 00000FFFF8000000FFC00025267DA52C>101 D<00FE00000000FFFE00000000FFFE0000 0000FFFE00000000FFFE0000000007FE0000000003FE0000000003FE0000000003FE0000 000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000 000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000 000003FE0000000003FE00FF800003FE03FFF00003FE0FFFF80003FE1E03FC0003FE3801 FE0003FE6001FF0003FEC000FF0003FFC000FF8003FF8000FF8003FF0000FF8003FF0000 FF8003FF0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF80FFFFF83F FFFEFFFFF83FFFFEFFFFF83FFFFEFFFFF83FFFFE2F3C7CBB36>104 D<00FE01FF0000FFFE1FFFF000FFFE7FFFFC00FFFEFC07FE00FFFFF001FF0007FFC0007F 8003FF00003FC003FE00003FE003FE00001FF003FE00001FF803FE00000FF803FE00000F FC03FE00000FFC03FE000007FC03FE000007FE03FE000007FE03FE000007FE03FE000007 FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007 FC03FE00000FFC03FE00000FFC03FE00000FFC03FE00000FF803FE00001FF003FE00001F F003FF00003FE003FF80007FC003FFC000FF8003FFE001FF0003FEFC07FE0003FE7FFFF8 0003FE1FFFE00003FE03FE000003FE0000000003FE0000000003FE0000000003FE000000 0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE000000 0003FE0000000003FE0000000003FE00000000FFFFF8000000FFFFF8000000FFFFF80000 00FFFFF80000002F377DA536>112 D<01FC03F000FFFC0FFC00FFFC1FFF00FFFC3C3F80 FFFC707F8007FCE0FFC003FCC0FFC003FD80FFC003FD80FFC003FF807F8003FF003F0003 FF001E0003FF00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE 00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00 000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE0000 0003FE000000FFFFFC0000FFFFFC0000FFFFFC0000FFFFFC000022267DA528>114 D<000F0000000F0000000F0000000F0000000F0000001F0000001F0000001F0000001F00 00003F0000003F0000007F0000007F000000FF000001FF000003FF000007FF00001FFFFF F0FFFFFFF0FFFFFFF0FFFFFFF001FF000001FF000001FF000001FF000001FF000001FF00 0001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF00 0001FF000001FF000001FF000001FF000001FF003C01FF003C01FF003C01FF003C01FF00 3C01FF003C01FF003C01FF003C00FF007800FF8078007F80F0003FC1E0001FFFC0000FFF 800001FE001E377EB626>116 D E /Fp 76 124 df<001F83E000F06E3001C078780380 F8780300F03007007000070070000700700007007000070070000700700007007000FFFF FF8007007000070070000700700007007000070070000700700007007000070070000700 700007007000070070000700700007007000070070000700700007007000070070000700 70007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007000007 0000070000070000070000070000FFFFE00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07F C3FE1720809F19>I<003FE000E0E001C1E00381E00700E00700E00700E00700E00700E0 0700E00700E00700E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E0 0700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FE7FE172080 9F19>I<001F81F80000F04F040001C07C06000380F80F000300F00F000700F00F000700 70000007007000000700700000070070000007007000000700700000FFFFFFFF00070070 070007007007000700700700070070070007007007000700700700070070070007007007 000700700700070070070007007007000700700700070070070007007007000700700700 0700700700070070070007007007007FE3FE3FF02420809F26>I<70F8F8F8F8F8F8F870 7070707070707070702020202020000000000070F8F8F87005217CA00D>33 D<7038F87CFC7EFC7E743A0402040204020804080410081008201040200F0E7E9F17>I< 70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C00 0C00180018003000300030007000600060006000E000E000E000E000E000E000E000E000 E000E000E000E0006000600060007000300030003000180018000C000C00060002000100 0080004000200B2E7DA112>I<800040002000100008000C000600060003000300018001 80018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E000 C000C000C001C001800180018003000300060006000C00080010002000400080000B2E7D A112>I<01800180018001800180C183F18F399C0FF003C003C00FF0399CF18FC1830180 018001800180018010147DA117>I<70F8FCFC74040404080810102040060E7C840D>44 DI<70F8F8F87005057C840D>I<03F0000E1C001C0E0018060038 0700700380700380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C0F003C0F003C07003807003807003807807803807001806001C 0E000E1C0003F000121F7E9D17>48 D<018003800F80F380038003800380038003800380 038003800380038003800380038003800380038003800380038003800380038003800380 07C0FFFE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F8 03C0F803C02007C00007C0000780000780000F00000E00001C0000380000700000600000 C0000180000300000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I< 03F0000C1C00100E00200F00780F80780780780780380F80000F80000F00000F00000E00 001C0000380003F000003C00000E00000F000007800007800007C02007C0F807C0F807C0 F807C0F00780400780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E 00000E00001E00002E00002E00004E00008E00008E00010E00020E00020E00040E00080E 00080E00100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E 00000E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE00010000010 000010000010000010000010000011F000161C00180E0010070010078000038000038000 03C00003C00003C07003C0F003C0F003C0E00380400380400700200600100E000C380003 E000121F7E9D17>I<007C000182000701000E03800C07801C0780380300380000780000 700000700000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0 F003C07003C07003C07003803803803807001807000C0E00061C0001F000121F7E9D17> I<4000007FFFC07FFF807FFF804001008002008002008004000008000008000010000020 0000200000400000400000C00000C00001C0000180000380000380000380000380000780 00078000078000078000078000078000078000030000121F7D9D17>I<03F0000C0C0010 06003003002001806001806001806001807001807803003E03003F06001FC8000FF00003 F80007FC000C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0C0008060 01802001001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C00380600380700 700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0 180BC00E13C003E3C0000380000380000380000700300700780600780E00700C00201800 1070000FC000121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C 930D>I<70F8F8F8700000000000000000000070F0F8F878080808101010202040051D7C 930D>I<7FFFFFE0FFFFFFF0000000000000000000000000000000000000000000000000 0000000000000000FFFFFFF07FFFFFE01C0C7D9023>61 D<000100000003800000038000 000380000007C0000007C0000007C0000009E0000009E0000009E0000010F0000010F000 0010F00000207800002078000020780000403C0000403C0000403C0000801E0000801E00 00FFFE0001000F0001000F0001000F00020007800200078002000780040003C00E0003C0 1F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C 0000C07C0000407C00004078000040F8000000F8000000F8000000F8000000F8000000F8 000000F8000000F8000000F8000000780000007C0000407C0000403C0000401C0000401E 0000800E000080070001000380020001C0040000703800000FC0001A217D9F21>I69 DI<000FE0200078186000E004E0038002E0070001E00F0000E01E0000 601E0000603C0000603C0000207C00002078000020F8000000F8000000F8000000F80000 00F8000000F8000000F8000000F8007FFCF80003E0780001E07C0001E03C0001E03C0001 E01E0001E01E0001E00F0001E0070001E0038002E000E0046000781820000FE0001E217D 9F24>III76 DII<001F800000F0F00001C038000780 1E000F000F000E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F800 01F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F07800 01E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0 380000F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F00000F000 007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C08001 C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7F FFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F001080 0F0010000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003 C0020001E0040001F00C0000F0080000781000007C1000003C2000003E4000001E400000 0F8000000F8000000780000003C0000007E0000005E0000009F0000018F8000010780000 207C0000603C0000401E0000801F0001800F0001000780020007C0070003C01F8007E0FF E01FFE1F1F7F9E22>I91 D<080410082010201040 204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E00 1E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317> 97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E0078 0E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F8 0E0C1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C 03F010147E9314>I<000380003F80000380000380000380000380000380000380000380 00038000038000038003E380061B801C0780380380380380700380700380F00380F00380 F00380F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E 9F19>I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F000 00F00000F000007000007000003800801800800C010007060001F80011147F9314>I<00 7C00C6018F038F07060700070007000700070007000700FFF00700070007000700070007 000700070007000700070007000700070007000700070007007FF01020809F0E>I<0000 E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E38 0033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C000 30C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F 01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C000000000000 000000000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E00000000000000000 00000000007007F000F00070007000700070007000700070007000700070007000700070 007000700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000 FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF0 0E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E00 0E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618 000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F 01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C0 7000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C0 1C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00 700E00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3 000E3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319 >I<03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F0 0380F003807003807803803803803807801C0B800E138003E38000038000038000038000 0380000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E00 0E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F 9030704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0 608F800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E00 0E000E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12> I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800 003C00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318> II<3FFF380E200E201C4038407840 7000E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>II E /Fq 45 122 df<70F8FCFC7404040404080810102040060F7C840E> 44 DI<70F8F8F87005057C840E>I<01F000071C000C06001803 003803803803807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001 E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803 C03803803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F38003 800380038003800380038003800380038003800380038003800380038003800380038003 800380038003800380038003800380038007C0FFFE0F217CA018>I<03F8000C1E001007 002007804007C07807C07803C07807C03807C0000780000780000700000F00000E000038 0003F000001C00000F000007800007800003C00003C00003E02003E07003E0F803E0F803 E0F003C04003C0400780200780100F000C1C0003F00013227EA018>51 D<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>I<10 00801E07001FFF001FFE001FF80013E00010000010000010000010000010000010000010 F800130E001407001803801003800001C00001C00001E00001E00001E00001E07001E0F0 01E0F001E0E001C08001C04003C04003802007001006000C1C0003F00013227EA018>I< 007E0001C1000300800601C00E03C01C03C0180180380000380000780000700000700000 F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013227EA018> I<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000 E0C00060C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018 >56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C0 0001C00001C0000380000380300300780700780600700C002018001030000FC00013227E A018>I<0001800000018000000180000003C0000003C0000003C0000005E0000005E000 000DF0000008F0000008F0000010F800001078000010780000203C0000203C0000203C00 00401E0000401E0000401E0000800F0000800F0000FFFF000100078001000780030007C0 020003C0020003C0040003E0040001E0040001E00C0000F00C0000F03E0001F8FF800FFF 20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E000070 1E0000701C0000303C0000303C0000307C0000107800001078000010F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F800000078000000780000107C000010 3C0000103C0000101C0000201E0000200E000040070000400380008001C0010000E00200 00381C000007E0001C247DA223>67 D69 DI73 D77 DI80 D82 D<03F0200C0C601802603001E07000E06000 60E00060E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF 0003FF80003FC00007E00001E00000F00000F0000070800070800070800070800070C000 60C00060E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF8780780786007 8018400780084007800840078008C007800C800780048007800480078004800780040007 800000078000000780000007800000078000000780000007800000078000000780000007 800000078000000780000007800000078000000780000007800000078000000780000007 80000007800000078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E 0018070000070000070000070000FF0007C7001E07003C0700780700700700F00708F007 08F00708F00F087817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C0 0E80600F00300E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E 0E001C0E003C0E00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07 801C0780380300780000700000F00000F00000F00000F00000F00000F00000F000007000 007800403800401C00800C010007060001F80012157E9416>I<0000E0000FE00001E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E001F8E007 04E00C02E01C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F0 00E07000E07800E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC00070700 0C03801C01C03801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000 7000007800203800201C00400E008007030000FC0013157F9416>I<003C00C6018F038F 030F070007000700070007000700070007000700FFF80700070007000700070007000700 0700070007000700070007000700070007000700070007807FF8102380A20F>I<000070 01F198071E180E0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E00 0F1C0019F0001000001000001800001800001FFE000FFFC00FFFE03800F0600030400018 C00018C00018C000186000306000303800E00E038003FE0015217F9518>I<0E0000FE00 001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E1F800E60C00E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C 001E003E001E001C00000000000000000000000000000000000E00FE001E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E >I<01C003E003E003E001C00000000000000000000000000000000001E00FE001E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01 F00E01C00E01800E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07 000E07800E03C00E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1F C07F00FE60E183801E807201C00F003C00E00F003C00E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A >I<0E1F80FE60C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I< 01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F00078 F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F 00FE61C00E80600F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E00 1E0E001E0E003C0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00 000E00000E00000E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F06 0F000E000E000E000E000E000E000E000E000E000E000E000E000E000F00FFF010157F94 13>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C80 0CC00CC008E018D0308FC00E157E9413>I<02000200020002000600060006000E001E00 3E00FFF80E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E04 0E040708030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F0060170 03827800FC7F18157F941B>III121 D E /Fr 24 122 df45 D<007F800001FFF0000701F8000C00 7E0018001F0010000F0020000F80400007C0400007C0400003E0B00003E0F80003E0FC00 03E0FC0003E0FC0003E0780003E0000003E0000007C0000007C00000078000000F800000 0F0000001F0000001E0000003C0000007800000070000000E0000001C000000380000007 0000000E0000000C000000180000003000000060000000C0002001800020030000200600 00400C00004008000040100000C03FFFFFC07FFFFF80FFFFFF80FFFFFF801B2F7DAE21> 50 D<00003FE0010001FFF8030007F01E03001F800307003E000087007800004F00F000 002F01E000001F03C000000F078000000F0F800000070F000000071F000000031E000000 033E000000033C000000017C000000017C000000017C000000017800000000F800000000 F800000000F800000000F800000000F800000000F800000000F800000000F800000000F8 00000000F800000000F80000000078000000007C000000007C000000017C000000013C00 0000013E000000011E000000011F000000020F000000020F80000006078000000403C000 000801E000000800F00000100078000020003E0000C0001F8003800007F00F000001FFFC 0000003FE00028337CB130>67 DI 73 D77 DI80 D<007F802001FFE02007C078600F001C601E0006E03C 0003E0380001E0780000E0700000E070000060F0000060F0000060F0000020F0000020F0 000020F8000020F80000007C0000007E0000003F0000003FC000001FF800000FFF800007 FFF80003FFFC0000FFFF00000FFF800000FFC000001FE0000007E0000003F0000001F000 0000F0000000F8000000F88000007880000078800000788000007880000078C0000078C0 000070E00000F0E00000E0F00000E0F80001C0EC000380C7000700C1F01E00807FFC0080 0FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E 001C003E001E003E001E0008001E0000001E0000001E0000001E00000FFE0000FC1E0003 E01E000F801E001F001E003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8 003E04F8003E0478003E047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700040F00041E001E1C003E3C003E7C003E7C0008780000F800 00F80000F80000F80000F80000F80000F80000F80000F800007800007C00007C00003C00 011E00011E00020F000207000403801800E060003F80181F7D9E1D>99 D<003F800000E0E0000380380007003C000E001E001E001E001C000F003C000F007C000F 0078000F8078000780F8000780F8000780FFFFFF80F8000000F8000000F8000000F80000 00F8000000F8000000780000007C0000003C0000003C0000801E0000800E0001000F0002 000780020001C00C0000F03000001FC000191F7E9E1D>101 D<0780000000FF80000000 FF800000000F800000000780000000078000000007800000000780000000078000000007 800000000780000000078000000007800000000780000000078000000007800000000780 000000078000000007800000000780FE00000783078000078C03C000079001E00007A001 E00007A000F00007C000F00007C000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078000F000 078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF8021327EB125>104 D<07000F801F801F800F8007000000000000000000000000000000000000000000000007 80FF80FF800F800780078007800780078007800780078007800780078007800780078007 800780078007800780078007800780078007800FC0FFF8FFF80D307EAF12>I<0780FF80 FF800F800780078007800780078007800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800780078007800780078007800FC0FFFCFFFC0E327EB112>108 D<0780FE001FC000FF83078060F000FF8C03C18078000F9001E2003C0007A001E4003C00 07A000F4001E0007C000F8001E0007C000F8001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E000780 00F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83 FFF0341F7E9E38>I<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A0 00F00007C000F00007C000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078000F000 078000F0000FC001F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001 C01C00070007000F0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F078 0000F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F878 0000F07C0001F03C0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000 F07800001FC0001D1F7E9E21>I<0781FC00FF860700FF8803C00F9001E007A000F007C0 0078078000780780003C0780003C0780003E0780001E0780001F0780001F0780001F0780 001F0780001F0780001F0780001F0780001F0780001F0780003E0780003E0780003C0780 007C0780007807C000F007A000F007A001E00798038007860F000781F800078000000780 000007800000078000000780000007800000078000000780000007800000078000000780 00000FC00000FFFC0000FFFC0000202D7E9E25>I<0783E0FF8C18FF907C0F907C07A07C 07C03807C00007C00007C000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000780000780000780000780000780000FC000 FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010 E00010E00010F00010F800007E00003FF0001FFF000FFFC003FFE0003FF00001F80000F8 80003C80003C80001CC0001CC0001CE0001CE00018F00038F00030CC0060C301C080FE00 161F7E9E1A>I<00400000400000400000400000400000C00000C00000C00001C00001C0 0003C00007C0000FC0001FFFE0FFFFE003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C01003C01003C0 1003C01003C01003C01003C01003C01001C02001E02000E0400078C0001F00142C7FAB19 >I119 D121 D E /Fs 5 85 df<00000000600000000000E00000000000E00000000001E00000000003 E00000000003E00000000007E00000000007F0000000000DF0000000000DF00000000019 F00000000039F00000000031F00000000061F00000000061F000000000C1F000000000C1 F00000000181F00000000381F00000000301F00000000601F00000000601F00000000C01 F80000000C00F80000001800F80000003800F80000003000F80000006000F80000006000 F8000000C000F8000000C000F8000001FFFFF8000001FFFFF80000030000F80000060000 F80000060000F800000C0000F800000C0000FC00001800007C00001800007C0000300000 7C00007000007C00006000007C0000C000007C0000C000007C0001C000007C0003C00000 7C000FE00000FE00FFF8001FFFE0FFF8001FFFE02B327CB134>65 D<000FFFFFFC0000000FFFFFFF800000007E000FE00000007C0003F00000007C0000F800 00007C00007C0000007C00007C000000F800003E000000F800001E000000F800001F0000 00F800001F000001F000001F000001F000000F800001F000000F800001F000000F800003 E000000F800003E000000F800003E000000F800003E000001F800007C000001F800007C0 00001F800007C000001F800007C000001F80000F8000001F00000F8000003F00000F8000 003F00000F8000003F00001F0000007E00001F0000007E00001F0000007E00001F000000 7C00003E000000FC00003E000000F800003E000001F800003E000001F000007C000003E0 00007C000003E000007C000007C000007C00000F800000F800000F000000F800001E0000 00F800003C000000F8000078000001F00001F0000001F00003E0000001F0000F80000003 F0007F000000FFFFFFFC000000FFFFFFC000000031317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007E0003F800007C0000F800007C00007800007C0000 7800007C0000300000F80000300000F80000300000F80000300000F80000300001F00000 300001F00000300001F00000300001F00000600003E00180600003E00180000003E00180 000003E00180000007C00300000007C00300000007C00700000007C01F0000000FFFFE00 00000FFFFE0000000F801E0000000F800E0000001F000C0000001F000C0000001F000C00 00001F000C0000003E00180000003E00180000003E00000000003E00000000007C000000 00007C00000000007C00000000007C0000000000F80000000000F80000000000F8000000 0000F80000000001F00000000001F00000000001F00000000003F000000000FFFFE00000 00FFFFE00000002D317BB02E>70 D<000FFFFFE000000FFFFFFC0000007E003F0000007C 000F8000007C0007C000007C0003E000007C0003E00000F80001F00000F80001F00000F8 0001F00000F80003F00001F00003F00001F00003F00001F00003F00001F00003E00003E0 0007E00003E00007C00003E0000FC00003E0000F800007C0001F000007C0003E000007C0 0078000007C001E000000F800F8000000FFFFC0000000F800E0000000F80070000001F00 038000001F0003C000001F0001E000001F0001E000003E0001F000003E0001F000003E00 01F000003E0001F000007C0003F000007C0003F000007C0003E000007C0003E00000F800 07E00000F80007E00000F80007E00000F80007E01801F00007E03001F00007E03001F000 07E02003F00003E060FFFF8001E0C0FFFF8000F180000000003E002D327BB033>82 D<07FFFFFFFFF007FFFFFFFFF00FE007E007F00F8007C001F00E0007C000E00C0007C000 E01C0007C000E018000F8000E018000F8000E030000F8000C030000F8000C030001F0000 C060001F0000C060001F0000C060001F0000C0C0003E00018000003E00000000003E0000 0000003E00000000007C00000000007C00000000007C00000000007C0000000000F80000 000000F80000000000F80000000000F80000000001F00000000001F00000000001F00000 000001F00000000003E00000000003E00000000003E00000000003E00000000007C00000 000007C00000000007C00000000007C0000000000F80000000000F80000000000F800000 00000F80000000001F00000000001F00000000001F00000000003F000000003FFFFF0000 003FFFFF0000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 799 911 a Fs(D)25 b(R)g(A)g(F)g(T)160 1002 y Fr(MPI-2)c(Dynamic)f(Pro)r(cess)i(Chapter)g(-)f(New)h(Spa)n(wn)e(Prop)r (osal)621 1196 y Fq(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)844 1323 y(Ma)o(y)h(3,)g(1996)77 1381 y(This)h(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1439 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1497 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 49 a Fp(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 361 a Fo(Chapter)31 b(3)75 568 y Fn(Dynamic)38 b(Pro)s(cesses)75 792 y Fm(3.0.3)49 b(Sta)o(rting)16 b(Pro)q(cesses)75 954 y Fl(Discussion:)34 b Fk(The)14 b(follo)o(wing)d(prop)q(osal)i(com)o(bines)f(all)h(the)h(functionalit)o (y)e(of)h(the)h(previous)g(prop)q(osal)f(in)o(to)g(a)75 1004 y(small)f(n)o(um)o(b)q(er)h(of)g(simple)f(functions.)19 b(The)14 b(prop)q(osal)f(include)h(four)g(basic)g(functions)g(on)g(the) g(paren)o(t)h(side:)145 1079 y Fj(\017)23 b Fi(MPI)p 266 1079 13 2 v 14 w(SP)m(A)-5 b(WN)145 1153 y Fj(\017)23 b Fi(MPI)p 266 1153 V 14 w(ISP)m(A)-5 b(WN)145 1228 y Fj(\017)23 b Fi(MPI)p 266 1228 V 14 w(CHILD)p 400 1228 V 15 w(A)m(TT)m(A)o(CH)145 1303 y Fj(\017)g Fi(MPI)p 266 1303 V 14 w(CHILD)p 400 1303 V 15 w(DET)m(A)o(CH)75 1378 y Fk(T)m(ogether,)c(these)h(pro)o(vide)e(all)f(the)i(functionalit) o(y)d(of)i(the)h(previous)f(prop)q(osal,)h(including)e(the)i(spa)o (wning)f(of)75 1427 y(indep)q(enden)o(t/non-indep)q(enden)o(t)13 b(MPI)e(pro)q(cesses,)j(m)o(ultiple)9 b(groups)i(of)g(pro)q(cesses,)i (and)e(non-MPI)g(pro)q(cesses..)75 1477 y(Here)k(is)f(a)f(brief)h (description)h(of)e(ho)o(w)h(it)f(w)o(orks.)134 1552 y(1.)22 b Fi(MPI)p 266 1552 V 14 w(SP)m(A)-5 b(WN)14 b Fk(tak)o(es)h(a)e(single)h(command)e(line,)h(an)h(MPI/NONMPI)h (\015ag,)e(and)h(returns)i(a)e(group.)19 b(It)14 b(is)189 1602 y(a)f(lo)q(cal)g(function,)g(called)h(on)g(a)f(single)h(no)q(de.) 134 1676 y(2.)22 b(F)m(or)13 b(pro)q(cesses)k(spa)o(wned)d(with)g(the)g (MPI)g(\015ag,)f(the)i(paren)o(t)f(is)g(required)h(to)e(call)h(either) 245 1757 y Fj(\017)23 b Fi(MPI)p 366 1757 V 14 w(CHILD)p 500 1757 V 15 w(A)m(TT)m(A)o(CH)p Fk(,)11 b(whic)o(h)i(tak)o(es)g(the)h (spa)o(wned)f(group)g(as)g(input,)g(is)g(collectiv)o(e)g(o)o(v)o(er)g (a)f(com-)289 1807 y(m)o(unicator,)f(and)j(returns)i(an)d(in)o(tercomm) o(unicator)f(\()p Fi(MPI)p 1228 1807 V 14 w(P)m(ARENT)h Fk(in)g(the)i(c)o(hild\).)289 1857 y(or)245 1923 y Fj(\017)23 b Fi(MPI)p 366 1923 V 14 w(CHILD)p 500 1923 V 15 w(DET)m(A)o(CH)p Fk(,)9 b(whic)o(h)h(tak)o(es)h(the)g(spa)o(wned)g(group)g(as)f(input)h (and)f(returns)i(nothing.)k(\(the)289 1973 y(paren)o(t)e(is)g Fi(MPI)p 538 1973 V 14 w(COMM)p 682 1973 V 15 w(NULL)h Fk(in)e(the)h(c)o(hild\).)189 2054 y(Both)g(the)g(attac)o(h)g(and)g (detac)o(h)h(functions)f(sync)o(hronize)h(with)e Fi(MPI)p 1279 2054 V 15 w(INIT)g Fk(in)g(the)i(c)o(hild.)134 2129 y(3.)22 b(F)m(or)12 b(pro)q(cesses)j(spa)o(wned)f(with)e(the)i(MPI)f (\015ag,)f(the)h(paren)o(t)h(is)e(allo)o(w)o(ed)g(to)g(merge)h(and)f (man)o(upulate)f(the)189 2179 y(groups)g(b)q(efore)g(attac)o(h/detac)o (h.)18 b(The)12 b(requiremen)o(t)e(is)h(that)g(either)h(attac)o(h)f(or) g(detac)o(h)h(b)q(e)f(called)g(exactly)189 2229 y(once)i(for)f(ev)o (ery)i(pro)q(cess)g(spa)o(wned)f(with)g(the)g(MPI)g(\015ag.)k(This)12 b(allo)o(ws)g(functionalit)o(y)f(equiv)n(alen)o(t)h(to)g(the)p 189 2278 V 201 2278 a Fi(MUL)m(TIPLE)h Fk(functions)h(of)g(previous)g (prop)q(osals.)134 2353 y(4.)22 b(F)m(or)10 b(pro)q(cess)j(spa)o(wned)e (with)g(the)g(NONMPI)h(\015ag,)f(the)g(paren)o(t)h(is)e(not)h(allo)o(w) o(ed)f(to)h(call)f(attac)o(h)h(or)g(detac)o(h.)189 2403 y(If)h(those)i(pro)q(cesses)i(call)c Fi(MPI)p 668 2403 V 14 w(INIT)h Fk(the)h(b)q(eha)o(vior)f(is)g(unde\014ned)h(\(though)f (there)h(will)e(b)q(e)i(a)e(prop)q(osal)h(to)189 2453 y(de\014ne)i(it\).)964 2828 y Fp(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fp(2)951 b Fh(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fg(MPI)p 160 49 14 2 v 16 w(SP)l(A)-5 b(WN\(command)p 529 49 V 16 w(line,)15 b(minp)o(ro)q(cs,)f(maxp)o(ro)q (cs,)h(\015ag,)f(info,)h(group,)g(a)o(rra)o(y)p 1468 49 V 14 w(of)p 1519 49 V 16 w(errco)q(des\))117 126 y Fk(IN)155 b Fg(command)p 506 126 V 16 w(line)321 b Fk(executable)15 b(and)f(argumen)o(ts,)f(in)g(a)h(single)f(string)h(\(string\))117 199 y(IN)155 b Fg(minp)o(ro)q(cs)417 b Fk(minim)n(um)10 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)d(start)g(\(in)o(teger\))117 272 y(IN)155 b Fg(maxp)o(ro)q(cs)408 b Fk(maxim)n(um)10 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)d(start)g(\(in)o(teger\))117 344 y(IN)155 b Fg(\015ag)518 b Fi(MPI)p 982 344 13 2 v 15 w(MPI)13 b Fk(or)h Fi(MPI)p 1211 344 V 14 w(NOTMPI)f Fk(\(in)o(teger\))117 417 y(IN)155 b Fg(info)516 b Fk(a)14 b(string)g(telling)f(the)i(run)o(time)e(system)g(where)j(and)d(ho)o(w)h (to)905 473 y(start)h(the)f(pro)q(cesses)j(\(string\))117 546 y(OUT)108 b Fg(group)479 b Fk(group)14 b(con)o(taining)f(newly)h (spa)o(wned)g(pro)q(cesses)i(\(handle\))117 618 y(OUT)108 b Fg(a)o(rra)o(y)p 416 618 14 2 v 15 w(of)p 468 618 V 16 w(errco)q(des)268 b Fk(one)14 b(co)q(de)h(p)q(er)g(pro)q(cess)h (\(arra)o(y)d(of)h(in)o(teger\))75 743 y Ff(int)23 b(MPI)p 245 743 15 2 v 17 w(Spawn\(char*)g(command)p 717 743 V 16 w(line,)g(int)g(minprocs,)g(int)h(maxprocs,)e(char*)h(info,)393 799 y(MPI)p 468 799 V 17 w(Group)g(group,)g(int*)g(array)p 1034 799 V 17 w(of)p 1099 799 V 17 w(errcodes\))75 885 y(MPI)p 150 885 V 17 w(SPAWN\(COMMAND)p 479 885 V 15 w(LINE,)g(MINPROCS,)g(MAXPROCS,)g(INFO,)g(GROUP)g(ARRAY)p 1521 885 V 16 w(OF)p 1585 885 V 17 w(ERRCODES,)393 942 y(IERR\))170 998 y(CHARACTER*\(*\))g(COMMAND)p 676 998 V 16 w(LINE,)g(INFO)170 1055 y(INTEGER)g(MINPROCS,)g(MAXPROCS,)g (GROUP,)g(ARRAY)p 1129 1055 V 16 w(OF)p 1193 1055 V 17 w(ERRCODES\(*\),)f(IERR)166 1141 y Fg(MPI)p 251 1141 14 2 v 16 w(SP)l(A)-5 b(WN)17 b Fp(tries)g(to)f(start)f Fg(maxp)o(ro)q(cs)h Fp(iden)o(tical)i(copies)g(of)e(the)g(program)g(sp) q(eci\014ed)i(b)o(y)f Fg(com-)75 1198 y(mand)p 183 1198 V 16 w(line)p Fp(.)36 b(It)20 b(returns)g(a)g Fg(group)g Fp(con)o(taining)h(the)f(newly)h(spa)o(wned)g(pro)q(cesses,)g(whic)o(h) g(are)f(called)75 1254 y(c)o(hildren.)166 1311 y(The)11 b Fg(\015ag)g Fp(ma)o(y)f(tak)o(e)g(the)h(v)m(alues)h Fg(MPI)p 813 1311 V 16 w(MPI)f Fp(or)f Fg(MPI)p 1055 1311 V 16 w(NONMPI)p Fp(.)h(The)g(v)m(alue)h Fg(MPI)p 1558 1311 V 16 w(MPI)e Fp(is)h(an)g(asser-)75 1367 y(tion)k(that)g(the) g(pro)q(cesses)g(b)q(eing)i(spa)o(wned)e(are)g(MPI)g(pro)q(cesses,)g (that)f(is,)h(that)g(they)g(call)h Fg(MPI)p 1761 1367 V 16 w(INIT)p Fp(.)75 1424 y(This)k(allo)o(ws)f(the)h(paren)o(t)f(to)f (determine)j(ho)o(w)e Fg(MPI)p 1012 1424 V 15 w(COMM)p 1169 1424 V 17 w(W)o(ORLD)g Fp(is)h(established)h(in)f(the)f(c)o(hil-) 75 1480 y(dren.)41 b(It)23 b(is)f(erroneous)h(for)e(c)o(hildren)j(spa)o (wned)f(with)f Fg(MPI)p 1181 1480 V 16 w(MPI)g Fp(not)f(to)h(call)h Fg(MPI)p 1623 1480 V 16 w(INIT)p Fp(.)e(If)i(the)75 1536 y(program)16 b(named)h(in)h Fg(command)p 650 1536 V 16 w(line)f Fp(do)q(es)h(not)f(call)h Fg(MPI)p 1103 1536 V 16 w(INIT)p Fp(,)e(but)h(instead)h(forks)e(a)h(pro)q(cess)h(that)75 1593 y(calls)i Fg(MPI)p 265 1593 V 16 w(INIT)p Fp(,)e(the)i(results)f (are)g(unde\014ned.)34 b(Some)19 b(implemen)o(tations)i(ma)o(y)d(allo)o (w)i(this)f(case)h(to)75 1649 y(w)o(ork.)189 1740 y Fe(A)n(dvic)n(e)15 b(to)i(users.)41 b Fp(MPI)15 b(do)q(es)h(not)f(sa)o(y)g(what)f(happ)q (ens)j(if)f(the)f(program)f(y)o(ou)h(start)f(is)i(a)f(shell)189 1796 y(script)20 b(and)g(that)f(shell)i(script)f(starts)e(a)i(program)e (that)h(calls)i Fg(MPI)p 1409 1796 V 16 w(INIT)p Fp(.)d(Some)i (implemen-)189 1853 y(tations)d(ma)o(y)h(allo)o(w)g(y)o(ou)g(to)g(do)g (this,)h(but)f(ma)o(y)f(ha)o(v)o(e)h(restrictions,)h(suc)o(h)g(as)e (requiring)j(that)189 1909 y(argumen)o(ts)14 b(supplied)k(to)d(the)h (shell)h(script)f(m)o(ust)f(b)q(e)h(supplied)i(to)d(the)h(program,)e (or)h(requiring)189 1966 y(that)f(certain)i(parts)e(of)h(the)g(en)o (vironmen)o(t)h(not)f(b)q(e)h(c)o(hanged.)k(\()p Fe(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fp(\))166 2056 y(F)l(or)e(ev)o(ery)i (group)e(spa)o(wned)i(with)f(the)h Fg(MPI)p 947 2056 V 15 w(MPI)f Fp(\015ag,)f(the)i(paren)o(t)f(m)o(ust)f(call)j(either)75 2113 y Fg(MPI)p 160 2113 V 16 w(CHILD)p 308 2113 V 15 w(A)l(TT)l(A)o(CH)h Fp(or)f Fg(MPI)p 655 2113 V 16 w(CHILD)p 803 2113 V 16 w(DET)l(A)o(CH)g Fp(as)g(describ)q(ed)i(b)q(elo)o(w.)26 b(It)18 b(is)f(legal)h(to)f(rearrange)75 2169 y(these)i(groups)g(with)h (the)f(group)g(functions)h(\(e.g.,)e Fg(MPI)p 1072 2169 V 16 w(GROUP)p 1240 2169 V 18 w(UNION)p Fp(\))h(prior)h(to)e(the)h (attac)o(h)f(or)75 2226 y(detac)o(h.)i(In)c(this)f(case,)g(the)g (requiremen)o(t)h(is)f(that)g(ev)o(ery)g(pro)q(cess)g(spa)o(wned)g (with)h(the)f Fg(MPI)p 1695 2226 V 16 w(MPI)f Fp(\015ag)75 2282 y(m)o(ust)h(b)q(e)g(either)h(attac)o(hed)f(or)g(detac)o(hed)g (exactly)h(once.)166 2339 y(If)h Fg(MPI)p 298 2339 V 16 w(NONMPI)g Fp(is)h(sp)q(eci\014ed,)h(the)e(spa)o(wned)g(pro)q (cesses)h(are)e(assumed)i(to)e(b)q(e)i(non-MPI)f(pro-)75 2395 y(cesses.)36 b(This)22 b(means)e(that)g Fg(MPI)p 681 2395 V 16 w(SP)l(A)-5 b(WN)21 b Fp(is)g(not)g(required)g(to)f(do)h (an)o(y)f(sp)q(ecial)j(setup)d(to)g(ensure)75 2452 y(that)15 b(a)g(call)i(to)e Fg(MPI)p 437 2452 V 16 w(INIT)g Fp(in)h(the)g(c)o (hildren)h(will)g(b)q(e)g(able)f(to)f(establish)i(comm)o(unication)f (among)f(the)75 2508 y(c)o(hildren.)22 b(The)15 b(e\013ect)g(of)g (calling)i Fg(MPI)p 760 2508 V 16 w(INIT)d Fp(in)i(the)g(c)o(hildren)h (is)e(unde\014ned.)166 2647 y Fl(Discussion:)48 b Fk(There)18 b(will)e(b)q(e)i(a)g(prop)q(osal)f(that)g Fi(MPI)p 1093 2647 13 2 v 14 w(INIT)h Fk(in)e(the)i(c)o(hild)f(creates)j(a)d (singleton)g(MPI)75 2704 y(application.)g(The)d(curren)o(t)h(draft)f (do)q(es)h(not)e(sp)q(ecify)i(this.)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 1852 -100 a Fp(3)166 49 y(The)22 b Fg(command)p 454 49 14 2 v 15 w(line)h Fp(argumen)o(t)e(is)h(a)g(string)g(con)o (taining)h(a)e(whitespace-separated)i(list)g(of)e(a)75 106 y(program)16 b(name)i(and)f(argumen)o(ts)g(whic)o(h)h(are)f(passed) g(to)g(the)h(program.)25 b(Argumen)o(ts)17 b(are)g(supplied)75 162 y(to)f(the)h(program)f(if)h(this)g(is)g(allo)o(w)o(ed)g(b)o(y)g (the)g(op)q(erating)g(system.)24 b(In)17 b(C,)f(the)h(argumen)o(ts)f (app)q(ear)h(in)75 219 y(the)f(argv[])f(argumen)o(t)g(of)h(main\(\).)22 b(If)17 b(a)e(F)l(ortran)g(implemen)o(tation)j(supplies)g(routines)f (that)e(allo)o(w)h(a)75 275 y(program)c(to)g(obtain)i(its)f(argumen)o (ts,)f(the)h(argumen)o(ts)g(ma)o(y)f(b)q(e)i(a)o(v)m(ailable)h(through) d(that)h(mec)o(hanism.)75 332 y(In)f(C,)g(if)g(the)g(op)q(erating)g (system)g(do)q(es)g(not)f(supp)q(ort)h(command)g(line)i(argumen)o(ts)d (app)q(earing)h(in)h(argv[],)75 388 y(and)k(if)g(the)g(program)f(is)h (an)g(MPI)g(program,)f(the)h(MPI)g(implemen)o(tation)h(ma)o(y)e(add)h (the)g(argumen)o(ts)75 444 y(to)e(the)g(argv[])e(that)i(is)h(passed)f (to)g Fg(MPI)p 766 444 V 15 w(INIT)p Fp(.)166 501 y(The)k(follo)o(wing) h(legalese)h(de\014nes)f(all)h(the)e(sp)q(ecial)i(cases)e(and)h (ensures)g(that)e(it)i(is)g(p)q(ossible)h(to)75 557 y(sp)q(ecify)d(an)o (y)e(argumen)o(ts.)23 b(It)16 b(is)h(unnecessary)g(for)f(normal)h(use.) 24 b(Whitespace)17 b(is)g(de\014ned)h(as)e(for)g(the)75 614 y(isspace\(\))21 b(routine)g(in)h(the)f(standard)f(C)h(library)h (\(space,)f(form-feed,)h(new)f(line,)j(carriage)c(return,)75 670 y(tab\).)j(All)17 b(other)f(c)o(haracters)g(are)g(in)o(terpreted)h (literally)h(except)f(for)f(a)g(small)h(n)o(um)o(b)q(er)g(of)f(t)o(w)o (o-letter)75 727 y(com)o(binations)g(starting)e(with)i(\\)p Ff(\045)p Fp(")e(that)h(ha)o(v)o(e)g(sp)q(ecial)i(meaning.)143 817 y Fc(\017)23 b Fp(\\)p Ff(\045\045)p Fp(")14 b(is)i(replaced)g(b)o (y)f(\\)p Ff(\045)p Fp(")143 909 y Fc(\017)23 b Fp(\\)p Ff(\045)p Fp(")11 b(where)j(\\)p Ff()p Fp(")e(is)i(a)g(single)h(whitespace)g(c)o(haracter)e(is)i(replaced)189 966 y(b)o(y)g(that)f(whitespace)i(c)o(haracter)f(but)g(do)q(es)h(not)e (separate)h(items)h(in)g(the)f(command)g(line.)143 1058 y Fc(\017)23 b Fp(\\)p Ff(\045_)p Fp(")13 b(is)j(discarded.)k(Ho)o(w)o (ev)o(er,)14 b(it)h(is)g(treated)f(as)g(a)h(non-whitespace)h(c)o (haracter)d(in)j(that)e(it)h(can)189 1115 y(b)q(egin)h(a)f(new)g(w)o (ord.)20 b(Its)15 b(in)o(tended)h(use)g(is)g(to)e(allo)o(w)i(the)f(sp)q (eci\014cation)i(of)e(empt)o(y)g(strings.)75 1205 y(F)l(or)e(example,)i (the)g(command-line)g(\\)p Ff(foo ba\045 r \045_ q\045\045u\045\045_ux) p Fp(")c(\(where)j(\\)p Ff( )p Fp(")f(is)i(a)f(space)g(c)o(harac-)75 1261 y(ter\))g(sp)q(eci\014es)i(the)e(command)h(\\)p Ff(foo)p Fp(")e(with)i(argumen)o(ts)e(\\)p Ff(ba r)p Fp(",)g(\\",)h(and)g(\\)p Ff(q\045u\045_ux)p Fp(".)19 b(The)14 b(sp)q(ecial)75 1318 y(t)o(w)o(o-letter)g(com)o(binations)h (are)g(translated)g(as)f(they)h(are)g(encoun)o(tered)g(when)g(reading)h (the)f(string,)f(so)75 1374 y(that)h(\\)p Ff(\045\045_)p Fp(")g(is)i(translated)f(to)f(\\)p Ff(\045_)p Fp(",)h(not)f(\\)p Ff(\045)p Fp(".)22 b(Note)16 b(that)f(\\)p Ff(\045)p Fp(")h(not)f(follo)o(w)o(ed)i(b)o(y)f(\\)p Ff(\045)p Fp(",)f(\\)p Ff(\045)p Fp(",)g(\\)p Ff(_)p Fp(")g(or)75 1431 y(\\)p Ff()p Fp(")e(is)j(not)e(treated)h(sp)q(ecially) l(.)166 1570 y Fl(Discussion:)34 b Fk(The)14 b(format)e(ma)o(y)f(lo)q (ok)i(complicated)g(to)g(users)i(but)f(is)g(only)f(messy)g(if)g(they)h (w)o(an)o(t)g(to)f(do)75 1626 y(something)h(strange.)25 b(Ho)o(w)o(ev)o(er,)16 b(in)f(en)o(vironmen)o(ts)g(where)i(\014lenames) e(tend)h(to)f(ha)o(v)o(e)h(em)o(b)q(edded)g(whitespace)75 1683 y(this)e(\(and)g(an)o(y)f(other)i(\\command-l)o(ine")c(in)o (terfaces\))k(could)e(b)q(ecome)h(a)g(nigh)o(tmare.)189 1868 y Fe(A)n(dvic)n(e)j(to)i(users.)53 b Fp(Though)17 b Fg(MPI)p 818 1868 V 16 w(SP)l(A)-5 b(WN)18 b Fp(tak)o(es)f(a)g (command)p 1353 1868 V 16 w(line)j(as)d(an)g(argumen)o(t,)g(it)g(is)189 1924 y(imp)q(ortan)o(t)10 b(to)h(remem)o(b)q(er)g(that)g(it)g(is)h(not) f(in)o(terpreted)g(b)o(y)g(a)g(shell)i(suc)o(h)e(as)g(csh)h({)f(the)g (argumen)o(ts)189 1981 y(are)i(in)o(terpreted)h(literally)l(.)22 b(F)l(or)13 b(instance,)h(\\)p Ff(foo)23 b(>)h(bar)p Fp(")13 b(will)i(spa)o(wn)e(a)h(program)e(\\)p Ff(foo)p Fp(")h(with)189 2037 y(argumen)o(ts)18 b(\\)p Ff(>)p Fp(")h(and)g(\\)p Ff(bar)p Fp(",)g(but)h(will)h(not)e(redirect)h (output.)32 b(\\)p Ff(mpirun)23 b(-np)g(4)h(foo)p Fp(")18 b(will)189 2093 y(spa)o(wn)j Fg(maxp)o(ro)q(cs)h Fp(copies)h(of)e(the)h (program)f(\\)p Ff(mpirun)p Fp(".)39 b(The)22 b(application)i(will)g (exp)q(ect)f(to)189 2150 y(establish)f(comm)o(unication)f(with)h(the)f (\\)p Ff(mpirun)p Fp(")e(program,)i(not)g(with)g(the)g(pro)q(cesses)h (that)189 2206 y(ma)o(y)14 b(b)q(e)i(started)e(b)o(y)i(the)f Ff(mpirun)f Fp(program.)19 b(\()p Fe(End)c(of)i(advic)n(e)f(to)h (users.)p Fp(\))166 2308 y(MPI)e(tries)g(to)f(start)g Fg(maxp)o(ro)q(cs)g Fp(copies)i(of)f(the)g(program)f(sp)q(eci\014ed)j (in)e(the)h Fg(command)p 1676 2308 V 15 w(line)p Fp(.)k(MPI)75 2365 y(do)q(es)f(not)f(sp)q(ecify)i(ho)o(w)e(to)g(\014nd)i(the)f (executable)h(or)e(ho)o(w)g(the)h(w)o(orking)f(directory)h(is)g (determined.)75 2421 y(These)k(rules)h(will)g(b)q(e)f(implemen)o (tation-dep)q(end)q(en)o(t)i(and)e(will)h(b)q(e)g(appropriate)f(for)f (the)h(run)o(time)75 2478 y(en)o(vironmen)o(t.)d(F)l(or)12 b(instance,)i(a)f(homogeneous)g(system)g(with)h(a)f(global)h(\014le)g (system)f(migh)o(t)g(lo)q(ok)g(\014rst)75 2534 y(in)h(the)f(w)o(orking) f(directory)h(of)g(the)g(spa)o(wning)g(pro)q(cess,)g(or)g(migh)o(t)g (searc)o(h)f(the)h(directories)h(in)g(a)f(P)l(A)l(TH)75 2591 y(en)o(vironmen)o(t)e(v)m(ariable)g(as)f(do)g(Unix)h(shells.)20 b(An)10 b(implemen)o(tation)i(on)e(top)g(of)g(PVM)g(w)o(ould)g(use)h (PVM's)75 2647 y(rules)k(for)f(\014nding)h(executables)h(\(usually)f (in)g Ff(~/pvm3/bin/$PVM_ARCH)p Fp(\).)c(An)j(MPI)h(implemen)o(tation) 75 2704 y(running)h(under)g(IBM's)f(POE)h(will)g(use)g(POE's)f(metho)q (d)g(of)g(\014nding)i(executables.)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fp(4)951 b Fh(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)166 49 y Fp(While)20 b(MPI)f(alw)o(a)o(ys)f(tries)i(to)e (spa)o(wn)h Fg(maxp)o(ro)q(cs)f Fp(pro)q(cesses,)i(the)f Fg(minp)o(ro)q(cs)g Fp(argumen)o(t)f(allo)o(ws)75 106 y(some)d(additional)h(\015exibilit)o(y)l(.)23 b(If)15 b(MPI)g(is)g(unable)h(to)f(spa)o(wn)f Fg(maxp)o(ro)q(cs)g Fp(pro)q(cesses,)h(it)h(ma)o(y)e(spa)o(wn)g(as)75 162 y(few)d(as)f Fg(minp)o(ro)q(cs)p Fp(.)18 b(The)12 b(n)o(um)o(b)q(er)f (of)f(pro)q(cesses)h(actually)h(spa)o(wned)f(can)g(b)q(e)h(determined)g (b)o(y)f(examining)75 219 y(the)17 b(size)h(of)f(the)g(returned)g (group.)25 b(If)17 b(MPI)g(is)h(unable)g(to)e(spa)o(wn)h(at)f(least)h Fg(minp)o(ro)q(cs)p Fp(,)g Fg(MPI)p 1711 219 14 2 v 16 w(SP)l(A)-5 b(WN)75 275 y Fp(will)18 b(return)f(an)f(error.)23 b(It)17 b(ma)o(y)f(b)q(e)h(p)q(ossible)h(to)e(additionally)i(constrain) f(the)g(n)o(um)o(b)q(er)f(of)h(pro)q(cesses)75 332 y(spa)o(wned)e (through)g(the)g Fg(info)h Fp(argumen)o(t.)189 438 y Fe(A)n(dvic)n(e)g(to)i(users.)50 b Fp(An)17 b(MPI)f(implemen)o(tation)i (is)g(allo)o(w)o(ed,)f(but)g(is)g(not)f(required,)i(to)e(spa)o(wn)189 494 y Fg(minp)o(ro)q(cs)d Fp(pro)q(cesses)h(if)g Fg(maxp)o(ro)q(cs)e Fp(are)i(not)f(a)o(v)m(ailable.)21 b(Th)o(us)13 b(it)h(w)o(ould)g(not)f (b)q(e)h(p)q(ortable)g(to)f(try)189 551 y(to)i(acquire)i(all)g(a)o(v)m (ailable)h(resources)e(b)o(y)g(sp)q(ecifying)i(a)e(v)o(ery)g(large)g(v) m(alue)h(for)f Fg(maxp)o(ro)q(cs)f Fp(and)h(a)189 607 y(small)g(v)m(alue)g(for)f Fg(minp)o(ro)q(cs)p Fp(.)k(\()p Fe(End)d(of)g(advic)n(e)g(to)h(users.)p Fp(\))166 713 y(The)12 b Fg(info)g Fp(argumen)o(t)g(is)h(opaque)f(to)f(MPI)i(and)f (is)h(passed)f(directly)i(to)d(the)h(run)o(time)h(en)o(vironmen)o(t,)75 770 y(whic)o(h)g(uses)f(it)g(to)f(determine)h(where)g(and)g(ho)o(w)f (to)h(spa)o(wn)f(pro)q(cesses.)19 b(In)12 b(C)g(it)g(is)g(a)g (NULL-terminated)75 826 y(string.)24 b(If)17 b(the)g(info)g(argumen)o (t)f(is)h(empt)o(y)f(\(NULL)i(or)e(an)g(empt)o(y)h(string)f(in)i(C,)e (or)g(')p Ff( )p Fp(')g(F)l(ortran\))f(the)75 883 y(run)o(time)j(en)o (vironmen)o(t)g(decides)h(where)f(to)f(spa)o(wn)g(pro)q(cesses.)28 b(P)o(ortable)17 b(programs)f(not)i(requiring)75 939 y(detailed)f(con)o(trol)e(o)o(v)o(er)f(pro)q(cess)h(lo)q(cations)h (should)h(use)e(an)g(empt)o(y)g(argumen)o(t.)166 996 y(MPI)k(do)q(es)h(not)f(sp)q(ecify)i(the)e(format)g(of)g(the)g Fg(info)g Fp(argumen)o(t,)h(whic)o(h)g(is)g(left)g(up)g(to)e(the)i(MPI) 75 1052 y(implemen)o(tation.)31 b(The)19 b(format)e(is)i(exp)q(ected)h (to)e(b)q(e)h(tailored)h(to)e(the)g(particular)h(run)o(time)g(system)75 1109 y(b)q(eing)d(used.)21 b(Examples)16 b(of)e(an)h(info)h(string)f (migh)o(t)g(b)q(e)h(\\hostname=dop)q(ey")f(to)g(spa)o(wn)g(all)h(tasks) e(on)75 1165 y(a)g(host)h(named)g(\\dop)q(ey",)f(or)g(\\2:hippi:sgi")i (to)e(spa)o(wn)g(on)h(t)o(w)o(o)e(hippi-connected)18 b(SGI)d(mac)o(hines,)g(or)75 1222 y(ev)o(en)h(\\\014le=info.cfg")g(to)e (indicate)j(that)d(detailed)j(information)e(is)h(sp)q(eci\014ed)h(in)f (the)g(\014le)g(\\info.cfg".)166 1278 y(Alternativ)o(ely)l(,)f(it)e(is) h(ev)o(en)g(p)q(ossible)i(to)c(use)i(the)g Fg(info)f Fp(argumen)o(t)g(to)g(sp)q(ecify)h(the)g(executable)h(and)75 1334 y(its)f(command-line)h(argumen)o(ts,)d(in)i(whic)o(h)h(case)e(the) g Fg(command)p 1198 1334 V 16 w(line)g Fp(argumen)o(t)g(to)g Fg(MPI)p 1630 1334 V 16 w(SP)l(A)-5 b(WN)14 b Fp(can)75 1391 y(b)q(e)k(empt)o(y)l(.)26 b(The)17 b(abilit)o(y)i(to)e(do)g(this)g (follo)o(ws)h(from)e(the)i(fact)e(that)h(MPI)g(do)q(es)h(not)f(sp)q (ecify)h(ho)o(w)f(an)75 1447 y(executable)k(is)f(found,)h(and)f(the)g Fg(info)g Fp(argumen)o(t)f(can)h(tell)g(the)g(run)o(time)g(system)g (where)g(to)f(\\\014nd")75 1504 y(the)h(executable)g(\\".)33 b(Of)20 b(course)f(a)g(program)g(that)g(do)q(es)h(this)g(will)h(not)e (b)q(e)h(p)q(ortable)g(across)f(MPI)75 1560 y(implemen)o(tations.)166 1617 y(The)c Fg(a)o(rra)o(y)p 357 1617 V 15 w(of)p 409 1617 V 16 w(errco)q(des)h Fp(is)g(an)f(arra)o(y)f(of)h(length)h(at)e (least)i Fg(maxp)o(ro)q(cs)e Fp(in)j(whic)o(h)f(MPI)f(rep)q(orts)g(the) 75 1673 y(status)20 b(of)g(eac)o(h)h(pro)q(cess)g(that)f(MPI)h(w)o(as)f (requested)h(to)f(start.)35 b(If)21 b(all)h Fg(maxp)o(ro)q(cs)e Fp(pro)q(cesses)h(w)o(ere)75 1730 y(spa)o(wned,)16 b Fg(a)o(rra)o(y)p 370 1730 V 14 w(of)p 421 1730 V 16 w(errco)q(des)g Fp(is)h(\014lled)g(in)g(with)f(the)g(v)m(alue)h Fg(MPI)p 1204 1730 V 15 w(SUCCESS)p Fp(.)g(If)f(only)g Fb(m)g Fp(\()p Fg(minp)o(ro)q(cs)d Fc(\024)75 1786 y Fb(m)g(<)g Fg(maxp)o(ro)q(cs)o Fp(\))g(pro)q(cesses)i(are)f(spa)o(wned,)g Fb(m)g Fp(of)f(the)h(en)o(tries)h(will)g(con)o(tain)g Fg(MPI)p 1506 1786 V 15 w(SUCCESS)h Fp(and)e(the)75 1843 y(rest)g(will)j(con)o(tain)e(an)g(error)f(co)q(de)i(indicating)g(an)f (implemen)o(tation-de\014ned)j(reason)d(MPI)f(could)i(not)75 1899 y(start)g(the)i(pro)q(cess.)27 b(MPI)18 b(do)q(es)f(not)h(sp)q (ecify)g(whic)o(h)h(en)o(tries)f(corresp)q(ond)g(to)f(failed)h(pro)q (cesses.)28 b(An)75 1955 y(implemen)o(tation)19 b(ma)o(y)l(,)g(for)e (instance,)i(\014ll)h(in)f(error)e(co)q(des)i(in)g(one-to-one)f (corresp)q(ondence)i(with)e(a)75 2012 y(detailed)g(sp)q(eci\014cation)g (in)g(the)e Fg(info)g Fp(argumen)o(t.)23 b(These)17 b(error)f(co)q(des) h(all)g(b)q(elong)h(to)d(the)i(error)f(class)75 2068 y Fg(MPI)p 160 2068 V 16 w(ERR)p 261 2068 V 17 w(SP)l(A)-5 b(WN)16 b Fp(if)h(there)e(w)o(as)g(no)h(error)f(in)h(the)g(argumen)o(t) f(list.)22 b(An)15 b(application)j(ma)o(y)d(pass)g(\(in)o(t)75 2125 y(*\)NULL)h(\(C-binding\))g(for)f Fg(a)o(rra)o(y)p 671 2125 V 14 w(of)p 722 2125 V 16 w(errco)q(des)p Fp(,)g(in)i(whic)o (h)f(case)f(it)g(is)h(ignored)g(b)o(y)f(MPI.)166 2181 y(If)10 b(no)g(pro)q(cesses)h(are)f(spa)o(wned,)h Fg(MPI)p 804 2181 V 15 w(SP)l(A)-5 b(WN)11 b Fp(returns)f(the)h(empt)o(y)f (group)f Fg(MPI)p 1545 2181 V 16 w(GROUP)p 1713 2181 V 18 w(EMPTY)p Fp(.)166 2238 y(The)15 b(return)g(v)m(alue)i(\(C\))d (and)i Fg(ierr)e Fp(argumen)o(t)g(\(F)l(ortran\))g(ha)o(v)o(e)g(their)i (usual)g(meanings.)75 2294 y Fg(MPI)p 160 2294 V 16 w(SUCCESS)21 b Fp(means)e(that)g(at)g(least)g Fg(minp)o(ro)q(cs)h Fp(pro)q(cesses)g(w)o(ere)f(spa)o(wned)h(successfully)l(.)35 b(Other)75 2351 y(v)m(alues)16 b(indicate)h(an)e(error.)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 1852 -100 a Fp(5)75 49 y Fm(3.0.4)49 b(Dealing)18 b(with)e(MPI)h(p)o(ro)q(cesses)75 230 y Fg(MPI)p 160 230 14 2 v 16 w(CHILD)p 308 230 V 15 w(A)l(TT)l(A)o(CH\(group,)f(ro)q (ot,)e(comm,)f(intercomm\))117 307 y Fk(IN)155 b Fg(group)479 b Fk(group)17 b(of)g(spa)o(wned)h(pro)q(cesses)i(\(handle,)e (signi\014can)o(t)f(only)905 363 y(at)d Fi(ro)q(ot)p Fk(\))117 438 y(IN)155 b Fg(ro)q(ot)508 b Fk(pro)q(cess)21 b(rank)e(in)f Fi(comm)f Fk(on)i(whic)o(h)f Fi(group)i Fk(w)o(as)e(spa)o(wned)905 495 y(\(in)o(teger\))117 570 y(IN)155 b Fg(comm)470 b Fk(comm)o(unicator)9 b(o)o(v)o(er)j(whic)o(h)f (op)q(eration)h(is)f(collectiv)o(e)h(\(han-)905 626 y(dle\))117 702 y(OUT)108 b Fg(intercomm)384 b Fk(in)o(tercomm)o(unicator)20 b(b)q(et)o(w)o(een)j(pro)q(cesses)i(in)c Fi(comm)f Fk(and)905 758 y Fi(group)15 b Fk(\(handle\))75 882 y Ff(int)23 b(MPI)p 245 882 15 2 v 17 w(Child)p 382 882 V 17 w(attach\(MPI)p 639 882 V 16 w(Group)g(group,)g(int)g(root,)g(MPI)p 1275 882 V 17 w(Comm)g(comm,)393 939 y(MPI)p 468 939 V 17 w(Comm*)g(intercomm\))75 1025 y(MPI)p 150 1025 V 17 w(CHILD)p 287 1025 V 16 w(ATTACH\(GROUP,)f(ROOT,)i(COMM,)f(INTERCOMM,)f(IERR\)) 170 1082 y(INTEGER)h(GROUP,)g(ROOT,)h(COMM,)f(INTERCOMM,)f(IERR)166 1168 y Fp(This)13 b(function)g(establishes)h(comm)o(unication)f(b)q(et) o(w)o(een)g(the)f(pro)q(cesses)h(in)g Fg(comm)e Fp(and)i(the)f(newly)75 1225 y(spa)o(wned)19 b(pro)q(cesses)g(in)g Fg(group)p Fp(.)30 b(It)19 b(returns)g(an)f(in)o(tercomm)o(unicator)h(con)o (taining)g(the)g(paren)o(t)f(pro-)75 1281 y(cesses)e(\(those)e(in)i Fg(comm)p Fp(\))d(in)j(one)g(group)e(and)i(the)f(c)o(hild)i(pro)q (cesses)f(in)g(the)f(other.)166 1338 y(The)h(pro)q(cesses)g(in)g Fg(group)g Fp(m)o(ust)f(ha)o(v)o(e)g(b)q(een)i(spa)o(wned)f(with)g(the) f(\015ag)h Fg(MPI)p 1498 1338 14 2 v 15 w(MPI)f Fp(and)h(m)o(ust)f(not) 75 1394 y(ha)o(v)o(e)g(participapted)h(in)g(another)f(call)h(to)f Fg(MPI)p 904 1394 V 16 w(CHILD)p 1052 1394 V 15 w(A)l(TT)l(A)o(CH)h Fp(or)f Fg(MPI)p 1395 1394 V 16 w(CHILD)p 1543 1394 V 16 w(DET)l(A)o(CH)p Fp(.)166 1451 y Fg(MPI)p 251 1451 V 16 w(CHILD)p 399 1451 V 15 w(A)l(TT)l(A)o(CH)j Fp(is)f(collectiv)o(e) h(with)f Fg(MPI)p 1038 1451 V 16 w(INIT)e Fp(in)j(the)e(c)o(hildren.)26 b(All)18 b(the)e(pro)q(cesses)h(in)75 1507 y Fg(group)h Fp(constitute)g(a)g(single)i Fg(MPI)p 672 1507 V 15 w(COMM)p 829 1507 V 17 w(W)o(ORLD)e Fp(whic)o(h)h(is)f(separate)g(from)f(that)h (of)g(the)g(paren)o(t.)75 1563 y(Pro)q(cess)g(ranks)g(in)g(the)h(c)o (hildren's)g Fg(MPI)p 796 1563 V 16 w(COMM)p 954 1563 V 16 w(W)o(ORLD)f Fp(are)g(iden)o(tical)i(to)d(those)h(in)h Fg(group)p Fp(.)28 b(The)75 1620 y(in)o(tercomm)o(unicator)20 b(b)q(et)o(w)o(een)h(paren)o(ts)f(and)h(c)o(hildren)h(ma)o(y)e(b)q(e)h (obtained)g(in)g(the)g(c)o(hildren)h(using)75 1676 y Fg(MPI)p 160 1676 V 16 w(P)l(ARENT)p Fp(.)75 1827 y Fg(MPI)p 160 1827 V 16 w(CHILD)p 308 1827 V 15 w(DET)l(A)o(CH\(group\))117 1905 y Fk(IN)155 b Fg(group)479 b Fk(group)17 b(of)g(spa)o(wned)h(pro)q (cesses)i(\(handle,)e(signi\014can)o(t)f(only)905 1961 y(at)d Fi(ro)q(ot)p Fk(\))75 2085 y Ff(int)23 b(MPI)p 245 2085 15 2 v 17 w(Child)p 382 2085 V 17 w(detach\(MPI)p 639 2085 V 16 w(Group)g(group\))75 2172 y(MPI)p 150 2172 V 17 w(CHILD)p 287 2172 V 16 w(DETACH\(GROUP,)f(IERR\))170 2228 y(INTEGER)h(GROUP,)g(IERR)166 2315 y Fp(This)d(function)g(sync)o (hronizes)g(with)g Fg(MPI)p 916 2315 14 2 v 16 w(INIT)f Fp(in)h(the)f(newly)h(spa)o(wned)g(pro)q(cesses)g(of)e Fg(group)p Fp(.)75 2371 y(Those)d(pro)q(cesses)h(form)e(a)h(single)i Fg(MPI)p 763 2371 V 15 w(COMM)p 920 2371 V 17 w(W)o(ORLD)p Fp(.)166 2428 y(No)11 b(MPI)h(comm)o(unication)g(is)g(p)q(ossible)i(b)q (et)o(w)o(een)e(the)f(spa)o(wned)h(pro)q(cesses)g(and)g(the)g(pro)q (cess)g(that)75 2484 y(spa)o(wned)18 b(them,)f(since)i(there)e(is)h(no) g(in)o(tercomm)o(unicator.)26 b(In)19 b(pro)q(cesses)e(of)g Fg(group)p Fp(,)h Fg(MPI)p 1687 2484 V 16 w(P)l(ARENT)75 2541 y Fp(returns)d Fg(MPI)p 317 2541 V 16 w(COMM)p 475 2541 V 16 w(NULL)p Fp(.)166 2597 y(The)h(pro)q(cesses)g(in)g Fg(group)g Fp(m)o(ust)f(ha)o(v)o(e)g(b)q(een)i(spa)o(wned)f(with)g(the) f(\015ag)h Fg(MPI)p 1498 2597 V 15 w(MPI)f Fp(and)h(m)o(ust)f(not)75 2654 y(ha)o(v)o(e)g(participapted)h(in)g(another)f(call)h(to)f Fg(MPI)p 904 2654 V 16 w(CHILD)p 1052 2654 V 15 w(A)l(TT)l(A)o(CH)h Fp(or)f Fg(MPI)p 1395 2654 V 16 w(CHILD)p 1543 2654 V 16 w(DET)l(A)o(CH)p Fp(.)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fp(6)951 b Fh(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)75 49 y Fg(MPI)p 160 49 14 2 v 16 w(P)l (ARENT\(intercomm\))117 126 y Fk(OUT)108 b Fg(intercomm)384 b Fk(paren)o(t)15 b(in)o(tercomm)o(unicator)c(\(handle\))75 251 y Ff(int)23 b(MPI)p 245 251 15 2 v 17 w(Parent\(MPI)p 502 251 V 16 w(Comm*)g(intercomm\))75 337 y(MPI)p 150 337 V 17 w(PARENT\(INTERCOMM,)e(IERR\))170 394 y(INTEGER)i(INTERCOMM,)g (IERR)166 480 y Fg(MPI)p 251 480 14 2 v 16 w(P)l(ARENT)f Fp(can)g(b)q(e)h(called)g(in)f(an)o(y)f(pro)q(cess.)40 b(If)22 b(there)g(is)g(a)f(paren)o(t,)h(\(i.e.,)h(the)f(pro)q(cess)75 537 y(w)o(as)15 b(started)g(with)h Fg(MPI)p 507 537 V 16 w(SP)l(A)-5 b(WN)16 b Fp(and)g(then)g(attac)o(hed)g(with)g Fg(MPI)p 1255 537 V 15 w(CHILD)p 1402 537 V 16 w(A)l(TT)l(A)o(CH)p Fp(\))g(it)g(returns)g(an)75 593 y(in)o(tercomm)o(unicator)d(that)e(w)o (as)h(formed)g(in)i Fg(MPI)p 928 593 V 16 w(INIT)p Fp(.)d(The)i(lo)q (cal)h(group)e(of)g(the)h(in)o(tercomm)o(unicator)75 650 y(consists)j(of)f(the)h(pro)q(cesses)h(attac)o(hed)e(with)h(the)g (same)f(call)i(to)f Fg(MPI)p 1281 650 V 15 w(CHILD)p 1428 650 V 16 w(A)l(TT)l(A)o(CH)p Fp(.)g(The)h(remote)75 706 y(group)c(consists)g(of)f(the)h(pro)q(cesses)h(that)e(co)q(op)q (erated)h(on)g(the)g(call)h(to)f Fg(MPI)p 1370 706 V 16 w(CHILD)p 1518 706 V 15 w(A)l(TT)l(A)o(CH)p Fp(.)h(If)f(there)75 763 y(is)g(no)g(paren)o(t,)g Fg(MPI)p 418 763 V 16 w(P)l(ARENT)h Fp(returns)e Fg(MPI)p 858 763 V 16 w(COMM)p 1016 763 V 17 w(NULL)p Fp(.)g Fg(MPI)p 1253 763 V 16 w(P)l(ARENT)i Fp(do)q(es)f(not)f(duplicate)j(an)75 819 y(existing)j(comm)o(unicator)e ({)h(it)g(merely)g(pro)o(vides)g(access)g(to)f(an)h(existing)h(comm)o (unicator.)24 b(Multiple)75 875 y(calls)14 b(to)e Fg(MPI)p 312 875 V 16 w(P)l(ARENT)h Fp(return)g(the)g(same)f(in)o(tercomm)o (unicator.)19 b(If)13 b(the)g(paren)o(t)f(in)o(tercomm)o(unicator)75 932 y(is)k(freed)f(with)h Fg(MPI)p 422 932 V 16 w(COMM)p 580 932 V 16 w(FREE)p Fp(,)f Fg(MPI)p 815 932 V 16 w(P)l(ARENT)h Fp(ma)o(y)f(return)g(an)g(in)o(v)m(alid)j(comm)o(unicator.)189 1038 y Fe(R)n(ationale.)82 b Fg(MPI)p 558 1038 V 16 w(P)l(ARENT)23 b Fp(basically)h(returns)e(a)g(v)m(alue)h(that)f(is)h(set)f(in)h Fg(MPI)p 1676 1038 V 16 w(INIT)p Fp(.)e(W)l(e)189 1095 y(could)14 b(ha)o(v)o(e)g(a)f(\\constan)o(t")g Fg(MPI)p 761 1095 V 16 w(COMM)p 919 1095 V 16 w(P)l(ARENT)i Fp(but)f(this)g(w)o (ouldn't)g(b)q(e)g(usable)h(as)f(a)f(static)189 1151 y(initializer,)29 b(since)c Fg(MPI)p 620 1151 V 16 w(INIT)e Fp(migh)o(t)h(need)h(to)f(set)f(the)i(v)m(alue)g(to)e Fg(MPI)p 1506 1151 V 16 w(COMM)p 1664 1151 V 17 w(NULL)p Fp(.)g(So)189 1208 y Fg(MPI)p 274 1208 V 15 w(P)l(ARENT)g Fp(just)d(returns)h(this)g(comm)o(unicator.)37 b(If)22 b(it)f(has)g(b)q(een)h(freed,)g Fg(MPI)p 1687 1208 V 16 w(P)l(ARENT)189 1264 y Fp(can't)14 b(\014nd)i(out)f(ab)q(out)g(it.) 189 1377 y Ff(MPI_Comm)22 b(__MPI_COMM_PARENT;)189 1433 y(MPI_Parent\(MPI_Comm)f(*comm\))189 1489 y({)260 1546 y(*comm)i(=)h(__MPI_COMM_PARENT;)189 1602 y(})189 1715 y Fp(\()p Fe(End)15 b(of)i(r)n(ationale.)p Fp(\))75 1837 y Fm(3.0.5)49 b(Nonblo)q(cking)18 b(requests)75 1922 y Fp(Spa)o(wning)e(new)g(pro)q(cesses)g(ma)o(y)f(b)q(e)h(an)g(exp)q (ensiv)o(e)h(op)q(eration.)k(In)16 b(order)f(to)g(allo)o(w)h(pro)q (cesses)g(to)f(do)75 1979 y(useful)20 b(w)o(ork)f(while)h(pro)q(cesses) g(are)f(b)q(eing)h(spa)o(wned,)g(MPI)f(pro)o(vides)h(a)f(non-blo)q(c)o (king)i(v)o(ersions)e(of)75 2035 y Fg(MPI)p 160 2035 V 16 w(SP)l(A)-5 b(WN)p Fp(.)166 2174 y Fl(Discussion:)32 b Fk(There)13 b(are)f(curren)o(tly)h(no)f(non-blo)q(c)o(king)f(v)o (ersions)h(of)f(the)i(attac)o(h)f(and)g(detac)o(h)g(functions.)75 2231 y(These)j(are)g(just)g(regular)f(collectiv)o(e)g(op)q(erations.)19 b Fi(MPI)p 965 2231 13 2 v 15 w(SP)m(A)-5 b(WN)14 b Fk(has)g(a)g (non-blo)q(c)o(king)f(v)o(ersion)h(since)h(it)f(migh)o(t)75 2287 y(b)q(e)f(extremely)g(exp)q(ensiv)o(e)h(\(e.g.,)e(submit)f(a)i (job)f(to)h(the)g(batc)o(h)g(queuing)g(system\).)18 b(Note)13 b(that)g Fi(MPI)p 1673 2287 V 14 w(ISP)m(A)-5 b(WN)12 b Fk(is)75 2344 y(not)i(collectiv)o(e.)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 1852 -100 a Fp(7)75 49 y Fg(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN\(command)p 542 49 V 15 w(line,)14 b(minp)o(ro)q(cs,)g(maxp)o(ro)q(cs,)e(\015ag,)i(info,)f(group,)h(a)o (rra)o(y)p 1473 49 V 14 w(of)p 1524 49 V 16 w(errco)q(des,)h(request\)) 117 183 y Fk(IN)155 b Fg(command)p 506 183 V 16 w(line)321 b Fk(executable)15 b(and)f(argumen)o(ts,)f(in)g(a)h(single)f(string)h (\(string\))117 258 y(IN)155 b Fg(minp)o(ro)q(cs)417 b Fk(minim)n(um)10 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)d(start)g (\(in)o(teger\))117 333 y(IN)155 b Fg(maxp)o(ro)q(cs)408 b Fk(maxim)n(um)10 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)d(start)g (\(in)o(teger\))117 408 y(IN)155 b Fg(\015ag)518 b Fi(MPI)p 982 408 13 2 v 15 w(MPI)13 b Fk(or)h Fi(MPI)p 1211 408 V 14 w(NOTMPI)f Fk(\(in)o(teger\))117 483 y(IN)155 b Fg(info)516 b Fk(a)14 b(string)g(telling)f(the)i(run)o(time)e(system)g (where)j(and)d(ho)o(w)h(to)905 540 y(start)h(the)f(pro)q(cesses)j (\(string\))117 615 y(OUT)108 b Fg(group)479 b Fk(group)14 b(con)o(taining)f(newly)h(spa)o(wned)g(pro)q(cesses)i(\(handle\))117 690 y(OUT)108 b Fg(a)o(rra)o(y)p 416 690 14 2 v 15 w(of)p 468 690 V 16 w(errco)q(des)268 b Fk(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))117 765 y(OUT)108 b Fg(request)452 b Fk(request)16 b(ob)r(ject)e(\(handle\))75 890 y Ff(int)23 b(MPI)p 245 890 15 2 v 17 w(Spawn\(char*)g(command)p 717 890 V 16 w(line,)g(int)g(minprocs,)g(int)h(maxprocs,)e(char*)h (info,)393 946 y(MPI)p 468 946 V 17 w(Group)g(group,)g(int*)g(array)p 1034 946 V 17 w(of)p 1099 946 V 17 w(errcodes,)g(MPI)p 1427 946 V 16 w(Request*)g(request\))75 1033 y(MPI)p 150 1033 V 17 w(SPAWN\(COMMAND)p 479 1033 V 15 w(LINE,)g(MINPROCS,)g (MAXPROCS,)g(INFO,)g(GROUP)g(ARRAY)p 1521 1033 V 16 w(OF)p 1585 1033 V 17 w(ERRCODES,)393 1089 y(REQUEST,)g(IERR\))170 1146 y(CHARACTER*\(*\))g(COMMAND)p 676 1146 V 16 w(LINE,)g(INFO)170 1202 y(INTEGER)g(MINPROCS,)g(MAXPROCS,)g(GROUP,)g(ARRAY)p 1129 1202 V 16 w(OF)p 1193 1202 V 17 w(ERRCODES\(*\),)f(REQUEST,)h (IERR)166 1289 y Fp(The)h(argumen)o(ts)e(of)h Fg(MPI)p 642 1289 14 2 v 16 w(ISP)l(A)-5 b(WN)24 b Fp(are)f(iden)o(tical)i(to)e (those)g(in)h Fg(MPI)p 1454 1289 V 16 w(SP)l(A)-5 b(WN)24 b Fp(except)g(that)75 1345 y(there)16 b(is)h(an)f(additional)i Fg(request)g Fp(argumen)o(t.)k(The)17 b(request)f(can)g(b)q(e)h(used)g (as)f(input)h(for)f Fg(MPI)p 1737 1345 V 15 w(W)l(AIT)p Fp(,)75 1401 y Fg(MPI)p 160 1401 V 16 w(TEST)p Fp(,)g(etc.)22 b(In)17 b(all)g(cases,)e(the)h(output)g Fg(group)g Fp(and)g(the)g Fg(a)o(rra)o(y)p 1277 1401 V 15 w(of)p 1329 1401 V 16 w(errco)q(des)g Fp(are)g(up)q(dated)h(asyn-)75 1458 y(c)o(hronously)f (and)f(are)g(not)g(v)m(alid)i(un)o(til)f(the)f(request)g(completes.)166 1514 y(The)f(status)f(ob)s(ject)h(returned)g(b)o(y)g Fg(MPI)p 858 1514 V 16 w(W)l(AIT)g Fp(con)o(tains)g(an)g(error)f(co)q (de)h(indicating)i(whether)e(or)75 1571 y(not)h(the)h(spa)o(wn)f(w)o (as)g(successful.)22 b(The)15 b(failure)i(or)e(success)h(of)f (individual)k(pro)q(cesses)d(is)g(indicated)h(in)75 1627 y Fg(a)o(rra)o(y)p 173 1627 V 15 w(of)p 225 1627 V 16 w(errco)q(des)p Fp(.)k(The)16 b(errorco)q(de)g(can)f(b)q(e)i(retriev)o (ed)f(as)f(the)h Fg(MPI)p 1280 1627 V 15 w(ERROR)i Fp(elemen)o(t)e(of)f (the)h(status)75 1684 y(ob)s(ject)f(returned)g(b)o(y)g Fg(MPI)p 545 1684 V 16 w(W)l(AIT)p Fp(,)g Fg(MPI)p 784 1684 V 16 w(TEST)p Fp(,)g(etc.)75 1805 y Fm(3.0.6)49 b(Spa)o(wn)17 b(Examples)75 1891 y Fp(In)j(the)f(follo)o(wing)i (examples,)f(arro)o(ws)e(indicate)j(collectiv)o(e)g(calls)g(b)q(et)o(w) o(een)e(paren)o(t)g(and)h(c)o(hild)h(pro-)75 1948 y(cesses.)31 b Fg(MPI)p 318 1948 V 15 w(SP)l(A)-5 b(WN)20 b Fp(is)f(called)h(on)f(a) f(single)i(paren)o(t)e(pro)q(cess,)i(while)g Fg(MPI)p 1448 1948 V 16 w(CHILD)p 1596 1948 V 15 w(A)l(TT)l(A)o(CH)g Fp(and)75 2004 y Fg(MPI)p 160 2004 V 16 w(CHILD)p 308 2004 V 15 w(DET)l(A)o(CH)15 b Fp(are)e(collectiv)o(e)j(o)o(v)o(er)d(a)g (comm)o(unicator)h(that)f(include)j(the)e(original)h(spa)o(wner.)131 2110 y(1.)22 b(P)o(aren)o(ts)14 b(spa)o(wn)h(c)o(hildren)i(and)e(comm)o (unicate)h(with)f(them:)189 2223 y Ff(Parents)786 b(Children)189 2279 y(-------)g(--------)189 2336 y(MPI_Spawn\(...,)22 b(MPI_MPI,)g(...\))189 2392 y(MPI_Child_attach\(\))69 b(<------------->)93 b(MPI_Init\(\))131 2505 y Fp(2.)22 b(P)o(aren)o(ts)14 b(spa)o(wn)h(indep)q(enden)o(t)i(c)o(hildren)g(whic) o(h)f(are)f(MPI)g(pro)q(cesses:)189 2617 y Ff(Parents)786 b(Children)189 2674 y(-------)g(--------)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fp(8)956 b Fh(CHAPTER)16 b(3.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)189 49 y Ff(MPI_Spawn\(...,)22 b(MPI_MPI,)g(...\))189 106 y(MPI_Child_detach\(\))69 b(<------------->)93 b(MPI_Init\(\))131 272 y Fp(3.)22 b(P)o(aren)o(ts)14 b(spa)o(wn)h(non-MPI)g(c)o(hildren:) 189 381 y Ff(Parents)786 b(Children)189 438 y(-------)g(--------)189 494 y(MPI_Spawn\(...,)22 b(MPI_NONMPI,)g(...\))131 604 y Fp(4.)g(P)o(aren)o(ts)12 b(spa)o(wn)g(m)o(ultiple)j(groups)e(of)f(c)o (hildren)j(that)e(share)g(a)f(common)h Fg(MPI)p 1542 604 14 2 v 16 w(COMM)p 1700 604 V 16 w(W)o(ORLD)189 660 y Fp(and)i(comm)o(unicate)h(with)f(them:)189 782 y Ff(Parents)786 b(Children)189 838 y(-------)g(--------)189 895 y(MPI_Spawn\(...,)22 b(MPI_MPI,)g(...,)i(&groupa\))189 951 y(MPI_Spawn\(...,)e(MPI_MPI,)g (...,)i(&groupb\))189 1064 y(MPI_Group_union\(groupa,)c(groupb,)j (&groupc\))189 1177 y(MPI_Child_attach\(groupc)o(,)e(...\))71 b(<--->)g(MPI_Init\(\))131 1299 y Fp(5.)22 b(Single)16 b(master)f(pro)q(cess)g(spa)o(wns)g(sla)o(v)o(es)g(dynamically:)189 1408 y Ff(MPI_Init\(\))189 1521 y(MPI_Parent\(&intercomm\))189 1634 y(if)23 b(\(intercomm)g(==)g(MPI_COMM_NULL\))f({)260 1691 y(MPI_Spawn\(...,)g(MPI_MPI,)h(...\))260 1747 y (MPI_Child_attach\(\))189 1803 y(})75 1946 y Fa(3.1)59 b(Examples)75 2048 y Fg(Manager-w)o(o)o(rk)o(er)13 b(example,)i(using)h (SP)l(A)-5 b(WN.)75 2139 y Ff(/*)24 b(manager)e(*/)75 2195 y(#include)h()75 2252 y(main\(int,)g(argc,)g(char)g (*arg[]\))75 2308 y({)147 2365 y(MPI_Status)f(status;)147 2421 y(int)h(world_size;)147 2478 y(MPI_Group)f(children;)147 2534 y(MPI_Comm)g(everyone;)262 b(/*)23 b(intercommunicator)f(*/)147 2647 y(MPI_Init\(&argc,)f(&argv\);)147 2704 y(MPI_Comm_size\(MPI_COMM_) o(WORLD,)f(&world_size\);)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fh(3.1.)29 b(EXAMPLES)1420 b Fp(9)147 49 y Ff(if)23 b(\(world_size)g(!=)g(1\))191 b(error\("Top)22 b(heavy)h(with)h(management"\);)147 106 y(if)f(\(MPI_UNIVERSE_SIZE)f (==)h(1\))h(error\("No)e(room)i(to)f(start)g(workers"\);)147 162 y(MPI_Spawn\("worker",)e(MPI_UNIVERSE_SIZE-1,)g (MPI_UNIVERSE_SIZE-1,)g(MPI_MPI,)385 219 y(&children,)i(NULL\))147 275 y(MPI_Child_attach\(childr)o(en,)e(0,)i(MPI_COMM_SELF,)f (&everyone\);)147 332 y(/*)170 388 y(*)i(Parallel)f(code)g(here.)g(The) h(communicator)e("everyone")h(can)g(be)h(used)170 444 y(*)g(to)g(communicate)e(with)h(the)h(spawned)f(processes,)f(which)h (have)h(ranks)f(0,..)170 501 y(*)h(MPI_UNIVERSE_SIZE-1)d(in)j(the)f (remote)h(group)f(of)g(the)h(intercommunicator)170 557 y(*)g("everyone".)170 614 y(*/)147 727 y(MPI_Finalize\(\);)75 783 y(})75 896 y(/*)g(worker)f(*/)75 1009 y(#include)g()75 1065 y(main\(int)g(argc,)g(char)g(*argv[]\))75 1122 y({)147 1178 y(MPI_Comm)f(myboss;)147 1235 y(MPI_Init\(&argc,)f(&argv\);)147 1348 y(MPI_Parent\(&myboss\);)147 1461 y(/*)170 1517 y(*)j(Parallel)f(code)g(here.)170 1574 y(*)h(The)g(manager)f(is)g (represented)f(as)i(the)g(process)e(with)i(rank)f(0)h(in)f(\(the)h (remote)170 1630 y(*)g(group)f(of\))h(``myboss''.)46 b(If)24 b(the)f(workers)g(need)g(to)h(communicate)e(among)170 1686 y(*)i(themselves,)f(they)g(can)g(either)g(extract)g(the)h(local)f (group)g(of)h(the)170 1743 y(*)g(intercommunicator)e(MPI_COMM_PARENT,)f (or)j(use)f(MPI_COMM_WORLD.)46 b(They)170 1799 y(*)24 b(should)f(be)h(the)f(same.)170 1856 y(*/)147 1969 y(MPI_Finalize\(\);) 75 2025 y(})75 2258 y Fg(T)l(ask)15 b(fa)o(rm)e(example.)44 b Fp(This)16 b(example)f(sho)o(ws)f(ho)o(w)g(to)g(manage)g(a)g(set)g (of)g(non-MPI)h(pro)q(cesses.)20 b(Note)75 2315 y(that)c(all)h(groups)g (con)o(tain)f(a)h(single)g(pro)q(cess)g(b)q(ecause)h(it)f(w)o(ould)g(b) q(e)g(a)f(mess)h(otherwise.)24 b(This)17 b(could)75 2371 y(b)q(e)d(\\impro)o(v)o(ed")f(b)o(y)g(using)i Fg(MPI)p 641 2371 14 2 v 15 w(SP)l(A)-5 b(WN)p 806 2371 V 18 w(MUL)l(TIPLE)p 1041 2371 V 15 w(INDEPENDENT)14 b Fp(but)f(this)h(w)o(ould)g(also)f (mak)o(e)75 2428 y(group)i(managemen)o(t)f(di\016cult.)75 2534 y Ff(#include)23 b()75 2590 y(#define)g(MAXPROC)g(1000)75 2647 y(main\(int)g(argc,)g(char)g(*argv[]\))75 2703 y({)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fp(10)928 b Fh(CHAPTER)16 b(3.)34 b(D)o(YNAMIC)15 b(PR)o(OCESSES)147 49 y Ff(MPI_Group)22 b(workers[MAXPROC];)147 106 y(MPI_Request)g(obituaries[MAXPROC];)147 162 y(int)h(nslots,)g(i,)h(deadone;)147 219 y(char)f(*args;)147 275 y(char)g(command_lines[MAXLEN];)147 388 y(ThingToDo)f(*thingstodo;) 147 444 y(int)h(nthingstodo,)f(nthingsdone;)147 557 y(MPI_Init\(&argc,) f(&argv\);)147 614 y(nslots)i(=)g(MPI_UNIVERSE_SIZE)f(-)i(1;)f(//)h (assume)f(only)g(one)h(copy)f(of)h(me!)147 670 y (GetThingsToDo\(&thingsto)o(do,)d(nthingstodo\);)147 783 y(/*)i(need)g(to)h(initialize)f(these)g(for)g(Waitany/all\(\))f (below)h(to)h(work)f(*/)147 840 y(for)g(\(i)h(=)f(0;)h(i)g(<)f (MAXPROC;)g(i++\))g(obituaries[i])g(=)g(MPI_REQUEST_NULL;)147 953 y(/*)g(start)g(up)h(initial)f(set)g(of)h(tasks)f(*/)147 1009 y(for)g(\(i)h(=)f(0;)h(i)g(<)f(nslots)g(&&)h(i)g(<)f(nthingstodo;) g(i++\))147 1065 y({)218 1122 y(//)h(we)f(want)h(to)f(run)h(the)f (command)g("cow")g(with)h(some)f(special)g(args)218 1178 y(SetArgs\(thingstodo[i],)e(&args\);)218 1235 y(sprintf\(command_line,) g("cow)i(\045s",)h(args\);)218 1291 y(MPI_Spawn\(command_line,)d(1,)i (1,)h(MPI_NONMPI,)e(NULL,)i(&workers[i],)e(NULL\);)218 1348 y(MPI_Process_monitor\(workers)o([i],)f(0,)i(MPI_PROCESS_DIED,)f (&obituaries[i]\);)147 1404 y(})147 1461 y(nthingsdone=i;)147 1574 y(/*)h(spawn)g(new)h(ones)f(as)h(old)f(ones)g(finish)h(*/)147 1630 y(while\(nthingsdone)d(<)j(nthingstodo\))147 1686 y({)218 1743 y(MPI_Waitany\(nslots,)d(obituaries,)i(&deadone,)f (NULL\);)218 1799 y(SetArgs\(thingstodo[nthingsd)o(one],)f(&args\);)218 1856 y(sprintf\(command_lines[0],)g("cow)i(\045s",)g(args\))218 1912 y(MPI_Spawn\(command_line,)e(1,)i(1,)h(MPI_NONMPI,)e(NULL,)457 1969 y(&workers[deadone],)f(NULL\);)218 2025 y (MPI_Process_monitor\(workers)o([deadone)o(],)g(0,)j(MPI_PROCESS_DIED,) 457 2082 y(&obituaries[deadone]\);)147 2138 y(})147 2195 y(MPI_Waitall\(nslots,)d(obituaries,)h(NULL\);)147 2307 y(MPI_Finalize\(\);)75 2364 y(})75 2540 y Fg(PVM-st)o(yle)16 b(SPMD)f(example)45 b Fp(This)16 b(is)g(ho)o(w)f(man)o(y)f(PVM)h (programs)f(are)h(t)o(ypically)i(written.)j(There)75 2597 y(is)f(no)g(reason)f(they)h(can't)f(b)q(e)h(done)g(with)g(MPI-1)g (\(except)f(for)g(p)q(erformance!\),)h(but)g(in)g(case)g(users)75 2653 y(w)o(an)o(t)e(the)i(app)q(earance)f(of)g(minimal)i(c)o(hange,)f (here)g(is)f(is.)30 b(\(V)l(ery)18 b(similar)i(to)d(the)i(manager-w)o (ork)o(er)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fh(3.1.)29 b(EXAMPLES)1398 b Fp(11)75 49 y(example)16 b(ab)q(o)o(v)o(e.\))75 156 y Ff(/*)24 b(myprog)f(*/)75 268 y(#include)g()75 325 y(main\(int)g(argc,)g (char)g(*argv[]\))75 381 y({)147 438 y(MPI_Comm)f(everyone;)238 b(/*)23 b(intercommunicator)f(*/)147 494 y(MPI_Comm)g(new_world;)214 b(/*)23 b(corresponding)f(intracommunicator)g(*/)147 551 y(MPI_Comm)g(myparent;)147 607 y(MPI_Group)g(childgroup;)147 664 y(MPI_Init\(&argc,)f(&argv\);)147 777 y(MPI_Parent\(myparent\);)147 833 y(if)i(\(myparent)g(==)g(MPI_COMM_NULL\))46 b(/*)24 b(I'm)f(the)h(parent)f(*/)147 889 y({)218 946 y(MPI_Spawn\("myproc",)e (MPI_UNIVERSE_SIZE-1,)h(MPI_UNIVERSE_SIZE-1,)457 1002 y(MPI_MPI,)h(NULL,)g(&childgroup,)f(NULL\);)218 1059 y(MPI_Child_attach\(&childgrou)o(p,)f(0,)j(MPI_COMM_SELF,)e (&everyone\);)218 1115 y(MPI_Intercomm_merge\(everyon)o(e,)f(FALSE,)i (&new_world\);)147 1172 y(})147 1228 y(else)218 1285 y(MPI_Intercomm_merge\(myparen)o(t,)e(TRUE,)i(&new_world\);)170 1398 y(/*)h(SPMD)f(parallel)g(code)g(here,)h(using)f(new_world)f (instead)h(of)h(MPI_COMM_WORLD)e(*/)170 1510 y(MPI_Finalize\(\);)75 1567 y(})75 1743 y Fg(Simple)e(client-server)i(example.)44 b Fp(This)21 b(is)g(a)g(simple)h(example;)h(the)e(serv)o(er)f(accepts)g (only)h(a)g(single)75 1800 y(connection)14 b(at)d(a)i(time)g(and)f (serv)o(es)h(that)e(connection)j(un)o(til)g(the)e(clien)o(t)i(requests) f(to)e(b)q(e)j(disconnected.)75 1856 y(The)h(serv)o(er)g(is)h(a)f (single)h(pro)q(cess.)166 1913 y(Here)21 b(is)h(the)g(serv)o(er.)38 b(It)21 b(accepts)h(a)f(single)h(connection)h(and)e(then)h(pro)q (cesses)g(data)e(un)o(til)j(it)75 1969 y(receiv)o(es)16 b(a)f(message)g(with)g(tag)f Ff(1)p Fp(.)20 b(A)15 b(message)g(with)h (tag)e Ff(0)h Fp(tells)h(the)g(serv)o(er)e(to)h(exit.)75 2063 y Ff(#include)23 b("mpi.h")75 2120 y(main\()g(int)h(argc,)f(char)g (**argv)g(\))75 2176 y({)170 2232 y(MPI_Comm)g(client;)170 2289 y(MPI_Status)g(status;)170 2345 y(MPI_Port)g(port;)170 2402 y(char)h(port_name[100];)170 2458 y(double)f(buf[MAX_DATA];)170 2515 y(char)71 b(internal_name[MAX_NAME_SIZE],)20 b (published_name[MAX_NAME_SIZE])o(;)170 2571 y(int)95 b(size,)24 b(again;)170 2684 y(MPI_Init\()f(&argc,)g(&argv)g(\);)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fp(12)933 b Fh(CHAPTER)16 b(3.)29 b(D)o(YNAMIC)15 b(PR)o(OCESSES)170 49 y Ff (MPI_Comm_size\(MPI_COMM_WORLD,)20 b(size\);)170 106 y(if)k(\(size)f(!=)h(0\))f(error\(FATAL,)g("Server)g(too)g(big"\);)170 162 y(MPI_Port_open\(NULL,)f(&port\);)170 219 y(MPI_Port_name\(port,)g (port_name,)g(100\);)170 275 y(printf\("server)g(available)h(at)h (\045s\\n",port_name\);)170 332 y(while)g(\(1\))f({)266 388 y(MPI_Accept\()f(port,)h(NULL,)h(NULL,)f(0,)g(MPI_COMM_WORLD,)f (&client)h(\);)266 444 y(again)g(=)h(1;)266 501 y(while)f(\(again\))g ({)361 557 y(MPI_Recv\()g(buf,)g(MAX_DATA,)g(MPI_DOUBLE,)f (MPI_ANY_SOURCE,)600 614 y(MPI_ANY_TAG,)g(client,)h(&status)g(\);)361 670 y(switch)g(\(status.tag\))g({)457 727 y(case)g(0:)h (MPI_Comm_free\()e(&client)h(\);)648 783 y(MPI_Finalize\(\);)648 840 y(return)g(0;)457 896 y(case)g(1:)h(MPI_Comm_disconnect\()d (&client)i(\);)648 953 y(again)g(=)h(0;)648 1009 y(break;)457 1065 y(case)f(2:)h(/*)f(do)h(something)f(*/)457 1122 y(...)457 1178 y(default:)648 1235 y(MPI_Abort\()f(MPI_COMM_WORLD,)910 1291 y("Unexpected)h(message)f(type")i(\);)457 1348 y(})361 1404 y(})266 1461 y(})75 1517 y(})166 1611 y Fp(Here)15 b(is)h(the)f(clien)o(t.)75 1717 y Ff(#include)23 b("mpi.h")75 1774 y(main\()g(int)h(argc,)f(char)g(**argv)g(\))75 1830 y({)170 1887 y(MPI_Comm)g(server;)170 1943 y(double)g(buf[MAX_DATA];) 170 1999 y(char)h(port_name[MAX_NAME_SIZE])o(;)170 2112 y(MPI_Init\()f(&argc,)g(&argv)g(\);)170 2169 y(strcpy\(port_name,)f (argv[1])h(\);/*)g(assume)g(server's)g(name)g(is)h(cmd-line)f(arg)g(*/) 170 2282 y(MPI_Connect\()g(port_name,)f(NULL,)h(NULL,)h(0,)f (MPI_COMM_WORLD,)f(&server)h(\);)170 2395 y(while)h(\(!done\))f({)266 2451 y(tag)g(=)h(2;)g(/*)f(Action)g(to)h(perform)f(*/)266 2508 y(MPI_Send\()g(buf,)g(n,)g(MPI_DOUBLE,)g(0,)g(tag,)h(server)f(\);) 266 2564 y(/*)g(etc)h(*/)266 2620 y(})170 2677 y(MPI_Send\()f(buf,)g (0,)h(MPI_DOUBLE,)e(0,)i(1,)g(server)f(\);)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fh(3.1.)34 b(EXAMPLES)1393 b Fp(13)170 49 y Ff(MPI_Comm_disconnect\()22 b(&server)g(\);)170 106 y(MPI_Finalize\(\);)75 162 y(})166 268 y Fp(If)17 b(the)g(serv)o(er)f(needs)i(to)e(manage)g(m)o(ultiple)i(connections)g (at)e(once,)h(it)g(can)g(use)g Fg(MPI)p 1675 268 14 2 v 16 w(IA)o(CCEPT)75 325 y Fp(instead.)j(The)c(clien)o(t)g(need)g(not)f (b)q(e)h(c)o(hanged.)75 445 y Fg(Elab)q(o)o(rate)g(client-server)g (example.)45 b Fp(This)16 b(example)h(should)f(sho)o(w)f(a)g(concurren) o(t)h(parallel)h(serv)o(er,)e(in)75 501 y(whic)o(h)i(a)f(serv)o(er)g (spa)o(wns)g(a)g(set)h(of)f(pro)q(cesses)g(to)g(clone)i(itself)f(to)f (manage)f(the)i(clien)o(t,)h(while)f(it)g(go)q(es)75 558 y(bac)o(k)j(in)o(to)g(iaccept)h(mo)q(de.)35 b(The)21 b(complicated)g(part)f(is)g(to)g(set)g(up)g(the)h(in)o(tercomm)o (unicator)f(that)75 614 y(connects)c(the)f(clien)o(t)h(with)g(the)f (clone)h(of)f(the)g(serv)o(er,)f(while)j(getting)e(the)g(original)i (serv)o(er)d(out)h(of)g(the)75 671 y(w)o(a)o(y)l(.)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Thu May 9 08:56:15 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA08030; Thu, 9 May 1996 08:56:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA21375 for mpi-core-out; Thu, 9 May 1996 07:57:04 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA21370 for ; Thu, 9 May 1996 07:56:54 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id HAA04578; Thu, 9 May 1996 07:56:53 -0500 Date: Thu, 9 May 1996 07:56:53 -0500 Message-Id: <199605091256.HAA04578@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: schedule/deadline for new MPI-2 document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As we attempt to start converging on the MPI-2 standard and begin voting, it becomes increasingly important that people have time to digest each chapter and understand the interrelationships before the meeting. As a result, here is the way we hope the schedule will go: 1) Within 2 weeks after a meeting, each subcommittee will produce a new version of their chapter which reflects decisions made at the last meeting. 2) a discussion will held on the appropriate mailing list about the lastest chapter. 3) 2 weeks before the next meeting (i.e., giving at least 2 weeks for discussion of the latest chapter), each subcommittee will forward me a new version of their chapter. I will assemble the document and make it available electronically at that time. 4) Minor/modest changes can occur to this draft over the next week before I assemble the draft for printing and distribution at the next meeting. For the upcoming meeting on June 5-7, this translates into: The next draft (TeX) needs to be to me on Wednesday, 5/22 so I can assemble a complete document. I appreciate your help, Steve From mpi-core-human@mcs.anl.gov Fri May 10 06:20:56 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA27483; Fri, 10 May 1996 06:20:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA10827 for mpi-core-out; Fri, 10 May 1996 05:22:23 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id FAA10822; Fri, 10 May 1996 05:22:16 -0500 Message-Id: <199605101022.FAA10822@antares.mcs.anl.gov> To: mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: dynamic counter-proposal Date: Fri, 10 May 1996 05:22:16 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The "problem" has arisen in the following way: People like adding functionality, and have voted for it. Additional functionality requires more functions, or more flags to existing functions. People don't like flags. People don't like having lots of functions. Thus we have a contradiction. At the time of the March draft, a compromise had been reached that people seemed to be able to live with, with all the functionality that people wanted, and the number of functions kept down by using flags to overload the semantics of some functions. Because some people don't like flags, this draft was rewritten for April (not by popular demand, nor by straw votes at the March meeting, but as a (useful) experiment by Bill S.)), so that multiple functions replaced functions with flags. Straw votes at the April meeting approved these functions, although not overwhelmingly. Now people are worried about having lots of functions again, and have proposed a major change to MPI_Spawn in order to bring the number down. The right way to reduce the number of functions is by voting out the functionality, not by messing up the core functionality, which is what I think the new proposal does. To me the minimal functionality of this chapter is the ability to expand one's communicator. Our current MPI_SPAWN does this just exactly right. It is a single call, collective, and returns a communicator ready to use. We have begun implementing it, and it works fine. No daemons or other third parties are required, and it is easy to see how to interface it to process managers like LSF. In fact we have already worked out a portable process manager interface that will smooth implementation in other environments as well. (Remember that workstation networks are not the only environment, and that an implementation may not have access to the internals of the process manager.) For this to work, it is critical that the setting up of the communicator be done in conjunction with the creation of the processes. Separating them constrains the implementation so that it can no longer do everything at once at a time when it has the participation of all the other processes, and can devise a strategy for either succeeding (delivering a communicator) or failing as cleanly as it can figure out how to do. Even just from the user's point of view, MPI_Spawn( ... , newcomm) is more convenient than MPI_Spawn( ... , group) /*introduction of spurious new MPI object */ MPI_Child_Attach( ... group, newcomm) So this version of the basic functionality, which we have had a year to think about and have found satisfactory, is better for *both* users and implementors. It is most important to get this minimal core functionality right, and I think that we already have it right. Greater functionality requires more functions. If we want to spawn multiple executables, then we need a spawn with more arguments. If we decide that we don't want the resulting doubling of the number of functions, then we can keep only the current version of MPI_SPAWN_MULTIPLE and throw out MPI_SPAWN (or rename it) because it becomes a convenience function only. If we want to spawn independent, and/or non_MPI programs, then we need more functions (or flags) to do so, so let's vote on those again, if people want to, although those votes were already held once. I don't object to having more votes on these functions, even if we vote out functionality that I have voted for in the past. I do object to casually changing the vital core function just in order to try to squeeze the additional functionality into a smaller total number of functions. The draft as currently stands (April version with the small changes voted in at the April meeting) is ideally organized for taking this approach. Each additional capability is represented by a new function, so the cost of capability in number of functions is clear. Let us start with the spawn and argue over how to get this function right as if it were going to be the only function in the chapter (it might end up that way). In this argument I will argue very vigorously, primarily as an implementor but also on behalf of users, that the current spawn is right the way it is. After we have settled on what the spawn looks like (collective or not, returns communicator or group, names one executable or many), only then let us look at how we want to specify additional functionality (independent MPI, non-MPI, etc.) If we don't change the current spawn, then we will have straightforward re-votes on the rest of the functions in the chapter, as they are currently presented. (Some syntax and minor tweaking, like minprocs, remain to be discussed, but I think this would be straightforwardly settled by voting.) I think that we could have a first official vote on the chapter by Friday. Those who want to reduce the number of functions should vote out the functions that supply capability for spawning independent and non-MPI processes, or vote out the ability to spawn multiple executables, or vote out MPI_Spawn as a convenience function for the special case of MPI_Spawn_multiple. I'll vote for them, but I don't mind if I lose on these issues. If we *do* change the current spawn, then we are essentially starting over on the chapter, since we will have changed its single core function. At that point we will need to begin work on a new version of the chapter, perhaps using the new draft as a starting point. I think we could get through a first reading for the whole Forum by Friday, but not an official vote. So it would both speed things up and promote getting things right to have the draft at the June meeting reflect the current state of the chapter, which the small changes that were directed by straw votes in April. The new, counter-proposal should be held in reserve to be brought into the discussion only if we decide we want to radically change the current semantics of MPI_Spawn. Rusty From mpi-core-human@mcs.anl.gov Fri May 10 14:19:00 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA01903; Fri, 10 May 1996 14:17:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA19480 for mpi-core-out; Fri, 10 May 1996 13:19:18 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA19473; Fri, 10 May 1996 13:19:08 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id LAA23408; Fri, 10 May 1996 11:19:05 -0700 From: "William C. Saphir" Message-Id: <9605101119.ZM23406@win233.nas.nasa.gov> Date: Fri, 10 May 1996 11:19:04 -0700 In-Reply-To: Rusty Lusk "dynamic counter-proposal" (May 10, 5:22am) References: <199605101022.FAA10822@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: Rusty Lusk , mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: dynamic counter-proposal Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Rusty, Can you provide some details on the reasons you think the dynamic counter-proposal is so different from the current one? Sure, it looks different, but the functionality is almost exactly the same (intentionally), and, despite your suggestions to the contrary, I think the implementation will be only slightly different. In a previous note I explained in detail why the people who generated the proposal (including several implementors) thought there was very little difference. Yet you say "major change to MPI_Spawn", "messing up the core functionality", "essentially starting over". The only argument I have seen to support this is that the old spawn() becomes spawn+attach. If this is a serious problem then a very simple convenience function can fix it. On the implementors side I have yet to hear any specific argument. I don't disagree with your historical observations, but I draw a different conclusion from them. We have spent many months deciding what functionality we want and understanding the issues in great detail. The draft of the last meeting reflects this historical process and is (to some) a confused jumble of functions, each one targeted towards a different piece of functionality. It is only now, after having gone through the whole process that we are able to step back and make sense of what has been produced. The counter-proposal does that. It is not a departure, but a reorganization designed to clarify and unify. > Those who want to reduce the number of functions should vote out the functions > that supply capability for spawning independent and non-MPI processes, or vote > out the ability to spawn multiple executables, or vote out MPI_Spawn as a > convenience function for the special case of MPI_Spawn_multiple. I'll vote > for them, but I don't mind if I lose on these issues. If (and only if) there is strong support for the counter-proposal, proceeding with votes on the current proposal would be a waste of time. It is appropriate if the debate is over functionality, but the counter-proposal provides the same functionality with a very different set of functions. As someone who has criticized MPI-1 for its bloat, I want to make sure that we give ourselves a chance to step back and see the big picture before locking the historical record into a standard. Bill From mpi-core-human@mcs.anl.gov Fri May 10 15:35:33 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA02721; Fri, 10 May 1996 15:35:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA21695 for mpi-core-out; Fri, 10 May 1996 14:38:35 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA21657; Fri, 10 May 1996 14:37:39 -0500 Message-Id: <199605101937.OAA21657@antares.mcs.anl.gov> To: "William C. Saphir" cc: Rusty Lusk , mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: dynamic counter-proposal In-Reply-to: Message from "William C. Saphir" of "Fri, 10 May 1996 11:19:04 -0700." <9605101119.ZM23406@win233.nas.nasa.gov> Date: Fri, 10 May 1996 14:37:37 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am concerned about the implementation complexity of the spawn/attach approach in the case of an MPP with an interconnect that must be reconfigured in order to provide high-performance connections. I am ignoring the user-hostile approach of only permitting "spawn" into a preallocated partition. This interconnect might be a physical device or the way an OS keeps track of seperate executable objectss (I'm avoiding threads/processes deliberately). In this case, the interconnect will need to be configured twice: once when the new jobs are spawned, and once when the final collection of members are known (unless the user is lucky and the implementations expected case matched theirs). Now, this isn't a scalability problem since it can at worst (roughly) double the time to spawn. But it does add to complexity to the MPP case, which is the case that MPI should be targeting (there ARE better ways to do this stuff for workstation clusters if that's all you're interested in; just to start with, there are the fault-tolerance issues). It may add enough complexity that only the user-hostile approach is implemented. This would be a major lose for the user community that MPI should be serving. I'll make three other points: 1) When considering implementation issues the concern must not be just "can we do it with current products" but "do we think that reasonable implementations in the future of MPI will be able to do this?" This is of course much harder, but it is important in developing a standard. I am concerned that some high-performance implementations may find this approach awkward. 2) There are other issues that are likely to need to know if the spawn is creating processes that are an extension of an existing set of a new set. For example, the user may want to gang or co-schedule the new processes with the existing set. 3) An interesting model that is popping up on some shared memory machines is "threads with private memory". To the MPI user, this looks like an MPI process (since it has its own memory space); to the implementation, it offers major advantages in performance (both with shared memory and intelligent scheduling). If you are doing a spawn of the same executable for the purposes of expanding your communicator, you can create a new such "thread"; if you are spawning to create a new independent process, you want to fork/exec. It is too late to do this when the the attach/detach occurs. Technically, it is possible to change the operating system to allow threads to be stripped off into another process, but I can't see why it should be done just to support MPI. Also note that in both cases the user is likely to use the NULL info argument, making it impossible to know what to do until it is too late. Bill From mpi-core-human@mcs.anl.gov Fri May 10 17:27:48 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA05155; Fri, 10 May 1996 17:27:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA24915 for mpi-core-out; Fri, 10 May 1996 16:30:31 -0500 Received: from msr.EPM.ORNL.GOV (geist@msr.epm.ornl.gov [128.219.8.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA24910; Fri, 10 May 1996 16:30:23 -0500 Received: (from geist@localhost) by msr.EPM.ORNL.GOV (8.7.4/8.7.3) id RAA07513; Fri, 10 May 1996 17:30:21 -0400 (EDT) Date: Fri, 10 May 1996 17:30:21 -0400 (EDT) From: Al Geist Message-Id: <199605102130.RAA07513@msr.EPM.ORNL.GOV> To: mpi-dynamic@mcs.anl.gov Subject: Re: dynamic counter-proposal Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk First, I just want to voice my STRONG support for the new dynamic proposal that Bill sent out. So far Rusty is the only person on the dynamic committee that doesn't like the proposal, and I feel his arguments are biased towards meeting deadlines rather than designing the best API. (Whatever "best" is) >The new, counter-proposal should be held in reserve to be brought >into the discussion only if we decide we want to radically change >the current semantics of MPI_Spawn. My reading is the majority of the dynamic committee is in agreement to present the new proposal. It is on the table, it has to be discussed. The new proposal was hashed out by a group that included several implementors, users, and international members. It is simple, elegant, complete, and provides all the functionality that we have spent the last several meetings voting in. > The "problem" has arisen in the following way: > ...flags... many functions... No Rusty, the problem arose when we finally understood the functionality we wanted SPAWN to provide and we stepped back to see what the API looked like (and it wasn't a pretty site). We realized the functionality was possible without the complexity. >The right way to reduce the number of functions is by voting out the >functionality, not by messing up the core functionality. >...Greater functionality requires more functions. I do not like Rusty's approach of reducing the number of functions by voting out functionality. I know from PVM experience that users want the functionality that we have decided on. The new proposal shows that greater functionality does not require more functions. Rusty argues for the trusty old MPI_SPAWN that returns an intercomm. This can be provided as a simple convenience function using the new proposal's routines. >Our current MPI_SPAWN does this just exactly right. It is a >single call, collective, and returns a communicator ready to use. We have >begun implementing it, and it works fine. Great, you have done the simplest case Rusty. Let's talk again when you try to implement the fortran version of MPI_SPAWN_MULTIPLE_INDEPENDENT_MPI SGI, Convex, LAM, and Intel implementors were deeply involved in designing the new dynamic proposal. So I feel comfortable saying I speak for the implementors and users that the proposal should be given serious consideration. Al Geist ----- From mpi-core-human@mcs.anl.gov Sat May 11 03:03:50 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA09194; Sat, 11 May 1996 03:03:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA29545 for mpi-core-out; Sat, 11 May 1996 02:06:53 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA29524; Sat, 11 May 1996 02:05:39 -0500 Message-Id: <199605110705.CAA29524@antares.mcs.anl.gov> To: Al Geist cc: mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: dynamic counter-proposal In-Reply-to: Message from Al Geist of "Fri, 10 May 1996 17:30:21 -0400." <199605102130.RAA07513@msr.EPM.ORNL.GOV> Date: Sat, 11 May 1996 02:05:38 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | SGI, Convex, LAM, and Intel implementors were deeply involved | in designing the new dynamic proposal. So I feel comfortable | saying I speak for the implementors and users that the | proposal should be given serious consideration. Al, this is going too far. You certainly do NOT speak for the implementors; you are presenting the views of some implementors. As implementors with significant experience in implementing MPI on a wide range of platforms, both Rusty and I are concerned that the separation of function may cause problems for SOME systems (which aren't workstation clusters, where it is easy). I have outlined several cases already where there are difficulties. One of the major reasons for the procedural issues Rusty brings up is to ensure proper deliberation of the proposals; the separation of spawn and attach/detach is a significant change which at the very least require significant review. Bill From mpi-core-human@mcs.anl.gov Sun May 12 10:35:52 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA02536; Sun, 12 May 1996 10:35:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA09797 for mpi-core-out; Sun, 12 May 1996 09:37:34 -0500 Received: from msr.EPM.ORNL.GOV (geist@msr.epm.ornl.gov [128.219.8.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA09775; Sun, 12 May 1996 09:35:16 -0500 Received: (from geist@localhost) by msr.EPM.ORNL.GOV (8.7.4/8.7.3) id KAA24179; Sun, 12 May 1996 10:35:14 -0400 (EDT) Date: Sun, 12 May 1996 10:35:14 -0400 (EDT) From: Al Geist Message-Id: <199605121435.KAA24179@msr.EPM.ORNL.GOV> To: geist@msr.epm.ornl.gov, gropp@mcs.anl.gov Subject: Re: dynamic counter-proposal Cc: mpi-core@mcs.anl.gov, mpi-dynamic@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >Al, this is going too far. You certainly do NOT speak for the implementors; >you are presenting the views of some implementors. This is correct Bill. It was just a counter to Rusty's claim that he spoke for the implementors - and he certainly does NOT either. He speaks for only one implementor (but an important one). >the procedural issues Rusty brings up is to ensure proper >deliberation of the proposals; Procedure is important. There is a written proposal on the table. Since there have been no official votes on dynamic. Procedure says we have to discuss it. The same procedure says we don't delete the current dynamic proposal unless it is voted out. I think it is good that we are having a lot of discussion on the newsgroup so we don't have to spend as much valuable meeting time discussing the pros and cons of both proposals. Al Geist From mpi-core-human@mcs.anl.gov Tue May 14 04:01:35 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA12405; Tue, 14 May 1996 04:01:34 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA06276 for mpi-core-out; Tue, 14 May 1996 03:03:50 -0500 Received: from tbag.convex.com (tbag.convex.com [130.168.75.174]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA06251; Tue, 14 May 1996 03:02:16 -0500 Message-Id: <199605140802.DAA06251@antares.mcs.anl.gov> Received: by tbag.convex.com (1.39.111.2/16.2) id AA223081194; Tue, 14 May 1996 03:06:35 -0500 From: Raja Daoud Subject: Re: dynamic counter-proposal To: mpi-dynamic@mcs.anl.gov Date: Tue, 14 May 1996 3:06:34 CDT Cc: mpi-core@mcs.anl.gov In-Reply-To: <199605101937.OAA21657@antares.mcs.anl.gov>; from "William Gropp" at May 10, 96 2:37 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The way I understand it, Bill's comment against the split spawn/attach design can be summarized as: if I know at spawn-time the set of parents, I can do some optimizations. As examples of such optim., he presents: 1) reconfigurable switches, 2) gang scheduling, and 3) thread vs. process spawning. Another point was the doubt about future architectures. I'd like to address these points. While it is generally true that, given more information, we can pull further optimizations, we also have to draw the line somewhere. In this particular case, we voted out many resource management features and added the "info" argument. It compromises portability, but allows system-specific optimizations that may not be of general nature. This certainly fits the three cases above. An implementation for a system that supports any (or all) of the above features is free to define specific "info" arguments that give spawn the necessary hints, including providing it the eventual set of parents. In any event, having the old spawn doesn't solve the problem since a user may later merge communicators (not known at spawn time), thus requiring further 1) reconfiguration, 2) hints to scheduler to form new gang. Since these would have to be done at merge time, then they might just was well be also done at child_attach time (consider it another merge). This scenario makes the I-have-to-know-it-at-spawn- time argument weaker. The point about reconfigurable networks deserves special mention. The argument is that MPPs (high performance + scalability) would quickly restructure their switches to accomodate new process groupings. From my experience, this is a slow serialized process, requiring the network to be brought to a "quiet" state before changing paths and route tables (whether in s/w or h/w). So it seems this 'requirement' pulls in two opposite directions. The "user hostile" solution of configuring at application-startup time is usually there because of concerns about performance. Reconfiguration requires global coordination, and will be costly. I'm not comfortable with an argument using it to backup a concern about performance at spawn time (another typically slow call). In my opinion, the examples given are less prevalent scenarios than many other resource management issues that we tossed out from the earlier drafts. If for all of these we said "use the 'info' argument", the same approach should be used to handle the special three cases above. Eric partially addressed the thread-vs-process case, but even for that case, and on systems that exhibit different behaviour, the point is still: "if you need to know more for further optimizations, have the user pass it in the 'info' argument." This is still a resource management issue. A "high quality" implementation will provide all the extensions it needs in 'info' to pull all the optim. its resource manager has up its sleeves. To handle the "fear of the future" part: 1- At the risk of sounding like a broken record, "info" provides lots of flexibility for all esoteric systems. 2- Further MPI std. efforts will add the necessary extensions (think Fortran-Forever). 3- We all are experienced people, no matter the differences of opinions, the 'sky isn't falling'. [yet?] :-) I support Eric's proposal for the following reasons: - It provides all the functionality we have converged on (single/multiple, blocking/non-blocking, mpi/indep/non-mpi). - It does so with only 4 functions (not 12). - It provides an elegant "multiple" mechanism (group unions). - It's portably fast on the vast majority of parallel systems. - It's no less parallel than the old proposal since the spawn prerequisites are only valid at the root, a single point. - It's easy to extend its functionality for systems that require more info for further optimizations (using "info"). All this info-sharing is really one-way, from the root parent to the external resource manager, and thus can also be done from the new spawn call. --Raja From mpi-core-human@mcs.anl.gov Fri May 17 15:38:43 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA29771; Fri, 17 May 1996 15:35:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA17646 for mpi-core-out; Fri, 17 May 1996 14:37:51 -0500 Received: from win24.nas.nasa.gov (win24.nas.nasa.gov [129.99.33.39]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA17623 for ; Fri, 17 May 1996 14:37:11 -0500 Received: from localhost (nitzberg@localhost) by win24.nas.nasa.gov (8.6.12/NAS.6.1) with SMTP id MAA03143; Fri, 17 May 1996 12:36:52 -0700 Message-Id: <199605171936.MAA03143@win24.nas.nasa.gov> X-Authentication-Warning: win24.nas.nasa.gov: Host localhost didn't use HELO protocol To: mpi-core@mcs.anl.gov, mpi-io@nas.nasa.gov Cc: sio-hlapi@cacr.caltech.edu, sio-os-api@cs.princeton.edu Subject: MPI-2 I/O Chapter Proposal Date: Fri, 17 May 1996 12:36:51 -0700 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The following is a proposed starting point for the MPI-2 I/O chapter. All interested parties are encouraged to participate in this design process via the MPI mailing lists, news group, and forum meetings. See the MPI-2 Web pages for details at: http://www.mcs.anl.gov/Projects/mpi/mpi2/mpi2.html This proposal is essentially a reformatting of the MPI-IO Version 0.5 draft (available via http://lovelace.nas.nasa.gov/MPI-IO/) with the following changes: 1. Retained only sections 5 through 9, and a modified section 3. 2. Renamed functions, types, etc., from "MPIO_xxx" to "MPI_xxx". 3. Edited (minor changes) to conform to MPI-2 chapter style. 4. Added clarification regarding: file consistency semantics, the use of MPI_Comm in MPI_Open, ----- cut here ----- io-2.ps ----- cut here ----- %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%Pages: 46 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: /usr/local/bin/dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.05.16:2306 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet %%BeginProcSet: special.pro TeXDict begin /SDict 200 dict N SDict begin /@SpecialDefaults{/hs 612 N /vs 792 N /ho 0 N /vo 0 N /hsc 1 N /vsc 1 N /ang 0 N /CLIP 0 N /rwiSeen false N /rhiSeen false N /letter{}N /note{}N /a4{}N /legal{}N}B /@scaleunit 100 N /@hscale{@scaleunit div /hsc X}B /@vscale{@scaleunit div /vsc X}B /@hsize{/hs X /CLIP 1 N}B /@vsize{/vs X /CLIP 1 N}B /@clip{ /CLIP 2 N}B /@hoffset{/ho X}B /@voffset{/vo X}B /@angle{/ang X}B /@rwi{ 10 div /rwi X /rwiSeen true N}B /@rhi{10 div /rhi X /rhiSeen true N}B /@llx{/llx X}B /@lly{/lly X}B /@urx{/urx X}B /@ury{/ury X}B /magscale true def end /@MacSetUp{userdict /md known{userdict /md get type /dicttype eq{userdict begin md length 10 add md maxlength ge{/md md dup length 20 add dict copy def}if end md begin /letter{}N /note{}N /legal{} N /od{txpose 1 0 mtx defaultmatrix dtransform S atan/pa X newpath clippath mark{transform{itransform moveto}}{transform{itransform lineto} }{6 -2 roll transform 6 -2 roll transform 6 -2 roll transform{ itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll curveto}}{{ closepath}}pathforall newpath counttomark array astore /gc xdf pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{PaintBlack}if}N /txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 -1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop 90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr 2 get ppr 0 get neg sub neg 0 S TR}if}ifelse scaleby96{ppr aload pop 4 -1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S TR}if}N /cp {pop pop showpage pm restore}N end}if}if}N /normalscale{Resolution 72 div VResolution 72 div neg scale magscale{DVImag dup scale}if 0 setgray} N /psfts{S 65781.76 div N}N /startTexFig{/psf$SavedState save N userdict maxlength dict begin /magscale true def normalscale currentpoint TR /psf$ury psfts /psf$urx psfts /psf$lly psfts /psf$llx psfts /psf$y psfts /psf$x psfts currentpoint /psf$cy X /psf$cx X /psf$sx psf$x psf$urx psf$llx sub div N /psf$sy psf$y psf$ury psf$lly sub div N psf$sx psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub TR /showpage{}N /erasepage{}N /copypage{}N /p 3 def @MacSetUp}N /doclip{ psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto closepath clip newpath moveto}N /endTexFig{end psf$SavedState restore}N /@beginspecial{SDict begin /SpecialSave save N gsave normalscale currentpoint TR @SpecialDefaults count /ocount X /dcount countdictstack N}N /@setspecial {CLIP 1 eq{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs neg 0 rlineto closepath clip}if ho vo TR hsc vsc scale ang rotate rwiSeen{rwi urx llx sub div rhiSeen{rhi ury lly sub div}{dup}ifelse scale llx neg lly neg TR }{rhiSeen{rhi ury lly sub div dup scale llx neg lly neg TR}if}ifelse CLIP 2 eq{newpath llx lly moveto urx lly lineto urx ury lineto llx ury lineto closepath clip}if /showpage{}N /erasepage{}N /copypage{}N newpath }N /@endspecial{count ocount sub{pop}repeat countdictstack dcount sub{ end}repeat grestore SpecialSave restore end}N /@defspecial{SDict begin} N /@fedspecial{end}B /li{lineto}B /rl{rlineto}B /rc{rcurveto}B /np{ /SaveX currentpoint /SaveY X N 1 setlinecap newpath}N /st{stroke SaveX SaveY moveto}N /fil{fill SaveX SaveY moveto}N /ellipse{/endangle X /startangle X /yrad X /xrad X /savematrix matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix}N end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (temp.dvi) @start /Fa 3 62 df<07C018303018701C600C600CE00EE00EE00EE00EE00EE00EE00E E00EE00E600C600C701C30181C7007C00F157F9412>48 D<03000700FF00070007000700 070007000700070007000700070007000700070007000700070007007FF00C157E9412> I61 D E /Fb 1 1 df0 D E /Fc 5 116 df<003E000C000C000C000C0018001800180018073018F0307060706060C060C060 C06080C080C480C4C1C446C838700F177E9612>100 D<03000380030000000000000000 00000000001C002400460046008C000C0018001800180031003100320032001C0009177F 960C>105 D<383C1E0044C6630047028100460301008E0703000C0603000C0603000C06 0600180C0600180C0620180C0C20180C0C4030180440301807801B0E7F8D1F>109 D<1C3C22462382230346030603060306030C060C060C0C0C081A3019E018001800300030 003000FC001014808D12>112 D<07C00C201870187038001E000FC003E000606060E060 C0C0C1803F000C0E7E8D10>115 D E /Fd 1 82 df81 D E /Fe 8 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000F800 00F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115 >99 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F 001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018E018 F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Ff 68 123 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0 000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703 C00E03C00E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C07F87F8151D809C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<6060F0 F0F8F86868080808080808101010102020404080800D0C7F9C15>34 D<60F0F8680808081010204080050C7C9C0C>39 D<004000800100020006000C000C0018 001800300030007000600060006000E000E000E000E000E000E000E000E000E000E000E0 00E000600060006000700030003000180018000C000C00060002000100008000400A2A7D 9E10>I<800040002000100018000C000C000600060003000300038001800180018001C0 01C001C001C001C001C001C001C001C001C001C001C00180018001800380030003000600 06000C000C00180010002000400080000A2A7E9E10>I<60F0F070101010102020408004 0C7C830C>44 DI<60F0F06004047C830C>I<030007003F00C700 070007000700070007000700070007000700070007000700070007000700070007000700 07000700070007000F80FFF80D1C7C9B15>49 D<07C01830201C400C400EF00FF80FF807 F8077007000F000E000E001C001C00380070006000C00180030006010C01180110023FFE 7FFEFFFE101C7E9B15>I<07E01830201C201C781E780E781E381E001C001C0018003000 6007E00030001C001C000E000F000F700FF80FF80FF80FF00E401C201C183007E0101D7E 9B15>I<000C00000C00001C00003C00003C00005C0000DC00009C00011C00031C00021C 00041C000C1C00081C00101C00301C00201C00401C00C01C00FFFFC0001C00001C00001C 00001C00001C00001C00001C0001FFC0121C7F9B15>I<60F0F060000000000000000000 0060F0F06004127C910C>58 D<0FE03038401CE00EF00EF00EF00E000C001C0030006000 C0008001800100010001000100010001000000000000000000000003000780078003000F 1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F00000017800000 178000001780000023C0000023C0000023C0000041E0000041E0000041E0000080F00000 80F0000180F8000100780001FFF80003007C0002003C0002003C0006003E0004001E0004 001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E0618001801980070007800E0003801C0003801C0001803800 0180780000807800008070000080F0000000F0000000F0000000F0000000F0000000F000 0000F0000000F0000000700000807800008078000080380000801C0001001C0001000E00 0200070004000180080000E03000001FC000191E7E9C1E>II< FFFFFC0F003C0F000C0F00040F00040F00060F00020F00020F02020F02000F02000F0200 0F06000FFE000F06000F02000F02000F02000F02010F00010F00020F00020F00020F0006 0F00060F000C0F003CFFFFFC181C7E9B1C>II<001F808000E0618001801980070007800E0003801C0003801C000180380001807800 00807800008070000080F0000000F0000000F0000000F0000000F0000000F0000000F000 FFF0F0000F80700007807800078078000780380007801C0007801C0007800E0007800700 0B800180118000E06080001F80001C1E7E9C21>III75 DIII<003F 800000E0E0000380380007001C000E000E001C0007003C00078038000380780003C07800 03C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F000 01E0700001C0780003C0780003C0380003803C0007801C0007000E000E0007001C000380 380000E0E000003F80001B1E7E9C20>II< FFFF00000F01E0000F0078000F003C000F001C000F001E000F001E000F001E000F001E00 0F001C000F003C000F0078000F01E0000FFF00000F03C0000F00E0000F00F0000F007800 0F0078000F0078000F0078000F0078000F0078000F0078100F0078100F0038100F003C20 FFF01C20000007C01C1D7E9B1F>82 D<07E0801C1980300580700380600180E00180E000 80E00080E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007 C00003C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E 7E9C17>I<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F00 20800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00001F800003FFFC001B1C7F9B1E>IIII<7FF0FFC00FC03E000780180003C0180003E0 100001E0200001F0600000F0400000788000007D8000003D0000001E0000001F0000000F 0000000F8000000F80000013C0000023E0000021E0000041F00000C0F800008078000100 7C0003003C0002001E0006001F001F003F80FFC0FFF01C1C7F9B1F>II91 D<08081010202040404040808080808080B0B0F8 F8787830300D0C7A9C15>II<1FC000307000783800781C00 301C00001C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40 304E801F870012127E9115>97 DI< 07E00C301878307870306000E000E000E000E000E000E00060007004300418080C3007C0 0E127E9112>I<003F000007000007000007000007000007000007000007000007000007 0000070003E7000C1700180F00300700700700600700E00700E00700E00700E00700E007 00E00700600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C30181830 0C700E6006E006FFFEE000E000E000E00060007002300218040C1803E00F127F9112>I< 00F8018C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C 38C01C3880181800381C00381C00381C00381C001818001C38000C300013C00010000030 00001800001FF8001FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C 0E0007F800121C7F9215>II<1800 3C003C0018000000000000000000000000000000FC001C001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C000 000000000000000000000000000FE000E000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>I< FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C3FC0 1C0F001C0C001C08001C10001C20001C40001CE0001DE0001E70001C78001C38001C3C00 1C1C001C0E001C0F001C0F80FF9FE0131D7F9C16>IIII<03F0000E1C0018 0600300300700380600180E001C0E001C0E001C0E001C0E001C0E001C060018070038030 03001806000E1C0003F00012127F9115>II<03C1000C33 00180B00300F00700700700700E00700E00700E00700E00700E00700E007006007007007 00300F00180F000C370007C700000700000700000700000700000700000700000700003F E0131A7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000 F880388018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C001C00 3C00FFE01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C100E20 03C00C1A7F9910>IIII<7F8FF00F0380 0F030007020003840001C80001D80000F00000700000780000F800009C00010E00020E00 0607000403801E07C0FF0FF81512809116>II<7FFC7038 6038407040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E127F91 12>I E /Fg 29 117 df<001C0000003E0000003E0000002E0000006700000067000000 E7800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F00007 00F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038 001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F00001E00003C00003C00 00780000780000780000F00000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800003C00003C00001E00000F000807801803C07800FFF0003F80151F 7D9D1B>IIII<00 3F8001FFF003C0F80780380F00181E00003C00003C0000780000780000780000F00000F0 0000F00000F00000F00000F00000F007F8F007F8F000387800387800387800383C00383C 00381E00380F003807803803C0F801FFF0003F80151F7D9D1C>III75 DIII<003F000001 FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F00780007807800078078 000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C078000780780007807C000F803C000F003E001F001F003E000F807C0007C0F80003 FFF00001FFE000003F00001A1F7E9D1F>II<003F000001FFE00003FFF00007C0F8000F807C001F003E003E001F003C000F00 780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C07800078078000780780E07803C0F0F003E079F001E03DE00 0F83FC0007C1F80003FFF00001FFF800003F780000003C0000003E0000001F0000000F80 1A237E9D1F>II<03F8000FFE001C 0F00380700700300600000E00000E00000E00000E00000F000007800007F00003FE0001F FC0007FE0001FF00001F800007800003C00003C00001C00001C00001C00001C0C00180E0 0380F007007C0E001FFC0007F000121F7E9D17>II< F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070 F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F000707800E0 7800E03C01C01E03800F078007FE0001F800141E7C9C1D>III<78000E007C001E003C003C001E0038000F0070000F00F000 0781E00003C1C00001C3C00001E7800000F70000007E0000003E0000003C0000003C0000 007E00000077000000E7800001E3800003C1C0000381E0000700F0000F00F8000E007800 1C003C003C003E0078001F0070000F00F0000F80191D7F9C1C>II<7FFFF07FFFF00001E00003E00003C00007C0000780000F00001F00001E00 003E00003C0000780000F80000F00001F00001E00003C00007C0000780000F80000F0000 1E00003E00003C00007C0000780000FFFFF0FFFFF0141D7E9C19>I<07E00FF81FFC3C1C 70047000E000E000E000E000E000E000700070043C1C1FFC0FF807E00E127E9112>99 D 108 D110 D<1C001C001C001C001C001C00FFE0FFE01C001C001C001C00 1C001C001C001C001C001C001C001C001C201FF00FF007C00C187F970F>116 D E /Fh 2 16 df<400004C0000C6000183000301800600C00C006018003030001860000 CC0000780000300000300000780000CC000186000303000601800C00C018006030003060 0018C0000C40000416187A9623>2 D<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFF FFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D E /Fi 17 123 df<00F1800389C00707800E03801C03803C0380380700780700780700780700F0 0E00F00E00F00E00F00E10F01C20F01C20703C20705C40308C400F078014147E9318>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207E9F14>I< 007C01C207010E0F1E0F1C0E3C04780078007800F000F000F000F000F000700170023004 18380FC010147E9314>I<0000780003F80000700000700000700000700000E00000E000 00E00000E00001C00001C000F1C00389C00707800E03801C03803C038038070078070078 0700780700F00E00F00E00F00E00F00E10F01C20F01C20703C20705C40308C400F078015 207E9F18>I<007C01C207010E011C013C013802780C7BF07C00F000F000F000F0007000 700170023004183807C010147E9315>I<00007C0000CE00019E00039E00030C00070000 0700000700000700000E00000E00000E0000FFF0000E00000E00001C00001C00001C0000 1C00001C0000380000380000380000380000380000700000700000700000700000700000 E00000E00000E00000E00000C00001C000318000798000F300006200003C000017297E9F 16>I<00E001E001E000C000000000000000000000000000000E00130023804380438043 808700070007000E000E001C001C001C20384038403840388019000E000B1F7E9E10> 105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C 001C0038003800380038007000700070007100E200E200E200E200640038000A207E9F0E >108 D<1E07802318C023A06043C0704380704380708700E00700E00700E00700E00E01 C00E01C00E01C00E03821C03841C07041C07081C03083803101801E017147E931B>110 D<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003 C0F003C0F00780F00700700F00700E0030180018700007C00013147E9316>I<03C1E004 621804741C08781C08701E08701E10E01E00E01E00E01E00E01E01C03C01C03C01C03C01 C0380380780380700380E003C1C0072380071E000700000700000E00000E00000E00000E 00001C00001C0000FFC000171D819317>I<007C018203010603060706060E00078007F8 03FC01FE001F00077007F006F006E004400820301FC010147E9315>115 D<00C000E001C001C001C001C003800380FFF8038007000700070007000E000E000E000E 001C001C001C001C10382038203820384018800F000D1C7F9B10>I<0F00601180702180 E021C0E041C0E04380E08381C00701C00701C00701C00E03800E03800E03800E03840E07 080C07080C07080E0F1006131003E1E016147E931A>I<0F006060118070F02180E0F821 C0E07841C0E0384380E0188381C0100701C0100701C0100701C0100E0380200E0380200E 0380200E0380400E0380400E0380800E078080060781000709860001F078001D147E9321 >119 D<0F00601180702180E021C0E041C0E04380E08381C00701C00701C00701C00E03 800E03800E03800E03800E07000C07000C07000E0F00061E0003EE00000E00000E00001C 0078180078380070700060600021C0001F0000141D7E9316>121 D<01E02003F04007F8C00C1F8008010000020000040000080000100000600000C0000100 000200000400800801001003003F060061FC0040F80080700013147E9315>I E /Fj 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C060404060C0 31801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C00 0C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006 000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F000080 00400060C060C060804060801F000B107F8F0F>I<0300030007000F000B001300330023 004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C00200020002000 2F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C060 00C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC0 8080808001000200040004000C0008000800180018001800180018000B117E900F>I<1F 00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F >I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B10 7F8F0F>I E /Fk 49 122 df<00003F03E00000C386700001878CF00003879CF0000303 1860000700380000070038000007003800000E003800000E007000000E007000000E0070 0000FFFFFF80001C007000001C00E000001C00E000001C00E000001C00E000003800E000 003801C000003801C000003801C000003801C000007001C0000070038000007003800000 700380000070038000006003800000E007000000E007000000E007000000E007000000C0 06000001C00E000001C00E000031860C0000798F180000F31E100000620C6000003C07C0 00002429829F1C>11 D<00003FE00000E010000180380003807800030078000700300007 00000007000000070000000E0000000E0000000E000000FFFFE0000E00E0001C01C0001C 01C0001C01C0001C01C0001C038000380380003803800038038000380700003807000070 07000070071000700E2000700E2000700E2000E00E2000E0064000E0038000E0000000C0 000001C0000001C000003180000079800000F3000000620000003C0000001D29829F1A> I<0E1F3F3F1D0102020404081020C0080E779F0E>39 D<70F8F8F0E005057B840E>46 D<00000040000000C000000180000001800000030000000300000006000000060000000C 0000001800000018000000300000003000000060000000C0000000C00000018000000180 00000300000003000000060000000C0000000C0000001800000018000000300000003000 000060000000C0000000C0000001800000018000000300000003000000060000000C0000 000C0000001800000018000000300000003000000060000000C0000000C0000000800000 001A2D7FA117>I<000200020006000E003C00DC031C001C003800380038003800700070 0070007000E000E000E000E001C001C001C001C003800380038003800780FFF80F1E7B9D 17>49 D<001F800060E00080700100300200380420380420380410380420700460700380 600000E00001C000030000FE00001C000006000007000007800007800007803007807807 80780780F00F00800F00401E00401C0040380020E0001F8000151F7C9D17>51 D<0000600000E00000E00000E00001C00001C00001C00003800003800003000007000007 00000600000E00000C0000180000180000300000300000630000C7000087000107000307 00060E00040E00080E003F8E00607C00801FC0001C00001C000038000038000038000038 0000700000700000600013277E9D17>I<00C06000FFC001FF8001FE0001000001000002 0000020000020000020000040000047800058C000606000C070008070000078000078000 0780000780000F00700F00F00F00F00E00E01E00801C0080380080300040600061C0001F 0000131F7B9D17>I<001F0000718000C0C00180C00380E00700E00F00E00F01E01E01E0 1E01E01E01E01E01C01C03C01C03C01C03C01C07C01C0F800C0F8006378003C700000F00 000E00000E00001C00601C00F03800F07000E0600080C0004380003E0000131F7B9D17> 57 D<00000200000006000000060000000E0000001E0000001E0000003F0000002F0000 004F0000004F0000008F0000010F0000010F0000020F0000020F0000040F00000C0F0000 080F0000100F0000100F0000200F80003FFF800040078000C00780008007800100078001 0007800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFC0001E00F0001E0078001E0038001E003C003C003C003C003C003C003C003C00 3C0078007800780078007800F0007801E000F0078000FFFE0000F00F8000F003C001E001 C001E001E001E001E001E001E003C001E003C001E003C001E003C001C0078003C0078007 8007800F0007801E000F007800FFFFE0001E1F7D9E20>I<0000FE0200078186001C004C 0038003C0060003C00C0001C01C0001803800018070000180F0000181E0000101E000010 3C0000003C00000078000000780000007800000078000000F0000000F0000000F0000000 F0000000F00000807000008070000080700001003800010038000200180004000C001800 060020000381C00000FE00001F217A9F21>I<01FFFF80001E00E0001E0070001E003800 1E001C003C001C003C000E003C000E003C000E0078000E0078000E0078000E0078000E00 F0001E00F0001E00F0001E00F0001E01E0003C01E0003C01E0003C01E0007803C0007003 C0007003C000E003C001C0078001C00780038007800E0007801C000F007000FFFFC0001F 1F7D9E22>I<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C 0004003C00040078080800780800007808000078180000F0300000FFF00000F0300000F0 300001E0200001E0200001E0200001E0001003C0002003C0002003C0004003C000400780 00800780018007800100078007000F001F00FFFFFE001F1F7D9E1F>I<01FFFFFC001E00 38001E0018001E0008001E0008003C0008003C0008003C0008003C000800780010007808 00007808000078080000F0100000F0300000FFF00000F0300001E0200001E0200001E020 0001E0200003C0000003C0000003C0000003C00000078000000780000007800000078000 000F800000FFF800001E1F7D9E1E>I<01FFF3FFE0001F003E00001E003C00001E003C00 001E003C00003C007800003C007800003C007800003C007800007800F000007800F00000 7800F000007800F00000F001E00000FFFFE00000F001E00000F001E00001E003C00001E0 03C00001E003C00001E003C00003C007800003C007800003C007800003C007800007800F 000007800F000007800F000007800F00000F801F0000FFF1FFE000231F7D9E22>72 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078000078 0000780000F00000F00000F00000F00001E00001E00001E00001E00003C00003C00003C0 0003C0000780000780000780000780000F8000FFF800141F7D9E12>I<001FFF0000F800 00F00000F00000F00001E00001E00001E00001E00003C00003C00003C00003C000078000 0780000780000780000F00000F00000F00000F00001E00001E00301E00781E00F83C00F8 3C00F0780080700040E00021C0001F000018207D9E18>I<01FE00007FC0001E0000FC00 001E0000F80000170001780000170001780000270002F00000270004F00000270004F000 00270008F00000470009E00000470011E00000470021E00000470021E00000870043C000 00838043C00000838083C00000838083C000010381078000010382078000010382078000 0103840780000203840F00000203880F00000203900F00000203900F00000401E01E0000 0401E01E00000401C01E00000C01801E00001C01803E0000FF8103FFC0002A1F7D9E29> 77 D<0001FC0000070700001C01C0003000E000E0006001C00070038000700780003807 0000380E0000381E0000381C0000383C0000383C00003878000078780000787800007878 000078F00000F0F00000F0F00000E0F00001E0F00001C0F00003C0700003807000070078 000F0038001E0038003C001C0070000E00E0000783800001FC00001D217A9F23>79 D<01FFFF80001E00E0001E0070001E0038001E003C003C003C003C003C003C003C003C00 3C0078007800780078007800F0007800E000F003C000F00F0000FFFC0000F0000001E000 0001E0000001E0000001E0000003C0000003C0000003C0000003C0000007800000078000 0007800000078000000F800000FFF000001E1F7D9E1F>I<0007E040001C18C000300580 0060038000C0038001C00180018001000380010003800100038001000380000003C00000 03C0000003F8000001FF800001FFE000007FF000001FF0000001F8000000780000007800 000038000000380020003800200038002000300060007000600060006000E0007000C000 E8038000C606000081F800001A217D9F1A>83 D<0FFFFFF01E0780E01807802010078020 20078020200F0020600F0020400F0020400F0020801E0040001E0000001E0000001E0000 003C0000003C0000003C0000003C00000078000000780000007800000078000000F00000 00F0000000F0000000F0000001E0000001E0000001E0000001E0000003E00000FFFF0000 1C1F789E21>I87 D<00F1800389C0070780 0E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40 F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E 000E000E000E001C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE0 1EE01CE03CE038607060E031C01F0010207B9F15>I<007E0001C1000300800E07801E07 801C07003C0200780000780000780000F00000F00000F00000F00000F000007001007002 0030040018380007C00011147C9315>I<0000780003F800007000007000007000007000 00E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C03803C038038 0700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C4030 8C800F070015207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000 F000F0007000700170023804183807C010147C9315>I<00007800019C00033C00033C00 0718000700000700000E00000E00000E00000E00000E0001FFE0001C00001C00001C0000 1C0000380000380000380000380000380000700000700000700000700000700000700000 E00000E00000E00000E00000C00001C00001C0000180003180007B0000F300006600003C 00001629829F0E>I<003C6000E27001C1E00380E00700E00F00E00E01C01E01C01E01C0 1E01C03C03803C03803C03803C03803C07003C07001C0F001C17000C2E0003CE00000E00 000E00001C00001C00301C00783800F0700060E0003F8000141D7E9315>I<01E0000FE0 0001C00001C00001C00001C000038000038000038000038000070000070000071E000763 000E81800F01C00E01C00E01C01C03801C03801C03801C0380380700380700380700380E 10700E20700C20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C00000 0000000000000000000000000E003300230043804300470087000E000E000E001C001C00 1C003840388030807080310033001C000B1F7C9E0E>I<01E0000FE00001C00001C00001 C00001C0000380000380000380000380000700000700000703C00704200E08E00E11E00E 21E00E40C01C80001D00001E00001FC00038E00038700038700038384070708070708070 7080703100E03100601E0013207D9F15>107 D<03C01FC0038003800380038007000700 070007000E000E000E000E001C001C001C001C0038003800380038007000700070007100 E200E200E200E200640038000A207C9F0C>I<1C0F80F0002630C318004740640C004780 680E004700700E004700700E008E00E01C000E00E01C000E00E01C000E00E01C001C01C0 38001C01C038001C01C038001C01C0708038038071003803806100380380E10038038062 007007006600300300380021147C9325>I<1C0F802630C0474060478060470070470070 8E00E00E00E00E00E00E00E01C01C01C01C01C01C01C0384380388380308380708380310 7003303001C016147C931A>I<007C0001C3000301800E01C01E01C01C01E03C01E07801 E07801E07801E0F003C0F003C0F003C0F00780F00700700F00700E0030180018700007C0 0013147C9317>I<01C1E002621804741C04781C04701E04701E08E01E00E01E00E01E00 E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0072380071E0007000007 00000E00000E00000E00000E00001C00001C0000FFC000171D809317>I<00F0400388C0 0705800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E00 F01C00F01C00703C00705C0030B8000F3800003800003800007000007000007000007000 00E00000E0000FFE00121D7C9315>I<1C1E002661004783804787804707804703008E00 000E00000E00000E00001C00001C00001C00001C00003800003800003800003800007000 0030000011147C9313>I<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00800E00800E 00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C00012147C93 15>I<0E00C1C03300E3C02301C3E04381C1E04301C0E04701C060870380400E0380400E 0380400E0380401C0700801C0700801C0700801C0701001C0701001C0602001C0F02000C 0F04000E13080003E1F0001B147C931E>I<0383800CC4401068E01071E02071E02070C0 40E00000E00000E00000E00001C00001C00001C00001C040638080F38080F38100E58100 84C60078780013147D9315>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07001C0E00180E00180E001C1E000C3C0007DC 00001C00001C00003800F03800F07000E06000C0C0004380003E0000131D7C9316>I E /Fl 54 123 df<000078007C7800FC7801FC7803C00003800007800007800007800007 8000078000078000078000078000FFFC78FFFC78FFFC7807807807807807807807807807 807807807807807807807807807807807807807807807807807807807807807807807807 8078078078078078152480A31A>12 D<007000E001C003C0078007000F000E001E001C00 3C003C00380078007800780078007000F000F000F000F000F000F000F000F000F000F000 F000F000F000F0007000780078007800780038003C003C001C001E000E000F0007000780 03C001C000E000700C327DA413>40 DI46 D<000080000180000180 000300000300000300000600000600000600000C00000C00000C00001800001800001800 00300000300000300000600000600000600000C00000C00000C000018000018000018000 0180000300000300000300000600000600000600000C00000C00000C0000180000180000 180000300000300000300000600000600000600000C00000C00000C0000011317DA418> I<01F00007FC000FFE001F1F003C07803C07807803C07803C07803C07001C0F001E0F001 E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001 E0F001E07803C07803C07803C07803C03C07803C07801F1F000FFE0007FC0001F0001323 7EA118>I<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFFFFFFFF FF10227CA118>I<03F0000FFC001FFE003C1F003007807007C06003C0E003E0C001E040 01E04001E00001E00001E00001E00003C00003C0000780000780000F00001E00003C0000 780000F00001E00001C0000380000700000E00001C0000380000700000FFFFE0FFFFE0FF FFE013227EA118>I<01F00007FC001FFF003E0F003807807003C02003C02003C00003C0 0003C00003C0000780000780000F00001E0003FC0003F80003FE00000F000007800003C0 0003C00001E00001E00001E00001E00001E08001E0C003C0E003C07007803C0F801FFF00 0FFC0003F00013237EA118>I<001F00001F00002F00002F00006F0000EF0000CF0001CF 0001CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F00780F 00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F00000F 00000F0015217FA018>I<3FFF803FFF803FFF803C00003C00003C00003C00003C00003C 00003C00003C00003C00003CF8003FFE003FFF003F0F803E07803C03C03803C00001E000 01E00001E00001E00001E00001E00001E04003C04003C0E003C07007807C1F003FFE000F FC0003F00013227EA018>I<007E0001FF0003FF0007C1000F00001E00001E00003C0000 3C000078000078000078F800F1FE00F7FF00FF0F80FC0780F803C0F803C0F801C0F001E0 F001E0F001E0F001E0F001E07001E07001E07801E07803C03803C03C03801C07801F0F00 0FFE0007FC0001F00013237EA118>II<01F00007FC000FFE001E0F003C07803C07807803C078 03C07803C07803C07803C03803803C07801E0F000F1E0007FC0003F8000FFE001E0F003C 07807803C07803C0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C03C07803E 0F801FFF0007FC0001F00013237EA118>I<01F00007FC000FFE001E0F003C0700380780 7803807803C0F003C0F001C0F001E0F001E0F001E0F001E0F001E0F001E07003E07803E0 7803E03C07E03E1FE01FFDE00FF1E003E3C00003C00003C0000380000780000700000F00 101E00183C003FF8001FF00007C00013237EA118>I<001F0000001F0000003F8000003F 8000003B8000007BC0000073C0000071C00000F1E00000F1E00000E0E00001E0F00001E0 F00001C0F00003C0780003C078000380780007803C0007803C0007003C000F001E000F00 1E000FFFFE001FFFFF001FFFFF001C000F003C0007803C00078038000780780003C07800 03C0700003C0F00001E0F00001E0E00001E01B237EA220>65 DI<000FF000007F FC0000FFFF0001F01F0003C00700078000000F0000001E0000003E0000003C0000003C00 0000780000007800000078000000F0000000F0000000F0000000F0000000F0000000F000 0000F0000000F0000000F00000007800000078000000780000003C0000003C0000003E00 00001E0000000F0000000780008003C0038001F00F8000FFFF00007FFC00000FF0001925 7DA31F>IIII<000FF000003FFE0000FFFF8001F80F8003E00380 078000000F0000001E0000001E0000003C0000003C000000780000007800000078000000 F0000000F0000000F0000000F0000000F0000000F0000000F000FFC0F000FFC0F000FFC0 780003C0780003C0780003C03C0003C03C0003C01E0003C01E0003C00F0003C0078003C0 03E003C001F807C000FFFFC0003FFF00000FF8001A257DA321>II I76 D78 D<001FC000007FF00001FFFC0003F07E0007C01F000F800F801F 0007C01E0003C03C0001E03C0001E0780000F0780000F0780000F070000070F0000078F0 000078F0000078F0000078F0000078F0000078F0000078F0000078F0000078780000F078 0000F0780000F07C0001F03C0001E03E0003E01E0003C01F0007C00F800F8007C01F0003 F07E0001FFFC00007FF000001FC0001D257DA324>II82 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800007800007800007800007800 007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F00000F00000 F8000078000078000078000078000078000078C000F0E000F0F801E07E07C03FFF800FFF 0001FC0015257EA31B>II87 D<07E01FF83FFC381E201E000F000F000F000F00FF 07FF1FFF3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180780000780000700000F00000F00000F0 0000F00000F00000F000007800007800007800003C00401F03C00FFFC007FF8001FC0012 167E9516>I<0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C003E3C00FFBC01FFFC03F0FC03C07C07803C07803C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C07803C07803C03C07C03E0FC01FFFC00FFBC0 03E3C012237EA219>I<03F00007FC001FFE003E0F003C0780780380780380F001C0FFFF C0FFFFC0FFFFC0F00000F00000F000007000007800007800003C00801F07800FFF8007FF 0001F80012167E9516>I<003F00FF01FF03C00380078007800780078007800780078007 80FFF8FFF8FFF80780078007800780078007800780078007800780078007800780078007 800780078007800780102380A20F>I<01F07807FFF80FFFF81F1F001E0F003C07803C07 803C07803C07803C07801E0F001F1F000FFE001FFC0019F0003800003800003C00001FFE 001FFFC01FFFE03FFFF07801F07800F8F00078F00078F00078F000787800F03E03E01FFF C00FFF8001FC0015217F9518>III107 DIII<01FC0007FF000FFF801F07C03C01E07800F07800F0700070F00078 F00078F00078F00078F00078F000787800F07800F07C01F03E03E01F07C00FFF8007FF00 01FC0015167F9518>II114 D<07F01FFC3FFE3C0E780678007800 7C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE3FFC0FF010167F9513> I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F000F000F000F000F000F 000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12>III121 D<7FFFC07FFFC07FFFC0000F80000F00001E00003E00007C0000780000F00001 F00003E00003C0000780000F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012 167F9515>I E /Fm 42 122 df<001FE03FC0007FE0FFC001FFE1FFC003FFE3FFC003E0 07E00007C007C00007800F80000F800F80000F800F80000F800F80000F800F80000F800F 80000F800F80000F800F80000F800F8000FFFE0FFE00FFFE0FFE00FFFE0FFE000F800F80 000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F8000 0F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F 800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F80 0F80000F800F8000222A7FA922>11 D<00000F80007E0F8000FE0F8001FE0F8003FE0F80 07C00000078000000F8000000F8000000F8000000F8000000F8000000F8000000F800000 0F8000000F800000FFFE0F80FFFE0F80FFFE0F800F800F800F800F800F800F800F800F80 0F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F80 0F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F80 0F800F800F800F80192B7FAA1F>I46 D<007E0001FF8007FFE00FFFF01FC3F81F00F83E007C3E007C3C003C7C003E7C003E7800 1E78001EF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF800 1FF8001FF8001FF8001FF8001F78001E7C003E7C003E7C003E7C003E3E007C3F00FC1F00 F81FC3F80FFFF007FFE001FF80007E0018297EA71D>48 D<00300000700001F0000FF000 FFF000FFF000F1F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F000FFFFE0FFFFE0FFFFE0 13287BA71D>I<00FE0003FF800FFFC01FFFE01F07F03C01F87800FC78007C70007CF000 3EF0003E60003E20003E00003E00003E00003E00007C00007C0000780000F80001F00001 E00003E00007C0000F80001F00003E00007C0000F80001F00003E00003C0000780000F00 001E00003C00007FFFFE7FFFFE7FFFFE7FFFFE17287EA71D>I<007E0003FF8007FFE01F FFF03F83F83E00F87C007C38007C30007C10007C00007C00007C0000F80000F80001F800 03F0000FE001FFC001FF8001FFC001FFE00001F80000FC00007C00003E00003E00001F00 001F00001F00001F00001F80001FC0003EE0003EF0007E7C00FC3F01F81FFFF00FFFE003 FFC000FE0018297EA71D>I<0007E0000FE0000BE0001BE0001BE0003BE0003BE00073E0 0073E000F3E000F3E001E3E001E3E003E3E003C3E007C3E00F83E00F83E01F03E01F03E0 3E03E03E03E07C03E07C03E0F803E0FFFFFFFFFFFFFFFFFFFFFFFF0003E00003E00003E0 0003E00003E00003E00003E00003E00003E00003E018277EA61D>I<3FFFF83FFFF83FFF F83FFFF83E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E3E 003EFF803FFFE03FFFF03FC1F03F00F83F007C3E007C00007C00003E00003E00003E0000 3E00003E00003E00003E40003E60007CE0007CF000F87801F87F07F03FFFE00FFFC007FF 0001FC0017287EA61D>I<000FE0003FF800FFF801FFF803F01807E0000F80000F00001F 00003E00003E00003C00007C00007C02007C3FC0F8FFE0F9FFF0FBFFF8FF80FCFE007CFE 003EFC003EFC003FFC001FF8001FF8001FF8001FF8001F78001F7C001F7C001F7C001E3C 003E3E003E1E007C1F00FC0FC1F807FFF003FFE001FF80007E0018297EA71D>II58 D<0003F000000003F000000007F800000007F8000000077800 00000F3C0000000F3C0000000F3C0000001E3E0000001E1E0000001E1E0000003E1F0000 003C1F0000007C0F8000007C0F800000780F800000F807C00000F807C00000F007C00001 F003E00001F003E00001E003E00003E001F00003E001F00003C001F00007C000F80007FF FFF80007FFFFF8000FFFFFFC000F80007C000F00007C001F00003E001F00003E001E0000 3E003E00001F003E00001F003C00001F007C00000F807C00000F80F800000FC0F8000007 C0F8000007C0222A7EA927>65 D<0003FF00000FFFE0003FFFF800FFFFF801FE01F803F8 007007E0001007C000000F8000001F8000001F0000003E0000003E0000007C0000007C00 00007C0000007C000000F8000000F8000000F8000000F8000000F8000000F8000000F800 0000F8000000F8000000F80000007C0000007C0000007C0000007C0000003E0000003E00 00001F0000001F8000000F80000007C0000007E0000803F8003801FE01F800FFFFF8003F FFF0000FFFC00003FE001D2C7CAA25>67 DIII73 D77 D<0003FC0000000FFF000000 3FFFC00000FFFFF00001FE07F80003F801FC0007E0007E000FC0003F000F80001F001F00 000F801F00000F803E000007C03E000007C07C000003E07C000003E07C000003E0780000 01E0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001 F0F8000001F0F8000001F0F8000001F07C000003E07C000003E07C000003E07E000007E0 3E000007C03E000007C01F00000F801F80001F800F80001F000FC0003F0007E0007E0003 F801FC0001FE07F80000FFFFF000003FFFC000001FFF80000003FC0000242C7DAA2B>79 DI<007FC000 01FFF80007FFFE000FFFFF001FC07F003F000F007E0006007C0000007C000000F8000000 F8000000F8000000F8000000F8000000FC0000007E0000007F0000003F8000001FF80000 0FFF000007FFE00003FFF80000FFFC00000FFE000000FF0000003F0000001F8000000F80 00000FC0000007C0000007C0000007C0000007C0000007C0000007C000000F8060000F80 F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE00000FF80001A2C7DAA21>83 D87 D<01FC000FFF803FFFC03FFFE03C03F03001F00001F8 0000F80000F80000F80000F80001F800FFF807FFF81FFFF83FC0F87E00F8FC00F8F800F8 F800F8F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1C>97 D<007F8001FFE007FFF80FFFF81FC0783F00103E00007C00007C00007C0000F80000F800 00F80000F80000F80000F80000F800007C00007C00007C00003E00003F00181FC0780FFF F807FFF801FFE0007F80151B7E9A1A>99 D<00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E00003E00003E00003E00003E01FC3E03FF3E0FFFBE 1FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003E F8003EF8003EFC003E7C003E7C003E7E007E3F00FE1FC1FE1FFFFE0FFFBE03FE3E01F83E 172A7EA91E>I<00FC0003FF0007FF800FFFC01F87E03F01E03E00F07C00F07C0070F800 78FFFFF8FFFFF8FFFFF8FFFFF8F80000F80000F800007800007C00007C00003E00003F00 181FC0F80FFFF807FFF801FFE0007F00151B7E9A1A>I104 DI108 DII<007E0001FF8007FFE00FFFF01F81F83F 00FC3E007C7C003E7C003E78001EF8001FF8001FF8001FF8001FF8001FF8001FF8001F7C 003E7C003E7C003E3E007C3F00FC1FC3F80FFFF007FFE003FFC0007E00181B7E9A1D>I< F83F00F9FFC0FBFFE0FFFFF0FF07F8FC01F8F800FCF800FCF8007CF8007EF8003EF8003E F8003EF8003EF8003EF8003EF8003EF8007EF8007CF800FCFC00F8FC01F8FF07F0FFFFE0 FBFFC0F9FF80F87E00F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F8000017277C9A1E>I114 D<07F8001FFF003FFF807FFF807C0780F80100F80000F80000F80000FC00 007F00007FF0003FFC001FFE0007FF0000FF80001FC0000FC00007C00007C04007C0E007 C0FC1F80FFFF807FFF001FFE0007F800121B7E9A16>I<0F80000F80000F80000F80000F 80000F80000F8000FFFF80FFFF80FFFF800F80000F80000F80000F80000F80000F80000F 80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F 80800FC38007FFC007FFC003FF0001F80012227FA115>IIII<7C001F807E003F003F003E001F007C000F80 FC0007C0F80007E1F00003E3E00001F7E00000FFC000007F8000007F0000003E0000003E 0000003F0000007F800000FFC00001F3E00003E1E00003E1F00007C0F8000F807C001F80 7C001F003E003E001F007E001F80FC000FC01A1B809A1B>II E /Fn 31 90 df<3078F8787005057C840D>46 D<00000400000C00000C0000 180000180000300000300000600000600000C00000C00001800001800003000003000006 00000600000C00000C0000180000180000300000300000600000600000C00000C0000180 000180000300000300000600000600000600000C00000C00001800001800003000003000 00600000600000C00000C00000800000162D7EA117>I<007E0001C3000301800701C00E 00C00E00E01C00E01C00E03C01E03801E07801E07801E07801E07801E07801E0F003C0F0 03C0F003C0F003C0F003C0F00380F00780E00780E00700E00700E00E00600E00701C0030 38003870000FC000131F7C9D17>I<000C001C00FC0F3800380038003800380038007000 70007000700070007000E000E000E000E000E000E001C001C001C001C001C001C0038003 C0FFFE0F1E7C9D17>I<003F8000C1E00100F00200780400780400780F007C0F807C0F80 7C0F00780600780000F80000F00001E00001C0000380000700000E00001C000038000060 0000C0000180000300200600200800401000403FFFC07FFF80FFFF80161E7E9D17>I<00 7F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E00003C000038000 0700000E0000F800000E000007000007800007C00003C00007C03007C07807C0F807C0F8 07C0F00780800F00400E00201C0018780007E000141F7D9D17>I<0000600000600000E0 0001C00003C00005C0000DC00009C00011C000238000438000C380008380010380020380 040700080700180700100700200700400700FFFFF0000E00000E00000E00000E00000E00 001C00001E0001FFE0141E7E9D17>I<01803001FFE003FFC003FF0003FC000200000200 00020000040000040000040000047C000587000603800C01800801C00001C00001E00001 E00001E00001E07003C0F803C0F003C0E00380800780400700400E00201C0018700007C0 00141F7D9D17>I<000F8000704000C0200180E00301E00701E00E00C01E00001C00003C 000038000078F800790E007A07007C0300F80380F80380F003C0F003C0F003C0F003C0F0 0780E00780E00780E00700E00F00600E00701C0030180018700007C000131F7C9D17>I< 2000003FFFE07FFFC07FFF80400100C00200800200800400000800001000002000004000 0040000080000180000300000300000700000600000E00000E00001E00001C00001C0000 3C00003C00003C0000780000780000780000300000131F799D17>I<0E1F1F1E0C000000 000000000000003078F8787008147C930D>58 D<00001000000018000000380000003800 00007800000078000000FC000001BC0000013C0000033C0000023C0000063C0000043E00 00081E0000081E0000101E0000101E0000201E0000200F0000400F0000400F0000FFFF00 00800F0001000F8001000780020007800200078004000780040007800C0007C03E0007C0 FF807FFC1E207E9F22>65 D<0001F808000E061800380138007000F801E0007803C00070 07800030078000300F0000301F0000301E0000303E0000203C0000007C0000007C000000 7C0000007C000000F8000000F8000000F8000000F8000000F80000007800004078000080 780000803C0000803C0001001C0002000E00020006000C000300100001C0E000003F0000 1D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E 0078001E0078001E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F 01E0001E01E0003E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F0 03C001E003C003C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF800 7C0078003C0038003C001800780018007800080078000800780008007800080078080800 F0100000F0100000F0100000F0300000FFF00000F0700001E0200001E0200001E0200001 E0200001E0000801E0001003C0001003C0001003C0002003C0002003C0006003C000C007 8001C0078007C0FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C00180078 0018007800080078000800780008007800080078000800F0100000F0100000F0100000F0 300000F0700000FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0 000003C0000003C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F 7E9E1E>I<07FFC7FFC0007C00F800003C007800003C007800007800F000007800F00000 7800F000007800F000007800F000007800F00000F001E00000F001E00000F001E00000F0 01E00000FFFFE00000F001E00001E003C00001E003C00001E003C00001E003C00001E003 C00001E003C00003C007800003C007800003C007800003C007800003C007800003C00780 0007800F000007C00F8000FFF8FFF800221F7E9E22>72 D<07FFE0007C00003C00003C00 00780000780000780000780000780000780000F00000F00000F00000F00000F00000F000 01E00001E00001E00001E00001E00001E00003C00003C00003C00003C00003C00003C000 07800007C000FFFC00131F7F9E10>I<07FFF000007E0000003C0000003C000000780000 007800000078000000780000007800000078000000F0000000F0000000F0000000F00000 00F0000000F0000001E0000001E0000001E0000001E0000001E0008001E0010003C00100 03C0010003C0030003C0020003C0060003C0060007801E0007807C00FFFFFC00191F7E9E 1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800004E0002F000004E 0002F000004E0004F000004E0004F000004E0008F000004E0008F00000870011E0000087 0011E00000870021E00000870021E00000870041E00000838041E00001038083C0000103 8083C00001038103C00001038203C0000101C203C0000101C403C0000201C40780000201 C80780000201C80780000201D00780000200F00780000600E00780000600E00F00000F00 C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E001800003E0018 00004F001000004F001000004780100000478010000043C010000043C010000083C02000 0081E020000081E020000080F020000080F020000080782000010078400001007C400001 003C400001003C400001001E400001001E400002000F800002000F800002000F80000200 0780000200078000060003800006000300000F00010000FFE0010000221F7E9E22>I<00 03F800001E0E000038070000E0038001C001C003C001E0078000E00F0000F00F0000F01E 0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8F80001F0F8 0001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C0007803C000F003C 001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07FFFF00007C 03C0003C01E0003C00F0007800F0007800F8007800F8007800F8007800F8007800F000F0 01F000F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001E0000001E0 000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003C000000780 000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C01E0007801 E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F0078000F00F 0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E01E0003C03E 0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC0F18000003 E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E0008 0E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E0 0000F00000F04000E04000E04000E04000E06000C0600180E00380F80300C60C0081F800 16217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010C00F001080 0F0010800F0010800F0010001E0000001E0000001E0000001E0000001E0000001E000000 3C0000003C0000003C0000003C0000003C0000003C000000780000007800000078000000 780000007800000078000000F0000001F800007FFFE0001C1F7A9E21>IIII<03FFC0FFC0 007F007E00003E003800001E003000001E002000000F004000000F008000000F81000000 078200000007C600000003C400000003E800000001F000000001F000000000F000000000 F800000000F8000000017C000000023C000000043C0000000C1E000000081E000000101F 000000200F000000400F800000C0078000008007C000010003C000070003E0001F8007E0 00FFE01FFE00221F7F9E22>II E /Fo 39 122 df<000FF07F00007FF9FF8000F83FC7C001E07F8FC003E07F0FC007C07F 0FC007C03F078007C01F000007C01F000007C01F000007C01F000007C01F0000FFFFFFF8 00FFFFFFF80007C01F000007C01F000007C01F000007C01F000007C01F000007C01F0000 07C01F000007C01F000007C01F000007C01F000007C01F000007C01F000007C01F000007 C01F000007C01F000007C01F00003FF8FFF0003FF8FFF0002220809F1F>11 D<000FE000007FF80000F81C0001E07C0003E07C0007C07C0007C07C0007C0380007C000 0007C0000007C0000007C1FE00FFFFFE00FFFFFE0007C03E0007C03E0007C03E0007C03E 0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E 0007C03E0007C03E0007C03E003FF9FFC03FF9FFC01A20809F1D>I<0018007000E001C0 0380038007000E000E001E001C003C003C007800780078007800F800F000F000F000F000 F000F000F000F000F000F80078007800780078003C003C001C001E000E000E0007000380 038001C000E0007000180D2D7DA114>40 DI45 D<387CFEFEFE7C3807077C860F>I< 387CFEFEFE7C38000000000000387CFEFEFE7C3807147C930F>58 D<0000E000000000E000000001F000000001F000000001F000000003F800000003F80000 0006FC00000006FC0000000EFE0000000C7E0000000C7E000000183F000000183F000000 303F800000301F800000701FC00000600FC00000600FC00000C007E00000FFFFE00001FF FFF000018003F000018003F000030001F800030001F800060001FC00060000FC000E0000 FE00FFE00FFFE0FFE00FFFE0231F7E9E28>65 D<0007FC02003FFF0E00FE03DE03F000FE 07E0003E0FC0001E1F80001E3F00000E3F00000E7F0000067E0000067E000006FE000000 FE000000FE000000FE000000FE000000FE000000FE0000007E0000007E0000067F000006 3F0000063F00000C1F80000C0FC0001807E0003803F0007000FE01C0003FFF800007FC00 1F1F7D9E26>67 D69 DI73 D76 DI<001FF80000FFFF0001F81F8007 E007E00FC003F01F8001F81F0000F83F0000FC7F0000FE7E00007E7E00007EFE00007FFE 00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007F7E00007E7F 0000FE7F0000FE3F0000FC3F8001FC1F8001F80FC003F007E007E001F81F8000FFFF0000 1FF800201F7D9E27>79 DI<03FC 080FFF381E03F83800F8700078700038F00038F00018F00018F80000FC00007FC0007FFE 003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC0003CC0003CC000 3CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>83 D91 D93 D<07FC001FFF003F0F803F07 C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803 E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C03 00FC0000FC0000FC0000FC0000FC0000FC00007C00007E00003E00603F00C01F81C007FF 0001FC0013147E9317>I<0007F80007F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F801F8F80FFEF81F83F83E01F87E00F87C00F87C00F8FC00F8FC 00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F87E00F83E01F81F07F80FFEFF03F8FF18 207E9F1D>I<01FE0007FF800F83C01E01E03E00F07C00F07C00F8FC00F8FFFFF8FFFFF8 FC0000FC0000FC00007C00007C00003E00181E00180F807007FFE000FF8015147F9318> I<001F8000FFC001F3E003E7E003C7E007C7E007C3C007C00007C00007C00007C00007C0 00FFFC00FFFC0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC3C07 FFFE0F079E1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC0018 00001800001C00001FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78 003C78003C3F01F80FFFE001FF00171E7F931A>II<1C003E007F007F007F003E001C00000000000000 000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F00 1F001F001F00FFE0FFE00B217EA00E>I108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007E FC007EFC007EFC007EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF00 17147F931A>II114 D<0FE63FFE701E600EE006 E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81EFFFCC7F010147E9315> I<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F800F800F800F 800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>III121 D E /Fp 82 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F870051C779B18> 33 D<4010E038F078E038E038E038E038E038E038E038E038E038E03860300D0E7B9C18> I<00C00001C00001C00001C00003F0000FFC003FFE007DCF0071C700E1C380E1C780E1C7 80E1C780F1C00079C0003DC0001FE0000FF80003FC0001DE0001CF0001C70061C380F1C3 80F1C380E1C380E1C70071C70079DE003FFE001FF80007E00001C00001C00001C00000C0 0011247D9F18>36 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F0 1C73F01CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800 E0F800E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18>I<007000F001E003C007800F001E00 1C00380038007000700070007000E000E000E000E000E000E000E000E000700070007000 7000380038001C001E000F00078003C001F000F000700C24799F18>I<6000F00078003C 001E000F000780038001C001C000E000E000E000E0007000700070007000700070007000 7000E000E000E000E001C001C0038007800F001E003C007800F00060000C247C9F18>I< 01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC00 7FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D9718>I<00600000F0 0000F00000F00000F00000F00000F00000F0007FFFC0FFFFE0FFFFE07FFFC000F00000F0 0000F00000F00000F00000F00000F00000600013147E9718>I<1C3E7E7F3F1F070E1E7C F860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC7830060677 8518>I<000300000780000780000F80000F00001F00001E00001E00003E00003C00007C 0000780000780000F80000F00001F00001E00003E00003C00003C00007C0000780000F80 000F00000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F000 0060000011247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001C070 01C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C070 01C07803C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>I<018003800380 07800F803F80FF80FB804380038003800380038003800380038003800380038003800380 03800380038003807FFCFFFE7FFC0F1C7B9B18>I<03F0000FFE003FFF007C0F807003C0 E001C0F000E0F000E06000E00000E00000E00001C00001C00003C0000780000F00001E00 003C0000780000F00001E00007C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E 9B18>I<07F8001FFE003FFF007807807803C07801C03001C00001C00003C0000380000F 0003FF0003FE0003FF000007800003C00001C00000E00000E00000E0F000E0F000E0F001 C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<1FFF803FFF803FFF8038000038 00003800003800003800003800003800003800003BF8003FFE003FFF003C07801803C000 01C00000E00000E06000E0F000E0F000E0E001C07003C07C0F803FFF001FFC0003F00013 1C7E9B18>53 D<007E0001FF0007FF800F83C01E03C01C03C03801803800007000007000 00E1F800E7FE00FFFF00FE0780F803C0F001C0F000E0E000E0F000E07000E07000E07000 E03801C03C03C01E07800FFF0007FE0001F800131C7E9B18>I<03F8000FFE001FFF003E 0F803803807001C07001C07001C07001C03803803C07801FFF0007FC000FFE001F1F003C 07807001C0F001E0E000E0E000E0E000E0E000E07001C07803C03E0F801FFF000FFE0003 F800131C7E9B18>56 D<3078FCFC783000000000000000003078FCFC78300614779318> 58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<00 0300000780001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC 00007E00003F00001FC00007E00003F00001FC00007E00003F00001F8000078000030011 187D9918>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0 FFFFE07FFFC0130C7E9318>I<600000F00000FC00007E00003F00001FC00007E00003F0 0001FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F00 007E0000FC0000F0000060000011187D9918>I<007C0001FE0007FF000F87801E03C03C 1DC0387FC070FFE071E3E071C1E0E1C1E0E380E0E380E0E380E0E380E0E380E0E380E0E1 C1C071C1C071E3C070FF80387F003C1C001E00E00F83E007FFC001FF80007E00131C7E9B 18>64 D<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E 00038E00038E000306000707000707000707000707000FFF800FFF800FFF800E03800E03 801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>II<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E0700000 E00000E00000E00000E00000E00000E00000E00000E000007000007000E07000E03800E0 3C00E01E01C00F07C007FF8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F 801C03C01C03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C00701C00 701C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF8 00141C7F9B18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001 C0700000E00000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001C07003 C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F 07F01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01F FFC01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF 8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7F07F0FF87 F87F07F01C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0001DF0001DF0 001FB8001FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03801C01C07F03 F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01D C1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C 39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I< 0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780700700 780F007FFF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E1E380E1E380F0E78070 F700787F007FFF003FFE000FFC00001C00001E00000E00000F0000070000070011227D9B 18>I<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C0380 1C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C039C 1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>I<03F3801FFF803FFF807C0F807007 80E00380E00380E00380E000007000007800003F00001FF00007FE0000FF00000F800003 C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C 7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E0703800700000700000700000 700000700000700000700000700000700000700000700000700000700000700000700000 700000700000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC00 01F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F00 0707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F0000 1E00001C00003C0000780000700000F00001E00001C00003C0000780000700000F00001E 00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>I I93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I<0038007C007C007C003800000000000000000FFC1FFC0FFC001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C6038F078FFF07FE03F800E277E9C18>II<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF800 1F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F0019148193 18>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0 000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001 E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807F FFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F 00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E 00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780 700780E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF80 0FFB8003E380000380000380000380000380000380000380000380003FF8003FF8003FF8 151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C0000380000380 000380000380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318 >I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE0000 1F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<018000 0380000380000380000380007FFFC0FFFFC0FFFFC0038000038000038000038000038000 0380000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E00 13197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318 >I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E0003 8E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8F F0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E 000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E 03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000 DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F80003F 00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00 003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF0 14147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000007800007F E0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003FE0003FE000 7FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00003C000FFC000FF00007C000013247E9F18>125 D E /Fq 59 123 df<007F07F001FF1FF003FF3FF007807800070070000F00F0000F00F0000F00F000 0F00F0000F00F0000F00F0000F00F000FFF8FF80FFF8FF80FFF8FF800F00F0000F00F000 0F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000 0F00F0000F00F0000F00F0000F00F0000F00F0000F00F0001C20809F1B>11 D<0000F000F8F001F8F003F8F00780000700000F00000F00000F00000F00000F00000F00 000F0000FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00 F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F0142180A018>I<00 7000E001C00380078007000E001E001E003C003C003C0078007800780078007000F000F0 00F000F000F000F000F000F000F000F000F000F000700078007800780078003C003C003C 001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<787878781830306060E0050A7D830D>44 DI<000100030003000600060006000C000C000C00180018 001800300030003000600060006000C000C000C001800180018003000300030006000600 06000C000C000C00180018001800300030003000600060006000C000C000C000102D7DA1 17>47 D<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17> 49 D<07F0000FFC001FFE00383F00700F00600780E00780E003C04003C04003C00003C0 0003C00003C0000780000780000F00000E00001C00003C0000780000E00001C000038000 0700000E00001C0000380000700000FFFFC0FFFFC0FFFFC0121F7E9E17>I58 D<001F0000001F0000003F800000 3B8000003B8000007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F00001 E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C000FFFFE000F FFFE000FFFFE001E000F001E000F003C000F803C0007803C000780780007C0780003C078 0003C0F00003E01B207F9F1E>65 DI<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00 007C0000780000780000780000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800007C00003C00003E00001E00000F000207800E03E03E01FFFC00FF F0001FC017227DA01D>II II<001FE000 FFF801FFFE03E03E07800E0F00001E00003E00003C00007C0000780000780000780000F0 0000F00000F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C 001E3E001E1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F000000FFC00003FFF00007E1F8000F807C001F003E001E001E00 3C000F003C000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F80007C0780007807800078078000780 3C000F003C000F001E001E001F003E000F807C0007E1F80003FFF00000FFC000003F0000 1A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C00 003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001 E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019 >IIIII<780007807C000F003E001F001E001E000F003C000F807C000780780003C0F0 0003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000003F00 00007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C000F003E 001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>III<07E03FF87FFC701E401F000F000F000F003F 07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000 F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F8 0014147F9317>II<03C7800FF780 1FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F80780 7807807C0F803E1F801FFF800FF78007C780000780000780000780000780000780000780 000780000780000780111D7E9317>II<07F01FFC3FFC780C78 00780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311 >I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E00 1E001E001E001E001E001E201FF00FF007C00C1A7F9910>II< F003C0F003C07803807807807807803C0F003C0F003C0F001E0E001E1E001E1E000F1C00 0F3C000F3C0007380007380007B80003F00003F00001E00012147F9315>II<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F8 0000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414 809315>II<7FFF7FFF7FFF003E00 3C007800F800F001E003E007C007800F001F001E003C007C00FFFFFFFFFFFF10147F9314 >I E /Fr 14 122 df<0000000000700000000000F80000000001F80000000001F80000 000003F80000000003F00000000007F00000000007E00000000007E0000000000FE00000 00000FC0000000001FC0000000001F80000000001F80000000003F80000000003F000000 00007F00000000007E00000000007E0000000000FE0000000000FC0000000001FC000000 0001F80000000001F80000000003F80000000003F00000000007F00000000007E0000000 0007E0000000000FE0000000000FC0000000001FC0000000001F80000000001F80000000 003F80000000003F00000000007F00000000007E00000000007E0000000000FE00000000 00FC0000000001FC0000000001F80000000001F80000000003F80000000003F000000000 07F00000000007E00000000007E0000000000FE0000000000FC0000000001FC000000000 1F80000000003F80000000003F00000000003F00000000007F00000000007E0000000000 FE0000000000FC0000000000FC0000000001FC0000000001F80000000003F80000000003 F00000000003F00000000007F00000000007E0000000000FE0000000000FC0000000000F C0000000001FC0000000001F80000000003F80000000003F00000000003F00000000007F 00000000007E0000000000FE0000000000FC0000000000FC0000000001FC0000000001F8 0000000003F80000000003F00000000003F00000000007F00000000007E0000000000FE0 000000000FC0000000000FC0000000001FC0000000001F80000000003F80000000003F00 000000003F00000000007F00000000007E0000000000FE0000000000FC0000000000FC00 00000000F800000000007800000000002D677ACC3A>47 D66 D73 D<00000003FFF000000000 0000007FFFFF80000000000003FFFFFFF000000000000FFF003FFC00000000003FF80007 FF0000000000FFE00001FFC000000003FF8000007FF000000007FF0000003FF80000000F FE0000001FFC0000001FFC0000000FFE0000003FF800000007FF0000007FF000000003FF 800000FFF000000003FFC00001FFE000000001FFE00003FFE000000001FFF00003FFC000 000000FFF00007FFC000000000FFF80007FF80000000007FF8000FFF80000000007FFC00 0FFF80000000007FFC001FFF00000000003FFE001FFF00000000003FFE003FFF00000000 003FFF003FFF00000000003FFF003FFF00000000003FFF007FFF00000000003FFF807FFE 00000000001FFF807FFE00000000001FFF807FFE00000000001FFF807FFE00000000001F FF80FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE0000 0000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0 FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000 001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC07FFE 00000000001FFF807FFF00000000003FFF807FFF00000000003FFF807FFF00000000003F FF803FFF00000000003FFF003FFF00000000003FFF003FFF00000000003FFF001FFF8000 0000007FFE001FFF80000000007FFE001FFF80000000007FFE000FFFC000000000FFFC00 0FFFC000000000FFFC0007FFC000000000FFF80003FFE000000001FFF00003FFE0000000 01FFF00001FFF000000003FFE00000FFF800000007FFC000007FF800000007FF8000007F FC0000000FFF8000003FFE0000001FFF0000000FFF0000003FFC00000007FFC00000FFF8 00000003FFE00001FFF000000000FFF80007FFC0000000007FFF807FFF80000000000FFF FFFFFC000000000003FFFFFFF00000000000007FFFFF8000000000000003FFF000000000 4A497AC757>79 D<000FFFF000000000FFFFFF00000003FFFFFFC0000007F8007FF00000 0FFC001FFC00000FFC0007FE00001FFE0003FF00001FFE0003FF80001FFE0001FF80001F FE0001FFC0001FFE0000FFC0000FFC0000FFE00007F80000FFE00003F00000FFE0000000 0000FFE00000000000FFE00000000000FFE00000000000FFE00000000000FFE000000000 7FFFE00000003FFFFFE0000003FFFFFFE000001FFFF0FFE000007FFE00FFE00001FFE000 FFE00003FF8000FFE0000FFE0000FFE0001FFC0000FFE0001FF80000FFE0003FF80000FF E0007FF00000FFE0007FF00000FFE000FFE00000FFE000FFE00000FFE000FFE00000FFE0 00FFE00000FFE000FFE00001FFE000FFE00001FFE0007FF00003FFE0007FF000077FE000 3FF8000F7FE0001FFC003E7FF8000FFF80F83FFFE003FFFFF01FFFE000FFFFC00FFFE000 0FFE0003FFE0332E7CAD38>97 D<007FC000000000FFFFC000000000FFFFC000000000FF FFC000000000FFFFC00000000003FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0 0000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000 000001FFC00000000001FFC00000000001FFC01FFC000001FFC0FFFF800001FFC3FFFFF0 0001FFCFE01FFC0001FFDF0003FE0001FFFC0001FF0001FFF80000FF8001FFF000007FC0 01FFE000003FE001FFC000003FF001FFC000001FF801FFC000001FF801FFC000001FFC01 FFC000001FFC01FFC000000FFE01FFC000000FFE01FFC000000FFE01FFC000000FFE01FF C000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC0 00000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000 000FFE01FFC000000FFE01FFC000000FFE01FFC000000FFC01FFC000001FFC01FFC00000 1FFC01FFC000001FF801FFC000003FF001FFC000003FF001FFE000007FE001FFF000007F C001FFF80000FF8001FF3C0001FF0001FE1F0007FE0001FC0FC03FF80001F803FFFFE000 01F000FFFF800001E0001FF8000038487CC741>I<00000000001F0000007FF000FFC000 07FFFF03FFE0001FFFFFC7E7E0007FE03FFF0FF000FF800FFC0FF001FF0007FC0FF003FE 0003FE0FF007FE0003FF07E007FC0001FF03C00FFC0001FF80000FFC0001FF80001FFC00 01FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001 FFC0001FFC0001FFC0001FFC0001FFC0000FFC0001FF80000FFC0001FF800007FC0001FF 000007FE0003FF000003FE0003FE000001FF0007FC000000FF800FF8000000FFE03FF000 0001DFFFFFC0000003C7FFFF00000003C07FF00000000780000000000007800000000000 0780000000000007C0000000000007C0000000000007E0000000000007E0000000000007 F8000000000007FFFFFFF0000003FFFFFFFF000003FFFFFFFFE00001FFFFFFFFF00001FF FFFFFFFC0000FFFFFFFFFE00007FFFFFFFFF0001FFFFFFFFFF8007FFFFFFFFFF800FF000 001FFFC01FC0000001FFC03F800000007FC07F800000003FE0FF000000003FE0FF000000 001FE0FF000000001FE0FF000000001FE0FF000000001FE0FF000000001FE07F80000000 3FC07F800000003FC03FC00000007F801FE0000000FF000FF8000003FE0003FE00000FF8 0001FFE000FFF000007FFFFFFFC000000FFFFFFE000000007FFFC0000034447DAE3A> 103 D<007FC000000000FFFFC000000000FFFFC000000000FFFFC000000000FFFFC00000 000003FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0000000 0001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000000000 01FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001 FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC007FE000001FFC03FFFC00001FFC07FFFF00001FFC1F81FF80001FFC3 C00FFC0001FFC70007FE0001FFCE0007FE0001FFDC0007FF0001FFF80003FF0001FFF800 03FF8001FFF00003FF8001FFF00003FF8001FFE00003FF8001FFE00003FF8001FFE00003 FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF 8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF80 01FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001 FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FF C00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF80FFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81 FFFFFF38487BC741>I<007C0001FF0003FF8007FFC007FFC00FFFE00FFFE00FFFE00FFF E00FFFE007FFC007FFC003FF8001FF00007C000000000000000000000000000000000000 00000000000000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC003FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C0FFFFFFFFFFFFFFFFFFFFFFFF18497CC820>I<007FC000FFFFC000FFFFC000FFFFC000 FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<00000FFE0000000001FFFFF000000007FFFFFC0000001FF803FF0000007FE000FFC000 00FF80003FE00001FF00001FF00003FE00000FF80007FC000007FC000FFC000007FE000F F8000003FE001FF8000003FF003FF8000003FF803FF0000001FF803FF0000001FF807FF0 000001FFC07FF0000001FFC07FF0000001FFC0FFF0000001FFE0FFF0000001FFE0FFF000 0001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF00000 01FFE0FFF0000001FFE0FFF0000001FFE07FF0000001FFC07FF0000001FFC07FF0000001 FFC07FF0000001FFC03FF8000003FF803FF8000003FF801FF8000003FF001FFC000007FF 000FFC000007FE0007FE00000FFC0003FE00000FF80001FF00001FF00000FF80003FE000 007FE000FFC000001FFC07FF00000007FFFFFC00000001FFFFF0000000001FFF00000033 2E7DAD3A>111 D<007FC01FFC0000FFFFC0FFFF8000FFFFC3FFFFF000FFFFCFE03FFC00 FFFFDF0007FE0003FFFC0003FF0001FFF80001FF8001FFF00000FFC001FFE000007FE001 FFC000007FF001FFC000003FF801FFC000003FF801FFC000001FFC01FFC000001FFC01FF C000001FFE01FFC000001FFE01FFC000000FFE01FFC000000FFE01FFC000000FFF01FFC0 00000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000 000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFE01FFC00000 1FFE01FFC000001FFE01FFC000001FFC01FFC000001FFC01FFC000003FFC01FFC000003F F801FFC000003FF001FFC000007FF001FFE00000FFE001FFF00000FFC001FFF80001FF80 01FFFC0003FF0001FFDF000FFE0001FFCFC03FF80001FFC3FFFFE00001FFC0FFFF800001 FFC01FF8000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0 0000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC000000000FFFFFF80000000FFFFFF80000000FFFFFF80000000FFFFFF8000 000038427CAD41>I<007F803F80FFFF80FFE0FFFF81FFF8FFFF83C3FCFFFF8707FE03FF 8E07FE01FF9C0FFF01FFB80FFF01FFB80FFF01FFF00FFF01FFF00FFF01FFF007FE01FFE0 03FC01FFE001F801FFE0000001FFE0000001FFC0000001FFC0000001FFC0000001FFC000 0001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC00000 01FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001 FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FF C00000FFFFFFE000FFFFFFE000FFFFFFE000FFFFFFE000282E7DAD2F>114 D121 D E /Fs 9 117 df<0001FF0000000FFFE000003FFFF80000FF83FE0001FE00FF00 03F8003F8007F8003FC00FF0001FE00FE0000FE01FE0000FF01FE0000FF03FE0000FF83F E0000FF83FC00007F87FC00007FC7FC00007FC7FC00007FC7FC00007FC7FC00007FCFFC0 0007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC000 07FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007 FEFFC00007FEFFC00007FEFFC00007FEFFC00007FE7FC00007FC7FC00007FC7FC00007FC 7FC00007FC7FC00007FC3FE0000FF83FE0000FF83FE0000FF81FE0000FF01FE0000FF00F F0001FE00FF0001FE007F8003FC003FC007F8001FE00FF0000FF83FE00007FFFFC00001F FFF0000001FF000027397CB730>48 D<00003C000000007C00000001FC00000007FC0000 003FFC0000FFFFFC0000FFFFFC0000FFFFFC0000FFCFFC0000000FFC0000000FFC000000 0FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000F FC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC 0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC00 00000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000 000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC000000 0FFC0000000FFC0000000FFC0000000FFC0000000FFC00007FFFFFFF807FFFFFFF807FFF FFFF807FFFFFFF80213879B730>I<0000001FFE000060000003FFFFE000E000001FFFFF F801E000007FFFFFFE03E00001FFFC00FF07E00007FFC0001FCFE0000FFF000007FFE000 1FFC000001FFE0003FF0000000FFE0007FE00000007FE000FFC00000003FE001FF800000 001FE003FF000000000FE007FF000000000FE00FFE0000000007E00FFE0000000007E01F FC0000000003E01FFC0000000003E03FF80000000003E03FF80000000001E03FF8000000 0001E07FF80000000001E07FF80000000001E07FF00000000000007FF0000000000000FF F0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000 000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FF F0000000000000FFF00000000000007FF00000000000007FF00000000000007FF8000000 0000007FF80000000001E03FF80000000001E03FF80000000001E03FF80000000001E01F FC0000000001E01FFC0000000003C00FFE0000000003C00FFE0000000003C007FF000000 00078003FF00000000078001FF800000000F0000FFC00000001F00007FE00000003E0000 3FF00000007C00001FFC000000F800000FFF000003F0000007FFC0000FE0000001FFFC00 7F800000007FFFFFFF000000001FFFFFFC0000000003FFFFE000000000001FFE0000003B 3D7BBB46>67 D<003FFE00000001FFFFE0000007FFFFF800000FE007FC00000FF001FE00 001FF800FF00001FF8007F80001FF8007FC0001FF8003FC0000FF0003FE00007E0003FE0 0003C0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000FFFFE0 00001FFFFFE000007FF83FE00003FF803FE00007FC003FE0000FF0003FE0001FE0003FE0 003FE0003FE0007FC0003FE0007FC0003FE000FF80003FE000FF80003FE000FF80003FE0 00FF80003FE000FF80007FE0007FC0007FE0007FC000DFE0003FE0039FF0001FF80F0FFF E007FFFE0FFFE001FFFC07FFE0003FE000FFE02B267DA52F>97 D<0001FF8000000FFFF0 00007FFFFC0000FF81FE0003FE007F8007F8003F800FF0001FC00FF0000FE01FE0000FE0 3FE0000FF03FE00007F07FC00007F07FC00007F87FC00007F8FFC00007F8FFC00007F8FF FFFFFFF8FFFFFFFFF8FFFFFFFFF8FFC0000000FFC0000000FFC0000000FFC00000007FC0 0000007FC00000007FC00000003FE00000003FE00000781FE00000781FF00000780FF000 00F007F80001F003FC0003E001FE000FC000FFC07F80003FFFFE00000FFFF8000000FFC0 0025267DA52C>101 D<00FE00000000FFFE00000000FFFE00000000FFFE00000000FFFE 0000000007FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 00FF800003FE03FFF00003FE0FFFF80003FE1E03FC0003FE3801FE0003FE6001FF0003FE C000FF0003FFC000FF8003FF8000FF8003FF0000FF8003FF0000FF8003FF0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF80FFFFF83FFFFEFFFFF83FFFFEFFFF F83FFFFEFFFFF83FFFFE2F3C7CBB36>104 D<00FE01FF0000FFFE1FFFF000FFFE7FFFFC 00FFFEFC07FE00FFFFF001FF0007FFC0007F8003FF00003FC003FE00003FE003FE00001F F003FE00001FF803FE00000FF803FE00000FFC03FE00000FFC03FE000007FC03FE000007 FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007 FE03FE000007FE03FE000007FE03FE000007FC03FE00000FFC03FE00000FFC03FE00000F FC03FE00000FF803FE00001FF003FE00001FF003FF00003FE003FF80007FC003FFC000FF 8003FFE001FF0003FEFC07FE0003FE7FFFF80003FE1FFFE00003FE03FE000003FE000000 0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE000000 0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE000000 00FFFFF8000000FFFFF8000000FFFFF8000000FFFFF80000002F377DA536>112 D<01FC03F000FFFC0FFC00FFFC1FFF00FFFC3C3F80FFFC707F8007FCE0FFC003FCC0FFC0 03FD80FFC003FD80FFC003FF807F8003FF003F0003FF001E0003FF00000003FE00000003 FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE 00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00 000003FE00000003FE00000003FE00000003FE00000003FE000000FFFFFC0000FFFFFC00 00FFFFFC0000FFFFFC000022267DA528>114 D<000F0000000F0000000F0000000F0000 000F0000001F0000001F0000001F0000001F0000003F0000003F0000007F0000007F0000 00FF000001FF000003FF000007FF00001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C00FF0078 00FF8078007F80F0003FC1E0001FFFC0000FFF800001FE001E377EB626>116 D E /Ft 84 125 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C000C00180018003000300030007000600060006000E0 00E000E000E000E000E000E000E000E000E000E000E00060006000600070003000300030 00180018000C000C000600020001000080004000200B2E7DA112>I<8000400020001000 08000C00060006000300030001800180018001C000C000C000C000E000E000E000E000E0 00E000E000E000E000E000E000E000C000C000C001C00180018001800300030006000600 0C00080010002000400080000B2E7DA112>I<01800180018001800180C183F18F399C0F F003C003C00FF0399CF18FC1830180018001800180018010147DA117>I<000600000006 000000060000000600000006000000060000000600000006000000060000000600000006 000000060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600000006 000000060000000600000006000000060000000600000006000000060000000600000006 00000006000000060000000600001C207D9A23>I<70F8FCFC7404040408081010204006 0E7C840D>II<70F8F8F87005057C840D>I<0001000300030006 00060006000C000C000C00180018001800300030003000600060006000C000C000C00180 018001800300030003000600060006000C000C000C001800180018003000300030006000 60006000C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070 0380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003 F000121F7E9D17>I<018003800F80F38003800380038003800380038003800380038003 800380038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C 9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007 C00007C0000780000780000F00000E00001C0000380000700000600000C0000180000300 000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C0010 0E00200F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003 F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040 0780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00 002E00002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00 200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E00 00FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000 0010000010000011F000161C00180E001007001007800003800003800003C00003C00003 C07003C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17 >I<007C000182000701000E03800C07801C0780380300380000780000700000700000F1 F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C070 03C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC0 7FFF807FFF80400100800200800200800400000800000800001000002000002000004000 00400000C00000C00001C000018000038000038000038000038000078000078000078000 078000078000078000078000030000121F7D9D17>I<03F0000C0C001006003003002001 806001806001806001807001807803003E03003F06001FC8000FF00003F80007FC000C7E 00103F00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001001002 000C0C0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700700380F0 0380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003 E3C0000380000380000380000700300700780600780E00700C002018001070000FC00012 1F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F8 700000000000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0 FFFFFFF00000000000000000000000000000000000000000000000000000000000000000 FFFFFFF07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C00380070 00E000C00180018001000300020002000200020002000200000000000000000000000700 0F800F800F8007000E207D9F15>63 D<000100000003800000038000000380000007C000 0007C0000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800 002078000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F00 01000F0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE 1F207F9F22>65 DI<000FC04000 7030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C 00004078000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8 000000F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007 0001000380020001C0040000703800000FC0001A217D9F21>III< FFFFFF000F800F0007800300078003000780010007800180078000800780008007800080 07800080078080000780800007808000078080000781800007FF80000781800007808000 078080000780800007808000078000000780000007800000078000000780000007800000 07800000078000000FC00000FFFE0000191F7E9E1E>I<000FE0200078186000E004E003 8002E0070001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8 000000F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E078 0001E07C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000 E0046000781820000FE0001E217D9F24>III<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C0000 3C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0020 3C00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>IIIII<001F800000F0F00001C0380007801E000F000F 000E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001 F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003 E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F0 00001F80001C217D9F23>II82 D<07E0800C1980100780 300380600180600180E00180E00080E00080E00080F00000F000007800007F00003FF000 1FFC000FFE0003FF00001F800007800003C00003C00001C08001C08001C08001C08001C0 C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F00 60400F0020400F0020C00F0030800F0010800F0010800F0010800F0010000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F80 0007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003C0020001E0040001F00C 0000F0080000781000007C1000003C2000003E4000001E4000000F8000000F8000000780 000003C0000007E0000005E0000009F0000018F8000010780000207C0000603C0000401E 0000801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I< FFF003FF1F8000F80F8000600780004007C0004003E0008001E0008001F0010000F00300 00F80200007C0400003C0400003E0800001E0800001F1000000FB0000007A0000007C000 0003C0000003C0000003C0000003C0000003C0000003C0000003C0000003C0000003C000 0003C0000003C0000007C000007FFE00201F7F9E22>I<7FFFF87C00F87000F06001E040 01E0C003C0C003C0800780800F80800F00001E00001E00003C00003C0000780000F80000 F00001E00001E00003C00403C0040780040F80040F000C1E000C1E00083C00183C001878 0038F801F8FFFFF8161F7D9E1C>II<0804100820 10201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE000303000781800781C00300E00000E00000E00000E0000FE 00078E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C01414 7E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E0078 0E00780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19> I<03F80E0C1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C 020E0C03F010147E9314>I<000380003F80000380000380000380000380000380000380 00038000038000038000038003E380061B801C0780380380380380700380700380F00380 F00380F00380F00380F00380F003807003807003803803803807801C07800E1B8003E3F8 15207E9F19>I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF 80F00000F00000F000007000007000003800801800800C010007060001F80011147F9314 >I<007C00C6018F038F07060700070007000700070007000700FFF00700070007000700 070007000700070007000700070007000700070007000700070007007FF01020809F0E> I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C 001E380033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C000 30C00030C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E 81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C000000 000000000000000000000E007E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E00000000000 00000000000000007007F000F00070007000700070007000700070007000700070007000 700070007000700070007000700070007000706070F060F0C061803F000C28829E0E>I< 0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C00 0E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE 618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E 81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C0 3801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C0 3801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00 F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01 C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D 7F9319>I<03E0800619801C05803C0780380380780380700380F00380F00380F00380F0 0380F00380F003807003807803803803803807801C0B800E138003E38000038000038000 0380000380000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F 0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312 >I<1F9030704030C010C010C010E00078007F803FE00FF00070803880188018C018C018 E030D0608F800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E 000E000E000E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F 9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800 003800003C00007C00004E000087000107000303800201C00601E01E01E0FF07FE171480 9318>II<3FFF380E200E201C4038 4078407000E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314> III E /Fu 45 122 df<70F8FCFC7404040404080810102040060F7C840E>44 D I<70F8F8F87005057C840E>I<01F000071C000C06001803003803803803807001C07001 C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001 E0F001E0F001E0F001E0F001E07001C07001C07001C07803C03803803803801C07000C06 00071C0001F00013227EA018>48 D<008003800F80F38003800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 800380038007C0FFFE0F217CA018>I<03F8000C1E001007002007804007C07807C07803 C07807C03807C0000780000780000700000F00000E0000380003F000001C00000F000007 800007800003C00003C00003E02003E07003E0F803E0F803E0F003C04003C04007802007 80100F000C1C0003F00013227EA018>51 D<000200000600000E00000E00001E00001E00 002E00004E00004E00008E00008E00010E00020E00020E00040E00040E00080E00100E00 100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E00000E00000E00 000E00001F0001FFF015217FA018>I<1000801E07001FFF001FFE001FF80013E0001000 0010000010000010000010000010000010F800130E001407001803801003800001C00001 C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C04003C04003802007 001006000C1C0003F00013227EA018>I<007E0001C1000300800601C00E03C01C03C018 0180380000380000780000700000700000F0F800F30C00F40600F40300F80380F801C0F0 01C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C03801C01803801C 03000C0600070C0001F00013227EA018>I<01F800060E000803001001802001802000C0 6000C06000C06000C07000C07801803E01003F02001FC4000FF80003F80003FC00067F00 083F80100F803007C06001C06000E0C000E0C00060C00060C00060C000606000406000C0 3000801803000E0E0003F00013227EA018>56 D<01F000060C000C060018070038038070 0380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003E038 03E01805E00C05E00619E003E1E00001C00001C00001C000038000038030030078070078 0600700C002018001030000FC00013227EA018>I<0001800000018000000180000003C0 000003C0000003C0000005E0000005E000000DF0000008F0000008F0000010F800001078 000010780000203C0000203C0000203C0000401E0000401E0000401E0000800F0000800F 0000FFFF000100078001000780030007C0020003C0020003C0040003E0040001E0040001 E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E006 3001C00170038000F0070000F00E0000701E0000701C0000303C0000303C0000307C0000 107800001078000010F8000000F8000000F8000000F8000000F8000000F8000000F80000 00F800000078000000780000107C0000103C0000103C0000101C0000201E0000200E0000 40070000400380008001C0010000E0020000381C000007E0001C247DA223>67 D69 DI< FFFC0FC00780078007800780078007800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800FC0FFFC0E227EA1 12>73 D 77 DI 80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020E00020 E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E00001E0 0000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0C80180 C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008400780 08C007800C80078004800780048007800480078004000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000FC0 0003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700000700 00FF0007C7001E07003C0700780700700700F00708F00708F00708F00F087817083C2390 0FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E003C0E 001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C 80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000700000 F00000F00000F00000F00000F00000F00000F000007000007800403800401C00800C0100 07060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800E07800 E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800E01801 E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C07801E070 00E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C00400E 008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700070007 00070007000700FFF8070007000700070007000700070007000700070007000700070007 00070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C07001C 07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010000018 00001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860003060 00303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E00F0070 0F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000000000 0000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E001C000 00000000000000000000000000000001E00FE001E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0F1 8061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E02000E0400 0E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C00E01E0 0E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00FE60E183801E807201C0 0F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60C01E80E00F00 700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E0038 0E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E0038 0F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E0000 0E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E 000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>III121 D E /Fv 20 118 df45 D68 D73 D77 D 80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E070 0000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF0000 0FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880 00007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0F0 0000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fw 5 85 df<00000000600000000000E00000000000E00000000001E0 0000000003E00000000003E00000000007E00000000007F0000000000DF0000000000DF0 0000000019F00000000039F00000000031F00000000061F00000000061F000000000C1F0 00000000C1F00000000181F00000000381F00000000301F00000000601F00000000601F0 0000000C01F80000000C00F80000001800F80000003800F80000003000F80000006000F8 0000006000F8000000C000F8000000C000F8000001FFFFF8000001FFFFF80000030000F8 0000060000F80000060000F800000C0000F800000C0000FC00001800007C00001800007C 00003000007C00007000007C00006000007C0000C000007C0000C000007C0001C000007C 0003C000007C000FE00000FE00FFF8001FFFE0FFF8001FFFE02B327CB134>65 D<000FFFFFFC0000000FFFFFFF800000007E000FE00000007C0003F00000007C0000F800 00007C00007C0000007C00007C000000F800003E000000F800001E000000F800001F0000 00F800001F000001F000001F000001F000000F800001F000000F800001F000000F800003 E000000F800003E000000F800003E000000F800003E000001F800007C000001F800007C0 00001F800007C000001F800007C000001F80000F8000001F00000F8000003F00000F8000 003F00000F8000003F00001F0000007E00001F0000007E00001F0000007E00001F000000 7C00003E000000FC00003E000000F800003E000001F800003E000001F000007C000003E0 00007C000003E000007C000007C000007C00000F800000F800000F000000F800001E0000 00F800003C000000F8000078000001F00001F0000001F00003E0000001F0000F80000003 F0007F000000FFFFFFFC000000FFFFFFC000000031317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007E0003F800007C0000F800007C00007800007C0000 7800007C0000300000F80000300000F80000300000F80000300000F80000300001F00000 300001F00000300001F00000300001F00000600003E00180600003E00180000003E00180 000003E00180000007C00300000007C00300000007C00700000007C01F0000000FFFFE00 00000FFFFE0000000F801E0000000F800E0000001F000C0000001F000C0000001F000C00 00001F000C0000003E00180000003E00180000003E00000000003E00000000007C000000 00007C00000000007C00000000007C0000000000F80000000000F80000000000F8000000 0000F80000000001F00000000001F00000000001F00000000003F000000000FFFFE00000 00FFFFE00000002D317BB02E>70 D<000FFFFFE000000FFFFFFC0000007E003F0000007C 000F8000007C0007C000007C0003E000007C0003E00000F80001F00000F80001F00000F8 0001F00000F80003F00001F00003F00001F00003F00001F00003F00001F00003E00003E0 0007E00003E00007C00003E0000FC00003E0000F800007C0001F000007C0003E000007C0 0078000007C001E000000F800F8000000FFFFC0000000F800E0000000F80070000001F00 038000001F0003C000001F0001E000001F0001E000003E0001F000003E0001F000003E00 01F000003E0001F000007C0003F000007C0003F000007C0003E000007C0003E00000F800 07E00000F80007E00000F80007E00000F80007E01801F00007E03001F00007E03001F000 07E02003F00003E060FFFF8001E0C0FFFF8000F180000000003E002D327BB033>82 D<07FFFFFFFFF007FFFFFFFFF00FE007E007F00F8007C001F00E0007C000E00C0007C000 E01C0007C000E018000F8000E018000F8000E030000F8000C030000F8000C030001F0000 C060001F0000C060001F0000C060001F0000C0C0003E00018000003E00000000003E0000 0000003E00000000007C00000000007C00000000007C00000000007C0000000000F80000 000000F80000000000F80000000000F80000000001F00000000001F00000000001F00000 000001F00000000003E00000000003E00000000003E00000000003E00000000007C00000 000007C00000000007C00000000007C0000000000F80000000000F80000000000F800000 00000F80000000001F00000000001F00000000001F00000000003F000000003FFFFF0000 003FFFFF0000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 799 911 a Fw(D)25 b(R)g(A)g(F)g(T)225 1002 y Fv(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n (terface)621 1196 y Fu(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum) 832 1323 y(Ma)o(y)h(16,)g(1996)77 1381 y(This)h(w)o(ork)f(w)o(as)h (supp)q(orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran) o(t)h(ASC-9310330,)i(the)192 1439 y(National)d(Science)f(F)l(oundation) i(Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e) 76 1497 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 49 a Ft(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 361 a Fs(Chapter)31 b(10)75 568 y Fr(I/O)75 798 y Ft(This)20 b(is)f(a)g(prop)q(osed)h(starting)e(p)q(oin)o(t)i(for) e(the)h Fq(MPI-2)g Ft(I/O)g(c)o(hapter.)31 b(It)20 b(is)f(directly)i (deriv)o(ed)f(from)75 855 y(MPI-IO)f(V)l(ersion)h(0.5)d([11])h(with)h (minor)f(editorial)i(c)o(hanges.)30 b(A)19 b(historical)h(p)q(ersp)q (ectiv)o(e)g(\(e.g.)29 b(the)75 911 y(argumen)o(ts)20 b(that)g(led)i(to)e(v)m(arious)h(design)g(decisions\))h(can)f(b)q(e)h (found)f(at)f(the)g(MPI-IO)i(W)l(eb)f(page)75 968 y Fp (http://lovelace.nas.nasa.g)o(ov/MPI-)o(IO/)p Ft(.)166 1024 y(Direct)f(all)h(commen)o(ts)f(regarding)g(the)g Fq(MPI-2)f Ft(I/O)i(Chapter)f(to)f(the)h Fp(mpi-io@mcs.anl.gov)75 1081 y Ft(mailing)d(list.)j(See)c Fp(http://www.mcs.anl.gov/Proj)o (ects/mp)o(i/mpi2/m)o(pi2.htm)o(l)c Ft(for)j(details.)166 1137 y(Ma)s(jor)e(issues)j(to)f(b)q(e)h(discussed)h(regarding)e(this)g (prop)q(osal:)75 1212 y Fo(La)o(y)o(erabilit)o(y)23 b Ft(Can/should)14 b(w)o(e)f(pro)o(vide)h(enough)f Fq(MPI-2)g Ft(facilities)i(to)e(allo)o(w)g(I/O)h(to)f(b)q(e)h(la)o(y)o(ered)f(on) 189 1268 y(top)h(of)h Fq(MPI-2)p Ft(?)75 1355 y Fo(P)o(ortable)j (O\013sets)k Ft(Ho)o(w)9 b(do)i(w)o(e)f(pro)o(vide)h(p)q(ortable)g (o\013sets)e(with)i(more)f(than)g(32)g(bits?)20 b(P)o(ossibilities:)189 1411 y(long)15 b(long/in)o(teger*8,)g(opaque)g(o\013sets,)f(in)o(t)h ([2]/in)o(teger)f(\(2\),)g(double/double)k(precision.)75 1497 y Fo(MPI)p 184 1497 16 2 v 18 w(File)g(vs.)k(MPI)p 497 1497 V 18 w(Comm)g Ft(Should)16 b(w)o(e)e(scrap)h(the)g(MPI-IO)g(v) o(er)g(0.5)e(idea)j(of)e(\014le)i(handles)g(and)189 1554 y(simply)g(use)g Fq(MPI)e Ft(comm)o(unicators?)75 1640 y Fo(MPI)p 184 1640 V 18 w(File)p 287 1640 V 20 w(Con)o(trol)22 b Ft(The)h(\014le)g(con)o(trol)f(in)o(terface)g(cannot)g(supp)q(ort)g (F)o(OR)l(TRAN)h(and)g(m)o(ust)e(b)q(e)189 1696 y(revised.)75 1783 y Fo(End-of-File)k Ft(The)14 b(curren)o(t)g(de\014nition)i(for)d (up)q(dating)i(\014le)g(p)q(oin)o(ters)g(when)f(end-of-\014le)i(is)e (reac)o(hed)g(is)189 1839 y(con)o(tradictory)l(.)k(The)12 b(seman)o(tics)g(de\014ned)h(in)g(the)f(last)f(paragraph)g(of)h (section)g(10.1.3)e(con)o(tradict)189 1896 y(the)15 b(language)g(in)h (the)g(in)o(tro)q(ductions)g(of)f(sections)g(10.4)f(and)i(10.5.)75 1982 y Fo(Shared)h(\014le)h(p)q(oin)o(ters)23 b Ft(There)15 b(is)h(some)e(con)o(tro)o(v)o(ersy)g(as)g(to)g(whether)h(w)o(e)g (should)h(k)o(eep)f(shared)g(\014le)189 2038 y(p)q(oin)o(ters,)g(and)g (if)h(w)o(e)f(do,)g(the)g(exact)g(seman)o(tics)g(of)g(shared)g(\014le)h (p)q(oin)o(ter)g(op)q(erations.)75 2125 y Fo(Filet)o(yp)q(e)i (constructors)23 b Ft(Since)15 b(\014let)o(yp)q(e)h(constructors)d(are) h(simply)h(new)g(datat)o(yp)q(e)e(constructors,)189 2181 y(should)19 b(w)o(e)e(mo)o(v)o(e)g(this)i(section)f(to)f(the)h(\\Datat) o(yp)q(e)f(Impro)o(v)o(emen)o(ts")g(subsection)i(in)g(Miscel-)189 2237 y(lan)o(y?)75 2324 y Fo(File)f(In)o(terop)q(erabilit)o(y)24 b Ft(Ho)o(w)16 b(m)o(uc)o(h)g(\014le)i(in)o(terop)q(erabilit)o(y)g (should)g(b)q(e)f(sp)q(eci\014ed.)26 b(Minimally)l(,)19 b(all)189 2380 y(applications)d(on)f(a)g(single)i(mac)o(hine,)e(using)h (a)f(single)h Fq(MPI)f Ft(implemen)o(tation)h(should)g(ha)o(v)o(e)f (the)189 2437 y(same)h(canonical)i(view)f(of)g(\014les.)25 b(What)16 b(ab)q(out)g(b)q(et)o(w)o(een)h Fq(MPI)g Ft(implemen)o (tations)g(or)g(b)q(et)o(w)o(een)189 2493 y(mac)o(hines)f(\(e.g.)j (heterogeneous)c(t)o(yp)q(es,)g(v)m(arying)h(ph)o(ysical)g(\014le)g (structures\).)75 2579 y Fo(Simpler)i(In)o(terface)k Ft(Sev)o(eral)15 b(users)f(ha)o(v)o(e)g(complained)i(of)e(the)h(steep)f (learning)i(curv)o(e)e(required)i(for)189 2636 y(this)g(in)o(terface.) 24 b(A)17 b(prop)q(osed)f(simpli\014cation)j(of)d Fq(MPI)p 1149 2636 14 2 v 16 w(OPEN)h Ft(\(see)f(section)h(10.7\))e(w)o(ould)i (ease)189 2692 y(migration)e(from)f(the)i(standard)e(UNIX)i(in)o (terface)g(to)e(the)h Fq(MPI)g Ft(I/O)h(in)o(terface.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Ft(2)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fm(10.1)60 b(Intro)r(duction)75 151 y Ft(Di\013eren)o(t) 11 b(parallel)i(\014le)g(systems)e(supp)q(ort)h(di\013eren)o(t)f(in)o (terfaces)h(and)g(the)f(lac)o(k)h(of)f(a)g(standard)g(p)q(ortable)75 207 y(in)o(terface)j(mak)o(es)f(it)h(imp)q(ossible)i(to)d(dev)o(elop)h (a)f(truly)h(p)q(ortable)g(parallel)h(program.)k(It)13 b(is)h(not)g(feasible)75 264 y(to)f(dev)o(elop)i(large)f(scien)o (ti\014c)h(applications)h(from)d(scratc)o(h)g(for)h(eac)o(h)g (generation)g(of)f(parallel)i(mac)o(hine,)75 320 y(and,)g(in)g(the)g (scien)o(ti\014c)h(w)o(orld,)f(p)q(ortabilit)o(y)g(and)g(p)q (erformance)g(are)f(inseparable|a)j(program)c(is)i(not)75 377 y(considered)i(truly)e(p)q(ortable)h(unless)g(it)g(not)e(only)i (compiles,)g(but)g(also)f(runs)g(e\016cien)o(tly)l(.)166 433 y(The)e(signi\014can)o(t)g(optimizations)g(required)h(for)d (e\016ciency)j(\(e.g.)k(grouping)13 b([7],)f(collectiv)o(e)i(bu\013er-) 75 489 y(ing)k([3,)f(8)o(,)h(9)o(,)g(10)o(],)f(and)h(disk-directed)i (I/O)e([5)o(]\))f(can)h(only)g(b)q(e)g(implemen)o(ted)i(as)d(part)g(of) g(a)g(parallel)75 546 y(I/O)d(en)o(vironmen)o(t)g(if)g(it)f(supp)q (orts)h(a)f(high-lev)o(el)j(in)o(terface)d(to)g(describ)q(e)i(the)f (partitioning)g(of)f(\014le)i(data)75 602 y(among)c(pro)q(cesses)g(and) h(a)f(collectiv)o(e)i(in)o(terface)e(describing)i(complete)f(transfers) f(of)f(global)i(data)f(struc-)75 659 y(tures)i(b)q(et)o(w)o(een)h(pro)q (cess)g(memories)g(and)f(\014les.)21 b(In)14 b(addition,)g(further)g (e\016ciencies)i(can)d(b)q(e)h(gained)h(via)75 715 y(supp)q(ort)d(for)g (async)o(hronous)f(I/O,)i(strided)f(requests,)h(and)f(con)o(trol)g(o)o (v)o(er)f(ph)o(ysical)i(\014le)g(la)o(y)o(out)f(on)g(stor-)75 772 y(age)i(devices)h(\(disks\).)20 b(The)14 b(I/O)g(en)o(vironmen)o(t) h(describ)q(ed)h(in)e(this)h(section)f(pro)o(vides)h(these)f (facilities.)166 828 y(Emphasis)h(has)f(b)q(een)i(put)e(in)i(remaining) f(as)f Fq(MPI)p Ft(-friendly)i(as)e(p)q(ossible.)22 b(When)14 b(op)q(ening)i(a)e(\014le,)75 885 y(a)k(comm)o(unicator)h(is)g(sp)q (eci\014ed)i(to)d(determine)i(whic)o(h)f(group)g(of)f(pro)q(cesses)h (can)g(get)f(access)h(to)f(the)75 941 y(\014le)g(in)g(subsequen)o(t)g (I/O)g(op)q(erations.)25 b(Accesses)18 b(to)f(a)g(\014le)h(can)f(b)q(e) h(indep)q(enden)o(t)h(\(no)e(co)q(ordination)75 998 y(b)q(et)o(w)o(een) f(pro)q(cesses)g(tak)o(es)f(place\))h(or)f(collectiv)o(e)i(\(eac)o(h)e (pro)q(cess)h(of)f(the)h(group)f(asso)q(ciated)h(with)g(the)75 1054 y(comm)o(unicator)h(m)o(ust)h(participate)g(in)h(the)f(collectiv)o (e)h(access\).)28 b Fq(MPI)17 b Ft(deriv)o(ed)i(datat)o(yp)q(es)e(are)g (used)75 1110 y(for)f(expressing)h(the)f(data)g(la)o(y)o(out)g(in)h (the)f(\014le)i(as)e(w)o(ell)h(as)f(the)g(distribution)i(of)e(the)h (\014le)g(data)f(among)75 1167 y(pro)q(cesses.)31 b(In)19 b(addition,)h(eac)o(h)f(access)f(\(read)h(or)f(write\))g(op)q(erates)h (on)f(a)g(n)o(um)o(b)q(er)h(of)g Fq(MPI)f Ft(ob)s(jects)75 1223 y(whic)o(h)e(can)f(b)q(e)h(of)f(an)o(y)g Fq(MPI)f Ft(basic)i(or)f(deriv)o(ed)h(datat)o(yp)q(es.)75 1345 y Fl(10.1.1)49 b(Data)17 b(P)o(a)o(rtitioning)75 1431 y Ft(Instead)d(of)g(de\014ning)h(\014le)g(access)f(mo)q(des)h(to)e (express)h(the)g(common)g(patterns)f(for)g(accessing)i(a)f(shared)75 1487 y(\014le)21 b(\(broadcast,)f(reduction,)h(scatter,)f(gather\),)g (w)o(e)g(c)o(hose)f(another)h(approac)o(h)f(whic)o(h)i(consists)f(of)75 1544 y(expressing)d(the)g(data)f(partitioning)h(via)g(deriv)o(ed)h (datat)o(yp)q(es.)23 b(Compared)17 b(to)e(a)i(limited)h(set)e(of)h (pre-)75 1600 y(de\014ned)j(access)f(patterns,)g(this)g(approac)o(h)f (has)h(the)g(adv)m(an)o(tage)f(of)g(added)h(\015exibili)q(t)o(y)i(and)d (expres-)75 1657 y(siv)o(eness.)166 1713 y Fq(MPI)11 b Ft(deriv)o(ed)i(datat)o(yp)q(es)f(are)f(used)i(to)e(describ)q(e)j(ho) o(w)e(data)f(is)i(laid)g(out)e(in)i(the)f(user's)g(bu\013er.)19 b(W)l(e)75 1770 y(extend)c(this)h(use)f(to)f(describ)q(e)j(ho)o(w)d (the)h(data)f(is)i(laid)g(out)e(in)i(the)f(\014le)h(as)f(w)o(ell.)20 b(Th)o(us)15 b(w)o(e)g(distinguish)75 1826 y(b)q(et)o(w)o(een)22 b(t)o(w)o(o)d(\(p)q(oten)o(tially)k(di\013eren)o(t\))e(deriv)o(ed)h (datat)o(yp)q(es)f(that)f(are)h(used:)33 b(the)21 b Fq(\014let)o(yp)q (e)p Ft(,)j(whic)o(h)75 1883 y(describ)q(es)c(the)f(la)o(y)o(out)f(in)i (the)e(\014le,)j(and)d(the)h Fq(buft)o(yp)q(e)p Ft(,)i(whic)o(h)f (describ)q(es)g(the)f(la)o(y)o(out)f(in)h(the)g(user's)75 1939 y(bu\013er.)g(In)12 b(addition,)h(b)q(oth)e Fq(\014let)o(yp)q(e)i Ft(and)f Fq(buft)o(yp)q(e)h Ft(are)e(deriv)o(ed)i(from)e(a)g(third)h Fq(MPI)f Ft(datat)o(yp)q(e,)g(referred)75 1995 y(to)k(as)g(the)h Fk(elementary)g Ft(datat)o(yp)q(e)f Fq(et)o(yp)q(e)p Ft(.)22 b(The)16 b(purp)q(ose)h(of)e(the)h(elemen)o(tary)g(datat)o(yp)q (e)f(is)h(to)f(ensure)75 2052 y(consistency)j(b)q(et)o(w)o(een)e(the)h (t)o(yp)q(e)g(signatures)f(of)h Fq(\014let)o(yp)q(e)h Ft(and)e Fq(buft)o(yp)q(e)p Ft(.)26 b(O\013sets)16 b(for)g(accessing)i (data)75 2108 y(within)e(the)g(\014le)g(are)f(expressed)h(as)f(an)g(in) o(tegral)g(n)o(um)o(b)q(er)h(of)e Fq(et)o(yp)q(e)j Ft(items.)166 2165 y(The)d(\014let)o(yp)q(e)g(de\014nes)h(a)e(data)f(pattern)h(that)g (is)h(replicated)h(throughout)e(the)g(\014le)i(\(or)d(part)h(of)g(the) 75 2221 y(\014le|see)19 b(the)f(concept)f(of)g(displacemen)o(t)i(b)q (elo)o(w\))f(to)f(tile)h(the)f(\014le)i(data.)25 b(It)18 b(should)g(b)q(e)g(noted)g(that)75 2278 y Fq(MPI)g Ft(deriv)o(ed)h (datat)o(yp)q(es)e(consist)i(of)e(\014elds)j(of)d(data)h(that)f(are)h (lo)q(cated)h(at)e(sp)q(eci\014ed)k(o\013sets.)27 b(This)75 2334 y(can)15 b(lea)o(v)o(e)h(\\holes")f(b)q(et)o(w)o(een)h(the)f (\014elds,)h(that)e(do)i(not)e(con)o(tain)i(an)o(y)f(data.)k(In)d(the)f (con)o(text)g(of)f(tiling)75 2391 y(the)g(\014le)g(with)g(the)g (\014let)o(yp)q(e,)g(the)g(pro)q(cess)g(can)g(only)g(access)f(the)h (\014le)h(data)e(that)g(matc)o(hes)g(items)g(in)i(the)75 2447 y(\014let)o(yp)q(e.)21 b(It)15 b(cannot)g(access)h(\014le)g(data)e (that)h(falls)h(under)g(holes)g(\(see)f(Figure)g(10.1\).)166 2504 y(Data)f(whic)o(h)i(resides)h(in)f(holes)g(can)f(b)q(e)i(accessed) f(b)o(y)f(other)g(pro)q(cesses)h(whic)o(h)g(use)g(complemen-)75 2560 y(tary)d(\014let)o(yp)q(es)i(\(see)f(Figure)h(10.2\).)j(Th)o(us,)c (\014le)h(data)e(can)h(b)q(e)h(distributed)h(among)d(parallel)j(pro)q (cesses)75 2616 y(in)i(disjoin)o(t)f(c)o(h)o(unks.)26 b(Filet)o(yp)q(e)18 b(constructors)e(help)j(the)e(user)g(create)f (complemen)o(tary)i(\014let)o(yp)q(es)g(for)75 2673 y(common)f (distribution)i(patterns,)d(suc)o(h)i(as)f(broadcast/reduce,)g (scatter/gather,)f(and)h(HPF)g(distri-)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(3)375 289 y @beginspecial 0 @llx 0 @lly 592 @urx 141 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-filetype.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -54.0 181.0 translate 0.900 -0.900 scale 0.500 setlinewidth % Polyline n 179 84 m 179 69 l 159 69 l 159 84 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 199 84 m 199 69 l 179 69 l 179 84 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 179 164 m 179 149 l 59 149 l 59 164 l clp gs col-1 s gr % Polyline n 99 164 m 99 149 l 79 149 l 79 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 119 164 m 119 149 l 99 149 l 99 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 299 164 m 299 149 l 179 149 l 179 164 l clp gs col-1 s gr % Polyline n 219 164 m 219 149 l 199 149 l 199 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 239 164 m 239 149 l 219 149 l 219 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 419 164 m 419 149 l 299 149 l 299 164 l clp gs col-1 s gr % Polyline n 339 164 m 339 149 l 319 149 l 319 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 359 164 m 359 149 l 339 149 l 339 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 539 164 m 539 149 l 419 149 l 419 164 l clp gs col-1 s gr % Polyline n 459 164 m 459 149 l 439 149 l 439 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 479 164 m 479 149 l 459 149 l 459 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 659 164 m 659 149 l 539 149 l 539 164 l clp gs col-1 s gr % Polyline n 579 164 m 579 149 l 559 149 l 559 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 84 m 259 69 l 139 69 l 139 84 l clp gs col-1 s gr % Polyline n 599 164 m 599 149 l 579 149 l 579 164 l clp gs 0.85 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 179 104 m gs 1 -1 scale (holes) dup stringwidth pop 2 div neg 0 rmoveto col-1 show gr 0.000 setlinewidth % Ellipse n 694 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 704 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 714 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 1.000 setlinewidth % Polyline n 699 149 m 59 149 l 59 164 l 699 164 l gs col-1 s gr 0.500 setlinewidth % Interpolated spline n 244 189 m 232.036 184.536 227.036 182.036 224 179 curveto 222.482 177.482 221.232 174.982 219 169 curveto gs col-1 s gr n 219.923 177.194 m 219.000 169.000 l 223.670 175.796 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 244 194 m 180.787 195.844 153.287 194.594 134 189 curveto 126.939 186.952 118.189 181.952 99 169 curveto gs col-1 s gr n 104.512 175.133 m 99.000 169.000 l 106.750 171.818 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 189 m 327.853 187.647 331.603 186.397 334 184 curveto 336.397 181.603 337.647 177.853 339 169 curveto gs col-1 s gr n 335.814 176.606 m 339.000 169.000 l 339.768 177.210 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 194 m 382.589 191.880 410.089 189.380 429 184 curveto 434.743 182.366 442.243 178.616 459 169 curveto gs col-1 s gr n 451.066 171.247 m 459.000 169.000 l 453.057 174.716 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 199 m 448.285 202.925 504.535 200.425 544 189 curveto 551.064 186.955 559.814 181.955 579 169 curveto gs col-1 s gr n 571.251 171.819 m 579.000 169.000 l 573.489 175.134 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 11.00 scalefont setfont 59 139 m gs 1 -1 scale (tiling a file with the filetype:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 249 199 m gs 1 -1 scale (accesible data) col-1 show gr % Polyline n 159 59 m 159 44 l 139 44 l 139 59 l clp gs 0.85 setgray fill gr gs col-1 s gr % Interpolated spline n 159 99 m 155.926 96.344 154.676 95.094 154 94 curveto 152.488 91.554 151.238 87.804 149 79 curveto gs col-1 s gr n 149.033 87.246 m 149.000 79.000 l 152.909 86.261 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 194 99 m 208.466 98.119 214.716 96.869 219 94 curveto 222.030 91.971 224.530 88.221 229 79 curveto gs col-1 s gr n 223.711 85.326 m 229.000 79.000 l 227.310 87.071 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 59 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 573 387 a(Figure)15 b(10.1:)k(Tiling)e(a)e(\014le)h(using) g(a)f(\014let)o(yp)q(e)75 525 y(butions)h(\(see)f(Section)h(10.6\).)375 859 y @beginspecial 0 @llx 0 @lly 593 @urx 135 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-comp-filetypes.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -35.0 175.0 translate 0.900 -0.900 scale % Ellipse n 684 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 694 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 159 59 m 139 59 l 139 44 l 159 44 l clp gs col-1 s gr % Polyline n 259 134 m 259 119 l 139 119 l 139 134 l clp gs col-1 s gr % Polyline n 259 109 m 259 94 l 139 94 l 139 109 l clp gs col-1 s gr % Polyline n 259 84 m 259 69 l 139 69 l 139 84 l clp gs col-1 s gr % Polyline n 179 84 m 159 84 l 159 69 l 179 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 159 84 m 139 84 l 139 69 l 159 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 659 194 m 639 194 l 639 179 l 659 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 59 194 m 39 194 l 39 179 l 59 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 79 194 m 59 194 l 59 179 l 79 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 194 m 159 194 l 159 179 l 179 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 194 m 179 194 l 179 179 l 199 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 194 m 279 194 l 279 179 l 299 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 194 m 299 194 l 299 179 l 319 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 419 194 m 399 194 l 399 179 l 419 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 439 194 m 419 194 l 419 179 l 439 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 539 194 m 519 194 l 519 179 l 539 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 559 194 m 539 194 l 539 179 l 559 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 99 194 m 79 194 l 79 179 l 99 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 119 194 m 99 194 l 99 179 l 119 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 139 194 m 119 194 l 119 179 l 139 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 159 194 m 139 194 l 139 179 l 159 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 219 194 m 199 194 l 199 179 l 219 179 l clp gs 0.65 setgray fill gr gs col-1 s gr 0.000 setlinewidth % Ellipse n 674 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 239 194 m 219 194 l 219 179 l 239 179 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 39 169 m gs 1 -1 scale (tiling a file with the filetypes:) col-1 show gr % Polyline n 259 194 m 239 194 l 239 179 l 259 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 279 194 m 259 194 l 259 179 l 279 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 339 194 m 319 194 l 319 179 l 339 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 359 194 m 339 194 l 339 179 l 359 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 194 m 359 194 l 359 179 l 379 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 399 194 m 379 194 l 379 179 l 399 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 479 194 m 459 194 l 459 179 l 479 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 459 194 m 439 194 l 439 179 l 459 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 499 194 m 479 194 l 479 179 l 499 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 519 194 m 499 194 l 499 179 l 519 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 579 194 m 559 194 l 559 179 l 579 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 599 194 m 579 194 l 579 179 l 599 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 619 194 m 599 194 l 599 179 l 619 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 639 194 m 619 194 l 619 179 l 639 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 199 109 m 179 109 l 179 94 l 199 94 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 219 109 m 199 109 l 199 94 l 219 94 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 134 m 219 134 l 219 119 l 239 119 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 259 134 m 239 134 l 239 119 l 259 119 l clp gs 0.40 setgray fill gr gs col-1 s gr 1.000 setlinewidth % Polyline n 679 179 m 39 179 l 39 194 l 679 194 l gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 59 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (process 1 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 109 m gs 1 -1 scale (process 2 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 134 m gs 1 -1 scale (process 3 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 419 957 a(Figure)f(10.2:)k(P)o(artitioning)c(a)g(\014le)i (among)d(parallel)j(pro)q(cesses)166 1061 y(In)k(order)f(to)g(b)q (etter)g(illustrate)i(these)e(concepts,)i(let)f(us)f(consider)h(a)f (2-D)g(matrix,)h(stored)f(in)75 1117 y(ro)o(w)14 b(ma)s(jor)g(order)i (in)g(a)f(\014le,)h(that)f(is)h(to)f(b)q(e)h(transp)q(osed)f(and)h (partitioned)g(among)f(a)g(group)g(of)g(three)75 1174 y(pro)q(cesses)f(\(see)g(Figure)g(10.3\).)j(The)d(matrix)g(is)g(to)f(b) q(e)h(distributed)h(among)e(the)h(parallel)h(pro)q(cesses)f(in)75 1230 y(a)e(ro)o(w)g(cyclic)i(manner.)19 b(Eac)o(h)12 b(pro)q(cess)h(w)o(an)o(ts)e(to)h(store)g(in)h(its)g(o)o(wn)f(memory)g (the)g(transp)q(osed)h(p)q(ortion)75 1287 y(of)k(the)h(matrix)g(whic)o (h)h(is)f(assigned)h(to)e(it.)28 b(Using)19 b(appropriate)f Fq(\014let)o(yp)q(e)p Ft(s)h(and)f Fq(buft)o(yp)q(e)p Ft(s)i(allo)o(ws)e(the)75 1343 y(user)f(to)g(p)q(erform)g(that)f(task)h (v)o(ery)f(easily)l(.)28 b(In)17 b(addition,)i(the)e(elemen)o(tary)g (datat)o(yp)q(e)g(allo)o(ws)g(one)h(to)75 1400 y(ha)o(v)o(e)d(a)g(v)o (ery)g(generic)h(co)q(de)g(that)e(applies)j(to)d(an)o(y)h(t)o(yp)q(e)g (of)g(2-D)g(matrix.)375 2437 y @beginspecial 0 @llx 0 @lly 673 @urx 549 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-cyclic-trans.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -36.0 562.0 translate 0.900 -0.900 scale 0.500 setlinewidth % Polyline n 399 344 m 319 344 l 319 269 l 399 269 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 279 m 139 279 l 139 264 l 239 264 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 234 m 139 234 l 139 219 l 239 219 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 189 m 139 189 l 139 174 l 239 174 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 144 m 139 144 l 139 129 l 239 129 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 264 m 139 264 l 139 249 l 239 249 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 399 244 m 319 244 l 319 169 l 399 169 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 219 m 139 219 l 139 204 l 239 204 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 249 m 139 249 l 139 234 l 239 234 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 174 m 139 174 l 139 159 l 239 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 129 m 139 129 l 139 114 l 239 114 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 399 144 m 319 144 l 319 69 l 399 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 114 m 139 114 l 139 99 l 239 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 159 m 139 159 l 139 144 l 239 144 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 204 m 139 204 l 139 189 l 239 189 l clp gs 0.90 setgray fill gr gs col-1 s gr % Ellipse n 754 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Ellipse n 769 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Ellipse n 784 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Polyline n 159 99 m 159 279 l gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 99 m 179 279 l 179 274 l gs col-1 s gr % Polyline n 199 99 m 199 279 l gs col-1 s gr % Polyline n 219 99 m 219 279 l gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 164 619 m gs 1 -1 scale (buftype \(all processes\)) dup stringwidth pop neg 0 rmoveto col-1 show gr % Polyline n 579 624 m 579 609 l 519 609 l 519 624 l clp gs col-1 s gr % Polyline n 519 624 m 519 609 l 499 609 l 499 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 499 624 m 499 609 l 439 609 l 439 624 l clp gs col-1 s gr % Polyline n 439 624 m 439 609 l 419 609 l 419 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 339 69 m 339 144 l gs col-1 s gr % Polyline n 359 69 m 359 144 l gs col-1 s gr % Polyline n 379 69 m 379 144 l gs col-1 s gr % Polyline n 319 84 m 399 84 l gs col-1 s gr % Polyline n 319 99 m 399 99 l gs col-1 s gr % Polyline n 319 114 m 399 114 l gs col-1 s gr % Polyline n 319 129 m 399 129 l gs col-1 s gr % Polyline n 319 184 m 399 184 l gs col-1 s gr % Polyline n 319 199 m 399 199 l gs col-1 s gr % Polyline n 319 214 m 399 214 l gs col-1 s gr % Polyline n 319 229 m 399 229 l gs col-1 s gr % Polyline n 319 284 m 399 284 l gs col-1 s gr % Polyline n 319 299 m 399 299 l gs col-1 s gr % Polyline n 319 329 m 399 329 l gs col-1 s gr % Polyline n 339 169 m 339 244 l gs col-1 s gr % Polyline n 359 169 m 359 244 l gs col-1 s gr % Polyline n 379 169 m 379 244 l gs col-1 s gr % Polyline n 379 269 m 379 344 l gs col-1 s gr % Polyline n 359 269 m 359 344 l gs col-1 s gr % Polyline n 339 269 m 339 344 l gs col-1 s gr % Polyline n 319 314 m 399 314 l gs col-1 s gr % Polyline n 179 434 m 179 419 l 159 419 l 159 434 l clp gs col-1 s gr % Polyline n 359 464 m 359 449 l 259 449 l 259 464 l clp gs col-1 s gr % Polyline n 459 449 m 459 464 l 359 464 l 359 449 l clp gs col-1 s gr % Polyline n 259 479 m 259 494 l 159 494 l 159 479 l clp gs col-1 s gr % Polyline n 459 479 m 459 494 l 359 494 l 359 479 l clp gs col-1 s gr % Polyline n 259 509 m 259 524 l 159 524 l 159 509 l clp gs col-1 s gr % Polyline n 359 509 m 359 524 l 259 524 l 259 509 l clp gs col-1 s gr % Polyline n 739 564 m 759 564 l gs col-1 s gr % Polyline n 739 579 m 759 579 l gs col-1 s gr % Polyline n 419 624 m 419 609 l 359 609 l 359 624 l clp gs col-1 s gr % Polyline n 359 624 m 359 609 l 339 609 l 339 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 339 624 m 339 609 l 279 609 l 279 624 l clp gs col-1 s gr % Polyline n 279 624 m 279 609 l 259 609 l 259 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 624 m 259 609 l 199 609 l 199 624 l clp gs col-1 s gr % Polyline n 199 624 m 199 609 l 179 609 l 179 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 464 m 239 464 l 239 449 l 259 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 464 m 219 464 l 219 449 l 239 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 219 464 m 199 464 l 199 449 l 219 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 464 m 179 464 l 179 449 l 199 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 464 m 159 464 l 159 449 l 179 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 59 579 m 39 579 l 39 564 l 59 564 l clp gs 0.90 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 139 524 m gs 1 -1 scale (process 3 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr % Polyline n 79 579 m 59 579 l 59 564 l 79 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 99 579 m 79 579 l 79 564 l 99 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 119 579 m 99 579 l 99 564 l 119 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 139 579 m 119 579 l 119 564 l 139 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 359 579 m 339 579 l 339 564 l 359 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 379 579 m 359 579 l 359 564 l 379 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 399 579 m 379 579 l 379 564 l 399 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 419 579 m 399 579 l 399 564 l 419 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 439 579 m 419 579 l 419 564 l 439 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 659 579 m 639 579 l 639 564 l 659 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 679 579 m 659 579 l 659 564 l 679 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 699 579 m 679 579 l 679 564 l 699 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 719 579 m 699 579 l 699 564 l 719 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 739 579 m 719 579 l 719 564 l 739 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 494 m 279 494 l 279 479 l 299 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 279 494 m 259 494 l 259 479 l 279 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 319 494 m 299 494 l 299 479 l 319 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 339 494 m 319 494 l 319 479 l 339 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 359 494 m 339 494 l 339 479 l 359 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 524 m 359 524 l 359 509 l 379 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 399 524 m 379 524 l 379 509 l 399 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 419 524 m 399 524 l 399 509 l 419 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 439 524 m 419 524 l 419 509 l 439 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 459 524 m 439 524 l 439 509 l 459 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 159 579 m 139 579 l 139 564 l 159 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 179 579 m 159 579 l 159 564 l 179 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 199 579 m 179 579 l 179 564 l 199 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 219 579 m 199 579 l 199 564 l 219 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 579 m 219 579 l 219 564 l 239 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 579 m 239 579 l 239 564 l 259 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 279 579 m 259 579 l 259 564 l 279 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 299 579 m 279 579 l 279 564 l 299 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 319 579 m 299 579 l 299 564 l 319 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 339 579 m 319 579 l 319 564 l 339 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 459 579 m 439 579 l 439 564 l 459 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 479 579 m 459 579 l 459 564 l 479 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 499 579 m 479 579 l 479 564 l 499 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 519 579 m 499 579 l 499 564 l 519 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 539 579 m 519 579 l 519 564 l 539 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 559 579 m 539 579 l 539 564 l 559 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 599 579 m 579 579 l 579 564 l 599 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 579 579 m 559 579 l 559 564 l 579 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 619 579 m 599 579 l 599 564 l 619 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 639 579 m 619 579 l 619 564 l 639 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Interpolated spline n 239 104 m 250.668 101.971 255.668 100.721 259 99 curveto 268.933 93.869 284.450 74.407 294 69 curveto 300.551 65.291 313.415 54.616 324 59 curveto 326.807 60.163 328.057 62.663 329 69 curveto gs col-1 s gr n 329.801 60.793 m 329.000 69.000 l 325.844 61.382 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 149 m 250.644 148.244 255.644 146.994 259 144 curveto 277.543 127.455 267.799 80.162 289 64 curveto 302.704 53.553 328.391 50.305 344 59 curveto 346.370 60.320 347.620 62.820 349 69 curveto gs col-1 s gr n 349.208 60.756 m 349.000 69.000 l 345.305 61.628 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 194 m 250.648 192.789 255.648 191.539 259 189 curveto 267.738 182.380 275.372 162.611 279 154 curveto 288.827 130.673 274.667 70.350 309 49 curveto 326.576 38.071 351.215 49.412 364 59 curveto 365.918 60.438 367.168 62.938 369 69 curveto gs col-1 s gr n 368.600 60.763 m 369.000 69.000 l 364.771 61.921 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 239 m 250.644 238.228 255.644 236.978 259 234 curveto 283.386 212.355 290.390 153.120 299 129 curveto 306.270 108.635 296.232 54.835 329 39 curveto 349.214 29.232 372.340 48.608 384 59 curveto 385.671 60.489 386.921 62.989 389 69 curveto gs col-1 s gr n 388.275 60.786 m 389.000 69.000 l 384.495 62.093 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 119 m 250.654 121.454 255.654 122.704 259 124 curveto 274.976 130.186 310.971 145.570 324 159 curveto 325.480 160.525 326.730 163.025 329 169 curveto gs col-1 s gr n 328.028 160.811 m 329.000 169.000 l 324.289 162.232 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 164 m 250.581 164.100 255.581 164.100 259 164 curveto 278.409 163.430 320.511 145.420 344 159 curveto 346.313 160.337 347.563 162.837 349 169 curveto gs col-1 s gr n 349.131 160.755 m 349.000 169.000 l 345.236 161.663 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 209 m 250.636 206.383 255.636 205.133 259 204 curveto 283.853 195.628 323.491 144.162 364 159 curveto 366.973 160.089 368.223 162.589 369 169 curveto gs col-1 s gr n 370.023 160.817 m 369.000 169.000 l 366.052 161.299 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 254 m 250.666 252.137 255.666 250.887 259 249 curveto 268.147 243.821 283.511 227.619 289 219 curveto 296.572 207.110 294.164 175.146 309 164 curveto 328.055 149.685 363.140 147.049 384 159 curveto 386.327 160.333 387.577 162.833 389 169 curveto gs col-1 s gr n 389.150 160.755 m 389.000 169.000 l 385.253 161.655 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 134 m 250.647 135.150 255.647 136.400 259 139 curveto 288.594 161.953 309.637 231.415 324 259 curveto 324.882 260.695 326.132 263.195 329 269 curveto gs col-1 s gr n 327.250 260.942 m 329.000 269.000 l 323.663 262.714 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 179 m 250.656 180.494 255.656 181.744 259 184 curveto 273.708 193.922 285.963 228.455 299 239 curveto 308.440 246.636 334.739 250.151 344 259 curveto 345.577 260.507 346.827 263.007 349 269 curveto gs col-1 s gr n 348.153 260.797 m 349.000 269.000 l 344.393 262.161 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 224 m 250.662 226.332 255.662 227.582 259 229 curveto 269.183 233.327 288.080 250.409 299 254 curveto 314.092 258.962 348.484 251.864 364 259 curveto 366.524 260.161 369.024 262.661 374 269 curveto gs col-1 s gr n 370.633 261.472 m 374.000 269.000 l 367.487 263.942 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 269 m 250.469 265.757 255.469 264.507 259 264 curveto 287.568 259.902 349.983 238.905 384 259 curveto 386.280 260.347 387.530 262.847 389 269 curveto gs col-1 s gr n 389.086 260.754 m 389.000 269.000 l 385.196 261.684 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 12.00 scalefont setfont 39 29 m gs 1 -1 scale (logical view: partition file in row cyclic pattern and transpose) col-1 show gr /Helvetica findfont 12.00 scalefont setfont 39 399 m gs 1 -1 scale (implementation using etype, filetypes, and buftypes) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 119 m gs 1 -1 scale (file structure) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 84 m gs 1 -1 scale (process 1 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 184 m gs 1 -1 scale (process 2 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 284 m gs 1 -1 scale (process 3 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 39 554 m gs 1 -1 scale (actual layout in the file:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 434 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 464 m gs 1 -1 scale (process 1 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 494 m gs 1 -1 scale (process 2 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 420 2535 a(Figure)h(10.3:)i(T)l(ransp)q(osing)e(and)f (partitioning)h(a)f(2-D)g(matrix)166 2641 y(Note)d(that)h(using)g Fq(MPI)f Ft(deriv)o(ed)i(datat)o(yp)q(es)e(leads)i(to)e(the)h(p)q (ossibilit)o(y)i(of)d(v)o(ery)h(\015exible)i(patterns.)75 2698 y(F)l(or)d(example,)i(the)f(\014let)o(yp)q(es)h(need)g(not)e (distribute)i(the)f(data)f(in)i(rank)f(order.)18 b(In)c(addition,)g (there)f(can)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Ft(4)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(b)q(e)17 b(o)o(v)o(erlaps)g(b)q(et)o(w)o(een)f(the)h (data)f(items)h(that)f(are)g(accessed)h(b)o(y)g(di\013eren)o(t)g(pro)q (cesses.)24 b(The)17 b(extreme)75 106 y(case)e(of)g(full)i(o)o(v)o (erlap)d(is)i(the)f(broadcast/reduce)h(pattern.)166 162 y(Using)c(the)f(\014let)o(yp)q(e)i(allo)o(ws)e(a)g(certain)h(access)g (pattern)f(to)f(b)q(e)i(established.)21 b(But)11 b(it)h(is)g(conceiv)m (able)75 219 y(that)j(a)h(single)i(pattern)d(w)o(ould)i(not)e(b)q(e)i (suitable)h(for)d(the)h(whole)h(\014le.)24 b(The)16 b(solution)h(is)f (to)g(de\014ne)h(an)75 275 y(absolute)d(b)o(yte)f(displacemen)o(t)i (from)e(the)g(b)q(eginning)j(of)d(the)h(\014le,)g(and)g(ha)o(v)o(e)f (the)h(access)f(pattern)g(start)75 332 y(from)f(that)f(displacemen)o (t.)21 b(Th)o(us)12 b(if)h(a)f(\014le)i(has)e(t)o(w)o(o)f(segmen)o(ts)h (that)f(need)j(to)d(b)q(e)j(accessed)f(in)g(di\013eren)o(t)75 388 y(patterns,)e(the)f(displacemen)o(t)j(for)d(the)g(second)h(pattern) g(will)h(skip)f(o)o(v)o(er)f(the)h(whole)g(\014rst)f(segmen)o(t.)18 b(This)75 444 y(mec)o(hanism)c(is)g(also)f(particularly)i(useful)f(for) f(handling)i(\014les)f(with)g(some)f(header)h(information)f(at)g(the)75 501 y(b)q(eginning)20 b(\(see)d(Figure)h(10.4\).)25 b(Use)17 b(of)g(\014le)i(headers)f(could)g(allo)o(w)g(the)f(supp)q(ort)h(of)f (heterogeneous)75 557 y(en)o(vironmen)o(ts)k(b)o(y)g(storing)g(a)f (\\standard")g(co)q(di\014cation)j(of)d(the)h(data)g(represen)o (tations)f(and)i(data)75 614 y(t)o(yp)q(es)15 b(of)g(the)g(\014le)h (data.)375 970 y @beginspecial 0 @llx 0 @lly 502 @urx 126 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-disp.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -36.0 189.0 translate 0.900 -0.900 scale % Ellipse n 574 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 594 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 1.000 setlinewidth % Polyline n 579 159 m 39 159 l 39 174 l 579 174 l gs col-1 s gr 0.500 setlinewidth % Polyline n 119 209 m 119 179 l gs col-1 s gr n 117.000 187.000 m 119.000 179.000 l 121.000 187.000 l gs 2 setlinejoin col-1 s gr % Polyline n 319 209 m 319 179 l gs col-1 s gr n 317.000 187.000 m 319.000 179.000 l 321.000 187.000 l gs 2 setlinejoin col-1 s gr % Polyline n 159 84 m 139 84 l 139 69 l 159 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 84 m 179 84 l 179 69 l 199 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 84 m 219 84 l 219 69 l 239 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 279 84 m 259 84 l 259 69 l 279 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 84 m 299 84 l 299 69 l 319 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 114 m 139 114 l 139 99 l 199 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 114 m 259 114 l 259 99 l 319 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 139 174 m 119 174 l 119 159 l 139 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 174 m 159 174 l 159 159 l 179 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 219 174 m 199 174 l 199 159 l 219 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 259 174 m 239 174 l 239 159 l 259 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 174 m 279 174 l 279 159 l 299 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 379 174 m 319 174 l 319 159 l 379 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 499 174 m 439 174 l 439 159 l 499 159 l clp gs 0.90 setgray fill gr gs col-1 s gr 0.000 setlinewidth % Ellipse n 584 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 179 84 m 159 84 l 159 69 l 179 69 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 324 204 m gs 1 -1 scale (second displacement) col-1 show gr % Polyline n 219 84 m 199 84 l 199 69 l 219 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 84 m 239 84 l 239 69 l 259 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 299 84 m 279 84 l 279 69 l 299 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 339 84 m 319 84 l 319 69 l 339 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 114 m 199 114 l 199 99 l 259 99 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 114 m 319 114 l 319 99 l 379 99 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 159 174 m 139 174 l 139 159 l 159 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 199 174 m 179 174 l 179 159 l 199 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 174 m 219 174 l 219 159 l 239 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 279 174 m 259 174 l 259 159 l 279 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 319 174 m 299 174 l 299 159 l 319 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 439 174 m 379 174 l 379 159 l 439 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 559 174 m 499 174 l 499 159 l 559 159 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (first tiling) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 109 m gs 1 -1 scale (second tiling) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 39 149 m gs 1 -1 scale (file structure:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 79 169 m gs 1 -1 scale (header) dup stringwidth pop 2 div neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 124 204 m gs 1 -1 scale (first displacement) col-1 show gr $F2psEnd %%EndDocument @endspecial 704 1068 a(Figure)g(10.4:)j(Displacemen)o(ts)75 1233 y Fl(10.1.2)49 b(Data)17 b(Access)e(F)o(unctions)75 1319 y Ft(Data)f(is)i(mo)o(v)o(ed)f(b)q(et)o(w)o(een)g(\014les)h(and)g (pro)q(cesses)f(b)o(y)h(issuing)g(read)f(and)h(write)f(calls.)21 b(There)16 b(are)f(three)75 1375 y(orthogonal)20 b(asp)q(ects)i(to)e (data)h(access:)32 b(p)q(ositioning)23 b(\(explicit)g(o\013set)d(vs.)38 b(implicit)24 b(\014le)e(p)q(oin)o(ter\),)75 1432 y(sync)o(hronism)14 b(\(blo)q(c)o(king)h(vs.)k(non)o(blo)q(c)o(king\),)c(and)e(co)q (ordination)i(\(indep)q(enden)o(t)h(vs.)j(collectiv)o(e\).)i(All)75 1488 y(com)o(binations)c(of)f(these)h(data)f(access)g(routines,)h (including)j(t)o(w)o(o)15 b(t)o(yp)q(es)h(of)g(\014le)i(p)q(oin)o (ters,)f(individual)75 1545 y(and)e(shared,)g(are)g(pro)o(vided.)p 190 1597 1570 2 v 189 1653 2 57 v 215 1636 a Fo(p)q(ositioning)p 491 1653 V 500 1653 V 61 w(sync)o(hronism)p 850 1653 V 859 1653 V 356 w(co)q(ordination)p 1759 1653 V 859 1655 901 2 v 189 1710 2 57 v 491 1710 V 500 1710 V 850 1710 V 859 1710 V 884 1693 a Fk(indep)n(endent)p 1276 1710 V 181 w(c)n(ol)r(le)n(ctive)p 1759 1710 V 190 1711 1570 2 v 190 1720 V 189 1776 2 57 v 215 1759 a(explicit)p 491 1776 V 500 1776 V 169 w(blo)n(cking)p 850 1776 V 859 1776 V 202 w Ft(MPI)p 976 1759 14 2 v 16 w(Read)p 1276 1776 2 57 v 212 w(MPI)p 1394 1759 14 2 v 17 w(Read)p 1512 1759 V 17 w(all)p 1759 1776 2 57 v 189 1833 V 215 1816 a Fk(o\013sets)p 491 1833 V 500 1833 V 185 w Ft(\(sync)o (hronous\))p 850 1833 V 859 1833 V 80 w(MPI)p 976 1816 14 2 v 16 w(W)l(rite)p 1276 1833 2 57 v 202 w(MPI)p 1394 1816 14 2 v 17 w(W)l(rite)p 1522 1816 V 16 w(all)p 1759 1833 2 57 v 500 1834 1261 2 v 189 1889 2 57 v 491 1889 V 500 1889 V 525 1872 a Fk(nonblo)n(cking)p 850 1889 V 859 1889 V 127 w Ft(MPI)p 976 1872 14 2 v 16 w(Iread)p 1276 1889 2 57 v 211 w(MPI)p 1394 1872 14 2 v 17 w(Iread)p 1513 1872 V 16 w(all)p 1759 1889 2 57 v 189 1945 V 491 1945 V 500 1945 V 525 1928 a(\(async)o(hronous\))p 850 1945 V 859 1945 V 57 w(MPI)p 976 1928 14 2 v 16 w(Iwrite)p 1276 1945 2 57 v 196 w(MPI)p 1394 1928 14 2 v 17 w(Iwrite)p 1528 1928 V 16 w(all)p 1759 1945 2 57 v 190 1947 1570 2 v 189 2004 2 57 v 215 1987 a Fk(individual)p 491 2004 V 500 2004 V 116 w(blo)n(cking)p 850 2004 V 859 2004 V 202 w Ft(MPI)p 976 1987 14 2 v 16 w(Read)p 1093 1987 V 17 w(next)p 1276 2004 2 57 v 108 w(MPI)p 1394 1987 14 2 v 17 w(Read)p 1512 1987 V 17 w(next)p 1616 1987 V 16 w(all)p 1759 2004 2 57 v 189 2060 V 215 2043 a Fk(\014le)g(p)n (ointers)p 491 2060 V 500 2060 V 78 w Ft(\(sync)o(hronous\))p 850 2060 V 859 2060 V 80 w(MPI)p 976 2043 14 2 v 16 w(W)l(rite)p 1103 2043 V 17 w(next)p 1276 2060 2 57 v 98 w(MPI)p 1394 2043 14 2 v 17 w(W)l(rite)p 1522 2043 V 16 w(next)p 1625 2043 V 17 w(all)p 1759 2060 2 57 v 500 2062 1261 2 v 189 2116 2 57 v 491 2116 V 500 2116 V 525 2100 a Fk(nonblo)n(cking)p 850 2116 V 859 2116 V 127 w Ft(MPI)p 976 2100 14 2 v 16 w(Iread)p 1094 2100 V 17 w(next)p 1276 2116 2 57 v 107 w(MPI)p 1394 2100 14 2 v 17 w(Iread)p 1513 2100 V 16 w(next)p 1616 2100 V 17 w(all)p 1759 2116 2 57 v 189 2173 V 491 2173 V 500 2173 V 525 2156 a(\(async)o(hronous\))p 850 2173 V 859 2173 V 57 w(MPI)p 976 2156 14 2 v 16 w(Iwrite)p 1109 2156 V 17 w(next)p 1276 2173 2 57 v 92 w(MPI)p 1394 2156 14 2 v 17 w(Iwrite)p 1528 2156 V 16 w(next)p 1631 2156 V 17 w(all)p 1759 2173 2 57 v 190 2175 1570 2 v 189 2231 2 57 v 215 2214 a Fk(shar)n(e)n(d)p 491 2231 V 500 2231 V 186 w(blo)n(cking)p 850 2231 V 859 2231 V 202 w Ft(MPI)p 976 2214 14 2 v 16 w(Read)p 1093 2214 V 17 w(shared)p 1276 2231 2 57 v 66 w(MPI)p 1394 2214 14 2 v 17 w(Read)p 1512 2214 V 17 w(shared)p 1658 2214 V 16 w(all)p 1759 2231 2 57 v 189 2287 V 215 2271 a Fk(\014le)g(p)n (ointer)p 491 2287 V 500 2287 V 97 w Ft(\(sync)o(hronous\))p 850 2287 V 859 2287 V 80 w(MPI)p 976 2271 14 2 v 16 w(W)l(rite)p 1103 2271 V 17 w(shared)p 1276 2287 2 57 v 56 w(MPI)p 1394 2271 14 2 v 17 w(W)l(rite)p 1522 2271 V 16 w(shared)p 1667 2271 V 17 w(all)p 1759 2287 2 57 v 500 2289 1261 2 v 189 2344 2 57 v 491 2344 V 500 2344 V 525 2327 a Fk(nonblo)n(cking)p 850 2344 V 859 2344 V 127 w Ft(MPI)p 976 2327 14 2 v 16 w(Iread)p 1094 2327 V 17 w(shared)p 1276 2344 2 57 v 65 w(MPI)p 1394 2327 14 2 v 17 w(Iread)p 1513 2327 V 16 w(shared)p 1658 2327 V 17 w(all)p 1759 2344 2 57 v 189 2400 V 491 2400 V 500 2400 V 525 2383 a(\(async)o(hronous\))p 850 2400 V 859 2400 V 57 w(MPI)p 976 2383 14 2 v 16 w(Iwrite)p 1109 2383 V 17 w(shared)p 1276 2400 2 57 v 50 w(MPI)p 1394 2383 14 2 v 17 w(Iwrite)p 1528 2383 V 16 w(shared)p 1673 2383 V 17 w(all)p 1759 2400 2 57 v 190 2402 1570 2 v 166 2478 a(UNIX)h(read\(\))e(and)i (write\(\))e(are)h(blo)q(c)o(king,)h(indep)q(enden)o(t)i(op)q (erations,)d(and)g(use)h(individual)i(\014le)75 2534 y(p)q(oin)o(ters.)i(The)c Fq(MPI)e Ft(equiv)m(alen)o(ts)j(are)e Fq(MPI)p 852 2534 14 2 v 16 w(READ)p 987 2534 V 17 w(NEXT)h Ft(and)f Fq(MPI)p 1310 2534 V 16 w(WRITE)p 1469 2534 V 16 w(NEXT)p Ft(.)166 2591 y(Op)q(erations)f(with)g(explicit)h (o\013sets)d(are)h(describ)q(ed)i(in)f(Section)h(10.3.)j(Op)q(erations) c(with)f(individ-)75 2647 y(ual)i(\014le)h(p)q(oin)o(ters)g(are)e (describ)q(ed)j(in)f(Section)g(10.4.)i(And)e(op)q(erations)f(with)g (shared)g(\014le)h(p)q(oin)o(ters)f(are)75 2704 y(describ)q(ed)i(in)f (Section)g(10.5.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(5)75 49 y Fl(10.1.3)49 b(P)o(ositioning)75 135 y Ft(UNIX)19 b(\014le)g(systems)f(traditionally)i(main)o(tain)f(a)f (system)g(\014le)h(p)q(oin)o(ter)g(sp)q(ecifying)h(what)e(o\013set)f (will)75 192 y(b)q(e)k(used)f(for)f(the)h(read)g(or)f(write)h(op)q (eration.)34 b(The)20 b(problem)h(with)f(this)g(in)o(terface)g(is)h (that)e(it)h(w)o(as)75 248 y(primarily)f(designed)g(for)e(\014les)h(b)q (eing)h(accessed)f(b)o(y)f(a)h(single)h(pro)q(cess.)27 b(In)18 b(a)f(parallel)i(en)o(vironmen)o(t,)75 304 y(w)o(e)14 b(m)o(ust)f(decide)j(whether)e(a)f(\014le)i(p)q(oin)o(ter)g(is)f (shared)g(b)o(y)g(m)o(ultiple)i(pro)q(cesses)e(or)g(if)g(an)g (individual)j(\014le)75 361 y(p)q(oin)o(ter)d(will)i(b)q(e)f(main)o (tained)g(b)o(y)f(eac)o(h)g(pro)q(cess.)19 b(In)c(addition,)g(parallel) h(programs)c(do)i(not)g(generally)75 417 y(exhibit)19 b(lo)q(calit)o(y)f(of)f(reference)h(within)h(a)e(\014le)h([6)o(].)26 b(Instead,)18 b(they)f(tend)h(to)f(mo)o(v)o(e)f(b)q(et)o(w)o(een)i (distinct)75 474 y(non-con)o(tiguous)h(regions)f(of)g(a)g(\014le.)29 b(This)19 b(means)f(that)g(the)g(pro)q(cess)g(m)o(ust)g(seek)g(on)g (almost)g(ev)o(ery)75 530 y(read)12 b(or)g(write)g(op)q(eration.)19 b(Moreo)o(v)o(er,)11 b(in)i(m)o(ulti-threaded)h(en)o(vironmen)o(ts)e (or)g(when)h(p)q(erforming)g(I/O)75 587 y(async)o(hronously)l(,)k(it)g (is)g(di\016cult)i(to)d(ensure)h(that)f(the)h(\014le)g(p)q(oin)o(ter)h (will)g(b)q(e)f(in)h(the)f(correct)f(p)q(osition)75 643 y(when)g(the)f(read)g(or)g(write)g(o)q(ccurs.)166 700 y Fq(MPI)10 b Ft(pro)o(vides)i(separate)e(routines)i(for)e(p)q (ositioning)j(with)e(explicit)j(o\013sets,)c(individual)k(\014le)e(p)q (oin)o(t-)75 756 y(ers,)18 b(and)g(shared)g(\014le)g(p)q(oin)o(ters.)28 b(The)18 b(explicit)i(o\013set)d(op)q(erations)h(require)g(the)g(user)g (to)f(sp)q(ecify)i(an)75 813 y(o\013set,)e(and)h(act)f(as)h(atomic)f (seek-and-read)i(or)e(seek-and-write)i(op)q(erations.)28 b(The)18 b(individual)j(and)75 869 y(shared)d(\014le)h(p)q(oin)o(ter)g (op)q(erations)f(use)g(distinct)i(system)d(main)o(tained)i(o\013sets)e (for)h(p)q(ositioning.)30 b(The)75 925 y(di\013eren)o(t)21 b(p)q(ositioning)i(metho)q(ds)f(are)f(orthogonal;)i(they)e(ma)o(y)f(b)q (e)i(mixed)g(within)h(the)e(same)g(pro-)75 982 y(gram,)15 b(and)i(they)f(do)g(not)g(a\013ect)f(eac)o(h)h(other.)23 b(In)17 b(other)e(w)o(ords,)h(an)g(individual)j(\014le)e(p)q(oin)o (ter's)g(v)m(alue)75 1038 y(will)h(b)q(e)e(unc)o(hanged)h(b)o(y)f (executing)h(explicit)h(o\013set)d(op)q(erations)h(or)g(shared)g (\014le)h(p)q(oin)o(ter)f(op)q(erations.)75 1095 y(The)g(data)g(access) g(routines)g(whic)o(h)h(accept)g(explicit)h(o\013sets)d(ha)o(v)o(e)h (no)g(extensions)g(\(e.g.)22 b Fq(MPI)p 1741 1095 14 2 v 16 w(XXX)p Ft(\),)75 1151 y(the)16 b(individual)k(\014le)d(p)q(oin) o(ter)f(routines)h(ha)o(v)o(e)p 884 1151 V 32 w Fq(NEXT)g Ft(app)q(ended)h(\(e.g.)j Fq(MPI)p 1435 1151 V 16 w(XXX)p 1541 1151 V 18 w(NEXT)p Ft(\),)16 b(and)g(the)75 1208 y(shared)i(\014le)i(p)q(oin)o(ter)f(routines)f(ha)o(v)o(e)p 747 1208 V 35 w Fq(SHARED)h Ft(app)q(ended)h(\(e.g.)29 b Fq(MPI)p 1366 1208 V 15 w(XXX)p 1471 1208 V 18 w(SHARED)p Ft(\).)18 b(In)h(order)75 1264 y(to)d(allo)o(w)g(the)g(implicit)j (o\013sets)c(to)h(b)q(e)h(set,)f(t)o(w)o(o)f(seek)h(routines)h(are)f (also)g(pro)o(vided)h(\()p Fq(MPI)p 1662 1264 V 15 w(SEEK)g Ft(and)75 1321 y Fq(MPI)p 160 1321 V 16 w(SEEK)p 287 1321 V 16 w(SHARED)p Ft(\).)166 1377 y(In)c(general,)h(\014le)g(p)q (oin)o(ter)f(op)q(erations)g(ha)o(v)o(e)g(the)g(same)g(seman)o(tics)g (as)f(explicit)j(o\013set)d(op)q(erations,)75 1434 y(with)k(the)f Fq(o\013set)g Ft(argumen)o(t)g(set)g(to)f(the)h(curren)o(t)h(v)m(alue)g (of)f(the)g(system-main)o(tained)h(\014le)g(p)q(oin)o(ter.)75 1554 y Fq(Explicit)g(O\013sets)75 1639 y Ft(There)h(are)g(t)o(w)o(o)f (\\k)o(eys")g(to)h(describ)q(e)h(lo)q(cations)g(in)g(a)f(\014le:)25 b(an)17 b Fq(MPI)f Ft(datat)o(yp)q(e)h(and)g(an)g(o\013set.)25 b Fq(MPI)75 1696 y Ft(datat)o(yp)q(es)14 b(are)h(used)h(as)f (templates,)g(tiling)h(the)f(\014le,)h(and)f(an)g(o\013set)g (determines)h(an)f(initial)i(p)q(osition)75 1752 y(for)g(transfers.)26 b(O\013sets)18 b(are)f(expressed)h(as)g(an)f(in)o(tegral)h(n)o(um)o(b)q (er)g(of)f(elemen)o(tary)h(datat)o(yp)q(e)f(\()p Fq(et)o(yp)q(e)p Ft(\))75 1809 y(items)d(relativ)o(e)h(to)e(the)i Fq(\014let)o(yp)q(e)p Ft(.)21 b(Holes)14 b(in)h(the)f Fq(\014let)o(yp)q(e)i Ft(are)e(ignored)g(when)h(calculating)h(the)e(p)q(osition)75 1865 y(of)f(an)h(o\013set)e(\(e.g.)19 b(an)13 b(o\013set)g(of)g(2)g (for)g(pro)q(cess)h(1)f(in)h(Figure)g(10.2)e(denotes)i(the)g(7th)f Fq(et)o(yp)q(e)i Ft(in)f(the)g(\014le\).)75 1922 y(The)j Fq(et)o(yp)q(e)i Ft(argumen)o(t)d(is)i(asso)q(ciated)g(with)f(a)g (\014le)h(and)g(used)f(to)g(express)h(the)f Fq(\014let)o(yp)q(e)p Ft(,)i Fq(buft)o(yp)q(e)g Ft(and)75 1978 y Fq(o\013set)h Ft(argumen)o(ts.)31 b(Therefore,)20 b(the)g Fq(\014let)o(yp)q(e)h Ft(and)e Fq(buft)o(yp)q(e)i Ft(datat)o(yp)q(es)e(m)o(ust)g(b)q(e)h (directly)h(deriv)o(ed)75 2035 y(from)14 b Fq(et)o(yp)q(e)p Ft(,)i(or)f(their)h(t)o(yp)q(e)f(signatures)g(m)o(ust)g(b)q(e)h(a)f(m)o (ultiple)i(of)e(the)g Fq(et)o(yp)q(e)h Ft(signature.)75 2155 y Fq(File)f(P)o(ointers)75 2240 y Ft(When)h(a)f(\014le)h(is)g(op)q (ened,)h(the)e(system)g(creates)g(a)h(set)f(of)g(\014le)h(p)q(oin)o (ters)g(to)f(k)o(eep)h(trac)o(k)e(of)i(the)f(curren)o(t)75 2297 y(\014le)j(p)q(osition.)25 b(One)18 b(is)f(a)g(global)g(\014le)h (p)q(oin)o(ter,)f Fk(shar)n(e)n(d)f Ft(b)o(y)h(all)h(the)f(pro)q (cesses)g(in)g(the)g(comm)o(unicator)75 2353 y(group.)29 b(The)19 b(others)f(are)g(individual)k(\014le)e(p)q(oin)o(ters)f(lo)q (cal)g(to)f(eac)o(h)h(pro)q(cess)f(in)i(the)e(comm)o(unicator)75 2410 y(group,)c(and)i(can)f(b)q(e)h(up)q(dated)g Fk(indep)n(endently.)i Ft(A)d(shared)h(\014le)g(p)q(oin)o(ter)f(only)h(mak)o(es)f(sense)g(if)h (all)g(the)75 2466 y(pro)q(cesses)j(can)f(access)h(the)f(same)g (dataset.)29 b(This)19 b(means)f(that)g(all)h(the)g(pro)q(cesses)g (should)g(use)g(the)75 2523 y(same)c Fq(\014let)o(yp)q(e)i Ft(when)e(op)q(ening)i(the)e(\014le.)166 2579 y(The)f(main)g(seman)o (tic)h(issue)f(with)h(system-main)o(tained)f(\014le)h(p)q(oin)o(ters)g (is)f(ho)o(w)f(they)i(are)e(up)q(dated)75 2636 y(b)o(y)18 b(I/O)h(op)q(erations.)29 b(In)19 b(general,)g(eac)o(h)g(I/O)f(op)q (eration)h(lea)o(v)o(es)f(the)h(\014le)g(p)q(oin)o(ter)g(p)q(oin)o (ting)g(to)f(the)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Ft(6)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(next)15 b(data)e(item)i(after)f(the)h(last)f(one)h (that)f(w)o(as)f(accessed.)21 b(This)15 b(principle)i(applies)f(to)e(b) q(oth)h(t)o(yp)q(es)f(of)75 106 y(\014le)i(p)q(oin)o(ters)g (\(individual)i(and)d(shared\),)g(and)g(to)g(all)h(t)o(yp)q(es)f(of)g (I/O)h(op)q(erations.)166 162 y(It)f(is)h(p)q(ossible)h(to)d(formalize) i(the)f(up)q(date)h(pro)q(cedure)g(as)f(follo)o(ws:)189 277 y Fi(new)p 274 277 14 2 v 17 w(f)5 b(il)q(e)p 369 277 V 16 w(position)13 b Ft(=)g Fi(ol)q(d)p 670 277 V 15 w(position)e Ft(+)906 247 y Fi(siz)r(e)p Ft(\()p Fi(buf)5 b(ty)r(pe)p Ft(\))10 b Fh(\002)g Fi(buf)5 b(count)p 906 267 514 2 v 1052 309 a(siz)r(e)p Ft(\()p Fi(ety)r(pe)p Ft(\))75 393 y(where)24 b Fi(buf)5 b(count)23 b Ft(is)h(the)f(n)o(um)o (b)q(er)h(of)f(elemen)o(ts)h(of)f(t)o(yp)q(e)g Fi(buf)5 b(ty)r(pe)24 b Ft(to)e(b)q(e)i(accessed)g(and)g(where)75 449 y Fi(siz)r(e)p Ft(\()p Fi(dataty)r(pe)p Ft(\))19 b(giv)o(es)f(the)h(n)o(um)o(b)q(er)g(of)f(b)o(ytes)h(of)f(actual)h (data)f(\(excluding)i(holes\))f(that)f(comp)q(oses)75 506 y(the)d Fq(MPI)g Ft(datat)o(yp)q(e)g Fi(dataty)r(pe)p Ft(.)166 562 y(Another)j(complication)j(with)d(UNIX)i(I/O)f(op)q (erations,)g(is)g(that)f(the)h(system-main)o(tained)g(\014le)75 619 y(p)q(oin)o(ter)c(is)g(normally)f(only)h(up)q(dated)g(when)g(the)g (op)q(eration)f(completes.)20 b(A)o(t)14 b(that)g(stage,)f(it)h(is)h (kno)o(wn)75 675 y(exactly)h(ho)o(w)f(m)o(uc)o(h)h(data)f(w)o(as)g (actually)h(accessed)h(\(whic)o(h)f(can)g(b)q(e)g(di\013eren)o(t)g (from)f(the)h(amoun)o(t)f(re-)75 732 y(quested\),)i(and)g(the)g(\014le) h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o(y)f(that)f(amoun)o(t.)24 b(F)l(or)16 b(non)o(blo)q(c)o(king)j(accesses)e(made)75 788 y(using)e(an)g(individual)j(or)c(the)g(shared)h(\014le)h(p)q(oin)o (ter,)e(the)h(up)q(date)g(cannot)g(b)q(e)g(dela)o(y)o(ed)g(un)o(til)h (the)f(op)q(er-)75 845 y(ation)g(completes,)h(b)q(ecause)g(additional)h (accesses)e(can)g(b)q(e)h(initiated)h(b)q(efore)f(that)e(time)i(b)o(y)f (the)g(same)75 901 y(pro)q(cess)i(\(for)e(b)q(oth)i(t)o(yp)q(es)g(of)f (\014le)h(p)q(oin)o(ters\))g(or)f(b)o(y)h(other)f(pro)q(cesses)h(\(for) e(the)i(shared)g(\014le)g(p)q(oin)o(ter\).)75 958 y(Therefore,)c(the)h (\014le)g(p)q(oin)o(ter)g(m)o(ust)f(b)q(e)h(up)q(dated)g(at)f(the)h (outset,)f(b)o(y)g(the)g(amoun)o(t)g(of)g(data)g(requested.)166 1014 y(Similarly)l(,)k(when)e(blo)q(c)o(king)i(accesses)e(are)g(made)g (using)g(the)h(shared)f(\014le)h(p)q(oin)o(ter,)f(up)q(dating)h(the)75 1070 y(\014le)k(p)q(oin)o(ter)f(at)e(the)i(completion)h(of)e(eac)o(h)g (access)h(w)o(ould)g(ha)o(v)o(e)f(the)h(same)f(e\013ect)g(as)g (serializing)j(all)75 1127 y(blo)q(c)o(king)e(accesses)f(to)e(the)i (\014le.)28 b(In)18 b(order)f(to)g(prev)o(en)o(t)g(this,)h(the)g (shared)f(\014le)i(p)q(oin)o(ter)f(for)f(blo)q(c)o(king)75 1183 y(accesses)g(is)g(up)q(dated)g(at)f(the)h(b)q(eginning)i(of)d(eac) o(h)g(access)h(b)o(y)g(the)f(amoun)o(t)g(of)g(data)g(requested.)25 b(F)l(or)75 1240 y(blo)q(c)o(king)13 b(accesses)f(using)h(an)f (individual)j(\014le)d(p)q(oin)o(ter,)h(up)q(dating)g(the)f(\014le)g(p) q(oin)o(ter)h(at)e(the)h(completion)75 1296 y(of)j(eac)o(h)h(access)f (w)o(ould)h(b)q(e)g(p)q(erfectly)h(v)m(alid.)22 b(Ho)o(w)o(ev)o(er,)14 b(in)i(order)f(to)g(main)o(tain)h(the)g(same)f(seman)o(tics)75 1353 y(for)f(all)i(t)o(yp)q(es)f(of)f(accesses)h(using)h(\014le)f(p)q (oin)o(ters,)g(the)g(up)q(date)h(of)e(the)h(\014le)h(p)q(oin)o(ter)f (in)h(this)f(case)g(is)g(also)75 1409 y(made)g(at)g(the)g(b)q(eginning) i(of)e(the)g(access)h(b)o(y)f(the)g(amoun)o(t)g(of)f(data)h(requested.) 166 1466 y(Although)c(consisten)o(t,)g(and)g(seman)o(tically)h (cleaner,)g(up)q(dating)g(the)f(\014le)g(p)q(oin)o(ter)g(at)f(the)h (initiation)75 1522 y(of)20 b(all)g(I/O)h(op)q(erations)f(di\013ers)g (from)f(accepted)i(UNIX)f(practice,)i(and)e(ma)o(y)f(lead)i(to)e(unexp) q(ected)75 1579 y(results.)h(Consider)c(the)f(follo)o(wing)h(scenario:) 266 1685 y Fp(MPI_Read_Next\(fh,)21 b(buff,)j(buftype,)e(bufcount,)h (&status\);)266 1741 y(MPI_Write_Next\(fh,)e(buff,)i(buftype,)g (bufcount,)g(&status\);)166 1848 y Ft(If)f(the)g(\014rst)g(read)g(reac) o(hes)g(the)g(end)h(of)e(the)i(\014le)g(b)q(efore)f(completing,)j(the)d (\014le)h(p)q(oin)o(ter)f(will)75 1904 y(b)q(e)g(incremen)o(ted)h(b)o (y)e(the)h(amoun)o(t)e(of)h(data)g Fk(r)n(e)n(queste)n(d)f Ft(rather)h(than)h(the)f(amoun)o(t)g(of)g(data)f(read.)75 1960 y(Therefore,)c(the)h(\014le)g(p)q(oin)o(ter)g(will)h(p)q(oin)o(t)f (b)q(ey)o(ond)g(the)f(curren)o(t)g(end)h(of)f(the)h(\014le,)g(and)f (the)h(write)f(will)75 2017 y(lea)o(v)o(e)f(a)g(hole)h(in)g(the)f (\014le.)21 b(Ho)o(w)o(ev)o(er,)14 b(suc)o(h)h(a)g(problem)h(only)g(o)q (ccurs)f(if)h(reads)f(and)g(writes)g(are)g(mixed)75 2073 y(without)g(c)o(hec)o(king)i(for)d(the)i(end)g(of)e(the)i(\014le.)21 b(Although)16 b(common)f(in)h(a)f(single)i(pro)q(cess)e(w)o(orkstation) 75 2130 y(en)o(vironmen)o(t,)g(w)o(e)g(b)q(eliev)o(e)i(this)f(is)g (uncommon)f(in)h(a)f(parallel)i(en)o(vironmen)o(t.)75 2252 y Fl(10.1.4)49 b(Synchronism)75 2337 y Ft(The)20 b(explicit)i(o)o(v)o(erlap)d(of)g(computation)h(with)g(I/O,)g(hop)q (efully)h(impro)o(ving)g(p)q(erformance,)f(is)g(sup-)75 2394 y(p)q(orted)13 b(through)f(the)h(use)g(of)g(non)o(blo)q(c)o(king)h (data)e(access)h(routines.)19 b(Both)13 b(blo)q(c)o(king)h(and)f(non)o (blo)q(c)o(king)75 2450 y(v)o(ersions)h(of)g(all)h(data)e(access)h (routines)g(are)g(supp)q(orted.)20 b(The)14 b(non)o(blo)q(c)o(king)i(v) o(ersions)e(of)f(the)i(routines)75 2507 y(are)g(named)g Fq(MPI)p 382 2507 14 2 v 16 w(IXXX)p Ft(,)g(where)h(the)f Fq(I)g Ft(stands)g(for)f(immediate.)166 2563 y(A)j Fk(blo)n(cking)e Ft(I/O)j(call)g(will)g(blo)q(c)o(k)g(un)o(til)g(the)f(I/O)g(request)g (is)h(completed.)26 b(A)17 b Fk(nonblo)n(cking)e Ft(I/O)75 2620 y(call)f(only)f(initiates)h(an)f(I/O)g(op)q(eration,)g(but)g(do)q (es)g(not)f(w)o(ait)h(for)f(it)h(to)f(complete.)20 b(A)12 b(separate)h Fk(r)n(e)n(quest)75 2676 y(c)n(omplete)18 b Ft(call)h(\()p Fq(MPI)p 453 2676 V 15 w(W)l(AIT)p Ft(,)f Fq(MPI)p 694 2676 V 15 w(TEST)p Ft(,)g(or)g(an)o(y)f(of)h(their)g(v)m (arian)o(ts\))f(is)i(needed)g(to)e(complete)i(the)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(7)75 49 y(I/O)17 b(request,)g(i.e.,)h(to)e(certify)h(that)f(data)h (has)g(b)q(een)h(read/written,)f(and)g(it)g(is)g(safe)g(for)f(the)h (user)h(to)75 106 y(reuse)d(the)g(bu\013er.)20 b(With)15 b(suitable)i(hardw)o(are,)d(the)h(transfer)f(of)g(data)h(out/in)g(the)g (user's)g(bu\013er)g(ma)o(y)75 162 y(pro)q(ceed)h(concurren)o(tly)g (with)f(computation.)166 219 y(Note)f(that)f(just)h(b)q(ecause)i(a)e (non)o(blo)q(c)o(king)h(\(or)f(blo)q(c)o(king\))h(data)f(access)g (routine)h(completes)g(do)q(es)75 275 y(not)g(mean)h(that)f(the)h(data) f(is)h(actually)h(written)f(to)f(\\p)q(ermanen)o(t")g(storage.)20 b(All)e(of)d(the)h(data)f(access)75 332 y(routines)22 b(ma)o(y)g(bu\013er)g(data)f(to)h(impro)o(v)o(e)g(p)q(erformance.)40 b(The)23 b(only)f(w)o(a)o(y)f(to)h(guaran)o(tee)f(data)g(is)75 388 y(actually)16 b(written)f(to)g(storage)e(is)j(b)o(y)f(using)h(the)f Fq(MPI)p 1010 388 14 2 v 16 w(FILE)p 1117 388 V 16 w(SYNC)h Ft(call.)21 b(Ho)o(w)o(ev)o(er,)13 b(one)j(need)g(not)e(b)q(e)75 444 y(concerned)i(with)e(the)h(con)o(v)o(erse)f(problem|once)i(a)e (read)h(op)q(eration)f(completes,)h(the)g(data)f(is)h(alw)o(a)o(ys)75 501 y(a)o(v)m(ailable)i(in)f(the)f(user's)g(bu\013er.)75 623 y Fl(10.1.5)49 b(Co)q(o)o(rdination)75 708 y Ft(Global)21 b(data)f(accesses)g(ha)o(v)o(e)g(signi\014can)o(t)h(p)q(oten)o(tial)g (for)f(automatic)g(optimization,)i(pro)o(vided)f(the)75 765 y(I/O)d(system)f(can)h(recognize)h(an)f(op)q(eration)f(as)h(a)f (global)h(access.)28 b(Collectiv)o(e)19 b(op)q(erations)f(are)f(used)75 821 y(for)e(this)i(purp)q(ose.)22 b Fq(MPI)15 b Ft(supp)q(orts)h(b)q (oth)g(indep)q(enden)o(t)j(and)d(collectiv)o(e)h(v)o(ersions)f(of)g (all)h(data)e(access)75 878 y(routines.)39 b(Ev)o(ery)21 b(indep)q(enden)o(t)i(data)e(access)h(routine)f Fq(MPI)p 1184 878 V 16 w(XXX)h Ft(has)g(a)f(collectiv)o(e)i(coun)o(terpart)75 934 y Fq(MPI)p 160 934 V 16 w(XXX)p 266 934 V 17 w(ALL)p Ft(,)17 b(where)p 527 934 V 35 w Fq(ALL)g Ft(means)h(that)f(\\all")i (pro)q(cesses)f(in)h(the)f(comm)o(unicator)f(group)h(whic)o(h)75 991 y(op)q(ened)e(the)g(\014le)g(m)o(ust)e(participate.)166 1047 y(Indep)q(enden)o(t)23 b(calls)f(do)f(not)f(imply)j(an)o(y)d(co)q (ordination)i(among)e(pro)q(cesses.)38 b(Collectiv)o(e)22 b(calls)75 1104 y(imply)13 b(that)e(all)h(pro)q(cesses)g(b)q(elonging)i (to)d(the)g(comm)o(unicator)h(asso)q(ciated)f(with)h(the)g(op)q(ened)h (\014le)g(m)o(ust)75 1160 y(participate.)20 b(Ho)o(w)o(ev)o(er,)12 b(no)h(sync)o(hronization)i(pattern)d(b)q(et)o(w)o(een)i(those)f(pro)q (cesses)g(is)h(enforced)g(\(with)75 1217 y(the)h(exception)i(of)d (collectiv)o(e)j(data)e(accesses)g(using)h(a)f(shared)g(\014le)i(p)q (oin)o(ter|see)f(Section)g(10.5.2\).)166 1273 y(An)h Fk(indep)n(endent)e Ft(I/O)h(request)h(is)g(a)f(request)g(whic)o(h)h (is)g(executed)g(individual)q(ly)j(b)o(y)c(an)o(y)g(of)g(the)75 1329 y(pro)q(cesses)21 b(within)i(a)e(comm)o(unicator)f(group.)38 b(A)21 b Fk(c)n(ol)r(le)n(ctive)e Ft(I/O)j(request)f(is)h(a)e(request)i (whic)o(h)f(is)75 1386 y(executed)15 b(b)o(y)e Fk(al)r(l)h Ft(pro)q(cesses)g(within)g(a)g(comm)o(unicator)f(group.)19 b(The)14 b(completion)h(of)e(an)h(indep)q(enden)o(t)75 1442 y(call)j(only)g(dep)q(ends)h(on)e(the)h(activit)o(y)f(of)g(the)h (calling)h(pro)q(cess.)23 b(On)17 b(the)g(other)f(hand,)g(a)g(pro)q (cess)h(can)75 1499 y(\(but)k(is)g(not)f(required)i(to\))e(return)h (from)f(a)g(collectiv)o(e)j(call)f(as)e(so)q(on)h(as)f(its)h (participation)h(in)g(the)75 1555 y(collectiv)o(e)e(op)q(eration)e(is)g (completed.)30 b(The)18 b(completion)h(of)f(the)g(call,)h(ho)o(w)o(ev)o (er,)f(do)q(es)g(not)f(indicate)75 1612 y(that)e(other)h(pro)q(cesses)h (ha)o(v)o(e)e(completed)i(or)f(ev)o(en)h(started)e(the)h(I/O)h(op)q (eration.)22 b(Th)o(us,)16 b(a)g(collectiv)o(e)75 1668 y(call)g(ma)o(y)l(,)f(or)f(ma)o(y)h(not,)f(ha)o(v)o(e)h(the)g(e\013ect) g(of)g(sync)o(hronizing)i(all)f(calling)h(pro)q(cesses.)166 1725 y(F)l(rom)c(a)g(seman)o(tic)i(viewp)q(oin)o(t,)f(the)g(only)h (di\013erence)g(b)q(et)o(w)o(een)f(collectiv)o(e)h(op)q(erations)f(and) g(their)75 1781 y(indep)q(enden)o(t)25 b(coun)o(terparts)d(is)h(p)q (oten)o(tial)g(sync)o(hronization.)43 b(F)l(rom)21 b(a)i(p)q (erformance)f(viewp)q(oin)o(t,)75 1838 y(ho)o(w)o(ev)o(er,)14 b(collectiv)o(e)j(op)q(erations)e(ha)o(v)o(e)g(the)g(p)q(oten)o(tial)h (to)e(b)q(e)i(m)o(uc)o(h)f(faster)f(than)h(their)h(indep)q(enden)o(t)75 1894 y(coun)o(terparts.)75 2016 y Fl(10.1.6)49 b(Consistency)16 b(Semantics)75 2102 y Ft(Consistency)21 b(seman)o(tics)f(de\014ne)h (the)f(outcome)g(of)g(m)o(ultiple)i(accesses)e(to)g(a)g(single)h (\014le,)h(either)f(b)o(y)75 2158 y(man)o(y)15 b(pro)q(cesses)g(or)g(b) o(y)g(a)g(single)i(pro)q(cess)e(whic)o(h)h(has)f(op)q(ened)h(the)g (\014le)g(more)f(than)g(once.)166 2214 y Fq(MPI)j Ft(guaran)o(tees)g (that)g(non-o)o(v)o(erlapping)i(\014le)g(accesses)f(within)h(a)f (single)h(application)g(are)f(se-)75 2271 y(quen)o(tially)k(consisten)o (t,)f(and)g(hence,)h(app)q(ear)e(to)g(the)g(application)i(as)e(atomic.) 37 b(Ho)o(w)o(ev)o(er,)21 b(unlik)o(e)75 2327 y(UNIX,)e(the)h(default)f (seman)o(tics)g(for)g(o)o(v)o(erlapping)h(accesses)f(do)q(es)g(not)g (guaran)o(tee)f(sequen)o(tial)j(con-)75 2384 y(sistency)l(.)28 b(This)19 b(default)f(mo)q(de)g(is)g(referred)g(to)f(as)g Fg(MPI)p 1066 2384 13 2 v 14 w(RECKLESS)g Ft(mo)q(de.)28 b(A)o(tomic)18 b(access)f(can)h(b)q(e)75 2440 y(guaran)o(teed)h(for)f (o)o(v)o(erlapping)i(accesses)f(b)o(y)g(enabling)i Fg(MPI)p 1140 2440 V 14 w(CA)o(UTIOUS)c Ft(mo)q(de)i(for)f(the)h(\014le)i(via)e (the)75 2497 y Fq(MPI)p 160 2497 14 2 v 16 w(FILE)p 267 2497 V 15 w(CONTROL)e Ft(routine.)166 2553 y(File)g(consistency)g (seman)o(tics)f(outside)g(of)g(a)f(single)j Fq(MPI)d Ft(application)j(are)d(implemen)o(tation)i(de-)75 2610 y(p)q(enden)o(t.)i(No)10 b(\014le)h(consistency)g(seman)o(tics)g(are)f (implied)i(b)o(y)e(either)h Fg(MPI)p 1322 2610 13 2 v 14 w(RECKLESS)f Ft(or)f Fg(MPI)p 1672 2610 V 15 w(CA)o(UTIOUS)75 2666 y Ft(b)q(et)o(w)o(een)17 b(m)o(ultiple)h Fq(MPI)e Ft(applications,)i(or)e(outside)h(of)g(the)f Fq(MPI)g Ft(en)o(vironmen)o(t.)24 b(In)18 b(other)e(w)o(ords,)f(if)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Ft(8)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(t)o(w)o(o)18 b(di\013eren)o(t)h Fq(MPI)f Ft(applications)j(op)q(en)f(the)f(same)g(\014le,)h(the)f(b)q(eha)o (vior)h(and)f(atomicit)o(y)g(of)g(the)g(\014le)75 106 y(accesses)k(are)f(unde\014ned.)43 b(The)23 b Fg(MPI)p 762 106 13 2 v 14 w(CA)o(UTIOUS)d Ft(mo)q(de)j(enforces)f(read/write)g (atomicit)o(y)h(in)g(the)75 162 y Fg(MPI)p 152 162 V 14 w(COMM)p 296 162 V 15 w(W)o(ORLD)15 b Ft(comm)o(unicator)g(group)g (only)l(.)75 284 y Fl(10.1.7)49 b(End)17 b(of)f(File)75 370 y Ft(Unlik)o(e)k(UNIX)e(\014les,)i(the)e(end)h(of)e(\014le)i(is)g (not)f(absolute)g(and)g(iden)o(tical)i(for)e(all)h(pro)q(cesses)f (accessing)75 426 y(the)d(\014le.)21 b(It)15 b(dep)q(ends)i(on)e(the)g (\014let)o(yp)q(e)h(used)g(to)f(access)g(the)g(\014le)h(and)g(is)f (de\014ned)i(for)d(a)h(giv)o(en)h(pro)q(cess)75 483 y(as)h(the)g(lo)q (cation)h(of)f(the)g(b)o(yte)g(follo)o(wing)h(the)f(last)g(elemen)o (tary)g(datat)o(yp)q(e)g(item)h(accessible)h(b)o(y)e(that)75 539 y(pro)q(cess)d(\(excluding)i(the)d(holes\).)20 b(It)14 b(ma)o(y)f(happ)q(en)i(that)e(data)g(is)h(lo)q(cated)h(b)q(ey)o(ond)f (the)g(end)g(of)g(\014le)h(for)75 596 y(a)g(giv)o(en)h(pro)q(cess.)k (This)15 b(data)g(is)h(accessible)h(only)f(b)o(y)f(other)f(pro)q (cesses.)166 652 y(When)d(a)f(\014le)i(gro)o(ws,)e(b)q(ecause)h(of)f (more)g(data)g(b)q(eing)i(written)f(to)f(the)g(\014le)i(or)e(the)h (\014le)g(b)q(eing)h(resized,)75 708 y(the)k(end)h(of)f(\014le)i(of)e (all)h(pro)q(cesses)g(accessing)g(the)f(\014le)i(ma)o(y)d(c)o(hange,)i (and)f(data)g(no)o(w)g(accessible,)i(but)75 765 y(not)d(y)o(et)g (written)g(b)o(y)g(an)o(y)g(of)g(these)g(pro)q(cesses,)g(will)i(b)q(e)f (read)f(as)g(zero)q(es.)75 887 y Fl(10.1.8)49 b(File)17 b(La)o(y)o(out)75 972 y Ft(The)k(basic)g(access)g(routines)g(only)g(sp) q(ecify)g(ho)o(w)f(the)h(data)f(should)h(b)q(e)g(laid)h(out)e(in)i(a)e (virtual)h(\014le)75 1029 y(structure)d(\(the)g(\014let)o(yp)q(e\),)i (not)d(ho)o(w)h(that)g(\014le)h(structure)f(is)h(to)f(b)q(e)h(stored)e (on)i(one)f(or)g(more)g(disks.)75 1085 y(This)g(w)o(as)f(a)o(v)o(oided) g(b)q(ecause)i(it)f(is)g(exp)q(ected)g(that)f(the)h(mapping)g(of)f (\014les)h(to)f(disks)h(will)h(b)q(e)f(system)75 1142 y(sp)q(eci\014c,)c(and)d(an)o(y)g(sp)q(eci\014c)i(con)o(trol)e(o)o(v)o (er)f(\014le)j(la)o(y)o(out)d(w)o(ould)i(therefore)f(restrict)g (program)f(p)q(ortabilit)o(y)l(.)75 1198 y(Ho)o(w)o(ev)o(er,)i(there)h (are)g(still)i(cases)e(where)g(some)g(information)g(will)i(b)q(e)f (necessary)f(in)h(order)f(to)g(optimize)75 1255 y(disk)i(la)o(y)o(out.) j(This)d(information)f(is)g(pro)o(vided)h(as)e Fk(hints)h Ft(sp)q(eci\014ed)i(when)e(a)f(\014le)i(is)g(created.)k(Hin)o(ts)14 b(are)75 1311 y(advisory)f(only)l(,)h(and)g(do)f(not)g(c)o(hange)h(the) f(seman)o(tics)h(of)f(an)o(y)g(routines.)19 b(Instead,)14 b(they)f(allo)o(w)h(sp)q(eci\014c)75 1368 y(implemen)o(tations)i(to)f (increase)h(I/O)g(p)q(erformance.)75 1511 y Fm(10.2)60 b(File)19 b(Manipulation)75 1614 y Fl(10.2.1)49 b(Op)q(ening)18 b(a)e(File)h(\(Collective\))75 1794 y Fq(MPI)p 160 1794 14 2 v 16 w(OPEN\(comm,)c(\014lename,)i(amo)q(de,)g(disp,)h(et)o(yp)q (e,)g(\014let)o(yp)q(e,)g(hints,)g(fh\))117 1871 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])14 b(Comm)n(unicator)d(that)j(op)q(ens)h(the) f(\014le)g(\(handle\))117 1947 y(IN)155 b Fq(\014lename)430 b Ff([SAME])14 b(Name)e(of)i(\014le)g(to)f(b)q(e)i(op)q(ened)g (\(string\))117 2022 y(IN)155 b Fq(amo)q(de)461 b Ff([SAME])14 b(File)f(access)j(mo)q(de)d(\(in)o(teger\))117 2097 y(IN)155 b Fq(disp)513 b Ff(Absolute)15 b(displacemen)o(t)e(\(nonnegativ)o(e)g (o\013set\))117 2172 y(IN)155 b Fq(et)o(yp)q(e)487 b Ff([SAME])14 b(Elemen)o(tary)f(datat)o(yp)q(e)h(\(handle\))117 2247 y(IN)155 b Fq(\014let)o(yp)q(e)452 b Ff(Filet)o(yp)q(e)14 b(\(handle\))117 2322 y(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(to)g(the)g(\014le)g(system)g(\(handle\))117 2397 y(OUT)108 b Fq(fh)550 b Ff(Returned)15 b(\014le)f(handle)g(\(handle\))75 2522 y Fp(int)23 b(MPI)p 245 2522 15 2 v 17 w(Open\(MPI)p 454 2522 V 16 w(Comm)h(comm,)f(char)g(*filename,)g(MPI)p 1187 2522 V 16 w(Mode)h(amode,)f(MPI)p 1562 2522 V 17 w(Offset)g(disp,)393 2578 y(MPI)p 468 2578 V 17 w(Datatype)g(etype,)g (MPI)p 939 2578 V 17 w(Datatype)f(filetype,)h(MPI)p 1481 2578 V 17 w(Hints)g(hints,)393 2635 y(MPI)p 468 2635 V 17 w(File)g(*fh\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1161 b Ft(9)75 49 y Fp(MPI)p 150 49 15 2 v 17 w(OPEN\(COMM,)22 b(FILENAME,)h(AMODE,)g(DISP,)g(ETYPE,)g(FILETYPE,)g(HINTS,)g(FH,)g (IERROR\))170 106 y(CHARACTER)g(FILENAME\(*\))170 162 y(INTEGER)g(COMM,)h(AMODE,)f(ETYPE,)g(FILETYPE)170 219 y(INTEGER)g(HINTS,)g(FH,)h(IERROR)170 275 y(INTEGER*8)f(DISP)166 362 y Fq(MPI)p 251 362 14 2 v 16 w(OPEN)c Ft(op)q(ens)h(the)e(\014le)i (iden)o(ti\014ed)h(b)o(y)e(the)f(\014le)i(name)f Fq(\014lename)g Ft(on)f(all)i(pro)q(cesses)f(in)h(the)75 418 y Fq(comm)13 b Ft(comm)o(unicator)h(group.)19 b Fq(MPI)p 732 418 V 16 w(OPEN)c Ft(is)g(a)g(collectiv)o(e)h(routine;)f(all)g(pro)q(cesses)g (in)h(the)e(comm)o(u-)75 474 y(nicator)j(group)g(m)o(ust)g(call)i Fq(MPI)p 648 474 V 15 w(OPEN)p Ft(,)f(with)g(iden)o(tical)h(v)m(alues)g (for)d Fq(comm)p Ft(,)g Fq(\014lename)p Ft(,)i Fq(amo)q(de)p Ft(,)f(and)75 531 y Fq(et)o(yp)q(e)p Ft(.)j(\(V)l(alues)12 b(for)f Fq(disp)p Ft(,)i Fq(\014let)o(yp)q(e)p Ft(,)g(and)f Fq(hints)h Ft(ma)o(y)d(v)m(ary)l(.\))19 b(A)11 b(pro)q(cess)h(can)f(op) q(en)h(a)g(\014le)g(indep)q(enden)o(tly)75 587 y(of)k(other)h(pro)q (cesses)g(b)o(y)f(using)i(the)f Fg(MPI)p 789 587 13 2 v 14 w(COMM)p 933 587 V 15 w(SELF)f Ft(comm)o(unicator.)24 b(The)17 b(\014le)h(handle)g(returned,)75 644 y Fq(fh)p Ft(,)d(can)g(b)q(e)h(subsequen)o(tly)h(used)f(to)e(access)h(the)h (\014le)g(un)o(til)g(the)g(\014le)g(is)g(closed)g(using)g Fq(MPI)p 1643 644 14 2 v 15 w(CLOSE)p Ft(.)166 700 y(The)f(follo)o (wing)h(access)g(mo)q(des,)f(sp)q(eci\014ed)i(via)e Fq(amo)q(de)p Ft(,)g(are)g(supp)q(orted:)143 790 y Fh(\017)23 b Fg(MPI)p 266 790 13 2 v 14 w(RDONL)m(Y)15 b Ft(-)g(reading)h(only)l(,)143 883 y Fh(\017)23 b Fg(MPI)p 266 883 V 14 w(RD)o(WR)15 b Ft(-)g(reading)h(and)f(writing,)143 975 y Fh(\017)23 b Fg(MPI)p 266 975 V 14 w(WRONL)m(Y)15 b Ft(-)g(writing)h(only)l(,)143 1068 y Fh(\017)23 b Fg(MPI)p 266 1068 V 14 w(CREA)m(TE)13 b Ft(-)j(creating)f(\014le,)143 1160 y Fh(\017)23 b Fg(MPI)p 266 1160 V 14 w(DELETE)p 435 1160 V 15 w(ON)p 510 1160 V 15 w(CLOSE)14 b Ft(-)i(deleting)g(on)g(close.)75 1250 y(These)c(can)g(b)q(e)g(com)o(bined)h(in)f(C)g(b)o(y)f(using)i(the)f (bit)o(wise)g(OR)h(op)q(erator,)e(and)h(in)g(F)o(OR)l(TRAN)h(b)o(y)e (adding)75 1307 y(them.)189 1408 y Fk(A)n(dvic)n(e)18 b(to)h(implementors.)58 b Ft(The)18 b(v)m(alues)i(of)d(these)i(constan) o(ts)e(m)o(ust)h(b)q(e)h(de\014ned)g(suc)o(h)g(that)189 1465 y(the)14 b(bit)o(wise)h(OR)h(and)e(the)h(sum)f(of)g(an)o(y)g (distinct)i(set)e(of)g(these)h(constan)o(ts)e(is)i(equiv)m(alen)o(t.)22 b(\()p Fk(End)189 1521 y(of)16 b(advic)n(e)g(to)h(implementors.)p Ft(\))189 1623 y Fk(A)n(dvic)n(e)k(to)i(users.)81 b Ft(Some)22 b(\014le)h(attributes)e(are)h(inheren)o(tly)i(implemen)o(tation)f(dep)q (enden)o(t.)189 1680 y(These)17 b(attributes)g(m)o(ust)g(b)q(e)h(set)f (using)h(facilities)h(outside)f(the)f(scop)q(e)h(of)f Fq(MPI)p Ft(.)f(F)l(or)g(example,)189 1736 y(UNIX)e(\014le)h(access)f (p)q(ermissions)h(can)f(b)q(e)h(sp)q(eci\014ed)h(via)e(the)g(UNIX)g (umask.)19 b(\()p Fk(End)c(of)g(advic)n(e)g(to)189 1793 y(users.)p Ft(\))189 1895 y Fk(A)n(dvic)n(e)j(to)h(users.)57 b Ft(Note)18 b(that)g(the)g(UNIX)h(app)q(end)g(mo)q(de)f(is)h(not)f (supp)q(orted.)29 b(This)19 b(mo)q(de)189 1951 y(can)f(b)q(e)h(em)o (ulated)f(b)o(y)g(requesting)h(the)f(curren)o(t)g(\014le)h(size)g (\(see)f(Section)h(10.2.3\))d(and)i(seeking)189 2007 y(to)f(the)h(end)h(of)f(\014le)h(b)q(efore)f(eac)o(h)h(write)f(op)q (eration.)29 b Fe(Discussion)o(:)22 b Ff(Actually)m(,)15 b(the)i(seman)o(tics)g(of)189 2147 y(UNIX)f(app)q(end)h(mo)q(de)e(are)i (at)f(the)h(\014lesystem)f(lev)o(el,)g(and)g(accoun)o(t)g(for)g(all)f (applications)g(running)h(on)189 2203 y(the)c(system)f(\(not)g(just)h (a)f(single)g(application\).)16 b(UNIX)c(app)q(end)g(mo)q(de)e(cannot)i (b)q(e)g(fully)e(em)o(ulated)g(using)189 2259 y(the)k(ab)q(o)o(v)o(e.)k (Ho)o(w)13 b(imp)q(ortan)o(t)g(is)g(a)h(true)h(UNIX)f(app)q(end)g(mo)q (de?)19 b Ft(\()p Fk(End)d(of)g(advic)n(e)g(to)h(users.)p Ft(\))166 2444 y(The)i Fq(disp)h Ft(displacemen)o(t)h(argumen)o(t)d(sp) q(eci\014es)i(the)f(p)q(osition)h(\(absolute)f(o\013set)f(in)i(b)o (ytes)f(from)75 2501 y(the)d(b)q(eginning)j(of)d(the)g(\014le\),)h (where)g(the)f(\014le)h(is)g(to)f(b)q(e)h(op)q(ened.)24 b(The)17 b(constan)o(t)e Fg(MPI)p 1586 2501 V 15 w(OFFSET)p 1756 2501 V 13 w(ZERO)75 2557 y Ft(is)h(de\014ned)g(to)f(b)q(e)h (o\013set)e(zero.)189 2647 y Fk(A)n(dvic)n(e)d(to)j(users.)37 b Fq(Disp)12 b Ft(can)g(b)q(e)g(used)g(to)f(skip)h(headers,)g(or)f (when)h(the)f(\014le)i(includes)h(a)d(sequence)189 2704 y(of)f(data)h(segmen)o(ts)g(that)f(are)h(to)g(b)q(e)g(accessed)h(in)g (di\013eren)o(t)g(patterns.)18 b(\()p Fk(End)12 b(of)g(advic)n(e)h(to)g (users.)p Ft(\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Ft(10)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Ft(The)18 b Fq(et)o(yp)q(e)i Ft(argumen)o(t)d(sp)q (eci\014es)j(the)e(elemen)o(tary)g(datat)o(yp)q(e)g(used)g(to)g (construct)g(the)g Fq(\014let)o(yp)q(e)p Ft(,)75 106 y(and)h(also)f(the)h Fq(buft)o(yp)q(e)h Ft(used)f(in)h(read/write)e(op) q(erations.)30 b(O\013sets)18 b(in)o(to)h(the)g(\014le)g(are)f (measured)h(in)75 162 y(units)d(of)f Fq(et)o(yp)q(e)p Ft(.)166 219 y(The)20 b Fq(\014let)o(yp)q(e)i Ft(argumen)o(t)e(describ) q(es)h(what)f(part)g(of)g(the)g(data)g(in)h(the)f(\014le)h(is)g(b)q (eing)h(accessed.)75 275 y(Conceptually)l(,)15 b(the)f(\014le)h (starting)e(from)g Fq(disp)i Ft(is)f(tiled)h(b)o(y)f(rep)q(eated)g (copies)h(of)e Fq(\014let)o(yp)q(e)p Ft(,)i(un)o(til)g(the)f(end.)75 332 y(If)k Fq(\014let)o(yp)q(e)g Ft(has)f(holes)h(in)h(it,)e(then)h (the)f(data)g(in)h(the)g(holes)g(is)f(inaccessible)j(b)o(y)e(the)f (calling)i(pro)q(cess.)75 388 y(Ho)o(w)o(ev)o(er,)d(the)h Fq(disp)p Ft(,)h Fq(et)o(yp)q(e)h Ft(and)e Fq(\014let)o(yp)q(e)h Ft(argumen)o(ts)e(can)i(b)q(e)f(c)o(hanged)h(later)f(to)f(access)h(a)g (di\013eren)o(t)75 444 y(part)e(of)f(the)i(\014le.)166 501 y(The)c Fq(hints)h Ft(argumen)o(t)e(is)i(used)f(to)f(pro)o(vide)i (information)f(regarding)g(\014le)h(access)f(patterns)f(and)h(\014le)75 557 y(system)j(sp)q(eci\014cs)j(to)d(direct)i(optimization)g(\(see)f (section)g(10.2.4\).)k(The)c(constan)o(t)f Fg(MPI)p 1623 557 13 2 v 14 w(HINTS)p 1758 557 V 16 w(NULL)75 614 y Ft(refers)g(to)g(the)g(n)o(ull)h(hin)o(t,)g(and)f(can)h(b)q(e)f(used)h (when)g(no)f(hin)o(ts)h(need)g(to)e(b)q(e)i(sp)q(eci\014ed.)166 670 y Fq(Filename)i Ft(can)h(b)q(e)h(pre\014xed)g(with)f("mpi-impl:/")i (to)d(iden)o(tify)i(the)f(implemen)o(tation)i(that)d(the)75 727 y(op)q(en)e(call)h(w)o(ould)f(lik)o(e)h(to)e(use)h(if)g(m)o (ultiple)i(implemen)o(tations)f(co-exist.)k(Examples)c(of)e(implemen)o (ta-)75 783 y(tions)g(are:)143 869 y Fh(\017)23 b Fq(ufs:/)15 b Ft(-)g(UNIX)h(\014le)g(system,)143 960 y Fh(\017)23 b Fq(mpirun:/)14 b Ft(-)h(serv)o(er)g(comm)o(unicating)h(to)f(clien)o (t)h(using)g(mpi,)143 1051 y Fh(\017)23 b Fq(so)q(ck)o(et:/)15 b Ft(-)h(serv)o(er)e(comm)o(unicating)i(to)f(clien)o(t)h(using)g(so)q (c)o(k)o(et,)143 1142 y Fh(\017)23 b Fq(piofs:/)15 b Ft(-)g(IBM)g(AIX)h(P)o(arallel)g(I/O)g(File)g(System)f([1,)f(2].)166 1228 y(Files)e(are)f(op)q(ened)h(b)o(y)f(default)h(using)g Fq(MPI)p 901 1228 14 2 v 16 w(RECKLESS)g Ft(\014le)g(consistency)g (seman)o(tics)f(\(see)g(section)75 1284 y(10.1.6\).)17 b(The)d(more)e(stringen)o(t)i Fg(MPI)p 711 1284 13 2 v 14 w(CA)o(UTIOUS)d Ft(consistency)j(seman)o(tics,)f(required)i(for)d (atomicit)o(y)h(of)75 1341 y(o)o(v)o(erlapping)j(accesses,)f(can)g(b)q (e)h(set)f(using)h Fq(MPI)p 924 1341 14 2 v 16 w(FILE)p 1031 1341 V 15 w(CONTROL)p Ft(.)75 1461 y Fl(10.2.2)49 b(Closing)18 b(a)f(\014le)f(\(Collective\))75 1641 y Fq(MPI)p 160 1641 V 16 w(CLOSE\(fh\))117 1718 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))75 1843 y Fp(int)23 b(MPI)p 245 1843 15 2 v 17 w(Close\(MPI)p 478 1843 V 16 w(File)g(fh\))75 1929 y(MPI)p 150 1929 V 17 w(CLOSE\(FH,)f(IERROR\))170 1986 y(INTEGER)h(FH,)h(IERROR)166 2072 y Fq(MPI)p 251 2072 14 2 v 16 w(CLOSE)d Ft(closes)g(the)f(\014le)i (asso)q(ciated)e(with)h Fq(fh)p Ft(.)36 b(After)20 b(closing,)i(the)f (con)o(ten)o(t)e(of)h(the)h(\014le)75 2129 y(handle)16 b Fq(fh)f Ft(is)h(destro)o(y)o(ed.)j(F)l(uture)c(use)g(of)f Fq(fh)i Ft(is)f(erroneous.)20 b(If)15 b(the)g(\014le)h(w)o(as)e(op)q (ened)i(with)f(the)g(access)75 2185 y(mo)q(de)22 b Fg(MPI)p 281 2185 13 2 v 14 w(DELETE)p 450 2185 V 15 w(ON)p 525 2185 V 15 w(CLOSE)p Ft(,)f(the)h(\014le)h(is)f(deleted.)41 b(Ho)o(w)o(ev)o(er,)22 b(if)g(there)g(are)f(other)h(pro)q(cesses)75 2242 y(curren)o(tly)e(accessing)h(the)f(\014le,)h(the)f(status)f(of)h (the)g(\014le)g(and)g(the)g(b)q(eha)o(vior)h(of)e(future)h(accesses)g (b)o(y)75 2298 y(these)15 b(pro)q(cesses)h(are)f(implemen)o(tation)h (dep)q(enden)o(t.)189 2395 y Fk(A)n(dvic)n(e)e(to)h(implementors.)40 b Ft(If)14 b(the)g(\014le)h(is)g(to)e(b)q(e)i(deleted)g(and)g(is)f(op)q (ened)h(b)o(y)f(other)g(pro)q(cesses,)189 2452 y(\014le)f(data)f(ma)o (y)g(still)i(b)q(e)g(accessible)g(b)o(y)f(these)g(pro)q(cesses)g(un)o (til)g(they)g(close)h(the)e(\014le)i(or)e(un)o(til)i(they)189 2508 y(exit.)20 b(The)14 b(delete)h(op)q(eration)f(should)h(b)q(e)g (e\013ectiv)o(e)f(only)h(on)f(the)g(last)g(close)h(of)e(that)h(\014le)h (within)189 2564 y(this)e(application.)21 b Fe(Discussion:)15 b Ff(This)d(implemen)o(tors)e(note)j(seems)f(to)g(con)o(tradict)h(the)g (last)f(sen)o(tence)189 2704 y(of)h(the)h(paragraph)g(ab)q(o)o(v)o(e.) 19 b Ft(\()p Fk(End)d(of)g(advic)n(e)h(to)f(implementors.)p Ft(\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fn(10.2.)29 b(FILE)15 b(MANIPULA)l(TION)1144 b Ft(11)75 49 y Fl(10.2.3)49 b(File)17 b(Control)g(\(Indep)q (endent/Collective\))75 211 y Fe(Discussion:)166 261 y Ff(It)e(is)g(not)g(p)q(ossible)g(to)g(supp)q(ort)h(F)o(OR)m(TRAN)e (with)g(the)i(curren)o(t)g(\014le)f(con)o(trol)g(in)o(terface.)22 b(The)16 b(follo)o(wing)75 311 y(alternativ)o(es)e(ha)o(v)o(e)g(b)q (een)h(suggested:)134 398 y(1.)22 b(Mak)o(e)13 b(MPI)p 383 398 13 2 v 16 w(Fcn)o(tl)h(w)o(ork)f(with)h(an)g(opaque)f(ob)r (ject)i(\(a)f(la)f(MPI)p 1207 398 V 15 w(Hin)o(ts\),)h(and)f(pro)o (vide)h(access)i(functions)189 448 y(for)d(eac)o(h)i(elemen)o(t.)134 522 y(2.)22 b(P)o(ass)14 b(all)f(thirteen)i(items)e(as)h(parameters,)f (no)h(structure.)134 597 y(3.)22 b(Pro)o(vide)13 b(MPI)p 424 597 V 15 w(Fcn)o(tl,)g(MPI)p 640 597 V 15 w(Fcn)o(tl)p 750 597 V 15 w(2,)g(MPI)p 892 597 V 15 w(Fcn)o(tl)p 1002 597 V 15 w(3,)g(etc.,)g(where)i(MPI)p 1353 597 V 15 w(Fcn)o(tl)e(tak)o (es)h(only)e(one)i(param-)189 647 y(eter,)g(MPI)p 366 647 V 16 w(Fcn)o(tl)p 477 647 V 15 w(2)f(tak)o(es)i(2)e(parameters,)h (etc.)134 722 y(4.)22 b(Group)13 b(the)i(items)e(in)g(some)g (meaningful)e(w)o(a)o(y)m(,)i(and)g(pro)o(vide)h(a)g(routine)g(for)f (eac)o(h)i(group)e(of)h(items.)75 948 y Fp(The)23 b(last)h(alternative) e(has)i(further)e(been)i(expanded)f(on:)75 1061 y(Grouping)g(the)g (items)g(and)h(providing)e(meaningful)h(routines)g(has)g(two)h (advantages:)75 1117 y(individual)e(pieces)i(of)f(this)g(standard)g (are)h(more)f(modular)75 1174 y(\(e.g.)g(file)g(pointer)g(manipulation) g(is)g(all)h(contained)e(in)i(the)f(file)h(pointer)f(section\),)75 1230 y(and)g(I/O)h(code)f(will)h(be)f(more)g(self-documenting)75 1287 y(\(e.g.)g(a)h(re-open)f(is)g(expressed)g(as)h(MPI_Reopen\).)75 1400 y(As)g(a)f(first)g(cut,)h(I)g(propose)e(we)i(group)f(the)h (MPI_Fcntl)e(options)h(into)h(the)75 1456 y(following)f(procedures:)75 1569 y(MPI_FCNTL_COMM)75 1625 y(MPI_FCNTL_FILENAME)75 1682 y(MPI_FCNTL_AMODE)75 1738 y(MPI_FCNTL_MOFFSET)75 1795 y(MPI_FCNTL_DISP)75 1851 y(MPI_FCNTL_ETYPE)75 1908 y(MPI_FCNTL_FILETYPE)75 1964 y(MPI_FCNTL_HINTS)75 2021 y(MPI_FCNTL_HINTS_TAKEN)75 2134 y(MPI_Open\(comm,)f(filename,)h(amode,) g(disp,)g(etype,)123 2190 y(filetype,)f(moffset,)h(hints,)g(*fh\);)75 2246 y(MPI_File_info\(*comm,)e(*filename,)i(*amode,)g(*disp,)g(*etype,) 242 2303 y(*filetype,)g(*moffset,)f(*hints_taken\);)75 2359 y(MPI_Reopen\(fh,)g(disp,)h(etype,)g(filetype,)g(hints,)g (*new_fh\);)75 2472 y(MPI_FCNTL_ATOMICITY)75 2585 y(MPI_Atomicity\(fh,) f(atomicity,)g(*old_atomicity\);)75 2698 y(MPI_FCNTL_FILE_SIZE)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Ft(12)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)75 106 y Fp(MPI_Resize\(fh,)22 b(MPI_Offset)g(disp\);)75 162 y(MPI_File_size\(fh,)g(MPI_Offset)g(*size_in_bytes\);)75 275 y(MPI_FCNTL_IND_FP)75 332 y(MPI_FCNTL_ABS_OFFSET)75 444 y(MPI_Seek\(fh,)g(offset,)h(whence\);)75 501 y (MPI_Next_position\(fh,)e(*offset,)i(*byte_offset\);)75 614 y(MPI_FCNTL_SHARED_FP)75 727 y(MPI_Seek_shared\(fh,)e(offset,)i (whence\);)75 783 y(MPI_Shared_position\(fh,)e(*offset,)h (*byte_offset\);)75 896 y(Note)h(that)h(we)f(only)h(need)f(to)g(add)h (6)g(routines)e(as)i(we)g(already)75 953 y(have)f(MPI_Open,)g (MPI_Resize,)f(MPI_Seek,)h(and)g(MPI_Seek_shared.)75 1153 y Fq(MPI)p 160 1153 14 2 v 16 w(FILE)p 267 1153 V 15 w(CONTROL\(fh,)16 b(cmd,)f(fcntl\))117 1230 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\)) 117 1306 y(IN)155 b Fq(cmd)507 b Ff([SAME])14 b(Comma)o(nd)117 1381 y(INOUT)62 b Fq(fcntl)503 b Ff(Argumen)o(ts)14 b(or)g(return)h(v)n (alues)e(to)h(the)g(command)75 1505 y Fp(int)23 b(MPI)p 245 1505 15 2 v 17 w(File)p 358 1505 V 17 w(control\(MPI)p 639 1505 V 16 w(File)g(fh,)g(unsigned)g(cmd,)g(MPI)p 1275 1505 V 17 w(Fcntl)g(*fcntl\))166 1592 y Ft(\(F)o(OR)l(TRAN)16 b(binding)h(to)d(b)q(e)i(pro)o(vided)g(in)g(v)o(ersion)g(0.6)e(of)h (the)g(draft.\))166 1648 y Fq(MPI)p 251 1648 14 2 v 16 w(FILE)p 358 1648 V 15 w(CONTROL)g Ft(gets)e(or)g(sets)g(\014le)i (information)f(ab)q(out)f(the)h(\014le)h(asso)q(ciated)e(with)h(the)g (\014le)75 1705 y(handle)j Fq(fh)p Ft(.)22 b(Multiple)c(commands)d(can) h(b)q(e)g(issued)h(in)g(one)f(call)h(using)f(the)g(bit)o(wise)h(OR)f (op)q(erator.)21 b(If)75 1761 y Fg(MPI)p 152 1761 13 2 v 14 w(F)o(CNTL)p 295 1761 V 15 w(SET)e Ft(is)h(set,)f(the)g (\014elds)h(indicated)h(b)o(y)e(the)g(bits)g(set)g(in)h Fq(cmd)e Ft(will)j(b)q(e)f(used)f(to)f(up)q(date)75 1817 y(the)i(\014le)h(information.)34 b(If)20 b Fg(MPI)p 641 1817 V 14 w(F)o(CNTL)p 784 1817 V 15 w(SET)g Ft(is)g(not)g(set,)g(the)g (\014le)h(information)f(requested)g(will)h(b)q(e)75 1874 y(returned.)f Fg(MPI)p 354 1874 V 15 w(Fcntl)15 b Ft(is)h(de\014ned)g (as)f(follo)o(ws:)170 1968 y Fp(typedef)23 b(struct)g(MPI_Fcntl)g({)266 2024 y(MPI_Comm)261 b(comm;)266 2081 y(char)333 b(*filename;)266 2137 y(MPI_Mode)237 b(amode;)266 2194 y(MPI_Offset)189 b(disp;)266 2250 y(MPI_Datatype)165 b(etype;)266 2306 y(MPI_Datatype)g(filetype;)266 2363 y(MPI_Hints)190 b(*hints;)266 2419 y(MPI_Hints)g(*hints_taken;)266 2476 y(MPI_Atomicity)117 b(atomicity;)266 2532 y(MPI_Offset)189 b(ind_fp;)266 2589 y(MPI_Offset)g(shared_fp;)266 2645 y(MPI_Offset)g(file_size;)266 2702 y(MPI_Offset)g(abs_offset;)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(13)170 49 y Fp(})24 b(MPI_Fcntl;)166 143 y Ft(The)15 b(commands)g(a)o(v)m(ailable)i(are:)143 237 y Fh(\017)23 b Fg(MPI)p 266 237 13 2 v 14 w(F)o(CNTL)p 409 237 V 15 w(SET)p Ft(:)15 b(Set)g(\014le)h(information.)143 331 y Fh(\017)23 b Fg(MPI)p 266 331 V 14 w(F)o(CNTL)p 409 331 V 15 w(COMM)p Ft(:)15 b(Get)f(the)i(comm)o(unicator)e(asso)q (ciated)i(with)f(the)h(\014le.)143 425 y Fh(\017)23 b Fg(MPI)p 266 425 V 14 w(F)o(CNTL)p 409 425 V 15 w(FILENAME)p Ft(:)14 b(Get)h(the)g(\014lename.)143 518 y Fh(\017)23 b Fg(MPI)p 266 518 V 14 w(F)o(CNTL)p 409 518 V 15 w(AMODE)p Ft(:)14 b(Get)h(or)g(set)f(the)i(\014le)g(access)f(mo)q(de)h(asso)q (ciated)f(with)h(the)f(\014le.)143 612 y Fh(\017)23 b Fg(MPI)p 266 612 V 14 w(F)o(CNTL)p 409 612 V 15 w(DISP)p Ft(:)14 b(Get)h(or)f(set)h(the)h(displacemen)o(t.)143 706 y Fh(\017)23 b Fg(MPI)p 266 706 V 14 w(F)o(CNTL)p 409 706 V 15 w(ETYPE)p Ft(:)14 b(Get)h(or)f(set)h(the)g(elemen)o(tary)h (datat)o(yp)q(e.)143 800 y Fh(\017)23 b Fg(MPI)p 266 800 V 14 w(F)o(CNTL)p 409 800 V 15 w(FILETYPE)p Ft(:)14 b(Get)g(or)h(set)g(the)g(\014let)o(yp)q(e.)143 894 y Fh(\017)23 b Fg(MPI)p 266 894 V 14 w(F)o(CNTL)p 409 894 V 15 w(HINTS)p Ft(:)15 b(Get)f(or)g(set)g(the)h(desired)g(v)m(alues)h (for)e(the)g(hin)o(ts)h(asso)q(ciated)g(with)g(the)f(\014le.)143 987 y Fh(\017)23 b Fg(MPI)p 266 987 V 14 w(F)o(CNTL)p 409 987 V 15 w(HINTS)p 545 987 V 16 w(T)m(AKEN)p Ft(:)14 b(Get)h(the)g(v)m(alues)h(for)f(the)g(hin)o(ts)h(actually)g(used.)143 1081 y Fh(\017)23 b Fg(MPI)p 266 1081 V 14 w(F)o(CNTL)p 409 1081 V 15 w(A)m(TOMICITY)p Ft(:)13 b(Get)i(or)g(set)g(the)h(curren) o(t)f(read/write)h(atomic)f(seman)o(tics)g(enforced)189 1138 y(mo)q(de.)143 1231 y Fh(\017)23 b Fg(MPI)p 266 1231 V 14 w(F)o(CNTL)p 409 1231 V 15 w(IND)p 495 1231 V 15 w(FP)p Ft(:)11 b(Get)g(the)h(curren)o(t)g(o\013set)g(of)f(the)h (individual)k(\014le)d(p)q(oin)o(ter)f(asso)q(ciated)h(with)189 1288 y(the)k(\014le)g(\(n)o(um)o(b)q(er)g(of)g(elemen)o(tary)g(datat)o (yp)q(e)f(items)h(within)h(the)f(\014le)h(after)e(the)h(displacemen)o (t)189 1344 y(p)q(osition\).)143 1438 y Fh(\017)23 b Fg(MPI)p 266 1438 V 14 w(F)o(CNTL)p 409 1438 V 15 w(SHARED)p 586 1438 V 15 w(FP)p Ft(:)18 b(Get)i(the)g(curren)o(t)g(o\013set)f(of)g (the)h(shared)g(\014le)h(p)q(oin)o(ter)g(asso)q(ciated)189 1495 y(with)15 b(the)h(\014le)g(\(n)o(um)o(b)q(er)f(of)g(elemen)o(tary) h(datat)o(yp)q(e)e(items)i(within)g(the)g(\014le)g(after)f(the)g (displace-)189 1551 y(men)o(t)g(p)q(osition\).)143 1645 y Fh(\017)23 b Fg(MPI)p 266 1645 V 14 w(F)o(CNTL)p 409 1645 V 15 w(FILE)p 507 1645 V 14 w(SIZE)p Ft(:)15 b(Get)g(the)g(size)h (of)f(the)g(\014le)h(asso)q(ciated)g(with)f(fh.)143 1739 y Fh(\017)23 b Fg(MPI)p 266 1739 V 14 w(F)o(CNTL)p 409 1739 V 15 w(ABS)p 503 1739 V 14 w(OFFSET)p Ft(:)14 b(Get)h(the)g (absolute)h(b)o(yte)f(o\013set)f(of)h(the)g(individual)j(\014le)f(p)q (oin)o(ter.)75 1833 y(Not)11 b(all)i(\014elds)g(in)g Fq(fcntl)g Ft(are)e(c)o(hangeable.)20 b(When)12 b Fq(et)o(yp)q(e)p Ft(,)i Fq(\014let)o(yp)q(e)p Ft(,)f(or)f Fq(disp)h Ft(is)f(c)o(hanged,) h(all)g(\014le)g(p)q(oin)o(ters)75 1889 y(asso)q(ciated)i(with)h Fq(fh)f Ft(are)g(reset)g(to)g Fg(MPI)p 765 1889 V 14 w(OFFSET)p 934 1889 V 14 w(ZERO)p Ft(.)166 1945 y(All)d(calls)f(that)f (get)g(\014le)i(information)e(are)h(indep)q(enden)o(t)h(calls.)20 b(All)12 b(calls)f(that)f(set)g(\014le)i(information)75 2002 y(are)j(collectiv)o(e)j(calls.)k(When)15 b(setting)h Fq(et)o(yp)q(e)p Ft(,)g Fq(amo)q(de)p Ft(,)f(and)h Fq(atomicit)o(y)p Ft(,)e(it)i(is)g(erroneous)g(for)f(pro)q(cesses)75 2058 y(to)g(pass)g(in)h(di\013eren)o(t)f(v)m(alues.)75 2180 y Fl(10.2.4)49 b(File)17 b(Hints)75 2266 y Ft(Hin)o(ts)k(allo)o(w)h(a)e (user)i(to)e(pro)o(vide)i(information)f(regarding)g(\014le)h(access)f (patterns)g(and)g(\014le)h(system)75 2322 y(sp)q(eci\014cs)i(to)e (direct)h(optimization.)42 b(Pro)o(viding)23 b(hin)o(ts)g(ma)o(y)e (enable)j(an)e(implemen)o(tation)i(to)d(de-)75 2379 y(liv)o(er)e (increased)g(I/O)f(p)q(erformance)g(or)f(minimize)j(the)e(use)g(of)f (system)g(resources.)28 b(Ho)o(w)o(ev)o(er,)17 b(hin)o(ts)75 2435 y(do)22 b(not)f(c)o(hange)h(the)f(seman)o(tics)h(of)f(an)o(y)h(of) f(the)h(in)o(terfaces.)39 b(In)23 b(other)e(w)o(ords,)h(an)g(implemen)o (ta-)75 2492 y(tion)d(is)g(free)f(to)g(ignore)g(all)i(hin)o(ts.)30 b(Hin)o(ts)18 b(are)g(sp)q(eci\014ed)j(on)d(a)g(p)q(er)h(\014le)h (basis)e(at)g Fq(MPI)p 1631 2492 14 2 v 16 w(OPEN)h Ft(\(and)75 2548 y Fq(MPI)p 160 2548 V 16 w(FILE)p 267 2548 V 15 w(CONTROL)p Ft(\))d(time.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Ft(14)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)189 49 y Fk(A)n(dvic)n(e)20 b(to)h(implementors.)71 b Ft(It)21 b(ma)o(y)e(happ)q(en)j(that)e(a)g(program)f(is)i(co)q(ded)g (with)g(hin)o(ts)g(for)189 106 y(one)15 b(system,)h(and)f(later)h (executes)g(on)g(another)f(system)h(that)f(do)q(es)h(not)f(supp)q(ort)h (these)g(hin)o(ts.)189 162 y(In)k(general,)h(unsupp)q(orted)g(hin)o(ts) f(should)h(simply)g(b)q(e)g(ignored.)34 b(In)20 b(particular,)h(\014le) g(con)o(trol)189 219 y(op)q(erations,)16 b(setting)g(or)g(inquiring)i (v)m(alues)g(to)e(unsupp)q(orted)h(hin)o(ts,)g(should)g(b)q(e)g (handled)h(as)d(if)189 275 y(these)h(hin)o(ts)g(w)o(ere)g(not)g(sp)q (eci\014ed.)25 b(Needless)17 b(to)f(sa)o(y)l(,)f(no)h(hin)o(t)h(can)f (b)q(e)h(mandatory)l(.)k(Ho)o(w)o(ev)o(er,)189 332 y(for)15 b(eac)o(h)h(hin)o(t)h(used)f(b)o(y)g(a)g(sp)q(eci\014c)i(implemen)o (tation,)f(a)f(default)h(v)m(alue)g(m)o(ust)f(b)q(e)g(pro)o(vided)h(in) 189 388 y(order)e(to)g(b)q(e)h(used)g(when)g(the)f(user)h(do)q(es)g (not)f(sp)q(ecify)i(a)e(v)m(alue)i(for)e(this)g(hin)o(t.)22 b(\()p Fk(End)15 b(of)i(advic)n(e)189 444 y(to)f(implementors.)p Ft(\))166 551 y(In)g(C,)e(hin)o(t)i(v)m(alues)g(are)f(de\014ned)i(as:) 170 657 y Fp(typedef)23 b(union)h(MPI_Hint_value)d({)266 713 y(MPI_Offset)h(hint_value;)266 770 y(void)h(*hint_ptr;)170 826 y(})h(MPI_Hint_value;)75 948 y Fl(10.2.5)49 b(Hints)17 b(Creation)f(and)h(Deallo)q(cation)75 1128 y Fq(MPI)p 160 1128 14 2 v 16 w(HINTS)p 309 1128 V 16 w(CREA)l(TE\(count,)g(hint)p 720 1128 V 18 w(names,)e(hint)p 958 1128 V 17 w(values,)h(hints\))117 1206 y Ff(IN)155 b Fq(count)482 b Ff(Num)o(b)q(er)14 b(of)f(hin)o(ts)h(supplied)g(\(in)o(teger\))117 1281 y(IN)155 b Fq(hint)p 394 1281 V 18 w(names)377 b Ff(Names)13 b(of)h(hin)o(ts)f(\(arra)o(y)h(of)g(in)o(teger\))117 1356 y(IN)155 b Fq(hint)p 394 1356 V 18 w(values)382 b Ff(V)m(alues)14 b(of)f(hin)o(ts)h(\(arra)o(y)g(of)f(in)o(teger\))117 1431 y(OUT)108 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))75 1555 y Fp(int)23 b(MPI)p 245 1555 15 2 v 17 w(Hints)p 382 1555 V 17 w(create\(int)f(count,)h(int)h(*hint)p 1044 1555 V 16 w(names,)f(MPI)p 1299 1555 V 17 w(Hint)p 1412 1555 V 17 w(value)g(*hint)p 1692 1555 V 16 w(values,)393 1612 y(MPI)p 468 1612 V 17 w(Hints)g(*hints\))75 1698 y(MPI)p 150 1698 V 17 w(HINTS)p 287 1698 V 16 w(CREATE\(COUNT,)f(HINT)p 733 1698 V 17 w(NAMES,)h(HINT)p 1013 1698 V 17 w(VALUES,)g(HINTS,)g (IERROR\))170 1755 y(INTEGER)g(COUNT,)g(HINT)p 627 1755 V 17 w(NAMES\(*\))170 1811 y(INTEGER*8)g(HINT)p 508 1811 V 17 w(VALUES\(*\))170 1868 y(INTEGER)g(HINTS,)g(IERROR)166 1954 y Fq(MPI)p 251 1954 14 2 v 16 w(HINTS)p 400 1954 V 16 w(CREA)l(TE)c Ft(returns)f(an)g(opaque)g(ob)s(ject,)g(it)h(is)f (the)g(resp)q(onsibilit)o(y)j(of)c(the)h(user)h(to)75 2011 y(delete)d(the)g(ob)s(ject)e(b)o(y)h(calling)i Fq(MPI)p 714 2011 V 16 w(HINTS)p 863 2011 V 17 w(FREE)e Ft(when)h(done.)75 2162 y Fq(MPI)p 160 2162 V 16 w(HINTS)p 309 2162 V 16 w(FREE\(hints\))117 2239 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))75 2363 y Fp(int)23 b(MPI)p 245 2363 15 2 v 17 w(Hints)p 382 2363 V 17 w(free\(MPI)p 591 2363 V 16 w(Hints)g(*hints\))75 2450 y(MPI)p 150 2450 V 17 w(HINTS)p 287 2450 V 16 w(FREE\(HINTS,)g(IERROR\))170 2506 y(INTEGER)g(HINTS,)g(IERROR)166 2593 y Fq(MPI)p 251 2593 14 2 v 16 w(HINTS)p 400 2593 V 16 w(FREE)11 b Ft(marks)e(the)i(hin)o(ts)f(ob)s(ject)g(for)f(deallo)q(cation)j(and)e (sets)g Fq(hints)i Ft(to)d Fg(MPI)p 1686 2593 13 2 v 15 w(HINTS)p 1822 2593 V 15 w(NULL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(15)166 49 y(Hin)o(t)11 b(names)f(smaller)h(or)f(equal)h(to)e Fg(MPI)p 844 49 13 2 v 15 w(HINTS)p 980 49 V 15 w(RESERVED)g Ft(when)i(implemen)o(ted)h(carry)e(the)g(same)75 106 y(meaning)h(across)e(all)i(implemen)o(tations)h(and)e(will)i(b)q(e)f (de\014ned)g(in)g(future)f(sp)q(eci\014cations.)20 b Fg(MPI)p 1697 106 V 15 w(HINTS)p 1833 106 V 15 w(RESERVED)75 162 y Ft(is)c(de\014ned)g(to)f(b)q(e)h(65535.)166 219 y(Hin)o(t)f(names)h(larger)f(than)g Fg(MPI)p 721 219 V 14 w(HINTS)p 856 219 V 16 w(RESERVED)e Ft(are)i(implemen)o(tation)i (sp)q(eci\014c)g(names.)166 275 y(The)g(constan)o(t)f Fg(MPI)p 523 275 V 15 w(HINTS)p 659 275 V 15 w(NULL)i Ft(refers)f(to)f(the)i(n)o(ull)g(hin)o(t,)g(and)f(can)g(b)q(e)h(used)f (in)h Fq(MPI)p 1740 275 14 2 v 16 w(OPEN)75 332 y Ft(when)e(no)f(hin)o (ts)g(need)i(to)d(b)q(e)i(sp)q(eci\014ed.)75 453 y Fl(10.2.6)49 b(Hints)17 b(Accesso)o(r)d(F)o(unctions)75 539 y Fq(MPI)p 160 539 V 16 w(HINTS)p 309 539 V 16 w(COUNT)i Ft(returns)f(the)h(n)o (um)o(b)q(er)f(of)g(hin)o(ts)h(in)g(the)f(opaque)g(ob)s(ject)g(hin)o (ts.)75 690 y Fq(MPI)p 160 690 V 16 w(HINTS)p 309 690 V 16 w(COUNT\(hints,)i(count\))117 767 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 842 y(OUT)108 b Fq(count)482 b Ff(Num)o(b)q(er)14 b(of)f(hin)o(ts)h(\(in)o(teger\))75 967 y Fp(int)23 b(MPI)p 245 967 15 2 v 17 w(Hints)p 382 967 V 17 w(count\(MPI)p 615 967 V 16 w(Hints)g(hints,)g(int)g(*count\)) 75 1053 y(MPI)p 150 1053 V 17 w(HINTS)p 287 1053 V 16 w(COUNT\(HINTS,)f(COUNT,)i(IERROR\))170 1110 y(INTEGER)f(HINTS,)g (COUNT,)g(IERROR)166 1196 y Fq(MPI)p 251 1196 14 2 v 16 w(HINTS)p 400 1196 V 16 w(GET)p 504 1196 V 17 w(ELEMENT)15 b Ft(returns)g(the)h(n-th)f(elemen)o(t)h(in)g(the)f(opaque)h(ob)s(ject) e(hin)o(ts.)189 1302 y Fk(A)n(dvic)n(e)h(to)i(users.)43 b Ft(The)15 b(order)h(of)f(the)g(elemen)o(ts)i(in)f(hin)o(ts)g(ma)o(y)f (or)g(ma)o(y)g(not)g(b)q(e)h(the)g(same)f(as)189 1359 y(the)g(order)g(passed)g(in)h Fq(MPI)p 668 1359 V 16 w(HINTS)p 817 1359 V 17 w(CREA)l(TE)p Ft(.)g(\()p Fk(End)f(of)i(advic)n (e)f(to)g(users.)p Ft(\))75 1560 y Fq(MPI)p 160 1560 V 16 w(HINTS)p 309 1560 V 16 w(GET)p 413 1560 V 17 w(ELEMENT\(hints,)g (n,)f(hint)p 900 1560 V 18 w(name,)f(hint)p 1120 1560 V 18 w(value\))117 1637 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 1712 y(IN)155 b Fq(n)564 b Ff(Numeric)14 b(selector)h(sp)q(ecifying)f(whic)o(h)f(hin)o(t)h(\(in) o(teger\))117 1787 y(OUT)108 b Fq(hint)p 394 1787 V 18 w(name)394 b Ff(Name)13 b(of)g(hin)o(t)h(\(in)o(teger\))117 1862 y(OUT)108 b Fq(hint)p 394 1862 V 18 w(value)399 b Ff(V)m(alue)14 b(of)f(hin)o(t)g(\(in)o(teger\))75 1987 y Fp(int)23 b(MPI)p 245 1987 15 2 v 17 w(Hints)p 382 1987 V 17 w(get)p 471 1987 V 16 w(element\(MPI)p 751 1987 V 16 w(Hints)g(hints,)g(int)h(n,)f(int)h(*hint)p 1460 1987 V 16 w(name,)393 2043 y(MPI)p 468 2043 V 17 w(Hint)p 581 2043 V 17 w(value)f(*hint)p 861 2043 V 16 w(value\))75 2130 y(MPI)p 150 2130 V 17 w(HINTS)p 287 2130 V 16 w(GET)p 375 2130 V 17 w(ELEMENT\(HINTS,)f(N,)i(HINT)p 918 2130 V 16 w(NAME,)f(HINT)p 1173 2130 V 17 w(VALUE,)g(IERROR\))170 2186 y(INTEGER)g(HINTS,)g(N,)h(HINT)p 699 2186 V 17 w(NAME)170 2243 y(INTEGER*8)f(HINT)p 508 2243 V 17 w(VALUE)170 2299 y(INTEGER)g(IERROR)166 2385 y Fq(MPI)p 251 2385 14 2 v 16 w(HINTS)p 400 2385 V 16 w(GET)p 504 2385 V 17 w(V)l(ALUE)13 b Ft(returns)f(the)g Fq(hint)p 975 2385 V 18 w(value)g Ft(of)g Fq(hint)p 1224 2385 V 18 w(name)g Ft(in)h(the)f(opaque)g(ob)s (ject)g(hin)o(ts.)75 2442 y(If)j Fq(hint)p 196 2442 V 18 w(name)g Ft(is)h(not)e(de\014ned)j(in)f Fq(hints)p Ft(,)g(an)g(error)e(will)j(b)q(e)f(returned,)f(and)g Fq(hint)p 1467 2442 V 18 w(value)g Ft(is)h(unde\014ned.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Ft(16)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(HINTS)p 309 49 V 16 w(GET)p 413 49 V 17 w(V)l(ALUE\(hints,)17 b(hint)p 780 49 V 18 w(name,)d(hint)p 1000 49 V 18 w(value\))117 126 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 200 y(IN)155 b Fq(hint)p 394 200 V 18 w(name)394 b Ff(Name)13 b(of)g(hin)o(t)h(\(in)o(teger\))117 274 y(OUT)108 b Fq(hint)p 394 274 V 18 w(value)399 b Ff(V)m(alue)14 b(of)f(hin)o(t)g(\(in)o (teger\))75 398 y Fp(int)23 b(MPI)p 245 398 15 2 v 17 w(Hints)p 382 398 V 17 w(get)p 471 398 V 16 w(value\(MPI)p 703 398 V 16 w(Hints)h(hints,)f(int)g(hint)p 1221 398 V 17 w(name,)393 455 y(MPI)p 468 455 V 17 w(Hint)p 581 455 V 17 w(value)g(*hint)p 861 455 V 16 w(value\))75 541 y(MPI)p 150 541 V 17 w(HINTS)p 287 541 V 16 w(GET)p 375 541 V 17 w(VALUE\(HINTS,)f(HINT)p 798 541 V 17 w(NAME,)h(HINT)p 1054 541 V 17 w(VALUE,)g(IERROR\))170 598 y(INTEGER)g(HINTS,)g(HINT)p 627 598 V 17 w(NAME)170 654 y(INTEGER*8)g(HINT)p 508 654 V 17 w(VALUE)170 711 y(INTEGER)g(IERROR)166 797 y Ft(Some)16 b(p)q(oten)o(tially)h(useful)f(hin)o(ts)g(are)g(outlined)h (b)q(elo)o(w.)22 b(Sp)q(eci\014c)c(implemen)o(tations)f(are)e(free)h (to)75 853 y(de\014ne)h(these)g(hin)o(ts)f(as)g(w)o(ell)h(as)f (additional)h(hin)o(ts.)23 b(Ho)o(w)o(ev)o(er,)15 b(in)i(order)f(to)g (main)o(tain)g(compatibilit)o(y)75 910 y(b)q(et)o(w)o(een)g(implemen)o (tations,)g(additional)h(hin)o(ts)e(should)i(not)e(reuse)g(the)h(same)f (hin)o(t)g(names)h(for)e(other)75 966 y(purp)q(oses)i(than)f(these)g (listed)i(here.)166 1023 y(These)c(hin)o(ts)h(are)f(mainly)h(concerned) g(with)f(la)o(y)o(out)g(of)f(data)h(on)g(parallel)h(I/O)g(devices,)g (and)g(with)75 1079 y(access)j(patterns.)24 b(F)l(or)16 b(eac)o(h)h(hin)o(t)g(name)g(in)o(tro)q(duced,)h(w)o(e)f(describ)q(e)h (the)f(purp)q(ose)g(of)g(the)f(hin)o(t,)i(and)75 1136 y(the)e(t)o(yp)q(e)f(of)h(the)f(hin)o(t)h(v)m(alue.)23 b(If)15 b(sp)q(eci\014ed,)j(the)e(\\)p Fo([SAME])p Ft(")d(annotation)i (stipulates)i(that)e(the)g(hin)o(t)75 1192 y(v)m(alues)j(pro)o(vided)g (b)o(y)f(all)g(participating)h(pro)q(cesses)g(m)o(ust)e(b)q(e)i(iden)o (tical,)h(otherwise)e(the)g(program)e(is)75 1249 y(erroneous.)75 1346 y Fg(MPI)p 152 1346 13 2 v 14 w(HINTS)p 287 1346 V 16 w(STRIPING)p 489 1346 V 14 w(UNIT)i Fo(\(in)o(teger\))h([SAME]:)j Ft(This)d(hin)o(t)h(sp)q(eci\014es)h(the)e(suggested)g(striping)189 1402 y(unit)c(to)e(b)q(e)i(used)g(for)f(this)g(\014le.)21 b(The)13 b(striping)h(unit)g(is)g(the)f(amoun)o(t)g(of)g(consecutiv)o (e)h(data)e(tak)o(en)189 1459 y(from)j(one)i(I/O)g(no)q(de)g(\(resp.)24 b(storage)15 b(device\))j(b)q(efore)f(progressing)f(to)g(the)h(next)f (no)q(de)i(\(resp.)189 1515 y(storage)c(device\),)j(when)f(striping)g (across)f(a)h(n)o(um)o(b)q(er)g(of)f(no)q(des)h(\(resp.)21 b(storage)15 b(devices\);)h(it)g(is)189 1572 y(expressed)j(in)g(b)o (ytes.)30 b(This)19 b(hin)o(t)g(is)g(relev)m(an)o(t)g(only)h(when)f (the)f(\014le)i(is)f(created,)g(and)g(if)g(used,)189 1628 y(this)13 b(data)f(should)h(b)q(e)h(main)o(tained)f(b)o(y)g(the)g (\014le)g(serv)o(er\(s\))f(as)g(part)g(of)h(the)f(metadata)g(asso)q (ciated)189 1685 y(with)g(the)g(\014le.)19 b(A)12 b(go)q(o)q(d)g(size)g (for)g(a)f(striping)i(unit)f(is)g(the)g(amoun)o(t)f(of)h(consecutiv)o (e)g(data)f(accessed)189 1741 y(indep)q(enden)o(tly)18 b(b)o(y)d(a)g(pro)q(cess,)g(or)g(if)g(this)h(is)g(to)q(o)e(small,)i(a)f (m)o(ultiple)i(of)d(that)h(size.)75 1832 y Fg(MPI)p 152 1832 V 14 w(HINTS)p 287 1832 V 16 w(STRIPING)p 489 1832 V 14 w(F)m(A)o(CTOR)g Fo(\(in)o(teger\))j([SAME]:)j Ft(This)e(hin)o(t)g (sp)q(eci\014es)h(the)e(n)o(um)o(b)q(er)h(of)f(I/O)189 1889 y(no)q(des)h(\(resp.)29 b(storage)17 b(devices\))j(that)e(the)g (\014le)i(should)f(b)q(e)g(strip)q(ed)h(across,)e(and)g(is)h(relev)m (an)o(t)189 1945 y(only)c(when)h(the)f(\014le)i(is)e(created.)75 2036 y Fg(MPI)p 152 2036 V 14 w(HINTS)p 287 2036 V 16 w(NUM)p 397 2036 V 15 w(IO)p 455 2036 V 14 w(NODES)i Fo(\(in)o(teger\))h([SAME]:)j Ft(The)c(n)o(um)o(b)q(er)g(of)f(I/O)h(no) q(des)h(in)f(the)g(system.)189 2092 y(Used)h(to)f(sp)q(ecify)i(the)f (ideal)h(n)o(um)o(b)q(er)f(of)f(I/O)h(no)q(des)h(for)e(this)h (application)h(as)f(w)o(ell)g(as)g(query)189 2149 y(the)d(n)o(um)o(b)q (er)g(of)g(I/O)h(no)q(des)g(actually)g(in)g(use.)75 2240 y Fg(MPI)p 152 2240 V 14 w(HINTS)p 287 2240 V 16 w(IO)p 346 2240 V 14 w(NODE)p 475 2240 V 15 w(LIST)h Fo(\(arra)o(y)g(of)h(in)o (teger\))f([SAME]:)k Ft(In)c(addition)g(to)f(de\014ning)h(the)g(strip-) 189 2296 y(ing)e(factor,)f(one)h(ma)o(y)f(w)o(an)o(t)g(to)g(giv)o(e)h (the)g(explicit)i(list)f(of)e(I/O)h(no)q(des)h(\(resp.)j(storage)14 b(devices\))189 2353 y(that)g(should)i(b)q(e)g(used.)21 b(This)15 b(hin)o(t)h(is)g(relev)m(an)o(t)g(only)f(when)h(the)f(\014le) i(is)e(created.)75 2443 y Fg(MPI)p 152 2443 V 14 w(HINTS)p 287 2443 V 16 w(NB)p 360 2443 V 15 w(PROC)h Fo(\(in)o(teger\))i ([SAME]:)j Ft(This)e(sp)q(eci\014es)h(the)f(n)o(um)o(b)q(er)g(of)f (parallel)j(pro)q(cesses)189 2500 y(that)15 b(will)i(t)o(ypically)g(b)q (e)f(assigned)g(to)f(run)h(programs)f(that)f(access)i(this)g(\014le,)h (and)e(can)h(b)q(e)g(used)189 2556 y(at)e(\014le)i(creation.)75 2647 y Fg(MPI)p 152 2647 V 14 w(HINTS)p 287 2647 V 16 w(A)o(CCESS)p 455 2647 V 13 w(STYLE)i Fo(\(in)o(teger\):)23 b Ft(This)16 b(hin)o(t)h(sp)q(eci\014es)h(the)e(manner)g(in)g(whic)o(h) h(the)f(\014le)h(will)189 2704 y(b)q(e)f(accessed)f(un)o(til)i(the)e (\014le)h(is)g(closed)g(or)f(un)o(til)h(the)f(program)f(terminates.) 1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(17)189 49 y(The)15 b(hin)o(t)h(v)m(alue)g(is)g(an)f(in)o(teger)g (equal)h(to)f(one)g(of)g(the)g(constan)o(ts:)230 159 y Fg(MPI)p 307 159 13 2 v 14 w(HINTS)p 442 159 V 16 w(READ)p 568 159 V 14 w(ONCE)230 203 y(MPI)p 307 203 V 14 w(HINTS)p 442 203 V 16 w(WRITE)p 589 203 V 15 w(ONCE)230 247 y(MPI)p 307 247 V 14 w(HINTS)p 442 247 V 16 w(READ)p 568 247 V 14 w(MOSTL)m(Y)230 291 y(MPI)p 307 291 V 14 w(HINTS)p 442 291 V 16 w(WRITE)p 589 291 V 15 w(MOSTL)m(Y)189 393 y Ft(P)o(ossibly)i(mo)q(di\014ed)i(b)o(y)e(one)g(of)g(the)g(follo)o (wing)h(constan)o(ts)e(\(via)h(the)g(bit)o(wise)h(OR)g(op)q(erator)e (in)189 449 y(C)f(or)f(simple)j(addition)f(in)g(F)o(OR)l(TRAN\):)230 559 y Fg(MPI)p 307 559 V 14 w(HINTS)p 442 559 V 16 w(SEQUENTIAL)230 603 y(MPI)p 307 603 V 14 w(HINTS)p 442 603 V 16 w(REVERSE)p 638 603 V 14 w(SEQUENTIAL)230 647 y(MPI)p 307 647 V 14 w(HINTS)p 442 647 V 16 w(RANDOM)75 767 y(MPI)p 152 767 V 14 w(HINTS)p 287 767 V 16 w(COLLECTIVE)p 550 767 V 13 w(BUFFERING)g Fo(\(logical\))k([SAME]:)h Ft(This)11 b(hin)o(t)f(sp)q(eci\014es)i(whether)f(the)f(ap-)189 824 y(plication)i(will)h(b)q(ene\014t)g(from)d(collectiv)o(e)j (bu\013ering)f(\(an)f(optimization)h(p)q(erformed)f(on)g(collectiv)o(e) 189 880 y(accesses)16 b(whic)o(h)g(coalesces)h(small)f(requests)g(in)o (to)g(large)g(disk)g(accesses\).)22 b(Collectiv)o(e)17 b(bu\013ering)189 937 y(parameters)d(are)h(further)g(directed)h(via)g Fg(MPI)p 979 937 V 14 w(HINTS)p 1114 937 V 16 w(CB)p 1185 937 V 14 w(XXX)e Ft(hin)o(ts.)75 1031 y Fg(MPI)p 152 1031 V 14 w(HINTS)p 287 1031 V 16 w(CB)p 358 1031 V 14 w(NODES)j Fo(\(in)o(teger\))h([SAME]:)j Ft(The)14 b(n)o(um)o(b)q(er)f(of)g(target)g(no)q(des)g(used)h(for)f(collectiv)o (e)189 1087 y(bu\013ering.)20 b(All)d(\014le)f(accesses)f(are)g(p)q (erformed)h(b)o(y)f(the)g(target)f(no)q(des.)75 1181 y Fg(MPI)p 152 1181 V 14 w(HINTS)p 287 1181 V 16 w(CB)p 358 1181 V 14 w(BLOCK)p 509 1181 V 14 w(SIZE)j Fo(\(o\013set\))h ([SAME]:)j Ft(The)d(blo)q(c)o(k)h(size)g(used)f(for)f(\014le)i(access.) 29 b(T)l(arget)189 1237 y(no)q(des)17 b(access)f(data)g(in)h(c)o(h)o (unks)g(of)e(this)i(size.)24 b(The)17 b(c)o(h)o(unks)g(are)f (distributed)i(to)d(target)g(no)q(des)189 1294 y(in)h(a)f(round-robin)h (\(CYCLIC\))f(pattern.)75 1388 y Fg(MPI)p 152 1388 V 14 w(HINTS)p 287 1388 V 16 w(CB)p 358 1388 V 14 w(BUF)p 453 1388 V 14 w(SIZE)i Fo(\(o\013set\))h([SAME]:)i Ft(The)11 b(total)e(bu\013er)h(space)h(used)g(for)e(collectiv)o(e)j(bu\013er-)189 1444 y(ing)j(on)h(eac)o(h)f(target)f(no)q(de;)h(usually)i(a)e(m)o (ultiple)i(of)d Fg(MPI)p 1186 1444 V 15 w(HINTS)p 1322 1444 V 15 w(CB)p 1392 1444 V 15 w(BLOCK)p 1544 1444 V 14 w(SIZE)p Ft(.)166 1550 y(Sp)q(eci\014c)19 b(implemen)o(tations)e (are)f(free)h(to)f(in)o(terpret)g(the)h(hin)o(ts)g(in)g(sligh)o(tly)g (di\013eren)o(t)g(w)o(a)o(ys.)23 b(F)l(or)75 1607 y(example,)17 b(the)f(follo)o(wing)h(table)f(outlines)i(p)q(ossible)g(in)o (terpretations)e(for)f(an)h(implemen)o(tation)i(based)75 1663 y(on)d(the)g(IBM)h(AIX)g(P)o(arallel)g(I/O)f(File)i(System)e([1)o (,)g(2]:)p 483 1718 985 2 v 482 1774 2 57 v 507 1757 a Fk(hint)i(name)p 1116 1774 V 433 w(interpr)n(etation)p 1467 1774 V 483 1776 985 2 v 482 1832 2 57 v 507 1815 a Fg(MPI)p 584 1815 13 2 v 15 w(HINTS)p 720 1815 V 15 w(STRIPING)p 921 1815 V 14 w(UNIT)p 1116 1832 2 57 v 111 w Ft(BSU)e(size)p 1467 1832 V 482 1889 V 507 1872 a Fg(MPI)p 584 1872 13 2 v 15 w(HINTS)p 720 1872 V 15 w(STRIPING)p 921 1872 V 14 w(F)m(A)o(CTOR)p 1116 1889 2 57 v 48 w Ft(n)o(um)o(b)q(er)g(of)g(cells)p 1467 1889 V 482 1945 V 507 1928 a Fg(MPI)p 584 1928 13 2 v 15 w(HINTS)p 720 1928 V 15 w(IO)p 778 1928 V 14 w(NODE)p 907 1928 V 15 w(LIST)p 1116 1945 2 57 v 137 w Ft(base)g(no)q(de)p 1467 1945 V 483 1947 985 2 v 75 2052 a Fl(10.2.7)49 b(Deleting)17 b(a)g(\014le)f(\(Indep)q(endent\))75 2232 y Fq(MPI)p 160 2232 14 2 v 16 w(DELETE\(\014lename\))117 2310 y Ff(IN)155 b Fq(\014lename)430 b Ff(Name)13 b(of)g(the)i(\014le)f(to)f (b)q(e)i(deleted)g(\(string\))75 2434 y Fp(int)47 b(MPI)p 269 2434 15 2 v 17 w(Delete\(char)22 b(*filename\))75 2521 y(MPI)p 150 2521 V 17 w(DELETE\(FILENAME,)f(IERROR\))170 2577 y(CHARACTER)i(FILENAME\(*\))170 2633 y(INTEGER)g(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Ft(18)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(DELETE)15 b Ft(deletes)h(a)e(\014le.)21 b(If)16 b(the)f(\014le)h(exists)f(it)h (is)f(remo)o(v)o(ed.)20 b(If)15 b(there)g(are)g(other)g(pro)q(cesses)75 106 y(curren)o(tly)k(accessing)h(the)f(\014le,)i(the)e(status)f(of)h (the)g(\014le)h(and)f(the)g(b)q(eha)o(vior)g(of)g(future)g(accesses)g (are)75 162 y(implemen)o(tation)d(dep)q(enden)o(t.)21 b(If)14 b(the)h(\014le)g(do)q(es)g(not)f(exist,)g Fq(MPI)p 1216 162 V 16 w(DELETE)g Ft(returns)g(a)g(w)o(arning)h(error)75 219 y(co)q(de.)189 325 y Fk(A)n(dvic)n(e)i(to)i(implementors.)54 b Ft(If)17 b(the)h(\014le)h(to)d(b)q(e)j(deleted)g(is)f(op)q(ened)g(b)o (y)g(other)f(pro)q(cesses,)h(\014le)189 381 y(data)e(ma)o(y)h(still)i (b)q(e)f(accessible)h(b)o(y)e(these)h(pro)q(cesses)g(un)o(til)g(they)g (close)g(the)f(\014le)i(or)d(un)o(til)j(they)189 438 y(exit.)h(\()p Fk(End)c(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))75 560 y Fl(10.2.8)49 b(Resizing)18 b(a)f(\014le)f (\(Collective\))75 740 y Fq(MPI)p 160 740 V 16 w(RESIZE\(MPI)p 425 740 V 16 w(File)f(fh,)g(MPI)p 671 740 V 15 w(O\013set)i(size\))117 817 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i (handle)g(\(handle\))117 892 y(IN)155 b Fq(size)519 b Ff([SAME])18 b(Size)h(whic)o(h)f(the)h(\014le)f(is)g(to)g(b)q(e)h (truncated)h(at)e(or)905 949 y(expanded)d(to)f(\(nonnegativ)o(e)f (o\013set\))75 1073 y Fp(int)23 b(MPI)p 245 1073 15 2 v 17 w(Resize\(MPI)p 502 1073 V 16 w(File)g(fh,)h(MPI)p 805 1073 V 17 w(Offset)f(size\))75 1160 y(MPI)p 150 1160 V 17 w(RESIZE\(FH,)f(SIZE,)h(IERROR\))170 1216 y(INTEGER)g(FH,)h (IERROR)170 1272 y(INTEGER*8)f(SIZE)166 1359 y Fq(MPI)p 251 1359 14 2 v 16 w(RESIZE)18 b Ft(resizes)h(the)f(\014le)g(asso)q (ciated)g(with)h(the)f(\014le)h(handle)g Fq(fh)p Ft(.)28 b(If)18 b Fq(size)g Ft(is)g(smaller)h(than)75 1415 y(the)14 b(curren)o(t)g(\014le)h(size,)g(the)f(\014le)h(is)f(truncated)g(at)f (the)h(p)q(osition)h(de\014ned)h(b)o(y)e Fq(size)g Ft(\(from)f(the)h(b) q(eginning)75 1472 y(of)f(the)g(\014le)h(and)f(measured)g(in)h(b)o (ytes\).)19 b(File)14 b(blo)q(c)o(ks)f(lo)q(cated)h(b)q(ey)o(ond)g (that)e(p)q(osition)i(are)f(deallo)q(cated.)75 1528 y(If)19 b Fq(size)h Ft(is)f(larger)g(than)g(the)g(curren)o(t)g(\014le)h(size,)h (additional)f(\014le)g(blo)q(c)o(ks)g(are)f(allo)q(cated)h(and)f(the)g (\014le)75 1585 y(size)14 b(b)q(ecomes)h Fq(size)p Ft(.)k(All)c(pro)q (cesses)f(in)g(the)g(comm)o(unicator)f(group)g(m)o(ust)g(call)h Fq(MPI)p 1534 1585 V 16 w(RESIZE)g Ft(with)g(the)75 1641 y(same)h Fq(size)p Ft(.)166 1780 y Fe(Discussion:)j Ff(There)e(are)g(t) o(w)o(o)e(issues)j(asso)q(ciated)f(with)e(\014le)h(blo)q(c)o(k)g (preallo)q(cation:)20 b(it)15 b(ma)o(y)e(b)q(e)j(slo)o(w)e(on)75 1837 y(systems)j(without)g(direct)g(\014lesystem)g(supp)q(ort,)h(and)f (w)o(e)g(m)o(ust)f(b)q(e)i(careful)f(to)f(de\014ne)i(what)f(happ)q(ens) h(if)e(not)75 1893 y(enough)e(space)h(is)f(a)o(v)n(ailable.)75 2098 y Fl(10.2.9)49 b(File)17 b(Sync)f(\(Collective\))75 2278 y Fq(MPI)p 160 2278 V 16 w(FILE)p 267 2278 V 15 w(SYNC\(fh\))117 2355 y Ff(IN)155 b Fq(fh)550 b Ff([SAME]V)m(alid)12 b(\014le)i(handle)g(\(handle\))75 2480 y Fp(int)23 b(MPI)p 245 2480 15 2 v 17 w(File)p 358 2480 V 17 w(sync\(MPI)p 567 2480 V 16 w(File)g(fh\))75 2566 y(MPI)p 150 2566 V 17 w(FILE)p 263 2566 V 16 w(SYNC\(FH,)g(IERROR\))170 2622 y(INTEGER)g(FH,)h(IERROR)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fn(10.3.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (EXPLICIT)h(OFFSETS)697 b Ft(19)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(FILE)p 358 49 V 15 w(SYNC)19 b Ft(causes)e(the)h(con)o (ten)o(ts)f(of)g(the)h(\014le)h(referenced)f(b)o(y)g Fq(fh)g Ft(to)e(b)q(e)j(\015ushed)f(to)f(p)q(er-)75 106 y(manen)o(t)g(storage.)26 b(All)18 b(pro)q(cesses)g(in)g(the)g(comm)o (unicator)f(group)g(asso)q(ciated)h(with)g(the)f(\014le)i(handle)75 162 y Fq(fh)c Ft(m)o(ust)e(call)j Fq(MPI)p 406 162 V 16 w(FILE)p 513 162 V 15 w(SYNC)p Ft(.)f(The)g Fq(MPI)p 847 162 V 16 w(FILE)p 954 162 V 15 w(SYNC)g Ft(call)h(returns)e Fk(after)h Ft(all)g(data)f(written,)g(since)75 219 y(the)h(\014le)i(w)o (as)d(op)q(ened,)i(has)f(b)q(een)h(\015ushed)g(to)f(p)q(ermanen)o(t)g (storage.)189 325 y Fk(A)n(dvic)n(e)j(to)h(users.)58 b Fq(MPI)p 653 325 V 15 w(FILE)p 759 325 V 16 w(SYNC)19 b Ft(guaran)o(tees)e(that)h(all)h Fk(c)n(omplete)n(d)f Ft(I/O)g(requests)g(ha)o(v)o(e)189 381 y(b)q(een)e(\015ushed)h(to)e(p)q (ermanen)o(t)g(storage.)20 b(P)o(ending)c(non)o(blo)q(c)o(king)h(I/O)f (requests)f(that)g(ha)o(v)o(e)g(not)189 438 y(completed)h(are)f(not)g (guaran)o(teed)f(to)h(b)q(e)h(\015ushed.)21 b(\()p Fk(End)15 b(of)i(advic)n(e)f(to)g(users.)p Ft(\))166 627 y Fe(Discussion:)d Ff(Do)q(es)d(this)g(imply)e(w)o(eak)h(consistency)i(b)q(et)o(w)o(een)h (applications)c(at)i Fg(MPI)p 1521 627 13 2 v 14 w(FILE)p 1618 627 V 15 w(SYNC)f Ff(b)q(ound-)75 683 y(aries?)18 b(Should)c(it)f(b)q(e)i(required?)75 909 y Fm(10.3)60 b(Data)19 b(Access)h(With)g(Explicit)e(O\013sets)75 1012 y Fl(10.3.1)49 b(Blo)q(cking)18 b(Read)75 1193 y Fq(MPI)p 160 1193 14 2 v 16 w(READ\(fh,)d(o\013set,)g(bu\013,)h(buft)o(yp)q(e,)g (bufcount,)h(status\))117 1270 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1345 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(nonnegativ)o(e)g (o\013set\))117 1420 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 1495 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat) o(yp)q(e)f(\(handle\))117 1570 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(in)o (teger\))117 1645 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1770 y Fp(int)23 b(MPI)p 245 1770 15 2 v 17 w(Read\(MPI)p 454 1770 V 16 w(File)h(fh,)f(MPI)p 757 1770 V 17 w(Offset)g(offset,)g(void)g(*buff,)393 1826 y(MPI)p 468 1826 V 17 w(Datatype)g(buftype,)f(int)i(bufcount,)f (MPI)p 1321 1826 V 16 w(Status)g(*status\))75 1913 y(MPI)p 150 1913 V 17 w(READ\(FH,)g(OFFSET,)f(BUFF,)i(BUFTYPE,)e(BUFCOUNT,)h (STATUS,)g(IERROR\))170 1969 y()g(BUFF\(*\))170 2026 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2026 V 16 w(STATUS)p 1313 2026 V 16 w(SIZE\),)h(IERROR)170 2082 y(INTEGER*8)g(OFFSET)166 2169 y Fq(MPI)p 251 2169 14 2 v 16 w(READ)c Ft(attempts)f(to)h(read)g(from)f(the)h(\014le)i (asso)q(ciated)e(with)g Fq(fh)h Ft(\(at)e(the)h Fq(o\013set)g Ft(p)q(osition\))75 2225 y(a)d(total)g(n)o(um)o(b)q(er)i(of)e Fq(bufcount)j Ft(data)d(items)h(ha)o(ving)g Fq(buft)o(yp)q(e)h Ft(datat)o(yp)q(e)e(in)o(to)h(the)g(user's)f(bu\013er)h Fq(bu\013)p Ft(.)75 2281 y(The)f(data)f(is)i(tak)o(en)e(out)h(of)f (those)h(parts)f(of)h(the)g(\014le)g(sp)q(eci\014ed)i(b)o(y)e(\014let)o (yp)q(e.)23 b Fq(MPI)p 1533 2281 V 16 w(READ)17 b Ft(stores)e(the)75 2338 y(n)o(um)o(b)q(er)h(of)e Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f (actually)g(read)f(in)h Fq(status)p Ft(.)75 2489 y Fq(MPI)p 160 2489 V 16 w(READ)p 295 2489 V 17 w(ALL\(fh,)e(o\013set,)h(bu\013,)g (buft)o(yp)q(e,)i(bufcount,)g(status\))75 2593 y Fp(int)23 b(MPI)p 245 2593 15 2 v 17 w(Read)p 358 2593 V 17 w(all\(MPI)p 543 2593 V 16 w(File)g(fh,)h(MPI)p 846 2593 V 17 w(Offset)f(offset,)g (void)g(*buff,)393 2649 y(MPI)p 468 2649 V 17 w(Datatype)g(buftype,)f (int)i(bufcount,)f(MPI)p 1321 2649 V 16 w(Status)g(*status\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Ft(20)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fp(MPI)p 150 49 15 2 v 17 w(READ)p 263 49 V 16 w(ALL\(FH,)23 b(OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (STATUS,)g(IERROR\))170 106 y()g(BUFF\(*\))170 162 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 162 V 16 w(STATUS)p 1313 162 V 16 w(SIZE\),)h(IERROR)170 219 y(INTEGER*8)g(OFFSET)166 305 y Fq(MPI)p 251 305 14 2 v 16 w(READ)p 386 305 V 17 w(ALL)10 b Ft(is)h(a)g(collectiv)o(e)i(v)o (ersion)e(of)f(the)h(blo)q(c)o(king)h Fq(MPI)p 1290 305 V 16 w(READ)f Ft(in)o(terface.)19 b(This)11 b(routine)75 362 y(is)i(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 694 362 V 16 w(READ)p Ft(,)g(except)h(that)f(all)h(pro)q(cesses)g(in)h (the)e(comm)o(unicator)g(group)75 418 y(asso)q(ciated)j(with)h(the)f (\014le)h(handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1026 418 V 16 w(READ)p 1161 418 V 17 w(ALL)p Ft(.)75 540 y Fl(10.3.2)49 b(Blo)q(cking)18 b(W)o(rite)75 720 y Fq(MPI)p 160 720 V 16 w(WRITE\(fh,)d(o\013set,)g(bu\013,)g(buft)o(yp) q(e,)i(bufcount,)g(status\))117 797 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 872 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(nonnegativ)o(e)g (o\013set\))117 947 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 1023 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat) o(yp)q(e)f(\(handle\))117 1098 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(in)o (teger\))117 1173 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1297 y Fp(int)47 b(MPI)p 269 1297 15 2 v 17 w(Write\(MPI)p 502 1297 V 16 w(File)23 b(fh,)h(MPI)p 805 1297 V 17 w(Offset)f(offset,)g(void)g(*buff,)393 1354 y(MPI)p 468 1354 V 17 w(Datatype)g(buftype,)f(int)i(bufcount,)f (MPI)p 1321 1354 V 16 w(Status)g(*status\))75 1440 y(MPI)p 150 1440 V 17 w(WRITE\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (STATUS,)g(IERROR\))170 1497 y()g(BUFF\(*\))170 1553 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1553 V 16 w(STATUS)p 1313 1553 V 16 w(SIZE\),)h(IERROR)170 1610 y(INTEGER*8)g(OFFSET)166 1696 y Fq(MPI)p 251 1696 14 2 v 16 w(WRITE)17 b Ft(attempts)f(to)h(write)g(in)o(to)h(the)f (\014le)h(asso)q(ciated)g(with)f Fq(fh)h Ft(\(at)e(the)i Fq(o\013set)f Ft(p)q(osition\))75 1753 y(a)e(total)g(n)o(um)o(b)q(er)h (of)f Fq(bufcount)j Ft(data)d(items)h(ha)o(ving)g Fq(buft)o(yp)q(e)h Ft(datat)o(yp)q(e)e(from)g(the)h(user's)f(bu\013er)h Fq(bu\013)p Ft(.)75 1809 y(The)f(data)f(is)h(written)g(in)o(to)g(those) g(parts)f(of)g(the)h(\014le)h(sp)q(eci\014ed)h(b)o(y)d(\014let)o(yp)q (e.)21 b Fq(MPI)p 1512 1809 V 16 w(WRITE)15 b Ft(stores)f(the)75 1865 y(n)o(um)o(b)q(er)i(of)e Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f (actually)g(written)f(in)h Fq(status)p Ft(.)75 2016 y Fq(MPI)p 160 2016 V 16 w(WRITE)p 319 2016 V 16 w(ALL\(fh,)f(o\013set,)g (bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g(status\))75 2120 y Fp(int)47 b(MPI)p 269 2120 15 2 v 17 w(Write)p 406 2120 V 17 w(all\(MPI)p 591 2120 V 16 w(File)23 b(fh,)h(MPI)p 894 2120 V 16 w(Offset)f(offset,)g(void)h(*buff,)393 2177 y(MPI)p 468 2177 V 17 w(Datatype)f(buftype,)f(int)i(bufcount,)f (MPI)p 1321 2177 V 16 w(Status)g(*status\))75 2263 y(MPI)p 150 2263 V 17 w(WRITE)p 287 2263 V 16 w(ALL\(FH,)g(OFFSET,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)g(STATUS,)g(IERROR\))170 2319 y()g (BUFF\(*\))170 2376 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (STATUS\(MPI)p 1153 2376 V 16 w(STATUS)p 1313 2376 V 16 w(SIZE\),)h(IERROR)170 2432 y(INTEGER*8)g(OFFSET)166 2519 y Fq(MPI)p 251 2519 14 2 v 16 w(WRITE)p 410 2519 V 16 w(ALL)12 b Ft(is)g(a)g(collectiv)o(e)i(v)o(ersion)f(of)e(the)i (blo)q(c)o(king)g Fq(MPI)p 1324 2519 V 16 w(WRITE)f Ft(in)o(terface.)19 b(This)13 b(rou-)75 2575 y(tine)h(is)h(seman)o(tically)f(iden)o(tical)i (to)d Fq(MPI)p 788 2575 V 16 w(WRITE)p Ft(,)g(except)h(that)f(all)i (pro)q(cesses)f(in)h(the)e(comm)o(unicator)75 2632 y(group)i(asso)q (ciated)g(with)h(the)f(\014le)h(handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1155 2632 V 16 w(WRITE)p 1314 2632 V 16 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fn(10.3.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (EXPLICIT)h(OFFSETS)697 b Ft(21)75 49 y Fl(10.3.3)49 b(Nonblo)q(cking)19 b(Read)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IREAD\(fh,)c(o\013set,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(O\013set)h(\(nonnegativ)o(e)f (o\013set\))117 457 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 532 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 607 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 682 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f(\(handle\))75 807 y Fp(int)23 b(MPI)p 245 807 15 2 v 17 w(Iread\(MPI)p 478 807 V 16 w(File)g(fh,)h(MPI)p 781 807 V 17 w(Offset)f(offset,)g(void)g (*buff,)393 863 y(MPI)p 468 863 V 17 w(Datatype)g(buftype,)f(int)i (bufcount,)f(MPI)p 1321 863 V 16 w(Request)g(*request\))75 950 y(MPI)p 150 950 V 17 w(IREAD\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)g(REQUEST,)g(IERROR\))170 1006 y()g(BUFF\(*\))170 1063 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)170 1119 y(INTEGER*8)g(OFFSET)166 1206 y Fq(MPI)p 251 1206 14 2 v 16 w(IREAD)12 b Ft(is)g(a)g(non)o(blo)q(c)o(king)h(v)o(ersion)f (of)g(the)g Fq(MPI)p 1094 1206 V 16 w(READ)g Ft(in)o(terface.)19 b Fq(MPI)p 1524 1206 V 16 w(IREAD)12 b Ft(asso)q(ciates)75 1262 y(a)f(request)g(handle)h Fq(request)h Ft(with)f(the)f(I/O)h (request.)18 b(The)12 b(request)f(handle)h(can)g(b)q(e)f(used)h(later)f (to)g(query)75 1319 y(the)k(status)f(of)g(the)h(read)g(request)g(\(via) f Fq(MPI)p 851 1319 V 16 w(TEST)p Ft(\),)g(or)h(w)o(ait)f(for)g(its)h (completion)h(\(via)f Fq(MPI)p 1719 1319 V 16 w(W)l(AIT)p Ft(\).)166 1375 y(The)k(non)o(blo)q(c)o(king)i(read)e(call)h(indicates) g(that)e(the)i(system)e(can)h(start)f(to)h(read)f(data)h(in)o(to)g(the) 75 1431 y(supplied)e(bu\013er.)j(The)15 b(user)g(should)h(not)e(access) h(an)o(y)g(part)f(of)g(the)h(bu\013er)g(after)f(a)h(non)o(blo)q(c)o (king)h(read)75 1488 y(is)h(called,)h(un)o(til)f(the)f(read)h (completes)g(\(as)e(indicated)j(b)o(y)e Fq(MPI)p 1187 1488 V 16 w(TEST)h Ft(or)f Fq(MPI)p 1473 1488 V 15 w(W)l(AIT)p Ft(\).)g Fq(MPI)p 1730 1488 V 16 w(IREAD)75 1544 y Ft(attempts)e(to)g (read)h(from)g(the)g(\014le)h(asso)q(ciated)f(with)h Fq(fh)f Ft(\(at)f(the)h Fq(o\013set)g Ft(p)q(osition\),)h(a)f(total)f (n)o(um)o(b)q(er)h(of)75 1601 y Fq(bufcount)i Ft(data)e(items)g(ha)o (ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)h(user's)f (bu\013er)g Fq(bu\013)p Ft(.)20 b(The)c(n)o(um)o(b)q(er)f(of)g Fq(buft)o(yp)q(e)75 1657 y Ft(elemen)o(ts)j(actually)h(read)e(can)h(b)q (e)g(extracted)f(from)g(the)h Fq(MPI)p 1171 1657 V 16 w(TEST)f Ft(or)g Fq(MPI)p 1458 1657 V 16 w(W)l(AIT)h Ft(return)f(status,)75 1714 y(using)f(the)f(routines)h Fq(MPI)p 532 1714 V 16 w(GET)p 636 1714 V 16 w(COUNT)g Ft(or)f Fq(MPI)p 962 1714 V 16 w(GET)p 1066 1714 V 17 w(ELEMENTS)p Ft(.)75 1865 y Fq(MPI)p 160 1865 V 16 w(IREAD)p 308 1865 V 16 w(ALL\(fh,)g(o\013set,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 1968 y Fp(int)23 b(MPI)p 245 1968 15 2 v 17 w(Iread)p 382 1968 V 17 w(all\(MPI)p 567 1968 V 16 w(File)g(fh,)h(MPI)p 870 1968 V 16 w(Offset)g(offset,)e(void) i(*buff,)393 2025 y(MPI)p 468 2025 V 17 w(Datatype)f(buftype,)f(int)i (bufcount,)f(MPI)p 1321 2025 V 16 w(Request)g(*request\))75 2111 y(MPI)p 150 2111 V 17 w(IREAD)p 287 2111 V 16 w(ALL\(FH,)g (OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(REQUEST,)f(IERROR\))170 2168 y()h(BUFF\(*\))170 2224 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)170 2281 y(INTEGER*8)g(OFFSET)166 2367 y Fq(MPI)p 251 2367 14 2 v 16 w(IREAD)p 399 2367 V 16 w(ALL)16 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)f(of)f(the)g (non)o(blo)q(c)o(king)i Fq(MPI)p 1419 2367 V 16 w(IREAD)f Ft(in)o(terface.)23 b(This)75 2424 y(routine)17 b(is)g(seman)o(tically) g(iden)o(tical)h(to)e Fq(MPI)p 867 2424 V 16 w(IREAD)p Ft(,)g(except)g(that)g(all)h(pro)q(cesses)g(in)g(the)f(comm)o(uni-)75 2480 y(cator)e(group)h(asso)q(ciated)h(with)f(the)g(\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)j Fq(MPI)p 1272 2480 V 15 w(IREAD)p 1419 2480 V 17 w(ALL)p Ft(.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 -100 a Ft(22)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.3.4)49 b(Nonblo)q(cking)19 b(W)o(rite)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IWRITE\(fh,)14 b(o\013set,)h (bu\013,)h(buft)o(yp)q(e,)g(bufcount,)h(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 382 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(O\013set)h(\(nonnegativ)o(e)f(o\013set\))117 457 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 532 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 607 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q (e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 682 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f (\(handle\))75 807 y Fp(int)47 b(MPI)p 269 807 15 2 v 17 w(Iwrite\(MPI)p 526 807 V 16 w(File)23 b(fh,)h(MPI)p 829 807 V 16 w(Offset)f(offset,)g(void)h(*buff,)393 863 y(MPI)p 468 863 V 17 w(Datatype)f(buftype,)f(int)i(bufcount,)f(MPI)p 1321 863 V 16 w(Request)g(*request\))75 950 y(MPI)p 150 950 V 17 w(IWRITE\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (REQUEST,)g(IERROR\))170 1006 y()g(BUFF\(*\))170 1063 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)170 1119 y(INTEGER*8)g(OFFSET)166 1206 y Fq(MPI)p 251 1206 14 2 v 16 w(IWRITE)14 b Ft(is)i(a)e(non)o(blo)q(c)o(king)i(v)o(ersion)f (of)g(the)g Fq(MPI)p 1138 1206 V 15 w(WRITE)g Ft(in)o(terface.)20 b Fq(MPI)p 1595 1206 V 16 w(IWRITE)15 b Ft(asso-)75 1262 y(ciates)g(a)f(request)h(handle)h Fq(request)h Ft(with)e(the)g(I/O)g (request.)20 b(The)15 b(request)f(handle)i(can)f(b)q(e)h(used)f(later) 75 1319 y(to)j(query)h(the)g(status)f(of)g(the)h(write)g(request)g (\(via)f Fq(MPI)p 1083 1319 V 16 w(TEST)p Ft(\),)g(or)h(w)o(ait)f(for)g (its)h(completion)h(\(via)75 1375 y Fq(MPI)p 160 1375 V 16 w(W)l(AIT)p Ft(\).)166 1431 y(The)c(non)o(blo)q(c)o(king)h(write)f (call)h(indicates)g(that)e(the)h(system)f(can)h(start)f(to)g(write)h (data)f(from)g(the)75 1488 y(supplied)e(bu\013er.)19 b(The)11 b(user)g(should)h(not)f(access)g(an)o(y)f(part)h(of)f(the)h (bu\013er)g(after)g(the)g(non)o(blo)q(c)o(king)h(write)75 1544 y(is)i(called,)g(un)o(til)h(the)e(write)g(completes)h(\(as)e (indicated)j(b)o(y)e Fq(MPI)p 1174 1544 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1452 1544 V 16 w(W)l(AIT)p Ft(\).)h Fq(MPI)p 1707 1544 V 15 w(IWRITE)75 1601 y Ft(attempts)h(to)g(write)i (in)o(to)f(the)g(\014le)h(asso)q(ciated)f(with)h Fq(fh)f Ft(\(at)f(the)h Fq(o\013set)g Ft(p)q(osition\),)h(a)f(total)f(n)o(um)o (b)q(er)h(of)75 1657 y Fq(bufcount)h Ft(data)e(items)g(ha)o(ving)g Fq(buft)o(yp)q(e)i Ft(t)o(yp)q(e)e(from)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)20 b(The)14 b(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)75 1714 y Ft(elemen)o(ts)f(actually)h(written)f(can)f (b)q(e)i(extracted)e(from)g(the)h Fq(MPI)p 1190 1714 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1468 1714 V 16 w(W)l(AIT)h Ft(return)f(status,)75 1770 y(using)k(the)f(routines)h Fq(MPI)p 532 1770 V 16 w(GET)p 636 1770 V 16 w(COUNT)g Ft(or)f Fq(MPI)p 962 1770 V 16 w(GET)p 1066 1770 V 17 w(ELEMENTS)p Ft(.)75 1921 y Fq(MPI)p 160 1921 V 16 w(IWRITE)p 332 1921 V 16 w(ALL\(fh,)f(o\013set,)h(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 2025 y Fp(int)23 b(MPI)p 245 2025 15 2 v 17 w(Iwrite)p 406 2025 V 17 w(all\(MPI)p 591 2025 V 16 w(File)g(fh,)h(MPI)p 894 2025 V 16 w(Offset)f(offset,)g (void)h(*buff,)393 2081 y(MPI)p 468 2081 V 17 w(Datatype)f(buftype,)f (int)i(bufcount,)f(MPI)p 1321 2081 V 16 w(Request)g(*request\))75 2168 y(MPI)p 150 2168 V 17 w(IWRITE)p 311 2168 V 16 w(ALL\(FH,)g (OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(REQUEST,)f(IERROR\))170 2224 y()h(BUFF\(*\))170 2281 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)170 2337 y(INTEGER*8)g(OFFSET)166 2424 y Fq(MPI)p 251 2424 14 2 v 16 w(IWRITE)p 423 2424 V 16 w(ALL)11 b Ft(is)h(a)g(collectiv)o(e)h(v)o(ersion)f(of)f(the)h (non)o(blo)q(c)o(king)h Fq(MPI)p 1405 2424 V 16 w(IWRITE)e Ft(in)o(terface.)19 b(This)75 2480 y(routine)c(is)g(seman)o(tically)g (iden)o(tical)h(to)e Fq(MPI)p 857 2480 V 16 w(IWRITE)p Ft(,)f(except)i(that)f(all)h(pro)q(cesses)g(in)g(the)f(comm)o(uni-)75 2536 y(cator)g(group)h(asso)q(ciated)h(with)f(the)g(\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)j Fq(MPI)p 1272 2536 V 15 w(IWRITE)p 1443 2536 V 16 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 23 24 bop 75 -100 a Fn(10.4.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (INDIVIDUAL)i(FILE)e(POINTERS)487 b Ft(23)75 49 y Fm(10.4)60 b(Data)19 b(Access)h(With)g(Individual)e(File)i(P)n(ointers)75 151 y Ft(The)14 b(system)f(main)o(tains)h(one)f(individual)k(\014le)e (p)q(oin)o(ter)f(p)q(er)g(pro)q(cess)g(p)q(er)g(collectiv)o(e)h Fq(MPI)p 1636 151 14 2 v 16 w(OPEN)p Ft(.)f(The)75 207 y(curren)o(t)k(v)m(alue)i(of)d(this)i(p)q(oin)o(ter)g(can)f(b)q(e)h (used)g(to)e(implicitly)22 b(sp)q(ecify)d(the)f(o\013set)g(of)f(data)h (accesses,)75 264 y(b)o(y)h(using)h(the)f(routines)h(describ)q(ed)h(in) f(this)g(section.)32 b(These)20 b(routines)f(only)h(use)g(and)f(up)q (date)h(the)75 320 y(individual)g(\014le)e(p)q(oin)o(ters)g(main)o (tained)g(b)o(y)f(the)g(system.)25 b(The)17 b(shared)g(\014le)h(p)q (oin)o(ter)f(is)h(not)f(used)g(nor)75 377 y(up)q(dated.)166 433 y(The)h(individual)j(\014le)e(p)q(oin)o(ter)f(routines)g(ha)o(v)o (e)g(the)g(same)f(seman)o(tics)h(as)g(the)f(data)h(access)g(with)75 489 y(explicit)f(o\013set)e(routines)g(describ)q(ed)i(in)f(Section)g (10.3)e(with)i(the)f(follo)o(wing)h(mo)q(di\014cation:)143 583 y Fh(\017)23 b Ft(the)17 b Fq(o\013set)h Ft(is)f(de\014ned)i(to)e (b)q(e)h(the)f(curren)o(t)h(v)m(alue)g(of)f(the)g(system-main)o(tained) h(individual)j(\014le)189 640 y(p)q(oin)o(ter.)75 734 y(After)15 b(an)g(individual)k(\014le)d(p)q(oin)o(ter)g(op)q(eration)f (is)h(initiated,)g(the)g(individual)i(\014le)f(p)q(oin)o(ter)e(is)h(up) q(dated)75 790 y(to)i(p)q(oin)o(t)h(to)f(the)h(next)g(data)f(item)h (after)f(the)g(last)h(one)g(that)f(will)i(b)q(e)f(actually)h(accessed,) f(p)q(ossibly)75 846 y(accoun)o(ting)d(for)e(an)h(end)h(of)f(\014le)h (o)q(ccurrence.)75 968 y Fl(10.4.1)49 b(Blo)q(cking)18 b(Read)75 1148 y Fq(MPI)p 160 1148 V 16 w(READ)p 295 1148 V 17 w(NEXT\(fh,)d(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (status\))117 1226 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1301 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 1376 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1451 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 1526 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1651 y Fp(int)23 b(MPI)p 245 1651 15 2 v 17 w(Read)p 358 1651 V 17 w(next\(MPI)p 567 1651 V 16 w(File)g(fh,)h(void)f(*buff,) g(MPI)p 1156 1651 V 17 w(Datatype)g(buftype,)393 1707 y(int)h(bufcount,)e(MPI)p 802 1707 V 17 w(Status)h(*status\))75 1794 y(MPI)p 150 1794 V 17 w(READ)p 263 1794 V 16 w(NEXT\(FH,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)g(STATUS,)g(IERROR\))170 1850 y()g (BUFF\(*\))170 1906 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (STATUS\(MPI)p 1153 1906 V 16 w(STATUS)p 1313 1906 V 16 w(SIZE\),)h(IERROR)166 1993 y Fq(MPI)p 251 1993 14 2 v 16 w(READ)p 386 1993 V 17 w(NEXT)14 b Ft(is)g(seman)o(tically)g (iden)o(tical)h(to)e Fq(MPI)p 1156 1993 V 16 w(READ)p Ft(,)g(with)h Fq(o\013set)g Ft(set)f(to)g(the)g(curren)o(t)75 2049 y(individual)21 b(\014le)e(p)q(oin)o(ter)g(p)q(osition)g(main)o (tained)g(b)o(y)f(the)g(system.)28 b(The)18 b(\014le)h(p)q(oin)o(ter)g (is)f(up)q(dated)h(b)o(y)75 2106 y(the)c(amoun)o(t)g(of)f(data)h (requested.)75 2257 y Fq(MPI)p 160 2257 V 16 w(READ)p 295 2257 V 17 w(NEXT)p 432 2257 V 17 w(ALL\(fh,)f(bu\013,)h(buft)o(yp)q (e,)i(bufcount,)g(status\))75 2360 y Fp(int)23 b(MPI)p 245 2360 15 2 v 17 w(Read)p 358 2360 V 17 w(next)p 471 2360 V 16 w(all\(MPI)p 655 2360 V 17 w(File)g(fh,)g(void)h(*buff,)f (MPI)p 1245 2360 V 17 w(Datatype)f(buftype,)393 2417 y(int)i(bufcount,)e(MPI)p 802 2417 V 17 w(Status)h(*status\))75 2503 y(MPI)p 150 2503 V 17 w(READ)p 263 2503 V 16 w(NEXT)p 375 2503 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(STATUS,)f (IERROR\))170 2560 y()h(BUFF\(*\))170 2616 y(INTEGER)g(FH,)h (BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2616 V 16 w(STATUS)p 1313 2616 V 16 w(SIZE\),)h(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 24 25 bop 75 -100 a Ft(24)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(READ)p 386 49 V 17 w(NEXT)p 523 49 V 17 w(ALL)17 b Ft(is)h(a)f(collectiv)o(e)i(v)o (ersion)f(of)f(the)h(blo)q(c)o(king)h Fq(MPI)p 1481 49 V 15 w(READ)p 1615 49 V 17 w(NEXT)g Ft(in)o(ter-)75 106 y(face.)g(This)12 b(routine)h(is)f(seman)o(tically)i(iden)o(tical)g(to) d Fq(MPI)p 1055 106 V 16 w(READ)p 1190 106 V 17 w(NEXT)p Ft(,)h(except)g(that)g(all)h(pro)q(cesses)f(in)75 162 y(the)g(comm)o(unicator)g(group)g(asso)q(ciated)g(with)h(the)f(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1497 162 V 16 w(READ)p 1632 162 V 17 w(NEXT)p 1769 162 V 17 w(ALL)p Ft(.)75 284 y Fl(10.4.2)49 b(Blo)q(cking)18 b(W)o(rite)75 464 y Fq(MPI)p 160 464 V 16 w(WRITE)p 319 464 V 16 w(NEXT\(fh,)e (bu\013,)f(buft)o(yp)q(e,)i(bufcount,)f(status\))117 541 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 617 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 692 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 767 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 842 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 966 y Fp(int)23 b(MPI)p 245 966 15 2 v 17 w(Write)p 382 966 V 17 w(next\(MPI)p 591 966 V 16 w(File)g(fh,)h(void)f(*buff,)g(MPI)p 1180 966 V 17 w(Datatype)f(buftype,)393 1023 y(int)i(bufcount,)e(MPI)p 802 1023 V 17 w(Status)h(*status\))75 1109 y(MPI)p 150 1109 V 17 w(WRITE)p 287 1109 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)g(STATUS,)g(IERROR\))170 1166 y()g(BUFF\(*\))170 1222 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1222 V 16 w(STATUS)p 1313 1222 V 16 w(SIZE\),)h(IERROR)166 1309 y Fq(MPI)p 251 1309 14 2 v 16 w(WRITE)p 410 1309 V 16 w(NEXT)16 b Ft(is)f(seman)o(tically)h(iden)o(tical)h(to)d Fq(MPI)p 1187 1309 V 16 w(WRITE)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)g(the)g(cur-)75 1365 y(ren)o(t)h(individual)j(\014le)e(p)q (oin)o(ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)21 b(The)16 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)75 1422 y(b)o(y)f(the)g(amoun)o(t)g(of)g(data)f(requested.)75 1573 y Fq(MPI)p 160 1573 V 16 w(WRITE)p 319 1573 V 16 w(NEXT)p 455 1573 V 17 w(ALL\(fh,)g(bu\013,)i(buft)o(yp)q(e,)g (bufcount,)h(status\))75 1676 y Fp(int)47 b(MPI)p 269 1676 15 2 v 17 w(Write)p 406 1676 V 17 w(next)p 519 1676 V 16 w(all\(MPI)p 703 1676 V 16 w(File)24 b(fh,)f(void)h(*buff,)f(MPI)p 1293 1676 V 16 w(Datatype)g(buftype,)393 1733 y(int)h(bufcount,)e(MPI)p 802 1733 V 17 w(Status)h(*status\))75 1819 y(MPI)p 150 1819 V 17 w(WRITE)p 287 1819 V 16 w(NEXT)p 399 1819 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(STATUS,)h(IERROR\))170 1876 y()g(BUFF\(*\))170 1932 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 1932 V 16 w(STATUS)p 1313 1932 V 16 w(SIZE\),)h(IERROR)166 2018 y Fq(MPI)p 251 2018 14 2 v 16 w(WRITE)p 410 2018 V 16 w(NEXT)p 546 2018 V 17 w(ALL)12 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the) h(blo)q(c)o(king)h Fq(MPI)p 1463 2018 V 15 w(WRITE)p 1621 2018 V 17 w(NEXT)f Ft(in)o(ter-)75 2075 y(face.)20 b(This)15 b(routine)g(is)g(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 1071 2075 V 16 w(WRITE)p 1230 2075 V 16 w(NEXT)p Ft(,)h(except)g(that)f(all)i(pro)q(cesses)75 2131 y(in)11 b(the)f(comm)o(unicator)g(group)g(asso)q(ciated)g(with)h(the)f(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1523 2131 V 16 w(WRITE)p 1682 2131 V 16 w(NEXT)p 1818 2131 V 17 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 25 26 bop 75 -100 a Fn(10.4.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (INDIVIDUAL)i(FILE)e(POINTERS)487 b Ft(25)75 49 y Fl(10.4.3)49 b(Nonblo)q(cking)19 b(Read)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IREAD)p 308 230 V 16 w(NEXT\(fh,)d(bu\013,)f(buft)o(yp)q (e,)i(bufcount,)g(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i (user's)f(bu\013er)h(\(in)o(teger\))117 457 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 532 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 607 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 732 y Fp(int)23 b(MPI)p 245 732 15 2 v 17 w(Iread)p 382 732 V 17 w(next\(MPI)p 591 732 V 16 w(File)g(fh,)h(void)f(*buff,)g(MPI)p 1180 732 V 17 w(Datatype)f (buftype,)393 788 y(int)i(bufcount,)e(MPI)p 802 788 V 17 w(Request)h(*request\))75 875 y(MPI)p 150 875 V 17 w(IREAD)p 287 875 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (REQUEST,)f(IERROR\))170 931 y()h(BUFF\(*\))170 988 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 1074 y Fq(MPI)p 251 1074 14 2 v 16 w(IREAD)p 399 1074 V 16 w(NEXT)12 b Ft(is)g(seman)o(tically)g(iden)o(tical)h(to)d Fq(MPI)p 1157 1074 V 16 w(IREAD)p Ft(,)h(with)g Fq(o\013set)h Ft(set)e(to)h(the)g(curren)o(t)75 1130 y(individual)21 b(\014le)e(p)q(oin)o(ter)g(p)q(osition)g(main)o(tained)g(b)o(y)f(the)g (system.)28 b(The)18 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o (y)75 1187 y(the)c(amoun)o(t)g(of)f(data)h(requested.)75 1338 y Fq(MPI)p 160 1338 V 16 w(IREAD)p 308 1338 V 16 w(NEXT)p 444 1338 V 17 w(ALL\(fh,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 1442 y Fp(int)23 b(MPI)p 245 1442 15 2 v 17 w(Iread)p 382 1442 V 17 w(next)p 495 1442 V 16 w(all\(MPI)p 679 1442 V 17 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1269 1442 V 16 w(Datatype)g(buftype,)393 1498 y(int)h(bufcount,)e(MPI)p 802 1498 V 17 w(Request)h(*request\))75 1584 y(MPI)p 150 1584 V 17 w(IREAD)p 287 1584 V 16 w(NEXT)p 399 1584 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\)) 170 1641 y()g(BUFF\(*\))170 1697 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)166 1784 y Fq(MPI)p 251 1784 14 2 v 16 w(IREAD)p 399 1784 V 16 w(NEXT)p 535 1784 V 17 w(ALL)15 b Ft(is)g(a)f(collectiv)o(e)j(v)o(ersion)e(of)f(the)h (blo)q(c)o(king)h Fq(MPI)p 1471 1784 V 16 w(IREAD)p 1619 1784 V 16 w(NEXT)g Ft(in)o(ter-)75 1840 y(face.)i(This)12 b(routine)f(is)h(seman)o(tically)g(iden)o(tical)h(to)d Fq(MPI)p 1048 1840 V 16 w(IREAD)p 1196 1840 V 16 w(NEXT)p Ft(,)h(except)h(that)e(all)i(pro)q(cesses)f(in)75 1897 y(the)g(comm)o(unicator)g(group)f(asso)q(ciated)i(with)f(the)g(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)f(call)i Fq(MPI)p 1484 1897 V 16 w(IREAD)p 1632 1897 V 17 w(NEXT)p 1769 1897 V 17 w(ALL)p Ft(.)75 2018 y Fl(10.4.4)49 b(Nonblo)q(cking)19 b(W)o(rite)75 2199 y Fq(MPI)p 160 2199 V 16 w(IWRITE)p 332 2199 V 16 w(NEXT\(fh,)c(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (request\))117 2276 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 2351 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 2426 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 2501 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 2577 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2701 y Fp(int)23 b(MPI)p 245 2701 15 2 v 17 w(Iwrite)p 406 2701 V 17 w(next\(MPI)p 615 2701 V 16 w(File)g(fh,)g(void)h(*buff,) f(MPI)p 1204 2701 V 17 w(Datatype)f(buftype,)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 26 27 bop 75 -100 a Ft(26)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)393 49 y Fp(int)24 b(bufcount,)e(MPI)p 802 49 15 2 v 17 w(Request)h(*request\))75 136 y(MPI)p 150 136 V 17 w(IWRITE)p 311 136 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(REQUEST,)h(IERROR\))170 192 y()g(BUFF\(*\))170 249 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 335 y Fq(MPI)p 251 335 14 2 v 16 w(IWRITE)p 423 335 V 16 w(NEXT)f Ft(is)g(seman)o(tically)h(iden)o(tical)h(to)d Fq(MPI)p 1234 335 V 16 w(IWRITE)p Ft(,)g(with)h Fq(o\013set)g Ft(set)g(to)f(the)75 392 y(curren)o(t)11 b(individual)j(\014le)e(p)q (oin)o(ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)18 b(The)11 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)75 448 y(b)o(y)k(the)g(amoun)o(t)g(of)g(data)f(requested.)75 599 y Fq(MPI)p 160 599 V 16 w(IWRITE)p 332 599 V 16 w(NEXT)p 468 599 V 17 w(ALL\(fh,)g(bu\013,)h(buft)o(yp)q(e,)i(bufcount,)g (request\))75 703 y Fp(int)23 b(MPI)p 245 703 15 2 v 17 w(Iwrite)p 406 703 V 17 w(next)p 519 703 V 16 w(all\(MPI)p 703 703 V 16 w(File)h(fh,)f(void)h(*buff,)f(MPI)p 1293 703 V 16 w(Datatype)g(buftype,)393 759 y(int)h(bufcount,)e(MPI)p 802 759 V 17 w(Request)h(*request\))75 846 y(MPI)p 150 846 V 17 w(IWRITE)p 311 846 V 16 w(NEXT)p 423 846 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\))170 902 y()g(BUFF\(*\))170 958 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)166 1045 y Fq(MPI)p 251 1045 14 2 v 16 w(IWRITE)p 423 1045 V 16 w(NEXT)p 559 1045 V 17 w(ALL)15 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)f(of)f(the) h(blo)q(c)o(king)h Fq(MPI)p 1501 1045 V 16 w(IWRITE)p 1673 1045 V 16 w(NEXT)f Ft(in-)75 1101 y(terface.)26 b(This)17 b(routine)h(is)g(seman)o(tically)g(iden)o(tical)h(to)e Fq(MPI)p 1149 1101 V 15 w(IWRITE)p 1320 1101 V 16 w(NEXT)p Ft(,)h(except)f(that)g(all)h(pro-)75 1158 y(cesses)10 b(in)h(the)g(comm)o(unicator)f(group)f(asso)q(ciated)i(with)f(the)g (\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)i Fq(MPI)p 1647 1158 V 16 w(IWRITE)p 1819 1158 V 16 w(NEXT)p 1955 1158 V 17 w(ALL)p Ft(.)75 1280 y Fl(10.4.5)49 b(Seek)16 b(\(indep)q(endent\))75 1460 y Fq(MPI)p 160 1460 V 16 w(SEEK\(fh,)f(o\013set,)g(whence\))117 1537 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1612 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(o\013set\)) 117 1687 y(IN)155 b Fq(whence)450 b Ff(Up)q(date)15 b(mo)q(de)e(\(in)o (teger\))75 1812 y Fp(int)23 b(MPI)p 245 1812 15 2 v 17 w(Seek\(MPI)p 454 1812 V 16 w(File)h(fh,)f(MPI)p 757 1812 V 17 w(Offset)g(offset,)g(MPI)p 1204 1812 V 17 w(Whence)g (whence\))75 1898 y(MPI)p 150 1898 V 17 w(SEEK\(FH,)g(OFFSET,)f (WHENCE\))170 1955 y(INTEGER)h(FH,)h(WHENCE)170 2011 y(INTEGER*8)f(OFFSET)166 2098 y Fq(MPI)p 251 2098 14 2 v 16 w(SEEK)17 b Ft(up)q(dates)h(the)g(individual)i(\014le)f(p)q(oin) o(ter)f(according)g(to)e Fq(whence)p Ft(,)k(whic)o(h)e(could)g(ha)o(v)o (e)75 2154 y(the)d(follo)o(wing)h(p)q(ossible)h(v)m(alues:)143 2248 y Fh(\017)23 b Fg(MPI)p 266 2248 13 2 v 14 w(SEEK)p 382 2248 V 14 w(SET)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)f Fq(o\013set)143 2342 y Fh(\017)23 b Fg(MPI)p 266 2342 V 14 w(SEEK)p 382 2342 V 14 w(CUR)p Ft(:)14 b(the)h(p)q(oin)o(ter)h(is) g(set)f(to)f(the)h(curren)o(t)h(p)q(oin)o(ter)f(p)q(osition)i(plus)f Fq(o\013set)143 2435 y Fh(\017)23 b Fg(MPI)p 266 2435 V 14 w(SEEK)p 382 2435 V 14 w(END)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is) h(set)f(to)g(the)g(end)h(of)e(the)i(\014le)g(plus)g Fq(o\013set)166 2529 y Ft(The)c Fq(o\013set)g Ft(is)g(relativ)o(e)h(to)e(the)h(\014let) o(yp)q(e)h(\(not)e(coun)o(ting)i(holes\),)f(in)h(units)f(of)g Fq(et)o(yp)q(e)p Ft(.)20 b(The)12 b(constan)o(t)75 2586 y Fg(MPI)p 152 2586 V 14 w(OFFSET)p 321 2586 V 14 w(ZERO)j Ft(is)h(de\014ned)g(to)f(b)q(e)h(o\013set)e(zero.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 27 28 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(27)75 49 y Fm(10.5)60 b(Data)19 b(Access)h(with)g(Sha)n(red)f(File)g(P)n(ointers)75 233 y Fe(Discussion:)c Ff(There)f(is)e(some)g(con)o(tro)o(v)o(ersy)h (as)g(to)f(whether)i(w)o(e)f(should)f(k)o(eep)i(shared)f(\014le)g(p)q (oin)o(ters,)g(and)f(if)g(w)o(e)75 290 y(do,)h(the)i(exact)f(seman)o (tics)g(of)f(shared)i(\014le)f(p)q(oin)o(ter)g(op)q(erations.)166 429 y Ft(The)g(system)f(main)o(tains)h(exactly)g(one)g(shared)g(\014le) h(p)q(oin)o(ter)f(p)q(er)g(collectiv)o(e)i Fq(MPI)p 1580 429 14 2 v 16 w(OPEN)e Ft(\(shared)75 485 y(among)19 b(pro)q(cesses)h(in)h(the)f(comm)o(unicator)f(group\).)33 b(The)20 b(curren)o(t)g(v)m(alue)h(of)e(this)h(p)q(oin)o(ter)g(can)g(b) q(e)75 542 y(used)h(to)f(implicitly)j(sp)q(ecify)f(the)e(o\013set)g(of) g(data)f(accesses,)j(b)o(y)e(using)i(the)e(routines)h(describ)q(ed)h (in)75 598 y(this)16 b(section.)k(These)c(routines)g(only)g(use)f(and)h (up)q(date)g(the)f(shared)g(\014le)i(p)q(oin)o(ter)e(main)o(tained)i(b) o(y)e(the)75 655 y(system.)k(The)d(individual)i(\014le)e(p)q(oin)o (ters)g(are)f(not)g(used)h(nor)f(up)q(dated.)166 711 y(The)c(shared)g(\014le)i(p)q(oin)o(ter)e(routines)h(ha)o(v)o(e)e(the)i (same)e(seman)o(tics)i(as)e(the)i(data)e(access)h(with)h(explicit)75 768 y(o\013set)i(routines)i(describ)q(ed)h(in)f(Section)g(10.3)e(with)i (the)f(follo)o(wing)h(mo)q(di\014cations:)143 862 y Fh(\017)23 b Ft(the)f Fq(o\013set)h Ft(is)g(de\014ned)h(to)e(b)q(e)i(the)e(curren) o(t)h(v)m(alue)h(of)e(the)h(system-main)o(tained)g(shared)g(\014le)189 918 y(p)q(oin)o(ter,)143 1012 y Fh(\017)g Ft(the)14 b(e\013ect)g(of)f (m)o(ultiple)j(calls)g(to)d(shared)h(\014le)h(p)q(oin)o(ter)g(routines) g(is)f(de\014ned)i(to)d(b)q(eha)o(v)o(e)i(as)e(if)i(the)189 1068 y(calls)h(w)o(ere)f(serialized,)i(and)143 1162 y Fh(\017)23 b Ft(the)17 b(use)h(of)e(shared)i(\014le)g(p)q(oin)o(ter)g (routines)f(is)h(erroneous)f(unless)i(all)f(pro)q(cesses)f(use)h(the)f (same)189 1219 y Fq(\014let)o(yp)q(e)f Ft(when)g(op)q(ening)g(the)g (\014le.)75 1312 y(The)e(serialization)h(ordering)e(is)h(not)f (deterministic.)21 b(The)14 b(user)f(needs)i(to)d(use)i(other)f(sync)o (hronization)75 1369 y(means)i(to)g(enforce)g(a)g(sp)q(eci\014c)i (order.)166 1425 y(After)j(a)h(shared)g(\014le)g(p)q(oin)o(ter)h(op)q (eration)f(is)g(initiated,)i(the)e(shared)g(\014le)h(p)q(oin)o(ter)f (is)g(up)q(dated)75 1482 y(to)d(p)q(oin)o(t)h(to)f(the)h(next)g(data)f (item)h(after)f(the)g(last)h(one)g(that)f(will)i(b)q(e)f(actually)h (accessed,)f(p)q(ossibly)75 1538 y(accoun)o(ting)d(for)e(an)h(end)h(of) f(\014le)h(o)q(ccurrence.)189 1644 y Fk(A)n(dvic)n(e)i(to)i(users.)60 b Ft(In)20 b(order)e(to)g(ensure)h(that)f(data)g(access)h(using)g(the)g (shared)g(\014le)g(p)q(oin)o(ter)189 1701 y(is)g(w)o(ell)h(de\014ned,)h (all)e(pro)q(cesses)h(m)o(ust)e(use)h(the)g(same)g Fq(\014let)o(yp)q(e) h Ft(when)g(op)q(ening)g(the)f(\014le.)32 b(F)l(or)189 1757 y(example,)14 b(consider)g(a)f(single)h Fq(MPI)p 798 1757 V 16 w(OPEN)g Ft(in)g(whic)o(h)g(pro)q(cess)f(1)g(uses)h (\014let)o(yp)q(e)g Fq(MPI)p 1655 1757 V 16 w(CHAR)g Ft(and)189 1814 y(pro)q(cess)k(2)f(uses)h(\014let)o(yp)q(e)h Fq(MPI)p 737 1814 V 16 w(DOUBLE)p Ft(.)f(If)g(pro)q(cess)g(1)g(reads)f (a)h(single)h(c)o(haracter)e(from)g(the)189 1870 y(\014le,)i(the)f (shared)g(\014le)g(p)q(oin)o(ter)h(p)q(oin)o(ts)f(to)f(the)h(second)g (c)o(haracter)f(relativ)o(e)i(to)e(pro)q(cess)h(1,)g(but)189 1927 y(p)q(oin)o(ts)13 b(in)o(to)f(the)h(middle)h(of)e(the)h(\014rst)f (double)i(relativ)o(e)f(to)f(pro)q(cess)h(2.)18 b(A)o(t)12 b(this)h(p)q(oin)o(t,)h(the)e(e\013ect)189 1983 y(of)i(a)h(shared)h(p)q (oin)o(ter)f(op)q(eration)h(on)f(pro)q(cess)g(2)g(is)h(ill-de\014ned.) 23 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(users.)p Ft(\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 28 30 28 29 bop 75 -100 a Ft(28)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.5.1)49 b(Indep)q(endent)16 b(Op)q(erations)75 135 y Fq(Blo)q(cking)g(Read)g(\(indep)q(endent)q(\))75 315 y(MPI)p 160 315 14 2 v 16 w(READ)p 295 315 V 17 w(SHARED\(fh,)g (bu\013,)f(buft)o(yp)q(e,)i(bufcount,)g(status\))117 393 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 468 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 543 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 618 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 693 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 818 y Fp(int)23 b(MPI)p 245 818 15 2 v 17 w(Read)p 358 818 V 17 w(shared\(MPI)p 615 818 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1204 818 V 17 w(Datatype)f(buftype,)393 874 y(int)i(bufcount,)e(MPI)p 802 874 V 17 w(Status)h(*status\))75 960 y(MPI)p 150 960 V 17 w(READ)p 263 960 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(STATUS,)h(IERROR\))170 1017 y()g(BUFF\(*\))170 1073 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1073 V 16 w(STATUS)p 1313 1073 V 16 w(SIZE\),)h(IERROR)166 1160 y Fq(MPI)p 251 1160 14 2 v 16 w(READ)p 386 1160 V 17 w(SHARED)15 b Ft(is)f(seman)o(tically)i(iden)o(tical)g(to)d Fq(MPI)p 1216 1160 V 16 w(READ)p Ft(,)h(with)h Fq(o\013set)f Ft(set)g(to)f(the)i(cur-)75 1216 y(ren)o(t)g(shared)g(\014le)i(p)q(oin) o(ter)f(p)q(osition)g(main)o(tained)g(b)o(y)g(the)f(system.)20 b(Shared)c(\014le)g(p)q(oin)o(ter)g(calls)h(within)75 1273 y(a)e(single)i(\014le)g(handle)g(comm)o(unicator)e(group)h(are)f (guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially)g(consisten)o(t.)22 b(The)75 1329 y(\014le)16 b(p)q(oin)o(ter)g(is)g(up)q(dated)g(b)o(y)f (the)g(amoun)o(t)f(of)h(data)g(requested.)75 1449 y Fq(Blo)q(cking)h(W) o(rite)f(\(indep)q(endent\))75 1630 y(MPI)p 160 1630 V 16 w(WRITE)p 319 1630 V 16 w(SHARED\(fh,)h(bu\013,)f(buft)o(yp)q(e,)i (bufcount,)g(status\))117 1707 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1782 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's) f(bu\013er)h(\(in)o(teger\))117 1857 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1932 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2007 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c (\(Status\))75 2132 y Fp(int)23 b(MPI)p 245 2132 15 2 v 17 w(Write)p 382 2132 V 17 w(shared\(MPI)p 639 2132 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1228 2132 V 16 w(Datatype)g(buftype,)393 2188 y(int)h(bufcount,)e(MPI)p 802 2188 V 17 w(Status)h(*status\))75 2275 y(MPI)p 150 2275 V 17 w(WRITE)p 287 2275 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(STATUS,)h(IERROR\))170 2331 y()g(BUFF\(*\))170 2388 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2388 V 16 w(STATUS)p 1313 2388 V 16 w(SIZE\),)h(IERROR)166 2474 y Fq(MPI)p 251 2474 14 2 v 16 w(WRITE)p 410 2474 V 16 w(SHARED)d Ft(is)f(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 1262 2474 V 16 w(WRITE)p Ft(,)g(with)h Fq(o\013set)g Ft(set)g(to)f(the)75 2531 y(curren)o(t)10 b(shared)h(\014le)g(p)q(oin)o (ter)g(p)q(osition)g(main)o(tained)g(b)o(y)f(the)h(system.)18 b(Shared)10 b(\014le)i(p)q(oin)o(ter)e(calls)i(within)75 2587 y(a)j(single)i(\014le)g(handle)g(comm)o(unicator)e(group)h(are)f (guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially)g(consisten)o(t.)22 b(The)75 2643 y(\014le)16 b(p)q(oin)o(ter)g(is)g(up)q(dated)g(b)o(y)f (the)g(amoun)o(t)f(of)h(data)g(requested.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 29 31 29 30 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(29)75 49 y Fq(Nonblo)q(cking)16 b(Read)g(\(indep)q(enden)q(t\))75 230 y(MPI)p 160 230 14 2 v 16 w(IREAD)p 308 230 V 16 w(SHARED\(fh,)g(bu\013,)g(buft)o(yp)q (e,)g(bufcount,)h(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i (user's)f(bu\013er)h(\(in)o(teger\))117 457 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 532 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 607 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 732 y Fp(int)23 b(MPI)p 245 732 15 2 v 17 w(Iread)p 382 732 V 17 w(shared\(MPI)p 639 732 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1228 732 V 16 w(Datatype)g (buftype,)393 788 y(int)h(bufcount,)e(MPI)p 802 788 V 17 w(Request)h(*request\))75 875 y(MPI)p 150 875 V 17 w(IREAD)p 287 875 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f (REQUEST,)h(IERROR\))170 931 y()g(BUFF\(*\))170 988 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 1074 y Fq(MPI)p 251 1074 14 2 v 16 w(IREAD)p 399 1074 V 16 w(SHARED)f Ft(is)g(seman)o(tically)g(iden)o(tical)g(to)e Fq(MPI)p 1262 1074 V 16 w(IREAD)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)f(the)75 1130 y(curren)o(t)h(shared)g(\014le)h(p)q(oin)o (ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)37 b(Shared)21 b(\014le)h(p)q(oin)o(ter)g(calls)75 1187 y(within)13 b(a)f(single)i(\014le)f(handle)g(comm)o(unicator)f(group)g (are)g(guaran)o(teed)g(to)f(b)q(e)i(sequen)o(tially)h(consisten)o(t.)75 1243 y(The)h(\014le)i(p)q(oin)o(ter)e(is)h(up)q(dated)g(b)o(y)f(the)h (amoun)o(t)e(of)h(data)f(requested.)75 1363 y Fq(Nonblo)q(cking)i(W)o (rite)f(\(indep)q(enden)q(t\))75 1544 y(MPI)p 160 1544 V 16 w(IWRITE)p 332 1544 V 16 w(SHARED\(fh,)h(bu\013,)f(buft)o(yp)q(e,) i(bufcount,)g(request\))117 1621 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1696 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's) f(bu\013er)h(\(in)o(teger\))117 1771 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1846 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 1922 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2046 y Fp(int)23 b(MPI)p 245 2046 15 2 v 17 w(Iwrite)p 406 2046 V 17 w(shared\(MPI)p 663 2046 V 15 w(File)h(fh,)f(void)g(*buff,)h(MPI)p 1252 2046 V 16 w(Datatype)f(buftype,)393 2102 y(int)h(bufcount,)e(MPI)p 802 2102 V 17 w(Request)h(*request\))75 2189 y(MPI)p 150 2189 V 17 w(IWRITE)p 311 2189 V 16 w(SHARED\(FH,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\))170 2245 y()g(BUFF\(*\))170 2302 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (REQUEST,)h(IERROR)166 2388 y Fq(MPI)p 251 2388 14 2 v 16 w(IWRITE)p 423 2388 V 16 w(SHARED)17 b Ft(is)g(seman)o(tically)g (iden)o(tical)h(to)d Fq(MPI)p 1262 2388 V 16 w(IWRITE)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)g(the)75 2445 y(curren)o(t)10 b(shared)h(\014le)g(p)q(oin)o(ter)g(p)q(osition)g(main) o(tained)g(b)o(y)f(the)h(system.)18 b(Shared)10 b(\014le)i(p)q(oin)o (ter)e(calls)i(within)75 2501 y(a)j(single)i(\014le)g(handle)g(comm)o (unicator)e(group)h(are)f(guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially) g(consisten)o(t.)22 b(The)75 2558 y(\014le)16 b(p)q(oin)o(ter)g(is)g (up)q(dated)g(b)o(y)f(the)g(amoun)o(t)f(of)h(data)g(requested.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 30 32 30 31 bop 75 -100 a Ft(30)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.5.2)49 b(Collective)18 b(Op)q(erations)75 135 y Ft(Unlik)o(e)h(other)e(collectiv)o(e)j(op)q(erations,)e(a)f (barrier)h(sync)o(hronization)g(\(among)f(the)g(pro)q(cesses)h(asso)q (ci-)75 192 y(ated)g(with)g(the)g(comm)o(unicator)f(in)o(v)o(olv)o(ed)i (in)g(the)f(op)q(eration\))g(is)g(enforced)g(for)g(shared)g(\014le)g(p) q(oin)o(ter)75 248 y(op)q(erations.)166 304 y(The)12 b(seman)o(tics)g(of)f(a)g(collectiv)o(e)j(access)d(using)i(a)e(shared)h (\014le)h(p)q(oin)o(ter)f(is)g(that)f(the)h(accesses)g(to)f(the)75 361 y(\014le)i(will)h(b)q(e)f(in)g(the)f(order)g(determined)h(b)o(y)f (the)g(ranks)g(of)g(the)g(pro)q(cesses)g(within)i(the)e(group)g(asso)q (ciated)75 417 y(with)k(the)g(comm)o(unicator.)22 b(In)16 b(addition,)h(in)g(order)e(to)h(prev)o(en)o(t)f(subsequen)o(t)i(shared) f(o\013set)f(accesses)75 474 y(b)o(y)c(the)h(same)e(pro)q(cesses)i(to)f (in)o(terfere)g(with)h(this)g(collectiv)o(e)h(access,)e(the)h(call)g (can)g(only)f(return)h(after)e(all)75 530 y(the)15 b(pro)q(cesses)g (within)h(the)f(group)g(ha)o(v)o(e)f(initiated)j(their)e(accesses.)20 b(In)c(other)e(terms,)h(this)g(seman)o(tics)75 587 y(implies)j(a)d (barrier)h(sync)o(hronization)g(among)f(all)h(the)g(pro)q(cesses)g(of)f (the)h(group,)f(that)f(tak)o(es)h(place)i(at)75 643 y(the)h(outset)f (of)h(a)g(blo)q(c)o(king)h(op)q(eration)f(or)g(at)f(the)h(initiation)i (of)d(a)h(non)o(blo)q(c)o(king)h(op)q(eration.)29 b(When)75 700 y(the)15 b(call)i(returns,)e(the)g(shared)g(\014le)i(p)q(oin)o(ter) e(p)q(oin)o(ts)h(to)f(the)g(next)h(b)o(yte)f(accessible,)h(according)g (to)f(the)75 756 y(\014let)o(yp)q(e)h(used)g(b)o(y)f(all)h(pro)q (cesses,)g(after)e(the)h(last)g(b)o(yte)g(requested.)75 873 y Fq(Blo)q(cking)h(Read)g(\(collective\))75 1053 y(MPI)p 160 1053 14 2 v 16 w(READ)p 295 1053 V 17 w(SHARED)p 488 1053 V 17 w(ALL\(fh,)f(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (status\))117 1130 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 1202 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 1274 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1346 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 1417 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c (\(Status\))75 1542 y Fp(int)47 b(MPI)p 269 1542 15 2 v 17 w(Read)p 382 1542 V 17 w(shared)p 543 1542 V 16 w(all\(MPI)p 727 1542 V 16 w(File)24 b(fh,)f(void)g(*buff,)g(MPI)p 1316 1542 V 17 w(Datatype)g(buftype,)393 1598 y(int)h(bufcount,)e(MPI)p 802 1598 V 17 w(Status)h(*status\))75 1685 y(MPI)p 150 1685 V 17 w(READ)p 263 1685 V 16 w(SHARED)p 423 1685 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(STATUS,)h(IERROR\))170 1741 y()g(BUFF\(*\))170 1798 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 1798 V 16 w(STATUS)p 1313 1798 V 16 w(SIZE\),)h(IERROR)166 1884 y Fq(MPI)p 251 1884 14 2 v 16 w(READ)p 386 1884 V 17 w(SHARED)p 579 1884 V 17 w(ALL)17 b Ft(is)g(a)g(collectiv)o(e)h(v)o(ersion)g(of)e (the)h Fq(MPI)p 1350 1884 V 16 w(READ)p 1485 1884 V 17 w(SHARED)h Ft(in)o(terface.)75 1940 y(This)i(routine)f(m)o(ust)g(b)q(e) h(called)h(b)o(y)e(all)h(pro)q(cesses)f(in)h(the)g(comm)o(unicator)e (group)h(asso)q(ciated)h(with)75 1997 y(the)f(\014le)g(handle)h Fq(fh)p Ft(.)30 b(Eac)o(h)18 b(pro)q(cess)h(ma)o(y)f(pass)g(di\013eren) o(t)h(argumen)o(t)f(v)m(alues)h(for)f(the)h Fq(buft)o(yp)q(e)p Ft(,)i(and)75 2053 y Fq(bufcount)16 b Ft(argumen)o(ts.)i(After)c(all)g (pro)q(cesses)g(of)f(the)h(group)f(ha)o(v)o(e)h(issued)g(their)h(resp)q (ectiv)o(e)f(calls,)h(eac)o(h)75 2110 y(pro)q(cess)h(attempts)e(to)g (read,)h(from)g(the)g(\014le)i(asso)q(ciated)e(with)h Fq(fh)p Ft(,)f(a)g(total)g(n)o(um)o(b)q(er)g(of)g Fq(bufcount)j Ft(data)75 2166 y(items)e(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)24 b(F)l(or)15 b(eac)o(h)h(pro)q(cess,)h(the)f(lo)q(cation)h(in)g (the)75 2223 y(\014le)g(at)d(whic)o(h)j(data)e(is)h(read)f(is)h(the)g (p)q(osition)g(at)f(whic)o(h)h(the)g(shared)g(\014le)g(p)q(oin)o(ter)g (w)o(ould)g(b)q(e)g(after)f(all)75 2279 y(pro)q(cesses)k(whose)g(ranks) f(within)i(the)e(group)h(are)f(less)h(than)g(that)f(of)g(this)h(pro)q (cess)g(had)g(read)f(their)75 2336 y(data.)h Fq(MPI)p 281 2336 V 15 w(READ)p 415 2336 V 17 w(SHARED)p 608 2336 V 18 w(ALL)13 b Ft(returns)g(the)h(n)o(um)o(b)q(er)f(of)g Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)e(read)f(in)h Fq(status)p Ft(.)22 b(The)75 2392 y(shared)f(\014le)g(p)q(oin)o(ter)g(is)g(up)q (dated)h(b)o(y)e(the)h(amoun)o(ts)f(of)g(data)g(requested)h(b)o(y)f (all)i(pro)q(cesses)f(of)f(the)75 2449 y(group.)189 2534 y Fk(A)n(dvic)n(e)14 b(to)h(implementors.)39 b Ft(Accesses)15 b(to)e(the)h(data)f(requested)i(b)o(y)f(all)h(pro)q(cesses)f(do)g(not)f (ha)o(v)o(e)189 2591 y(to)f(b)q(e)i(serialized.)21 b(Once)14 b(all)g(pro)q(cesses)f(ha)o(v)o(e)g(issued)h(their)f(requests,)g(lo)q (cations)h(within)g(the)f(\014le)189 2647 y(for)i(all)i(accesses)g(can) f(b)q(e)h(computed,)g(and)f(accesses)h(can)f(pro)q(ceed)h(indep)q (enden)o(tly)i(from)d(eac)o(h)189 2704 y(other,)e(p)q(ossibly)j(in)f (parallel.)21 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 31 33 31 32 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(31)75 49 y Fq(Blo)q(cking)16 b(W)o(rite)f(\(collective\))75 230 y(MPI)p 160 230 14 2 v 16 w(WRITE)p 319 230 V 16 w(SHARED)p 511 230 V 18 w(ALL\(fh,)f(bu\013,)h(buft)o(yp)q(e,)i(bufcount,)g(status\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i (handle)g(\(handle\))117 381 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 455 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 530 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 604 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 728 y Fp(int)47 b(MPI)p 269 728 15 2 v 17 w(Write)p 406 728 V 17 w(shared)p 567 728 V 16 w(all\(MPI)p 751 728 V 16 w(File)23 b(fh,)h(void)f(*buff,)g(MPI)p 1340 728 V 17 w(Datatype)g(buftype,)393 785 y(int)h(bufcount,)e(MPI)p 802 785 V 17 w(Status)h(*status\))75 871 y(MPI)p 150 871 V 17 w(WRITE)p 287 871 V 16 w(SHARED)p 447 871 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)f(BUFCOUNT,)h(STATUS,)g(IERROR\))170 928 y()g(BUFF\(*\))170 984 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 984 V 16 w(STATUS)p 1313 984 V 16 w(SIZE\),)h(IERROR)166 1071 y Fq(MPI)p 251 1071 14 2 v 16 w(WRITE)p 410 1071 V 16 w(SHARED)p 602 1071 V 18 w(ALL)12 b Ft(is)i(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the)g (blo)q(c)o(king)i Fq(MPI)p 1526 1071 V 16 w(WRITE)p 1685 1071 V 16 w(SHARED)75 1127 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f (in)g(the)f(comm)o(unicator)g(group)g(asso)q(ciated)h(with)g(the)f (\014le)h(handle)h Fq(fh)f Ft(m)o(ust)75 1183 y(call)k Fq(MPI)p 247 1183 V 16 w(WRITE)p 406 1183 V 16 w(SHARED)p 598 1183 V 18 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)g(ma)o(y)g(pass)g (di\013eren)o(t)g(argumen)o(t)f(v)m(alues)i(for)f(the)75 1240 y Fq(buft)o(yp)q(e)13 b Ft(and)f Fq(bufcount)h Ft(argumen)o(ts.)18 b(After)11 b(all)h(pro)q(cesses)g(of)e(the)i(group)f(ha)o(v)o(e)g (issued)h(their)g(resp)q(ectiv)o(e)75 1296 y(calls,)22 b(eac)o(h)d(pro)q(cess)h(attempts)f(to)g(write,)i(in)o(to)e(the)h (\014le)h(asso)q(ciated)f(with)g Fq(fh)p Ft(,)h(a)e(total)g(n)o(um)o(b) q(er)h(of)75 1353 y Fq(bufcount)13 b Ft(data)e(items)g(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(con)o(tained)h(in)g(the)f(user's)g (bu\013er)g Fq(bu\013)p Ft(.)19 b(F)l(or)11 b(eac)o(h)g(pro)q(cess,)75 1409 y(the)g(lo)q(cation)h(in)h(the)e(\014le)h(at)f(whic)o(h)h(data)f (is)h(written)f(is)h(the)f(p)q(osition)h(at)f(whic)o(h)h(the)g(shared)f (\014le)h(p)q(oin)o(ter)75 1466 y(w)o(ould)j(b)q(e)g(after)e(all)i(pro) q(cesses)g(whose)f(ranks)g(within)h(the)g(group)f(are)g(less)h(than)f (that)f(of)h(this)h(pro)q(cess)75 1522 y(had)g(written)g(their)g(data.) k Fq(MPI)p 636 1522 V 16 w(WRITE)p 795 1522 V 17 w(SHARED)p 988 1522 V 17 w(ALL)14 b Ft(returns)h(the)g(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)h Ft(elemen)o(ts)75 1579 y(written)g(in)h Fq(status)p Ft(.)26 b(The)16 b(shared)h(\014le)g(p)q(oin)o(ter)g(is)g (up)q(dated)g(b)o(y)f(the)g(amoun)o(ts)g(of)g(data)f(requested)i(b)o(y) 75 1635 y(all)f(pro)q(cesses)g(of)e(the)i(group.)189 1736 y Fk(A)n(dvic)n(e)e(to)h(implementors.)39 b Ft(Accesses)15 b(to)e(the)h(data)f(requested)i(b)o(y)f(all)h(pro)q(cesses)f(do)g(not)f (ha)o(v)o(e)189 1792 y(to)f(b)q(e)i(serialized.)21 b(Once)14 b(all)g(pro)q(cesses)f(ha)o(v)o(e)g(issued)h(their)f(requests,)g(lo)q (cations)h(within)g(the)f(\014le)189 1849 y(for)i(all)i(accesses)g(can) f(b)q(e)h(computed,)g(and)f(accesses)h(can)f(pro)q(ceed)h(indep)q (enden)o(tly)i(from)d(eac)o(h)189 1905 y(other,)e(p)q(ossibly)j(in)f (parallel.)21 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))75 2025 y Fq(Nonblo)q(cking)f(Read)g(\(collective\))75 2205 y(MPI)p 160 2205 V 16 w(IREAD)p 308 2205 V 16 w(SHARED)p 500 2205 V 18 w(ALL\(fh,)e(bu\013,)i(buft)o(yp)q(e,)g(bufcount,)h (request\))117 2282 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 2356 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 2431 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 2505 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2579 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 2704 y Fp(int)23 b(MPI)p 245 2704 15 2 v 17 w(Iread)p 382 2704 V 17 w(shared)p 543 2704 V 16 w(all\(MPI)p 727 2704 V 16 w(File)h(fh,)f(void)g(*buff,)g(MPI)p 1316 2704 V 17 w(Datatype)g(buftype,)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 32 34 32 33 bop 75 -100 a Ft(32)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)393 49 y Fp(int)24 b(bufcount,)e(MPI)p 802 49 15 2 v 17 w(Request)h(*request\))75 136 y(MPI)p 150 136 V 17 w(IREAD)p 287 136 V 16 w(SHARED)p 447 136 V 17 w(ALL\(FH,)g(BUFF,) g(BUFTYPE,)f(BUFCOUNT,)h(REQUEST,)g(IERROR\))170 192 y()g(BUFF\(*\))170 249 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (REQUEST,)h(IERROR)166 335 y Fq(MPI)p 251 335 14 2 v 16 w(IREAD)p 399 335 V 16 w(SHARED)p 591 335 V 18 w(ALL)9 b Ft(is)i(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the)g(non)o(blo)q(c)o (king)i Fq(MPI)p 1563 335 V 15 w(IREAD)p 1710 335 V 17 w(SHARED)75 392 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f(in)g(the)f (comm)o(unicator)g(group)g(asso)q(ciated)h(with)g(the)f(\014le)h (handle)h Fq(fh)f Ft(m)o(ust)75 448 y(call)21 b Fq(MPI)p 248 448 V 16 w(IREAD)p 396 448 V 17 w(SHARED)p 589 448 V 17 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)h(ma)o(y)e(pass)h(di\013eren)o (t)h(argumen)o(t)e(v)m(alues)i(for)f(the)75 504 y Fq(buft)o(yp)q(e)e Ft(and)e Fq(bufcount)i Ft(argumen)o(ts.)j(After)15 b(all)i(pro)q (cesses)f(of)g(the)g(group)f(ha)o(v)o(e)h(issued)h(their)f(resp)q(ec-) 75 561 y(tiv)o(e)i(calls,)g(eac)o(h)g(pro)q(cess)f(attempts)g(to)f (read,)i(from)f(the)g(\014le)i(asso)q(ciated)e(with)h Fq(fh)p Ft(,)g(a)f(total)g(n)o(um)o(b)q(er)75 617 y(of)i Fq(bufcount)i Ft(data)d(items)h(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)32 b(F)l(or)18 b(eac)o(h)h(pro)q(cess,)75 674 y(the)f(lo)q(cation) g(in)g(the)g(\014le)h(at)e(whic)o(h)h(data)f(will)i(b)q(e)f(read)g(is)g (the)g(p)q(osition)g(at)f(whic)o(h)i(the)e(shared)h(\014le)75 730 y(p)q(oin)o(ter)d(w)o(ould)g(b)q(e)g(after)e(all)j(pro)q(cesses)e (whose)h(ranks)f(within)h(the)g(group)f(are)g(less)h(than)f(that)g(of)g (this)75 787 y(pro)q(cess)j(had)f(read)h(their)g(data.)23 b Fq(MPI)p 747 787 V 15 w(IREAD)p 894 787 V 17 w(SHARED)p 1087 787 V 18 w(ALL)16 b Ft(returns)g(immediately)i(and)f(do)q(es)f (not)75 843 y(w)o(ait)d(for)f(the)i(data)e(to)h(b)q(e)h(presen)o(t)f (in)h(the)f(user's)g(bu\013er.)19 b Fq(MPI)p 1172 843 V 16 w(IREAD)p 1320 843 V 17 w(SHARED)p 1513 843 V 17 w(ALL)13 b Ft(asso)q(ciates)g(an)75 900 y(individual)20 b(request)c(handle)i Fq(request)g Ft(to)d(the)i(I/O)g(request)f(for)g (eac)o(h)g(pro)q(cess.)24 b(The)16 b(request)h(handle)75 956 y(can)i(b)q(e)h(used)f(later)g(b)o(y)g(a)f(pro)q(cess)h(to)g(query) g(the)g(status)f(of)g(its)h(individual)j(read)d(request)g(or)f(w)o(ait) 75 1013 y(for)f(its)i(completion.)29 b(On)19 b(eac)o(h)f(pro)q(cess,)h Fq(MPI)p 916 1013 V 16 w(IREAD)p 1064 1013 V 16 w(SHARED)p 1256 1013 V 18 w(ALL)f Ft(completes)g(when)h(the)f(indi-)75 1069 y(vidual)f(request)e(has)g(completed)i(\(i.e.)j(a)15 b(pro)q(cess)g(do)q(es)h(not)f(ha)o(v)o(e)g(to)g(w)o(ait)f(for)h(all)h (other)f(requests)h(to)75 1125 y(complete\).)21 b(The)16 b(user)g(should)g(not)f(access)h(an)o(y)f(part)g(of)g(the)h(receiving)h (bu\013er)f(after)e(a)i(non)o(blo)q(c)o(king)75 1182 y(read)f(is)g(called,)i(un)o(til)f(the)f(read)g(completes.)20 b(The)15 b(shared)g(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o(y)f (the)g(amoun)o(ts)75 1238 y(of)j(data)f(requested)i(b)o(y)e(all)i(pro)q (cesses)g(of)f(the)g(group.)28 b(F)l(or)17 b(eac)o(h)h(pro)q(cess,)h (the)f(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)75 1295 y Ft(elemen)o(ts)d(actually)h(read)f(b)o(y)f(that)g(pro)q(cess)h(can)g (b)q(e)h(extracted)e(from)g(the)h Fq(MPI)p 1467 1295 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1749 1295 V 16 w(W)l(AIT)75 1351 y Ft(return)h(status,)f(using)i(the)f(routines)h Fq(MPI)p 818 1351 V 16 w(GET)p 922 1351 V 17 w(COUNT)g Ft(or)e Fq(MPI)p 1248 1351 V 16 w(GET)p 1352 1351 V 17 w(ELEMENTS)p Ft(.)75 1468 y Fq(Nonblo)q(cking)i(W)o(rite)f (\(collective\))75 1648 y(MPI)p 160 1648 V 16 w(IWRITE)p 332 1648 V 16 w(SHARED)p 524 1648 V 17 w(ALL\(fh,)g(bu\013,)g(buft)o (yp)q(e,)i(bufcount,)g(request\))117 1725 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\)) 117 1797 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the) i(user's)f(bu\013er)h(\(in)o(teger\))117 1868 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 1940 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2011 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2135 y Fp(int)47 b(MPI)p 269 2135 15 2 v 17 w(Iwrite)p 430 2135 V 16 w(shared)p 590 2135 V 17 w(all\(MPI)p 775 2135 V 16 w(File)23 b(fh,)h(void)f (*buff,)g(MPI)p 1364 2135 V 17 w(Datatype)g(buftype,)393 2192 y(int)h(bufcount,)e(MPI)p 802 2192 V 17 w(Request)h(*request\))75 2278 y(MPI)p 150 2278 V 17 w(IWRITE)p 311 2278 V 16 w(SHARED)p 471 2278 V 16 w(ALL\(FH,)g(BUFF,)h(BUFTYPE,)e(BUFCOUNT,)h(REQUEST,)g (IERROR\))170 2335 y()g(BUFF\(*\))170 2391 y(INTEGER)g(FH,)h (BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 2478 y Fq(MPI)p 251 2478 14 2 v 16 w(IWRITE)p 423 2478 V 16 w(SHARED)p 615 2478 V 17 w(ALL)10 b Ft(is)g(a)g(collectiv)o(e)i(v)o (ersion)f(of)e(the)i(non)o(blo)q(c)o(king)g Fq(MPI)p 1586 2478 V 16 w(IWRITE)p 1758 2478 V 16 w(SHARED)75 2534 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f(in)g(the)f(comm)o (unicator)g(group)g(asso)q(ciated)h(with)g(the)f(\014le)h(handle)h Fq(fh)f Ft(m)o(ust)75 2591 y(call)j Fq(MPI)p 246 2591 V 16 w(IWRITE)p 418 2591 V 16 w(SHARED)p 610 2591 V 17 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)g(ma)o(y)f(pass)h(di\013eren)o(t)g (argumen)o(t)f(v)m(alues)i(for)e(the)75 2647 y Fq(buft)o(yp)q(e)h Ft(and)e Fq(bufcount)i Ft(argumen)o(ts.)j(After)15 b(all)i(pro)q (cesses)f(of)g(the)g(group)f(ha)o(v)o(e)h(issued)h(their)f(resp)q(ec-) 75 2704 y(tiv)o(e)e(calls,)h(eac)o(h)f(pro)q(cess)h(attempts)e(to)g (write,)h(in)o(to)g(the)h(\014le)g(asso)q(ciated)f(with)g Fq(fh)p Ft(,)h(a)e(total)h(n)o(um)o(b)q(er)g(of)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 33 35 33 34 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(33)75 49 y Fq(bufcount)19 b Ft(data)d(items)g(ha)o(ving)h Fq(buft)o(yp)q(e)i Ft(t)o(yp)q(e)d(con)o(tained)h(in)h(the)e(user's)h (bu\013er)f Fq(bu\013)p Ft(.)25 b(F)l(or)15 b(eac)o(h)i(pro-)75 106 y(cess,)c(the)g(lo)q(cation)g(in)g(the)g(\014le)h(at)e(whic)o(h)h (data)f(will)i(b)q(e)g(written)e(is)h(the)g(p)q(osition)h(at)e(whic)o (h)h(the)g(shared)75 162 y(\014le)j(p)q(oin)o(ter)g(w)o(ould)g(b)q(e)g (after)e(all)j(pro)q(cesses)e(whose)h(ranks)f(within)h(the)g(group)f (are)g(less)h(than)f(that)f(of)75 219 y(this)k(pro)q(cess)g(had)g (written)g(their)g(data.)27 b Fq(MPI)p 905 219 14 2 v 16 w(IWRITE)p 1077 219 V 16 w(SHARED)p 1269 219 V 17 w(ALL)18 b Ft(returns)f(immediately)j(and)75 275 y(do)q(es)e(not)g(w)o (ait)f(for)h(the)g(data)f(to)g(b)q(e)i(presen)o(t)f(in)h(the)f(user's)f (bu\013er.)28 b Fq(MPI)p 1417 275 V 16 w(IWRITE)p 1589 275 V 16 w(SHARED)p 1781 275 V 17 w(ALL)75 332 y Ft(asso)q(ciates)18 b(an)h(individual)j(request)c(handle)i Fq(request)g Ft(to)e(the)h(I/O)g (request)f(for)g(eac)o(h)h(pro)q(cess.)30 b(The)75 388 y(request)19 b(handle)h(can)f(b)q(e)g(used)g(later)g(b)o(y)g(a)f(pro)q (cess)h(to)f(query)h(the)g(status)f(of)g(its)h(individual)j(write)75 444 y(request)15 b(or)g(w)o(ait)g(for)g(its)g(completion.)22 b(On)16 b(eac)o(h)f(pro)q(cess,)g Fq(MPI)p 1207 444 V 16 w(IWRITE)p 1379 444 V 16 w(SHARED)p 1571 444 V 17 w(ALL)g Ft(completes)75 501 y(when)i(the)g(individual)j(write)d (request)g(has)f(completed)i(\(i.e.)25 b(a)16 b(pro)q(cess)h(do)q(es)g (not)g(ha)o(v)o(e)f(to)g(w)o(ait)h(for)75 557 y(all)f(other)f(requests) g(to)g(complete\).)20 b(The)c(user)f(should)h(not)f(access)h(an)o(y)f (part)f(of)h(the)g(supplied)j(bu\013er)75 614 y(after)g(a)h(non)o(blo)q (c)o(king)i(write)e(is)g(called,)j(un)o(til)e(the)f(write)g(is)h (completed.)33 b(The)19 b(shared)g(\014le)h(p)q(oin)o(ter)75 670 y(is)d(up)q(dated)h(b)o(y)f(the)f(amoun)o(ts)g(of)h(data)f (requested)h(b)o(y)g(all)g(pro)q(cesses)h(of)e(the)h(group.)24 b(F)l(or)16 b(eac)o(h)h(pro-)75 727 y(cess,)22 b(the)f(n)o(um)o(b)q(er) g(of)f Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f(actually)f(written)g(b)o(y)g (that)f(pro)q(cess)h(can)g(b)q(e)g(extracted)75 783 y(from)c(the)h Fq(MPI)p 351 783 V 16 w(TEST)h Ft(or)e Fq(MPI)p 640 783 V 16 w(W)l(AIT)h Ft(return)g(status,)g(using)g(the)h(routines)f Fq(MPI)p 1542 783 V 16 w(GET)p 1646 783 V 17 w(COUNT)g Ft(or)75 840 y Fq(MPI)p 160 840 V 16 w(GET)p 264 840 V 17 w(ELEMENTS)p Ft(.)75 957 y Fq(Seek)e(\(collective\))75 1137 y(MPI)p 160 1137 V 16 w(SEEK)p 287 1137 V 16 w(SHARED\(fh,)g (o\013set,)f(whence\))117 1214 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 1286 y(IN)155 b Fq(o\013set)484 b Ff([SAME])14 b(File)f(o\013set)i (\(o\013set\))117 1358 y(IN)155 b Fq(whence)450 b Ff([SAME])14 b(Up)q(date)g(mo)q(de)f(\(in)o(teger\))75 1482 y Fp(int)23 b(MPI)p 245 1482 15 2 v 17 w(Seek)p 358 1482 V 17 w(shared\(MPI)p 615 1482 V 16 w(File)g(fh,)g(MPI)p 917 1482 V 17 w(Offset)g(offset,)g (MPI)p 1364 1482 V 17 w(Whence)g(whence\))75 1569 y(MPI)p 150 1569 V 17 w(SEEK)p 263 1569 V 16 w(SHARED\(FH,)g(OFFSET,)g (WHENCE\))170 1625 y(INTEGER)g(FH,)h(WHENCE)170 1682 y(INTEGER*8)f(OFFSET)166 1768 y Fq(MPI)p 251 1768 14 2 v 16 w(SEEK)p 378 1768 V 16 w(SHARED)17 b Ft(up)q(dates)f(the)g (shared)g(\014le)h(p)q(oin)o(ter)f(according)g(to)f Fq(whence)p Ft(,)i(whic)o(h)g(could)75 1824 y(ha)o(v)o(e)e(the)g(follo)o(wing)h(p)q (ossible)h(v)m(alues:)143 1902 y Fh(\017)23 b Fg(MPI)p 266 1902 13 2 v 14 w(SEEK)p 382 1902 V 14 w(SET)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)f Fq(o\013set)143 1989 y Fh(\017)23 b Fg(MPI)p 266 1989 V 14 w(SEEK)p 382 1989 V 14 w(CUR)p Ft(:)14 b(the)h(p)q(oin)o(ter)h(is)g(set)f(to)f(the)h (curren)o(t)h(p)q(oin)o(ter)f(p)q(osition)i(plus)f Fq(o\013set)143 2076 y Fh(\017)23 b Fg(MPI)p 266 2076 V 14 w(SEEK)p 382 2076 V 14 w(END)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)g(the)g (end)h(of)e(the)i(\014le)g(plus)g Fq(o\013set)166 2154 y Ft(All)d(the)f(pro)q(cesses)g(in)g(the)g(comm)o(unicator)g(group)f (asso)q(ciated)h(with)g(the)g(\014le)h(handle)g Fq(fh)f Ft(m)o(ust)f(call)75 2210 y Fq(MPI)p 160 2210 14 2 v 16 w(SEEK)p 287 2210 V 16 w(SHARED)18 b Ft(with)g(the)f(same)g Fq(o\013set)g Ft(and)g Fq(whence)p Ft(.)27 b(All)19 b(pro)q(cesses)e (in)h(the)f(comm)o(unicator)75 2266 y(group)e(are)g(sync)o(hronized)h (with)g(a)f(barrier)g(b)q(efore)h(the)f(shared)g(\014le)h(p)q(oin)o (ter)g(is)g(up)q(dated.)166 2323 y(The)c Fq(o\013set)g Ft(is)g(relativ)o(e)h(to)e(the)h(\014let)o(yp)q(e)h(\(not)e(coun)o (ting)i(holes\),)f(in)h(units)f(of)g Fq(et)o(yp)q(e)p Ft(.)20 b(The)12 b(constan)o(t)75 2379 y Fg(MPI)p 152 2379 13 2 v 14 w(OFFSET)p 321 2379 V 14 w(ZERO)j Ft(is)h(de\014ned)g (to)f(b)q(e)h(o\013set)e(zero.)75 2519 y Fm(10.6)60 b(Filet)n(yp)r(e)19 b(Constructo)n(rs)75 2704 y Fe(Discussion:)14 b Ff(Should)c(w)o(e)h(mo) o(v)o(e)e(these)k(datat)o(yp)q(e)e(constructors)h(to)f(the)h(\\Datat)o (yp)q(e)e(Impro)o(v)o(emen)o(ts")f(section?)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 34 36 34 35 bop 75 -100 a Ft(34)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 254 y Fl(10.6.1)49 b(Intro)q(duction)75 339 y Ft(Common)20 b(I/O)h(op)q(erations)f(\(e.g.,)h(rank-ordered)f(blo)q (c)o(ks,)i(structured)f(arra)o(y)e(accesses,)j(etc.\))35 b(are)75 396 y(easily)20 b(expressed)g(using)g(the)f(previously)h (de\014ned)h(read/write)e(op)q(erations)g(and)g(carefully)i(de\014ned) 75 452 y(\014let)o(yp)q(es.)i(In)17 b(order)f(to)f(simplify)j (generation)e(of)g(common)f(\014let)o(yp)q(es,)i(the)f(follo)o(wing)h Fq(MPI)e Ft(datat)o(yp)q(e)75 509 y(constructors)f(are)h(pro)o(vided.) 166 565 y(Although)c(it)f(is)h(p)q(ossible)h(to)d(implemen)o(t)j(these) e(t)o(yp)q(e)h(constructors)e(as)h(lo)q(cal)h(op)q(erations,)g(in)g (order)75 622 y(to)g(facilitate)h(e\016cien)o(t)g(implemen)o(tations)g (of)f(\014le)h(I/O)f(op)q(erations,)h(most)f(of)f(the)h(\014let)o(yp)q (e)i(constructors)75 678 y(ha)o(v)o(e)j(b)q(een)i(de\014ned)g(to)e(b)q (e)i Fk(c)n(ol)r(le)n(ctive)d Ft(op)q(erations.)24 b(\(Recall)19 b(that)d(a)g(collectiv)o(e)i(op)q(eration)f(do)q(es)g(not)75 735 y(imply)k(a)e(barrier)g(sync)o(hronization.\))33 b(The)20 b(only)g(exception)g(to)f(this)g(is)h(the)g(lo)q(cal)g (sub-arra)o(y)f(t)o(yp)q(e)75 791 y(constructor)c(whic)o(h)i(is)f(lo)q (cal)i(and)e(uses)g(no)g(pro)q(cess)g(rank)g(information.)22 b(Because)17 b(of)e(this,)h(the)g(sub-)75 848 y(arra)o(y)e(constructor) g(can)i(b)q(e)g(used)f(for)g(b)q(oth)g(buft)o(yp)q(es)h(and)f(\014let)o (yp)q(es.)166 904 y(The)j(set)g(of)f(\014let)o(yp)q(es)i(created)f(b)o (y)g(a)g(single)h(collectiv)o(e)h(\014let)o(yp)q(e)f(constructor)e (should)i(b)q(e)f(used)75 960 y(together)c(when)i(op)q(ening)h(a)d (\014le.)166 1017 y(Eac)o(h)20 b(new)h(\014let)o(yp)q(e)h(created)e Fq(newt)o(yp)q(e)j Ft(consists)d(of)h(zero)f(or)g(more)g(copies)i(of)e (the)g(base)h(t)o(yp)q(e)75 1073 y Fq(oldt)o(yp)q(e)p Ft(,)14 b(p)q(ossibly)h(separated)e(b)o(y)g(holes.)20 b(The)13 b(exten)o(t)g(of)f(the)h(new)h(\014let)o(yp)q(e)g(is)g(a)e (nonnegativ)o(e)i(in)o(teger)75 1130 y(m)o(ultiple)j(of)d(the)h(exten)o (t)g(of)f(the)h(base)g(t)o(yp)q(e.)20 b(All)c(\014let)o(yp)q(e)g (constructors)f(return)f(a)h(success)g(or)g(failure)75 1186 y(co)q(de.)75 1308 y Fl(10.6.2)49 b(Scatter)15 b(/)i(Gather)e(T)l (yp)q(e)i(Constructo)o(rs)75 1394 y Fq(MPI)p 160 1394 14 2 v 16 w(T)l(yp)q(e)p 268 1394 V 17 w(scatter)p 412 1394 V 18 w(gather)e(\(collective\))75 1574 y(MPI)p 160 1574 V 16 w(TYPE)p 293 1574 V 17 w(SCA)l(TTER)p 508 1574 V 17 w(GA)l(THER\(comm,)f(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 1651 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e (to)h(b)q(e)i(used)g(in)e(MPI)p 1655 1651 13 2 v 15 w(Op)q(en)i(\(han-) 905 1708 y(dle\))117 1783 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp)q(e)h(\(handle\))117 1858 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f (\(handle\))75 1983 y Fp(int)23 b(MPI)p 245 1983 15 2 v 17 w(Type)p 358 1983 V 17 w(scatter)p 543 1983 V 16 w(gather\(MPI)p 799 1983 V 16 w(Comm)g(comm,)g(MPI)p 1149 1983 V 17 w(Datatype)g(oldtype,)393 2039 y(MPI)p 468 2039 V 17 w(Datatype)g(*newtype\))75 2125 y(MPI)p 150 2125 V 17 w(TYPE)p 263 2125 V 16 w(SCATTER)p 447 2125 V 17 w(GATHER\(COMM,)f(OLDTYPE,)h(NEWTYPE,)f(IERROR\))170 2182 y(INTEGER)h(COMM,)h(OLDTYPE,)e(NEWTYPE,)h(IERROR)166 2268 y Ft(This)18 b(\014let)o(yp)q(e)h(allo)o(ws)f(eac)o(h)g(pro)q (cess)f(in)i(the)f(group)f(to)g(access)h(a)f(distinct)i Fq(blo)q(ck)f Ft(of)g(the)f(\014le)i(in)75 2325 y(rank)14 b(order.)20 b(The)15 b(blo)q(c)o(ks)g(are)f(iden)o(tical)j(in)f(size)f (and)g(datat)o(yp)q(e;)f(eac)o(h)h(item)g(is)g(of)f(datat)o(yp)q(e)g Fq(oldt)o(yp)q(e)p Ft(.)166 2381 y(T)l(o)g(ac)o(hiev)o(e)h(the)f (scatter)g(or)g(gather)f(op)q(eration,)i(the)f(\014les)i(op)q(ened)f (with)g(the)f(returned)h(\014let)o(yp)q(es)75 2438 y(should)f(b)q(e)f (accessed)g(with)g(collectiv)o(e)i(read)e(or)f(write)h(op)q(erations)f (with)h(iden)o(tical)i(o\013sets.)j(Generated)75 2494 y(\014let)o(yp)q(es)e(ma)o(y)f(not)g(b)q(e)g(iden)o(tical,)i(but)e (will)i(ha)o(v)o(e)e(the)g(same)g(exten)o(t.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 35 37 35 36 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(35)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(T)l(yp)q(e)p 268 49 V 17 w(scatterv)p 433 49 V 18 w(gatherv)15 b(\(collective\))75 230 y(MPI)p 160 230 V 16 w(TYPE)p 293 230 V 17 w(SCA)l(TTERV)p 538 230 V 17 w(GA)l(THERV\(comm,)g(count,)h(oldt)o(yp)q(e,)g(newt)o(yp) q(e\))117 307 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to)h(b)q(e)i(used)g(in)e(MPI)p 1655 307 13 2 v 15 w(Op)q(en)i(\(han-)905 363 y(dle\))117 438 y(IN)155 b Fq(count)482 b Ff(n)o(um)o(b)q(er)14 b(of)h(elemen)o(ts) f(of)g(oldt)o(yp)q(e)h(in)g(this)f(blo)q(c)o(k)h(\(nonneg-)905 495 y(ativ)o(e)f(in)o(teger\))117 570 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff(old)13 b(datat)o(yp)q(e)i(\(handle\))117 645 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 770 y Fp(int)23 b(MPI)p 245 770 15 2 v 17 w(Type)p 358 770 V 17 w(scatterv)p 567 770 V 16 w(gatherv\(MPI)p 847 770 V 16 w(Comm)g(comm,)g(int)h(count,)393 826 y(MPI)p 468 826 V 17 w(Datatype)f(oldtype,)f(MPI)p 986 826 V 17 w(Datatype)h(*newtype\))75 912 y(MPI)p 150 912 V 17 w(TYPE)p 263 912 V 16 w(SCATTERV)p 471 912 V 16 w(GATHERV\(COMM,)g (COUNT,)g(OLDTYPE,)f(NEWTYPE,)h(IERROR\))170 969 y(INTEGER)g(COMM,)h (COUNT,)f(OLDTYPE,)f(NEWTYPE,)h(IERROR)166 1055 y Ft(This)16 b(t)o(yp)q(e)f(allo)o(ws)g(eac)o(h)g(pro)q(cess)g(in)h(the)g(group)e (to)h(access)g(a)g(distinct)h Fk(blo)n(ck)e Ft(of)h(the)g(\014le)h(in)g (rank)75 1112 y(order)11 b(as)h(in)g(MPI)p 384 1112 14 2 v 16 w(T)o(yp)q(e)p 502 1112 V 17 w(scatter)p 654 1112 V 16 w(gather,)f(except)h(that)f(the)h(blo)q(c)o(k)g(sizes)g(and)g(t)o (yp)q(es)g(ma)o(y)f(b)q(e)h(di\013eren)o(t.)75 1168 y(Eac)o(h)j(blo)q (c)o(k)h(is)g(de\014ned)g(as)f Fq(count)i Ft(con)o(tiguous)e(copies)h (of)f(the)g(passed)g(datat)o(yp)q(e)g Fq(oldt)o(yp)q(e)p Ft(.)75 1290 y Fl(10.6.3)49 b(Lo)q(cal)18 b(Arra)o(y)e(Filet)o(yp)q(e)g (Constructo)o(r)f(\(indep)q(endent\))75 1470 y Fq(MPI)p 160 1470 V 16 w(TYPE)p 293 1470 V 17 w(SUBARRA)l(Y\(ndims,)i(sizes,)e (subsizes,)i(sta)o(rts,)f(o)o(rder,)e(oldt)o(yp)q(e,)i(newt)o(yp)q(e\)) 117 1547 y Ff(IN)155 b Fq(ndims)476 b Ff(n)o(um)o(b)q(er)13 b(of)h(arra)o(y)f(dimensions)g(\(nonnegativ)o(e)h(in)o(teger\))117 1623 y(IN)155 b Fq(sizes)502 b Ff(size)15 b(of)e(the)i(full)d(arra)o(y) i(in)f(eac)o(h)i(dimension)d(\(arra)o(y)i(of)f(non-)905 1679 y(negativ)o(e)h(in)o(tegers\))117 1754 y(IN)155 b Fq(subsizes)439 b Ff(size)14 b(of)f(the)h(sub-arra)o(y)g(in)f(eac)o (h)h(dimension)e(\(arra)o(y)h(of)g(non-)905 1811 y(negativ)o(e)h(in)o (tegers\))117 1886 y(IN)155 b Fq(sta)o(rts)484 b Ff(starting)18 b(co)q(ordinates)g(of)e(the)i(sub-arra)o(y)g(in)f(eac)o(h)g(dimen-)905 1942 y(sion)d(\(arra)o(y)g(of)f(nonnegativ)o(e)g(in)o(tegers\))117 2017 y(IN)155 b Fq(o)o(rder)490 b Ff(arra)o(y)14 b(storage)g(order)h (\015ag)e(\(in)o(teger\))117 2093 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff(arra)o(y)14 b(elemen)o(t)f(datat)o(yp)q(e)h(\(handle\))117 2168 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f (\(handle\))75 2292 y Fp(int)23 b(MPI)p 245 2292 15 2 v 17 w(Type)p 358 2292 V 17 w(subarray\(int)f(ndims,)h(int)h(*sizes,)e (int)i(*subsizes,)e(int)i(*starts,)393 2349 y(int)g(order,)f(MPI)p 731 2349 V 16 w(Datatype)g(oldtype,)g(MPI)p 1249 2349 V 17 w(Datatype)g(*newtype\))75 2435 y(MPI)p 150 2435 V 17 w(TYPE)p 263 2435 V 16 w(SUBARRAY\(NDIMS,)f(SIZES,)h(SUBSIZES,)g (STARTS,)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)393 2491 y(IERROR\))170 2548 y(INTEGER)g(NDIMS,)g(SIZES\(*\),)g(SUBSIZES\(*\),)f(STARTS\(*\),)h (ORDER,)g(OLDTYPE,)170 2604 y(NEWTYPE,)g(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 36 38 36 37 bop 75 -100 a Ft(36)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)166 49 y Ft(The)23 b(lo)q(cal)h(sub-arra)o(y)e(\014let)o(yp)q(e)i (constructor)e(creates)h(an)g Fq(MPI)f Ft(datat)o(yp)q(e)g(describing)j (an)e(n-)75 106 y(dimensional)c(sub-arra)o(y)c(of)h(a)g(n-dimensional)j (arra)o(y)l(.)k(The)17 b(sub-arra)o(y)e(ma)o(y)h(b)q(e)h(situated)g(an) o(ywhere)75 162 y(within)f(the)f(full)i(arra)o(y)l(,)c(and)j(ma)o(y)e (b)q(e)i(of)e(an)o(y)h(size)h(up)f(to)g(the)g(size)h(of)e(the)h(larger) g(arra)o(y)f(as)h(long)g(as)g(it)75 219 y(is)h(con\014ned)i(within)f (this)f(arra)o(y)l(.)21 b(This)c(t)o(yp)q(e)f(constructor)f (facilitates)i(accessing)g(arra)o(ys)d(distributed)75 275 y(in)i(blo)q(c)o(ks)g(among)e(pro)q(cesses)i(to)f(a)f(single)j (\014le)f(that)f(con)o(tains)g(the)g(global)h(arra)o(y)l(.)166 332 y(This)h(t)o(yp)q(e)f(constructor)f(can)i(handle)g(arra)o(ys)e (with)i(an)f(arbitrary)g(n)o(um)o(b)q(er)g(of)g(dimensions)i(and)75 388 y(w)o(orks)11 b(for)h(b)q(oth)g(C)g(and)h(F)o(OR)l(TRAN)g(ordered)f (matrices)g(\(i.e.,)g(ro)o(w-ma)s(jor)e(or)i(column-ma)s(jor\).)18 b(Note)75 444 y(that)d(a)g(C)g(program)f(ma)o(y)g(use)i(F)o(OR)l(TRAN)g (order)f(and)h(a)f(F)o(OR)l(TRAN)h(program)e(ma)o(y)h(use)g(C)h(order,) 75 501 y(the)e(names)h(are)f(only)g(mean)o(t)g(to)g(sp)q(ecify)i(the)e (default)h(language)f(b)q(eha)o(vior.)20 b(T)l(o)14 b(sp)q(ecify)i (whic)o(h)f(order)75 557 y(the)g(arra)o(y)f(is)i(stored,)e(the)i Fq(o)o(rder)e Ft(argumen)o(t)g(is)i(set)f(to)g(one)g(of)g(the)g(follo)o (wing:)143 645 y Fh(\017)23 b Fg(MPI)p 266 645 13 2 v 14 w(C)p 307 645 V 14 w(ORDER)15 b Ft(-)g(The)g(ordering)h(used)g(b)o (y)f(C)g(arra)o(ys,)e(\(i.e.,)i(ro)o(w-ma)s(jor)e(order\))143 736 y Fh(\017)23 b Fg(MPI)p 266 736 V 14 w(F)o(ORTRAN)p 473 736 V 14 w(ORDER)13 b Ft(-)i(The)f(ordering)h(used)g(b)o(y)f(F)o (OR)l(TRAN)i(arra)o(ys,)c(\(i.e.,)i(column-ma)s(jor)189 793 y(order\))675 1572 y @beginspecial 195 @llx 269 @lly 416 @urx 523 @ury 1440 @rwi @setspecial %%BeginDocument: figures/io-array-file.eps /$F2psDict 32 dict def $F2psDict begin $F2psDict /mtrx matrix put end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 57.000000 681.000000 translate 0.900000 -0.900000 scale % Polyline newpath 163 177 moveto 215 177 lineto 215 380 lineto 163 380 lineto closepath 0.95 setgray gsave fill grestore 0.00 setgray % Polyline newpath 217 177 moveto 268 177 lineto 268 379 lineto 217 379 lineto closepath gsave fill grestore % Polyline newpath 270 177 moveto 321 177 lineto 321 379 lineto 270 379 lineto closepath 0.60 setgray gsave fill grestore 0.00 setgray 1.000 setlinewidth % Polyline newpath 323 178 moveto 374 178 lineto 374 377 lineto 323 377 lineto closepath 0.35 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 294 439 moveto 334 439 lineto 334 458 lineto 294 458 lineto closepath 0.35 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 294 400 moveto 335 400 lineto 335 418 lineto 294 418 lineto closepath 0.60 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 153 438 moveto 195 438 lineto 195 456 lineto 153 456 lineto closepath gsave fill grestore stroke % Polyline newpath 154 400 moveto 194 400 lineto 194 420 lineto 154 420 lineto closepath 0.95 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 216 176 moveto 216 380 lineto stroke % Polyline newpath 268 176 moveto 268 380 lineto stroke % Polyline newpath 322 176 moveto 322 380 lineto stroke 4.000 setlinewidth % Polyline newpath 163 176 moveto 163 379 lineto 375 379 lineto 375 176 lineto closepath stroke /fn0.12 /Times-Roman findfont 13.333334 scalefont def fn0.12 setfont (Process 0) 202 414 moveto 1 -1 scale show 1 -1 scale (Process 2) 348 414 moveto 1 -1 scale show 1 -1 scale (Process 1) 204 452 moveto 1 -1 scale show 1 -1 scale (Process 3) 349 452 moveto 1 -1 scale show 1 -1 scale $F2psEnd %%EndDocument @endspecial 595 1670 a(Figure)h(10.5:)k(Example)d(arra)o(y)e(\014le)i (la)o(y)o(out)166 1772 y(F)l(or)11 b(example,)j(assume)e(w)o(e)f(are)h (writing)h(out)f(a)f(100x100)g(2D)g(arra)o(y)g(of)h(double)h(precision) h(\015oating)75 1828 y(p)q(oin)o(t)i(n)o(um)o(b)q(ers)g(that)e(is)i (distributed)h(among)e(4)g(pro)q(cesses)g(suc)o(h)h(that)f(eac)o(h)g (pro)q(cess)h(has)f(a)g(blo)q(c)o(k)h(of)75 1885 y(25)e(columns)i (\(e.g.,)d(pro)q(cess)i(0)g(has)f(columns)i(0-24,)e(pro)q(cess)h(1)g (has)f(columns)i(25-49,)d(etc.;)i(see)g(Figure)75 1941 y(10.5)f(\).)19 b(T)l(o)c(create)g(the)h(\014let)o(yp)q(es)g(for)e(eac) o(h)i(pro)q(cess)f(one)h(could)g(use)f(the)h(follo)o(wing)g(C)e (program:)147 2040 y Fp(double)23 b(subarray[100][25];)147 2097 y(MPI_Datatype)f(filetype;)147 2153 y(int)h(sizes[2],)g (subsizes[2],)f(starts[2];)147 2209 y(int)h(rank;)147 2322 y(MPI_Comm_rank\(MPI_COMM_)o(WORLD,)d(&rank\);)147 2379 y(sizes[0]=100;)i(sizes[1]=100;)g(subsizes[0]=100;)147 2435 y(subsizes[1]=25;)f(starts[0]=0;)i(starts[1]=rank*25;)147 2548 y(MPI_Type_subarray\(2,)e(sizes,)i(subsizes,)f(starts,)h (MPI_C_ORDER,)600 2605 y(MPI_DOUBLE,)f(&filetype\);)166 2704 y Ft(Or,)15 b(equiv)m(alen)o(tly)i(in)f(F)o(OR)l(TRAN:)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 37 39 37 38 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(37)242 49 y Fp(double)23 b(precision)g(subarray\(100,25\))242 106 y(integer)g(filetype,)g(rank,)g(ierror)242 162 y(integer)g (sizes\(2\),)g(subsizes\(2\),)f(starts\(2\))242 275 y(call)h (MPI_COMM_RANK\(MPI_COMM_WORLD,)d(rank,)j(ierror\))242 332 y(sizes\(1\)=100)242 388 y(sizes\(2\)=100)242 444 y(subsizes\(1\)=100)242 501 y(subsizes\(2\)=25)242 557 y(starts\(1\)=1)242 614 y(starts\(2\)=rank*25)f(+)h(1)242 727 y(call)g(MPI_TYPE_SUBARRAY\(2,)e(sizes,)j(subsizes,)e(starts,)194 783 y($)287 b(MPI_FORTRAN_ORDER,)21 b(MPI_DOUBLE_PRECISION,)194 840 y($)287 b(filetype,)22 b(ierror\))166 946 y Ft(The)c(generated)f (\014let)o(yp)q(e)i(will)g(then)f(describ)q(e)h(the)f(p)q(ortion)g(of)f (the)g(\014le)i(con)o(tained)f(within)h(the)75 1002 y(pro)q(cess')c (subarra)o(y)g(with)g(holes)h(for)f(the)g(space)h(tak)o(en)f(b)o(y)g (the)g(other)g(pro)q(cesses.)21 b(Figure)16 b(10.6)e(sho)o(ws)75 1059 y(the)h(\014let)o(yp)q(e)i(created)e(for)f(pro)q(cess)i(1.)675 1626 y @beginspecial 198 @llx 304 @lly 414 @urx 487 @ury 1440 @rwi @setspecial %%BeginDocument: figures/io-array-p1.eps /$F2psDict 32 dict def $F2psDict begin $F2psDict /mtrx matrix put end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 54.000000 681.000000 translate 0.900000 -0.900000 scale % Polyline newpath 230 217 moveto 269 217 lineto 269 380 lineto 230 380 lineto closepath 0.00 setgray gsave fill grestore 1.000 setlinewidth % Polyline newpath 160 400 moveto 199 400 lineto 199 420 lineto 160 420 lineto closepath gsave fill grestore stroke % Polyline newpath 229 216 moveto 229 381 lineto stroke % Polyline newpath 270 216 moveto 270 381 lineto stroke 4.000 setlinewidth % Polyline newpath 188 216 moveto 188 380 lineto 353 380 lineto 353 216 lineto closepath stroke 5.000 setlinewidth % Polyline newpath 319 398 moveto 358 398 lineto 358 418 lineto 319 418 lineto closepath stroke /fn0.12 /Times-Roman findfont 13.333334 scalefont def fn0.12 setfont (Holes) 370 413 moveto 1 -1 scale show 1 -1 scale (MPI_DOUBLE) 215 415 moveto 1 -1 scale show 1 -1 scale $F2psEnd %%EndDocument @endspecial 453 1724 a(Figure)f(10.6:)k(Example)d(subarra)o(y)e (\014let)o(yp)q(e)i(for)f(pro)q(cess)g(1)75 1895 y Fl(10.6.4)49 b(Distributed)16 b(Arra)o(y)g(Filet)o(yp)q(e)g(Constructo)o(rs)75 1981 y Ft(The)g(distributed)i(arra)o(y)d(\014let)o(yp)q(e)i (constructors)e(create,)h(for)f(eac)o(h)h(pro)q(cess)h(in)g(a)e(group,) h(a)g(\(p)q(ossibly)75 2038 y(di\013eren)o(t\))j(\014let)o(yp)q(e.)34 b(When)20 b(a)g(collectiv)o(e)h(I/O)f(op)q(eration)g(\(with)f(iden)o (tical)j(o\013sets\))c(is)i(p)q(erformed)75 2094 y(on)d(a)f(\014le)i (op)q(ened)g(with)f(the)g(returned)g(\014let)o(yp)q(e,)h(an)f(HPF-lik)o (e)h([4)o(])f(distribution)h(of)f(the)f(\014le)i(data)e(is)75 2150 y(realized.)26 b(Ho)o(w)o(ev)o(er,)16 b(unlik)o(e)i(in)f(HPF,)f (the)h(storage)e(order)i(ma)o(y)f(b)q(e)h(sp)q(eci\014ed)i(for)d(C)g (arra)o(ys)g(as)g(w)o(ell)75 2207 y(as)f(for)f(F)o(OR)l(TRAN)i(arra)o (ys.)75 2327 y Fq(MPI)p 160 2327 14 2 v 16 w(T)l(yp)q(e)p 268 2327 V 17 w(da)o(rra)o(y)e(\(collective\))75 2413 y Ft(Distribution)i(of)f(an)g(N-dimensional)j(arra)o(y)c(on)o(to)g(a)h (P-dimensional)i(grid)f(of)e(logical)j(pro)q(cessors:)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 38 40 38 39 bop 75 -100 a Ft(38)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y\(comm,)14 b(ndims,)i(dsizes,)h(distribs,)g(da)o (rgs,)e(p)q(dims,)h(psizes,)h(o)o(rder,)d(oldt)o(yp)q(e,)j(new-)75 106 y(t)o(yp)q(e\))117 183 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to)h(b)q(e)i(used)g(in)e(MPI)p 1655 183 13 2 v 15 w(Op)q(en)i(\(han-)905 239 y(dle\))117 314 y(IN)155 b Fq(ndims)476 b Ff([SAME])13 b(n)o(um)o(b)q(er)g(of)g (arra)o(y)g(dimensions)f(\(nonnegativ)o(e)i(in-)905 370 y(teger\))117 445 y(IN)155 b Fq(dsizes)479 b Ff([SAME])19 b(size)i(of)e(distributed)i(arra)o(y)e(in)g(eac)o(h)i(dimension)905 501 y(\(arra)o(y)14 b(of)f(nonnegativ)o(e)h(in)o(teger\))117 576 y(IN)155 b Fq(distribs)453 b Ff([SAME])16 b(distribution)g(of)g (arra)o(y)g(in)g(eac)o(h)h(dimension,)e(e.g.)905 632 y(BLOCK)g(or)f(CYCLIC)g(\(arra)o(y)f(of)h(in)o(teger\))117 707 y(IN)155 b Fq(da)o(rgs)487 b Ff([SAME])14 b(distribution)f(argumen) o(t)g(of)g(dimension,)905 763 y(e.g.)25 b(BLOCK\(dargs\))18 b(or)e(CYCLIC\(dargs\))h(\(arra)o(y)f(of)g(in-)905 820 y(teger\))117 894 y(IN)155 b Fq(p)q(dims)475 b Ff([SAME])20 b(n)o(um)o(b)q(er)f(of)h(pro)q(cessor)i(grid)d(dimensions)g(\(non-)905 951 y(negativ)o(e)14 b(in)o(teger\))117 1025 y(IN)155 b Fq(psizes)479 b Ff([SAME])10 b(size)h(of)e(pro)q(cessor)j(grid)e(in)g (eac)o(h)h(dimension)d(\(arra)o(y)905 1082 y(of)13 b(nonnegativ)o(e)h (in)o(teger\))117 1156 y(IN)155 b Fq(o)o(rder)490 b Ff([SAME])14 b(arra)o(y)f(storage)i(order)f(\015ag)g(\(in)o(teger\))117 1231 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 1305 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 1430 y Fp(int)23 b(MPI)p 245 1430 15 2 v 17 w(Type)p 358 1430 V 17 w(darray\(MPI)p 615 1430 V 16 w(Comm)g(comm,)g(int)h(ndims,)f(int)g(*dsizes,)g(int)g (*distribs,)393 1486 y(int)h(*dargs,)f(int)g(pdims,)g(int)g(*psizes,)g (int)h(order,)393 1543 y(MPI)p 468 1543 V 17 w(Datatype)f(oldtype,)f (MPI)p 986 1543 V 17 w(Datatype)h(*newtype\))75 1629 y(MPI)p 150 1629 V 17 w(TYPE)p 263 1629 V 16 w(DARRAY\(COMM,)g(NDIMS,)g (DSIZES,)g(DISTRIBS,)f(DARGS,)h(PDIMS,)g(PSIZES,)g(ORDER,)393 1686 y(OLDTYPE,)g(NEWTYPE,)g(IERROR\))170 1742 y(INTEGER)g(COMM,)h (NDIMS,)f(DSIZES\(*\),)f(DISTRIBS\(*\),)g(DARGS\(*\),)h(PDIMS,)g (PSIZES\(*\))170 1799 y(INTEGER)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR)166 1885 y Fq(MPI)p 251 1885 14 2 v 16 w(TYPE)p 384 1885 V 17 w(D)o(ARRA)l(Y)f Ft(generates)f(the)h(\014let)o(yp)q(es)g (corresp)q(onding)g(to)f(the)g(distribution)i(of)e(an)75 1941 y Fq(ndims)p Ft(-dimensional)15 b(arra)o(y)c(of)h Fq(oldt)o(yp)q(e)h Ft(elemen)o(ts)g(on)o(to)e(a)h Fq(p)q(dims)p Ft(-dimensional)j(grid)e(of)f(logical)h(pro)q(ces-)75 1998 y(sors.)18 b(F)l(or)10 b(a)g(call)h(to)f Fq(MPI)p 507 1998 V 16 w(TYPE)p 640 1998 V 17 w(D)o(ARRA)l(Y)i Ft(to)d(b)q(e)j(correct,)e(the)h(equation)1365 1966 y Fd(Q)1404 1976 y Fc(pdims)p Fb(\000)p Fa(1)1404 2011 y Fc(i)p Fa(=0)1554 1998 y Fi(psiz)r(es)p Ft([)p Fi(i)p Ft(])h(=)h Fi(siz)r(e)p Ft(,)75 2054 y(where)i Fi(siz)r(e)h Ft(denotes)f(the)h(size)g(of)e(comm)o(unicator)h Fq(comm)p Ft(,)e(m)o(ust)i(b)q(e)h(satis\014ed.)166 2111 y(Eac)o(h)f(dimension)i (of)e(the)g(arra)o(y)f(is)i(distributed)g(in)g(one)g(of)e(three)i(w)o (a)o(ys:)143 2202 y Fh(\017)23 b Fg(MPI)p 266 2202 13 2 v 14 w(DISTRIBUTE)p 522 2202 V 13 w(BLOCK)15 b Ft(-)g(Blo)q(c)o(k)h (distribution)143 2294 y Fh(\017)23 b Fg(MPI)p 266 2294 V 14 w(DISTRIBUTE)p 522 2294 V 13 w(CYCLIC)14 b Ft(-)h(Cyclic)i (distribution)143 2387 y Fh(\017)23 b Fg(MPI)p 266 2387 V 14 w(DISTRIBUTE)p 522 2387 V 13 w(NONE)16 b Ft(-)f(Dimension)h(not)f (distributed.)166 2478 y(T)l(o)e(sp)q(ecify)i(a)f(default)g (distribution)i(argumen)o(t,)c(the)i(constan)o(t)f Fg(MPI)p 1365 2478 V 14 w(DISTRIBUTE)p 1621 2478 V 14 w(DFL)m(T)p 1736 2478 V 15 w(ARG)f Ft(is)75 2534 y(used.)20 b(The)c(distribution)h (argumen)o(t)d(for)h(a)g(dimension)i(that)d(is)i(not)f(distributed)h (is)g(ignored.)166 2591 y(F)l(or)j(example,)i(ARRA)l(Y\(CYCLIC\(15\)\)) e(corresp)q(onds)h(to)e Fg(MPI)p 1316 2591 V 15 w(DISTRIBUTE)p 1573 2591 V 13 w(CYCLIC)g Ft(with)h(a)75 2647 y(distribution)13 b(argumen)o(t)d(of)h(15,)g(and)h(ARRA)l(Y\(BLOCK\))h(corresp)q(onds)f (to)e Fg(MPI)p 1471 2647 V 15 w(DISTRIBUTE)p 1728 2647 V 13 w(BLOCK)75 2704 y Ft(with)16 b(a)e(distribution)j(argumen)o(t)e (of)f Fg(MPI)p 796 2704 V 15 w(DISTRIBUTE)p 1053 2704 V 13 w(DFL)m(T)p 1167 2704 V 15 w(ARG)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 39 41 39 40 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(39)166 49 y(The)17 b Fq(o)o(rder)f Ft(argumen)o(t)g(is)h(used)g (as)f(in)i Fq(MPI)p 928 49 14 2 v 16 w(TYPE)p 1061 49 V 17 w(SUBARRA)l(Y)h Ft(to)d(sp)q(ecify)i(the)f(storage)e(order)75 106 y(for)h(m)o(ultidimensional)k(arra)o(ys.)k(Therefore,)17 b(arra)o(ys)f(describ)q(ed)j(b)o(y)e(this)h(t)o(yp)q(e)f(constructor)f (ma)o(y)g(b)q(e)75 162 y(stored)11 b(in)h(ro)o(w-ma)s(jor)d(or)i (column-ma)s(jor)g(order.)18 b(V)l(alid)13 b(v)m(alues)g(for)d Fq(o)o(rder)h Ft(are)g Fg(MPI)p 1517 162 13 2 v 14 w(F)o(ORTRAN)p 1724 162 V 14 w(ORDER)75 219 y Ft(or)k Fg(MPI)p 208 219 V 14 w(C)p 249 219 V 14 w(ORDER)p Ft(.)166 275 y(F)l(or)f(example,)i (let)g(us)f(generate)g(the)g(\014let)o(yp)q(es)i(corresp)q(onding)f(to) e(the)i(HPF)e(distribution:)218 369 y Fp()23 b(FILEARRAY\(100,)f(200,)h(300\))75 425 y(!HPF$)g(PROCESSORS)g (PROCESSES\(2,)f(3\))75 482 y(!HPF$)h(DISTRIBUTE)g (FILEARRAY\(CYCLIC\(10\),)e(*,)i(BLOCK\))g(ONTO)h(PROCESSES)75 576 y Ft(This)13 b(can)g(b)q(e)g(ac)o(hiev)o(ed)g(b)o(y)f(the)h(follo)o (wing)g(F)o(OR)l(TRAN)g(co)q(de,)g(assuming)g(there)g(will)h(b)q(e)f (six)g(pro)q(cesses)75 632 y(attac)o(hed)i(to)f(the)h(run:)170 726 y Fp(ndims)24 b(=)f(3)170 782 y(dsizes\(1\))g(=)h(100)170 839 y(distribs\(1\))f(=)h(MPI_DISTRIBUTE_CYCLIC)170 895 y(dargs\(1\))f(=)h(10)170 952 y(dsizes\(2\))f(=)h(200)170 1008 y(distribs\(2\))f(=)h(MPI_DISTRIBUTE_NONE)170 1065 y(dargs\(2\))f(=)h(0)170 1121 y(dsizes\(3\))f(=)h(300)170 1178 y(distribs\(3\))f(=)h(MPI_DISTRIBUTE_BLOCK)170 1234 y(dargs\(3\))f(=)h(MPI_DISTRIBUTE_DFLT_ARG)170 1290 y(pdims)g(=)f(2)170 1347 y(psizes\(1\))g(=)h(2)170 1403 y(psizes\(2\))f(=)h(3)170 1460 y(call)g(MPI_TYPE_DARRAY\(MPI_COMM)o(_WORLD,)c(ndims,)j(dsizes,)g (distribs,)g(dargs,)147 1516 y($)119 b(pdims,)23 b(psizes,)g (MPI_FORTRAN_ORDER,)e(oldtype,)i(newtype,)g(ierr\))75 1636 y Fq(MPI)p 160 1636 14 2 v 16 w(T)l(yp)q(e)p 268 1636 V 17 w(da)o(rra)o(y)p 403 1636 V 15 w(2d)16 b(\(collective\))75 1722 y Ft(Distribution)g(of)f(a)g Fk(two)p Ft(-dimensional)j(arra)o(y:) -32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 40 42 40 41 bop 75 -100 a Ft(40)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y)p 486 49 V 17 w(2D\(comm,)13 b(dsize1,)i (distrib1,)h(da)o(rg1,)e(psize1,)i(dsize2,)f(distrib2,)h(da)o(rg2,)e (psize2,)75 106 y(o)o(rder,)g(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 183 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to) h(b)q(e)i(used)g(in)e(MPI)p 1655 183 13 2 v 15 w(Op)q(en)i(\(han-)905 239 y(dle\))117 315 y(IN)155 b Fq(dsize1)473 b Ff([SAME])21 b(size)g(of)f(distributed)i(arra)o(y)f(in)f(\014rst)i(dimension)905 371 y(\(nonnegativ)o(e)14 b(in)o(teger\))117 446 y(IN)155 b Fq(distrib1)447 b Ff([SAME])14 b(distribution)f(for)h(\014rst)g (dimension)e(\(in)o(teger\))117 521 y(IN)155 b Fq(da)o(rg1)481 b Ff([SAME])12 b(distribution)g(argumen)o(t)f(for)h(\014rst)h (dimension)d(\(in-)905 578 y(teger\))117 653 y(IN)155 b Fq(psize1)473 b Ff([SAME])12 b(size)i(of)e(pro)q(cessor)j(grid)d(in)g (\014rst)i(dimension)d(\(in)o(te-)905 709 y(ger\))117 784 y(IN)155 b Fq(dsize2)473 b Ff([SAME])14 b(size)g(of)g(distributed)g (arra)o(y)g(in)g(second)h(dimension)905 841 y(\(nonnegativ)o(e)f(in)o (teger\))117 916 y(IN)155 b Fq(distrib2)447 b Ff([SAME])14 b(distribution)f(for)h(second)g(dimension)f(\(in)o(teger\))117 991 y(IN)155 b Fq(da)o(rg2)481 b Ff([SAME])18 b(distribution)f(argumen) o(t)g(for)h(second)h(dimension)905 1048 y(\(in)o(teger\))117 1123 y(IN)155 b Fq(psize2)473 b Ff([SAME])11 b(size)g(of)g(pro)q (cessor)h(grid)f(in)f(second)i(dimension)d(\(in-)905 1179 y(teger\))117 1254 y(IN)155 b Fq(o)o(rder)490 b Ff([SAME])14 b(arra)o(y)f(storage)i(order)f(\015ag)g(\(in)o(teger\))117 1329 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 1405 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 1529 y Fp(int)23 b(MPI)p 245 1529 15 2 v 17 w(Type)p 358 1529 V 17 w(darray)p 519 1529 V 16 w(2d\(MPI)p 679 1529 V 17 w(Comm)g(comm,)g(int)h(dsize1,) e(int)i(distrib1,)f(int)g(darg1,)393 1585 y(int)h(psize1,)f(int)g (dsize2,)g(int)g(distrib2,)g(int)g(darg2,)g(int)h(psize2,)393 1642 y(int)g(order,)f(MPI)p 731 1642 V 16 w(Datatype)g(oldtype,)g(MPI)p 1249 1642 V 17 w(Datatype)g(*newtype\))75 1728 y(MPI)p 150 1728 V 17 w(TYPE)p 263 1728 V 16 w(DARRAY)p 423 1728 V 17 w(2D\(COMM,)g(DSIZE1,)f(DISTRIB1,)h(DARG1,)g(PSIZE1,)g(DSIZE2,)g (DISTRIB2,)393 1785 y(DARG2,)g(PSIZE2,)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR\))170 1841 y(INTEGER)g(COMM,)h(DSIZE1,)e(DISTRIB1,)h(DARG1,)g (PSIZE1,)g(DSIZE2,)g(DISTRIB2,)g(DARG2)170 1898 y(INTEGER)g(PSIZE2,)g (ORDER,)g(OLDTYPE,)g(NEWTYPE,)g(IERROR)166 1984 y Ft(This)12 b(call)h(is)g(a)e(shorthand)h(for)f(the)h(t)o(w)o(o-dimensional)h (distributed)g(arra)o(y)e(case)h(and)g(is)g(equiv)m(alen)o(t)75 2041 y(to:)170 2134 y Fp(dsize[0]=dsize1;)22 b(distrib[0]=distrib1;)f (dargs[0]=darg1;)h(psize[0])h(=)h(psize1;)170 2191 y(dsize[1]=dsize2;)e (distrib[1]=distrib2;)f(dargs[1]=darg2;)h(psize[1])h(=)h(psize2;)170 2247 y(MPI_Type_darray\(comm,)d(2,)j(dsize,)f(distrib,)g(dargs,)g(2,)g (psize,)576 2304 y(order,)g(oldtype,)g(&newtype\);)75 2424 y Fq(MPI)p 160 2424 14 2 v 16 w(T)l(yp)q(e)p 268 2424 V 17 w(da)o(rra)o(y)p 403 2424 V 15 w(blo)q(ck)16 b(\(collective\))75 2510 y Ft(BLOCK)g(distribution)h(of)e(a)g (one-dimensional)i(arra)o(y:)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 41 43 41 42 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(41)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y)p 486 49 V 17 w(BLOCK\(comm,)13 b(dsize,)i(da)o(rg,)g(oldt)o(yp)q(e,)g(newt)o(yp)q(e\))117 126 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to) h(b)q(e)i(used)g(in)e(MPI)p 1655 126 13 2 v 15 w(Op)q(en)i(\(han-)905 183 y(dle\))117 258 y(IN)155 b Fq(dsize)496 b Ff([SAME])10 b(size)h(of)f(distributed)h(arra)o(y)f(\(nonnegativ)o(e)g(in)o(teger\)) 117 333 y(IN)155 b Fq(da)o(rg)504 b Ff([SAME])21 b(distribution)f (argumen)o(t,)i(e.g.)39 b(BLOCK\(darg\))905 390 y(\(in)o(teger\))117 465 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp)q (e)h(\(handle\))117 540 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 664 y Fp(int)23 b(MPI)p 245 664 15 2 v 17 w(Type)p 358 664 V 17 w(darray)p 519 664 V 16 w(block\(MPI)p 751 664 V 16 w(Comm)g(comm,)h(int)f(dsize,) g(int)h(darg,)393 721 y(MPI)p 468 721 V 17 w(Datatype)f(oldtype,)f(MPI) p 986 721 V 17 w(Datatype)h(*newtype\))75 807 y(MPI)p 150 807 V 17 w(TYPE)p 263 807 V 16 w(DARRAY)p 423 807 V 17 w(BLOCK\(COMM,)f(DSIZE,)h(DARG,)g(OLDTYPE,)g(NEWTYPE,)g(IERROR\)) 170 864 y(INTEGER)g(COMM,)h(DSIZE,)f(DARG,)g(OLDTYPE,)g(NEWTYPE,)f (IERROR)166 950 y Fq(MPI)p 251 950 14 2 v 16 w(TYPE)p 384 950 V 17 w(D)o(ARRA)l(Y)p 577 950 V 17 w(BLOCK)14 b Ft(generates)f(the)h(\014let)o(yp)q(es)g(corresp)q(onding)h(to)e(the) g(BLOCK)i(dis-)75 1007 y(tribution)h(of)f(a)g(one-dimensional)i Fq(dsize)f Ft(elemen)o(t)g(arra)o(y)e(of)h Fq(oldt)o(yp)q(e)p Ft(.)166 1063 y(This)h(call)g(is)g(a)f(shorthand)g(for:)170 1157 y Fp(MPI_Comm_size\(comm,)22 b(&size\);)170 1213 y(distrib)h(=)h(MPI_DISTRIBUTE_BLOCK;)170 1270 y (MPI_Type_darray\(comm,)d(1,)j(dsize,)f(distrib,)g(darg,)g(1,)h(size,) 576 1326 y(order,)f(oldtype,)g(&newtype\);)166 1420 y Ft(Note)10 b(that)f(order)h(is)h(not)e(sp)q(eci\014ed)k(for)c(this)i (constructor,)f(nor)g(for)f Fq(MPI)p 1395 1420 V 16 w(TYPE)p 1528 1420 V 17 w(D)o(ARRA)l(Y)p 1721 1420 V 18 w(CYCLIC)p Ft(,)75 1477 y(b)q(ecause)16 b(it)g(is)f(not)g(meaningful)i(for)d(1D)h (arra)o(ys.)75 1597 y Fq(MPI)p 160 1597 V 16 w(T)l(yp)q(e)p 268 1597 V 17 w(da)o(rra)o(y)p 403 1597 V 15 w(cyclic)h(\(collective\)) 75 1682 y Ft(CYCLIC)g(distribution)g(of)f(a)g(one-dimensional)j(arra)o (y:)75 1833 y Fq(MPI)p 160 1833 V 16 w(TYPE)p 293 1833 V 17 w(D)o(ARRA)l(Y)p 486 1833 V 17 w(CYCLIC\(comm,)13 b(dsize,)i(da)o(rg,)f(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 1911 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e (to)h(b)q(e)i(used)g(in)e(MPI)p 1655 1911 13 2 v 15 w(Op)q(en)i(\(han-) 905 1967 y(dle\))117 2042 y(IN)155 b Fq(dsize)496 b Ff([SAME])10 b(size)h(of)f(distributed)h(arra)o(y)f(\(nonnegativ)o(e)g(in)o(teger\)) 117 2117 y(IN)155 b Fq(da)o(rg)504 b Ff([SAME])19 b(distribution)f (argumen)o(t,)h(e.g.)34 b(CYCLIC\(darg\))905 2174 y(\(in)o(teger\))117 2249 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 2324 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 2448 y Fp(int)23 b(MPI)p 245 2448 15 2 v 17 w(Type)p 358 2448 V 17 w(darray)p 519 2448 V 16 w(cyclic\(MPI)p 775 2448 V 16 w(Comm)g(comm,)h(int)f (dsize,)g(int)g(darg,)393 2505 y(MPI)p 468 2505 V 17 w(Datatype)g(oldtype,)f(MPI)p 986 2505 V 17 w(Datatype)h(*newtype\))75 2591 y(MPI)p 150 2591 V 17 w(TYPE)p 263 2591 V 16 w(DARRAY)p 423 2591 V 17 w(CYCLIC\(COMM,)f(DSIZE,)h(DARG,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR\))170 2648 y(INTEGER)g(COMM,)h(DSIZE,)f(DARG,)g(OLDTYPE,)g (NEWTYPE,)f(IERROR\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 42 44 42 43 bop 75 -100 a Ft(42)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(TYPE)p 384 49 V 17 w(D)o(ARRA)l(Y)p 577 49 V 17 w(CYCLIC)11 b Ft(generates)g(the)g (\014let)o(yp)q(es)i(corresp)q(onding)f(to)e(the)h(CYCLIC)h(dis-)75 106 y(tribution)k(of)f(a)g(one-dimensional)i Fq(dsize)f Ft(elemen)o(t)g(arra)o(y)e(of)h Fq(oldt)o(yp)q(e)p Ft(.)166 162 y(This)h(call)g(is)g(a)f(shorthand)g(for:)170 256 y Fp(MPI_Comm_size\(comm,)22 b(&size\);)170 312 y(distrib)h(=)h (MPI_DISTRIBUTE_CYCLIC;)170 369 y(MPI_Type_darray\(comm,)d(1,)j(dsize,) f(distrib,)g(darg,)g(1,)h(size,)576 425 y(order,)f(oldtype,)g (&newtype\);)75 569 y Fm(10.7)60 b(Prop)r(osal:)26 b(Simpli\014ed)19 b(Op)r(en)75 726 y Fp(Several)k(scientific)f(users)i(have)f(complained) f(of)i(the)f(steep)h(learning)75 783 y(curve)f(of)h(this)f(interface)g (and)g(have)g(suggested)g(we)h(provide)f(simplified)75 839 y("wrapper")g(functions.)46 b(Rather)23 b(than)g(adding)g(extra)h (functions)e(to)i(the)75 896 y(specification,)e(the)h(following)g (proposal)g(was)g(made)h(which)f(simplifies)75 952 y(I/O)g(for)h(the)f (user)h(that)f(does)g(not)h(need)f(to)h(use)f(the)h(more)f (sophisticated)75 1009 y(features,)g(and)g(also)g(allows)g(the)h (filetype)f(to)g(be)h(changed)f(without)75 1065 y(re-opening)f(the)i (file)f(or)h(using)f(file)g(control.)75 1178 y(Break)g(MPI_OPEN)g(in)h (to)f(two)h(calls:)46 b(MPI_OPEN)23 b(and)h(MPI_LAYOUT.)75 1291 y(MPI_OPEN\(comm,)e(filename,)h(amode,)g(hints,)g(fh\))75 1404 y(Opens)g("filename")g(with:)75 1517 y(disp)g(=)h(0)75 1573 y(etype)f(=)h(MPI_BYTE)75 1630 y(filetype)f(=)g(MPI_BYTE)75 1743 y(This)g(essentially)g(mimic's)g(a)g(UNIX)h(open)f(call.)75 1856 y(MPI_LAYOUT\(fh,)f(disp,)h(etype,)g(filetype,)g(hints\))75 1968 y(Changes)g(the)g(process's)g(view)g(of)h(the)f(file.)47 b(It)24 b(is)g(assumed)e(to)75 2025 y(be)i(an)f(expensive)g(operation,) f(and)i(is)f(collective.)47 b(It)23 b(would)75 2081 y(be)h(allowed)e (to)i(be)g(done)f(at)h(any)f(time)g(on)h(an)g(open)f(file)g(handle.)75 2194 y(An)h(MPI_GET_LAYOUT)e(routine)g(could)i(also)f(be)h(provided)e (\(see)i(the)f(proposed)75 2251 y(replacemenet)f(for)i (MPI_FILE_CONTROL)d(above\).)75 2364 y(By)j(adding)f(this)g(change,)g (a)h(user)f(can)g(open,)h(read,)f(and)g(write)g(a)h(file)75 2420 y(without)f(having)g(to)h(know)f(anything)g(about)g(etypes,)g (filetypes,)f(displacement,)75 2477 y(etc.)47 b(It)24 b(also)f(means)g(that)h(you)f(can)g(open)h(a)g(file,)f(write)g(the)g (header)g(using)75 2533 y(etype=filetype=MPI_BYTE,)e(and)i(then,)g (after)g(the)h(header)f(has)g(been)h(written,)75 2589 y(set)f(the)h(layout)f(to)h(a)f(more)h(complicated)e(filetype)h(with)g (a)h(displacement.)75 2646 y(You)f(need)h(not)f(close)g(and)h(re-open)f (the)g(file)h(or)f(use)h(a)f(file)h(control)f(command.)75 2702 y(This)g(makes)g(it)h(easier)f(for)h(people)f(that)g(plan)g(to)h (write)f(simple)g(headers)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 43 45 43 44 bop 75 -100 a Fn(10.7.)34 b(PR)o(OPOSAL:)16 b(SIMPLIFIED)g(OPEN) 929 b Ft(43)75 49 y Fp(on)24 b(to)f(the)h(start)f(of)g(their)h(files.) -32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 44 46 44 45 bop 75 381 a Fr(Bibliograph)m(y)98 604 y Ft([1])21 b(F)l(ern)11 b(E.)f(Basso)o(w.)i(Installing,)h(managing,)e(and)g(using) g(the)g(IBM)g(AIX)g(P)o(arallel)h(I/O)f(File)g(System.)168 660 y(IBM)16 b(Do)q(cumen)o(t)f(Num)o(b)q(er)h(SH34-6065-00,)d(F)l (ebruary)i(1995.)k(IBM)c(Kingston,)g(NY.)98 754 y([2])21 b(P)o(eter)11 b(F.)g(Corb)q(ett,)g(Dror)f(G.)g(F)l(eitelson,)j (Jean-Pierre)f(Prost,)f(George)g(S.)g(Almasi,)h(Sandra)f(John-)168 811 y(son)20 b(Ba)o(ylor,)g(An)o(thon)o(y)g(S.)f(Bolmarcic)o(h,)j(Y)l (arsun)e(Hsu,)h(Julian)g(Satran,)f(Marc)f(Snir,)j(Rob)q(ert)168 867 y(Colao,)15 b(Brian)g(Herr,)g(Joseph)h(Ka)o(v)m(aky)l(,)e(Thomas)h (R.)g(Morgan,)e(and)j(An)o(thon)o(y)e(Zlotek.)20 b(P)o(arallel)168 923 y(\014le)15 b(systems)e(for)f(the)i(IBM)g(SP)f(computers.)k Fk(IBM)d(Systems)f(Journal)p Ft(,)h(34\(2\):222{2)o(48,)c(Jan)o(uary) 168 980 y(1995.)98 1074 y([3])21 b(Juan)15 b(Miguel)g(del)f(Rosario,)g (Ra)s(jesh)g(Borda)o(w)o(ek)m(ar,)f(and)h(Alok)g(Choudhary)l(.)k(Impro) o(v)o(ed)13 b(parallel)168 1130 y(I/O)i(via)f(a)f(t)o(w)o(o-phase)g (run-time)i(access)e(strategy)l(.)k(In)d Fk(IPPS)g('93)h(Workshop)h(on) f(Input/Output)168 1187 y(in)h(Par)n(al)r(lel)f(Computer)i(Systems)p Ft(,)c(pages)i(56{70,)e(1993.)18 b(Also)d(published)j(in)e(Computer)e (Arc)o(hi-)168 1243 y(tecture)i(News)f(21\(5\),)e(Decem)o(b)q(er)j (1993,)d(pages)i(31{38.)98 1337 y([4])21 b(Charles)e(H.)f(Ko)q(elb)q (el,)i(Da)o(vid)e(B.)g(Lo)o(v)o(eman,)g(Rob)q(ert)h(S.)f(Sc)o(hreib)q (er,)i(Guy)d(L.)h(Steele)i(Jr.,)e(and)168 1393 y(Mary)d(E.)f(Zosel.)21 b Fk(The)15 b(High)i(Performanc)n(e)f(F)m(ortr)n(an)f(Handb)n(o)n(ok)p Ft(.)k(MIT)c(Press,)g(1993.)98 1487 y([5])21 b(Da)o(vid)15 b(Kotz.)20 b(Disk-directed)c(I/O)f(for)g(MIMD)f(m)o(ultipro)q(cessors.) 20 b(In)15 b Fk(Pr)n(o)n(c)n(e)n(e)n(dings)f(of)i(the)g(1994)168 1544 y(Symp)n(osium)e(on)g(Op)n(er)n(ating)f(Systems)f(Design)h(and)h (Implementation)p Ft(,)e(pages)g(61{74,)f(No)o(v)o(em)o(b)q(er)168 1600 y(1994.)19 b(Up)q(dated)d(as)f(Dartmouth)f(TR)h(PCS-TR94-226)g(on) g(No)o(v)o(em)o(b)q(er)f(8,)h(1994.)98 1694 y([6])21 b(Da)o(vid)16 b(Kotz)g(and)f(Nils)i(Nieu)o(w)o(ejaar.)k(File-system)c (w)o(orkload)e(on)g(a)g(scien)o(ti\014c)j(m)o(ultipro)q(cessor.)168 1750 y Fk(IEEE)e(Par)n(al)r(lel)f(and)h(Distribute)n(d)h(T)m(e)n (chnolo)n(gy)p Ft(,)c(pages)i(51{60,)e(Spring)j(1995.)98 1844 y([7])21 b(Bill)15 b(Nitzb)q(erg.)h(P)o(erformance)c(of)g(the)h (iPSC/860)f(Concurren)o(t)g(File)i(System.)h(T)l(ec)o(hnical)g(Rep)q (ort)168 1901 y(RND-92-020,)f(NAS)i(Systems)f(Division,)h(NASA)g(Ames,) f(Decem)o(b)q(er)g(1992.)98 1994 y([8])21 b(William)f(J.)f(Nitzb)q (erg.)29 b Fk(Col)r(le)n(ctive)18 b(Par)n(al)r(lel)g(I/O)p Ft(.)29 b(PhD)18 b(thesis,)h(Departmen)o(t)f(of)f(Computer)168 2051 y(and)f(Information)f(Science,)i(Univ)o(ersit)o(y)e(of)g(Oregon,)g (Decem)o(b)q(er)h(1995.)98 2145 y([9])21 b(K.)h(E.)g(Seamons,)h(Y.)f (Chen,)i(P)l(.)e(Jones,)h(J.)f(Jozwiak,)i(and)e(M.)g(Winslett.)41 b(Serv)o(er-directed)168 2201 y(collectiv)o(e)17 b(I/O)f(in)g(P)o (anda.)j(In)d Fk(Pr)n(o)n(c)n(e)n(e)n(dings)e(of)j(Sup)n(er)n(c)n (omputing)f('95)p Ft(,)f(Decem)o(b)q(er)g(1995.)75 2295 y([10])21 b(Ra)s(jeev)f(Thakur)f(and)g(Alok)h(Choudhary)l(.)32 b(An)20 b(extended)g(t)o(w)o(o-phase)f(metho)q(d)g(for)g(accessing)168 2351 y(sections)h(of)f(out-of-core)f(arra)o(ys.)30 b(T)l(ec)o(hnical)21 b(Rep)q(ort)f(CA)o(CR-103,)f(Scalable)h(I/O)g(Initiativ)o(e,)168 2408 y(Cen)o(ter)g(for)g(Adv)m(anced)i(Computing)e(Researc)o(h,)i (Caltec)o(h,)f(June)h(1995.)33 b(Revised)22 b(No)o(v)o(em)o(b)q(er)168 2464 y(1995.)75 2558 y([11])f(The)f(MPI-IO)f(Committee.)31 b(MPI-IO:)19 b(a)g(parallel)h(\014le)g(I/O)f(in)o(terface)h(for)e(MPI,) g(April)i(1996.)168 2615 y(V)l(ersion)c(0.5.)952 2828 y(44)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Fri May 17 15:39:43 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA29770; Fri, 17 May 1996 15:35:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA17645 for mpi-core-out; Fri, 17 May 1996 14:37:46 -0500 Received: from win24.nas.nasa.gov (win24.nas.nasa.gov [129.99.33.39]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA17614 for ; Fri, 17 May 1996 14:36:58 -0500 Received: from localhost (nitzberg@localhost) by win24.nas.nasa.gov (8.6.12/NAS.6.1) with SMTP id MAA03143; Fri, 17 May 1996 12:36:52 -0700 Message-Id: <199605171936.MAA03143@win24.nas.nasa.gov> X-Authentication-Warning: win24.nas.nasa.gov: Host localhost didn't use HELO protocol To: mpi-core@mcs.anl.gov, mpi-io@nas.nasa.gov Cc: sio-hlapi@cacr.caltech.edu, sio-os-api@cs.princeton.edu Subject: MPI-2 I/O Chapter Proposal Date: Fri, 17 May 1996 12:36:51 -0700 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The following is a proposed starting point for the MPI-2 I/O chapter. All interested parties are encouraged to participate in this design process via the MPI mailing lists, news group, and forum meetings. See the MPI-2 Web pages for details at: http://www.mcs.anl.gov/Projects/mpi/mpi2/mpi2.html This proposal is essentially a reformatting of the MPI-IO Version 0.5 draft (available via http://lovelace.nas.nasa.gov/MPI-IO/) with the following changes: 1. Retained only sections 5 through 9, and a modified section 3. 2. Renamed functions, types, etc., from "MPIO_xxx" to "MPI_xxx". 3. Edited (minor changes) to conform to MPI-2 chapter style. 4. Added clarification regarding: file consistency semantics, the use of MPI_Comm in MPI_Open, ----- cut here ----- io-2.ps ----- cut here ----- %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%Pages: 46 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: /usr/local/bin/dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.05.16:2306 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet %%BeginProcSet: special.pro TeXDict begin /SDict 200 dict N SDict begin /@SpecialDefaults{/hs 612 N /vs 792 N /ho 0 N /vo 0 N /hsc 1 N /vsc 1 N /ang 0 N /CLIP 0 N /rwiSeen false N /rhiSeen false N /letter{}N /note{}N /a4{}N /legal{}N}B /@scaleunit 100 N /@hscale{@scaleunit div /hsc X}B /@vscale{@scaleunit div /vsc X}B /@hsize{/hs X /CLIP 1 N}B /@vsize{/vs X /CLIP 1 N}B /@clip{ /CLIP 2 N}B /@hoffset{/ho X}B /@voffset{/vo X}B /@angle{/ang X}B /@rwi{ 10 div /rwi X /rwiSeen true N}B /@rhi{10 div /rhi X /rhiSeen true N}B /@llx{/llx X}B /@lly{/lly X}B /@urx{/urx X}B /@ury{/ury X}B /magscale true def end /@MacSetUp{userdict /md known{userdict /md get type /dicttype eq{userdict begin md length 10 add md maxlength ge{/md md dup length 20 add dict copy def}if end md begin /letter{}N /note{}N /legal{} N /od{txpose 1 0 mtx defaultmatrix dtransform S atan/pa X newpath clippath mark{transform{itransform moveto}}{transform{itransform lineto} }{6 -2 roll transform 6 -2 roll transform 6 -2 roll transform{ itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll curveto}}{{ closepath}}pathforall newpath counttomark array astore /gc xdf pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{PaintBlack}if}N /txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 -1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop 90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr 2 get ppr 0 get neg sub neg 0 S TR}if}ifelse scaleby96{ppr aload pop 4 -1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S TR}if}N /cp {pop pop showpage pm restore}N end}if}if}N /normalscale{Resolution 72 div VResolution 72 div neg scale magscale{DVImag dup scale}if 0 setgray} N /psfts{S 65781.76 div N}N /startTexFig{/psf$SavedState save N userdict maxlength dict begin /magscale true def normalscale currentpoint TR /psf$ury psfts /psf$urx psfts /psf$lly psfts /psf$llx psfts /psf$y psfts /psf$x psfts currentpoint /psf$cy X /psf$cx X /psf$sx psf$x psf$urx psf$llx sub div N /psf$sy psf$y psf$ury psf$lly sub div N psf$sx psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub TR /showpage{}N /erasepage{}N /copypage{}N /p 3 def @MacSetUp}N /doclip{ psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto closepath clip newpath moveto}N /endTexFig{end psf$SavedState restore}N /@beginspecial{SDict begin /SpecialSave save N gsave normalscale currentpoint TR @SpecialDefaults count /ocount X /dcount countdictstack N}N /@setspecial {CLIP 1 eq{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs neg 0 rlineto closepath clip}if ho vo TR hsc vsc scale ang rotate rwiSeen{rwi urx llx sub div rhiSeen{rhi ury lly sub div}{dup}ifelse scale llx neg lly neg TR }{rhiSeen{rhi ury lly sub div dup scale llx neg lly neg TR}if}ifelse CLIP 2 eq{newpath llx lly moveto urx lly lineto urx ury lineto llx ury lineto closepath clip}if /showpage{}N /erasepage{}N /copypage{}N newpath }N /@endspecial{count ocount sub{pop}repeat countdictstack dcount sub{ end}repeat grestore SpecialSave restore end}N /@defspecial{SDict begin} N /@fedspecial{end}B /li{lineto}B /rl{rlineto}B /rc{rcurveto}B /np{ /SaveX currentpoint /SaveY X N 1 setlinecap newpath}N /st{stroke SaveX SaveY moveto}N /fil{fill SaveX SaveY moveto}N /ellipse{/endangle X /startangle X /yrad X /xrad X /savematrix matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix}N end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (temp.dvi) @start /Fa 3 62 df<07C018303018701C600C600CE00EE00EE00EE00EE00EE00EE00E E00EE00E600C600C701C30181C7007C00F157F9412>48 D<03000700FF00070007000700 070007000700070007000700070007000700070007000700070007007FF00C157E9412> I61 D E /Fb 1 1 df0 D E /Fc 5 116 df<003E000C000C000C000C0018001800180018073018F0307060706060C060C060 C06080C080C480C4C1C446C838700F177E9612>100 D<03000380030000000000000000 00000000001C002400460046008C000C0018001800180031003100320032001C0009177F 960C>105 D<383C1E0044C6630047028100460301008E0703000C0603000C0603000C06 0600180C0600180C0620180C0C20180C0C4030180440301807801B0E7F8D1F>109 D<1C3C22462382230346030603060306030C060C060C0C0C081A3019E018001800300030 003000FC001014808D12>112 D<07C00C201870187038001E000FC003E000606060E060 C0C0C1803F000C0E7E8D10>115 D E /Fd 1 82 df81 D E /Fe 8 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000F800 00F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115 >99 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F 001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018E018 F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Ff 68 123 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0 000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703 C00E03C00E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C07F87F8151D809C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<6060F0 F0F8F86868080808080808101010102020404080800D0C7F9C15>34 D<60F0F8680808081010204080050C7C9C0C>39 D<004000800100020006000C000C0018 001800300030007000600060006000E000E000E000E000E000E000E000E000E000E000E0 00E000600060006000700030003000180018000C000C00060002000100008000400A2A7D 9E10>I<800040002000100018000C000C000600060003000300038001800180018001C0 01C001C001C001C001C001C001C001C001C001C001C00180018001800380030003000600 06000C000C00180010002000400080000A2A7E9E10>I<60F0F070101010102020408004 0C7C830C>44 DI<60F0F06004047C830C>I<030007003F00C700 070007000700070007000700070007000700070007000700070007000700070007000700 07000700070007000F80FFF80D1C7C9B15>49 D<07C01830201C400C400EF00FF80FF807 F8077007000F000E000E001C001C00380070006000C00180030006010C01180110023FFE 7FFEFFFE101C7E9B15>I<07E01830201C201C781E780E781E381E001C001C0018003000 6007E00030001C001C000E000F000F700FF80FF80FF80FF00E401C201C183007E0101D7E 9B15>I<000C00000C00001C00003C00003C00005C0000DC00009C00011C00031C00021C 00041C000C1C00081C00101C00301C00201C00401C00C01C00FFFFC0001C00001C00001C 00001C00001C00001C00001C0001FFC0121C7F9B15>I<60F0F060000000000000000000 0060F0F06004127C910C>58 D<0FE03038401CE00EF00EF00EF00E000C001C0030006000 C0008001800100010001000100010001000000000000000000000003000780078003000F 1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F00000017800000 178000001780000023C0000023C0000023C0000041E0000041E0000041E0000080F00000 80F0000180F8000100780001FFF80003007C0002003C0002003C0006003E0004001E0004 001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E0618001801980070007800E0003801C0003801C0001803800 0180780000807800008070000080F0000000F0000000F0000000F0000000F0000000F000 0000F0000000F0000000700000807800008078000080380000801C0001001C0001000E00 0200070004000180080000E03000001FC000191E7E9C1E>II< FFFFFC0F003C0F000C0F00040F00040F00060F00020F00020F02020F02000F02000F0200 0F06000FFE000F06000F02000F02000F02000F02010F00010F00020F00020F00020F0006 0F00060F000C0F003CFFFFFC181C7E9B1C>II<001F808000E0618001801980070007800E0003801C0003801C000180380001807800 00807800008070000080F0000000F0000000F0000000F0000000F0000000F0000000F000 FFF0F0000F80700007807800078078000780380007801C0007801C0007800E0007800700 0B800180118000E06080001F80001C1E7E9C21>III75 DIII<003F 800000E0E0000380380007001C000E000E001C0007003C00078038000380780003C07800 03C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F000 01E0700001C0780003C0780003C0380003803C0007801C0007000E000E0007001C000380 380000E0E000003F80001B1E7E9C20>II< FFFF00000F01E0000F0078000F003C000F001C000F001E000F001E000F001E000F001E00 0F001C000F003C000F0078000F01E0000FFF00000F03C0000F00E0000F00F0000F007800 0F0078000F0078000F0078000F0078000F0078000F0078100F0078100F0038100F003C20 FFF01C20000007C01C1D7E9B1F>82 D<07E0801C1980300580700380600180E00180E000 80E00080E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007 C00003C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E 7E9C17>I<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F00 20800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00001F800003FFFC001B1C7F9B1E>IIII<7FF0FFC00FC03E000780180003C0180003E0 100001E0200001F0600000F0400000788000007D8000003D0000001E0000001F0000000F 0000000F8000000F80000013C0000023E0000021E0000041F00000C0F800008078000100 7C0003003C0002001E0006001F001F003F80FFC0FFF01C1C7F9B1F>II91 D<08081010202040404040808080808080B0B0F8 F8787830300D0C7A9C15>II<1FC000307000783800781C00 301C00001C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40 304E801F870012127E9115>97 DI< 07E00C301878307870306000E000E000E000E000E000E00060007004300418080C3007C0 0E127E9112>I<003F000007000007000007000007000007000007000007000007000007 0000070003E7000C1700180F00300700700700600700E00700E00700E00700E00700E007 00E00700600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C30181830 0C700E6006E006FFFEE000E000E000E00060007002300218040C1803E00F127F9112>I< 00F8018C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C 38C01C3880181800381C00381C00381C00381C001818001C38000C300013C00010000030 00001800001FF8001FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C 0E0007F800121C7F9215>II<1800 3C003C0018000000000000000000000000000000FC001C001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C000 000000000000000000000000000FE000E000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>I< FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C3FC0 1C0F001C0C001C08001C10001C20001C40001CE0001DE0001E70001C78001C38001C3C00 1C1C001C0E001C0F001C0F80FF9FE0131D7F9C16>IIII<03F0000E1C0018 0600300300700380600180E001C0E001C0E001C0E001C0E001C0E001C060018070038030 03001806000E1C0003F00012127F9115>II<03C1000C33 00180B00300F00700700700700E00700E00700E00700E00700E00700E007006007007007 00300F00180F000C370007C700000700000700000700000700000700000700000700003F E0131A7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000 F880388018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C001C00 3C00FFE01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C100E20 03C00C1A7F9910>IIII<7F8FF00F0380 0F030007020003840001C80001D80000F00000700000780000F800009C00010E00020E00 0607000403801E07C0FF0FF81512809116>II<7FFC7038 6038407040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E127F91 12>I E /Fg 29 117 df<001C0000003E0000003E0000002E0000006700000067000000 E7800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F00007 00F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038 001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F00001E00003C00003C00 00780000780000780000F00000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800003C00003C00001E00000F000807801803C07800FFF0003F80151F 7D9D1B>IIII<00 3F8001FFF003C0F80780380F00181E00003C00003C0000780000780000780000F00000F0 0000F00000F00000F00000F00000F007F8F007F8F000387800387800387800383C00383C 00381E00380F003807803803C0F801FFF0003F80151F7D9D1C>III75 DIII<003F000001 FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F00780007807800078078 000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C078000780780007807C000F803C000F003E001F001F003E000F807C0007C0F80003 FFF00001FFE000003F00001A1F7E9D1F>II<003F000001FFE00003FFF00007C0F8000F807C001F003E003E001F003C000F00 780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C07800078078000780780E07803C0F0F003E079F001E03DE00 0F83FC0007C1F80003FFF00001FFF800003F780000003C0000003E0000001F0000000F80 1A237E9D1F>II<03F8000FFE001C 0F00380700700300600000E00000E00000E00000E00000F000007800007F00003FE0001F FC0007FE0001FF00001F800007800003C00003C00001C00001C00001C00001C0C00180E0 0380F007007C0E001FFC0007F000121F7E9D17>II< F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070 F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F000707800E0 7800E03C01C01E03800F078007FE0001F800141E7C9C1D>III<78000E007C001E003C003C001E0038000F0070000F00F000 0781E00003C1C00001C3C00001E7800000F70000007E0000003E0000003C0000003C0000 007E00000077000000E7800001E3800003C1C0000381E0000700F0000F00F8000E007800 1C003C003C003E0078001F0070000F00F0000F80191D7F9C1C>II<7FFFF07FFFF00001E00003E00003C00007C0000780000F00001F00001E00 003E00003C0000780000F80000F00001F00001E00003C00007C0000780000F80000F0000 1E00003E00003C00007C0000780000FFFFF0FFFFF0141D7E9C19>I<07E00FF81FFC3C1C 70047000E000E000E000E000E000E000700070043C1C1FFC0FF807E00E127E9112>99 D 108 D110 D<1C001C001C001C001C001C00FFE0FFE01C001C001C001C00 1C001C001C001C001C001C001C001C001C201FF00FF007C00C187F970F>116 D E /Fh 2 16 df<400004C0000C6000183000301800600C00C006018003030001860000 CC0000780000300000300000780000CC000186000303000601800C00C018006030003060 0018C0000C40000416187A9623>2 D<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFF FFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D E /Fi 17 123 df<00F1800389C00707800E03801C03803C0380380700780700780700780700F0 0E00F00E00F00E00F00E10F01C20F01C20703C20705C40308C400F078014147E9318>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207E9F14>I< 007C01C207010E0F1E0F1C0E3C04780078007800F000F000F000F000F000700170023004 18380FC010147E9314>I<0000780003F80000700000700000700000700000E00000E000 00E00000E00001C00001C000F1C00389C00707800E03801C03803C038038070078070078 0700780700F00E00F00E00F00E00F00E10F01C20F01C20703C20705C40308C400F078015 207E9F18>I<007C01C207010E011C013C013802780C7BF07C00F000F000F000F0007000 700170023004183807C010147E9315>I<00007C0000CE00019E00039E00030C00070000 0700000700000700000E00000E00000E0000FFF0000E00000E00001C00001C00001C0000 1C00001C0000380000380000380000380000380000700000700000700000700000700000 E00000E00000E00000E00000C00001C000318000798000F300006200003C000017297E9F 16>I<00E001E001E000C000000000000000000000000000000E00130023804380438043 808700070007000E000E001C001C001C20384038403840388019000E000B1F7E9E10> 105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C 001C0038003800380038007000700070007100E200E200E200E200640038000A207E9F0E >108 D<1E07802318C023A06043C0704380704380708700E00700E00700E00700E00E01 C00E01C00E01C00E03821C03841C07041C07081C03083803101801E017147E931B>110 D<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003 C0F003C0F00780F00700700F00700E0030180018700007C00013147E9316>I<03C1E004 621804741C08781C08701E08701E10E01E00E01E00E01E00E01E01C03C01C03C01C03C01 C0380380780380700380E003C1C0072380071E000700000700000E00000E00000E00000E 00001C00001C0000FFC000171D819317>I<007C018203010603060706060E00078007F8 03FC01FE001F00077007F006F006E004400820301FC010147E9315>115 D<00C000E001C001C001C001C003800380FFF8038007000700070007000E000E000E000E 001C001C001C001C10382038203820384018800F000D1C7F9B10>I<0F00601180702180 E021C0E041C0E04380E08381C00701C00701C00701C00E03800E03800E03800E03840E07 080C07080C07080E0F1006131003E1E016147E931A>I<0F006060118070F02180E0F821 C0E07841C0E0384380E0188381C0100701C0100701C0100701C0100E0380200E0380200E 0380200E0380400E0380400E0380800E078080060781000709860001F078001D147E9321 >119 D<0F00601180702180E021C0E041C0E04380E08381C00701C00701C00701C00E03 800E03800E03800E03800E07000C07000C07000E0F00061E0003EE00000E00000E00001C 0078180078380070700060600021C0001F0000141D7E9316>121 D<01E02003F04007F8C00C1F8008010000020000040000080000100000600000C0000100 000200000400800801001003003F060061FC0040F80080700013147E9315>I E /Fj 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C060404060C0 31801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C00 0C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006 000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F000080 00400060C060C060804060801F000B107F8F0F>I<0300030007000F000B001300330023 004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C00200020002000 2F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C060 00C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC0 8080808001000200040004000C0008000800180018001800180018000B117E900F>I<1F 00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F >I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B10 7F8F0F>I E /Fk 49 122 df<00003F03E00000C386700001878CF00003879CF0000303 1860000700380000070038000007003800000E003800000E007000000E007000000E0070 0000FFFFFF80001C007000001C00E000001C00E000001C00E000001C00E000003800E000 003801C000003801C000003801C000003801C000007001C0000070038000007003800000 700380000070038000006003800000E007000000E007000000E007000000E007000000C0 06000001C00E000001C00E000031860C0000798F180000F31E100000620C6000003C07C0 00002429829F1C>11 D<00003FE00000E010000180380003807800030078000700300007 00000007000000070000000E0000000E0000000E000000FFFFE0000E00E0001C01C0001C 01C0001C01C0001C01C0001C038000380380003803800038038000380700003807000070 07000070071000700E2000700E2000700E2000E00E2000E0064000E0038000E0000000C0 000001C0000001C000003180000079800000F3000000620000003C0000001D29829F1A> I<0E1F3F3F1D0102020404081020C0080E779F0E>39 D<70F8F8F0E005057B840E>46 D<00000040000000C000000180000001800000030000000300000006000000060000000C 0000001800000018000000300000003000000060000000C0000000C00000018000000180 00000300000003000000060000000C0000000C0000001800000018000000300000003000 000060000000C0000000C0000001800000018000000300000003000000060000000C0000 000C0000001800000018000000300000003000000060000000C0000000C0000000800000 001A2D7FA117>I<000200020006000E003C00DC031C001C003800380038003800700070 0070007000E000E000E000E001C001C001C001C003800380038003800780FFF80F1E7B9D 17>49 D<001F800060E00080700100300200380420380420380410380420700460700380 600000E00001C000030000FE00001C000006000007000007800007800007803007807807 80780780F00F00800F00401E00401C0040380020E0001F8000151F7C9D17>51 D<0000600000E00000E00000E00001C00001C00001C00003800003800003000007000007 00000600000E00000C0000180000180000300000300000630000C7000087000107000307 00060E00040E00080E003F8E00607C00801FC0001C00001C000038000038000038000038 0000700000700000600013277E9D17>I<00C06000FFC001FF8001FE0001000001000002 0000020000020000020000040000047800058C000606000C070008070000078000078000 0780000780000F00700F00F00F00F00E00E01E00801C0080380080300040600061C0001F 0000131F7B9D17>I<001F0000718000C0C00180C00380E00700E00F00E00F01E01E01E0 1E01E01E01E01E01C01C03C01C03C01C03C01C07C01C0F800C0F8006378003C700000F00 000E00000E00001C00601C00F03800F07000E0600080C0004380003E0000131F7B9D17> 57 D<00000200000006000000060000000E0000001E0000001E0000003F0000002F0000 004F0000004F0000008F0000010F0000010F0000020F0000020F0000040F00000C0F0000 080F0000100F0000100F0000200F80003FFF800040078000C00780008007800100078001 0007800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFC0001E00F0001E0078001E0038001E003C003C003C003C003C003C003C003C00 3C0078007800780078007800F0007801E000F0078000FFFE0000F00F8000F003C001E001 C001E001E001E001E001E001E003C001E003C001E003C001E003C001C0078003C0078007 8007800F0007801E000F007800FFFFE0001E1F7D9E20>I<0000FE0200078186001C004C 0038003C0060003C00C0001C01C0001803800018070000180F0000181E0000101E000010 3C0000003C00000078000000780000007800000078000000F0000000F0000000F0000000 F0000000F00000807000008070000080700001003800010038000200180004000C001800 060020000381C00000FE00001F217A9F21>I<01FFFF80001E00E0001E0070001E003800 1E001C003C001C003C000E003C000E003C000E0078000E0078000E0078000E0078000E00 F0001E00F0001E00F0001E00F0001E01E0003C01E0003C01E0003C01E0007803C0007003 C0007003C000E003C001C0078001C00780038007800E0007801C000F007000FFFFC0001F 1F7D9E22>I<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C 0004003C00040078080800780800007808000078180000F0300000FFF00000F0300000F0 300001E0200001E0200001E0200001E0001003C0002003C0002003C0004003C000400780 00800780018007800100078007000F001F00FFFFFE001F1F7D9E1F>I<01FFFFFC001E00 38001E0018001E0008001E0008003C0008003C0008003C0008003C000800780010007808 00007808000078080000F0100000F0300000FFF00000F0300001E0200001E0200001E020 0001E0200003C0000003C0000003C0000003C00000078000000780000007800000078000 000F800000FFF800001E1F7D9E1E>I<01FFF3FFE0001F003E00001E003C00001E003C00 001E003C00003C007800003C007800003C007800003C007800007800F000007800F00000 7800F000007800F00000F001E00000FFFFE00000F001E00000F001E00001E003C00001E0 03C00001E003C00001E003C00003C007800003C007800003C007800003C007800007800F 000007800F000007800F000007800F00000F801F0000FFF1FFE000231F7D9E22>72 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078000078 0000780000F00000F00000F00000F00001E00001E00001E00001E00003C00003C00003C0 0003C0000780000780000780000780000F8000FFF800141F7D9E12>I<001FFF0000F800 00F00000F00000F00001E00001E00001E00001E00003C00003C00003C00003C000078000 0780000780000780000F00000F00000F00000F00001E00001E00301E00781E00F83C00F8 3C00F0780080700040E00021C0001F000018207D9E18>I<01FE00007FC0001E0000FC00 001E0000F80000170001780000170001780000270002F00000270004F00000270004F000 00270008F00000470009E00000470011E00000470021E00000470021E00000870043C000 00838043C00000838083C00000838083C000010381078000010382078000010382078000 0103840780000203840F00000203880F00000203900F00000203900F00000401E01E0000 0401E01E00000401C01E00000C01801E00001C01803E0000FF8103FFC0002A1F7D9E29> 77 D<0001FC0000070700001C01C0003000E000E0006001C00070038000700780003807 0000380E0000381E0000381C0000383C0000383C00003878000078780000787800007878 000078F00000F0F00000F0F00000E0F00001E0F00001C0F00003C0700003807000070078 000F0038001E0038003C001C0070000E00E0000783800001FC00001D217A9F23>79 D<01FFFF80001E00E0001E0070001E0038001E003C003C003C003C003C003C003C003C00 3C0078007800780078007800F0007800E000F003C000F00F0000FFFC0000F0000001E000 0001E0000001E0000001E0000003C0000003C0000003C0000003C0000007800000078000 0007800000078000000F800000FFF000001E1F7D9E1F>I<0007E040001C18C000300580 0060038000C0038001C00180018001000380010003800100038001000380000003C00000 03C0000003F8000001FF800001FFE000007FF000001FF0000001F8000000780000007800 000038000000380020003800200038002000300060007000600060006000E0007000C000 E8038000C606000081F800001A217D9F1A>83 D<0FFFFFF01E0780E01807802010078020 20078020200F0020600F0020400F0020400F0020801E0040001E0000001E0000001E0000 003C0000003C0000003C0000003C00000078000000780000007800000078000000F00000 00F0000000F0000000F0000001E0000001E0000001E0000001E0000003E00000FFFF0000 1C1F789E21>I87 D<00F1800389C0070780 0E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40 F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E 000E000E000E001C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE0 1EE01CE03CE038607060E031C01F0010207B9F15>I<007E0001C1000300800E07801E07 801C07003C0200780000780000780000F00000F00000F00000F00000F000007001007002 0030040018380007C00011147C9315>I<0000780003F800007000007000007000007000 00E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C03803C038038 0700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C4030 8C800F070015207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000 F000F0007000700170023804183807C010147C9315>I<00007800019C00033C00033C00 0718000700000700000E00000E00000E00000E00000E0001FFE0001C00001C00001C0000 1C0000380000380000380000380000380000700000700000700000700000700000700000 E00000E00000E00000E00000C00001C00001C0000180003180007B0000F300006600003C 00001629829F0E>I<003C6000E27001C1E00380E00700E00F00E00E01C01E01C01E01C0 1E01C03C03803C03803C03803C03803C07003C07001C0F001C17000C2E0003CE00000E00 000E00001C00001C00301C00783800F0700060E0003F8000141D7E9315>I<01E0000FE0 0001C00001C00001C00001C000038000038000038000038000070000070000071E000763 000E81800F01C00E01C00E01C01C03801C03801C03801C0380380700380700380700380E 10700E20700C20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C00000 0000000000000000000000000E003300230043804300470087000E000E000E001C001C00 1C003840388030807080310033001C000B1F7C9E0E>I<01E0000FE00001C00001C00001 C00001C0000380000380000380000380000700000700000703C00704200E08E00E11E00E 21E00E40C01C80001D00001E00001FC00038E00038700038700038384070708070708070 7080703100E03100601E0013207D9F15>107 D<03C01FC0038003800380038007000700 070007000E000E000E000E001C001C001C001C0038003800380038007000700070007100 E200E200E200E200640038000A207C9F0C>I<1C0F80F0002630C318004740640C004780 680E004700700E004700700E008E00E01C000E00E01C000E00E01C000E00E01C001C01C0 38001C01C038001C01C038001C01C0708038038071003803806100380380E10038038062 007007006600300300380021147C9325>I<1C0F802630C0474060478060470070470070 8E00E00E00E00E00E00E00E01C01C01C01C01C01C01C0384380388380308380708380310 7003303001C016147C931A>I<007C0001C3000301800E01C01E01C01C01E03C01E07801 E07801E07801E0F003C0F003C0F003C0F00780F00700700F00700E0030180018700007C0 0013147C9317>I<01C1E002621804741C04781C04701E04701E08E01E00E01E00E01E00 E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0072380071E0007000007 00000E00000E00000E00000E00001C00001C0000FFC000171D809317>I<00F0400388C0 0705800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E00 F01C00F01C00703C00705C0030B8000F3800003800003800007000007000007000007000 00E00000E0000FFE00121D7C9315>I<1C1E002661004783804787804707804703008E00 000E00000E00000E00001C00001C00001C00001C00003800003800003800003800007000 0030000011147C9313>I<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00800E00800E 00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C00012147C93 15>I<0E00C1C03300E3C02301C3E04381C1E04301C0E04701C060870380400E0380400E 0380400E0380401C0700801C0700801C0700801C0701001C0701001C0602001C0F02000C 0F04000E13080003E1F0001B147C931E>I<0383800CC4401068E01071E02071E02070C0 40E00000E00000E00000E00001C00001C00001C00001C040638080F38080F38100E58100 84C60078780013147D9315>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07001C0E00180E00180E001C1E000C3C0007DC 00001C00001C00003800F03800F07000E06000C0C0004380003E0000131D7C9316>I E /Fl 54 123 df<000078007C7800FC7801FC7803C00003800007800007800007800007 8000078000078000078000078000FFFC78FFFC78FFFC7807807807807807807807807807 807807807807807807807807807807807807807807807807807807807807807807807807 8078078078078078152480A31A>12 D<007000E001C003C0078007000F000E001E001C00 3C003C00380078007800780078007000F000F000F000F000F000F000F000F000F000F000 F000F000F000F0007000780078007800780038003C003C001C001E000E000F0007000780 03C001C000E000700C327DA413>40 DI46 D<000080000180000180 000300000300000300000600000600000600000C00000C00000C00001800001800001800 00300000300000300000600000600000600000C00000C00000C000018000018000018000 0180000300000300000300000600000600000600000C00000C00000C0000180000180000 180000300000300000300000600000600000600000C00000C00000C0000011317DA418> I<01F00007FC000FFE001F1F003C07803C07807803C07803C07803C07001C0F001E0F001 E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001 E0F001E07803C07803C07803C07803C03C07803C07801F1F000FFE0007FC0001F0001323 7EA118>I<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFFFFFFFF FF10227CA118>I<03F0000FFC001FFE003C1F003007807007C06003C0E003E0C001E040 01E04001E00001E00001E00001E00003C00003C0000780000780000F00001E00003C0000 780000F00001E00001C0000380000700000E00001C0000380000700000FFFFE0FFFFE0FF FFE013227EA118>I<01F00007FC001FFF003E0F003807807003C02003C02003C00003C0 0003C00003C0000780000780000F00001E0003FC0003F80003FE00000F000007800003C0 0003C00001E00001E00001E00001E00001E08001E0C003C0E003C07007803C0F801FFF00 0FFC0003F00013237EA118>I<001F00001F00002F00002F00006F0000EF0000CF0001CF 0001CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F00780F 00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F00000F 00000F0015217FA018>I<3FFF803FFF803FFF803C00003C00003C00003C00003C00003C 00003C00003C00003C00003CF8003FFE003FFF003F0F803E07803C03C03803C00001E000 01E00001E00001E00001E00001E00001E04003C04003C0E003C07007807C1F003FFE000F FC0003F00013227EA018>I<007E0001FF0003FF0007C1000F00001E00001E00003C0000 3C000078000078000078F800F1FE00F7FF00FF0F80FC0780F803C0F803C0F801C0F001E0 F001E0F001E0F001E0F001E07001E07001E07801E07803C03803C03C03801C07801F0F00 0FFE0007FC0001F00013237EA118>II<01F00007FC000FFE001E0F003C07803C07807803C078 03C07803C07803C07803C03803803C07801E0F000F1E0007FC0003F8000FFE001E0F003C 07807803C07803C0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C03C07803E 0F801FFF0007FC0001F00013237EA118>I<01F00007FC000FFE001E0F003C0700380780 7803807803C0F003C0F001C0F001E0F001E0F001E0F001E0F001E0F001E07003E07803E0 7803E03C07E03E1FE01FFDE00FF1E003E3C00003C00003C0000380000780000700000F00 101E00183C003FF8001FF00007C00013237EA118>I<001F0000001F0000003F8000003F 8000003B8000007BC0000073C0000071C00000F1E00000F1E00000E0E00001E0F00001E0 F00001C0F00003C0780003C078000380780007803C0007803C0007003C000F001E000F00 1E000FFFFE001FFFFF001FFFFF001C000F003C0007803C00078038000780780003C07800 03C0700003C0F00001E0F00001E0E00001E01B237EA220>65 DI<000FF000007F FC0000FFFF0001F01F0003C00700078000000F0000001E0000003E0000003C0000003C00 0000780000007800000078000000F0000000F0000000F0000000F0000000F0000000F000 0000F0000000F0000000F00000007800000078000000780000003C0000003C0000003E00 00001E0000000F0000000780008003C0038001F00F8000FFFF00007FFC00000FF0001925 7DA31F>IIII<000FF000003FFE0000FFFF8001F80F8003E00380 078000000F0000001E0000001E0000003C0000003C000000780000007800000078000000 F0000000F0000000F0000000F0000000F0000000F0000000F000FFC0F000FFC0F000FFC0 780003C0780003C0780003C03C0003C03C0003C01E0003C01E0003C00F0003C0078003C0 03E003C001F807C000FFFFC0003FFF00000FF8001A257DA321>II I76 D78 D<001FC000007FF00001FFFC0003F07E0007C01F000F800F801F 0007C01E0003C03C0001E03C0001E0780000F0780000F0780000F070000070F0000078F0 000078F0000078F0000078F0000078F0000078F0000078F0000078F0000078780000F078 0000F0780000F07C0001F03C0001E03E0003E01E0003C01F0007C00F800F8007C01F0003 F07E0001FFFC00007FF000001FC0001D257DA324>II82 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800007800007800007800007800 007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F00000F00000 F8000078000078000078000078000078000078C000F0E000F0F801E07E07C03FFF800FFF 0001FC0015257EA31B>II87 D<07E01FF83FFC381E201E000F000F000F000F00FF 07FF1FFF3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180780000780000700000F00000F00000F0 0000F00000F00000F000007800007800007800003C00401F03C00FFFC007FF8001FC0012 167E9516>I<0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C003E3C00FFBC01FFFC03F0FC03C07C07803C07803C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C07803C07803C03C07C03E0FC01FFFC00FFBC0 03E3C012237EA219>I<03F00007FC001FFE003E0F003C0780780380780380F001C0FFFF C0FFFFC0FFFFC0F00000F00000F000007000007800007800003C00801F07800FFF8007FF 0001F80012167E9516>I<003F00FF01FF03C00380078007800780078007800780078007 80FFF8FFF8FFF80780078007800780078007800780078007800780078007800780078007 800780078007800780102380A20F>I<01F07807FFF80FFFF81F1F001E0F003C07803C07 803C07803C07803C07801E0F001F1F000FFE001FFC0019F0003800003800003C00001FFE 001FFFC01FFFE03FFFF07801F07800F8F00078F00078F00078F000787800F03E03E01FFF C00FFF8001FC0015217F9518>III107 DIII<01FC0007FF000FFF801F07C03C01E07800F07800F0700070F00078 F00078F00078F00078F00078F000787800F07800F07C01F03E03E01F07C00FFF8007FF00 01FC0015167F9518>II114 D<07F01FFC3FFE3C0E780678007800 7C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE3FFC0FF010167F9513> I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F000F000F000F000F000F 000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12>III121 D<7FFFC07FFFC07FFFC0000F80000F00001E00003E00007C0000780000F00001 F00003E00003C0000780000F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012 167F9515>I E /Fm 42 122 df<001FE03FC0007FE0FFC001FFE1FFC003FFE3FFC003E0 07E00007C007C00007800F80000F800F80000F800F80000F800F80000F800F80000F800F 80000F800F80000F800F80000F800F8000FFFE0FFE00FFFE0FFE00FFFE0FFE000F800F80 000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F8000 0F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F 800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F80 0F80000F800F8000222A7FA922>11 D<00000F80007E0F8000FE0F8001FE0F8003FE0F80 07C00000078000000F8000000F8000000F8000000F8000000F8000000F8000000F800000 0F8000000F800000FFFE0F80FFFE0F80FFFE0F800F800F800F800F800F800F800F800F80 0F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F80 0F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F80 0F800F800F800F80192B7FAA1F>I46 D<007E0001FF8007FFE00FFFF01FC3F81F00F83E007C3E007C3C003C7C003E7C003E7800 1E78001EF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF800 1FF8001FF8001FF8001FF8001F78001E7C003E7C003E7C003E7C003E3E007C3F00FC1F00 F81FC3F80FFFF007FFE001FF80007E0018297EA71D>48 D<00300000700001F0000FF000 FFF000FFF000F1F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F000FFFFE0FFFFE0FFFFE0 13287BA71D>I<00FE0003FF800FFFC01FFFE01F07F03C01F87800FC78007C70007CF000 3EF0003E60003E20003E00003E00003E00003E00007C00007C0000780000F80001F00001 E00003E00007C0000F80001F00003E00007C0000F80001F00003E00003C0000780000F00 001E00003C00007FFFFE7FFFFE7FFFFE7FFFFE17287EA71D>I<007E0003FF8007FFE01F FFF03F83F83E00F87C007C38007C30007C10007C00007C00007C0000F80000F80001F800 03F0000FE001FFC001FF8001FFC001FFE00001F80000FC00007C00003E00003E00001F00 001F00001F00001F00001F80001FC0003EE0003EF0007E7C00FC3F01F81FFFF00FFFE003 FFC000FE0018297EA71D>I<0007E0000FE0000BE0001BE0001BE0003BE0003BE00073E0 0073E000F3E000F3E001E3E001E3E003E3E003C3E007C3E00F83E00F83E01F03E01F03E0 3E03E03E03E07C03E07C03E0F803E0FFFFFFFFFFFFFFFFFFFFFFFF0003E00003E00003E0 0003E00003E00003E00003E00003E00003E00003E018277EA61D>I<3FFFF83FFFF83FFF F83FFFF83E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E3E 003EFF803FFFE03FFFF03FC1F03F00F83F007C3E007C00007C00003E00003E00003E0000 3E00003E00003E00003E40003E60007CE0007CF000F87801F87F07F03FFFE00FFFC007FF 0001FC0017287EA61D>I<000FE0003FF800FFF801FFF803F01807E0000F80000F00001F 00003E00003E00003C00007C00007C02007C3FC0F8FFE0F9FFF0FBFFF8FF80FCFE007CFE 003EFC003EFC003FFC001FF8001FF8001FF8001FF8001F78001F7C001F7C001F7C001E3C 003E3E003E1E007C1F00FC0FC1F807FFF003FFE001FF80007E0018297EA71D>II58 D<0003F000000003F000000007F800000007F8000000077800 00000F3C0000000F3C0000000F3C0000001E3E0000001E1E0000001E1E0000003E1F0000 003C1F0000007C0F8000007C0F800000780F800000F807C00000F807C00000F007C00001 F003E00001F003E00001E003E00003E001F00003E001F00003C001F00007C000F80007FF FFF80007FFFFF8000FFFFFFC000F80007C000F00007C001F00003E001F00003E001E0000 3E003E00001F003E00001F003C00001F007C00000F807C00000F80F800000FC0F8000007 C0F8000007C0222A7EA927>65 D<0003FF00000FFFE0003FFFF800FFFFF801FE01F803F8 007007E0001007C000000F8000001F8000001F0000003E0000003E0000007C0000007C00 00007C0000007C000000F8000000F8000000F8000000F8000000F8000000F8000000F800 0000F8000000F8000000F80000007C0000007C0000007C0000007C0000003E0000003E00 00001F0000001F8000000F80000007C0000007E0000803F8003801FE01F800FFFFF8003F FFF0000FFFC00003FE001D2C7CAA25>67 DIII73 D77 D<0003FC0000000FFF000000 3FFFC00000FFFFF00001FE07F80003F801FC0007E0007E000FC0003F000F80001F001F00 000F801F00000F803E000007C03E000007C07C000003E07C000003E07C000003E0780000 01E0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001 F0F8000001F0F8000001F0F8000001F07C000003E07C000003E07C000003E07E000007E0 3E000007C03E000007C01F00000F801F80001F800F80001F000FC0003F0007E0007E0003 F801FC0001FE07F80000FFFFF000003FFFC000001FFF80000003FC0000242C7DAA2B>79 DI<007FC000 01FFF80007FFFE000FFFFF001FC07F003F000F007E0006007C0000007C000000F8000000 F8000000F8000000F8000000F8000000FC0000007E0000007F0000003F8000001FF80000 0FFF000007FFE00003FFF80000FFFC00000FFE000000FF0000003F0000001F8000000F80 00000FC0000007C0000007C0000007C0000007C0000007C0000007C000000F8060000F80 F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE00000FF80001A2C7DAA21>83 D87 D<01FC000FFF803FFFC03FFFE03C03F03001F00001F8 0000F80000F80000F80000F80001F800FFF807FFF81FFFF83FC0F87E00F8FC00F8F800F8 F800F8F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1C>97 D<007F8001FFE007FFF80FFFF81FC0783F00103E00007C00007C00007C0000F80000F800 00F80000F80000F80000F80000F800007C00007C00007C00003E00003F00181FC0780FFF F807FFF801FFE0007F80151B7E9A1A>99 D<00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E00003E00003E00003E00003E01FC3E03FF3E0FFFBE 1FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003E F8003EF8003EFC003E7C003E7C003E7E007E3F00FE1FC1FE1FFFFE0FFFBE03FE3E01F83E 172A7EA91E>I<00FC0003FF0007FF800FFFC01F87E03F01E03E00F07C00F07C0070F800 78FFFFF8FFFFF8FFFFF8FFFFF8F80000F80000F800007800007C00007C00003E00003F00 181FC0F80FFFF807FFF801FFE0007F00151B7E9A1A>I104 DI108 DII<007E0001FF8007FFE00FFFF01F81F83F 00FC3E007C7C003E7C003E78001EF8001FF8001FF8001FF8001FF8001FF8001FF8001F7C 003E7C003E7C003E3E007C3F00FC1FC3F80FFFF007FFE003FFC0007E00181B7E9A1D>I< F83F00F9FFC0FBFFE0FFFFF0FF07F8FC01F8F800FCF800FCF8007CF8007EF8003EF8003E F8003EF8003EF8003EF8003EF8003EF8007EF8007CF800FCFC00F8FC01F8FF07F0FFFFE0 FBFFC0F9FF80F87E00F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F8000017277C9A1E>I114 D<07F8001FFF003FFF807FFF807C0780F80100F80000F80000F80000FC00 007F00007FF0003FFC001FFE0007FF0000FF80001FC0000FC00007C00007C04007C0E007 C0FC1F80FFFF807FFF001FFE0007F800121B7E9A16>I<0F80000F80000F80000F80000F 80000F80000F8000FFFF80FFFF80FFFF800F80000F80000F80000F80000F80000F80000F 80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F 80800FC38007FFC007FFC003FF0001F80012227FA115>IIII<7C001F807E003F003F003E001F007C000F80 FC0007C0F80007E1F00003E3E00001F7E00000FFC000007F8000007F0000003E0000003E 0000003F0000007F800000FFC00001F3E00003E1E00003E1F00007C0F8000F807C001F80 7C001F003E003E001F007E001F80FC000FC01A1B809A1B>II E /Fn 31 90 df<3078F8787005057C840D>46 D<00000400000C00000C0000 180000180000300000300000600000600000C00000C00001800001800003000003000006 00000600000C00000C0000180000180000300000300000600000600000C00000C0000180 000180000300000300000600000600000600000C00000C00001800001800003000003000 00600000600000C00000C00000800000162D7EA117>I<007E0001C3000301800701C00E 00C00E00E01C00E01C00E03C01E03801E07801E07801E07801E07801E07801E0F003C0F0 03C0F003C0F003C0F003C0F00380F00780E00780E00700E00700E00E00600E00701C0030 38003870000FC000131F7C9D17>I<000C001C00FC0F3800380038003800380038007000 70007000700070007000E000E000E000E000E000E001C001C001C001C001C001C0038003 C0FFFE0F1E7C9D17>I<003F8000C1E00100F00200780400780400780F007C0F807C0F80 7C0F00780600780000F80000F00001E00001C0000380000700000E00001C000038000060 0000C0000180000300200600200800401000403FFFC07FFF80FFFF80161E7E9D17>I<00 7F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E00003C000038000 0700000E0000F800000E000007000007800007C00003C00007C03007C07807C0F807C0F8 07C0F00780800F00400E00201C0018780007E000141F7D9D17>I<0000600000600000E0 0001C00003C00005C0000DC00009C00011C000238000438000C380008380010380020380 040700080700180700100700200700400700FFFFF0000E00000E00000E00000E00000E00 001C00001E0001FFE0141E7E9D17>I<01803001FFE003FFC003FF0003FC000200000200 00020000040000040000040000047C000587000603800C01800801C00001C00001E00001 E00001E00001E07003C0F803C0F003C0E00380800780400700400E00201C0018700007C0 00141F7D9D17>I<000F8000704000C0200180E00301E00701E00E00C01E00001C00003C 000038000078F800790E007A07007C0300F80380F80380F003C0F003C0F003C0F003C0F0 0780E00780E00780E00700E00F00600E00701C0030180018700007C000131F7C9D17>I< 2000003FFFE07FFFC07FFF80400100C00200800200800400000800001000002000004000 0040000080000180000300000300000700000600000E00000E00001E00001C00001C0000 3C00003C00003C0000780000780000780000300000131F799D17>I<0E1F1F1E0C000000 000000000000003078F8787008147C930D>58 D<00001000000018000000380000003800 00007800000078000000FC000001BC0000013C0000033C0000023C0000063C0000043E00 00081E0000081E0000101E0000101E0000201E0000200F0000400F0000400F0000FFFF00 00800F0001000F8001000780020007800200078004000780040007800C0007C03E0007C0 FF807FFC1E207E9F22>65 D<0001F808000E061800380138007000F801E0007803C00070 07800030078000300F0000301F0000301E0000303E0000203C0000007C0000007C000000 7C0000007C000000F8000000F8000000F8000000F8000000F80000007800004078000080 780000803C0000803C0001001C0002000E00020006000C000300100001C0E000003F0000 1D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E 0078001E0078001E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F 01E0001E01E0003E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F0 03C001E003C003C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF800 7C0078003C0038003C001800780018007800080078000800780008007800080078080800 F0100000F0100000F0100000F0300000FFF00000F0700001E0200001E0200001E0200001 E0200001E0000801E0001003C0001003C0001003C0002003C0002003C0006003C000C007 8001C0078007C0FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C00180078 0018007800080078000800780008007800080078000800F0100000F0100000F0100000F0 300000F0700000FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0 000003C0000003C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F 7E9E1E>I<07FFC7FFC0007C00F800003C007800003C007800007800F000007800F00000 7800F000007800F000007800F000007800F00000F001E00000F001E00000F001E00000F0 01E00000FFFFE00000F001E00001E003C00001E003C00001E003C00001E003C00001E003 C00001E003C00003C007800003C007800003C007800003C007800003C007800003C00780 0007800F000007C00F8000FFF8FFF800221F7E9E22>72 D<07FFE0007C00003C00003C00 00780000780000780000780000780000780000F00000F00000F00000F00000F00000F000 01E00001E00001E00001E00001E00001E00003C00003C00003C00003C00003C00003C000 07800007C000FFFC00131F7F9E10>I<07FFF000007E0000003C0000003C000000780000 007800000078000000780000007800000078000000F0000000F0000000F0000000F00000 00F0000000F0000001E0000001E0000001E0000001E0000001E0008001E0010003C00100 03C0010003C0030003C0020003C0060003C0060007801E0007807C00FFFFFC00191F7E9E 1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800004E0002F000004E 0002F000004E0004F000004E0004F000004E0008F000004E0008F00000870011E0000087 0011E00000870021E00000870021E00000870041E00000838041E00001038083C0000103 8083C00001038103C00001038203C0000101C203C0000101C403C0000201C40780000201 C80780000201C80780000201D00780000200F00780000600E00780000600E00F00000F00 C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E001800003E0018 00004F001000004F001000004780100000478010000043C010000043C010000083C02000 0081E020000081E020000080F020000080F020000080782000010078400001007C400001 003C400001003C400001001E400001001E400002000F800002000F800002000F80000200 0780000200078000060003800006000300000F00010000FFE0010000221F7E9E22>I<00 03F800001E0E000038070000E0038001C001C003C001E0078000E00F0000F00F0000F01E 0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8F80001F0F8 0001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C0007803C000F003C 001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07FFFF00007C 03C0003C01E0003C00F0007800F0007800F8007800F8007800F8007800F8007800F000F0 01F000F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001E0000001E0 000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003C000000780 000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C01E0007801 E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F0078000F00F 0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E01E0003C03E 0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC0F18000003 E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E0008 0E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E0 0000F00000F04000E04000E04000E04000E06000C0600180E00380F80300C60C0081F800 16217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010C00F001080 0F0010800F0010800F0010001E0000001E0000001E0000001E0000001E0000001E000000 3C0000003C0000003C0000003C0000003C0000003C000000780000007800000078000000 780000007800000078000000F0000001F800007FFFE0001C1F7A9E21>IIII<03FFC0FFC0 007F007E00003E003800001E003000001E002000000F004000000F008000000F81000000 078200000007C600000003C400000003E800000001F000000001F000000000F000000000 F800000000F8000000017C000000023C000000043C0000000C1E000000081E000000101F 000000200F000000400F800000C0078000008007C000010003C000070003E0001F8007E0 00FFE01FFE00221F7F9E22>II E /Fo 39 122 df<000FF07F00007FF9FF8000F83FC7C001E07F8FC003E07F0FC007C07F 0FC007C03F078007C01F000007C01F000007C01F000007C01F000007C01F0000FFFFFFF8 00FFFFFFF80007C01F000007C01F000007C01F000007C01F000007C01F000007C01F0000 07C01F000007C01F000007C01F000007C01F000007C01F000007C01F000007C01F000007 C01F000007C01F000007C01F00003FF8FFF0003FF8FFF0002220809F1F>11 D<000FE000007FF80000F81C0001E07C0003E07C0007C07C0007C07C0007C0380007C000 0007C0000007C0000007C1FE00FFFFFE00FFFFFE0007C03E0007C03E0007C03E0007C03E 0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E 0007C03E0007C03E0007C03E003FF9FFC03FF9FFC01A20809F1D>I<0018007000E001C0 0380038007000E000E001E001C003C003C007800780078007800F800F000F000F000F000 F000F000F000F000F000F80078007800780078003C003C001C001E000E000E0007000380 038001C000E0007000180D2D7DA114>40 DI45 D<387CFEFEFE7C3807077C860F>I< 387CFEFEFE7C38000000000000387CFEFEFE7C3807147C930F>58 D<0000E000000000E000000001F000000001F000000001F000000003F800000003F80000 0006FC00000006FC0000000EFE0000000C7E0000000C7E000000183F000000183F000000 303F800000301F800000701FC00000600FC00000600FC00000C007E00000FFFFE00001FF FFF000018003F000018003F000030001F800030001F800060001FC00060000FC000E0000 FE00FFE00FFFE0FFE00FFFE0231F7E9E28>65 D<0007FC02003FFF0E00FE03DE03F000FE 07E0003E0FC0001E1F80001E3F00000E3F00000E7F0000067E0000067E000006FE000000 FE000000FE000000FE000000FE000000FE000000FE0000007E0000007E0000067F000006 3F0000063F00000C1F80000C0FC0001807E0003803F0007000FE01C0003FFF800007FC00 1F1F7D9E26>67 D69 DI73 D76 DI<001FF80000FFFF0001F81F8007 E007E00FC003F01F8001F81F0000F83F0000FC7F0000FE7E00007E7E00007EFE00007FFE 00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007F7E00007E7F 0000FE7F0000FE3F0000FC3F8001FC1F8001F80FC003F007E007E001F81F8000FFFF0000 1FF800201F7D9E27>79 DI<03FC 080FFF381E03F83800F8700078700038F00038F00018F00018F80000FC00007FC0007FFE 003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC0003CC0003CC000 3CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>83 D91 D93 D<07FC001FFF003F0F803F07 C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803 E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C03 00FC0000FC0000FC0000FC0000FC0000FC00007C00007E00003E00603F00C01F81C007FF 0001FC0013147E9317>I<0007F80007F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F801F8F80FFEF81F83F83E01F87E00F87C00F87C00F8FC00F8FC 00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F87E00F83E01F81F07F80FFEFF03F8FF18 207E9F1D>I<01FE0007FF800F83C01E01E03E00F07C00F07C00F8FC00F8FFFFF8FFFFF8 FC0000FC0000FC00007C00007C00003E00181E00180F807007FFE000FF8015147F9318> I<001F8000FFC001F3E003E7E003C7E007C7E007C3C007C00007C00007C00007C00007C0 00FFFC00FFFC0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC3C07 FFFE0F079E1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC0018 00001800001C00001FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78 003C78003C3F01F80FFFE001FF00171E7F931A>II<1C003E007F007F007F003E001C00000000000000 000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F00 1F001F001F00FFE0FFE00B217EA00E>I108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007E FC007EFC007EFC007EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF00 17147F931A>II114 D<0FE63FFE701E600EE006 E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81EFFFCC7F010147E9315> I<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F800F800F800F 800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>III121 D E /Fp 82 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F870051C779B18> 33 D<4010E038F078E038E038E038E038E038E038E038E038E038E03860300D0E7B9C18> I<00C00001C00001C00001C00003F0000FFC003FFE007DCF0071C700E1C380E1C780E1C7 80E1C780F1C00079C0003DC0001FE0000FF80003FC0001DE0001CF0001C70061C380F1C3 80F1C380E1C380E1C70071C70079DE003FFE001FF80007E00001C00001C00001C00000C0 0011247D9F18>36 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F0 1C73F01CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800 E0F800E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18>I<007000F001E003C007800F001E00 1C00380038007000700070007000E000E000E000E000E000E000E000E000700070007000 7000380038001C001E000F00078003C001F000F000700C24799F18>I<6000F00078003C 001E000F000780038001C001C000E000E000E000E0007000700070007000700070007000 7000E000E000E000E001C001C0038007800F001E003C007800F00060000C247C9F18>I< 01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC00 7FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D9718>I<00600000F0 0000F00000F00000F00000F00000F00000F0007FFFC0FFFFE0FFFFE07FFFC000F00000F0 0000F00000F00000F00000F00000F00000600013147E9718>I<1C3E7E7F3F1F070E1E7C F860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC7830060677 8518>I<000300000780000780000F80000F00001F00001E00001E00003E00003C00007C 0000780000780000F80000F00001F00001E00003E00003C00003C00007C0000780000F80 000F00000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F000 0060000011247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001C070 01C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C070 01C07803C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>I<018003800380 07800F803F80FF80FB804380038003800380038003800380038003800380038003800380 03800380038003807FFCFFFE7FFC0F1C7B9B18>I<03F0000FFE003FFF007C0F807003C0 E001C0F000E0F000E06000E00000E00000E00001C00001C00003C0000780000F00001E00 003C0000780000F00001E00007C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E 9B18>I<07F8001FFE003FFF007807807803C07801C03001C00001C00003C0000380000F 0003FF0003FE0003FF000007800003C00001C00000E00000E00000E0F000E0F000E0F001 C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<1FFF803FFF803FFF8038000038 00003800003800003800003800003800003800003BF8003FFE003FFF003C07801803C000 01C00000E00000E06000E0F000E0F000E0E001C07003C07C0F803FFF001FFC0003F00013 1C7E9B18>53 D<007E0001FF0007FF800F83C01E03C01C03C03801803800007000007000 00E1F800E7FE00FFFF00FE0780F803C0F001C0F000E0E000E0F000E07000E07000E07000 E03801C03C03C01E07800FFF0007FE0001F800131C7E9B18>I<03F8000FFE001FFF003E 0F803803807001C07001C07001C07001C03803803C07801FFF0007FC000FFE001F1F003C 07807001C0F001E0E000E0E000E0E000E0E000E07001C07803C03E0F801FFF000FFE0003 F800131C7E9B18>56 D<3078FCFC783000000000000000003078FCFC78300614779318> 58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<00 0300000780001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC 00007E00003F00001FC00007E00003F00001FC00007E00003F00001F8000078000030011 187D9918>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0 FFFFE07FFFC0130C7E9318>I<600000F00000FC00007E00003F00001FC00007E00003F0 0001FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F00 007E0000FC0000F0000060000011187D9918>I<007C0001FE0007FF000F87801E03C03C 1DC0387FC070FFE071E3E071C1E0E1C1E0E380E0E380E0E380E0E380E0E380E0E380E0E1 C1C071C1C071E3C070FF80387F003C1C001E00E00F83E007FFC001FF80007E00131C7E9B 18>64 D<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E 00038E00038E000306000707000707000707000707000FFF800FFF800FFF800E03800E03 801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>II<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E0700000 E00000E00000E00000E00000E00000E00000E00000E000007000007000E07000E03800E0 3C00E01E01C00F07C007FF8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F 801C03C01C03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C00701C00 701C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF8 00141C7F9B18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001 C0700000E00000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001C07003 C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F 07F01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01F FFC01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF 8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7F07F0FF87 F87F07F01C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0001DF0001DF0 001FB8001FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03801C01C07F03 F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01D C1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C 39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I< 0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780700700 780F007FFF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E1E380E1E380F0E78070 F700787F007FFF003FFE000FFC00001C00001E00000E00000F0000070000070011227D9B 18>I<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C0380 1C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C039C 1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>I<03F3801FFF803FFF807C0F807007 80E00380E00380E00380E000007000007800003F00001FF00007FE0000FF00000F800003 C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C 7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E0703800700000700000700000 700000700000700000700000700000700000700000700000700000700000700000700000 700000700000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC00 01F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F00 0707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F0000 1E00001C00003C0000780000700000F00001E00001C00003C0000780000700000F00001E 00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>I I93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I<0038007C007C007C003800000000000000000FFC1FFC0FFC001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C6038F078FFF07FE03F800E277E9C18>II<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF800 1F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F0019148193 18>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0 000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001 E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807F FFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F 00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E 00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780 700780E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF80 0FFB8003E380000380000380000380000380000380000380000380003FF8003FF8003FF8 151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C0000380000380 000380000380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318 >I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE0000 1F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<018000 0380000380000380000380007FFFC0FFFFC0FFFFC0038000038000038000038000038000 0380000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E00 13197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318 >I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E0003 8E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8F F0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E 000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E 03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000 DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F80003F 00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00 003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF0 14147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000007800007F E0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003FE0003FE000 7FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00003C000FFC000FF00007C000013247E9F18>125 D E /Fq 59 123 df<007F07F001FF1FF003FF3FF007807800070070000F00F0000F00F0000F00F000 0F00F0000F00F0000F00F0000F00F000FFF8FF80FFF8FF80FFF8FF800F00F0000F00F000 0F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000 0F00F0000F00F0000F00F0000F00F0000F00F0000F00F0001C20809F1B>11 D<0000F000F8F001F8F003F8F00780000700000F00000F00000F00000F00000F00000F00 000F0000FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00 F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F0142180A018>I<00 7000E001C00380078007000E001E001E003C003C003C0078007800780078007000F000F0 00F000F000F000F000F000F000F000F000F000F000700078007800780078003C003C003C 001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<787878781830306060E0050A7D830D>44 DI<000100030003000600060006000C000C000C00180018 001800300030003000600060006000C000C000C001800180018003000300030006000600 06000C000C000C00180018001800300030003000600060006000C000C000C000102D7DA1 17>47 D<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17> 49 D<07F0000FFC001FFE00383F00700F00600780E00780E003C04003C04003C00003C0 0003C00003C0000780000780000F00000E00001C00003C0000780000E00001C000038000 0700000E00001C0000380000700000FFFFC0FFFFC0FFFFC0121F7E9E17>I58 D<001F0000001F0000003F800000 3B8000003B8000007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F00001 E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C000FFFFE000F FFFE000FFFFE001E000F001E000F003C000F803C0007803C000780780007C0780003C078 0003C0F00003E01B207F9F1E>65 DI<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00 007C0000780000780000780000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800007C00003C00003E00001E00000F000207800E03E03E01FFFC00FF F0001FC017227DA01D>II II<001FE000 FFF801FFFE03E03E07800E0F00001E00003E00003C00007C0000780000780000780000F0 0000F00000F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C 001E3E001E1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F000000FFC00003FFF00007E1F8000F807C001F003E001E001E00 3C000F003C000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F80007C0780007807800078078000780 3C000F003C000F001E001E001F003E000F807C0007E1F80003FFF00000FFC000003F0000 1A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C00 003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001 E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019 >IIIII<780007807C000F003E001F001E001E000F003C000F807C000780780003C0F0 0003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000003F00 00007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C000F003E 001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>III<07E03FF87FFC701E401F000F000F000F003F 07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000 F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F8 0014147F9317>II<03C7800FF780 1FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F80780 7807807C0F803E1F801FFF800FF78007C780000780000780000780000780000780000780 000780000780000780111D7E9317>II<07F01FFC3FFC780C78 00780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311 >I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E00 1E001E001E001E001E001E201FF00FF007C00C1A7F9910>II< F003C0F003C07803807807807807803C0F003C0F003C0F001E0E001E1E001E1E000F1C00 0F3C000F3C0007380007380007B80003F00003F00001E00012147F9315>II<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F8 0000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414 809315>II<7FFF7FFF7FFF003E00 3C007800F800F001E003E007C007800F001F001E003C007C00FFFFFFFFFFFF10147F9314 >I E /Fr 14 122 df<0000000000700000000000F80000000001F80000000001F80000 000003F80000000003F00000000007F00000000007E00000000007E0000000000FE00000 00000FC0000000001FC0000000001F80000000001F80000000003F80000000003F000000 00007F00000000007E00000000007E0000000000FE0000000000FC0000000001FC000000 0001F80000000001F80000000003F80000000003F00000000007F00000000007E0000000 0007E0000000000FE0000000000FC0000000001FC0000000001F80000000001F80000000 003F80000000003F00000000007F00000000007E00000000007E0000000000FE00000000 00FC0000000001FC0000000001F80000000001F80000000003F80000000003F000000000 07F00000000007E00000000007E0000000000FE0000000000FC0000000001FC000000000 1F80000000003F80000000003F00000000003F00000000007F00000000007E0000000000 FE0000000000FC0000000000FC0000000001FC0000000001F80000000003F80000000003 F00000000003F00000000007F00000000007E0000000000FE0000000000FC0000000000F C0000000001FC0000000001F80000000003F80000000003F00000000003F00000000007F 00000000007E0000000000FE0000000000FC0000000000FC0000000001FC0000000001F8 0000000003F80000000003F00000000003F00000000007F00000000007E0000000000FE0 000000000FC0000000000FC0000000001FC0000000001F80000000003F80000000003F00 000000003F00000000007F00000000007E0000000000FE0000000000FC0000000000FC00 00000000F800000000007800000000002D677ACC3A>47 D66 D73 D<00000003FFF000000000 0000007FFFFF80000000000003FFFFFFF000000000000FFF003FFC00000000003FF80007 FF0000000000FFE00001FFC000000003FF8000007FF000000007FF0000003FF80000000F FE0000001FFC0000001FFC0000000FFE0000003FF800000007FF0000007FF000000003FF 800000FFF000000003FFC00001FFE000000001FFE00003FFE000000001FFF00003FFC000 000000FFF00007FFC000000000FFF80007FF80000000007FF8000FFF80000000007FFC00 0FFF80000000007FFC001FFF00000000003FFE001FFF00000000003FFE003FFF00000000 003FFF003FFF00000000003FFF003FFF00000000003FFF007FFF00000000003FFF807FFE 00000000001FFF807FFE00000000001FFF807FFE00000000001FFF807FFE00000000001F FF80FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE0000 0000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0 FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000 001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC07FFE 00000000001FFF807FFF00000000003FFF807FFF00000000003FFF807FFF00000000003F FF803FFF00000000003FFF003FFF00000000003FFF003FFF00000000003FFF001FFF8000 0000007FFE001FFF80000000007FFE001FFF80000000007FFE000FFFC000000000FFFC00 0FFFC000000000FFFC0007FFC000000000FFF80003FFE000000001FFF00003FFE0000000 01FFF00001FFF000000003FFE00000FFF800000007FFC000007FF800000007FF8000007F FC0000000FFF8000003FFE0000001FFF0000000FFF0000003FFC00000007FFC00000FFF8 00000003FFE00001FFF000000000FFF80007FFC0000000007FFF807FFF80000000000FFF FFFFFC000000000003FFFFFFF00000000000007FFFFF8000000000000003FFF000000000 4A497AC757>79 D<000FFFF000000000FFFFFF00000003FFFFFFC0000007F8007FF00000 0FFC001FFC00000FFC0007FE00001FFE0003FF00001FFE0003FF80001FFE0001FF80001F FE0001FFC0001FFE0000FFC0000FFC0000FFE00007F80000FFE00003F00000FFE0000000 0000FFE00000000000FFE00000000000FFE00000000000FFE00000000000FFE000000000 7FFFE00000003FFFFFE0000003FFFFFFE000001FFFF0FFE000007FFE00FFE00001FFE000 FFE00003FF8000FFE0000FFE0000FFE0001FFC0000FFE0001FF80000FFE0003FF80000FF E0007FF00000FFE0007FF00000FFE000FFE00000FFE000FFE00000FFE000FFE00000FFE0 00FFE00000FFE000FFE00001FFE000FFE00001FFE0007FF00003FFE0007FF000077FE000 3FF8000F7FE0001FFC003E7FF8000FFF80F83FFFE003FFFFF01FFFE000FFFFC00FFFE000 0FFE0003FFE0332E7CAD38>97 D<007FC000000000FFFFC000000000FFFFC000000000FF FFC000000000FFFFC00000000003FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0 0000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000 000001FFC00000000001FFC00000000001FFC01FFC000001FFC0FFFF800001FFC3FFFFF0 0001FFCFE01FFC0001FFDF0003FE0001FFFC0001FF0001FFF80000FF8001FFF000007FC0 01FFE000003FE001FFC000003FF001FFC000001FF801FFC000001FF801FFC000001FFC01 FFC000001FFC01FFC000000FFE01FFC000000FFE01FFC000000FFE01FFC000000FFE01FF C000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC0 00000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000 000FFE01FFC000000FFE01FFC000000FFE01FFC000000FFC01FFC000001FFC01FFC00000 1FFC01FFC000001FF801FFC000003FF001FFC000003FF001FFE000007FE001FFF000007F C001FFF80000FF8001FF3C0001FF0001FE1F0007FE0001FC0FC03FF80001F803FFFFE000 01F000FFFF800001E0001FF8000038487CC741>I<00000000001F0000007FF000FFC000 07FFFF03FFE0001FFFFFC7E7E0007FE03FFF0FF000FF800FFC0FF001FF0007FC0FF003FE 0003FE0FF007FE0003FF07E007FC0001FF03C00FFC0001FF80000FFC0001FF80001FFC00 01FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001 FFC0001FFC0001FFC0001FFC0001FFC0000FFC0001FF80000FFC0001FF800007FC0001FF 000007FE0003FF000003FE0003FE000001FF0007FC000000FF800FF8000000FFE03FF000 0001DFFFFFC0000003C7FFFF00000003C07FF00000000780000000000007800000000000 0780000000000007C0000000000007C0000000000007E0000000000007E0000000000007 F8000000000007FFFFFFF0000003FFFFFFFF000003FFFFFFFFE00001FFFFFFFFF00001FF FFFFFFFC0000FFFFFFFFFE00007FFFFFFFFF0001FFFFFFFFFF8007FFFFFFFFFF800FF000 001FFFC01FC0000001FFC03F800000007FC07F800000003FE0FF000000003FE0FF000000 001FE0FF000000001FE0FF000000001FE0FF000000001FE0FF000000001FE07F80000000 3FC07F800000003FC03FC00000007F801FE0000000FF000FF8000003FE0003FE00000FF8 0001FFE000FFF000007FFFFFFFC000000FFFFFFE000000007FFFC0000034447DAE3A> 103 D<007FC000000000FFFFC000000000FFFFC000000000FFFFC000000000FFFFC00000 000003FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0000000 0001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000000000 01FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001 FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC007FE000001FFC03FFFC00001FFC07FFFF00001FFC1F81FF80001FFC3 C00FFC0001FFC70007FE0001FFCE0007FE0001FFDC0007FF0001FFF80003FF0001FFF800 03FF8001FFF00003FF8001FFF00003FF8001FFE00003FF8001FFE00003FF8001FFE00003 FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF 8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF80 01FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001 FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FF C00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF80FFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81 FFFFFF38487BC741>I<007C0001FF0003FF8007FFC007FFC00FFFE00FFFE00FFFE00FFF E00FFFE007FFC007FFC003FF8001FF00007C000000000000000000000000000000000000 00000000000000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC003FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C0FFFFFFFFFFFFFFFFFFFFFFFF18497CC820>I<007FC000FFFFC000FFFFC000FFFFC000 FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<00000FFE0000000001FFFFF000000007FFFFFC0000001FF803FF0000007FE000FFC000 00FF80003FE00001FF00001FF00003FE00000FF80007FC000007FC000FFC000007FE000F F8000003FE001FF8000003FF003FF8000003FF803FF0000001FF803FF0000001FF807FF0 000001FFC07FF0000001FFC07FF0000001FFC0FFF0000001FFE0FFF0000001FFE0FFF000 0001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF00000 01FFE0FFF0000001FFE0FFF0000001FFE07FF0000001FFC07FF0000001FFC07FF0000001 FFC07FF0000001FFC03FF8000003FF803FF8000003FF801FF8000003FF001FFC000007FF 000FFC000007FE0007FE00000FFC0003FE00000FF80001FF00001FF00000FF80003FE000 007FE000FFC000001FFC07FF00000007FFFFFC00000001FFFFF0000000001FFF00000033 2E7DAD3A>111 D<007FC01FFC0000FFFFC0FFFF8000FFFFC3FFFFF000FFFFCFE03FFC00 FFFFDF0007FE0003FFFC0003FF0001FFF80001FF8001FFF00000FFC001FFE000007FE001 FFC000007FF001FFC000003FF801FFC000003FF801FFC000001FFC01FFC000001FFC01FF C000001FFE01FFC000001FFE01FFC000000FFE01FFC000000FFE01FFC000000FFF01FFC0 00000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000 000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFE01FFC00000 1FFE01FFC000001FFE01FFC000001FFC01FFC000001FFC01FFC000003FFC01FFC000003F F801FFC000003FF001FFC000007FF001FFE00000FFE001FFF00000FFC001FFF80001FF80 01FFFC0003FF0001FFDF000FFE0001FFCFC03FF80001FFC3FFFFE00001FFC0FFFF800001 FFC01FF8000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0 0000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC000000000FFFFFF80000000FFFFFF80000000FFFFFF80000000FFFFFF8000 000038427CAD41>I<007F803F80FFFF80FFE0FFFF81FFF8FFFF83C3FCFFFF8707FE03FF 8E07FE01FF9C0FFF01FFB80FFF01FFB80FFF01FFF00FFF01FFF00FFF01FFF007FE01FFE0 03FC01FFE001F801FFE0000001FFE0000001FFC0000001FFC0000001FFC0000001FFC000 0001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC00000 01FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001 FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FF C00000FFFFFFE000FFFFFFE000FFFFFFE000FFFFFFE000282E7DAD2F>114 D121 D E /Fs 9 117 df<0001FF0000000FFFE000003FFFF80000FF83FE0001FE00FF00 03F8003F8007F8003FC00FF0001FE00FE0000FE01FE0000FF01FE0000FF03FE0000FF83F E0000FF83FC00007F87FC00007FC7FC00007FC7FC00007FC7FC00007FC7FC00007FCFFC0 0007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC000 07FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007 FEFFC00007FEFFC00007FEFFC00007FEFFC00007FE7FC00007FC7FC00007FC7FC00007FC 7FC00007FC7FC00007FC3FE0000FF83FE0000FF83FE0000FF81FE0000FF01FE0000FF00F F0001FE00FF0001FE007F8003FC003FC007F8001FE00FF0000FF83FE00007FFFFC00001F FFF0000001FF000027397CB730>48 D<00003C000000007C00000001FC00000007FC0000 003FFC0000FFFFFC0000FFFFFC0000FFFFFC0000FFCFFC0000000FFC0000000FFC000000 0FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000F FC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC 0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC00 00000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000 000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC000000 0FFC0000000FFC0000000FFC0000000FFC0000000FFC00007FFFFFFF807FFFFFFF807FFF FFFF807FFFFFFF80213879B730>I<0000001FFE000060000003FFFFE000E000001FFFFF F801E000007FFFFFFE03E00001FFFC00FF07E00007FFC0001FCFE0000FFF000007FFE000 1FFC000001FFE0003FF0000000FFE0007FE00000007FE000FFC00000003FE001FF800000 001FE003FF000000000FE007FF000000000FE00FFE0000000007E00FFE0000000007E01F FC0000000003E01FFC0000000003E03FF80000000003E03FF80000000001E03FF8000000 0001E07FF80000000001E07FF80000000001E07FF00000000000007FF0000000000000FF F0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000 000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FF F0000000000000FFF00000000000007FF00000000000007FF00000000000007FF8000000 0000007FF80000000001E03FF80000000001E03FF80000000001E03FF80000000001E01F FC0000000001E01FFC0000000003C00FFE0000000003C00FFE0000000003C007FF000000 00078003FF00000000078001FF800000000F0000FFC00000001F00007FE00000003E0000 3FF00000007C00001FFC000000F800000FFF000003F0000007FFC0000FE0000001FFFC00 7F800000007FFFFFFF000000001FFFFFFC0000000003FFFFE000000000001FFE0000003B 3D7BBB46>67 D<003FFE00000001FFFFE0000007FFFFF800000FE007FC00000FF001FE00 001FF800FF00001FF8007F80001FF8007FC0001FF8003FC0000FF0003FE00007E0003FE0 0003C0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000FFFFE0 00001FFFFFE000007FF83FE00003FF803FE00007FC003FE0000FF0003FE0001FE0003FE0 003FE0003FE0007FC0003FE0007FC0003FE000FF80003FE000FF80003FE000FF80003FE0 00FF80003FE000FF80007FE0007FC0007FE0007FC000DFE0003FE0039FF0001FF80F0FFF E007FFFE0FFFE001FFFC07FFE0003FE000FFE02B267DA52F>97 D<0001FF8000000FFFF0 00007FFFFC0000FF81FE0003FE007F8007F8003F800FF0001FC00FF0000FE01FE0000FE0 3FE0000FF03FE00007F07FC00007F07FC00007F87FC00007F8FFC00007F8FFC00007F8FF FFFFFFF8FFFFFFFFF8FFFFFFFFF8FFC0000000FFC0000000FFC0000000FFC00000007FC0 0000007FC00000007FC00000003FE00000003FE00000781FE00000781FF00000780FF000 00F007F80001F003FC0003E001FE000FC000FFC07F80003FFFFE00000FFFF8000000FFC0 0025267DA52C>101 D<00FE00000000FFFE00000000FFFE00000000FFFE00000000FFFE 0000000007FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 00FF800003FE03FFF00003FE0FFFF80003FE1E03FC0003FE3801FE0003FE6001FF0003FE C000FF0003FFC000FF8003FF8000FF8003FF0000FF8003FF0000FF8003FF0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF80FFFFF83FFFFEFFFFF83FFFFEFFFF F83FFFFEFFFFF83FFFFE2F3C7CBB36>104 D<00FE01FF0000FFFE1FFFF000FFFE7FFFFC 00FFFEFC07FE00FFFFF001FF0007FFC0007F8003FF00003FC003FE00003FE003FE00001F F003FE00001FF803FE00000FF803FE00000FFC03FE00000FFC03FE000007FC03FE000007 FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007 FE03FE000007FE03FE000007FE03FE000007FC03FE00000FFC03FE00000FFC03FE00000F FC03FE00000FF803FE00001FF003FE00001FF003FF00003FE003FF80007FC003FFC000FF 8003FFE001FF0003FEFC07FE0003FE7FFFF80003FE1FFFE00003FE03FE000003FE000000 0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE000000 0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE000000 00FFFFF8000000FFFFF8000000FFFFF8000000FFFFF80000002F377DA536>112 D<01FC03F000FFFC0FFC00FFFC1FFF00FFFC3C3F80FFFC707F8007FCE0FFC003FCC0FFC0 03FD80FFC003FD80FFC003FF807F8003FF003F0003FF001E0003FF00000003FE00000003 FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE 00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00 000003FE00000003FE00000003FE00000003FE00000003FE000000FFFFFC0000FFFFFC00 00FFFFFC0000FFFFFC000022267DA528>114 D<000F0000000F0000000F0000000F0000 000F0000001F0000001F0000001F0000001F0000003F0000003F0000007F0000007F0000 00FF000001FF000003FF000007FF00001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C00FF0078 00FF8078007F80F0003FC1E0001FFFC0000FFF800001FE001E377EB626>116 D E /Ft 84 125 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C000C00180018003000300030007000600060006000E0 00E000E000E000E000E000E000E000E000E000E000E00060006000600070003000300030 00180018000C000C000600020001000080004000200B2E7DA112>I<8000400020001000 08000C00060006000300030001800180018001C000C000C000C000E000E000E000E000E0 00E000E000E000E000E000E000E000C000C000C001C00180018001800300030006000600 0C00080010002000400080000B2E7DA112>I<01800180018001800180C183F18F399C0F F003C003C00FF0399CF18FC1830180018001800180018010147DA117>I<000600000006 000000060000000600000006000000060000000600000006000000060000000600000006 000000060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600000006 000000060000000600000006000000060000000600000006000000060000000600000006 00000006000000060000000600001C207D9A23>I<70F8FCFC7404040408081010204006 0E7C840D>II<70F8F8F87005057C840D>I<0001000300030006 00060006000C000C000C00180018001800300030003000600060006000C000C000C00180 018001800300030003000600060006000C000C000C001800180018003000300030006000 60006000C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070 0380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003 F000121F7E9D17>I<018003800F80F38003800380038003800380038003800380038003 800380038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C 9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007 C00007C0000780000780000F00000E00001C0000380000700000600000C0000180000300 000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C0010 0E00200F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003 F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040 0780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00 002E00002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00 200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E00 00FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000 0010000010000011F000161C00180E001007001007800003800003800003C00003C00003 C07003C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17 >I<007C000182000701000E03800C07801C0780380300380000780000700000700000F1 F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C070 03C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC0 7FFF807FFF80400100800200800200800400000800000800001000002000002000004000 00400000C00000C00001C000018000038000038000038000038000078000078000078000 078000078000078000078000030000121F7D9D17>I<03F0000C0C001006003003002001 806001806001806001807001807803003E03003F06001FC8000FF00003F80007FC000C7E 00103F00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001001002 000C0C0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700700380F0 0380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003 E3C0000380000380000380000700300700780600780E00700C002018001070000FC00012 1F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F8 700000000000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0 FFFFFFF00000000000000000000000000000000000000000000000000000000000000000 FFFFFFF07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C00380070 00E000C00180018001000300020002000200020002000200000000000000000000000700 0F800F800F8007000E207D9F15>63 D<000100000003800000038000000380000007C000 0007C0000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800 002078000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F00 01000F0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE 1F207F9F22>65 DI<000FC04000 7030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C 00004078000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8 000000F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007 0001000380020001C0040000703800000FC0001A217D9F21>III< FFFFFF000F800F0007800300078003000780010007800180078000800780008007800080 07800080078080000780800007808000078080000781800007FF80000781800007808000 078080000780800007808000078000000780000007800000078000000780000007800000 07800000078000000FC00000FFFE0000191F7E9E1E>I<000FE0200078186000E004E003 8002E0070001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8 000000F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E078 0001E07C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000 E0046000781820000FE0001E217D9F24>III<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C0000 3C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0020 3C00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>IIIII<001F800000F0F00001C0380007801E000F000F 000E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001 F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003 E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F0 00001F80001C217D9F23>II82 D<07E0800C1980100780 300380600180600180E00180E00080E00080E00080F00000F000007800007F00003FF000 1FFC000FFE0003FF00001F800007800003C00003C00001C08001C08001C08001C08001C0 C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F00 60400F0020400F0020C00F0030800F0010800F0010800F0010800F0010000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F80 0007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003C0020001E0040001F00C 0000F0080000781000007C1000003C2000003E4000001E4000000F8000000F8000000780 000003C0000007E0000005E0000009F0000018F8000010780000207C0000603C0000401E 0000801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I< FFF003FF1F8000F80F8000600780004007C0004003E0008001E0008001F0010000F00300 00F80200007C0400003C0400003E0800001E0800001F1000000FB0000007A0000007C000 0003C0000003C0000003C0000003C0000003C0000003C0000003C0000003C0000003C000 0003C0000003C0000007C000007FFE00201F7F9E22>I<7FFFF87C00F87000F06001E040 01E0C003C0C003C0800780800F80800F00001E00001E00003C00003C0000780000F80000 F00001E00001E00003C00403C0040780040F80040F000C1E000C1E00083C00183C001878 0038F801F8FFFFF8161F7D9E1C>II<0804100820 10201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE000303000781800781C00300E00000E00000E00000E0000FE 00078E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C01414 7E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E0078 0E00780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19> I<03F80E0C1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C 020E0C03F010147E9314>I<000380003F80000380000380000380000380000380000380 00038000038000038000038003E380061B801C0780380380380380700380700380F00380 F00380F00380F00380F00380F003807003807003803803803807801C07800E1B8003E3F8 15207E9F19>I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF 80F00000F00000F000007000007000003800801800800C010007060001F80011147F9314 >I<007C00C6018F038F07060700070007000700070007000700FFF00700070007000700 070007000700070007000700070007000700070007000700070007007FF01020809F0E> I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C 001E380033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C000 30C00030C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E 81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C000000 000000000000000000000E007E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E00000000000 00000000000000007007F000F00070007000700070007000700070007000700070007000 700070007000700070007000700070007000706070F060F0C061803F000C28829E0E>I< 0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C00 0E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE 618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E 81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C0 3801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C0 3801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00 F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01 C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D 7F9319>I<03E0800619801C05803C0780380380780380700380F00380F00380F00380F0 0380F00380F003807003807803803803803807801C0B800E138003E38000038000038000 0380000380000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F 0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312 >I<1F9030704030C010C010C010E00078007F803FE00FF00070803880188018C018C018 E030D0608F800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E 000E000E000E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F 9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800 003800003C00007C00004E000087000107000303800201C00601E01E01E0FF07FE171480 9318>II<3FFF380E200E201C4038 4078407000E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314> III E /Fu 45 122 df<70F8FCFC7404040404080810102040060F7C840E>44 D I<70F8F8F87005057C840E>I<01F000071C000C06001803003803803803807001C07001 C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001 E0F001E0F001E0F001E0F001E07001C07001C07001C07803C03803803803801C07000C06 00071C0001F00013227EA018>48 D<008003800F80F38003800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 800380038007C0FFFE0F217CA018>I<03F8000C1E001007002007804007C07807C07803 C07807C03807C0000780000780000700000F00000E0000380003F000001C00000F000007 800007800003C00003C00003E02003E07003E0F803E0F803E0F003C04003C04007802007 80100F000C1C0003F00013227EA018>51 D<000200000600000E00000E00001E00001E00 002E00004E00004E00008E00008E00010E00020E00020E00040E00040E00080E00100E00 100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E00000E00000E00 000E00001F0001FFF015217FA018>I<1000801E07001FFF001FFE001FF80013E0001000 0010000010000010000010000010000010F800130E001407001803801003800001C00001 C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C04003C04003802007 001006000C1C0003F00013227EA018>I<007E0001C1000300800601C00E03C01C03C018 0180380000380000780000700000700000F0F800F30C00F40600F40300F80380F801C0F0 01C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C03801C01803801C 03000C0600070C0001F00013227EA018>I<01F800060E000803001001802001802000C0 6000C06000C06000C07000C07801803E01003F02001FC4000FF80003F80003FC00067F00 083F80100F803007C06001C06000E0C000E0C00060C00060C00060C000606000406000C0 3000801803000E0E0003F00013227EA018>56 D<01F000060C000C060018070038038070 0380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003E038 03E01805E00C05E00619E003E1E00001C00001C00001C000038000038030030078070078 0600700C002018001030000FC00013227EA018>I<0001800000018000000180000003C0 000003C0000003C0000005E0000005E000000DF0000008F0000008F0000010F800001078 000010780000203C0000203C0000203C0000401E0000401E0000401E0000800F0000800F 0000FFFF000100078001000780030007C0020003C0020003C0040003E0040001E0040001 E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E006 3001C00170038000F0070000F00E0000701E0000701C0000303C0000303C0000307C0000 107800001078000010F8000000F8000000F8000000F8000000F8000000F8000000F80000 00F800000078000000780000107C0000103C0000103C0000101C0000201E0000200E0000 40070000400380008001C0010000E0020000381C000007E0001C247DA223>67 D69 DI< FFFC0FC00780078007800780078007800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800FC0FFFC0E227EA1 12>73 D 77 DI 80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020E00020 E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E00001E0 0000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0C80180 C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008400780 08C007800C80078004800780048007800480078004000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000FC0 0003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700000700 00FF0007C7001E07003C0700780700700700F00708F00708F00708F00F087817083C2390 0FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E003C0E 001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C 80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000700000 F00000F00000F00000F00000F00000F00000F000007000007800403800401C00800C0100 07060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800E07800 E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800E01801 E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C07801E070 00E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C00400E 008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700070007 00070007000700FFF8070007000700070007000700070007000700070007000700070007 00070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C07001C 07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010000018 00001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860003060 00303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E00F0070 0F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000000000 0000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E001C000 00000000000000000000000000000001E00FE001E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0F1 8061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E02000E0400 0E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C00E01E0 0E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00FE60E183801E807201C0 0F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60C01E80E00F00 700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E0038 0E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E0038 0F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E0000 0E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E 000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>III121 D E /Fv 20 118 df45 D68 D73 D77 D 80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E070 0000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF0000 0FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880 00007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0F0 0000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fw 5 85 df<00000000600000000000E00000000000E00000000001E0 0000000003E00000000003E00000000007E00000000007F0000000000DF0000000000DF0 0000000019F00000000039F00000000031F00000000061F00000000061F000000000C1F0 00000000C1F00000000181F00000000381F00000000301F00000000601F00000000601F0 0000000C01F80000000C00F80000001800F80000003800F80000003000F80000006000F8 0000006000F8000000C000F8000000C000F8000001FFFFF8000001FFFFF80000030000F8 0000060000F80000060000F800000C0000F800000C0000FC00001800007C00001800007C 00003000007C00007000007C00006000007C0000C000007C0000C000007C0001C000007C 0003C000007C000FE00000FE00FFF8001FFFE0FFF8001FFFE02B327CB134>65 D<000FFFFFFC0000000FFFFFFF800000007E000FE00000007C0003F00000007C0000F800 00007C00007C0000007C00007C000000F800003E000000F800001E000000F800001F0000 00F800001F000001F000001F000001F000000F800001F000000F800001F000000F800003 E000000F800003E000000F800003E000000F800003E000001F800007C000001F800007C0 00001F800007C000001F800007C000001F80000F8000001F00000F8000003F00000F8000 003F00000F8000003F00001F0000007E00001F0000007E00001F0000007E00001F000000 7C00003E000000FC00003E000000F800003E000001F800003E000001F000007C000003E0 00007C000003E000007C000007C000007C00000F800000F800000F000000F800001E0000 00F800003C000000F8000078000001F00001F0000001F00003E0000001F0000F80000003 F0007F000000FFFFFFFC000000FFFFFFC000000031317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007E0003F800007C0000F800007C00007800007C0000 7800007C0000300000F80000300000F80000300000F80000300000F80000300001F00000 300001F00000300001F00000300001F00000600003E00180600003E00180000003E00180 000003E00180000007C00300000007C00300000007C00700000007C01F0000000FFFFE00 00000FFFFE0000000F801E0000000F800E0000001F000C0000001F000C0000001F000C00 00001F000C0000003E00180000003E00180000003E00000000003E00000000007C000000 00007C00000000007C00000000007C0000000000F80000000000F80000000000F8000000 0000F80000000001F00000000001F00000000001F00000000003F000000000FFFFE00000 00FFFFE00000002D317BB02E>70 D<000FFFFFE000000FFFFFFC0000007E003F0000007C 000F8000007C0007C000007C0003E000007C0003E00000F80001F00000F80001F00000F8 0001F00000F80003F00001F00003F00001F00003F00001F00003F00001F00003E00003E0 0007E00003E00007C00003E0000FC00003E0000F800007C0001F000007C0003E000007C0 0078000007C001E000000F800F8000000FFFFC0000000F800E0000000F80070000001F00 038000001F0003C000001F0001E000001F0001E000003E0001F000003E0001F000003E00 01F000003E0001F000007C0003F000007C0003F000007C0003E000007C0003E00000F800 07E00000F80007E00000F80007E00000F80007E01801F00007E03001F00007E03001F000 07E02003F00003E060FFFF8001E0C0FFFF8000F180000000003E002D327BB033>82 D<07FFFFFFFFF007FFFFFFFFF00FE007E007F00F8007C001F00E0007C000E00C0007C000 E01C0007C000E018000F8000E018000F8000E030000F8000C030000F8000C030001F0000 C060001F0000C060001F0000C060001F0000C0C0003E00018000003E00000000003E0000 0000003E00000000007C00000000007C00000000007C00000000007C0000000000F80000 000000F80000000000F80000000000F80000000001F00000000001F00000000001F00000 000001F00000000003E00000000003E00000000003E00000000003E00000000007C00000 000007C00000000007C00000000007C0000000000F80000000000F80000000000F800000 00000F80000000001F00000000001F00000000001F00000000003F000000003FFFFF0000 003FFFFF0000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 799 911 a Fw(D)25 b(R)g(A)g(F)g(T)225 1002 y Fv(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n (terface)621 1196 y Fu(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum) 832 1323 y(Ma)o(y)h(16,)g(1996)77 1381 y(This)h(w)o(ork)f(w)o(as)h (supp)q(orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran) o(t)h(ASC-9310330,)i(the)192 1439 y(National)d(Science)f(F)l(oundation) i(Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e) 76 1497 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 49 a Ft(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 361 a Fs(Chapter)31 b(10)75 568 y Fr(I/O)75 798 y Ft(This)20 b(is)f(a)g(prop)q(osed)h(starting)e(p)q(oin)o(t)i(for) e(the)h Fq(MPI-2)g Ft(I/O)g(c)o(hapter.)31 b(It)20 b(is)f(directly)i (deriv)o(ed)f(from)75 855 y(MPI-IO)f(V)l(ersion)h(0.5)d([11])h(with)h (minor)f(editorial)i(c)o(hanges.)30 b(A)19 b(historical)h(p)q(ersp)q (ectiv)o(e)g(\(e.g.)29 b(the)75 911 y(argumen)o(ts)20 b(that)g(led)i(to)e(v)m(arious)h(design)g(decisions\))h(can)f(b)q(e)h (found)f(at)f(the)g(MPI-IO)i(W)l(eb)f(page)75 968 y Fp (http://lovelace.nas.nasa.g)o(ov/MPI-)o(IO/)p Ft(.)166 1024 y(Direct)f(all)h(commen)o(ts)f(regarding)g(the)g Fq(MPI-2)f Ft(I/O)i(Chapter)f(to)f(the)h Fp(mpi-io@mcs.anl.gov)75 1081 y Ft(mailing)d(list.)j(See)c Fp(http://www.mcs.anl.gov/Proj)o (ects/mp)o(i/mpi2/m)o(pi2.htm)o(l)c Ft(for)j(details.)166 1137 y(Ma)s(jor)e(issues)j(to)f(b)q(e)h(discussed)h(regarding)e(this)g (prop)q(osal:)75 1212 y Fo(La)o(y)o(erabilit)o(y)23 b Ft(Can/should)14 b(w)o(e)f(pro)o(vide)h(enough)f Fq(MPI-2)g Ft(facilities)i(to)e(allo)o(w)g(I/O)h(to)f(b)q(e)h(la)o(y)o(ered)f(on) 189 1268 y(top)h(of)h Fq(MPI-2)p Ft(?)75 1355 y Fo(P)o(ortable)j (O\013sets)k Ft(Ho)o(w)9 b(do)i(w)o(e)f(pro)o(vide)h(p)q(ortable)g (o\013sets)e(with)i(more)f(than)g(32)g(bits?)20 b(P)o(ossibilities:)189 1411 y(long)15 b(long/in)o(teger*8,)g(opaque)g(o\013sets,)f(in)o(t)h ([2]/in)o(teger)f(\(2\),)g(double/double)k(precision.)75 1497 y Fo(MPI)p 184 1497 16 2 v 18 w(File)g(vs.)k(MPI)p 497 1497 V 18 w(Comm)g Ft(Should)16 b(w)o(e)e(scrap)h(the)g(MPI-IO)g(v) o(er)g(0.5)e(idea)j(of)e(\014le)i(handles)g(and)189 1554 y(simply)g(use)g Fq(MPI)e Ft(comm)o(unicators?)75 1640 y Fo(MPI)p 184 1640 V 18 w(File)p 287 1640 V 20 w(Con)o(trol)22 b Ft(The)h(\014le)g(con)o(trol)f(in)o(terface)g(cannot)g(supp)q(ort)g (F)o(OR)l(TRAN)h(and)g(m)o(ust)e(b)q(e)189 1696 y(revised.)75 1783 y Fo(End-of-File)k Ft(The)14 b(curren)o(t)g(de\014nition)i(for)d (up)q(dating)i(\014le)g(p)q(oin)o(ters)g(when)f(end-of-\014le)i(is)e (reac)o(hed)g(is)189 1839 y(con)o(tradictory)l(.)k(The)12 b(seman)o(tics)g(de\014ned)h(in)g(the)f(last)f(paragraph)g(of)h (section)g(10.1.3)e(con)o(tradict)189 1896 y(the)15 b(language)g(in)h (the)g(in)o(tro)q(ductions)g(of)f(sections)g(10.4)f(and)i(10.5.)75 1982 y Fo(Shared)h(\014le)h(p)q(oin)o(ters)23 b Ft(There)15 b(is)h(some)e(con)o(tro)o(v)o(ersy)g(as)g(to)g(whether)h(w)o(e)g (should)h(k)o(eep)f(shared)g(\014le)189 2038 y(p)q(oin)o(ters,)g(and)g (if)h(w)o(e)f(do,)g(the)g(exact)g(seman)o(tics)g(of)g(shared)g(\014le)h (p)q(oin)o(ter)g(op)q(erations.)75 2125 y Fo(Filet)o(yp)q(e)i (constructors)23 b Ft(Since)15 b(\014let)o(yp)q(e)h(constructors)d(are) h(simply)h(new)g(datat)o(yp)q(e)e(constructors,)189 2181 y(should)19 b(w)o(e)e(mo)o(v)o(e)g(this)i(section)f(to)f(the)h(\\Datat) o(yp)q(e)f(Impro)o(v)o(emen)o(ts")g(subsection)i(in)g(Miscel-)189 2237 y(lan)o(y?)75 2324 y Fo(File)f(In)o(terop)q(erabilit)o(y)24 b Ft(Ho)o(w)16 b(m)o(uc)o(h)g(\014le)i(in)o(terop)q(erabilit)o(y)g (should)g(b)q(e)f(sp)q(eci\014ed.)26 b(Minimally)l(,)19 b(all)189 2380 y(applications)d(on)f(a)g(single)i(mac)o(hine,)e(using)h (a)f(single)h Fq(MPI)f Ft(implemen)o(tation)h(should)g(ha)o(v)o(e)f (the)189 2437 y(same)h(canonical)i(view)f(of)g(\014les.)25 b(What)16 b(ab)q(out)g(b)q(et)o(w)o(een)h Fq(MPI)g Ft(implemen)o (tations)g(or)g(b)q(et)o(w)o(een)189 2493 y(mac)o(hines)f(\(e.g.)j (heterogeneous)c(t)o(yp)q(es,)g(v)m(arying)h(ph)o(ysical)g(\014le)g (structures\).)75 2579 y Fo(Simpler)i(In)o(terface)k Ft(Sev)o(eral)15 b(users)f(ha)o(v)o(e)g(complained)i(of)e(the)h(steep)f (learning)i(curv)o(e)e(required)i(for)189 2636 y(this)g(in)o(terface.) 24 b(A)17 b(prop)q(osed)f(simpli\014cation)j(of)d Fq(MPI)p 1149 2636 14 2 v 16 w(OPEN)h Ft(\(see)f(section)h(10.7\))e(w)o(ould)i (ease)189 2692 y(migration)e(from)f(the)i(standard)e(UNIX)i(in)o (terface)g(to)e(the)h Fq(MPI)g Ft(I/O)h(in)o(terface.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Ft(2)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fm(10.1)60 b(Intro)r(duction)75 151 y Ft(Di\013eren)o(t) 11 b(parallel)i(\014le)g(systems)e(supp)q(ort)h(di\013eren)o(t)f(in)o (terfaces)h(and)g(the)f(lac)o(k)h(of)f(a)g(standard)g(p)q(ortable)75 207 y(in)o(terface)j(mak)o(es)f(it)h(imp)q(ossible)i(to)d(dev)o(elop)h (a)f(truly)h(p)q(ortable)g(parallel)h(program.)k(It)13 b(is)h(not)g(feasible)75 264 y(to)f(dev)o(elop)i(large)f(scien)o (ti\014c)h(applications)h(from)d(scratc)o(h)g(for)h(eac)o(h)g (generation)g(of)f(parallel)i(mac)o(hine,)75 320 y(and,)g(in)g(the)g (scien)o(ti\014c)h(w)o(orld,)f(p)q(ortabilit)o(y)g(and)g(p)q (erformance)g(are)f(inseparable|a)j(program)c(is)i(not)75 377 y(considered)i(truly)e(p)q(ortable)h(unless)g(it)g(not)e(only)i (compiles,)g(but)g(also)f(runs)g(e\016cien)o(tly)l(.)166 433 y(The)e(signi\014can)o(t)g(optimizations)g(required)h(for)d (e\016ciency)j(\(e.g.)k(grouping)13 b([7],)f(collectiv)o(e)i(bu\013er-) 75 489 y(ing)k([3,)f(8)o(,)h(9)o(,)g(10)o(],)f(and)h(disk-directed)i (I/O)e([5)o(]\))f(can)h(only)g(b)q(e)g(implemen)o(ted)i(as)d(part)g(of) g(a)g(parallel)75 546 y(I/O)d(en)o(vironmen)o(t)g(if)g(it)f(supp)q (orts)h(a)f(high-lev)o(el)j(in)o(terface)d(to)g(describ)q(e)i(the)f (partitioning)g(of)f(\014le)i(data)75 602 y(among)c(pro)q(cesses)g(and) h(a)f(collectiv)o(e)i(in)o(terface)e(describing)i(complete)f(transfers) f(of)f(global)i(data)f(struc-)75 659 y(tures)i(b)q(et)o(w)o(een)h(pro)q (cess)g(memories)g(and)f(\014les.)21 b(In)14 b(addition,)g(further)g (e\016ciencies)i(can)d(b)q(e)h(gained)h(via)75 715 y(supp)q(ort)d(for)g (async)o(hronous)f(I/O,)i(strided)f(requests,)h(and)f(con)o(trol)g(o)o (v)o(er)f(ph)o(ysical)i(\014le)g(la)o(y)o(out)f(on)g(stor-)75 772 y(age)i(devices)h(\(disks\).)20 b(The)14 b(I/O)g(en)o(vironmen)o(t) h(describ)q(ed)h(in)e(this)h(section)f(pro)o(vides)h(these)f (facilities.)166 828 y(Emphasis)h(has)f(b)q(een)i(put)e(in)i(remaining) f(as)f Fq(MPI)p Ft(-friendly)i(as)e(p)q(ossible.)22 b(When)14 b(op)q(ening)i(a)e(\014le,)75 885 y(a)k(comm)o(unicator)h(is)g(sp)q (eci\014ed)i(to)d(determine)i(whic)o(h)f(group)g(of)f(pro)q(cesses)h (can)g(get)f(access)h(to)f(the)75 941 y(\014le)g(in)g(subsequen)o(t)g (I/O)g(op)q(erations.)25 b(Accesses)18 b(to)f(a)g(\014le)h(can)f(b)q(e) h(indep)q(enden)o(t)h(\(no)e(co)q(ordination)75 998 y(b)q(et)o(w)o(een) f(pro)q(cesses)g(tak)o(es)f(place\))h(or)f(collectiv)o(e)i(\(eac)o(h)e (pro)q(cess)h(of)f(the)h(group)f(asso)q(ciated)h(with)g(the)75 1054 y(comm)o(unicator)h(m)o(ust)h(participate)g(in)h(the)f(collectiv)o (e)h(access\).)28 b Fq(MPI)17 b Ft(deriv)o(ed)i(datat)o(yp)q(es)e(are)g (used)75 1110 y(for)f(expressing)h(the)f(data)g(la)o(y)o(out)g(in)h (the)f(\014le)i(as)e(w)o(ell)h(as)f(the)g(distribution)i(of)e(the)h (\014le)g(data)f(among)75 1167 y(pro)q(cesses.)31 b(In)19 b(addition,)h(eac)o(h)f(access)f(\(read)h(or)f(write\))g(op)q(erates)h (on)f(a)g(n)o(um)o(b)q(er)h(of)g Fq(MPI)f Ft(ob)s(jects)75 1223 y(whic)o(h)e(can)f(b)q(e)h(of)f(an)o(y)g Fq(MPI)f Ft(basic)i(or)f(deriv)o(ed)h(datat)o(yp)q(es.)75 1345 y Fl(10.1.1)49 b(Data)17 b(P)o(a)o(rtitioning)75 1431 y Ft(Instead)d(of)g(de\014ning)h(\014le)g(access)f(mo)q(des)h(to)e (express)h(the)g(common)g(patterns)f(for)g(accessing)i(a)f(shared)75 1487 y(\014le)21 b(\(broadcast,)f(reduction,)h(scatter,)f(gather\),)g (w)o(e)g(c)o(hose)f(another)h(approac)o(h)f(whic)o(h)i(consists)f(of)75 1544 y(expressing)d(the)g(data)f(partitioning)h(via)g(deriv)o(ed)h (datat)o(yp)q(es.)23 b(Compared)17 b(to)e(a)i(limited)h(set)e(of)h (pre-)75 1600 y(de\014ned)j(access)f(patterns,)g(this)g(approac)o(h)f (has)h(the)g(adv)m(an)o(tage)f(of)g(added)h(\015exibili)q(t)o(y)i(and)d (expres-)75 1657 y(siv)o(eness.)166 1713 y Fq(MPI)11 b Ft(deriv)o(ed)i(datat)o(yp)q(es)f(are)f(used)i(to)e(describ)q(e)j(ho) o(w)e(data)f(is)i(laid)g(out)e(in)i(the)f(user's)g(bu\013er.)19 b(W)l(e)75 1770 y(extend)c(this)h(use)f(to)f(describ)q(e)j(ho)o(w)d (the)h(data)f(is)i(laid)g(out)e(in)i(the)f(\014le)h(as)f(w)o(ell.)20 b(Th)o(us)15 b(w)o(e)g(distinguish)75 1826 y(b)q(et)o(w)o(een)22 b(t)o(w)o(o)d(\(p)q(oten)o(tially)k(di\013eren)o(t\))e(deriv)o(ed)h (datat)o(yp)q(es)f(that)f(are)h(used:)33 b(the)21 b Fq(\014let)o(yp)q (e)p Ft(,)j(whic)o(h)75 1883 y(describ)q(es)c(the)f(la)o(y)o(out)f(in)i (the)e(\014le,)j(and)d(the)h Fq(buft)o(yp)q(e)p Ft(,)i(whic)o(h)f (describ)q(es)g(the)f(la)o(y)o(out)f(in)h(the)g(user's)75 1939 y(bu\013er.)g(In)12 b(addition,)h(b)q(oth)e Fq(\014let)o(yp)q(e)i Ft(and)f Fq(buft)o(yp)q(e)h Ft(are)e(deriv)o(ed)i(from)e(a)g(third)h Fq(MPI)f Ft(datat)o(yp)q(e,)g(referred)75 1995 y(to)k(as)g(the)h Fk(elementary)g Ft(datat)o(yp)q(e)f Fq(et)o(yp)q(e)p Ft(.)22 b(The)16 b(purp)q(ose)h(of)e(the)h(elemen)o(tary)g(datat)o(yp)q (e)f(is)h(to)f(ensure)75 2052 y(consistency)j(b)q(et)o(w)o(een)e(the)h (t)o(yp)q(e)g(signatures)f(of)h Fq(\014let)o(yp)q(e)h Ft(and)e Fq(buft)o(yp)q(e)p Ft(.)26 b(O\013sets)16 b(for)g(accessing)i (data)75 2108 y(within)e(the)g(\014le)g(are)f(expressed)h(as)f(an)g(in) o(tegral)g(n)o(um)o(b)q(er)h(of)e Fq(et)o(yp)q(e)j Ft(items.)166 2165 y(The)d(\014let)o(yp)q(e)g(de\014nes)h(a)e(data)f(pattern)h(that)g (is)h(replicated)h(throughout)e(the)g(\014le)i(\(or)d(part)h(of)g(the) 75 2221 y(\014le|see)19 b(the)f(concept)f(of)g(displacemen)o(t)i(b)q (elo)o(w\))f(to)f(tile)h(the)f(\014le)i(data.)25 b(It)18 b(should)g(b)q(e)g(noted)g(that)75 2278 y Fq(MPI)g Ft(deriv)o(ed)h (datat)o(yp)q(es)e(consist)i(of)e(\014elds)j(of)d(data)h(that)f(are)h (lo)q(cated)h(at)e(sp)q(eci\014ed)k(o\013sets.)27 b(This)75 2334 y(can)15 b(lea)o(v)o(e)h(\\holes")f(b)q(et)o(w)o(een)h(the)f (\014elds,)h(that)e(do)i(not)e(con)o(tain)i(an)o(y)f(data.)k(In)d(the)f (con)o(text)g(of)f(tiling)75 2391 y(the)g(\014le)g(with)g(the)g (\014let)o(yp)q(e,)g(the)g(pro)q(cess)g(can)g(only)g(access)f(the)h (\014le)h(data)e(that)g(matc)o(hes)g(items)g(in)i(the)75 2447 y(\014let)o(yp)q(e.)21 b(It)15 b(cannot)g(access)h(\014le)g(data)e (that)h(falls)h(under)g(holes)g(\(see)f(Figure)g(10.1\).)166 2504 y(Data)f(whic)o(h)i(resides)h(in)f(holes)g(can)f(b)q(e)i(accessed) f(b)o(y)f(other)g(pro)q(cesses)h(whic)o(h)g(use)g(complemen-)75 2560 y(tary)d(\014let)o(yp)q(es)i(\(see)f(Figure)h(10.2\).)j(Th)o(us,)c (\014le)h(data)e(can)h(b)q(e)h(distributed)h(among)d(parallel)j(pro)q (cesses)75 2616 y(in)i(disjoin)o(t)f(c)o(h)o(unks.)26 b(Filet)o(yp)q(e)18 b(constructors)e(help)j(the)e(user)g(create)f (complemen)o(tary)i(\014let)o(yp)q(es)g(for)75 2673 y(common)f (distribution)i(patterns,)d(suc)o(h)i(as)f(broadcast/reduce,)g (scatter/gather,)f(and)h(HPF)g(distri-)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(3)375 289 y @beginspecial 0 @llx 0 @lly 592 @urx 141 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-filetype.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -54.0 181.0 translate 0.900 -0.900 scale 0.500 setlinewidth % Polyline n 179 84 m 179 69 l 159 69 l 159 84 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 199 84 m 199 69 l 179 69 l 179 84 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 179 164 m 179 149 l 59 149 l 59 164 l clp gs col-1 s gr % Polyline n 99 164 m 99 149 l 79 149 l 79 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 119 164 m 119 149 l 99 149 l 99 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 299 164 m 299 149 l 179 149 l 179 164 l clp gs col-1 s gr % Polyline n 219 164 m 219 149 l 199 149 l 199 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 239 164 m 239 149 l 219 149 l 219 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 419 164 m 419 149 l 299 149 l 299 164 l clp gs col-1 s gr % Polyline n 339 164 m 339 149 l 319 149 l 319 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 359 164 m 359 149 l 339 149 l 339 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 539 164 m 539 149 l 419 149 l 419 164 l clp gs col-1 s gr % Polyline n 459 164 m 459 149 l 439 149 l 439 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 479 164 m 479 149 l 459 149 l 459 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 659 164 m 659 149 l 539 149 l 539 164 l clp gs col-1 s gr % Polyline n 579 164 m 579 149 l 559 149 l 559 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 84 m 259 69 l 139 69 l 139 84 l clp gs col-1 s gr % Polyline n 599 164 m 599 149 l 579 149 l 579 164 l clp gs 0.85 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 179 104 m gs 1 -1 scale (holes) dup stringwidth pop 2 div neg 0 rmoveto col-1 show gr 0.000 setlinewidth % Ellipse n 694 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 704 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 714 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 1.000 setlinewidth % Polyline n 699 149 m 59 149 l 59 164 l 699 164 l gs col-1 s gr 0.500 setlinewidth % Interpolated spline n 244 189 m 232.036 184.536 227.036 182.036 224 179 curveto 222.482 177.482 221.232 174.982 219 169 curveto gs col-1 s gr n 219.923 177.194 m 219.000 169.000 l 223.670 175.796 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 244 194 m 180.787 195.844 153.287 194.594 134 189 curveto 126.939 186.952 118.189 181.952 99 169 curveto gs col-1 s gr n 104.512 175.133 m 99.000 169.000 l 106.750 171.818 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 189 m 327.853 187.647 331.603 186.397 334 184 curveto 336.397 181.603 337.647 177.853 339 169 curveto gs col-1 s gr n 335.814 176.606 m 339.000 169.000 l 339.768 177.210 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 194 m 382.589 191.880 410.089 189.380 429 184 curveto 434.743 182.366 442.243 178.616 459 169 curveto gs col-1 s gr n 451.066 171.247 m 459.000 169.000 l 453.057 174.716 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 199 m 448.285 202.925 504.535 200.425 544 189 curveto 551.064 186.955 559.814 181.955 579 169 curveto gs col-1 s gr n 571.251 171.819 m 579.000 169.000 l 573.489 175.134 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 11.00 scalefont setfont 59 139 m gs 1 -1 scale (tiling a file with the filetype:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 249 199 m gs 1 -1 scale (accesible data) col-1 show gr % Polyline n 159 59 m 159 44 l 139 44 l 139 59 l clp gs 0.85 setgray fill gr gs col-1 s gr % Interpolated spline n 159 99 m 155.926 96.344 154.676 95.094 154 94 curveto 152.488 91.554 151.238 87.804 149 79 curveto gs col-1 s gr n 149.033 87.246 m 149.000 79.000 l 152.909 86.261 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 194 99 m 208.466 98.119 214.716 96.869 219 94 curveto 222.030 91.971 224.530 88.221 229 79 curveto gs col-1 s gr n 223.711 85.326 m 229.000 79.000 l 227.310 87.071 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 59 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 573 387 a(Figure)15 b(10.1:)k(Tiling)e(a)e(\014le)h(using) g(a)f(\014let)o(yp)q(e)75 525 y(butions)h(\(see)f(Section)h(10.6\).)375 859 y @beginspecial 0 @llx 0 @lly 593 @urx 135 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-comp-filetypes.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -35.0 175.0 translate 0.900 -0.900 scale % Ellipse n 684 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 694 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 159 59 m 139 59 l 139 44 l 159 44 l clp gs col-1 s gr % Polyline n 259 134 m 259 119 l 139 119 l 139 134 l clp gs col-1 s gr % Polyline n 259 109 m 259 94 l 139 94 l 139 109 l clp gs col-1 s gr % Polyline n 259 84 m 259 69 l 139 69 l 139 84 l clp gs col-1 s gr % Polyline n 179 84 m 159 84 l 159 69 l 179 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 159 84 m 139 84 l 139 69 l 159 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 659 194 m 639 194 l 639 179 l 659 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 59 194 m 39 194 l 39 179 l 59 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 79 194 m 59 194 l 59 179 l 79 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 194 m 159 194 l 159 179 l 179 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 194 m 179 194 l 179 179 l 199 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 194 m 279 194 l 279 179 l 299 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 194 m 299 194 l 299 179 l 319 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 419 194 m 399 194 l 399 179 l 419 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 439 194 m 419 194 l 419 179 l 439 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 539 194 m 519 194 l 519 179 l 539 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 559 194 m 539 194 l 539 179 l 559 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 99 194 m 79 194 l 79 179 l 99 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 119 194 m 99 194 l 99 179 l 119 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 139 194 m 119 194 l 119 179 l 139 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 159 194 m 139 194 l 139 179 l 159 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 219 194 m 199 194 l 199 179 l 219 179 l clp gs 0.65 setgray fill gr gs col-1 s gr 0.000 setlinewidth % Ellipse n 674 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 239 194 m 219 194 l 219 179 l 239 179 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 39 169 m gs 1 -1 scale (tiling a file with the filetypes:) col-1 show gr % Polyline n 259 194 m 239 194 l 239 179 l 259 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 279 194 m 259 194 l 259 179 l 279 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 339 194 m 319 194 l 319 179 l 339 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 359 194 m 339 194 l 339 179 l 359 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 194 m 359 194 l 359 179 l 379 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 399 194 m 379 194 l 379 179 l 399 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 479 194 m 459 194 l 459 179 l 479 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 459 194 m 439 194 l 439 179 l 459 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 499 194 m 479 194 l 479 179 l 499 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 519 194 m 499 194 l 499 179 l 519 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 579 194 m 559 194 l 559 179 l 579 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 599 194 m 579 194 l 579 179 l 599 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 619 194 m 599 194 l 599 179 l 619 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 639 194 m 619 194 l 619 179 l 639 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 199 109 m 179 109 l 179 94 l 199 94 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 219 109 m 199 109 l 199 94 l 219 94 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 134 m 219 134 l 219 119 l 239 119 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 259 134 m 239 134 l 239 119 l 259 119 l clp gs 0.40 setgray fill gr gs col-1 s gr 1.000 setlinewidth % Polyline n 679 179 m 39 179 l 39 194 l 679 194 l gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 59 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (process 1 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 109 m gs 1 -1 scale (process 2 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 134 m gs 1 -1 scale (process 3 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 419 957 a(Figure)f(10.2:)k(P)o(artitioning)c(a)g(\014le)i (among)d(parallel)j(pro)q(cesses)166 1061 y(In)k(order)f(to)g(b)q (etter)g(illustrate)i(these)e(concepts,)i(let)f(us)f(consider)h(a)f (2-D)g(matrix,)h(stored)f(in)75 1117 y(ro)o(w)14 b(ma)s(jor)g(order)i (in)g(a)f(\014le,)h(that)f(is)h(to)f(b)q(e)h(transp)q(osed)f(and)h (partitioned)g(among)f(a)g(group)g(of)g(three)75 1174 y(pro)q(cesses)f(\(see)g(Figure)g(10.3\).)j(The)d(matrix)g(is)g(to)f(b) q(e)h(distributed)h(among)e(the)h(parallel)h(pro)q(cesses)f(in)75 1230 y(a)e(ro)o(w)g(cyclic)i(manner.)19 b(Eac)o(h)12 b(pro)q(cess)h(w)o(an)o(ts)e(to)h(store)g(in)h(its)g(o)o(wn)f(memory)g (the)g(transp)q(osed)h(p)q(ortion)75 1287 y(of)k(the)h(matrix)g(whic)o (h)h(is)f(assigned)h(to)e(it.)28 b(Using)19 b(appropriate)f Fq(\014let)o(yp)q(e)p Ft(s)h(and)f Fq(buft)o(yp)q(e)p Ft(s)i(allo)o(ws)e(the)75 1343 y(user)f(to)g(p)q(erform)g(that)f(task)h (v)o(ery)f(easily)l(.)28 b(In)17 b(addition,)i(the)e(elemen)o(tary)g (datat)o(yp)q(e)g(allo)o(ws)g(one)h(to)75 1400 y(ha)o(v)o(e)d(a)g(v)o (ery)g(generic)h(co)q(de)g(that)e(applies)j(to)d(an)o(y)h(t)o(yp)q(e)g (of)g(2-D)g(matrix.)375 2437 y @beginspecial 0 @llx 0 @lly 673 @urx 549 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-cyclic-trans.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -36.0 562.0 translate 0.900 -0.900 scale 0.500 setlinewidth % Polyline n 399 344 m 319 344 l 319 269 l 399 269 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 279 m 139 279 l 139 264 l 239 264 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 234 m 139 234 l 139 219 l 239 219 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 189 m 139 189 l 139 174 l 239 174 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 144 m 139 144 l 139 129 l 239 129 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 264 m 139 264 l 139 249 l 239 249 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 399 244 m 319 244 l 319 169 l 399 169 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 219 m 139 219 l 139 204 l 239 204 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 249 m 139 249 l 139 234 l 239 234 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 174 m 139 174 l 139 159 l 239 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 129 m 139 129 l 139 114 l 239 114 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 399 144 m 319 144 l 319 69 l 399 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 114 m 139 114 l 139 99 l 239 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 159 m 139 159 l 139 144 l 239 144 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 204 m 139 204 l 139 189 l 239 189 l clp gs 0.90 setgray fill gr gs col-1 s gr % Ellipse n 754 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Ellipse n 769 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Ellipse n 784 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Polyline n 159 99 m 159 279 l gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 99 m 179 279 l 179 274 l gs col-1 s gr % Polyline n 199 99 m 199 279 l gs col-1 s gr % Polyline n 219 99 m 219 279 l gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 164 619 m gs 1 -1 scale (buftype \(all processes\)) dup stringwidth pop neg 0 rmoveto col-1 show gr % Polyline n 579 624 m 579 609 l 519 609 l 519 624 l clp gs col-1 s gr % Polyline n 519 624 m 519 609 l 499 609 l 499 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 499 624 m 499 609 l 439 609 l 439 624 l clp gs col-1 s gr % Polyline n 439 624 m 439 609 l 419 609 l 419 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 339 69 m 339 144 l gs col-1 s gr % Polyline n 359 69 m 359 144 l gs col-1 s gr % Polyline n 379 69 m 379 144 l gs col-1 s gr % Polyline n 319 84 m 399 84 l gs col-1 s gr % Polyline n 319 99 m 399 99 l gs col-1 s gr % Polyline n 319 114 m 399 114 l gs col-1 s gr % Polyline n 319 129 m 399 129 l gs col-1 s gr % Polyline n 319 184 m 399 184 l gs col-1 s gr % Polyline n 319 199 m 399 199 l gs col-1 s gr % Polyline n 319 214 m 399 214 l gs col-1 s gr % Polyline n 319 229 m 399 229 l gs col-1 s gr % Polyline n 319 284 m 399 284 l gs col-1 s gr % Polyline n 319 299 m 399 299 l gs col-1 s gr % Polyline n 319 329 m 399 329 l gs col-1 s gr % Polyline n 339 169 m 339 244 l gs col-1 s gr % Polyline n 359 169 m 359 244 l gs col-1 s gr % Polyline n 379 169 m 379 244 l gs col-1 s gr % Polyline n 379 269 m 379 344 l gs col-1 s gr % Polyline n 359 269 m 359 344 l gs col-1 s gr % Polyline n 339 269 m 339 344 l gs col-1 s gr % Polyline n 319 314 m 399 314 l gs col-1 s gr % Polyline n 179 434 m 179 419 l 159 419 l 159 434 l clp gs col-1 s gr % Polyline n 359 464 m 359 449 l 259 449 l 259 464 l clp gs col-1 s gr % Polyline n 459 449 m 459 464 l 359 464 l 359 449 l clp gs col-1 s gr % Polyline n 259 479 m 259 494 l 159 494 l 159 479 l clp gs col-1 s gr % Polyline n 459 479 m 459 494 l 359 494 l 359 479 l clp gs col-1 s gr % Polyline n 259 509 m 259 524 l 159 524 l 159 509 l clp gs col-1 s gr % Polyline n 359 509 m 359 524 l 259 524 l 259 509 l clp gs col-1 s gr % Polyline n 739 564 m 759 564 l gs col-1 s gr % Polyline n 739 579 m 759 579 l gs col-1 s gr % Polyline n 419 624 m 419 609 l 359 609 l 359 624 l clp gs col-1 s gr % Polyline n 359 624 m 359 609 l 339 609 l 339 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 339 624 m 339 609 l 279 609 l 279 624 l clp gs col-1 s gr % Polyline n 279 624 m 279 609 l 259 609 l 259 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 624 m 259 609 l 199 609 l 199 624 l clp gs col-1 s gr % Polyline n 199 624 m 199 609 l 179 609 l 179 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 464 m 239 464 l 239 449 l 259 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 464 m 219 464 l 219 449 l 239 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 219 464 m 199 464 l 199 449 l 219 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 464 m 179 464 l 179 449 l 199 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 464 m 159 464 l 159 449 l 179 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 59 579 m 39 579 l 39 564 l 59 564 l clp gs 0.90 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 139 524 m gs 1 -1 scale (process 3 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr % Polyline n 79 579 m 59 579 l 59 564 l 79 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 99 579 m 79 579 l 79 564 l 99 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 119 579 m 99 579 l 99 564 l 119 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 139 579 m 119 579 l 119 564 l 139 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 359 579 m 339 579 l 339 564 l 359 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 379 579 m 359 579 l 359 564 l 379 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 399 579 m 379 579 l 379 564 l 399 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 419 579 m 399 579 l 399 564 l 419 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 439 579 m 419 579 l 419 564 l 439 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 659 579 m 639 579 l 639 564 l 659 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 679 579 m 659 579 l 659 564 l 679 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 699 579 m 679 579 l 679 564 l 699 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 719 579 m 699 579 l 699 564 l 719 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 739 579 m 719 579 l 719 564 l 739 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 494 m 279 494 l 279 479 l 299 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 279 494 m 259 494 l 259 479 l 279 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 319 494 m 299 494 l 299 479 l 319 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 339 494 m 319 494 l 319 479 l 339 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 359 494 m 339 494 l 339 479 l 359 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 524 m 359 524 l 359 509 l 379 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 399 524 m 379 524 l 379 509 l 399 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 419 524 m 399 524 l 399 509 l 419 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 439 524 m 419 524 l 419 509 l 439 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 459 524 m 439 524 l 439 509 l 459 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 159 579 m 139 579 l 139 564 l 159 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 179 579 m 159 579 l 159 564 l 179 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 199 579 m 179 579 l 179 564 l 199 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 219 579 m 199 579 l 199 564 l 219 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 579 m 219 579 l 219 564 l 239 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 579 m 239 579 l 239 564 l 259 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 279 579 m 259 579 l 259 564 l 279 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 299 579 m 279 579 l 279 564 l 299 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 319 579 m 299 579 l 299 564 l 319 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 339 579 m 319 579 l 319 564 l 339 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 459 579 m 439 579 l 439 564 l 459 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 479 579 m 459 579 l 459 564 l 479 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 499 579 m 479 579 l 479 564 l 499 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 519 579 m 499 579 l 499 564 l 519 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 539 579 m 519 579 l 519 564 l 539 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 559 579 m 539 579 l 539 564 l 559 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 599 579 m 579 579 l 579 564 l 599 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 579 579 m 559 579 l 559 564 l 579 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 619 579 m 599 579 l 599 564 l 619 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 639 579 m 619 579 l 619 564 l 639 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Interpolated spline n 239 104 m 250.668 101.971 255.668 100.721 259 99 curveto 268.933 93.869 284.450 74.407 294 69 curveto 300.551 65.291 313.415 54.616 324 59 curveto 326.807 60.163 328.057 62.663 329 69 curveto gs col-1 s gr n 329.801 60.793 m 329.000 69.000 l 325.844 61.382 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 149 m 250.644 148.244 255.644 146.994 259 144 curveto 277.543 127.455 267.799 80.162 289 64 curveto 302.704 53.553 328.391 50.305 344 59 curveto 346.370 60.320 347.620 62.820 349 69 curveto gs col-1 s gr n 349.208 60.756 m 349.000 69.000 l 345.305 61.628 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 194 m 250.648 192.789 255.648 191.539 259 189 curveto 267.738 182.380 275.372 162.611 279 154 curveto 288.827 130.673 274.667 70.350 309 49 curveto 326.576 38.071 351.215 49.412 364 59 curveto 365.918 60.438 367.168 62.938 369 69 curveto gs col-1 s gr n 368.600 60.763 m 369.000 69.000 l 364.771 61.921 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 239 m 250.644 238.228 255.644 236.978 259 234 curveto 283.386 212.355 290.390 153.120 299 129 curveto 306.270 108.635 296.232 54.835 329 39 curveto 349.214 29.232 372.340 48.608 384 59 curveto 385.671 60.489 386.921 62.989 389 69 curveto gs col-1 s gr n 388.275 60.786 m 389.000 69.000 l 384.495 62.093 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 119 m 250.654 121.454 255.654 122.704 259 124 curveto 274.976 130.186 310.971 145.570 324 159 curveto 325.480 160.525 326.730 163.025 329 169 curveto gs col-1 s gr n 328.028 160.811 m 329.000 169.000 l 324.289 162.232 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 164 m 250.581 164.100 255.581 164.100 259 164 curveto 278.409 163.430 320.511 145.420 344 159 curveto 346.313 160.337 347.563 162.837 349 169 curveto gs col-1 s gr n 349.131 160.755 m 349.000 169.000 l 345.236 161.663 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 209 m 250.636 206.383 255.636 205.133 259 204 curveto 283.853 195.628 323.491 144.162 364 159 curveto 366.973 160.089 368.223 162.589 369 169 curveto gs col-1 s gr n 370.023 160.817 m 369.000 169.000 l 366.052 161.299 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 254 m 250.666 252.137 255.666 250.887 259 249 curveto 268.147 243.821 283.511 227.619 289 219 curveto 296.572 207.110 294.164 175.146 309 164 curveto 328.055 149.685 363.140 147.049 384 159 curveto 386.327 160.333 387.577 162.833 389 169 curveto gs col-1 s gr n 389.150 160.755 m 389.000 169.000 l 385.253 161.655 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 134 m 250.647 135.150 255.647 136.400 259 139 curveto 288.594 161.953 309.637 231.415 324 259 curveto 324.882 260.695 326.132 263.195 329 269 curveto gs col-1 s gr n 327.250 260.942 m 329.000 269.000 l 323.663 262.714 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 179 m 250.656 180.494 255.656 181.744 259 184 curveto 273.708 193.922 285.963 228.455 299 239 curveto 308.440 246.636 334.739 250.151 344 259 curveto 345.577 260.507 346.827 263.007 349 269 curveto gs col-1 s gr n 348.153 260.797 m 349.000 269.000 l 344.393 262.161 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 224 m 250.662 226.332 255.662 227.582 259 229 curveto 269.183 233.327 288.080 250.409 299 254 curveto 314.092 258.962 348.484 251.864 364 259 curveto 366.524 260.161 369.024 262.661 374 269 curveto gs col-1 s gr n 370.633 261.472 m 374.000 269.000 l 367.487 263.942 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 269 m 250.469 265.757 255.469 264.507 259 264 curveto 287.568 259.902 349.983 238.905 384 259 curveto 386.280 260.347 387.530 262.847 389 269 curveto gs col-1 s gr n 389.086 260.754 m 389.000 269.000 l 385.196 261.684 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 12.00 scalefont setfont 39 29 m gs 1 -1 scale (logical view: partition file in row cyclic pattern and transpose) col-1 show gr /Helvetica findfont 12.00 scalefont setfont 39 399 m gs 1 -1 scale (implementation using etype, filetypes, and buftypes) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 119 m gs 1 -1 scale (file structure) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 84 m gs 1 -1 scale (process 1 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 184 m gs 1 -1 scale (process 2 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 284 m gs 1 -1 scale (process 3 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 39 554 m gs 1 -1 scale (actual layout in the file:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 434 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 464 m gs 1 -1 scale (process 1 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 494 m gs 1 -1 scale (process 2 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 420 2535 a(Figure)h(10.3:)i(T)l(ransp)q(osing)e(and)f (partitioning)h(a)f(2-D)g(matrix)166 2641 y(Note)d(that)h(using)g Fq(MPI)f Ft(deriv)o(ed)i(datat)o(yp)q(es)e(leads)i(to)e(the)h(p)q (ossibilit)o(y)i(of)d(v)o(ery)h(\015exible)i(patterns.)75 2698 y(F)l(or)d(example,)i(the)f(\014let)o(yp)q(es)h(need)g(not)e (distribute)i(the)f(data)f(in)i(rank)f(order.)18 b(In)c(addition,)g (there)f(can)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Ft(4)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(b)q(e)17 b(o)o(v)o(erlaps)g(b)q(et)o(w)o(een)f(the)h (data)f(items)h(that)f(are)g(accessed)h(b)o(y)g(di\013eren)o(t)g(pro)q (cesses.)24 b(The)17 b(extreme)75 106 y(case)e(of)g(full)i(o)o(v)o (erlap)d(is)i(the)f(broadcast/reduce)h(pattern.)166 162 y(Using)c(the)f(\014let)o(yp)q(e)i(allo)o(ws)e(a)g(certain)h(access)g (pattern)f(to)f(b)q(e)i(established.)21 b(But)11 b(it)h(is)g(conceiv)m (able)75 219 y(that)j(a)h(single)i(pattern)d(w)o(ould)i(not)e(b)q(e)i (suitable)h(for)d(the)h(whole)h(\014le.)24 b(The)16 b(solution)h(is)f (to)g(de\014ne)h(an)75 275 y(absolute)d(b)o(yte)f(displacemen)o(t)i (from)e(the)g(b)q(eginning)j(of)d(the)h(\014le,)g(and)g(ha)o(v)o(e)f (the)h(access)f(pattern)g(start)75 332 y(from)f(that)f(displacemen)o (t.)21 b(Th)o(us)12 b(if)h(a)f(\014le)i(has)e(t)o(w)o(o)f(segmen)o(ts)h (that)f(need)j(to)d(b)q(e)j(accessed)f(in)g(di\013eren)o(t)75 388 y(patterns,)e(the)f(displacemen)o(t)j(for)d(the)g(second)h(pattern) g(will)h(skip)f(o)o(v)o(er)f(the)h(whole)g(\014rst)f(segmen)o(t.)18 b(This)75 444 y(mec)o(hanism)c(is)g(also)f(particularly)i(useful)f(for) f(handling)i(\014les)f(with)g(some)f(header)h(information)f(at)g(the)75 501 y(b)q(eginning)20 b(\(see)d(Figure)h(10.4\).)25 b(Use)17 b(of)g(\014le)i(headers)f(could)g(allo)o(w)g(the)f(supp)q(ort)h(of)f (heterogeneous)75 557 y(en)o(vironmen)o(ts)k(b)o(y)g(storing)g(a)f (\\standard")g(co)q(di\014cation)j(of)d(the)h(data)g(represen)o (tations)f(and)i(data)75 614 y(t)o(yp)q(es)15 b(of)g(the)g(\014le)h (data.)375 970 y @beginspecial 0 @llx 0 @lly 502 @urx 126 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-disp.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -36.0 189.0 translate 0.900 -0.900 scale % Ellipse n 574 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 594 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 1.000 setlinewidth % Polyline n 579 159 m 39 159 l 39 174 l 579 174 l gs col-1 s gr 0.500 setlinewidth % Polyline n 119 209 m 119 179 l gs col-1 s gr n 117.000 187.000 m 119.000 179.000 l 121.000 187.000 l gs 2 setlinejoin col-1 s gr % Polyline n 319 209 m 319 179 l gs col-1 s gr n 317.000 187.000 m 319.000 179.000 l 321.000 187.000 l gs 2 setlinejoin col-1 s gr % Polyline n 159 84 m 139 84 l 139 69 l 159 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 84 m 179 84 l 179 69 l 199 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 84 m 219 84 l 219 69 l 239 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 279 84 m 259 84 l 259 69 l 279 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 84 m 299 84 l 299 69 l 319 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 114 m 139 114 l 139 99 l 199 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 114 m 259 114 l 259 99 l 319 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 139 174 m 119 174 l 119 159 l 139 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 174 m 159 174 l 159 159 l 179 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 219 174 m 199 174 l 199 159 l 219 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 259 174 m 239 174 l 239 159 l 259 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 174 m 279 174 l 279 159 l 299 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 379 174 m 319 174 l 319 159 l 379 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 499 174 m 439 174 l 439 159 l 499 159 l clp gs 0.90 setgray fill gr gs col-1 s gr 0.000 setlinewidth % Ellipse n 584 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 179 84 m 159 84 l 159 69 l 179 69 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 324 204 m gs 1 -1 scale (second displacement) col-1 show gr % Polyline n 219 84 m 199 84 l 199 69 l 219 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 84 m 239 84 l 239 69 l 259 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 299 84 m 279 84 l 279 69 l 299 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 339 84 m 319 84 l 319 69 l 339 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 114 m 199 114 l 199 99 l 259 99 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 114 m 319 114 l 319 99 l 379 99 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 159 174 m 139 174 l 139 159 l 159 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 199 174 m 179 174 l 179 159 l 199 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 174 m 219 174 l 219 159 l 239 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 279 174 m 259 174 l 259 159 l 279 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 319 174 m 299 174 l 299 159 l 319 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 439 174 m 379 174 l 379 159 l 439 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 559 174 m 499 174 l 499 159 l 559 159 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (first tiling) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 109 m gs 1 -1 scale (second tiling) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 39 149 m gs 1 -1 scale (file structure:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 79 169 m gs 1 -1 scale (header) dup stringwidth pop 2 div neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 124 204 m gs 1 -1 scale (first displacement) col-1 show gr $F2psEnd %%EndDocument @endspecial 704 1068 a(Figure)g(10.4:)j(Displacemen)o(ts)75 1233 y Fl(10.1.2)49 b(Data)17 b(Access)e(F)o(unctions)75 1319 y Ft(Data)f(is)i(mo)o(v)o(ed)f(b)q(et)o(w)o(een)g(\014les)h(and)g (pro)q(cesses)f(b)o(y)h(issuing)g(read)f(and)h(write)f(calls.)21 b(There)16 b(are)f(three)75 1375 y(orthogonal)20 b(asp)q(ects)i(to)e (data)h(access:)32 b(p)q(ositioning)23 b(\(explicit)g(o\013set)d(vs.)38 b(implicit)24 b(\014le)e(p)q(oin)o(ter\),)75 1432 y(sync)o(hronism)14 b(\(blo)q(c)o(king)h(vs.)k(non)o(blo)q(c)o(king\),)c(and)e(co)q (ordination)i(\(indep)q(enden)o(t)h(vs.)j(collectiv)o(e\).)i(All)75 1488 y(com)o(binations)c(of)f(these)h(data)f(access)g(routines,)h (including)j(t)o(w)o(o)15 b(t)o(yp)q(es)h(of)g(\014le)i(p)q(oin)o (ters,)f(individual)75 1545 y(and)e(shared,)g(are)g(pro)o(vided.)p 190 1597 1570 2 v 189 1653 2 57 v 215 1636 a Fo(p)q(ositioning)p 491 1653 V 500 1653 V 61 w(sync)o(hronism)p 850 1653 V 859 1653 V 356 w(co)q(ordination)p 1759 1653 V 859 1655 901 2 v 189 1710 2 57 v 491 1710 V 500 1710 V 850 1710 V 859 1710 V 884 1693 a Fk(indep)n(endent)p 1276 1710 V 181 w(c)n(ol)r(le)n(ctive)p 1759 1710 V 190 1711 1570 2 v 190 1720 V 189 1776 2 57 v 215 1759 a(explicit)p 491 1776 V 500 1776 V 169 w(blo)n(cking)p 850 1776 V 859 1776 V 202 w Ft(MPI)p 976 1759 14 2 v 16 w(Read)p 1276 1776 2 57 v 212 w(MPI)p 1394 1759 14 2 v 17 w(Read)p 1512 1759 V 17 w(all)p 1759 1776 2 57 v 189 1833 V 215 1816 a Fk(o\013sets)p 491 1833 V 500 1833 V 185 w Ft(\(sync)o (hronous\))p 850 1833 V 859 1833 V 80 w(MPI)p 976 1816 14 2 v 16 w(W)l(rite)p 1276 1833 2 57 v 202 w(MPI)p 1394 1816 14 2 v 17 w(W)l(rite)p 1522 1816 V 16 w(all)p 1759 1833 2 57 v 500 1834 1261 2 v 189 1889 2 57 v 491 1889 V 500 1889 V 525 1872 a Fk(nonblo)n(cking)p 850 1889 V 859 1889 V 127 w Ft(MPI)p 976 1872 14 2 v 16 w(Iread)p 1276 1889 2 57 v 211 w(MPI)p 1394 1872 14 2 v 17 w(Iread)p 1513 1872 V 16 w(all)p 1759 1889 2 57 v 189 1945 V 491 1945 V 500 1945 V 525 1928 a(\(async)o(hronous\))p 850 1945 V 859 1945 V 57 w(MPI)p 976 1928 14 2 v 16 w(Iwrite)p 1276 1945 2 57 v 196 w(MPI)p 1394 1928 14 2 v 17 w(Iwrite)p 1528 1928 V 16 w(all)p 1759 1945 2 57 v 190 1947 1570 2 v 189 2004 2 57 v 215 1987 a Fk(individual)p 491 2004 V 500 2004 V 116 w(blo)n(cking)p 850 2004 V 859 2004 V 202 w Ft(MPI)p 976 1987 14 2 v 16 w(Read)p 1093 1987 V 17 w(next)p 1276 2004 2 57 v 108 w(MPI)p 1394 1987 14 2 v 17 w(Read)p 1512 1987 V 17 w(next)p 1616 1987 V 16 w(all)p 1759 2004 2 57 v 189 2060 V 215 2043 a Fk(\014le)g(p)n (ointers)p 491 2060 V 500 2060 V 78 w Ft(\(sync)o(hronous\))p 850 2060 V 859 2060 V 80 w(MPI)p 976 2043 14 2 v 16 w(W)l(rite)p 1103 2043 V 17 w(next)p 1276 2060 2 57 v 98 w(MPI)p 1394 2043 14 2 v 17 w(W)l(rite)p 1522 2043 V 16 w(next)p 1625 2043 V 17 w(all)p 1759 2060 2 57 v 500 2062 1261 2 v 189 2116 2 57 v 491 2116 V 500 2116 V 525 2100 a Fk(nonblo)n(cking)p 850 2116 V 859 2116 V 127 w Ft(MPI)p 976 2100 14 2 v 16 w(Iread)p 1094 2100 V 17 w(next)p 1276 2116 2 57 v 107 w(MPI)p 1394 2100 14 2 v 17 w(Iread)p 1513 2100 V 16 w(next)p 1616 2100 V 17 w(all)p 1759 2116 2 57 v 189 2173 V 491 2173 V 500 2173 V 525 2156 a(\(async)o(hronous\))p 850 2173 V 859 2173 V 57 w(MPI)p 976 2156 14 2 v 16 w(Iwrite)p 1109 2156 V 17 w(next)p 1276 2173 2 57 v 92 w(MPI)p 1394 2156 14 2 v 17 w(Iwrite)p 1528 2156 V 16 w(next)p 1631 2156 V 17 w(all)p 1759 2173 2 57 v 190 2175 1570 2 v 189 2231 2 57 v 215 2214 a Fk(shar)n(e)n(d)p 491 2231 V 500 2231 V 186 w(blo)n(cking)p 850 2231 V 859 2231 V 202 w Ft(MPI)p 976 2214 14 2 v 16 w(Read)p 1093 2214 V 17 w(shared)p 1276 2231 2 57 v 66 w(MPI)p 1394 2214 14 2 v 17 w(Read)p 1512 2214 V 17 w(shared)p 1658 2214 V 16 w(all)p 1759 2231 2 57 v 189 2287 V 215 2271 a Fk(\014le)g(p)n (ointer)p 491 2287 V 500 2287 V 97 w Ft(\(sync)o(hronous\))p 850 2287 V 859 2287 V 80 w(MPI)p 976 2271 14 2 v 16 w(W)l(rite)p 1103 2271 V 17 w(shared)p 1276 2287 2 57 v 56 w(MPI)p 1394 2271 14 2 v 17 w(W)l(rite)p 1522 2271 V 16 w(shared)p 1667 2271 V 17 w(all)p 1759 2287 2 57 v 500 2289 1261 2 v 189 2344 2 57 v 491 2344 V 500 2344 V 525 2327 a Fk(nonblo)n(cking)p 850 2344 V 859 2344 V 127 w Ft(MPI)p 976 2327 14 2 v 16 w(Iread)p 1094 2327 V 17 w(shared)p 1276 2344 2 57 v 65 w(MPI)p 1394 2327 14 2 v 17 w(Iread)p 1513 2327 V 16 w(shared)p 1658 2327 V 17 w(all)p 1759 2344 2 57 v 189 2400 V 491 2400 V 500 2400 V 525 2383 a(\(async)o(hronous\))p 850 2400 V 859 2400 V 57 w(MPI)p 976 2383 14 2 v 16 w(Iwrite)p 1109 2383 V 17 w(shared)p 1276 2400 2 57 v 50 w(MPI)p 1394 2383 14 2 v 17 w(Iwrite)p 1528 2383 V 16 w(shared)p 1673 2383 V 17 w(all)p 1759 2400 2 57 v 190 2402 1570 2 v 166 2478 a(UNIX)h(read\(\))e(and)i (write\(\))e(are)h(blo)q(c)o(king,)h(indep)q(enden)o(t)i(op)q (erations,)d(and)g(use)h(individual)i(\014le)75 2534 y(p)q(oin)o(ters.)i(The)c Fq(MPI)e Ft(equiv)m(alen)o(ts)j(are)e Fq(MPI)p 852 2534 14 2 v 16 w(READ)p 987 2534 V 17 w(NEXT)h Ft(and)f Fq(MPI)p 1310 2534 V 16 w(WRITE)p 1469 2534 V 16 w(NEXT)p Ft(.)166 2591 y(Op)q(erations)f(with)g(explicit)h (o\013sets)d(are)h(describ)q(ed)i(in)f(Section)h(10.3.)j(Op)q(erations) c(with)f(individ-)75 2647 y(ual)i(\014le)h(p)q(oin)o(ters)g(are)e (describ)q(ed)j(in)f(Section)g(10.4.)i(And)e(op)q(erations)f(with)g (shared)g(\014le)h(p)q(oin)o(ters)f(are)75 2704 y(describ)q(ed)i(in)f (Section)g(10.5.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(5)75 49 y Fl(10.1.3)49 b(P)o(ositioning)75 135 y Ft(UNIX)19 b(\014le)g(systems)f(traditionally)i(main)o(tain)f(a)f (system)g(\014le)h(p)q(oin)o(ter)g(sp)q(ecifying)h(what)e(o\013set)f (will)75 192 y(b)q(e)k(used)f(for)f(the)h(read)g(or)f(write)h(op)q (eration.)34 b(The)20 b(problem)h(with)f(this)g(in)o(terface)g(is)h (that)e(it)h(w)o(as)75 248 y(primarily)f(designed)g(for)e(\014les)h(b)q (eing)h(accessed)f(b)o(y)f(a)h(single)h(pro)q(cess.)27 b(In)18 b(a)f(parallel)i(en)o(vironmen)o(t,)75 304 y(w)o(e)14 b(m)o(ust)f(decide)j(whether)e(a)f(\014le)i(p)q(oin)o(ter)g(is)f (shared)g(b)o(y)g(m)o(ultiple)i(pro)q(cesses)e(or)g(if)g(an)g (individual)j(\014le)75 361 y(p)q(oin)o(ter)d(will)i(b)q(e)f(main)o (tained)g(b)o(y)f(eac)o(h)g(pro)q(cess.)19 b(In)c(addition,)g(parallel) h(programs)c(do)i(not)g(generally)75 417 y(exhibit)19 b(lo)q(calit)o(y)f(of)f(reference)h(within)h(a)e(\014le)h([6)o(].)26 b(Instead,)18 b(they)f(tend)h(to)f(mo)o(v)o(e)f(b)q(et)o(w)o(een)i (distinct)75 474 y(non-con)o(tiguous)h(regions)f(of)g(a)g(\014le.)29 b(This)19 b(means)f(that)g(the)g(pro)q(cess)g(m)o(ust)g(seek)g(on)g (almost)g(ev)o(ery)75 530 y(read)12 b(or)g(write)g(op)q(eration.)19 b(Moreo)o(v)o(er,)11 b(in)i(m)o(ulti-threaded)h(en)o(vironmen)o(ts)e (or)g(when)h(p)q(erforming)g(I/O)75 587 y(async)o(hronously)l(,)k(it)g (is)g(di\016cult)i(to)d(ensure)h(that)f(the)h(\014le)g(p)q(oin)o(ter)h (will)g(b)q(e)f(in)h(the)f(correct)f(p)q(osition)75 643 y(when)g(the)f(read)g(or)g(write)g(o)q(ccurs.)166 700 y Fq(MPI)10 b Ft(pro)o(vides)i(separate)e(routines)i(for)e(p)q (ositioning)j(with)e(explicit)j(o\013sets,)c(individual)k(\014le)e(p)q (oin)o(t-)75 756 y(ers,)18 b(and)g(shared)g(\014le)g(p)q(oin)o(ters.)28 b(The)18 b(explicit)i(o\013set)d(op)q(erations)h(require)g(the)g(user)g (to)f(sp)q(ecify)i(an)75 813 y(o\013set,)e(and)h(act)f(as)h(atomic)f (seek-and-read)i(or)e(seek-and-write)i(op)q(erations.)28 b(The)18 b(individual)j(and)75 869 y(shared)d(\014le)h(p)q(oin)o(ter)g (op)q(erations)f(use)g(distinct)i(system)d(main)o(tained)i(o\013sets)e (for)h(p)q(ositioning.)30 b(The)75 925 y(di\013eren)o(t)21 b(p)q(ositioning)i(metho)q(ds)f(are)f(orthogonal;)i(they)e(ma)o(y)f(b)q (e)i(mixed)g(within)h(the)e(same)g(pro-)75 982 y(gram,)15 b(and)i(they)f(do)g(not)g(a\013ect)f(eac)o(h)h(other.)23 b(In)17 b(other)e(w)o(ords,)h(an)g(individual)j(\014le)e(p)q(oin)o (ter's)g(v)m(alue)75 1038 y(will)h(b)q(e)e(unc)o(hanged)h(b)o(y)f (executing)h(explicit)h(o\013set)d(op)q(erations)h(or)g(shared)g (\014le)h(p)q(oin)o(ter)f(op)q(erations.)75 1095 y(The)g(data)g(access) g(routines)g(whic)o(h)h(accept)g(explicit)h(o\013sets)d(ha)o(v)o(e)h (no)g(extensions)g(\(e.g.)22 b Fq(MPI)p 1741 1095 14 2 v 16 w(XXX)p Ft(\),)75 1151 y(the)16 b(individual)k(\014le)d(p)q(oin) o(ter)f(routines)h(ha)o(v)o(e)p 884 1151 V 32 w Fq(NEXT)g Ft(app)q(ended)h(\(e.g.)j Fq(MPI)p 1435 1151 V 16 w(XXX)p 1541 1151 V 18 w(NEXT)p Ft(\),)16 b(and)g(the)75 1208 y(shared)i(\014le)i(p)q(oin)o(ter)f(routines)f(ha)o(v)o(e)p 747 1208 V 35 w Fq(SHARED)h Ft(app)q(ended)h(\(e.g.)29 b Fq(MPI)p 1366 1208 V 15 w(XXX)p 1471 1208 V 18 w(SHARED)p Ft(\).)18 b(In)h(order)75 1264 y(to)d(allo)o(w)g(the)g(implicit)j (o\013sets)c(to)h(b)q(e)h(set,)f(t)o(w)o(o)f(seek)h(routines)h(are)f (also)g(pro)o(vided)h(\()p Fq(MPI)p 1662 1264 V 15 w(SEEK)g Ft(and)75 1321 y Fq(MPI)p 160 1321 V 16 w(SEEK)p 287 1321 V 16 w(SHARED)p Ft(\).)166 1377 y(In)c(general,)h(\014le)g(p)q (oin)o(ter)f(op)q(erations)g(ha)o(v)o(e)g(the)g(same)g(seman)o(tics)g (as)f(explicit)j(o\013set)d(op)q(erations,)75 1434 y(with)k(the)f Fq(o\013set)g Ft(argumen)o(t)g(set)g(to)f(the)h(curren)o(t)h(v)m(alue)g (of)f(the)g(system-main)o(tained)h(\014le)g(p)q(oin)o(ter.)75 1554 y Fq(Explicit)g(O\013sets)75 1639 y Ft(There)h(are)g(t)o(w)o(o)f (\\k)o(eys")g(to)h(describ)q(e)h(lo)q(cations)g(in)g(a)f(\014le:)25 b(an)17 b Fq(MPI)f Ft(datat)o(yp)q(e)h(and)g(an)g(o\013set.)25 b Fq(MPI)75 1696 y Ft(datat)o(yp)q(es)14 b(are)h(used)h(as)f (templates,)g(tiling)h(the)f(\014le,)h(and)f(an)g(o\013set)g (determines)h(an)f(initial)i(p)q(osition)75 1752 y(for)g(transfers.)26 b(O\013sets)18 b(are)f(expressed)h(as)g(an)f(in)o(tegral)h(n)o(um)o(b)q (er)g(of)f(elemen)o(tary)h(datat)o(yp)q(e)f(\()p Fq(et)o(yp)q(e)p Ft(\))75 1809 y(items)d(relativ)o(e)h(to)e(the)i Fq(\014let)o(yp)q(e)p Ft(.)21 b(Holes)14 b(in)h(the)f Fq(\014let)o(yp)q(e)i Ft(are)e(ignored)g(when)h(calculating)h(the)e(p)q(osition)75 1865 y(of)f(an)h(o\013set)e(\(e.g.)19 b(an)13 b(o\013set)g(of)g(2)g (for)g(pro)q(cess)h(1)f(in)h(Figure)g(10.2)e(denotes)i(the)g(7th)f Fq(et)o(yp)q(e)i Ft(in)f(the)g(\014le\).)75 1922 y(The)j Fq(et)o(yp)q(e)i Ft(argumen)o(t)d(is)i(asso)q(ciated)g(with)f(a)g (\014le)h(and)g(used)f(to)g(express)h(the)f Fq(\014let)o(yp)q(e)p Ft(,)i Fq(buft)o(yp)q(e)g Ft(and)75 1978 y Fq(o\013set)h Ft(argumen)o(ts.)31 b(Therefore,)20 b(the)g Fq(\014let)o(yp)q(e)h Ft(and)e Fq(buft)o(yp)q(e)i Ft(datat)o(yp)q(es)e(m)o(ust)g(b)q(e)h (directly)h(deriv)o(ed)75 2035 y(from)14 b Fq(et)o(yp)q(e)p Ft(,)i(or)f(their)h(t)o(yp)q(e)f(signatures)g(m)o(ust)g(b)q(e)h(a)f(m)o (ultiple)i(of)e(the)g Fq(et)o(yp)q(e)h Ft(signature.)75 2155 y Fq(File)f(P)o(ointers)75 2240 y Ft(When)h(a)f(\014le)h(is)g(op)q (ened,)h(the)e(system)g(creates)g(a)h(set)f(of)g(\014le)h(p)q(oin)o (ters)g(to)f(k)o(eep)h(trac)o(k)e(of)i(the)f(curren)o(t)75 2297 y(\014le)j(p)q(osition.)25 b(One)18 b(is)f(a)g(global)g(\014le)h (p)q(oin)o(ter,)f Fk(shar)n(e)n(d)f Ft(b)o(y)h(all)h(the)f(pro)q (cesses)g(in)g(the)g(comm)o(unicator)75 2353 y(group.)29 b(The)19 b(others)f(are)g(individual)k(\014le)e(p)q(oin)o(ters)f(lo)q (cal)g(to)f(eac)o(h)h(pro)q(cess)f(in)i(the)e(comm)o(unicator)75 2410 y(group,)c(and)i(can)f(b)q(e)h(up)q(dated)g Fk(indep)n(endently.)i Ft(A)d(shared)h(\014le)g(p)q(oin)o(ter)f(only)h(mak)o(es)f(sense)g(if)h (all)g(the)75 2466 y(pro)q(cesses)j(can)f(access)h(the)f(same)g (dataset.)29 b(This)19 b(means)f(that)g(all)h(the)g(pro)q(cesses)g (should)g(use)g(the)75 2523 y(same)c Fq(\014let)o(yp)q(e)i Ft(when)e(op)q(ening)i(the)e(\014le.)166 2579 y(The)f(main)g(seman)o (tic)h(issue)f(with)h(system-main)o(tained)f(\014le)h(p)q(oin)o(ters)g (is)f(ho)o(w)f(they)i(are)e(up)q(dated)75 2636 y(b)o(y)18 b(I/O)h(op)q(erations.)29 b(In)19 b(general,)g(eac)o(h)g(I/O)f(op)q (eration)h(lea)o(v)o(es)f(the)h(\014le)g(p)q(oin)o(ter)g(p)q(oin)o (ting)g(to)f(the)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Ft(6)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(next)15 b(data)e(item)i(after)f(the)h(last)f(one)h (that)f(w)o(as)f(accessed.)21 b(This)15 b(principle)i(applies)f(to)e(b) q(oth)h(t)o(yp)q(es)f(of)75 106 y(\014le)i(p)q(oin)o(ters)g (\(individual)i(and)d(shared\),)g(and)g(to)g(all)h(t)o(yp)q(es)f(of)g (I/O)h(op)q(erations.)166 162 y(It)f(is)h(p)q(ossible)h(to)d(formalize) i(the)f(up)q(date)h(pro)q(cedure)g(as)f(follo)o(ws:)189 277 y Fi(new)p 274 277 14 2 v 17 w(f)5 b(il)q(e)p 369 277 V 16 w(position)13 b Ft(=)g Fi(ol)q(d)p 670 277 V 15 w(position)e Ft(+)906 247 y Fi(siz)r(e)p Ft(\()p Fi(buf)5 b(ty)r(pe)p Ft(\))10 b Fh(\002)g Fi(buf)5 b(count)p 906 267 514 2 v 1052 309 a(siz)r(e)p Ft(\()p Fi(ety)r(pe)p Ft(\))75 393 y(where)24 b Fi(buf)5 b(count)23 b Ft(is)h(the)f(n)o(um)o (b)q(er)h(of)f(elemen)o(ts)h(of)f(t)o(yp)q(e)g Fi(buf)5 b(ty)r(pe)24 b Ft(to)e(b)q(e)i(accessed)g(and)g(where)75 449 y Fi(siz)r(e)p Ft(\()p Fi(dataty)r(pe)p Ft(\))19 b(giv)o(es)f(the)h(n)o(um)o(b)q(er)g(of)f(b)o(ytes)h(of)f(actual)h (data)f(\(excluding)i(holes\))f(that)f(comp)q(oses)75 506 y(the)d Fq(MPI)g Ft(datat)o(yp)q(e)g Fi(dataty)r(pe)p Ft(.)166 562 y(Another)j(complication)j(with)d(UNIX)i(I/O)f(op)q (erations,)g(is)g(that)f(the)h(system-main)o(tained)g(\014le)75 619 y(p)q(oin)o(ter)c(is)g(normally)f(only)h(up)q(dated)g(when)g(the)g (op)q(eration)f(completes.)20 b(A)o(t)14 b(that)g(stage,)f(it)h(is)h (kno)o(wn)75 675 y(exactly)h(ho)o(w)f(m)o(uc)o(h)h(data)f(w)o(as)g (actually)h(accessed)h(\(whic)o(h)f(can)g(b)q(e)g(di\013eren)o(t)g (from)f(the)h(amoun)o(t)f(re-)75 732 y(quested\),)i(and)g(the)g(\014le) h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o(y)f(that)f(amoun)o(t.)24 b(F)l(or)16 b(non)o(blo)q(c)o(king)j(accesses)e(made)75 788 y(using)e(an)g(individual)j(or)c(the)g(shared)h(\014le)h(p)q(oin)o (ter,)e(the)h(up)q(date)g(cannot)g(b)q(e)g(dela)o(y)o(ed)g(un)o(til)h (the)f(op)q(er-)75 845 y(ation)g(completes,)h(b)q(ecause)g(additional)h (accesses)e(can)g(b)q(e)h(initiated)h(b)q(efore)f(that)e(time)i(b)o(y)f (the)g(same)75 901 y(pro)q(cess)i(\(for)e(b)q(oth)i(t)o(yp)q(es)g(of)f (\014le)h(p)q(oin)o(ters\))g(or)f(b)o(y)h(other)f(pro)q(cesses)h(\(for) e(the)i(shared)g(\014le)g(p)q(oin)o(ter\).)75 958 y(Therefore,)c(the)h (\014le)g(p)q(oin)o(ter)g(m)o(ust)f(b)q(e)h(up)q(dated)g(at)f(the)h (outset,)f(b)o(y)g(the)g(amoun)o(t)g(of)g(data)g(requested.)166 1014 y(Similarly)l(,)k(when)e(blo)q(c)o(king)i(accesses)e(are)g(made)g (using)g(the)h(shared)f(\014le)h(p)q(oin)o(ter,)f(up)q(dating)h(the)75 1070 y(\014le)k(p)q(oin)o(ter)f(at)e(the)i(completion)h(of)e(eac)o(h)g (access)h(w)o(ould)g(ha)o(v)o(e)f(the)h(same)f(e\013ect)g(as)g (serializing)j(all)75 1127 y(blo)q(c)o(king)e(accesses)f(to)e(the)i (\014le.)28 b(In)18 b(order)f(to)g(prev)o(en)o(t)g(this,)h(the)g (shared)f(\014le)i(p)q(oin)o(ter)f(for)f(blo)q(c)o(king)75 1183 y(accesses)g(is)g(up)q(dated)g(at)f(the)h(b)q(eginning)i(of)d(eac) o(h)g(access)h(b)o(y)g(the)f(amoun)o(t)g(of)g(data)g(requested.)25 b(F)l(or)75 1240 y(blo)q(c)o(king)13 b(accesses)f(using)h(an)f (individual)j(\014le)d(p)q(oin)o(ter,)h(up)q(dating)g(the)f(\014le)g(p) q(oin)o(ter)h(at)e(the)h(completion)75 1296 y(of)j(eac)o(h)h(access)f (w)o(ould)h(b)q(e)g(p)q(erfectly)h(v)m(alid.)22 b(Ho)o(w)o(ev)o(er,)14 b(in)i(order)f(to)g(main)o(tain)h(the)g(same)f(seman)o(tics)75 1353 y(for)f(all)i(t)o(yp)q(es)f(of)f(accesses)h(using)h(\014le)f(p)q (oin)o(ters,)g(the)g(up)q(date)h(of)e(the)h(\014le)h(p)q(oin)o(ter)f (in)h(this)f(case)g(is)g(also)75 1409 y(made)g(at)g(the)g(b)q(eginning) i(of)e(the)g(access)h(b)o(y)f(the)g(amoun)o(t)g(of)f(data)h(requested.) 166 1466 y(Although)c(consisten)o(t,)g(and)g(seman)o(tically)h (cleaner,)g(up)q(dating)g(the)f(\014le)g(p)q(oin)o(ter)g(at)f(the)h (initiation)75 1522 y(of)20 b(all)g(I/O)h(op)q(erations)f(di\013ers)g (from)f(accepted)i(UNIX)f(practice,)i(and)e(ma)o(y)f(lead)i(to)e(unexp) q(ected)75 1579 y(results.)h(Consider)c(the)f(follo)o(wing)h(scenario:) 266 1685 y Fp(MPI_Read_Next\(fh,)21 b(buff,)j(buftype,)e(bufcount,)h (&status\);)266 1741 y(MPI_Write_Next\(fh,)e(buff,)i(buftype,)g (bufcount,)g(&status\);)166 1848 y Ft(If)f(the)g(\014rst)g(read)g(reac) o(hes)g(the)g(end)h(of)e(the)i(\014le)g(b)q(efore)f(completing,)j(the)d (\014le)h(p)q(oin)o(ter)f(will)75 1904 y(b)q(e)g(incremen)o(ted)h(b)o (y)e(the)h(amoun)o(t)e(of)h(data)g Fk(r)n(e)n(queste)n(d)f Ft(rather)h(than)h(the)f(amoun)o(t)g(of)g(data)f(read.)75 1960 y(Therefore,)c(the)h(\014le)g(p)q(oin)o(ter)g(will)h(p)q(oin)o(t)f (b)q(ey)o(ond)g(the)f(curren)o(t)g(end)h(of)f(the)h(\014le,)g(and)f (the)h(write)f(will)75 2017 y(lea)o(v)o(e)f(a)g(hole)h(in)g(the)f (\014le.)21 b(Ho)o(w)o(ev)o(er,)14 b(suc)o(h)h(a)g(problem)h(only)g(o)q (ccurs)f(if)h(reads)f(and)g(writes)g(are)g(mixed)75 2073 y(without)g(c)o(hec)o(king)i(for)d(the)i(end)g(of)e(the)i(\014le.)21 b(Although)16 b(common)f(in)h(a)f(single)i(pro)q(cess)e(w)o(orkstation) 75 2130 y(en)o(vironmen)o(t,)g(w)o(e)g(b)q(eliev)o(e)i(this)f(is)g (uncommon)f(in)h(a)f(parallel)i(en)o(vironmen)o(t.)75 2252 y Fl(10.1.4)49 b(Synchronism)75 2337 y Ft(The)20 b(explicit)i(o)o(v)o(erlap)d(of)g(computation)h(with)g(I/O,)g(hop)q (efully)h(impro)o(ving)g(p)q(erformance,)f(is)g(sup-)75 2394 y(p)q(orted)13 b(through)f(the)h(use)g(of)g(non)o(blo)q(c)o(king)h (data)e(access)h(routines.)19 b(Both)13 b(blo)q(c)o(king)h(and)f(non)o (blo)q(c)o(king)75 2450 y(v)o(ersions)h(of)g(all)h(data)e(access)h (routines)g(are)g(supp)q(orted.)20 b(The)14 b(non)o(blo)q(c)o(king)i(v) o(ersions)e(of)f(the)i(routines)75 2507 y(are)g(named)g Fq(MPI)p 382 2507 14 2 v 16 w(IXXX)p Ft(,)g(where)h(the)f Fq(I)g Ft(stands)g(for)f(immediate.)166 2563 y(A)j Fk(blo)n(cking)e Ft(I/O)j(call)g(will)g(blo)q(c)o(k)g(un)o(til)g(the)f(I/O)g(request)g (is)h(completed.)26 b(A)17 b Fk(nonblo)n(cking)e Ft(I/O)75 2620 y(call)f(only)f(initiates)h(an)f(I/O)g(op)q(eration,)g(but)g(do)q (es)g(not)f(w)o(ait)h(for)f(it)h(to)f(complete.)20 b(A)12 b(separate)h Fk(r)n(e)n(quest)75 2676 y(c)n(omplete)18 b Ft(call)h(\()p Fq(MPI)p 453 2676 V 15 w(W)l(AIT)p Ft(,)f Fq(MPI)p 694 2676 V 15 w(TEST)p Ft(,)g(or)g(an)o(y)f(of)h(their)g(v)m (arian)o(ts\))f(is)i(needed)g(to)e(complete)i(the)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(7)75 49 y(I/O)17 b(request,)g(i.e.,)h(to)e(certify)h(that)f(data)h (has)g(b)q(een)h(read/written,)f(and)g(it)g(is)g(safe)g(for)f(the)h (user)h(to)75 106 y(reuse)d(the)g(bu\013er.)20 b(With)15 b(suitable)i(hardw)o(are,)d(the)h(transfer)f(of)g(data)h(out/in)g(the)g (user's)g(bu\013er)g(ma)o(y)75 162 y(pro)q(ceed)h(concurren)o(tly)g (with)f(computation.)166 219 y(Note)f(that)f(just)h(b)q(ecause)i(a)e (non)o(blo)q(c)o(king)h(\(or)f(blo)q(c)o(king\))h(data)f(access)g (routine)h(completes)g(do)q(es)75 275 y(not)g(mean)h(that)f(the)h(data) f(is)h(actually)h(written)f(to)f(\\p)q(ermanen)o(t")g(storage.)20 b(All)e(of)d(the)h(data)f(access)75 332 y(routines)22 b(ma)o(y)g(bu\013er)g(data)f(to)h(impro)o(v)o(e)g(p)q(erformance.)40 b(The)23 b(only)f(w)o(a)o(y)f(to)h(guaran)o(tee)f(data)g(is)75 388 y(actually)16 b(written)f(to)g(storage)e(is)j(b)o(y)f(using)h(the)f Fq(MPI)p 1010 388 14 2 v 16 w(FILE)p 1117 388 V 16 w(SYNC)h Ft(call.)21 b(Ho)o(w)o(ev)o(er,)13 b(one)j(need)g(not)e(b)q(e)75 444 y(concerned)i(with)e(the)h(con)o(v)o(erse)f(problem|once)i(a)e (read)h(op)q(eration)f(completes,)h(the)g(data)f(is)h(alw)o(a)o(ys)75 501 y(a)o(v)m(ailable)i(in)f(the)f(user's)g(bu\013er.)75 623 y Fl(10.1.5)49 b(Co)q(o)o(rdination)75 708 y Ft(Global)21 b(data)f(accesses)g(ha)o(v)o(e)g(signi\014can)o(t)h(p)q(oten)o(tial)g (for)f(automatic)g(optimization,)i(pro)o(vided)f(the)75 765 y(I/O)d(system)f(can)h(recognize)h(an)f(op)q(eration)f(as)h(a)f (global)h(access.)28 b(Collectiv)o(e)19 b(op)q(erations)f(are)f(used)75 821 y(for)e(this)i(purp)q(ose.)22 b Fq(MPI)15 b Ft(supp)q(orts)h(b)q (oth)g(indep)q(enden)o(t)j(and)d(collectiv)o(e)h(v)o(ersions)f(of)g (all)h(data)e(access)75 878 y(routines.)39 b(Ev)o(ery)21 b(indep)q(enden)o(t)i(data)e(access)h(routine)f Fq(MPI)p 1184 878 V 16 w(XXX)h Ft(has)g(a)f(collectiv)o(e)i(coun)o(terpart)75 934 y Fq(MPI)p 160 934 V 16 w(XXX)p 266 934 V 17 w(ALL)p Ft(,)17 b(where)p 527 934 V 35 w Fq(ALL)g Ft(means)h(that)f(\\all")i (pro)q(cesses)f(in)h(the)f(comm)o(unicator)f(group)h(whic)o(h)75 991 y(op)q(ened)e(the)g(\014le)g(m)o(ust)e(participate.)166 1047 y(Indep)q(enden)o(t)23 b(calls)f(do)f(not)f(imply)j(an)o(y)d(co)q (ordination)i(among)e(pro)q(cesses.)38 b(Collectiv)o(e)22 b(calls)75 1104 y(imply)13 b(that)e(all)h(pro)q(cesses)g(b)q(elonging)i (to)d(the)g(comm)o(unicator)h(asso)q(ciated)f(with)h(the)g(op)q(ened)h (\014le)g(m)o(ust)75 1160 y(participate.)20 b(Ho)o(w)o(ev)o(er,)12 b(no)h(sync)o(hronization)i(pattern)d(b)q(et)o(w)o(een)i(those)f(pro)q (cesses)g(is)h(enforced)g(\(with)75 1217 y(the)h(exception)i(of)d (collectiv)o(e)j(data)e(accesses)g(using)h(a)f(shared)g(\014le)i(p)q (oin)o(ter|see)f(Section)g(10.5.2\).)166 1273 y(An)h Fk(indep)n(endent)e Ft(I/O)h(request)h(is)g(a)f(request)g(whic)o(h)h (is)g(executed)g(individual)q(ly)j(b)o(y)c(an)o(y)g(of)g(the)75 1329 y(pro)q(cesses)21 b(within)i(a)e(comm)o(unicator)f(group.)38 b(A)21 b Fk(c)n(ol)r(le)n(ctive)e Ft(I/O)j(request)f(is)h(a)e(request)i (whic)o(h)f(is)75 1386 y(executed)15 b(b)o(y)e Fk(al)r(l)h Ft(pro)q(cesses)g(within)g(a)g(comm)o(unicator)f(group.)19 b(The)14 b(completion)h(of)e(an)h(indep)q(enden)o(t)75 1442 y(call)j(only)g(dep)q(ends)h(on)e(the)h(activit)o(y)f(of)g(the)h (calling)h(pro)q(cess.)23 b(On)17 b(the)g(other)f(hand,)g(a)g(pro)q (cess)h(can)75 1499 y(\(but)k(is)g(not)f(required)i(to\))e(return)h (from)f(a)g(collectiv)o(e)j(call)f(as)e(so)q(on)h(as)f(its)h (participation)h(in)g(the)75 1555 y(collectiv)o(e)e(op)q(eration)e(is)g (completed.)30 b(The)18 b(completion)h(of)f(the)g(call,)h(ho)o(w)o(ev)o (er,)f(do)q(es)g(not)f(indicate)75 1612 y(that)e(other)h(pro)q(cesses)h (ha)o(v)o(e)e(completed)i(or)f(ev)o(en)h(started)e(the)h(I/O)h(op)q (eration.)22 b(Th)o(us,)16 b(a)g(collectiv)o(e)75 1668 y(call)g(ma)o(y)l(,)f(or)f(ma)o(y)h(not,)f(ha)o(v)o(e)h(the)g(e\013ect) g(of)g(sync)o(hronizing)i(all)f(calling)h(pro)q(cesses.)166 1725 y(F)l(rom)c(a)g(seman)o(tic)i(viewp)q(oin)o(t,)f(the)g(only)h (di\013erence)g(b)q(et)o(w)o(een)f(collectiv)o(e)h(op)q(erations)f(and) g(their)75 1781 y(indep)q(enden)o(t)25 b(coun)o(terparts)d(is)h(p)q (oten)o(tial)g(sync)o(hronization.)43 b(F)l(rom)21 b(a)i(p)q (erformance)f(viewp)q(oin)o(t,)75 1838 y(ho)o(w)o(ev)o(er,)14 b(collectiv)o(e)j(op)q(erations)e(ha)o(v)o(e)g(the)g(p)q(oten)o(tial)h (to)e(b)q(e)i(m)o(uc)o(h)f(faster)f(than)h(their)h(indep)q(enden)o(t)75 1894 y(coun)o(terparts.)75 2016 y Fl(10.1.6)49 b(Consistency)16 b(Semantics)75 2102 y Ft(Consistency)21 b(seman)o(tics)f(de\014ne)h (the)f(outcome)g(of)g(m)o(ultiple)i(accesses)e(to)g(a)g(single)h (\014le,)h(either)f(b)o(y)75 2158 y(man)o(y)15 b(pro)q(cesses)g(or)g(b) o(y)g(a)g(single)i(pro)q(cess)e(whic)o(h)h(has)f(op)q(ened)h(the)g (\014le)g(more)f(than)g(once.)166 2214 y Fq(MPI)j Ft(guaran)o(tees)g (that)g(non-o)o(v)o(erlapping)i(\014le)g(accesses)f(within)h(a)f (single)h(application)g(are)f(se-)75 2271 y(quen)o(tially)k(consisten)o (t,)f(and)g(hence,)h(app)q(ear)e(to)g(the)g(application)i(as)e(atomic.) 37 b(Ho)o(w)o(ev)o(er,)21 b(unlik)o(e)75 2327 y(UNIX,)e(the)h(default)f (seman)o(tics)g(for)g(o)o(v)o(erlapping)h(accesses)f(do)q(es)g(not)g (guaran)o(tee)f(sequen)o(tial)j(con-)75 2384 y(sistency)l(.)28 b(This)19 b(default)f(mo)q(de)g(is)g(referred)g(to)f(as)g Fg(MPI)p 1066 2384 13 2 v 14 w(RECKLESS)g Ft(mo)q(de.)28 b(A)o(tomic)18 b(access)f(can)h(b)q(e)75 2440 y(guaran)o(teed)h(for)f (o)o(v)o(erlapping)i(accesses)f(b)o(y)g(enabling)i Fg(MPI)p 1140 2440 V 14 w(CA)o(UTIOUS)c Ft(mo)q(de)i(for)f(the)h(\014le)i(via)e (the)75 2497 y Fq(MPI)p 160 2497 14 2 v 16 w(FILE)p 267 2497 V 15 w(CONTROL)e Ft(routine.)166 2553 y(File)g(consistency)g (seman)o(tics)f(outside)g(of)g(a)f(single)j Fq(MPI)d Ft(application)j(are)d(implemen)o(tation)i(de-)75 2610 y(p)q(enden)o(t.)i(No)10 b(\014le)h(consistency)g(seman)o(tics)g(are)f (implied)i(b)o(y)e(either)h Fg(MPI)p 1322 2610 13 2 v 14 w(RECKLESS)f Ft(or)f Fg(MPI)p 1672 2610 V 15 w(CA)o(UTIOUS)75 2666 y Ft(b)q(et)o(w)o(een)17 b(m)o(ultiple)h Fq(MPI)e Ft(applications,)i(or)e(outside)h(of)g(the)f Fq(MPI)g Ft(en)o(vironmen)o(t.)24 b(In)18 b(other)e(w)o(ords,)f(if)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Ft(8)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(t)o(w)o(o)18 b(di\013eren)o(t)h Fq(MPI)f Ft(applications)j(op)q(en)f(the)f(same)g(\014le,)h(the)f(b)q(eha)o (vior)h(and)f(atomicit)o(y)g(of)g(the)g(\014le)75 106 y(accesses)k(are)f(unde\014ned.)43 b(The)23 b Fg(MPI)p 762 106 13 2 v 14 w(CA)o(UTIOUS)d Ft(mo)q(de)j(enforces)f(read/write)g (atomicit)o(y)h(in)g(the)75 162 y Fg(MPI)p 152 162 V 14 w(COMM)p 296 162 V 15 w(W)o(ORLD)15 b Ft(comm)o(unicator)g(group)g (only)l(.)75 284 y Fl(10.1.7)49 b(End)17 b(of)f(File)75 370 y Ft(Unlik)o(e)k(UNIX)e(\014les,)i(the)e(end)h(of)e(\014le)i(is)g (not)f(absolute)g(and)g(iden)o(tical)i(for)e(all)h(pro)q(cesses)f (accessing)75 426 y(the)d(\014le.)21 b(It)15 b(dep)q(ends)i(on)e(the)g (\014let)o(yp)q(e)h(used)g(to)f(access)g(the)g(\014le)h(and)g(is)f (de\014ned)i(for)d(a)h(giv)o(en)h(pro)q(cess)75 483 y(as)h(the)g(lo)q (cation)h(of)f(the)g(b)o(yte)g(follo)o(wing)h(the)f(last)g(elemen)o (tary)g(datat)o(yp)q(e)g(item)h(accessible)h(b)o(y)e(that)75 539 y(pro)q(cess)d(\(excluding)i(the)d(holes\).)20 b(It)14 b(ma)o(y)f(happ)q(en)i(that)e(data)g(is)h(lo)q(cated)h(b)q(ey)o(ond)f (the)g(end)g(of)g(\014le)h(for)75 596 y(a)g(giv)o(en)h(pro)q(cess.)k (This)15 b(data)g(is)h(accessible)h(only)f(b)o(y)f(other)f(pro)q (cesses.)166 652 y(When)d(a)f(\014le)i(gro)o(ws,)e(b)q(ecause)h(of)f (more)g(data)g(b)q(eing)i(written)f(to)f(the)g(\014le)i(or)e(the)h (\014le)g(b)q(eing)h(resized,)75 708 y(the)k(end)h(of)f(\014le)i(of)e (all)h(pro)q(cesses)g(accessing)g(the)f(\014le)i(ma)o(y)d(c)o(hange,)i (and)f(data)g(no)o(w)g(accessible,)i(but)75 765 y(not)d(y)o(et)g (written)g(b)o(y)g(an)o(y)g(of)g(these)g(pro)q(cesses,)g(will)i(b)q(e)f (read)f(as)g(zero)q(es.)75 887 y Fl(10.1.8)49 b(File)17 b(La)o(y)o(out)75 972 y Ft(The)k(basic)g(access)g(routines)g(only)g(sp) q(ecify)g(ho)o(w)f(the)h(data)f(should)h(b)q(e)g(laid)h(out)e(in)i(a)e (virtual)h(\014le)75 1029 y(structure)d(\(the)g(\014let)o(yp)q(e\),)i (not)d(ho)o(w)h(that)g(\014le)h(structure)f(is)h(to)f(b)q(e)h(stored)e (on)i(one)f(or)g(more)g(disks.)75 1085 y(This)g(w)o(as)f(a)o(v)o(oided) g(b)q(ecause)i(it)f(is)g(exp)q(ected)g(that)f(the)h(mapping)g(of)f (\014les)h(to)f(disks)h(will)h(b)q(e)f(system)75 1142 y(sp)q(eci\014c,)c(and)d(an)o(y)g(sp)q(eci\014c)i(con)o(trol)e(o)o(v)o (er)f(\014le)j(la)o(y)o(out)d(w)o(ould)i(therefore)f(restrict)g (program)f(p)q(ortabilit)o(y)l(.)75 1198 y(Ho)o(w)o(ev)o(er,)i(there)h (are)g(still)i(cases)e(where)g(some)g(information)g(will)i(b)q(e)f (necessary)f(in)h(order)f(to)g(optimize)75 1255 y(disk)i(la)o(y)o(out.) j(This)d(information)f(is)g(pro)o(vided)h(as)e Fk(hints)h Ft(sp)q(eci\014ed)i(when)e(a)f(\014le)i(is)g(created.)k(Hin)o(ts)14 b(are)75 1311 y(advisory)f(only)l(,)h(and)g(do)f(not)g(c)o(hange)h(the) f(seman)o(tics)h(of)f(an)o(y)g(routines.)19 b(Instead,)14 b(they)f(allo)o(w)h(sp)q(eci\014c)75 1368 y(implemen)o(tations)i(to)f (increase)h(I/O)g(p)q(erformance.)75 1511 y Fm(10.2)60 b(File)19 b(Manipulation)75 1614 y Fl(10.2.1)49 b(Op)q(ening)18 b(a)e(File)h(\(Collective\))75 1794 y Fq(MPI)p 160 1794 14 2 v 16 w(OPEN\(comm,)c(\014lename,)i(amo)q(de,)g(disp,)h(et)o(yp)q (e,)g(\014let)o(yp)q(e,)g(hints,)g(fh\))117 1871 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])14 b(Comm)n(unicator)d(that)j(op)q(ens)h(the) f(\014le)g(\(handle\))117 1947 y(IN)155 b Fq(\014lename)430 b Ff([SAME])14 b(Name)e(of)i(\014le)g(to)f(b)q(e)i(op)q(ened)g (\(string\))117 2022 y(IN)155 b Fq(amo)q(de)461 b Ff([SAME])14 b(File)f(access)j(mo)q(de)d(\(in)o(teger\))117 2097 y(IN)155 b Fq(disp)513 b Ff(Absolute)15 b(displacemen)o(t)e(\(nonnegativ)o(e)g (o\013set\))117 2172 y(IN)155 b Fq(et)o(yp)q(e)487 b Ff([SAME])14 b(Elemen)o(tary)f(datat)o(yp)q(e)h(\(handle\))117 2247 y(IN)155 b Fq(\014let)o(yp)q(e)452 b Ff(Filet)o(yp)q(e)14 b(\(handle\))117 2322 y(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(to)g(the)g(\014le)g(system)g(\(handle\))117 2397 y(OUT)108 b Fq(fh)550 b Ff(Returned)15 b(\014le)f(handle)g(\(handle\))75 2522 y Fp(int)23 b(MPI)p 245 2522 15 2 v 17 w(Open\(MPI)p 454 2522 V 16 w(Comm)h(comm,)f(char)g(*filename,)g(MPI)p 1187 2522 V 16 w(Mode)h(amode,)f(MPI)p 1562 2522 V 17 w(Offset)g(disp,)393 2578 y(MPI)p 468 2578 V 17 w(Datatype)g(etype,)g (MPI)p 939 2578 V 17 w(Datatype)f(filetype,)h(MPI)p 1481 2578 V 17 w(Hints)g(hints,)393 2635 y(MPI)p 468 2635 V 17 w(File)g(*fh\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1161 b Ft(9)75 49 y Fp(MPI)p 150 49 15 2 v 17 w(OPEN\(COMM,)22 b(FILENAME,)h(AMODE,)g(DISP,)g(ETYPE,)g(FILETYPE,)g(HINTS,)g(FH,)g (IERROR\))170 106 y(CHARACTER)g(FILENAME\(*\))170 162 y(INTEGER)g(COMM,)h(AMODE,)f(ETYPE,)g(FILETYPE)170 219 y(INTEGER)g(HINTS,)g(FH,)h(IERROR)170 275 y(INTEGER*8)f(DISP)166 362 y Fq(MPI)p 251 362 14 2 v 16 w(OPEN)c Ft(op)q(ens)h(the)e(\014le)i (iden)o(ti\014ed)h(b)o(y)e(the)f(\014le)i(name)f Fq(\014lename)g Ft(on)f(all)i(pro)q(cesses)f(in)h(the)75 418 y Fq(comm)13 b Ft(comm)o(unicator)h(group.)19 b Fq(MPI)p 732 418 V 16 w(OPEN)c Ft(is)g(a)g(collectiv)o(e)h(routine;)f(all)g(pro)q(cesses)g (in)h(the)e(comm)o(u-)75 474 y(nicator)j(group)g(m)o(ust)g(call)i Fq(MPI)p 648 474 V 15 w(OPEN)p Ft(,)f(with)g(iden)o(tical)h(v)m(alues)g (for)d Fq(comm)p Ft(,)g Fq(\014lename)p Ft(,)i Fq(amo)q(de)p Ft(,)f(and)75 531 y Fq(et)o(yp)q(e)p Ft(.)j(\(V)l(alues)12 b(for)f Fq(disp)p Ft(,)i Fq(\014let)o(yp)q(e)p Ft(,)g(and)f Fq(hints)h Ft(ma)o(y)d(v)m(ary)l(.\))19 b(A)11 b(pro)q(cess)h(can)f(op) q(en)h(a)g(\014le)g(indep)q(enden)o(tly)75 587 y(of)k(other)h(pro)q (cesses)g(b)o(y)f(using)i(the)f Fg(MPI)p 789 587 13 2 v 14 w(COMM)p 933 587 V 15 w(SELF)f Ft(comm)o(unicator.)24 b(The)17 b(\014le)h(handle)g(returned,)75 644 y Fq(fh)p Ft(,)d(can)g(b)q(e)h(subsequen)o(tly)h(used)f(to)e(access)h(the)h (\014le)g(un)o(til)g(the)g(\014le)g(is)g(closed)g(using)g Fq(MPI)p 1643 644 14 2 v 15 w(CLOSE)p Ft(.)166 700 y(The)f(follo)o (wing)h(access)g(mo)q(des,)f(sp)q(eci\014ed)i(via)e Fq(amo)q(de)p Ft(,)g(are)g(supp)q(orted:)143 790 y Fh(\017)23 b Fg(MPI)p 266 790 13 2 v 14 w(RDONL)m(Y)15 b Ft(-)g(reading)h(only)l(,)143 883 y Fh(\017)23 b Fg(MPI)p 266 883 V 14 w(RD)o(WR)15 b Ft(-)g(reading)h(and)f(writing,)143 975 y Fh(\017)23 b Fg(MPI)p 266 975 V 14 w(WRONL)m(Y)15 b Ft(-)g(writing)h(only)l(,)143 1068 y Fh(\017)23 b Fg(MPI)p 266 1068 V 14 w(CREA)m(TE)13 b Ft(-)j(creating)f(\014le,)143 1160 y Fh(\017)23 b Fg(MPI)p 266 1160 V 14 w(DELETE)p 435 1160 V 15 w(ON)p 510 1160 V 15 w(CLOSE)14 b Ft(-)i(deleting)g(on)g(close.)75 1250 y(These)c(can)g(b)q(e)g(com)o(bined)h(in)f(C)g(b)o(y)f(using)i(the)f (bit)o(wise)g(OR)h(op)q(erator,)e(and)h(in)g(F)o(OR)l(TRAN)h(b)o(y)e (adding)75 1307 y(them.)189 1408 y Fk(A)n(dvic)n(e)18 b(to)h(implementors.)58 b Ft(The)18 b(v)m(alues)i(of)d(these)i(constan) o(ts)e(m)o(ust)h(b)q(e)h(de\014ned)g(suc)o(h)g(that)189 1465 y(the)14 b(bit)o(wise)h(OR)h(and)e(the)h(sum)f(of)g(an)o(y)g (distinct)i(set)e(of)g(these)h(constan)o(ts)e(is)i(equiv)m(alen)o(t.)22 b(\()p Fk(End)189 1521 y(of)16 b(advic)n(e)g(to)h(implementors.)p Ft(\))189 1623 y Fk(A)n(dvic)n(e)k(to)i(users.)81 b Ft(Some)22 b(\014le)h(attributes)e(are)h(inheren)o(tly)i(implemen)o(tation)f(dep)q (enden)o(t.)189 1680 y(These)17 b(attributes)g(m)o(ust)g(b)q(e)h(set)f (using)h(facilities)h(outside)f(the)f(scop)q(e)h(of)f Fq(MPI)p Ft(.)f(F)l(or)g(example,)189 1736 y(UNIX)e(\014le)h(access)f (p)q(ermissions)h(can)f(b)q(e)h(sp)q(eci\014ed)h(via)e(the)g(UNIX)g (umask.)19 b(\()p Fk(End)c(of)g(advic)n(e)g(to)189 1793 y(users.)p Ft(\))189 1895 y Fk(A)n(dvic)n(e)j(to)h(users.)57 b Ft(Note)18 b(that)g(the)g(UNIX)h(app)q(end)g(mo)q(de)f(is)h(not)f (supp)q(orted.)29 b(This)19 b(mo)q(de)189 1951 y(can)f(b)q(e)h(em)o (ulated)f(b)o(y)g(requesting)h(the)f(curren)o(t)g(\014le)h(size)g (\(see)f(Section)h(10.2.3\))d(and)i(seeking)189 2007 y(to)f(the)h(end)h(of)f(\014le)h(b)q(efore)f(eac)o(h)h(write)f(op)q (eration.)29 b Fe(Discussion)o(:)22 b Ff(Actually)m(,)15 b(the)i(seman)o(tics)g(of)189 2147 y(UNIX)f(app)q(end)h(mo)q(de)e(are)i (at)f(the)h(\014lesystem)f(lev)o(el,)g(and)g(accoun)o(t)g(for)g(all)f (applications)g(running)h(on)189 2203 y(the)c(system)f(\(not)g(just)h (a)f(single)g(application\).)16 b(UNIX)c(app)q(end)g(mo)q(de)e(cannot)i (b)q(e)g(fully)e(em)o(ulated)g(using)189 2259 y(the)k(ab)q(o)o(v)o(e.)k (Ho)o(w)13 b(imp)q(ortan)o(t)g(is)g(a)h(true)h(UNIX)f(app)q(end)g(mo)q (de?)19 b Ft(\()p Fk(End)d(of)g(advic)n(e)g(to)h(users.)p Ft(\))166 2444 y(The)i Fq(disp)h Ft(displacemen)o(t)h(argumen)o(t)d(sp) q(eci\014es)i(the)f(p)q(osition)h(\(absolute)f(o\013set)f(in)i(b)o (ytes)f(from)75 2501 y(the)d(b)q(eginning)j(of)d(the)g(\014le\),)h (where)g(the)f(\014le)h(is)g(to)f(b)q(e)h(op)q(ened.)24 b(The)17 b(constan)o(t)e Fg(MPI)p 1586 2501 V 15 w(OFFSET)p 1756 2501 V 13 w(ZERO)75 2557 y Ft(is)h(de\014ned)g(to)f(b)q(e)h (o\013set)e(zero.)189 2647 y Fk(A)n(dvic)n(e)d(to)j(users.)37 b Fq(Disp)12 b Ft(can)g(b)q(e)g(used)g(to)f(skip)h(headers,)g(or)f (when)h(the)f(\014le)i(includes)h(a)d(sequence)189 2704 y(of)f(data)h(segmen)o(ts)g(that)f(are)h(to)g(b)q(e)g(accessed)h(in)g (di\013eren)o(t)g(patterns.)18 b(\()p Fk(End)12 b(of)g(advic)n(e)h(to)g (users.)p Ft(\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Ft(10)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Ft(The)18 b Fq(et)o(yp)q(e)i Ft(argumen)o(t)d(sp)q (eci\014es)j(the)e(elemen)o(tary)g(datat)o(yp)q(e)g(used)g(to)g (construct)g(the)g Fq(\014let)o(yp)q(e)p Ft(,)75 106 y(and)h(also)f(the)h Fq(buft)o(yp)q(e)h Ft(used)f(in)h(read/write)e(op) q(erations.)30 b(O\013sets)18 b(in)o(to)h(the)g(\014le)g(are)f (measured)h(in)75 162 y(units)d(of)f Fq(et)o(yp)q(e)p Ft(.)166 219 y(The)20 b Fq(\014let)o(yp)q(e)i Ft(argumen)o(t)e(describ) q(es)h(what)f(part)g(of)g(the)g(data)g(in)h(the)f(\014le)h(is)g(b)q (eing)h(accessed.)75 275 y(Conceptually)l(,)15 b(the)f(\014le)h (starting)e(from)g Fq(disp)i Ft(is)f(tiled)h(b)o(y)f(rep)q(eated)g (copies)h(of)e Fq(\014let)o(yp)q(e)p Ft(,)i(un)o(til)g(the)f(end.)75 332 y(If)k Fq(\014let)o(yp)q(e)g Ft(has)f(holes)h(in)h(it,)e(then)h (the)f(data)g(in)h(the)g(holes)g(is)f(inaccessible)j(b)o(y)e(the)f (calling)i(pro)q(cess.)75 388 y(Ho)o(w)o(ev)o(er,)d(the)h Fq(disp)p Ft(,)h Fq(et)o(yp)q(e)h Ft(and)e Fq(\014let)o(yp)q(e)h Ft(argumen)o(ts)e(can)i(b)q(e)f(c)o(hanged)h(later)f(to)f(access)h(a)g (di\013eren)o(t)75 444 y(part)e(of)f(the)i(\014le.)166 501 y(The)c Fq(hints)h Ft(argumen)o(t)e(is)i(used)f(to)f(pro)o(vide)i (information)f(regarding)g(\014le)h(access)f(patterns)f(and)h(\014le)75 557 y(system)j(sp)q(eci\014cs)j(to)d(direct)i(optimization)g(\(see)f (section)g(10.2.4\).)k(The)c(constan)o(t)f Fg(MPI)p 1623 557 13 2 v 14 w(HINTS)p 1758 557 V 16 w(NULL)75 614 y Ft(refers)g(to)g(the)g(n)o(ull)h(hin)o(t,)g(and)f(can)h(b)q(e)f(used)h (when)g(no)f(hin)o(ts)h(need)g(to)e(b)q(e)i(sp)q(eci\014ed.)166 670 y Fq(Filename)i Ft(can)h(b)q(e)h(pre\014xed)g(with)f("mpi-impl:/")i (to)d(iden)o(tify)i(the)f(implemen)o(tation)i(that)d(the)75 727 y(op)q(en)e(call)h(w)o(ould)f(lik)o(e)h(to)e(use)h(if)g(m)o (ultiple)i(implemen)o(tations)f(co-exist.)k(Examples)c(of)e(implemen)o (ta-)75 783 y(tions)g(are:)143 869 y Fh(\017)23 b Fq(ufs:/)15 b Ft(-)g(UNIX)h(\014le)g(system,)143 960 y Fh(\017)23 b Fq(mpirun:/)14 b Ft(-)h(serv)o(er)g(comm)o(unicating)h(to)f(clien)o (t)h(using)g(mpi,)143 1051 y Fh(\017)23 b Fq(so)q(ck)o(et:/)15 b Ft(-)h(serv)o(er)e(comm)o(unicating)i(to)f(clien)o(t)h(using)g(so)q (c)o(k)o(et,)143 1142 y Fh(\017)23 b Fq(piofs:/)15 b Ft(-)g(IBM)g(AIX)h(P)o(arallel)g(I/O)g(File)g(System)f([1,)f(2].)166 1228 y(Files)e(are)f(op)q(ened)h(b)o(y)f(default)h(using)g Fq(MPI)p 901 1228 14 2 v 16 w(RECKLESS)g Ft(\014le)g(consistency)g (seman)o(tics)f(\(see)g(section)75 1284 y(10.1.6\).)17 b(The)d(more)e(stringen)o(t)i Fg(MPI)p 711 1284 13 2 v 14 w(CA)o(UTIOUS)d Ft(consistency)j(seman)o(tics,)f(required)i(for)d (atomicit)o(y)h(of)75 1341 y(o)o(v)o(erlapping)j(accesses,)f(can)g(b)q (e)h(set)f(using)h Fq(MPI)p 924 1341 14 2 v 16 w(FILE)p 1031 1341 V 15 w(CONTROL)p Ft(.)75 1461 y Fl(10.2.2)49 b(Closing)18 b(a)f(\014le)f(\(Collective\))75 1641 y Fq(MPI)p 160 1641 V 16 w(CLOSE\(fh\))117 1718 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))75 1843 y Fp(int)23 b(MPI)p 245 1843 15 2 v 17 w(Close\(MPI)p 478 1843 V 16 w(File)g(fh\))75 1929 y(MPI)p 150 1929 V 17 w(CLOSE\(FH,)f(IERROR\))170 1986 y(INTEGER)h(FH,)h(IERROR)166 2072 y Fq(MPI)p 251 2072 14 2 v 16 w(CLOSE)d Ft(closes)g(the)f(\014le)i (asso)q(ciated)e(with)h Fq(fh)p Ft(.)36 b(After)20 b(closing,)i(the)f (con)o(ten)o(t)e(of)h(the)h(\014le)75 2129 y(handle)16 b Fq(fh)f Ft(is)h(destro)o(y)o(ed.)j(F)l(uture)c(use)g(of)f Fq(fh)i Ft(is)f(erroneous.)20 b(If)15 b(the)g(\014le)h(w)o(as)e(op)q (ened)i(with)f(the)g(access)75 2185 y(mo)q(de)22 b Fg(MPI)p 281 2185 13 2 v 14 w(DELETE)p 450 2185 V 15 w(ON)p 525 2185 V 15 w(CLOSE)p Ft(,)f(the)h(\014le)h(is)f(deleted.)41 b(Ho)o(w)o(ev)o(er,)22 b(if)g(there)g(are)f(other)h(pro)q(cesses)75 2242 y(curren)o(tly)e(accessing)h(the)f(\014le,)h(the)f(status)f(of)h (the)g(\014le)g(and)g(the)g(b)q(eha)o(vior)h(of)e(future)h(accesses)g (b)o(y)75 2298 y(these)15 b(pro)q(cesses)h(are)f(implemen)o(tation)h (dep)q(enden)o(t.)189 2395 y Fk(A)n(dvic)n(e)e(to)h(implementors.)40 b Ft(If)14 b(the)g(\014le)h(is)g(to)e(b)q(e)i(deleted)g(and)g(is)f(op)q (ened)h(b)o(y)f(other)g(pro)q(cesses,)189 2452 y(\014le)f(data)f(ma)o (y)g(still)i(b)q(e)g(accessible)g(b)o(y)f(these)g(pro)q(cesses)g(un)o (til)g(they)g(close)h(the)e(\014le)i(or)e(un)o(til)i(they)189 2508 y(exit.)20 b(The)14 b(delete)h(op)q(eration)f(should)h(b)q(e)g (e\013ectiv)o(e)f(only)h(on)f(the)g(last)g(close)h(of)e(that)h(\014le)h (within)189 2564 y(this)e(application.)21 b Fe(Discussion:)15 b Ff(This)d(implemen)o(tors)e(note)j(seems)f(to)g(con)o(tradict)h(the)g (last)f(sen)o(tence)189 2704 y(of)h(the)h(paragraph)g(ab)q(o)o(v)o(e.) 19 b Ft(\()p Fk(End)d(of)g(advic)n(e)h(to)f(implementors.)p Ft(\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fn(10.2.)29 b(FILE)15 b(MANIPULA)l(TION)1144 b Ft(11)75 49 y Fl(10.2.3)49 b(File)17 b(Control)g(\(Indep)q (endent/Collective\))75 211 y Fe(Discussion:)166 261 y Ff(It)e(is)g(not)g(p)q(ossible)g(to)g(supp)q(ort)h(F)o(OR)m(TRAN)e (with)g(the)i(curren)o(t)g(\014le)f(con)o(trol)g(in)o(terface.)22 b(The)16 b(follo)o(wing)75 311 y(alternativ)o(es)e(ha)o(v)o(e)g(b)q (een)h(suggested:)134 398 y(1.)22 b(Mak)o(e)13 b(MPI)p 383 398 13 2 v 16 w(Fcn)o(tl)h(w)o(ork)f(with)h(an)g(opaque)f(ob)r (ject)i(\(a)f(la)f(MPI)p 1207 398 V 15 w(Hin)o(ts\),)h(and)f(pro)o (vide)h(access)i(functions)189 448 y(for)d(eac)o(h)i(elemen)o(t.)134 522 y(2.)22 b(P)o(ass)14 b(all)f(thirteen)i(items)e(as)h(parameters,)f (no)h(structure.)134 597 y(3.)22 b(Pro)o(vide)13 b(MPI)p 424 597 V 15 w(Fcn)o(tl,)g(MPI)p 640 597 V 15 w(Fcn)o(tl)p 750 597 V 15 w(2,)g(MPI)p 892 597 V 15 w(Fcn)o(tl)p 1002 597 V 15 w(3,)g(etc.,)g(where)i(MPI)p 1353 597 V 15 w(Fcn)o(tl)e(tak)o (es)h(only)e(one)i(param-)189 647 y(eter,)g(MPI)p 366 647 V 16 w(Fcn)o(tl)p 477 647 V 15 w(2)f(tak)o(es)i(2)e(parameters,)h (etc.)134 722 y(4.)22 b(Group)13 b(the)i(items)e(in)g(some)g (meaningful)e(w)o(a)o(y)m(,)i(and)g(pro)o(vide)h(a)g(routine)g(for)f (eac)o(h)i(group)e(of)h(items.)75 948 y Fp(The)23 b(last)h(alternative) e(has)i(further)e(been)i(expanded)f(on:)75 1061 y(Grouping)g(the)g (items)g(and)h(providing)e(meaningful)h(routines)g(has)g(two)h (advantages:)75 1117 y(individual)e(pieces)i(of)f(this)g(standard)g (are)h(more)f(modular)75 1174 y(\(e.g.)g(file)g(pointer)g(manipulation) g(is)g(all)h(contained)e(in)i(the)f(file)h(pointer)f(section\),)75 1230 y(and)g(I/O)h(code)f(will)h(be)f(more)g(self-documenting)75 1287 y(\(e.g.)g(a)h(re-open)f(is)g(expressed)g(as)h(MPI_Reopen\).)75 1400 y(As)g(a)f(first)g(cut,)h(I)g(propose)e(we)i(group)f(the)h (MPI_Fcntl)e(options)h(into)h(the)75 1456 y(following)f(procedures:)75 1569 y(MPI_FCNTL_COMM)75 1625 y(MPI_FCNTL_FILENAME)75 1682 y(MPI_FCNTL_AMODE)75 1738 y(MPI_FCNTL_MOFFSET)75 1795 y(MPI_FCNTL_DISP)75 1851 y(MPI_FCNTL_ETYPE)75 1908 y(MPI_FCNTL_FILETYPE)75 1964 y(MPI_FCNTL_HINTS)75 2021 y(MPI_FCNTL_HINTS_TAKEN)75 2134 y(MPI_Open\(comm,)f(filename,)h(amode,) g(disp,)g(etype,)123 2190 y(filetype,)f(moffset,)h(hints,)g(*fh\);)75 2246 y(MPI_File_info\(*comm,)e(*filename,)i(*amode,)g(*disp,)g(*etype,) 242 2303 y(*filetype,)g(*moffset,)f(*hints_taken\);)75 2359 y(MPI_Reopen\(fh,)g(disp,)h(etype,)g(filetype,)g(hints,)g (*new_fh\);)75 2472 y(MPI_FCNTL_ATOMICITY)75 2585 y(MPI_Atomicity\(fh,) f(atomicity,)g(*old_atomicity\);)75 2698 y(MPI_FCNTL_FILE_SIZE)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Ft(12)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)75 106 y Fp(MPI_Resize\(fh,)22 b(MPI_Offset)g(disp\);)75 162 y(MPI_File_size\(fh,)g(MPI_Offset)g(*size_in_bytes\);)75 275 y(MPI_FCNTL_IND_FP)75 332 y(MPI_FCNTL_ABS_OFFSET)75 444 y(MPI_Seek\(fh,)g(offset,)h(whence\);)75 501 y (MPI_Next_position\(fh,)e(*offset,)i(*byte_offset\);)75 614 y(MPI_FCNTL_SHARED_FP)75 727 y(MPI_Seek_shared\(fh,)e(offset,)i (whence\);)75 783 y(MPI_Shared_position\(fh,)e(*offset,)h (*byte_offset\);)75 896 y(Note)h(that)h(we)f(only)h(need)f(to)g(add)h (6)g(routines)e(as)i(we)g(already)75 953 y(have)f(MPI_Open,)g (MPI_Resize,)f(MPI_Seek,)h(and)g(MPI_Seek_shared.)75 1153 y Fq(MPI)p 160 1153 14 2 v 16 w(FILE)p 267 1153 V 15 w(CONTROL\(fh,)16 b(cmd,)f(fcntl\))117 1230 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\)) 117 1306 y(IN)155 b Fq(cmd)507 b Ff([SAME])14 b(Comma)o(nd)117 1381 y(INOUT)62 b Fq(fcntl)503 b Ff(Argumen)o(ts)14 b(or)g(return)h(v)n (alues)e(to)h(the)g(command)75 1505 y Fp(int)23 b(MPI)p 245 1505 15 2 v 17 w(File)p 358 1505 V 17 w(control\(MPI)p 639 1505 V 16 w(File)g(fh,)g(unsigned)g(cmd,)g(MPI)p 1275 1505 V 17 w(Fcntl)g(*fcntl\))166 1592 y Ft(\(F)o(OR)l(TRAN)16 b(binding)h(to)d(b)q(e)i(pro)o(vided)g(in)g(v)o(ersion)g(0.6)e(of)h (the)g(draft.\))166 1648 y Fq(MPI)p 251 1648 14 2 v 16 w(FILE)p 358 1648 V 15 w(CONTROL)g Ft(gets)e(or)g(sets)g(\014le)i (information)f(ab)q(out)f(the)h(\014le)h(asso)q(ciated)e(with)h(the)g (\014le)75 1705 y(handle)j Fq(fh)p Ft(.)22 b(Multiple)c(commands)d(can) h(b)q(e)g(issued)h(in)g(one)f(call)h(using)f(the)g(bit)o(wise)h(OR)f (op)q(erator.)21 b(If)75 1761 y Fg(MPI)p 152 1761 13 2 v 14 w(F)o(CNTL)p 295 1761 V 15 w(SET)e Ft(is)h(set,)f(the)g (\014elds)h(indicated)h(b)o(y)e(the)g(bits)g(set)g(in)h Fq(cmd)e Ft(will)j(b)q(e)f(used)f(to)f(up)q(date)75 1817 y(the)i(\014le)h(information.)34 b(If)20 b Fg(MPI)p 641 1817 V 14 w(F)o(CNTL)p 784 1817 V 15 w(SET)g Ft(is)g(not)g(set,)g(the)g (\014le)h(information)f(requested)g(will)h(b)q(e)75 1874 y(returned.)f Fg(MPI)p 354 1874 V 15 w(Fcntl)15 b Ft(is)h(de\014ned)g (as)f(follo)o(ws:)170 1968 y Fp(typedef)23 b(struct)g(MPI_Fcntl)g({)266 2024 y(MPI_Comm)261 b(comm;)266 2081 y(char)333 b(*filename;)266 2137 y(MPI_Mode)237 b(amode;)266 2194 y(MPI_Offset)189 b(disp;)266 2250 y(MPI_Datatype)165 b(etype;)266 2306 y(MPI_Datatype)g(filetype;)266 2363 y(MPI_Hints)190 b(*hints;)266 2419 y(MPI_Hints)g(*hints_taken;)266 2476 y(MPI_Atomicity)117 b(atomicity;)266 2532 y(MPI_Offset)189 b(ind_fp;)266 2589 y(MPI_Offset)g(shared_fp;)266 2645 y(MPI_Offset)g(file_size;)266 2702 y(MPI_Offset)g(abs_offset;)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(13)170 49 y Fp(})24 b(MPI_Fcntl;)166 143 y Ft(The)15 b(commands)g(a)o(v)m(ailable)i(are:)143 237 y Fh(\017)23 b Fg(MPI)p 266 237 13 2 v 14 w(F)o(CNTL)p 409 237 V 15 w(SET)p Ft(:)15 b(Set)g(\014le)h(information.)143 331 y Fh(\017)23 b Fg(MPI)p 266 331 V 14 w(F)o(CNTL)p 409 331 V 15 w(COMM)p Ft(:)15 b(Get)f(the)i(comm)o(unicator)e(asso)q (ciated)i(with)f(the)h(\014le.)143 425 y Fh(\017)23 b Fg(MPI)p 266 425 V 14 w(F)o(CNTL)p 409 425 V 15 w(FILENAME)p Ft(:)14 b(Get)h(the)g(\014lename.)143 518 y Fh(\017)23 b Fg(MPI)p 266 518 V 14 w(F)o(CNTL)p 409 518 V 15 w(AMODE)p Ft(:)14 b(Get)h(or)g(set)f(the)i(\014le)g(access)f(mo)q(de)h(asso)q (ciated)f(with)h(the)f(\014le.)143 612 y Fh(\017)23 b Fg(MPI)p 266 612 V 14 w(F)o(CNTL)p 409 612 V 15 w(DISP)p Ft(:)14 b(Get)h(or)f(set)h(the)h(displacemen)o(t.)143 706 y Fh(\017)23 b Fg(MPI)p 266 706 V 14 w(F)o(CNTL)p 409 706 V 15 w(ETYPE)p Ft(:)14 b(Get)h(or)f(set)h(the)g(elemen)o(tary)h (datat)o(yp)q(e.)143 800 y Fh(\017)23 b Fg(MPI)p 266 800 V 14 w(F)o(CNTL)p 409 800 V 15 w(FILETYPE)p Ft(:)14 b(Get)g(or)h(set)g(the)g(\014let)o(yp)q(e.)143 894 y Fh(\017)23 b Fg(MPI)p 266 894 V 14 w(F)o(CNTL)p 409 894 V 15 w(HINTS)p Ft(:)15 b(Get)f(or)g(set)g(the)h(desired)g(v)m(alues)h (for)e(the)g(hin)o(ts)h(asso)q(ciated)g(with)g(the)f(\014le.)143 987 y Fh(\017)23 b Fg(MPI)p 266 987 V 14 w(F)o(CNTL)p 409 987 V 15 w(HINTS)p 545 987 V 16 w(T)m(AKEN)p Ft(:)14 b(Get)h(the)g(v)m(alues)h(for)f(the)g(hin)o(ts)h(actually)g(used.)143 1081 y Fh(\017)23 b Fg(MPI)p 266 1081 V 14 w(F)o(CNTL)p 409 1081 V 15 w(A)m(TOMICITY)p Ft(:)13 b(Get)i(or)g(set)g(the)h(curren) o(t)f(read/write)h(atomic)f(seman)o(tics)g(enforced)189 1138 y(mo)q(de.)143 1231 y Fh(\017)23 b Fg(MPI)p 266 1231 V 14 w(F)o(CNTL)p 409 1231 V 15 w(IND)p 495 1231 V 15 w(FP)p Ft(:)11 b(Get)g(the)h(curren)o(t)g(o\013set)g(of)f(the)h (individual)k(\014le)d(p)q(oin)o(ter)f(asso)q(ciated)h(with)189 1288 y(the)k(\014le)g(\(n)o(um)o(b)q(er)g(of)g(elemen)o(tary)g(datat)o (yp)q(e)f(items)h(within)h(the)f(\014le)h(after)e(the)h(displacemen)o (t)189 1344 y(p)q(osition\).)143 1438 y Fh(\017)23 b Fg(MPI)p 266 1438 V 14 w(F)o(CNTL)p 409 1438 V 15 w(SHARED)p 586 1438 V 15 w(FP)p Ft(:)18 b(Get)i(the)g(curren)o(t)g(o\013set)f(of)g (the)h(shared)g(\014le)h(p)q(oin)o(ter)g(asso)q(ciated)189 1495 y(with)15 b(the)h(\014le)g(\(n)o(um)o(b)q(er)f(of)g(elemen)o(tary) h(datat)o(yp)q(e)e(items)i(within)g(the)g(\014le)g(after)f(the)g (displace-)189 1551 y(men)o(t)g(p)q(osition\).)143 1645 y Fh(\017)23 b Fg(MPI)p 266 1645 V 14 w(F)o(CNTL)p 409 1645 V 15 w(FILE)p 507 1645 V 14 w(SIZE)p Ft(:)15 b(Get)g(the)g(size)h (of)f(the)g(\014le)h(asso)q(ciated)g(with)f(fh.)143 1739 y Fh(\017)23 b Fg(MPI)p 266 1739 V 14 w(F)o(CNTL)p 409 1739 V 15 w(ABS)p 503 1739 V 14 w(OFFSET)p Ft(:)14 b(Get)h(the)g (absolute)h(b)o(yte)f(o\013set)f(of)h(the)g(individual)j(\014le)f(p)q (oin)o(ter.)75 1833 y(Not)11 b(all)i(\014elds)g(in)g Fq(fcntl)g Ft(are)e(c)o(hangeable.)20 b(When)12 b Fq(et)o(yp)q(e)p Ft(,)i Fq(\014let)o(yp)q(e)p Ft(,)f(or)f Fq(disp)h Ft(is)f(c)o(hanged,) h(all)g(\014le)g(p)q(oin)o(ters)75 1889 y(asso)q(ciated)i(with)h Fq(fh)f Ft(are)g(reset)g(to)g Fg(MPI)p 765 1889 V 14 w(OFFSET)p 934 1889 V 14 w(ZERO)p Ft(.)166 1945 y(All)d(calls)f(that)f (get)g(\014le)i(information)e(are)h(indep)q(enden)o(t)h(calls.)20 b(All)12 b(calls)f(that)f(set)g(\014le)i(information)75 2002 y(are)j(collectiv)o(e)j(calls.)k(When)15 b(setting)h Fq(et)o(yp)q(e)p Ft(,)g Fq(amo)q(de)p Ft(,)f(and)h Fq(atomicit)o(y)p Ft(,)e(it)i(is)g(erroneous)g(for)f(pro)q(cesses)75 2058 y(to)g(pass)g(in)h(di\013eren)o(t)f(v)m(alues.)75 2180 y Fl(10.2.4)49 b(File)17 b(Hints)75 2266 y Ft(Hin)o(ts)k(allo)o(w)h(a)e (user)i(to)e(pro)o(vide)i(information)f(regarding)g(\014le)h(access)f (patterns)g(and)g(\014le)h(system)75 2322 y(sp)q(eci\014cs)i(to)e (direct)h(optimization.)42 b(Pro)o(viding)23 b(hin)o(ts)g(ma)o(y)e (enable)j(an)e(implemen)o(tation)i(to)d(de-)75 2379 y(liv)o(er)e (increased)g(I/O)f(p)q(erformance)g(or)f(minimize)j(the)e(use)g(of)f (system)g(resources.)28 b(Ho)o(w)o(ev)o(er,)17 b(hin)o(ts)75 2435 y(do)22 b(not)f(c)o(hange)h(the)f(seman)o(tics)h(of)f(an)o(y)h(of) f(the)h(in)o(terfaces.)39 b(In)23 b(other)e(w)o(ords,)h(an)g(implemen)o (ta-)75 2492 y(tion)d(is)g(free)f(to)g(ignore)g(all)i(hin)o(ts.)30 b(Hin)o(ts)18 b(are)g(sp)q(eci\014ed)j(on)d(a)g(p)q(er)h(\014le)h (basis)e(at)g Fq(MPI)p 1631 2492 14 2 v 16 w(OPEN)h Ft(\(and)75 2548 y Fq(MPI)p 160 2548 V 16 w(FILE)p 267 2548 V 15 w(CONTROL)p Ft(\))d(time.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Ft(14)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)189 49 y Fk(A)n(dvic)n(e)20 b(to)h(implementors.)71 b Ft(It)21 b(ma)o(y)e(happ)q(en)j(that)e(a)g(program)f(is)i(co)q(ded)g (with)g(hin)o(ts)g(for)189 106 y(one)15 b(system,)h(and)f(later)h (executes)g(on)g(another)f(system)h(that)f(do)q(es)h(not)f(supp)q(ort)h (these)g(hin)o(ts.)189 162 y(In)k(general,)h(unsupp)q(orted)g(hin)o(ts) f(should)h(simply)g(b)q(e)g(ignored.)34 b(In)20 b(particular,)h(\014le) g(con)o(trol)189 219 y(op)q(erations,)16 b(setting)g(or)g(inquiring)i (v)m(alues)g(to)e(unsupp)q(orted)h(hin)o(ts,)g(should)g(b)q(e)g (handled)h(as)d(if)189 275 y(these)h(hin)o(ts)g(w)o(ere)g(not)g(sp)q (eci\014ed.)25 b(Needless)17 b(to)f(sa)o(y)l(,)f(no)h(hin)o(t)h(can)f (b)q(e)h(mandatory)l(.)k(Ho)o(w)o(ev)o(er,)189 332 y(for)15 b(eac)o(h)h(hin)o(t)h(used)f(b)o(y)g(a)g(sp)q(eci\014c)i(implemen)o (tation,)f(a)f(default)h(v)m(alue)g(m)o(ust)f(b)q(e)g(pro)o(vided)h(in) 189 388 y(order)e(to)g(b)q(e)h(used)g(when)g(the)f(user)h(do)q(es)g (not)f(sp)q(ecify)i(a)e(v)m(alue)i(for)e(this)g(hin)o(t.)22 b(\()p Fk(End)15 b(of)i(advic)n(e)189 444 y(to)f(implementors.)p Ft(\))166 551 y(In)g(C,)e(hin)o(t)i(v)m(alues)g(are)f(de\014ned)i(as:) 170 657 y Fp(typedef)23 b(union)h(MPI_Hint_value)d({)266 713 y(MPI_Offset)h(hint_value;)266 770 y(void)h(*hint_ptr;)170 826 y(})h(MPI_Hint_value;)75 948 y Fl(10.2.5)49 b(Hints)17 b(Creation)f(and)h(Deallo)q(cation)75 1128 y Fq(MPI)p 160 1128 14 2 v 16 w(HINTS)p 309 1128 V 16 w(CREA)l(TE\(count,)g(hint)p 720 1128 V 18 w(names,)e(hint)p 958 1128 V 17 w(values,)h(hints\))117 1206 y Ff(IN)155 b Fq(count)482 b Ff(Num)o(b)q(er)14 b(of)f(hin)o(ts)h(supplied)g(\(in)o(teger\))117 1281 y(IN)155 b Fq(hint)p 394 1281 V 18 w(names)377 b Ff(Names)13 b(of)h(hin)o(ts)f(\(arra)o(y)h(of)g(in)o(teger\))117 1356 y(IN)155 b Fq(hint)p 394 1356 V 18 w(values)382 b Ff(V)m(alues)14 b(of)f(hin)o(ts)h(\(arra)o(y)g(of)f(in)o(teger\))117 1431 y(OUT)108 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))75 1555 y Fp(int)23 b(MPI)p 245 1555 15 2 v 17 w(Hints)p 382 1555 V 17 w(create\(int)f(count,)h(int)h(*hint)p 1044 1555 V 16 w(names,)f(MPI)p 1299 1555 V 17 w(Hint)p 1412 1555 V 17 w(value)g(*hint)p 1692 1555 V 16 w(values,)393 1612 y(MPI)p 468 1612 V 17 w(Hints)g(*hints\))75 1698 y(MPI)p 150 1698 V 17 w(HINTS)p 287 1698 V 16 w(CREATE\(COUNT,)f(HINT)p 733 1698 V 17 w(NAMES,)h(HINT)p 1013 1698 V 17 w(VALUES,)g(HINTS,)g (IERROR\))170 1755 y(INTEGER)g(COUNT,)g(HINT)p 627 1755 V 17 w(NAMES\(*\))170 1811 y(INTEGER*8)g(HINT)p 508 1811 V 17 w(VALUES\(*\))170 1868 y(INTEGER)g(HINTS,)g(IERROR)166 1954 y Fq(MPI)p 251 1954 14 2 v 16 w(HINTS)p 400 1954 V 16 w(CREA)l(TE)c Ft(returns)f(an)g(opaque)g(ob)s(ject,)g(it)h(is)f (the)g(resp)q(onsibilit)o(y)j(of)c(the)h(user)h(to)75 2011 y(delete)d(the)g(ob)s(ject)e(b)o(y)h(calling)i Fq(MPI)p 714 2011 V 16 w(HINTS)p 863 2011 V 17 w(FREE)e Ft(when)h(done.)75 2162 y Fq(MPI)p 160 2162 V 16 w(HINTS)p 309 2162 V 16 w(FREE\(hints\))117 2239 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))75 2363 y Fp(int)23 b(MPI)p 245 2363 15 2 v 17 w(Hints)p 382 2363 V 17 w(free\(MPI)p 591 2363 V 16 w(Hints)g(*hints\))75 2450 y(MPI)p 150 2450 V 17 w(HINTS)p 287 2450 V 16 w(FREE\(HINTS,)g(IERROR\))170 2506 y(INTEGER)g(HINTS,)g(IERROR)166 2593 y Fq(MPI)p 251 2593 14 2 v 16 w(HINTS)p 400 2593 V 16 w(FREE)11 b Ft(marks)e(the)i(hin)o(ts)f(ob)s(ject)g(for)f(deallo)q(cation)j(and)e (sets)g Fq(hints)i Ft(to)d Fg(MPI)p 1686 2593 13 2 v 15 w(HINTS)p 1822 2593 V 15 w(NULL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(15)166 49 y(Hin)o(t)11 b(names)f(smaller)h(or)f(equal)h(to)e Fg(MPI)p 844 49 13 2 v 15 w(HINTS)p 980 49 V 15 w(RESERVED)g Ft(when)i(implemen)o(ted)h(carry)e(the)g(same)75 106 y(meaning)h(across)e(all)i(implemen)o(tations)h(and)e(will)i(b)q(e)f (de\014ned)g(in)g(future)f(sp)q(eci\014cations.)20 b Fg(MPI)p 1697 106 V 15 w(HINTS)p 1833 106 V 15 w(RESERVED)75 162 y Ft(is)c(de\014ned)g(to)f(b)q(e)h(65535.)166 219 y(Hin)o(t)f(names)h(larger)f(than)g Fg(MPI)p 721 219 V 14 w(HINTS)p 856 219 V 16 w(RESERVED)e Ft(are)i(implemen)o(tation)i (sp)q(eci\014c)g(names.)166 275 y(The)g(constan)o(t)f Fg(MPI)p 523 275 V 15 w(HINTS)p 659 275 V 15 w(NULL)i Ft(refers)f(to)f(the)i(n)o(ull)g(hin)o(t,)g(and)f(can)g(b)q(e)h(used)f (in)h Fq(MPI)p 1740 275 14 2 v 16 w(OPEN)75 332 y Ft(when)e(no)f(hin)o (ts)g(need)i(to)d(b)q(e)i(sp)q(eci\014ed.)75 453 y Fl(10.2.6)49 b(Hints)17 b(Accesso)o(r)d(F)o(unctions)75 539 y Fq(MPI)p 160 539 V 16 w(HINTS)p 309 539 V 16 w(COUNT)i Ft(returns)f(the)h(n)o (um)o(b)q(er)f(of)g(hin)o(ts)h(in)g(the)f(opaque)g(ob)s(ject)g(hin)o (ts.)75 690 y Fq(MPI)p 160 690 V 16 w(HINTS)p 309 690 V 16 w(COUNT\(hints,)i(count\))117 767 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 842 y(OUT)108 b Fq(count)482 b Ff(Num)o(b)q(er)14 b(of)f(hin)o(ts)h(\(in)o(teger\))75 967 y Fp(int)23 b(MPI)p 245 967 15 2 v 17 w(Hints)p 382 967 V 17 w(count\(MPI)p 615 967 V 16 w(Hints)g(hints,)g(int)g(*count\)) 75 1053 y(MPI)p 150 1053 V 17 w(HINTS)p 287 1053 V 16 w(COUNT\(HINTS,)f(COUNT,)i(IERROR\))170 1110 y(INTEGER)f(HINTS,)g (COUNT,)g(IERROR)166 1196 y Fq(MPI)p 251 1196 14 2 v 16 w(HINTS)p 400 1196 V 16 w(GET)p 504 1196 V 17 w(ELEMENT)15 b Ft(returns)g(the)h(n-th)f(elemen)o(t)h(in)g(the)f(opaque)h(ob)s(ject) e(hin)o(ts.)189 1302 y Fk(A)n(dvic)n(e)h(to)i(users.)43 b Ft(The)15 b(order)h(of)f(the)g(elemen)o(ts)i(in)f(hin)o(ts)g(ma)o(y)f (or)g(ma)o(y)g(not)g(b)q(e)h(the)g(same)f(as)189 1359 y(the)g(order)g(passed)g(in)h Fq(MPI)p 668 1359 V 16 w(HINTS)p 817 1359 V 17 w(CREA)l(TE)p Ft(.)g(\()p Fk(End)f(of)i(advic)n (e)f(to)g(users.)p Ft(\))75 1560 y Fq(MPI)p 160 1560 V 16 w(HINTS)p 309 1560 V 16 w(GET)p 413 1560 V 17 w(ELEMENT\(hints,)g (n,)f(hint)p 900 1560 V 18 w(name,)f(hint)p 1120 1560 V 18 w(value\))117 1637 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 1712 y(IN)155 b Fq(n)564 b Ff(Numeric)14 b(selector)h(sp)q(ecifying)f(whic)o(h)f(hin)o(t)h(\(in) o(teger\))117 1787 y(OUT)108 b Fq(hint)p 394 1787 V 18 w(name)394 b Ff(Name)13 b(of)g(hin)o(t)h(\(in)o(teger\))117 1862 y(OUT)108 b Fq(hint)p 394 1862 V 18 w(value)399 b Ff(V)m(alue)14 b(of)f(hin)o(t)g(\(in)o(teger\))75 1987 y Fp(int)23 b(MPI)p 245 1987 15 2 v 17 w(Hints)p 382 1987 V 17 w(get)p 471 1987 V 16 w(element\(MPI)p 751 1987 V 16 w(Hints)g(hints,)g(int)h(n,)f(int)h(*hint)p 1460 1987 V 16 w(name,)393 2043 y(MPI)p 468 2043 V 17 w(Hint)p 581 2043 V 17 w(value)f(*hint)p 861 2043 V 16 w(value\))75 2130 y(MPI)p 150 2130 V 17 w(HINTS)p 287 2130 V 16 w(GET)p 375 2130 V 17 w(ELEMENT\(HINTS,)f(N,)i(HINT)p 918 2130 V 16 w(NAME,)f(HINT)p 1173 2130 V 17 w(VALUE,)g(IERROR\))170 2186 y(INTEGER)g(HINTS,)g(N,)h(HINT)p 699 2186 V 17 w(NAME)170 2243 y(INTEGER*8)f(HINT)p 508 2243 V 17 w(VALUE)170 2299 y(INTEGER)g(IERROR)166 2385 y Fq(MPI)p 251 2385 14 2 v 16 w(HINTS)p 400 2385 V 16 w(GET)p 504 2385 V 17 w(V)l(ALUE)13 b Ft(returns)f(the)g Fq(hint)p 975 2385 V 18 w(value)g Ft(of)g Fq(hint)p 1224 2385 V 18 w(name)g Ft(in)h(the)f(opaque)g(ob)s (ject)g(hin)o(ts.)75 2442 y(If)j Fq(hint)p 196 2442 V 18 w(name)g Ft(is)h(not)e(de\014ned)j(in)f Fq(hints)p Ft(,)g(an)g(error)e(will)j(b)q(e)f(returned,)f(and)g Fq(hint)p 1467 2442 V 18 w(value)g Ft(is)h(unde\014ned.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Ft(16)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(HINTS)p 309 49 V 16 w(GET)p 413 49 V 17 w(V)l(ALUE\(hints,)17 b(hint)p 780 49 V 18 w(name,)d(hint)p 1000 49 V 18 w(value\))117 126 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 200 y(IN)155 b Fq(hint)p 394 200 V 18 w(name)394 b Ff(Name)13 b(of)g(hin)o(t)h(\(in)o(teger\))117 274 y(OUT)108 b Fq(hint)p 394 274 V 18 w(value)399 b Ff(V)m(alue)14 b(of)f(hin)o(t)g(\(in)o (teger\))75 398 y Fp(int)23 b(MPI)p 245 398 15 2 v 17 w(Hints)p 382 398 V 17 w(get)p 471 398 V 16 w(value\(MPI)p 703 398 V 16 w(Hints)h(hints,)f(int)g(hint)p 1221 398 V 17 w(name,)393 455 y(MPI)p 468 455 V 17 w(Hint)p 581 455 V 17 w(value)g(*hint)p 861 455 V 16 w(value\))75 541 y(MPI)p 150 541 V 17 w(HINTS)p 287 541 V 16 w(GET)p 375 541 V 17 w(VALUE\(HINTS,)f(HINT)p 798 541 V 17 w(NAME,)h(HINT)p 1054 541 V 17 w(VALUE,)g(IERROR\))170 598 y(INTEGER)g(HINTS,)g(HINT)p 627 598 V 17 w(NAME)170 654 y(INTEGER*8)g(HINT)p 508 654 V 17 w(VALUE)170 711 y(INTEGER)g(IERROR)166 797 y Ft(Some)16 b(p)q(oten)o(tially)h(useful)f(hin)o(ts)g(are)g(outlined)h (b)q(elo)o(w.)22 b(Sp)q(eci\014c)c(implemen)o(tations)f(are)e(free)h (to)75 853 y(de\014ne)h(these)g(hin)o(ts)f(as)g(w)o(ell)h(as)f (additional)h(hin)o(ts.)23 b(Ho)o(w)o(ev)o(er,)15 b(in)i(order)f(to)g (main)o(tain)g(compatibilit)o(y)75 910 y(b)q(et)o(w)o(een)g(implemen)o (tations,)g(additional)h(hin)o(ts)e(should)i(not)e(reuse)g(the)h(same)f (hin)o(t)g(names)h(for)e(other)75 966 y(purp)q(oses)i(than)f(these)g (listed)i(here.)166 1023 y(These)c(hin)o(ts)h(are)f(mainly)h(concerned) g(with)f(la)o(y)o(out)g(of)f(data)h(on)g(parallel)h(I/O)g(devices,)g (and)g(with)75 1079 y(access)j(patterns.)24 b(F)l(or)16 b(eac)o(h)h(hin)o(t)g(name)g(in)o(tro)q(duced,)h(w)o(e)f(describ)q(e)h (the)f(purp)q(ose)g(of)g(the)f(hin)o(t,)i(and)75 1136 y(the)e(t)o(yp)q(e)f(of)h(the)f(hin)o(t)h(v)m(alue.)23 b(If)15 b(sp)q(eci\014ed,)j(the)e(\\)p Fo([SAME])p Ft(")d(annotation)i (stipulates)i(that)e(the)g(hin)o(t)75 1192 y(v)m(alues)j(pro)o(vided)g (b)o(y)f(all)g(participating)h(pro)q(cesses)g(m)o(ust)e(b)q(e)i(iden)o (tical,)h(otherwise)e(the)g(program)e(is)75 1249 y(erroneous.)75 1346 y Fg(MPI)p 152 1346 13 2 v 14 w(HINTS)p 287 1346 V 16 w(STRIPING)p 489 1346 V 14 w(UNIT)i Fo(\(in)o(teger\))h([SAME]:)j Ft(This)d(hin)o(t)h(sp)q(eci\014es)h(the)e(suggested)g(striping)189 1402 y(unit)c(to)e(b)q(e)i(used)g(for)f(this)g(\014le.)21 b(The)13 b(striping)h(unit)g(is)g(the)f(amoun)o(t)g(of)g(consecutiv)o (e)h(data)e(tak)o(en)189 1459 y(from)j(one)i(I/O)g(no)q(de)g(\(resp.)24 b(storage)15 b(device\))j(b)q(efore)f(progressing)f(to)g(the)h(next)f (no)q(de)i(\(resp.)189 1515 y(storage)c(device\),)j(when)f(striping)g (across)f(a)h(n)o(um)o(b)q(er)g(of)f(no)q(des)h(\(resp.)21 b(storage)15 b(devices\);)h(it)g(is)189 1572 y(expressed)j(in)g(b)o (ytes.)30 b(This)19 b(hin)o(t)g(is)g(relev)m(an)o(t)g(only)h(when)f (the)f(\014le)i(is)f(created,)g(and)g(if)g(used,)189 1628 y(this)13 b(data)f(should)h(b)q(e)h(main)o(tained)f(b)o(y)g(the)g (\014le)g(serv)o(er\(s\))f(as)g(part)g(of)h(the)f(metadata)g(asso)q (ciated)189 1685 y(with)g(the)g(\014le.)19 b(A)12 b(go)q(o)q(d)g(size)g (for)g(a)f(striping)i(unit)f(is)g(the)g(amoun)o(t)f(of)h(consecutiv)o (e)g(data)f(accessed)189 1741 y(indep)q(enden)o(tly)18 b(b)o(y)d(a)g(pro)q(cess,)g(or)g(if)g(this)h(is)g(to)q(o)e(small,)i(a)f (m)o(ultiple)i(of)d(that)h(size.)75 1832 y Fg(MPI)p 152 1832 V 14 w(HINTS)p 287 1832 V 16 w(STRIPING)p 489 1832 V 14 w(F)m(A)o(CTOR)g Fo(\(in)o(teger\))j([SAME]:)j Ft(This)e(hin)o(t)g (sp)q(eci\014es)h(the)e(n)o(um)o(b)q(er)h(of)f(I/O)189 1889 y(no)q(des)h(\(resp.)29 b(storage)17 b(devices\))j(that)e(the)g (\014le)i(should)f(b)q(e)g(strip)q(ed)h(across,)e(and)g(is)h(relev)m (an)o(t)189 1945 y(only)c(when)h(the)f(\014le)i(is)e(created.)75 2036 y Fg(MPI)p 152 2036 V 14 w(HINTS)p 287 2036 V 16 w(NUM)p 397 2036 V 15 w(IO)p 455 2036 V 14 w(NODES)i Fo(\(in)o(teger\))h([SAME]:)j Ft(The)c(n)o(um)o(b)q(er)g(of)f(I/O)h(no) q(des)h(in)f(the)g(system.)189 2092 y(Used)h(to)f(sp)q(ecify)i(the)f (ideal)h(n)o(um)o(b)q(er)f(of)f(I/O)h(no)q(des)h(for)e(this)h (application)h(as)f(w)o(ell)g(as)g(query)189 2149 y(the)d(n)o(um)o(b)q (er)g(of)g(I/O)h(no)q(des)g(actually)g(in)g(use.)75 2240 y Fg(MPI)p 152 2240 V 14 w(HINTS)p 287 2240 V 16 w(IO)p 346 2240 V 14 w(NODE)p 475 2240 V 15 w(LIST)h Fo(\(arra)o(y)g(of)h(in)o (teger\))f([SAME]:)k Ft(In)c(addition)g(to)f(de\014ning)h(the)g(strip-) 189 2296 y(ing)e(factor,)f(one)h(ma)o(y)f(w)o(an)o(t)g(to)g(giv)o(e)h (the)g(explicit)i(list)f(of)e(I/O)h(no)q(des)h(\(resp.)j(storage)14 b(devices\))189 2353 y(that)g(should)i(b)q(e)g(used.)21 b(This)15 b(hin)o(t)h(is)g(relev)m(an)o(t)g(only)f(when)h(the)f(\014le) i(is)e(created.)75 2443 y Fg(MPI)p 152 2443 V 14 w(HINTS)p 287 2443 V 16 w(NB)p 360 2443 V 15 w(PROC)h Fo(\(in)o(teger\))i ([SAME]:)j Ft(This)e(sp)q(eci\014es)h(the)f(n)o(um)o(b)q(er)g(of)f (parallel)j(pro)q(cesses)189 2500 y(that)15 b(will)i(t)o(ypically)g(b)q (e)f(assigned)g(to)f(run)h(programs)f(that)f(access)i(this)g(\014le,)h (and)e(can)h(b)q(e)g(used)189 2556 y(at)e(\014le)i(creation.)75 2647 y Fg(MPI)p 152 2647 V 14 w(HINTS)p 287 2647 V 16 w(A)o(CCESS)p 455 2647 V 13 w(STYLE)i Fo(\(in)o(teger\):)23 b Ft(This)16 b(hin)o(t)h(sp)q(eci\014es)h(the)e(manner)g(in)g(whic)o(h) h(the)f(\014le)h(will)189 2704 y(b)q(e)f(accessed)f(un)o(til)i(the)e (\014le)h(is)g(closed)g(or)f(un)o(til)h(the)f(program)f(terminates.) 1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(17)189 49 y(The)15 b(hin)o(t)h(v)m(alue)g(is)g(an)f(in)o(teger)g (equal)h(to)f(one)g(of)g(the)g(constan)o(ts:)230 159 y Fg(MPI)p 307 159 13 2 v 14 w(HINTS)p 442 159 V 16 w(READ)p 568 159 V 14 w(ONCE)230 203 y(MPI)p 307 203 V 14 w(HINTS)p 442 203 V 16 w(WRITE)p 589 203 V 15 w(ONCE)230 247 y(MPI)p 307 247 V 14 w(HINTS)p 442 247 V 16 w(READ)p 568 247 V 14 w(MOSTL)m(Y)230 291 y(MPI)p 307 291 V 14 w(HINTS)p 442 291 V 16 w(WRITE)p 589 291 V 15 w(MOSTL)m(Y)189 393 y Ft(P)o(ossibly)i(mo)q(di\014ed)i(b)o(y)e(one)g(of)g(the)g(follo)o (wing)h(constan)o(ts)e(\(via)h(the)g(bit)o(wise)h(OR)g(op)q(erator)e (in)189 449 y(C)f(or)f(simple)j(addition)f(in)g(F)o(OR)l(TRAN\):)230 559 y Fg(MPI)p 307 559 V 14 w(HINTS)p 442 559 V 16 w(SEQUENTIAL)230 603 y(MPI)p 307 603 V 14 w(HINTS)p 442 603 V 16 w(REVERSE)p 638 603 V 14 w(SEQUENTIAL)230 647 y(MPI)p 307 647 V 14 w(HINTS)p 442 647 V 16 w(RANDOM)75 767 y(MPI)p 152 767 V 14 w(HINTS)p 287 767 V 16 w(COLLECTIVE)p 550 767 V 13 w(BUFFERING)g Fo(\(logical\))k([SAME]:)h Ft(This)11 b(hin)o(t)f(sp)q(eci\014es)i(whether)f(the)f(ap-)189 824 y(plication)i(will)h(b)q(ene\014t)g(from)d(collectiv)o(e)j (bu\013ering)f(\(an)f(optimization)h(p)q(erformed)f(on)g(collectiv)o(e) 189 880 y(accesses)16 b(whic)o(h)g(coalesces)h(small)f(requests)g(in)o (to)g(large)g(disk)g(accesses\).)22 b(Collectiv)o(e)17 b(bu\013ering)189 937 y(parameters)d(are)h(further)g(directed)h(via)g Fg(MPI)p 979 937 V 14 w(HINTS)p 1114 937 V 16 w(CB)p 1185 937 V 14 w(XXX)e Ft(hin)o(ts.)75 1031 y Fg(MPI)p 152 1031 V 14 w(HINTS)p 287 1031 V 16 w(CB)p 358 1031 V 14 w(NODES)j Fo(\(in)o(teger\))h([SAME]:)j Ft(The)14 b(n)o(um)o(b)q(er)f(of)g(target)g(no)q(des)g(used)h(for)f(collectiv)o (e)189 1087 y(bu\013ering.)20 b(All)d(\014le)f(accesses)f(are)g(p)q (erformed)h(b)o(y)f(the)g(target)f(no)q(des.)75 1181 y Fg(MPI)p 152 1181 V 14 w(HINTS)p 287 1181 V 16 w(CB)p 358 1181 V 14 w(BLOCK)p 509 1181 V 14 w(SIZE)j Fo(\(o\013set\))h ([SAME]:)j Ft(The)d(blo)q(c)o(k)h(size)g(used)f(for)f(\014le)i(access.) 29 b(T)l(arget)189 1237 y(no)q(des)17 b(access)f(data)g(in)h(c)o(h)o (unks)g(of)e(this)i(size.)24 b(The)17 b(c)o(h)o(unks)g(are)f (distributed)i(to)d(target)g(no)q(des)189 1294 y(in)h(a)f(round-robin)h (\(CYCLIC\))f(pattern.)75 1388 y Fg(MPI)p 152 1388 V 14 w(HINTS)p 287 1388 V 16 w(CB)p 358 1388 V 14 w(BUF)p 453 1388 V 14 w(SIZE)i Fo(\(o\013set\))h([SAME]:)i Ft(The)11 b(total)e(bu\013er)h(space)h(used)g(for)e(collectiv)o(e)j(bu\013er-)189 1444 y(ing)j(on)h(eac)o(h)f(target)f(no)q(de;)h(usually)i(a)e(m)o (ultiple)i(of)d Fg(MPI)p 1186 1444 V 15 w(HINTS)p 1322 1444 V 15 w(CB)p 1392 1444 V 15 w(BLOCK)p 1544 1444 V 14 w(SIZE)p Ft(.)166 1550 y(Sp)q(eci\014c)19 b(implemen)o(tations)e (are)f(free)h(to)f(in)o(terpret)g(the)h(hin)o(ts)g(in)g(sligh)o(tly)g (di\013eren)o(t)g(w)o(a)o(ys.)23 b(F)l(or)75 1607 y(example,)17 b(the)f(follo)o(wing)h(table)f(outlines)i(p)q(ossible)g(in)o (terpretations)e(for)f(an)h(implemen)o(tation)i(based)75 1663 y(on)d(the)g(IBM)h(AIX)g(P)o(arallel)g(I/O)f(File)i(System)e([1)o (,)g(2]:)p 483 1718 985 2 v 482 1774 2 57 v 507 1757 a Fk(hint)i(name)p 1116 1774 V 433 w(interpr)n(etation)p 1467 1774 V 483 1776 985 2 v 482 1832 2 57 v 507 1815 a Fg(MPI)p 584 1815 13 2 v 15 w(HINTS)p 720 1815 V 15 w(STRIPING)p 921 1815 V 14 w(UNIT)p 1116 1832 2 57 v 111 w Ft(BSU)e(size)p 1467 1832 V 482 1889 V 507 1872 a Fg(MPI)p 584 1872 13 2 v 15 w(HINTS)p 720 1872 V 15 w(STRIPING)p 921 1872 V 14 w(F)m(A)o(CTOR)p 1116 1889 2 57 v 48 w Ft(n)o(um)o(b)q(er)g(of)g(cells)p 1467 1889 V 482 1945 V 507 1928 a Fg(MPI)p 584 1928 13 2 v 15 w(HINTS)p 720 1928 V 15 w(IO)p 778 1928 V 14 w(NODE)p 907 1928 V 15 w(LIST)p 1116 1945 2 57 v 137 w Ft(base)g(no)q(de)p 1467 1945 V 483 1947 985 2 v 75 2052 a Fl(10.2.7)49 b(Deleting)17 b(a)g(\014le)f(\(Indep)q(endent\))75 2232 y Fq(MPI)p 160 2232 14 2 v 16 w(DELETE\(\014lename\))117 2310 y Ff(IN)155 b Fq(\014lename)430 b Ff(Name)13 b(of)g(the)i(\014le)f(to)f (b)q(e)i(deleted)g(\(string\))75 2434 y Fp(int)47 b(MPI)p 269 2434 15 2 v 17 w(Delete\(char)22 b(*filename\))75 2521 y(MPI)p 150 2521 V 17 w(DELETE\(FILENAME,)f(IERROR\))170 2577 y(CHARACTER)i(FILENAME\(*\))170 2633 y(INTEGER)g(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Ft(18)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(DELETE)15 b Ft(deletes)h(a)e(\014le.)21 b(If)16 b(the)f(\014le)h(exists)f(it)h (is)f(remo)o(v)o(ed.)20 b(If)15 b(there)g(are)g(other)g(pro)q(cesses)75 106 y(curren)o(tly)k(accessing)h(the)f(\014le,)i(the)e(status)f(of)h (the)g(\014le)h(and)f(the)g(b)q(eha)o(vior)g(of)g(future)g(accesses)g (are)75 162 y(implemen)o(tation)d(dep)q(enden)o(t.)21 b(If)14 b(the)h(\014le)g(do)q(es)g(not)f(exist,)g Fq(MPI)p 1216 162 V 16 w(DELETE)g Ft(returns)g(a)g(w)o(arning)h(error)75 219 y(co)q(de.)189 325 y Fk(A)n(dvic)n(e)i(to)i(implementors.)54 b Ft(If)17 b(the)h(\014le)h(to)d(b)q(e)j(deleted)g(is)f(op)q(ened)g(b)o (y)g(other)f(pro)q(cesses,)h(\014le)189 381 y(data)e(ma)o(y)h(still)i (b)q(e)f(accessible)h(b)o(y)e(these)h(pro)q(cesses)g(un)o(til)g(they)g (close)g(the)f(\014le)i(or)d(un)o(til)j(they)189 438 y(exit.)h(\()p Fk(End)c(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))75 560 y Fl(10.2.8)49 b(Resizing)18 b(a)f(\014le)f (\(Collective\))75 740 y Fq(MPI)p 160 740 V 16 w(RESIZE\(MPI)p 425 740 V 16 w(File)f(fh,)g(MPI)p 671 740 V 15 w(O\013set)i(size\))117 817 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i (handle)g(\(handle\))117 892 y(IN)155 b Fq(size)519 b Ff([SAME])18 b(Size)h(whic)o(h)f(the)h(\014le)f(is)g(to)g(b)q(e)h (truncated)h(at)e(or)905 949 y(expanded)d(to)f(\(nonnegativ)o(e)f (o\013set\))75 1073 y Fp(int)23 b(MPI)p 245 1073 15 2 v 17 w(Resize\(MPI)p 502 1073 V 16 w(File)g(fh,)h(MPI)p 805 1073 V 17 w(Offset)f(size\))75 1160 y(MPI)p 150 1160 V 17 w(RESIZE\(FH,)f(SIZE,)h(IERROR\))170 1216 y(INTEGER)g(FH,)h (IERROR)170 1272 y(INTEGER*8)f(SIZE)166 1359 y Fq(MPI)p 251 1359 14 2 v 16 w(RESIZE)18 b Ft(resizes)h(the)f(\014le)g(asso)q (ciated)g(with)h(the)f(\014le)h(handle)g Fq(fh)p Ft(.)28 b(If)18 b Fq(size)g Ft(is)g(smaller)h(than)75 1415 y(the)14 b(curren)o(t)g(\014le)h(size,)g(the)f(\014le)h(is)f(truncated)g(at)f (the)h(p)q(osition)h(de\014ned)h(b)o(y)e Fq(size)g Ft(\(from)f(the)h(b) q(eginning)75 1472 y(of)f(the)g(\014le)h(and)f(measured)g(in)h(b)o (ytes\).)19 b(File)14 b(blo)q(c)o(ks)f(lo)q(cated)h(b)q(ey)o(ond)g (that)e(p)q(osition)i(are)f(deallo)q(cated.)75 1528 y(If)19 b Fq(size)h Ft(is)f(larger)g(than)g(the)g(curren)o(t)g(\014le)h(size,)h (additional)f(\014le)g(blo)q(c)o(ks)g(are)f(allo)q(cated)h(and)f(the)g (\014le)75 1585 y(size)14 b(b)q(ecomes)h Fq(size)p Ft(.)k(All)c(pro)q (cesses)f(in)g(the)g(comm)o(unicator)f(group)g(m)o(ust)g(call)h Fq(MPI)p 1534 1585 V 16 w(RESIZE)g Ft(with)g(the)75 1641 y(same)h Fq(size)p Ft(.)166 1780 y Fe(Discussion:)j Ff(There)e(are)g(t) o(w)o(o)e(issues)j(asso)q(ciated)f(with)e(\014le)h(blo)q(c)o(k)g (preallo)q(cation:)20 b(it)15 b(ma)o(y)e(b)q(e)j(slo)o(w)e(on)75 1837 y(systems)j(without)g(direct)g(\014lesystem)g(supp)q(ort,)h(and)f (w)o(e)g(m)o(ust)f(b)q(e)i(careful)f(to)f(de\014ne)i(what)f(happ)q(ens) h(if)e(not)75 1893 y(enough)e(space)h(is)f(a)o(v)n(ailable.)75 2098 y Fl(10.2.9)49 b(File)17 b(Sync)f(\(Collective\))75 2278 y Fq(MPI)p 160 2278 V 16 w(FILE)p 267 2278 V 15 w(SYNC\(fh\))117 2355 y Ff(IN)155 b Fq(fh)550 b Ff([SAME]V)m(alid)12 b(\014le)i(handle)g(\(handle\))75 2480 y Fp(int)23 b(MPI)p 245 2480 15 2 v 17 w(File)p 358 2480 V 17 w(sync\(MPI)p 567 2480 V 16 w(File)g(fh\))75 2566 y(MPI)p 150 2566 V 17 w(FILE)p 263 2566 V 16 w(SYNC\(FH,)g(IERROR\))170 2622 y(INTEGER)g(FH,)h(IERROR)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fn(10.3.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (EXPLICIT)h(OFFSETS)697 b Ft(19)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(FILE)p 358 49 V 15 w(SYNC)19 b Ft(causes)e(the)h(con)o (ten)o(ts)f(of)g(the)h(\014le)h(referenced)f(b)o(y)g Fq(fh)g Ft(to)e(b)q(e)j(\015ushed)f(to)f(p)q(er-)75 106 y(manen)o(t)g(storage.)26 b(All)18 b(pro)q(cesses)g(in)g(the)g(comm)o (unicator)f(group)g(asso)q(ciated)h(with)g(the)f(\014le)i(handle)75 162 y Fq(fh)c Ft(m)o(ust)e(call)j Fq(MPI)p 406 162 V 16 w(FILE)p 513 162 V 15 w(SYNC)p Ft(.)f(The)g Fq(MPI)p 847 162 V 16 w(FILE)p 954 162 V 15 w(SYNC)g Ft(call)h(returns)e Fk(after)h Ft(all)g(data)f(written,)g(since)75 219 y(the)h(\014le)i(w)o (as)d(op)q(ened,)i(has)f(b)q(een)h(\015ushed)g(to)f(p)q(ermanen)o(t)g (storage.)189 325 y Fk(A)n(dvic)n(e)j(to)h(users.)58 b Fq(MPI)p 653 325 V 15 w(FILE)p 759 325 V 16 w(SYNC)19 b Ft(guaran)o(tees)e(that)h(all)h Fk(c)n(omplete)n(d)f Ft(I/O)g(requests)g(ha)o(v)o(e)189 381 y(b)q(een)e(\015ushed)h(to)e(p)q (ermanen)o(t)g(storage.)20 b(P)o(ending)c(non)o(blo)q(c)o(king)h(I/O)f (requests)f(that)g(ha)o(v)o(e)g(not)189 438 y(completed)h(are)f(not)g (guaran)o(teed)f(to)h(b)q(e)h(\015ushed.)21 b(\()p Fk(End)15 b(of)i(advic)n(e)f(to)g(users.)p Ft(\))166 627 y Fe(Discussion:)d Ff(Do)q(es)d(this)g(imply)e(w)o(eak)h(consistency)i(b)q(et)o(w)o(een)h (applications)c(at)i Fg(MPI)p 1521 627 13 2 v 14 w(FILE)p 1618 627 V 15 w(SYNC)f Ff(b)q(ound-)75 683 y(aries?)18 b(Should)c(it)f(b)q(e)i(required?)75 909 y Fm(10.3)60 b(Data)19 b(Access)h(With)g(Explicit)e(O\013sets)75 1012 y Fl(10.3.1)49 b(Blo)q(cking)18 b(Read)75 1193 y Fq(MPI)p 160 1193 14 2 v 16 w(READ\(fh,)d(o\013set,)g(bu\013,)h(buft)o(yp)q(e,)g (bufcount,)h(status\))117 1270 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1345 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(nonnegativ)o(e)g (o\013set\))117 1420 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 1495 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat) o(yp)q(e)f(\(handle\))117 1570 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(in)o (teger\))117 1645 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1770 y Fp(int)23 b(MPI)p 245 1770 15 2 v 17 w(Read\(MPI)p 454 1770 V 16 w(File)h(fh,)f(MPI)p 757 1770 V 17 w(Offset)g(offset,)g(void)g(*buff,)393 1826 y(MPI)p 468 1826 V 17 w(Datatype)g(buftype,)f(int)i(bufcount,)f (MPI)p 1321 1826 V 16 w(Status)g(*status\))75 1913 y(MPI)p 150 1913 V 17 w(READ\(FH,)g(OFFSET,)f(BUFF,)i(BUFTYPE,)e(BUFCOUNT,)h (STATUS,)g(IERROR\))170 1969 y()g(BUFF\(*\))170 2026 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2026 V 16 w(STATUS)p 1313 2026 V 16 w(SIZE\),)h(IERROR)170 2082 y(INTEGER*8)g(OFFSET)166 2169 y Fq(MPI)p 251 2169 14 2 v 16 w(READ)c Ft(attempts)f(to)h(read)g(from)f(the)h(\014le)i (asso)q(ciated)e(with)g Fq(fh)h Ft(\(at)e(the)h Fq(o\013set)g Ft(p)q(osition\))75 2225 y(a)d(total)g(n)o(um)o(b)q(er)i(of)e Fq(bufcount)j Ft(data)d(items)h(ha)o(ving)g Fq(buft)o(yp)q(e)h Ft(datat)o(yp)q(e)e(in)o(to)h(the)g(user's)f(bu\013er)h Fq(bu\013)p Ft(.)75 2281 y(The)f(data)f(is)i(tak)o(en)e(out)h(of)f (those)h(parts)f(of)h(the)g(\014le)g(sp)q(eci\014ed)i(b)o(y)e(\014let)o (yp)q(e.)23 b Fq(MPI)p 1533 2281 V 16 w(READ)17 b Ft(stores)e(the)75 2338 y(n)o(um)o(b)q(er)h(of)e Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f (actually)g(read)f(in)h Fq(status)p Ft(.)75 2489 y Fq(MPI)p 160 2489 V 16 w(READ)p 295 2489 V 17 w(ALL\(fh,)e(o\013set,)h(bu\013,)g (buft)o(yp)q(e,)i(bufcount,)g(status\))75 2593 y Fp(int)23 b(MPI)p 245 2593 15 2 v 17 w(Read)p 358 2593 V 17 w(all\(MPI)p 543 2593 V 16 w(File)g(fh,)h(MPI)p 846 2593 V 17 w(Offset)f(offset,)g (void)g(*buff,)393 2649 y(MPI)p 468 2649 V 17 w(Datatype)g(buftype,)f (int)i(bufcount,)f(MPI)p 1321 2649 V 16 w(Status)g(*status\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Ft(20)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fp(MPI)p 150 49 15 2 v 17 w(READ)p 263 49 V 16 w(ALL\(FH,)23 b(OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (STATUS,)g(IERROR\))170 106 y()g(BUFF\(*\))170 162 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 162 V 16 w(STATUS)p 1313 162 V 16 w(SIZE\),)h(IERROR)170 219 y(INTEGER*8)g(OFFSET)166 305 y Fq(MPI)p 251 305 14 2 v 16 w(READ)p 386 305 V 17 w(ALL)10 b Ft(is)h(a)g(collectiv)o(e)i(v)o (ersion)e(of)f(the)h(blo)q(c)o(king)h Fq(MPI)p 1290 305 V 16 w(READ)f Ft(in)o(terface.)19 b(This)11 b(routine)75 362 y(is)i(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 694 362 V 16 w(READ)p Ft(,)g(except)h(that)f(all)h(pro)q(cesses)g(in)h (the)e(comm)o(unicator)g(group)75 418 y(asso)q(ciated)j(with)h(the)f (\014le)h(handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1026 418 V 16 w(READ)p 1161 418 V 17 w(ALL)p Ft(.)75 540 y Fl(10.3.2)49 b(Blo)q(cking)18 b(W)o(rite)75 720 y Fq(MPI)p 160 720 V 16 w(WRITE\(fh,)d(o\013set,)g(bu\013,)g(buft)o(yp) q(e,)i(bufcount,)g(status\))117 797 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 872 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(nonnegativ)o(e)g (o\013set\))117 947 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 1023 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat) o(yp)q(e)f(\(handle\))117 1098 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(in)o (teger\))117 1173 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1297 y Fp(int)47 b(MPI)p 269 1297 15 2 v 17 w(Write\(MPI)p 502 1297 V 16 w(File)23 b(fh,)h(MPI)p 805 1297 V 17 w(Offset)f(offset,)g(void)g(*buff,)393 1354 y(MPI)p 468 1354 V 17 w(Datatype)g(buftype,)f(int)i(bufcount,)f (MPI)p 1321 1354 V 16 w(Status)g(*status\))75 1440 y(MPI)p 150 1440 V 17 w(WRITE\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (STATUS,)g(IERROR\))170 1497 y()g(BUFF\(*\))170 1553 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1553 V 16 w(STATUS)p 1313 1553 V 16 w(SIZE\),)h(IERROR)170 1610 y(INTEGER*8)g(OFFSET)166 1696 y Fq(MPI)p 251 1696 14 2 v 16 w(WRITE)17 b Ft(attempts)f(to)h(write)g(in)o(to)h(the)f (\014le)h(asso)q(ciated)g(with)f Fq(fh)h Ft(\(at)e(the)i Fq(o\013set)f Ft(p)q(osition\))75 1753 y(a)e(total)g(n)o(um)o(b)q(er)h (of)f Fq(bufcount)j Ft(data)d(items)h(ha)o(ving)g Fq(buft)o(yp)q(e)h Ft(datat)o(yp)q(e)e(from)g(the)h(user's)f(bu\013er)h Fq(bu\013)p Ft(.)75 1809 y(The)f(data)f(is)h(written)g(in)o(to)g(those) g(parts)f(of)g(the)h(\014le)h(sp)q(eci\014ed)h(b)o(y)d(\014let)o(yp)q (e.)21 b Fq(MPI)p 1512 1809 V 16 w(WRITE)15 b Ft(stores)f(the)75 1865 y(n)o(um)o(b)q(er)i(of)e Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f (actually)g(written)f(in)h Fq(status)p Ft(.)75 2016 y Fq(MPI)p 160 2016 V 16 w(WRITE)p 319 2016 V 16 w(ALL\(fh,)f(o\013set,)g (bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g(status\))75 2120 y Fp(int)47 b(MPI)p 269 2120 15 2 v 17 w(Write)p 406 2120 V 17 w(all\(MPI)p 591 2120 V 16 w(File)23 b(fh,)h(MPI)p 894 2120 V 16 w(Offset)f(offset,)g(void)h(*buff,)393 2177 y(MPI)p 468 2177 V 17 w(Datatype)f(buftype,)f(int)i(bufcount,)f (MPI)p 1321 2177 V 16 w(Status)g(*status\))75 2263 y(MPI)p 150 2263 V 17 w(WRITE)p 287 2263 V 16 w(ALL\(FH,)g(OFFSET,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)g(STATUS,)g(IERROR\))170 2319 y()g (BUFF\(*\))170 2376 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (STATUS\(MPI)p 1153 2376 V 16 w(STATUS)p 1313 2376 V 16 w(SIZE\),)h(IERROR)170 2432 y(INTEGER*8)g(OFFSET)166 2519 y Fq(MPI)p 251 2519 14 2 v 16 w(WRITE)p 410 2519 V 16 w(ALL)12 b Ft(is)g(a)g(collectiv)o(e)i(v)o(ersion)f(of)e(the)i (blo)q(c)o(king)g Fq(MPI)p 1324 2519 V 16 w(WRITE)f Ft(in)o(terface.)19 b(This)13 b(rou-)75 2575 y(tine)h(is)h(seman)o(tically)f(iden)o(tical)i (to)d Fq(MPI)p 788 2575 V 16 w(WRITE)p Ft(,)g(except)h(that)f(all)i (pro)q(cesses)f(in)h(the)e(comm)o(unicator)75 2632 y(group)i(asso)q (ciated)g(with)h(the)f(\014le)h(handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1155 2632 V 16 w(WRITE)p 1314 2632 V 16 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fn(10.3.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (EXPLICIT)h(OFFSETS)697 b Ft(21)75 49 y Fl(10.3.3)49 b(Nonblo)q(cking)19 b(Read)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IREAD\(fh,)c(o\013set,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(O\013set)h(\(nonnegativ)o(e)f (o\013set\))117 457 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 532 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 607 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 682 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f(\(handle\))75 807 y Fp(int)23 b(MPI)p 245 807 15 2 v 17 w(Iread\(MPI)p 478 807 V 16 w(File)g(fh,)h(MPI)p 781 807 V 17 w(Offset)f(offset,)g(void)g (*buff,)393 863 y(MPI)p 468 863 V 17 w(Datatype)g(buftype,)f(int)i (bufcount,)f(MPI)p 1321 863 V 16 w(Request)g(*request\))75 950 y(MPI)p 150 950 V 17 w(IREAD\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)g(REQUEST,)g(IERROR\))170 1006 y()g(BUFF\(*\))170 1063 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)170 1119 y(INTEGER*8)g(OFFSET)166 1206 y Fq(MPI)p 251 1206 14 2 v 16 w(IREAD)12 b Ft(is)g(a)g(non)o(blo)q(c)o(king)h(v)o(ersion)f (of)g(the)g Fq(MPI)p 1094 1206 V 16 w(READ)g Ft(in)o(terface.)19 b Fq(MPI)p 1524 1206 V 16 w(IREAD)12 b Ft(asso)q(ciates)75 1262 y(a)f(request)g(handle)h Fq(request)h Ft(with)f(the)f(I/O)h (request.)18 b(The)12 b(request)f(handle)h(can)g(b)q(e)f(used)h(later)f (to)g(query)75 1319 y(the)k(status)f(of)g(the)h(read)g(request)g(\(via) f Fq(MPI)p 851 1319 V 16 w(TEST)p Ft(\),)g(or)h(w)o(ait)f(for)g(its)h (completion)h(\(via)f Fq(MPI)p 1719 1319 V 16 w(W)l(AIT)p Ft(\).)166 1375 y(The)k(non)o(blo)q(c)o(king)i(read)e(call)h(indicates) g(that)e(the)i(system)e(can)h(start)f(to)h(read)f(data)h(in)o(to)g(the) 75 1431 y(supplied)e(bu\013er.)j(The)15 b(user)g(should)h(not)e(access) h(an)o(y)g(part)f(of)g(the)h(bu\013er)g(after)f(a)h(non)o(blo)q(c)o (king)h(read)75 1488 y(is)h(called,)h(un)o(til)f(the)f(read)h (completes)g(\(as)e(indicated)j(b)o(y)e Fq(MPI)p 1187 1488 V 16 w(TEST)h Ft(or)f Fq(MPI)p 1473 1488 V 15 w(W)l(AIT)p Ft(\).)g Fq(MPI)p 1730 1488 V 16 w(IREAD)75 1544 y Ft(attempts)e(to)g (read)h(from)g(the)g(\014le)h(asso)q(ciated)f(with)h Fq(fh)f Ft(\(at)f(the)h Fq(o\013set)g Ft(p)q(osition\),)h(a)f(total)f (n)o(um)o(b)q(er)h(of)75 1601 y Fq(bufcount)i Ft(data)e(items)g(ha)o (ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)h(user's)f (bu\013er)g Fq(bu\013)p Ft(.)20 b(The)c(n)o(um)o(b)q(er)f(of)g Fq(buft)o(yp)q(e)75 1657 y Ft(elemen)o(ts)j(actually)h(read)e(can)h(b)q (e)g(extracted)f(from)g(the)h Fq(MPI)p 1171 1657 V 16 w(TEST)f Ft(or)g Fq(MPI)p 1458 1657 V 16 w(W)l(AIT)h Ft(return)f(status,)75 1714 y(using)f(the)f(routines)h Fq(MPI)p 532 1714 V 16 w(GET)p 636 1714 V 16 w(COUNT)g Ft(or)f Fq(MPI)p 962 1714 V 16 w(GET)p 1066 1714 V 17 w(ELEMENTS)p Ft(.)75 1865 y Fq(MPI)p 160 1865 V 16 w(IREAD)p 308 1865 V 16 w(ALL\(fh,)g(o\013set,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 1968 y Fp(int)23 b(MPI)p 245 1968 15 2 v 17 w(Iread)p 382 1968 V 17 w(all\(MPI)p 567 1968 V 16 w(File)g(fh,)h(MPI)p 870 1968 V 16 w(Offset)g(offset,)e(void) i(*buff,)393 2025 y(MPI)p 468 2025 V 17 w(Datatype)f(buftype,)f(int)i (bufcount,)f(MPI)p 1321 2025 V 16 w(Request)g(*request\))75 2111 y(MPI)p 150 2111 V 17 w(IREAD)p 287 2111 V 16 w(ALL\(FH,)g (OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(REQUEST,)f(IERROR\))170 2168 y()h(BUFF\(*\))170 2224 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)170 2281 y(INTEGER*8)g(OFFSET)166 2367 y Fq(MPI)p 251 2367 14 2 v 16 w(IREAD)p 399 2367 V 16 w(ALL)16 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)f(of)f(the)g (non)o(blo)q(c)o(king)i Fq(MPI)p 1419 2367 V 16 w(IREAD)f Ft(in)o(terface.)23 b(This)75 2424 y(routine)17 b(is)g(seman)o(tically) g(iden)o(tical)h(to)e Fq(MPI)p 867 2424 V 16 w(IREAD)p Ft(,)g(except)g(that)g(all)h(pro)q(cesses)g(in)g(the)f(comm)o(uni-)75 2480 y(cator)e(group)h(asso)q(ciated)h(with)f(the)g(\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)j Fq(MPI)p 1272 2480 V 15 w(IREAD)p 1419 2480 V 17 w(ALL)p Ft(.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 -100 a Ft(22)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.3.4)49 b(Nonblo)q(cking)19 b(W)o(rite)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IWRITE\(fh,)14 b(o\013set,)h (bu\013,)h(buft)o(yp)q(e,)g(bufcount,)h(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 382 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(O\013set)h(\(nonnegativ)o(e)f(o\013set\))117 457 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 532 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 607 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q (e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 682 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f (\(handle\))75 807 y Fp(int)47 b(MPI)p 269 807 15 2 v 17 w(Iwrite\(MPI)p 526 807 V 16 w(File)23 b(fh,)h(MPI)p 829 807 V 16 w(Offset)f(offset,)g(void)h(*buff,)393 863 y(MPI)p 468 863 V 17 w(Datatype)f(buftype,)f(int)i(bufcount,)f(MPI)p 1321 863 V 16 w(Request)g(*request\))75 950 y(MPI)p 150 950 V 17 w(IWRITE\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (REQUEST,)g(IERROR\))170 1006 y()g(BUFF\(*\))170 1063 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)170 1119 y(INTEGER*8)g(OFFSET)166 1206 y Fq(MPI)p 251 1206 14 2 v 16 w(IWRITE)14 b Ft(is)i(a)e(non)o(blo)q(c)o(king)i(v)o(ersion)f (of)g(the)g Fq(MPI)p 1138 1206 V 15 w(WRITE)g Ft(in)o(terface.)20 b Fq(MPI)p 1595 1206 V 16 w(IWRITE)15 b Ft(asso-)75 1262 y(ciates)g(a)f(request)h(handle)h Fq(request)h Ft(with)e(the)g(I/O)g (request.)20 b(The)15 b(request)f(handle)i(can)f(b)q(e)h(used)f(later) 75 1319 y(to)j(query)h(the)g(status)f(of)g(the)h(write)g(request)g (\(via)f Fq(MPI)p 1083 1319 V 16 w(TEST)p Ft(\),)g(or)h(w)o(ait)f(for)g (its)h(completion)h(\(via)75 1375 y Fq(MPI)p 160 1375 V 16 w(W)l(AIT)p Ft(\).)166 1431 y(The)c(non)o(blo)q(c)o(king)h(write)f (call)h(indicates)g(that)e(the)h(system)f(can)h(start)f(to)g(write)h (data)f(from)g(the)75 1488 y(supplied)e(bu\013er.)19 b(The)11 b(user)g(should)h(not)f(access)g(an)o(y)f(part)h(of)f(the)h (bu\013er)g(after)g(the)g(non)o(blo)q(c)o(king)h(write)75 1544 y(is)i(called,)g(un)o(til)h(the)e(write)g(completes)h(\(as)e (indicated)j(b)o(y)e Fq(MPI)p 1174 1544 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1452 1544 V 16 w(W)l(AIT)p Ft(\).)h Fq(MPI)p 1707 1544 V 15 w(IWRITE)75 1601 y Ft(attempts)h(to)g(write)i (in)o(to)f(the)g(\014le)h(asso)q(ciated)f(with)h Fq(fh)f Ft(\(at)f(the)h Fq(o\013set)g Ft(p)q(osition\),)h(a)f(total)f(n)o(um)o (b)q(er)h(of)75 1657 y Fq(bufcount)h Ft(data)e(items)g(ha)o(ving)g Fq(buft)o(yp)q(e)i Ft(t)o(yp)q(e)e(from)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)20 b(The)14 b(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)75 1714 y Ft(elemen)o(ts)f(actually)h(written)f(can)f (b)q(e)i(extracted)e(from)g(the)h Fq(MPI)p 1190 1714 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1468 1714 V 16 w(W)l(AIT)h Ft(return)f(status,)75 1770 y(using)k(the)f(routines)h Fq(MPI)p 532 1770 V 16 w(GET)p 636 1770 V 16 w(COUNT)g Ft(or)f Fq(MPI)p 962 1770 V 16 w(GET)p 1066 1770 V 17 w(ELEMENTS)p Ft(.)75 1921 y Fq(MPI)p 160 1921 V 16 w(IWRITE)p 332 1921 V 16 w(ALL\(fh,)f(o\013set,)h(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 2025 y Fp(int)23 b(MPI)p 245 2025 15 2 v 17 w(Iwrite)p 406 2025 V 17 w(all\(MPI)p 591 2025 V 16 w(File)g(fh,)h(MPI)p 894 2025 V 16 w(Offset)f(offset,)g (void)h(*buff,)393 2081 y(MPI)p 468 2081 V 17 w(Datatype)f(buftype,)f (int)i(bufcount,)f(MPI)p 1321 2081 V 16 w(Request)g(*request\))75 2168 y(MPI)p 150 2168 V 17 w(IWRITE)p 311 2168 V 16 w(ALL\(FH,)g (OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(REQUEST,)f(IERROR\))170 2224 y()h(BUFF\(*\))170 2281 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)170 2337 y(INTEGER*8)g(OFFSET)166 2424 y Fq(MPI)p 251 2424 14 2 v 16 w(IWRITE)p 423 2424 V 16 w(ALL)11 b Ft(is)h(a)g(collectiv)o(e)h(v)o(ersion)f(of)f(the)h (non)o(blo)q(c)o(king)h Fq(MPI)p 1405 2424 V 16 w(IWRITE)e Ft(in)o(terface.)19 b(This)75 2480 y(routine)c(is)g(seman)o(tically)g (iden)o(tical)h(to)e Fq(MPI)p 857 2480 V 16 w(IWRITE)p Ft(,)f(except)i(that)f(all)h(pro)q(cesses)g(in)g(the)f(comm)o(uni-)75 2536 y(cator)g(group)h(asso)q(ciated)h(with)f(the)g(\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)j Fq(MPI)p 1272 2536 V 15 w(IWRITE)p 1443 2536 V 16 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 23 24 bop 75 -100 a Fn(10.4.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (INDIVIDUAL)i(FILE)e(POINTERS)487 b Ft(23)75 49 y Fm(10.4)60 b(Data)19 b(Access)h(With)g(Individual)e(File)i(P)n(ointers)75 151 y Ft(The)14 b(system)f(main)o(tains)h(one)f(individual)k(\014le)e (p)q(oin)o(ter)f(p)q(er)g(pro)q(cess)g(p)q(er)g(collectiv)o(e)h Fq(MPI)p 1636 151 14 2 v 16 w(OPEN)p Ft(.)f(The)75 207 y(curren)o(t)k(v)m(alue)i(of)d(this)i(p)q(oin)o(ter)g(can)f(b)q(e)h (used)g(to)e(implicitly)22 b(sp)q(ecify)d(the)f(o\013set)g(of)f(data)h (accesses,)75 264 y(b)o(y)h(using)h(the)f(routines)h(describ)q(ed)h(in) f(this)g(section.)32 b(These)20 b(routines)f(only)h(use)g(and)f(up)q (date)h(the)75 320 y(individual)g(\014le)e(p)q(oin)o(ters)g(main)o (tained)g(b)o(y)f(the)g(system.)25 b(The)17 b(shared)g(\014le)h(p)q (oin)o(ter)f(is)h(not)f(used)g(nor)75 377 y(up)q(dated.)166 433 y(The)h(individual)j(\014le)e(p)q(oin)o(ter)f(routines)g(ha)o(v)o (e)g(the)g(same)f(seman)o(tics)h(as)g(the)f(data)h(access)g(with)75 489 y(explicit)f(o\013set)e(routines)g(describ)q(ed)i(in)f(Section)g (10.3)e(with)i(the)f(follo)o(wing)h(mo)q(di\014cation:)143 583 y Fh(\017)23 b Ft(the)17 b Fq(o\013set)h Ft(is)f(de\014ned)i(to)e (b)q(e)h(the)f(curren)o(t)h(v)m(alue)g(of)f(the)g(system-main)o(tained) h(individual)j(\014le)189 640 y(p)q(oin)o(ter.)75 734 y(After)15 b(an)g(individual)k(\014le)d(p)q(oin)o(ter)g(op)q(eration)f (is)h(initiated,)g(the)g(individual)i(\014le)f(p)q(oin)o(ter)e(is)h(up) q(dated)75 790 y(to)i(p)q(oin)o(t)h(to)f(the)h(next)g(data)f(item)h (after)f(the)g(last)h(one)g(that)f(will)i(b)q(e)f(actually)h(accessed,) f(p)q(ossibly)75 846 y(accoun)o(ting)d(for)e(an)h(end)h(of)f(\014le)h (o)q(ccurrence.)75 968 y Fl(10.4.1)49 b(Blo)q(cking)18 b(Read)75 1148 y Fq(MPI)p 160 1148 V 16 w(READ)p 295 1148 V 17 w(NEXT\(fh,)d(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (status\))117 1226 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1301 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 1376 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1451 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 1526 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1651 y Fp(int)23 b(MPI)p 245 1651 15 2 v 17 w(Read)p 358 1651 V 17 w(next\(MPI)p 567 1651 V 16 w(File)g(fh,)h(void)f(*buff,) g(MPI)p 1156 1651 V 17 w(Datatype)g(buftype,)393 1707 y(int)h(bufcount,)e(MPI)p 802 1707 V 17 w(Status)h(*status\))75 1794 y(MPI)p 150 1794 V 17 w(READ)p 263 1794 V 16 w(NEXT\(FH,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)g(STATUS,)g(IERROR\))170 1850 y()g (BUFF\(*\))170 1906 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (STATUS\(MPI)p 1153 1906 V 16 w(STATUS)p 1313 1906 V 16 w(SIZE\),)h(IERROR)166 1993 y Fq(MPI)p 251 1993 14 2 v 16 w(READ)p 386 1993 V 17 w(NEXT)14 b Ft(is)g(seman)o(tically)g (iden)o(tical)h(to)e Fq(MPI)p 1156 1993 V 16 w(READ)p Ft(,)g(with)h Fq(o\013set)g Ft(set)f(to)g(the)g(curren)o(t)75 2049 y(individual)21 b(\014le)e(p)q(oin)o(ter)g(p)q(osition)g(main)o (tained)g(b)o(y)f(the)g(system.)28 b(The)18 b(\014le)h(p)q(oin)o(ter)g (is)f(up)q(dated)h(b)o(y)75 2106 y(the)c(amoun)o(t)g(of)f(data)h (requested.)75 2257 y Fq(MPI)p 160 2257 V 16 w(READ)p 295 2257 V 17 w(NEXT)p 432 2257 V 17 w(ALL\(fh,)f(bu\013,)h(buft)o(yp)q (e,)i(bufcount,)g(status\))75 2360 y Fp(int)23 b(MPI)p 245 2360 15 2 v 17 w(Read)p 358 2360 V 17 w(next)p 471 2360 V 16 w(all\(MPI)p 655 2360 V 17 w(File)g(fh,)g(void)h(*buff,)f (MPI)p 1245 2360 V 17 w(Datatype)f(buftype,)393 2417 y(int)i(bufcount,)e(MPI)p 802 2417 V 17 w(Status)h(*status\))75 2503 y(MPI)p 150 2503 V 17 w(READ)p 263 2503 V 16 w(NEXT)p 375 2503 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(STATUS,)f (IERROR\))170 2560 y()h(BUFF\(*\))170 2616 y(INTEGER)g(FH,)h (BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2616 V 16 w(STATUS)p 1313 2616 V 16 w(SIZE\),)h(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 24 25 bop 75 -100 a Ft(24)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(READ)p 386 49 V 17 w(NEXT)p 523 49 V 17 w(ALL)17 b Ft(is)h(a)f(collectiv)o(e)i(v)o (ersion)f(of)f(the)h(blo)q(c)o(king)h Fq(MPI)p 1481 49 V 15 w(READ)p 1615 49 V 17 w(NEXT)g Ft(in)o(ter-)75 106 y(face.)g(This)12 b(routine)h(is)f(seman)o(tically)i(iden)o(tical)g(to) d Fq(MPI)p 1055 106 V 16 w(READ)p 1190 106 V 17 w(NEXT)p Ft(,)h(except)g(that)g(all)h(pro)q(cesses)f(in)75 162 y(the)g(comm)o(unicator)g(group)g(asso)q(ciated)g(with)h(the)f(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1497 162 V 16 w(READ)p 1632 162 V 17 w(NEXT)p 1769 162 V 17 w(ALL)p Ft(.)75 284 y Fl(10.4.2)49 b(Blo)q(cking)18 b(W)o(rite)75 464 y Fq(MPI)p 160 464 V 16 w(WRITE)p 319 464 V 16 w(NEXT\(fh,)e (bu\013,)f(buft)o(yp)q(e,)i(bufcount,)f(status\))117 541 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 617 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 692 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 767 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 842 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 966 y Fp(int)23 b(MPI)p 245 966 15 2 v 17 w(Write)p 382 966 V 17 w(next\(MPI)p 591 966 V 16 w(File)g(fh,)h(void)f(*buff,)g(MPI)p 1180 966 V 17 w(Datatype)f(buftype,)393 1023 y(int)i(bufcount,)e(MPI)p 802 1023 V 17 w(Status)h(*status\))75 1109 y(MPI)p 150 1109 V 17 w(WRITE)p 287 1109 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)g(STATUS,)g(IERROR\))170 1166 y()g(BUFF\(*\))170 1222 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1222 V 16 w(STATUS)p 1313 1222 V 16 w(SIZE\),)h(IERROR)166 1309 y Fq(MPI)p 251 1309 14 2 v 16 w(WRITE)p 410 1309 V 16 w(NEXT)16 b Ft(is)f(seman)o(tically)h(iden)o(tical)h(to)d Fq(MPI)p 1187 1309 V 16 w(WRITE)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)g(the)g(cur-)75 1365 y(ren)o(t)h(individual)j(\014le)e(p)q (oin)o(ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)21 b(The)16 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)75 1422 y(b)o(y)f(the)g(amoun)o(t)g(of)g(data)f(requested.)75 1573 y Fq(MPI)p 160 1573 V 16 w(WRITE)p 319 1573 V 16 w(NEXT)p 455 1573 V 17 w(ALL\(fh,)g(bu\013,)i(buft)o(yp)q(e,)g (bufcount,)h(status\))75 1676 y Fp(int)47 b(MPI)p 269 1676 15 2 v 17 w(Write)p 406 1676 V 17 w(next)p 519 1676 V 16 w(all\(MPI)p 703 1676 V 16 w(File)24 b(fh,)f(void)h(*buff,)f(MPI)p 1293 1676 V 16 w(Datatype)g(buftype,)393 1733 y(int)h(bufcount,)e(MPI)p 802 1733 V 17 w(Status)h(*status\))75 1819 y(MPI)p 150 1819 V 17 w(WRITE)p 287 1819 V 16 w(NEXT)p 399 1819 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(STATUS,)h(IERROR\))170 1876 y()g(BUFF\(*\))170 1932 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 1932 V 16 w(STATUS)p 1313 1932 V 16 w(SIZE\),)h(IERROR)166 2018 y Fq(MPI)p 251 2018 14 2 v 16 w(WRITE)p 410 2018 V 16 w(NEXT)p 546 2018 V 17 w(ALL)12 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the) h(blo)q(c)o(king)h Fq(MPI)p 1463 2018 V 15 w(WRITE)p 1621 2018 V 17 w(NEXT)f Ft(in)o(ter-)75 2075 y(face.)20 b(This)15 b(routine)g(is)g(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 1071 2075 V 16 w(WRITE)p 1230 2075 V 16 w(NEXT)p Ft(,)h(except)g(that)f(all)i(pro)q(cesses)75 2131 y(in)11 b(the)f(comm)o(unicator)g(group)g(asso)q(ciated)g(with)h(the)f(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1523 2131 V 16 w(WRITE)p 1682 2131 V 16 w(NEXT)p 1818 2131 V 17 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 25 26 bop 75 -100 a Fn(10.4.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (INDIVIDUAL)i(FILE)e(POINTERS)487 b Ft(25)75 49 y Fl(10.4.3)49 b(Nonblo)q(cking)19 b(Read)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IREAD)p 308 230 V 16 w(NEXT\(fh,)d(bu\013,)f(buft)o(yp)q (e,)i(bufcount,)g(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i (user's)f(bu\013er)h(\(in)o(teger\))117 457 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 532 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 607 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 732 y Fp(int)23 b(MPI)p 245 732 15 2 v 17 w(Iread)p 382 732 V 17 w(next\(MPI)p 591 732 V 16 w(File)g(fh,)h(void)f(*buff,)g(MPI)p 1180 732 V 17 w(Datatype)f (buftype,)393 788 y(int)i(bufcount,)e(MPI)p 802 788 V 17 w(Request)h(*request\))75 875 y(MPI)p 150 875 V 17 w(IREAD)p 287 875 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (REQUEST,)f(IERROR\))170 931 y()h(BUFF\(*\))170 988 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 1074 y Fq(MPI)p 251 1074 14 2 v 16 w(IREAD)p 399 1074 V 16 w(NEXT)12 b Ft(is)g(seman)o(tically)g(iden)o(tical)h(to)d Fq(MPI)p 1157 1074 V 16 w(IREAD)p Ft(,)h(with)g Fq(o\013set)h Ft(set)e(to)h(the)g(curren)o(t)75 1130 y(individual)21 b(\014le)e(p)q(oin)o(ter)g(p)q(osition)g(main)o(tained)g(b)o(y)f(the)g (system.)28 b(The)18 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o (y)75 1187 y(the)c(amoun)o(t)g(of)f(data)h(requested.)75 1338 y Fq(MPI)p 160 1338 V 16 w(IREAD)p 308 1338 V 16 w(NEXT)p 444 1338 V 17 w(ALL\(fh,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 1442 y Fp(int)23 b(MPI)p 245 1442 15 2 v 17 w(Iread)p 382 1442 V 17 w(next)p 495 1442 V 16 w(all\(MPI)p 679 1442 V 17 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1269 1442 V 16 w(Datatype)g(buftype,)393 1498 y(int)h(bufcount,)e(MPI)p 802 1498 V 17 w(Request)h(*request\))75 1584 y(MPI)p 150 1584 V 17 w(IREAD)p 287 1584 V 16 w(NEXT)p 399 1584 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\)) 170 1641 y()g(BUFF\(*\))170 1697 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)166 1784 y Fq(MPI)p 251 1784 14 2 v 16 w(IREAD)p 399 1784 V 16 w(NEXT)p 535 1784 V 17 w(ALL)15 b Ft(is)g(a)f(collectiv)o(e)j(v)o(ersion)e(of)f(the)h (blo)q(c)o(king)h Fq(MPI)p 1471 1784 V 16 w(IREAD)p 1619 1784 V 16 w(NEXT)g Ft(in)o(ter-)75 1840 y(face.)i(This)12 b(routine)f(is)h(seman)o(tically)g(iden)o(tical)h(to)d Fq(MPI)p 1048 1840 V 16 w(IREAD)p 1196 1840 V 16 w(NEXT)p Ft(,)h(except)h(that)e(all)i(pro)q(cesses)f(in)75 1897 y(the)g(comm)o(unicator)g(group)f(asso)q(ciated)i(with)f(the)g(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)f(call)i Fq(MPI)p 1484 1897 V 16 w(IREAD)p 1632 1897 V 17 w(NEXT)p 1769 1897 V 17 w(ALL)p Ft(.)75 2018 y Fl(10.4.4)49 b(Nonblo)q(cking)19 b(W)o(rite)75 2199 y Fq(MPI)p 160 2199 V 16 w(IWRITE)p 332 2199 V 16 w(NEXT\(fh,)c(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (request\))117 2276 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 2351 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 2426 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 2501 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 2577 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2701 y Fp(int)23 b(MPI)p 245 2701 15 2 v 17 w(Iwrite)p 406 2701 V 17 w(next\(MPI)p 615 2701 V 16 w(File)g(fh,)g(void)h(*buff,) f(MPI)p 1204 2701 V 17 w(Datatype)f(buftype,)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 26 27 bop 75 -100 a Ft(26)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)393 49 y Fp(int)24 b(bufcount,)e(MPI)p 802 49 15 2 v 17 w(Request)h(*request\))75 136 y(MPI)p 150 136 V 17 w(IWRITE)p 311 136 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(REQUEST,)h(IERROR\))170 192 y()g(BUFF\(*\))170 249 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 335 y Fq(MPI)p 251 335 14 2 v 16 w(IWRITE)p 423 335 V 16 w(NEXT)f Ft(is)g(seman)o(tically)h(iden)o(tical)h(to)d Fq(MPI)p 1234 335 V 16 w(IWRITE)p Ft(,)g(with)h Fq(o\013set)g Ft(set)g(to)f(the)75 392 y(curren)o(t)11 b(individual)j(\014le)e(p)q (oin)o(ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)18 b(The)11 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)75 448 y(b)o(y)k(the)g(amoun)o(t)g(of)g(data)f(requested.)75 599 y Fq(MPI)p 160 599 V 16 w(IWRITE)p 332 599 V 16 w(NEXT)p 468 599 V 17 w(ALL\(fh,)g(bu\013,)h(buft)o(yp)q(e,)i(bufcount,)g (request\))75 703 y Fp(int)23 b(MPI)p 245 703 15 2 v 17 w(Iwrite)p 406 703 V 17 w(next)p 519 703 V 16 w(all\(MPI)p 703 703 V 16 w(File)h(fh,)f(void)h(*buff,)f(MPI)p 1293 703 V 16 w(Datatype)g(buftype,)393 759 y(int)h(bufcount,)e(MPI)p 802 759 V 17 w(Request)h(*request\))75 846 y(MPI)p 150 846 V 17 w(IWRITE)p 311 846 V 16 w(NEXT)p 423 846 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\))170 902 y()g(BUFF\(*\))170 958 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)166 1045 y Fq(MPI)p 251 1045 14 2 v 16 w(IWRITE)p 423 1045 V 16 w(NEXT)p 559 1045 V 17 w(ALL)15 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)f(of)f(the) h(blo)q(c)o(king)h Fq(MPI)p 1501 1045 V 16 w(IWRITE)p 1673 1045 V 16 w(NEXT)f Ft(in-)75 1101 y(terface.)26 b(This)17 b(routine)h(is)g(seman)o(tically)g(iden)o(tical)h(to)e Fq(MPI)p 1149 1101 V 15 w(IWRITE)p 1320 1101 V 16 w(NEXT)p Ft(,)h(except)f(that)g(all)h(pro-)75 1158 y(cesses)10 b(in)h(the)g(comm)o(unicator)f(group)f(asso)q(ciated)i(with)f(the)g (\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)i Fq(MPI)p 1647 1158 V 16 w(IWRITE)p 1819 1158 V 16 w(NEXT)p 1955 1158 V 17 w(ALL)p Ft(.)75 1280 y Fl(10.4.5)49 b(Seek)16 b(\(indep)q(endent\))75 1460 y Fq(MPI)p 160 1460 V 16 w(SEEK\(fh,)f(o\013set,)g(whence\))117 1537 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1612 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(o\013set\)) 117 1687 y(IN)155 b Fq(whence)450 b Ff(Up)q(date)15 b(mo)q(de)e(\(in)o (teger\))75 1812 y Fp(int)23 b(MPI)p 245 1812 15 2 v 17 w(Seek\(MPI)p 454 1812 V 16 w(File)h(fh,)f(MPI)p 757 1812 V 17 w(Offset)g(offset,)g(MPI)p 1204 1812 V 17 w(Whence)g (whence\))75 1898 y(MPI)p 150 1898 V 17 w(SEEK\(FH,)g(OFFSET,)f (WHENCE\))170 1955 y(INTEGER)h(FH,)h(WHENCE)170 2011 y(INTEGER*8)f(OFFSET)166 2098 y Fq(MPI)p 251 2098 14 2 v 16 w(SEEK)17 b Ft(up)q(dates)h(the)g(individual)i(\014le)f(p)q(oin) o(ter)f(according)g(to)e Fq(whence)p Ft(,)k(whic)o(h)e(could)g(ha)o(v)o (e)75 2154 y(the)d(follo)o(wing)h(p)q(ossible)h(v)m(alues:)143 2248 y Fh(\017)23 b Fg(MPI)p 266 2248 13 2 v 14 w(SEEK)p 382 2248 V 14 w(SET)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)f Fq(o\013set)143 2342 y Fh(\017)23 b Fg(MPI)p 266 2342 V 14 w(SEEK)p 382 2342 V 14 w(CUR)p Ft(:)14 b(the)h(p)q(oin)o(ter)h(is) g(set)f(to)f(the)h(curren)o(t)h(p)q(oin)o(ter)f(p)q(osition)i(plus)f Fq(o\013set)143 2435 y Fh(\017)23 b Fg(MPI)p 266 2435 V 14 w(SEEK)p 382 2435 V 14 w(END)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is) h(set)f(to)g(the)g(end)h(of)e(the)i(\014le)g(plus)g Fq(o\013set)166 2529 y Ft(The)c Fq(o\013set)g Ft(is)g(relativ)o(e)h(to)e(the)h(\014let) o(yp)q(e)h(\(not)e(coun)o(ting)i(holes\),)f(in)h(units)f(of)g Fq(et)o(yp)q(e)p Ft(.)20 b(The)12 b(constan)o(t)75 2586 y Fg(MPI)p 152 2586 V 14 w(OFFSET)p 321 2586 V 14 w(ZERO)j Ft(is)h(de\014ned)g(to)f(b)q(e)h(o\013set)e(zero.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 27 28 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(27)75 49 y Fm(10.5)60 b(Data)19 b(Access)h(with)g(Sha)n(red)f(File)g(P)n(ointers)75 233 y Fe(Discussion:)c Ff(There)f(is)e(some)g(con)o(tro)o(v)o(ersy)h (as)g(to)f(whether)i(w)o(e)f(should)f(k)o(eep)i(shared)f(\014le)g(p)q (oin)o(ters,)g(and)f(if)g(w)o(e)75 290 y(do,)h(the)i(exact)f(seman)o (tics)g(of)f(shared)i(\014le)f(p)q(oin)o(ter)g(op)q(erations.)166 429 y Ft(The)g(system)f(main)o(tains)h(exactly)g(one)g(shared)g(\014le) h(p)q(oin)o(ter)f(p)q(er)g(collectiv)o(e)i Fq(MPI)p 1580 429 14 2 v 16 w(OPEN)e Ft(\(shared)75 485 y(among)19 b(pro)q(cesses)h(in)h(the)f(comm)o(unicator)f(group\).)33 b(The)20 b(curren)o(t)g(v)m(alue)h(of)e(this)h(p)q(oin)o(ter)g(can)g(b) q(e)75 542 y(used)h(to)f(implicitly)j(sp)q(ecify)f(the)e(o\013set)g(of) g(data)f(accesses,)j(b)o(y)e(using)i(the)e(routines)h(describ)q(ed)h (in)75 598 y(this)16 b(section.)k(These)c(routines)g(only)g(use)f(and)h (up)q(date)g(the)f(shared)g(\014le)i(p)q(oin)o(ter)e(main)o(tained)i(b) o(y)e(the)75 655 y(system.)k(The)d(individual)i(\014le)e(p)q(oin)o (ters)g(are)f(not)g(used)h(nor)f(up)q(dated.)166 711 y(The)c(shared)g(\014le)i(p)q(oin)o(ter)e(routines)h(ha)o(v)o(e)e(the)i (same)e(seman)o(tics)i(as)e(the)i(data)e(access)h(with)h(explicit)75 768 y(o\013set)i(routines)i(describ)q(ed)h(in)f(Section)g(10.3)e(with)i (the)f(follo)o(wing)h(mo)q(di\014cations:)143 862 y Fh(\017)23 b Ft(the)f Fq(o\013set)h Ft(is)g(de\014ned)h(to)e(b)q(e)i(the)e(curren) o(t)h(v)m(alue)h(of)e(the)h(system-main)o(tained)g(shared)g(\014le)189 918 y(p)q(oin)o(ter,)143 1012 y Fh(\017)g Ft(the)14 b(e\013ect)g(of)f (m)o(ultiple)j(calls)g(to)d(shared)h(\014le)h(p)q(oin)o(ter)g(routines) g(is)f(de\014ned)i(to)d(b)q(eha)o(v)o(e)i(as)e(if)i(the)189 1068 y(calls)h(w)o(ere)f(serialized,)i(and)143 1162 y Fh(\017)23 b Ft(the)17 b(use)h(of)e(shared)i(\014le)g(p)q(oin)o(ter)g (routines)f(is)h(erroneous)f(unless)i(all)f(pro)q(cesses)f(use)h(the)f (same)189 1219 y Fq(\014let)o(yp)q(e)f Ft(when)g(op)q(ening)g(the)g (\014le.)75 1312 y(The)e(serialization)h(ordering)e(is)h(not)f (deterministic.)21 b(The)14 b(user)f(needs)i(to)d(use)i(other)f(sync)o (hronization)75 1369 y(means)i(to)g(enforce)g(a)g(sp)q(eci\014c)i (order.)166 1425 y(After)j(a)h(shared)g(\014le)g(p)q(oin)o(ter)h(op)q (eration)f(is)g(initiated,)i(the)e(shared)g(\014le)h(p)q(oin)o(ter)f (is)g(up)q(dated)75 1482 y(to)d(p)q(oin)o(t)h(to)f(the)h(next)g(data)f (item)h(after)f(the)g(last)h(one)g(that)f(will)i(b)q(e)f(actually)h (accessed,)f(p)q(ossibly)75 1538 y(accoun)o(ting)d(for)e(an)h(end)h(of) f(\014le)h(o)q(ccurrence.)189 1644 y Fk(A)n(dvic)n(e)i(to)i(users.)60 b Ft(In)20 b(order)e(to)g(ensure)h(that)f(data)g(access)h(using)g(the)g (shared)g(\014le)g(p)q(oin)o(ter)189 1701 y(is)g(w)o(ell)h(de\014ned,)h (all)e(pro)q(cesses)h(m)o(ust)e(use)h(the)g(same)g Fq(\014let)o(yp)q(e) h Ft(when)g(op)q(ening)g(the)f(\014le.)32 b(F)l(or)189 1757 y(example,)14 b(consider)g(a)f(single)h Fq(MPI)p 798 1757 V 16 w(OPEN)g Ft(in)g(whic)o(h)g(pro)q(cess)f(1)g(uses)h (\014let)o(yp)q(e)g Fq(MPI)p 1655 1757 V 16 w(CHAR)g Ft(and)189 1814 y(pro)q(cess)k(2)f(uses)h(\014let)o(yp)q(e)h Fq(MPI)p 737 1814 V 16 w(DOUBLE)p Ft(.)f(If)g(pro)q(cess)g(1)g(reads)f (a)h(single)h(c)o(haracter)e(from)g(the)189 1870 y(\014le,)i(the)f (shared)g(\014le)g(p)q(oin)o(ter)h(p)q(oin)o(ts)f(to)f(the)h(second)g (c)o(haracter)f(relativ)o(e)i(to)e(pro)q(cess)h(1,)g(but)189 1927 y(p)q(oin)o(ts)13 b(in)o(to)f(the)h(middle)h(of)e(the)h(\014rst)f (double)i(relativ)o(e)f(to)f(pro)q(cess)h(2.)18 b(A)o(t)12 b(this)h(p)q(oin)o(t,)h(the)e(e\013ect)189 1983 y(of)i(a)h(shared)h(p)q (oin)o(ter)f(op)q(eration)h(on)f(pro)q(cess)g(2)g(is)h(ill-de\014ned.) 23 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(users.)p Ft(\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 28 30 28 29 bop 75 -100 a Ft(28)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.5.1)49 b(Indep)q(endent)16 b(Op)q(erations)75 135 y Fq(Blo)q(cking)g(Read)g(\(indep)q(endent)q(\))75 315 y(MPI)p 160 315 14 2 v 16 w(READ)p 295 315 V 17 w(SHARED\(fh,)g (bu\013,)f(buft)o(yp)q(e,)i(bufcount,)g(status\))117 393 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 468 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 543 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 618 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 693 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 818 y Fp(int)23 b(MPI)p 245 818 15 2 v 17 w(Read)p 358 818 V 17 w(shared\(MPI)p 615 818 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1204 818 V 17 w(Datatype)f(buftype,)393 874 y(int)i(bufcount,)e(MPI)p 802 874 V 17 w(Status)h(*status\))75 960 y(MPI)p 150 960 V 17 w(READ)p 263 960 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(STATUS,)h(IERROR\))170 1017 y()g(BUFF\(*\))170 1073 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1073 V 16 w(STATUS)p 1313 1073 V 16 w(SIZE\),)h(IERROR)166 1160 y Fq(MPI)p 251 1160 14 2 v 16 w(READ)p 386 1160 V 17 w(SHARED)15 b Ft(is)f(seman)o(tically)i(iden)o(tical)g(to)d Fq(MPI)p 1216 1160 V 16 w(READ)p Ft(,)h(with)h Fq(o\013set)f Ft(set)g(to)f(the)i(cur-)75 1216 y(ren)o(t)g(shared)g(\014le)i(p)q(oin) o(ter)f(p)q(osition)g(main)o(tained)g(b)o(y)g(the)f(system.)20 b(Shared)c(\014le)g(p)q(oin)o(ter)g(calls)h(within)75 1273 y(a)e(single)i(\014le)g(handle)g(comm)o(unicator)e(group)h(are)f (guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially)g(consisten)o(t.)22 b(The)75 1329 y(\014le)16 b(p)q(oin)o(ter)g(is)g(up)q(dated)g(b)o(y)f (the)g(amoun)o(t)f(of)h(data)g(requested.)75 1449 y Fq(Blo)q(cking)h(W) o(rite)f(\(indep)q(endent\))75 1630 y(MPI)p 160 1630 V 16 w(WRITE)p 319 1630 V 16 w(SHARED\(fh,)h(bu\013,)f(buft)o(yp)q(e,)i (bufcount,)g(status\))117 1707 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1782 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's) f(bu\013er)h(\(in)o(teger\))117 1857 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1932 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2007 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c (\(Status\))75 2132 y Fp(int)23 b(MPI)p 245 2132 15 2 v 17 w(Write)p 382 2132 V 17 w(shared\(MPI)p 639 2132 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1228 2132 V 16 w(Datatype)g(buftype,)393 2188 y(int)h(bufcount,)e(MPI)p 802 2188 V 17 w(Status)h(*status\))75 2275 y(MPI)p 150 2275 V 17 w(WRITE)p 287 2275 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(STATUS,)h(IERROR\))170 2331 y()g(BUFF\(*\))170 2388 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2388 V 16 w(STATUS)p 1313 2388 V 16 w(SIZE\),)h(IERROR)166 2474 y Fq(MPI)p 251 2474 14 2 v 16 w(WRITE)p 410 2474 V 16 w(SHARED)d Ft(is)f(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 1262 2474 V 16 w(WRITE)p Ft(,)g(with)h Fq(o\013set)g Ft(set)g(to)f(the)75 2531 y(curren)o(t)10 b(shared)h(\014le)g(p)q(oin)o (ter)g(p)q(osition)g(main)o(tained)g(b)o(y)f(the)h(system.)18 b(Shared)10 b(\014le)i(p)q(oin)o(ter)e(calls)i(within)75 2587 y(a)j(single)i(\014le)g(handle)g(comm)o(unicator)e(group)h(are)f (guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially)g(consisten)o(t.)22 b(The)75 2643 y(\014le)16 b(p)q(oin)o(ter)g(is)g(up)q(dated)g(b)o(y)f (the)g(amoun)o(t)f(of)h(data)g(requested.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 29 31 29 30 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(29)75 49 y Fq(Nonblo)q(cking)16 b(Read)g(\(indep)q(enden)q(t\))75 230 y(MPI)p 160 230 14 2 v 16 w(IREAD)p 308 230 V 16 w(SHARED\(fh,)g(bu\013,)g(buft)o(yp)q (e,)g(bufcount,)h(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i (user's)f(bu\013er)h(\(in)o(teger\))117 457 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 532 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 607 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 732 y Fp(int)23 b(MPI)p 245 732 15 2 v 17 w(Iread)p 382 732 V 17 w(shared\(MPI)p 639 732 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1228 732 V 16 w(Datatype)g (buftype,)393 788 y(int)h(bufcount,)e(MPI)p 802 788 V 17 w(Request)h(*request\))75 875 y(MPI)p 150 875 V 17 w(IREAD)p 287 875 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f (REQUEST,)h(IERROR\))170 931 y()g(BUFF\(*\))170 988 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 1074 y Fq(MPI)p 251 1074 14 2 v 16 w(IREAD)p 399 1074 V 16 w(SHARED)f Ft(is)g(seman)o(tically)g(iden)o(tical)g(to)e Fq(MPI)p 1262 1074 V 16 w(IREAD)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)f(the)75 1130 y(curren)o(t)h(shared)g(\014le)h(p)q(oin)o (ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)37 b(Shared)21 b(\014le)h(p)q(oin)o(ter)g(calls)75 1187 y(within)13 b(a)f(single)i(\014le)f(handle)g(comm)o(unicator)f(group)g (are)g(guaran)o(teed)g(to)f(b)q(e)i(sequen)o(tially)h(consisten)o(t.)75 1243 y(The)h(\014le)i(p)q(oin)o(ter)e(is)h(up)q(dated)g(b)o(y)f(the)h (amoun)o(t)e(of)h(data)f(requested.)75 1363 y Fq(Nonblo)q(cking)i(W)o (rite)f(\(indep)q(enden)q(t\))75 1544 y(MPI)p 160 1544 V 16 w(IWRITE)p 332 1544 V 16 w(SHARED\(fh,)h(bu\013,)f(buft)o(yp)q(e,) i(bufcount,)g(request\))117 1621 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1696 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's) f(bu\013er)h(\(in)o(teger\))117 1771 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1846 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 1922 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2046 y Fp(int)23 b(MPI)p 245 2046 15 2 v 17 w(Iwrite)p 406 2046 V 17 w(shared\(MPI)p 663 2046 V 15 w(File)h(fh,)f(void)g(*buff,)h(MPI)p 1252 2046 V 16 w(Datatype)f(buftype,)393 2102 y(int)h(bufcount,)e(MPI)p 802 2102 V 17 w(Request)h(*request\))75 2189 y(MPI)p 150 2189 V 17 w(IWRITE)p 311 2189 V 16 w(SHARED\(FH,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\))170 2245 y()g(BUFF\(*\))170 2302 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (REQUEST,)h(IERROR)166 2388 y Fq(MPI)p 251 2388 14 2 v 16 w(IWRITE)p 423 2388 V 16 w(SHARED)17 b Ft(is)g(seman)o(tically)g (iden)o(tical)h(to)d Fq(MPI)p 1262 2388 V 16 w(IWRITE)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)g(the)75 2445 y(curren)o(t)10 b(shared)h(\014le)g(p)q(oin)o(ter)g(p)q(osition)g(main) o(tained)g(b)o(y)f(the)h(system.)18 b(Shared)10 b(\014le)i(p)q(oin)o (ter)e(calls)i(within)75 2501 y(a)j(single)i(\014le)g(handle)g(comm)o (unicator)e(group)h(are)f(guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially) g(consisten)o(t.)22 b(The)75 2558 y(\014le)16 b(p)q(oin)o(ter)g(is)g (up)q(dated)g(b)o(y)f(the)g(amoun)o(t)f(of)h(data)g(requested.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 30 32 30 31 bop 75 -100 a Ft(30)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.5.2)49 b(Collective)18 b(Op)q(erations)75 135 y Ft(Unlik)o(e)h(other)e(collectiv)o(e)j(op)q(erations,)e(a)f (barrier)h(sync)o(hronization)g(\(among)f(the)g(pro)q(cesses)h(asso)q (ci-)75 192 y(ated)g(with)g(the)g(comm)o(unicator)f(in)o(v)o(olv)o(ed)i (in)g(the)f(op)q(eration\))g(is)g(enforced)g(for)g(shared)g(\014le)g(p) q(oin)o(ter)75 248 y(op)q(erations.)166 304 y(The)12 b(seman)o(tics)g(of)f(a)g(collectiv)o(e)j(access)d(using)i(a)e(shared)h (\014le)h(p)q(oin)o(ter)f(is)g(that)f(the)h(accesses)g(to)f(the)75 361 y(\014le)i(will)h(b)q(e)f(in)g(the)f(order)g(determined)h(b)o(y)f (the)g(ranks)g(of)g(the)g(pro)q(cesses)g(within)i(the)e(group)g(asso)q (ciated)75 417 y(with)k(the)g(comm)o(unicator.)22 b(In)16 b(addition,)h(in)g(order)e(to)h(prev)o(en)o(t)f(subsequen)o(t)i(shared) f(o\013set)f(accesses)75 474 y(b)o(y)c(the)h(same)e(pro)q(cesses)i(to)f (in)o(terfere)g(with)h(this)g(collectiv)o(e)h(access,)e(the)h(call)g (can)g(only)f(return)h(after)e(all)75 530 y(the)15 b(pro)q(cesses)g (within)h(the)f(group)g(ha)o(v)o(e)f(initiated)j(their)e(accesses.)20 b(In)c(other)e(terms,)h(this)g(seman)o(tics)75 587 y(implies)j(a)d (barrier)h(sync)o(hronization)g(among)f(all)h(the)g(pro)q(cesses)g(of)f (the)h(group,)f(that)f(tak)o(es)h(place)i(at)75 643 y(the)h(outset)f (of)h(a)g(blo)q(c)o(king)h(op)q(eration)f(or)g(at)f(the)h(initiation)i (of)d(a)h(non)o(blo)q(c)o(king)h(op)q(eration.)29 b(When)75 700 y(the)15 b(call)i(returns,)e(the)g(shared)g(\014le)i(p)q(oin)o(ter) e(p)q(oin)o(ts)h(to)f(the)g(next)h(b)o(yte)f(accessible,)h(according)g (to)f(the)75 756 y(\014let)o(yp)q(e)h(used)g(b)o(y)f(all)h(pro)q (cesses,)g(after)e(the)h(last)g(b)o(yte)g(requested.)75 873 y Fq(Blo)q(cking)h(Read)g(\(collective\))75 1053 y(MPI)p 160 1053 14 2 v 16 w(READ)p 295 1053 V 17 w(SHARED)p 488 1053 V 17 w(ALL\(fh,)f(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (status\))117 1130 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 1202 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 1274 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1346 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 1417 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c (\(Status\))75 1542 y Fp(int)47 b(MPI)p 269 1542 15 2 v 17 w(Read)p 382 1542 V 17 w(shared)p 543 1542 V 16 w(all\(MPI)p 727 1542 V 16 w(File)24 b(fh,)f(void)g(*buff,)g(MPI)p 1316 1542 V 17 w(Datatype)g(buftype,)393 1598 y(int)h(bufcount,)e(MPI)p 802 1598 V 17 w(Status)h(*status\))75 1685 y(MPI)p 150 1685 V 17 w(READ)p 263 1685 V 16 w(SHARED)p 423 1685 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(STATUS,)h(IERROR\))170 1741 y()g(BUFF\(*\))170 1798 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 1798 V 16 w(STATUS)p 1313 1798 V 16 w(SIZE\),)h(IERROR)166 1884 y Fq(MPI)p 251 1884 14 2 v 16 w(READ)p 386 1884 V 17 w(SHARED)p 579 1884 V 17 w(ALL)17 b Ft(is)g(a)g(collectiv)o(e)h(v)o(ersion)g(of)e (the)h Fq(MPI)p 1350 1884 V 16 w(READ)p 1485 1884 V 17 w(SHARED)h Ft(in)o(terface.)75 1940 y(This)i(routine)f(m)o(ust)g(b)q(e) h(called)h(b)o(y)e(all)h(pro)q(cesses)f(in)h(the)g(comm)o(unicator)e (group)h(asso)q(ciated)h(with)75 1997 y(the)f(\014le)g(handle)h Fq(fh)p Ft(.)30 b(Eac)o(h)18 b(pro)q(cess)h(ma)o(y)f(pass)g(di\013eren) o(t)h(argumen)o(t)f(v)m(alues)h(for)f(the)h Fq(buft)o(yp)q(e)p Ft(,)i(and)75 2053 y Fq(bufcount)16 b Ft(argumen)o(ts.)i(After)c(all)g (pro)q(cesses)g(of)f(the)h(group)f(ha)o(v)o(e)h(issued)g(their)h(resp)q (ectiv)o(e)f(calls,)h(eac)o(h)75 2110 y(pro)q(cess)h(attempts)e(to)g (read,)h(from)g(the)g(\014le)i(asso)q(ciated)e(with)h Fq(fh)p Ft(,)f(a)g(total)g(n)o(um)o(b)q(er)g(of)g Fq(bufcount)j Ft(data)75 2166 y(items)e(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)24 b(F)l(or)15 b(eac)o(h)h(pro)q(cess,)h(the)f(lo)q(cation)h(in)g (the)75 2223 y(\014le)g(at)d(whic)o(h)j(data)e(is)h(read)f(is)h(the)g (p)q(osition)g(at)f(whic)o(h)h(the)g(shared)g(\014le)g(p)q(oin)o(ter)g (w)o(ould)g(b)q(e)g(after)f(all)75 2279 y(pro)q(cesses)k(whose)g(ranks) f(within)i(the)e(group)h(are)f(less)h(than)g(that)f(of)g(this)h(pro)q (cess)g(had)g(read)f(their)75 2336 y(data.)h Fq(MPI)p 281 2336 V 15 w(READ)p 415 2336 V 17 w(SHARED)p 608 2336 V 18 w(ALL)13 b Ft(returns)g(the)h(n)o(um)o(b)q(er)f(of)g Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)e(read)f(in)h Fq(status)p Ft(.)22 b(The)75 2392 y(shared)f(\014le)g(p)q(oin)o(ter)g(is)g(up)q (dated)h(b)o(y)e(the)h(amoun)o(ts)f(of)g(data)g(requested)h(b)o(y)f (all)i(pro)q(cesses)f(of)f(the)75 2449 y(group.)189 2534 y Fk(A)n(dvic)n(e)14 b(to)h(implementors.)39 b Ft(Accesses)15 b(to)e(the)h(data)f(requested)i(b)o(y)f(all)h(pro)q(cesses)f(do)g(not)f (ha)o(v)o(e)189 2591 y(to)f(b)q(e)i(serialized.)21 b(Once)14 b(all)g(pro)q(cesses)f(ha)o(v)o(e)g(issued)h(their)f(requests,)g(lo)q (cations)h(within)g(the)f(\014le)189 2647 y(for)i(all)i(accesses)g(can) f(b)q(e)h(computed,)g(and)f(accesses)h(can)f(pro)q(ceed)h(indep)q (enden)o(tly)i(from)d(eac)o(h)189 2704 y(other,)e(p)q(ossibly)j(in)f (parallel.)21 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 31 33 31 32 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(31)75 49 y Fq(Blo)q(cking)16 b(W)o(rite)f(\(collective\))75 230 y(MPI)p 160 230 14 2 v 16 w(WRITE)p 319 230 V 16 w(SHARED)p 511 230 V 18 w(ALL\(fh,)f(bu\013,)h(buft)o(yp)q(e,)i(bufcount,)g(status\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i (handle)g(\(handle\))117 381 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 455 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 530 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 604 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 728 y Fp(int)47 b(MPI)p 269 728 15 2 v 17 w(Write)p 406 728 V 17 w(shared)p 567 728 V 16 w(all\(MPI)p 751 728 V 16 w(File)23 b(fh,)h(void)f(*buff,)g(MPI)p 1340 728 V 17 w(Datatype)g(buftype,)393 785 y(int)h(bufcount,)e(MPI)p 802 785 V 17 w(Status)h(*status\))75 871 y(MPI)p 150 871 V 17 w(WRITE)p 287 871 V 16 w(SHARED)p 447 871 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)f(BUFCOUNT,)h(STATUS,)g(IERROR\))170 928 y()g(BUFF\(*\))170 984 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 984 V 16 w(STATUS)p 1313 984 V 16 w(SIZE\),)h(IERROR)166 1071 y Fq(MPI)p 251 1071 14 2 v 16 w(WRITE)p 410 1071 V 16 w(SHARED)p 602 1071 V 18 w(ALL)12 b Ft(is)i(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the)g (blo)q(c)o(king)i Fq(MPI)p 1526 1071 V 16 w(WRITE)p 1685 1071 V 16 w(SHARED)75 1127 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f (in)g(the)f(comm)o(unicator)g(group)g(asso)q(ciated)h(with)g(the)f (\014le)h(handle)h Fq(fh)f Ft(m)o(ust)75 1183 y(call)k Fq(MPI)p 247 1183 V 16 w(WRITE)p 406 1183 V 16 w(SHARED)p 598 1183 V 18 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)g(ma)o(y)g(pass)g (di\013eren)o(t)g(argumen)o(t)f(v)m(alues)i(for)f(the)75 1240 y Fq(buft)o(yp)q(e)13 b Ft(and)f Fq(bufcount)h Ft(argumen)o(ts.)18 b(After)11 b(all)h(pro)q(cesses)g(of)e(the)i(group)f(ha)o(v)o(e)g (issued)h(their)g(resp)q(ectiv)o(e)75 1296 y(calls,)22 b(eac)o(h)d(pro)q(cess)h(attempts)f(to)g(write,)i(in)o(to)e(the)h (\014le)h(asso)q(ciated)f(with)g Fq(fh)p Ft(,)h(a)e(total)g(n)o(um)o(b) q(er)h(of)75 1353 y Fq(bufcount)13 b Ft(data)e(items)g(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(con)o(tained)h(in)g(the)f(user's)g (bu\013er)g Fq(bu\013)p Ft(.)19 b(F)l(or)11 b(eac)o(h)g(pro)q(cess,)75 1409 y(the)g(lo)q(cation)h(in)h(the)e(\014le)h(at)f(whic)o(h)h(data)f (is)h(written)f(is)h(the)f(p)q(osition)h(at)f(whic)o(h)h(the)g(shared)f (\014le)h(p)q(oin)o(ter)75 1466 y(w)o(ould)j(b)q(e)g(after)e(all)i(pro) q(cesses)g(whose)f(ranks)g(within)h(the)g(group)f(are)g(less)h(than)f (that)f(of)h(this)h(pro)q(cess)75 1522 y(had)g(written)g(their)g(data.) k Fq(MPI)p 636 1522 V 16 w(WRITE)p 795 1522 V 17 w(SHARED)p 988 1522 V 17 w(ALL)14 b Ft(returns)h(the)g(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)h Ft(elemen)o(ts)75 1579 y(written)g(in)h Fq(status)p Ft(.)26 b(The)16 b(shared)h(\014le)g(p)q(oin)o(ter)g(is)g (up)q(dated)g(b)o(y)f(the)g(amoun)o(ts)g(of)g(data)f(requested)i(b)o(y) 75 1635 y(all)f(pro)q(cesses)g(of)e(the)i(group.)189 1736 y Fk(A)n(dvic)n(e)e(to)h(implementors.)39 b Ft(Accesses)15 b(to)e(the)h(data)f(requested)i(b)o(y)f(all)h(pro)q(cesses)f(do)g(not)f (ha)o(v)o(e)189 1792 y(to)f(b)q(e)i(serialized.)21 b(Once)14 b(all)g(pro)q(cesses)f(ha)o(v)o(e)g(issued)h(their)f(requests,)g(lo)q (cations)h(within)g(the)f(\014le)189 1849 y(for)i(all)i(accesses)g(can) f(b)q(e)h(computed,)g(and)f(accesses)h(can)f(pro)q(ceed)h(indep)q (enden)o(tly)i(from)d(eac)o(h)189 1905 y(other,)e(p)q(ossibly)j(in)f (parallel.)21 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))75 2025 y Fq(Nonblo)q(cking)f(Read)g(\(collective\))75 2205 y(MPI)p 160 2205 V 16 w(IREAD)p 308 2205 V 16 w(SHARED)p 500 2205 V 18 w(ALL\(fh,)e(bu\013,)i(buft)o(yp)q(e,)g(bufcount,)h (request\))117 2282 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 2356 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 2431 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 2505 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2579 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 2704 y Fp(int)23 b(MPI)p 245 2704 15 2 v 17 w(Iread)p 382 2704 V 17 w(shared)p 543 2704 V 16 w(all\(MPI)p 727 2704 V 16 w(File)h(fh,)f(void)g(*buff,)g(MPI)p 1316 2704 V 17 w(Datatype)g(buftype,)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 32 34 32 33 bop 75 -100 a Ft(32)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)393 49 y Fp(int)24 b(bufcount,)e(MPI)p 802 49 15 2 v 17 w(Request)h(*request\))75 136 y(MPI)p 150 136 V 17 w(IREAD)p 287 136 V 16 w(SHARED)p 447 136 V 17 w(ALL\(FH,)g(BUFF,) g(BUFTYPE,)f(BUFCOUNT,)h(REQUEST,)g(IERROR\))170 192 y()g(BUFF\(*\))170 249 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (REQUEST,)h(IERROR)166 335 y Fq(MPI)p 251 335 14 2 v 16 w(IREAD)p 399 335 V 16 w(SHARED)p 591 335 V 18 w(ALL)9 b Ft(is)i(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the)g(non)o(blo)q(c)o (king)i Fq(MPI)p 1563 335 V 15 w(IREAD)p 1710 335 V 17 w(SHARED)75 392 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f(in)g(the)f (comm)o(unicator)g(group)g(asso)q(ciated)h(with)g(the)f(\014le)h (handle)h Fq(fh)f Ft(m)o(ust)75 448 y(call)21 b Fq(MPI)p 248 448 V 16 w(IREAD)p 396 448 V 17 w(SHARED)p 589 448 V 17 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)h(ma)o(y)e(pass)h(di\013eren)o (t)h(argumen)o(t)e(v)m(alues)i(for)f(the)75 504 y Fq(buft)o(yp)q(e)e Ft(and)e Fq(bufcount)i Ft(argumen)o(ts.)j(After)15 b(all)i(pro)q (cesses)f(of)g(the)g(group)f(ha)o(v)o(e)h(issued)h(their)f(resp)q(ec-) 75 561 y(tiv)o(e)i(calls,)g(eac)o(h)g(pro)q(cess)f(attempts)g(to)f (read,)i(from)f(the)g(\014le)i(asso)q(ciated)e(with)h Fq(fh)p Ft(,)g(a)f(total)g(n)o(um)o(b)q(er)75 617 y(of)i Fq(bufcount)i Ft(data)d(items)h(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)32 b(F)l(or)18 b(eac)o(h)h(pro)q(cess,)75 674 y(the)f(lo)q(cation) g(in)g(the)g(\014le)h(at)e(whic)o(h)h(data)f(will)i(b)q(e)f(read)g(is)g (the)g(p)q(osition)g(at)f(whic)o(h)i(the)e(shared)h(\014le)75 730 y(p)q(oin)o(ter)d(w)o(ould)g(b)q(e)g(after)e(all)j(pro)q(cesses)e (whose)h(ranks)f(within)h(the)g(group)f(are)g(less)h(than)f(that)g(of)g (this)75 787 y(pro)q(cess)j(had)f(read)h(their)g(data.)23 b Fq(MPI)p 747 787 V 15 w(IREAD)p 894 787 V 17 w(SHARED)p 1087 787 V 18 w(ALL)16 b Ft(returns)g(immediately)i(and)f(do)q(es)f (not)75 843 y(w)o(ait)d(for)f(the)i(data)e(to)h(b)q(e)h(presen)o(t)f (in)h(the)f(user's)g(bu\013er.)19 b Fq(MPI)p 1172 843 V 16 w(IREAD)p 1320 843 V 17 w(SHARED)p 1513 843 V 17 w(ALL)13 b Ft(asso)q(ciates)g(an)75 900 y(individual)20 b(request)c(handle)i Fq(request)g Ft(to)d(the)i(I/O)g(request)f(for)g (eac)o(h)g(pro)q(cess.)24 b(The)16 b(request)h(handle)75 956 y(can)i(b)q(e)h(used)f(later)g(b)o(y)g(a)f(pro)q(cess)h(to)g(query) g(the)g(status)f(of)g(its)h(individual)j(read)d(request)g(or)f(w)o(ait) 75 1013 y(for)f(its)i(completion.)29 b(On)19 b(eac)o(h)f(pro)q(cess,)h Fq(MPI)p 916 1013 V 16 w(IREAD)p 1064 1013 V 16 w(SHARED)p 1256 1013 V 18 w(ALL)f Ft(completes)g(when)h(the)f(indi-)75 1069 y(vidual)f(request)e(has)g(completed)i(\(i.e.)j(a)15 b(pro)q(cess)g(do)q(es)h(not)f(ha)o(v)o(e)g(to)g(w)o(ait)f(for)h(all)h (other)f(requests)h(to)75 1125 y(complete\).)21 b(The)16 b(user)g(should)g(not)f(access)h(an)o(y)f(part)g(of)g(the)h(receiving)h (bu\013er)f(after)e(a)i(non)o(blo)q(c)o(king)75 1182 y(read)f(is)g(called,)i(un)o(til)f(the)f(read)g(completes.)20 b(The)15 b(shared)g(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o(y)f (the)g(amoun)o(ts)75 1238 y(of)j(data)f(requested)i(b)o(y)e(all)i(pro)q (cesses)g(of)f(the)g(group.)28 b(F)l(or)17 b(eac)o(h)h(pro)q(cess,)h (the)f(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)75 1295 y Ft(elemen)o(ts)d(actually)h(read)f(b)o(y)f(that)g(pro)q(cess)h(can)g (b)q(e)h(extracted)e(from)g(the)h Fq(MPI)p 1467 1295 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1749 1295 V 16 w(W)l(AIT)75 1351 y Ft(return)h(status,)f(using)i(the)f(routines)h Fq(MPI)p 818 1351 V 16 w(GET)p 922 1351 V 17 w(COUNT)g Ft(or)e Fq(MPI)p 1248 1351 V 16 w(GET)p 1352 1351 V 17 w(ELEMENTS)p Ft(.)75 1468 y Fq(Nonblo)q(cking)i(W)o(rite)f (\(collective\))75 1648 y(MPI)p 160 1648 V 16 w(IWRITE)p 332 1648 V 16 w(SHARED)p 524 1648 V 17 w(ALL\(fh,)g(bu\013,)g(buft)o (yp)q(e,)i(bufcount,)g(request\))117 1725 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\)) 117 1797 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the) i(user's)f(bu\013er)h(\(in)o(teger\))117 1868 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 1940 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2011 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2135 y Fp(int)47 b(MPI)p 269 2135 15 2 v 17 w(Iwrite)p 430 2135 V 16 w(shared)p 590 2135 V 17 w(all\(MPI)p 775 2135 V 16 w(File)23 b(fh,)h(void)f (*buff,)g(MPI)p 1364 2135 V 17 w(Datatype)g(buftype,)393 2192 y(int)h(bufcount,)e(MPI)p 802 2192 V 17 w(Request)h(*request\))75 2278 y(MPI)p 150 2278 V 17 w(IWRITE)p 311 2278 V 16 w(SHARED)p 471 2278 V 16 w(ALL\(FH,)g(BUFF,)h(BUFTYPE,)e(BUFCOUNT,)h(REQUEST,)g (IERROR\))170 2335 y()g(BUFF\(*\))170 2391 y(INTEGER)g(FH,)h (BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 2478 y Fq(MPI)p 251 2478 14 2 v 16 w(IWRITE)p 423 2478 V 16 w(SHARED)p 615 2478 V 17 w(ALL)10 b Ft(is)g(a)g(collectiv)o(e)i(v)o (ersion)f(of)e(the)i(non)o(blo)q(c)o(king)g Fq(MPI)p 1586 2478 V 16 w(IWRITE)p 1758 2478 V 16 w(SHARED)75 2534 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f(in)g(the)f(comm)o (unicator)g(group)g(asso)q(ciated)h(with)g(the)f(\014le)h(handle)h Fq(fh)f Ft(m)o(ust)75 2591 y(call)j Fq(MPI)p 246 2591 V 16 w(IWRITE)p 418 2591 V 16 w(SHARED)p 610 2591 V 17 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)g(ma)o(y)f(pass)h(di\013eren)o(t)g (argumen)o(t)f(v)m(alues)i(for)e(the)75 2647 y Fq(buft)o(yp)q(e)h Ft(and)e Fq(bufcount)i Ft(argumen)o(ts.)j(After)15 b(all)i(pro)q (cesses)f(of)g(the)g(group)f(ha)o(v)o(e)h(issued)h(their)f(resp)q(ec-) 75 2704 y(tiv)o(e)e(calls,)h(eac)o(h)f(pro)q(cess)h(attempts)e(to)g (write,)h(in)o(to)g(the)h(\014le)g(asso)q(ciated)f(with)g Fq(fh)p Ft(,)h(a)e(total)h(n)o(um)o(b)q(er)g(of)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 33 35 33 34 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(33)75 49 y Fq(bufcount)19 b Ft(data)d(items)g(ha)o(ving)h Fq(buft)o(yp)q(e)i Ft(t)o(yp)q(e)d(con)o(tained)h(in)h(the)e(user's)h (bu\013er)f Fq(bu\013)p Ft(.)25 b(F)l(or)15 b(eac)o(h)i(pro-)75 106 y(cess,)c(the)g(lo)q(cation)g(in)g(the)g(\014le)h(at)e(whic)o(h)h (data)f(will)i(b)q(e)g(written)e(is)h(the)g(p)q(osition)h(at)e(whic)o (h)h(the)g(shared)75 162 y(\014le)j(p)q(oin)o(ter)g(w)o(ould)g(b)q(e)g (after)e(all)j(pro)q(cesses)e(whose)h(ranks)f(within)h(the)g(group)f (are)g(less)h(than)f(that)f(of)75 219 y(this)k(pro)q(cess)g(had)g (written)g(their)g(data.)27 b Fq(MPI)p 905 219 14 2 v 16 w(IWRITE)p 1077 219 V 16 w(SHARED)p 1269 219 V 17 w(ALL)18 b Ft(returns)f(immediately)j(and)75 275 y(do)q(es)e(not)g(w)o (ait)f(for)h(the)g(data)f(to)g(b)q(e)i(presen)o(t)f(in)h(the)f(user's)f (bu\013er.)28 b Fq(MPI)p 1417 275 V 16 w(IWRITE)p 1589 275 V 16 w(SHARED)p 1781 275 V 17 w(ALL)75 332 y Ft(asso)q(ciates)18 b(an)h(individual)j(request)c(handle)i Fq(request)g Ft(to)e(the)h(I/O)g (request)f(for)g(eac)o(h)h(pro)q(cess.)30 b(The)75 388 y(request)19 b(handle)h(can)f(b)q(e)g(used)g(later)g(b)o(y)g(a)f(pro)q (cess)h(to)f(query)h(the)g(status)f(of)g(its)h(individual)j(write)75 444 y(request)15 b(or)g(w)o(ait)g(for)g(its)g(completion.)22 b(On)16 b(eac)o(h)f(pro)q(cess,)g Fq(MPI)p 1207 444 V 16 w(IWRITE)p 1379 444 V 16 w(SHARED)p 1571 444 V 17 w(ALL)g Ft(completes)75 501 y(when)i(the)g(individual)j(write)d (request)g(has)f(completed)i(\(i.e.)25 b(a)16 b(pro)q(cess)h(do)q(es)g (not)g(ha)o(v)o(e)f(to)g(w)o(ait)h(for)75 557 y(all)f(other)f(requests) g(to)g(complete\).)20 b(The)c(user)f(should)h(not)f(access)h(an)o(y)f (part)f(of)h(the)g(supplied)j(bu\013er)75 614 y(after)g(a)h(non)o(blo)q (c)o(king)i(write)e(is)g(called,)j(un)o(til)e(the)f(write)g(is)h (completed.)33 b(The)19 b(shared)g(\014le)h(p)q(oin)o(ter)75 670 y(is)d(up)q(dated)h(b)o(y)f(the)f(amoun)o(ts)g(of)h(data)f (requested)h(b)o(y)g(all)g(pro)q(cesses)h(of)e(the)h(group.)24 b(F)l(or)16 b(eac)o(h)h(pro-)75 727 y(cess,)22 b(the)f(n)o(um)o(b)q(er) g(of)f Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f(actually)f(written)g(b)o(y)g (that)f(pro)q(cess)h(can)g(b)q(e)g(extracted)75 783 y(from)c(the)h Fq(MPI)p 351 783 V 16 w(TEST)h Ft(or)e Fq(MPI)p 640 783 V 16 w(W)l(AIT)h Ft(return)g(status,)g(using)g(the)h(routines)f Fq(MPI)p 1542 783 V 16 w(GET)p 1646 783 V 17 w(COUNT)g Ft(or)75 840 y Fq(MPI)p 160 840 V 16 w(GET)p 264 840 V 17 w(ELEMENTS)p Ft(.)75 957 y Fq(Seek)e(\(collective\))75 1137 y(MPI)p 160 1137 V 16 w(SEEK)p 287 1137 V 16 w(SHARED\(fh,)g (o\013set,)f(whence\))117 1214 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 1286 y(IN)155 b Fq(o\013set)484 b Ff([SAME])14 b(File)f(o\013set)i (\(o\013set\))117 1358 y(IN)155 b Fq(whence)450 b Ff([SAME])14 b(Up)q(date)g(mo)q(de)f(\(in)o(teger\))75 1482 y Fp(int)23 b(MPI)p 245 1482 15 2 v 17 w(Seek)p 358 1482 V 17 w(shared\(MPI)p 615 1482 V 16 w(File)g(fh,)g(MPI)p 917 1482 V 17 w(Offset)g(offset,)g (MPI)p 1364 1482 V 17 w(Whence)g(whence\))75 1569 y(MPI)p 150 1569 V 17 w(SEEK)p 263 1569 V 16 w(SHARED\(FH,)g(OFFSET,)g (WHENCE\))170 1625 y(INTEGER)g(FH,)h(WHENCE)170 1682 y(INTEGER*8)f(OFFSET)166 1768 y Fq(MPI)p 251 1768 14 2 v 16 w(SEEK)p 378 1768 V 16 w(SHARED)17 b Ft(up)q(dates)f(the)g (shared)g(\014le)h(p)q(oin)o(ter)f(according)g(to)f Fq(whence)p Ft(,)i(whic)o(h)g(could)75 1824 y(ha)o(v)o(e)e(the)g(follo)o(wing)h(p)q (ossible)h(v)m(alues:)143 1902 y Fh(\017)23 b Fg(MPI)p 266 1902 13 2 v 14 w(SEEK)p 382 1902 V 14 w(SET)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)f Fq(o\013set)143 1989 y Fh(\017)23 b Fg(MPI)p 266 1989 V 14 w(SEEK)p 382 1989 V 14 w(CUR)p Ft(:)14 b(the)h(p)q(oin)o(ter)h(is)g(set)f(to)f(the)h (curren)o(t)h(p)q(oin)o(ter)f(p)q(osition)i(plus)f Fq(o\013set)143 2076 y Fh(\017)23 b Fg(MPI)p 266 2076 V 14 w(SEEK)p 382 2076 V 14 w(END)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)g(the)g (end)h(of)e(the)i(\014le)g(plus)g Fq(o\013set)166 2154 y Ft(All)d(the)f(pro)q(cesses)g(in)g(the)g(comm)o(unicator)g(group)f (asso)q(ciated)h(with)g(the)g(\014le)h(handle)g Fq(fh)f Ft(m)o(ust)f(call)75 2210 y Fq(MPI)p 160 2210 14 2 v 16 w(SEEK)p 287 2210 V 16 w(SHARED)18 b Ft(with)g(the)f(same)g Fq(o\013set)g Ft(and)g Fq(whence)p Ft(.)27 b(All)19 b(pro)q(cesses)e (in)h(the)f(comm)o(unicator)75 2266 y(group)e(are)g(sync)o(hronized)h (with)g(a)f(barrier)g(b)q(efore)h(the)f(shared)g(\014le)h(p)q(oin)o (ter)g(is)g(up)q(dated.)166 2323 y(The)c Fq(o\013set)g Ft(is)g(relativ)o(e)h(to)e(the)h(\014let)o(yp)q(e)h(\(not)e(coun)o (ting)i(holes\),)f(in)h(units)f(of)g Fq(et)o(yp)q(e)p Ft(.)20 b(The)12 b(constan)o(t)75 2379 y Fg(MPI)p 152 2379 13 2 v 14 w(OFFSET)p 321 2379 V 14 w(ZERO)j Ft(is)h(de\014ned)g (to)f(b)q(e)h(o\013set)e(zero.)75 2519 y Fm(10.6)60 b(Filet)n(yp)r(e)19 b(Constructo)n(rs)75 2704 y Fe(Discussion:)14 b Ff(Should)c(w)o(e)h(mo) o(v)o(e)e(these)k(datat)o(yp)q(e)e(constructors)h(to)f(the)h(\\Datat)o (yp)q(e)e(Impro)o(v)o(emen)o(ts")f(section?)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 34 36 34 35 bop 75 -100 a Ft(34)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 254 y Fl(10.6.1)49 b(Intro)q(duction)75 339 y Ft(Common)20 b(I/O)h(op)q(erations)f(\(e.g.,)h(rank-ordered)f(blo)q (c)o(ks,)i(structured)f(arra)o(y)e(accesses,)j(etc.\))35 b(are)75 396 y(easily)20 b(expressed)g(using)g(the)f(previously)h (de\014ned)h(read/write)e(op)q(erations)g(and)g(carefully)i(de\014ned) 75 452 y(\014let)o(yp)q(es.)i(In)17 b(order)f(to)f(simplify)j (generation)e(of)g(common)f(\014let)o(yp)q(es,)i(the)f(follo)o(wing)h Fq(MPI)e Ft(datat)o(yp)q(e)75 509 y(constructors)f(are)h(pro)o(vided.) 166 565 y(Although)c(it)f(is)h(p)q(ossible)h(to)d(implemen)o(t)j(these) e(t)o(yp)q(e)h(constructors)e(as)h(lo)q(cal)h(op)q(erations,)g(in)g (order)75 622 y(to)g(facilitate)h(e\016cien)o(t)g(implemen)o(tations)g (of)f(\014le)h(I/O)f(op)q(erations,)h(most)f(of)f(the)h(\014let)o(yp)q (e)i(constructors)75 678 y(ha)o(v)o(e)j(b)q(een)i(de\014ned)g(to)e(b)q (e)i Fk(c)n(ol)r(le)n(ctive)d Ft(op)q(erations.)24 b(\(Recall)19 b(that)d(a)g(collectiv)o(e)i(op)q(eration)f(do)q(es)g(not)75 735 y(imply)k(a)e(barrier)g(sync)o(hronization.\))33 b(The)20 b(only)g(exception)g(to)f(this)g(is)h(the)g(lo)q(cal)g (sub-arra)o(y)f(t)o(yp)q(e)75 791 y(constructor)c(whic)o(h)i(is)f(lo)q (cal)i(and)e(uses)g(no)g(pro)q(cess)g(rank)g(information.)22 b(Because)17 b(of)e(this,)h(the)g(sub-)75 848 y(arra)o(y)e(constructor) g(can)i(b)q(e)g(used)f(for)g(b)q(oth)g(buft)o(yp)q(es)h(and)f(\014let)o (yp)q(es.)166 904 y(The)j(set)g(of)f(\014let)o(yp)q(es)i(created)f(b)o (y)g(a)g(single)h(collectiv)o(e)h(\014let)o(yp)q(e)f(constructor)e (should)i(b)q(e)f(used)75 960 y(together)c(when)i(op)q(ening)h(a)d (\014le.)166 1017 y(Eac)o(h)20 b(new)h(\014let)o(yp)q(e)h(created)e Fq(newt)o(yp)q(e)j Ft(consists)d(of)h(zero)f(or)g(more)g(copies)i(of)e (the)g(base)h(t)o(yp)q(e)75 1073 y Fq(oldt)o(yp)q(e)p Ft(,)14 b(p)q(ossibly)h(separated)e(b)o(y)g(holes.)20 b(The)13 b(exten)o(t)g(of)f(the)h(new)h(\014let)o(yp)q(e)g(is)g(a)e (nonnegativ)o(e)i(in)o(teger)75 1130 y(m)o(ultiple)j(of)d(the)h(exten)o (t)g(of)f(the)h(base)g(t)o(yp)q(e.)20 b(All)c(\014let)o(yp)q(e)g (constructors)f(return)f(a)h(success)g(or)g(failure)75 1186 y(co)q(de.)75 1308 y Fl(10.6.2)49 b(Scatter)15 b(/)i(Gather)e(T)l (yp)q(e)i(Constructo)o(rs)75 1394 y Fq(MPI)p 160 1394 14 2 v 16 w(T)l(yp)q(e)p 268 1394 V 17 w(scatter)p 412 1394 V 18 w(gather)e(\(collective\))75 1574 y(MPI)p 160 1574 V 16 w(TYPE)p 293 1574 V 17 w(SCA)l(TTER)p 508 1574 V 17 w(GA)l(THER\(comm,)f(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 1651 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e (to)h(b)q(e)i(used)g(in)e(MPI)p 1655 1651 13 2 v 15 w(Op)q(en)i(\(han-) 905 1708 y(dle\))117 1783 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp)q(e)h(\(handle\))117 1858 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f (\(handle\))75 1983 y Fp(int)23 b(MPI)p 245 1983 15 2 v 17 w(Type)p 358 1983 V 17 w(scatter)p 543 1983 V 16 w(gather\(MPI)p 799 1983 V 16 w(Comm)g(comm,)g(MPI)p 1149 1983 V 17 w(Datatype)g(oldtype,)393 2039 y(MPI)p 468 2039 V 17 w(Datatype)g(*newtype\))75 2125 y(MPI)p 150 2125 V 17 w(TYPE)p 263 2125 V 16 w(SCATTER)p 447 2125 V 17 w(GATHER\(COMM,)f(OLDTYPE,)h(NEWTYPE,)f(IERROR\))170 2182 y(INTEGER)h(COMM,)h(OLDTYPE,)e(NEWTYPE,)h(IERROR)166 2268 y Ft(This)18 b(\014let)o(yp)q(e)h(allo)o(ws)f(eac)o(h)g(pro)q (cess)f(in)i(the)f(group)f(to)g(access)h(a)f(distinct)i Fq(blo)q(ck)f Ft(of)g(the)f(\014le)i(in)75 2325 y(rank)14 b(order.)20 b(The)15 b(blo)q(c)o(ks)g(are)f(iden)o(tical)j(in)f(size)f (and)g(datat)o(yp)q(e;)f(eac)o(h)h(item)g(is)g(of)f(datat)o(yp)q(e)g Fq(oldt)o(yp)q(e)p Ft(.)166 2381 y(T)l(o)g(ac)o(hiev)o(e)h(the)f (scatter)g(or)g(gather)f(op)q(eration,)i(the)f(\014les)i(op)q(ened)f (with)g(the)f(returned)h(\014let)o(yp)q(es)75 2438 y(should)f(b)q(e)f (accessed)g(with)g(collectiv)o(e)i(read)e(or)f(write)h(op)q(erations)f (with)h(iden)o(tical)i(o\013sets.)j(Generated)75 2494 y(\014let)o(yp)q(es)e(ma)o(y)f(not)g(b)q(e)g(iden)o(tical,)i(but)e (will)i(ha)o(v)o(e)e(the)g(same)g(exten)o(t.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 35 37 35 36 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(35)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(T)l(yp)q(e)p 268 49 V 17 w(scatterv)p 433 49 V 18 w(gatherv)15 b(\(collective\))75 230 y(MPI)p 160 230 V 16 w(TYPE)p 293 230 V 17 w(SCA)l(TTERV)p 538 230 V 17 w(GA)l(THERV\(comm,)g(count,)h(oldt)o(yp)q(e,)g(newt)o(yp) q(e\))117 307 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to)h(b)q(e)i(used)g(in)e(MPI)p 1655 307 13 2 v 15 w(Op)q(en)i(\(han-)905 363 y(dle\))117 438 y(IN)155 b Fq(count)482 b Ff(n)o(um)o(b)q(er)14 b(of)h(elemen)o(ts) f(of)g(oldt)o(yp)q(e)h(in)g(this)f(blo)q(c)o(k)h(\(nonneg-)905 495 y(ativ)o(e)f(in)o(teger\))117 570 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff(old)13 b(datat)o(yp)q(e)i(\(handle\))117 645 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 770 y Fp(int)23 b(MPI)p 245 770 15 2 v 17 w(Type)p 358 770 V 17 w(scatterv)p 567 770 V 16 w(gatherv\(MPI)p 847 770 V 16 w(Comm)g(comm,)g(int)h(count,)393 826 y(MPI)p 468 826 V 17 w(Datatype)f(oldtype,)f(MPI)p 986 826 V 17 w(Datatype)h(*newtype\))75 912 y(MPI)p 150 912 V 17 w(TYPE)p 263 912 V 16 w(SCATTERV)p 471 912 V 16 w(GATHERV\(COMM,)g (COUNT,)g(OLDTYPE,)f(NEWTYPE,)h(IERROR\))170 969 y(INTEGER)g(COMM,)h (COUNT,)f(OLDTYPE,)f(NEWTYPE,)h(IERROR)166 1055 y Ft(This)16 b(t)o(yp)q(e)f(allo)o(ws)g(eac)o(h)g(pro)q(cess)g(in)h(the)g(group)e (to)h(access)g(a)g(distinct)h Fk(blo)n(ck)e Ft(of)h(the)g(\014le)h(in)g (rank)75 1112 y(order)11 b(as)h(in)g(MPI)p 384 1112 14 2 v 16 w(T)o(yp)q(e)p 502 1112 V 17 w(scatter)p 654 1112 V 16 w(gather,)f(except)h(that)f(the)h(blo)q(c)o(k)g(sizes)g(and)g(t)o (yp)q(es)g(ma)o(y)f(b)q(e)h(di\013eren)o(t.)75 1168 y(Eac)o(h)j(blo)q (c)o(k)h(is)g(de\014ned)g(as)f Fq(count)i Ft(con)o(tiguous)e(copies)h (of)f(the)g(passed)g(datat)o(yp)q(e)g Fq(oldt)o(yp)q(e)p Ft(.)75 1290 y Fl(10.6.3)49 b(Lo)q(cal)18 b(Arra)o(y)e(Filet)o(yp)q(e)g (Constructo)o(r)f(\(indep)q(endent\))75 1470 y Fq(MPI)p 160 1470 V 16 w(TYPE)p 293 1470 V 17 w(SUBARRA)l(Y\(ndims,)i(sizes,)e (subsizes,)i(sta)o(rts,)f(o)o(rder,)e(oldt)o(yp)q(e,)i(newt)o(yp)q(e\)) 117 1547 y Ff(IN)155 b Fq(ndims)476 b Ff(n)o(um)o(b)q(er)13 b(of)h(arra)o(y)f(dimensions)g(\(nonnegativ)o(e)h(in)o(teger\))117 1623 y(IN)155 b Fq(sizes)502 b Ff(size)15 b(of)e(the)i(full)d(arra)o(y) i(in)f(eac)o(h)i(dimension)d(\(arra)o(y)i(of)f(non-)905 1679 y(negativ)o(e)h(in)o(tegers\))117 1754 y(IN)155 b Fq(subsizes)439 b Ff(size)14 b(of)f(the)h(sub-arra)o(y)g(in)f(eac)o (h)h(dimension)e(\(arra)o(y)h(of)g(non-)905 1811 y(negativ)o(e)h(in)o (tegers\))117 1886 y(IN)155 b Fq(sta)o(rts)484 b Ff(starting)18 b(co)q(ordinates)g(of)e(the)i(sub-arra)o(y)g(in)f(eac)o(h)g(dimen-)905 1942 y(sion)d(\(arra)o(y)g(of)f(nonnegativ)o(e)g(in)o(tegers\))117 2017 y(IN)155 b Fq(o)o(rder)490 b Ff(arra)o(y)14 b(storage)g(order)h (\015ag)e(\(in)o(teger\))117 2093 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff(arra)o(y)14 b(elemen)o(t)f(datat)o(yp)q(e)h(\(handle\))117 2168 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f (\(handle\))75 2292 y Fp(int)23 b(MPI)p 245 2292 15 2 v 17 w(Type)p 358 2292 V 17 w(subarray\(int)f(ndims,)h(int)h(*sizes,)e (int)i(*subsizes,)e(int)i(*starts,)393 2349 y(int)g(order,)f(MPI)p 731 2349 V 16 w(Datatype)g(oldtype,)g(MPI)p 1249 2349 V 17 w(Datatype)g(*newtype\))75 2435 y(MPI)p 150 2435 V 17 w(TYPE)p 263 2435 V 16 w(SUBARRAY\(NDIMS,)f(SIZES,)h(SUBSIZES,)g (STARTS,)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)393 2491 y(IERROR\))170 2548 y(INTEGER)g(NDIMS,)g(SIZES\(*\),)g(SUBSIZES\(*\),)f(STARTS\(*\),)h (ORDER,)g(OLDTYPE,)170 2604 y(NEWTYPE,)g(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 36 38 36 37 bop 75 -100 a Ft(36)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)166 49 y Ft(The)23 b(lo)q(cal)h(sub-arra)o(y)e(\014let)o(yp)q(e)i (constructor)e(creates)h(an)g Fq(MPI)f Ft(datat)o(yp)q(e)g(describing)j (an)e(n-)75 106 y(dimensional)c(sub-arra)o(y)c(of)h(a)g(n-dimensional)j (arra)o(y)l(.)k(The)17 b(sub-arra)o(y)e(ma)o(y)h(b)q(e)h(situated)g(an) o(ywhere)75 162 y(within)f(the)f(full)i(arra)o(y)l(,)c(and)j(ma)o(y)e (b)q(e)i(of)e(an)o(y)h(size)h(up)f(to)g(the)g(size)h(of)e(the)h(larger) g(arra)o(y)f(as)h(long)g(as)g(it)75 219 y(is)h(con\014ned)i(within)f (this)f(arra)o(y)l(.)21 b(This)c(t)o(yp)q(e)f(constructor)f (facilitates)i(accessing)g(arra)o(ys)d(distributed)75 275 y(in)i(blo)q(c)o(ks)g(among)e(pro)q(cesses)i(to)f(a)f(single)j (\014le)f(that)f(con)o(tains)g(the)g(global)h(arra)o(y)l(.)166 332 y(This)h(t)o(yp)q(e)f(constructor)f(can)i(handle)g(arra)o(ys)e (with)i(an)f(arbitrary)g(n)o(um)o(b)q(er)g(of)g(dimensions)i(and)75 388 y(w)o(orks)11 b(for)h(b)q(oth)g(C)g(and)h(F)o(OR)l(TRAN)g(ordered)f (matrices)g(\(i.e.,)g(ro)o(w-ma)s(jor)e(or)i(column-ma)s(jor\).)18 b(Note)75 444 y(that)d(a)g(C)g(program)f(ma)o(y)g(use)i(F)o(OR)l(TRAN)g (order)f(and)h(a)f(F)o(OR)l(TRAN)h(program)e(ma)o(y)h(use)g(C)h(order,) 75 501 y(the)e(names)h(are)f(only)g(mean)o(t)g(to)g(sp)q(ecify)i(the)e (default)h(language)f(b)q(eha)o(vior.)20 b(T)l(o)14 b(sp)q(ecify)i (whic)o(h)f(order)75 557 y(the)g(arra)o(y)f(is)i(stored,)e(the)i Fq(o)o(rder)e Ft(argumen)o(t)g(is)i(set)f(to)g(one)g(of)g(the)g(follo)o (wing:)143 645 y Fh(\017)23 b Fg(MPI)p 266 645 13 2 v 14 w(C)p 307 645 V 14 w(ORDER)15 b Ft(-)g(The)g(ordering)h(used)g(b)o (y)f(C)g(arra)o(ys,)e(\(i.e.,)i(ro)o(w-ma)s(jor)e(order\))143 736 y Fh(\017)23 b Fg(MPI)p 266 736 V 14 w(F)o(ORTRAN)p 473 736 V 14 w(ORDER)13 b Ft(-)i(The)f(ordering)h(used)g(b)o(y)f(F)o (OR)l(TRAN)i(arra)o(ys,)c(\(i.e.,)i(column-ma)s(jor)189 793 y(order\))675 1572 y @beginspecial 195 @llx 269 @lly 416 @urx 523 @ury 1440 @rwi @setspecial %%BeginDocument: figures/io-array-file.eps /$F2psDict 32 dict def $F2psDict begin $F2psDict /mtrx matrix put end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 57.000000 681.000000 translate 0.900000 -0.900000 scale % Polyline newpath 163 177 moveto 215 177 lineto 215 380 lineto 163 380 lineto closepath 0.95 setgray gsave fill grestore 0.00 setgray % Polyline newpath 217 177 moveto 268 177 lineto 268 379 lineto 217 379 lineto closepath gsave fill grestore % Polyline newpath 270 177 moveto 321 177 lineto 321 379 lineto 270 379 lineto closepath 0.60 setgray gsave fill grestore 0.00 setgray 1.000 setlinewidth % Polyline newpath 323 178 moveto 374 178 lineto 374 377 lineto 323 377 lineto closepath 0.35 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 294 439 moveto 334 439 lineto 334 458 lineto 294 458 lineto closepath 0.35 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 294 400 moveto 335 400 lineto 335 418 lineto 294 418 lineto closepath 0.60 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 153 438 moveto 195 438 lineto 195 456 lineto 153 456 lineto closepath gsave fill grestore stroke % Polyline newpath 154 400 moveto 194 400 lineto 194 420 lineto 154 420 lineto closepath 0.95 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 216 176 moveto 216 380 lineto stroke % Polyline newpath 268 176 moveto 268 380 lineto stroke % Polyline newpath 322 176 moveto 322 380 lineto stroke 4.000 setlinewidth % Polyline newpath 163 176 moveto 163 379 lineto 375 379 lineto 375 176 lineto closepath stroke /fn0.12 /Times-Roman findfont 13.333334 scalefont def fn0.12 setfont (Process 0) 202 414 moveto 1 -1 scale show 1 -1 scale (Process 2) 348 414 moveto 1 -1 scale show 1 -1 scale (Process 1) 204 452 moveto 1 -1 scale show 1 -1 scale (Process 3) 349 452 moveto 1 -1 scale show 1 -1 scale $F2psEnd %%EndDocument @endspecial 595 1670 a(Figure)h(10.5:)k(Example)d(arra)o(y)e(\014le)i (la)o(y)o(out)166 1772 y(F)l(or)11 b(example,)j(assume)e(w)o(e)f(are)h (writing)h(out)f(a)f(100x100)g(2D)g(arra)o(y)g(of)h(double)h(precision) h(\015oating)75 1828 y(p)q(oin)o(t)i(n)o(um)o(b)q(ers)g(that)e(is)i (distributed)h(among)e(4)g(pro)q(cesses)g(suc)o(h)h(that)f(eac)o(h)g (pro)q(cess)h(has)f(a)g(blo)q(c)o(k)h(of)75 1885 y(25)e(columns)i (\(e.g.,)d(pro)q(cess)i(0)g(has)f(columns)i(0-24,)e(pro)q(cess)h(1)g (has)f(columns)i(25-49,)d(etc.;)i(see)g(Figure)75 1941 y(10.5)f(\).)19 b(T)l(o)c(create)g(the)h(\014let)o(yp)q(es)g(for)e(eac) o(h)i(pro)q(cess)f(one)h(could)g(use)f(the)h(follo)o(wing)g(C)e (program:)147 2040 y Fp(double)23 b(subarray[100][25];)147 2097 y(MPI_Datatype)f(filetype;)147 2153 y(int)h(sizes[2],)g (subsizes[2],)f(starts[2];)147 2209 y(int)h(rank;)147 2322 y(MPI_Comm_rank\(MPI_COMM_)o(WORLD,)d(&rank\);)147 2379 y(sizes[0]=100;)i(sizes[1]=100;)g(subsizes[0]=100;)147 2435 y(subsizes[1]=25;)f(starts[0]=0;)i(starts[1]=rank*25;)147 2548 y(MPI_Type_subarray\(2,)e(sizes,)i(subsizes,)f(starts,)h (MPI_C_ORDER,)600 2605 y(MPI_DOUBLE,)f(&filetype\);)166 2704 y Ft(Or,)15 b(equiv)m(alen)o(tly)i(in)f(F)o(OR)l(TRAN:)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 37 39 37 38 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(37)242 49 y Fp(double)23 b(precision)g(subarray\(100,25\))242 106 y(integer)g(filetype,)g(rank,)g(ierror)242 162 y(integer)g (sizes\(2\),)g(subsizes\(2\),)f(starts\(2\))242 275 y(call)h (MPI_COMM_RANK\(MPI_COMM_WORLD,)d(rank,)j(ierror\))242 332 y(sizes\(1\)=100)242 388 y(sizes\(2\)=100)242 444 y(subsizes\(1\)=100)242 501 y(subsizes\(2\)=25)242 557 y(starts\(1\)=1)242 614 y(starts\(2\)=rank*25)f(+)h(1)242 727 y(call)g(MPI_TYPE_SUBARRAY\(2,)e(sizes,)j(subsizes,)e(starts,)194 783 y($)287 b(MPI_FORTRAN_ORDER,)21 b(MPI_DOUBLE_PRECISION,)194 840 y($)287 b(filetype,)22 b(ierror\))166 946 y Ft(The)c(generated)f (\014let)o(yp)q(e)i(will)g(then)f(describ)q(e)h(the)f(p)q(ortion)g(of)f (the)g(\014le)i(con)o(tained)f(within)h(the)75 1002 y(pro)q(cess')c (subarra)o(y)g(with)g(holes)h(for)f(the)g(space)h(tak)o(en)f(b)o(y)g (the)g(other)g(pro)q(cesses.)21 b(Figure)16 b(10.6)e(sho)o(ws)75 1059 y(the)h(\014let)o(yp)q(e)i(created)e(for)f(pro)q(cess)i(1.)675 1626 y @beginspecial 198 @llx 304 @lly 414 @urx 487 @ury 1440 @rwi @setspecial %%BeginDocument: figures/io-array-p1.eps /$F2psDict 32 dict def $F2psDict begin $F2psDict /mtrx matrix put end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 54.000000 681.000000 translate 0.900000 -0.900000 scale % Polyline newpath 230 217 moveto 269 217 lineto 269 380 lineto 230 380 lineto closepath 0.00 setgray gsave fill grestore 1.000 setlinewidth % Polyline newpath 160 400 moveto 199 400 lineto 199 420 lineto 160 420 lineto closepath gsave fill grestore stroke % Polyline newpath 229 216 moveto 229 381 lineto stroke % Polyline newpath 270 216 moveto 270 381 lineto stroke 4.000 setlinewidth % Polyline newpath 188 216 moveto 188 380 lineto 353 380 lineto 353 216 lineto closepath stroke 5.000 setlinewidth % Polyline newpath 319 398 moveto 358 398 lineto 358 418 lineto 319 418 lineto closepath stroke /fn0.12 /Times-Roman findfont 13.333334 scalefont def fn0.12 setfont (Holes) 370 413 moveto 1 -1 scale show 1 -1 scale (MPI_DOUBLE) 215 415 moveto 1 -1 scale show 1 -1 scale $F2psEnd %%EndDocument @endspecial 453 1724 a(Figure)f(10.6:)k(Example)d(subarra)o(y)e (\014let)o(yp)q(e)i(for)f(pro)q(cess)g(1)75 1895 y Fl(10.6.4)49 b(Distributed)16 b(Arra)o(y)g(Filet)o(yp)q(e)g(Constructo)o(rs)75 1981 y Ft(The)g(distributed)i(arra)o(y)d(\014let)o(yp)q(e)i (constructors)e(create,)h(for)f(eac)o(h)h(pro)q(cess)h(in)g(a)e(group,) h(a)g(\(p)q(ossibly)75 2038 y(di\013eren)o(t\))j(\014let)o(yp)q(e.)34 b(When)20 b(a)g(collectiv)o(e)h(I/O)f(op)q(eration)g(\(with)f(iden)o (tical)j(o\013sets\))c(is)i(p)q(erformed)75 2094 y(on)d(a)f(\014le)i (op)q(ened)g(with)f(the)g(returned)g(\014let)o(yp)q(e,)h(an)f(HPF-lik)o (e)h([4)o(])f(distribution)h(of)f(the)f(\014le)i(data)e(is)75 2150 y(realized.)26 b(Ho)o(w)o(ev)o(er,)16 b(unlik)o(e)i(in)f(HPF,)f (the)h(storage)e(order)i(ma)o(y)f(b)q(e)h(sp)q(eci\014ed)i(for)d(C)g (arra)o(ys)g(as)g(w)o(ell)75 2207 y(as)f(for)f(F)o(OR)l(TRAN)i(arra)o (ys.)75 2327 y Fq(MPI)p 160 2327 14 2 v 16 w(T)l(yp)q(e)p 268 2327 V 17 w(da)o(rra)o(y)e(\(collective\))75 2413 y Ft(Distribution)i(of)f(an)g(N-dimensional)j(arra)o(y)c(on)o(to)g(a)h (P-dimensional)i(grid)f(of)e(logical)j(pro)q(cessors:)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 38 40 38 39 bop 75 -100 a Ft(38)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y\(comm,)14 b(ndims,)i(dsizes,)h(distribs,)g(da)o (rgs,)e(p)q(dims,)h(psizes,)h(o)o(rder,)d(oldt)o(yp)q(e,)j(new-)75 106 y(t)o(yp)q(e\))117 183 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to)h(b)q(e)i(used)g(in)e(MPI)p 1655 183 13 2 v 15 w(Op)q(en)i(\(han-)905 239 y(dle\))117 314 y(IN)155 b Fq(ndims)476 b Ff([SAME])13 b(n)o(um)o(b)q(er)g(of)g (arra)o(y)g(dimensions)f(\(nonnegativ)o(e)i(in-)905 370 y(teger\))117 445 y(IN)155 b Fq(dsizes)479 b Ff([SAME])19 b(size)i(of)e(distributed)i(arra)o(y)e(in)g(eac)o(h)i(dimension)905 501 y(\(arra)o(y)14 b(of)f(nonnegativ)o(e)h(in)o(teger\))117 576 y(IN)155 b Fq(distribs)453 b Ff([SAME])16 b(distribution)g(of)g (arra)o(y)g(in)g(eac)o(h)h(dimension,)e(e.g.)905 632 y(BLOCK)g(or)f(CYCLIC)g(\(arra)o(y)f(of)h(in)o(teger\))117 707 y(IN)155 b Fq(da)o(rgs)487 b Ff([SAME])14 b(distribution)f(argumen) o(t)g(of)g(dimension,)905 763 y(e.g.)25 b(BLOCK\(dargs\))18 b(or)e(CYCLIC\(dargs\))h(\(arra)o(y)f(of)g(in-)905 820 y(teger\))117 894 y(IN)155 b Fq(p)q(dims)475 b Ff([SAME])20 b(n)o(um)o(b)q(er)f(of)h(pro)q(cessor)i(grid)d(dimensions)g(\(non-)905 951 y(negativ)o(e)14 b(in)o(teger\))117 1025 y(IN)155 b Fq(psizes)479 b Ff([SAME])10 b(size)h(of)e(pro)q(cessor)j(grid)e(in)g (eac)o(h)h(dimension)d(\(arra)o(y)905 1082 y(of)13 b(nonnegativ)o(e)h (in)o(teger\))117 1156 y(IN)155 b Fq(o)o(rder)490 b Ff([SAME])14 b(arra)o(y)f(storage)i(order)f(\015ag)g(\(in)o(teger\))117 1231 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 1305 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 1430 y Fp(int)23 b(MPI)p 245 1430 15 2 v 17 w(Type)p 358 1430 V 17 w(darray\(MPI)p 615 1430 V 16 w(Comm)g(comm,)g(int)h(ndims,)f(int)g(*dsizes,)g(int)g (*distribs,)393 1486 y(int)h(*dargs,)f(int)g(pdims,)g(int)g(*psizes,)g (int)h(order,)393 1543 y(MPI)p 468 1543 V 17 w(Datatype)f(oldtype,)f (MPI)p 986 1543 V 17 w(Datatype)h(*newtype\))75 1629 y(MPI)p 150 1629 V 17 w(TYPE)p 263 1629 V 16 w(DARRAY\(COMM,)g(NDIMS,)g (DSIZES,)g(DISTRIBS,)f(DARGS,)h(PDIMS,)g(PSIZES,)g(ORDER,)393 1686 y(OLDTYPE,)g(NEWTYPE,)g(IERROR\))170 1742 y(INTEGER)g(COMM,)h (NDIMS,)f(DSIZES\(*\),)f(DISTRIBS\(*\),)g(DARGS\(*\),)h(PDIMS,)g (PSIZES\(*\))170 1799 y(INTEGER)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR)166 1885 y Fq(MPI)p 251 1885 14 2 v 16 w(TYPE)p 384 1885 V 17 w(D)o(ARRA)l(Y)f Ft(generates)f(the)h(\014let)o(yp)q(es)g (corresp)q(onding)g(to)f(the)g(distribution)i(of)e(an)75 1941 y Fq(ndims)p Ft(-dimensional)15 b(arra)o(y)c(of)h Fq(oldt)o(yp)q(e)h Ft(elemen)o(ts)g(on)o(to)e(a)h Fq(p)q(dims)p Ft(-dimensional)j(grid)e(of)f(logical)h(pro)q(ces-)75 1998 y(sors.)18 b(F)l(or)10 b(a)g(call)h(to)f Fq(MPI)p 507 1998 V 16 w(TYPE)p 640 1998 V 17 w(D)o(ARRA)l(Y)i Ft(to)d(b)q(e)j(correct,)e(the)h(equation)1365 1966 y Fd(Q)1404 1976 y Fc(pdims)p Fb(\000)p Fa(1)1404 2011 y Fc(i)p Fa(=0)1554 1998 y Fi(psiz)r(es)p Ft([)p Fi(i)p Ft(])h(=)h Fi(siz)r(e)p Ft(,)75 2054 y(where)i Fi(siz)r(e)h Ft(denotes)f(the)h(size)g(of)e(comm)o(unicator)h Fq(comm)p Ft(,)e(m)o(ust)i(b)q(e)h(satis\014ed.)166 2111 y(Eac)o(h)f(dimension)i (of)e(the)g(arra)o(y)f(is)i(distributed)g(in)g(one)g(of)e(three)i(w)o (a)o(ys:)143 2202 y Fh(\017)23 b Fg(MPI)p 266 2202 13 2 v 14 w(DISTRIBUTE)p 522 2202 V 13 w(BLOCK)15 b Ft(-)g(Blo)q(c)o(k)h (distribution)143 2294 y Fh(\017)23 b Fg(MPI)p 266 2294 V 14 w(DISTRIBUTE)p 522 2294 V 13 w(CYCLIC)14 b Ft(-)h(Cyclic)i (distribution)143 2387 y Fh(\017)23 b Fg(MPI)p 266 2387 V 14 w(DISTRIBUTE)p 522 2387 V 13 w(NONE)16 b Ft(-)f(Dimension)h(not)f (distributed.)166 2478 y(T)l(o)e(sp)q(ecify)i(a)f(default)g (distribution)i(argumen)o(t,)c(the)i(constan)o(t)f Fg(MPI)p 1365 2478 V 14 w(DISTRIBUTE)p 1621 2478 V 14 w(DFL)m(T)p 1736 2478 V 15 w(ARG)f Ft(is)75 2534 y(used.)20 b(The)c(distribution)h (argumen)o(t)d(for)h(a)g(dimension)i(that)d(is)i(not)f(distributed)h (is)g(ignored.)166 2591 y(F)l(or)j(example,)i(ARRA)l(Y\(CYCLIC\(15\)\)) e(corresp)q(onds)h(to)e Fg(MPI)p 1316 2591 V 15 w(DISTRIBUTE)p 1573 2591 V 13 w(CYCLIC)g Ft(with)h(a)75 2647 y(distribution)13 b(argumen)o(t)d(of)h(15,)g(and)h(ARRA)l(Y\(BLOCK\))h(corresp)q(onds)f (to)e Fg(MPI)p 1471 2647 V 15 w(DISTRIBUTE)p 1728 2647 V 13 w(BLOCK)75 2704 y Ft(with)16 b(a)e(distribution)j(argumen)o(t)e (of)f Fg(MPI)p 796 2704 V 15 w(DISTRIBUTE)p 1053 2704 V 13 w(DFL)m(T)p 1167 2704 V 15 w(ARG)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 39 41 39 40 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(39)166 49 y(The)17 b Fq(o)o(rder)f Ft(argumen)o(t)g(is)h(used)g (as)f(in)i Fq(MPI)p 928 49 14 2 v 16 w(TYPE)p 1061 49 V 17 w(SUBARRA)l(Y)h Ft(to)d(sp)q(ecify)i(the)f(storage)e(order)75 106 y(for)h(m)o(ultidimensional)k(arra)o(ys.)k(Therefore,)17 b(arra)o(ys)f(describ)q(ed)j(b)o(y)e(this)h(t)o(yp)q(e)f(constructor)f (ma)o(y)g(b)q(e)75 162 y(stored)11 b(in)h(ro)o(w-ma)s(jor)d(or)i (column-ma)s(jor)g(order.)18 b(V)l(alid)13 b(v)m(alues)g(for)d Fq(o)o(rder)h Ft(are)g Fg(MPI)p 1517 162 13 2 v 14 w(F)o(ORTRAN)p 1724 162 V 14 w(ORDER)75 219 y Ft(or)k Fg(MPI)p 208 219 V 14 w(C)p 249 219 V 14 w(ORDER)p Ft(.)166 275 y(F)l(or)f(example,)i (let)g(us)f(generate)g(the)g(\014let)o(yp)q(es)i(corresp)q(onding)f(to) e(the)i(HPF)e(distribution:)218 369 y Fp()23 b(FILEARRAY\(100,)f(200,)h(300\))75 425 y(!HPF$)g(PROCESSORS)g (PROCESSES\(2,)f(3\))75 482 y(!HPF$)h(DISTRIBUTE)g (FILEARRAY\(CYCLIC\(10\),)e(*,)i(BLOCK\))g(ONTO)h(PROCESSES)75 576 y Ft(This)13 b(can)g(b)q(e)g(ac)o(hiev)o(ed)g(b)o(y)f(the)h(follo)o (wing)g(F)o(OR)l(TRAN)g(co)q(de,)g(assuming)g(there)g(will)h(b)q(e)f (six)g(pro)q(cesses)75 632 y(attac)o(hed)i(to)f(the)h(run:)170 726 y Fp(ndims)24 b(=)f(3)170 782 y(dsizes\(1\))g(=)h(100)170 839 y(distribs\(1\))f(=)h(MPI_DISTRIBUTE_CYCLIC)170 895 y(dargs\(1\))f(=)h(10)170 952 y(dsizes\(2\))f(=)h(200)170 1008 y(distribs\(2\))f(=)h(MPI_DISTRIBUTE_NONE)170 1065 y(dargs\(2\))f(=)h(0)170 1121 y(dsizes\(3\))f(=)h(300)170 1178 y(distribs\(3\))f(=)h(MPI_DISTRIBUTE_BLOCK)170 1234 y(dargs\(3\))f(=)h(MPI_DISTRIBUTE_DFLT_ARG)170 1290 y(pdims)g(=)f(2)170 1347 y(psizes\(1\))g(=)h(2)170 1403 y(psizes\(2\))f(=)h(3)170 1460 y(call)g(MPI_TYPE_DARRAY\(MPI_COMM)o(_WORLD,)c(ndims,)j(dsizes,)g (distribs,)g(dargs,)147 1516 y($)119 b(pdims,)23 b(psizes,)g (MPI_FORTRAN_ORDER,)e(oldtype,)i(newtype,)g(ierr\))75 1636 y Fq(MPI)p 160 1636 14 2 v 16 w(T)l(yp)q(e)p 268 1636 V 17 w(da)o(rra)o(y)p 403 1636 V 15 w(2d)16 b(\(collective\))75 1722 y Ft(Distribution)g(of)f(a)g Fk(two)p Ft(-dimensional)j(arra)o(y:) -32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 40 42 40 41 bop 75 -100 a Ft(40)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y)p 486 49 V 17 w(2D\(comm,)13 b(dsize1,)i (distrib1,)h(da)o(rg1,)e(psize1,)i(dsize2,)f(distrib2,)h(da)o(rg2,)e (psize2,)75 106 y(o)o(rder,)g(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 183 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to) h(b)q(e)i(used)g(in)e(MPI)p 1655 183 13 2 v 15 w(Op)q(en)i(\(han-)905 239 y(dle\))117 315 y(IN)155 b Fq(dsize1)473 b Ff([SAME])21 b(size)g(of)f(distributed)i(arra)o(y)f(in)f(\014rst)i(dimension)905 371 y(\(nonnegativ)o(e)14 b(in)o(teger\))117 446 y(IN)155 b Fq(distrib1)447 b Ff([SAME])14 b(distribution)f(for)h(\014rst)g (dimension)e(\(in)o(teger\))117 521 y(IN)155 b Fq(da)o(rg1)481 b Ff([SAME])12 b(distribution)g(argumen)o(t)f(for)h(\014rst)h (dimension)d(\(in-)905 578 y(teger\))117 653 y(IN)155 b Fq(psize1)473 b Ff([SAME])12 b(size)i(of)e(pro)q(cessor)j(grid)d(in)g (\014rst)i(dimension)d(\(in)o(te-)905 709 y(ger\))117 784 y(IN)155 b Fq(dsize2)473 b Ff([SAME])14 b(size)g(of)g(distributed)g (arra)o(y)g(in)g(second)h(dimension)905 841 y(\(nonnegativ)o(e)f(in)o (teger\))117 916 y(IN)155 b Fq(distrib2)447 b Ff([SAME])14 b(distribution)f(for)h(second)g(dimension)f(\(in)o(teger\))117 991 y(IN)155 b Fq(da)o(rg2)481 b Ff([SAME])18 b(distribution)f(argumen) o(t)g(for)h(second)h(dimension)905 1048 y(\(in)o(teger\))117 1123 y(IN)155 b Fq(psize2)473 b Ff([SAME])11 b(size)g(of)g(pro)q (cessor)h(grid)f(in)f(second)i(dimension)d(\(in-)905 1179 y(teger\))117 1254 y(IN)155 b Fq(o)o(rder)490 b Ff([SAME])14 b(arra)o(y)f(storage)i(order)f(\015ag)g(\(in)o(teger\))117 1329 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 1405 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 1529 y Fp(int)23 b(MPI)p 245 1529 15 2 v 17 w(Type)p 358 1529 V 17 w(darray)p 519 1529 V 16 w(2d\(MPI)p 679 1529 V 17 w(Comm)g(comm,)g(int)h(dsize1,) e(int)i(distrib1,)f(int)g(darg1,)393 1585 y(int)h(psize1,)f(int)g (dsize2,)g(int)g(distrib2,)g(int)g(darg2,)g(int)h(psize2,)393 1642 y(int)g(order,)f(MPI)p 731 1642 V 16 w(Datatype)g(oldtype,)g(MPI)p 1249 1642 V 17 w(Datatype)g(*newtype\))75 1728 y(MPI)p 150 1728 V 17 w(TYPE)p 263 1728 V 16 w(DARRAY)p 423 1728 V 17 w(2D\(COMM,)g(DSIZE1,)f(DISTRIB1,)h(DARG1,)g(PSIZE1,)g(DSIZE2,)g (DISTRIB2,)393 1785 y(DARG2,)g(PSIZE2,)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR\))170 1841 y(INTEGER)g(COMM,)h(DSIZE1,)e(DISTRIB1,)h(DARG1,)g (PSIZE1,)g(DSIZE2,)g(DISTRIB2,)g(DARG2)170 1898 y(INTEGER)g(PSIZE2,)g (ORDER,)g(OLDTYPE,)g(NEWTYPE,)g(IERROR)166 1984 y Ft(This)12 b(call)h(is)g(a)e(shorthand)h(for)f(the)h(t)o(w)o(o-dimensional)h (distributed)g(arra)o(y)e(case)h(and)g(is)g(equiv)m(alen)o(t)75 2041 y(to:)170 2134 y Fp(dsize[0]=dsize1;)22 b(distrib[0]=distrib1;)f (dargs[0]=darg1;)h(psize[0])h(=)h(psize1;)170 2191 y(dsize[1]=dsize2;)e (distrib[1]=distrib2;)f(dargs[1]=darg2;)h(psize[1])h(=)h(psize2;)170 2247 y(MPI_Type_darray\(comm,)d(2,)j(dsize,)f(distrib,)g(dargs,)g(2,)g (psize,)576 2304 y(order,)g(oldtype,)g(&newtype\);)75 2424 y Fq(MPI)p 160 2424 14 2 v 16 w(T)l(yp)q(e)p 268 2424 V 17 w(da)o(rra)o(y)p 403 2424 V 15 w(blo)q(ck)16 b(\(collective\))75 2510 y Ft(BLOCK)g(distribution)h(of)e(a)g (one-dimensional)i(arra)o(y:)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 41 43 41 42 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(41)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y)p 486 49 V 17 w(BLOCK\(comm,)13 b(dsize,)i(da)o(rg,)g(oldt)o(yp)q(e,)g(newt)o(yp)q(e\))117 126 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to) h(b)q(e)i(used)g(in)e(MPI)p 1655 126 13 2 v 15 w(Op)q(en)i(\(han-)905 183 y(dle\))117 258 y(IN)155 b Fq(dsize)496 b Ff([SAME])10 b(size)h(of)f(distributed)h(arra)o(y)f(\(nonnegativ)o(e)g(in)o(teger\)) 117 333 y(IN)155 b Fq(da)o(rg)504 b Ff([SAME])21 b(distribution)f (argumen)o(t,)i(e.g.)39 b(BLOCK\(darg\))905 390 y(\(in)o(teger\))117 465 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp)q (e)h(\(handle\))117 540 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 664 y Fp(int)23 b(MPI)p 245 664 15 2 v 17 w(Type)p 358 664 V 17 w(darray)p 519 664 V 16 w(block\(MPI)p 751 664 V 16 w(Comm)g(comm,)h(int)f(dsize,) g(int)h(darg,)393 721 y(MPI)p 468 721 V 17 w(Datatype)f(oldtype,)f(MPI) p 986 721 V 17 w(Datatype)h(*newtype\))75 807 y(MPI)p 150 807 V 17 w(TYPE)p 263 807 V 16 w(DARRAY)p 423 807 V 17 w(BLOCK\(COMM,)f(DSIZE,)h(DARG,)g(OLDTYPE,)g(NEWTYPE,)g(IERROR\)) 170 864 y(INTEGER)g(COMM,)h(DSIZE,)f(DARG,)g(OLDTYPE,)g(NEWTYPE,)f (IERROR)166 950 y Fq(MPI)p 251 950 14 2 v 16 w(TYPE)p 384 950 V 17 w(D)o(ARRA)l(Y)p 577 950 V 17 w(BLOCK)14 b Ft(generates)f(the)h(\014let)o(yp)q(es)g(corresp)q(onding)h(to)e(the) g(BLOCK)i(dis-)75 1007 y(tribution)h(of)f(a)g(one-dimensional)i Fq(dsize)f Ft(elemen)o(t)g(arra)o(y)e(of)h Fq(oldt)o(yp)q(e)p Ft(.)166 1063 y(This)h(call)g(is)g(a)f(shorthand)g(for:)170 1157 y Fp(MPI_Comm_size\(comm,)22 b(&size\);)170 1213 y(distrib)h(=)h(MPI_DISTRIBUTE_BLOCK;)170 1270 y (MPI_Type_darray\(comm,)d(1,)j(dsize,)f(distrib,)g(darg,)g(1,)h(size,) 576 1326 y(order,)f(oldtype,)g(&newtype\);)166 1420 y Ft(Note)10 b(that)f(order)h(is)h(not)e(sp)q(eci\014ed)k(for)c(this)i (constructor,)f(nor)g(for)f Fq(MPI)p 1395 1420 V 16 w(TYPE)p 1528 1420 V 17 w(D)o(ARRA)l(Y)p 1721 1420 V 18 w(CYCLIC)p Ft(,)75 1477 y(b)q(ecause)16 b(it)g(is)f(not)g(meaningful)i(for)d(1D)h (arra)o(ys.)75 1597 y Fq(MPI)p 160 1597 V 16 w(T)l(yp)q(e)p 268 1597 V 17 w(da)o(rra)o(y)p 403 1597 V 15 w(cyclic)h(\(collective\)) 75 1682 y Ft(CYCLIC)g(distribution)g(of)f(a)g(one-dimensional)j(arra)o (y:)75 1833 y Fq(MPI)p 160 1833 V 16 w(TYPE)p 293 1833 V 17 w(D)o(ARRA)l(Y)p 486 1833 V 17 w(CYCLIC\(comm,)13 b(dsize,)i(da)o(rg,)f(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 1911 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e (to)h(b)q(e)i(used)g(in)e(MPI)p 1655 1911 13 2 v 15 w(Op)q(en)i(\(han-) 905 1967 y(dle\))117 2042 y(IN)155 b Fq(dsize)496 b Ff([SAME])10 b(size)h(of)f(distributed)h(arra)o(y)f(\(nonnegativ)o(e)g(in)o(teger\)) 117 2117 y(IN)155 b Fq(da)o(rg)504 b Ff([SAME])19 b(distribution)f (argumen)o(t,)h(e.g.)34 b(CYCLIC\(darg\))905 2174 y(\(in)o(teger\))117 2249 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 2324 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 2448 y Fp(int)23 b(MPI)p 245 2448 15 2 v 17 w(Type)p 358 2448 V 17 w(darray)p 519 2448 V 16 w(cyclic\(MPI)p 775 2448 V 16 w(Comm)g(comm,)h(int)f (dsize,)g(int)g(darg,)393 2505 y(MPI)p 468 2505 V 17 w(Datatype)g(oldtype,)f(MPI)p 986 2505 V 17 w(Datatype)h(*newtype\))75 2591 y(MPI)p 150 2591 V 17 w(TYPE)p 263 2591 V 16 w(DARRAY)p 423 2591 V 17 w(CYCLIC\(COMM,)f(DSIZE,)h(DARG,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR\))170 2648 y(INTEGER)g(COMM,)h(DSIZE,)f(DARG,)g(OLDTYPE,)g (NEWTYPE,)f(IERROR\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 42 44 42 43 bop 75 -100 a Ft(42)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(TYPE)p 384 49 V 17 w(D)o(ARRA)l(Y)p 577 49 V 17 w(CYCLIC)11 b Ft(generates)g(the)g (\014let)o(yp)q(es)i(corresp)q(onding)f(to)e(the)h(CYCLIC)h(dis-)75 106 y(tribution)k(of)f(a)g(one-dimensional)i Fq(dsize)f Ft(elemen)o(t)g(arra)o(y)e(of)h Fq(oldt)o(yp)q(e)p Ft(.)166 162 y(This)h(call)g(is)g(a)f(shorthand)g(for:)170 256 y Fp(MPI_Comm_size\(comm,)22 b(&size\);)170 312 y(distrib)h(=)h (MPI_DISTRIBUTE_CYCLIC;)170 369 y(MPI_Type_darray\(comm,)d(1,)j(dsize,) f(distrib,)g(darg,)g(1,)h(size,)576 425 y(order,)f(oldtype,)g (&newtype\);)75 569 y Fm(10.7)60 b(Prop)r(osal:)26 b(Simpli\014ed)19 b(Op)r(en)75 726 y Fp(Several)k(scientific)f(users)i(have)f(complained) f(of)i(the)f(steep)h(learning)75 783 y(curve)f(of)h(this)f(interface)g (and)g(have)g(suggested)g(we)h(provide)f(simplified)75 839 y("wrapper")g(functions.)46 b(Rather)23 b(than)g(adding)g(extra)h (functions)e(to)i(the)75 896 y(specification,)e(the)h(following)g (proposal)g(was)g(made)h(which)f(simplifies)75 952 y(I/O)g(for)h(the)f (user)h(that)f(does)g(not)h(need)f(to)h(use)f(the)h(more)f (sophisticated)75 1009 y(features,)g(and)g(also)g(allows)g(the)h (filetype)f(to)g(be)h(changed)f(without)75 1065 y(re-opening)f(the)i (file)f(or)h(using)f(file)g(control.)75 1178 y(Break)g(MPI_OPEN)g(in)h (to)f(two)h(calls:)46 b(MPI_OPEN)23 b(and)h(MPI_LAYOUT.)75 1291 y(MPI_OPEN\(comm,)e(filename,)h(amode,)g(hints,)g(fh\))75 1404 y(Opens)g("filename")g(with:)75 1517 y(disp)g(=)h(0)75 1573 y(etype)f(=)h(MPI_BYTE)75 1630 y(filetype)f(=)g(MPI_BYTE)75 1743 y(This)g(essentially)g(mimic's)g(a)g(UNIX)h(open)f(call.)75 1856 y(MPI_LAYOUT\(fh,)f(disp,)h(etype,)g(filetype,)g(hints\))75 1968 y(Changes)g(the)g(process's)g(view)g(of)h(the)f(file.)47 b(It)24 b(is)g(assumed)e(to)75 2025 y(be)i(an)f(expensive)g(operation,) f(and)i(is)f(collective.)47 b(It)23 b(would)75 2081 y(be)h(allowed)e (to)i(be)g(done)f(at)h(any)f(time)g(on)h(an)g(open)f(file)g(handle.)75 2194 y(An)h(MPI_GET_LAYOUT)e(routine)g(could)i(also)f(be)h(provided)e (\(see)i(the)f(proposed)75 2251 y(replacemenet)f(for)i (MPI_FILE_CONTROL)d(above\).)75 2364 y(By)j(adding)f(this)g(change,)g (a)h(user)f(can)g(open,)h(read,)f(and)g(write)g(a)h(file)75 2420 y(without)f(having)g(to)h(know)f(anything)g(about)g(etypes,)g (filetypes,)f(displacement,)75 2477 y(etc.)47 b(It)24 b(also)f(means)g(that)h(you)f(can)g(open)h(a)g(file,)f(write)g(the)g (header)g(using)75 2533 y(etype=filetype=MPI_BYTE,)e(and)i(then,)g (after)g(the)h(header)f(has)g(been)h(written,)75 2589 y(set)f(the)h(layout)f(to)h(a)f(more)h(complicated)e(filetype)h(with)g (a)h(displacement.)75 2646 y(You)f(need)h(not)f(close)g(and)h(re-open)f (the)g(file)h(or)f(use)h(a)f(file)h(control)f(command.)75 2702 y(This)g(makes)g(it)h(easier)f(for)h(people)f(that)g(plan)g(to)h (write)f(simple)g(headers)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 43 45 43 44 bop 75 -100 a Fn(10.7.)34 b(PR)o(OPOSAL:)16 b(SIMPLIFIED)g(OPEN) 929 b Ft(43)75 49 y Fp(on)24 b(to)f(the)h(start)f(of)g(their)h(files.) -32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 44 46 44 45 bop 75 381 a Fr(Bibliograph)m(y)98 604 y Ft([1])21 b(F)l(ern)11 b(E.)f(Basso)o(w.)i(Installing,)h(managing,)e(and)g(using) g(the)g(IBM)g(AIX)g(P)o(arallel)h(I/O)f(File)g(System.)168 660 y(IBM)16 b(Do)q(cumen)o(t)f(Num)o(b)q(er)h(SH34-6065-00,)d(F)l (ebruary)i(1995.)k(IBM)c(Kingston,)g(NY.)98 754 y([2])21 b(P)o(eter)11 b(F.)g(Corb)q(ett,)g(Dror)f(G.)g(F)l(eitelson,)j (Jean-Pierre)f(Prost,)f(George)g(S.)g(Almasi,)h(Sandra)f(John-)168 811 y(son)20 b(Ba)o(ylor,)g(An)o(thon)o(y)g(S.)f(Bolmarcic)o(h,)j(Y)l (arsun)e(Hsu,)h(Julian)g(Satran,)f(Marc)f(Snir,)j(Rob)q(ert)168 867 y(Colao,)15 b(Brian)g(Herr,)g(Joseph)h(Ka)o(v)m(aky)l(,)e(Thomas)h (R.)g(Morgan,)e(and)j(An)o(thon)o(y)e(Zlotek.)20 b(P)o(arallel)168 923 y(\014le)15 b(systems)e(for)f(the)i(IBM)g(SP)f(computers.)k Fk(IBM)d(Systems)f(Journal)p Ft(,)h(34\(2\):222{2)o(48,)c(Jan)o(uary) 168 980 y(1995.)98 1074 y([3])21 b(Juan)15 b(Miguel)g(del)f(Rosario,)g (Ra)s(jesh)g(Borda)o(w)o(ek)m(ar,)f(and)h(Alok)g(Choudhary)l(.)k(Impro) o(v)o(ed)13 b(parallel)168 1130 y(I/O)i(via)f(a)f(t)o(w)o(o-phase)g (run-time)i(access)e(strategy)l(.)k(In)d Fk(IPPS)g('93)h(Workshop)h(on) f(Input/Output)168 1187 y(in)h(Par)n(al)r(lel)f(Computer)i(Systems)p Ft(,)c(pages)i(56{70,)e(1993.)18 b(Also)d(published)j(in)e(Computer)e (Arc)o(hi-)168 1243 y(tecture)i(News)f(21\(5\),)e(Decem)o(b)q(er)j (1993,)d(pages)i(31{38.)98 1337 y([4])21 b(Charles)e(H.)f(Ko)q(elb)q (el,)i(Da)o(vid)e(B.)g(Lo)o(v)o(eman,)g(Rob)q(ert)h(S.)f(Sc)o(hreib)q (er,)i(Guy)d(L.)h(Steele)i(Jr.,)e(and)168 1393 y(Mary)d(E.)f(Zosel.)21 b Fk(The)15 b(High)i(Performanc)n(e)f(F)m(ortr)n(an)f(Handb)n(o)n(ok)p Ft(.)k(MIT)c(Press,)g(1993.)98 1487 y([5])21 b(Da)o(vid)15 b(Kotz.)20 b(Disk-directed)c(I/O)f(for)g(MIMD)f(m)o(ultipro)q(cessors.) 20 b(In)15 b Fk(Pr)n(o)n(c)n(e)n(e)n(dings)f(of)i(the)g(1994)168 1544 y(Symp)n(osium)e(on)g(Op)n(er)n(ating)f(Systems)f(Design)h(and)h (Implementation)p Ft(,)e(pages)g(61{74,)f(No)o(v)o(em)o(b)q(er)168 1600 y(1994.)19 b(Up)q(dated)d(as)f(Dartmouth)f(TR)h(PCS-TR94-226)g(on) g(No)o(v)o(em)o(b)q(er)f(8,)h(1994.)98 1694 y([6])21 b(Da)o(vid)16 b(Kotz)g(and)f(Nils)i(Nieu)o(w)o(ejaar.)k(File-system)c (w)o(orkload)e(on)g(a)g(scien)o(ti\014c)j(m)o(ultipro)q(cessor.)168 1750 y Fk(IEEE)e(Par)n(al)r(lel)f(and)h(Distribute)n(d)h(T)m(e)n (chnolo)n(gy)p Ft(,)c(pages)i(51{60,)e(Spring)j(1995.)98 1844 y([7])21 b(Bill)15 b(Nitzb)q(erg.)h(P)o(erformance)c(of)g(the)h (iPSC/860)f(Concurren)o(t)g(File)i(System.)h(T)l(ec)o(hnical)g(Rep)q (ort)168 1901 y(RND-92-020,)f(NAS)i(Systems)f(Division,)h(NASA)g(Ames,) f(Decem)o(b)q(er)g(1992.)98 1994 y([8])21 b(William)f(J.)f(Nitzb)q (erg.)29 b Fk(Col)r(le)n(ctive)18 b(Par)n(al)r(lel)g(I/O)p Ft(.)29 b(PhD)18 b(thesis,)h(Departmen)o(t)f(of)f(Computer)168 2051 y(and)f(Information)f(Science,)i(Univ)o(ersit)o(y)e(of)g(Oregon,)g (Decem)o(b)q(er)h(1995.)98 2145 y([9])21 b(K.)h(E.)g(Seamons,)h(Y.)f (Chen,)i(P)l(.)e(Jones,)h(J.)f(Jozwiak,)i(and)e(M.)g(Winslett.)41 b(Serv)o(er-directed)168 2201 y(collectiv)o(e)17 b(I/O)f(in)g(P)o (anda.)j(In)d Fk(Pr)n(o)n(c)n(e)n(e)n(dings)e(of)j(Sup)n(er)n(c)n (omputing)f('95)p Ft(,)f(Decem)o(b)q(er)g(1995.)75 2295 y([10])21 b(Ra)s(jeev)f(Thakur)f(and)g(Alok)h(Choudhary)l(.)32 b(An)20 b(extended)g(t)o(w)o(o-phase)f(metho)q(d)g(for)g(accessing)168 2351 y(sections)h(of)f(out-of-core)f(arra)o(ys.)30 b(T)l(ec)o(hnical)21 b(Rep)q(ort)f(CA)o(CR-103,)f(Scalable)h(I/O)g(Initiativ)o(e,)168 2408 y(Cen)o(ter)g(for)g(Adv)m(anced)i(Computing)e(Researc)o(h,)i (Caltec)o(h,)f(June)h(1995.)33 b(Revised)22 b(No)o(v)o(em)o(b)q(er)168 2464 y(1995.)75 2558 y([11])f(The)f(MPI-IO)f(Committee.)31 b(MPI-IO:)19 b(a)g(parallel)h(\014le)g(I/O)f(in)o(terface)h(for)e(MPI,) g(April)i(1996.)168 2615 y(V)l(ersion)c(0.5.)952 2828 y(44)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Fri May 17 15:45:56 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA29936; Fri, 17 May 1996 15:43:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA17839 for mpi-core-out; Fri, 17 May 1996 14:46:19 -0500 Received: from win24.nas.nasa.gov (win24.nas.nasa.gov [129.99.33.39]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA17803 for ; Fri, 17 May 1996 14:45:27 -0500 Received: from localhost (nitzberg@localhost) by win24.nas.nasa.gov (8.6.12/NAS.6.1) with SMTP id MAA03222; Fri, 17 May 1996 12:45:14 -0700 Message-Id: <199605171945.MAA03222@win24.nas.nasa.gov> X-Authentication-Warning: win24.nas.nasa.gov: Host localhost didn't use HELO protocol To: mpi-core@mcs.anl.gov, mpi-io@nas.nasa.gov Cc: sio-hlapi@cacr.caltech.edu, sio-os-api@cs.princeton.edu Subject: MPI-2 I/O Chapter Proposal Date: Fri, 17 May 1996 12:45:13 -0700 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The following is a proposed starting point for the MPI-2 I/O chapter. All interested parties are encouraged to participate in this design process via the MPI mailing lists, news group, and forum meetings. See the MPI-2 Web pages for details at: http://www.mcs.anl.gov/Projects/mpi/mpi2/mpi2.html This proposal is essentially a reformatting of the MPI-IO Version 0.5 draft (available via http://lovelace.nas.nasa.gov/MPI-IO/) with the following changes: 1. Retained only sections 5 through 9, and a modified section 3. 2. Renamed functions, types, etc., from "MPIO_xxx" to "MPI_xxx". 3. Edited (minor changes) to conform to MPI-2 chapter style. 4. Added clarification regarding: file consistency semantics, the use of MPI_Comm in MPI_Open, ----- cut here ----- io-2.ps ----- cut here ----- %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%Pages: 46 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: /usr/local/bin/dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.05.16:2306 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet %%BeginProcSet: special.pro TeXDict begin /SDict 200 dict N SDict begin /@SpecialDefaults{/hs 612 N /vs 792 N /ho 0 N /vo 0 N /hsc 1 N /vsc 1 N /ang 0 N /CLIP 0 N /rwiSeen false N /rhiSeen false N /letter{}N /note{}N /a4{}N /legal{}N}B /@scaleunit 100 N /@hscale{@scaleunit div /hsc X}B /@vscale{@scaleunit div /vsc X}B /@hsize{/hs X /CLIP 1 N}B /@vsize{/vs X /CLIP 1 N}B /@clip{ /CLIP 2 N}B /@hoffset{/ho X}B /@voffset{/vo X}B /@angle{/ang X}B /@rwi{ 10 div /rwi X /rwiSeen true N}B /@rhi{10 div /rhi X /rhiSeen true N}B /@llx{/llx X}B /@lly{/lly X}B /@urx{/urx X}B /@ury{/ury X}B /magscale true def end /@MacSetUp{userdict /md known{userdict /md get type /dicttype eq{userdict begin md length 10 add md maxlength ge{/md md dup length 20 add dict copy def}if end md begin /letter{}N /note{}N /legal{} N /od{txpose 1 0 mtx defaultmatrix dtransform S atan/pa X newpath clippath mark{transform{itransform moveto}}{transform{itransform lineto} }{6 -2 roll transform 6 -2 roll transform 6 -2 roll transform{ itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll curveto}}{{ closepath}}pathforall newpath counttomark array astore /gc xdf pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{PaintBlack}if}N /txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 -1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop 90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr 2 get ppr 0 get neg sub neg 0 S TR}if}ifelse scaleby96{ppr aload pop 4 -1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S TR}if}N /cp {pop pop showpage pm restore}N end}if}if}N /normalscale{Resolution 72 div VResolution 72 div neg scale magscale{DVImag dup scale}if 0 setgray} N /psfts{S 65781.76 div N}N /startTexFig{/psf$SavedState save N userdict maxlength dict begin /magscale true def normalscale currentpoint TR /psf$ury psfts /psf$urx psfts /psf$lly psfts /psf$llx psfts /psf$y psfts /psf$x psfts currentpoint /psf$cy X /psf$cx X /psf$sx psf$x psf$urx psf$llx sub div N /psf$sy psf$y psf$ury psf$lly sub div N psf$sx psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub TR /showpage{}N /erasepage{}N /copypage{}N /p 3 def @MacSetUp}N /doclip{ psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto closepath clip newpath moveto}N /endTexFig{end psf$SavedState restore}N /@beginspecial{SDict begin /SpecialSave save N gsave normalscale currentpoint TR @SpecialDefaults count /ocount X /dcount countdictstack N}N /@setspecial {CLIP 1 eq{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs neg 0 rlineto closepath clip}if ho vo TR hsc vsc scale ang rotate rwiSeen{rwi urx llx sub div rhiSeen{rhi ury lly sub div}{dup}ifelse scale llx neg lly neg TR }{rhiSeen{rhi ury lly sub div dup scale llx neg lly neg TR}if}ifelse CLIP 2 eq{newpath llx lly moveto urx lly lineto urx ury lineto llx ury lineto closepath clip}if /showpage{}N /erasepage{}N /copypage{}N newpath }N /@endspecial{count ocount sub{pop}repeat countdictstack dcount sub{ end}repeat grestore SpecialSave restore end}N /@defspecial{SDict begin} N /@fedspecial{end}B /li{lineto}B /rl{rlineto}B /rc{rcurveto}B /np{ /SaveX currentpoint /SaveY X N 1 setlinecap newpath}N /st{stroke SaveX SaveY moveto}N /fil{fill SaveX SaveY moveto}N /ellipse{/endangle X /startangle X /yrad X /xrad X /savematrix matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix}N end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (temp.dvi) @start /Fa 3 62 df<07C018303018701C600C600CE00EE00EE00EE00EE00EE00EE00E E00EE00E600C600C701C30181C7007C00F157F9412>48 D<03000700FF00070007000700 070007000700070007000700070007000700070007000700070007007FF00C157E9412> I61 D E /Fb 1 1 df0 D E /Fc 5 116 df<003E000C000C000C000C0018001800180018073018F0307060706060C060C060 C06080C080C480C4C1C446C838700F177E9612>100 D<03000380030000000000000000 00000000001C002400460046008C000C0018001800180031003100320032001C0009177F 960C>105 D<383C1E0044C6630047028100460301008E0703000C0603000C0603000C06 0600180C0600180C0620180C0C20180C0C4030180440301807801B0E7F8D1F>109 D<1C3C22462382230346030603060306030C060C060C0C0C081A3019E018001800300030 003000FC001014808D12>112 D<07C00C201870187038001E000FC003E000606060E060 C0C0C1803F000C0E7E8D10>115 D E /Fd 1 82 df81 D E /Fe 8 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000F800 00F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115 >99 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F 001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018E018 F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Ff 68 123 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0 000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703 C00E03C00E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C07F87F8151D809C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<6060F0 F0F8F86868080808080808101010102020404080800D0C7F9C15>34 D<60F0F8680808081010204080050C7C9C0C>39 D<004000800100020006000C000C0018 001800300030007000600060006000E000E000E000E000E000E000E000E000E000E000E0 00E000600060006000700030003000180018000C000C00060002000100008000400A2A7D 9E10>I<800040002000100018000C000C000600060003000300038001800180018001C0 01C001C001C001C001C001C001C001C001C001C001C00180018001800380030003000600 06000C000C00180010002000400080000A2A7E9E10>I<60F0F070101010102020408004 0C7C830C>44 DI<60F0F06004047C830C>I<030007003F00C700 070007000700070007000700070007000700070007000700070007000700070007000700 07000700070007000F80FFF80D1C7C9B15>49 D<07C01830201C400C400EF00FF80FF807 F8077007000F000E000E001C001C00380070006000C00180030006010C01180110023FFE 7FFEFFFE101C7E9B15>I<07E01830201C201C781E780E781E381E001C001C0018003000 6007E00030001C001C000E000F000F700FF80FF80FF80FF00E401C201C183007E0101D7E 9B15>I<000C00000C00001C00003C00003C00005C0000DC00009C00011C00031C00021C 00041C000C1C00081C00101C00301C00201C00401C00C01C00FFFFC0001C00001C00001C 00001C00001C00001C00001C0001FFC0121C7F9B15>I<60F0F060000000000000000000 0060F0F06004127C910C>58 D<0FE03038401CE00EF00EF00EF00E000C001C0030006000 C0008001800100010001000100010001000000000000000000000003000780078003000F 1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F00000017800000 178000001780000023C0000023C0000023C0000041E0000041E0000041E0000080F00000 80F0000180F8000100780001FFF80003007C0002003C0002003C0006003E0004001E0004 001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E0618001801980070007800E0003801C0003801C0001803800 0180780000807800008070000080F0000000F0000000F0000000F0000000F0000000F000 0000F0000000F0000000700000807800008078000080380000801C0001001C0001000E00 0200070004000180080000E03000001FC000191E7E9C1E>II< FFFFFC0F003C0F000C0F00040F00040F00060F00020F00020F02020F02000F02000F0200 0F06000FFE000F06000F02000F02000F02000F02010F00010F00020F00020F00020F0006 0F00060F000C0F003CFFFFFC181C7E9B1C>II<001F808000E0618001801980070007800E0003801C0003801C000180380001807800 00807800008070000080F0000000F0000000F0000000F0000000F0000000F0000000F000 FFF0F0000F80700007807800078078000780380007801C0007801C0007800E0007800700 0B800180118000E06080001F80001C1E7E9C21>III75 DIII<003F 800000E0E0000380380007001C000E000E001C0007003C00078038000380780003C07800 03C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F000 01E0700001C0780003C0780003C0380003803C0007801C0007000E000E0007001C000380 380000E0E000003F80001B1E7E9C20>II< FFFF00000F01E0000F0078000F003C000F001C000F001E000F001E000F001E000F001E00 0F001C000F003C000F0078000F01E0000FFF00000F03C0000F00E0000F00F0000F007800 0F0078000F0078000F0078000F0078000F0078000F0078100F0078100F0038100F003C20 FFF01C20000007C01C1D7E9B1F>82 D<07E0801C1980300580700380600180E00180E000 80E00080E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007 C00003C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E 7E9C17>I<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F00 20800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00001F800003FFFC001B1C7F9B1E>IIII<7FF0FFC00FC03E000780180003C0180003E0 100001E0200001F0600000F0400000788000007D8000003D0000001E0000001F0000000F 0000000F8000000F80000013C0000023E0000021E0000041F00000C0F800008078000100 7C0003003C0002001E0006001F001F003F80FFC0FFF01C1C7F9B1F>II91 D<08081010202040404040808080808080B0B0F8 F8787830300D0C7A9C15>II<1FC000307000783800781C00 301C00001C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40 304E801F870012127E9115>97 DI< 07E00C301878307870306000E000E000E000E000E000E00060007004300418080C3007C0 0E127E9112>I<003F000007000007000007000007000007000007000007000007000007 0000070003E7000C1700180F00300700700700600700E00700E00700E00700E00700E007 00E00700600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C30181830 0C700E6006E006FFFEE000E000E000E00060007002300218040C1803E00F127F9112>I< 00F8018C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C 38C01C3880181800381C00381C00381C00381C001818001C38000C300013C00010000030 00001800001FF8001FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C 0E0007F800121C7F9215>II<1800 3C003C0018000000000000000000000000000000FC001C001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C000 000000000000000000000000000FE000E000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>I< FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C3FC0 1C0F001C0C001C08001C10001C20001C40001CE0001DE0001E70001C78001C38001C3C00 1C1C001C0E001C0F001C0F80FF9FE0131D7F9C16>IIII<03F0000E1C0018 0600300300700380600180E001C0E001C0E001C0E001C0E001C0E001C060018070038030 03001806000E1C0003F00012127F9115>II<03C1000C33 00180B00300F00700700700700E00700E00700E00700E00700E00700E007006007007007 00300F00180F000C370007C700000700000700000700000700000700000700000700003F E0131A7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000 F880388018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C001C00 3C00FFE01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C100E20 03C00C1A7F9910>IIII<7F8FF00F0380 0F030007020003840001C80001D80000F00000700000780000F800009C00010E00020E00 0607000403801E07C0FF0FF81512809116>II<7FFC7038 6038407040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E127F91 12>I E /Fg 29 117 df<001C0000003E0000003E0000002E0000006700000067000000 E7800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F00007 00F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038 001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F00001E00003C00003C00 00780000780000780000F00000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800003C00003C00001E00000F000807801803C07800FFF0003F80151F 7D9D1B>IIII<00 3F8001FFF003C0F80780380F00181E00003C00003C0000780000780000780000F00000F0 0000F00000F00000F00000F00000F007F8F007F8F000387800387800387800383C00383C 00381E00380F003807803803C0F801FFF0003F80151F7D9D1C>III75 DIII<003F000001 FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F00780007807800078078 000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C078000780780007807C000F803C000F003E001F001F003E000F807C0007C0F80003 FFF00001FFE000003F00001A1F7E9D1F>II<003F000001FFE00003FFF00007C0F8000F807C001F003E003E001F003C000F00 780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C07800078078000780780E07803C0F0F003E079F001E03DE00 0F83FC0007C1F80003FFF00001FFF800003F780000003C0000003E0000001F0000000F80 1A237E9D1F>II<03F8000FFE001C 0F00380700700300600000E00000E00000E00000E00000F000007800007F00003FE0001F FC0007FE0001FF00001F800007800003C00003C00001C00001C00001C00001C0C00180E0 0380F007007C0E001FFC0007F000121F7E9D17>II< F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070 F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F000707800E0 7800E03C01C01E03800F078007FE0001F800141E7C9C1D>III<78000E007C001E003C003C001E0038000F0070000F00F000 0781E00003C1C00001C3C00001E7800000F70000007E0000003E0000003C0000003C0000 007E00000077000000E7800001E3800003C1C0000381E0000700F0000F00F8000E007800 1C003C003C003E0078001F0070000F00F0000F80191D7F9C1C>II<7FFFF07FFFF00001E00003E00003C00007C0000780000F00001F00001E00 003E00003C0000780000F80000F00001F00001E00003C00007C0000780000F80000F0000 1E00003E00003C00007C0000780000FFFFF0FFFFF0141D7E9C19>I<07E00FF81FFC3C1C 70047000E000E000E000E000E000E000700070043C1C1FFC0FF807E00E127E9112>99 D 108 D110 D<1C001C001C001C001C001C00FFE0FFE01C001C001C001C00 1C001C001C001C001C001C001C001C001C201FF00FF007C00C187F970F>116 D E /Fh 2 16 df<400004C0000C6000183000301800600C00C006018003030001860000 CC0000780000300000300000780000CC000186000303000601800C00C018006030003060 0018C0000C40000416187A9623>2 D<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFF FFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D E /Fi 17 123 df<00F1800389C00707800E03801C03803C0380380700780700780700780700F0 0E00F00E00F00E00F00E10F01C20F01C20703C20705C40308C400F078014147E9318>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207E9F14>I< 007C01C207010E0F1E0F1C0E3C04780078007800F000F000F000F000F000700170023004 18380FC010147E9314>I<0000780003F80000700000700000700000700000E00000E000 00E00000E00001C00001C000F1C00389C00707800E03801C03803C038038070078070078 0700780700F00E00F00E00F00E00F00E10F01C20F01C20703C20705C40308C400F078015 207E9F18>I<007C01C207010E011C013C013802780C7BF07C00F000F000F000F0007000 700170023004183807C010147E9315>I<00007C0000CE00019E00039E00030C00070000 0700000700000700000E00000E00000E0000FFF0000E00000E00001C00001C00001C0000 1C00001C0000380000380000380000380000380000700000700000700000700000700000 E00000E00000E00000E00000C00001C000318000798000F300006200003C000017297E9F 16>I<00E001E001E000C000000000000000000000000000000E00130023804380438043 808700070007000E000E001C001C001C20384038403840388019000E000B1F7E9E10> 105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C 001C0038003800380038007000700070007100E200E200E200E200640038000A207E9F0E >108 D<1E07802318C023A06043C0704380704380708700E00700E00700E00700E00E01 C00E01C00E01C00E03821C03841C07041C07081C03083803101801E017147E931B>110 D<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003 C0F003C0F00780F00700700F00700E0030180018700007C00013147E9316>I<03C1E004 621804741C08781C08701E08701E10E01E00E01E00E01E00E01E01C03C01C03C01C03C01 C0380380780380700380E003C1C0072380071E000700000700000E00000E00000E00000E 00001C00001C0000FFC000171D819317>I<007C018203010603060706060E00078007F8 03FC01FE001F00077007F006F006E004400820301FC010147E9315>115 D<00C000E001C001C001C001C003800380FFF8038007000700070007000E000E000E000E 001C001C001C001C10382038203820384018800F000D1C7F9B10>I<0F00601180702180 E021C0E041C0E04380E08381C00701C00701C00701C00E03800E03800E03800E03840E07 080C07080C07080E0F1006131003E1E016147E931A>I<0F006060118070F02180E0F821 C0E07841C0E0384380E0188381C0100701C0100701C0100701C0100E0380200E0380200E 0380200E0380400E0380400E0380800E078080060781000709860001F078001D147E9321 >119 D<0F00601180702180E021C0E041C0E04380E08381C00701C00701C00701C00E03 800E03800E03800E03800E07000C07000C07000E0F00061E0003EE00000E00000E00001C 0078180078380070700060600021C0001F0000141D7E9316>121 D<01E02003F04007F8C00C1F8008010000020000040000080000100000600000C0000100 000200000400800801001003003F060061FC0040F80080700013147E9315>I E /Fj 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C060404060C0 31801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C00 0C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006 000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F000080 00400060C060C060804060801F000B107F8F0F>I<0300030007000F000B001300330023 004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C00200020002000 2F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C060 00C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC0 8080808001000200040004000C0008000800180018001800180018000B117E900F>I<1F 00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F >I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B10 7F8F0F>I E /Fk 49 122 df<00003F03E00000C386700001878CF00003879CF0000303 1860000700380000070038000007003800000E003800000E007000000E007000000E0070 0000FFFFFF80001C007000001C00E000001C00E000001C00E000001C00E000003800E000 003801C000003801C000003801C000003801C000007001C0000070038000007003800000 700380000070038000006003800000E007000000E007000000E007000000E007000000C0 06000001C00E000001C00E000031860C0000798F180000F31E100000620C6000003C07C0 00002429829F1C>11 D<00003FE00000E010000180380003807800030078000700300007 00000007000000070000000E0000000E0000000E000000FFFFE0000E00E0001C01C0001C 01C0001C01C0001C01C0001C038000380380003803800038038000380700003807000070 07000070071000700E2000700E2000700E2000E00E2000E0064000E0038000E0000000C0 000001C0000001C000003180000079800000F3000000620000003C0000001D29829F1A> I<0E1F3F3F1D0102020404081020C0080E779F0E>39 D<70F8F8F0E005057B840E>46 D<00000040000000C000000180000001800000030000000300000006000000060000000C 0000001800000018000000300000003000000060000000C0000000C00000018000000180 00000300000003000000060000000C0000000C0000001800000018000000300000003000 000060000000C0000000C0000001800000018000000300000003000000060000000C0000 000C0000001800000018000000300000003000000060000000C0000000C0000000800000 001A2D7FA117>I<000200020006000E003C00DC031C001C003800380038003800700070 0070007000E000E000E000E001C001C001C001C003800380038003800780FFF80F1E7B9D 17>49 D<001F800060E00080700100300200380420380420380410380420700460700380 600000E00001C000030000FE00001C000006000007000007800007800007803007807807 80780780F00F00800F00401E00401C0040380020E0001F8000151F7C9D17>51 D<0000600000E00000E00000E00001C00001C00001C00003800003800003000007000007 00000600000E00000C0000180000180000300000300000630000C7000087000107000307 00060E00040E00080E003F8E00607C00801FC0001C00001C000038000038000038000038 0000700000700000600013277E9D17>I<00C06000FFC001FF8001FE0001000001000002 0000020000020000020000040000047800058C000606000C070008070000078000078000 0780000780000F00700F00F00F00F00E00E01E00801C0080380080300040600061C0001F 0000131F7B9D17>I<001F0000718000C0C00180C00380E00700E00F00E00F01E01E01E0 1E01E01E01E01E01C01C03C01C03C01C03C01C07C01C0F800C0F8006378003C700000F00 000E00000E00001C00601C00F03800F07000E0600080C0004380003E0000131F7B9D17> 57 D<00000200000006000000060000000E0000001E0000001E0000003F0000002F0000 004F0000004F0000008F0000010F0000010F0000020F0000020F0000040F00000C0F0000 080F0000100F0000100F0000200F80003FFF800040078000C00780008007800100078001 0007800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFC0001E00F0001E0078001E0038001E003C003C003C003C003C003C003C003C00 3C0078007800780078007800F0007801E000F0078000FFFE0000F00F8000F003C001E001 C001E001E001E001E001E001E003C001E003C001E003C001E003C001C0078003C0078007 8007800F0007801E000F007800FFFFE0001E1F7D9E20>I<0000FE0200078186001C004C 0038003C0060003C00C0001C01C0001803800018070000180F0000181E0000101E000010 3C0000003C00000078000000780000007800000078000000F0000000F0000000F0000000 F0000000F00000807000008070000080700001003800010038000200180004000C001800 060020000381C00000FE00001F217A9F21>I<01FFFF80001E00E0001E0070001E003800 1E001C003C001C003C000E003C000E003C000E0078000E0078000E0078000E0078000E00 F0001E00F0001E00F0001E00F0001E01E0003C01E0003C01E0003C01E0007803C0007003 C0007003C000E003C001C0078001C00780038007800E0007801C000F007000FFFFC0001F 1F7D9E22>I<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C 0004003C00040078080800780800007808000078180000F0300000FFF00000F0300000F0 300001E0200001E0200001E0200001E0001003C0002003C0002003C0004003C000400780 00800780018007800100078007000F001F00FFFFFE001F1F7D9E1F>I<01FFFFFC001E00 38001E0018001E0008001E0008003C0008003C0008003C0008003C000800780010007808 00007808000078080000F0100000F0300000FFF00000F0300001E0200001E0200001E020 0001E0200003C0000003C0000003C0000003C00000078000000780000007800000078000 000F800000FFF800001E1F7D9E1E>I<01FFF3FFE0001F003E00001E003C00001E003C00 001E003C00003C007800003C007800003C007800003C007800007800F000007800F00000 7800F000007800F00000F001E00000FFFFE00000F001E00000F001E00001E003C00001E0 03C00001E003C00001E003C00003C007800003C007800003C007800003C007800007800F 000007800F000007800F000007800F00000F801F0000FFF1FFE000231F7D9E22>72 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078000078 0000780000F00000F00000F00000F00001E00001E00001E00001E00003C00003C00003C0 0003C0000780000780000780000780000F8000FFF800141F7D9E12>I<001FFF0000F800 00F00000F00000F00001E00001E00001E00001E00003C00003C00003C00003C000078000 0780000780000780000F00000F00000F00000F00001E00001E00301E00781E00F83C00F8 3C00F0780080700040E00021C0001F000018207D9E18>I<01FE00007FC0001E0000FC00 001E0000F80000170001780000170001780000270002F00000270004F00000270004F000 00270008F00000470009E00000470011E00000470021E00000470021E00000870043C000 00838043C00000838083C00000838083C000010381078000010382078000010382078000 0103840780000203840F00000203880F00000203900F00000203900F00000401E01E0000 0401E01E00000401C01E00000C01801E00001C01803E0000FF8103FFC0002A1F7D9E29> 77 D<0001FC0000070700001C01C0003000E000E0006001C00070038000700780003807 0000380E0000381E0000381C0000383C0000383C00003878000078780000787800007878 000078F00000F0F00000F0F00000E0F00001E0F00001C0F00003C0700003807000070078 000F0038001E0038003C001C0070000E00E0000783800001FC00001D217A9F23>79 D<01FFFF80001E00E0001E0070001E0038001E003C003C003C003C003C003C003C003C00 3C0078007800780078007800F0007800E000F003C000F00F0000FFFC0000F0000001E000 0001E0000001E0000001E0000003C0000003C0000003C0000003C0000007800000078000 0007800000078000000F800000FFF000001E1F7D9E1F>I<0007E040001C18C000300580 0060038000C0038001C00180018001000380010003800100038001000380000003C00000 03C0000003F8000001FF800001FFE000007FF000001FF0000001F8000000780000007800 000038000000380020003800200038002000300060007000600060006000E0007000C000 E8038000C606000081F800001A217D9F1A>83 D<0FFFFFF01E0780E01807802010078020 20078020200F0020600F0020400F0020400F0020801E0040001E0000001E0000001E0000 003C0000003C0000003C0000003C00000078000000780000007800000078000000F00000 00F0000000F0000000F0000001E0000001E0000001E0000001E0000003E00000FFFF0000 1C1F789E21>I87 D<00F1800389C0070780 0E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40 F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E 000E000E000E001C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE0 1EE01CE03CE038607060E031C01F0010207B9F15>I<007E0001C1000300800E07801E07 801C07003C0200780000780000780000F00000F00000F00000F00000F000007001007002 0030040018380007C00011147C9315>I<0000780003F800007000007000007000007000 00E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C03803C038038 0700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C4030 8C800F070015207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000 F000F0007000700170023804183807C010147C9315>I<00007800019C00033C00033C00 0718000700000700000E00000E00000E00000E00000E0001FFE0001C00001C00001C0000 1C0000380000380000380000380000380000700000700000700000700000700000700000 E00000E00000E00000E00000C00001C00001C0000180003180007B0000F300006600003C 00001629829F0E>I<003C6000E27001C1E00380E00700E00F00E00E01C01E01C01E01C0 1E01C03C03803C03803C03803C03803C07003C07001C0F001C17000C2E0003CE00000E00 000E00001C00001C00301C00783800F0700060E0003F8000141D7E9315>I<01E0000FE0 0001C00001C00001C00001C000038000038000038000038000070000070000071E000763 000E81800F01C00E01C00E01C01C03801C03801C03801C0380380700380700380700380E 10700E20700C20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C00000 0000000000000000000000000E003300230043804300470087000E000E000E001C001C00 1C003840388030807080310033001C000B1F7C9E0E>I<01E0000FE00001C00001C00001 C00001C0000380000380000380000380000700000700000703C00704200E08E00E11E00E 21E00E40C01C80001D00001E00001FC00038E00038700038700038384070708070708070 7080703100E03100601E0013207D9F15>107 D<03C01FC0038003800380038007000700 070007000E000E000E000E001C001C001C001C0038003800380038007000700070007100 E200E200E200E200640038000A207C9F0C>I<1C0F80F0002630C318004740640C004780 680E004700700E004700700E008E00E01C000E00E01C000E00E01C000E00E01C001C01C0 38001C01C038001C01C038001C01C0708038038071003803806100380380E10038038062 007007006600300300380021147C9325>I<1C0F802630C0474060478060470070470070 8E00E00E00E00E00E00E00E01C01C01C01C01C01C01C0384380388380308380708380310 7003303001C016147C931A>I<007C0001C3000301800E01C01E01C01C01E03C01E07801 E07801E07801E0F003C0F003C0F003C0F00780F00700700F00700E0030180018700007C0 0013147C9317>I<01C1E002621804741C04781C04701E04701E08E01E00E01E00E01E00 E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0072380071E0007000007 00000E00000E00000E00000E00001C00001C0000FFC000171D809317>I<00F0400388C0 0705800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E00 F01C00F01C00703C00705C0030B8000F3800003800003800007000007000007000007000 00E00000E0000FFE00121D7C9315>I<1C1E002661004783804787804707804703008E00 000E00000E00000E00001C00001C00001C00001C00003800003800003800003800007000 0030000011147C9313>I<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00800E00800E 00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C00012147C93 15>I<0E00C1C03300E3C02301C3E04381C1E04301C0E04701C060870380400E0380400E 0380400E0380401C0700801C0700801C0700801C0701001C0701001C0602001C0F02000C 0F04000E13080003E1F0001B147C931E>I<0383800CC4401068E01071E02071E02070C0 40E00000E00000E00000E00001C00001C00001C00001C040638080F38080F38100E58100 84C60078780013147D9315>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07001C0E00180E00180E001C1E000C3C0007DC 00001C00001C00003800F03800F07000E06000C0C0004380003E0000131D7C9316>I E /Fl 54 123 df<000078007C7800FC7801FC7803C00003800007800007800007800007 8000078000078000078000078000FFFC78FFFC78FFFC7807807807807807807807807807 807807807807807807807807807807807807807807807807807807807807807807807807 8078078078078078152480A31A>12 D<007000E001C003C0078007000F000E001E001C00 3C003C00380078007800780078007000F000F000F000F000F000F000F000F000F000F000 F000F000F000F0007000780078007800780038003C003C001C001E000E000F0007000780 03C001C000E000700C327DA413>40 DI46 D<000080000180000180 000300000300000300000600000600000600000C00000C00000C00001800001800001800 00300000300000300000600000600000600000C00000C00000C000018000018000018000 0180000300000300000300000600000600000600000C00000C00000C0000180000180000 180000300000300000300000600000600000600000C00000C00000C0000011317DA418> I<01F00007FC000FFE001F1F003C07803C07807803C07803C07803C07001C0F001E0F001 E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001 E0F001E07803C07803C07803C07803C03C07803C07801F1F000FFE0007FC0001F0001323 7EA118>I<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFFFFFFFF FF10227CA118>I<03F0000FFC001FFE003C1F003007807007C06003C0E003E0C001E040 01E04001E00001E00001E00001E00003C00003C0000780000780000F00001E00003C0000 780000F00001E00001C0000380000700000E00001C0000380000700000FFFFE0FFFFE0FF FFE013227EA118>I<01F00007FC001FFF003E0F003807807003C02003C02003C00003C0 0003C00003C0000780000780000F00001E0003FC0003F80003FE00000F000007800003C0 0003C00001E00001E00001E00001E00001E08001E0C003C0E003C07007803C0F801FFF00 0FFC0003F00013237EA118>I<001F00001F00002F00002F00006F0000EF0000CF0001CF 0001CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F00780F 00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F00000F 00000F0015217FA018>I<3FFF803FFF803FFF803C00003C00003C00003C00003C00003C 00003C00003C00003C00003CF8003FFE003FFF003F0F803E07803C03C03803C00001E000 01E00001E00001E00001E00001E00001E04003C04003C0E003C07007807C1F003FFE000F FC0003F00013227EA018>I<007E0001FF0003FF0007C1000F00001E00001E00003C0000 3C000078000078000078F800F1FE00F7FF00FF0F80FC0780F803C0F803C0F801C0F001E0 F001E0F001E0F001E0F001E07001E07001E07801E07803C03803C03C03801C07801F0F00 0FFE0007FC0001F00013237EA118>II<01F00007FC000FFE001E0F003C07803C07807803C078 03C07803C07803C07803C03803803C07801E0F000F1E0007FC0003F8000FFE001E0F003C 07807803C07803C0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C03C07803E 0F801FFF0007FC0001F00013237EA118>I<01F00007FC000FFE001E0F003C0700380780 7803807803C0F003C0F001C0F001E0F001E0F001E0F001E0F001E0F001E07003E07803E0 7803E03C07E03E1FE01FFDE00FF1E003E3C00003C00003C0000380000780000700000F00 101E00183C003FF8001FF00007C00013237EA118>I<001F0000001F0000003F8000003F 8000003B8000007BC0000073C0000071C00000F1E00000F1E00000E0E00001E0F00001E0 F00001C0F00003C0780003C078000380780007803C0007803C0007003C000F001E000F00 1E000FFFFE001FFFFF001FFFFF001C000F003C0007803C00078038000780780003C07800 03C0700003C0F00001E0F00001E0E00001E01B237EA220>65 DI<000FF000007F FC0000FFFF0001F01F0003C00700078000000F0000001E0000003E0000003C0000003C00 0000780000007800000078000000F0000000F0000000F0000000F0000000F0000000F000 0000F0000000F0000000F00000007800000078000000780000003C0000003C0000003E00 00001E0000000F0000000780008003C0038001F00F8000FFFF00007FFC00000FF0001925 7DA31F>IIII<000FF000003FFE0000FFFF8001F80F8003E00380 078000000F0000001E0000001E0000003C0000003C000000780000007800000078000000 F0000000F0000000F0000000F0000000F0000000F0000000F000FFC0F000FFC0F000FFC0 780003C0780003C0780003C03C0003C03C0003C01E0003C01E0003C00F0003C0078003C0 03E003C001F807C000FFFFC0003FFF00000FF8001A257DA321>II I76 D78 D<001FC000007FF00001FFFC0003F07E0007C01F000F800F801F 0007C01E0003C03C0001E03C0001E0780000F0780000F0780000F070000070F0000078F0 000078F0000078F0000078F0000078F0000078F0000078F0000078F0000078780000F078 0000F0780000F07C0001F03C0001E03E0003E01E0003C01F0007C00F800F8007C01F0003 F07E0001FFFC00007FF000001FC0001D257DA324>II82 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800007800007800007800007800 007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F00000F00000 F8000078000078000078000078000078000078C000F0E000F0F801E07E07C03FFF800FFF 0001FC0015257EA31B>II87 D<07E01FF83FFC381E201E000F000F000F000F00FF 07FF1FFF3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180780000780000700000F00000F00000F0 0000F00000F00000F000007800007800007800003C00401F03C00FFFC007FF8001FC0012 167E9516>I<0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C003E3C00FFBC01FFFC03F0FC03C07C07803C07803C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C07803C07803C03C07C03E0FC01FFFC00FFBC0 03E3C012237EA219>I<03F00007FC001FFE003E0F003C0780780380780380F001C0FFFF C0FFFFC0FFFFC0F00000F00000F000007000007800007800003C00801F07800FFF8007FF 0001F80012167E9516>I<003F00FF01FF03C00380078007800780078007800780078007 80FFF8FFF8FFF80780078007800780078007800780078007800780078007800780078007 800780078007800780102380A20F>I<01F07807FFF80FFFF81F1F001E0F003C07803C07 803C07803C07803C07801E0F001F1F000FFE001FFC0019F0003800003800003C00001FFE 001FFFC01FFFE03FFFF07801F07800F8F00078F00078F00078F000787800F03E03E01FFF C00FFF8001FC0015217F9518>III107 DIII<01FC0007FF000FFF801F07C03C01E07800F07800F0700070F00078 F00078F00078F00078F00078F000787800F07800F07C01F03E03E01F07C00FFF8007FF00 01FC0015167F9518>II114 D<07F01FFC3FFE3C0E780678007800 7C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE3FFC0FF010167F9513> I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F000F000F000F000F000F 000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12>III121 D<7FFFC07FFFC07FFFC0000F80000F00001E00003E00007C0000780000F00001 F00003E00003C0000780000F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012 167F9515>I E /Fm 42 122 df<001FE03FC0007FE0FFC001FFE1FFC003FFE3FFC003E0 07E00007C007C00007800F80000F800F80000F800F80000F800F80000F800F80000F800F 80000F800F80000F800F80000F800F8000FFFE0FFE00FFFE0FFE00FFFE0FFE000F800F80 000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F8000 0F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F 800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F800F80000F80 0F80000F800F8000222A7FA922>11 D<00000F80007E0F8000FE0F8001FE0F8003FE0F80 07C00000078000000F8000000F8000000F8000000F8000000F8000000F8000000F800000 0F8000000F800000FFFE0F80FFFE0F80FFFE0F800F800F800F800F800F800F800F800F80 0F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F80 0F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F800F80 0F800F800F800F80192B7FAA1F>I46 D<007E0001FF8007FFE00FFFF01FC3F81F00F83E007C3E007C3C003C7C003E7C003E7800 1E78001EF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF800 1FF8001FF8001FF8001FF8001F78001E7C003E7C003E7C003E7C003E3E007C3F00FC1F00 F81FC3F80FFFF007FFE001FF80007E0018297EA71D>48 D<00300000700001F0000FF000 FFF000FFF000F1F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F000FFFFE0FFFFE0FFFFE0 13287BA71D>I<00FE0003FF800FFFC01FFFE01F07F03C01F87800FC78007C70007CF000 3EF0003E60003E20003E00003E00003E00003E00007C00007C0000780000F80001F00001 E00003E00007C0000F80001F00003E00007C0000F80001F00003E00003C0000780000F00 001E00003C00007FFFFE7FFFFE7FFFFE7FFFFE17287EA71D>I<007E0003FF8007FFE01F FFF03F83F83E00F87C007C38007C30007C10007C00007C00007C0000F80000F80001F800 03F0000FE001FFC001FF8001FFC001FFE00001F80000FC00007C00003E00003E00001F00 001F00001F00001F00001F80001FC0003EE0003EF0007E7C00FC3F01F81FFFF00FFFE003 FFC000FE0018297EA71D>I<0007E0000FE0000BE0001BE0001BE0003BE0003BE00073E0 0073E000F3E000F3E001E3E001E3E003E3E003C3E007C3E00F83E00F83E01F03E01F03E0 3E03E03E03E07C03E07C03E0F803E0FFFFFFFFFFFFFFFFFFFFFFFF0003E00003E00003E0 0003E00003E00003E00003E00003E00003E00003E018277EA61D>I<3FFFF83FFFF83FFF F83FFFF83E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E3E 003EFF803FFFE03FFFF03FC1F03F00F83F007C3E007C00007C00003E00003E00003E0000 3E00003E00003E00003E40003E60007CE0007CF000F87801F87F07F03FFFE00FFFC007FF 0001FC0017287EA61D>I<000FE0003FF800FFF801FFF803F01807E0000F80000F00001F 00003E00003E00003C00007C00007C02007C3FC0F8FFE0F9FFF0FBFFF8FF80FCFE007CFE 003EFC003EFC003FFC001FF8001FF8001FF8001FF8001F78001F7C001F7C001F7C001E3C 003E3E003E1E007C1F00FC0FC1F807FFF003FFE001FF80007E0018297EA71D>II58 D<0003F000000003F000000007F800000007F8000000077800 00000F3C0000000F3C0000000F3C0000001E3E0000001E1E0000001E1E0000003E1F0000 003C1F0000007C0F8000007C0F800000780F800000F807C00000F807C00000F007C00001 F003E00001F003E00001E003E00003E001F00003E001F00003C001F00007C000F80007FF FFF80007FFFFF8000FFFFFFC000F80007C000F00007C001F00003E001F00003E001E0000 3E003E00001F003E00001F003C00001F007C00000F807C00000F80F800000FC0F8000007 C0F8000007C0222A7EA927>65 D<0003FF00000FFFE0003FFFF800FFFFF801FE01F803F8 007007E0001007C000000F8000001F8000001F0000003E0000003E0000007C0000007C00 00007C0000007C000000F8000000F8000000F8000000F8000000F8000000F8000000F800 0000F8000000F8000000F80000007C0000007C0000007C0000007C0000003E0000003E00 00001F0000001F8000000F80000007C0000007E0000803F8003801FE01F800FFFFF8003F FFF0000FFFC00003FE001D2C7CAA25>67 DIII73 D77 D<0003FC0000000FFF000000 3FFFC00000FFFFF00001FE07F80003F801FC0007E0007E000FC0003F000F80001F001F00 000F801F00000F803E000007C03E000007C07C000003E07C000003E07C000003E0780000 01E0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001 F0F8000001F0F8000001F0F8000001F07C000003E07C000003E07C000003E07E000007E0 3E000007C03E000007C01F00000F801F80001F800F80001F000FC0003F0007E0007E0003 F801FC0001FE07F80000FFFFF000003FFFC000001FFF80000003FC0000242C7DAA2B>79 DI<007FC000 01FFF80007FFFE000FFFFF001FC07F003F000F007E0006007C0000007C000000F8000000 F8000000F8000000F8000000F8000000FC0000007E0000007F0000003F8000001FF80000 0FFF000007FFE00003FFF80000FFFC00000FFE000000FF0000003F0000001F8000000F80 00000FC0000007C0000007C0000007C0000007C0000007C0000007C000000F8060000F80 F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE00000FF80001A2C7DAA21>83 D87 D<01FC000FFF803FFFC03FFFE03C03F03001F00001F8 0000F80000F80000F80000F80001F800FFF807FFF81FFFF83FC0F87E00F8FC00F8F800F8 F800F8F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1C>97 D<007F8001FFE007FFF80FFFF81FC0783F00103E00007C00007C00007C0000F80000F800 00F80000F80000F80000F80000F800007C00007C00007C00003E00003F00181FC0780FFF F807FFF801FFE0007F80151B7E9A1A>99 D<00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E00003E00003E00003E00003E01FC3E03FF3E0FFFBE 1FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003E F8003EF8003EFC003E7C003E7C003E7E007E3F00FE1FC1FE1FFFFE0FFFBE03FE3E01F83E 172A7EA91E>I<00FC0003FF0007FF800FFFC01F87E03F01E03E00F07C00F07C0070F800 78FFFFF8FFFFF8FFFFF8FFFFF8F80000F80000F800007800007C00007C00003E00003F00 181FC0F80FFFF807FFF801FFE0007F00151B7E9A1A>I104 DI108 DII<007E0001FF8007FFE00FFFF01F81F83F 00FC3E007C7C003E7C003E78001EF8001FF8001FF8001FF8001FF8001FF8001FF8001F7C 003E7C003E7C003E3E007C3F00FC1FC3F80FFFF007FFE003FFC0007E00181B7E9A1D>I< F83F00F9FFC0FBFFE0FFFFF0FF07F8FC01F8F800FCF800FCF8007CF8007EF8003EF8003E F8003EF8003EF8003EF8003EF8003EF8007EF8007CF800FCFC00F8FC01F8FF07F0FFFFE0 FBFFC0F9FF80F87E00F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F8000017277C9A1E>I114 D<07F8001FFF003FFF807FFF807C0780F80100F80000F80000F80000FC00 007F00007FF0003FFC001FFE0007FF0000FF80001FC0000FC00007C00007C04007C0E007 C0FC1F80FFFF807FFF001FFE0007F800121B7E9A16>I<0F80000F80000F80000F80000F 80000F80000F8000FFFF80FFFF80FFFF800F80000F80000F80000F80000F80000F80000F 80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F 80800FC38007FFC007FFC003FF0001F80012227FA115>IIII<7C001F807E003F003F003E001F007C000F80 FC0007C0F80007E1F00003E3E00001F7E00000FFC000007F8000007F0000003E0000003E 0000003F0000007F800000FFC00001F3E00003E1E00003E1F00007C0F8000F807C001F80 7C001F003E003E001F007E001F80FC000FC01A1B809A1B>II E /Fn 31 90 df<3078F8787005057C840D>46 D<00000400000C00000C0000 180000180000300000300000600000600000C00000C00001800001800003000003000006 00000600000C00000C0000180000180000300000300000600000600000C00000C0000180 000180000300000300000600000600000600000C00000C00001800001800003000003000 00600000600000C00000C00000800000162D7EA117>I<007E0001C3000301800701C00E 00C00E00E01C00E01C00E03C01E03801E07801E07801E07801E07801E07801E0F003C0F0 03C0F003C0F003C0F003C0F00380F00780E00780E00700E00700E00E00600E00701C0030 38003870000FC000131F7C9D17>I<000C001C00FC0F3800380038003800380038007000 70007000700070007000E000E000E000E000E000E001C001C001C001C001C001C0038003 C0FFFE0F1E7C9D17>I<003F8000C1E00100F00200780400780400780F007C0F807C0F80 7C0F00780600780000F80000F00001E00001C0000380000700000E00001C000038000060 0000C0000180000300200600200800401000403FFFC07FFF80FFFF80161E7E9D17>I<00 7F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E00003C000038000 0700000E0000F800000E000007000007800007C00003C00007C03007C07807C0F807C0F8 07C0F00780800F00400E00201C0018780007E000141F7D9D17>I<0000600000600000E0 0001C00003C00005C0000DC00009C00011C000238000438000C380008380010380020380 040700080700180700100700200700400700FFFFF0000E00000E00000E00000E00000E00 001C00001E0001FFE0141E7E9D17>I<01803001FFE003FFC003FF0003FC000200000200 00020000040000040000040000047C000587000603800C01800801C00001C00001E00001 E00001E00001E07003C0F803C0F003C0E00380800780400700400E00201C0018700007C0 00141F7D9D17>I<000F8000704000C0200180E00301E00701E00E00C01E00001C00003C 000038000078F800790E007A07007C0300F80380F80380F003C0F003C0F003C0F003C0F0 0780E00780E00780E00700E00F00600E00701C0030180018700007C000131F7C9D17>I< 2000003FFFE07FFFC07FFF80400100C00200800200800400000800001000002000004000 0040000080000180000300000300000700000600000E00000E00001E00001C00001C0000 3C00003C00003C0000780000780000780000300000131F799D17>I<0E1F1F1E0C000000 000000000000003078F8787008147C930D>58 D<00001000000018000000380000003800 00007800000078000000FC000001BC0000013C0000033C0000023C0000063C0000043E00 00081E0000081E0000101E0000101E0000201E0000200F0000400F0000400F0000FFFF00 00800F0001000F8001000780020007800200078004000780040007800C0007C03E0007C0 FF807FFC1E207E9F22>65 D<0001F808000E061800380138007000F801E0007803C00070 07800030078000300F0000301F0000301E0000303E0000203C0000007C0000007C000000 7C0000007C000000F8000000F8000000F8000000F8000000F80000007800004078000080 780000803C0000803C0001001C0002000E00020006000C000300100001C0E000003F0000 1D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E 0078001E0078001E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F 01E0001E01E0003E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F0 03C001E003C003C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF800 7C0078003C0038003C001800780018007800080078000800780008007800080078080800 F0100000F0100000F0100000F0300000FFF00000F0700001E0200001E0200001E0200001 E0200001E0000801E0001003C0001003C0001003C0002003C0002003C0006003C000C007 8001C0078007C0FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C00180078 0018007800080078000800780008007800080078000800F0100000F0100000F0100000F0 300000F0700000FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0 000003C0000003C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F 7E9E1E>I<07FFC7FFC0007C00F800003C007800003C007800007800F000007800F00000 7800F000007800F000007800F000007800F00000F001E00000F001E00000F001E00000F0 01E00000FFFFE00000F001E00001E003C00001E003C00001E003C00001E003C00001E003 C00001E003C00003C007800003C007800003C007800003C007800003C007800003C00780 0007800F000007C00F8000FFF8FFF800221F7E9E22>72 D<07FFE0007C00003C00003C00 00780000780000780000780000780000780000F00000F00000F00000F00000F00000F000 01E00001E00001E00001E00001E00001E00003C00003C00003C00003C00003C00003C000 07800007C000FFFC00131F7F9E10>I<07FFF000007E0000003C0000003C000000780000 007800000078000000780000007800000078000000F0000000F0000000F0000000F00000 00F0000000F0000001E0000001E0000001E0000001E0000001E0008001E0010003C00100 03C0010003C0030003C0020003C0060003C0060007801E0007807C00FFFFFC00191F7E9E 1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800004E0002F000004E 0002F000004E0004F000004E0004F000004E0008F000004E0008F00000870011E0000087 0011E00000870021E00000870021E00000870041E00000838041E00001038083C0000103 8083C00001038103C00001038203C0000101C203C0000101C403C0000201C40780000201 C80780000201C80780000201D00780000200F00780000600E00780000600E00F00000F00 C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E001800003E0018 00004F001000004F001000004780100000478010000043C010000043C010000083C02000 0081E020000081E020000080F020000080F020000080782000010078400001007C400001 003C400001003C400001001E400001001E400002000F800002000F800002000F80000200 0780000200078000060003800006000300000F00010000FFE0010000221F7E9E22>I<00 03F800001E0E000038070000E0038001C001C003C001E0078000E00F0000F00F0000F01E 0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8F80001F0F8 0001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C0007803C000F003C 001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07FFFF00007C 03C0003C01E0003C00F0007800F0007800F8007800F8007800F8007800F8007800F000F0 01F000F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001E0000001E0 000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003C000000780 000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C01E0007801 E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F0078000F00F 0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E01E0003C03E 0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC0F18000003 E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E0008 0E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E0 0000F00000F04000E04000E04000E04000E06000C0600180E00380F80300C60C0081F800 16217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010C00F001080 0F0010800F0010800F0010001E0000001E0000001E0000001E0000001E0000001E000000 3C0000003C0000003C0000003C0000003C0000003C000000780000007800000078000000 780000007800000078000000F0000001F800007FFFE0001C1F7A9E21>IIII<03FFC0FFC0 007F007E00003E003800001E003000001E002000000F004000000F008000000F81000000 078200000007C600000003C400000003E800000001F000000001F000000000F000000000 F800000000F8000000017C000000023C000000043C0000000C1E000000081E000000101F 000000200F000000400F800000C0078000008007C000010003C000070003E0001F8007E0 00FFE01FFE00221F7F9E22>II E /Fo 39 122 df<000FF07F00007FF9FF8000F83FC7C001E07F8FC003E07F0FC007C07F 0FC007C03F078007C01F000007C01F000007C01F000007C01F000007C01F0000FFFFFFF8 00FFFFFFF80007C01F000007C01F000007C01F000007C01F000007C01F000007C01F0000 07C01F000007C01F000007C01F000007C01F000007C01F000007C01F000007C01F000007 C01F000007C01F000007C01F00003FF8FFF0003FF8FFF0002220809F1F>11 D<000FE000007FF80000F81C0001E07C0003E07C0007C07C0007C07C0007C0380007C000 0007C0000007C0000007C1FE00FFFFFE00FFFFFE0007C03E0007C03E0007C03E0007C03E 0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E 0007C03E0007C03E0007C03E003FF9FFC03FF9FFC01A20809F1D>I<0018007000E001C0 0380038007000E000E001E001C003C003C007800780078007800F800F000F000F000F000 F000F000F000F000F000F80078007800780078003C003C001C001E000E000E0007000380 038001C000E0007000180D2D7DA114>40 DI45 D<387CFEFEFE7C3807077C860F>I< 387CFEFEFE7C38000000000000387CFEFEFE7C3807147C930F>58 D<0000E000000000E000000001F000000001F000000001F000000003F800000003F80000 0006FC00000006FC0000000EFE0000000C7E0000000C7E000000183F000000183F000000 303F800000301F800000701FC00000600FC00000600FC00000C007E00000FFFFE00001FF FFF000018003F000018003F000030001F800030001F800060001FC00060000FC000E0000 FE00FFE00FFFE0FFE00FFFE0231F7E9E28>65 D<0007FC02003FFF0E00FE03DE03F000FE 07E0003E0FC0001E1F80001E3F00000E3F00000E7F0000067E0000067E000006FE000000 FE000000FE000000FE000000FE000000FE000000FE0000007E0000007E0000067F000006 3F0000063F00000C1F80000C0FC0001807E0003803F0007000FE01C0003FFF800007FC00 1F1F7D9E26>67 D69 DI73 D76 DI<001FF80000FFFF0001F81F8007 E007E00FC003F01F8001F81F0000F83F0000FC7F0000FE7E00007E7E00007EFE00007FFE 00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007F7E00007E7F 0000FE7F0000FE3F0000FC3F8001FC1F8001F80FC003F007E007E001F81F8000FFFF0000 1FF800201F7D9E27>79 DI<03FC 080FFF381E03F83800F8700078700038F00038F00018F00018F80000FC00007FC0007FFE 003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC0003CC0003CC000 3CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>83 D91 D93 D<07FC001FFF003F0F803F07 C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803 E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C03 00FC0000FC0000FC0000FC0000FC0000FC00007C00007E00003E00603F00C01F81C007FF 0001FC0013147E9317>I<0007F80007F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F801F8F80FFEF81F83F83E01F87E00F87C00F87C00F8FC00F8FC 00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F87E00F83E01F81F07F80FFEFF03F8FF18 207E9F1D>I<01FE0007FF800F83C01E01E03E00F07C00F07C00F8FC00F8FFFFF8FFFFF8 FC0000FC0000FC00007C00007C00003E00181E00180F807007FFE000FF8015147F9318> I<001F8000FFC001F3E003E7E003C7E007C7E007C3C007C00007C00007C00007C00007C0 00FFFC00FFFC0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC3C07 FFFE0F079E1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC0018 00001800001C00001FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78 003C78003C3F01F80FFFE001FF00171E7F931A>II<1C003E007F007F007F003E001C00000000000000 000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F00 1F001F001F00FFE0FFE00B217EA00E>I108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007E FC007EFC007EFC007EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF00 17147F931A>II114 D<0FE63FFE701E600EE006 E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81EFFFCC7F010147E9315> I<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F800F800F800F 800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>III121 D E /Fp 82 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F870051C779B18> 33 D<4010E038F078E038E038E038E038E038E038E038E038E038E03860300D0E7B9C18> I<00C00001C00001C00001C00003F0000FFC003FFE007DCF0071C700E1C380E1C780E1C7 80E1C780F1C00079C0003DC0001FE0000FF80003FC0001DE0001CF0001C70061C380F1C3 80F1C380E1C380E1C70071C70079DE003FFE001FF80007E00001C00001C00001C00000C0 0011247D9F18>36 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F0 1C73F01CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800 E0F800E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18>I<007000F001E003C007800F001E00 1C00380038007000700070007000E000E000E000E000E000E000E000E000700070007000 7000380038001C001E000F00078003C001F000F000700C24799F18>I<6000F00078003C 001E000F000780038001C001C000E000E000E000E0007000700070007000700070007000 7000E000E000E000E001C001C0038007800F001E003C007800F00060000C247C9F18>I< 01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC00 7FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D9718>I<00600000F0 0000F00000F00000F00000F00000F00000F0007FFFC0FFFFE0FFFFE07FFFC000F00000F0 0000F00000F00000F00000F00000F00000600013147E9718>I<1C3E7E7F3F1F070E1E7C F860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC7830060677 8518>I<000300000780000780000F80000F00001F00001E00001E00003E00003C00007C 0000780000780000F80000F00001F00001E00003E00003C00003C00007C0000780000F80 000F00000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F000 0060000011247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001C070 01C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C070 01C07803C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>I<018003800380 07800F803F80FF80FB804380038003800380038003800380038003800380038003800380 03800380038003807FFCFFFE7FFC0F1C7B9B18>I<03F0000FFE003FFF007C0F807003C0 E001C0F000E0F000E06000E00000E00000E00001C00001C00003C0000780000F00001E00 003C0000780000F00001E00007C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E 9B18>I<07F8001FFE003FFF007807807803C07801C03001C00001C00003C0000380000F 0003FF0003FE0003FF000007800003C00001C00000E00000E00000E0F000E0F000E0F001 C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<1FFF803FFF803FFF8038000038 00003800003800003800003800003800003800003BF8003FFE003FFF003C07801803C000 01C00000E00000E06000E0F000E0F000E0E001C07003C07C0F803FFF001FFC0003F00013 1C7E9B18>53 D<007E0001FF0007FF800F83C01E03C01C03C03801803800007000007000 00E1F800E7FE00FFFF00FE0780F803C0F001C0F000E0E000E0F000E07000E07000E07000 E03801C03C03C01E07800FFF0007FE0001F800131C7E9B18>I<03F8000FFE001FFF003E 0F803803807001C07001C07001C07001C03803803C07801FFF0007FC000FFE001F1F003C 07807001C0F001E0E000E0E000E0E000E0E000E07001C07803C03E0F801FFF000FFE0003 F800131C7E9B18>56 D<3078FCFC783000000000000000003078FCFC78300614779318> 58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<00 0300000780001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC 00007E00003F00001FC00007E00003F00001FC00007E00003F00001F8000078000030011 187D9918>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0 FFFFE07FFFC0130C7E9318>I<600000F00000FC00007E00003F00001FC00007E00003F0 0001FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F00 007E0000FC0000F0000060000011187D9918>I<007C0001FE0007FF000F87801E03C03C 1DC0387FC070FFE071E3E071C1E0E1C1E0E380E0E380E0E380E0E380E0E380E0E380E0E1 C1C071C1C071E3C070FF80387F003C1C001E00E00F83E007FFC001FF80007E00131C7E9B 18>64 D<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E 00038E00038E000306000707000707000707000707000FFF800FFF800FFF800E03800E03 801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>II<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E0700000 E00000E00000E00000E00000E00000E00000E00000E000007000007000E07000E03800E0 3C00E01E01C00F07C007FF8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F 801C03C01C03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C00701C00 701C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF8 00141C7F9B18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001 C0700000E00000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001C07003 C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F 07F01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01F FFC01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF 8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7F07F0FF87 F87F07F01C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0001DF0001DF0 001FB8001FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03801C01C07F03 F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01D C1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C 39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I< 0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780700700 780F007FFF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E1E380E1E380F0E78070 F700787F007FFF003FFE000FFC00001C00001E00000E00000F0000070000070011227D9B 18>I<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C0380 1C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C039C 1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>I<03F3801FFF803FFF807C0F807007 80E00380E00380E00380E000007000007800003F00001FF00007FE0000FF00000F800003 C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C 7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E0703800700000700000700000 700000700000700000700000700000700000700000700000700000700000700000700000 700000700000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC00 01F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F00 0707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F0000 1E00001C00003C0000780000700000F00001E00001C00003C0000780000700000F00001E 00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>I I93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I<0038007C007C007C003800000000000000000FFC1FFC0FFC001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C6038F078FFF07FE03F800E277E9C18>II<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF800 1F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F0019148193 18>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0 000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001 E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807F FFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F 00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E 00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780 700780E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF80 0FFB8003E380000380000380000380000380000380000380000380003FF8003FF8003FF8 151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C0000380000380 000380000380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318 >I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE0000 1F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<018000 0380000380000380000380007FFFC0FFFFC0FFFFC0038000038000038000038000038000 0380000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E00 13197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318 >I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E0003 8E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8F F0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E 000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E 03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000 DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F80003F 00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00 003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF0 14147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000007800007F E0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003FE0003FE000 7FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00003C000FFC000FF00007C000013247E9F18>125 D E /Fq 59 123 df<007F07F001FF1FF003FF3FF007807800070070000F00F0000F00F0000F00F000 0F00F0000F00F0000F00F0000F00F000FFF8FF80FFF8FF80FFF8FF800F00F0000F00F000 0F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000 0F00F0000F00F0000F00F0000F00F0000F00F0000F00F0001C20809F1B>11 D<0000F000F8F001F8F003F8F00780000700000F00000F00000F00000F00000F00000F00 000F0000FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00 F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F0142180A018>I<00 7000E001C00380078007000E001E001E003C003C003C0078007800780078007000F000F0 00F000F000F000F000F000F000F000F000F000F000700078007800780078003C003C003C 001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<787878781830306060E0050A7D830D>44 DI<000100030003000600060006000C000C000C00180018 001800300030003000600060006000C000C000C001800180018003000300030006000600 06000C000C000C00180018001800300030003000600060006000C000C000C000102D7DA1 17>47 D<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17> 49 D<07F0000FFC001FFE00383F00700F00600780E00780E003C04003C04003C00003C0 0003C00003C0000780000780000F00000E00001C00003C0000780000E00001C000038000 0700000E00001C0000380000700000FFFFC0FFFFC0FFFFC0121F7E9E17>I58 D<001F0000001F0000003F800000 3B8000003B8000007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F00001 E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C000FFFFE000F FFFE000FFFFE001E000F001E000F003C000F803C0007803C000780780007C0780003C078 0003C0F00003E01B207F9F1E>65 DI<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00 007C0000780000780000780000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800007C00003C00003E00001E00000F000207800E03E03E01FFFC00FF F0001FC017227DA01D>II II<001FE000 FFF801FFFE03E03E07800E0F00001E00003E00003C00007C0000780000780000780000F0 0000F00000F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C 001E3E001E1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F000000FFC00003FFF00007E1F8000F807C001F003E001E001E00 3C000F003C000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F80007C0780007807800078078000780 3C000F003C000F001E001E001F003E000F807C0007E1F80003FFF00000FFC000003F0000 1A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C00 003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001 E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019 >IIIII<780007807C000F003E001F001E001E000F003C000F807C000780780003C0F0 0003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000003F00 00007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C000F003E 001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>III<07E03FF87FFC701E401F000F000F000F003F 07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000 F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F8 0014147F9317>II<03C7800FF780 1FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F80780 7807807C0F803E1F801FFF800FF78007C780000780000780000780000780000780000780 000780000780000780111D7E9317>II<07F01FFC3FFC780C78 00780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311 >I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E00 1E001E001E001E001E001E201FF00FF007C00C1A7F9910>II< F003C0F003C07803807807807807803C0F003C0F003C0F001E0E001E1E001E1E000F1C00 0F3C000F3C0007380007380007B80003F00003F00001E00012147F9315>II<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F8 0000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414 809315>II<7FFF7FFF7FFF003E00 3C007800F800F001E003E007C007800F001F001E003C007C00FFFFFFFFFFFF10147F9314 >I E /Fr 14 122 df<0000000000700000000000F80000000001F80000000001F80000 000003F80000000003F00000000007F00000000007E00000000007E0000000000FE00000 00000FC0000000001FC0000000001F80000000001F80000000003F80000000003F000000 00007F00000000007E00000000007E0000000000FE0000000000FC0000000001FC000000 0001F80000000001F80000000003F80000000003F00000000007F00000000007E0000000 0007E0000000000FE0000000000FC0000000001FC0000000001F80000000001F80000000 003F80000000003F00000000007F00000000007E00000000007E0000000000FE00000000 00FC0000000001FC0000000001F80000000001F80000000003F80000000003F000000000 07F00000000007E00000000007E0000000000FE0000000000FC0000000001FC000000000 1F80000000003F80000000003F00000000003F00000000007F00000000007E0000000000 FE0000000000FC0000000000FC0000000001FC0000000001F80000000003F80000000003 F00000000003F00000000007F00000000007E0000000000FE0000000000FC0000000000F C0000000001FC0000000001F80000000003F80000000003F00000000003F00000000007F 00000000007E0000000000FE0000000000FC0000000000FC0000000001FC0000000001F8 0000000003F80000000003F00000000003F00000000007F00000000007E0000000000FE0 000000000FC0000000000FC0000000001FC0000000001F80000000003F80000000003F00 000000003F00000000007F00000000007E0000000000FE0000000000FC0000000000FC00 00000000F800000000007800000000002D677ACC3A>47 D66 D73 D<00000003FFF000000000 0000007FFFFF80000000000003FFFFFFF000000000000FFF003FFC00000000003FF80007 FF0000000000FFE00001FFC000000003FF8000007FF000000007FF0000003FF80000000F FE0000001FFC0000001FFC0000000FFE0000003FF800000007FF0000007FF000000003FF 800000FFF000000003FFC00001FFE000000001FFE00003FFE000000001FFF00003FFC000 000000FFF00007FFC000000000FFF80007FF80000000007FF8000FFF80000000007FFC00 0FFF80000000007FFC001FFF00000000003FFE001FFF00000000003FFE003FFF00000000 003FFF003FFF00000000003FFF003FFF00000000003FFF007FFF00000000003FFF807FFE 00000000001FFF807FFE00000000001FFF807FFE00000000001FFF807FFE00000000001F FF80FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE0000 0000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0 FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000 001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC07FFE 00000000001FFF807FFF00000000003FFF807FFF00000000003FFF807FFF00000000003F FF803FFF00000000003FFF003FFF00000000003FFF003FFF00000000003FFF001FFF8000 0000007FFE001FFF80000000007FFE001FFF80000000007FFE000FFFC000000000FFFC00 0FFFC000000000FFFC0007FFC000000000FFF80003FFE000000001FFF00003FFE0000000 01FFF00001FFF000000003FFE00000FFF800000007FFC000007FF800000007FF8000007F FC0000000FFF8000003FFE0000001FFF0000000FFF0000003FFC00000007FFC00000FFF8 00000003FFE00001FFF000000000FFF80007FFC0000000007FFF807FFF80000000000FFF FFFFFC000000000003FFFFFFF00000000000007FFFFF8000000000000003FFF000000000 4A497AC757>79 D<000FFFF000000000FFFFFF00000003FFFFFFC0000007F8007FF00000 0FFC001FFC00000FFC0007FE00001FFE0003FF00001FFE0003FF80001FFE0001FF80001F FE0001FFC0001FFE0000FFC0000FFC0000FFE00007F80000FFE00003F00000FFE0000000 0000FFE00000000000FFE00000000000FFE00000000000FFE00000000000FFE000000000 7FFFE00000003FFFFFE0000003FFFFFFE000001FFFF0FFE000007FFE00FFE00001FFE000 FFE00003FF8000FFE0000FFE0000FFE0001FFC0000FFE0001FF80000FFE0003FF80000FF E0007FF00000FFE0007FF00000FFE000FFE00000FFE000FFE00000FFE000FFE00000FFE0 00FFE00000FFE000FFE00001FFE000FFE00001FFE0007FF00003FFE0007FF000077FE000 3FF8000F7FE0001FFC003E7FF8000FFF80F83FFFE003FFFFF01FFFE000FFFFC00FFFE000 0FFE0003FFE0332E7CAD38>97 D<007FC000000000FFFFC000000000FFFFC000000000FF FFC000000000FFFFC00000000003FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0 0000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000 000001FFC00000000001FFC00000000001FFC01FFC000001FFC0FFFF800001FFC3FFFFF0 0001FFCFE01FFC0001FFDF0003FE0001FFFC0001FF0001FFF80000FF8001FFF000007FC0 01FFE000003FE001FFC000003FF001FFC000001FF801FFC000001FF801FFC000001FFC01 FFC000001FFC01FFC000000FFE01FFC000000FFE01FFC000000FFE01FFC000000FFE01FF C000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC0 00000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000 000FFE01FFC000000FFE01FFC000000FFE01FFC000000FFC01FFC000001FFC01FFC00000 1FFC01FFC000001FF801FFC000003FF001FFC000003FF001FFE000007FE001FFF000007F C001FFF80000FF8001FF3C0001FF0001FE1F0007FE0001FC0FC03FF80001F803FFFFE000 01F000FFFF800001E0001FF8000038487CC741>I<00000000001F0000007FF000FFC000 07FFFF03FFE0001FFFFFC7E7E0007FE03FFF0FF000FF800FFC0FF001FF0007FC0FF003FE 0003FE0FF007FE0003FF07E007FC0001FF03C00FFC0001FF80000FFC0001FF80001FFC00 01FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001 FFC0001FFC0001FFC0001FFC0001FFC0000FFC0001FF80000FFC0001FF800007FC0001FF 000007FE0003FF000003FE0003FE000001FF0007FC000000FF800FF8000000FFE03FF000 0001DFFFFFC0000003C7FFFF00000003C07FF00000000780000000000007800000000000 0780000000000007C0000000000007C0000000000007E0000000000007E0000000000007 F8000000000007FFFFFFF0000003FFFFFFFF000003FFFFFFFFE00001FFFFFFFFF00001FF FFFFFFFC0000FFFFFFFFFE00007FFFFFFFFF0001FFFFFFFFFF8007FFFFFFFFFF800FF000 001FFFC01FC0000001FFC03F800000007FC07F800000003FE0FF000000003FE0FF000000 001FE0FF000000001FE0FF000000001FE0FF000000001FE0FF000000001FE07F80000000 3FC07F800000003FC03FC00000007F801FE0000000FF000FF8000003FE0003FE00000FF8 0001FFE000FFF000007FFFFFFFC000000FFFFFFE000000007FFFC0000034447DAE3A> 103 D<007FC000000000FFFFC000000000FFFFC000000000FFFFC000000000FFFFC00000 000003FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0000000 0001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000000000 01FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001 FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC007FE000001FFC03FFFC00001FFC07FFFF00001FFC1F81FF80001FFC3 C00FFC0001FFC70007FE0001FFCE0007FE0001FFDC0007FF0001FFF80003FF0001FFF800 03FF8001FFF00003FF8001FFF00003FF8001FFE00003FF8001FFE00003FF8001FFE00003 FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF 8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF80 01FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001 FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FF C00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF80FFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81 FFFFFF38487BC741>I<007C0001FF0003FF8007FFC007FFC00FFFE00FFFE00FFFE00FFF E00FFFE007FFC007FFC003FF8001FF00007C000000000000000000000000000000000000 00000000000000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC003FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C0FFFFFFFFFFFFFFFFFFFFFFFF18497CC820>I<007FC000FFFFC000FFFFC000FFFFC000 FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<00000FFE0000000001FFFFF000000007FFFFFC0000001FF803FF0000007FE000FFC000 00FF80003FE00001FF00001FF00003FE00000FF80007FC000007FC000FFC000007FE000F F8000003FE001FF8000003FF003FF8000003FF803FF0000001FF803FF0000001FF807FF0 000001FFC07FF0000001FFC07FF0000001FFC0FFF0000001FFE0FFF0000001FFE0FFF000 0001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF00000 01FFE0FFF0000001FFE0FFF0000001FFE07FF0000001FFC07FF0000001FFC07FF0000001 FFC07FF0000001FFC03FF8000003FF803FF8000003FF801FF8000003FF001FFC000007FF 000FFC000007FE0007FE00000FFC0003FE00000FF80001FF00001FF00000FF80003FE000 007FE000FFC000001FFC07FF00000007FFFFFC00000001FFFFF0000000001FFF00000033 2E7DAD3A>111 D<007FC01FFC0000FFFFC0FFFF8000FFFFC3FFFFF000FFFFCFE03FFC00 FFFFDF0007FE0003FFFC0003FF0001FFF80001FF8001FFF00000FFC001FFE000007FE001 FFC000007FF001FFC000003FF801FFC000003FF801FFC000001FFC01FFC000001FFC01FF C000001FFE01FFC000001FFE01FFC000000FFE01FFC000000FFE01FFC000000FFF01FFC0 00000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000 000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFF01FFC000000FFE01FFC00000 1FFE01FFC000001FFE01FFC000001FFC01FFC000001FFC01FFC000003FFC01FFC000003F F801FFC000003FF001FFC000007FF001FFE00000FFE001FFF00000FFC001FFF80001FF80 01FFFC0003FF0001FFDF000FFE0001FFCFC03FF80001FFC3FFFFE00001FFC0FFFF800001 FFC01FF8000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FF C00000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC0 0000000001FFC00000000001FFC00000000001FFC00000000001FFC00000000001FFC000 00000001FFC000000000FFFFFF80000000FFFFFF80000000FFFFFF80000000FFFFFF8000 000038427CAD41>I<007F803F80FFFF80FFE0FFFF81FFF8FFFF83C3FCFFFF8707FE03FF 8E07FE01FF9C0FFF01FFB80FFF01FFB80FFF01FFF00FFF01FFF00FFF01FFF007FE01FFE0 03FC01FFE001F801FFE0000001FFE0000001FFC0000001FFC0000001FFC0000001FFC000 0001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC00000 01FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001 FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FF C00000FFFFFFE000FFFFFFE000FFFFFFE000FFFFFFE000282E7DAD2F>114 D121 D E /Fs 9 117 df<0001FF0000000FFFE000003FFFF80000FF83FE0001FE00FF00 03F8003F8007F8003FC00FF0001FE00FE0000FE01FE0000FF01FE0000FF03FE0000FF83F E0000FF83FC00007F87FC00007FC7FC00007FC7FC00007FC7FC00007FC7FC00007FCFFC0 0007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC000 07FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007 FEFFC00007FEFFC00007FEFFC00007FEFFC00007FE7FC00007FC7FC00007FC7FC00007FC 7FC00007FC7FC00007FC3FE0000FF83FE0000FF83FE0000FF81FE0000FF01FE0000FF00F F0001FE00FF0001FE007F8003FC003FC007F8001FE00FF0000FF83FE00007FFFFC00001F FFF0000001FF000027397CB730>48 D<00003C000000007C00000001FC00000007FC0000 003FFC0000FFFFFC0000FFFFFC0000FFFFFC0000FFCFFC0000000FFC0000000FFC000000 0FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000F FC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC 0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC00 00000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000 000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC000000 0FFC0000000FFC0000000FFC0000000FFC0000000FFC00007FFFFFFF807FFFFFFF807FFF FFFF807FFFFFFF80213879B730>I<0000001FFE000060000003FFFFE000E000001FFFFF F801E000007FFFFFFE03E00001FFFC00FF07E00007FFC0001FCFE0000FFF000007FFE000 1FFC000001FFE0003FF0000000FFE0007FE00000007FE000FFC00000003FE001FF800000 001FE003FF000000000FE007FF000000000FE00FFE0000000007E00FFE0000000007E01F FC0000000003E01FFC0000000003E03FF80000000003E03FF80000000001E03FF8000000 0001E07FF80000000001E07FF80000000001E07FF00000000000007FF0000000000000FF F0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000 000000FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FF F0000000000000FFF00000000000007FF00000000000007FF00000000000007FF8000000 0000007FF80000000001E03FF80000000001E03FF80000000001E03FF80000000001E01F FC0000000001E01FFC0000000003C00FFE0000000003C00FFE0000000003C007FF000000 00078003FF00000000078001FF800000000F0000FFC00000001F00007FE00000003E0000 3FF00000007C00001FFC000000F800000FFF000003F0000007FFC0000FE0000001FFFC00 7F800000007FFFFFFF000000001FFFFFFC0000000003FFFFE000000000001FFE0000003B 3D7BBB46>67 D<003FFE00000001FFFFE0000007FFFFF800000FE007FC00000FF001FE00 001FF800FF00001FF8007F80001FF8007FC0001FF8003FC0000FF0003FE00007E0003FE0 0003C0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000FFFFE0 00001FFFFFE000007FF83FE00003FF803FE00007FC003FE0000FF0003FE0001FE0003FE0 003FE0003FE0007FC0003FE0007FC0003FE000FF80003FE000FF80003FE000FF80003FE0 00FF80003FE000FF80007FE0007FC0007FE0007FC000DFE0003FE0039FF0001FF80F0FFF E007FFFE0FFFE001FFFC07FFE0003FE000FFE02B267DA52F>97 D<0001FF8000000FFFF0 00007FFFFC0000FF81FE0003FE007F8007F8003F800FF0001FC00FF0000FE01FE0000FE0 3FE0000FF03FE00007F07FC00007F07FC00007F87FC00007F8FFC00007F8FFC00007F8FF FFFFFFF8FFFFFFFFF8FFFFFFFFF8FFC0000000FFC0000000FFC0000000FFC00000007FC0 0000007FC00000007FC00000003FE00000003FE00000781FE00000781FF00000780FF000 00F007F80001F003FC0003E001FE000FC000FFC07F80003FFFFE00000FFFF8000000FFC0 0025267DA52C>101 D<00FE00000000FFFE00000000FFFE00000000FFFE00000000FFFE 0000000007FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE 00FF800003FE03FFF00003FE0FFFF80003FE1E03FC0003FE3801FE0003FE6001FF0003FE C000FF0003FFC000FF8003FF8000FF8003FF0000FF8003FF0000FF8003FF0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE 0000FF8003FE0000FF8003FE0000FF8003FE0000FF80FFFFF83FFFFEFFFFF83FFFFEFFFF F83FFFFEFFFFF83FFFFE2F3C7CBB36>104 D<00FE01FF0000FFFE1FFFF000FFFE7FFFFC 00FFFEFC07FE00FFFFF001FF0007FFC0007F8003FF00003FC003FE00003FE003FE00001F F003FE00001FF803FE00000FF803FE00000FFC03FE00000FFC03FE000007FC03FE000007 FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007 FE03FE000007FE03FE000007FE03FE000007FC03FE00000FFC03FE00000FFC03FE00000F FC03FE00000FF803FE00001FF003FE00001FF003FF00003FE003FF80007FC003FFC000FF 8003FFE001FF0003FEFC07FE0003FE7FFFF80003FE1FFFE00003FE03FE000003FE000000 0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE000000 0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE000000 00FFFFF8000000FFFFF8000000FFFFF8000000FFFFF80000002F377DA536>112 D<01FC03F000FFFC0FFC00FFFC1FFF00FFFC3C3F80FFFC707F8007FCE0FFC003FCC0FFC0 03FD80FFC003FD80FFC003FF807F8003FF003F0003FF001E0003FF00000003FE00000003 FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE 00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00 000003FE00000003FE00000003FE00000003FE00000003FE000000FFFFFC0000FFFFFC00 00FFFFFC0000FFFFFC000022267DA528>114 D<000F0000000F0000000F0000000F0000 000F0000001F0000001F0000001F0000001F0000003F0000003F0000007F0000007F0000 00FF000001FF000003FF000007FF00001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C00FF0078 00FF8078007F80F0003FC1E0001FFFC0000FFF800001FE001E377EB626>116 D E /Ft 84 125 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C000C00180018003000300030007000600060006000E0 00E000E000E000E000E000E000E000E000E000E000E00060006000600070003000300030 00180018000C000C000600020001000080004000200B2E7DA112>I<8000400020001000 08000C00060006000300030001800180018001C000C000C000C000E000E000E000E000E0 00E000E000E000E000E000E000E000C000C000C001C00180018001800300030006000600 0C00080010002000400080000B2E7DA112>I<01800180018001800180C183F18F399C0F F003C003C00FF0399CF18FC1830180018001800180018010147DA117>I<000600000006 000000060000000600000006000000060000000600000006000000060000000600000006 000000060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600000006 000000060000000600000006000000060000000600000006000000060000000600000006 00000006000000060000000600001C207D9A23>I<70F8FCFC7404040408081010204006 0E7C840D>II<70F8F8F87005057C840D>I<0001000300030006 00060006000C000C000C00180018001800300030003000600060006000C000C000C00180 018001800300030003000600060006000C000C000C001800180018003000300030006000 60006000C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070 0380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003 F000121F7E9D17>I<018003800F80F38003800380038003800380038003800380038003 800380038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C 9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007 C00007C0000780000780000F00000E00001C0000380000700000600000C0000180000300 000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C0010 0E00200F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003 F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040 0780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00 002E00002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00 200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E00 00FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000 0010000010000011F000161C00180E001007001007800003800003800003C00003C00003 C07003C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17 >I<007C000182000701000E03800C07801C0780380300380000780000700000700000F1 F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C070 03C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC0 7FFF807FFF80400100800200800200800400000800000800001000002000002000004000 00400000C00000C00001C000018000038000038000038000038000078000078000078000 078000078000078000078000030000121F7D9D17>I<03F0000C0C001006003003002001 806001806001806001807001807803003E03003F06001FC8000FF00003F80007FC000C7E 00103F00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001001002 000C0C0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700700380F0 0380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003 E3C0000380000380000380000700300700780600780E00700C002018001070000FC00012 1F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F8 700000000000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0 FFFFFFF00000000000000000000000000000000000000000000000000000000000000000 FFFFFFF07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C00380070 00E000C00180018001000300020002000200020002000200000000000000000000000700 0F800F800F8007000E207D9F15>63 D<000100000003800000038000000380000007C000 0007C0000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800 002078000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F00 01000F0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE 1F207F9F22>65 DI<000FC04000 7030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C 00004078000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8 000000F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007 0001000380020001C0040000703800000FC0001A217D9F21>III< FFFFFF000F800F0007800300078003000780010007800180078000800780008007800080 07800080078080000780800007808000078080000781800007FF80000781800007808000 078080000780800007808000078000000780000007800000078000000780000007800000 07800000078000000FC00000FFFE0000191F7E9E1E>I<000FE0200078186000E004E003 8002E0070001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8 000000F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E078 0001E07C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000 E0046000781820000FE0001E217D9F24>III<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C0000 3C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0020 3C00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>IIIII<001F800000F0F00001C0380007801E000F000F 000E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001 F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003 E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F0 00001F80001C217D9F23>II82 D<07E0800C1980100780 300380600180600180E00180E00080E00080E00080F00000F000007800007F00003FF000 1FFC000FFE0003FF00001F800007800003C00003C00001C08001C08001C08001C08001C0 C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F00 60400F0020400F0020C00F0030800F0010800F0010800F0010800F0010000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F80 0007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003C0020001E0040001F00C 0000F0080000781000007C1000003C2000003E4000001E4000000F8000000F8000000780 000003C0000007E0000005E0000009F0000018F8000010780000207C0000603C0000401E 0000801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I< FFF003FF1F8000F80F8000600780004007C0004003E0008001E0008001F0010000F00300 00F80200007C0400003C0400003E0800001E0800001F1000000FB0000007A0000007C000 0003C0000003C0000003C0000003C0000003C0000003C0000003C0000003C0000003C000 0003C0000003C0000007C000007FFE00201F7F9E22>I<7FFFF87C00F87000F06001E040 01E0C003C0C003C0800780800F80800F00001E00001E00003C00003C0000780000F80000 F00001E00001E00003C00403C0040780040F80040F000C1E000C1E00083C00183C001878 0038F801F8FFFFF8161F7D9E1C>II<0804100820 10201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE000303000781800781C00300E00000E00000E00000E0000FE 00078E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C01414 7E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E0078 0E00780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19> I<03F80E0C1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C 020E0C03F010147E9314>I<000380003F80000380000380000380000380000380000380 00038000038000038000038003E380061B801C0780380380380380700380700380F00380 F00380F00380F00380F00380F003807003807003803803803807801C07800E1B8003E3F8 15207E9F19>I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF 80F00000F00000F000007000007000003800801800800C010007060001F80011147F9314 >I<007C00C6018F038F07060700070007000700070007000700FFF00700070007000700 070007000700070007000700070007000700070007000700070007007FF01020809F0E> I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C 001E380033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C000 30C00030C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E 81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C000000 000000000000000000000E007E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E00000000000 00000000000000007007F000F00070007000700070007000700070007000700070007000 700070007000700070007000700070007000706070F060F0C061803F000C28829E0E>I< 0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C00 0E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE 618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E 81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C0 3801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C0 3801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00 F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01 C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D 7F9319>I<03E0800619801C05803C0780380380780380700380F00380F00380F00380F0 0380F00380F003807003807803803803803807801C0B800E138003E38000038000038000 0380000380000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F 0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312 >I<1F9030704030C010C010C010E00078007F803FE00FF00070803880188018C018C018 E030D0608F800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E 000E000E000E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F 9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800 003800003C00007C00004E000087000107000303800201C00601E01E01E0FF07FE171480 9318>II<3FFF380E200E201C4038 4078407000E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314> III E /Fu 45 122 df<70F8FCFC7404040404080810102040060F7C840E>44 D I<70F8F8F87005057C840E>I<01F000071C000C06001803003803803803807001C07001 C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001 E0F001E0F001E0F001E0F001E07001C07001C07001C07803C03803803803801C07000C06 00071C0001F00013227EA018>48 D<008003800F80F38003800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 800380038007C0FFFE0F217CA018>I<03F8000C1E001007002007804007C07807C07803 C07807C03807C0000780000780000700000F00000E0000380003F000001C00000F000007 800007800003C00003C00003E02003E07003E0F803E0F803E0F003C04003C04007802007 80100F000C1C0003F00013227EA018>51 D<000200000600000E00000E00001E00001E00 002E00004E00004E00008E00008E00010E00020E00020E00040E00040E00080E00100E00 100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E00000E00000E00 000E00001F0001FFF015217FA018>I<1000801E07001FFF001FFE001FF80013E0001000 0010000010000010000010000010000010F800130E001407001803801003800001C00001 C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C04003C04003802007 001006000C1C0003F00013227EA018>I<007E0001C1000300800601C00E03C01C03C018 0180380000380000780000700000700000F0F800F30C00F40600F40300F80380F801C0F0 01C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C03801C01803801C 03000C0600070C0001F00013227EA018>I<01F800060E000803001001802001802000C0 6000C06000C06000C07000C07801803E01003F02001FC4000FF80003F80003FC00067F00 083F80100F803007C06001C06000E0C000E0C00060C00060C00060C000606000406000C0 3000801803000E0E0003F00013227EA018>56 D<01F000060C000C060018070038038070 0380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003E038 03E01805E00C05E00619E003E1E00001C00001C00001C000038000038030030078070078 0600700C002018001030000FC00013227EA018>I<0001800000018000000180000003C0 000003C0000003C0000005E0000005E000000DF0000008F0000008F0000010F800001078 000010780000203C0000203C0000203C0000401E0000401E0000401E0000800F0000800F 0000FFFF000100078001000780030007C0020003C0020003C0040003E0040001E0040001 E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E006 3001C00170038000F0070000F00E0000701E0000701C0000303C0000303C0000307C0000 107800001078000010F8000000F8000000F8000000F8000000F8000000F8000000F80000 00F800000078000000780000107C0000103C0000103C0000101C0000201E0000200E0000 40070000400380008001C0010000E0020000381C000007E0001C247DA223>67 D69 DI< FFFC0FC00780078007800780078007800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800FC0FFFC0E227EA1 12>73 D 77 DI 80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020E00020 E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E00001E0 0000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0C80180 C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008400780 08C007800C80078004800780048007800480078004000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000FC0 0003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700000700 00FF0007C7001E07003C0700780700700700F00708F00708F00708F00F087817083C2390 0FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E003C0E 001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C 80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000700000 F00000F00000F00000F00000F00000F00000F000007000007800403800401C00800C0100 07060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800E07800 E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800E01801 E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C07801E070 00E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C00400E 008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700070007 00070007000700FFF8070007000700070007000700070007000700070007000700070007 00070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C07001C 07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010000018 00001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860003060 00303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E00F0070 0F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000000000 0000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E001C000 00000000000000000000000000000001E00FE001E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0F1 8061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E02000E0400 0E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C00E01E0 0E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00FE60E183801E807201C0 0F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60C01E80E00F00 700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E0038 0E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E0038 0F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E0000 0E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E 000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>III121 D E /Fv 20 118 df45 D68 D73 D77 D 80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E070 0000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF0000 0FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880 00007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0F0 0000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fw 5 85 df<00000000600000000000E00000000000E00000000001E0 0000000003E00000000003E00000000007E00000000007F0000000000DF0000000000DF0 0000000019F00000000039F00000000031F00000000061F00000000061F000000000C1F0 00000000C1F00000000181F00000000381F00000000301F00000000601F00000000601F0 0000000C01F80000000C00F80000001800F80000003800F80000003000F80000006000F8 0000006000F8000000C000F8000000C000F8000001FFFFF8000001FFFFF80000030000F8 0000060000F80000060000F800000C0000F800000C0000FC00001800007C00001800007C 00003000007C00007000007C00006000007C0000C000007C0000C000007C0001C000007C 0003C000007C000FE00000FE00FFF8001FFFE0FFF8001FFFE02B327CB134>65 D<000FFFFFFC0000000FFFFFFF800000007E000FE00000007C0003F00000007C0000F800 00007C00007C0000007C00007C000000F800003E000000F800001E000000F800001F0000 00F800001F000001F000001F000001F000000F800001F000000F800001F000000F800003 E000000F800003E000000F800003E000000F800003E000001F800007C000001F800007C0 00001F800007C000001F800007C000001F80000F8000001F00000F8000003F00000F8000 003F00000F8000003F00001F0000007E00001F0000007E00001F0000007E00001F000000 7C00003E000000FC00003E000000F800003E000001F800003E000001F000007C000003E0 00007C000003E000007C000007C000007C00000F800000F800000F000000F800001E0000 00F800003C000000F8000078000001F00001F0000001F00003E0000001F0000F80000003 F0007F000000FFFFFFFC000000FFFFFFC000000031317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007E0003F800007C0000F800007C00007800007C0000 7800007C0000300000F80000300000F80000300000F80000300000F80000300001F00000 300001F00000300001F00000300001F00000600003E00180600003E00180000003E00180 000003E00180000007C00300000007C00300000007C00700000007C01F0000000FFFFE00 00000FFFFE0000000F801E0000000F800E0000001F000C0000001F000C0000001F000C00 00001F000C0000003E00180000003E00180000003E00000000003E00000000007C000000 00007C00000000007C00000000007C0000000000F80000000000F80000000000F8000000 0000F80000000001F00000000001F00000000001F00000000003F000000000FFFFE00000 00FFFFE00000002D317BB02E>70 D<000FFFFFE000000FFFFFFC0000007E003F0000007C 000F8000007C0007C000007C0003E000007C0003E00000F80001F00000F80001F00000F8 0001F00000F80003F00001F00003F00001F00003F00001F00003F00001F00003E00003E0 0007E00003E00007C00003E0000FC00003E0000F800007C0001F000007C0003E000007C0 0078000007C001E000000F800F8000000FFFFC0000000F800E0000000F80070000001F00 038000001F0003C000001F0001E000001F0001E000003E0001F000003E0001F000003E00 01F000003E0001F000007C0003F000007C0003F000007C0003E000007C0003E00000F800 07E00000F80007E00000F80007E00000F80007E01801F00007E03001F00007E03001F000 07E02003F00003E060FFFF8001E0C0FFFF8000F180000000003E002D327BB033>82 D<07FFFFFFFFF007FFFFFFFFF00FE007E007F00F8007C001F00E0007C000E00C0007C000 E01C0007C000E018000F8000E018000F8000E030000F8000C030000F8000C030001F0000 C060001F0000C060001F0000C060001F0000C0C0003E00018000003E00000000003E0000 0000003E00000000007C00000000007C00000000007C00000000007C0000000000F80000 000000F80000000000F80000000000F80000000001F00000000001F00000000001F00000 000001F00000000003E00000000003E00000000003E00000000003E00000000007C00000 000007C00000000007C00000000007C0000000000F80000000000F80000000000F800000 00000F80000000001F00000000001F00000000001F00000000003F000000003FFFFF0000 003FFFFF0000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 799 911 a Fw(D)25 b(R)g(A)g(F)g(T)225 1002 y Fv(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n (terface)621 1196 y Fu(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum) 832 1323 y(Ma)o(y)h(16,)g(1996)77 1381 y(This)h(w)o(ork)f(w)o(as)h (supp)q(orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran) o(t)h(ASC-9310330,)i(the)192 1439 y(National)d(Science)f(F)l(oundation) i(Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e) 76 1497 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 49 a Ft(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 361 a Fs(Chapter)31 b(10)75 568 y Fr(I/O)75 798 y Ft(This)20 b(is)f(a)g(prop)q(osed)h(starting)e(p)q(oin)o(t)i(for) e(the)h Fq(MPI-2)g Ft(I/O)g(c)o(hapter.)31 b(It)20 b(is)f(directly)i (deriv)o(ed)f(from)75 855 y(MPI-IO)f(V)l(ersion)h(0.5)d([11])h(with)h (minor)f(editorial)i(c)o(hanges.)30 b(A)19 b(historical)h(p)q(ersp)q (ectiv)o(e)g(\(e.g.)29 b(the)75 911 y(argumen)o(ts)20 b(that)g(led)i(to)e(v)m(arious)h(design)g(decisions\))h(can)f(b)q(e)h (found)f(at)f(the)g(MPI-IO)i(W)l(eb)f(page)75 968 y Fp (http://lovelace.nas.nasa.g)o(ov/MPI-)o(IO/)p Ft(.)166 1024 y(Direct)f(all)h(commen)o(ts)f(regarding)g(the)g Fq(MPI-2)f Ft(I/O)i(Chapter)f(to)f(the)h Fp(mpi-io@mcs.anl.gov)75 1081 y Ft(mailing)d(list.)j(See)c Fp(http://www.mcs.anl.gov/Proj)o (ects/mp)o(i/mpi2/m)o(pi2.htm)o(l)c Ft(for)j(details.)166 1137 y(Ma)s(jor)e(issues)j(to)f(b)q(e)h(discussed)h(regarding)e(this)g (prop)q(osal:)75 1212 y Fo(La)o(y)o(erabilit)o(y)23 b Ft(Can/should)14 b(w)o(e)f(pro)o(vide)h(enough)f Fq(MPI-2)g Ft(facilities)i(to)e(allo)o(w)g(I/O)h(to)f(b)q(e)h(la)o(y)o(ered)f(on) 189 1268 y(top)h(of)h Fq(MPI-2)p Ft(?)75 1355 y Fo(P)o(ortable)j (O\013sets)k Ft(Ho)o(w)9 b(do)i(w)o(e)f(pro)o(vide)h(p)q(ortable)g (o\013sets)e(with)i(more)f(than)g(32)g(bits?)20 b(P)o(ossibilities:)189 1411 y(long)15 b(long/in)o(teger*8,)g(opaque)g(o\013sets,)f(in)o(t)h ([2]/in)o(teger)f(\(2\),)g(double/double)k(precision.)75 1497 y Fo(MPI)p 184 1497 16 2 v 18 w(File)g(vs.)k(MPI)p 497 1497 V 18 w(Comm)g Ft(Should)16 b(w)o(e)e(scrap)h(the)g(MPI-IO)g(v) o(er)g(0.5)e(idea)j(of)e(\014le)i(handles)g(and)189 1554 y(simply)g(use)g Fq(MPI)e Ft(comm)o(unicators?)75 1640 y Fo(MPI)p 184 1640 V 18 w(File)p 287 1640 V 20 w(Con)o(trol)22 b Ft(The)h(\014le)g(con)o(trol)f(in)o(terface)g(cannot)g(supp)q(ort)g (F)o(OR)l(TRAN)h(and)g(m)o(ust)e(b)q(e)189 1696 y(revised.)75 1783 y Fo(End-of-File)k Ft(The)14 b(curren)o(t)g(de\014nition)i(for)d (up)q(dating)i(\014le)g(p)q(oin)o(ters)g(when)f(end-of-\014le)i(is)e (reac)o(hed)g(is)189 1839 y(con)o(tradictory)l(.)k(The)12 b(seman)o(tics)g(de\014ned)h(in)g(the)f(last)f(paragraph)g(of)h (section)g(10.1.3)e(con)o(tradict)189 1896 y(the)15 b(language)g(in)h (the)g(in)o(tro)q(ductions)g(of)f(sections)g(10.4)f(and)i(10.5.)75 1982 y Fo(Shared)h(\014le)h(p)q(oin)o(ters)23 b Ft(There)15 b(is)h(some)e(con)o(tro)o(v)o(ersy)g(as)g(to)g(whether)h(w)o(e)g (should)h(k)o(eep)f(shared)g(\014le)189 2038 y(p)q(oin)o(ters,)g(and)g (if)h(w)o(e)f(do,)g(the)g(exact)g(seman)o(tics)g(of)g(shared)g(\014le)h (p)q(oin)o(ter)g(op)q(erations.)75 2125 y Fo(Filet)o(yp)q(e)i (constructors)23 b Ft(Since)15 b(\014let)o(yp)q(e)h(constructors)d(are) h(simply)h(new)g(datat)o(yp)q(e)e(constructors,)189 2181 y(should)19 b(w)o(e)e(mo)o(v)o(e)g(this)i(section)f(to)f(the)h(\\Datat) o(yp)q(e)f(Impro)o(v)o(emen)o(ts")g(subsection)i(in)g(Miscel-)189 2237 y(lan)o(y?)75 2324 y Fo(File)f(In)o(terop)q(erabilit)o(y)24 b Ft(Ho)o(w)16 b(m)o(uc)o(h)g(\014le)i(in)o(terop)q(erabilit)o(y)g (should)g(b)q(e)f(sp)q(eci\014ed.)26 b(Minimally)l(,)19 b(all)189 2380 y(applications)d(on)f(a)g(single)i(mac)o(hine,)e(using)h (a)f(single)h Fq(MPI)f Ft(implemen)o(tation)h(should)g(ha)o(v)o(e)f (the)189 2437 y(same)h(canonical)i(view)f(of)g(\014les.)25 b(What)16 b(ab)q(out)g(b)q(et)o(w)o(een)h Fq(MPI)g Ft(implemen)o (tations)g(or)g(b)q(et)o(w)o(een)189 2493 y(mac)o(hines)f(\(e.g.)j (heterogeneous)c(t)o(yp)q(es,)g(v)m(arying)h(ph)o(ysical)g(\014le)g (structures\).)75 2579 y Fo(Simpler)i(In)o(terface)k Ft(Sev)o(eral)15 b(users)f(ha)o(v)o(e)g(complained)i(of)e(the)h(steep)f (learning)i(curv)o(e)e(required)i(for)189 2636 y(this)g(in)o(terface.) 24 b(A)17 b(prop)q(osed)f(simpli\014cation)j(of)d Fq(MPI)p 1149 2636 14 2 v 16 w(OPEN)h Ft(\(see)f(section)h(10.7\))e(w)o(ould)i (ease)189 2692 y(migration)e(from)f(the)i(standard)e(UNIX)i(in)o (terface)g(to)e(the)h Fq(MPI)g Ft(I/O)h(in)o(terface.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Ft(2)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fm(10.1)60 b(Intro)r(duction)75 151 y Ft(Di\013eren)o(t) 11 b(parallel)i(\014le)g(systems)e(supp)q(ort)h(di\013eren)o(t)f(in)o (terfaces)h(and)g(the)f(lac)o(k)h(of)f(a)g(standard)g(p)q(ortable)75 207 y(in)o(terface)j(mak)o(es)f(it)h(imp)q(ossible)i(to)d(dev)o(elop)h (a)f(truly)h(p)q(ortable)g(parallel)h(program.)k(It)13 b(is)h(not)g(feasible)75 264 y(to)f(dev)o(elop)i(large)f(scien)o (ti\014c)h(applications)h(from)d(scratc)o(h)g(for)h(eac)o(h)g (generation)g(of)f(parallel)i(mac)o(hine,)75 320 y(and,)g(in)g(the)g (scien)o(ti\014c)h(w)o(orld,)f(p)q(ortabilit)o(y)g(and)g(p)q (erformance)g(are)f(inseparable|a)j(program)c(is)i(not)75 377 y(considered)i(truly)e(p)q(ortable)h(unless)g(it)g(not)e(only)i (compiles,)g(but)g(also)f(runs)g(e\016cien)o(tly)l(.)166 433 y(The)e(signi\014can)o(t)g(optimizations)g(required)h(for)d (e\016ciency)j(\(e.g.)k(grouping)13 b([7],)f(collectiv)o(e)i(bu\013er-) 75 489 y(ing)k([3,)f(8)o(,)h(9)o(,)g(10)o(],)f(and)h(disk-directed)i (I/O)e([5)o(]\))f(can)h(only)g(b)q(e)g(implemen)o(ted)i(as)d(part)g(of) g(a)g(parallel)75 546 y(I/O)d(en)o(vironmen)o(t)g(if)g(it)f(supp)q (orts)h(a)f(high-lev)o(el)j(in)o(terface)d(to)g(describ)q(e)i(the)f (partitioning)g(of)f(\014le)i(data)75 602 y(among)c(pro)q(cesses)g(and) h(a)f(collectiv)o(e)i(in)o(terface)e(describing)i(complete)f(transfers) f(of)f(global)i(data)f(struc-)75 659 y(tures)i(b)q(et)o(w)o(een)h(pro)q (cess)g(memories)g(and)f(\014les.)21 b(In)14 b(addition,)g(further)g (e\016ciencies)i(can)d(b)q(e)h(gained)h(via)75 715 y(supp)q(ort)d(for)g (async)o(hronous)f(I/O,)i(strided)f(requests,)h(and)f(con)o(trol)g(o)o (v)o(er)f(ph)o(ysical)i(\014le)g(la)o(y)o(out)f(on)g(stor-)75 772 y(age)i(devices)h(\(disks\).)20 b(The)14 b(I/O)g(en)o(vironmen)o(t) h(describ)q(ed)h(in)e(this)h(section)f(pro)o(vides)h(these)f (facilities.)166 828 y(Emphasis)h(has)f(b)q(een)i(put)e(in)i(remaining) f(as)f Fq(MPI)p Ft(-friendly)i(as)e(p)q(ossible.)22 b(When)14 b(op)q(ening)i(a)e(\014le,)75 885 y(a)k(comm)o(unicator)h(is)g(sp)q (eci\014ed)i(to)d(determine)i(whic)o(h)f(group)g(of)f(pro)q(cesses)h (can)g(get)f(access)h(to)f(the)75 941 y(\014le)g(in)g(subsequen)o(t)g (I/O)g(op)q(erations.)25 b(Accesses)18 b(to)f(a)g(\014le)h(can)f(b)q(e) h(indep)q(enden)o(t)h(\(no)e(co)q(ordination)75 998 y(b)q(et)o(w)o(een) f(pro)q(cesses)g(tak)o(es)f(place\))h(or)f(collectiv)o(e)i(\(eac)o(h)e (pro)q(cess)h(of)f(the)h(group)f(asso)q(ciated)h(with)g(the)75 1054 y(comm)o(unicator)h(m)o(ust)h(participate)g(in)h(the)f(collectiv)o (e)h(access\).)28 b Fq(MPI)17 b Ft(deriv)o(ed)i(datat)o(yp)q(es)e(are)g (used)75 1110 y(for)f(expressing)h(the)f(data)g(la)o(y)o(out)g(in)h (the)f(\014le)i(as)e(w)o(ell)h(as)f(the)g(distribution)i(of)e(the)h (\014le)g(data)f(among)75 1167 y(pro)q(cesses.)31 b(In)19 b(addition,)h(eac)o(h)f(access)f(\(read)h(or)f(write\))g(op)q(erates)h (on)f(a)g(n)o(um)o(b)q(er)h(of)g Fq(MPI)f Ft(ob)s(jects)75 1223 y(whic)o(h)e(can)f(b)q(e)h(of)f(an)o(y)g Fq(MPI)f Ft(basic)i(or)f(deriv)o(ed)h(datat)o(yp)q(es.)75 1345 y Fl(10.1.1)49 b(Data)17 b(P)o(a)o(rtitioning)75 1431 y Ft(Instead)d(of)g(de\014ning)h(\014le)g(access)f(mo)q(des)h(to)e (express)h(the)g(common)g(patterns)f(for)g(accessing)i(a)f(shared)75 1487 y(\014le)21 b(\(broadcast,)f(reduction,)h(scatter,)f(gather\),)g (w)o(e)g(c)o(hose)f(another)h(approac)o(h)f(whic)o(h)i(consists)f(of)75 1544 y(expressing)d(the)g(data)f(partitioning)h(via)g(deriv)o(ed)h (datat)o(yp)q(es.)23 b(Compared)17 b(to)e(a)i(limited)h(set)e(of)h (pre-)75 1600 y(de\014ned)j(access)f(patterns,)g(this)g(approac)o(h)f (has)h(the)g(adv)m(an)o(tage)f(of)g(added)h(\015exibili)q(t)o(y)i(and)d (expres-)75 1657 y(siv)o(eness.)166 1713 y Fq(MPI)11 b Ft(deriv)o(ed)i(datat)o(yp)q(es)f(are)f(used)i(to)e(describ)q(e)j(ho) o(w)e(data)f(is)i(laid)g(out)e(in)i(the)f(user's)g(bu\013er.)19 b(W)l(e)75 1770 y(extend)c(this)h(use)f(to)f(describ)q(e)j(ho)o(w)d (the)h(data)f(is)i(laid)g(out)e(in)i(the)f(\014le)h(as)f(w)o(ell.)20 b(Th)o(us)15 b(w)o(e)g(distinguish)75 1826 y(b)q(et)o(w)o(een)22 b(t)o(w)o(o)d(\(p)q(oten)o(tially)k(di\013eren)o(t\))e(deriv)o(ed)h (datat)o(yp)q(es)f(that)f(are)h(used:)33 b(the)21 b Fq(\014let)o(yp)q (e)p Ft(,)j(whic)o(h)75 1883 y(describ)q(es)c(the)f(la)o(y)o(out)f(in)i (the)e(\014le,)j(and)d(the)h Fq(buft)o(yp)q(e)p Ft(,)i(whic)o(h)f (describ)q(es)g(the)f(la)o(y)o(out)f(in)h(the)g(user's)75 1939 y(bu\013er.)g(In)12 b(addition,)h(b)q(oth)e Fq(\014let)o(yp)q(e)i Ft(and)f Fq(buft)o(yp)q(e)h Ft(are)e(deriv)o(ed)i(from)e(a)g(third)h Fq(MPI)f Ft(datat)o(yp)q(e,)g(referred)75 1995 y(to)k(as)g(the)h Fk(elementary)g Ft(datat)o(yp)q(e)f Fq(et)o(yp)q(e)p Ft(.)22 b(The)16 b(purp)q(ose)h(of)e(the)h(elemen)o(tary)g(datat)o(yp)q (e)f(is)h(to)f(ensure)75 2052 y(consistency)j(b)q(et)o(w)o(een)e(the)h (t)o(yp)q(e)g(signatures)f(of)h Fq(\014let)o(yp)q(e)h Ft(and)e Fq(buft)o(yp)q(e)p Ft(.)26 b(O\013sets)16 b(for)g(accessing)i (data)75 2108 y(within)e(the)g(\014le)g(are)f(expressed)h(as)f(an)g(in) o(tegral)g(n)o(um)o(b)q(er)h(of)e Fq(et)o(yp)q(e)j Ft(items.)166 2165 y(The)d(\014let)o(yp)q(e)g(de\014nes)h(a)e(data)f(pattern)h(that)g (is)h(replicated)h(throughout)e(the)g(\014le)i(\(or)d(part)h(of)g(the) 75 2221 y(\014le|see)19 b(the)f(concept)f(of)g(displacemen)o(t)i(b)q (elo)o(w\))f(to)f(tile)h(the)f(\014le)i(data.)25 b(It)18 b(should)g(b)q(e)g(noted)g(that)75 2278 y Fq(MPI)g Ft(deriv)o(ed)h (datat)o(yp)q(es)e(consist)i(of)e(\014elds)j(of)d(data)h(that)f(are)h (lo)q(cated)h(at)e(sp)q(eci\014ed)k(o\013sets.)27 b(This)75 2334 y(can)15 b(lea)o(v)o(e)h(\\holes")f(b)q(et)o(w)o(een)h(the)f (\014elds,)h(that)e(do)i(not)e(con)o(tain)i(an)o(y)f(data.)k(In)d(the)f (con)o(text)g(of)f(tiling)75 2391 y(the)g(\014le)g(with)g(the)g (\014let)o(yp)q(e,)g(the)g(pro)q(cess)g(can)g(only)g(access)f(the)h (\014le)h(data)e(that)g(matc)o(hes)g(items)g(in)i(the)75 2447 y(\014let)o(yp)q(e.)21 b(It)15 b(cannot)g(access)h(\014le)g(data)e (that)h(falls)h(under)g(holes)g(\(see)f(Figure)g(10.1\).)166 2504 y(Data)f(whic)o(h)i(resides)h(in)f(holes)g(can)f(b)q(e)i(accessed) f(b)o(y)f(other)g(pro)q(cesses)h(whic)o(h)g(use)g(complemen-)75 2560 y(tary)d(\014let)o(yp)q(es)i(\(see)f(Figure)h(10.2\).)j(Th)o(us,)c (\014le)h(data)e(can)h(b)q(e)h(distributed)h(among)d(parallel)j(pro)q (cesses)75 2616 y(in)i(disjoin)o(t)f(c)o(h)o(unks.)26 b(Filet)o(yp)q(e)18 b(constructors)e(help)j(the)e(user)g(create)f (complemen)o(tary)i(\014let)o(yp)q(es)g(for)75 2673 y(common)f (distribution)i(patterns,)d(suc)o(h)i(as)f(broadcast/reduce,)g (scatter/gather,)f(and)h(HPF)g(distri-)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(3)375 289 y @beginspecial 0 @llx 0 @lly 592 @urx 141 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-filetype.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -54.0 181.0 translate 0.900 -0.900 scale 0.500 setlinewidth % Polyline n 179 84 m 179 69 l 159 69 l 159 84 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 199 84 m 199 69 l 179 69 l 179 84 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 179 164 m 179 149 l 59 149 l 59 164 l clp gs col-1 s gr % Polyline n 99 164 m 99 149 l 79 149 l 79 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 119 164 m 119 149 l 99 149 l 99 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 299 164 m 299 149 l 179 149 l 179 164 l clp gs col-1 s gr % Polyline n 219 164 m 219 149 l 199 149 l 199 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 239 164 m 239 149 l 219 149 l 219 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 419 164 m 419 149 l 299 149 l 299 164 l clp gs col-1 s gr % Polyline n 339 164 m 339 149 l 319 149 l 319 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 359 164 m 359 149 l 339 149 l 339 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 539 164 m 539 149 l 419 149 l 419 164 l clp gs col-1 s gr % Polyline n 459 164 m 459 149 l 439 149 l 439 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 479 164 m 479 149 l 459 149 l 459 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 659 164 m 659 149 l 539 149 l 539 164 l clp gs col-1 s gr % Polyline n 579 164 m 579 149 l 559 149 l 559 164 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 84 m 259 69 l 139 69 l 139 84 l clp gs col-1 s gr % Polyline n 599 164 m 599 149 l 579 149 l 579 164 l clp gs 0.85 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 179 104 m gs 1 -1 scale (holes) dup stringwidth pop 2 div neg 0 rmoveto col-1 show gr 0.000 setlinewidth % Ellipse n 694 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 704 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 714 156 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 1.000 setlinewidth % Polyline n 699 149 m 59 149 l 59 164 l 699 164 l gs col-1 s gr 0.500 setlinewidth % Interpolated spline n 244 189 m 232.036 184.536 227.036 182.036 224 179 curveto 222.482 177.482 221.232 174.982 219 169 curveto gs col-1 s gr n 219.923 177.194 m 219.000 169.000 l 223.670 175.796 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 244 194 m 180.787 195.844 153.287 194.594 134 189 curveto 126.939 186.952 118.189 181.952 99 169 curveto gs col-1 s gr n 104.512 175.133 m 99.000 169.000 l 106.750 171.818 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 189 m 327.853 187.647 331.603 186.397 334 184 curveto 336.397 181.603 337.647 177.853 339 169 curveto gs col-1 s gr n 335.814 176.606 m 339.000 169.000 l 339.768 177.210 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 194 m 382.589 191.880 410.089 189.380 429 184 curveto 434.743 182.366 442.243 178.616 459 169 curveto gs col-1 s gr n 451.066 171.247 m 459.000 169.000 l 453.057 174.716 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 319 199 m 448.285 202.925 504.535 200.425 544 189 curveto 551.064 186.955 559.814 181.955 579 169 curveto gs col-1 s gr n 571.251 171.819 m 579.000 169.000 l 573.489 175.134 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 11.00 scalefont setfont 59 139 m gs 1 -1 scale (tiling a file with the filetype:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 249 199 m gs 1 -1 scale (accesible data) col-1 show gr % Polyline n 159 59 m 159 44 l 139 44 l 139 59 l clp gs 0.85 setgray fill gr gs col-1 s gr % Interpolated spline n 159 99 m 155.926 96.344 154.676 95.094 154 94 curveto 152.488 91.554 151.238 87.804 149 79 curveto gs col-1 s gr n 149.033 87.246 m 149.000 79.000 l 152.909 86.261 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 194 99 m 208.466 98.119 214.716 96.869 219 94 curveto 222.030 91.971 224.530 88.221 229 79 curveto gs col-1 s gr n 223.711 85.326 m 229.000 79.000 l 227.310 87.071 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 59 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 573 387 a(Figure)15 b(10.1:)k(Tiling)e(a)e(\014le)h(using) g(a)f(\014let)o(yp)q(e)75 525 y(butions)h(\(see)f(Section)h(10.6\).)375 859 y @beginspecial 0 @llx 0 @lly 593 @urx 135 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-comp-filetypes.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -35.0 175.0 translate 0.900 -0.900 scale % Ellipse n 684 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 694 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 159 59 m 139 59 l 139 44 l 159 44 l clp gs col-1 s gr % Polyline n 259 134 m 259 119 l 139 119 l 139 134 l clp gs col-1 s gr % Polyline n 259 109 m 259 94 l 139 94 l 139 109 l clp gs col-1 s gr % Polyline n 259 84 m 259 69 l 139 69 l 139 84 l clp gs col-1 s gr % Polyline n 179 84 m 159 84 l 159 69 l 179 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 159 84 m 139 84 l 139 69 l 159 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 659 194 m 639 194 l 639 179 l 659 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 59 194 m 39 194 l 39 179 l 59 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 79 194 m 59 194 l 59 179 l 79 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 194 m 159 194 l 159 179 l 179 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 194 m 179 194 l 179 179 l 199 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 194 m 279 194 l 279 179 l 299 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 194 m 299 194 l 299 179 l 319 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 419 194 m 399 194 l 399 179 l 419 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 439 194 m 419 194 l 419 179 l 439 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 539 194 m 519 194 l 519 179 l 539 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 559 194 m 539 194 l 539 179 l 559 179 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 99 194 m 79 194 l 79 179 l 99 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 119 194 m 99 194 l 99 179 l 119 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 139 194 m 119 194 l 119 179 l 139 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 159 194 m 139 194 l 139 179 l 159 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 219 194 m 199 194 l 199 179 l 219 179 l clp gs 0.65 setgray fill gr gs col-1 s gr 0.000 setlinewidth % Ellipse n 674 186 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 239 194 m 219 194 l 219 179 l 239 179 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 39 169 m gs 1 -1 scale (tiling a file with the filetypes:) col-1 show gr % Polyline n 259 194 m 239 194 l 239 179 l 259 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 279 194 m 259 194 l 259 179 l 279 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 339 194 m 319 194 l 319 179 l 339 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 359 194 m 339 194 l 339 179 l 359 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 194 m 359 194 l 359 179 l 379 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 399 194 m 379 194 l 379 179 l 399 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 479 194 m 459 194 l 459 179 l 479 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 459 194 m 439 194 l 439 179 l 459 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 499 194 m 479 194 l 479 179 l 499 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 519 194 m 499 194 l 499 179 l 519 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 579 194 m 559 194 l 559 179 l 579 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 599 194 m 579 194 l 579 179 l 599 179 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 619 194 m 599 194 l 599 179 l 619 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 639 194 m 619 194 l 619 179 l 639 179 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 199 109 m 179 109 l 179 94 l 199 94 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 219 109 m 199 109 l 199 94 l 219 94 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 134 m 219 134 l 219 119 l 239 119 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 259 134 m 239 134 l 239 119 l 259 119 l clp gs 0.40 setgray fill gr gs col-1 s gr 1.000 setlinewidth % Polyline n 679 179 m 39 179 l 39 194 l 679 194 l gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 59 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (process 1 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 109 m gs 1 -1 scale (process 2 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 134 m gs 1 -1 scale (process 3 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 419 957 a(Figure)f(10.2:)k(P)o(artitioning)c(a)g(\014le)i (among)d(parallel)j(pro)q(cesses)166 1061 y(In)k(order)f(to)g(b)q (etter)g(illustrate)i(these)e(concepts,)i(let)f(us)f(consider)h(a)f (2-D)g(matrix,)h(stored)f(in)75 1117 y(ro)o(w)14 b(ma)s(jor)g(order)i (in)g(a)f(\014le,)h(that)f(is)h(to)f(b)q(e)h(transp)q(osed)f(and)h (partitioned)g(among)f(a)g(group)g(of)g(three)75 1174 y(pro)q(cesses)f(\(see)g(Figure)g(10.3\).)j(The)d(matrix)g(is)g(to)f(b) q(e)h(distributed)h(among)e(the)h(parallel)h(pro)q(cesses)f(in)75 1230 y(a)e(ro)o(w)g(cyclic)i(manner.)19 b(Eac)o(h)12 b(pro)q(cess)h(w)o(an)o(ts)e(to)h(store)g(in)h(its)g(o)o(wn)f(memory)g (the)g(transp)q(osed)h(p)q(ortion)75 1287 y(of)k(the)h(matrix)g(whic)o (h)h(is)f(assigned)h(to)e(it.)28 b(Using)19 b(appropriate)f Fq(\014let)o(yp)q(e)p Ft(s)h(and)f Fq(buft)o(yp)q(e)p Ft(s)i(allo)o(ws)e(the)75 1343 y(user)f(to)g(p)q(erform)g(that)f(task)h (v)o(ery)f(easily)l(.)28 b(In)17 b(addition,)i(the)e(elemen)o(tary)g (datat)o(yp)q(e)g(allo)o(ws)g(one)h(to)75 1400 y(ha)o(v)o(e)d(a)g(v)o (ery)g(generic)h(co)q(de)g(that)e(applies)j(to)d(an)o(y)h(t)o(yp)q(e)g (of)g(2-D)g(matrix.)375 2437 y @beginspecial 0 @llx 0 @lly 673 @urx 549 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-cyclic-trans.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -36.0 562.0 translate 0.900 -0.900 scale 0.500 setlinewidth % Polyline n 399 344 m 319 344 l 319 269 l 399 269 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 279 m 139 279 l 139 264 l 239 264 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 234 m 139 234 l 139 219 l 239 219 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 189 m 139 189 l 139 174 l 239 174 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 144 m 139 144 l 139 129 l 239 129 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 239 264 m 139 264 l 139 249 l 239 249 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 399 244 m 319 244 l 319 169 l 399 169 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 219 m 139 219 l 139 204 l 239 204 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 249 m 139 249 l 139 234 l 239 234 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 174 m 139 174 l 139 159 l 239 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 129 m 139 129 l 139 114 l 239 114 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 399 144 m 319 144 l 319 69 l 399 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 114 m 139 114 l 139 99 l 239 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 159 m 139 159 l 139 144 l 239 144 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 204 m 139 204 l 139 189 l 239 189 l clp gs 0.90 setgray fill gr gs col-1 s gr % Ellipse n 754 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Ellipse n 769 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Ellipse n 784 571 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr gs col-1 s gr % Polyline n 159 99 m 159 279 l gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 99 m 179 279 l 179 274 l gs col-1 s gr % Polyline n 199 99 m 199 279 l gs col-1 s gr % Polyline n 219 99 m 219 279 l gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 164 619 m gs 1 -1 scale (buftype \(all processes\)) dup stringwidth pop neg 0 rmoveto col-1 show gr % Polyline n 579 624 m 579 609 l 519 609 l 519 624 l clp gs col-1 s gr % Polyline n 519 624 m 519 609 l 499 609 l 499 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 499 624 m 499 609 l 439 609 l 439 624 l clp gs col-1 s gr % Polyline n 439 624 m 439 609 l 419 609 l 419 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 339 69 m 339 144 l gs col-1 s gr % Polyline n 359 69 m 359 144 l gs col-1 s gr % Polyline n 379 69 m 379 144 l gs col-1 s gr % Polyline n 319 84 m 399 84 l gs col-1 s gr % Polyline n 319 99 m 399 99 l gs col-1 s gr % Polyline n 319 114 m 399 114 l gs col-1 s gr % Polyline n 319 129 m 399 129 l gs col-1 s gr % Polyline n 319 184 m 399 184 l gs col-1 s gr % Polyline n 319 199 m 399 199 l gs col-1 s gr % Polyline n 319 214 m 399 214 l gs col-1 s gr % Polyline n 319 229 m 399 229 l gs col-1 s gr % Polyline n 319 284 m 399 284 l gs col-1 s gr % Polyline n 319 299 m 399 299 l gs col-1 s gr % Polyline n 319 329 m 399 329 l gs col-1 s gr % Polyline n 339 169 m 339 244 l gs col-1 s gr % Polyline n 359 169 m 359 244 l gs col-1 s gr % Polyline n 379 169 m 379 244 l gs col-1 s gr % Polyline n 379 269 m 379 344 l gs col-1 s gr % Polyline n 359 269 m 359 344 l gs col-1 s gr % Polyline n 339 269 m 339 344 l gs col-1 s gr % Polyline n 319 314 m 399 314 l gs col-1 s gr % Polyline n 179 434 m 179 419 l 159 419 l 159 434 l clp gs col-1 s gr % Polyline n 359 464 m 359 449 l 259 449 l 259 464 l clp gs col-1 s gr % Polyline n 459 449 m 459 464 l 359 464 l 359 449 l clp gs col-1 s gr % Polyline n 259 479 m 259 494 l 159 494 l 159 479 l clp gs col-1 s gr % Polyline n 459 479 m 459 494 l 359 494 l 359 479 l clp gs col-1 s gr % Polyline n 259 509 m 259 524 l 159 524 l 159 509 l clp gs col-1 s gr % Polyline n 359 509 m 359 524 l 259 524 l 259 509 l clp gs col-1 s gr % Polyline n 739 564 m 759 564 l gs col-1 s gr % Polyline n 739 579 m 759 579 l gs col-1 s gr % Polyline n 419 624 m 419 609 l 359 609 l 359 624 l clp gs col-1 s gr % Polyline n 359 624 m 359 609 l 339 609 l 339 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 339 624 m 339 609 l 279 609 l 279 624 l clp gs col-1 s gr % Polyline n 279 624 m 279 609 l 259 609 l 259 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 624 m 259 609 l 199 609 l 199 624 l clp gs col-1 s gr % Polyline n 199 624 m 199 609 l 179 609 l 179 624 l clp gs 0.85 setgray fill gr gs col-1 s gr % Polyline n 259 464 m 239 464 l 239 449 l 259 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 464 m 219 464 l 219 449 l 239 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 219 464 m 199 464 l 199 449 l 219 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 464 m 179 464 l 179 449 l 199 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 464 m 159 464 l 159 449 l 179 449 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 59 579 m 39 579 l 39 564 l 59 564 l clp gs 0.90 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 139 524 m gs 1 -1 scale (process 3 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr % Polyline n 79 579 m 59 579 l 59 564 l 79 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 99 579 m 79 579 l 79 564 l 99 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 119 579 m 99 579 l 99 564 l 119 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 139 579 m 119 579 l 119 564 l 139 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 359 579 m 339 579 l 339 564 l 359 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 379 579 m 359 579 l 359 564 l 379 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 399 579 m 379 579 l 379 564 l 399 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 419 579 m 399 579 l 399 564 l 419 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 439 579 m 419 579 l 419 564 l 439 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 659 579 m 639 579 l 639 564 l 659 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 679 579 m 659 579 l 659 564 l 679 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 699 579 m 679 579 l 679 564 l 699 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 719 579 m 699 579 l 699 564 l 719 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 739 579 m 719 579 l 719 564 l 739 564 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 494 m 279 494 l 279 479 l 299 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 279 494 m 259 494 l 259 479 l 279 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 319 494 m 299 494 l 299 479 l 319 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 339 494 m 319 494 l 319 479 l 339 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 359 494 m 339 494 l 339 479 l 359 479 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 524 m 359 524 l 359 509 l 379 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 399 524 m 379 524 l 379 509 l 399 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 419 524 m 399 524 l 399 509 l 419 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 439 524 m 419 524 l 419 509 l 439 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 459 524 m 439 524 l 439 509 l 459 509 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 159 579 m 139 579 l 139 564 l 159 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 179 579 m 159 579 l 159 564 l 179 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 199 579 m 179 579 l 179 564 l 199 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 219 579 m 199 579 l 199 564 l 219 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 579 m 219 579 l 219 564 l 239 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 579 m 239 579 l 239 564 l 259 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 279 579 m 259 579 l 259 564 l 279 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 299 579 m 279 579 l 279 564 l 299 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 319 579 m 299 579 l 299 564 l 319 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 339 579 m 319 579 l 319 564 l 339 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 459 579 m 439 579 l 439 564 l 459 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 479 579 m 459 579 l 459 564 l 479 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 499 579 m 479 579 l 479 564 l 499 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 519 579 m 499 579 l 499 564 l 519 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 539 579 m 519 579 l 519 564 l 539 564 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 559 579 m 539 579 l 539 564 l 559 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 599 579 m 579 579 l 579 564 l 599 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 579 579 m 559 579 l 559 564 l 579 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 619 579 m 599 579 l 599 564 l 619 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Polyline n 639 579 m 619 579 l 619 564 l 639 564 l clp gs 0.40 setgray fill gr gs col-1 s gr % Interpolated spline n 239 104 m 250.668 101.971 255.668 100.721 259 99 curveto 268.933 93.869 284.450 74.407 294 69 curveto 300.551 65.291 313.415 54.616 324 59 curveto 326.807 60.163 328.057 62.663 329 69 curveto gs col-1 s gr n 329.801 60.793 m 329.000 69.000 l 325.844 61.382 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 149 m 250.644 148.244 255.644 146.994 259 144 curveto 277.543 127.455 267.799 80.162 289 64 curveto 302.704 53.553 328.391 50.305 344 59 curveto 346.370 60.320 347.620 62.820 349 69 curveto gs col-1 s gr n 349.208 60.756 m 349.000 69.000 l 345.305 61.628 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 194 m 250.648 192.789 255.648 191.539 259 189 curveto 267.738 182.380 275.372 162.611 279 154 curveto 288.827 130.673 274.667 70.350 309 49 curveto 326.576 38.071 351.215 49.412 364 59 curveto 365.918 60.438 367.168 62.938 369 69 curveto gs col-1 s gr n 368.600 60.763 m 369.000 69.000 l 364.771 61.921 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 239 m 250.644 238.228 255.644 236.978 259 234 curveto 283.386 212.355 290.390 153.120 299 129 curveto 306.270 108.635 296.232 54.835 329 39 curveto 349.214 29.232 372.340 48.608 384 59 curveto 385.671 60.489 386.921 62.989 389 69 curveto gs col-1 s gr n 388.275 60.786 m 389.000 69.000 l 384.495 62.093 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 119 m 250.654 121.454 255.654 122.704 259 124 curveto 274.976 130.186 310.971 145.570 324 159 curveto 325.480 160.525 326.730 163.025 329 169 curveto gs col-1 s gr n 328.028 160.811 m 329.000 169.000 l 324.289 162.232 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 164 m 250.581 164.100 255.581 164.100 259 164 curveto 278.409 163.430 320.511 145.420 344 159 curveto 346.313 160.337 347.563 162.837 349 169 curveto gs col-1 s gr n 349.131 160.755 m 349.000 169.000 l 345.236 161.663 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 209 m 250.636 206.383 255.636 205.133 259 204 curveto 283.853 195.628 323.491 144.162 364 159 curveto 366.973 160.089 368.223 162.589 369 169 curveto gs col-1 s gr n 370.023 160.817 m 369.000 169.000 l 366.052 161.299 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 254 m 250.666 252.137 255.666 250.887 259 249 curveto 268.147 243.821 283.511 227.619 289 219 curveto 296.572 207.110 294.164 175.146 309 164 curveto 328.055 149.685 363.140 147.049 384 159 curveto 386.327 160.333 387.577 162.833 389 169 curveto gs col-1 s gr n 389.150 160.755 m 389.000 169.000 l 385.253 161.655 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 134 m 250.647 135.150 255.647 136.400 259 139 curveto 288.594 161.953 309.637 231.415 324 259 curveto 324.882 260.695 326.132 263.195 329 269 curveto gs col-1 s gr n 327.250 260.942 m 329.000 269.000 l 323.663 262.714 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 179 m 250.656 180.494 255.656 181.744 259 184 curveto 273.708 193.922 285.963 228.455 299 239 curveto 308.440 246.636 334.739 250.151 344 259 curveto 345.577 260.507 346.827 263.007 349 269 curveto gs col-1 s gr n 348.153 260.797 m 349.000 269.000 l 344.393 262.161 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 224 m 250.662 226.332 255.662 227.582 259 229 curveto 269.183 233.327 288.080 250.409 299 254 curveto 314.092 258.962 348.484 251.864 364 259 curveto 366.524 260.161 369.024 262.661 374 269 curveto gs col-1 s gr n 370.633 261.472 m 374.000 269.000 l 367.487 263.942 l gs 2 setlinejoin col-1 s gr % Interpolated spline n 239 269 m 250.469 265.757 255.469 264.507 259 264 curveto 287.568 259.902 349.983 238.905 384 259 curveto 386.280 260.347 387.530 262.847 389 269 curveto gs col-1 s gr n 389.086 260.754 m 389.000 269.000 l 385.196 261.684 l gs 2 setlinejoin col-1 s gr /Helvetica findfont 12.00 scalefont setfont 39 29 m gs 1 -1 scale (logical view: partition file in row cyclic pattern and transpose) col-1 show gr /Helvetica findfont 12.00 scalefont setfont 39 399 m gs 1 -1 scale (implementation using etype, filetypes, and buftypes) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 119 m gs 1 -1 scale (file structure) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 84 m gs 1 -1 scale (process 1 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 184 m gs 1 -1 scale (process 2 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 419 284 m gs 1 -1 scale (process 3 buffer) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 39 554 m gs 1 -1 scale (actual layout in the file:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 434 m gs 1 -1 scale (etype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 464 m gs 1 -1 scale (process 1 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 139 494 m gs 1 -1 scale (process 2 filetype) dup stringwidth pop neg 0 rmoveto col-1 show gr $F2psEnd %%EndDocument @endspecial 420 2535 a(Figure)h(10.3:)i(T)l(ransp)q(osing)e(and)f (partitioning)h(a)f(2-D)g(matrix)166 2641 y(Note)d(that)h(using)g Fq(MPI)f Ft(deriv)o(ed)i(datat)o(yp)q(es)e(leads)i(to)e(the)h(p)q (ossibilit)o(y)i(of)d(v)o(ery)h(\015exible)i(patterns.)75 2698 y(F)l(or)d(example,)i(the)f(\014let)o(yp)q(es)h(need)g(not)e (distribute)i(the)f(data)f(in)i(rank)f(order.)18 b(In)c(addition,)g (there)f(can)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Ft(4)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(b)q(e)17 b(o)o(v)o(erlaps)g(b)q(et)o(w)o(een)f(the)h (data)f(items)h(that)f(are)g(accessed)h(b)o(y)g(di\013eren)o(t)g(pro)q (cesses.)24 b(The)17 b(extreme)75 106 y(case)e(of)g(full)i(o)o(v)o (erlap)d(is)i(the)f(broadcast/reduce)h(pattern.)166 162 y(Using)c(the)f(\014let)o(yp)q(e)i(allo)o(ws)e(a)g(certain)h(access)g (pattern)f(to)f(b)q(e)i(established.)21 b(But)11 b(it)h(is)g(conceiv)m (able)75 219 y(that)j(a)h(single)i(pattern)d(w)o(ould)i(not)e(b)q(e)i (suitable)h(for)d(the)h(whole)h(\014le.)24 b(The)16 b(solution)h(is)f (to)g(de\014ne)h(an)75 275 y(absolute)d(b)o(yte)f(displacemen)o(t)i (from)e(the)g(b)q(eginning)j(of)d(the)h(\014le,)g(and)g(ha)o(v)o(e)f (the)h(access)f(pattern)g(start)75 332 y(from)f(that)f(displacemen)o (t.)21 b(Th)o(us)12 b(if)h(a)f(\014le)i(has)e(t)o(w)o(o)f(segmen)o(ts)h (that)f(need)j(to)d(b)q(e)j(accessed)f(in)g(di\013eren)o(t)75 388 y(patterns,)e(the)f(displacemen)o(t)j(for)d(the)g(second)h(pattern) g(will)h(skip)f(o)o(v)o(er)f(the)h(whole)g(\014rst)f(segmen)o(t.)18 b(This)75 444 y(mec)o(hanism)c(is)g(also)f(particularly)i(useful)f(for) f(handling)i(\014les)f(with)g(some)f(header)h(information)f(at)g(the)75 501 y(b)q(eginning)20 b(\(see)d(Figure)h(10.4\).)25 b(Use)17 b(of)g(\014le)i(headers)f(could)g(allo)o(w)g(the)f(supp)q(ort)h(of)f (heterogeneous)75 557 y(en)o(vironmen)o(ts)k(b)o(y)g(storing)g(a)f (\\standard")g(co)q(di\014cation)j(of)d(the)h(data)g(represen)o (tations)f(and)i(data)75 614 y(t)o(yp)q(es)15 b(of)g(the)g(\014le)h (data.)375 970 y @beginspecial 0 @llx 0 @lly 502 @urx 126 @ury 2880 @rwi @setspecial %%BeginDocument: figures/io-disp.eps /$F2psDict 200 dict def $F2psDict begin $F2psDict /mtrx matrix put /l {lineto} bind def /m {moveto} bind def /s {stroke} bind def /n {newpath} bind def /gs {gsave} bind def /gr {grestore} bind def /clp {closepath} bind def /graycol {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul 4 -2 roll mul setrgbcolor} bind def /col-1 {} def /col0 {0 0 0 setrgbcolor} bind def /col1 {0 0 1 setrgbcolor} bind def /col2 {0 1 0 setrgbcolor} bind def /col3 {0 1 1 setrgbcolor} bind def /col4 {1 0 0 setrgbcolor} bind def /col5 {1 0 1 setrgbcolor} bind def /col6 {1 1 0 setrgbcolor} bind def /col7 {1 1 1 setrgbcolor} bind def /col8 {.68 .85 .9 setrgbcolor} bind def /col9 {0 .39 0 setrgbcolor} bind def /col10 {.65 .17 .17 setrgbcolor} bind def /col11 {1 .51 0 setrgbcolor} bind def /col12 {.63 .13 .94 setrgbcolor} bind def /col13 {1 .75 .8 setrgbcolor} bind def /col14 {.7 .13 .13 setrgbcolor} bind def /col15 {1 .84 0 setrgbcolor} bind def /DrawEllipse { /endangle exch def /startangle exch def /yrad exch def /xrad exch def /y exch def /x exch def /savematrix mtrx currentmatrix def x y translate xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix } def end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 0 setlinecap 0 setlinejoin -36.0 189.0 translate 0.900 -0.900 scale % Ellipse n 574 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr % Ellipse n 594 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 1.000 setlinewidth % Polyline n 579 159 m 39 159 l 39 174 l 579 174 l gs col-1 s gr 0.500 setlinewidth % Polyline n 119 209 m 119 179 l gs col-1 s gr n 117.000 187.000 m 119.000 179.000 l 121.000 187.000 l gs 2 setlinejoin col-1 s gr % Polyline n 319 209 m 319 179 l gs col-1 s gr n 317.000 187.000 m 319.000 179.000 l 321.000 187.000 l gs 2 setlinejoin col-1 s gr % Polyline n 159 84 m 139 84 l 139 69 l 159 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 84 m 179 84 l 179 69 l 199 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 239 84 m 219 84 l 219 69 l 239 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 279 84 m 259 84 l 259 69 l 279 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 84 m 299 84 l 299 69 l 319 69 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 199 114 m 139 114 l 139 99 l 199 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 319 114 m 259 114 l 259 99 l 319 99 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 139 174 m 119 174 l 119 159 l 139 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 179 174 m 159 174 l 159 159 l 179 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 219 174 m 199 174 l 199 159 l 219 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 259 174 m 239 174 l 239 159 l 259 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 299 174 m 279 174 l 279 159 l 299 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 379 174 m 319 174 l 319 159 l 379 159 l clp gs 0.90 setgray fill gr gs col-1 s gr % Polyline n 499 174 m 439 174 l 439 159 l 499 159 l clp gs 0.90 setgray fill gr gs col-1 s gr 0.000 setlinewidth % Ellipse n 584 166 3 3 0 360 DrawEllipse gs 0.00 setgray fill gr 0.500 setlinewidth % Polyline n 179 84 m 159 84 l 159 69 l 179 69 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 324 204 m gs 1 -1 scale (second displacement) col-1 show gr % Polyline n 219 84 m 199 84 l 199 69 l 219 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 84 m 239 84 l 239 69 l 259 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 299 84 m 279 84 l 279 69 l 299 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 339 84 m 319 84 l 319 69 l 339 69 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 259 114 m 199 114 l 199 99 l 259 99 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 379 114 m 319 114 l 319 99 l 379 99 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 159 174 m 139 174 l 139 159 l 159 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 199 174 m 179 174 l 179 159 l 199 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 239 174 m 219 174 l 219 159 l 239 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 279 174 m 259 174 l 259 159 l 279 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 319 174 m 299 174 l 299 159 l 319 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 439 174 m 379 174 l 379 159 l 439 159 l clp gs 0.65 setgray fill gr gs col-1 s gr % Polyline n 559 174 m 499 174 l 499 159 l 559 159 l clp gs 0.65 setgray fill gr gs col-1 s gr /Helvetica findfont 11.00 scalefont setfont 119 84 m gs 1 -1 scale (first tiling) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 119 109 m gs 1 -1 scale (second tiling) dup stringwidth pop neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 39 149 m gs 1 -1 scale (file structure:) col-1 show gr /Helvetica findfont 11.00 scalefont setfont 79 169 m gs 1 -1 scale (header) dup stringwidth pop 2 div neg 0 rmoveto col-1 show gr /Helvetica findfont 11.00 scalefont setfont 124 204 m gs 1 -1 scale (first displacement) col-1 show gr $F2psEnd %%EndDocument @endspecial 704 1068 a(Figure)g(10.4:)j(Displacemen)o(ts)75 1233 y Fl(10.1.2)49 b(Data)17 b(Access)e(F)o(unctions)75 1319 y Ft(Data)f(is)i(mo)o(v)o(ed)f(b)q(et)o(w)o(een)g(\014les)h(and)g (pro)q(cesses)f(b)o(y)h(issuing)g(read)f(and)h(write)f(calls.)21 b(There)16 b(are)f(three)75 1375 y(orthogonal)20 b(asp)q(ects)i(to)e (data)h(access:)32 b(p)q(ositioning)23 b(\(explicit)g(o\013set)d(vs.)38 b(implicit)24 b(\014le)e(p)q(oin)o(ter\),)75 1432 y(sync)o(hronism)14 b(\(blo)q(c)o(king)h(vs.)k(non)o(blo)q(c)o(king\),)c(and)e(co)q (ordination)i(\(indep)q(enden)o(t)h(vs.)j(collectiv)o(e\).)i(All)75 1488 y(com)o(binations)c(of)f(these)h(data)f(access)g(routines,)h (including)j(t)o(w)o(o)15 b(t)o(yp)q(es)h(of)g(\014le)i(p)q(oin)o (ters,)f(individual)75 1545 y(and)e(shared,)g(are)g(pro)o(vided.)p 190 1597 1570 2 v 189 1653 2 57 v 215 1636 a Fo(p)q(ositioning)p 491 1653 V 500 1653 V 61 w(sync)o(hronism)p 850 1653 V 859 1653 V 356 w(co)q(ordination)p 1759 1653 V 859 1655 901 2 v 189 1710 2 57 v 491 1710 V 500 1710 V 850 1710 V 859 1710 V 884 1693 a Fk(indep)n(endent)p 1276 1710 V 181 w(c)n(ol)r(le)n(ctive)p 1759 1710 V 190 1711 1570 2 v 190 1720 V 189 1776 2 57 v 215 1759 a(explicit)p 491 1776 V 500 1776 V 169 w(blo)n(cking)p 850 1776 V 859 1776 V 202 w Ft(MPI)p 976 1759 14 2 v 16 w(Read)p 1276 1776 2 57 v 212 w(MPI)p 1394 1759 14 2 v 17 w(Read)p 1512 1759 V 17 w(all)p 1759 1776 2 57 v 189 1833 V 215 1816 a Fk(o\013sets)p 491 1833 V 500 1833 V 185 w Ft(\(sync)o (hronous\))p 850 1833 V 859 1833 V 80 w(MPI)p 976 1816 14 2 v 16 w(W)l(rite)p 1276 1833 2 57 v 202 w(MPI)p 1394 1816 14 2 v 17 w(W)l(rite)p 1522 1816 V 16 w(all)p 1759 1833 2 57 v 500 1834 1261 2 v 189 1889 2 57 v 491 1889 V 500 1889 V 525 1872 a Fk(nonblo)n(cking)p 850 1889 V 859 1889 V 127 w Ft(MPI)p 976 1872 14 2 v 16 w(Iread)p 1276 1889 2 57 v 211 w(MPI)p 1394 1872 14 2 v 17 w(Iread)p 1513 1872 V 16 w(all)p 1759 1889 2 57 v 189 1945 V 491 1945 V 500 1945 V 525 1928 a(\(async)o(hronous\))p 850 1945 V 859 1945 V 57 w(MPI)p 976 1928 14 2 v 16 w(Iwrite)p 1276 1945 2 57 v 196 w(MPI)p 1394 1928 14 2 v 17 w(Iwrite)p 1528 1928 V 16 w(all)p 1759 1945 2 57 v 190 1947 1570 2 v 189 2004 2 57 v 215 1987 a Fk(individual)p 491 2004 V 500 2004 V 116 w(blo)n(cking)p 850 2004 V 859 2004 V 202 w Ft(MPI)p 976 1987 14 2 v 16 w(Read)p 1093 1987 V 17 w(next)p 1276 2004 2 57 v 108 w(MPI)p 1394 1987 14 2 v 17 w(Read)p 1512 1987 V 17 w(next)p 1616 1987 V 16 w(all)p 1759 2004 2 57 v 189 2060 V 215 2043 a Fk(\014le)g(p)n (ointers)p 491 2060 V 500 2060 V 78 w Ft(\(sync)o(hronous\))p 850 2060 V 859 2060 V 80 w(MPI)p 976 2043 14 2 v 16 w(W)l(rite)p 1103 2043 V 17 w(next)p 1276 2060 2 57 v 98 w(MPI)p 1394 2043 14 2 v 17 w(W)l(rite)p 1522 2043 V 16 w(next)p 1625 2043 V 17 w(all)p 1759 2060 2 57 v 500 2062 1261 2 v 189 2116 2 57 v 491 2116 V 500 2116 V 525 2100 a Fk(nonblo)n(cking)p 850 2116 V 859 2116 V 127 w Ft(MPI)p 976 2100 14 2 v 16 w(Iread)p 1094 2100 V 17 w(next)p 1276 2116 2 57 v 107 w(MPI)p 1394 2100 14 2 v 17 w(Iread)p 1513 2100 V 16 w(next)p 1616 2100 V 17 w(all)p 1759 2116 2 57 v 189 2173 V 491 2173 V 500 2173 V 525 2156 a(\(async)o(hronous\))p 850 2173 V 859 2173 V 57 w(MPI)p 976 2156 14 2 v 16 w(Iwrite)p 1109 2156 V 17 w(next)p 1276 2173 2 57 v 92 w(MPI)p 1394 2156 14 2 v 17 w(Iwrite)p 1528 2156 V 16 w(next)p 1631 2156 V 17 w(all)p 1759 2173 2 57 v 190 2175 1570 2 v 189 2231 2 57 v 215 2214 a Fk(shar)n(e)n(d)p 491 2231 V 500 2231 V 186 w(blo)n(cking)p 850 2231 V 859 2231 V 202 w Ft(MPI)p 976 2214 14 2 v 16 w(Read)p 1093 2214 V 17 w(shared)p 1276 2231 2 57 v 66 w(MPI)p 1394 2214 14 2 v 17 w(Read)p 1512 2214 V 17 w(shared)p 1658 2214 V 16 w(all)p 1759 2231 2 57 v 189 2287 V 215 2271 a Fk(\014le)g(p)n (ointer)p 491 2287 V 500 2287 V 97 w Ft(\(sync)o(hronous\))p 850 2287 V 859 2287 V 80 w(MPI)p 976 2271 14 2 v 16 w(W)l(rite)p 1103 2271 V 17 w(shared)p 1276 2287 2 57 v 56 w(MPI)p 1394 2271 14 2 v 17 w(W)l(rite)p 1522 2271 V 16 w(shared)p 1667 2271 V 17 w(all)p 1759 2287 2 57 v 500 2289 1261 2 v 189 2344 2 57 v 491 2344 V 500 2344 V 525 2327 a Fk(nonblo)n(cking)p 850 2344 V 859 2344 V 127 w Ft(MPI)p 976 2327 14 2 v 16 w(Iread)p 1094 2327 V 17 w(shared)p 1276 2344 2 57 v 65 w(MPI)p 1394 2327 14 2 v 17 w(Iread)p 1513 2327 V 16 w(shared)p 1658 2327 V 17 w(all)p 1759 2344 2 57 v 189 2400 V 491 2400 V 500 2400 V 525 2383 a(\(async)o(hronous\))p 850 2400 V 859 2400 V 57 w(MPI)p 976 2383 14 2 v 16 w(Iwrite)p 1109 2383 V 17 w(shared)p 1276 2400 2 57 v 50 w(MPI)p 1394 2383 14 2 v 17 w(Iwrite)p 1528 2383 V 16 w(shared)p 1673 2383 V 17 w(all)p 1759 2400 2 57 v 190 2402 1570 2 v 166 2478 a(UNIX)h(read\(\))e(and)i (write\(\))e(are)h(blo)q(c)o(king,)h(indep)q(enden)o(t)i(op)q (erations,)d(and)g(use)h(individual)i(\014le)75 2534 y(p)q(oin)o(ters.)i(The)c Fq(MPI)e Ft(equiv)m(alen)o(ts)j(are)e Fq(MPI)p 852 2534 14 2 v 16 w(READ)p 987 2534 V 17 w(NEXT)h Ft(and)f Fq(MPI)p 1310 2534 V 16 w(WRITE)p 1469 2534 V 16 w(NEXT)p Ft(.)166 2591 y(Op)q(erations)f(with)g(explicit)h (o\013sets)d(are)h(describ)q(ed)i(in)f(Section)h(10.3.)j(Op)q(erations) c(with)f(individ-)75 2647 y(ual)i(\014le)h(p)q(oin)o(ters)g(are)e (describ)q(ed)j(in)f(Section)g(10.4.)i(And)e(op)q(erations)f(with)g (shared)g(\014le)h(p)q(oin)o(ters)f(are)75 2704 y(describ)q(ed)i(in)f (Section)g(10.5.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(5)75 49 y Fl(10.1.3)49 b(P)o(ositioning)75 135 y Ft(UNIX)19 b(\014le)g(systems)f(traditionally)i(main)o(tain)f(a)f (system)g(\014le)h(p)q(oin)o(ter)g(sp)q(ecifying)h(what)e(o\013set)f (will)75 192 y(b)q(e)k(used)f(for)f(the)h(read)g(or)f(write)h(op)q (eration.)34 b(The)20 b(problem)h(with)f(this)g(in)o(terface)g(is)h (that)e(it)h(w)o(as)75 248 y(primarily)f(designed)g(for)e(\014les)h(b)q (eing)h(accessed)f(b)o(y)f(a)h(single)h(pro)q(cess.)27 b(In)18 b(a)f(parallel)i(en)o(vironmen)o(t,)75 304 y(w)o(e)14 b(m)o(ust)f(decide)j(whether)e(a)f(\014le)i(p)q(oin)o(ter)g(is)f (shared)g(b)o(y)g(m)o(ultiple)i(pro)q(cesses)e(or)g(if)g(an)g (individual)j(\014le)75 361 y(p)q(oin)o(ter)d(will)i(b)q(e)f(main)o (tained)g(b)o(y)f(eac)o(h)g(pro)q(cess.)19 b(In)c(addition,)g(parallel) h(programs)c(do)i(not)g(generally)75 417 y(exhibit)19 b(lo)q(calit)o(y)f(of)f(reference)h(within)h(a)e(\014le)h([6)o(].)26 b(Instead,)18 b(they)f(tend)h(to)f(mo)o(v)o(e)f(b)q(et)o(w)o(een)i (distinct)75 474 y(non-con)o(tiguous)h(regions)f(of)g(a)g(\014le.)29 b(This)19 b(means)f(that)g(the)g(pro)q(cess)g(m)o(ust)g(seek)g(on)g (almost)g(ev)o(ery)75 530 y(read)12 b(or)g(write)g(op)q(eration.)19 b(Moreo)o(v)o(er,)11 b(in)i(m)o(ulti-threaded)h(en)o(vironmen)o(ts)e (or)g(when)h(p)q(erforming)g(I/O)75 587 y(async)o(hronously)l(,)k(it)g (is)g(di\016cult)i(to)d(ensure)h(that)f(the)h(\014le)g(p)q(oin)o(ter)h (will)g(b)q(e)f(in)h(the)f(correct)f(p)q(osition)75 643 y(when)g(the)f(read)g(or)g(write)g(o)q(ccurs.)166 700 y Fq(MPI)10 b Ft(pro)o(vides)i(separate)e(routines)i(for)e(p)q (ositioning)j(with)e(explicit)j(o\013sets,)c(individual)k(\014le)e(p)q (oin)o(t-)75 756 y(ers,)18 b(and)g(shared)g(\014le)g(p)q(oin)o(ters.)28 b(The)18 b(explicit)i(o\013set)d(op)q(erations)h(require)g(the)g(user)g (to)f(sp)q(ecify)i(an)75 813 y(o\013set,)e(and)h(act)f(as)h(atomic)f (seek-and-read)i(or)e(seek-and-write)i(op)q(erations.)28 b(The)18 b(individual)j(and)75 869 y(shared)d(\014le)h(p)q(oin)o(ter)g (op)q(erations)f(use)g(distinct)i(system)d(main)o(tained)i(o\013sets)e (for)h(p)q(ositioning.)30 b(The)75 925 y(di\013eren)o(t)21 b(p)q(ositioning)i(metho)q(ds)f(are)f(orthogonal;)i(they)e(ma)o(y)f(b)q (e)i(mixed)g(within)h(the)e(same)g(pro-)75 982 y(gram,)15 b(and)i(they)f(do)g(not)g(a\013ect)f(eac)o(h)h(other.)23 b(In)17 b(other)e(w)o(ords,)h(an)g(individual)j(\014le)e(p)q(oin)o (ter's)g(v)m(alue)75 1038 y(will)h(b)q(e)e(unc)o(hanged)h(b)o(y)f (executing)h(explicit)h(o\013set)d(op)q(erations)h(or)g(shared)g (\014le)h(p)q(oin)o(ter)f(op)q(erations.)75 1095 y(The)g(data)g(access) g(routines)g(whic)o(h)h(accept)g(explicit)h(o\013sets)d(ha)o(v)o(e)h (no)g(extensions)g(\(e.g.)22 b Fq(MPI)p 1741 1095 14 2 v 16 w(XXX)p Ft(\),)75 1151 y(the)16 b(individual)k(\014le)d(p)q(oin) o(ter)f(routines)h(ha)o(v)o(e)p 884 1151 V 32 w Fq(NEXT)g Ft(app)q(ended)h(\(e.g.)j Fq(MPI)p 1435 1151 V 16 w(XXX)p 1541 1151 V 18 w(NEXT)p Ft(\),)16 b(and)g(the)75 1208 y(shared)i(\014le)i(p)q(oin)o(ter)f(routines)f(ha)o(v)o(e)p 747 1208 V 35 w Fq(SHARED)h Ft(app)q(ended)h(\(e.g.)29 b Fq(MPI)p 1366 1208 V 15 w(XXX)p 1471 1208 V 18 w(SHARED)p Ft(\).)18 b(In)h(order)75 1264 y(to)d(allo)o(w)g(the)g(implicit)j (o\013sets)c(to)h(b)q(e)h(set,)f(t)o(w)o(o)f(seek)h(routines)h(are)f (also)g(pro)o(vided)h(\()p Fq(MPI)p 1662 1264 V 15 w(SEEK)g Ft(and)75 1321 y Fq(MPI)p 160 1321 V 16 w(SEEK)p 287 1321 V 16 w(SHARED)p Ft(\).)166 1377 y(In)c(general,)h(\014le)g(p)q (oin)o(ter)f(op)q(erations)g(ha)o(v)o(e)g(the)g(same)g(seman)o(tics)g (as)f(explicit)j(o\013set)d(op)q(erations,)75 1434 y(with)k(the)f Fq(o\013set)g Ft(argumen)o(t)g(set)g(to)f(the)h(curren)o(t)h(v)m(alue)g (of)f(the)g(system-main)o(tained)h(\014le)g(p)q(oin)o(ter.)75 1554 y Fq(Explicit)g(O\013sets)75 1639 y Ft(There)h(are)g(t)o(w)o(o)f (\\k)o(eys")g(to)h(describ)q(e)h(lo)q(cations)g(in)g(a)f(\014le:)25 b(an)17 b Fq(MPI)f Ft(datat)o(yp)q(e)h(and)g(an)g(o\013set.)25 b Fq(MPI)75 1696 y Ft(datat)o(yp)q(es)14 b(are)h(used)h(as)f (templates,)g(tiling)h(the)f(\014le,)h(and)f(an)g(o\013set)g (determines)h(an)f(initial)i(p)q(osition)75 1752 y(for)g(transfers.)26 b(O\013sets)18 b(are)f(expressed)h(as)g(an)f(in)o(tegral)h(n)o(um)o(b)q (er)g(of)f(elemen)o(tary)h(datat)o(yp)q(e)f(\()p Fq(et)o(yp)q(e)p Ft(\))75 1809 y(items)d(relativ)o(e)h(to)e(the)i Fq(\014let)o(yp)q(e)p Ft(.)21 b(Holes)14 b(in)h(the)f Fq(\014let)o(yp)q(e)i Ft(are)e(ignored)g(when)h(calculating)h(the)e(p)q(osition)75 1865 y(of)f(an)h(o\013set)e(\(e.g.)19 b(an)13 b(o\013set)g(of)g(2)g (for)g(pro)q(cess)h(1)f(in)h(Figure)g(10.2)e(denotes)i(the)g(7th)f Fq(et)o(yp)q(e)i Ft(in)f(the)g(\014le\).)75 1922 y(The)j Fq(et)o(yp)q(e)i Ft(argumen)o(t)d(is)i(asso)q(ciated)g(with)f(a)g (\014le)h(and)g(used)f(to)g(express)h(the)f Fq(\014let)o(yp)q(e)p Ft(,)i Fq(buft)o(yp)q(e)g Ft(and)75 1978 y Fq(o\013set)h Ft(argumen)o(ts.)31 b(Therefore,)20 b(the)g Fq(\014let)o(yp)q(e)h Ft(and)e Fq(buft)o(yp)q(e)i Ft(datat)o(yp)q(es)e(m)o(ust)g(b)q(e)h (directly)h(deriv)o(ed)75 2035 y(from)14 b Fq(et)o(yp)q(e)p Ft(,)i(or)f(their)h(t)o(yp)q(e)f(signatures)g(m)o(ust)g(b)q(e)h(a)f(m)o (ultiple)i(of)e(the)g Fq(et)o(yp)q(e)h Ft(signature.)75 2155 y Fq(File)f(P)o(ointers)75 2240 y Ft(When)h(a)f(\014le)h(is)g(op)q (ened,)h(the)e(system)g(creates)g(a)h(set)f(of)g(\014le)h(p)q(oin)o (ters)g(to)f(k)o(eep)h(trac)o(k)e(of)i(the)f(curren)o(t)75 2297 y(\014le)j(p)q(osition.)25 b(One)18 b(is)f(a)g(global)g(\014le)h (p)q(oin)o(ter,)f Fk(shar)n(e)n(d)f Ft(b)o(y)h(all)h(the)f(pro)q (cesses)g(in)g(the)g(comm)o(unicator)75 2353 y(group.)29 b(The)19 b(others)f(are)g(individual)k(\014le)e(p)q(oin)o(ters)f(lo)q (cal)g(to)f(eac)o(h)h(pro)q(cess)f(in)i(the)e(comm)o(unicator)75 2410 y(group,)c(and)i(can)f(b)q(e)h(up)q(dated)g Fk(indep)n(endently.)i Ft(A)d(shared)h(\014le)g(p)q(oin)o(ter)f(only)h(mak)o(es)f(sense)g(if)h (all)g(the)75 2466 y(pro)q(cesses)j(can)f(access)h(the)f(same)g (dataset.)29 b(This)19 b(means)f(that)g(all)h(the)g(pro)q(cesses)g (should)g(use)g(the)75 2523 y(same)c Fq(\014let)o(yp)q(e)i Ft(when)e(op)q(ening)i(the)e(\014le.)166 2579 y(The)f(main)g(seman)o (tic)h(issue)f(with)h(system-main)o(tained)f(\014le)h(p)q(oin)o(ters)g (is)f(ho)o(w)f(they)i(are)e(up)q(dated)75 2636 y(b)o(y)18 b(I/O)h(op)q(erations.)29 b(In)19 b(general,)g(eac)o(h)g(I/O)f(op)q (eration)h(lea)o(v)o(es)f(the)h(\014le)g(p)q(oin)o(ter)g(p)q(oin)o (ting)g(to)f(the)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Ft(6)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(next)15 b(data)e(item)i(after)f(the)h(last)f(one)h (that)f(w)o(as)f(accessed.)21 b(This)15 b(principle)i(applies)f(to)e(b) q(oth)h(t)o(yp)q(es)f(of)75 106 y(\014le)i(p)q(oin)o(ters)g (\(individual)i(and)d(shared\),)g(and)g(to)g(all)h(t)o(yp)q(es)f(of)g (I/O)h(op)q(erations.)166 162 y(It)f(is)h(p)q(ossible)h(to)d(formalize) i(the)f(up)q(date)h(pro)q(cedure)g(as)f(follo)o(ws:)189 277 y Fi(new)p 274 277 14 2 v 17 w(f)5 b(il)q(e)p 369 277 V 16 w(position)13 b Ft(=)g Fi(ol)q(d)p 670 277 V 15 w(position)e Ft(+)906 247 y Fi(siz)r(e)p Ft(\()p Fi(buf)5 b(ty)r(pe)p Ft(\))10 b Fh(\002)g Fi(buf)5 b(count)p 906 267 514 2 v 1052 309 a(siz)r(e)p Ft(\()p Fi(ety)r(pe)p Ft(\))75 393 y(where)24 b Fi(buf)5 b(count)23 b Ft(is)h(the)f(n)o(um)o (b)q(er)h(of)f(elemen)o(ts)h(of)f(t)o(yp)q(e)g Fi(buf)5 b(ty)r(pe)24 b Ft(to)e(b)q(e)i(accessed)g(and)g(where)75 449 y Fi(siz)r(e)p Ft(\()p Fi(dataty)r(pe)p Ft(\))19 b(giv)o(es)f(the)h(n)o(um)o(b)q(er)g(of)f(b)o(ytes)h(of)f(actual)h (data)f(\(excluding)i(holes\))f(that)f(comp)q(oses)75 506 y(the)d Fq(MPI)g Ft(datat)o(yp)q(e)g Fi(dataty)r(pe)p Ft(.)166 562 y(Another)j(complication)j(with)d(UNIX)i(I/O)f(op)q (erations,)g(is)g(that)f(the)h(system-main)o(tained)g(\014le)75 619 y(p)q(oin)o(ter)c(is)g(normally)f(only)h(up)q(dated)g(when)g(the)g (op)q(eration)f(completes.)20 b(A)o(t)14 b(that)g(stage,)f(it)h(is)h (kno)o(wn)75 675 y(exactly)h(ho)o(w)f(m)o(uc)o(h)h(data)f(w)o(as)g (actually)h(accessed)h(\(whic)o(h)f(can)g(b)q(e)g(di\013eren)o(t)g (from)f(the)h(amoun)o(t)f(re-)75 732 y(quested\),)i(and)g(the)g(\014le) h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o(y)f(that)f(amoun)o(t.)24 b(F)l(or)16 b(non)o(blo)q(c)o(king)j(accesses)e(made)75 788 y(using)e(an)g(individual)j(or)c(the)g(shared)h(\014le)h(p)q(oin)o (ter,)e(the)h(up)q(date)g(cannot)g(b)q(e)g(dela)o(y)o(ed)g(un)o(til)h (the)f(op)q(er-)75 845 y(ation)g(completes,)h(b)q(ecause)g(additional)h (accesses)e(can)g(b)q(e)h(initiated)h(b)q(efore)f(that)e(time)i(b)o(y)f (the)g(same)75 901 y(pro)q(cess)i(\(for)e(b)q(oth)i(t)o(yp)q(es)g(of)f (\014le)h(p)q(oin)o(ters\))g(or)f(b)o(y)h(other)f(pro)q(cesses)h(\(for) e(the)i(shared)g(\014le)g(p)q(oin)o(ter\).)75 958 y(Therefore,)c(the)h (\014le)g(p)q(oin)o(ter)g(m)o(ust)f(b)q(e)h(up)q(dated)g(at)f(the)h (outset,)f(b)o(y)g(the)g(amoun)o(t)g(of)g(data)g(requested.)166 1014 y(Similarly)l(,)k(when)e(blo)q(c)o(king)i(accesses)e(are)g(made)g (using)g(the)h(shared)f(\014le)h(p)q(oin)o(ter,)f(up)q(dating)h(the)75 1070 y(\014le)k(p)q(oin)o(ter)f(at)e(the)i(completion)h(of)e(eac)o(h)g (access)h(w)o(ould)g(ha)o(v)o(e)f(the)h(same)f(e\013ect)g(as)g (serializing)j(all)75 1127 y(blo)q(c)o(king)e(accesses)f(to)e(the)i (\014le.)28 b(In)18 b(order)f(to)g(prev)o(en)o(t)g(this,)h(the)g (shared)f(\014le)i(p)q(oin)o(ter)f(for)f(blo)q(c)o(king)75 1183 y(accesses)g(is)g(up)q(dated)g(at)f(the)h(b)q(eginning)i(of)d(eac) o(h)g(access)h(b)o(y)g(the)f(amoun)o(t)g(of)g(data)g(requested.)25 b(F)l(or)75 1240 y(blo)q(c)o(king)13 b(accesses)f(using)h(an)f (individual)j(\014le)d(p)q(oin)o(ter,)h(up)q(dating)g(the)f(\014le)g(p) q(oin)o(ter)h(at)e(the)h(completion)75 1296 y(of)j(eac)o(h)h(access)f (w)o(ould)h(b)q(e)g(p)q(erfectly)h(v)m(alid.)22 b(Ho)o(w)o(ev)o(er,)14 b(in)i(order)f(to)g(main)o(tain)h(the)g(same)f(seman)o(tics)75 1353 y(for)f(all)i(t)o(yp)q(es)f(of)f(accesses)h(using)h(\014le)f(p)q (oin)o(ters,)g(the)g(up)q(date)h(of)e(the)h(\014le)h(p)q(oin)o(ter)f (in)h(this)f(case)g(is)g(also)75 1409 y(made)g(at)g(the)g(b)q(eginning) i(of)e(the)g(access)h(b)o(y)f(the)g(amoun)o(t)g(of)f(data)h(requested.) 166 1466 y(Although)c(consisten)o(t,)g(and)g(seman)o(tically)h (cleaner,)g(up)q(dating)g(the)f(\014le)g(p)q(oin)o(ter)g(at)f(the)h (initiation)75 1522 y(of)20 b(all)g(I/O)h(op)q(erations)f(di\013ers)g (from)f(accepted)i(UNIX)f(practice,)i(and)e(ma)o(y)f(lead)i(to)e(unexp) q(ected)75 1579 y(results.)h(Consider)c(the)f(follo)o(wing)h(scenario:) 266 1685 y Fp(MPI_Read_Next\(fh,)21 b(buff,)j(buftype,)e(bufcount,)h (&status\);)266 1741 y(MPI_Write_Next\(fh,)e(buff,)i(buftype,)g (bufcount,)g(&status\);)166 1848 y Ft(If)f(the)g(\014rst)g(read)g(reac) o(hes)g(the)g(end)h(of)e(the)i(\014le)g(b)q(efore)f(completing,)j(the)d (\014le)h(p)q(oin)o(ter)f(will)75 1904 y(b)q(e)g(incremen)o(ted)h(b)o (y)e(the)h(amoun)o(t)e(of)h(data)g Fk(r)n(e)n(queste)n(d)f Ft(rather)h(than)h(the)f(amoun)o(t)g(of)g(data)f(read.)75 1960 y(Therefore,)c(the)h(\014le)g(p)q(oin)o(ter)g(will)h(p)q(oin)o(t)f (b)q(ey)o(ond)g(the)f(curren)o(t)g(end)h(of)f(the)h(\014le,)g(and)f (the)h(write)f(will)75 2017 y(lea)o(v)o(e)f(a)g(hole)h(in)g(the)f (\014le.)21 b(Ho)o(w)o(ev)o(er,)14 b(suc)o(h)h(a)g(problem)h(only)g(o)q (ccurs)f(if)h(reads)f(and)g(writes)g(are)g(mixed)75 2073 y(without)g(c)o(hec)o(king)i(for)d(the)i(end)g(of)e(the)i(\014le.)21 b(Although)16 b(common)f(in)h(a)f(single)i(pro)q(cess)e(w)o(orkstation) 75 2130 y(en)o(vironmen)o(t,)g(w)o(e)g(b)q(eliev)o(e)i(this)f(is)g (uncommon)f(in)h(a)f(parallel)i(en)o(vironmen)o(t.)75 2252 y Fl(10.1.4)49 b(Synchronism)75 2337 y Ft(The)20 b(explicit)i(o)o(v)o(erlap)d(of)g(computation)h(with)g(I/O,)g(hop)q (efully)h(impro)o(ving)g(p)q(erformance,)f(is)g(sup-)75 2394 y(p)q(orted)13 b(through)f(the)h(use)g(of)g(non)o(blo)q(c)o(king)h (data)e(access)h(routines.)19 b(Both)13 b(blo)q(c)o(king)h(and)f(non)o (blo)q(c)o(king)75 2450 y(v)o(ersions)h(of)g(all)h(data)e(access)h (routines)g(are)g(supp)q(orted.)20 b(The)14 b(non)o(blo)q(c)o(king)i(v) o(ersions)e(of)f(the)i(routines)75 2507 y(are)g(named)g Fq(MPI)p 382 2507 14 2 v 16 w(IXXX)p Ft(,)g(where)h(the)f Fq(I)g Ft(stands)g(for)f(immediate.)166 2563 y(A)j Fk(blo)n(cking)e Ft(I/O)j(call)g(will)g(blo)q(c)o(k)g(un)o(til)g(the)f(I/O)g(request)g (is)h(completed.)26 b(A)17 b Fk(nonblo)n(cking)e Ft(I/O)75 2620 y(call)f(only)f(initiates)h(an)f(I/O)g(op)q(eration,)g(but)g(do)q (es)g(not)f(w)o(ait)h(for)f(it)h(to)f(complete.)20 b(A)12 b(separate)h Fk(r)n(e)n(quest)75 2676 y(c)n(omplete)18 b Ft(call)h(\()p Fq(MPI)p 453 2676 V 15 w(W)l(AIT)p Ft(,)f Fq(MPI)p 694 2676 V 15 w(TEST)p Ft(,)g(or)g(an)o(y)f(of)h(their)g(v)m (arian)o(ts\))f(is)i(needed)g(to)e(complete)i(the)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fn(10.1.)34 b(INTR)o(ODUCTION)1278 b Ft(7)75 49 y(I/O)17 b(request,)g(i.e.,)h(to)e(certify)h(that)f(data)h (has)g(b)q(een)h(read/written,)f(and)g(it)g(is)g(safe)g(for)f(the)h (user)h(to)75 106 y(reuse)d(the)g(bu\013er.)20 b(With)15 b(suitable)i(hardw)o(are,)d(the)h(transfer)f(of)g(data)h(out/in)g(the)g (user's)g(bu\013er)g(ma)o(y)75 162 y(pro)q(ceed)h(concurren)o(tly)g (with)f(computation.)166 219 y(Note)f(that)f(just)h(b)q(ecause)i(a)e (non)o(blo)q(c)o(king)h(\(or)f(blo)q(c)o(king\))h(data)f(access)g (routine)h(completes)g(do)q(es)75 275 y(not)g(mean)h(that)f(the)h(data) f(is)h(actually)h(written)f(to)f(\\p)q(ermanen)o(t")g(storage.)20 b(All)e(of)d(the)h(data)f(access)75 332 y(routines)22 b(ma)o(y)g(bu\013er)g(data)f(to)h(impro)o(v)o(e)g(p)q(erformance.)40 b(The)23 b(only)f(w)o(a)o(y)f(to)h(guaran)o(tee)f(data)g(is)75 388 y(actually)16 b(written)f(to)g(storage)e(is)j(b)o(y)f(using)h(the)f Fq(MPI)p 1010 388 14 2 v 16 w(FILE)p 1117 388 V 16 w(SYNC)h Ft(call.)21 b(Ho)o(w)o(ev)o(er,)13 b(one)j(need)g(not)e(b)q(e)75 444 y(concerned)i(with)e(the)h(con)o(v)o(erse)f(problem|once)i(a)e (read)h(op)q(eration)f(completes,)h(the)g(data)f(is)h(alw)o(a)o(ys)75 501 y(a)o(v)m(ailable)i(in)f(the)f(user's)g(bu\013er.)75 623 y Fl(10.1.5)49 b(Co)q(o)o(rdination)75 708 y Ft(Global)21 b(data)f(accesses)g(ha)o(v)o(e)g(signi\014can)o(t)h(p)q(oten)o(tial)g (for)f(automatic)g(optimization,)i(pro)o(vided)f(the)75 765 y(I/O)d(system)f(can)h(recognize)h(an)f(op)q(eration)f(as)h(a)f (global)h(access.)28 b(Collectiv)o(e)19 b(op)q(erations)f(are)f(used)75 821 y(for)e(this)i(purp)q(ose.)22 b Fq(MPI)15 b Ft(supp)q(orts)h(b)q (oth)g(indep)q(enden)o(t)j(and)d(collectiv)o(e)h(v)o(ersions)f(of)g (all)h(data)e(access)75 878 y(routines.)39 b(Ev)o(ery)21 b(indep)q(enden)o(t)i(data)e(access)h(routine)f Fq(MPI)p 1184 878 V 16 w(XXX)h Ft(has)g(a)f(collectiv)o(e)i(coun)o(terpart)75 934 y Fq(MPI)p 160 934 V 16 w(XXX)p 266 934 V 17 w(ALL)p Ft(,)17 b(where)p 527 934 V 35 w Fq(ALL)g Ft(means)h(that)f(\\all")i (pro)q(cesses)f(in)h(the)f(comm)o(unicator)f(group)h(whic)o(h)75 991 y(op)q(ened)e(the)g(\014le)g(m)o(ust)e(participate.)166 1047 y(Indep)q(enden)o(t)23 b(calls)f(do)f(not)f(imply)j(an)o(y)d(co)q (ordination)i(among)e(pro)q(cesses.)38 b(Collectiv)o(e)22 b(calls)75 1104 y(imply)13 b(that)e(all)h(pro)q(cesses)g(b)q(elonging)i (to)d(the)g(comm)o(unicator)h(asso)q(ciated)f(with)h(the)g(op)q(ened)h (\014le)g(m)o(ust)75 1160 y(participate.)20 b(Ho)o(w)o(ev)o(er,)12 b(no)h(sync)o(hronization)i(pattern)d(b)q(et)o(w)o(een)i(those)f(pro)q (cesses)g(is)h(enforced)g(\(with)75 1217 y(the)h(exception)i(of)d (collectiv)o(e)j(data)e(accesses)g(using)h(a)f(shared)g(\014le)i(p)q (oin)o(ter|see)f(Section)g(10.5.2\).)166 1273 y(An)h Fk(indep)n(endent)e Ft(I/O)h(request)h(is)g(a)f(request)g(whic)o(h)h (is)g(executed)g(individual)q(ly)j(b)o(y)c(an)o(y)g(of)g(the)75 1329 y(pro)q(cesses)21 b(within)i(a)e(comm)o(unicator)f(group.)38 b(A)21 b Fk(c)n(ol)r(le)n(ctive)e Ft(I/O)j(request)f(is)h(a)e(request)i (whic)o(h)f(is)75 1386 y(executed)15 b(b)o(y)e Fk(al)r(l)h Ft(pro)q(cesses)g(within)g(a)g(comm)o(unicator)f(group.)19 b(The)14 b(completion)h(of)e(an)h(indep)q(enden)o(t)75 1442 y(call)j(only)g(dep)q(ends)h(on)e(the)h(activit)o(y)f(of)g(the)h (calling)h(pro)q(cess.)23 b(On)17 b(the)g(other)f(hand,)g(a)g(pro)q (cess)h(can)75 1499 y(\(but)k(is)g(not)f(required)i(to\))e(return)h (from)f(a)g(collectiv)o(e)j(call)f(as)e(so)q(on)h(as)f(its)h (participation)h(in)g(the)75 1555 y(collectiv)o(e)e(op)q(eration)e(is)g (completed.)30 b(The)18 b(completion)h(of)f(the)g(call,)h(ho)o(w)o(ev)o (er,)f(do)q(es)g(not)f(indicate)75 1612 y(that)e(other)h(pro)q(cesses)h (ha)o(v)o(e)e(completed)i(or)f(ev)o(en)h(started)e(the)h(I/O)h(op)q (eration.)22 b(Th)o(us,)16 b(a)g(collectiv)o(e)75 1668 y(call)g(ma)o(y)l(,)f(or)f(ma)o(y)h(not,)f(ha)o(v)o(e)h(the)g(e\013ect) g(of)g(sync)o(hronizing)i(all)f(calling)h(pro)q(cesses.)166 1725 y(F)l(rom)c(a)g(seman)o(tic)i(viewp)q(oin)o(t,)f(the)g(only)h (di\013erence)g(b)q(et)o(w)o(een)f(collectiv)o(e)h(op)q(erations)f(and) g(their)75 1781 y(indep)q(enden)o(t)25 b(coun)o(terparts)d(is)h(p)q (oten)o(tial)g(sync)o(hronization.)43 b(F)l(rom)21 b(a)i(p)q (erformance)f(viewp)q(oin)o(t,)75 1838 y(ho)o(w)o(ev)o(er,)14 b(collectiv)o(e)j(op)q(erations)e(ha)o(v)o(e)g(the)g(p)q(oten)o(tial)h (to)e(b)q(e)i(m)o(uc)o(h)f(faster)f(than)h(their)h(indep)q(enden)o(t)75 1894 y(coun)o(terparts.)75 2016 y Fl(10.1.6)49 b(Consistency)16 b(Semantics)75 2102 y Ft(Consistency)21 b(seman)o(tics)f(de\014ne)h (the)f(outcome)g(of)g(m)o(ultiple)i(accesses)e(to)g(a)g(single)h (\014le,)h(either)f(b)o(y)75 2158 y(man)o(y)15 b(pro)q(cesses)g(or)g(b) o(y)g(a)g(single)i(pro)q(cess)e(whic)o(h)h(has)f(op)q(ened)h(the)g (\014le)g(more)f(than)g(once.)166 2214 y Fq(MPI)j Ft(guaran)o(tees)g (that)g(non-o)o(v)o(erlapping)i(\014le)g(accesses)f(within)h(a)f (single)h(application)g(are)f(se-)75 2271 y(quen)o(tially)k(consisten)o (t,)f(and)g(hence,)h(app)q(ear)e(to)g(the)g(application)i(as)e(atomic.) 37 b(Ho)o(w)o(ev)o(er,)21 b(unlik)o(e)75 2327 y(UNIX,)e(the)h(default)f (seman)o(tics)g(for)g(o)o(v)o(erlapping)h(accesses)f(do)q(es)g(not)g (guaran)o(tee)f(sequen)o(tial)j(con-)75 2384 y(sistency)l(.)28 b(This)19 b(default)f(mo)q(de)g(is)g(referred)g(to)f(as)g Fg(MPI)p 1066 2384 13 2 v 14 w(RECKLESS)g Ft(mo)q(de.)28 b(A)o(tomic)18 b(access)f(can)h(b)q(e)75 2440 y(guaran)o(teed)h(for)f (o)o(v)o(erlapping)i(accesses)f(b)o(y)g(enabling)i Fg(MPI)p 1140 2440 V 14 w(CA)o(UTIOUS)c Ft(mo)q(de)i(for)f(the)h(\014le)i(via)e (the)75 2497 y Fq(MPI)p 160 2497 14 2 v 16 w(FILE)p 267 2497 V 15 w(CONTROL)e Ft(routine.)166 2553 y(File)g(consistency)g (seman)o(tics)f(outside)g(of)g(a)f(single)j Fq(MPI)d Ft(application)j(are)d(implemen)o(tation)i(de-)75 2610 y(p)q(enden)o(t.)i(No)10 b(\014le)h(consistency)g(seman)o(tics)g(are)f (implied)i(b)o(y)e(either)h Fg(MPI)p 1322 2610 13 2 v 14 w(RECKLESS)f Ft(or)f Fg(MPI)p 1672 2610 V 15 w(CA)o(UTIOUS)75 2666 y Ft(b)q(et)o(w)o(een)17 b(m)o(ultiple)h Fq(MPI)e Ft(applications,)i(or)e(outside)h(of)g(the)f Fq(MPI)g Ft(en)o(vironmen)o(t.)24 b(In)18 b(other)e(w)o(ords,)f(if)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Ft(8)1365 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Ft(t)o(w)o(o)18 b(di\013eren)o(t)h Fq(MPI)f Ft(applications)j(op)q(en)f(the)f(same)g(\014le,)h(the)f(b)q(eha)o (vior)h(and)f(atomicit)o(y)g(of)g(the)g(\014le)75 106 y(accesses)k(are)f(unde\014ned.)43 b(The)23 b Fg(MPI)p 762 106 13 2 v 14 w(CA)o(UTIOUS)d Ft(mo)q(de)j(enforces)f(read/write)g (atomicit)o(y)h(in)g(the)75 162 y Fg(MPI)p 152 162 V 14 w(COMM)p 296 162 V 15 w(W)o(ORLD)15 b Ft(comm)o(unicator)g(group)g (only)l(.)75 284 y Fl(10.1.7)49 b(End)17 b(of)f(File)75 370 y Ft(Unlik)o(e)k(UNIX)e(\014les,)i(the)e(end)h(of)e(\014le)i(is)g (not)f(absolute)g(and)g(iden)o(tical)i(for)e(all)h(pro)q(cesses)f (accessing)75 426 y(the)d(\014le.)21 b(It)15 b(dep)q(ends)i(on)e(the)g (\014let)o(yp)q(e)h(used)g(to)f(access)g(the)g(\014le)h(and)g(is)f (de\014ned)i(for)d(a)h(giv)o(en)h(pro)q(cess)75 483 y(as)h(the)g(lo)q (cation)h(of)f(the)g(b)o(yte)g(follo)o(wing)h(the)f(last)g(elemen)o (tary)g(datat)o(yp)q(e)g(item)h(accessible)h(b)o(y)e(that)75 539 y(pro)q(cess)d(\(excluding)i(the)d(holes\).)20 b(It)14 b(ma)o(y)f(happ)q(en)i(that)e(data)g(is)h(lo)q(cated)h(b)q(ey)o(ond)f (the)g(end)g(of)g(\014le)h(for)75 596 y(a)g(giv)o(en)h(pro)q(cess.)k (This)15 b(data)g(is)h(accessible)h(only)f(b)o(y)f(other)f(pro)q (cesses.)166 652 y(When)d(a)f(\014le)i(gro)o(ws,)e(b)q(ecause)h(of)f (more)g(data)g(b)q(eing)i(written)f(to)f(the)g(\014le)i(or)e(the)h (\014le)g(b)q(eing)h(resized,)75 708 y(the)k(end)h(of)f(\014le)i(of)e (all)h(pro)q(cesses)g(accessing)g(the)f(\014le)i(ma)o(y)d(c)o(hange,)i (and)f(data)g(no)o(w)g(accessible,)i(but)75 765 y(not)d(y)o(et)g (written)g(b)o(y)g(an)o(y)g(of)g(these)g(pro)q(cesses,)g(will)i(b)q(e)f (read)f(as)g(zero)q(es.)75 887 y Fl(10.1.8)49 b(File)17 b(La)o(y)o(out)75 972 y Ft(The)k(basic)g(access)g(routines)g(only)g(sp) q(ecify)g(ho)o(w)f(the)h(data)f(should)h(b)q(e)g(laid)h(out)e(in)i(a)e (virtual)h(\014le)75 1029 y(structure)d(\(the)g(\014let)o(yp)q(e\),)i (not)d(ho)o(w)h(that)g(\014le)h(structure)f(is)h(to)f(b)q(e)h(stored)e (on)i(one)f(or)g(more)g(disks.)75 1085 y(This)g(w)o(as)f(a)o(v)o(oided) g(b)q(ecause)i(it)f(is)g(exp)q(ected)g(that)f(the)h(mapping)g(of)f (\014les)h(to)f(disks)h(will)h(b)q(e)f(system)75 1142 y(sp)q(eci\014c,)c(and)d(an)o(y)g(sp)q(eci\014c)i(con)o(trol)e(o)o(v)o (er)f(\014le)j(la)o(y)o(out)d(w)o(ould)i(therefore)f(restrict)g (program)f(p)q(ortabilit)o(y)l(.)75 1198 y(Ho)o(w)o(ev)o(er,)i(there)h (are)g(still)i(cases)e(where)g(some)g(information)g(will)i(b)q(e)f (necessary)f(in)h(order)f(to)g(optimize)75 1255 y(disk)i(la)o(y)o(out.) j(This)d(information)f(is)g(pro)o(vided)h(as)e Fk(hints)h Ft(sp)q(eci\014ed)i(when)e(a)f(\014le)i(is)g(created.)k(Hin)o(ts)14 b(are)75 1311 y(advisory)f(only)l(,)h(and)g(do)f(not)g(c)o(hange)h(the) f(seman)o(tics)h(of)f(an)o(y)g(routines.)19 b(Instead,)14 b(they)f(allo)o(w)h(sp)q(eci\014c)75 1368 y(implemen)o(tations)i(to)f (increase)h(I/O)g(p)q(erformance.)75 1511 y Fm(10.2)60 b(File)19 b(Manipulation)75 1614 y Fl(10.2.1)49 b(Op)q(ening)18 b(a)e(File)h(\(Collective\))75 1794 y Fq(MPI)p 160 1794 14 2 v 16 w(OPEN\(comm,)c(\014lename,)i(amo)q(de,)g(disp,)h(et)o(yp)q (e,)g(\014let)o(yp)q(e,)g(hints,)g(fh\))117 1871 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])14 b(Comm)n(unicator)d(that)j(op)q(ens)h(the) f(\014le)g(\(handle\))117 1947 y(IN)155 b Fq(\014lename)430 b Ff([SAME])14 b(Name)e(of)i(\014le)g(to)f(b)q(e)i(op)q(ened)g (\(string\))117 2022 y(IN)155 b Fq(amo)q(de)461 b Ff([SAME])14 b(File)f(access)j(mo)q(de)d(\(in)o(teger\))117 2097 y(IN)155 b Fq(disp)513 b Ff(Absolute)15 b(displacemen)o(t)e(\(nonnegativ)o(e)g (o\013set\))117 2172 y(IN)155 b Fq(et)o(yp)q(e)487 b Ff([SAME])14 b(Elemen)o(tary)f(datat)o(yp)q(e)h(\(handle\))117 2247 y(IN)155 b Fq(\014let)o(yp)q(e)452 b Ff(Filet)o(yp)q(e)14 b(\(handle\))117 2322 y(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(to)g(the)g(\014le)g(system)g(\(handle\))117 2397 y(OUT)108 b Fq(fh)550 b Ff(Returned)15 b(\014le)f(handle)g(\(handle\))75 2522 y Fp(int)23 b(MPI)p 245 2522 15 2 v 17 w(Open\(MPI)p 454 2522 V 16 w(Comm)h(comm,)f(char)g(*filename,)g(MPI)p 1187 2522 V 16 w(Mode)h(amode,)f(MPI)p 1562 2522 V 17 w(Offset)g(disp,)393 2578 y(MPI)p 468 2578 V 17 w(Datatype)g(etype,)g (MPI)p 939 2578 V 17 w(Datatype)f(filetype,)h(MPI)p 1481 2578 V 17 w(Hints)g(hints,)393 2635 y(MPI)p 468 2635 V 17 w(File)g(*fh\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1161 b Ft(9)75 49 y Fp(MPI)p 150 49 15 2 v 17 w(OPEN\(COMM,)22 b(FILENAME,)h(AMODE,)g(DISP,)g(ETYPE,)g(FILETYPE,)g(HINTS,)g(FH,)g (IERROR\))170 106 y(CHARACTER)g(FILENAME\(*\))170 162 y(INTEGER)g(COMM,)h(AMODE,)f(ETYPE,)g(FILETYPE)170 219 y(INTEGER)g(HINTS,)g(FH,)h(IERROR)170 275 y(INTEGER*8)f(DISP)166 362 y Fq(MPI)p 251 362 14 2 v 16 w(OPEN)c Ft(op)q(ens)h(the)e(\014le)i (iden)o(ti\014ed)h(b)o(y)e(the)f(\014le)i(name)f Fq(\014lename)g Ft(on)f(all)i(pro)q(cesses)f(in)h(the)75 418 y Fq(comm)13 b Ft(comm)o(unicator)h(group.)19 b Fq(MPI)p 732 418 V 16 w(OPEN)c Ft(is)g(a)g(collectiv)o(e)h(routine;)f(all)g(pro)q(cesses)g (in)h(the)e(comm)o(u-)75 474 y(nicator)j(group)g(m)o(ust)g(call)i Fq(MPI)p 648 474 V 15 w(OPEN)p Ft(,)f(with)g(iden)o(tical)h(v)m(alues)g (for)d Fq(comm)p Ft(,)g Fq(\014lename)p Ft(,)i Fq(amo)q(de)p Ft(,)f(and)75 531 y Fq(et)o(yp)q(e)p Ft(.)j(\(V)l(alues)12 b(for)f Fq(disp)p Ft(,)i Fq(\014let)o(yp)q(e)p Ft(,)g(and)f Fq(hints)h Ft(ma)o(y)d(v)m(ary)l(.\))19 b(A)11 b(pro)q(cess)h(can)f(op) q(en)h(a)g(\014le)g(indep)q(enden)o(tly)75 587 y(of)k(other)h(pro)q (cesses)g(b)o(y)f(using)i(the)f Fg(MPI)p 789 587 13 2 v 14 w(COMM)p 933 587 V 15 w(SELF)f Ft(comm)o(unicator.)24 b(The)17 b(\014le)h(handle)g(returned,)75 644 y Fq(fh)p Ft(,)d(can)g(b)q(e)h(subsequen)o(tly)h(used)f(to)e(access)h(the)h (\014le)g(un)o(til)g(the)g(\014le)g(is)g(closed)g(using)g Fq(MPI)p 1643 644 14 2 v 15 w(CLOSE)p Ft(.)166 700 y(The)f(follo)o (wing)h(access)g(mo)q(des,)f(sp)q(eci\014ed)i(via)e Fq(amo)q(de)p Ft(,)g(are)g(supp)q(orted:)143 790 y Fh(\017)23 b Fg(MPI)p 266 790 13 2 v 14 w(RDONL)m(Y)15 b Ft(-)g(reading)h(only)l(,)143 883 y Fh(\017)23 b Fg(MPI)p 266 883 V 14 w(RD)o(WR)15 b Ft(-)g(reading)h(and)f(writing,)143 975 y Fh(\017)23 b Fg(MPI)p 266 975 V 14 w(WRONL)m(Y)15 b Ft(-)g(writing)h(only)l(,)143 1068 y Fh(\017)23 b Fg(MPI)p 266 1068 V 14 w(CREA)m(TE)13 b Ft(-)j(creating)f(\014le,)143 1160 y Fh(\017)23 b Fg(MPI)p 266 1160 V 14 w(DELETE)p 435 1160 V 15 w(ON)p 510 1160 V 15 w(CLOSE)14 b Ft(-)i(deleting)g(on)g(close.)75 1250 y(These)c(can)g(b)q(e)g(com)o(bined)h(in)f(C)g(b)o(y)f(using)i(the)f (bit)o(wise)g(OR)h(op)q(erator,)e(and)h(in)g(F)o(OR)l(TRAN)h(b)o(y)e (adding)75 1307 y(them.)189 1408 y Fk(A)n(dvic)n(e)18 b(to)h(implementors.)58 b Ft(The)18 b(v)m(alues)i(of)d(these)i(constan) o(ts)e(m)o(ust)h(b)q(e)h(de\014ned)g(suc)o(h)g(that)189 1465 y(the)14 b(bit)o(wise)h(OR)h(and)e(the)h(sum)f(of)g(an)o(y)g (distinct)i(set)e(of)g(these)h(constan)o(ts)e(is)i(equiv)m(alen)o(t.)22 b(\()p Fk(End)189 1521 y(of)16 b(advic)n(e)g(to)h(implementors.)p Ft(\))189 1623 y Fk(A)n(dvic)n(e)k(to)i(users.)81 b Ft(Some)22 b(\014le)h(attributes)e(are)h(inheren)o(tly)i(implemen)o(tation)f(dep)q (enden)o(t.)189 1680 y(These)17 b(attributes)g(m)o(ust)g(b)q(e)h(set)f (using)h(facilities)h(outside)f(the)f(scop)q(e)h(of)f Fq(MPI)p Ft(.)f(F)l(or)g(example,)189 1736 y(UNIX)e(\014le)h(access)f (p)q(ermissions)h(can)f(b)q(e)h(sp)q(eci\014ed)h(via)e(the)g(UNIX)g (umask.)19 b(\()p Fk(End)c(of)g(advic)n(e)g(to)189 1793 y(users.)p Ft(\))189 1895 y Fk(A)n(dvic)n(e)j(to)h(users.)57 b Ft(Note)18 b(that)g(the)g(UNIX)h(app)q(end)g(mo)q(de)f(is)h(not)f (supp)q(orted.)29 b(This)19 b(mo)q(de)189 1951 y(can)f(b)q(e)h(em)o (ulated)f(b)o(y)g(requesting)h(the)f(curren)o(t)g(\014le)h(size)g (\(see)f(Section)h(10.2.3\))d(and)i(seeking)189 2007 y(to)f(the)h(end)h(of)f(\014le)h(b)q(efore)f(eac)o(h)h(write)f(op)q (eration.)29 b Fe(Discussion)o(:)22 b Ff(Actually)m(,)15 b(the)i(seman)o(tics)g(of)189 2147 y(UNIX)f(app)q(end)h(mo)q(de)e(are)i (at)f(the)h(\014lesystem)f(lev)o(el,)g(and)g(accoun)o(t)g(for)g(all)f (applications)g(running)h(on)189 2203 y(the)c(system)f(\(not)g(just)h (a)f(single)g(application\).)16 b(UNIX)c(app)q(end)g(mo)q(de)e(cannot)i (b)q(e)g(fully)e(em)o(ulated)g(using)189 2259 y(the)k(ab)q(o)o(v)o(e.)k (Ho)o(w)13 b(imp)q(ortan)o(t)g(is)g(a)h(true)h(UNIX)f(app)q(end)g(mo)q (de?)19 b Ft(\()p Fk(End)d(of)g(advic)n(e)g(to)h(users.)p Ft(\))166 2444 y(The)i Fq(disp)h Ft(displacemen)o(t)h(argumen)o(t)d(sp) q(eci\014es)i(the)f(p)q(osition)h(\(absolute)f(o\013set)f(in)i(b)o (ytes)f(from)75 2501 y(the)d(b)q(eginning)j(of)d(the)g(\014le\),)h (where)g(the)f(\014le)h(is)g(to)f(b)q(e)h(op)q(ened.)24 b(The)17 b(constan)o(t)e Fg(MPI)p 1586 2501 V 15 w(OFFSET)p 1756 2501 V 13 w(ZERO)75 2557 y Ft(is)h(de\014ned)g(to)f(b)q(e)h (o\013set)e(zero.)189 2647 y Fk(A)n(dvic)n(e)d(to)j(users.)37 b Fq(Disp)12 b Ft(can)g(b)q(e)g(used)g(to)f(skip)h(headers,)g(or)f (when)h(the)f(\014le)i(includes)h(a)d(sequence)189 2704 y(of)f(data)h(segmen)o(ts)g(that)f(are)h(to)g(b)q(e)g(accessed)h(in)g (di\013eren)o(t)g(patterns.)18 b(\()p Fk(End)12 b(of)g(advic)n(e)h(to)g (users.)p Ft(\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Ft(10)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Ft(The)18 b Fq(et)o(yp)q(e)i Ft(argumen)o(t)d(sp)q (eci\014es)j(the)e(elemen)o(tary)g(datat)o(yp)q(e)g(used)g(to)g (construct)g(the)g Fq(\014let)o(yp)q(e)p Ft(,)75 106 y(and)h(also)f(the)h Fq(buft)o(yp)q(e)h Ft(used)f(in)h(read/write)e(op) q(erations.)30 b(O\013sets)18 b(in)o(to)h(the)g(\014le)g(are)f (measured)h(in)75 162 y(units)d(of)f Fq(et)o(yp)q(e)p Ft(.)166 219 y(The)20 b Fq(\014let)o(yp)q(e)i Ft(argumen)o(t)e(describ) q(es)h(what)f(part)g(of)g(the)g(data)g(in)h(the)f(\014le)h(is)g(b)q (eing)h(accessed.)75 275 y(Conceptually)l(,)15 b(the)f(\014le)h (starting)e(from)g Fq(disp)i Ft(is)f(tiled)h(b)o(y)f(rep)q(eated)g (copies)h(of)e Fq(\014let)o(yp)q(e)p Ft(,)i(un)o(til)g(the)f(end.)75 332 y(If)k Fq(\014let)o(yp)q(e)g Ft(has)f(holes)h(in)h(it,)e(then)h (the)f(data)g(in)h(the)g(holes)g(is)f(inaccessible)j(b)o(y)e(the)f (calling)i(pro)q(cess.)75 388 y(Ho)o(w)o(ev)o(er,)d(the)h Fq(disp)p Ft(,)h Fq(et)o(yp)q(e)h Ft(and)e Fq(\014let)o(yp)q(e)h Ft(argumen)o(ts)e(can)i(b)q(e)f(c)o(hanged)h(later)f(to)f(access)h(a)g (di\013eren)o(t)75 444 y(part)e(of)f(the)i(\014le.)166 501 y(The)c Fq(hints)h Ft(argumen)o(t)e(is)i(used)f(to)f(pro)o(vide)i (information)f(regarding)g(\014le)h(access)f(patterns)f(and)h(\014le)75 557 y(system)j(sp)q(eci\014cs)j(to)d(direct)i(optimization)g(\(see)f (section)g(10.2.4\).)k(The)c(constan)o(t)f Fg(MPI)p 1623 557 13 2 v 14 w(HINTS)p 1758 557 V 16 w(NULL)75 614 y Ft(refers)g(to)g(the)g(n)o(ull)h(hin)o(t,)g(and)f(can)h(b)q(e)f(used)h (when)g(no)f(hin)o(ts)h(need)g(to)e(b)q(e)i(sp)q(eci\014ed.)166 670 y Fq(Filename)i Ft(can)h(b)q(e)h(pre\014xed)g(with)f("mpi-impl:/")i (to)d(iden)o(tify)i(the)f(implemen)o(tation)i(that)d(the)75 727 y(op)q(en)e(call)h(w)o(ould)f(lik)o(e)h(to)e(use)h(if)g(m)o (ultiple)i(implemen)o(tations)f(co-exist.)k(Examples)c(of)e(implemen)o (ta-)75 783 y(tions)g(are:)143 869 y Fh(\017)23 b Fq(ufs:/)15 b Ft(-)g(UNIX)h(\014le)g(system,)143 960 y Fh(\017)23 b Fq(mpirun:/)14 b Ft(-)h(serv)o(er)g(comm)o(unicating)h(to)f(clien)o (t)h(using)g(mpi,)143 1051 y Fh(\017)23 b Fq(so)q(ck)o(et:/)15 b Ft(-)h(serv)o(er)e(comm)o(unicating)i(to)f(clien)o(t)h(using)g(so)q (c)o(k)o(et,)143 1142 y Fh(\017)23 b Fq(piofs:/)15 b Ft(-)g(IBM)g(AIX)h(P)o(arallel)g(I/O)g(File)g(System)f([1,)f(2].)166 1228 y(Files)e(are)f(op)q(ened)h(b)o(y)f(default)h(using)g Fq(MPI)p 901 1228 14 2 v 16 w(RECKLESS)g Ft(\014le)g(consistency)g (seman)o(tics)f(\(see)g(section)75 1284 y(10.1.6\).)17 b(The)d(more)e(stringen)o(t)i Fg(MPI)p 711 1284 13 2 v 14 w(CA)o(UTIOUS)d Ft(consistency)j(seman)o(tics,)f(required)i(for)d (atomicit)o(y)h(of)75 1341 y(o)o(v)o(erlapping)j(accesses,)f(can)g(b)q (e)h(set)f(using)h Fq(MPI)p 924 1341 14 2 v 16 w(FILE)p 1031 1341 V 15 w(CONTROL)p Ft(.)75 1461 y Fl(10.2.2)49 b(Closing)18 b(a)f(\014le)f(\(Collective\))75 1641 y Fq(MPI)p 160 1641 V 16 w(CLOSE\(fh\))117 1718 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))75 1843 y Fp(int)23 b(MPI)p 245 1843 15 2 v 17 w(Close\(MPI)p 478 1843 V 16 w(File)g(fh\))75 1929 y(MPI)p 150 1929 V 17 w(CLOSE\(FH,)f(IERROR\))170 1986 y(INTEGER)h(FH,)h(IERROR)166 2072 y Fq(MPI)p 251 2072 14 2 v 16 w(CLOSE)d Ft(closes)g(the)f(\014le)i (asso)q(ciated)e(with)h Fq(fh)p Ft(.)36 b(After)20 b(closing,)i(the)f (con)o(ten)o(t)e(of)h(the)h(\014le)75 2129 y(handle)16 b Fq(fh)f Ft(is)h(destro)o(y)o(ed.)j(F)l(uture)c(use)g(of)f Fq(fh)i Ft(is)f(erroneous.)20 b(If)15 b(the)g(\014le)h(w)o(as)e(op)q (ened)i(with)f(the)g(access)75 2185 y(mo)q(de)22 b Fg(MPI)p 281 2185 13 2 v 14 w(DELETE)p 450 2185 V 15 w(ON)p 525 2185 V 15 w(CLOSE)p Ft(,)f(the)h(\014le)h(is)f(deleted.)41 b(Ho)o(w)o(ev)o(er,)22 b(if)g(there)g(are)f(other)h(pro)q(cesses)75 2242 y(curren)o(tly)e(accessing)h(the)f(\014le,)h(the)f(status)f(of)h (the)g(\014le)g(and)g(the)g(b)q(eha)o(vior)h(of)e(future)h(accesses)g (b)o(y)75 2298 y(these)15 b(pro)q(cesses)h(are)f(implemen)o(tation)h (dep)q(enden)o(t.)189 2395 y Fk(A)n(dvic)n(e)e(to)h(implementors.)40 b Ft(If)14 b(the)g(\014le)h(is)g(to)e(b)q(e)i(deleted)g(and)g(is)f(op)q (ened)h(b)o(y)f(other)g(pro)q(cesses,)189 2452 y(\014le)f(data)f(ma)o (y)g(still)i(b)q(e)g(accessible)g(b)o(y)f(these)g(pro)q(cesses)g(un)o (til)g(they)g(close)h(the)e(\014le)i(or)e(un)o(til)i(they)189 2508 y(exit.)20 b(The)14 b(delete)h(op)q(eration)f(should)h(b)q(e)g (e\013ectiv)o(e)f(only)h(on)f(the)g(last)g(close)h(of)e(that)h(\014le)h (within)189 2564 y(this)e(application.)21 b Fe(Discussion:)15 b Ff(This)d(implemen)o(tors)e(note)j(seems)f(to)g(con)o(tradict)h(the)g (last)f(sen)o(tence)189 2704 y(of)h(the)h(paragraph)g(ab)q(o)o(v)o(e.) 19 b Ft(\()p Fk(End)d(of)g(advic)n(e)h(to)f(implementors.)p Ft(\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fn(10.2.)29 b(FILE)15 b(MANIPULA)l(TION)1144 b Ft(11)75 49 y Fl(10.2.3)49 b(File)17 b(Control)g(\(Indep)q (endent/Collective\))75 211 y Fe(Discussion:)166 261 y Ff(It)e(is)g(not)g(p)q(ossible)g(to)g(supp)q(ort)h(F)o(OR)m(TRAN)e (with)g(the)i(curren)o(t)g(\014le)f(con)o(trol)g(in)o(terface.)22 b(The)16 b(follo)o(wing)75 311 y(alternativ)o(es)e(ha)o(v)o(e)g(b)q (een)h(suggested:)134 398 y(1.)22 b(Mak)o(e)13 b(MPI)p 383 398 13 2 v 16 w(Fcn)o(tl)h(w)o(ork)f(with)h(an)g(opaque)f(ob)r (ject)i(\(a)f(la)f(MPI)p 1207 398 V 15 w(Hin)o(ts\),)h(and)f(pro)o (vide)h(access)i(functions)189 448 y(for)d(eac)o(h)i(elemen)o(t.)134 522 y(2.)22 b(P)o(ass)14 b(all)f(thirteen)i(items)e(as)h(parameters,)f (no)h(structure.)134 597 y(3.)22 b(Pro)o(vide)13 b(MPI)p 424 597 V 15 w(Fcn)o(tl,)g(MPI)p 640 597 V 15 w(Fcn)o(tl)p 750 597 V 15 w(2,)g(MPI)p 892 597 V 15 w(Fcn)o(tl)p 1002 597 V 15 w(3,)g(etc.,)g(where)i(MPI)p 1353 597 V 15 w(Fcn)o(tl)e(tak)o (es)h(only)e(one)i(param-)189 647 y(eter,)g(MPI)p 366 647 V 16 w(Fcn)o(tl)p 477 647 V 15 w(2)f(tak)o(es)i(2)e(parameters,)h (etc.)134 722 y(4.)22 b(Group)13 b(the)i(items)e(in)g(some)g (meaningful)e(w)o(a)o(y)m(,)i(and)g(pro)o(vide)h(a)g(routine)g(for)f (eac)o(h)i(group)e(of)h(items.)75 948 y Fp(The)23 b(last)h(alternative) e(has)i(further)e(been)i(expanded)f(on:)75 1061 y(Grouping)g(the)g (items)g(and)h(providing)e(meaningful)h(routines)g(has)g(two)h (advantages:)75 1117 y(individual)e(pieces)i(of)f(this)g(standard)g (are)h(more)f(modular)75 1174 y(\(e.g.)g(file)g(pointer)g(manipulation) g(is)g(all)h(contained)e(in)i(the)f(file)h(pointer)f(section\),)75 1230 y(and)g(I/O)h(code)f(will)h(be)f(more)g(self-documenting)75 1287 y(\(e.g.)g(a)h(re-open)f(is)g(expressed)g(as)h(MPI_Reopen\).)75 1400 y(As)g(a)f(first)g(cut,)h(I)g(propose)e(we)i(group)f(the)h (MPI_Fcntl)e(options)h(into)h(the)75 1456 y(following)f(procedures:)75 1569 y(MPI_FCNTL_COMM)75 1625 y(MPI_FCNTL_FILENAME)75 1682 y(MPI_FCNTL_AMODE)75 1738 y(MPI_FCNTL_MOFFSET)75 1795 y(MPI_FCNTL_DISP)75 1851 y(MPI_FCNTL_ETYPE)75 1908 y(MPI_FCNTL_FILETYPE)75 1964 y(MPI_FCNTL_HINTS)75 2021 y(MPI_FCNTL_HINTS_TAKEN)75 2134 y(MPI_Open\(comm,)f(filename,)h(amode,) g(disp,)g(etype,)123 2190 y(filetype,)f(moffset,)h(hints,)g(*fh\);)75 2246 y(MPI_File_info\(*comm,)e(*filename,)i(*amode,)g(*disp,)g(*etype,) 242 2303 y(*filetype,)g(*moffset,)f(*hints_taken\);)75 2359 y(MPI_Reopen\(fh,)g(disp,)h(etype,)g(filetype,)g(hints,)g (*new_fh\);)75 2472 y(MPI_FCNTL_ATOMICITY)75 2585 y(MPI_Atomicity\(fh,) f(atomicity,)g(*old_atomicity\);)75 2698 y(MPI_FCNTL_FILE_SIZE)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Ft(12)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)75 106 y Fp(MPI_Resize\(fh,)22 b(MPI_Offset)g(disp\);)75 162 y(MPI_File_size\(fh,)g(MPI_Offset)g(*size_in_bytes\);)75 275 y(MPI_FCNTL_IND_FP)75 332 y(MPI_FCNTL_ABS_OFFSET)75 444 y(MPI_Seek\(fh,)g(offset,)h(whence\);)75 501 y (MPI_Next_position\(fh,)e(*offset,)i(*byte_offset\);)75 614 y(MPI_FCNTL_SHARED_FP)75 727 y(MPI_Seek_shared\(fh,)e(offset,)i (whence\);)75 783 y(MPI_Shared_position\(fh,)e(*offset,)h (*byte_offset\);)75 896 y(Note)h(that)h(we)f(only)h(need)f(to)g(add)h (6)g(routines)e(as)i(we)g(already)75 953 y(have)f(MPI_Open,)g (MPI_Resize,)f(MPI_Seek,)h(and)g(MPI_Seek_shared.)75 1153 y Fq(MPI)p 160 1153 14 2 v 16 w(FILE)p 267 1153 V 15 w(CONTROL\(fh,)16 b(cmd,)f(fcntl\))117 1230 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\)) 117 1306 y(IN)155 b Fq(cmd)507 b Ff([SAME])14 b(Comma)o(nd)117 1381 y(INOUT)62 b Fq(fcntl)503 b Ff(Argumen)o(ts)14 b(or)g(return)h(v)n (alues)e(to)h(the)g(command)75 1505 y Fp(int)23 b(MPI)p 245 1505 15 2 v 17 w(File)p 358 1505 V 17 w(control\(MPI)p 639 1505 V 16 w(File)g(fh,)g(unsigned)g(cmd,)g(MPI)p 1275 1505 V 17 w(Fcntl)g(*fcntl\))166 1592 y Ft(\(F)o(OR)l(TRAN)16 b(binding)h(to)d(b)q(e)i(pro)o(vided)g(in)g(v)o(ersion)g(0.6)e(of)h (the)g(draft.\))166 1648 y Fq(MPI)p 251 1648 14 2 v 16 w(FILE)p 358 1648 V 15 w(CONTROL)g Ft(gets)e(or)g(sets)g(\014le)i (information)f(ab)q(out)f(the)h(\014le)h(asso)q(ciated)e(with)h(the)g (\014le)75 1705 y(handle)j Fq(fh)p Ft(.)22 b(Multiple)c(commands)d(can) h(b)q(e)g(issued)h(in)g(one)f(call)h(using)f(the)g(bit)o(wise)h(OR)f (op)q(erator.)21 b(If)75 1761 y Fg(MPI)p 152 1761 13 2 v 14 w(F)o(CNTL)p 295 1761 V 15 w(SET)e Ft(is)h(set,)f(the)g (\014elds)h(indicated)h(b)o(y)e(the)g(bits)g(set)g(in)h Fq(cmd)e Ft(will)j(b)q(e)f(used)f(to)f(up)q(date)75 1817 y(the)i(\014le)h(information.)34 b(If)20 b Fg(MPI)p 641 1817 V 14 w(F)o(CNTL)p 784 1817 V 15 w(SET)g Ft(is)g(not)g(set,)g(the)g (\014le)h(information)f(requested)g(will)h(b)q(e)75 1874 y(returned.)f Fg(MPI)p 354 1874 V 15 w(Fcntl)15 b Ft(is)h(de\014ned)g (as)f(follo)o(ws:)170 1968 y Fp(typedef)23 b(struct)g(MPI_Fcntl)g({)266 2024 y(MPI_Comm)261 b(comm;)266 2081 y(char)333 b(*filename;)266 2137 y(MPI_Mode)237 b(amode;)266 2194 y(MPI_Offset)189 b(disp;)266 2250 y(MPI_Datatype)165 b(etype;)266 2306 y(MPI_Datatype)g(filetype;)266 2363 y(MPI_Hints)190 b(*hints;)266 2419 y(MPI_Hints)g(*hints_taken;)266 2476 y(MPI_Atomicity)117 b(atomicity;)266 2532 y(MPI_Offset)189 b(ind_fp;)266 2589 y(MPI_Offset)g(shared_fp;)266 2645 y(MPI_Offset)g(file_size;)266 2702 y(MPI_Offset)g(abs_offset;)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(13)170 49 y Fp(})24 b(MPI_Fcntl;)166 143 y Ft(The)15 b(commands)g(a)o(v)m(ailable)i(are:)143 237 y Fh(\017)23 b Fg(MPI)p 266 237 13 2 v 14 w(F)o(CNTL)p 409 237 V 15 w(SET)p Ft(:)15 b(Set)g(\014le)h(information.)143 331 y Fh(\017)23 b Fg(MPI)p 266 331 V 14 w(F)o(CNTL)p 409 331 V 15 w(COMM)p Ft(:)15 b(Get)f(the)i(comm)o(unicator)e(asso)q (ciated)i(with)f(the)h(\014le.)143 425 y Fh(\017)23 b Fg(MPI)p 266 425 V 14 w(F)o(CNTL)p 409 425 V 15 w(FILENAME)p Ft(:)14 b(Get)h(the)g(\014lename.)143 518 y Fh(\017)23 b Fg(MPI)p 266 518 V 14 w(F)o(CNTL)p 409 518 V 15 w(AMODE)p Ft(:)14 b(Get)h(or)g(set)f(the)i(\014le)g(access)f(mo)q(de)h(asso)q (ciated)f(with)h(the)f(\014le.)143 612 y Fh(\017)23 b Fg(MPI)p 266 612 V 14 w(F)o(CNTL)p 409 612 V 15 w(DISP)p Ft(:)14 b(Get)h(or)f(set)h(the)h(displacemen)o(t.)143 706 y Fh(\017)23 b Fg(MPI)p 266 706 V 14 w(F)o(CNTL)p 409 706 V 15 w(ETYPE)p Ft(:)14 b(Get)h(or)f(set)h(the)g(elemen)o(tary)h (datat)o(yp)q(e.)143 800 y Fh(\017)23 b Fg(MPI)p 266 800 V 14 w(F)o(CNTL)p 409 800 V 15 w(FILETYPE)p Ft(:)14 b(Get)g(or)h(set)g(the)g(\014let)o(yp)q(e.)143 894 y Fh(\017)23 b Fg(MPI)p 266 894 V 14 w(F)o(CNTL)p 409 894 V 15 w(HINTS)p Ft(:)15 b(Get)f(or)g(set)g(the)h(desired)g(v)m(alues)h (for)e(the)g(hin)o(ts)h(asso)q(ciated)g(with)g(the)f(\014le.)143 987 y Fh(\017)23 b Fg(MPI)p 266 987 V 14 w(F)o(CNTL)p 409 987 V 15 w(HINTS)p 545 987 V 16 w(T)m(AKEN)p Ft(:)14 b(Get)h(the)g(v)m(alues)h(for)f(the)g(hin)o(ts)h(actually)g(used.)143 1081 y Fh(\017)23 b Fg(MPI)p 266 1081 V 14 w(F)o(CNTL)p 409 1081 V 15 w(A)m(TOMICITY)p Ft(:)13 b(Get)i(or)g(set)g(the)h(curren) o(t)f(read/write)h(atomic)f(seman)o(tics)g(enforced)189 1138 y(mo)q(de.)143 1231 y Fh(\017)23 b Fg(MPI)p 266 1231 V 14 w(F)o(CNTL)p 409 1231 V 15 w(IND)p 495 1231 V 15 w(FP)p Ft(:)11 b(Get)g(the)h(curren)o(t)g(o\013set)g(of)f(the)h (individual)k(\014le)d(p)q(oin)o(ter)f(asso)q(ciated)h(with)189 1288 y(the)k(\014le)g(\(n)o(um)o(b)q(er)g(of)g(elemen)o(tary)g(datat)o (yp)q(e)f(items)h(within)h(the)f(\014le)h(after)e(the)h(displacemen)o (t)189 1344 y(p)q(osition\).)143 1438 y Fh(\017)23 b Fg(MPI)p 266 1438 V 14 w(F)o(CNTL)p 409 1438 V 15 w(SHARED)p 586 1438 V 15 w(FP)p Ft(:)18 b(Get)i(the)g(curren)o(t)g(o\013set)f(of)g (the)h(shared)g(\014le)h(p)q(oin)o(ter)g(asso)q(ciated)189 1495 y(with)15 b(the)h(\014le)g(\(n)o(um)o(b)q(er)f(of)g(elemen)o(tary) h(datat)o(yp)q(e)e(items)i(within)g(the)g(\014le)g(after)f(the)g (displace-)189 1551 y(men)o(t)g(p)q(osition\).)143 1645 y Fh(\017)23 b Fg(MPI)p 266 1645 V 14 w(F)o(CNTL)p 409 1645 V 15 w(FILE)p 507 1645 V 14 w(SIZE)p Ft(:)15 b(Get)g(the)g(size)h (of)f(the)g(\014le)h(asso)q(ciated)g(with)f(fh.)143 1739 y Fh(\017)23 b Fg(MPI)p 266 1739 V 14 w(F)o(CNTL)p 409 1739 V 15 w(ABS)p 503 1739 V 14 w(OFFSET)p Ft(:)14 b(Get)h(the)g (absolute)h(b)o(yte)f(o\013set)f(of)h(the)g(individual)j(\014le)f(p)q (oin)o(ter.)75 1833 y(Not)11 b(all)i(\014elds)g(in)g Fq(fcntl)g Ft(are)e(c)o(hangeable.)20 b(When)12 b Fq(et)o(yp)q(e)p Ft(,)i Fq(\014let)o(yp)q(e)p Ft(,)f(or)f Fq(disp)h Ft(is)f(c)o(hanged,) h(all)g(\014le)g(p)q(oin)o(ters)75 1889 y(asso)q(ciated)i(with)h Fq(fh)f Ft(are)g(reset)g(to)g Fg(MPI)p 765 1889 V 14 w(OFFSET)p 934 1889 V 14 w(ZERO)p Ft(.)166 1945 y(All)d(calls)f(that)f (get)g(\014le)i(information)e(are)h(indep)q(enden)o(t)h(calls.)20 b(All)12 b(calls)f(that)f(set)g(\014le)i(information)75 2002 y(are)j(collectiv)o(e)j(calls.)k(When)15 b(setting)h Fq(et)o(yp)q(e)p Ft(,)g Fq(amo)q(de)p Ft(,)f(and)h Fq(atomicit)o(y)p Ft(,)e(it)i(is)g(erroneous)g(for)f(pro)q(cesses)75 2058 y(to)g(pass)g(in)h(di\013eren)o(t)f(v)m(alues.)75 2180 y Fl(10.2.4)49 b(File)17 b(Hints)75 2266 y Ft(Hin)o(ts)k(allo)o(w)h(a)e (user)i(to)e(pro)o(vide)i(information)f(regarding)g(\014le)h(access)f (patterns)g(and)g(\014le)h(system)75 2322 y(sp)q(eci\014cs)i(to)e (direct)h(optimization.)42 b(Pro)o(viding)23 b(hin)o(ts)g(ma)o(y)e (enable)j(an)e(implemen)o(tation)i(to)d(de-)75 2379 y(liv)o(er)e (increased)g(I/O)f(p)q(erformance)g(or)f(minimize)j(the)e(use)g(of)f (system)g(resources.)28 b(Ho)o(w)o(ev)o(er,)17 b(hin)o(ts)75 2435 y(do)22 b(not)f(c)o(hange)h(the)f(seman)o(tics)h(of)f(an)o(y)h(of) f(the)h(in)o(terfaces.)39 b(In)23 b(other)e(w)o(ords,)h(an)g(implemen)o (ta-)75 2492 y(tion)d(is)g(free)f(to)g(ignore)g(all)i(hin)o(ts.)30 b(Hin)o(ts)18 b(are)g(sp)q(eci\014ed)j(on)d(a)g(p)q(er)h(\014le)h (basis)e(at)g Fq(MPI)p 1631 2492 14 2 v 16 w(OPEN)h Ft(\(and)75 2548 y Fq(MPI)p 160 2548 V 16 w(FILE)p 267 2548 V 15 w(CONTROL)p Ft(\))d(time.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Ft(14)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)189 49 y Fk(A)n(dvic)n(e)20 b(to)h(implementors.)71 b Ft(It)21 b(ma)o(y)e(happ)q(en)j(that)e(a)g(program)f(is)i(co)q(ded)g (with)g(hin)o(ts)g(for)189 106 y(one)15 b(system,)h(and)f(later)h (executes)g(on)g(another)f(system)h(that)f(do)q(es)h(not)f(supp)q(ort)h (these)g(hin)o(ts.)189 162 y(In)k(general,)h(unsupp)q(orted)g(hin)o(ts) f(should)h(simply)g(b)q(e)g(ignored.)34 b(In)20 b(particular,)h(\014le) g(con)o(trol)189 219 y(op)q(erations,)16 b(setting)g(or)g(inquiring)i (v)m(alues)g(to)e(unsupp)q(orted)h(hin)o(ts,)g(should)g(b)q(e)g (handled)h(as)d(if)189 275 y(these)h(hin)o(ts)g(w)o(ere)g(not)g(sp)q (eci\014ed.)25 b(Needless)17 b(to)f(sa)o(y)l(,)f(no)h(hin)o(t)h(can)f (b)q(e)h(mandatory)l(.)k(Ho)o(w)o(ev)o(er,)189 332 y(for)15 b(eac)o(h)h(hin)o(t)h(used)f(b)o(y)g(a)g(sp)q(eci\014c)i(implemen)o (tation,)f(a)f(default)h(v)m(alue)g(m)o(ust)f(b)q(e)g(pro)o(vided)h(in) 189 388 y(order)e(to)g(b)q(e)h(used)g(when)g(the)f(user)h(do)q(es)g (not)f(sp)q(ecify)i(a)e(v)m(alue)i(for)e(this)g(hin)o(t.)22 b(\()p Fk(End)15 b(of)i(advic)n(e)189 444 y(to)f(implementors.)p Ft(\))166 551 y(In)g(C,)e(hin)o(t)i(v)m(alues)g(are)f(de\014ned)i(as:) 170 657 y Fp(typedef)23 b(union)h(MPI_Hint_value)d({)266 713 y(MPI_Offset)h(hint_value;)266 770 y(void)h(*hint_ptr;)170 826 y(})h(MPI_Hint_value;)75 948 y Fl(10.2.5)49 b(Hints)17 b(Creation)f(and)h(Deallo)q(cation)75 1128 y Fq(MPI)p 160 1128 14 2 v 16 w(HINTS)p 309 1128 V 16 w(CREA)l(TE\(count,)g(hint)p 720 1128 V 18 w(names,)e(hint)p 958 1128 V 17 w(values,)h(hints\))117 1206 y Ff(IN)155 b Fq(count)482 b Ff(Num)o(b)q(er)14 b(of)f(hin)o(ts)h(supplied)g(\(in)o(teger\))117 1281 y(IN)155 b Fq(hint)p 394 1281 V 18 w(names)377 b Ff(Names)13 b(of)h(hin)o(ts)f(\(arra)o(y)h(of)g(in)o(teger\))117 1356 y(IN)155 b Fq(hint)p 394 1356 V 18 w(values)382 b Ff(V)m(alues)14 b(of)f(hin)o(ts)h(\(arra)o(y)g(of)f(in)o(teger\))117 1431 y(OUT)108 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))75 1555 y Fp(int)23 b(MPI)p 245 1555 15 2 v 17 w(Hints)p 382 1555 V 17 w(create\(int)f(count,)h(int)h(*hint)p 1044 1555 V 16 w(names,)f(MPI)p 1299 1555 V 17 w(Hint)p 1412 1555 V 17 w(value)g(*hint)p 1692 1555 V 16 w(values,)393 1612 y(MPI)p 468 1612 V 17 w(Hints)g(*hints\))75 1698 y(MPI)p 150 1698 V 17 w(HINTS)p 287 1698 V 16 w(CREATE\(COUNT,)f(HINT)p 733 1698 V 17 w(NAMES,)h(HINT)p 1013 1698 V 17 w(VALUES,)g(HINTS,)g (IERROR\))170 1755 y(INTEGER)g(COUNT,)g(HINT)p 627 1755 V 17 w(NAMES\(*\))170 1811 y(INTEGER*8)g(HINT)p 508 1811 V 17 w(VALUES\(*\))170 1868 y(INTEGER)g(HINTS,)g(IERROR)166 1954 y Fq(MPI)p 251 1954 14 2 v 16 w(HINTS)p 400 1954 V 16 w(CREA)l(TE)c Ft(returns)f(an)g(opaque)g(ob)s(ject,)g(it)h(is)f (the)g(resp)q(onsibilit)o(y)j(of)c(the)h(user)h(to)75 2011 y(delete)d(the)g(ob)s(ject)e(b)o(y)h(calling)i Fq(MPI)p 714 2011 V 16 w(HINTS)p 863 2011 V 17 w(FREE)e Ft(when)h(done.)75 2162 y Fq(MPI)p 160 2162 V 16 w(HINTS)p 309 2162 V 16 w(FREE\(hints\))117 2239 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))75 2363 y Fp(int)23 b(MPI)p 245 2363 15 2 v 17 w(Hints)p 382 2363 V 17 w(free\(MPI)p 591 2363 V 16 w(Hints)g(*hints\))75 2450 y(MPI)p 150 2450 V 17 w(HINTS)p 287 2450 V 16 w(FREE\(HINTS,)g(IERROR\))170 2506 y(INTEGER)g(HINTS,)g(IERROR)166 2593 y Fq(MPI)p 251 2593 14 2 v 16 w(HINTS)p 400 2593 V 16 w(FREE)11 b Ft(marks)e(the)i(hin)o(ts)f(ob)s(ject)g(for)f(deallo)q(cation)j(and)e (sets)g Fq(hints)i Ft(to)d Fg(MPI)p 1686 2593 13 2 v 15 w(HINTS)p 1822 2593 V 15 w(NULL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(15)166 49 y(Hin)o(t)11 b(names)f(smaller)h(or)f(equal)h(to)e Fg(MPI)p 844 49 13 2 v 15 w(HINTS)p 980 49 V 15 w(RESERVED)g Ft(when)i(implemen)o(ted)h(carry)e(the)g(same)75 106 y(meaning)h(across)e(all)i(implemen)o(tations)h(and)e(will)i(b)q(e)f (de\014ned)g(in)g(future)f(sp)q(eci\014cations.)20 b Fg(MPI)p 1697 106 V 15 w(HINTS)p 1833 106 V 15 w(RESERVED)75 162 y Ft(is)c(de\014ned)g(to)f(b)q(e)h(65535.)166 219 y(Hin)o(t)f(names)h(larger)f(than)g Fg(MPI)p 721 219 V 14 w(HINTS)p 856 219 V 16 w(RESERVED)e Ft(are)i(implemen)o(tation)i (sp)q(eci\014c)g(names.)166 275 y(The)g(constan)o(t)f Fg(MPI)p 523 275 V 15 w(HINTS)p 659 275 V 15 w(NULL)i Ft(refers)f(to)f(the)i(n)o(ull)g(hin)o(t,)g(and)f(can)g(b)q(e)h(used)f (in)h Fq(MPI)p 1740 275 14 2 v 16 w(OPEN)75 332 y Ft(when)e(no)f(hin)o (ts)g(need)i(to)d(b)q(e)i(sp)q(eci\014ed.)75 453 y Fl(10.2.6)49 b(Hints)17 b(Accesso)o(r)d(F)o(unctions)75 539 y Fq(MPI)p 160 539 V 16 w(HINTS)p 309 539 V 16 w(COUNT)i Ft(returns)f(the)h(n)o (um)o(b)q(er)f(of)g(hin)o(ts)h(in)g(the)f(opaque)g(ob)s(ject)g(hin)o (ts.)75 690 y Fq(MPI)p 160 690 V 16 w(HINTS)p 309 690 V 16 w(COUNT\(hints,)i(count\))117 767 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 842 y(OUT)108 b Fq(count)482 b Ff(Num)o(b)q(er)14 b(of)f(hin)o(ts)h(\(in)o(teger\))75 967 y Fp(int)23 b(MPI)p 245 967 15 2 v 17 w(Hints)p 382 967 V 17 w(count\(MPI)p 615 967 V 16 w(Hints)g(hints,)g(int)g(*count\)) 75 1053 y(MPI)p 150 1053 V 17 w(HINTS)p 287 1053 V 16 w(COUNT\(HINTS,)f(COUNT,)i(IERROR\))170 1110 y(INTEGER)f(HINTS,)g (COUNT,)g(IERROR)166 1196 y Fq(MPI)p 251 1196 14 2 v 16 w(HINTS)p 400 1196 V 16 w(GET)p 504 1196 V 17 w(ELEMENT)15 b Ft(returns)g(the)h(n-th)f(elemen)o(t)h(in)g(the)f(opaque)h(ob)s(ject) e(hin)o(ts.)189 1302 y Fk(A)n(dvic)n(e)h(to)i(users.)43 b Ft(The)15 b(order)h(of)f(the)g(elemen)o(ts)i(in)f(hin)o(ts)g(ma)o(y)f (or)g(ma)o(y)g(not)g(b)q(e)h(the)g(same)f(as)189 1359 y(the)g(order)g(passed)g(in)h Fq(MPI)p 668 1359 V 16 w(HINTS)p 817 1359 V 17 w(CREA)l(TE)p Ft(.)g(\()p Fk(End)f(of)i(advic)n (e)f(to)g(users.)p Ft(\))75 1560 y Fq(MPI)p 160 1560 V 16 w(HINTS)p 309 1560 V 16 w(GET)p 413 1560 V 17 w(ELEMENT\(hints,)g (n,)f(hint)p 900 1560 V 18 w(name,)f(hint)p 1120 1560 V 18 w(value\))117 1637 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 1712 y(IN)155 b Fq(n)564 b Ff(Numeric)14 b(selector)h(sp)q(ecifying)f(whic)o(h)f(hin)o(t)h(\(in) o(teger\))117 1787 y(OUT)108 b Fq(hint)p 394 1787 V 18 w(name)394 b Ff(Name)13 b(of)g(hin)o(t)h(\(in)o(teger\))117 1862 y(OUT)108 b Fq(hint)p 394 1862 V 18 w(value)399 b Ff(V)m(alue)14 b(of)f(hin)o(t)g(\(in)o(teger\))75 1987 y Fp(int)23 b(MPI)p 245 1987 15 2 v 17 w(Hints)p 382 1987 V 17 w(get)p 471 1987 V 16 w(element\(MPI)p 751 1987 V 16 w(Hints)g(hints,)g(int)h(n,)f(int)h(*hint)p 1460 1987 V 16 w(name,)393 2043 y(MPI)p 468 2043 V 17 w(Hint)p 581 2043 V 17 w(value)f(*hint)p 861 2043 V 16 w(value\))75 2130 y(MPI)p 150 2130 V 17 w(HINTS)p 287 2130 V 16 w(GET)p 375 2130 V 17 w(ELEMENT\(HINTS,)f(N,)i(HINT)p 918 2130 V 16 w(NAME,)f(HINT)p 1173 2130 V 17 w(VALUE,)g(IERROR\))170 2186 y(INTEGER)g(HINTS,)g(N,)h(HINT)p 699 2186 V 17 w(NAME)170 2243 y(INTEGER*8)f(HINT)p 508 2243 V 17 w(VALUE)170 2299 y(INTEGER)g(IERROR)166 2385 y Fq(MPI)p 251 2385 14 2 v 16 w(HINTS)p 400 2385 V 16 w(GET)p 504 2385 V 17 w(V)l(ALUE)13 b Ft(returns)f(the)g Fq(hint)p 975 2385 V 18 w(value)g Ft(of)g Fq(hint)p 1224 2385 V 18 w(name)g Ft(in)h(the)f(opaque)g(ob)s (ject)g(hin)o(ts.)75 2442 y(If)j Fq(hint)p 196 2442 V 18 w(name)g Ft(is)h(not)e(de\014ned)j(in)f Fq(hints)p Ft(,)g(an)g(error)e(will)j(b)q(e)f(returned,)f(and)g Fq(hint)p 1467 2442 V 18 w(value)g Ft(is)h(unde\014ned.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Ft(16)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(HINTS)p 309 49 V 16 w(GET)p 413 49 V 17 w(V)l(ALUE\(hints,)17 b(hint)p 780 49 V 18 w(name,)d(hint)p 1000 49 V 18 w(value\))117 126 y Ff(IN)155 b Fq(hints)497 b Ff(Hin)o(ts)14 b(\(handle\))117 200 y(IN)155 b Fq(hint)p 394 200 V 18 w(name)394 b Ff(Name)13 b(of)g(hin)o(t)h(\(in)o(teger\))117 274 y(OUT)108 b Fq(hint)p 394 274 V 18 w(value)399 b Ff(V)m(alue)14 b(of)f(hin)o(t)g(\(in)o (teger\))75 398 y Fp(int)23 b(MPI)p 245 398 15 2 v 17 w(Hints)p 382 398 V 17 w(get)p 471 398 V 16 w(value\(MPI)p 703 398 V 16 w(Hints)h(hints,)f(int)g(hint)p 1221 398 V 17 w(name,)393 455 y(MPI)p 468 455 V 17 w(Hint)p 581 455 V 17 w(value)g(*hint)p 861 455 V 16 w(value\))75 541 y(MPI)p 150 541 V 17 w(HINTS)p 287 541 V 16 w(GET)p 375 541 V 17 w(VALUE\(HINTS,)f(HINT)p 798 541 V 17 w(NAME,)h(HINT)p 1054 541 V 17 w(VALUE,)g(IERROR\))170 598 y(INTEGER)g(HINTS,)g(HINT)p 627 598 V 17 w(NAME)170 654 y(INTEGER*8)g(HINT)p 508 654 V 17 w(VALUE)170 711 y(INTEGER)g(IERROR)166 797 y Ft(Some)16 b(p)q(oten)o(tially)h(useful)f(hin)o(ts)g(are)g(outlined)h (b)q(elo)o(w.)22 b(Sp)q(eci\014c)c(implemen)o(tations)f(are)e(free)h (to)75 853 y(de\014ne)h(these)g(hin)o(ts)f(as)g(w)o(ell)h(as)f (additional)h(hin)o(ts.)23 b(Ho)o(w)o(ev)o(er,)15 b(in)i(order)f(to)g (main)o(tain)g(compatibilit)o(y)75 910 y(b)q(et)o(w)o(een)g(implemen)o (tations,)g(additional)h(hin)o(ts)e(should)i(not)e(reuse)g(the)h(same)f (hin)o(t)g(names)h(for)e(other)75 966 y(purp)q(oses)i(than)f(these)g (listed)i(here.)166 1023 y(These)c(hin)o(ts)h(are)f(mainly)h(concerned) g(with)f(la)o(y)o(out)g(of)f(data)h(on)g(parallel)h(I/O)g(devices,)g (and)g(with)75 1079 y(access)j(patterns.)24 b(F)l(or)16 b(eac)o(h)h(hin)o(t)g(name)g(in)o(tro)q(duced,)h(w)o(e)f(describ)q(e)h (the)f(purp)q(ose)g(of)g(the)f(hin)o(t,)i(and)75 1136 y(the)e(t)o(yp)q(e)f(of)h(the)f(hin)o(t)h(v)m(alue.)23 b(If)15 b(sp)q(eci\014ed,)j(the)e(\\)p Fo([SAME])p Ft(")d(annotation)i (stipulates)i(that)e(the)g(hin)o(t)75 1192 y(v)m(alues)j(pro)o(vided)g (b)o(y)f(all)g(participating)h(pro)q(cesses)g(m)o(ust)e(b)q(e)i(iden)o (tical,)h(otherwise)e(the)g(program)e(is)75 1249 y(erroneous.)75 1346 y Fg(MPI)p 152 1346 13 2 v 14 w(HINTS)p 287 1346 V 16 w(STRIPING)p 489 1346 V 14 w(UNIT)i Fo(\(in)o(teger\))h([SAME]:)j Ft(This)d(hin)o(t)h(sp)q(eci\014es)h(the)e(suggested)g(striping)189 1402 y(unit)c(to)e(b)q(e)i(used)g(for)f(this)g(\014le.)21 b(The)13 b(striping)h(unit)g(is)g(the)f(amoun)o(t)g(of)g(consecutiv)o (e)h(data)e(tak)o(en)189 1459 y(from)j(one)i(I/O)g(no)q(de)g(\(resp.)24 b(storage)15 b(device\))j(b)q(efore)f(progressing)f(to)g(the)h(next)f (no)q(de)i(\(resp.)189 1515 y(storage)c(device\),)j(when)f(striping)g (across)f(a)h(n)o(um)o(b)q(er)g(of)f(no)q(des)h(\(resp.)21 b(storage)15 b(devices\);)h(it)g(is)189 1572 y(expressed)j(in)g(b)o (ytes.)30 b(This)19 b(hin)o(t)g(is)g(relev)m(an)o(t)g(only)h(when)f (the)f(\014le)i(is)f(created,)g(and)g(if)g(used,)189 1628 y(this)13 b(data)f(should)h(b)q(e)h(main)o(tained)f(b)o(y)g(the)g (\014le)g(serv)o(er\(s\))f(as)g(part)g(of)h(the)f(metadata)g(asso)q (ciated)189 1685 y(with)g(the)g(\014le.)19 b(A)12 b(go)q(o)q(d)g(size)g (for)g(a)f(striping)i(unit)f(is)g(the)g(amoun)o(t)f(of)h(consecutiv)o (e)g(data)f(accessed)189 1741 y(indep)q(enden)o(tly)18 b(b)o(y)d(a)g(pro)q(cess,)g(or)g(if)g(this)h(is)g(to)q(o)e(small,)i(a)f (m)o(ultiple)i(of)d(that)h(size.)75 1832 y Fg(MPI)p 152 1832 V 14 w(HINTS)p 287 1832 V 16 w(STRIPING)p 489 1832 V 14 w(F)m(A)o(CTOR)g Fo(\(in)o(teger\))j([SAME]:)j Ft(This)e(hin)o(t)g (sp)q(eci\014es)h(the)e(n)o(um)o(b)q(er)h(of)f(I/O)189 1889 y(no)q(des)h(\(resp.)29 b(storage)17 b(devices\))j(that)e(the)g (\014le)i(should)f(b)q(e)g(strip)q(ed)h(across,)e(and)g(is)h(relev)m (an)o(t)189 1945 y(only)c(when)h(the)f(\014le)i(is)e(created.)75 2036 y Fg(MPI)p 152 2036 V 14 w(HINTS)p 287 2036 V 16 w(NUM)p 397 2036 V 15 w(IO)p 455 2036 V 14 w(NODES)i Fo(\(in)o(teger\))h([SAME]:)j Ft(The)c(n)o(um)o(b)q(er)g(of)f(I/O)h(no) q(des)h(in)f(the)g(system.)189 2092 y(Used)h(to)f(sp)q(ecify)i(the)f (ideal)h(n)o(um)o(b)q(er)f(of)f(I/O)h(no)q(des)h(for)e(this)h (application)h(as)f(w)o(ell)g(as)g(query)189 2149 y(the)d(n)o(um)o(b)q (er)g(of)g(I/O)h(no)q(des)g(actually)g(in)g(use.)75 2240 y Fg(MPI)p 152 2240 V 14 w(HINTS)p 287 2240 V 16 w(IO)p 346 2240 V 14 w(NODE)p 475 2240 V 15 w(LIST)h Fo(\(arra)o(y)g(of)h(in)o (teger\))f([SAME]:)k Ft(In)c(addition)g(to)f(de\014ning)h(the)g(strip-) 189 2296 y(ing)e(factor,)f(one)h(ma)o(y)f(w)o(an)o(t)g(to)g(giv)o(e)h (the)g(explicit)i(list)f(of)e(I/O)h(no)q(des)h(\(resp.)j(storage)14 b(devices\))189 2353 y(that)g(should)i(b)q(e)g(used.)21 b(This)15 b(hin)o(t)h(is)g(relev)m(an)o(t)g(only)f(when)h(the)f(\014le) i(is)e(created.)75 2443 y Fg(MPI)p 152 2443 V 14 w(HINTS)p 287 2443 V 16 w(NB)p 360 2443 V 15 w(PROC)h Fo(\(in)o(teger\))i ([SAME]:)j Ft(This)e(sp)q(eci\014es)h(the)f(n)o(um)o(b)q(er)g(of)f (parallel)j(pro)q(cesses)189 2500 y(that)15 b(will)i(t)o(ypically)g(b)q (e)f(assigned)g(to)f(run)h(programs)f(that)f(access)i(this)g(\014le,)h (and)e(can)h(b)q(e)g(used)189 2556 y(at)e(\014le)i(creation.)75 2647 y Fg(MPI)p 152 2647 V 14 w(HINTS)p 287 2647 V 16 w(A)o(CCESS)p 455 2647 V 13 w(STYLE)i Fo(\(in)o(teger\):)23 b Ft(This)16 b(hin)o(t)h(sp)q(eci\014es)h(the)e(manner)g(in)g(whic)o(h) h(the)f(\014le)h(will)189 2704 y(b)q(e)f(accessed)f(un)o(til)i(the)e (\014le)h(is)g(closed)g(or)f(un)o(til)h(the)f(program)f(terminates.) 1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fn(10.2.)34 b(FILE)15 b(MANIPULA)l(TION)1139 b Ft(17)189 49 y(The)15 b(hin)o(t)h(v)m(alue)g(is)g(an)f(in)o(teger)g (equal)h(to)f(one)g(of)g(the)g(constan)o(ts:)230 159 y Fg(MPI)p 307 159 13 2 v 14 w(HINTS)p 442 159 V 16 w(READ)p 568 159 V 14 w(ONCE)230 203 y(MPI)p 307 203 V 14 w(HINTS)p 442 203 V 16 w(WRITE)p 589 203 V 15 w(ONCE)230 247 y(MPI)p 307 247 V 14 w(HINTS)p 442 247 V 16 w(READ)p 568 247 V 14 w(MOSTL)m(Y)230 291 y(MPI)p 307 291 V 14 w(HINTS)p 442 291 V 16 w(WRITE)p 589 291 V 15 w(MOSTL)m(Y)189 393 y Ft(P)o(ossibly)i(mo)q(di\014ed)i(b)o(y)e(one)g(of)g(the)g(follo)o (wing)h(constan)o(ts)e(\(via)h(the)g(bit)o(wise)h(OR)g(op)q(erator)e (in)189 449 y(C)f(or)f(simple)j(addition)f(in)g(F)o(OR)l(TRAN\):)230 559 y Fg(MPI)p 307 559 V 14 w(HINTS)p 442 559 V 16 w(SEQUENTIAL)230 603 y(MPI)p 307 603 V 14 w(HINTS)p 442 603 V 16 w(REVERSE)p 638 603 V 14 w(SEQUENTIAL)230 647 y(MPI)p 307 647 V 14 w(HINTS)p 442 647 V 16 w(RANDOM)75 767 y(MPI)p 152 767 V 14 w(HINTS)p 287 767 V 16 w(COLLECTIVE)p 550 767 V 13 w(BUFFERING)g Fo(\(logical\))k([SAME]:)h Ft(This)11 b(hin)o(t)f(sp)q(eci\014es)i(whether)f(the)f(ap-)189 824 y(plication)i(will)h(b)q(ene\014t)g(from)d(collectiv)o(e)j (bu\013ering)f(\(an)f(optimization)h(p)q(erformed)f(on)g(collectiv)o(e) 189 880 y(accesses)16 b(whic)o(h)g(coalesces)h(small)f(requests)g(in)o (to)g(large)g(disk)g(accesses\).)22 b(Collectiv)o(e)17 b(bu\013ering)189 937 y(parameters)d(are)h(further)g(directed)h(via)g Fg(MPI)p 979 937 V 14 w(HINTS)p 1114 937 V 16 w(CB)p 1185 937 V 14 w(XXX)e Ft(hin)o(ts.)75 1031 y Fg(MPI)p 152 1031 V 14 w(HINTS)p 287 1031 V 16 w(CB)p 358 1031 V 14 w(NODES)j Fo(\(in)o(teger\))h([SAME]:)j Ft(The)14 b(n)o(um)o(b)q(er)f(of)g(target)g(no)q(des)g(used)h(for)f(collectiv)o (e)189 1087 y(bu\013ering.)20 b(All)d(\014le)f(accesses)f(are)g(p)q (erformed)h(b)o(y)f(the)g(target)f(no)q(des.)75 1181 y Fg(MPI)p 152 1181 V 14 w(HINTS)p 287 1181 V 16 w(CB)p 358 1181 V 14 w(BLOCK)p 509 1181 V 14 w(SIZE)j Fo(\(o\013set\))h ([SAME]:)j Ft(The)d(blo)q(c)o(k)h(size)g(used)f(for)f(\014le)i(access.) 29 b(T)l(arget)189 1237 y(no)q(des)17 b(access)f(data)g(in)h(c)o(h)o (unks)g(of)e(this)i(size.)24 b(The)17 b(c)o(h)o(unks)g(are)f (distributed)i(to)d(target)g(no)q(des)189 1294 y(in)h(a)f(round-robin)h (\(CYCLIC\))f(pattern.)75 1388 y Fg(MPI)p 152 1388 V 14 w(HINTS)p 287 1388 V 16 w(CB)p 358 1388 V 14 w(BUF)p 453 1388 V 14 w(SIZE)i Fo(\(o\013set\))h([SAME]:)i Ft(The)11 b(total)e(bu\013er)h(space)h(used)g(for)e(collectiv)o(e)j(bu\013er-)189 1444 y(ing)j(on)h(eac)o(h)f(target)f(no)q(de;)h(usually)i(a)e(m)o (ultiple)i(of)d Fg(MPI)p 1186 1444 V 15 w(HINTS)p 1322 1444 V 15 w(CB)p 1392 1444 V 15 w(BLOCK)p 1544 1444 V 14 w(SIZE)p Ft(.)166 1550 y(Sp)q(eci\014c)19 b(implemen)o(tations)e (are)f(free)h(to)f(in)o(terpret)g(the)h(hin)o(ts)g(in)g(sligh)o(tly)g (di\013eren)o(t)g(w)o(a)o(ys.)23 b(F)l(or)75 1607 y(example,)17 b(the)f(follo)o(wing)h(table)f(outlines)i(p)q(ossible)g(in)o (terpretations)e(for)f(an)h(implemen)o(tation)i(based)75 1663 y(on)d(the)g(IBM)h(AIX)g(P)o(arallel)g(I/O)f(File)i(System)e([1)o (,)g(2]:)p 483 1718 985 2 v 482 1774 2 57 v 507 1757 a Fk(hint)i(name)p 1116 1774 V 433 w(interpr)n(etation)p 1467 1774 V 483 1776 985 2 v 482 1832 2 57 v 507 1815 a Fg(MPI)p 584 1815 13 2 v 15 w(HINTS)p 720 1815 V 15 w(STRIPING)p 921 1815 V 14 w(UNIT)p 1116 1832 2 57 v 111 w Ft(BSU)e(size)p 1467 1832 V 482 1889 V 507 1872 a Fg(MPI)p 584 1872 13 2 v 15 w(HINTS)p 720 1872 V 15 w(STRIPING)p 921 1872 V 14 w(F)m(A)o(CTOR)p 1116 1889 2 57 v 48 w Ft(n)o(um)o(b)q(er)g(of)g(cells)p 1467 1889 V 482 1945 V 507 1928 a Fg(MPI)p 584 1928 13 2 v 15 w(HINTS)p 720 1928 V 15 w(IO)p 778 1928 V 14 w(NODE)p 907 1928 V 15 w(LIST)p 1116 1945 2 57 v 137 w Ft(base)g(no)q(de)p 1467 1945 V 483 1947 985 2 v 75 2052 a Fl(10.2.7)49 b(Deleting)17 b(a)g(\014le)f(\(Indep)q(endent\))75 2232 y Fq(MPI)p 160 2232 14 2 v 16 w(DELETE\(\014lename\))117 2310 y Ff(IN)155 b Fq(\014lename)430 b Ff(Name)13 b(of)g(the)i(\014le)f(to)f (b)q(e)i(deleted)g(\(string\))75 2434 y Fp(int)47 b(MPI)p 269 2434 15 2 v 17 w(Delete\(char)22 b(*filename\))75 2521 y(MPI)p 150 2521 V 17 w(DELETE\(FILENAME,)f(IERROR\))170 2577 y(CHARACTER)i(FILENAME\(*\))170 2633 y(INTEGER)g(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Ft(18)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(DELETE)15 b Ft(deletes)h(a)e(\014le.)21 b(If)16 b(the)f(\014le)h(exists)f(it)h (is)f(remo)o(v)o(ed.)20 b(If)15 b(there)g(are)g(other)g(pro)q(cesses)75 106 y(curren)o(tly)k(accessing)h(the)f(\014le,)i(the)e(status)f(of)h (the)g(\014le)h(and)f(the)g(b)q(eha)o(vior)g(of)g(future)g(accesses)g (are)75 162 y(implemen)o(tation)d(dep)q(enden)o(t.)21 b(If)14 b(the)h(\014le)g(do)q(es)g(not)f(exist,)g Fq(MPI)p 1216 162 V 16 w(DELETE)g Ft(returns)g(a)g(w)o(arning)h(error)75 219 y(co)q(de.)189 325 y Fk(A)n(dvic)n(e)i(to)i(implementors.)54 b Ft(If)17 b(the)h(\014le)h(to)d(b)q(e)j(deleted)g(is)f(op)q(ened)g(b)o (y)g(other)f(pro)q(cesses,)h(\014le)189 381 y(data)e(ma)o(y)h(still)i (b)q(e)f(accessible)h(b)o(y)e(these)h(pro)q(cesses)g(un)o(til)g(they)g (close)g(the)f(\014le)i(or)d(un)o(til)j(they)189 438 y(exit.)h(\()p Fk(End)c(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))75 560 y Fl(10.2.8)49 b(Resizing)18 b(a)f(\014le)f (\(Collective\))75 740 y Fq(MPI)p 160 740 V 16 w(RESIZE\(MPI)p 425 740 V 16 w(File)f(fh,)g(MPI)p 671 740 V 15 w(O\013set)i(size\))117 817 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i (handle)g(\(handle\))117 892 y(IN)155 b Fq(size)519 b Ff([SAME])18 b(Size)h(whic)o(h)f(the)h(\014le)f(is)g(to)g(b)q(e)h (truncated)h(at)e(or)905 949 y(expanded)d(to)f(\(nonnegativ)o(e)f (o\013set\))75 1073 y Fp(int)23 b(MPI)p 245 1073 15 2 v 17 w(Resize\(MPI)p 502 1073 V 16 w(File)g(fh,)h(MPI)p 805 1073 V 17 w(Offset)f(size\))75 1160 y(MPI)p 150 1160 V 17 w(RESIZE\(FH,)f(SIZE,)h(IERROR\))170 1216 y(INTEGER)g(FH,)h (IERROR)170 1272 y(INTEGER*8)f(SIZE)166 1359 y Fq(MPI)p 251 1359 14 2 v 16 w(RESIZE)18 b Ft(resizes)h(the)f(\014le)g(asso)q (ciated)g(with)h(the)f(\014le)h(handle)g Fq(fh)p Ft(.)28 b(If)18 b Fq(size)g Ft(is)g(smaller)h(than)75 1415 y(the)14 b(curren)o(t)g(\014le)h(size,)g(the)f(\014le)h(is)f(truncated)g(at)f (the)h(p)q(osition)h(de\014ned)h(b)o(y)e Fq(size)g Ft(\(from)f(the)h(b) q(eginning)75 1472 y(of)f(the)g(\014le)h(and)f(measured)g(in)h(b)o (ytes\).)19 b(File)14 b(blo)q(c)o(ks)f(lo)q(cated)h(b)q(ey)o(ond)g (that)e(p)q(osition)i(are)f(deallo)q(cated.)75 1528 y(If)19 b Fq(size)h Ft(is)f(larger)g(than)g(the)g(curren)o(t)g(\014le)h(size,)h (additional)f(\014le)g(blo)q(c)o(ks)g(are)f(allo)q(cated)h(and)f(the)g (\014le)75 1585 y(size)14 b(b)q(ecomes)h Fq(size)p Ft(.)k(All)c(pro)q (cesses)f(in)g(the)g(comm)o(unicator)f(group)g(m)o(ust)g(call)h Fq(MPI)p 1534 1585 V 16 w(RESIZE)g Ft(with)g(the)75 1641 y(same)h Fq(size)p Ft(.)166 1780 y Fe(Discussion:)j Ff(There)e(are)g(t) o(w)o(o)e(issues)j(asso)q(ciated)f(with)e(\014le)h(blo)q(c)o(k)g (preallo)q(cation:)20 b(it)15 b(ma)o(y)e(b)q(e)j(slo)o(w)e(on)75 1837 y(systems)j(without)g(direct)g(\014lesystem)g(supp)q(ort,)h(and)f (w)o(e)g(m)o(ust)f(b)q(e)i(careful)f(to)f(de\014ne)i(what)f(happ)q(ens) h(if)e(not)75 1893 y(enough)e(space)h(is)f(a)o(v)n(ailable.)75 2098 y Fl(10.2.9)49 b(File)17 b(Sync)f(\(Collective\))75 2278 y Fq(MPI)p 160 2278 V 16 w(FILE)p 267 2278 V 15 w(SYNC\(fh\))117 2355 y Ff(IN)155 b Fq(fh)550 b Ff([SAME]V)m(alid)12 b(\014le)i(handle)g(\(handle\))75 2480 y Fp(int)23 b(MPI)p 245 2480 15 2 v 17 w(File)p 358 2480 V 17 w(sync\(MPI)p 567 2480 V 16 w(File)g(fh\))75 2566 y(MPI)p 150 2566 V 17 w(FILE)p 263 2566 V 16 w(SYNC\(FH,)g(IERROR\))170 2622 y(INTEGER)g(FH,)h(IERROR)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fn(10.3.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (EXPLICIT)h(OFFSETS)697 b Ft(19)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(FILE)p 358 49 V 15 w(SYNC)19 b Ft(causes)e(the)h(con)o (ten)o(ts)f(of)g(the)h(\014le)h(referenced)f(b)o(y)g Fq(fh)g Ft(to)e(b)q(e)j(\015ushed)f(to)f(p)q(er-)75 106 y(manen)o(t)g(storage.)26 b(All)18 b(pro)q(cesses)g(in)g(the)g(comm)o (unicator)f(group)g(asso)q(ciated)h(with)g(the)f(\014le)i(handle)75 162 y Fq(fh)c Ft(m)o(ust)e(call)j Fq(MPI)p 406 162 V 16 w(FILE)p 513 162 V 15 w(SYNC)p Ft(.)f(The)g Fq(MPI)p 847 162 V 16 w(FILE)p 954 162 V 15 w(SYNC)g Ft(call)h(returns)e Fk(after)h Ft(all)g(data)f(written,)g(since)75 219 y(the)h(\014le)i(w)o (as)d(op)q(ened,)i(has)f(b)q(een)h(\015ushed)g(to)f(p)q(ermanen)o(t)g (storage.)189 325 y Fk(A)n(dvic)n(e)j(to)h(users.)58 b Fq(MPI)p 653 325 V 15 w(FILE)p 759 325 V 16 w(SYNC)19 b Ft(guaran)o(tees)e(that)h(all)h Fk(c)n(omplete)n(d)f Ft(I/O)g(requests)g(ha)o(v)o(e)189 381 y(b)q(een)e(\015ushed)h(to)e(p)q (ermanen)o(t)g(storage.)20 b(P)o(ending)c(non)o(blo)q(c)o(king)h(I/O)f (requests)f(that)g(ha)o(v)o(e)g(not)189 438 y(completed)h(are)f(not)g (guaran)o(teed)f(to)h(b)q(e)h(\015ushed.)21 b(\()p Fk(End)15 b(of)i(advic)n(e)f(to)g(users.)p Ft(\))166 627 y Fe(Discussion:)d Ff(Do)q(es)d(this)g(imply)e(w)o(eak)h(consistency)i(b)q(et)o(w)o(een)h (applications)c(at)i Fg(MPI)p 1521 627 13 2 v 14 w(FILE)p 1618 627 V 15 w(SYNC)f Ff(b)q(ound-)75 683 y(aries?)18 b(Should)c(it)f(b)q(e)i(required?)75 909 y Fm(10.3)60 b(Data)19 b(Access)h(With)g(Explicit)e(O\013sets)75 1012 y Fl(10.3.1)49 b(Blo)q(cking)18 b(Read)75 1193 y Fq(MPI)p 160 1193 14 2 v 16 w(READ\(fh,)d(o\013set,)g(bu\013,)h(buft)o(yp)q(e,)g (bufcount,)h(status\))117 1270 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1345 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(nonnegativ)o(e)g (o\013set\))117 1420 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 1495 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat) o(yp)q(e)f(\(handle\))117 1570 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(in)o (teger\))117 1645 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1770 y Fp(int)23 b(MPI)p 245 1770 15 2 v 17 w(Read\(MPI)p 454 1770 V 16 w(File)h(fh,)f(MPI)p 757 1770 V 17 w(Offset)g(offset,)g(void)g(*buff,)393 1826 y(MPI)p 468 1826 V 17 w(Datatype)g(buftype,)f(int)i(bufcount,)f (MPI)p 1321 1826 V 16 w(Status)g(*status\))75 1913 y(MPI)p 150 1913 V 17 w(READ\(FH,)g(OFFSET,)f(BUFF,)i(BUFTYPE,)e(BUFCOUNT,)h (STATUS,)g(IERROR\))170 1969 y()g(BUFF\(*\))170 2026 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2026 V 16 w(STATUS)p 1313 2026 V 16 w(SIZE\),)h(IERROR)170 2082 y(INTEGER*8)g(OFFSET)166 2169 y Fq(MPI)p 251 2169 14 2 v 16 w(READ)c Ft(attempts)f(to)h(read)g(from)f(the)h(\014le)i (asso)q(ciated)e(with)g Fq(fh)h Ft(\(at)e(the)h Fq(o\013set)g Ft(p)q(osition\))75 2225 y(a)d(total)g(n)o(um)o(b)q(er)i(of)e Fq(bufcount)j Ft(data)d(items)h(ha)o(ving)g Fq(buft)o(yp)q(e)h Ft(datat)o(yp)q(e)e(in)o(to)h(the)g(user's)f(bu\013er)h Fq(bu\013)p Ft(.)75 2281 y(The)f(data)f(is)i(tak)o(en)e(out)h(of)f (those)h(parts)f(of)h(the)g(\014le)g(sp)q(eci\014ed)i(b)o(y)e(\014let)o (yp)q(e.)23 b Fq(MPI)p 1533 2281 V 16 w(READ)17 b Ft(stores)e(the)75 2338 y(n)o(um)o(b)q(er)h(of)e Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f (actually)g(read)f(in)h Fq(status)p Ft(.)75 2489 y Fq(MPI)p 160 2489 V 16 w(READ)p 295 2489 V 17 w(ALL\(fh,)e(o\013set,)h(bu\013,)g (buft)o(yp)q(e,)i(bufcount,)g(status\))75 2593 y Fp(int)23 b(MPI)p 245 2593 15 2 v 17 w(Read)p 358 2593 V 17 w(all\(MPI)p 543 2593 V 16 w(File)g(fh,)h(MPI)p 846 2593 V 17 w(Offset)f(offset,)g (void)g(*buff,)393 2649 y(MPI)p 468 2649 V 17 w(Datatype)g(buftype,)f (int)i(bufcount,)f(MPI)p 1321 2649 V 16 w(Status)g(*status\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Ft(20)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fp(MPI)p 150 49 15 2 v 17 w(READ)p 263 49 V 16 w(ALL\(FH,)23 b(OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (STATUS,)g(IERROR\))170 106 y()g(BUFF\(*\))170 162 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 162 V 16 w(STATUS)p 1313 162 V 16 w(SIZE\),)h(IERROR)170 219 y(INTEGER*8)g(OFFSET)166 305 y Fq(MPI)p 251 305 14 2 v 16 w(READ)p 386 305 V 17 w(ALL)10 b Ft(is)h(a)g(collectiv)o(e)i(v)o (ersion)e(of)f(the)h(blo)q(c)o(king)h Fq(MPI)p 1290 305 V 16 w(READ)f Ft(in)o(terface.)19 b(This)11 b(routine)75 362 y(is)i(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 694 362 V 16 w(READ)p Ft(,)g(except)h(that)f(all)h(pro)q(cesses)g(in)h (the)e(comm)o(unicator)g(group)75 418 y(asso)q(ciated)j(with)h(the)f (\014le)h(handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1026 418 V 16 w(READ)p 1161 418 V 17 w(ALL)p Ft(.)75 540 y Fl(10.3.2)49 b(Blo)q(cking)18 b(W)o(rite)75 720 y Fq(MPI)p 160 720 V 16 w(WRITE\(fh,)d(o\013set,)g(bu\013,)g(buft)o(yp) q(e,)i(bufcount,)g(status\))117 797 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 872 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(nonnegativ)o(e)g (o\013set\))117 947 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 1023 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat) o(yp)q(e)f(\(handle\))117 1098 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(in)o (teger\))117 1173 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1297 y Fp(int)47 b(MPI)p 269 1297 15 2 v 17 w(Write\(MPI)p 502 1297 V 16 w(File)23 b(fh,)h(MPI)p 805 1297 V 17 w(Offset)f(offset,)g(void)g(*buff,)393 1354 y(MPI)p 468 1354 V 17 w(Datatype)g(buftype,)f(int)i(bufcount,)f (MPI)p 1321 1354 V 16 w(Status)g(*status\))75 1440 y(MPI)p 150 1440 V 17 w(WRITE\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (STATUS,)g(IERROR\))170 1497 y()g(BUFF\(*\))170 1553 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1553 V 16 w(STATUS)p 1313 1553 V 16 w(SIZE\),)h(IERROR)170 1610 y(INTEGER*8)g(OFFSET)166 1696 y Fq(MPI)p 251 1696 14 2 v 16 w(WRITE)17 b Ft(attempts)f(to)h(write)g(in)o(to)h(the)f (\014le)h(asso)q(ciated)g(with)f Fq(fh)h Ft(\(at)e(the)i Fq(o\013set)f Ft(p)q(osition\))75 1753 y(a)e(total)g(n)o(um)o(b)q(er)h (of)f Fq(bufcount)j Ft(data)d(items)h(ha)o(ving)g Fq(buft)o(yp)q(e)h Ft(datat)o(yp)q(e)e(from)g(the)h(user's)f(bu\013er)h Fq(bu\013)p Ft(.)75 1809 y(The)f(data)f(is)h(written)g(in)o(to)g(those) g(parts)f(of)g(the)h(\014le)h(sp)q(eci\014ed)h(b)o(y)d(\014let)o(yp)q (e.)21 b Fq(MPI)p 1512 1809 V 16 w(WRITE)15 b Ft(stores)f(the)75 1865 y(n)o(um)o(b)q(er)i(of)e Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f (actually)g(written)f(in)h Fq(status)p Ft(.)75 2016 y Fq(MPI)p 160 2016 V 16 w(WRITE)p 319 2016 V 16 w(ALL\(fh,)f(o\013set,)g (bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g(status\))75 2120 y Fp(int)47 b(MPI)p 269 2120 15 2 v 17 w(Write)p 406 2120 V 17 w(all\(MPI)p 591 2120 V 16 w(File)23 b(fh,)h(MPI)p 894 2120 V 16 w(Offset)f(offset,)g(void)h(*buff,)393 2177 y(MPI)p 468 2177 V 17 w(Datatype)f(buftype,)f(int)i(bufcount,)f (MPI)p 1321 2177 V 16 w(Status)g(*status\))75 2263 y(MPI)p 150 2263 V 17 w(WRITE)p 287 2263 V 16 w(ALL\(FH,)g(OFFSET,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)g(STATUS,)g(IERROR\))170 2319 y()g (BUFF\(*\))170 2376 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (STATUS\(MPI)p 1153 2376 V 16 w(STATUS)p 1313 2376 V 16 w(SIZE\),)h(IERROR)170 2432 y(INTEGER*8)g(OFFSET)166 2519 y Fq(MPI)p 251 2519 14 2 v 16 w(WRITE)p 410 2519 V 16 w(ALL)12 b Ft(is)g(a)g(collectiv)o(e)i(v)o(ersion)f(of)e(the)i (blo)q(c)o(king)g Fq(MPI)p 1324 2519 V 16 w(WRITE)f Ft(in)o(terface.)19 b(This)13 b(rou-)75 2575 y(tine)h(is)h(seman)o(tically)f(iden)o(tical)i (to)d Fq(MPI)p 788 2575 V 16 w(WRITE)p Ft(,)g(except)h(that)f(all)i (pro)q(cesses)f(in)h(the)e(comm)o(unicator)75 2632 y(group)i(asso)q (ciated)g(with)h(the)f(\014le)h(handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1155 2632 V 16 w(WRITE)p 1314 2632 V 16 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fn(10.3.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (EXPLICIT)h(OFFSETS)697 b Ft(21)75 49 y Fl(10.3.3)49 b(Nonblo)q(cking)19 b(Read)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IREAD\(fh,)c(o\013set,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(O\013set)h(\(nonnegativ)o(e)f (o\013set\))117 457 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 532 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 607 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 682 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f(\(handle\))75 807 y Fp(int)23 b(MPI)p 245 807 15 2 v 17 w(Iread\(MPI)p 478 807 V 16 w(File)g(fh,)h(MPI)p 781 807 V 17 w(Offset)f(offset,)g(void)g (*buff,)393 863 y(MPI)p 468 863 V 17 w(Datatype)g(buftype,)f(int)i (bufcount,)f(MPI)p 1321 863 V 16 w(Request)g(*request\))75 950 y(MPI)p 150 950 V 17 w(IREAD\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)g(REQUEST,)g(IERROR\))170 1006 y()g(BUFF\(*\))170 1063 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)170 1119 y(INTEGER*8)g(OFFSET)166 1206 y Fq(MPI)p 251 1206 14 2 v 16 w(IREAD)12 b Ft(is)g(a)g(non)o(blo)q(c)o(king)h(v)o(ersion)f (of)g(the)g Fq(MPI)p 1094 1206 V 16 w(READ)g Ft(in)o(terface.)19 b Fq(MPI)p 1524 1206 V 16 w(IREAD)12 b Ft(asso)q(ciates)75 1262 y(a)f(request)g(handle)h Fq(request)h Ft(with)f(the)f(I/O)h (request.)18 b(The)12 b(request)f(handle)h(can)g(b)q(e)f(used)h(later)f (to)g(query)75 1319 y(the)k(status)f(of)g(the)h(read)g(request)g(\(via) f Fq(MPI)p 851 1319 V 16 w(TEST)p Ft(\),)g(or)h(w)o(ait)f(for)g(its)h (completion)h(\(via)f Fq(MPI)p 1719 1319 V 16 w(W)l(AIT)p Ft(\).)166 1375 y(The)k(non)o(blo)q(c)o(king)i(read)e(call)h(indicates) g(that)e(the)i(system)e(can)h(start)f(to)h(read)f(data)h(in)o(to)g(the) 75 1431 y(supplied)e(bu\013er.)j(The)15 b(user)g(should)h(not)e(access) h(an)o(y)g(part)f(of)g(the)h(bu\013er)g(after)f(a)h(non)o(blo)q(c)o (king)h(read)75 1488 y(is)h(called,)h(un)o(til)f(the)f(read)h (completes)g(\(as)e(indicated)j(b)o(y)e Fq(MPI)p 1187 1488 V 16 w(TEST)h Ft(or)f Fq(MPI)p 1473 1488 V 15 w(W)l(AIT)p Ft(\).)g Fq(MPI)p 1730 1488 V 16 w(IREAD)75 1544 y Ft(attempts)e(to)g (read)h(from)g(the)g(\014le)h(asso)q(ciated)f(with)h Fq(fh)f Ft(\(at)f(the)h Fq(o\013set)g Ft(p)q(osition\),)h(a)f(total)f (n)o(um)o(b)q(er)h(of)75 1601 y Fq(bufcount)i Ft(data)e(items)g(ha)o (ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)h(user's)f (bu\013er)g Fq(bu\013)p Ft(.)20 b(The)c(n)o(um)o(b)q(er)f(of)g Fq(buft)o(yp)q(e)75 1657 y Ft(elemen)o(ts)j(actually)h(read)e(can)h(b)q (e)g(extracted)f(from)g(the)h Fq(MPI)p 1171 1657 V 16 w(TEST)f Ft(or)g Fq(MPI)p 1458 1657 V 16 w(W)l(AIT)h Ft(return)f(status,)75 1714 y(using)f(the)f(routines)h Fq(MPI)p 532 1714 V 16 w(GET)p 636 1714 V 16 w(COUNT)g Ft(or)f Fq(MPI)p 962 1714 V 16 w(GET)p 1066 1714 V 17 w(ELEMENTS)p Ft(.)75 1865 y Fq(MPI)p 160 1865 V 16 w(IREAD)p 308 1865 V 16 w(ALL\(fh,)g(o\013set,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 1968 y Fp(int)23 b(MPI)p 245 1968 15 2 v 17 w(Iread)p 382 1968 V 17 w(all\(MPI)p 567 1968 V 16 w(File)g(fh,)h(MPI)p 870 1968 V 16 w(Offset)g(offset,)e(void) i(*buff,)393 2025 y(MPI)p 468 2025 V 17 w(Datatype)f(buftype,)f(int)i (bufcount,)f(MPI)p 1321 2025 V 16 w(Request)g(*request\))75 2111 y(MPI)p 150 2111 V 17 w(IREAD)p 287 2111 V 16 w(ALL\(FH,)g (OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(REQUEST,)f(IERROR\))170 2168 y()h(BUFF\(*\))170 2224 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)170 2281 y(INTEGER*8)g(OFFSET)166 2367 y Fq(MPI)p 251 2367 14 2 v 16 w(IREAD)p 399 2367 V 16 w(ALL)16 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)f(of)f(the)g (non)o(blo)q(c)o(king)i Fq(MPI)p 1419 2367 V 16 w(IREAD)f Ft(in)o(terface.)23 b(This)75 2424 y(routine)17 b(is)g(seman)o(tically) g(iden)o(tical)h(to)e Fq(MPI)p 867 2424 V 16 w(IREAD)p Ft(,)g(except)g(that)g(all)h(pro)q(cesses)g(in)g(the)f(comm)o(uni-)75 2480 y(cator)e(group)h(asso)q(ciated)h(with)f(the)g(\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)j Fq(MPI)p 1272 2480 V 15 w(IREAD)p 1419 2480 V 17 w(ALL)p Ft(.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 -100 a Ft(22)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.3.4)49 b(Nonblo)q(cking)19 b(W)o(rite)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IWRITE\(fh,)14 b(o\013set,)h (bu\013,)h(buft)o(yp)q(e,)g(bufcount,)h(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 382 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(O\013set)h(\(nonnegativ)o(e)f(o\013set\))117 457 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 532 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 607 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q (e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 682 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f (\(handle\))75 807 y Fp(int)47 b(MPI)p 269 807 15 2 v 17 w(Iwrite\(MPI)p 526 807 V 16 w(File)23 b(fh,)h(MPI)p 829 807 V 16 w(Offset)f(offset,)g(void)h(*buff,)393 863 y(MPI)p 468 863 V 17 w(Datatype)f(buftype,)f(int)i(bufcount,)f(MPI)p 1321 863 V 16 w(Request)g(*request\))75 950 y(MPI)p 150 950 V 17 w(IWRITE\(FH,)f(OFFSET,)h(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (REQUEST,)g(IERROR\))170 1006 y()g(BUFF\(*\))170 1063 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)170 1119 y(INTEGER*8)g(OFFSET)166 1206 y Fq(MPI)p 251 1206 14 2 v 16 w(IWRITE)14 b Ft(is)i(a)e(non)o(blo)q(c)o(king)i(v)o(ersion)f (of)g(the)g Fq(MPI)p 1138 1206 V 15 w(WRITE)g Ft(in)o(terface.)20 b Fq(MPI)p 1595 1206 V 16 w(IWRITE)15 b Ft(asso-)75 1262 y(ciates)g(a)f(request)h(handle)h Fq(request)h Ft(with)e(the)g(I/O)g (request.)20 b(The)15 b(request)f(handle)i(can)f(b)q(e)h(used)f(later) 75 1319 y(to)j(query)h(the)g(status)f(of)g(the)h(write)g(request)g (\(via)f Fq(MPI)p 1083 1319 V 16 w(TEST)p Ft(\),)g(or)h(w)o(ait)f(for)g (its)h(completion)h(\(via)75 1375 y Fq(MPI)p 160 1375 V 16 w(W)l(AIT)p Ft(\).)166 1431 y(The)c(non)o(blo)q(c)o(king)h(write)f (call)h(indicates)g(that)e(the)h(system)f(can)h(start)f(to)g(write)h (data)f(from)g(the)75 1488 y(supplied)e(bu\013er.)19 b(The)11 b(user)g(should)h(not)f(access)g(an)o(y)f(part)h(of)f(the)h (bu\013er)g(after)g(the)g(non)o(blo)q(c)o(king)h(write)75 1544 y(is)i(called,)g(un)o(til)h(the)e(write)g(completes)h(\(as)e (indicated)j(b)o(y)e Fq(MPI)p 1174 1544 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1452 1544 V 16 w(W)l(AIT)p Ft(\).)h Fq(MPI)p 1707 1544 V 15 w(IWRITE)75 1601 y Ft(attempts)h(to)g(write)i (in)o(to)f(the)g(\014le)h(asso)q(ciated)f(with)h Fq(fh)f Ft(\(at)f(the)h Fq(o\013set)g Ft(p)q(osition\),)h(a)f(total)f(n)o(um)o (b)q(er)h(of)75 1657 y Fq(bufcount)h Ft(data)e(items)g(ha)o(ving)g Fq(buft)o(yp)q(e)i Ft(t)o(yp)q(e)e(from)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)20 b(The)14 b(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)75 1714 y Ft(elemen)o(ts)f(actually)h(written)f(can)f (b)q(e)i(extracted)e(from)g(the)h Fq(MPI)p 1190 1714 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1468 1714 V 16 w(W)l(AIT)h Ft(return)f(status,)75 1770 y(using)k(the)f(routines)h Fq(MPI)p 532 1770 V 16 w(GET)p 636 1770 V 16 w(COUNT)g Ft(or)f Fq(MPI)p 962 1770 V 16 w(GET)p 1066 1770 V 17 w(ELEMENTS)p Ft(.)75 1921 y Fq(MPI)p 160 1921 V 16 w(IWRITE)p 332 1921 V 16 w(ALL\(fh,)f(o\013set,)h(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 2025 y Fp(int)23 b(MPI)p 245 2025 15 2 v 17 w(Iwrite)p 406 2025 V 17 w(all\(MPI)p 591 2025 V 16 w(File)g(fh,)h(MPI)p 894 2025 V 16 w(Offset)f(offset,)g (void)h(*buff,)393 2081 y(MPI)p 468 2081 V 17 w(Datatype)f(buftype,)f (int)i(bufcount,)f(MPI)p 1321 2081 V 16 w(Request)g(*request\))75 2168 y(MPI)p 150 2168 V 17 w(IWRITE)p 311 2168 V 16 w(ALL\(FH,)g (OFFSET,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(REQUEST,)f(IERROR\))170 2224 y()h(BUFF\(*\))170 2281 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)170 2337 y(INTEGER*8)g(OFFSET)166 2424 y Fq(MPI)p 251 2424 14 2 v 16 w(IWRITE)p 423 2424 V 16 w(ALL)11 b Ft(is)h(a)g(collectiv)o(e)h(v)o(ersion)f(of)f(the)h (non)o(blo)q(c)o(king)h Fq(MPI)p 1405 2424 V 16 w(IWRITE)e Ft(in)o(terface.)19 b(This)75 2480 y(routine)c(is)g(seman)o(tically)g (iden)o(tical)h(to)e Fq(MPI)p 857 2480 V 16 w(IWRITE)p Ft(,)f(except)i(that)f(all)h(pro)q(cesses)g(in)g(the)f(comm)o(uni-)75 2536 y(cator)g(group)h(asso)q(ciated)h(with)f(the)g(\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)j Fq(MPI)p 1272 2536 V 15 w(IWRITE)p 1443 2536 V 16 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 23 24 bop 75 -100 a Fn(10.4.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (INDIVIDUAL)i(FILE)e(POINTERS)487 b Ft(23)75 49 y Fm(10.4)60 b(Data)19 b(Access)h(With)g(Individual)e(File)i(P)n(ointers)75 151 y Ft(The)14 b(system)f(main)o(tains)h(one)f(individual)k(\014le)e (p)q(oin)o(ter)f(p)q(er)g(pro)q(cess)g(p)q(er)g(collectiv)o(e)h Fq(MPI)p 1636 151 14 2 v 16 w(OPEN)p Ft(.)f(The)75 207 y(curren)o(t)k(v)m(alue)i(of)d(this)i(p)q(oin)o(ter)g(can)f(b)q(e)h (used)g(to)e(implicitly)22 b(sp)q(ecify)d(the)f(o\013set)g(of)f(data)h (accesses,)75 264 y(b)o(y)h(using)h(the)f(routines)h(describ)q(ed)h(in) f(this)g(section.)32 b(These)20 b(routines)f(only)h(use)g(and)f(up)q (date)h(the)75 320 y(individual)g(\014le)e(p)q(oin)o(ters)g(main)o (tained)g(b)o(y)f(the)g(system.)25 b(The)17 b(shared)g(\014le)h(p)q (oin)o(ter)f(is)h(not)f(used)g(nor)75 377 y(up)q(dated.)166 433 y(The)h(individual)j(\014le)e(p)q(oin)o(ter)f(routines)g(ha)o(v)o (e)g(the)g(same)f(seman)o(tics)h(as)g(the)f(data)h(access)g(with)75 489 y(explicit)f(o\013set)e(routines)g(describ)q(ed)i(in)f(Section)g (10.3)e(with)i(the)f(follo)o(wing)h(mo)q(di\014cation:)143 583 y Fh(\017)23 b Ft(the)17 b Fq(o\013set)h Ft(is)f(de\014ned)i(to)e (b)q(e)h(the)f(curren)o(t)h(v)m(alue)g(of)f(the)g(system-main)o(tained) h(individual)j(\014le)189 640 y(p)q(oin)o(ter.)75 734 y(After)15 b(an)g(individual)k(\014le)d(p)q(oin)o(ter)g(op)q(eration)f (is)h(initiated,)g(the)g(individual)i(\014le)f(p)q(oin)o(ter)e(is)h(up) q(dated)75 790 y(to)i(p)q(oin)o(t)h(to)f(the)h(next)g(data)f(item)h (after)f(the)g(last)h(one)g(that)f(will)i(b)q(e)f(actually)h(accessed,) f(p)q(ossibly)75 846 y(accoun)o(ting)d(for)e(an)h(end)h(of)f(\014le)h (o)q(ccurrence.)75 968 y Fl(10.4.1)49 b(Blo)q(cking)18 b(Read)75 1148 y Fq(MPI)p 160 1148 V 16 w(READ)p 295 1148 V 17 w(NEXT\(fh,)d(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (status\))117 1226 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1301 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 1376 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1451 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 1526 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 1651 y Fp(int)23 b(MPI)p 245 1651 15 2 v 17 w(Read)p 358 1651 V 17 w(next\(MPI)p 567 1651 V 16 w(File)g(fh,)h(void)f(*buff,) g(MPI)p 1156 1651 V 17 w(Datatype)g(buftype,)393 1707 y(int)h(bufcount,)e(MPI)p 802 1707 V 17 w(Status)h(*status\))75 1794 y(MPI)p 150 1794 V 17 w(READ)p 263 1794 V 16 w(NEXT\(FH,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)g(STATUS,)g(IERROR\))170 1850 y()g (BUFF\(*\))170 1906 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (STATUS\(MPI)p 1153 1906 V 16 w(STATUS)p 1313 1906 V 16 w(SIZE\),)h(IERROR)166 1993 y Fq(MPI)p 251 1993 14 2 v 16 w(READ)p 386 1993 V 17 w(NEXT)14 b Ft(is)g(seman)o(tically)g (iden)o(tical)h(to)e Fq(MPI)p 1156 1993 V 16 w(READ)p Ft(,)g(with)h Fq(o\013set)g Ft(set)f(to)g(the)g(curren)o(t)75 2049 y(individual)21 b(\014le)e(p)q(oin)o(ter)g(p)q(osition)g(main)o (tained)g(b)o(y)f(the)g(system.)28 b(The)18 b(\014le)h(p)q(oin)o(ter)g (is)f(up)q(dated)h(b)o(y)75 2106 y(the)c(amoun)o(t)g(of)f(data)h (requested.)75 2257 y Fq(MPI)p 160 2257 V 16 w(READ)p 295 2257 V 17 w(NEXT)p 432 2257 V 17 w(ALL\(fh,)f(bu\013,)h(buft)o(yp)q (e,)i(bufcount,)g(status\))75 2360 y Fp(int)23 b(MPI)p 245 2360 15 2 v 17 w(Read)p 358 2360 V 17 w(next)p 471 2360 V 16 w(all\(MPI)p 655 2360 V 17 w(File)g(fh,)g(void)h(*buff,)f (MPI)p 1245 2360 V 17 w(Datatype)f(buftype,)393 2417 y(int)i(bufcount,)e(MPI)p 802 2417 V 17 w(Status)h(*status\))75 2503 y(MPI)p 150 2503 V 17 w(READ)p 263 2503 V 16 w(NEXT)p 375 2503 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g(STATUS,)f (IERROR\))170 2560 y()h(BUFF\(*\))170 2616 y(INTEGER)g(FH,)h (BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2616 V 16 w(STATUS)p 1313 2616 V 16 w(SIZE\),)h(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 24 25 bop 75 -100 a Ft(24)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(READ)p 386 49 V 17 w(NEXT)p 523 49 V 17 w(ALL)17 b Ft(is)h(a)f(collectiv)o(e)i(v)o (ersion)f(of)f(the)h(blo)q(c)o(king)h Fq(MPI)p 1481 49 V 15 w(READ)p 1615 49 V 17 w(NEXT)g Ft(in)o(ter-)75 106 y(face.)g(This)12 b(routine)h(is)f(seman)o(tically)i(iden)o(tical)g(to) d Fq(MPI)p 1055 106 V 16 w(READ)p 1190 106 V 17 w(NEXT)p Ft(,)h(except)g(that)g(all)h(pro)q(cesses)f(in)75 162 y(the)g(comm)o(unicator)g(group)g(asso)q(ciated)g(with)h(the)f(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1497 162 V 16 w(READ)p 1632 162 V 17 w(NEXT)p 1769 162 V 17 w(ALL)p Ft(.)75 284 y Fl(10.4.2)49 b(Blo)q(cking)18 b(W)o(rite)75 464 y Fq(MPI)p 160 464 V 16 w(WRITE)p 319 464 V 16 w(NEXT\(fh,)e (bu\013,)f(buft)o(yp)q(e,)i(bufcount,)f(status\))117 541 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 617 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 692 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 767 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 842 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 966 y Fp(int)23 b(MPI)p 245 966 15 2 v 17 w(Write)p 382 966 V 17 w(next\(MPI)p 591 966 V 16 w(File)g(fh,)h(void)f(*buff,)g(MPI)p 1180 966 V 17 w(Datatype)f(buftype,)393 1023 y(int)i(bufcount,)e(MPI)p 802 1023 V 17 w(Status)h(*status\))75 1109 y(MPI)p 150 1109 V 17 w(WRITE)p 287 1109 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)g(STATUS,)g(IERROR\))170 1166 y()g(BUFF\(*\))170 1222 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1222 V 16 w(STATUS)p 1313 1222 V 16 w(SIZE\),)h(IERROR)166 1309 y Fq(MPI)p 251 1309 14 2 v 16 w(WRITE)p 410 1309 V 16 w(NEXT)16 b Ft(is)f(seman)o(tically)h(iden)o(tical)h(to)d Fq(MPI)p 1187 1309 V 16 w(WRITE)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)g(the)g(cur-)75 1365 y(ren)o(t)h(individual)j(\014le)e(p)q (oin)o(ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)21 b(The)16 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)75 1422 y(b)o(y)f(the)g(amoun)o(t)g(of)g(data)f(requested.)75 1573 y Fq(MPI)p 160 1573 V 16 w(WRITE)p 319 1573 V 16 w(NEXT)p 455 1573 V 17 w(ALL\(fh,)g(bu\013,)i(buft)o(yp)q(e,)g (bufcount,)h(status\))75 1676 y Fp(int)47 b(MPI)p 269 1676 15 2 v 17 w(Write)p 406 1676 V 17 w(next)p 519 1676 V 16 w(all\(MPI)p 703 1676 V 16 w(File)24 b(fh,)f(void)h(*buff,)f(MPI)p 1293 1676 V 16 w(Datatype)g(buftype,)393 1733 y(int)h(bufcount,)e(MPI)p 802 1733 V 17 w(Status)h(*status\))75 1819 y(MPI)p 150 1819 V 17 w(WRITE)p 287 1819 V 16 w(NEXT)p 399 1819 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(STATUS,)h(IERROR\))170 1876 y()g(BUFF\(*\))170 1932 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 1932 V 16 w(STATUS)p 1313 1932 V 16 w(SIZE\),)h(IERROR)166 2018 y Fq(MPI)p 251 2018 14 2 v 16 w(WRITE)p 410 2018 V 16 w(NEXT)p 546 2018 V 17 w(ALL)12 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the) h(blo)q(c)o(king)h Fq(MPI)p 1463 2018 V 15 w(WRITE)p 1621 2018 V 17 w(NEXT)f Ft(in)o(ter-)75 2075 y(face.)20 b(This)15 b(routine)g(is)g(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 1071 2075 V 16 w(WRITE)p 1230 2075 V 16 w(NEXT)p Ft(,)h(except)g(that)f(all)i(pro)q(cesses)75 2131 y(in)11 b(the)f(comm)o(unicator)g(group)g(asso)q(ciated)g(with)h(the)f(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)g(call)h Fq(MPI)p 1523 2131 V 16 w(WRITE)p 1682 2131 V 16 w(NEXT)p 1818 2131 V 17 w(ALL)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 25 26 bop 75 -100 a Fn(10.4.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (INDIVIDUAL)i(FILE)e(POINTERS)487 b Ft(25)75 49 y Fl(10.4.3)49 b(Nonblo)q(cking)19 b(Read)75 230 y Fq(MPI)p 160 230 14 2 v 16 w(IREAD)p 308 230 V 16 w(NEXT\(fh,)d(bu\013,)f(buft)o(yp)q (e,)i(bufcount,)g(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i (user's)f(bu\013er)h(\(in)o(teger\))117 457 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 532 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 607 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 732 y Fp(int)23 b(MPI)p 245 732 15 2 v 17 w(Iread)p 382 732 V 17 w(next\(MPI)p 591 732 V 16 w(File)g(fh,)h(void)f(*buff,)g(MPI)p 1180 732 V 17 w(Datatype)f (buftype,)393 788 y(int)i(bufcount,)e(MPI)p 802 788 V 17 w(Request)h(*request\))75 875 y(MPI)p 150 875 V 17 w(IREAD)p 287 875 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)g (REQUEST,)f(IERROR\))170 931 y()h(BUFF\(*\))170 988 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 1074 y Fq(MPI)p 251 1074 14 2 v 16 w(IREAD)p 399 1074 V 16 w(NEXT)12 b Ft(is)g(seman)o(tically)g(iden)o(tical)h(to)d Fq(MPI)p 1157 1074 V 16 w(IREAD)p Ft(,)h(with)g Fq(o\013set)h Ft(set)e(to)h(the)g(curren)o(t)75 1130 y(individual)21 b(\014le)e(p)q(oin)o(ter)g(p)q(osition)g(main)o(tained)g(b)o(y)f(the)g (system.)28 b(The)18 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o (y)75 1187 y(the)c(amoun)o(t)g(of)f(data)h(requested.)75 1338 y Fq(MPI)p 160 1338 V 16 w(IREAD)p 308 1338 V 16 w(NEXT)p 444 1338 V 17 w(ALL\(fh,)g(bu\013,)g(buft)o(yp)q(e,)i (bufcount,)g(request\))75 1442 y Fp(int)23 b(MPI)p 245 1442 15 2 v 17 w(Iread)p 382 1442 V 17 w(next)p 495 1442 V 16 w(all\(MPI)p 679 1442 V 17 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1269 1442 V 16 w(Datatype)g(buftype,)393 1498 y(int)h(bufcount,)e(MPI)p 802 1498 V 17 w(Request)h(*request\))75 1584 y(MPI)p 150 1584 V 17 w(IREAD)p 287 1584 V 16 w(NEXT)p 399 1584 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\)) 170 1641 y()g(BUFF\(*\))170 1697 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)166 1784 y Fq(MPI)p 251 1784 14 2 v 16 w(IREAD)p 399 1784 V 16 w(NEXT)p 535 1784 V 17 w(ALL)15 b Ft(is)g(a)f(collectiv)o(e)j(v)o(ersion)e(of)f(the)h (blo)q(c)o(king)h Fq(MPI)p 1471 1784 V 16 w(IREAD)p 1619 1784 V 16 w(NEXT)g Ft(in)o(ter-)75 1840 y(face.)i(This)12 b(routine)f(is)h(seman)o(tically)g(iden)o(tical)h(to)d Fq(MPI)p 1048 1840 V 16 w(IREAD)p 1196 1840 V 16 w(NEXT)p Ft(,)h(except)h(that)e(all)i(pro)q(cesses)f(in)75 1897 y(the)g(comm)o(unicator)g(group)f(asso)q(ciated)i(with)f(the)g(\014le)h (handle)h Fq(fh)e Ft(m)o(ust)f(call)i Fq(MPI)p 1484 1897 V 16 w(IREAD)p 1632 1897 V 17 w(NEXT)p 1769 1897 V 17 w(ALL)p Ft(.)75 2018 y Fl(10.4.4)49 b(Nonblo)q(cking)19 b(W)o(rite)75 2199 y Fq(MPI)p 160 2199 V 16 w(IWRITE)p 332 2199 V 16 w(NEXT\(fh,)c(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (request\))117 2276 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 2351 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 2426 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 2501 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 2577 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2701 y Fp(int)23 b(MPI)p 245 2701 15 2 v 17 w(Iwrite)p 406 2701 V 17 w(next\(MPI)p 615 2701 V 16 w(File)g(fh,)g(void)h(*buff,) f(MPI)p 1204 2701 V 17 w(Datatype)f(buftype,)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 26 27 bop 75 -100 a Ft(26)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)393 49 y Fp(int)24 b(bufcount,)e(MPI)p 802 49 15 2 v 17 w(Request)h(*request\))75 136 y(MPI)p 150 136 V 17 w(IWRITE)p 311 136 V 16 w(NEXT\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(REQUEST,)h(IERROR\))170 192 y()g(BUFF\(*\))170 249 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 335 y Fq(MPI)p 251 335 14 2 v 16 w(IWRITE)p 423 335 V 16 w(NEXT)f Ft(is)g(seman)o(tically)h(iden)o(tical)h(to)d Fq(MPI)p 1234 335 V 16 w(IWRITE)p Ft(,)g(with)h Fq(o\013set)g Ft(set)g(to)f(the)75 392 y(curren)o(t)11 b(individual)j(\014le)e(p)q (oin)o(ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)18 b(The)11 b(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)75 448 y(b)o(y)k(the)g(amoun)o(t)g(of)g(data)f(requested.)75 599 y Fq(MPI)p 160 599 V 16 w(IWRITE)p 332 599 V 16 w(NEXT)p 468 599 V 17 w(ALL\(fh,)g(bu\013,)h(buft)o(yp)q(e,)i(bufcount,)g (request\))75 703 y Fp(int)23 b(MPI)p 245 703 15 2 v 17 w(Iwrite)p 406 703 V 17 w(next)p 519 703 V 16 w(all\(MPI)p 703 703 V 16 w(File)h(fh,)f(void)h(*buff,)f(MPI)p 1293 703 V 16 w(Datatype)g(buftype,)393 759 y(int)h(bufcount,)e(MPI)p 802 759 V 17 w(Request)h(*request\))75 846 y(MPI)p 150 846 V 17 w(IWRITE)p 311 846 V 16 w(NEXT)p 423 846 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\))170 902 y()g(BUFF\(*\))170 958 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(REQUEST,)h(IERROR)166 1045 y Fq(MPI)p 251 1045 14 2 v 16 w(IWRITE)p 423 1045 V 16 w(NEXT)p 559 1045 V 17 w(ALL)15 b Ft(is)h(a)f(collectiv)o(e)i(v)o(ersion)f(of)f(the) h(blo)q(c)o(king)h Fq(MPI)p 1501 1045 V 16 w(IWRITE)p 1673 1045 V 16 w(NEXT)f Ft(in-)75 1101 y(terface.)26 b(This)17 b(routine)h(is)g(seman)o(tically)g(iden)o(tical)h(to)e Fq(MPI)p 1149 1101 V 15 w(IWRITE)p 1320 1101 V 16 w(NEXT)p Ft(,)h(except)f(that)g(all)h(pro-)75 1158 y(cesses)10 b(in)h(the)g(comm)o(unicator)f(group)f(asso)q(ciated)i(with)f(the)g (\014le)i(handle)f Fq(fh)g Ft(m)o(ust)e(call)i Fq(MPI)p 1647 1158 V 16 w(IWRITE)p 1819 1158 V 16 w(NEXT)p 1955 1158 V 17 w(ALL)p Ft(.)75 1280 y Fl(10.4.5)49 b(Seek)16 b(\(indep)q(endent\))75 1460 y Fq(MPI)p 160 1460 V 16 w(SEEK\(fh,)f(o\013set,)g(whence\))117 1537 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1612 y(IN)155 b Fq(o\013set)484 b Ff(File)14 b(o\013set)g(\(o\013set\)) 117 1687 y(IN)155 b Fq(whence)450 b Ff(Up)q(date)15 b(mo)q(de)e(\(in)o (teger\))75 1812 y Fp(int)23 b(MPI)p 245 1812 15 2 v 17 w(Seek\(MPI)p 454 1812 V 16 w(File)h(fh,)f(MPI)p 757 1812 V 17 w(Offset)g(offset,)g(MPI)p 1204 1812 V 17 w(Whence)g (whence\))75 1898 y(MPI)p 150 1898 V 17 w(SEEK\(FH,)g(OFFSET,)f (WHENCE\))170 1955 y(INTEGER)h(FH,)h(WHENCE)170 2011 y(INTEGER*8)f(OFFSET)166 2098 y Fq(MPI)p 251 2098 14 2 v 16 w(SEEK)17 b Ft(up)q(dates)h(the)g(individual)i(\014le)f(p)q(oin) o(ter)f(according)g(to)e Fq(whence)p Ft(,)k(whic)o(h)e(could)g(ha)o(v)o (e)75 2154 y(the)d(follo)o(wing)h(p)q(ossible)h(v)m(alues:)143 2248 y Fh(\017)23 b Fg(MPI)p 266 2248 13 2 v 14 w(SEEK)p 382 2248 V 14 w(SET)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)f Fq(o\013set)143 2342 y Fh(\017)23 b Fg(MPI)p 266 2342 V 14 w(SEEK)p 382 2342 V 14 w(CUR)p Ft(:)14 b(the)h(p)q(oin)o(ter)h(is) g(set)f(to)f(the)h(curren)o(t)h(p)q(oin)o(ter)f(p)q(osition)i(plus)f Fq(o\013set)143 2435 y Fh(\017)23 b Fg(MPI)p 266 2435 V 14 w(SEEK)p 382 2435 V 14 w(END)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is) h(set)f(to)g(the)g(end)h(of)e(the)i(\014le)g(plus)g Fq(o\013set)166 2529 y Ft(The)c Fq(o\013set)g Ft(is)g(relativ)o(e)h(to)e(the)h(\014let) o(yp)q(e)h(\(not)e(coun)o(ting)i(holes\),)f(in)h(units)f(of)g Fq(et)o(yp)q(e)p Ft(.)20 b(The)12 b(constan)o(t)75 2586 y Fg(MPI)p 152 2586 V 14 w(OFFSET)p 321 2586 V 14 w(ZERO)j Ft(is)h(de\014ned)g(to)f(b)q(e)h(o\013set)e(zero.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 27 28 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(27)75 49 y Fm(10.5)60 b(Data)19 b(Access)h(with)g(Sha)n(red)f(File)g(P)n(ointers)75 233 y Fe(Discussion:)c Ff(There)f(is)e(some)g(con)o(tro)o(v)o(ersy)h (as)g(to)f(whether)i(w)o(e)f(should)f(k)o(eep)i(shared)f(\014le)g(p)q (oin)o(ters,)g(and)f(if)g(w)o(e)75 290 y(do,)h(the)i(exact)f(seman)o (tics)g(of)f(shared)i(\014le)f(p)q(oin)o(ter)g(op)q(erations.)166 429 y Ft(The)g(system)f(main)o(tains)h(exactly)g(one)g(shared)g(\014le) h(p)q(oin)o(ter)f(p)q(er)g(collectiv)o(e)i Fq(MPI)p 1580 429 14 2 v 16 w(OPEN)e Ft(\(shared)75 485 y(among)19 b(pro)q(cesses)h(in)h(the)f(comm)o(unicator)f(group\).)33 b(The)20 b(curren)o(t)g(v)m(alue)h(of)e(this)h(p)q(oin)o(ter)g(can)g(b) q(e)75 542 y(used)h(to)f(implicitly)j(sp)q(ecify)f(the)e(o\013set)g(of) g(data)f(accesses,)j(b)o(y)e(using)i(the)e(routines)h(describ)q(ed)h (in)75 598 y(this)16 b(section.)k(These)c(routines)g(only)g(use)f(and)h (up)q(date)g(the)f(shared)g(\014le)i(p)q(oin)o(ter)e(main)o(tained)i(b) o(y)e(the)75 655 y(system.)k(The)d(individual)i(\014le)e(p)q(oin)o (ters)g(are)f(not)g(used)h(nor)f(up)q(dated.)166 711 y(The)c(shared)g(\014le)i(p)q(oin)o(ter)e(routines)h(ha)o(v)o(e)e(the)i (same)e(seman)o(tics)i(as)e(the)i(data)e(access)h(with)h(explicit)75 768 y(o\013set)i(routines)i(describ)q(ed)h(in)f(Section)g(10.3)e(with)i (the)f(follo)o(wing)h(mo)q(di\014cations:)143 862 y Fh(\017)23 b Ft(the)f Fq(o\013set)h Ft(is)g(de\014ned)h(to)e(b)q(e)i(the)e(curren) o(t)h(v)m(alue)h(of)e(the)h(system-main)o(tained)g(shared)g(\014le)189 918 y(p)q(oin)o(ter,)143 1012 y Fh(\017)g Ft(the)14 b(e\013ect)g(of)f (m)o(ultiple)j(calls)g(to)d(shared)h(\014le)h(p)q(oin)o(ter)g(routines) g(is)f(de\014ned)i(to)d(b)q(eha)o(v)o(e)i(as)e(if)i(the)189 1068 y(calls)h(w)o(ere)f(serialized,)i(and)143 1162 y Fh(\017)23 b Ft(the)17 b(use)h(of)e(shared)i(\014le)g(p)q(oin)o(ter)g (routines)f(is)h(erroneous)f(unless)i(all)f(pro)q(cesses)f(use)h(the)f (same)189 1219 y Fq(\014let)o(yp)q(e)f Ft(when)g(op)q(ening)g(the)g (\014le.)75 1312 y(The)e(serialization)h(ordering)e(is)h(not)f (deterministic.)21 b(The)14 b(user)f(needs)i(to)d(use)i(other)f(sync)o (hronization)75 1369 y(means)i(to)g(enforce)g(a)g(sp)q(eci\014c)i (order.)166 1425 y(After)j(a)h(shared)g(\014le)g(p)q(oin)o(ter)h(op)q (eration)f(is)g(initiated,)i(the)e(shared)g(\014le)h(p)q(oin)o(ter)f (is)g(up)q(dated)75 1482 y(to)d(p)q(oin)o(t)h(to)f(the)h(next)g(data)f (item)h(after)f(the)g(last)h(one)g(that)f(will)i(b)q(e)f(actually)h (accessed,)f(p)q(ossibly)75 1538 y(accoun)o(ting)d(for)e(an)h(end)h(of) f(\014le)h(o)q(ccurrence.)189 1644 y Fk(A)n(dvic)n(e)i(to)i(users.)60 b Ft(In)20 b(order)e(to)g(ensure)h(that)f(data)g(access)h(using)g(the)g (shared)g(\014le)g(p)q(oin)o(ter)189 1701 y(is)g(w)o(ell)h(de\014ned,)h (all)e(pro)q(cesses)h(m)o(ust)e(use)h(the)g(same)g Fq(\014let)o(yp)q(e) h Ft(when)g(op)q(ening)g(the)f(\014le.)32 b(F)l(or)189 1757 y(example,)14 b(consider)g(a)f(single)h Fq(MPI)p 798 1757 V 16 w(OPEN)g Ft(in)g(whic)o(h)g(pro)q(cess)f(1)g(uses)h (\014let)o(yp)q(e)g Fq(MPI)p 1655 1757 V 16 w(CHAR)g Ft(and)189 1814 y(pro)q(cess)k(2)f(uses)h(\014let)o(yp)q(e)h Fq(MPI)p 737 1814 V 16 w(DOUBLE)p Ft(.)f(If)g(pro)q(cess)g(1)g(reads)f (a)h(single)h(c)o(haracter)e(from)g(the)189 1870 y(\014le,)i(the)f (shared)g(\014le)g(p)q(oin)o(ter)h(p)q(oin)o(ts)f(to)f(the)h(second)g (c)o(haracter)f(relativ)o(e)i(to)e(pro)q(cess)h(1,)g(but)189 1927 y(p)q(oin)o(ts)13 b(in)o(to)f(the)h(middle)h(of)e(the)h(\014rst)f (double)i(relativ)o(e)f(to)f(pro)q(cess)h(2.)18 b(A)o(t)12 b(this)h(p)q(oin)o(t,)h(the)e(e\013ect)189 1983 y(of)i(a)h(shared)h(p)q (oin)o(ter)f(op)q(eration)h(on)f(pro)q(cess)g(2)g(is)h(ill-de\014ned.) 23 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(users.)p Ft(\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 28 30 28 29 bop 75 -100 a Ft(28)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.5.1)49 b(Indep)q(endent)16 b(Op)q(erations)75 135 y Fq(Blo)q(cking)g(Read)g(\(indep)q(endent)q(\))75 315 y(MPI)p 160 315 14 2 v 16 w(READ)p 295 315 V 17 w(SHARED\(fh,)g (bu\013,)f(buft)o(yp)q(e,)i(bufcount,)g(status\))117 393 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h (\(handle\))117 468 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o(teger\))117 543 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o (yp)q(e)f(\(handle\))117 618 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ) o(e)g(in)o(teger\))117 693 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 818 y Fp(int)23 b(MPI)p 245 818 15 2 v 17 w(Read)p 358 818 V 17 w(shared\(MPI)p 615 818 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1204 818 V 17 w(Datatype)f(buftype,)393 874 y(int)i(bufcount,)e(MPI)p 802 874 V 17 w(Status)h(*status\))75 960 y(MPI)p 150 960 V 17 w(READ)p 263 960 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(STATUS,)h(IERROR\))170 1017 y()g(BUFF\(*\))170 1073 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 1073 V 16 w(STATUS)p 1313 1073 V 16 w(SIZE\),)h(IERROR)166 1160 y Fq(MPI)p 251 1160 14 2 v 16 w(READ)p 386 1160 V 17 w(SHARED)15 b Ft(is)f(seman)o(tically)i(iden)o(tical)g(to)d Fq(MPI)p 1216 1160 V 16 w(READ)p Ft(,)h(with)h Fq(o\013set)f Ft(set)g(to)f(the)i(cur-)75 1216 y(ren)o(t)g(shared)g(\014le)i(p)q(oin) o(ter)f(p)q(osition)g(main)o(tained)g(b)o(y)g(the)f(system.)20 b(Shared)c(\014le)g(p)q(oin)o(ter)g(calls)h(within)75 1273 y(a)e(single)i(\014le)g(handle)g(comm)o(unicator)e(group)h(are)f (guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially)g(consisten)o(t.)22 b(The)75 1329 y(\014le)16 b(p)q(oin)o(ter)g(is)g(up)q(dated)g(b)o(y)f (the)g(amoun)o(t)f(of)h(data)g(requested.)75 1449 y Fq(Blo)q(cking)h(W) o(rite)f(\(indep)q(endent\))75 1630 y(MPI)p 160 1630 V 16 w(WRITE)p 319 1630 V 16 w(SHARED\(fh,)h(bu\013,)f(buft)o(yp)q(e,)i (bufcount,)g(status\))117 1707 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1782 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's) f(bu\013er)h(\(in)o(teger\))117 1857 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1932 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2007 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c (\(Status\))75 2132 y Fp(int)23 b(MPI)p 245 2132 15 2 v 17 w(Write)p 382 2132 V 17 w(shared\(MPI)p 639 2132 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1228 2132 V 16 w(Datatype)g(buftype,)393 2188 y(int)h(bufcount,)e(MPI)p 802 2188 V 17 w(Status)h(*status\))75 2275 y(MPI)p 150 2275 V 17 w(WRITE)p 287 2275 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g (BUFCOUNT,)f(STATUS,)h(IERROR\))170 2331 y()g(BUFF\(*\))170 2388 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(STATUS\(MPI)p 1153 2388 V 16 w(STATUS)p 1313 2388 V 16 w(SIZE\),)h(IERROR)166 2474 y Fq(MPI)p 251 2474 14 2 v 16 w(WRITE)p 410 2474 V 16 w(SHARED)d Ft(is)f(seman)o(tically)h(iden)o(tical)g(to)e Fq(MPI)p 1262 2474 V 16 w(WRITE)p Ft(,)g(with)h Fq(o\013set)g Ft(set)g(to)f(the)75 2531 y(curren)o(t)10 b(shared)h(\014le)g(p)q(oin)o (ter)g(p)q(osition)g(main)o(tained)g(b)o(y)f(the)h(system.)18 b(Shared)10 b(\014le)i(p)q(oin)o(ter)e(calls)i(within)75 2587 y(a)j(single)i(\014le)g(handle)g(comm)o(unicator)e(group)h(are)f (guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially)g(consisten)o(t.)22 b(The)75 2643 y(\014le)16 b(p)q(oin)o(ter)g(is)g(up)q(dated)g(b)o(y)f (the)g(amoun)o(t)f(of)h(data)g(requested.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 29 31 29 30 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(29)75 49 y Fq(Nonblo)q(cking)16 b(Read)g(\(indep)q(enden)q(t\))75 230 y(MPI)p 160 230 14 2 v 16 w(IREAD)p 308 230 V 16 w(SHARED\(fh,)g(bu\013,)g(buft)o(yp)q (e,)g(bufcount,)h(request\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 382 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i (user's)f(bu\013er)h(\(in)o(teger\))117 457 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 532 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 607 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 732 y Fp(int)23 b(MPI)p 245 732 15 2 v 17 w(Iread)p 382 732 V 17 w(shared\(MPI)p 639 732 V 16 w(File)g(fh,)g(void)h(*buff,)f(MPI)p 1228 732 V 16 w(Datatype)g (buftype,)393 788 y(int)h(bufcount,)e(MPI)p 802 788 V 17 w(Request)h(*request\))75 875 y(MPI)p 150 875 V 17 w(IREAD)p 287 875 V 16 w(SHARED\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f (REQUEST,)h(IERROR\))170 931 y()g(BUFF\(*\))170 988 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 1074 y Fq(MPI)p 251 1074 14 2 v 16 w(IREAD)p 399 1074 V 16 w(SHARED)f Ft(is)g(seman)o(tically)g(iden)o(tical)g(to)e Fq(MPI)p 1262 1074 V 16 w(IREAD)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)f(the)75 1130 y(curren)o(t)h(shared)g(\014le)h(p)q(oin)o (ter)f(p)q(osition)h(main)o(tained)g(b)o(y)f(the)g(system.)37 b(Shared)21 b(\014le)h(p)q(oin)o(ter)g(calls)75 1187 y(within)13 b(a)f(single)i(\014le)f(handle)g(comm)o(unicator)f(group)g (are)g(guaran)o(teed)g(to)f(b)q(e)i(sequen)o(tially)h(consisten)o(t.)75 1243 y(The)h(\014le)i(p)q(oin)o(ter)e(is)h(up)q(dated)g(b)o(y)f(the)h (amoun)o(t)e(of)h(data)f(requested.)75 1363 y Fq(Nonblo)q(cking)i(W)o (rite)f(\(indep)q(enden)q(t\))75 1544 y(MPI)p 160 1544 V 16 w(IWRITE)p 332 1544 V 16 w(SHARED\(fh,)h(bu\013,)f(buft)o(yp)q(e,) i(bufcount,)g(request\))117 1621 y Ff(IN)155 b Fq(fh)550 b Ff(V)m(alid)13 b(\014le)g(handle)h(\(handle\))117 1696 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's) f(bu\013er)h(\(in)o(teger\))117 1771 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1846 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 1922 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2046 y Fp(int)23 b(MPI)p 245 2046 15 2 v 17 w(Iwrite)p 406 2046 V 17 w(shared\(MPI)p 663 2046 V 15 w(File)h(fh,)f(void)g(*buff,)h(MPI)p 1252 2046 V 16 w(Datatype)f(buftype,)393 2102 y(int)h(bufcount,)e(MPI)p 802 2102 V 17 w(Request)h(*request\))75 2189 y(MPI)p 150 2189 V 17 w(IWRITE)p 311 2189 V 16 w(SHARED\(FH,)g(BUFF,)g (BUFTYPE,)g(BUFCOUNT,)f(REQUEST,)h(IERROR\))170 2245 y()g(BUFF\(*\))170 2302 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (REQUEST,)h(IERROR)166 2388 y Fq(MPI)p 251 2388 14 2 v 16 w(IWRITE)p 423 2388 V 16 w(SHARED)17 b Ft(is)g(seman)o(tically)g (iden)o(tical)h(to)d Fq(MPI)p 1262 2388 V 16 w(IWRITE)p Ft(,)h(with)g Fq(o\013set)g Ft(set)g(to)g(the)75 2445 y(curren)o(t)10 b(shared)h(\014le)g(p)q(oin)o(ter)g(p)q(osition)g(main) o(tained)g(b)o(y)f(the)h(system.)18 b(Shared)10 b(\014le)i(p)q(oin)o (ter)e(calls)i(within)75 2501 y(a)j(single)i(\014le)g(handle)g(comm)o (unicator)e(group)h(are)f(guaran)o(teed)g(to)g(b)q(e)i(sequen)o(tially) g(consisten)o(t.)22 b(The)75 2558 y(\014le)16 b(p)q(oin)o(ter)g(is)g (up)q(dated)g(b)o(y)f(the)g(amoun)o(t)f(of)h(data)g(requested.)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 30 32 30 31 bop 75 -100 a Ft(30)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fl(10.5.2)49 b(Collective)18 b(Op)q(erations)75 135 y Ft(Unlik)o(e)h(other)e(collectiv)o(e)j(op)q(erations,)e(a)f (barrier)h(sync)o(hronization)g(\(among)f(the)g(pro)q(cesses)h(asso)q (ci-)75 192 y(ated)g(with)g(the)g(comm)o(unicator)f(in)o(v)o(olv)o(ed)i (in)g(the)f(op)q(eration\))g(is)g(enforced)g(for)g(shared)g(\014le)g(p) q(oin)o(ter)75 248 y(op)q(erations.)166 304 y(The)12 b(seman)o(tics)g(of)f(a)g(collectiv)o(e)j(access)d(using)i(a)e(shared)h (\014le)h(p)q(oin)o(ter)f(is)g(that)f(the)h(accesses)g(to)f(the)75 361 y(\014le)i(will)h(b)q(e)f(in)g(the)f(order)g(determined)h(b)o(y)f (the)g(ranks)g(of)g(the)g(pro)q(cesses)g(within)i(the)e(group)g(asso)q (ciated)75 417 y(with)k(the)g(comm)o(unicator.)22 b(In)16 b(addition,)h(in)g(order)e(to)h(prev)o(en)o(t)f(subsequen)o(t)i(shared) f(o\013set)f(accesses)75 474 y(b)o(y)c(the)h(same)e(pro)q(cesses)i(to)f (in)o(terfere)g(with)h(this)g(collectiv)o(e)h(access,)e(the)h(call)g (can)g(only)f(return)h(after)e(all)75 530 y(the)15 b(pro)q(cesses)g (within)h(the)f(group)g(ha)o(v)o(e)f(initiated)j(their)e(accesses.)20 b(In)c(other)e(terms,)h(this)g(seman)o(tics)75 587 y(implies)j(a)d (barrier)h(sync)o(hronization)g(among)f(all)h(the)g(pro)q(cesses)g(of)f (the)h(group,)f(that)f(tak)o(es)h(place)i(at)75 643 y(the)h(outset)f (of)h(a)g(blo)q(c)o(king)h(op)q(eration)f(or)g(at)f(the)h(initiation)i (of)d(a)h(non)o(blo)q(c)o(king)h(op)q(eration.)29 b(When)75 700 y(the)15 b(call)i(returns,)e(the)g(shared)g(\014le)i(p)q(oin)o(ter) e(p)q(oin)o(ts)h(to)f(the)g(next)h(b)o(yte)f(accessible,)h(according)g (to)f(the)75 756 y(\014let)o(yp)q(e)h(used)g(b)o(y)f(all)h(pro)q (cesses,)g(after)e(the)h(last)g(b)o(yte)g(requested.)75 873 y Fq(Blo)q(cking)h(Read)g(\(collective\))75 1053 y(MPI)p 160 1053 14 2 v 16 w(READ)p 295 1053 V 17 w(SHARED)p 488 1053 V 17 w(ALL\(fh,)f(bu\013,)g(buft)o(yp)q(e,)i(bufcount,)g (status\))117 1130 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 1202 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 1274 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 1346 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 1417 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c (\(Status\))75 1542 y Fp(int)47 b(MPI)p 269 1542 15 2 v 17 w(Read)p 382 1542 V 17 w(shared)p 543 1542 V 16 w(all\(MPI)p 727 1542 V 16 w(File)24 b(fh,)f(void)g(*buff,)g(MPI)p 1316 1542 V 17 w(Datatype)g(buftype,)393 1598 y(int)h(bufcount,)e(MPI)p 802 1598 V 17 w(Status)h(*status\))75 1685 y(MPI)p 150 1685 V 17 w(READ)p 263 1685 V 16 w(SHARED)p 423 1685 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)g(BUFCOUNT,)f(STATUS,)h(IERROR\))170 1741 y()g(BUFF\(*\))170 1798 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 1798 V 16 w(STATUS)p 1313 1798 V 16 w(SIZE\),)h(IERROR)166 1884 y Fq(MPI)p 251 1884 14 2 v 16 w(READ)p 386 1884 V 17 w(SHARED)p 579 1884 V 17 w(ALL)17 b Ft(is)g(a)g(collectiv)o(e)h(v)o(ersion)g(of)e (the)h Fq(MPI)p 1350 1884 V 16 w(READ)p 1485 1884 V 17 w(SHARED)h Ft(in)o(terface.)75 1940 y(This)i(routine)f(m)o(ust)g(b)q(e) h(called)h(b)o(y)e(all)h(pro)q(cesses)f(in)h(the)g(comm)o(unicator)e (group)h(asso)q(ciated)h(with)75 1997 y(the)f(\014le)g(handle)h Fq(fh)p Ft(.)30 b(Eac)o(h)18 b(pro)q(cess)h(ma)o(y)f(pass)g(di\013eren) o(t)h(argumen)o(t)f(v)m(alues)h(for)f(the)h Fq(buft)o(yp)q(e)p Ft(,)i(and)75 2053 y Fq(bufcount)16 b Ft(argumen)o(ts.)i(After)c(all)g (pro)q(cesses)g(of)f(the)h(group)f(ha)o(v)o(e)h(issued)g(their)h(resp)q (ectiv)o(e)f(calls,)h(eac)o(h)75 2110 y(pro)q(cess)h(attempts)e(to)g (read,)h(from)g(the)g(\014le)i(asso)q(ciated)e(with)h Fq(fh)p Ft(,)f(a)g(total)g(n)o(um)o(b)q(er)g(of)g Fq(bufcount)j Ft(data)75 2166 y(items)e(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)24 b(F)l(or)15 b(eac)o(h)h(pro)q(cess,)h(the)f(lo)q(cation)h(in)g (the)75 2223 y(\014le)g(at)d(whic)o(h)j(data)e(is)h(read)f(is)h(the)g (p)q(osition)g(at)f(whic)o(h)h(the)g(shared)g(\014le)g(p)q(oin)o(ter)g (w)o(ould)g(b)q(e)g(after)f(all)75 2279 y(pro)q(cesses)k(whose)g(ranks) f(within)i(the)e(group)h(are)f(less)h(than)g(that)f(of)g(this)h(pro)q (cess)g(had)g(read)f(their)75 2336 y(data.)h Fq(MPI)p 281 2336 V 15 w(READ)p 415 2336 V 17 w(SHARED)p 608 2336 V 18 w(ALL)13 b Ft(returns)g(the)h(n)o(um)o(b)q(er)f(of)g Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)e(read)f(in)h Fq(status)p Ft(.)22 b(The)75 2392 y(shared)f(\014le)g(p)q(oin)o(ter)g(is)g(up)q (dated)h(b)o(y)e(the)h(amoun)o(ts)f(of)g(data)g(requested)h(b)o(y)f (all)i(pro)q(cesses)f(of)f(the)75 2449 y(group.)189 2534 y Fk(A)n(dvic)n(e)14 b(to)h(implementors.)39 b Ft(Accesses)15 b(to)e(the)h(data)f(requested)i(b)o(y)f(all)h(pro)q(cesses)f(do)g(not)f (ha)o(v)o(e)189 2591 y(to)f(b)q(e)i(serialized.)21 b(Once)14 b(all)g(pro)q(cesses)f(ha)o(v)o(e)g(issued)h(their)f(requests,)g(lo)q (cations)h(within)g(the)f(\014le)189 2647 y(for)i(all)i(accesses)g(can) f(b)q(e)h(computed,)g(and)f(accesses)h(can)f(pro)q(ceed)h(indep)q (enden)o(tly)i(from)d(eac)o(h)189 2704 y(other,)e(p)q(ossibly)j(in)f (parallel.)21 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 31 33 31 32 bop 75 -100 a Fn(10.5.)34 b(D)o(A)l(T)l(A)15 b(A)o(CCESS)g(WITH)g (SHARED)h(FILE)f(POINTERS)578 b Ft(31)75 49 y Fq(Blo)q(cking)16 b(W)o(rite)f(\(collective\))75 230 y(MPI)p 160 230 14 2 v 16 w(WRITE)p 319 230 V 16 w(SHARED)p 511 230 V 18 w(ALL\(fh,)f(bu\013,)h(buft)o(yp)q(e,)i(bufcount,)g(status\))117 307 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i (handle)g(\(handle\))117 381 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f(bu\013er)h(\(in)o (teger\))117 455 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 530 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o (ts)g(\(nonnegativ)o(e)g(in)o(teger\))117 604 y(OUT)108 b Fq(status)476 b Ff(Status)15 b(information)c(\(Status\))75 728 y Fp(int)47 b(MPI)p 269 728 15 2 v 17 w(Write)p 406 728 V 17 w(shared)p 567 728 V 16 w(all\(MPI)p 751 728 V 16 w(File)23 b(fh,)h(void)f(*buff,)g(MPI)p 1340 728 V 17 w(Datatype)g(buftype,)393 785 y(int)h(bufcount,)e(MPI)p 802 785 V 17 w(Status)h(*status\))75 871 y(MPI)p 150 871 V 17 w(WRITE)p 287 871 V 16 w(SHARED)p 447 871 V 17 w(ALL\(FH,)g(BUFF,)g(BUFTYPE,)f(BUFCOUNT,)h(STATUS,)g(IERROR\))170 928 y()g(BUFF\(*\))170 984 y(INTEGER)g(FH,)h(BUFTYPE,)f (BUFCOUNT,)f(STATUS\(MPI)p 1153 984 V 16 w(STATUS)p 1313 984 V 16 w(SIZE\),)h(IERROR)166 1071 y Fq(MPI)p 251 1071 14 2 v 16 w(WRITE)p 410 1071 V 16 w(SHARED)p 602 1071 V 18 w(ALL)12 b Ft(is)i(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the)g (blo)q(c)o(king)i Fq(MPI)p 1526 1071 V 16 w(WRITE)p 1685 1071 V 16 w(SHARED)75 1127 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f (in)g(the)f(comm)o(unicator)g(group)g(asso)q(ciated)h(with)g(the)f (\014le)h(handle)h Fq(fh)f Ft(m)o(ust)75 1183 y(call)k Fq(MPI)p 247 1183 V 16 w(WRITE)p 406 1183 V 16 w(SHARED)p 598 1183 V 18 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)g(ma)o(y)g(pass)g (di\013eren)o(t)g(argumen)o(t)f(v)m(alues)i(for)f(the)75 1240 y Fq(buft)o(yp)q(e)13 b Ft(and)f Fq(bufcount)h Ft(argumen)o(ts.)18 b(After)11 b(all)h(pro)q(cesses)g(of)e(the)i(group)f(ha)o(v)o(e)g (issued)h(their)g(resp)q(ectiv)o(e)75 1296 y(calls,)22 b(eac)o(h)d(pro)q(cess)h(attempts)f(to)g(write,)i(in)o(to)e(the)h (\014le)h(asso)q(ciated)f(with)g Fq(fh)p Ft(,)h(a)e(total)g(n)o(um)o(b) q(er)h(of)75 1353 y Fq(bufcount)13 b Ft(data)e(items)g(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(con)o(tained)h(in)g(the)f(user's)g (bu\013er)g Fq(bu\013)p Ft(.)19 b(F)l(or)11 b(eac)o(h)g(pro)q(cess,)75 1409 y(the)g(lo)q(cation)h(in)h(the)e(\014le)h(at)f(whic)o(h)h(data)f (is)h(written)f(is)h(the)f(p)q(osition)h(at)f(whic)o(h)h(the)g(shared)f (\014le)h(p)q(oin)o(ter)75 1466 y(w)o(ould)j(b)q(e)g(after)e(all)i(pro) q(cesses)g(whose)f(ranks)g(within)h(the)g(group)f(are)g(less)h(than)f (that)f(of)h(this)h(pro)q(cess)75 1522 y(had)g(written)g(their)g(data.) k Fq(MPI)p 636 1522 V 16 w(WRITE)p 795 1522 V 17 w(SHARED)p 988 1522 V 17 w(ALL)14 b Ft(returns)h(the)g(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)h Ft(elemen)o(ts)75 1579 y(written)g(in)h Fq(status)p Ft(.)26 b(The)16 b(shared)h(\014le)g(p)q(oin)o(ter)g(is)g (up)q(dated)g(b)o(y)f(the)g(amoun)o(ts)g(of)g(data)f(requested)i(b)o(y) 75 1635 y(all)f(pro)q(cesses)g(of)e(the)i(group.)189 1736 y Fk(A)n(dvic)n(e)e(to)h(implementors.)39 b Ft(Accesses)15 b(to)e(the)h(data)f(requested)i(b)o(y)f(all)h(pro)q(cesses)f(do)g(not)f (ha)o(v)o(e)189 1792 y(to)f(b)q(e)i(serialized.)21 b(Once)14 b(all)g(pro)q(cesses)f(ha)o(v)o(e)g(issued)h(their)f(requests,)g(lo)q (cations)h(within)g(the)f(\014le)189 1849 y(for)i(all)i(accesses)g(can) f(b)q(e)h(computed,)g(and)f(accesses)h(can)f(pro)q(ceed)h(indep)q (enden)o(tly)i(from)d(eac)o(h)189 1905 y(other,)e(p)q(ossibly)j(in)f (parallel.)21 b(\()p Fk(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Ft(\))75 2025 y Fq(Nonblo)q(cking)f(Read)g(\(collective\))75 2205 y(MPI)p 160 2205 V 16 w(IREAD)p 308 2205 V 16 w(SHARED)p 500 2205 V 18 w(ALL\(fh,)e(bu\013,)i(buft)o(yp)q(e,)g(bufcount,)h (request\))117 2282 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 2356 y(OUT)108 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the)i(user's)f (bu\013er)h(\(in)o(teger\))117 2431 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f(\(handle\))117 2505 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2579 y(OUT)108 b Fq(request)452 b Ff(Read)14 b(request)h(handle)f (\(handle\))75 2704 y Fp(int)23 b(MPI)p 245 2704 15 2 v 17 w(Iread)p 382 2704 V 17 w(shared)p 543 2704 V 16 w(all\(MPI)p 727 2704 V 16 w(File)h(fh,)f(void)g(*buff,)g(MPI)p 1316 2704 V 17 w(Datatype)g(buftype,)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 32 34 32 33 bop 75 -100 a Ft(32)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)393 49 y Fp(int)24 b(bufcount,)e(MPI)p 802 49 15 2 v 17 w(Request)h(*request\))75 136 y(MPI)p 150 136 V 17 w(IREAD)p 287 136 V 16 w(SHARED)p 447 136 V 17 w(ALL\(FH,)g(BUFF,) g(BUFTYPE,)f(BUFCOUNT,)h(REQUEST,)g(IERROR\))170 192 y()g(BUFF\(*\))170 249 y(INTEGER)g(FH,)h(BUFTYPE,)f(BUFCOUNT,)f (REQUEST,)h(IERROR)166 335 y Fq(MPI)p 251 335 14 2 v 16 w(IREAD)p 399 335 V 16 w(SHARED)p 591 335 V 18 w(ALL)9 b Ft(is)i(a)f(collectiv)o(e)i(v)o(ersion)e(of)g(the)g(non)o(blo)q(c)o (king)i Fq(MPI)p 1563 335 V 15 w(IREAD)p 1710 335 V 17 w(SHARED)75 392 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f(in)g(the)f (comm)o(unicator)g(group)g(asso)q(ciated)h(with)g(the)f(\014le)h (handle)h Fq(fh)f Ft(m)o(ust)75 448 y(call)21 b Fq(MPI)p 248 448 V 16 w(IREAD)p 396 448 V 17 w(SHARED)p 589 448 V 17 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)h(ma)o(y)e(pass)h(di\013eren)o (t)h(argumen)o(t)e(v)m(alues)i(for)f(the)75 504 y Fq(buft)o(yp)q(e)e Ft(and)e Fq(bufcount)i Ft(argumen)o(ts.)j(After)15 b(all)i(pro)q (cesses)f(of)g(the)g(group)f(ha)o(v)o(e)h(issued)h(their)f(resp)q(ec-) 75 561 y(tiv)o(e)i(calls,)g(eac)o(h)g(pro)q(cess)f(attempts)g(to)f (read,)i(from)f(the)g(\014le)i(asso)q(ciated)e(with)h Fq(fh)p Ft(,)g(a)f(total)g(n)o(um)o(b)q(er)75 617 y(of)i Fq(bufcount)i Ft(data)d(items)h(ha)o(ving)h Fq(buft)o(yp)q(e)h Ft(t)o(yp)q(e)e(in)o(to)g(the)g(user's)g(bu\013er)g Fq(bu\013)p Ft(.)32 b(F)l(or)18 b(eac)o(h)h(pro)q(cess,)75 674 y(the)f(lo)q(cation) g(in)g(the)g(\014le)h(at)e(whic)o(h)h(data)f(will)i(b)q(e)f(read)g(is)g (the)g(p)q(osition)g(at)f(whic)o(h)i(the)e(shared)h(\014le)75 730 y(p)q(oin)o(ter)d(w)o(ould)g(b)q(e)g(after)e(all)j(pro)q(cesses)e (whose)h(ranks)f(within)h(the)g(group)f(are)g(less)h(than)f(that)g(of)g (this)75 787 y(pro)q(cess)j(had)f(read)h(their)g(data.)23 b Fq(MPI)p 747 787 V 15 w(IREAD)p 894 787 V 17 w(SHARED)p 1087 787 V 18 w(ALL)16 b Ft(returns)g(immediately)i(and)f(do)q(es)f (not)75 843 y(w)o(ait)d(for)f(the)i(data)e(to)h(b)q(e)h(presen)o(t)f (in)h(the)f(user's)g(bu\013er.)19 b Fq(MPI)p 1172 843 V 16 w(IREAD)p 1320 843 V 17 w(SHARED)p 1513 843 V 17 w(ALL)13 b Ft(asso)q(ciates)g(an)75 900 y(individual)20 b(request)c(handle)i Fq(request)g Ft(to)d(the)i(I/O)g(request)f(for)g (eac)o(h)g(pro)q(cess.)24 b(The)16 b(request)h(handle)75 956 y(can)i(b)q(e)h(used)f(later)g(b)o(y)g(a)f(pro)q(cess)h(to)g(query) g(the)g(status)f(of)g(its)h(individual)j(read)d(request)g(or)f(w)o(ait) 75 1013 y(for)f(its)i(completion.)29 b(On)19 b(eac)o(h)f(pro)q(cess,)h Fq(MPI)p 916 1013 V 16 w(IREAD)p 1064 1013 V 16 w(SHARED)p 1256 1013 V 18 w(ALL)f Ft(completes)g(when)h(the)f(indi-)75 1069 y(vidual)f(request)e(has)g(completed)i(\(i.e.)j(a)15 b(pro)q(cess)g(do)q(es)h(not)f(ha)o(v)o(e)g(to)g(w)o(ait)f(for)h(all)h (other)f(requests)h(to)75 1125 y(complete\).)21 b(The)16 b(user)g(should)g(not)f(access)h(an)o(y)f(part)g(of)g(the)h(receiving)h (bu\013er)f(after)e(a)i(non)o(blo)q(c)o(king)75 1182 y(read)f(is)g(called,)i(un)o(til)f(the)f(read)g(completes.)20 b(The)15 b(shared)g(\014le)h(p)q(oin)o(ter)g(is)f(up)q(dated)h(b)o(y)f (the)g(amoun)o(ts)75 1238 y(of)j(data)f(requested)i(b)o(y)e(all)i(pro)q (cesses)g(of)f(the)g(group.)28 b(F)l(or)17 b(eac)o(h)h(pro)q(cess,)h (the)f(n)o(um)o(b)q(er)g(of)g Fq(buft)o(yp)q(e)75 1295 y Ft(elemen)o(ts)d(actually)h(read)f(b)o(y)f(that)g(pro)q(cess)h(can)g (b)q(e)h(extracted)e(from)g(the)h Fq(MPI)p 1467 1295 V 16 w(TEST)g Ft(or)f Fq(MPI)p 1749 1295 V 16 w(W)l(AIT)75 1351 y Ft(return)h(status,)f(using)i(the)f(routines)h Fq(MPI)p 818 1351 V 16 w(GET)p 922 1351 V 17 w(COUNT)g Ft(or)e Fq(MPI)p 1248 1351 V 16 w(GET)p 1352 1351 V 17 w(ELEMENTS)p Ft(.)75 1468 y Fq(Nonblo)q(cking)i(W)o(rite)f (\(collective\))75 1648 y(MPI)p 160 1648 V 16 w(IWRITE)p 332 1648 V 16 w(SHARED)p 524 1648 V 17 w(ALL\(fh,)g(bu\013,)g(buft)o (yp)q(e,)i(bufcount,)g(request\))117 1725 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\)) 117 1797 y(IN)155 b Fq(bu\013)514 b Ff(Initial)13 b(address)i(of)e(the) i(user's)f(bu\013er)h(\(in)o(teger\))117 1868 y(IN)155 b Fq(buft)o(yp)q(e)447 b Ff(User's)15 b(bu\013er)g(datat)o(yp)q(e)f (\(handle\))117 1940 y(IN)155 b Fq(bufcount)422 b Ff(Num)o(b)q(er)14 b(of)f(buft)o(yp)q(e)h(elemen)o(ts)g(\(nonnegativ)o(e)g(in)o(teger\)) 117 2011 y(OUT)108 b Fq(request)452 b Ff(W)m(rite)14 b(request)h(handle)f(\(handle\))75 2135 y Fp(int)47 b(MPI)p 269 2135 15 2 v 17 w(Iwrite)p 430 2135 V 16 w(shared)p 590 2135 V 17 w(all\(MPI)p 775 2135 V 16 w(File)23 b(fh,)h(void)f (*buff,)g(MPI)p 1364 2135 V 17 w(Datatype)g(buftype,)393 2192 y(int)h(bufcount,)e(MPI)p 802 2192 V 17 w(Request)h(*request\))75 2278 y(MPI)p 150 2278 V 17 w(IWRITE)p 311 2278 V 16 w(SHARED)p 471 2278 V 16 w(ALL\(FH,)g(BUFF,)h(BUFTYPE,)e(BUFCOUNT,)h(REQUEST,)g (IERROR\))170 2335 y()g(BUFF\(*\))170 2391 y(INTEGER)g(FH,)h (BUFTYPE,)f(BUFCOUNT,)f(REQUEST,)h(IERROR)166 2478 y Fq(MPI)p 251 2478 14 2 v 16 w(IWRITE)p 423 2478 V 16 w(SHARED)p 615 2478 V 17 w(ALL)10 b Ft(is)g(a)g(collectiv)o(e)i(v)o (ersion)f(of)e(the)i(non)o(blo)q(c)o(king)g Fq(MPI)p 1586 2478 V 16 w(IWRITE)p 1758 2478 V 16 w(SHARED)75 2534 y Ft(in)o(terface.)20 b(All)d(pro)q(cesses)f(in)g(the)f(comm)o (unicator)g(group)g(asso)q(ciated)h(with)g(the)f(\014le)h(handle)h Fq(fh)f Ft(m)o(ust)75 2591 y(call)j Fq(MPI)p 246 2591 V 16 w(IWRITE)p 418 2591 V 16 w(SHARED)p 610 2591 V 17 w(ALL)p Ft(.)e(Eac)o(h)h(pro)q(cess)g(ma)o(y)f(pass)h(di\013eren)o(t)g (argumen)o(t)f(v)m(alues)i(for)e(the)75 2647 y Fq(buft)o(yp)q(e)h Ft(and)e Fq(bufcount)i Ft(argumen)o(ts.)j(After)15 b(all)i(pro)q (cesses)f(of)g(the)g(group)f(ha)o(v)o(e)h(issued)h(their)f(resp)q(ec-) 75 2704 y(tiv)o(e)e(calls,)h(eac)o(h)f(pro)q(cess)h(attempts)e(to)g (write,)h(in)o(to)g(the)h(\014le)g(asso)q(ciated)f(with)g Fq(fh)p Ft(,)h(a)e(total)h(n)o(um)o(b)q(er)g(of)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 33 35 33 34 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(33)75 49 y Fq(bufcount)19 b Ft(data)d(items)g(ha)o(ving)h Fq(buft)o(yp)q(e)i Ft(t)o(yp)q(e)d(con)o(tained)h(in)h(the)e(user's)h (bu\013er)f Fq(bu\013)p Ft(.)25 b(F)l(or)15 b(eac)o(h)i(pro-)75 106 y(cess,)c(the)g(lo)q(cation)g(in)g(the)g(\014le)h(at)e(whic)o(h)h (data)f(will)i(b)q(e)g(written)e(is)h(the)g(p)q(osition)h(at)e(whic)o (h)h(the)g(shared)75 162 y(\014le)j(p)q(oin)o(ter)g(w)o(ould)g(b)q(e)g (after)e(all)j(pro)q(cesses)e(whose)h(ranks)f(within)h(the)g(group)f (are)g(less)h(than)f(that)f(of)75 219 y(this)k(pro)q(cess)g(had)g (written)g(their)g(data.)27 b Fq(MPI)p 905 219 14 2 v 16 w(IWRITE)p 1077 219 V 16 w(SHARED)p 1269 219 V 17 w(ALL)18 b Ft(returns)f(immediately)j(and)75 275 y(do)q(es)e(not)g(w)o (ait)f(for)h(the)g(data)f(to)g(b)q(e)i(presen)o(t)f(in)h(the)f(user's)f (bu\013er.)28 b Fq(MPI)p 1417 275 V 16 w(IWRITE)p 1589 275 V 16 w(SHARED)p 1781 275 V 17 w(ALL)75 332 y Ft(asso)q(ciates)18 b(an)h(individual)j(request)c(handle)i Fq(request)g Ft(to)e(the)h(I/O)g (request)f(for)g(eac)o(h)h(pro)q(cess.)30 b(The)75 388 y(request)19 b(handle)h(can)f(b)q(e)g(used)g(later)g(b)o(y)g(a)f(pro)q (cess)h(to)f(query)h(the)g(status)f(of)g(its)h(individual)j(write)75 444 y(request)15 b(or)g(w)o(ait)g(for)g(its)g(completion.)22 b(On)16 b(eac)o(h)f(pro)q(cess,)g Fq(MPI)p 1207 444 V 16 w(IWRITE)p 1379 444 V 16 w(SHARED)p 1571 444 V 17 w(ALL)g Ft(completes)75 501 y(when)i(the)g(individual)j(write)d (request)g(has)f(completed)i(\(i.e.)25 b(a)16 b(pro)q(cess)h(do)q(es)g (not)g(ha)o(v)o(e)f(to)g(w)o(ait)h(for)75 557 y(all)f(other)f(requests) g(to)g(complete\).)20 b(The)c(user)f(should)h(not)f(access)h(an)o(y)f (part)f(of)h(the)g(supplied)j(bu\013er)75 614 y(after)g(a)h(non)o(blo)q (c)o(king)i(write)e(is)g(called,)j(un)o(til)e(the)f(write)g(is)h (completed.)33 b(The)19 b(shared)g(\014le)h(p)q(oin)o(ter)75 670 y(is)d(up)q(dated)h(b)o(y)f(the)f(amoun)o(ts)g(of)h(data)f (requested)h(b)o(y)g(all)g(pro)q(cesses)h(of)e(the)h(group.)24 b(F)l(or)16 b(eac)o(h)h(pro-)75 727 y(cess,)22 b(the)f(n)o(um)o(b)q(er) g(of)f Fq(buft)o(yp)q(e)j Ft(elemen)o(ts)f(actually)f(written)g(b)o(y)g (that)f(pro)q(cess)h(can)g(b)q(e)g(extracted)75 783 y(from)c(the)h Fq(MPI)p 351 783 V 16 w(TEST)h Ft(or)e Fq(MPI)p 640 783 V 16 w(W)l(AIT)h Ft(return)g(status,)g(using)g(the)h(routines)f Fq(MPI)p 1542 783 V 16 w(GET)p 1646 783 V 17 w(COUNT)g Ft(or)75 840 y Fq(MPI)p 160 840 V 16 w(GET)p 264 840 V 17 w(ELEMENTS)p Ft(.)75 957 y Fq(Seek)e(\(collective\))75 1137 y(MPI)p 160 1137 V 16 w(SEEK)p 287 1137 V 16 w(SHARED\(fh,)g (o\013set,)f(whence\))117 1214 y Ff(IN)155 b Fq(fh)550 b Ff([SAME])14 b(V)m(alid)e(\014le)i(handle)g(\(handle\))117 1286 y(IN)155 b Fq(o\013set)484 b Ff([SAME])14 b(File)f(o\013set)i (\(o\013set\))117 1358 y(IN)155 b Fq(whence)450 b Ff([SAME])14 b(Up)q(date)g(mo)q(de)f(\(in)o(teger\))75 1482 y Fp(int)23 b(MPI)p 245 1482 15 2 v 17 w(Seek)p 358 1482 V 17 w(shared\(MPI)p 615 1482 V 16 w(File)g(fh,)g(MPI)p 917 1482 V 17 w(Offset)g(offset,)g (MPI)p 1364 1482 V 17 w(Whence)g(whence\))75 1569 y(MPI)p 150 1569 V 17 w(SEEK)p 263 1569 V 16 w(SHARED\(FH,)g(OFFSET,)g (WHENCE\))170 1625 y(INTEGER)g(FH,)h(WHENCE)170 1682 y(INTEGER*8)f(OFFSET)166 1768 y Fq(MPI)p 251 1768 14 2 v 16 w(SEEK)p 378 1768 V 16 w(SHARED)17 b Ft(up)q(dates)f(the)g (shared)g(\014le)h(p)q(oin)o(ter)f(according)g(to)f Fq(whence)p Ft(,)i(whic)o(h)g(could)75 1824 y(ha)o(v)o(e)e(the)g(follo)o(wing)h(p)q (ossible)h(v)m(alues:)143 1902 y Fh(\017)23 b Fg(MPI)p 266 1902 13 2 v 14 w(SEEK)p 382 1902 V 14 w(SET)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)f Fq(o\013set)143 1989 y Fh(\017)23 b Fg(MPI)p 266 1989 V 14 w(SEEK)p 382 1989 V 14 w(CUR)p Ft(:)14 b(the)h(p)q(oin)o(ter)h(is)g(set)f(to)f(the)h (curren)o(t)h(p)q(oin)o(ter)f(p)q(osition)i(plus)f Fq(o\013set)143 2076 y Fh(\017)23 b Fg(MPI)p 266 2076 V 14 w(SEEK)p 382 2076 V 14 w(END)p Ft(:)15 b(the)h(p)q(oin)o(ter)f(is)h(set)f(to)g(the)g (end)h(of)e(the)i(\014le)g(plus)g Fq(o\013set)166 2154 y Ft(All)d(the)f(pro)q(cesses)g(in)g(the)g(comm)o(unicator)g(group)f (asso)q(ciated)h(with)g(the)g(\014le)h(handle)g Fq(fh)f Ft(m)o(ust)f(call)75 2210 y Fq(MPI)p 160 2210 14 2 v 16 w(SEEK)p 287 2210 V 16 w(SHARED)18 b Ft(with)g(the)f(same)g Fq(o\013set)g Ft(and)g Fq(whence)p Ft(.)27 b(All)19 b(pro)q(cesses)e (in)h(the)f(comm)o(unicator)75 2266 y(group)e(are)g(sync)o(hronized)h (with)g(a)f(barrier)g(b)q(efore)h(the)f(shared)g(\014le)h(p)q(oin)o (ter)g(is)g(up)q(dated.)166 2323 y(The)c Fq(o\013set)g Ft(is)g(relativ)o(e)h(to)e(the)h(\014let)o(yp)q(e)h(\(not)e(coun)o (ting)i(holes\),)f(in)h(units)f(of)g Fq(et)o(yp)q(e)p Ft(.)20 b(The)12 b(constan)o(t)75 2379 y Fg(MPI)p 152 2379 13 2 v 14 w(OFFSET)p 321 2379 V 14 w(ZERO)j Ft(is)h(de\014ned)g (to)f(b)q(e)h(o\013set)e(zero.)75 2519 y Fm(10.6)60 b(Filet)n(yp)r(e)19 b(Constructo)n(rs)75 2704 y Fe(Discussion:)14 b Ff(Should)c(w)o(e)h(mo) o(v)o(e)e(these)k(datat)o(yp)q(e)e(constructors)h(to)f(the)h(\\Datat)o (yp)q(e)e(Impro)o(v)o(emen)o(ts")f(section?)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 34 36 34 35 bop 75 -100 a Ft(34)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 254 y Fl(10.6.1)49 b(Intro)q(duction)75 339 y Ft(Common)20 b(I/O)h(op)q(erations)f(\(e.g.,)h(rank-ordered)f(blo)q (c)o(ks,)i(structured)f(arra)o(y)e(accesses,)j(etc.\))35 b(are)75 396 y(easily)20 b(expressed)g(using)g(the)f(previously)h (de\014ned)h(read/write)e(op)q(erations)g(and)g(carefully)i(de\014ned) 75 452 y(\014let)o(yp)q(es.)i(In)17 b(order)f(to)f(simplify)j (generation)e(of)g(common)f(\014let)o(yp)q(es,)i(the)f(follo)o(wing)h Fq(MPI)e Ft(datat)o(yp)q(e)75 509 y(constructors)f(are)h(pro)o(vided.) 166 565 y(Although)c(it)f(is)h(p)q(ossible)h(to)d(implemen)o(t)j(these) e(t)o(yp)q(e)h(constructors)e(as)h(lo)q(cal)h(op)q(erations,)g(in)g (order)75 622 y(to)g(facilitate)h(e\016cien)o(t)g(implemen)o(tations)g (of)f(\014le)h(I/O)f(op)q(erations,)h(most)f(of)f(the)h(\014let)o(yp)q (e)i(constructors)75 678 y(ha)o(v)o(e)j(b)q(een)i(de\014ned)g(to)e(b)q (e)i Fk(c)n(ol)r(le)n(ctive)d Ft(op)q(erations.)24 b(\(Recall)19 b(that)d(a)g(collectiv)o(e)i(op)q(eration)f(do)q(es)g(not)75 735 y(imply)k(a)e(barrier)g(sync)o(hronization.\))33 b(The)20 b(only)g(exception)g(to)f(this)g(is)h(the)g(lo)q(cal)g (sub-arra)o(y)f(t)o(yp)q(e)75 791 y(constructor)c(whic)o(h)i(is)f(lo)q (cal)i(and)e(uses)g(no)g(pro)q(cess)g(rank)g(information.)22 b(Because)17 b(of)e(this,)h(the)g(sub-)75 848 y(arra)o(y)e(constructor) g(can)i(b)q(e)g(used)f(for)g(b)q(oth)g(buft)o(yp)q(es)h(and)f(\014let)o (yp)q(es.)166 904 y(The)j(set)g(of)f(\014let)o(yp)q(es)i(created)f(b)o (y)g(a)g(single)h(collectiv)o(e)h(\014let)o(yp)q(e)f(constructor)e (should)i(b)q(e)f(used)75 960 y(together)c(when)i(op)q(ening)h(a)d (\014le.)166 1017 y(Eac)o(h)20 b(new)h(\014let)o(yp)q(e)h(created)e Fq(newt)o(yp)q(e)j Ft(consists)d(of)h(zero)f(or)g(more)g(copies)i(of)e (the)g(base)h(t)o(yp)q(e)75 1073 y Fq(oldt)o(yp)q(e)p Ft(,)14 b(p)q(ossibly)h(separated)e(b)o(y)g(holes.)20 b(The)13 b(exten)o(t)g(of)f(the)h(new)h(\014let)o(yp)q(e)g(is)g(a)e (nonnegativ)o(e)i(in)o(teger)75 1130 y(m)o(ultiple)j(of)d(the)h(exten)o (t)g(of)f(the)h(base)g(t)o(yp)q(e.)20 b(All)c(\014let)o(yp)q(e)g (constructors)f(return)f(a)h(success)g(or)g(failure)75 1186 y(co)q(de.)75 1308 y Fl(10.6.2)49 b(Scatter)15 b(/)i(Gather)e(T)l (yp)q(e)i(Constructo)o(rs)75 1394 y Fq(MPI)p 160 1394 14 2 v 16 w(T)l(yp)q(e)p 268 1394 V 17 w(scatter)p 412 1394 V 18 w(gather)e(\(collective\))75 1574 y(MPI)p 160 1574 V 16 w(TYPE)p 293 1574 V 17 w(SCA)l(TTER)p 508 1574 V 17 w(GA)l(THER\(comm,)f(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 1651 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e (to)h(b)q(e)i(used)g(in)e(MPI)p 1655 1651 13 2 v 15 w(Op)q(en)i(\(han-) 905 1708 y(dle\))117 1783 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp)q(e)h(\(handle\))117 1858 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f (\(handle\))75 1983 y Fp(int)23 b(MPI)p 245 1983 15 2 v 17 w(Type)p 358 1983 V 17 w(scatter)p 543 1983 V 16 w(gather\(MPI)p 799 1983 V 16 w(Comm)g(comm,)g(MPI)p 1149 1983 V 17 w(Datatype)g(oldtype,)393 2039 y(MPI)p 468 2039 V 17 w(Datatype)g(*newtype\))75 2125 y(MPI)p 150 2125 V 17 w(TYPE)p 263 2125 V 16 w(SCATTER)p 447 2125 V 17 w(GATHER\(COMM,)f(OLDTYPE,)h(NEWTYPE,)f(IERROR\))170 2182 y(INTEGER)h(COMM,)h(OLDTYPE,)e(NEWTYPE,)h(IERROR)166 2268 y Ft(This)18 b(\014let)o(yp)q(e)h(allo)o(ws)f(eac)o(h)g(pro)q (cess)f(in)i(the)f(group)f(to)g(access)h(a)f(distinct)i Fq(blo)q(ck)f Ft(of)g(the)f(\014le)i(in)75 2325 y(rank)14 b(order.)20 b(The)15 b(blo)q(c)o(ks)g(are)f(iden)o(tical)j(in)f(size)f (and)g(datat)o(yp)q(e;)f(eac)o(h)h(item)g(is)g(of)f(datat)o(yp)q(e)g Fq(oldt)o(yp)q(e)p Ft(.)166 2381 y(T)l(o)g(ac)o(hiev)o(e)h(the)f (scatter)g(or)g(gather)f(op)q(eration,)i(the)f(\014les)i(op)q(ened)f (with)g(the)f(returned)h(\014let)o(yp)q(es)75 2438 y(should)f(b)q(e)f (accessed)g(with)g(collectiv)o(e)i(read)e(or)f(write)h(op)q(erations)f (with)h(iden)o(tical)i(o\013sets.)j(Generated)75 2494 y(\014let)o(yp)q(es)e(ma)o(y)f(not)g(b)q(e)g(iden)o(tical,)i(but)e (will)i(ha)o(v)o(e)e(the)g(same)g(exten)o(t.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 35 37 35 36 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(35)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(T)l(yp)q(e)p 268 49 V 17 w(scatterv)p 433 49 V 18 w(gatherv)15 b(\(collective\))75 230 y(MPI)p 160 230 V 16 w(TYPE)p 293 230 V 17 w(SCA)l(TTERV)p 538 230 V 17 w(GA)l(THERV\(comm,)g(count,)h(oldt)o(yp)q(e,)g(newt)o(yp) q(e\))117 307 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to)h(b)q(e)i(used)g(in)e(MPI)p 1655 307 13 2 v 15 w(Op)q(en)i(\(han-)905 363 y(dle\))117 438 y(IN)155 b Fq(count)482 b Ff(n)o(um)o(b)q(er)14 b(of)h(elemen)o(ts) f(of)g(oldt)o(yp)q(e)h(in)g(this)f(blo)q(c)o(k)h(\(nonneg-)905 495 y(ativ)o(e)f(in)o(teger\))117 570 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff(old)13 b(datat)o(yp)q(e)i(\(handle\))117 645 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 770 y Fp(int)23 b(MPI)p 245 770 15 2 v 17 w(Type)p 358 770 V 17 w(scatterv)p 567 770 V 16 w(gatherv\(MPI)p 847 770 V 16 w(Comm)g(comm,)g(int)h(count,)393 826 y(MPI)p 468 826 V 17 w(Datatype)f(oldtype,)f(MPI)p 986 826 V 17 w(Datatype)h(*newtype\))75 912 y(MPI)p 150 912 V 17 w(TYPE)p 263 912 V 16 w(SCATTERV)p 471 912 V 16 w(GATHERV\(COMM,)g (COUNT,)g(OLDTYPE,)f(NEWTYPE,)h(IERROR\))170 969 y(INTEGER)g(COMM,)h (COUNT,)f(OLDTYPE,)f(NEWTYPE,)h(IERROR)166 1055 y Ft(This)16 b(t)o(yp)q(e)f(allo)o(ws)g(eac)o(h)g(pro)q(cess)g(in)h(the)g(group)e (to)h(access)g(a)g(distinct)h Fk(blo)n(ck)e Ft(of)h(the)g(\014le)h(in)g (rank)75 1112 y(order)11 b(as)h(in)g(MPI)p 384 1112 14 2 v 16 w(T)o(yp)q(e)p 502 1112 V 17 w(scatter)p 654 1112 V 16 w(gather,)f(except)h(that)f(the)h(blo)q(c)o(k)g(sizes)g(and)g(t)o (yp)q(es)g(ma)o(y)f(b)q(e)h(di\013eren)o(t.)75 1168 y(Eac)o(h)j(blo)q (c)o(k)h(is)g(de\014ned)g(as)f Fq(count)i Ft(con)o(tiguous)e(copies)h (of)f(the)g(passed)g(datat)o(yp)q(e)g Fq(oldt)o(yp)q(e)p Ft(.)75 1290 y Fl(10.6.3)49 b(Lo)q(cal)18 b(Arra)o(y)e(Filet)o(yp)q(e)g (Constructo)o(r)f(\(indep)q(endent\))75 1470 y Fq(MPI)p 160 1470 V 16 w(TYPE)p 293 1470 V 17 w(SUBARRA)l(Y\(ndims,)i(sizes,)e (subsizes,)i(sta)o(rts,)f(o)o(rder,)e(oldt)o(yp)q(e,)i(newt)o(yp)q(e\)) 117 1547 y Ff(IN)155 b Fq(ndims)476 b Ff(n)o(um)o(b)q(er)13 b(of)h(arra)o(y)f(dimensions)g(\(nonnegativ)o(e)h(in)o(teger\))117 1623 y(IN)155 b Fq(sizes)502 b Ff(size)15 b(of)e(the)i(full)d(arra)o(y) i(in)f(eac)o(h)i(dimension)d(\(arra)o(y)i(of)f(non-)905 1679 y(negativ)o(e)h(in)o(tegers\))117 1754 y(IN)155 b Fq(subsizes)439 b Ff(size)14 b(of)f(the)h(sub-arra)o(y)g(in)f(eac)o (h)h(dimension)e(\(arra)o(y)h(of)g(non-)905 1811 y(negativ)o(e)h(in)o (tegers\))117 1886 y(IN)155 b Fq(sta)o(rts)484 b Ff(starting)18 b(co)q(ordinates)g(of)e(the)i(sub-arra)o(y)g(in)f(eac)o(h)g(dimen-)905 1942 y(sion)d(\(arra)o(y)g(of)f(nonnegativ)o(e)g(in)o(tegers\))117 2017 y(IN)155 b Fq(o)o(rder)490 b Ff(arra)o(y)14 b(storage)g(order)h (\015ag)e(\(in)o(teger\))117 2093 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff(arra)o(y)14 b(elemen)o(t)f(datat)o(yp)q(e)h(\(handle\))117 2168 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f (\(handle\))75 2292 y Fp(int)23 b(MPI)p 245 2292 15 2 v 17 w(Type)p 358 2292 V 17 w(subarray\(int)f(ndims,)h(int)h(*sizes,)e (int)i(*subsizes,)e(int)i(*starts,)393 2349 y(int)g(order,)f(MPI)p 731 2349 V 16 w(Datatype)g(oldtype,)g(MPI)p 1249 2349 V 17 w(Datatype)g(*newtype\))75 2435 y(MPI)p 150 2435 V 17 w(TYPE)p 263 2435 V 16 w(SUBARRAY\(NDIMS,)f(SIZES,)h(SUBSIZES,)g (STARTS,)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)393 2491 y(IERROR\))170 2548 y(INTEGER)g(NDIMS,)g(SIZES\(*\),)g(SUBSIZES\(*\),)f(STARTS\(*\),)h (ORDER,)g(OLDTYPE,)170 2604 y(NEWTYPE,)g(IERROR)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 36 38 36 37 bop 75 -100 a Ft(36)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)166 49 y Ft(The)23 b(lo)q(cal)h(sub-arra)o(y)e(\014let)o(yp)q(e)i (constructor)e(creates)h(an)g Fq(MPI)f Ft(datat)o(yp)q(e)g(describing)j (an)e(n-)75 106 y(dimensional)c(sub-arra)o(y)c(of)h(a)g(n-dimensional)j (arra)o(y)l(.)k(The)17 b(sub-arra)o(y)e(ma)o(y)h(b)q(e)h(situated)g(an) o(ywhere)75 162 y(within)f(the)f(full)i(arra)o(y)l(,)c(and)j(ma)o(y)e (b)q(e)i(of)e(an)o(y)h(size)h(up)f(to)g(the)g(size)h(of)e(the)h(larger) g(arra)o(y)f(as)h(long)g(as)g(it)75 219 y(is)h(con\014ned)i(within)f (this)f(arra)o(y)l(.)21 b(This)c(t)o(yp)q(e)f(constructor)f (facilitates)i(accessing)g(arra)o(ys)d(distributed)75 275 y(in)i(blo)q(c)o(ks)g(among)e(pro)q(cesses)i(to)f(a)f(single)j (\014le)f(that)f(con)o(tains)g(the)g(global)h(arra)o(y)l(.)166 332 y(This)h(t)o(yp)q(e)f(constructor)f(can)i(handle)g(arra)o(ys)e (with)i(an)f(arbitrary)g(n)o(um)o(b)q(er)g(of)g(dimensions)i(and)75 388 y(w)o(orks)11 b(for)h(b)q(oth)g(C)g(and)h(F)o(OR)l(TRAN)g(ordered)f (matrices)g(\(i.e.,)g(ro)o(w-ma)s(jor)e(or)i(column-ma)s(jor\).)18 b(Note)75 444 y(that)d(a)g(C)g(program)f(ma)o(y)g(use)i(F)o(OR)l(TRAN)g (order)f(and)h(a)f(F)o(OR)l(TRAN)h(program)e(ma)o(y)h(use)g(C)h(order,) 75 501 y(the)e(names)h(are)f(only)g(mean)o(t)g(to)g(sp)q(ecify)i(the)e (default)h(language)f(b)q(eha)o(vior.)20 b(T)l(o)14 b(sp)q(ecify)i (whic)o(h)f(order)75 557 y(the)g(arra)o(y)f(is)i(stored,)e(the)i Fq(o)o(rder)e Ft(argumen)o(t)g(is)i(set)f(to)g(one)g(of)g(the)g(follo)o (wing:)143 645 y Fh(\017)23 b Fg(MPI)p 266 645 13 2 v 14 w(C)p 307 645 V 14 w(ORDER)15 b Ft(-)g(The)g(ordering)h(used)g(b)o (y)f(C)g(arra)o(ys,)e(\(i.e.,)i(ro)o(w-ma)s(jor)e(order\))143 736 y Fh(\017)23 b Fg(MPI)p 266 736 V 14 w(F)o(ORTRAN)p 473 736 V 14 w(ORDER)13 b Ft(-)i(The)f(ordering)h(used)g(b)o(y)f(F)o (OR)l(TRAN)i(arra)o(ys,)c(\(i.e.,)i(column-ma)s(jor)189 793 y(order\))675 1572 y @beginspecial 195 @llx 269 @lly 416 @urx 523 @ury 1440 @rwi @setspecial %%BeginDocument: figures/io-array-file.eps /$F2psDict 32 dict def $F2psDict begin $F2psDict /mtrx matrix put end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 57.000000 681.000000 translate 0.900000 -0.900000 scale % Polyline newpath 163 177 moveto 215 177 lineto 215 380 lineto 163 380 lineto closepath 0.95 setgray gsave fill grestore 0.00 setgray % Polyline newpath 217 177 moveto 268 177 lineto 268 379 lineto 217 379 lineto closepath gsave fill grestore % Polyline newpath 270 177 moveto 321 177 lineto 321 379 lineto 270 379 lineto closepath 0.60 setgray gsave fill grestore 0.00 setgray 1.000 setlinewidth % Polyline newpath 323 178 moveto 374 178 lineto 374 377 lineto 323 377 lineto closepath 0.35 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 294 439 moveto 334 439 lineto 334 458 lineto 294 458 lineto closepath 0.35 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 294 400 moveto 335 400 lineto 335 418 lineto 294 418 lineto closepath 0.60 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 153 438 moveto 195 438 lineto 195 456 lineto 153 456 lineto closepath gsave fill grestore stroke % Polyline newpath 154 400 moveto 194 400 lineto 194 420 lineto 154 420 lineto closepath 0.95 setgray gsave fill grestore 0.00 setgray stroke % Polyline newpath 216 176 moveto 216 380 lineto stroke % Polyline newpath 268 176 moveto 268 380 lineto stroke % Polyline newpath 322 176 moveto 322 380 lineto stroke 4.000 setlinewidth % Polyline newpath 163 176 moveto 163 379 lineto 375 379 lineto 375 176 lineto closepath stroke /fn0.12 /Times-Roman findfont 13.333334 scalefont def fn0.12 setfont (Process 0) 202 414 moveto 1 -1 scale show 1 -1 scale (Process 2) 348 414 moveto 1 -1 scale show 1 -1 scale (Process 1) 204 452 moveto 1 -1 scale show 1 -1 scale (Process 3) 349 452 moveto 1 -1 scale show 1 -1 scale $F2psEnd %%EndDocument @endspecial 595 1670 a(Figure)h(10.5:)k(Example)d(arra)o(y)e(\014le)i (la)o(y)o(out)166 1772 y(F)l(or)11 b(example,)j(assume)e(w)o(e)f(are)h (writing)h(out)f(a)f(100x100)g(2D)g(arra)o(y)g(of)h(double)h(precision) h(\015oating)75 1828 y(p)q(oin)o(t)i(n)o(um)o(b)q(ers)g(that)e(is)i (distributed)h(among)e(4)g(pro)q(cesses)g(suc)o(h)h(that)f(eac)o(h)g (pro)q(cess)h(has)f(a)g(blo)q(c)o(k)h(of)75 1885 y(25)e(columns)i (\(e.g.,)d(pro)q(cess)i(0)g(has)f(columns)i(0-24,)e(pro)q(cess)h(1)g (has)f(columns)i(25-49,)d(etc.;)i(see)g(Figure)75 1941 y(10.5)f(\).)19 b(T)l(o)c(create)g(the)h(\014let)o(yp)q(es)g(for)e(eac) o(h)i(pro)q(cess)f(one)h(could)g(use)f(the)h(follo)o(wing)g(C)e (program:)147 2040 y Fp(double)23 b(subarray[100][25];)147 2097 y(MPI_Datatype)f(filetype;)147 2153 y(int)h(sizes[2],)g (subsizes[2],)f(starts[2];)147 2209 y(int)h(rank;)147 2322 y(MPI_Comm_rank\(MPI_COMM_)o(WORLD,)d(&rank\);)147 2379 y(sizes[0]=100;)i(sizes[1]=100;)g(subsizes[0]=100;)147 2435 y(subsizes[1]=25;)f(starts[0]=0;)i(starts[1]=rank*25;)147 2548 y(MPI_Type_subarray\(2,)e(sizes,)i(subsizes,)f(starts,)h (MPI_C_ORDER,)600 2605 y(MPI_DOUBLE,)f(&filetype\);)166 2704 y Ft(Or,)15 b(equiv)m(alen)o(tly)i(in)f(F)o(OR)l(TRAN:)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 37 39 37 38 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(37)242 49 y Fp(double)23 b(precision)g(subarray\(100,25\))242 106 y(integer)g(filetype,)g(rank,)g(ierror)242 162 y(integer)g (sizes\(2\),)g(subsizes\(2\),)f(starts\(2\))242 275 y(call)h (MPI_COMM_RANK\(MPI_COMM_WORLD,)d(rank,)j(ierror\))242 332 y(sizes\(1\)=100)242 388 y(sizes\(2\)=100)242 444 y(subsizes\(1\)=100)242 501 y(subsizes\(2\)=25)242 557 y(starts\(1\)=1)242 614 y(starts\(2\)=rank*25)f(+)h(1)242 727 y(call)g(MPI_TYPE_SUBARRAY\(2,)e(sizes,)j(subsizes,)e(starts,)194 783 y($)287 b(MPI_FORTRAN_ORDER,)21 b(MPI_DOUBLE_PRECISION,)194 840 y($)287 b(filetype,)22 b(ierror\))166 946 y Ft(The)c(generated)f (\014let)o(yp)q(e)i(will)g(then)f(describ)q(e)h(the)f(p)q(ortion)g(of)f (the)g(\014le)i(con)o(tained)f(within)h(the)75 1002 y(pro)q(cess')c (subarra)o(y)g(with)g(holes)h(for)f(the)g(space)h(tak)o(en)f(b)o(y)g (the)g(other)g(pro)q(cesses.)21 b(Figure)16 b(10.6)e(sho)o(ws)75 1059 y(the)h(\014let)o(yp)q(e)i(created)e(for)f(pro)q(cess)i(1.)675 1626 y @beginspecial 198 @llx 304 @lly 414 @urx 487 @ury 1440 @rwi @setspecial %%BeginDocument: figures/io-array-p1.eps /$F2psDict 32 dict def $F2psDict begin $F2psDict /mtrx matrix put end /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def /$F2psEnd {$F2psEnteredState restore end} def $F2psBegin 54.000000 681.000000 translate 0.900000 -0.900000 scale % Polyline newpath 230 217 moveto 269 217 lineto 269 380 lineto 230 380 lineto closepath 0.00 setgray gsave fill grestore 1.000 setlinewidth % Polyline newpath 160 400 moveto 199 400 lineto 199 420 lineto 160 420 lineto closepath gsave fill grestore stroke % Polyline newpath 229 216 moveto 229 381 lineto stroke % Polyline newpath 270 216 moveto 270 381 lineto stroke 4.000 setlinewidth % Polyline newpath 188 216 moveto 188 380 lineto 353 380 lineto 353 216 lineto closepath stroke 5.000 setlinewidth % Polyline newpath 319 398 moveto 358 398 lineto 358 418 lineto 319 418 lineto closepath stroke /fn0.12 /Times-Roman findfont 13.333334 scalefont def fn0.12 setfont (Holes) 370 413 moveto 1 -1 scale show 1 -1 scale (MPI_DOUBLE) 215 415 moveto 1 -1 scale show 1 -1 scale $F2psEnd %%EndDocument @endspecial 453 1724 a(Figure)f(10.6:)k(Example)d(subarra)o(y)e (\014let)o(yp)q(e)i(for)f(pro)q(cess)g(1)75 1895 y Fl(10.6.4)49 b(Distributed)16 b(Arra)o(y)g(Filet)o(yp)q(e)g(Constructo)o(rs)75 1981 y Ft(The)g(distributed)i(arra)o(y)d(\014let)o(yp)q(e)i (constructors)e(create,)h(for)f(eac)o(h)h(pro)q(cess)h(in)g(a)e(group,) h(a)g(\(p)q(ossibly)75 2038 y(di\013eren)o(t\))j(\014let)o(yp)q(e.)34 b(When)20 b(a)g(collectiv)o(e)h(I/O)f(op)q(eration)g(\(with)f(iden)o (tical)j(o\013sets\))c(is)i(p)q(erformed)75 2094 y(on)d(a)f(\014le)i (op)q(ened)g(with)f(the)g(returned)g(\014let)o(yp)q(e,)h(an)f(HPF-lik)o (e)h([4)o(])f(distribution)h(of)f(the)f(\014le)i(data)e(is)75 2150 y(realized.)26 b(Ho)o(w)o(ev)o(er,)16 b(unlik)o(e)i(in)f(HPF,)f (the)h(storage)e(order)i(ma)o(y)f(b)q(e)h(sp)q(eci\014ed)i(for)d(C)g (arra)o(ys)g(as)g(w)o(ell)75 2207 y(as)f(for)f(F)o(OR)l(TRAN)i(arra)o (ys.)75 2327 y Fq(MPI)p 160 2327 14 2 v 16 w(T)l(yp)q(e)p 268 2327 V 17 w(da)o(rra)o(y)e(\(collective\))75 2413 y Ft(Distribution)i(of)f(an)g(N-dimensional)j(arra)o(y)c(on)o(to)g(a)h (P-dimensional)i(grid)f(of)e(logical)j(pro)q(cessors:)-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 38 40 38 39 bop 75 -100 a Ft(38)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y\(comm,)14 b(ndims,)i(dsizes,)h(distribs,)g(da)o (rgs,)e(p)q(dims,)h(psizes,)h(o)o(rder,)d(oldt)o(yp)q(e,)j(new-)75 106 y(t)o(yp)q(e\))117 183 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to)h(b)q(e)i(used)g(in)e(MPI)p 1655 183 13 2 v 15 w(Op)q(en)i(\(han-)905 239 y(dle\))117 314 y(IN)155 b Fq(ndims)476 b Ff([SAME])13 b(n)o(um)o(b)q(er)g(of)g (arra)o(y)g(dimensions)f(\(nonnegativ)o(e)i(in-)905 370 y(teger\))117 445 y(IN)155 b Fq(dsizes)479 b Ff([SAME])19 b(size)i(of)e(distributed)i(arra)o(y)e(in)g(eac)o(h)i(dimension)905 501 y(\(arra)o(y)14 b(of)f(nonnegativ)o(e)h(in)o(teger\))117 576 y(IN)155 b Fq(distribs)453 b Ff([SAME])16 b(distribution)g(of)g (arra)o(y)g(in)g(eac)o(h)h(dimension,)e(e.g.)905 632 y(BLOCK)g(or)f(CYCLIC)g(\(arra)o(y)f(of)h(in)o(teger\))117 707 y(IN)155 b Fq(da)o(rgs)487 b Ff([SAME])14 b(distribution)f(argumen) o(t)g(of)g(dimension,)905 763 y(e.g.)25 b(BLOCK\(dargs\))18 b(or)e(CYCLIC\(dargs\))h(\(arra)o(y)f(of)g(in-)905 820 y(teger\))117 894 y(IN)155 b Fq(p)q(dims)475 b Ff([SAME])20 b(n)o(um)o(b)q(er)f(of)h(pro)q(cessor)i(grid)d(dimensions)g(\(non-)905 951 y(negativ)o(e)14 b(in)o(teger\))117 1025 y(IN)155 b Fq(psizes)479 b Ff([SAME])10 b(size)h(of)e(pro)q(cessor)j(grid)e(in)g (eac)o(h)h(dimension)d(\(arra)o(y)905 1082 y(of)13 b(nonnegativ)o(e)h (in)o(teger\))117 1156 y(IN)155 b Fq(o)o(rder)490 b Ff([SAME])14 b(arra)o(y)f(storage)i(order)f(\015ag)g(\(in)o(teger\))117 1231 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 1305 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 1430 y Fp(int)23 b(MPI)p 245 1430 15 2 v 17 w(Type)p 358 1430 V 17 w(darray\(MPI)p 615 1430 V 16 w(Comm)g(comm,)g(int)h(ndims,)f(int)g(*dsizes,)g(int)g (*distribs,)393 1486 y(int)h(*dargs,)f(int)g(pdims,)g(int)g(*psizes,)g (int)h(order,)393 1543 y(MPI)p 468 1543 V 17 w(Datatype)f(oldtype,)f (MPI)p 986 1543 V 17 w(Datatype)h(*newtype\))75 1629 y(MPI)p 150 1629 V 17 w(TYPE)p 263 1629 V 16 w(DARRAY\(COMM,)g(NDIMS,)g (DSIZES,)g(DISTRIBS,)f(DARGS,)h(PDIMS,)g(PSIZES,)g(ORDER,)393 1686 y(OLDTYPE,)g(NEWTYPE,)g(IERROR\))170 1742 y(INTEGER)g(COMM,)h (NDIMS,)f(DSIZES\(*\),)f(DISTRIBS\(*\),)g(DARGS\(*\),)h(PDIMS,)g (PSIZES\(*\))170 1799 y(INTEGER)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR)166 1885 y Fq(MPI)p 251 1885 14 2 v 16 w(TYPE)p 384 1885 V 17 w(D)o(ARRA)l(Y)f Ft(generates)f(the)h(\014let)o(yp)q(es)g (corresp)q(onding)g(to)f(the)g(distribution)i(of)e(an)75 1941 y Fq(ndims)p Ft(-dimensional)15 b(arra)o(y)c(of)h Fq(oldt)o(yp)q(e)h Ft(elemen)o(ts)g(on)o(to)e(a)h Fq(p)q(dims)p Ft(-dimensional)j(grid)e(of)f(logical)h(pro)q(ces-)75 1998 y(sors.)18 b(F)l(or)10 b(a)g(call)h(to)f Fq(MPI)p 507 1998 V 16 w(TYPE)p 640 1998 V 17 w(D)o(ARRA)l(Y)i Ft(to)d(b)q(e)j(correct,)e(the)h(equation)1365 1966 y Fd(Q)1404 1976 y Fc(pdims)p Fb(\000)p Fa(1)1404 2011 y Fc(i)p Fa(=0)1554 1998 y Fi(psiz)r(es)p Ft([)p Fi(i)p Ft(])h(=)h Fi(siz)r(e)p Ft(,)75 2054 y(where)i Fi(siz)r(e)h Ft(denotes)f(the)h(size)g(of)e(comm)o(unicator)h Fq(comm)p Ft(,)e(m)o(ust)i(b)q(e)h(satis\014ed.)166 2111 y(Eac)o(h)f(dimension)i (of)e(the)g(arra)o(y)f(is)i(distributed)g(in)g(one)g(of)e(three)i(w)o (a)o(ys:)143 2202 y Fh(\017)23 b Fg(MPI)p 266 2202 13 2 v 14 w(DISTRIBUTE)p 522 2202 V 13 w(BLOCK)15 b Ft(-)g(Blo)q(c)o(k)h (distribution)143 2294 y Fh(\017)23 b Fg(MPI)p 266 2294 V 14 w(DISTRIBUTE)p 522 2294 V 13 w(CYCLIC)14 b Ft(-)h(Cyclic)i (distribution)143 2387 y Fh(\017)23 b Fg(MPI)p 266 2387 V 14 w(DISTRIBUTE)p 522 2387 V 13 w(NONE)16 b Ft(-)f(Dimension)h(not)f (distributed.)166 2478 y(T)l(o)e(sp)q(ecify)i(a)f(default)g (distribution)i(argumen)o(t,)c(the)i(constan)o(t)f Fg(MPI)p 1365 2478 V 14 w(DISTRIBUTE)p 1621 2478 V 14 w(DFL)m(T)p 1736 2478 V 15 w(ARG)f Ft(is)75 2534 y(used.)20 b(The)c(distribution)h (argumen)o(t)d(for)h(a)g(dimension)i(that)d(is)i(not)f(distributed)h (is)g(ignored.)166 2591 y(F)l(or)j(example,)i(ARRA)l(Y\(CYCLIC\(15\)\)) e(corresp)q(onds)h(to)e Fg(MPI)p 1316 2591 V 15 w(DISTRIBUTE)p 1573 2591 V 13 w(CYCLIC)g Ft(with)h(a)75 2647 y(distribution)13 b(argumen)o(t)d(of)h(15,)g(and)h(ARRA)l(Y\(BLOCK\))h(corresp)q(onds)f (to)e Fg(MPI)p 1471 2647 V 15 w(DISTRIBUTE)p 1728 2647 V 13 w(BLOCK)75 2704 y Ft(with)16 b(a)e(distribution)j(argumen)o(t)e (of)f Fg(MPI)p 796 2704 V 15 w(DISTRIBUTE)p 1053 2704 V 13 w(DFL)m(T)p 1167 2704 V 15 w(ARG)p Ft(.)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 39 41 39 40 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(39)166 49 y(The)17 b Fq(o)o(rder)f Ft(argumen)o(t)g(is)h(used)g (as)f(in)i Fq(MPI)p 928 49 14 2 v 16 w(TYPE)p 1061 49 V 17 w(SUBARRA)l(Y)h Ft(to)d(sp)q(ecify)i(the)f(storage)e(order)75 106 y(for)h(m)o(ultidimensional)k(arra)o(ys.)k(Therefore,)17 b(arra)o(ys)f(describ)q(ed)j(b)o(y)e(this)h(t)o(yp)q(e)f(constructor)f (ma)o(y)g(b)q(e)75 162 y(stored)11 b(in)h(ro)o(w-ma)s(jor)d(or)i (column-ma)s(jor)g(order.)18 b(V)l(alid)13 b(v)m(alues)g(for)d Fq(o)o(rder)h Ft(are)g Fg(MPI)p 1517 162 13 2 v 14 w(F)o(ORTRAN)p 1724 162 V 14 w(ORDER)75 219 y Ft(or)k Fg(MPI)p 208 219 V 14 w(C)p 249 219 V 14 w(ORDER)p Ft(.)166 275 y(F)l(or)f(example,)i (let)g(us)f(generate)g(the)g(\014let)o(yp)q(es)i(corresp)q(onding)f(to) e(the)i(HPF)e(distribution:)218 369 y Fp()23 b(FILEARRAY\(100,)f(200,)h(300\))75 425 y(!HPF$)g(PROCESSORS)g (PROCESSES\(2,)f(3\))75 482 y(!HPF$)h(DISTRIBUTE)g (FILEARRAY\(CYCLIC\(10\),)e(*,)i(BLOCK\))g(ONTO)h(PROCESSES)75 576 y Ft(This)13 b(can)g(b)q(e)g(ac)o(hiev)o(ed)g(b)o(y)f(the)h(follo)o (wing)g(F)o(OR)l(TRAN)g(co)q(de,)g(assuming)g(there)g(will)h(b)q(e)f (six)g(pro)q(cesses)75 632 y(attac)o(hed)i(to)f(the)h(run:)170 726 y Fp(ndims)24 b(=)f(3)170 782 y(dsizes\(1\))g(=)h(100)170 839 y(distribs\(1\))f(=)h(MPI_DISTRIBUTE_CYCLIC)170 895 y(dargs\(1\))f(=)h(10)170 952 y(dsizes\(2\))f(=)h(200)170 1008 y(distribs\(2\))f(=)h(MPI_DISTRIBUTE_NONE)170 1065 y(dargs\(2\))f(=)h(0)170 1121 y(dsizes\(3\))f(=)h(300)170 1178 y(distribs\(3\))f(=)h(MPI_DISTRIBUTE_BLOCK)170 1234 y(dargs\(3\))f(=)h(MPI_DISTRIBUTE_DFLT_ARG)170 1290 y(pdims)g(=)f(2)170 1347 y(psizes\(1\))g(=)h(2)170 1403 y(psizes\(2\))f(=)h(3)170 1460 y(call)g(MPI_TYPE_DARRAY\(MPI_COMM)o(_WORLD,)c(ndims,)j(dsizes,)g (distribs,)g(dargs,)147 1516 y($)119 b(pdims,)23 b(psizes,)g (MPI_FORTRAN_ORDER,)e(oldtype,)i(newtype,)g(ierr\))75 1636 y Fq(MPI)p 160 1636 14 2 v 16 w(T)l(yp)q(e)p 268 1636 V 17 w(da)o(rra)o(y)p 403 1636 V 15 w(2d)16 b(\(collective\))75 1722 y Ft(Distribution)g(of)f(a)g Fk(two)p Ft(-dimensional)j(arra)o(y:) -32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 40 42 40 41 bop 75 -100 a Ft(40)1342 b Fn(CHAPTER)15 b(10.)35 b(I/O)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y)p 486 49 V 17 w(2D\(comm,)13 b(dsize1,)i (distrib1,)h(da)o(rg1,)e(psize1,)i(dsize2,)f(distrib2,)h(da)o(rg2,)e (psize2,)75 106 y(o)o(rder,)g(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 183 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to) h(b)q(e)i(used)g(in)e(MPI)p 1655 183 13 2 v 15 w(Op)q(en)i(\(han-)905 239 y(dle\))117 315 y(IN)155 b Fq(dsize1)473 b Ff([SAME])21 b(size)g(of)f(distributed)i(arra)o(y)f(in)f(\014rst)i(dimension)905 371 y(\(nonnegativ)o(e)14 b(in)o(teger\))117 446 y(IN)155 b Fq(distrib1)447 b Ff([SAME])14 b(distribution)f(for)h(\014rst)g (dimension)e(\(in)o(teger\))117 521 y(IN)155 b Fq(da)o(rg1)481 b Ff([SAME])12 b(distribution)g(argumen)o(t)f(for)h(\014rst)h (dimension)d(\(in-)905 578 y(teger\))117 653 y(IN)155 b Fq(psize1)473 b Ff([SAME])12 b(size)i(of)e(pro)q(cessor)j(grid)d(in)g (\014rst)i(dimension)d(\(in)o(te-)905 709 y(ger\))117 784 y(IN)155 b Fq(dsize2)473 b Ff([SAME])14 b(size)g(of)g(distributed)g (arra)o(y)g(in)g(second)h(dimension)905 841 y(\(nonnegativ)o(e)f(in)o (teger\))117 916 y(IN)155 b Fq(distrib2)447 b Ff([SAME])14 b(distribution)f(for)h(second)g(dimension)f(\(in)o(teger\))117 991 y(IN)155 b Fq(da)o(rg2)481 b Ff([SAME])18 b(distribution)f(argumen) o(t)g(for)h(second)h(dimension)905 1048 y(\(in)o(teger\))117 1123 y(IN)155 b Fq(psize2)473 b Ff([SAME])11 b(size)g(of)g(pro)q (cessor)h(grid)f(in)f(second)i(dimension)d(\(in-)905 1179 y(teger\))117 1254 y(IN)155 b Fq(o)o(rder)490 b Ff([SAME])14 b(arra)o(y)f(storage)i(order)f(\015ag)g(\(in)o(teger\))117 1329 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 1405 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 1529 y Fp(int)23 b(MPI)p 245 1529 15 2 v 17 w(Type)p 358 1529 V 17 w(darray)p 519 1529 V 16 w(2d\(MPI)p 679 1529 V 17 w(Comm)g(comm,)g(int)h(dsize1,) e(int)i(distrib1,)f(int)g(darg1,)393 1585 y(int)h(psize1,)f(int)g (dsize2,)g(int)g(distrib2,)g(int)g(darg2,)g(int)h(psize2,)393 1642 y(int)g(order,)f(MPI)p 731 1642 V 16 w(Datatype)g(oldtype,)g(MPI)p 1249 1642 V 17 w(Datatype)g(*newtype\))75 1728 y(MPI)p 150 1728 V 17 w(TYPE)p 263 1728 V 16 w(DARRAY)p 423 1728 V 17 w(2D\(COMM,)g(DSIZE1,)f(DISTRIB1,)h(DARG1,)g(PSIZE1,)g(DSIZE2,)g (DISTRIB2,)393 1785 y(DARG2,)g(PSIZE2,)g(ORDER,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR\))170 1841 y(INTEGER)g(COMM,)h(DSIZE1,)e(DISTRIB1,)h(DARG1,)g (PSIZE1,)g(DSIZE2,)g(DISTRIB2,)g(DARG2)170 1898 y(INTEGER)g(PSIZE2,)g (ORDER,)g(OLDTYPE,)g(NEWTYPE,)g(IERROR)166 1984 y Ft(This)12 b(call)h(is)g(a)e(shorthand)h(for)f(the)h(t)o(w)o(o-dimensional)h (distributed)g(arra)o(y)e(case)h(and)g(is)g(equiv)m(alen)o(t)75 2041 y(to:)170 2134 y Fp(dsize[0]=dsize1;)22 b(distrib[0]=distrib1;)f (dargs[0]=darg1;)h(psize[0])h(=)h(psize1;)170 2191 y(dsize[1]=dsize2;)e (distrib[1]=distrib2;)f(dargs[1]=darg2;)h(psize[1])h(=)h(psize2;)170 2247 y(MPI_Type_darray\(comm,)d(2,)j(dsize,)f(distrib,)g(dargs,)g(2,)g (psize,)576 2304 y(order,)g(oldtype,)g(&newtype\);)75 2424 y Fq(MPI)p 160 2424 14 2 v 16 w(T)l(yp)q(e)p 268 2424 V 17 w(da)o(rra)o(y)p 403 2424 V 15 w(blo)q(ck)16 b(\(collective\))75 2510 y Ft(BLOCK)g(distribution)h(of)e(a)g (one-dimensional)i(arra)o(y:)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 41 43 41 42 bop 75 -100 a Fn(10.6.)34 b(FILETYPE)15 b(CONSTR)o(UCTORS)992 b Ft(41)75 49 y Fq(MPI)p 160 49 14 2 v 16 w(TYPE)p 293 49 V 17 w(D)o(ARRA)l(Y)p 486 49 V 17 w(BLOCK\(comm,)13 b(dsize,)i(da)o(rg,)g(oldt)o(yp)q(e,)g(newt)o(yp)q(e\))117 126 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e(to) h(b)q(e)i(used)g(in)e(MPI)p 1655 126 13 2 v 15 w(Op)q(en)i(\(han-)905 183 y(dle\))117 258 y(IN)155 b Fq(dsize)496 b Ff([SAME])10 b(size)h(of)f(distributed)h(arra)o(y)f(\(nonnegativ)o(e)g(in)o(teger\)) 117 333 y(IN)155 b Fq(da)o(rg)504 b Ff([SAME])21 b(distribution)f (argumen)o(t,)i(e.g.)39 b(BLOCK\(darg\))905 390 y(\(in)o(teger\))117 465 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp)q (e)h(\(handle\))117 540 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 664 y Fp(int)23 b(MPI)p 245 664 15 2 v 17 w(Type)p 358 664 V 17 w(darray)p 519 664 V 16 w(block\(MPI)p 751 664 V 16 w(Comm)g(comm,)h(int)f(dsize,) g(int)h(darg,)393 721 y(MPI)p 468 721 V 17 w(Datatype)f(oldtype,)f(MPI) p 986 721 V 17 w(Datatype)h(*newtype\))75 807 y(MPI)p 150 807 V 17 w(TYPE)p 263 807 V 16 w(DARRAY)p 423 807 V 17 w(BLOCK\(COMM,)f(DSIZE,)h(DARG,)g(OLDTYPE,)g(NEWTYPE,)g(IERROR\)) 170 864 y(INTEGER)g(COMM,)h(DSIZE,)f(DARG,)g(OLDTYPE,)g(NEWTYPE,)f (IERROR)166 950 y Fq(MPI)p 251 950 14 2 v 16 w(TYPE)p 384 950 V 17 w(D)o(ARRA)l(Y)p 577 950 V 17 w(BLOCK)14 b Ft(generates)f(the)h(\014let)o(yp)q(es)g(corresp)q(onding)h(to)e(the) g(BLOCK)i(dis-)75 1007 y(tribution)h(of)f(a)g(one-dimensional)i Fq(dsize)f Ft(elemen)o(t)g(arra)o(y)e(of)h Fq(oldt)o(yp)q(e)p Ft(.)166 1063 y(This)h(call)g(is)g(a)f(shorthand)g(for:)170 1157 y Fp(MPI_Comm_size\(comm,)22 b(&size\);)170 1213 y(distrib)h(=)h(MPI_DISTRIBUTE_BLOCK;)170 1270 y (MPI_Type_darray\(comm,)d(1,)j(dsize,)f(distrib,)g(darg,)g(1,)h(size,) 576 1326 y(order,)f(oldtype,)g(&newtype\);)166 1420 y Ft(Note)10 b(that)f(order)h(is)h(not)e(sp)q(eci\014ed)k(for)c(this)i (constructor,)f(nor)g(for)f Fq(MPI)p 1395 1420 V 16 w(TYPE)p 1528 1420 V 17 w(D)o(ARRA)l(Y)p 1721 1420 V 18 w(CYCLIC)p Ft(,)75 1477 y(b)q(ecause)16 b(it)g(is)f(not)g(meaningful)i(for)d(1D)h (arra)o(ys.)75 1597 y Fq(MPI)p 160 1597 V 16 w(T)l(yp)q(e)p 268 1597 V 17 w(da)o(rra)o(y)p 403 1597 V 15 w(cyclic)h(\(collective\)) 75 1682 y Ft(CYCLIC)g(distribution)g(of)f(a)g(one-dimensional)j(arra)o (y:)75 1833 y Fq(MPI)p 160 1833 V 16 w(TYPE)p 293 1833 V 17 w(D)o(ARRA)l(Y)p 486 1833 V 17 w(CYCLIC\(comm,)13 b(dsize,)i(da)o(rg,)f(oldt)o(yp)q(e,)i(newt)o(yp)q(e\))117 1911 y Ff(IN)155 b Fq(comm)470 b Ff([SAME])12 b(comm)o(uni)o(cator)e (to)h(b)q(e)i(used)g(in)e(MPI)p 1655 1911 13 2 v 15 w(Op)q(en)i(\(han-) 905 1967 y(dle\))117 2042 y(IN)155 b Fq(dsize)496 b Ff([SAME])10 b(size)h(of)f(distributed)h(arra)o(y)f(\(nonnegativ)o(e)g(in)o(teger\)) 117 2117 y(IN)155 b Fq(da)o(rg)504 b Ff([SAME])19 b(distribution)f (argumen)o(t,)h(e.g.)34 b(CYCLIC\(darg\))905 2174 y(\(in)o(teger\))117 2249 y(IN)155 b Fq(oldt)o(yp)q(e)450 b Ff([SAME])14 b(old)f(datat)o(yp) q(e)h(\(handle\))117 2324 y(OUT)108 b Fq(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))75 2448 y Fp(int)23 b(MPI)p 245 2448 15 2 v 17 w(Type)p 358 2448 V 17 w(darray)p 519 2448 V 16 w(cyclic\(MPI)p 775 2448 V 16 w(Comm)g(comm,)h(int)f (dsize,)g(int)g(darg,)393 2505 y(MPI)p 468 2505 V 17 w(Datatype)g(oldtype,)f(MPI)p 986 2505 V 17 w(Datatype)h(*newtype\))75 2591 y(MPI)p 150 2591 V 17 w(TYPE)p 263 2591 V 16 w(DARRAY)p 423 2591 V 17 w(CYCLIC\(COMM,)f(DSIZE,)h(DARG,)g(OLDTYPE,)g(NEWTYPE,)g (IERROR\))170 2648 y(INTEGER)g(COMM,)h(DSIZE,)f(DARG,)g(OLDTYPE,)g (NEWTYPE,)f(IERROR\))-32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 42 44 42 43 bop 75 -100 a Ft(42)1347 b Fn(CHAPTER)15 b(10.)30 b(I/O)166 49 y Fq(MPI)p 251 49 14 2 v 16 w(TYPE)p 384 49 V 17 w(D)o(ARRA)l(Y)p 577 49 V 17 w(CYCLIC)11 b Ft(generates)g(the)g (\014let)o(yp)q(es)i(corresp)q(onding)f(to)e(the)h(CYCLIC)h(dis-)75 106 y(tribution)k(of)f(a)g(one-dimensional)i Fq(dsize)f Ft(elemen)o(t)g(arra)o(y)e(of)h Fq(oldt)o(yp)q(e)p Ft(.)166 162 y(This)h(call)g(is)g(a)f(shorthand)g(for:)170 256 y Fp(MPI_Comm_size\(comm,)22 b(&size\);)170 312 y(distrib)h(=)h (MPI_DISTRIBUTE_CYCLIC;)170 369 y(MPI_Type_darray\(comm,)d(1,)j(dsize,) f(distrib,)g(darg,)g(1,)h(size,)576 425 y(order,)f(oldtype,)g (&newtype\);)75 569 y Fm(10.7)60 b(Prop)r(osal:)26 b(Simpli\014ed)19 b(Op)r(en)75 726 y Fp(Several)k(scientific)f(users)i(have)f(complained) f(of)i(the)f(steep)h(learning)75 783 y(curve)f(of)h(this)f(interface)g (and)g(have)g(suggested)g(we)h(provide)f(simplified)75 839 y("wrapper")g(functions.)46 b(Rather)23 b(than)g(adding)g(extra)h (functions)e(to)i(the)75 896 y(specification,)e(the)h(following)g (proposal)g(was)g(made)h(which)f(simplifies)75 952 y(I/O)g(for)h(the)f (user)h(that)f(does)g(not)h(need)f(to)h(use)f(the)h(more)f (sophisticated)75 1009 y(features,)g(and)g(also)g(allows)g(the)h (filetype)f(to)g(be)h(changed)f(without)75 1065 y(re-opening)f(the)i (file)f(or)h(using)f(file)g(control.)75 1178 y(Break)g(MPI_OPEN)g(in)h (to)f(two)h(calls:)46 b(MPI_OPEN)23 b(and)h(MPI_LAYOUT.)75 1291 y(MPI_OPEN\(comm,)e(filename,)h(amode,)g(hints,)g(fh\))75 1404 y(Opens)g("filename")g(with:)75 1517 y(disp)g(=)h(0)75 1573 y(etype)f(=)h(MPI_BYTE)75 1630 y(filetype)f(=)g(MPI_BYTE)75 1743 y(This)g(essentially)g(mimic's)g(a)g(UNIX)h(open)f(call.)75 1856 y(MPI_LAYOUT\(fh,)f(disp,)h(etype,)g(filetype,)g(hints\))75 1968 y(Changes)g(the)g(process's)g(view)g(of)h(the)f(file.)47 b(It)24 b(is)g(assumed)e(to)75 2025 y(be)i(an)f(expensive)g(operation,) f(and)i(is)f(collective.)47 b(It)23 b(would)75 2081 y(be)h(allowed)e (to)i(be)g(done)f(at)h(any)f(time)g(on)h(an)g(open)f(file)g(handle.)75 2194 y(An)h(MPI_GET_LAYOUT)e(routine)g(could)i(also)f(be)h(provided)e (\(see)i(the)f(proposed)75 2251 y(replacemenet)f(for)i (MPI_FILE_CONTROL)d(above\).)75 2364 y(By)j(adding)f(this)g(change,)g (a)h(user)f(can)g(open,)h(read,)f(and)g(write)g(a)h(file)75 2420 y(without)f(having)g(to)h(know)f(anything)g(about)g(etypes,)g (filetypes,)f(displacement,)75 2477 y(etc.)47 b(It)24 b(also)f(means)g(that)h(you)f(can)g(open)h(a)g(file,)f(write)g(the)g (header)g(using)75 2533 y(etype=filetype=MPI_BYTE,)e(and)i(then,)g (after)g(the)h(header)f(has)g(been)h(written,)75 2589 y(set)f(the)h(layout)f(to)h(a)f(more)h(complicated)e(filetype)h(with)g (a)h(displacement.)75 2646 y(You)f(need)h(not)f(close)g(and)h(re-open)f (the)g(file)h(or)f(use)h(a)f(file)h(control)f(command.)75 2702 y(This)g(makes)g(it)h(easier)f(for)h(people)f(that)g(plan)g(to)h (write)f(simple)g(headers)1967 46 y Fj(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 43 45 43 44 bop 75 -100 a Fn(10.7.)34 b(PR)o(OPOSAL:)16 b(SIMPLIFIED)g(OPEN) 929 b Ft(43)75 49 y Fp(on)24 b(to)f(the)h(start)f(of)g(their)h(files.) -32 46 y Fj(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 44 46 44 45 bop 75 381 a Fr(Bibliograph)m(y)98 604 y Ft([1])21 b(F)l(ern)11 b(E.)f(Basso)o(w.)i(Installing,)h(managing,)e(and)g(using) g(the)g(IBM)g(AIX)g(P)o(arallel)h(I/O)f(File)g(System.)168 660 y(IBM)16 b(Do)q(cumen)o(t)f(Num)o(b)q(er)h(SH34-6065-00,)d(F)l (ebruary)i(1995.)k(IBM)c(Kingston,)g(NY.)98 754 y([2])21 b(P)o(eter)11 b(F.)g(Corb)q(ett,)g(Dror)f(G.)g(F)l(eitelson,)j (Jean-Pierre)f(Prost,)f(George)g(S.)g(Almasi,)h(Sandra)f(John-)168 811 y(son)20 b(Ba)o(ylor,)g(An)o(thon)o(y)g(S.)f(Bolmarcic)o(h,)j(Y)l (arsun)e(Hsu,)h(Julian)g(Satran,)f(Marc)f(Snir,)j(Rob)q(ert)168 867 y(Colao,)15 b(Brian)g(Herr,)g(Joseph)h(Ka)o(v)m(aky)l(,)e(Thomas)h (R.)g(Morgan,)e(and)j(An)o(thon)o(y)e(Zlotek.)20 b(P)o(arallel)168 923 y(\014le)15 b(systems)e(for)f(the)i(IBM)g(SP)f(computers.)k Fk(IBM)d(Systems)f(Journal)p Ft(,)h(34\(2\):222{2)o(48,)c(Jan)o(uary) 168 980 y(1995.)98 1074 y([3])21 b(Juan)15 b(Miguel)g(del)f(Rosario,)g (Ra)s(jesh)g(Borda)o(w)o(ek)m(ar,)f(and)h(Alok)g(Choudhary)l(.)k(Impro) o(v)o(ed)13 b(parallel)168 1130 y(I/O)i(via)f(a)f(t)o(w)o(o-phase)g (run-time)i(access)e(strategy)l(.)k(In)d Fk(IPPS)g('93)h(Workshop)h(on) f(Input/Output)168 1187 y(in)h(Par)n(al)r(lel)f(Computer)i(Systems)p Ft(,)c(pages)i(56{70,)e(1993.)18 b(Also)d(published)j(in)e(Computer)e (Arc)o(hi-)168 1243 y(tecture)i(News)f(21\(5\),)e(Decem)o(b)q(er)j (1993,)d(pages)i(31{38.)98 1337 y([4])21 b(Charles)e(H.)f(Ko)q(elb)q (el,)i(Da)o(vid)e(B.)g(Lo)o(v)o(eman,)g(Rob)q(ert)h(S.)f(Sc)o(hreib)q (er,)i(Guy)d(L.)h(Steele)i(Jr.,)e(and)168 1393 y(Mary)d(E.)f(Zosel.)21 b Fk(The)15 b(High)i(Performanc)n(e)f(F)m(ortr)n(an)f(Handb)n(o)n(ok)p Ft(.)k(MIT)c(Press,)g(1993.)98 1487 y([5])21 b(Da)o(vid)15 b(Kotz.)20 b(Disk-directed)c(I/O)f(for)g(MIMD)f(m)o(ultipro)q(cessors.) 20 b(In)15 b Fk(Pr)n(o)n(c)n(e)n(e)n(dings)f(of)i(the)g(1994)168 1544 y(Symp)n(osium)e(on)g(Op)n(er)n(ating)f(Systems)f(Design)h(and)h (Implementation)p Ft(,)e(pages)g(61{74,)f(No)o(v)o(em)o(b)q(er)168 1600 y(1994.)19 b(Up)q(dated)d(as)f(Dartmouth)f(TR)h(PCS-TR94-226)g(on) g(No)o(v)o(em)o(b)q(er)f(8,)h(1994.)98 1694 y([6])21 b(Da)o(vid)16 b(Kotz)g(and)f(Nils)i(Nieu)o(w)o(ejaar.)k(File-system)c (w)o(orkload)e(on)g(a)g(scien)o(ti\014c)j(m)o(ultipro)q(cessor.)168 1750 y Fk(IEEE)e(Par)n(al)r(lel)f(and)h(Distribute)n(d)h(T)m(e)n (chnolo)n(gy)p Ft(,)c(pages)i(51{60,)e(Spring)j(1995.)98 1844 y([7])21 b(Bill)15 b(Nitzb)q(erg.)h(P)o(erformance)c(of)g(the)h (iPSC/860)f(Concurren)o(t)g(File)i(System.)h(T)l(ec)o(hnical)g(Rep)q (ort)168 1901 y(RND-92-020,)f(NAS)i(Systems)f(Division,)h(NASA)g(Ames,) f(Decem)o(b)q(er)g(1992.)98 1994 y([8])21 b(William)f(J.)f(Nitzb)q (erg.)29 b Fk(Col)r(le)n(ctive)18 b(Par)n(al)r(lel)g(I/O)p Ft(.)29 b(PhD)18 b(thesis,)h(Departmen)o(t)f(of)f(Computer)168 2051 y(and)f(Information)f(Science,)i(Univ)o(ersit)o(y)e(of)g(Oregon,)g (Decem)o(b)q(er)h(1995.)98 2145 y([9])21 b(K.)h(E.)g(Seamons,)h(Y.)f (Chen,)i(P)l(.)e(Jones,)h(J.)f(Jozwiak,)i(and)e(M.)g(Winslett.)41 b(Serv)o(er-directed)168 2201 y(collectiv)o(e)17 b(I/O)f(in)g(P)o (anda.)j(In)d Fk(Pr)n(o)n(c)n(e)n(e)n(dings)e(of)j(Sup)n(er)n(c)n (omputing)f('95)p Ft(,)f(Decem)o(b)q(er)g(1995.)75 2295 y([10])21 b(Ra)s(jeev)f(Thakur)f(and)g(Alok)h(Choudhary)l(.)32 b(An)20 b(extended)g(t)o(w)o(o-phase)f(metho)q(d)g(for)g(accessing)168 2351 y(sections)h(of)f(out-of-core)f(arra)o(ys.)30 b(T)l(ec)o(hnical)21 b(Rep)q(ort)f(CA)o(CR-103,)f(Scalable)h(I/O)g(Initiativ)o(e,)168 2408 y(Cen)o(ter)g(for)g(Adv)m(anced)i(Computing)e(Researc)o(h,)i (Caltec)o(h,)f(June)h(1995.)33 b(Revised)22 b(No)o(v)o(em)o(b)q(er)168 2464 y(1995.)75 2558 y([11])f(The)f(MPI-IO)f(Committee.)31 b(MPI-IO:)19 b(a)g(parallel)h(\014le)g(I/O)f(in)o(terface)h(for)e(MPI,) g(April)i(1996.)168 2615 y(V)l(ersion)c(0.5.)952 2828 y(44)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Fri May 17 17:16:32 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA01162; Fri, 17 May 1996 17:16:31 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA20121 for mpi-core-out; Fri, 17 May 1996 16:19:36 -0500 Received: from piper.cs.colorado.edu (root@piper.cs.colorado.edu [128.138.236.20]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA20074 for ; Fri, 17 May 1996 16:17:55 -0500 Received: from [204.228.69.101] ([204.228.69.101]) by piper.cs.colorado.edu (8.7.5/8.7.3) with ESMTP id PAA22815; Fri, 17 May 1996 15:17:46 -0600 (MDT) X-Sender: mcbr@piper.cs.colorado.edu Message-Id: In-Reply-To: <199605171936.MAA03143@win24.nas.nasa.gov> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Fri, 17 May 1996 15:20:14 -0600 To: Bill Nitzberg , mpi-core@mcs.anl.gov, mpi-io@nas.nasa.gov From: "Oliver A. McBryan" Subject: Re: MPI-2 I/O Chapter Proposal Cc: sio-hlapi@cacr.caltech.edu, sio-os-api@cs.princeton.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I've just received a single email message of 1.2MB on my powerbook while traveling. This is ridiculuous. Could you please put items like this on a Web page or in an FTP archive - and mail the address instead of the content. Oliver McBryan; Oliver.McBryan@cs.colorado.edu Phone: 303-6650544 and 303-4923898; Cell: 303-8097804; Fax: 303-4922844 Dept of Computer Science, Univ. of Colorado, Boulder, CO 80309-0430. WWW: http://www.cs.colorado.edu/~mcbryan/Home.html From mpi-core-human@mcs.anl.gov Sun May 19 23:46:05 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA19805; Sun, 19 May 1996 23:46:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA11280 for mpi-core-out; Sun, 19 May 1996 22:45:34 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA11275 for ; Sun, 19 May 1996 22:45:28 -0500 Message-Id: <199605200345.WAA11275@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: June MPI Forum meeting Date: Sun, 19 May 1996 22:45:27 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, The next meeting of the MPI Forum will take place on Wed.- Fri., June 5-7. It is time to make your reservations. Register at the hotel by calling (847) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. We will follow the usual schedule, beginning at 1:30 on Wednesday and ending at noon on Friday. We will charge the usual registration fee of $75, which can be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. If you have any problems or questions, let me know, either by email (lusk@mcs.anl.gov) or phone (708-252-7852). Here is the schedule of future MPI Forum meetings. Jun 5-7 Jul 17-19 Sep 4-6 Oct 9-11 Supercomputing '96 is Nov 17-22. Regards, Rusty From mpi-core-human@mcs.anl.gov Tue May 21 21:03:44 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA05715; Tue, 21 May 1996 21:03:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA21319 for mpi-core-out; Tue, 21 May 1996 20:00:12 -0500 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA13288 for ; Mon, 20 May 1996 04:31:00 -0500 Received: from ry70.rz.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Mon, 20 May 1996 11:09:28 +0200 Received: by ry70.rz.uni-karlsruhe.de (1.37.109.16/16.2) id AA119143357; Mon, 20 May 1996 11:09:17 +0200 Subject: MPI v1.1 errors To: mpi-core@mcs.anl.gov Date: Mon, 20 May 1996 11:09:17 +0200 (CES) From: hennecke@rz.uni-karlsruhe.de X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 7bit Content-Length: 1222 Message-ID: <"nz11.rz.un.647:20.05.96.09.10.17"@rz.uni-karlsruhe.de> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hello! Some errors in the MPI Spec v1.1: 195:48 "HANDLER," -> "ERRHANDLER," 197: 2 shouldn't errhandler have intent INOUT ? 204: 1 "level" -> "LEVEL" 223:34 "INTEGER REQUEST," -> "INTEGER" 223:38 "INTEGER REQUEST," -> "INTEGER" 228:19 "INTRACOMM" -> "NEWINTRACOMM" 228:20 "INTRACOMM" -> "NEWINTRACOMM" 229:46 "HANDLER," -> "ERRHANDLER," 230:33 "level" -> "LEVEL" Some minor Typos: 9:33 "begining" -> "beginning" 17: 8 "string {\bf message}" -> "variable {\bf message}" (see 16:46) 19: 2 "longlong" -> "long long" 21:23 "intercommunicators" -> "inter-communicators" Best regards, Michael Hennecke ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) BITNET: RZ48@DKAUNI2.BITNET Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Wed May 22 01:13:15 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id BAA13100; Wed, 22 May 1996 01:13:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id AAA24057 for mpi-core-out; Wed, 22 May 1996 00:16:01 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id AAA24039; Wed, 22 May 1996 00:15:26 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id WAA02668; Tue, 21 May 1996 22:12:53 -0700 Date: Tue, 21 May 1996 22:12:53 -0700 From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199605220512.WAA02668@win233.nas.nasa.gov> To: mpi-1sided@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: we should wait for 1sided implementations Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At the last meeting, I was the sole dissenter in a number of the formal votes on the one-sided communication chapter. A colleague has since pointed out the essence of my unease, which is that we are voting to standardize an interface that has not been implemented (even partially) and which is not closely related to common practice. I know of no implmentation of the proposed MPI interface. My understanding is that at this stage in MPI-1, there was an implementation that tracked the current spec and provided substantial feedback. The major existing model is the Cray shmem interface, which is only superficially similar. While the shmem interface has put and get routines, the core of the proposal is really the semantics and the synchronization mechanisms. The MPI mechanisms are quite different from the Cray mechanisms. The Global Arrays package has been mentioned but the MPI interface does not have equivalent functionality - it would be a building block for GA. The vast majority of "existing practice", shared memory and synchronization mechanisms on SMPs, is completely different from the MPI proposal. The lack of implementations would be ok if we had common practice, or vice-versa. But it seems we are making a big mistake to standardize out of thin air. So rather than rushing to take a second and final vote on the 1sided chapter, I suggest that we await feedback from initial implementations. The fact that we have a first formal vote should give implementors the confidence that they are building on a fairly solid surface. At the same time, if implementors find problems, or if performance is unacceptable, or if users don't like the functionality, we are not locked into a bad standard. I suggest the delay even though I don't have any particular objections to the proposal at this point. Further discussion of the parts which have one formal vote is not necessary until there is feedback. I think it is wise in any case to wait. If the proposal is solid, we lose nothing and possibly gain earlier implementations. If the proposal is broken, we have a much better chance to do something about it. I do not see how this could delay progress, except superficially. Someone will say that we can always override our earlier decision if there is a problem. I submit that we will not have any implementations by Supercomputing if there is no incentive to work on a prototype, so this opportunity will not arise. Moreover, procedurally it may be very difficult to make changes for all but the most serious problems. I would like to hold other chapters involving major change (including the Dynamic Process chapter) to the same standard. Comments? Bill From mpi-core-human@mcs.anl.gov Wed May 22 03:18:36 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA21956; Wed, 22 May 1996 03:18:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA24827 for mpi-core-out; Wed, 22 May 1996 02:20:43 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA24822; Wed, 22 May 1996 02:20:36 -0500 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) id AAA03813; Wed, 22 May 1996 00:20:19 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) id AAA11450; Wed, 22 May 1996 00:19:07 -0700 From: "Eric Salo" Message-Id: <9605220019.ZM11448@mrjones.engr.sgi.com> Date: Wed, 22 May 1996 00:19:07 -0700 In-Reply-To: wcs@nas.nasa.gov (William C. Saphir) "we should wait for 1sided implementations" (May 21, 10:12pm) References: <199605220512.WAA02668@win233.nas.nasa.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-1sided@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: we should wait for 1sided implementations Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > At the last meeting, I was the sole dissenter in a number of the > formal votes on the one-sided communication chapter. A colleague has > since pointed out the essence of my unease, which is that we are > voting to standardize an interface that has not been implemented (even > partially) and which is not closely related to common practice. You were only the sole dissenter because I had to catch a plane after the first vote! I've been saying for months that the 1sided subcommittee has been putting the cart before the horse, so to speak. (I know, this isn't quite what you're saying, but it's close enough for me to jump in!) > I know of no implmentation of the proposed MPI interface. My > understanding is that at this stage in MPI-1, there was an > implementation that tracked the current spec and provided substantial > feedback. It's worse than that. At the moment, I am unaware of any implementors other than maybe IBM who are willing to commit to implementing it *ever*. On the other hand, I can think of at least three that are seriously interested in *not* implementing it. > The major existing model is the Cray shmem interface, which is only > superficially similar. While the shmem interface has put and get > routines, the core of the proposal is really the semantics and the > synchronization mechanisms. The MPI mechanisms are quite different > from the Cray mechanisms. The Global Arrays package has been > mentioned but the MPI interface does not have equivalent functionality > - it would be a building block for GA. The vast majority of "existing > practice", shared memory and synchronization mechanisms on SMPs, is > completely different from the MPI proposal. Yup. > The lack of implementations would be ok if we had common practice, > or vice-versa. But it seems we are making a big mistake to standardize > out of thin air. So rather than rushing to take a second and final > vote on the 1sided chapter, I suggest that we await feedback from > initial implementations. The fact that we have a first formal vote > should give implementors the confidence that they are building on a > fairly solid surface. At the same time, if implementors find problems, > or if performance is unacceptable, or if users don't like the > functionality, we are not locked into a bad standard. We know right now that the performance is gonna be awful. Unfortunately, this does not appear to be a priority; the subcommittee has made it quite clear on numerous occasions that "flexibility" comes first. Which is strange, considering the historical emphasis that MPI has placed on high performance. The result is a whole lot of apathy from those of us with performance concerns. But let's hear from the implementors! Who is willing to commit to a prototype in the next 6 months? Anyone? > I suggest the delay even though I don't have any particular objections > to the proposal at this point. Further discussion of the parts which > have one formal vote is not necessary until there is feedback. Well, we certainly do need to follow proper procedure. Much as I hate it, there *has* been a formal vote and so that does commit us to some degree to those parts. On the other hand, we could always decide to "unvote" them... I really think that, more than any other chapter in MPI-1 *or* MPI-2, the 1sided chapter has suffered tremendously from the committee nature of the MPI Forum. I would suggest that if we're ever going to get it right, we need to look at the chapter as a whole instead of voting in our favorite pieces one at a time. I'll even propose a starting point: What features (read: restrictions) must a get/put interface have that, given reasonably intelligent machines, will allow implementations to transfer data at hardware speeds? Neither latency nor bandwidth may be sacrificed. Then, given a model which satisfies that, how do we make it portable to the widest range of systems? Discuss. > I think it is wise in any case to wait. If the proposal is solid, we > lose nothing and possibly gain earlier implementations. If the > proposal is broken, we have a much better chance to do something about > it. I do not see how this could delay progress, except superficially. > > Someone will say that we can always override our earlier decision > if there is a problem. I submit that we will not have any > implementations by Supercomputing if there is no incentive to work on > a prototype, so this opportunity will not arise. Moreover, > procedurally it may be very difficult to make changes for all but the > most serious problems. I am a bit more pessimistic. IMHO, waiting is highly unlikely to generate any substantial feedback because prototype implementations will probably not appear any time soon (see above). And this chapter has proven to be extremely difficult to change in any meaningful way; in the past year, no less than three different counter-proposals (Paul's, Lloyd's, and my own) have been suggested in the names of performance and/or simplicity, only to be swallowed up. It seems quite apparent that for whatever reason, the get/put proposal is not going to change fundamentally from what it looks like now. Unless something very significant happens to change that, we will eventually find ourselves in the awkward position of either voting it in as it stands (and being left with a mess) or killing it entirely (and having nothing to show for almost two years of effort). Anyone wanna guess which outcome is the more likely? > I would like to hold other chapters involving major change (including > the Dynamic Process chapter) to the same standard. Is there anything in MPI-2 that *doesn't* involve major change? Maybe the misc chapter, but for the others I think that this is definitely the correct thing to do. Taking the specific example of the Dynamic Process chapter, I believe that both LAM and MPICH already have prototype spawn() implementations in place, so it looks like that particular chapter is well on its way. And I believe that the MSU people have already done experiments with some of the collective extensions, so that area also appears to be solidifying nicely. SGI will follow both of these examples as soon as we are able, hopefully well before Supercomputing. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Wed May 22 11:39:52 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA08518; Wed, 22 May 1996 11:39:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA01254 for mpi-core-out; Wed, 22 May 1996 10:42:19 -0500 Received: from Phoenix.ERC.MsState.Edu (root@Phoenix.ERC.MsState.Edu [192.208.145.65]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA01156; Wed, 22 May 1996 10:39:23 -0500 Received: from lucy.cs.msstate.edu (lucy.cs.msstate.edu [192.208.157.69]); by Phoenix.ERC.MsState.Edu using SMTP (8.7.4/7.0m-FWP-MsState); id KAA10073; Wed, 22 May 1996 10:39:06 -0500 (CDT) Received: by lucy.cs.msstate.edu with Microsoft Mail id <01BB47CA.F61444F0@lucy.cs.msstate.edu>; Wed, 22 May 1996 10:39:32 -0500 Message-ID: <01BB47CA.F61444F0@lucy.cs.msstate.edu> From: Shane Hebert To: "mpi-1sided@mcs.anl.gov" , "mpi-core@mcs.anl.gov" Subject: RE: we should wait for 1sided implementations Date: Wed, 22 May 1996 10:39:30 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk But let's hear from the implementors! Who is willing to commit to a prototype in the next 6 months? Anyone? ============================== I have been thinking about how to implement this on Windows NT. I cannot commit to prototype it but I believe I can see how it can be done. I agree that it would be an expensive procedure requiring a collective operation. On a machine using only a shared memory device, it is not too bad but on a machine using both shared memory and TCP, it gets trickier. I am sure there are others out there who have thought as much as I have on what is required for your implementation but here are my ideas for the shared memory version and the problem (with a solution) with TCP. [stream-of-consciousness alert:] Shared memory for Windows NT: Basically, when the RMA_Malloc is called, map a region of shared memory and pass that OS_handle around to all the other processes. Each process gets this message and maps that region based on that handle. Now, the time consuming part is when a put or a get to that shared memory is called, the OS_handle has to be looked up (possibly the time consuming part) and associated with the memory address then the put/get can proceed simply as a memcpy(). Perhaps an MPID_SMHANDLE can be defined and the result of the RMA_Malloc returns one of those to the calling process and all communications then proceed through that data structure (should speed it up some since no lookup) but this definitely calls for RMA_MALLOC to be a collective operation that returns an MPID_SMHANDLE to each calling process. One process actually does the allocation and the rest simply map it, of course. From then on, it is easy to read and write to that section of memory (even in a mutually exclusive way if you include a mutex in the MPID_SMHANDLE). Having RMA_Malloc be a collective operation is expensive of course. Using a collective RMA_Malloc that every process must participate in would simplify the TCP case greatly. Without it, there must be a way to generate unique identifiers (numbers, strings, whatever) across the platforms. Actually thinking about it now, I think the collective type of RMA_Malloc is the *only* way to do these operations. The MPID_SMHANDLE can also contain the information about on which machine the RMA_memory actually exists. For machines without shared memory, a new packet type should be defined that contains the location and length to read/write the data. If the data must be broken across multiple packets, each packet should contain at least the base address and an offset to read/write from/to. The packet receiving code must become smarter of course to perform these put/get functions behind the scenes. It may not be pretty, easy, or fast but it can be done. On machines that actually have some sort of shared memory architecture, it can be fairly easy and have some speed. On machines without shared memory, it becomes slower. There was talk of having an "agent" or something to handle these put/get requests on machines without shared memory. This would be a prime target for (close your eyes Eric and Raja :-) threads. Sorry if this disjoint and confusing, I'm in a hurry and didn't even proofread beyond the spell-checker... (besides, I did give a warning...) Shane From mpi-core-human@mcs.anl.gov Wed May 22 13:16:36 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA10727; Wed, 22 May 1996 13:16:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA03191 for mpi-core-out; Wed, 22 May 1996 12:19:20 -0500 Received: from cs.sandia.gov (cs.sandia.gov [132.175.13.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA02789; Wed, 22 May 1996 11:57:30 -0500 Received: from cheetah.cs.sandia.gov.noname by cs.sandia.gov with smtp (Smail3.1.28.1 #5) id m0uMHDW-000XRuC; Wed, 22 May 96 10:56 MDT Received: by cheetah.cs.sandia.gov.noname (4.1/SMI-4.1) id AA10958; Wed, 22 May 96 10:56:57 MDT From: bright@cs.sandia.gov (Ron Brightwell) Message-Id: <9605221656.AA10958@cheetah.cs.sandia.gov.noname> Subject: Re: we should wait for 1sided implementations To: wcs@nas.nasa.gov (William C. Saphir) Date: Wed, 22 May 96 10:56:57 MDT Cc: mpi-1sided@mcs.anl.gov, mpi-core@mcs.anl.gov In-Reply-To: <199605220512.WAA02668@win233.nas.nasa.gov>; from "William C. Saphir" at May 21, 96 10:12 pm X-Mailer: ELM [version 2.3 PL11] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > > At the last meeting, I was the sole dissenter in a number of the > formal votes on the one-sided communication chapter. A colleague has > since pointed out the essence of my unease, which is that we are > voting to standardize an interface that has not been implemented (even > partially) and which is not closely related to common practice. > > I know of no implmentation of the proposed MPI interface. My > understanding is that at this stage in MPI-1, there was an > implementation that tracked the current spec and provided substantial > feedback. > The initialization and get/put functions have existed in our MPI implementation on top of Puma portals for the Paragon and TeraFLOPS machine for over two months. We have several ASCI users who are in the process of porting their shmem codes from the T3D to our MPI-2 one sided interface, which is essentially what was proposed in the March 4th edition of the document. Our implementation currently ignores the target counter argument to the put functions (which will require a handler or something at the target), so I voted against having this functionality at the last meeting. I have documentation that is dated December '95 for an MPI-2 one sided communications implementation that EPCC has done for the Cray T3D also. -Ron From mpi-core-human@mcs.anl.gov Wed May 22 14:47:09 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA12074; Wed, 22 May 1996 14:42:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA05060 for mpi-core-out; Wed, 22 May 1996 13:44:12 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA05055; Wed, 22 May 1996 13:43:56 -0500 Received: from porthos.lsc.nd.edu (porthos.lsc.nd.edu [129.74.25.91]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id NAA02174; Wed, 22 May 1996 13:43:54 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.7.5/8.7.3) with SMTP id NAA02642; Wed, 22 May 1996 13:43:53 -0500 (EST) Date: Wed, 22 May 1996 13:43:53 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Chapter 7 Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the text for Chapter 7, the C++ bindings. We do not yet have any text for the Fortran 90 bindings, so that section has been left empty. A lot of the text has been changed to clarify points raised at the last forum meeting as well as bring out new issues that were also discussed at the last forum. Please note that the inter-language operability issue has still not been settled; there is a nasty name clash problem (see section 7.1.6). Also please note that the "const" constants issue hasn't been fully resolved yet -- we'd like to hear from some more implementers first. Particularly, the int vs. enum issue has not been resolved yet. Here's my $.02 on the issue: It shouldn't matter whether (for example) the return codes are const ints or const enums, because both are constant in the eyes of the compiler, both can be assigned initial values, and both provide better type safety than #define. Sorry this took so long -- we got caught up in finals and graduation. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness. {+} "I came to ND for 4 years and ended up staying for a decade." ------ Postscript follows this line ------ %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: chapter-7.dvi %%Pages: 18 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o chapter-7.ps chapter-7 %DVIPSParameters: dpi=300, compressed, comments removed %DVIPSSource: TeX output 1996.05.22:1343 %%BeginProcSet: texc.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]/id ch-image N /rw ch-width 7 add 8 idiv string N /rc 0 N /gp 0 N /cp 0 N{rc 0 ne{rc 1 sub /rc X rw}{G}ifelse}imagemask restore}B /G{{id gp get /gp gp 1 add N dup 18 mod S 18 idiv pl S get exec}loop}B /adv{cp add /cp X}B /chg{rw cp id gp 4 index getinterval putinterval dup gp add /gp X adv}B /nd{/cp 0 N rw exit}B /lsh{rw cp 2 copy get dup 0 eq{pop 1}{ dup 255 eq{pop 254}{dup dup add 255 and S 1 and or}ifelse}ifelse put 1 adv}B /rsh{rw cp 2 copy get dup 0 eq{pop 128}{dup 255 eq{pop 127}{dup 2 idiv S 128 and or}ifelse}ifelse put 1 adv}B /clr{rw cp 2 index string putinterval adv}B /set{rw cp fillstr 0 4 index getinterval putinterval adv}B /fillstr 18 string 0 1 17{2 copy 255 put pop}for N /pl[{adv 1 chg} {adv 1 chg nd}{1 add chg}{1 add chg nd}{adv lsh}{adv lsh nd}{adv rsh}{ adv rsh nd}{1 add adv}{/rc X nd}{1 add set}{1 add clr}{adv 2 chg}{adv 2 chg nd}{pop nd}]dup{bind pop}forall N /D{/cc X dup type /stringtype ne{] }if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (chapter-7.dvi) @start /Fa 1 59 df<127012F8A3127005057C840D>58 D E /Fb 25 122 df<13E01201EA0380EA0700120E5AA25AA25AA35AA91270A37EA27EA27E7EEA03 80EA01E012000B217A9C16>40 D<12C07E12707E7E7EA27EA2EA0380A3EA01C0A9EA0380 A3EA0700A2120EA25A5A5A5A5A0A217B9C16>I<1238127C127EA2123E120E121E121C12 7812F01260070B798416>44 D<127012F8A312700505788416>46 D<1238127CA312381200A812381278127CA2123C121CA21238127012E012400618799116 >59 D<3801F180EA07FBEA0FFFEA1F0FEA3C07EA38031270A200F0C7FC5AA77E38700380 A21238383C0700EA1F0FEA0FFE6C5AEA01F011197E9816>67 D73 D<38FC07E0EAFE0FA2383A0B80EA3B1BA513BBEA39B3A413F3EA38 E3A21303A538FE0FE0A313197F9816>77 D80 D97 D99 D<133F5B7F1307A4EA03C7EA 0FF748B4FCEA3C1F487EEA700712E0A6EA700FA2EA3C1F381FFFE0380FE7F03807C7E014 197F9816>II<131FEB7F8013FFEA01E7EBC30013C0 A2EA7FFFB5FCA2EA01C0ACEA3FFE487E6C5A11197F9816>I<127E12FE127E120EA4133C 13FEEA0FFFEB87801303120EAA387FC7F038FFE7F8387FC7F01519809816>104 DI108 D<38F9C38038FFEFC0EBFFE0EA3C78A2EA3870 AA38FE7CF8A2EB3C781512809116>III115 D<12035AA4EA7FFFB5FCA20007C7FCA75BEB0380A3EB 8700EA03FE6C5A6C5A11177F9616>I<387F1FC000FF13E0007F13C0381C0700EA1E0FEA 0E0EA36C5AA4EA03B8A3EA01F0A26C5A13127F9116>118 D<387F1FC0133F131F380F1C 00EA073CEA03B813F012016C5A12017FEA03B8EA073C131CEA0E0E387F1FC038FF3FE038 7F1FC013127F9116>120 D<387F1FC038FF9FE0387F1FC0381C0700120E130EA212075B A2EA039CA21398EA01B8A2EA00F0A35BA3485A1279127BEA7F806CC7FC123C131B7F9116 >I E /Fc 45 125 df<137E3801C180EA0301380703C0120EEB018090C7FCA5B512C0EA 0E01B0387F87F8151D809C17>12 D34 D<13401380EA0100120212065AA25AA25AA2127012 60A312E0AC1260A312701230A27EA27EA27E12027EEA008013400A2A7D9E10>40 D<7E12407E7E12187EA27EA27EA213801201A313C0AC1380A312031300A21206A25AA25A 12105A5A5A0A2A7E9E10>I<1306ADB612E0A2D80006C7FCAD1B1C7E9720>43 D<126012F0A212701210A41220A212401280040C7C830C>II<12 6012F0A2126004047C830C>I<130113031306A3130CA31318A31330A31360A213C0A3EA 0180A3EA0300A31206A25AA35AA35AA35AA35AA210297E9E15>I<1306A3130FA3EB1780 A3EB23C0A3EB41E0A3EB80F0A200017FEB0078EBFFF83803007C0002133CA20006133E00 04131EA2000C131F121E39FF80FFF01C1D7F9C1F>65 D<90381F8080EBE0613801801938 070007000E13035A14015A00781300A2127000F01400A8007014801278A212386CEB0100 A26C13026C5B380180083800E030EB1FC0191E7E9C1E>67 D69 DI 73 D77 D80 D<007FB512C038700F010060130000401440A200C014201280A300001400 B1497E3803FFFC1B1C7F9B1E>84 D92 D97 D<12FC121CAA137CEA1D86EA1E03381C018014C0130014E0A614C013011480381E0300EA 1906EA10F8131D7F9C17>II<133F1307AAEA03E7EA0C17EA180F487E12 70126012E0A61260127012306C5AEA0C373807C7E0131D7E9C17>II<13F8EA018CEA071E1206EA0E0C1300A6EAFFE0EA0E00B0EA7FE00F1D809C 0D>II<12FC121CAA137C1387EA1D03001E1380121CAD38FF9FF0141D 7F9C17>I<1218123CA21218C7FCA712FC121CB0EAFF80091D7F9C0C>I<13C0EA01E0A2EA 00C01300A7EA0FE01200B3A21260EAF0C012F1EA6180EA3E000B25839C0D>I<12FC121C AAEB3FC0EB0F00130C13085B5B5B13E0121DEA1E70EA1C781338133C131C7F130F148038 FF9FE0131D7F9C16>I<12FC121CB3A9EAFF80091D7F9C0C>I<39FC7E07E0391C83883839 1D019018001EEBE01C001C13C0AD3AFF8FF8FF8021127F9124>IIIIIII<1204A4120CA212 1C123CEAFFE0EA1C00A91310A5120CEA0E20EA03C00C1A7F9910>I<38FC1F80EA1C03AD 1307120CEA0E1B3803E3F014127F9117>I<38FF07E0383C0380381C0100A2EA0E02A26C 5AA3EA0388A213D8EA01D0A2EA00E0A3134013127F9116>I<39FF3FCFE0393C0F038038 1C07011500130B000E1382A21311000713C4A213203803A0E8A2EBC06800011370A2EB80 30000013201B127F911E>I<387F8FF0380F03801400EA0702EA0384EA01C813D8EA00F0 1370137813F8139CEA010E1202EA060738040380381E07C038FF0FF81512809116>I<38 FF07E0383C0380381C0100A2EA0E02A26C5AA3EA0388A213D8EA01D0A2EA00E0A31340A2 5BA212F000F1C7FC12F31266123C131A7F9116>I I124 D E /Fd 10 58 df<121FEA3180EA60C0EA4040EAC060A8EA 4040EA60C0EA3180EA1F000B107F8F0F>48 D<120C123C12CC120CACEAFF8009107E8F0F >I<121FEA6180EA40C0EA806012C01200A213C0EA0180EA030012065AEA10201220EA7F C012FF0B107F8F0F>I<121FEA2180EA60C0A2120013801201EA0F00EA00801340136012 C0A2EA8040EA6080EA1F000B107F8F0F>I<1203A25A5A120B121312331223124312C3EA FFE0EA0300A4EA1FE00B107F8F0F>III<1240EA7FE013C0 EA8080A2EA010012025AA2120C1208A21218A50B117E900F>I<121FEA3180EA60C0A3EA 7180EA3F00120FEA3380EA61C0EAC060A3EA4040EA6080EA1F000B107F8F0F>I<121FEA 3180EA60C0EAC0401360A3EA40E01221EA1E6012001340EA60C01380EA4300123E0B107F 8F0F>I E /Fe 20 122 df<13011303A21306A3130CA31318A31330A31360A313C0A3EA 0180A3EA0300A31206A35AA35AA35AA35AA35AA3102D7DA117>47 D<131FA2497E133BA2EB7BC013731371EBF1E013E113E000017FA213C000031378A21380 00077FA21300380FFFFEA3381E000FA24814801407A24814C01403A24814E01B207F9F1E >65 D67 DI70 D83 D97 D99 D 101 D<3803E0F0EA0FFF5A383E3E00EA3C1E487EA5EA3C1EEA3E3EEA1FFC485AEA33E000 30C7FC1238EA3FFEEBFF806C13C04813E0387803F0EAF000A3EAF801387E07E0383FFFC0 000F1300EA03FC141E7F9317>103 D<12F0A41200A812F0B3A204207D9F0B>105 D<39F0FC07E039F3FE1FF039F7FF3FF839FE0FF07C39F807C03CA200F01380AE1E147D93 25>109 DIII114 D I<121EA6EAFFF0A3EA1E00AD1320EA1FF0120FEA07C00C1A7F9910>II<38F003C0A238780780A2127C383C0F00A2121E13 1E120EEA0F1CA2EA073C1338EA03B8A213B0EA01F0A26C5AA2485AA3485A1207007FC7FC 127E127C121D7F9315>121 D E /Ff 17 121 df<1238127C12FEA3127C123807077C86 0F>46 D<13E01201120F12FF12F31203B3A4B51280A2111D7C9C1A>49 DIII< 1260387FFFF8A214F014E014C038E0018038C00300A21306C65A5B1338A21378137013F0 A21201A41203A66C5A6C5A151E7D9D1A>55 D69 D97 DI101 D<121C123E127FA3123E121CC7FCA6B4FCA2121FB0EAFFE0A20B217EA00E> 105 D108 D<3AFE0FE03F8090391FF07FC03A 1E70F9C3E09039407D01F0EB807E121FEB007CAC3AFFE3FF8FFEA227147D932C>I<38FE 0FC0EB3FE0381E61F0EBC0F81380EA1F00AD38FFE3FFA218147D931D>I<38FF1FC0EB7F F0381FE1F8EB80FCEB007EA2143E143FA6143E147E147CEB80FCEBC1F8EB7FE0EB1F8090 C7FCA7EAFFE0A2181D7E931D>112 D116 D<38FFE1FFA2380F80706C6C5A6D5A 3803E180EA01F36CB4C7FC137E133E133F497E136FEBC7C0380183E0380381F0380701F8 380E00FC39FF81FF80A219147F931C>120 D E /Fg 71 126 df<127012F8B012701200 A5127012F8A31270051C779B18>33 DI38 D<137013F0EA01E0EA03C0EA0780EA0F00121E121C5AA25AA45AA81270A47E A27E121E7EEA0780EA03C0EA01F0120013700C24799F18>40 D<126012F012787E7E7EEA 07801203EA01C0A2EA00E0A41370A813E0A4EA01C0A2EA03801207EA0F00121E5A5A5A12 600C247C9F18>II<136013F0A7387FFFC0B512 E0A26C13C03800F000A7136013147E9718>I<121C123E127E127F123F121F1207120E12 1E127C12F81260080C788518>II<1230127812FC A2127812300606778518>I<1303EB0780A2130F14005B131EA2133E133C137C1378A213 F85B12015B12035BA212075B120F90C7FCA25A121E123E123CA2127C127812F85AA21260 11247D9F18>I49 DII<1230127812FCA2127812301200A81230127812FCA212781230 0614779318>58 D<1218123C127EA2123C12181200A81218123C127EA2123E121E120E12 1C123C127812F01260071A789318>I<1303EB0780131FEB3F00137E485AEA03F0485AEA 1FC048C7FC127E5AA2127E7EEA1FC0EA07E06C7EEA01FCEA007E7FEB1F801307EB030011 187D9918>I<387FFFC0B512E0A3C8FCA4B512E0A36C13C0130C7E9318>I<137013F8A213 D8A2EA01DCA3138CEA038EA41306EA0707A4380FFF80A3EA0E03A2381C01C0A2387F07F0 38FF8FF8387F07F0151C7F9B18>65 DIIIII<3801F1C0EA03FDEA0FFFEA1F0FEA 1C03123813011270A290C7FC5AA5EB0FF0131F130F387001C0A213031238A2EA1C07EA1F 0FEA0FFFEA03FDEA01F1141C7E9B18>I<387F07F038FF8FF8387F07F0381C01C0A9EA1F FFA3EA1C01AA387F07F038FF8FF8387F07F0151C7F9B18>II<387F07F038FF87F8387F07F0381C03C0EB 07801400130E131E5B13385B13F0121DA2EA1FB8A2131C121EEA1C0EA27FA2EB0380A2EB 01C0387F03F038FF87F8387F03F0151C7F9B18>75 DI<38FC01F8EAFE03A2383B06E0A4138EA2EA398CA213 DCA3EA38D8A213F81370A21300A638FE03F8A3151C7F9B18>I<387E07F038FF0FF8387F 07F0381D81C0A313C1121CA213E1A313611371A213311339A31319A2131D130DA3EA7F07 EAFF87EA7F03151C7F9B18>II III<3803F380EA1FFF5AEA7C0FEA7007EAE003A390C7FC 12701278123FEA1FF0EA07FEC67EEB0F80EB03C01301EB00E0A2126012E0130100F013C0 38F80780B5FCEBFE00EAE7F8131C7E9B18>I<387FFFF8B5FCA238E07038A400001300B2 EA07FFA3151C7F9B18>I<38FF83FEA3381C0070B36C13E0EA0F01380783C03803FF806C 1300EA007C171C809B18>I<38FE03F8A338700070A36C13E0A513F8A2EA39DCA2001913 C0A3138CEA1D8DA4000D13801305EA0F07A2EA0E03151C7F9B18>87 D<387F8FE0139F138F380E0700120FEA070E138EEA039C13DCEA01F8A26C5AA2137013F0 7F120113DCEA039E138EEA070F7F000E13801303001E13C0387F07F038FF8FF8387F07F0 151C7F9B18>I<38FF07F8A3381C01C0EA1E03000E1380EA0F0700071300A2EA038EA2EA 01DCA3EA00F8A21370A9EA01FC487E6C5A151C7F9B18>I91 D93 D95 D97 D<127E12FE127E120EA5133EEBFF80000F13C0EBC1E01380EB0070120E1438A6000F1370 A2EB80E013C1EBFFC0000E138038063E00151C809B18>IIIII<3801E1F03807FFF85A381E1E3038 1C0E00487EA5EA1C0EEA1E1EEA1FFC5BEA39E00038C7FC7EEA1FFEEBFFC04813E0387801 F038700070481338A4007813F0EA7E03381FFFC06C13803801FC00151F7F9318>I<127E 12FE127E120EA5133EEBFF80000F13C013C1EB80E01300120EAB387FC3FC38FFE7FE387F C3FC171C809B18>II<12FEA3120EA5EB3FF0137F133FEB0780EB0F00131E5B5B5BEA0FF87F13 9C130EEA0E0F7FEB038014C038FFC7F8A3151C7F9B18>107 DI<387CE0E038FFFBF8EA7FFF381F1F1CEA1E1E A2EA1C1CAB387F1F1F39FFBFBF80397F1F1F001914819318>IIII<3801E380EA07FBEA1FFFEA3E1FEA380FEA7007A2EAE003A6EA7007A2 EA380FEA3C1FEA1FFFEA0FFBEA03E3EA0003A7EB3FF8A3151E7E9318>I<387F87E038FF 9FF0387FBFF83803F878EBF030EBE0005BA25BA9EA7FFEB5FC6C5A15147F9318>II<487E1203A4387FFFC0B5FCA238038000 A9144014E0A33801C1C013FF6C1380EB3E0013197F9818>I<387E07E0EAFE0FEA7E07EA 0E00AC1301EA0F033807FFFC6C13FE3801FCFC1714809318>I<387F8FF000FF13F8007F 13F0381E03C0000E1380A338070700A3EA038EA4EA01DCA3EA00F8A2137015147F9318> I<38FF8FF8A3383800E0A3381C01C0A2137113F9A213D9A2380DDD80A3138DEA0F8FA238 07070015147F9318>I<387F8FF0139F138F38070700138EEA039EEA01DC13F812001370 13F07FEA01DCEA039E138EEA0707000F1380387F8FF000FF13F8007F13F015147F9318> I<387F8FF000FF13F8007F13F0380E01C0EB0380A21207EB0700A2EA03871386138EEA01 CEA2EA00CCA213DC1378A31370A313F05B1279EA7BC0EA7F806CC7FC121E151E7F9318> I<383FFFF05AA2387001E0EB03C0EB078038000F00131E5B13F8485AEA03C0485A380F00 70121E5A5AB512F0A314147F9318>II<127CB4FC13C01203C67EAB 7FEB7FC0EB3FE0A2EB7FC0EBF0005BABEA03C012FF90C7FC127C13247E9F18>125 D E /Fh 33 90 df<801301A549C7FCA61306A3B612E0A2D80006C7FC5BA65BA65B1310 1B207B9A23>43 D45 D<1230127812F81278127005057C840D> I<1404140CA21418A21430A21460A214C0A2EB0180A2EB0300A21306A25BA25BA25BA25B A25BA2485AA248C7FCA21206A35AA25AA25AA25AA25AA25A162D7EA117>I<137EEA01C3 38030180000713C0EA0E0014E05AA2EA3C0112381278A538F003C0A51480130712E01400 A2130E1260EA701CEA3038EA3870EA0FC0131F7C9D17>I<130C131C13FCEA0F381200A5 1370A613E0A6EA01C0A6EA038013C0EAFFFE0F1E7C9D17>II<137F38 0183C0380201E0380400F012075A1301A2000013E0A2EB03C01480EB0700130E13F8130E 7F148014C0130313071230127812F8A200F0138038800F00EA400E6C5AEA1878EA07E014 1F7D9D17>I<1460A214E0EB01C013031305130D13091311EB2380134313C31383EA0103 12023804070012081218121012201240B512F038000E00A55B131E3801FFE0141E7E9D17 >I<38018030EBFFE04813C0140013FC0002C7FCA35AA3137CEA058738060380EA0C0100 0813C0120014E0A4387003C012F812F000E01380EA800700401300130E6C5AEA1870EA07 C0141F7D9D17>I<1220383FFFE04813C0148038400100EAC00212805B485A5B5B5BA25B 120148C7FCA25A1206120EA2121E121CA2123CA35AA31230131F799D17>55 D<137EEA01C338030180000613C0120EEA1C00003C13E0A213011278A41303A2003813C0 1307EA1C0BEA0C133803E380EA0007A21400130E1260EAF01C1318485AEA8060EA41C000 3FC7FC131F7C9D17>57 D<141014181438A21478A214FCEB01BC143C130313021306EB04 3EEB081EA21310A21320801340A290B5FCEB800FD80100138014071202A25AA2000C14C0 123E39FF807FFC1E207E9F22>65 D<0007B5FC39007C01C0013C13E0EC00F04913F8A515 F0EBF00115E0EC03C0EC0F809038FFFE00EBF00F3901E007C0140315E01401A3EA03C014 03A215C01407EC0F80D807801300143EB512F01D1F7E9E20>I<903801F80890380E0618 903838013890387000F8484813784848137048481330A248C7FC5A121E003E1420003C14 00127CA45AA5007814401580A27EEC0100001C13027E0006130C6C13103801C0E0D8003F C7FC1D217B9F21>I<0007B5FC39007C01E090383C00F0157849133CA2151EA3151F5BA6 4848131E153EA3153C157C4848137815F0A2EC01E0EC03C0EC07803907800F00143CB512 E0201F7E9E23>I<0007B512F839007C0078013C133815185B1508A414089038F01000A3 1430EBFFF0EBF0703801E020A4EC00081510485AA21520A2156015C0380780011407B612 801D1F7E9E1F>I<0007B512F839007C0078013C133815185B1508A59038F01000A31430 1470EBFFF03801E0601420A491C7FC485AA6485A7FEAFFFE1D1F7E9E1E>I<903801FC04 90380F030C90383C009C0170137C49133C4848133848481318120748C7FC5A121E003E14 10003C1400127CA45AA2EC7FFCEC03E000781301A2EC03C0A27EA2121C6C13076CEB0B80 380380113801E06039003F80001E217B9F24>I<3A07FFC7FFC03A007C00F800013C1378 A2495BA649485AA490B5FCEBF0014848485AA64848485AA6484848C7FC01C07F39FFF8FF F8221F7E9E22>I<3807FFE038007C00133CA25BA65BA6485AA6485AA6485A7FEAFFFC13 1F7F9E10>I<3807FFF0D8007EC7FC133CA25BA65BA6485AA41580EC0100EA03C0A25C14 021406A23807801E147CB512FC191F7E9E1C>76 DI<3A07FC01FFC03A003E003E001518A2014F1310A2EB4780A2EB43 C0A201835BEB81E0A2EB80F0A21478D801005B147C143CA2141EA200026D5AA31407A200 06130392C7FC000F7FEAFFE0221F7E9E22>II<0007B5FC39007C03C090383C01E0EC00F05B15F8A415F0EBF00115E0EC03C0EC 0780EC0F00EBFFF8D801E0C7FCA6485AA6485A7FEAFFFC1D1F7E9E1F>I<3807FFFC3800 7C0790383C03C0EC01E0137815F0A415E0EBF00315C0EC0780EC0F00143CEBFFF03801E0 30143880A3141E3803C03EA51502D807801304EBC01F39FFFC0F18C7EA03E01F207E9E21 >82 DI<003FB512F0383C07800030143012603940 0F0010A212C01280A3D8001E1300A65BA65BA65B487E387FFFE01C1F7A9E21>I<39FFFC 3FF8390F8007C00007EB0300A2380F0002A6001E5BA6485BA600385BA35C003C5B121C5C 6C48C7FCEA0706EA01F81D20799E22>I<39FFF003FE391F8000F86CC712601540A26D13 800007130115001402A26D5A12035CA25C13E000015BA25CA26D5AD800F1C7FCA213F2A2 13FC137C1378A21370A213201F207A9E22>I<3A03FFC0FFC03A007F007E00013E133801 1E133015206D5B5D0281C7FCEB078214C6EB03C414E8EB01F0A2130080A2EB017CEB023C 1304EB0C1E1308EB101F497E01407FEBC00701807F38010003000780381F800739FFE01F FE221F7F9E22>88 D<39FFF003FF391F8000F86CC712606D134000071480EBC001000314 00EBE00200015BEBF00C140800005B6D5AEB7860EB7C40EB3C80013FC7FC7F131EA3131C 133CA513381378137C380FFFC0201F7A9E22>I E /Fi 27 119 df<14C0A3495AA449C7 FCA41306A2B612E0A2D8000CC7FCA45BA45BA413201B1C7A9823>43 D<127012F8A212F012E005057B840E>46 D<14021406A2140E141EA2143F142F144FA214 8FEB010FA21302A21304130C13081310A201201380EB3FFFEB400713C01380EA0100A212 02A21206001E130F39FF807FF81D207E9F22>65 D67 D<48B512FE39001E001C150C1504A25BA490387808081500A21418495AEBFFF0EBF030A2 3801E020A3EC001048481320A21540A248481380140115001407380F001FB512FE1F1F7D 9E1F>69 D77 D<48B5FC39001E03C0EC00E0157015785BA44913F0A2EC01E015C09038F00700141EEBFF F0EBF03848487E141E140E140F3803C01EA448485A1508A21510000F131C39FFF00C20C7 EA07C01D207D9E21>82 D<000FB512F0391E0780E00018142012101220EB0F0012601240 A2D8801E134000001400A35BA45BA45BA4485AA41203B5FC1C1F789E21>84 D97 DI<137EEA01C138030080EA0E07121E001C1300EA3C0248C7FCA35AA5EA 70011302EA3004EA1838EA07C011147C9315>I<1478EB03F8EB0070A414E0A4EB01C0A2 13F1EA038938070780EA0E03121C123C383807001278A3EAF00EA31420EB1C40A2EA703C 135C38308C80380F070015207C9F17>I<137CEA01C2EA0701120E121C123CEA3802EA78 0CEA7BF0EA7C0012F0A4127013011302EA3804EA1838EA07C010147C9315>I<1478EB01 9CEB033CA2EB07181400A2130EA53801FFE038001C00A45BA55BA65BA45B1201A25B1231 007BC7FC12F31266123C1629829F0E>III<13C0EA01E0A213C0C7FCA7120E12331223EA4380130012 471287120EA35AA3EA3840138012301270EA31001233121C0B1F7C9E0E>I108 D<391C0F80F0392630C318394740640C903880680EEB0070A2008E495A120EA34848485A A3ED70803A3803807100156115E115623970070066D83003133821147C9325>I<381C0F 80382630C0384740601380EB0070A2008E13E0120EA3381C01C0A3EB0384003813881408 1307EB031000701330383001C016147C931A>I<137CEA01C338030180000E13C0121E00 1C13E0123C1278A338F003C0A3EB07801400EA700F130EEA3018EA1870EA07C013147C93 17>I<3801C1E0380262183804741C1378EB701EA2EA08E01200A33801C03CA314383803 8078147014E0EBC1C038072380EB1E0090C7FCA2120EA45AA2EAFFC0171D809317>I 114 D<13FCEA0302EA0601EA0C03130713061300EA0F8013F0EA07F8EA03FCEA003E130E 1270EAF00CA2EAE008EA4010EA2060EA1F8010147D9313>II< 000E13C0003313E0382301C0EA43811301124738870380120EA3381C0700A31410EB0E20 1218A2EA1C1E380C26403807C38014147C9318>I<380E0380EA3307002313C0EA438313 01EA470000871380120EA3381C0100A31302A25BA25BEA0E30EA03C012147C9315>I E /Fj 36 122 df<1478137C13FC12013803C0005B1207A838FFFC78A3EA0780B3152480 A31A>12 D<497EB0B612FEA23900018000B01F227D9C26>43 D 45 D<12F0A404047B830E>I<13C01201120712FFA212FB1203B3A7B5FCA310227CA118> 49 DII<131FA2132FA2136F13EF13CF1201A2 EA038FA21207130F120F120E121E123CA21278A212F0B512F8A338000F00A915217FA018 >I<383FFF80A3003CC7FCA913F8EA3FFE7FEB0F80EA3E07383C03C01238380001E0A738 4003C0A212E038700780387C1F00EA3FFEEA0FFCEA03F013227EA018>I<137E48B4FC5A EA07C148C7FC121EA25AA25AA213F8EAF1FEEAF7FF38FF0F80EAFC0738F803C0A2130100 F013E0A51270A21278EB03C01238003C1380EA1C07381F0F00EA0FFE6C5AEA01F013237E A118>II66 DII<12F0B3B104237CA20D>73 D<00FC147E6C14FEA300F7EB01 DEA3EB800300F3149EA2EBC007A200F1141EA2EBE00F00F0130EA2EBF01EA2EB701CEB78 3CA2EB3838EB3C78A2EB1C70A2EB1EF0EB0EE0A2130FEB07C0A390C7FC1F237BA22A>77 D79 DI<13FE3803FFC04813E0EA0F81381E0060481300A25AA5 127C123C123FEA1FC0EA0FFC6CB4FC0001138038003FC0EB07E0EB01F0130014F81478A6 00C013F07E38F801E0387E07C0383FFF80000F1300EA01FC15257EA31B>83 D97 D<12F0ADEAF1F0EAF7FCB47EEAFC1F487E38F007 80A2EB03C0A8EB0780A238F80F00EAFC3EEAFFFEEAF7F8EAF1F012237CA219>IIII<3801F0783807FFF85A381F1F00EA1E0F383C0780A538 1E0F00EA1F1FEA0FFE485AEA19F00038C7FCA2123CEA1FFEEBFFC014E04813F0EA7801EB 00F8481378A4007813F0383E03E0381FFFC06C13803801FC0015217F9518>103 D<12F0A41200A912F0B3A404237DA20B>105 D<12F0B3B104237DA20B>108 D<39F0F807C039F3FE1FF039F7FF3FF839FE0F707800FCEBE07C39F807C03CA200F01380 AF1E167C9527>IIII114 DI<120FA6EAFFF8A3EA0F00AE 1308131CEA07FC13F8EA03E00E1C7F9B12>II<38F001E0A2387803C0A2127C383C0780A2121E14005B120F130EEA071E 139EEA039CA3EA019813D8EA00F85BA35BA212015BA2485A127F90C7FC127E13207F9516 >121 D E /Fk 70 124 df<90381F83E09038F06E303901C07878380380F8903800F030 48EB7000A7B612803907007000B2387FE3FF1D20809F1B>11 D<133FEBE0C0EA01C03803 81E0EA0701A290C7FCA6B512E0EA0700B2387FC3FE1720809F19>I<90381F81F89038F0 4F043901C07C06390380F80FEB00F05A0270C7FCA6B7FC3907007007B23A7FE3FE3FF024 20809F26>14 D34 D<127012F812FCA212741204A31208A21210A212201240060E7C9F 0D>39 D<132013401380EA0100120212065AA25AA25AA312701260A312E0AC1260A31270 1230A37EA27EA27E12027EEA0080134013200B2E7DA112>I<7E12407E7E7E120C7EA27E A2EA0180A313C01200A313E0AC13C0A312011380A3EA0300A21206A25A12085A5A5A5A0B 2E7DA112>I<1306AFB612F0A2D80006C7FCAF1C207D9A23>43 D<127012F812FCA21274 1204A31208A21210A212201240060E7C840D>II<127012F8A312 7005057C840D>I48 DIII<1306A2130EA2131E132EA2134E13 8EA2EA010E1202A212041208A212101220A2124012C0B512F038000E00A7EBFFE0141E7F 9D17>I I<137CEA0182EA0701380E0380EA0C07121C3838030090C7FC12781270A2EAF1F0EAF21C EAF406EAF807EB0380A200F013C0A51270A214801238EB07001218EA0C0E6C5AEA01F012 1F7E9D17>I<1240387FFFC01480A238400100EA8002A25B485AA25B5BA25BA213C0A212 015B1203A41207A76CC7FC121F7D9D17>III<1270 12F8A312701200AA127012F8A3127005147C930D>I<127012F8A312701200AA127012F0 12F8A212781208A31210A31220A21240051D7C930D>I<5B497EA3497EA3EB09E0A3EB10 F0A3EB2078A3497EA3497EA2EBFFFE3801000FA30002EB0780A348EB03C0120E001FEB07 E039FFC03FFE1F207F9F22>65 D I<90380FC04090387030C03801C0093803800538070003000E1301001E1300121C123C00 7C1440A2127800F81400A91278007C1440A2123C121C001E1480120E6CEB010038038002 6C6C5A38007038EB0FC01A217D9F21>IIII< 39FFF8FFF8390F800F8000071400AC90B5FCEB800FAE000F148039FFF8FFF81D1F7E9E22 >72 DI77 D<39FF803FF83907C007C0EC03803905E00100A2EA04F01378A2133CA2131E130FA2EB07 81A2EB03C1EB01E1A2EB00F1A21479143DA2141FA28080A2000E7F121F38FFE0011D1F7E 9E22>III82 D<3807E080EA0C19EA10 07EA3003EA6001A212E01300A36C1300A21278127FEA3FF0EA1FFC6C7EEA03FF38001F80 1307EB03C0A213011280A400C01380130300E01300EAF006EACE0CEA81F812217D9F19> I<007FB512E038780F010060EB006000401420A200C0143000801410A400001400B3497E 3807FFFE1C1F7E9E21>I<39FFFC3FF8390FC007C03907800380EC0100B3A300031302A2 EA01C05C6C6C5AEB7018EB3820EB0FC01D207E9E22>I<3BFFF07FF81FF03B1F800FC007 C03B0F0007800380EE010015C0D807801402A33A03C009E004A39039E010F00C00011508 A29038F0207800005DA2EC403C01785CA2ECC03E90393C801E40A390391F000F80A3011E 1307010E91C7FCA2010C7F010413022C207F9E2F>87 D<12FEA212C0B3B3A512FEA2072D 7CA10D>91 DI<12FEA21206B3B3A512FEA2072D7FA10D>I97 D<120E12FE120EAA133EEBC380380F01C0EB00E0120E1470A21478A61470 A214E0120F380D01C0380CC300EA083E15207F9F19>IIII<137C13C6EA018F1203EA070613 00A7EAFFF0EA0700B2EA7FF01020809F0E>I<14E03803E330EA0E3CEA1C1C38380E00EA 780FA5EA380E6C5AEA1E38EA33E00020C7FCA21230A2EA3FFE381FFF806C13C0383001E0 38600070481330A4006013606C13C0381C03803803FC00141F7F9417>I<120E12FE120E AA133E1343EB8180380F01C0A2120EAE38FFE7FC16207F9F19>I<121C121E123E121E12 1CC7FCA6120E127E120EB1EAFFC00A1F809E0C>I<13E0EA01F0A3EA00E01300A61370EA 07F012001370B3A31260EAF06013C0EA6180EA3F000C28829E0E>I<120E12FE120EAAEB 0FF0EB03C0140013025B5B5B1330137013F8EA0F38EA0E1C131E130E7F1480130314C014 E038FFCFF815207F9F18>I<120E12FE120EB3ABEAFFE00B20809F0C>I<390E1F01F039FE 618618390E81C81C390F00F00EA2000E13E0AE3AFFE7FE7FE023147F9326>IIII<3803E080EA0619EA1C05EA3C 07EA38031278127012F0A61270127812381307EA1C0BEA0E13EA03E3EA0003A8EB3FF815 1D7E9318>III<1202A31206A2120EA2123EEAFFF8EA0E00AB1308A5EA 06101203EA01E00D1C7F9B12>I<380E01C0EAFE1FEA0E01AE13031206EA030D3801F1FC 16147F9319>I<38FF83F8381E01E0381C00C06C1380A338070100A2EA0382A3EA01C4A2 13ECEA00E8A21370A3132015147F9318>I<39FF9FE1FC393C078070391C030060EC8020 000E1440A214C0D807071380130414E039038861001471EBC8733801D032143A3800F03C EBE01CA2EB6018EB40081E147F9321>I<387FC3FC380F01E0000713C0148038038100EA 01C2EA00E413EC13781338133C137C134E1387EA010738030380380201C0000613E0121E 38FF07FE1714809318>I<38FF83F8381E01E0381C00C06C1380A338070100A2EA0382A3 EA01C4A213ECEA00E8A21370A31320A25BA3EAF080A200F1C7FC1262123C151D7F9318> III E /Fl 12 118 df<13201340EA0180120313001206120E5AA2123C1238A21278A312F85A A97E1278A31238A2123C121CA27E12067E13801201EA004013200B297C9E13>40 D<7E12401230123812187E120E7EA213801203A213C0A313E01201A9120313C0A31380A2 12071300A2120E120C5A1238123012405A0B297D9E13>I<127812FCA412781200A61278 12FCA4127806127D910D>58 D68 D<3807FFF0A238001F80B21230127812FCA21400EA783EEA387CEA0FF0141C7F9B19>74 D99 D101 D<121E123FA4121EC7FCA6B4FCA2121FAEEAFFE0A20B1E7F9D0E>105 D<38FF0FC0EB31E0381F40F0EB80F8A21300AB38FFE7FFA218127F911B>110 DI115 D<38FF07F8A2EA1F00AC1301120F380786FFEA01F818127F911B>117 D E /Fm 2 64 df62 D<1304130CB3A7B612E0A21B1C7D9B21>I E /Fn 48 122 df12 D<1438B2B712FEA3C70038C7FCB227277DA12E>43 D<12F8A5123812301270A21260A212 E0050C7B8410>II<12F8A505057B8410>I<137E3801FF800007 13E04813F0381FC3F81300003E137CA2003C133C007C133EA20078131EA200F8131FAF00 78131E007C133EA46C137C003F13FC6C13F813C3380FFFF06C13E00001138038007E0018 297EA71D>48 D<13301370EA01F0120F12FFA212F11201B3ACB512E0A313287BA71D>I< 13FE3803FF80000F13C04813E0EB07F0383C01F8387800FC147C127000F0133EA2126012 20C7FCA3147CA2147814F8EB01F014E01303EB07C0EB0F80EB1F00133E5B5B485A485A5B 485A48C7FC121E5A387FFFFEA417287EA71D>I<137E3803FF804813E0001F13F0383F83 F8EA3E0048137C123812301210C7FCA214F8A21301EB03F0EB0FE03801FFC0148014C014 E0380001F8EB00FC147C143EA2141FA57E6C133E7E6C137E007C13FC383F01F8381FFFF0 6C13E0000313C03800FE0018297EA71D>II<383FFFF8A4003EC7FCAA133EEBFF80003F13E014F013C1EB00F8147C123E C7FC143EA712400060137C12E06C13F8EA7801387F07F0383FFFE0000F13C06C1300EA01 FC17287EA61D>III<137E3803 FFC04813E04813F0381F81F8383F00FC003E137C007E137E007C133EA56C137CA26C13F8 380FC3F03807FFE06C13C0A2000F13F0381F81F8383F00FC003E137C48133EA248131FA6 007C133EA2007E137E6C13FC381F81F8380FFFF06C13E06C13C038007E0018297EA71D> I<137E3801FF80000713C04813E0381F83F0EB00F8003E137848137C143C0078133E12F8 A2141E141FA4143F7E127C147F7E383F01FF381FFFDF6C139F6C131F3803FC1E3800403E 1300A2147CA214F8A2EB01F0381803E0381E0FC0383FFF806C13006C5AEA03F818297EA7 1D>I65 DIIIIII<12F8B3B3A6052A7BA910>73 D78 DII82 D84 D97 D99 D<143EAFEA01FCEA03FF000F13BE4813FE13C1383F007E003E133E5AA212 FC5AA77E127CA2007E137E6C13FEEA1FC113FF6C13BE3803FE3EEA01F8172A7EA91E>I< 13FCEA03FF4813804813C0381F87E0EA3F01383E00F05A1470481378B512F8A400F8C7FC A31278127CA27E003F1318381FC0F8EA0FFF7E000113E038007F00151B7E9A1A>II<90387C07803901 FF3FC0000713FF5A903883E000381F01F0A2383E00F8A56C485AA2380F83E013FF485B00 1D90C7FCEA1C7C003CC8FCA2123E381FFFE014F86C13FE487F481480387E003F007CEB0F C0481307A46C130F007EEB1F80393F807F00381FFFFE6C5B000313F038007F801A287F9A 1D>I<12F8A51200AA12F8B3A9052A7CA90E>105 D<12F8B3B3A6052A7CA90E>108 DII<137E3801FF80000713E04813F0381F81F8383F00FC003E137C4813 3EA20078131E00F8131FA7007C133EA36C137C003F13FC381FC3F8380FFFF06C13E06C13 C038007E00181B7E9A1D>IIIIII<00F813 F8B3A213011303EAFC07B5FCEA7FFEEA3FF8EA1FC0151B7C9A1E>I<00F8131FA2007C13 3EA36C137CA36C13F8A3380F81F0A33807C3E0A3000313C013E700011380A30000130013 F713FF137EA2181B7F9A1B>I<007CEB1F80007EEB3F006C133E6C5B380F80FC6C6C5AEB E1F03803E3E0EA01F76CB45A6D5A91C7FC133EA2133F497E497E3801F3E0EA03E1803807 C0F8380F807C121F497E003E7F007E148048EB0FC01A1B809A1B>120 D<00F8131F7E007C133EA27E147CA27E14F81380000F13F01381EA07C114E013C31203EB E3C0120113E71480EA00F714001377A2137E133EA2133CA2137C1378A25BA21201EA7FE0 5B5B90C7FC18277F9A1B>I E /Fo 29 122 df43 D46 D<177017F81601A2160317F01607 17E0A2160F17C0161F1780A2163F17005E167EA216FE5E15015EA215035E15075EA2150F 5E151F5EA2153F93C7FC5D157EA215FE5D14015DA214035D14075DA2140F5D141F5D143F 92C8FCA25C147E14FE5CA213015C13035CA213075C130F5CA2131F5C133F91C9FCA25B13 7E13FE5BA212015B12035BA212075B120F5BA2121F5B123F90CAFCA25A127E12FE5AA25A 12782D677ACC3A>III57 D66 DI70 D73 D76 DI80 D<90380FFFF090B6FC000315C03A07F8007FF0486CEB1FFCED07FE486C6D7E838183816C 48816C5A6C5AC9FCA5157F023FB5FC0103B6FC011F13F090387FFE003801FFE0481380D8 0FFEC7FC485A5B123F485AA2485AA45DA26C6C5BED077F6C6C130F6C6C013E13F83C0FFF 80F83FFFE000039038FFF01FC6ECC00F90390FFE0003332E7CAD38>97 DI100 DI<171FDA7FF0EBFFC00107B5000313E0011FECC7E7903B7FE03FFF0FF0 9039FF800FFC48EB00074848EB03FE00079238FF07E0496DEB03C0000FEE8000A2001F82 A8000F5EA2000793C7FC6D5B00035D6C6C495A6C6D485A9138E03FF0D801DFB512C0D803 C791C8FC9038C07FF04848CAFCA37FA27FA213F890B612F06C15FF17E06C8217FC6C826D 8148B812801207D80FF0C7001F13C0D81FC014014848EC007F007FEE3FE048C9FC171FA5 6C6CED3FC0A26C6CED7F806C6CEDFF00D80FF8EC03FED803FEEC0FF82601FFE0EBFFF06C 6CB612C0010F4AC7FCD9007F13C034447DAE3A>103 DI<137C48B4FC4813804813C0A24813E0 A56C13C0A26C13806C1300EA007C90C7FCACEB7FC0B5FCA412037EB3B2B6FCA418497CC8 20>I108 D<9039FF8007FEB590383FFFC04B13F0913981F81FF8913983C00FFC00039039870007FE 6C138E029C8002B87F188014F0A25CA35CB3A9B60081B6FCA4382E7BAD41>110 DI<90397F C01FFCB590B57E02C314F09139CFE03FFC9139DF0007FE000301FC6D7E6C496D13804A6D 13C04AEC7FE04A15F0EF3FF8A2EF1FFCA218FEA2170FA218FFAA18FE171FA218FCA2173F 18F818F0177F6EECFFE06E15C06E4913806E49130002DFEB0FFE9139CFC03FF802C3B512 E002C01480DB1FF8C7FC92C9FCB0B67EA438427CAD41>I<90397F803F80B5EBFFE00281 13F8913883C3FC91388707FE0003138E6C90389C0FFF14B8A214F0A2ED07FE9138E003FC ED01F892C7FCA25CB3A8B612E0A4282E7DAD2F>114 D<90390FFE01C090B512C7000314 FF3807F801390FC0007F48C7121F48140F007E1407A2150312FEA27E7F01E090C7FC13F8 387FFFC014FF6C14E015F86C806C14FF6C1580000115C06C6C14E0131F010014F0140391 38007FF80070141F00F0140F15077E1503A26C15F0A27E6CEC07E07F6DEB0FC001F0EB1F 80D8FEFCEBFF0039F87FFFFCD8F01F13F0D8E0031380252E7CAD2E>III121 D E /Fp 11 121 df<123C123EEA3FE090B71280A41700485D5E5E5EA2007CC7EA03E000 784A5A4B5A93C7FC48141E153E5D5DC7485A5D14034A5A5D140F4AC8FCA25C143E147E14 FE5CA21301A21303A2495AA4130FA5131FA86D5AA2EB03C0293B7BB930>55 D65 D67 D97 D<49B47E010F13F0017F13FC9038FF81FE3A03FE007F80D807F8133F 4848EB1FC0ED0FE0485A003F15F01507485A16F8A212FFA290B6FCA301C0C8FCA4127FA3 6C7E1678121F7F000F15F06C6C13016C6CEB03E06C6CEB0FC03A00FFC07F8090393FFFFE 00010F13F8010013C025267DA52C>101 D<13FE12FFA412071203B0EDFF80020313F002 0F7F91381E03FC91383801FE02607F4A7E01FF15805C91C7FCA35BB3A4B5D8F83F13FEA4 2F3C7CBB36>104 D110 D<9038FE01FF00FF011F13F0027F13FC9138FC07FE9039FFF001FF00079039C0007F806C 90C7EA3FC04915E0EE1FF017F8160F17FCA2160717FEA917FC160FA317F8EE1FF0A26DEC 3FE06EEB7FC06EEBFF806E4813009039FEFC07FE91387FFFF8021F13E0DA03FEC7FC91C9 FCADB512F8A42F377DA536>112 D<3901FC03F000FFEB0FFC4AB4FC91383C3F80EC707F 00079038E0FFC000035BEBFD80A201FFEB7F809138003F00151E92C7FC5BB3A3B512FCA4 22267DA528>114 D<130FA55BA45BA25BA25B5A5A5A001FEBFFF0B6FCA3000190C7FCB3 153CA86C14781480017F13F090383FC1E090381FFFC06D1380903801FE001E377EB626> 116 D120 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 1 1 1 0 bop 75 361 a Fp(Chapter)31 b(7)75 568 y Fo(MPI)40 b(1.1)f(C++)g(/)g(F)-10 b(ortran)40 b(90)75 693 y(Language)f(Bindings) 75 933 y Fn(7.1)59 b(C++)19 b(Intro)r(duction)1875 1039 y Fm(>)p Fl(\(June\))75 1035 y Fk(This)14 b(section)g(presen)o(ts)f (the)h(design)g(of)f(C++)h([1)o(,)f(2])f(bindings)k(for)c(MPI)i(1.1.)k (The)c(C++)f(bindings)j(for)75 1091 y(MPI)g(1.1)f(functions)h(are)g (listed)h(in)f(App)q(endix)i(A.)k(C++)16 b(bindings)i(for)d(MPI)h(2)g (functions)g(are)g(listed)75 1148 y(in)g(their)g(resp)q(ectiv)o(e)g(c)o (hapters.)75 1270 y Fj(7.1.1)49 b(Design)75 1355 y Fk(The)15 b(MPI)h(C++)f(bindings)i(w)o(ere)e(designed)i(in)f(accordance)f(to)g (the)g(follo)o(wing)h(criteria:)131 1462 y(1.)22 b(The)13 b(C++)h(language)g(bindings)h(should)f(consist)g(of)f(a)g(small)h(set)f (of)g(classes)h(with)g(a)f(ligh)o(t)o(w)o(eigh)o(t)189 1518 y(functional)20 b(in)o(terface)g(to)f(MPI.)g(The)h(set)f(of)g (classes)h(should)h(b)q(e)f(as)f(small)i(as)e(p)q(ossible,)j(and)189 1575 y(the)15 b(classes)h(should)g(corresp)q(ond)f(to)g(fundamen)o(tal) g(MPI)g(ob)s(ject)g(t)o(yp)q(es)g(\(e.g.,)f(comm)o(unicator,)189 1631 y(group,)g(etc.,)h(as)f(w)o(ell)j(as)d(an)o(y)h(new)h(t)o(yp)q(es) f(that)f(ma)o(y)h(emerge)g(from)g(MPI-2\).)131 1725 y(2.)22 b(The)e(MPI)f(C++)h(language)g(bindings)i(m)o(ust)d(pro)o(vide)h(a)g (seman)o(tically)g(correct)g(in)o(terface)g(to)189 1781 y(MPI-2.)131 1875 y(3.)i(The)16 b(C++)g(function)h(bindings)h(should)e (seman)o(tically)h(and)g(syn)o(tactically)f(corresp)q(ond)h(to)e(the) 189 1932 y(MPI)21 b(functional)h(sp)q(eci\014cation.)40 b(T)l(o)21 b(the)h(greatest)e(exten)o(t)h(p)q(ossible,)j(the)d(C++)h (functions)189 1988 y(should)16 b(b)q(e)g(mem)o(b)q(er)f(functions)h (of)f(MPI)g(classes.)189 2094 y Fi(R)n(ationale.)62 b Fk(There)20 b(are)e(three)h(principle)j(options)d(to)g(consider)g(for)g (C++)g(language)g(bind-)189 2151 y(ings:)j(use)16 b(the)g(existing)h(C) f(language)g(bindings,)i(pro)o(vide)e(a)g(comprehensiv)o(e)h(class)g (library)l(,)f(or)189 2207 y(pro)o(vide)j(a)f(ligh)o(t)o(w)o(eigh)o(t)g (set)h(of)f(MPI)g(ob)s(jects.)29 b(Using)19 b(the)g(existing)g(C)f (bindings)i(has)f(signif-)189 2264 y(ican)o(t)f(dra)o(wbac)o(ks)e(in)j (that)e(m)o(uc)o(h)h(of)f(the)h(expressiv)o(e)h(p)q(o)o(w)o(er)e(of)g (the)h(C++)g(language)g(is)g(lost.)189 2320 y(While)h(pro)o(viding)g(a) f(comprehensiv)o(e)h(class)g(library)g(w)o(ould)f(mak)o(e)g(user)g (programming)f(more)189 2377 y(elegan)o(t,)f(it)g(is)h(not)f(suitable)h (as)f(a)g Fi(binding)f Fk(since)i(a)f(binding)i(m)o(ust)e(pro)o(vide)g (a)g(direct)h(and)f(un-)189 2433 y(am)o(biguous)e(mapping)g(to)g(the)g (sp)q(eci\014ed)i(functionalit)o(y)f(of)f(MPI.)g(Pro)o(viding)g(a)g (ligh)o(t)o(w)o(eigh)o(t)g(set)189 2489 y(of)j(MPI)g(ob)s(jects)g(that) g(corresp)q(ond)h(to)f(the)g(basic)i(MPI)e(t)o(yp)q(es)h(is)g(the)f(b)q (est)h(\014t)f(to)g(MPI's)g(im-)189 2546 y(plicit)h(ob)s(ject-based)f (design;)h(metho)q(ds)f(can)g(b)q(e)g(supplied)i(for)d(these)h(ob)s (jects)f(to)g(realize)i(MPI)189 2602 y(functionalit)o(y)l(.)189 2677 y(\()p Fi(End)d(of)i(r)n(ationale.)p Fk(\))964 2828 y(1)p eop %%Page: 2 2 2 1 bop 75 -100 a Fk(2)315 b Fh(CHAPTER)15 b(7.)35 b(MPI)15 b(1.1)f(C++)i(/)f(F)o(OR)l(TRAN)h(90)e(LANGUA)o(GE)h(BINDINGS)75 49 y Fj(7.1.2)49 b(MPI)17 b(1.1)f(C++)h(Bindings)75 135 y Fk(The)f(complete)g(set)f(of)g(C++)h(language)g(bindings)h(is)f (presen)o(ted)g(in)h(App)q(endix)g(A.)k(There)16 b(is)g(a)f(small)75 192 y(set)i(of)h(classes)g(de\014ned)h(for)e(these)h(bindings)i (corresp)q(onding)e(to)g(ob)s(jects)f(implicitly)j(used)f(b)o(y)e(MPI.) 75 248 y(Sp)q(eci\014cally)l(,)h(the)d(follo)o(wing)h(are)f(the)h (classes)f(pro)o(vided)h(with)g(the)f(C++)h(language)f(bindings:)170 338 y Fg(class)24 b(MPI_Comm;)170 395 y(class)g(MPI_Datatype;)170 451 y(class)g(MPI_Errorhandler;)170 508 y(class)g(MPI_Group;)170 564 y(class)g(MPI_Op;)170 621 y(class)g(MPI_Request;)170 677 y(class)g(MPI_Status;)166 768 y Fk(T)l(o)14 b(main)o(tain)i (consistency)g(with)f(the)g(binding)i(de\014nitions,)f(w)o(e)e(list)i (the)f(C++)g(bindings)i(in)f(the)75 824 y(same)d(order)f(as)h(giv)o(en) g(for)f(the)h(C)g(bindings)h(in)g([3)o(].)19 b(Note)12 b(that)h(the)f(functions)i(in)g(App)q(endix)h(A)e(do)f(not)75 881 y(use)17 b(p)q(olymorphism)h(or)e(o)o(v)o(erloading)g(for)g(the)h (reasons)f(describ)q(ed)i(in)g(Section)f(7.1.1.)22 b(The)17 b(bindings)75 937 y(also)e(use)h(reference)g(and)f Fg(const)f Fk(seman)o(tics)i(\(an)f(imp)q(ortan)o(t)f(feature)h(of)g(the)g(C++)h (language\).)-1927 b Fl(\(June\))p Fm(?)166 994 y Fk(In)16 b(order)f(to)g(mak)o(e)g(the)h(function)g(list)g(less)g(cluttered,)g (the)g(k)o(eyw)o(ord)f Fg(virtual)f Fk(has)h(b)q(een)i(omit-)75 1050 y(ted)h(from)g(eac)o(h)g(declaration)h(with)g(the)g(implicit)h (understanding)g(that)d(all)j(mem)o(b)q(er)e(functions)h(are)75 1106 y Fg(virtual)e Fk(\(except)g(for)g(constructors)g(and)h(the)g (assignmen)o(t)f(op)q(erator\).)26 b(This)18 b(allo)o(ws)g(class)g (library)75 1163 y(ob)s(jects)d(to)f(b)q(e)i(deriv)o(ed)g(from)f(the)g (MPI)g(ob)s(jects)g(in)h(order)f(to)f(pro)o(vide)i(additional)h (functionalit)o(y)l(.)-160 1213 y Fl(\(June\))p Fm(>)75 1265 y Ff(Example)h(7.1)k Fg(class)i(foo_comm)e(:)i(public)f(class)g (MPI_Comm)g({)75 1322 y(public:)123 1378 y(...)123 1435 y(Dup\(const)f(MPI_Comm&)h(comm\))g({)170 1491 y(//)h(Class)f(library)g (functionality)170 1548 y(MPI_Comm::Dup\(comm\);)f(//)h(Call)g(base)h (MPI)f(function)170 1604 y(//)h(More)f(class)h(library)e(functionality) 123 1660 y(})123 1717 y(...)75 1773 y(};)75 1894 y Fj(7.1.3)49 b(Semantics)75 1980 y Fe(F)o(unction)18 b(Scoping)47 b Fk(T)l(o)16 b(the)i(greatest)e(p)q(ossible)j(exten)o(t,)e(C++)g (language)h(bindings)h(for)d(MPI)h(func-)75 2037 y(tions)11 b(should)h(b)q(e)g(mem)o(b)q(er)f(functions)h(of)f(the)g(sp)q (eci\014ed)i(MPI)f(C++)f(classes.)19 b(The)12 b(bindings)g(presen)o (ted)75 2093 y(in)20 b(App)q(endix)h(A)e(w)o(ere)f(generated)h(b)o(y)g (applying)h(a)f(simple)h(set)f(of)f(name)h(generation)g(rules)h(to)e (the)75 2150 y(MPI-1)d(function)i(sp)q(eci\014cations.)22 b(These)16 b(guidelines)i(ha)o(v)o(e)d(also)g(b)q(een)i(applied)g(to)e (MPI-2)g(functions)75 2206 y(to)j(create)g(the)g(MPI-2)g(C++)h (bindings)h(con)o(tained)f(in)g(other)f(c)o(hapters)g(of)f(this)i(do)q (cumen)o(t.)30 b(While)75 2263 y(these)17 b(guidelines)i(ma)o(y)d(b)q (e)h(su\016cien)o(t)h(in)f(most)f(cases)g(for)g(creating)h(C++)g (bindings,)i(they)d(ma)o(y)g(not)75 2319 y(b)q(e)g(suitable)h(for)e (all)i(situations.)k(In)c(cases)e(of)g(am)o(biguit)o(y)h(or)f(where)h (a)g(sp)q(eci\014c)h(seman)o(tic)f(statemen)o(t)75 2375 y(is)g(desired,)g(these)f(guidelines)j(ma)o(y)d(b)q(e)g(sup)q(erseded)i (as)e(the)g(situation)h(dictates.)131 2478 y(1.)22 b(If)c(the)h(MPI)f (function)h(con)o(tains)f(one)h(or)f(more)f(scalar)i(MPI)f(handles)h (in)h(the)e(argumen)o(t)g(list,)189 2534 y(the)13 b(function)h(should)g (b)q(e)g(made)f(a)g(mem)o(b)q(er)g(function)h(of)f(an)g(MPI)g(class)g (corresp)q(onding)h(to)f(one)189 2591 y(of)18 b(the)g(MPI)h(handles)g (in)h(the)e(argumen)o(t)g(list.)31 b(In)19 b(the)f(case)h(when)g(there) f(is)h(more)f(than)h(one)189 2647 y(candidate)12 b(class,)h(the)e(c)o (hoice)i(of)e(whic)o(h)i(class)f(to)f(asso)q(ciate)g(the)h(mem)o(b)q (er)g(function)g(will)i(usually)189 2704 y(b)q(e)k(ob)o(vious)g(from)f (seman)o(tic)h(\(and)f(syn)o(tactic\))h(con)o(text.)26 b(The)18 b(mem)o(b)q(er)g(function)h(should)f(b)q(e)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 3 3 2 bop 75 -100 a Fh(7.1.)34 b(C++)16 b(INTR)o(ODUCTION)1182 b Fk(3)189 49 y(named)18 b(according)g(to)f(the)h(corresp)q(onding)g (MPI)g(function)g(name,)g(but)g(without)g(the)g(\\)p Fg(MPI)p 1839 49 15 2 v 16 w Fk(")189 106 y(pre\014x)d(and)h(without)f (the)g(ob)s(ject)g(name)g(pre\014x)h(\(if)f(applicable\).)189 195 y Ff(Example)i(7.2)23 b Fk(The)15 b(C++)h(binding)h(for)d Fg(MPI)p 1014 195 V 17 w(COMM)p 1127 195 V 16 w(COMPARE\(\))g Fk(is)i Fg(MPI)p 1491 195 V 16 w(Comm::Compare\(\))p Fk(.)189 284 y(The)f(follo)o(wing)h(\\rules)g(of)e(th)o(um)o(b")h(can)g (usually)i(b)q(e)f(applied:)208 369 y(\(a\))22 b(If)c(the)h(argumen)o (t)f(list)h(con)o(tains)g(one)f(or)g(more)h(arra)o(ys)e(of)h(MPI)g (handles,)j(these)d(arra)o(ys)289 426 y(should)e(b)q(e)g(left)f(in)h (the)g(argumen)o(t)e(list,)i(ev)o(en)f(if)h(they)f(are)g(\\out")f (paramem)o(ters.)205 491 y(\(b\))23 b(If)f(the)h(argumen)o(t)e(list)j (con)o(tains)e(a)g(single)i(scalar)e(MPI)h(handle,)i(mak)o(e)d(the)g (function)289 548 y(a)h(metho)q(d)i(of)e(the)h(MPI)g(ob)s(ject)g (corresp)q(onding)h(to)e(the)h(MPI)g(handle)i(\(e.g.,)e Fg(MPI)p 1837 548 V 17 w(-)289 604 y(Comm::Compare\(\))p Fk(\).)210 669 y(\(c\))f(If)17 b(a)f(function)i(can)f(b)q(e)h(ob)o (viously)f(assigned)h(to)e(a)g(particular)i(class)f(and)g(it)g(has)g(a) g(single)289 726 y(argumen)o(t)12 b(of)i(that)f(t)o(yp)q(e,)g(that)g (argumen)o(t)g(should)i(b)q(e)f(dropp)q(ed)h(from)e(the)g(argumen)o(t)g (list)289 782 y(\(and)i(b)q(ecome)h Fg(this)p Fk(\).)205 847 y(\(d\))23 b(If)18 b(a)g(function)i(can)e(b)q(e)h(ob)o(viously)g (assigned)g(to)f(a)g(particular)h(class)g(and)g(it)f(has)g(t)o(w)o(o)f (or)289 904 y(more)i(argumen)o(ts)g(of)g(that)g(t)o(yp)q(e,)i(the)f (\\out")f(v)m(ariable)i(should)g(b)q(e)f(dropp)q(ed)h(from)e(the)289 960 y(argumen)o(t)14 b(list)i(\(and)f(b)q(ecome)h Fg(this)p Fk(\).)289 1036 y Ff(Example)h(7.3)23 b Fk(The)16 b(C++)h(binding)g (for)f Fg(MPI)p 1118 1036 V 16 w(Comm)p 1230 1036 V 17 w(dup\(MPI)p 1415 1036 V 16 w(Comm)24 b(comm,)f(MPI)p 1766 1036 V 17 w(Comm)289 1092 y(*newcomm\))14 b Fk(b)q(ecomes)h Fg(MPI)p 774 1092 V 17 w(Comm::Dup\(const)22 b(MPI)p 1245 1092 V 17 w(Comm&)h(comm\))p Fk(.)210 1168 y(\(e\))g(If)12 b(the)g(original)h(function)g(do)q(es)f(not)g(con)o(tain)g(an)o(y)g (scalar)g(MPI)g(handles,)i(the)e(full)h(function)289 1224 y(name)i(should)h(b)q(e)g(retained.)289 1300 y Ff(Example)h(7.4)23 b Fk(The)10 b(C++)h(binding)h(for)e Fg(MPI)p 1095 1300 V 16 w(Buffer)p 1255 1300 V 17 w(attach\(void*)22 b(buffer,)h(int)g (size\))289 1356 y Fk(b)q(ecomes)15 b Fg(::MPI)p 592 1356 V 17 w(Buffer)p 753 1356 V 16 w(attach\(void*)23 b(buffer,)f(int)i(size\))p Fk(.)1875 1377 y Fm(?)p Fl(\(June\))131 1457 y Fk(2.)e(F)l(ollo)o(w)15 b(the)g(MPI-1)g(capitalization)i(rules)f (for)f(the)g(metho)q(d)g(names.)131 1543 y(3.)22 b(Metho)q(ds)15 b(should)h(b)q(e)g(declared)h Fg(const)e Fk(if)g(they)h(return)f (information)h(to)e(the)i(user)f(and)h(do)f(not)189 1599 y(c)o(hange)g(the)g(in)o(ternal)h(state)f(of)f(the)i(calling)h(ob)s (ject.)131 1685 y(4.)22 b(An)o(y)16 b(reference)i(or)e(arra)o(y)f (argumen)o(t)h(that)g(will)i(not)e(b)q(e)i(mo)q(di\014ed)g(in)f(the)g (metho)q(d)g(should)g(b)q(e)189 1742 y(declared)f Fg(const)p Fk(.)131 1828 y(5.)22 b(P)o(oin)o(ter)12 b(argumen)o(ts)g(should)i(b)q (e)f(c)o(hanged)g(to)f(references)h(when)h(a)e(reference)h(is)h(what)e (is)h(seman-)189 1884 y(tically)j(implied.)1875 1941 y Fm(>)p Fl(\(June\))131 1970 y Fk(6.)22 b(Arra)o(y)16 b(argumen)o(ts)g(should)i(b)q(e)g(denoted)g(with)f(square)g(brac)o(k)o (ets)f(\()p Fg([])p Fk(\),)g(not)h(p)q(oin)o(ters,)h(as)f(this)189 2027 y(is)e(more)g(seman)o(ticly)h(precise.)1156 b Fm(?)p Fl(\(June\))75 2143 y Fe(Cop)o(y)18 b(/)g(Assignment)h(Semantics)46 b Fk(The)19 b(cop)o(y)f(and)g(assignmen)o(t)g(seman)o(tics)g(in)h(C++)g (should)g(b)q(e)f(as)75 2199 y(implicitly)25 b(sp)q(eci\014ed)e(b)o(y)f (the)g(MPI-1)f(do)q(cumen)o(t)h(\(the)g(same)f(as)g(those)g(sp)q (eci\014ed)j(b)o(y)e(the)f(C)h(and)75 2256 y(F)l(ortran)d(77)h (bindings\).)37 b(That)19 b(is,)j(in)f(terms)f(of)g(cop)o(ying)h(and)f (assignmen)o(t,)h(the)g(MPI)f(user)h(lev)o(el)75 2312 y(ob)s(jects)13 b(should)i(b)q(eha)o(v)o(e)f(lik)o(e)g(handles.)21 b(Cop)o(y)13 b(constructors)g(should)i(p)q(erform)e(handle-based)i (copies.)189 2395 y Fi(A)n(dvic)n(e)g(to)h(implementors.)40 b Fk(Eac)o(h)14 b(MPI)h(user)g(lev)o(el)h(ob)s(ject)f(is)g(lik)o(ely)i (to)d(con)o(tain,)h(b)o(y)f(v)m(alue)i(or)189 2452 y(b)o(y)i (reference,)i(an)f(implemen)o(tation-dep)q(endent)i(state)d (information.)30 b(The)19 b(assignmen)o(t)g(and)189 2508 y(cop)o(ying)g(of)f(MPI)h(ob)s(ject)g(handles)h(ma)o(y)e(simply)i(cop)o (y)f(this)g(v)m(alue)h(\(or)e(reference\).)32 b(\()p Fi(End)19 b(of)189 2564 y(advic)n(e)d(to)g(implementors.)p Fk(\))166 2647 y(The)g Fg(MPI)p 335 2647 V 17 w(Request)e Fk(and)i Fg(MPI)p 695 2647 V 17 w(Status)f Fk(ob)s(jects)g(are)g (exceptions)i(to)e(this)i(\\handle-based")f(rule.)75 2704 y(These)g(ob)s(jects)e(should)i(p)q(erform)f(deep)h(copies)g(for)f (assignmen)o(t)g(and)g(cop)o(y)h(constructors.)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 4 4 3 bop 75 -100 a Fk(4)315 b Fh(CHAPTER)15 b(7.)35 b(MPI)15 b(1.1)f(C++)i(/)f(F)o(OR)l(TRAN)h(90)e(LANGUA)o(GE)h(BINDINGS)75 49 y Fe(Construction)h(/)e(Destruction)h(Semantics)47 b Fk(The)14 b(construction)g(and)h(destruction)f(seman)o(tics)h(in)f (C++)75 106 y(should)i(b)q(e)g(the)g(same)f(as)g(those)g(sp)q (eci\014ed)j(b)o(y)d(the)h(C)f(and)g(F)l(ortran)g(77)f(bindings.)23 b(That)15 b(is,)g(in)i(terms)75 162 y(of)h(construction)h(and)g (destruction)g(seman)o(tics,)g(the)f(MPI)h(user)g(lev)o(el)h(ob)s (jects)d(should)j(b)q(eha)o(v)o(e)f(lik)o(e)75 219 y(handles.)-396 b Fl(\(June\))p Fm(?)189 325 y Fi(A)n(dvic)n(e)20 b(to)j(implementors.) 76 b Fk(Default)21 b(constructors)f(for)h(all)h(MPI)f(ob)s(jects)g (should)h(create)189 381 y(corresp)q(onding)c Fg(MPI)p 558 381 15 2 v 17 w(*)p 599 381 V 17 w(NULL)g Fk(handles.)29 b(That)17 b(is,)i(when)f(an)g(MPI)g(ob)s(ject)f(is)h(instan)o(tiated,)h (it)189 438 y(should)14 b(b)q(e)g(comparable)f(with)h(its)f(corresp)q (onding)i Fg(MPI)p 1160 438 V 16 w(*)p 1200 438 V 17 w(NULL)p Fk(.)e(F)l(or)f(example,)i Fg(result)f Fk(in)h(the)189 494 y(follo)o(wing)i(co)q(de)f(fragmen)o(t)f(will)j(get)e(the)g(v)m (alue)i Fg(MPI)p 1110 494 V 16 w(IDENT)p Fk(:)189 619 y Fg(void)23 b(foo\(void\))189 676 y({)236 732 y(int)h(result;)236 789 y(MPI_Comm)f(foo;)236 901 y(foo.Compare\(MPI_COMM_NULL,)e (result\);)236 958 y(if)j(\(result)f(==)g(MPI_IDENT\))284 1014 y(cout)g(<<)h("foo)f(is)h(MPI_COMM_NULL")e(<<)h(endl;)189 1071 y(})189 1196 y Fk(The)c(destructor)f(for)g(eac)o(h)h(MPI)g(user)g (lev)o(el)h(ob)s(ject)e(should)i Fi(not)e Fk(in)o(v)o(ok)o(e)h(the)g (corresp)q(onding)189 1252 y Fg(MPI)p 264 1252 V 16 w(*)p 304 1252 V 17 w(free\(\))c Fk(function)h(\(if)f(it)h(exists\))f(for)f (the)i(follo)o(wing)g(reasons:)231 1359 y(1.)22 b(Suc)o(h)15 b(a)f(sc)o(heme)h(w)o(ould)g(not)f(b)q(e)i(consisten)o(t)f(with)f(the)h (C)g(and)f(F)l(ortran)g(77)g(functionalit)o(y)l(.)231 1432 y(2.)22 b(The)d(mo)q(del)h(put)f(forth)f(in)i(MPI-1)e(mak)o(es)h (memory)f(allo)q(cation)i(and)f(deallo)q(cation)i(the)289 1488 y(resp)q(onsibilit)o(y)c(of)e(the)g(user,)g(not)g(the)g(implemen)o (tation.)-160 1538 y Fl(\(June\))p Fm(>)231 1561 y Fk(3.)22 b(MPI)15 b(ob)s(jects)g(going)h(out)f(of)h(scop)q(e)g(ma)o(y)f(pro)q (duce)i(collectiv)o(e)g(op)q(erations,)f(whic)o(h)g(is)g(not)289 1618 y(in)o(tuitiv)o(e,)f(and)f(ma)o(y)f(not)g(b)q(e)i(what)e(the)h (user)g(w)o(an)o(ts)e(\(this)i(also)g(a\013ects)f(the)h(cop)o(y)l(,)g (assign-)289 1674 y(men)o(t,)h(and)g(construction)h(seman)o(tics\).)21 b(F)l(or)15 b(example,)h(the)g Fg(bar)f Fk(comm)o(unicator)h(that)e(is) 289 1730 y(created)g(at)g(the)h(b)q(eginning)i(of)d(the)g(function)i (sho)o(wn)e(b)q(elo)o(w)h(is)g(freed)g(when)g(the)g(function)289 1787 y(exits,)g(whic)o(h)h(ma)o(y)e(trigger)h(a)g(collectiv)o(e)i(op)q (eration:)289 1883 y Fg(void)23 b(foo\(\))289 1939 y({)336 1996 y(MPI_Comm)g(bar;)336 2052 y(bar.Dup\(MPI_COMM_WORLD\);)336 2109 y(//)h(Rest)f(of)h(the)f(function)336 2165 y(//)h(bar)f(goes)h (out)f(of)h(scope)289 2222 y(})189 2328 y Fk(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fk(\))75 2448 y Fe(Compa)o(rison)k(Semantics)46 b Fk(Sev)o(eral)21 b(MPI)g(ob)s(jects)f(already)h(ha)o(v)o(e)f(MPI)g(functions)i(for)e (comparison)75 2504 y(\(i.e.,)12 b Fg(MPI)p 251 2504 V 17 w(Comm::Compare\(\))d Fk(and)k Fg(MPI)p 795 2504 V 16 w(Group::Compare\(\))p Fk(\).)k(These)12 b(functions)h(can)f (return)f(more)75 2561 y(than)20 b(t)o(w)o(o)f(v)m(alues;)24 b(a)c(b)q(o)q(olean)i(v)m(alue)f(is)g(not)f(su\016cien)o(t)i(to)d (describ)q(e)j(their)f(relationship.)38 b(There-)75 2617 y(fore,)16 b(the)h(equalit)o(y)h(and)f(inequalit)o(y)i(op)q(erators)c (\()p Fg(operator==\(\))g Fk(and)i Fg(operator!=\(\))p Fk(\))e(m)o(ust)h(ha)o(v)o(e)75 2674 y(a)f(sligh)o(tly)h(di\013eren)o (t)g(meaning)f(than)h(simply)g(\\equal")f(or)g(\\not)g(equal".)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 5 5 4 bop 75 -100 a Fh(7.1.)34 b(C++)16 b(INTR)o(ODUCTION)1182 b Fk(5)166 49 y(Ho)o(w)o(ev)o(er,)16 b(man)o(y)h(C)g(and)h(F)l(ortran)e (implemen)o(tations)i(that)f(use)g(p)q(oin)o(ters)h(or)f(in)o(tegers)g (as)g(MPI)75 106 y(handles)i(indirectly)h(allo)o(w)d(the)h(use)g(of)f (the)h Fg(==)f Fk(and)h Fg(!=)f Fk(op)q(erators;)g(the)h(only)g (meaningful)h(b)q(o)q(olean)75 162 y(resp)q(onse)f(is)h Fg(true)p Fk(,)f(whic)o(h)h(is)f(equiv)m(alen)o(t)i(to)d Fg(MPI)p 970 162 15 2 v 17 w(IDENT)p Fk(.)g(As)h(suc)o(h,)g(it)h(is)f (desirable)i(to)d(adopt)h(this)75 219 y(functionalit)o(y)h(in)o(to)f (the)g(C++)g(bindings)i(so)e(as)f(to)g(remain)i(consisten)o(t)f(with)g (C)g(and)g(F)l(ortran.)27 b(F)l(or)75 275 y(example,)19 b(after)e(the)h(assignmen)o(t)f(of)g(t)o(w)o(o)g Fg(MPI)p 914 275 V 16 w(Comm)h Fk(ob)s(jects)f(suc)o(h)h(as)f Fg(a)24 b(=)f(b)p Fk(,)18 b(the)g(test)f Fg(\(a)24 b(==)f(b\))75 332 y Fk(should)16 b(return)f Fg(true)p Fk(.)1408 b Fm(?)p Fl(\(June\))166 388 y Fk(Sev)o(eral)18 b(ob)s(jects)f(do)h(not)f(ha)o (v)o(e)g(metho)q(ds)h(equiv)m(alen)o(t)h(to)e(the)h(C)f Fg(MPI)p 1407 388 V 17 w(*)p 1448 388 V 17 w(compare\(\))f Fk(functions)75 444 y(\(e.g.,)h Fg(MPI)p 267 444 V 17 w(Datatype)p Fk(\).)28 b(Since)19 b(no)f(other)g(form)g(of)g (comparison)g(is)h(a)o(v)m(ailable,)h(and)e(since)i(a)d(simple)75 501 y(b)q(o)q(olean)23 b(answ)o(er)f(is)h(all)g(that)e(is)i(required,)i (the)d(equalit)o(y)h(and)g(inequalit)o(y)h(op)q(erators)d(should)i(b)q (e)75 557 y(o)o(v)o(erridden)16 b(to)f(pro)o(vide)i(this)f (functionalit)o(y)l(.)23 b(This)16 b(is)h(also)e(consisten)o(t)h(with)h (the)e(C)h(and)g(F)l(ortran)f(77)75 614 y(functionalit)o(y)l(.)1875 670 y Fm(>)p Fl(\(June\))-1881 b Fk(The)25 b(follo)o(wing)g(table)g (summarizes)g(what)f(return)g(v)m(alues)i(mean)e(for)g Fg(operator==\(\))f Fk(\(and)75 727 y Fg(operator!=\(\))p Fk(\).)38 b(The)23 b(ob)s(jects)e(in)i(the)f(\014rst)g(column)h(only)f (ha)o(v)o(e)g(t)o(w)o(o)e(p)q(ossible)k(outcomes)e(and)75 783 y(therefore)17 b(a)g(b)q(o)q(olean)h(result)f(is)h(su\016cien)o(t.) 26 b(The)18 b(ob)s(jects)e(in)i(the)f(second)h(column)g(return)f Fg(true)g Fk(for)75 840 y Fg(operator==\(\))g Fk(if)j(the)f(ob)s(jects) g(compared)g(are)g Fg(MPI)p 1034 840 V 17 w(IDENT)p Fk(,)f Fg(false)g Fk(otherwise.)32 b(F)l(or)19 b(all)h(ob)s(jects,)75 896 y Fg(operator!=\(\))14 b Fk(returns)h(the)g(opp)q(osite)h(of)e Fg(operator==\(\))p Fk(.)p 543 962 865 2 v 542 1018 2 57 v 568 1001 a(Standard)h(b)q(o)q(olean)p 943 1018 V 50 w(Chec)o(k)h(for)e Fg(MPI)p 1249 1001 15 2 v 17 w(IDENT)p 1407 1018 2 57 v 543 1020 865 2 v 542 1076 2 57 v 603 1059 a(MPI)p 678 1059 15 2 v 17 w(Datatype)p 943 1076 2 57 v 199 w(MPI)p 1158 1059 15 2 v 17 w(Comm)p 1407 1076 2 57 v 542 1133 V 675 1116 a(MPI)p 750 1116 15 2 v 17 w(Op)p 943 1133 2 57 v 259 w(MPI)p 1146 1116 15 2 v 17 w(Group)p 1407 1133 2 57 v 542 1189 V 627 1172 a(MPI)p 702 1172 15 2 v 17 w(Status)p 943 1189 2 57 v 1407 1189 V 542 1246 V 615 1229 a(MPI)p 690 1229 15 2 v 17 w(Request)p 943 1246 2 57 v 1407 1246 V 543 1247 865 2 v 75 1347 a Fe(Constants)48 b Fk(Constan)o(ts)13 b(should)j(b)q(e)f(singleton)h(ob)s(jects)e(and)h(should)g(b)q(e)h (declared)g Fg(const)p Fk(.)i(The)d(only)75 1403 y(exceptions)j(to)d (this)i(rule)h(are)e(the)h(follo)o(wing)g(ob)s(jects,)f(whic)o(h)h(do)g (not)f(need)h(to)f(b)q(e)h(declared)h Fg(const)p Fk(:)75 1460 y Fg(MPI)p 150 1460 15 2 v 17 w(BOTTOM)p Fk(,)c Fg(MPI)p 410 1460 V 17 w(COMM)p 523 1460 V 16 w(WORLD)p Fk(,)g(and)i Fg(MPI)p 847 1460 V 16 w(COMM)p 959 1460 V 17 w(SELF)p Fk(.)189 1566 y Fi(R)n(ationale.)62 b Fk(The)19 b(three)g(ob)s(jects)f(men)o(tioned)i(ab)q(o)o(v)o(e)f(do)f(not)h(need) h(to)e(b)q(e)i(declared)g Fg(const)189 1622 y Fk(b)q(ecause)14 b(some)e(implemen)o(tations)i(ma)o(y)e(need)i(to)f(c)o(hange)g(the)g (in)o(ternal)g(state)f(of)h(these)g(ob)s(jects,)189 1679 y(particularly)j(in)g(the)f(case)h(of)e(dynamic)i(pro)q(cesses.)21 b(\()p Fi(End)15 b(of)i(r)n(ationale.)p Fk(\))166 1868 y Fl(Discussion:)j Fc(This)c(matter)f(ma)o(y)f(still)h(b)q(e)i(up)f (for)g(debate.)25 b(If)16 b(implem)o(en)o(table,)e Fb(const)h Fc(constan)o(ts)i(are)75 1924 y(extremely)d(useful)g(for)f (optimization)e(and)j(seman)o(tic)f(purp)q(oses.)75 2129 y Fj(7.1.4)49 b(C++)16 b(Datat)o(yp)q(es)75 2214 y Fk(In)g(Section)g (A.2,)e(sev)o(eral)i(new)f(datat)o(yp)q(es)g(are)f(in)o(tro)q(duced)j (for)d(the)i(C++)f(bindings:)513 2317 y Fg(MPI)p 588 2317 V 17 w(BOOL)210 b(MPI)p 983 2317 V 17 w(DOUBLE)p 1144 2317 V 16 w(COMPLEX)513 2373 y(MPI)p 588 2373 V 17 w(LONG)p 701 2373 V 17 w(DOUBLE)49 b(MPI)p 983 2373 V 17 w(LONG)p 1096 2373 V 16 w(DOUBLE)p 1256 2373 V 17 w(COMPLEX)513 2430 y(MPI)p 588 2430 V 17 w(COMPLEX)166 2532 y Fk(These)20 b(datat)o(yp)q(es)f(are)g(included)j(b)q(ecause)f (the)e(ANSI)i(C++)f(standard)f(de\014nes)i(these)e(t)o(yp)q(es;)75 2589 y(C++)k(compilers)g(can)g(b)q(e)g(exp)q(ected)g(to)f(supp)q(ort)g (them)g(in)h(a)f(standard)g(manner.)41 b(Ho)o(w)o(ev)o(er,)23 b(in)75 2645 y(order)14 b(to)f(b)q(e)h(consisten)o(t)g(with)h(the)f(F)l (ortran)e(treatmen)o(t)h(of)h(comparable)g(datat)o(yp)q(es,)f(not)g (all)i(of)f(these)75 2701 y(datat)o(yp)q(es)h(are)h(not)f(eligible)k (for)d(all)g(t)o(yp)q(es)g(of)g(reduction)h(op)q(erations.)22 b(The)16 b(tables)g(in)h(section)g(4.9.2)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 6 6 5 bop 75 -100 a Fk(6)315 b Fh(CHAPTER)15 b(7.)35 b(MPI)15 b(1.1)f(C++)i(/)f(F)o(OR)l(TRAN)h(90)e(LANGUA)o(GE)h(BINDINGS)75 49 y Fk(of)f(the)g(MPI)g(1.1)g(do)q(cumen)o(t)g(sp)q(ecify)i(whic)o(h)f (datat)o(yp)q(es)e(are)h(v)m(alid)i(for)e(whic)o(h)h(reduction)g(op)q (erations;)75 106 y(the)g(follo)o(wing)h(table)g(sp)q(eci\014es)h(whic) o(h)f(C++)g(datat)o(yp)q(es)e(are)h(v)m(alid)i(with)e(reduction)i(op)q (erations:)191 208 y(Op)574 b(Allo)o(w)o(ed)16 b(T)o(yp)q(es)191 264 y Fg(MPI)p 266 264 15 2 v 17 w(SUM)p Fk(,)e Fg(MPI)p 454 264 V 17 w(PROD)261 b(MPI)p 900 264 V 17 w(DOUBLE)p 1061 264 V 16 w(COMPLEX)p Fk(,)14 b Fg(MPI)p 1344 264 V 17 w(LONG)p 1457 264 V 16 w(DOUBLE)p 1617 264 V 17 w(COMPLEX)191 321 y(MPI)p 266 321 V 17 w(LAND)p Fk(,)g Fg(MPI)p 478 321 V 17 w(LOR)p Fk(,)g Fg(MPI)p 666 321 V 17 w(LXOR)49 b(MPI)p 900 321 V 17 w(BOOL)75 440 y Fj(7.1.5)g (Inter-language)17 b(Op)q(erabilit)o(y)75 602 y Fl(Discussion:)47 b Fc(In)o(ter-language)17 b(op)q(erabilit)o(y)f(is)h(problematic)f(b)q (ecause)j(of)d(name)g(clashing)h(issues.)29 b(F)m(or)17 b(in-)75 652 y(stance,)e(supp)q(ose)h(w)o(e)f(wish)f(to)g(call)g(a)g(C) g(library)g(function)g(from)f(our)h(C++)h(MPI)g(program.)j(The)d(C)f (function)75 702 y(migh)o(t)e(b)q(e)i(protot)o(yp)q(ed)h(\(sa)o(y)m(,)e (in)g(the)i(C)e(library)h(header)g(\014les\))h(as)75 764 y Fb(void)21 b(foo\(MPI)p 341 764 14 2 v 14 w(Comm)g(comm,)g(int)g (x,)g(int)h(y\);)75 826 y Fc(Ev)o(en)15 b(if)e(there)j(is)e(some)f (easy)i(w)o(a)o(y)e(extract)j(the)e(C)g(handle)h(from)d(the)j(C++)g(ob) r(ject,)f(there)i(is)e(no)g(w)o(a)o(y)f(to)h(pass)75 876 y(that)i(handle)f(to)h(the)h(function)e Fb(foo\(\))g Fc(b)q(ecause)i(the)g(\014rst)f(argumen)o(t)f(to)g Fb(foo\(\))g Fc(m)o(ust)g(b)q(e)h(of)f(t)o(yp)q(e)i Fb(MPI)p 1776 876 V 15 w(Comm)75 926 y Fc(|)c(whic)o(h)h(the)h(compiler)d(will)g (require)j(to)f(b)q(e)g(an)g(ob)r(ject,)g(not)g(a)g(C)g(handle.)j(In)d (other)h(w)o(ords,)e(a)h(call)75 988 y Fb(MPI)p 144 988 V 15 w(Comm)21 b(a;)75 1038 y(foo\(a.C)p 232 1038 V 14 w(handle\(\),)f(x,)i(y\);)75 1100 y Fc(cannot)14 b(w)o(ork)g(\(at)g (least)g(not)f(without)h(generating)g(compiler)e(w)o(arnings/errors\).) 166 1157 y(A)17 b(w)o(ork)n(around)e(is)i(to)f(pro)o(vide)h(a)f(C)g (handle)h(t)o(yp)q(e)g(corresp)q(onding)g(to)g(the)g(\\real")f(C)h (handle)f(t)o(yp)q(e)h(and)75 1213 y(then)e(to)e(rewrite)i(the)g(C)f (library)f(header)i(\014le)e(to)h(use)h(that)f(t)o(yp)q(e)g(in)g(its)g (function)f(protot)o(yp)q(es.)75 1418 y Fj(7.1.6)49 b(Pro\014ling)75 1503 y Fk(The)13 b(C++)g(bindings)h(pro)o(vide)f(m)o(ultiple)i(en)o (try)d(p)q(oin)o(ts)h(for)f(MPI)h(functions)g(b)o(y)f(de\014ning)j(an)d (alternate)75 1560 y(set)k(of)f(classes)h(ha)o(ving)h Fg(PMPI)p 591 1560 15 2 v 32 w Fk(pre\014x.)22 b(Alternate)16 b(en)o(try)g(p)q(oin)o(ts)g(for)g(globally-scop)q(ed)i(functions)e(are) 75 1616 y(similarly)23 b(pro)o(vided)f(b)o(y)f(an)g(alternate)g(set)g (of)g(globally)i(scop)q(ed)f(functions)g(ha)o(ving)f Fg(PMPI)p 1715 1616 V 38 w Fk(pre\014x.)75 1673 y(Pro\014ling)15 b(co)q(de)g(can)f(b)q(e)h(inserted)g(b)o(y)f(deriving)i(pro\014ling)f (classes)g(from)f(the)g Fg(PMPI)p 1519 1673 V 31 w Fk(classes)g(and)h (o)o(v)o(er-)75 1729 y(riding)h(mem)o(b)q(er)g(functions)g(as)f (desired.)189 1835 y Fi(A)n(dvic)n(e)j(to)i(implementors.)63 b Fk(Pro)o(viding)19 b(t)o(w)o(o)f(en)o(try)g(p)q(oin)o(ts)i(can)f(b)q (e)h(done)f(almost)f(trivially)189 1892 y(using)e(inheritance.)284 2004 y Fg(class)23 b(MPI_Comm)g(:)h(public)f(PMPI_Comm)f({)284 2061 y(//)i(constructors,)e(destructor,)g(assignment)h(are)g(not)h (inherited)284 2117 y(})189 2230 y Fk(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fk(\))-160 2291 y Fl(\(June\))p Fm(?)75 2423 y Fn(7.2)59 b(F)n(ORTRAN)20 b(90)g(Intro)r(duction)75 2524 y Fk(Still)d(needs)f(to)f(b)q(e)g (added.)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 7 7 6 bop 75 381 a Fo(Bibliograph)m(y)75 604 y Fk([1])22 b(Margaret)13 b(A.)h(Ellis)j(and)d(Bjarne)h(Stroustrup.)k Fi(The)c(A)o(nnotate)n(d)g(C++)g(R)n(efer)n(enc)n(e)g(Manual)p Fk(.)k(Ad-)146 660 y(dison)d(W)l(esley)l(,)g(1990.)75 754 y([2])22 b(C++)16 b(F)l(orum.)j(W)l(orking)d(pap)q(er)g(for)e (draft)h(prop)q(osed)h(in)o(ternational)g(standard)f(for)g(information) 146 811 y(systems)e({)g(programming)g(language)h(c++.)k(T)l(ec)o (hnical)d(rep)q(ort,)f(American)g(National)g(Standards)146 867 y(Institute,)h(1995.)75 961 y([3])22 b(Message)15 b(P)o(assing)g(In)o(terface)g(F)l(orum.)21 b(Do)q(cumen)o(t)15 b(for)g(a)g(standard)g(message-passing)h(in)o(terface.)146 1017 y(T)l(ec)o(hnical)21 b(Rep)q(ort)e(T)l(ec)o(hnical)i(Rep)q(ort)e (No.)g(CS-93-214)f(\(revised\),)i(Univ)o(ersit)o(y)g(of)e(T)l (ennessee,)146 1074 y(April)e(1994.)j(Av)m(ailable)e(on)f Ff(netlib)p Fk(.)964 2828 y(7)p eop %%Page: 8 8 8 7 bop 75 361 a Fp(Annex)32 b(A)75 568 y Fo(MPI)40 b(1.1)f(C++)g (Language)g(Binding)75 809 y Fn(A.1)59 b(C++)19 b(Classes)75 910 y Fk(The)c(follo)o(wing)h(are)f(the)h(classes)f(pro)o(vided)h(with) g(the)f(C++)h(language)f(bindings:)75 1002 y Fg(class)23 b(MPI_Comm;)75 1059 y(class)g(MPI_Datatype;)-160 1154 y Fl(\(June\))p Fm(>)75 1198 y Fg(class)g(MPI_Errorhandler;)-160 1251 y Fl(\(June\))p Fm(?)75 1349 y Fg(class)g(MPI_Group;)75 1405 y(class)g(MPI_Op;)75 1461 y(class)g(MPI_Request;)75 1518 y(class)g(MPI_Status;)166 1622 y Fk(Note)15 b(that)f Fg(MPI)p 449 1622 15 2 v 17 w(Status)g Fk(has)h(public)j(data)c(mem)o (b)q(ers,)h(as)g(sp)q(eci\014ed)i(in)f(the)g(C)f(bindings.)75 1764 y Fn(A.2)59 b(De\014ned)20 b(Constants)75 1866 y Fk(These)g(are)g(required)h(constan)o(ts,)f(de\014ned)i(in)f(the)f (\014le)h Fg(mpi++.h)p Fk(.)33 b(F)l(or)20 b(brevit)o(y)l(,)h(the)f(t)o (yp)q(es)g(of)g(the)75 1922 y(constan)o(ts)c(are)h(de\014ned)i(b)q(elo) o(w)f(are)f(de\014ned)i(in)f(the)g(commen)o(ts.)25 b(All)19 b(constan)o(ts)d(are)i(also)f(assumed)-160 1972 y Fl(\(June\))p Fm(>)75 1979 y Fk(to)e(b)q(e)g Fg(const)g Fk(\(except)g(where)h(sp)q (eci\014cally)i(noted\).)-160 2034 y Fl(\(June\))p Fm(?)75 2083 y Fg(//)24 b(return)f(codes)75 2139 y(//)h(Type:)f(int)g(\(or)h (enum\))75 2195 y(MPI_SUCCESS)75 2252 y(MPI_ERR_BUFFER)75 2308 y(MPI_ERR_COUNT)75 2365 y(MPI_ERR_TYPE)75 2421 y(MPI_ERR_TAG)75 2478 y(MPI_ERR_COMM)75 2534 y(MPI_ERR_RANK)75 2591 y(MPI_ERR_REQUEST)75 2647 y(MPI_ERR_ROOT)75 2704 y(MPI_ERR_GROUP)964 2828 y Fk(8)p eop %%Page: 9 9 9 8 bop 75 -100 a Fh(A.2.)29 b(DEFINED)15 b(CONST)l(ANTS)1156 b Fk(9)75 49 y Fg(MPI_ERR_OP)75 106 y(MPI_ERR_TOPOLOGY)75 162 y(MPI_ERR_DIMS)75 219 y(MPI_ERR_ARG)75 275 y(MPI_ERR_UNKNOWN)75 332 y(MPI_ERR_TRUNCATE)75 388 y(MPI_ERR_OTHER)75 444 y(MPI_ERR_INTERN)75 501 y(MPI_ERR_PENDING)75 557 y(MPI_ERR_LASTCODE)75 670 y(//)24 b(assorted)e(constants)1875 780 y Fm(>)p Fl(\(June\))75 826 y Fg(//)i(Type:)f(MPI_Aint)75 882 y(//)h(MPI_BOTTOM)e(is)i(not)f(required)g(to)g(be)h(const)1875 942 y Fm(?)p Fl(\(June\))75 1038 y Fg(MPI_BOTTOM)75 1094 y(//)g(Type:)f(int)g(\(or)h(enum\))75 1151 y(MPI_PROC_NULL)75 1207 y(MPI_ANY_SOURCE)75 1264 y(MPI_ANY_TAG)75 1320 y(MPI_UNDEFINED) 1875 1430 y Fm(>)p Fl(\(June\))75 1475 y Fg(//)g(Type:)f(MPI_Datatype) 1875 1535 y Fm(?)p Fl(\(June\))75 1631 y Fg(MPI_UB)75 1687 y(MPI_LB)75 1800 y(//)h(Error-handling)e(specifiers)75 1857 y(//)i(Type:)f(int)g(\(or)h(enum\))75 1913 y(MPI_ERRORS_ARE_FATAL) 75 1970 y(MPI_ERRORS_RETURN)75 2083 y(//)g(Maximum)e(sizes)i(for)f (strings)75 2139 y(//)h(Type:)f(int)75 2195 y(MPI_MAX_PROCESSOR_NAME)75 2252 y(MPI_MAX_ERROR_STRING)75 2365 y(//)h(elementary)e(datatypes)75 2421 y(//)i(Type:)f(MPI_Datatype)75 2478 y(MPI_CHAR)75 2534 y(MPI_SHORT)75 2591 y(MPI_INT)75 2647 y(MPI_LONG)75 2704 y(MPI_UNSIGNED_CHAR)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 10 10 9 bop 75 -100 a Fk(10)714 b Fh(ANNEX)15 b(A.)30 b(MPI)16 b(1.1)e(C++)h(LANGUA)o(GE)g(BINDING)75 49 y Fg(MPI_UNSIGNED_SHORT)75 106 y(MPI_UNSIGNED)75 162 y(MPI_UNSIGNED_LONG)75 219 y(MPI_FLOAT)75 275 y(MPI_DOUBLE)75 332 y(MPI_LONG_DOUBLE)75 388 y(MPI_BYTE)75 444 y(MPI_PACKED)75 557 y(//)24 b(datatypes)e(for)i (reduction)e(functions)75 614 y(//)i(Type:)f(MPI_Datatype)75 670 y(MPI_FLOAT_INT)75 727 y(MPI_DOUBLE_INT)75 783 y(MPI_LONG_INT)75 840 y(MPI_2INT)75 896 y(MPI_SHORT_INT)75 953 y(MPI_LONG_DOUBLE_INT)75 1065 y(//)h(optional)e(datatypes)75 1122 y(//)i(Type:)f(MPI_Datatype)75 1178 y(MPI_LONG_LONG_INT)75 1291 y(//)h(optional)e(datatypes)75 1348 y(//)i(Type:)f(MPI_Datatype)75 1404 y(MPI_BOOL)75 1461 y(MPI_LONG_DOUBLE)75 1517 y(MPI_COMPLEX)75 1574 y(MPI_DOUBLE_COMPLEX)75 1630 y(MPI_LONG_DOUBLE_COMPLEX)75 1743 y(//)h(reserved)e(communicators)75 1799 y(//)i(Type:)f(MPI_Comm) -160 1902 y Fl(\(June\))p Fm(>)75 1941 y Fg(//)h(MPI_COMM_WORLD)e(and)h (MPI_COMM_SELF)f(are)i(not)f(required)g(to)g(be)h(const)-160 1994 y Fl(\(June\))p Fm(?)75 2083 y Fg(MPI_COMM_WORLD)75 2139 y(MPI_COMM_SELF)75 2252 y(//)g(results)e(of)i(communicator)e(and)i (group)f(comparisons)75 2308 y(//)h(Type:)f(int)g(\(or)h(enum\))75 2365 y(MPI_IDENT)75 2421 y(MPI_CONGRUENT)75 2478 y(MPI_SIMILAR)75 2534 y(MPI_UNEQUAL)75 2647 y(//)g(environmental)e(inquiry)h(keys)75 2704 y(//)h(Type:)f(int)g(\(or)h(enum\))1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 11 11 10 bop 75 -100 a Fh(A.3.)29 b(TYPEDEFS)1393 b Fk(11)75 49 y Fg(MPI_TAG_UB)75 106 y(MPI_IO)75 162 y(MPI_HOST)75 275 y(//)24 b(collective)e(operations)75 332 y(//)i(Type:)f(MPI_Op)75 388 y(MPI_MAX)75 444 y(MPI_MIN)75 501 y(MPI_SUM)75 557 y(MPI_PROD)75 614 y(MPI_MAXLOC)75 670 y(MPI_MINLOC)75 727 y(MPI_BAND)75 783 y(MPI_BOR)75 840 y(MPI_BXOR)75 896 y(MPI_LAND)75 953 y(MPI_LOR)75 1009 y(MPI_LXOR)75 1122 y(//)h(Null)f(handles)75 1178 y(//)h(Type:)f(MPI_Group)75 1235 y(MPI_GROUP_NULL)75 1291 y(//)h(Type:)f(MPI_Comm)75 1348 y(MPI_COMM_NULL)75 1404 y(//)h(Type:)f(MPI_Datatype)75 1461 y(MPI_DATATYPE_NULL)75 1517 y(//)h(Type:)f(MPI_Request)75 1574 y(MPI_REQUEST_NULL)75 1630 y(//)h(Type:)f(MPI_Op)75 1686 y(MPI_OP_NULL)75 1743 y(//)h(Type:)f(int)g(\(or)h(enum\))75 1799 y(MPI_ERRHANDLER_NULL)75 1912 y(//)g(Empty)f(group)75 1969 y(//)h(Type:)f(MPI_Group)75 2025 y(MPI_GROUP_EMPTY)75 2138 y(//)h(topologies)75 2195 y(//)g(Type:)f(int)g(\(or)h(enum\))75 2251 y(MPI_GRAPH)75 2307 y(MPI_CART)75 2506 y Fn(A.3)59 b(T)-5 b(yp)r(edefs)75 2607 y Fk(The)15 b(follo)o(wing)h(are)f (de\014ned)i(C++)e(t)o(yp)q(es,)g(also)g(included)j(in)e(the)g(\014le)g Fg(mpi++.h)p Fk(.)75 2704 y Fg(//)24 b(opaque)f(types)-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 12 12 11 bop 75 -100 a Fk(12)714 b Fh(ANNEX)15 b(A.)30 b(MPI)16 b(1.1)e(C++)h(LANGUA)o(GE)g(BINDING)75 49 y Fg(MPI_Aint)75 162 y(//)24 b(prototypes)e(for)i(user-defined)e(functions)75 219 y(typedef)h(int)g(MPI_Copy_function\(MPI_Comm)e(&oldcomm,)h (MPI_Comm)h(&newcomm,)791 275 y(int)g(&keyval,)g(void)g (*extra_state\);)75 388 y(typedef)g(int)g (MPI_Delete_function\(MPI_Comm)d(&comm,)j(int)h(&keyval,)839 444 y(void)f(*extra_state\);)75 557 y(typedef)g(void)g (MPI_Handler_function\(MPI_Com)o(m)e(&,)j(int)f(&,)h(...\);)75 670 y(typedef)f(void)g(MPI_User_function\(void)e(invec[],)i(void)g (inoutvec[],)g(int)g(&len,)815 727 y(MPI_Datatype)f(&datatype\);)75 870 y Fn(A.4)59 b(C++)19 b(Bindings)g(fo)n(r)h(P)n(oint-to-P)n(oint)h (Communication)75 971 y Fg(int)i(MPI)p 245 971 15 2 v 17 w(Comm::Send\(const)f(void*)h(buf,)g(int)h(count,)f(const)393 1028 y(MPI)p 468 1028 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int) g(tag\))75 1114 y(int)g(MPI)p 245 1114 V 17 w(Comm::Recv\(void*)f(buf,) h(int)h(count,)f(const)g(MPI)p 1265 1114 V 17 w(Datatype&)f(datatype,) 393 1171 y(int)i(source,)f(int)g(tag,)g(MPI)p 969 1171 V 17 w(Status&)g(status\))75 1257 y(int)g(MPI)p 245 1257 V 17 w(Comm::Bsend\(const)f(void*)h(buf,)g(int)h(count,)f(const)393 1314 y(MPI)p 468 1314 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int) g(tag\))75 1400 y(int)g(MPI)p 245 1400 V 17 w(Comm::Ssend\(const)f (void*)h(buf,)g(int)h(count,)f(const)393 1457 y(MPI)p 468 1457 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag\))75 1543 y(int)g(MPI)p 245 1543 V 17 w(Comm::Rsend\(const)f(void*)h(buf,)g (int)h(count,)f(const)393 1599 y(MPI)p 468 1599 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag\))75 1686 y(int)g(MPI)p 245 1686 V 17 w(Comm::Isend\(const)f(void*)h(buf,)g (int)h(count,)f(const)393 1742 y(MPI)p 468 1742 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag,)393 1799 y(MPI)p 468 1799 V 17 w(Request&)g(request\))75 1885 y(int)g(MPI)p 245 1885 V 17 w(Comm::Ibsend\(const)f(void*)h(buf,)g (int)h(count,)f(const)393 1942 y(MPI)p 468 1942 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag,)393 1998 y(MPI)p 468 1998 V 17 w(Request&)g(request\))75 2085 y(int)g(MPI)p 245 2085 V 17 w(Comm::Issend\(const)f(void*)h(buf,)g (int)h(count,)f(const)393 2141 y(MPI)p 468 2141 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag,)393 2198 y(MPI)p 468 2198 V 17 w(Request&)g(request\))75 2284 y(int)g(MPI)p 245 2284 V 17 w(Comm::Irsend\(const)f(void*)h(buf,)g (int)h(count,)f(const)393 2341 y(MPI)p 468 2341 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g(tag,)393 2397 y(MPI)p 468 2397 V 17 w(Request&)g(request\))75 2483 y(int)g(MPI)p 245 2483 V 17 w(Comm::Irecv\(void*)f(buf,)h(int)h (count,)f(const)g(MPI)p 1289 2483 V 17 w(Datatype&)f(datatype,)393 2540 y(int)i(source,)f(int)g(tag,)g(MPI)p 969 2540 V 17 w(Request&)g(request\))75 2626 y(int)g(MPI)p 245 2626 V 17 w(Comm::Send)p 502 2626 V 16 w(init\(const)g(void*)g(buf,)g(int)g (count,)h(const)393 2683 y(MPI)p 468 2683 V 17 w(Datatype&)f(datatype,) f(int)i(dest,)f(int)g(tag,)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 13 13 12 bop 75 -100 a Fh(A.4.)34 b(C++)16 b(BINDINGS)g(F)o(OR)f (POINT-TO-POINT)i(COMMUNICA)l(TION)333 b Fk(13)393 49 y Fg(MPI)p 468 49 15 2 v 17 w(Request&)23 b(request\))75 136 y(int)g(MPI)p 245 136 V 17 w(Comm::Bsend)p 526 136 V 16 w(init\(const)f(void*)i(buf,)f(int)g(count,)g(const)393 192 y(MPI)p 468 192 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g (tag,)393 249 y(MPI)p 468 249 V 17 w(Request&)g(request\))75 335 y(int)g(MPI)p 245 335 V 17 w(Comm::Ssend)p 526 335 V 16 w(init\(const)f(void*)i(buf,)f(int)g(count,)g(const)393 392 y(MPI)p 468 392 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g (tag,)393 448 y(MPI)p 468 448 V 17 w(Request&)g(request\))75 534 y(int)g(MPI)p 245 534 V 17 w(Comm::Rsend)p 526 534 V 16 w(init\(const)f(void*)i(buf,)f(int)g(count,)g(const)393 591 y(MPI)p 468 591 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int)g (tag,)393 647 y(MPI)p 468 647 V 17 w(Request&)g(request\))75 734 y(int)g(MPI)p 245 734 V 17 w(Comm::Recv)p 502 734 V 16 w(init\(void*)g(buf,)g(int)g(count,)g(const)h(MPI)p 1378 734 V 16 w(Datatype&)f(datatype,)393 790 y(int)h(source,)f(int)g (tag,)g(MPI)p 969 790 V 17 w(Request&)g(request\))75 877 y(int)g(MPI)p 245 877 V 17 w(Comm::Iprobe\(int)f(source,)h(int)g (tag,)h(int&)f(flag,)g(MPI)p 1408 877 V 17 w(Status&)g(status\))75 963 y(int)g(MPI)p 245 963 V 17 w(Comm::Probe\(int)f(source,)h(int)g (tag,)h(MPI)p 1122 963 V 16 w(Status&)f(status\))75 1050 y(int)g(MPI)p 245 1050 V 17 w(Comm::Sendrecv\(const)e(void)j(*sendbuf,) e(int)i(sendcount,)e(const)393 1106 y(MPI)p 468 1106 V 17 w(Datatype&)h(sendtype,)f(int)i(dest,)f(int)g(sendtag,)g(void)g (*recvbuf,)393 1163 y(int)h(recvcount,)e(const)h(MPI)p 969 1163 V 17 w(Datatype&)g(recvtype,)f(int)i(source,)393 1219 y(int)g(recvtag,)e(MPI)p 778 1219 V 17 w(Status&)h(status\))75 1305 y(int)g(MPI)p 245 1305 V 17 w(Comm::Sendrecv)p 598 1305 V 15 w(replace\(void*)g(buf,)g(int)g(count,)g(const)393 1362 y(MPI)p 468 1362 V 17 w(Datatype&)g(datatype,)f(int)i(dest,)f(int) g(sendtag,)g(int)h(source,)393 1418 y(int)g(recvtag,)e(MPI)p 778 1418 V 17 w(Status&)h(status\))75 1505 y(int)g(MPI)p 245 1505 V 17 w(Status::Get)p 526 1505 V 16 w(count\(const)f(MPI)p 900 1505 V 17 w(Datatype&)h(datatype,)f(int&)i(count\))f(const)1875 1601 y Fm(>)p Fl(\(June\))75 1591 y Fg(int)g(MPI)p 245 1591 V 17 w(Status::Test)p 550 1591 V 16 w(cancelled\(int&)f(flag\)) 1875 1664 y Fm(?)p Fl(\(June\))75 1678 y Fg(int)h(MPI)p 245 1678 V 17 w(Request::Wait\(MPI)p 670 1678 V 15 w(Status&)g (status\))1875 1774 y Fm(>)p Fl(\(June\))75 1764 y Fg(int)g(MPI)p 245 1764 V 17 w(Request::Test\(int&)f(flag,)h(MPI)p 931 1764 V 17 w(Status&)f(status\))1875 1837 y Fm(?)p Fl(\(June\))75 1851 y Fg(int)h(MPI)p 245 1851 V 17 w(Request::Request)p 646 1851 V 15 w(free\(void\))75 1937 y(int)g(MPI)p 245 1937 V 17 w(Request::Cancel\(void\))75 2024 y(int)g(MPI)p 245 2024 V 17 w(Request::Start\(void\))75 2110 y(int)g(::MPI)p 293 2110 V 17 w(Buffer)p 454 2110 V 16 w(attach\(void*)g(buffer,)f(int) i(size\))1875 2203 y Fm(>)p Fl(\(June\))75 2196 y Fg(int)f(::MPI)p 293 2196 V 17 w(Buffer)p 454 2196 V 16 w(detach\(void*&)f(buffer,)h (int&)h(size\))1875 2265 y Fm(?)p Fl(\(June\))75 2283 y Fg(int)f(::MPI)p 293 2283 V 17 w(Waitany\(int)f(count,)h(MPI)p 835 2283 V 17 w(Request)g(array)p 1163 2283 V 17 w(of)p 1228 2283 V 16 w(requests[],)g(int&)g(index,)393 2339 y(MPI)p 468 2339 V 17 w(Status&)g(status\))75 2426 y(int)g(::MPI)p 293 2426 V 17 w(Testany\(int)f(count,)h(MPI)p 835 2426 V 17 w(Request)g(array)p 1163 2426 V 17 w(of)p 1228 2426 V 16 w(requests[],)g(int&)g(index,)393 2482 y(int&)g(flag,)h(MPI)p 731 2482 V 16 w(Status&)f(status\))75 2569 y(int)g(::MPI)p 293 2569 V 17 w(Waitall\(int)f(count,)h(MPI)p 835 2569 V 17 w(Request)g(array)p 1163 2569 V 17 w(of)p 1228 2569 V 16 w(requests[],)393 2625 y(MPI)p 468 2625 V 17 w(Status)g(array)p 772 2625 V 16 w(of)p 836 2625 V 17 w(statuses[]\))-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 14 14 13 bop 75 -100 a Fk(14)714 b Fh(ANNEX)15 b(A.)30 b(MPI)16 b(1.1)e(C++)h(LANGUA)o(GE)g(BINDING)75 49 y Fg(int)23 b(::MPI)p 293 49 15 2 v 17 w(Testall\(int)f(count,)h(MPI)p 835 49 V 17 w(Request)g(array)p 1163 49 V 17 w(of)p 1228 49 V 16 w(requests[],)g(int&)g(flag,)393 106 y(MPI)p 468 106 V 17 w(Status)g(array)p 772 106 V 16 w(of)p 836 106 V 17 w(statuses[]\))-160 196 y Fl(\(June\))p Fm(>)75 192 y Fg(int)g(::MPI)p 293 192 V 17 w(Waitsome\(int)f(incount,)h(MPI)p 907 192 V 17 w(Request)g(array)p 1235 192 V 16 w(of)p 1299 192 V 17 w(requests[],)393 249 y(int&)g(outcount,)g(int)h(array)p 970 249 V 16 w(of)p 1034 249 V 17 w(indices[],)393 305 y(MPI)p 468 305 V 17 w(Status)f(array)p 772 305 V 16 w(of)p 836 305 V 17 w(statuses[]\))-160 345 y Fl(\(June\))p Fm(?)75 392 y Fg(int)g(::MPI)p 293 392 V 17 w(Testsome\(int)f(incount,) h(MPI)p 907 392 V 17 w(Request)g(array)p 1235 392 V 16 w(of)p 1299 392 V 17 w(requests[],)393 448 y(int&)g(outcount,)g(int)h (array)p 970 448 V 16 w(of)p 1034 448 V 17 w(indices[],)393 504 y(MPI)p 468 504 V 17 w(Status)f(array)p 772 504 V 16 w(of)p 836 504 V 17 w(statuses[]\))75 591 y(int)g(::MPI)p 293 591 V 17 w(Startall\(int)f(count,)h(MPI)p 859 591 V 17 w(Request)g(array)p 1187 591 V 16 w(of)p 1251 591 V 17 w(requests[]\))75 677 y(int)g(MPI)p 245 677 V 17 w(Datatype::Type)p 598 677 V 15 w(contiguous\(int)f(count,)h(const)h (MPI)p 1354 677 V 16 w(Datatype&)f(type\))75 764 y(int)g(MPI)p 245 764 V 17 w(Datatype::Type)p 598 764 V 15 w(vector\(int)g(count,)g (int)g(blocklength,)g(int)g(stride,)g(const)393 820 y(MPI)p 468 820 V 17 w(Datatype&)g(type\))75 907 y(int)g(MPI)p 245 907 V 17 w(Datatype::Type)p 598 907 V 15 w(hvector\(int)g(count,)g (int)g(blocklength,)g(MPI)p 1545 907 V 16 w(Aint)h(stride,)393 963 y(const)f(MPI)p 611 963 V 17 w(Datatype&)g(type\))75 1050 y(int)g(MPI)p 245 1050 V 17 w(Datatype::Type)p 598 1050 V 15 w(indexed\(int)g(count,)g(int)g(array)p 1282 1050 V 17 w(of)p 1347 1050 V 17 w(blocklengths[],)393 1106 y(int)h(array)p 612 1106 V 16 w(of)p 676 1106 V 17 w(displacements[],)e(const)h(MPI)p 1314 1106 V 17 w(Datatype&)f(type\))75 1193 y(int)h(MPI)p 245 1193 V 17 w(Datatype::Type)p 598 1193 V 15 w(hindexed\(int)g(count,)g(int)g (array)p 1306 1193 V 17 w(of)p 1371 1193 V 17 w(blocklengths[],)393 1249 y(MPI)p 468 1249 V 17 w(Aint)g(array)p 724 1249 V 17 w(of)p 789 1249 V 17 w(displacements[],)e(const)j(MPI)p 1427 1249 V 16 w(Datatype&)f(type\))75 1335 y(int)g(MPI)p 245 1335 V 17 w(Datatype::MPI)p 574 1335 V 16 w(Type)p 686 1335 V 16 w(struct\(int)g(count,)g(int)g(array)p 1347 1335 V 17 w(of)p 1412 1335 V 17 w(blocklengths[],)393 1392 y(MPI)p 468 1392 V 17 w(Aint)g(array)p 724 1392 V 17 w(of)p 789 1392 V 17 w(displacements[],)393 1448 y(MPI)p 468 1448 V 17 w(Datatype)g(array)p 820 1448 V 16 w(of)p 884 1448 V 17 w(types[]\))75 1535 y(int)g(::MPI)p 293 1535 V 17 w(Address\(void*)f(location,)h(MPI)p 955 1535 V 16 w(Aint&)h(address\))75 1621 y(int)f(MPI)p 245 1621 V 17 w(Datatype::Type)p 598 1621 V 15 w(extent\(int&)g(extent\))g (const)75 1708 y(int)g(MPI)p 245 1708 V 17 w(Datatype::Type)p 598 1708 V 15 w(size\(int&)g(size\))g(const)75 1794 y(int)g(MPI)p 245 1794 V 17 w(Datatype::Type)p 598 1794 V 15 w(count\(int&)g(count\)) g(const)75 1881 y(int)g(MPI)p 245 1881 V 17 w(Datatype::Type)p 598 1881 V 15 w(lb\(int&)g(displacement\))f(const)75 1967 y(int)h(MPI)p 245 1967 V 17 w(Datatype::Type)p 598 1967 V 15 w(ub\(int&)g(displacement\))f(const)75 2054 y(int)h(MPI)p 245 2054 V 17 w(Datatype::Type)p 598 2054 V 15 w(commit\(void\))75 2140 y(int)g(MPI)p 245 2140 V 17 w(Datatype::Type)p 598 2140 V 15 w(free\(void\))75 2226 y(int)g(MPI)p 245 2226 V 17 w(Datatype::Get)p 574 2226 V 16 w(elements\(const)f(MPI)p 1020 2226 V 16 w(Status&)h(status,) g(int&)g(count\))h(const)-160 2316 y Fl(\(June\))p Fm(>)75 2313 y Fg(int)f(MPI)p 245 2313 V 17 w(Datatype::Pack\(const)e(void*)j (inbuf,)f(int)g(incount,)g(void)g(*outbuf,)393 2369 y(int)h(outsize,)e (int&)i(position,)e(const)i(MPI)p 1280 2369 V 16 w(Comm)g(&comm\))f (const)75 2456 y(int)g(MPI)p 245 2456 V 17 w(Datatype::Unpack\(const)e (void*)i(inbuf,)g(int)h(insize,)f(int&)g(position,)393 2512 y(void)g(*outbuf,)g(int)h(outcount,)e(const)i(MPI)p 1280 2512 V 16 w(Comm&)f(comm\))h(const)75 2599 y(int)f(MPI)p 245 2599 V 17 w(Datatype::Pack)p 598 2599 V 15 w(size\(int)g(incount,)g (const)g(MPI)p 1258 2599 V 17 w(Datatype&)g(comm,)393 2655 y(int&)g(size\))h(const)-160 2689 y Fl(\(June\))p Fm(?)1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 15 15 14 bop 75 -100 a Fh(A.5.)34 b(C++)16 b(BINDINGS)g(F)o(OR)f (COLLECTIVE)i(COMMUNICA)l(TION)427 b Fk(15)75 49 y Fn(A.5)59 b(C++)19 b(Bindings)g(fo)n(r)h(Collective)g(Communication)75 151 y Fg(int)j(MPI)p 245 151 15 2 v 17 w(Comm::Barrier\(void\))75 237 y(int)g(MPI)p 245 237 V 17 w(Comm::Bcast\(void*)f(buffer,)h(int)g (count,)g(const)g(MPI)p 1360 237 V 17 w(Datatype&)g(datatype,)393 294 y(int)h(root\))75 380 y(int)f(MPI)p 245 380 V 17 w(Comm::Gather\(const)f(void*)h(sendbuf,)g(int)g(sendcount,)g(const)393 437 y(MPI)p 468 437 V 17 w(Datatype&)g(sendtype,)f(void*)h(recvbuf,)g (int)h(recvcount,)e(const)393 493 y(MPI)p 468 493 V 17 w(Datatype&)h(recvtype,)f(int)i(root\))75 579 y(int)f(MPI)p 245 579 V 17 w(Comm::Gatherv\(const)e(void*)j(sendbuf,)e(int)i (sendcount,)e(const)393 636 y(MPI)p 468 636 V 17 w(Datatype&)h (sendtype,)f(void*)h(recvbuf,)g(int)h(recvcounts[],)393 692 y(int)g(displs[],)e(const)h(MPI)p 945 692 V 17 w(Datatype&)g (recvtype,)f(int)i(root\))75 779 y(int)f(MPI)p 245 779 V 17 w(Comm::Scatter\(const)e(void*)j(sendbuf,)e(int)i(sendcount,)e (const)393 835 y(MPI)p 468 835 V 17 w(Datatype&)h(sendtype,)f(void*)h (recvbuf,)g(int)h(recvcount,)e(const)393 892 y(MPI)p 468 892 V 17 w(Datatype&)h(recvtype,)f(int)i(root\))75 978 y(int)f(MPI)p 245 978 V 17 w(Comm::Scatterv\(const)e(void*)j (sendbuf,)e(int)i(sendcounts[],)e(int)h(displs[],)393 1035 y(const)g(MPI)p 611 1035 V 17 w(Datatype&)g(sendtype,)f(void*)i (recvbuf,)e(int)i(recvcount,)393 1091 y(const)f(MPI)p 611 1091 V 17 w(Datatype&)g(recvtype,)f(int)i(root\))75 1178 y(int)f(MPI)p 245 1178 V 17 w(Comm::Allgather\(const)e(void*)i (sendbuf,)g(int)h(sendcount,)e(const)393 1234 y(MPI)p 468 1234 V 17 w(Datatype&)h(sendtype,)f(void*)h(recvbuf,)g(int)h (recvcount,)e(const)393 1290 y(MPI)p 468 1290 V 17 w(Datatype&)h (recvtype\))75 1377 y(int)g(MPI)p 245 1377 V 17 w (Comm::Allgatherv\(const)e(void*)i(sendbuf,)g(int)g(sendcount,)g(const) 393 1433 y(MPI)p 468 1433 V 17 w(Datatype&)g(sendtype,)f(void*)h (recvbuf,)g(int)h(recvcounts[],)393 1490 y(int)g(displs[],)e(const)h (MPI)p 945 1490 V 17 w(Datatype&)g(recvtype\))75 1576 y(int)g(MPI)p 245 1576 V 17 w(Comm::Alltoall\(const)e(void*)j(sendbuf,) e(int)i(sendcount,)e(const)393 1633 y(MPI)p 468 1633 V 17 w(Datatype&)h(sendtype,)f(void*)h(recvbuf,)g(int)h(recvcount,)e (const)393 1689 y(MPI)p 468 1689 V 17 w(Datatype&)h(recvtype\))75 1776 y(int)g(MPI)p 245 1776 V 17 w(Comm::Alltoallv\(const)e(void*)i (sendbuf,)g(int)h(sendcounts[],)393 1832 y(int)g(sdispls[],)e(const)h (MPI)p 969 1832 V 17 w(Datatype&)g(sendtype,)f(void*)i(recvbuf,)393 1889 y(int)g(recvcounts[],)e(int)h(rdispls[],)g(const)g(MPI)p 1399 1889 V 17 w(Datatype&)f(recvtype\))75 1975 y(int)h(MPI)p 245 1975 V 17 w(Comm::Reduce\(const)f(void*)h(sendbuf,)g(void*)g (recvbuf,)g(int)g(count,)g(const)393 2031 y(MPI)p 468 2031 V 17 w(Datatype&)g(datatype,)f(const)h(MPI)p 1177 2031 V 17 w(Op&)h(op,)f(int)h(root\))1875 2128 y Fm(>)p Fl(\(June\))75 2118 y Fg(int)f(MPI)p 245 2118 V 17 w(Op::Create\(const) f(MPI)p 740 2118 V 17 w(User)p 853 2118 V 16 w(function&)h(function,)g (int)g(commute\))75 2204 y(int)g(MPI)p 245 2204 V 17 w(Op::Free\(void\))1875 2244 y Fm(?)p Fl(\(June\))75 2291 y Fg(int)g(MPI)p 245 2291 V 17 w(Comm::Allreduce\(const)e(void*)i (sendbuf,)g(void*)g(recvbuf,)g(int)h(count,)393 2347 y(const)f(MPI)p 611 2347 V 17 w(Datatype&)g(datatype,)f(const)i(MPI)p 1321 2347 V 16 w(Op&)g(op\))75 2434 y(int)f(MPI)p 245 2434 V 17 w(Comm::Reduce)p 550 2434 V 16 w(scatter\(const)f(void*)h (sendbuf,)g(void*)g(recvbuf,)393 2490 y(int)h(recvcounts[],)e(const)h (MPI)p 1041 2490 V 17 w(Datatype&)f(datatype,)h(const)393 2547 y(MPI)p 468 2547 V 17 w(Op&)g(op\))75 2633 y(int)g(MPI)p 245 2633 V 17 w(Comm::Scan\(const)f(void*)h(sendbuf,)g(void*)g (recvbuf,)g(int)g(count,)g(const)393 2690 y(MPI)p 468 2690 V 17 w(Datatype&)g(datatype,)f(const)h(MPI)p 1177 2690 V 17 w(Op&)h(op\))-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 16 16 15 bop 75 -100 a Fk(16)714 b Fh(ANNEX)15 b(A.)30 b(MPI)16 b(1.1)e(C++)h(LANGUA)o(GE)g(BINDING)75 49 y Fn(A.6)59 b(C++)19 b(Bindings)g(fo)n(r)h(Groups,)h(Contexts,)g(and)e(Communicato) n(rs)75 151 y Fg(int)k(MPI)p 245 151 15 2 v 17 w(Group::Size\(int&)f (size\))h(const)75 237 y(int)g(MPI)p 245 237 V 17 w(Group::Rank\(int&)f (rank\))h(const)-160 327 y Fl(\(June\))p Fm(>)75 324 y Fg(int)g(MPI)p 245 324 V 17 w(Group::Translate)p 646 324 V 15 w(ranks)g(\(int)h(n,)f(int)h(ranks1[],)e(const)i(MPI)p 1545 324 V 16 w(Group&)f(group,)393 380 y(int)h(ranks2[]\))e(const)75 467 y(int)h(MPI)p 245 467 V 17 w(Group::Compare\(const)e(MPI)p 835 467 V 17 w(Group&)i(group,)g(int&)h(result\))e(const)75 553 y(int)h(MPI)p 245 553 V 17 w(Comm::Group\(MPI)p 622 553 V 15 w(Group&)g(group\))g(const)75 639 y(int)g(MPI)p 245 639 V 17 w(Group::Union\(const)f(MPI)p 788 639 V 16 w(Group&)h(group1,)g(const)h(MPI)p 1378 639 V 16 w(Group&)f (group2\))75 726 y(int)g(MPI)p 245 726 V 17 w (Group::Intersection\(const)e(MPI)p 955 726 V 16 w(Group&)i(group1,)g (const)393 782 y(MPI)p 468 782 V 17 w(Group&)g(group2\))75 869 y(int)g(MPI)p 245 869 V 17 w(Group::Difference\(const)e(MPI)p 907 869 V 17 w(Group&)i(group1,)g(const)g(MPI)p 1497 869 V 17 w(Group&)g(group2\))-160 909 y Fl(\(June\))p Fm(?)75 955 y Fg(int)g(MPI)p 245 955 V 17 w(Group::Incl\(int)f(n,)i (int)f(ranks[],)g(const)g(MPI)p 1241 955 V 17 w(Group&)g(group\))75 1042 y(int)g(MPI)p 245 1042 V 17 w(Group::Excl\(int)f(n,)i(int)f (ranks[],)g(const)g(MPI)p 1241 1042 V 17 w(Group&)g(group\))75 1128 y(int)g(MPI)p 245 1128 V 17 w(Group::Range)p 550 1128 V 16 w(incl\(int)g(n,)g(int)h(ranges[][3],)e(const)h(MPI)p 1473 1128 V 17 w(Group&)g(group\))75 1215 y(int)g(MPI)p 245 1215 V 17 w(Group::Range)p 550 1215 V 16 w(excl\(int)g(n,)g(int)h (ranges[][3],)e(const)h(MPI)p 1473 1215 V 17 w(Group&)g(group\))75 1301 y(int)g(MPI)p 245 1301 V 17 w(Group::Free\(void\))75 1388 y(int)g(MPI)p 245 1388 V 17 w(Comm::Size\(int&)f(size\))h(const)75 1474 y(int)g(MPI)p 245 1474 V 17 w(Comm::Rank\(int&)f(rank\))h(const) -160 1558 y Fl(\(June\))p Fm(>)75 1560 y Fg(int)g(MPI)p 245 1560 V 17 w(Comm::Compare\(const)e(MPI)p 811 1560 V 17 w(Comm&)i(comm,)h(int&)f(result\))g(const)75 1647 y(int)g(MPI)p 245 1647 V 17 w(Comm::Dup\(const)f(MPI)p 716 1647 V 17 w(Comm&)h(comm\))-160 1687 y Fl(\(June\))p Fm(?)75 1733 y Fg(int)g(MPI)p 245 1733 V 17 w(Comm::Create\(const)f (MPI)p 788 1733 V 16 w(Comm&)i(comm,)f(const)g(MPI)p 1306 1733 V 17 w(Group&)g(group\))75 1820 y(int)g(MPI)p 245 1820 V 17 w(Comm::Split\(const)f(MPI)p 764 1820 V 17 w(Comm&)h(comm,)g(int)g(color,)g(int)h(key\))75 1906 y(int)f(MPI)p 245 1906 V 17 w(Comm::Free\(void\))75 1993 y(int)g(MPI)p 245 1993 V 17 w(Comm::Test)p 502 1993 V 16 w(inter\(int&)g(flag\))g(const)75 2079 y(int)g(MPI)p 245 2079 V 17 w(Comm::Remote)p 550 2079 V 16 w(size\(int&)f(size\))i (const)75 2166 y(int)f(MPI)p 245 2166 V 17 w(Comm::Remote)p 550 2166 V 16 w(group\(MPI)p 782 2166 V 16 w(Group&)g(group\))g(const) 75 2252 y(int)g(MPI)p 245 2252 V 17 w(Comm::Intercomm)p 622 2252 V 15 w(create\(const)g(MPI)p 1020 2252 V 16 w(Comm&)h(local)p 1300 2252 V 16 w(comm,)f(int)h(local)p 1675 2252 V 16 w(leader,)393 2309 y(const)f(MPI)p 611 2309 V 17 w(Comm&)g(peer)p 867 2309 V 17 w(comm,)g(int)h(remote)p 1267 2309 V 16 w(leader,)f(int)g(tag\))75 2395 y(int)g(MPI)p 245 2395 V 17 w(Comm::Intercomm)p 622 2395 V 15 w(merge\(const)g(MPI)p 996 2395 V 16 w(Comm&)h(intercomm,)e(int)i(high\))-160 2485 y Fl(\(June\))p Fm(>)75 2482 y Fg(int)f(::MPI)p 293 2482 V 17 w(Keyval)p 454 2482 V 16 w(create\(const)g(MPI)p 853 2482 V 16 w(Copy)p 965 2482 V 17 w(function)g(©)p 1317 2482 V 16 w(fn,)h(const)393 2538 y(MPI)p 468 2538 V 17 w(Delete)p 629 2538 V 16 w(function)f(*delete)p 1028 2538 V 16 w(fn,)h(int&)f(keyval,)g(void*)g(extra)p 1713 2538 V 17 w(state\))-160 2578 y Fl(\(June\))p Fm(?)75 2624 y Fg(int)g(::MPI)p 293 2624 V 17 w(Keyval)p 454 2624 V 16 w(free\(int&)g(keyval\))1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 17 17 16 bop 75 -100 a Fh(A.7.)34 b(C++)16 b(BINDINGS)g(F)o(OR)f(PR)o (OCESS)h(TOPOLOGIES)630 b Fk(17)75 49 y Fg(int)23 b(MPI)p 245 49 15 2 v 17 w(Comm::Attr)p 502 49 V 16 w(put\(int)g(keyval,)g (void*)g(attribute)p 1259 49 V 16 w(val\))1875 146 y Fm(>)p Fl(\(June\))75 136 y Fg(int)g(MPI)p 245 136 V 17 w(Comm::Attr)p 502 136 V 16 w(get\(int)g(keyval,)g(void*&)g (attribute)p 1283 136 V 16 w(val,)g(int&)g(flag\))h(const)1875 233 y Fm(>)p Fl(\(June\))75 222 y Fg(int)f(MPI)p 245 222 V 17 w(Comm::Attr)p 502 222 V 16 w(delete\(int)g(keyval\))75 395 y Fn(A.7)59 b(C++)19 b(Bindings)g(fo)n(r)h(Pro)r(cess)h(T)-5 b(op)r(ologies)75 497 y Fg(int)23 b(MPI)p 245 497 V 17 w(Comm::Cart)p 502 497 V 16 w(create\(const)f(MPI)p 900 497 V 17 w(Com&)h(comm)p 1132 497 V 17 w(old,)g(int)h(ndims,)f(int)g (dims[],)393 553 y(int)h(periods[],)e(int)i(reorder\))1875 650 y Fm(>)p Fl(\(June\))75 640 y Fg(int)f(::MPI)p 293 640 V 17 w(Dims)p 406 640 V 17 w(create\(int)f(nnodes,)h(int)g(ndims,)g (int)h(dims[]\))1875 712 y Fm(?)p Fl(\(June\))75 726 y Fg(int)f(MPI)p 245 726 V 17 w(Comm::Graph)p 526 726 V 16 w(create\(const)f(MPI)p 924 726 V 17 w(Comm&)h(comm)p 1180 726 V 17 w(old,)g(int)g(nnodes,)393 783 y(int)h(index[],)e(int)i (edges[],)f(int)g(reorder\))75 869 y(int)g(MPI)p 245 869 V 17 w(Comm::Topo)p 502 869 V 16 w(test\(int&)g(status\))g(const)75 956 y(int)g(MPI)p 245 956 V 17 w(Comm::Graphdims)p 622 956 V 15 w(get\(int)g(nnodes[],)g(int)g(nedges[]\))g(const)75 1042 y(int)g(MPI)p 245 1042 V 17 w(Comm::Graph)p 526 1042 V 16 w(get\(int)g(maxindex,)f(int)i(maxedges,)f(int)g(index[],)393 1099 y(int)h(edges[]\))e(const)75 1185 y(int)h(MPI)p 245 1185 V 17 w(Comm::Cartdim)p 574 1185 V 16 w(get\(int&)f(ndims\))h (const)75 1271 y(int)g(MPI)p 245 1271 V 17 w(Comm::Cart)p 502 1271 V 16 w(get\(int)g(maxdims,)g(int)g(dims[],)g(int)h(periods[],) 393 1328 y(int)g(coords[]\))e(const)75 1414 y(int)h(MPI)p 245 1414 V 17 w(Comm::Cart)p 502 1414 V 16 w(rank\(int)g(coords[],)f (int&)i(rank\))f(const)75 1501 y(int)g(MPI)p 245 1501 V 17 w(Comm::Cart)p 502 1501 V 16 w(coords\(int)g(rank,)g(int)g (maxdims,)g(int)g(coords[]\))g(const)75 1587 y(int)g(MPI)p 245 1587 V 17 w(Comm::Graph)p 526 1587 V 16 w(neighbors)p 758 1587 V 16 w(count\(int)f(rank,)i(int&)f(nneighbors\))f(const)75 1674 y(int)h(MPI)p 245 1674 V 17 w(Comm::Graph)p 526 1674 V 16 w(neighbors\(int)f(rank,)h(int)h(maxneighbors,)e(int)h (neighbors[]\))393 1730 y(const)75 1817 y(int)g(MPI)p 245 1817 V 17 w(Comm::Cart)p 502 1817 V 16 w(shift\(int)g(direction,)f (int)i(disp,)f(int&)g(rank)p 1473 1817 V 17 w(source,)393 1873 y(int&)g(rank)p 611 1873 V 17 w(dest\))g(const)75 1960 y(int)g(MPI)p 245 1960 V 17 w(Comm::Cart)p 502 1960 V 16 w(sub\(const)g(MPI)p 829 1960 V 16 w(Comm&)h(comm,)f(int)g(remain) p 1371 1960 V 17 w(dims[]\))75 2046 y(int)g(MPI)p 245 2046 V 17 w(Comm::Cart)p 502 2046 V 16 w(map\(int)g(ndims,)g(int)g (dims[],)g(int)h(periods[],)e(int&)i(newrank\))393 2102 y(const)75 2189 y(int)f(MPI)p 245 2189 V 17 w(Comm::Graph)p 526 2189 V 16 w(map\(int)g(nnodes,)g(int)g(index[],)g(int)g(edges[],)g (int&)g(newrank\))393 2245 y(const)75 2419 y Fn(A.8)59 b(C++)19 b(Bindings)g(fo)n(r)h(Environmental)g(Inquiry)1875 2532 y Fm(>)p Fl(\(June\))75 2520 y Fg(int)j(::MPI)p 293 2520 V 17 w(Get)p 382 2520 V 17 w(processor)p 615 2520 V 16 w(name\(char*&)f(name,)h(int&)h(resultlen\))75 2606 y(int)f(MPI)p 245 2606 V 17 w(Errhandler::Create\(const)e(MPI)p 931 2606 V 17 w(Handler)p 1116 2606 V 16 w(function)i(*function\))75 2693 y(int)g(MPI)p 245 2693 V 17 w(Comm::Errhandler)p 646 2693 V 15 w(set\(const)g(MPI)p 972 2693 V 17 w(Errhandler&)f (errhandler\))-32 46 y Fd(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 18 18 17 bop 75 -100 a Fk(18)714 b Fh(ANNEX)15 b(A.)30 b(MPI)16 b(1.1)e(C++)h(LANGUA)o(GE)g(BINDING)75 49 y Fg(int)23 b(MPI)p 245 49 15 2 v 17 w(Comm::Errhandler)p 646 49 V 15 w(get\(MPI)p 829 49 V 16 w(Errhandler&)g(errhandler\))f(const)75 136 y(int)h(MPI)p 245 136 V 17 w(Errhandler::Free\(void\))-160 169 y Fl(\(June\))p Fm(?)75 222 y Fg(int)g(::MPI)p 293 222 V 17 w(Error)p 430 222 V 16 w(string\(int)g(errorcode,)f(char*)i (string,)f(int&)g(resultlen\))75 309 y(int)g(::MPI)p 293 309 V 17 w(Error)p 430 309 V 16 w(class\(int)g(errorcode,)g(int&)g (errorclass\))75 395 y(int)g(double)g(::MPI)p 460 395 V 17 w(Wtime\(void\))75 482 y(int)g(double)g(::MPI)p 460 482 V 17 w(Wtick\(void\))-160 565 y Fl(\(June\))p Fm(>)75 568 y Fg(int)g(::MPI)p 293 568 V 17 w(Init\(int&)g(argc,)g (char**&)g(argv\))75 654 y(int)g(::MPI)p 293 654 V 17 w(Finalize\(void\))75 741 y(int)g(::MPI)p 293 741 V 17 w(Initialized\(int&)f(flag\))-160 781 y Fl(\(June\))p Fm(?)75 827 y Fg(int)h(MPI)p 245 827 V 17 w(Comm::Abort\(int)f (errorcode\))75 1001 y Fn(A.9)59 b(C++)19 b(Bindings)g(fo)n(r)h (Pro\014ling)75 1102 y Fg(int)j(::MPI)p 293 1102 V 17 w(Pcontrol\(const)f(int)h(level,)g Fa(:)8 b(:)g(:)e Fg(\))1967 46 y Fd(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Wed May 22 16:35:16 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA13339; Wed, 22 May 1996 16:33:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA08133 for mpi-core-out; Wed, 22 May 1996 15:32:25 -0500 Received: from daedalus.epcc.ed.ac.uk (root@[129.215.56.21]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA08068; Wed, 22 May 1996 15:29:00 -0500 Received: from subnode.epcc.ed.ac.uk (smith@serpentine.epcc.ed.ac.uk [129.215.158.2]) by daedalus.epcc.ed.ac.uk (8.6.13/8.6.12) with SMTP id VAA26223; Wed, 22 May 1996 21:28:57 +0100 Date: Wed, 22 May 96 21:28:55 BST Message-Id: <17400.9605222028@subnode.epcc.ed.ac.uk> To: bright@cs.sandia.gov (Ron Brightwell) Cc: mpi-1sided@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: we should wait for 1sided implementations In-Reply-To: <9605221656.AA10958@cheetah.cs.sandia.gov.noname> References: <199605220512.WAA02668@win233.nas.nasa.gov> <9605221656.AA10958@cheetah.cs.sandia.gov.noname> From: "A Gordon Smith" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Ron Brightwell writes: >> At the last meeting, I was the sole dissenter in a number of the >> formal votes on the one-sided communication chapter. A colleague has >> since pointed out the essence of my unease, which is that we are >> voting to standardize an interface that has not been implemented (even >> partially) and which is not closely related to common practice. >> >> I know of no implmentation of the proposed MPI interface. My >> understanding is that at this stage in MPI-1, there was an >> implementation that tracked the current spec and provided substantial >> feedback. >> > I have documentation that is dated December '95 for an MPI-2 one sided > communications implementation that EPCC has done for the Cray T3D also. > -Ron Indeed, we have implemented a subset of the MPI2 One-sided proposal that was current prior to SuperComputing '95. This is available as an extension of the CRI/EPCC MPI for Cray T3D software. I admit to not having followed all of the subsequent changes to the proposal (RMC becomes RMA right?). At the time, the subset omissions were broadly as follows: * Basic MPI datatypes only for window declaration * Window, origin and target datatypes must match in Put/Get * (I)Accumulate, (I)Rmw, and Type_global omitted * Handler-related sections omitted Counter requests have been implemented. Unsurprisingly this fits snugly on CRI's SHMEM interface. The software has been installed and used at various T3D sites. We have had limited feedback, but not much to report. I hope to discuss this with attendees of a European Cray MPP Workshop to be held here in Edinburgh in July. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- -=-=- A. Gordon Smith -=- Edinburgh Parallel Computing Centre -=-=- =- http://www.epcc.ed.ac.uk/~smith -=- Phone +44 (0)131 650 6712 -= =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= From mpi-core-human@mcs.anl.gov Thu May 23 09:28:54 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA28089; Thu, 23 May 1996 09:28:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA22874 for mpi-core-out; Thu, 23 May 1996 08:31:02 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA22864; Thu, 23 May 1996 08:30:48 -0500 Received: by felix.dircon.co.uk id AA07017 (5.67b/IDA-1.5); Thu, 23 May 1996 14:30:36 +0100 Message-Id: <199605231330.AA07017@felix.dircon.co.uk> Received: from gw2-129.pool.dircon.co.uk(194.112.35.129) by amnesiac via smap (V1.3) id sma006815; Thu May 23 14:28:52 1996 Received: from localhost by jim (SMI-8.6) id OAA05524; Thu, 23 May 1996 14:28:46 +0100 To: Jeff Squyres Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Chapter 7 In-Reply-To: Your message of "Wed, 22 May 1996 13:43:53 CDT." Date: Thu, 23 May 1996 14:28:45 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Please note that the inter-language operability issue has still not been > settled; there is a nasty name clash problem (see section 7.1.6). I think this is pointing out a fundamental issue. You're presuming that the C++ MPI_Comm is a different object from the C MPI_Comm (and so on). Why ? This implies that the C++ "binding" is actually rather more than that, it's more like a re-implementation, since it is using different structures to hold the information. I was expecting that these data types would be the same. Remember that "A structure *is* a class declared with the class-key 'struct'; its members and base classes are public by default." (C++ Programming Language Rule 9). [my emphasis]. An implication of this is that a struct and a class have identical store layouts. (Which will be the same as that generated in C provided that the class has no virtual methods). Therefore I would have expected the C++ MPI_Comm to be exactly the same as the C MPI_Comm, but with a whole set of non-virtual member functions defined on it. Most of these would then look something like int MPI_Comm::Send(void* buf, int count, MPI_Datatype &datatype, int dest, int tag) { return MPI_Send(buf, count, datatype, dest, tag, this); } The inter-operability and name clash problems then disappear, and implementing the C++ binding becomes rather easy... > Also please note that the "const" constants issue hasn't been fully > resolved yet -- we'd like to hear from some more implementers first. > Particularly, the int vs. enum issue has not been resolved yet. > > Here's my $.02 on the issue: It shouldn't matter whether (for > example) the return codes are const ints or const enums, because > both are constant in the eyes of the compiler, both can be assigned > initial values, and both provide better type safety than #define. Right. The standard doesn't need to lay down whether the values are const ints or enum names from an anonymous enum. In effect the two things are indistinguishable for the uses we want to put them to, since an enum silently converts to a const int where required. Though they are in general distinguishable, for instance const int i = 10; const int * const p = &i; is valid, whereas enum { i = 10 } const int * const p = &i; is not. If we want to allow enums, we should therefore at least state that you cannot take the address of one of these constants. -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Thu May 23 09:56:20 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA28438; Thu, 23 May 1996 09:56:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA23898 for mpi-core-out; Thu, 23 May 1996 08:57:26 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA23827; Thu, 23 May 1996 08:55:11 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id IAA03428; Thu, 23 May 1996 08:55:08 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id IAA14523; Thu, 23 May 1996 08:55:08 -0500 (EST) Date: Thu, 23 May 1996 08:55:08 -0500 (EST) Message-Id: <199605231355.IAA14523@aramis.lsc.nd.edu> To: jcownie@bbn.com CC: jsquyres@lsc.nd.edu, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <199605231330.AA07017@felix.dircon.co.uk> (message from James Cownie on Thu, 23 May 1996 14:28:45 +0100) Subject: Re: Chapter 7 Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode James Cownie wrote: > I think this is pointing out a fundamental issue. You're presuming > that the C++ MPI_Comm is a different object from the C MPI_Comm (and > so on). Why ? My understanding of the MPI standard is that it does not specify exactly what an MPI handle is. In particular, I believe that it does not specify for the C bindings that the handles must be pointers to structures. The IBM implementation, for instance, uses integers for its handles. The C++ bindings on the other hand, do (seek to) mandate the use of references to objects as the handles. > Therefore I would have expected the C++ MPI_Comm to be exactly the > same as the C MPI_Comm, but with a whole set of non-virtual member > functions defined on it. Most of these would then look something like > int MPI_Comm::Send(void* buf, int count, MPI_Datatype &datatype, > int dest, int tag) > { > return MPI_Send(buf, count, datatype, dest, tag, this); > } > The inter-operability and name clash problems then disappear, and > implementing the C++ binding becomes rather easy... I agree that this would be a very elegant solution to the inter-operability problem. However, given that it does not seem to be possible to require that the C and C++ data types be the same, I think our problems with name clashing remain. Moreover, for reasons of "design for inheritance" I think it is desirable to have the member functions of the C++ objects to be virtual. This would break the equivalence between the C struct and the C++ object at any rate. -------------------------------------------------------------------- Andrew Lumsdaine Dept. Comp. Sci. & Engr. email: Andrew.Lumsdaine@nd.edu 353 Fitzpatrick Hall phone: (219) 631-8716 University of Notre Dame fax: (219) 631-9260 Notre Dame, IN 46556 -------------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Thu May 23 11:51:32 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA29910; Thu, 23 May 1996 11:51:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA27311 for mpi-core-out; Thu, 23 May 1996 10:50:08 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA27304; Thu, 23 May 1996 10:49:52 -0500 Received: by felix.dircon.co.uk id AA17278 (5.67b/IDA-1.5); Thu, 23 May 1996 16:49:38 +0100 Message-Id: <199605231549.AA17278@felix.dircon.co.uk> Received: from gw2-185.pool.dircon.co.uk(194.112.35.185) by amnesiac via smap (V1.3) id sma017102; Thu May 23 16:48:10 1996 Received: from localhost by jim (SMI-8.6) id QAA06246; Thu, 23 May 1996 16:47:49 +0100 To: Andrew.Lumsdaine@nd.edu Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Chapter 7 In-Reply-To: Your message of "Thu, 23 May 1996 08:55:08 CDT." <199605231355.IAA14523@aramis.lsc.nd.edu> Date: Thu, 23 May 1996 16:47:48 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Andrew says > My understanding of the MPI standard is that it does not specify > exactly what an MPI handle is. In particular, I believe that it > does not specify for the C bindings that the handles must be > pointers to structures. The IBM implementation, for instance, uses > integers for its handles. The C++ bindings on the other hand, do > (seek to) mandate the use of references to objects as the handles. No problem... 1) The tacky way, (which is guaranteed to work). /* mpi++.h */ #define MPI_Comm __C_MPI_Comm ... #include "mpi.h" #undef MPI_Comm ... class MPI_Comm { __C_MPI_Comm the_comm; public: int Send(void* buf, int count, MPI_Datatype &datatype, int dest, int tag) { return MPI_Send(buf, count, datatype.get_C_handle(), dest, tag, the_comm); } ... etc ... } 2) The clean way. (Assuming that your C++ compiler has namespace and that the way I read the standard is what is intended. None of the three C++ compilers I have will eat this, though YMMV). /* mpi++.h */ namespace __MPI_impl { #include "mpi.h" } class MPI_Comm { __MPI_impl::MPI_Comm the_comm; public: int Send(void* buf, int count, MPI_Datatype &datatype, int dest, int tag) { return __MPI_impl::MPI_Send(buf, count, datatype.get_C_handle(), dest, tag, the_comm); } ... etc ... } In both these cases you'll actually very likely get away with just passing (*this) to the underlying C MPI routine where it wants the relevant handle. In neither case do we rely on knowing the type of the underlying MPI handle. > Moreover, for reasons of "design for inheritance" I think it is > desirable to have the member functions of the C++ objects to be > virtual. This would break the equivalence between the C struct and > the C++ object at any rate. What's the objective in making these virtual ? When will it ever make sense for a derived class re-implement these functions and have that affect the base class ? (Which is what virtual is for.) You can always override these functions in the derived class if you want them to be different there, but it seems to me that making these virtual can only lead to disaster. (Do I want the derived class's collective ops which are provided by the base MPI_Comm class to use the derived class's send and recv ? It seems unlikely and confusing given that we don't say how the collective operations are implemented.) Actually, of course, an implementation like either of those above will work whether or not the class has virtual functions. My objection to them is that I can't see the benefit, and there's a cost on every call. You can always insert one more class if you *really* need the functions virtual, of course. class vmpi_comm : private MPI_Comm { public: virtual int Send(...) { return MPI_Comm::Send(...); } }; Now derive all your classes from vmpi_comm. -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Thu May 23 13:43:49 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA02461; Thu, 23 May 1996 13:43:48 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA00487 for mpi-core-out; Thu, 23 May 1996 12:45:01 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA00482 for ; Thu, 23 May 1996 12:44:51 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id MAA04914; Thu, 23 May 1996 12:44:49 -0500 Date: Thu, 23 May 1996 12:44:49 -0500 Message-Id: <199605231744.MAA04914@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: Latest release of full document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z. That directory also has all the sources and figures. The document is now almost 300 pages so send it to a big printer. Besides the usual updates, this version has the first cut of the I/O chapter. If you want to make changes for the version to be printed for the meeting, I need to receive them by 10:00 AM (central US time) on Friday, 31 May. I have had to make changes to several chapters to get them to integrate. Please get the latest sources for beginning new edits. Also, the macros and refs.bib were updated slightly. A complete new copy of all sources, scripts, etc. can be gotten from http://www.cs.wisc.edu/~lederman/mpi2/mpi2.tar.Z Steve From mpi-core-human@mcs.anl.gov Thu May 23 15:19:58 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA29601; Thu, 23 May 1996 15:19:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA02995 for mpi-core-out; Thu, 23 May 1996 14:20:47 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA02965; Thu, 23 May 1996 14:20:15 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id OAA04307; Thu, 23 May 1996 14:20:13 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id OAA15569; Thu, 23 May 1996 14:20:12 -0500 (EST) Date: Thu, 23 May 1996 14:20:12 -0500 (EST) Message-Id: <199605231920.OAA15569@aramis.lsc.nd.edu> To: jcownie@bbn.com CC: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <199605231549.AA17278@felix.dircon.co.uk> (message from James Cownie on Thu, 23 May 1996 16:47:48 +0100) Subject: Re: Chapter 7 Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode James Cownie wrote: > No problem... > 1) The tacky way, (which is guaranteed to work). > /* mpi++.h */ > #define MPI_Comm __C_MPI_Comm > ... > #include "mpi.h" > #undef MPI_Comm > ... > 2) The clean way. > (Assuming that your C++ compiler has namespace and that > the way I read the standard is what is intended. None of the > three C++ compilers I have will eat this, though YMMV). > /* mpi++.h */ > namespace __MPI_impl > { > #include "mpi.h" > } Interoperability with C++ and C MPI bindings was not really the problem we were raising -- both solutions 1 and 2 had occurred to us for these particular problems. The more problematic issue is this. Suppose I am writing a C++ program and I want to use a library (say, Petsc) that uses the C bindings. If the library function expects a C handle as an argument, there is a problem. For instance (this is completely fictitious petsc code by the way), suppose a petsc call is prototyped in "petsc.h" as #include "mpi.h" ... extern Petsc_call(MPI_Comm, Vector v1, Vector v2); and the function is called in C++ code like this #include "petsc.h" Petsc_call(MPI_COMM_WORLD.C_handle(), vec_1, vec_2); Now, for this to work, we have to make the user wrap up "petsc.h" with either one of the mechanisms you mentioned. I think this might be our only choice but I can't help but feel uneasy about it. If we are wrapping up mpi.h it's one thing because we have control over what goes in there. Requiring users to do it seems dangerous -- not because it wouldn't work but because I can see people screwing things up. Sincerely, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Thu May 23 16:08:34 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA16306; Thu, 23 May 1996 16:08:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA04192 for mpi-core-out; Thu, 23 May 1996 15:10:11 -0500 Received: from tbag.osc.edu (tbag.osc.edu [128.146.36.50]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA04187 for ; Thu, 23 May 1996 15:10:00 -0500 Received: for gdburns@tbag.osc.edu by tbag.osc.edu (8.7.1/950822.1) id QAA13839; Thu, 23 May 1996 16:09:55 -0400 (EDT) Date: Thu, 23 May 1996 16:09:55 -0400 (EDT) From: Greg Burns Message-Id: <199605232009.QAA13839@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: MPI rantings (was: Re: we should wait for 1sided implementations) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk some rantings on MPI-2: I agree with the general tone of caution in Bill's note but I don't think waiting until just before SC96 will significantly increase my confidence in the proposal. That is just not enough time. I'm mostly concerned with distributed memory so I don't have the kind of specific objections that Eric and others have raised. I have few technical objections to anything in MPI-2 vis-a-vis LAM. We have a powerful daemon and a powerful underlying OS - we can implement most anything, though I do not relish the mandated work. I would rather be doing something else to build MPI than implementing MPI-2. Beyond dynamic processes, a few other misc. issues and maybe some external interfaces, we will definitely have a "wait and see" attitude toward MPI-2. I am interested in the evolution of parallel computing technology and MPI's impact on it. MPI-1 was very much standardizing current practice. It was overdue. The only obvious marketing weakness of MPI-1 was dynamic processes. Lots of people (especially PVM start-up coders) who think they need it don't really need it. Moreover, I have started thinking that people who want to program heterogeneous distributed systems (like factory stuff or toaster/lights/doorbell stuff) might be better off with PVM. Still, marketing rules and MPI must have dynamic processes. The rest of MPI-2 seems to me, frankly, like an academic research project. This is fine for a .edu, but not for a Forum with serious impact potential on the industry at large. Look at the similarities. All the current proposals are high quality and represent a great effort, albeit by a limited number of people [how many people who will vote 1-sided into the standard have put their all into shaping and validating it?]. The work is far more original than standard practice. There is probably enough for a Phd for the principle authors, if they need an extra one. The motivation is more "here's something that should be done" rather than the aforementioned motivation for dynamic processes. The justification is more "lots of users I know" rather than the PVM/P4/PARMACS/Express situation we had with message-passing. 1-sided and MPI-IO demonstrate that MPIF is a forum for standardizing anything relevant to parallel computing, not just message-passing. We have become standard-happy. MPI-1 is barely a toddler. It needs to be nurtured with tools and implementations. Companies like SGI and HP are just now getting their implementations on-line while at the same time the Forum is raising the bar. What the industry needs, IMO, is lots of MPI-IO-like projects before they got the gleam of standard in their eyes (maybe they had it at the beginning). I mean multiple projects in each of the areas, just like message-passing was. Then we can sit down in a few years and repeat the MPI-1 exercise. We have definitely got the cart before the horse in MPI-2 today. There is every good chance that serious glitches will pop up in 1-sided after a larger number of implementations are attempted. It has nothing to do with the care and quality that are obvious in the current proposal. It is just a risky low-level topic. Is portability enough of a reason to target my shared memory application to MPI-2 instead of native facilities on my expensive new Power Challenge? I only see 1-sided as a win on something like a T3D where the model fits so well and portability is icing on the cake. I would not assume that the Forum has the last say on MPI-2. To avoid the standards-that-never-made-it file, there will have to be implementations for all the major w/s vendors, perhaps through a PD implementations, and for the current supercomputer elite. If enough people "wait-and-see", and the substantial offering of IBM does not begin a panic attack, the whole thing could stall. So I urge everyone to vote against most everything except the dynamic chapter. This doesn't mean the proposals are trashed. They just become research projects in name as well as substance. -- Greg From mpi-core-human@mcs.anl.gov Thu May 23 17:17:57 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA16968; Thu, 23 May 1996 17:17:57 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA05801 for mpi-core-out; Thu, 23 May 1996 16:18:04 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [129.34.139.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA05795 for ; Thu, 23 May 1996 16:17:55 -0500 Received: from snir.watson.ibm.com (snir.watson.ibm.com [9.2.215.24]) by igw2.watson.ibm.com (8.7.4/8.7.1) with SMTP id RAA11496 for ; Thu, 23 May 1996 17:18:10 -0400 Received: from localhost by snir.watson.ibm.com (AIX 3.2/UCB 5.64/5/18/96) id AA24644; Thu, 23 May 1996 17:17:52 -0400 Message-Id: <9605232117.AA24644@snir.watson.ibm.com> X-Mailer: exmh version 1.5.3 12/28/94 To: mpi-core@mcs.anl.gov Subject: Greg's message Date: Thu, 23 May 1996 17:17:51 -0400 From: Marc Snir Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk -:) -:) -:) *** (:- (:- (:- I find it ironic to see a university researcher cautioning against the "researchiness" of proposals that are pushed by vendors or national lab people. MPI2 is by no means an academic or research exercise (at least not for vendors). With the possible exception of one chapter, all proposals are driven either by the desire of some vendors to take advantage of their hardware or strong pressure from the user community to agree on standards. MPI-IO is driven by strong user community pressure for a standard on parallel IO, and the existence of parallel IO libraries. Dynamic is driven by pressure of the research community (although I personally believe it to be the least useful extension to MPI, since we have no good concept of how to mix the "user control resource management" view of PVM and the "system control resource management" view of any vendor system). One-sided is driven by the recognition that memory coupling, without full shared memory support, is increasingly widespread (Cray, Dec clusters, SGI clusters, etc.). Misc is driven by requirements of library developers. And so on. On the other hand, I find it very pleasing that Greg states that MPI-1 was very much standardizing current practice. As a person that participated in MPI-1 I can testify that this was far from from true. When we started, there was nothing ressembling MPI communicators, nothing ressembling MPI datatypes, nothing ressembling attribute caching, nothing as extensive as MPI collectives, and so on. I think it is a good testimony to the success of MPI that people believe that MPI1 standardized existing practice. That said, 300 new pages of MPI text should worry us (even if the final text is likely to be much shorter, when discussions, alternatives, repetitions, etc. are pruned out). We should carefully think how we want to package MPI2, what is part of MPI, and what is part of separated, stand-alone libraries (MPI-IO?). Marc Snir IBM T.J. Watson Research Center P.O. Box 218, Yorktown Heights, NY 10598 email: snir@watson.ibm.com phone: 914-945-3204 fax: 914-945-4425 URL: http://www.research.ibm.com/people/s/snir From mpi-core-human@mcs.anl.gov Fri May 24 04:38:47 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA11947; Fri, 24 May 1996 04:38:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA13040 for mpi-core-out; Fri, 24 May 1996 03:41:10 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id DAA13035; Fri, 24 May 1996 03:41:02 -0500 Received: by felix.dircon.co.uk id AA02236 (5.67b/IDA-1.5); Fri, 24 May 1996 09:40:59 +0100 Message-Id: <199605240840.AA02236@felix.dircon.co.uk> Received: from gw2-120.pool.dircon.co.uk(194.112.35.120) by amnesiac via smap (V1.3) id sma002229; Fri May 24 09:40:42 1996 Received: from localhost by jim (SMI-8.6) id JAA00592; Fri, 24 May 1996 09:28:38 +0100 To: Andrew.Lumsdaine@nd.edu Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Chapter 7 In-Reply-To: Your message of "Thu, 23 May 1996 14:20:12 CDT." <199605231920.OAA15569@aramis.lsc.nd.edu> Date: Fri, 24 May 1996 09:28:38 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Andrew points out a serious problem, for which there is an obvious and correct answer (though it may not be popular). The problem is that we're using the same names in the C++ binding as in the C binding, but for *different* objects. The solution is to use different names in the C++ binding for the new (C++) objects. (Since we can't change the existing C binding this is the only solution !) The effect of this is to make the C++ binding look slightly more different from the C binding than it does in the current draft. However this seems a small price to pay, since 1) Having two different things with the same name is extremely confusing. 2) You need to know which object you are dealing with. 3) We can then provide implicit cast operators to make the C++ objects convert to the C objects where appropriate, making calls to C libraries wanting the C handle trivial. 4) We avoid all the mess with macros or namespaces. Therefore I strongly recommend that we come up with new names for the C++ objects. I have no suggestion that I really like yet as to what names to give these, something like MPIX_... might be OK ? -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Fri May 24 17:42:50 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA07408; Fri, 24 May 1996 17:42:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA27232 for mpi-core-out; Fri, 24 May 1996 16:44:58 -0500 Received: from paloalto.access.hp.com (daemon@paloalto.access.hp.com [15.254.56.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA27227 for ; Fri, 24 May 1996 16:44:50 -0500 Received: from tbag.rsn.hp.com by paloalto.access.hp.com with ESMTP (1.37.109.16/15.5+ECS 3.3) id AA022694285; Fri, 24 May 1996 14:44:46 -0700 Message-Id: <199605242144.AA022694285@paloalto.access.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA104394594; Fri, 24 May 1996 16:49:54 -0500 From: Raja Daoud Subject: Re: Greg's message To: mpi-core@mcs.anl.gov Date: Fri, 24 May 1996 16:49:54 CDT In-Reply-To: <9605232117.AA24644@snir.watson.ibm.com>; from "Marc Snir" at May 23, 96 5:17 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I think Greg's point is valid and on target (independently of what substring is at the end of his email address :-) ). MPI-1 standardized more than a decade of experience and expertise in message passing. Communicators and datatypes are nifty but ultimately simple extensions to what previous message passing systems did. They did not alter the message passing "paradigm" (BTW, MPI_Pack was included in the end, but that's another story). Likewise for the collective routines, the key ones were already being used, MPI-1 added some more to allow further optimizations. Attribute caching is a nice mechanism, but as we all know, still a bag-on-the-side; I believe MPI-1's success would be the same without it. Compare MPI-1 to Express, NX, and MPL and things become clear. While MPI-1 didn't "just bless" a pre-existing system, it did indeed standardize the existing way of doing message passing. > that memory coupling, without full shared memory support, is increasingly > widespread (Cray, Dec clusters, SGI clusters, etc.). Misc is driven by I agree that MPI-2 is standardizing functionality that is "increasingly widespread", but to me, that means it was not given time to evolve and mature in the marketplace before being clamped down in a standard. In MPI-1's case, a large community of users had significant experience with message passing (point-to-point and collective) prior to standardization. This is not the case with major parts of MPI-2. I think if some technology is just becoming widespread, now is not the time to standardize it. As for the "strong pressure" argument, sometimes it just means a highly vocal minority, not necessarily representing a wide-based consensus. Whatever the outcome, I'm glad Greg (and Eric and Bill S. on other occasions) brought up that point. It's something to keep in mind in this last phase of MPI-2 work. --Raja -=- Raja Daoud Hewlett-Packard Co. raja@tbag.rsn.hp.com Convex Division From mpi-core-human@mcs.anl.gov Tue May 28 12:59:14 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA04022; Tue, 28 May 1996 12:59:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA23770 for mpi-core-out; Tue, 28 May 1996 11:05:04 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA23649; Tue, 28 May 1996 11:01:39 -0500 Message-Id: <199605281601.LAA23649@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov, mpi-comm@antares.mcs.anl.gov Subject: tutorial on Shared memory issues Date: Tue, 28 May 1996 11:01:37 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear group, For those who would like to better understand the issues surrounding some of the discussions in the 1-sided communication subcommittee, I highly recommend the excellent paper Shared Memory Consistency Models: A Tutorial by Sarita Adve and Kourosh Gharachorloo This is quite easy to read and uses as examples just exactly the sorts of things people are going to want to do with put/get. It presents the memory model that most *users* have of shared memory, explains how modern CPUs do not support this model, and what kinds of steps (e.g. fence) need to be taken by programmers (or compilers, or libraries) to reconstruct the model while still allowing optimizations. I wish that I had read this months ago, but it is not too late. It is available from the Web page: http://www.research.digital.com/wrl/techreports/abstracts/95.7.html (A link on that page will get you the postscript of the full 22-page paper.) Regards, Rusty From mpi-core-human@mcs.anl.gov Tue May 28 12:59:21 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA04026; Tue, 28 May 1996 12:59:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA23250 for mpi-core-out; Tue, 28 May 1996 10:46:45 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA23227; Tue, 28 May 1996 10:45:59 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id KAA10613; Tue, 28 May 1996 10:45:56 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id KAA05265; Tue, 28 May 1996 10:45:56 -0500 (EST) Date: Tue, 28 May 1996 10:45:56 -0500 (EST) Message-Id: <199605281545.KAA05265@aramis.lsc.nd.edu> To: tony@Aurora.CS.MsState.Edu CC: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: (message from Anthony Skjellum on Mon, 27 May 1996 16:29:06 -0500 (CDT)) Subject: Re: Chapter 7 Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode Anthony Skjellum wrote: > The issue is implicitly mixed language programming between C and C++. > If one were only talking C++, then presumably, one would not care > about the C binding. You should require the use of the C++ binding > only. This is not possible in the real world. Consider a C++ programmer wanting to use Petsc (for instance). > To get compatibility... > The object MPI could be the name space for all MPI-related stuff. > MPI.whatever or MPI->whatever ... then, it could co-exist with > the C name space, and interactions between the class form, and the C > form would have to be done with global functions. I don't understand this suggestion. The proposed C++ bindings use MPI_Comm, MPI_Group, etc., as the classes. Are you suggesting to instead use a single class named "MPI"? This seems to contradict your later suggestion about design for inheritance/reuse. > Rules for C/C++ interchange (just like between C and Fortran) must be > worked out. I think the C/C++ interchange issue is different than the C/Fortran issue. For one thing, since C and C++ are so intimately related (in particular, C is a proper subset of C++), the C and C++ bindings should inter-operate in like fashion. I think Jim Cownie's suggestion of giving the objects different names so that there is no conflict with the C name space solves all the inter-operability problems and seems not to introduce any new ones. Best Wishes, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Tue May 28 13:01:46 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA04297; Tue, 28 May 1996 13:01:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA08017 for mpi-core-out; Mon, 27 May 1996 16:29:43 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA08000; Mon, 27 May 1996 16:29:09 -0500 Received: from aurora.cs.msstate.edu (aurora.cs.msstate.edu [130.18.208.91]); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id QAA12795; Mon, 27 May 1996 16:29:07 -0500 Date: Mon, 27 May 1996 16:29:06 -0500 (CDT) From: Anthony Skjellum To: Andrew.Lumsdaine@nd.edu cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Chapter 7 In-Reply-To: <199605262017.PAA23291@aramis.lsc.nd.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk MPJ_ is really bad. The issue is implicitly mixed language programming between C and C++. If one were only talking C++, then presumably, one would not care about the C binding. You should require the use of the C++ binding only. To get compatibility... The object MPI could be the name space for all MPI-related stuff. MPI.whatever or MPI->whatever ... then, it could co-exist with the C name space, and interactions between the class form, and the C form would have to be done with global functions. Rules for C/C++ interchange (just like between C and Fortran) must be worked out. The point of the language binding in C++, among other things, is to promote "best practice" C++ programming. The virtual function naming represents a respected idiom (cf, Coplien's book of 1991), on how to build classes for reuse. If reuse is not an issue, then we should make the minimalistic global bindings, and let people build there own classes, and explicitly assume a C implementation, with a C++ interface. -Tony On Sun, 26 May 1996 Andrew.Lumsdaine@nd.edu wrote: > Date: Sun, 26 May 1996 15:17:22 -0500 (EST) > From: Andrew.Lumsdaine@nd.edu > To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov > Subject: Re: Chapter 7 > > > > In our last exciting episode James Cownie wrote: > > > Therefore I strongly recommend that we come up with new names for the > > C++ objects. > > > I have no suggestion that I really like yet as to what names to give > > these, something like MPIX_... might be OK ? > > Having different names for the C++ objects does seem to provide some > important advantages. Are there any substantive disadvantages? > > What should the prefix for the C++ objects be? I have seen MPIX_ used > elsewhere (with some unofficial extensions), but it seems as good as > any other, e.g. MPIPP_, MPJ_. > > Sincerely, > Andrew Lumsdaine > Anthony Skjellum, PhD, Asst. Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Tue May 28 13:01:48 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA04300; Tue, 28 May 1996 13:01:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA06977 for mpi-core-out; Mon, 27 May 1996 15:18:40 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA06956; Mon, 27 May 1996 15:17:59 -0500 Received: from aurora.cs.msstate.edu (aurora.cs.msstate.edu [130.18.208.91]); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id PAA12325; Mon, 27 May 1996 15:17:42 -0500 Date: Mon, 27 May 1996 15:17:42 -0500 (CDT) From: Anthony Skjellum To: James Cownie cc: Andrew.Lumsdaine@nd.edu, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Chapter 7 In-Reply-To: <199605231549.AA17278@felix.dircon.co.uk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk If the class is known to the compiler, then the virtual function table is not consulted. The case where the type must be consulted is when a pointer to a class instance is used, which is polymorphic. -Tony On Thu, 23 May 1996, James Cownie wrote: > Date: Thu, 23 May 1996 16:47:48 +0100 > From: James Cownie > To: Andrew.Lumsdaine@nd.edu > Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov > Subject: Re: Chapter 7 > > > Andrew says > > My understanding of the MPI standard is that it does not specify > > exactly what an MPI handle is. In particular, I believe that it > > does not specify for the C bindings that the handles must be > > pointers to structures. The IBM implementation, for instance, uses > > integers for its handles. The C++ bindings on the other hand, do > > (seek to) mandate the use of references to objects as the handles. > > No problem... > > 1) The tacky way, (which is guaranteed to work). > > /* mpi++.h */ > #define MPI_Comm __C_MPI_Comm > ... > > #include "mpi.h" > #undef MPI_Comm > ... > > class MPI_Comm > { > __C_MPI_Comm the_comm; > > public: > > int Send(void* buf, int count, MPI_Datatype &datatype, > int dest, int tag) > { > return MPI_Send(buf, count, datatype.get_C_handle(), > dest, tag, the_comm); > } > > ... etc ... > } > > 2) The clean way. > (Assuming that your C++ compiler has namespace and that > the way I read the standard is what is intended. None of the > three C++ compilers I have will eat this, though YMMV). > > /* mpi++.h */ > > namespace __MPI_impl > { > #include "mpi.h" > } > > > class MPI_Comm > { > __MPI_impl::MPI_Comm the_comm; > > public: > > int Send(void* buf, int count, MPI_Datatype &datatype, > int dest, int tag) > { > return __MPI_impl::MPI_Send(buf, count, datatype.get_C_handle(), > dest, tag, the_comm); > } > > ... etc ... > } > > > In both these cases you'll actually very likely get away with just passing > (*this) to the underlying C MPI routine where it wants the relevant handle. > > In neither case do we rely on knowing the type of the underlying MPI > handle. > > > Moreover, for reasons of "design for inheritance" I think it is > > desirable to have the member functions of the C++ objects to be > > virtual. This would break the equivalence between the C struct and > > the C++ object at any rate. > > What's the objective in making these virtual ? > > When will it ever make sense for a derived class re-implement these > functions and have that affect the base class ? (Which is what virtual is > for.) > > You can always override these functions in the derived class if you want > them to be different there, but it seems to me that making these virtual > can only lead to disaster. > > (Do I want the derived class's collective ops which are provided by the > base MPI_Comm class to use the derived class's send and recv ? It seems > unlikely and confusing given that we don't say how the collective > operations are implemented.) > > Actually, of course, an implementation like either of those above will work > whether or not the class has virtual functions. My objection to them is > that I can't see the benefit, and there's a cost on every call. > > You can always insert one more class if you *really* need the functions > virtual, of course. > > class vmpi_comm : private MPI_Comm { > > public: > virtual int Send(...) { return MPI_Comm::Send(...); } > > }; > > Now derive all your classes from vmpi_comm. > > -- Jim > > James Cownie > BBN UK Ltd > Phone : +44 117 9071438 > E-Mail: jcownie@bbn.com > > Anthony Skjellum, PhD, Asst. Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Tue May 28 13:02:37 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA04332; Tue, 28 May 1996 13:02:36 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA21927 for mpi-core-out; Sun, 26 May 1996 15:19:12 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA21906; Sun, 26 May 1996 15:17:58 -0500 From: Andrew.Lumsdaine@nd.edu Received: from aramis.lsc.nd.edu (aramis.lsc.nd.edu [129.74.25.93]) by lsc.nd.edu (8.7.3/8.7.3) with ESMTP id PAA08125; Sun, 26 May 1996 15:17:32 -0500 (EST) Received: (from lums@localhost) by aramis.lsc.nd.edu (8.7.4/8.7.3) id PAA23291; Sun, 26 May 1996 15:17:22 -0500 (EST) Date: Sun, 26 May 1996 15:17:22 -0500 (EST) Message-Id: <199605262017.PAA23291@aramis.lsc.nd.edu> To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <199605240840.AA02236@felix.dircon.co.uk> (message from James Cownie on Fri, 24 May 1996 09:28:38 +0100) Subject: Re: Chapter 7 Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode James Cownie wrote: > Therefore I strongly recommend that we come up with new names for the > C++ objects. > I have no suggestion that I really like yet as to what names to give > these, something like MPIX_... might be OK ? Having different names for the C++ objects does seem to provide some important advantages. Are there any substantive disadvantages? What should the prefix for the C++ objects be? I have seen MPIX_ used elsewhere (with some unofficial extensions), but it seems as good as any other, e.g. MPIPP_, MPJ_. Sincerely, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Tue May 28 17:26:15 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA11334; Tue, 28 May 1996 17:26:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA03024 for mpi-core-out; Tue, 28 May 1996 16:29:18 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA03006; Tue, 28 May 1996 16:28:52 -0500 Received: from aurora.cs.msstate.edu (aurora.cs.msstate.edu [130.18.208.91]); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id QAA24607; Tue, 28 May 1996 16:28:47 -0500 Date: Tue, 28 May 1996 16:28:45 -0500 (CDT) From: Anthony Skjellum To: Andrew.Lumsdaine@nd.edu cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Chapter 7 In-Reply-To: <199605281545.KAA05265@aramis.lsc.nd.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Tue, 28 May 1996 Andrew.Lumsdaine@nd.edu wrote: > Date: Tue, 28 May 1996 10:45:56 -0500 (EST) > From: Andrew.Lumsdaine@nd.edu > To: tony@aurora.cs.msstate.edu > Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov > Subject: Re: Chapter 7 > > > In our last exciting episode Anthony Skjellum wrote: > > > The issue is implicitly mixed language programming between C and C++. > > If one were only talking C++, then presumably, one would not care > > about the C binding. You should require the use of the C++ binding > > only. > > This is not possible in the real world. Consider a C++ programmer > wanting to use Petsc (for instance). This is true if there are to be C libraries that are using MPI. That is not true of all C++ parallel programs. If a price must be paid for all C++ parallel programs to support this capability, that should be made apparent. > > > To get compatibility... > > > The object MPI could be the name space for all MPI-related stuff. > > MPI.whatever or MPI->whatever ... then, it could co-exist with > > the C name space, and interactions between the class form, and the C > > form would have to be done with global functions. > > I don't understand this suggestion. The proposed C++ bindings use > MPI_Comm, MPI_Group, etc., as the classes. Are you suggesting to > instead use a single class named "MPI"? This seems to contradict your > later suggestion about design for inheritance/reuse. > I offered a suggestion to help with the name space problem as an alt. to above. Someone else said something like that too. MPI as a singleton class can hide all the names. Then MPI::Comm could be a base class as before. One just uses the prefix to get the avoidance of conflict. Then, in the STL style, all the types inside the MPI singleton class are used as MPI::Group, MPI::Op, etc. -Tony > > > Rules for C/C++ interchange (just like between C and Fortran) must be > > worked out. > > I think the C/C++ interchange issue is different than the C/Fortran > issue. For one thing, since C and C++ are so intimately related (in > particular, C is a proper subset of C++), the C and C++ bindings > should inter-operate in like fashion. > > I think Jim Cownie's suggestion of giving the objects different names > so that there is no conflict with the C name space solves all the > inter-operability problems and seems not to introduce any new ones. > > > Best Wishes, > Andrew Lumsdaine > Anthony Skjellum, PhD, Asst. Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Thu May 30 12:01:19 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA28433; Thu, 30 May 1996 12:01:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA13321 for mpi-core-out; Thu, 30 May 1996 11:01:05 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA13314 for ; Thu, 30 May 1996 11:00:54 -0500 Message-Id: <199605301600.LAA13314@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Attendance at June meeting Date: Thu, 30 May 1996 11:00:53 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am putting together the final count for the hotel, which I need by tomorrow. Can you please tell if you are planning for certain on attending the MPI Forum meeting next week, June 5-7? The only people I already have commitments from are: Eric Brunner Lyndon Clarke Dennis Cottel Terry Dontje Bill Gropp Steve Landherr Rusty Lusk Perry Partow Elsie Pierce Jean-Pierre Prost Rolf Rabenseifner Joe Rieken Marc Snir Marydell Tholburn Klaus Wolf If you are not on this list and are coming, please let me know as soon as possible. Thanks. Rusty From mpi-core-human@mcs.anl.gov Thu May 30 17:25:45 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA03665; Thu, 30 May 1996 17:25:39 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA21942 for mpi-core-out; Thu, 30 May 1996 16:26:32 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA21936 for ; Thu, 30 May 1996 16:26:17 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA05791; Thu, 30 May 96 15:26:15 MDT From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9605302126.AA05791@nipmuc.fsl.noaa.gov> Subject: Proposal for Enumerating Recoverable Errors (PostScript) To: mpi-core@mcs.anl.gov Date: Thu, 30 May 1996 15:26:14 -0600 (MDT) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 52407 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A while back I volunteered to attempt to enumerate recoverable errors in MPI. Well I finally got around to it, and below is the proposal. Leslie Hart hart@fsl.noaa.gov NOAA/FSL %!PS-Adobe-2.0 %%Creator: dvipsk 5.515b Copyright 1986, 1993 Radical Eye Software %%Title: temp.dvi %%Pages: 2 -1 %%PageOrder: Descend %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /@rigin{isls{[ 0 -1 1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{Resolution hsize -72 div mul 0 TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (temp.dvi) @start /Fa 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C0604040 60C031801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C00 0C000C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C0018003 0006000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F00 008000400060C060C060804060801F000B107F8F0F>I<0300030007000F000B00130033 0023004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C0020002000 20002F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060 C06000C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE0 7FC08080808001000200040004000C0008000800180018001800180018000B117E900F> I<1F00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F 8F0F>I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E00 0B107F8F0F>I E /Fb 33 120 df<004000800100020006000C000C0018001800300030 007000600060006000E000E000E000E000E000E000E000E000E000E000E000E000600060 006000700030003000180018000C000C00060002000100008000400A2A7D9E10>40 D<800040002000100018000C000C000600060003000300038001800180018001C001C001 C001C001C001C001C001C001C001C001C001C0018001800180038003000300060006000C 000C00180010002000400080000A2A7E9E10>I45 D66 D69 D73 D77 DI<003F800000E0E0000380380007001C000E000E00 1C0007003C00078038000380780003C0780003C0700001C0F00001E0F00001E0F00001E0 F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C0780003C038000380 3C0007801C0007000E000E0007001C000380380000E0E000003F80001B1E7E9C20>II<07E0801C1980300580700380600180E00180 E00080E00080E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F80 0007C00003C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800 121E7E9C17>83 D<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020 800F0020800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000001F800003FFFC001B1C7F9B1E>II<1FC0 00307000783800781C00301C00001C00001C0001FC000F1C00381C00701C00601C00E01C 40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E0006000 7004300418080C3007C00E127E9112>I<003F0000070000070000070000070000070000 070000070000070000070000070003E7000C1700180F00300700700700600700E00700E0 0700E00700E00700E00700E00700600700700700300700180F000C370007C7E0131D7E9C 17>I<03E00C301818300C700E6006E006FFFEE000E000E000E00060007002300218040C 1803E00F127F9112>I<00F8018C071E061E0E0C0E000E000E000E000E000E00FFE00E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E007FE00F1D809C 0D>I<00038003C4C00C38C01C3880181800381C00381C00381C00381C001818001C3800 0C300013C0001000003000001800001FF8001FFF001FFF803003806001C0C000C0C000C0 C000C06001803003001C0E0007F800121C7F9215>II<18003C003C0018000000000000000000000000000000FC001C001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80091D7F9C0C> I108 DII<03F0000E1C00180600300300700380600180E001C0 E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C0003F00012127F 9115>II114 D<1F9030704030C010C010 E010F8007F803FE00FF000F880388018C018C018E010D0608FC00D127F9110>I<040004 00040004000C000C001C003C00FFE01C001C001C001C001C001C001C001C001C001C101C 101C101C101C100C100E2003C00C1A7F9910>IIII E /Fc 23 115 329 300 dfs[<00E001C0038007000E000E001C001C003800 380038007000700070007000E000E000E000E000E000E000E000E000E000E000E000E000 70007000700070003800380038001C001C000E000E000700038001C000E0>11 42 126 158 18 40 D[11 42 126 158 18 I[3 8 124 130 13 44 D[<001C0000003E0000003E0000 002E0000006700000067000000E7800000C7800000C3800001C3C0000183C0000181C000 0381E0000381E0000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C003C00 1C003C0018003C0038001E0038001E0070001F0070000F0070000F00E0000780>25 29 127 156 31 65 D[21 29 124 156 31 I[<003FC000FFF003C0F00780300F00001E00003C00003C0000780000 780000780000F00000F00000F00000F00000F00000F00000F00000F00000F00000780000 7800007800003C00003C00001E00000F000807801803C07800FFF0003F80>21 31 125 157 30 I[18 29 124 156 27 69 D[17 29 124 156 26 I[4 29 124 156 13 73 D[15 29 124 156 24 76 D[27 29 124 156 39 I[20 29 124 156 32 I[<003F000001FFE000 03FFF00007C0F8000F807C001E001E003E001F003C000F00780007807800078078000780 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C0 78000780780007807C000F803C000F003E001F001F003E000F807C0007C0F80003FFF000 01FFE000003F0000>26 31 126 157 34 I[20 29 124 156 30 I[21 29 124 156 30 82 D[<03F8000FFE001C0F00380700700300600000E00000E00000E00000 E00000F000007800007F00003FE0001FFC0007FE0001FF00001F800007800003C00003C0 0001C00001C00001C00001C0C00180E00380F007007C0E001FFC0007F000>18 31 126 157 25 I[25 29 127 156 31 I[20 30 124 156 32 I[<0FC03FF07FF87038401C001C001C00 FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC3F1C>14 18 126 145 22 97 D[<03C3C00FFFC01FFFC01C3800381C00381C00381C00381C00381C001C38 001FF8001FF0003BC0003800003800001FFC001FFF003FFF80700780E001C0E001C0E001 C0F003C07C0F803FFF001FFE0007F800>18 27 127 145 23 103 D[3 29 125 156 11 108 D[<03F0000FFC001FFE003C0F00780780700380E001C0E001C0E0 01C0E001C0E001C0F003C07003807807803C0F001FFE000FFC0003F000>18 18 127 145 23 111 D[9 18 125 145 15 114 D E /Fd 51 122 329 300 dfs[<00E001E0038007000E001C001C0038003800700070007000E000E0 00E000E000E000E000E000E000E000700070007000380038001C001C000E000700038001 E000E0>11 33 122 156 24 40 D[10 33 123 156 24 I[<01C00001C00001C00001C00071C700F9 CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF8071C70001C00001C00001C00001 C000>17 18 126 149 24 I[<387C7E7E3E0E1E1C78F060>7 11 121 132 24 44 D[<000180000380000380000700000700000E00000E00001C00001C00 00380000380000700000700000E00000E00001C00001C000038000038000070000070000 0E00000E00001C00001C0000380000380000700000700000E00000E00000C00000>17 32 126 156 24 47 D[<00E00001F00001F00001B00001B00003B80003B80003B8000318 00071C00071C00071C00071C00071C000E0E000E0E000FFE000FFE001FFF001C07001C07 001C07007F1FC0FF1FE07F1FC0>19 25 127 152 24 65 D[<7FF800FFFE007FFF001C0F 001C07801C03801C03801C03801C07801C07001FFF001FFE001FFE001C1F001C03801C03 C01C01C01C01C01C01C01C01C01C03C01C07807FFF80FFFF007FFC00>18 25 127 152 24 I[<01F18007FB800FFF801F0F803C0780380380700380700380F00000 E00000E00000E00000E00000E00000E00000E00000F000007003807003803803803C0700 1F0F000FFE0007FC0001F000>17 25 126 152 24 I[<7FF800FFFE007FFF001C0F001C 07801C03C01C01C01C01C01C01E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C 00E01C01C01C01C01C03C01C07801C0F807FFF00FFFE007FF800>19 25 128 152 24 I[<7FFFC0FFFFC07FFFC01C01C01C01C01C01C01C01C01C00001C0000 1C1C001C1C001FFC001FFC001FFC001C1C001C1C001C00001C00E01C00E01C00E01C00E0 1C00E07FFFE0FFFFE07FFFE0>19 25 127 152 24 I[19 25 127 152 24 I[<03E30007FF000FFF001E1F003C0F00380700700700700700F00000 E00000E00000E00000E00000E03F80E07FC0E03F80F00700700700700700380F003C0F00 1E1F000FFF0007F70003E700>18 25 126 152 24 I[<7F1FC0FFBFE07F1FC01C07001C 07001C07001C07001C07001C07001C07001FFF001FFF001FFF001C07001C07001C07001C 07001C07001C07001C07001C07001C07007F1FC0FFBFE07F1FC0>19 25 127 152 24 I[15 25 125 152 24 I[<7F0FE0FF8FF07F0FE01C07801C0F001C0E001C1C001C3C001C78001CF0 001CE0001DF0001FF0001FF8001F38001E1C001C1C001C0E001C0E001C07001C07001C03 807F07E0FF8FF07F07E0>20 25 128 152 24 75 D[19 25 127 152 24 I[19 25 127 152 24 I[<7E1FC0FF3FE07F1FC01D07001D 87001D87001D87001DC7001DC7001CC7001CC7001CE7001CE7001CE7001C67001C67001C 77001C77001C37001C37001C37001C17007F1F00FF9F007F0F00>19 25 127 152 24 I[<1FFC003FFE007FFF00780F00F00780E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780F00780 780F007FFF003FFE001FFC00>17 25 126 152 24 I[<7FF800FFFE007FFF001C0F801C 03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FF8001C00001C 00001C00001C00001C00001C00001C00007F0000FF80007F0000>18 25 127 152 24 I[<1FFC003FFE007FFF00780F00F00780E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E0E380E1E380F0F780 787F007FFF003FFE001FFC00001C00001E00000E00000F00000700000700>17 31 126 152 24 I[<7FE000FFF8007FFC001C1E001C0F001C07001C07001C07001C0700 1C0F001C1E001FFC001FF8001FFC001C1C001C0E001C0E001C0E001C0E001C0E201C0E70 1C0E707F07E0FF87E07F03C0>20 25 127 152 24 I[<07E3001FFF003FFF00781F00F0 0700E00700E00700E00000F000007800003F80001FF00007FC0000FE00000F0000070000 0380000380600380E00380E00700F80F00FFFE00FFFC00C7F000>17 25 126 152 24 I[<7FFFE0FFFFE0FFFFE0E0E0E0E0E0E0E0E0E0E0E0E000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00007FC000FFE0007FC00>19 25 127 152 24 I[<7F07F0FF8FF87F07F01C01C01C 01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C 01C01C01C01C01C00E03800E038007070007FF0003FE0000F800>21 25 128 152 24 I[19 25 127 152 24 87 D[19 25 127 152 24 89 D[17 3 126 126 24 95 D[<1FE0003FF0007FF800783C00300E00000E00000E0003FE001FFE003E0E00700E00 E00E00E00E00E00E00783E007FFFE03FE7E00F83E0>19 18 126 145 24 97 D[<7E0000FE00007E00000E00000E00000E00000E00000E3E000EFF000FFF 800F83C00F00E00E00E00E00700E00700E00700E00700E00700E00700E00E00F01E00F83 C00FFF800EFF00063C00>20 25 128 152 24 I[<03F80FFC1FFE3C1E780C7000E000E0 00E000E000E000F000700778073E0E1FFC0FF803F0>16 18 125 145 24 I[<003F00007F00003F0000070000070000070000070003C7000FF7001FFF003C 1F00780F00700700E00700E00700E00700E00700E00700E00700700F00700F003C1F001F FFE00FE7F007C7E0>20 25 127 152 24 I[<03E00FF81FFC3C1E780E7007E007FFFFFF FFFFFFE000E000700778073C0F1FFE0FFC03F0>16 18 125 145 24 I[<001F00007F8000FF8001E78001C30001C00001C0007FFF00FFFF00FFFF0001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0003FFE00 7FFF003FFE00>17 25 127 152 24 I[<03E3C007F7E00FFFE01C1CC0380E00380E0038 0E00380E00380E001C1C000FF8001FF0001BE0003800001800001FFC001FFF003FFF8078 03C0E000E0E000E0E000E0E000E07001C07C07C03FFF800FFE0003F800>19 28 127 145 24 I[<7E0000FE00007E00000E00000E00000E00000E00000E3C000EFE00 0FFF000F87800F03800E03800E03800E03800E03800E03800E03800E03800E03800E0380 0E03807FC7F0FFE7F87FC7F0>21 25 128 152 24 I[<018003C003C001800000000000 0000007FC07FC07FC001C001C001C001C001C001C001C001C001C001C001C001C07FFFFF FF7FFF>16 26 125 153 24 I[<7E0000FE00007E00000E00000E00000E00000E00000E 7FE00E7FE00E7FE00E0F000E1E000E3C000E78000EF0000FF0000FF8000FBC000F1E000E 0E000E07000E07807F87F0FFCFF07F87F0>20 25 128 152 24 107 D[17 25 126 152 24 I[21 18 128 145 24 I[<7E3C00FEFE007FFF000F87800F03800E03800E03800E03800E0380 0E03800E03800E03800E03800E03800E03807FC7F0FFE7F87FC7F0>21 18 128 145 24 I[<03E0000FF8001FFC003C1E00780F00700700E00380E00380E00380 E00380E00380F00780700700780F003C1E001FFC000FF80003E000>17 18 126 145 24 I[<7E3E00FEFF007FFF800F83C00F00E00E00E00E00700E00700E0070 0E00700E00700E00700E00E00F01E00F83C00FFF800EFF000E3C000E00000E00000E0000 0E00000E00000E00007FC000FFE0007FC000>20 27 128 145 24 I[<07C7000FE7001FF7003C1F00700F00700F00E00700E00700E00700E00700E00700E0 0700700F00700F003C3F003FF7001FE70007C70000070000070000070000070000070000 0700003FE0007FF0003FE0>20 27 126 145 24 I[19 18 127 145 24 I[<0FEC3FFC7FFCF03CE01CE01C70007F801FF007F800 3C600EE00EF00EF81EFFFCFFF8C7E0>15 18 125 145 24 I[<03000007000007000007 00000700007FFF00FFFF00FFFF0007000007000007000007000007000007000007000007 010007038007038007038007870003FE0001FC0000F800>17 23 127 150 24 I[<7E1F80FE3F807E1F800E03800E03800E03800E03800E03800E03800E03 800E03800E03800E03800E03800E0F800FFFF007FBF803E3F0>21 18 128 145 24 I[<7F1FC0FF1FE07F1FC01C07001E0F000E0E000E0E000E0E00071C00 071C00071C00071C0003B80003B80003B80001F00001F00000E000>19 18 127 145 24 I[19 18 127 145 24 I[<7F1FC0FF9FE07F1FC01C07000E07000E0E000E0E00070E00071C00 071C00039C00039C0003980001B80001B80000F00000F00000F00000E00000E00000E000 01C00079C0007BC0007F80003F00003C0000>19 27 127 145 24 121 D E /Fe 22 118 432 300 dfs[<0001C001F1C003F1C007F0000700000E00000E00 000E00000E00000E00000E00000E0000FFF1C0FFF1C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0>18 30 128 157 32 12 D[3 3 124 130 17 46 D[<03C00FF01FF83C3C381C700E 700E700EE007E007E007E007E007E007E007E007E007E007E007E007E007700E700E700E 381C3C3C1FF80FF007E0>16 29 126 155 30 48 D[<010007003F00FF00C70007000700 070007000700070007000700070007000700070007000700070007000700070007000700 0700FFF8FFF8>13 28 124 155 30 I[3 18 124 145 17 58 D[<003FC000FFF003C0F00780300F00001E00003C00003C00007800 00780000780000F00000F00000F00000F00000F00000F00000F00000F00000F000007800 007800007800003C00003C00001E00000F000807801803C07800FFF0003F80>21 31 125 157 39 67 D[23 29 124 156 43 I[18 29 124 156 36 I[20 29 124 156 39 80 D[<0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF0 7C7FFC7FDC3F1C>14 18 126 145 29 97 D[15 29 125 156 30 I[<07C01FE03FF078787018601CFFFCFFFCFFFCE000 E000E000700070043C1C3FFC1FF807E0>14 18 126 145 26 101 D[<03C3C00FFFC01FFFC01C3800381C00381C00381C00381C00381C001C38001FF8001F F0003BC0003800003800001FFC001FFF003FFF80700780E001C0E001C0E001C0F003C07C 0F803FFF001FFE0007F800>18 27 127 145 30 103 D[4 29 126 156 14 105 D[3 29 125 156 14 108 D[14 18 125 145 30 110 D[<03F0000FFC001FFE003C0F00 780780700380E001C0E001C0E001C0E001C0E001C0F003C07003807807803C0F001FFE00 0FFC0003F000>18 18 127 145 30 I[15 26 125 145 30 I[9 18 125 145 20 114 D[<1FC03FF07FF0F030E000E000F0007F 003FC01FE000F0003800388038F078FFF07FE01FC0>13 18 127 145 23 I[<1C001C001C001C001C001C00FFE0FFE01C001C001C001C001C001C001C001C 001C001C001C001C001C201FF00FF007C0>12 24 127 151 22 I[14 18 125 145 30 I E /Fg 38 122 329 300 dfs[<6060F0F0F8F8686808080808080810 101010202040408080>13 12 127 156 23 34 D[<60F0F0701010101020204080>4 12 124 131 13 44 D[<60F0F060>4 4 124 131 13 46 D[<030007003F00C700070007 000700070007000700070007000700070007000700070007000700070007000700070007 00070007000F80FFF8>13 28 124 155 23 49 D[<07C01830201C400C400EF00FF80FF8 07F8077007000F000E000E001C001C00380070006000C00180030006010C01180110023F FE7FFEFFFE>16 28 126 155 23 I[<60F0F0600000000000000000000060F0F060>4 18 124 145 13 58 D[22 28 126 155 30 70 D[12 28 127 155 16 73 D[21 28 126 155 29 76 D[33 28 126 155 42 I[23 28 126 155 31 80 D[28 29 126 155 34 82 D[<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F002080 0F0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 1F800003FFFC00>27 28 127 155 33 84 D[40 29 127 155 47 87 D[<7FF0FFC00FC03E000780180003C0180003E0100001E0200001F0600000F040 0000788000007D8000003D0000001E0000001F0000000F0000000F8000000F80000013C0 000023E0000021E0000041F00000C0F8000080780001007C0003003C0002001E0006001F 001F003F80FFC0FFF0>28 28 127 155 34 I[<1FC000307000783800781C00301C0000 1C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E801F 8700>18 18 126 145 23 97 D[19 29 127 156 25 I[<07E00C301878307870306000E000E000E000E000E000E000600070 04300418080C3007C0>14 18 126 145 20 I[<003F0000070000070000070000070000 070000070000070000070000070000070003E7000C1700180F00300700700700600700E0 0700E00700E00700E00700E00700E00700600700700700300700180F000C370007C7E0> 19 29 126 156 25 I[<03E00C301818300C700E6006E006FFFEE000E000E000E0006000 7002300218040C1803E0>15 18 127 145 20 I[<00F8018C071E061E0E0C0E000E000E 000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E007FE0>15 29 128 156 14 I[<00038003C4C00C38C01C3880181800381C00 381C00381C00381C001818001C38000C300013C0001000003000001800001FF8001FFF00 1FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007F800>18 28 127 146 23 I[20 29 127 156 25 I[<18003C003C0018000000000000000000000000000000FC001C001C001C 001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80>9 29 127 156 13 I[19 29 127 156 24 107 D[9 29 127 156 13 I[33 18 127 145 39 I[20 18 127 145 25 I[<03F0000E1C00180600300300700380600180E001C0E001C0E001C0 E001C0E001C0E001C06001807003803003001806000E1C0003F000>18 18 127 145 23 I[19 26 127 145 25 I[13 18 127 145 18 114 D[<1F9030704030C010C010E010F8007F803FE00FF000F8803880 18C018C018E010D0608FC0>13 18 127 145 18 I[<04000400040004000C000C001C00 3C00FFE01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C100E20 03C0>12 26 127 153 18 I[20 18 127 145 25 I[19 18 127 145 24 I[27 18 127 145 33 I[<7F8FF00F03800F030007020003840001 C80001D80000F00000700000780000F800009C00010E00020E000607000403801E07C0FF 0FF8>21 18 128 145 24 I[19 26 127 145 24 I E /Fh 45 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000071C000C060018030038 03803803807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C038 03803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038007C0FFFE0F217CA018>I<03F8000C1E0010070020 07804007C07807C07803C07807C03807C0000780000780000700000F00000E0000380003 F000001C00000F000007800007800003C00003C00003E02003E07003E0F803E0F803E0F0 03C04003C0400780200780100F000C1C0003F00013227EA018>51 D<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>I<10 00801E07001FFF001FFE001FF80013E00010000010000010000010000010000010000010 F800130E001407001803801003800001C00001C00001E00001E00001E00001E07001E0F0 01E0F001E0E001C08001C04003C04003802007001006000C1C0003F00013227EA018>I< 007E0001C1000300800601C00E03C01C03C0180180380000380000780000700000700000 F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013227EA018> I<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000 E0C00060C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018 >56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C0 0001C00001C0000380000380300300780700780600700C002018001030000FC00013227E A018>I<0001800000018000000180000003C0000003C0000003C0000005E0000005E000 000DF0000008F0000008F0000010F800001078000010780000203C0000203C0000203C00 00401E0000401E0000401E0000800F0000800F0000FFFF000100078001000780030007C0 020003C0020003C0040003E0040001E0040001E00C0000F00C0000F03E0001F8FF800FFF 20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E000070 1E0000701C0000303C0000303C0000307C0000107800001078000010F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F800000078000000780000107C000010 3C0000103C0000101C0000201E0000200E000040070000400380008001C0010000E00200 00381C000007E0001C247DA223>67 D69 DI73 D77 DI80 D82 D<03F0200C0C601802603001E07000E06000 60E00060E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF 0003FF80003FC00007E00001E00000F00000F0000070800070800070800070800070C000 60C00060E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF8780780786007 8018400780084007800840078008C007800C800780048007800480078004800780040007 800000078000000780000007800000078000000780000007800000078000000780000007 800000078000000780000007800000078000000780000007800000078000000780000007 80000007800000078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E 0018070000070000070000070000FF0007C7001E07003C0700780700700700F00708F007 08F00708F00F087817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C0 0E80600F00300E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E 0E001C0E003C0E00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07 801C0780380300780000700000F00000F00000F00000F00000F00000F00000F000007000 007800403800401C00800C010007060001F80012157E9416>I<0000E0000FE00001E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E001F8E007 04E00C02E01C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F0 00E07000E07800E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC00070700 0C03801C01C03801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000 7000007800203800201C00400E008007030000FC0013157F9416>I<003C00C6018F038F 030F070007000700070007000700070007000700FFF80700070007000700070007000700 0700070007000700070007000700070007000700070007807FF8102380A20F>I<000070 01F198071E180E0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E00 0F1C0019F0001000001000001800001800001FFE000FFFC00FFFE03800F0600030400018 C00018C00018C000186000306000303800E00E038003FE0015217F9518>I<0E0000FE00 001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E1F800E60C00E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C 003E003E003E001C00000000000000000000000000000000000E00FE001E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E >I<01C003E003E003E001C00000000000000000000000000000000001E00FE001E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01 F00E01C00E01800E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07 000E07800E03C00E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1F C07F00FE60E183801E807201C00F003C00E00F003C00E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A >I<0E1F80FE60C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I< 01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F00078 F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F 00FE61C00E80600F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E00 1E0E001E0E003C0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00 000E00000E00000E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F06 0F000E000E000E000E000E000E000E000E000E000E000E000E000E000F00FFF010157F94 13>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C80 0CC00CC008E018D0308FC00E157E9413>I<02000200020002000600060006000E001E00 3E00FFF80E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E04 0E040708030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F0060170 03827800FC7F18157F941B>III121 D E /Fi 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E0 700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F8000000 7C0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF00 000FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F8 8000007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0 F00000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<0F001F801F801F801F800F00 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fj 5 85 518 300 dfs[<000018000000180000003800000038000000 7800000078000000B8000001B800000138000002380000023C0000041C0000041C000008 1C0000181C0000101C0000201C0000201C00007FFC0000401C0000801C0001801C000100 1C0002001C0002001C0004000E000C000E001C001E00FF00FFC0>26 29 126 156 54 65 D[<01FFFE00003C0780003801C0003801C0003800E0003800E00070 00F00070007000700070007000F000E000F000E000F000E000F000E000F001C001E001C0 01E001C001E001C001C0038003C003800380038007800380070007000E0007001C000700 3800070070000E01C000FFFF0000>28 28 125 155 54 68 D[<01FFFFC0003C01C00038 00C00038008000380080003800800070008000700080007020800070200000E0400000E0 400000E0C00000FFC00001C0800001C0800001C0800001C0800003810000038000000380 000003800000070000000700000007000000070000000F000000FFF00000>26 28 125 155 47 70 D[<01FFF800003C0E00003807000038038000380380003803800070 0780007007800070078000700F0000E00E0000E01C0000E0700000FFC00001C0C00001C0 600001C0700001C07000038070000380700003807000038070000700F0000700F0400700 F0400700F0800F007880FFE0790000001E00>26 29 125 155 52 82 D[<1FFFFFC01C0701C0300E00C0200E0080600E0080400E0080401C0080801C008080 1C0080001C00000038000000380000003800000038000000700000007000000070000000 70000000E0000000E0000000E0000000E0000001C0000001C0000001C0000001C0000003 C000007FFE0000>26 28 121 155 52 84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 2 1 2 0 bop 75 -125 a Fg(2)147 20 y Fd(LOGICAL)22 b(FLAG)147 77 y(INTEGER)g(IERROR)1967 21 y Fa(1)1967 78 y(2)1967 134 y(3)1967 191 y(4)1967 247 y(5)1967 303 y(6)1967 360 y(7)1967 416 y(8)1967 473 y(9)1959 529 y(10)1959 586 y(11)1959 642 y(12)1959 699 y(13)1959 755 y(14)1959 812 y(15)1959 868 y(16)1959 924 y(17)1959 981 y(18)1959 1037 y(19)1959 1094 y(20)1959 1150 y(21)1959 1207 y(22)1959 1263 y(23)1959 1320 y(24)1959 1376 y(25)1959 1433 y(26)1959 1489 y(27)1959 1545 y(28)1959 1602 y(29)1959 1658 y(30)1959 1715 y(31)1959 1771 y(32)1959 1828 y(33)1959 1884 y(34)1959 1941 y(35)1959 1997 y(36)1959 2054 y(37)1959 2110 y(38)1959 2166 y(39)1959 2223 y(40)1959 2279 y(41)1959 2336 y(42)1959 2392 y(43)1959 2449 y(44)1959 2505 y(45)1959 2562 y(46)1959 2618 y(47)1959 2675 y(48)p eop %%Page: 1 2 1 1 bop 1852 -125 a Fg(1)75 20 y Fe(0.1)59 b(Prop)r(osal:)26 b(De\014ning)d(Continuable)g(Erro)n(rs)75 122 y Fg(T)o(ypically)l(,)c (MPI)g(sa)o(ys)g(nothing)g(ab)q(out)h(the)f(in)o(ternal)f(state)h(of)g (itself)g(after)f(an)i(error)f(handler)g(has)75 178 y(b)q(een)14 b(in)o(v)o(ok)o(ed.)19 b(There)13 b(are)g(a)g(n)o(um)o(b)q(er)f(of)h (error)f(classes)h(within)f(MPI)h(that)g(should)g(ha)o(v)o(e)g(no)g (negativ)o(e)75 235 y(impact)k(on)h(the)g(in)o(ternal)f(state)h(of)g (MPI.)g(That)g(is)f(to)h(sa)o(y)l(,)g(if)g(one)h(of)e(the)i("con)o(tin) o(uable")e(errors)g(is)75 291 y(detected)g(and)h(no)e(other)h(external) f(ev)o(en)o(ts)h(cause)g(degradation,)f(it)g(is)g(safe)h(to)f(con)o (tin)o(ue)g(using)h(MPI)75 347 y(within)d(an)i(application.)166 404 y(The)f("con)o(tin)o(uable")f(error)h(classes)f(are:)75 507 y Fd(MPI_ERR_BUFFER)237 b(/*)23 b(invalid)g(buffer)g(pointer)g(*/) 75 564 y(MPI_ERR_COUNT)261 b(/*)23 b(invalid)g(count)g(argument)g(*/)75 620 y(MPI_ERR_TYPE)285 b(/*)23 b(invalid)g(datatype)g(argument)g(*/)75 676 y(MPI_ERR_TAG)309 b(/*)23 b(invalid)g(tag)h(argument)e(*/)75 733 y(MPI_ERR_COMM)285 b(/*)23 b(invalid)g(communicator)f(*/)75 789 y(MPI_ERR_ROOT)285 b(/*)23 b(invalid)g(root)h(*/)75 846 y(MPI_ERR_RANK)285 b(/*)23 b(invalid)g(rank)h(*/)75 902 y(MPI_ERR_GROUP)261 b(/*)23 b(invalid)g(group)g(*/)75 959 y(MPI_ERR_TOPOLOGY)189 b(/*)23 b(invalid)g(topology)g(*/)75 1015 y(MPI_ERR_DIMS)285 b(/*)23 b(invalid)g(dimension)g(argument)g(*/) 75 1072 y(MPI_ERR_ARG)309 b(/*)23 b(invalid)g(argument)g(*/)75 1175 y Fg(The)14 b(follo)o(wing)d(errors)h(pro)o(vide)h(no)g(guaran)o (tees,)g(although)f(implemen)o(ters)f(are)i(encouraged)h(to)e(mak)o(e) 75 1231 y(errors)i("con)o(tin)o(uable")g(whenev)o(er)i(p)q(ossible:)75 1323 y Fd(MPI_ERR_OP)333 b(/*)23 b(invalid)g(operation)g(*/)75 1379 y(MPI_ERR_REQUEST)213 b(/*)23 b(invalid)g(request)g(handle)g(*/)75 1436 y(MPI_ERR_UNKNOWN)213 b(/*)23 b(unknown)g(error)g(*/)75 1492 y(MPI_ERR_TRUNCATE)189 b(/*)23 b(message)g(truncated)g(on)g (receive)g(*/)75 1549 y(MPI_ERR_OTHER)261 b(/*)23 b(Other)g(error)h(*/) 75 1605 y(MPI_ERR_INTERN)237 b(/*)23 b(internal)g(MPI)h(error)f(*/)75 1662 y(MPI_ERR_IN_STATUS)165 b(/*)23 b(error)g(code)h(is)f(in)h(status) f(*/)75 1718 y(MPI_ERR_PENDING)213 b(/*)23 b(pending)g(request)g(*/)75 1774 y(MPI_ERR_SYSRESOURCE)117 b(/*)23 b(out)h(of)f(system)g(resources) g(*/)75 1866 y Fg(W)l(e)15 b(also)e(pro)o(vide)h(the)h(function)f(to)g (indicate)g(whether)g(MPI)h(b)q(eliev)o(es)f(an)h(error)e(is)h("con)o (tin)o(uable")f(or)75 1922 y(not:)75 2073 y Fc(MPI)p 160 2073 14 2 v 16 w(ERR)p 263 2073 V 15 w(IS)p 316 2073 V 16 w(CONTINUABLE)f(\(Erro)o(r,)k(Flag\))117 2151 y Fb(IN)155 b Fc(Erro)o(r)493 b Fb(MPI)14 b(Error)h(Status)f(\(actual)g (status)h(or)f(a)f(class\))117 2225 y(OUT)108 b Fc(Flag)505 b Fb(Bo)q(olean)13 b(indication)f(of)h(whether)h(the)g(implemen)o (tation)c(al-)905 2282 y(lo)o(ws)19 b(safe)h(con)o(tin)o(ued)g(op)q (eration)g(in)f(the)i(ev)o(en)o(t)f(of)f(Error)905 2338 y(b)q(eing)14 b(encoun)o(tered)75 2509 y Fd(int)23 b (MPI_Err_is_continuable)e(\(int)j(Error,)f(int)g(Flag\))75 2622 y(MPI_ERR_IS_CONTINUABLE)e(\(ERROR,)i(FLAG,)g(IERROR\))147 2679 y(INTEGER)f(ERROR)-32 21 y Fa(1)-32 78 y(2)-32 134 y(3)-32 191 y(4)-32 247 y(5)-32 303 y(6)-32 360 y(7)-32 416 y(8)-32 473 y(9)-40 529 y(10)-40 586 y(11)-40 642 y(12)-40 699 y(13)-40 755 y(14)-40 812 y(15)-40 868 y(16)-40 924 y(17)-40 981 y(18)-40 1037 y(19)-40 1094 y(20)-40 1150 y(21)-40 1207 y(22)-40 1263 y(23)-40 1320 y(24)-40 1376 y(25)-40 1433 y(26)-40 1489 y(27)-40 1545 y(28)-40 1602 y(29)-40 1658 y(30)-40 1715 y(31)-40 1771 y(32)-40 1828 y(33)-40 1884 y(34)-40 1941 y(35)-40 1997 y(36)-40 2054 y(37)-40 2110 y(38)-40 2166 y(39)-40 2223 y(40)-40 2279 y(41)-40 2336 y(42)-40 2392 y(43)-40 2449 y(44)-40 2505 y(45)-40 2562 y(46)-40 2618 y(47)-40 2675 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Thu May 30 17:29:41 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA03687; Thu, 30 May 1996 17:29:39 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA22055 for mpi-core-out; Thu, 30 May 1996 16:32:54 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA22050 for ; Thu, 30 May 1996 16:32:40 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA05913; Thu, 30 May 96 15:32:38 MDT From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9605302132.AA05913@nipmuc.fsl.noaa.gov> Subject: Proposal for Enumerating Recoverable Errors (TeX) To: mpi-core@mcs.anl.gov Date: Thu, 30 May 1996 15:32:37 -0600 (MDT) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 2424 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A while back I volunteered to attempt to enumerate recoverable errors in MPI. Well I finally got around to it, and below is the proposal. (In TeX this time.) Leslie Hart hart@fsl.noaa.gov NOAA/FSL \section{Proposal: Defining Continuable Errors} Typically, MPI says nothing about the internal state of itself after an error handler has been invoked. There are a number of error classes within MPI that should have no negative impact on the internal state of MPI. That is to say, if one of the "continuable" errors is detected and no other external events cause degradation, it is safe to continue using MPI within an application. The "continuable" error classes are: \begin{verbatim} MPI_ERR_BUFFER /* invalid buffer pointer */ MPI_ERR_COUNT /* invalid count argument */ MPI_ERR_TYPE /* invalid datatype argument */ MPI_ERR_TAG /* invalid tag argument */ MPI_ERR_COMM /* invalid communicator */ MPI_ERR_ROOT /* invalid root */ MPI_ERR_RANK /* invalid rank */ MPI_ERR_GROUP /* invalid group */ MPI_ERR_TOPOLOGY /* invalid topology */ MPI_ERR_DIMS /* invalid dimension argument */ MPI_ERR_ARG /* invalid argument */ \end{verbatim} The following errors provide no guarantees, although implementers are encouraged to make errors "continuable" whenever possible: \begin{verbatim} MPI_ERR_OP /* invalid operation */ MPI_ERR_REQUEST /* invalid request handle */ MPI_ERR_UNKNOWN /* unknown error */ MPI_ERR_TRUNCATE /* message truncated on receive */ MPI_ERR_OTHER /* Other error */ MPI_ERR_INTERN /* internal MPI error */ MPI_ERR_IN_STATUS /* error code is in status */ MPI_ERR_PENDING /* pending request */ MPI_ERR_SYSRESOURCE /* out of system resources */ \end{verbatim} We also provide the function to indicate whether MPI believes an error is "continuable" or not: \begin{funcdef}{MPI\_ERR\_IS\_CONTINUABLE (Error, Flag)} \funcarg{\IN}{Error}{MPI Error Status (actual status or a class)} \funcarg{\OUT}{Flag}{Boolean indication of whether the implementation allows safe continued operation in the event of Error being encountered} \end{funcdef} \begin{verbatim} int MPI_Err_is_continuable (int Error, int Flag) MPI_ERR_IS_CONTINUABLE (ERROR, FLAG, IERROR) INTEGER ERROR LOGICAL FLAG INTEGER IERROR \end{verbatim} From mpi-core-human@mcs.anl.gov Thu May 30 17:32:04 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA03692; Thu, 30 May 1996 17:31:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA22086 for mpi-core-out; Thu, 30 May 1996 16:35:06 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA22081 for ; Thu, 30 May 1996 16:34:53 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA05965; Thu, 30 May 96 15:34:51 MDT From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9605302134.AA05965@nipmuc.fsl.noaa.gov> Subject: Proposal for two processes to "share" a buffer (PostScript) To: mpi-core@mcs.anl.gov Date: Thu, 30 May 1996 15:34:50 -0600 (MDT) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 84035 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This proposal is prompted by Tony Skjellum's proposal titled: "Infinite asymptotic bandwidth for thread communication" We have a similiar protocol inside our system when we work on shared memory machines. Important to us is not only the bandwidth and latency, but also lightweight flow-control. The proposal in its current form solves our problem in particular, but we feel it may prompt discussion that can shape this proposal and Tony's proposal into something that will provide a portable high bandwidth/low-latency mechanism on true SMPs as well as provide high performance in distributed memory machines. Questions for discussion: 1) Is this really going to be faster on a shared memory machine with a "high-quality" implementation. 2) Does the user visible flow-control provide significant added value? 3) For both shared and distributed memory machines, does predefining the communication pattern provide options for optimization? Leslie Hart Tom Henderson NOAA/FSL %!PS-Adobe-2.0 %%Creator: dvipsk 5.515b Copyright 1986, 1993 Radical Eye Software %%Title: temp.dvi %%Pages: 6 -1 %%PageOrder: Descend %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /@rigin{isls{[ 0 -1 1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{Resolution hsize -72 div mul 0 TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (temp.dvi) @start /Fa 40 122 329 300 dfs[<00FC1F8001FC3F8003FC7F800700E0000E01C000 0E01C0000E01C0000E01C0000E01C0000E01C0000E01C000FFE1FC00FFE1FC000E01C000 0E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C000 0E01C0000E01C0000E01C0000E01C0000E01C0000E01C000>25 29 128 156 26 11 D[<00E001C0038007000E000E001C001C003800380038007000700070 007000E000E000E000E000E000E000E000E000E000E000E000E000700070007000700038 00380038001C001C000E000E000700038001C000E0>11 42 126 158 18 40 D[11 42 126 158 18 I[3 8 124 130 13 44 D[11 2 128 137 15 I[<001C0000003E0000003E0000002E0000006700000067000000E78000 00C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F0000700F000 0600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038001E00 70001F0070000F0070000F00E0000780>25 29 127 156 31 65 D[21 29 124 156 31 I[<003FC000FFF003C0 F00780300F00001E00003C00003C0000780000780000780000F00000F00000F00000F000 00F00000F00000F00000F00000F000007800007800007800003C00003C00001E00000F00 0807801803C07800FFF0003F80>21 31 125 157 30 I[23 29 124 156 33 I[18 29 124 156 27 I[17 29 124 156 26 I[20 29 124 156 32 72 D[ 4 29 124 156 13 I[22 29 124 156 32 75 D[15 29 124 156 24 I[27 29 124 156 39 I[20 29 124 156 30 80 D[<003F000001FFE00003FFF00007C0F8000F 807C001F003E003E001F003C000F00780007807800078078000780F00003C0F00003C0F0 0003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0780007807800078078 0E07803C0F0F003E079F001E03DE000F83FC0007C1F80003FFF00001FFF800003F780000 003C0000003E0000001F0000000F80>26 35 126 157 34 I[21 29 124 156 30 I[<03F8000FFE001C0F00380700700300600000E000 00E00000E00000E00000F000007800007F00003FE0001FFC0007FE0001FF00001F800007 800003C00003C00001C00001C00001C00001C0C00180E00380F007007C0E001FFC0007F0 00>18 31 126 157 25 I[25 29 127 156 31 I[20 30 124 156 32 I[<0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7F FC7FDC3F1C>14 18 126 145 22 97 D[15 29 125 156 23 I[<07E00FF81FFC3C1C70047000E000E000E000E000E000 E000700070043C1C1FFC0FF807E0>14 18 126 145 20 I[<000E000E000E000E000E00 0E000E000E000E000E000E0F8E1FEE3FFE7C3E700E700EE00EE00EE00EE00EE00EE00EF0 0E701E7C3E3FFE1FEE0F8E>15 29 126 156 23 I[<07C01FE03FF078787018601CFFFC FFFCFFFCE000E000E000700070043C1C3FFC1FF807E0>14 18 126 145 20 I[<03C3C00FFFC01FFFC01C3800381C00381C00381C00381C00381C001C38001F F8001FF0003BC0003800003800001FFC001FFF003FFF80700780E001C0E001C0E001C0F0 03C07C0F803FFF001FFE0007F800>18 27 127 145 23 103 D[14 29 125 156 23 I[4 29 126 156 11 I[3 29 125 156 11 108 D[27 18 125 145 37 I[14 18 125 145 23 I[<03F0000FFC001FFE00 3C0F00780780700380E001C0E001C0E001C0E001C0E001C0F003C07003807807803C0F00 1FFE000FFC0003F000>18 18 127 145 23 I[15 26 125 145 23 I[9 18 125 145 15 114 D[<1FC03FF07FF0F030E000E000F0 007F003FC01FE000F0003800388038F078FFF07FE01FC0>13 18 127 145 18 I[<1C001C001C001C001C001C00FFE0FFE01C001C001C001C001C001C001C 001C001C001C001C001C001C201FF00FF007C0>12 24 127 151 16 I[14 18 125 145 23 I[16 26 127 145 21 121 D E /Fb 10 58 df<1F00318060C04040C060C060C060C060C060 C060C060C060404060C031801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C00 0C000C000C000C000C000C000C000C00FF8009107E8F0F>I<1F00618040C08060C06000 60006000C00180030006000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C0 00C0008001800F00008000400060C060C060804060801F000B107F8F0F>I<0300030007 000F000B001300330023004300C300FFE003000300030003001FE00B107F8F0F>I<2080 3F002C002000200020002F0030802040006000600060C06080C061801F000B107F8F0F> I<0780184030C060C06000C000CF00F080E040C060C060C060406060C030801F000B107F 8F0F>I<40007FE07FC08080808001000200040004000C00080008001800180018001800 18000B117E900F>I<1F00318060C060C060C071803F000F00338061C0C060C060C06040 4060801F000B107F8F0F>I<1F00318060C0C040C060C060C06040E021E01E6000600040 60C0608043003E000B107F8F0F>I E /Fc 71 126 329 300 dfs[<60F0F0F0F0F0F0F0 F0F0F0F0F0F0F0F0600000000060F0F060>4 25 119 152 24 33 D[13 11 124 152 24 I[<03800007E0000FE0001E70001C70001C70001C70001C77E01CE7E01DE7 E00FC7000F8E000F0E001E0E003F1C007F1C00739C00E3F800E1F800E0F1C0E0F1C071F9 C07FFFC03F9F801E0700>19 25 127 152 24 38 D[<30787C3C1C1C1C1C3878F0E040> 6 13 120 152 24 I[<00E001E0038007000E001C001C0038003800700070007000E000 E000E000E000E000E000E000E000E000700070007000380038001C001C000E0007000380 01E000E0>11 33 122 156 24 I[10 33 123 156 24 I[<01C00001C00001C00001C00071C700F9 CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF8071C70001C00001C00001C00001 C000>17 18 126 149 24 I[<01C00001C00001C00001C00001C00001C00001C00001C0 00FFFF80FFFF80FFFF8001C00001C00001C00001C00001C00001C00001C00001C000>17 19 126 149 24 I[<387C7E7E3E0E1E1C78F060>7 11 121 132 24 I[17 3 126 141 24 I[<70F8F8F870>5 5 120 132 24 I[<000180000380000380000700000700000E00000E00001C00001C0000 380000380000700000700000E00000E00001C00001C0000380000380000700000700000E 00000E00001C00001C0000380000380000700000700000E00000E00000C00000>17 32 126 156 24 I[<03E0000FF8001FFC001E3C00380E00780F00700700700700E00380 E00380E00380E00380E00380E00380E00380E00380F00780700700700700780F003C1E00 1E3C001FFC000FF80003E000>17 25 126 152 24 I[<01800380038007800F807F80FF 807380038003800380038003800380038003800380038003800380038003807FF87FFC7F F8>14 25 124 152 24 I[<70F8F8F870000000000000000070F8F8F870>5 18 120 145 24 58 D[<387C7C7C38000000000000000038787C7C3C1C1C3870E040>6 24 121 145 24 I[<7FFF00FFFF80FFFF80000000000000000000000000000000FFFF80 FFFF807FFF00>17 11 126 145 24 61 D[17 21 126 150 24 I[<0FE03FF87FFCF01EF00EF00E601E003C 007800F001C003800380038003800380030000000000000000000300078007800300>15 25 125 152 24 I[<00E00001F00001F00001B00001B00003B80003B80003B800031800 071C00071C00071C00071C00071C000E0E000E0E000FFE000FFE001FFF001C07001C0700 1C07007F1FC0FF1FE07F1FC0>19 25 127 152 24 65 D[<7FF800FFFE007FFF001C0F00 1C07801C03801C03801C03801C07801C07001FFF001FFE001FFE001C1F001C03801C03C0 1C01C01C01C01C01C01C01C01C03C01C07807FFF80FFFF007FFC00>18 25 127 152 24 I[<01F18007FB800FFF801F0F803C0780380380700380700380F00000 E00000E00000E00000E00000E00000E00000E00000F000007003807003803803803C0700 1F0F000FFE0007FC0001F000>17 25 126 152 24 I[<7FF800FFFE007FFF001C0F001C 07801C03C01C01C01C01C01C01E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C 00E01C01C01C01C01C03C01C07801C0F807FFF00FFFE007FF800>19 25 128 152 24 I[<7FFFC0FFFFC07FFFC01C01C01C01C01C01C01C01C01C00001C0000 1C1C001C1C001FFC001FFC001FFC001C1C001C1C001C00001C00E01C00E01C00E01C00E0 1C00E07FFFE0FFFFE07FFFE0>19 25 127 152 24 I[19 25 127 152 24 I[<03E30007FF000FFF001E1F003C0F00380700700700700700F00000 E00000E00000E00000E00000E03F80E07FC0E03F80F00700700700700700380F003C0F00 1E1F000FFF0007F70003E700>18 25 126 152 24 I[<7F1FC0FFBFE07F1FC01C07001C 07001C07001C07001C07001C07001C07001FFF001FFF001FFF001C07001C07001C07001C 07001C07001C07001C07001C07001C07007F1FC0FFBFE07F1FC0>19 25 127 152 24 I[15 25 125 152 24 I[<7F0FE0FF8FF07F0FE01C07801C0F001C0E001C1C001C3C001C78001CF0 001CE0001DF0001FF0001FF8001F38001E1C001C1C001C0E001C0E001C07001C07001C03 807F07E0FF8FF07F07E0>20 25 128 152 24 75 D[19 25 127 152 24 I[19 25 127 152 24 I[<7E1FC0FF3FE07F1FC01D07001D 87001D87001D87001DC7001DC7001CC7001CC7001CE7001CE7001CE7001C67001C67001C 77001C77001C37001C37001C37001C17007F1F00FF9F007F0F00>19 25 127 152 24 I[<1FFC003FFE007FFF00780F00F00780E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780F00780 780F007FFF003FFE001FFC00>17 25 126 152 24 I[<7FF800FFFE007FFF001C0F801C 03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FF8001C00001C 00001C00001C00001C00001C00001C00007F0000FF80007F0000>18 25 127 152 24 I[<1FFC003FFE007FFF00780F00F00780E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E0E380E1E380F0F780 787F007FFF003FFE001FFC00001C00001E00000E00000F00000700000700>17 31 126 152 24 I[<7FE000FFF8007FFC001C1E001C0F001C07001C07001C07001C0700 1C0F001C1E001FFC001FF8001FFC001C1C001C0E001C0E001C0E001C0E001C0E201C0E70 1C0E707F07E0FF87E07F03C0>20 25 127 152 24 I[<07E3001FFF003FFF00781F00F0 0700E00700E00700E00000F000007800003F80001FF00007FC0000FE00000F0000070000 0380000380600380E00380E00700F80F00FFFE00FFFC00C7F000>17 25 126 152 24 I[<7FFFE0FFFFE0FFFFE0E0E0E0E0E0E0E0E0E0E0E0E000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00007FC000FFE0007FC00>19 25 127 152 24 I[<7F07F0FF8FF87F07F01C01C01C 01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C 01C01C01C01C01C00E03800E038007070007FF0003FE0000F800>21 25 128 152 24 I[19 25 127 152 24 I[19 25 127 152 24 I[<7F1F807F3F807F1F800E1E000E1C00073C0007380003B80003F000 01F00001E00000E00001E00001F00003F00003B80007B800071C00071C000E0E000E0E00 1C07007F1FC0FF1FE07F1FC0>19 25 127 152 24 I[19 25 127 152 24 I[17 3 126 126 24 95 D[<1FE0003FF0007FF800783C00300E00000E00000E0003FE001FFE003E0E00700E00E0 0E00E00E00E00E00783E007FFFE03FE7E00F83E0>19 18 126 145 24 97 D[<7E0000FE00007E00000E00000E00000E00000E00000E3E000EFF000FFF800F 83C00F00E00E00E00E00700E00700E00700E00700E00700E00700E00E00F01E00F83C00F FF800EFF00063C00>20 25 128 152 24 I[<03F80FFC1FFE3C1E780C7000E000E000E0 00E000E000F000700778073E0E1FFC0FF803F0>16 18 125 145 24 I[<003F00007F00003F0000070000070000070000070003C7000FF7001FFF003C1F00 780F00700700E00700E00700E00700E00700E00700E00700700F00700F003C1F001FFFE0 0FE7F007C7E0>20 25 127 152 24 I[<03E00FF81FFC3C1E780E7007E007FFFFFFFFFF FFE000E000700778073C0F1FFE0FFC03F0>16 18 125 145 24 I[<001F00007F8000FF 8001E78001C30001C00001C0007FFF00FFFF00FFFF0001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C0003FFE007FFF003FFE00>17 25 127 152 24 I[<03E3C007F7E00FFFE01C1CC0380E00380E00380E00380E00380E00 1C1C000FF8001FF0001BE0003800001800001FFC001FFF003FFF807803C0E000E0E000E0 E000E0E000E07001C07C07C03FFF800FFE0003F800>19 28 127 145 24 I[<7E0000FE00007E00000E00000E00000E00000E00000E3C000EFE000FFF000F 87800F03800E03800E03800E03800E03800E03800E03800E03800E03800E03800E03807F C7F0FFE7F87FC7F0>21 25 128 152 24 I[<018003C003C0018000000000000000007F C07FC07FC001C001C001C001C001C001C001C001C001C001C001C001C07FFFFFFF7FFF> 16 26 125 153 24 I[<7E0000FE00007E00000E00000E00000E00000E00000E7FE00E7F E00E7FE00E0F000E1E000E3C000E78000EF0000FF0000FF8000FBC000F1E000E0E000E07 000E07807F87F0FFCFF07F87F0>20 25 128 152 24 107 D[17 25 126 152 24 I[21 18 128 145 24 I[<7E3C00FEFE007FFF000F87800F03800E03800E03800E03800E0380 0E03800E03800E03800E03800E03800E03807FC7F0FFE7F87FC7F0>21 18 128 145 24 I[<03E0000FF8001FFC003C1E00780F00700700E00380E00380E00380 E00380E00380F00780700700780F003C1E001FFC000FF80003E000>17 18 126 145 24 I[<7E3E00FEFF007FFF800F83C00F00E00E00E00E00700E00700E0070 0E00700E00700E00700E00E00F01E00F83C00FFF800EFF000E3C000E00000E00000E0000 0E00000E00000E00007FC000FFE0007FC000>20 27 128 145 24 I[<07C7000FE7001FF7003C1F00700F00700F00E00700E00700E00700E00700E00700E0 0700700F00700F003C3F003FF7001FE70007C70000070000070000070000070000070000 0700003FE0007FF0003FE0>20 27 126 145 24 I[19 18 127 145 24 I[<0FEC3FFC7FFCF03CE01CE01C70007F801FF007F800 3C600EE00EF00EF81EFFFCFFF8C7E0>15 18 125 145 24 I[<03000007000007000007 00000700007FFF00FFFF00FFFF0007000007000007000007000007000007000007000007 010007038007038007038007870003FE0001FC0000F800>17 23 127 150 24 I[<7E1F80FE3F807E1F800E03800E03800E03800E03800E03800E03800E03 800E03800E03800E03800E03800E0F800FFFF007FBF803E3F0>21 18 128 145 24 I[<7F1FC0FF1FE07F1FC01C07001E0F000E0E000E0E000E0E00071C00 071C00071C00071C0003B80003B80003B80001F00001F00000E000>19 18 127 145 24 I[19 18 127 145 24 I[<7F1FC07F3FC07F1FC00F1C00073C0003B80003F00001F00000E000 01E00001F00003B800073C00071C000E0E007F1FC0FF3FE07F1FC0>19 18 127 145 24 I[<7F1FC0FF9FE07F1FC01C07000E07000E0E000E0E00070E00071C00 071C00039C00039C0003980001B80001B80000F00000F00000F00000E00000E00000E000 01C00079C0007BC0007F80003F00003C0000>19 27 127 145 24 I[<3FFFC07FFFC07FFFC0700780700F00701E00003C0000780001F00003E0000780000F 00001E01C03C01C07801C0FFFFC0FFFFC0FFFFC0>18 18 127 145 24 I[<001F80007F8000FF8001E00001C00001C00001C00001C00001C00001C00001C000 01C00001C00003C0007F8000FF0000FF00007F800003C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001E00000FF80007F80001F80>17 32 126 156 24 I[<7C0000FF0000FF800003C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001E00000FF00007F80007F8000FF0001E00001C00001C000 01C00001C00001C00001C00001C00001C00001C00003C000FF8000FF00007C0000>17 32 126 156 24 125 D E /Fd 61 123 df<007E1F0001C1B1800303E3C00703C3C00E03 C1800E01C0000E01C0000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01 C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01 C0000E01C0000E01C0000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C00E01800E00000E00000E00000E00000E0000FFFF C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C07F87F8151D809C17>I<007FC001C1C00303C00703C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F CFF8151D809C17>I<6060F0F0F8F86868080808080808101010102020404080800D0C7F 9C15>34 D<60F0F8680808081010204080050C7C9C0C>39 D<004000800100020006000C 000C0018001800300030007000600060006000E000E000E000E000E000E000E000E000E0 00E000E000E000600060006000700030003000180018000C000C00060002000100008000 400A2A7D9E10>I<800040002000100018000C000C000600060003000300038001800180 018001C001C001C001C001C001C001C001C001C001C001C001C001800180018003800300 0300060006000C000C00180010002000400080000A2A7E9E10>I<60F0F0701010101020 204080040C7C830C>44 DI<60F0F06004047C830C>I<00010003 000600060006000C000C000C0018001800180030003000300060006000C000C000C00180 01800180030003000300060006000C000C000C0018001800180030003000300060006000 6000C000C00010297E9E15>I<030007003F00C700070007000700070007000700070007 00070007000700070007000700070007000700070007000700070007000F80FFF80D1C7C 9B15>49 D<07C01830201C400C400EF00FF80FF807F8077007000F000E000E001C001C00 380070006000C00180030006010C01180110023FFE7FFEFFFE101C7E9B15>I<07E01830 201C201C781E780E781E381E001C001C00180030006007E00030001C001C000E000F000F 700FF80FF80FF80FF00E401C201C183007E0101D7E9B15>I<60F0F06000000000000000 00000060F0F06004127C910C>58 D<0FE03038401CE00EF00EF00EF00E000C001C003000 6000C0008001800100010001000100010001000000000000000000000003000780078003 000F1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F0000001780 0000178000001780000023C0000023C0000023C0000041E0000041E0000041E0000080F0 000080F0000180F8000100780001FFF80003007C0002003C0002003C0006003E0004001E 0004001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E0618001801980070007800E0003801C0003801C000180 38000180780000807800008070000080F0000000F0000000F0000000F0000000F0000000 F0000000F0000000F0000000700000807800008078000080380000801C0001001C000100 0E000200070004000180080000E03000001FC000191E7E9C1E>IIII72 DI76 DII<003F800000E0E0000380380007001C000E000E001C0007003C000780380003 80780003C0780003C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001 E0F00001E0F00001E0700001C0780003C0780003C0380003803C0007801C0007000E000E 0007001C000380380000E0E000003F80001B1E7E9C20>II<003F800000E0E0000380380007001C000E000E001C0007003C00078038 000380780003C0780003C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F0 0001E0F00001E0F00001E0700001C0780003C0780003C0380003803C0E07801C1107000E 208E000720DC0003A0F80000F0E020003FE0200000602000007060000078E000003FC000 003FC000001F8000000F001B257E9C20>II<07E080 1C1980300580700380600180E00180E00080E00080E00080F00000F800007C00007FC000 3FF8001FFE0007FF0000FF80000F800007C00003C00001C08001C08001C08001C0C00180 C00180E00300D00200CC0C0083F800121E7E9C17>I<7FFFFFC0700F01C0600F00C0400F 0040400F0040C00F0020800F0020800F0020800F0020000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000001F800003FFFC001B1C7F9B1E>II87 D<1FC000307000783800781C00301C00 001C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E80 1F870012127E9115>97 DI<07E00C 301878307870306000E000E000E000E000E000E00060007004300418080C3007C00E127E 9112>I<003F000007000007000007000007000007000007000007000007000007000007 0003E7000C1700180F00300700700700600700E00700E00700E00700E00700E00700E007 00600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E 6006E006FFFEE000E000E000E00060007002300218040C1803E00F127F9112>I<00F801 8C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C 3880181800381C00381C00381C00381C001818001C38000C300013C00010000030000018 00001FF8001FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007 F800121C7F9215>II<18003C003C 0018000000000000000000000000000000FC001C001C001C001C001C001C001C001C001C 001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C000000000 000000000000000000000FE000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>III II<03F0000E1C0018060030 0300700380600180E001C0E001C0E001C0E001C0E001C0E001C060018070038030030018 06000E1C0003F00012127F9115>II<03C1000C3300180B 00300F00700700700700E00700E00700E00700E00700E00700E00700600700700700300F 00180F000C370007C700000700000700000700000700000700000700000700003FE0131A 7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000F88038 8018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C001C003C00FF E01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C100E2003C00C 1A7F9910>IIII<7F8FF00F03800F0300 07020003840001C80001D80000F00000700000780000F800009C00010E00020E00060700 0403801E07C0FF0FF81512809116>II<7FFC7038603840 7040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E127F9112>I E /Fe 8 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F00781F00780E00F80000F800 00F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115 >99 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F 001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018E018 F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Ff 22 120 432 300 dfs[<00FC1F8001FC3F8003FC7F800700E0000E01C0000E01 C0000E01C0000E01C0000E01C0000E01C0000E01C000FFE1FC00FFE1FC000E01C0000E01 C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01 C0000E01C0000E01C0000E01C0000E01C0000E01C000>25 29 128 156 35 11 D[11 8 127 156 30 34 D[3 3 124 130 17 46 D[<03C00FF01FF83C3C381C700E700E70 0EE007E007E007E007E007E007E007E007E007E007E007E007E007700E700E700E381C3C 3C1FF80FF007E0>16 29 126 155 30 48 D[<010007003F00FF00C70007000700070007 0007000700070007000700070007000700070007000700070007000700070007000700FF F8FFF8>13 28 124 155 30 I[3 18 124 145 17 58 D[21 29 124 156 40 66 D[20 29 124 156 39 80 D[<03F8000FFE001C0F00380700700300600000E00000E00000E000 00E00000F000007800007F00003FE0001FFC0007FE0001FF00001F800007800003C00003 C00001C00001C00001C00001C0C00180E00380F007007C0E001FFC0007F000>18 31 126 157 33 83 D[25 29 127 156 40 I[<0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC 3F1C>14 18 126 145 29 97 D[<07E00FF81FFC3C1C70047000E000E000E000E000E000 E000700070043C1C1FFC0FF807E0>14 18 126 145 26 99 D[<000E000E000E000E000E 000E000E000E000E000E000E0F8E1FEE3FFE7C3E700E700EE00EE00EE00EE00EE00EE00E F00E701E7C3E3FFE1FEE0F8E>15 29 126 156 30 I[<07C01FE03FF078787018601CFF FCFFFCFFFCE000E000E000700070043C1C3FFC1FF807E0>14 18 126 145 26 I[14 29 125 156 30 104 D[3 29 125 156 14 108 D[<03F0000FFC001FFE003C0F00780780700380E001 C0E001C0E001C0E001C0E001C0F003C07003807807803C0F001FFE000FFC0003F000>18 18 127 145 30 111 D[15 26 125 145 30 I[9 18 125 145 20 114 D[<1FC03FF07FF0F030E000E000F0007F 003FC01FE000F0003800388038F078FFF07FE01FC0>13 18 127 145 23 I[14 18 125 145 30 117 D[25 18 127 145 40 119 D E /Fh 58 122 329 300 dfs[<007E1F0001C1B1800303E3C00703C3C00E03C180 0E01C0000E01C0000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C000 0E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C000 0E01C0000E01C0000E01C0000E01C0000E01C0007F87FC00>26 29 128 156 26 11 D[<007E0001C1800301800703C00E03C00E01800E00000E00000E0000 0E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F8>21 29 128 156 25 I[<003F07E00001C09C18000380F018000701F03C000E01E03C000E00E018 000E00E000000E00E000000E00E000000E00E000000E00E00000FFFFFFFC000E00E01C00 0E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E 00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00 E01C007FC7FCFF80>33 29 128 156 38 14 D[<6060F0F0F8F868680808080808081010 1010202040408080>13 12 127 156 23 34 D[<004000800100020006000C000C001800 1800300030007000600060006000E000E000E000E000E000E000E000E000E000E000E000 E000600060006000700030003000180018000C000C0006000200010000800040>10 42 125 158 18 40 D[<800040002000100018000C000C00060006000300030003800180 0180018001C001C001C001C001C001C001C001C001C001C001C001C00180018001800380 03000300060006000C000C0018001000200040008000>10 42 126 158 18 I[<60F0F0701010101020204080>4 12 124 131 13 44 D[11 2 128 137 15 I[<60F0F060>4 4 124 131 13 I[<00010003000600060006000C000C000C0018001800180030003000300060006000C0 00C000C0018001800180030003000300060006000C000C000C0018001800180030003000 3000600060006000C000C000>16 41 126 158 23 I[<030007003F00C7000700070007 000700070007000700070007000700070007000700070007000700070007000700070007 0007000F80FFF8>13 28 124 155 23 49 D[<07C01830201C400C400EF00FF80FF807F8 077007000F000E000E001C001C00380070006000C00180030006010C01180110023FFE7F FEFFFE>16 28 126 155 23 I[<07E01830201C201C781E780E781E381E001C001C0018 0030006007E00030001C001C000E000F000F700FF80FF80FF80FF00E401C201C183007E0 >16 29 126 155 23 I[<000C00000C00001C00003C00003C00005C0000DC00009C0001 1C00031C00021C00041C000C1C00081C00101C00301C00201C00401C00C01C00FFFFC000 1C00001C00001C00001C00001C00001C00001C0001FFC0>18 28 127 155 23 I[<300C3FF83FF03FC020002000200020002000200023E024302818301C20 0E000E000F000F000F600FF00FF00FF00F800E401E401C2038187007C0>16 29 126 155 23 I[<00F0030C06040C0E181E301E300C700070006000E3E0E430E818F0 0CF00EE006E007E007E007E007E007600760077006300E300C18180C3003E0>16 29 126 155 23 I[<000600000006000000060000000F0000000F0000000F0000001780 0000178000001780000023C0000023C0000023C0000041E0000041E0000041E0000080F0 000080F0000180F8000100780001FFF80003007C0002003C0002003C0006003E0004001E 0004001E000C001F001E001F00FF80FFF0>28 29 127 156 34 65 D[24 28 126 155 32 I[<001F808000E06180018019 80070007800E0003801C0003801C00018038000180780000807800008070000080F00000 00F0000000F0000000F0000000F0000000F0000000F0000000F000000070000080780000 8078000080380000801C0001001C0001000E000200070004000180080000E03000001FC0 00>25 30 126 156 33 I[27 28 126 155 35 I[24 28 126 155 31 I[22 28 126 155 30 I[12 28 127 155 16 73 D[21 28 126 155 29 76 D[33 28 126 155 42 I[26 28 126 155 34 I[<003F800000E0E0000380380007001C000E000E001C0007003C000780380003807800 03C0780003C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F000 01E0F00001E0700001C0780003C0780003C0380003803C0007801C0007000E000E000700 1C000380380000E0E000003F8000>27 30 126 156 35 I[23 28 126 155 31 I[28 29 126 155 34 82 D[<07E0801C1980300580700380600180E00180E00080E00080E00080F00000F8 00007C00007FC0003FF8001FFE0007FF0000FF80000F800007C00003C00001C08001C080 01C08001C0C00180C00180E00300D00200CC0C0083F800>18 30 126 156 25 I[<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F002080 0F0020800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000001F800003FFFC00>27 28 127 155 33 I[26 29 126 155 34 I[40 29 127 155 47 87 D[<7FF0FFC00FC03E000780180003C0 180003E0100001E0200001F0600000F0400000788000007D8000003D0000001E0000001F 0000000F0000000F8000000F80000013C0000023E0000021E0000041F00000C0F8000080 780001007C0003003C0002001E0006001F001F003F80FFC0FFF0>28 28 127 155 34 I[<1FC000307000783800781C00301C00001C00001C0001FC000F1C00 381C00701C00601C00E01C40E01C40E01C40603C40304E801F8700>18 18 126 145 23 97 D[19 29 127 156 25 I[<07E00C301878307870306000E000E000E000E000E000E000600070 04300418080C3007C0>14 18 126 145 20 I[<003F0000070000070000070000070000 070000070000070000070000070000070003E7000C1700180F00300700700700600700E0 0700E00700E00700E00700E00700E00700600700700700300700180F000C370007C7E0> 19 29 126 156 25 I[<03E00C301818300C700E6006E006FFFEE000E000E000E0006000 7002300218040C1803E0>15 18 127 145 20 I[<00F8018C071E061E0E0C0E000E000E 000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E007FE0>15 29 128 156 14 I[<00038003C4C00C38C01C3880181800381C00 381C00381C00381C001818001C38000C300013C0001000003000001800001FF8001FFF00 1FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007F800>18 28 127 146 23 I[20 29 127 156 25 I[<18003C003C0018000000000000000000000000000000FC001C001C001C 001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80>9 29 127 156 13 I[19 29 127 156 24 107 D[9 29 127 156 13 I[33 18 127 145 39 I[20 18 127 145 25 I[<03F0000E1C00180600300300700380600180E001C0E001C0E001C0 E001C0E001C0E001C06001807003803003001806000E1C0003F000>18 18 127 145 23 I[19 26 127 145 25 I[<03C1000C3300180B0030 0F00700700700700E00700E00700E00700E00700E00700E00700600700700700300F0018 0F000C370007C700000700000700000700000700000700000700000700003FE0>19 26 126 145 24 I[13 18 127 145 18 I[<1F9030704030C010C010E010F8007F80 3FE00FF000F880388018C018C018E010D0608FC0>13 18 127 145 18 I[<04000400040004000C000C001C003C00FFE01C001C001C001C001C001C001C001C 001C001C101C101C101C101C100C100E2003C0>12 26 127 153 18 I[20 18 127 145 25 I[19 18 127 145 24 I[27 18 127 145 33 I[<7F8FF00F03800F030007020003840001C800 01D80000F00000700000780000F800009C00010E00020E000607000403801E07C0FF0FF8 >21 18 128 145 24 I[19 26 127 145 24 I E /Fi 45 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000071C000C060018030038 03803803807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C038 03803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038007C0FFFE0F217CA018>I<03F8000C1E0010070020 07804007C07807C07803C07807C03807C0000780000780000700000F00000E0000380003 F000001C00000F000007800007800003C00003C00003E02003E07003E0F803E0F803E0F0 03C04003C0400780200780100F000C1C0003F00013227EA018>51 D<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>I<10 00801E07001FFF001FFE001FF80013E00010000010000010000010000010000010000010 F800130E001407001803801003800001C00001C00001E00001E00001E00001E07001E0F0 01E0F001E0E001C08001C04003C04003802007001006000C1C0003F00013227EA018>I< 007E0001C1000300800601C00E03C01C03C0180180380000380000780000700000700000 F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013227EA018> I<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000 E0C00060C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018 >56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C0 0001C00001C0000380000380300300780700780600700C002018001030000FC00013227E A018>I<0001800000018000000180000003C0000003C0000003C0000005E0000005E000 000DF0000008F0000008F0000010F800001078000010780000203C0000203C0000203C00 00401E0000401E0000401E0000800F0000800F0000FFFF000100078001000780030007C0 020003C0020003C0040003E0040001E0040001E00C0000F00C0000F03E0001F8FF800FFF 20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E000070 1E0000701C0000303C0000303C0000307C0000107800001078000010F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F800000078000000780000107C000010 3C0000103C0000101C0000201E0000200E000040070000400380008001C0010000E00200 00381C000007E0001C247DA223>67 D69 DI73 D77 DI80 D82 D<03F0200C0C601802603001E07000E06000 60E00060E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF 0003FF80003FC00007E00001E00000F00000F0000070800070800070800070800070C000 60C00060E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF8780780786007 8018400780084007800840078008C007800C800780048007800480078004800780040007 800000078000000780000007800000078000000780000007800000078000000780000007 800000078000000780000007800000078000000780000007800000078000000780000007 80000007800000078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E 0018070000070000070000070000FF0007C7001E07003C0700780700700700F00708F007 08F00708F00F087817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C0 0E80600F00300E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E 0E001C0E003C0E00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07 801C0780380300780000700000F00000F00000F00000F00000F00000F00000F000007000 007800403800401C00800C010007060001F80012157E9416>I<0000E0000FE00001E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E001F8E007 04E00C02E01C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F0 00E07000E07800E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC00070700 0C03801C01C03801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000 7000007800203800201C00400E008007030000FC0013157F9416>I<003C00C6018F038F 030F070007000700070007000700070007000700FFF80700070007000700070007000700 0700070007000700070007000700070007000700070007807FF8102380A20F>I<000070 01F198071E180E0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E00 0F1C0019F0001000001000001800001800001FFE000FFFC00FFFE03800F0600030400018 C00018C00018C000186000306000303800E00E038003FE0015217F9518>I<0E0000FE00 001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E1F800E60C00E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C 003E003E003E001C00000000000000000000000000000000000E00FE001E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E >I<01C003E003E003E001C00000000000000000000000000000000001E00FE001E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01 F00E01C00E01800E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07 000E07800E03C00E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1F C07F00FE60E183801E807201C00F003C00E00F003C00E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A >I<0E1F80FE60C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I< 01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F00078 F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F 00FE61C00E80600F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E00 1E0E001E0E003C0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00 000E00000E00000E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F06 0F000E000E000E000E000E000E000E000E000E000E000E000E000E000F00FFF010157F94 13>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C80 0CC00CC008E018D0308FC00E157E9413>I<02000200020002000600060006000E001E00 3E00FFF80E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E04 0E040708030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F0060170 03827800FC7F18157F941B>III121 D E /Fj 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E0 700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F8000000 7C0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF00 000FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F8 8000007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0 F00000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<0F001F801F801F801F800F00 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fk 5 85 518 300 dfs[<000018000000180000003800000038000000 7800000078000000B8000001B800000138000002380000023C0000041C0000041C000008 1C0000181C0000101C0000201C0000201C00007FFC0000401C0000801C0001801C000100 1C0002001C0002001C0004000E000C000E001C001E00FF00FFC0>26 29 126 156 54 65 D[<01FFFE00003C0780003801C0003801C0003800E0003800E00070 00F00070007000700070007000F000E000F000E000F000E000F000E000F001C001E001C0 01E001C001E001C001C0038003C003800380038007800380070007000E0007001C000700 3800070070000E01C000FFFF0000>28 28 125 155 54 68 D[<01FFFFC0003C01C00038 00C00038008000380080003800800070008000700080007020800070200000E0400000E0 400000E0C00000FFC00001C0800001C0800001C0800001C0800003810000038000000380 000003800000070000000700000007000000070000000F000000FFF00000>26 28 125 155 47 70 D[<01FFF800003C0E00003807000038038000380380003803800070 0780007007800070078000700F0000E00E0000E01C0000E0700000FFC00001C0C00001C0 600001C0700001C07000038070000380700003807000038070000700F0000700F0400700 F0400700F0800F007880FFE0790000001E00>26 29 125 155 52 82 D[<1FFFFFC01C0701C0300E00C0200E0080600E0080400E0080401C0080801C008080 1C0080001C00000038000000380000003800000038000000700000007000000070000000 70000000E0000000E0000000E0000000E0000001C0000001C0000001C0000001C0000003 C000007FFE0000>26 28 121 155 52 84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 6 1 6 0 bop 75 -125 a Fh(6)123 20 y Fc(if)23 b(\(myProc)g(==)h(source\))170 77 y({)170 133 y(i)g(=)g(myProc;)170 190 y(memcpy)f (\(sharedBuffHandle->semaphore,)d(&i,)k(sizeof\(int\)\);)170 246 y(sharedBuffHandle->partner)d(=)j(dest;)170 302 y(Send)g(\(comm,)f (sharedTag,)f(dest,)h(sharedBuffHandle->semaphore,)e(MPI_Pointer,)h (...\);)170 359 y(})123 415 y(else)170 472 y({)170 528 y(sharedBuffHandle->partner)f(=)j(source;)170 585 y(Recv)g(\(comm,)f (sharedTag,)f(source,)h(sharedBuffHandle->semaphore,)457 641 y(MPI_Pointer,)f(...\);)170 698 y(})123 754 y(*buffer)h(=)g (\(\(char)g(*\))h(sharedBuffHandle->semaphor)o(e\))d(+)j (cacheLineSize;)123 811 y(})75 923 y(int)f(MPI_Acquire_shared_buffer)e (\(MPI_Shared_buff)h(*sharedBuffHandle\))123 980 y({)123 1036 y(i)h(=)h(myProc)f(-)h(1;)123 1093 y(while)f(\(i)g(!=)h(myProc\)) 170 1149 y({)170 1206 y(memcpy)f(\(&i,)h(sharedBuffHandle->semapho)o (re,)d(sizeof\(int\)\);)170 1262 y(/*)j(A)g(smart)f(implementation)f (would)h(release)g(it's)g(timeslice)242 1319 y(after)g(some)h(number)f (of)g(times)g(through)g(this)h(loop)f(*/)170 1375 y(})123 1432 y(})75 1544 y(int)g(MPI_Release_shared_buffer)e(\(MPI_Shared_buff) h(*sharedBuffHandle,)314 1601 y(int)h(releaseAction\))123 1657 y({)123 1714 y(i)g(=)h(sharedBuffHandle->partner;)123 1770 y(memcpy)f(\(sharedBuffHandle->semap)o(hore,)e(&i,)i (sizeof\(int\)\);)123 1827 y(})75 1940 y(int)g(MPI_Free_shared_buffer)e (\(MPI_Shared_buff)h(*sharedBuffHandle\))123 1996 y({)123 2053 y(/*)170 2109 y(Free)i(all)f(structures)123 2165 y(*/)123 2222 y(})1967 21 y Fb(1)1967 78 y(2)1967 134 y(3)1967 191 y(4)1967 247 y(5)1967 303 y(6)1967 360 y(7)1967 416 y(8)1967 473 y(9)1959 529 y(10)1959 586 y(11)1959 642 y(12)1959 699 y(13)1959 755 y(14)1959 812 y(15)1959 868 y(16)1959 924 y(17)1959 981 y(18)1959 1037 y(19)1959 1094 y(20)1959 1150 y(21)1959 1207 y(22)1959 1263 y(23)1959 1320 y(24)1959 1376 y(25)1959 1433 y(26)1959 1489 y(27)1959 1545 y(28)1959 1602 y(29)1959 1658 y(30)1959 1715 y(31)1959 1771 y(32)1959 1828 y(33)1959 1884 y(34)1959 1941 y(35)1959 1997 y(36)1959 2054 y(37)1959 2110 y(38)1959 2166 y(39)1959 2223 y(40)1959 2279 y(41)1959 2336 y(42)1959 2392 y(43)1959 2449 y(44)1959 2505 y(45)1959 2562 y(46)1959 2618 y(47)1959 2675 y(48)p eop %%Page: 5 2 5 1 bop 1852 -125 a Fh(5)123 20 y Fc(wait)23 b (\(sharedBuffHandle->waitHan)o(dle\);)123 77 y(})75 190 y(int)g(MPI_Release_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle,)314 246 y(int)h(releaseAction\))123 302 y({)123 359 y(if)g(\(releaseAction)f(==)i(MPI_CONSUME\))170 415 y(count)g(=)f(0;)123 472 y(else)170 528 y(count)h(=)f (sharedBuffHandle->count;)123 585 y(sharedBuffHandle->waitHa)o(ndle)e (=)i(IRECV)h(\(comm,)f(sharedBuffHandle->tag,)1030 641 y(sharedBuffHandle->partn)o(er,)1030 698 y(sharedBuffHandle->buffe)o (r,)e(count,...\);)123 754 y(/*)i(Note)h(that)f(a)h(non-buffering)e (send)h(can)h(be)f(used)g(here)h(*/)123 811 y(/*)f(Note)h(that)f(a)h ("ready")f(protocol)f(can)i(also)f(be)h(used)f(since)g(we)h(guarantee) 194 867 y(the)g(receive)f(is)g(always)g(posted)g(before)g(the)h(send)f (*/)123 923 y(localHandle)f(=)i(ISSEND)f(\(comm,)g (sharedBuffHandle->tag,)e(sharedBuffHandle->partne)o(r,)648 980 y(sharedBuffHandle->buffer)o(,)g(count,...\);)123 1036 y(freeHandle)h(\(localHandle\);)123 1093 y(})75 1206 y(int)h(MPI_Free_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle\))123 1262 y({)123 1319 y(/*)170 1375 y(Either)h(cancel)h(the)f(outstanding)f(receive)h(or)h(\(if)f(you)h (don't)f(beleive)170 1432 y(cancel)g(works\))h(send)f(to)g(complete)g (the)h(receive.)123 1488 y(*/)123 1544 y(/*)170 1601 y(Free)g(all)f(structures)123 1657 y(*/)123 1714 y(})75 1820 y(Possible)g(Shared)g(Memory)g(Implementation:)75 1933 y(int)g(MPI_Make_shared_buffer)e(\(MPI_Datatype)h(dataType,)h (int,)g(count,)361 1989 y(MPI_Comm)g(comm,)g(int)h(source,)f(int)g (dest,)361 2046 y(void)h(*buffer,)e(MPI_Shared_buff)g (*sharedBuffHandle\))123 2102 y({)123 2159 y(/*)h(Ok,)h(ok)f(this)h (isn't)f(really)g(C,)g(but)h(you)f(get)h(the)f(idea\))g(*/)123 2215 y(sharedBuffHandle)e(=)j(malloc)f(\(sizeof\(sharedBuffHandle\)\);) 123 2272 y(if)g(\(myProc)g(==)h(source\))170 2328 y({)170 2385 y(sharedBuffHandle->semaphore)d(=)457 2441 y(shared_malloc)h (\(sizeof\(dataType\))f(*)j(count)f(+)h(cacheLineSize\);)170 2498 y(})123 2554 y(sharedBuffHandle->buffer)c(=)k(buffer;)123 2610 y(sharedBuffHandle->count)c(=)k(count;)123 2667 y(sharedBuffHandle->dataTy)o(pe)d(=)j(dataType;)e(/*)i(Dup)f(here?)h (*/)-32 21 y Fb(1)-32 78 y(2)-32 134 y(3)-32 191 y(4)-32 247 y(5)-32 303 y(6)-32 360 y(7)-32 416 y(8)-32 473 y(9)-40 529 y(10)-40 586 y(11)-40 642 y(12)-40 699 y(13)-40 755 y(14)-40 812 y(15)-40 868 y(16)-40 924 y(17)-40 981 y(18)-40 1037 y(19)-40 1094 y(20)-40 1150 y(21)-40 1207 y(22)-40 1263 y(23)-40 1320 y(24)-40 1376 y(25)-40 1433 y(26)-40 1489 y(27)-40 1545 y(28)-40 1602 y(29)-40 1658 y(30)-40 1715 y(31)-40 1771 y(32)-40 1828 y(33)-40 1884 y(34)-40 1941 y(35)-40 1997 y(36)-40 2054 y(37)-40 2110 y(38)-40 2166 y(39)-40 2223 y(40)-40 2279 y(41)-40 2336 y(42)-40 2392 y(43)-40 2449 y(44)-40 2505 y(45)-40 2562 y(46)-40 2618 y(47)-40 2675 y(48)p eop %%Page: 4 3 4 2 bop 75 -125 a Fh(4)75 20 y Fa(MPI)p 160 20 14 2 v 16 w(FREE)p 286 20 V 16 w(SHARED)p 480 20 V 15 w(BUFFER)13 b(\(sha)o(redBu\013Handle\))117 97 y Fd(INOUT)62 b Fa(sha)o (redBu\013Handle)256 b Fd(A)14 b(shared)h(bu\013er)g(handle)75 269 y Fc(int)23 b(MPI_Free_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle\))75 382 y(MPI_FREE_SHARED_BUFFER)f (\(SHARED_BUFF_HANDLE,)g(IERROR\))147 439 y(INTEGER)h (SHARED_BUFF_HANDLE)147 495 y(INTEGER)g(IERROR)166 599 y Fh(This)d(function)h(frees)f(the)h(bu\013er)h(asso)q(ciated)d(with)h (the)h(shared)g(bu\013er)h(handler,)f(the)g(handle)75 655 y(itself,)15 b(and)i(satis\014es)f(an)o(y)g(outstanding)g(comm)o (unications.)21 b(This)16 b(function)g(needs)i(to)e(b)q(e)h(view)o(ed)f (as)75 712 y("bi-collectiv)o(e".)75 816 y Fc(Possible)23 b(Send/Recv)f(Implementation:)75 929 y(int)h(MPI_Make_shared_buffer)e (\(MPI_Datatype)h(dataType,)h(int,)g(count,)361 985 y(MPI_Comm)g(comm,) g(int)h(source,)f(int)g(dest,)361 1041 y(void)h(*buffer,)e (MPI_Shared_buff)g(*sharedBuffHandle\))123 1098 y({)123 1154 y(/*)h(Ok,)h(ok)f(this)h(isn't)f(really)g(C,)g(but)h(you)f(get)h (the)f(idea\))g(*/)123 1211 y(sharedBuffHandle)e(=)j(malloc)f (\(sizeof\(sharedBuffHandle\)\);)123 1267 y(*buffer)g(=)g(malloc)g (\(sizeof\(dataType\))f(*)i(count\);)123 1324 y (sharedBuffHandle->buffer)c(=)k(buffer;)123 1380 y (sharedBuffHandle->count)c(=)k(count;)123 1437 y (sharedBuffHandle->dataTy)o(pe)d(=)j(dataType;)e(/*)i(Dup)f(here?)h(*/) 123 1493 y(sharedBuffHandle->tag)d(=)i(getSharedBuffTag\(\);)f(/*)h (Some)h(magic?)f(*/)123 1550 y(/*)g(At)h(this)f(point,)g(a)h(robust)f (application)f(might)i(check)f(with)g(it's)g(partner)194 1606 y(to)h(see)f(if)h(the)f(parameters)g(are)g(correct)g(and)h(the)f (malloc)g(worked.)g(But)h(it)194 1662 y(would)f(take)h(too)f(much)h (space)f(\(besides)g(if)g(programming)f(text)i(books)194 1719 y(can)g(omit)f(error)g(checking,)g(why)g(can't)h(I?)f(:-\))h(*/) 123 1775 y(if)f(\(myProc)g(==)h(src\))170 1832 y({)170 1888 y(Recv)g(\(comm,)f(sharedBuffHandle->tag,)e(dest,)i(\(count=0,)f (buff=NULL,...\)\);)170 1945 y(sharedBuffHandle->partner)f(=)j(dest;) 170 2001 y(})123 2058 y(else)170 2114 y({)170 2171 y (sharedBuffHandle->waitHandle)c(=)k(IRECV)f(\(comm,)g (sharedBuffHandle->tag,)1077 2227 y(source,)g(buffer,)g(count,...\))170 2283 y(Send)h(\(comm,)f(sharedBuffHandle->tag,)e(source,)i(\(count=0,)f (buff=NULL,...\)\);)170 2340 y(sharedBuffHandle->partner)f(=)j(source;) 170 2396 y(})123 2453 y(return;)123 2509 y(})75 2622 y(int)f(MPI_Acquire_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle\))123 2679 y({)1967 21 y Fb(1)1967 78 y(2)1967 134 y(3)1967 191 y(4)1967 247 y(5)1967 303 y(6)1967 360 y(7)1967 416 y(8)1967 473 y(9)1959 529 y(10)1959 586 y(11)1959 642 y(12)1959 699 y(13)1959 755 y(14)1959 812 y(15)1959 868 y(16)1959 924 y(17)1959 981 y(18)1959 1037 y(19)1959 1094 y(20)1959 1150 y(21)1959 1207 y(22)1959 1263 y(23)1959 1320 y(24)1959 1376 y(25)1959 1433 y(26)1959 1489 y(27)1959 1545 y(28)1959 1602 y(29)1959 1658 y(30)1959 1715 y(31)1959 1771 y(32)1959 1828 y(33)1959 1884 y(34)1959 1941 y(35)1959 1997 y(36)1959 2054 y(37)1959 2110 y(38)1959 2166 y(39)1959 2223 y(40)1959 2279 y(41)1959 2336 y(42)1959 2392 y(43)1959 2449 y(44)1959 2505 y(45)1959 2562 y(46)1959 2618 y(47)1959 2675 y(48)p eop %%Page: 3 4 3 3 bop 1852 -125 a Fh(3)75 20 y(Dest)16 b(then)h(has)f(the)g(same)g (priviledgs)f(as)h(source)g(previously)g(had.)24 b(T)l(o)16 b(access)h(a)f(released)g(bu\013er)h(is)75 77 y(considered)f(to)e(b)q (e)i(erroneous.)k(This)15 b(function)g(needs)h(to)e(b)q(e)i(view)o(ed)f (as)g("bi-collectiv)o(e".)166 216 y Fe(Discussion:)40 b Fd(Do)15 b(w)o(e)h(w)o(an)o(t)f(to)h(ha)o(v)o(e)f(a)g(v)o(ersion)h (that)f(allo)o(ws)g(the)h(user)g(to)g(supply)f(a)g(bu\013er.)24 b(In)16 b(that)75 272 y(ev)o(en)o(t)f(on)f(a)g(distributed)h(memory)d (mac)o(hine)h(this)i(b)q(ecomes)g(a)f("safe")g(in)o(terface)h(to)f (ready)h(send,)g(on)f(a)g(shared)75 329 y(memory)d(mac)o(hine)i(it)g (ma)o(y)f(in)o(tro)q(duce)j(extra)f(copies.)75 562 y Fa(MPI)p 160 562 14 2 v 16 w(A)o(CQUI)o(RE)p 370 562 V 13 w(SHARED)p 562 562 V 16 w(BUFFER)f(\(sha)o(redBu\013Handle\))117 640 y Fd(INOUT)62 b Fa(sha)o(redBu\013Handle)256 b Fd(A)14 b(shared)h(bu\013er)g(handle)75 814 y Fc(int)23 b (MPI_Acquire_shared_buffer)e(\(MPI_Shared_buff)h(*sharedBuffHandle\))75 927 y(MPI_ACQUIRE_SHARED_BUFFER)e(\(SHARED_BUFF_HANDLE,)i(IERROR\))147 983 y(INTEGER)g(SHARED_BUFF_HANDLE)147 1040 y(INTEGER)g(IERROR)166 1146 y Fh(This)15 b(routine)f(w)o(aits)g(for)h(the)g(shared)g(bu\013er) h(to)f(b)q(e)h(released)f(b)o(y)g(the)g(partner.)20 b(It)15 b(is)g(a)g(blo)q(c)o(king)75 1202 y(call.)36 b(When)21 b(it)f(returns,)h(the)g(bu\013er)h(asso)q(ciated)e(with)g(the)h(shared) g(bu\013er)g(handle)g(can)g(b)q(e)h(fully)75 1259 y(accessed.)e(A)12 b(subsequen)o(t)h(acquire)f(without)f(an)i(in)o(terv)o(ening)e(release) h(is)g(erroneous)g(\(this)f(includes)i(an)75 1315 y(acquire)h(b)o(y)h (the)g(source)f(immediately)e(after)i(the)g(mak)o(e)p 1062 1315 V 15 w(shared)p 1206 1315 V 17 w(bu\013er\).)20 b(An)15 b(acquire)f(will)g(correctly)75 1372 y(return)h(only)g(when)h (the)f(partner)g(pro)q(cess)g(p)q(erforms)f(a)h(release.)75 1523 y Fa(MPI)p 160 1523 V 16 w(RELEASE)p 367 1523 V 16 w(SHARED)p 561 1523 V 16 w(BUFFER)e(\(sha)o(redBu\013Handle,)j (releaseAction\))117 1600 y Fd(INOUT)62 b Fa(sha)o(redBu\013Handle)256 b Fd(A)14 b(shared)h(bu\013er)g(handle)117 1675 y(IN)155 b Fa(releaseAction)337 b Fd(Indicate)11 b(whether)g(a)f(pro)q(duce)h (\(MPI)p 1501 1675 13 2 v 16 w(PR)o(ODUCE\))f(or)g(con-)905 1731 y(sume)i(\(MPI)p 1109 1731 V 15 w(CONSUME\))i(action)e(is)g(asso)q (ciated)h(with)f(the)905 1788 y(release)75 1962 y Fc(int)23 b(MPI_Release_shared_buffer)e(\(MPI_Shared_buff)h(*sharedBuffHandle,) 314 2019 y(int)h(releaseAction\))75 2132 y(MPI_RELEASE_SHARED_BUFFER)d (\(SHARED_BUFF_HANDLE,)i(RELEASE_ACTION,)f(IERROR\))147 2188 y(INTEGER)h(SHARED_BUFF_HANDLE)147 2244 y(INTEGER)g (RELEASE_ACTION)147 2301 y(INTEGER)g(IERROR)166 2407 y Fh(This)10 b(routine)h(releases)f(a)h(shared)g(bu\013er)h(so)e(that)g (it)g(can)h(b)q(e)h(acquired)f(b)o(y)g(the)f(partner)h(pro)q(cess.)19 b(If)75 2464 y(the)11 b(pro)q(cess)h(has)f(only)g(read)g(the)g (bu\013er)i(asso)q(ciated)d(with)h(the)g(shared)g(bu\013er)i(handler,)e (it)g(ma)o(y)e(use)j(the)75 2520 y(releaseAction)j(of)h(MPI)p 500 2520 14 2 v 16 w(CONSUME.)g(If)g(the)h(pro)q(cess)f(has)g(written)f (to)g(the)h(bu\013er)h(asso)q(ciated)f(with)75 2577 y(the)e(shared)g (bu\013er)i(handler,)e(it)f(MUST)h(use)g(the)h(releaseAction)e(of)h (MPI)p 1369 2577 V 16 w(PR)o(ODUCE)f(to)h(guaran)o(tee)75 2633 y(that)g(the)i(partner)f(sees)g(the)g(c)o(hanges.)-32 21 y Fb(1)-32 78 y(2)-32 134 y(3)-32 191 y(4)-32 247 y(5)-32 303 y(6)-32 360 y(7)-32 416 y(8)-32 473 y(9)-40 529 y(10)-40 586 y(11)-40 642 y(12)-40 699 y(13)-40 755 y(14)-40 812 y(15)-40 868 y(16)-40 924 y(17)-40 981 y(18)-40 1037 y(19)-40 1094 y(20)-40 1150 y(21)-40 1207 y(22)-40 1263 y(23)-40 1320 y(24)-40 1376 y(25)-40 1433 y(26)-40 1489 y(27)-40 1545 y(28)-40 1602 y(29)-40 1658 y(30)-40 1715 y(31)-40 1771 y(32)-40 1828 y(33)-40 1884 y(34)-40 1941 y(35)-40 1997 y(36)-40 2054 y(37)-40 2110 y(38)-40 2166 y(39)-40 2223 y(40)-40 2279 y(41)-40 2336 y(42)-40 2392 y(43)-40 2449 y(44)-40 2505 y(45)-40 2562 y(46)-40 2618 y(47)-40 2675 y(48)p eop %%Page: 2 5 2 4 bop 75 -125 a Fh(2)75 20 y Fc(!)24 b(Information)e(produced)h (before)214 b(...Read)23 b(buff)g(and)g(do)h(something...)75 77 y(!)g(creating)e(more)i(data)75 133 y(ACQUIRE_SHARED_BUFFER)d (\(sbh\))262 b(!)24 b(Process)e(1)i(releases)f(the)g(buffer,)g(but)982 190 y(!)h(has)f(not)g(modified)g(it)h(\(CONSUMEd\))982 246 y(RELEASE_SHARED_BUFFER)d(\(sbh,)i(CONSUME\))75 302 y(!)h(At)f(this)h(point)f(process)g(0)g(unblocks)75 359 y(!)h(and)f(is)h(free)f(to)h(modify)f(buff)314 415 y(...)739 b(...)75 472 y(FREE_SHARED_BUFF\(sbh\))403 b(FREE_SHARED_BUFF\(sbh\))75 673 y Fa(MPI)p 160 673 14 2 v 16 w(MAKE)p 305 673 V 16 w(SHARED)p 499 673 V 15 w(BUFFER)13 b(\(dataT)l(yp)q(e,)i(count,)h (comm,)c(source,)j(dest,)g(bu\013er,)h(sha)o(redBu\013Han-)75 729 y(dle\))117 806 y Fd(IN)155 b Fa(dataT)l(yp)q(e)412 b Fd(MPI)14 b(Data)g(T)o(yp)q(e)g(\(con)o(tiguous\))117 881 y(IN)155 b Fa(count)482 b Fd(Num)o(b)q(er)14 b(of)f(dataT)o(yp)q (es)117 957 y(IN)155 b Fa(comm)470 b Fd(Comm)o(unicator)117 1032 y(IN)155 b Fa(source)468 b Fd(First)14 b(pro)q(cess)i(in)e(pair)f (\(initially)f(o)o(wns)h(bu\013er\))117 1107 y(IN)155 b Fa(dest)510 b Fd(Second)15 b(pro)q(cess)h(in)d(pair)117 1182 y(IN)155 b Fa(tag)526 b Fd(T)m(ag)13 b(for)h(use)g(in)g(comm)o (unicatio)o(ns)e(and)h(for)h(uniqueness)117 1257 y(OUT)108 b Fa(bu\013er)480 b Fd(Bu\013er)15 b(creaated)h(b)o(y)d(mak)o(e)p 1352 1257 13 2 v 14 w(shared)p 1483 1257 V 16 w(bu\013er)117 1332 y(OUT)108 b Fa(sha)o(redBu\013Handle)256 b Fd(Handle)14 b(to)g(describ)q(e)h(shared)g(ob)r(ject)75 1506 y Fc(int)23 b(MPI_Make_shared_buffer)e(\(MPI_Datatype)h(dataType,)h(int,)g(count,) 361 1563 y(MPI_Comm)g(comm,)g(int)h(source,)f(int)g(dest,)361 1619 y(void)h(*buffer,)e(MPI_Shared_buff)g(*sharedBuffHandle\))75 1732 y(MPI_MAKE_SHARED_BUFFER)f(\(DATA_TYPE,)h(COUNT,)h(COMM,)g (SOURCE,)g(DEST,)624 1789 y(BUFF_REF,)f(BUFFER_IX,)h (SHARED_BUFF_HANDLE,)e(IERROR\))147 1845 y(INTEGER)h(DATA_TYPE)147 1902 y(INTEGER)g(COUNT)147 1958 y(INTEGER)g(COMM)147 2015 y(INTEGER)g(SOURCE)147 2071 y(INTEGER)g(DEST)147 2127 y(INTEGER)g(BUFF_REFF\(*\))147 2184 y(INTEGER)g(BUFFER_IX)147 2240 y(INTEGER)g(SHARED_BUFF_HANDLE)147 2297 y(INTEGER)g(IERROR)166 2403 y Fh(This)e(routine)h(creates)f(a)h(bu\013er)h(shared)f(b)q(et)o (w)o(een)g(t)o(w)o(o)e(pro)q(cesses.)38 b(The)21 b(pro)q(cess)g(iden)o (ti\014ed)75 2460 y(b)o(y)c(source)g(initially)e("o)o(wns")h(the)h (bu\013er.)27 b(Source)18 b(ma)o(y)d(write)h(to)h(the)g(bu\013er)h(and) g(manipulate)d(the)75 2516 y(bu\013er)g(in)g(an)o(y)f(w)o(a)o(y)f(that) h(mak)o(es)f(sense)i(with)f(standard)g(language)g(access)g(\(assignmen) o(t)e(statemen)o(ts,)75 2572 y(functions)18 b(calls,)f(etc.\).)28 b(When)19 b(source)f(is)f(done)i(with)e(the)h(bu\013er,)i(it)d(m)o(ust) g(release)g(the)i(bu\013er)g(for)75 2629 y(dest)h(to)f(acquire)g(it.)33 b(Dest)19 b(\(and)h(only)f(dest\))g(can)h(acquire)g(the)f(bu\013er)i (once)f(released)g(b)o(y)f(source.)1967 21 y Fb(1)1967 78 y(2)1967 134 y(3)1967 191 y(4)1967 247 y(5)1967 303 y(6)1967 360 y(7)1967 416 y(8)1967 473 y(9)1959 529 y(10)1959 586 y(11)1959 642 y(12)1959 699 y(13)1959 755 y(14)1959 812 y(15)1959 868 y(16)1959 924 y(17)1959 981 y(18)1959 1037 y(19)1959 1094 y(20)1959 1150 y(21)1959 1207 y(22)1959 1263 y(23)1959 1320 y(24)1959 1376 y(25)1959 1433 y(26)1959 1489 y(27)1959 1545 y(28)1959 1602 y(29)1959 1658 y(30)1959 1715 y(31)1959 1771 y(32)1959 1828 y(33)1959 1884 y(34)1959 1941 y(35)1959 1997 y(36)1959 2054 y(37)1959 2110 y(38)1959 2166 y(39)1959 2223 y(40)1959 2279 y(41)1959 2336 y(42)1959 2392 y(43)1959 2449 y(44)1959 2505 y(45)1959 2562 y(46)1959 2618 y(47)1959 2675 y(48)p eop %%Page: 1 6 1 5 bop 1852 -125 a Fh(1)75 20 y Ff(0.1)59 b(Prop)r(osal:)26 b(Tw)n(o)c(Pro)r(cess)e("Sha)n(red")j(Bu\013ers)75 122 y Fh(It)15 b(is)f(not)h(uncommon)e(for)h(a)g(pair)h(of)f(pro)q(cesses)h (to)g(engage)f(in)h(frequen)o(t)g(pre-determined)f(comm)o(uni-)75 178 y(cations.)19 b(On)14 b(example)f(of)h(suc)o(h)h(a)f(co)q(de)h(is)e (a)h(pip)q(eline)h(algorithm)c(in)j(whic)o(h)g(pro)q(cess)g(A)h (creates)f(data)75 235 y(that)i(is)g(subsequen)o(tly)h(used)g(b)o(y)f (pro)q(cess)h(B.)f(Another)g(example)g(is)g(a)g(\014nite)g (di\013erence)i(appro)o(xima-)75 291 y(tion)12 b(mo)q(del)g(when)i (comm)o(unicating)c(b)q(oundary)j(v)m(alues)g(\(a.k.a.)18 b(ghost)12 b(p)q(oin)o(ts\).)18 b(On)c(a)f(mac)o(hine)f(with)75 347 y(shared)18 b(memory)l(,)e(it)h(is)h(p)q(ossible)g(to)f(share)h(a)f (common)f(bu\013er)j(to)e(use)i(for)e(exc)o(hanging)h(suc)o(h)g(data.) 75 404 y(Ownership)13 b(of)g(the)g(bu\013er)h(m)o(ust)d(b)q(e)j (established)e(in)h(order)g(for)f(a)h(pro)q(cess)g(to)f(safely)g (\014ll)h(or)f(empt)o(y)g(the)75 460 y(bu\013er.)26 b(This)16 b(can)h(b)q(e)g(expressed)h(as)e(a)h(pro)q(duce/comsume)e (relationship.)23 b(This)17 b(functionalit)o(y)e(can)75 517 y(b)q(e)f(expressed)f(in)g(suc)o(h)g(a)g(w)o(a)o(y)f(that)g(it)h (should)g(b)q(e)g(no)g(less)g(e\016cien)o(t)f(when)i(using)f(MPI)g (Send/Recv)g(on)75 573 y(a)i(distributed)g(memory)d(mac)o(hine)i(and)i (is)e(simply)g(a)h(p)q(oin)o(ter)f(pass)h(on)g(shared)h(memory)c(mac)o (hines.)166 706 y Fe(Discussion:)34 b Fd(This)13 b(prop)q(osal)g(is)h (prompted)f(b)o(y)g(T)m(on)o(y)g(Skjellum's)e(prop)q(osal)i(titled:)18 b("In\014nite)13 b(asymp-)75 756 y(totic)h(bandwidth)f(for)h(thread)h (comm)o(uni)o(cation")166 805 y(W)m(e)j(ha)o(v)o(e)h(a)g(similia)o(r)e (proto)q(col)h(inside)h(our)g(system)g(when)g(w)o(e)g(w)o(ork)g(on)f (shared)i(memory)c(mac)o(hines.)75 855 y(Imp)q(ortan)o(t)i(to)i(us)g (is)f(not)h(only)e(the)j(bandwidth)e(and)g(latency)m(,)i(but)e(also)g (ligh)o(t)o(w)o(eigh)o(t)f(\015o)o(w-con)o(trol.)35 b(The)75 905 y(prop)q(osal)14 b(in)g(its)g(curren)o(t)i(form)c(solv)o(es)j(our)f (problem)f(in)h(particular,)g(but)g(w)o(e)h(feel)f(it)g(ma)o(y)e (prompt)h(discussion)75 955 y(that)i(can)f(shap)q(e)i(this)f(prop)q (osal)f(and)g(T)m(on)o(y's)g(prop)q(osal)g(in)o(to)g(something)f(that)i (will)e(pro)o(vide)i(a)f(p)q(ortable)h(high)75 1005 y(bandwidth/lo)o (w-latency)c(mec)o(hanism)e(on)j(true)h(SMPs)f(as)h(w)o(ell)e(as)h(pro) o(vide)g(high)f(p)q(erformance)h(in)f(distributed)75 1054 y(memory)g(mac)o(hines.)166 1104 y(Questions)k(for)e(discussion:) 166 1154 y(1\))i(Is)h(this)f(really)g(going)f(to)h(b)q(e)h(faster)g(on) f(a)g(shared)h(memory)d(mac)o(hine)h(with)h(a)g("high-qualit)o(y")e (imple-)75 1204 y(men)o(tation.)166 1254 y(2\))h(Do)q(es)g(the)g(user)h (visible)f(\015o)o(w-con)o(trol)f(pro)o(vide)g(signi\014can)o(t)h (added)g(v)n(alue?)166 1304 y(3\))19 b(F)m(or)f(b)q(oth)h(shared)g(and) g(distributed)g(memory)e(mac)o(hines,)h(do)q(es)i(prede\014ning)f(the)g (comm)o(unication)75 1353 y(pattern)c(pro)o(vide)e(options)h(for)f (optimization?)166 1403 y(Leslie)h(Hart)166 1453 y(T)m(om)e(Henderson) 166 1509 y(NO)o(AA/FSL)75 1698 y Fc(Example:)75 1811 y(Process)23 b(0:)668 b(Process)23 b(1:)75 1924 y(!)h(Make)f(a)h (shared)f(buffer)g(with)g(process)g(0)h(as)f(the)h(source,)f(process)g (1)g(as)h(the)f(dest)75 1981 y(!)h(sbh)f(is)h(the)f(shared)g(buffer)g (handle)g(and)h(buff)f(is)h(some)f(memory)g(allocated)75 2037 y(!)h(by)f(MAKE_SHARED_BUFFER)75 2094 y(MAKE_SHARED_BUFFER)e (\(...,)334 b(MPI_MAKE_SHARED_BUFFER)21 b(\(...,)218 2150 y(0,)j(1,)f(buff,)h(sbh\))619 b(0,)24 b(1,)g(buff,)f(sbh\))75 2206 y(!)h(Process)f(0)g(owns)h(the)f(buffer)262 b(!)24 b(Process)e(1)i(waits)f(for)h(buffer)99 2263 y(...Fill)f(buff)g(with)g (useful)g(stuff...)95 b(ACQUIRE_SHARED_BUFFER)21 b(\(sbh\))75 2376 y(!)j(Process)f(1)g(releases)g(the)h(buffer)75 2432 y(!)g(communicating)e(that)h(it)h(was)75 2489 y(!)g(modified)e (\(PRODUCEd\))75 2545 y(RELEASE_SHARED_BUFFER)f(\(sbh,)i(PRODUCE\))982 2602 y(!)h(At)f(this)g(point)h(process)f(1)g(unblocks)75 2658 y(!)h(Wait)f(for)g(process)g(0)h(to)g(use)f(the)167 b(!)24 b(and)f(has)g(full)h(access)f(to)g(buff)-32 21 y Fb(1)-32 78 y(2)-32 134 y(3)-32 191 y(4)-32 247 y(5)-32 303 y(6)-32 360 y(7)-32 416 y(8)-32 473 y(9)-40 529 y(10)-40 586 y(11)-40 642 y(12)-40 699 y(13)-40 755 y(14)-40 812 y(15)-40 868 y(16)-40 924 y(17)-40 981 y(18)-40 1037 y(19)-40 1094 y(20)-40 1150 y(21)-40 1207 y(22)-40 1263 y(23)-40 1320 y(24)-40 1376 y(25)-40 1433 y(26)-40 1489 y(27)-40 1545 y(28)-40 1602 y(29)-40 1658 y(30)-40 1715 y(31)-40 1771 y(32)-40 1828 y(33)-40 1884 y(34)-40 1941 y(35)-40 1997 y(36)-40 2054 y(37)-40 2110 y(38)-40 2166 y(39)-40 2223 y(40)-40 2279 y(41)-40 2336 y(42)-40 2392 y(43)-40 2449 y(44)-40 2505 y(45)-40 2562 y(46)-40 2618 y(47)-40 2675 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Thu May 30 17:32:40 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA03696; Thu, 30 May 1996 17:32:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA22120 for mpi-core-out; Thu, 30 May 1996 16:35:48 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA22111 for ; Thu, 30 May 1996 16:35:37 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA06006; Thu, 30 May 96 15:35:35 MDT From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9605302135.AA06006@nipmuc.fsl.noaa.gov> Subject: Proposal for two processes to "share" a buffer (TeX) To: mpi-core@mcs.anl.gov Date: Thu, 30 May 1996 15:35:35 -0600 (MDT) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 12711 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This proposal is prompted by Tony Skjellum's proposal titled: "Infinite asymptotic bandwidth for thread communication" We have a similiar protocol inside our system when we work on shared memory machines. Important to us is not only the bandwidth and latency, but also lightweight flow-control. The proposal in its current form solves our problem in particular, but we feel it may prompt discussion that can shape this proposal and Tony's proposal into something that will provide a portable high bandwidth/low-latency mechanism on true SMPs as well as provide high performance in distributed memory machines. Questions for discussion: 1) Is this really going to be faster on a shared memory machine with a "high-quality" implementation. 2) Does the user visible flow-control provide significant added value? 3) For both shared and distributed memory machines, does predefining the communication pattern provide options for optimization? Leslie Hart Tom Henderson NOAA/FSL \section{Proposal: Two Process "Shared" Buffers} It is not uncommon for a pair of processes to engage in frequent pre-determined communications. On example of such a code is a pipeline algorithm in which process A creates data that is subsequently used by process B. Another example is a finite difference approximation model when communicating boundary values (a.k.a. ghost points). On a machine with shared memory, it is possible to share a common buffer to use for exchanging such data. Ownership of the buffer must be established in order for a process to safely fill or empty the buffer. This can be expressed as a produce/comsume relationship. This functionality can be expressed in such a way that it should be no less efficient when using MPI Send/Recv on a distributed memory machine and is simply a pointer pass on shared memory machines. \discuss{ This proposal is prompted by Tony Skjellum's proposal titled: "Infinite asymptotic bandwidth for thread communication" We have a similiar protocol inside our system when we work on shared memory machines. Important to us is not only the bandwidth and latency, but also lightweight flow-control. The proposal in its current form solves our problem in particular, but we feel it may prompt discussion that can shape this proposal and Tony's proposal into something that will provide a portable high bandwidth/low-latency mechanism on true SMPs as well as provide high performance in distributed memory machines. Questions for discussion: 1) Is this really going to be faster on a shared memory machine with a "high-quality" implementation. 2) Does the user visible flow-control provide significant added value? 3) For both shared and distributed memory machines, does predefining the communication pattern provide options for optimization? Leslie Hart Tom Henderson NOAA/FSL } \begin{verbatim} Example: Process 0: Process 1: ! Make a shared buffer with process 0 as the source, process 1 as the dest ! sbh is the shared buffer handle and buff is some memory allocated ! by MAKE_SHARED_BUFFER MAKE_SHARED_BUFFER (..., MPI_MAKE_SHARED_BUFFER (..., 0, 1, buff, sbh) 0, 1, buff, sbh) ! Process 0 owns the buffer ! Process 1 waits for buffer ...Fill buff with useful stuff... ACQUIRE_SHARED_BUFFER (sbh) ! Process 1 releases the buffer ! communicating that it was ! modified (PRODUCEd) RELEASE_SHARED_BUFFER (sbh, PRODUCE) ! At this point process 1 unblocks ! Wait for process 0 to use the ! and has full access to buff ! Information produced before ...Read buff and do something... ! creating more data ACQUIRE_SHARED_BUFFER (sbh) ! Process 1 releases the buffer, but ! has not modified it (CONSUMEd) RELEASE_SHARED_BUFFER (sbh, CONSUME) ! At this point process 0 unblocks ! and is free to modify buff ... ... FREE_SHARED_BUFF(sbh) FREE_SHARED_BUFF(sbh) \end{verbatim} \begin{funcdef}{MPI\_MAKE\_SHARED\_BUFFER (dataType, count, comm, source, dest, buffer, sharedBuffHandle)} \funcarg{\IN}{dataType}{MPI Data Type (contiguous)} \funcarg{\IN}{count}{Number of dataTypes} \funcarg{\IN}{comm}{Communicator} \funcarg{\IN}{source}{First process in pair (initially owns buffer)} \funcarg{\IN}{dest}{Second process in pair} \funcarg{\IN}{tag}{Tag for use in communications and for uniqueness} \funcarg{\OUT}{buffer}{Buffer creaated by make\_shared\_buffer} \funcarg{\OUT}{sharedBuffHandle}{Handle to describe shared object} \end{funcdef} \begin{verbatim} int MPI_Make_shared_buffer (MPI_Datatype dataType, int, count, MPI_Comm comm, int source, int dest, void *buffer, MPI_Shared_buff *sharedBuffHandle) MPI_MAKE_SHARED_BUFFER (DATA_TYPE, COUNT, COMM, SOURCE, DEST, BUFF_REF, BUFFER_IX, SHARED_BUFF_HANDLE, IERROR) INTEGER DATA_TYPE INTEGER COUNT INTEGER COMM INTEGER SOURCE INTEGER DEST INTEGER BUFF_REFF(*) INTEGER BUFFER_IX INTEGER SHARED_BUFF_HANDLE INTEGER IERROR \end{verbatim} This routine creates a buffer shared between two processes. The process identified by source initially "owns" the buffer. Source may write to the buffer and manipulate the buffer in any way that makes sense with standard language access (assignment statements, functions calls, etc.). When source is done with the buffer, it must release the buffer for dest to acquire it. Dest (and only dest) can acquire the buffer once released by source. Dest then has the same priviledgs as source previously had. To access a released buffer is considered to be erroneous. This function needs to be viewed as "bi-collective". \discuss{ Do we want to have a version that allows the user to supply a buffer. In that event on a distributed memory machine this becomes a "safe" interface to ready send, on a shared memory machine it may introduce extra copies. } %\discuss{ %Do we want to have the tag "consumed"? In an implementation layered over %MPI\_Send/Recv, we need a tag for communications. It could be in a hidden %context or we could simply require that the tag not be used when communicating %with the partner process. %} \begin{funcdef}{MPI\_ACQUIRE\_SHARED\_BUFFER (sharedBuffHandle)} \funcarg{\INOUT}{sharedBuffHandle}{A shared buffer handle} \end{funcdef} \begin{verbatim} int MPI_Acquire_shared_buffer (MPI_Shared_buff *sharedBuffHandle) MPI_ACQUIRE_SHARED_BUFFER (SHARED_BUFF_HANDLE, IERROR) INTEGER SHARED_BUFF_HANDLE INTEGER IERROR \end{verbatim} This routine waits for the shared buffer to be released by the partner. It is a blocking call. When it returns, the buffer associated with the shared buffer handle can be fully accessed. A subsequent acquire without an intervening release is erroneous (this includes an acquire by the source immediately after the make\_shared\_buffer). An acquire will correctly return only when the partner process performs a release. \begin{funcdef}{MPI\_RELEASE\_SHARED\_BUFFER (sharedBuffHandle, releaseAction)} \funcarg{\INOUT}{sharedBuffHandle}{A shared buffer handle} \funcarg{\IN}{releaseAction}{Indicate whether a produce (MPI\_PRODUCE) or consume (MPI\_CONSUME) action is associated with the release} \end{funcdef} \begin{verbatim} int MPI_Release_shared_buffer (MPI_Shared_buff *sharedBuffHandle, int releaseAction) MPI_RELEASE_SHARED_BUFFER (SHARED_BUFF_HANDLE, RELEASE_ACTION, IERROR) INTEGER SHARED_BUFF_HANDLE INTEGER RELEASE_ACTION INTEGER IERROR \end{verbatim} This routine releases a shared buffer so that it can be acquired by the partner process. If the process has only read the buffer associated with the shared buffer handler, it may use the releaseAction of MPI\_CONSUME. If the process has written to the buffer associated with the shared buffer handler, it MUST use the releaseAction of MPI\_PRODUCE to guarantee that the partner sees the changes. \begin{funcdef}{MPI\_FREE\_SHARED\_BUFFER (sharedBuffHandle)} \funcarg{\INOUT}{sharedBuffHandle}{A shared buffer handle} \end{funcdef} \begin{verbatim} int MPI_Free_shared_buffer (MPI_Shared_buff *sharedBuffHandle) MPI_FREE_SHARED_BUFFER (SHARED_BUFF_HANDLE, IERROR) INTEGER SHARED_BUFF_HANDLE INTEGER IERROR \end{verbatim} This function frees the buffer associated with the shared buffer handler, the handle itself, and satisfies any outstanding communications. This function needs to be viewed as "bi-collective". \begin{verbatim} Possible Send/Recv Implementation: int MPI_Make_shared_buffer (MPI_Datatype dataType, int, count, MPI_Comm comm, int source, int dest, void *buffer, MPI_Shared_buff *sharedBuffHandle) { /* Ok, ok this isn't really C, but you get the idea) */ sharedBuffHandle = malloc (sizeof(sharedBuffHandle)); *buffer = malloc (sizeof(dataType) * count); sharedBuffHandle->buffer = buffer; sharedBuffHandle->count = count; sharedBuffHandle->dataType = dataType; /* Dup here? */ sharedBuffHandle->tag = getSharedBuffTag(); /* Some magic? */ /* At this point, a robust application might check with it's partner to see if the parameters are correct and the malloc worked. But it would take too much space (besides if programming text books can omit error checking, why can't I? :-) */ if (myProc == src) { Recv (comm, sharedBuffHandle->tag, dest, (count=0, buff=NULL,...)); sharedBuffHandle->partner = dest; } else { sharedBuffHandle->waitHandle = IRECV (comm, sharedBuffHandle->tag, source, buffer, count,...) Send (comm, sharedBuffHandle->tag, source, (count=0, buff=NULL,...)); sharedBuffHandle->partner = source; } return; } int MPI_Acquire_shared_buffer (MPI_Shared_buff *sharedBuffHandle) { wait (sharedBuffHandle->waitHandle); } int MPI_Release_shared_buffer (MPI_Shared_buff *sharedBuffHandle, int releaseAction) { if (releaseAction == MPI_CONSUME) count = 0; else count = sharedBuffHandle->count; sharedBuffHandle->waitHandle = IRECV (comm, sharedBuffHandle->tag, sharedBuffHandle->partner, sharedBuffHandle->buffer, count,...); /* Note that a non-buffering send can be used here */ /* Note that a "ready" protocol can also be used since we guarantee the receive is always posted before the send */ localHandle = ISSEND (comm, sharedBuffHandle->tag, sharedBuffHandle->partner, sharedBuffHandle->buffer, count,...); freeHandle (localHandle); } int MPI_Free_shared_buffer (MPI_Shared_buff *sharedBuffHandle) { /* Either cancel the outstanding receive or (if you don't beleive cancel works) send to complete the receive. */ /* Free all structures */ } \end{verbatim} \begin{verbatim} Possible Shared Memory Implementation: int MPI_Make_shared_buffer (MPI_Datatype dataType, int, count, MPI_Comm comm, int source, int dest, void *buffer, MPI_Shared_buff *sharedBuffHandle) { /* Ok, ok this isn't really C, but you get the idea) */ sharedBuffHandle = malloc (sizeof(sharedBuffHandle)); if (myProc == source) { sharedBuffHandle->semaphore = shared_malloc (sizeof(dataType) * count + cacheLineSize); } sharedBuffHandle->buffer = buffer; sharedBuffHandle->count = count; sharedBuffHandle->dataType = dataType; /* Dup here? */ if (myProc == source) { i = myProc; memcpy (sharedBuffHandle->semaphore, &i, sizeof(int)); sharedBuffHandle->partner = dest; Send (comm, sharedTag, dest, sharedBuffHandle->semaphore, MPI_Pointer, ...); } else { sharedBuffHandle->partner = source; Recv (comm, sharedTag, source, sharedBuffHandle->semaphore, MPI_Pointer, ...); } *buffer = ((char *) sharedBuffHandle->semaphore) + cacheLineSize; } int MPI_Acquire_shared_buffer (MPI_Shared_buff *sharedBuffHandle) { i = myProc - 1; while (i != myProc) { memcpy (&i, sharedBuffHandle->semaphore, sizeof(int)); /* A smart implementation would release it's timeslice after some number of times through this loop */ } } int MPI_Release_shared_buffer (MPI_Shared_buff *sharedBuffHandle, int releaseAction) { i = sharedBuffHandle->partner; memcpy (sharedBuffHandle->semaphore, &i, sizeof(int)); } int MPI_Free_shared_buffer (MPI_Shared_buff *sharedBuffHandle) { /* Free all structures */ } \end{verbatim} From mpi-core-human@mcs.anl.gov Thu May 30 20:53:13 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA06207; Thu, 30 May 1996 20:53:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA25222 for mpi-core-out; Thu, 30 May 1996 19:56:13 -0500 Received: from win119.nas.nasa.gov (win119.nas.nasa.gov [129.99.66.85]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA25217 for ; Thu, 30 May 1996 19:56:07 -0500 Received: from win119 (localhost [127.0.0.1]) by win119.nas.nasa.gov (8.6.12/NAS.6.1) with SMTP id RAA29677; Thu, 30 May 1996 17:56:01 -0700 Message-ID: <31AE43A0.167E@nas.nasa.gov> Date: Thu, 30 May 1996 17:56:00 -0700 From: "David C. DiNucci" X-Mailer: Mozilla 2.01 (X11; U; IRIX 5.3 IP22) MIME-Version: 1.0 To: Leslie Hart CC: mpi-core@mcs.anl.gov Subject: Re: Proposal for two processes to "share" a buffer (TeX) References: <9605302135.AA06006@nipmuc.fsl.noaa.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Leslie Hart wrote: > > This proposal is prompted by Tony Skjellum's proposal titled: "Infinite > asymptotic bandwidth for thread communication" > Tony's original proposal mentioned an RK-like package that we (well, I) am working on at NAS called MPK. It's now called CDS1 (Cooperative Data Sharing, Level 1), and there is info at http://www.nas.nasa.gov/NAS/Tools/Projects/CDS (Please don't be thrown by the cutesy graphics.) CDS1 contains a direct superset of the functionality you describe in the NOAA system, but is much more general. Briefly, each process has a "comm heap", out of which regions are allocated (just like MPI_RMA_MALLOC and your MAKE_SHARED_BUFFER), and a set of "comm cells", which are used for actually passing these regions between processes. The comm heap is logically local to each process, the comm cells are logically global. Each comm cell is capable of holding many regions (in reality, pointers to regions) in a queue structure. Communication is performed by putting a region from the comm heap into a comm cell, and allowing another process to retrieve it from there. There are actually 5 basic comm cell operations: write: erase contents of comm cell, replace with new region enq: add new region to end of regions already in cell read: get a copy of first region from comm cell deq: remove first region from comm cell zap: erase contents of comm cell (In CDS1, message passing primitives "send" and "recv" are built on top of these as well, effectively using the comm cells as tags.) In CDS1, multiple processes on the same processor automatically share read access to a *single* region. This is enabled by implementing what might be called "virtual shared regions". A process is obligated to call a special CDS1 routine before modifying a region (in most cases), and CDS1 will make a copy of the region if it is currently being read by another process. (This is, in fact, the only case in which CDS1 makes physical copies on the same processor.) In other words, passing a region (i.e. a pointer) through a cell to another process (or the same process) on the same processor does not create a physical copy of the region -- a reference count is incremented internally and they just both point to the same region -- until one of the processes wants to write to the region. CDS1 also has lots of other stuff -- dynamic process creation, handlers, data translation -- but it also doesn't have lots of stuff that MPI does have -- e.g. communicators and most collective operations. That is being built on top of CDS1 (as was originally envisioned) as "CDS2". This keeps CDS1 small and simple -- about 29 routines total. (CDS1 also doesn't have much use for blocking or synchronous communication, so those are omitted.) Now, having described this, I'm not sure how it would fit into MPI. I would like vendors to support this functionality, but it works in CDS1 partially because it was designed in from the ground floor. I get the sense that adding this functionality to MPI might really mean adding most all of CDS1 to MPI, and MPI is already pretty big. To fit into MPI's style, it will at least need some added typing/automatic data translation arguments. (Translation is a separate step in CDS1.) CDS1 also may not meet some of the requirements that I've been hearing. -Dave -- =============================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst |USMail: NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Group | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Fri May 31 02:58:40 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA10519; Fri, 31 May 1996 02:58:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA28190 for mpi-core-out; Fri, 31 May 1996 02:01:30 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA28185 for ; Fri, 31 May 1996 02:01:19 -0500 Received: from mrjones.engr.sgi.com by sgi.sgi.com via ESMTP (950405.SGI.8.6.12/910110.SGI) id AAA15690; Fri, 31 May 1996 00:00:39 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) id AAA13391; Fri, 31 May 1996 00:00:38 -0700 From: "Eric Salo" Message-Id: <9605310000.ZM13389@mrjones.engr.sgi.com> Date: Fri, 31 May 1996 00:00:37 -0700 In-Reply-To: hart@nipmuc.fsl.noaa.gov (Leslie Hart) "Proposal for two processes to "share" a buffer (TeX)" (May 30, 3:35pm) References: <9605302135.AA06006@nipmuc.fsl.noaa.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: Proposal for two processes to "share" a buffer (TeX) Cc: dinucci@nas.nasa.gov, hart@nipmuc.fsl.noaa.gov (Leslie Hart) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Leslie - If I understand your message correctly, it looks like what you are trying to do is find a way to improve upon the current MPI "best case" of a single-copy algorithm by providing an abstraction that allows for a zero-copy algorithm on some machines and is at worst no slower than send/recv on others. This is an admirable goal but I have some questions: 1) Let's say that the shared buffer is very large. It looks to me like the entire buffer will always be sent to the receiver, even if we only modify a single word. On shared memory systems this is of course not an issue, but I don't think this is likely to be acceptable for NOWs. 2) Now let's say that the shared buffer is very small. Will the handshaking that is required between partners to obtain/free the buffer lock be a source of significant latency, and in general can it be masked by careful coding? I confess that I don't quite understand your sample implementation, but it seems logical that a consumer (for example) would need to receive first a message from the producer stating that the buffer was available, and then potentially a second message containing the new buffer contents. This might or might not be a significant source of delay, depending on how things are "reset" for the next transaction. 3) Limiting each buffer to a single sender and a single receiver seems very painful, although I can see how it simplifies implementations nicely. If we're going to try to simulate a true shared buffer it would be nice to be able to do it for an arbitrary set of readers and writers. (Just a small matter of design and implementation, right?) I can't make up my mind yet whether David's CDS1 does this - I'm still trying to wrap my brain around it - but at first glance it appears to be more of a "pull" design than a "push" design, which again makes me worry about round-trip latency. - Eric -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri May 31 10:47:28 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA14756; Fri, 31 May 1996 10:47:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA02952 for mpi-core-out; Fri, 31 May 1996 09:45:25 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA02916 for ; Fri, 31 May 1996 09:44:41 -0500 Message-Id: <199605311444.JAA02916@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: final attendance poll for June meeting Date: Fri, 31 May 1996 09:44:39 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk By noon today I have to have a final count for the hotel. Here is the list of (51!) people who have told me they are coming. If you are coming to the meeting and are not on this list, or if you are on this list but are not coming, please let me know immediately. Thanks. Rusty Greg Astfalk Ron Brightwell Eric Brunner Margaret Cahir Pang Chen Joel Clark Lyndon Clarke Dennis Cottel Terry Dontje Nathan Doss Mark Fallon Al Geist Bill Gropp Leslie Hart Shane Hebert Rolf Hempel Hans-Christian Hoppe Steve Huss-Lederman Koichi Konishi Susan Kraus Steve Landherr Lloyd Lewins Andrew Lumsdaine Rusty Lusk Brian McCandless Harish Nag Nick Nevin Jarek Nieplocha Bill Nitzberg Ron Oldfield Yoonho Park Perry Partow Elsie Pierce Heidi Poxon Jean-Pierre Prost Rolf Rabenseifner Joe Rieken Eric Salo Bill Saphir Anthony Skjellum Marc Snir Jeff Squyres Rajeev Thakur Marydell Tholburn Simon Tsang Manuel Ujaldon Jerrell Watts Sanjiva Weerawarana Klaus Wolf Parkson Wong Dave Wright From mpi-core-human@mcs.anl.gov Fri May 31 14:04:37 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA17392; Fri, 31 May 1996 14:04:36 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA08091 for mpi-core-out; Fri, 31 May 1996 13:07:05 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA08084 for ; Fri, 31 May 1996 13:06:58 -0500 Message-Id: <199605311806.NAA08084@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: new intro Date: Fri, 31 May 1996 13:06:56 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is a proposal for how we might deal with the state we will find ourselves in when we are finishing up. It is in the new "Introduction" chapter. Comments are welcome. Rusty ______________________________________________________________________ Background Beginning March 1995, the MPI Forum began meeting to consider corrections and extensions to the original MPI Standard document. The first product of these deliberations was Version 1.1 of the MPI specification, released in May of 1995. Since that time, effort has been focused in four types of areas. 1. Further corrections and clarifications for the MPI-1.1 document. 2. Modifications to MPI-1 that do not significantly change its types of functionality (new datatypes, version identification, etc.). 3. Completely new types of functionality (dynamic processes, one-sided communication, etc.) that are what everyone thinks of as ``MPI-2 functionality''. 4. Discussions of areas in which the MPI process and framework seem likely to be useful, but where more discussion and experience are needed before standardization (e.g. Real-time extensions and perhaps I/O). Opinions will vary about what category various sections and subsections belong to. It is proposed that as soon as possible after Supercomputing'96, three documents be produced: + MPI-1.2, containing items of category (1) and perhaps some of (2) above. + MPI-2.0, containing fully-voted-on (reading + two official votes) items in category (3) above, plus the rest of (2). + MPI Working Notes, containing items of category (3) that are still in progress, together with items of category (4). This will make it easier for vendors and users to have a discourse about what level of MPI compliance a given implementation has: + MPI-1 compliance will mean compliance with MPI 1.2. This is a useful level of compliance. The idea is that the original message-passing model, as envisioned during the MPI-1 process, is fully supported, and there will be changes required from Version 1.1, but no significant new types of functionality. + MPI-2 compliance will mean compliance with MPI 2.0. Guidance for future work will be provided in the MPI Working Notes. From mpi-core-human@mcs.anl.gov Fri May 31 17:20:11 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA20923; Fri, 31 May 1996 17:20:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA12382 for mpi-core-out; Fri, 31 May 1996 16:20:30 -0500 Received: from win119.nas.nasa.gov (win119.nas.nasa.gov [129.99.66.85]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA12344; Fri, 31 May 1996 16:19:10 -0500 Received: from win119 (localhost [127.0.0.1]) by win119.nas.nasa.gov (8.6.12/NAS.6.1) with SMTP id OAA02607; Fri, 31 May 1996 14:18:48 -0700 Message-ID: <31AF6237.794B@nas.nasa.gov> Date: Fri, 31 May 1996 14:18:47 -0700 From: "David C. DiNucci" X-Mailer: Mozilla 2.01 (X11; U; IRIX 5.3 IP22) MIME-Version: 1.0 To: Eric Salo CC: mpi-1sided@mcs.anl.gov, mpi-core@mcs.anl.gov, hart@nipmuc.fsl.noaa.gov Subject: Re: (Fwd) Re: Proposal for two processes to "share" a buffer (TeX) References: <9605311335.ZM14217@mrjones.engr.sgi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I'll try to answer your questions with respect to CDS1, for completeness. Eric Salo wrote: >1) Let's say that the shared buffer is very large. It looks to me like the >entire buffer will always be sent to the receiver, even if we only modify a >single word. On shared memory systems this is of course not an issue, but I >don't think this is likely to be acceptable for NOWs. The objective of communication in CDS1 is to get data which is accessible by one process to be accessible by another. If one of those processes wants that data in a particular place, it is up to the process to move it to or from that spot. There are CDS1 primitives ("send" and "recv") which integrate the movement with the transfer to allow for optimization, but there is *no* capability in CDS1 for the producer to specify where the data should physically appear in the consumer, or for a consumer to specify where the data should come from in the producer, nor do I believe there should be. Such semantics are directly contrary to information hiding principals. So, to answer your specific question, the buffer (i.e. "region" in CDS1) *is* the unit of transfer. If you want to transfer parts of a buffer, then you make each part a separate buffer. Each CDS1 comm cell can hold one element of an array, or an entire array, so the unit of transfer is up to the user. >2) Now let's say that the shared buffer is very small. Will the handshaking >that is required between partners to obtain/free the buffer lock be a source of >significant latency, and in general can it be masked by careful coding? I >confess that I don't quite understand your sample implementation, but it seems >logical that a consumer (for example) would need to receive first a message >from the producer stating that the buffer was available, and then potentially a >second message containing the new buffer contents. This might or might not be a >significant source of delay, depending on how things are "reset" for the next >transaction. I don't know why you believe that more overhead would be needed than for standard message passing. In fact, in CDS1, the buffer transfer is essentially a ready send, because it is the user's responsibility to ensure that there is room in the comm heap for any incoming region. (It sounds like the same could be said of the NOAA system, since there is always a buffer there waiting for the message.) In CDS1 (though perhaps not in the NOAA system), there is some overhead required to allocate a region in the comm heap for the incoming message. >3) Limiting each buffer to a single sender and a single receiver seems very >painful, although I can see how it simplifies implementations nicely. If we're >going to try to simulate a true shared buffer it would be nice to be able to do >it for an arbitrary set of readers and writers. (Just a small matter of design >and implementation, right?) CDS1 allows an arbitrary set of readers and writers. >I can't make up my mind yet whether David's CDS1 does this - I'm still trying >to wrap my brain around it - but at first glance it appears to be more of a >"pull" design than a "push" design, which again makes me worry about round-trip >latency. CDS1 allows both "push" and "pull". If the process puts a region into a cell in another process, and that process gets the region from the cell, it is a "push". If a process puts a region into a cell in its own process, and another process gets the region from that cell, it is a "pull". Even on pulls, CDS1 offers a very simple and portable mechanism similar to asynchronous receive to allow the pulling process to perform other work after requesting a region from a distant cell -- i.e. while incurring the latency of the request and the resultant data. -Dave -- =============================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst |USMail: NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Group | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Fri May 31 17:30:35 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA20977; Fri, 31 May 1996 17:28:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA12535 for mpi-core-out; Fri, 31 May 1996 16:30:05 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA12530 for ; Fri, 31 May 1996 16:29:51 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id PAA08689; Fri, 31 May 1996 15:31:31 -0500 Date: Fri, 31 May 1996 15:31:31 -0500 Message-Id: <199605312031.PAA08689@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: MPI-2 document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The version of the MPI-2 document to be distributed at the June meeting is now available. It is in http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z It contains minor updates to several chapters as well as first shots at the intro and terms chapters. See you soon in Chicago..... Steve From mpi-core-human@mcs.anl.gov Mon Jun 3 07:06:11 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA06162; Mon, 3 Jun 1996 07:06:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id GAA04597 for mpi-core-out; Mon, 3 Jun 1996 06:08:00 -0500 Received: from bedrock.osc.edu (bedrock.osc.edu [128.146.36.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA04582; Mon, 3 Jun 1996 06:07:37 -0500 Received: from alex.osc.edu for nevin@osc.edu by bedrock.osc.edu (8.7.5/950822.1) id HAA12146; Mon, 3 Jun 1996 07:07:33 -0400 (EDT) Received: for nevin@osc.edu by alex.osc.edu (8.7.1/920428.1525) id HAA14496; Mon, 3 Jun 1996 07:07:31 -0400 (EDT) Date: Mon, 3 Jun 1996 07:07:31 -0400 (EDT) Message-Id: <199606031107.HAA14496@alex.osc.edu> From: Nick Nevin To: mpi-impl@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: mpirun In-Reply-To: <199606030923.AA12272@felix.dircon.co.uk> References: <199605311907.PAA08484@alex.osc.edu> <199606030923.AA12272@felix.dircon.co.uk> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > James Cownie writes: > This is certainly *not* the way that you are supposed to use -- > in normal unix commands. Normally -- means that all subsequent > arguments should be interpreted literally, and not treated as > flags. Well this is kind of what we are after. One could think of -- as meaning that all subsequent arguments are not to be interpreted by mpirun and to be passed literally to the MPI program to deal with as it pleases. I don't see a major semantic clash with "normal" unix commands. > Since this has a sane meaning for mpirun I'd be very nervous > about adopting the Lam scheme. > For instance > mpirun -np 4 -- -np > would mean run four instances of the program '-np' Yes but how many people do you know who start their file names with - ?. The commands mv and rm use the -- syntax so one can get rid of or rename mistakes if you ask me. > In any case it seems unnecessary to me, surely the simple thing > to say is that all the flags before the program name are for > mpirun, all those after for the program. This has the following > advantages We allow arguments to mpirun (apart from the filename) that do not look like flags and having a separator between what is for mpirun and what is for the program makes it simple for us. We also allow the program or schema name to appear anywhere in the command line before the --. If -- makes you so nervous how about a different separator, say ++ ? > 1) It's consistent with standard unix usage of -- > -- e.g. mpirun -np 4 -- -np -foo is well defined Our propsosed use may not be completely consistent but I think its fairly close. I don't have any problem with disallowing the usage given in the example since I don't envisage any users naming their program -np unless they are just trying to be difficult. > 2) It means that mpirun and its arguments are always a prefix on > a normal command (useful if you use it for other unix commands... > e.g. mpirun -np 4 !! will (assuming a csh like shell re-run the > last command, but in four instances)) Pretty marginal benefit assuming you want to use mpirun to start non-MPI programs which in itself seems pretty weird. If not then how does this help? If you previously ran a MPI program you would have used mpirun no? > 3) It's simple to see to whom the flags belong. Same with --. > 4) It's simple to implement. Same with --. Nick. -=- Nick Nevin nevin@osc.edu Ohio Supercomputer Center http://www.osc.edu/lam.html From mpi-core-human@mcs.anl.gov Mon Jun 3 09:19:18 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA06865; Mon, 3 Jun 1996 09:19:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA05694 for mpi-core-out; Mon, 3 Jun 1996 08:22:06 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA05689; Mon, 3 Jun 1996 08:21:57 -0500 Received: by felix.dircon.co.uk id AB28508 (5.67b/IDA-1.5); Mon, 3 Jun 1996 14:21:47 +0100 Message-Id: <199606031321.AB28508@felix.dircon.co.uk> Received: from gw2-183.pool.dircon.co.uk(194.112.35.183) by amnesiac via smap (V1.3) id sma028452; Mon Jun 3 14:21:13 1996 Received: from localhost by jim (SMI-8.6) id OAA02413; Mon, 3 Jun 1996 14:11:14 +0100 To: mpi-impl@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: mpirun In-Reply-To: Nick's message of "Mon, 03 Jun 1996 07:07:31 EDT." <199606031107.HAA14496@alex.osc.edu> Date: Mon, 03 Jun 1996 14:11:13 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I don't see a major semantic clash with "normal" unix commands. Maybe not major, but it's certainly different and that seems unnecessary. > Yes but how many people do you know who start their file names > with - ?. The commands mv and rm use the -- syntax so one can get rid of > or rename mistakes if you ask me. Of course, but Unix has a way to handle this. Why should we do something different ? > We allow arguments to mpirun (apart from the filename) that do not look > like flags This seems to imply that you have further potential ambiguities. How can you tell these arguments from the executable filename ? e.g. what does this mean ? mpirun n1 n2 n3 > Our proposed use may not be completely consistent but I think its > fairly close. Which IMHO is even worse than being completely inconsistent ! We should do the same as the Unix standard (if "Unix standard" is not a contradiction in terms :-)) or do something completely different. Doing something which is "fairly close" seems to me to be the worst of all possible worlds. (You have not only to explain it, but also explain how it differs from what was expected, and people who have an expectation just have it confounded. It's better to have to remember a new rule than to have to remember an exception to an old rule.) -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Mon Jun 3 09:26:54 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA07009; Mon, 3 Jun 1996 09:26:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA05828 for mpi-core-out; Mon, 3 Jun 1996 08:30:10 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA05812; Mon, 3 Jun 1996 08:29:41 -0500 Message-Id: <199606031329.IAA05812@antares.mcs.anl.gov> To: James Cownie cc: mpi-impl@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: mpirun In-Reply-to: Message from James Cownie of "Mon, 03 Jun 1996 14:11:13 +0100." <199606031321.AB28508@felix.dircon.co.uk> Date: Mon, 03 Jun 1996 08:29:41 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | Doing something which is "fairly close" seems to me to be the worst of | all possible worlds. (You have not only to explain it, but also | explain how it differs from what was expected, and people who have an | expectation just have it confounded. It's better to have to remember a | new rule than to have to remember an exception to an old rule.) I agree with Jim. This IS the worst possible situation, and vendors should remember how they get raked over the coals for other supposedly small inconsistencies. The cognitive burden of all of these inconsistencies is very high, to say nothing of the documentation and more importantlyly maintenance cost of explaining to users why things are "slightly" different. Bill From mpi-core-human@mcs.anl.gov Mon Jun 3 10:00:16 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA07447; Mon, 3 Jun 1996 10:00:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA06469 for mpi-core-out; Mon, 3 Jun 1996 09:03:03 -0500 Received: from bedrock.osc.edu (bedrock.osc.edu [128.146.36.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA06444; Mon, 3 Jun 1996 09:02:28 -0500 Received: from alex.osc.edu for nevin@osc.edu by bedrock.osc.edu (8.7.5/950822.1) id KAA13773; Mon, 3 Jun 1996 10:02:25 -0400 (EDT) Received: for nevin@osc.edu by alex.osc.edu (8.7.1/920428.1525) id KAA17003; Mon, 3 Jun 1996 10:02:24 -0400 (EDT) Date: Mon, 3 Jun 1996 10:02:24 -0400 (EDT) Message-Id: <199606031402.KAA17003@alex.osc.edu> From: Nick Nevin To: mpi-impl@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: re: mpirun In-Reply-To: <199606031321.AB28508@felix.dircon.co.uk> References: <199606031107.HAA14496@alex.osc.edu> <199606031321.AB28508@felix.dircon.co.uk> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >>>>> "James" == James Cownie writes: >> I don't see a major semantic clash with "normal" unix commands. James> Maybe not major, but it's certainly different and that seems James> unnecessary. >> Yes but how many people do you know who start their file names >> with - ?. The commands mv and rm use the -- syntax so one can get >> rid of or rename mistakes if you ask me. James> Of course, but Unix has a way to handle this. Why should we James> do something different ? How many current mpirun's support the "Unix" -- syntax? Just tried mpich and it doesn't. Does POSIX say anything about --? >> We allow arguments to mpirun (apart from the filename) that do >> not look like flags James> This seems to imply that you have further potential James> ambiguities. How can you tell these arguments from the James> executable filename ? James> e.g. what does this mean ? mpirun n1 n2 n3 It's an error. We don't allow program names that clash with our node naming syntax. We have never had any complaints. >> Our proposed use may not be completely consistent but I think its >> fairly close. James> Which IMHO is even worse than being completely inconsistent ! Good point. Still it works fine for us and I haven't heard any users complain, so we are unlikely to change it. James> We should do the same as the Unix standard (if "Unix James> standard" is not a contradiction in terms :-)) or do James> something completely different. Fair enough. I have never held much hope for standardizing mpirun in any case. Nick. From mpi-core-human@mcs.anl.gov Mon Jun 3 17:35:06 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA14347; Mon, 3 Jun 1996 17:35:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA18098 for mpi-core-out; Mon, 3 Jun 1996 16:37:49 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA18085 for ; Mon, 3 Jun 1996 16:37:40 -0500 Message-Id: <199606032137.QAA18085@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Final announcement of June MPI Forum meeting Date: Mon, 03 Jun 1996 16:37:38 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This is a resend of the usual announcement. You need to read it only if you are unfamiliar with the location, schedule, or cost of our usual meetings. The list of expected attendees for the meeting is attached. Correction of any errors would be appreciated. Regards, Rusty Dear Group, The next meeting of the MPI Forum will take place on Wed.- Fri., June 5-7. It is time to make your reservations. Register at the hotel by calling (847) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. We will follow the usual schedule, beginning at 1:30 on Wednesday and ending at noon on Friday. We will charge the usual registration fee of $75, which can be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. If you have any problems or questions, let me know, either by email (lusk@mcs.anl.gov) or phone (708-252-7852). Here is the schedule of future MPI Forum meetings. Jun 5-7 Jul 17-19 Sep 4-6 Oct 9-11 Supercomputing '96 is Nov 17-22. Attendees for the June meeting: Greg Astfalk Ron Brightwell Eric Brunner Margaret Cahir Pang Chen Ying Chen Albert Cheng Yong Cho Joel Clark Lyndon Clarke Dennis Cottel Raja Daoud Terry Dontje Mark Fallon Richard Frost Al Geist Bill Gropp Leslie Hart Shane Hebert Rolf Hempel HansChristian Hoppe Steve HussLederman Arkady Kanevsky Koichi Konishi Susan Kraus Steve Landherr Lloyd Lewins Andrew Lumsdaine Rusty Lusk Brian McCandless Harish Nag Nick Nevin Jarek Nieplocha Bill Nitzberg Ron Oldfield Yoonho Park Perry Partow Elsie Pierce Heidi Poxon JeanPierre Prost Rolf Rabenseifner Joe Rieken Eric Salo Bill Saphir Anthony Skjellum Marc Snir Jeff Squyres Rajeev Thakur Marydell Tholburn Dick Treumann Simon Tsang Manuel Ujaldon Jerrell Watts Sanjiva Weerawarana Klaus Wolf Parkson Wong Dave Wright From mpi-core-human@mcs.anl.gov Tue Jun 4 16:57:46 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA17630; Tue, 4 Jun 1996 16:57:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA09418 for mpi-core-out; Tue, 4 Jun 1996 15:58:53 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA09398 for ; Tue, 4 Jun 1996 15:58:40 -0500 Message-Id: <199606042058.PAA09398@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Tentative agenda for meeting Date: Tue, 04 Jun 1996 15:58:38 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Comments welcome. - Rusty Tentative Agenda for the June '96 MPI Forum meeting My assessment of the priorities for this meeting are: Begin finishing those sections that are getting ripe for voting on: most of Dynamic, One-sided, and parts of Miscellany. Launch the effort in I/O Begin getting other chapters prepared for first official reading. The resources we have are: 1 room capable of holding all of us, all three days. (Call this Room A) 2 rooms capable of holding 25 people each, Wed. and Thurs. only. (B and C) 57 dedicated, intelligent, articulate, and thoughtful attendees. Therefore we can have as many as 3 parallel sessions on Wed. and Thurs., and on Friday we should have formal votes. Here is a tentative agenda, nearly 24 hours earlier than usual: Wednesday --------- 1:30-2:00 Discussion of procedures, what will be delivered in Supercomputing'96 time frame, and this agenda 2:00-3:00 (Room A, everyone) Overview of MPI-IO as it currently stands 3:00-3:30 Break 3:30-5:30 Room A: Dynamic (read counter-proposal, vote on direction) Room B: I/O (start) Room C: Bindings (prepare (again) for official reading. Fortran-90?) 6:00-8:00 Dinner 8:00-10:00 Room A: Dynamic (read amendments) Room B: External Room C: Real time Thursday -------- 8:30-12:00 Room A: One-sided Room B: Collective Room C: I/O (Break at 10:00-10:30) 12:00-1:00 Lunch 1:00-3:00 Room A: Dynamic (client/server) Room B: External (especially coordination with I/O) Room C: Real time 3:00-3:30 Break 3:30-5:30 Room A: I/O Room B: Miscellany 6:00-8:00 Dinner 8:00-10:00 Room A: Votes on One-Sided Friday ------ 8:30-10:00 Room A: Votes on Dynamic, if ready 10:00-10:15 Break 10:15-12:00 Room A: Progress Reports from I/O, External, Bindings, Collective, Real-Time, and Miscellany. 12:00 Adjourn From mpi-core-human@mcs.anl.gov Tue Jun 4 18:28:32 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA21488; Tue, 4 Jun 1996 18:28:31 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA11241 for mpi-core-out; Tue, 4 Jun 1996 17:28:08 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA11234 for ; Tue, 4 Jun 1996 17:28:00 -0500 Message-Id: <199606042228.RAA11234@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: thoughts on the number of functions Date: Tue, 04 Jun 1996 17:27:59 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Some people believe that reducing the number of functions in proposed additions to the MPI library specification is an important objective. Let me disagree with this view. A library is a toolkit, and the best feature for a toolkit to have is that a user with a job to do finds in the toolkit a tool that does the job. It can be elegant to design the toolkit so that it contains a small number of tools out of which the user can build the tool that he needs, but it is not as handy as finding the exact tool for the job. And it is especially likely that the custom tool will be more efficient for the job at hand than one that it is assembled out of a small number of more basic tools. There are only too many functions if they are so badly organized that the user has a hard time finding the one he needs. I don't think we have this problem; even in chapters with many functions, they are organized along orthogonal lines. We argued at various times during the MPI-1 process that we were getting too many functions, but users quickly found that the organization of the MPI specification meant that the relatively large numbers of functions did not mean even a linear increase in complexity, and that when they needed a particular function, it was there. I have never heard a user say that they wished MPI did *not* have a particular function. It is a far greater mistake to have the wrong functions than to have too many. A function is wrong if it is unclear what it means or if it is defined in such a way that it prevents efficient implementation. In MPI-1 we do have a few wrong functions, but not too many functions. I worry that in trying to reduce the number of functions in MPI-2, we will end up either with wrong functions, or without ones that we really do need. Rusty From mpi-core-human@mcs.anl.gov Tue Jun 4 18:46:34 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA21650; Tue, 4 Jun 1996 18:46:33 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA11709 for mpi-core-out; Tue, 4 Jun 1996 17:49:49 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA11704 for ; Tue, 4 Jun 1996 17:49:42 -0500 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA22287; Tue, 4 Jun 96 16:49:35 MDT From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9606042249.AA22287@nipmuc.fsl.noaa.gov> Subject: Re: thoughts on the number of functions To: lusk@mcs.anl.gov (Rusty Lusk) Date: Tue, 4 Jun 1996 16:49:35 -0600 (MDT) Cc: mpi-core@antares.mcs.anl.gov In-Reply-To: <199606042228.RAA11234@antares.mcs.anl.gov> from "Rusty Lusk" at Jun 4, 96 05:27:59 pm X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1130 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I tend to agree with Rusty, when I've talked to people who believe MPI is too big, their seems to be no consensus on what should be eliminated. What it usually comes down to is: "I don't use ..., therefore it should be eliminated". It is much easier for those who think MPI is too big to think of features to add to MPI :-) The only compelling arguments I have heard in favor of a small MPI is from developers who have to worry about test procedures, etc. I think for these folks, there is not much difference in a testing procedure if we have two funcions or a boolean flag that changes the nature of a single function. I think we were faced with a choice to do something the size of PICL or to do a more full blown standard (the choice we made is obvious). Leslie > > Some people believe that reducing the number of functions in proposed > additions to the MPI library specification is an important objective. > Let me disagree with this view. > > A library is a toolkit, and the best feature for a toolkit to have is > that a user with a job to do finds in the toolkit a tool that does the job. > > ... > > Rusty > From mpi-core-human@mcs.anl.gov Tue Jun 4 20:06:58 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA22478; Tue, 4 Jun 1996 20:06:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA12570 for mpi-core-out; Tue, 4 Jun 1996 19:09:05 -0500 Received: from paloalto.access.hp.com (daemon@paloalto.access.hp.com [15.254.56.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA12565 for ; Tue, 4 Jun 1996 19:08:58 -0500 Received: from tbag.rsn.hp.com by paloalto.access.hp.com with ESMTP (1.37.109.16/15.5+ECS 3.3) id AA264523329; Tue, 4 Jun 1996 17:08:49 -0700 Message-Id: <199606050008.AA264523329@paloalto.access.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA021213655; Tue, 4 Jun 1996 19:14:16 -0500 From: Raja Daoud Subject: Re: thoughts on the number of functions To: mpi-core@antares.mcs.anl.gov Date: Tue, 04 Jun 1996 19:14:15 CDT In-Reply-To: <9606042249.AA22287@nipmuc.fsl.noaa.gov>; from "Leslie Hart" at Jun 4, 96 4:49 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Good points in general Rusty and Leslie, but not necessarily applicable to all cases. I equally dislike functions that end up being: foo(int flag) { if (flag) foo1(); else foo2(); } But this is not always the case. In some cases, the several functions considered may be "artifically" subdivided into a larger number of cases then really necessary. In other cases they may also be inherently slow and the performance/optimization argument becomes much weaker for them. I see no reason in such cases to keep the large # of functions just because, out of analogy, it worked well in other cases, especially if no functionality is lost in the process. To give an example from MPI-1. We have a single "pack" function and it takes the datatype as argument. Inside of it, deep down, there is a switch statement that handles int/short/long/float/double/... in distinct cases. Should we be using your argument to justify one pack function per datatype? I don't think so, and I think you'd agree. Likewise, to get back to a more current scenario, I see little reason to have 12 spawn functions when 5 do the same (modulo a few objections that I'm sure we'll debate at length the next 3 days :-) ). Anyway, the arguments posted are valid, but they do not form an all-sweeping mantra applicable to all cases. As usual, we need to consider each case on its own carefully. See you in Chicago, --Raja From mpi-core-human@mcs.anl.gov Tue Jun 4 22:45:23 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA23560; Tue, 4 Jun 1996 22:45:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA14134 for mpi-core-out; Tue, 4 Jun 1996 21:48:01 -0500 Received: from msr.EPM.ORNL.GOV (geist@msr.epm.ornl.gov [128.219.8.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA14129 for ; Tue, 4 Jun 1996 21:47:53 -0500 Received: (from geist@localhost) by msr.EPM.ORNL.GOV (8.7.4/8.7.3) id WAA01715 for mpi-core@antares.mcs.anl.gov; Tue, 4 Jun 1996 22:47:52 -0400 (EDT) Date: Tue, 4 Jun 1996 22:47:52 -0400 (EDT) From: Al Geist Message-Id: <199606050247.WAA01715@msr.EPM.ORNL.GOV> To: mpi-core@antares.mcs.anl.gov Subject: Re: thoughts on the number of functions Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk It is easy to inflate the number of functions in a library and often easy (using extra args) to decrease the number. I would like to argue that it is not the number but rather the clarity of the functions that we should be considering. There are cases like MPI_PACK where an extra arg makes sense and saves us from adding many functions to MPI-1 that even if "organized well" wouldn't add to the clarity. There are other cases like MPI_SEND where I would not want to see an extra arg that specified (blocking, nonblocking, sycn, ready,...) because I think it is clearer to the user to have separate functions. At the upcoming meeting we need to decide in what category we want MPI_SPAWN to be in. >I have never heard a user say that they >wished MPI did *not* have a particular function. Why should they? They do not have to implement, test, or maintain MPI. The vendors get stuck with this burden. Let them scream. What I do hear regularly is a perception that MPI is bloated. Perception can scare people off from even trying MPI and this isn't good. We need to carefully determine if enumerating the different versions of a function really buys us anything. Performance isn't much of an issue with inherently slow MPI_SPAWN, but there are other considerations that have been aired in the newsgoup. In the case of MPI_SPAWN, I still favor the new proposal. Not because it has 5 functions vs. 24, but because I think the users will find this dynamic process API clearer, and less bloated. Al Geist From mpi-core-human@mcs.anl.gov Sat Jun 8 19:17:23 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA15627; Sat, 8 Jun 1996 19:17:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA09166 for mpi-core-out; Sat, 8 Jun 1996 18:17:59 -0500 Received: from bambam (root@bambam.hpc.uh.edu [129.7.102.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id PAA07992 for ; Sat, 8 Jun 1996 15:53:05 -0500 Received: by bambam (Smail3.1.29.1 #1) id m0uSV03-000IbbC; Sat, 8 Jun 96 15:52 CDT Message-Id: From: yoonho@sina.hpc.uh.edu (Yoonho Park) To: mpi-core@mcs.anl.gov Date: Sat, 8 Jun 1996 15:52:47 -0500 (CDT) X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 28 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk subscribe yoonho@hpc.uh.edu From mpi-core-human@mcs.anl.gov Wed Jun 12 10:30:16 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA01827; Wed, 12 Jun 1996 10:30:15 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA29902 for mpi-core-out; Wed, 12 Jun 1996 09:31:11 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA29897 for ; Wed, 12 Jun 1996 09:31:00 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id JAA08533; Wed, 12 Jun 1996 09:30:57 -0500 Date: Wed, 12 Jun 1996 09:30:57 -0500 Message-Id: <199606121430.JAA08533@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: Timetable for July meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Before I and others forget the schedule for the next meeting, here are the dates: June 21: Informal date for chapter authors to get out a new draft of their chapters which reflect changes from the June meeting. July 5: A first cut at the next version of each chapter is due to me by 12:00 PM CST (noon). It will be announced on mpi-core and should be out by late afternoon. July 12: The final version of each chapter is due to me by 10:00 AM CST. This is the version that will be printed for the meeting. It will also be announced on mpi-core and should be out by late afternoon. The current (and future documents) are available via the web. See http://www.cs.wisc.edu/~lederman/mpi2 and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2. The full current document is in mpi2-report.ps.Z. The sources and scripts are also available. From mpi-core-human@mcs.anl.gov Wed Jun 12 11:36:40 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA02778; Wed, 12 Jun 1996 11:36:37 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA02066 for mpi-core-out; Wed, 12 Jun 1996 10:38:41 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA02050; Wed, 12 Jun 1996 10:38:20 -0500 Received: by felix.dircon.co.uk id AA04141 (5.67b/IDA-1.5); Wed, 12 Jun 1996 16:37:51 +0100 Message-Id: <199606121537.AA04141@felix.dircon.co.uk> Received: from gw2-128.pool.dircon.co.uk(194.112.35.128) by amnesiac via smap (V1.3) id sma004090; Wed Jun 12 16:37:15 1996 Received: from localhost by jim (SMI-8.6) id OAA08537; Wed, 12 Jun 1996 14:49:31 +0100 To: mpi-misc@mcs.anl.gov Cc: mpi-core@mcs.anl.gov Subject: Naming objects Reply-To: jcownie@bbn.com Date: Wed, 12 Jun 1996 14:49:30 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk For a debugger to be able to provide a maningful display of the message passing state of a program it would be useful to have a method by which the user could associate a print name with a communicator (and potentially with other MPI types as well ?). The debugger could then use this name when displaying the outstanding communication operations, thus avoiding any need for the user to deal with context identifers (or other implementation specific details). Such a capability would probably also be useful to a profiler. The apparently obvious way to do this would be to add an additional pre-defined attribute key, something like MPI_NAME However there are a number of problems with this approach :- 1) All currently pre-defined attributes are read-only. MPI_NAME would need to be modifiable to be useful. 2) The copy behaviour is clear (the name does not get copied), but the delete behaviour is unclear (should the storage for the string be deleted ???) 3) It's not possible to pass a string into put_attr from Fortran. Since these problems seem rather hard to solve, I suggest instead a pair of new functions... int MPI_Comm_set_name (IN MPI_Comm comm, IN string name) int MPI_Comm_set_name(MPI_Comm comm, char * name); integer MPI_Comm_set_name( integer comm, character*(*) name) int MPI_Comm_get_name (IN MPI_Comm comm, OUT string name) int MPI_Comm_get_name(MPI_Comm comm, char ** name); integer MPI_Comm_get_name( integer comm, character*(*) name) MPI_Comm_set_name allows a user to associate a name string with a communicator. The passed in character string will be saved inside the MPI library (so need not be allocated in persistent store). MPI_Comm_get_name returns any name which had previously been associated with the given communicator. (Note that in the C binding it returns a reference to the internal copy of the name, which should not be modified by the user, and which will become invalid should the name be altered (by another call to MPI_Comm_set_name), or the communicator deleted). If the user has not associated a name with a communicator the system should return a null string (all spaces in Fortran, "" in C), or a system dependent name. (This allows the MPI system to give default print names to communicators, one might expect MPI_COMM_WORLD to return "MPI_COMM_WORLD", for instance). Notes: I have called these MPI_Comm_set_name and MPI_Comm_get_name, in case we decide that we also want to have similar functions for other data structures. Comments ? -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Wed Jun 12 13:18:45 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA03928; Wed, 12 Jun 1996 13:18:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA05115 for mpi-core-out; Wed, 12 Jun 1996 12:21:14 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA05106 for ; Wed, 12 Jun 1996 12:21:03 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id MAA12505; Wed, 12 Jun 1996 12:20:22 -0500 Date: Wed, 12 Jun 1996 12:20:22 -0500 Message-Id: <199606121720.MAA12505@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: official votes available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk To try and keep a permanent record of the many official votes we will be taking, I have added a file to the archives called votes-official. It has a record of all the official votes during MPI-2. Let me know if you think there are any mistakes. See http://www.cs.wisc.edu/~lederman/mpi2/votes-official or ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/votes-official. Steve From mpi-core-human@mcs.anl.gov Thu Jun 13 16:16:46 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA25489; Thu, 13 Jun 1996 16:16:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA05939 for mpi-core-out; Thu, 13 Jun 1996 15:13:04 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA05931 for ; Thu, 13 Jun 1996 15:12:57 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id NAA27771 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Thu, 13 Jun 1996 13:12:55 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id NAA01171; Thu, 13 Jun 1996 13:12:54 -0700 From: "Eric Salo" Message-Id: <9606131312.ZM1169@mrjones.engr.sgi.com> Date: Thu, 13 Jun 1996 13:12:53 -0700 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: YASM Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I suspect that there is probably no greater sin in the MPI community than proposing Yet Another Send Mode, but here's one anyway: In the 1-sided chapter, we have introduced the notion of "good" memory vs. "bad" memory for gets and puts. The idea is that if your system supports shared memory, then you can call MPI_RMA_MALLOC to request a shared buffer and as a reward for doing this your gets and puts will be faster. The thought occurs that 2-sided communication could benefit from this as well. In MPI-1, for example, many shared memory implementations still use a two-copy algorithm to pass messages around; the sender copies the data into a shared buffer, and the receiver copies it out again. While some systems are able to do better than this in some cases, many are not. So, given that we already are going to be adding MPI_RMA_MALLOC to MPI-2, why not also add a new send mode to take advantage of it? Something like this: MPI_Xsend MPI_Ixsend MPI_Xsend_init (We can debate the proper value of 'X' later...) In this mode, the sender asserts that the buffer being passed is one which was returned by a call to MPI_RMA_MALLOC. It otherwise has the same semantics as standard mode. With this mode, processes calling MPI_RECV on shared memory machines would be able to copy the data directly out of the send buffer. There could also be advantages on NOWs, because (for example) high performance implementations might opt to pin down the send buffers in advance to prepare for subsequent DMAs. Machines not capable of optimizing this case could simply use MPI_SEND to implement the new calls, much as often happens today with MPI_RSEND. Adding additional smarts to the already-existing send modes is probably not a practical solution because of the added latency that would be involved in building internal lists of shared buffers and comparing against them with every sent message. Doing this would probably involve making some minor changes to MPI_RMA_MALLOC. For example, it would probably have to become a collective function, and we would perhaps want to tweak the syntax slightly. Does anyone else think that this idea has merit? -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-808 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Thu Jun 13 16:34:45 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA25844; Thu, 13 Jun 1996 16:34:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA06821 for mpi-core-out; Thu, 13 Jun 1996 15:36:28 -0500 Received: from SSD.intel.com (ssd.intel.com [137.46.3.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id PAA06813 for ; Thu, 13 Jun 1996 15:36:18 -0500 From: joel@ssd.intel.com Received: from sail.ssd.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA17022; Thu, 13 Jun 96 13:35:44 PDT Message-Id: <9606132035.AA17022@SSD.intel.com> To: "Eric Salo" , mpi-core@mcs.anl.gov Subject: Re: YASM In-Reply-To: Your message of "Thu, 13 Jun 96 13:12:53 PDT." <9606131312.ZM1169@mrjones.engr.sgi.com> Date: Thu, 13 Jun 96 13:35:43 -0700 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk --- Your Message -> From: "Eric Salo" -> Date: Thu, 13 Jun 1996 13:12:53 -0700 -> To: mpi-core@mcs.anl.gov -> Subject: YASM -> -> I suspect that there is probably no greater sin in the MPI community than -> proposing Yet Another Send Mode, but here's one anyway: -> -> In the 1-sided chapter, we have introduced the notion of "good" memory vs. -> "bad" memory for gets and puts. The idea is that if your system supports shared -> memory, then you can call MPI_RMA_MALLOC to request a shared buffer and as a -> reward for doing this your gets and puts will be faster. We voted nothing like the above paragraph. There is no "good" vs "bad" memory in the spec. There is no "reward" in the spec. There is no guarantee that any particular memory will be "faster" than any other memory in the spec. Please refrain from projecting your desire despite several votes to the contrary. -> -> The thought occurs that 2-sided communication could benefit from this as well. -> In MPI-1, for example, many shared memory implementations still use a two-copy -> algorithm to pass messages around; the sender copies the data into a shared -> buffer, and the receiver copies it out again. While some systems are able to do -> better than this in some cases, many are not. -> -> So, given that we already are going to be adding MPI_RMA_MALLOC to MPI-2, why -> not also add a new send mode to take advantage of it? Something like this: I for one am willing to vote out MPI_RMA_MALLOC if it is going to cause these kinds of problems! -> -> MPI_Xsend -> MPI_Ixsend -> MPI_Xsend_init (We can debate the proper value of 'X' later...) -> -> In this mode, the sender asserts that the buffer being passed is one which was -> returned by a call to MPI_RMA_MALLOC. It otherwise has the same semantics as -> standard mode. I would think an implementation would be smart enough to tell when it can take advantage of this kind of special hardware without burdening the application writer. -> With this mode, processes calling MPI_RECV on shared memory -> machines would be able to copy the data directly out of the send buffer. There -> could also be advantages on NOWs, because (for example) high performance -> implementations might opt to pin down the send buffers in advance to prepare -> for subsequent DMAs. Machines not capable of optimizing this case could simply -> use MPI_SEND to implement the new calls, much as often happens today with -> MPI_RSEND. -> -> Adding additional smarts to the already-existing send modes is probably not a -> practical solution because of the added latency that would be involved in -> building internal lists of shared buffers and comparing against them with every -> sent message. The lists can be built during MPI_RMA_MALLOC. If the compare of a address costs so much latency there must be no advantage to avoiding the copies you mention above. -> -> Doing this would probably involve making some minor changes to MPI_RMA_MALLOC. -> For example, it would probably have to become a collective function, and we -> would perhaps want to tweak the syntax slightly. So now creating a pt2pt message buffer is a collective operation?! Ugh! -> -> Does anyone else think that this idea has merit? Need I say more? joel clark -> -> -- -> Eric Salo Silicon Graphics Inc. "Do you know what the -> (415)933-2998 2011 N. Shoreline Blvd, 8U-808 last Xon said, just -> salo@sgi.com Mountain View, CA 94043-1389 before he died?" --- End Of Your Message From mpi-core-human@mcs.anl.gov Thu Jun 13 18:32:59 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA27131; Thu, 13 Jun 1996 18:32:58 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA09606 for mpi-core-out; Thu, 13 Jun 1996 17:34:24 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA09601 for ; Thu, 13 Jun 1996 17:34:15 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id PAA06631 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Thu, 13 Jun 1996 15:34:12 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id PAA01408; Thu, 13 Jun 1996 15:34:11 -0700 From: "Eric Salo" Message-Id: <9606131534.ZM1406@mrjones.engr.sgi.com> Date: Thu, 13 Jun 1996 15:34:11 -0700 In-Reply-To: joel@SSD.intel.com "Re: YASM" (Jun 13, 1:35pm) References: <9606132035.AA17022@SSD.intel.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: YASM Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > We voted nothing like the above paragraph. There is no "good" vs "bad" > memory in the spec. There is no "reward" in the spec. There is no > guarantee that any particular memory will be "faster" than any other memory > in the spec. I would suggest that you reread the first paragraph in section 4.2: "In some systems, RMA operations will run faster when accessing specially allocated memory (e.g., memory that is shared by the other processes in the communicating group, on an SMP)." MPI provides a mechanism for allocating and freeing such special memory. The use of such memory for RMA is not mandatory." Seems pretty straightforward to me. Which part don't you understand? > Please refrain from projecting your desire despite several votes to the > contrary. Joel, I am doing what I can to *embrace* the current proposal. My own preference would still be to mandate RMA_MALLOC for every put/get message. That has been voted down and so I have moved on; the subcommittee has made it quite clear that restricting get/put windows is not acceptable, we shall instead allow codes to specify "special" memory for added performance. I am simply extending this idea forward to the next logical step. > I for one am willing to vote out MPI_RMA_MALLOC if it is going to cause these > kinds of problems! I was unaware that providing optimized data paths was a "problem". Are you saying that MPI should force all implementations to run uniformly slowly? > I would think an implementation would be smart enough to tell when it can > take advantage of this kind of special hardware without burdening the > application writer. Well, then perhaps you should think again. In principle, and MPI implementation could certainly maintain a list of all RMA_MALLOC buffers, yes. But now you have to check that list every time you send a message. So, what if the application allocates 20 different buffers? Now MPI has to check all 20 entries every time a message is sent to see if it can optimize it, which rather defeats the whole purpose. > The lists can be built during MPI_RMA_MALLOC. If the compare of a address > costs so much latency there must be no advantage to avoiding the copies you > mention above. This is just plain bullshit. The problem is not in building the list, it is in checking it. Extra memory references are a *big* problem, which is my whole point! I don't want to have to check some list every time I call MPI_SEND, and I don't want to have to copy my send buffer twice if I can just do it once. > So now creating a pt2pt message buffer is a collective operation?! Ugh! No, it's only collective if you choose to take advantage of this new option. Nothing else in MPI-1 changes. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-808 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Thu Jun 13 19:36:07 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA27558; Thu, 13 Jun 1996 19:36:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA09442 for mpi-core-out; Thu, 13 Jun 1996 17:28:11 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA09436 for ; Thu, 13 Jun 1996 17:28:00 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id PAA13657; Thu, 13 Jun 1996 15:27:42 -0700 From: "William C. Saphir" Message-Id: <9606131527.ZM13655@win233.nas.nasa.gov> Date: Thu, 13 Jun 1996 15:27:42 -0700 In-Reply-To: joel@ssd.intel.com "Re: YASM" (Jun 13, 1:35pm) References: <9606132035.AA17022@SSD.intel.com> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: joel@ssd.intel.com Subject: Re: YASM Cc: mpi-core@mcs.anl.gov, "Eric Salo" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk -> In the 1-sided chapter, we have introduced the notion of "good" memory vs. -> "bad" memory for gets and puts. The idea is that if your system supports shared -> memory, then you can call MPI_RMA_MALLOC to request a shared buffer and as a -> reward for doing this your gets and puts will be faster. > We voted nothing like the above paragraph. There is no "good" vs "bad" > memory in the spec. There is no "reward" in the spec. There is no > guarantee that any particular memory will be "faster" than any other memory > in the spec. > Please refrain from projecting your desire despite several votes to the > contrary. I don't know about you, but I and the rest of the Forum voted for the following text: "In some systems, RMA operations will run faster when accessing specially allocated memory (e.g. memory that is shared by the other processes in the communicating group, on an SMP). MPI provides a mechanism for allocating and freeing such special memory. The use of such memory for RMA is not mandatory" I expect that on certain machines I will be rewarded for using MPI_Rma_malloc, and I will encourage users to use it whenever possible. I believe I am guaranteed that such memory will sometimes be faster and will never be slower. What we (as a group, not me in particular) voted against was to replace the last sentence by "An MPI implementation may require that this specially-allocated be used for all RMA operations". Please refrain from projecting your desire despite several votes to the contrary. :-) Bill ps. I take no position on the proposed Xsend(). Perhaps we could trade it for Rsend(). From mpi-core-human@mcs.anl.gov Thu Jun 13 19:42:57 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA27606; Thu, 13 Jun 1996 19:42:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA10827 for mpi-core-out; Thu, 13 Jun 1996 18:44:32 -0500 Received: from SSD.intel.com (ssd.intel.com [137.46.3.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA10822 for ; Thu, 13 Jun 1996 18:44:25 -0500 From: joel@ssd.intel.com Received: from sail.ssd.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA05319; Thu, 13 Jun 96 16:43:46 PDT Message-Id: <9606132343.AA05319@SSD.intel.com> To: "Eric Salo" Subject: Re: YASM In-Reply-To: Your message of "Thu, 13 Jun 96 15:34:11 PDT." <9606131534.ZM1406@mrjones.engr.sgi.com> Cc: mpi-core@mcs.anl.gov Date: Thu, 13 Jun 96 16:43:45 -0700 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Well Bill and Eric's replies have certainly set me in my (deserved) place One reason I rarely contribute by email is my difficulty in using it as a carefully considered medium. I apologize for getting so hot under the collar. I am very frustrated with the process and direction of the Forum. I feel like Eric was a couple of months ago when he proposed tossing out the whole 1-sided chapter. We keep re-arguing many discussions and adding new functionality for (to me) border line cases and specialized limited hardware. One comment I felt like making in my last mail was we might was well call it SGI-MPI-2. This obviously is a result of my limited experience with shared memory architectures. While many of these new functions may be useful on certain shared memory machines and (simply) no-ops on distributed memory machines, the complexity and wording of the spec. is beginning to greatly obscure the usability of MPI for our system. Eric's email further emphasized this bias by suggesting systems have good and bad, faster and slower memory. It's just not true for our system, and many of the functions being added and required by the spec are (to me) useless and irrelevant. I'm afraid mnay users will find understanding the useful functions from the irrelevant overwhelming especially as the spec more and more implies requirements and bias that don't exist for our system. I also seriously object to any effort to add complexity to the user API in order to make implementations easier. I also am biased to resist any thing that looks like a change to MPI-1. As to providing optimized data paths being a problem, there are many trade offs that have to be made in writing a portable interface. If the Forum insists on providing all the functionality required to implement the absolute optimal communication for each type of hardware for now and into the future, MPI will be unusable. 20%-50% of the latency of MPI in our system is implementing MPI semantics above our primitives. This is something we have to live with (though we keep grinding away at it) for the sake of a widely accepted portable interface. Again I don't know much about shared memory architectures, I assumed an MPI implementation would be able to tell which kind of memory (shared or not shared) was being used without comparing each address to a list of addresses. Aren't the physical addresses of shared memory somehow different that non-shared physical addresses? Again I apologize for responding off-the-cuff to Eric's with my frustration. joel --- Your Message -> From: "Eric Salo" -> Message-Id: <9606131534.ZM1406@mrjones.engr.sgi.com> -> Date: Thu, 13 Jun 1996 15:34:11 -0700 -> To: mpi-core@mcs.anl.gov -> Subject: Re: YASM -> -> > We voted nothing like the above paragraph. There is no "good" vs "bad" -> > memory in the spec. There is no "reward" in the spec. There is no -> > guarantee that any particular memory will be "faster" than any other memory -> > in the spec. -> -> I would suggest that you reread the first paragraph in section 4.2: -> -> "In some systems, RMA operations will run faster when accessing -> specially allocated memory (e.g., memory that is shared by the other processes -> in the communicating group, on an SMP)." MPI provides a mechanism for -> allocating and freeing such special memory. The use of such memory for RMA is -> not mandatory." -> -> Seems pretty straightforward to me. Which part don't you understand? -> -> > Please refrain from projecting your desire despite several votes to the -> > contrary. -> -> Joel, I am doing what I can to *embrace* the current proposal. My own -> preference would still be to mandate RMA_MALLOC for every put/get message. That -> has been voted down and so I have moved on; the subcommittee has made it quite -> clear that restricting get/put windows is not acceptable, we shall instead -> allow codes to specify "special" memory for added performance. I am simply -> extending this idea forward to the next logical step. -> -> > I for one am willing to vote out MPI_RMA_MALLOC if it is going to cause these -> > kinds of problems! -> -> I was unaware that providing optimized data paths was a "problem". Are you -> saying that MPI should force all implementations to run uniformly slowly? -> -> > I would think an implementation would be smart enough to tell when it can -> > take advantage of this kind of special hardware without burdening the -> > application writer. -> -> Well, then perhaps you should think again. In principle, and MPI implementation -> could certainly maintain a list of all RMA_MALLOC buffers, yes. But now you -> have to check that list every time you send a message. So, what if the -> application allocates 20 different buffers? Now MPI has to check all 20 entries -> every time a message is sent to see if it can optimize it, which rather defeats -> the whole purpose. -> -> > The lists can be built during MPI_RMA_MALLOC. If the compare of a address -> > costs so much latency there must be no advantage to avoiding the copies you -> > mention above. -> -> This is just plain bullshit. The problem is not in building the list, it is in -> checking it. Extra memory references are a *big* problem, which is my whole -> point! I don't want to have to check some list every time I call MPI_SEND, and -> I don't want to have to copy my send buffer twice if I can just do it once. -> -> > So now creating a pt2pt message buffer is a collective operation?! Ugh! -> -> No, it's only collective if you choose to take advantage of this new option. -> Nothing else in MPI-1 changes. -> -> -- -> Eric Salo Silicon Graphics Inc. "Do you know what the -> (415)933-2998 2011 N. Shoreline Blvd, 8U-808 last Xon said, just -> salo@sgi.com Mountain View, CA 94043-1389 before he died?" --- End Of Your Message From mpi-core-human@mcs.anl.gov Thu Jun 13 19:45:22 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA27627; Thu, 13 Jun 1996 19:45:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA10858 for mpi-core-out; Thu, 13 Jun 1996 18:47:16 -0500 Received: from SSD.intel.com (ssd.intel.com [137.46.3.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA10849 for ; Thu, 13 Jun 1996 18:47:05 -0500 From: joel@ssd.intel.com Received: from sail.ssd.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA05379; Thu, 13 Jun 96 16:46:33 PDT Message-Id: <9606132346.AA05379@SSD.intel.com> To: mpi-core@mcs.anl.gov Subject: YASM Date: Thu, 13 Jun 96 16:46:32 -0700 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Well Bill and Eric's replies have certainly set me in my (deserved) place One reason I rarely contribute by email is my difficulty in using it as a carefully considered medium. I apologize for getting so hot under the collar. I am very frustrated with the process and direction of the Forum. I feel like Eric was a couple of months ago when he proposed tossing out the whole 1-sided chapter. We keep re-arguing many discussions and adding new functionality for (to me) border line cases and specialized limited hardware. One comment I felt like making in my last mail was we might was well call it SGI-MPI-2. This obviously is a result of my limited experience with shared memory architectures. While many of these new functions may be useful on certain shared memory machines and (simply) no-ops on distributed memory machines, the complexity and wording of the spec. is beginning to greatly obscure the usability of MPI for our system. Eric's email further emphasized this bias by suggesting systems have good and bad, faster and slower memory. It's just not true for our system, and many of the functions being added and required by the spec are (to me) useless and irrelevant. I'm afraid mnay users will find understanding the useful functions from the irrelevant overwhelming especially as the spec more and more implies requirements and bias that don't exist for our system. I also seriously object to any effort to add complexity to the user API in order to make implementations easier. I also am biased to resist any thing that looks like a change to MPI-1. As to providing optimized data paths being a problem, there are many trade offs that have to be made in writing a portable interface. If the Forum insists on providing all the functionality required to implement the absolute optimal communication for each type of hardware for now and into the future, MPI will be unusable. 20%-50% of the latency of MPI in our system is implementing MPI semantics above our primitives. This is something we have to live with (though we keep grinding away at it) for the sake of a widely accepted portable interface. Again I don't know much about shared memory architectures, I assumed an MPI implementation would be able to tell which kind of memory (shared or not shared) was being used without comparing each address to a list of addresses. Aren't the physical addresses of shared memory somehow different that non-shared physical addresses? Again I apologize for responding off-the-cuff to Eric's with my frustration. joel - --- Your Message - -> From: "Eric Salo" - -> Message-Id: <9606131534.ZM1406@mrjones.engr.sgi.com> - -> Date: Thu, 13 Jun 1996 15:34:11 -0700 - -> To: mpi-core@mcs.anl.gov - -> Subject: Re: YASM - -> - -> > We voted nothing like the above paragraph. There is no "good" vs "bad" - -> > memory in the spec. There is no "reward" in the spec. There is no - -> > guarantee that any particular memory will be "faster" than any other memory - -> > in the spec. - -> - -> I would suggest that you reread the first paragraph in section 4.2: - -> - -> "In some systems, RMA operations will run faster when accessing - -> specially allocated memory (e.g., memory that is shared by the other processes - -> in the communicating group, on an SMP)." MPI provides a mechanism for - -> allocating and freeing such special memory. The use of such memory for RMA is - -> not mandatory." - -> - -> Seems pretty straightforward to me. Which part don't you understand? - -> - -> > Please refrain from projecting your desire despite several votes to the - -> > contrary. - -> - -> Joel, I am doing what I can to *embrace* the current proposal. My own - -> preference would still be to mandate RMA_MALLOC for every put/get message. That - -> has been voted down and so I have moved on; the subcommittee has made it quite - -> clear that restricting get/put windows is not acceptable, we shall instead - -> allow codes to specify "special" memory for added performance. I am simply - -> extending this idea forward to the next logical step. - -> - -> > I for one am willing to vote out MPI_RMA_MALLOC if it is going to cause these - -> > kinds of problems! - -> - -> I was unaware that providing optimized data paths was a "problem". Are you - -> saying that MPI should force all implementations to run uniformly slowly? - -> - -> > I would think an implementation would be smart enough to tell when it can - -> > take advantage of this kind of special hardware without burdening the - -> > application writer. - -> - -> Well, then perhaps you should think again. In principle, and MPI implementation - -> could certainly maintain a list of all RMA_MALLOC buffers, yes. But now you - -> have to check that list every time you send a message. So, what if the - -> application allocates 20 different buffers? Now MPI has to check all 20 entries - -> every time a message is sent to see if it can optimize it, which rather defeats - -> the whole purpose. - -> - -> > The lists can be built during MPI_RMA_MALLOC. If the compare of a address - -> > costs so much latency there must be no advantage to avoiding the copies you - -> > mention above. - -> - -> This is just plain bullshit. The problem is not in building the list, it is in - -> checking it. Extra memory references are a *big* problem, which is my whole - -> point! I don't want to have to check some list every time I call MPI_SEND, and - -> I don't want to have to copy my send buffer twice if I can just do it once. - -> - -> > So now creating a pt2pt message buffer is a collective operation?! Ugh! - -> - -> No, it's only collective if you choose to take advantage of this new option. - -> Nothing else in MPI-1 changes. - -> - -> -- - -> Eric Salo Silicon Graphics Inc. "Do you know what the - -> (415)933-2998 2011 N. Shoreline Blvd, 8U-808 last Xon said, just - -> salo@sgi.com Mountain View, CA 94043-1389 before he died?" - --- End Of Your Message ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Thu Jun 13 20:35:01 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA27830; Thu, 13 Jun 1996 20:35:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA11383 for mpi-core-out; Thu, 13 Jun 1996 19:36:39 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA11378 for ; Thu, 13 Jun 1996 19:36:31 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id RAA08542 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Thu, 13 Jun 1996 17:36:30 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id RAA01566; Thu, 13 Jun 1996 17:36:29 -0700 From: "Eric Salo" Message-Id: <9606131736.ZM1564@mrjones.engr.sgi.com> Date: Thu, 13 Jun 1996 17:36:28 -0700 In-Reply-To: joel@ssd.intel.com "Re: YASM" (Jun 13, 4:43pm) References: <9606132343.AA05319@SSD.intel.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: YASM Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Again I apologize for responding off-the-cuff to Eric's with my frustration. Apology accepted, and I can certainly sympathize! I too am very concerned about the increased complexity of MPI-2, both as a user and as someone who is probably gonna have to implement it at some point. In regard to my specific suggestion of adding a new send mode, I see here a potential for a truly significant performance boost on any system that supports multiple processes with shared memory segments, including uniprocessors. For systems which would not see a boost the cost of implementation would be basically zero, and the learning curve for MPI-1 users who wished to take advantage of it would be minimal. There is probably not much else in MPI-2 for which all of that can equally be said! To answer your question, no, in general there is no difference between the addresses of shared buffers and non-shared buffers. What commonly happens is that the application asks the OS to allocate a shared buffer, and that buffer is then mapped into the address space at some location. The "shared-ness" comes from the fact that virtual addresses in two different address spaces now map to the same physical page(s) of memory. Fancier machines may work a bit differently, but that's how (for example) System V shared segments work. - Eric P.S. Calling it SGI-MPI-2 would be a great irony, since virtually every time I propose a change it gets defeated! :-) -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-808 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Jun 14 15:44:34 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA15139; Fri, 14 Jun 1996 15:44:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA26592 for mpi-core-out; Fri, 14 Jun 1996 14:35:45 -0500 Received: from SSD.intel.com (ssd.intel.com [137.46.3.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA26583 for ; Fri, 14 Jun 1996 14:35:32 -0500 From: joel@ssd.intel.com Received: from sail.ssd.intel.com by SSD.intel.com (4.1/SMI-4.1) id AA12239; Fri, 14 Jun 96 12:35:00 PDT Message-Id: <9606141935.AA12239@SSD.intel.com> To: mpi-core@mcs.anl.gov Subject: Re: YASM In-Reply-To: Your message of "Thu, 13 Jun 96 17:36:28 PDT." <9606131736.ZM1564@mrjones.engr.sgi.com> Date: Fri, 14 Jun 96 12:34:59 -0700 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk --- Your Message -> From: "Eric Salo" -> Date: Thu, 13 Jun 1996 17:36:28 -0700 -> To: mpi-core@mcs.anl.gov -> Subject: Re: YASM -> -> In regard to my specific suggestion of adding a new send mode, I see here a -> potential for a truly significant performance boost on any system that supports -> multiple processes with shared memory segments, including uniprocessors. For -> systems which would not see a boost the cost of implementation would be -> basically zero, .... The cost is never basically zero. For the implementor there may be relatively little costs to these irrelevant functions to prevent linker errors. However we cannot ignore the cost of documentation and evaluation. This are significant The standard as resulting from the MPI-Forum is poor documentation, rarely even clearing indicate possible return values of functions. And I have not met an Evaluator yet who believes me when I say a function is a no-op. Take a function like MPI_Window_in that was first advertise as basically a no-op on many systems. Some evaluators would interpret (page 94, line 21) "but the local copy of the window should not be updated locally" to imply that a high quality implementation should prevent a local update. Also (page 91, line 46,47) "Before using loads and stores on the target process to access data updated by a prior RMA operation, the target process 'must' first issue a call to MPI_WINDOW_IN", would surely be interpreted by most evaluators as some kind of error needs to be generated if this is not the case. And what exactly are the possible return values of MPI_WINDOW_IN??? Portable evaluation suite writers might spend a great deal of time trying to evaluate these meaningfully. joel --- End Of Your Message From mpi-core-human@mcs.anl.gov Mon Jun 17 20:29:08 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA16199; Mon, 17 Jun 1996 20:29:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA29491 for mpi-core-out; Mon, 17 Jun 1996 19:31:48 -0500 Received: from relay1.UU.NET (relay1.UU.NET [192.48.96.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA26523 for ; Mon, 17 Jun 1996 16:08:24 -0500 Received: from sky.com by relay1.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQauns00373; Mon, 17 Jun 1996 17:07:58 -0400 (EDT) Received: from gg.sky.com by sky.com (4.1/SMI-4.1-DNI-7.0.1) id AA01911; Mon, 17 Jun 96 17:08:02 EDT Date: Mon, 17 Jun 96 17:08:02 EDT From: anna@sky.com (Anna Rounbehler - Technical Staff) Message-Id: <9606172108.AA01911@sky.com> To: salo@sgi.com Subject: New MPI Send Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I took some time to think about some of your suggestions, which I thought may have captured issues that many of us have been mulling over. Your ideas have been very helpful, since they cause us to think about the balance between generic and specific. One reason why MPI-2 extensions are valubale is that they provide a vehicle for unique and specific MPI functionality. If the basic MPI functions grow too large in scope, some implementors end up with too much overlap between extensions and MPI-1 functionality. A new MPI_ Xsend command may fall into this category. Some shared memory architectures encapsulate RMA, DMA, and local transfers as part of an implementation layer for the standard MPI send commands. An additional MPI Send encourages a decoupling of data transfer type, at a high level. It may be more implementation & user friendly to encapsulate this functionality in the MPI-2 extensions as unique function calls, instead of an MPI basic function. There is a fine line between an advantage derived from adding MPI basic functions/extensions and leaving room for the implementators. I suspect this line wiggles around due to the variety of architectures and users that MPI supports. If sometimes a proposal is rejected, it is not because it is poor, but possibly because the advantage to some is an equal disadvantage to others. That makes it a tough call. So I think, therein lies the balance. From mpi-core-human@mcs.anl.gov Tue Jun 18 10:26:58 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04071; Tue, 18 Jun 1996 10:26:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA05239 for mpi-core-out; Tue, 18 Jun 1996 09:28:39 -0500 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA05234 for ; Tue, 18 Jun 1996 09:28:32 -0500 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.18.106]) by franklin.sdsc.edu (8.7.5/8.7.3/SDSCserver-12) with SMTP id HAA20316 for ; Tue, 18 Jun 1996 07:28:30 -0700 (PDT) Received: from localhost by rogue.sdsc.edu (940406.SGI/1.11-client) with SMTP id AA01205; Tue, 18 Jun 96 07:28:30 -0700 Date: Tue, 18 Jun 1996 07:28:29 -0700 (PDT) From: Richard Frost To: mpi-core@mcs.anl.gov Subject: Re: YASM In-Reply-To: <9606132343.AA05319@SSD.intel.com> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I also share Joel's concerns. Our site serves approximately 2,000 academic researchers. For many of them, programming is a means of solving a computational problem. It is not a daily profession. >From our experiences in training and consulting these users, I can't imagine more than a few of them using the 1-sided constructs voted in last week. Further, the task of library writers has been complicated by the 1-sided API. I doubt that I could now write efficient, portable put/get Fortran code. With C, it appears that "#ifdef SGI" will be required. Perhaps extensive examples in the 1-sided chapter can demonstrate otherwise? - Richard On Thu, 13 Jun 1996 joel@ssd.intel.com wrote: > ... > > While many of these new functions may be useful on certain shared memory > machines and (simply) no-ops on distributed memory machines, the complexity > and wording of the spec. is beginning to greatly obscure the usability of > MPI for our system. From mpi-core-human@mcs.anl.gov Thu Jun 20 09:45:52 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA02481; Thu, 20 Jun 1996 09:45:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA15926 for mpi-core-out; Thu, 20 Jun 1996 08:48:06 -0500 Received: from relay1.UU.NET (relay1.UU.NET [192.48.96.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA15853; Thu, 20 Jun 1996 08:46:47 -0500 Received: from sky.com by relay1.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQauxr15635; Thu, 20 Jun 1996 09:46:45 -0400 (EDT) Received: from gg.sky.com by sky.com (4.1/SMI-4.1-DNI-7.0.1) id AA19682; Thu, 20 Jun 96 09:46:39 EDT Date: Thu, 20 Jun 96 09:46:39 EDT From: anna@sky.com (Anna Rounbehler - Technical Staff) Message-Id: <9606201346.AA19682@sky.com> To: mpi-core@mcs.anl.gov, mpi-impl@mcs.anl.gov Subject: Styleguide & standards Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Is there a styleguide and coding standard document for MPI,MPICH to insure consistency in the development of functions across the individual forum groups? Some of the types of things found in such documents might be: Coding ------- Naming conventions Recommendations for a limit on number of params for a function ANSI C compliant Data structure design recommendations Style ----- - Standardization of data structures used in MPI-2 by all groups - Ergonomics - guidelines for user-friendly interface - Guidelines for common functions - Grouping of functions into "toolbox" functions. This could include functions that "edit". - Organization of MPI function calls to insure minimized library builds - This would allow a user to build libraries by functionality. And there are many others. Two issues that affect success of a product are software quality and user friendliness. If the user has to invest too much time using the interface, despair sets in, schedules slip and code becomes unmanageable. The balance of a good R&D product with commercial guidelines usually produces a friendlier, more easily manageable product. Any little step that we can make in that direction is ok. If we have "written" or implied guidelines, I would be glad to collect them and create a document for all to review, comment & add. Regards, Anna ---------------------------------------------------------------------------- Anna Rounbehler SKY Computer rounbehler@sky.com From mpi-core-human@mcs.anl.gov Fri Jun 21 18:53:00 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA02647; Fri, 21 Jun 1996 18:52:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA19925 for mpi-core-out; Fri, 21 Jun 1996 17:54:56 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA19919; Fri, 21 Jun 1996 17:54:44 -0500 Message-Id: <199606212254.RAA19919@antares.mcs.anl.gov> To: mpi-misc@antares.mcs.anl.gov, mpi-core@antares.mcs.anl.gov Subject: New Miscellany Chapter Date: Fri, 21 Jun 1996 17:54:41 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Many people have contributed sections to this chapter, some indirectly by making proposals in the mpi-misc mailing list. Particularly if you are one of these, please read this to see whether I have accurately represented your material. Changes and new stuff are marked with the July change markers. This is intended to reflect the informal votes taken at the June meeting and most of the email discussion since then. Rusty %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Fri Jun 21 17:51:02 1996 %%Pages: 43 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.06.21:1750 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2.new/temp.dvi) @start /Fa 2 52 df<0F8030E040708030C038E0384038003800700070006000C00180 030006000C08080810183FF07FF0FFF00D157E9412>50 D<0FE030306018701C701C001C 00180038006007E000300018000C000E000EE00EE00EC00C401830300FE00F157F9412> I E /Fb 1 59 df<70F8F8F87005057C840D>58 D E /Fc 49 123 df<00E001C0038007000E000E001C001C003800380038007000700070007000E000E000 E000E000E000E000E000E000E000E000E000E00070007000700070003800380038001C00 1C000E000E000700038001C000E00B2A7E9E10>40 DI<018001C0018001806186F99F7DBE1FF807E007E01FF87DBEF99F6186018001 8001C0018010127E9E15>I44 DII<03C00FF01FF83C3C381C700E700E70 0EE007E007E007E007E007E007E007E007E007E007E007E007E007700E700E700E381C3C 3C1FF80FF007E0101D7E9B15>48 D<010007003F00FF00C7000700070007000700070007 00070007000700070007000700070007000700070007000700070007000700FFF8FFF80D 1C7C9B15>I<7FFFFFC0FFFFFFE000000000000000000000000000000000000000000000 00000000000000000000FFFFFFE07FFFFFC01B0C7E8F20>61 D<001C0000003E0000003E 0000002E0000006700000067000000E7800000C7800000C3800001C3C0000183C0000181 C0000381E0000381E0000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C00 3C001C003C0018003C0038001E0038001E0070001F0070000F0070000F00E0000780191D 7F9C1C>65 DI<003FC000FFF003C0 F00780300F00001E00003C00003C0000780000780000780000F00000F00000F00000F000 00F00000F00000F00000F00000F000007800007800007800003C00003C00001E00000F00 0807801803C07800FFF0003F80151F7D9D1B>IIII<003F8001FFF003C0F80780380F00181E00003C00003C 0000780000780000780000F00000F00000F00000F00000F00000F00000F007F8F007F8F0 00387800387800387800383C00383C00381E00380F003807803803C0F801FFF0003F8015 1F7D9D1C>III75 DIII<003F000001FFE00003FFF00007C0F8000F807C001E001E003E001F 003C000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003 C0F00003C0F00003C0F00003C0F80007C078000780780007807C000F803C000F003E001F 001F003E000F807C0007C0F80003FFF00001FFE000003F00001A1F7E9D1F>II<003F000001FFE00003FFF00007C0F8000F80 7C001F003E003E001F003C000F00780007807800078078000780F00003C0F00003C0F000 03C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C07800078078000780780E 07803C0F0F003E079F001E03DE000F83FC0007C1F80003FFF00001FFF800003F78000000 3C0000003E0000001F0000000F801A237E9D1F>II<03F8000FFE001C0F00380700700300600000E00000E00000E00000E000 00F000007800007F00003FE0001FFC0007FE0001FF00001F800007800003C00003C00001 C00001C00001C00001C0C00180E00380F007007C0E001FFC0007F000121F7E9D17>III87 D89 D<7FFFF07FFFF00001E00003E00003C00007C0000780000F00001F00001E00003E00003C 0000780000F80000F00001F00001E00003C00007C0000780000F80000F00001E00003E00 003C00007C0000780000FFFFF0FFFFF0141D7E9C19>I<0FC03FF07FF87038401C001C00 1C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC3F1C0E127E9114>97 D<07E00FF81FFC3C1C70047000E000E000E000E000E000E000700070043C1C1FFC0FF807 E00E127E9112>99 D<000E000E000E000E000E000E000E000E000E000E000E0F8E1FEE3F FE7C3E700E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E0F1D7E9C15 >I<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C3FFC1FF8 07E00E127E9112>I104 DI109 DI<03F0000FFC001FFE003C0F0078078070 0380E001C0E001C0E001C0E001C0E001C0F003C07003807807803C0F001FFE000FFC0003 F00012127F9115>II114 D<1FC03FF07FF0F030E000E000F0007F003FC01FE000F0003800388038F078FF F07FE01FC00D127F9110>I<1C001C001C001C001C001C00FFE0FFE01C001C001C001C00 1C001C001C001C001C001C001C001C001C201FF00FF007C00C187F970F>IIII<7FFC7FFC7FFC007800F000E001E003C003800700 0F001E001C003C007800FFFCFFFCFFFC0E127F9112>122 D E /Fd 15 119 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D<00038000000380000007C0000007C0000007C000000FE000000FE000001FF000001BF0 00001BF0000031F8000031F8000061FC000060FC0000E0FE0000C07E0000C07E0001803F 0001FFFF0003FFFF8003001F8003001F8006000FC006000FC00E000FE00C0007E0FFC07F FEFFC07FFE1F1C7E9B24>65 D68 D<0FF8001C1E003E0F803E07 803E07C01C07C00007C0007FC007E7C01F07C03C07C07C07C0F807C0F807C0F807C0780B C03E13F80FE1F815127F9117>97 D<03FC000E0E001C1F003C1F00781F00780E00F80000 F80000F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E 9115>99 D<01FC000F07001C03803C01C07801C07801E0F801E0F801E0FFFFE0F80000F8 0000F800007800007C00603C00601E00C00F038001FC0013127F9116>101 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F001F 001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D108 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I114 D<1FD830786018E018E018F000FF807FE07FF01F F807FC007CC01CC01CE01CE018F830CFC00E127E9113>I<030003000300030007000700 0F000F003FFCFFFC1F001F001F001F001F001F001F001F001F001F0C1F0C1F0C1F0C0F08 079803F00E1A7F9913>II< FFC1FCFFC1FC1F00601F80E00F80C00FC0C007C18007C18003E30003E30001F60001F600 01FE0000FC0000FC0000780000780000300016127F9119>I E /Fe 47 123 df45 D<387CFEFEFE7C3807077C860F>I<00 E00001E0000FE000FFE000F3E00003E00003E00003E00003E00003E00003E00003E00003 E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003 E00003E00003E000FFFF80FFFF80111D7C9C1A>49 D<07F0001FFE00383F007C1F80FE0F C0FE0FC0FE0FE0FE07E07C07E03807E0000FE0000FC0000FC0001F80001F00003E000078 0000F00000E00001C0000380600700600E00601C00E01FFFC03FFFC07FFFC0FFFFC0FFFF C0131D7D9C1A>I<01FC0007FF000E0F801E0FC03F07E03F07E03F07E03F07E01E0FC000 0FC0000F80001F0001FC0001FC00000F800007C00003E00003F00003F83803F87C03F8FE 03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003FC00151D7E9C1A>I<0001C00003C0 0007C00007C0000FC0001FC0003BC00073C00063C000C3C00183C00383C00703C00E03C0 0C03C01803C03803C07003C0E003C0FFFFFEFFFFFE0007C00007C00007C00007C00007C0 0007C000FFFE00FFFE171D7F9C1A>I<3803803FFF803FFF003FFE003FFC003FF0003F80 0030000030000030000030000033F80037FE003C1F00380F801007C00007C00007E00007 E07807E0FC07E0FC07E0FC07E0FC07C0780FC0600F80381F001FFC0007F000131D7D9C1A >I<01FC000FFF001F07803E03C07C03E07C01E0FC01F0FC01F0FC01F0FC01F8FC01F8FC 01F8FC01F87C03F87C03F83E05F81FFDF807F9F80041F80001F03C01F07E01F07E03E07E 03E07E07C03C0780381F001FFC0007F000151D7E9C1A>57 D<387CFEFEFE7C3800000000 0000387CFEFEFE7C3807147C930F>I<0000E000000000E000000001F000000001F00000 0001F000000003F800000003F800000006FC00000006FC0000000EFE0000000C7E000000 0C7E000000183F000000183F000000303F800000301F800000701FC00000600FC0000060 0FC00000C007E00000FFFFE00001FFFFF000018003F000018003F000030001F800030001 F800060001FC00060000FC000E0000FE00FFE00FFFE0FFE00FFFE0231F7E9E28>65 D<0007FC02003FFF0E00FE03DE03F000FE07E0003E0FC0001E1F80001E3F00000E3F0000 0E7F0000067E0000067E000006FE000000FE000000FE000000FE000000FE000000FE0000 00FE0000007E0000007E0000067F0000063F0000063F00000C1F80000C0FC0001807E000 3803F0007000FE01C0003FFF800007FC001F1F7D9E26>67 DIII<0007FC0200003FFF0E0000FE03DE0003F000FE 0007E0003E000FC0001E001F80001E003F00000E003F00000E007F000006007E00000600 7E00000600FE00000000FE00000000FE00000000FE00000000FE00000000FE003FFFE0FE 003FFFE07E00007E007E00007E007F00007E003F00007E003F00007E001F80007E000FC0 007E0007E0007E0003F000FE0000FE01FE00003FFF8E000007FC0600231F7D9E29>III 76 DII80 D82 D<03FC080FFF381E03F83800F8700078700038F00038F00018F00018F80000FC 00007FC0007FFE003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC0 003CC0003CC0003CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>I<7FFFFF FC7FFFFFFC7C07E07C7007E01C6007E00C6007E00CE007E00EC007E006C007E006C007E0 06C007E0060007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0 000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0 0003FFFFC003FFFFC01F1E7E9D24>II<07FC 001FFF003F0F803F07C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03 E0F803E0F803E0F803E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007 FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000FC0000FC00007C 00007E00003E00603F00C01F81C007FF0001FC0013147E9317>I<0007F80007F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F801F8F80FFEF81F83F8 3E01F87E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F8 7E00F83E01F81F07F80FFEFF03F8FF18207E9F1D>I<01FE0007FF800F83C01E01E03E00 F07C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E00 180F807007FFE000FF8015147F9318>I<001F8000FFC001F3E003E7E003C7E007C7E007 C3C007C00007C00007C00007C00007C000FFFC00FFFC0007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0003F FC003FFC0013207F9F10>I<01FC3C07FFFE0F079E1E03DE3E03E03E03E03E03E03E03E0 3E03E01E03C00F07800FFF0009FC001800001800001C00001FFF800FFFF007FFF81FFFFC 3C007C70003EF0001EF0001EF0001E78003C78003C3F01F80FFFE001FF00171E7F931A> II<1C003E00 7F007F007F003E001C00000000000000000000000000FF00FF001F001F001F001F001F00 1F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B217EA00E>I<003800 7C00FE00FE00FE007C003800000000000000000000000001FE01FE003E003E003E003E00 3E003E003E003E003E003E003E003E003E003E003E003E003E003E003E003E303E783EFC 3CFC7C78783FF01FC00F2A83A010>I108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC 007EFC007EFC007EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017 147F931A>II<01F81807FE381F87 783F01F83E01F87E00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87E00 F87E00F83F01F81F87F80FFEF803F8F80000F80000F80000F80000F80000F80000F80000 F80007FF0007FF181D7E931C>II<0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC0 07C007E007F006F81EFFFCC7F010147E9315>I<01800180018003800380038007800F80 3F80FFFCFFFC0F800F800F800F800F800F800F800F800F800F800F860F860F860F860F86 07CC03F801F00F1D7F9C14>II120 D<3FFFE03FFFE03C07C0380F 80701F80603F00603E00607C0000F80001F80003F00003E06007C0600F80601F80E03F00 C03E01C07C03C0FFFFC0FFFFC013147F9317>122 D E /Ff 74 124 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0000E01C0000E01 C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0000E01C0000E01 C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01 C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C00E0180 0E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F8151D80 9C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<003F07E00001C09C1800 0380F018000701F03C000E01E03C000E00E018000E00E000000E00E000000E00E000000E 00E000000E00E00000FFFFFFFC000E00E01C000E00E01C000E00E01C000E00E01C000E00 E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E0 1C000E00E01C000E00E01C000E00E01C000E00E01C007FC7FCFF80211D809C23>I<6060 F0F0F8F86868080808080808101010102020404080800D0C7F9C15>34 D<00E0000001900000030800000308000007080000070800000708000007080000071000 0007100000072000000740000003C03FE003800F00038006000380040005C0040009C008 0010E0100030E010006070200060702000E0384000E03C4000E01C8000E00F0020E00700 20700780403009C0401830E18007C03E001B1F7E9D20>38 D<60F0F86808080810102040 80050C7C9C0C>I<004000800100020006000C000C001800180030003000700060006000 6000E000E000E000E000E000E000E000E000E000E000E000E00060006000600070003000 3000180018000C000C00060002000100008000400A2A7D9E10>I<800040002000100018 000C000C000600060003000300038001800180018001C001C001C001C001C001C001C001 C001C001C001C001C0018001800180038003000300060006000C000C0018001000200040 0080000A2A7E9E10>I<60F0F0701010101020204080040C7C830C>44 DI<60F0F06004047C830C>I<00010003000600060006000C000C 000C0018001800180030003000300060006000C000C000C0018001800180030003000300 060006000C000C000C00180018001800300030003000600060006000C000C00010297E9E 15>I<03C00C301818300C300C700E60066006E007E007E007E007E007E007E007E007E0 07E007E007E007E00760066006700E300C300C18180C3007E0101D7E9B15>I<03000700 3F00C7000700070007000700070007000700070007000700070007000700070007000700 0700070007000700070007000F80FFF80D1C7C9B15>I<07C01830201C400C400EF00FF8 0FF807F8077007000F000E000E001C001C00380070006000C00180030006010C01180110 023FFE7FFEFFFE101C7E9B15>I<07E01830201C201C781E780E781E381E001C001C0018 0030006007E00030001C001C000E000F000F700FF80FF80FF80FF00E401C201C183007E0 101D7E9B15>I<000C00000C00001C00003C00003C00005C0000DC00009C00011C00031C 00021C00041C000C1C00081C00101C00301C00201C00401C00C01C00FFFFC0001C00001C 00001C00001C00001C00001C00001C0001FFC0121C7F9B15>I<300C3FF83FF03FC02000 2000200020002000200023E024302818301C200E000E000F000F000F600FF00FF00FF00F 800E401E401C2038187007C0101D7E9B15>I<00F0030C06040C0E181E301E300C700070 006000E3E0E430E818F00CF00EE006E007E007E007E007E007600760077006300E300C18 180C3003E0101D7E9B15>I<4000007FFF807FFF007FFF00400200800400800400800800 00100000100000200000600000400000C00000C00001C000018000018000038000038000 038000038000078000078000078000078000078000078000030000111D7E9B15>I<03C0 0C301818300C700C600EE006E006E007E007E007E007E0076007700F300F18170C2707C7 00060006000E300C780C78187010203030C00F80101D7E9B15>57 D<60F0F0600000000000000000000060F0F06004127C910C>I<60F0F060000000000000 0000000060F0F0701010101020204080041A7C910C>I<0FE03038401CE00EF00EF00EF0 0E000C001C0030006000C000800180010001000100010001000100000000000000000000 0003000780078003000F1D7E9C14>63 D<000600000006000000060000000F0000000F00 00000F00000017800000178000001780000023C0000023C0000023C0000041E0000041E0 000041E0000080F0000080F0000180F8000100780001FFF80003007C0002003C0002003C 0006003E0004001E0004001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E061800180198007000780 0E0003801C0003801C00018038000180780000807800008070000080F0000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000700000807800008078000080 380000801C0001001C0001000E000200070004000180080000E03000001FC000191E7E9C 1E>IIII<001F808000E0618001801980070007800E000380 1C0003801C00018038000180780000807800008070000080F0000000F0000000F0000000 F0000000F0000000F0000000F000FFF0F0000F8070000780780007807800078038000780 1C0007801C0007800E00078007000B800180118000E06080001F80001C1E7E9C21>II I<1FFF00F800780078007800780078007800780078007800780078007800780078007800 780078007800787078F878F878F878F0F040E021C01F00101D7F9B15>I76 DII<003F800000E0E000 0380380007001C000E000E001C0007003C00078038000380780003C0780003C0700001C0 F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0 780003C0780003C0380003803C0007801C0007000E000E0007001C000380380000E0E000 003F80001B1E7E9C20>II<003F800000E0 E0000380380007001C000E000E001C0007003C00078038000380780003C0780003C07000 01C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E07000 01C0780003C0780003C0380003803C0E07801C1107000E208E0007205C0003A0780000F0 E020003FE0200000602000003060000038E000003FC000003FC000001F8000000F001B25 7E9C20>II<07E0801C1980300580700380600180E0 0180E00080E00080E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF8000 0F800007C00003C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083 F800121E7E9C17>I<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F00 20800F0020800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000001F800003FFFC001B1C7F9B1E>II87 D<08081010202040404040808080808080B0B0F8F8787830300D0C7A9C 15>92 D<1FC000307000783800781C00301C00001C00001C0001FC000F1C00381C00701C 00601C00E01C40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E0 00E000E000E000E000E00060007004300418080C3007C00E127E9112>I<003F00000700 00070000070000070000070000070000070000070000070000070003E7000C1700180F00 300700700700600700E00700E00700E00700E00700E00700E00700600700700700300700 180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006E006FFFEE000E000 E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E061E0E0C0E000E 000E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C 00381C00381C001818001C38000C300013C0001000003000001800001FF8001FFF001FFF 803003806001C0C000C0C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C00180000000000000000 00000000000000FC001C001C001C001C001C001C001C001C001C001C001C001C001C001C 001C001C00FF80091D7F9C0C>I<00C001E001E000C00000000000000000000000000000 0FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E060E0F0C0F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180E001 C0E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C0003F0001212 7F9115>II<03C1000C3300180B00300F00700700700700 E00700E00700E00700E00700E00700E00700600700700700300F00180F000C370007C700 000700000700000700000700000700000700000700003FE0131A7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000F880388018C018C018E010D060 8FC00D127F9110>I<04000400040004000C000C001C003C00FFE01C001C001C001C001C 001C001C001C001C001C101C101C101C101C100C100E2003C00C1A7F9910>IIII<7F8FF00F03800F030007020003840001C80001 D80000F00000700000780000F800009C00010E00020E000607000403801E07C0FF0FF815 12809116>II<7FFC70386038407040F040E041C003C003 8007000F040E041C043C0C380870087038FFF80E127F9112>II E /Fg 58 123 df<000078007C7800FC7801FC7803C00003800007800007800007800007 8000078000078000078000078000FFFC78FFFC78FFFC7807807807807807807807807807 807807807807807807807807807807807807807807807807807807807807807807807807 8078078078078078152480A31A>12 D45 DI<01F00007FC000FFE001F1F003C07803C07807803C07803C0 7803C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0 F001E0F001E0F001E0F001E0F001E07803C07803C07803C07803C03C07803C07801F1F00 0FFE0007FC0001F00013237EA118>48 D<00C001C007C0FFC0FFC0FBC003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C0FFFFFFFFFFFF10227CA118>I<03F0000FFC001FFE003C1F0030078070 07C06003C0E003E0C001E04001E04001E00001E00001E00001E00003C00003C000078000 0780000F00001E00003C0000780000F00001E00001C0000380000700000E00001C000038 0000700000FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC001FFF003E0F00380780 7003C02003C02003C00003C00003C00003C0000780000780000F00001E0003FC0003F800 03FE00000F000007800003C00003C00001E00001E00001E00001E00001E08001E0C003C0 E003C07007803C0F801FFF000FFC0003F00013237EA118>I<001F00001F00002F00002F 00006F0000EF0000CF0001CF0001CF00038F00038F00078F00070F000F0F000E0F001E0F 003C0F003C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F 00000F00000F00000F00000F00000F0015217FA018>I<3FFF803FFF803FFF803C00003C 00003C00003C00003C00003C00003C00003C00003C00003CF8003FFE003FFF003F0F803E 07803C03C03803C00001E00001E00001E00001E00001E00001E00001E04003C04003C0E0 03C07007807C1F003FFE000FFC0003F00013227EA018>I<007E0001FF0003FF0007C100 0F00001E00001E00003C00003C000078000078000078F800F1FE00F7FF00FF0F80FC0780 F803C0F803C0F801C0F001E0F001E0F001E0F001E0F001E07001E07001E07801E07803C0 3803C03C03801C07801F0F000FFE0007FC0001F00013237EA118>II<01F00007FC000FFE001E 0F003C07803C07807803C07803C07803C07803C07803C03803803C07801E0F000F1E0007 FC0003F8000FFE001E0F003C07807803C07803C0F001E0F001E0F001E0F001E0F001E0F0 01E07803C07803C03C07803E0F801FFF0007FC0001F00013237EA118>I<01F00007FC00 0FFE001E0F003C07003807807803807803C0F003C0F001C0F001E0F001E0F001E0F001E0 F001E0F001E07003E07803E07803E03C07E03E1FE01FFDE00FF1E003E3C00003C00003C0 000380000780000700000F00101E00183C003FF8001FF00007C00013237EA118>I<001F 0000001F0000003F8000003F8000003B8000007BC0000073C0000071C00000F1E00000F1 E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C000780 3C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF001C000F003C0007803C00 078038000780780003C0780003C0700003C0F00001E0F00001E0E00001E01B237EA220> 65 DI<000FF000007FFC0000FFFF0001F01F0003C00700078000000F0000001E 0000003E0000003C0000003C000000780000007800000078000000F0000000F0000000F0 000000F0000000F0000000F0000000F0000000F0000000F0000000780000007800000078 0000003C0000003C0000003E0000001E0000000F0000000780008003C0038001F00F8000 FFFF00007FFC00000FF00019257DA31F>IIII<000FF000003FFE 0000FFFF8001F80F8003E00380078000000F0000001E0000001E0000003C0000003C0000 00780000007800000078000000F0000000F0000000F0000000F0000000F0000000F00000 00F000FFC0F000FFC0F000FFC0780003C0780003C0780003C03C0003C03C0003C01E0003 C01E0003C00F0003C0078003C003E003C001F807C000FFFFC0003FFF00000FF8001A257D A321>III75 DIII<001FC000007FF0 0001FFFC0003F07E0007C01F000F800F801F0007C01E0003C03C0001E03C0001E0780000 F0780000F0780000F070000070F0000078F0000078F0000078F0000078F0000078F00000 78F0000078F0000078F0000078780000F0780000F0780000F07C0001F03C0001E03E0003 E01E0003C01F0007C00F800F8007C01F0003F07E0001FFFC00007FF000001FC0001D257D A324>II82 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800 007800007800007800007800007C00003C00003F00001FC0000FFC0007FF0001FF80003F C00007E00001F00000F00000F8000078000078000078000078000078000078C000F0E000 F0F801E07E07C03FFF800FFF0001FC0015257EA31B>III 90 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF0 0FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C 0180780000780000700000F00000F00000F00000F00000F00000F0000078000078000078 00003C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC01FFFC0 3F0FC03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0 7803C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC001FFE 003E0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F000007000 007800007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01FF03 C0038007800780078007800780078007800780FFF8FFF8FFF80780078007800780078007 800780078007800780078007800780078007800780078007800780102380A20F>I<01F0 7807FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F000FFE 001FFC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8F000 78F00078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>I II<00F000F000F000F000000000000000000000000000000000000000F000 F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000 F000F000F000F000F000F000F000F080F0E3E0FFE07FC01F000C2D83A20D>II< F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F004 237DA20B>III<01FC0007FF000FFF801F07C03C01E07800F07800F0700070F00078F0 0078F00078F00078F00078F000787800F07800F07C01F03E03E01F07C00FFF8007FF0001 FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07C07C03C07803C0F803C0F003C0F003C0F003 C0F003C0F003C0F003C0F803C07803C07803C03C07C03F0FC01FFFC00FFBC003E3C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C012207E9519>II<07F01FFC3FFE3C0E7806780078007C003F003FF01FF8 0FFC01FE001F000F000F000FC00FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F 000F000F00FFF8FFF8FFF80F000F000F000F000F000F000F000F000F000F000F000F000F 000F000F080F1C07FC07F803E00E1C7F9B12>III< 7801F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F80000700000F800 01FC0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F81516809516> 120 DI<7FFF C07FFFC07FFFC0000F80000F00001E00003E00007C0000780000F00001F00003E00003C0 000780000F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012167F9515>I E /Fh 36 90 df<381C7C3E7C3E7E3F3A1D040204020402080408041008201040208040 100E7A9F17>34 D<1C3E7E7E3A0202040408081020C0070E7D840D>44 DI<3078F8787005057C840D>I<007E0001C3000301800701C00E 00C00E00E01C00E01C00E03C01E03801E07801E07801E07801E07801E07801E0F003C0F0 03C0F003C0F003C0F003C0F00380F00780E00780E00700E00700E00E00600E00701C0030 38003870000FC000131F7C9D17>48 D<000C001C00FC0F38003800380038003800380070 0070007000700070007000E000E000E000E000E000E001C001C001C001C001C001C00380 03C0FFFE0F1E7C9D17>I<003F8000C1E00100F00200780400780400780F007C0F807C0F 807C0F00780600780000F80000F00001E00001C0000380000700000E00001C0000380000 600000C0000180000300200600200800401000403FFFC07FFF80FFFF80161E7E9D17>I< 007F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E00003C0000380 000700000E0000F800000E000007000007800007C00003C00007C03007C07807C0F807C0 F807C0F00780800F00400E00201C0018780007E000141F7D9D17>I<0000600000600000 E00001C00003C00005C0000DC00009C00011C000238000438000C3800083800103800203 80040700080700180700100700200700400700FFFFF0000E00000E00000E00000E00000E 00001C00001E0001FFE0141E7E9D17>I<01803001FFE003FFC003FF0003FC0002000002 0000020000040000040000040000047C000587000603800C01800801C00001C00001E000 01E00001E00001E07003C0F803C0F003C0E00380800780400700400E00201C0018700007 C000141F7D9D17>I<2000003FFFE07FFFC07FFF80400100C00200800200800400000800 0010000020000040000040000080000180000300000300000700000600000E00000E0000 1E00001C00001C00003C00003C00003C0000780000780000780000300000131F799D17> 55 D<003F0000C1C00100600200600400300C00300C00300C00300C00600E00600F80C0 0FC18007F60003FC0001FC0001FF00063F800C0F801007C03003C06001C06000C0C000C0 C000C0C000C0C00080C0010060030030040018180007E000141F7D9D17>I<007E0001C3 000301800601C00E01C01C00C03C00E03C00E03C01E07801E07801E07801E07801E07803 E07803E03803C03807C01C0BC00C13C003E380000780000780000700000E00600E00F01C 00F01800E0300080600041C0003F0000131F7C9D17>I<0E1F1F1E0C0000000000000000 00003078F8787008147C930D>I<00001000000018000000380000003800000078000000 78000000FC000001BC0000013C0000033C0000023C0000063C0000043E0000081E000008 1E0000101E0000101E0000201E0000200F0000400F0000400F0000FFFF0000800F000100 0F8001000780020007800200078004000780040007800C0007C03E0007C0FF807FFC1E20 7E9F22>65 D<07FFFF00007C01C0003C01E0003C00F0007800F8007800F8007800F80078 00F8007800F8007800F000F001F000F001E000F003C000F00F8000FFFE0000F00F0001E0 07C001E003C001E003E001E001E001E001E001E001E003C001E003C003E003C003E003C0 03C003C007C003C00F8007800F0007803E00FFFFF0001D1F7E9E20>I<0001F808000E06 1800380138007000F801E0007803C0007007800030078000300F0000301F0000301E0000 303E0000203C0000007C0000007C0000007C0000007C000000F8000000F8000000F80000 00F8000000F80000007800004078000080780000803C0000803C0001001C0002000E0002 0006000C000300100001C0E000003F00001D217B9F21>I<07FFFF00007C01E0003C00F0 003C00780078003C0078003C0078001E0078001E0078001E0078001F00F0001F00F0001F 00F0001F00F0001F00F0001F00F0001F01E0001E01E0003E01E0003E01E0003E01E0003C 01E0007C03C0007803C000F003C000F003C001E003C003C003C0078007800F0007803C00 FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038003C0018007800180078000800 78000800780008007800080078080800F0100000F0100000F0100000F0300000FFF00000 F0700001E0200001E0200001E0200001E0200001E0000801E0001003C0001003C0001003 C0002003C0002003C0006003C000C0078001C0078007C0FFFFFF801D1F7E9E1F>I<07FF FFF8007C0078003C0038003C001800780018007800080078000800780008007800080078 000800F0100000F0100000F0100000F0300000F0700000FFF00001E0600001E0200001E0 200001E0200001E0200001E0000003C0000003C0000003C0000003C0000003C0000003C0 00000780000007C00000FFFE00001D1F7E9E1E>I<0001FC04000F030C003C009C007000 7C00E0003C01C0003803800018078000180F0000181F0000181E0000183E0000103C0000 007C0000007C0000007C0000007C000000F8000000F8000000F8007FFCF80003E0780001 E0780001E0780003C0780003C03C0003C03C0003C01C0003C00E0007C007000B80038011 8001E06080003F80001E217B9F24>I<07FFC7FFC0007C00F800003C007800003C007800 007800F000007800F000007800F000007800F000007800F000007800F00000F001E00000 F001E00000F001E00000F001E00000FFFFE00000F001E00001E003C00001E003C00001E0 03C00001E003C00001E003C00001E003C00003C007800003C007800003C007800003C007 800003C007800003C007800007800F000007C00F8000FFF8FFF800221F7E9E22>I<07FF E0007C00003C00003C0000780000780000780000780000780000780000F00000F00000F0 0000F00000F00000F00001E00001E00001E00001E00001E00001E00003C00003C00003C0 0003C00003C00003C00007800007C000FFFC00131F7F9E10>I<00FFFC0007C00003C000 03C0000780000780000780000780000780000780000F00000F00000F00000F00000F0000 0F00001E00001E00001E00001E00001E00001E00003C00303C00783C00F83C00F83800F0 780080700040E00021C0001F000016207D9E17>I<07FFF000007E0000003C0000003C00 0000780000007800000078000000780000007800000078000000F0000000F0000000F000 0000F0000000F0000000F0000001E0000001E0000001E0000001E0000001E0008001E001 0003C0010003C0010003C0030003C0020003C0060003C0060007801E0007807C00FFFFFC 00191F7E9E1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800004E00 02F000004E0002F000004E0004F000004E0004F000004E0008F000004E0008F000008700 11E00000870011E00000870021E00000870021E00000870041E00000838041E000010380 83C00001038083C00001038103C00001038203C0000101C203C0000101C403C0000201C4 0780000201C80780000201C80780000201D00780000200F00780000600E00780000600E0 0F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E0018 00003E001800004F001000004F001000004780100000478010000043C010000043C01000 0083C020000081E020000081E020000080F020000080F020000080782000010078400001 007C400001003C400001003C400001001E400001001E400002000F800002000F80000200 0F800002000780000200078000060003800006000300000F00010000FFE0010000221F7E 9E22>I<0003F800001E0E000038070000E0038001C001C003C001E0078000E00F0000F0 0F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8 F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C000780 3C000F003C001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07 FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F8007800F8007800F800 7800F000F001F000F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001 E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003 C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C 01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F0 078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E0 1E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC 0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E 00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E000 01E00000E00000F00000F04000E04000E04000E04000E06000C0600180E00380F80300C6 0C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010 C00F0010800F0010800F0010800F0010001E0000001E0000001E0000001E0000001E0000 001E0000003C0000003C0000003C0000003C0000003C0000003C00000078000000780000 0078000000780000007800000078000000F0000001F800007FFFE0001C1F7A9E21>IIII< FFF003FF1F8000F80F0000600F8000400780008007C0018003C0010003E0020001E00400 01F00C0001F0080000F0100000F8200000786000007C4000003C8000003F0000001F0000 001E0000001E0000001E0000001C0000003C0000003C0000003C0000003C0000003C0000 0038000000780000007C00000FFFC000201F7A9E22>89 D E /Fi 24 122 df<7FF0FFE07FE00C037D8A10>45 D<70F8F8F0E005057B840E>I<0000020000 0006000000060000000E0000001E0000001E0000003F0000002F0000004F0000004F0000 008F0000010F0000010F0000020F0000020F0000040F00000C0F0000080F0000100F0000 100F0000200F80003FFF800040078000C007800080078001000780010007800200078002 000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFFE001E001C001E000C00 1E0004001E0004003C0004003C0004003C0004003C000400780808007808000078080000 78180000F0300000FFF00000F0300000F0300001E0200001E0200001E0200001E0001003 C0002003C0002003C0004003C00040078000800780018007800100078007000F001F00FF FFFE001F1F7D9E1F>69 D<01FFFF00001E03C0001E00E0001E0070001E0078003C007800 3C0078003C0078003C0078007800F0007800F0007801E0007801C000F0070000F01E0000 FFF00000F0380001E01C0001E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003 C01E0007803C0007803C0807803C0807803C100F801C10FFF00C20000007C01D207D9E21 >82 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00 F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F000 00F00000F00000F0000070010070020030040018380007C00011147C9315>99 D<0000780003F80000700000700000700000700000E00000E00000E00000E00001C00001 C000F1C00389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C01C2 07010E011C013C013802780C7BF07C00F000F000F000F0007000700170023804183807C0 10147C9315>I<00007800019C00033C00033C000718000700000700000E00000E00000E 00000E00000E0001FFE0001C00001C00001C00001C000038000038000038000038000038 0000700000700000700000700000700000700000E00000E00000E00000E00000C00001C0 0001C0000180003180007B0000F300006600003C00001629829F0E>I<003C6000E27001 C1E00380E00700E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C03803C 07003C07001C0F001C17000C2E0003CE00000E00000E00001C00001C00301C00783800F0 700060E0003F8000141D7E9315>I<01E0000FE00001C00001C00001C00001C000038000 038000038000038000070000070000071E000763000E81800F01C00E01C00E01C01C0380 1C03801C03801C0380380700380700380700380E10700E20700C20701C20700C40E00CC0 60070014207D9F17>I<00C001E001E001C000000000000000000000000000000E003300 230043804300470087000E000E000E001C001C001C003840388030807080310033001C00 0B1F7C9E0E>I<03C01FC0038003800380038007000700070007000E000E000E000E001C 001C001C001C0038003800380038007000700070007100E200E200E200E200640038000A 207C9F0C>108 D<1C0F80F0002630C318004740640C004780680E004700700E00470070 0E008E00E01C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038 001C01C0708038038071003803806100380380E100380380620070070066003003003800 21147C9325>I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00 E01C01C01C01C01C01C01C03843803883803083807083803107003303001C016147C931A >I<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F0 03C0F003C0F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E0 02621804741C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C 01C0380380780380700380E003C1C0072380071E000700000700000E00000E00000E0000 0E00001C00001C0000FFC000171D809317>I<1C1E002661004783804787804707804703 008E00000E00000E00000E00001C00001C00001C00001C00003800003800003800003800 0070000030000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F8 03FC003E000E700EF00CF00CE008401020601F8010147D9313>I<018001C00380038003 80038007000700FFF007000E000E000E000E001C001C001C001C00380038003800382070 4070407080708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C0 8703800E03800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E20 0C264007C38014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00 800E00800E00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C0 0012147C9315>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E 03801C07001C07001C07001C07001C0E00180E00180E001C1E000C3C0007DC00001C0000 1C00003800F03800F07000E06000C0C0004380003E0000131D7C9316>121 D E /Fj 61 123 df<007F07F001FF1FF003FF3FF007807800070070000F00F0000F00F0 000F00F0000F00F0000F00F0000F00F0000F00F000FFF8FF80FFF8FF80FFF8FF800F00F0 000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0 000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0001C20809F1B>11 D<00F8F001F8F003F8F00780F00700F00F00F00F00F00F00F00F00F00F00F00F00F00F00 F0FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00 F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F01420809F18>13 D<007000E001C00380078007000E001E001E003C003C003C0078007800780078007000F0 00F000F000F000F000F000F000F000F000F000F000F000700078007800780078003C003C 003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DII<00C001 C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17>49 D<07F0000FFC001FFE00383F00700F00600780E00780E003C04003C04003C00003C00003 C00003C0000780000780000F00000E00001C00003C0000780000E00001C0000380000700 000E00001C0000380000700000FFFFC0FFFFC0FFFFC0121F7E9E17>I58 D61 D<001F0000001F0000003F8000003B8000003B8000007BC0000073C0000071C0 0000F1E00000E1E00000E0E00001E0F00001E0F00001C0F00003C0780003C07800038078 0007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F001E000F003C000F 803C0007803C000780780007C0780003C0780003C0F00003E01B207F9F1E>65 DI<001FC000 FFF801FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000780000F0 0000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00003C 00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>IIII<001FE000FFF801FFFE03E03E07800E0F0000 1E00003E00003C00007C0000780000780000780000F00000F00000F00000F00000F00000 F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07801E 03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F000000FF C00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F00780007807800 078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F80007C07800078078000780780007803C000F003C000F001E001E001F00 3E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC00003FFF000 07E1F8000F807C001F003E001E001E003C000F003C000F00780007807800078078000780 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE000F83FC00 07E1F80003FFF00000FFF800003F780000007C0000003E0000001E0000001F0000000F80 1A277DA021>II<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C 00003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E000 01E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA0 19>IIIII<780007807C000F003E001F001E001E000F003C000F807C000780780003C0 F00003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000003F 0000007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C000F00 3E001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>III<07E03FF87FFC701E401F000F000F000F00 3F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000 F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F8 0014147F9317>II<03C7800FF780 1FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F80780 7807807C0F803E1F801FFF800FF78007C780000780000780000780000780000780000780 000780000780000780111D7E9317>II<07F01FFC3FFC780C78 00780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311 >I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E00 1E001E001E001E001E001E201FF00FF007C00C1A7F9910>II< F003C0F003C07803807807807807803C0F003C0F003C0F001E0E001E1E001E1E000F1C00 0F3C000F3C0007380007380007B80003F00003F00001E00012147F9315>II<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F8 0000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414 809315>II<7FFF7FFF7FFF003E00 3C007800F800F001E003E007C007800F001F001E003C007C00FFFFFFFFFFFF10147F9314 >I E /Fk 87 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8 F870051C779B18>33 D<4010E038F078E038E038E038E038E038E038E038E038E038E038 60300D0E7B9C18>I<030600078F00078F00078F00078F00078F00078F007FFFC0FFFFE0 FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFFC0 1E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>I<01C00007E0000FF0 000E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3800FC7000F87000F07 001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E0787070FC707FFFE03FCF E00F03C0141C7F9B18>38 D<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18>I<007000 F001E003C007800F001E001C00380038007000700070007000E000E000E000E000E000E0 00E000E0007000700070007000380038001C001E000F00078003C001F000F000700C2479 9F18>I<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070 00700070007000700070007000E000E000E000E001C001C0038007800F001E003C007800 F00060000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001F FC0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011 147D9718>I<00600000F00000F00000F00000F00000F00000F00000F0007FFFC0FFFFE0 FFFFE07FFFC000F00000F00000F00000F00000F00000F00000F00000600013147E9718> I<1C3E7E7F3F1F070E1E7CF860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D8F 18>I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00001E00 001E00003E00003C00007C0000780000780000F80000F00001F00001E00003E00003C000 03C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C00007C0000 780000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F001C07 003803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000 E0E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F000131C 7E9B18>I<01800380038007800F803F80FF80FB80438003800380038003800380038003 800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03F0000F FE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C00001C000 03C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C00E07F FFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C07801C03001C0 0001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C00000E00000E0 0000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<001F 00003F0000770000770000E70001E70001C7000387000787000707000E07001E07003C07 00380700780700F00700FFFFF8FFFFF8FFFFF80007000007000007000007000007000007 00007FF000FFF8007FF0151C7F9B18>I<1FFF803FFF803FFF8038000038000038000038 00003800003800003800003800003BF8003FFE003FFF003C07801803C00001C00000E000 00E06000E0F000E0F000E0E001C07003C07C0F803FFF001FFC0003F000131C7E9B18>I< 007E0001FF0007FF800F83C01E03C01C03C0380180380000700000700000E1F800E7FE00 FFFF00FE0780F803C0F001C0F000E0E000E0F000E07000E07000E07000E03801C03C03C0 1E07800FFF0007FE0001F800131C7E9B18>II<03F8000FFE001FFF003E0F803803807001C07001C07001C07001C03803803C 07801FFF0007FC000FFE001F1F003C07807001C0F001E0E000E0E000E0E000E0E000E070 01C07803C03E0F801FFF000FFE0003F800131C7E9B18>I<03F0000FFC001FFE003C0F00 780780700380E001C0E001C0E001C0E001E0E001E07001E07803E03C0FE01FFFE00FFEE0 03F0E00000E00001C00001C00001C0300380780780780F00783E003FFC001FF00007C000 131C7E9B18>I<3078FCFC783000000000000000003078FCFC78300614779318>I<183C7E 7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<000300000780 001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC00007E0000 3F00001FC00007E00003F00001FC00007E00003F00001F8000078000030011187D9918> I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07FFF C0130C7E9318>I<600000F00000FC00007E00003F00001FC00007E00003F00001FC0000 7E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC 0000F0000060000011187D9918>I<0FF0003FFC007FFF00700F00F00380F00380600780 000F00003E00007C0001F00001E00003C00003C00003C00003C00003C000038000000000 00000000000000000000000003800007C00007C00007C000038000111C7D9B18>I<0070 0000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E00038E00038E 000306000707000707000707000707000FFF800FFF800FFF800E03800E03801C01C01C01 C07F07F0FF8FF87F07F0151C7F9B18>65 D I<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E0700000E00000E000 00E00000E00000E00000E00000E00000E000007000007000E07000E03800E03C00E01E01 C00F07C007FF8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C 03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C00701C00701C00701C 00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B 18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001C0700000E0 0000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001C07003C03803C038 03C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C0 1C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C0 1C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0 151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7F07F0FF87F87F07F01C 03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0001DF0001DF0001FB8001F B8001F1C001E1C001C0E001C0E001C07001C07001C03801C03801C01C07F03F0FF87F87F 03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C0 1CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C0 1C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE 007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380E00380E00380E00380F00780700700780F007FFF 003FFE000FF800111C7D9B18>II<0FF800 3FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E1E380E1E380F0E78070F700787F00 7FFF003FFE000FFC00001C00001E00000E00000F0000070000070011227D9B18>I<7FF8 00FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF 001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C039C1C039C1C03 9C7F01F8FF81F87F00F0161C7F9B18>I<03F3801FFF803FFF807C0F80700780E00380E0 0380E00380E000007000007800003F00001FF00007FE0000FF00000F800003C00001C000 00E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I< 7FFFF8FFFFF8FFFFF8E07038E07038E07038E07038007000007000007000007000007000 007000007000007000007000007000007000007000007000007000007000007000007000 00700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F80001F8 0000F00000F00000700000F00000F80001F80001DC00039E00038E00070F000707000E07 800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F00001E00001C00 003C0000780000700000F00001E00001C00003C0000780000700000F00001E00E01C00E0 3C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II<600000F0 0000F00000F800007800007C00003C00003C00003E00001E00001F00000F00000F00000F 800007800007C00003C00003C00003E00001E00001F00000F00000F80000780000780000 7C00003C00003E00001E00001E00001F00000F00000F8000078000078000030011247D9F 18>II<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<061E3E387070E0E0E0F8FC7C7C38070E789E18>I<1FE0003FF8007FFC00781E00300E 0000070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F00F00781F 003FFFF01FFBF007E1F014147D9318>I<7E0000FE00007E00000E00000E00000E00000E 00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E 00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C809B18>I< 01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E00000E00000 E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80003F 80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80380F807007 80700380E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF F00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F00380780700380700380E0 01C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF8007FF0001 FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001C0007FFFC0 FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFF F81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC001FF80039E0 003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E000387800 F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00000E00000E 00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B 18>I<03800007C00007C00007C0000380000000000000000000000000007FC000FFC000 7FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I<0038007C007C007C003800 000000000000000FFC1FFC0FFC001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C001C001C6038F078FFF07FE03F800E277E9C18 >II<7FE000FFE0007FE00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E 9B18>I<7CE0E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 7F1F1F00FFBFBF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3 FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E0 00E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001 F00013147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E0038 0E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E0000 0E00000E00000E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<01E3 8007FB801FFF803E1F80380F80700780700780E00380E00380E00380E00380E00380E003 80700780700780380F803C1F801FFF800FFB8003E3800003800003800003800003800003 80000380000380003FF8003FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F87803 F03003E00003C00003C00003800003800003800003800003800003800003800003800003 80007FFE00FFFF007FFE0015147F9318>I<07F7003FFF007FFF00780F00E00700E00700 E007007C00007FE0001FFC0003FE00001F00600780E00380E00380F00380F80F00FFFF00 FFFC00E7F00011147D9318>I<0180000380000380000380000380007FFFC0FFFFC0FFFF C00380000380000380000380000380000380000380000380000380000380400380E00380 E00380E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F 03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E0380 0E0380070700070700070700038E00038E00038E00038E0001DC0001DC0001DC0000F800 00F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F80000F80000 700000F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF87F8FF015147F93 18>I<7F8FF0FF8FF87F8FF00E01C00E03800E0380070380070700070700038700038600 038E0001CE0001CE0000CC0000CC0000DC00007800007800007800007000007000007000 00F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<3FFFF07FFFF07FFF F07001E07003C0700780000F00001E00003C0000F80001F00003C0000780000F00701E00 703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I<0007E0001FE0007FE000780000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00001E0007F C000FF8000FF80007FC00001E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E000007800007FE0001FE00007E013247E9F18>I<7C0000FF0000 FFC00003C00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000F000007FC0003FE0003FE0007FC000F00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00003C000FFC000FF00007C000013247E9F18> 125 D E /Fl 16 122 df<020408103020604040C0C0C0C0C0C0C0C04040602030100804 02071A7F920C>40 D<8040201018080C0404060606060606060604040C08181020408007 1A7E920C>I<1F00318060C04040C060C060C060C060C060C060C060C060404060C03180 1F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C000C00 0C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006000C 00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F0000800040 0060C060C060804060801F000B107F8F0F>I<0300030007000F000B0013003300230043 00C300FFE003000300030003001FE00B107F8F0F>I<20803F002C002000200020002F00 30802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C06000C0 00CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC08080 808001000200040004000C0008000800180018001800180018000B117E900F>I<1F0031 8060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F>I< 1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B107F8F 0F>I<0FE0018001800180018001800180018001800180018001800180C180C18043003E 000B117E9010>74 D108 D117 D121 D E /Fm 4 64 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFF FF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D21 D62 D<00040000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000FFFFFFE0FFFFFFE01B1C7C9B23>I E /Fn 54 123 df<000FF01FE0003FF07FE000FFF0FFE001FFF1FFE001F003F00003E003E00003 C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C0 07C00007C007C000FFFF07FF00FFFF07FF00FFFF07FF0007C007C00007C007C00007C007 C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C0 0007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C000 07C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00023 2A7FA923>11 D<00000F80003F8F80007F8F8000FF8F8001FF8F8003E0000003C0000007 C0000007C0000007C0000007C0000007C0000007C0000007C0000007C0000007C00000FF FF8F80FFFF8F80FFFF8F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007 C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007 C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8019 2B7FAA20>I34 D44 DII<007F000001FFC00007FFF0 000FFFF8000FC1F8001F007C003F007E003E003E003C001E007C001F007C001F007C001F 0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F 80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F8078000F007C001F 007C001F007C001F003E003E003E003E003F007E001F80FC000FC1F8000FFFF80007FFF0 0001FFC000007F000019297EA71E>48 D<00180000380000F80007F800FFF800FFF800F8 F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF07FFFF014287BA71E>I< 00FE0003FFC007FFE00FFFF01F03F83C00FC38007E78003E70003EF0001FF0001F60001F 20001F00001F00001F00001F00003E00003E00007C00007C0000F80001F00001E00003C0 000780000F00001E00003C0000780000F00001E00003C0000780000F00001E00003C0000 7FFFFF7FFFFF7FFFFF7FFFFF18287EA71E>I<007F000001FFC00007FFF0000FFFF8001F C1F8003E007C003C003E0078003E0038003E0010003E0000003E0000003E0000003C0000 007C000000FC000001F8000007F00000FFE00000FFC00000FFE00000FFF0000001FC0000 007C0000003E0000001F0000001F0000000F8000000F8000000F8000000F8000000F8040 000F8060001F00F0001F00F8003F007E007E003F81FC001FFFF8000FFFF00003FFE00000 7F000019297EA71E>I<0003F0000007F0000005F000000DF000000DF000001DF0000039 F0000039F0000079F0000079F00000F1F00000F1F00001E1F00003E1F00003E1F00007C1 F00007C1F0000F81F0000F81F0001F01F0001F01F0003E01F0007C01F0007C01F000F801 F000FFFFFF80FFFFFF80FFFFFF80FFFFFF800001F0000001F0000001F0000001F0000001 F0000001F0000001F0000001F0000001F0000001F00019277EA61E>I<3FFFFC3FFFFC3F FFFC3FFFFC3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E 3F003EFFC03FFFE03FFFF03FE1F83F807C3F003E3E003E00003E00001F00001F00001F00 001F00001F00001F00001F20001F60003E70003EF8007C7C00FC3F03F81FFFF00FFFE007 FF8000FE0018287EA61E>I<000FF000003FFC0000FFFC0001FFFC0003F80C0007E00000 0FC000000F8000001F0000001E0000003E0000003C0000007C0000007C0000007C3FE000 F8FFF000F9FFF800FBFFFC00FF807E00FF003E00FE003F00FC001F00FC001F00FC000F80 F8000F80F8000F80F8000F80F8000F8078000F807C000F807C000F807C000F003E001F00 3E001F001F003E001F807C000FC1FC0007FFF80003FFF00001FFC000007F000019297EA7 1E>II<007F000001FFC00007FFF0000FFF F8001FC1FC003F007E003E003E007E003F007C001F007C001F007C001F007C001F007C00 1F003E003E003E003E001F007C000FC1F80007FFF00003FFE00003FFE0000FFFF8001FC1 FC003F007E003E003E007C001F007C001F00F8000F80F8000F80F8000F80F8000F80F800 0F80F8000F807C001F007C001F007E003F003F007E001FC1FC000FFFF80007FFF00003FF E000007F000019297EA71E>I<007F000001FFC00003FFE0000FFFF0000FC1F8001F007C 003E007C007C003E007C001E007C001F00F8001F00F8001F00F8000F00F8000F80F8000F 80F8000F80F8000F80F8001F807C001F807C001F807E003F803E007F803F00FF801FFFEF 800FFFCF8007FF8F8003FE1F0000001F0000001F0000001E0000003E0000003E0000007C 0000007C000000F8001801F0001E07E0003FFFC0001FFF80000FFE000003F8000019297E A71E>I I<0001F000000003F800000003F800000007FC00000007BC00000007BC0000000F3E0000 000F1E0000000F1E0000001F1F0000001E1F0000001E0F0000003E0F8000003C0F800000 3C078000007C07C000007807C00000F803E00000F803E00000F003E00001F001F00001F0 01F00001E001F00003E000F80003E000F80003C000F80007FFFFFC0007FFFFFC000FFFFF FE000F80003E000F80003E001F00003F001F00001F001E00001F003E00000F803E00000F 803C00000F807C000007C07C000007C078000007C0F8000003E0F8000003E0232A7EA928 >65 DI<0001 FF00000FFFE0003FFFF8007FFFF800FE01F801F8003003F0001007C000000F8000001F80 00001F0000003E0000003E0000007E0000007C0000007C0000007C000000F8000000F800 0000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F80000007C00 00007C0000007C0000007E0000003E0000003E0000001F0000001F8000000F80000007C0 000003F0000401F8001C00FE00FC007FFFFC003FFFF8000FFFE00001FF001E2C7CAA26> III73 D77 DI<0001FC0000000FFF8000003FFFE000007FFFF00001FE 03FC0003F800FE0007E0003F0007C0001F000F80000F801F000007C01F000007C03E0000 03E03E000003E07C000001F07C000001F07C000001F078000000F0F8000000F8F8000000 F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8 F8000000F87C000001F07C000001F07C000001F07E000003F03E000003E03F000007E01F 000007C01F80000FC00FC0001F8007E0003F0007F0007F0003F800FE0001FE03FC0000FF FFF800003FFFE000000FFF80000001FC0000252C7DAA2C>II<007FC00001FFF80007FFFE000FFFFF00 1FC07F003F000F007E0006007C0000007C000000F8000000F8000000F8000000F8000000 F8000000FC0000007E0000007F0000003F8000001FF800000FFF800007FFE00003FFF800 00FFFC00000FFE000000FF0000003F0000001F8000000F8000000FC0000007C0000007C0 000007C0000007C0000007C0000007C000000F8060000F80F0001F00FC003F00FF80FE00 7FFFFC001FFFF80007FFE00000FF80001A2C7DAA21>83 DI86 D89 D<01FE000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F80000F80000F80000 F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F800F8F800F8FC01F87E07F87FFF F83FFFF81FFCF80FE0F8151B7E9A1D>97 DI<007FC001FFF007FFFC0FFFFC1FC07C1F00083E00007C00007C00007C00 00F80000F80000F80000F80000F80000F80000F800007C00007C00007E00003E00001F00 0C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<00003E00003E00003E00003E00 003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00FC3E03 FF3E07FFFE0FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8 003EF8003EF8003EF8003EFC003E7C003E7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03 FF3E00FC3E172A7EA91F>I<007E0003FF8007FFC00FFFE01F83F03F00F03E00787C0078 7C003878003CFFFFFCFFFFFCFFFFFCFFFFFCF80000F80000F800007800007C00007C0000 3E00003F000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<001FC0007FC000FF C001FFC003F00003E00007C00007C00007C00007C00007C00007C00007C00007C00007C0 00FFFE00FFFE00FFFE0007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C000122A7FA912>I<00F8078003FE7FC00FFFFFC01FFFFFC01F07C0 003E03E0003E03E0007C01F0007C01F0007C01F0007C01F0007C01F0007C01F0003E03E0 003E03E0001F07C0001FFFC0003FFF80003BFE000038F8000078000000780000003C0000 003FFFC0003FFFF8001FFFFC001FFFFE003FFFFF007C007F00F8001F80F8000F80F8000F 80F8000F80FC001F807E003F003F80FE003FFFFE000FFFF80007FFF00000FF80001A287E 9A1E>III<007C007C007C007C007C0000000000000000000000000000000000000000 007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C 007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C40FCF1F8FFF8 FFF07FE00F800E3683A910>I108 DII<007F000001FFC00007FFF0000FFFF8001FC1FC003F 007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001FC1FC000F FFF80007FFF00001FFC000007F0000191B7E9A1E>II114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8 003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFF C07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007 C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003 FFE003FFE001FF8000FC0013227FA116>IIII121 DI E /Fo 9 122 df77 D<0007FFFC000000007FFFFFC0000001FFFFFFF8000003FFFFFFFE000007FE 001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000FFF0000FFF80007FF0000FFF80 007FF8000FFF80007FF80007FF00003FFC0007FF00003FFC0003FE00003FFC0000F80000 3FFC00000000003FFC00000000003FFC00000000003FFC00000000003FFC00000007FFFF FC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC0003FFF0003FFC 0007FFC0003FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC007FF800003FFC00 7FF800003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FF F000003FFC00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC0003EFFC003FFF 0007CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFFF001FFFC0003FF 80007FF8362E7DAD3A>97 D<00001FFFC0000000FFFFF8000007FFFFFE00001FFFFFFF80 007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FFE007FF0003FFE00FFE0003FFE0 0FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF803FFC00003E007FF800000000 7FF8000000007FF800000000FFF800000000FFF800000000FFF800000000FFF800000000 FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000 7FF8000000007FF8000000007FFC000000003FFC000000003FFC000000001FFC000000F8 1FFE000000F80FFE000000F80FFF000001F007FF800003F003FFC00007E001FFE0000FC0 00FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8000000FFFFE00000001FFE0000 2D2E7CAD35>99 D<00001FFE00000001FFFFE0000007FFFFF800001FFFFFFE00007FFC07 FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00003FF00FFE00001FF01FFE0000 0FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC000007FC7FF8000007FC7FF80000 07FE7FF8000007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFF FFFEFFFFFFFFFFFCFFF800000000FFF800000000FFF800000000FFF8000000007FF80000 00007FF8000000007FFC000000003FFC000000003FFC000000003FFC0000001C1FFE0000 003E0FFE0000003E07FF0000007E07FF000000FC03FF800001F801FFC00003F0007FF000 1FE0003FFE00FFC0001FFFFFFF800007FFFFFE000000FFFFF80000000FFF80002F2E7DAD 36>101 D<00FC0001FE0003FF0007FF800FFFC01FFFE01FFFE01FFFE01FFFE01FFFE01F FFE00FFFC007FF8003FF0001FE0000FC0000000000000000000000000000000000000000 0000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFF FFFFFFFFFFFFFFFF18497CC820>105 D<007FC000FFFFC000FFFFC000FFFFC000FFFFC0 00FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FFFFFF 80FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<007FC001FFC00000FFFF C00FFFF80000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1FC07FF8000FFFFC3E003FF C00003FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE00001FFDC0000FFE00001FF FC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF00000FFF00001FFF00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF000FFFFFFC07FFF FFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B2E 7CAD42>110 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FFE01FC0001F E03F80000FE03F000007E07F000003E07F000003E0FF000003E0FF000003E0FF800003E0 FFC0000000FFF0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFFFFFC001F FFFFFF000FFFFFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FFFFF80000 1FFFF8000000FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003FCFC0000 03FCFE000003F8FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0FFFC00FF C0FFFFFFFF80FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>115 D<7FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFF C000FFFF8000FFF000000FE00000FFF800000FC00000FFF800000FC000007FFC00000F80 00007FFC00001F8000003FFC00001F0000003FFE00003F0000001FFE00003E0000001FFF 00007E0000000FFF00007C0000000FFF8000FC00000007FF8000F800000007FFC001F800 000003FFC001F000000003FFE003F000000003FFE003F000000001FFF003E000000001FF F007E000000000FFF007C000000000FFF80FC0000000007FF80F80000000007FFC1F8000 0000003FFC1F00000000003FFE3F00000000001FFE3E00000000001FFF7E00000000000F FF7C00000000000FFFFC00000000000FFFFC000000000007FFF8000000000007FFF80000 00000003FFF0000000000003FFF0000000000001FFE0000000000001FFE0000000000000 FFC0000000000000FFC00000000000007F800000000000007F800000000000003F000000 000000003F000000000000003F000000000000003E000000000000007E00000000000000 7C00000000000000FC000000001F8000F8000000003FC001F8000000007FE001F0000000 00FFF003F000000000FFF003E000000000FFF007E000000000FFF00FC000000000FFF01F 8000000000FFF03F80000000007FE07F00000000007F43FE00000000003FFFF800000000 001FFFF0000000000007FFC0000000000001FE00000000000039427EAD3F>121 D E /Fp 8 117 df<0001FF8000001FFFF000007FFFFC0000FF81FE0001FE007F0007FC 003F800FF8003FC00FF8001FE01FF8001FF03FF0001FF03FF0000FF87FF0000FF87FF000 0FFCFFF0000FFCFFF0000FFCFFF0000FFEFFF0000FFEFFF0000FFEFFF0000FFEFFF0000F FEFFF0000FFFFFF0000FFFFFF0000FFF7FF0000FFF7FF0001FFF7FF0001FFF3FF0001FFF 3FF0001FFF1FF0003FFF0FF8003FFF07F8007FFF03FC006FFF01FE01CFFF00FFEF8FFF00 7FFF0FFF000FFE0FFE0000000FFE0000000FFE0000000FFE0000000FFC0000000FFC03E0 001FFC07F0001FF80FF8001FF81FFC001FF01FFC003FF01FFC003FE01FFC003FC01FF800 7FC00FF000FF800FE001FF0007C003FE0007F01FFC0003FFFFF00001FFFFC000007FFF00 00000FF8000028397CB731>57 D<0000001FFF000030000001FFFFE000F000000FFFFFFC 01F000007FFFFFFE03F00001FFFE007F87F00003FFE0000FCFF0000FFF000003FFF0001F FC000001FFF0003FF80000007FF0007FF00000003FF000FFC00000003FF001FFC0000000 1FF003FF800000000FF007FF000000000FF00FFF0000000007F00FFE0000000007F01FFE 0000000003F01FFE0000000003F03FFC0000000003F03FFC0000000001F03FFC00000000 01F07FFC0000000001F07FF80000000001F07FF80000000000007FF8000000000000FFF8 000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800000000 0000FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF8 000000000000FFF80000000000007FF80000000000007FF80000000000007FF800000000 00007FFC0000000000F03FFC0000000000F03FFC0000000000F03FFC0000000000F01FFE 0000000000F01FFE0000000001E00FFE0000000001E00FFF0000000001E007FF00000000 03C003FF8000000003C001FFC0000000078000FFE00000000F00007FF00000001F00003F F80000003E00001FFC0000007C00000FFF000001F8000003FFE00007F0000001FFFE003F C00000007FFFFFFF000000000FFFFFFC0000000001FFFFF000000000001FFF0000003C3D 7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF0000 1FFC00FF80001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF000 07F0003FF00001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000 000FFFFFF000007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF000 3FE0003FF0007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000 FFC0003FF000FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF0 07FFFE0FFFF001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800 003FFFFE0000FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03F F00007F83FF00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFF FFFFFCFFFFFFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE000 00007FE00000007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF80000 7807FC0000F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE000 26267DA52D>101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF00 00000007FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 00000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 00000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 7FC00003FF01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF60 00FF8003FFE000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC 3FFFFFFFFFFC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00 FFFFFE03FF00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF8 03FF00001FFC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF 03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF 03FF000007FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC 03FF00000FFC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC0 03FFF001FF8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF00000000 03FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000 03FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000 FFFFFC000000FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fq 76 124 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<00780000008400000184000003020000070200000702 00000702000007020000070400000704000007080000070800000310000003A00FFC03C0 03E0038001C001C0008001C0010003E0010004E0020008F0020018700400307808007038 0800701C1000F01E1000F00E2000F0074000F003C0087003C0087801C010380670301C18 386007E00F801E227EA023>38 D<70F8FCFC74040404080810102040060E7C9F0D>I<00 20004000800100020006000C000C00180018003000300030007000600060006000E000E0 00E000E000E000E000E000E000E000E000E000E000600060006000700030003000300018 0018000C000C000600020001000080004000200B2E7DA112>I<80004000200010000800 0C00060006000300030001800180018001C000C000C000C000E000E000E000E000E000E0 00E000E000E000E000E000E000C000C000C001C001800180018003000300060006000C00 080010002000400080000B2E7DA112>I<70F8FCFC74040404080810102040060E7C840D> 44 DI<70F8F8F87005057C840D>I<0001000300030006000600 06000C000C000C00180018001800300030003000600060006000C000C000C00180018001 800300030003000600060006000C000C000C001800180018003000300030006000600060 00C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070038070038070 0380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C07003807003807003807807803807001806001C0E000E1C0003F00012 1F7E9D17>I<018003800F80F38003800380038003800380038003800380038003800380 038003800380038003800380038003800380038003800380038007C0FFFE0F1E7C9D17> I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007C00007 C0000780000780000F00000E00001C0000380000700000600000C0000180000300000600 400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E0020 0F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003F00000 3C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040078040 0F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00002E00 002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00200E00 200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0 141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000001000 0010000011F000161C00180E001007001007800003800003800003C00003C00003C07003 C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17>I<00 7C000182000701000E03800C07801C0780380300380000780000700000700000F1F000F2 1C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003C070 03803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF80 7FFF80400100800200800200800400000800000800001000002000002000004000004000 00C00000C00001C000018000038000038000038000038000078000078000078000078000 078000078000078000030000121F7D9D17>I<03F0000C0C001006003003002001806001 806001806001807001807803003E03003F06001FC8000FF00003F80007FC000C7E00103F 00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C 0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700700380F00380F0 0380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C000 0380000380000380000700300700780600780E00700C002018001070000FC000121F7E9D 17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70F8F8F8700000 000000000000000070F0F8F878080808101010202040051D7C930D>I<7FFFFFE0FFFFFF F00000000000000000000000000000000000000000000000000000000000000000FFFFFF F07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000 C001800180010003000200020002000200020002000000000000000000000007000F800F 800F8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C0 000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800002078 000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F 0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F 9F22>65 DI<000FC040007030C0 01C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C000040 78000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007000100 0380020001C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0 070001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000 F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E0 7C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E00460 00781820000FE0001E217D9F24>III<0F FFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0000 3C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00203C00F8 3C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>I76 DII<001F800000F0F000 01C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E0780001E0 780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0 F80001F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F000F00 07801E0001C0380000F0F000001F80001C217D9F23>II 82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F00000 F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C0 8001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19> I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F00 10800F0010000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000001F800007FFFE001C1F7E9E21>II87 D<7FF83FF80FE00FC007C0070003C0020001E0040001F00C0000F0080000781000007C10 00003C2000003E4000001E4000000F8000000F8000000780000003C0000007E0000005E0 000009F0000018F8000010780000207C0000603C0000401E0000801F0001800F00010007 80020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I<080410082010201040204020 804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>92 D<1FE000303000781800781C00 300E00000E00000E00000E0000FE00078E001E0E00380E00780E00F00E10F00E10F00E10 F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E3E000EC3800F01C00F00E00E00E00E 00700E00700E00780E00780E00780E00780E00780E00780E00700E00700E00E00F00E00D 01C00CC300083E0015207F9F19>I<03F80E0C1C1E381E380C70007000F000F000F000F0 00F000F00070007000380138011C020E0C03F010147E9314>I<000380003F8000038000 038000038000038000038000038000038000038000038000038003E380061B801C078038 0380380380700380700380F00380F00380F00380F00380F00380F0038070038070038038 03803807801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E00380700380700 700700700380F00380F00380FFFF80F00000F00000F00000700000700000380080180080 0C010007060001F80011147F9314>I<007C00C6018F038F070607000700070007000700 07000700FFF0070007000700070007000700070007000700070007000700070007000700 0700070007007FF01020809F0E>I<0000E003E3300E3C301C1C30380E00780F00780F00 780F00780F00780F00380E001C1C001E380033E0002000002000003000003000003FFE00 1FFF800FFFC03001E0600070C00030C00030C00030C000306000603000C01C038003FC00 141F7F9417>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E3E000E43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19 >I<1C001E003E001E001C000000000000000000000000000E007E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A1F809E0C>I<00 E001F001F001F000E0000000000000000000000000007007F000F0007000700070007000 7000700070007000700070007000700070007000700070007000700070007000706070F0 60F0C061803F000C28829E0E>I<0E0000FE00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E0FF00E03C00E03000E02000E04000E08000E10000E 30000E70000EF8000F38000E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FF CFF815207F9F18>I<0E00FE000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FF E00B20809F0C>I<0E1F01F000FE618618000E81C81C000F00F00E000F00F00E000E00E0 0E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E 000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE0 23147F9326>I<0E3E00FE43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319 >I<01F800070E001C03803801C03801C07000E07000E0F000F0F000F0F000F0F000F0F0 00F0F000F07000E07000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00 FEC3800F01C00F00E00E00E00E00F00E00700E00780E00780E00780E00780E00780E0078 0E00700E00F00E00E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E0000 0E00000E00000E0000FFE000151D7F9319>I<03E0800619801C05803C07803803807803 80700380F00380F00380F00380F00380F00380F003807003807803803803803807801C0B 800E138003E380000380000380000380000380000380000380000380000380003FF8151D 7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFE00F147F9312>I<1F9030704030C010C010C010E00078007F803FE0 0FF00070803880188018C018C018E030D0608F800D147E9312>I<020002000200060006 000E000E003E00FFF80E000E000E000E000E000E000E000E000E000E000E000E080E080E 080E080E080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0 030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003 810001C20000E40000EC00007800003800003C00007C00004E0000870001070003038002 01C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000E001E001C00380078007010E011E011C03 38027006700EFFFE10147F9314>II E /Fr 47 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000071C000C060018030038 03803803807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C038 03803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038007C0FFFE0F217CA018>I<03F0000C1C0010070020 07804003C04003C08003E0F003E0F801E0F801E0F801E02003E00003E00003C00003C000 0780000700000E00001C0000180000300000600000C00001800001000002002004002008 00201800603000403FFFC07FFFC0FFFFC013217EA018>I<03F8000C1E00100700200780 4007C07807C07803C07807C03807C0000780000780000700000F00000E0000380003F000 001C00000F000007800007800003C00003C00003E02003E07003E0F803E0F803E0F003C0 4003C0400780200780100F000C1C0003F00013227EA018>I<000200000600000E00000E 00001E00001E00002E00004E00004E00008E00008E00010E00020E00020E00040E00040E 00080E00100E00100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E 00000E00000E00000E00001F0001FFF015217FA018>I<1000801E07001FFF001FFE001F F80013E00010000010000010000010000010000010000010F800130E0014070018038010 03800001C00001C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C040 03C04003802007001006000C1C0003F00013227EA018>I<007E0001C1000300800601C0 0E03C01C03C0180180380000380000780000700000700000F0F800F30C00F40600F40300 F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C0 3801C01803801C03000C0600070C0001F00013227EA018>I<01F800060E000803001001 802001802000C06000C06000C06000C07000C07801803E01003F02001FC4000FF80003F8 0003FC00067F00083F80100F803007C06001C06000E0C000E0C00060C00060C00060C000 606000406000C03000801803000E0E0003F00013227EA018>56 D<01F000060C000C0600 180700380380700380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07003E03803E01805E00C05E00619E003E1E00001C00001C00001C0000380000380 300300780700780600700C002018001030000FC00013227EA018>I<0001800000018000 000180000003C0000003C0000003C0000005E0000005E000000DF0000008F0000008F000 0010F800001078000010780000203C0000203C0000203C0000401E0000401E0000401E00 00800F0000800F0000FFFF000100078001000780030007C0020003C0020003C0040003E0 040001E0040001E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E0000701E0000701C0000 303C0000303C0000307C0000107800001078000010F8000000F8000000F8000000F80000 00F8000000F8000000F8000000F800000078000000780000107C0000103C0000103C0000 101C0000201E0000200E000040070000400380008001C0010000E0020000381C000007E0 001C247DA223>67 D69 DI73 D<03FFF0001F00000F00000F00000F00000F00000F00000F00 000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00000F00 000F00000F00000F00000F00000F00000F00700F00F80F00F80F00F80E00F01E00401C00 20380018700007C00014237EA119>I77 DI80 D82 D<03F0200C0C601802603001E07000E0600060E000 60E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF 80003FC00007E00001E00000F00000F0000070800070800070800070800070C00060C000 60E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF8780780786007801840 0780084007800840078008C007800C800780048007800480078004800780040007800000 078000000780000007800000078000000780000007800000078000000780000007800000 078000000780000007800000078000000780000007800000078000000780000007800000 07800000078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E001807 0000070000070000070000FF0007C7001E07003C0700780700700700F00708F00708F007 08F00F087817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C00E8060 0F00300E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C 0E003C0E00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07801C07 80380300780000700000F00000F00000F00000F00000F00000F00000F000007000007800 403800401C00800C010007060001F80012157E9416>I<0000E0000FE00001E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E001F8E00704E00C 02E01C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E070 00E07800E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C0380 1C01C03801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000700000 7800203800201C00400E008007030000FC0013157F9416>I<003C00C6018F038F030F07 0007000700070007000700070007000700FFF80700070007000700070007000700070007 0007000700070007000700070007000700070007807FF8102380A20F>I<00007001F198 071E180E0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E000F1C00 19F0001000001000001800001800001FFE000FFFC00FFFE03800F0600030400018C00018 C00018C000186000306000303800E00E038003FE0015217F9518>I<0E0000FE00001E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F 800E60C00E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E00 3E001E001C00000000000000000000000000000000000E00FE001E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01 C003E003E003E001C00000000000000000000000000000000001E00FE001E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000 E000E000E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01F00E01 C00E01800E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07 800E03C00E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00 FE60E183801E807201C00F003C00E00F003C00E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E 1F80FE60C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC00 0707000C01801800C03800E0700070700070F00078F00078F00078F00078F00078F00078 F000787000707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61 C00E80600F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E00 1E0E003C0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00000E00 000E00000E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E 000E000E000E000E000E000E000E000E000E000E000E000E000F00FFF010157F9413> 114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC0 0CC008E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00 FFF80E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E04 0708030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F00601700382 7800FC7F18157F941B>III121 D E /Fs 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E0 700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F8000000 7C0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF00 000FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F8 8000007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0 F00000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Ft 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 908 a Ft(D)26 b(R)g(A)f(F)h(T)225 999 y Fs(Do)r(cumen)n(t) 20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1194 y Fr(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)828 1320 y(June)i(21,)f(1996)77 1378 y(This)h(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1436 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1494 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1552 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 45 a Fq(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fp(Chapter)34 b(9)75 564 y Fo(Miscellan)m(y)75 786 y Fq(There)12 b(are)g(a)g(n)o(um)o(b)q(er)g(of)g(topics)g(that)f (do)h(not)g(\014t)g(con)o(v)o(enien)o(tly)h(in)o(to)f(the)g(other)g(c)o (hapters.)19 b(W)l(e)12 b(collect)75 843 y(them)j(here.)75 985 y Fn(9.1)59 b(Standa)n(rdize)20 b(mpirun)1875 1042 y Fm(>)c Fl(\(July\))75 1087 y Fq(This)j(will)g(b)q(e)g(c)o(hanged)g (to)e(advice)i(to)f(implemen)o(tors,)h(and)f(the)g(syn)o(tax)f(will)j (b)q(e)f(aligned)g(with)g(the)75 1143 y(spa)o(wn)c(syn)o(tax.)1518 b Fm(?)16 b Fl(\(July\))166 1200 y Fq(A)e(n)o(um)o(b)q(er)g(of)f (implemen)o(tations)i(of)e(MPI-1)g(pro)o(vide)i(a)e(startup)g(command)g (for)h(MPI)f(programs)75 1256 y(that)h(is)i(of)f(the)g(form)170 1344 y Fk(mpirun)23 b()g()g()75 1433 y Fq(Separating)16 b(the)g(command)g(to)g(start)f (the)h(program)f(from)g(the)h(program)f(itself)i(pro)o(vides)g (\015exibilit)o(y)l(,)75 1489 y(particularly)23 b(for)e(net)o(w)o(ork)g (and)h(heterogeneous)g(implemen)o(tations.)41 b(F)l(or)21 b(example,)j(the)e(startup)75 1546 y(script)16 b(need)g(not)f(run)g(on) g(one)g(of)g(the)g(mac)o(hines)h(that)f(will)i(b)q(e)f(executing)g(the) f(MPI)g(program)f(itself.)c Fm(>)16 b Fl(\(July\))166 1602 y Fq(F)l(or)e(example,)i(it)g(has)f(b)q(een)h(prop)q(osed)g(that) 170 1690 y Fk(mpirun)23 b(-np)h()e()75 1779 y Fq(b)q(e)c(at)f(least)g(one)g(w)o(a)o(y)g(to)f(start)g Fk()g Fq(with)i(an)f(initial)j Fj(MPI)p 1275 1779 14 2 v 15 w(COMM)p 1432 1779 V 17 w(W)o(ORLD)d Fq(whose)g(group)75 1835 y(con)o(tains)25 b Fk()e Fq(pro)q(cesses.)48 b(Other)24 b(argumen)o(ts)g(to)g Fk(mpirun)g Fq(ma)o(y)f(b)q(e)j (implemen)o(tation-)75 1892 y(dep)q(enden)o(t.)166 1948 y(Rather)16 b(than)g(require)i(this)e(of)g(all)h(implemen)o(tations,)h (implemen)o(tors)f(are)f(advised,)h(if)g(they)f(do)75 2005 y(pro)o(vide)k(a)g(startup)f(command)h(for)f Fj(MPI)h Fq(programs,)f(that)g(in)i(conform)f(to)f(the)h(syn)o(tax)f(describ)q (ed)75 2061 y(here.)189 2160 y Fi(A)n(dvic)n(e)11 b(to)h(implementors.) 36 b Fq(Eac)o(h)11 b Fk(mpirung)e Fq(is)i(lik)o(ely)h(to)e(ha)o(v)o(e)g (a)g(v)m(ariet)o(y)h(of)f(argumen)o(ts)f(sp)q(eci\014c)189 2216 y(to)19 b(the)h(implemen)o(tation.)36 b(The)20 b(only)g(required)i (argumen)o(ts)d(are)g(those)h(used)h Fj(MPI)p 1699 2216 V 15 w(SP)l(A)-5 b(WN)p Fq(,)189 2273 y(together)17 b(with)i(a)g (\014le)g(in)h(whic)o(h)f(more)f(inofmration)h(can)g(b)q(e)g(supplied.) 32 b(As)19 b(in)g(the)g(case)g(for)189 2329 y(the)h Fj(info)g Fq(argumen)o(t)f(of)h Fj(MPI)p 713 2329 V 15 w(SP)l(A)-5 b(WN)p Fq(,)21 b(the)f(format)f(of)g(the)h(\014le)h(is)g(not)f(sp)q (eci\014ed.)36 b(P)o(ending)189 2386 y(\014naization)16 b(of)f Fj(MPI)p 548 2386 V 15 w(SP)l(A)-5 b(WN)16 b Fq(syn)o(tax,)f (the)g(syn)o(tas)f(of)h Fk(mpirun)f Fq(is)i(suggested)f(to)g(b)q(e:)260 2492 y Fk(mpirun)23 b(-np)h()e()h()189 2598 y Fq(or)260 2704 y Fk(mpirun)g(-info)g()964 2828 y Fq(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fq(2)1161 b Fh(CHAPTER)15 b(9.)30 b(MISCELLANY)189 45 y Fq(or)260 153 y Fk(mpirun)23 b()g()189 262 y Fq(In)18 b(the)f(last)g(case)g(the)h(default)g(n)o(um)o(b)q(er)f (of)g(pro)q(cesses)h(is)g(tak)o(en)e(to)h(b)q(e)h(1.)26 b(\()p Fi(End)17 b(of)i(advic)n(e)f(to)189 318 y(implementors.)p Fq(\))75 476 y Fk(Discussion:)46 b(What)24 b(about)f(multiple)f (binaries)h(and)h(multiple)e(machines?)75 589 y(NAS)h(supports)g(a)h (syntax)f(for)g(running)g(multiple)g(executables:)170 702 y(mpirun)g(-np)h(2)g(foo)f(:)h(-np)f(4)h(bar)f(:)h(-np)g(3)f(bazz) 75 815 y(and)g(MPICH)h(supports)e(a)i(mechanism)f(for)g(finding)g(and)h (running)e(similarly-named)75 871 y(binaries)h(for)g(different)g (architectures:)170 984 y(mpirun)g(-arch)h(sun4)f(-np)g(4)h(-arch)f (rs6000)g(-np)h(3)g(myprog)75 1097 y(All)f(three)h(have)f(the)g(option) g(of)h(a)g(``configuration)e(file'')h(for)g(closer)g(control.)-117 1198 y Fm(?)15 b Fl(\(July\))75 1341 y Fn(9.2)59 b(Datat)n(yp)r(e)18 b(Imp)n(rovements)75 1444 y Fg(9.2.1)49 b(Simple)17 b(Struct)d(T)l(yp)q (es)-117 1483 y Fm(>)h Fl(\(July\))75 1530 y Fq(It)g(is)h(prop)q(osed)g (to)e(drop)h(this)h(section)g(or)e(mo)o(v)o(e)h(to)f(1-sided.)-117 1542 y Fm(?)h Fl(\(July\))75 1681 y Fj(MPI)p 160 1681 14 2 v 16 w(TYPE)p 293 1681 V 17 w(SIMPLE)p 469 1681 V 15 w(STRUCT\(count,)i(a)o(rra)o(y)p 908 1681 V 14 w(of)p 959 1681 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1334 1681 V 14 w(of)p 1385 1681 V 16 w(t)o(yp)q(es,)e(newt)o(yp)q(e\))117 1758 y Ff(IN)155 b Fj(count)482 b Ff(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 1832 y(IN)155 b Fj(a)o(rra)o(y)p 416 1832 V 15 w(of)p 468 1832 V 16 w(blo)q(cklengths)191 b Ff(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h (eac)o(h)g(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 1907 y(IN)155 b Fj(a)o(rra)o(y)p 416 1907 V 15 w(of)p 468 1907 V 16 w(t)o(yp)q(es)327 b Ff(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h (in)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f(handle\))117 1981 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f (\(handle\))166 2106 y Fq(This)22 b(t)o(yp)q(e)g(constructor)f(is)i (similar)f(to)g Fj(MPI)p 988 2106 V 15 w(TYPE)p 1120 2106 V 17 w(STRUCT)p Fq(,)g(except)h(that)e(the)h(user)g(do)q(es)75 2162 y(not)e(supply)h(an)f(arra)o(y)e(of)i(displacemen)o(ts.)35 b(Instead,)22 b(the)e(successiv)o(e)h(blo)q(c)o(ks)f(are)g(assumed)g (to)f(b)q(e)75 2219 y(con)o(tiguous.)g(P)o(adding)11 b(spaces)h(are)e(added,)j(according)e(to)g(the)g(default)h(alignmen)o (t)g(rules)g(for)e(structure)75 2275 y(comp)q(onen)o(ts.)21 b(This)c(facilitates)f(the)g(construction)g(of)f(datat)o(yp)q(es)g(for) g(structures,)g(as)h(the)f(user)h(need)75 2332 y(not)f(compute)g (displacemen)o(ts.)75 2433 y Fe(Example)j(9.1)k Fq(The)14 b(follo)o(wing)g(co)q(de)f(declares)h(a)f(structure)g(t)o(yp)q(e)g(and) h(creates)f(a)g(datat)o(yp)q(e)f(for)h(that)75 2489 y(structure)i(t)o (yp)q(e.)75 2591 y Fk(struct)23 b(record)g({)147 2647 y(char)g(name;)147 2704 y(double)g(position[3];)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fh(9.2.)34 b(D)o(A)l(T)l(A)l(TYPE)15 b(IMPR)o(O)o(VEMENTS)1019 b Fq(3)147 45 y Fk(float)47 b(mass;)147 102 y(})75 214 y(MPI_Datatype)22 b(record_type;)75 271 y(MPI_datatype)g(types[3])h(=)h({MPI_CHAR,)e(MPI_DOUBLE,)h (MPI_FLOAT};)75 327 y(int)238 b(lenghts[3])23 b(=)g({1,)h(3,)f(1};)75 440 y(MPI_Type_simple_struct\()e(3,)i(lengths,)g(types,)g (record_type\);)166 587 y Fd(Discussion:)40 b Ff(The)16 b(co)q(de)g(ab)q(o)o(v)o(e)f(is)h(m)o(uc)o(h)e(simpler)g(and)i(more)e (natural)h(than)g(the)i(curren)o(t)f(approac)o(h,)75 637 y(that)e(uses)h Fc(MPI)p 330 637 13 2 v 14 w(TYPE)p 452 637 V 15 w(STRUCT)p Ff(,)d(and)i(requires)h(one)f(to)g(compute)f (displacemen)o(ts.)166 687 y(It)g(also)g(pro)o(vides)g(a)g (de\014nition)f(of)h(the)h(record)g(t)o(yp)q(e)g(that)f(is)g(arc)o (hitecture)i(indep)q(enden)o(t)f(\(assuming)e(that)75 737 y(compilers)i(don't)i(\014ddle)f(with)g(the)i(order)f(of)f (\014elds,)h(whic)o(h)f(is)g(true)i(in)e(C\))h({)f(this)g(ma)o(y)f (\014t)i(w)o(ell)e(with)i(v)n(arious)75 786 y(prop)q(osals)e(for)f (third)h(part)o(y)g(transfers)h(and/or)f(one)g(sided)g(comm)o (unications.)166 836 y(Ho)o(w)o(ev)o(er,)22 b(this)f(simpli\014cation)d (places)j(a)f(signi\014can)o(t)h(burden)g(on)g(the)g(MPI)g(implem)o(en) o(tation.)36 b(The)75 886 y(implemen)o(tatio)o(n)17 b(will)g(ha)o(v)o (e)i(to)g(\\kno)o(w")f(all)g(the)h(padding)g(rules)g(that)g(the)h (compiler)e(uses)i(for)f(la)o(ying)e(out)75 936 y(structure)g(t)o(yp)q (es)f(in)f(memory)m(.)k(W)m(orse)d(still,)e(the)i(padding)e(rules)i (often)f(v)n(ary)g(from)e(compiler)h(to)h(compiler)f(or)75 986 y(v)o(ersion)h(to)g(v)o(ersion.)22 b(They)15 b(ma)o(y)f(also)g(b)q (e)i(mo)q(di\014ed)d(b)o(y)i(user)i(sp)q(eci\014ed)f(pragmas)e(or)h (options.)21 b(F)m(urther,)16 b(this)75 1035 y(function)g(will)e(only)i (b)q(e)g(v)n(alid)f(for)h(languages)f(whic)o(h)h(guaran)o(tee)h(not)f (to)g(reorder)h(\014elds)g(in)f(a)f(structure)k(\(e.g.)75 1085 y(C\),)13 b(but)i(not)e(languages)h(lik)o(e)f(Ada)h(\(or)g(F)m (ortran?\).)166 1142 y(Th)o(us,)f(this)g(function)g(will)f(b)q(e)i (di\016cult)e(\(or)h(imp)q(ossible\))f(to)h(include)g(in)g(a)g (library-only)e(implemen)o(tation)75 1198 y(of)i(MPI;)h(instead)g(it)g (will)e(require)j(close)f(in)o(tegration)f(with)h(the)g(compiler.)75 1367 y Fg(9.2.2)49 b(Contiguous)17 b(Struct)e(T)l(yp)q(es)1875 1406 y Fm(>)h Fl(\(July\))75 1453 y Fq(This)g(section)g(needs)g(a)e (clari\014cation)j(of)e(ho)o(w)f(it)i(w)o(ould)g(w)o(ork)e(in)i(a)f (heterogeneous)g(en)o(vironmen)o(t.)1875 1465 y Fm(?)h Fl(\(July\))166 1509 y Fq(The)i(de\014nition)h(of)e(a)g(datat)o(yp)q(e) g(exten)o(t)h(ma)o(y)e(sp)q(ecify)j(that)e(a)g(space)h(b)q(e)g(added)h (after)d(the)i(last)75 1566 y(datat)o(yp)q(e)13 b(comp)q(onen)o(t,)h (for)e(alignmen)o(t)j(purp)q(oses.)k(If)14 b(suc)o(h)g(datat)o(yp)q(e)f (is)h(replicated)h(con)o(tiguously)f(in)75 1622 y(a)j(blo)q(c)o(k)h(b)o (y)f(one)g(of)g(the)g(datat)o(yp)q(e)g(constructors,)f(then)i (successiv)o(e,)g(con)o(tiguous)f(comp)q(onen)o(ts)h(of)e(a)75 1679 y(blo)q(c)o(k)g(will)h(b)q(e)f(separated)f(b)o(y)g(these)g (padding)h(spaces.)k(Also,)c(a)e(datat)o(yp)q(e)h(constructed)g(b)o(y) 75 1735 y Fj(MPI)p 160 1735 14 2 v 16 w(TYPE)p 293 1735 V 17 w(STRUCT)h Fq(ma)o(y)e(con)o(tain)h(in)o(ternal)h(holes,)g(b)q (ecause)g(of)f(alignmen)o(t)h(requiremen)o(ts.)166 1792 y(Suc)o(h)f(holes)g(\(padding)g(space)g(b)q(et)o(w)o(een)f(successiv)o (e)i(comp)q(onen)o(ts)e(of)g(a)g(blo)q(c)o(k)h(or)f(padding)i(space)75 1848 y(b)q(et)o(w)o(een)c(comp)q(onen)o(ts)f(of)g(a)g(datat)o(yp)q(e)g (constructed)g(with)h Fj(MPI)p 1188 1848 V 16 w(TYPE)p 1321 1848 V 17 w(STRUCT)p Fq(\))f(generally)i(con)o(tain)75 1905 y(no)18 b(signi\014can)o(t)i(information.)29 b(Ho)o(w)o(ev)o(er,) 18 b(an)g(MPI)h(implemen)o(tation)g(is)g(not)f(allo)o(w)o(ed)h(to)f(o)o (v)o(erwrite)75 1961 y(these)d(holes)g(when)g(data)f(is)h(receiv)o(ed)h (in)o(to)f(a)f(comm)o(unication)i(bu\013er,)e(th)o(us)g(precluding)j (the)e(natural)75 2018 y(optimization)h(of)f(blo)q(c)o(k)h(cop)o(ying)g (data)e(in)o(to)h(the)h(bu\013er.)166 2074 y(The)23 b(t)o(yp)q(e)h (constructor)e Fj(MPI)p 711 2074 V 16 w(TYPE)p 844 2074 V 17 w(CONTIGUOUS)p 1149 2074 V 18 w(STRUCT)i Fq(marks)f(an)o(y)g(suc)o (h)g(holes)h(as)75 2130 y(\\don)o(t-care")14 b(lo)q(cations,)i(p)q (ermiting)g(the)g(implemen)o(tation)g(to)f(o)o(v)o(erwrite)f(them.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fq(4)1156 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(TYPE)p 293 45 V 17 w(CONTIGUOUS)p 598 45 V 18 w(STRUCT\(count,)18 b(a)o(rra)o(y)p 1041 45 V 14 w(of)p 1092 45 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1468 45 V 14 w(of)p 1519 45 V 16 w(displacements,)g(a)o(r-)75 102 y(ra)o(y)p 136 102 V 15 w(of)p 188 102 V 16 w(t)o(yp)q(es,)e(newt)o (yp)q(e\))117 179 y Ff(IN)155 b Fj(count)482 b Ff(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 253 y(IN)155 b Fj(a)o(rra)o(y)p 416 253 V 15 w(of)p 468 253 V 16 w(blo)q(cklengths) 191 b Ff(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h(eac)o(h)g(blo)q (c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 327 y(IN)155 b Fj(a)o(rra)o(y)p 416 327 V 15 w(of)p 468 327 V 16 w(displacements)164 b Ff(b)o(yte)14 b(displacemen)o(t)g(of)f(eac)o(h)h(blo)q(c)o(k)g (\(arra)o(y)g(of)f(in)o(teger\))117 401 y(IN)155 b Fj(a)o(rra)o(y)p 416 401 V 15 w(of)p 468 401 V 16 w(t)o(yp)q(es)327 b Ff(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h(in)f(eac)o(h)h(blo)q(c)o(k)g (\(arra)o(y)g(of)f(handle\))117 476 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)f(\(handle\))166 600 y Fq(The)21 b(seman)o(tics)h(of)e Fj(MPI)p 620 600 V 16 w(TYPE)p 753 600 V 17 w(CONTIGUOUS)p 1058 600 V 18 w(STRUCT)i Fq(are)f(otherwise)g(iden)o(tical)i(to)e(the)75 657 y(function)16 b Fj(MPI)p 338 657 V 16 w(TYPE)p 471 657 V 17 w(STRUCT)p Fq(.)-791 b Fm(>)15 b Fl(\(July\))166 713 y Fq(The)j(part)f(of)g(the)h (bu\013er)g(sen)o(t)f(or)h(pac)o(k)o(ed)f(with)h Fj(MPI)p 1130 713 V 16 w(TYPE)p 1263 713 V 17 w(CONTIGUOUS)p 1568 713 V 18 w(STRUCT)h Fq(m)o(ust)75 770 y(b)q(e)d(receiv)o(ed)g(or)f (unpac)o(k)o(ed)h(with)f(a)g(corresp)q(onding)h Fj(MPI)p 1086 770 V 16 w(TYPE)p 1219 770 V 17 w(CONTIGUOUS)p 1524 770 V 18 w(STRUCT)p Fq(.)-1845 b Fm(?)15 b Fl(\(July\))189 870 y Fi(R)n(ationale.)38 b Fq(C)12 b(struct)f(datat)o(yp)q(es)h(ma)o (y)g(con)o(tain)g(holes)h(due)g(to)e(the)i(need)g(to)e(align)i(basic)g (t)o(yp)q(es.)189 926 y(F)l(or)g(example,)i(the)f(C)g(struct)f (de\014ned)j(in)f(Example)g(9.1)e(ma)o(y)g(con)o(tain)h(a)g(3)g(b)o (yte)g(hole)h(after)e(the)189 982 y(c)o(har)h(on)h(a)f(32)g(bit)h(mac)o (hine,)g(where)g(doubles)h(are)e(w)o(ord)g(aligned,)i(or)e(a)g(7)h(b)o (yte)f(hole)i(if)f(doubles)189 1039 y(are)g(doublew)o(ord)g(aligned.) 189 1113 y(Although)k(the)f(ab)q(o)o(v)o(e)g(t)o(yp)q(e)h(con)o(tains)g (a)f(hole,)h(for)f(e\016ciency)i(it)f(is)g(desirable)h(that)e(the)h Fj(MPI)189 1170 y Fq(implemen)o(tation)j(treat)e(the)i(ab)q(o)o(v)o(e)e (structure)h(as)g(a)g(sequence)h(of)f(con)o(tiguous)h(b)o(ytes)e(when) 189 1226 y(sending)13 b(the)g(t)o(yp)q(e)f(b)q(et)o(w)o(een)h (homogeneous)f(mac)o(hines,)h(and)g(hence)g(mo)o(v)o(e)f(it)h(using)g (an)f(e\016cien)o(t)189 1282 y(memory)20 b(cop)o(y)l(,)j(e.g.)38 b Fk(mempcpy\(\))p Fq(.)f(Otherwise,)23 b(the)f(implemen)o(tation,)h (when)f(receiving)h(a)189 1339 y(message)14 b(of)g(the)h(struct)f (datat)o(yp)q(e)g(m)o(ust)g(actually)i(p)q(erform)e(an)h(elemen)o(t)g (b)o(y)g(elemen)o(t)g(cop)o(y)g(to)189 1395 y(ensure)c(that)g(it)g (preserv)o(es)h(the)f(con)o(ten)o(ts)f(of)h(the)g(holes)h(in)g(the)g (destination.)19 b(\()p Fi(End)12 b(of)h(r)n(ationale.)p Fq(\))189 1495 y Fi(A)n(dvic)n(e)18 b(to)i(users.)61 b Fq(If)19 b Fj(MPI)p 706 1495 V 15 w(TYPE)p 838 1495 V 17 w(STRUCT)h Fq(is)f(used)g(to)f(create)g(an)h Fj(MPI)f Fq(datat)o(yp)q(e)g(corre-)189 1552 y(sp)q(onding)h(to)f(the)h(C)f (datat)o(yp)q(e)g(de\014ned)i(in)g(Example)f(9.1,)f(there)h(is)g(no)f (w)o(a)o(y)g(to)g(indicate)i(to)189 1608 y(the)f Fj(MPI)h Fq(implemen)o(tation)h(that)e(the)h(hole)g(is)g(actually)h(a)e (sequence)i(of)f(\\don)o(t-care")f(b)o(ytes.)189 1665 y(Ho)o(w)o(ev)o(er,)f(if)h Fj(MPI)p 519 1665 V 16 w(TYPE)p 652 1665 V 17 w(CONTIGUOUS)p 957 1665 V 18 w(STRUCT)g Fq(is)g(used)h(to)e(create)g(an)h Fj(MPI)f Fq(datat)o(yp)q(e,)189 1721 y(this)d(indicates)i(to)d(the)i(implemen)o(tation)g(that)f(the)g (padding)h(spaces)g(are)f(not)f(signi\014can)o(t.)189 1795 y(Only)j(padding)f(spaces)g(b)q(et)o(w)o(een)g(\014elds)h(of)e (the)h(struct)g(can)f(b)q(e)i(o)o(v)o(erwritten,)d(an)o(y)i(hole)g (added)189 1852 y(implicitly)22 b(b)o(y)d Fj(MPI)g Fq(due)h(to)f(the)g (default)h(exten)o(t)f(will)i(not)e(b)q(e)h(o)o(v)o(erwritten.)32 b(On)20 b(the)f(other)189 1908 y(hand,)13 b(if)g(the)g(user)g (explicitly)j(de\014ned)e(the)f(upp)q(er)h(b)q(ound)g(of)e(a)h(datat)o (yp)q(e,)f(using)i Fc(MPI)p 1693 1908 13 2 v 14 w(UB)p Fq(,)e(then)189 1965 y(holes)i(in)g(the)f(datat)o(yp)q(e)g(that)f (preceded)j(the)e(user)g(de\014ned)i(upp)q(er)f(b)q(ound)g(will)h(b)q (e)f(o)o(v)o(erwritten.)189 2039 y(Users)j(should)i(arrange)f(the)g (\014elds)h(in)f(a)g(C)g(struct)f(t)o(yp)q(e)h(in)h(descending)h(order) d(of)h(their)g(size.)189 2096 y(This)j(arrangemen)o(t)e(has)h(t)o(w)o (o)f(adv)m(an)o(tages:)30 b(\014rstly)20 b(it)h(ma)o(y)e(reduce)i(the)g (size)g(of)f(the)g(struct)189 2152 y(b)o(y)c(eliminating)i(padding)f(b) q(et)o(w)o(een)f(elemen)o(ts)h(with)f(di\013eren)o(t)h(alignmen)o(ts)f (\(this)g(is)h(generally)189 2208 y(true)h(for)f(C)h(programs\).)28 b(Secondly)l(,)20 b(it)f(ma)o(y)e(increase)i(the)g(n)o(um)o(b)q(er)f (of)g(cases)g(for)g(whic)o(h)h(the)189 2265 y(implemen)o(tation)i(can)e (p)q(erform)h(blo)q(c)o(k)g(copies)h(b)o(y)f(ensuring)g(that)f(the)h (\014rst)f(elemen)o(t)i(of)e(the)189 2321 y(structure)c(has)g(the)g (most)f(stringen)o(t)h(alignmen)o(t)h(requiremen)o(ts.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fq(\))189 2421 y Fi(A)n(dvic)n(e)i(to)i(implementors.)62 b Fq(An)19 b(implemen)o(tation)i(can)e(tak)o(e)f(adv)m(an)o(tage)g(of)h(the)g (leew)o(a)o(y)f(of)189 2478 y(o)o(v)o(erwriting)12 b(holes,)i(if)f (data)f(sen)o(t)h(is)g(alw)o(a)o(ys)f(aligned)i(using)g(the)f(same)f (rules)i(that)e(the)h(compiler)189 2534 y(uses)18 b(for)f(\014elds)j (within)f(a)e(struct)h(t)o(yp)q(e.)28 b(E.g.,)18 b(if)g(the)g(compiler) i(forces)d(half-w)o(ord)h(en)o(tries)h(to)189 2591 y(b)q(e)e(half-w)o (ord)f(aligned,)i(and)f(w)o(ord)f(en)o(tries)h(to)e(b)q(e)j(w)o(ord)d (aligned,)j(then)f(the)g(implemen)o(tation)189 2647 y(should)j(add)g (\\don)o(t-care")e(b)o(ytes)h(to)g(the)g(message)g(to)g(matc)o(h.)32 b(The)19 b(receiving)i(pro)q(cess)f(can)189 2704 y(iden)o(tify)f(the)f (lo)q(cations)h(where)f(\\don)o(t-care")f(b)o(ytes)h(where)g(added)g (from)g(the)g(basic)g(t)o(yp)q(es)g(of)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fh(9.2.)29 b(D)o(A)l(T)l(A)l(TYPE)15 b(IMPR)o(O)o(VEMENTS)1024 b Fq(5)189 45 y(the)12 b(receiv)o(ed)i(en)o (tries.)20 b(It)12 b(can)h(iden)o(tify)h(\(at)d(datat)o(yp)q(e)i (commit)f(time\))h(when)g(suc)o(h)g(\\don)o(t-care")189 102 y(b)o(ytes)d(can)h(b)q(e)g(copied)h(in)o(to)e(the)h(receiv)o(e)h (bu\013er,)f(without)f(o)o(v)o(erwriting)h(signi\014can)o(t)g (information.)189 177 y(NOTE:)17 b(Compilers)i(generally)g(align)g (structures)e(based)i(on)f(the)f(most)h(stringen)o(t)f(alignmen)o(t)189 233 y(constrain)o(ts)f(of)f(their)i(elemen)o(ts.)24 b(F)l(or)16 b(example,)h(the)g(C)f(struct)g(de\014ned)i(in)f(Example)g(9.1)e(will) 189 290 y(often)c(b)q(e)i(aligned)g(on)f(an)g(8)g(b)o(yte)g(b)q (oundary)l(,)g(ev)o(en)h(though)f(the)g(\014rst)f(elemen)o(t)i(could)g (b)q(e)g(aligned)189 346 y(on)g(a)f(1)h(b)o(yte)g(b)q(oundary)l(.)20 b(Th)o(us)13 b(arra)o(ys)f(of)g(this)i(structure)f(will)h(often)f(ha)o (v)o(e)g(4)f(b)o(ytes)h(of)g(padding)189 403 y(b)q(et)o(w)o(een)g (successiv)o(e)h(replications.)21 b(In)14 b(con)o(trast,)d(the)i (elemen)o(ts)h(pac)o(k)o(ed)f(in)o(to)g(a)g(message)g(m)o(ust)189 459 y(alw)o(a)o(ys)h(use)i(an)g(alignmen)o(t)g(rule)g(based)g(solely)h (on)e(their)h(o)o(wn)f(t)o(yp)q(e)h(\(and)f(not)g(their)h(con)o (text\).)189 515 y(Th)o(us,)d(ev)o(en)g(though)g(the)g(\014rst)g Fk(char)f Fq(is)i(part)e(of)h(a)g Fk(struct)f Fq(t)o(yp)q(e,)h(there)g (should)h(b)q(e)g(no)f(padding)189 572 y(b)q(et)o(w)o(een)j(the)g (\015oat)f(at)h(the)g(end)h(of)e(one)h(replication,)i(and)e(the)g(c)o (har)g(at)f(the)h(b)q(eginning)i(of)e(the)189 628 y(next)g (replication.)24 b(This)16 b(is)h(neccessary)f(to)g(meet)g(the)g(full)h Fj(MPI)e Fq(t)o(yp)q(e)h(matc)o(hing)g(rules,)h(whic)o(h)189 685 y(allo)o(w)d(an)o(y)h(t)o(w)o(o)e(t)o(yp)q(es)i(to)f(matc)o(h)g(so) g(long)h(as)f(their)h(basic)g(elemen)o(ts)h(matc)o(h.)j(This)c(will)h (reduce)189 741 y(the)d(o)q(ccasions)g(in)h(whic)o(h)f(the)g(implemen)o (tation)h(can)f(optimize)h(the)f(mo)o(v)o(emen)o(t)f(of)h(struct)f(t)o (yp)q(es)189 798 y(con)o(taining)k(holes.)k(\()p Fi(End)c(of)g(advic)n (e)g(to)h(implementors.)p Fq(\))166 945 y Fd(Discussion:)52 b Ff(Supp)q(ose)20 b(w)o(e)e(concatenated)i(10)e(copies)h(of)f(a)h (datat)o(yp)q(e.)32 b(The)19 b(sp)q(eci\014cation)g(in)f(this)75 994 y(section)g(sa)o(ys)f(that)h(the)f(9)g(holes)h(b)q(et)o(w)o(een)g (the)g(copies)g(and)f(the)h(10th)f(hole)g(after)g(the)h(last)f(cop)o(y) g(can)g(all)f(b)q(e)75 1044 y(o)o(v)o(erwritten)e(\(so)f(long)f(as)h (the)h(user)g(has)f(explicitly)f(set)i(the)g(exten)o(t)g(of)e(the)i (struct)g(t)o(yp)q(e)g(using)f Fc(MPI)p 1685 1044 13 2 v 14 w(UB)p Ff(\).)f(This)75 1094 y(migh)o(t)h(not)i(b)q(e)g(safe)g (if)f(the)i(space)g(allo)q(cated)e(to)h(an)f(arra)o(y)h(is)g(not)f(a)h (m)o(ultiple)e(of)h(the)h(\\exten)o(t")h(of)e(eac)o(h)h(arra)o(y)75 1144 y(comp)q(onen)o(t.)i(Do)q(es)e(an)o(y)o(one)e(kno)o(w)h(of)f(a)g (compiler)g(that)h(do)q(es)g(not)g(satisfy)g(this)g(rule?)166 1200 y(T)m(o)d(increase)j(the)f(n)o(um)o(b)q(er)f(of)f(o)q(ccasions)i (for)f(whic)o(h)g(the)h Fc(MPI)f Ff(implemen)o(tati)o(on)e(can)i (optimize)f(the)i(mo)o(v)o(e-)75 1257 y(men)o(t)e(of)h(struct)h(t)o(yp) q(es)g(con)o(taining)f(holes)g(\(without)g(relying)g(on)f(the)i(user)h (ordering)e(\014elds)g(in)g(decreasing)i(size\),)75 1313 y(w)o(e)k(migh)o(t)f(add)h(a)g(further)h(restriction)g(to)f(the)h Fc(MPI)e Ff(t)o(yp)q(e)i(matc)o(hing)d(rules.)32 b(This)18 b(restriction)h(w)o(ould)f(only)75 1370 y(allo)o(w)11 b(t)o(yp)q(es)j(created)g(b)o(y)f Fc(MPI)p 568 1370 V 14 w(TYPE)p 690 1370 V 14 w(CONTIGUOUS)p 971 1370 V 13 w(STRUCT)f Ff(to)g(matc)o(h)g(iden)o(tical)g(t)o(yp)q(es)i(also)e (created)i(b)o(y)75 1426 y Fc(MPI)p 152 1426 V 14 w(TYPE)p 274 1426 V 14 w(CONTIGUOUS)p 555 1426 V 13 w(STRUCT)d Ff(\(or)g(with)g Fc(MPI)p 972 1426 V 14 w(P)m(A)o(CKED)p Ff(\).)d(This)j(restriction)h(w)o(ould)f(allo)o(w)e(the)j(imple-)75 1483 y(men)o(tation)f(to)i(add)g(the)g(same)f(padding)g(b)q(et)o(w)o (een)i(replications)f(of)f(the)i(t)o(yp)q(e)f(in)g(a)f(message)h(that)g (the)g(compiler)75 1539 y(uses)i(b)q(et)o(w)o(een)g(replications)f(of)f (the)i(struct)g(t)o(yp)q(e)g(in)e(an)h(arra)o(y)m(.)75 1708 y Fg(9.2.3)49 b(MPI)p 313 1708 15 2 v 18 w(T)l(yp)q(e)p 430 1708 V 18 w(Datat)o(yp)q(e)1875 1747 y Fm(>)16 b Fl(\(July\))75 1794 y Fq(It)f(is)h(prop)q(osed)g(that)e(this)i(b)q(e)g (mo)o(v)o(ed)e(to)h(1-sided.)1875 1806 y Fm(>)h Fl(\(July\))75 1957 y Fk(Here)23 b(is)h(a)g(draft;)f(it)g(has)h(a)f(number)g(of)h (questions/issues)e(for)h(discussion)g(in)g(it.)75 2013 y(The)g(main)h(problem)f(is)g(the)h(data-dependence)e(on)h(the)h(size)f (of)g(the)h(representation)e(of)75 2069 y(the)h(MPI_DATATYPE,)g(and)g (the)g(need)h(to)f(avoid)h(constraining)e(the)h(implementation)f(of)75 2126 y(MPI_PACK/UNPACK.)75 2239 y(Datatype)h(encoding)g(with)g (MPI_Pack/MPI_Unpack)75 2352 y(Rationale)75 2408 y(An)h(MPI)f(datatype) g(is)g(an)h(object)f(with)g(a)h(well-defined)e(representation)g(\(the)i (type)f(map\))75 2465 y(which)g(is)h(representable)e(on)h(all)h (platforms.)46 b(Certain)23 b(applications)f(need)i(to)f(\(a\))75 2521 y(send)g(a)h(datatype)f(between)g(processors)f(and)i(\(b\))f (examine)g(and)g(create)h(MPI)f(datatypes)75 2578 y(\(e.g.,)g(I/O)g (operations)g(or)h(user-defined)e(operations)g(\(MPI_Op\))h(on)h (complex)75 2634 y(datatypes\).)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fq(6)1161 b Fh(CHAPTER)15 b(9.)30 b(MISCELLANY)75 45 y Fk(MPI)23 b(already)g(has)h(routines)f(to)g(convert)g(to)h(and)f (from)g(an)h(opaque)f(external)75 102 y(representation;)f(these)h(are)g (MPI_Pack)g(and)h(MPI_Unpack.)46 b(These)23 b(can)h(be)f(used)g(to)75 158 y(convert)g(an)g(MPI_Datatype)g(into)g(a)h(form)f(that)g(can)h(be)f (sent)h(between)f(processors)f(\(case)75 214 y(\(a\))h(above\).)75 327 y(Propose:)75 384 y(A)h(new)f(datatype,)g(MPI_DATATYPE,)f(which)h (may)h(be)f(used)g(in)h(MPI_PACK/MPI_UNPACK.)75 497 y(Issues:)75 553 y(Unlike)f(other)g(MPI)h(datatypes,)e(the)i(amount)f(of)g(space)g (needed)g(to)h(pack)f(an)h(MPI_DATATYPE)75 610 y(depends)f(on)g(the)h (data.)47 b(To)24 b(determine)e(the)i(amount)f(of)g(space,)g(we)h(add) 75 723 y(MPI_PACK_SIZE_DATA\()d(void)j(*,)f(MPI_Datatype,)f(MPI_Comm,)h (int)g(*)h(\))75 835 y(where)f(the)h(first)f(argument)g(points)g(to)g (the)h(data)f(and)g(the)h(last)f(is)h(the)f(length)75 892 y(required.)46 b(Unlike)24 b(MPI_PACK_SIZE,)d(there)j(is)f(no)h (count)f(argument;)75 948 y(MPI_PACK_SIZE_DATA)e(gives)j(you)f(only)g (the)h(size)f(to)h(pack)f(the)g(one)h(instance)f(of)g(the)75 1005 y(datatype.)75 1118 y(Example)147 1231 y(MPI_PACK_SIZE_DATA\()e (MPI_INT,)i(MPI_DATATYPE,)f(MPI_COMM_WORLD,)g(&len)h(\))75 1344 y(In)h(order)f(to)g(expose)g(the)h(representation)e(used)h(by)h (MPI_PACK,)e(we)i(must)f(choose)g(both)75 1400 y(a)h(representation)e (form)h(and)g(how)h(to)f(access)h(it.)47 b(The)23 b(form)h(in)f(the)h (previous)e(drafts)75 1456 y(is)i(fine.)75 1569 y(Binding)f(issue:)75 1626 y(Nothing)g(in)g(MPI)h(requires)f(an)g(MPI_Datatype)g(to)g(be)h (either)f(a)g(pointer)g(or)h(an)g(integral)75 1682 y(data)f(item;)g (thus)h(using)f("void)g(*")h(may)f(be)h(inappropriate.)46 b(Perhaps)23 b(an)g(MPI_Handle_t)75 1739 y(basic)g(type)g(should)h(be)f (defined)g(that)g(all)h(MPI)f(opaque)g(objects)g(can)h(be)f(cast)h(to.) 75 1852 y(Issue)75 1965 y(MPI_PACK)f(does)g(not)h(say)f(anything)g (about)g(how)g(the)h(data)f(is)h(represented.)46 b(For)75 2021 y(example,)23 b(an)g(implementation)f(may)i(put)f(a)h(header)f(on) h(the)f(data)g(\("I)h(am)f(packed)g(with)75 2077 y(XDR"\))g(or)h (\("Characters)e(are)h(in)h(Unicode"\).)46 b(An)24 b(implementation)e (may)h(include)75 2134 y(additional)f(information)h(within)g(the)g (data)h(\(for)f(example,)g(the)g(type-map)g(may)g(be)75 2190 y(included)g(to)g(give)h(better)f(error)g(detection\).)46 b(To)24 b(use)f(MPI_PACK)g(to)h(create)f(a)75 2247 y(canonical)g (representation,)e(we)j(need)f(to)170 2360 y(\(a\))h(restrict)f(the)g (formats)g(used)g(by)h(MPI_PACK)f(\(for)g(example,)g(expect)266 2416 y(a)h(header)f(of)g(fixed)g(MPI_PACK_HEADER_SIZE)f(length\))170 2473 y(\(b\))i(specify)f(a)h(canonical)e(output)h(format)g(as)h(part)f (of)h(the)f(communicator)266 2529 y(\(either)g(by)g(a)h(user-setable)e (attribute)h(or)h(special)e(communicator,)266 2586 y(including)h(a)g (null)h(communicator\))170 2642 y(\(c\))g(specify)f(special)g(behavior) f(when)i(packing)f(MPI_DATATYPE)f(\(e.g.,)h(must)g(use)266 2698 y(canonical)g(representation)e(with)j(no)f(header\))1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fh(9.2.)34 b(D)o(A)l(T)l(A)l(TYPE)15 b(IMPR)o(O)o(VEMENTS)1019 b Fq(7)170 45 y Fk(\(d\))24 b(use)f(MPI_PACK_DATATYPE)f(\(or)h(MPI_PACK_CANONICAL\))f(instead)75 158 y(On)i(the)f(MPI_UNPACK)g(side,)g(we)g(need)75 271 y(MPI_GET_COUNT_DATA\()e(void)j(*,)f(MPI_Status)g(*,)g(MPI_Datatype,)f (int)i(*)g(\))75 384 y(\(in)f(case)h(you)f(sent)h(3)f(MPI_DATATYPES,)f (all)i(with)f(different)g(sizes)g(of)75 440 y(canonical)g (representations\))75 553 y(This)g(is)h(similar)f(to)g (MPI_PACK_SIZE_DATA)f(in)h(that)h(the)f(result)g(depends)g(on)h(the)75 610 y(data)f(that)h(was)f(sent.)75 772 y(Possible)g(restrictions)f (discussed)h(at)g(the)h(April)f(meeting)g(are:)123 885 y(0\))g(none)123 942 y(1\))g(use)h(only)f(in)h(PACK/UNPACK)123 998 y(2\))f(\(1\))h(+)f(only)h(1)g(datatype)e(per)i(message)123 1055 y(3\))f(\(2\))h(+)f(standard)g(packing)75 1176 y Fg(9.2.4)49 b(Convenient)17 b(F)o(o)o(rm)d(of)j(MPI)p 731 1176 15 2 v 18 w(T)l(yp)q(e)p 848 1176 V 18 w(indexed)p 1018 1176 V 18 w(blo)q(ck)75 1262 y Fq(This)e(prop)q(osal)g(is)g(to)f (allo)o(w)h(constan)o(t)f(blo)q(c)o(ksize)i(and)f(arbitrary)f (displacemen)o(ts)i(in)f(the)g(in)o(terface)g(to)75 1319 y Fj(MPI)p 160 1319 14 2 v 16 w(TYPE)p 293 1319 V 17 w(INDEXED)p Fq(.)9 b(The)i(routine)f(no)o(w)g(tak)o(es)f Fj(a)o(rra)o(y)p 1063 1319 V 15 w(of)p 1115 1319 V 16 w(blo)q(cklengths)j Fq(and)f Fj(a)o(rra)o(y)p 1555 1319 V 14 w(of)p 1606 1319 V 16 w(displacements)75 1375 y Fq(as)17 b(argumen)o(ts.)27 b(There)18 b(are)f(man)o(y)h(co)q(des)g (using)g(indirect)i(addressing)e(arising)g(from)f(unstructured)75 1432 y(grids)f(where)h(the)f(blo)q(c)o(ksize)i(is)e(alw)o(a)o(ys)f(1)h (\(gather/scatter\).)k(W)l(e)c(prop)q(ose)g(this)h(in)o(terface,)f (allo)o(wing)75 1488 y(for)f(constan)o(t)f(blo)q(c)o(ksize)j(and)e (arbitrary)g(displacemen)o(ts.)788 b Fm(>)16 b Fl(\(July\))75 1639 y Fj(MPI)p 160 1639 V 16 w(TYPE)p 293 1639 V 17 w(INDEXED)p 505 1639 V 16 w(BLOCK\()i(count,)j(blo)q(cklength,)g(a)o (rra)o(y)p 1190 1639 V 15 w(of)p 1242 1639 V 16 w(displacements,)g (oldt)o(yp)q(e,)f(newt)o(yp)q(e)75 1695 y(\))117 1773 y Ff(IN)155 b Fj(count)482 b Ff(length)14 b(of)f(arra)o(y)h(of)f (displacemen)o(ts)117 1848 y(IN)155 b Fj(blo)q(cklength)371 b Ff(size)15 b(of)e(blo)q(c)o(k)117 1923 y(IN)155 b Fj(a)o(rra)o(y)p 416 1923 V 15 w(of)p 468 1923 V 16 w(displacements)164 b Ff(arra)o(y)14 b(of)f(displacemen)o(ts)117 1998 y(IN)155 b Fj(oldt)o(yp)q(e)450 b Ff(old)13 b(datat)o(yp)q(e)117 2073 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Ff(new)15 b(datat)o(yp)q(e)75 2198 y Fk(int)23 b(MPI)p 245 2198 15 2 v 17 w(Type)p 358 2198 V 17 w(Indexed)p 543 2198 V 16 w(Block\()g(int)h(count,)f(int) g(blocklength,)393 2254 y(int*)g(array)p 635 2254 V 17 w(of)p 700 2254 V 17 w(displacements,)f(MPI)p 1147 2254 V 17 w(Datatype)g(oldtype,)393 2311 y(MPI)p 468 2311 V 17 w(Datatype*)h(newtype)f(\))75 2397 y(MPI)p 150 2397 V 17 w(TYPE)p 263 2397 V 16 w(INDEXED)p 447 2397 V 17 w(BLOCK\(COUNT,)g(BLOCKLENGTH,)g(ARRAY)p 1204 2397 V 17 w(OF)p 1269 2397 V 16 w(DISPLACEMENTS,)g(OLDTYPE,)393 2453 y(NEWTYPE,)h(IERROR\))170 2510 y(INTEGER)g(COUNT,)g(INTEGER)g (BLOCKLENGTH,)g(ARRAY)p 1153 2510 V 16 w(OF)p 1217 2510 V 17 w(DISPLACEMENT\(*\),)f(OLDTYPE,)170 2566 y(NEWTYPE,)h(IERROR)1875 2603 y Fm(?)16 b Fl(\(July\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fq(8)1156 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fn(9.3)59 b(Interlanguage)18 b(Interop)r(erabilit)n(y)75 148 y Fg(9.3.1)49 b(Intro)q(duction)75 234 y Fq(It)17 b(is)h(not)e(uncommon)h(for)g(library)h(dev)o(elop)q(ers)g(to)e(use)i (one)f(language)g(to)f(dev)o(elop)i(an)f(applications)75 290 y(library)g(to)f(b)q(e)h(used)g(b)o(y)g(applications)h(dev)o(elop)q (ers)g(using)f(another)f(language.)24 b(MPI)16 b(curren)o(tly)h(sup-)75 347 y(p)q(orts)d(b)q(oth)i(an)e(ANSI)i(C)f(binding)i(and)e(a)f(F)l (ortran)g(77)g(binding.)22 b(It)15 b(should)h(b)q(e)f(p)q(ossible)i (for)d(F)l(ortran)75 403 y(77)h(to)f(call)i(C)f(and)h(vice)g(v)o(ersa)f (that)f(use)i(MPI)f(in)h(b)q(oth)f(languages.)166 460 y(MPI)h(allo)o(ws)h(the)f(dev)o(elopmen)o(t)h(of)f(clien)o(t-serv)o(er) i(co)q(de,)f(with)g(MPI)f(comm)o(unication)h(used)g(b)q(e-)75 516 y(t)o(w)o(een)d(a)h(parallel)h(clien)o(t)g(and)f(a)g(parallel)h (serv)o(er.)k(It)14 b(should)i(b)q(e)g(p)q(ossible)g(to)e(co)q(de)i (the)f(serv)o(er)f(in)i(one)75 573 y(language,)e(and)g(the)f(clien)o (ts)i(in)g(another)e(language.)19 b(T)l(o)14 b(do)f(so,)g(it)h(should)h (b)q(e)f(p)q(ossible)i(for)d(F)l(ortran77)75 629 y(pro)q(cesses)j(to)e (comm)o(unicate)i(with)f(C)g(pro)q(cesses.)166 686 y(There)g(are)g(sev) o(eral)h(issues)g(that)e(need)i(to)f(b)q(e)h(addressed,)f(to)f(ac)o (hiev)o(e)i(in)o(terop)q(erabilit)o(y)l(.)75 778 y Fe(Initializat)q (ion)26 b Fq(W)l(e)13 b(need)h(to)f(sp)q(ecify)h(ho)o(w)f(is)g(the)g (MPI)h(en)o(vironmen)o(t)f(initialized)j(for)d(b)q(oth)g(F)l(ortran)189 834 y(and)i(C.)75 927 y Fe(Comm)o(unication)j(of)g(MPI)e(opaque)i(ob)s (jects)23 b Fq(W)l(e)13 b(need)h(to)f(sp)q(ecify)i(ho)o(w)e(are)g(MPI)g (ob)s(ject)g(han-)189 984 y(dles)i(passed)g(from)e(F)l(ortran)h(to)f (C,)h(and)h(vice)g(v)o(ersa.)k(W)l(e)c(also)f(need)h(to)f(sp)q(ecify)i (what)e(happ)q(ens)189 1040 y(when)g(an)g(MPI)g(ob)s(ject)g(is)g (accessed)h(in)g(one)f(language,)g(to)g(retriev)o(e)g(information)g (set)g(with)h(this)189 1097 y(ob)s(ject)f(in)i(another)f(language.)75 1190 y Fe(In)o(terlanguage)j(comm)o(unication)24 b Fq(W)l(e)18 b(need)g(to)e(sp)q(ecify)i(ho)o(w)f(messages)g(sen)o(t)g(in)g(one)h (language)189 1246 y(can)d(b)q(e)h(receiv)o(ed)g(in)g(another)f (language.)166 1339 y(It)f(is)h(highly)h(desirable)g(that)d(the)i (solution)g(for)f(in)o(terlanguage)h(in)o(terop)q(erabilit)o(y)h(b)q(e) f(extendable)75 1395 y(to)g(new)g(languages,)g(should)h(MPI)f(bindings) i(b)q(e)f(de\014ned)h(for)d(suc)o(h)i(languages.)75 1516 y Fg(9.3.2)49 b(Assumptions)75 1602 y Fq(W)l(e)21 b(assume)f(that)g (con)o(v)o(en)o(tions)g(exist)h(for)f(C)h(programs)e(to)h(call)i(F)l (ortran)d(subroutines,)j(and)f(vice)75 1659 y(v)o(ersa.)e(These)c(con)o (v)o(en)o(tions)f(sp)q(ecify)i(ho)o(w)e(to)g(link)h(C)g(and)f(F)l (ortran)f(co)q(de;)i(ho)o(w)f(to)g(call)h(from)f(F)l(ortran)75 1715 y(a)j(C)g(function,)h(so)f(that)g(the)g(correct)g(C)g(calling)i (in)o(terface)f(b)q(e)g(generated,)g(and)f(vice)h(v)o(ersa;)g(ho)o(w)f (to)75 1772 y(pass)i(from)g(F)l(ortran)g(parameters)g(b)o(y)g(v)m(alue) i(to)e(C)h(functions;)i(and)e(the)g(corresp)q(ondence)h(b)q(et)o(w)o (een)75 1828 y(F)l(ortran)g(basic)h(data)f(t)o(yp)q(es)h(and)g(C)g (basic)g(datat)o(yp)q(es.)39 b(In)22 b(general,)i(these)e(con)o(v)o(en) o(tions)g(will)h(b)q(e)75 1885 y(implemen)o(tation)16 b(dep)q(enden)o(t.)22 b(\(Ho)o(w)o(ev)o(er,)13 b(there)i(is)g(an)g (ongoing)g(e\013ort)f(to)g(standardize)i(a)e(C)h(calling)75 1941 y(in)o(terface)k(in)h(High)f(P)o(erformance)g(F)l(ortran.\))29 b(F)l(urthermore,)19 b(not)f(ev)o(ery)h(F)l(ortran)f(t)o(yp)q(e)h(need) g(ha)o(v)o(e)75 1997 y(a)g(matc)o(hing)g(C)f(t)o(yp)q(e,)i(and)f(vice)h (v)o(ersa:)26 b(often)o(times,)20 b(for)e(example,)i(C)f(c)o(haracter)f (strings)h(are)f(not)75 2054 y(compatible)k(with)g(F)l(ortran)e(CHARA)o (CTER)i(v)m(ariables.)39 b(Ho)o(w)o(ev)o(er,)21 b(w)o(e)g(assume)h (that)e(a)h(F)l(ortran)75 2110 y Fc(INTEGER)c Fq(can)h(b)q(e)g(passed)g (to)f(a)g(C)h(program.)26 b(Also,)18 b(w)o(e)f(assume)h(that)f(b)q(oth) h(F)l(ortran)e(and)i(C)f(ha)o(v)o(e)75 2167 y(address)e(sized)i(in)o (tegers.)75 2288 y Fg(9.3.3)49 b(Initialization)75 2374 y Fq(A)18 b(call)i(to)e Fk(MPI)p 348 2374 15 2 v 16 w(INIT\(\))p Fq(,)g(either)h(from)f(C)g(or)g(F)l(ortran,)f(initialize)q(s)k(MPI)d (for)g(execution,)i(b)q(oth)e(in)h(C)75 2431 y(and)c(F)l(ortran.)166 2534 y Fd(Discussion:)36 b Ff(Certain)14 b(implemen)o(tations)e(use)j (the)g(\(inout\))f(argc,)g(argv)g(argumen)o(ts)g(of)g(the)h(C)f(v)o (ersion)75 2591 y(of)i Fc(MPI)p 202 2591 13 2 v 14 w(INIT)g Ff(in)f(order)i(to)f(propagate)g(the)h(righ)o(t)f(v)n(alue)f(for)h (argc,)g(argv)g(to)g(all)f(executing)i(pro)q(cesses.)27 b(Users)75 2647 y(should)13 b(b)q(e)i(w)o(arned)f(that)g(use)g(of)f (the)h(F)m(ortran)g(v)o(ersion)g(of)f Fc(MPI)p 1102 2647 V 14 w(INIT)h Ff(to)f(initialize)f(MPI)i(ma)o(y)e(result)i(in)g(a)f (loss)75 2704 y(of)j(this)g(abilit)o(y)m(.)22 b(This)16 b(should)g(b)q(e)h(acceptable:)24 b(if)15 b(C)h(execution)h(is)f(not)g (started)h(from)d(main,)g(but)j(is)f(started)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fh(9.3.)29 b(INTERLANGUA)o(GE)16 b(INTER)o (OPERABILITY)778 b Fq(9)75 45 y Ff(from)13 b(the)j(in)o(v)o(o)q(cation) e(of)h(a)g(\(library\))f(function)h(call,)g(then)g(one)h(do)q(es)g(not) f(necessarily)h(exp)q(ect)h(argc,)e(argv)g(to)75 102 y(b)q(e)g(de\014ned.)1875 149 y Fm(>)h Fl(\(July\))189 255 y Fi(A)n(dvic)n(e)i(to)i(implementors.)62 b Fq(It)19 b(should)g(b)q(e)h(p)q(ossible)h(for)d(an)g(arbitrary)h(program,)f(no)h (mat-)189 311 y(ter)h(ho)o(w)h(it)g(has)g(b)q(een)i(started,)e(to)f (call)j Fj(MPI)p 1028 311 14 2 v 15 w(INIT)e Fq(and)g(b)q(ecome)h(an)f Fj(MPI)g Fq(program,)g(with)189 368 y Fj(MPI)p 274 368 V 15 w(COMM)p 431 368 V 17 w(W)o(ORLD)d Fq(then)g(set)f(to)g(ha)o(v)o (e)h(size)g(1.)28 b(The)18 b(idea)g(is)g(that)g(it)g(migh)o(t)f(then)h (spa)o(wn)189 424 y(further)d Fj(MPI)f Fq(pro)q(cesses.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))1875 485 y Fm(?)g Fl(\(July\))166 530 y Fq(The)21 b(function)g Fk(MPI)p 523 530 15 2 v 17 w(INITIALIZED)e Fq(returns)h(the)h(same)f(answ)o(er,)h(whether)f(called)i(from)e(C)g (or)75 587 y(F)l(ortran.)166 643 y(The)15 b(function)h Fk(MPI)p 512 643 V 17 w(FINALIZE)e Fq(\014nalizes)j(b)q(oth)e(C)g(and)h (F)l(ortran)e(MPI)h(en)o(vironmen)o(t.)166 700 y(The)i(function)g Fk(MPI)p 515 700 V 17 w(ABORT)e Fq(kill)k(pro)q(cesses,)e(irresp)q (ectiv)o(e)h(of)e(the)g(language)h(used)g(b)o(y)f(the)h(caller)75 756 y(or)e(b)o(y)g(the)g(pro)q(cesses)h(killed.)166 813 y(When)22 b Fj(MPI)p 390 813 14 2 v 16 w(INIT)g Fq(is)g(called,)k (either)c(from)g(F)l(ortran)f(or)h(C,)f(then)i(the)f(MPI)g(en)o (vironmen)o(t)g(is)75 869 y(initialized)17 b(in)e(the)f(same)f(manner,) h(b)q(oth)g(for)f(F)l(ortran)g(and)h(C.)g(I.e.,)f Fc(MPI)p 1350 869 13 2 v 14 w(COMM)p 1494 869 V 15 w(W)o(ORLD)h Fq(carries)g(the)75 925 y(same)i(information,)g(b)q(oth)g(in)h(F)l (ortran)e(and)i(C:)e(same)h(pro)q(cesses,)h(same)f(en)o(vironmen)o(tal) h(attributes,)75 982 y(same)e(error)f(handlers.)75 1104 y Fg(9.3.4)49 b(T)l(ransfer)15 b(of)i(handles)75 1189 y Fq(The)g(data)g(t)o(yp)q(e)g Fc(MPI)p 457 1189 V 14 w(Fint)g Fq(is)h(pro)o(vided)f(in)h(C)f(for)g(an)g(in)o(teger)g(of)f (the)i(size)f(that)g(matc)o(hes)f(a)h(F)l(ortran)75 1246 y Fc(INTEGER)p Fq(;)d(usually)l(,)i Fc(MPI)p 517 1246 V 15 w(Fint)f Fq(will)i(b)q(e)f(equiv)m(alen)o(t)h(to)d Fc(int)p Fq(.)166 1302 y(The)f(follo)o(wing)h(t)o(w)o(o)e(functions)i (are)f(pro)o(vided)h(in)g(C)f(to)g(con)o(v)o(ert)f(from)h(a)g(F)l (ortran)f(handle)i(\(whic)o(h)75 1359 y(is)i(an)f(in)o(teger\))g(to)f (a)h(C)g(handle,)h(and)g(vice)g(v)o(ersa.)75 1415 y Fk(void)23 b(*MPI)p 293 1415 15 2 v 17 w(Int2handle\()f(MPI)p 668 1415 V 17 w(Fint)h(*f)p 852 1415 V 17 w(handle\))75 1502 y(MPI)p 150 1502 V 17 w(Fint)g(MPI)p 358 1502 V 17 w(Handle2int\()f (void)i(*c)p 829 1502 V 16 w(handle\))166 1588 y Fq(If)e Fj(f)p 235 1588 14 2 v 17 w(handle)h Fq(is)g(a)f(v)m(alid)i(F)l(ortran) d(handle)j(to)d(an)h(opaque)h(ob)s(ject,)g(then)f Fj(MPI)p 1584 1588 V 16 w(INT2HANDLE)75 1645 y Fq(returns)e(a)g(v)m(alid)i(C)e (handle)i(to)e(that)f(same)h(ob)s(ject;)i(if)f Fj(f)p 1105 1645 V 17 w(handle)g Fq(is)g(a)f(n)o(ull)i(F)l(ortran)d(handle,)k (then)75 1701 y Fj(MPI)p 160 1701 V 16 w(INT2HANDLE)18 b Fq(returns)g(a)f(n)o(ull)j(C)e(handle;)i(if)f Fj(f)p 1038 1701 V 16 w(handle)g Fq(is)g(an)f(in)o(v)m(alid)i(F)l(ortran)d (handle,)j(then)75 1758 y Fj(MPI)p 160 1758 V 16 w(INT2HANDLE)15 b Fq(returns)g(an)g(in)o(v)m(alid)i(C)e(handle;)h(and)g(similarly)l(,)h (for)d Fj(MPI)p 1477 1758 V 16 w(HANDLE2INT)p Fq(.)75 1864 y Fe(Example)k(9.2)k Fq(The)11 b(example)g(b)q(elo)o(w)f (illustrates)i(ho)o(w)d(the)i(F)l(ortran)d(MPI)j(function)g Fj(MPI)p 1651 1864 V 15 w(TYPE)p 1783 1864 V 17 w(COMMIT)75 1920 y Fq(can)k(b)q(e)h(implemen)o(ted)h(b)o(y)d(wrapping)i(the)f(C)g (MPI)g(function)h Fj(MPI)p 1244 1920 V 15 w(T)l(yp)q(e)p 1351 1920 V 18 w(commit)d Fq(with)i(a)g(C)g(wrapp)q(er)75 1977 y(to)g(do)g(handle)h(con)o(v)o(ersions.)75 2083 y Fk(!)24 b(FORTRAN)f(PROCEDURE)75 2139 y(SUBROUTINE)f (MPI_TYPE_COMMIT\()g(DATATYPE,)h(IERR\))75 2195 y(INTEGER)g(DATATYPE,)g (IERR)75 2252 y(CALL)g(MPI_X_TYPE_COMMIT\(DATATYPE,)d(IERR\))75 2308 y(RETURN)75 2365 y(END)75 2478 y(!)k(C)f(wrapper)75 2591 y(void)g(MPI_X_TYPE_COMMIT\()f(MPI_Fint)g(*f_handle,)h(*ierr\))75 2647 y({)75 2704 y(MPI_Datatype)f(datatype;)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fq(10)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)75 102 y Fk(datatype)23 b(=)g (\(MPI_Datatype\)MPI_int2handle\()d(*f_handle\);)75 158 y(*ierr)j(=)h(\(MPI_Fint\)MPI_Type_commit)o(\()d(&datatype\);)75 214 y(*f_handle)i(=)g(MPI_handle2int\()f(datatype\);)75 271 y(return\(\);)75 327 y(})166 432 y Fq(The)10 b(same)g(approac)o(h)g (can)g(b)q(e)h(used)g(for)e(all)i(other)f(MPI)g(functions.)19 b(The)11 b(call)g(to)e Fj(MPI)p 1637 432 14 2 v 16 w(INT2HANDLE)75 489 y Fq(\(resp.)k Fj(MPI)p 285 489 V 16 w(HANDLE2INT)p Fq(\))g(can)i(b)q(e)f(omitted)g(when)g(the)g(handle)i(is)e(an)g Ff(OUT)g Fq(\(resp.)f Ff(IN)p Fq(\))h(argumen)o(t,)75 545 y(rather)h(than)g Ff(INOUT)p Fq(.)166 697 y Fd(Discussion:)46 b Ff(The)17 b(design)g(assumes)g(that)g(C)g(handles)g(can)g(b)q(e)h (con)o(v)o(erted)g(to)e(t)o(yp)q(e)i Fc(void*)p Ff(,)f(and)g(vice)75 753 y(v)o(ersa,)e(without)g(loss.)22 b(This)15 b(is)g(a)g(sligh)o(t)f (restriction)i(on)f(MPI1)g(whic)o(h)g(only)f(requires)i(that)g(handles) f(b)q(e)h(of)e(an)75 810 y(in)o(teger)g(or)g(p)q(oin)o(ter)g(t)o(yp)q (e.)189 962 y Fi(R)n(ationale.)62 b Fq(The)19 b(design)h(here)g(pro)o (vides)f(a)g(con)o(v)o(enien)o(t)g(solution)h(for)e(the)h(prev)m(alen)o (t)h(case,)189 1018 y(where)f(a)f(C)h(wrapp)q(er)g(is)h(used)f(to)f (allo)o(w)i(F)l(ortran)d(co)q(de)j(to)e(call)i(a)f(C)f(library)l(,)j (or)d(C)h(co)q(de)h(to)189 1075 y(call)g(a)f(F)l(ortran)f(library)l(.) 32 b(The)19 b(use)h(of)f(C)f(wrapp)q(ers)i(is)f(m)o(uc)o(h)g(more)g (lik)o(ely)i(than)e(the)g(use)g(of)189 1131 y(F)l(ortran)12 b(wrapp)q(ers,)i(b)q(ecause)h(it)f(is)g(m)o(uc)o(h)g(more)f(lik)o(ely)j (that)d(a)g(v)m(ariable)j(of)d(t)o(yp)q(e)h Fc(INTEGER)f Fq(can)189 1188 y(b)q(e)j(passed)f(to)g(C,)f(than)h(a)g(C)g(handle)h (can)g(b)q(e)g(passed)f(to)g(F)l(ortran.)189 1263 y(Assuming)e(that)g (the)g(F)l(ortran)f(handle)i(is)g(used)f(as)g(the)g(language)g(indep)q (enden)o(t)j(represen)o(tation)189 1319 y(for)21 b(handles,)j(then)e (this)g(design)g(can)g(b)q(e)g(extended)h(to)e(allo)o(w)h(wrapp)q(ers)f (written)h(in)g(other)189 1375 y(languages)15 b(b)o(y)g(pro)o(viding)h (con)o(v)o(ersion)g(functions)g(in)g(these)f(languages.)189 1450 y(The)21 b(use)h(of)f(con)o(v)o(ersion)g(functions,)j(rather)c (than)i(pro)q(cedures,)h(allo)o(w)f(us)f(to)g(use)g(inlinin)q(g,)189 1507 y(esp)q(ecially)c(in)f(the)f(case)g(where)g(these)g(functions)h (are)e(the)h(iden)o(tit)o(y)l(.)21 b(The)15 b(con)o(v)o(ersion)g (function)189 1563 y(in)f(the)g(wrapp)q(er)g(do)q(es)g(not)f(catc)o(h)h (an)f(in)o(v)m(alid)j(handle)f(argumen)o(t.)k(Instead,)14 b(an)g(in)o(v)m(alid)i(handle)189 1620 y(is)11 b(passed)h(b)q(elo)o(w)g (to)f(the)g(library)h(function,)h(whic)o(h,)f(presumably)l(,)h(c)o(hec) o(ks)e(its)h(input)g(argumen)o(ts.)189 1676 y(\()p Fi(End)j(of)i(r)n (ationale.)p Fq(\))189 1781 y Fi(A)n(dvic)n(e)c(to)h(users.)38 b Fq(The)13 b(user)g(needs)g(to)g(co)q(erce)g(the)g(result)g(returned)g (b)o(y)f Fj(MPI)p 1560 1781 V 16 w(int2handle)j Fq(from)189 1837 y Fc(void*)g Fq(to)f(a)h(suitable)i(handle)f(t)o(yp)q(e.)k(\()p Fi(End)c(of)g(advic)n(e)g(to)h(users.)p Fq(\))75 1959 y Fg(9.3.5)49 b(MPI)17 b(Opaque)f(objects)75 2045 y Fq(In)11 b(general,)g(opaque)g(ob)s(jects)e(are)h(\\the)g(same")g(in)h(F)l (ortran)e(and)h(C:)g(they)g(carry)g(the)g(same)g(information,)75 2101 y(and)17 b(ha)o(v)o(e)f(the)h(same)f(meaning)h(in)h(b)q(oth)f (languages.)24 b(The)17 b(mec)o(hanism)g(describ)q(ed)i(in)e(the)g (previous)75 2158 y(section)c(can)g(b)q(e)g(used)h(to)e(pass)g (references)i(to)e(MPI)g(ob)s(jects)g(from)g(language)h(to)f(language.) 20 b(An)13 b(ob)s(ject)75 2214 y(created)i(in)h(one)g(language)f(can)g (b)q(e)h(accessed,)g(mo)q(di\014ed)g(or)f(freed)g(in)h(another)f (language.)166 2270 y(W)l(e)g(examine)h(b)q(elo)o(w)g(in)g(more)f (detail,)h(issues)g(that)e(arise)i(for)e(eac)o(h)h(t)o(yp)q(e)h(of)f (MPI)g(ob)s(ject.)75 2392 y Fg(9.3.6)49 b(Datat)o(yp)q(es)75 2478 y Fq(Datat)o(yp)q(es)19 b(enco)q(des)h(the)g(same)g(information)f (in)i(F)l(ortran)e(and)h(in)g(C.)f(E.g.,)h(a)f(datat)o(yp)q(e)g (accessor)75 2534 y(lik)o(e)i Fj(MPI)p 248 2534 V 15 w(TYPE)p 380 2534 V 17 w(EXTENT)f Fq(will)i(return)d(the)h(same)f (information,)h(b)q(oth)g(in)g(F)l(ortran)f(and)g(in)i(C.)e(If)75 2591 y(a)g(datat)o(yp)q(e)f(de\014ned)j(in)f(F)l(ortran)e(is)h(used)h (in)g(a)f(C)f(comm)o(unication)i(call,)h(then)e(the)h(message)e(sen)o (t)75 2647 y(will)e(b)q(e)f(iden)o(tical)h(to)e(the)g(message)g(that)g (w)o(ould)h(b)q(e)g(sen)o(t)f(from)f(F)l(ortran:)19 b(the)14 b(same)g(comm)o(unication)75 2704 y(bu\013er)h(is)h(accessed,)f(and)h (the)f(same)g(represen)o(tation)g(con)o(v)o(ersion)g(is)h(p)q (erformed,)f(if)h(needed.)21 b(Both)15 b(C)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fh(9.3.)34 b(INTERLANGUA)o(GE)16 b(INTER)o(OPERABILITY)751 b Fq(11)75 45 y(and)16 b(F)l(ortran)e(basic)i (datat)o(yp)q(es)f(can)g(b)q(e)h(used)g(in)h(datat)o(yp)q(e)d (constructors,)h(in)h(either)g(language.)21 b(If)15 b(a)75 102 y(datat)o(yp)q(e)h(is)i(committed)e(in)i(either)g(language,)f(then) g(it)g(can)g(b)q(e)g(used)h(for)e(comm)o(unication)h(in)h(b)q(oth)75 158 y(languages.)166 214 y(The)c(function)g Fj(MPI)p 519 214 14 2 v 16 w(ADDRESS)g Fq(returns)f(the)h(same)f(v)m(alue,)i(b)q (oth)e(in)i(F)l(ortran)d(and)i(in)g(C.)f(On)h(the)75 271 y(other)h(hand,)h(w)o(e)g(do)g(not)f(require)i(that)e(the)h (constan)o(t)f Fc(MPI)p 1127 271 13 2 v 14 w(BOTTOM)g Fq(has)h(the)g(same)f(v)m(alue)i(in)g(b)q(oth)75 327 y(languages.)75 427 y Fe(Example)h(9.3)75 515 y Fk(!)24 b(FORTRAN)f(CODE)75 572 y(REAL)g(R\(5\))75 628 y(INTEGER)g(TYPE,)g (ADDR,)g(IERR)75 741 y(!)h(create)f(an)g(absolute)g(datatype)g(for)g (array)h(R)75 798 y(CALL)f(MPI_ADDRESS\()g(R,)g(ADDR,)g(IERR\))75 854 y(CALL)g(MPI_TYPE_STRUCT\(1,)f(5,)h(ADDR,)g(MPI_REAL,)g(TYPE,)g (IERR\))75 911 y(CALL)g(C_ROUTINE\(TYPE\))75 1024 y(/*)h(C)f(code)h(*/) 75 1136 y(void)f(C_ROUTINE\(MPI_Fint)f(*ftype\))75 1193 y({)75 1249 y(int)h(count)h(=)f(5;)75 1306 y(void)g(*handle;)75 1362 y(int)g(lens[2])g(=)h({1,1};)75 1419 y(MPI_Aint)f(displs[2];)75 1475 y(MPI_Datatype)f(types[2],)h(newtype;)75 1588 y(/*)h(create)f(an)g (absolute)g(datatype)g(for)g(buffer)g(that)h(consists)70 b(*/)75 1645 y(/*)47 b(of)24 b(count,)f(followed)g(by)g(R\(5\))668 b(*/)75 1757 y(MPI_Addr\(&count,)22 b(displs[0]\);)75 1814 y(displs[1])h(=)g(0;)75 1870 y(types[0])g(=)g(MPI_INT;)75 1927 y(types[1])g(=)g(\(MPI_Datatype\)MPI_Int2handle\(f)o(type\);)75 1983 y(MPI_Type_struct\(2,)e(lens,)j(displs,)e(types,)i(&newtype\);)75 2040 y(MPI_Type_commit\(&newtype\);)75 2153 y(MPI_Send\(MPI_BOTTOM,)d (1,)j(newtype,)e(1,)i(0,)g(MPI_COMM_WORLD\);)75 2209 y(/*)g(the)f(message)g(sent)g(contains)g(an)h(int)f(count)g(of)h(5,)f (followed)47 b(*/)75 2266 y(/*)24 b(by)f(the)h(5)f(REAL)h(entries)f(of) g(the)h(Fortran)f(array)g(R.)238 b(*/)75 2322 y(})189 2421 y Fi(R)n(ationale.)58 b Fq(The)18 b(curren)o(t)g(MPI)g(standard)g (sp)q(eci\014es)i(that)d Fc(MPI)p 1363 2421 V 15 w(ADDRESS)g Fq(can)h(b)q(e)h(used)g(in)189 2478 y(initialization)i(expressions)f (in)g(C,)e(but)h(not)f(in)i(F)l(ortran.)29 b(Since)21 b(F)l(ortran)c(do)q(es)i(not)g(supp)q(ort)189 2534 y(normally)11 b(call)h(b)q(e)g(v)m(alue,)g(then)g Fc(MPI)p 815 2534 V 14 w(ADDRESS)e Fq(m)o(ust)g(b)q(e)i(in)g(F)l(ortran)d(the)i(name)g (of)g(a)f(prede\014ned)189 2591 y(static)19 b(v)m(ariable,)k(e.g.,)d(a) f(v)m(ariable)j(in)f(an)f(MPI)g(declared)h Fc(COMMON)f Fq(blo)q(c)o(k.)35 b(On)20 b(the)g(other)189 2647 y(hand,)e(in)h(C,)e (it)h(is)g(natural)f(to)g(tak)o(e)g Fc(MPI)p 930 2647 V 15 w(BOTTOM)f(=)g(0)h Fq(\(Ca)o(v)o(eat:)23 b(De\014ning)c Fc(MPI)p 1681 2647 V 14 w(BOTTOM)189 2704 y(=)e(0)h Fq(implies)i(that)e Fc(NULL)i Fq(p)q(oin)o(ter)e(cannot)h(b)q(e)g(distinguished)i(from)d Fc(MPI)p 1502 2704 V 14 w(BOTTOM)p Fq(;)g(ma)o(y)f(b)q(e)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fq(12)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)189 45 y Fc(MPI)p 266 45 13 2 v 14 w(BOTTOM)12 b(=)i(1)f Fq(is)i(b)q(etter)f Fb(:)8 b(:)g(:)e Fq(\))13 b(Requiring)j(that)e(the)g(F)l(ortran)f(and)h(C)g(v)m(alues)h(b)q(e)g (the)f(same)189 102 y(will)i(complicate)h(the)e(initialization)j(pro)q (cess.)i(\()p Fi(End)c(of)g(r)n(ationale.)p Fq(\))189 208 y Fi(A)n(dvic)n(e)e(to)h(implementors.)39 b Fq(The)14 b(follo)o(wing)g(implemen)o(tation)h(can)f(b)q(e)h(used:)k(MPI)14 b(addresses,)189 264 y(as)c(returned)h(b)o(y)g Fj(MPI)p 564 264 14 2 v 15 w(ADDRESS)p Fq(,)h(will)g(ha)o(v)o(e)e(the)h(same)f (v)m(alue,)j(b)q(oth)e(in)g(F)l(ortran)f(and)h(in)g(C.)f(One)189 321 y(ob)o(vious)17 b(c)o(hoice)h(is)f(that)f(MPI)h(addresses)g(b)q(e)h (iden)o(tical)h(to)d(regular)h(addresses.)26 b(The)17 b(address)189 377 y(is)i(stored)f(in)i(the)f(datat)o(yp)q(e,)f(when)i (datat)o(yp)q(es)e(with)h(absolute)g(addresses)g(are)f(constructed.)189 434 y(When)f(a)f(send)i(or)e(receiv)o(e)i(op)q(eration)f(is)g(p)q (erformed,)g(then)g(addresses)g(stored)f(in)i(a)e(datat)o(yp)q(e)189 490 y(are)e(in)o(terpreted)h(as)f(displacemen)o(ts)i(that)d(are)i(all)g (augmen)o(ted)f(b)o(y)h(a)f(base)g(address.)20 b(This)15 b(base)189 547 y(address)d(is)h(\(the)f(address)g(of)t(\))f Fj(buf)p Fq(,)j(or)d(zero,)i(if)g Fj(buf)g(=)f(MPI)p 1203 547 V 16 w(BOTTOM)p Fq(.)h(Th)o(us,)f(if)h Fc(MPI)p 1681 547 13 2 v 14 w(BOTTOM)189 603 y Fq(is)j(zero)g(then)h(a)f(send)g (or)g(receiv)o(e)h(call)g(with)g Fj(buf)g(=)f(MPI)p 1189 603 14 2 v 16 w(BOTTOM)h Fq(is)g(implemen)o(ted)h(exactly)189 659 y(as)12 b(a)h(call)i(with)e(a)g(regular)g(bu\013er)g(argumen)o(t:) 18 b(in)c(b)q(oth)g(cases)f(the)g(base)g(address)g(is)h Fj(buf)p Fq(.)20 b(On)14 b(the)189 716 y(other)j(hand,)h(if)f Fc(MPI)p 559 716 13 2 v 15 w(BOTTOM)g Fq(is)h(not)f(zero,)g(then)h(the) f(implemen)o(tation)i(has)e(to)g(b)q(e)h(sligh)o(tly)189 772 y(di\013eren)o(t.)h(A)12 b(test)g(is)h(p)q(erformed)g(to)f(c)o(hec) o(k)g(whether)h Fj(buf)g(=)g(MPI)p 1325 772 14 2 v 16 w(BOTTOM)p Fq(.)g(If)g(true,)f(then)h(the)189 829 y(base)i(address)g (is)h(zero,)f(otherwise)g(it)h(is)g Fj(buf)p Fq(.)21 b(In)15 b(particular,)h(if)g Fc(MPI)p 1393 829 13 2 v 14 w(BOTTOM)f Fq(do)q(es)g(not)g(ha)o(v)o(e)189 885 y(the)f(same)g(v)m (alue)i(in)f(F)l(ortran)f(and)g(C,)g(then)h(an)f(additional)i(test)e (for)g Fj(bu\013)h(=)f(MPI)p 1618 885 14 2 v 16 w(BOTTOM)h Fq(is)189 942 y(needed)h(in)g(at)f(least)g(one)g(of)g(the)g(t)o(w)o(o)f (languages.)189 1017 y(It)h(ma)o(y)g(b)q(e)i(desirable)g(to)e(use)h(a)f (v)m(alue)i(other)e(than)h(zero)f(for)g Fc(MPI)p 1350 1017 13 2 v 14 w(BOTTOM)h Fq(ev)o(en)g(in)g(C,)f(so)g(as)189 1073 y(to)g(distinguish)k(it)e(from)f(a)g(NULL)h(p)q(oin)o(ter.)24 b(If)17 b Fc(MPI)p 1115 1073 V 14 w(BOTTOM)e(=)g(c)i Fq(then)g(one)g(can)f(still)i(a)o(v)o(oid)189 1130 y(the)h(test)g Fj(bu\013)h(=)g(MPI)p 597 1130 14 2 v 16 w(BOTTOM)p Fq(,)f(b)o(y)h (using)g(the)g(displacemen)o(t)h(from)d Fc(MPI)p 1578 1130 13 2 v 15 w(BOTTOM)p Fq(,)g(i.e.,)189 1186 y(the)c(regular)g (address)g(-)h(c,)f(as)g(the)g(MPI)g(address)g(returned)h(b)o(y)f Fj(MPI)p 1384 1186 14 2 v 16 w(ADDRESS)h Fq(and)f(stored)g(in)189 1243 y(absolute)h(datat)o(yp)q(es.)k(\()p Fi(End)d(of)g(advic)n(e)h(to) f(implementors.)p Fq(\))75 1363 y Fj(Addresses)75 1449 y Fq(Some)f(of)f(the)h(datat)o(yp)q(e)f(accessors)g(and)h(constructors) f(ha)o(v)o(e)g(argumen)o(ts)g(of)g(t)o(yp)q(e)h Fc(MPI)p 1608 1449 13 2 v 15 w(Aint)p Fq(,)f(in)h(C,)g(to)75 1505 y(hold)g(addresses.)20 b(The)15 b(corresp)q(onding)h(argumen)o(ts,)d (in)j(F)l(ortran,)d(ha)o(v)o(e)h(t)o(yp)q(e)h Fc(INTEGER)p Fq(.)f(This)h(causes)75 1561 y(F)l(ortran)j(and)h(C)g(to)f(b)q(e)h (incompatible,)j(in)e(an)f(en)o(vironmen)o(t)g(where)g(addresses)g(ha)o (v)o(e)g(64)f(bits,)i(but)75 1618 y(F)l(ortran)14 b Fc(INTEGER)p Fq(s)g(ha)o(v)o(e)h(32)g(bits.)166 1674 y(This)k(is)h(a)e(problem,)i (irresp)q(ectiv)o(e)h(of)d(in)o(terlanguage)h(issues.)32 b(Supp)q(ose)20 b(that)e(a)g(F)l(ortran)g(pro-)75 1731 y(cess)h(has)g(an)g(address)g(space)g(of)g Fm(\025)g Fq(4)g(GB.)g(What)f(should)i(b)q(e)g(the)f(v)m(alue)h(returned)f(in)h (F)l(ortran)e(b)o(y)75 1787 y Fj(MPI)p 160 1787 14 2 v 16 w(ADDRESS)p Fq(,)i(for)e(a)h(v)m(ariable)i(with)e(an)h(address)f (ab)q(o)o(v)o(e)g(2)1195 1771 y Fa(32)1232 1787 y Fq(?)33 b(The)19 b(prop)q(osed)h(design)g(aims)g(to)75 1844 y(solv)o(e)15 b(this)h(issue,)g(while)g(main)o(taining)h(compatibilit)o(y)f(with)g (curren)o(t)f(F)l(ortran)f(co)q(des.)166 1900 y(The)h(constan)o(t)g Fc(MPI)p 520 1900 13 2 v 14 w(ADDRESS)p 720 1900 V 14 w(SIZE)g Fq(is)h(de\014ned)g(so)f(that,)f(in)i(F)l(ortran)e(90,)75 1957 y Fc(INTEGER\(KIND=MPI)p 474 1957 V 14 w(ADDRESS)p 674 1957 V 14 w(SIZE\))c Fq(is)h(an)f(address)g(sized)h(in)o(teger)g(t) o(yp)q(e)f(\(t)o(ypically)l(,)i Fc(MPI)p 1687 1957 V 14 w(ADDRESS)p 1887 1957 V 15 w(SIZE)75 2013 y Fq(is)k(4)f(on)g(32)g (bit)g(address)h(mac)o(hines)g(and)f(8)g(on)g(64)g(bit)h(address)f(mac) o(hines\).)20 b(Similarly)l(,)d(the)f(constan)o(t)75 2070 y Fc(MPI)p 152 2070 V 14 w(INTEGER)p 340 2070 V 15 w(SIZE)j Fq(is)i(de\014ned)g(so)f(that)f Fc(INTEGER\(KIND=MPI)p 1232 2070 V 14 w(INTEGER)p 1420 2070 V 14 w(SIZE\))h Fq(is)h(a)e(default)i(size)75 2126 y Fc(INTEGER)p Fq(.)166 2182 y(There)15 b(are)g(sev)o(en)h(functions)g(that)e(ha)o(v)o(e)h (address)g(argumen)o(ts:)k Fj(MPI)p 1380 2182 14 2 v 16 w(TYPE)p 1513 2182 V 17 w(STRUCT,)75 2239 y(MPI)p 160 2239 V 16 w(TYPE)p 293 2239 V 17 w(HVECTOR,)d(MPI)p 632 2239 V 16 w(TYPE)p 765 2239 V 16 w(HINDEXED,)f(MPI)p 1118 2239 V 16 w(TYPE)p 1251 2239 V 17 w(EXTENT,)g(MPI)p 1556 2239 V 16 w(TYPE)p 1689 2239 V 17 w(LB,)75 2295 y(MPI)p 160 2295 V 16 w(TYPE)p 293 2295 V 17 w(UB)f Fq(and)g Fj(MPI)p 554 2295 V 15 w(ADDRESS)h Fq(\(the)e(address)h(is)g(an)g Ff(IN)f Fq(argumen)o(t)g(in)i(the)e(\014rst)h(three)f(and)h(an)75 2352 y Ff(OUT)k Fq(argumen)o(t)e(in)i(the)g(last)f(four\).)25 b(These)18 b(functions)g(should)g(accept)f(address)h(argumen)o(ts)e (whic)o(h)75 2408 y(are)10 b(either)h(address)g(sized,)h(or)e(are)g (default)h(in)o(tegers.)18 b(The)10 b(en)o(vironmen)o(tal)i(v)m (ariable)f Fc(MPI)p 1625 2408 13 2 v 15 w(AINT)p 1737 2408 V 14 w(SIZE)f Fq(is)75 2465 y(pro)o(vided)h(to)e(indicate)j(the)e (curren)o(t)g(size)h(for)f(address)g(argumen)o(ts.)17 b(If)11 b Fc(MPI)p 1358 2465 V 14 w(AINT)p 1469 2465 V 15 w(SIZE)e(=)g(MPI)p 1694 2465 V 15 w(ADDRESS)p 1895 2465 V 14 w(SIZE)75 2521 y Fq(then)15 b(these)f(sev)o(en)h(functions)g (accept)f(\(or)f(return\))h(address)g(sized)i(in)o(teger)e(v)m(alues)i (\(32)d(bits)i(on)f(32)f(bit)75 2578 y(mac)o(hines)g(and)g(64)f(bits)h (on)f(64)g(bit)h(mac)o(hines\).)20 b(If)12 b Fc(MPI)p 1034 2578 V 15 w(AINT)p 1146 2578 V 14 w(SIZE)g(=)f(MPI)p 1375 2578 V 15 w(INTEGER)p 1564 2578 V 14 w(SIZE)h Fq(then)h(these)75 2634 y(functions)j(accept)f(or)g(return)g(address)g(v)m(alues)i(that)d (ha)o(v)o(e)h Fc(INTEGER)f Fq(t)o(yp)q(e.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fh(9.3.)34 b(INTERLANGUA)o(GE)16 b(INTER)o(OPERABILITY)751 b Fq(13)166 45 y Fc(MPI)p 243 45 13 2 v 14 w(AINT)p 354 45 V 15 w(SIZE)15 b Fq(has)g(initial)j(v)m (alue)e Fc(MPI)p 874 45 V 14 w(INTEGER)p 1062 45 V 15 w(SIZE)p Fq(.)e(Implemen)o(ters)i(ma)o(y)f(sp)q(ecify)i(a)e(mec)o(h-)75 102 y(anism)i(for)e(c)o(hanging)i(the)f(initial)i(v)m(alue,)g(as)d (part)h(of)g(the)g(program)f(start-up)h(pro)q(cedure)h(\(e.g.,)e(with) 75 158 y(argumen)o(ts)e(to)h(the)g(link)o(er)i(or)d(to)h Fc(mpirun)p Fq(\).)19 b(The)c(v)m(ariable)g Fc(MPI)p 1172 158 V 15 w(AINT)p 1284 158 V 15 w(SIZE)f Fq(can)g(b)q(e)h(rep)q (eatedly)g(mo)q(di-)75 214 y(\014ed)g(during)g(program)f(execution.)20 b(In)15 b(a)g(m)o(ultithreaded)g(en)o(vironmen)o(t,)g Fc(MPI)p 1443 214 V 14 w(AINT)p 1554 214 V 15 w(SIZE)f Fq(should)h(not)75 271 y(b)q(e)h(mo)q(di\014ed)g(while)h(an)e(MPI)g (function)h(with)g(address)f(argumen)o(t)g(executes.)166 375 y Fd(Discussion:)34 b Ff(W)m(e)14 b(ma)o(y)e(also)h(w)o(an)o(t)h (to)f(pro)o(vide)h(a)g(deriv)o(ed)g(datat)o(yp)q(e)g Fc(MPI)p 1410 375 V 15 w(AINT)f Ff(to)h(b)q(e)h(equiv)n(alen)o(t)e(to) 75 431 y Fc(INTEGER\(KIND)f(=)g(MPI)p 498 431 V 15 w(ADDRESS)p 699 431 V 14 w(SIZE\))p Ff(,)g(on)g(those)i(F)m(ortran)e(systems)h (that)f(supp)q(ort)i(deriv)o(ed)f(datat)o(yp)q(es.)189 641 y Fi(A)n(dvic)n(e)20 b(to)h(users.)72 b Fq(Curren)o(t)20 b(F)l(ortran)f(MPI)h(co)q(des)h(will)h(run)f(unmo)q(di\014ed,)i(with)e (the)g(cur-)189 697 y(ren)o(t)i(prop)q(osal.)44 b(New)24 b(co)q(des)f(can)h(b)q(e)g(written)f(so)g(that)g(they)g(alw)o(a)o(ys)g (use)g(address)h(sized)189 754 y(address)c(argumen)o(ts.)35 b(Suc)o(h)21 b(new)g(co)q(de)g(declares)h(address)e(v)m(ariables)i(to)e (b)q(e)h(of)g(t)o(yp)q(e)f Fc(INTE-)189 810 y(GER\(KIND=MPI)p 494 810 V 13 w(ADDRESS)p 693 810 V 15 w(SIZE\))p Fq(,)h(and)g (initializes)k Fc(MPI)p 1221 810 V 14 w(AINT)p 1332 810 V 15 w(SIZE)19 b(=)h(MPI)p 1578 810 V 14 w(ADDRESS)p 1778 810 V 14 w(SIZE)189 867 y Fq(\(or)9 b(starts)g(up)h(the)h (computation)f(so)f(that)h(the)g(initial)i(v)m(alue)g(of)e Fc(MPI)p 1353 867 V 14 w(AINT)p 1464 867 V 15 w(SIZE)f Fq(is)i Fc(MPI)p 1689 867 V 14 w(ADDRESS)p 1889 867 V 15 w(SIZE)p Fq(\).)189 923 y(\\Old")16 b(co)q(de)f(and)h(\\new")f(co)q (de)h(can)f(b)q(e)h(mixed.)21 b(E.g.,)14 b(a)h(new)h(st)o(yle)f (library)h(can)g(co)q(exist)g(with)189 980 y(an)i(old)i(st)o(yle)e (user)h(co)q(de,)h(if)f(the)g(v)m(alue)h(of)e Fc(MPI)p 1046 980 V 15 w(AINT)p 1158 980 V 14 w(SIZE)h Fq(is)g(mo)q(di\014ed)h (up)q(on)f(en)o(try)g(to)f(the)189 1036 y(library)h(and)f(restored)g (up)q(on)h(exit.)30 b(This)19 b(assumes)f(that)g(the)h(user)f(co)q(de)h (do)q(es)g(not)f(pass)g(ad-)189 1093 y(dresses)d(\(or)f(datat)o(yp)q (es)h(con)o(taining)h(addresses\))f(to)g(the)g(library)l(.)21 b(In)16 b(the)f(later)g(case,)g(then)h(the)189 1149 y(library)f(has)f (to)g(written)g(so)g(as)g(to)g(function)h(correctly)g(with)f(either)i (t)o(yp)q(es)e(of)g(addresses.)20 b(\()p Fi(End)189 1206 y(of)c(advic)n(e)g(to)h(users.)p Fq(\))189 1312 y Fi(A)n(dvic)n(e)22 b(to)h(implementors.)84 b Fq(In)23 b(F)l(ortran)e(90,)j(o)o(v)o (erloading)e(can)h(b)q(e)g(used)g(so)f(as)g(to)g(pro-)189 1368 y(vide)d(di\013eren)o(t)f(v)o(ersions)g(of)f(the)h(functions)h Fj(MPI)p 1065 1368 14 2 v 15 w(TYPE)p 1197 1368 V 17 w(STRUCT,)f(MPI)p 1503 1368 V 16 w(TYPE)p 1636 1368 V 17 w(HVECTOR,)189 1425 y(MPI)p 274 1425 V 15 w(TYPE)p 406 1425 V 17 w(HINDEXED,)10 b(MPI)p 755 1425 V 16 w(TYPE)p 888 1425 V 17 w(EXTENT,)g(MPI)p 1188 1425 V 16 w(TYPE)p 1321 1425 V 17 w(LB)p Fq(,)f(and)i Fj(MPI)p 1581 1425 V 16 w(TYPE)p 1714 1425 V 16 w(UB)p Fq(,)g(one)189 1481 y(for)e(eac)o(h)h(address)h(size.)19 b(The)10 b(function)h(v)o(ersions) f(can)h(c)o(hec)o(k)f(the)g(curren)o(t)g(v)m(alue)i(of)d Fc(MPI)p 1696 1481 13 2 v 15 w(AINT)p 1808 1481 V 15 w(SIZE)p Fq(,)189 1538 y(to)14 b(catc)o(h)h(errors.)189 1613 y(This)h(do)q(es)g(not)f(w)o(ork)g(with)h(the)g(function)g Fj(MPI)p 1035 1613 14 2 v 16 w(ADDRESS)p Fq(,)g(since)h(this)f(tak)o (es)f(a)h(c)o(hoice)g(argu-)189 1669 y(men)o(t,)j(and)g(an)f(in)o (terface)h(blo)q(c)o(k)h(cannot)e(b)q(e)i(pro)o(vided)g(for)e(it.)31 b(This)19 b(function)h(will)g(need)g(to)189 1726 y(b)q(e)g(reco)q(ded)g (so)f(as)g(to)f(branc)o(h)i(according)f(to)g(the)g(curren)o(t)h(v)m (alue)g(of)f Fc(MPI)p 1528 1726 13 2 v 14 w(AINT)p 1639 1726 V 15 w(SIZE)p Fq(.)g(\(This)189 1782 y(function)d(returns)f(an)g (address,)g(so)g(that)f(error)h(c)o(hec)o(king)h(is)f(not)g(an)g (issue.\))189 1857 y(If)g(o)o(v)o(erloading)h(is)g(not)f(a)o(v)m (ailable,)h(then)g(all)g(sev)o(en)g(functions)g(need)h(b)q(e)f(reco)q (ded)g(so)f(as)g(to)g(tak)o(e)189 1914 y(in)o(to)j(consideration)h(the) g(curren)o(t)f(v)m(alue)i(of)d Fc(MPI)p 1060 1914 V 15 w(AINT)p 1172 1914 V 15 w(SIZE)h Fq(when)g(in)o(terpreting)i(an)e (address)189 1970 y(input)e(parameter,)d(or)i(deciding)i(whic)o(h)e (address)g(v)m(alue)h(to)f(return.)k(\()p Fi(End)d(of)g(advic)n(e)g(to) g(imple-)189 2027 y(mentors.)p Fq(\))166 2174 y Fd(Alternativ)o(es)o(:) 134 2248 y Ff(1.)22 b(A)17 b(w)o(eak)o(er)h(v)o(ersion)g(of)f(this)h (prop)q(osal)f(is)h(to)f(sp)q(ecify)h(that)g Fc(MPI)p 1242 2248 V 14 w(AINT)p 1353 2248 V 15 w(SIZE)f Ff(cannot)h(b)q(e)g(mo) q(difed)e(b)o(y)189 2298 y(the)f(user,)g(and)g(require)h(implem)o(en)o (tors)d(to)h(pro)o(vide)h(an)f(MPI)h(en)o(vironmen)o(t)f(where)i Fc(MPI)p 1634 2298 V 14 w(AINT)p 1745 2298 V 15 w(SIZE=)189 2348 y(MPI)p 266 2348 V 14 w(ADDRESS)p 466 2348 V 14 w(SIZE)d Ff(and)h(an)f(en)o(vironmen)o(t)f(where)j Fc(MPI)p 1149 2348 V 14 w(AINT)p 1260 2348 V 15 w(SIZE)e(=)g(MPI)p 1493 2348 V 15 w(INTEGER)p 1682 2348 V 14 w(SIZE)p Ff(.)g(The)189 2398 y(selection)h(of)e(the)i(en)o(vironmen)o(t)e(can)h(b)q(e)h(done)g (at)f(link)f(time)g(or)h(with)g(argumen)o(ts)f(to)h Fc(mpirun)p Ff(.)18 b(Old)13 b(co)q(de)189 2448 y(can)j(b)q(e)h(run)g(with)f(the)h (former)e(en)o(vironmen)o(t.)25 b(New)17 b(co)q(de,)g(compatible)e (with)h(C,)g(can)h(b)q(e)g(run)f(in)g(the)189 2497 y(later.)29 b(Ho)o(w)o(ev)o(er,)19 b(w)o(e)f(lo)q(ose)g(the)g(abilit)o(y)e(to)h (mix)f(old)h(and)h(new)g(co)q(des:)27 b(If)17 b(a)h(library)f(that)g (calls)h(one)189 2547 y(of)e(the)h(sev)o(en)h(MPI)f(functions)g(with)g (address)h(argumen)o(ts)e(has)h(to)g(run)g(in)g(b)q(oth)g(en)o (vironmen)o(ts,)f(then)189 2597 y(w)o(e)f(need)h(either)g(t)o(w)o(o)e (di\013eren)o(t)i(v)o(ersions)f(of)g(the)g(library)m(,)f(or)h(a)g (library)f(that)h(can)g(accommo)q(date)e(b)q(oth)189 2647 y(en)o(vironmen)o(ts.)j(\(If)10 b(w)o(e)h(go)f(this)g(w)o(a)o(y)m (,)g(should)g(w)o(e)h(also)f(assume)g(that)g(there)i(are)f(t)o(w)o(o)f (di\013eren)o(t)h Fc(mpi.h)f Ff(\014les,)189 2697 y(with)j(t)o(w)o(o)g (di\013eren)o(t)h(de\014nitions)f(of)g Fc(MPI)p 848 2697 V 14 w(AINT)p 959 2697 V 15 w(SIZE)p Ff(?)g(I.e.,)f(do)i(w)o(e)f(supp)q (ort)h(declarations)g(of)f(the)h(form)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fq(14)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)189 45 y Fc(INTEGER\(KIND=MPI)p 588 45 13 2 v 13 w(AINT)p 698 45 V 15 w(SIZE\))p Ff(?)17 b(This)h(will)e(allo)o (w)h(to)g(write)h(source)i(co)q(de)e(that)g(is)g(compatible)189 95 y(with)13 b(either)i(en)o(vironmen)o(ts;)e(but)h(this)g(co)q(de)g (will)f(need)i(to)e(b)q(e)i(compiled)d(with)i(the)g(righ)o(t)g Fc(mpi.h)f Ff(\014le.\))134 170 y(2.)22 b(An)14 b(alternativ)o(e)h(is)g (to)f(c)o(hange)h(the)g(F)m(ortran)g(90)f(binding)g(so)h(that)g (functions)f(that)h(ha)o(v)o(e)g(argumen)o(ts)f(of)189 219 y(t)o(yp)q(e)k Fc(MPI)p 363 219 V 15 w(Aint)g Ff(will)e(accept)j (an)f(additional)e(optional)h(parameter,)h Fc(aint)p 1394 219 V 16 w(size)p Ff(,)g(with)g(a)g(default)g(v)n(alue)189 269 y(whic)o(h)d(is)h(the)h(curren)o(t)g(size)f(of)g(an)f Fc(INTEGER)p Ff(.)g(Th)o(us,)h(curren)o(t)h(MPI)f(F)m(ortran)g(co)q(de) h(will)d(run)i(without)189 319 y(c)o(hanges;)h(co)q(de)g(that)f(w)o(an) o(ts)g(to)g(use)h(an)e(address)j(size)e(that)g(is)g(di\013eren)o(t)h (from)d(the)j(default)f Fc(INTEGER)189 369 y Ff(will)f(need)k(to)e(pro) o(vide)g(the)g(additional)f(parameter.)27 b(Note)18 b(that)f(the)h (same)e(implemen)o(tation)e(can)j(b)q(e)189 419 y(used)h(as)f(for)g (the)h(main)e(prop)q(osal.)28 b(The)18 b(di\013erence)h(is)e(whether)i Fc(aint)p 1339 419 V 15 w(size)f Ff(is)f(a)g(global)f(v)n(ariable)g(or) h(a)189 469 y(function)c(argumen)o(t.)k(This)d(design)g(is)g(cleaner,)g (but)g(cannot)g(b)q(e)h(supp)q(orted)g(in)e(F)m(ortran)h(77.)75 662 y Fg(9.3.7)49 b(Group)75 748 y Fq(A)15 b(group)h(ob)s(ject)e (represen)o(t)i(the)f(same)g(group)g(in)i(b)q(oth)e(F)l(ortran)f(and)i (C:)f(the)g(same)g(pro)q(cesses,)h(with)75 805 y(the)f(same)g(ranks.)20 b(Group)15 b(accessors)f(suc)o(h)i(as)f Fj(MPI)p 986 805 14 2 v 15 w(GROUP)p 1153 805 V 18 w(SIZE)g Fq(return)g(the)h(same)f (v)m(alue.)75 926 y Fg(9.3.8)49 b(Communicato)o(rs)75 1012 y Fq(A)14 b(comm)o(unicator)f(represen)o(ts)h(the)g(same)f(group)h (and)g(same)f(comm)o(unication)i(domain,)f(b)q(oth)g(in)g(F)l(or-)75 1069 y(tran)c(and)h(C.)f(In)i(particular,)g(a)e(message)h(sen)o(t)f (from)g(F)l(ortran)g(using)h(a)g(comm)o(unicator)f(can)h(b)q(e)h (receiv)o(ed)75 1125 y(from)18 b(C,)h(using)g(the)g(matc)o(hing)h(comm) o(unicator)e(on)h(another)g(pro)q(cess;)h(collectiv)o(e)h(comm)o (unication)75 1182 y(can)c(in)o(v)o(olv)o(e)h(b)q(oth)f(C)g(and)g(F)l (ortran)f(pro)q(cesses,)h(using)h(matc)o(hing)f(comm)o(unicators;)g (accessors)g(suc)o(h)75 1238 y(as)e Fj(MPI)p 216 1238 V 16 w(COMM)p 374 1238 V 16 w(SIZE)g Fq(return)g(the)g(same)g(v)m (alue;)h(etc.)166 1294 y(T)l(op)q(ology)g(information)h(asso)q(ciated)g (to)f(a)g(comm)o(unicator)g(in)i(C)e(can)h(b)q(e)g(accessed)g(in)g(F)l (ortran,)75 1351 y(and)e(vice)i(v)o(ersa.)75 1471 y Fj(A)o(ttributes)75 1557 y Fq(A)o(ttribute)g(k)o(eys)g(allo)q(cated)i(in)f(F)l(ortran)e (can)h(b)q(e)h(used)g(or)f(freed)h(in)g(C,)f(and)g(vice)h(v)o(ersa.)26 b(A)o(ttributes)75 1613 y(v)m(alues)17 b(set)f(in)i(F)l(ortran)d(can)h (b)q(e)h(accessed)g(in)g(C,)f(and)g(vice)h(v)o(ersa.)23 b(T)l(o)16 b(ac)o(hiev)o(e)g(this,)h(attribute)f(k)o(eys)75 1670 y(will)f(b)q(e)e(allo)q(cated)h(in)g(a)f(range)f(that)h(is)g(v)m (alid)i(b)q(oth)e(for)f(C)h Fc(int)g Fq(t)o(yp)q(e)g(and)h(F)l(ortran)d Fc(INTEGER)i Fq(t)o(yp)q(e.)19 b(The)75 1726 y(same)c(holds)h(true)f (for)f(system)h(de\014ned)i(attribute)e(v)m(alues)h(\(suc)o(h)f(as)g Fc(MPI)p 1361 1726 13 2 v 15 w(T)m(A)o(G)p 1456 1726 V 13 w(UB,)e(MPI)p 1626 1726 V 14 w(IO)p Fq(,)h(etc.\))166 1783 y(A)o(ttribute)20 b(k)o(eys)g(declared)i(in)f(C)f(are)g(asso)q (ciated)h(with)g(\\C)f(attributes")f(and)i(attribute)f(k)o(eys)75 1839 y(declared)15 b(in)f(F)l(ortran)e(are)h(asso)q(ciated)h(with)g (\\F)l(ortran)e(attributes".)19 b(C)13 b(attributes)g(ha)o(v)o(e)g(C)g (cop)o(y)h(and)75 1896 y(delete)20 b(callbac)o(k)f(functions)g(\(the)g (functions)g(pro)o(vided)g(b)o(y)g(the)f Fj(MPI)p 1308 1896 14 2 v 16 w(Keyval)p 1451 1896 V 16 w(create)h Fq(call\).)30 b(F)l(ortran)75 1952 y(attributes)14 b(ha)o(v)o(e)g(F)l(ortran)g (delete)h(and)g(callbac)o(k)g(functions.)21 b(When)15 b(a)f(comm)o(unicator)g(is)h(duplicated)75 2008 y(then,)21 b(for)e(eac)o(h)h(attribute,)h(the)f(corresp)q(onding)h(cop)o(y)f (function)h(is)f(called,)i(using)f(the)f(righ)o(t)g(C)f(or)75 2065 y(F)l(ortran)14 b(calling)j(con)o(v)o(en)o(tion;)e(and)g (similarly)l(,)i(for)e(the)g(delete)h(callbac)o(k)g(function.)189 2171 y Fi(A)n(dvic)n(e)i(to)i(implementors.)59 b Fq(This)19 b(requires)h(that)d(attributes)i(b)q(e)g(tagged)e(either)j(as)e(\\C")f (or)189 2228 y(\\F)l(ortran",)j(and)h(that)f(the)g(language)h(tag)f(b)q (e)i(c)o(hec)o(k)o(ed)f(in)g(order)g(to)f(use)h(the)g(righ)o(t)f (calling)189 2284 y(con)o(v)o(en)o(tion)15 b(for)f(the)i(callbac)o(k)g (function.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))166 2390 y(C)k(attributes)f(are)h(of)f(t)o(yp)q(e)h Fc(void*)p Fq(,)h(whereas)f(F)l(ortran)f(attributes)g(are)h(of)g(t)o (yp)q(e)g Fc(INTEGER)p Fq(.)e(On)75 2447 y(some)h(systems,)g Fc(INTEGER)p Fq(s)g(will)i(ha)o(v)o(e)d(32)h(bits,)h(while)h(C)e(p)q (oin)o(ters)h(will)h(ha)o(v)o(e)e(64)f(bits.)33 b(This)20 b(is)f(a)75 2503 y(problem)d(if)g(comm)o(unicator)f(attributes)g(are)h (used)g(to)f(mo)o(v)o(e)f(information)i(from)f(a)g(F)l(ortran)f(caller) j(to)75 2560 y(a)e(C)g(callee,)h(or)f(vice-v)o(ersa.)166 2616 y(W)l(e)f(prop)q(ose)g(to)f(use)h(the)g(en)o(vironmen)o(t)g(v)m (ariable)h Fc(MPI)p 1124 2616 13 2 v 14 w(AINT)p 1235 2616 V 15 w(SIZE)e Fq(to)g(indicate)j(the)d(curren)o(t)h(size)75 2673 y(of)19 b(attributes)g(in)h(F)l(ortran.)31 b(If)19 b(an)g(attribute)g(is)h(set)f(in)h(4)f(b)o(yte)g(mo)q(de)g(and)h(is)f (accessed)h(in)g(8)f(b)o(yte)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fh(9.3.)34 b(INTERLANGUA)o(GE)16 b(INTER)o(OPERABILITY)751 b Fq(15)75 45 y(mo)q(de,)19 b(then)g(the)g(4)f(b)o(yte)g(v)m(alue)i(is)f(sign)g(extended)g(to)f(8)g (b)o(ytes.)30 b(If)19 b(an)f(attribute)g(is)h(set)g(in)g(8)f(b)o(yte)75 102 y(mo)q(de,)d(next)g(read)h(in)g(4)e(b)o(yte)h(mo)q(de,)h(then)f (the)g(8)g(b)o(yte)g(v)m(alue)i(is)e(truncated)h(to)e(4)h(b)o(ytes.)166 205 y Fd(Discussion:)34 b Ff(W)m(e)13 b(ma)o(y)f(w)o(an)o(t)h(to)h (raise)g(an)g(error)h(if)e(the)h(truncation)g(causes)h(loss)f(of)f (information)e(\(i.e.,)75 262 y(if)i(the)i(64)e(bit)g(in)o(teger)i (cannot)f(b)q(e)g(con)o(v)o(erted)h(to)f(a)g(32)f(bit)h(in)o(teger.)189 389 y Fi(A)n(dvic)n(e)h(to)h(implementors.)40 b Fq(Comm)o(unicator)14 b(structures)h(are)f(set)h(for)f(8)h(b)o(yte)f(attributes,)h(but)189 446 y(F)l(ortran)d(access)i(functions)h(will)g(access)f(either)g(4)g (or)f(8)h(b)o(ytes,)f(according)h(to)f(the)h(curren)o(t)g(v)m(alue)189 502 y(of)g Fc(MPI)p 317 502 13 2 v 15 w(AINT)p 429 502 V 15 w(SIZE)p Fq(.)189 573 y(Ov)o(erloading)k(can)g(b)q(e)h(used)f(in)h (F)l(ortran)d(90)i(to)f(supp)q(ort)h(the)f(t)o(w)o(o)g(v)o(ersions)h (of)f(the)h(attribute)189 630 y(access)d(functions.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))166 751 y Fd(Alternativ)o(es)o(:)134 818 y Ff(1.)22 b(No)13 b(c)o(hange.)19 b(On)14 b(a)g(64)f(bit)g(system,)h(the)g (attributes)h(will)d(b)q(e,)i(in)o(ternally)m(,)e(of)i(size)g(64)g (bits.)189 868 y Fc(MPI)p 266 868 V 14 w(A)m(TTRIBUTE)p 510 868 V 13 w(GET)p Ff(,)20 b(in)h(F)m(ortran,)h(will)e(truncate)i (the)f(64)g(bit)g(v)n(alue)f(to)h(a)f(32)h(bit)f(v)n(alue,)i(and)189 917 y Fc(MPI)p 266 917 V 14 w(A)m(TTRIBUTE)p 510 917 V 13 w(SET)14 b Ff(will)f(con)o(v)o(ert)h(the)h(32)e(bit)h(argumen)o(t) e(to)i(a)g(64)f(bit)h(in)o(teger.)134 985 y(2.)22 b(Add)14 b(an)f(optional)g Fc(aint)p 567 985 V 15 w(size)h Ff(argumen)o(t)f(to)h (attribute)g(access)i(functions)e(in)f(F)m(ortran)h(90.)166 1059 y(The)k(problem)f(w)o(e)h(face)g(in)f(this)h(section)h(is)e(the)i (same)e(as)h(the)g(problem)f(w)o(e)h(faced)g(with)f(addresses)j(in)75 1115 y(Section)14 b(9.3.6;)e(w)o(e)i(should)g(adopt)f(here)i(the)g (same)e(solution)g(that)h(w)o(e)g(adopt)g(for)f(addresses)j(there.)75 1281 y Fg(9.3.9)49 b(Requests)75 1366 y Fq(A)17 b(request)h(created)f (in)h(C)g(can)f(b)q(e)h(accessed)g(and)g(up)q(dated)g(in)g(F)l(ortran)e (b)o(y)i(w)o(ait)f(or)f(test)h(functions)75 1423 y(and)e(can)h(b)q(e)g (freed)f(in)h(F)l(ortran;)e(and)h(vice)h(v)o(ersa.)75 1541 y Fg(9.3.10)49 b(Erro)o(r)15 b(handlers)75 1627 y Fq(Error)g(handlers)j(allo)q(cated)f(in)g(F)l(ortran)f(can)g(b)q(e)h (in)o(v)o(ok)o(ed)g(or)f(deallo)q(cated)i(from)d(C,)h(and)h(vice)g(v)o (ersa.)75 1683 y(When)i(an)g(MPI)f(exception)i(o)q(ccurs,)f(then)g(the) g(last)f(error)g(handler)i(asso)q(ciated)f(with)g(the)f(relev)m(an)o(t) 75 1740 y(comm)o(unicator)12 b(is)g(in)o(v)o(ok)o(ed,)g(irresp)q(ectiv) o(e)i(of)d(the)h(language)h(en)o(vironmen)o(t)f(where)g(the)g(error)f (o)q(ccurred)75 1796 y(and)g(the)h(en)o(vironmen)o(t)f(where)h(the)f (handler)h(w)o(as)f(asso)q(ciated)g(with)h(the)f(comm)o(unicator.)18 b(The)12 b(handler)75 1853 y(should)k(b)q(e)g(in)o(v)o(ok)o(ed)f(with)h (the)f(righ)o(t)g(parameter)g(passing)g(con)o(v)o(en)o(tion.)189 1933 y Fi(A)n(dvic)n(e)k(to)i(implementors.)69 b Fq(error)19 b(handler)i(ob)s(jects)e(need)i(to)f(ha)o(v)o(e)f(a)h(language)g(tag,)g (lik)o(e)189 1989 y(attribute)14 b(k)o(eys,)g(so)g(as)h(to)e(use)i(the) g(righ)o(t)f(calling)j(con)o(v)o(en)o(tion.)i(These)c(handlers)h(ha)o (v)o(e,)e(in)h(C,)f(a)189 2046 y(\\)p Fk(stdargs)p Fq(")e(argumen)o(t)h (list.)20 b(It)13 b(migh)o(t)h(b)q(e)g(useful)h(to)e(pro)o(vide)h(to)f (the)h(handler)g(information)g(on)189 2102 y(the)e(language)h(en)o (vironmen)o(t)g(where)g(the)g(error)f(o)q(ccurred.)20 b(\()p Fi(End)13 b(of)i(advic)n(e)e(to)i(implementors.)p Fq(\))75 2220 y Fg(9.3.11)49 b(Reduce)16 b(op)q(erations)75 2306 y Fq(Reduce)h(op)q(erations)e(de\014ned)i(in)f(C)f(can)g(b)q(e)h (used)g(or)e(deallo)q(cated)j(in)f(F)l(ortran,)e(and)h(vice)h(v)o (ersa.)189 2386 y Fi(A)n(dvic)n(e)g(to)h(users.)47 b Fq(Reduce)18 b(op)q(erations)e(receiv)o(e,)h(as)f(one)g(of)g(their)h (argumen)o(t,)e(the)h(datat)o(yp)q(e)189 2443 y(of)f(the)g(op)q (erands.)22 b(Th)o(us,)15 b(one)h(can)g(de\014ne,)g(either)g(in)h(C)e (or)g(F)l(ortran,)f(\\p)q(olymorphic")j(reduce)189 2499 y(op)q(erations)e(that)f(w)o(ork)h(b)q(oth)g(for)g(C)g(and)g(F)l (ortran)f(datat)o(yp)q(es.)19 b(\()p Fi(End)d(of)g(advic)n(e)g(to)h (users.)p Fq(\))189 2579 y Fi(A)n(dvic)n(e)c(to)i(implementors.)39 b Fq(Reduce)15 b(op)q(eration)f(ob)s(jects)f(need)h(to)f(carry)g(a)h (language)f(tag,)g(lik)o(e)189 2636 y(attribute)h(k)o(eys,)g(so)h(as)f (to)g(use)h(the)g(righ)o(t)f(calling)j(con)o(v)o(en)o(tion.)i(\()p Fi(End)d(of)g(advic)n(e)f(to)h(implemen-)189 2692 y(tors.)p Fq(\))-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Fq(16)1138 b Fh(CHAPTER)15 b(9.)30 b(MISCELLANY)75 45 y Fg(9.3.12)49 b(Status)75 131 y Fq(The)15 b(MPI)f(status)f(ob)s(ject)h(is)h(not)f(really)h(an)f(opaque)g(ob)s (ject,)g(since)h(its)g(structure)f(is)g(partially)i(visible)75 187 y(to)i(the)h(user,)h(and)f(the)g(ob)s(ject)f(is)h(allo)q(cated)h(b) o(y)f(the)g(user.)31 b(A)19 b(C)f(status)g(ob)s(ject)g(is)i(a)e (structure;)i(a)75 244 y(F)l(ortran)12 b(status)g(ob)s(ject)h(is)g(an)g (arra)o(y)l(.)19 b Fj(MPI)12 b Fq(do)q(es)h(not)g(guaran)o(tee)g(that)f (status)g(ob)s(jects)h(can)g(b)q(e)h(passed)75 300 y(from)g(one)i (language)f(to)g(another.)166 397 y Fd(Alternativ)o(es)o(:)134 472 y Ff(1.)22 b(W)m(e)9 b(pro)o(vide)h(status)h(con)o(v)o(ersion)g (functions,)f(that)h(con)o(v)o(ert)g(a)e(status)i(ob)r(ject)g(to)f(a)g (language)g(indep)q(enden)o(t)189 522 y(status)k(ob)r(ject,)f(and)g (vice)h(v)o(ersa.)k(The)c(language)e(indep)q(enden)o(t)i(status)g(ob)r (ject)g(is)f(an)g(arra)o(y)g(of)g(in)o(tegers.)134 597 y(2.)22 b(W)m(e)14 b(require)i(that)f(status)h(ob)r(jects)h(ha)o(v)o(e) d(the)i(same)e(memory)f(la)o(y)o(out)h(in)g(C)h(and)g(F)m(ortran)g(so)g (that)g(they)189 646 y(can)f(b)q(e)g(passed)h(across)g(languages.)75 840 y Fg(9.3.13)49 b(Constants)75 926 y Fq(MPI)14 b(constan)o(ts)g(ha)o (v)o(e)g(the)g(same)g(v)m(alue)i(in)f(F)l(ortran)f(and)g(C.)g(This)h (includes,)h(error)e(co)q(des,)h(constan)o(ts)75 983 y(suc)o(h)23 b(as)f Fk(MPI)p 323 983 15 2 v 17 w(PROC)p 436 983 V 17 w(NULL)g Fq(or)g Fk(MPI)p 706 983 V 17 w(ANY)p 795 983 V 17 w(TAG)p Fq(,)g(etc.)42 b(This)24 b(do)q(es)f(not)f(apply)i (to)e(constan)o(t)g(handles)75 1039 y(\()p Fc(MPI)p 170 1039 13 2 v 14 w(INT,)17 b(MPI)p 357 1039 V 15 w(COMM)p 502 1039 V 14 w(W)o(ORLD,)h(MPI)p 769 1039 V 14 w(ERRORS)p 943 1039 V 14 w(RETURN,)f(MPI)p 1226 1039 V 14 w(SUM)p Fq(,)i(etc.\))33 b(These)19 b(handles)i(need)75 1095 y(to)16 b(b)q(e)i(con)o(v)o(erted,)e(as)h(explained)i(in)e(Section)h (9.3.4.)23 b Fc(MPI)p 1102 1095 V 14 w(BOTTOM)17 b Fq(ma)o(y)f(ha)o(v)o (e)g(di\013eren)o(t)h(v)m(alues)h(in)75 1152 y(F)l(ortran)c(and)h(C.)75 1274 y Fg(9.3.14)49 b(Interlanguage)17 b(communication)75 1359 y Fq(The)c(t)o(yp)q(e)f(matc)o(hing)g(rules)h(for)f(comm)o (unications)h(in)g(MPI)f(are)g(not)g(c)o(hanged:)19 b(The)12 b(datat)o(yp)q(e)g(sp)q(eci\014-)75 1416 y(cation)i(for)e(eac)o(h)i (item)f(sen)o(t)g(should)i(matc)o(h,)e(textually)l(,)h(the)f(datat)o (yp)q(e)g(sp)q(eci\014cation)i(used)f(to)f(receiv)o(e)75 1472 y(this)j(item)g(\(unless)g(one)g(of)f(the)g(t)o(yp)q(es)h(is)g Fc(MPI)p 878 1472 V 14 w(P)m(A)o(CKED)p Fq(\).)d(And)j(the)f(t)o(yp)q (e)h(of)f(a)g(message)g(item)h(should)75 1529 y(matc)o(h)c(the)i(t)o (yp)q(e)f(declaration)h(for)e(the)h(corresp)q(onding)h(comm)o (unication)g(bu\013er)f(lo)q(cation,)h(unless)g(the)75 1585 y(t)o(yp)q(e)k(is)g Fc(MPI)p 305 1585 V 14 w(BYTE)f Fq(or)g Fc(MPI)p 578 1585 V 15 w(P)m(A)o(CKED)p Fq(.)e(In)o (terlanguage)j(comm)o(unication)g(is)g(allo)o(w)o(ed,)g(if)h(it)f (complies)75 1642 y(with)e(these)f(rules.)75 1748 y Fe(Example)j(9.4)k Fq(In)17 b(the)e(example)i(b)q(elo)o(w,)e(a)h(F)l(ortran)e(arra)o(y)g (is)i(sen)o(t)g(from)e(F)l(ortran)h(and)g(receiv)o(ed)i(in)75 1804 y(C.)75 1911 y Fk(!)24 b(FORTRAN)f(CODE)75 1967 y(REAL)g(R\(5\))75 2024 y(INTEGER)g(TYPE,)g(ADDR,)g(IERR,)g(MYRANK)75 2137 y(!)h(create)f(an)g(absolute)g(datatype)g(for)g(array)h(R)75 2193 y(CALL)f(MPI_ADDRESS\()g(R,)g(ADDR\))75 2249 y(CALL)g (MPI_TYPE_STRUCT\(1,)f(5,)h(ADDR,)g(MPI_REAL,)g(TYPE,)g(IERR\))75 2306 y(CALL)g(MPI_TYPE_COMMIT\()f(TYPE\))75 2419 y(CALL)h (MPI_COMM_RANK\()f(MPI_COMM_WORLD,)g(MYRANK,)h(IERR\))75 2475 y(IF)h(\(MYRANK.EQ.0\))e(THEN)147 2532 y(CALL)h(MPI_SEND\()g (MPI_BOTTOM,)f(1,)h(TYPE,)h(1,)f(0,)h(MPI_COMM_WORLD,)e(IERR\))75 2588 y(ELSE)147 2645 y(CALL)h(MPI_HANDLE2INT\(TYPE,)e(CODE\))147 2701 y(CALL)i(C_ROUTINE\(CODE\))1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fh(9.4.)34 b(TREA)l(TMENT)15 b(OF)g Fj(MPI)p 661 -100 14 2 v 16 w(ST)l(A)l(TUS)991 b Fq(17)75 45 y Fk(END)23 b(IF)75 214 y(/*)h(C)f(code)h(*/)75 327 y(void)f(C_ROUTINE\(MPI_Fint)f(*fhandle\))75 384 y({)75 440 y(void)h(*handle;)75 497 y(MPI_Datatype)f(type;)75 553 y(MPI_Status)g(status;)75 666 y(type)h(=)h (\(MPI_Datatype\)MPI_Int2hand)o(le\(fhand)o(le\);)75 779 y(MPI_Recv\()f(MPI_BOTTOM,)f(1,)i(type,)f(0,)g(0,)h (MPI_COMM_WORLD,)e(status\);)75 835 y(})166 942 y Fq(MPI)17 b(implemen)o(tors)h(ma)o(y)f(w)o(eak)o(en)g(these)h(t)o(yp)q(e)f(matc)o (hing)h(rules,)g(and)g(allo)o(w)g(messages)e(to)h(b)q(e)75 998 y(sen)o(t)c(with)g(F)l(ortran)f(t)o(yp)q(es)h(and)g(receiv)o(ed)h (with)f(C)g(t)o(yp)q(es,)g(and)g(vice)h(v)o(ersa,)e(when)h(those)g(t)o (yp)q(es)g(matc)o(h.)75 1055 y(I.e.,)g(if)h(the)f(F)l(ortran)f(t)o(yp)q (e)h Fc(INTEGER)f Fq(is)i(iden)o(tical)h(to)d(the)i(C)e(t)o(yp)q(e)i Fc(int)p Fq(,)f(then)h(an)f(MPI)g(implemen)o(tation)75 1111 y(ma)o(y)22 b(allo)o(w)g(data)g(to)f(b)q(e)i(sen)o(t)g(with)f (datat)o(yp)q(e)g Fc(MPI)p 1035 1111 13 2 v 14 w(INTEGER)g Fq(and)g(b)q(e)h(receiv)o(ed)h(with)e(datat)o(yp)q(e)75 1168 y Fc(MPI)p 152 1168 V 14 w(INT)p Fq(.)15 b(Ho)o(w)o(ev)o(er,)f (suc)o(h)i(co)q(de)f(is)h(not)f(p)q(ortable.)75 1311 y Fn(9.4)59 b(T)-5 b(reatment)18 b(of)i(MPI)p 670 1311 18 2 v 21 w(Status)75 1412 y Fq(Tw)o(o)10 b(prop)q(osals)h(are)g(b)q (eing)i(made)e(that)f(add)h(to,)g(rather)g(than)g(c)o(hange,)g(the)g (treatmen)o(t)f(of)h Fj(MPI)p 1729 1412 14 2 v 16 w(Status)p Fq(.)75 1534 y Fg(9.4.1)49 b(P)o(assing)17 b(MPI)p 481 1534 15 2 v 18 w(ST)l(A)l(TUS)p 676 1534 V 18 w(NULL)h(fo)o(r)d(MPI)p 987 1534 V 19 w(Status)75 1620 y Fq(There)f(are)f(a)g(n)o(um)o(b)q(er)h (of)f(places)i(in)f(the)g(MPI-1)g(sp)q(eci\014cation)h(where)f (e\016ciency)h(could)g(b)q(e)f(impro)o(v)o(ed)75 1676 y(if)h(a)e(NULL)i(or)f(0)g(argumen)o(t)f(could)i(b)q(e)g(passed)f (instead)h(of)f(the)g(curren)o(tly-sp)q(eci\014ed)j(argumen)o(t)c(as)h (an)75 1733 y(indication)20 b(that)e(the)h(argumen)o(t)f(need)h(not)f (b)q(e)h(pro)q(cessed)g(b)o(y)g(the)f(MPI)h(function.)31 b(In)19 b(particular,)75 1789 y(the)14 b Fj(MPI)p 237 1789 14 2 v 16 w(Status)h Fq(argumen)o(t)e(passed)h(to)f(certain)i(MPI) e(functions)i(could)g(b)q(e)f(allo)o(w)o(ed)g(to)f(b)q(e)i(NULL,)f(in) 75 1846 y(whic)o(h)i(case)f(status)g(\014elds)h(need)g(not)f(b)q(e)h (\014lled)h(in.)913 b Fm(>)16 b Fl(\(July\))166 1902 y Fq(Instead)g(of)e Fk(NULL)p Fq(,)h(a)f(new)i Fj(MPI)e Fq(constan)o(t)h Fj(MPI)p 998 1902 V 16 w(ST)l(A)l(TUS)p 1179 1902 V 17 w(NULL)g Fq(is)h(de\014ned)g(for)f(this)h(purp)q(ose.) 166 1959 y(In)c(general,)h(this)f(optimization)g(can)g(apply)g(to)f (all)i(functions)f(for)f(whic)o(h)h Fj(MPI)p 1514 1959 V 16 w(Status)i Fq(or)d(an)g(arra)o(y)75 2015 y(of)k Fj(MPI)p 212 2015 V 16 w(Status)p Fq('s)i(is)f(an)g(argumen)o(t.)k (These)c(are)f(all)i(the)e(v)m(arious)h(forms)f(of)g Fj(MPI)p 1493 2015 V 16 w(RECV)p Fq(,)h Fj(MPI)p 1735 2015 V 16 w(TEST)p Fq(,)75 2071 y(and)i Fj(MPI)p 251 2071 V 16 w(W)l(AIT)p Fq(.)g(When)h(an)f(arra)o(y)f(is)i(passed,)f (either)h(all)h(or)d(none)i(of)f(the)g(elemen)o(ts)h(of)f(the)g(arra)o (y)75 2128 y(should)e(b)q(e)g(set)f(to)g Fj(MPI)p 493 2128 V 15 w(ST)l(A)l(TUS)p 673 2128 V 18 w(NULL)p Fq(.)1061 b Fm(?)16 b Fl(\(July\))75 2250 y Fg(9.4.2)49 b(Non-destructive)16 b(T)l(est)f(of)i(MPI)p 808 2250 15 2 v 18 w(Status)75 2335 y Fq(This)e(call)h(is)f(useful)h(for)e(accessing)i(the)e (information)h(asso)q(ciated)g(with)g(a)f(request,)h(without)f (deleting)75 2392 y(the)22 b(request)f(\(in)h(case)g(the)f(user)h(is)g (exp)q(ected)h(to)d(access)i(it)g(after)f(the)g(handler\).)40 b(It)21 b(allo)o(ws)h(one)75 2448 y(to)16 b(la)o(y)o(er)g(libraries)i (more)f(con)o(v)o(enien)o(tly)l(,)h(since)f(m)o(ultiple)i(la)o(y)o(ers) d(of)g(soft)o(w)o(are)f(ma)o(y)h(access)h(the)f(same)75 2505 y(completed)c(request)f(and)h(extract)e(from)h(it)g(the)h(status)e (information.)19 b(This)11 b(will)i(b)q(e)f(also)f(b)q(e)h(imp)q(ortan) o(t)75 2561 y(for)k(language)i(in)o(terop)q(erabilit)o(y)h(if)e(w)o(e)g (decide)i(that)d(status)g(ob)s(jects)h(cannot)f(b)q(e)i(transferred)f (across)75 2618 y(language)e(b)q(oundaries,)h(since)h(with)e(this)h (function)g(the)f(status)g(ob)s(ject)f(can)i(b)q(e)f(transferred)g (instead.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Fq(18)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(GET)p 264 45 V 17 w(ST)l(A)l(TUS\()16 b(request,)g(\015ag,)f(status\))117 122 y Ff(IN)155 b Fj(request)452 b Ff(an)14 b Fc(MPI)p 1040 122 13 2 v 14 w(REQUEST)f Ff(ob)r(ject)117 193 y(OUT)108 b Fj(\015ag)518 b Ff(\015ag,)13 b(same)g(as)h(from)e Fc(MPI)p 1325 193 V 15 w(TEST)117 264 y Ff(OUT)108 b Fj(status)476 b Fc(MPI)p 982 264 V 15 w(ST)m(A)m(TUS)12 b Ff(ob)r(ject)j(if)e(\015ag)g(is)h(true)75 389 y Fk(int)23 b(MPI)p 245 389 15 2 v 17 w(Get)p 334 389 V 17 w(status\()g(MPI)p 614 389 V 17 w(Request)g(request,)f(int)i(*flag,)f(MPI)p 1371 389 V 17 w(Status)g(*status\))75 475 y(MPI)p 150 475 V 17 w(GET)p 239 475 V 17 w(STATUS\()f(REQUEST,)h(FLAG,)g(STATUS,)g (IERROR\))170 532 y(INTEGER)g(REQUEST,)g(FLAG,)g(STATUS\(MPI)p 962 532 V 16 w(STATUS)p 1122 532 V 16 w(SIZE\),)h(IERROR)166 618 y Fq(Sets)18 b(\015ag=true)g(if)h(the)f(request)h(has)f(completed,) h(and,)g(if)g(so,)f(returns)g(in)h(status)f(the)g(request)75 675 y(status.)34 b(Ho)o(w)o(ev)o(er,)20 b(unlik)o(e)i(test)d(or)h(w)o (ait,)g(it)h(do)q(es)f(not)g(deallo)q(cate)h(or)f(inactiv)m(ate)h(the)f (request;)i(a)75 731 y(subsequen)o(t)16 b(call)g(to)f(test,)f(w)o(ait)h (or)f(free)i(should)g(b)q(e)g(executed)g(with)f(that)g(request.)75 870 y Fn(9.5)59 b(MPI-1.0)19 b(and)h(MPI-1.1)f(Issues)75 973 y Fg(9.5.1)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 973 V 18 w(INTERCOMM)p 936 973 V 20 w(CREA)l(TE)75 1059 y Fj(The)11 b(Problem:)44 b Fq(The)12 b(MPI)e(1.1)g(standard)h(sa) o(ys,)f(in)i(the)f(discussion)h(of)f Fj(MPI)p 1389 1059 14 2 v 16 w(INTERCOMM)p 1679 1059 V 16 w(CREA)l(TE)p Fq(,)75 1116 y(b)q(oth)k(that)189 1189 y(The)g(groups)g(m)o(ust)g(b)q (e)g(disjoin)o(t)75 1262 y(and)g(that)189 1335 y(The)g(leaders)h(ma)o (y)e(b)q(e)i(the)g(same)e(pro)q(cess.)75 1408 y(T)l(o)d(further)h(m)o (uddy)g(the)g(w)o(aters,)e(the)i(reason)f(giv)o(en)i(for)e("The)g (groups)g(m)o(ust)h(b)q(e)g(disjoin)o(t")g(is)g(based)g(on)75 1465 y(concerns)18 b(ab)q(out)g(the)g(implmen)o(tation)g(of)g Fj(MPI)p 922 1465 V 15 w(INTERCOMM)p 1211 1465 V 17 w(CREA)l(TE)h Fq(that)e(are)g(not)h(applicable)75 1521 y(for)d(the)g(case)g(where)h (the)f(leaders)h(are)f(the)g(same)g(pro)q(cess.)75 1637 y Fj(The)h(Fix:)45 b Fq(Delete)15 b(the)h(text:)189 1710 y(\(the)f(t)o(w)o(o)e(leaders)j(could)h(b)q(e)e(the)h(same)f(pro)q (cess\))75 1783 y(from)f(the)i(discussion)h(of)d Fj(MPI)p 610 1783 V 16 w(INTERCOMM)p 900 1783 V 17 w(CREA)l(TE)p Fq(.)166 1840 y(Replace)j(the)e(text:)189 1913 y(All)f(in)o(ter-comm)o (unicator)g(constructors)e(are)h(blo)q(c)o(king)h(and)g(require)g(that) e(the)i(lo)q(cal)g(and)189 1969 y(remote)g(groups)h(b)q(e)h(disjoin)o (t)g(in)g(order)f(to)f(a)o(v)o(oid)h(deadlo)q(c)o(k.)75 2043 y(with)189 2116 y(All)f(in)o(ter-comm)o(unicator)g(constructors)e (are)h(blo)q(c)o(king)h(and)g(require)g(that)e(the)i(lo)q(cal)g(and)189 2172 y(remote)g(groups)h(b)q(e)h(disjoin)o(t.)189 2243 y(Advice)21 b(to)e(users:)29 b(The)20 b(groups)f(m)o(ust)g(b)q(e)h (disjoin)o(t)h(for)e(sev)o(eral)h(reasons.)33 b(Primarily)l(,)189 2300 y(this)19 b(is)h(the)f(in)o(ten)o(t)g(of)g(the)g(in)o(tercomm)o (unicators)g({)g(to)g(pro)o(vide)g(a)g(comm)o(unicator)g(for)189 2356 y(comm)o(unication)h(b)q(et)o(w)o(een)g(disjoin)o(t)g(groups.)33 b(This)20 b(is)g(re\015ected)g(in)h(the)e(de\014nition)j(of)189 2412 y Fj(MPI)p 274 2412 V 15 w(INTERCOMM)p 563 2412 V 17 w(MERGE)p Fq(,)15 b(whic)o(h)h(allo)o(ws)f(the)g(user)f(to)h(con)o (trol)f(the)h(ranking)g(of)f(the)189 2469 y(pro)q(cesses)19 b(in)h(the)f(created)g(in)o(tracomm)o(unicator;)g(this)h(ranking)f(mak) o(es)f(little)i(sense)g(if)189 2525 y(the)i(groups)f(are)h(not)g (disjoin)o(t.)41 b(In)22 b(addition,)j(the)d(natural)g(extension)h(of)f (collectiv)o(e)189 2582 y(op)q(erations)13 b(to)g(in)o(tercomm)o (unicators)h(\(b)q(eing)g(considered)h(for)e(MPI-2\))g(mak)o(es)g(the)h (most)189 2638 y(sense)h(when)g(the)g(groups)g(are)f(disjoin)o(t.)21 b(It)15 b(is)g(the)g(in)o(ten)o(t)g(of)f(the)h(MPI)g(de\014nition)i (not)d(to)189 2695 y(preclude)j(future)e(extensions.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fh(9.5.)34 b(MPI-1.0)14 b(AND)i(MPI-1.1)e(ISSUES) 1020 b Fq(19)75 45 y Fg(9.5.2)49 b(Erro)o(r)15 b(Class)i(fo)o(r)f (Invalid)i(Keyval)75 131 y Fq(There)d(is)h(a)f(new)g(MPI)h(error)e (class:)20 b Fj(MPI)p 808 131 14 2 v 16 w(ERR)p 909 131 V 17 w(KEYV)l(AL)166 235 y Fd(Discussion:)43 b Ff(A)o(t)16 b(the)h(momen)o(t)c(the)k(b)q(est)g(w)o(e)g(can)f(do)g(with)g(an)g(in)o (v)n(alid)e(k)o(eyv)n(al)h(argumen)o(t)g(is)h(return)75 291 y Fc(MPI)p 152 291 13 2 v 14 w(ERR)p 245 291 V 15 w(ARG)p Ff(,)g(or)h(p)q(erhaps)h Fc(MPI)p 659 291 V 15 w(ERR)p 753 291 V 14 w(OTHER)p Ff(.)f(W)m(e're)g(free)h(to)f(de\014ne)h (whatev)o(er)g(extra)g(error)g(co)q(des)h(w)o(e)75 348 y(w)o(an)o(t)12 b(to)h(in)f(an)h(implemen)o(tatio)o(n,)d(but)j(this)g (one)g(seems)g(ob)o(vious;)f(it's)g(prett)o(y)i(m)o(uc)o(h)d(the)j (only)e(sp)q(eci\014c)i(kind)f(of)75 404 y(MPI)h(argumen)o(t)f(that)h (do)q(esn't)g(curren)o(tly)h(ha)o(v)o(e)f(its)g(o)o(wn)f(class.)75 573 y Fg(9.5.3)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 573 15 2 v 18 w(FINALIZE)75 659 y Fq(This)f(routine)f(cleans)h(up)f (all)h(MPI)f(state.)k(Eac)o(h)c(pro)q(cess)g(m)o(ust)g(call)h Fj(MPI)p 1371 659 14 2 v 16 w(FINALIZE)e Fq(b)q(efore)h(it)g(exits.)75 715 y(Once)e(this)g(routine)f(is)h(called,)h(no)e(MPI)h(routine)f(\(ev) o(en)g(not)g Fj(MPI)p 1195 715 V 16 w(INIT)p Fq(\))f(ma)o(y)h(b)q(e)h (called.)20 b(Eac)o(h)11 b(pro)q(cess)75 772 y(m)o(ust)i(complete)i(an) o(y)f(p)q(ending)i(comm)o(unication)e(it)h(initiated)g(b)q(efore)f(it)h (calls)g Fj(MPI)p 1539 772 V 16 w(FINALIZE)p Fq(.)d(After)75 828 y(the)j(call)g(returns,)g(eac)o(h)f(pro)q(cess)h(ma)o(y)f(con)o (tin)o(ue)h(lo)q(cal)h(computations,)e(or)g(exit,)h(without)f(in)o (terfering)75 885 y(with)i(other)e(pro)q(cesses.)21 b Fj(MPI)p 597 885 V 15 w(FINALIZE)15 b Fq(is)g(collectiv)o(e)i(on)f Fj(MPI)p 1209 885 V 15 w(COMM)p 1366 885 V 17 w(W)o(ORLD)p Fq(.)166 988 y Fd(Discussion:)54 b Ff(Need)20 b(to)e(discuss)i(whether) g(\014nalize)f(should,)h(indeed,)g(b)q(e)f(collectiv)o(e,)h(and)e(ho)o (w)h(this)75 1045 y(extends)14 b(to)e(dynamic)e(pro)q(cesses.)20 b(F)m(or)12 b(dynamic)e(pro)q(cesses,)15 b(w)o(e)d(ma)o(y)f(w)o(an)o(t) g(\014nalize)h(to)g(b)q(e)h(collectiv)o(e)f(o)o(v)o(er)g(the)75 1101 y(connected)k(comp)q(onen)o(t)d(including)g(the)h(calling)f(pro)q (cess.)166 1252 y Fd(Discussion:)34 b Ff(Should)13 b(a)g(program)f (that)i(do)q(es)g(not)g(complete)f(its)g(outstanding)h(op)q(erations)g (b)q(e)g(consid-)75 1309 y(ered)j(erroneous,)h(or)e(is)g(it)g Fc(MPI)p 587 1309 13 2 v 15 w(FINALIZE)p Ff('s)f(resp)q(onsibilit)o(y)h (to)g(complete)g(all)f(op)q(erations?)25 b(Do)16 b(w)o(e)g(w)o(an)o(t)g (to)75 1365 y(encourage)f(programmer)c(reliance)k(on)e(this)h (functionalit)o(y?)189 1519 y Fi(A)n(dvic)n(e)d(to)i(implementors.)38 b Fq(Ev)o(en)11 b(though)g(a)g(pro)q(cess)h(has)f(completed)i(all)f (the)f(comm)o(unication)189 1575 y(it)j(initiated,)h(suc)o(h)g(comm)o (unication)f(ma)o(y)f(not)h(y)o(et)g(b)q(e)g(completed)h(from)e(the)h (viewp)q(oin)o(t)h(of)f(the)189 1631 y(underlying)j(MPI)f(system.)k (E.g.,)14 b(a)h(blo)q(c)o(king)i(send)f(ma)o(y)f(ha)o(v)o(e)g (completed,)i(ev)o(en)e(though)h(the)189 1688 y(data)k(is)h(still)h (bu\013ered)f(at)f(the)h(sender.)37 b(The)20 b(MPI)h(implemen)o(tation) h(m)o(ust)e(ensure)h(that)f(a)189 1744 y(pro)q(cess)14 b(has)g(completed)h(an)o(y)f(in)o(v)o(olv)o(emen)o(t)g(in)h(MPI)f(comm) o(unication)h(b)q(efore)f Fj(MPI)p 1669 1744 14 2 v 16 w(FINALIZE)189 1801 y Fq(returns.)26 b(Th)o(us,)17 b(if)h(a)e(pro)q (cess)i(exits)g(after)e(the)h(call)i(to)d Fj(MPI)p 1271 1801 V 16 w(FINALIZE)p Fq(,)g(this)i(will)h(not)e(cause)189 1857 y(an)e(ongoing)g(comm)o(unication)h(to)e(fail.)21 b(\()p Fi(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Fq(\))75 1979 y Fg(9.5.4)49 b(MPI)p 313 1979 15 2 v 18 w(UB)1875 2008 y Fm(>)16 b Fl(\(July\))75 2065 y Fq(This)g(section)g (needs)g(to)e(b)q(e)i(\015eshed)g(out,)f(probably)g(b)o(y)h(Bill)h (Gropp.)1875 2067 y Fm(?)f Fl(\(July\))166 2121 y Fq(The)e(problem)h (with)f Fj(MPI)p 621 2121 14 2 v 16 w(UB)h Fq(is)f(that)f(it)i(o)o(v)o (errides)f(ev)o(erything)g(else.)21 b(F)l(or)13 b(example,)i(if)f(y)o (ou)g(put)75 2178 y(an)j Fj(MPI)p 225 2178 V 15 w(UB)h Fq(in)f(a)g(struct)f(t)o(yp)q(e)g(and)h(then)g(use)g(that)f(new)h(t)o (yp)q(e)g(in)h(another)e(t)o(yp)q(e)h(\(e.g.)23 b(a)16 b(v)o(ector)g(or)75 2234 y(another)c(struct)h(or)f(ev)o(en)h(con)o (tiguous\),)g(then)g(that)f Fj(MPI)p 1060 2234 V 15 w(UB)i Fq(o)o(v)o(errides)e(the)h("natural")f(and)h(probably)75 2291 y(in)o(tended)23 b(form)d(of)h(the)g(datat)o(yp)q(e.)38 b(W)l(e)21 b(need)h(an)g Fj(MPI)p 1100 2291 V 15 w(UB)p 1176 2291 V 17 w(SOFT)g Fq(or)f(something)g(that)g(sets)g(the)75 2347 y(\\stride")15 b(for)g(that)f(datat)o(yp)q(e)h(but)g(not)g(for)g (the)g(datat)o(yp)q(es)f(that)h(con)o(tain)g(that)g(datat)o(yp)q(e.)75 2469 y Fg(9.5.5)49 b(Cla)o(ri\014cation)18 b(of)e(Binding)i(of)f(MPI)p 854 2469 15 2 v 18 w(T)l(yp)q(e)p 971 2469 V 18 w(size)75 2555 y Fq(This)i(clari\014cation)g(is)g(needed)g(in)g(the)f(MPI-1)g (description)i(of)e Fj(MPI)p 1293 2555 14 2 v 15 w(T)l(yp)q(e)p 1400 2555 V 18 w(size)p Fq(,)g(since)i(the)e(issue)h(re-)75 2611 y(p)q(eatedly)d(arises.)k(It)c(is)f(a)g(clari\014cation)i(of)e (the)g(binding.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Fq(20)1138 b Fh(CHAPTER)15 b(9.)30 b(MISCELLANY)189 45 y Fi(A)n(dvic)n(e)20 b(to)j(users.)76 b Fq(The)21 b(binding)i(of)e Fj(MPI)p 1004 45 14 2 v 16 w(T)l(yp)q(e)p 1112 45 V 17 w(size)p Fq(,)i(sp)q(eci\014cally)l(,)i (whether)d(the)f(output)189 102 y(argumen)o(t)e(in)i(C)g(is)f(of)g(t)o (yp)q(e)h Fk(int)f Fq(as)g(in)h(the)f(curren)o(t)h(standard)f(or)f (should)j(b)q(e)f(c)o(hanged)g(to)189 158 y(something)14 b(else,)i(has)e(b)q(een)i(extensiv)o(ely)f(discussed)h(b)o(y)f(the)g (MPI)f(F)l(orum.)19 b(The)c(\014nal)g(decision)189 214 y(is)g(to)g(lea)o(v)o(e)g(it)h(an)f Fk(int)p Fq(.)k(\()p Fi(End)d(of)g(advic)n(e)g(to)h(users.)p Fq(\))-117 275 y Fm(>)e Fl(\(July\))75 386 y Fg(9.5.6)49 b(MPI)p 313 386 15 2 v 18 w(REDUCE)75 471 y Fq(The)15 b(curren)o(t)h(text)e(on)h (p.)20 b(115)15 b(lines)i(25-28)d(from)g(MPI-1.1)h(\(June)h(12,)e (1995\))f(sa)o(ys:)166 528 y(The)j Fj(datat)o(yp)q(e)i Fq(argumen)o(t)e(of)f Fj(MPI)p 783 528 14 2 v 16 w(REDUCE)i Fq(m)o(ust)f(b)q(e)g(compatible)i(with)e Fj(op)p Fq(.)23 b(Prede\014ned)18 b(op-)75 584 y(erators)h(w)o(ork)g(only)i(with)f(the) g Fj(MPI)g Fq(t)o(yp)q(es)g(listed)h(in)g(Sec.)35 b(4.9.2)18 b(and)j(Sec.)35 b(4.9.3.)d(User-de\014ned)75 641 y(op)q(erators)14 b(ma)o(y)h(op)q(erate)g(on)g(general,)g(deriv)o(ed)h(datat)o(yp)q(es.) 166 697 y(This)g(text)e(is)i(c)o(hanged)g(to:)166 754 y(The)g Fj(datat)o(yp)q(e)i Fq(argumen)o(t)e(of)f Fj(MPI)p 783 754 V 16 w(REDUCE)i Fq(m)o(ust)f(b)q(e)g(compatible)i(with)e Fj(op)p Fq(.)23 b(Prede\014ned)18 b(op-)75 810 y(erators)c(w)o(ork)h (only)h(with)f(the)h Fj(MPI)f Fq(t)o(yp)q(es)g(listed)i(in)f(Sec.)21 b(4.9.2)14 b(and)h(Sec.)21 b(4.9.3.)e(F)l(urthermore,)c(the)75 867 y Fj(datat)o(yp)q(e)20 b Fq(and)e Fj(op)h Fq(giv)o(en)f(for)g (prede\014ned)i(op)q(erators)d(m)o(ust)h(b)q(e)h(the)f(same)g(on)g(all) h(pro)q(cesses.)30 b(User-)75 923 y(de\014ned)15 b(op)q(erators)e(ma)o (y)g(op)q(erate)g(on)h(general,)g(deriv)o(ed)h(datat)o(yp)q(es)e (without)g(the)h(restrictions)g(ab)q(o)o(v)o(e)75 979 y(for)h(prede\014ned)h(op)q(erators.)-681 b Fm(?)15 b Fl(\(July\))75 1101 y Fg(9.5.7)49 b(Prob)q(e)75 1187 y Fq(The)20 b(seman)o(tics)g(of)f Fj(MPI)p 525 1187 V 16 w(PROBE)i Fq(and)f Fj(MPI)p 885 1187 V 16 w(IPROBE)h Fq(in)f Fj(MPI-1)f Fq(is)i(inconsisten)o(t)f(with)g(the)g(other)75 1243 y(uses)15 b(of)f(the)g(\\I")g(pre\014x.)21 b(The)14 b(follo)o(wing)h(new)g(functions)g(are)f(prop)q(osed,)h(so)f(that)g (one)g(can)h(w)o(ait)f(on)g(a)75 1300 y(request)h(returned)h(b)o(y)f (an)g(\\IPR)o(OBE":)170 1393 y Fk(try_recv)47 b(=)h(probe)147 1450 y(itry_recv)e(=)i(iprobe)23 b(returning)f(a)i(request.)75 1543 y Fq(It)15 b(is)h(not)f(prop)q(osed)g(to)g(c)o(hange)g(the)g (existing)i Fj(PROBE)f Fq(functions.)75 1665 y Fg(9.5.8)49 b(Mino)o(r)17 b(Co)o(rrections)75 1751 y Fq(These)f(are)e(here)i(temp)q (orarily)g(so)f(that)f(w)o(e)h(don't)g(lose)g(them.)20 b(\(F)l(rom)14 b(Mic)o(hael)i(Hennec)o(k)o(e\))75 1857 y Fk(Some)23 b(errors)g(in)h(the)f(MPI)h(Spec)f(v1.1:)75 1970 y(195:48)47 b("HANDLER,")f(->)i("ERRHANDLER,")75 2026 y(197:)23 b(2)48 b(shouldn't)23 b(errhandler)f(have)h(intent)g (INOUT)h(?)75 2083 y(204:)f(1)48 b("level")f(->)g("LEVEL")75 2139 y(223:34)g("INTEGER)23 b(REQUEST,")46 b(->)i("INTEGER")75 2195 y(223:38)f("INTEGER)23 b(REQUEST,")46 b(->)i("INTEGER")75 2252 y(228:19)f("INTRACOMM")f(->)i("NEWINTRACOMM")75 2308 y(228:20)f("INTRACOMM")f(->)i("NEWINTRACOMM")75 2365 y(229:46)f("HANDLER,")f(->)i("ERRHANDLER,")75 2421 y(230:33)f("level")g(->)g("LEVEL")75 2534 y(Some)23 b(minor)g(Typos:) 123 2647 y(9:33)47 b("begining")f(->)i("beginning")99 2704 y(17:)23 b(8)48 b("string)23 b({\\bf)g(message}")47 b(->)g("variable)23 b({\\bf)g(message}")g(\(see)g(16:46\))1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fh(9.5.)34 b(MPI-1.0)14 b(AND)i(MPI-1.1)e(ISSUES) 1020 b Fq(21)99 45 y Fk(19:)23 b(2)48 b("longlong")e(->)i("long)23 b(long")99 102 y(21:23)47 b("intercommunicators")e(->)i ("inter-communicators")1875 208 y Fm(>)16 b Fl(\(July\))166 264 y Fq(The)h(follo)o(wing)i(corrections)e(to)g Fj(MPI-1.1)f Fq(are)h(\(all)h(page)f(and)g(line)i(n)o(um)o(b)q(ers)f(are)f(for)g (the)g(June)75 321 y(12,)d(1995)g(v)o(ersion)i(without)f(c)o (hangebars\):)143 427 y Fm(\017)23 b Fq(P)o(age)14 b(19,)g(lines)j(1-2) e(reads)189 483 y(for)f(\(64)g(bit\))i(C)f(in)o(tegers)g(declared)i(to) d(b)q(e)i(of)f(t)o(yp)q(e)g Fj(longlong)g(int)189 540 y Fq(but)g(should)h(read)189 596 y(for)e(\(64)g(bit\))i(C)f(in)o (tegers)g(declared)i(to)d(b)q(e)i(of)f(t)o(yp)q(e)g Fj(long)g(long)g (int)189 769 y Fd(Discussion)o(:)189 837 y Ff(Just)f(to)g(mak)o(e)f(it) g(correct)j(C)d(so)h(it)g(is)g(long)f(long.)143 978 y Fm(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(3)e(reads)189 1035 y(MPI)p 281 1035 14 2 v 16 w(P)o(ac)o(k)p 393 1035 V 16 w(size\(coun)o(t,)g(MPI)p 724 1035 V 16 w(CHAR,)h(&k2\);)189 1091 y(but)f(should)h(read)189 1147 y(MPI)p 281 1147 V 16 w(P)o(ac)o(k)p 393 1147 V 16 w(size\(coun)o(t,)f(MPI)p 724 1147 V 16 w(CHAR,)h(comm,)e(&k2\);)189 1320 y Fd(Discussion)o(:)189 1388 y Ff(This)f(is)h(a)g(minor)e(\014x)i(to)f(an)h(example)f(for)g(a)h (missing)e(argumen)o(t.)143 1529 y Fm(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(10)e(reads)189 1586 y(MPI)p 281 1586 V 16 w(P)o(ac)o(k\(c)o(hr,)f(coun)o(t,)g(MPI)p 726 1586 V 17 w(CHAR,)h(&lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 1642 y(but)f(should)h(read)189 1699 y(MPI)p 281 1699 V 16 w(P)o(ac)o(k\(c)o(hr,)e(coun)o(t,)g(MPI)p 726 1699 V 17 w(CHAR,)h(lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 1871 y Fd(Discussion)o(:)189 1939 y Ff(This)e(is)h(a)f(minor)f(\014x)i (to)g(an)f(example)g(for)g(an)g(argumen)o(t)g(that)h(is)g(declared)g (to)g(b)q(e)g(a)g(p)q(oin)o(ter)g(and)f(then)189 1989 y(passed)h(with)e(&.)143 2130 y Fm(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(10)e(reads)189 2187 y(MPI)p 281 2187 V 16 w(P)o(ac)o(k\(c)o(hr,)f(coun)o(t,)g(MPI)p 726 2187 V 17 w(CHAR,)h(&lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 2243 y(but)f(should)h(read)189 2300 y(MPI)p 281 2300 V 16 w(P)o(ac)o(k\(c)o(hr,)e(coun)o(t,)g(MPI)p 726 2300 V 17 w(CHAR,)h(lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 2472 y Fd(Discussion)o(:)189 2540 y Ff(This)e(is)h(a)f(minor)f(\014x)i (to)g(an)f(example)g(for)g(an)g(argumen)o(t)g(that)h(is)g(declared)g (to)g(b)q(e)g(a)g(p)q(oin)o(ter)g(and)f(then)189 2590 y(passed)h(with)e(&.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 -100 a Fq(22)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)143 45 y Fm(\017)23 b Fq(P)o(age)14 b(125,)g(line)j(1)e (reads)189 102 y(CALL)h(MPI)p 420 102 14 2 v 16 w(ALLREDUCE\(sum,)f(c,) g(n,)g(MPI)p 1040 102 V 17 w(REAL,)g(MPI)p 1300 102 V 17 w(SUM,)g(0,)f(comm,)g(ierr\))189 158 y(but)h(should)h(read)189 214 y(CALL)g(MPI)p 420 214 V 16 w(ALLREDUCE\(sum,)f(c,)g(n,)g(MPI)p 1040 214 V 17 w(REAL,)g(MPI)p 1300 214 V 17 w(SUM,)g(comm,)f(ierr\))189 387 y Fd(Discussion)o(:)189 455 y Ff(The)g(extra)h(argumen)o(t)e(app)q (ears)j(to)e(b)q(e)h(left)f(o)o(v)o(er)g(from)f(the)i(previous)f (example)f(with)h(MPI)p 1670 455 13 2 v 16 w(REDUCE.)189 505 y(This)f(is)h(a)g(mistak)o(e.)143 646 y Fm(\017)23 b Fq(P)o(age)14 b(192,)g(lines)j(1-2)e(reads)204 702 y Fj(MPI)p 289 702 14 2 v 16 w(ERRHANDLER)p 598 702 V 17 w(FREE\()h(errhandler)f(\))230 893 y Ff(IN)156 b Fj(errhandler)397 b Ff(MPI)14 b(error)h(handler)f(\(handle\))189 1036 y Fq(but)h(should)h(read)204 1092 y Fj(MPI)p 289 1092 V 16 w(ERRHANDLER)p 598 1092 V 17 w(FREE\()g(errhandler)f(\))230 1283 y Ff(INOUT)63 b Fj(errhandler)397 b Ff(MPI)14 b(error)h(handler)f (\(handle\))189 1467 y Fd(Discussion)o(:)189 1535 y Ff(This)c(app)q (ears)h(to)f(ha)o(v)o(e)h(b)q(een)g(a)f(mistak)o(e.)16 b(This)10 b(function)g(frees)i(the)f(function)f(p)q(oin)o(ter)g(and)h (sets)g(its)g(v)n(alue)189 1585 y(to)h(MPI)p 321 1585 13 2 v 15 w(ERRHANDLER)p 636 1585 V 14 w(NULL.)g(Th)o(us,)h(it)f(is)g (b)q(oth)h(input)f(and)g(output)h(as)f(are)h(the)g(free)g(routines)g (for)189 1635 y(Groups,)g(Comms,)e(etc.)143 1776 y Fm(\017)23 b Fq(P)o(age)14 b(203,)g(line)j(1)e(reads)189 1832 y Fj(MPI)p 274 1832 14 2 v 15 w(PCONTROL\(level\))189 1889 y Fq(but)g(should)h(read)189 1945 y Fj(MPI)p 274 1945 V 15 w(PCONTROL\(LEVEL\))189 2117 y Fd(Discussion)o(:)189 2186 y Ff(This)d(is)h(a)g(minor)e(t)o(yp)q(o.)143 2327 y Fm(\017)23 b Fq(P)o(age)14 b(210,)g(line)j(43)e(reads)189 2383 y Fc(MPI)p 266 2383 13 2 v 14 w(PENDING)189 2440 y Fq(but)g(should)h(read)189 2496 y Fc(MPI)p 266 2496 V 14 w(ERR)p 359 2496 V 14 w(PENDING)189 2668 y Fd(Discussion)o(:)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 23 24 bop 75 -100 a Fh(9.5.)34 b(MPI-1.0)14 b(AND)i(MPI-1.1)e(ISSUES) 1020 b Fq(23)189 45 y Ff(This)13 b(is)h(a)g(t)o(yp)q(o.)k(See)c(page)g (197,)f(line)g(20)h(for)f(the)i(original)d(de\014nition.)143 184 y Fm(\017)23 b Fq(P)o(age)14 b(229,)g(lines)j(19-20)d(reads)189 240 y Fj(int)i(double)g(MPI)p 479 240 14 2 v 16 w(Wtime\(void\))189 297 y(int)g(double)g(MPI)p 479 297 V 16 w(Wtick\(void\))189 353 y Fq(but)f(should)h(read)189 410 y Fj(double)g(MPI)p 413 410 V 16 w(Wtime\(void\))189 466 y(double)g(MPI)p 413 466 V 16 w(Wtick\(void\))189 637 y Fd(Discussion)o(:)189 705 y Ff(This)d(w)o(as)h(a)g(mistak)o(e)e(caused)j(b)o(y)f(the)g (macros.)143 844 y Fm(\017)23 b Fq(P)o(age)14 b(222,)g(line)j(34)e (reads)189 900 y Fj(INTEGER)f(REQUEST,)g(COUNT,)g(D)o(A)l(T)l(A)l (TYPE,)g(DEST,)g(T)l(A)o(G,)f(COMM,)g(REQUEST,)i(IERROR)189 957 y Fq(but)g(should)h(read)189 1013 y Fj(INTEGER)f(COUNT,)h(D)o(A)l (T)l(A)l(TYPE,)g(DEST,)f(T)l(A)o(G,)g(COMM,)g(REQUEST,)h(IERROR)189 1184 y Fd(Discussion)o(:)189 1252 y Ff(An)e(extra)g(argumen)o(t)f(got)g (added.)19 b(It)14 b(needs)h(to)f(b)q(e)g(remo)o(v)o(ed.)143 1391 y Fm(\017)23 b Fq(P)o(age)14 b(222,)g(line)j(38)e(reads)189 1447 y Fj(INTEGER)f(REQUEST,)g(COUNT,)g(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)g (T)l(A)o(G,)f(COMM,)g(REQUEST,)i(IERROR)189 1504 y Fq(but)g(should)h (read)189 1560 y Fj(INTEGER)f(COUNT,)h(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)f (T)l(A)o(G,)g(COMM,)g(REQUEST,)h(IERROR)189 1731 y Fd(Discussion)o(:) 189 1799 y Ff(An)e(extra)g(argumen)o(t)f(got)g(added.)19 b(It)14 b(needs)h(to)f(b)q(e)g(remo)o(v)o(ed.)143 1938 y Fm(\017)23 b Fq(P)o(age)14 b(227,)g(lines)j(19-20)d(reads)189 1994 y Fj(MPI)p 274 1994 V 15 w(INTERCOMM)p 563 1994 V 17 w(MERGE\(INTERCOMM,)i(HIGH,)f(INTRA)o(COMM,)g(IERROR\))189 2050 y(INTEGER)g(INTERCOMM,)h(INTRA)o(COMM,)f(IERROR)189 2107 y Fq(but)g(should)h(read)189 2163 y Fj(MPI)p 274 2163 V 15 w(INTERCOMM)p 563 2163 V 17 w(MERGE\(INTERCOMM,)g(HIGH,)f (NEWINTRA)o(COMM,)g(IERROR\))189 2220 y(INTEGER)g(INTERCOMM,)h (NEWINTRA)o(COMM,)f(IERROR)189 2391 y Fd(Discussion)o(:)189 2458 y Ff(This)g(mak)o(es)e(it)i(matc)o(h)f(the)i(de\014nition)e(in)h (the)g(text)h(on)f(P)o(age)g(159,)f(lines)h(54-56.)20 b(It)15 b(only)g(c)o(hanges)g(the)189 2508 y(name)d(of)i(the)g(argumen) o(t.)143 2647 y Fm(\017)23 b Fq(P)o(age)14 b(228,)g(line)j(46)e(reads) 189 2704 y Fj(MPI)p 274 2704 V 15 w(ERRHANDLER)p 582 2704 V 18 w(CREA)l(TE\(FUNCTION,)h(HANDLER,)g(IERROR\))-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 24 25 bop 75 -100 a Fq(24)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)189 45 y Fq(but)15 b(should)h(read)189 102 y Fj(MPI)p 274 102 14 2 v 15 w(ERRHANDLER)p 582 102 V 18 w(CREA)l(TE\(FUNCTION,)g(ERRHANDLER,)h(IERROR\))189 274 y Fd(Discussion)o(:)189 342 y Ff(This)c(is)h(a)g(small)d(t)o(yp)q (o)j(to)g(matc)o(h)f(all)f(the)j(other)f(names)f(for)h(this)g(argumen)o (t.)143 483 y Fm(\017)23 b Fq(P)o(age)14 b(229,)g(line)j(33)e(reads)189 540 y Fj(MPI)p 274 540 V 15 w(PCONTROL\(level\))189 596 y Fq(but)g(should)h(read)189 653 y Fj(MPI)p 274 653 V 15 w(PCONTROL\(LEVEL\))189 825 y Fd(Discussion)o(:)189 893 y Ff(This)d(is)h(to)g(mak)o(e)e(the)j(language)e(binding)g(the)h (same)f(as)h(the)h(c)o(hange)f(to)g(P)o(age)f(203,)g(line)g(1.)75 1047 y Fk(The)23 b(definition)g(of)h(MPI_ALLGATHER)e(in)h(the)h(MPI)f (document)g(says:)75 1160 y("The)g(jth)h(block)f(of)h(data)f(sent)g (from)g(each)h(proess)f(is)g(received)g(by)h(every)75 1216 y(process)f(and)g(placed)g(in)h(the)f(jth)h(block)f(of)h(the)f (buffer)g(recvbuf.")75 1329 y(This)g(is)h(confusing.)46 b(It)24 b(should)f(be)75 1442 y("The)g(block)g(of)h(data)f(sent)h(from) f(the)g(jth)h(process)f(is)g(received)g(by)h(every)75 1499 y(process)f(and)g(placed)g(in)h(the)f(jth)h(block)f(of)h(the)f (buffer)g(recvbuf.")75 1611 y(Same)g(for)h(MPI_ALLGATHERV.)-117 1718 y Fm(?)15 b Fl(\(July\))75 1861 y Fn(9.6)59 b(V)n(ersion)20 b(Numb)r(er)75 1962 y Fq(In)c(order)f(to)g(cop)q(e)h(with)g(c)o(hanges) f(to)g(the)g(MPI)h(Standard,)f(there)g(are)g(b)q(oth)h(compile-time)h (and)f(run-)75 2019 y(time)h(w)o(a)o(ys)f(to)h(determine)h(whic)o(h)g (v)o(ersion)f(of)g(the)g(standard)g(is)g(in)h(use)g(in)g(the)f(en)o (vironmen)o(t)g(one)g(is)75 2075 y(using.)166 2132 y(The)d(\\v)o (ersion")g(will)h(b)q(e)g(represen)o(ted)f(b)o(y)g(t)o(w)o(o)e (separate)i(in)o(tegers,)g(for)f(the)h(v)o(ersion)g(and)g(sub)o(v)o (er-)75 2188 y(sion:)166 2245 y(In)i(C,)170 2338 y Fk(#define)23 b(MPI_VERSION)94 b(1)170 2395 y(#define)23 b(MPI_SUBVERSION)f(2)75 2489 y Fq(and)15 b(in)h(F)l(ortran,)170 2583 y Fk(INTEGER)23 b(MPI_VERSION,)g(MPI_SUBVERSION)170 2639 y(PARAMETER)g(\(MPI_VERSION)94 b(=)24 b(1\))170 2695 y(PARAMETER)f(\(MPI_SUBVERSION)f(=)i(2\))1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 25 26 bop 75 -100 a Fh(9.7.)34 b(MPI)15 b(AND)g(THREADS)1202 b Fq(25)75 45 y(F)l(or)15 b(run)o(time)g(determination,)75 196 y Fj(MPI)p 160 196 14 2 v 16 w(VERSION\()h(version,)f(subversion)i (\))117 273 y Ff(OUT)108 b Fj(version)456 b Ff(v)o(ersion)14 b(n)o(um)o(b)q(er)117 347 y(OUT)108 b Fj(subversion)393 b Ff(sub)o(v)o(ersion)15 b(n)o(um)o(b)q(er)75 472 y Fk(int)23 b(MPI)p 245 472 15 2 v 17 w(Version\()g(int)g(*version,)g(int)h (*subversion)e(\))75 558 y(MPI)p 150 558 V 17 w(VERSION\()h(VERSION,)f (SUBVERSION,)h(IERROR)g(\))170 614 y(INTEGER)g(VERSION,)g(SUBVERSION,)f (IERROR)166 701 y Fq(This)11 b(function)g(is)f(one)h(of)e(the)i(few)f (that)f(can)h(b)q(e)h(called)h(b)q(efore)e Fj(MPI)p 1322 701 14 2 v 16 w(INIT)g Fq(or)f(after)h Fj(MPI)p 1672 701 V 15 w(FINALIZE)p Fq(.)-16 b Fm(>)16 b Fl(\(July\))75 843 y Fn(9.7)59 b(MPI)20 b(and)g(Threads)75 944 y Fq(This)f(section)f (sp)q(eci\014es)i(the)e(in)o(teraction)g(b)q(et)o(w)o(een)g(MPI)g (calls)h(and)f(P)o(osix)g(threads.)28 b(MPI)18 b(ma)o(y)f(b)q(e)75 1001 y(implemen)o(ted)g(in)g(en)o(vironmen)o(ts)f(where)g(threads)f (are)h(not)f(supp)q(orted,)h(or)f(p)q(erform)g(p)q(o)q(orly)l(.)23 b(There-)75 1057 y(fore,)c(it)h(is)g(not)f(required)h(that)e(all)j(MPI) e(implemen)o(tations)h(b)q(eha)o(v)o(e)g(as)f(sp)q(eci\014ed)i(in)f (this)g(section.)75 1114 y(The)15 b(requiremen)o(ts)h(here)g(apply)g (to)e Fe(thread-complian)o(t)k Fq(implemen)o(tations.)166 1217 y Fd(Discussion:)42 b Ff(The)17 b(design)f(is)g(lik)o(ely)e(to)i (b)q(e)h(compatible)d(with)i(non)g(P)o(osix)f(thread)i(pac)o(k)n(ages)f (as)g(w)o(ell,)75 1274 y(but)e(I)g(ha)o(v)o(e)g(not)f(c)o(hec)o(k)o(ed) j(NT)e(threads)h(or)e(threads)i(on)f(other)g(systems.)75 1442 y Fg(9.7.1)49 b(General)75 1527 y Fq(A)15 b(prede\014ned)i (attribute)f(k)o(ey)f Fc(MPI)p 696 1527 13 2 v 14 w(THREADS)h Fq(is)f(asso)q(ciated)h(with)g Fj(MPI)p 1365 1527 14 2 v 16 w(COMM)p 1523 1527 V 16 w(W)o(ORLD)p Fq(.)f(The)g(at-)75 1584 y(tribute)h(is)f(set)g(if)h(the)f(implmen)o(tation)i(is)e(thread)g (complian)o(t.)166 1688 y Fd(Discussion:)50 b Ff(W)m(e)17 b(ma)o(y)f(use)j(the)f(attribute)g(v)n(alue)f(to)h(pro)o(vide)g (additional)e(information)f(on)i(thread)75 1744 y(supp)q(ort,)d(or)g (use)h(additional)d(attribute)i(k)o(eys)h(for)e(that)h(purp)q(ose.)166 1848 y Fq(All)19 b(MPI)f(calls)h(should)g(b)q(e)g Fi(thr)n(e)n(ad-safe) p Fq(.)29 b(I.e.,)18 b(t)o(w)o(o)f(concurren)o(tly)h(running)i(threads) d(ma)o(y)h(call)75 1904 y(MPI)f(calls)i(and)e(the)h(outcome)f(should)h (b)q(e)g(as)f(if)h(the)g(calls)g(executed)g(in)h(some)e(order,)g(ev)o (en)h(if)f(their)75 1961 y(execution)f(is)g(in)o(terlea)o(v)o(ed.)189 2058 y Fi(A)n(dvic)n(e)g(to)i(implementors.)49 b Fq(MPI)17 b(calls)h(can)f(b)q(e)g(made)g(thread-safe)f(b)o(y)h(executing)h(only)f (one)189 2114 y(at)c(a)h(time,)h(e.g.,)e(b)o(y)h(protecting)g(MPI)h(co) q(de)f(with)h(one)f(global)h(lo)q(c)o(k.)20 b(High-qualit)o(y)c (implemen-)189 2171 y(tations)j(will)j(pro)o(vide)f(more)e(concurrency) i(in)g(MPI)f(execution,)i(using)f(\014ner)f(grain)g(lo)q(c)o(king.)189 2227 y(Concurrency)d(can)h(also)f(b)q(e)h(ac)o(hiev)o(ed)h(b)o(y)e(ha)o (ving)h(some)f(of)g(the)g(MPI)g(proto)q(col)h(executed)g(b)o(y)189 2284 y(separate)c(serv)o(er)h(threads.)20 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))166 2381 y(Blo)q(c)o(king)j(MPI)e(calls)i(will)g(blo)q(c)o(k)g(the)e(calling)j (thread)d(only)l(,)i(allo)o(wing)f(another)f(thread)h(to)f(ex-)75 2437 y(ecute,)j(if)g(a)o(v)m(ailable.)33 b(The)19 b(completion)h(of)e (the)i(blo)q(c)o(king)g(MPI)f(call)h(will)h(un)o(blo)q(c)o(k)f(the)f (thread.)31 b(A)75 2494 y(blo)q(c)o(k)o(ed)23 b(thread)f(should)i(not)e (prev)o(en)o(t)g(progress)f(of)h(other)g(threads)g(on)g(the)h(same)f (pro)q(cess,)i(and)75 2550 y(should)16 b(not)f(prev)o(en)o(t)g(them)g (from)g(executing)h(MPI)f(calls.)75 2647 y Fe(Example)j(9.5)k Fq(Pro)q(cess)15 b(0)e(consists)i(of)f(t)o(w)o(o)e(threads.)20 b(The)14 b(\014rst)g(thread)g(executes)h(a)f(blo)q(c)o(king)h(send)75 2704 y(call)h Fk(MPI)p 233 2704 15 2 v 17 w(Send\(buff1,)22 b(count,)h(type,)g(0,)h(0,)f(comm\))p Fq(,)14 b(whereas)h(the)f(second) h(thread)g(executes)g(a)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 26 27 bop 75 -100 a Fq(26)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fq(blo)q(c)o(king)15 b(receiv)o(e)g(call)g Fk(MPI)p 559 45 15 2 v 17 w(Irecv\(buff2,)22 b(count,)h(type,)g(0,)h (0,)f(comm\))p Fq(.)c(I.e.,)14 b(the)g(\014rst)f(thread)75 102 y(sends)j(a)f(message)g(that)f(is)i(receiv)o(ed)h(b)o(y)e(the)g (second)h(thread.)k(This)c(comm)o(unication)g(should)h(alw)o(a)o(ys)75 158 y(succeed.)j(This,)14 b(b)q(ecause)h(if)e(the)h(\014rst)e(thread)i (blo)q(c)o(ks,)g(it)f(should)h(yield)h(to)e(the)g(second)h(thread,)f (whic)o(h)75 214 y(will)19 b(start)d(the)i(receiv)o(e.)27 b(After)18 b(b)q(oth)f(send)h(and)g(receiv)o(e)g(are)f(p)q(osted,)h (the)g(comm)o(unication)g(should)75 271 y(complete.)189 375 y Fi(A)n(dvic)n(e)c(to)h(implementors.)39 b Fq(This)14 b(requires)h(that)e(the)g(MPI)h(library)h(in)o(teract)e(with)h(the)g (thread)189 431 y(sc)o(heduler)j(\(the)f(library)h(has)e(to)h(k)o(eep)g (trac)o(k)f(of)h(threads)f(blo)q(c)o(k)o(ed)i(b)q(ecause)g(of)f(MPI)g (calls)h(and)189 488 y(un)o(blo)q(c)o(k)f(them,)f(as)f(appropriate\).) 20 b(\()p Fi(End)c(of)g(advic)n(e)g(to)h(implementors.)p Fq(\))75 609 y Fg(9.7.2)49 b(Cla)o(ri\014cations)75 695 y Fj(Prob)q(e)d Fq(A)23 b(receiv)o(e)h(call)g(that)e(uses)i(source)f (and)g(tag)f(v)m(alues)j(returned)e(b)o(y)g(a)g(preceding)h(call)h(to) 75 751 y Fj(MPI)p 160 751 14 2 v 16 w(PROBE)c Fq(or)e Fj(MPI)p 487 751 V 16 w(IPROBE)h Fq(will)h(receiv)o(e)g(the)f(message)f (matc)o(hed)g(b)o(y)h(the)f(prob)q(e)i(call)f(only)g(if)75 808 y(there)g(w)o(as)f(no)g(other)h(receiv)o(e)g(after)f(the)h(prob)q (e)g(and)g(b)q(efore)g(that)f(receiv)o(e.)35 b(In)20 b(a)g(m)o(ulti-threaded)75 864 y(en)o(vironmen)o(t,)j(it)f(is)g(up)g (to)f(the)g(user)h(to)f(enforce)g(this)h(condition)h(using)f(suitble)h (m)o(utual)f(thread)75 921 y(exclusion)17 b(co)q(de.)75 1040 y Fj(Multiple)e(threads)g(w)o(ait)f(on)g(same)g(request.)47 b Fq(A)14 b(program)e(where)j(t)o(w)o(o)d(threads)i(blo)q(c)o(k,)h(w)o (aiting)f(on)g(the)75 1097 y(same)h(request,)g(is)g(erroneous.)166 1194 y Fd(Alternativ)o(es)o(:)34 b Ff(W)m(e)14 b(could)f(allo)o(w)f (suc)o(h)j(m)o(ultiple)c(w)o(aits)i(and)h(un)o(blo)q(c)o(k)f(all)f(w)o (aiting)g(threads)j(when)f(the)75 1244 y(comm)o(unication)c(completes)k ({)g(this)g(is)f(further)i(discussed)h(in)d(the)i(hnalder)f (subsection.)166 1300 y(W)m(e)k(ma)o(y)f(w)o(an)o(t)i(to)f(designate)i (this)e(error)i(as)f(\\con)o(tin)o(uable":)27 b(a)19 b(thread)g(gets)h(a)e(nonfatal)g(error)h(if)f(it)75 1357 y(attempts)13 b(to)h(w)o(ait)f(on)h(a)g(request)h(another)f(thread)h(w) o(aits)e(on.)75 1523 y Fj(Collective)g(calls)46 b Fq(Matc)o(hing)12 b(of)g(\(blo)q(c)o(king)i(or)e(non)o(blo)q(c)o(king\))h(collectiv)o(e)i (calls)e(on)g(a)f(comm)o(unicator)g(is)75 1580 y(done)g(according)f(to) g(the)g(order)g(the)g(calls)h(are)f(issued)h(at)f(eac)o(h)g(pro)q (cessor.)18 b(If)12 b(concurren)o(t)f(threads)g(issue)75 1636 y(suc)o(h)j(call,)i(it)e(is)h(up)f(to)g(the)g(user)g(to)g(mak)o(e) g(sure)g(the)g(calls)h(are)f(correctly)h(ordered,)f(using)h(in)o (terthread)75 1693 y(sync)o(hronization.)75 1813 y Fj(Exception)i (handlers)47 b Fq(An)15 b(exception)i(handler)f(executes)h(in)f(the)f (con)o(text)g(of)g(the)h(thread)f(that)g(made)75 1869 y(the)j(exception)h(raising)g(MPI)f(call)h(\(the)f(thread)g(that)f (returns)h(the)g(error)f(co)q(de)i(is)g(the)f(thread)g(that)75 1925 y(executes)e(the)f(handler\).)189 2029 y Fi(R)n(ationale.)54 b Fq(This)18 b(is)g(in)h(order)e(to)g(\014gure)h(out)f(whic)o(h)h(call) h(caused)f(the)g(exception.)28 b(\()p Fi(End)18 b(of)189 2086 y(r)n(ationale.)p Fq(\))166 2230 y Fd(Alternativ)o(es)o(:)34 b Ff(W)m(e)13 b(ma)o(y)f(prefer)i(to)f(allo)o(w)f(the)i(exception)g (handlers)g(to)g(execute)h(in)e(an)o(y)g(con)o(text.)18 b(W)m(e)75 2280 y(then)d(need)f(to)g(pro)o(vide)g(a)f(mec)o(hanism)f (for)i(iden)o(tifying)e(the)i(faulting)f(thread.)166 2336 y(It)c(ma)o(y)f(b)q(e)h(w)o(orth)o(while)g(to)g(pro)o(vide)g (additional)f(default)h(exception)h(handlers,)g(suc)o(h)g(as)f Fc(MPI)p 1634 2336 13 2 v 15 w(KILL)p 1734 2336 V 15 w(PROCESS)p Ff(,)75 2393 y(and)14 b Fc(MPI)p 233 2393 V 14 w(KILL)p 332 2393 V 15 w(THREAD)p Ff(.)75 2561 y Fg(9.7.3)49 b(User-de\014ned)15 b(Handlers)75 2647 y Fq(In)f(man)o(y)e(situations)h(it)h(is)f(desirable)h(to)f(in)o(v)o(ok)o (e,)g(up)q(on)g(completion)h(of)f(a)g(comm)o(unication,)g(a)g(handler) 75 2704 y(routine.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 27 28 bop 75 -100 a Fh(9.7.)34 b(MPI)15 b(AND)g(THREADS)1202 b Fq(27)166 45 y(The)12 b(function)g(in)o(tro)q(duced)g(b)q(elo)o(w)g (can)g(b)q(e)g(used)g(to)e(implemen)o(t)j(the)f(generalized)h(requests) e(in)o(tro-)75 102 y(duced)16 b(in)f(the)g(External)g(In)o(terface)f(c) o(hapter;)g(it)h(can)g(also)g(b)q(e)g(used)g(to)f(implemen)o(t)i(new)f (non)o(blo)q(c)o(king)75 158 y(calls,)h(suc)o(h)f(as)g(non)o(blo)q(c)o (king)i(comm)o(unication)f(or)e(task)h(spa)o(wning.)75 309 y Fj(MPI)p 160 309 14 2 v 16 w(POST)p 294 309 V 17 w(HANDLER\(request,)i(handler)p 840 309 V 17 w(attr,)e(handler,)h (extra)p 1217 309 V 16 w(state,)g(handler)p 1491 309 V 17 w(t)o(yp)q(e\))117 386 y Ff(IN)155 b Fj(request)452 b Ff(comm)o(unication)11 b(request)k(\(handle\))117 461 y(IN)155 b Fj(handler)p 459 461 V 18 w(attr)361 b Ff(handler)14 b(attributes)117 536 y(IN)155 b Fj(handler)449 b Ff(receiv)o(e)15 b(handler)f(\(function\))117 612 y(IN)155 b Fj(extra)16 b(state)385 b Ff(extra)15 b(state)f(for)g(handler)117 687 y(IN)155 b Fj(handler)p 459 687 V 18 w(t)o(yp)q(e)351 b Ff(t)o(yp)q(e)15 b(of)e(handler)h(function)75 811 y Fk(int)23 b(MPI)p 245 811 15 2 v 17 w(Post)p 358 811 V 17 w(handler\(MPI)p 639 811 V 16 w(Request)f(request,)h(MPI)p 1132 811 V 17 w(Handler)p 1317 811 V 16 w(attr,)g(*handler)p 1668 811 V 16 w(attr,)393 868 y(MPI)p 468 868 V 17 w(Request)p 653 868 V 16 w(handler)g(handler,)g(void)g(*extra)p 1338 868 V 17 w(state,)g(int)393 924 y(MPI)p 468 924 V 17 w(handler)p 653 924 V 16 w(type\))75 1011 y(MPI)p 150 1011 V 17 w(POST)p 263 1011 V 16 w(HANDLER\(REQUEST,)f(HANDLER)p 853 1011 V 16 w(ATTR,)h(HANDLER,)g(EXTRA)p 1347 1011 V 17 w(STATE,)g(HANDLER)p 1699 1011 V 16 w(TYPE,)393 1067 y(IERROR\))170 1123 y(INTEGER)g(REQUEST,)g(EXTRA)p 699 1123 V 17 w(STATE,)g(HANDLER)p 1051 1123 V 16 w(TYPE,)g(IERROR)170 1180 y(PROCEDURE)g(HANDLER)166 1266 y Fq(A)o(ttac)o(hes)14 b Fj(handler)i Fq(to)f Fj(request)p Fq(.)21 b(This)16 b(can)g(b)q(e)f(a)g(p)q(ersisten)o(t,)h(or)e(non)i(p)q(ersisten)o(t)f (request.)166 1323 y(The)g(handler)i(is)e(a)g(routine)h(of)f(t)o(yp)q (e)75 1429 y Fk(typedef)23 b(int)g(MPI_Request_handler\(MPI_Reque)o(st) e(*request,)i(void)g(*extra_state\))166 1592 y Fq(in)16 b(C,)f(and)g(of)g(t)o(yp)q(e)75 1698 y Fk(PROCEDURE)46 b(MPI_REQUEST_HANDLER\()22 b(REQUEST,)g(EXTRA_STATE,)h(IERR\))99 1755 y(INTEGER)70 b(REQUEST,)23 b(EXTRA_STATE,)f(IERR)166 1861 y Fq(in)16 b(F)l(ortran.)166 1917 y(The)g(handler)g(is)g(in)o(v)o (ok)o(ed)g(as)f(so)q(on)g(as)g(the)h(request)f(completes,)h(subsequen)o (t)g(to)f(its)h(p)q(osting.)21 b(If)75 1974 y(the)13 b(request)h(w)o(as)e(already)i(complete)g(when)g(the)g(handler)g(w)o (as)f(p)q(osted,)g(then)h(the)f(handler)i(is)f(in)o(v)o(ok)o(ed)75 2030 y(as)d(so)q(on)f(as)h(p)q(osted.)19 b(When)11 b(in)o(v)o(ok)o(ed,) h(the)f(handler)h(is)g(passed)f(in)h Fj(request)g Fq(a)f(handle)h(to)f (the)g(request)g(the)75 2087 y(handler)16 b(w)o(as)e(p)q(osted)h(to,)g (and)g(in)h Fj(extra)p 779 2087 14 2 v 16 w(state)g Fq(the)f(extra)g (state)f(that)g(w)o(as)g(passed)h(when)h(the)f(request)75 2143 y(w)o(as)g(p)q(osted.)23 b(The)17 b(handler)g(can)g(access)f(the)g (status)f(asso)q(ciated)i(with)f(the)h(request)f(b)o(y)g(executing)h(a) 75 2200 y(call)f(to)f Fj(MPI)p 299 2200 V 16 w(GET)p 403 2200 V 16 w(ST)l(A)l(TUS)p Fq(.)166 2256 y(The)g(in)o(v)o(o)q (cation)f(of)g(the)h(handler)g(should)h(b)q(eha)o(v)o(e)e(as)g(if)h(a)f (call)i(w)o(as)d(made)i(to)e(create)h(a)g(separate)75 2312 y(thread)f(for)f(handler)j(execution,)f(when)f(the)h(request)f (completed.)20 b(The)13 b Fj(handler)p 1476 2312 V 17 w(attr)h Fq(argumen)o(t)e(sp)q(ec-)75 2369 y(i\014es)k(thread)g (attributes)f(for)g(the)g(newly)i(created)e(thread,)g(suc)o(h)h(as)f (priorit)o(y)l(,)h(sc)o(heduling)h(algorithm,)75 2425 y(stac)o(k)e(size,)i(etc.)23 b(The)17 b(t)o(yp)q(e)f(of)g(attributes)g (that)f(can)h(b)q(e)h(sp)q(eci\014ed)i(for)c(a)h(handler,)h(and)g(the)f (mec)o(ha-)75 2482 y(nisms)11 b(for)f(sp)q(ecifying)i(attribute)f(v)m (alues,)h(is)f(implemen)o(tation)h(dep)q(enden)o(t.)19 b(An)11 b(MPI)g(implemen)o(tation)75 2538 y(m)o(ust)k(pro)o(vide)g(a)g (constan)o(t,)f(prede\014ned)j(default)f(handler)g(attribute)f Fc(MPI)p 1389 2538 13 2 v 14 w(HANDLER)p 1593 2538 V 16 w(A)m(TTRIBUTE)p Fq(.)166 2635 y Fd(Discussion:)36 b Ff(P)o(osix)14 b(uses)i(a)e(structure)j(of)d(t)o(yp)q(e)h Fc(pthread)p 1125 2635 V 17 w(attr)p 1206 2635 V 15 w(t)f Ff(for)g(thread)h(attributes)h(suc)o(h)f(as)g(stac)o(k)75 2685 y(size)j(and)f(stac)o(k)h(address,)h(but)e(uses)h(a)f(di\013eren)o (t)h(mec)o(hanism)d(to)i(set)i(the)e(thread)h(priorit)o(y)f(\(a)g (function)g(call)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 28 30 28 29 bop 75 -100 a Fq(28)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Ff(that)16 b(is)f(passed)i(a)e(the)h(thread)g (id\).)23 b(Since)16 b(handler)g(instances)h(are)f(\\anon)o(ymous",)d (it)i(mak)o(es)g(more)f(sense)j(in)75 95 y(our)d(case)h(to)f(use)g(the) h(attribute)f(mec)o(hanism)e(for)h(b)q(oth.)166 151 y(W)m(e)d(should)f (use)i(here)h(the)e(same)g(mec)o(hanism)d(used)k(elsewhere)h(in)e(MPI)g (to)g(sp)q(ecify)h(attribute)f(v)n(alues.)17 b(The,)75 208 y(the)d(mec)o(hanism)d(used)j(for)f(sp)q(ecifying)g(attributes)i(b) q(ecomes)e(part)h(of)f(the)h(MPI)f(standard,)h(but)f(the)h(attributes) 75 264 y(are)g(still)f(implemen)o(tation)e(dep)q(enden)o(t.)19 b(W)m(e)14 b(ma)o(y)e(go)h(further,)h(and)g(ha)o(v)o(e)g(some)f (standard)h(attributes.)166 368 y Fq(The)h Fj(handler)p 400 368 14 2 v 17 w(t)o(yp)q(e)g Fq(argumen)o(t)f(pro)o(vides)h (information)g(on)f(restrictions)h(ob)q(ey)o(ed)g(b)o(y)f(the)h (handler)75 424 y(co)q(de.)20 b(P)o(ossible)d(v)m(alues)f(are)75 518 y Fe(MPI)p 184 518 16 2 v 18 w(HANDLER)p 468 518 V 19 w(DEF)-6 b(A)o(UL)l(T:)23 b Fq(Handler)14 b(is)g(thread-safe,)g (but)g(ob)o(y)o(es)f(no)g(further)h(restrictions.)75 612 y Fe(MPI)p 184 612 V 18 w(HANDLER)p 468 612 V 19 w(SIGNAL)p 689 612 V 19 w(SAFE:)22 b Fq(Handler)16 b(is)g(signal-safe.) 75 706 y(On)g(some)f(systems,)f(it)h(will)i(b)q(e)f(p)q(ossible)h(to)e (implemen)o(t)h(more)f(e\016cien)o(tly)h(restricted)g(handlers.)166 762 y(A)k(function)h(is)f(signal)p 569 762 14 2 v 17 w(safe)g(if)h(it)f(can)g(b)q(e)h(in)o(v)o(ok)o(ed)f(from)f(a)h (signal-catc)o(hing)h(function.)35 b(\(The)75 819 y(P)o(osix)19 b(terminology)g(is)h Fi(asynch-signal-safe)p Fq(\).)30 b(F)l(or)19 b(example,)h(a)f(signal-safe)h(handler)g(can)g(not)f(use)75 875 y(m)o(utex)c(calls.)166 979 y Fd(Discussion:)33 b Ff(The)14 b(list)e(ab)q(o)o(v)o(e)h(is)g(a)g(placeholder:)18 b(w)o(e)13 b(need)h(to)f(discuss)h(what)f(restrictions)i(mak)o(e)c (sense)75 1035 y(for)j(v)n(arious)f(implem)o(en)o(tations.)166 1139 y Fq(Handler)h(co)q(de)h(should)f(b)q(e)h(thread)e(safe.)19 b(Mutual)14 b(exclusion)h(with)f(other)f(handler)i(instances,)f(or)75 1196 y(with)i(computation)f(co)q(de)g(is)h(ac)o(hiev)o(ed)g(b)o(y)f (using)h(thread)f(sync)o(hronization)i(calls.)166 1252 y(An)d(MPI)g(exception)h(o)q(ccurs)f(if)h(the)f(handler)h(returns)f (with)g(a)g(v)m(alue)h(other)f(than)f Fc(MPI)p 1674 1252 13 2 v 15 w(SUCCESS)p Fq(.)189 1358 y Fi(A)n(dvic)n(e)19 b(to)j(implementors.)69 b Fq(Since)22 b(the)e(p)q(ost)p 1043 1358 14 2 v 16 w(handle)h(call)h(do)q(es)e(not)g(return)g(a)g (thread)f(id,)189 1415 y(implemen)o(tations)14 b(are)f(not)f(required)i (to)f(spa)o(wn)f(a)h(thread)g(when)h(the)f(call)h(o)q(ccurs;)g(rather)e (than)189 1471 y(spa)o(wning)20 b(a)g(new)g(thread)g(for)f(eac)o(h)h (handler)h(instance,)h(one)e(can)h(\\recycle")f(a)g(\014xed)h(set)e(of) 189 1528 y(threads.)189 1603 y(Note)13 b(that)h(handler)h(in)o(v)o(o)q (cation)f(requires)h(some)f(wrapp)q(er)g(co)q(de,)h(on)f(top)f(of)h Fj(pthread)p 1691 1603 V 18 w(create)p Fq(:)19 b(a)189 1659 y(P)o(osix)c(thread)h(has)g(only)g(one)g(argumen)o(t,)f(wheres)h (an)g(MPI)f(handler)i(function)g(has)e(t)o(w)o(o)g(input)189 1716 y(argumen)o(ts)f(and)h(one)h(return)f(v)m(alue.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)h(implementors.)p Fq(\))75 1836 y Fj(First)e(Alternative)75 1922 y Fq(The)h(request)f (handle)i(is)f(set)f(to)g Fc(MPI)p 724 1922 13 2 v 14 w(REQUEST)p 926 1922 V 14 w(NULL)h Fq(b)o(y)f(the)h(call)h(to)d Fj(MPI)p 1422 1922 14 2 v 16 w(POST)p 1556 1922 V 17 w(HANDLER)i Fq(\(the)75 1978 y(binding)i(has)f(to)f(b)q(e)h(c)o(hanged) g(to)f(mak)o(e)g(this)h(argumen)o(t)e Ff(INOUT)p Fq(\).)i(It)f(is)h (erroneous)f(for)g(the)h(request)75 2034 y(to)d(b)q(e)i(accessed,)f (except)g(b)o(y)g(the)f(handler,)i(once)f(the)g(handler)h(is)f(p)q (osted.)20 b(If)15 b(the)g(request)g(is)g(nonp)q(er-)75 2091 y(sisten)o(t,)h(then)g(it)g(will)h(b)q(e)g(deallo)q(cated)g(once)g (the)e(handler)i(completes.)23 b(If)16 b(it)g(is)h(p)q(ersisten)o(t,)f (then)g(the)75 2147 y(handler)i(has)e(to)h(reactiv)m(ate)g(the)f (request)h(with)g(a)g(call)h(to)e Fj(MPI)p 1202 2147 V 16 w(ST)l(ART)h Fq(or)f Fj(MPI)p 1516 2147 V 16 w(ST)l(ART)l(ALL)p Fq(,)h(or)g(to)75 2204 y(free)d(the)h(request,)f(with)h(a)f(call)h(to)f Fj(MPI)p 771 2204 V 15 w(REQUEST)p 989 2204 V 18 w(FREE)p Fq(.)h(The)f(handler)i(can)e(also)g(asso)q(ciate)h(a)f(new)75 2260 y(handler)i(with)g(the)f(request,)g(b)o(y)g(calling)i Fj(MPI)p 884 2260 V 16 w(POST)p 1018 2260 V 17 w(HANDLER)p Fq(.)166 2317 y(A)e(request)g(cannot)g(b)q(e)h(canceled,)h(once)e(a)g (handler)h(is)g(p)q(osted.)166 2420 y Fd(Discussion:)34 b Ff(This)13 b(mo)q(del)g(has)g(clean)h(seman)o(tics:)k(the)c(request)i (is)d(\\o)o(wned")h(b)o(y)f(the)h(handler,)g(once)g(it)75 2477 y(is)g(p)q(osted.)19 b(The)14 b(do)o(wn-side)g(is)g(no)f (cancelations.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 29 31 29 30 bop 75 -100 a Fh(9.7.)34 b(MPI)15 b(AND)g(THREADS)1202 b Fq(29)75 45 y Fj(Second)17 b(Alternative)75 131 y Fq(A)c(call)i(to)e Fj(MPI)p 343 131 14 2 v 15 w(POST)p 476 131 V 17 w(HANDLER)h Fq(do)q(es)g(not)f(mo)q(dify)h(the)f(request)h(handle,)g(and)g(the)f (request)h(ob)s(ject)75 187 y(can)h(still)h(b)q(e)g(accessed)f(b)o(y)g (the)g(caller,)g(after)f(the)h(call)h(to)e Fj(MPI)p 1169 187 V 16 w(POST)p 1303 187 V 17 w(HANDLER)h Fq(completed.)21 b(Also,)75 244 y(nonp)q(ersisten)o(t)e(requests)f(are)f(not)h (automatically)g(deallo)q(cated)h(at)f(handler)h(completion,)g(and)f(p) q(er-)75 300 y(sisten)o(t)e(request)f(are)h(not)f(automatically)h (deactiv)m(ated.)22 b(Instead,)16 b(a)f(call)i(to)e Fj(MPI)p 1519 300 V 16 w(REQUEST)p 1738 300 V 18 w(FREE,)75 357 y(MPI)p 160 357 V 16 w(W)l(AIT,)e(MPI)p 397 357 V 15 w(TEST)h Fq(or)e(an)o(y)h(of)g(the)g(other)g(deriv)o(ed)h(function)g (is)g(needed,)g(either)g(inside)h(the)f(han-)75 413 y(dler)k(or)e (outside)h(it.)25 b(The)17 b(user)g(has)f(complete)i(freedom)f(to)f (decide)i(who)f(\\o)o(wns")e(the)i(request,)g(and)75 470 y(who)12 b(should)h(deallo)q(cate)h(or)e(inactiv)m(ate)h(this)g (request)f(ob)s(ject.)19 b(He)12 b(or)g(she)h(also)f(has)g(the)g(resp)q (onsibili)q(t)o(y)75 526 y(to)j(sync)o(hronize)h(correctly)f (con\014cting)h(accesses)g(to)e(a)h(request)g(ob)s(ject.)166 583 y(A)f(request)g(ma)o(y)g(b)q(e)h(cancelled)h(at)e(an)o(y)g(time,)g (b)o(y)g(calling)i Fj(MPI)p 1268 583 V 16 w(CANCEL)p Fq(.)e(If)h Fj(MPI)p 1610 583 V 16 w(CANCEL)f Fq(has)75 639 y(b)q(een)g(called)h(b)q(efore)e(the)g(handler)h(w)o(as)e(in)o(v)o (ok)o(ed,)i(then)f(the)g(request)g(is)g(complete,)h(and)f(the)g (handler)h(is)75 695 y(in)o(v)o(ok)o(ed,)d(when)g(either)g(the)g(comm)o (unication)g(completes)g(successfully)l(,)i(or)d(the)g(cancellation)i (succeeds.)75 752 y(In)k(either)f(case,)g(the)g(handler)h(is)g(passed)f (a)f(v)m(alid)j(handle)f(to)f(the)g(request,)f(and)h(the)g(handler)i (can)e(use)75 808 y(it)g(to)g(c)o(hec)o(k)g(whic)o(h)i(is)e(whic)o(h,)h (using)g Fj(MPI)p 821 808 V 16 w(TEST)p 951 808 V 16 w(CANCELLED)p Fq(.)166 865 y(The)i(request)f(ma)o(y)g(b)q(e)h(freed)g (at)e(an)o(y)h(time,)h(b)o(y)g(calling)h Fj(MPI)p 1263 865 V 16 w(REQUEST)p 1482 865 V 18 w(FREE)p Fq(.)e(The)h(request)75 921 y(ob)s(ject)13 b(is)i(deallo)q(cated)g(only)f(after)g(the)f (request)h(completed)h(and)f(the)g(handler)h(w)o(as)e(in)o(v)o(ok)o (ed,)h(so)g(that)75 978 y(the)k(handler)i(will)g(b)q(e)f(passed)g(a)f (v)m(alid)i(request)e(handle,)i(when)f(in)o(v)o(ok)o(ed,)g(ev)o(en)g (if)g(the)f(request)h(w)o(as)75 1034 y(freed)d(b)q(efore)f(the)g (handler)i(started)d(executing.)166 1091 y(A)i(new)g(handler)h(can)e(b) q(e)i(asso)q(ciated)f(with)g(a)f(request)h(at)f(an)o(y)h(time,)g (replacing)h(the)f(previously)75 1147 y(p)q(osted)i(handler.)29 b(If)18 b(the)g(previous)h(handler)g(has)f(started)f(executing)i(b)q (efore)f(b)q(eing)h(replaced)g(b)o(y)f(a)75 1204 y(call)f(to)f Fj(MPI)p 301 1204 V 16 w(POST)p 435 1204 V 17 w(HANDLER)p Fq(,)g(then)h(it)f(will)i(complete)f(executing.)23 b(If)17 b Fj(MPI)p 1472 1204 V 16 w(POST)p 1606 1204 V 17 w(HANDLER)f Fq(is)75 1260 y(called)e(with)e Fj(handler)h(=)f(MPI)p 584 1260 V 16 w(HANDLER)p 808 1260 V 17 w(NULL)g Fq(then)g(no)g (handler)h(is)f(attac)o(hed)g(to)f(the)h(request)g(up)q(on)75 1316 y(return.)20 b(This)c(allo)o(ws)f(handlers)h(to)f(b)q(e)h(detac)o (hed.)166 1373 y(A)h(call)h(to)e Fj(MPI)p 444 1373 V 16 w(W)l(AIT)p Fq(,)g(or)g Fj(MPI)p 741 1373 V 16 w(TEST)p Fq(,)h(or)f(an)o(y)g(of)h(the)g(other)f(deriv)o(ed)i(functions,)f(ma)o (y)g(o)q(ccur)75 1429 y(at)h(an)o(y)g(time.)29 b(A)19 b(call)g(to)f Fj(MPI)p 636 1429 V 16 w(W)l(AIT)g Fq(or)g(an)o(y)g(of)f (the)i(deriv)o(ed)g(functions,)h(or)d(a)h(successful)i(call)g(to)75 1486 y Fj(MPI)p 160 1486 V 16 w(TEST)h Fq(or)g(an)o(y)h(of)f(the)g (deriv)o(ed)i(functions,)g(is)f(analogous)f(to)g(the)g(replacing)i(of)e (the)h(p)q(osted)75 1542 y(handler:)f(request)15 b(completion)h(is)g (handled)h(no)o(w)e(b)o(y)g(w)o(ait)f(or)h(test,)f(rather)h(than)g(b)o (y)g(the)h(handler.)166 1639 y Fd(Discussion:)33 b Ff(I)13 b(b)q(eliev)o(e)h(implemen)o(tati)o(on)c(is)j(easier)h(and)f(logic)g (clearer)h(if)e(a)h(request)i(handler)e(is)g(analo-)75 1689 y(gous)g(to)g(a)g(thread)h(blo)q(c)o(k)o(ed)f(on)g(a)g(w)o(ait:)k (in)12 b(b)q(oth)i(cases,)g(when)f(the)h(request)h(is)e(complete,)f (some)h(piece)h(of)e(co)q(de)75 1739 y(is)i(\(re\)sc)o(heduled)i(for)d (execution.)19 b(In)14 b(this)g(case,)g(there)i(are)e(t)o(w)o(o)f (consisten)o(t)i(approac)o(hes:)166 1789 y(\(1\))i(Only)g(one)g(con)o (tin)o(uation)f(w)o(aiting)f(to)i(execute)i(when)f(the)f(comm)o (unication)d(completes.)27 b(This)17 b(can)75 1839 y(b)q(e)e(either)g (one)f(handler)h(p)q(osted)g(on)f(a)f(request)j(or)e(one)h(thread)g (blo)q(c)o(k)o(ed)f(on)g(a)g(w)o(ait)f(to)h(that)g(request,)i(but)e (not)75 1888 y(b)q(oth.)166 1938 y(\(2\))19 b(Multiple)f(con)o(tin)o (uations)g(w)o(aiting)f(to)i(execute)i(when)e(the)g(comm)o(unication)d (completes.)32 b(In)19 b(suc)o(h)75 1988 y(case,)c(w)o(e)g(could)f(ha)o (v)o(e)h(m)o(ultiple)d(handlers)j(p)q(osted)g(on)g(the)g(same)f (request)i(\(a)e(new)h(mec)o(hanisms)e(is)h(needed)i(to)75 2038 y(\\unp)q(ost")i(them\),)f(as)g(w)o(ell)g(as)h(m)o(ultiple)d (threads)k(w)o(aiting)d(on)h(the)h(same)f(request,)i(or)f(an)o(y)f(com) o(bination)e(of)75 2088 y(b)q(oth.)26 b(In)16 b(the)h(later)g(case,)g (w)o(e)g(can)g(sp)q(ecify)g(that)f(these)i(v)n(arious)e(threads)h(are)g (activ)n(ated)f(in)g(the)h(order)g(they)75 2137 y(w)o(ere)e(p)q(osted.) 166 2194 y(The)f(curren)o(t)i(prop)q(osal)d(follo)o(ws)f(the)j(\014rst) g(approac)o(h.)75 2363 y Fg(9.7.4)49 b(Global)18 b(thread)d(functions) 75 2449 y Fq(It)j(is)g(useful)h(to)e(extend)h(thread)g(co)q(ordination) g(functions)g(to)f(w)o(ork)g(across)g(an)h(MPI)f(group)h(of)f(pro-)75 2505 y(cesses,)d(including)i(pro)q(cesses)e(not)g(under)g(the)g(same)f (k)o(ernel.)20 b(Suc)o(h)15 b(extensions)f(can)g(b)q(e)g(implemen)o (ted)75 2562 y(p)q(ortably)k(using)h(MPI)f(comm)o(unication)h(\(with)f (handlers\))h(and)g(pro)o(xies)f(at)f(eac)o(h)i(no)q(de)f(to)g(supp)q (ort)75 2618 y(the)f(functions)h(globally)g(\(of)e(course,)h(m)o(uc)o (h)g(more)g(e\016cien)o(t)h(implemen)o(tations)g(will)h(b)q(e)e(p)q (ossible)i(on)75 2675 y(man)o(y)c(systems\).)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 30 32 30 31 bop 75 -100 a Fq(30)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)166 45 y Fq(W)l(e)15 b(outline)i(b)q(elo)o(w)e(some)g(p)q (ossible)i(c)o(hoices.)75 165 y Fj(Mutexes)75 345 y(MPI)p 160 345 14 2 v 16 w(MUTEX)p 335 345 V 17 w(CREA)l(TE\()f(comm,)c(mutex) p 814 345 V 17 w(attr,)j(mutex\))117 423 y Ff(IN)155 b Fj(comm)470 b Ff(comm)o(unicator)11 b(for)j(group)g(sharing)f(m)o (utex)117 498 y(IN)155 b Fj(mutex)p 438 498 V 17 w(attr)383 b Ff(m)o(utex)13 b(attributes)117 573 y(OUT)108 b Fj(mutex)470 b Ff(m)o(utex)13 b(ob)r(ject)166 697 y Fq(Creates)e(a)g(m)o(utex)g(ob)s (ject)g(that)g(is)h(shared)g(b)o(y)f(all)i(pro)q(cesses)f(in)g(the)g (group)f(of)g Fj(comm)p Fq(.)17 b(\(W)l(e)11 b(could,)75 754 y(instead,)17 b(return)e(a)h Fj(new)o(comm)f Fq(and)h(further)g(o)o (v)o(erload)g(comm)o(unicators.\))21 b(This)c(is)f(the)h(extension)f (of)75 810 y Fj(pthread)p 221 810 V 18 w(mutex)p 356 810 V 16 w(init)75 961 y(MPI)p 160 961 V 16 w(MUTEX)p 335 961 V 17 w(FREE\()f(mutex\))117 1038 y Ff(INOUT)62 b Fj(mutex)470 b Ff(m)o(utex)13 b(ob)r(ject)166 1163 y Fq(F)l(ree)d(the)g(m)o(utex)g(ob)s(ject)g(\(should)h(that)e(b)q(e)i (collectiv)o(e?\))20 b(This)11 b(is)g(the)f(extension)h(to)e Fj(pthread)p 1751 1163 V 18 w(mutex)p 1886 1163 V 17 w(destro)o(y)75 1219 y Fq(\(P)o(osix)16 b(sa)o(ys)g(that)h(the)f(b)q (eha)o(vior)i(of)e(a)h(pthread)p 932 1219 V 16 w(m)o(utex)p 1072 1219 V 16 w(destro)o(y)f(call)i(on)f(a)g(lo)q(c)o(k)o(ed)g(m)o (utex)g(is)g(unde-)75 1276 y(\014ned.)j(W)l(e)13 b(can)g(lea)o(v)o(e)h (it)f(this)g(w)o(a)o(y)l(,)g(or)f(ha)o(v)o(e)h(our)g(usual,)g(more)g (lenien)o(t)i(implemen)o(tation)f(of)f(free,)g(that)75 1332 y(w)o(aits)g(un)o(til)i(ob)s(jects)e(are)g(not)g(busy)l(.)20 b(The)14 b(later)f(is)h(consisten)o(t)g(b)q(oth)g(with)g(P)o(osix)f (and)h(with)g(MPI,)f(but)75 1389 y(ma)o(y)h(not)h(b)q(e)h(consisten)o (t)g(with)f(curren)o(t)g(implemen)o(tations)i(of)d(P)o(osix)h (threads.\))75 1540 y Fj(MPI)p 160 1540 V 16 w(MUTEX)p 335 1540 V 17 w(LOCK\()f(mutex\))117 1617 y Ff(IN)155 b Fj(mutex)470 b Ff(m)o(utex)13 b(to)h(b)q(e)g(lo)q(c)o(k)o(ed)75 1836 y Fj(MPI)p 160 1836 V 16 w(MUTEX)p 335 1836 V 17 w(TRYLOCK)h(\(mutex,)g(\015ag\))117 1913 y Ff(IN)155 b Fj(mutex)470 b Ff(m)o(utex)13 b(to)h(b)q(e)g(lo)q(c)o(k)o(ed)117 1988 y(OUT)108 b Fj(\015ag)518 b Ff(success)17 b(\015ag)c(\(b)q(o)q (olean\))75 2207 y Fj(MPI)p 160 2207 V 16 w(MUTEX)p 335 2207 V 17 w(UNLOCK\()i(mutex\))117 2284 y Ff(IN)155 b Fj(mutex)470 b Ff(m)o(utex)13 b(to)h(b)q(e)g(unlo)q(c)o(k)o(ed)166 2409 y Fq(These)19 b(are)g(the)f(extensions)i(of)e(the)h(P)o(osix)g (calls)h Fj(pthread)p 1199 2409 V 17 w(mutex)p 1333 2409 V 17 w(lo)q(ck,)f(pthread)p 1602 2409 V 18 w(mutex)p 1737 2409 V 17 w(unlo)q(ck)75 2465 y Fq(and)k Fj(pthread)p 317 2465 V 18 w(mutex)p 452 2465 V 16 w(unlo)q(ck)p Fq(.)43 b(The)23 b(lo)q(c)o(k)g(call)g(is)g(blo)q(c)o(king)h(\(lik)o(e)g(a)e(w) o(ait\),)h(whereas)f(the)h(trylo)q(c)o(k)75 2522 y(function)16 b(in)g(non)o(blo)q(c)o(king,)g(lik)o(e)h(a)d(test.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 31 33 31 32 bop 75 -100 a Fh(9.8.)34 b(SEND)p 309 -100 14 2 v 16 w(BUFFER)p 515 -100 V 17 w(INIT,)15 b(ETC.)1064 b Fq(31)75 45 y Fj(Condition)16 b(va)o(riables)75 225 y(MPI)p 160 225 V 16 w(COND)p 303 225 V 17 w(CREA)l(TE\()g(comm,)d (cond)p 755 225 V 17 w(attr,)i(cond\))117 303 y Ff(IN)155 b Fj(comm)470 b Ff(comm)o(unicator)11 b(for)j(group)g(sharing)f (condition)g(v)n(ariable)117 378 y(IN)155 b Fj(cond)p 410 378 V 18 w(attr)410 b Ff(condition)13 b(attributes)117 453 y(OUT)108 b Fj(cond)498 b Ff(condition)13 b(ob)r(ject)166 577 y Fq(Creates)g(a)h(condition)i(v)m(ariable)f(that)e(is)i(shared)f (b)o(y)g(all)h(pro)q(cesses)g(in)g(the)f(group)g(of)g Fj(comm)p Fq(.)j(\(W)l(e)75 634 y(could,)f(here,)f(to)q(o,)f(o)o(v)o (erload)h(comm)o(unicators,)f(rah)o(ter)h(than)g(ha)o(ving)g(a)g(new)h (opaque)f(ob)s(ject.\))75 785 y Fj(MPI)p 160 785 V 16 w(COND)p 303 785 V 17 w(FREE\()g(cond\))117 862 y Ff(INOUT)62 b Fj(cond)498 b Ff(condition)13 b(ob)r(ject)166 986 y Fq(F)l(ree)i(the)g(condition)i(ob)s(ject.)166 1043 y(These)e(are)g(the) h(MPI)f(extensions)h(to)e Fj(pthread)p 976 1043 V 18 w(cond)p 1083 1043 V 17 w(init)i Fq(and)g Fj(pthread)p 1409 1043 V 17 w(cond)p 1515 1043 V 18 w(destro)o(y)p Fq(.)75 1194 y Fj(MPI)p 160 1194 V 16 w(COND)p 303 1194 V 17 w(SIGNAL\()f(cond\))117 1271 y Ff(IN)155 b Fj(cond)498 b Ff(condition)13 b(to)h(b)q(e)h(signaled)75 1490 y Fj(MPI)p 160 1490 V 16 w(COND)p 303 1490 V 17 w(BRO)o(ADCAST)h(\()f(cond\))117 1567 y Ff(IN)155 b Fj(cond)498 b Ff(condition)13 b(to)h(b)q(e)h (signaled)75 1786 y Fj(MPI)p 160 1786 V 16 w(COND)p 303 1786 V 17 w(W)l(AIT\()g(cond\))117 1863 y Ff(IN)155 b Fj(cond)498 b Ff(condition)13 b(to)h(w)o(ait)f(on)166 1988 y Fq(These)d(are)g(the)g(MPI)h(extensions)f(of)g(the)g(P)o(osix)g (calls)i Fj(pthread)p 1229 1988 V 18 w(cond)p 1336 1988 V 17 w(signal,)f(pthread)p 1627 1988 V 18 w(cond)p 1734 1988 V 17 w(b)o(roadcast)75 2044 y Fq(and)g Fj(pthread)p 305 2044 V 18 w(cond)p 412 2044 V 17 w(w)o(ait)p Fq(.)18 b(The)11 b(cond)p 721 2044 V 17 w(w)o(ait)g(call)g(is)h(blo)q(c)o (king.)20 b(The)11 b(cond)p 1333 2044 V 16 w(signal)h(call)g(un)o(blo)q (c)o(ks)g(at)e(least)75 2101 y(one)17 b(of)f(the)h(threads)f(w)o (aiting)h(on)g(the)g(condition)h(\(if)f(an)o(y\))f(and)h(the)f(cond)p 1395 2101 V 17 w(broadcast)g(call)i(un)o(blo)q(c)o(ks)75 2157 y(all)e(threads)f(that)g(are)f(w)o(aiting)i(on)f(the)g(condition.) 166 2261 y Fd(Discussion:)34 b Ff(I)14 b(ha)o(v)o(e)f(left)h(the)g (extension)h(of)e Fc(pthread)p 1065 2261 13 2 v 17 w(cond)p 1163 2261 V 16 w(timedw)o(ait)h Ff(to)g(the)g(real-time)f(p)q(eople.) 166 2365 y Fq(Prede\014ned)j(attributes)f(will)i(b)q(e)f(pro)o(vided,)g (b)q(oth)f(for)g(m)o(utexes)g(and)g(for)g(condition)h(v)m(ariables.)30 b Fm(?)16 b Fl(\(July\))75 2508 y Fn(9.8)59 b(Send)p 337 2508 18 2 v 21 w(Bu\013er)p 511 2508 V 21 w(Init,)19 b(etc.)1875 2560 y Fm(>)d Fl(\(July\))75 2656 y Fd(Discussion:)34 b Ff(This)14 b(section)g(is)g(subsumed)g(b)o(y)g(the)g(prop)q(osal)g (on)f(\\shared")i(bu\013ers)g(-)e(RL)1875 2712 y Fm(?)j Fl(\(July\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 32 34 32 33 bop 75 -100 a Fq(32)1138 b Fh(CHAPTER)15 b(9.)30 b(MISCELLANY)1875 104 y Fm(>)16 b Fl(\(July\))75 180 y Fn(9.9)59 b(Naming)20 b(Objects)75 282 y Fq(There)12 b(are)f(man)o(y)g(o)q(ccasions)h(on)f(whic)o(h)h(it)g(w)o(ould)g(p)q (oten)o(tially)h(b)q(e)f(useful)g(to)f(allo)o(w)h(a)f(user)g(to)g(asso) q(ciate)75 338 y(a)g(prin)o(t)g(name)g(with)g(an)g(MPI)g(comm)o (unicator,)h(for)e(instance:)19 b(error)10 b(rep)q(orting,)i (debugging,)g(pro\014ling,)75 395 y(etc.)166 451 y(This)k(ob)s(jectiv)o (e)f(can)g(b)q(e)h(ac)o(hiev)o(ed)g(using)g(the)f(follo)o(wing)h(t)o(w) o(o)e(functions)75 614 y Fk(int)23 b(MPI_Comm_set_name)f(\(IN)h (MPI_Comm)g(comm,)g(IN)48 b(string)23 b(name\))123 670 y(int)g(MPI_Comm_set_name\(MPI_Comm)d(comm,)k(char)f(*)h(name\);)123 727 y(integer)f(MPI_Comm_set_name\()e(integer)i(comm,)g (character*\(*\))f(name\))75 840 y(int)h(MPI_Comm_get_name)f(\(IN)h (MPI_Comm)g(comm,)g(OUT)h(string)f(name\))123 896 y(int)g (MPI_Comm_get_name\(MPI_Comm)d(comm,)k(char)f(**)h(name\);)123 953 y(integer)f(MPI_Comm_get_name\()e(integer)i(comm,)g (character*\(*\))f(name\))166 1115 y Fj(MPI)p 251 1115 14 2 v 16 w(Comm)p 393 1115 V 14 w(set)p 460 1115 V 17 w(name)10 b Fq(allo)o(ws)h(a)f(user)g(to)g(asso)q(ciate)g(a)g(name)h (string)f(with)h(a)f(comm)o(unicator.)18 b(The)75 1172 y(passed)d(in)g(c)o(haracter)f(string)h(will)h(b)q(e)g(sa)o(v)o(ed)e (inside)i(the)f(MPI)g(library)g(\(so)f(need)i(not)e(b)q(e)h(allo)q (cated)h(in)75 1228 y(p)q(ersisten)o(t)g(store\).)166 1285 y Fj(MPI)p 251 1285 V 16 w(Comm)p 393 1285 V 14 w(get)p 466 1285 V 17 w(name)d Fq(returns)h(an)o(y)g(name)g(whic)o(h)h (had)f(previously)h(b)q(een)g(asso)q(ciated)g(with)f(the)75 1341 y(giv)o(en)21 b(comm)o(unicator.)34 b(\(Note)20 b(that)f(in)i(the)f(C)g(binding)i(it)f(returns)f(a)f(reference)i(to)f (the)g(in)o(ternal)75 1398 y(cop)o(y)14 b(of)f(the)h(name,)g(whic)o(h)g (should)h(not)f(b)q(e)g(mo)q(di\014ed)h(b)o(y)f(the)g(user,)g(and)g (whic)o(h)h(will)g(b)q(ecome)g(in)o(v)m(alid)75 1454 y(should)g(the)f(name)g(b)q(e)g(altered)h(\(b)o(y)e(another)h(call)h (to)e Fj(MPI)p 1092 1454 V 16 w(Comm)p 1234 1454 V 14 w(set)p 1301 1454 V 18 w(name)p Fq(\),)g(or)g(the)h(comm)o(unicator)75 1510 y(deleted\).)166 1567 y(If)22 b(the)f(user)h(has)f(not)g(asso)q (ciated)h(a)f(name)g(with)h(a)f(comm)o(unicator)g(the)g(system)g (should)i(re-)75 1623 y(turn)d(a)g(n)o(ull)h(string)g(\(all)f(spaces)h (in)g(F)l(ortran,)e("")h(in)h(C\),)e(or)h(a)g(system)f(dep)q(enden)o(t) j(name.)35 b(\(This)75 1680 y(allo)o(ws)19 b(the)h(MPI)f(system)g(to)f (giv)o(e)i(default)f(prin)o(t)h(names)f(to)f(comm)o(unicators,)i(one)f (migh)o(t)g(exp)q(ect)75 1736 y Fj(MPI)p 160 1736 V 16 w(COMM)p 318 1736 V 16 w(W)o(ORLD)c Fq(to)g(return)g("MPI)p 818 1736 V 16 w(COMM)p 986 1736 V 16 w(W)o(ORLD",)g(for)f(instance\).) 166 1793 y(W)l(e)22 b(also)f(add)h(a)f(pre-de\014ned)i(attribute)f(k)o (ey)f(MPI)p 1107 1793 V 16 w(NAME)h(to)e(the)i(C)f(binding)j(only)l(.) 39 b(This)75 1849 y(b)q(eha)o(v)o(es)16 b(as)f(if)i(the)e(follo)o(wing) i(co)q(de)f(had)g(b)q(een)h(executed)g(in)f Fj(MPI)p 1244 1849 V 16 w(Init)g Fq(\(giv)o(e)g(or)f(tak)o(e)g(the)h(issue)h (that)75 1906 y Fj(MPI)p 160 1906 V 16 w(NAME)c Fq(is)g(here)f(treated) g(as)g(a)g(v)m(ariable)i(whereas)f(it)f(should)i(lo)q(ok)e(lik)o(e)i(a) e(constan)o(t)g(to)f(user)i(co)q(de\).)75 2068 y Fk(static)23 b(int)g(name_delete\(MPI_Comm)f(comm,)h(int)g(keyval,)g(void)g(*name,) 648 2125 y(void)g(*extra\))75 2181 y({)147 2238 y(free\(name\);)147 2294 y(return)g(MPI_SUCCESS;)75 2351 y(})75 2464 y (MPI_Keyval_create\(MPI_Null)o(_copy_f)o(n,)e(name_delete,)h (&MPI_NAME,)h(0\);)75 2576 y(Then)g(the)h(C)f(version)g(of)h (MPI_Comm_set_name)e(can)h(be)h(implemented)e(as)75 2689 y(int)h(MPI_Comm_set_name\(MPI_Comm)e(comm,)i(char)g(*)h(name\))1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 33 35 33 34 bop 75 -100 a Fh(9.9.)34 b(NAMING)15 b(OBJECTS)1226 b Fq(33)75 45 y Fk({)147 102 y(char)23 b(*)h(local_name)e(=)i (strdup\(name\);)147 214 y(return)f(MPI_Attr_put\(comm,)e(MPI_NAME,)i (local_name\);)75 271 y(})75 384 y(and)g(the)h(C)g(version)f(of)g (MPI_Comm_get_name)f(is)h(simply)75 497 y(int)g (MPI_Comm_get_name\(MPI_Comm)e(comm,)i(char)g(**name\))75 553 y({)147 610 y(int)g(flag;)147 666 y(int)g(res)g(=)h (MPI_Attr_get\(comm,)e(MPI_NAME,)g(name,)h(&flag\);)147 779 y(if)g(\(!flag\))194 835 y(*name)g(=)h("";)g(/*)f(No)h(name,)f (give)g(it)h(a)g(null)f(string)g(*/)147 892 y(return)g(res;)75 948 y(})75 1061 y(Rationale:)75 1174 y(We)h(provide)e(the)i(full)f(set) h(of)f(functions)g(above)g(as)h(well)f(as)h(the)f(pre-defined)75 1231 y(attribute)g(key)g(for)h(the)f(following)g(reasons)75 1344 y(1\))h(It)f(is)h(not)f(in)h(general)f(possible)f(to)i(store)f(a)h (string)f(as)h(an)f(attribute)147 1400 y(from)g(Fortran.)75 1456 y(2\))h(To)f(make)g(the)h(attribute)f(key)g(useful)g(the)h (additional)e(code)h(to)h(call)f(strdup)147 1513 y(is)g(necessary.)g (If)g(this)h(is)f(not)h(standardised)e(then)h(users)g(have)h(to)f (write)147 1569 y(it.)g(This)g(is)h(extra)f(unneeded)g(work)g(which)g (we)h(can)f(easily)g(eliminate.)75 1626 y(3\))h(The)f(Fortran)g (binding)g(is)g(not)h(trivial)f(to)g(write)h(\(it)f(will)g(depend)g(on) 147 1682 y(details)f(of)i(the)g(Fortran)e(compilation)h(system\),)g (and)g(will)g(not)h(be)147 1739 y(portable.)e(Therefore)h(it)g(should)g (be)h(in)g(the)f(library)g(rather)g(than)g(in)h(user)147 1795 y(code.)75 1908 y(***Open)f(issues***)75 1965 y(1\))h(Does)f(the)g (MPI_Info)g(mechanism)g(provide)g(a)g(way)h(of)f(creating)g(an)h (object)f(which)147 2021 y(could)g(then)g(be)h(cached)f(allowing)f(us)i (to)g(bundle)f(up)g(a)h(string)f(?)147 2134 y(Even)g(if)g(we)h(could)f (implement)g(this)g(way)h(I'd)f(still)g(like)h(to)f(keep)h(simple)f (user)147 2190 y(level)g(functions)f(as)i(described)f(above,)g(because) g(people)g(won't)g(use)g(them)147 2247 y(otherwise...)75 2360 y(2\))h(If)f(we're)g(allowing)g(attributes)g(on)g(many)h(objects)e (we)i(should)f(allow)g(names)147 2416 y(there)g(too.)1875 2522 y Fm(?)16 b Fl(\(July\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 34 36 34 35 bop 75 -100 a Fq(34)1133 b Fh(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fn(9.10)59 b(Prop)r(osal:)26 b(De\014ning)20 b(Continuable)f(Erro)n(rs)75 147 y Fq(T)o(ypically)l(,)j(MPI)d(sa)o(ys) g(nothing)h(ab)q(out)g(the)f(in)o(ternal)h(state)f(of)g(itself)i(after) d(an)i(error)f(handler)h(has)75 203 y(b)q(een)14 b(in)o(v)o(ok)o(ed.)20 b(There)13 b(are)g(a)g(n)o(um)o(b)q(er)g(of)g(error)f(classes)i(within) g(MPI)f(that)g(should)h(ha)o(v)o(e)f(no)g(negativ)o(e)75 259 y(impact)19 b(on)f(the)g(in)o(ternal)h(state)f(of)g(MPI.)g(That)g (is)g(to)g(sa)o(y)l(,)g(if)h(one)g(of)e(the)i("con)o(tin)o(uable")g (errors)e(is)75 316 y(detected)g(and)h(no)e(other)h(external)g(ev)o(en) o(ts)g(cause)g(degradation,)g(it)g(is)g(safe)g(to)f(con)o(tin)o(ue)h (using)h(MPI)75 372 y(within)e(an)g(application.)166 429 y(The)f("con)o(tin)o(uable")h(error)f(classes)g(are:)75 532 y Fk(MPI_ERR_BUFFER)237 b(/*)23 b(invalid)g(buffer)g(pointer)g(*/) 75 588 y(MPI_ERR_COUNT)261 b(/*)23 b(invalid)g(count)g(argument)g(*/)75 645 y(MPI_ERR_TYPE)285 b(/*)23 b(invalid)g(datatype)g(argument)g(*/)75 701 y(MPI_ERR_TAG)309 b(/*)23 b(invalid)g(tag)h(argument)e(*/)75 758 y(MPI_ERR_COMM)285 b(/*)23 b(invalid)g(communicator)f(*/)75 814 y(MPI_ERR_ROOT)285 b(/*)23 b(invalid)g(root)h(*/)75 871 y(MPI_ERR_RANK)285 b(/*)23 b(invalid)g(rank)h(*/)75 927 y(MPI_ERR_GROUP)261 b(/*)23 b(invalid)g(group)g(*/)75 984 y(MPI_ERR_TOPOLOGY)189 b(/*)23 b(invalid)g(topology)g(*/)75 1040 y(MPI_ERR_DIMS)285 b(/*)23 b(invalid)g(dimension)g(argument)g(*/) 75 1097 y(MPI_ERR_ARG)309 b(/*)23 b(invalid)g(argument)g(*/)75 1200 y Fq(The)14 b(follo)o(wing)g(errors)e(pro)o(vide)i(no)f(guaran)o (tees,)g(although)g(implemen)o(ters)i(are)e(encouraged)h(to)e(mak)o(e) 75 1256 y(errors)i("con)o(tin)o(uable")i(whenev)o(er)g(p)q(ossible:)75 1348 y Fk(MPI_ERR_OP)333 b(/*)23 b(invalid)g(operation)g(*/)75 1404 y(MPI_ERR_REQUEST)213 b(/*)23 b(invalid)g(request)g(handle)g(*/)75 1461 y(MPI_ERR_UNKNOWN)213 b(/*)23 b(unknown)g(error)g(*/)75 1517 y(MPI_ERR_TRUNCATE)189 b(/*)23 b(message)g(truncated)g(on)g (receive)g(*/)75 1574 y(MPI_ERR_OTHER)261 b(/*)23 b(Other)g(error)h(*/) 75 1630 y(MPI_ERR_INTERN)237 b(/*)23 b(internal)g(MPI)h(error)f(*/)75 1686 y(MPI_ERR_IN_STATUS)165 b(/*)23 b(error)g(code)h(is)f(in)h(status) f(*/)75 1743 y(MPI_ERR_PENDING)213 b(/*)23 b(pending)g(request)g(*/)75 1799 y(MPI_ERR_SYSRESOURCE)117 b(/*)23 b(out)h(of)f(system)g(resources) g(*/)75 1891 y Fq(W)l(e)15 b(also)f(pro)o(vide)h(the)g(function)g(to)f (indicate)i(whether)e(MPI)h(b)q(eliev)o(es)h(an)f(error)e(is)i("con)o (tin)o(uable")g(or)75 1947 y(not:)75 2098 y Fj(MPI)p 160 2098 14 2 v 16 w(ERR)p 261 2098 V 17 w(IS)p 316 2098 V 16 w(CONTINUABLE)h(\(Erro)o(r,)d(Flag\))117 2176 y Ff(IN)155 b Fj(Erro)o(r)490 b Ff(MPI)14 b(Error)h(Status)f(\(actual)g (status)h(or)f(a)f(class\))117 2250 y(OUT)108 b Fj(Flag)505 b Ff(Bo)q(olean)13 b(indication)f(of)h(whether)h(the)g(implemen)o (tation)c(al-)905 2307 y(lo)o(ws)19 b(safe)h(con)o(tin)o(ued)g(op)q (eration)g(in)f(the)i(ev)o(en)o(t)f(of)f(Error)905 2363 y(b)q(eing)14 b(encoun)o(tered)75 2534 y Fk(int)23 b (MPI_Err_is_continuable)e(\(int)j(Error,)f(int)g(Flag\))75 2647 y(MPI_ERR_IS_CONTINUABLE)e(\(ERROR,)i(FLAG,)g(IERROR\))147 2704 y(INTEGER)f(ERROR)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 35 37 35 36 bop 75 -100 a Fh(9.11.)34 b(YET)15 b(ANOTHER)h(SEND)f(MODE)980 b Fq(35)147 45 y Fk(LOGICAL)22 b(FLAG)147 102 y(INTEGER)g(IERROR)1875 151 y Fm(>)16 b Fl(\(July\))75 295 y Fn(9.11)59 b(Y)-5 b(et)19 b(Another)g(Send)g(Mo)r(de)75 396 y Fk(In)24 b(the)f(1-sided)g(chapter,)g(we)g(have)h(introduced)e(the)i(notion)f (of)g("good")g(memory)g(vs.)75 453 y("bad")g(memory)g(for)h(gets)f(and) g(puts.)h(The)f(idea)g(is)h(that)f(if)h(your)f(system)g(supports)g (shared)75 509 y(memory,)g(then)g(you)h(can)f(call)g(MPI_RMA_MALLOC)f (to)i(request)f(a)h(shared)f(buffer)g(and)g(as)h(a)75 565 y(reward)f(for)g(doing)h(this)f(your)g(gets)h(and)f(puts)g(will)h (be)f(faster.)75 678 y(The)g(thought)g(occurs)g(that)h(2-sided)f (communication)f(could)h(benefit)g(from)g(this)g(as)h(well.)75 735 y(In)g(MPI-1,)f(for)g(example,)g(many)g(shared)g(memory)g (implementations)f(still)h(use)h(a)f(two-copy)75 791 y(algorithm)g(to)g(pass)h(messages)e(around;)h(the)h(sender)f(copies)g (the)g(data)h(into)f(a)h(shared)75 848 y(buffer,)f(and)g(the)h (receiver)f(copies)g(it)g(out)h(again.)f(While)g(some)g(systems)g(are)h (able)f(to)g(do)75 904 y(better)g(than)g(this)h(in)f(some)h(cases,)f (many)g(are)g(not.)75 1017 y(So,)g(given)h(that)f(we)h(already)e(are)i (going)f(to)h(be)f(adding)g(MPI_RMA_MALLOC)f(to)i(MPI-2,)f(why)75 1074 y(not)g(also)h(add)f(a)h(new)f(send)h(mode)f(to)h(take)f (advantage)g(of)g(it?)h(Something)e(like)i(this:)266 1186 y(MPI_Xsend)266 1243 y(MPI_Ixsend)266 1299 y(MPI_Xsend_init)70 b(\(We)23 b(can)g(debate)h(the)f(proper)g(value)g(of)h('X')f (later...\))75 1412 y(In)h(this)f(mode,)g(the)h(sender)f(asserts)f (that)i(the)f(buffer)g(being)g(passed)h(is)f(one)h(which)f(was)75 1469 y(returned)g(by)g(a)h(call)f(to)h(MPI_RMA_MALLOC.)e(It)h (otherwise)g(has)g(the)h(same)f(semantics)g(as)75 1525 y(standard)g(mode.)g(With)g(this)h(mode,)f(processes)f(calling)h (MPI_RECV)g(on)h(shared)f(memory)75 1582 y(machines)g(would)g(be)h (able)f(to)g(copy)h(the)f(data)h(directly)e(out)i(of)f(the)h(send)f (buffer.)g(There)75 1638 y(could)g(also)g(be)h(advantages)f(on)g(NOWs,) g(because)g(\(for)h(example\))e(high)i(performance)75 1695 y(implementations)e(might)h(opt)g(to)h(pin)f(down)h(the)f(send)h (buffers)e(in)i(advance)f(to)h(prepare)75 1751 y(for)f(subsequent)g (DMAs.)g(Machines)g(not)g(capable)g(of)h(optimizing)e(this)i(case)f (could)g(simply)75 1807 y(use)g(MPI_SEND)g(to)h(implement)e(the)i(new)f (calls,)g(much)h(as)f(often)g(happens)g(today)h(with)75 1864 y(MPI_RSEND.)75 1977 y(Adding)f(additional)g(smarts)g(to)g(the)h (already-existing)d(send)j(modes)f(is)g(probably)g(not)h(a)75 2033 y(practical)f(solution)f(because)h(of)h(the)f(added)h(latency)e (that)i(would)f(be)h(involved)e(in)75 2090 y(building)h(internal)g (lists)g(of)g(shared)g(buffers)g(and)h(comparing)e(against)h(them)h (with)f(every)75 2146 y(sent)g(message.)75 2259 y(Doing)g(this)g(would) h(probably)e(involve)h(making)g(some)h(minor)f(changes)g(to)g (MPI_RMA_MALLOC.)75 2316 y(For)g(example,)g(it)h(would)f(probably)g (have)g(to)h(become)f(a)g(collective)g(function,)g(and)g(we)75 2372 y(would)g(perhaps)g(want)g(to)h(tweak)f(the)h(syntax)f(slightly.) 75 2485 y(Does)g(anyone)g(else)h(think)f(that)g(this)g(idea)h(has)f (merit?)1875 2591 y Fm(?)16 b Fl(\(July\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 36 38 36 37 bop 75 -100 a Fq(36)1138 b Fh(CHAPTER)15 b(9.)30 b(MISCELLANY)75 45 y Fn(9.12)59 b(Prop)r(osal:)26 b(Tw)n(o)20 b(Pro)r(cess)f("Sha)n(red")h(Bu\013ers)75 147 y Fq(It)15 b(is)g(not)g(uncommon)g(for)f(a)g(pair)i(of)e(pro)q(cesses)h(to)g (engage)f(in)i(frequen)o(t)f(pre-determined)h(comm)o(uni-)75 203 y(cations.)k(On)14 b(example)h(of)f(suc)o(h)h(a)f(co)q(de)h(is)f(a) g(pip)q(eline)k(algorithm)c(in)h(whic)o(h)g(pro)q(cess)f(A)h(creates)f (data)75 259 y(that)i(is)h(subsequen)o(tly)h(used)f(b)o(y)f(pro)q(cess) h(B.)f(Another)g(example)i(is)f(a)f(\014nite)h(di\013erence)h(appro)o (xima-)75 316 y(tion)13 b(mo)q(del)h(when)g(comm)o(unicating)g(b)q (oundary)f(v)m(alues)h(\(a.k.a.)k(ghost)12 b(p)q(oin)o(ts\).)19 b(On)14 b(a)f(mac)o(hine)h(with)75 372 y(shared)k(memory)l(,)g(it)g(is) h(p)q(ossible)h(to)d(share)h(a)f(common)h(bu\013er)g(to)f(use)i(for)e (exc)o(hanging)i(suc)o(h)f(data.)75 429 y(Ownership)c(of)f(the)g (bu\013er)g(m)o(ust)f(b)q(e)i(established)g(in)g(order)f(for)f(a)h(pro) q(cess)g(to)f(safely)h(\014ll)i(or)d(empt)o(y)h(the)75 485 y(bu\013er.)25 b(This)17 b(can)g(b)q(e)g(expressed)h(as)e(a)h(pro)q (duce/comsume)g(relationship.)26 b(This)18 b(functionalit)o(y)g(can)75 542 y(b)q(e)c(expressed)f(in)h(suc)o(h)f(a)g(w)o(a)o(y)f(that)g(it)i (should)g(b)q(e)f(no)g(less)h(e\016cien)o(t)f(when)h(using)g(MPI)f (Send/Recv)h(on)75 598 y(a)h(distributed)i(memory)d(mac)o(hine)i(and)g (is)f(simply)i(a)e(p)q(oin)o(ter)g(pass)g(on)g(shared)h(memory)e(mac)o (hines.)166 695 y Fd(Discussion:)34 b Ff(This)13 b(prop)q(osal)g(is)h (prompted)f(b)o(y)g(T)m(on)o(y)g(Skjellum's)e(prop)q(osal)i(titled:)18 b("In\014nite)13 b(asymp-)75 745 y(totic)h(bandwidth)f(for)h(thread)h (comm)o(uni)o(cation")166 795 y(W)m(e)j(ha)o(v)o(e)h(a)g(similia)o(r)e (proto)q(col)h(inside)h(our)g(system)g(when)g(w)o(e)g(w)o(ork)g(on)f (shared)i(memory)c(mac)o(hines.)75 845 y(Imp)q(ortan)o(t)i(to)i(us)g (is)f(not)h(only)e(the)j(bandwidth)e(and)g(latency)m(,)i(but)e(also)g (ligh)o(t)o(w)o(eigh)o(t)f(\015o)o(w-con)o(trol.)35 b(The)75 895 y(prop)q(osal)14 b(in)g(its)g(curren)o(t)i(form)c(solv)o(es)j(our)f (problem)f(in)h(particular,)g(but)g(w)o(e)h(feel)f(it)g(ma)o(y)e (prompt)h(discussion)75 944 y(that)i(can)f(shap)q(e)i(this)f(prop)q (osal)f(and)g(T)m(on)o(y's)g(prop)q(osal)g(in)o(to)g(something)f(that)i (will)e(pro)o(vide)i(a)f(p)q(ortable)h(high)75 994 y(bandwidth/lo)o (w-latency)c(mec)o(hanism)e(on)j(true)h(SMPs)f(as)h(w)o(ell)e(as)h(pro) o(vide)g(high)f(p)q(erformance)h(in)f(distributed)75 1044 y(memory)g(mac)o(hines.)166 1094 y(Questions)k(for)e(discussion:) 166 1144 y(1\))i(Is)h(this)f(really)g(going)f(to)h(b)q(e)h(faster)g(on) f(a)g(shared)h(memory)d(mac)o(hine)h(with)h(a)g("high-qualit)o(y")e (imple-)75 1193 y(men)o(tation.)166 1243 y(2\))h(Do)q(es)g(the)g(user)h (visible)f(\015o)o(w-con)o(trol)f(pro)o(vide)g(signi\014can)o(t)h (added)g(v)n(alue?)166 1293 y(3\))19 b(F)m(or)f(b)q(oth)h(shared)g(and) g(distributed)g(memory)e(mac)o(hines,)h(do)q(es)i(prede\014ning)f(the)g (comm)o(unication)75 1343 y(pattern)c(pro)o(vide)e(options)h(for)f (optimization?)166 1393 y(Leslie)h(Hart)166 1442 y(T)m(om)e(Henderson) 166 1499 y(NO)o(AA/FSL)75 1652 y Fk(Example:)75 1765 y(Process)23 b(0:)668 b(Process)23 b(1:)75 1878 y(!)h(Make)f(a)h (shared)f(buffer)g(with)g(process)g(0)h(as)f(the)h(source,)f(process)g (1)g(as)h(the)f(dest)75 1935 y(!)h(sbh)f(is)h(the)f(shared)g(buffer)g (handle)g(and)h(buff)f(is)h(some)f(memory)g(allocated)75 1991 y(!)h(by)f(MAKE_SHARED_BUFFER)75 2048 y(MAKE_SHARED_BUFFER)e (\(...,)334 b(MPI_MAKE_SHARED_BUFFER)21 b(\(...,)218 2104 y(0,)j(1,)f(buff,)h(sbh\))619 b(0,)24 b(1,)g(buff,)f(sbh\))75 2161 y(!)h(Process)f(0)g(owns)h(the)f(buffer)262 b(!)24 b(Process)e(1)i(waits)f(for)h(buffer)99 2217 y(...Fill)f(buff)g(with)g (useful)g(stuff...)95 b(ACQUIRE_SHARED_BUFFER)21 b(\(sbh\))75 2330 y(!)j(Process)f(1)g(releases)g(the)h(buffer)75 2386 y(!)g(communicating)e(that)h(it)h(was)75 2443 y(!)g(modified)e (\(PRODUCEd\))75 2499 y(RELEASE_SHARED_BUFFER)f(\(sbh,)i(PRODUCE\))982 2556 y(!)h(At)f(this)g(point)h(process)f(1)g(unblocks)75 2612 y(!)h(Wait)f(for)g(process)g(0)h(to)g(use)f(the)167 b(!)24 b(and)f(has)g(full)h(access)f(to)g(buff)75 2669 y(!)h(Information)e(produced)h(before)214 b(...Read)23 b(buff)g(and)g(do)h(something...)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 37 39 37 38 bop 75 -100 a Fh(9.12.)34 b(PR)o(OPOSAL:)16 b(TW)o(O)f(PR)o (OCESS)h("SHARED")f(BUFFERS)521 b Fq(37)75 45 y Fk(!)24 b(creating)e(more)i(data)75 102 y(ACQUIRE_SHARED_BUFFER)d(\(sbh\))262 b(!)24 b(Process)e(1)i(releases)f(the)g(buffer,)g(but)982 158 y(!)h(has)f(not)g(modified)g(it)h(\(CONSUMEd\))982 214 y(RELEASE_SHARED_BUFFER)d(\(sbh,)i(CONSUME\))75 271 y(!)h(At)f(this)h(point)f(process)g(0)g(unblocks)75 327 y(!)h(and)f(is)h(free)f(to)h(modify)f(buff)314 384 y(...)739 b(...)75 440 y(FREE_SHARED_BUFF\(sbh\))403 b(FREE_SHARED_BUFF\(sbh\))75 639 y Fj(MPI)p 160 639 14 2 v 16 w(MAKE)p 305 639 V 16 w(SHARED)p 497 639 V 17 w(BUFFER)16 b(\(dataT)l(yp)q(e,)f(count,)h (comm,)c(source,)j(dest,)h(bu\013er,)e(sha)o(redBu\013Han-)75 696 y(dle\))117 773 y Ff(IN)155 b Fj(dataT)l(yp)q(e)412 b Ff(MPI)14 b(Data)g(T)o(yp)q(e)g(\(con)o(tiguous\))117 848 y(IN)155 b Fj(count)482 b Ff(Num)o(b)q(er)14 b(of)f(dataT)o(yp)q (es)117 923 y(IN)155 b Fj(comm)470 b Ff(Comm)o(unicator)117 998 y(IN)155 b Fj(source)468 b Ff(First)14 b(pro)q(cess)i(in)e(pair)f (\(initially)f(o)o(wns)h(bu\013er\))117 1073 y(IN)155 b Fj(dest)511 b Ff(Second)15 b(pro)q(cess)h(in)d(pair)117 1148 y(IN)155 b Fj(tag)526 b Ff(T)m(ag)13 b(for)h(use)g(in)g(comm)o (unicatio)o(ns)e(and)h(for)h(uniqueness)117 1222 y(OUT)108 b Fj(bu\013er)478 b Ff(Bu\013er)15 b(creaated)h(b)o(y)d(mak)o(e)p 1352 1222 13 2 v 14 w(shared)p 1483 1222 V 16 w(bu\013er)117 1297 y(OUT)108 b Fj(sha)o(redBu\013Handle)256 b Ff(Handle)14 b(to)g(describ)q(e)h(shared)g(ob)r(ject)75 1470 y Fk(int)23 b(MPI_Make_shared_buffer)e(\(MPI_Datatype)h(dataType,)h(int,)g(count,) 361 1526 y(MPI_Comm)g(comm,)g(int)h(source,)f(int)g(dest,)361 1583 y(void)h(*buffer,)e(MPI_Shared_buff)g(*sharedBuffHandle\))75 1696 y(MPI_MAKE_SHARED_BUFFER)f(\(DATA_TYPE,)h(COUNT,)h(COMM,)g (SOURCE,)g(DEST,)624 1752 y(BUFF_REF,)f(BUFFER_IX,)h (SHARED_BUFF_HANDLE,)e(IERROR\))147 1809 y(INTEGER)h(DATA_TYPE)147 1865 y(INTEGER)g(COUNT)147 1922 y(INTEGER)g(COMM)147 1978 y(INTEGER)g(SOURCE)147 2034 y(INTEGER)g(DEST)147 2091 y(INTEGER)g(BUFF_REFF\(*\))147 2147 y(INTEGER)g(BUFFER_IX)147 2204 y(INTEGER)g(SHARED_BUFF_HANDLE)147 2260 y(INTEGER)g(IERROR)166 2365 y Fq(This)f(routine)h(creates)e(a)h(bu\013er)g(shared)g(b)q(et)o (w)o(een)g(t)o(w)o(o)e(pro)q(cesses.)38 b(The)21 b(pro)q(cess)g(iden)o (ti\014ed)75 2421 y(b)o(y)c(source)g(initially)j("o)o(wns")c(the)h (bu\013er.)26 b(Source)18 b(ma)o(y)e(write)h(to)g(the)g(bu\013er)g(and) h(manipulate)g(the)75 2478 y(bu\013er)c(in)i(an)o(y)e(w)o(a)o(y)f(that) h(mak)o(es)g(sense)h(with)g(standard)f(language)h(access)f(\(assignmen) o(t)g(statemen)o(ts,)75 2534 y(functions)19 b(calls,)g(etc.\).)28 b(When)19 b(source)f(is)g(done)h(with)f(the)g(bu\013er,)h(it)f(m)o(ust) g(release)g(the)h(bu\013er)f(for)75 2591 y(dest)i(to)f(acquire)h(it.)34 b(Dest)19 b(\(and)h(only)g(dest\))f(can)h(acquire)h(the)e(bu\013er)h (once)g(released)h(b)o(y)e(source.)75 2647 y(Dest)d(then)h(has)f(the)g (same)h(priviledgs)h(as)e(source)g(previously)i(had.)24 b(T)l(o)16 b(access)h(a)f(released)h(bu\013er)f(is)75 2704 y(considered)h(to)d(b)q(e)i(erroneous.)k(This)c(function)g(needs)g (to)e(b)q(e)i(view)o(ed)g(as)f("bi-collectiv)o(e".)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 38 40 38 39 bop 75 -100 a Fq(38)1138 b Fh(CHAPTER)15 b(9.)30 b(MISCELLANY)166 45 y Fd(Discussion:)40 b Ff(Do)15 b(w)o(e)h(w)o(an)o (t)f(to)h(ha)o(v)o(e)f(a)g(v)o(ersion)h(that)f(allo)o(ws)g(the)h(user)g (to)g(supply)f(a)g(bu\013er.)24 b(In)16 b(that)75 102 y(ev)o(en)o(t)f(on)f(a)g(distributed)h(memory)d(mac)o(hine)h(this)i(b)q (ecomes)g(a)f("safe")g(in)o(terface)h(to)f(ready)h(send,)g(on)f(a)g (shared)75 158 y(memory)d(mac)o(hine)i(it)g(ma)o(y)f(in)o(tro)q(duce)j (extra)f(copies.)75 356 y Fj(MPI)p 160 356 14 2 v 16 w(A)o(CQUIRE)p 367 356 V 17 w(SHARED)p 560 356 V 18 w(BUFFER)i(\(sha)o (redBu\013Handle\))117 433 y Ff(INOUT)62 b Fj(sha)o(redBu\013Handle)256 b Ff(A)14 b(shared)h(bu\013er)g(handle)75 603 y Fk(int)23 b(MPI_Acquire_shared_buffer)e(\(MPI_Shared_buff)h(*sharedBuffHandle\)) 75 716 y(MPI_ACQUIRE_SHARED_BUFFER)e(\(SHARED_BUFF_HANDLE,)i(IERROR\)) 147 772 y(INTEGER)g(SHARED_BUFF_HANDLE)147 829 y(INTEGER)g(IERROR)166 930 y Fq(This)16 b(routine)f(w)o(aits)g(for)g(the)g(shared)g(bu\013er)g (to)g(b)q(e)h(released)g(b)o(y)f(the)g(partner.)20 b(It)15 b(is)h(a)f(blo)q(c)o(king)75 987 y(call.)38 b(When)21 b(it)g(returns,)g(the)g(bu\013er)g(asso)q(ciated)g(with)g(the)g(shared) g(bu\013er)f(handle)i(can)f(b)q(e)h(fully)75 1043 y(accessed.)e(A)12 b(subsequen)o(t)h(acquire)g(without)f(an)h(in)o(terv)o(ening)g(release) g(is)g(erroneous)f(\(this)g(includes)j(an)75 1100 y(acquire)g(b)o(y)g (the)g(source)f(immediately)j(after)d(the)g(mak)o(e)p 1061 1100 V 16 w(shared)p 1206 1100 V 17 w(bu\013er\).)19 b(An)c(acquire)g(will)i(correctly)75 1156 y(return)e(only)h(when)g(the) f(partner)g(pro)q(cess)g(p)q(erforms)g(a)g(release.)75 1307 y Fj(MPI)p 160 1307 V 16 w(RELEASE)p 366 1307 V 17 w(SHARED)p 559 1307 V 18 w(BUFFER)h(\(sha)o(redBu\013Handle,)g (releaseAction\))117 1384 y Ff(INOUT)62 b Fj(sha)o(redBu\013Handle)256 b Ff(A)14 b(shared)h(bu\013er)g(handle)117 1459 y(IN)155 b Fj(releaseAction)338 b Ff(Indicate)11 b(whether)g(a)f(pro)q(duce)h (\(MPI)p 1501 1459 13 2 v 16 w(PR)o(ODUCE\))f(or)g(con-)905 1515 y(sume)i(\(MPI)p 1109 1515 V 15 w(CONSUME\))i(action)e(is)g(asso)q (ciated)h(with)f(the)905 1572 y(release)75 1741 y Fk(int)23 b(MPI_Release_shared_buffer)e(\(MPI_Shared_buff)h(*sharedBuffHandle,) 314 1798 y(int)h(releaseAction\))75 1911 y(MPI_RELEASE_SHARED_BUFFER)d (\(SHARED_BUFF_HANDLE,)i(RELEASE_ACTION,)f(IERROR\))147 1967 y(INTEGER)h(SHARED_BUFF_HANDLE)147 2024 y(INTEGER)g (RELEASE_ACTION)147 2080 y(INTEGER)g(IERROR)166 2182 y Fq(This)11 b(routine)h(releases)f(a)g(shared)g(bu\013er)g(so)f(that)g (it)h(can)g(b)q(e)h(acquired)g(b)o(y)f(the)f(partner)h(pro)q(cess.)19 b(If)75 2238 y(the)11 b(pro)q(cess)h(has)f(only)h(read)f(the)g (bu\013er)h(asso)q(ciated)f(with)h(the)f(shared)g(bu\013er)h(handler,)g (it)g(ma)o(y)e(use)i(the)75 2295 y(releaseAction)17 b(of)f(MPI)p 500 2295 14 2 v 16 w(CONSUME.)g(If)g(the)h(pro)q(cess)f(has)g(written)g (to)f(the)h(bu\013er)g(asso)q(ciated)h(with)75 2351 y(the)d(shared)g (bu\013er)h(handler,)g(it)f(MUST)g(use)g(the)h(releaseAction)g(of)f (MPI)p 1369 2351 V 16 w(PR)o(ODUCE)g(to)g(guaran)o(tee)75 2407 y(that)g(the)i(partner)f(sees)g(the)g(c)o(hanges.)75 2558 y Fj(MPI)p 160 2558 V 16 w(FREE)p 285 2558 V 17 w(SHARED)p 478 2558 V 17 w(BUFFER)h(\(sha)o(redBu\013Handle\))117 2636 y Ff(INOUT)62 b Fj(sha)o(redBu\013Handle)256 b Ff(A)14 b(shared)h(bu\013er)g(handle)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 39 41 39 40 bop 75 -100 a Fh(9.12.)29 b(PR)o(OPOSAL:)16 b(TW)o(O)f(PR)o (OCESS)h("SHARED")f(BUFFERS)526 b Fq(39)75 45 y Fk(int)23 b(MPI_Free_shared_buffer)e(\(MPI_Shared_buff)h(*sharedBuffHandle\))75 158 y(MPI_FREE_SHARED_BUFFER)f(\(SHARED_BUFF_HANDLE,)g(IERROR\))147 214 y(INTEGER)h(SHARED_BUFF_HANDLE)147 271 y(INTEGER)g(IERROR)166 377 y Fq(This)e(function)h(frees)e(the)h(bu\013er)g(asso)q(ciated)f (with)h(the)g(shared)g(bu\013er)g(handler,)h(the)f(handle)75 434 y(itself,)d(and)g(satis\014es)g(an)o(y)f(outstanding)h(comm)o (unications.)25 b(This)17 b(function)g(needs)h(to)e(b)q(e)h(view)o(ed)g (as)75 490 y("bi-collectiv)o(e".)75 596 y Fk(Possible)23 b(Send/Recv)f(Implementation:)75 709 y(int)h(MPI_Make_shared_buffer)e (\(MPI_Datatype)h(dataType,)h(int,)g(count,)361 766 y(MPI_Comm)g(comm,) g(int)h(source,)f(int)g(dest,)361 822 y(void)h(*buffer,)e (MPI_Shared_buff)g(*sharedBuffHandle\))123 879 y({)123 935 y(/*)h(Ok,)h(ok)f(this)h(isn't)f(really)g(C,)g(but)h(you)f(get)h (the)f(idea\))g(*/)123 992 y(sharedBuffHandle)e(=)j(malloc)f (\(sizeof\(sharedBuffHandle\)\);)123 1048 y(*buffer)g(=)g(malloc)g (\(sizeof\(dataType\))f(*)i(count\);)123 1104 y (sharedBuffHandle->buffer)c(=)k(buffer;)123 1161 y (sharedBuffHandle->count)c(=)k(count;)123 1217 y (sharedBuffHandle->dataTy)o(pe)d(=)j(dataType;)e(/*)i(Dup)f(here?)h(*/) 123 1274 y(sharedBuffHandle->tag)d(=)i(getSharedBuffTag\(\);)f(/*)h (Some)h(magic?)f(*/)123 1330 y(/*)g(At)h(this)f(point,)g(a)h(robust)f (application)f(might)i(check)f(with)g(it's)g(partner)194 1387 y(to)h(see)f(if)h(the)f(parameters)g(are)g(correct)g(and)h(the)f (malloc)g(worked.)g(But)h(it)194 1443 y(would)f(take)h(too)f(much)h (space)f(\(besides)g(if)g(programming)f(text)i(books)194 1500 y(can)g(omit)f(error)g(checking,)g(why)g(can't)h(I?)f(:-\))h(*/) 123 1556 y(if)f(\(myProc)g(==)h(src\))170 1613 y({)170 1669 y(Recv)g(\(comm,)f(sharedBuffHandle->tag,)e(dest,)i(\(count=0,)f (buff=NULL,...\)\);)170 1725 y(sharedBuffHandle->partner)f(=)j(dest;) 170 1782 y(})123 1838 y(else)170 1895 y({)170 1951 y (sharedBuffHandle->waitHandle)c(=)k(IRECV)f(\(comm,)g (sharedBuffHandle->tag,)1077 2008 y(source,)g(buffer,)g(count,...\))170 2064 y(Send)h(\(comm,)f(sharedBuffHandle->tag,)e(source,)i(\(count=0,)f (buff=NULL,...\)\);)170 2121 y(sharedBuffHandle->partner)f(=)j(source;) 170 2177 y(})123 2234 y(return;)123 2290 y(})75 2403 y(int)f(MPI_Acquire_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle\))123 2459 y({)123 2516 y(wait)h (\(sharedBuffHandle->waitHan)o(dle\);)123 2572 y(})75 2685 y(int)g(MPI_Release_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle,)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 40 42 40 41 bop 75 -100 a Fq(40)1138 b Fh(CHAPTER)15 b(9.)30 b(MISCELLANY)314 45 y Fk(int)23 b(releaseAction\))123 102 y({)123 158 y(if)g(\(releaseAction)f(==)i(MPI_CONSUME\))170 214 y(count)g(=)f(0;)123 271 y(else)170 327 y(count)h(=)f (sharedBuffHandle->count;)123 384 y(sharedBuffHandle->waitHa)o(ndle)e (=)i(IRECV)h(\(comm,)f(sharedBuffHandle->tag,)1030 440 y(sharedBuffHandle->partn)o(er,)1030 497 y(sharedBuffHandle->buffe)o (r,)e(count,...\);)123 553 y(/*)i(Note)h(that)f(a)h(non-buffering)e (send)h(can)h(be)f(used)g(here)h(*/)123 610 y(/*)f(Note)h(that)f(a)h ("ready")f(protocol)f(can)i(also)f(be)h(used)f(since)g(we)h(guarantee) 194 666 y(the)g(receive)f(is)g(always)g(posted)g(before)g(the)h(send)f (*/)123 723 y(localHandle)f(=)i(ISSEND)f(\(comm,)g (sharedBuffHandle->tag,)e(sharedBuffHandle->partne)o(r,)648 779 y(sharedBuffHandle->buffer)o(,)g(count,...\);)123 835 y(freeHandle)h(\(localHandle\);)123 892 y(})75 1005 y(int)h(MPI_Free_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle\))123 1061 y({)123 1118 y(/*)170 1174 y(Either)h(cancel)h(the)f(outstanding)f(receive)h(or)h(\(if)f(you)h (don't)f(beleive)170 1231 y(cancel)g(works\))h(send)f(to)g(complete)g (the)h(receive.)123 1287 y(*/)123 1344 y(/*)170 1400 y(Free)g(all)f(structures)123 1456 y(*/)123 1513 y(})75 1619 y(Possible)g(Shared)g(Memory)g(Implementation:)75 1732 y(int)g(MPI_Make_shared_buffer)e(\(MPI_Datatype)h(dataType,)h (int,)g(count,)361 1789 y(MPI_Comm)g(comm,)g(int)h(source,)f(int)g (dest,)361 1845 y(void)h(*buffer,)e(MPI_Shared_buff)g (*sharedBuffHandle\))123 1901 y({)123 1958 y(/*)h(Ok,)h(ok)f(this)h (isn't)f(really)g(C,)g(but)h(you)f(get)h(the)f(idea\))g(*/)123 2014 y(sharedBuffHandle)e(=)j(malloc)f(\(sizeof\(sharedBuffHandle\)\);) 123 2071 y(if)g(\(myProc)g(==)h(source\))170 2127 y({)170 2184 y(sharedBuffHandle->semaphore)d(=)457 2240 y(shared_malloc)h (\(sizeof\(dataType\))f(*)j(count)f(+)h(cacheLineSize\);)170 2297 y(})123 2353 y(sharedBuffHandle->buffer)c(=)k(buffer;)123 2410 y(sharedBuffHandle->count)c(=)k(count;)123 2466 y(sharedBuffHandle->dataTy)o(pe)d(=)j(dataType;)e(/*)i(Dup)f(here?)h (*/)123 2522 y(if)f(\(myProc)g(==)h(source\))170 2579 y({)170 2635 y(i)g(=)g(myProc;)170 2692 y(memcpy)f (\(sharedBuffHandle->semaphore,)d(&i,)k(sizeof\(int\)\);)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 41 43 41 42 bop 75 -100 a Fh(9.12.)34 b(PR)o(OPOSAL:)16 b(TW)o(O)f(PR)o (OCESS)h("SHARED")f(BUFFERS)521 b Fq(41)170 45 y Fk (sharedBuffHandle->partner)21 b(=)j(dest;)170 102 y(Send)g(\(comm,)f (sharedTag,)f(dest,)h(sharedBuffHandle->semaphore,)e(MPI_Pointer,)h (...\);)170 158 y(})123 214 y(else)170 271 y({)170 327 y(sharedBuffHandle->partner)f(=)j(source;)170 384 y(Recv)g(\(comm,)f (sharedTag,)f(source,)h(sharedBuffHandle->semaphore,)457 440 y(MPI_Pointer,)f(...\);)170 497 y(})123 553 y(*buffer)h(=)g (\(\(char)g(*\))h(sharedBuffHandle->semaphor)o(e\))d(+)j (cacheLineSize;)123 610 y(})75 723 y(int)f(MPI_Acquire_shared_buffer)e (\(MPI_Shared_buff)h(*sharedBuffHandle\))123 779 y({)123 835 y(i)h(=)h(myProc)f(-)h(1;)123 892 y(while)f(\(i)g(!=)h(myProc\))170 948 y({)170 1005 y(memcpy)f(\(&i,)h(sharedBuffHandle->semapho)o(re,)d (sizeof\(int\)\);)170 1061 y(/*)j(A)g(smart)f(implementation)f(would)h (release)g(it's)g(timeslice)242 1118 y(after)g(some)h(number)f(of)g (times)g(through)g(this)h(loop)f(*/)170 1174 y(})123 1231 y(})75 1344 y(int)g(MPI_Release_shared_buffer)e(\(MPI_Shared_buff) h(*sharedBuffHandle,)314 1400 y(int)h(releaseAction\))123 1456 y({)123 1513 y(i)g(=)h(sharedBuffHandle->partner;)123 1569 y(memcpy)f(\(sharedBuffHandle->semap)o(hore,)e(&i,)i (sizeof\(int\)\);)123 1626 y(})75 1739 y(int)g(MPI_Free_shared_buffer)e (\(MPI_Shared_buff)h(*sharedBuffHandle\))123 1795 y({)123 1852 y(/*)170 1908 y(Free)i(all)f(structures)123 1965 y(*/)123 2021 y(})-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Fri Jul 5 15:49:49 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA11856; Fri, 5 Jul 1996 15:49:48 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA25476 for mpi-core-out; Fri, 5 Jul 1996 14:51:48 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA25471 for ; Fri, 5 Jul 1996 14:51:40 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id OAA07157; Fri, 5 Jul 1996 14:51:38 -0500 Date: Fri, 5 Jul 1996 14:51:38 -0500 Message-Id: <199607051951.OAA07157@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. These directories also has all the sources and figures. The document is now almost 300 pages so send it to a big printer. This version, probably due to the holidays, is missing some updates. I expect to release another version early next week with the final updates. The chapters I think need updating are collective, language binding and one sided. If you want to make changes for the version to be printed for the meeting, I need to receive them by 10:00 AM (central US time) on Friday, 12 July. Steve From mpi-core-human@mcs.anl.gov Fri Jul 5 17:03:11 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA15705; Fri, 5 Jul 1996 17:03:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA26626 for mpi-core-out; Fri, 5 Jul 1996 16:06:04 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA26621 for ; Fri, 5 Jul 1996 16:05:54 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id QAA08188; Fri, 5 Jul 1996 16:05:53 -0500 Date: Fri, 5 Jul 1996 16:05:53 -0500 Message-Id: <199607052105.QAA08188@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: problem with document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My quick scan of the document missed the fact that page 179 is very messed up. Unfortunitely I need to leave right now. I will try and fix this problem soon and repost when it is fixed. Sorry for the mistake. Steve From mpi-core-human@mcs.anl.gov Mon Jul 8 05:12:25 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA01790; Mon, 8 Jul 1996 05:12:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA17895 for mpi-core-out; Mon, 8 Jul 1996 04:13:39 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id EAA17889 for ; Mon, 8 Jul 1996 04:13:18 -0500 Received: by felix.dircon.co.uk id AA22111 (5.67b/IDA-1.5 for ); Mon, 8 Jul 1996 10:13:09 +0100 Message-Id: <199607080913.AA22111@felix.dircon.co.uk> Received: from gw2-185.pool.dircon.co.uk(194.112.35.185) by amnesiac via smap (V1.3) id sma022094; Mon Jul 8 10:13:01 1996 Received: from localhost by jim (SMI-8.6) id JAA00628; Mon, 8 Jul 1996 09:41:08 +0100 To: mpi-core@mcs.anl.gov Subject: What is the MPI 2 document ? Reply-To: jcownie@bbn.com Date: Mon, 08 Jul 1996 09:41:08 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk It seems to me that it is about time that we decided what whether we're intending the MPI-2 document to be complete in itself, or an add on to the MPI-1.x document, since this has significant implications. If we were to follow the style of other standards which have changed (for instance Fortran), then we would produce a single self contained MPI (97 ?) standard, which would consist of a union of the existing MPI-1(.x) document and the new chapters which are being created in MPI-2. However at the moment we seem to be expecting to produce a new MPI-2 document which adds to the MPI-1 document. Users then need both documents (and a meta-index ?) to use the full power (and glory :-)) of MPI. Creating a merged MPI-97 document is clearly more work than leaving the MPI-1.x document alone and adding to it. However it has a number of benefits 1) MPI-2 features which extend MPI-1 can be placed in the logical places in the document. (e.g. new data-type constructors, testrecv additional collective ops ...) 2) Where we want to deprecate some of the MPI-1 usages (because there are now better ways of achieving the same thing), this can be easily inserted and users will learn the better way, rather than never noticing it because it's in a different document. 3) The index can be consistent. 4) Users need only one document to find out all about MPI. 5) Producing the merged document forces us to look closely at the MPI-2 extensions in the context of MPI-1. Clearly we would need to ensure that MPI-2 features are somehow flagged in such a document because they will likely not be implemented yet. This also does not address the issue of whether we're going to have optional sections. -- Jim James Cownie BBN UK Ltd Phone : +44 117 9071438 E-Mail: jcownie@bbn.com From mpi-core-human@mcs.anl.gov Mon Jul 8 14:08:52 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA10516; Mon, 8 Jul 1996 14:08:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA25545 for mpi-core-out; Mon, 8 Jul 1996 13:10:02 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA25540 for ; Mon, 8 Jul 1996 13:09:53 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id NAA10882; Mon, 8 Jul 1996 13:09:51 -0500 Date: Mon, 8 Jul 1996 13:09:51 -0500 Message-Id: <199607081809.NAA10882@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: new version of report Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A corrected and more complete version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. These directories also have all the sources and figures. The document is now almost 300 pages so send it to a big printer. This version has an updated one side chapter as well as an appendix which represents an alternative to this chapter. The binding and collective chapters have not been updated but are expected in the final version at the end of the week. If you want to make changes for the version to be printed for the meeting, I need to receive them by 10:00 AM (central US time) on Friday, 12 July. Steve From mpi-core-human@mcs.anl.gov Tue Jul 9 13:54:53 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA03215; Tue, 9 Jul 1996 13:54:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA13938 for mpi-core-out; Tue, 9 Jul 1996 12:51:09 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA13929 for ; Tue, 9 Jul 1996 12:50:57 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id MAA10345; Tue, 9 Jul 1996 12:50:55 -0500 Date: Tue, 9 Jul 1996 12:50:55 -0500 Message-Id: <199607091750.MAA10345@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: request, status, cancel Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk While trying to tie up loose ends in the external chapter, I looked at the other chapters in MPI-2. As a result, I am becoming concerned about requests, status and cancel. I am sorry to post to mpi-core, but the topic crosses many chapters. Here is my summary of where they get used/mentioned. Sorry if it has mistakes, there was a lot of text to look at: Dynamic ------- ISPAWN variants: status has error code/other fields undefined. Cancel is not allowed MONITOR,IACCEPT,ICONNECT: what status returns and cancelling not defined One-sided --------- IPUT, IGET, IACCUMULATE, ICONSUME_COUNTER: what status returns and cancelling not defined Collective ---------- all non-blocking variants: status not undefined, cancel only if all processes use non-blocking variants and ignores cancel by default persistant collective, ICALL, channels: what status returns and cancelling not defined External Interfaces ------------------- Generalize request: status and cancel discussed but nothing concrete IO -- IREAD, IWRITE variants: return size in status, cancel undefined I think we are heading for problems since we keep reusing request and status and it means different things at different points in MPI-2. We already overloaded requests in MPI-1 by allowing non-blocking and persistant requests. However, there they all related to point-to-point communication. Now we are further overloading requests and they don't always involve point-to-point communication. For requests I already see two choices: 1) continue to overload them: advantage is that the current MPI functions can be used, e.g., START, WAIT, ... 2) create new objects: maybe cleaner but orphans persistant requests and requires new functions. We also use wait on all these requests. Many of the requests do not have meaningful information to put in the status fields and also have other information they want to return. This issue if further complicated by the fact that we made part of the status field be non-opaque. Here are some possibilities: 1) NULL out the fields that are not relavant: this means that the GET_COUNT, etc calls need to check for this. This does not solve the need for additional information. This is not exclusive of other choices. 2) Add a new pointer field so can add info: We have discussed that we want status to be fast so making the status object bigger may cause more cache misses in the MPI-1 cases were the pointer is not used. 3) reuse the fields: First, this is ugly and dangerous. How will GET_COUNT know that the fields have special data and not try and interprete them. Second, this limits the amount of information that can be returned. It is not clear that it will meet everyones needs. 4) create new object(s) Finally, there is cancel. It may be fine to allow each call to decide what is allowed/not. However, we are going to have to mark the requests so we know what to do. We all need to determine what we want for our own sections. I do not have slick ideas here. I just want to get the discussion going. This effects lots of chapters and they currently are not consistant. Steve From mpi-core-human@mcs.anl.gov Wed Jul 10 13:19:51 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA13588; Wed, 10 Jul 1996 13:19:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA02478 for mpi-core-out; Wed, 10 Jul 1996 12:14:38 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA02472 for ; Wed, 10 Jul 1996 12:14:28 -0500 Message-Id: <199607101714.MAA02472@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: July MPI Forum meeting Date: Wed, 10 Jul 1996 12:14:27 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, The next meeting of the MPI Forum will take place on Wed.- Fri., July 17-19, at the Ramada Hotel O'Hare (same place as the last few meetings). It is time to make your reservations. Register at the hotel by calling (847) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). Please let me know as soon as possible if you are coming, so that I can give the hotel a head count. Thanks. When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. We will follow the usual schedule, beginning at 1:30 on Wednesday and ending at noon on Friday. We will charge the usual registration fee of $75, which should be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. If you have any problems or questions, let me know, either by email (lusk@mcs.anl.gov) or phone (708-252-7852). Here is the schedule of future MPI Forum meetings, which are also scheduled for the Ramada O'Hare. Sep 4-6 Oct 9-11 Supercomputing '96 is Nov 17-22. Regards, Rusty From mpi-core-human@mcs.anl.gov Wed Jul 10 14:43:31 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA16148; Wed, 10 Jul 1996 14:43:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA04564 for mpi-core-out; Wed, 10 Jul 1996 13:45:39 -0500 Received: from tbag.osc.edu (tbag.osc.edu [128.146.36.50]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA04549 for ; Wed, 10 Jul 1996 13:45:17 -0500 Received: for gdburns@tbag.osc.edu by tbag.osc.edu (8.7.1/950822.1) id OAA12281; Wed, 10 Jul 1996 14:45:13 -0400 (EDT) Date: Wed, 10 Jul 1996 14:45:13 -0400 (EDT) From: Greg Burns Message-Id: <199607101845.OAA12281@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: if impl-a then mpi-a else mpi-b Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk ...from generalized requests... > 1) If your implementation has threads then your request progresses > without any MPI calls on that process. > > 2) If you are emulating threads, then your request may only progress > if you call an MPI function. Also, issuing a blocking call in your > handler is likely to hang your code. ...and from 1-sided... >MPI_WINDOW_RELAXED_MODEL Boolean variable that indicates whether > the "relaxed-model" is followed. > >If the value is set to 1, the 1-sided usage is restricted to the >relaxed-model on that window (i.e. the window owner must participate by >calling MPI for lock/unlock operations to complete). If the key is not >defined, or the value is set to 0, the strict shared memory model can be >assumed by the user. ...both aptly summarized by Raja as... >It's the "big tent" inclusive view, with >different qualities of implementation. I am against the big tent. It's a nice way of achieving consensus on the Forum but it is not good for MPI. I have enjoyed telling users: "It's MPI; it's a standard; just recompile and run." Yes, there are caveats wrt other aspects of the code, but at least not [big] caveats within MPI itself. Proposals like the ones above place an important fork on the application development path. Furthermore, when I buy a canned MPI app, it will have to be marked "requires thread-capable implementation". I will have to match it to an implementation that is clearly marked, "thread-capable". All of this weakens the power of the MPI standard. We are already in for a substantial period where any conversation about MPI includes clarification about how much of MPI-2 is intended. Quality of Implementation cannot be tied to functionality. It can only mean things like how fast it runs, how gracefully it dies, etc. "Extent of Implementation" is a more accurate term for the above proposals. -- Greg From mpi-core-human@mcs.anl.gov Fri Jul 12 12:37:28 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA18656; Fri, 12 Jul 1996 12:37:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA13330 for mpi-core-out; Fri, 12 Jul 1996 11:39:12 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA13325 for ; Fri, 12 Jul 1996 11:39:04 -0500 Message-Id: <199607121639.LAA13325@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Final head count for next week's MPI meeting Date: Fri, 12 Jul 1996 11:39:03 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I need to give the hotel a count early this afternoon for the chairs and meals for the MPI Forum meeting next week at the Ramada O'Hare. Below is the list of people who have told me they are coming. This is fewer than half of the number who came to the June meeting. Is everyone else going to be on vacation? Is Cray really going to be represented by Eric? Has Tony been eaten by an alligator? Has Notre Dame been obliterated by Ohio State? Please let me know right away if you plan to come but are not on this list. Rusty Ron Brightwell Joel Clark Dennis Cottel James Cownie Raja Daoud David DiNucci Nathan Doss Hans-Christian Hoppe Steve Huss-Lederman Arkady Kanevsky Koichi Konishi Rusty Lusk Nick Nevin Perry Partow Elsie Pierce Jean-Pierre Prost Rolf Rabenseifner Anna Rounbehler Eric Salo William Saphir Marc Snir Alan Sussman Rajeev Thakur Dick Treumann Klaus Wolf Parkson Wong Dave Wright From mpi-core-human@mcs.anl.gov Fri Jul 12 13:28:33 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA20384; Fri, 12 Jul 1996 13:28:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA14312 for mpi-core-out; Fri, 12 Jul 1996 12:30:03 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA14307 for ; Fri, 12 Jul 1996 12:29:51 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id MAA09438; Fri, 12 Jul 1996 12:29:48 -0500 Date: Fri, 12 Jul 1996 12:29:48 -0500 Message-Id: <199607121729.MAA09438@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document for meeting available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The version to be printed and distributed at the next meeting is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. These directories also have all the sources and figures. The document is now almost 300 pages so send it to a big printer. Most chapters were updated since the draft a week ago. Also, the collective chapter has had no revision since the last meeting so I left in the old text. Steve From mpi-core-human@mcs.anl.gov Wed Jul 17 12:27:44 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA06368; Wed, 17 Jul 1996 12:27:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA29254 for mpi-core-out; Wed, 17 Jul 1996 11:28:38 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA29248; Wed, 17 Jul 1996 11:28:22 -0500 Received: from aurora.cs.msstate.edu (aurora.cs.msstate.edu [130.18.208.91]); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id LAA06446; Wed, 17 Jul 1996 11:28:26 -0500 Date: Wed, 17 Jul 1996 11:28:25 -0500 (CDT) From: Anthony Skjellum To: mpi-core@mcs.anl.gov, mpi-comm@mcs.anl.gov Subject: Collective chapter Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As the person with the Mac/Eudora complains when anyone puts postscript on these reflectors, I have placed the new collective chapter. aurora.cs.msstate.edu, pub/mpi/collective-17jul96.ps (or .ps.Z) Those on the mpi-coll group got the postscript also. -Tony Anthony Skjellum, PhD, Asst. Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Mon Jul 22 20:24:17 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA08192; Mon, 22 Jul 1996 20:24:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA26657 for mpi-core-out; Mon, 22 Jul 1996 19:22:52 -0500 Received: from lsc.nd.edu (root@lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA26651; Mon, 22 Jul 1996 19:22:42 -0500 Received: from porthos.lsc.nd.edu (porthos.lsc.nd.edu [129.74.25.91]) by lsc.nd.edu (8.7.4/8.7.3) with ESMTP id TAA01769; Mon, 22 Jul 1996 19:22:39 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.7.5/8.7.3) with SMTP id TAA03188; Mon, 22 Jul 1996 19:22:38 -0500 (EST) Date: Mon, 22 Jul 1996 19:22:37 -0500 (EST) From: Jeff Squyres To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: C++ profiling example Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Since there were several concerns raised about profiling in the July meeting, I have written a short example implementation of the profiling using the proposed C++ bindings. For the user, it works just like the C profiling: the user only needs to *relink* their code with the additional profiling library -- the user does not need to recompile. That is, normally the user would do something like: g++ mycode.o -L/usr/local/mpi -lmpi++ -o mycode To profile, they can do: g++ mycode.o -L/usr/local/mpi -lpmpi++ -lmpi++ -o mycode A tarfile containing the source code of the example implementation is available from: http://www.cse.nd.edu/~lsc/mpi2/ The example only shows the functions MPI_COMM_SIZE and MPI_BUFFER_ATTACH, and the constants MPI_COMM_WORLD and MPI_SUCCESS. There is a simple configure script and makefile that should be suitable for most flavors of Unix. There is also a README file which explains how it works in detail. There should also be ample comments in the code. Note: to override the default C++ compiler that configure chooses, set the environment variable CXX before running ./configure. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness. {+} "I came to ND for 4 years and ended up staying for a decade." From mpi-core-human@mcs.anl.gov Wed Jul 24 12:08:11 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA13589; Wed, 24 Jul 1996 12:08:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA29369 for mpi-core-out; Wed, 24 Jul 1996 11:09:28 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA29362 for ; Wed, 24 Jul 1996 11:09:18 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id LAA25905; Wed, 24 Jul 1996 11:09:17 -0500 Date: Wed, 24 Jul 1996 11:09:17 -0500 Message-Id: <199607241609.LAA25905@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: Deadlines for next meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here are the document deadlines for the September meeting. NOTE: THE DATES ARE DIFFERENT THAN PREVIOUS MEETINGS. THIS IS DUE TO MOVING THE MEETING UP ONE DAY, LABOR DAY WEEKEND AND MY TRAVEL. BECAUSE OF MY COMMITMENTS AND THE HOLIDAY, IT WILL BE DIFFICULT FOR ME TO GIVE EXTRA TIME. FOR THOSE NOT PRESENT AT THE LAST MEETING, WE VOTED TO ADD ONE DAY TO THE SEPTEMBER AND OCTOBER MEETING. THUS, WE WILL BEGIN ON TUESDAY AT 1:30 AND GO TO FRIDAY 12:00. Friday, August 2: Informal date for chapter authors to get out a new draft of their chapters which reflect changes from the July meeting. Tuesday, August 20: A first cut at the next version of each chapter is due to me by 12:00 PM CST (noon). When the full document is ready it will be announced on mpi-core and should be out by late afternoon. Tuesday, August 27: The final version of each chapter is due to me by 10:00 AM CST. This is the version that will be printed for the meeting. It will also be announced on mpi-core and should be out by late afternoon. The current (and future documents) are available via the web. See http://www.cs.wisc.edu/~lederman/mpi2 and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2. The full current document is in mpi2-report.ps.Z. The sources and scripts are also available. The official votes through the July meeting are available in the file votes-official. Steve From mpi-core-human@mcs.anl.gov Wed Jul 24 16:26:05 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA15660; Wed, 24 Jul 1996 16:26:05 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA05244 for mpi-core-out; Wed, 24 Jul 1996 15:27:50 -0500 Received: from hp.com (hp.com [15.255.152.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA05239 for ; Wed, 24 Jul 1996 15:27:39 -0500 Received: from mailhost.rsn.hp.com by hp.com with ESMTP (1.37.109.16/15.5+ECS 3.3) id AA153690040; Wed, 24 Jul 1996 13:27:21 -0700 Received: from hydra.rsn.hp.com by mailhost.rsn.hp.com with SMTP (1.37.109.18/15.5+ECS 3.4 ) id AA289009723; Wed, 24 Jul 1996 15:22:03 -0500 Received: by hydra.rsn.hp.com (5.64/1.28) id AA21406; Wed, 24 Jul 96 15:27:13 -0500 Date: Wed, 24 Jul 96 15:27:13 -0500 Message-Id: <9607242027.AA21406@hydra.rsn.hp.com> From: Greg Astfalk To: mpi-core@mcs.anl.gov Subject: Meeting schedule Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Since the next two metings are a day longer then normal it would be nice if a schedule for the various sessions were made available before the meeting. For those of us that are only interested/involved in one chapter, say MPI-IO for instance, it would be beneficial if we only had to attend for the, say, two days necessary. From mpi-core-human@mcs.anl.gov Wed Jul 24 16:59:01 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA15933; Wed, 24 Jul 1996 16:59:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA05993 for mpi-core-out; Wed, 24 Jul 1996 16:02:09 -0500 Received: from timbuk.cray.com (root@timbuk.cray.com [128.162.19.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA05988 for ; Wed, 24 Jul 1996 16:02:02 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.7.5/CRI-gate-8-2.11) with SMTP id PAA19262; Wed, 24 Jul 1996 15:57:48 -0500 (CDT) Received: from cherry340.cray.com (maggie@cherry340 [128.162.174.40]) by ironwood.cray.com (8.6.12/CRI-ccm_serv-8-2.8) with ESMTP id PAA27801; Wed, 24 Jul 1996 15:57:36 -0500 From: Margaret Cahir Received: by cherry340.cray.com (8.6.12/btd-b3) id PAA22047; Wed, 24 Jul 1996 15:57:35 -0500 Message-Id: <199607242057.PAA22047@cherry340.cray.com> Date: Wed, 24 Jul 1996 15:57:35 -0500 To: mpi-core@mcs.anl.gov Cc: astfalk@hydra.rsn.hp.com Subject: Re: Meeting schedule Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I'd like to second the motion. Having a schedule ahead of time would be enormously helpful. > Since the next two metings are a day longer then normal it would > be nice if a schedule for the various sessions were made > available before the meeting. For those of us that are only > interested/involved in one chapter, say MPI-IO for instance, it > would be beneficial if we only had to attend for the, say, two > days necessary. From mpi-core-human@mcs.anl.gov Thu Jul 25 08:20:43 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA24756; Thu, 25 Jul 1996 08:20:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA25105 for mpi-core-out; Thu, 25 Jul 1996 07:23:43 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA25098 for ; Thu, 25 Jul 1996 07:23:31 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id HAA01692; Thu, 25 Jul 1996 07:23:25 -0500 Date: Thu, 25 Jul 1996 07:23:25 -0500 Message-Id: <199607251223.HAA01692@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov In-reply-to: <199607242057.PAA22047@cherry340.cray.com> (message from Margaret Cahir on Wed, 24 Jul 1996 15:57:35 -0500) Subject: Re: Meeting schedule Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree that having a schedule would be nice. Rusty presented an outline at the last meeting - maybe he could post it. However, I think the idea of coming for your favorite parts of the meeting is a bad idea. One of the real strengths of MPI is its integration. The solutions reached by the full group are often better than the combination of all the subcommittees. The MPI Forum has not been the sum of the subcomittees meeting in isolation; it has been the integration of them all. Also, the upcoming meetings will have many meetings of the whole for official votes. Everyone should be at these. I feel that working on the MPI Forum is a commitment to the full standard, not just your favorite parts. Yes, parts of MPI-2 are not in my normal interest area, however, I read the document and attend the full meeting. It is in the interest of MPI and an extra benefit is I have learned a lot. Steve > Since the next two metings are a day longer then normal it would > be nice if a schedule for the various sessions were made > available before the meeting. For those of us that are only > interested/involved in one chapter, say MPI-IO for instance, it > would be beneficial if we only had to attend for the, say, two > days necessary. From mpi-core-human@mcs.anl.gov Thu Jul 25 10:14:35 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA27213; Thu, 25 Jul 1996 10:14:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA27844 for mpi-core-out; Thu, 25 Jul 1996 09:17:17 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA27835 for ; Thu, 25 Jul 1996 09:16:59 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id JAA02206; Thu, 25 Jul 1996 09:16:57 -0500 Date: Thu, 25 Jul 1996 09:16:57 -0500 Message-Id: <199607251416.JAA02206@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: REVISED Document schedule Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A consequence of having a larger document is that it takes longer to reproduce. Since I just found this out, I need to modify the dates I sent out yesterday. The deadlines move back one day. Attached is the revised notice. Steve ---------------------------------------------------------------------- Here are the document deadlines for the September meeting. NOTE: THE DATES ARE DIFFERENT THAN PREVIOUS MEETINGS. THIS IS DUE TO MOVING THE MEETING UP ONE DAY, LABOR DAY WEEKEND, AND REPRODUCING THE LARGER DOCUMENT. BECAUSE OF THE HOLIDAY AND REPRO DEADLINES, IT WILL BE DIFFICULT FOR ME TO GIVE EXTRA TIME. GETTING THE DOCUMENT LATE TO REPRO WILL COST HUNDREDS OF DOLLARS. FOR THOSE NOT PRESENT AT THE LAST MEETING, WE VOTED TO ADD ONE DAY TO THE SEPTEMBER AND OCTOBER MEETING. THUS, WE WILL BEGIN ON TUESDAY AT 1:30 AND GO TO FRIDAY 12:00. Friday, August 2: Informal date for chapter authors to get out a new draft of their chapters which reflect changes from the July meeting. Monday, August 19: A first cut at the next version of each chapter is due to me by 12:00 PM CST (noon). When the full document is ready it will be announced on mpi-core and should be out by late afternoon. Monday, August 26: The final version of each chapter is due to me by 10:00 AM CST. This is the version that will be printed for the meeting. It will also be announced on mpi-core and should be out by late afternoon. The current (and future documents) are available via the web. See http://www.cs.wisc.edu/~lederman/mpi2 and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2. The full current document is in mpi2-report.ps.Z. The sources and scripts are also available. The official votes through the July meeting are available in the file votes-official. From mpi-core-human@mcs.anl.gov Thu Jul 25 13:35:02 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA29558; Thu, 25 Jul 1996 13:35:01 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA04902 for mpi-core-out; Thu, 25 Jul 1996 12:36:51 -0500 Received: from relay1.UU.NET (relay1.UU.NET [192.48.96.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA04883 for ; Thu, 25 Jul 1996 12:36:39 -0500 Received: from sky.com by relay1.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQazxm24201; Thu, 25 Jul 1996 13:35:58 -0400 (EDT) Received: from sky (localhost) by sky.com (4.1/SMI-4.1-DNI-7.0.1) id AA06247; Thu, 25 Jul 96 13:35:57 EDT Message-Id: <31F7B07B.167EB0E7@sky.com> Date: Thu, 25 Jul 1996 13:35:55 -0400 From: Anna Rounbehler Organization: Sky Computers X-Mailer: Mozilla 3.0b5a (X11; I; SunOS 4.1.3 sun4) Mime-Version: 1.0 To: Steve Huss-Lederman Cc: mpi-core@mcs.anl.gov Subject: Re: Meeting schedule References: <199607251223.HAA01692@romano.cs.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve Huss-Lederman wrote: > > I agree that having a schedule would be nice. Rusty presented an > outline at the last meeting - maybe he could post it. However, I > think the idea of coming for your favorite parts of the meeting is a > bad idea. One of the real strengths of MPI is its integration. The > solutions reached by the full group are often better than the > combination of all the subcommittees. The MPI Forum has not been the > sum of the subcomittees meeting in isolation; it has been the > integration of them all. Also, the upcoming meetings will have many > meetings of the whole for official votes. Everyone should be at > these. I feel that working on the MPI Forum is a commitment to the > full standard, not just your favorite parts. Yes, parts of MPI-2 are > not in my normal interest area, however, I read the document and > attend the full meeting. It is in the interest of MPI and an extra > benefit is I have learned a lot. > > Steve > > > Since the next two metings are a day longer then normal it would > > be nice if a schedule for the various sessions were made > > available before the meeting. For those of us that are only > > interested/involved in one chapter, say MPI-IO for instance, it > > would be beneficial if we only had to attend for the, say, two > > days necessary. Steve, I think it is our English (or lack of it) that is probably at fault here. I think anyone who knows they can not commit to four days, would like to view the schedule so they can attend the committees of interest. It is like a rock and a hard spot - we know we must attend the meetings that we actively participate in, as well as be available for the others. For some people, this might turn out to be mutually exclusive. I think everyone wants this to work. Try not to be too hard on us. Let's try to find supporting media like email, etc to help those who can not attend the full four days. I am willing to help. The goal here is Supercomputing'96 and to show the good work that reflects the committment of the many that helped to make it happen. Anna ----------------------- Anna Rounbehler SKY Computer From mpi-core-human@mcs.anl.gov Thu Jul 25 19:13:04 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA02549; Thu, 25 Jul 1996 19:13:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA15390 for mpi-core-out; Thu, 25 Jul 1996 18:13:54 -0500 Received: from relay1.UU.NET (relay1.UU.NET [192.48.96.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA15384 for ; Thu, 25 Jul 1996 18:13:45 -0500 Received: from sky.com by relay1.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQazyi19026; Thu, 25 Jul 1996 19:13:03 -0400 (EDT) Received: from sky (localhost) by sky.com (4.1/SMI-4.1-DNI-7.0.1) id AA07634; Wed, 24 Jul 96 17:55:48 EDT Message-Id: <31F69BE2.167EB0E7@sky.com> Date: Wed, 24 Jul 1996 17:55:47 -0400 From: Anna Rounbehler Organization: Sky Computers X-Mailer: Mozilla 3.0b5a (X11; I; SunOS 4.1.3 sun4) Mime-Version: 1.0 To: Greg Astfalk Cc: mpi-core@mcs.anl.gov Subject: Re: Meeting schedule References: <9607242027.AA21406@hydra.rsn.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Greg Astfalk wrote: > > Since the next two metings are a day longer then normal it would > be nice if a schedule for the various sessions were made > available before the meeting. For those of us that are only > interested/involved in one chapter, say MPI-IO for instance, it > would be beneficial if we only had to attend for the, say, two > days necessary. A very good idea. Also, the chapters with more work to be done can have longer sessions to catch up. We probably need a show of hands as to who will be attending the full four days. This will help to organize the days and groups. I will be there all four days. Is it possible to set up email at the conference to allow members who can not attend to vote or to keep up? We can post the information hot off the press in the form of email after each session. I think I can bring a laptop, if it will help. Regards, Anna -------------------------------------------------------------- Anna Rounbehler SKY Computer (508) 250 1920 ex 248 anna@sky.com From mpi-core-human@mcs.anl.gov Wed Jul 31 17:54:09 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA29491; Wed, 31 Jul 1996 17:54:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA20655 for mpi-core-out; Wed, 31 Jul 1996 16:52:17 -0500 Received: from lsc.nd.edu (root@lsc.nd.edu [129.74.25.114]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA20638; Wed, 31 Jul 1996 16:51:59 -0500 Received: from porthos.lsc.nd.edu (porthos.lsc.nd.edu [129.74.25.91]) by lsc.nd.edu (8.7.4/8.7.3) with ESMTP id QAA20764; Wed, 31 Jul 1996 16:51:56 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.7.5/8.7.3) with SMTP id QAA05710; Wed, 31 Jul 1996 16:51:55 -0500 (EST) Date: Wed, 31 Jul 1996 16:51:55 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: New chapter 7 (fwd) Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Chapter 7 is now available in postscript from: http://www.cse.nd.edu/~lsc/mpi2/ Notable changes: - Much of the language was rewritten to be clearer. No content changed (except for the datatypes and profiling sections, see below) since it has been voted on once already. Change bars indicate where large sections of language changed. - Section 7.1.1 was split into sections 7.1.1 and a new section 7.1.2. Other section numbers were renumbered accordingly (this only matters for the formal record of the votes). - The datatypes section has been re-written to include all datatypes (C, C++, and Fortran). - The profiling section has been completely rewritten. We have tested the C++ profiling interface scheme; example code is available at the URL listed above. - Fortran constants have been added to the appendix. A new discussion item was added: when we added the Fortran constants, a question came up about MPI_BYTE and MPI_PACKED. It would seem to make sense that MPI_BYTE is the same in all languages, but about about MPI_PACKED? Because C and C++ datatypes are interchangable, it has been established that (C MPI_PACKED == C++ MPI::PACKED), but is (fortran MPI_PACKED == C++ MPI::PACKED)? That is, we already had to make MPI::F_COMPLEX to denote the difference between the fortran complex datatype and the C++ datatype MPI::COMPLEX. Do we need to create an MPI::F_PACKED? Not being implementors, we don't have a good answer to this question. Comments? {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness. {+} "I came to ND for 4 years and ended up staying for a decade." From mpi-core-human@mcs.anl.gov Mon Aug 5 13:54:43 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA11929; Mon, 5 Aug 1996 13:54:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA10711 for mpi-core-out; Mon, 5 Aug 1996 12:56:28 -0500 Received: from xipe-totec.cacr.caltech.edu (root@xipe-totec.cacr.caltech.edu [131.215.145.166]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA10702 for ; Mon, 5 Aug 1996 12:56:19 -0500 Received: from willow.cacr.caltech.edu (rajesh@willow.cacr.caltech.edu [131.215.151.104]) by xipe-totec.cacr.caltech.edu (8.6.12/8.6.4) with ESMTP id KAA11375 for ; Mon, 5 Aug 1996 10:55:12 -0700 Received: by willow.cacr.caltech.edu (8.6.12) id KAA08090; Mon, 5 Aug 1996 10:55:56 -0700 Date: Mon, 5 Aug 1996 10:55:45 -0700 (PDT) From: "Rajesh R. Bordawekar" X-Sender: rajesh@willow To: mpi-core@mcs.anl.gov Subject: MPI-2 Meeting Schedule Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk hi Can someone post tentative schedule of the MPI-2 meeting? I am specifically interested in the schedule of the I/O sub-committee. Thanks Rajesh From mpi-core-human@mcs.anl.gov Mon Aug 5 15:21:28 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA29293; Mon, 5 Aug 1996 15:21:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA13087 for mpi-core-out; Mon, 5 Aug 1996 14:19:57 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA13081 for ; Mon, 5 Aug 1996 14:19:49 -0500 Message-Id: <199608051919.OAA13081@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: September MPI Forum meeting Date: Mon, 05 Aug 1996 14:19:49 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, This is a reminder that the next MPI Forum meeting will be held at the currently usual place, at the Ramada Hotel O'Hare, on September 3-6. We voted at the last meeting to extend the next two meetings by an extra day. That is, we will start at 1:30 on *Tuesday*, September 3, and adjourn at noon on Friday, September 6. Beginning at this meeting, we will have to devote considerable time to plenary sessions, in order that official readings and votes can proceed on schedule, but there will also be time for subcommittee meetings. A tentative agenda will be put together shortly before the meeting, when it is more clear from the state of the chapters and the requirements of the chapter authors, after the next version of the draft has taken shape, how we should proceed. Rusty Lusk From mpi-core-human@mcs.anl.gov Mon Aug 12 18:22:44 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA10429; Mon, 12 Aug 1996 18:22:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA20049 for mpi-core-out; Mon, 12 Aug 1996 17:20:39 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA20032 for ; Mon, 12 Aug 1996 17:20:25 -0500 Message-Id: <199608122220.RAA20032@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: new introduction Date: Mon, 12 Aug 1996 17:20:23 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Enclosed is the updated version of the introduction chapter. It has a further explanation of why the new document is in three parts: the MPI-1.2 part, the MPI-2 part, and the MPI Journal of Development. The Miscellany Chapter has been split up into contributions to these three parts, and other chapters might find this type of split useful. It will be sent (in three parts) shortly. The overall plan is currently to merge the current large, old MPI-1.1 and new, small MPI-1.2 after we are finished to make a definitive 1.2 that replaces 1.1. Then the MPI-2 document will focus on the new "MPI-2" capabilities. We can use the JoD for useful ideas and discussions that don't get into either one. This structure is of course still open to discussion. New Intro: %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Mon Aug 12 17:14:00 1996 %%Pages: 5 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.08.12:1713 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 25 118 df<00000200000006000000060000000E0000001E0000001E0000 003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F0000020F0000 040F00000C0F0000080F0000100F0000100F0000200F80003FFF800040078000C0078000 80078001000780010007800200078002000780060007801E000F80FF807FF81D207E9F22 >65 D<0000FE0200078186001C004C0038003C0060003C00C0001C01C000180380001807 0000180F0000181E0000101E0000103C0000003C00000078000000780000007800000078 000000F0000000F0000000F0000000F0000000F000008070000080700000807000010038 00010038000200180004000C001800060020000381C00000FE00001F217A9F21>67 D<01FFF3FFE0001F003E00001E003C00001E003C00001E003C00003C007800003C007800 003C007800003C007800007800F000007800F000007800F000007800F00000F001E00000 FFFFE00000F001E00000F001E00001E003C00001E003C00001E003C00001E003C00003C0 07800003C007800003C007800003C007800007800F000007800F000007800F000007800F 00000F801F0000FFF1FFE000231F7D9E22>72 D<01FFF0001F00001E00001E00001E0000 3C00003C00003C00003C0000780000780000780000780000F00000F00000F00000F00001 E00001E00001E00001E00003C00003C00003C00003C0000780000780000780000780000F 8000FFF800141F7D9E12>I<001FFF0000F80000F00000F00000F00001E00001E00001E0 0001E00003C00003C00003C00003C0000780000780000780000780000F00000F00000F00 000F00001E00001E00301E00781E00F83C00F83C00F0780080700040E00021C0001F0000 18207D9E18>I<01FFFF80001E00E0001E0070001E0038001E003C003C003C003C003C00 3C003C003C003C0078007800780078007800F0007800E000F003C000F00F0000FFFC0000 F0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000007 8000000780000007800000078000000F800000FFF000001E1F7D9E1F>80 D<0007E040001C18C0003005800060038000C0038001C001800180010003800100038001 00038001000380000003C0000003C0000003F8000001FF800001FFE000007FF000001FF0 000001F80000007800000078000000380000003800200038002000380020003000600070 00600060006000E0007000C000E8038000C606000081F800001A217D9F1A>83 D<0FFFFFF01E0780E0180780201007802020078020200F0020600F0020400F0020400F00 20801E0040001E0000001E0000001E0000003C0000003C0000003C0000003C0000007800 0000780000007800000078000000F0000000F0000000F0000000F0000001E0000001E000 0001E0000001E0000003E00000FFFF00001C1F789E21>I<00F1800389C00707800E0380 1C03803C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40 703C40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C 07003C0200780000780000780000F00000F00000F00000F00000F0000070010070020030 040018380007C00011147C9315>99 D<0000780003F80000700000700000700000700000 E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C03803C03803807 00780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C40308C 800F070015207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000F0 00F0007000700170023804183807C010147C9315>I<00007800019C00033C00033C0007 18000700000700000E00000E00000E00000E00000E0001FFE0001C00001C00001C00001C 0000380000380000380000380000380000700000700000700000700000700000700000E0 0000E00000E00000E00000C00001C00001C0000180003180007B0000F300006600003C00 001629829F0E>I<003C6000E27001C1E00380E00700E00F00E00E01C01E01C01E01C01E 01C03C03803C03803C03803C03803C07003C07001C0F001C17000C2E0003CE00000E0000 0E00001C00001C00301C00783800F0700060E0003F8000141D7E9315>I<01E0000FE000 01C00001C00001C00001C000038000038000038000038000070000070000071E00076300 0E81800F01C00E01C00E01C01C03801C03801C03801C0380380700380700380700380E10 700E20700C20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C0000000 00000000000000000000000E003300230043804300470087000E000E000E001C001C001C 003840388030807080310033001C000B1F7C9E0E>I<03C01FC003800380038003800700 0700070007000E000E000E000E001C001C001C001C003800380038003800700070007000 7100E200E200E200E200640038000A207C9F0C>108 D<1C0F80F0002630C31800474064 0C004780680E004700700E004700700E008E00E01C000E00E01C000E00E01C000E00E01C 001C01C038001C01C038001C01C038001C01C0708038038071003803806100380380E100 38038062007007006600300300380021147C9325>I<1C0F802630C04740604780604700 704700708E00E00E00E00E00E00E00E01C01C01C01C01C01C01C03843803883803083807 083803107003303001C016147C931A>I<007C0001C3000301800E01C01E01C01C01E03C 01E07801E07801E07801E0F003C0F003C0F003C0F00780F00700700F00700E0030180018 700007C00013147C9317>I<01C1E002621804741C04781C04701E04701E08E01E00E01E 00E01E00E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0072380071E00 0700000700000E00000E00000E00000E00001C00001C0000FFC000171D809317>I<1C1E 002661004783804787804707804703008E00000E00000E00000E00001C00001C00001C00 001C000038000038000038000038000070000030000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E000E700EF00CF00CE00840 1020601F8010147D9313>I<018001C0038003800380038007000700FFF007000E000E00 0E000E001C001C001C001C003800380038003820704070407080708031001E000C1C7C9B 0F>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E03801C0700 1C07001C07001C07101C0E20180E20180E201C1E200C264007C38014147C9318>I E /Fb 1 64 df<07F8001FFE00381F80780F80FC0FC0FC0FC0FC0FC0780FC0301F80001F 00003E00007C0000700000E00000E00000C00000C00000C00000C00000C00000C0000000 0000000000000000000001C00003E00007F00007F00007F00003E00001C00012207D9F19 >63 D E /Fc 16 117 df<020408103020604040C0C0C0C0C0C0C0C04040602030100804 02071A7F920C>40 D<8040201018080C0404060606060606060604040C08181020408007 1A7E920C>I<1F00318060C04040C060C060C060C060C060C060C060C060404060C03180 1F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C000C00 0C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006000C 00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F0000800040 0060C060C060804060801F000B107F8F0F>I<0300030007000F000B0013003300230043 00C300FFE003000300030003001FE00B107F8F0F>I<20803F002C002000200020002F00 30802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C06000C0 00CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC08080 808001000200040004000C0008000800180018001800180018000B117E900F>I<1F0031 8060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F>I< 1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B107F8F 0F>I<1F2060E0006080208020800060003E001F8000C00060002080208020C040E0C09F 000B117E9011>83 D<1F00318060C0C0C0FFC0C000C000C000604030801F000A0B7F8A0E >101 D112 D<10103030FE3030303030323232321C070F7F8E0C>116 D E /Fd 38 122 df<0000F000F8F001F8F003F8F00780000700000F00000F00000F0000 0F00000F00000F00000F0000FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F0 0F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F0 142180A018>12 D45 DI<000100 030003000600060006000C000C000C00180018001800300030003000600060006000C000 C000C00180018001800300030003000600060006000C000C000C00180018001800300030 003000600060006000C000C000C000102D7DA117>I<00C001C00FC0FFC0FFC0F3C003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C003C003C0FFFEFFFEFFFE0F1F7C9E17>49 D<07F0000FFC001FFE00383F00700F0060 0780E00780E003C04003C04003C00003C00003C00003C0000780000780000F00000E0000 1C00003C0000780000E00001C0000380000700000E00001C0000380000700000FFFFC0FF FFC0FFFFC0121F7E9E17>I66 D<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C 0000780000780000780000F00000F00000F00000F00000F00000F00000F00000F0000078 00007800007800007C00003C00003E00001E00000F000207800E03E03E01FFFC00FFF000 1FC017227DA01D>IIII73 D76 DI<003F000000FFC000 03FFF00007E1F8000F807C001F003E001E001E003C000F003C000F007800078078000780 78000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F80007C07800078078000780780007803C000F003C000F001E001E001F003E00 0F807C0007E1F80003FFF00000FFC000003F00001A227DA021>79 DI82 D<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C00 003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E00001 E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019 >II<07E03FF87FFC70 1E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E 9316>97 D<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C 013E0F1FFF0FFE03F010147E9314>99 D<00078000078000078000078000078000078000 078000078000078000078000078000078007C7800FF7801FFF803E1F807C0780780780F8 0780F00780F00780F00780F00780F00780F00780F00780780780780F803E1F801FFF800F F78007C78011207E9F17>I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80 FFFF80FFFF80F00000F00000F000007000007800003C01003E07001FFF0007FE0001F800 11147F9314>I<007E01FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FF F00F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F 20809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F00 3C1E003E3E001FFC003FF80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0 F000F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00141E7F9317>I105 D108 DII<01F80007 FE001FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000F078 01E07801E03C03C03F0FC01FFF8007FE0001F80014147F9317>II114 D<07F01FFC3FFC780C7800 780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311> I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E001E 001E001E001E001E001E201FF00FF007C00C1A7F9910>III<7801E07C03C0 3E07801E0F000F0F00079E0003FC0003F80001F80000F00001F00001F80003FC00079E00 0F0F000E0F001E07803C03C07801E0F801F01414809315>120 DI E /Fe 14 86 df<3078F8787005057C840D>46 D<000C001C00FC0F380038003800380038003800700070007000700070007000E000E000 E000E000E000E001C001C001C001C001C001C0038003C0FFFE0F1E7C9D17>49 D<0000100000001800000038000000380000007800000078000000FC000001BC0000013C 0000033C0000023C0000063C0000043E0000081E0000081E0000101E0000101E0000201E 0000200F0000400F0000400F0000FFFF0000800F0001000F800100078002000780020007 8004000780040007800C0007C03E0007C0FF807FFC1E207E9F22>65 D<0001F808000E061800380138007000F801E0007803C0007007800030078000300F0000 301F0000301E0000303E0000203C0000007C0000007C0000007C0000007C000000F80000 00F8000000F8000000F8000000F80000007800004078000080780000803C0000803C0001 001C0002000E00020006000C000300100001C0E000003F00001D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E007800 1E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E01E000 3E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C003 C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038 003C001800780018007800080078000800780008007800080078080800F0100000F01000 00F0100000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001E00008 01E0001003C0001003C0001003C0002003C0002003C0006003C000C0078001C0078007C0 FFFFFF801D1F7E9E1F>I<07FFC7FFC0007C00F800003C007800003C007800007800F000 007800F000007800F000007800F000007800F000007800F00000F001E00000F001E00000 F001E00000F001E00000FFFFE00000F001E00001E003C00001E003C00001E003C00001E0 03C00001E003C00001E003C00003C007800003C007800003C007800003C007800003C007 800003C007800007800F000007C00F8000FFF8FFF800221F7E9E22>72 D<07FFE0007C00003C00003C0000780000780000780000780000780000780000F00000F0 0000F00000F00000F00000F00001E00001E00001E00001E00001E00001E00003C00003C0 0003C00003C00003C00003C00007800007C000FFFC00131F7F9E10>I<07FC01FFC0003E 003E00003E001800003E001800004F001000004F001000004780100000478010000043C0 10000043C010000083C020000081E020000081E020000080F020000080F0200000807820 00010078400001007C400001003C400001003C400001001E400001001E400002000F8000 02000F800002000F800002000780000200078000060003800006000300000F00010000FF E0010000221F7E9E22>78 D<0003F800001E0E000038070000E0038001C001C003C001E0 078000E00F0000F00F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F8 7C0000F87C0000F8F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0 780007C07C0007803C000F003C001E001E001C000E0038000700F00003C3C00000FE0000 1D217B9F23>I<07FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F800 7800F8007800F8007800F000F001F000F001E000F003C000F0078000F00F0000FFF80001 E0000001E0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003 C0000003C0000003C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C 0700003C03C0003C01E0007801E0007801F0007801F0007801F0007801F0007801E000F0 03E000F003C000F0078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E0 1C0001E01C0001E01E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E020780 3E0407C01F04FFFC0F18000003E01F207E9E21>82 D<3FFFFFF03C0780F0300780306007 8030400F0010400F0010C00F0010800F0010800F0010800F0010001E0000001E0000001E 0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003C 000000780000007800000078000000780000007800000078000000F0000001F800007FFF E0001C1F7A9E21>84 D I E /Ff 3 64 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE 3FFC1FF80FF003C010127D9317>15 D62 D<00040000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000FFFFFF E0FFFFFFE01B1C7C9B23>I E /Fg 23 123 df46 D<00180000380000F80007F800FFF800FFF800F8F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F8007FFFF07FFFF07FFFF014287BA71E>49 D<00FE0003FFC007FFE00FFFF01F03F8 3C00FC38007E78003E70003EF0001FF0001F60001F20001F00001F00001F00001F00003E 00003E00007C00007C0000F80001F00001E00003C0000780000F00001E00003C00007800 00F00001E00003C0000780000F00001E00003C00007FFFFF7FFFFF7FFFFF7FFFFF18287E A71E>I66 D68 D<0001FC0000000FFF8000003FFFE000007FFFF00001FE03FC0003F800FE0007E000 3F0007C0001F000F80000F801F000007C01F000007C03E000003E03E000003E07C000001 F07C000001F07C000001F078000000F0F8000000F8F8000000F8F8000000F8F8000000F8 F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F87C000001F07C 000001F07C000001F07E000003F03E000003E03F000007E01F000007C01F80000FC00FC0 001F8007E0003F0007F0007F0003F800FE0001FE03FC0000FFFFF800003FFFE000000FFF 80000001FC0000252C7DAA2C>79 D<01FE000FFF803FFFC03FFFE03C03F03001F00001F8 0000F80000F80000F80000F80000F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8 F800F8F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1D>97 D<007FC001FFF007FFFC0FFFFC1FC07C1F00083E00007C00007C00007C0000F80000F800 00F80000F80000F80000F80000F800007C00007C00007E00003E00001F000C1FC07C0FFF FC07FFFC01FFF0007F80161B7E9A1B>99 D<00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E00003E00003E00003E00003E00FC3E03FF3E07FFFE 0FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003E F8003EF8003EFC003E7C003E7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03FF3E00FC3E 172A7EA91F>I<007E0003FF8007FFC00FFFE01F83F03F00F03E00787C00787C00387800 3CFFFFFCFFFFFCFFFFFCFFFFFCF80000F80000F800007800007C00007C00003E00003F00 0C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<001FC0007FC000FFC001FFC003 F00003E00007C00007C00007C00007C00007C00007C00007C00007C00007C000FFFE00FF FE00FFFE0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C000122A7FA912>I<00F8078003FE7FC00FFFFFC01FFFFFC01F07C0003E03E000 3E03E0007C01F0007C01F0007C01F0007C01F0007C01F0007C01F0003E03E0003E03E000 1F07C0001FFFC0003FFF80003BFE000038F8000078000000780000003C0000003FFFC000 3FFFF8001FFFFC001FFFFE003FFFFF007C007F00F8001F80F8000F80F8000F80F8000F80 FC001F807E003F003F80FE003FFFFE000FFFF80007FFF00000FF80001A287E9A1E>II I107 D109 DI<007F000001FFC00007FFF0000FFFF8001FC1FC 003F007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F 80F8000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001FC1FC 000FFFF80007FFF00001FFC000007F0000191B7E9A1E>I114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000 F80000F80000FC00007F80007FF8003FFE001FFF0007FF8000FFC0000FE00007E00003E0 0003E04003E0E007E0FC0FC0FFFFC07FFF801FFE0003F800131B7E9A17>I<07C00007C0 0007C00007C00007C00007C00007C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C04007E1C003FFE003FFE001FF8000FC0013227FA116>II122 D E /Fh 5 81 df45 D<0001FF000000000FFFF00000003FFFFC000000FC03FF000001E000FFC00003C0 003FE0000700001FF0000E00000FF8001C000007FC001C000007FC0038000003FE007000 0003FE0070000003FF007C000001FF00FF000001FF00FF800001FF80FFC00001FF80FFC0 0001FF80FFC00001FF80FFC00001FF807F800001FF807F800001FF803F000001FF800000 0001FF0000000001FF0000000003FF0000000003FE0000000003FE0000000007FC000000 0007FC000000000FF8000000000FF0000000001FF0000000001FE0000000003FC0000000 003F80000000007F0000000000FE0000000001FC0000000001F80000000003F000000000 07E0000000000FC0000000000F00000000001E00000000003C0000000000780000000000 F00000000001E00000000003C0000000000780000380000700000380000E00000380001C 0000070000380000070000700000070000E00000070001C000000F00018000000F000300 00001E0007FFFFFFFE000FFFFFFFFE001FFFFFFFFE003FFFFFFFFE007FFFFFFFFE00FFFF FFFFFC00FFFFFFFFFC00FFFFFFFFFC0029447BC334>50 D73 D77 D80 D E /Fi 17 122 df66 D73 D<0007FFFC000000007FFFFFC0000001FFFFFFF8000003FFFFFF FE000007FE001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000FFF0000FFF80007F F0000FFF80007FF8000FFF80007FF80007FF00003FFC0007FF00003FFC0003FE00003FFC 0000F800003FFC00000000003FFC00000000003FFC00000000003FFC00000000003FFC00 000007FFFFFC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC0003 FFF0003FFC0007FFC0003FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC007FF8 00003FFC007FF800003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000 003FFC00FFF000003FFC00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC0003 EFFC003FFF0007CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFFF001 FFFC0003FF80007FF8362E7DAD3A>97 D<007FC00000000000FFFFC00000000000FFFFC0 0000000000FFFFC00000000000FFFFC00000000000FFFFC0000000000003FFC000000000 0001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0 000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC000000000 0001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0 000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC000000000 0001FFC0000000000001FFC00FFC00000001FFC07FFFC0000001FFC3FFFFF0000001FFCF FFFFFC000001FFDFF00FFF000001FFFF8003FF800001FFFE0001FFC00001FFF800007FE0 0001FFF000007FF00001FFE000003FF80001FFE000001FFC0001FFE000001FFC0001FFE0 00001FFE0001FFE000001FFE0001FFE000000FFF0001FFE000000FFF0001FFE000000FFF 0001FFE000000FFF0001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE0 00000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF 8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF0001FFE000000FFF0001FFE0 00000FFF0001FFE000001FFE0001FFE000001FFE0001FFE000001FFC0001FFE000001FFC 0001FFE000003FF80001FFF000003FF80001FFF800007FF00001FFFC0000FFE00001FFFE 0001FFC00001FFBF0007FF800001FF1FE01FFE000001FE0FFFFFFC000001FC03FFFFF000 0001F800FFFF80000001F0001FF800000039487CC742>I<00001FFFC0000000FFFFF800 0007FFFFFE00001FFFFFFF80007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FFE0 07FF0003FFE00FFE0003FFE00FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF80 3FFC00003E007FF8000000007FF8000000007FF800000000FFF800000000FFF800000000 FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000 FFF800000000FFF8000000007FF8000000007FF8000000007FFC000000003FFC00000000 3FFC000000001FFC000000F81FFE000000F80FFE000000F80FFF000001F007FF800003F0 03FFC00007E001FFE0000FC000FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF800 0000FFFFE00000001FFE00002D2E7CAD35>I<00000000007FC00000000000FFFFC00000 000000FFFFC00000000000FFFFC00000000000FFFFC00000000000FFFFC0000000000003 FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC00000 00000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001 FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC00000 00000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001 FFC0000000000001FFC00000000FFC01FFC0000000FFFF81FFC0000007FFFFE1FFC00000 1FFFFFF9FFC000007FFC03FFFFC00000FFF0007FFFC00001FFC0001FFFC00003FF80000F FFC00007FF000007FFC0000FFE000003FFC0000FFE000003FFC0001FFC000003FFC0001F FC000003FFC0003FFC000003FFC0003FFC000003FFC0007FF8000003FFC0007FF8000003 FFC0007FF8000003FFC000FFF8000003FFC000FFF8000003FFC000FFF8000003FFC000FF F8000003FFC000FFF8000003FFC000FFF8000003FFC000FFF8000003FFC000FFF8000003 FFC000FFF8000003FFC000FFF8000003FFC0007FF8000003FFC0007FF8000003FFC0007F F8000003FFC0003FF8000003FFC0003FFC000003FFC0003FFC000003FFC0001FFC000003 FFC0001FFC000003FFC0000FFE000007FFC00007FF00000FFFC00003FF00001FFFC00001 FFC0003FFFC00000FFE000FFFFE000007FF807FBFFFF80001FFFFFF3FFFF800007FFFFC3 FFFF800001FFFF03FFFF8000001FF803FFFF8039487CC742>I<00000000001F8000007F F000FFE00007FFFF03FFF0001FFFFFC7FFF0007FFFFFFFC7F800FFE03FFE0FF801FF800F FC0FF803FF0007FE0FF807FE0003FF07F007FE0003FF07F00FFC0001FF81C00FFC0001FF 80000FFC0001FF80001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0 001FFC0001FFC0001FFC0001FFC0000FFC0001FF80000FFC0001FF80000FFC0001FF8000 07FE0003FF000007FE0003FF000003FF0007FE000001FF800FFC000000FFE03FF8000001 FFFFFFF0000001DFFFFFC0000003C7FFFF00000003C07FF000000007C0000000000007C0 000000000007C0000000000007C0000000000007E0000000000007F0000000000007F800 0000000007FFFFFFF0000007FFFFFFFF000003FFFFFFFFE00003FFFFFFFFF80001FFFFFF FFFE0001FFFFFFFFFF0000FFFFFFFFFF80007FFFFFFFFF8003FFFFFFFFFFC00FFFFFFFFF FFC01FF800001FFFE03FE0000001FFE07FC00000007FF07FC00000003FF0FF800000001F F0FF800000001FF0FF800000001FF0FF800000001FF0FF800000001FF07FC00000003FE0 7FC00000003FE03FE00000007FC03FF0000000FFC01FFC000003FF800FFF00000FFF0003 FFF000FFFC0000FFFFFFFFF000003FFFFFFFC0000007FFFFFE000000003FFFC000003544 7DAE3B>103 D<007FC00000000000FFFFC00000000000FFFFC00000000000FFFFC00000 000000FFFFC00000000000FFFFC0000000000003FFC0000000000001FFC0000000000001 FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC00000 00000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001 FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001FFC00000 00000001FFC0000000000001FFC0000000000001FFC0000000000001FFC0000000000001 FFC001FFC0000001FFC00FFFF8000001FFC03FFFFE000001FFC0FFFFFF000001FFC1FC07 FF800001FFC3E003FFC00001FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE00001 FFDC0000FFE00001FFFC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF00000 FFF00001FFF00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001 FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000 FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001 FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000 FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001 FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000 FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FF FFFFC07FFFFFE03B487CC742>I<00FC0001FE0003FF0007FF800FFFC01FFFE01FFFE01F FFE01FFFE01FFFE01FFFE00FFFC007FF8003FF0001FE0000FC0000000000000000000000 0000000000000000000000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FF FFC0FFFFC003FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC0FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF18497CC820>I<007FC000FFFFC000FFFFC000 FFFFC000FFFFC000FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<007FC001FFC00000FFFFC00FFFF80000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1 FC07FF8000FFFFC3E003FFC00003FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE0 0001FFDC0000FFE00001FFFC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF0 0000FFF00001FFF00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFF E0FFFFFFC07FFFFFE03B2E7CAD42>110 D<00000FFF0000000000FFFFF000000007FFFF FE0000001FFFFFFF8000003FFC03FFC00000FFE0007FF00001FF80001FF80003FF00000F FC0007FE000007FE000FFE000007FF000FFC000003FF001FFC000003FF803FFC000003FF C03FF8000001FFC03FF8000001FFC07FF8000001FFE07FF8000001FFE07FF8000001FFE0 FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FF F8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF07FF8 000001FFE07FF8000001FFE07FF8000001FFE07FF8000001FFE03FFC000003FFC03FFC00 0003FFC01FFC000003FF801FFE000007FF800FFE000007FF0007FF00000FFE0003FF8000 1FFC0001FFC0003FF80000FFE0007FF000007FFC03FFE000001FFFFFFF80000007FFFFFE 00000000FFFFF0000000000FFF000000342E7DAD3B>I<007FC00FFC000000FFFFC07FFF C00000FFFFC3FFFFF00000FFFFCFFFFFFC0000FFFFDFF01FFF0000FFFFFF8007FF800003 FFFE0001FFC00001FFF80000FFE00001FFF00000FFF00001FFE000007FF80001FFE00000 3FFC0001FFE000003FFC0001FFE000003FFE0001FFE000001FFE0001FFE000001FFF0001 FFE000001FFF0001FFE000001FFF0001FFE000000FFF0001FFE000000FFF8001FFE00000 0FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001 FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE000000FFF8001FFE00000 0FFF0001FFE000001FFF0001FFE000001FFF0001FFE000001FFE0001FFE000001FFE0001 FFE000003FFC0001FFE000003FFC0001FFE000007FF80001FFF000007FF80001FFF80000 FFF00001FFFC0001FFE00001FFFE0003FFC00001FFFF0007FF800001FFFFE03FFE000001 FFEFFFFFFC000001FFE3FFFFF0000001FFE0FFFF80000001FFE01FF800000001FFE00000 00000001FFE0000000000001FFE0000000000001FFE0000000000001FFE0000000000001 FFE0000000000001FFE0000000000001FFE0000000000001FFE0000000000001FFE00000 00000001FFE0000000000001FFE0000000000001FFE0000000000001FFE0000000000001 FFE00000000000FFFFFFC000000000FFFFFFC000000000FFFFFFC000000000FFFFFFC000 000000FFFFFFC00000000039427CAD42>I<00FF803F8000FFFF80FFF000FFFF83FFFC00 FFFF87FFFE00FFFF8FC3FF00FFFF8F07FF0003FF9E0FFF8001FFBC0FFF8001FFB80FFF80 01FFF80FFF8001FFF00FFF8001FFF007FF0001FFF007FF0001FFE003FE0001FFE000F800 01FFE000000001FFE000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC0000000 01FFC000000001FFC0000000FFFFFFE00000FFFFFFE00000FFFFFFE00000FFFFFFE00000 FFFFFFE00000292E7CAD31>114 D<0001F000000001F000000001F000000001F0000000 01F000000001F000000003F000000003F000000003F000000007F000000007F000000007 F00000000FF00000000FF00000001FF00000003FF00000003FF00000007FF0000001FFF0 000003FFF000000FFFFFFFC0FFFFFFFFC0FFFFFFFFC0FFFFFFFFC0FFFFFFFFC000FFF000 0000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF00000 00FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000 FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FFF0000000FF F0000000FFF001F000FFF001F000FFF001F000FFF001F000FFF001F000FFF001F000FFF0 01F000FFF001F000FFF001F0007FF001E0007FF803E0003FF803E0003FFC07C0001FFE0F 80000FFFFF800007FFFE000001FFFC0000001FF00024427EC12E>116 D<007FE000003FF000FFFFE0007FFFF000FFFFE0007FFFF000FFFFE0007FFFF000FFFFE0 007FFFF000FFFFE0007FFFF00003FFE00001FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00001FFF00001FFE00001FFF00001FFE00001FFF0 0001FFE00003FFF00000FFE00007FFF00000FFE0000F7FF000007FE0001F7FF000007FF0 003E7FF800003FFC00FC7FFFE0001FFFFFF87FFFE00007FFFFE07FFFE00001FFFF807FFF E000003FFE007FFFE03B2E7CAD42>I<7FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC0 00FFFF807FFFFFC000FFFF807FFFFFC000FFFF8000FFF000000FE00000FFF800000FC000 00FFF800000FC000007FFC00000F8000007FFC00001F8000003FFC00001F0000003FFE00 003F0000001FFE00003E0000001FFF00007E0000000FFF00007C0000000FFF8000FC0000 0007FF8000F800000007FFC001F800000003FFC001F000000003FFE003F000000003FFE0 03F000000001FFF003E000000001FFF007E000000000FFF007C000000000FFF80FC00000 00007FF80F80000000007FFC1F80000000003FFC1F00000000003FFE3F00000000001FFE 3E00000000001FFF7E00000000000FFF7C00000000000FFFFC00000000000FFFFC000000 000007FFF8000000000007FFF8000000000003FFF0000000000003FFF0000000000001FF E0000000000001FFE0000000000000FFC0000000000000FFC00000000000007F80000000 0000007F800000000000003F000000000000003F000000000000003F000000000000003E 000000000000007E000000000000007C00000000000000FC000000001F8000F800000000 3FC001F8000000007FE001F000000000FFF003F000000000FFF003E000000000FFF007E0 00000000FFF00FC000000000FFF01F8000000000FFF03F80000000007FE07F0000000000 7F43FE00000000003FFFF800000000001FFFF0000000000007FFC0000000000001FE0000 0000000039427EAD3F>121 D E /Fj 8 117 df<00001E000000003E00000000FE000000 03FE0000003FFE0000FFFFFE0000FFFFFE0000FFFFFE0000FFCFFE0000000FFE0000000F FE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE00 00000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000 000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE000000 0FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000F FE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE00007FFFFFFFC07FFFFF FFC07FFFFFFFC07FFFFFFFC0223879B731>49 D<0000001FFF000030000001FFFFE000F0 00000FFFFFFC01F000007FFFFFFE03F00001FFFE007F87F00003FFE0000FCFF0000FFF00 0003FFF0001FFC000001FFF0003FF80000007FF0007FF00000003FF000FFC00000003FF0 01FFC00000001FF003FF800000000FF007FF000000000FF00FFF0000000007F00FFE0000 000007F01FFE0000000003F01FFE0000000003F03FFC0000000003F03FFC0000000001F0 3FFC0000000001F07FFC0000000001F07FF80000000001F07FF80000000000007FF80000 00000000FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000 FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF80000 00000000FFF8000000000000FFF80000000000007FF80000000000007FF8000000000000 7FF80000000000007FFC0000000000F03FFC0000000000F03FFC0000000000F03FFC0000 000000F01FFE0000000000F01FFE0000000001E00FFE0000000001E00FFF0000000001E0 07FF0000000003C003FF8000000003C001FFC0000000078000FFE00000000F00007FF000 00001F00003FF80000003E00001FFC0000007C00000FFF000001F8000003FFE00007F000 0001FFFE003FC00000007FFFFFFF000000000FFFFFFC0000000001FFFFF000000000001F FF0000003C3D7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE0000 0FF801FF00001FFC00FF80001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF000 0FF8003FF00007F0003FF00001C0003FF0000000003FF0000000003FF0000000003FF000 0000FFFFF000000FFFFFF000007FF83FF00001FF803FF00007FE003FF0000FF8003FF000 1FF0003FF0003FE0003FF0007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000 FFC0003FF000FFC0003FF000FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF800 1FFC0F0FFFF007FFFE0FFFF001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFFFE0000FF80FF0001FE003F8007FC001FC00FF8000FE0 0FF8000FF01FF00007F03FF00007F83FF00007F87FE00007F87FE00003FC7FE00003FC7F E00003FCFFE00003FCFFFFFFFFFCFFFFFFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0 000000FFE00000007FE00000007FE00000007FE00000003FE00000003FF000003C1FF000 003C1FF000003C0FF800007807FC0000F803FE0001F001FF0007E000FFC03FC0003FFFFF 000007FFFC000000FFE00026267DA52D>101 D<00FF00000000FFFF00000000FFFF0000 0000FFFF00000000FFFF0000000007FF0000000003FF0000000003FF0000000003FF0000 000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000 000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000 000003FF0000000003FF007FC00003FF01FFF80003FF07FFFC0003FF0F03FE0003FF1C01 FF0003FF3001FF8003FF6000FF8003FFE000FFC003FFC000FFC003FF8000FFC003FF8000 FFC003FF8000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000 FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000 FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000 FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC0FFFFFC3F FFFFFFFFFC3FFFFFFFFFFC3FFFFFFFFFFC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00FFFFFE03FF00FFFFF000FF8003FFC0007F C003FF80003FE003FF00003FF003FF00001FF803FF00001FFC03FF00000FFC03FF00000F FE03FF00000FFE03FF000007FE03FF000007FF03FF000007FF03FF000007FF03FF000007 FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007 FE03FF000007FE03FF00000FFE03FF00000FFC03FF00000FFC03FF00001FF803FF00001F F803FF00003FF003FF80003FE003FFC0007FC003FFF001FF8003FFFC07FF0003FF3FFFFC 0003FF0FFFF00003FF01FF000003FF0000000003FF0000000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF000000 0003FF0000000003FF0000000003FF00000000FFFFFC000000FFFFFC000000FFFFFC0000 00FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80 FFFE707FC007FE60FFE003FEE0FFE003FEC0FFE003FFC0FFE003FF807FC003FF807FC003 FF803F8003FF800E0003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF0000 0003FF000000FFFFFE0000FFFFFE0000FFFFFE0000FFFFFE000023267DA529>114 D<0007800000078000000780000007800000078000000F8000000F8000000F8000000F80 00001F8000001F8000003F8000003F8000007F800000FF800001FF800007FF80001FFFFF F0FFFFFFF0FFFFFFF0FFFFFFF001FF800001FF800001FF800001FF800001FF800001FF80 0001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF80 0001FF800001FF800001FF800001FF800001FF803C01FF803C01FF803C01FF803C01FF80 3C01FF803C01FF803C01FF803C00FF807800FFC078007FC070003FE0E0001FFFC00007FF 800001FF001E377EB626>116 D E /Fk 70 123 df<001F83E000F06E3001C078780380 F8780300F03007007000070070000700700007007000070070000700700007007000FFFF FF8007007000070070000700700007007000070070000700700007007000070070000700 700007007000070070000700700007007000070070000700700007007000070070000700 70007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007000007 0000070000070000070000070000FFFFE00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07F C3FE1720809F19>I<001F81F80000F04F040001C07C06000380F80F000300F00F000700 F00F00070070000007007000000700700000070070000007007000000700700000FFFFFF FF0007007007000700700700070070070007007007000700700700070070070007007007 000700700700070070070007007007000700700700070070070007007007000700700700 07007007000700700700070070070007007007007FE3FE3FF02420809F26>14 D<7038F87CFC7EFC7E743A0402040204020804080410081008201040200F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<0020004000800100020006000C 000C00180018003000300030007000600060006000E000E000E000E000E000E000E000E0 00E000E000E000E0006000600060007000300030003000180018000C000C000600020001 000080004000200B2E7DA112>I<800040002000100008000C0006000600030003000180 0180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E0 00C000C000C001C001800180018003000300060006000C00080010002000400080000B2E 7DA112>I<00060000000600000006000000060000000600000006000000060000000600 0000060000000600000006000000060000000600000006000000060000FFFFFFF0FFFFFF F00006000000060000000600000006000000060000000600000006000000060000000600 000006000000060000000600000006000000060000000600001C207D9A23>43 D<70F8FCFC74040404080810102040060E7C840D>II<70F8F8F8 7005057C840D>I<000100030003000600060006000C000C000C00180018001800300030 003000600060006000C000C000C00180018001800300030003000600060006000C000C00 0C00180018001800300030003000600060006000C000C000C000102D7DA117>I<03F000 0E1C001C0E00180600380700700380700380700380700380F003C0F003C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0700380700380700380 7807803807001806001C0E000E1C0003F000121F7E9D17>I<018003800F80F380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038003800380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E0020070040078080 0780F007C0F803C0F803C0F803C02007C00007C0000780000780000F00000E00001C0000 380000700000600000C0000180000300000600400C00401800401000803FFF807FFF80FF FF80121E7E9D17>I<03F0000C1C00100E00200F00780F80780780780780380F80000F80 000F00000F00000E00001C0000380003F000003C00000E00000F000007800007800007C0 2007C0F807C0F807C0F807C0F00780400780400F00200E001C3C0003F000121F7E9D17> I<000600000600000E00000E00001E00002E00002E00004E00008E00008E00010E00020E 00020E00040E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E00000E 00000E00000E00000E00000E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001F F8001FE00010000010000010000010000010000010000011F000161C00180E0010070010 07800003800003800003C00003C00003C07003C0F003C0F003C0E0038040038040070020 0600100E000C380003E000121F7E9D17>I<007C000182000701000E03800C07801C0780 380300380000780000700000700000F1F000F21C00F40600F80700F80380F80380F003C0 F003C0F003C0F003C0F003C07003C07003C07003803803803807001807000C0E00061C00 01F000121F7E9D17>I<4000007FFFC07FFF807FFF804001008002008002008004000008 0000080000100000200000200000400000400000C00000C00001C0000180000380000380 00038000038000078000078000078000078000078000078000078000030000121F7D9D17 >I<03F0000C0C001006003003002001806001806001806001807001807803003E03003F 06001FC8000FF00003F80007FC000C7E00103F00300F806003804001C0C001C0C000C0C0 00C0C000C0C000806001802001001002000C0C0003F000121F7E9D17>I<03F0000E1800 1C0C00380600380700700700700380F00380F00380F003C0F003C0F003C0F003C0F003C0 7007C07007C03807C0180BC00E13C003E3C0000380000380000380000700300700780600 780E00700C002018001070000FC000121F7E9D17>I<70F8F8F870000000000000000000 0070F8F8F87005147C930D>I<000100000003800000038000000380000007C0000007C0 000007C0000009E0000009E0000009E0000010F0000010F0000010F00000207800002078 000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F 0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F 9F22>65 DI<000FC040007030C0 01C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C000040 78000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000780000007C0000407C0000403C0000401C0000401E0000800E00008007000100 0380020001C0040000703800000FC0001A217D9F21>II70 D<000FE0200078186000E004E0038002E0070001E00F0000E01E0000601E0000603C0000 603C0000207C00002078000020F8000000F8000000F8000000F8000000F8000000F80000 00F8000000F8007FFCF80003E0780001E07C0001E03C0001E03C0001E01E0001E01E0001 E00F0001E0070001E0038002E000E0046000781820000FE0001E217D9F24>I73 D<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00203C 00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>I76 DI<001F800000F0F00001C0380007801E000F000F000E0007001E0007803C 0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F80001F0F80001F0F8 0001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C0003E03C0003C03C 0003C01E0007800E0007000F000F0007801E0001C0380000F0F000001F80001C217D9F23 >79 DI82 D<07E0800C1980100780300380600180600180 E00180E00080E00080E00080F00000F000007800007F00003FF0001FFC000FFE0003FF00 001F800007800003C00003C00001C08001C08001C08001C08001C0C00180C00380E00300 F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060400F0020400F0020 C00F0030800F0010800F0010800F0010800F0010000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000001F800007FFFE001C1F7E9E 21>I86 DI<7FF83FF80FE00FC007C0070003C00200 01E0040001F00C0000F0080000781000007C1000003C2000003E4000001E4000000F8000 000F8000000780000003C0000007E0000005E0000009F0000018F8000010780000207C00 00603C0000401E0000801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE 1F1F7F9E22>I91 D<080410082010201040204020 804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E001E0E00 380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>I E /Fl 46 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000071C000C060018030038 03803803807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C038 03803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038007C0FFFE0F217CA018>I<03F0000C1C0010070020 07804003C04003C08003E0F003E0F801E0F801E0F801E02003E00003E00003C00003C000 0780000700000E00001C0000180000300000600000C00001800001000002002004002008 00201800603000403FFFC07FFFC0FFFFC013217EA018>I<03F8000C1E00100700200780 4007C07807C07803C07807C03807C0000780000780000700000F00000E0000380003F000 001C00000F000007800007800003C00003C00003E02003E07003E0F803E0F803E0F003C0 4003C0400780200780100F000C1C0003F00013227EA018>I<000200000600000E00000E 00001E00001E00002E00004E00004E00008E00008E00010E00020E00020E00040E00040E 00080E00100E00100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E 00000E00000E00000E00001F0001FFF015217FA018>I<1000801E07001FFF001FFE001F F80013E00010000010000010000010000010000010000010F800130E0014070018038010 03800001C00001C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C040 03C04003802007001006000C1C0003F00013227EA018>I<007E0001C1000300800601C0 0E03C01C03C0180180380000380000780000700000700000F0F800F30C00F40600F40300 F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C0 3801C01803801C03000C0600070C0001F00013227EA018>I<01F800060E000803001001 802001802000C06000C06000C06000C07000C07801803E01003F02001FC4000FF80003F8 0003FC00067F00083F80100F803007C06001C06000E0C000E0C00060C00060C00060C000 606000406000C03000801803000E0E0003F00013227EA018>56 D<01F000060C000C0600 180700380380700380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07003E03803E01805E00C05E00619E003E1E00001C00001C00001C0000380000380 300300780700780600700C002018001030000FC00013227EA018>I<0001800000018000 000180000003C0000003C0000003C0000005E0000005E000000DF0000008F0000008F000 0010F800001078000010780000203C0000203C0000203C0000401E0000401E0000401E00 00800F0000800F0000FFFF000100078001000780030007C0020003C0020003C0040003E0 040001E0040001E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E0000701E0000701C0000 303C0000303C0000307C0000107800001078000010F8000000F8000000F8000000F80000 00F8000000F8000000F8000000F800000078000000780000107C0000103C0000103C0000 101C0000201E0000200E000040070000400380008001C0010000E0020000381C000007E0 001C247DA223>67 D69 DI73 D77 DI80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060 E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC0 0007E00001E00000F00000F0000070800070800070800070800070C00060C00060E000C0 F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840 07800840078008C007800C80078004800780048007800480078004000780000007800000 078000000780000007800000078000000780000007800000078000000780000007800000 078000000780000007800000078000000780000007800000078000000780000007800000 078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700 00070000070000FF0007C7001E07003C0700780700700700F00708F00708F00708F00F08 7817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E 00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E 00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300 780000700000F00000F00000F00000F00000F00000F00000F00000700000780040380040 1C00800C010007060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01 E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800 E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C038 01C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F0000070000078002038 00201C00400E008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007 00070007000700070007000700FFF8070007000700070007000700070007000700070007 00070007000700070007000700070007807FF8102380A20F>I<00007001F198071E180E 0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010 00001000001800001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0 00186000306000303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C0 0E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E00 1C00000000000000000000000000000000000E00FE001E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003 E003E001C00000000000000000000000000000000001E00FE001E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000 E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E0180 0E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C0 0E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00FE60E183 801E807201C00F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E0 0E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60 C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C 01801800C03800E0700070700070F00078F00078F00078F00078F00078F00078F0007870 00707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E8060 0F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C 0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E0000 0E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E 000E000E000E000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>III121 D E /Fm 20 118 df45 D68 D73 D77 D 80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E070 0000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF0000 0FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880 00007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0F0 0000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fn 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 908 a Fn(D)26 b(R)g(A)f(F)h(T)225 999 y Fm(Do)r(cumen)n(t) 20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1194 y Fl(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)802 1320 y(August)i(12,)f(1996)77 1378 y(This)h(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1436 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1494 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1552 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 45 a Fk(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fj(Chapter)34 b(1)75 564 y Fi(In)m(tro)s(duction)41 b(to)g Fh(MPI-2)75 805 y Fg(1.1)59 b(Background)75 906 y Fk(Beginning)19 b(Marc)o(h)d(1995,)g(the)h(MPI)f(F)l(orum)h(b)q(egan) g(meeting)g(to)g(consider)g(corrections)g(and)h(exten-)75 963 y(sions)f(to)f(the)g(original)i(MPI)e(Standard)g(do)q(cumen)o(t)h ([1].)23 b(The)16 b(\014rst)g(pro)q(duct)h(of)f(these)h(delib)q (erations)75 1019 y(w)o(as)f(V)l(ersion)i(1.1)e(of)g(the)h(MPI)g(sp)q (eci\014cation,)i(released)f(in)g(Ma)o(y)e(of)g(1995.)24 b(Since)19 b(that)d(time,)h(e\013ort)75 1076 y(has)e(b)q(een)h(fo)q (cused)g(in)g(four)f(t)o(yp)q(es)g(of)g(areas.)131 1169 y(1.)22 b(F)l(urther)15 b(corrections)g(and)g(clari\014cations)i(for)d (the)i(MPI-1.1)e(do)q(cumen)o(t.)131 1263 y(2.)22 b(Mo)q(di\014cations) 13 b(to)e(MPI-1)h(that)f(do)h(not)g(signi\014can)o(tly)i(c)o(hange)e (its)g(t)o(yp)q(es)g(of)g(functionalit)o(y)h(\(new)189 1320 y(datat)o(yp)q(es,)h(v)o(ersion)h(iden)o(ti\014cation,)i(etc.\).) 131 1413 y(3.)22 b(Completely)14 b(new)g(t)o(yp)q(es)g(of)g (functionalit)o(y)h(\(dynamic)f(pro)q(cesses,)g(one-sided)i(comm)o (unication,)189 1470 y(etc.\))j(that)c(are)g(what)f(ev)o(ery)o(one)h (thinks)h(of)f(as)g(\\MPI-2)f(functionalit)o(y".)131 1564 y(4.)22 b(Discussions)13 b(of)g(areas)f(in)h(whic)o(h)h(the)f(MPI) f(pro)q(cess)h(and)g(framew)o(ork)e(seem)i(lik)o(ely)i(to)d(b)q(e)h (useful,)189 1620 y(but)21 b(where)h(more)e(discussion)j(and)f(exp)q (erience)h(are)e(needed)i(b)q(efore)e(standardization)h(\(e.g.)189 1677 y(Real-time)16 b(extensions)g(and)g(p)q(erhaps)f(I/O.)75 1770 y(Opinions)i(will)g(v)m(ary)e(ab)q(out)g(what)g(category)f(v)m (arious)i(sections)f(and)h(subsections)g(b)q(elong)g(to.)166 1827 y(It)h(is)h(prop)q(osed)g(that)e(as)h(so)q(on)h(as)e(p)q(ossible)k (after)c(Sup)q(ercomputing'96,)i(three)g(do)q(cumen)o(ts)f(b)q(e)75 1883 y(pro)q(duced:)143 1977 y Ff(\017)23 b Fk(MPI-1.2,)14 b(con)o(taining)i(items)f(of)g(category)f(\(1\))g(and)i(p)q(erhaps)g (some)e(of)h(\(2\))f(ab)q(o)o(v)o(e.)143 2071 y Ff(\017)23 b Fk(MPI-2.0,)12 b(con)o(taining)i(fully-v)o(oted-on)g(\(reading)f(+)g (t)o(w)o(o)f(o\016cial)h(v)o(otes\))f(items)h(in)h(category)e(\(3\))189 2127 y(ab)q(o)o(v)o(e.)143 2221 y Ff(\017)23 b Fk(MPI)10 b(Journal)h(of)f(Dev)o(elopmen)o(t,)h(con)o(taining)g(items)g(of)f (category)f(\(3\))g(that)h(are)g(still)i(in)f(progress,)189 2278 y(together)j(with)i(items)f(of)g(category)f(\(4\).)166 2372 y(This)i(will)i(mak)o(e)e(it)g(easier)g(for)g(v)o(endors)f(and)h (users)g(to)g(ha)o(v)o(e)f(a)h(discourse)h(ab)q(out)f(what)f(lev)o(el)i (of)75 2428 y(MPI)e(compliance)i(a)e(giv)o(en)h(implemen)o(tation)g (has:)143 2522 y Ff(\017)23 b Fk(MPI-1)f(compliance)i(will)h(mean)d (compliance)i(with)f(MPI)g(1.2.)41 b(This)23 b(is)g(a)g(useful)g(lev)o (el)h(of)189 2578 y(compliance.)c(The)13 b(idea)g(is)g(that)e(the)i (original)g(message-passing)g(mo)q(del,)g(as)f(en)o(visioned)i(during) 189 2635 y(the)e(MPI-1)h(pro)q(cess,)g(is)g(fully)g(supp)q(orted,)h (and)e(there)h(will)h(b)q(e)f(c)o(hanges)g(required)g(from)f(V)l (ersion)189 2691 y(1.1,)h(but)j(no)f(signi\014can)o(t)h(new)g(t)o(yp)q (es)f(of)g(functionalit)o(y)l(.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fk(2)873 b Fe(CHAPTER)15 b(1.)35 b(INTR)o(ODUCTION)16 b(TO)g Fd(MPI-2)143 45 y Ff(\017)23 b Fk(MPI-2)15 b(compliance)i(will)f (mean)g(compliance)g(with)g(MPI)f(2.0.)75 126 y(Guidance)h(for)f (future)g(w)o(ork)f(will)j(b)q(e)f(pro)o(vided)g(in)g(the)f(MPI)h (Journal)f(of)g(Dev)o(elopmen)o(t.)-1768 b Ff(>)15 b Fc(\(Sept\))75 266 y Fg(1.2)59 b(Organization)20 b(of)g(this)f(Do)r (cument)75 368 y Fk(This)14 b(do)q(cumen)o(t)g(is)h(in)f(three)g (parts.)19 b(The)14 b(\014rst)f(con)o(tains)h(prop)q(osed)g (corrections,)g(clari\014cations,)h(and)75 424 y(mo)q(di\014cations)e (to)f(the)g Fd(MPI-1.1)f Fk(Standard,)h(whic)o(h)h(together)f(with)g (these)h(c)o(hanges)f(will)i(b)q(ecome)e Fd(MPI-)75 481 y(1.2)p Fk(.)18 b(The)c(small)h(n)o(um)o(b)q(er)f(of)f(new)i(functions) f(in)h Fd(MPI-1.2)d Fk(are)i(extensions)g(to)f Fd(MPI-1.1)g Fk(concepts,)h(but)75 537 y(do)j(not)h(add)f(an)o(y)g(new)h(t)o(yp)q (es)g(of)f(functionalit)o(y)l(.)28 b(They)17 b(are)h(not)f(an)o (ticipated)h(to)f(require)h(extensiv)o(e)75 594 y(additions)e(or)f(c)o (hanges)h(to)e(existing)j(implemen)o(tations.)k(They)16 b(are)f(exp)q(ected)i(to)d(require)j(some)e(suc)o(h)75 650 y(mo)q(di\014cations.)166 707 y(The)i(second)h(of)f(the)g(this)h (do)q(cumen)o(t)f(con)o(tains)h(the)f Fd(MPI-2)f Fk(Standard)h(Sp)q (eci\014cation.)29 b(It)17 b(adds)75 763 y(substan)o(tial)e(new)g(t)o (yp)q(es)g(of)f(functionalit)o(y)i(to)e Fd(MPI)p Fk(,)g(in)i(most)e (cases)h(sp)q(ecifying)h(functions)g(for)e(an)h(ex-)75 819 y(tended)f(computational)f(mo)q(del)g(\(e.g.,)f(dynamic)h(pro)q (cess)g(creation)g(and)g(one-sided)i(comm)o(unication\))75 876 y(or)g(for)f(a)h(signi\014can)o(t)h(new)g(capabilit)o(y)g(\(e.g.,)e (parallel)j(I/O\).)166 932 y(The)e(follo)o(wing)h(is)g(a)f(list)h(of)f (the)g(c)o(hapters)g(in)h Fd(MPI-2)p Fk(,)e(along)h(with)h(a)f(brief)h (description)g(of)f(eac)o(h.)143 1020 y Ff(\017)23 b Fk(Chapter)13 b Fb(??)p Fk(,)g Fd(MPI-2)g(T)l(erms)g(and)i(Conventions) p Fk(,)g(explains)g(notational)f(terms)f(and)h(con)o(v)o(en)o(tions)189 1077 y(used)h(throughout)g(the)g Fd(MPI-2)g Fk(do)q(cumen)o(t.)143 1165 y Ff(\017)23 b Fk(Chapter)17 b Fb(??)p Fk(,)f Fd(Dynamic)h(Pro)q (cesses)p Fk(,)i(discusses)f(the)f(extension)h(of)f Fd(MPI)g Fk(to)f(remo)o(v)o(e)h(the)g(static)189 1222 y(pro)q(cess)c(mo)q(del)i (in)f Fd(MPI)p Fk(.)e(It)i(giv)o(es)f(routines)h(to)f(allo)o(w)g(for)g (creation)h(and)f(remo)o(v)m(al)h(of)f(pro)q(cesses.)143 1310 y Ff(\017)23 b Fk(Chapter)d Fb(??)p Fk(,)g Fd(One)h(Sided)h (Communications)p Fk(,)e(de\014nes)i(comm)o(unication)f(routines)f (whic)o(h)h(are)189 1366 y(one)16 b(directional.)23 b(These)17 b(include)h(shared)e(memory)f(t)o(yp)q(e)h(op)q(erations)h (\(put/get\),)d(and)i(Read-)189 1423 y(Mo)q(dify-W)l(rite)g(op)q (erations.)143 1511 y Ff(\017)23 b Fk(Chapter)13 b Fb(??)p Fk(,)h Fd(Extended)i(Collective)f(Op)q(erations)p Fk(,)g(giv)o(es)g (the)f(extensions)h(to)e Fd(MPI)h Fk(to)f(allo)o(w)i(non-)189 1568 y(blo)q(c)o(king)23 b(collectiv)o(e)i(op)q(erations,)f(apply)f (collectiv)o(e)h(op)q(erations)e(and)h(top)q(ologies)g(to)f(in)o(ter-) 189 1624 y(comm)o(unicators,)14 b(and)h(additional)i(features.)143 1713 y Ff(\017)23 b Fk(Chapter)11 b Fb(??)p Fk(,)h Fd(External)h (Interfaces)p Fk(,)g(de\014nes)g(those)f(routines)g(designed)h(for)f (allo)o(wing)h(dev)o(elop)q(ers)189 1769 y(to)f(la)o(y)o(er)i(on)f(top) g(of)g Fd(MPI)p Fk(.)g(This)h(includes)i(generalized)f(requests)f(and)f (routines)h(to)f(deco)q(de)i Fd(MPI)189 1826 y Fk(opaque)g(ob)s(jects.) 143 1914 y Ff(\017)23 b Fk(Chapter)15 b Fb(??)p Fk(,)f Fd(Language)h(Binding)p Fk(,)h(extends)f(the)h(binding)h(to)d(F)l (ortran)g(90)h(and)g(C++.)143 2003 y Ff(\017)23 b Fk(Chapter)15 b Fb(??)p Fk(,)f Fd(Real)h(Time)p Fk(,)f(discusses)i Fd(MPI-2)f Fk(supp)q(ort)g(for)g(real)g(time)h(pro)q(cessing.)143 2091 y Ff(\017)23 b Fk(Chapter)15 b Fb(??)p Fk(,)f Fd(Miscellaneous)p Fk(,)i(discusses)g(issues)g(left)g(o)o(v)o(er)e(from)h(the)g(initial)i Fd(MPI-2)e Fk(e\013ort.)143 2180 y Ff(\017)23 b Fk(Chapter)15 b Fb(??)p Fk(,)f Fd(I/O)p Fk(,)g(discusses)j Fd(MPI-2)d Fk(supp)q(ort)h(for)g(I/O.)143 2268 y Ff(\017)23 b Fk(Annex)17 b Fb(??)p Fk(,)f Fd(Sp)q(eci\014c)j(Language)d(Bindings)p Fk(,)j(giv)o(es)e(sp)q(eci\014c)h(syn)o(tax)e(in)i(F)l(ortran)d(77)i (and)f(C,)h(for)189 2325 y(all)f Fd(MPI-2)e Fk(functions,)i(constan)o (ts,)e(and)h(t)o(yp)q(es.)166 2412 y(The)d(third)h(part)f(of)f(this)i (do)q(cumen)o(t)f(consists)h(of)e(ideas)i(that)f(w)o(ere)g(discussed)h (in)g(the)f(MPI)g(F)l(orum)75 2469 y(and)i(deemed)i(to)d(ha)o(v)o(e)h (v)m(alue,)i(but)e(are)g(not)g(included)j(in)e(the)g Fd(MPI)e Fk(Standard.)20 b(They)14 b(are)g(part)g(of)g(the)75 2525 y(\\Journal)i(of)f(Dev)o(elopmen)o(t",)g(lest)h(go)q(o)q(d)g (ideas)g(b)o(y)f(lost)h(and)g(in)g(order)f(to)g(pro)o(vide)h(a)g (starting)e(p)q(oin)o(t)75 2582 y(for)h(further)g(w)o(ork.)166 2638 y(The)f Fd(MPI)g(F)o(unction)h(Index)g Fk(is)f(a)g(simple)i(index) f(sho)o(wing)f(the)g(lo)q(cation)h(of)f(the)g(precise)h(de\014nition)75 2695 y(of)g(eac)o(h)g Fd(MPI-2)g Fk(function,)g(together)g(with)g(b)q (oth)g(C)g(and)h(F)l(ortran)e(bindings.)-1541 b Ff(?)15 b Fc(\(Sept\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 377 a Fi(Biblio)q(graph)m(y)75 600 y Fk([1])22 b(Message)d(P)o(assing)g(In)o(terface)g(F)l(orum.)32 b(MPI:)19 b(A)h(Message-P)o(assing)f(In)o(terface)g(standard.)32 b Fa(The)146 656 y(International)13 b(Journal)h(of)h(Sup)n(er)n(c)n (omputer)g(Applic)n(atons)e(and)i(High)f(Performanc)n(e)g(Computing)p Fk(,)146 713 y(8,)g(1994.)964 2828 y(3)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Aug 12 18:23:55 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA10456; Mon, 12 Aug 1996 18:23:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA20135 for mpi-core-out; Mon, 12 Aug 1996 17:23:36 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA20060; Mon, 12 Aug 1996 17:21:15 -0500 Message-Id: <199608122221.RAA20060@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov, mpi-misc@antares.mcs.anl.gov Subject: new misc chapter, MPI-1.2 part Date: Mon, 12 Aug 1996 17:21:12 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Mon Aug 12 16:48:43 1996 %%Pages: 16 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.08.12:1648 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 1 106 df<030007800780030000000000000000007F807F80038003800380 038003800380038003800380038003800380FFFCFFFC0E187D9714>105 D E /Fb 10 121 df<387CFEFEFE7C3807077C860F>46 D<00E00001E0000FE000FFE000 F3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000 03E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000FFFF80 FFFF80111D7C9C1A>49 D<01FC0007FF000E0F801E0FC03F07E03F07E03F07E03F07E01E 0FC0000FC0000F80001F0001FC0001FC00000F800007C00003E00003F00003F83803F87C 03F8FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003FC00151D7E9C1A>51 D69 D<07FC001FFF003F0F803F07 C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803 E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 D<01FE0007FF800F83C01E01E03E00F0 7C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E0018 0F807007FFE000FF8015147F9318>101 D108 DI112 D120 D E /Fc 24 119 df<00003F03E00000C386700001878CF00003879CF0000303186000070038000007 0038000007003800000E003800000E007000000E007000000E00700000FFFFFF80001C00 7000001C00E000001C00E000001C00E000001C00E000003800E000003801C000003801C0 00003801C000003801C000007001C0000070038000007003800000700380000070038000 006003800000E007000000E007000000E007000000E007000000C006000001C00E000001 C00E000031860C0000798F180000F31E100000620C6000003C07C000002429829F1C>11 D<70F8F8F0E005057B840E>46 D<00000200000006000000060000000E0000001E000000 1E0000003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F000002 0F0000040F00000C0F0000080F0000100F0000100F0000200F80003FFF800040078000C0 07800080078001000780010007800200078002000780060007801E000F80FF807FF81D20 7E9F22>65 D<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C 0004003C00040078080800780800007808000078180000F0300000FFF00000F0300000F0 300001E0200001E0200001E0200001E0001003C0002003C0002003C0004003C000400780 00800780018007800100078007000F001F00FFFFFE001F1F7D9E1F>69 D<01FFFF00001E03C0001E00E0001E0070001E0078003C0078003C0078003C0078003C00 78007800F0007800F0007801E0007801C000F0070000F01E0000FFF00000F0380001E01C 0001E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003C01E0007803C0007803C 0807803C0807803C100F801C10FFF00C20000007C01D207D9E21>82 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207B9F15>I< 007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F00000 F00000F00000F0000070010070020030040018380007C00011147C9315>I<0000780003 F80000700000700000700000700000E00000E00000E00000E00001C00001C000F1C00389 C00707800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E 20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C01C207010E011C01 3C013802780C7BF07C00F000F000F000F0007000700170023804183807C010147C9315> I<00007800019C00033C00033C000718000700000700000E00000E00000E00000E00000E 0001FFE0001C00001C00001C00001C000038000038000038000038000038000070000070 0000700000700000700000700000E00000E00000E00000E00000C00001C00001C0000180 003180007B0000F300006600003C00001629829F0E>I<003C6000E27001C1E00380E007 00E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C03803C07003C07001C 0F001C17000C2E0003CE00000E00000E00001C00001C00301C00783800F0700060E0003F 8000141D7E9315>I<01E0000FE00001C00001C00001C00001C000038000038000038000 038000070000070000071E000763000E81800F01C00E01C00E01C01C03801C03801C0380 1C0380380700380700380700380E10700E20700C20701C20700C40E00CC060070014207D 9F17>I<00C001E001E001C000000000000000000000000000000E003300230043804300 470087000E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E> I<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030 000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00800E00800E 00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C00012147C93 15>I E /Fd 22 91 df<001C0000003E0000003E0000002E0000006700000067000000E7 800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F0000700 F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E003800 1E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F00001E00003C00003C0000 780000780000780000F00000F00000F00000F00000F00000F00000F00000F00000F00000 7800007800007800003C00003C00001E00000F000807801803C07800FFF0003F80151F7D 9D1B>IIII<003F 8001FFF003C0F80780380F00181E00003C00003C0000780000780000780000F00000F000 00F00000F00000F00000F00000F007F8F007F8F000387800387800387800383C00383C00 381E00380F003807803803C0F801FFF0003F80151F7D9D1C>III76 DII<003F000001FFE00003FFF00007C0F8000F807C001E001E003E001F003C 000F00780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F0 0003C0F00003C0F00003C0F80007C078000780780007807C000F803C000F003E001F001F 003E000F807C0007C0F80003FFF00001FFE000003F00001A1F7E9D1F>II<003F000001FFE00003FFF00007C0F8000F807C00 1F003E003E001F003C000F00780007807800078078000780F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C07800078078000780780E0780 3C0F0F003E079F001E03DE000F83FC0007C1F80003FFF00001FFF800003F780000003C00 00003E0000001F0000000F801A237E9D1F>II<03F8000FFE001C0F00380700700300600000E00000E00000E00000E00000F0 00007800007F00003FE0001FFC0007FE0001FF00001F800007800003C00003C00001C000 01C00001C00001C0C00180E00380F007007C0E001FFC0007F000121F7E9D17>II I<78000E007C001E003C003C001E0038000F0070000F00F0000781E00003C1C00001C3C0 0001E7800000F70000007E0000003E0000003C0000003C0000007E00000077000000E780 0001E3800003C1C0000381E0000700F0000F00F8000E0078001C003C003C003E0078001F 0070000F00F0000F80191D7F9C1C>88 DI<7FFFF07F FFF00001E00003E00003C00007C0000780000F00001F00001E00003E00003C0000780000 F80000F00001F00001E00003C00007C0000780000F80000F00001E00003E00003C00007C 0000780000FFFFF0FFFFF0141D7E9C19>I E /Fe 49 123 df<000078007C7800FC7801 FC7803C000038000078000078000078000078000078000078000078000078000FFFC78FF FC78FFFC7807807807807807807807807807807807807807807807807807807807807807 8078078078078078078078078078078078078078078078078078152480A31A>12 D45 DI<00C001C007C0FFC0FFC0 FBC003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C0FFFFFFFFFFFF10227CA118>49 D<03F0000FFC001FFE003C1F003007807007C06003C0E003E0C001E04001E04001E00001 E00001E00001E00003C00003C0000780000780000F00001E00003C0000780000F00001E0 0001C0000380000700000E00001C0000380000700000FFFFE0FFFFE0FFFFE013227EA118 >I<01F00007FC001FFF003E0F003807807003C02003C02003C00003C00003C00003C000 0780000780000F00001E0003FC0003F80003FE00000F000007800003C00003C00001E000 01E00001E00001E00001E08001E0C003C0E003C07007803C0F801FFF000FFC0003F00013 237EA118>I<001F00001F00002F00002F00006F0000EF0000CF0001CF0001CF00038F00 038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F00780F00F00F00FFFFF8 FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F00000F00000F0015217F A018>I<3FFF803FFF803FFF803C00003C00003C00003C00003C00003C00003C00003C00 003C00003CF8003FFE003FFF003F0F803E07803C03C03803C00001E00001E00001E00001 E00001E00001E00001E04003C04003C0E003C07007807C1F003FFE000FFC0003F0001322 7EA018>I<007E0001FF0003FF0007C1000F00001E00001E00003C00003C000078000078 000078F800F1FE00F7FF00FF0F80FC0780F803C0F803C0F801C0F001E0F001E0F001E0F0 01E0F001E07001E07001E07801E07803C03803C03C03801C07801F0F000FFE0007FC0001 F00013237EA118>I<001F0000001F0000003F8000003F8000003B8000007BC0000073C0 000071C00000F1E00000F1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078 000380780007803C0007803C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF 001C000F003C0007803C00078038000780780003C0780003C0700003C0F00001E0F00001 E0E00001E01B237EA220>65 DI<000FF000007FFC0000FFFF0001F01F0003C007 00078000000F0000001E0000003E0000003C0000003C0000007800000078000000780000 00F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00000 007800000078000000780000003C0000003C0000003E0000001E0000000F000000078000 8003C0038001F00F8000FFFF00007FFC00000FF00019257DA31F>IIII<000FF000003FFE0000FFFF8001F80F8003E00380078000000F0000001E0000001E 0000003C0000003C000000780000007800000078000000F0000000F0000000F0000000F0 000000F0000000F0000000F000FFC0F000FFC0F000FFC0780003C0780003C0780003C03C 0003C03C0003C01E0003C01E0003C00F0003C0078003C003E003C001F807C000FFFFC000 3FFF00000FF8001A257DA321>I73 D76 DII<001FC0 00007FF00001FFFC0003F07E0007C01F000F800F801F0007C01E0003C03C0001E03C0001 E0780000F0780000F0780000F070000070F0000078F0000078F0000078F0000078F00000 78F0000078F0000078F0000078F0000078780000F0780000F0780000F07C0001F03C0001 E03E0003E01E0003C01F0007C00F800F8007C01F0003F07E0001FFFC00007FF000001FC0 001D257DA324>II82 D<00FE0003FFC007FFE00F81E01E00603C00003C 00007800007800007800007800007800007C00003C00003F00001FC0000FFC0007FF0001 FF80003FC00007E00001F00000F00000F8000078000078000078000078000078000078C0 00F0E000F0F801E07E07C03FFF800FFF0001FC0015257EA31B>II I90 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00F F00FF00FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F 03803C0180780000780000700000F00000F00000F00000F00000F00000F0000078000078 00007800003C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC0 1FFFC03F0FC03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0 F003C07803C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC 001FFE003E0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F000 007000007800007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF 01FF03C0038007800780078007800780078007800780FFF8FFF8FFF80780078007800780 078007800780078007800780078007800780078007800780078007800780102380A20F> I<01F07807FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F 000FFE001FFC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800 F8F00078F00078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>I105 D107 DIII<01FC0007FF000FFF801F07C03C01E0 7800F07800F0700070F00078F00078F00078F00078F00078F000787800F07800F07C01F0 3E03E01F07C00FFF8007FF0001FC0015167F9518>II114 D<07F01FFC3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC0 0FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F00 0F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E0 0E1C7F9B12>III<7801F07C01E03E03C01E07C00F 0780078F0007DE0003FC0001FC0000F80000700000F80001FC0003DC00039E00078F000F 07801E07801E03C03C01E07800F0F800F81516809516>120 DI<7FFFC07FFFC07FFFC0000F80000F00 001E00003E00007C0000780000F00001F00003E00003C0000780000F80001F00001E0000 3C00007C0000FFFFC0FFFFC0FFFFC012167F9515>I E /Ff 25 90 df<3078F8787005057C840D>46 D<000C001C00FC0F3800380038003800380038007000 70007000700070007000E000E000E000E000E000E001C001C001C001C001C001C0038003 C0FFFE0F1E7C9D17>49 D<003F8000C1E00100F00200780400780400780F007C0F807C0F 807C0F00780600780000F80000F00001E00001C0000380000700000E00001C0000380000 600000C0000180000300200600200800401000403FFFC07FFF80FFFF80161E7E9D17>I< 007F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E00003C0000380 000700000E0000F800000E000007000007800007C00003C00007C03007C07807C0F807C0 F807C0F00780800F00400E00201C0018780007E000141F7D9D17>I<000F8000704000C0 200180E00301E00701E00E00C01E00001C00003C000038000078F800790E007A07007C03 00F80380F80380F003C0F003C0F003C0F003C0F00780E00780E00780E00700E00F00600E 00701C0030180018700007C000131F7C9D17>54 D<2000003FFFE07FFFC07FFF80400100 C00200800200800400000800001000002000004000004000008000018000030000030000 0700000600000E00000E00001E00001C00001C00003C00003C00003C0000780000780000 780000300000131F799D17>I<0000100000001800000038000000380000007800000078 000000FC000001BC0000013C0000033C0000023C0000063C0000043E0000081E0000081E 0000101E0000101E0000201E0000200F0000400F0000400F0000FFFF0000800F0001000F 8001000780020007800200078004000780040007800C0007C03E0007C0FF807FFC1E207E 9F22>65 D<0001F808000E061800380138007000F801E0007803C0007007800030078000 300F0000301F0000301E0000303E0000203C0000007C0000007C0000007C0000007C0000 00F8000000F8000000F8000000F8000000F80000007800004078000080780000803C0000 803C0001001C0002000E00020006000C000300100001C0E000003F00001D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E007800 1E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E01E000 3E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C003 C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038 003C001800780018007800080078000800780008007800080078080800F0100000F01000 00F0100000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001E00008 01E0001003C0001003C0001003C0002003C0002003C0006003C000C0078001C0078007C0 FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C0018007800180078000800 78000800780008007800080078000800F0100000F0100000F0100000F0300000F0700000 FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0000003C0000003 C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F7E9E1E>I<07FF C7FFC0007C00F800003C007800003C007800007800F000007800F000007800F000007800 F000007800F000007800F00000F001E00000F001E00000F001E00000F001E00000FFFFE0 0000F001E00001E003C00001E003C00001E003C00001E003C00001E003C00001E003C000 03C007800003C007800003C007800003C007800003C007800003C007800007800F000007 C00F8000FFF8FFF800221F7E9E22>72 D<07FFE0007C00003C00003C0000780000780000 780000780000780000780000F00000F00000F00000F00000F00000F00001E00001E00001 E00001E00001E00001E00003C00003C00003C00003C00003C00003C00007800007C000FF FC00131F7F9E10>I<07FFE07FE0007C001F00003C000C00003C00180000780010000078 004000007800800000780100000078020000007804000000F008000000F010000000F060 000000F0F0000000F1F0000000F278000001E478000001E878000001F03C000001E03C00 0001E01E000001E01E000003C00F000003C00F000003C00F000003C007800003C0078000 03C003C000078003C00007C007E000FFFC3FFC00231F7E9E23>75 D<07FFF000007E0000003C0000003C000000780000007800000078000000780000007800 000078000000F0000000F0000000F0000000F0000000F0000000F0000001E0000001E000 0001E0000001E0000001E0008001E0010003C0010003C0010003C0030003C0020003C006 0003C0060007801E0007807C00FFFFFC00191F7E9E1C>I<07FC0000FFC0007C0000F800 003C00017800003C00017800004E0002F000004E0002F000004E0004F000004E0004F000 004E0008F000004E0008F00000870011E00000870011E00000870021E00000870021E000 00870041E00000838041E00001038083C00001038083C00001038103C00001038203C000 0101C203C0000101C403C0000201C40780000201C80780000201C80780000201D0078000 0200F00780000600E00780000600E00F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A> I<07FC01FFC0003E003E00003E001800003E001800004F001000004F0010000047801000 00478010000043C010000043C010000083C020000081E020000081E020000080F0200000 80F020000080782000010078400001007C400001003C400001003C400001001E40000100 1E400002000F800002000F800002000F8000020007800002000780000600038000060003 00000F00010000FFE0010000221F7E9E22>I<0003F800001E0E000038070000E0038001 C001C003C001E0078000E00F0000F00F0000F01E0000F01E0000F83E0000F83C0000F87C 0000F87C0000F87C0000F87C0000F8F80001F0F80001F0F80001F0F80001F0F80003E078 0003E0780003C0780007C07C0007803C000F003C001E001E001C000E0038000700F00003 C3C00000FE00001D217B9F23>I<07FFFF00007C03C0003C01E0003C00F0007800F00078 00F8007800F8007800F8007800F8007800F000F001F000F001E000F003C000F0078000F0 0F0000FFF80001E0000001E0000001E0000001E0000001E0000001E0000003C0000003C0 000003C0000003C0000003C0000003C000000780000007C00000FFFC00001D1F7E9E1F> I<07FFFC00007C0700003C03C0003C01E0007801E0007801F0007801F0007801F0007801 F0007801E000F003E000F003C000F0078000F00F0000F03C0000FFF00001E0300001E038 0001E01C0001E01C0001E01C0001E01E0003C03E0003C03E0003C03E0003C03E0003C03E 0003C03E0207803E0407C01F04FFFC0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E00080E00080E00000F00 000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E00000F00000F04000 E04000E04000E04000E06000C0600180E00380F80300C60C0081F80016217D9F19>I<3F FFFFF03C0780F03007803060078030400F0010400F0010C00F0010800F0010800F001080 0F0010001E0000001E0000001E0000001E0000001E0000001E0000003C0000003C000000 3C0000003C0000003C0000003C0000007800000078000000780000007800000078000000 78000000F0000001F800007FFFE0001C1F7A9E21>I86 DI89 D E /Fg 19 119 df<03800007E0000FE0001E70001C70001C70001C70001C77E01CE7E01DE7E00FC7000F 8E000F0E001E0E003F1C007F1C00739C00E3F800E1F800E0F1C0E0F1C071F9C07FFFC03F 9F801E070013197F9816>38 D<00E001E0038007000E001C001C00380038007000700070 00E000E000E000E000E000E000E000E000E000700070007000380038001C001C000E0007 00038001E000E00B217A9C16>40 DI<387C7E7E3E0E1E1C78F060070B798416>44 D<70F8F8F870000000000000000070F8F8F8700512789116>58 D73 D77 D<7FF800FFFE007FFF001C0F80 1C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FF8001C0000 1C00001C00001C00001C00001C00001C00007F0000FF80007F000012197F9816>80 D86 D<7E0000FE00007E00000E00000E00000E00000E00000E3E00 0EFF000FFF800F83C00F00E00E00E00E00700E00700E00700E00700E00700E00700E00E0 0F01E00F83C00FFF800EFF00063C001419809816>98 D<03E00FF81FFC3C1E780E7007E0 07FFFFFFFFFFFFE000E000700778073C0F1FFE0FFC03F010127D9116>101 D<018003C003C0018000000000000000007FC07FC07FC001C001C001C001C001C001C001 C001C001C001C001C001C07FFFFFFF7FFF101A7D9916>105 D<7E3C00FEFE007FFF000F 87800F03800E03800E03800E03800E03800E03800E03800E03800E03800E03800E03807F C7F0FFE7F87FC7F01512809116>110 D<03E0000FF8001FFC003C1E00780F00700700E0 0380E00380E00380E00380E00380F00780700700780F003C1E001FFC000FF80003E00011 127E9116>I114 D<0FEC3FFC7FFCF03CE01CE01C70007F801FF007F8003C600EE00EF00EF81EFFFCFFF8C7 E00F127D9116>I<0300000700000700000700000700007FFF00FFFF00FFFF0007000007 000007000007000007000007000007000007010007038007038007038007870003FE0001 FC0000F80011177F9616>I<7E1F80FE3F807E1F800E03800E03800E03800E03800E0380 0E03800E03800E03800E03800E03800E03800E0F800FFFF007FBF803E3F01512809116> I<7F1FC0FF1FE07F1FC01C07001E0F000E0E000E0E000E0E00071C00071C00071C00071C 0003B80003B80003B80001F00001F00000E00013127F9116>I E /Fh 15 119 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D<00038000000380000007C0000007C0000007C000000FE000000FE000001FF000001BF0 00001BF0000031F8000031F8000061FC000060FC0000E0FE0000C07E0000C07E0001803F 0001FFFF0003FFFF8003001F8003001F8006000FC006000FC00E000FE00C0007E0FFC07F FEFFC07FFE1F1C7E9B24>65 D68 D<0FF8001C1E003E0F803E07 803E07C01C07C00007C0007FC007E7C01F07C03C07C07C07C0F807C0F807C0F807C0780B C03E13F80FE1F815127F9117>97 D<03FC000E0E001C1F003C1F00781F00780E00F80000 F80000F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E 9115>99 D<01FC000F07001C03803C01C07801C07801E0F801E0F801E0FFFFE0F80000F8 0000F800007800007C00603C00601E00C00F038001FC0013127F9116>101 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F001F 001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D108 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I114 D<1FD830786018E018E018F000FF807FE07FF01F F807FC007CC01CC01CE01CE018F830CFC00E127E9113>I<030003000300030007000700 0F000F003FFCFFFC1F001F001F001F001F001F001F001F001F001F0C1F0C1F0C1F0C0F08 079803F00E1A7F9913>II< FFC1FCFFC1FC1F00601F80E00F80C00FC0C007C18007C18003E30003E30001F60001F600 01FE0000FC0000FC0000780000780000300016127F9119>I E /Fi 16 117 df<020408103020604040C0C0C0C0C0C0C0C0404060203010080402071A7F920C >40 D<8040201018080C0404060606060606060604040C081810204080071A7E920C>I< 1F00318060C04040C060C060C060C060C060C060C060C060404060C031801F000B107F8F 0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C000C000C000C00FF80 09107E8F0F>I<1F00618040C08060C0600060006000C00180030006000C00102020207F C0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F00008000400060C060C060 804060801F000B107F8F0F>I<0300030007000F000B001300330023004300C300FFE003 000300030003001FE00B107F8F0F>I<20803F002C002000200020002F00308020400060 00600060C06080C061801F000B107F8F0F>I<0780184030C060C06000C000CF00F080E0 40C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC08080808001000200 040004000C0008000800180018001800180018000B117E900F>I<1F00318060C060C060 C071803F000F00338061C0C060C060C060404060801F000B107F8F0F>I<1F00318060C0 C040C060C060C06040E021E01E600060004060C0608043003E000B107F8F0F>I<1F2060 E0006080208020800060003E001F8000C00060002080208020C040E0C09F000B117E9011 >83 D<1F00318060C0C0C0FFC0C000C000C000604030801F000A0B7F8A0E>101 D112 D<10103030FE3030303030323232321C070F7F8E0C>116 D E /Fj 3 64 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE 3FFC1FF80FF003C010127D9317>15 D62 D<00040000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000FFFFFF E0FFFFFFE01B1C7C9B23>I E /Fk 71 123 df<007E1F0001C1B1800303E3C00703C3C0 0E03C1800E01C0000E01C0000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C000 0E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C000 0E01C0000E01C0000E01C0000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C00E01800E00000E00000E00000E00000E0000FFFF C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C07F87F8151D809C17>I<007FC001C1C00303C00703C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F CFF8151D809C17>I<6060F0F0F8F86868080808080808101010102020404080800D0C7F 9C15>34 D<00E00000019000000308000003080000070800000708000007080000070800 000710000007100000072000000740000003C03FE003800F00038006000380040005C004 0009C0080010E0100030E010006070200060702000E0384000E03C4000E01C8000E00F00 20E0070020700780403009C0401830E18007C03E001B1F7E9D20>38 D<60F0F8680808081010204080050C7C9C0C>I<004000800100020006000C000C001800 1800300030007000600060006000E000E000E000E000E000E000E000E000E000E000E000 E000600060006000700030003000180018000C000C00060002000100008000400A2A7D9E 10>I<800040002000100018000C000C000600060003000300038001800180018001C001 C001C001C001C001C001C001C001C001C001C001C0018001800180038003000300060006 000C000C00180010002000400080000A2A7E9E10>I<60F0F0701010101020204080040C 7C830C>44 DI<60F0F06004047C830C>I<03C00C301818300C30 0C700E60066006E007E007E007E007E007E007E007E007E007E007E007E007E007600660 06700E300C300C18180C3007E0101D7E9B15>48 D<030007003F00C70007000700070007 000700070007000700070007000700070007000700070007000700070007000700070007 000F80FFF80D1C7C9B15>I<07C01830201C400C400EF00FF80FF807F8077007000F000E 000E001C001C00380070006000C00180030006010C01180110023FFE7FFEFFFE101C7E9B 15>I<07E01830201C201C781E780E781E381E001C001C00180030006007E00030001C00 1C000E000F000F700FF80FF80FF80FF00E401C201C183007E0101D7E9B15>I<000C0000 0C00001C00003C00003C00005C0000DC00009C00011C00031C00021C00041C000C1C0008 1C00101C00301C00201C00401C00C01C00FFFFC0001C00001C00001C00001C00001C0000 1C00001C0001FFC0121C7F9B15>I<300C3FF83FF03FC020002000200020002000200023 E024302818301C200E000E000F000F000F600FF00FF00FF00F800E401E401C2038187007 C0101D7E9B15>I<00F0030C06040C0E181E301E300C700070006000E3E0E430E818F00C F00EE006E007E007E007E007E007600760077006300E300C18180C3003E0101D7E9B15> I<4000007FFF807FFF007FFF004002008004008004008008000010000010000020000060 0000400000C00000C00001C0000180000180000380000380000380000380000780000780 00078000078000078000078000030000111D7E9B15>I<03E00C301008200C2006600660 0660067006780C3E083FB01FE007F007F818FC307E601E600FC007C003C003C003C00360 026004300C1C1007E0101D7E9B15>I<03C00C301818300C700C600EE006E006E007E007 E007E007E0076007700F300F18170C2707C700060006000E300C780C78187010203030C0 0F80101D7E9B15>I<60F0F0600000000000000000000060F0F06004127C910C>I<60F0F0 600000000000000000000060F0F0701010101020204080041A7C910C>I<0FE03038401C E00EF00EF00EF00E000C001C0030006000C0008001800100010001000100010001000000 000000000000000003000780078003000F1D7E9C14>63 D<000600000006000000060000 000F0000000F0000000F00000017800000178000001780000023C0000023C0000023C000 0041E0000041E0000041E0000080F0000080F0000180F8000100780001FFF80003007C00 02003C0002003C0006003E0004001E0004001E000C001F001E001F00FF80FFF01C1D7F9C 1F>65 DI<001F808000E061800180198007 0007800E0003801C0003801C00018038000180780000807800008070000080F0000000F0 000000F0000000F0000000F0000000F0000000F0000000F0000000700000807800008078 000080380000801C0001001C0001000E000200070004000180080000E03000001FC00019 1E7E9C1E>III< FFFFF80F00780F00180F00080F00080F000C0F00040F00040F02040F02000F02000F0200 0F06000FFE000F06000F02000F02000F02000F02000F00000F00000F00000F00000F0000 0F00000F00000F8000FFF800161C7E9B1B>I<001F808000E0618001801980070007800E 0003801C0003801C00018038000180780000807800008070000080F0000000F0000000F0 000000F0000000F0000000F0000000F000FFF0F0000F8070000780780007807800078038 0007801C0007801C0007800E00078007000B800180118000E06080001F80001C1E7E9C21 >III<1FFF00F800780078007800780078007800780078007800780078007800780078 007800780078007800787078F878F878F878F0F040E021C01F00101D7F9B15>I76 DII<003F800000 E0E0000380380007001C000E000E001C0007003C00078038000380780003C0780003C070 0001C0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E070 0001C0780003C0780003C0380003803C0007801C0007000E000E0007001C000380380000 E0E000003F80001B1E7E9C20>II82 D<07E0801C1980300580700380600180E00180E00080E000 80E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007C00003 C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E7E9C17 >I<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F0020800F 0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F 800003FFFC001B1C7F9B1E>II87 D<08081010202040404040808080808080B0B0F8F8787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C0001FC000F1C00381C00701C00601C 00E01C40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E0 0060007004300418080C3007C00E127E9112>I<003F0000070000070000070000070000 070000070000070000070000070000070003E7000C1700180F00300700700700600700E0 0700E00700E00700E00700E00700E00700600700700700300700180F000C370007C7E013 1D7E9C17>I<03E00C301818300C700E6006E006FFFEE000E000E000E000600070023002 18040C1803E00F127F9112>I<00F8018C071E061E0E0C0E000E000E000E000E000E00FF E00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E007FE00F 1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C00381C00381C00181800 1C38000C300013C0001000003000001800001FF8001FFF001FFF803003806001C0C000C0 C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C0018000000000000000000000000000000FC001C 001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80091D7F 9C0C>I<00C001E001E000C000000000000000000000000000000FE000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0 F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180E001C0E001C0E001C0E001C0 E001C0E001C06001807003803003001806000E1C0003F00012127F9115>II<03C1000C3300180B00300F00700700700700E00700E00700E00700E0 0700E00700E00700600700700700300F00180F000C370007C70000070000070000070000 0700000700000700000700003FE0131A7E9116>II<1F9030704030C010 C010E010F8007F803FE00FF000F880388018C018C018E010D0608FC00D127F9110>I<04 000400040004000C000C001C003C00FFE01C001C001C001C001C001C001C001C001C001C 101C101C101C101C100C100E2003C00C1A7F9910>III< FF3FCFE03C0F03801C0701801C0701001C0B01000E0B82000E0B82000E1182000711C400 0711C4000720C40003A0E80003A0E80003C0680001C0700001C070000180300000802000 1B127F911E>I<7F8FF00F03800F030007020003840001C80001D80000F0000070000078 0000F800009C00010E00020E000607000403801E07C0FF0FF81512809116>II<7FFC70386038407040F040E041C003C0038007000F040E041C043C 0C380870087038FFF80E127F9112>I E /Fl 58 123 df<00F8F001F8F003F8F00780F0 0700F00F00F00F00F00F00F00F00F00F00F00F00F00F00F0FFF8F0FFF8F0FFF8F00F00F0 0F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F0 0F00F00F00F00F00F00F00F01420809F18>13 D<007000E001C00380078007000E001E00 1E003C003C003C0078007800780078007000F000F000F000F000F000F000F000F000F000 F000F000F000700078007800780078003C003C003C001E001E000E0007000780038001C0 00E000700C2E7EA112>40 DI<787878781830306060E0050A7D830D>44 DII<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F 7C9E17>49 D58 D<001F0000001F0000003F8000003B8000003B8000007BC0000073C0000071C00000F1E0 0000E1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C 0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F001E000F003C000F803C0007 803C000780780007C0780003C0780003C0F00003E01B207F9F1E>65 DI<001FC000 FFF801FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000780000F0 0000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00003C 00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>IIII<001FE000FFF801FFFE03E03E07800E0F0000 1E00003E00003C00007C0000780000780000780000F00000F00000F00000F00000F00000 F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07801E 03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F000000FF C00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F00780007807800 078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F80007C07800078078000780780007803C000F003C000F001E001E001F00 3E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC00003FFF000 07E1F8000F807C001F003E001E001E003C000F003C000F00780007807800078078000780 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE000F83FC00 07E1F80003FFF00000FFF800003F780000007C0000003E0000001E0000001F0000000F80 1A277DA021>II<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800007C 00003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001E000 01E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA0 19>IIIII<780007807C000F003E001F001E001E000F003C000F807C000780780003C0 F00003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000003F 0000007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C000F00 3E001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>III<07E03FF87FFC701E401F000F000F000F00 3F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000 F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F8 0014147F9317>II<03C7800FF780 1FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F80780 7807807C0F803E1F801FFF800FF78007C780000780000780000780000780000780000780 000780000780000780111D7E9317>II<07F01FFC3FFC780C78 00780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311 >I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E00 1E001E001E001E001E001E201FF00FF007C00C1A7F9910>II< F003C0F003C07803807807807807803C0F003C0F003C0F001E0E001E1E001E1E000F1C00 0F3C000F3C0007380007380007B80003F00003F00001E00012147F9315>II<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F8 0000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414 809315>II<7FFF7FFF7FFF003E00 3C007800F800F001E003E007C007800F001F001E003C007C00FFFFFFFFFFFF10147F9314 >I E /Fm 68 126 df<030600078F00078F00078F00078F00078F00078F007FFFC0FFFF E0FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFF C01E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>35 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3 800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E078 7070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F001E001C 00380038007000700070007000E000E000E000E000E000E000E000E00070007000700070 00380038001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070007000 70007000700070007000E000E000E000E001C001C0038007800F001E003C007800F00060 000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007 F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<3078FCFC78300606778518>46 D<01800380038007800F803F80FF80FB8043800380038003800380038003800380038003 8003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>49 D<03F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001 C00001C00003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00 E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C078 01C03001C00001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C000 00E00000E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B 18>I<3078FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<7FFF C0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C 7E9318>61 D<00700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00 038E00038E00038E000306000707000707000707000707000FFF800FFF800FFF800E0380 0E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E0 700000E00000E00000E00000E00000E00000E00000E00000E000007000007000E07000E0 3800E03C00E01E01C00F07C007FF8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF 001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C00 701C00701C00701C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE 007FF800141C7F9B18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001 C07001C0700000E00000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001 C07003C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF 8FF87F07F01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01FFFC01F FFC01FFFC01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C07F 07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7F07 F0FF87F87F07F01C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0001DF0 001DF0001FB8001FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03801C01 C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18> II<7E07F0FF0FF87F07F01D81C01D81C01D 81C01DC1C01CC1C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C 39C01C39C01C19C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B 18>I<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380F00780 700700780F007FFF003FFE000FF800111C7D9B18>II<0FF8003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380E00380E1E380E1E380F0 E78070F700787F007FFF003FFE000FFC00001C00001E00000E00000F0000070000070011 227D9B18>I<7FF800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C0 1C03801C0F801FFF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C0380 1C039C1C039C1C039C7F01F8FF81F87F00F0161C7F9B18>I<03F3801FFF803FFF807C0F 80700780E00380E00380E00380E000007000007800003F00001FF00007FE0000FF00000F 800003C00001C00000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F8 00131C7E9B18>I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E0703800700000700000 700000700000700000700000700000700000700000700000700000700000700000700000 700000700000700000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C00 03DC0001F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E00 070F000707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C070078070070000 0F00001E00001C00003C0000780000700000F00001E00001C00003C0000780000700000F 00001E00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I<0038007C007C007C003800000000000000000FFC1FFC0FFC001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C6038F078FFF07FE03F800E277E9C18>II<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF800 1F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F0019148193 18>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0 000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001 E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807F FFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F 00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E 00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780 700780E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF80 0FFB8003E380000380000380000380000380000380000380000380003FF8003FF8003FF8 151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C0000380000380 000380000380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318 >I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE0000 1F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<018000 0380000380000380000380007FFFC0FFFFC0FFFFC0038000038000038000038000038000 0380000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E00 13197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318 >I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E0003 8E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8F F0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E 000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E 03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000 DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F80003F 00001E0000151E7F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC000 01E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 007800007FE0001FE00007E013247E9F18>123 D<7C0000FF0000FFC00003C00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0 003FE0003FE0007FC000F00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00003C000FFC000FF00007C000013247E9F18>125 D E /Fn 43 122 df<00000F80003F8F80007F8F8000FF8F8001FF8F8003E0000003C000 0007C0000007C0000007C0000007C0000007C0000007C0000007C0000007C0000007C000 00FFFF8F80FFFF8F80FFFF8F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F 8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F 8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F 80192B7FAA20>12 D45 DI<00 7F000001FFC00007FFF0000FFFF8000FC1F8001F007C003F007E003E003E003C001E007C 001F007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8 000F8078000F007C001F007C001F007C001F003E003E003E003E003F007E001F80FC000F C1F8000FFFF80007FFF00001FFC000007F000019297EA71E>48 D<00180000380000F800 07F800FFF800FFF800F8F80000F80000F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F800 00F80000F80000F80000F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF0 7FFFF014287BA71E>I<00FE0003FFC007FFE00FFFF01F03F83C00FC38007E78003E7000 3EF0001FF0001F60001F20001F00001F00001F00001F00003E00003E00007C00007C0000 F80001F00001E00003C0000780000F00001E00003C0000780000F00001E00003C0000780 000F00001E00003C00007FFFFF7FFFFF7FFFFF7FFFFF18287EA71E>I<007F000001FFC0 0007FFF0000FFFF8001FC1F8003E007C003C003E0078003E0038003E0010003E0000003E 0000003E0000003C0000007C000000FC000001F8000007F00000FFE00000FFC00000FFE0 0000FFF0000001FC0000007C0000003E0000001F0000001F0000000F8000000F8000000F 8000000F8000000F8040000F8060001F00F0001F00F8003F007E007E003F81FC001FFFF8 000FFFF00003FFE000007F000019297EA71E>I<0003F0000007F0000005F000000DF000 000DF000001DF0000039F0000039F0000079F0000079F00000F1F00000F1F00001E1F000 03E1F00003E1F00007C1F00007C1F0000F81F0000F81F0001F01F0001F01F0003E01F000 7C01F0007C01F000F801F000FFFFFF80FFFFFF80FFFFFF80FFFFFF800001F0000001F000 0001F0000001F0000001F0000001F0000001F0000001F0000001F0000001F00019277EA6 1E>I<3FFFFC3FFFFC3FFFFC3FFFFC3E00003E00003E00003E00003E00003E00003E0000 3E00003E00003E00003E3F003EFFC03FFFE03FFFF03FE1F83F807C3F003E3E003E00003E 00001F00001F00001F00001F00001F00001F00001F20001F60003E70003EF8007C7C00FC 3F03F81FFFF00FFFE007FF8000FE0018287EA61E>I<000FF000003FFC0000FFFC0001FF FC0003F80C0007E000000FC000000F8000001F0000001E0000003E0000003C0000007C00 00007C0000007C3FE000F8FFF000F9FFF800FBFFFC00FF807E00FF003E00FE003F00FC00 1F00FC001F00FC000F80F8000F80F8000F80F8000F80F8000F8078000F807C000F807C00 0F807C000F003E001F003E001F001F003E001F807C000FC1FC0007FFF80003FFF00001FF C000007F000019297EA71E>II66 D<0001FF00000FFFE0003FFFF8007FFFF800FE01F801F8003003F0001007C000000F8000 001F8000001F0000003E0000003E0000007E0000007C0000007C0000007C000000F80000 00F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F80000 007C0000007C0000007C0000007E0000003E0000003E0000001F0000001F8000000F8000 0007C0000003F0000401F8001C00FE00FC007FFFFC003FFFF8000FFFE00001FF001E2C7C AA26>III< F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8 F8F8F8F8F8F8052A7AA911>73 D75 D77 DI80 D<007FC00001FFF80007FFFE000FFFFF001FC07F003F 000F007E0006007C0000007C000000F8000000F8000000F8000000F8000000F8000000FC 0000007E0000007F0000003F8000001FF800000FFF800007FFE00003FFF80000FFFC0000 0FFE000000FF0000003F0000001F8000000F8000000FC0000007C0000007C0000007C000 0007C0000007C0000007C000000F8060000F80F0001F00FC003F00FF80FE007FFFFC001F FFF80007FFE00000FF80001A2C7DAA21>83 DIII<01FE000FFF803FFFC03FFFE03C03F030 01F00001F80000F80000F80000F80000F80000F8007FF807FFF81FFFF83FE0F87F00F8FC 00F8F800F8F800F8F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1D>97 DI<007FC001FFF007FFFC0F FFFC1FC07C1F00083E00007C00007C00007C0000F80000F80000F80000F80000F80000F8 0000F800007C00007C00007E00003E00001F000C1FC07C0FFFFC07FFFC01FFF0007F8016 1B7E9A1B>I<00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E00FC3E03FF3E07FFFE0FFFFE1FC1FE3F007E3E003E 7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E 7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03FF3E00FC3E172A7EA91F>I<007E0003FF 8007FFC00FFFE01F83F03F00F03E00787C00787C003878003CFFFFFCFFFFFCFFFFFCFFFF FCF80000F80000F800007800007C00007C00003E00003F000C1FC07C0FFFFC07FFFC01FF F0007F80161B7E9A1B>I<001FC0007FC000FFC001FFC003F00003E00007C00007C00007 C00007C00007C00007C00007C00007C00007C000FFFE00FFFE00FFFE0007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C000122A7FA912>I< F8F8F8F8F800000000000000000000F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8 F8F8F8F8F8F8052A7CA90E>105 D108 DII<007F000001FFC00007FFF0000FFFF8001FC1FC003F 007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001FC1FC000F FFF80007FFF00001FFC000007F0000191B7E9A1E>II114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8 003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFF C07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007 C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003 FFE003FFE001FF8000FC0013227FA116>IIII121 D E /Fo 15 122 df<07E00FF03FFC3FFC7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE3FFC3FFC0FF007 E01010788F21>46 D<000001E00000000003F0000000000FF0000000003FF000000000FF F00000000FFFF0000003FFFFF00000FFFFFFF00000FFFFFFF00000FFFFFFF00000FFF0FF F00000FC00FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000003FFFFFFFFFC03FFFFF FFFFC03FFFFFFFFFC03FFFFFFFFFC03FFFFFFFFFC02A4478C33B>49 D<0000FFE00000000FFFFE0000003FFFFFC00000FFFFFFF00003FFFFFFFC0007FC01FFFE 000FE0007FFF001FC0001FFF803F80000FFFC03FE00007FFE07FF80007FFE07FFC0003FF F0FFFC0003FFF0FFFE0001FFF8FFFE0001FFF8FFFE0001FFF8FFFE0000FFFCFFFE0000FF FC7FFC0000FFFC7FFC0000FFFC3FF80000FFFC1FF00000FFFC0FE00000FFFC00000001FF FC00000001FFF800000001FFF800000001FFF000000003FFF000000003FFE000000003FF E000000007FFC000000007FF800000000FFF800000001FFF000000001FFE000000003FFC 000000007FF8000000007FE000000000FFC000000001FF8000000003FF0000000007FC00 0000000FF8000000001FF0000000003FC0000000007F80007C0000FF00007C0000FE0000 7C0001FC00007C0003F00000F80007E00000F8000FC00000F8001F800000F8003F000001 F8007C000003F800FFFFFFFFF801FFFFFFFFF003FFFFFFFFF007FFFFFFFFF00FFFFFFFFF F01FFFFFFFFFF03FFFFFFFFFF07FFFFFFFFFF07FFFFFFFFFF0FFFFFFFFFFE0FFFFFFFFFF E0FFFFFFFFFFE0FFFFFFFFFFE02E447AC33B>I77 D<0007FFFC000000007FFFFFC0000001FFFFFFF80000 03FFFFFFFE000007FE001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000FFF0000F FF80007FF0000FFF80007FF8000FFF80007FF80007FF00003FFC0007FF00003FFC0003FE 00003FFC0000F800003FFC00000000003FFC00000000003FFC00000000003FFC00000000 003FFC00000007FFFFFC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE00 3FFC0003FFF0003FFC0007FFC0003FFC000FFF00003FFC001FFE00003FFC003FFC00003F FC007FF800003FFC007FF800003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC 00FFF000003FFC00FFF000003FFC00FFF000007FFC007FF80000FFFC007FF80001EFFC00 3FFC0003EFFC003FFF0007CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC00 7FFFF001FFFC0003FF80007FF8362E7DAD3A>97 D<00001FFFC0000000FFFFF8000007FF FFFE00001FFFFFFF80007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FFE007FF00 03FFE00FFE0003FFE00FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF803FFC00 003E007FF8000000007FF8000000007FF800000000FFF800000000FFF800000000FFF800 000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800 000000FFF8000000007FF8000000007FF8000000007FFC000000003FFC000000003FFC00 0000001FFC000000F81FFE000000F80FFE000000F80FFF000001F007FF800003F003FFC0 0007E001FFE0000FC000FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8000000FF FFE00000001FFE00002D2E7CAD35>99 D<00001FFE00000001FFFFE0000007FFFFF80000 1FFFFFFE00007FFC07FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00003FF00F FE00001FF01FFE00000FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC000007FC7F F8000007FC7FF8000007FE7FF8000007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFF FFFFFFFFFEFFFFFFFFFFFEFFFFFFFFFFFCFFF800000000FFF800000000FFF800000000FF F8000000007FF8000000007FF8000000007FFC000000003FFC000000003FFC000000003F FC0000001C1FFE0000003E0FFE0000003E07FF0000007E07FF000000FC03FF800001F801 FFC00003F0007FF0001FE0003FFE00FFC0001FFFFFFF800007FFFFFE000000FFFFF80000 000FFF80002F2E7DAD36>101 D<000000FFC000000007FFF80000003FFFFC000000FFFF FF000001FFC1FF000007FF03FF80000FFC03FF80000FF807FFC0001FF807FFC0003FF007 FFC0003FF007FFC0003FE003FF80007FE003FF80007FE001FF00007FE000FE00007FE000 3800007FE0000000007FE0000000007FE0000000007FE0000000007FE0000000007FE000 0000007FE0000000007FE0000000007FE0000000007FE0000000FFFFFFFE0000FFFFFFFE 0000FFFFFFFE0000FFFFFFFE0000FFFFFFFE0000007FF0000000007FF0000000007FF000 0000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF000 0000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF000 0000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF000 0000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF000 0000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF000 0000007FF0000000007FF0000000007FF00000003FFFFFF800003FFFFFF800003FFFFFF8 00003FFFFFF800003FFFFFF800002A487DC724>I<00FC0001FE0003FF0007FF800FFFC0 1FFFE01FFFE01FFFE01FFFE01FFFE01FFFE00FFFC007FF8003FF0001FE0000FC00000000 000000000000000000000000000000000000000000000000000000000000007FC0FFFFC0 FFFFC0FFFFC0FFFFC0FFFFC003FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0 01FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0 01FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0 01FFC001FFC001FFC0FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF18497CC820>105 D<007FC000FFFFC000FFFFC000FFFFC000FFFFC000FFFFC00003FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF 8019487CC720>108 D<007FC001FFC00000FFFFC00FFFF80000FFFFC03FFFFE0000FFFF C0FFFFFF0000FFFFC1FC07FF8000FFFFC3E003FFC00003FFC7C001FFC00001FFCF0001FF E00001FFDE0000FFE00001FFDC0000FFE00001FFFC0000FFF00001FFF80000FFF00001FF F00000FFF00001FFF00000FFF00001FFF00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFF FFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B2E7CAD42>110 D<00000FFF0000000000 FFFFF000000007FFFFFE0000001FFFFFFF8000003FFC03FFC00000FFE0007FF00001FF80 001FF80003FF00000FFC0007FE000007FE000FFE000007FF000FFC000003FF001FFC0000 03FF803FFC000003FFC03FF8000001FFC03FF8000001FFC07FF8000001FFE07FF8000001 FFE07FF8000001FFE0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FF F0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0 FFF8000001FFF07FF8000001FFE07FF8000001FFE07FF8000001FFE07FF8000001FFE03F FC000003FFC03FFC000003FFC01FFC000003FF801FFE000007FF800FFE000007FF0007FF 00000FFE0003FF80001FFC0001FFC0003FF80000FFE0007FF000007FFC03FFE000001FFF FFFF80000007FFFFFE00000000FFFFF0000000000FFF000000342E7DAD3B>I<00FF803F 8000FFFF80FFF000FFFF83FFFC00FFFF87FFFE00FFFF8FC3FF00FFFF8F07FF0003FF9E0F FF8001FFBC0FFF8001FFB80FFF8001FFF80FFF8001FFF00FFF8001FFF007FF0001FFF007 FF0001FFE003FE0001FFE000F80001FFE000000001FFE000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC0000000FFFFFFE00000FFFFFFE0 0000FFFFFFE00000FFFFFFE00000FFFFFFE00000292E7CAD31>114 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FFE01FC0001FE03F80000FE0 3F000007E07F000003E07F000003E0FF000003E0FF000003E0FF800003E0FFC0000000FF F0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFFFFFC001FFFFFFF000FFF FFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FFFFF800001FFFF8000000 FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003FCFC000003FCFE000003 F8FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0FFFC00FFC0FFFFFFFF80 FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>I<7FFFFFC000FFFF807FFFFFC000FF FF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF8000FFF000000FE00000FF F800000FC00000FFF800000FC000007FFC00000F8000007FFC00001F8000003FFC00001F 0000003FFE00003F0000001FFE00003E0000001FFF00007E0000000FFF00007C0000000F FF8000FC00000007FF8000F800000007FFC001F800000003FFC001F000000003FFE003F0 00000003FFE003F000000001FFF003E000000001FFF007E000000000FFF007C000000000 FFF80FC0000000007FF80F80000000007FFC1F80000000003FFC1F00000000003FFE3F00 000000001FFE3E00000000001FFF7E00000000000FFF7C00000000000FFFFC0000000000 0FFFFC000000000007FFF8000000000007FFF8000000000003FFF0000000000003FFF000 0000000001FFE0000000000001FFE0000000000000FFC0000000000000FFC00000000000 007F800000000000007F800000000000003F000000000000003F000000000000003F0000 00000000003E000000000000007E000000000000007C00000000000000FC000000001F80 00F8000000003FC001F8000000007FE001F000000000FFF003F000000000FFF003E00000 0000FFF007E000000000FFF00FC000000000FFF01F8000000000FFF03F80000000007FE0 7F00000000007F43FE00000000003FFFF800000000001FFFF0000000000007FFC0000000 000001FE00000000000039427EAD3F>121 D E /Fp 8 117 df<0003FF8000001FFFF000 007FFFFE0000FE03FF0001F000FF8003C000FFC00780007FE00FF0007FF00FF8007FF01F FC007FF81FFE007FF81FFE007FF81FFE007FF81FFE007FF81FFE007FF80FFC007FF007F8 007FF003F0007FF0000000FFE0000000FFC0000001FF80000001FF00000003FE00000007 FC0000001FF000000FFFC000000FFF8000000FFFF800000003FE00000000FF800000007F E00000003FF00000003FF80000003FFC0000001FFC0000001FFE0000001FFE0200001FFF 1FC0001FFF3FE0001FFF7FF0001FFF7FF0001FFFFFF8001FFFFFF8001FFFFFF8001FFEFF F8001FFEFFF0001FFE7FF0003FFC7FE0003FFC3FC0003FF81F80007FF01FE000FFE007FC 03FFC003FFFFFF0001FFFFFE00003FFFF0000007FF800028397CB731>51 D<0000001FFF000030000001FFFFE000F000000FFFFFFC01F000007FFFFFFE03F00001FF FE007F87F00003FFE0000FCFF0000FFF000003FFF0001FFC000001FFF0003FF80000007F F0007FF00000003FF000FFC00000003FF001FFC00000001FF003FF800000000FF007FF00 0000000FF00FFF0000000007F00FFE0000000007F01FFE0000000003F01FFE0000000003 F03FFC0000000003F03FFC0000000001F03FFC0000000001F07FFC0000000001F07FF800 00000001F07FF80000000000007FF8000000000000FFF8000000000000FFF80000000000 00FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800 0000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF80000000000 007FF80000000000007FF80000000000007FF80000000000007FFC0000000000F03FFC00 00000000F03FFC0000000000F03FFC0000000000F01FFE0000000000F01FFE0000000001 E00FFE0000000001E00FFF0000000001E007FF0000000003C003FF8000000003C001FFC0 000000078000FFE00000000F00007FF00000001F00003FF80000003E00001FFC0000007C 00000FFF000001F8000003FFE00007F0000001FFFE003FC00000007FFFFFFF000000000F FFFFFC0000000001FFFFF000000000001FFF0000003C3D7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF00001FFC00FF80 001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF00007F0003FF0 0001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000000FFFFFF0 00007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF0003FE0003FF0 007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000FFC0003FF0 00FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF007FFFE0FFF F001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFFFE00 00FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03FF00007F83F F00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFFFFFFFCFFFF FFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE00000007FE000 00007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF800007807FC0000 F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE00026267DA52D >101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF0000000007FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF007FC00003FF 01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF6000FF8003FF E000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC3FFFFFFFFF FC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00FFFFFE03FF 00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF803FF00001F FC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF03FF000007 FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007 FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC03FF00000F FC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC003FFF001FF 8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000FFFFFC0000 00FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fq 75 124 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<7038F87CFC7EFC7E74 3A0402040204020804080410081008201040200F0E7E9F17>34 D<007800000084000001 840000030200000702000007020000070200000702000007040000070400000708000007 0800000310000003A00FFC03C003E0038001C001C0008001C0010003E0010004E0020008 F00200187004003078080070380800701C1000F01E1000F00E2000F0074000F003C00870 03C0087801C010380670301C18386007E00F801E227EA023>38 D<70F8FCFC7404040408 0810102040060E7C9F0D>I<0020004000800100020006000C000C001800180030003000 30007000600060006000E000E000E000E000E000E000E000E000E000E000E000E0006000 600060007000300030003000180018000C000C000600020001000080004000200B2E7DA1 12>I<800040002000100008000C00060006000300030001800180018001C000C000C000 C000E000E000E000E000E000E000E000E000E000E000E000E000C000C000C001C0018001 80018003000300060006000C00080010002000400080000B2E7DA112>I<70F8FCFC7404 0404080810102040060E7C840D>44 DI<70F8F8F87005057C84 0D>I<000100030003000600060006000C000C000C001800180018003000300030006000 60006000C000C000C00180018001800300030003000600060006000C000C000C00180018 001800300030003000600060006000C000C000C000102D7DA117>I<03F0000E1C001C0E 00180600380700700380700380700380700380F003C0F003C0F003C0F003C0F003C0F003 C0F003C0F003C0F003C0F003C0F003C0F003C0F003C07003807003807003807807803807 001806001C0E000E1C0003F000121F7E9D17>I<018003800F80F3800380038003800380 038003800380038003800380038003800380038003800380038003800380038003800380 0380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0 F803C0F803C0F803C02007C00007C0000780000780000F00000E00001C00003800007000 00600000C0000180000300000600400C00401800401000803FFF807FFF80FFFF80121E7E 9D17>I<03F0000C1C00100E00200F00780F80780780780780380F80000F80000F00000F 00000E00001C0000380003F000003C00000E00000F000007800007800007C02007C0F807 C0F807C0F807C0F00780400780400F00200E001C3C0003F000121F7E9D17>I<00060000 0600000E00000E00001E00002E00002E00004E00008E00008E00010E00020E00020E0004 0E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E0000 0E00000E00000E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE000 10000010000010000010000010000010000011F000161C00180E00100700100780000380 0003800003C00003C00003C07003C0F003C0F003C0E00380400380400700200600100E00 0C380003E000121F7E9D17>I<007C000182000701000E03800C07801C07803803003800 00780000700000700000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003 C0F003C0F003C07003C07003C07003803803803807001807000C0E00061C0001F000121F 7E9D17>I<4000007FFFC07FFF807FFF8040010080020080020080040000080000080000 100000200000200000400000400000C00000C00001C00001800003800003800003800003 8000078000078000078000078000078000078000078000030000121F7D9D17>I<03F000 0C0C001006003003002001806001806001806001807001807803003E03003F06001FC800 0FF00003F80007FC000C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0 C000806001802001001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C003806 00380700700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007 C03807C0180BC00E13C003E3C0000380000380000380000700300700780600780E00700C 002018001070000FC000121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F8 7005147C930D>I<70F8F8F8700000000000000000000070F0F8F8780808081010102020 40051D7C930D>I<7FFFFFE0FFFFFFF00000000000000000000000000000000000000000 000000000000000000000000FFFFFFF07FFFFFE01C0C7D9023>61 D<000100000003800000038000000380000007C0000007C0000007C0000009E0000009E0 000009E0000010F0000010F0000010F00000207800002078000020780000403C0000403C 0000403C0000801E0000801E0000FFFE0001000F0001000F0001000F0002000780020007 8002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0 038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C00004078000040 F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 780000007C0000407C0000403C0000401C0000401E0000800E0000800700010003800200 01C0040000703800000FC0001A217D9F21>IIII72 D I<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00203C 00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>I76 DII<001F800000F0 F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E07800 01E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F800 01F0F80001F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F00 0F0007801E0001C0380000F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F000 00F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001 C08001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19 >I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F 0010800F0010000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000001F800007FFFE001C1F7E9E21>II87 D<7FF83FF80FE00FC007C0070003C0020001E0040001F00C0000F0080000781000007C10 00003C2000003E4000001E4000000F8000000F8000000780000003C0000007E0000005E0 000009F0000018F8000010780000207C0000603C0000401E0000801F0001800F00010007 80020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I91 D<080410082010201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B 9F17>II<1FE000303000781800781C00300E0000 0E00000E00000E0000FE00078E001E0E00380E00780E00F00E10F00E10F00E10F01E1078 1E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00 700E00780E00780E00780E00780E00780E00780E00700E00700E00E00F00E00D01C00CC3 00083E0015207F9F19>I<03F80E0C1C1E381E380C70007000F000F000F000F000F000F0 0070007000380138011C020E0C03F010147E9314>I<000380003F800003800003800003 8000038000038000038000038000038000038000038003E380061B801C07803803803803 80700380700380F00380F00380F00380F00380F00380F003807003807003803803803807 801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E0038070038070070070070 0380F00380F00380FFFF80F00000F00000F000007000007000003800801800800C010007 060001F80011147F9314>I<007C00C6018F038F07060700070007000700070007000700 FFF007000700070007000700070007000700070007000700070007000700070007000700 07007FF01020809F0E>I<0000E003E3300E3C301C1C30380E00780F00780F00780F0078 0F00780F00380E001C1C001E380033E0002000002000003000003000003FFE001FFF800F FFC03001E0600070C00030C00030C00030C000306000603000C01C038003FC00141F7F94 17>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E3E000E43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C00 1E003E001E001C000000000000000000000000000E007E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E00FFC00A1F809E0C>I<00E001F001 F001F000E0000000000000000000000000007007F000F000700070007000700070007000 70007000700070007000700070007000700070007000700070007000706070F060F0C061 803F000C28829E0E>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E0FF00E03C00E03000E02000E04000E08000E10000E30000E70 000EF8000F38000E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF81520 7F9F18>I<0E00FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B2080 9F0C>I<0E1F01F000FE618618000E81C81C000F00F00E000F00F00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F93 26>I<0E3E00FE43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F8 00070E001C03803801C03801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000 F07000E07000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F 01C00F00E00E00E00E00F00E00700E00780E00780E00780E00780E00780E00780E00700E 00F00E00E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E 00000E0000FFE000151D7F9319>I<03E0800619801C05803C0780380380780380700380 F00380F00380F00380F00380F00380F003807003807803803803803807801C0B800E1380 03E380000380000380000380000380000380000380000380000380003FF8151D7E9318> I<0E78FE8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E000E000E 000E00FFE00F147F9312>I<1F9030704030C010C010C010E00078007F803FE00FF00070 803880188018C018C018E030D0608F800D147E9312>I<020002000200060006000E000E 003E00FFF80E000E000E000E000E000E000E000E000E000E000E000E080E080E080E080E 080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001 F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C2 0000E40000EC00007800003800003C00007C00004E000087000107000303800201C00601 E01E01E0FF07FE1714809318>II< 3FFF380E200E201C40384078407000E001E001C00380078007010E011E011C0338027006 700EFFFE10147F9314>II E /Fr 46 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70 F8F8F87005057C840E>I<01F000071C000C06001803003803803803807001C07001C070 01C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E07001C07001C07001C07803C03803803803801C07000C060007 1C0001F00013227EA018>48 D<008003800F80F380038003800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038007C0FFFE0F217CA018>I<03F0000C1C001007002007804003C04003C08003E0F0 03E0F801E0F801E0F801E02003E00003E00003C00003C0000780000700000E00001C0000 180000300000600000C0000180000100000200200400200800201800603000403FFFC07F FFC0FFFFC013217EA018>I<03F8000C1E001007002007804007C07807C07803C07807C0 3807C0000780000780000700000F00000E0000380003F000001C00000F00000780000780 0003C00003C00003E02003E07003E0F803E0F803E0F003C04003C0400780200780100F00 0C1C0003F00013227EA018>I<000200000600000E00000E00001E00001E00002E00004E 00004E00008E00008E00010E00020E00020E00040E00040E00080E00100E00100E00200E 00200E00400E00800E00FFFFF8000E00000E00000E00000E00000E00000E00000E00001F 0001FFF015217FA018>I<1000801E07001FFF001FFE001FF80013E00010000010000010 000010000010000010000010F800130E001407001803801003800001C00001C00001E000 01E00001E00001E07001E0F001E0F001E0E001C08001C04003C04003802007001006000C 1C0003F00013227EA018>I<007E0001C1000300800601C00E03C01C03C0180180380000 380000780000700000700000F0F800F30C00F40600F40300F80380F801C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07001E07001E03801C03801C01803801C03000C0600 070C0001F00013227EA018>I<01F800060E000803001001802001802000C06000C06000 C06000C07000C07801803E01003F02001FC4000FF80003F80003FC00067F00083F80100F 803007C06001C06000E0C000E0C00060C00060C00060C000606000406000C03000801803 000E0E0003F00013227EA018>56 D<01F000060C000C0600180700380380700380700380 F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003E03803E01805E0 0C05E00619E003E1E00001C00001C00001C0000380000380300300780700780600700C00 2018001030000FC00013227EA018>I<0001800000018000000180000003C0000003C000 0003C0000005E0000005E000000DF0000008F0000008F0000010F8000010780000107800 00203C0000203C0000203C0000401E0000401E0000401E0000800F0000800F0000FFFF00 0100078001000780030007C0020003C0020003C0040003E0040001E0040001E00C0000F0 0C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C00170 038000F0070000F00E0000701E0000701C0000303C0000303C0000307C00001078000010 78000010F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 78000000780000107C0000103C0000103C0000101C0000201E0000200E00004007000040 0380008001C0010000E0020000381C000007E0001C247DA223>67 D69 DI< FFFC0FC00780078007800780078007800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800FC0FFFC0E227EA1 12>73 D 77 DI 80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020E00020 E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E00001E0 0000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0C80180 C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008400780 08C007800C80078004800780048007800480078004000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000FC0 0003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700000700 00FF0007C7001E07003C0700780700700700F00708F00708F00708F00F087817083C2390 0FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E003C0E 001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C 80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000700000 F00000F00000F00000F00000F00000F00000F000007000007800403800401C00800C0100 07060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800E07800 E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800E01801 E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C07801E070 00E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C00400E 008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700070007 00070007000700FFF8070007000700070007000700070007000700070007000700070007 00070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C07001C 07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010000018 00001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860003060 00303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E00F0070 0F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000000000 0000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E001C000 00000000000000000000000000000001E00FE001E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0F1 8061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E02000E0400 0E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C00E01E0 0E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00FE60E183801E807201C0 0F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60C01E80E00F00 700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E0038 0E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E0038 0F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E0000 0E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E 000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>III121 D E /Fs 20 118 df45 D68 D73 D77 D 80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E070 0000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF0000 0FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880 00007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0F0 0000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Ft 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 908 a Ft(D)26 b(R)g(A)f(F)h(T)225 999 y Fs(Do)r(cumen)n(t) 20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1194 y Fr(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)802 1320 y(August)i(12,)f(1996)77 1378 y(This)h(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1436 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1494 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1552 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 45 a Fq(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fp(Chapter)34 b(3)75 564 y Fo(Miscellan)m(y)41 b(for)g(1.2)75 805 y Fn(3.1)59 b(V)n(ersion)20 b(Numb)r(er)75 906 y Fq(In)c(order)f(to)g(cop)q(e)h(with)g(c)o(hanges)f(to)g(the)g (MPI)h(Standard,)f(there)g(are)g(b)q(oth)h(compile-time)h(and)f(run-)75 963 y(time)h(w)o(a)o(ys)f(to)h(determine)h(whic)o(h)g(v)o(ersion)f(of)g (the)g(standard)g(is)g(in)h(use)g(in)g(the)f(en)o(vironmen)o(t)g(one)g (is)75 1019 y(using.)166 1076 y(The)d(\\v)o(ersion")g(will)h(b)q(e)g (represen)o(ted)f(b)o(y)g(t)o(w)o(o)e(separate)i(in)o(tegers,)g(for)f (the)h(v)o(ersion)g(and)g(sub)o(v)o(er-)75 1132 y(sion:)166 1188 y(In)i(C,)170 1282 y Fm(#define)23 b(MPI_VERSION)94 b(1)170 1339 y(#define)23 b(MPI_SUBVERSION)f(2)75 1433 y Fq(and)15 b(in)h(F)l(ortran,)170 1526 y Fm(INTEGER)23 b(MPI_VERSION,)g(MPI_SUBVERSION)170 1583 y(PARAMETER)g(\(MPI_VERSION)94 b(=)24 b(1\))170 1639 y(PARAMETER)f(\(MPI_SUBVERSION)f(=)i(2\))75 1733 y Fq(F)l(or)15 b(run)o(time)g(determination,)75 1884 y Fl(MPI)p 160 1884 14 2 v 16 w(VERSION\()h(version,)f(subversion) i(\))117 1961 y Fk(OUT)108 b Fl(version)456 b Fk(v)o(ersion)14 b(n)o(um)o(b)q(er)117 2036 y(OUT)108 b Fl(subversion)393 b Fk(sub)o(v)o(ersion)15 b(n)o(um)o(b)q(er)75 2161 y Fm(int)23 b(MPI)p 245 2161 15 2 v 17 w(Version\()g(int)g(*version,)g (int)h(*subversion)e(\))75 2247 y(MPI)p 150 2247 V 17 w(VERSION\()h(VERSION,)f(SUBVERSION,)h(IERROR)g(\))170 2304 y(INTEGER)g(VERSION,)g(SUBVERSION,)f(IERROR)1875 2340 y Fj(>)16 b Fi(\(Sept\))75 2390 y Fm(int)23 b(MPI::Version\()g (int*)g(version,)g(int*)g(subversion)f(\))166 2517 y Fh(Discussion:)34 b Fk(Should)13 b(this)h(b)q(e)h(instead)75 2567 y Fg(int)21 b(MPI::Version\()e(int&)i(version,)f(int&)h (subversion)f(\))1875 2650 y Fj(?)c Fi(\(Sept\))166 2701 y Fq(This)11 b(function)g(is)f(one)h(of)e(the)i(few)f(that)f(can)h(b)q (e)h(called)h(b)q(efore)e Fl(MPI)p 1322 2701 14 2 v 16 w(INIT)g Fq(or)f(after)h Fl(MPI)p 1672 2701 V 15 w(FINALIZE)p Fq(.)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fq(2)970 b Ff(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)75 45 y Fn(3.2)59 b(T)-5 b(reatment)18 b(of)i(MPI)p 670 45 18 2 v 21 w(Status)75 147 y Fq(The)25 b(follo)o(wing)g(prop)q(osals)g(add)g(to,)g(but)g(do)g(not)f(c)o (hange,)i(the)f(functionalit)o(y)h(asso)q(ciated)f(with)75 203 y Fl(MPI)p 160 203 14 2 v 16 w(ST)l(A)l(TUS)p Fq(.)-468 b Fj(>)15 b Fi(\(Sept\))75 325 y Fe(3.2.1)49 b(P)o(assing)17 b(MPI)p 481 325 15 2 v 18 w(ST)l(A)l(TUS)p 676 325 V 18 w(IGNORE)g(fo)o(r)f(MPI)p 1044 325 V 18 w(Status)75 411 y Fq(Ev)o(ery)h(call)i(to)e Fl(MPI)p 437 411 14 2 v 16 w(RECV)h Fq(includes)i(a)e Fl(status)h Fq(argumen)o(t,)f(where)f (the)h(system)g(can)f(return)h(details)75 467 y(ab)q(out)d(the)h (message)f(receiv)o(ed.)21 b(An)16 b(ob)s(ject)f(of)g(t)o(yp)q(e)g Fl(MPI)p 1103 467 V 16 w(ST)l(A)l(TUS)h Fq(is)g(not)f(an)h Fl(MPI)e Fq(opaque)i(ob)s(ject;)75 523 y(its)f(structure)f(is)g (declared)i(in)f Fm(mpi.h)f Fq(and)g Fm(mpif.h)p Fq(,)g(and)g(it)h (exists)f(in)i(the)e(users')g(program.)k(In)d(man)o(y)75 580 y(cases)i(application)i(programs)e(are)g(constructed)g(so)g(that)g (it)g(is)h(unnecessary)g(for)f(them)g(to)g(examine)75 636 y(the)f Fm(status)g Fq(\014elds.)25 b(In)17 b(these)f(cases)h(it)f (is)h(a)f(w)o(aste)g(for)g(the)g(user)h(to)e(allo)q(cate)i(a)f(status)g (ob)s(ject,)g(and)75 693 y(it)f(is)h(particularly)h(w)o(asteful)e(for)f (the)i Fl(MPI)e Fq(implemen)o(tation)j(to)d(\014ll)j(in)f(\014elds)g (in)g(this)g(ob)s(ject.)166 749 y(T)l(o)21 b(cop)q(e)h(with)g(this)g (problem,)i(there)d(is)h(a)g(pre-de\014ned)h(constan)o(t,)f Fl(MPI)p 1502 749 V 16 w(ST)l(A)l(TUS)p 1683 749 V 17 w(IGNORE)p Fq(,)75 806 y(whic)o(h,)i(when)e(passed)g(to)f(a)h(receiv)o (e)h(or)e(test)g(function,)j(informs)e(the)g(implemen)o(tation)h(that)e (the)75 862 y(status)13 b(\014elds)h(are)f(not)g(to)g(b)q(e)h(\014lled) i(in.)k(Note)13 b(that)g Fl(MPI)p 1059 862 V 15 w(ST)l(A)l(TUS)p 1239 862 V 18 w(IGNORE)i Fq(is)e(not)h(a)f(sp)q(ecial)i(t)o(yp)q(e)e (of)75 919 y Fl(MPI)p 160 919 V 16 w(ST)l(A)l(TUS)i Fq(ob)s(ject;)f (rather,)f(it)h(is)h(a)f(sp)q(ecial)i(v)m(alue)f(for)f(the)g(argumen)o (t.)k(That)c(is,)h(in)f(C)g(one)h(w)o(ould)75 975 y(exp)q(ect)h(it)f (to)g(b)q(e)h(NULL,)g(not)e(the)i(address)f(of)g(a)g(sp)q(ecial)i Fl(MPI)p 1179 975 V 15 w(ST)l(A)l(TUS)p Fq(.)-1486 b Fj(?)15 b Fi(\(Sept\))166 1032 y Fq(In)d(general,)h(this)f (optimization)g(can)g(apply)g(to)f(all)i(functions)f(for)f(whic)o(h)h Fl(MPI)p 1514 1032 V 16 w(Status)i Fq(or)d(an)g(arra)o(y)75 1088 y(of)k Fl(MPI)p 212 1088 V 16 w(Status)p Fq('s)i(is)f(an)g (argumen)o(t.)k(These)c(are)f(all)i(the)e(v)m(arious)h(forms)f(of)g Fl(MPI)p 1493 1088 V 16 w(RECV)p Fq(,)h Fl(MPI)p 1735 1088 V 16 w(TEST)p Fq(,)75 1144 y(and)i Fl(MPI)p 251 1144 V 15 w(W)l(AIT)p Fq(.)f(When)h(an)f(arra)o(y)g(is)g(passed,)h(as)f (in)h(the)p 1132 1144 V 34 w Fl(ANY)g Fq(and)p 1349 1144 V 34 w Fl(ALL)f Fq(functions,)h(a)f(separate)75 1201 y(constan)o(t,)d Fl(MPI)p 356 1201 V 16 w(ST)l(A)l(TUSES)p 589 1201 V 18 w(IGNORE)p Fq(,)i(is)f(passed)h(for)e(the)i(arra)o(y)e (argumen)o(t.)-1573 b Fj(>)15 b Fi(\(Sept\))166 1257 y Fl(MPI)p 251 1257 V 16 w(ST)l(A)l(TUS)p 432 1257 V 17 w(IGNORE)e Fq(and)e Fl(MPI)p 792 1257 V 16 w(ST)l(A)l(TUSES)p 1025 1257 V 18 w(IGNORE)h Fq(are)f(not)g(exp)q(ected)h(to)e(ha)o(v)o(e) h(the)g(same)75 1314 y(v)m(alues)16 b(in)g(C)f(and)h(F)l(ortran.)-681 b Fj(?)15 b Fi(\(Sept\))75 1436 y Fe(3.2.2)49 b(Non-destructive)16 b(T)l(est)f(of)i(MPI)p 808 1436 15 2 v 18 w(Status)75 1521 y Fq(This)e(call)h(is)f(useful)h(for)e(accessing)i(the)e (information)h(asso)q(ciated)g(with)g(a)f(request,)h(without)f (deleting)75 1578 y(the)22 b(request)f(\(in)h(case)g(the)f(user)h(is)g (exp)q(ected)h(to)d(access)i(it)g(after)f(the)g(handler\).)40 b(It)21 b(allo)o(ws)h(one)75 1634 y(to)16 b(la)o(y)o(er)g(libraries)i (more)f(con)o(v)o(enien)o(tly)l(,)h(since)f(m)o(ultiple)i(la)o(y)o(ers) d(of)g(soft)o(w)o(are)f(ma)o(y)h(access)h(the)f(same)75 1691 y(completed)c(request)f(and)h(extract)e(from)h(it)g(the)h(status)e (information.)19 b(This)11 b(will)i(b)q(e)f(also)f(b)q(e)h(imp)q(ortan) o(t)75 1747 y(for)k(language)i(in)o(terop)q(erabilit)o(y)h(if)e(w)o(e)g (decide)i(that)d(status)g(ob)s(jects)h(cannot)f(b)q(e)i(transferred)f (across)75 1804 y(language)e(b)q(oundaries,)h(since)h(with)e(this)h (function)g(the)f(status)g(ob)s(ject)f(can)i(b)q(e)f(transferred)g (instead.)75 1955 y Fl(MPI)p 160 1955 14 2 v 16 w(GET)p 264 1955 V 17 w(ST)l(A)l(TUS\()h(request,)g(\015ag,)f(status)i(\))117 2032 y Fk(IN)155 b Fl(request)452 b Fk(an)14 b Fd(MPI)p 1040 2032 13 2 v 14 w(REQUEST)f Fk(ob)r(ject)117 2107 y(OUT)108 b Fl(\015ag)518 b Fk(\015ag,)13 b(same)g(as)h(from)e Fd(MPI)p 1325 2107 V 15 w(TEST)117 2182 y Fk(OUT)108 b Fl(status)476 b Fd(MPI)p 982 2182 V 15 w(ST)m(A)m(TUS)12 b Fk(ob)r(ject)j(if)e(\015ag)g(is)h(true)75 2307 y Fm(int)23 b(MPI)p 245 2307 15 2 v 17 w(Get)p 334 2307 V 17 w(status\(MPI)p 591 2307 V 16 w(Request)g(request,)f(int)i(*flag,)f(MPI)p 1347 2307 V 17 w(Status)g(*status\))75 2393 y(MPI)p 150 2393 V 17 w(GET)p 239 2393 V 17 w(STATUS\()f(REQUEST,)h(FLAG,)g (STATUS,)g(IERROR\))170 2449 y(INTEGER)g(REQUEST,)g(FLAG,)g (STATUS\(MPI)p 962 2449 V 16 w(STATUS)p 1122 2449 V 16 w(SIZE\),)h(IERROR)75 2536 y(int)f(MPI::Request::Get)p 581 2536 V 15 w(status\(int&)g(flag,)g(MPI::Status&)f(status\))166 2622 y Fq(Sets)c(\015ag=true)g(if)h(the)f(request)h(has)f(completed,)h (and,)g(if)g(so,)f(returns)g(in)h(status)f(the)g(request)75 2679 y(status.)34 b(Ho)o(w)o(ev)o(er,)20 b(unlik)o(e)i(test)d(or)h(w)o (ait,)g(it)h(do)q(es)f(not)g(deallo)q(cate)h(or)f(inactiv)m(ate)h(the)f (request;)i(a)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Ff(3.3.)34 b(ERR)o(OR)17 b(CLASS)f(F)o(OR)f(INV)-5 b(ALID)16 b(KEYV)-5 b(AL)818 b Fq(3)75 45 y(subsequen)o(t)16 b(call)g(to)f(test,)f(w)o(ait)h(or)f(free)i(should)g(b)q(e)g(executed)g (with)f(that)g(request)75 188 y Fn(3.3)59 b(Erro)n(r)21 b(Class)f(fo)n(r)h(Invalid)e(Keyval)75 290 y Fq(There)c(is)h(a)f(new)g (MPI)h(error)e(class:)20 b Fl(MPI)p 808 290 14 2 v 16 w(ERR)p 909 290 V 17 w(KEYV)l(AL)166 393 y Fh(Discussion:)43 b Fk(A)o(t)16 b(the)h(momen)o(t)c(the)k(b)q(est)g(w)o(e)g(can)f(do)g (with)g(an)g(in)o(v)n(alid)e(k)o(eyv)n(al)h(argumen)o(t)g(is)h(return) 75 450 y Fd(MPI)p 152 450 13 2 v 14 w(ERR)p 245 450 V 15 w(ARG)p Fk(,)g(or)h(p)q(erhaps)h Fd(MPI)p 659 450 V 15 w(ERR)p 753 450 V 14 w(OTHER)p Fk(.)f(W)m(e're)g(free)h(to)f (de\014ne)h(whatev)o(er)g(extra)g(error)g(co)q(des)h(w)o(e)75 506 y(w)o(an)o(t)12 b(to)h(in)f(an)h(implemen)o(tatio)o(n,)d(but)j (this)g(one)g(seems)g(ob)o(vious;)f(it's)g(prett)o(y)i(m)o(uc)o(h)d (the)j(only)e(sp)q(eci\014c)i(kind)f(of)75 563 y(MPI)h(argumen)o(t)f (that)h(do)q(esn't)g(curren)o(tly)h(ha)o(v)o(e)f(its)g(o)o(wn)f(class.) 75 753 y Fn(3.4)59 b(MPI)p 322 753 18 2 v 22 w(UB)75 855 y Fq(The)16 b(problem)h(with)f Fl(MPI)p 536 855 14 2 v 16 w(UB)g Fq(is)h(that)e(it)h(o)o(v)o(errides)g(ev)o(erything)g (else.)23 b(F)l(or)15 b(example,)i(if)f(y)o(ou)g(put)g(an)75 911 y Fl(MPI)p 160 911 V 16 w(UB)11 b Fq(in)h(a)f(struct)f(t)o(yp)q(e)h (and)g(then)g(use)h(that)e(new)h(t)o(yp)q(e)g(in)h(another)e(t)o(yp)q (e)h(\(e.g.)18 b(a)11 b(v)o(ector)f(or)g(another)75 968 y(struct)h(or)f(ev)o(en)i(con)o(tiguous\),)f(then)h(that)e Fl(MPI)p 886 968 V 16 w(UB)h Fq(o)o(v)o(errides)h(the)f("natural")f (and)i(probably)g(in)o(tended)75 1024 y(form)17 b(of)g(the)g(datat)o (yp)q(e.)26 b(W)l(e)17 b(need)i(an)e Fl(MPI)p 873 1024 V 16 w(UB)p 950 1024 V 17 w(SOFT)h Fq(or)e(something)i(that)f(sets)g (the)g(\\stride")h(for)75 1081 y(that)d(datat)o(yp)q(e)g(but)h(not)f (for)g(the)h(datat)o(yp)q(es)f(that)g(con)o(tain)h(that)f(datat)o(yp)q (e.)21 b Fh(Discussion:)36 b Fk(W)m(e)14 b(ma)o(y)75 1184 y(need)h(to)f(reexamine)f(Llo)o(yd's)g(Lewins's)h(original)e(prop) q(osal)h(from)f(MPI-1.)75 1375 y Fn(3.5)59 b(MPI-1.0)19 b(and)h(MPI-1.1)f(Cla)n(ri\014cations)75 1478 y Fe(3.5.1)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 1478 15 2 v 18 w(FINALIZE)75 1564 y Fq(This)f(routine)f(cleans)h(up)f(all)h(MPI)f (state.)k(Eac)o(h)c(pro)q(cess)g(m)o(ust)g(call)h Fl(MPI)p 1371 1564 14 2 v 16 w(FINALIZE)e Fq(b)q(efore)h(it)g(exits.)75 1620 y(Once)e(this)g(routine)f(is)h(called,)h(no)e(MPI)h(routine)f (\(ev)o(en)g(not)g Fl(MPI)p 1195 1620 V 16 w(INIT)p Fq(\))f(ma)o(y)h(b) q(e)h(called.)20 b(Eac)o(h)11 b(pro)q(cess)75 1677 y(m)o(ust)i (complete)i(an)o(y)f(p)q(ending)i(comm)o(unication)e(it)h(initiated)g (b)q(efore)f(it)h(calls)g Fl(MPI)p 1539 1677 V 16 w(FINALIZE)p Fq(.)d(After)75 1733 y(the)j(call)g(returns,)g(eac)o(h)f(pro)q(cess)h (ma)o(y)f(con)o(tin)o(ue)h(lo)q(cal)h(computations,)e(or)g(exit,)h (without)f(in)o(terfering)75 1790 y(with)i(other)e(pro)q(cesses.)21 b Fl(MPI)p 597 1790 V 15 w(FINALIZE)15 b Fq(is)g(collectiv)o(e)i(on)f Fl(MPI)p 1209 1790 V 15 w(COMM)p 1366 1790 V 17 w(W)o(ORLD)p Fq(.)166 1893 y Fh(Discussion:)54 b Fk(Need)20 b(to)e(discuss)i (whether)g(\014nalize)f(should,)h(indeed,)g(b)q(e)f(collectiv)o(e,)h (and)e(ho)o(w)h(this)75 1950 y(extends)14 b(to)e(dynamic)e(pro)q (cesses.)20 b(F)m(or)12 b(dynamic)e(pro)q(cesses,)15 b(w)o(e)d(ma)o(y)f(w)o(an)o(t)g(\014nalize)h(to)g(b)q(e)h(collectiv)o (e)f(o)o(v)o(er)g(the)75 2006 y(connected)k(comp)q(onen)o(t)d (including)g(the)h(calling)f(pro)q(cess.)166 2157 y Fh(Discussion:)34 b Fk(Should)13 b(a)g(program)f(that)i(do)q(es)g(not)g(complete)f(its)g (outstanding)h(op)q(erations)g(b)q(e)g(consid-)75 2214 y(ered)j(erroneous,)h(or)e(is)g(it)g Fd(MPI)p 587 2214 13 2 v 15 w(FINALIZE)p Fk('s)f(resp)q(onsibilit)o(y)h(to)g(complete)g (all)f(op)q(erations?)25 b(Do)16 b(w)o(e)g(w)o(an)o(t)g(to)75 2270 y(encourage)f(programmer)c(reliance)k(on)e(this)h(functionalit)o (y?)189 2423 y Fc(A)n(dvic)n(e)d(to)i(implementors.)38 b Fq(Ev)o(en)11 b(though)g(a)g(pro)q(cess)h(has)f(completed)i(all)f (the)f(comm)o(unication)189 2480 y(it)j(initiated,)h(suc)o(h)g(comm)o (unication)f(ma)o(y)f(not)h(y)o(et)g(b)q(e)g(completed)h(from)e(the)h (viewp)q(oin)o(t)h(of)f(the)189 2536 y(underlying)j(MPI)f(system.)k (E.g.,)14 b(a)h(blo)q(c)o(king)i(send)f(ma)o(y)f(ha)o(v)o(e)g (completed,)i(ev)o(en)e(though)h(the)189 2593 y(data)k(is)h(still)h (bu\013ered)f(at)f(the)h(sender.)37 b(The)20 b(MPI)h(implemen)o(tation) h(m)o(ust)e(ensure)h(that)f(a)189 2649 y(pro)q(cess)14 b(has)g(completed)h(an)o(y)f(in)o(v)o(olv)o(emen)o(t)g(in)h(MPI)f(comm) o(unication)h(b)q(efore)f Fl(MPI)p 1669 2649 14 2 v 16 w(FINALIZE)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fq(4)970 b Ff(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fq(returns.)26 b(Th)o(us,)17 b(if)h(a)e(pro)q(cess)i(exits)g(after)e(the)h(call)i(to)d Fl(MPI)p 1271 45 14 2 v 16 w(FINALIZE)p Fq(,)g(this)i(will)h(not)e (cause)189 102 y(an)e(ongoing)g(comm)o(unication)h(to)e(fail.)21 b(\()p Fc(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Fq(\))75 223 y Fe(3.5.2)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 223 15 2 v 18 w(INTERCOMM)p 936 223 V 20 w(CREA)l(TE)75 309 y Fl(The)11 b(Problem:)44 b Fq(The)12 b(MPI)e(1.1)g(standard)h(sa)o (ys,)f(in)i(the)f(discussion)h(of)f Fl(MPI)p 1389 309 14 2 v 16 w(INTERCOMM)p 1679 309 V 16 w(CREA)l(TE)p Fq(,)75 366 y(b)q(oth)k(that)189 459 y(The)g(groups)g(m)o(ust)g(b)q(e)g (disjoin)o(t)75 553 y(and)g(that)189 647 y(The)g(leaders)h(ma)o(y)e(b)q (e)i(the)g(same)e(pro)q(cess.)75 741 y(T)l(o)d(further)h(m)o(uddy)g (the)g(w)o(aters,)e(the)i(reason)f(giv)o(en)i(for)e(\\The)g(groups)g(m) o(ust)h(b)q(e)g(disjoin)o(t")g(is)g(based)g(on)75 797 y(concerns)18 b(ab)q(out)g(the)g(implmen)o(tation)g(of)g Fl(MPI)p 922 797 V 15 w(INTERCOMM)p 1211 797 V 17 w(CREA)l(TE)h Fq(that)e(are)g(not)h(applicable)75 854 y(for)d(the)g(case)g(where)h (the)f(leaders)h(are)f(the)g(same)g(pro)q(cess.)75 974 y Fl(The)h(Fix:)45 b Fq(Delete)15 b(the)h(text:)189 1068 y(\(the)f(t)o(w)o(o)e(leaders)j(could)h(b)q(e)e(the)h(same)f(pro)q (cess\))75 1161 y(from)f(the)i(discussion)h(of)d Fl(MPI)p 610 1161 V 16 w(INTERCOMM)p 900 1161 V 17 w(CREA)l(TE)p Fq(.)166 1218 y(Replace)j(the)e(text:)189 1312 y(All)f(in)o(ter-comm)o (unicator)g(constructors)e(are)h(blo)q(c)o(king)h(and)g(require)g(that) e(the)i(lo)q(cal)g(and)189 1368 y(remote)g(groups)h(b)q(e)h(disjoin)o (t)g(in)g(order)f(to)f(a)o(v)o(oid)h(deadlo)q(c)o(k.)75 1462 y(with)189 1556 y(All)f(in)o(ter-comm)o(unicator)g(constructors)e (are)h(blo)q(c)o(king)h(and)g(require)g(that)e(the)i(lo)q(cal)g(and)189 1612 y(remote)g(groups)h(b)q(e)h(disjoin)o(t.)189 1687 y(Advice)21 b(to)e(users:)29 b(The)20 b(groups)f(m)o(ust)g(b)q(e)h (disjoin)o(t)h(for)e(sev)o(eral)h(reasons.)33 b(Primarily)l(,)189 1744 y(this)19 b(is)h(the)f(in)o(ten)o(t)g(of)g(the)g(in)o(tercomm)o (unicators)g({)g(to)g(pro)o(vide)g(a)g(comm)o(unicator)g(for)189 1800 y(comm)o(unication)h(b)q(et)o(w)o(een)g(disjoin)o(t)g(groups.)33 b(This)20 b(is)g(re\015ected)g(in)h(the)e(de\014nition)j(of)189 1857 y Fl(MPI)p 274 1857 V 15 w(INTERCOMM)p 563 1857 V 17 w(MERGE)p Fq(,)15 b(whic)o(h)h(allo)o(ws)f(the)g(user)f(to)h(con)o (trol)f(the)h(ranking)g(of)f(the)189 1913 y(pro)q(cesses)19 b(in)h(the)f(created)g(in)o(tracomm)o(unicator;)g(this)h(ranking)f(mak) o(es)f(little)i(sense)g(if)189 1970 y(the)i(groups)f(are)h(not)g (disjoin)o(t.)41 b(In)22 b(addition,)j(the)d(natural)g(extension)h(of)f (collectiv)o(e)189 2026 y(op)q(erations)13 b(to)g(in)o(tercomm)o (unicators)h(\(b)q(eing)g(considered)h(for)e(MPI-2\))g(mak)o(es)g(the)h (most)189 2083 y(sense)h(when)g(the)g(groups)g(are)f(disjoin)o(t.)21 b(It)15 b(is)g(the)g(in)o(ten)o(t)g(of)f(the)h(MPI)g(de\014nition)i (not)d(to)189 2139 y(preclude)j(future)e(extensions.)75 2261 y Fe(3.5.3)49 b(Cla)o(ri\014cation)18 b(of)e(Binding)i(of)f(MPI)p 854 2261 15 2 v 18 w(T)l(yp)q(e)p 971 2261 V 18 w(size)75 2347 y Fq(This)i(clari\014cation)g(is)g(needed)g(in)g(the)f(MPI-1)g (description)i(of)e Fl(MPI)p 1293 2347 14 2 v 15 w(T)l(yp)q(e)p 1400 2347 V 18 w(size)p Fq(,)g(since)i(the)e(issue)h(re-)75 2403 y(p)q(eatedly)d(arises.)k(It)c(is)f(a)g(clari\014cation)i(of)e (the)g(binding.)189 2509 y Fc(A)n(dvic)n(e)20 b(to)j(users.)76 b Fq(The)21 b(binding)i(of)e Fl(MPI)p 1004 2509 V 16 w(T)l(yp)q(e)p 1112 2509 V 17 w(size)p Fq(,)i(sp)q(eci\014cally)l(,)i (whether)d(the)f(output)189 2566 y(argumen)o(t)e(in)i(C)g(is)f(of)g(t)o (yp)q(e)h Fm(int)f Fq(as)g(in)h(the)f(curren)o(t)h(standard)f(or)f (should)j(b)q(e)f(c)o(hanged)g(to)189 2622 y(something)14 b(else,)i(has)e(b)q(een)i(extensiv)o(ely)f(discussed)h(b)o(y)f(the)g (MPI)f(F)l(orum.)19 b(The)c(\014nal)g(decision)189 2679 y(is)g(to)g(lea)o(v)o(e)g(it)h(an)f Fm(int)p Fq(.)k(\()p Fc(End)d(of)g(advic)n(e)g(to)h(users.)p Fq(\))1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Ff(3.6.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPES)1267 b Fq(5)75 45 y Fe(3.5.4)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 45 15 2 v 18 w(REDUCE)75 131 y Fq(The)f(curren)o(t)h(text)e(on)h (p.)20 b(115)15 b(lines)i(25-28)d(from)g(MPI-1.1)h(\(June)h(12,)e (1995\))f(sa)o(ys:)166 187 y(The)j Fl(datat)o(yp)q(e)i Fq(argumen)o(t)e(of)f Fl(MPI)p 783 187 14 2 v 16 w(REDUCE)i Fq(m)o(ust)f(b)q(e)g(compatible)i(with)e Fl(op)p Fq(.)23 b(Prede\014ned)18 b(op-)75 244 y(erators)h(w)o(ork)g(only)i(with)f(the) g Fl(MPI)g Fq(t)o(yp)q(es)g(listed)h(in)g(Sec.)35 b(4.9.2)18 b(and)j(Sec.)35 b(4.9.3.)d(User-de\014ned)75 300 y(op)q(erators)14 b(ma)o(y)h(op)q(erate)g(on)g(general,)g(deriv)o(ed)h(datat)o(yp)q(es.) 166 357 y(This)g(text)e(is)i(c)o(hanged)g(to:)166 413 y(The)g Fl(datat)o(yp)q(e)i Fq(argumen)o(t)e(of)f Fl(MPI)p 783 413 V 16 w(REDUCE)i Fq(m)o(ust)f(b)q(e)g(compatible)i(with)e Fl(op)p Fq(.)23 b(Prede\014ned)18 b(op-)75 470 y(erators)c(w)o(ork)h (only)h(with)f(the)h Fl(MPI)f Fq(t)o(yp)q(es)g(listed)i(in)f(Sec.)21 b(4.9.2)14 b(and)h(Sec.)21 b(4.9.3.)e(F)l(urthermore,)c(the)75 526 y Fl(datat)o(yp)q(e)20 b Fq(and)e Fl(op)h Fq(giv)o(en)f(for)g (prede\014ned)i(op)q(erators)d(m)o(ust)h(b)q(e)h(the)f(same)g(on)g(all) h(pro)q(cesses.)30 b(User-)75 583 y(de\014ned)15 b(op)q(erators)e(ma)o (y)g(op)q(erate)g(on)h(general,)g(deriv)o(ed)h(datat)o(yp)q(es)e (without)g(the)h(restrictions)g(ab)q(o)o(v)o(e)75 639 y(for)h(prede\014ned)h(op)q(erators.)75 781 y Fn(3.6)59 b(New)20 b(Datat)n(yp)r(es)75 884 y Fe(3.6.1)49 b(Simple)17 b(Struct)d(T)l(yp)q(es)75 1065 y Fl(MPI)p 160 1065 V 16 w(TYPE)p 293 1065 V 17 w(SIMPLE)p 469 1065 V 15 w(STRUCT\(count,)j (a)o(rra)o(y)p 908 1065 V 14 w(of)p 959 1065 V 16 w(blo)q(cklengths,)h (a)o(rra)o(y)p 1334 1065 V 14 w(of)p 1385 1065 V 16 w(t)o(yp)q(es,)e (newt)o(yp)q(e\))117 1142 y Fk(IN)155 b Fl(count)482 b Fk(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 1216 y(IN)155 b Fl(a)o(rra)o(y)p 416 1216 V 15 w(of)p 468 1216 V 16 w(blo)q(cklengths)191 b Fk(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h(eac)o(h)g(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in) o(teger\))117 1290 y(IN)155 b Fl(a)o(rra)o(y)p 416 1290 V 15 w(of)p 468 1290 V 16 w(t)o(yp)q(es)327 b Fk(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h(in)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f (handle\))117 1364 y(OUT)108 b Fl(newt)o(yp)q(e)433 b Fk(new)15 b(datat)o(yp)q(e)f(\(handle\))166 1489 y Fq(Eac)o(h)c(t)o(yp) q(e)h(in)g(the)g Fl(a)o(rra)o(y)p 592 1489 V 14 w(of)p 643 1489 V 16 w(t)o(yp)q(es)h Fq(list)g(m)o(ust)e(b)q(e)h(a)f Fc(p)n(ortable)h Fq(datat)o(yp)q(e:)17 b(I.e.,)10 b(a)h(basic)g(datat)o (yp)q(e,)f(or)75 1545 y(a)g(datat)o(yp)q(e)f(constructed)i(from)e(a)h (p)q(ortable)h(datat)o(yp)q(e)e(using)i(one)f(of)g(the)g(constructors)g Fl(MPI)p 1681 1545 V 15 w(TYPE)p 1813 1545 V 17 w(CONTIGUOUS,)75 1602 y(MPI)p 160 1602 V 16 w(TYPE)p 293 1602 V 17 w(VECTOR,)16 b(MPI)p 600 1602 V 15 w(TYPE)p 732 1602 V 17 w(INDEXED,)f Fq(or)g Fl(MPI)p 1110 1602 V 16 w(TYPE)p 1243 1602 V 16 w(SIMPLE)p 1418 1602 V 16 w(STRUCT)p Fq(.)166 1658 y(This)22 b(t)o(yp)q(e)g(constructor)f(is)i(similar)f(to)g Fl(MPI)p 988 1658 V 15 w(TYPE)p 1120 1658 V 17 w(STRUCT)p Fq(,)g(except)h(that)e(the)h(user)g(do)q(es)75 1715 y(not)e(supply)h (an)f(arra)o(y)e(of)i(displacemen)o(ts.)35 b(Instead,)22 b(the)e(successiv)o(e)h(blo)q(c)o(ks)f(are)g(assumed)g(to)f(b)q(e)75 1771 y(con)o(tiguous.)g(P)o(adding)11 b(spaces)h(are)e(added,)j (according)e(to)g(the)g(default)h(alignmen)o(t)g(rules)g(for)e (structure)75 1828 y(comp)q(onen)o(ts.)21 b(This)c(facilitates)f(the)g (construction)g(of)f(datat)o(yp)q(es)g(for)g(structures,)g(as)h(the)f (user)h(need)75 1884 y(not)f(compute)g(displacemen)o(ts.)75 1983 y Fb(Example)j(3.1)k Fq(The)14 b(follo)o(wing)g(co)q(de)f (declares)h(a)f(structure)g(t)o(yp)q(e)g(and)h(creates)f(a)g(datat)o (yp)q(e)f(for)h(that)75 2040 y(structure)i(t)o(yp)q(e.)75 2139 y Fm(struct)23 b(record)g({)147 2195 y(char)g(name;)147 2252 y(double)g(position[3];)147 2308 y(float)47 b(mass;)147 2365 y(})75 2478 y(MPI_Datatype)22 b(record_type;)75 2534 y(MPI_datatype)g(types[3])h(=)h({MPI_CHAR,)e(MPI_DOUBLE,)h (MPI_FLOAT};)75 2591 y(int)238 b(lengths[3])23 b(=)g({1,)h(3,)f(1};)75 2704 y(MPI_Type_simple_struct\()e(3,)i(lengths,)g(types,)g (record_type\);)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fq(6)970 b Ff(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fc(R)n(ationale.)76 b Fq(The)21 b(co)q(de)h(ab)q(o)o(v)o(e)f(is)h(m)o(uc)o(h)f(simpler)h(and)g(more)e (natural)i(than)f(the)g(curren)o(t)189 102 y(approac)o(h,)14 b(that)h(uses)g Fl(MPI)p 677 102 14 2 v 16 w(TYPE)p 810 102 V 17 w(STRUCT)p Fq(,)g(and)h(requires)g(to)e(compute)h(displacemen) o(ts.)189 177 y(It)i(also)g(pro)o(vides)h(a)f(de\014nition)i(of)e(the)h (record)f(t)o(yp)q(e)h(that)e(is)i(arc)o(hitecture)g(indep)q(enden)o(t) i(\(as-)189 233 y(suming)c(that)e(compilers)j(don't)e(\014ddle)i(with)e (the)h(order)f(of)g(\014elds,)h(whic)o(h)g(is)g(true)f(in)i(C\))d({)i (this)189 290 y(facilitates)f(transfer)e(of)h(records)g(using)h(RMA,)f (an)g(\014ts)g(w)o(ell)h(with)g(v)m(arious)f(prop)q(osals)h(for)e (third)189 346 y(part)o(y)h(transfers.)19 b(\()p Fc(End)d(of)g(r)n (ationale.)p Fq(\))189 452 y Fc(A)n(dvic)n(e)h(to)h(users.)50 b Fq(Datat)o(yp)q(es)16 b(constructed)h(using)h Fl(MPI)p 1231 452 V 16 w(TYPE)p 1364 452 V 16 w(SIMPLE)p 1539 452 V 16 w(STRUCT)g Fq(should)189 509 y(b)q(e)i(used)g(only)g(for)f (structures)g(that)g(use)h(the)g(default)g(alignmen)o(t)g(rules.)34 b(They)20 b(should)g(not)189 565 y(b)q(e)c(used)h(if)f(alignmen)o(t)g (rules)h(ha)o(v)o(e)f(b)q(een)h(c)o(hanged,)f(using)g(compiler)h (options)f(or)g(compilation)189 622 y(pragmas.)j(\()p Fc(End)c(of)i(advic)n(e)f(to)g(users.)p Fq(\))189 728 y Fc(A)n(dvic)n(e)g(to)h(implementors.)47 b Fq(The)16 b(datat)o(yp)q(e)g(constructor)f Fl(MPI)p 1330 728 V 16 w(TYPE)p 1463 728 V 17 w(SIMPLE)p 1639 728 V 15 w(STRUCT)i Fq(is)189 784 y(most)e(easily)i(implemen)o(ted)h(on)e(systems)g(that)f (use)h Fc(natur)n(al)g Fq(data)g(alignmen)o(t)h(rules.)23 b(On)17 b(suc)o(h)189 841 y(systems,)12 b(eac)o(h)h(basic)g(datat)o(yp) q(e)f(has)g(an)h(exten)o(t)f(and)h(an)g(alignmen)o(t;)h(the)e(alignmen) o(t)i(of)e(a)g(com-)189 897 y(p)q(ound)j(t)o(yp)q(e)g(is)h(the)f (strictest)f(alignmen)o(t)i(of)e(a)g(comp)q(onen)o(t.)20 b(P)o(adding)c(is)f(added)g(so)g(that)f(eac)o(h)189 954 y(comp)q(onen)o(t)f(is)h(aligned)g(at)f(its)g(natural)g(alignmen)o(t.) 20 b(W)l(e)13 b(can)g(then)h(deriv)o(e)g(ob)o(vious)f(de\014nitions)189 1010 y(for)h(the)h(exten)o(t)g(and)h(alignmen)o(t)g(of)e(p)q(ortable)i (MPI)f(datat)o(yp)q(es:)189 1067 y(The)g(exten)o(t)f(and)h(alignmen)o (t)g(of)f(a)h(basic)g(datat)o(yp)q(e)f(is)i(the)e(exten)o(t)h(and)g (alignmen)o(t)g(of)f(the)h(cor-)189 1123 y(resp)q(onding)h(language)f (t)o(yp)q(e.)189 1180 y(The)10 b(alignmen)o(t)h(of)f(a)f(datat)o(yp)q (e)h(built)i(with)e(one)g(of)g(the)g(constructors)g Fl(MPI)p 1487 1180 V 15 w(TYPE)p 1619 1180 V 17 w(CONTIGUOUS,)189 1236 y(MPI)p 274 1236 V 15 w(TYPE)p 406 1236 V 17 w(VECTOR,)26 b Fq(and)e Fl(PI)p 780 1236 V 16 w(TYPE)p 913 1236 V 17 w(INDEXED)h Fq(is)g(the)f(alignmen)o(t)h(of)f(the)h(comp)q(onen)o(t) 189 1293 y(datat)o(yp)q(e;)14 b(the)h(exten)o(t)g(of)g(the)g(comp)q (ound)h(datat)o(yp)q(e)f(is)g(as)g(de\014ned)i(in)f(MPI1.)189 1349 y(The)h(alignmen)o(t)g(of)f(a)g(datat)o(yp)q(e)h(built)h(b)o(y)e (a)h(call)g(to)f Fl(MPI)p 1221 1349 V 16 w(TYPE)p 1354 1349 V 17 w(SIMPLE)p 1530 1349 V 16 w(STRUCT\(n,)h(lens,)189 1405 y(t)o(yp)q(es,)i(newt)o(yp)q(e\))h Fq(is)e(max)643 1412 y Fa(i)672 1405 y Fm(alignment)p Fq(\()p Fm(types)p Fq([)p Fm(i)p Fq(]\);)e(the)i(exten)o(t)g(is)g(computed)h(b)o(y)f(la)o (ying)g(out)189 1462 y(the)h(comp)q(onen)o(ts)g(with)h(minimal)h (padding)f(added)g(so)f(that)g(eac)o(h)h(comp)q(onen)o(t)f(starts)f(at) h(its)189 1518 y(alignmen)o(t.)189 1593 y(Some)g(compilers)h(treat)f (the)g(\014rst)g(comp)q(onen)o(t)g(of)g(a)g(structure)g(di\013eren)o (tly)l(.)33 b(E.g.,)18 b(the)i(IBM)189 1650 y(compiler)d(aligns)h (doubles)f(at)f(w)o(ord)g(b)q(oundaries,)i(but)f(align)g(a)f(structure) g(that)g(starts)f(with)i(a)189 1706 y(double)j(comp)q(onen)o(t)f(at)f (doublew)o(ord)i(b)q(oundaries.)32 b(The)19 b(de\014nition)i(of)d(the)h (alignmen)o(t)h(of)e(a)189 1763 y(datat)o(yp)q(e)h(built)i(with)f Fl(MPI)p 690 1763 V 16 w(TYPE)p 823 1763 V 16 w(SIMPLE)p 998 1763 V 16 w(STRUCT)h Fq(is)f(c)o(hanged,)h(accordingly)l(.)34 b(\()p Fc(End)20 b(of)189 1819 y(advic)n(e)c(to)g(implementors.)p Fq(\))75 1941 y Fe(3.6.2)49 b(Contiguous)17 b(Struct)e(T)l(yp)q(es)75 2027 y Fq(Data)e(structures)g(ma)o(y)g(con)o(tain)h(padding)h(holes,)f (added)h(for)e(alignmen)o(t.)20 b(The)14 b(de\014nition)i(of)d(an)h (MPI)75 2083 y(datat)o(yp)q(e,)20 b(done)f(using)i(the)e Fl(MPI)p 687 2083 V 16 w(TYPE)p 820 2083 V 17 w(STRUCT)h Fq(constructor,)f(do)q(es)h(not)f(distinguish)j(b)q(et)o(w)o(een)75 2140 y(holes)g(that)e(con)o(tains)h(no)g(signi\014can)o(t)h(data,)f (and)h(are)e(there)h(only)h(for)e(alignmen)o(t)i(purp)q(oses,)h(and)75 2196 y(b)q(et)o(w)o(een)18 b(holes)h(that)e(ma)o(y)g(con)o(tain)h (signi\014can)o(t)h(data)f(and)g(w)o(ere)f(purp)q(osefully)j(left)f (out)e(b)o(y)h(a)g(user)75 2253 y(that)i(wishes)i(not)e(to)g(o)o(v)o (erwrite)h(some)f(of)h(the)f(information)h(in)h(a)f(structure.)36 b(As)21 b(a)g(result,)h(MPI)75 2309 y(cannot)f(o)o(v)o(erwrite)f(these) i(holes,)g(this)g(prev)o(en)o(ts)f(the)g(ob)o(vious)g(optimization)h (of)f(shipping)i(a)e(data)75 2366 y(structure)g(as)g(a)g(con)o(tiguous) h(blo)q(c)o(k)g(of)f(data,)h(padding)g(holes)g(included.)42 b(The)21 b(t)o(yp)q(e)h(constructor)75 2422 y Fl(MPI)p 160 2422 V 16 w(TYPE)p 293 2422 V 17 w(CONTIGUOUS)p 598 2422 V 18 w(STRUCT)11 b Fq(marks)f(an)o(y)g(suc)o(h)h(holes)g(as)g (\\don)o(t-care")e(lo)q(cations,)j(allo)o(wing)75 2478 y(the)j(implemen)o(tation)i(to)d(o)o(v)o(erwrite)h(them.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Ff(3.6.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPES)1267 b Fq(7)75 45 y Fl(MPI)p 160 45 14 2 v 16 w(TYPE)p 293 45 V 17 w(CONTIGUOUS)p 598 45 V 18 w(STRUCT\(count,)18 b(a)o(rra)o(y)p 1041 45 V 14 w(of)p 1092 45 V 16 w(blo)q(cklengths,)h (a)o(rra)o(y)p 1468 45 V 14 w(of)p 1519 45 V 16 w(displacements,)g(a)o (r-)75 102 y(ra)o(y)p 136 102 V 15 w(of)p 188 102 V 16 w(t)o(yp)q(es,)e(newt)o(yp)q(e\))117 179 y Fk(IN)155 b Fl(count)482 b Fk(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o (teger\))117 253 y(IN)155 b Fl(a)o(rra)o(y)p 416 253 V 15 w(of)p 468 253 V 16 w(blo)q(cklengths)191 b Fk(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h(eac)o(h)g(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in) o(teger\))117 328 y(IN)155 b Fl(a)o(rra)o(y)p 416 328 V 15 w(of)p 468 328 V 16 w(displacements)164 b Fk(b)o(yte)14 b(displacemen)o(t)g(of)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in)o (teger\))117 402 y(IN)155 b Fl(a)o(rra)o(y)p 416 402 V 15 w(of)p 468 402 V 16 w(t)o(yp)q(es)327 b Fk(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h(in)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f (handle\))117 476 y(OUT)108 b Fl(newt)o(yp)q(e)433 b Fk(new)15 b(datat)o(yp)q(e)f(\(handle\))166 601 y Fq(The)c(seman)o (tics)h(of)e Fl(MPI)p 587 601 V 16 w(TYPE)p 720 601 V 17 w(CONTIGUOUS)p 1025 601 V 18 w(STRUCT)i Fq(are)f(iden)o(tical)i(to)d Fl(MPI)p 1611 601 V 16 w(TYPE)p 1744 601 V 17 w(STRUCT)p Fq(,)75 657 y(except)15 b(that,)e(if)h(a)g(receiv)o(e)h(call)g(uses)g (a)e(datat)o(yp)q(e)h(constructed)g(with)h(this)f(constructor,)f(then)i (the)f(in-)75 714 y(ternal)f(holes)h(in)f(the)g(structure)g(can)g(b)q (e)g(o)o(v)o(erwritten.)18 b(The)13 b(same)g(rule)h(\(namely)l(,)f (that)f(holes)i(b)q(et)o(w)o(een)75 770 y(comp)q(onen)o(ts)c(can)g(b)q (e)h(o)o(v)o(erwritten\))e(applies)j(also)e(to)g(datat)o(yp)q(es)f (constructed)h(using)h Fl(MPI)p 1635 770 V 16 w(TYPE)p 1768 770 V 17 w(SIMPLE)p 1944 770 V 16 w(STRUCT)p Fq(.)166 874 y Fh(Discussion:)29 b Fk(It)10 b(is)f(reasonable)g(to)g(assume)g (that)h(holes)f(in)g(la)o(y)o(outs)f(de\014ned)j(using)e Fd(MPI)p 1576 874 13 2 v 14 w(TYPE)p 1698 874 V 14 w(SIMPLE)p 1857 874 V 15 w(STRUCT)75 930 y Fk(con)o(tain)14 b(no)h(signi\014can)o (t)f(information,)e(since)k(this)f(datat)o(yp)q(e)g(constructor)h(is)f (exp)q(ected)i(to)e(b)q(e)g(used)h(for)f(sp)q(ec-)75 987 y(ifying)f(the)h(la)o(y)o(out)f(of)h(structures.)24 b(Users)16 b(will)e(b)q(e)i(required)g(to)f(use)h Fd(MPI)p 1271 987 V 14 w(TYPE)p 1393 987 V 14 w(STRUCT)f Fk(when)g(holes)h(hold) 75 1043 y(signi\014can)o(t)d(information.)189 1191 y Fc(A)n(dvic)n(e)e(to)i(users.)38 b Fq(Users)11 b(should)i(arrange)d (the)i(\014elds)g(in)g(a)f(C)h(struct)e(t)o(yp)q(e)i(in)g(descending)h (order)189 1248 y(of)i(their)i(size.)24 b(This)17 b(arrangemen)o(t)e (has)h(t)o(w)o(o)f(adv)m(an)o(tages:)21 b(\014rstly)c(it)f(ma)o(y)g (reduce)h(the)f(size)h(of)189 1304 y(the)i(struct)f(b)o(y)h (eliminating)i(padding)f(b)q(et)o(w)o(een)f(elemen)o(ts)h(with)f (di\013eren)o(t)g(alignmen)o(ts)h(\(this)189 1361 y(is)c(generally)i (true)e(for)g(C)g(programs\).)22 b(Secondly)l(,)c(it)e(ma)o(y)g (increase)h(the)g(n)o(um)o(b)q(er)f(of)g(cases)h(for)189 1417 y(whic)o(h)d(the)f(implemen)o(tation)h(can)g(p)q(erform)f(blo)q(c) o(k)g(copies)i(b)o(y)e(ensuring)h(that)e(the)h(\014rst)g(elemen)o(t)189 1474 y(of)19 b(the)g(structure)g(has)h(the)f(most)g(stringen)o(t)g (alignmen)o(t)h(requiremen)o(ts.)33 b(\()p Fc(End)19 b(of)i(advic)n(e)f(to)189 1530 y(users.)p Fq(\))189 1631 y Fc(A)n(dvic)n(e)15 b(to)i(implementors.)39 b Fq(An)16 b(implemen)o(tation)g(ma)o(y)f(ignore)g(the)h(constructor)189 1687 y Fl(MPI)p 274 1687 14 2 v 15 w(TYPE)p 406 1687 V 17 w(CONTIGUOUS)p 711 1687 V 18 w(STRUCT)p Fq(,)21 b(and)f(handle)i(it)e(exactly)h(as)f Fl(MPI)p 1539 1687 V 16 w(TYPE)p 1672 1687 V 17 w(STRUCT)p Fq(.)189 1744 y(Th)o(us,)f(there)g(is)h(\(almost\))d(no)i(o)o(v)o(erhead)g(in)g(supp) q(orting)h(this)f(constructor,)g(without)g(taking)189 1800 y(adv)m(an)o(tage)f(of)h(it.)32 b(One)20 b(p)q(ossible)h(w)o(a)o (y)d(of)h(taking)g(adv)m(an)o(tage)g(of)g(it,)h(is)g(to)e(adopt)h(a)g (canoni-)189 1857 y(cal)d(\\wire")g(represen)o(tation)g(for)g (structures.)22 b(Supp)q(ose)17 b(that)f(the)g(compiler)h(uses,)g(b)o (y)f(default,)189 1913 y(natural)g(alignmen)o(t)i(rules)f(for)f (structures:)23 b(eac)o(h)16 b(basic)i(comp)q(onen)o(t)e(is)i(aligned)g (at)e(a)g(natural)189 1970 y(b)q(oundary)l(.)k(Supp)q(ose)14 b(that)f(one)h(adopts)f(natural)h(alignmen)o(t)g(rules)h(for)e (messages)g(on)g(the)h(wire,)189 2026 y(in)19 b(a)f(homogeneous)g(en)o (vironmen)o(t:)27 b(messages)18 b(are)g(padded)h(so)f(that)g(eac)o(h)g (basic)h(elemen)o(t)g(is)189 2083 y(aligned)d(at)f(a)g(natural)g(b)q (oundary)l(.)20 b(Then)189 2139 y(\(1\))14 b(A)i(receiv)o(er)g(can)f (alw)o(a)o(ys)g(deco)q(de)h(an)g(incoming)h(message)d(and)i(retriev)o (e)g(correctly)f(the)h(ba-)189 2195 y(sic)k(comp)q(onen)o(ts.)31 b(This)20 b(b)q(ecause)g(the)g(basic)g(datat)o(yp)q(e)e(of)h(the)g (elemen)o(t)h(indicates)h(at)e(what)189 2252 y(b)q(oundary)c(it)h(is)f (aligned.)189 2308 y(\(2\))f(A)h(sender)g(can)h(blo)q(c)o(k)f(cop)o(y)g (a)g(structure)g(on)o(to)f(the)h(\\wire")g(\(most)f(lik)o(ely)l(,)j(on) o(to)d(a)g(message)189 2365 y(bu\013er\),)19 b(when)g(the)g(structure)g (is)g(naturally)h(aligned,)h Fc(r)n(elative)e(to)h(the)g(start)g(of)f (the)h(sending)189 2421 y(bu\013er)p Fq(.)26 b(This)18 b(optimization)g(do)q(es)g(not)e(require)j(that)d(the)h(sending)i (datat)o(yp)q(e)e(b)q(e)h(built)g(using)189 2478 y Fl(MPI)p 274 2478 V 15 w(TYPE)p 406 2478 V 17 w(CONTIGUOUS)p 711 2478 V 18 w(STRUCT)p Fq(:)c(The)g(MPI)f(library)i(can)e(scan)h(datat)o (yp)q(es,)f(when)h(they)189 2534 y(are)h(committed,)f(and)i(recognize)g (when)g(a)e(datat)o(yp)q(e)h(\(or)f(a)h(datat)o(yp)q(e)g(fragmen)o(t\)) f(is)h(naturally)189 2591 y(aligned.)24 b(Blo)q(c)o(k)17 b(cop)o(ying)g(can)g(b)q(e)g(used)g(ev)o(en)f(if)h(the)g(holes)g(in)g (the)f(datat)o(yp)q(e)g(con)o(tain)g(signi\014-)189 2647 y(can)o(t)e(information.)189 2704 y(\(3\))h(A)h(receiv)o(er)h(can)f (blo)q(c)o(k)h(cop)o(y)f(an)g(incoming)i(message)d(on)o(to)h(the)g (receiv)o(e)h(bu\013er)f(when)h(the)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fq(8)970 b Ff(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fq(receiving)g(structure)f(is)h(naturally)g (aligned)h Fc(r)n(elative)f(to)g(the)h(start)g(of)f(the)h(r)n(e)n(c)n (eiving)d(bu\013er)i Fq(and)189 102 y(holes)10 b(can)h(b)q(e)g (ignored.)18 b(I.e.,)11 b(when)g(the)f(receiving)i(datat)o(yp)q(e)d(is) i(built)g(using)g Fl(MPI)p 1597 102 14 2 v 16 w(TYPE)p 1730 102 V 17 w(CONTIGUOUS)p 2035 102 V 18 w(STRUCT)189 158 y Fq(or)i Fl(MPI)p 328 158 V 16 w(TYPE)p 461 158 V 17 w(SIMPLE)p 637 158 V 16 w(STRUCT)h Fq(and)g(is)h(naturally)f (aligned,)i(relativ)o(e)e(to)g(its)g(\014rst)f(p)q(osition.)189 231 y(The)k(same)h(wire)g(represen)o(tation)f(ma)o(y)g(b)q(e)h(used)g (in)h(a)e(heterogenous)h(en)o(vironmen)o(t,)g(in)g(cases)189 288 y(where)h(data)e(con)o(v)o(ersion)i(is)g(done)g(at)f(the)h (destination;)i(the)e(destination)g(has)g(to)f(kno)o(w,)g(not)189 344 y(only)c(the)h(basic)g(datat)o(yp)q(es)e(used)i(b)o(y)f(the)h (source,)f(but)g(also)h(the)f(alignmen)o(t)h(rules)g(used)g(b)o(y)f (the)189 401 y(source.)-443 b Fj(>)15 b Fi(\(Sept\))189 474 y Fq(Example:)34 b(Consider)23 b(a)f(C)g(structure)g Fm(struct)h({char)g(a[2];)g(float)g(b[2];)h(double)f(c})p Fq(;)189 531 y(assume)16 b(that)g(\015oats)g(are)g(4)h(b)o(ytes)f(and)h (doubles)h(are)e(8)g(b)o(ytes,)h(and)g(structures)f(are)g(naturally)189 587 y(aligned.)36 b(The)20 b(memory)f(la)o(y)o(out)h(is)44 b Fm([c][c]xx[ffff][ffff]xxxx[dddd)o(dddd])p Fq(,)18 b(and)i(the)189 643 y(structure)15 b(is)g(aligned)i(to)d(an)i(8)e(b)o (yte)h(b)q(oundary)l(.)189 717 y(Case)c(1:)18 b(The)12 b(en)o(tire)g(structure)f(is)h(comm)o(unicated.)20 b(Sender)12 b(constructs)f(a)h(datat)o(yp)q(e)f(for)g(struc-)189 773 y(ture)f(using)h Fl(MPI)p 479 773 V 15 w(TYPE)p 611 773 V 17 w(STRUCT,)g(MPI)p 910 773 V 15 w(TYPE)p 1042 773 V 17 w(SIMPLE)p 1218 773 V 16 w(STRUCT)g Fq(or)f Fl(MPI)p 1554 773 V 15 w(TYPE)p 1686 773 V 17 w(CONTIGUOUS)p 1991 773 V 18 w(STRUCT)p Fq(;)189 830 y(receiv)o(er)16 b(constructs)g(a)g(datat)o(yp)q(e)g(for)f(this)i(structure,)e(using)i Fl(MPI)p 1379 830 V 16 w(CONTIGUOUS)p 1683 830 V 18 w(STRUCT)189 886 y Fq(or)12 b Fl(MPI)p 327 886 V 16 w(TYPE)p 460 886 V 16 w(SIMPLE)p 635 886 V 16 w(STRUCT)p Fq(.)h(Then)g(the)g(la)o(y)o (outs)f(in)h(the)g(sender)g(memory)l(,)g(on)f(the)h(wire,)189 943 y(and)i(in)h(the)f(receiv)o(er)h(memory)f(are)g(iden)o(tical,)h (and)g(data)e(can)i(b)q(e)g(simply)g(copied.)189 1016 y(Case)10 b(2:)17 b(comm)o(unication)12 b(in)o(v)o(olv)o(es)f(only)h (the)f(second)g(of)f(the)h(t)o(w)o(o)f(c)o(haracters,)g(and)h(the)g (remain-)189 1073 y(der)f(of)g(the)g(structure.)18 b(Receiv)o(er)11 b(constructs)f(the)g(datat)o(yp)q(e)g(using)h Fl(MPI)p 1441 1073 V 16 w(CONTIGUOUS)p 1745 1073 V 18 w(STRUCT)p Fq(.)189 1129 y(The)19 b(memory)f(la)o(y)o(out)g(is)h Fm([c]xx[ffff][ffff]xxxx[dddddd)o(dd])p Fq(,)d(and)j(the)g(wire)g(la)o (y)o(out)f(is)189 1185 y Fm([c]xxx[ffff][ffff]xxxx[)o(dddddddd)o(])p Fq(.)e(The)c(datat)o(yp)q(e)f(is)h(not)g(naturally)g(aligned,)h(so)f (that)189 1242 y(blo)q(c)o(k)k(cop)o(ying)g(cannot)f(b)q(e)h(used)g(at) f(the)h(sender)g(or)f(the)g(receiv)o(er.)22 b(Ho)o(w)o(ev)o(er,)14 b(a)h(smart)f(imple-)189 1298 y(men)o(tation)h(ma)o(y)g(realize)i(that) e(the)h(fragmen)o(t)f(starting)g(from)g(the)h(\015oat)f(is)i(naturally) f(aligned,)189 1355 y(relativ)o(e)e(to)e(the)i(start)e(of)h(the)g (bu\013er,)h(so)f(that)f(blo)q(c)o(k)j(cop)o(ying)e(can)h(b)q(e)g(used) g(for)f(this)g(fragmen)o(t.)189 1428 y(Case)f(3:)19 b(comm)o(unication) 14 b(in)o(v)o(olv)o(es)g(only)f(the)g(t)o(w)o(o)f(\015oats)h(and)g(the) g(double.)21 b(The)13 b(memory)g(la)o(y-)189 1485 y(out)c(is)i Fm([ffff][ffff]xxxx[dddddddd])o Fq(.)16 b(The)10 b(la)o(y)o(out)g(on)g (the)g(wire)g(is)h Fm([ffff][ffff][dddddddd])p Fq(.)189 1541 y(The)j(datat)o(yp)q(e)f(is)i(not)f(naturally)g(aligned,)i (relativ)o(e)e(to)g(its)g(start,)e(so)i(that)f(blo)q(c)o(k)i(cop)o (ying)g(can-)189 1598 y(not)f(b)q(e)i(used.)21 b(\()p Fc(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))-1227 b Fj(?)15 b Fi(\(Sept\))166 1732 y Fh(Alternativ)o(es)o(:)134 1803 y Fk(1.)22 b(No)14 b(new)h(function)f Fd(MPI)p 581 1803 13 2 v 14 w(TYPE)p 703 1803 V 15 w(CONTIGUOUS)p 985 1803 V 12 w(STRUCT)p Fk(:)g(instead,)g(holes)h(can)g(b)q(e)g(o)o(v) o(erwritten)g(only)189 1853 y(for)f(datat)o(yp)q(es)i(built)e(with)g Fd(MPI)p 717 1853 V 15 w(TYPE)p 840 1853 V 14 w(SIMPLE)p 999 1853 V 15 w(STRUCT)p Fk(.)f(The)j(argumen)o(t)e(is)g(that,)h(at)g (least)g(in)f(the)189 1903 y(scenario)f(I)g(presen)o(ted,)i(hole)d(o)o (v)o(erwriting)h(is)g(tak)o(en)g(adv)n(an)o(tage)f(of)g(only)g(for)h (structures)i(with)e(a)f(natural)189 1953 y(alignmen)o(t.)j(But,)f (then,)g(their)h(la)o(y)o(out)d(can)i(b)q(e)h(sp)q(eci\014ed)g(using)f Fd(MPI)p 1308 1953 V 14 w(TYPE)p 1430 1953 V 15 w(SIMPLE)p 1590 1953 V 14 w(STRUCT)p Fk(.)134 2024 y(2.)22 b(Both)d(sender)h(and)f (receiv)o(er)h(are)g(required)f(to)g(use)h Fd(MPI)p 1126 2024 V 14 w(TYPE)p 1248 2024 V 14 w(CONTIGUOUS)p 1529 2024 V 13 w(STRUCT)p Fk(,)e(and)g(the)189 2074 y(datat)o(yp)q(es)c(on)f (b)q(oth)h(ends)g(ha)o(v)o(e)g(to)f(b)q(e)h(structurally)g(equiv)n (alen)o(t,)f(i.e.,)f(de\014ned)j(b)o(y)e(the)h(same)f(sequence)189 2124 y(of)g(de\014nitions.)18 b(There)d(are)f(sev)o(eral)h(p)q(ossible) f(v)n(arian)o(ts)f(for)g(suc)o(h)i(prop)q(osal)189 2184 y(\(a\))20 b(This)g(construct)i(can)e(b)q(e)h(only)e(used)i(in)f(a)g (homogeneous)f(en)o(vironmen)o(t.)36 b(But,)22 b(then,)g(w)o(e)e(lose) 189 2234 y(p)q(ortabilit)o(y)11 b(to)h(heterogeneous)j(en)o(vironmen)o (t,)c(and)i(w)o(e)g(ac)o(hiev)o(e)f(little)g(that)h(could)f(not)h(b)q (e)g(ac)o(hiev)o(ed)g(b)o(y)189 2284 y(using)g Fd(MPI)p 374 2284 V 15 w(BYTE)p Fk(.)189 2344 y(\(b\))18 b(This)g(construct)i (can)e(b)q(e)g(used)h(p)q(ortably)m(,)f(with)g(enhanced)h(p)q (erformance)f(obtained)g(in)f(a)h(homo-)189 2394 y(geneous)i(en)o (vironmen)o(t.)36 b(E.g.,)20 b(a)g(homogeneous)f(implem)o(en)o(tation)e (will)i(use)h(blo)q(c)o(k)g(cop)o(ying,)g(while)189 2444 y(a)d(heterogeneous)j(implemen)o(tatio)o(n)15 b(will)i(mo)o(v)o(e)f (elemen)o(t)h(b)o(y)h(elemen)o(t)f(\(at)h(least,)g(for)g(heterogeneous) 189 2494 y(comm)o(unicators\).)j(This)16 b(assumes)g(that,)g(in)f(a)h (homogeneous)e(en)o(vironmen)o(t,)h(di\013eren)o(t)i(pro)q(cesses)i (are)189 2543 y(compiled)8 b(with)i(the)g(same)f(data)h(alignmen)o(t)d (options.)17 b(Also,)10 b(this)g(runs)g(coun)o(ter)h(the)g(datat)o(yp)q (e)f(matc)o(hing)189 2593 y(rules)k(in)g(MPI,)f(where)i(only)e (signature)i(matc)o(hing)d(is)h(required.)189 2654 y(\(c\))d Fd(MPI)p 326 2654 V 14 w(TYPE)p 448 2654 V 14 w(CONTIGUOUS)p 729 2654 V 13 w(STRUCT)f Fk(can)g(b)q(e)i(matc)o(hed)d(with)h(other)i (datat)o(yp)q(es,)f(as)g(long)e(as)i(signa-)189 2704 y(tures)g(matc)o(h;)f(impro)o(v)o(ed)f(p)q(erformance)h(is)g(ac)o(hiev) o(ed)h(when)f(b)q(oth)h(ends)g(use)g Fd(MPI)p 1463 2704 V 14 w(TYPE)p 1585 2704 V 15 w(CONTIGUOUS)p 1867 2704 V 12 w(STRUCT)p Fk(.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Ff(3.7.)34 b(MINOR)16 b(CORRECTIONS)1154 b Fq(9)189 45 y Fk(But,)12 b(then,)h(the)g(wire)g(proto)q(col)f(m)o (ust)f(b)q(e)i(so)f(that)h(individual)d(elemen)o(ts)i(can)h(b)q(e)f (extracted,)i(in)e(case)h(the)189 95 y(receiv)o(er)k(do)q(es)f(not)f (use)i Fd(MPI)p 664 95 13 2 v 14 w(TYPE)p 786 95 V 14 w(CONTIGUOUS)p 1067 95 V 13 w(STRUCT)p Fk(.)d(In)i(this)f(case,)i (asking)e(the)h(sender)h(to)189 145 y(use)g Fd(MPI)p 340 145 V 14 w(TYPE)p 462 145 V 15 w(CONTIGUOUS)p 744 145 V 12 w(STRUCT)f Fk(do)q(es)h(not)g(pro)o(vide)f(additional)f (function)h(o)o(v)o(er)h(the)g(main)189 195 y(prop)q(osal.)g(The)d (di\013erence)h(is)f(only)e(whether)j(the)f(use)g(asserts)i(that)d(the) h(sending)g(datat)o(yp)q(e)f(has)h(a)f(nat-)189 244 y(ural)i(la)o(y)o (out)g(\(using)h Fd(MPI)p 608 244 V 14 w(TYPE)p 730 244 V 14 w(CONTIGUOUS)p 1011 244 V 13 w(STRUCT)p Fk(\),)f(or)h(whether)h (MPI)f(disco)o(v)o(ers)h(that)f(fact)189 294 y(b)o(y)d(itself.)75 488 y Fe(3.6.3)49 b(Convenient)17 b(F)o(o)o(rm)d(of)j(MPI)p 731 488 15 2 v 18 w(T)l(yp)q(e)p 848 488 V 18 w(indexed)p 1018 488 V 18 w(blo)q(ck)75 574 y Fq(This)e(prop)q(osal)g(is)g(to)f (allo)o(w)h(constan)o(t)f(blo)q(c)o(ksize)i(and)f(arbitrary)f (displacemen)o(ts)i(in)f(the)g(in)o(terface)g(to)75 630 y Fl(MPI)p 160 630 14 2 v 16 w(TYPE)p 293 630 V 17 w(INDEXED)p Fq(.)9 b(The)i(routine)f(no)o(w)g(tak)o(es)f Fl(a)o(rra)o(y)p 1063 630 V 15 w(of)p 1115 630 V 16 w(blo)q(cklengths)j Fq(and)f Fl(a)o(rra)o(y)p 1555 630 V 14 w(of)p 1606 630 V 16 w(displacements)75 687 y Fq(as)17 b(argumen)o(ts.)27 b(There)18 b(are)f(man)o(y)h(co)q(des)g(using)g(indirect)i(addressing)e (arising)g(from)f(unstructured)75 743 y(grids)f(where)h(the)f(blo)q(c)o (ksize)i(is)e(alw)o(a)o(ys)f(1)h(\(gather/scatter\).)k(W)l(e)c(prop)q (ose)g(this)h(in)o(terface,)f(allo)o(wing)75 800 y(for)f(constan)o(t)f (blo)q(c)o(ksize)j(and)e(arbitrary)g(displacemen)o(ts.)75 951 y Fl(MPI)p 160 951 V 16 w(TYPE)p 293 951 V 17 w(INDEXED)p 505 951 V 16 w(BLOCK\()g(count,)i(blo)q(cklength,)h(a)o(rra)o(y)p 1180 951 V 14 w(of)p 1231 951 V 16 w(displacements,)g(oldt)o(yp)q(e,)f (newt)o(yp)q(e\))117 1084 y Fk(IN)155 b Fl(count)482 b Fk(length)14 b(of)f(arra)o(y)h(of)f(displacemen)o(ts)117 1159 y(IN)155 b Fl(blo)q(cklength)371 b Fk(size)15 b(of)e(blo)q(c)o(k) 117 1235 y(IN)155 b Fl(a)o(rra)o(y)p 416 1235 V 15 w(of)p 468 1235 V 16 w(displacements)164 b Fk(arra)o(y)14 b(of)f(displacemen)o (ts)117 1310 y(IN)155 b Fl(oldt)o(yp)q(e)450 b Fk(old)13 b(datat)o(yp)q(e)117 1385 y(OUT)108 b Fl(newt)o(yp)q(e)433 b Fk(new)15 b(datat)o(yp)q(e)75 1509 y Fm(int)23 b(MPI)p 245 1509 15 2 v 17 w(Type)p 358 1509 V 17 w(Indexed)p 543 1509 V 16 w(Block\()g(int)h(count,)f(int)g(blocklength,)393 1566 y(int*)g(array)p 635 1566 V 17 w(of)p 700 1566 V 17 w(displacements,)f(MPI)p 1147 1566 V 17 w(Datatype)g(oldtype,)393 1622 y(MPI)p 468 1622 V 17 w(Datatype*)h(newtype)f(\))75 1709 y(MPI)p 150 1709 V 17 w(TYPE)p 263 1709 V 16 w(INDEXED)p 447 1709 V 17 w(BLOCK\(COUNT,)g(BLOCKLENGTH,)g(ARRAY)p 1204 1709 V 17 w(OF)p 1269 1709 V 16 w(DISPLACEMENTS,)g(OLDTYPE,)393 1765 y(NEWTYPE,)h(IERROR\))170 1822 y(INTEGER)g(COUNT,)g(INTEGER)g (BLOCKLENGTH,)g(ARRAY)p 1153 1822 V 16 w(OF)p 1217 1822 V 17 w(DISPLACEMENT\(*\),)f(OLDTYPE,)170 1878 y(NEWTYPE,)h(IERROR)75 1964 y(int)g(MPI::Datatype::Indexed)p 701 1964 V 15 w(Block\()g(int)g (count,)g(int)h(blocklength,)393 2021 y(int*)f(array)p 635 2021 V 17 w(of)p 700 2021 V 17 w(displacements,)f(MPI::Datatype&)g (oldtype)h(\))75 2194 y Fn(3.7)59 b(Mino)n(r)21 b(Co)n(rrections)75 2296 y Fq(The)e(follo)o(wing)g(corrections)f(to)g Fl(MPI-1.1)f Fq(are)h(\(all)h(page)f(and)h(line)h(n)o(um)o(b)q(ers)e(are)g(for)g (the)h(June)g(12,)75 2352 y(1995)14 b(v)o(ersion)h(without)h(c)o (hangebars\):)143 2458 y Fj(\017)23 b Fq(P)o(age)14 b(19,)g(lines)j (1-2)e(reads)189 2515 y(for)f(\(64)g(bit\))i(C)f(in)o(tegers)g (declared)i(to)d(b)q(e)i(of)f(t)o(yp)q(e)g Fl(longlong)g(int)189 2571 y Fq(but)g(should)h(read)189 2628 y(for)e(\(64)g(bit\))i(C)f(in)o (tegers)g(declared)i(to)d(b)q(e)i(of)f(t)o(yp)q(e)g Fl(long)g(long)g (int)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fq(10)947 b Ff(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fh(Discussion)o(:)189 114 y Fk(Just)f(to)g(mak)o(e)f(it)g(correct)j(C)d(so)h(it)g(is)g(long)f (long.)143 255 y Fj(\017)23 b Fq(P)o(age)14 b(74,)g(lines)j(39-45)d (read)189 311 y(A)i(datat)o(yp)q(e)f(ma)o(y)g(sp)q(ecify)i(o)o(v)o (erlapping)g(en)o(tries.)22 b(The)16 b(use)g(of)g(suc)o(h)g(a)f(datat)o (yp)q(e)h(in)g(a)g(receiv)o(e)189 368 y(op)q(eration)e(is)h(erroneous.) k(\(This)14 b(is)h(erroneous)f(ev)o(en)g(if)h(the)f(actual)g(message)g (receiv)o(ed)h(is)g(short)189 424 y(enough)g(not)g(to)g(write)g(an)o(y) g(en)o(try)g(more)f(than)h(once.\))189 499 y(A)i(datat)o(yp)q(e)g(ma)o (y)g(sp)q(ecify)i(o)o(v)o(erlapping)g(en)o(tries.)27 b(If)18 b(suc)o(h)g(a)g(datat)o(yp)q(e)f(is)h(used)g(in)h(a)e(receiv)o (e)189 556 y(op)q(eration,)d(that)h(is,)g(if)g(some)g(part)f(of)h(the)f (receiv)o(e)i(bu\013er)f(is)h(written)f(more)f(than)h(once)g(b)o(y)g (the)189 612 y(receiv)o(e)h(op)q(eration,)f(then)g(the)h(call)g(is)g (erroneous.)189 687 y(The)i(\014rst)f(part)g(w)o(as)h(an)f Fl(MPI-1.1)g Fq(addition.)29 b(The)18 b(second)g(part)f(o)o(v)o(erlaps) h(with)g(it.)28 b(The)18 b(old)189 744 y(text)c(will)j(b)q(e)f(remo)o (v)o(ed)f(so)g(it)g(no)o(w)g(reads)189 800 y(A)h(datat)o(yp)q(e)f(ma)o (y)g(sp)q(ecify)i(o)o(v)o(erlapping)g(en)o(tries.)22 b(The)16 b(use)g(of)g(suc)o(h)g(a)f(datat)o(yp)q(e)h(in)g(a)g(receiv)o (e)189 857 y(op)q(eration)e(is)h(erroneous.)k(\(This)14 b(is)h(erroneous)f(ev)o(en)g(if)h(the)f(actual)g(message)g(receiv)o(ed) h(is)g(short)189 913 y(enough)g(not)g(to)g(write)g(an)o(y)g(en)o(try)g (more)f(than)h(once.\))189 1029 y Fh(Discussion)o(:)189 1097 y Fk(The)f(in)o(ten)o(t)h(is)f(to)g(remo)o(v)o(e)f(duplicate)h (text)h(whic)o(h)f(seems)h(to)f(ha)o(v)o(e)g(b)q(een)h(left.)k(No)14 b(c)o(hange)h(in)f(meaning)189 1147 y(is)f(in)o(tended.)143 1288 y Fj(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(3)e(reads)189 1345 y(MPI)p 281 1345 14 2 v 16 w(P)o(ac)o(k)p 393 1345 V 16 w(size\(coun)o(t,)g(MPI)p 724 1345 V 16 w(CHAR,)h(&k2\);)189 1401 y(but)f(should)h(read)189 1457 y(MPI)p 281 1457 V 16 w(P)o(ac)o(k)p 393 1457 V 16 w(size\(coun)o(t,)f(MPI)p 724 1457 V 16 w(CHAR,)h(comm,)e(&k2\);)189 1630 y Fh(Discussion)o(:)189 1698 y Fk(This)f(is)h(a)g(minor)e(\014x)i(to)f(an)h(example)f(for)g(a)h (missing)e(argumen)o(t.)143 1839 y Fj(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(10)e(reads)189 1896 y(MPI)p 281 1896 V 16 w(P)o(ac)o(k\(c)o(hr,)f(coun)o(t,)g(MPI)p 726 1896 V 17 w(CHAR,)h(&lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 1952 y(but)f(should)h(read)189 2009 y(MPI)p 281 2009 V 16 w(P)o(ac)o(k\(c)o(hr,)e(coun)o(t,)g(MPI)p 726 2009 V 17 w(CHAR,)h(lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 2181 y Fh(Discussion)o(:)189 2249 y Fk(This)e(is)h(a)f(minor)f(\014x)i (to)g(an)f(example)g(for)g(an)g(argumen)o(t)g(that)h(is)g(declared)g (to)g(b)q(e)g(a)g(p)q(oin)o(ter)g(and)f(then)189 2299 y(passed)h(with)e(&.)143 2440 y Fj(\017)23 b Fq(P)o(age)14 b(109,)g(lines)j(26/27)d(and)h(page)g(110,)f(lines)j(28/29)d(reads)189 2497 y(The)h Fm(j)p Fq(th)g(blo)q(c)o(k)g(of)g(data)f(sen)o(t)h(from)f (eac)o(h)h(pro)q(cess)g(is)h(receiv)o(ed)g(b)o(y)f(ev)o(ery)f(pro)q (cess)i(and)f(placed)189 2553 y(in)h(the)f Fm(j)p Fq(th)g(blo)q(c)o(k)h (of)f(the)g(bu\013er)g Fl(recvbuf)p Fq(.)189 2610 y(but)g(should)h (read)189 2666 y(The)g(blo)q(c)o(k)i(of)e(data)g(sen)o(t)g(from)g(the)g Fm(j)p Fq(th)h(pro)q(cess)g(is)g(receiv)o(ed)g(b)o(y)g(ev)o(ery)f(pro)q (cess)h(and)g(placed)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Ff(3.7.)34 b(MINOR)16 b(CORRECTIONS)1132 b Fq(11)189 45 y(in)16 b(the)f Fm(j)p Fq(th)g(blo)q(c)o(k)h(of)f(the)g (bu\013er)g Fl(recvbuf)p Fq(.)189 216 y Fh(Discussion)o(:)189 284 y Fk(This)e(is)g(related)h(to)f(a)f(subtle)i(p)q(oin)o(t)f(in)g (usage.)18 b(The)13 b(sendbuf)h(is)f(a)g(single)g(item)f(whereas)i(the) g(recvbuf)g(is)189 334 y(an)f(\\arra)o(y")g(of)h(items.)143 473 y Fj(\017)23 b Fq(P)o(age)14 b(117,)g(lines)j(22/23)d(reads)189 529 y(MPI)h(pro)o(vides)g(sev)o(en)h(suc)o(h)f(prede\014ned)i(datat)o (yp)q(es.)189 585 y(but)e(should)h(read)189 642 y(MPI)f(pro)o(vides)g (nine)i(suc)o(h)e(prede\014ned)i(datat)o(yp)q(es.)189 870 y Fh(Discussion)o(:)189 937 y Fk(This)c(is)h(an)g(ob)o(vious)f (mistak)o(e.)k(There)e(are)f(nine)g(listed)g(on)f(line)h(28-40.)143 1076 y Fj(\017)23 b Fq(P)o(age)14 b(122,)g(lines)j(35-36)d(read)204 1132 y Fl(MPI)p 289 1132 14 2 v 16 w(OP)p 367 1132 V 16 w(FREE\()i(op\))230 1322 y Fk(IN)156 b Fl(op)541 b Fk(op)q(eration)14 b(\(handle\))189 1464 y Fq(but)h(should)h(read)204 1520 y Fl(MPI)p 289 1520 V 16 w(OP)p 367 1520 V 16 w(FREE\()g(op\))230 1710 y Fk(INOUT)63 b Fl(op)541 b Fk(op)q(eration)14 b(\(handle\))189 1892 y Fh(Discussion)o(:)189 1960 y Fk(This)j(app)q(ears)g(to)g(ha)o(v) o(e)g(b)q(een)h(a)f(mistak)o(e.)26 b(This)17 b(function)g(frees)h(the)g (function)e(p)q(oin)o(ter)i(and)f(sets)h(its)189 2010 y(v)n(alue)10 b(to)h(MPI)p 424 2010 13 2 v 15 w(OP)p 499 2010 V 15 w(NULL.)g(Th)o(us,)g(it)g(is)g(b)q(oth)g(input)g(and)f (output)i(as)f(are)g(the)g(free)h(routines)g(for)e(Groups,)189 2060 y(Comms,)g(etc.)143 2199 y Fj(\017)23 b Fq(P)o(age)14 b(125,)g(line)j(1)e(reads)189 2255 y(CALL)h(MPI)p 420 2255 14 2 v 16 w(ALLREDUCE\(sum,)f(c,)g(n,)g(MPI)p 1040 2255 V 17 w(REAL,)g(MPI)p 1300 2255 V 17 w(SUM,)g(0,)f(comm,)g(ierr\)) 189 2311 y(but)h(should)h(read)189 2368 y(CALL)g(MPI)p 420 2368 V 16 w(ALLREDUCE\(sum,)f(c,)g(n,)g(MPI)p 1040 2368 V 17 w(REAL,)g(MPI)p 1300 2368 V 17 w(SUM,)g(comm,)f(ierr\))189 2539 y Fh(Discussion)o(:)189 2607 y Fk(The)g(extra)h(argumen)o(t)e(app) q(ears)j(to)e(b)q(e)h(left)f(o)o(v)o(er)g(from)f(the)i(previous)f (example)f(with)h(MPI)p 1670 2607 13 2 v 16 w(REDUCE.)189 2656 y(This)f(is)h(a)g(mistak)o(e.)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fq(12)947 b Ff(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)143 45 y Fj(\017)23 b Fq(P)o(age)14 b(194,)g(line)j(48)e(reads)189 102 y Fl(MPI)p 274 102 14 2 v 15 w(ERRHANDLER)p 582 102 V 18 w(CREA)l(TE\(FUNCTION,)h (HANDLER,)g(IERROR\))189 158 y Fq(but)f(should)h(read)189 214 y Fl(MPI)p 274 214 V 15 w(ERRHANDLER)p 582 214 V 18 w(CREA)l(TE\(FUNCTION,)g(ERRHANDLER,)h(IERROR\))189 384 y Fh(Discussion)o(:)189 451 y Fk(This)c(is)h(a)g(small)d(t)o(yp)q (o)j(to)g(matc)o(h)f(all)f(the)j(other)f(names)f(for)h(this)g(argumen)o (t.)143 587 y Fj(\017)23 b Fq(P)o(age)14 b(196,)g(lines)j(1-2)e(reads) 204 644 y Fl(MPI)p 289 644 V 16 w(ERRHANDLER)p 598 644 V 17 w(FREE\()h(errhandler)f(\))230 832 y Fk(IN)156 b Fl(errhandler)397 b Fk(MPI)14 b(error)h(handler)f(\(handle\))189 972 y Fq(but)h(should)h(read)204 1029 y Fl(MPI)p 289 1029 V 16 w(ERRHANDLER)p 598 1029 V 17 w(FREE\()g(errhandler)f(\))230 1217 y Fk(INOUT)63 b Fl(errhandler)397 b Fk(MPI)14 b(error)h(handler)f (\(handle\))189 1398 y Fh(Discussion)o(:)189 1465 y Fk(This)c(app)q (ears)h(to)f(ha)o(v)o(e)h(b)q(een)g(a)f(mistak)o(e.)16 b(This)10 b(function)g(frees)i(the)f(function)f(p)q(oin)o(ter)g(and)h (sets)g(its)g(v)n(alue)189 1515 y(to)h(MPI)p 321 1515 13 2 v 15 w(ERRHANDLER)p 636 1515 V 14 w(NULL.)g(Th)o(us,)h(it)f(is)g (b)q(oth)h(input)f(and)g(output)h(as)f(are)h(the)g(free)g(routines)g (for)189 1564 y(Groups,)g(Comms,)e(etc.)143 1701 y Fj(\017)23 b Fq(P)o(age)14 b(203,)g(line)j(1)e(reads)189 1757 y Fl(MPI)p 274 1757 14 2 v 15 w(PCONTROL\(level\))189 1814 y Fq(but)g(should)h(read)189 1870 y Fl(MPI)p 274 1870 V 15 w(PCONTROL\(LEVEL\))189 2040 y Fh(Discussion)o(:)189 2106 y Fk(This)d(is)h(a)g(minor)e(t)o(yp)q(o.)143 2243 y Fj(\017)23 b Fq(P)o(age)14 b(210,)g(line)j(44)e(reads)189 2299 y Fd(MPI)p 266 2299 13 2 v 14 w(PENDING)189 2356 y Fq(but)g(should)h(read)189 2412 y Fd(MPI)p 266 2412 V 14 w(ERR)p 359 2412 V 14 w(PENDING)189 2582 y Fh(Discussion)o(:)189 2648 y Fk(This)d(is)h(a)g(t)o(yp)q(o.)k(See)c(page)g(197,)f(line)g(20)h (for)f(the)i(original)d(de\014nition.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Ff(3.7.)34 b(MINOR)16 b(CORRECTIONS)1132 b Fq(13)143 45 y Fj(\017)23 b Fq(P)o(age)14 b(211,)g(line)j(44)e(reads) 189 102 y Fd(MPI)p 266 102 13 2 v 14 w(DOUBLE)p 445 102 V 14 w(COMPLEX)189 158 y Fq(but)g(should)h(b)q(e)g(mo)o(v)o(ed)f(to)f (P)o(age)h(212,)f(line)j(22)d(since)j(it)e(is)h(an)f(optional)h(F)l (ortran)e(datat)o(yp)q(e.)189 274 y Fh(Discussion)o(:)189 342 y Fk(On)20 b(page)h(19,)f(lines)h(2-3,)f(it)g(clearly)h(states)g (this)f(is)h(an)f(optional)f(F)m(ortran)h(datat)o(yp)q(e.)37 b(It)21 b(is)f(listed)189 392 y(incorrectly)14 b(in)g(the)g(app)q (endix.)143 533 y Fj(\017)23 b Fq(P)o(age)14 b(220,)g(lines)j(19-20)d (reads)189 590 y Fl(int)i(double)g(MPI)p 479 590 14 2 v 16 w(Wtime\(void\))189 646 y(int)g(double)g(MPI)p 479 646 V 16 w(Wtick\(void\))189 702 y Fq(but)f(should)h(read)189 759 y Fl(double)g(MPI)p 413 759 V 16 w(Wtime\(void\))189 815 y(double)g(MPI)p 413 815 V 16 w(Wtick\(void\))189 988 y Fh(Discussion)o(:)189 1056 y Fk(This)d(w)o(as)h(a)g(mistak)o(e)e (caused)j(b)o(y)f(the)g(macros.)143 1197 y Fj(\017)23 b Fq(P)o(age)14 b(222,)g(line)j(34)e(reads)189 1254 y Fl(INTEGER)f(REQUEST,)g(COUNT,)g(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)g(T)l(A) o(G,)f(COMM,)g(REQUEST,)i(IERROR)189 1310 y Fq(but)g(should)h(read)189 1366 y Fl(INTEGER)f(COUNT,)h(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)f(T)l(A)o (G,)g(COMM,)g(REQUEST,)h(IERROR)189 1539 y Fh(Discussion)o(:)189 1607 y Fk(An)e(extra)g(argumen)o(t)f(got)g(added.)19 b(It)14 b(needs)h(to)f(b)q(e)g(remo)o(v)o(ed.)143 1748 y Fj(\017)23 b Fq(P)o(age)14 b(222,)g(line)j(38)e(reads)189 1805 y Fl(INTEGER)f(REQUEST,)g(COUNT,)g(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)g (T)l(A)o(G,)f(COMM,)g(REQUEST,)i(IERROR)189 1861 y Fq(but)g(should)h (read)189 1918 y Fl(INTEGER)f(COUNT,)h(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)f (T)l(A)o(G,)g(COMM,)g(REQUEST,)h(IERROR)189 2090 y Fh(Discussion)o(:) 189 2158 y Fk(An)e(extra)g(argumen)o(t)f(got)g(added.)19 b(It)14 b(needs)h(to)f(b)q(e)g(remo)o(v)o(ed.)143 2299 y Fj(\017)23 b Fq(P)o(age)14 b(227,)g(lines)j(19-20)d(reads)189 2356 y Fl(MPI)p 274 2356 V 15 w(INTERCOMM)p 563 2356 V 17 w(MERGE\(INTERCOMM,)i(HIGH,)f(INTRA)o(COMM,)g(IERROR\))189 2412 y(INTEGER)g(INTERCOMM,)h(INTRA)o(COMM,)f(IERROR)189 2469 y Fq(but)g(should)h(read)189 2525 y Fl(MPI)p 274 2525 V 15 w(INTERCOMM)p 563 2525 V 17 w(MERGE\(INTERCOMM,)g(HIGH,)f (NEWINTRA)o(COMM,)g(IERROR\))189 2582 y(INTEGER)g(INTERCOMM,)h (NEWINTRA)o(COMM,)f(IERROR)-32 46 y Fi(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fq(14)947 b Ff(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fh(Discussion)o(:)189 114 y Fk(This)g(mak)o(es)e(it)i(matc)o(h)f(the)i(de\014nition)e(in)h (the)g(text)h(on)f(P)o(age)g(159,)f(lines)h(34-36.)20 b(It)15 b(only)g(c)o(hanges)g(the)189 163 y(name)d(of)i(the)g(argumen)o (t.)143 304 y Fj(\017)23 b Fq(P)o(age)14 b(228,)g(line)j(46)e(reads)189 361 y Fl(MPI)p 274 361 14 2 v 15 w(ERRHANDLER)p 582 361 V 18 w(CREA)l(TE\(FUNCTION,)h(HANDLER,)g(IERROR\))189 417 y Fq(but)f(should)h(read)189 474 y Fl(MPI)p 274 474 V 15 w(ERRHANDLER)p 582 474 V 18 w(CREA)l(TE\(FUNCTION,)g(ERRHANDLER,)h (IERROR\))189 646 y Fh(Discussion)o(:)189 715 y Fk(This)c(c)o(hange)i (mak)o(es)d(the)j(app)q(endix)f(consistan)o(t)g(with)g(c)o(hange)g(on)g (p.)k(194,)12 b(line)i(48.)143 856 y Fj(\017)23 b Fq(P)o(age)14 b(229,)g(line)j(33)e(reads)189 912 y Fl(MPI)p 274 912 V 15 w(PCONTROL\(level\))189 968 y Fq(but)g(should)h(read)189 1025 y Fl(MPI)p 274 1025 V 15 w(PCONTROL\(LEVEL\))189 1197 y Fh(Discussion)o(:)189 1266 y Fk(This)d(is)h(to)g(mak)o(e)e(the)j (language)e(binding)g(the)h(same)f(as)h(the)h(c)o(hange)f(to)g(P)o(age) f(203,)g(line)g(1.)1967 46 y Fi(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Aug 12 18:24:11 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA10465; Mon, 12 Aug 1996 18:23:53 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA20156 for mpi-core-out; Mon, 12 Aug 1996 17:24:01 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA20069; Mon, 12 Aug 1996 17:21:45 -0500 Message-Id: <199608122221.RAA20069@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov, mpi-misc@antares.mcs.anl.gov Subject: new misc chapter, MPI-2 part Date: Mon, 12 Aug 1996 17:21:42 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Mon Aug 12 16:52:49 1996 %%Pages: 16 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.08.12:1652 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 2 52 df<0F8030E040708030C038E0384038003800700070006000C00180 030006000C08080810183FF07FF0FFF00D157E9412>50 D<0FE030306018701C701C001C 00180038006007E000300018000C000E000EE00EE00EC00C401830300FE00F157F9412> I E /Fb 1 59 df<70F8F8F87005057C840D>58 D E /Fc 62 123 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0000E01C0000E01 C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0000E01C0000E01 C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01 C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C00E0180 0E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F8151D80 9C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<6060F0F0F8F868680808 08080808101010102020404080800D0C7F9C15>34 D<004000800100020006000C000C00 18001800300030007000600060006000E000E000E000E000E000E000E000E000E000E000 E000E000600060006000700030003000180018000C000C00060002000100008000400A2A 7D9E10>40 D<800040002000100018000C000C0006000600030003000380018001800180 01C001C001C001C001C001C001C001C001C001C001C001C0018001800180038003000300 060006000C000C00180010002000400080000A2A7E9E10>I<60F0F07010101010202040 80040C7C830C>44 DI<60F0F06004047C830C>I<03C00C301818 300C300C700E60066006E007E007E007E007E007E007E007E007E007E007E007E007E007 60066006700E300C300C18180C3007E0101D7E9B15>48 D<030007003F00C70007000700 070007000700070007000700070007000700070007000700070007000700070007000700 070007000F80FFF80D1C7C9B15>I<07C01830201C400C400EF00FF80FF807F807700700 0F000E000E001C001C00380070006000C00180030006010C01180110023FFE7FFEFFFE10 1C7E9B15>I<07E01830201C201C781E780E781E381E001C001C00180030006007E00030 001C001C000E000F000F700FF80FF80FF80FF00E401C201C183007E0101D7E9B15>I<00 0C00000C00001C00003C00003C00005C0000DC00009C00011C00031C00021C00041C000C 1C00081C00101C00301C00201C00401C00C01C00FFFFC0001C00001C00001C00001C0000 1C00001C00001C0001FFC0121C7F9B15>I<00F0030C06040C0E181E301E300C70007000 6000E3E0E430E818F00CF00EE006E007E007E007E007E007600760077006300E300C1818 0C3003E0101D7E9B15>54 D<4000007FFF807FFF007FFF00400200800400800400800800 00100000100000200000600000400000C00000C00001C000018000018000038000038000 038000038000078000078000078000078000078000078000030000111D7E9B15>I<03C0 0C301818300C700C600EE006E006E007E007E007E007E0076007700F300F18170C2707C7 00060006000E300C780C78187010203030C00F80101D7E9B15>57 D<60F0F0600000000000000000000060F0F06004127C910C>I<60F0F060000000000000 0000000060F0F0701010101020204080041A7C910C>I<0FE03038401CE00EF00EF00EF0 0E000C001C0030006000C000800180010001000100010001000100000000000000000000 0003000780078003000F1D7E9C14>63 D<000600000006000000060000000F0000000F00 00000F00000017800000178000001780000023C0000023C0000023C0000041E0000041E0 000041E0000080F0000080F0000180F8000100780001FFF80003007C0002003C0002003C 0006003E0004001E0004001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E061800180198007000780 0E0003801C0003801C00018038000180780000807800008070000080F0000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000700000807800008078000080 380000801C0001001C0001000E000200070004000180080000E03000001FC000191E7E9C 1E>I69 DI72 DI77 DI<003F800000E0E00003803800 07001C000E000E001C0007003C00078038000380780003C0780003C0700001C0F00001E0 F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C0 780003C0380003803C0007801C0007000E000E0007001C000380380000E0E000003F8000 1B1E7E9C20>II<07E0801C198030058070 0380600180E00180E00080E00080E00080F00000F800007C00007FC0003FF8001FFE0007 FF0000FF80000F800007C00003C00001C08001C08001C08001C0C00180C00180E00300D0 0200CC0C0083F800121E7E9C17>83 D<7FFFFFC0700F01C0600F00C0400F0040400F0040 C00F0020800F0020800F0020800F0020000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000001F800003FFFC001B1C7F9B1E>II87 D<08081010202040404040808080808080B0B0F8F878 7830300D0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C0001FC000F 1C00381C00701C00601C00E01C40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E0 00E000E000E000E000E00060007004300418080C3007C00E127E9112>I<003F00000700 00070000070000070000070000070000070000070000070000070003E7000C1700180F00 300700700700600700E00700E00700E00700E00700E00700E00700600700700700300700 180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006E006FFFEE000E000 E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E061E0E0C0E000E 000E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C 00381C00381C001818001C38000C300013C0001000003000001800001FF8001FFF001FFF 803003806001C0C000C0C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C00180000000000000000 00000000000000FC001C001C001C001C001C001C001C001C001C001C001C001C001C001C 001C001C00FF80091D7F9C0C>I<00C001E001E000C00000000000000000000000000000 0FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E060E0F0C0F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180E001 C0E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C0003F0001212 7F9115>II<03C1000C3300180B00300F00700700700700 E00700E00700E00700E00700E00700E00700600700700700300F00180F000C370007C700 000700000700000700000700000700000700000700003FE0131A7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000F880388018C018C018E010D060 8FC00D127F9110>I<04000400040004000C000C001C003C00FFE01C001C001C001C001C 001C001C001C001C001C101C101C101C101C100C100E2003C00C1A7F9910>IIII<7F8FF00F03800F030007020003840001C80001 D80000F00000700000780000F800009C00010E00020E000607000403801E07C0FF0FF815 12809116>II<7FFC70386038407040F040E041C003C003 8007000F040E041C043C0C380870087038FFF80E127F9112>I E /Fd 15 119 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D<00038000000380000007C0000007C0000007C000000FE000000FE000001FF000001BF0 00001BF0000031F8000031F8000061FC000060FC0000E0FE0000C07E0000C07E0001803F 0001FFFF0003FFFF8003001F8003001F8006000FC006000FC00E000FE00C0007E0FFC07F FEFFC07FFE1F1C7E9B24>65 D68 D<0FF8001C1E003E0F803E07 803E07C01C07C00007C0007FC007E7C01F07C03C07C07C07C0F807C0F807C0F807C0780B C03E13F80FE1F815127F9117>97 D<03FC000E0E001C1F003C1F00781F00780E00F80000 F80000F80000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E 9115>99 D<01FC000F07001C03803C01C07801C07801E0F801E0F801E0FFFFE0F80000F8 0000F800007800007C00603C00601E00C00F038001FC0013127F9116>101 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F001F 001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D108 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I114 D<1FD830786018E018E018F000FF807FE07FF01F F807FC007CC01CC01CE01CE018F830CFC00E127E9113>I<030003000300030007000700 0F000F003FFCFFFC1F001F001F001F001F001F001F001F001F001F0C1F0C1F0C1F0C0F08 079803F00E1A7F9913>II< FFC1FCFFC1FC1F00601F80E00F80C00FC0C007C18007C18003E30003E30001F60001F600 01FE0000FC0000FC0000780000780000300016127F9119>I E /Fe 50 123 df<00E001C0038007000E000E001C001C003800380038007000700070007000E0 00E000E000E000E000E000E000E000E000E000E000E00070007000700070003800380038 001C001C000E000E000700038001C000E00B2A7E9E10>40 DI<018001C0018001806186F99F7DBE1FF807E007E01FF87DBEF99F6186 0180018001C0018010127E9E15>I44 D46 D<03C00FF01FF83C3C381C700E700E700EE007E007E007E007 E007E007E007E007E007E007E007E007E007700E700E700E381C3C3C1FF80FF007E0101D 7E9B15>48 D<010007003F00FF00C7000700070007000700070007000700070007000700 07000700070007000700070007000700070007000700FFF8FFF80D1C7C9B15>I<07E01F F83C3C700E700EE007E007E007E007700E700E3C3C1FF807E01FF83C3C700E700EE007E0 07E007E007E007E007700E781E3C3C1FF807E0101D7E9B15>56 D<7FFFFFC0FFFFFFE000 00000000000000000000000000000000000000000000000000000000000000FFFFFFE07F FFFFC01B0C7E8F20>61 D<001C0000003E0000003E0000002E0000006700000067000000 E7800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F00007 00F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038 001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F00001E00003C00003C00 00780000780000780000F00000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800003C00003C00001E00000F000807801803C07800FFF0003F80151F 7D9D1B>IIII<00 3F8001FFF003C0F80780380F00181E00003C00003C0000780000780000780000F00000F0 0000F00000F00000F00000F00000F007F8F007F8F000387800387800387800383C00383C 00381E00380F003807803803C0F801FFF0003F80151F7D9D1C>III75 DIII<003F000001 FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F00780007807800078078 000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C078000780780007807C000F803C000F003E001F001F003E000F807C0007C0F80003 FFF00001FFE000003F00001A1F7E9D1F>II82 D<03F8000FFE001C0F00 380700700300600000E00000E00000E00000E00000F000007800007F00003FE0001FFC00 07FE0001FF00001F800007800003C00003C00001C00001C00001C00001C0C00180E00380 F007007C0E001FFC0007F000121F7E9D17>III87 D89 D<7FFFF07FFFF00001E00003E00003C00007C000 0780000F00001F00001E00003E00003C0000780000F80000F00001F00001E00003C00007 C0000780000F80000F00001E00003E00003C00007C0000780000FFFFF0FFFFF0141D7E9C 19>I<0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7F DC3F1C0E127E9114>97 D<07E00FF81FFC3C1C70047000E000E000E000E000E000E00070 0070043C1C1FFC0FF807E00E127E9112>99 D<000E000E000E000E000E000E000E000E00 0E000E000E0F8E1FEE3FFE7C3E700E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3F FE1FEE0F8E0F1D7E9C15>I<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000 700070043C1C3FFC1FF807E00E127E9112>I104 DI107 DIII<03F0000FFC001FFE003C0F0078 0780700380E001C0E001C0E001C0E001C0E001C0F003C07003807807803C0F001FFE000F FC0003F00012127F9115>II114 D<1FC03FF07FF0F030E000E000F0007F003FC01FE000F0003800388038 F078FFF07FE01FC00D127F9110>I<1C001C001C001C001C001C00FFE0FFE01C001C001C 001C001C001C001C001C001C001C001C001C001C201FF00FF007C00C187F970F>III121 D<7FFC7FFC7FFC007800F000E001E003C0038007000F001E001C003C007800FFFCFFFCFF FC0E127F9112>I E /Ff 45 123 df46 D<01F00007FC000FFE001F1F003C07803C07807803C07803C07803C07001C0F001E0F001 E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001 E0F001E07803C07803C07803C07803C03C07803C07801F1F000FFE0007FC0001F0001323 7EA118>48 D<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFFFFFF FFFF10227CA118>I<03F0000FFC001FFE003C1F003007807007C06003C0E003E0C001E0 4001E04001E00001E00001E00001E00003C00003C0000780000780000F00001E00003C00 00780000F00001E00001C0000380000700000E00001C0000380000700000FFFFE0FFFFE0 FFFFE013227EA118>I<01F00007FC001FFF003E0F003807807003C02003C02003C00003 C00003C00003C0000780000780000F00001E0003FC0003F80003FE00000F000007800003 C00003C00001E00001E00001E00001E00001E08001E0C003C0E003C07007803C0F801FFF 000FFC0003F00013237EA118>I<001F00001F00002F00002F00006F0000EF0000CF0001 CF0001CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F003C0F00780F0078 0F00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00000F00000F0000 0F00000F0015217FA018>I<3FFF803FFF803FFF803C00003C00003C00003C00003C0000 3C00003C00003C00003C00003CF8003FFE003FFF003F0F803E07803C03C03803C00001E0 0001E00001E00001E00001E00001E00001E04003C04003C0E003C07007807C1F003FFE00 0FFC0003F00013227EA018>I<007E0001FF0003FF0007C1000F00001E00001E00003C00 003C000078000078000078F800F1FE00F7FF00FF0F80FC0780F803C0F803C0F801C0F001 E0F001E0F001E0F001E0F001E07001E07001E07801E07803C03803C03C03801C07801F0F 000FFE0007FC0001F00013237EA118>II<01F00007FC000FFE001E0F003C07803C07807803C0 7803C07803C07803C07803C03803803C07801E0F000F1E0007FC0003F8000FFE001E0F00 3C07807803C07803C0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C03C0780 3E0F801FFF0007FC0001F00013237EA118>I<01F00007FC000FFE001E0F003C07003807 807803807803C0F003C0F001C0F001E0F001E0F001E0F001E0F001E0F001E07003E07803 E07803E03C07E03E1FE01FFDE00FF1E003E3C00003C00003C0000380000780000700000F 00101E00183C003FF8001FF00007C00013237EA118>I<001F0000001F0000003F800000 3F8000003B8000007BC0000073C0000071C00000F1E00000F1E00000E0E00001E0F00001 E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C000F001E000F 001E000FFFFE001FFFFF001FFFFF001C000F003C0007803C00078038000780780003C078 0003C0700003C0F00001E0F00001E0E00001E01B237EA220>65 D<000FF000007FFC0000 FFFF0001F01F0003C00700078000000F0000001E0000003E0000003C0000003C00000078 0000007800000078000000F0000000F0000000F0000000F0000000F0000000F0000000F0 000000F0000000F00000007800000078000000780000003C0000003C0000003E0000001E 0000000F0000000780008003C0038001F00F8000FFFF00007FFC00000FF00019257DA31F >67 DII<000FF000003FFE0000FFFF8001F80F8003E00380078000000F00 00001E0000001E0000003C0000003C000000780000007800000078000000F0000000F000 0000F0000000F0000000F0000000F0000000F000FFC0F000FFC0F000FFC0780003C07800 03C0780003C03C0003C03C0003C01E0003C01E0003C00F0003C0078003C003E003C001F8 07C000FFFFC0003FFF00000FF8001A257DA321>71 D73 D77 D<001FC000007FF00001FFFC0003F07E0007C01F000F800F801F0007C01E0003 C03C0001E03C0001E0780000F0780000F0780000F070000070F0000078F0000078F00000 78F0000078F0000078F0000078F0000078F0000078F0000078780000F0780000F0780000 F07C0001F03C0001E03E0003E01E0003C01F0007C00F800F8007C01F0003F07E0001FFFC 00007FF000001FC0001D257DA324>79 DI82 D<00FE0003FFC007FFE00F 81E01E00603C00003C00007800007800007800007800007800007C00003C00003F00001F C0000FFC0007FF0001FF80003FC00007E00001F00000F00000F800007800007800007800 0078000078000078C000F0E000F0F801E07E07C03FFF800FFF0001FC0015257EA31B>I< FFFFFFF0FFFFFFF0FFFFFFF0000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00001C237EA2 21>I<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF0 0FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C 0180780000780000700000F00000F00000F00000F00000F00000F0000078000078000078 00003C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC01FFFC0 3F0FC03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0 7803C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC001FFE 003E0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F000007000 007800007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01FF03 C0038007800780078007800780078007800780FFF8FFF8FFF80780078007800780078007 800780078007800780078007800780078007800780078007800780102380A20F>I<01F0 7807FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F000FFE 001FFC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8F000 78F00078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>I II<00F000F000F000F000000000000000000000000000000000000000F000 F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000 F000F000F000F000F000F000F000F080F0E3E0FFE07FC01F000C2D83A20D>I108 DII<01FC0007FF000FFF801F07C03C01E07800F07800F0700070F00078F00078 F00078F00078F00078F000787800F07800F07C01F03E03E01F07C00FFF8007FF0001FC00 15167F9518>II<03E3C007FBC00FFFC01F0FC03C07C07C03C07803C0F803C0F003C0F003C0F003C0F0 03C0F003C0F003C0F803C07803C07803C03C07C03F0FC01FFFC00FFBC003E3C00003C000 03C00003C00003C00003C00003C00003C00003C00003C00003C012207E9519>II<07F01FFC3FFE3C0E7806780078007C003F003FF01FF80FFC 01FE001F000F000F000FC00FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F 000F00FFF8FFF8FFF80F000F000F000F000F000F000F000F000F000F000F000F000F000F 000F080F1C07FC07F803E00E1C7F9B12>II121 D<7FFFC07FFFC07FFFC0000F80000F00001E00003E00007C0000780000F00001F00003E0 0003C0000780000F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012167F9515 >I E /Fg 22 90 df<3078F8787005057C840D>46 D<007F000183C00201E00400F00700 F00F00F00F01F00F01F00001E00001E00003C0000380000700000E0000F800000E000007 000007800007C00003C00007C03007C07807C0F807C0F807C0F00780800F00400E00201C 0018780007E000141F7D9D17>51 D<000F8000704000C0200180E00301E00701E00E00C0 1E00001C00003C000038000078F800790E007A07007C0300F80380F80380F003C0F003C0 F003C0F003C0F00780E00780E00780E00700E00F00600E00701C0030180018700007C000 131F7C9D17>54 D<007E0001C3000301800601C00E01C01C00C03C00E03C00E03C01E078 01E07801E07801E07801E07803E07803E03803C03807C01C0BC00C13C003E38000078000 0780000700000E00600E00F01C00F01800E0300080600041C0003F0000131F7C9D17>57 D<0000100000001800000038000000380000007800000078000000FC000001BC0000013C 0000033C0000023C0000063C0000043E0000081E0000081E0000101E0000101E0000201E 0000200F0000400F0000400F0000FFFF0000800F0001000F800100078002000780020007 8004000780040007800C0007C03E0007C0FF807FFC1E207E9F22>65 D<07FFFF00007C01C0003C01E0003C00F0007800F8007800F8007800F8007800F8007800 F8007800F000F001F000F001E000F003C000F00F8000FFFE0000F00F0001E007C001E003 C001E003E001E001E001E001E001E001E003C001E003C003E003C003E003C003C003C007 C003C00F8007800F0007803E00FFFFF0001D1F7E9E20>I<0001F808000E061800380138 007000F801E0007803C0007007800030078000300F0000301F0000301E0000303E000020 3C0000007C0000007C0000007C0000007C000000F8000000F8000000F8000000F8000000 F80000007800004078000080780000803C0000803C0001001C0002000E00020006000C00 0300100001C0E000003F00001D217B9F21>I<07FFFF00007C01E0003C00F0003C007800 78003C0078003C0078001E0078001E0078001E0078001F00F0001F00F0001F00F0001F00 F0001F00F0001F00F0001F01E0001E01E0003E01E0003E01E0003E01E0003C01E0007C03 C0007803C000F003C000F003C001E003C003C003C0078007800F0007803C00FFFFE00020 1F7E9E23>I<07FFFFF8007C0078003C0038003C00180078001800780008007800080078 0008007800080078080800F0100000F0100000F0100000F0300000FFF00000F0700001E0 200001E0200001E0200001E0200001E0000801E0001003C0001003C0001003C0002003C0 002003C0006003C000C0078001C0078007C0FFFFFF801D1F7E9E1F>I<0001FC04000F03 0C003C009C0070007C00E0003C01C0003803800018078000180F0000181F0000181E0000 183E0000103C0000007C0000007C0000007C0000007C000000F8000000F8000000F8007F FCF80003E0780001E0780001E0780003C0780003C03C0003C03C0003C01C0003C00E0007 C007000B800380118001E06080003F80001E217B9F24>71 D<07FFC7FFC0007C00F80000 3C007800003C007800007800F000007800F000007800F000007800F000007800F0000078 00F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000F001E00001E003 C00001E003C00001E003C00001E003C00001E003C00001E003C00003C007800003C00780 0003C007800003C007800003C007800003C007800007800F000007C00F8000FFF8FFF800 221F7E9E22>I<07FFE0007C00003C00003C000078000078000078000078000078000078 0000F00000F00000F00000F00000F00000F00001E00001E00001E00001E00001E00001E0 0003C00003C00003C00003C00003C00003C00007800007C000FFFC00131F7F9E10>I<07 FFF000007E0000003C0000003C0000007800000078000000780000007800000078000000 78000000F0000000F0000000F0000000F0000000F0000000F0000001E0000001E0000001 E0000001E0000001E0008001E0010003C0010003C0010003C0030003C0020003C0060003 C0060007801E0007807C00FFFFFC00191F7E9E1C>76 D<07FC0000FFC0007C0000F80000 3C00017800003C00017800004E0002F000004E0002F000004E0004F000004E0004F00000 4E0008F000004E0008F00000870011E00000870011E00000870021E00000870021E00000 870041E00000838041E00001038083C00001038083C00001038103C00001038203C00001 01C203C0000101C403C0000201C40780000201C80780000201C80780000201D007800002 00F00780000600E00780000600E00F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I< 07FC01FFC0003E003E00003E001800003E001800004F001000004F001000004780100000 478010000043C010000043C010000083C020000081E020000081E020000080F020000080 F020000080782000010078400001007C400001003C400001003C400001001E400001001E 400002000F800002000F800002000F800002000780000200078000060003800006000300 000F00010000FFE0010000221F7E9E22>I<0003F800001E0E000038070000E0038001C0 01C003C001E0078000E00F0000F00F0000F01E0000F01E0000F83E0000F83C0000F87C00 00F87C0000F87C0000F87C0000F8F80001F0F80001F0F80001F0F80001F0F80003E07800 03E0780003C0780007C07C0007803C000F003C001E001E001C000E0038000700F00003C3 C00000FE00001D217B9F23>I<07FFFF00007C03C0003C01E0003C00F0007800F0007800 F8007800F8007800F8007800F8007800F000F001F000F001E000F003C000F0078000F00F 0000FFF80001E0000001E0000001E0000001E0000001E0000001E0000003C0000003C000 0003C0000003C0000003C0000003C000000780000007C00000FFFC00001D1F7E9E1F>I< 07FFFC00007C0700003C03C0003C01E0007801E0007801F0007801F0007801F0007801F0 007801E000F003E000F003C000F0078000F00F0000F03C0000FFF00001E0300001E03800 01E01C0001E01C0001E01C0001E01E0003C03E0003C03E0003C03E0003C03E0003C03E00 03C03E0207803E0407C01F04FFFC0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E00080E00080E00000F00 000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E00000F00000F04000 E04000E04000E04000E06000C0600180E00380F80300C60C0081F80016217D9F19>I<3F FFFFF03C0780F03007803060078030400F0010400F0010C00F0010800F0010800F001080 0F0010001E0000001E0000001E0000001E0000001E0000001E0000003C0000003C000000 3C0000003C0000003C0000003C0000007800000078000000780000007800000078000000 78000000F0000001F800007FFFE0001C1F7A9E21>II89 D E /Fh 35 123 df<387CFEFEFE7C3807077C860F>46 D<00E00001E0000FE000FFE000 F3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000 03E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000FFFF80 FFFF80111D7C9C1A>49 D<07F0001FFE00383F007C1F80FE0FC0FE0FC0FE0FE0FE07E07C 07E03807E0000FE0000FC0000FC0001F80001F00003E0000780000F00000E00001C00003 80600700600E00601C00E01FFFC03FFFC07FFFC0FFFFC0FFFFC0131D7D9C1A>I<01FC00 07FF000E0F801E0FC03F07E03F07E03F07E03F07E01E0FC0000FC0000F80001F0001FC00 01FC00000F800007C00003E00003F00003F83803F87C03F8FE03F8FE03F8FE03F0FC03F0 7807E03C0FC01FFF8003FC00151D7E9C1A>I<0001C00003C00007C00007C0000FC0001F C0003BC00073C00063C000C3C00183C00383C00703C00E03C00C03C01803C03803C07003 C0E003C0FFFFFEFFFFFE0007C00007C00007C00007C00007C00007C000FFFE00FFFE171D 7F9C1A>I<3803803FFF803FFF003FFE003FFC003FF0003F800030000030000030000030 000033F80037FE003C1F00380F801007C00007C00007E00007E07807E0FC07E0FC07E0FC 07E0FC07C0780FC0600F80381F001FFC0007F000131D7D9C1A>I<003F0001FFC007E0E0 0F81E01F03F01E03F03E03F07C03F07C01E07C0000FC1000FCFF00FDFFC0FD03E0FE01F0 FE01F0FC01F8FC01F8FC01F8FC01F87C01F87C01F87C01F83C01F03E01F01E03E00F07C0 07FF8001FE00151D7E9C1A>I<6000007FFFF87FFFF87FFFF07FFFE07FFFC0E00180C003 00C00300C00600000C0000180000380000380000780000700000F00000F00001F00001F0 0001F00001F00003F00003F00003F00003F00003F00003F00001E00000C000151E7D9D1A >I<01FC000FFF001F07803E03C07C03E07C01E0FC01F0FC01F0FC01F0FC01F8FC01F8FC 01F8FC01F87C03F87C03F83E05F81FFDF807F9F80041F80001F03C01F07E01F07E03E07E 03E07E07C03C0780381F001FFC0007F000151D7E9C1A>57 D<07F8001FFE00381F80780F 80FC0FC0FC0FC0FC0FC0780FC0301F80001F00003E00007C0000700000E00000E00000C0 0000C00000C00000C00000C00000C00000000000000000000000000001C00003E00007F0 0007F00007F00003E00001C00012207D9F19>63 D<0007FC02003FFF0E00FE03DE03F000 FE07E0003E0FC0001E1F80001E3F00000E3F00000E7F0000067E0000067E000006FE0000 00FE000000FE000000FE000000FE000000FE000000FE0000007E0000007E0000067F0000 063F0000063F00000C1F80000C0FC0001807E0003803F0007000FE01C0003FFF800007FC 001F1F7D9E26>67 D69 D73 D77 D80 D<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003E0007FE007FBE01F03E0 3C03E07C03E0F803E0F803E0F803E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007 FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000FC0000FC00007C 00007E00003E00603F00C01F81C007FF0001FC0013147E9317>I<01FE0007FF800F83C0 1E01E03E00F07C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C0000 3E00181E00180F807007FFE000FF8015147F9318>101 D<001F8000FFC001F3E003E7E0 03C7E007C7E007C3C007C00007C00007C00007C00007C000FFFC00FFFC0007C00007C000 07C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C000 07C00007C0003FFC003FFC0013207F9F10>I<01FC3C07FFFE0F079E1E03DE3E03E03E03 E03E03E03E03E03E03E01E03C00F07800FFF0009FC001800001800001C00001FFF800FFF F007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78003C78003C3F01F80FFFE001FF 00171E7F931A>I<1C003E007F007F007F003E001C00000000000000000000000000FF00 FF001F001F001F001F001F001F001F001F001F001F001F001F001F001F001F001F00FFE0 FFE00B217EA00E>105 D<0038007C00FE00FE00FE007C00380000000000000000000000 0001FE01FE003E003E003E003E003E003E003E003E003E003E003E003E003E003E003E00 3E003E003E003E003E303E783EFC3CFC7C78783FF01FC00F2A83A010>I108 DI< FE0FC0FE3FE01E61F01EC0F81E80F81F00F81F00F81F00F81F00F81F00F81F00F81F00F8 1F00F81F00F81F00F81F00F81F00F81F00F8FFE3FFFFE3FF18147D931D>I<01FF0007FF C01F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC007EFC007E7C00 7C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>II<01F81807FE381F87783F01F83E01F87E00F87C00F8FC00F8 FC00F8FC00F8FC00F8FC00F8FC00F87C00F87E00F87E00F83F01F81F87F80FFEF803F8F8 0000F80000F80000F80000F80000F80000F80000F80007FF0007FF181D7E931C>II<0FE63FFE701E60 0EE006E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81EFFFCC7F010147E 9315>I<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F800F80 0F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>II120 D<3FFFE03FFFE03C07C0380F80701F80603F00603E00607C0000F800 01F80003F00003E06007C0600F80601F80E03F00C03E01C07C03C0FFFFC0FFFFC013147F 9317>122 D E /Fi 33 120 df<00003F03E00000C386700001878CF00003879CF00003 031860000700380000070038000007003800000E003800000E007000000E007000000E00 700000FFFFFF80001C007000001C00E000001C00E000001C00E000001C00E000003800E0 00003801C000003801C000003801C000003801C000007001C00000700380000070038000 00700380000070038000006003800000E007000000E007000000E007000000E007000000 C006000001C00E000001C00E000031860C0000798F180000F31E100000620C6000003C07 C000002429829F1C>11 D<00003FE00000E0100001803800038078000300780007003000 0700000007000000070000000E0000000E0000000E000000FFFFE0000E00E0001C01C000 1C01C0001C01C0001C01C0001C0380003803800038038000380380003807000038070000 7007000070071000700E2000700E2000700E2000E00E2000E0064000E0038000E0000000 C0000001C0000001C000003180000079800000F3000000620000003C0000001D29829F1A >I<0E1F3F3F1D0102020404081020C0080E779F0E>39 D<7FF0FFE07FE00C037D8A10> 45 D<70F8F8F0E005057B840E>I<00000040000000C00000018000000180000003000000 0300000006000000060000000C0000001800000018000000300000003000000060000000 C0000000C0000001800000018000000300000003000000060000000C0000000C00000018 00000018000000300000003000000060000000C0000000C0000001800000018000000300 000003000000060000000C0000000C000000180000001800000030000000300000006000 0000C0000000C0000000800000001A2D7FA117>I<000F800030E000E07001C070038030 0380380700380F00780F00780E00781E00781E00703C00F03C00F03C00F03C00F07801E0 7801E07801E07801C07003C0F003C0F00380F00780F00700700700700E00701C00303800 1870000FC000151F7C9D17>I<000200020006000E003C00DC031C001C00380038003800 38007000700070007000E000E000E000E001C001C001C001C003800380038003800780FF F80F1E7B9D17>I<0007C0001C200030200060E000C1E00181E00380C00700000F00000E 00001E00001E78001D84003E06003E07003C07007C0780780780780780780780700F0070 0F00F00F00F00E00F01E00701C00601C0070380030700010C0000F8000131F7B9D17>54 D<070F1F1F0E0000000000000000000070F8F8F0E008147B930E>58 D<00000200000006000000060000000E0000001E0000001E0000003F0000002F0000004F 0000004F0000008F0000010F0000010F0000020F0000020F0000040F00000C0F0000080F 0000100F0000100F0000200F80003FFF800040078000C007800080078001000780010007 800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C0004003C00 040078080800780800007808000078180000F0300000FFF00000F0300000F0300001E020 0001E0200001E0200001E0001003C0002003C0002003C0004003C0004007800080078001 8007800100078007000F001F00FFFFFE001F1F7D9E1F>69 D<01FFFF00001E03C0001E00 E0001E0070001E0078003C0078003C0078003C0078003C0078007800F0007800F0007801 E0007801C000F0070000F01E0000FFF00000F0380001E01C0001E01E0001E00E0001E00F 0003C01E0003C01E0003C01E0003C01E0007803C0007803C0807803C0807803C100F801C 10FFF00C20000007C01D207D9E21>82 D<0007E040001C18C0003005800060038000C003 8001C00180018001000380010003800100038001000380000003C0000003C0000003F800 0001FF800001FFE000007FF000001FF0000001F800000078000000780000003800000038 0020003800200038002000300060007000600060006000E0007000C000E8038000C60600 0081F800001A217D9F1A>I<00F1800389C00707800E03801C03803C0380380700780700 780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F0700 13147C9317>97 D<007E0001C1000300800E07801E07801C07003C020078000078000078 0000F00000F00000F00000F00000F0000070010070020030040018380007C00011147C93 15>99 D<0000780003F80000700000700000700000700000E00000E00000E00000E00001 C00001C000F1C00389C00707800E03801C03803C0380380700780700780700780700F00E 00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F070015207C9F17>I<00 7C01C207010E011C013C013802780C7BF07C00F000F000F000F000700070017002380418 3807C010147C9315>I<00007800019C00033C00033C000718000700000700000E00000E 00000E00000E00000E0001FFE0001C00001C00001C00001C000038000038000038000038 0000380000700000700000700000700000700000700000E00000E00000E00000E00000C0 0001C00001C0000180003180007B0000F300006600003C00001629829F0E>I<003C6000 E27001C1E00380E00700E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C 03803C07003C07001C0F001C17000C2E0003CE00000E00000E00001C00001C00301C0078 3800F0700060E0003F8000141D7E9315>I<01E0000FE00001C00001C00001C00001C000 038000038000038000038000070000070000071E000763000E81800F01C00E01C00E01C0 1C03801C03801C03801C0380380700380700380700380E10700E20700C20701C20700C40 E00CC060070014207D9F17>I<00C001E001E001C000000000000000000000000000000E 003300230043804300470087000E000E000E001C001C001C003840388030807080310033 001C000B1F7C9E0E>I<03C01FC0038003800380038007000700070007000E000E000E00 0E001C001C001C001C0038003800380038007000700070007100E200E200E200E2006400 38000A207C9F0C>108 D<1C0F80F0002630C318004740640C004780680E004700700E00 4700700E008E00E01C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C 01C038001C01C0708038038071003803806100380380E100380380620070070066003003 00380021147C9325>I<1C0F802630C04740604780604700704700708E00E00E00E00E00 E00E00E01C01C01C01C01C01C01C03843803883803083807083803107003303001C01614 7C931A>I<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F0 03C0F003C0F003C0F00780F00700700F00700E0030180018700007C00013147C9317>I< 01C1E002621804741C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C 01C03C01C0380380780380700380E003C1C0072380071E000700000700000E00000E0000 0E00000E00001C00001C0000FFC000171D809317>I<1C1E002661004783804787804707 804703008E00000E00000E00000E00001C00001C00001C00001C00003800003800003800 0038000070000030000011147C9313>114 D<00FC030206010C030C070C060C000F800F F007F803FC003E000E700EF00CF00CE008401020601F8010147D9313>I<018001C00380 03800380038007000700FFF007000E000E000E000E001C001C001C001C00380038003800 3820704070407080708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C0 4701C08703800E03800E03800E03801C07001C07001C07001C07101C0E20180E20180E20 1C1E200C264007C38014147C9318>I<0E03803307802307C04383C04301C04700C08700 800E00800E00800E00801C01001C01001C01001C02001C02001C04001C04001C08000E30 0003C00012147C9315>I<0E00C1C03300E3C02301C3E04381C1E04301C0E04701C06087 0380400E0380400E0380400E0380401C0700801C0700801C0700801C0701001C0701001C 0602001C0F02000C0F04000E13080003E1F0001B147C931E>I E /Fj 58 122 df<007F07F001FF1FF003FF3FF007807800070070000F00F0000F00F0000F 00F0000F00F0000F00F0000F00F0000F00F000FFF8FF80FFF8FF80FFF8FF800F00F0000F 00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F 00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0001C20809F1B>11 D<00F8F001F8F003F8F00780F00700F00F00F00F00F00F00F00F00F00F00F00F00F00F00 F0FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00 F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F01420809F18>13 D<007000E001C00380078007000E001E001E003C003C003C0078007800780078007000F0 00F000F000F000F000F000F000F000F000F000F000F000700078007800780078003C003C 003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DII<00C001 C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17>49 D<07F0000FFC001FFE00383F00700F00600780E00780E003C04003C04003C00003C00003 C00003C0000780000780000F00000E00001C00003C0000780000E00001C0000380000700 000E00001C0000380000700000FFFFC0FFFFC0FFFFC0121F7E9E17>I61 D<001F0000001F0000003F8000003B8000003B8000007BC0 000073C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0F00003C078 0003C078000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F 001E000F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B207F 9F1E>65 DI< 001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000 780000F00000F00000F00000F00000F00000F00000F00000F00000780000780000780000 7C00003C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D> IIII<001FE000FFF801FFFE03E03E0780 0E0F00001E00003E00003C00007C0000780000780000780000F00000F00000F00000F000 00F00000F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F00 1E07801E03E03E01FFFE00FFF8001FC017227DA01E>III<003C003C003C003C003C003C003C00 3C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C00 3C003C003CC078E078FFF07FE00FC00E217E9F15>IIIII<003F 000000FFC00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F007800 07807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F00003C0F80007C07800078078000780780007803C000F003C000F001E00 1E001F003E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC000 03FFF00007E1F8000F807C001F003E001E001E003C000F003C000F007800078078000780 78000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE00 0F83FC0007E1F80003FFF00000FFF800003F780000007C0000003E0000001E0000001F00 00000F801A277DA021>II<01FC0007FF800FFFC01F03C03C00C03C0000780000780000780000780000 7800007C00003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E0 0001E00001E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F800 13227EA019>IIIII<780007807C000F003E001F001E001E000F003C000F807C000780 780003C0F00003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E 0000003F0000007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C078000780 3C000F003E001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E> III<07E03FF87FFC701E401F000F00 0F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000 F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F8 0014147F9317>II<03C7800FF780 1FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F80780 7807807C0F803E1F801FFF800FF78007C780000780000780000780000780000780000780 000780000780000780111D7E9317>II<07F01FFC3FFC780C78 00780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311 >I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E00 1E001E001E001E001E001E201FF00FF007C00C1A7F9910>II< F003C0F003C07803807807807807803C0F003C0F003C0F001E0E001E1E001E1E000F1C00 0F3C000F3C0007380007380007B80003F00003F00001E00012147F9315>I121 D E /Fk 16 117 df<020408103020604040 C0C0C0C0C0C0C0C0404060203010080402071A7F920C>40 D<8040201018080C04040606 06060606060604040C081810204080071A7E920C>I<1F00318060C04040C060C060C060 C060C060C060C060C060404060C031801F000B107F8F0F>48 D<0C003C00CC000C000C00 0C000C000C000C000C000C000C000C000C000C00FF8009107E8F0F>I<1F00618040C080 60C0600060006000C00180030006000C00102020207FC0FFC00B107F8F0F>I<1F002180 60C060C000C0008001800F00008000400060C060C060804060801F000B107F8F0F>I<03 00030007000F000B001300330023004300C300FFE003000300030003001FE00B107F8F0F >I<20803F002C002000200020002F0030802040006000600060C06080C061801F000B10 7F8F0F>I<0780184030C060C06000C000CF00F080E040C060C060C060406060C030801F 000B107F8F0F>I<40007FE07FC08080808001000200040004000C000800080018001800 1800180018000B117E900F>I<1F00318060C060C060C071803F000F00338061C0C060C0 60C060404060801F000B107F8F0F>I<1F00318060C0C040C060C060C06040E021E01E60 0060004060C0608043003E000B107F8F0F>I<1F2060E0006080208020800060003E001F 8000C00060002080208020C040E0C09F000B117E9011>83 D<1F00318060C0C0C0FFC0C0 00C000C000604030801F000A0B7F8A0E>101 D112 D<10103030FE303030303032 3232321C070F7F8E0C>116 D E /Fl 3 64 df21 D62 D<00040000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000FFFFFFE0FFFFFFE01B1C7C9B23>I E /Fm 73 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F870051C77 9B18>33 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01C E3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E0 7070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F 001E001C00380038007000700070007000E000E000E000E000E000E000E000E000700070 0070007000380038001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070007000 70007000700070007000E000E000E000E001C001C0038007800F001E003C007800F00060 000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007 F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011 047D8F18>I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00 001E00001E00003E00003C00007C0000780000780000F80000F00001F00001E00003E000 03C00003C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C0000 7C0000780000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F 001C07003803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000 E0E000E0E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F0 00131C7E9B18>I<01800380038007800F803F80FF80FB80438003800380038003800380 038003800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03 F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C000 01C00003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C 00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C07801C0 3001C00001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C00000E0 0000E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B18> I<1FFF803FFF803FFF803800003800003800003800003800003800003800003800003BF8 003FFE003FFF003C07801803C00001C00000E00000E06000E0F000E0F000E0E001C07003 C07C0F803FFF001FFC0003F000131C7E9B18>53 D<007E0001FF0007FF800F83C01E03C0 1C03C0380180380000700000700000E1F800E7FE00FFFF00FE0780F803C0F001C0F000E0 E000E0F000E07000E07000E07000E03801C03C03C01E07800FFF0007FE0001F800131C7E 9B18>I<3078FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<0003 00000780001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC00 007E00003F00001FC00007E00003F00001FC00007E00003F00001F800007800003001118 7D9918>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FF FFE07FFFC0130C7E9318>I<600000F00000FC00007E00003F00001FC00007E00003F000 01FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F0000 7E0000FC0000F0000060000011187D9918>I<00700000F80000F80000D80000D80001DC 0001DC0001DC00018C00038E00038E00038E00038E000306000707000707000707000707 000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18 >65 DI<00F8E003FEE007FFE00F07E01E03 E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E00000E000 00E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F800131C 7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C 00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C 01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00FFFC01F 0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E00000E0 0FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003FDC001 F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C01C01C01C0 1C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C01C01C0 1C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF 807FFF00111C7D9B18>I<7F07F0FF87F87F07F01C03C01C07801C07001C0E001C1E001C 3C001C38001C70001CF0001DF0001DF0001FB8001FB8001F1C001E1C001C0E001C0E001C 07001C07001C03801C03801C01C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00700E00700E00700E00 700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C0 1CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C0DC0 1C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F007007 00F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C 7D9B18>II<0FF8003FFE007FFF00780F00 700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00 001C00001E00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF001C0F 801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F 001C07001C03801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F87F00 F0161C7F9B18>I<03F3801FFF803FFF807C0F80700780E00380E00380E00380E0000070 00007800003F00001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E0 00E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8 E07038E07038E07038E07038007000007000007000007000007000007000007000007000 00700000700000700000700000700000700000700000700000700000700007FF0007FF00 07FF00151C7F9B18>II87 D<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C 0003DC0001F80001F80000F00000F00000700000F00000F80001F80001DC00039E00038E 00070F000707000E07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700 000F00001E00001C00003C0000780000700000F00001E00001C00003C000078000070000 0F00001E00E01C00E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF8001F1F1C00 1E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F001914819318>I<7E 3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0000FFE00 1FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001E07001C0 7803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E00007FC0 00FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780700780E0 0380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF800FFB8003 E380000380000380000380000380000380000380000380003FF8003FF8003FF8151E7E93 18>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C000038000038000038000 0380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318>I<07F7 003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE00001F006007 80E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<01800003800003 80000380000380007FFFC0FFFFC0FFFFC003800003800003800003800003800003800003 80000380000380000380400380E00380E00380E001C1C001FFC000FF80003E0013197F98 18>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318>I<7F8F F0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E00038E00038E 00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700 078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E00070700 0F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03 80070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000DC000078 0000780000780000700000700000700000F00000E00079E0007BC0007F80003F00001E00 00151E7F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000780000 7FE0001FE00007E013247E9F18>123 D<7C0000FF0000FFC00003C00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003FE000 3FE0007FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00003C000FFC000FF00007C000013247E9F18>125 D E /Fn 46 123 df<000FF01FE0003FF07FE000FFF0FFE001FFF1FFE001F003F00003E003E0 0003C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C000 07C007C00007C007C000FFFF07FF00FFFF07FF00FFFF07FF0007C007C00007C007C00007 C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C0 07C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007 C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C0 00232A7FA923>11 D34 D46 D<00180000380000F80007F800FFF800FFF800F8F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F8007FFFF07FFFF07FFFF014287BA71E>49 D<00FE0003FFC007FFE00FFFF01F03F8 3C00FC38007E78003E70003EF0001FF0001F60001F20001F00001F00001F00001F00003E 00003E00007C00007C0000F80001F00001E00003C0000780000F00001E00003C00007800 00F00001E00003C0000780000F00001E00003C00007FFFFF7FFFFF7FFFFF7FFFFF18287E A71E>I<007F000001FFC00007FFF0000FFFF8001FC1F8003E007C003C003E0078003E00 38003E0010003E0000003E0000003E0000003C0000007C000000FC000001F8000007F000 00FFE00000FFC00000FFE00000FFF0000001FC0000007C0000003E0000001F0000001F00 00000F8000000F8000000F8000000F8000000F8040000F8060001F00F0001F00F8003F00 7E007E003F81FC001FFFF8000FFFF00003FFE000007F000019297EA71E>I<0003F00000 07F0000005F000000DF000000DF000001DF0000039F0000039F0000079F0000079F00000 F1F00000F1F00001E1F00003E1F00003E1F00007C1F00007C1F0000F81F0000F81F0001F 01F0001F01F0003E01F0007C01F0007C01F000F801F000FFFFFF80FFFFFF80FFFFFF80FF FFFF800001F0000001F0000001F0000001F0000001F0000001F0000001F0000001F00000 01F0000001F00019277EA61E>I<3FFFFC3FFFFC3FFFFC3FFFFC3E00003E00003E00003E 00003E00003E00003E00003E00003E00003E00003E3F003EFFC03FFFE03FFFF03FE1F83F 807C3F003E3E003E00003E00001F00001F00001F00001F00001F00001F00001F20001F60 003E70003EF8007C7C00FC3F03F81FFFF00FFFE007FF8000FE0018287EA61E>I<000FF0 00003FFC0000FFFC0001FFFC0003F80C0007E000000FC000000F8000001F0000001E0000 003E0000003C0000007C0000007C0000007C3FE000F8FFF000F9FFF800FBFFFC00FF807E 00FF003E00FE003F00FC001F00FC001F00FC000F80F8000F80F8000F80F8000F80F8000F 8078000F807C000F807C000F807C000F003E001F003E001F001F003E001F807C000FC1FC 0007FFF80003FFF00001FFC000007F000019297EA71E>II<007F000001FFC00007FFF0000FFFF8001FC1FC003F007E003E003E007E 003F007C001F007C001F007C001F007C001F007C001F003E003E003E003E001F007C000F C1F80007FFF00003FFE00003FFE0000FFFF8001FC1FC003F007E003E003E007C001F007C 001F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F807C001F007C001F007E 003F003F007E001FC1FC000FFFF80007FFF00003FFE000007F000019297EA71E>I<007F 000001FFC00003FFE0000FFFF0000FC1F8001F007C003E007C007C003E007C001E007C00 1F00F8001F00F8001F00F8000F00F8000F80F8000F80F8000F80F8000F80F8001F807C00 1F807C001F807E003F803E007F803F00FF801FFFEF800FFFCF8007FF8F8003FE1F000000 1F0000001F0000001E0000003E0000003E0000007C0000007C000000F8001801F0001E07 E0003FFFC0001FFF80000FFE000003F8000019297EA71E>I<0001F000000003F8000000 03F800000007FC00000007BC00000007BC0000000F3E0000000F1E0000000F1E0000001F 1F0000001E1F0000001E0F0000003E0F8000003C0F8000003C078000007C07C000007807 C00000F803E00000F803E00000F003E00001F001F00001F001F00001E001F00003E000F8 0003E000F80003C000F80007FFFFFC0007FFFFFC000FFFFFFE000F80003E000F80003E00 1F00003F001F00001F001E00001F003E00000F803E00000F803C00000F807C000007C07C 000007C078000007C0F8000003E0F8000003E0232A7EA928>65 DI<0001FF00000FFFE0003FFFF8007F FFF800FE01F801F8003003F0001007C000000F8000001F8000001F0000003E0000003E00 00007E0000007C0000007C0000007C000000F8000000F8000000F8000000F8000000F800 0000F8000000F8000000F8000000F8000000F80000007C0000007C0000007C0000007E00 00003E0000003E0000001F0000001F8000000F80000007C0000003F0000401F8001C00FE 00FC007FFFFC003FFFF8000FFFE00001FF001E2C7CAA26>III73 D76 DI80 D<007FC00001FFF80007FFFE000FFFFF001FC07F003F000F007E0006007C0000007C0000 00F8000000F8000000F8000000F8000000F8000000FC0000007E0000007F0000003F8000 001FF800000FFF800007FFE00003FFF80000FFFC00000FFE000000FF0000003F0000001F 8000000F8000000FC0000007C0000007C0000007C0000007C0000007C0000007C000000F 8060000F80F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE00000FF80001A2C7D AA21>83 DI89 D<01FE000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F800 00F80000F80000F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F800F8F800F8FC 01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1D>97 DI<007FC001FFF007FFFC0FFFFC1FC07C1F00083E00007C 00007C00007C0000F80000F80000F80000F80000F80000F80000F800007C00007C00007E 00003E00001F000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<00003E00003E 00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00FC3E03FF3E07FFFE0FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003E F8003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E7C003E3E007E3F00FE1FC1FE 0FFFFE07FFBE03FF3E00FC3E172A7EA91F>I<007E0003FF8007FFC00FFFE01F83F03F00 F03E00787C00787C003878003CFFFFFCFFFFFCFFFFFCFFFFFCF80000F80000F800007800 007C00007C00003E00003F000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<00 1FC0007FC000FFC001FFC003F00003E00007C00007C00007C00007C00007C00007C00007 C00007C00007C000FFFE00FFFE00FFFE0007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C000122A7FA912>I<00F8078003FE7FC00FFFFFC0 1FFFFFC01F07C0003E03E0003E03E0007C01F0007C01F0007C01F0007C01F0007C01F000 7C01F0003E03E0003E03E0001F07C0001FFFC0003FFF80003BFE000038F8000078000000 780000003C0000003FFFC0003FFFF8001FFFFC001FFFFE003FFFFF007C007F00F8001F80 F8000F80F8000F80F8000F80FC001F807E003F003F80FE003FFFFE000FFFF80007FFF000 00FF80001A287E9A1E>II< F8F8F8F8F800000000000000000000F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8 F8F8F8F8F8F8052A7CA90E>I108 DII<007F000001FFC00007FFF0000FFFF8001FC1FC003F 007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001FC1FC000F FFF80007FFF00001FFC000007F0000191B7E9A1E>II114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8 003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFF C07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007 C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003 FFE003FFE001FF8000FC0013227FA116>IIII121 DI E /Fo 9 122 df77 D<0007FFFC000000007FFFFFC0000001FFFFFFF8000003FFFFFFFE000007FE 001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000FFF0000FFF80007FF0000FFF80 007FF8000FFF80007FF80007FF00003FFC0007FF00003FFC0003FE00003FFC0000F80000 3FFC00000000003FFC00000000003FFC00000000003FFC00000000003FFC00000007FFFF FC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC0003FFF0003FFC 0007FFC0003FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC007FF800003FFC00 7FF800003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FF F000003FFC00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC0003EFFC003FFF 0007CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFFF001FFFC0003FF 80007FF8362E7DAD3A>97 D<00001FFFC0000000FFFFF8000007FFFFFE00001FFFFFFF80 007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FFE007FF0003FFE00FFE0003FFE0 0FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF803FFC00003E007FF800000000 7FF8000000007FF800000000FFF800000000FFF800000000FFF800000000FFF800000000 FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000 7FF8000000007FF8000000007FFC000000003FFC000000003FFC000000001FFC000000F8 1FFE000000F80FFE000000F80FFF000001F007FF800003F003FFC00007E001FFE0000FC0 00FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8000000FFFFE00000001FFE0000 2D2E7CAD35>99 D<00001FFE00000001FFFFE0000007FFFFF800001FFFFFFE00007FFC07 FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00003FF00FFE00001FF01FFE0000 0FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC000007FC7FF8000007FC7FF80000 07FE7FF8000007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFF FFFEFFFFFFFFFFFCFFF800000000FFF800000000FFF800000000FFF8000000007FF80000 00007FF8000000007FFC000000003FFC000000003FFC000000003FFC0000001C1FFE0000 003E0FFE0000003E07FF0000007E07FF000000FC03FF800001F801FFC00003F0007FF000 1FE0003FFE00FFC0001FFFFFFF800007FFFFFE000000FFFFF80000000FFF80002F2E7DAD 36>101 D<00FC0001FE0003FF0007FF800FFFC01FFFE01FFFE01FFFE01FFFE01FFFE01F FFE00FFFC007FF8003FF0001FE0000FC0000000000000000000000000000000000000000 0000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFF FFFFFFFFFFFFFFFF18497CC820>105 D<007FC000FFFFC000FFFFC000FFFFC000FFFFC0 00FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FFFFFF 80FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<007FC001FFC00000FFFF C00FFFF80000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1FC07FF8000FFFFC3E003FF C00003FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE00001FFDC0000FFE00001FF FC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF00000FFF00001FFF00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF000FFFFFFC07FFF FFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B2E 7CAD42>110 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FFE01FC0001F E03F80000FE03F000007E07F000003E07F000003E0FF000003E0FF000003E0FF800003E0 FFC0000000FFF0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFFFFFC001F FFFFFF000FFFFFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FFFFF80000 1FFFF8000000FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003FCFC0000 03FCFE000003F8FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0FFFC00FF C0FFFFFFFF80FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>115 D<7FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFF C000FFFF8000FFF000000FE00000FFF800000FC00000FFF800000FC000007FFC00000F80 00007FFC00001F8000003FFC00001F0000003FFE00003F0000001FFE00003E0000001FFF 00007E0000000FFF00007C0000000FFF8000FC00000007FF8000F800000007FFC001F800 000003FFC001F000000003FFE003F000000003FFE003F000000001FFF003E000000001FF F007E000000000FFF007C000000000FFF80FC0000000007FF80F80000000007FFC1F8000 0000003FFC1F00000000003FFE3F00000000001FFE3E00000000001FFF7E00000000000F FF7C00000000000FFFFC00000000000FFFFC000000000007FFF8000000000007FFF80000 00000003FFF0000000000003FFF0000000000001FFE0000000000001FFE0000000000000 FFC0000000000000FFC00000000000007F800000000000007F800000000000003F000000 000000003F000000000000003F000000000000003E000000000000007E00000000000000 7C00000000000000FC000000001F8000F8000000003FC001F8000000007FE001F0000000 00FFF003F000000000FFF003E000000000FFF007E000000000FFF00FC000000000FFF01F 8000000000FFF03F80000000007FE07F00000000007F43FE00000000003FFFF800000000 001FFFF0000000000007FFC0000000000001FE00000000000039427EAD3F>121 D E /Fp 8 117 df<0001FF8000001FFFF000007FFFFC0000FF81FE0001FE007F0007FC 003F800FF8003FC00FF8001FE01FF8001FF03FF0001FF03FF0000FF87FF0000FF87FF000 0FFCFFF0000FFCFFF0000FFCFFF0000FFEFFF0000FFEFFF0000FFEFFF0000FFEFFF0000F FEFFF0000FFFFFF0000FFFFFF0000FFF7FF0000FFF7FF0001FFF7FF0001FFF3FF0001FFF 3FF0001FFF1FF0003FFF0FF8003FFF07F8007FFF03FC006FFF01FE01CFFF00FFEF8FFF00 7FFF0FFF000FFE0FFE0000000FFE0000000FFE0000000FFE0000000FFC0000000FFC03E0 001FFC07F0001FF80FF8001FF81FFC001FF01FFC003FF01FFC003FE01FFC003FC01FF800 7FC00FF000FF800FE001FF0007C003FE0007F01FFC0003FFFFF00001FFFFC000007FFF00 00000FF8000028397CB731>57 D<0000001FFF000030000001FFFFE000F000000FFFFFFC 01F000007FFFFFFE03F00001FFFE007F87F00003FFE0000FCFF0000FFF000003FFF0001F FC000001FFF0003FF80000007FF0007FF00000003FF000FFC00000003FF001FFC0000000 1FF003FF800000000FF007FF000000000FF00FFF0000000007F00FFE0000000007F01FFE 0000000003F01FFE0000000003F03FFC0000000003F03FFC0000000001F03FFC00000000 01F07FFC0000000001F07FF80000000001F07FF80000000000007FF8000000000000FFF8 000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800000000 0000FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF8 000000000000FFF80000000000007FF80000000000007FF80000000000007FF800000000 00007FFC0000000000F03FFC0000000000F03FFC0000000000F03FFC0000000000F01FFE 0000000000F01FFE0000000001E00FFE0000000001E00FFF0000000001E007FF00000000 03C003FF8000000003C001FFC0000000078000FFE00000000F00007FF00000001F00003F F80000003E00001FFC0000007C00000FFF000001F8000003FFE00007F0000001FFFE003F C00000007FFFFFFF000000000FFFFFFC0000000001FFFFF000000000001FFF0000003C3D 7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF0000 1FFC00FF80001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF000 07F0003FF00001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000 000FFFFFF000007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF000 3FE0003FF0007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000 FFC0003FF000FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF0 07FFFE0FFFF001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800 003FFFFE0000FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03F F00007F83FF00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFF FFFFFCFFFFFFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE000 00007FE00000007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF80000 7807FC0000F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE000 26267DA52D>101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF00 00000007FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 00000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 00000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 7FC00003FF01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF60 00FF8003FFE000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF00 00FFC003FF0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC 3FFFFFFFFFFC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00 FFFFFE03FF00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF8 03FF00001FFC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF 03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF 03FF000007FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC 03FF00000FFC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC0 03FFF001FF8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF00000000 03FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000 03FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000 FFFFFC000000FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fq 71 123 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<0020004000800100020006000C000C00180018003000 300030007000600060006000E000E000E000E000E000E000E000E000E000E000E000E000 6000600060007000300030003000180018000C000C000600020001000080004000200B2E 7DA112>40 D<800040002000100008000C00060006000300030001800180018001C000C0 00C000C000E000E000E000E000E000E000E000E000E000E000E000E000C000C000C001C0 01800180018003000300060006000C00080010002000400080000B2E7DA112>I<70F8FC FC74040404080810102040060E7C840D>44 DI<70F8F8F87005 057C840D>I<000100030003000600060006000C000C000C001800180018003000300030 00600060006000C000C000C00180018001800300030003000600060006000C000C000C00 180018001800300030003000600060006000C000C000C000102D7DA117>I<03F0000E1C 001C0E00180600380700700380700380700380700380F003C0F003C0F003C0F003C0F003 C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C07003807003807003807807 803807001806001C0E000E1C0003F000121F7E9D17>I<018003800F80F3800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 8003800380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780 F007C0F803C0F803C0F803C02007C00007C0000780000780000F00000E00001C00003800 00700000600000C0000180000300000600400C00401800401000803FFF807FFF80FFFF80 121E7E9D17>I<03F0000C1C00100E00200F00780F80780780780780380F80000F80000F 00000F00000E00001C0000380003F000003C00000E00000F000007800007800007C02007 C0F807C0F807C0F807C0F00780400780400F00200E001C3C0003F000121F7E9D17>I<00 0600000600000E00000E00001E00002E00002E00004E00008E00008E00010E00020E0002 0E00040E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E00000E0000 0E00000E00000E00000E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF800 1FE00010000010000010000010000010000010000011F000161C00180E00100700100780 0003800003800003C00003C00003C07003C0F003C0F003C0E00380400380400700200600 100E000C380003E000121F7E9D17>I<007C000182000701000E03800C07801C07803803 00380000780000700000700000F1F000F21C00F40600F80700F80380F80380F003C0F003 C0F003C0F003C0F003C07003C07003C07003803803803807001807000C0E00061C0001F0 00121F7E9D17>I<4000007FFFC07FFF807FFF8040010080020080020080040000080000 080000100000200000200000400000400000C00000C00001C00001800003800003800003 8000038000078000078000078000078000078000078000078000030000121F7D9D17>I< 03F0000C0C001006003003002001806001806001806001807001807803003E03003F0600 1FC8000FF00003F80007FC000C7E00103F00300F806003804001C0C001C0C000C0C000C0 C000C0C000806001802001001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C 00380600380700700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007 C07007C03807C0180BC00E13C003E3C0000380000380000380000700300700780600780E 00700C002018001070000FC000121F7E9D17>I<70F8F8F8700000000000000000000070 F8F8F87005147C930D>I<70F8F8F8700000000000000000000070F0F8F8780808081010 10202040051D7C930D>I<0FC0307040384038E03CF03CF03C603C0038007000E000C001 800180010003000200020002000200020002000000000000000000000007000F800F800F 8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C00000 07C0000009E0000009E0000009E0000010F0000010F0000010F000002078000020780000 20780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F0001 000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22 >65 DI<000FC040007030C001C0 09C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C0000407800 0040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F800 0000780000007C0000407C0000403C0000401C0000401E0000800E000080070001000380 020001C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E00700 01E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000F800 0000F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E07C00 01E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E004600078 1820000FE0001E217D9F24>II< FFFC0FC00780078007800780078007800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800FC0FFFC0E1F7F9E10>I76 DII<001F800000 F0F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E078 0001E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F8 0001F0F80001F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F 000F0007801E0001C0380000F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F0 0000F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C000 01C08001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F 19>I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F001080 0F0010800F0010000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000001F800007FFFE001C1F7E9E21>II87 D<7FF83FF80FE00FC007C0070003C0020001E0040001F00C0000F0080000781000007C10 00003C2000003E4000001E4000000F8000000F8000000780000003C0000007E0000005E0 000009F0000018F8000010780000207C0000603C0000401E0000801F0001800F00010007 80020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I<080410082010201040204020 804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>92 D<1FE000303000781800781C00 300E00000E00000E00000E0000FE00078E001E0E00380E00780E00F00E10F00E10F00E10 F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E3E000EC3800F01C00F00E00E00E00E 00700E00700E00780E00780E00780E00780E00780E00780E00700E00700E00E00F00E00D 01C00CC300083E0015207F9F19>I<03F80E0C1C1E381E380C70007000F000F000F000F0 00F000F00070007000380138011C020E0C03F010147E9314>I<000380003F8000038000 038000038000038000038000038000038000038000038000038003E380061B801C078038 0380380380700380700380F00380F00380F00380F00380F00380F0038070038070038038 03803807801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E00380700380700 700700700380F00380F00380FFFF80F00000F00000F00000700000700000380080180080 0C010007060001F80011147F9314>I<007C00C6018F038F070607000700070007000700 07000700FFF0070007000700070007000700070007000700070007000700070007000700 0700070007007FF01020809F0E>I<0000E003E3300E3C301C1C30380E00780F00780F00 780F00780F00780F00380E001C1C001E380033E0002000002000003000003000003FFE00 1FFF800FFFC03001E0600070C00030C00030C00030C000306000603000C01C038003FC00 141F7F9417>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E3E000E43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19 >I<1C001E003E001E001C000000000000000000000000000E007E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A1F809E0C>I<00 E001F001F001F000E0000000000000000000000000007007F000F0007000700070007000 7000700070007000700070007000700070007000700070007000700070007000706070F0 60F0C061803F000C28829E0E>I<0E0000FE00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E0FF00E03C00E03000E02000E04000E08000E10000E 30000E70000EF8000F38000E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FF CFF815207F9F18>I<0E00FE000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FF E00B20809F0C>I<0E1F01F000FE618618000E81C81C000F00F00E000F00F00E000E00E0 0E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E 000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE0 23147F9326>I<0E3E00FE43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319 >I<01F800070E001C03803801C03801C07000E07000E0F000F0F000F0F000F0F000F0F0 00F0F000F07000E07000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00 FEC3800F01C00F00E00E00E00E00F00E00700E00780E00780E00780E00780E00780E0078 0E00700E00F00E00E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E0000 0E00000E00000E0000FFE000151D7F9319>I<03E0800619801C05803C07803803807803 80700380F00380F00380F00380F00380F00380F003807003807803803803803807801C0B 800E138003E380000380000380000380000380000380000380000380000380003FF8151D 7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFE00F147F9312>I<1F9030704030C010C010C010E00078007F803FE0 0FF00070803880188018C018C018E030D0608F800D147E9312>I<020002000200060006 000E000E003E00FFF80E000E000E000E000E000E000E000E000E000E000E000E080E080E 080E080E080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0 030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003 810001C20000E40000EC00007800003800003C00007C00004E0000870001070003038002 01C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000E001E001C00380078007010E011E011C03 38027006700EFFFE10147F9314>I E /Fr 46 122 df<70F8FCFC740404040408081010 2040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F00007 1C000C06001803003803803803807001C07001C07001C07001C0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C070 01C07001C07803C03803803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038007C0FFFE0F217CA018 >I<03F0000C1C001007002007804003C04003C08003E0F003E0F801E0F801E0F801E020 03E00003E00003C00003C0000780000700000E00001C0000180000300000600000C00001 80000100000200200400200800201800603000403FFFC07FFFC0FFFFC013217EA018>I< 03F8000C1E001007002007804007C07807C07803C07807C03807C0000780000780000700 000F00000E0000380003F000001C00000F000007800007800003C00003C00003E02003E0 7003E0F803E0F803E0F003C04003C0400780200780100F000C1C0003F00013227EA018> I<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>I<10 00801E07001FFF001FFE001FF80013E00010000010000010000010000010000010000010 F800130E001407001803801003800001C00001C00001E00001E00001E00001E07001E0F0 01E0F001E0E001C08001C04003C04003802007001006000C1C0003F00013227EA018>I< 007E0001C1000300800601C00E03C01C03C0180180380000380000780000700000700000 F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013227EA018> I<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000 E0C00060C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018 >56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C0 0001C00001C0000380000380300300780700780600700C002018001030000FC00013227E A018>I<0001800000018000000180000003C0000003C0000003C0000005E0000005E000 000DF0000008F0000008F0000010F800001078000010780000203C0000203C0000203C00 00401E0000401E0000401E0000800F0000800F0000FFFF000100078001000780030007C0 020003C0020003C0040003E0040001E0040001E00C0000F00C0000F03E0001F8FF800FFF 20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E000070 1E0000701C0000303C0000303C0000307C0000107800001078000010F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F800000078000000780000107C000010 3C0000103C0000101C0000201E0000200E000040070000400380008001C0010000E00200 00381C000007E0001C247DA223>67 D69 DI73 D77 DI80 D82 D<03F0200C0C601802603001E07000E06000 60E00060E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF 0003FF80003FC00007E00001E00000F00000F0000070800070800070800070800070C000 60C00060E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF8780780786007 8018400780084007800840078008C007800C800780048007800480078004800780040007 800000078000000780000007800000078000000780000007800000078000000780000007 800000078000000780000007800000078000000780000007800000078000000780000007 80000007800000078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E 0018070000070000070000070000FF0007C7001E07003C0700780700700700F00708F007 08F00708F00F087817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C0 0E80600F00300E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E 0E001C0E003C0E00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07 801C0780380300780000700000F00000F00000F00000F00000F00000F00000F000007000 007800403800401C00800C010007060001F80012157E9416>I<0000E0000FE00001E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E001F8E007 04E00C02E01C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F0 00E07000E07800E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC00070700 0C03801C01C03801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000 7000007800203800201C00400E008007030000FC0013157F9416>I<003C00C6018F038F 030F070007000700070007000700070007000700FFF80700070007000700070007000700 0700070007000700070007000700070007000700070007807FF8102380A20F>I<000070 01F198071E180E0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E00 0F1C0019F0001000001000001800001800001FFE000FFFC00FFFE03800F0600030400018 C00018C00018C000186000306000303800E00E038003FE0015217F9518>I<0E0000FE00 001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E1F800E60C00E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C 001E003E001E001C00000000000000000000000000000000000E00FE001E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E >I<01C003E003E003E001C00000000000000000000000000000000001E00FE001E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01 F00E01C00E01800E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07 000E07800E03C00E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1F C07F00FE60E183801E807201C00F003C00E00F003C00E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A >I<0E1F80FE60C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I< 01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F00078 F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F 00FE61C00E80600F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E00 1E0E001E0E003C0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00 000E00000E00000E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F06 0F000E000E000E000E000E000E000E000E000E000E000E000E000E000F00FFF010157F94 13>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C80 0CC00CC008E018D0308FC00E157E9413>I<02000200020002000600060006000E001E00 3E00FFF80E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E04 0E040708030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F0060170 03827800FC7F18157F941B>III121 D E /Fs 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E0 700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F8000000 7C0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF00 000FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F8 8000007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0 F00000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Ft 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 908 a Ft(D)26 b(R)g(A)f(F)h(T)225 999 y Fs(Do)r(cumen)n(t) 20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1194 y Fr(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)802 1320 y(August)i(12,)f(1996)77 1378 y(This)h(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1436 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1494 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1552 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 45 a Fq(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fp(Chapter)34 b(9)75 564 y Fo(Miscellan)m(y)75 786 y Fq(There)12 b(are)g(a)g(n)o(um)o(b)q(er)g(of)g(topics)g(that)f (do)h(not)g(\014t)g(con)o(v)o(enien)o(tly)h(in)o(to)f(the)g(other)g(c)o (hapters.)19 b(W)l(e)12 b(collect)75 843 y(them)j(here.)75 986 y Fn(9.1)59 b(Standa)n(rdize)20 b(mpirun)75 1088 y Fq(A)13 b(n)o(um)o(b)q(er)h(of)f(implemen)o(tations)h(of)f(MPI-1)g (pro)o(vide)h(a)e(startup)h(command)g(for)g(MPI)g(programs)f(that)75 1144 y(is)k(of)e(the)i(form)170 1238 y Fm(mpirun)23 b()g()g()75 1332 y Fq(Separating)e(the)g(command)g(to)f(start)g(the)h(program)f(from)g (the)h(program)f(itself)i(pro)o(vides)f(\015exibil-)75 1388 y(it)o(y)l(,)15 b(particularly)h(for)e(net)o(w)o(ork)g(and)h (heterogeneous)g(implemen)o(tations.)21 b(F)l(or)14 b(example,)i(the)f (startup)75 1445 y(script)20 b(need)h(not)e(run)h(on)f(one)h(of)f(the)h (mac)o(hines)g(that)f(will)j(b)q(e)e(executing)h(the)e(MPI)h(program)f (it-)75 1501 y(self.)25 b(Ha)o(ving)17 b(a)f(standard)h(startup)f(mec)o (hanism)h(also)g(extends)g(the)g(p)q(ortabiliey)h(of)f(F)l(ortran)e (and)i(C)p Fl(>)f Fk(\(Sept\))75 1558 y Fq(programs)e(one)h(step)h (further,)e(to)h(the)g(command)g(lines)i(and)e(scripts)h(that)e(manage) h(them.)166 1614 y(F)l(or)f(example,)i(it)g(has)f(b)q(een)h(prop)q (osed)g(that)170 1708 y Fm(mpirun)23 b(-np)h()e()75 1802 y Fq(b)q(e)c(at)f(least)g(one)g(w)o(a)o(y)g(to)f(start)g Fm()g Fq(with)i(an)f(initial)j Fj(MPI)p 1275 1802 14 2 v 15 w(COMM)p 1432 1802 V 17 w(W)o(ORLD)d Fq(whose)g(group)75 1858 y(con)o(tains)25 b Fm()e Fq(pro)q(cesses.)48 b(Other)24 b(argumen)o(ts)g(to)g Fm(mpirun)g Fq(ma)o(y)f(b)q(e)j (implemen)o(tation-)75 1915 y(dep)q(enden)o(t.)166 1971 y(Curren)o(tly)14 b(this)g(is)g(prop)q(osed)g(as)f(advice)h(to)f (implemen)o(tors,)i(rather)e(than)g(as)g(a)h(required)g(part)f(of)75 2027 y(MPI-2.)20 b(It)15 b(is)h(not)e(suggested)i(that)e(this)i(b)q(e)g (the)f(only)h(w)o(a)o(y)e(to)g(start)g(MPI)h(programs.)189 2134 y Fi(A)n(dvic)n(e)h(to)h(implementors.)44 b Fq(Implemen)o(tors,)16 b(if)h(they)f(do)g(pro)o(vide)g(a)g(sp)q(ecialed)i(startup)d(com-)189 2190 y(mand)10 b(for)g(MPI)g(programs,)f(are)h(advised)i(to)d(giv)o(e)i (it)f(the)h(follo)o(wing)f(form.)18 b(The)10 b(syn)o(tax)g(is)h(c)o (hosen)189 2247 y(in)k(order)f(that)g Fm(mpirun)g Fq(b)q(e)h(able)h(to) e(b)q(e)h(view)o(ed)g(as)g(a)f(command-line)i(v)o(ersion)f(of)f Fj(MPI)p 1711 2247 V 16 w(SP)l(A)-5 b(WN)189 2303 y Fq(and)15 b Fj(MPI)p 362 2303 V 16 w(SP)l(A)-5 b(WN)p 528 2303 V 17 w(INDEPENDENT)16 b Fq(\(See)f(Section)h Fh(??)p Fq(\).)284 2416 y Fm(mpirun)23 b(-np)95 b()451 2472 y(-soft)47 b(<)191 b(>)451 2528 y(-host)47 b(<)191 b(>)451 2585 y(-arch)47 b(<)191 b(>)451 2641 y(-wdir)47 b(<)191 b(>)451 2698 y(-path)47 b(<)191 b(>)964 2828 y Fq(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fq(2)1156 b Fg(CHAPTER)15 b(9.)35 b(MISCELLANY)451 45 y Fm(-file)47 b(<)191 b(>)475 102 y(...)451 158 y()189 271 y Fq(for)9 b(the)i(case)f(where)g(a)g(single)i(command) e(line)i(for)e(the)g(application)i(program)d(and)h(its)h(argumen)o(ts) 189 327 y(will)16 b(su\016ce,)g(and)284 439 y Fm(mpirun)23 b({)h()f(})i(:)g({)g(...)f(})h(:)f({)h(...)g(})f(:)h (...)g(:)f({)h(...)f(})189 552 y Fq(As)15 b(with)h Fj(MPI)p 445 552 14 2 v 16 w(SP)l(A)-5 b(WN)p Fq(,)16 b(all)g(the)g(argumen)o (ts)e(are)h(optional.)21 b(\(Ev)o(en)16 b(the)f Fm(-np)24 b(x)39 b Fq(argumen)o(t)14 b(is)189 608 y(optional;)20 b(it)f(defaults)g(to)g Fm(1)p Fq(.)30 b(The)19 b(names)f(of)h(the)f (argumen)o(ts)g(are)h(tak)o(en)f(from)g(the)h(k)o(eys)f(in)189 665 y(the)e Fm(info)g Fq(argumen)o(t)f(to)h Fj(MPI)p 726 665 V 16 w(SP)l(A)-5 b(WN)p Fq(.)16 b(There)h(ma)o(y)e(b)q(e)i (other,)f(implemen)o(tation-dep)q(endent)189 721 y(argumen)o(ts)e(as)h (w)o(ell.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))75 828 y Fh(Example)i(9.1)k Fi(Start)17 b(16)g(instanc)n(es)d(of) i Fm(myprog)g Fi(on)g(the)g(curr)n(ent)g(or)h(default)g(machine:)170 921 y Fm(mpirun)23 b(-np)h(16)g(myprog)75 1028 y Fh(Example)18 b(9.2)k Fi(Start)17 b(10)g(pr)n(o)n(c)n(esses)d(on)i(the)g(machine)h(c) n(al)r(le)n(d)e Fm(ferrari)p Fi(:)170 1121 y Fm(mpirun)23 b(-np)h(10)g(-host)f(ferrari)g(myprog)75 1228 y Fh(Example)18 b(9.3)k Fi(Start)12 b(the)g Fm(ocean)f Fi(pr)n(o)n(gr)n(am)h(on)f (\014ve)g(suns)g(and)g(the)h Fm(atmos)f Fi(pr)n(o)n(gr)n(am)h(on)f(10)h (RS/6000's:)170 1322 y Fm(mpirun)23 b(-np)h(5)g(-arch)f(sun)g(ocean)g (:)h(-np)g(10)f(-arch)g(rs6000)g(atmos)75 1428 y Fh(Example)18 b(9.4)k Fi(Start)f(thr)n(e)n(e)f(c)n(opies)g(of)g(the)h(same)f(pr)n(o)n (gr)n(am)g(with)h(di\013er)n(ent)f(c)n(ommand-line)f(ar)n(gu-)75 1484 y(ments:)170 1578 y Fm(mpirun)k(myprog)h(infile1)e(:)i(myprog)f (infile2)g(:)h(myprog)f(infile3)-117 1638 y Fl(?)15 b Fk(\(Sept\))75 1771 y Fn(9.2)59 b(Datat)n(yp)r(e)18 b(Imp)n(rovements) -117 1828 y Fl(>)d Fk(\(Sept\))75 1873 y Fq(This)h(section)g(has)f(b)q (een)h(mo)o(v)o(ed)f(to)f Fj(MPI-1.2)g Fq(and)h(renamed)h(\\New)f (Datat)o(yp)q(es".)-117 1886 y Fl(?)g Fk(\(Sept\))75 2016 y Fn(9.3)59 b(Language)19 b(Interop)r(erabilit)n(y)75 2119 y Ff(9.3.1)49 b(Intro)q(duction)75 2205 y Fq(It)17 b(is)h(not)e(uncommon)h(for)g(library)h(dev)o(elop)q(ers)g(to)e(use)i (one)f(language)g(to)f(dev)o(elop)i(an)f(applications)75 2261 y(library)g(to)f(b)q(e)h(used)g(b)o(y)g(applications)h(dev)o(elop) q(ers)g(using)f(another)f(language.)24 b(MPI)16 b(curren)o(tly)h(sup-) 75 2318 y(p)q(orts)d(b)q(oth)i(an)e(ANSI)i(C)f(binding)i(and)e(a)f(F)l (ortran)g(77)g(binding.)22 b(It)15 b(should)h(b)q(e)f(p)q(ossible)i (for)d(F)l(ortran)75 2374 y(77)h(to)f(call)i(C)f(and)h(vice)g(v)o(ersa) f(that)f(use)i(MPI)f(in)h(b)q(oth)f(languages.)166 2431 y(MPI)h(allo)o(ws)h(the)f(dev)o(elopmen)o(t)h(of)f(clien)o(t-serv)o(er) i(co)q(de,)f(with)g(MPI)f(comm)o(unication)h(used)g(b)q(e-)75 2487 y(t)o(w)o(een)d(a)h(parallel)h(clien)o(t)g(and)f(a)g(parallel)h (serv)o(er.)k(It)14 b(should)i(b)q(e)g(p)q(ossible)g(to)e(co)q(de)i (the)f(serv)o(er)f(in)i(one)75 2543 y(language,)e(and)g(the)f(clien)o (ts)i(in)g(another)e(language.)19 b(T)l(o)14 b(do)f(so,)g(it)h(should)h (b)q(e)f(p)q(ossible)i(for)d(F)l(ortran77)75 2600 y(pro)q(cesses)j(to)e (comm)o(unicate)i(with)f(C)g(pro)q(cesses.)166 2656 y(There)g(are)g (sev)o(eral)h(issues)g(that)e(need)i(to)f(b)q(e)h(addressed,)f(to)f(ac) o(hiev)o(e)i(in)o(terop)q(erabilit)o(y)l(.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fg(9.3.)34 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY)921 b Fq(3)75 45 y Fh(Initializat)q(ion)26 b Fq(W)l(e)13 b(need)h(to)f(sp)q(ecify)h(ho)o(w)f(is)g(the)g(MPI)h(en)o(vironmen)o(t) f(initialized)j(for)d(b)q(oth)g(F)l(ortran)189 102 y(and)i(C.)75 195 y Fh(Comm)o(unication)j(of)g(MPI)e(opaque)i(ob)s(jects)23 b Fq(W)l(e)13 b(need)h(to)f(sp)q(ecify)i(ho)o(w)e(are)g(MPI)g(ob)s (ject)g(han-)189 252 y(dles)i(passed)g(from)e(F)l(ortran)h(to)f(C,)h (and)h(vice)g(v)o(ersa.)k(W)l(e)c(also)f(need)h(to)f(sp)q(ecify)i(what) e(happ)q(ens)189 308 y(when)g(an)g(MPI)g(ob)s(ject)g(is)g(accessed)h (in)g(one)f(language,)g(to)g(retriev)o(e)g(information)g(set)g(with)h (this)189 365 y(ob)s(ject)f(in)i(another)f(language.)75 459 y Fh(In)o(terlanguage)j(comm)o(unication)24 b Fq(W)l(e)18 b(need)g(to)e(sp)q(ecify)i(ho)o(w)f(messages)g(sen)o(t)g(in)g(one)h (language)189 515 y(can)d(b)q(e)h(receiv)o(ed)g(in)g(another)f (language.)166 609 y(It)f(is)h(highly)h(desirable)g(that)d(the)i (solution)g(for)f(in)o(terlanguage)h(in)o(terop)q(erabilit)o(y)h(b)q(e) f(extendable)75 665 y(to)g(new)g(languages,)g(should)h(MPI)f(bindings)i (b)q(e)f(de\014ned)h(for)d(suc)o(h)i(languages.)75 787 y Ff(9.3.2)49 b(Assumptions)75 873 y Fq(W)l(e)21 b(assume)f(that)g(con) o(v)o(en)o(tions)g(exist)h(for)f(C)h(programs)e(to)h(call)i(F)l(ortran) d(subroutines,)j(and)f(vice)75 929 y(v)o(ersa.)e(These)c(con)o(v)o(en)o (tions)f(sp)q(ecify)i(ho)o(w)e(to)g(link)h(C)g(and)f(F)l(ortran)f(co)q (de;)i(ho)o(w)f(to)g(call)h(from)f(F)l(ortran)75 986 y(a)j(C)g(function,)h(so)f(that)g(the)g(correct)g(C)g(calling)i(in)o (terface)f(b)q(e)g(generated,)g(and)f(vice)h(v)o(ersa;)g(ho)o(w)f(to)75 1042 y(pass)i(from)g(F)l(ortran)g(parameters)g(b)o(y)g(v)m(alue)i(to)e (C)h(functions;)i(and)e(the)g(corresp)q(ondence)h(b)q(et)o(w)o(een)75 1099 y(F)l(ortran)g(basic)h(data)f(t)o(yp)q(es)h(and)g(C)g(basic)g (datat)o(yp)q(es.)39 b(In)22 b(general,)i(these)e(con)o(v)o(en)o(tions) g(will)h(b)q(e)75 1155 y(implemen)o(tation)16 b(dep)q(enden)o(t.)22 b(\(Ho)o(w)o(ev)o(er,)13 b(there)i(is)g(an)g(ongoing)g(e\013ort)f(to)g (standardize)i(a)e(C)h(calling)75 1212 y(in)o(terface)k(in)h(High)f(P)o (erformance)g(F)l(ortran.\))29 b(F)l(urthermore,)19 b(not)f(ev)o(ery)h (F)l(ortran)f(t)o(yp)q(e)h(need)g(ha)o(v)o(e)75 1268 y(a)g(matc)o(hing)g(C)f(t)o(yp)q(e,)i(and)f(vice)h(v)o(ersa:)26 b(often)o(times,)20 b(for)e(example,)i(C)f(c)o(haracter)f(strings)h (are)f(not)75 1324 y(compatible)k(with)g(F)l(ortran)e(CHARA)o(CTER)i(v) m(ariables.)39 b(Ho)o(w)o(ev)o(er,)21 b(w)o(e)g(assume)h(that)e(a)h(F)l (ortran)75 1381 y Fe(INTEGER)c Fq(can)h(b)q(e)g(passed)g(to)f(a)g(C)h (program.)26 b(Also,)18 b(w)o(e)f(assume)h(that)f(b)q(oth)h(F)l(ortran) e(and)i(C)f(ha)o(v)o(e)75 1437 y(address)e(sized)i(in)o(tegers.)75 1559 y Ff(9.3.3)49 b(Initialization)75 1645 y Fq(A)18 b(call)i(to)e Fm(MPI)p 348 1645 15 2 v 16 w(INIT\(\))p Fq(,)g(either)h(from)f(C)g(or)g(F)l(ortran,)f(initialize)q(s)k(MPI)d (for)g(execution,)i(b)q(oth)e(in)h(C)75 1701 y(and)c(F)l(ortran.)166 1805 y Fd(Discussion:)36 b Fc(Certain)14 b(implemen)o(tations)e(use)j (the)g(\(inout\))f(argc,)g(argv)g(argumen)o(ts)g(of)g(the)h(C)f(v)o (ersion)75 1861 y(of)i Fe(MPI)p 202 1861 13 2 v 14 w(INIT)g Fc(in)f(order)i(to)f(propagate)g(the)h(righ)o(t)f(v)n(alue)f(for)h (argc,)g(argv)g(to)g(all)f(executing)i(pro)q(cesses.)27 b(Users)75 1918 y(should)13 b(b)q(e)i(w)o(arned)f(that)g(use)g(of)f (the)h(F)m(ortran)g(v)o(ersion)g(of)f Fe(MPI)p 1102 1918 V 14 w(INIT)h Fc(to)f(initialize)f(MPI)i(ma)o(y)e(result)i(in)g(a)f (loss)75 1974 y(of)j(this)g(abilit)o(y)m(.)22 b(This)16 b(should)g(b)q(e)h(acceptable:)24 b(if)15 b(C)h(execution)h(is)f(not)g (started)h(from)d(main,)g(but)j(is)f(started)75 2031 y(from)d(the)j(in)o(v)o(o)q(cation)e(of)h(a)g(\(library\))f(function)h (call,)g(then)g(one)h(do)q(es)g(not)f(necessarily)h(exp)q(ect)h(argc,)e (argv)g(to)75 2087 y(b)q(e)g(de\014ned.)166 2191 y Fq(The)21 b(function)g Fm(MPI)p 523 2191 15 2 v 17 w(INITIALIZED)e Fq(returns)h(the)h(same)f(answ)o(er,)h(whether)f(called)i(from)e(C)g (or)75 2247 y(F)l(ortran.)166 2304 y(The)15 b(function)h Fm(MPI)p 512 2304 V 17 w(FINALIZE)e Fq(\014nalizes)j(b)q(oth)e(C)g(and) h(F)l(ortran)e(MPI)h(en)o(vironmen)o(t.)166 2360 y(The)i(function)g Fm(MPI)p 515 2360 V 17 w(ABORT)e Fq(kill)k(pro)q(cesses,)e(irresp)q (ectiv)o(e)h(of)e(the)g(language)h(used)g(b)o(y)f(the)h(caller)75 2417 y(or)e(b)o(y)g(the)g(pro)q(cesses)h(killed.)166 2473 y(When)22 b Fj(MPI)p 390 2473 14 2 v 16 w(INIT)g Fq(is)g(called,)k(either)c(from)g(F)l(ortran)f(or)h(C,)f(then)i(the)f (MPI)g(en)o(vironmen)o(t)g(is)75 2530 y(initialized)17 b(in)e(the)f(same)f(manner,)h(b)q(oth)g(for)f(F)l(ortran)g(and)h(C.)g (I.e.,)f Fe(MPI)p 1350 2530 13 2 v 14 w(COMM)p 1494 2530 V 15 w(W)o(ORLD)h Fq(carries)g(the)75 2586 y(same)i(information,)g(b)q (oth)g(in)h(F)l(ortran)e(and)i(C:)e(same)h(pro)q(cesses,)h(same)f(en)o (vironmen)o(tal)h(attributes,)75 2643 y(same)e(error)f(handlers.)1396 b Fl(>)16 b Fk(\(Sept\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fq(4)1156 b Fg(CHAPTER)15 b(9.)35 b(MISCELLANY)189 45 y Fi(A)n(dvic)n(e)17 b(to)h(users.)52 b Fq(The)17 b(use)h(of)f(b)q(oth)g(C)g(and)g(F)l(ortran)f(in)i(one)f(MPI)h(program) e(ma)o(y)g(require)189 102 y(the)f(use)g(of)g(sp)q(ecial)i(options)f (at)e(compile)j(and/or)d(link)j(time.)j(\()p Fi(End)c(of)g(advic)n(e)g (to)h(users.)p Fq(\))189 205 y Fi(A)n(dvic)n(e)i(to)i(implementors.)68 b Fq(Implemen)o(tation)21 b(ma)o(y)e(selectiv)o(ely)i(link)h(the)e(C)f (and)h(F)l(ortran)189 262 y(MPI)15 b(libraries)i(only)f(to)f(co)q(des)h (that)f(need)h(them,)g(so)f(as)g(not)g(to)g(increase)h(the)g(size)g(of) g(binaries)189 318 y(for)21 b(co)q(des)i(that)e(use)i(only)f(one)g (language.)41 b(The)22 b(MPI)g(initializati)q(on)j(co)q(de)d(need)h(p)q (erform)189 375 y(initialization)18 b(for)c(a)h(language)g(only)g(if)h (that)e(language)h(library)h(is)g(loaded.)k(\()p Fi(End)c(of)g(advic)n (e)g(to)189 431 y(implementors.)p Fq(\))-117 490 y Fl(?)f Fk(\(Sept\))75 600 y Ff(9.3.4)49 b(T)l(ransfer)15 b(of)i(handles)75 686 y Fq(The)g(data)g(t)o(yp)q(e)g Fe(MPI)p 457 686 13 2 v 14 w(Fint)g Fq(is)h(pro)o(vided)f(in)h(C)f(for)g(an)g(in)o(teger)g (of)f(the)i(size)f(that)g(matc)o(hes)f(a)h(F)l(ortran)75 742 y Fe(INTEGER)p Fq(;)d(usually)l(,)i Fe(MPI)p 517 742 V 15 w(Fint)f Fq(will)i(b)q(e)f(equiv)m(alen)o(t)h(to)d Fe(int)p Fq(.)166 799 y(The)f(follo)o(wing)h(t)o(w)o(o)e(functions)i (are)f(pro)o(vided)h(in)g(C)f(to)g(con)o(v)o(ert)f(from)h(a)g(F)l (ortran)f(handle)i(\(whic)o(h)75 855 y(is)i(an)f(in)o(teger\))g(to)f(a) h(C)g(handle,)h(and)g(vice)g(v)o(ersa.)-1065 b Fl(>)15 b Fk(\(Sept\))75 912 y Fm(MPI)p 150 912 15 2 v 17 w(Handle)p 311 912 V 16 w(type)23 b(MPI)p 518 912 V 17 w(Int2handle\()g(MPI)p 894 912 V 16 w(Fint)h(f)p 1054 912 V 17 w(handle,)393 968 y(MPI)p 468 968 V 17 w(Handle)p 629 968 V 16 w(enum)g(handle)p 909 968 V 16 w(kind\))75 1055 y(MPI)p 150 1055 V 17 w(Fint)f(MPI)p 358 1055 V 17 w(Handle2int\()f(MPI)p 733 1055 V 17 w(Handle)p 894 1055 V 16 w(type)i(c)p 1054 1055 V 17 w(handle,)393 1111 y(MPI)p 468 1111 V 17 w(Handle)p 629 1111 V 16 w(enum)g(handle)p 909 1111 V 16 w(kind\))166 1197 y Fq(If)e Fj(f)p 235 1197 14 2 v 16 w(handle)g Fq(is)g(a)f(v)m(alid)i(F)l(ortran)e(handle)h (to)f(an)g(opaque)h(ob)s(ject,)g(and)f Fj(handle)p 1598 1197 V 18 w(kind)h Fq(sp)q(eci\014es)75 1254 y(the)17 b(t)o(yp)q(e)f(of)h(ob)s(ject,)f(then)h Fj(MPI)p 654 1254 V 16 w(INT2HANDLE)f Fq(returns)h(a)f(v)m(alid)i(C)f(handle)h(to)e (that)g(same)g(ob)s(ject;)75 1310 y(if)21 b Fj(f)p 139 1310 V 16 w(handle)h Fq(is)f(a)g(n)o(ull)h(F)l(ortran)d(handle,)k(then) e Fj(MPI)p 1016 1310 V 16 w(INT2HANDLE)f Fq(returns)h(a)f(n)o(ull)i(C)e (handle;)k(if)75 1367 y Fj(f)p 92 1367 V 16 w(handle)17 b Fq(is)e(an)g(in)o(v)m(alid)j(F)l(ortran)c(handle,)i(then)f Fj(MPI)p 1011 1367 V 16 w(INT2HANDLE)g Fq(returns)g(an)g(in)o(v)m(alid) i(C)e(handle;)75 1423 y(and)g(similarly)l(,)i(for)e Fj(MPI)p 513 1423 V 16 w(HANDLE2INT)p Fq(.)75 1527 y Fh(Example)j(9.5)k Fq(The)11 b(example)g(b)q(elo)o(w)f(illustrates)i(ho)o(w)d(the)i(F)l (ortran)d(MPI)j(function)g Fj(MPI)p 1651 1527 V 15 w(TYPE)p 1783 1527 V 17 w(COMMIT)75 1584 y Fq(can)k(b)q(e)h(implemen)o(ted)h(b)o (y)d(wrapping)i(the)f(C)g(MPI)g(function)h Fj(MPI)p 1244 1584 V 15 w(T)l(yp)q(e)p 1351 1584 V 18 w(commit)d Fq(with)i(a)g(C)g (wrapp)q(er)75 1640 y(to)g(do)g(handle)h(con)o(v)o(ersions.)75 1744 y Fm(!)24 b(FORTRAN)f(PROCEDURE)75 1800 y(SUBROUTINE)f (MPI_TYPE_COMMIT\()g(DATATYPE,)h(IERR\))75 1857 y(INTEGER)g(DATATYPE,)g (IERR)75 1913 y(CALL)g(MPI_X_TYPE_COMMIT\(DATATYPE,)d(IERR\))75 1970 y(RETURN)75 2026 y(END)75 2139 y(!)k(C)f(wrapper)75 2252 y(void)g(MPI_X_TYPE_COMMIT\()f(MPI_Fint)g(*f_handle,)h(*ierr\))75 2308 y({)75 2365 y(MPI_Datatype)f(datatype;)75 2478 y(datatype)h(=)g (\(MPI_Datatype\)MPI_int2handle\()d(*f_handle,)j (MPI_DATATYPE_HANDLE\);)75 2534 y(*ierr)g(=)h (\(MPI_Fint\)MPI_Type_commit)o(\()d(&datatype\);)75 2591 y(*f_handle)i(=)g(MPI_handle2int\()f(\(MPI_Handle_type\)datatype,)e (MPI_DATATYPE_HANDLE\);)75 2647 y(return\(\);)75 2704 y(})1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fg(9.3.)29 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY)926 b Fq(5)166 45 y(The)10 b(same)g(approac)o(h)g(can)g(b)q(e)h(used)g(for) e(all)i(other)f(MPI)g(functions.)19 b(The)11 b(call)g(to)e Fj(MPI)p 1637 45 14 2 v 16 w(INT2HANDLE)75 102 y Fq(\(resp.)k Fj(MPI)p 285 102 V 16 w(HANDLE2INT)p Fq(\))g(can)i(b)q(e)f(omitted)g (when)g(the)g(handle)i(is)e(an)g Fc(OUT)g Fq(\(resp.)f Fc(IN)p Fq(\))h(argumen)o(t,)75 158 y(rather)h(than)g Fc(INOUT)p Fq(.)166 312 y Fd(Discussion:)29 b Fc(The)10 b(design)g(assumes)f(that)g(all)g(C)g(handles)g(can)h(b)q(e)g(con)o(v)o (erted)h(to)e(t)o(yp)q(e)h Fe(MPI)p 1631 312 13 2 v 14 w(Handle)p 1764 312 V 16 w(T)m(yp)q(e)p Fc(,)75 368 y(and)15 b(vice)g(v)o(ersa,)g(without)g(loss.)21 b(This)15 b(is)g(a)g(sligh)o(t) f(restriction)i(on)e(MPI1)h(whic)o(h)g(only)f(requires)j(that)e (handles)75 424 y(b)q(e)g(of)e(an)g(in)o(teger)i(or)f(p)q(oin)o(ter)g (t)o(yp)q(e.)1875 480 y Fl(?)i Fk(\(Sept\))189 578 y Fi(R)n(ationale.)62 b Fq(The)19 b(design)h(here)g(pro)o(vides)f(a)g (con)o(v)o(enien)o(t)g(solution)h(for)e(the)h(prev)m(alen)o(t)h(case,) 189 634 y(where)f(a)f(C)h(wrapp)q(er)g(is)h(used)f(to)f(allo)o(w)i(F)l (ortran)d(co)q(de)j(to)e(call)i(a)f(C)f(library)l(,)j(or)d(C)h(co)q(de) h(to)189 691 y(call)g(a)f(F)l(ortran)f(library)l(.)32 b(The)19 b(use)h(of)f(C)f(wrapp)q(ers)i(is)f(m)o(uc)o(h)g(more)g(lik)o (ely)i(than)e(the)g(use)g(of)189 747 y(F)l(ortran)12 b(wrapp)q(ers,)i(b)q(ecause)h(it)f(is)g(m)o(uc)o(h)g(more)f(lik)o(ely)j (that)d(a)g(v)m(ariable)j(of)d(t)o(yp)q(e)h Fe(INTEGER)f Fq(can)189 804 y(b)q(e)j(passed)f(to)g(C,)f(than)h(a)g(C)g(handle)h (can)g(b)q(e)g(passed)f(to)g(F)l(ortran.)189 879 y(Assuming)e(that)g (the)g(F)l(ortran)f(handle)i(is)g(used)f(as)g(the)g(language)g(indep)q (enden)o(t)j(represen)o(tation)189 935 y(for)21 b(handles,)j(then)e (this)g(design)g(can)g(b)q(e)g(extended)h(to)e(allo)o(w)h(wrapp)q(ers)f (written)h(in)g(other)189 992 y(languages)15 b(b)o(y)g(pro)o(viding)h (con)o(v)o(ersion)g(functions)g(in)g(these)f(languages.)189 1067 y(The)21 b(use)h(of)f(con)o(v)o(ersion)g(functions,)j(rather)c (than)i(pro)q(cedures,)h(allo)o(w)f(us)f(to)g(use)g(inlinin)q(g,)189 1123 y(esp)q(ecially)c(in)f(the)f(case)g(where)g(these)g(functions)h (are)e(the)h(iden)o(tit)o(y)l(.)21 b(The)15 b(con)o(v)o(ersion)g (function)189 1180 y(in)f(the)g(wrapp)q(er)g(do)q(es)g(not)f(catc)o(h)h (an)f(in)o(v)m(alid)j(handle)f(argumen)o(t.)k(Instead,)14 b(an)g(in)o(v)m(alid)i(handle)189 1236 y(is)11 b(passed)h(b)q(elo)o(w)g (to)f(the)g(library)h(function,)h(whic)o(h,)f(presumably)l(,)h(c)o(hec) o(ks)e(its)h(input)g(argumen)o(ts.)189 1293 y(\()p Fi(End)j(of)i(r)n (ationale.)p Fq(\))189 1399 y Fi(A)n(dvic)n(e)c(to)h(users.)38 b Fq(The)13 b(user)g(needs)g(to)g(co)q(erce)g(the)g(result)g(returned)g (b)o(y)f Fj(MPI)p 1560 1399 14 2 v 16 w(int2handle)j Fq(from)189 1455 y Fe(void*)g Fq(to)f(a)h(suitable)i(handle)f(t)o(yp)q (e.)k(\()p Fi(End)c(of)g(advic)n(e)g(to)h(users.)p Fq(\))75 1577 y Ff(9.3.5)49 b(MPI)17 b(Opaque)f(objects)75 1663 y Fq(In)11 b(general,)g(opaque)g(ob)s(jects)e(are)h(\\the)g(same")g(in) h(F)l(ortran)e(and)h(C:)g(they)g(carry)g(the)g(same)g(information,)75 1719 y(and)17 b(ha)o(v)o(e)f(the)h(same)f(meaning)h(in)h(b)q(oth)f (languages.)24 b(The)17 b(mec)o(hanism)g(describ)q(ed)i(in)e(the)g (previous)75 1776 y(section)c(can)g(b)q(e)g(used)h(to)e(pass)g (references)i(to)e(MPI)g(ob)s(jects)g(from)g(language)h(to)f(language.) 20 b(An)13 b(ob)s(ject)75 1832 y(created)i(in)h(one)g(language)f(can)g (b)q(e)h(accessed,)g(mo)q(di\014ed)g(or)f(freed)g(in)h(another)f (language.)166 1889 y(W)l(e)g(examine)h(b)q(elo)o(w)g(in)g(more)f (detail,)h(issues)g(that)e(arise)i(for)e(eac)o(h)h(t)o(yp)q(e)h(of)f (MPI)g(ob)s(ject.)75 2011 y Ff(9.3.6)49 b(Datat)o(yp)q(es)75 2096 y Fq(Datat)o(yp)q(es)19 b(enco)q(des)h(the)g(same)g(information)f (in)i(F)l(ortran)e(and)h(in)g(C.)f(E.g.,)h(a)f(datat)o(yp)q(e)g (accessor)75 2153 y(lik)o(e)i Fj(MPI)p 248 2153 V 15 w(TYPE)p 380 2153 V 17 w(EXTENT)f Fq(will)i(return)d(the)h(same)f (information,)h(b)q(oth)g(in)g(F)l(ortran)f(and)g(in)i(C.)e(If)75 2209 y(a)g(datat)o(yp)q(e)f(de\014ned)j(in)f(F)l(ortran)e(is)h(used)h (in)g(a)f(C)f(comm)o(unication)i(call,)h(then)e(the)h(message)e(sen)o (t)75 2266 y(will)e(b)q(e)f(iden)o(tical)h(to)e(the)g(message)g(that)g (w)o(ould)h(b)q(e)g(sen)o(t)f(from)f(F)l(ortran:)19 b(the)14 b(same)g(comm)o(unication)75 2322 y(bu\013er)h(is)h(accessed,)f(and)h (the)f(same)g(represen)o(tation)g(con)o(v)o(ersion)g(is)h(p)q (erformed,)f(if)h(needed.)21 b(Both)15 b(C)75 2379 y(and)h(F)l(ortran)e (basic)i(datat)o(yp)q(es)f(can)g(b)q(e)h(used)g(in)h(datat)o(yp)q(e)d (constructors,)h(in)h(either)g(language.)21 b(If)15 b(a)75 2435 y(datat)o(yp)q(e)h(is)i(committed)e(in)i(either)g(language,)f (then)g(it)g(can)g(b)q(e)g(used)h(for)e(comm)o(unication)h(in)h(b)q (oth)75 2492 y(languages.)166 2548 y(The)c(function)g Fj(MPI)p 519 2548 V 16 w(ADDRESS)g Fq(returns)f(the)h(same)f(v)m(alue,) i(b)q(oth)e(in)i(F)l(ortran)d(and)i(in)g(C.)f(On)h(the)75 2604 y(other)h(hand,)h(w)o(e)g(do)g(not)f(require)i(that)e(the)h (constan)o(t)f Fe(MPI)p 1127 2604 13 2 v 14 w(BOTTOM)g Fq(has)h(the)g(same)f(v)m(alue)i(in)g(b)q(oth)75 2661 y(languages.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fq(6)1156 b Fg(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fh(Example)18 b(9.6)75 134 y Fm(!)24 b(FORTRAN)f(CODE)75 190 y(REAL)g(R\(5\))75 247 y(INTEGER)g(TYPE,)g(ADDR,)g(IERR)75 360 y(!)h(create)f(an)g(absolute)g(datatype)g(for)g(array)h(R)75 416 y(CALL)f(MPI_ADDRESS\()g(R,)g(ADDR,)g(IERR\))75 473 y(CALL)g(MPI_TYPE_STRUCT\(1,)f(5,)h(ADDR,)g(MPI_REAL,)g(TYPE,)g(IERR\)) 75 529 y(CALL)g(C_ROUTINE\(TYPE\))75 642 y(/*)h(C)f(code)h(*/)75 755 y(void)f(C_ROUTINE\(MPI_Fint)f(*ftype\))75 811 y({)75 868 y(int)h(count)h(=)f(5;)75 924 y(void)g(*handle;)75 981 y(int)g(lens[2])g(=)h({1,1};)75 1037 y(MPI_Aint)f(displs[2];)75 1094 y(MPI_Datatype)f(types[2],)h(newtype;)75 1207 y(/*)h(create)f(an)g (absolute)g(datatype)g(for)g(buffer)g(that)h(consists)70 b(*/)75 1263 y(/*)47 b(of)24 b(count,)f(followed)g(by)g(R\(5\))668 b(*/)75 1376 y(MPI_Addr\(&count,)22 b(displs[0]\);)75 1432 y(displs[1])h(=)g(0;)75 1489 y(types[0])g(=)g(MPI_INT;)75 1545 y(types[1])g(=)g(\(MPI_Datatype\)MPI_Int2handle\(f)o(type\);)75 1602 y(MPI_Type_struct\(2,)e(lens,)j(displs,)e(types,)i(&newtype\);)75 1658 y(MPI_Type_commit\(&newtype\);)75 1771 y(MPI_Send\(MPI_BOTTOM,)d (1,)j(newtype,)e(1,)i(0,)g(MPI_COMM_WORLD\);)75 1828 y(/*)g(the)f(message)g(sent)g(contains)g(an)h(int)f(count)g(of)h(5,)f (followed)47 b(*/)75 1884 y(/*)24 b(by)f(the)h(5)f(REAL)h(entries)f(of) g(the)h(Fortran)f(array)g(R.)238 b(*/)75 1940 y(})189 2040 y Fi(R)n(ationale.)58 b Fq(The)18 b(curren)o(t)g(MPI)g(standard)g (sp)q(eci\014es)i(that)d Fe(MPI)p 1363 2040 13 2 v 15 w(ADDRESS)g Fq(can)h(b)q(e)h(used)g(in)189 2096 y(initialization)i (expressions)f(in)g(C,)e(but)h(not)f(in)i(F)l(ortran.)29 b(Since)21 b(F)l(ortran)c(do)q(es)i(not)g(supp)q(ort)189 2153 y(normally)11 b(call)h(b)q(e)g(v)m(alue,)g(then)g Fe(MPI)p 815 2153 V 14 w(ADDRESS)e Fq(m)o(ust)g(b)q(e)i(in)g(F)l (ortran)d(the)i(name)g(of)g(a)f(prede\014ned)189 2209 y(static)19 b(v)m(ariable,)k(e.g.,)d(a)f(v)m(ariable)j(in)f(an)f(MPI)g (declared)h Fe(COMMON)f Fq(blo)q(c)o(k.)35 b(On)20 b(the)g(other)189 2266 y(hand,)e(in)h(C,)e(it)h(is)g(natural)f(to)g(tak)o(e)g Fe(MPI)p 930 2266 V 15 w(BOTTOM)f(=)g(0)h Fq(\(Ca)o(v)o(eat:)23 b(De\014ning)c Fe(MPI)p 1681 2266 V 14 w(BOTTOM)189 2322 y(=)e(0)h Fq(implies)i(that)e Fe(NULL)i Fq(p)q(oin)o(ter)e(cannot)h(b)q (e)g(distinguished)i(from)d Fe(MPI)p 1502 2322 V 14 w(BOTTOM)p Fq(;)g(ma)o(y)f(b)q(e)189 2378 y Fe(MPI)p 266 2378 V 14 w(BOTTOM)12 b(=)i(1)f Fq(is)i(b)q(etter)f Fb(:)8 b(:)g(:)e Fq(\))13 b(Requiring)j(that)e(the)g(F)l(ortran)f(and)h(C)g(v)m(alues)h (b)q(e)g(the)f(same)189 2435 y(will)i(complicate)h(the)e (initialization)j(pro)q(cess.)i(\()p Fi(End)c(of)g(r)n(ationale.)p Fq(\))189 2534 y Fi(A)n(dvic)n(e)e(to)h(implementors.)39 b Fq(The)14 b(follo)o(wing)g(implemen)o(tation)h(can)f(b)q(e)h(used:)k (MPI)14 b(addresses,)189 2591 y(as)c(returned)h(b)o(y)g Fj(MPI)p 564 2591 14 2 v 15 w(ADDRESS)p Fq(,)h(will)g(ha)o(v)o(e)e(the) h(same)f(v)m(alue,)j(b)q(oth)e(in)g(F)l(ortran)f(and)h(in)g(C.)f(One) 189 2647 y(ob)o(vious)17 b(c)o(hoice)h(is)f(that)f(MPI)h(addresses)g(b) q(e)h(iden)o(tical)h(to)d(regular)h(addresses.)26 b(The)17 b(address)189 2704 y(is)i(stored)f(in)i(the)f(datat)o(yp)q(e,)f(when)i (datat)o(yp)q(es)e(with)h(absolute)g(addresses)g(are)f(constructed.) 1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fg(9.3.)34 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY)921 b Fq(7)189 45 y(When)17 b(a)f(send)i(or)e(receiv)o(e)i(op)q(eration)f (is)g(p)q(erformed,)g(then)g(addresses)g(stored)f(in)i(a)e(datat)o(yp)q (e)189 102 y(are)e(in)o(terpreted)h(as)f(displacemen)o(ts)i(that)d(are) i(all)g(augmen)o(ted)f(b)o(y)h(a)f(base)g(address.)20 b(This)15 b(base)189 158 y(address)d(is)h(\(the)f(address)g(of)t(\))f Fj(buf)p Fq(,)j(or)d(zero,)i(if)g Fj(buf)g(=)f(MPI)p 1203 158 14 2 v 16 w(BOTTOM)p Fq(.)h(Th)o(us,)f(if)h Fe(MPI)p 1681 158 13 2 v 14 w(BOTTOM)189 214 y Fq(is)j(zero)g(then)h(a) f(send)g(or)g(receiv)o(e)h(call)g(with)g Fj(buf)g(=)f(MPI)p 1189 214 14 2 v 16 w(BOTTOM)h Fq(is)g(implemen)o(ted)h(exactly)189 271 y(as)12 b(a)h(call)i(with)e(a)g(regular)g(bu\013er)g(argumen)o(t:) 18 b(in)c(b)q(oth)g(cases)f(the)g(base)g(address)g(is)h Fj(buf)p Fq(.)20 b(On)14 b(the)189 327 y(other)j(hand,)h(if)f Fe(MPI)p 559 327 13 2 v 15 w(BOTTOM)g Fq(is)h(not)f(zero,)g(then)h(the) f(implemen)o(tation)i(has)e(to)g(b)q(e)h(sligh)o(tly)189 384 y(di\013eren)o(t.)h(A)12 b(test)g(is)h(p)q(erformed)g(to)f(c)o(hec) o(k)g(whether)h Fj(buf)g(=)g(MPI)p 1325 384 14 2 v 16 w(BOTTOM)p Fq(.)g(If)g(true,)f(then)h(the)189 440 y(base)i(address)g (is)h(zero,)f(otherwise)g(it)h(is)g Fj(buf)p Fq(.)21 b(In)15 b(particular,)h(if)g Fe(MPI)p 1393 440 13 2 v 14 w(BOTTOM)f Fq(do)q(es)g(not)g(ha)o(v)o(e)189 497 y(the)f(same)g(v)m (alue)i(in)f(F)l(ortran)f(and)g(C,)g(then)h(an)f(additional)i(test)e (for)g Fj(bu\013)h(=)f(MPI)p 1618 497 14 2 v 16 w(BOTTOM)h Fq(is)189 553 y(needed)h(in)g(at)f(least)g(one)g(of)g(the)g(t)o(w)o(o)f (languages.)189 628 y(It)h(ma)o(y)g(b)q(e)i(desirable)g(to)e(use)h(a)f (v)m(alue)i(other)e(than)h(zero)f(for)g Fe(MPI)p 1350 628 13 2 v 14 w(BOTTOM)h Fq(ev)o(en)g(in)g(C,)f(so)g(as)189 685 y(to)g(distinguish)k(it)e(from)f(a)g(NULL)h(p)q(oin)o(ter.)24 b(If)17 b Fe(MPI)p 1115 685 V 14 w(BOTTOM)e(=)g(c)i Fq(then)g(one)g (can)f(still)i(a)o(v)o(oid)189 741 y(the)h(test)g Fj(bu\013)h(=)g(MPI)p 597 741 14 2 v 16 w(BOTTOM)p Fq(,)f(b)o(y)h(using)g(the)g(displacemen)o (t)h(from)d Fe(MPI)p 1578 741 13 2 v 15 w(BOTTOM)p Fq(,)g(i.e.,)189 798 y(the)c(regular)g(address)g(-)h(c,)f(as)g(the)g(MPI)g(address)g (returned)h(b)o(y)f Fj(MPI)p 1384 798 14 2 v 16 w(ADDRESS)h Fq(and)f(stored)g(in)189 854 y(absolute)h(datat)o(yp)q(es.)k(\()p Fi(End)d(of)g(advic)n(e)h(to)f(implementors.)p Fq(\))75 976 y Ff(9.3.7)49 b(Addresses)75 1062 y Fq(Some)15 b(of)f(the)h(datat)o (yp)q(e)f(accessors)g(and)h(constructors)f(ha)o(v)o(e)g(argumen)o(ts)g (of)g(t)o(yp)q(e)h Fe(MPI)p 1608 1062 13 2 v 15 w(Aint)p Fq(,)f(in)h(C,)g(to)75 1118 y(hold)g(addresses.)20 b(The)15 b(corresp)q(onding)h(argumen)o(ts,)d(in)j(F)l(ortran,)d(ha)o(v)o(e)h(t) o(yp)q(e)h Fe(INTEGER)p Fq(.)f(This)h(causes)75 1175 y(F)l(ortran)j(and)h(C)g(to)f(b)q(e)h(incompatible,)j(in)e(an)f(en)o (vironmen)o(t)g(where)g(addresses)g(ha)o(v)o(e)g(64)f(bits,)i(but)75 1231 y(F)l(ortran)14 b Fe(INTEGER)p Fq(s)g(ha)o(v)o(e)h(32)g(bits.)166 1288 y(This)k(is)h(a)e(problem,)i(irresp)q(ectiv)o(e)h(of)d(in)o (terlanguage)h(issues.)32 b(Supp)q(ose)20 b(that)e(a)g(F)l(ortran)g (pro-)75 1344 y(cess)h(has)g(an)g(address)g(space)g(of)g Fl(\025)g Fq(4)g(GB.)g(What)f(should)i(b)q(e)g(the)f(v)m(alue)h (returned)f(in)h(F)l(ortran)e(b)o(y)75 1400 y Fj(MPI)p 160 1400 14 2 v 16 w(ADDRESS)p Fq(,)i(for)e(a)h(v)m(ariable)i(with)e (an)h(address)f(ab)q(o)o(v)o(e)g(2)1195 1384 y Fa(32)1232 1400 y Fq(?)33 b(The)19 b(prop)q(osed)h(design)g(aims)g(to)75 1457 y(solv)o(e)15 b(this)h(issue,)g(while)g(main)o(taining)h (compatibilit)o(y)f(with)g(curren)o(t)f(F)l(ortran)f(co)q(des.)284 b Fl(>)16 b Fk(\(Sept\))166 1513 y Fq(The)f(constan)o(t)g Fe(MPI)p 520 1513 13 2 v 14 w(ADDRESS)p 720 1513 V 14 w(KIND)g Fq(is)h(de\014ned)g(so)f(that,)f(in)i(F)l(ortran)e(90,)75 1570 y Fe(INTEGER\(KIND=MPI)p 474 1570 V 14 w(ADDRESS)p 674 1570 V 14 w(KIND\))22 b Fq(is)g(an)g(address)g(sized)h(in)o(teger)f (t)o(yp)q(e)g(\(t)o(ypically)l(,)i(but)e(not)75 1626 y(necessarily)l(,)15 b Fe(MPI)p 386 1626 V 14 w(ADDRESS)p 586 1626 V 14 w(KIND)e Fq(is)g(4)g(on)f(32)h(bit)g(address)g(mac)o (hines)g(and)g(8)g(on)g(64)f(bit)h(address)g(ma-)75 1683 y(c)o(hines\).)19 b(Similarly)l(,)13 b(the)d(constan)o(t)g Fe(MPI)p 771 1683 V 14 w(INTEGER)p 959 1683 V 14 w(KIND)g Fq(is)h(de\014ned)g(so)f(that)f Fe(INTEGER\(KIND=MPI)p 1816 1683 V 14 w(INTEGER)p 2004 1683 V 14 w(KIND\))75 1739 y Fq(is)16 b(a)f(default)g(size)h Fe(INTEGER)p Fq(.)1291 b Fl(?)16 b Fk(\(Sept\))166 1796 y Fq(There)f(are)g(sev)o(en)h (functions)g(that)e(ha)o(v)o(e)h(address)g(argumen)o(ts:)k Fj(MPI)p 1380 1796 14 2 v 16 w(TYPE)p 1513 1796 V 17 w(STRUCT,)75 1852 y(MPI)p 160 1852 V 16 w(TYPE)p 293 1852 V 17 w(HVECTOR,)d(MPI)p 632 1852 V 16 w(TYPE)p 765 1852 V 16 w(HINDEXED,)f(MPI)p 1118 1852 V 16 w(TYPE)p 1251 1852 V 17 w(EXTENT,)g(MPI)p 1556 1852 V 16 w(TYPE)p 1689 1852 V 17 w(LB,)75 1909 y(MPI)p 160 1909 V 16 w(TYPE)p 293 1909 V 17 w(UB)f Fq(and)g Fj(MPI)p 554 1909 V 15 w(ADDRESS)h Fq(\(the)e(address)h(is)g(an)g Fc(IN)f Fq(argumen)o(t)g(in) i(the)e(\014rst)h(three)f(and)h(an)75 1965 y Fc(OUT)k Fq(argumen)o(t)e(in)i(the)g(last)f(four\).)25 b(These)18 b(functions)g(should)g(accept)f(address)h(argumen)o(ts)e(whic)o(h)75 2021 y(are)e(either)g(address)g(sized,)h(or)f(are)g(default)g(in)o (tegers.)20 b(The)14 b(v)m(ariable)h Fe(MPI)p 1366 2021 13 2 v 15 w(AINT)p 1478 2021 V 15 w(KIND)e Fq(is)i(pro)o(vided)g(to)o Fl(>)h Fk(\(Sept\))75 2078 y Fq(indicate)j(the)f(curren)o(t)g(size)g (for)f(address)h(argumen)o(ts.)26 b(If)18 b(the)g(curren)o(t)g(v)m (alue)h(of)e Fe(MPI)p 1603 2078 V 14 w(AINT)p 1714 2078 V 15 w(KIND)g Fq(is)1875 2080 y Fl(?)f Fk(\(Sept\))75 2134 y Fq(equal)f(to)e Fe(MPI)p 325 2134 V 14 w(ADDRESS)p 525 2134 V 15 w(KIND)g Fq(then)h(these)g(sev)o(en)h(functions)f(accept) g(\(or)f(return\))h(address)g(sized)h(in-)75 2191 y(teger)c(v)m(alues)h (\(32)f(bits)h(on)f(32)g(bit)h(mac)o(hines)g(and)f(64)g(bits)h(on)f(64) g(bit)h(mac)o(hines\).)19 b(If)12 b Fe(MPI)p 1609 2191 V 14 w(AINT)p 1720 2191 V 15 w(KIND)e(=)75 2247 y(MPI)p 152 2247 V 14 w(INTEGER)p 340 2247 V 15 w(KIND)j Fq(then)h(these)f (functions)i(accept)e(or)g(return)h(address)f(v)m(alues)i(that)e(ha)o (v)o(e)g Fe(INTEGER)75 2304 y Fq(t)o(yp)q(e.)166 2360 y Fe(MPI)p 243 2360 V 14 w(AINT)p 354 2360 V 15 w(KIND)f Fq(has)f(initial)k(v)m(alue)e Fe(MPI)p 876 2360 V 14 w(INTEGER)p 1064 2360 V 14 w(KIND)p Fq(.)f(Implemen)o(ters)g(ma)o(y)g (sp)q(ecify)h(a)f(mec)o(h-)75 2417 y(anism)17 b(for)e(c)o(hanging)i (the)f(initial)i(v)m(alue,)g(as)d(part)h(of)g(the)g(program)f(start-up) h(pro)q(cedure)h(\(e.g.,)e(with)75 2473 y(argumen)o(ts)d(to)g(the)i (link)o(er)g(or)e(to)h Fe(mpirun)p Fq(\).)19 b(The)13 b(v)m(ariable)h Fe(MPI)p 1162 2473 V 15 w(AINT)p 1274 2473 V 14 w(KIND)f Fq(can)g(b)q(e)h(rep)q(eatedly)g(mo)q(di-)75 2530 y(\014ed)f(during)h(program)e(execution.)20 b(In)13 b(a)g(m)o(ultithreaded)h(en)o(vironmen)o(t,)f Fe(MPI)p 1431 2530 V 14 w(AINT)p 1542 2530 V 15 w(KIND)g Fq(should)g(not)75 2586 y(b)q(e)j(mo)q(di\014ed)g(while)h(an)e(MPI)g(function)h(with)g (address)f(argumen)o(t)g(executes.)166 2690 y Fd(Discussion:)34 b Fc(W)m(e)14 b(ma)o(y)e(also)h(w)o(an)o(t)h(to)f(pro)o(vide)h(a)g (deriv)o(ed)g(datat)o(yp)q(e)g Fe(MPI)p 1410 2690 V 15 w(AINT)f Fc(to)h(b)q(e)h(equiv)n(alen)o(t)e(to)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fq(8)1156 b Fg(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fe(INTEGER\(KIND)10 b(=)h(MPI)p 495 45 13 2 v 15 w(ADDRESS)p 696 45 V 14 w(KIND\))p Fc(,)f(on)h(those)h(F)m(ortran)f (systems)g(that)g(supp)q(ort)h(deriv)o(ed)g(datat)o(yp)q(es.)189 255 y Fi(A)n(dvic)n(e)20 b(to)h(users.)72 b Fq(Curren)o(t)20 b(F)l(ortran)f(MPI)h(co)q(des)h(will)h(run)f(unmo)q(di\014ed,)i(with)e (the)g(cur-)189 312 y(ren)o(t)c(prop)q(osal.)29 b(New)18 b(co)q(des)g(can)g(b)q(e)h(written)f(so)f(that)g(they)h(alw)o(a)o(ys)g (use)g(address)g(sized)h(ad-)189 368 y(dress)g(argumen)o(ts.)31 b(Suc)o(h)20 b(new)f(co)q(de,)i(written)e(in)h(F)l(ortran)e(90,)h (declares)h(address)g(v)m(ariables)-1991 b Fl(>)15 b Fk(\(Sept\))189 424 y Fq(to)i(b)q(e)i(of)e(t)o(yp)q(e)h Fe(INTEGER\(KIND=MPI)p 870 424 V 14 w(ADDRESS)p 1070 424 V 15 w(KIND\))p Fq(,)f(and)h(initializes)j Fe(MPI)p 1602 424 V 15 w(AINT)p 1714 424 V 15 w(KIND)16 b(=)189 481 y(MPI)p 266 481 V 14 w(ADDRESS)p 466 481 V 14 w(KIND)10 b Fq(\(or)f(starts)g(up)i(the)f(computation)g(so)g(that)f(the)i (initial)h(v)m(alue)f(of)f Fe(MPI)p 1752 481 V 14 w(AINT)p 1863 481 V 15 w(KIND)189 537 y Fq(is)19 b Fe(MPI)p 315 537 V 14 w(ADDRESS)p 515 537 V 14 w(KIND)p Fq(\).)f(On)h(man)o(y)f(F)l (ortran)f(77)h(systems,)g(suc)o(h)h(co)q(de)g(can)f(b)q(e)i(written)e (b)o(y)189 594 y(declaring)d(address)f(v)m(ariables)h(to)e(b)q(e)i(of)e (t)o(yp)q(e)h Fe(INTEGER*8)p Fq(.)k(\\Old")c(co)q(de)h(and)f(\\new")f (co)q(de)i(can)-1992 b Fl(?)15 b Fk(\(Sept\))189 650 y Fq(b)q(e)20 b(mixed.)33 b(E.g.,)19 b(a)g(new)g(st)o(yle)h(library)g (can)g(co)q(exist)g(with)f(an)h(old)f(st)o(yle)h(user)f(co)q(de,)i(if)f (the)189 707 y(v)m(alue)15 b(of)f Fe(MPI)p 433 707 V 14 w(AINT)p 544 707 V 15 w(KIND)g Fq(is)h(mo)q(di\014ed)h(up)q(on)f(en) o(try)f(to)g(the)g(library)h(and)g(restored)f(up)q(on)h(exit.)189 763 y(This)20 b(assumes)g(that)g(the)g(user)g(co)q(de)h(do)q(es)f(not)g (pass)g(addresses)g(\(or)g(datat)o(yp)q(es)f(con)o(taining)189 820 y(addresses\))e(to)g(the)h(library)l(.)28 b(In)18 b(the)g(later)g(case,)g(then)g(the)f(library)i(has)e(to)g(written)h(so) f(as)h(to)189 876 y(function)e(correctly)f(with)h(either)g(t)o(yp)q(es) f(of)f(addresses.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fq(\))189 982 y Fi(A)n(dvic)n(e)22 b(to)h(implementors.)84 b Fq(In)23 b(F)l(ortran)e(90,)j(o)o(v)o(erloading)e(can)h(b)q(e)g(used) g(so)f(as)g(to)g(pro-)189 1039 y(vide)d(di\013eren)o(t)f(v)o(ersions)g (of)f(the)h(functions)h Fj(MPI)p 1065 1039 14 2 v 15 w(TYPE)p 1197 1039 V 17 w(STRUCT,)f(MPI)p 1503 1039 V 16 w(TYPE)p 1636 1039 V 17 w(HVECTOR,)189 1095 y(MPI)p 274 1095 V 15 w(TYPE)p 406 1095 V 17 w(HINDEXED,)10 b(MPI)p 755 1095 V 16 w(TYPE)p 888 1095 V 17 w(EXTENT,)g(MPI)p 1188 1095 V 16 w(TYPE)p 1321 1095 V 17 w(LB)p Fq(,)f(and)i Fj(MPI)p 1581 1095 V 16 w(TYPE)p 1714 1095 V 16 w(UB)p Fq(,)g(one)189 1152 y(for)e(eac)o(h)h(address)h(size.)19 b(The)10 b(function)h(v)o(ersions)f(can)h(c)o(hec)o(k)f(the)g(curren)o (t)g(v)m(alue)i(of)d Fe(MPI)p 1696 1152 13 2 v 15 w(AINT)p 1808 1152 V 15 w(KIND)p Fq(,)189 1208 y(to)14 b(catc)o(h)h(errors.)189 1283 y(This)h(do)q(es)g(not)f(w)o(ork)g(with)h(the)g(function)g Fj(MPI)p 1035 1283 14 2 v 16 w(ADDRESS)p Fq(,)g(since)h(this)f(tak)o (es)f(a)h(c)o(hoice)g(argu-)189 1340 y(men)o(t,)j(and)g(an)f(in)o (terface)h(blo)q(c)o(k)h(cannot)e(b)q(e)i(pro)o(vided)g(for)e(it.)31 b(This)19 b(function)h(will)g(need)g(to)189 1396 y(b)q(e)e(reco)q(ded)h (so)f(as)g(to)f(branc)o(h)i(according)f(to)g(the)g(curren)o(t)g(v)m (alue)h(of)f Fe(MPI)p 1514 1396 13 2 v 14 w(AINT)p 1625 1396 V 15 w(KIND)p Fq(.)g(\(This)189 1453 y(function)e(returns)f(an)g (address,)g(so)g(that)f(error)h(c)o(hec)o(king)h(is)f(not)g(an)g (issue.\))189 1528 y(If)g(o)o(v)o(erloading)h(is)g(not)f(a)o(v)m (ailable,)h(then)g(all)g(sev)o(en)g(functions)g(need)h(b)q(e)f(reco)q (ded)g(so)f(as)g(to)g(tak)o(e)189 1584 y(in)o(to)h(consideration)i(the) f(curren)o(t)g(v)m(alue)h(of)e Fe(MPI)p 1051 1584 V 15 w(AINT)p 1163 1584 V 15 w(KIND)g Fq(when)h(in)o(terpreting)h(an)f (address)189 1641 y(input)f(parameter,)d(or)i(deciding)i(whic)o(h)e (address)g(v)m(alue)h(to)f(return.)k(\()p Fi(End)d(of)g(advic)n(e)g(to) g(imple-)189 1697 y(mentors.)p Fq(\))166 1844 y Fd(Alternativ)o(es)o(:) 134 1919 y Fc(1.)22 b(A)16 b(w)o(eak)o(er)h(v)o(ersion)g(of)f(this)g (prop)q(osal)h(is)f(to)g(sp)q(ecify)i(that)e Fe(MPI)p 1231 1919 V 14 w(AINT)p 1342 1919 V 15 w(KIND)h Fc(cannot)f(b)q(e)h(mo) q(difed)e(b)o(y)189 1969 y(the)f(user,)g(and)f(require)h(implemen)o (tors)d(to)i(pro)o(vide)g(an)g(MPI)h(en)o(vironmen)o(t)e(where)j Fe(MPI)p 1619 1969 V 14 w(AINT)p 1730 1969 V 15 w(KIND=)189 2018 y(MPI)p 266 2018 V 14 w(ADDRESS)p 466 2018 V 14 w(KIND)k Fc(and)g(an)g(en)o(vironmen)o(t)f(where)i Fe(MPI)p 1192 2018 V 14 w(AINT)p 1303 2018 V 15 w(KIND)f(=)g(MPI)p 1563 2018 V 15 w(INTEGER)p 1752 2018 V 14 w(KIND)p Fc(.)189 2068 y(The)14 b(selection)g(of)g(the)g(en)o(vironmen)o(t)f(can)h(b)q(e) g(done)h(at)e(link)g(time)g(or)g(with)h(argumen)o(ts)f(to)h Fe(mpirun)p Fc(.)k(Old)189 2118 y(co)q(de)g(can)h(b)q(e)f(run)g(with)g (the)h(former)e(en)o(vironmen)o(t.)29 b(New)18 b(co)q(de,)i(compatible) c(with)h(C,)h(can)g(b)q(e)h(run)189 2168 y(in)c(the)h(later.)23 b(Ho)o(w)o(ev)o(er,)16 b(w)o(e)f(lo)q(ose)h(the)g(abilit)o(y)e(to)h (mix)e(old)i(and)g(new)h(co)q(des:)23 b(If)15 b(a)g(library)g(that)g (calls)189 2218 y(one)j(of)g(the)h(sev)o(en)g(MPI)f(functions)h(with)f (address)h(argumen)o(ts)f(has)g(to)g(run)h(in)f(b)q(oth)g(en)o (vironmen)o(ts,)189 2267 y(then)d(w)o(e)h(need)g(either)g(t)o(w)o(o)f (di\013eren)o(t)h(v)o(ersions)g(of)e(the)i(library)m(,)e(or)h(a)g (library)f(that)i(can)f(accommo)q(date)189 2317 y(b)q(oth)i(en)o (vironmen)o(ts.)27 b(\(If)17 b(w)o(e)g(go)g(this)g(w)o(a)o(y)m(,)g (should)f(w)o(e)i(also)e(assume)h(that)g(there)i(are)e(t)o(w)o(o)g (di\013eren)o(t)189 2367 y Fe(mpi.h)12 b Fc(\014les,)h(with)g(t)o(w)o (o)f(di\013eren)o(t)i(de\014nitions)f(of)f Fe(MPI)p 1050 2367 V 15 w(AINT)p 1162 2367 V 15 w(KIND)p Fc(?)g(I.e.,)g(do)h(w)o(e)g (supp)q(ort)h(declarations)189 2417 y(of)h(the)h(form)d Fe(INTEGER\(KIND=MPI)p 809 2417 V 14 w(AINT)p 920 2417 V 15 w(KIND\))p Fc(?)i(This)g(will)f(allo)o(w)g(to)i(write)f(source)i (co)q(de)g(that)e(is)189 2467 y(compatible)f(with)i(either)h(en)o (vironmen)o(ts;)f(but)g(this)g(co)q(de)h(will)e(need)i(to)f(b)q(e)g (compiled)f(with)h(the)g(righ)o(t)189 2517 y Fe(mpi.h)d Fc(\014le.\))134 2591 y(2.)22 b(An)14 b(alternativ)o(e)h(is)g(to)f(c)o (hange)h(the)g(F)m(ortran)g(90)f(binding)g(so)h(that)g(functions)f (that)h(ha)o(v)o(e)g(argumen)o(ts)f(of)189 2641 y(t)o(yp)q(e)j Fe(MPI)p 362 2641 V 15 w(Aint)g Fc(will)f(accept)i(an)f(additional)e (optional)h(parameter,)h Fe(aint)p 1387 2641 V 15 w(kind)p Fc(,)i(with)e(a)g(default)g(v)n(alue)189 2691 y(whic)o(h)e(is)h(the)h (curren)o(t)g(size)f(of)g(an)f Fe(INTEGER)p Fc(.)g(Th)o(us,)h(curren)o (t)h(MPI)f(F)m(ortran)g(co)q(de)h(will)d(run)i(without)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fg(9.3.)34 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY) 921 b Fq(9)189 45 y Fc(c)o(hanges;)17 b(co)q(de)g(that)f(w)o(an)o(ts)g (to)g(use)h(an)e(address)j(size)e(that)g(is)g(di\013eren)o(t)h(from)d (the)j(default)f Fe(INTEGER)189 95 y Fc(will)f(need)k(to)e(pro)o(vide)g (the)g(additional)f(parameter.)27 b(Note)18 b(that)f(the)h(same)e (implemen)o(tation)e(can)j(b)q(e)189 145 y(used)h(as)f(for)g(the)h (main)e(prop)q(osal.)28 b(The)18 b(di\013erence)h(is)e(whether)i Fe(aint)p 1339 145 13 2 v 15 w(size)f Fc(is)f(a)g(global)f(v)n(ariable) g(or)h(a)189 195 y(function)c(argumen)o(t.)k(This)d(design)g(is)g (cleaner,)g(but)g(cannot)g(b)q(e)h(supp)q(orted)g(in)e(F)m(ortran)h (77.)75 384 y Ff(9.3.8)49 b(Groups)75 470 y Fq(A)15 b(group)h(ob)s (ject)e(represen)o(t)i(the)f(same)g(group)g(in)i(b)q(oth)e(F)l(ortran)f (and)i(C:)f(the)g(same)g(pro)q(cesses,)h(with)75 527 y(the)f(same)g(ranks.)20 b(Group)15 b(accessors)f(suc)o(h)i(as)f Fj(MPI)p 986 527 14 2 v 15 w(GROUP)p 1153 527 V 18 w(SIZE)g Fq(return)g(the)h(same)f(v)m(alue.)75 647 y Ff(9.3.9)49 b(Communicato)o(rs)75 733 y Fq(A)14 b(comm)o(unicator)f(represen)o(ts)h (the)g(same)f(group)h(and)g(same)f(comm)o(unication)i(domain,)f(b)q (oth)g(in)g(F)l(or-)75 789 y(tran)c(and)h(C.)f(In)i(particular,)g(a)e (message)h(sen)o(t)f(from)g(F)l(ortran)g(using)h(a)g(comm)o(unicator)f (can)h(b)q(e)h(receiv)o(ed)75 846 y(from)18 b(C,)h(using)g(the)g(matc)o (hing)h(comm)o(unicator)e(on)h(another)g(pro)q(cess;)h(collectiv)o(e)h (comm)o(unication)75 902 y(can)c(in)o(v)o(olv)o(e)h(b)q(oth)f(C)g(and)g (F)l(ortran)f(pro)q(cesses,)h(using)h(matc)o(hing)f(comm)o(unicators;)g (accessors)g(suc)o(h)75 959 y(as)e Fj(MPI)p 216 959 V 16 w(COMM)p 374 959 V 16 w(SIZE)g Fq(return)g(the)g(same)g(v)m(alue;)h (etc.)166 1015 y(T)l(op)q(ology)g(information)h(asso)q(ciated)g(to)f(a) g(comm)o(unicator)g(in)i(C)e(can)h(b)q(e)g(accessed)g(in)g(F)l(ortran,) 75 1072 y(and)e(vice)i(v)o(ersa.)75 1192 y Ff(9.3.10)49 b(A)o(ttributes)75 1278 y Fq(A)o(ttribute)17 b(k)o(eys)g(allo)q(cated)i (in)f(F)l(ortran)e(can)h(b)q(e)h(used)g(or)f(freed)h(in)g(C,)f(and)g (vice)h(v)o(ersa.)26 b(A)o(ttributes)75 1334 y(v)m(alues)17 b(set)f(in)i(F)l(ortran)d(can)h(b)q(e)h(accessed)g(in)g(C,)f(and)g (vice)h(v)o(ersa.)23 b(T)l(o)16 b(ac)o(hiev)o(e)g(this,)h(attribute)f (k)o(eys)75 1391 y(will)f(b)q(e)e(allo)q(cated)h(in)g(a)f(range)f(that) h(is)g(v)m(alid)i(b)q(oth)e(for)f(C)h Fe(int)g Fq(t)o(yp)q(e)g(and)h(F) l(ortran)d Fe(INTEGER)i Fq(t)o(yp)q(e.)19 b(The)75 1447 y(same)c(holds)h(true)f(for)f(system)h(de\014ned)i(attribute)e(v)m (alues)h(\(suc)o(h)f(as)g Fe(MPI)p 1361 1447 13 2 v 15 w(T)m(A)o(G)p 1456 1447 V 13 w(UB,)e(MPI)p 1626 1447 V 14 w(IO)p Fq(,)h(etc.\))166 1504 y(A)o(ttribute)20 b(k)o(eys)g(declared)i(in)f(C)f(are)g(asso)q(ciated)h(with)g(\\C)f (attributes")f(and)i(attribute)f(k)o(eys)75 1560 y(declared)15 b(in)f(F)l(ortran)e(are)h(asso)q(ciated)h(with)g(\\F)l(ortran)e (attributes".)19 b(C)13 b(attributes)g(ha)o(v)o(e)g(C)g(cop)o(y)h(and) 75 1617 y(delete)20 b(callbac)o(k)f(functions)g(\(the)g(functions)g (pro)o(vided)g(b)o(y)g(the)f Fj(MPI)p 1308 1617 14 2 v 16 w(Keyval)p 1451 1617 V 16 w(create)h Fq(call\).)30 b(F)l(ortran)75 1673 y(attributes)14 b(ha)o(v)o(e)g(F)l(ortran)g (delete)h(and)g(callbac)o(k)g(functions.)21 b(When)15 b(a)f(comm)o(unicator)g(is)h(duplicated)75 1730 y(then,)21 b(for)e(eac)o(h)h(attribute,)h(the)f(corresp)q(onding)h(cop)o(y)f (function)h(is)f(called,)i(using)f(the)f(righ)o(t)g(C)f(or)75 1786 y(F)l(ortran)14 b(calling)j(con)o(v)o(en)o(tion;)e(and)g (similarly)l(,)i(for)e(the)g(delete)h(callbac)o(k)g(function.)189 1883 y Fi(A)n(dvic)n(e)i(to)i(implementors.)59 b Fq(This)19 b(requires)h(that)d(attributes)i(b)q(e)g(tagged)e(either)j(as)e(\\C")f (or)189 1940 y(\\F)l(ortran",)j(and)h(that)f(the)g(language)h(tag)f(b)q (e)i(c)o(hec)o(k)o(ed)f(in)g(order)g(to)f(use)h(the)g(righ)o(t)f (calling)189 1996 y(con)o(v)o(en)o(tion)15 b(for)f(the)i(callbac)o(k)g (function.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))166 2093 y(C)k(attributes)f(are)h(of)f(t)o(yp)q(e)h Fe(void*)p Fq(,)h(whereas)f(F)l(ortran)f(attributes)g(are)h(of)g(t)o (yp)q(e)g Fe(INTEGER)p Fq(.)e(On)75 2149 y(some)h(systems,)g Fe(INTEGER)p Fq(s)g(will)i(ha)o(v)o(e)d(32)h(bits,)h(while)h(C)e(p)q (oin)o(ters)h(will)h(ha)o(v)o(e)e(64)f(bits.)33 b(This)20 b(is)f(a)75 2206 y(problem)d(if)g(comm)o(unicator)f(attributes)g(are)h (used)g(to)f(mo)o(v)o(e)f(information)i(from)f(a)g(F)l(ortran)f(caller) j(to)75 2262 y(a)e(C)g(callee,)h(or)f(vice-v)o(ersa.)166 2319 y(W)l(e)f(prop)q(ose)g(to)f(use)h(the)g(en)o(vironmen)o(t)g(v)m (ariable)h Fe(MPI)p 1124 2319 13 2 v 14 w(AINT)p 1235 2319 V 15 w(SIZE)e Fq(to)g(indicate)j(the)d(curren)o(t)h(size)75 2375 y(of)19 b(attributes)g(in)h(F)l(ortran.)31 b(If)19 b(an)g(attribute)g(is)h(set)f(in)h(4)f(b)o(yte)g(mo)q(de)g(and)h(is)f (accessed)h(in)g(8)f(b)o(yte)75 2432 y(mo)q(de,)g(then)g(the)g(4)f(b)o (yte)g(v)m(alue)i(is)f(sign)g(extended)g(to)f(8)g(b)o(ytes.)30 b(If)19 b(an)f(attribute)g(is)h(set)g(in)g(8)f(b)o(yte)75 2488 y(mo)q(de,)d(next)g(read)h(in)g(4)e(b)o(yte)h(mo)q(de,)h(then)f (the)g(8)g(b)o(yte)g(v)m(alue)i(is)e(truncated)h(to)e(4)h(b)o(ytes.)166 2592 y Fd(Discussion:)34 b Fc(W)m(e)13 b(ma)o(y)f(w)o(an)o(t)h(to)h (raise)g(an)g(error)h(if)e(the)h(truncation)g(causes)h(loss)f(of)f (information)e(\(i.e.,)75 2648 y(if)i(the)i(64)e(bit)g(in)o(teger)i (cannot)f(b)q(e)g(con)o(v)o(erted)h(to)f(a)g(32)f(bit)h(in)o(teger.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fq(10)1133 b Fg(CHAPTER)15 b(9.)35 b(MISCELLANY)189 45 y Fi(A)n(dvic)n(e)15 b(to)h(implementors.)40 b Fq(Comm)o(unicator)14 b(structures)h(are)f(set)h(for)f(8)h(b)o(yte)f (attributes,)h(but)189 102 y(F)l(ortran)d(access)i(functions)h(will)g (access)f(either)g(4)g(or)f(8)h(b)o(ytes,)f(according)h(to)f(the)h (curren)o(t)g(v)m(alue)189 158 y(of)g Fe(MPI)p 317 158 13 2 v 15 w(AINT)p 429 158 V 15 w(SIZE)p Fq(.)189 233 y(Ov)o(erloading)k(can)g(b)q(e)h(used)f(in)h(F)l(ortran)d(90)i(to)f (supp)q(ort)h(the)f(t)o(w)o(o)g(v)o(ersions)h(of)f(the)h(attribute)189 290 y(access)d(functions.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g (implementors.)p Fq(\))166 436 y Fd(Alternativ)o(es)o(:)134 511 y Fc(1.)22 b(No)13 b(c)o(hange.)19 b(On)14 b(a)g(64)f(bit)g (system,)h(the)g(attributes)h(will)d(b)q(e,)i(in)o(ternally)m(,)e(of)i (size)g(64)g(bits.)189 561 y Fe(MPI)p 266 561 V 14 w(A)m(TTRIBUTE)p 510 561 V 13 w(GET)p Fc(,)20 b(in)h(F)m(ortran,)h(will)e(truncate)i (the)f(64)g(bit)g(v)n(alue)f(to)h(a)f(32)h(bit)f(v)n(alue,)i(and)189 611 y Fe(MPI)p 266 611 V 14 w(A)m(TTRIBUTE)p 510 611 V 13 w(SET)14 b Fc(will)f(con)o(v)o(ert)h(the)h(32)e(bit)h(argumen)o(t) e(to)i(a)g(64)f(bit)h(in)o(teger.)134 686 y(2.)22 b(Add)14 b(an)f(optional)g Fe(aint)p 567 686 V 15 w(size)h Fc(argumen)o(t)f(to)h (attribute)g(access)i(functions)e(in)f(F)m(ortran)h(90.)166 767 y(The)k(problem)f(w)o(e)h(face)g(in)f(this)h(section)h(is)e(the)i (same)e(as)h(the)g(problem)f(w)o(e)h(faced)g(with)f(addresses)j(in)75 823 y(Section)14 b(9.3.7;)e(w)o(e)i(should)g(adopt)f(here)i(the)g(same) e(solution)g(that)h(w)o(e)g(adopt)g(for)f(addresses)j(there.)75 992 y Ff(9.3.11)49 b(Requests)75 1078 y Fq(A)17 b(request)h(created)f (in)h(C)g(can)f(b)q(e)h(accessed)g(and)g(up)q(dated)g(in)g(F)l(ortran)e (b)o(y)i(w)o(ait)f(or)f(test)h(functions)75 1135 y(and)e(can)h(b)q(e)g (freed)f(in)h(F)l(ortran;)e(and)h(vice)h(v)o(ersa.)75 1256 y Ff(9.3.12)49 b(Erro)o(r)15 b(handlers)75 1342 y Fq(Error)g(handlers)j(allo)q(cated)f(in)g(F)l(ortran)f(can)g(b)q(e)h (in)o(v)o(ok)o(ed)g(or)f(deallo)q(cated)i(from)d(C,)h(and)h(vice)g(v)o (ersa.)75 1399 y(When)i(an)g(MPI)f(exception)i(o)q(ccurs,)f(then)g(the) g(last)f(error)g(handler)i(asso)q(ciated)f(with)g(the)f(relev)m(an)o(t) 75 1455 y(comm)o(unicator)12 b(is)g(in)o(v)o(ok)o(ed,)g(irresp)q(ectiv) o(e)i(of)d(the)h(language)h(en)o(vironmen)o(t)f(where)g(the)g(error)f (o)q(ccurred)75 1511 y(and)g(the)h(en)o(vironmen)o(t)f(where)h(the)f (handler)h(w)o(as)f(asso)q(ciated)g(with)h(the)f(comm)o(unicator.)18 b(The)12 b(handler)75 1568 y(should)k(b)q(e)g(in)o(v)o(ok)o(ed)f(with)h (the)f(righ)o(t)g(parameter)g(passing)g(con)o(v)o(en)o(tion.)189 1674 y Fi(A)n(dvic)n(e)k(to)i(implementors.)69 b Fq(error)19 b(handler)i(ob)s(jects)e(need)i(to)f(ha)o(v)o(e)f(a)h(language)g(tag,)g (lik)o(e)189 1731 y(attribute)14 b(k)o(eys,)g(so)g(as)h(to)e(use)i(the) g(righ)o(t)f(calling)j(con)o(v)o(en)o(tion.)i(These)c(handlers)h(ha)o (v)o(e,)e(in)h(C,)f(a)189 1787 y(\\)p Fm(stdargs)p Fq(")e(argumen)o(t)h (list.)20 b(It)13 b(migh)o(t)h(b)q(e)g(useful)h(to)e(pro)o(vide)h(to)f (the)h(handler)g(information)g(on)189 1844 y(the)e(language)h(en)o (vironmen)o(t)g(where)g(the)g(error)f(o)q(ccurred.)20 b(\()p Fi(End)13 b(of)i(advic)n(e)e(to)i(implementors.)p Fq(\))75 1965 y Ff(9.3.13)49 b(Reduce)16 b(op)q(erations)75 2051 y Fq(Reduce)h(op)q(erations)e(de\014ned)i(in)f(C)f(can)g(b)q(e)h (used)g(or)e(deallo)q(cated)j(in)f(F)l(ortran,)e(and)h(vice)h(v)o (ersa.)189 2157 y Fi(A)n(dvic)n(e)g(to)h(users.)47 b Fq(Reduce)18 b(op)q(erations)e(receiv)o(e,)h(as)f(one)g(of)g(their)h (argumen)o(t,)e(the)h(datat)o(yp)q(e)189 2214 y(of)f(the)g(op)q (erands.)22 b(Th)o(us,)15 b(one)h(can)g(de\014ne,)g(either)g(in)h(C)e (or)g(F)l(ortran,)f(\\p)q(olymorphic")j(reduce)189 2270 y(op)q(erations)e(that)f(w)o(ork)h(b)q(oth)g(for)g(C)g(and)g(F)l (ortran)f(datat)o(yp)q(es.)19 b(\()p Fi(End)d(of)g(advic)n(e)g(to)h (users.)p Fq(\))189 2377 y Fi(A)n(dvic)n(e)c(to)i(implementors.)39 b Fq(Reduce)15 b(op)q(eration)f(ob)s(jects)f(need)h(to)f(carry)g(a)h (language)f(tag,)g(lik)o(e)189 2433 y(attribute)h(k)o(eys,)g(so)h(as)f (to)g(use)h(the)g(righ)o(t)f(calling)j(con)o(v)o(en)o(tion.)i(\()p Fi(End)d(of)g(advic)n(e)f(to)h(implemen-)189 2489 y(tors.)p Fq(\))1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fg(9.3.)29 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY) 904 b Fq(11)75 45 y Ff(9.3.14)49 b(Status)75 131 y Fq(The)15 b(MPI)f(status)f(ob)s(ject)h(is)h(not)f(really)h(an)f(opaque)g(ob)s (ject,)g(since)h(its)g(structure)f(is)g(partially)i(visible)75 187 y(to)i(the)h(user,)h(and)f(the)g(ob)s(ject)f(is)h(allo)q(cated)h(b) o(y)f(the)g(user.)31 b(A)19 b(C)f(status)g(ob)s(ject)g(is)i(a)e (structure;)i(a)75 244 y(F)l(ortran)12 b(status)g(ob)s(ject)h(is)g(an)g (arra)o(y)l(.)19 b Fj(MPI)12 b Fq(do)q(es)h(not)g(guaran)o(tee)g(that)f (status)g(ob)s(jects)h(can)g(b)q(e)h(passed)75 300 y(from)g(one)i (language)f(to)g(another.)166 397 y Fd(Alternativ)o(es)o(:)134 472 y Fc(1.)22 b(W)m(e)9 b(pro)o(vide)h(status)h(con)o(v)o(ersion)g (functions,)f(that)h(con)o(v)o(ert)g(a)e(status)i(ob)r(ject)g(to)f(a)g (language)g(indep)q(enden)o(t)189 522 y(status)k(ob)r(ject,)f(and)g (vice)h(v)o(ersa.)k(The)c(language)e(indep)q(enden)o(t)i(status)g(ob)r (ject)g(is)f(an)g(arra)o(y)g(of)g(in)o(tegers.)134 597 y(2.)22 b(W)m(e)14 b(require)i(that)f(status)h(ob)r(jects)h(ha)o(v)o(e) d(the)i(same)e(memory)f(la)o(y)o(out)h(in)g(C)h(and)g(F)m(ortran)g(so)g (that)g(they)189 646 y(can)f(b)q(e)g(passed)h(across)g(languages.)75 840 y Ff(9.3.15)49 b(Constants)75 926 y Fq(MPI)14 b(constan)o(ts)g(ha)o (v)o(e)g(the)g(same)g(v)m(alue)i(in)f(F)l(ortran)f(and)g(C.)g(This)h (includes,)h(error)e(co)q(des,)h(constan)o(ts)75 983 y(suc)o(h)23 b(as)f Fm(MPI)p 323 983 15 2 v 17 w(PROC)p 436 983 V 17 w(NULL)g Fq(or)g Fm(MPI)p 706 983 V 17 w(ANY)p 795 983 V 17 w(TAG)p Fq(,)g(etc.)42 b(This)24 b(do)q(es)f(not)f(apply)i (to)e(constan)o(t)g(handles)75 1039 y(\()p Fe(MPI)p 170 1039 13 2 v 14 w(INT,)17 b(MPI)p 357 1039 V 15 w(COMM)p 502 1039 V 14 w(W)o(ORLD,)h(MPI)p 769 1039 V 14 w(ERRORS)p 943 1039 V 14 w(RETURN,)f(MPI)p 1226 1039 V 14 w(SUM)p Fq(,)i(etc.\))33 b(These)19 b(handles)i(need)75 1095 y(to)16 b(b)q(e)i(con)o(v)o(erted,)e(as)h(explained)i(in)e(Section)h (9.3.4.)23 b Fe(MPI)p 1102 1095 V 14 w(BOTTOM)17 b Fq(ma)o(y)f(ha)o(v)o (e)g(di\013eren)o(t)h(v)m(alues)h(in)75 1152 y(F)l(ortran)c(and)h(C.)75 1274 y Ff(9.3.16)49 b(Interlanguage)17 b(communication)75 1359 y Fq(The)c(t)o(yp)q(e)f(matc)o(hing)g(rules)h(for)f(comm)o (unications)h(in)g(MPI)f(are)g(not)g(c)o(hanged:)19 b(The)12 b(datat)o(yp)q(e)g(sp)q(eci\014-)75 1416 y(cation)i(for)e(eac)o(h)i (item)f(sen)o(t)g(should)i(matc)o(h,)e(textually)l(,)h(the)f(datat)o (yp)q(e)g(sp)q(eci\014cation)i(used)f(to)f(receiv)o(e)75 1472 y(this)j(item)g(\(unless)g(one)g(of)f(the)g(t)o(yp)q(es)h(is)g Fe(MPI)p 878 1472 V 14 w(P)m(A)o(CKED)p Fq(\).)d(And)j(the)f(t)o(yp)q (e)h(of)f(a)g(message)g(item)h(should)75 1529 y(matc)o(h)c(the)i(t)o (yp)q(e)f(declaration)h(for)e(the)h(corresp)q(onding)h(comm)o (unication)g(bu\013er)f(lo)q(cation,)h(unless)g(the)75 1585 y(t)o(yp)q(e)k(is)g Fe(MPI)p 305 1585 V 14 w(BYTE)f Fq(or)g Fe(MPI)p 578 1585 V 15 w(P)m(A)o(CKED)p Fq(.)e(In)o (terlanguage)j(comm)o(unication)g(is)g(allo)o(w)o(ed,)g(if)h(it)f (complies)75 1642 y(with)e(these)f(rules.)75 1748 y Fh(Example)j(9.7)k Fq(In)17 b(the)e(example)i(b)q(elo)o(w,)e(a)h(F)l(ortran)e(arra)o(y)g (is)i(sen)o(t)g(from)e(F)l(ortran)h(and)g(receiv)o(ed)i(in)75 1804 y(C.)75 1911 y Fm(!)24 b(FORTRAN)f(CODE)75 1967 y(REAL)g(R\(5\))75 2024 y(INTEGER)g(TYPE,)g(ADDR,)g(IERR,)g(MYRANK)75 2137 y(!)h(create)f(an)g(absolute)g(datatype)g(for)g(array)h(R)75 2193 y(CALL)f(MPI_ADDRESS\()g(R,)g(ADDR\))75 2249 y(CALL)g (MPI_TYPE_STRUCT\(1,)f(5,)h(ADDR,)g(MPI_REAL,)g(TYPE,)g(IERR\))75 2306 y(CALL)g(MPI_TYPE_COMMIT\()f(TYPE\))75 2419 y(CALL)h (MPI_COMM_RANK\()f(MPI_COMM_WORLD,)g(MYRANK,)h(IERR\))75 2475 y(IF)h(\(MYRANK.EQ.0\))e(THEN)147 2532 y(CALL)h(MPI_SEND\()g (MPI_BOTTOM,)f(1,)h(TYPE,)h(1,)f(0,)h(MPI_COMM_WORLD,)e(IERR\))75 2588 y(ELSE)147 2645 y(CALL)h(MPI_HANDLE2INT\(TYPE,)e(CODE\))147 2701 y(CALL)i(C_ROUTINE\(CODE\))-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fq(12)1133 b Fg(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fm(END)23 b(IF)75 214 y(/*)h(C)f(code)h(*/)75 327 y(void)f(C_ROUTINE\(MPI_Fint)f(*fhandle\))75 384 y({)75 440 y(void)h(*handle;)75 497 y(MPI_Datatype)f(type;)75 553 y(MPI_Status)g(status;)75 666 y(type)h(=)h (\(MPI_Datatype\)MPI_Int2hand)o(le\(fhand)o(le\);)75 779 y(MPI_Recv\()f(MPI_BOTTOM,)f(1,)i(type,)f(0,)g(0,)h (MPI_COMM_WORLD,)e(status\);)75 835 y(})166 919 y Fq(MPI)17 b(implemen)o(tors)h(ma)o(y)f(w)o(eak)o(en)g(these)h(t)o(yp)q(e)f(matc)o (hing)h(rules,)g(and)g(allo)o(w)g(messages)e(to)h(b)q(e)75 975 y(sen)o(t)c(with)g(F)l(ortran)f(t)o(yp)q(es)h(and)g(receiv)o(ed)h (with)f(C)g(t)o(yp)q(es,)g(and)g(vice)h(v)o(ersa,)e(when)h(those)g(t)o (yp)q(es)g(matc)o(h.)75 1031 y(I.e.,)g(if)h(the)f(F)l(ortran)f(t)o(yp)q (e)h Fe(INTEGER)f Fq(is)i(iden)o(tical)h(to)d(the)i(C)e(t)o(yp)q(e)i Fe(int)p Fq(,)f(then)h(an)f(MPI)g(implemen)o(tation)75 1088 y(ma)o(y)22 b(allo)o(w)g(data)g(to)f(b)q(e)i(sen)o(t)g(with)f (datat)o(yp)q(e)g Fe(MPI)p 1035 1088 13 2 v 14 w(INTEGER)g Fq(and)g(b)q(e)h(receiv)o(ed)h(with)e(datat)o(yp)q(e)75 1144 y Fe(MPI)p 152 1144 V 14 w(INT)p Fq(.)15 b(Ho)o(w)o(ev)o(er,)f (suc)o(h)i(co)q(de)f(is)h(not)f(p)q(ortable.)75 1284 y Fn(9.4)59 b(T)-5 b(reatment)18 b(of)i(MPI)p 670 1284 18 2 v 21 w(Status)-117 1329 y Fl(>)15 b Fk(\(Sept\))75 1386 y Fq(This)h(section)g(has)f(b)q(een)h(mo)o(v)o(ed)f(to)f Fj(MPI-1.2)p Fq(.)-117 1388 y Fl(?)h Fk(\(Sept\))75 1526 y Fn(9.5)59 b(Prob)r(e)-117 1571 y Fl(>)15 b Fk(\(Sept\))75 1627 y Fq(The)j(seman)o(tics)f(of)g Fj(MPI)p 518 1627 14 2 v 16 w(PROBE)h Fq(and)g Fj(MPI)p 873 1627 V 15 w(IPROBE)g Fq(in)h Fj(MPI-1)d Fq(are)h(inconsisten)o(t)h(with)g(the)f(other)75 1684 y(uses)c(of)f(the)h(\\I")g(pre\014x.)19 b(Also,)14 b(the)f(curren)o(t)f(de\014n)o(tion)i(of)e(prob)q(e)i(is)f(problematic) h(in)f(a)g(m)o(ultithreaded)75 1740 y(en)o(vironmen)o(t:)19 b(while)d(one)e(thread)f(executes)i(a)e(prob)q(e,)i(follo)o(w)o(ed)f(b) o(y)f(a)h(receiv)o(e,)h(another)e(thread)h(ma)o(y)75 1797 y(consume)k(the)f(message)h(after)e(the)i(prob)q(e)g(and)f(b)q (efore)h(the)g(receiv)o(e)g(of)f(the)h(\014rst)f(thread.)27 b(The)17 b(\014rst)75 1853 y(problem)j(is)g(easily)g(solv)o(ed.)33 b(The)20 b(second)g(is)f(harder)h(to)e(solv)o(e.)33 b(The)20 b(curren)o(t)f(implemen)o(tation)h(of)75 1910 y(prob)q(e)e(do)q(es)h (not)e(require)i(that)e(en)o(tries)h(on)g(the)g(receiv)o(e)h(queue)g(b) q(e)f(lo)q(c)o(k)o(ed)h(b)o(y)f(a)f(call)i(to)e(prob)q(e.)29 b(A)75 1966 y(prob)q(e-lik)o(e)19 b(call)f(that)f(applies)i(suc)o(h)e (lo)q(c)o(king)i(is)e(thread)g(safe,)g(but)h(ma)o(y)e(imply)j (signi\014can)o(t)f(c)o(hanges)75 2023 y(\(and)e(ine\016ciencies?)q(\)) 26 b(in)17 b(message-passing)g(supp)q(ort.)23 b(A)17 b(\\try)p 1206 2023 V 15 w(receiv)o(e")g(that)f(attempts)f(to)h(receiv) o(e)75 2079 y(and)g(returns)f(immediately)i(if)f(no)f(message)g(is)h(a) o(v)m(ailable)h(is)e(a)g(thread-safe)h(construct,)e(but)i(do)q(es)f (not)75 2136 y(ha)o(v)o(e)g(the)g(\015exibilit)o(y)j(of)d(prob)q(e,)g (since)h(a)f(receiv)o(e)h(bu\013er)f(m)o(ust)g(b)q(e)h(pro)o(vided)g (upfron)o(t.)75 2286 y Fj(MPI)p 160 2286 V 16 w(QUERY\(source,)g(tag,)f (comm,)e(status\))117 2364 y Fc(IN)155 b Fj(source)468 b Fc(rank)14 b(of)f(source)117 2436 y(IN)155 b Fj(tag)526 b Fc(message)14 b(tag)117 2507 y(IN)155 b Fj(comm)470 b Fc(comm)o(unicator)117 2579 y(OUT)108 b Fj(status)476 b Fc(status)15 b(ob)r(ject)166 2704 y Fq(This)h(call)g(is)g(synon)o (ymous)f(to)f Fj(MPI)p 796 2704 V 16 w(PROBE)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fg(9.6.)29 b(MPI)15 b(AND)g(THREADS)1207 b Fq(13)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(IQUERY\(source,)16 b(tag,)f(comm,)e(request\))117 122 y Fc(IN)155 b Fj(source)468 b Fc(rank)14 b(of)f(source)117 197 y(IN)155 b Fj(tag)526 b Fc(message)14 b(tag)117 273 y(IN)155 b Fj(comm)470 b Fc(comm)o(unicator)117 348 y(OUT)108 b Fj(request)452 b Fc(request)16 b(ob)r(ject)166 472 y Fq(This)23 b(is)g(a)f(non)o(blo)q (c)o(king)h(v)o(ersion)g(of)f Fj(MPI)p 940 472 V 15 w(PROBE)p Fq(.)i(The)e(call)i(completes)f(when)f(a)g(message)75 529 y(matc)o(hing)h Fj(source,)i(tag,)f Fq(and)f Fj(comm)e Fq(is)i(a)o(v)m(ailable)h(for)f(reception.)43 b(Users)23 b(can)g(test)f(or)g(w)o(ait)h(for)75 585 y(completion)18 b(using)g Fj(MPI)p 515 585 V 15 w(W)l(AIT,)f(MPI)p 755 585 V 16 w(TEST)p Fq(,)f(or)h(an)o(y)f(of)h(the)g(deriv)o(ed)h(calls.) 25 b(The)18 b(status)e(returned)75 642 y(b)o(y)f(these)h(calls)g(will)h (pro)o(vide)e(information)h(on)f(the)g(matc)o(hed)g(message:)20 b(tag,)14 b(source,)g(and)i(size.)75 792 y Fj(MPI)p 160 792 V 16 w(TRYRECV\(buf,)h(count,)f(datat)o(yp)q(e,)g(source,)f(tag,)g (comm,)e(status,)k(\015ag\))117 870 y Fc(OUT)108 b Fj(buf)527 b Fc(initial)12 b(address)j(of)f(receiv)o(e)h(bu\013er)117 945 y(IN)155 b Fj(count)482 b Fc(max)12 b(n)o(um)o(b)q(er)h(of)h(en)o (tries)h(to)e(receiv)o(e)117 1020 y(IN)155 b Fj(datat)o(yp)q(e)424 b Fc(datat)o(yp)q(e)14 b(of)g(eac)o(h)g(en)o(try)117 1095 y(IN)155 b Fj(source)468 b Fc(rank)14 b(of)f(source)117 1170 y(IN)155 b Fj(tag)526 b Fc(message)14 b(tag)117 1245 y(IN)155 b Fj(comm)470 b Fc(comm)o(unicator)117 1320 y(OUT)108 b Fj(\015ag)518 b Fc(success)17 b(\015ag)166 1445 y Fq(This)h(call)g(b)q(eha)o(v)o(es)g(as)f(a)g(call)i(to)e Fj(MPI)p 856 1445 V 15 w(RECV)i Fq(excepts)e(that)g(it)h(is)g(non)o (blo)q(c)o(king:)25 b(it)18 b(receiv)o(es)g(a)75 1501 y(matc)o(hing)f(message,)f(if)h(suc)o(h)g(is)g(a)o(v)m(ailable,)h(and)f (returns)f Fj(\015ag)g(=)h(true)p Fq(,)g(and)g(returns)g Fj(\015ag)f(=)h(false)p Fq(,)f(if)75 1558 y(none)g(is)f(a)o(v)m (ailable.)166 1662 y Fd(Discussion:)34 b Fc(W)m(e)13 b(can)h(use)h(a)f(sp)q(ecial)g(\\n)o(ull")e(status,)i(rather)h(than)f (an)g(additional)e(\015ag)h(argumen)o(t.)1875 1717 y Fl(?)j Fk(\(Sept\))75 1852 y Fn(9.6)59 b(MPI)20 b(and)g(Threads)1875 1897 y Fl(>)c Fk(\(Sept\))75 1953 y Fq(This)g(section)g(has)f(b)q(een)h (mo)o(v)o(ed)f(to)f(the)h(External)h(c)o(hapter,)e(to)h(b)q(e)h(near)f (handlers.)1875 1956 y Fl(?)h Fk(\(Sept\))75 2097 y Fn(9.7)59 b(Continuable)19 b(Erro)n(rs)75 2198 y Fq(T)o(ypically)l(,)j(MPI)d(sa)o (ys)g(nothing)h(ab)q(out)g(the)f(in)o(ternal)h(state)f(of)g(itself)i (after)d(an)i(error)f(handler)h(has)75 2255 y(b)q(een)14 b(in)o(v)o(ok)o(ed.)20 b(There)13 b(are)g(a)g(n)o(um)o(b)q(er)g(of)g (error)f(classes)i(within)g(MPI)f(that)g(should)h(ha)o(v)o(e)f(no)g (negativ)o(e)75 2311 y(impact)19 b(on)f(the)g(in)o(ternal)h(state)f(of) g(MPI.)g(That)g(is)g(to)g(sa)o(y)l(,)g(if)h(one)g(of)e(the)i("con)o (tin)o(uable")g(errors)e(is)75 2368 y(detected)g(and)h(no)e(other)h (external)g(ev)o(en)o(ts)g(cause)g(degradation,)g(it)g(is)g(safe)g(to)f (con)o(tin)o(ue)h(using)h(MPI)75 2424 y(within)e(an)g(application.)166 2480 y(The)f("con)o(tin)o(uable")h(error)f(classes)g(are:)75 2587 y Fm(MPI_ERR_BUFFER)237 b(/*)23 b(invalid)g(buffer)g(pointer)g(*/) 75 2643 y(MPI_ERR_COUNT)261 b(/*)23 b(invalid)g(count)g(argument)g(*/) 75 2700 y(MPI_ERR_TYPE)285 b(/*)23 b(invalid)g(datatype)g(argument)g (*/)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fq(14)1133 b Fg(CHAPTER)15 b(9.)35 b(MISCELLANY)75 45 y Fm(MPI_ERR_TAG)309 b(/*)23 b(invalid)g(tag)h (argument)e(*/)75 102 y(MPI_ERR_COMM)285 b(/*)23 b(invalid)g (communicator)f(*/)75 158 y(MPI_ERR_ROOT)285 b(/*)23 b(invalid)g(root)h(*/)75 214 y(MPI_ERR_RANK)285 b(/*)23 b(invalid)g(rank)h(*/)75 271 y(MPI_ERR_GROUP)261 b(/*)23 b(invalid)g(group)g(*/)75 327 y(MPI_ERR_TOPOLOGY)189 b(/*)23 b(invalid)g(topology)g(*/)75 384 y(MPI_ERR_DIMS)285 b(/*)23 b(invalid)g(dimension)g(argument)g(*/)75 440 y(MPI_ERR_ARG)309 b(/*)23 b(invalid)g(argument)g(*/)75 547 y Fq(The)14 b(follo)o(wing)g(errors)e(pro)o(vide)i(no)f(guaran)o (tees,)g(although)g(implemen)o(ters)i(are)e(encouraged)h(to)e(mak)o(e) 75 603 y(errors)i("con)o(tin)o(uable")i(whenev)o(er)g(p)q(ossible:)75 697 y Fm(MPI_ERR_OP)333 b(/*)23 b(invalid)g(operation)g(*/)75 753 y(MPI_ERR_REQUEST)213 b(/*)23 b(invalid)g(request)g(handle)g(*/)75 810 y(MPI_ERR_UNKNOWN)213 b(/*)23 b(unknown)g(error)g(*/)75 866 y(MPI_ERR_TRUNCATE)189 b(/*)23 b(message)g(truncated)g(on)g (receive)g(*/)75 923 y(MPI_ERR_OTHER)261 b(/*)23 b(Other)g(error)h(*/) 75 979 y(MPI_ERR_INTERN)237 b(/*)23 b(internal)g(MPI)h(error)f(*/)75 1036 y(MPI_ERR_IN_STATUS)165 b(/*)23 b(error)g(code)h(is)f(in)h(status) f(*/)75 1092 y(MPI_ERR_PENDING)213 b(/*)23 b(pending)g(request)g(*/)75 1148 y(MPI_ERR_SYSRESOURCE)117 b(/*)23 b(out)h(of)f(system)g(resources) g(*/)75 1242 y Fq(W)l(e)15 b(also)f(pro)o(vide)h(the)g(function)g(to)f (indicate)i(whether)e(MPI)h(b)q(eliev)o(es)h(an)f(error)e(is)i("con)o (tin)o(uable")g(or)75 1299 y(not:)75 1450 y Fj(MPI)p 160 1450 14 2 v 16 w(ERR)p 261 1450 V 17 w(IS)p 316 1450 V 16 w(CONTINUABLE)h(\(erro)o(r,)d(\015ag\))117 1527 y Fc(IN)155 b Fj(Erro)o(r)490 b Fc(MPI)14 b(Error)h(Status)f(\(actual)g (status)h(or)f(a)f(class\))117 1602 y(OUT)108 b Fj(Flag)505 b Fc(Bo)q(olean)13 b(indication)f(of)h(whether)h(the)g(implemen)o (tation)c(al-)905 1658 y(lo)o(ws)19 b(safe)h(con)o(tin)o(ued)g(op)q (eration)g(in)f(the)i(ev)o(en)o(t)f(of)f(Error)905 1715 y(b)q(eing)14 b(encoun)o(tered)75 1839 y Fm(int)23 b(MPI)p 245 1839 15 2 v 17 w(Err)p 334 1839 V 17 w(is)p 399 1839 V 17 w(continuable\()f(int)i(error,)f(int)g(*flag)g(\))75 1926 y(MPI)p 150 1926 V 17 w(ERR)p 239 1926 V 17 w(IS)p 304 1926 V 16 w(CONTINUABLE\()g(ERROR,)g(FLAG,)g(IERROR)g(\))170 1982 y(INTEGER)g(ERROR,)g(IERROR)170 2039 y(LOGICAL)g(FLAG)75 2125 y(int)g(MPI::Err)p 365 2125 V 17 w(is)p 430 2125 V 16 w(continuable\()g(int)g(error,)g(int&)g(flag)h(\))75 2298 y Fn(9.8)59 b(Y)-5 b(et)19 b(Another)g(Send)h(Mo)r(de)-117 2343 y Fl(>)15 b Fk(\(Sept\))75 2400 y Fq(This)h(section)g(has)f(b)q (een)h(mo)o(v)o(ed)f(to)f(the)h Fj(MPI-JOD)p Fq(.)-117 2402 y Fl(?)g Fk(\(Sept\))75 2543 y Fn(9.9)59 b(Tw)n(o)20 b(Pro)r(cess)f("Sha)n(red")h(Bu\013ers)-117 2588 y Fl(>)15 b Fk(\(Sept\))75 2645 y Fq(This)h(section)g(has)f(b)q(een)h(mo)o(v)o (ed)f(to)f(the)h Fj(MPI-JOD)p Fq(.)-117 2647 y Fl(?)g Fk(\(Sept\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Aug 12 18:25:33 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA10524; Mon, 12 Aug 1996 18:25:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA20198 for mpi-core-out; Mon, 12 Aug 1996 17:24:34 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA20098; Mon, 12 Aug 1996 17:22:28 -0500 Message-Id: <199608122222.RAA20098@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov, mpi-misc@antares.mcs.anl.gov Subject: new misc, J.O.D. part Date: Mon, 12 Aug 1996 17:22:24 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Mon Aug 12 16:39:27 1996 %%Pages: 9 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.08.12:1639 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 10 58 df<1F00318060C04040C060C060C060C060C060C060C060C0604040 60C031801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C00 0C000C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C0018003 0006000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F00 008000400060C060C060804060801F000B107F8F0F>I<0300030007000F000B00130033 0023004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C0020002000 20002F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060 C06000C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE0 7FC08080808001000200040004000C0008000800180018001800180018000B117E900F> I<1F00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F 8F0F>I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E00 0B107F8F0F>I E /Fb 43 122 df<007F07F001FF1FF003FF3FF007807800070070000F 00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000FFF8FF80FFF8FF80FF F8FF800F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F 00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0001C 20809F1B>11 D<007000E001C00380078007000E001E001E003C003C003C007800780078 0078007000F000F000F000F000F000F000F000F000F000F000F000F00070007800780078 0078003C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DI<001F0000001F0000003F800000 3B8000003B8000007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F00001 E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C000FFFFE000F FFFE000FFFFE001E000F001E000F003C000F803C0007803C000780780007C0780003C078 0003C0F00003E01B207F9F1E>65 DI<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00 007C0000780000780000780000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800007C00003C00003E00001E00000F000207800E03E03E01FFFC00FF F0001FC017227DA01D>II II72 DI75 DIII<003F000000FFC00003FFF00007E1F8000F80 7C001F003E001E001E003C000F003C000F00780007807800078078000780F00003C0F000 03C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C07800 078078000780780007803C000F003C000F001E001E001F003E000F807C0007E1F80003FF F00000FFC000003F00001A227DA021>II<003F000000FFC00003FFF00007E1F8000F807C001F003E00 1E001E003C000F003C000F00780007807800078078000780F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C07800078078000780 781E07803C0F0F003C0F8F001E079E001F03FE000F83FC0007E1F80003FFF00000FFF800 003F780000007C0000003E0000001E0000001F0000000F801A277DA021>II<01FC0007FF800FFFC01F 03C03C00C03C00007800007800007800007800007800007C00003C00003F00001FE0000F FC0007FE0001FF00003F800007C00003C00003E00001E00001E00001E00001E00001E000 01C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019>IIII<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF0 0FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F000F000F000F000 F000F000780078003C013E0F1FFF0FFE03F010147E9314>I<0007800007800007800007 8000078000078000078000078000078000078000078000078007C7800FF7801FFF803E1F 807C0780780780F80780F00780F00780F00780F00780F00780F00780F00780780780780F 803E1F801FFF800FF78007C78011207E9F17>I<03F0000FFC001FFE003E1F003C070078 0700700380FFFF80FFFF80FFFF80F00000F00000F000007000007800003C01003E07001F FF0007FE0001F80011147F9314>I<03E0F00FFFF01FFFF03E3E003C1E00780F00780F00 780F00780F00780F003C1E003E3E001FFC003FF80033E0003000003800003FFE003FFF80 1FFFC03FFFE07803F0F000F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00141E7F 9317>103 DII108 DII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0 F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F80014147F9317>II114 D<07F01FFC3FFC780C7800780078007C003FC01FF00FF803F8007C003C003CC03CF07CFF F87FF00FC00E147F9311>I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E00 1E001E001E001E001E001E001E001E001E001E001E201FF00FF007C00C1A7F9910>II121 D E /Fc 24 90 df<381C7C3E7C3E7E3F3A1D0402040204020804080410082010402080 40100E7A9F17>34 D<3078F8787005057C840D>46 D<000C001C00FC0F38003800380038 0038003800700070007000700070007000E000E000E000E000E000E001C001C001C001C0 01C001C0038003C0FFFE0F1E7C9D17>49 D<003F8000C1E00100F0020078040078040078 0F007C0F807C0F807C0F00780600780000F80000F00001E00001C0000380000700000E00 001C0000380000600000C0000180000300200600200800401000403FFFC07FFF80FFFF80 161E7E9D17>I<0000100000001800000038000000380000007800000078000000FC0000 01BC0000013C0000033C0000023C0000063C0000043E0000081E0000081E0000101E0000 101E0000201E0000200F0000400F0000400F0000FFFF0000800F0001000F800100078002 0007800200078004000780040007800C0007C03E0007C0FF807FFC1E207E9F22>65 D<07FFFF00007C01C0003C01E0003C00F0007800F8007800F8007800F8007800F8007800 F8007800F000F001F000F001E000F003C000F00F8000FFFE0000F00F0001E007C001E003 C001E003E001E001E001E001E001E001E003C001E003C003E003C003E003C003C003C007 C003C00F8007800F0007803E00FFFFF0001D1F7E9E20>I<0001F808000E061800380138 007000F801E0007803C0007007800030078000300F0000301F0000301E0000303E000020 3C0000007C0000007C0000007C0000007C000000F8000000F8000000F8000000F8000000 F80000007800004078000080780000803C0000803C0001001C0002000E00020006000C00 0300100001C0E000003F00001D217B9F21>I<07FFFF00007C01E0003C00F0003C007800 78003C0078003C0078001E0078001E0078001E0078001F00F0001F00F0001F00F0001F00 F0001F00F0001F00F0001F01E0001E01E0003E01E0003E01E0003E01E0003C01E0007C03 C0007803C000F003C000F003C001E003C003C003C0078007800F0007803C00FFFFE00020 1F7E9E23>I<07FFFFF8007C0078003C0038003C00180078001800780008007800080078 0008007800080078080800F0100000F0100000F0100000F0300000FFF00000F0700001E0 200001E0200001E0200001E0200001E0000801E0001003C0001003C0001003C0002003C0 002003C0006003C000C0078001C0078007C0FFFFFF801D1F7E9E1F>I<07FFFFF8007C00 78003C0038003C001800780018007800080078000800780008007800080078000800F010 0000F0100000F0100000F0300000F0700000FFF00001E0600001E0200001E0200001E020 0001E0200001E0000003C0000003C0000003C0000003C0000003C0000003C00000078000 0007C00000FFFE00001D1F7E9E1E>I<07FFC7FFC0007C00F800003C007800003C007800 007800F000007800F000007800F000007800F000007800F000007800F00000F001E00000 F001E00000F001E00000F001E00000FFFFE00000F001E00001E003C00001E003C00001E0 03C00001E003C00001E003C00001E003C00003C007800003C007800003C007800003C007 800003C007800003C007800007800F000007C00F8000FFF8FFF800221F7E9E22>72 D<07FFE0007C00003C00003C0000780000780000780000780000780000780000F00000F0 0000F00000F00000F00000F00001E00001E00001E00001E00001E00001E00003C00003C0 0003C00003C00003C00003C00007800007C000FFFC00131F7F9E10>I<00FFFC0007C000 03C00003C0000780000780000780000780000780000780000F00000F00000F00000F0000 0F00000F00001E00001E00001E00001E00001E00001E00003C00303C00783C00F83C00F8 3800F0780080700040E00021C0001F000016207D9E17>I<07FFF000007E0000003C0000 003C000000780000007800000078000000780000007800000078000000F0000000F00000 00F0000000F0000000F0000000F0000001E0000001E0000001E0000001E0000001E00080 01E0010003C0010003C0010003C0030003C0020003C0060003C0060007801E0007807C00 FFFFFC00191F7E9E1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800 004E0002F000004E0002F000004E0004F000004E0004F000004E0008F000004E0008F000 00870011E00000870011E00000870021E00000870021E00000870041E00000838041E000 01038083C00001038083C00001038103C00001038203C0000101C203C0000101C403C000 0201C40780000201C80780000201C80780000201D00780000200F00780000600E0078000 0600E00F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E0000 3E001800003E001800004F001000004F001000004780100000478010000043C010000043 C010000083C020000081E020000081E020000080F020000080F020000080782000010078 400001007C400001003C400001003C400001001E400001001E400002000F800002000F80 0002000F800002000780000200078000060003800006000300000F00010000FFE0010000 221F7E9E22>I<0003F800001E0E000038070000E0038001C001C003C001E0078000E00F 0000F00F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C 0000F8F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C 0007803C000F003C001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23 >I<07FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F8007800F80078 00F8007800F000F001F000F001E000F003C000F0078000F00F0000FFF80001E0000001E0 000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C0 000003C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03 C0003C01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F003 C000F0078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C 0001E01E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F 04FFFC0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C07001806 00080E00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE000 07E00001E00000E00000F00000F04000E04000E04000E04000E06000C0600180E00380F8 0300C60C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030400F001040 0F0010C00F0010800F0010800F0010800F0010001E0000001E0000001E0000001E000000 1E0000001E0000003C0000003C0000003C0000003C0000003C0000003C00000078000000 7800000078000000780000007800000078000000F0000001F800007FFFE0001C1F7A9E21 >II87 D89 D E /Fd 71 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F8 70051C779B18>33 D<4010E038F078E038E038E038E038E038E038E038E038E038E03860 300D0E7B9C18>I<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C 73F01CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0 F800E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18>I<007000F001E003C007800F001E00 1C00380038007000700070007000E000E000E000E000E000E000E000E000700070007000 7000380038001C001E000F00078003C001F000F000700C24799F18>I<6000F00078003C 001E000F000780038001C001C000E000E000E000E0007000700070007000700070007000 7000E000E000E000E001C001C0038007800F001E003C007800F00060000C247C9F18>I< 01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007F00007F0001FFC00 7FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D9718>I<00600000F0 0000F00000F00000F00000F00000F00000F0007FFFC0FFFFE0FFFFE07FFFC000F00000F0 0000F00000F00000F00000F00000F00000600013147E9718>I<1C3E7E7F3F1F070E1E7C F860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC7830060677 8518>I<000300000780000780000F80000F00001F00001E00001E00003E00003C00007C 0000780000780000F80000F00001F00001E00003E00003C00003C00007C0000780000F80 000F00000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F000 0060000011247D9F18>I<01F00007FC000FFE001F1F001C07003803807803C07001C070 01C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C070 01C07803C03803801C07001F1F000FFE0007FC0001F000131C7E9B18>I<018003800380 07800F803F80FF80FB804380038003800380038003800380038003800380038003800380 03800380038003807FFCFFFE7FFC0F1C7B9B18>I<3078FCFC7830000000000000000030 78FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C 78F060071A789318>I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFF E0FFFFE0FFFFE07FFFC0130C7E9318>61 D<600000F00000FC00007E00003F00001FC000 07E00003F00001FC00007E00003F00001F80001F80003F00007E0001FC0003F00007E000 1FC0003F00007E0000FC0000F0000060000011187D9918>I<0FF0003FFC007FFF00700F 00F00380F00380600780000F00003E00007C0001F00001E00003C00003C00003C00003C0 0003C00003800000000000000000000000000000000003800007C00007C00007C0000380 00111C7D9B18>I<00700000F80000F80000D80000D80001DC0001DC0001DC00018C0003 8E00038E00038E00038E000306000707000707000707000707000FFF800FFF800FFF800E 03800E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE00F07E01E03E03C 01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E00000E00000E0 00007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F800131C7E9B 18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C0070 1C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C01C0 1C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00FFFC01F0FC0 1C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E00000E00FF0 E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0 141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C01C01C01C01C01 C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C01C01C01C01 C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF807F FF00111C7D9B18>I<7F07F0FF87F87F07F01C03C01C07801C07001C0E001C1E001C3C00 1C38001C70001CF0001DF0001DF0001FB8001FB8001F1C001E1C001C0E001C0E001C0700 1C07001C03801C03801C01C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00700E00700E00700E00 700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C0 1CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C0DC0 1C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F007007 00F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C 7D9B18>II<0FF8003FFE007FFF00780F00 700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00 001C00001E00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF001C0F 801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F 001C07001C03801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F87F00 F0161C7F9B18>I<03F3801FFF803FFF807C0F80700780E00380E00380E00380E0000070 00007800003F00001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E0 00E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8 E07038E07038E07038E07038007000007000007000007000007000007000007000007000 00700000700000700000700000700000700000700000700000700000700007FF0007FF00 07FF00151C7F9B18>IIII<7F8FE07F9F E07F8FE00E07000F0700070E00078E00039C0003DC0001F80001F80000F00000F0000070 0000F00000F80001F80001DC00039E00038E00070F000707000E07800E03801E03C07F07 F0FF8FF87F07F0151C7F9B18>II<7FFF00 FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000 070000FF0007FF001FFF007F0700780700E00700E00700E00700F00F00781F003FFFF01F FBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00 000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00 380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007 FF001FFF803E0780380300700000700000E00000E00000E00000E00000E00000E0000070 00007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80003F80001F80 00038000038000038000038000038003E3800FFB801FFF803C1F80380F80700780700380 E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFFF00FFBF8 03E3F0151C7E9B18>I<01F00007FC001FFE003E0F00380780700380700380E001C0E001 C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF8007FF0001FC001214 7D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FF FFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E30 1C0E003807003807003807003807003807001C0E001E1E001FFC001FF80039E000380000 1C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E000387800F07E03F0 1FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00000E00000E00000E00 000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03 800007C00007C00007C0000380000000000000000000000000007FC000FFC0007FC00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C000FFFF00FFFF80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF8 001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F00191481 9318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01 F0000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F0 01E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF80 7FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F0070 0F00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E0000 0E00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F807007 80700780E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF 800FFB8003E380000380000380000380000380000380000380000380003FF8003FF8003F F8151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C00003800003 80000380000380000380000380000380000380000380007FFE00FFFF007FFE0015147F93 18>I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE00 001F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<0180 000380000380000380000380007FFFC0FFFFC0FFFFC00380000380000380000380000380 000380000380000380000380000380400380E00380E00380E001C1C001FFC000FF80003E 0013197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC17148093 18>I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E00 038E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F 8FF0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E0003 8E000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C0 0E03800E0380070380070700070700038700038600038E0001CE0001CE0000CC0000CC00 00DC0000780000780000780000700000700000700000F00000E00079E0007BC0007F8000 3F00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E 00003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFF F014147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000780000 7FE0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003FE0003FE0 007FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00003C000FFC000FF00007C000013247E9F18>125 D E /Fe 60 123 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E01C0000E01C0 000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C0 0E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F8 151D809C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<6060F0F0F8F868 68080808080808101010102020404080800D0C7F9C15>34 D<60F0F86808080810102040 80050C7C9C0C>39 D<004000800100020006000C000C0018001800300030007000600060 006000E000E000E000E000E000E000E000E000E000E000E000E000600060006000700030 003000180018000C000C00060002000100008000400A2A7D9E10>I<8000400020001000 18000C000C000600060003000300038001800180018001C001C001C001C001C001C001C0 01C001C001C001C001C0018001800180038003000300060006000C000C00180010002000 400080000A2A7E9E10>I<60F0F0701010101020204080040C7C830C>44 DI<60F0F06004047C830C>I<00010003000600060006000C000C 000C0018001800180030003000300060006000C000C000C0018001800180030003000300 060006000C000C000C00180018001800300030003000600060006000C000C00010297E9E 15>I<030007003F00C70007000700070007000700070007000700070007000700070007 000700070007000700070007000700070007000F80FFF80D1C7C9B15>49 D<07C01830201C400C400EF00FF80FF807F8077007000F000E000E001C001C0038007000 6000C00180030006010C01180110023FFE7FFEFFFE101C7E9B15>I<07E01830201C201C 781E780E781E381E001C001C00180030006007E00030001C001C000E000F000F700FF80F F80FF80FF00E401C201C183007E0101D7E9B15>I<60F0F0600000000000000000000060 F0F06004127C910C>58 D<0FE03038401CE00EF00EF00EF00E000C001C0030006000C000 8001800100010001000100010001000000000000000000000003000780078003000F1D7E 9C14>63 D<000600000006000000060000000F0000000F0000000F000000178000001780 00001780000023C0000023C0000023C0000041E0000041E0000041E0000080F0000080F0 000180F8000100780001FFF80003007C0002003C0002003C0006003E0004001E0004001E 000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E0618001801980070007800E0003801C0003801C00018038000180 780000807800008070000080F0000000F0000000F0000000F0000000F0000000F0000000 F0000000F0000000700000807800008078000080380000801C0001001C0001000E000200 070004000180080000E03000001FC000191E7E9C1E>IIII< FFF3FFC00F003C000F003C000F003C000F003C000F003C000F003C000F003C000F003C00 0F003C000F003C000F003C000F003C000FFFFC000F003C000F003C000F003C000F003C00 0F003C000F003C000F003C000F003C000F003C000F003C000F003C000F003C000F003C00 FFF3FFC01A1C7E9B1F>72 DI77 DI<003F800000E0E0000380380007001C000E00 0E001C0007003C00078038000380780003C0780003C0700001C0F00001E0F00001E0F000 01E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C0780003C03800 03803C0007801C0007000E000E0007001C000380380000E0E000003F80001B1E7E9C20> II<003F800000E0E0000380380007001C00 0E000E001C0007003C00078038000380780003C0780003C0700001C0F00001E0F00001E0 F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C0780003C0 380003803C0E07801C1107000E208E0007205C0003A0780000F0E020003FE02000006020 00003060000038E000003FC000003FC000001F8000000F001B257E9C20>II<07E0801C1980300580700380600180E00180E00080E00080E000 80F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007C00003C00001 C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E7E9C17>I<7F FFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F0020800F002000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F800003 FFFC001B1C7F9B1E>II87 D<1FC000307000783800781C00301C00001C00001C0001FC000F1C00381C00701C00601C 00E01C40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E0 0060007004300418080C3007C00E127E9112>I<003F0000070000070000070000070000 070000070000070000070000070000070003E7000C1700180F00300700700700600700E0 0700E00700E00700E00700E00700E00700600700700700300700180F000C370007C7E013 1D7E9C17>I<03E00C301818300C700E6006E006FFFEE000E000E000E000600070023002 18040C1803E00F127F9112>I<00F8018C071E061E0E0C0E000E000E000E000E000E00FF E00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E007FE00F 1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C00381C00381C00181800 1C38000C300013C0001000003000001800001FF8001FFF001FFF803003806001C0C000C0 C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C0018000000000000000000000000000000FC001C 001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80091D7F 9C0C>I<00C001E001E000C000000000000000000000000000000FE000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0 F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180E001C0E001C0E001C0E001C0 E001C0E001C06001807003803003001806000E1C0003F00012127F9115>II<03C1000C3300180B00300F00700700700700E00700E00700E00700E0 0700E00700E00700600700700700300F00180F000C370007C70000070000070000070000 0700000700000700000700003FE0131A7E9116>II<1F9030704030C010 C010E010F8007F803FE00FF000F880388018C018C018E010D0608FC00D127F9110>I<04 000400040004000C000C001C003C00FFE01C001C001C001C001C001C001C001C001C001C 101C101C101C101C100C100E2003C00C1A7F9910>III< FF3FCFE03C0F03801C0701801C0701001C0B01000E0B82000E0B82000E1182000711C400 0711C4000720C40003A0E80003A0E80003C0680001C0700001C070000180300000802000 1B127F911E>I<7F8FF00F03800F030007020003840001C80001D80000F0000070000078 0000F800009C00010E00020E000607000403801E07C0FF0FF81512809116>II<7FFC70386038407040F040E041C003C0038007000F040E041C043C 0C380870087038FFF80E127F9112>I E /Ff 8 118 df<78FCFCFCFC7800000000000078 FCFCFCFC7806127D910D>58 D68 D<03FC000E0E001C1F003C1F 00781F00780E00F80000F80000F80000F80000F80000F800007800007801803C01801C03 000E0E0003F80011127E9115>99 D<1E003F003F003F003F001E00000000000000000000 000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F001F00FF E0FFE00B1E7F9D0E>105 D 110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8 F800F87800F07800F03C01E01E03C00F078001FC0015127F9118>I<1FD830786018E018 E018F000FF807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>115 D117 D E /Fg 24 120 df<000FF01FE0003FF07FE000FFF0FFE001FFF1FFE001F003F00003E0 03E00003C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007 C00007C007C00007C007C000FFFF07FF00FFFF07FF00FFFF07FF0007C007C00007C007C0 0007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C000 07C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007 C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C007C00007C0 07C000232A7FA923>11 D34 D46 D<00180000380000F80007F800FFF800FFF800F8F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F8007FFFF07FFFF07FFFF014287BA71E>49 D<00FE0003FFC007FFE00FFFF01F03F8 3C00FC38007E78003E70003EF0001FF0001F60001F20001F00001F00001F00001F00003E 00003E00007C00007C0000F80001F00001E00003C0000780000F00001E00003C00007800 00F00001E00003C0000780000F00001E00003C00007FFFFF7FFFFF7FFFFF7FFFFF18287E A71E>I<0001F000000003F800000003F800000007FC00000007BC00000007BC0000000F 3E0000000F1E0000000F1E0000001F1F0000001E1F0000001E0F0000003E0F8000003C0F 8000003C078000007C07C000007807C00000F803E00000F803E00000F003E00001F001F0 0001F001F00001E001F00003E000F80003E000F80003C000F80007FFFFFC0007FFFFFC00 0FFFFFFE000F80003E000F80003E001F00003F001F00001F001E00001F003E00000F803E 00000F803C00000F807C000007C07C000007C078000007C0F8000003E0F8000003E0232A 7EA928>65 D I77 D80 D<007FC00001FFF80007FFFE000FFFFF001FC07F003F000F007E0006007C0000 007C000000F8000000F8000000F8000000F8000000F8000000FC0000007E0000007F0000 003F8000001FF800000FFF800007FFE00003FFF80000FFFC00000FFE000000FF0000003F 0000001F8000000F8000000FC0000007C0000007C0000007C0000007C0000007C0000007 C000000F8060000F80F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE00000FF80 001A2C7DAA21>83 DI89 D<01FE000FFF803FFFC03FFFE03C03F03001F00001F80000F8 0000F80000F80000F80000F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F800F8 F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1D>97 D<007FC001FFF007FFFC0FFFFC1FC07C1F00083E00007C00007C00007C0000F80000F800 00F80000F80000F80000F80000F800007C00007C00007E00003E00001F000C1FC07C0FFF FC07FFFC01FFF0007F80161B7E9A1B>99 D<00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E00003E00003E00003E00003E00FC3E03FF3E07FFFE 0FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003E F8003EF8003EFC003E7C003E7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03FF3E00FC3E 172A7EA91F>I<007E0003FF8007FFC00FFFE01F83F03F00F03E00787C00787C00387800 3CFFFFFCFFFFFCFFFFFCFFFFFCF80000F80000F800007800007C00007C00003E00003F00 0C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I104 D110 D<007F000001FFC00007FFF0000FFFF8001FC1FC003F007E003E003E007C001F007C001F 0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F807C001F 007C001F007E003F003E003E003F007E001FC1FC000FFFF80007FFF00001FFC000007F00 00191B7E9A1E>I114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8 003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFF C07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007 C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003 FFE003FFE001FF8000FC0013227FA116>II119 D E /Fh 15 122 df68 D<0003FFFFFFFFE00003FFFFFFFFE00003FFFFFFFFE00003FFFFFFFFE000 03FFFFFFFFE000000007FFE00000000007FFE00000000007FFE00000000007FFE0000000 0007FFE00000000007FFE00000000007FFE00000000007FFE00000000007FFE000000000 07FFE00000000007FFE00000000007FFE00000000007FFE00000000007FFE00000000007 FFE00000000007FFE00000000007FFE00000000007FFE00000000007FFE00000000007FF E00000000007FFE00000000007FFE00000000007FFE00000000007FFE00000000007FFE0 0000000007FFE00000000007FFE00000000007FFE00000000007FFE00000000007FFE000 00000007FFE00000000007FFE00000000007FFE00000000007FFE00000000007FFE00000 000007FFE00000000007FFE00000000007FFE00000000007FFE00000000007FFE0000000 0007FFE00000000007FFE00000000007FFE00000000007FFE00000000007FFE000000000 07FFE00007E00007FFE0000FF00007FFE0001FF80007FFE0003FFC0007FFE0007FFE0007 FFE000FFFF0007FFE000FFFF0007FFE000FFFF0007FFE000FFFF0007FFE000FFFF000FFF C000FFFF000FFFC000FFFE000FFF80007FFE001FFF80007FFC001FFF00003FF8003FFE00 001FE0007FFC00000FFE03FFF8000007FFFFFFE0000001FFFFFF800000007FFFFE000000 0007FFE000000033487EC63D>74 D77 D<00000003FFF0000000000000007FFFFF80000000000003FFFFFF F000000000000FFFFFFFFC00000000003FFF807FFF0000000000FFF80007FFC000000003 FFE00001FFF000000007FFC00000FFF80000000FFF0000003FFC0000001FFE0000001FFE 0000003FFC0000000FFF0000007FFC0000000FFF800000FFF800000007FFC00001FFF000 000003FFE00003FFF000000003FFF00003FFE000000001FFF00007FFE000000001FFF800 07FFC000000000FFF8000FFFC000000000FFFC000FFFC000000000FFFC001FFF80000000 007FFE001FFF80000000007FFE003FFF80000000007FFF003FFF80000000007FFF003FFF 80000000007FFF007FFF00000000003FFF807FFF00000000003FFF807FFF00000000003F FF807FFF00000000003FFF807FFF00000000003FFF80FFFF00000000003FFFC0FFFF0000 0000003FFFC0FFFF00000000003FFFC0FFFF00000000003FFFC0FFFF00000000003FFFC0 FFFF00000000003FFFC0FFFF00000000003FFFC0FFFF00000000003FFFC0FFFF00000000 003FFFC0FFFF00000000003FFFC0FFFF00000000003FFFC0FFFF00000000003FFFC0FFFF 00000000003FFFC0FFFF00000000003FFFC07FFF00000000003FFF807FFF00000000003F FF807FFF80000000007FFF807FFF80000000007FFF803FFF80000000007FFF003FFF8000 0000007FFF003FFF80000000007FFF001FFFC000000000FFFE001FFFC000000000FFFE00 1FFFC000000000FFFE000FFFE000000001FFFC000FFFE000000001FFFC0007FFE0000000 01FFF80003FFF000000003FFF00003FFF800000007FFF00001FFF800000007FFE00000FF FC0000000FFFC000007FFE0000001FFF8000007FFF0000003FFF8000003FFF8000007FFF 0000000FFFC00000FFFC00000007FFF00003FFF800000003FFFC000FFFF000000000FFFF 807FFFC0000000007FFFFFFFFF80000000000FFFFFFFFC000000000003FFFFFFF0000000 0000007FFFFF8000000000000003FFF0000000004A4979C759>79 D<0007FFFC000000007FFFFFC0000001FFFFFFF8000003FFFFFFFE000007FE001FFF0000 07FF0003FFC0000FFF8001FFE0000FFF8000FFF0000FFF80007FF0000FFF80007FF8000F FF80007FF80007FF00003FFC0007FF00003FFC0003FE00003FFC0000F800003FFC000000 00003FFC00000000003FFC00000000003FFC00000000003FFC00000007FFFFFC000000FF FFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC0003FFF0003FFC0007FFC000 3FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC007FF800003FFC007FF800003F FC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC 00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC0003EFFC003FFF0007CFFF00 0FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFFF001FFFC0003FF80007FF836 2E7DAD3A>97 D<00001FFFC0000000FFFFF8000007FFFFFE00001FFFFFFF80007FFC00FF C000FFE001FFC001FFC003FFE003FF8003FFE007FF0003FFE00FFE0003FFE00FFE0003FF E01FFC0001FFC01FFC0001FFC03FFC0000FF803FFC00003E007FF8000000007FF8000000 007FF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF8000000 00FFF800000000FFF800000000FFF800000000FFF800000000FFF8000000007FF8000000 007FF8000000007FFC000000003FFC000000003FFC000000001FFC000000F81FFE000000 F80FFE000000F80FFF000001F007FF800003F003FFC00007E001FFE0000FC000FFF0001F 80007FFE00FF00001FFFFFFE000007FFFFF8000000FFFFE00000001FFE00002D2E7CAD35 >99 D<00001FFE00000001FFFFE0000007FFFFF800001FFFFFFE00007FFC07FF0000FFE0 01FF8001FFC0007FC003FF80003FE007FF00003FF00FFE00001FF01FFE00000FF81FFC00 000FF83FFC00000FFC3FFC000007FC7FFC000007FC7FF8000007FC7FF8000007FE7FF800 0007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFF FFFFFCFFF800000000FFF800000000FFF800000000FFF8000000007FF8000000007FF800 0000007FFC000000003FFC000000003FFC000000003FFC0000001C1FFE0000003E0FFE00 00003E07FF0000007E07FF000000FC03FF800001F801FFC00003F0007FF0001FE0003FFE 00FFC0001FFFFFFF800007FFFFFE000000FFFFF80000000FFF80002F2E7DAD36>101 D<000000FFC000000007FFF80000003FFFFC000000FFFFFF000001FFC1FF000007FF03FF 80000FFC03FF80000FF807FFC0001FF807FFC0003FF007FFC0003FF007FFC0003FE003FF 80007FE003FF80007FE001FF00007FE000FE00007FE0003800007FE0000000007FE00000 00007FE0000000007FE0000000007FE0000000007FE0000000007FE0000000007FE00000 00007FE0000000007FE0000000FFFFFFFE0000FFFFFFFE0000FFFFFFFE0000FFFFFFFE00 00FFFFFFFE0000007FF0000000007FF0000000007FF0000000007FF0000000007FF00000 00007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF00000 00007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF00000 00007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF00000 00007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF00000 00007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF00000 00007FF00000003FFFFFF800003FFFFFF800003FFFFFF800003FFFFFF800003FFFFFF800 002A487DC724>I<00FC0001FE0003FF0007FF800FFFC01FFFE01FFFE01FFFE01FFFE01F FFE01FFFE00FFFC007FF8003FF0001FE0000FC0000000000000000000000000000000000 0000000000000000000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001 FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0FFFFFFFF FFFFFFFFFFFFFFFFFFFFFF18497CC820>105 D<007FC000FFFFC000FFFFC000FFFFC000 FFFFC000FFFFC00003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 01FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000 FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<007FC001FFC00000FFFFC00FFFF80000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1 FC07FF8000FFFFC3E003FFC00003FFC7C001FFC00001FFCF0001FFE00001FFDE0000FFE0 0001FFDC0000FFE00001FFFC0000FFF00001FFF80000FFF00001FFF00000FFF00001FFF0 0000FFF00001FFF00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF0 0001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE0 0000FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFF E0FFFFFFC07FFFFFE03B2E7CAD42>110 D<00000FFF0000000000FFFFF000000007FFFF FE0000001FFFFFFF8000003FFC03FFC00000FFE0007FF00001FF80001FF80003FF00000F FC0007FE000007FE000FFE000007FF000FFC000003FF001FFC000003FF803FFC000003FF C03FF8000001FFC03FF8000001FFC07FF8000001FFE07FF8000001FFE07FF8000001FFE0 FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FF F8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF07FF8 000001FFE07FF8000001FFE07FF8000001FFE07FF8000001FFE03FFC000003FFC03FFC00 0003FFC01FFC000003FF801FFE000007FF800FFE000007FF0007FF00000FFE0003FF8000 1FFC0001FFC0003FF80000FFE0007FF000007FFC03FFE000001FFFFFFF80000007FFFFFE 00000000FFFFF0000000000FFF000000342E7DAD3B>I<00FF803F8000FFFF80FFF000FF FF83FFFC00FFFF87FFFE00FFFF8FC3FF00FFFF8F07FF0003FF9E0FFF8001FFBC0FFF8001 FFB80FFF8001FFF80FFF8001FFF00FFF8001FFF007FF0001FFF007FF0001FFE003FE0001 FFE000F80001FFE000000001FFE000000001FFC000000001FFC000000001FFC000000001 FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001 FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001 FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001 FFC000000001FFC000000001FFC0000000FFFFFFE00000FFFFFFE00000FFFFFFE00000FF FFFFE00000FFFFFFE00000292E7CAD31>114 D<000FFF00E0007FFFF3E001FFFFFFE007 FFFFFFE00FF800FFE01FC0001FE03F80000FE03F000007E07F000003E07F000003E0FF00 0003E0FF000003E0FF800003E0FFC0000000FFF0000000FFFE000000FFFFF800007FFFFF C0007FFFFFF0003FFFFFFC001FFFFFFF000FFFFFFF8007FFFFFFC003FFFFFFE000FFFFFF F0003FFFFFF00003FFFFF800001FFFF8000000FFFC0000001FFC7800000FFCF8000007FC F8000003FCFC000003FCFC000003FCFE000003F8FE000003F8FF000003F8FF800007F0FF C0000FF0FFF0001FE0FFFC00FFC0FFFFFFFF80FC7FFFFE00F81FFFF800E003FF8000262E 7CAD2F>I<7FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF 807FFFFFC000FFFF8000FFF000000FE00000FFF800000FC00000FFF800000FC000007FFC 00000F8000007FFC00001F8000003FFC00001F0000003FFE00003F0000001FFE00003E00 00001FFF00007E0000000FFF00007C0000000FFF8000FC00000007FF8000F800000007FF C001F800000003FFC001F000000003FFE003F000000003FFE003F000000001FFF003E000 000001FFF007E000000000FFF007C000000000FFF80FC0000000007FF80F80000000007F FC1F80000000003FFC1F00000000003FFE3F00000000001FFE3E00000000001FFF7E0000 0000000FFF7C00000000000FFFFC00000000000FFFFC000000000007FFF8000000000007 FFF8000000000003FFF0000000000003FFF0000000000001FFE0000000000001FFE00000 00000000FFC0000000000000FFC00000000000007F800000000000007F80000000000000 3F000000000000003F000000000000003F000000000000003E000000000000007E000000 000000007C00000000000000FC000000001F8000F8000000003FC001F8000000007FE001 F000000000FFF003F000000000FFF003E000000000FFF007E000000000FFF00FC0000000 00FFF01F8000000000FFF03F80000000007FE07F00000000007F43FE00000000003FFFF8 00000000001FFFF0000000000007FFC0000000000001FE00000000000039427EAD3F> 121 D E /Fi 9 117 df<00001E000000003E00000000FE00000003FE0000003FFE0000 FFFFFE0000FFFFFE0000FFFFFE0000FFCFFE0000000FFE0000000FFE0000000FFE000000 0FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000F FE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE00 00000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000 000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE000000 0FFE0000000FFE0000000FFE0000000FFE00007FFFFFFFC07FFFFFFFC07FFFFFFFC07FFF FFFFC0223879B731>49 D<0007FE0000007FFFE00001FFFFF80003FFFFFE0007F01FFF00 0F8007FF801F0001FFC03E0000FFE07F8000FFF07FC0007FF0FFE0007FF8FFF0003FF8FF F0003FFCFFF0003FFCFFF0003FFCFFF0003FFC7FE0003FFC3FC0003FFC1F80003FFC0000 003FFC0000003FF80000007FF80000007FF00000007FF0000000FFE0000000FFC0000001 FF80000001FF00000003FE00000007FC00000007F80000000FF00000001FC00000003F80 0000007F00000000FC00000001F800000001F0003C0003E0003C0007C0003C000F000078 001E000078003C00007800780000F800F00000F801FFFFFFF803FFFFFFF007FFFFFFF00F FFFFFFF01FFFFFFFF03FFFFFFFF07FFFFFFFF0FFFFFFFFF0FFFFFFFFE0FFFFFFFFE0FFFF FFFFE026387BB731>I<0000001FFF000030000001FFFFE000F000000FFFFFFC01F00000 7FFFFFFE03F00001FFFE007F87F00003FFE0000FCFF0000FFF000003FFF0001FFC000001 FFF0003FF80000007FF0007FF00000003FF000FFC00000003FF001FFC00000001FF003FF 800000000FF007FF000000000FF00FFF0000000007F00FFE0000000007F01FFE00000000 03F01FFE0000000003F03FFC0000000003F03FFC0000000001F03FFC0000000001F07FFC 0000000001F07FF80000000001F07FF80000000000007FF8000000000000FFF800000000 0000FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF8 000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800000000 0000FFF80000000000007FF80000000000007FF80000000000007FF80000000000007FFC 0000000000F03FFC0000000000F03FFC0000000000F03FFC0000000000F01FFE00000000 00F01FFE0000000001E00FFE0000000001E00FFF0000000001E007FF0000000003C003FF 8000000003C001FFC0000000078000FFE00000000F00007FF00000001F00003FF8000000 3E00001FFC0000007C00000FFF000001F8000003FFE00007F0000001FFFE003FC0000000 7FFFFFFF000000000FFFFFFC0000000001FFFFF000000000001FFF0000003C3D7BBB47> 67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF00001FFC00 FF80001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF00007F000 3FF00001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000000FFF FFF000007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF0003FE000 3FF0007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000FFC000 3FF000FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF007FFFE 0FFFF001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFF FE0000FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03FF00007 F83FF00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFFFFFFFC FFFFFFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE00000007F E00000007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF800007807FC 0000F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE00026267D A52D>101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF00000000 07FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000 03FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000 03FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF007FC000 03FF01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF6000FF80 03FFE000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF0000FFC0 03FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC0 03FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC0 03FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC0 03FF0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC3FFFFF FFFFFC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00FFFFFE 03FF00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF803FF00 001FFC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF03FF00 0007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF00 0007FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC03FF00 000FFC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC003FFF0 01FF8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF0000000003FF00 00000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF00 00000003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000FFFFFC 000000FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fj 64 123 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<001F81F8 0000F04F040001C07C06000380F80F000300F00F000700F00F0007007000000700700000 0700700000070070000007007000000700700000FFFFFFFF000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 070007007007007FE3FE3FF02420809F26>14 D<7038F87CFC7EFC7E743A040204020402 0804080410081008201040200F0E7E9F17>34 D<0020004000800100020006000C000C00 180018003000300030007000600060006000E000E000E000E000E000E000E000E000E000 E000E000E0006000600060007000300030003000180018000C000C000600020001000080 004000200B2E7DA112>40 D<800040002000100008000C00060006000300030001800180 018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E000C0 00C000C001C001800180018003000300060006000C00080010002000400080000B2E7DA1 12>I<70F8FCFC74040404080810102040060E7C840D>44 DI<70 F8F8F87005057C840D>I<000100030003000600060006000C000C000C00180018001800 300030003000600060006000C000C000C00180018001800300030003000600060006000C 000C000C00180018001800300030003000600060006000C000C000C000102D7DA117>I< 018003800F80F38003800380038003800380038003800380038003800380038003800380 038003800380038003800380038003800380038007C0FFFE0F1E7C9D17>49 D<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C02007C00007 C0000780000780000F00000E00001C0000380000700000600000C0000180000300000600 400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E0020 0F00780F80780780780780380F80000F80000F00000F00000E00001C0000380003F00000 3C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0078040078040 0F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00001E00002E00 002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00100E00200E00 200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0 141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000001000001000 0010000011F000161C00180E001007001007800003800003800003C00003C00003C07003 C0F003C0F003C0E00380400380400700200600100E000C380003E000121F7E9D17>I<00 7C000182000701000E03800C07801C0780380300380000780000700000700000F1F000F2 1C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C07003C07003C070 03803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF80 7FFF80400100800200800200800400000800000800001000002000002000004000004000 00C00000C00001C000018000038000038000038000038000078000078000078000078000 078000078000078000030000121F7D9D17>I<70F8F8F8700000000000000000000070F8 F8F87005147C930D>58 D<70F8F8F8700000000000000000000070F0F8F8780808081010 10202040051D7C930D>I<0FC0307040384038E03CF03CF03C603C0038007000E000C001 800180010003000200020002000200020002000000000000000000000007000F800F800F 8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C00000 07C0000009E0000009E0000009E0000010F0000010F0000010F000002078000020780000 20780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F0001 000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22 >65 DI<000FC040007030C001C0 09C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C0000407800 0040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F800 0000780000007C0000407C0000403C0000401C0000401E0000800E000080070001000380 020001C0040000703800000FC0001A217D9F21>IIII73 D76 DII<001F800000F0F00001C0380007801E000F000F000E00 07001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F800 01F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C00 03E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F000001F 80001C217D9F23>II82 D<07E0800C1980100780300380 600180600180E00180E00080E00080E00080F00000F000007800007F00003FF0001FFC00 0FFE0003FF00001F800007800003C00003C00001C08001C08001C08001C08001C0C00180 C00380E00300F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060400F 0020400F0020C00F0030800F0010800F0010800F0010800F0010000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F800007FF FE001C1F7E9E21>II< FFF07FF81FF01F800FC007C00F00078003800F00078001000F0007C00100078007C00200 078007C00200078007C0020003C009E0040003C009E0040003C009E0040003E010F00C00 01E010F0080001E010F0080001F02078080000F02078100000F02078100000F0403C1000 0078403C20000078403C20000078C03E2000003C801E4000003C801E4000003C801E4000 001F000F8000001F000F8000001F000F8000001E00078000000E00070000000E00070000 000C000300000004000200002C207F9E2F>87 D<7FF83FF80FE00FC007C0070003C00200 01E0040001F00C0000F0080000781000007C1000003C2000003E4000001E4000000F8000 000F8000000780000003C0000007E0000005E0000009F0000018F8000010780000207C00 00603C0000401E0000801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE 1F1F7F9E22>I<080410082010201040204020804080408040B85CFC7EFC7E7C3E381C0F 0E7B9F17>92 D<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E 001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317 >97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E0078 0E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F8 0E0C1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C 03F010147E9314>I<000380003F80000380000380000380000380000380000380000380 00038000038000038003E380061B801C0780380380380380700380700380F00380F00380 F00380F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E 9F19>I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F000 00F00000F000007000007000003800801800800C010007060001F80011147F9314>I<00 7C00C6018F038F07060700070007000700070007000700FFF00700070007000700070007 000700070007000700070007000700070007000700070007007FF01020809F0E>I<0000 E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E38 0033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C000 30C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F 01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C000000000000 000000000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E00FFC00A1F809E0C>I<0E0000FE00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E0FF00E03C00E03000E02000E04000E0800 0E10000E30000E70000EF8000F38000E1C000E1E000E0E000E07000E07800E03800E03C0 0E03E0FFCFF815207F9F18>107 D<0E00FE000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E81C81C000F00F00E000F00 F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E 00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C00F01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7 FC16147F9319>I<01F800070E001C03803801C03801C07000E07000E0F000F0F000F0F0 00F0F000F0F000F0F000F07000E07000E03801C03801C01C0380070E0001F80014147F93 17>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E00780E00780E00780E0078 0E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E3E000E00000E00000E0000 0E00000E00000E00000E00000E0000FFE000151D7F9319>I<03E0800619801C05803C07 80380380780380700380F00380F00380F00380F00380F00380F003807003807803803803 803807801C0B800E138003E3800003800003800003800003800003800003800003800003 80003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E 000E000E000E000E000E000E00FFE00F147F9312>I<1F9030704030C010C010C010E000 78007F803FE00FF00070803880188018C018C018E030D0608F800D147E9312>I<020002 000200060006000E000E003E00FFF80E000E000E000E000E000E000E000E000E000E000E 000E080E080E080E080E080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E007 01C007018003810001C20000E40000EC00007800003800003C00007C00004E0000870001 07000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000E001E001C0038007800701 0E011E011C0338027006700EFFFE10147F9314>I E /Fk 46 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70 F8F8F87005057C840E>I<01F000071C000C06001803003803803803807001C07001C070 01C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E07001C07001C07001C07803C03803803803801C07000C060007 1C0001F00013227EA018>48 D<008003800F80F380038003800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038007C0FFFE0F217CA018>I<03F0000C1C001007002007804003C04003C08003E0F0 03E0F801E0F801E0F801E02003E00003E00003C00003C0000780000700000E00001C0000 180000300000600000C0000180000100000200200400200800201800603000403FFFC07F FFC0FFFFC013217EA018>I<03F8000C1E001007002007804007C07807C07803C07807C0 3807C0000780000780000700000F00000E0000380003F000001C00000F00000780000780 0003C00003C00003E02003E07003E0F803E0F803E0F003C04003C0400780200780100F00 0C1C0003F00013227EA018>I<000200000600000E00000E00001E00001E00002E00004E 00004E00008E00008E00010E00020E00020E00040E00040E00080E00100E00100E00200E 00200E00400E00800E00FFFFF8000E00000E00000E00000E00000E00000E00000E00001F 0001FFF015217FA018>I<1000801E07001FFF001FFE001FF80013E00010000010000010 000010000010000010000010F800130E001407001803801003800001C00001C00001E000 01E00001E00001E07001E0F001E0F001E0E001C08001C04003C04003802007001006000C 1C0003F00013227EA018>I<007E0001C1000300800601C00E03C01C03C0180180380000 380000780000700000700000F0F800F30C00F40600F40300F80380F801C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07001E07001E03801C03801C01803801C03000C0600 070C0001F00013227EA018>I<01F800060E000803001001802001802000C06000C06000 C06000C07000C07801803E01003F02001FC4000FF80003F80003FC00067F00083F80100F 803007C06001C06000E0C000E0C00060C00060C00060C000606000406000C03000801803 000E0E0003F00013227EA018>56 D<01F000060C000C0600180700380380700380700380 F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003E03803E01805E0 0C05E00619E003E1E00001C00001C00001C0000380000380300300780700780600700C00 2018001030000FC00013227EA018>I<0001800000018000000180000003C0000003C000 0003C0000005E0000005E000000DF0000008F0000008F0000010F8000010780000107800 00203C0000203C0000203C0000401E0000401E0000401E0000800F0000800F0000FFFF00 0100078001000780030007C0020003C0020003C0040003E0040001E0040001E00C0000F0 0C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C00170 038000F0070000F00E0000701E0000701C0000303C0000303C0000307C00001078000010 78000010F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 78000000780000107C0000103C0000103C0000101C0000201E0000200E00004007000040 0380008001C0010000E0020000381C000007E0001C247DA223>67 D69 DI< FFFC0FC00780078007800780078007800780078007800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800FC0FFFC0E227EA1 12>73 D 77 DI 80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020E00020 E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E00001E0 0000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0C80180 C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008400780 08C007800C80078004800780048007800480078004000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000FC0 0003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700000700 00FF0007C7001E07003C0700780700700700F00708F00708F00708F00F087817083C2390 0FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E003C0E 001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C 80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000700000 F00000F00000F00000F00000F00000F00000F000007000007800403800401C00800C0100 07060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800E07800 E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800E01801 E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C07801E070 00E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C00400E 008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700070007 00070007000700FFF8070007000700070007000700070007000700070007000700070007 00070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C07001C 07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010000018 00001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860003060 00303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E00F0070 0F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000000000 0000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E001C000 00000000000000000000000000000001E00FE001E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0F1 8061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E02000E0400 0E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C00E01E0 0E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E00FFE00B237FA20E>I<0E1FC07F00FE60E183801E807201C0 0F003C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E00E003800E00E003800E0FFE3FF8FFE27157F942A>I<0E1F80FE60C01E80E00F00 700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E0038 0E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E0038 0F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E0000 0E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E 000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>III121 D E /Fl 20 118 df45 D68 D73 D77 D 80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E0780000E070 0000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F80000007C 0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000FFFF0000 0FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F8000000F880 00007880000078800000788000007880000078C0000078C0000070E00000F0E00000E0F0 0000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E001E 0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F001E 003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478003E 047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E0600380180700 040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000F800 00F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F00020700 0403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E00000 01E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E00000 01E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E00780 05E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0F800 01E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E07800 01E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF003F 81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C000F 003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F80000 00F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E0000 800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007E000 1C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF007F FF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E003C 001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E003C 000F007800070070000780E00009C1C000087F0000180000001800000018000000180000 00180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E0300000 F070000070E0000038E0000038E0000038E0000038E00000387000007070000070380000 E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F800700 000000000000000000000000000000000000000000000780FF80FF800F80078007800780 078007800780078007800780078007800780078007800780078007800780078007800780 0780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F000FF 8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E0007C0 00F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fm 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 908 a Fm(D)26 b(R)g(A)f(F)h(T)225 999 y Fl(Do)r(cumen)n(t) 20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n(terface)621 1194 y Fk(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum)802 1320 y(August)i(12,)f(1996)77 1378 y(This)h(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1436 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1494 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1552 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 45 a Fj(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fi(Chapter)34 b(12)75 564 y Fh(Miscellan)m(y)41 b(for)g(JOD)75 805 y Fg(12.1)59 b(Tw)n(o)20 b(Pro)r(cess)f("Sha)n(red") h(Bu\013ers)75 906 y Fj(It)15 b(is)g(not)g(uncommon)g(for)f(a)g(pair)i (of)e(pro)q(cesses)h(to)g(engage)f(in)i(frequen)o(t)f(pre-determined)h (comm)o(uni-)75 963 y(cations.)k(On)14 b(example)h(of)f(suc)o(h)h(a)f (co)q(de)h(is)f(a)g(pip)q(eline)k(algorithm)c(in)h(whic)o(h)g(pro)q (cess)f(A)h(creates)f(data)75 1019 y(that)i(is)h(subsequen)o(tly)h (used)f(b)o(y)f(pro)q(cess)h(B.)f(Another)g(example)i(is)f(a)f (\014nite)h(di\013erence)h(appro)o(xima-)75 1076 y(tion)13 b(mo)q(del)h(when)g(comm)o(unicating)g(b)q(oundary)f(v)m(alues)h (\(a.k.a.)k(ghost)12 b(p)q(oin)o(ts\).)19 b(On)14 b(a)f(mac)o(hine)h (with)75 1132 y(shared)k(memory)l(,)g(it)g(is)h(p)q(ossible)h(to)d (share)h(a)f(common)h(bu\013er)g(to)f(use)i(for)e(exc)o(hanging)i(suc)o (h)f(data.)75 1188 y(Ownership)c(of)f(the)g(bu\013er)g(m)o(ust)f(b)q(e) i(established)g(in)g(order)f(for)f(a)h(pro)q(cess)g(to)f(safely)h (\014ll)i(or)d(empt)o(y)h(the)75 1245 y(bu\013er.)25 b(This)17 b(can)g(b)q(e)g(expressed)h(as)e(a)h(pro)q(duce/comsume)g (relationship.)26 b(This)18 b(functionalit)o(y)g(can)75 1301 y(b)q(e)c(expressed)f(in)h(suc)o(h)f(a)g(w)o(a)o(y)f(that)g(it)i (should)g(b)q(e)f(no)g(less)h(e\016cien)o(t)f(when)h(using)g(MPI)f (Send/Recv)h(on)75 1358 y(a)h(distributed)i(memory)d(mac)o(hine)i(and)g (is)f(simply)i(a)e(p)q(oin)o(ter)g(pass)g(on)g(shared)h(memory)e(mac)o (hines.)166 1455 y Ff(Discussion:)34 b Fe(This)13 b(prop)q(osal)g(is)h (prompted)f(b)o(y)g(T)m(on)o(y)g(Skjellum's)e(prop)q(osal)i(titled:)18 b("In\014nite)13 b(asymp-)75 1505 y(totic)h(bandwidth)f(for)h(thread)h (comm)o(uni)o(cation")166 1555 y(W)m(e)j(ha)o(v)o(e)h(a)g(similia)o(r)e (proto)q(col)h(inside)h(our)g(system)g(when)g(w)o(e)g(w)o(ork)g(on)f (shared)i(memory)c(mac)o(hines.)75 1604 y(Imp)q(ortan)o(t)i(to)i(us)g (is)f(not)h(only)e(the)j(bandwidth)e(and)g(latency)m(,)i(but)e(also)g (ligh)o(t)o(w)o(eigh)o(t)f(\015o)o(w-con)o(trol.)35 b(The)75 1654 y(prop)q(osal)14 b(in)g(its)g(curren)o(t)i(form)c(solv)o(es)j(our) f(problem)f(in)h(particular,)g(but)g(w)o(e)h(feel)f(it)g(ma)o(y)e (prompt)h(discussion)75 1704 y(that)i(can)f(shap)q(e)i(this)f(prop)q (osal)f(and)g(T)m(on)o(y's)g(prop)q(osal)g(in)o(to)g(something)f(that)i (will)e(pro)o(vide)i(a)f(p)q(ortable)h(high)75 1754 y(bandwidth/lo)o (w-latency)c(mec)o(hanism)e(on)j(true)h(SMPs)f(as)h(w)o(ell)e(as)h(pro) o(vide)g(high)f(p)q(erformance)h(in)f(distributed)75 1804 y(memory)g(mac)o(hines.)166 1853 y(Questions)k(for)e(discussion:) 166 1903 y(1\))i(Is)h(this)f(really)g(going)f(to)h(b)q(e)h(faster)g(on) f(a)g(shared)h(memory)d(mac)o(hine)h(with)h(a)g("high-qualit)o(y")e (imple-)75 1953 y(men)o(tation.)166 2003 y(2\))h(Do)q(es)g(the)g(user)h (visible)f(\015o)o(w-con)o(trol)f(pro)o(vide)g(signi\014can)o(t)h (added)g(v)n(alue?)166 2053 y(3\))19 b(F)m(or)f(b)q(oth)h(shared)g(and) g(distributed)g(memory)e(mac)o(hines,)h(do)q(es)i(prede\014ning)f(the)g (comm)o(unication)75 2102 y(pattern)c(pro)o(vide)e(options)h(for)f (optimization?)75 2252 y Fd(Example:)75 2365 y(Process)23 b(0:)668 b(Process)23 b(1:)75 2478 y(!)h(Make)f(a)h(shared)f(buffer)g (with)g(process)g(0)h(as)f(the)h(source,)f(process)g(1)g(as)h(the)f (dest)75 2534 y(!)h(sbh)f(is)h(the)f(shared)g(buffer)g(handle)g(and)h (buff)f(is)h(some)f(memory)g(allocated)75 2591 y(!)h(by)f (MAKE_SHARED_BUFFER)75 2647 y(MAKE_SHARED_BUFFER)e(\(...,)334 b(MPI_MAKE_SHARED_BUFFER)21 b(\(...,)218 2704 y(0,)j(1,)f(buff,)h (sbh\))619 b(0,)24 b(1,)g(buff,)f(sbh\))964 2828 y Fj(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fj(2)917 b Fc(CHAPTER)15 b(12.)30 b(MISCELLANY)16 b(F)o(OR)g(JOD)75 45 y Fd(!)24 b(Process)f(0)g(owns)h(the)f(buffer)262 b(!)24 b(Process)e(1)i(waits)f(for)h(buffer)99 102 y(...Fill)f(buff)g (with)g(useful)g(stuff...)95 b(ACQUIRE_SHARED_BUFFER)21 b(\(sbh\))75 214 y(!)j(Process)f(1)g(releases)g(the)h(buffer)75 271 y(!)g(communicating)e(that)h(it)h(was)75 327 y(!)g(modified)e (\(PRODUCEd\))75 384 y(RELEASE_SHARED_BUFFER)f(\(sbh,)i(PRODUCE\))982 440 y(!)h(At)f(this)g(point)h(process)f(1)g(unblocks)75 497 y(!)h(Wait)f(for)g(process)g(0)h(to)g(use)f(the)167 b(!)24 b(and)f(has)g(full)h(access)f(to)g(buff)75 553 y(!)h(Information)e(produced)h(before)214 b(...Read)23 b(buff)g(and)g(do)h(something...)75 610 y(!)g(creating)e(more)i(data)75 666 y(ACQUIRE_SHARED_BUFFER)d(\(sbh\))262 b(!)24 b(Process)e(1)i (releases)f(the)g(buffer,)g(but)982 723 y(!)h(has)f(not)g(modified)g (it)h(\(CONSUMEd\))982 779 y(RELEASE_SHARED_BUFFER)d(\(sbh,)i (CONSUME\))75 835 y(!)h(At)f(this)h(point)f(process)g(0)g(unblocks)75 892 y(!)h(and)f(is)h(free)f(to)h(modify)f(buff)314 948 y(...)739 b(...)75 1005 y(FREE_SHARED_BUFF\(sbh\))403 b(FREE_SHARED_BUFF\(sbh\))75 1201 y Fb(MPI)p 160 1201 14 2 v 16 w(MAKE)p 305 1201 V 16 w(SHARED)p 497 1201 V 17 w(BUFFER)16 b(\(dataT)l(yp)q(e,)f(count,)h(comm,)c(source,)j (dest,)h(bu\013er,)e(sha)o(redBu\013Han-)75 1258 y(dle\))117 1335 y Fe(IN)155 b Fb(dataT)l(yp)q(e)412 b Fe(MPI)14 b(Data)g(T)o(yp)q(e)g(\(con)o(tiguous\))117 1409 y(IN)155 b Fb(count)482 b Fe(Num)o(b)q(er)14 b(of)f(dataT)o(yp)q(es)117 1484 y(IN)155 b Fb(comm)470 b Fe(Comm)o(unicator)117 1558 y(IN)155 b Fb(source)468 b Fe(First)14 b(pro)q(cess)i(in)e(pair)f (\(initially)f(o)o(wns)h(bu\013er\))117 1633 y(IN)155 b Fb(dest)511 b Fe(Second)15 b(pro)q(cess)h(in)d(pair)117 1707 y(IN)155 b Fb(tag)526 b Fe(T)m(ag)13 b(for)h(use)g(in)g(comm)o (unicatio)o(ns)e(and)h(for)h(uniqueness)117 1782 y(OUT)108 b Fb(bu\013er)478 b Fe(Bu\013er)15 b(creaated)h(b)o(y)d(mak)o(e)p 1352 1782 13 2 v 14 w(shared)p 1483 1782 V 16 w(bu\013er)117 1856 y(OUT)108 b Fb(sha)o(redBu\013Handle)256 b Fe(Handle)14 b(to)g(describ)q(e)h(shared)g(ob)r(ject)75 2026 y Fd(int)23 b(MPI_Make_shared_buffer)e(\(MPI_Datatype)h(dataType,)h(int,)g(count,) 361 2083 y(MPI_Comm)g(comm,)g(int)h(source,)f(int)g(dest,)361 2139 y(void)h(*buffer,)e(MPI_Shared_buff)g(*sharedBuffHandle\))75 2252 y(MPI_MAKE_SHARED_BUFFER)f(\(DATA_TYPE,)h(COUNT,)h(COMM,)g (SOURCE,)g(DEST,)624 2308 y(BUFF_REF,)f(BUFFER_IX,)h (SHARED_BUFF_HANDLE,)e(IERROR\))147 2365 y(INTEGER)h(DATA_TYPE)147 2421 y(INTEGER)g(COUNT)147 2478 y(INTEGER)g(COMM)147 2534 y(INTEGER)g(SOURCE)147 2591 y(INTEGER)g(DEST)147 2647 y(INTEGER)g(BUFF_REFF\(*\))147 2704 y(INTEGER)g(BUFFER_IX)1967 46 y Fa(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fc(12.1.)34 b(TW)o(O)15 b(PR)o(OCESS)g("SHARED")h (BUFFERS)823 b Fj(3)147 45 y Fd(INTEGER)22 b(SHARED_BUFF_HANDLE)147 102 y(INTEGER)g(IERROR)166 208 y Fj(This)f(routine)h(creates)e(a)h (bu\013er)g(shared)g(b)q(et)o(w)o(een)g(t)o(w)o(o)e(pro)q(cesses.)38 b(The)21 b(pro)q(cess)g(iden)o(ti\014ed)75 264 y(b)o(y)c(source)g (initially)j("o)o(wns")c(the)h(bu\013er.)26 b(Source)18 b(ma)o(y)e(write)h(to)g(the)g(bu\013er)g(and)h(manipulate)g(the)75 321 y(bu\013er)c(in)i(an)o(y)e(w)o(a)o(y)f(that)h(mak)o(es)g(sense)h (with)g(standard)f(language)h(access)f(\(assignmen)o(t)g(statemen)o (ts,)75 377 y(functions)19 b(calls,)g(etc.\).)28 b(When)19 b(source)f(is)g(done)h(with)f(the)g(bu\013er,)h(it)f(m)o(ust)g(release) g(the)h(bu\013er)f(for)75 434 y(dest)i(to)f(acquire)h(it.)34 b(Dest)19 b(\(and)h(only)g(dest\))f(can)h(acquire)h(the)e(bu\013er)h (once)g(released)h(b)o(y)e(source.)75 490 y(Dest)d(then)h(has)f(the)g (same)h(priviledgs)h(as)e(source)g(previously)i(had.)24 b(T)l(o)16 b(access)h(a)f(released)h(bu\013er)f(is)75 547 y(considered)h(to)d(b)q(e)i(erroneous.)k(This)c(function)g(needs)g (to)e(b)q(e)i(view)o(ed)g(as)f("bi-collectiv)o(e".)166 650 y Ff(Discussion:)40 b Fe(Do)15 b(w)o(e)h(w)o(an)o(t)f(to)h(ha)o(v)o (e)f(a)g(v)o(ersion)h(that)f(allo)o(ws)g(the)h(user)g(to)g(supply)f(a)g (bu\013er.)24 b(In)16 b(that)75 707 y(ev)o(en)o(t)f(on)f(a)g (distributed)h(memory)d(mac)o(hine)h(this)i(b)q(ecomes)g(a)f("safe")g (in)o(terface)h(to)f(ready)h(send,)g(on)f(a)g(shared)75 763 y(memory)d(mac)o(hine)i(it)g(ma)o(y)f(in)o(tro)q(duce)j(extra)f (copies.)75 961 y Fb(MPI)p 160 961 14 2 v 16 w(A)o(CQUIRE)p 367 961 V 17 w(SHARED)p 560 961 V 18 w(BUFFER)i(\(sha)o (redBu\013Handle\))117 1039 y Fe(INOUT)62 b Fb(sha)o(redBu\013Handle) 256 b Fe(A)14 b(shared)h(bu\013er)g(handle)75 1213 y Fd(int)23 b(MPI_Acquire_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle\))75 1326 y(MPI_ACQUIRE_SHARED_BUFFER)e (\(SHARED_BUFF_HANDLE,)i(IERROR\))147 1382 y(INTEGER)g (SHARED_BUFF_HANDLE)147 1439 y(INTEGER)g(IERROR)166 1545 y Fj(This)16 b(routine)f(w)o(aits)g(for)g(the)g(shared)g(bu\013er)g(to) g(b)q(e)h(released)g(b)o(y)f(the)g(partner.)20 b(It)15 b(is)h(a)f(blo)q(c)o(king)75 1601 y(call.)38 b(When)21 b(it)g(returns,)g(the)g(bu\013er)g(asso)q(ciated)g(with)g(the)g(shared) g(bu\013er)f(handle)i(can)f(b)q(e)h(fully)75 1658 y(accessed.)e(A)12 b(subsequen)o(t)h(acquire)g(without)f(an)h(in)o(terv)o(ening)g(release) g(is)g(erroneous)f(\(this)g(includes)j(an)75 1714 y(acquire)g(b)o(y)g (the)g(source)f(immediately)j(after)d(the)g(mak)o(e)p 1061 1714 V 16 w(shared)p 1206 1714 V 17 w(bu\013er\).)19 b(An)c(acquire)g(will)i(correctly)75 1771 y(return)e(only)h(when)g(the) f(partner)g(pro)q(cess)g(p)q(erforms)g(a)g(release.)75 1922 y Fb(MPI)p 160 1922 V 16 w(RELEASE)p 366 1922 V 17 w(SHARED)p 559 1922 V 18 w(BUFFER)h(\(sha)o(redBu\013Handle,)g (releaseAction\))117 1999 y Fe(INOUT)62 b Fb(sha)o(redBu\013Handle)256 b Fe(A)14 b(shared)h(bu\013er)g(handle)117 2074 y(IN)155 b Fb(releaseAction)338 b Fe(Indicate)11 b(whether)g(a)f(pro)q(duce)h (\(MPI)p 1501 2074 13 2 v 16 w(PR)o(ODUCE\))f(or)g(con-)905 2130 y(sume)i(\(MPI)p 1109 2130 V 15 w(CONSUME\))i(action)e(is)g(asso)q (ciated)h(with)f(the)905 2187 y(release)75 2361 y Fd(int)23 b(MPI_Release_shared_buffer)e(\(MPI_Shared_buff)h(*sharedBuffHandle,) 314 2418 y(int)h(releaseAction\))75 2531 y(MPI_RELEASE_SHARED_BUFFER)d (\(SHARED_BUFF_HANDLE,)i(RELEASE_ACTION,)f(IERROR\))147 2587 y(INTEGER)h(SHARED_BUFF_HANDLE)147 2643 y(INTEGER)g (RELEASE_ACTION)147 2700 y(INTEGER)g(IERROR)-32 46 y Fa(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fj(4)917 b Fc(CHAPTER)15 b(12.)30 b(MISCELLANY)16 b(F)o(OR)g(JOD)166 45 y Fj(This)11 b(routine)h(releases)f(a)g(shared)g (bu\013er)g(so)f(that)g(it)h(can)g(b)q(e)h(acquired)g(b)o(y)f(the)f (partner)h(pro)q(cess.)19 b(If)75 102 y(the)11 b(pro)q(cess)h(has)f (only)h(read)f(the)g(bu\013er)h(asso)q(ciated)f(with)h(the)f(shared)g (bu\013er)h(handler,)g(it)g(ma)o(y)e(use)i(the)75 158 y(releaseAction)17 b(of)f(MPI)p 500 158 14 2 v 16 w(CONSUME.)g(If)g (the)h(pro)q(cess)f(has)g(written)g(to)f(the)h(bu\013er)g(asso)q (ciated)h(with)75 214 y(the)d(shared)g(bu\013er)h(handler,)g(it)f(MUST) g(use)g(the)h(releaseAction)g(of)f(MPI)p 1369 214 V 16 w(PR)o(ODUCE)g(to)g(guaran)o(tee)75 271 y(that)g(the)i(partner)f(sees)g (the)g(c)o(hanges.)75 422 y Fb(MPI)p 160 422 V 16 w(FREE)p 285 422 V 17 w(SHARED)p 478 422 V 17 w(BUFFER)h(\(sha)o (redBu\013Handle\))117 499 y Fe(INOUT)62 b Fb(sha)o(redBu\013Handle)256 b Fe(A)14 b(shared)h(bu\013er)g(handle)75 673 y Fd(int)23 b(MPI_Free_shared_buffer)e(\(MPI_Shared_buff)h(*sharedBuffHandle\))75 786 y(MPI_FREE_SHARED_BUFFER)f(\(SHARED_BUFF_HANDLE,)g(IERROR\))147 843 y(INTEGER)h(SHARED_BUFF_HANDLE)147 899 y(INTEGER)g(IERROR)166 1005 y Fj(This)e(function)h(frees)e(the)h(bu\013er)g(asso)q(ciated)f (with)h(the)g(shared)g(bu\013er)g(handler,)h(the)f(handle)75 1062 y(itself,)d(and)g(satis\014es)g(an)o(y)f(outstanding)h(comm)o (unications.)25 b(This)17 b(function)g(needs)h(to)e(b)q(e)h(view)o(ed)g (as)75 1118 y("bi-collectiv)o(e".)75 1225 y Fd(Possible)23 b(Send/Recv)f(Implementation:)75 1338 y(int)h(MPI_Make_shared_buffer)e (\(MPI_Datatype)h(dataType,)h(int,)g(count,)361 1394 y(MPI_Comm)g(comm,)g(int)h(source,)f(int)g(dest,)361 1450 y(void)h(*buffer,)e(MPI_Shared_buff)g(*sharedBuffHandle\))123 1507 y({)123 1563 y(/*)h(Ok,)h(ok)f(this)h(isn't)f(really)g(C,)g(but)h (you)f(get)h(the)f(idea\))g(*/)123 1620 y(sharedBuffHandle)e(=)j (malloc)f(\(sizeof\(sharedBuffHandle\)\);)123 1676 y(*buffer)g(=)g (malloc)g(\(sizeof\(dataType\))f(*)i(count\);)123 1733 y(sharedBuffHandle->buffer)c(=)k(buffer;)123 1789 y (sharedBuffHandle->count)c(=)k(count;)123 1846 y (sharedBuffHandle->dataTy)o(pe)d(=)j(dataType;)e(/*)i(Dup)f(here?)h(*/) 123 1902 y(sharedBuffHandle->tag)d(=)i(getSharedBuffTag\(\);)f(/*)h (Some)h(magic?)f(*/)123 1959 y(/*)g(At)h(this)f(point,)g(a)h(robust)f (application)f(might)i(check)f(with)g(it's)g(partner)194 2015 y(to)h(see)f(if)h(the)f(parameters)g(are)g(correct)g(and)h(the)f (malloc)g(worked.)g(But)h(it)194 2071 y(would)f(take)h(too)f(much)h (space)f(\(besides)g(if)g(programming)f(text)i(books)194 2128 y(can)g(omit)f(error)g(checking,)g(why)g(can't)h(I?)f(:-\))h(*/) 123 2184 y(if)f(\(myProc)g(==)h(src\))170 2241 y({)170 2297 y(Recv)g(\(comm,)f(sharedBuffHandle->tag,)e(dest,)i(\(count=0,)f (buff=NULL,...\)\);)170 2354 y(sharedBuffHandle->partner)f(=)j(dest;) 170 2410 y(})123 2467 y(else)170 2523 y({)170 2580 y (sharedBuffHandle->waitHandle)c(=)k(IRECV)f(\(comm,)g (sharedBuffHandle->tag,)1077 2636 y(source,)g(buffer,)g(count,...\))170 2692 y(Send)h(\(comm,)f(sharedBuffHandle->tag,)e(source,)i(\(count=0,)f (buff=NULL,...\)\);)1967 46 y Fa(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fc(12.1.)29 b(TW)o(O)15 b(PR)o(OCESS)g("SHARED")h (BUFFERS)828 b Fj(5)170 45 y Fd(sharedBuffHandle->partner)21 b(=)j(source;)170 102 y(})123 158 y(return;)123 214 y(})75 327 y(int)f(MPI_Acquire_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle\))123 384 y({)123 440 y(wait)h (\(sharedBuffHandle->waitHan)o(dle\);)123 497 y(})75 610 y(int)g(MPI_Release_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle,)314 666 y(int)h(releaseAction\))123 723 y({)123 779 y(if)g(\(releaseAction)f(==)i(MPI_CONSUME\))170 835 y(count)g(=)f(0;)123 892 y(else)170 948 y(count)h(=)f (sharedBuffHandle->count;)123 1005 y(sharedBuffHandle->waitHa)o(ndle)e (=)i(IRECV)h(\(comm,)f(sharedBuffHandle->tag,)1030 1061 y(sharedBuffHandle->partn)o(er,)1030 1118 y(sharedBuffHandle->buffe)o (r,)e(count,...\);)123 1174 y(/*)i(Note)h(that)f(a)h(non-buffering)e (send)h(can)h(be)f(used)g(here)h(*/)123 1231 y(/*)f(Note)h(that)f(a)h ("ready")f(protocol)f(can)i(also)f(be)h(used)f(since)g(we)h(guarantee) 194 1287 y(the)g(receive)f(is)g(always)g(posted)g(before)g(the)h(send)f (*/)123 1344 y(localHandle)f(=)i(ISSEND)f(\(comm,)g (sharedBuffHandle->tag,)e(sharedBuffHandle->partne)o(r,)648 1400 y(sharedBuffHandle->buffer)o(,)g(count,...\);)123 1456 y(freeHandle)h(\(localHandle\);)123 1513 y(})75 1626 y(int)h(MPI_Free_shared_buffer)e(\(MPI_Shared_buff)h (*sharedBuffHandle\))123 1682 y({)123 1739 y(/*)170 1795 y(Either)h(cancel)h(the)f(outstanding)f(receive)h(or)h(\(if)f(you)h (don't)f(beleive)170 1852 y(cancel)g(works\))h(send)f(to)g(complete)g (the)h(receive.)123 1908 y(*/)123 1965 y(/*)170 2021 y(Free)g(all)f(structures)123 2077 y(*/)123 2134 y(})75 2240 y(Possible)g(Shared)g(Memory)g(Implementation:)75 2353 y(int)g(MPI_Make_shared_buffer)e(\(MPI_Datatype)h(dataType,)h (int,)g(count,)361 2410 y(MPI_Comm)g(comm,)g(int)h(source,)f(int)g (dest,)361 2466 y(void)h(*buffer,)e(MPI_Shared_buff)g (*sharedBuffHandle\))123 2522 y({)123 2579 y(/*)h(Ok,)h(ok)f(this)h (isn't)f(really)g(C,)g(but)h(you)f(get)h(the)f(idea\))g(*/)123 2635 y(sharedBuffHandle)e(=)j(malloc)f(\(sizeof\(sharedBuffHandle\)\);) 123 2692 y(if)g(\(myProc)g(==)h(source\))-32 46 y Fa(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fj(6)912 b Fc(CHAPTER)15 b(12.)35 b(MISCELLANY)16 b(F)o(OR)g(JOD)170 45 y Fd({)170 102 y(sharedBuffHandle->semaphore)21 b(=)457 158 y(shared_malloc)h(\(sizeof\(dataType\))f(*)j(count)f(+)h (cacheLineSize\);)170 214 y(})123 271 y(sharedBuffHandle->buffer)c(=)k (buffer;)123 327 y(sharedBuffHandle->count)c(=)k(count;)123 384 y(sharedBuffHandle->dataTy)o(pe)d(=)j(dataType;)e(/*)i(Dup)f(here?) h(*/)123 440 y(if)f(\(myProc)g(==)h(source\))170 497 y({)170 553 y(i)g(=)g(myProc;)170 610 y(memcpy)f (\(sharedBuffHandle->semaphore,)d(&i,)k(sizeof\(int\)\);)170 666 y(sharedBuffHandle->partner)d(=)j(dest;)170 723 y(Send)g(\(comm,)f (sharedTag,)f(dest,)h(sharedBuffHandle->semaphore,)e(MPI_Pointer,)h (...\);)170 779 y(})123 835 y(else)170 892 y({)170 948 y(sharedBuffHandle->partner)f(=)j(source;)170 1005 y(Recv)g(\(comm,)f (sharedTag,)f(source,)h(sharedBuffHandle->semaphore,)457 1061 y(MPI_Pointer,)f(...\);)170 1118 y(})123 1174 y(*buffer)h(=)g (\(\(char)g(*\))h(sharedBuffHandle->semaphor)o(e\))d(+)j (cacheLineSize;)123 1231 y(})75 1344 y(int)f(MPI_Acquire_shared_buffer) e(\(MPI_Shared_buff)h(*sharedBuffHandle\))123 1400 y({)123 1456 y(i)h(=)h(myProc)f(-)h(1;)123 1513 y(while)f(\(i)g(!=)h(myProc\)) 170 1569 y({)170 1626 y(memcpy)f(\(&i,)h(sharedBuffHandle->semapho)o (re,)d(sizeof\(int\)\);)170 1682 y(/*)j(A)g(smart)f(implementation)f (would)h(release)g(it's)g(timeslice)242 1739 y(after)g(some)h(number)f (of)g(times)g(through)g(this)h(loop)f(*/)170 1795 y(})123 1852 y(})75 1965 y(int)g(MPI_Release_shared_buffer)e(\(MPI_Shared_buff) h(*sharedBuffHandle,)314 2021 y(int)h(releaseAction\))123 2077 y({)123 2134 y(i)g(=)h(sharedBuffHandle->partner;)123 2190 y(memcpy)f(\(sharedBuffHandle->semap)o(hore,)e(&i,)i (sizeof\(int\)\);)123 2247 y(})75 2360 y(int)g(MPI_Free_shared_buffer)e (\(MPI_Shared_buff)h(*sharedBuffHandle\))123 2416 y({)123 2473 y(/*)170 2529 y(Free)i(all)f(structures)123 2586 y(*/)123 2642 y(})1967 46 y Fa(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fc(12.2.)34 b(YET)15 b(ANOTHER)h(SEND)f(MODE)1002 b Fj(7)75 45 y Fg(12.2)59 b(Y)-5 b(et)19 b(Another)g(Send)g(Mo)r(de)75 147 y Fj(In)13 b(the)f(1-sided)i(c)o(hapter,)e(w)o(e)g(ha)o(v)o(e)g(in) o(tro)q(duced)i(the)f(notion)f(of)g(\\go)q(o)q(d")g(memory)g(vs.)18 b(\\bad")13 b(memory)75 203 y(for)j(gets)g(and)h(puts.)25 b(The)16 b(idea)i(is)f(that)f(if)h(y)o(our)f(system)h(supp)q(orts)f (shared)h(memory)l(,)f(then)h(y)o(ou)g(can)75 259 y(call)e Fb(MPI)p 242 259 14 2 v 16 w(RMA)p 357 259 V 17 w(MALLOC)f Fj(to)f(request)h(a)g(shared)g(bu\013er)g(and)g(as)g(a)g(rew)o(ard)f (for)h(doing)g(this)h(y)o(our)e(gets)75 316 y(and)i(puts)h(will)h(b)q (e)e(faster.)166 372 y(The)20 b(though)o(t)e(o)q(ccurs)i(that)f (2-sided)i(comm)o(unication)f(could)g(b)q(ene\014t)h(from)e(this)h(as)f (w)o(ell.)33 b(In)75 429 y(MPI-1,)16 b(for)f(example,)i(man)o(y)f (shared)g(memory)f(implemen)o(tations)j(still)f(use)f(a)g(t)o(w)o (o-cop)o(y)f(algorithm)75 485 y(to)g(pass)h(messages)g(around;)g(the)g (sender)h(copies)g(the)f(data)g(in)o(to)g(a)g(shared)g(bu\013er,)g(and) g(the)g(receiv)o(er)75 542 y(copies)e(it)f(out)f(again.)20 b(While)14 b(some)e(systems)h(are)f(able)i(to)e(do)h(b)q(etter)g(than)g (this)g(in)h(some)e(cases,)h(man)o(y)75 598 y(are)i(not.)166 655 y(So,)i(giv)o(en)g(that)g(w)o(e)f(already)h(are)g(going)g(to)f(b)q (e)i(adding)g Fb(MPI)p 1253 655 V 16 w(RMA)p 1368 655 V 16 w(MALLOC)f Fj(to)f(MPI-2,)h(wh)o(y)75 711 y(not)e(also)g(add)g(a)g (new)h(send)f(mo)q(de)h(to)e(tak)o(e)h(adv)m(an)o(tage)g(of)f(it?)21 b(Something)16 b(lik)o(e)g(this:)266 817 y Fd(MPI_Xsend)266 874 y(MPI_Ixsend)266 930 y(MPI_Xsend_init)70 b(\(We)23 b(can)g(debate)h(the)f(proper)g(value)g(of)h('X')f(later...\))166 1037 y Fj(In)14 b(this)f(mo)q(de,)h(the)f(sender)h(asserts)f(that)f (the)i(bu\013er)f(b)q(eing)h(passed)g(is)g(one)f(whic)o(h)h(w)o(as)f (returned)75 1093 y(b)o(y)18 b(a)g(call)i(to)d Fb(MPI)p 412 1093 V 16 w(RMA)p 527 1093 V 17 w(MALLOC)p Fj(.)h(It)g(otherwise)g (has)h(the)f(same)g(seman)o(tics)g(as)g(standard)g(mo)q(de.)75 1149 y(With)c(this)g(mo)q(de,)g(pro)q(cesses)g(calling)h Fb(MPI)p 834 1149 V 16 w(RECV)f Fj(on)g(shared)g(memory)f(mac)o(hines)h (w)o(ould)g(b)q(e)g(able)h(to)75 1206 y(cop)o(y)i(the)h(data)e (directly)j(out)e(of)g(the)h(send)g(bu\013er.)26 b(There)18 b(could)g(also)f(b)q(e)h(adv)m(an)o(tages)f(on)h(NO)o(Ws,)75 1262 y(b)q(ecause)f(\(for)f(example\))h(high)g(p)q(erformance)f (implemen)o(tations)i(migh)o(t)e(opt)g(to)f(pin)j(do)o(wn)e(the)g(send) 75 1319 y(bu\013ers)i(in)g(adv)m(ance)h(to)e(prepare)h(for)f(subsequen) o(t)i(DMAs.)27 b(Mac)o(hines)18 b(not)f(capable)i(of)e(optimizing)75 1375 y(this)g(case)g(could)g(simply)h(use)f Fb(MPI)p 697 1375 V 16 w(SEND)g Fj(to)f(implemen)o(t)i(the)e(new)h(calls,)h(m)o (uc)o(h)e(as)h(often)f(happ)q(ens)75 1432 y(to)q(da)o(y)f(with)g Fb(MPI)p 391 1432 V 16 w(RSEND)p Fj(.)166 1488 y(Adding)d(additional)h (smarts)d(to)h(the)g(already-existing)i(send)f(mo)q(des)g(is)f (probably)h(not)f(a)g(practical)75 1545 y(solution)19 b(b)q(ecause)g(of)e(the)h(added)h(latency)f(that)g(w)o(ould)g(b)q(e)h (in)o(v)o(olv)o(ed)g(in)f(building)j(in)o(ternal)e(lists)f(of)75 1601 y(shared)d(bu\013ers)g(and)h(comparing)f(against)g(them)g(with)h (ev)o(ery)f(sen)o(t)g(message.)166 1658 y(Doing)d(this)h(w)o(ould)g (probably)h(in)o(v)o(olv)o(e)f(making)f(some)h(minor)f(c)o(hanges)h(to) f Fb(MPI)p 1552 1658 V 16 w(RMA)p 1667 1658 V 17 w(MALLOC)p Fj(.)75 1714 y(F)l(or)g(example,)i(it)f(w)o(ould)g(probably)g(ha)o(v)o (e)g(to)f(b)q(ecome)h(a)g(collectiv)o(e)h(function,)g(and)f(w)o(e)f(w)o (ould)h(p)q(erhaps)75 1770 y(w)o(an)o(t)h(to)h(t)o(w)o(eak)f(the)h(syn) o(tax)f(sligh)o(tly)l(.)166 1827 y(Do)q(es)h(an)o(y)o(one)g(else)h (think)g(that)e(this)i(idea)g(has)f(merit?)-32 46 y Fa(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Fri Aug 16 16:38:57 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA10838; Fri, 16 Aug 1996 16:38:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA24789 for mpi-core-out; Fri, 16 Aug 1996 15:41:19 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA24727; Fri, 16 Aug 1996 15:40:04 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id NAA06025; Fri, 16 Aug 1996 13:40:01 -0700 From: "William C. Saphir" Message-Id: <9608161340.ZM6023@win233.nas.nasa.gov> Date: Fri, 16 Aug 1996 13:40:00 -0700 X-Mailer: Z-Mail (3.2.2 10apr95 MediaMail) To: mpi-dynamic@mcs.anl.gov Subject: new dynamic chapter Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, The next message will contain a new version of the dynamic chapter. I apologize for the length of time it took to get the chapter out. The new chapter has a large number of small changes from the previous version. The biggest changes have to do with splitting the command_line argument into command and argv, and making the independent routines not collective. Many other things have been clarified (hopefully). Steve wants a version by Monday, for which I will add C++ bindings. I will also incorporate any changes received by Sunday night. The current plan is to take second and final votes at the next meeting, so please read carefully. Bill From mpi-core-human@mcs.anl.gov Fri Aug 16 16:44:06 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA10855; Fri, 16 Aug 1996 16:42:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA24858 for mpi-core-out; Fri, 16 Aug 1996 15:45:41 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA24795; Fri, 16 Aug 1996 15:42:04 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.6.12/NAS.6.1) id NAA06033; Fri, 16 Aug 1996 13:42:01 -0700 From: "William C. Saphir" Message-Id: <9608161342.ZM6031@win233.nas.nasa.gov> Date: Fri, 16 Aug 1996 13:42:01 -0700 X-Mailer: Z-Mail (3.2.2 10apr95 MediaMail) To: mpi-dynamic@mcs.anl.gov Subject: postscript for new dynamic chapter Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: main.dvi %%Pages: 42 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: /usr/local/bin/dvips -o main.ps main.dvi %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.08.16:1335 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (main.dvi) @start /Fa 1 81 df80 D E /Fb 4 62 df<03000700FF000700070007000700070007 00070007000700070007000700070007000700070007007FF00C157E9412>49 D<0F8030E040708030C038E0384038003800700070006000C00180030006000C08080810 183FF07FF0FFF00D157E9412>I<0FE030306018701C701C001C00180038006007E00030 0018000C000E000EE00EE00EC00C401830300FE00F157F9412>I61 D E /Fc 6 118 df<07C00C20107020706000C000C000C00080008000C010C02060C03F 000C0E7E8D0F>99 D<0300038003000000000000000000000000001C002400460046008C 000C0018001800180031003100320032001C0009177F960C>105 D<383C0044C6004702004602008E06000C06000C06000C0C00180C00180C401818401818 80300880300F00120E7F8D15>110 D<07C00C20101020186018C018C018C01880308030 C060C0C061803E000D0E7E8D11>I<030003000600060006000600FFC00C000C000C0018 00180018001800300030803080310031001E000A147F930D>116 D<1C0200260600460600460600860C000C0C000C0C000C0C001818001818801818801838 800C5900078E00110E7F8D14>I E /Fd 10 111 df<70F8F8F87005057C840D>58 D<70F8FCFC74040404080810102040060E7C840D>I<000001C00000078000001E000000 78000001E00000078000000E00000038000000F0000003C000000F0000003C000000F000 0000F00000003C0000000F00000003C0000000F0000000380000000E0000000780000001 E0000000780000001E0000000780000001C01A1A7C9723>I<00FF803FF0000F80078000 0F800200000BC00200000BC002000013C004000011E004000011E004000011E004000020 F008000020F008000020F808000020780800004078100000403C100000403C100000403C 100000801E200000801E200000801E200000800F200001000F400001000F4000010007C0 00010007C00002000780000200038000020003800006000380000F00010000FFE0010000 241F7E9E25>78 D<00F1800389C00707800E03801C03803C038038070078070078070078 0700F00E00F00E00F00E00F00E10F01C20F01C20703C20705C40308C400F078014147E93 18>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E 380F380F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207E9F 14>I<007C01C207010E0F1E0F1C0E3C04780078007800F000F000F000F000F000700170 02300418380FC010147E9314>I<00E001E001E000C00000000000000000000000000000 0E00130023804380438043808700070007000E000E001C001C001C203840384038403880 19000E000B1F7E9E10>105 D<1E07C07C00231861860023A032030043C0340300438038 0380438038038087007007000700700700070070070007007007000E00E00E000E00E00E 000E00E00E000E00E01C101C01C01C201C01C038201C01C038401C01C018403803801880 1801800F0024147E9328>109 D<1E07802318C023A06043C0704380704380708700E007 00E00700E00700E00E01C00E01C00E01C00E03821C03841C07041C07081C030838031018 01E017147E931B>I E /Fe 37 118 df<001C0000003E0000003E0000002E0000006700 000067000000E7800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0 000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C 0038001E0038001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F 00001E00003C00003C0000780000780000780000F00000F00000F00000F00000F00000F0 0000F00000F00000F000007800007800007800003C00003C00001E00000F000807801803 C07800FFF0003F80151F7D9D1B>I III<003F8001FFF003C0F80780380F00181E00003C00003C0000780000 780000780000F00000F00000F00000F00000F00000F00000F007F8F007F8F00038780038 7800387800383C00383C00381E00380F003807803803C0F801FFF0003F80151F7D9D1C> III75 DIII<003F000001FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F00 780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F80007C078000780780007807C000F803C000F003E001F001F003E00 0F807C0007C0F80003FFF00001FFE000003F00001A1F7E9D1F>II82 D<03F8000FFE001C0F00380700700300600000E00000E00000E00000E00000F000007800 007F00003FE0001FFC0007FE0001FF00001F800007800003C00003C00001C00001C00001 C00001C0C00180E00380F007007C0E001FFC0007F000121F7E9D17>IIIII<78000E007C001E003C003C001E003800 0F0070000F00F0000781E00003C1C00001C3C00001E7800000F70000007E0000003E0000 003C0000003C0000007E00000077000000E7800001E3800003C1C0000381E0000700F000 0F00F8000E0078001C003C003C003E0078001F0070000F00F0000F80191D7F9C1C>II<7FFFF07FFFF00001E00003E00003C00007C000078000 0F00001F00001E00003E00003C0000780000F80000F00001F00001E00003C00007C00007 80000F80000F00001E00003E00003C00007C0000780000FFFFF0FFFFF0141D7E9C19>I< 0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC3F1C 0E127E9114>97 D<07E00FF81FFC3C1C70047000E000E000E000E000E000E00070007004 3C1C1FFC0FF807E00E127E9112>99 D<000E000E000E000E000E000E000E000E000E000E 000E0F8E1FEE3FFE7C3E700E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE 0F8E0F1D7E9C15>I<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070 043C1C3FFC1FF807E00E127E9112>I<00FC01FC03FC07000E000E000E000E000E000E00 0E00FFE0FFE00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E1D809C0D>I108 DII<03F0000FFC001FFE003C0F 00780780700380E001C0E001C0E001C0E001C0E001C0F003C07003807807803C0F001FFE 000FFC0003F00012127F9115>I114 D<1FC03FF07FF0F030E000E000F0 007F003FC01FE000F0003800388038F078FFF07FE01FC00D127F9110>I<1C001C001C00 1C001C001C00FFE0FFE01C001C001C001C001C001C001C001C001C001C001C001C001C20 1FF00FF007C00C187F970F>II E /Ff 39 122 df<00003FE00000E010 00018038000380780003007800070030000700000007000000070000000E0000000E0000 000E000000FFFFE0000E00E0001C01C0001C01C0001C01C0001C01C0001C038000380380 00380380003803800038070000380700007007000070071000700E2000700E2000700E20 00E00E2000E0064000E0038000E0000000C0000001C0000001C000003180000079800000 F3000000620000003C0000001D29829F1A>12 D<0E0E1F1F3F3F3F3F1D1D010102020202 04040404080810102020C0C0100E799F17>34 D<0E1F3F3F1D0102020404081020C0080E 779F0E>39 D<1C3C3C3C3C040408081020204080060E7D840E>44 D<7FF0FFE07FE00C037D8A10>I<70F8F8F0E005057B840E>I<000200020006000E003C00 DC031C001C0038003800380038007000700070007000E000E000E000E001C001C001C001 C003800380038003800780FFF80F1E7B9D17>49 D<070F1F1F0E00000000000000000000 70F8F8F0E008147B930E>58 D<00000200000006000000060000000E0000001E0000001E 0000003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F0000020F 0000040F00000C0F0000080F0000100F0000100F0000200F80003FFF800040078000C007 800080078001000780010007800200078002000780060007801E000F80FF807FF81D207E 9F22>65 D<0000FE0200078186001C004C0038003C0060003C00C0001C01C00018038000 18070000180F0000181E0000101E0000103C0000003C0000007800000078000000780000 0078000000F0000000F0000000F0000000F0000000F00000807000008070000080700001 003800010038000200180004000C001800060020000381C00000FE00001F217A9F21>67 D<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C0004003C00 040078080800780800007808000078180000F0300000FFF00000F0300000F0300001E020 0001E0200001E0200001E0001003C0002003C0002003C0004003C0004007800080078001 8007800100078007000F001F00FFFFFE001F1F7D9E1F>69 D<01FFFFFC001E0038001E00 18001E0008001E0008003C0008003C0008003C0008003C00080078001000780800007808 000078080000F0100000F0300000FFF00000F0300001E0200001E0200001E0200001E020 0003C0000003C0000003C0000003C00000078000000780000007800000078000000F8000 00FFF800001E1F7D9E1E>I<01FFF3FFE0001F003E00001E003C00001E003C00001E003C 00003C007800003C007800003C007800003C007800007800F000007800F000007800F000 007800F00000F001E00000FFFFE00000F001E00000F001E00001E003C00001E003C00001 E003C00001E003C00003C007800003C007800003C007800003C007800007800F00000780 0F000007800F000007800F00000F801F0000FFF1FFE000231F7D9E22>72 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078000078 0000780000F00000F00000F00000F00001E00001E00001E00001E00003C00003C00003C0 0003C0000780000780000780000780000F8000FFF800141F7D9E12>I<01FFFF00001E03 C0001E00E0001E0070001E0078003C0078003C0078003C0078003C0078007800F0007800 F0007801E0007801C000F0070000F01E0000FFF00000F0380001E01C0001E01E0001E00E 0001E00F0003C01E0003C01E0003C01E0003C01E0007803C0007803C0807803C0807803C 100F801C10FFF00C20000007C01D207D9E21>82 D<0FFFFFF01E0780E018078020100780 2020078020200F0020600F0020400F0020400F0020801E0040001E0000001E0000001E00 00003C0000003C0000003C0000003C00000078000000780000007800000078000000F000 0000F0000000F0000000F0000001E0000001E0000001E0000001E0000003E00000FFFF00 001C1F789E21>84 D<020204040808101020204040404080808080B0B0F8F8F8F8F8F870 700F0E749F17>92 D<00F1800389C00707800E03801C03803C0380380700780700780700 780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C 9317>97 D<007E0001C1000300800E07801E07801C07003C0200780000780000780000F0 0000F00000F00000F00000F0000070010070020030040018380007C00011147C9315>99 D<0000780003F80000700000700000700000700000E00000E00000E00000E00001C00001 C000F1C00389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C01C2 07010E011C013C013802780C7BF07C00F000F000F000F0007000700170023804183807C0 10147C9315>I<00007800019C00033C00033C000718000700000700000E00000E00000E 00000E00000E0001FFE0001C00001C00001C00001C000038000038000038000038000038 0000700000700000700000700000700000700000E00000E00000E00000E00000C00001C0 0001C0000180003180007B0000F300006600003C00001629829F0E>I<003C6000E27001 C1E00380E00700E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C03803C 07003C07001C0F001C17000C2E0003CE00000E00000E00001C00001C00301C00783800F0 700060E0003F8000141D7E9315>I<01E0000FE00001C00001C00001C00001C000038000 038000038000038000070000070000071E000763000E81800F01C00E01C00E01C01C0380 1C03801C03801C0380380700380700380700380E10700E20700C20701C20700C40E00CC0 60070014207D9F17>I<00C001E001E001C000000000000000000000000000000E003300 230043804300470087000E000E000E001C001C001C003840388030807080310033001C00 0B1F7C9E0E>I<01E0000FE00001C00001C00001C00001C0000380000380000380000380 000700000700000703C00704200E08E00E11E00E21E00E40C01C80001D00001E00001FC0 0038E000387000387000383840707080707080707080703100E03100601E0013207D9F15 >107 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C00 1C001C0038003800380038007000700070007100E200E200E200E200640038000A207C9F 0C>I<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<00F0400388C00705800E03801C03803C038038070078 0700780700780700F00E00F00E00F00E00F00E00F01C00F01C00703C00705C0030B8000F 380000380000380000700000700000700000700000E00000E0000FFE00121D7C9315>I< 1C1E002661004783804787804707804703008E00000E00000E00000E00001C00001C0000 1C00001C000038000038000038000038000070000030000011147C9313>I<00FC030206 010C030C070C060C000F800FF007F803FC003E000E700EF00CF00CE008401020601F8010 147D9313>I<018001C0038003800380038007000700FFF007000E000E000E000E001C00 1C001C001C003800380038003820704070407080708031001E000C1C7C9B0F>I<0E00C0 3300E02301C04381C04301C04701C08703800E03800E03800E03801C07001C07001C0700 1C07101C0E20180E20180E201C1E200C264007C38014147C9318>I<0E03803307802307 C04383C04301C04700C08700800E00800E00800E00801C01001C01001C01001C02001C02 001C04001C04001C08000E300003C00012147C9315>I<0E00C1C03300E3C02301C3E043 81C1E04301C0E04701C060870380400E0380400E0380400E0380401C0700801C0700801C 0700801C0701001C0701001C0602001C0F02000C0F04000E13080003E1F0001B147C931E >I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E03801C07001C 07001C07001C07001C0E00180E00180E001C1E000C3C0007DC00001C00001C00003800F0 3800F07000E06000C0C0004380003E0000131D7C9316>121 D E /Fg 88 126 df32 D<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F870051C779B18>I< 4010E038F078E038E038E038E038E038E038E038E038E038E03860300D0E7B9C18>I<03 0600078F00078F00078F00078F00078F00078F007FFFC0FFFFE0FFFFE07FFFC00F1E000F 1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFFC01E3C001E3C001E3C001E 3C001E3C001E3C000C1800131C7E9B18>I<00C00001C00001C00001C00003F0000FFC00 3FFE007DCF0071C700E1C380E1C780E1C780E1C780F1C00079C0003DC0001FE0000FF800 03FC0001DE0001CF0001C70061C380F1C380F1C380E1C380E1C70071C70079DE003FFE00 1FF80007E00001C00001C00001C00000C00011247D9F18>I<3803007C07807C0780EE0F 80EE0F00EE0F00EE1F00EE1E00EE1E00EE3E007C3C007C3C00387C0000780000780000F8 0000F00001F00001E00001E00003E00003C00003C00007C0000783800787C00F87C00F0E E00F0EE01F0EE01E0EE01E0EE03E0EE03C07C03C07C018038013247E9F18>I<01C00007 E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3800FC7000F 87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E0787070FC707F FFE03FCFE00F03C0141C7F9B18>I<387C7C7E3E0E0E0E1C1C38F8F0C0070E789B18>I<00 7000F001E003C007800F001E001C00380038007000700070007000E000E000E000E000E0 00E000E000E0007000700070007000380038001C001E000F00078003C001F000F000700C 24799F18>I<6000F00078003C001E000F000780038001C001C000E000E000E000E00070 007000700070007000700070007000E000E000E000E001C001C0038007800F001E003C00 7800F00060000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF 001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C0 0011147D9718>I<00600000F00000F00000F00000F00000F00000F00000F0007FFFC0FF FFE0FFFFE07FFFC000F00000F00000F00000F00000F00000F00000F00000600013147E97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>I<7FFF00FFFF80FFFF807FFF0011047D 8F18>I<3078FCFC78300606778518>I<000300000780000780000F80000F00001F00001E 00001E00003E00003C00007C0000780000780000F80000F00001F00001E00003E00003C0 0003C00007C0000780000F80000F00000F00001F00001E00003E00003C00003C00007C00 00780000F80000F00000F0000060000011247D9F18>I<01F00007FC000FFE001F1F001C 07003803807803C07001C07001C0E000E0E000E0E000E0E000E0E000E0E000E0E000E0E0 00E0E000E0F001E07001C07001C07803C03803801C07001F1F000FFE0007FC0001F00013 1C7E9B18>I<01800380038007800F803F80FF80FB804380038003800380038003800380 03800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>I<03F000 0FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001C00001C0 0003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00E03C00E0 7FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C07801C03001 C00001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C00000E00000 E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B18>I<00 1F00003F0000770000770000E70001E70001C7000387000787000707000E07001E07003C 0700380700780700F00700FFFFF8FFFFF8FFFFF800070000070000070000070000070000 0700007FF000FFF8007FF0151C7F9B18>I<1FFF803FFF803FFF80380000380000380000 3800003800003800003800003800003BF8003FFE003FFF003C07801803C00001C00000E0 0000E06000E0F000E0F000E0E001C07003C07C0F803FFF001FFC0003F000131C7E9B18> I<007E0001FF0007FF800F83C01E03C01C03C0380180380000700000700000E1F800E7FE 00FFFF00FE0780F803C0F001C0F000E0E000E0F000E07000E07000E07000E03801C03C03 C01E07800FFF0007FE0001F800131C7E9B18>II<03F8000FFE001FFF003E0F803803807001C07001C07001C07001C0380380 3C07801FFF0007FC000FFE001F1F003C07807001C0F001E0E000E0E000E0E000E0E000E0 7001C07803C03E0F801FFF000FFE0003F800131C7E9B18>I<03F0000FFC001FFE003C0F 00780780700380E001C0E001C0E001C0E001E0E001E07001E07803E03C0FE01FFFE00FFE E003F0E00000E00001C00001C00001C0300380780780780F00783E003FFC001FF00007C0 00131C7E9B18>I<3078FCFC783000000000000000003078FCFC78300614779318>I<183C 7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<0003000007 80001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000FC00007E00 003F00001FC00007E00003F00001FC00007E00003F00001F8000078000030011187D9918 >I<7FFFC0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07F FFC0130C7E9318>I<600000F00000FC00007E00003F00001FC00007E00003F00001FC00 007E00003F00001F80001F80003F00007E0001FC0003F00007E0001FC0003F00007E0000 FC0000F0000060000011187D9918>I<0FF0003FFC007FFF00700F00F00380F003806007 80000F00003E00007C0001F00001E00003C00003C00003C00003C00003C0000380000000 0000000000000000000000000003800007C00007C00007C000038000111C7D9B18>I<00 700000F80000F80000D80000D80001DC0001DC0001DC00018C00038E00038E00038E0003 8E000306000707000707000707000707000FFF800FFF800FFF800E03800E03801C01C01C 01C07F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE00F07E01E03E03C01E03800E07000E07000E0700000E00000E0 0000E00000E00000E00000E00000E00000E000007000007000E07000E03800E03C00E01E 01C00F07C007FF8003FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C0 1C03C01C01E01C00E01C00E01C00F01C00701C00701C00701C00701C00701C00701C0070 1C00701C00F01C00E01C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F 9B18>III<01F1C003FDC00FFFC01F0FC01C03C03803C03801C07001C07001C0700000 E00000E00000E00000E00000E00000E00FF0E01FF0E00FF07001C07001C07003C03803C0 3803C01C07C01F0FC00FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01 C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01 C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF8FF87F07 F0151C7F9B18>I<7FFF00FFFF807FFF0001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C0007FFF00FFFF807FFF00111C7D9B18>I<7F07F0FF87F87F07F0 1C03C01C07801C07001C0E001C1E001C3C001C38001C70001CF0001DF0001DF0001FB800 1FB8001F1C001E1C001C0E001C0E001C07001C07001C03801C03801C01C07F03F0FF87F8 7F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E 00700E00700E00700E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1 C01CC1C01CE1C01CE1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19 C01C19C01C1DC01C0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003F FE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E00380E00380E00380E00380F00780700700780F007F FF003FFE000FF800111C7D9B18>II<0FF8 003FFE007FFF00780F00700700F00780E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380E00380E00380E1E380E1E380F0E78070F700787F 007FFF003FFE000FFC00001C00001E00000E00000F0000070000070011227D9B18>I<7F F800FFFE007FFF001C0F801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801F FF001FFE001FFE001C0F001C07001C03801C03801C03801C03801C03801C039C1C039C1C 039C7F01F8FF81F87F00F0161C7F9B18>I<03F3801FFF803FFF807C0F80700780E00380 E00380E00380E000007000007800003F00001FF00007FE0000FF00000F800003C00001C0 0000E00000E06000E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18> I<7FFFF8FFFFF8FFFFF8E07038E07038E07038E070380070000070000070000070000070 000070000070000070000070000070000070000070000070000070000070000070000070 0000700007FF0007FF0007FF00151C7F9B18>IIII<7F8FE07F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F80001 F80000F00000F00000700000F00000F80001F80001DC00039E00038E00070F000707000E 07800E03801E03C07F07F0FF8FF87F07F0151C7F9B18>II<3FFFE07FFFE07FFFE07001C07003C0700780700700000F00001E00001C 00003C0000780000700000F00001E00001C00003C0000780000700000F00001E00E01C00 E03C00E07800E07000E0FFFFE0FFFFE0FFFFE0131C7E9B18>II<600000 F00000F00000F800007800007C00003C00003C00003E00001E00001F00000F00000F0000 0F800007800007C00003C00003C00003E00001E00001F00000F00000F800007800007800 007C00003C00003E00001E00001E00001F00000F00000F8000078000078000030011247D 9F18>II<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E0000070000070000FF0007FF001FFF007F07007807 00E00700E00700E00700F00F00781F003FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF80063E00151C809B18>I<01FE0007FF001FFF803E078038030070 0000700000E00000E00000E00000E00000E00000E000007000007001C03801C03E03C01F FF8007FF0001FC0012147D9318>I<001F80003F80001F80000380000380000380000380 00038003E3800FFB801FFF803C1F80380F80700780700380E00380E00380E00380E00380 E00380E00380700780700780380F803C1F801FFFF00FFBF803E3F0151C7E9B18>I<01F0 0007FC001FFE003E0F00380780700380700380E001C0E001C0FFFFC0FFFFC0FFFFC0E000 007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80007FC000 FFE000E1E001C0C001C00001C00001C0007FFFC0FFFFC0FFFFC001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF007F FF007FFF00131C7F9B18>I<01E1F007FFF80FFFF81E1E301C0E00380700380700380700 3807003807001C0E001E1E001FFC001FF80039E0003800001C00001FFE001FFFC03FFFE0 7801F0700070E00038E00038E00038E000387800F07E03F01FFFC00FFF8001FC00151F7F 9318>I<7E0000FE00007E00000E00000E00000E00000E00000E00000E3E000EFF800FFF C00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00 E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B18>I<03800007C00007C00007C00003 80000000000000000000000000007FC000FFC0007FC00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C000FFFF00FFFF80FF FF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8007FFFF8001F1F1C00 1E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F001914819318>I<7E 3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC1714809318>I<01F0000FFE00 1FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0E000E0F001E07001C0 7803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E00FEFF807FFFC00FC1 E00F80E00F00700E00700E00380E00380E00380E00380E00380E00380F00700F00700F80 E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00000E00000E00007FC0 00FFE0007FC000151E809318>I<01E38007FB801FFF803E1F80380F80700780700780E0 0380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF800FFB8003 E380000380000380000380000380000380000380000380003FF8003FF8003FF8151E7E93 18>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C000038000038000038000 0380000380000380000380000380000380007FFE00FFFF007FFE0015147F9318>I<07F7 003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC0003FE00001F006007 80E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318>I<01800003800003 80000380000380007FFFC0FFFFC0FFFFC003800003800003800003800003800003800003 80000380000380000380400380E00380E00380E001C1C001FFC000FF80003E0013197F98 18>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC1714809318>I<7F8F F0FF8FF87F8FF01E03C00E03800E03800E0380070700070700070700038E00038E00038E 00038E0001DC0001DC0001DC0000F80000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700 078E00039E0001DC0001F80000F80000700000F00000F80001DC00039E00038E00070700 0F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03 80070380070700070700038700038600038E0001CE0001CE0000CC0000CC0000DC000078 0000780000780000700000700000700000F00000E00079E0007BC0007F80003F00001E00 00151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C0700780000F00001E00003C0000 F80001F00003C0000780000F00701E00703C0070780070FFFFF0FFFFF0FFFFF014147F93 18>I<0007E0001FE0007FE000780000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00001E0007FC000FF8000FF80007FC00001E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E000007800007FE0001FE0 0007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000F000007FC0003FE0003FE0007FC000F0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00003C0 00FFC000FF00007C000013247E9F18>125 D E /Fh 49 123 df<000078007C7800FC78 01FC7803C000038000078000078000078000078000078000078000078000078000FFFC78 FFFC78FFFC78078078078078078078078078078078078078078078078078078078078078 078078078078078078078078078078078078078078078078078078152480A31A>12 D<787878781830306060E0050A7C830E>44 DII<000080000180000180000300000300000300000600000600000600000C 00000C00000C0000180000180000180000300000300000300000600000600000600000C0 0000C00000C0000180000180000180000180000300000300000300000600000600000600 000C00000C00000C00001800001800001800003000003000003000006000006000006000 00C00000C00000C0000011317DA418>I<00C001C007C0FFC0FFC0FBC003C003C003C003 C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C0FFFFFFFFFFFF10227CA118>49 D<03F0000FFC001FFE003C1F00300780 7007C06003C0E003E0C001E04001E04001E00001E00001E00001E00003C00003C0000780 000780000F00001E00003C0000780000F00001E00001C0000380000700000E00001C0000 380000700000FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC001FFF003E0F003807 807003C02003C02003C00003C00003C00003C0000780000780000F00001E0003FC0003F8 0003FE00000F000007800003C00003C00001E00001E00001E00001E00001E08001E0C003 C0E003C07007803C0F801FFF000FFC0003F00013237EA118>I<001F00001F00002F0000 2F00006F0000EF0000CF0001CF0001CF00038F00038F00078F00070F000F0F000E0F001E 0F003C0F003C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F0000 0F00000F00000F00000F00000F00000F0015217FA018>I<3FFF803FFF803FFF803C0000 3C00003C00003C00003C00003C00003C00003C00003C00003CF8003FFE003FFF003F0F80 3E07803C03C03803C00001E00001E00001E00001E00001E00001E00001E04003C04003C0 E003C07007807C1F003FFE000FFC0003F00013227EA018>I<007E0001FF0003FF0007C1 000F00001E00001E00003C00003C000078000078000078F800F1FE00F7FF00FF0F80FC07 80F803C0F803C0F801C0F001E0F001E0F001E0F001E0F001E07001E07001E07801E07803 C03803C03C03801C07801F0F000FFE0007FC0001F00013237EA118>II<01F00007FC000FFE00 1E0F003C07803C07807803C07803C07803C07803C07803C03803803C07801E0F000F1E00 07FC0003F8000FFE001E0F003C07807803C07803C0F001E0F001E0F001E0F001E0F001E0 F001E07803C07803C03C07803E0F801FFF0007FC0001F00013237EA118>I<000FF00000 7FFC0000FFFF0001F01F0003C00700078000000F0000001E0000003E0000003C0000003C 000000780000007800000078000000F0000000F0000000F0000000F0000000F0000000F0 000000F0000000F0000000F00000007800000078000000780000003C0000003C0000003E 0000001E0000000F0000000780008003C0038001F00F8000FFFF00007FFC00000FF00019 257DA31F>67 D69 D73 D77 DI80 D82 D<00FE0003FFC007FFE00F81E01E00603C00003C0000780000780000780000 7800007800007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F0 0000F00000F8000078000078000078000078000078000078C000F0E000F0F801E07E07C0 3FFF800FFF0001FC0015257EA31B>III87 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF00FF0 0FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180 780000780000700000F00000F00000F00000F00000F00000F00000780000780000780000 3C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC01FFFC03F0F C03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C07803 C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC001FFE003E 0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F0000070000078 00007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01FF03C003 8007800780078007800780078007800780FFF8FFF8FFF807800780078007800780078007 80078007800780078007800780078007800780078007800780102380A20F>I<01F07807 FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F000FFE001F FC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8F00078F0 0078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>III107 DIII<01FC0007FF000FFF801F07C03C01E07800 F07800F0700070F00078F00078F00078F00078F00078F000787800F07800F07C01F03E03 E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07C07C03C07803C0 F803C0F003C0F003C0F003C0F003C0F003C0F003C0F803C07803C07803C03C07C03F0FC0 1FFFC00FFBC003E3C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C012207E9519>II<07F01FFC3FFE3C0E780678 0078007C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE3FFC0FF010167F 9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F000F000F000F00 0F000F000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12>IIII<7801 F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F80000700000F80001FC 0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F81516809516>II<7FFFC07FFFC0 7FFFC0000F80000F00001E00003E00007C0000780000F00001F00003E00003C000078000 0F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012167F9515>I E /Fi 26 90 df45 D<3078F8787005057C840D>I<003F8000C1 E00100F00200780400780400780F007C0F807C0F807C0F00780600780000F80000F00001 E00001C0000380000700000E00001C0000380000600000C0000180000300200600200800 401000403FFFC07FFF80FFFF80161E7E9D17>50 D<007F000183C00201E00400F00700F0 0F00F00F01F00F01F00001E00001E00003C0000380000700000E0000F800000E00000700 0007800007C00003C00007C03007C07807C0F807C0F807C0F00780800F00400E00201C00 18780007E000141F7D9D17>I<0000600000600000E00001C00003C00005C0000DC00009 C00011C000238000438000C3800083800103800203800407000807001807001007002007 00400700FFFFF0000E00000E00000E00000E00000E00001C00001E0001FFE0141E7E9D17 >I<01803001FFE003FFC003FF0003FC0002000002000002000004000004000004000004 7C000587000603800C01800801C00001C00001E00001E00001E00001E07003C0F803C0F0 03C0E00380800780400700400E00201C0018700007C000141F7D9D17>I<000010000000 1800000038000000380000007800000078000000FC000001BC0000013C0000033C000002 3C0000063C0000043E0000081E0000081E0000101E0000101E0000201E0000200F000040 0F0000400F0000FFFF0000800F0001000F80010007800200078002000780040007800400 07800C0007C03E0007C0FF807FFC1E207E9F22>65 D<07FFFF00007C01C0003C01E0003C 00F0007800F8007800F8007800F8007800F8007800F8007800F000F001F000F001E000F0 03C000F00F8000FFFE0000F00F0001E007C001E003C001E003E001E001E001E001E001E0 01E003C001E003C003E003C003E003C003C003C007C003C00F8007800F0007803E00FFFF F0001D1F7E9E20>I<0001F808000E061800380138007000F801E0007803C00070078000 30078000300F0000301F0000301E0000303E0000203C0000007C0000007C0000007C0000 007C000000F8000000F8000000F8000000F8000000F80000007800004078000080780000 803C0000803C0001001C0002000E00020006000C000300100001C0E000003F00001D217B 9F21>I<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E 0078001E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E 01E0003E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E0 03C003C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C007800 3C0038003C001800780018007800080078000800780008007800080078080800F0100000 F0100000F0100000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001 E0000801E0001003C0001003C0001003C0002003C0002003C0006003C000C0078001C007 8007C0FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C0018007800180078 00080078000800780008007800080078000800F0100000F0100000F0100000F0300000F0 700000FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0000003C0 000003C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F7E9E1E> I<0001FC04000F030C003C009C0070007C00E0003C01C0003803800018078000180F0000 181F0000181E0000183E0000103C0000007C0000007C0000007C0000007C000000F80000 00F8000000F8007FFCF80003E0780001E0780001E0780003C0780003C03C0003C03C0003 C01C0003C00E0007C007000B800380118001E06080003F80001E217B9F24>I<07FFC7FF C0007C00F800003C007800003C007800007800F000007800F000007800F000007800F000 007800F000007800F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000 F001E00001E003C00001E003C00001E003C00001E003C00001E003C00001E003C00003C0 07800003C007800003C007800003C007800003C007800003C007800007800F000007C00F 8000FFF8FFF800221F7E9E22>I<07FFE0007C00003C00003C0000780000780000780000 780000780000780000F00000F00000F00000F00000F00000F00001E00001E00001E00001 E00001E00001E00003C00003C00003C00003C00003C00003C00007800007C000FFFC0013 1F7F9E10>I<07FFF000007E0000003C0000003C00000078000000780000007800000078 0000007800000078000000F0000000F0000000F0000000F0000000F0000000F0000001E0 000001E0000001E0000001E0000001E0008001E0010003C0010003C0010003C0030003C0 020003C0060003C0060007801E0007807C00FFFFFC00191F7E9E1C>76 D<07FC0000FFC0007C0000F800003C00017800003C00017800004E0002F000004E0002F0 00004E0004F000004E0004F000004E0008F000004E0008F00000870011E00000870011E0 0000870021E00000870021E00000870041E00000838041E00001038083C00001038083C0 0001038103C00001038203C0000101C203C0000101C403C0000201C40780000201C80780 000201C80780000201D00780000200F00780000600E00780000600E00F00000F00C00F80 00FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E001800003E001800004F 001000004F001000004780100000478010000043C010000043C010000083C020000081E0 20000081E020000080F020000080F020000080782000010078400001007C400001003C40 0001003C400001001E400001001E400002000F800002000F800002000F80000200078000 0200078000060003800006000300000F00010000FFE0010000221F7E9E22>I<0003F800 001E0E000038070000E0038001C001C003C001E0078000E00F0000F00F0000F01E0000F0 1E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8F80001F0F80001F0 F80001F0F80001F0F80003E0780003E0780003C0780007C07C0007803C000F003C001E00 1E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07FFFF00007C03C000 3C01E0003C00F0007800F0007800F8007800F8007800F8007800F8007800F000F001F000 F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001E0000001E0000001 E0000001E0000003C0000003C0000003C0000003C0000003C0000003C000000780000007 C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C01E0007801E00078 01F0007801F0007801F0007801F0007801E000F003E000F003C000F0078000F00F0000F0 3C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E01E0003C03E0003C0 3E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC0F18000003E01F20 7E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E00080E0008 0E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E00000F0 0000F04000E04000E04000E04000E06000C0600180E00380F80300C60C0081F80016217D 9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010C00F0010800F0010 800F0010800F0010001E0000001E0000001E0000001E0000001E0000001E0000003C0000 003C0000003C0000003C0000003C0000003C000000780000007800000078000000780000 007800000078000000F0000001F800007FFFE0001C1F7A9E21>II 87 D89 D E /Fj 37 122 df<000FE000007FF80000F81C0001E07C0003E07C0007C07C0007C07C00 07C0380007C0000007C0000007C0000007C1FE00FFFFFE00FFFFFE0007C03E0007C03E00 07C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E00 07C03E0007C03E0007C03E0007C03E0007C03E003FF9FFC03FF9FFC01A20809F1D>12 D<387CFEFEFE7C3807077C860F>46 D<00E00001E0000FE000FFE000F3E00003E00003E0 0003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E0 0003E00003E00003E00003E00003E00003E00003E00003E000FFFF80FFFF80111D7C9C1A >49 D<07F0001FFE00383F007C1F80FE0FC0FE0FC0FE0FE0FE07E07C07E03807E0000FE0 000FC0000FC0001F80001F00003E0000780000F00000E00001C0000380600700600E0060 1C00E01FFFC03FFFC07FFFC0FFFFC0FFFFC0131D7D9C1A>I<01FC0007FF000E0F801E0F C03F07E03F07E03F07E03F07E01E0FC0000FC0000F80001F0001FC0001FC00000F800007 C00003E00003F00003F83803F87C03F8FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF 8003FC00151D7E9C1A>I<07F8001FFE00381F80780F80FC0FC0FC0FC0FC0FC0780FC030 1F80001F00003E00007C0000700000E00000E00000C00000C00000C00000C00000C00000 C00000000000000000000000000001C00003E00007F00007F00007F00003E00001C00012 207D9F19>63 D69 D76 DII80 D<03FC080FFF381E03F83800F8700078700038F00038F00018F00018F8 0000FC00007FC0007FFE003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00 003CC0003CC0003CC0003CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>83 D85 DI<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C 03E07C03E0F803E0F803E0F803E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007 FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000FC0000FC00007C 00007E00003E00603F00C01F81C007FF0001FC0013147E9317>I<0007F80007F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F801F8F80FFEF81F83F8 3E01F87E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F8 7E00F83E01F81F07F80FFEFF03F8FF18207E9F1D>I<01FE0007FF800F83C01E01E03E00 F07C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E00 180F807007FFE000FF8015147F9318>I<001F8000FFC001F3E003E7E003C7E007C7E007 C3C007C00007C00007C00007C00007C000FFFC00FFFC0007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0003F FC003FFC0013207F9F10>I<01FC3C07FFFE0F079E1E03DE3E03E03E03E03E03E03E03E0 3E03E01E03C00F07800FFF0009FC001800001800001C00001FFF800FFFF007FFF81FFFFC 3C007C70003EF0001EF0001EF0001E78003C78003C3F01F80FFFE001FF00171E7F931A> II<1C003E00 7F007F007F003E001C00000000000000000000000000FF00FF001F001F001F001F001F00 1F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B217EA00E>I107 DI< FE0FE03F80FE1FF07FC01E70F9C3E01E407D01F01E807E01F01F807E01F01F007C01F01F 007C01F01F007C01F01F007C01F01F007C01F01F007C01F01F007C01F01F007C01F01F00 7C01F01F007C01F01F007C01F01F007C01F0FFE3FF8FFEFFE3FF8FFE27147D932C>II<01FF0007FFC0 1F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC007EFC007E7C007C 7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>II<01F81807FE381F87783F01F83E01F87E00F87C00F8FC00F8FC 00F8FC00F8FC00F8FC00F8FC00F87C00F87E00F87E00F83F01F81F87F80FFEF803F8F800 00F80000F80000F80000F80000F80000F80000F80007FF0007FF181D7E931C>II<0FE63FFE701E600E E006E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81EFFFCC7F010147E93 15>I<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F800F800F 800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>II119 DII E /Fk 61 122 df<00F8F001F8F003F8F00780F00700F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F0FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F01420809F 18>13 D<007000E001C00380078007000E001E001E003C003C003C007800780078007800 7000F000F000F000F000F000F000F000F000F000F000F000F00070007800780078007800 3C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<018001C001800180 C183E187F99F7DBE1FF807E007E01FF87DBEF99FE187C1830180018001C0018010147DA1 17>I<787878781830306060E0050A7D830D>44 DII<000100030003000600060006000C000C000C00180018001800300030 003000600060006000C000C000C00180018001800300030003000600060006000C000C00 0C00180018001800300030003000600060006000C000C000C000102D7DA117>I<03F000 07F8001FFE001E1E003C0F00380700780780780780700380F003C0F003C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0780780780780 7807803807003C0F001E1E001FFE000FFC0003F00012207E9E17>I<00C001C00FC0FFC0 FFC0F3C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17>I<001F0000001F0000003F80 00003B8000003B8000007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F0 0001E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C000FFFFE 000FFFFE000FFFFE001E000F001E000F003C000F803C0007803C000780780007C0780003 C0780003C0F00003E01B207F9F1E>65 DI<001FC000FFF801FFFC03E03C07800C0F00001E00003E0000 3C00007C0000780000780000780000F00000F00000F00000F00000F00000F00000F00000 F000007800007800007800007C00003C00003E00001E00000F000207800E03E03E01FFFC 00FFF0001FC017227DA01D>IIII<001F E000FFF801FFFE03E03E07800E0F00001E00003E00003C00007C00007800007800007800 00F00000F00000F00000F00000F00000F00000F003FEF003FE7803FE78001E78001E7C00 1E3C001E3E001E1E001E0F001E07801E03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F000000FFC00003FFF00007E1F8000F80 7C001F003E001E001E003C000F003C000F00780007807800078078000780F00003C0F000 03C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F80007C07800 078078000780780007803C000F003C000F001E001E001F003E000F807C0007E1F80003FF F00000FFC000003F00001A227DA021>II<003F000000FFC00003FFF00007E1F8000F807C001F003E00 1E001E003C000F003C000F00780007807800078078000780F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C07800078078000780 781E07803C0F0F003C0F8F001E079E001F03FE000F83FC0007E1F80003FFF00000FFF800 003F780000007C0000003E0000001E0000001F0000000F801A277DA021>II<01FC0007FF800FFFC01F 03C03C00C03C00007800007800007800007800007800007C00003C00003F00001FE0000F FC0007FE0001FF00003F800007C00003C00003E00001E00001E00001E00001E00001E000 01C0C003C0F007C0FC0F807FFF001FFE0003F80013227EA019>IIIII89 DII93 D<07E03FF87FFC701E401F000F000F000F003F07FF1FFF 7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F0 00F000F000F000F000F000780078003C013E0F1FFF0FFE03F010147E9314>I<00078000 078000078000078000078000078000078000078000078000078000078000078007C7800F F7801FFF803E1F807C0780780780F80780F00780F00780F00780F00780F00780F00780F0 0780780780780F803E1F801FFF800FF78007C78011207E9F17>I<03F0000FFC001FFE00 3E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F00000F00000700000780000 3C01003E07001FFF0007FE0001F80011147F9314>I<007E01FE03FE078007000F000F00 0F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F000F000F000F000F000F00 0F000F000F000F000F000F000F000F20809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00 780F00780F00780F00780F00780F003C1E003E3E001FFC003FF80033E000300000380000 3FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F801F07E07E03FFFC00FFF00 03FC00141E7F9317>III<01E001E001E001E000000000000000000000000000000000 01E001E001E001E001E001E001E001E001E001E001E001E001E001E001E001E001E001E0 01E001E001E001E001E001E001E0C3C0FFC0FF803F000B29839F0C>IIIII<01F80007FE001FFF803F0FC03C03C07801E078 01E0F000F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007 FE0001F80014147F9317>II<03C7 800FF7801FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F007 80F807807807807C0F803E1F801FFF800FF78007C7800007800007800007800007800007 80000780000780000780000780111D7E9317>II<07F01FFC3F FC780C7800780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E 147F9311>I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E00 1E001E001E001E001E001E001E001E201FF00FF007C00C1A7F9910>IIII<7801E07C03C03E07801E0F000F0F00079E0003FC0003 F80001F80000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F8 01F01414809315>II E /Fl 16 117 df<020408103020604040C0C0C0C0C0C0C0C0404060203010080402071A 7F920C>40 D<8040201018080C0404060606060606060604040C081810204080071A7E92 0C>I<1F00318060C04040C060C060C060C060C060C060C060C060404060C031801F000B 107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C000C000C000C 00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006000C001020 20207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F00008000400060C0 60C060804060801F000B107F8F0F>I<0300030007000F000B001300330023004300C300 FFE003000300030003001FE00B107F8F0F>I<20803F002C002000200020002F00308020 40006000600060C06080C061801F000B107F8F0F>I<0780184030C060C06000C000CF00 F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC08080808001 000200040004000C0008000800180018001800180018000B117E900F>I<1F00318060C0 60C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F>I<1F0031 8060C0C040C060C060C06040E021E01E600060004060C0608043003E000B107F8F0F>I< 1F2060E0006080208020800060003E001F8000C00060002080208020C040E0C09F000B11 7E9011>83 D<1F00318060C0C0C0FFC0C000C000C000604030801F000A0B7F8A0E>101 D112 D<10103030FE3030303030323232321C070F7F8E0C>116 D E /Fm 7 104 df0 D<03C00FF01FF83FFC7FFE7FFE FFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D<000000C0000003C000000F0000003C000000F0000003C00000070000001C0000007800 0001E00000078000001E00000078000000E0000000780000001E0000000780000001E000 0000780000001C0000000700000003C0000000F00000003C0000000F00000003C0000000 C0000000000000000000000000000000000000000000000000000000007FFFFF80FFFFFF C01A247C9C23>20 D62 D<00040000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000FFFFFFE0FFFFFFE01B1C7C9B23>I<000F0038 006000E001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00380 07001E00F8001E000700038001C001C001C001C001C001C001C001C001C001C001C001C0 01C001C001C000E000600038000F102D7DA117>102 DI E /Fn 46 123 df<007E0001C1800301800703C00E03C00E 01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F87F815 1D809C17>12 D<007FC001C1C00303C00703C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<6060F0F0F8F868 68080808080808101010102020404080800D0C7F9C15>34 D<60F0F86808080810102040 80050C7C9C0C>39 D<004000800100020006000C000C0018001800300030007000600060 006000E000E000E000E000E000E000E000E000E000E000E000E000600060006000700030 003000180018000C000C00060002000100008000400A2A7D9E10>I<8000400020001000 18000C000C000600060003000300038001800180018001C001C001C001C001C001C001C0 01C001C001C001C001C0018001800180038003000300060006000C000C00180010002000 400080000A2A7E9E10>I<60F0F0701010101020204080040C7C830C>44 DI<60F0F06004047C830C>I<0FE03038401CE00EF00EF00EF00E 000C001C0030006000C00080018001000100010001000100010000000000000000000000 03000780078003000F1D7E9C14>63 D73 D77 DI<003F800000E0E0000380380007 001C000E000E001C0007003C00078038000380780003C0780003C0700001C0F00001E0F0 0001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C078 0003C0380003803C0007801C0007000E000E0007001C000380380000E0E000003F80001B 1E7E9C20>II82 D<07E0801C1980300580700380600180E00180E00080E00080E00080F00000F8 00007C00007FC0003FF8001FFE0007FF0000FF80000F800007C00003C00001C08001C080 01C08001C0C00180C00180E00300D00200CC0C0083F800121E7E9C17>I<7FFFFFC0700F 01C0600F00C0400F0040400F0040C00F0020800F0020800F0020800F0020000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000001F800003FFFC001B1C 7F9B1E>II<08081010202040404040808080808080 B0B0F8F8787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00001C00001C 0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E801F87001212 7E9115>97 DI<07E00C3018783078 70306000E000E000E000E000E000E00060007004300418080C3007C00E127E9112>I<00 3F0000070000070000070000070000070000070000070000070000070000070003E7000C 1700180F00300700700700600700E00700E00700E00700E00700E00700E0070060070070 0700300700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006E006FF FEE000E000E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E061E 0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C3880181800 381C00381C00381C00381C001818001C38000C300013C0001000003000001800001FF800 1FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007F800121C7F 9215>II<18003C003C0018000000 000000000000000000000000FC001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C0000000000000000000 00000000000FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380 600180E001C0E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C00 03F00012127F9115>II<03C1000C3300180B00300F0070 0700700700E00700E00700E00700E00700E00700E00700600700700700300F00180F000C 370007C700000700000700000700000700000700000700000700003FE0131A7E9116>I< FCE01D301E781E781C301C001C001C001C001C001C001C001C001C001C001C001C00FFC0 0D127F9110>I<1F9030704030C010C010E010F8007F803FE00FF000F880388018C018C0 18E010D0608FC00D127F9110>I<04000400040004000C000C001C003C00FFE01C001C00 1C001C001C001C001C001C001C001C101C101C101C101C100C100E2003C00C1A7F9910> IIII<7F8FF00F03800F03000702000384 0001C80001D80000F00000700000780000F800009C00010E00020E000607000403801E07 C0FF0FF81512809116>II<7FFC70386038407040F040E0 41C003C0038007000F040E041C043C0C380870087038FFF80E127F9112>I E /Fo 14 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D<001FE02000FFF8E003F80FE007C003E00F8001E01F0000E03E0000E03E0000607E0000 607C000060FC000000FC000000FC000000FC000000FC000000FC000000FC000000FC0000 007C0000607E0000603E0000603E0000C01F0000C00F80018007C0030003F80E0000FFFC 00001FE0001B1C7D9B22>67 DI<07F8201FFEE03C07E07801E0 7000E0F000E0F00060F00060F80000FE0000FFE0007FFE003FFF003FFF800FFFC007FFE0 007FE00003F00001F00000F0C000F0C000F0C000E0E000E0F001C0FC03C0EFFF0083FC00 141C7D9B1B>83 D<0FF8001C1E003E0F803E07803E07C01C07C00007C0007FC007E7C01F 07C03C07C07C07C0F807C0F807C0F807C0780BC03E13F80FE1F815127F9117>97 D<03FC000E0E001C1F003C1F00781F00780E00F80000F80000F80000F80000F80000F800 007800007801803C01801C03000E0E0003F80011127E9115>99 D<01FC000F07001C0380 3C01C07801C07801E0F801E0F801E0FFFFE0F80000F80000F800007800007C00603C0060 1E00C00F038001FC0013127F9116>101 D<1E003F003F003F003F001E00000000000000 000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F00 1F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8 F800F8F800F87800F07800F03C01E01E03C00F078001FC0015127F9118>I114 D<1FD830786018E018E018F000FF80 7FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>I<03000300030003 00070007000F000F003FFCFFFC1F001F001F001F001F001F001F001F001F001F0C1F0C1F 0C1F0C0F08079803F00E1A7F9913>II E /Fp 36 122 df45 DI<00300000700001F0000FF000FFF000FFF000F1F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F000 01F00001F000FFFFE0FFFFE0FFFFE013287BA71D>49 D<00FE0003FF800FFFC01FFFE01F 07F03C01F87800FC78007C70007CF0003EF0003E60003E20003E00003E00003E00003E00 007C00007C0000780000F80001F00001E00003E00007C0000F80001F00003E00007C0000 F80001F00003E00003C0000780000F00001E00003C00007FFFFE7FFFFE7FFFFE7FFFFE17 287EA71D>I<007E0003FF8007FFE01FFFF03F83F83E00F87C007C38007C30007C10007C 00007C00007C0000F80000F80001F80003F0000FE001FFC001FF8001FFC001FFE00001F8 0000FC00007C00003E00003E00001F00001F00001F00001F00001F80001FC0003EE0003E F0007E7C00FC3F01F81FFFF00FFFE003FFC000FE0018297EA71D>I<0007E0000FE0000B E0001BE0001BE0003BE0003BE00073E00073E000F3E000F3E001E3E001E3E003E3E003C3 E007C3E00F83E00F83E01F03E01F03E03E03E03E03E07C03E07C03E0F803E0FFFFFFFFFF FFFFFFFFFFFFFF0003E00003E00003E00003E00003E00003E00003E00003E00003E00003 E018277EA61D>I<3FFFF83FFFF83FFFF83FFFF83E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E3E003EFF803FFFE03FFFF03FC1F03F00F83F007C3E 007C00007C00003E00003E00003E00003E00003E00003E00003E40003E60007CE0007CF0 00F87801F87F07F03FFFE00FFFC007FF0001FC0017287EA61D>I66 D<0003FF00000FFFE0003FFFF800 FFFFF801FE01F803F8007007E0001007C000000F8000001F8000001F0000003E0000003E 0000007C0000007C0000007C0000007C000000F8000000F8000000F8000000F8000000F8 000000F8000000F8000000F8000000F8000000F80000007C0000007C0000007C0000007C 0000003E0000003E0000001F0000001F8000000F80000007C0000007E0000803F8003801 FE01F800FFFFF8003FFFF0000FFFC00003FE001D2C7CAA25>III73 D77 D<0003FC0000000FFF0000003FFFC00000FFFFF00001FE07F80003F801FC0007E000 7E000FC0003F000F80001F001F00000F801F00000F803E000007C03E000007C07C000003 E07C000003E07C000003E078000001E0F8000001F0F8000001F0F8000001F0F8000001F0 F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F07C000003E07C 000003E07C000003E07E000007E03E000007C03E000007C01F00000F801F80001F800F80 001F000FC0003F0007E0007E0003F801FC0001FE07F80000FFFFF000003FFFC000001FFF 80000003FC0000242C7DAA2B>79 DI84 D<01FC000FFF803FFFC03FFFE03C03F03001F00001F80000F8 0000F80000F80000F80001F800FFF807FFF81FFFF83FC0F87E00F8FC00F8F800F8F800F8 F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1C>97 DI<007F8001FFE007FFF80F FFF81FC0783F00103E00007C00007C00007C0000F80000F80000F80000F80000F80000F8 0000F800007C00007C00007C00003E00003F00181FC0780FFFF807FFF801FFE0007F8015 1B7E9A1A>I<00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E01FC3E03FF3E0FFFBE1FFFFE1FC1FE3F007E3E003E 7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E 7C003E7E007E3F00FE1FC1FE1FFFFE0FFFBE03FE3E01F83E172A7EA91E>I<00FC0003FF 0007FF800FFFC01F87E03F01E03E00F07C00F07C0070F80078FFFFF8FFFFF8FFFFF8FFFF F8F80000F80000F800007800007C00007C00003E00003F00181FC0F80FFFF807FFF801FF E0007F00151B7E9A1A>I<003FC000FFC001FFC003FFC007E00007C0000F80000F80000F 80000F80000F80000F80000F80000F80000F8000FFFE00FFFE00FFFE000F80000F80000F 80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F 80000F80000F80000F80000F80000F80000F80000F80000F80000F8000122A7FA912>I< 007C078001FF3FC007FFFFC00FFFFFC00F83E0001F01F0001F01F0003E00F8003E00F800 3E00F8003E00F8003E00F8001F01F0001F01F0000F83E0000FFFE0001FFFC0001DFF0000 1C7C00003C0000003C0000003E0000001FFFE0001FFFF8000FFFFE001FFFFF003FFFFF80 7E003F807C000FC0F80007C0F80007C0F80007C0F80007C0FC000FC07E001F803F807F00 1FFFFE000FFFFC0003FFF000007F80001A287F9A1D>III108 DII<007E0001FF8007FFE00FFFF01F81F83F00FC 3E007C7C003E7C003E78001EF8001FF8001FF8001FF8001FF8001FF8001FF8001F7C003E 7C003E7C003E3E007C3F00FC1FC3F80FFFF007FFE003FFC0007E00181B7E9A1D>II114 D<07F8001FFF003FFF807FFF807C0780F80100F80000F80000F80000FC00007F 00007FF0003FFC001FFE0007FF0000FF80001FC0000FC00007C00007C04007C0E007C0FC 1F80FFFF807FFF001FFE0007F800121B7E9A16>I<0F80000F80000F80000F80000F8000 0F80000F8000FFFF80FFFF80FFFF800F80000F80000F80000F80000F80000F80000F8000 0F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8080 0FC38007FFC007FFC003FF0001F80012227FA115>II 119 D121 D E /Fq 15 117 df<000000007FFE0000060000000FFFFFC0001E0000007FFFFFF8003E000003FFFFFFFE 007E00000FFFF800FF80FE00003FFF80001FC1FE0000FFFC000007F3FE0001FFF0000001 FFFE0007FFC00000007FFE000FFF000000003FFE001FFE000000001FFE003FFC00000000 0FFE007FF80000000007FE00FFF00000000007FE01FFF00000000003FE01FFE000000000 01FE03FFE00000000001FE07FFC00000000000FE07FFC00000000000FE0FFF8000000000 00FE0FFF8000000000007E1FFF8000000000007E1FFF0000000000007E3FFF0000000000 003E3FFF0000000000003E3FFF0000000000003E7FFF0000000000003E7FFE0000000000 00007FFE000000000000007FFE00000000000000FFFE00000000000000FFFE0000000000 0000FFFE00000000000000FFFE00000000000000FFFE00000000000000FFFE0000000000 0000FFFE00000000000000FFFE00000000000000FFFE00000000000000FFFE0000000000 0000FFFE00000000000000FFFE00000000000000FFFE000000000000007FFE0000000000 00007FFE000000000000007FFE000000000000007FFF000000000000003FFF0000000000 001E3FFF0000000000001E3FFF0000000000001E1FFF0000000000001E1FFF8000000000 001E0FFF8000000000003E0FFF8000000000003C07FFC000000000003C07FFC000000000 003C03FFE000000000007801FFE000000000007801FFF00000000000F000FFF000000000 01F0007FF80000000001E0003FFC0000000003C0001FFE000000000780000FFF80000000 0F000007FFC00000003E000001FFF00000007C000000FFFC000001F80000003FFF80000F F00000000FFFF8007FC000000003FFFFFFFF00000000007FFFFFFC00000000000FFFFFE0 0000000000007FFE00000047497AC754>67 D77 D80 D<000FFFF000000000FFFFFF00000003FFFFFFC0000007F8 007FF000000FFC001FFC00000FFC0007FE00001FFE0003FF00001FFE0003FF80001FFE00 01FF80001FFE0001FFC0001FFE0000FFC0000FFC0000FFE00007F80000FFE00003F00000 FFE00000000000FFE00000000000FFE00000000000FFE00000000000FFE00000000000FF E0000000007FFFE00000003FFFFFE0000003FFFFFFE000001FFFF0FFE000007FFE00FFE0 0001FFE000FFE00003FF8000FFE0000FFE0000FFE0001FFC0000FFE0001FF80000FFE000 3FF80000FFE0007FF00000FFE0007FF00000FFE000FFE00000FFE000FFE00000FFE000FF E00000FFE000FFE00000FFE000FFE00001FFE000FFE00001FFE0007FF00003FFE0007FF0 00077FE0003FF8000F7FE0001FFC003E7FF8000FFF80F83FFFE003FFFFF01FFFE000FFFF C00FFFE0000FFE0003FFE0332E7CAD38>97 D<00001FFFC0000001FFFFFC000007FFFFFF 00001FF8007F80007FE000FFC000FF8000FFC001FF0001FFE003FE0001FFE007FE0001FF E00FFC0001FFE00FFC0001FFE01FF80000FFC03FF800007F803FF800003F007FF8000000 007FF0000000007FF0000000007FF000000000FFF000000000FFF000000000FFF0000000 00FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF0000000 00FFF0000000007FF0000000007FF0000000007FF8000000007FF8000000003FF8000000 003FF8000000001FFC000000F00FFC000000F00FFE000001E007FE000001E003FF000003 C001FF8000078000FFC0000F00007FF0003E00001FFC01FC000007FFFFF0000001FFFFC0 0000001FFC00002C2E7CAD34>99 D<0000000000FF8000000001FFFF8000000001FFFF80 00000001FFFF8000000001FFFF800000000007FF800000000003FF800000000003FF8000 00000003FF800000000003FF800000000003FF800000000003FF800000000003FF800000 000003FF800000000003FF800000000003FF800000000003FF800000000003FF80000000 0003FF800000000003FF800000000003FF800000000003FF800000000003FF8000000000 03FF800000000003FF800000000003FF8000001FFC03FF800001FFFF83FF800007FFFFE3 FF80001FFC03F3FF80007FE0007FFF8000FF80003FFF8001FF00000FFF8003FE000007FF 8007FE000003FF800FFC000003FF800FFC000003FF801FF8000003FF803FF8000003FF80 3FF8000003FF803FF0000003FF807FF0000003FF807FF0000003FF807FF0000003FF80FF F0000003FF80FFF0000003FF80FFF0000003FF80FFF0000003FF80FFF0000003FF80FFF0 000003FF80FFF0000003FF80FFF0000003FF80FFF0000003FF80FFF0000003FF807FF000 0003FF807FF0000003FF807FF0000003FF807FF0000003FF803FF8000003FF803FF80000 03FF801FF8000003FF801FF8000003FF800FFC000003FF8007FC000007FF8003FE00000F FF8001FF00001FFF8000FF80007FFF80007FC000FBFFC0003FF807E3FFFF000FFFFFC3FF FF0001FFFF03FFFF00003FF003FFFF38487CC741>I<00001FFC00000001FFFFC0000007 FFFFF000001FF80FFC00007FC001FE0000FF8000FF0001FF00007F8003FE00003FC007FC 00001FE00FFC00001FE01FF800001FF01FF800000FF03FF800000FF83FF800000FF83FF0 00000FF87FF0000007F87FF0000007FC7FF0000007FCFFF0000007FCFFF0000007FCFFFF FFFFFFFCFFFFFFFFFFFCFFFFFFFFFFFCFFF000000000FFF000000000FFF000000000FFF0 00000000FFF0000000007FF0000000007FF0000000007FF0000000003FF8000000003FF8 000000001FF80000003C1FF80000003C0FFC0000003C0FFC0000007807FE000000F803FF 000001F001FF800003E000FFC00007C0003FE0001F80001FFC01FF000007FFFFFC000000 FFFFE00000000FFF00002E2E7DAD35>I<00000000001F0000007FF000FFC00007FFFF03 FFE0001FFFFFC7E7E0007FE03FFF0FF000FF800FFC0FF001FF0007FC0FF003FE0003FE0F F007FE0003FF07E007FC0001FF03C00FFC0001FF80000FFC0001FF80001FFC0001FFC000 1FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001FFC0001F FC0001FFC0001FFC0001FFC0000FFC0001FF80000FFC0001FF800007FC0001FF000007FE 0003FF000003FE0003FE000001FF0007FC000000FF800FF8000000FFE03FF0000001DFFF FFC0000003C7FFFF00000003C07FF0000000078000000000000780000000000007800000 00000007C0000000000007C0000000000007E0000000000007E0000000000007F8000000 000007FFFFFFF0000003FFFFFFFF000003FFFFFFFFE00001FFFFFFFFF00001FFFFFFFFFC 0000FFFFFFFFFE00007FFFFFFFFF0001FFFFFFFFFF8007FFFFFFFFFF800FF000001FFFC0 1FC0000001FFC03F800000007FC07F800000003FE0FF000000003FE0FF000000001FE0FF 000000001FE0FF000000001FE0FF000000001FE0FF000000001FE07F800000003FC07F80 0000003FC03FC00000007F801FE0000000FF000FF8000003FE0003FE00000FF80001FFE0 00FFF000007FFFFFFFC000000FFFFFFE000000007FFFC0000034447DAE3A>103 D<007C0001FF0003FF8007FFC007FFC00FFFE00FFFE00FFFE00FFFE00FFFE007FFC007FF C003FF8001FF00007C000000000000000000000000000000000000000000000000000000 00000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFFFF FFFFFFFF18497CC820>105 D<00FF8003FF000003FF000000FFFF801FFFE0001FFFE000 00FFFF807FFFF8007FFFF80000FFFF81F80FFC01F80FFC0000FFFF83C007FE03C007FE00 0003FF878003FF078003FF000001FF8E0003FF0E0003FF000001FF9C0003FF9C0003FF80 0001FFB80001FFB80001FF800001FFB80001FFF80001FFC00001FFF00001FFF00001FFC0 0001FFF00001FFF00001FFC00001FFE00001FFE00001FFC00001FFE00001FFE00001FFC0 0001FFE00001FFE00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF 80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF80592E7BAD62>109 D<00FF8007FE0000FFFF803FFFC000FFFF807FFFF000FFFF81F81FF800FFFF83C00FFC00 03FF870007FE0001FF8E0007FE0001FF9C0007FF0001FFB80003FF0001FFB80003FF8001 FFF00003FF8001FFF00003FF8001FFE00003FF8001FFE00003FF8001FFE00003FF8001FF C00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC0 0003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC000 03FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003 FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF 8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF8001FFC00003FF80 01FFC00003FF80FFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFFFFFFFF81FFFFFF38 2E7BAD41>I<00000FFE0000000001FFFFF000000007FFFFFC0000001FF803FF0000007F E000FFC00000FF80003FE00001FF00001FF00003FE00000FF80007FC000007FC000FFC00 0007FE000FF8000003FE001FF8000003FF003FF8000003FF803FF0000001FF803FF00000 01FF807FF0000001FFC07FF0000001FFC07FF0000001FFC0FFF0000001FFE0FFF0000001 FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE0FFF0000001FF E0FFF0000001FFE0FFF0000001FFE0FFF0000001FFE07FF0000001FFC07FF0000001FFC0 7FF0000001FFC07FF0000001FFC03FF8000003FF803FF8000003FF801FF8000003FF001F FC000007FF000FFC000007FE0007FE00000FFC0003FE00000FF80001FF00001FF00000FF 80003FE000007FE000FFC000001FFC07FF00000007FFFFFC00000001FFFFF0000000001F FF000000332E7DAD3A>I<007F803F80FFFF80FFE0FFFF81FFF8FFFF83C3FCFFFF8707FE 03FF8E07FE01FF9C0FFF01FFB80FFF01FFB80FFF01FFF00FFF01FFF00FFF01FFF007FE01 FFE003FC01FFE001F801FFE0000001FFE0000001FFC0000001FFC0000001FFC0000001FF C0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0 000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC000 0001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC0000001FFC00000 01FFC00000FFFFFFE000FFFFFFE000FFFFFFE000FFFFFFE000282E7DAD2F>114 D<000FFE01C000FFFFC7C003FFFFFFC007F801FFC00FC0007FC01F00001FC03F00000FC0 7E000007C07E000007C07E000003C0FE000003C0FE000003C0FF000003C0FF800003C0FF E0000000FFF80000007FFFC000007FFFFF00003FFFFFE0003FFFFFF8001FFFFFFC000FFF FFFF0007FFFFFF8001FFFFFFC0007FFFFFE0001FFFFFE00000FFFFF0000003FFF0000000 7FF87000001FF8F000000FF8F0000007F8F8000007F8F8000003F8F8000003F8FC000003 F0FC000003F0FE000003F0FF000007E0FF800007E0FFC0000FC0FFF0001F80FEFC00FF00 F87FFFFC00F01FFFF000E003FF8000252E7CAD2E>I<0001E000000001E000000001E000 000001E000000001E000000001E000000003E000000003E000000003E000000007E00000 0007E000000007E00000000FE00000000FE00000001FE00000003FE00000003FE0000000 7FE0000001FFE0000003FFE000000FFFFFFFC0FFFFFFFFC0FFFFFFFFC0FFFFFFFFC000FF E0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0 000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE000 0000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE00000 00FFE0000000FFE0000000FFE000F000FFE000F000FFE000F000FFE000F000FFE000F000 FFE000F000FFE000F000FFE000F000FFE000F0007FE000E0007FF001E0003FF001E0003F F003C0001FF80380000FFE0F000007FFFE000001FFFC0000001FF00024427EC12D>I E /Fr 8 117 df<0003FF0000001FFFF000007FFFFC0000FC07FE0001F001FF0003C000 FF800780007FC00FE0007FE00FF0007FE01FF8007FF01FFC007FF01FFC007FF01FFC007F F01FFC007FF00FF8007FF007F0007FE003E0007FE0008000FFE0000000FFC0000000FF80 000001FF00000001FE00000003FC00000007F80000001FF000000FFFC000000FFF800000 0FFFF000000007FE00000000FF000000007FC00000007FE00000003FF00000003FF80000 003FF80000001FFC0000001FFC0000001FFE0000001FFE1F80001FFE3FC0001FFE7FE000 1FFEFFF0001FFEFFF0001FFEFFF0001FFCFFF0001FFCFFF0001FFCFFE0003FF87FC0003F F87F80007FF03E00007FE01FC000FFC00FF803FF8007FFFFFF0001FFFFFC00007FFFF000 0007FF000027397CB730>51 D<0000001FFE000060000003FFFFE000E000001FFFFFF801 E000007FFFFFFE03E00001FFFC00FF07E00007FFC0001FCFE0000FFF000007FFE0001FFC 000001FFE0003FF0000000FFE0007FE00000007FE000FFC00000003FE001FF800000001F E003FF000000000FE007FF000000000FE00FFE0000000007E00FFE0000000007E01FFC00 00000003E01FFC0000000003E03FF80000000003E03FF80000000001E03FF80000000001 E07FF80000000001E07FF80000000001E07FF00000000000007FF0000000000000FFF000 0000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF00000000000 00FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF000 0000000000FFF00000000000007FF00000000000007FF00000000000007FF80000000000 007FF80000000001E03FF80000000001E03FF80000000001E03FF80000000001E01FFC00 00000001E01FFC0000000003C00FFE0000000003C00FFE0000000003C007FF0000000007 8003FF00000000078001FF800000000F0000FFC00000001F00007FE00000003E00003FF0 0000007C00001FFC000000F800000FFF000003F0000007FFC0000FE0000001FFFC007F80 0000007FFFFFFF000000001FFFFFFC0000000003FFFFE000000000001FFE0000003B3D7B BB46>67 D<003FFE00000001FFFFE0000007FFFFF800000FE007FC00000FF001FE00001F F800FF00001FF8007F80001FF8007FC0001FF8003FC0000FF0003FE00007E0003FE00003 C0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000FFFFE00000 1FFFFFE000007FF83FE00003FF803FE00007FC003FE0000FF0003FE0001FE0003FE0003F E0003FE0007FC0003FE0007FC0003FE000FF80003FE000FF80003FE000FF80003FE000FF 80003FE000FF80007FE0007FC0007FE0007FC000DFE0003FE0039FF0001FF80F0FFFE007 FFFE0FFFE001FFFC07FFE0003FE000FFE02B267DA52F>97 D<0001FF8000000FFFF00000 7FFFFC0000FF81FE0003FE007F8007F8003F800FF0001FC00FF0000FE01FE0000FE03FE0 000FF03FE00007F07FC00007F07FC00007F87FC00007F8FFC00007F8FFC00007F8FFFFFF FFF8FFFFFFFFF8FFFFFFFFF8FFC0000000FFC0000000FFC0000000FFC00000007FC00000 007FC00000007FC00000003FE00000003FE00000781FE00000781FF00000780FF00000F0 07F80001F003FC0003E001FE000FC000FFC07F80003FFFFE00000FFFF8000000FFC00025 267DA52C>101 D<00FE00000000FFFE00000000FFFE00000000FFFE00000000FFFE0000 000007FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000 000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000 000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE00FF 800003FE03FFF00003FE0FFFF80003FE1E03FC0003FE3801FE0003FE6001FF0003FEC000 FF0003FFC000FF8003FF8000FF8003FF0000FF8003FF0000FF8003FF0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF80FFFFF83FFFFEFFFFF83FFFFEFFFFF83F FFFEFFFFF83FFFFE2F3C7CBB36>104 D<00FE01FF0000FFFE1FFFF000FFFE7FFFFC00FF FEFC07FE00FFFFF001FF0007FFC0007F8003FF00003FC003FE00003FE003FE00001FF003 FE00001FF803FE00000FF803FE00000FFC03FE00000FFC03FE000007FC03FE000007FE03 FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03 FE000007FE03FE000007FE03FE000007FC03FE00000FFC03FE00000FFC03FE00000FFC03 FE00000FF803FE00001FF003FE00001FF003FF00003FE003FF80007FC003FFC000FF8003 FFE001FF0003FEFC07FE0003FE7FFFF80003FE1FFFE00003FE03FE000003FE0000000003 FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003 FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE00000000FF FFF8000000FFFFF8000000FFFFF8000000FFFFF80000002F377DA536>112 D<01FC03F000FFFC0FFC00FFFC1FFF00FFFC3C3F80FFFC707F8007FCE0FFC003FCC0FFC0 03FD80FFC003FD80FFC003FF807F8003FF003F0003FF001E0003FF00000003FE00000003 FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE 00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00 000003FE00000003FE00000003FE00000003FE00000003FE000000FFFFFC0000FFFFFC00 00FFFFFC0000FFFFFC000022267DA528>114 D<000F0000000F0000000F0000000F0000 000F0000001F0000001F0000001F0000001F0000003F0000003F0000007F0000007F0000 00FF000001FF000003FF000007FF00001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF0000 01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C01FF003C00FF0078 00FF8078007F80F0003FC1E0001FFFC0000FFF800001FE001E377EB626>116 D E /Fs 80 124 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<70F8F8F8F8F8F8F87070707070707070707020202020 20000000000070F8F8F87005217CA00D>33 D<7038F87CFC7EFC7E743A04020402040208 04080410081008201040200F0E7E9F17>I<70F8FCFC74040404080810102040060E7C9F 0D>39 D<0020004000800100020006000C000C0018001800300030003000700060006000 6000E000E000E000E000E000E000E000E000E000E000E000E00060006000600070003000 30003000180018000C000C000600020001000080004000200B2E7DA112>I<8000400020 00100008000C00060006000300030001800180018001C000C000C000C000E000E000E000 E000E000E000E000E000E000E000E000E000C000C000C001C00180018001800300030006 0006000C00080010002000400080000B2E7DA112>I<01800180018001800180C183F18F 399C0FF003C003C00FF0399CF18FC1830180018001800180018010147DA117>I<000600 000006000000060000000600000006000000060000000600000006000000060000000600 000006000000060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600 000006000000060000000600000006000000060000000600000006000000060000000600 00000600000006000000060000000600001C207D9A23>I<70F8FCFC7404040408081010 2040060E7C840D>II<70F8F8F87005057C840D>I<0001000300 03000600060006000C000C000C00180018001800300030003000600060006000C000C000 C00180018001800300030003000600060006000C000C000C001800180018003000300030 00600060006000C000C000C000102D7DA117>I<03F0000E1C001C0E0018060038070070 0380700380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F0 03C0F003C0F003C0F003C0F003C07003807003807003807807803807001806001C0E000E 1C0003F000121F7E9D17>I<018003800F80F38003800380038003800380038003800380 038003800380038003800380038003800380038003800380038003800380038007C0FFFE 0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803 C02007C00007C0000780000780000F00000E00001C0000380000700000600000C0000180 000300000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C 1C00100E00200F00780F80780780780780380F80000F80000F00000F00000E00001C0000 380003F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0F0 0780400780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E00 001E00002E00002E00004E00008E00008E00010E00020E00020E00040E00080E00080E00 100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E00 000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE0001000001000001000 0010000010000010000011F000161C00180E001007001007800003800003800003C00003 C00003C07003C0F003C0F003C0E00380400380400700200600100E000C380003E000121F 7E9D17>I<007C000182000701000E03800C07801C078038030038000078000070000070 0000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C070 03C07003C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<400000 7FFFC07FFF807FFF80400100800200800200800400000800000800001000002000002000 00400000400000C00000C00001C000018000038000038000038000038000078000078000 078000078000078000078000078000030000121F7D9D17>I<03F0000C0C001006003003 002001806001806001806001807001807803003E03003F06001FC8000FF00003F80007FC 000C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0C000806001802001 001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C0038060038070070070070 0380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC00E 13C003E3C0000380000380000380000700300700780600780E00700C002018001070000F C000121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I<70 F8F8F8700000000000000000000070F0F8F878080808101010202040051D7C930D>I<0F C0307040384038E03CF03CF03C603C0038007000E000C001800180010003000200020002 000200020002000000000000000000000007000F800F800F8007000E207D9F15>63 D<000100000003800000038000000380000007C0000007C0000007C0000009E0000009E0 000009E0000010F0000010F0000010F00000207800002078000020780000403C0000403C 0000403C0000801E0000801E0000FFFE0001000F0001000F0001000F0002000780020007 8002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0 038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C00004078000040 F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 780000007C0000407C0000403C0000401C0000401E0000800E0000800700010003800200 01C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0070001E0 0F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E07C0001E0 3C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E0046000781820 000FE0001E217D9F24>III75 DIII<001F800000F0F00001C0380007801E000F000F00 0E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0 F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E0 7C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F000 001F80001C217D9F23>II82 D<07E0800C198010078030 0380600180600180E00180E00080E00080E00080F00000F000007800007F00003FF0001F FC000FFE0003FF00001F800007800003C00003C00001C08001C08001C08001C08001C0C0 0180C00380E00300F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060 400F0020400F0020C00F0030800F0010800F0010800F0010800F0010000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F8000 07FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003C0020001E0040001F00C00 00F0080000781000007C1000003C2000003E4000001E4000000F8000000F800000078000 0003C0000007E0000005E0000009F0000018F8000010780000207C0000603C0000401E00 00801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I91 D<080410082010201040204020804080408040B85C FC7EFC7E7C3E381C0F0E7B9F17>II<1FE0003030 00781800781C00300E00000E00000E00000E0000FE00078E001E0E00380E00780E00F00E 10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000EC3800F01C0 0F00E00E00E00E00700E00700E00780E00780E00780E00780E00780E00780E00700E0070 0E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C1C1E381E380C70007000 F000F000F000F000F000F00070007000380138011C020E0C03F010147E9314>I<000380 003F8000038000038000038000038000038000038000038000038000038000038003E380 061B801C0780380380380380700380700380F00380F00380F00380F00380F00380F00380 7003807003803803803807801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E 00380700380700700700700380F00380F00380FFFF80F00000F00000F000007000007000 003800801800800C010007060001F80011147F9314>I<007C00C6018F038F0706070007 0007000700070007000700FFF00700070007000700070007000700070007000700070007 000700070007000700070007007FF01020809F0E>I<0000E003E3300E3C301C1C30380E 00780F00780F00780F00780F00780F00380E001C1C001E380033E0002000002000003000 003000003FFE001FFF800FFFC03001E0600070C00030C00030C00030C000306000603000 C01C038003FC00141F7F9417>I<0E0000FE00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E3E000E43000E81800F01C00F01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FF E7FC16207F9F19>I<1C001E003E001E001C000000000000000000000000000E007E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A 1F809E0C>I<00E001F001F001F000E0000000000000000000000000007007F000F00070 007000700070007000700070007000700070007000700070007000700070007000700070 007000706070F060F0C061803F000C28829E0E>I<0E0000FE00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E0FF00E03C00E03000E02000E0400 0E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E000E07000E07800E0380 0E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E81C81C000F00F00E000F 00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C00F01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FF E7FC16147F9319>I<01F800070E001C03803801C03801C07000E07000E0F000F0F000F0 F000F0F000F0F000F0F000F07000E07000E03801C03801C01C0380070E0001F80014147F 9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E00780E00780E00780E00 780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E3E000E00000E00000E00 000E00000E00000E00000E00000E0000FFE000151D7F9319>I<03E0800619801C05803C 0780380380780380700380F00380F00380F00380F00380F00380F0038070038078038038 03803807801C0B800E138003E38000038000038000038000038000038000038000038000 0380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E000E000E000E000E000E00 0E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030704030C010C010C010E0 0078007F803FE00FF00070803880188018C018C018E030D0608F800D147E9312>I<0200 02000200060006000E000E003E00FFF80E000E000E000E000E000E000E000E000E000E00 0E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E0 0701C007018003810001C20000E40000EC00007800003800003C00007C00004E00008700 0107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000E001E001C00380078007 010E011E011C0338027006700EFFFE10147F9314>II E /Ft 45 122 df<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<01F000071C000C060018030038 03803803807001C07001C07001C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07001C07001C07001C07803C038 03803803801C07000C0600071C0001F00013227EA018>48 D<008003800F80F380038003 800380038003800380038003800380038003800380038003800380038003800380038003 80038003800380038003800380038007C0FFFE0F217CA018>I<03F8000C1E0010070020 07804007C07807C07803C07807C03807C0000780000780000700000F00000E0000380003 F000001C00000F000007800007800003C00003C00003E02003E07003E0F803E0F803E0F0 03C04003C0400780200780100F000C1C0003F00013227EA018>51 D<000200000600000E00000E00001E00001E00002E00004E00004E00008E00008E00010E 00020E00020E00040E00040E00080E00100E00100E00200E00200E00400E00800E00FFFF F8000E00000E00000E00000E00000E00000E00000E00001F0001FFF015217FA018>I<10 00801E07001FFF001FFE001FF80013E00010000010000010000010000010000010000010 F800130E001407001803801003800001C00001C00001E00001E00001E00001E07001E0F0 01E0F001E0E001C08001C04003C04003802007001006000C1C0003F00013227EA018>I< 007E0001C1000300800601C00E03C01C03C0180180380000380000780000700000700000 F0F800F30C00F40600F40300F80380F801C0F001C0F001E0F001E0F001E0F001E0F001E0 7001E07001E07001E03801C03801C01803801C03000C0600070C0001F00013227EA018> I<01F800060E000803001001802001802000C06000C06000C06000C07000C07801803E01 003F02001FC4000FF80003F80003FC00067F00083F80100F803007C06001C06000E0C000 E0C00060C00060C00060C000606000406000C03000801803000E0E0003F00013227EA018 >56 D<01F000060C000C0600180700380380700380700380F001C0F001C0F001C0F001E0 F001E0F001E0F001E0F001E07001E07003E03803E01805E00C05E00619E003E1E00001C0 0001C00001C0000380000380300300780700780600700C002018001030000FC00013227E A018>I<0001800000018000000180000003C0000003C0000003C0000005E0000005E000 000DF0000008F0000008F0000010F800001078000010780000203C0000203C0000203C00 00401E0000401E0000401E0000800F0000800F0000FFFF000100078001000780030007C0 020003C0020003C0040003E0040001E0040001E00C0000F00C0000F03E0001F8FF800FFF 20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E000070 1E0000701C0000303C0000303C0000307C0000107800001078000010F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F800000078000000780000107C000010 3C0000103C0000101C0000201E0000200E000040070000400380008001C0010000E00200 00381C000007E0001C247DA223>67 D69 DI73 D77 DI80 D82 D<03F0200C0C601802603001E07000E06000 60E00060E00060E00020E00020E00020F00000F000007800007F00003FF0001FFE000FFF 0003FF80003FC00007E00001E00000F00000F0000070800070800070800070800070C000 60C00060E000C0F000C0C80180C6070081FC0014247DA21B>I<7FFFFFF8780780786007 8018400780084007800840078008C007800C800780048007800480078004800780040007 800000078000000780000007800000078000000780000007800000078000000780000007 800000078000000780000007800000078000000780000007800000078000000780000007 80000007800000078000000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E 0018070000070000070000070000FF0007C7001E07003C0700780700700700F00708F007 08F00708F00F087817083C23900FC1E015157E9418>97 D<0E0000FE00001E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F000E61C0 0E80600F00300E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E 0E001C0E003C0E00380F00700C80600C41C0083F0017237FA21B>I<01FE000703000C07 801C0780380300780000700000F00000F00000F00000F00000F00000F00000F000007000 007800403800401C00800C010007060001F80012157E9416>I<0000E0000FE00001E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E001F8E007 04E00C02E01C01E03800E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F0 00E07000E07800E03800E01801E00C02E0070CF001F0FE17237EA21B>I<01FC00070700 0C03801C01C03801C07801E07000E0F000E0FFFFE0F00000F00000F00000F00000F00000 7000007800203800201C00400E008007030000FC0013157F9416>I<003C00C6018F038F 030F070007000700070007000700070007000700FFF80700070007000700070007000700 0700070007000700070007000700070007000700070007807FF8102380A20F>I<000070 01F198071E180E0E181C07001C07003C07803C07803C07803C07801C07001C07000E0E00 0F1C0019F0001000001000001800001800001FFE000FFFC00FFFE03800F0600030400018 C00018C00018C000186000306000303800E00E038003FE0015217F9518>I<0E0000FE00 001E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E1F800E60C00E80E00F00700F00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C 001E003E001E001C00000000000000000000000000000000000E00FE001E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A227FA10E >I<01C003E003E003E001C00000000000000000000000000000000001E00FE001E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E060E0F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E03FC0E01 F00E01C00E01800E02000E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07 000E07800E03C00E01C00E01E00E00F00E00F8FFE3FE17237FA21A>I<0E00FE001E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B237FA20E>I<0E1F C07F00FE60E183801E807201C00F003C00E00F003C00E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E00E003800E00E003800E00E003800E0FFE3FF8FFE27157F942A >I<0E1F80FE60C01E80E00F00700F00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E0070FFE7FF18157F941B>I< 01FC000707000C01801800C03800E0700070700070F00078F00078F00078F00078F00078 F00078F000787000707800F03800E01C01C00E038007070001FC0015157F9418>I<0E1F 00FE61C00E80600F00700E00380E003C0E001C0E001E0E001E0E001E0E001E0E001E0E00 1E0E001E0E003C0E003C0E00380F00700E80E00E41C00E3F000E00000E00000E00000E00 000E00000E00000E00000E00000E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F06 0F000E000E000E000E000E000E000E000E000E000E000E000E000E000F00FFF010157F94 13>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C80 0CC00CC008E018D0308FC00E157E9413>I<02000200020002000600060006000E001E00 3E00FFF80E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E04 0E040708030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E0070 0E00700E00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F0060170 03827800FC7F18157F941B>III121 D E /Fu 20 122 df45 D<007F800001FFF0000701F8000C00 7E0018001F0010000F0020000F80400007C0400007C0400003E0B00003E0F80003E0FC00 03E0FC0003E0FC0003E0780003E0000003E0000007C0000007C00000078000000F800000 0F0000001F0000001E0000003C0000007800000070000000E0000001C000000380000007 0000000E0000000C000000180000003000000060000000C0002001800020030000200600 00400C00004008000040100000C03FFFFFC07FFFFF80FFFFFF80FFFFFF801B2F7DAE21> 50 D<00003FE0010001FFF8030007F01E03001F800307003E000087007800004F00F000 002F01E000001F03C000000F078000000F0F800000070F000000071F000000031E000000 033E000000033C000000017C000000017C000000017C000000017800000000F800000000 F800000000F800000000F800000000F800000000F800000000F800000000F800000000F8 00000000F800000000F80000000078000000007C000000007C000000017C000000013C00 0000013E000000011E000000011F000000020F000000020F80000006078000000403C000 000801E000000800F00000100078000020003E0000C0001F8003800007F00F000001FFFC 0000003FE00028337CB130>67 DI 73 D77 D 80 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E 001E0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F 001E003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478 003E047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E060038018 0700040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000 F80000F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F0002 07000403801800E060003F80181F7D9E1D>99 D<003F800000E0E0000380380007003C00 0E001E001E001E001C000F003C000F007C000F0078000F8078000780F8000780F8000780 FFFFFF80F8000000F8000000F8000000F8000000F8000000F8000000780000007C000000 3C0000003C0000801E0000800E0001000F0002000780020001C00C0000F03000001FC000 191F7E9E1D>101 D<0780000000FF80000000FF800000000F8000000007800000000780 000000078000000007800000000780000000078000000007800000000780000000078000 00000780000000078000000007800000000780000000078000000007800000000780FE00 000783078000078C03C000079001E00007A001E00007A000F00007C000F00007C000F000 078000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F0000FC001F800FFFC1F FF80FFFC1FFF8021327EB125>104 D<07000F801F801F800F8007000000000000000000 00000000000000000000000000000780FF80FF800F800780078007800780078007800780 078007800780078007800780078007800780078007800780078007800780078007800FC0 FFF8FFF80D307EAF12>I<0780FE001FC000FF83078060F000FF8C03C18078000F9001E2 003C0007A001E4003C0007A000F4001E0007C000F8001E0007C000F8001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00 078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E00078000F0001E000FC001F8003F00FFFC 1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C000 0F9001E00007A001E00007A000F00007C000F00007C000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF80211F7E9E 25>I<001FC00000F0780001C01C00070007000F0007801E0003C01C0001C03C0001E03C 0001E0780000F0780000F0780000F0F80000F8F80000F8F80000F8F80000F8F80000F8F8 0000F8F80000F8F80000F8780000F07C0001F03C0001E03C0001E01E0003C01E0003C00F 00078007800F0001C01C0000F07800001FC0001D1F7E9E21>I<0781FC00FF860700FF88 03C00F9001E007A000F007C00078078000780780003C0780003C0780003E0780001E0780 001F0780001F0780001F0780001F0780001F0780001F0780001F0780001F0780001F0780 003E0780003E0780003C0780007C0780007807C000F007A000F007A001E0079803800786 0F000781F800078000000780000007800000078000000780000007800000078000000780 00000780000007800000078000000FC00000FFFC0000FFFC0000202D7E9E25>I<0783E0 FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000078000078000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0 300070600030E00030E00010E00010E00010F00010F800007E00003FF0001FFF000FFFC0 03FFE0003FF00001F80000F880003C80003C80001CC0001CC0001CE0001CE00018F00038 F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000400000400000400000C0 0000C00000C00001C00001C00003C00007C0000FC0001FFFE0FFFFE003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C01003C01003C01003C01003C01003C01003C01003C01001C02001E02000E0 400078C0001F00142C7FAB19>I121 D E /Fv 5 85 df<00000000600000000000E00000000000E00000000001E00000000003 E00000000003E00000000007E00000000007F0000000000DF0000000000DF00000000019 F00000000039F00000000031F00000000061F00000000061F000000000C1F000000000C1 F00000000181F00000000381F00000000301F00000000601F00000000601F00000000C01 F80000000C00F80000001800F80000003800F80000003000F80000006000F80000006000 F8000000C000F8000000C000F8000001FFFFF8000001FFFFF80000030000F80000060000 F80000060000F800000C0000F800000C0000FC00001800007C00001800007C0000300000 7C00007000007C00006000007C0000C000007C0000C000007C0001C000007C0003C00000 7C000FE00000FE00FFF8001FFFE0FFF8001FFFE02B327CB134>65 D<000FFFFFFC0000000FFFFFFF800000007E000FE00000007C0003F00000007C0000F800 00007C00007C0000007C00007C000000F800003E000000F800001E000000F800001F0000 00F800001F000001F000001F000001F000000F800001F000000F800001F000000F800003 E000000F800003E000000F800003E000000F800003E000001F800007C000001F800007C0 00001F800007C000001F800007C000001F80000F8000001F00000F8000003F00000F8000 003F00000F8000003F00001F0000007E00001F0000007E00001F0000007E00001F000000 7C00003E000000FC00003E000000F800003E000001F800003E000001F000007C000003E0 00007C000003E000007C000007C000007C00000F800000F800000F000000F800001E0000 00F800003C000000F8000078000001F00001F0000001F00003E0000001F0000F80000003 F0007F000000FFFFFFFC000000FFFFFFC000000031317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007E0003F800007C0000F800007C00007800007C0000 7800007C0000300000F80000300000F80000300000F80000300000F80000300001F00000 300001F00000300001F00000300001F00000600003E00180600003E00180000003E00180 000003E00180000007C00300000007C00300000007C00700000007C01F0000000FFFFE00 00000FFFFE0000000F801E0000000F800E0000001F000C0000001F000C0000001F000C00 00001F000C0000003E00180000003E00180000003E00000000003E00000000007C000000 00007C00000000007C00000000007C0000000000F80000000000F80000000000F8000000 0000F80000000001F00000000001F00000000001F00000000003F000000000FFFFE00000 00FFFFE00000002D317BB02E>70 D<000FFFFFE000000FFFFFFC0000007E003F0000007C 000F8000007C0007C000007C0003E000007C0003E00000F80001F00000F80001F00000F8 0001F00000F80003F00001F00003F00001F00003F00001F00003F00001F00003E00003E0 0007E00003E00007C00003E0000FC00003E0000F800007C0001F000007C0003E000007C0 0078000007C001E000000F800F8000000FFFFC0000000F800E0000000F80070000001F00 038000001F0003C000001F0001E000001F0001E000003E0001F000003E0001F000003E00 01F000003E0001F000007C0003F000007C0003F000007C0003E000007C0003E00000F800 07E00000F80007E00000F80007E00000F80007E01801F00007E03001F00007E03001F000 07E02003F00003E060FFFF8001E0C0FFFF8000F180000000003E002D327BB033>82 D<07FFFFFFFFF007FFFFFFFFF00FE007E007F00F8007C001F00E0007C000E00C0007C000 E01C0007C000E018000F8000E018000F8000E030000F8000C030000F8000C030001F0000 C060001F0000C060001F0000C060001F0000C0C0003E00018000003E00000000003E0000 0000003E00000000007C00000000007C00000000007C00000000007C0000000000F80000 000000F80000000000F80000000000F80000000001F00000000001F00000000001F00000 000001F00000000003E00000000003E00000000003E00000000003E00000000007C00000 000007C00000000007C00000000007C0000000000F80000000000F80000000000F800000 00000F80000000001F00000000001F00000000001F00000000003F000000003FFFFF0000 003FFFFF0000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 799 911 a Fv(D)25 b(R)g(A)g(F)g(T)497 1002 y Fu(MPI-2)d(Dynamic)e(Pro)r(cess)i(Chapter)621 1196 y Ft(Message)16 b(P)o(assing)h(In)o(terface)e(F)l(orum)802 1323 y(August)i(16,)f(1996)77 1381 y(This)h(w)o(ork)f(w)o(as)h(supp)q (orted)g(in)f(part)h(b)o(y)e(ARP)l(A)h(and)h(NSF)e(under)h(gran)o(t)h (ASC-9310330,)i(the)192 1439 y(National)d(Science)f(F)l(oundation)i (Science)e(and)i(T)l(ec)o(hnology)f(Cen)o(ter)f(Co)q(op)q(erativ)o(e)76 1497 y(Agreemen)o(t)e(No.)22 b(CCR-8809615,)d(and)e(b)o(y)e(the)h (Commission)e(of)j(the)f(Europ)q(ean)i(Comm)o(unit)n(y)654 1555 y(through)f(Esprit)f(pro)s(ject)g(P6643.)p eop %%Page: 1 2 1 1 bop 166 49 a Fs(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 106 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 361 a Fr(Chapter)31 b(3)75 568 y Fq(Pro)s(cess)39 b(Creation)h(and)e(Managemen)m(t)75 809 y Fp(3.1)59 b(Intro)r(duction) 75 958 y Fo(Curren)o(t)14 b(Status:)35 b Fn(No)14 b(v)o(otes)1875 1005 y Fm(>)i Fl(\(Sept\))166 1061 y Fk(MPI-1)11 b Fs(pro)o(vides)i(an) g(in)o(terface)f(that)g(allo)o(ws)g(pro)q(cesses)h(in)g(a)f(parallel)i (program)d(to)h(comm)o(unicate)75 1118 y(with)21 b(one)g(another.)36 b Fk(MPI-1)20 b Fs(sp)q(eci\014es)i(neither)g(ho)o(w)e(the)h(pro)q (cesses)g(are)g(created,)g(nor)g(ho)o(w)f(they)75 1174 y(establish)g(comm)o(unication.)31 b(Moreo)o(v)o(er,)18 b(an)h Fk(MPI-1)f Fs(application)i(is)g(static,)f(that)f(is,)i(no)f (pro)q(cesses)75 1231 y(can)c(b)q(e)h(added)g(to)f(or)f(deleted)j(from) d(an)h(application)i(after)e(it)g(has)g(b)q(een)i(started.)166 1287 y Fk(MPI)f Fs(users)h(ha)o(v)o(e)g(ask)o(ed)f(that)h(the)g Fk(MPI-1)f Fs(mo)q(del)i(b)q(e)f(relaxed)h(to)e(allo)o(w)h(pro)q(cess)h (creation)f(and)75 1344 y(managemen)o(t)f(after)h(an)g Fk(MPI)g Fs(application)i(has)e(b)q(een)h(started.)25 b(A)17 b(main)h(imp)q(etus)g(comes)f(from)g(the)75 1400 y(PVM)f([)p Fj(?)p Fs(])g(researc)o(h)g(e\013ort,)g(whic)o(h)h(has)g (pro)o(vided)g(a)g(w)o(ealth)f(of)g(exp)q(erience)j(with)e(dynamic)h (pro)q(cess)75 1456 y(managemen)o(t)c(and)i(resource)f(con)o(trol)g (that)f(illustrates)j(their)e(b)q(ene\014ts)i(and)e(p)q(oten)o(tial)h (pitfalls.)166 1513 y(The)11 b Fk(MPI)f Fs(F)l(orum)g(decided)i(not)f (to)f(address)g(resource)h(con)o(trol)f(in)i Fk(MPI-1)e Fs(b)q(ecause)h(it)g(w)o(as)f(not)g(able)75 1569 y(to)k(design)i(a)e(p) q(ortable)h(in)o(terface)g(that)f(w)o(ould)i(b)q(e)f(appropriate)g(for) f(the)h(wide)g(range)g(of)f(existing)i(and)75 1626 y(p)q(oten)o(tial)f (resource)f(and)g(pro)q(cess)g(con)o(trollers.)20 b(Resource)15 b(con)o(trol)e(can)h(encompass)g(a)g(wide)h(range)e(of)75 1682 y(abilities)j(including)h(adding)d(and)g(deleting)i(no)q(des)e (from)f(a)h(virtual)g(parallel)i(mac)o(hine,)e(reserving)h(and)75 1739 y(sc)o(heduling)j(resources,)e(managing)g(compute)h(partitions)f (of)g(an)g(MPP)l(,)g(and)g(returning)h(information)75 1795 y(ab)q(out)e(a)o(v)m(ailable)i(resources.)166 1852 y(MPI-2)j(assumes)g(that)g(resource)h(con)o(trol)f(is)h(pro)o(vided)g (exernally)h({)e(probably)h(b)o(y)f(computer)75 1908 y(v)o(endors)d(in)h(the)f(case)g(of)g(tigh)o(tly)g(coupled)i(systems)e (or)f(b)o(y)h(a)g(third)h(part)o(y)e(soft)o(w)o(are)f(pac)o(k)m(age)i (when)75 1965 y(the)e(en)o(vironmen)o(t)h(is)f(a)g(cluster)h(of)f (hosts.)166 2021 y(The)i(reasons)g(for)f(adding)i(dynamic)g(pro)q(cess) f(managemen)o(t)g(to)f(MPI)h(are)g(b)q(oth)g(tec)o(hnical)i(and)75 2078 y(practical.)24 b(Imp)q(ortan)o(t)16 b(classes)h(of)f(message)g (passing)h(applications)h(require)g(dynamic)f(pro)q(cess)g(con-)75 2134 y(trol.)i(These)14 b(include)h(task)e(farms,)f(serial)i (applications)h(with)e(parallel)i(mo)q(dules,)f(and)g(problems)g(that) 75 2190 y(require)i(a)f(run-time)h(assessmen)o(t)f(of)g(the)g(n)o(um)o (b)q(er)h(and)f(t)o(yp)q(e)h(of)f(pro)q(cesses)g(that)g(should)h(b)q(e) g(started.)75 2247 y(On)h(the)g(practical)h(side,)g(users)f(of)f(w)o (orkstation)g(clusters)h(who)g(are)f(migrating)h(from)f(PVM)h(to)f(MPI) 75 2303 y(ma)o(y)c(b)q(e)i(accustomed)f(to)f(using)i(PVM's)e (capabilities)j(for)d(pro)q(cess)h(and)g(resource)g(managemen)o(t.)19 b(The)75 2360 y(lac)o(k)c(of)g(these)h(features)e(is)i(a)f(practical)h (stum)o(bling)g(blo)q(c)o(k)g(to)f(migration.)166 2416 y(While)g(dynamic)h(pro)q(cess)e(managemen)o(t)f(is)i(essen)o(tial,)g (adding)g(it)g(to)e(MPI)h(should)h(not)f(compro-)75 2473 y(mise)i(the)f(p)q(ortabilit)o(y)h(or)f(p)q(erformance)g(of)g(MPI)g (applications.)22 b(In)15 b(particular:)143 2567 y Fm(\017)23 b Fs(The)f(MPI-2)f(dynamic)i(pro)q(cess)e(managemen)o(t)g(mo)q(del)i(m) o(ust)e(apply)h(to)f(the)h(v)m(ast)f(ma)s(jorit)o(y)189 2623 y(of)e(curren)o(t)h(parallel)h(en)o(vironmen)o(ts.)34 b(These)20 b(include)j(ev)o(erything)d(from)f(tigh)o(tly)h(in)o (tegrated)189 2679 y(MPPs)d(suc)o(h)i(as)f(the)g(In)o(tel)h(P)o(aragon) e(and)h(the)g(Meik)o(o)g(CS-2)h(to)e(heterogeneous)h(net)o(w)o(orks)f (of)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fs(2)506 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)189 49 y Fs(w)o(orkstations.)143 143 y Fm(\017)23 b Fs(MPI)16 b(m)o(ust)g(not)g(tak)o(e)g(o)o(v)o(er)f (op)q(erating)i(system)f(resp)q(onsibiliti)q(es.)26 b(It)17 b(should)g(instead)g(pro)o(vide)189 200 y(a)e(clean)h(in)o(terface)f(b) q(et)o(w)o(een)h(an)f(application)i(and)e(system)g(soft)o(w)o(are.)143 293 y Fm(\017)23 b Fs(MPI)17 b(m)o(ust)g(con)o(tin)o(ue)h(to)f(guaran)o (tee)g(comm)o(unication)h(determinism,)h(i.e.,)e(dynamic)i(pro)q(cess) 189 350 y(managemen)o(t)14 b(m)o(ust)h(not)f(in)o(tro)q(duce)j(una)o(v) o(oidable)f(race)f(conditions.)143 444 y Fm(\017)23 b Fs(MPI)15 b(m)o(ust)f(not)h(con)o(tain)h(features)e(that)h(compromise)g (p)q(erformance.)143 537 y Fm(\017)23 b Fs(MPI-1)16 b(programs)f(m)o (ust)h(w)o(ork)g(under)h(MPI-2,)f(i.e.,)g(the)h(MPI-1)f(static)g(pro)q (cess)h(mo)q(del)h(m)o(ust)189 594 y(b)q(e)e(a)e(sp)q(ecial)j(case)f (of)e(the)i(MPI-2)f(dynamic)h(mo)q(del.)166 688 y(The)22 b(MPI)h(dynamic)g(pro)q(cess)f(managemen)o(t)g(mo)q(del)h(addresses)g (these)f(issues)h(in)g(t)o(w)o(o)e(w)o(a)o(ys.)75 744 y(First,)f(MPI)g(remains)g(primarily)h(a)e(comm)o(unication)h(library)l (.)35 b(It)19 b(do)q(es)h(not)g(manage)f(the)g(parallel)75 801 y(en)o(vironmen)o(t)e(in)g(whic)o(h)h(a)e(parallel)i(program)e (executes,)h(though)f(it)h(pro)o(vides)g(a)g(minimal)h(in)o(terface)75 857 y(b)q(et)o(w)o(een)e(an)f(application)h(and)g(external)g(resource)f (and)g(pro)q(cess)h(managers.)166 914 y(Second,)h(MPI-2)g(do)q(es)g (not)f(c)o(hange)h(the)g(concept)g(of)f(comm)o(unicator.)24 b(Once)18 b(a)e(comm)o(unicator)75 970 y(is)h(built,)g(it)g(b)q(eha)o (v)o(es)f(as)g(sp)q(eci\014ed)j(in)e(MPI-1.)23 b(A)16 b(comm)o(unicator)g(is)h(nev)o(er)f(c)o(hanged)h(once)f(created,)75 1026 y(and)f(it)h(is)g(alw)o(a)o(ys)e(created)h(using)h(deterministic)h (collectiv)o(e)g(seman)o(tics.)75 1170 y Fp(3.2)59 b(The)20 b(MPI-2)f(Dynamic)g(Pro)r(cess)i(Mo)r(del)75 1271 y Fs(The)g(MPI-2)g (dynamic)h(pro)q(cess)g(mo)q(del)g(allo)o(ws)f(for)f(the)i(creation)f (and)g(destruction)h(of)f(pro)q(cesses)75 1328 y(after)14 b(an)g(MPI)h(application)h(has)e(started.)19 b(It)14 b(pro)o(vides)h(a)g(mec)o(hanism)g(to)f(establish)h(comm)o(unication)75 1384 y(b)q(et)o(w)o(een)i(the)g(newly)h(created)f(pro)q(cesses)g(and)g (the)g(existing)h(MPI)f(application.)27 b(It)17 b(also)g(pro)o(vides)g (a)75 1440 y(mec)o(hanism)f(to)e(establish)j(comm)o(unication)f(b)q(et) o(w)o(een)f(t)o(w)o(o)f(existing)i(MPI)f(applications,)i(ev)o(en)e (when)75 1497 y(one)g(did)i(not)d(\\start")g(the)h(other.)75 1619 y Fh(3.2.1)49 b(Sta)o(rting)16 b(and)h(Managing)h(Pro)q(cesses)75 1704 y Fs(MPI)10 b(applications)i(ma)o(y)e(start)f(new)h(pro)q(cesses)h (\(including)i(non-MPI)d(pro)q(cesses\),)h(send)g(them)g(signals,)75 1761 y(and)17 b(\014nd)h(out)e(when)i(they)f(die)h(or)e(b)q(ecome)i (unreac)o(hable.)26 b(They)17 b(do)g(this)g(through)g(an)f(in)o (terface)i(to)75 1817 y(an)d(external)h(pro)q(cess)g(manager,)f(whic)o (h)h(can)g(range)f(from)g(a)g(parallel)i(op)q(erating)f(system)f (\(CMOST\))75 1874 y(to)g(la)o(y)o(ered)g(soft)o(w)o(are)e(\(POE\))i (to)g(an)g Fg(rsh)g Fs(command)g(\(p4\).)166 1930 y(There)g(are)g(t)o (w)o(o)f(w)o(a)o(ys)g(to)h(start)f(new)h(pro)q(cesses.)143 2024 y Fm(\017)23 b Fk(MPI)p 274 2024 14 2 v 15 w(SP)l(A)-5 b(WN)16 b Fs(starts)d(MPI)h(pro)q(cesses)h(and)f(establishes)i(comm)o (unication)f(with)g(them,)f(return-)189 2081 y(ing)h(an)h(in)o(tercomm) o(unicator.)143 2174 y Fm(\017)23 b Fk(MPI)p 274 2174 V 15 w(SP)l(A)-5 b(WN)p 439 2174 V 18 w(INDEPENDENT)17 b Fs(starts)e(pro)q(cesses)i(but)g(do)q(es)g(not)f(establish)i(comm)o (unication)189 2231 y(with)d(them.)20 b(It)15 b(returns)g(a)g(group.) 166 2325 y(Corresp)q(onding)e(to)f(these)h(routines)g(are)g(t)o(w)o(o)e (more)h(routines)i(that)e(start)f(sev)o(eral)i(di\013eren)o(t)g(bina-) 75 2381 y(ries)e(\(or)e(the)h(same)g(binary)h(with)f(di\013eren)o(t)h (argumen)o(ts\))e(at)g(the)h(same)g(time:)18 b Fk(MPI)p 1485 2381 V 16 w(SP)l(A)-5 b(WN)p 1651 2381 V 17 w(MUL)l(TIPLE)p Fs(,)75 2438 y(and)15 b Fk(MPI)p 248 2438 V 16 w(SP)l(A)-5 b(WN)p 414 2438 V 17 w(MUL)l(TIPLE)p 648 2438 V 16 w(INDEPENDENT)p Fs(.)166 2494 y(MPI)14 b(uses)h(the)g(existing)g(group)g(abstraction)f (to)g(represen)o(t)g(pro)q(cesses.)20 b(A)15 b(pro)q(cess)g(is)g(iden)o (ti\014ed)75 2550 y(b)o(y)g(a)g(\(group,)f(rank\))h(pair.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fi(3.2.)34 b(THE)15 b(MPI-2)g(D)o(YNAMIC)g(PR)o (OCESS)h(MODEL)775 b Fs(3)75 49 y Fh(3.2.2)49 b(The)16 b(Runtime)g(Environment)75 135 y Fs(The)h Fk(MPI)p 255 135 14 2 v 16 w(SP)l(A)-5 b(WN)18 b Fs(and)f Fk(MPI)p 611 135 V 16 w(SP)l(A)-5 b(WN)p 777 135 V 17 w(INDEPENDENT)17 b Fs(routines)g(pro)o(vide)h(an)e(in)o(terface)i(b)q(et)o(w)o(een)75 192 y(MPI)d(and)h(the)f Ff(runtime)h(envir)n(onment)e Fs(of)h(an)g(MPI)g(application.)166 248 y(The)f(di\016cult)o(y)h(is)f (that)f(there)h(is)g(an)g(enormous)g(range)f(of)g(run)o(time)h(en)o (vironmen)o(ts)g(and)g(applica-)75 304 y(tion)i(requiremen)o(ts,)g(and) g(MPI)g(m)o(ust)f(not)h(b)q(e)g(tailored)h(to)e(an)o(y)g(particular)i (one.)22 b(Examples)16 b(of)g(suc)o(h)75 361 y(en)o(vironmen)o(ts)f (are:)143 462 y Fm(\017)23 b Fj(MPP)15 b(managed)i(b)o(y)e(a)i(batc)o (h)g(queueing)f(system)p Fs(.)j(Batc)o(h)13 b(queueing)j(systems)e (generally)189 519 y(allo)q(cate)j(resources)g(b)q(efore)g(an)g (application)i(b)q(egins,)f(enforce)f(limits)h(on)f(resource)g(use)g (\(CPU)189 575 y(time,)f(memory)g(use,)g(etc\),)g(and)g(do)h(not)e (allo)o(w)i(a)f(c)o(hange)g(in)h(resource)f(allo)q(cation)i(after)d(a)h (job)189 632 y(b)q(egins.)21 b(Moreo)o(v)o(er,)13 b(man)o(y)i(MPPs)f (ha)o(v)o(e)h(sp)q(ecial)h(limitations)h(or)d(extensions,)i(suc)o(h)f (as)g(a)g(limit)189 688 y(the)c(n)o(um)o(b)q(er)h(of)f(pro)q(cesses)h (that)f(ma)o(y)g(run)h(on)g(one)f(pro)q(cessor,)h(or)f(the)h(abilit)o (y)h(to)e(gang-sc)o(hedule)189 745 y(pro)q(cesses)k(of)g(a)g(parallel)i (application.)143 837 y Fm(\017)23 b Fj(Net)o(w)o(ork)17 b(of)i(w)o(orkstations)g(with)h(PVM)p Fs(.)15 b(PVM)h(\(P)o(arallel)h (Virtual)h(Mac)o(hine\))e(allo)o(ws)h(a)189 893 y(user)f(to)g(create)g (a)g(\\virtual)h(mac)o(hine")g(out)f(of)g(a)g(net)o(w)o(ork)f(of)h(w)o (orkstations.)23 b(An)16 b(application)189 950 y(ma)o(y)i(extend)j(the) e(virtual)h(mac)o(hine)h(or)e(manage)g(pro)q(cesses)h(\(create,)f (kill,)j(redirect)f(output,)189 1006 y(etc.\))26 b(through)18 b(the)f(PVM)g(library)l(.)28 b(Requests)19 b(to)e(manage)g(the)g(mac)o (hine)i(or)e(pro)q(cesses)h(ma)o(y)189 1063 y(b)q(e)e(in)o(tercepted)g (and)f(handled)i(b)o(y)e(an)g(external)h(resource)f(manager.)143 1155 y Fm(\017)23 b Fj(Net)o(w)o(ork)10 b(of)i(w)o(orkstations)f (managed)i(b)o(y)e(a)i(load)g(balancing)h(system)p Fs(.)j(A)10 b(load)h(balanc-)189 1211 y(ing)h(system)f(ma)o(y)f(c)o(ho)q(ose)i(the) f(lo)q(cation)h(of)f(spa)o(wned)h(pro)q(cesses)f(based)h(on)f(dynamic)i (quan)o(tities,)189 1268 y(suc)o(h)19 b(as)f(load)h(a)o(v)o(erage.)29 b(It)18 b(ma)o(y)g(transparen)o(tly)g(migrate)h(pro)q(cesses)g(from)e (one)i(mac)o(hine)h(to)189 1324 y(another)14 b(when)i(a)f(resource)g(b) q(ecomes)h(una)o(v)m(ailable.)143 1417 y Fm(\017)23 b Fj(Large)j(SMP)g(with)g(Unix)p Fs(.)43 b(Applications)24 b(are)f(run)g(directly)h(b)o(y)e(the)h(user.)42 b(They)23 b(are)189 1473 y(sc)o(heduled)15 b(at)d(a)h(lo)o(w)g(lev)o(el)h(b)o(y)f (the)g(op)q(erating)h(system.)k(Pro)q(cesses)13 b(ma)o(y)g(ha)o(v)o(e)g (sp)q(ecial)h(sc)o(hedul-)189 1529 y(ing)g(c)o(haracteristics)h (\(gang-sc)o(heduling,)g(pro)q(cessor)g(a\016nit)o(y)l(,)f(deadline)i (sc)o(heduling,)g(pro)q(cessor)189 1586 y(lo)q(c)o(king,)h(etc.\))24 b(and)16 b(b)q(e)i(sub)s(ject)e(to)g(OS)h(resource)f(limits)i(\(n)o(um) o(b)q(er)f(of)f(pro)q(cesses,)h(amoun)o(t)e(of)189 1642 y(memory)l(,)f(etc.\).)166 1744 y(MPI)f(assumes,)g(implicitly)l(,)k (the)c(existence)h(of)f(an)g(en)o(vironmen)o(t)h(in)g(whic)o(h)g(an)f (application)i(runs.)75 1800 y(It)k(do)q(es)g(not)g(pro)o(vide)g(\\op)q (erating)g(system")f(services,)i(suc)o(h)g(as)e(a)h(general)g(abilit)o (y)h(to)e(query)i(what)75 1857 y(pro)q(cesses)i(are)f(running,)j(to)c (kill)k(arbitrary)c(pro)q(cesses,)j(to)e(\014nd)h(out)f(prop)q(erties)h (of)f(the)h(run)o(time)75 1913 y(en)o(vironmen)o(t)15 b(\(ho)o(w)g(man)o(y)g(pro)q(cessors,)f(ho)o(w)h(m)o(uc)o(h)g(memory)l (,)g(etc\).)166 1970 y(Complex)20 b(in)o(teraction)g(of)f(an)g(MPI)g (application)i(with)f(its)f(run)o(time)h(en)o(vironmen)o(t)g(should)g (b)q(e)75 2026 y(done)14 b(through)g(an)g(en)o(vironmen)o(t-sp)q (eci\014c)i(API.)e(An)g(example)h(of)e(suc)o(h)h(an)g(API)h(w)o(ould)f (b)q(e)g(the)g(PVM)75 2083 y(task)22 b(and)g(mac)o(hine)i(managemen)o (t)d(routines)i({)f Fg(pvm)p 1036 2083 15 2 v 17 w(addhosts)p Fs(,)h Fg(pvm)p 1353 2083 V 17 w(config)p Fs(,)g Fg(pvm)p 1622 2083 V 16 w(tasks)p Fs(,)h(etc.,)75 2139 y(p)q(ossibly)c(mo)q (di\014ed)g(to)e(return)h(an)g(MPI)f(\(group,rank\))f(when)i(p)q (ossible.)32 b(A)19 b(Condor)f(or)h(PBS)g(API)75 2195 y(w)o(ould)d(b)q(e)f(another)g(p)q(ossibilit)o(y)l(.)166 2252 y(A)o(t)h(some)h(lo)o(w)f(lev)o(el,)i(ob)o(viously)l(,)g(MPI)f(m)o (ust)f(b)q(e)h(able)h(to)e(in)o(teract)h(with)g(the)g(run)o(time)g (system,)75 2308 y(but)f(the)h(in)o(teraction)g(is)f(not)g(visible)j (at)c(the)i(application)h(lev)o(el)f(and)g(the)f(details)h(of)f(the)g (in)o(teraction)75 2365 y(are)f(not)g(sp)q(eci\014ed)i(b)o(y)e(the)g (MPI)h(standard.)166 2421 y(MPI)d(do)q(es)g(not)f(require)i(the)f (existence)h(of)e(an)h(underlying)h(\\virtual)f(mac)o(hine")h(mo)q (del,)g(in)f(whic)o(h)75 2478 y(there)h(is)g(a)g(consisten)o(t)g (global)g(view)h(of)e(an)h(MPI)g(application,)h(and)f(an)g(implicit)i (\\op)q(erating)e(system")75 2534 y(managing)k(resources)f(and)h(pro)q (cesses.)28 b(F)l(or)17 b(instance,)i(pro)q(cesses)f(spa)o(wned)g(b)o (y)g(one)f(task)g(ma)o(y)g(not)75 2591 y(b)q(e)i(visible)h(to)e (another;)h(additional)h(hosts)e(added)h(to)e(the)i(run)o(time)f(en)o (vironmen)o(t)h(b)o(y)f(one)h(pro)q(cess)75 2647 y(ma)o(y)13 b(not)h(b)q(e)h(visible)i(in)e(another)e(pro)q(cess;)i(tasks)e(spa)o (wned)i(b)o(y)f(di\013eren)o(t)g(pro)q(cesses)h(ma)o(y)e(not)h(b)q(e)h (au-)75 2704 y(tomatically)g(distributed)h(o)o(v)o(er)d(a)o(v)m (ailable)k(resources.)i(MPI)c(do)q(es)f(require,)h(ho)o(w)o(ev)o(er,)f (that)g(a)g(pro)q(cess)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fs(4)506 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)75 49 y Fs(b)q(e)j(a)o(w)o(are)e (of)h(its)g(o)o(wn)g(c)o(hanges)h(to)e(the)i(run)o(time)f(en)o (vironmen)o(t.)27 b(F)l(or)17 b(instance,)h(it)g(can)f(b)q(e)h (noti\014ed)75 106 y(when)e(its)f(c)o(hildren)i(die;)f(tasks)e(that)h (it)g(spa)o(wns)g(will)i(b)q(e)f(distributed)h(in)f(some)f(sane)g (manner,)g(etc.)189 200 y Ff(A)n(dvic)n(e)22 b(to)h(implementors.)86 b Fs(A)23 b(virtual)g(mac)o(hine)h(abstraction)e(is)h(appropriate)g(in) h(most)189 256 y(en)o(vironmen)o(ts)15 b(and)g(should)g(b)q(e)h(part)e (of)g(a)h(\\high-qualit)o(y")h(implemen)o(tation.)21 b(In)15 b(other)f(cases,)189 312 y(pro)o(viding)j(the)f(virtual)h(mac)o (hine)g(abstraction)f(ma)o(y)g(add)h(to)q(o)e(m)o(uc)o(h)i(baggage)e (\(e.g.,)g(daemon)189 369 y(pro)q(cesses\))g(or)g(inhibit)i(p)q (ortabilit)o(y)l(.)k(\()p Ff(End)16 b(of)g(advic)n(e)g(to)h (implementors.)p Fs(\))166 463 y(In)o(teraction)12 b(b)q(et)o(w)o(een)g (MPI)g(and)g(the)f(run)o(time)i(en)o(vironmen)o(t)f(is)g(limited)i(to)d (the)h(follo)o(wing)g(areas:)143 569 y Fm(\017)23 b Fs(A)17 b(pro)q(cess)h(ma)o(y)f(start)g(new)h(pro)q(cesses)g(with)g Fk(MPI)p 1100 569 14 2 v 16 w(SP)l(A)-5 b(WN)p Fs(,)18 b Fk(MPI)p 1379 569 V 15 w(SP)l(A)-5 b(WN)p 1544 569 V 18 w(INDEPENDENT)189 625 y Fs(and)21 b(their)g(m)o(ultiple-executabl) q(e)j(v)m(arian)o(ts.)37 b(It)21 b(ma)o(y)f(kill)j(pro)q(cesses)f(it)f (has)g(spa)o(wned)g(\(and)189 682 y(p)q(ossibly)16 b(others\))f(and)g (ma)o(y)g(b)q(e)h(noti\014ed)g(when)f(pro)q(cesses)h(die)g(or)f(b)q (ecome)h(unreac)o(hable.)143 776 y Fm(\017)23 b Fs(When)16 b(a)f(pro)q(cess)h(spa)o(wns)f(a)g(c)o(hild)i(pro)q(cess,)f(it)f (optionally)i(passes)e(extra)g(information)h(to)f(the)189 832 y(run)o(time)j(en)o(vironmen)o(t)h(to)e(indicate)j(where)e(or)g(ho) o(w)g(the)g(pro)q(cess)g(should)h(b)q(e)g(started.)28 b(This)189 889 y(extra)14 b(information)i(ma)o(y)e(b)q(e)i(opaque)f(to) g(MPI.)143 982 y Fm(\017)23 b Fs(An)c(attribute)g(v)m(ariable)h Fk(MPI)p 722 982 V 16 w(UNIVERSE)p 952 982 V 17 w(SIZE)p Fs(,)f(tells)h(a)e(program)g(ho)o(w)g(\\large")h(the)g(initial)189 1039 y(run)o(time)h(en)o(vironmen)o(t)g(is,)h(namely)g(ho)o(w)e(man)o (y)h(pro)q(cesses)g(can)g(usefully)i(b)q(e)e(started)f(allto-)189 1095 y(gether.)g(One)d(can)f(subtract)f(the)h(size)h(of)e Fk(MPI)p 1003 1095 V 16 w(COMM)p 1161 1095 V 16 w(W)o(ORLD)h Fs(from)f(this)i(v)m(alue)g(to)e(\014nd)h(out)189 1152 y(ho)o(w)f(man)o(y)h(pro)q(cesses)h(migh)o(t)f(usefully)i(b)q(e)f (started)e(in)i(addition)g(to)f(those)g(already)g(running.)75 1295 y Fp(3.3)59 b(Pro)r(cess)21 b(Manager)e(Interface)75 1444 y Fo(Curren)o(t)14 b(Status:)j Fn(Reading)c(and)h(\014rst)g(v)o (ote)g(on)g(en)o(tire)g(section.)75 1613 y Fh(3.3.1)49 b(Pro)q(cesses)15 b(in)i(MPI)75 1698 y Fs(A)22 b(pro)q(cess)h(is)f (represen)o(ted)h(in)g(MPI)f(b)o(y)g(a)g(\(group,)h(rank\))f(pair.)41 b(A)22 b(pro)q(cess)h(ma)o(y)e(or)h(ma)o(y)f(not)75 1755 y(b)q(e)f(an)g(\\MPI)f(pro)q(cess")g(in)i(that)e(it)g(ma)o(y)g(or)g(ma) o(y)g(not)g(call)i Fk(MPI)p 1259 1755 V 16 w(INIT)e Fs(and)g(th)o(us)h (b)q(e)g(assigned)g(an)75 1811 y Fk(MPI)p 160 1811 V 16 w(COMM)p 318 1811 V 16 w(W)o(ORLD)p Fs(.)15 b(MPI)g(can't)g(comm)o (unicate)g(directly)i(with)e(a)g(pro)q(cess)g(unless)i(it)e(has)g(a)g (com-)75 1868 y(m)o(unicator,)21 b(but)f(an)g(application)i(can)e(send) h(signals)g(to)f(an)o(y)g(pro)q(cess)g(if)h(can)f(iden)o(tify)h (through)f(a)75 1924 y(\(group,)13 b(rank)h(pair\).)19 b(Note)13 b(that)g(a)h(\(group,)f(rank\))g(iden)o(ti\014cation)j(is)e (not)f(unique)j(b)q(ecause)e(a)g(pro)q(cess)75 1981 y(ma)o(y)g(b)q (elong)j(to)d(sev)o(eral)i(groups.)75 2102 y Fh(3.3.2)49 b(Sta)o(rting)16 b(Pro)q(cesses)f(and)h(Establishing)i(Communication)f (With)f(Them)-117 2142 y Fm(>)f Fl(\(Sept\))75 2188 y Fs(The)21 b(follo)o(wing)g(routine)g(starts)f(a)g(n)o(um)o(b)q(er)h(of) f(MPI)h(pro)q(cesses)g(and)f(establishes)i(comm)o(unication)75 2245 y(with)16 b(them,)e(returning)i(an)f(in)o(tercomm)o(unicator.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fi(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)916 b Fs(5)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(SP)l(A)-5 b(WN\(command,)14 b(a)o(rgv,)g(maxp)o(ro)q(cs,)g(info,)g (ro)q(ot,)h(comm,)e(intercomm,)g(a)o(rra)o(y)p 1542 49 V 15 w(of)p 1594 49 V 16 w(errco)q(des\))117 126 y Fn(IN)155 b Fk(command)402 b Fn(name)19 b(of)g(program)f(to)h(b)q(e)h(spa)o(wned) h(\(string,)f(signi\014can)o(t)905 183 y(only)13 b(at)h(ro)q(ot\))117 254 y(IN)155 b Fk(a)o(rgv)506 b Fn(argumen)o(ts)19 b(to)g Fe(command)f Fn(\(arra)o(y)h(of)g(strings,)h(signi\014can)o(t)905 311 y(only)13 b(at)h(ro)q(ot\))117 382 y(IN)155 b Fk(maxp)o(ro)q(cs)408 b Fn(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g (\(in)o(teger,)h(sig-)905 438 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\)) 117 510 y(IN)155 b Fk(info)516 b Fn(a)21 b(set)i(of)e(k)o(ey-v)n(alue)g (pairs)g(telling)g(the)h(run)o(time)e(system)905 566 y(where)15 b(and)f(ho)o(w)f(to)h(start)h(the)f(pro)q(cesses)i (\(handle,)e(signi\014-)905 623 y(can)o(t)g(only)f(at)h(ro)q(ot\))117 694 y(IN)155 b Fk(ro)q(ot)508 b Fn(rank)13 b(of)f(pro)q(cess)j(in)d (whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 750 y(\(in)o(teger\))117 822 y(IN)155 b Fk(comm)470 b Fn(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 893 y(OUT)108 b Fk(intercomm)384 b Fn(in)o(tercomm)o (unicator)12 b(b)q(et)o(w)o(een)j(original)d(group)i(and)f(the)905 950 y(newly)h(spa)o(wned)g(group)g(\(handle\))117 1021 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1021 V 15 w(of)p 468 1021 V 16 w(errco)q(des)268 b Fn(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))75 1145 y Fg(int)23 b(MPI)p 245 1145 15 2 v 17 w(Spawn\(char*)g(command,)f(char**)h(argv,)h (int)f(maxprocs,)g(MPI)p 1480 1145 V 16 w(Info)h(info,)393 1202 y(int)g(root,)f(MPI)p 707 1202 V 17 w(Comm)g(comm,)393 1258 y(MPI)p 468 1258 V 17 w(Comm*)g(intercomm,int*)f(array)p 1106 1258 V 17 w(of)p 1171 1258 V 16 w(errcodes\))75 1345 y(MPI)p 150 1345 V 17 w(SPAWN\(COMMAND,)g(ARGV,)h(MAXPROCS,)g (INFO,)g(ROOT,)g(COMM,)g(INTERCOMM,)393 1401 y(ARRAY)p 516 1401 V 17 w(OF)p 581 1401 V 17 w(ERRCODES,)f(IERR\))170 1458 y(CHARACTER*\(*\))h(COMMAND,)f(ARGV\(*\))170 1514 y(INTEGER)h(INFO,)h(MAXPROCS,)e(ROOT,)h(COMM,)h(INTERCOMM,)e(ARRAY)p 1415 1514 V 17 w(OF)p 1480 1514 V 16 w(ERRCODES\(*\),)170 1570 y(IERR)166 1657 y Fk(MPI)p 251 1657 14 2 v 16 w(SP)l(A)-5 b(WN)23 b Fs(tries)g(to)g(start)e Fk(maxp)o(ro)q(cs)h Fs(iden)o(tical)j(copies)f(of)e(the)h(MPI)g(program)f(sp)q(eci\014ed)75 1713 y(b)o(y)f Fk(command)p Fs(,)h(establishing)h(comm)o(unication)g (with)e(them)h(and)g(returning)g(an)f(in)o(tercomm)o(unica-)75 1770 y(tor.)35 b Fk(MPI)p 267 1770 V 16 w(SP)l(A)-5 b(WN)21 b Fs(is)g(collectiv)o(e)i(o)o(v)o(er)c(pro)q(cesses)i(in)h Fk(comm)d Fs(and)h(the)h(newly)g(created)g(pro)q(cesses.)75 1826 y(The)13 b(spa)o(wned)h(pro)q(cesses,)f(kno)o(wn)g(as)g(c)o (hildren,)i(are)e Ff(r)n(e)n(quir)n(e)n(d)f Fs(to)g(call)j Fk(MPI)p 1395 1826 V 16 w(INIT)p Fs(,)d(whic)o(h)i(is)f(collectiv)o(e) 75 1883 y(with)f Fk(MPI)p 260 1883 V 16 w(SP)l(A)-5 b(WN)12 b Fs(in)g(the)g(paren)o(t)f(pro)q(cesses.)19 b(The)12 b(in)o(tercomm)o(unicator)f(returned)h(b)o(y)f Fk(MPI)p 1711 1883 V 16 w(SP)l(A)-5 b(WN)75 1939 y Fs(con)o(tains)18 b(the)f(paren)o(t)h(pro)q(cesses)g(in)g(one)g(group)f(and)h(the)g(c)o (hild)h(pro)q(cesses)f(in)g(the)g(other.)27 b(This)18 b(in-)75 1996 y(tercomm)o(unicator)g(is)g(kno)o(wn)g(in)i(the)e(c)o (hildren)i(as)e Fk(MPI)p 1084 1996 V 16 w(P)l(ARENT)p Fs(.)h(The)g(c)o(hildren)h(ha)o(v)o(e)e(their)g(o)o(wn)75 2052 y Fk(MPI)p 160 2052 V 16 w(COMM)p 318 2052 V 16 w(W)o(ORLD)p Fs(,)d(whic)o(h)h(is)g(separate)e(from)h(that)f(of)h(the)g (paren)o(t.)189 2135 y Ff(A)n(dvic)n(e)k(to)i(users.)66 b Fs(An)20 b(implemen)o(tation)h(ma)o(y)e(automatically)h(establish)h (comm)o(unication)189 2192 y(b)q(efore)d Fk(MPI)p 413 2192 V 16 w(INIT)f Fs(is)h(called.)30 b(Th)o(us,)18 b(completion)h(of)e Fk(MPI)p 1240 2192 V 16 w(SP)l(A)-5 b(WN)19 b Fs(in)g(the)f(paren)o(t)f (do)q(es)h(not)189 2248 y(necessarily)13 b(mean)f(that)g Fk(MPI)p 711 2248 V 15 w(INIT)g Fs(has)g(b)q(een)h(called)g(in)g(the)g (c)o(hildren)g(\(although)f(the)h(returned)189 2305 y(in)o(tercomm)o (unicator)i(can)g(b)q(e)h(used)g(immediately\).)21 b(\()p Ff(End)16 b(of)g(advic)n(e)g(to)h(users.)p Fs(\))75 2421 y Fk(The)23 b(command)e(a)o(rgument)45 b Fs(The)22 b Fk(command)f Fs(argumen)o(t)h(is)g(a)g(string)g(con)o(taining)h(the)f (name)h(of)e(a)75 2478 y(program)12 b(to)g(b)q(e)i(spa)o(wned.)19 b(The)14 b(string)f(is)g(n)o(ull-terminated)i(in)f(C.)f(In)g(F)l (ortran,)f(leading)j(and)e(trailing)75 2534 y(spaces)i(are)g(stripp)q (ed.)166 2591 y(MPI)22 b(do)q(es)h(not)g(sp)q(ecify)h(ho)o(w)e(to)g (\014nd)h(the)g(executable)g(or)g(ho)o(w)f(the)g(w)o(orking)h (directory)f(is)75 2647 y(determined.)f(These)16 b(rules)f(will)i(b)q (e)f(implemen)o(tation-dep)q(enden)o(t)i(and)d(will)i(b)q(e)e (appropriate)g(for)g(the)75 2704 y(run)o(time)h(en)o(vironmen)o(t.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fs(6)506 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)189 49 y Ff(A)n(dvic)n(e)f(to)h (implementors.)39 b Fs(The)14 b(implemen)o(tation)h(should)g(use)f(a)g (\\natural")f(rule)h(for)g(\014nding)189 106 y(executables)19 b(and)f(determining)i(w)o(orking)d(directories.)30 b(F)l(or)17 b(instance,)i(a)f(homogeneous)g(sys-)189 162 y(tem)12 b(with)h(a)g(global)g(\014le)h(system)e(migh)o(t)h(lo)q(ok)g(\014rst)f (in)i(the)f(w)o(orking)f(directory)h(of)g(the)g(spa)o(wning)189 219 y(pro)q(cess,)j(or)g(migh)o(t)g(searc)o(h)g(the)h(directories)g(in) h(a)e(P)l(A)l(TH)g(en)o(vironmen)o(t)h(v)m(ariable)h(as)e(do)g(Unix)189 275 y(shells.)32 b(An)20 b(implemen)o(tation)g(on)f(top)g(of)f(PVM)h(w) o(ould)g(use)h(PVM's)e(rules)i(for)e(\014nding)i(exe-)189 332 y(cutables)d(\(usually)h(in)41 b Fg(/pvm3/bin/$PVM)p 959 332 15 2 v 15 w(ARCH)16 b Fs(An)h(MPI)g(implemen)o(tation)h (running)f(under)189 388 y(POE)c(on)h(an)f(IBM)h(SP2)f(w)o(ould)h(use)g (POE's)f(metho)q(d)h(of)f(\014nding)h(executables.)21 b(A)13 b(high-qualit)o(y)189 444 y(implemen)o(tation)19 b(will)g(giv)o(e)f(the)g(user)g(some)f(con)o(trol)h(o)o(v)o(er)f(ho)o (w)g(executables)i(are)f(found)g(and)189 501 y(w)o(orking)c (directories)j(are)e(determined.)21 b(\()p Ff(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fs(\))166 607 y(If)f(the)f(program)g(named)h(in)g Fk(command)f Fs(do)q(es)g(not)h (call)g Fk(MPI)p 1216 607 14 2 v 16 w(INIT)p Fs(,)f(but)g(instead)i (forks)d(a)i(pro)q(cess)75 663 y(that)g(calls)j Fk(MPI)p 362 663 V 15 w(INIT)p Fs(,)e(the)g(results)g(are)g(unde\014ned.)25 b(Some)16 b(implemen)o(tations)h(ma)o(y)f(allo)o(w)g(this)h(case)75 720 y(to)e(w)o(ork.)189 825 y Ff(A)n(dvic)n(e)g(to)i(users.)41 b Fs(MPI)15 b(do)q(es)h(not)f(sa)o(y)g(what)f(happ)q(ens)j(if)f(the)f (program)f(y)o(ou)h(start)f(is)i(a)f(shell)189 882 y(script)20 b(and)g(that)f(shell)i(script)f(starts)e(a)i(program)e(that)h(calls)i Fk(MPI)p 1409 882 V 16 w(INIT)p Fs(.)d(Some)i(implemen-)189 938 y(tations)d(ma)o(y)h(allo)o(w)g(y)o(ou)g(to)g(do)g(this,)h(but)f (ma)o(y)f(ha)o(v)o(e)h(restrictions,)h(suc)o(h)g(as)e(requiring)j(that) 189 995 y(argumen)o(ts)14 b(supplied)k(to)d(the)h(shell)h(script)f(m)o (ust)f(b)q(e)h(supplied)i(to)d(the)h(program,)e(or)h(requiring)189 1051 y(that)f(certain)i(parts)e(of)h(the)g(en)o(vironmen)o(t)h(not)f(b) q(e)h(c)o(hanged.)k(\()p Ff(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fs(\))75 1171 y Fk(The)f(a)o(rgv)e(a)o(rgument)45 b(a)o(rgv)13 b Fs(is)i(an)g(arra)o(y)e(of)h(strings)h(con)o(taining)g(argumen)o(ts)e (that)h(are)g(passed)h(to)f(the)75 1228 y(program.)k(The)c(\014rst)f (elemen)o(t)h(of)f Fk(a)o(rgv)g Fs(is)h(the)f(\014rst)g(argumen)o(t)g (passed)h(to)f Fk(command)p Fs(.)18 b(The)c(argumen)o(t)75 1284 y(list)k(is)h(terminated)e(b)o(y)h Fe(NULL)g Fs(in)h(C)e(and)h(an) g(empt)o(y)f(string)g(in)i(F)l(ortran.)26 b(In)18 b(F)l(ortran,)f (leading)i(and)75 1340 y(trailing)c(spaces)g(are)f(alw)o(a)o(ys)f (stripp)q(ed,)j(so)e(that)f(a)h(string)h(consisting)g(of)f(all)h (spaces)f(is)h(considered)h(an)75 1397 y(empt)o(y)c(string.)19 b(The)13 b(constan)o(t)f Fe(MPI)p 706 1397 13 2 v 15 w(ARGV)p 832 1397 V 13 w(NULL)i Fs(ma)o(y)e(b)q(e)h(used)h(in)f(b)q (oth)g(C)g(and)g(F)l(ortran)e(to)h(indicate)75 1453 y(an)j(empt)o(y)g (argumen)o(t)f(list.)21 b(In)16 b(C,)f(this)g(constan)o(t)g(is)g(the)h (same)e(as)h(NULL.)75 1559 y Fj(Example)j(3.1)k Fs(Examples)16 b(of)f Fk(a)o(rgv)f Fs(in)i(C)f(and)g(F)l(ortran)75 1616 y Ff(T)m(o)h(run)g(the)g(pr)n(o)n(gr)n(am)h(\\o)n(c)n(e)n(an)-5 b(")16 b(with)h(ar)n(guments)f(\\-grid\014le")g(and)h(\\o)n(c)n(e)n (an1.gr)n(d")f(in)f(C:)242 1709 y Fg(char)23 b(command[])g(=)h ("ocean";)242 1765 y(char)f(*argv[])g(=)h({"-gridfile",)e ("ocean1.grd",)g(NULL};)242 1822 y(MPI_Spawn\(command,)f(argv,)j (...\);)75 1915 y Ff(or,)17 b(if)f(not)g(everything)g(is)g(known)f(at)i (c)n(ompile)f(time:)242 2009 y Fg(char)23 b(*command;)242 2065 y(char)g(**argv;)242 2122 y(command)g(=)h("ocean";)242 2178 y(argv=\(char)f(**\)malloc\(3)f(*)i(sizeof\(char)e(*\)\);)242 2235 y(argv[0])h(=)h("-gridfile";)242 2291 y(argv[1])f(=)h ("ocean1.grd";)242 2347 y(argv[2])f(=)h(NULL;)242 2404 y(MPI_Spawn\(command,)d(argv,)j(...\);)75 2497 y Ff(In)15 b(F)m(ortr)n(an:)242 2591 y Fg(CHARACTER*25)22 b(command,)h(argv\(3\)) 242 2647 y(command)g(=)h(')f(ocean)h(')242 2704 y(argv\(1\))f(=)h(')f (-gridfile)g(')1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fi(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)916 b Fs(7)242 49 y Fg(argv\(2\))23 b(=)h(')f(ocean1.grd')242 106 y(argv\(3\))g(=)h(')f(')242 162 y(call)g(MPI_SPAWN\(command,)f (argv,)h(...\))166 268 y Fs(Argumen)o(ts)15 b(are)h(supplied)i(to)e (the)g(program)f(if)h(this)h(is)f(allo)o(w)o(ed)g(b)o(y)g(the)g(op)q (erating)h(system.)k(In)75 325 y(C,)d(the)h(argumen)o(ts)e(app)q(ear)i (in)h(the)e(argv[])f(argumen)o(t)h(of)g(main\(\).)30 b(If)19 b(a)f(F)l(ortran)g(implemen)o(tation)75 381 y(supplies)23 b(routines)f(that)e(allo)o(w)h(a)g(program)f(to)h(obtain)g(its)g (argumen)o(ts,)g(the)h(argumen)o(ts)e(ma)o(y)g(b)q(e)75 438 y(a)o(v)m(ailable)g(through)f(that)f(mec)o(hanism.)31 b(In)19 b(C,)f(if)h(the)g(op)q(erating)g(system)f(do)q(es)h(not)f(supp) q(ort)h(com-)75 494 y(mand)c(line)i(argumen)o(ts)d(app)q(earing)i(in)f (argv[],)e(and)j(if)f(the)g(program)f(is)i(an)e(MPI)h(program,)f(the)h (MPI)75 551 y(implemen)o(tation)g(ma)o(y)e(add)g(the)h(argumen)o(ts)e (to)h(the)h(argv[])e(that)h(is)h(passed)f(to)g Fk(MPI)p 1556 551 14 2 v 16 w(INIT)p Fs(.)g(Note)g(that)75 607 y(the)h Fk(a)o(rgv)f Fs(argumen)o(t)g(to)h Fk(MPI)p 588 607 V 16 w(SP)l(A)-5 b(WN)15 b Fs(con)o(tains)f(only)h(argumen)o(ts,)e (where)h Fk(a)o(rgv[0])f Fs(to)g Fg(main)h Fs(con)o(tains)75 664 y(the)h(name)g(of)g(the)h(program.)75 784 y Fk(The)h(maxp)o(ro)q (cs)g(a)o(rgument)44 b Fs(MPI)17 b(tries)g(to)g(spa)o(wn)f Fk(maxp)o(ro)q(cs)g Fs(pro)q(cesses.)25 b(If)17 b(it)h(is)f(unable)h (to)e(spa)o(wn)75 840 y Fk(maxp)o(ro)q(cs)d Fs(pro)q(cesses,)h(it)g (returns)g(b)o(y)g(default)g(a)f(con)o(tin)o(uable)i(error)f(of)f (class)h Fe(MPI)p 1508 840 13 2 v 14 w(ERR)p 1601 840 V 15 w(SP)m(A)-5 b(WN)13 b Fs(and)h(a)75 897 y(half-empt)o(y)c(in)o (tercomm)o(unicator.)19 b(A)10 b(half-empt)o(y)g(in)o(tercomm)o (unicator)g(is)h(one)f(for)g(whic)o(h)h Fk(MPI)p 1749 897 14 2 v 16 w(COMM)p 1907 897 V 16 w(REMOTE)p 2110 897 V 17 w(GROUP)75 953 y Fs(returns)k Fk(MPI)p 317 953 V 16 w(GROUP)p 485 953 V 18 w(EMPTY)g Fs(and)g Fk(MPI)p 845 953 V 16 w(COMM)p 1003 953 V 16 w(REMOTE)p 1206 953 V 18 w(SIZE)g Fs(returns)g(0.)k(The)d(reasons)e(wh)o(y)75 1009 y Fk(MPI)p 160 1009 V 16 w(SP)l(A)-5 b(WN)18 b Fs(w)o(as)e(not)h (able)h(to)e(spa)o(wn)h(sp)q(eci\014c)i(pro)q(cesses)f(are)e(indicated) j(in)f Fk(a)o(rra)o(y)p 1596 1009 V 15 w(of)p 1648 1009 V 16 w(errco)q(des)f Fs(as)75 1066 y(describ)q(ed)g(b)q(elo)o(w.)166 1122 y(An)f(implemen)o(tation)h(ma)o(y)d(allo)o(w)i(the)g Fk(info)f Fs(argumen)o(t)g(to)g(c)o(hange)g(the)h(default)g(b)q(eha)o (vior,)g(suc)o(h)75 1179 y(that)22 b(if)g(the)h(implemen)o(tation)g(is) g(unable)h(to)d(spa)o(wn)h(all)i Fk(maxp)o(ro)q(cs)d Fs(pro)q(cesses,)j(it)f(ma)o(y)e(spa)o(wn)h(a)75 1235 y(smaller)c(n)o(um)o(b)q(er)f(of)f(pro)q(cesses.)26 b(In)17 b(principle,)j(the)d Fk(info)g Fs(argumen)o(t)f(ma)o(y)g(sp)q(ecify)i (an)f(arbitrary)f(set)75 1292 y Fm(f)p Fd(m)138 1299 y Fc(i)171 1292 y Fs(:)i(0)g Fd(<)i(m)338 1299 y Fc(i)370 1292 y Fm(\024)g Fk(maxp)o(ro)q(cs)o Fm(g)e Fs(of)h(allo)o(w)o(ed)g(v)m (alues)h(for)f(the)g(n)o(um)o(b)q(er)g(of)f(pro)q(cesses)i(spa)o(wned.) 31 b(The)75 1348 y(set)17 b Fm(f)p Fd(m)211 1355 y Fc(i)224 1348 y Fm(g)g Fs(do)q(es)g(not)f(necessarily)j(include)g(the)e(v)m (alue)h Fk(maxp)o(ro)q(cs)p Fs(.)24 b(If)17 b(an)g(implemen)o(tation)h (is)f(able)h(to)75 1405 y(spa)o(wn)e(one)h(of)e(these)i(allo)o(w)o(ed)g (n)o(um)o(b)q(ers)g(of)e(pro)q(cesses,)i(no)f(error)g(is)h(returned,)g (and)f(the)h(n)o(um)o(b)q(er)f(of)75 1461 y(spa)o(wned)f(pro)q(cesses,) g Fd(m)p Fs(,)f(is)i(giv)o(en)f(b)o(y)g(the)g(size)h(of)e(the)h(remote) f(group.)20 b(In)15 b(this)h(case,)e(if)i(the)f(n)o(um)o(b)q(er)75 1518 y(of)k(spa)o(wned)g(pro)q(cesses)g(is)h(smaller)g(than)f Fk(maxp)o(ro)q(c)p Fs(,)f(reasons)h(wh)o(y)f(the)h(other)g(pro)q (cesses)h(w)o(ere)e(not)75 1574 y(spa)o(wned)i(are)f(giv)o(en)h(in)h Fk(a)o(rra)o(y)p 622 1574 V 14 w(of)p 673 1574 V 16 w(errco)q(des)p Fs(.)34 b(If)20 b(it)g(is)g(not)f(p)q(ossible)j(to)d(spa)o(wn)g(one)h (of)f(the)h(allo)o(w)o(ed)75 1631 y(n)o(um)o(b)q(ers)11 b(of)g(pro)q(cesses,)h(no)f(pro)q(cesses)g(are)g(spa)o(wned)g(and)g Fk(MPI)p 1179 1631 V 16 w(SP)l(A)-5 b(WN)12 b Fs(returns)f(a)f(con)o (tin)o(uable)j(error)75 1687 y(of)g(class)h Fe(MPI)p 307 1687 13 2 v 14 w(ERR)p 400 1687 V 14 w(SP)m(A)-5 b(WN)13 b Fs(and)h(a)f(half-empt)o(y)h(in)o(tercomm)o(unicator,)f (indicating)i(sp)q(eci\014c)g(reasons)e(in)75 1743 y Fk(a)o(rra)o(y)p 173 1743 14 2 v 15 w(of)p 225 1743 V 16 w(errco)q(des)p Fs(.)166 1800 y(A)f(spa)o(wn)f(call)i(with)e(the)h (default)g(b)q(eha)o(vior)g(is)h(called)g Ff(har)n(d)p Fs(.)19 b(A)11 b(spa)o(wn)h(call)g(for)f(whic)o(h)i(few)o(er)e(than)75 1856 y Fk(maxp)o(ro)q(cs)17 b Fs(pro)q(cesses)g(ma)o(y)g(b)q(e)h (returned)f(is)h(called)h Ff(soft)p Fs(.)26 b(See)18 b(section)g(3.3.7)d(for)i(more)g(information)75 1913 y(on)e(the)g Ff(soft)g Fs(k)o(ey)g(for)g Fk(info)p Fs(.)166 1969 y(In)j(trying)f(to)f(ful\014ll)k(a)d(spa)o(wn)g(request,)g(an)g (implemen)o(tation)h(ma)o(y)f(ha)o(v)o(e)g(to)f(kill)j(o\013)e(pro)q (cesses)75 2026 y(that)h(ha)o(v)o(e)g(b)q(een)i(started.)29 b(F)l(or)18 b(instance,)i(in)f(trying)g(to)f(ful\014ll)j(a)d(hard)g (request)h(for)f(10)g(pro)q(cesses,)75 2082 y(an)f(implemen)o(tation)h (ma)o(y)e(encoun)o(ter)h(a)f(resource)h(error)f(when)h(spa)o(wning)g (the)g(10th.)24 b(In)17 b(this)g(case,)75 2139 y(it)i(m)o(ust)f(kill)j (o\013)d(the)h(\014rst)f(nine)i(pro)q(cesses.)31 b(If)19 b(the)g(same)f(request)h(w)o(ere)f(soft,)h(but)g(sp)q(eci\014ed)i(that) 75 2195 y(only)d(ev)o(en)f(v)m(alues)h(of)f Fd(m)f Fs(w)o(ere)h(allo)o (w)o(ed,)h(the)f(implemen)o(tation)h(migh)o(t)f(ha)o(v)o(e)g(to)f (terminate)h(a)g(single)75 2252 y(pro)q(cess.)39 b(MPI)21 b(guaran)o(tees)f(that)h(suc)o(h)h(spa)o(wned)f(pro)q(cesses)h(will)h (not)e(pro)q(ceed)h(past)f Fk(MPI)p 1761 2252 V 16 w(INIT)p Fs(.)75 2308 y(Programs)14 b(that)g(p)q(erform)h(w)o(ork)f(b)q(efore)i Fk(MPI)p 884 2308 V 16 w(INIT)e Fs(is)i(called)h(ma)o(y)d(ha)o(v)o(e)h (unin)o(tended)i(side-e\013ects.)75 2428 y Fk(The)23 b(info)f(a)o(rgument)45 b Fs(The)22 b Fk(info)g Fs(argumen)o(t)f(to)g (all)i(of)f(the)g(routines)g(in)h(this)g(c)o(hapter)f(c)o(hapter)f(is) 75 2484 y(an)f(opaque)h(handle)g(of)f(t)o(yp)q(e)g Fk(MPI)p 704 2484 V 16 w(Info)g Fs(in)h(C)f(and)h(in)o(teger)f(in)h(F)l(ortran.) 34 b(It)20 b(is)h(a)f(con)o(tainer)h(for)e(a)75 2541 y(n)o(um)o(b)q(er)14 b(of)g(user-sp)q(eci\014ed)i(\(k)o(ey)l(,v)m (alue\))f(pairs.)k(k)o(ey)14 b(and)g(v)m(alue)i(are)d(strings)h(\(n)o (ull-terminated)i(c)o(har*)75 2597 y(in)h(C,)f(c)o(haracter*\(*\))e(in) j(F)l(ortran\).)22 b(Routines)17 b(to)f(create)g(and)g(manipulate)i (the)e(info)h(argumen)o(t)e(are)75 2654 y(describ)q(ed)i(in)f(section)g (3.4.1.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fs(8)506 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)166 49 y Fs(F)l(or)10 b(the)h Fk(SP)l(A)-5 b(WN)12 b Fs(calls,)g Fk(info)f Fs(pro)o(vides)g(additional)h(\(and)f(p)q(ossibly)h(implemen)o (tation-dep)q(ende)q(n)o(t\))75 106 y(instructions)i(to)f(MPI)g(and)h (the)f(run)o(time)h(system)f(on)g(ho)o(w)g(to)f(start)g(pro)q(cesses.) 20 b(An)14 b(application)h(ma)o(y)75 162 y(pass)e Fe(MPI)p 249 162 13 2 v 15 w(INF)o(O)p 359 162 V 14 w(NULL)h Fs(in)g(C)f(or)g(F) l(ortran.)18 b(P)o(ortable)c(programs)e(not)h(requiring)i(detailed)g (con)o(trol)e(o)o(v)o(er)75 219 y(pro)q(cess)i(lo)q(cations)h(should)h (use)e Fe(MPI)p 724 219 V 14 w(INF)o(O)p 833 219 V 14 w(NULL)p Fs(.)166 275 y(MPI)h(do)q(es)g(not)f(sp)q(ecify)i(the)f(con)o (ten)o(t)f(of)h(the)f Fk(info)h Fs(argumen)o(t,)f(except)h(to)f(reserv) o(e)h(a)f(n)o(um)o(b)q(er)h(of)75 332 y(sp)q(ecial)h Fk(k)o(ey)d Fs(v)m(alues)i(\(see)f(section)g(3.3.7\).)j(The)d Fk(info)g Fs(argumen)o(t)f(is)h(quite)h(\015exible)g(and)f(could)h(ev)o (en)f(b)q(e)75 388 y(used,)h(for)f(example,)h(to)f(sp)q(ecify)i(the)e (executable)i(and)f(its)f(command-line)j(argumen)o(ts.)i(In)c(this)g (case)75 444 y(the)e Fk(command)e Fs(argumen)o(t)h(to)g Fk(MPI)p 690 444 14 2 v 16 w(SP)l(A)-5 b(WN)15 b Fs(could)g(b)q(e)f (empt)o(y)l(.)19 b(The)14 b(abilit)o(y)h(to)e(do)h(this)g(follo)o(ws)g (from)75 501 y(the)i(fact)e(that)h(MPI)h(do)q(es)f(not)g(sp)q(ecify)i (ho)o(w)e(an)g(executable)i(is)f(found,)g(and)f(the)h Fk(info)f Fs(argumen)o(t)g(can)75 557 y(tell)i(the)f(run)o(time)g (system)f(where)h(to)f(\\\014nd")h(the)g(executable)h(\\".)k(Of)16 b(course)g(a)f(program)g(that)g(do)q(es)75 614 y(this)h(will)g(not)f(b) q(e)h(p)q(ortable)g(across)e(MPI)h(implemen)o(tations.)75 734 y Fk(The)22 b(ro)q(ot)g(a)o(rgument)45 b Fs(All)23 b(argumen)o(ts)e(b)q(efore)h(the)g Fk(ro)q(ot)f Fs(argumen)o(t)g(are)h (examined)h(only)f(on)g(the)75 790 y(pro)q(cess)g(whose)h(rank)f(in)h Fk(comm)d Fs(is)j(equal)g(to)e Fk(ro)q(ot)p Fs(.)41 b(The)22 b(v)m(alue)i(of)d(these)i(argumen)o(ts)e(on)h(other)75 847 y(pro)q(cesses)16 b(is)f(ignored.)75 967 y Fk(The)c(a)o(rra)o(y)p 258 967 V 14 w(of)p 309 967 V 16 w(errco)q(des)g(a)o(rgument)45 b Fs(The)10 b Fk(a)o(rra)o(y)p 899 967 V 15 w(of)p 951 967 V 16 w(errco)q(des)h Fs(is)f(an)g(arra)o(y)f(of)h(length)h Fk(maxp)o(ro)q(cs)e Fs(in)i(whic)o(h)75 1023 y(MPI)18 b(rep)q(orts)g(the)g(status)g(of)g(eac)o(h)g(pro)q(cess)h(that)e(MPI)h (w)o(as)g(requested)h(to)e(start.)28 b(If)19 b(all)g Fk(maxp)o(ro)q(cs)75 1080 y Fs(pro)q(cesses)12 b(w)o(ere)g(spa)o(wned,) g Fk(a)o(rra)o(y)p 661 1080 V 14 w(of)p 712 1080 V 16 w(errco)q(des)h Fs(is)f(\014lled)i(in)e(with)g(the)g(v)m(alue)h Fk(MPI)p 1467 1080 V 16 w(SUCCESS)p Fs(.)g(If)f(only)g Fd(m)75 1136 y Fs(\(0)g Fm(\024)h Fd(m)g(<)g Fk(maxp)o(ro)q(cs)o Fs(\))e(pro)q(cesses)h(are)g(spa)o(wned,)g Fd(m)g Fs(of)f(the)h(en)o (tries)g(will)i(con)o(tain)e Fk(MPI)p 1585 1136 V 16 w(SUCCESS)h Fs(and)75 1193 y(the)i(rest)g(will)i(con)o(tain)e(an)g (error)f(co)q(de)i(indicating)h(an)e(implemen)o(tation-de\014ned)j (reason)d(MPI)g(could)75 1249 y(not)i(start)g(the)h(pro)q(cess.)28 b(MPI)17 b(do)q(es)h(not)g(sp)q(ecify)h(whic)o(h)f(en)o(tries)h (corresp)q(ond)f(to)f(failed)i(pro)q(cesses.)75 1306 y(An)d(implemen)o(tation)g(ma)o(y)l(,)e(for)h(instance,)h(\014ll)g(in)h (error)d(co)q(des)i(in)g(one-to-one)f(corresp)q(ondence)i(with)75 1362 y(a)j(detailed)j(sp)q(eci\014cation)f(in)g(the)f Fk(info)f Fs(argumen)o(t.)36 b(These)21 b(error)f(co)q(des)h(all)h(b)q (elong)g(to)e(the)h(error)75 1418 y(class)15 b Fk(MPI)p 266 1418 V 15 w(ERR)p 366 1418 V 18 w(SP)l(A)-5 b(WN)15 b Fs(if)f(there)h(w)o(as)e(no)h(error)g(in)h(the)f(argumen)o(t)g(list.) 20 b(An)15 b(application)h(ma)o(y)d(pass)75 1475 y Fe(MPI)p 152 1475 13 2 v 14 w(ERRCODES)p 381 1475 V 14 w(DONTCARE)h Fs(in)i(C)f(or)f(F)l(ortran.)75 1595 y Fe(MPI)p 152 1595 V 14 w(COMM)p 296 1595 V 15 w(P)m(ARENT)44 b Fs(MPI)20 b(pro)o(vides)g(a)g(prede\014ned)h(comm)o(unicator)f Fe(MPI)p 1453 1595 V 14 w(COMM)p 1597 1595 V 15 w(P)m(ARENT)p Fs(.)e(If)i(a)75 1651 y(pro)q(cess)15 b(w)o(as)f(started)g(with)g Fk(MPI)p 661 1651 14 2 v 16 w(SP)l(A)-5 b(WN)16 b Fs(or)e Fk(MPI)p 980 1651 V 16 w(SP)l(A)-5 b(WN)p 1146 1651 V 17 w(MUL)l(TIPLE)p Fs(,)13 b Fe(MPI)p 1480 1651 13 2 v 15 w(COMM)p 1625 1651 V 14 w(P)m(ARENT)h Fs(the)75 1708 y(other)d(side)h(of)f(the)g(in)o(tercomm)o(unicator)g(returned)h (b)o(y)f(those)g(calls.)20 b(Otherwise,)12 b Fe(MPI)p 1558 1708 V 14 w(COMM)p 1702 1708 V 15 w(P)m(ARENT)75 1764 y Fs(is)k(an)f(in)o(tercomm)o(unicator)g(with)h(a)e(remote)h (group)g(size)h(of)f(0.)-1286 b Fm(?)15 b Fl(\(Sept\))75 1886 y Fh(3.3.3)49 b(Sta)o(rting)16 b(Multiple)h(Executables)f(and)h (Establishing)g(Communication)g(with)g(Them)75 1972 y Fs(While)h Fk(MPI)p 294 1972 14 2 v 16 w(SP)l(A)-5 b(WN)17 b Fs(is)h(su\016cien)o(t)f(for)f(most)g(cases,)h(it)g(do)q(es)g(not)f (allo)o(w)h(the)g(spa)o(wning)g(of)f(m)o(ultiple)75 2028 y(binaries,)23 b(or)e(of)f(the)h(same)g(binary)g(with)h(m)o(ultiple)g (sets)f(of)f(argumen)o(ts.)37 b(The)21 b(follo)o(wing)g(routine)75 2085 y(spa)o(wns)16 b(m)o(ultiple)j(binaries)f(or)e(the)g(same)h (binary)g(with)g(m)o(ultiple)h(sets)e(of)g(argumen)o(ts,)g (establishing)75 2141 y(comm)o(unication)g(with)f(them.)20 b(All)d(spa)o(wned)e(programs)f(ha)o(v)o(e)h(the)g(same)g Fk(MPI)p 1469 2141 V 16 w(COMM)p 1627 2141 V 16 w(W)o(ORLD)p Fs(.)-1932 b Fm(>)15 b Fl(\(Sept\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fi(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)916 b Fs(9)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(SP)l(A)-5 b(WN)p 326 49 V 17 w(MUL)l(TIPLE\(count,)11 b(a)o(rra)o(y)p 802 49 V 15 w(of)p 854 49 V 16 w(commands,)f(a)o(rra)o (y)p 1190 49 V 14 w(of)p 1241 49 V 16 w(a)o(rgv,)g(a)o(rra)o(y)p 1456 49 V 14 w(of)p 1507 49 V 16 w(maxp)o(ro)q(cs,)g(a)o(rra)o(y)p 1820 49 V 15 w(of)p 1872 49 V 16 w(info,)75 106 y(ro)q(ot,)k(comm,)f (intercomm,)h(a)o(rra)o(y)p 652 106 V 14 w(of)p 703 106 V 16 w(errco)q(des\))117 183 y Fn(IN)155 b Fk(count)482 b Fn(Num)o(b)q(er)12 b(of)g(command)e(lines)j(\(in)o(teger,)g(size)g (of)f(eac)o(h)h(of)f(the)905 239 y(follo)o(wing)f(arra)o(ys\))117 313 y(IN)155 b Fk(a)o(rra)o(y)p 416 313 V 15 w(of)p 468 313 V 16 w(commands)222 b Fn(programs)14 b(to)g(b)q(e)i(executed)h (\(arra)o(y)d(of)g(strings,)h(signi\014can)o(t)905 369 y(only)e(at)h(ro)q(ot\))117 442 y(IN)155 b Fk(a)o(rra)o(y)p 416 442 V 15 w(of)p 468 442 V 16 w(a)o(rgv)343 b Fn(argumen)o(ts)20 b(for)g Fe(commands)g Fn(\(arra)o(y)g(of)g(arra)o(y)g(of)g(strings,)905 499 y(signi\014can)o(t)14 b(only)f(at)g(ro)q(ot\))117 572 y(IN)155 b Fk(a)o(rra)o(y)p 416 572 V 15 w(of)p 468 572 V 16 w(maxp)o(ro)q(cs)245 b Fn(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of) f(pro)q(cesses)k(for)d(eac)o(h)g(command)d(line)905 629 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can)o(t)h (only)f(at)h(ro)q(ot\))117 702 y(IN)155 b Fk(a)o(rra)o(y)p 416 702 V 15 w(of)p 468 702 V 16 w(info)353 b Fn(info)11 b(ob)r(jects)h(telling)f(the)h(run)o(time)e(system)h(where)i(and)e(ho)o (w)905 758 y(to)h(start)g(pro)q(cesses)i(\(arra)o(y)d(of)g(handles,)g (signi\014can)o(t)g(only)g(at)905 815 y(ro)q(ot)117 888 y(IN)155 b Fk(ro)q(ot)508 b Fn(rank)17 b(of)f(pro)q(cess)j(in)d(whic)o (h)h(previous)g(argumen)o(ts)g(are)g(ex-)905 944 y(amined)c(\(in)o (teger\))117 1018 y(IN)155 b Fk(comm)470 b Fn(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j(\(handle\))117 1091 y(OUT)108 b Fk(intercomm)384 b Fn(in)o(tercomm)o(unicator)12 b(b)q(et)o(w)o(een)j(original)d(group)i(and)g(newly)905 1147 y(spa)o(wned)h(group)e(\(handle\))117 1221 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1221 V 15 w(of)p 468 1221 V 16 w(errco)q(des)268 b Fn(one)14 b(error)h(co)q(de)g(p)q(er)g(pro)q(cess)g (\(arra)o(y)f(of)f(in)o(teger\))75 1345 y Fg(int)23 b(MPI)p 245 1345 15 2 v 17 w(Spawn)p 382 1345 V 17 w(multiple\(int)f(count,)h (char*)g(array)p 1139 1345 V 17 w(of)p 1204 1345 V 17 w(commands[],)f(char**)h(argv[],)393 1402 y(int*)g(array)p 635 1402 V 17 w(of)p 700 1402 V 17 w(maxprocs,)g(MPI)p 1028 1402 V 16 w(Info)h(array)p 1284 1402 V 16 w(of)p 1348 1402 V 17 w(info[],)f(int)g(root,)393 1458 y(MPI)p 468 1458 V 17 w(Comm)g(comm,MPI)p 796 1458 V 16 w(Comm*)h(intercomm,)e (int*)h(array)p 1457 1458 V 17 w(of)p 1522 1458 V 17 w(errcodes\))75 1544 y(MPI)p 150 1544 V 17 w(SPAWN)p 287 1544 V 16 w(MULTIPLE\(COUNT,)f(ARRAY)p 805 1544 V 17 w(OF)p 870 1544 V 16 w(COMMANDS,)h(ARRAY)p 1245 1544 V 17 w(OF)p 1310 1544 V 17 w(ARGV,)393 1601 y(ARRAY)p 516 1601 V 17 w(OF)p 581 1601 V 17 w(MAXPROCS,)f(ARRAY)p 956 1601 V 17 w(OF)p 1021 1601 V 17 w(INFO,)h(ROOT,)g(COMM,)g (INTERCOMM,)393 1657 y(ARRAY)p 516 1657 V 17 w(OF)p 581 1657 V 17 w(ERRCODES,)f(IERR\))170 1714 y(INTEGER)h(COUNT,)g(ARRAY)p 651 1714 V 17 w(OF)p 716 1714 V 17 w(INFO,)g(ARRAY)p 996 1714 V 16 w(OF)p 1060 1714 V 17 w(MAXPROCS,)g(ROOT,)g(COMM,)g (INTERCOMM,)170 1770 y(ARRAY)p 293 1770 V 17 w(OF)p 358 1770 V 17 w(ERRCODES\(*\),)f(IERR)170 1827 y(CHARACTER*\(*\))h(ARRAY)p 628 1827 V 16 w(OF)p 692 1827 V 17 w(COMMANDS\(*\))f(ARRAY)p 1115 1827 V 17 w(OF)p 1180 1827 V 17 w(ARGV\(COUNT,)g(*\))1875 1863 y Fm(?)16 b Fl(\(Sept\))166 1913 y Fk(MPI)p 251 1913 14 2 v 16 w(SP)l(A)-5 b(WN)p 417 1913 V 17 w(MUL)l(TIPLE)12 b Fs(is)h(iden)o(tical)h(to)e Fk(MPI)p 1021 1913 V 16 w(SP)l(A)-5 b(WN)13 b Fs(except)g(that)f(there)h(are)f(m)o(ultiple)i (exe-)75 1970 y(cutable)e(sp)q(eci\014cations.)20 b(The)12 b(\014rst)e(argumen)o(t,)h Fk(count)p Fs(,)i(giv)o(es)e(the)g(n)o(um)o (b)q(er)h(of)e(sp)q(eci\014cations.)21 b(Eac)o(h)10 b(of)75 2026 y(the)k(next)g(four)g(argumen)o(ts)f(are)h(simply)i(arra)o(ys)c (of)i(the)g(corresp)q(onding)h(argumen)o(ts)f(in)h Fk(MPI)p 1699 2026 V 15 w(SP)l(A)-5 b(WN)p Fs(.)75 2083 y(F)l(or)13 b(the)i(F)l(ortran)e(v)o(ersion)h(of)g Fk(a)o(rra)o(y)p 699 2083 V 14 w(of)p 750 2083 V 16 w(a)o(rgv)p Fs(,)f(the)h(elemen)o(t) h Fk(a)o(rra)o(y)p 1212 2083 V 15 w(of)p 1264 2083 V 15 w(a)o(rgv\(i,j\))d Fs(is)j(the)f Fk(j)p Fs(th)g(argumen)o(t)g(to)75 2139 y(command)j(n)o(um)o(b)q(er)h Fk(i)p Fs(.)25 b(This)18 b(ma)o(y)e(seem)i(bac)o(kw)o(ards)e(to)g(F)l(ortran)g(programmers)g (who)h(are)g(familiar)75 2195 y(with)g(F)l(ortran's)e(column-ma)s(jor)h (ordering.)24 b(Ho)o(w)o(ev)o(er,)15 b(it)i(is)g(necessary)g(to)e(do)i (it)g(this)f(w)o(a)o(y)g(to)g(allo)o(w)75 2252 y Fk(MPI)p 160 2252 V 16 w(SP)l(A)-5 b(WN)16 b Fs(to)f(sort)g(out)g(argumen)o(ts.) 21 b(Note)15 b(that)g(the)h(leading)h(dimension)h(of)d Fk(a)o(rra)o(y)p 1620 2252 V 14 w(of)p 1671 2252 V 16 w(a)o(rgv)g Ff(must)75 2308 y Fs(b)q(e)h(the)f(same)g(as)g Fk(count)p Fs(.)166 2365 y(All)25 b(of)f(the)g(spa)o(wned)g(pro)q (cesses)h(ha)o(v)o(e)e(the)h(same)g Fk(MPI)p 1203 2365 V 16 w(COMM)p 1361 2365 V 16 w(W)o(ORLD)p Fs(.)g(Their)g(ranks)g(in)75 2421 y Fk(MPI)p 160 2421 V 16 w(COMM)p 318 2421 V 16 w(W)o(ORLD)13 b Fs(corresp)q(ond)g(directly)h(to)e(the)g(order)h(in)g (whic)o(h)g(the)g(command)g(lines)h(are)e(sp)q(ec-)75 2478 y(i\014ed)j(in)g Fk(MPI)p 309 2478 V 16 w(SP)l(A)-5 b(WN)p 475 2478 V 17 w(MUL)l(TIPLE)p Fs(.)13 b(Assume)i(that)e Fd(m)1037 2485 y Fb(1)1071 2478 y Fs(pro)q(cesses)h(are)g(generated)g (b)o(y)g(the)g(\014rst)g(com-)75 2534 y(mand,)h Fd(m)254 2541 y Fb(2)289 2534 y Fs(b)o(y)g(the)h(second,)f(etc.)20 b(The)c(pro)q(cesses)g(corresp)q(onding)g(to)f(the)g(\014rst)g(command) g(line)i(ha)o(v)o(e)75 2591 y(ranks)11 b(0)p Fd(;)d Fs(1)p Fd(;)g(:)g(:)g(:)t(;)g(m)402 2598 y Fb(1)424 2591 y Fm(\000)s Fs(1.)19 b(The)12 b(pro)q(cesses)g(in)h(the)e(second)i(command)e(ha)o (v)o(e)h(ranks)f Fd(m)1532 2598 y Fb(1)1552 2591 y Fd(;)d(m)1613 2598 y Fb(1)1635 2591 y Fs(+)s(1)p Fd(;)g(:)g(:)g(:)d(m)1817 2598 y Fb(1)1840 2591 y Fs(+)75 2647 y Fd(m)115 2654 y Fb(2)141 2647 y Fm(\000)i Fs(1.)19 b(The)14 b(pro)q(cesses)f(in)i (the)e(third)h(ha)o(v)o(e)f(ranks)g Fd(m)1028 2654 y Fb(1)1054 2647 y Fs(+)7 b Fd(m)1136 2654 y Fb(2)1156 2647 y Fd(;)h(m)1217 2654 y Fb(1)1242 2647 y Fs(+)f Fd(m)1324 2654 y Fb(2)1350 2647 y Fs(+)g(1)p Fd(;)h(:)g(:)g(:)t(;)g(m)1556 2654 y Fb(1)1582 2647 y Fs(+)f Fd(m)1664 2654 y Fb(2)1690 2647 y Fs(+)g Fd(m)1772 2654 y Fb(3)1798 2647 y Fm(\000)g Fs(1,)75 2704 y(etc.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fs(10)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)189 49 y Ff(A)n(dvic)n(e)g(to)h(users.)39 b Fs(Calling)17 b Fk(MPI)p 785 49 14 2 v 15 w(SP)l(A)-5 b(WN)16 b Fs(m)o(ultiple)h (times)e(w)o(ould)g(create)f(man)o(y)h(sets)f(of)g(c)o(hil-)189 106 y(dren)23 b(with)f(di\013eren)o(t)h Fk(MPI)p 682 106 V 16 w(COMM)p 840 106 V 16 w(W)o(ORLD)p Fs(s)f(whereas)h Fk(MPI)p 1320 106 V 15 w(SP)l(A)-5 b(WN)p 1485 106 V 18 w(MUL)l(TIPLE)21 b Fs(creates)189 162 y(c)o(hildren)e(with)f(a)f (single)i Fk(MPI)p 724 162 V 16 w(COMM)p 882 162 V 16 w(W)o(ORLD)p Fs(,)e(so)h(the)f(t)o(w)o(o)f(metho)q(ds)i(are)f(not)h (completely)189 219 y(equiv)m(alen)o(t.)j(There)13 b(are)g(also)h(t)o (w)o(o)e(p)q(erformance-related)i(reasons)f(wh)o(y)l(,)g(if)h(y)o(ou)f (need)i(to)d(spa)o(wn)189 275 y(m)o(ultiple)20 b(executables,)h(y)o(ou) d(ma)o(y)g(w)o(an)o(t)g(to)g(use)h Fk(MPI)p 1161 275 V 16 w(SP)l(A)-5 b(WN)p 1327 275 V 17 w(MUL)l(TIPLE)18 b Fs(instead)h(of)g(call-)189 332 y(ing)e Fk(MPI)p 351 332 V 16 w(SP)l(A)-5 b(WN)17 b Fs(sev)o(eral)g(times.)24 b(First,)17 b(spa)o(wning)g(sev)o(eral)f(things)i(at)e(once)h(ma)o(y)e (b)q(e)j(faster)189 388 y(than)e(spa)o(wning)h(them)f(sequen)o(tially)l (.)25 b(Second,)17 b(in)h(some)e(implemen)o(tations,)h(comm)o (unication)189 444 y(b)q(et)o(w)o(een)f(pro)q(cesses)g(spa)o(wned)g(at) g(the)g(same)f(time)i(ma)o(y)e(b)q(e)h(faster)g(than)f(comm)o (unication)i(b)q(e-)189 501 y(t)o(w)o(een)d(pro)q(cesses)i(spa)o(wned)f (separately)l(.)21 b(\()p Ff(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fs(\))166 607 y(The)e Fk(a)o(rra)o(y)p 356 607 V 14 w(of)p 407 607 V 16 w(errco)q(des)g Fs(argumen)o(t)f(is)h(1-dimensional)i (arra)o(y)c(of)h(size)1375 575 y Fa(P)1419 588 y Fc(count)1419 619 y(i)p Fb(=1)1515 607 y Fd(n)1542 614 y Fc(i)1557 607 y Fs(,)g(where)h Fd(n)1740 614 y Fc(i)1768 607 y Fs(is)g(the)75 664 y Fd(i)p Fs(th)19 b(elemen)o(t)i(of)e Fk(a)o(rra)o(y)p 480 664 V 14 w(of)p 531 664 V 16 w(maxp)o(ro)q(cs)p Fs(.)33 b(Command)19 b(line)j(n)o(um)o(b)q(er)e Fd(i)f Fs(corresp)q(onds)h(to)f Fd(n)1630 671 y Fc(i)1664 664 y Fs(con)o(tiguous)75 720 y(slots)c(in)h(this)g(arra)o(y)l(,)d(and)j (error)e(co)q(des)i(are)f(treated)g(as)f(for)h Fk(MPI)p 1214 720 V 16 w(SP)l(A)-5 b(WN)p Fs(.)75 826 y Fj(Example)18 b(3.2)k Fs(Examples)16 b(of)f Fk(a)o(rra)o(y)p 737 826 V 14 w(of)p 788 826 V 16 w(a)o(rgv)f Fs(in)i(C)f(and)h(F)l(ortran)75 883 y Ff(T)m(o)h(run)h(the)g(pr)n(o)n(gr)n(am)g(\\o)n(c)n(e)n(an)-5 b(")18 b(with)g(ar)n(guments)f(\\-grid\014le")i(and)e(\\o)n(c)n(e)n (an1.gr)n(d")h(and)g(the)g(pr)n(o)n(gr)n(am)75 939 y(\\atmos")f(with)g (ar)n(gument)f(\\atmos.gr)n(d")i(in)d(C:)242 1033 y Fg(char)23 b(*array_of_command[2])e(=)j({"ocean",)f("atmos"};)242 1090 y(char)g(**array_of_argv[3];)242 1146 y(char)g(*argv0[])g(=)h ({"-gridfile",)e("ocean1.grd",)g(\(char)h(*\)0};)242 1202 y(char)g(*argv1[])g(=)h({"atmos.grd",)e(\(char)h(*\)0};)242 1259 y(array_of_argv[0])f(=)h(argv0;)242 1315 y(array_of_argv[1])f(=)h (argv1;)242 1372 y(MPI_Spawn\(array_of_command)o(s,)e(array_of_argv,)h (...\);)75 1466 y Ff(Her)n(e's)15 b(how)i(you)g(do)g(it)f(in)g(F)m (ortr)n(an:)242 1559 y Fg(CHARACTER*25)22 b(command\(2\),)h (array_of_argv\(2,)e(3\))242 1616 y(command\(1\))i(=)g(')h(ocean)f(') 242 1672 y(array_of_argv\(1,)f(1\))h(=)h(')g(-gridfile)e(')242 1729 y(array_of_argv\(1,)g(2\))h(=)h(')g(ocean1.grd')242 1785 y(array_of_argv\(1,)e(3\))h(=)h(')g(')242 1898 y(command\(2\))f(=) g(')h(atmos)f(')242 1955 y(array_of_argv\(2,)f(1\))h(=)h(')g(atmos.grd) e(')242 2011 y(array_of_argv\(2,)g(2\))h(=)h(')g(')242 2124 y(call)f(MPI_SPAWN\(array_of_commands,)d(array_of_argv,)i(...\))75 2246 y Fh(3.3.4)49 b(Sta)o(rting)16 b(Indep)q(endent)g(Pro)q(cesses)75 2332 y Fs(The)j(previous)g(section)g(describ)q(ed)h(ho)o(w)e(an)g(MPI)g (application)i(could)g(spa)o(wn)e(new)g(MPI)h(pro)q(cesses)75 2388 y(and)c(establish)h(comm)o(unication)g(with)f(them.)20 b(This)15 b(section)h(describ)q(es)g(ho)o(w)e(to)h(spa)o(wn)f Ff(indep)n(endent)75 2444 y Fs(pro)q(cesses,)f(that)f(is,)h(pro)q (cesses)g(that)f(do)g(not)g(comm)o(unicate)h(with)g(their)g(paren)o(t.) 18 b(These)13 b(ma)o(y)f(b)q(e)h(MPI)75 2501 y(pro)q(cesses)j(\(i.e.,)e (pro)q(cesses)i(that)e(call)j Fk(MPI)p 839 2501 V 15 w(INIT)p Fs(\))e(or)f(not.)-1224 b Fm(>)15 b Fl(\(Sept\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fi(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fs(11)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(SP)l(A)-5 b(WN)p 326 49 V 17 w(INDEPENDENT\(command,)15 b(a)o(rgv,)g(maxp)o(ro)q(cs,)g(info,)h(\015ag,)g(group,)g(a)o(rra)o(y)p 1636 49 V 15 w(of)p 1688 49 V 16 w(errco)q(des\))117 183 y Fn(IN)155 b Fk(command)402 b Fn(name)13 b(of)g(program)f(to)i(b)q (e)h(started)g(\(string\))117 258 y(IN)155 b Fk(a)o(rgv)506 b Fn(argumen)o(ts)13 b(to)h Fe(command)f Fn(\(arra)o(y)h(of)f (strings\))117 333 y(IN)155 b Fk(maxp)o(ro)q(cs)408 b Fn(maxim)n(um)10 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)k(to)d(start)g (\(in)o(teger\))117 408 y(IN)155 b Fk(info)516 b Fn(info)13 b(ob)r(ject)h(telling)f(the)h(run)o(time)e(system)h(where)i(and)e(ho)o (w)905 465 y(to)e(start)g(the)h(pro)q(cesses)i(\(handle,)d (signi\014can)o(t)f(only)g(at)h(ro)q(ot\))117 540 y(IN)155 b Fk(\015ag)518 b Fn(sa)o(ys)16 b(whether)g(pro)q(cesses)i(are)d(MPI)h (pro)q(cesses)h(or)e(not)g(\(in-)905 596 y(teger,)g(signi\014can)o(t)e (only)g(at)h(ro)q(ot\))117 672 y(OUT)108 b Fk(group)479 b Fn(group)14 b(of)f(spa)o(wned)i(pro)q(cesses)h(\(handle\))117 747 y(OUT)108 b Fk(a)o(rra)o(y)p 416 747 V 15 w(of)p 468 747 V 16 w(errco)q(des)268 b Fn(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))75 871 y Fg(int)23 b(MPI)p 245 871 15 2 v 17 w(Spawn)p 382 871 V 17 w(independent\(char*)e (command,)i(char*)g(argv[],)g(int)h(maxprocs,)393 928 y(MPI)p 468 928 V 17 w(Info)f(info,)g(int)h(flag,)f(MPI)p 1058 928 V 17 w(Group*)g(group,)393 984 y(int*)g(array)p 635 984 V 17 w(of)p 700 984 V 17 w(errcodes\))75 1070 y(MPI)p 150 1070 V 17 w(SPAWN)p 287 1070 V 16 w(INDEPENDENT\(COMMAND,)e (ARGV,)j(MAXPROCS,)e(INFO,)h(FLAG,)h(INTERCOMM,)393 1127 y(ARRAY)p 516 1127 V 17 w(OF)p 581 1127 V 17 w(ERRCODES,)e(IERR\))170 1183 y(CHARACTER*\(*\))h(COMMAND,)f(ARGV\(*\))170 1240 y(INTEGER)h(INFO,)h(MAXPROCS,)e(FLAG,)h(GROUP,)g(ARRAY)p 1176 1240 V 17 w(OF)p 1241 1240 V 17 w(ERRCODES\(*\),)f(IERR)166 1326 y Fs(This)17 b(routine)h(spa)o(wns)e(a)h(set)g(of)f(pro)q(cesses)h (that)g(do)f(not)h(establish)h(comm)o(unication)g(with)f(the)75 1383 y(paren)o(t)g(pro)q(cesses.)27 b(It)18 b(therefore)f(returns)h(a)f (group)g(instead)h(of)f(an)h(in)o(tercomm)o(unicator.)27 b(It)17 b(is)h(not)75 1439 y(collectiv)o(e,)e(and)f(is)g(called)i(on)d (a)h(single)h(pro)q(cess.)k(The)15 b Fk(\015ag)f Fs(argumen)o(t)g(sp)q (eci\014es)j(whether)d(or)h(not)f(the)75 1496 y(c)o(hildren)19 b(are)d(MPI)h(pro)q(cesses,)g(that)f(is,)i(whether)f(or)f(not)h(they)f (call)i Fk(MPI)p 1394 1496 14 2 v 16 w(INIT)p Fs(.)e(In)h(other)g(resp) q(ects,)75 1552 y(this)f(routine)f(is)h(similar)g(to)f Fk(MPI)p 656 1552 V 16 w(SP)l(A)-5 b(WN)p Fs(.)75 1672 y Fk(MPI)15 b(p)o(ro)q(cesses)75 1758 y Fs(If)f Fk(\015ag)f Fs(has)h(the)f(v)m(alue)i Fe(MPI)p 550 1758 13 2 v 14 w(MPI)e Fs(then)h Fk(MPI)p 837 1758 14 2 v 16 w(SP)l(A)-5 b(WN)p 1003 1758 V 17 w(INDEPENDENT)14 b Fs(is)g(iden)o(tical)h(to)e Fk(MPI)p 1711 1758 V 16 w(SP)l(A)-5 b(WN)75 1814 y Fs(except)14 b(that)f(it)g(do)q(es)h(not)f(establish)i(comm)o(unication)f(with)g (the)f(c)o(hildren,)i(returning)f(an)g(MPI)f(group)75 1871 y(rather)f(than)g(a)h(comm)o(unicator,)f(and)h(it)f(is)i(not)e (collectiv.)21 b(Successful)14 b(completion)f(do)q(es)g(not)f(indicate) 75 1927 y(that)k(the)h(pro)q(cesses)h(ha)o(v)o(e)e(called)j Fk(MPI)p 776 1927 V 16 w(INIT)d Fs(successfully)l(,)k(only)d(that)f (the)i(pro)q(cesses)f(w)o(ere)g(started)75 1984 y(successfully)l(.)166 2040 y(The)11 b(c)o(hild)i(pro)q(cesses)f(are)f Ff(r)n(e)n(quir)n(e)n (d)f Fs(to)h(call)h Fk(MPI)p 1006 2040 V 16 w(INIT)p Fs(.)e(All)i(c)o(hildren)i(spa)o(wned)d(in)h(a)f(single)h(call)h(to)75 2097 y Fk(MPI)p 160 2097 V 16 w(SP)l(A)-5 b(WN)p 326 2097 V 17 w(INDEPENDENT)17 b Fs(ha)o(v)o(e)g(the)f(same)h Fk(MPI)p 1060 2097 V 16 w(COMM)p 1218 2097 V 16 w(W)o(ORLD)p Fs(,)f(whic)o(h)i(is)f(separate)g(from)75 2153 y(that)d(of)h(the)h (paren)o(t.)j(In)d(the)f(c)o(hildren,)i Fk(MPI)p 870 2153 V 16 w(COMM)p 1028 2153 V 16 w(P)l(ARENT)f Fs(has)f(an)h(empt)o(y) e(remote)h(group.)75 2273 y Fk(Non-MPI)g(p)o(ro)q(cesses)75 2359 y Fs(If)21 b Fk(\015ag)h Fs(has)f(the)g(v)m(alue)h Fe(MPI)p 588 2359 13 2 v 15 w(NONMPI)f Fs(then)g(the)g(c)o(hild)i(pro)q (cesses)f(are)f(assumed)g(to)g(b)q(e)h(non-MPI)75 2415 y(pro)q(cesses.)28 b(MPI)17 b(is)h(not)g(required)g(to)f(do)h(an)o(y)f (sp)q(ecial)j(setup)e(to)f(ensure)h(that)f(a)g(call)i(to)e Fk(MPI)p 1774 2415 14 2 v 16 w(INIT)75 2472 y Fs(in)h(the)f(c)o (hildren)h(will)h(b)q(e)e(able)h(to)e(establish)i(comm)o(unication)g (among)e(the)h(c)o(hildren.)27 b(The)17 b(e\013ect)f(of)75 2528 y(calling)h Fk(MPI)p 304 2528 V 16 w(INIT)d Fs(in)i(the)g(c)o (hildren)h(is)e(unde\014ned.)902 b Fm(?)16 b Fl(\(Sept\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fs(12)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)75 49 y Fh(3.3.5)49 b(Sta)o(rting)16 b(multiple)h(indep)q(endent)f(p)o(ro) q(cesses)75 230 y Fk(MPI)p 160 230 14 2 v 16 w(SP)l(A)-5 b(WN)p 326 230 V 17 w(MUL)l(TIPLE)p 560 230 V 16 w(INDEPENDENT\(count,) 12 b(a)o(rra)o(y)p 1135 230 V 14 w(of)p 1186 230 V 16 w(commands,)e(a)o(rra)o(y)p 1522 230 V 15 w(of)p 1574 230 V 16 w(a)o(rgv,)f(a)o(rra)o(y)p 1788 230 V 15 w(of)p 1840 230 V 16 w(maxp)o(ro)q(cs,)75 286 y(a)o(rra)o(y)p 173 286 V 15 w(of)p 225 286 V 16 w(info,)14 b(\015ag,)h(group,)g(a)o (rra)o(y)p 667 286 V 14 w(of)p 718 286 V 16 w(errco)q(des\))117 363 y Fn(IN)155 b Fk(count)482 b Fn(Num)o(b)q(er)13 b(of)g(commands)e (\(in)o(teger,)i(size)h(of)f(eac)o(h)h(of)e(the)i(fol-)905 420 y(lo)o(wing)f(arra)o(ys\))117 495 y(IN)155 b Fk(a)o(rra)o(y)p 416 495 V 15 w(of)p 468 495 V 16 w(commands)222 b Fn(programs)13 b(to)h(b)q(e)g(executed)i(\(arra)o(y)e(of)f(strings\))117 570 y(IN)155 b Fk(a)o(rra)o(y)p 416 570 V 15 w(of)p 468 570 V 16 w(a)o(rgv)343 b Fn(argumen)o(ts)13 b(for)h Fe(commands)f Fn(\(arra)o(y)g(of)h(arra)o(y)f(of)h(strings\))117 645 y(IN)155 b Fk(a)o(rra)o(y)p 416 645 V 15 w(of)p 468 645 V 16 w(maxp)o(ro)q(cs)245 b Fn(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of)f (pro)q(cesses)k(for)d(eac)o(h)g(command)d(line)905 702 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o(tegers\))117 777 y(IN)155 b Fk(a)o(rra)o(y)p 416 777 V 15 w(of)p 468 777 V 16 w(info)353 b Fn(info)11 b(ob)r(jects)h(telling)f(the)h(run)o (time)e(system)h(where)i(and)e(ho)o(w)905 833 y(to)j(start)g(pro)q (cesses)j(\(arra)o(y)d(of)f(handles\))117 908 y(IN)155 b Fk(\015ag)518 b Fn(sa)o(ys)16 b(whether)g(pro)q(cesses)i(are)d(MPI)h (pro)q(cesses)h(or)e(not)g(\(in-)905 965 y(teger,)g(signi\014can)o(t)e (only)g(at)h(ro)q(ot\))117 1040 y(OUT)108 b Fk(group)479 b Fn(group)14 b(of)f(spa)o(wned)i(pro)q(cesses)h(\(handle\))117 1115 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1115 V 15 w(of)p 468 1115 V 16 w(errco)q(des)268 b Fn(one)14 b(error)h(co)q(de)g(p)q(er) g(pro)q(cess)g(\(arra)o(y)f(of)f(in)o(teger\))75 1239 y Fg(int)23 b(MPI)p 245 1239 15 2 v 17 w(Spawn)p 382 1239 V 17 w(multiple)p 591 1239 V 16 w(independent\(int)f(count,)h (char*)g(array)p 1419 1239 V 16 w(of)p 1483 1239 V 17 w(commands[],)393 1296 y(char**)g(array)p 683 1296 V 17 w(of)p 748 1296 V 17 w(argv[],)g(int*)g(array)p 1195 1296 V 16 w(of)p 1259 1296 V 17 w(maxprocs,)393 1352 y(MPI)p 468 1352 V 17 w(Info*)g(array)p 748 1352 V 17 w(of)p 813 1352 V 17 w(info[],)f(int)i(flag,)f(MPI)p 1331 1352 V 17 w(Group*)g(group,)393 1409 y(int*)g(array)p 635 1409 V 17 w(of)p 700 1409 V 17 w(errcodes\))75 1495 y(MPI)p 150 1495 V 17 w(SPAWN)p 287 1495 V 16 w(MULTIPLE)p 495 1495 V 16 w(INDEPENDENT\(COUNT,)f(ARRAY)p 1085 1495 V 16 w(OF)p 1149 1495 V 17 w(COMMANDS,)h(ARRAY)p 1525 1495 V 16 w(OF)p 1589 1495 V 17 w(ARGV,)393 1552 y(ARRAY)p 516 1552 V 17 w(OF)p 581 1552 V 17 w(MAXPROCS,)f(ARRAY)p 956 1552 V 17 w(OF)p 1021 1552 V 17 w(INFO,)h(FLAG,)g(GROUP,)393 1608 y(ARRAY)p 516 1608 V 17 w(OF)p 581 1608 V 17 w(ERRCODES,)f(IERR\)) 170 1665 y(INTEGER)h(COUNT,)g(ARRAY)p 651 1665 V 17 w(OF)p 716 1665 V 17 w(INFO,)g(ARRAY)p 996 1665 V 16 w(OF)p 1060 1665 V 17 w(MAXPROCS,)g(FLAG,)g(GROUP,)170 1721 y(ARRAY)p 293 1721 V 17 w(OF)p 358 1721 V 17 w(ERRCODES\(*\),)f(IERR) 170 1778 y(CHARACTER*\(*\))h(ARRAY)p 628 1778 V 16 w(OF)p 692 1778 V 17 w(COMMANDS\(*\),)f(ARRAY)p 1139 1778 V 17 w(OF)p 1204 1778 V 17 w(ARGV\(COUNT,)g(*\))166 1864 y Fk(MPI)p 251 1864 14 2 v 16 w(SP)l(A)-5 b(WN)p 417 1864 V 17 w(MUL)l(TIPLE)p 651 1864 V 16 w(INDEPENDENT)19 b Fs(is)g(iden)o(tical)i(to)d Fk(MPI)p 1379 1864 V 15 w(SP)l(A)-5 b(WN)p 1544 1864 V 18 w(INDEPENDENT)75 1920 y Fs(except)18 b(that)g(there)g(are)f(m)o(ultiple)j(executable)f(sp)q (eci\014cations.)30 b(The)18 b(\014rst)g(argumen)o(t,)f Fk(count)p Fs(,)j(giv)o(es)75 1977 y(the)g(n)o(um)o(b)q(er)g(of)g(sp)q (eci\014cations.)35 b(Eac)o(h)20 b(of)g(the)g(next)g(four)f(argumen)o (ts)g(are)h(simply)h(arra)o(ys)d(of)i(the)75 2033 y(corresp)q(onding)c (argumen)o(ts)f(in)h Fk(MPI)p 727 2033 V 15 w(SP)l(A)-5 b(WN)p 892 2033 V 18 w(INDEPENDENT)p Fs(.)75 2155 y Fh(3.3.6)49 b(Nonblo)q(cking)18 b(requests)75 2241 y Fs(Spa)o(wning)e(new)g(pro)q (cesses)g(ma)o(y)f(b)q(e)h(an)g(exp)q(ensiv)o(e)h(op)q(eration.)k(In)16 b(order)f(to)g(allo)o(w)h(pro)q(cesses)g(to)f(do)75 2297 y(useful)i(w)o(ork)e(while)i(pro)q(cesses)g(are)e(b)q(eing)j(spa)o (wned,)d(MPI)h(pro)o(vides)h(non-blo)q(c)o(king)g(v)o(ersions)f(of)g (the)75 2354 y(eac)o(h)f(of)g(the)g(routines)h(de\014ned)h(ab)q(o)o(v)o (e.)-883 b Fm(>)15 b Fl(\(Sept\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fi(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fs(13)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN\(command,)16 b(a)o(rgv,)g(maxp)o(ro)q(cs,)h(info,)g (ro)q(ot,)g(comm,)f(intercomm,)g(a)o(rra)o(y)p 1573 49 V 14 w(of)p 1624 49 V 16 w(errco)q(des,)i(re-)75 106 y(quest\))117 183 y Fn(IN)155 b Fk(command)402 b Fn(name)19 b(of)g(program)f(to)h(b)q(e)h(spa)o(wned)h(\(string,)f(signi\014can)o (t)905 239 y(only)13 b(at)h(ro)q(ot\))117 315 y(IN)155 b Fk(a)o(rgv)506 b Fn(argumen)o(ts)19 b(to)g Fe(command)f Fn(\(arra)o(y)h(of)g(strings,)h(signi\014can)o(t)905 371 y(only)13 b(at)h(ro)q(ot\))117 446 y(IN)155 b Fk(maxp)o(ro)q(cs)408 b Fn(maxim)n(um)13 b(n)o(um)o(b)q(er)j(of)g(pro)q(cesses)j(to)e(start)g (\(in)o(teger,)h(sig-)905 503 y(ni\014can)o(t)c(only)f(at)h(ro)q(ot\)) 117 578 y(IN)155 b Fk(info)516 b Fn(a)21 b(set)i(of)e(k)o(ey-v)n(alue)g (pairs)g(telling)g(the)h(run)o(time)e(system)905 634 y(where)15 b(and)f(ho)o(w)f(to)h(start)h(the)f(pro)q(cesses)i (\(handle,)e(signi\014-)905 691 y(can)o(t)g(only)f(at)h(ro)q(ot\))117 766 y(IN)155 b Fk(ro)q(ot)508 b Fn(rank)13 b(of)f(pro)q(cess)j(in)d (whic)o(h)h(previous)g(argumen)o(ts)f(are)i(v)n(alid)905 822 y(\(in)o(teger\))117 897 y(IN)155 b Fk(comm)470 b Fn(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j (\(handle\))117 972 y(OUT)108 b Fk(intercomm)384 b Fn(in)o(tercomm)o (unicator)12 b(b)q(et)o(w)o(een)j(original)d(group)i(and)f(the)905 1029 y(newly)h(spa)o(wned)g(group)g(\(handle\))117 1104 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1104 V 15 w(of)p 468 1104 V 16 w(errco)q(des)268 b Fn(one)14 b(co)q(de)h(p)q(er)g(pro)q (cess)h(\(arra)o(y)d(of)h(in)o(teger\))117 1179 y(OUT)108 b Fk(request)452 b Fn(request)16 b(ob)r(ject)e(\(handle\))75 1304 y Fg(int)23 b(MPI)p 245 1304 15 2 v 17 w(Ispawn\(char*)f(command,) h(char**)g(argv,)g(int)h(maxprocs,)f(MPI)p 1504 1304 V 16 w(Info)h(info,)393 1360 y(int)g(root,)f(MPI)p 707 1360 V 17 w(Comm)g(comm,)393 1417 y(MPI)p 468 1417 V 17 w(Comm*)g(intercomm,int*)f(array)p 1106 1417 V 17 w(of)p 1171 1417 V 16 w(errcodes,)393 1473 y(MPI)p 468 1473 V 17 w(Request*)h(request\))75 1559 y(MPI)p 150 1559 V 17 w(ISPAWN\(COMMAND,)f(ARGV,)h(MAXPROCS,)f(INFO,)i(ROOT,)f (COMM,)g(INTERCOMM,)393 1616 y(ARRAY)p 516 1616 V 17 w(OF)p 581 1616 V 17 w(ERRCODES,)f(REQUEST,)h(IERR\))170 1672 y(CHARACTER*\(*\))g(COMMAND,)f(ARGV\(*\))170 1729 y(INTEGER)h(INFO,)h(MAXPROCS,)e(ROOT,)h(COMM,)h(INTERCOMM,)e(ARRAY)p 1415 1729 V 17 w(OF)p 1480 1729 V 16 w(ERRCODES\(*\),)170 1785 y(REQUEST,)h(IERR)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fs(14)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN)p 339 49 V 17 w(MUL)l(TIPLE\(count,)11 b(a)o(rra)o(y)p 815 49 V 14 w(of)p 866 49 V 16 w(commands,)f(a)o(rra)o(y)p 1202 49 V 15 w(of)p 1254 49 V 16 w(a)o(rgv,)f(a)o(rra)o(y)p 1468 49 V 15 w(of)p 1520 49 V 16 w(maxp)o(ro)q(cs,)h(a)o(rra)o(y)p 1833 49 V 14 w(of)p 1884 49 V 16 w(info,)75 106 y(ro)q(ot,)k(comm,)f (intercomm,)h(a)o(rra)o(y)p 652 106 V 14 w(of)p 703 106 V 16 w(errco)q(des,)i(request\))117 183 y Fn(IN)155 b Fk(count)482 b Fn(Num)o(b)q(er)12 b(of)g(command)e(lines)j(\(in)o (teger,)g(size)g(of)f(eac)o(h)h(of)f(the)905 239 y(follo)o(wing)f(arra) o(ys\))117 315 y(IN)155 b Fk(a)o(rra)o(y)p 416 315 V 15 w(of)p 468 315 V 16 w(commands)222 b Fn(programs)14 b(to)g(b)q(e)i(executed)h(\(arra)o(y)d(of)g(strings,)h(signi\014can)o (t)905 371 y(only)e(at)h(ro)q(ot\))117 446 y(IN)155 b Fk(a)o(rra)o(y)p 416 446 V 15 w(of)p 468 446 V 16 w(a)o(rgv)343 b Fn(argumen)o(ts)20 b(for)g Fe(commands)g Fn(\(arra)o(y)g(of)g(arra)o (y)g(of)g(strings,)905 503 y(signi\014can)o(t)14 b(only)f(at)g(ro)q (ot\))117 578 y(IN)155 b Fk(a)o(rra)o(y)p 416 578 V 15 w(of)p 468 578 V 16 w(maxp)o(ro)q(cs)245 b Fn(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o(h)g(command)d(line) 905 634 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o(teger,)f(signi\014can) o(t)h(only)f(at)h(ro)q(ot\))117 709 y(IN)155 b Fk(a)o(rra)o(y)p 416 709 V 15 w(of)p 468 709 V 16 w(info)353 b Fn(info)11 b(ob)r(jects)h(telling)f(the)h(run)o(time)e(system)h(where)i(and)e(ho)o (w)905 766 y(to)h(start)g(pro)q(cesses)i(\(arra)o(y)d(of)g(handles,)g (signi\014can)o(t)g(only)g(at)905 822 y(ro)q(ot)117 897 y(IN)155 b Fk(ro)q(ot)508 b Fn(rank)17 b(of)f(pro)q(cess)j(in)d(whic)o (h)h(previous)g(argumen)o(ts)g(are)g(ex-)905 954 y(amined)c(\(in)o (teger\))117 1029 y(IN)155 b Fk(comm)470 b Fn(comm)o(unicator)7 b(of)i(group)g(of)g(spa)o(wning)g(pro)q(cesses)j(\(handle\))117 1104 y(OUT)108 b Fk(intercomm)384 b Fn(in)o(tercomm)o(unicator)12 b(b)q(et)o(w)o(een)j(original)d(group)i(and)g(newly)905 1161 y(spa)o(wned)h(group)e(\(handle\))117 1236 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1236 V 15 w(of)p 468 1236 V 16 w(errco)q(des)268 b Fn(one)14 b(error)h(co)q(de)g(p)q(er)g(pro)q(cess)g (\(arra)o(y)f(of)f(in)o(teger\))117 1311 y(OUT)108 b Fk(request)452 b Fn(request)16 b(ob)r(ject)e(\(handle\))75 1435 y Fg(int)23 b(MPI)p 245 1435 15 2 v 17 w(Ispawn)p 406 1435 V 17 w(multiple\(int)f(count,)h(char*)g(array)p 1163 1435 V 17 w(of)p 1228 1435 V 16 w(commands[],)g(char**)g(argv[],) 393 1492 y(int*)g(array)p 635 1492 V 17 w(of)p 700 1492 V 17 w(maxprocs,)g(MPI)p 1028 1492 V 16 w(Info)h(array)p 1284 1492 V 16 w(of)p 1348 1492 V 17 w(info[],)f(int)g(root,)393 1548 y(MPI)p 468 1548 V 17 w(Comm)g(comm,MPI)p 796 1548 V 16 w(Comm*)h(intercomm,)e(int*)h(array)p 1457 1548 V 17 w(of)p 1522 1548 V 17 w(errcodes,)393 1605 y(MPI)p 468 1605 V 17 w(Request)g(*request\))75 1691 y(MPI)p 150 1691 V 17 w(ISPAWN)p 311 1691 V 16 w(MULTIPLE\(COUNT,)f(ARRAY)p 829 1691 V 16 w(OF)p 893 1691 V 17 w(COMMANDS,)h(ARRAY)p 1269 1691 V 16 w(OF)p 1333 1691 V 17 w(ARGV,)393 1748 y(ARRAY)p 516 1748 V 17 w(OF)p 581 1748 V 17 w(MAXPROCS,)f(ARRAY)p 956 1748 V 17 w(OF)p 1021 1748 V 17 w(INFO,)h(ROOT,)g(COMM,)g (INTERCOMM,)393 1804 y(ARRAY)p 516 1804 V 17 w(OF)p 581 1804 V 17 w(ERRCODES,)f(REQUEST,)h(IERR\))170 1860 y(INTEGER)g(COUNT,)g (ARRAY)p 651 1860 V 17 w(OF)p 716 1860 V 17 w(INFO,)g(ARRAY)p 996 1860 V 16 w(OF)p 1060 1860 V 17 w(MAXPROCS,)g(ROOT,)g(COMM,)g (INTERCOMM,)170 1917 y(ARRAY)p 293 1917 V 17 w(OF)p 358 1917 V 17 w(ERRCODES\(*\),)f(REQUEST,)h(IERR)170 1973 y(CHARACTER*\(*\))g(ARRAY)p 628 1973 V 16 w(OF)p 692 1973 V 17 w(COMMANDS\(*\))f(ARRAY)p 1115 1973 V 17 w(OF)p 1180 1973 V 17 w(ARGV\(COUNT,)g(*\))1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fi(3.3.)34 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)894 b Fs(15)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(ISP)l(A)-5 b(WN)p 339 49 V 17 w(INDEPENDENT\(command,)13 b(a)o(rgv,)h(maxp)o(ro)q(cs,)g(info,)h(\015ag,)g(group,)f(a)o(rra)o(y)p 1641 49 V 15 w(of)p 1693 49 V 16 w(errco)q(des,)75 106 y(request\))117 183 y Fn(IN)155 b Fk(command)402 b Fn(name)13 b(of)g(program)f(to)i(b)q(e)h(started)g(\(string\))117 258 y(IN)155 b Fk(a)o(rgv)506 b Fn(argumen)o(ts)13 b(to)h Fe(command)f Fn(\(arra)o(y)h(of)f(strings\))117 333 y(IN)155 b Fk(maxp)o(ro)q(cs)408 b Fn(maxim)n(um)10 b(n)o(um)o(b)q(er)j(of)g (pro)q(cesses)k(to)d(start)g(\(in)o(teger\))117 408 y(IN)155 b Fk(info)516 b Fn(info)13 b(ob)r(ject)h(telling)f(the)h(run)o(time)e (system)h(where)i(and)e(ho)o(w)905 465 y(to)e(start)g(the)h(pro)q (cesses)i(\(handle,)d(signi\014can)o(t)f(only)g(at)h(ro)q(ot\))117 540 y(IN)155 b Fk(\015ag)518 b Fn(sa)o(ys)16 b(whether)g(pro)q(cesses)i (are)d(MPI)h(pro)q(cesses)h(or)e(not)g(\(in-)905 596 y(teger,)g(signi\014can)o(t)e(only)g(at)h(ro)q(ot\))117 672 y(OUT)108 b Fk(group)479 b Fn(group)14 b(of)f(spa)o(wned)i(pro)q (cesses)h(\(handle\))117 747 y(OUT)108 b Fk(a)o(rra)o(y)p 416 747 V 15 w(of)p 468 747 V 16 w(errco)q(des)268 b Fn(one)14 b(co)q(de)h(p)q(er)g(pro)q(cess)h(\(arra)o(y)d(of)h(in)o (teger\))117 822 y(OUT)108 b Fk(request)452 b Fn(request)16 b(ob)r(ject)e(\(handle\))75 946 y Fg(int)23 b(MPI)p 245 946 15 2 v 17 w(Ispawn)p 406 946 V 17 w(independent\(char*)e(command,)i (char*)g(argv[],)g(int)g(maxprocs,)393 1003 y(MPI)p 468 1003 V 17 w(Info)g(info,)g(int)h(flag,)f(MPI)p 1058 1003 V 17 w(Group*)g(group,)393 1059 y(int*)g(array)p 635 1059 V 17 w(of)p 700 1059 V 17 w(errcodes,)g(MPI)p 1028 1059 V 16 w(Request*)g(request\))75 1146 y(MPI)p 150 1146 V 17 w(SPAWN)p 287 1146 V 16 w(INDEPENDENT\(COMMAND,)e(ARGV,)j (MAXPROCS,)e(INFO,)h(FLAG,)h(INTERCOMM,)393 1202 y(ARRAY)p 516 1202 V 17 w(OF)p 581 1202 V 17 w(ERRCODES,)e(REQUEST,)h(IERR\))170 1259 y(CHARACTER*\(*\))g(COMMAND,)f(ARGV\(*\))170 1315 y(INTEGER)h(INFO,)h(MAXPROCS,)e(FLAG,)h(GROUP,)g(ARRAY)p 1176 1315 V 17 w(OF)p 1241 1315 V 17 w(ERRCODES\(*\),)f(REQUEST,)170 1371 y(IERR)75 1552 y Fk(MPI)p 160 1552 14 2 v 16 w(SP)l(A)-5 b(WN)p 326 1552 V 17 w(MUL)l(TIPLE)p 560 1552 V 16 w (INDEPENDENT\(count,)12 b(a)o(rra)o(y)p 1135 1552 V 14 w(of)p 1186 1552 V 16 w(commands,)e(a)o(rra)o(y)p 1522 1552 V 15 w(of)p 1574 1552 V 16 w(a)o(rgv,)f(a)o(rra)o(y)p 1788 1552 V 15 w(of)p 1840 1552 V 16 w(maxp)o(ro)q(cs,)75 1609 y(a)o(rra)o(y)p 173 1609 V 15 w(of)p 225 1609 V 16 w(info,)14 b(\015ag,)h(group,)g(a)o(rra)o(y)p 667 1609 V 14 w(of)p 718 1609 V 16 w(errco)q(des,)h(request\))117 1686 y Fn(IN)155 b Fk(count)482 b Fn(Num)o(b)q(er)13 b(of)g(commands)e(\(in)o(teger,)i(size)h(of)f(eac)o(h)h(of)e(the)i (fol-)905 1742 y(lo)o(wing)f(arra)o(ys\))117 1818 y(IN)155 b Fk(a)o(rra)o(y)p 416 1818 V 15 w(of)p 468 1818 V 16 w(commands)222 b Fn(programs)13 b(to)h(b)q(e)g(executed)i(\(arra)o(y)e (of)f(strings\))117 1893 y(IN)155 b Fk(a)o(rra)o(y)p 416 1893 V 15 w(of)p 468 1893 V 16 w(a)o(rgv)343 b Fn(argumen)o(ts)13 b(for)h Fe(commands)f Fn(\(arra)o(y)g(of)h(arra)o(y)f(of)h(strings\)) 117 1968 y(IN)155 b Fk(a)o(rra)o(y)p 416 1968 V 15 w(of)p 468 1968 V 16 w(maxp)o(ro)q(cs)245 b Fn(maxim)n(um)8 b(n)o(um)o(b)q(er)k(of)f(pro)q(cesses)k(for)d(eac)o(h)g(command)d(line) 905 2024 y(to)14 b(start)g(\(arra)o(y)g(of)g(in)o(tegers\))117 2099 y(IN)155 b Fk(a)o(rra)o(y)p 416 2099 V 15 w(of)p 468 2099 V 16 w(info)353 b Fn(info)11 b(ob)r(jects)h(telling)f(the)h (run)o(time)e(system)h(where)i(and)e(ho)o(w)905 2156 y(to)j(start)g(pro)q(cesses)j(\(arra)o(y)d(of)f(handles\))117 2231 y(IN)155 b Fk(\015ag)518 b Fn(sa)o(ys)16 b(whether)g(pro)q(cesses) i(are)d(MPI)h(pro)q(cesses)h(or)e(not)g(\(in-)905 2288 y(teger,)g(signi\014can)o(t)e(only)g(at)h(ro)q(ot\))117 2363 y(OUT)108 b Fk(group)479 b Fn(group)14 b(of)f(spa)o(wned)i(pro)q (cesses)h(\(handle\))117 2438 y(OUT)108 b Fk(a)o(rra)o(y)p 416 2438 V 15 w(of)p 468 2438 V 16 w(errco)q(des)268 b Fn(one)14 b(error)h(co)q(de)g(p)q(er)g(pro)q(cess)g(\(arra)o(y)f(of)f (in)o(teger\))117 2513 y(OUT)108 b Fk(request)452 b Fn(request)16 b(ob)r(ject)e(\(handle\))75 2637 y Fg(int)23 b(MPI)p 245 2637 15 2 v 17 w(Ispawn)p 406 2637 V 17 w(multiple)p 615 2637 V 16 w(independent\(int)e(count,)j(char*)f(array)p 1443 2637 V 16 w(of)p 1507 2637 V 17 w(commands[],)393 2694 y(char**)g(array)p 683 2694 V 17 w(of)p 748 2694 V 17 w(argv[],)g(int*)g(array)p 1195 2694 V 16 w(of)p 1259 2694 V 17 w(maxprocs,)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Fs(16)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)393 49 y Fg(MPI)p 468 49 15 2 v 17 w(Info*)23 b(array)p 748 49 V 17 w(of)p 813 49 V 17 w(info[],)f(int)i(flag,)f(MPI)p 1331 49 V 17 w(Group*)g(group,)393 106 y(int*)g(array)p 635 106 V 17 w(of)p 700 106 V 17 w(errcodes,)g(MPI)p 1028 106 V 16 w(Request*)g(request\))75 192 y(MPI)p 150 192 V 17 w(ISPAWN)p 311 192 V 16 w(MULTIPLE)p 519 192 V 16 w(INDEPENDENT\(COUNT,)f(ARRAY)p 1109 192 V 16 w(OF)p 1173 192 V 17 w(COMMANDS,)h(ARRAY)p 1549 192 V 16 w(OF)p 1613 192 V 17 w(ARGV,)393 249 y(ARRAY)p 516 249 V 17 w(OF)p 581 249 V 17 w(MAXPROCS,)f(ARRAY)p 956 249 V 17 w(OF)p 1021 249 V 17 w(INFO,)h(FLAG,)g(GROUP,)393 305 y(ARRAY)p 516 305 V 17 w(OF)p 581 305 V 17 w(ERRCODES,)f(REQUEST,)h (IERR\))170 362 y(INTEGER)g(COUNT,)g(ARRAY)p 651 362 V 17 w(OF)p 716 362 V 17 w(INFO,)g(ARRAY)p 996 362 V 16 w(OF)p 1060 362 V 17 w(MAXPROCS,)g(FLAG,)g(GROUP,)170 418 y(ARRAY)p 293 418 V 17 w(OF)p 358 418 V 17 w(ERRCODES\(*\),)f (REQUEST,)h(IERR)170 474 y(CHARACTER*\(*\))g(ARRAY)p 628 474 V 16 w(OF)p 692 474 V 17 w(COMMANDS\(*\),)f(ARRAY)p 1139 474 V 17 w(OF)p 1204 474 V 17 w(ARGV\(COUNT,)g(*\))166 561 y Fs(F)l(or)11 b(all)i(of)e(the)h(routines)g(in)h(this)f(section,)h (the)f(argumen)o(ts)f(are)h(iden)o(tical)h(those)f(of)f(their)i(blo)q (c)o(king)75 617 y(coun)o(terparts)h(except)h(that)g(there)g(is)g(an)g (additional)h Fk(request)h Fs(argumen)o(t.)i(The)c(request)g(can)g(b)q (e)g(used)75 674 y(as)c(input)i(for)e Fk(MPI)p 396 674 14 2 v 16 w(W)l(AIT)p Fs(,)g Fk(MPI)p 631 674 V 16 w(TEST)p Fs(,)g(etc.)19 b(In)12 b(all)h(cases,)f(the)f(output)h Fk(group)f Fs(or)g Fk(intercomm)p Fs(,)g(and)h(the)75 730 y Fk(a)o(rra)o(y)p 173 730 V 15 w(of)p 225 730 V 16 w(errco)q(des)i Fs(are)g(up)q(dated)h(async)o(hronously)f(and)h(are) f(not)f(v)m(alid)j(un)o(til)f(the)g(request)f(completes.)166 787 y(The)g(status)f(ob)s(ject)h(returned)g(b)o(y)g Fk(MPI)p 858 787 V 16 w(W)l(AIT)g Fs(con)o(tains)g(an)g(error)f(co)q(de)h (indicating)i(whether)e(or)75 843 y(not)h(the)h(spa)o(wn)f(w)o(as)g (successful.)22 b(The)15 b(failure)i(or)e(success)h(of)f(individual)k (pro)q(cesses)d(is)g(indicated)h(in)75 900 y Fk(a)o(rra)o(y)p 173 900 V 15 w(of)p 225 900 V 16 w(errco)q(des)p Fs(.)k(The)16 b(errorco)q(de)g(can)f(b)q(e)i(retriev)o(ed)f(as)f(the)h Fk(MPI)p 1280 900 V 15 w(ERROR)i Fs(elemen)o(t)e(of)f(the)h(status)75 956 y(ob)s(ject)k(returned)h(b)o(y)f Fk(MPI)p 561 956 V 16 w(W)l(AIT)p Fs(,)g Fk(MPI)p 805 956 V 16 w(TEST)p Fs(,)g(etc.)36 b(The)21 b(v)m(alues)g(of)f(other)g(\014elds)i(in)f(the) g(status)75 1013 y(ob)s(ject)15 b(are)g(not)f(de\014ned.)166 1069 y Fk(MPI)p 251 1069 V 16 w(ISP)l(A)-5 b(WN)17 b Fs(and)f Fk(MPI)p 618 1069 V 16 w(ISP)l(A)-5 b(WN)p 797 1069 V 17 w(MUL)l(TIPLE)16 b Fs(are)g(collectiv)o(e)i(o)o(v)o(er)e(pro) q(cesses)h(in)g Fk(comm)e Fs(and)75 1125 y(the)g(newly)h(created)g(pro) q(cesses.)k(They)15 b(ob)q(ey)h(the)f(the)g(follo)o(wing)h(rules:)143 1231 y Fm(\017)23 b Fs(A)c(non-blo)q(c)o(king)i(collectiv)o(e)h (routine)e(m)o(ust)f(complete)h(in)g(ev)o(ery)f(pro)q(cess)h(of)f(the)h (asso)q(ciated)189 1287 y(comm)o(unicator)14 b(b)q(efore)i Fk(MPI)p 707 1287 V 16 w(FINALIZE)e Fs(is)i(called.)143 1381 y Fm(\017)23 b Fs(The)13 b(op)q(eration)g(is)h(lo)q(cal.)20 b(A)14 b(non-blo)q(c)o(king)h(collectiv)o(e)g(op)q(eration)e(will)i (return)e(ev)o(en)h(if)f(it)h(is)f(not)189 1437 y(called)j(immediately) h(on)e(all)i(pro)q(cesses.)143 1531 y Fm(\017)23 b Fs(Multiple)17 b(outstanding)f(non-blo)q(c)o(king)h(op)q(erations)e(of)h(the)f(same)g (t)o(yp)q(e)h(on)f(the)h(same)f(comm)o(u-)189 1587 y(nicator)f(are)h (allo)o(w)o(ed,)g(and)g(matc)o(h)f(up)h(in)h(the)f(order)g(they)f(w)o (ere)h(called,)h(since)g(there)f(is)g(no)g Fk(tag)189 1643 y Fs(argumen)o(t.)25 b(Op)q(erations)18 b(of)f(di\013eren)o(t)h(t) o(yp)q(es)f(and/or)g(on)g(di\013eren)o(t)h(comm)o(unicators)f(ma)o(y)f (b)q(e)189 1700 y(in)o(terlea)o(v)o(ed.)143 1793 y Fm(\017)23 b Fs(A)e(single)i(collectiv)o(e)g(op)q(eration)f(ma)o(y)f(b)q(e)h(blo)q (c)o(king)h(on)e(some)g(no)q(des)i(and)e(non)o(blo)q(c)o(king)i(on)189 1850 y(others.)143 1943 y Fm(\017)g Fs(Spa)o(wn)18 b(requests)f(cannot) h(b)q(e)h(cancelled.)30 b(An)18 b(attempt)f(to)g(cancel)i(a)f (collectiv)o(e)h(request)f(will)189 2000 y(return)d(an)g(error)f(of)h (error)g(class)g Fk(MPI)p 857 2000 V 16 w(ERR)p 958 2000 V 17 w(REQUEST)p Fs(.)-117 2054 y Fm(?)g Fl(\(Sept\))75 2170 y Fh(3.3.7)49 b(Reserved)15 b(k)o(ey)h(values)75 2256 y Fs(The)h(follo)o(wing)h(k)o(ey)e(v)m(alues)i(are)f(reserv)o(ed)g (for)f(all)i Fk(info)f Fs(argumen)o(ts)f(in)h(section)h(3.3.)24 b(An)17 b(implemen-)75 2312 y(tation)d(is)h(not)f(required)i(to)e(in)o (terpret)h(these)f(k)o(ey)h(v)m(alues,)g(but)g(if)g(it)g(do)q(es)f(in)o (terpret)h(the)g(k)o(ey)f(v)m(alue,)i(it)75 2369 y(m)o(ust)f(pro)o (vide)g(the)h(functionalit)o(y)g(describ)q(ed.)-1008 b Fm(>)15 b Fl(\(Sept\))189 2474 y Ff(A)n(dvic)n(e)d(to)h (implementors.)38 b Fs(Implemen)o(tation-sp)q(eci\014c)15 b(k)o(ey)c(names)h(should)h(b)q(e)f(c)o(hosen)g(so)g(that)189 2531 y(they)j(are)g(unlik)o(ely)j(to)d(con\015ict)i(with)e(k)o(eys)h(c) o(hosen)g(b)o(y)f(other)g(implemen)o(tors.)22 b(A)15 b(high-qualit)o(y)189 2587 y(implemen)o(tation)20 b(will)h(ensure)e (this)g(b)o(y)g(adding)h(an)f(implemen)o(tation-sp)q(eci\014c)j (pre\014x)e(to)e(k)o(ey)189 2643 y(names.)h(\()p Ff(End)d(of)g(advic)n (e)h(to)f(implementors.)p Fs(\))-117 2704 y Fm(?)f Fl(\(Sept\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fi(3.3.)29 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)899 b Fs(17)75 49 y Fj(host)23 b Fs(V)l(alue)16 b(is)g(a)f(hostname,)f(format)g(of)h(hostname)g(determined)h(b)o(y)f (implemen)o(tation)75 143 y Fj(arc)o(h)22 b Fs(V)l(alue)f(is)f(an)f (arc)o(hitecture)h(name.)32 b(V)l(alid)21 b(arc)o(hitecture)f(names)f (and)h(what)e(they)i(mean)f(are)189 200 y(determined)d(b)o(y)f(an)g (implemen)o(tation)75 293 y Fj(wdir)23 b Fs(V)l(alue)13 b(is)h(the)e(name)h(of)g(a)f(directory)h(on)g(remote)f(mac)o(hine)h(in) h(whic)o(h)g(the)e(spa)o(wned)h(pro)q(cess\(es\))189 350 y(execute\(s\).)19 b(The)d(format)e(of)h(the)g(directory)g(name)g (determined)i(b)o(y)e(implemen)o(tation.)75 444 y Fj(path)24 b Fs(V)l(alue)13 b(is)f(a)g(directory)g(or)g(set)g(of)g(directories)h (on)f(remote)f(mac)o(hine)i(where)f(the)h(implemen)o(tation)189 500 y(should)j(lo)q(ok)f(for)g(the)g(executable.)21 b(format)14 b(of)h(path)g(determined)i(b)o(y)e(implemen)o(tation.)75 594 y Fj(\014le)23 b Fs(V)l(alue)17 b(is)f(the)g(name)f(of)g(a)h (\014le)g(in)h(whic)o(h)f(additional)h(information)f(is)g(sp)q (eci\014ed.)23 b(The)16 b(format)e(of)189 650 y(the)h(\014lename)h(and) g(in)o(ternal)g(format)e(of)g(the)i(\014le)g(are)f(determined)h(b)o(y)f (the)h(implemen)o(tation.)75 744 y Fj(soft)23 b Fs(V)l(alue)16 b(sp)q(eci\014es)h(a)e(set)f(of)h(n)o(um)o(b)q(ers)g(whic)o(h)h(are)f (allo)o(w)o(ed)h(v)m(alues)g(for)e(the)h(n)o(um)o(b)q(er)h(of)f(pro)q (cesses)189 801 y(that)k Fk(MPI)p 377 801 14 2 v 16 w(SP)l(A)-5 b(WN)21 b Fs(\(et)e(al.\))34 b(ma)o(y)20 b(create.)34 b(The)20 b(format)f(of)g(v)m(alue)i(is)g(a)f(comma-separated)189 857 y(list)d(of)f(F)l(ortran-90)g(triplets.)25 b(Non-p)q(ositiv)o(e)17 b(v)m(alues)h(and)f(v)m(alues)h(greater)e(than)g Fk(maxp)o(ro)q(cs)g Fs(are)189 914 y(ignored.)189 989 y(By)f(F)l(ortran-90)f(triplets,)h(w) o(e)g(mean:)231 1082 y(1.)22 b Fg(a)15 b Fs(means)g Fd(a)231 1156 y Fs(2.)22 b Fg(a:b)14 b Fs(means)i Fd(a;)8 b(a)h Fs(+)h(1)p Fd(;)e(a)h Fs(+)i(2)p Fd(;)d(:)g(:)g(:)t(;)g(b)231 1229 y Fs(3.)22 b Fg(a:b:c)14 b Fs(means)h Fd(a;)8 b(a)i Fs(+)g Fd(c;)e(a)h Fs(+)i(2)p Fd(c;)d(:)g(:)g(:)t(;)g(b)189 1322 y Fs(Examples:)231 1416 y(1.)22 b Fg(a:b)14 b Fs(giv)o(es)i(a)f (range)f(b)q(et)o(w)o(een)i Fd(a)f Fs(and)h Fd(b)231 1489 y Fs(2.)22 b Fg(0:N)14 b Fs(giv)o(es)i(full)g(\\soft")e (functionalit)o(y)231 1562 y(3.)22 b Fg(1,2,4,8,16,32,64,128,25)o (6,512,10)o(24,2048)o(,4096)8 b Fs(allo)o(ws)k(p)q(o)o(w)o(er-of-t)o(w) o(o)e(n)o(um)o(b)q(er)289 1619 y(of)k(pro)q(cesses.)231 1692 y(4.)22 b Fg(2:10000:2)14 b Fs(allo)o(ws)h(ev)o(en)g(n)o(um)o(b)q (er)h(of)f(pro)q(cesses.)231 1765 y(5.)22 b Fg(2:10:2,7)14 b Fs(same)h(as)f(ab)q(o)o(v)o(e,)h(and)g(also)g(7)g(pro)q(cesses.)75 1887 y Fh(3.3.8)49 b(Spa)o(wn)17 b(Examples)75 1972 y Fk(Manager-w)o(o)o(rk)o(er)c(example,)i(using)h(SP)l(A)-5 b(WN.)75 2058 y Fg(/*)24 b(manager)e(*/)75 2115 y(#include)h()75 2171 y(main\(int,)g(argc,)g(char)g(*arg[]\))75 2228 y({)147 2284 y(MPI_Status)f(status;)147 2341 y(int)h(world_size,)f (universe_size,)g(flag;)147 2397 y(MPI_Comm)g(everyone;)262 b(/*)23 b(intercommunicator)f(*/)147 2566 y(MPI_Init\(&argc,)f (&argv\);)147 2623 y(MPI_Comm_size\(MPI_COMM_)o(WORLD,)f (&world_size\);)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Fs(18)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)147 49 y Fg(if)23 b(\(world_size)g(!=)g(1\))191 b(error\("Top)22 b(heavy)h(with)h(management"\);)147 162 y(MPI_Attr_get\(MPI_COMM_W)o (ORLD,)d(MPI_UNIVERSE_SIZE,)g(&universe_size,)h(&flag\);)147 219 y(if)h(\(!flag\))g({)75 275 y(printf\("This)f(MPI)i(does)f(not)g (support)g(UNIVERSE_SIZE.)f(How)i(many\\n")433 332 y("processes)e (should)h(I)h(spawn?)f("\))75 388 y(scanf\("\045d",)f (&universe_size\);)147 444 y(})147 501 y(if)h(\(universe_size)f(==)i (1\))f(error\("No)g(room)g(to)h(start)f(workers"\);)147 614 y(MPI_Spawn\("worker",)e(MPI_ARGV_NULL,)h(universe_size-1,)385 670 y(MPI_INFO_NULL,)g(0,)i(MPI_COMM_SELF,)e(&everyone,)385 727 y(MPI_ERRCODES_DONTCARE\);)147 783 y(/*)170 840 y(*)i(Parallel)f (code)g(here.)g(The)h(communicator)e("everyone")h(can)g(be)h(used)170 896 y(*)g(to)g(communicate)e(with)h(the)h(spawned)f(processes,)f(which) h(have)h(ranks)f(0,..)170 953 y(*)h(MPI_UNIVERSE_SIZE-1)d(in)j(the)f (remote)h(group)f(of)g(the)h(intercommunicator)170 1009 y(*)g("everyone".)170 1065 y(*/)147 1178 y(MPI_Finalize\(\);)75 1235 y(})75 1348 y(/*)g(worker)f(*/)75 1461 y(#include)g()75 1517 y(main\(int)g(argc,)g(char)g(*argv[]\))75 1574 y({)147 1630 y(int)g(size;)147 1686 y(MPI_Init\(&argc,)e(&argv\);)147 1743 y(MPI_Comm_remote_size\(MP)o(I_COMM_)o(PARENT,)f(&size\);)147 1799 y(if)j(\(size)g(!=)h(1\))g(error\("Something's)d(wrong)i(with)g (the)h(parent"\);)147 1912 y(/*)170 1969 y(*)g(Parallel)f(code)g(here.) 170 2025 y(*)h(The)g(manager)f(is)g(represented)f(as)i(the)g(process)e (with)i(rank)f(0)h(in)f(\(the)h(remote)170 2082 y(*)g(group)f(of\))h (MPI_COMM_PARENT.)45 b(If)24 b(the)f(workers)g(need)h(to)f(communicate) g(among)170 2138 y(*)h(themselves,)f(they)g(can)g(either)g(extract)g (the)h(local)f(group)g(of)170 2195 y(*)h(MPI_COMM_PARENT,)e(or)h(use)h (MPI_COMM_WORLD.)46 b(These)23 b(two)170 2251 y(*)h(should)f(be)h(the)f (same.)170 2307 y(*/)147 2420 y(MPI_Finalize\(\);)75 2477 y(})1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fi(3.3.)29 b(PR)o(OCESS)16 b(MANA)o(GER)f(INTERF)-5 b(A)o(CE)899 b Fs(19)75 49 y Fk(T)l(ask)16 b(fa)o(rm)d(example.)75 135 y Fs(This)f(example)f(sho)o(ws)g(ho)o(w)f(to)h(manage)f(a)h(set)g (of)f(non-MPI)h(pro)q(cesses.)19 b(Note)11 b(that)f(all)i(groups)f(con) o(tain)75 192 y(a)18 b(single)h(pro)q(cess)g(b)q(ecause)g(it)f(w)o (ould)h(b)q(e)f(a)g(mess)g(otherwise.)29 b(This)19 b(could)g(b)q(e)g (\\impro)o(v)o(ed")f(b)o(y)g(us-)75 248 y(ing)e Fk(MPI)p 236 248 14 2 v 16 w(SP)l(A)-5 b(WN)p 402 248 V 17 w(MUL)l(TIPLE)p 636 248 V 15 w(INDEPENDENT)16 b Fs(but)f(this)h(w)o(ould)g(also)f(mak)o (e)g(group)g(managemen)o(t)75 304 y(di\016cult.)75 389 y Fg(#include)23 b()75 445 y(#define)g(MAXPROC)g(1000)75 502 y(main\(int)g(argc,)g(char)g(*argv[]\))75 558 y({)147 615 y(MPI_Group)f(workers[MAXPROC];)147 671 y(MPI_Request)g (obituaries[MAXPROC];)147 728 y(int)h(nslots,)g(i,)h(deadone,)e(flag;) 147 784 y(char)h(**argv;)147 841 y(char)g(command_lines[MAXLEN];)147 953 y(ThingToDo)f(*thingstodo;)147 1010 y(int)h(nthingstodo,)f (nthingsdone;)147 1123 y(MPI_Init\(&argc,)f(&argv\);)147 1179 y(MPI_Attr_get\(MPI_COMM_W)o(ORLD,)g(MPI_UNIVERSE_SIZE,)g (&nslots,)i(&flag\);)147 1236 y(nslots)g(=)g(nslots)g(-)h(1;)g(//)f (subtract)g(off)h(myself,)e(assume)h(only)h(one)f(of)h(me.)147 1292 y(GetThingsToDo\(&thingsto)o(do,)d(nthingstodo\);)147 1405 y(/*)i(need)g(to)h(initialize)f(these)g(for)g(Waitany/all\(\))f (below)h(to)h(work)f(*/)147 1462 y(for)g(\(i)h(=)f(0;)h(i)g(<)f (MAXPROC;)g(i++\))g(obituaries[i])g(=)g(MPI_REQUEST_NULL;)147 1574 y(/*)g(start)g(up)h(initial)f(set)g(of)h(tasks)f(*/)147 1631 y(for)g(\(i)h(=)f(0;)h(i)g(<)f(nslots)g(&&)h(i)g(<)f(nthingstodo;) g(i++\))147 1687 y({)218 1744 y(//)h(we)f(want)h(to)f(run)h(the)f (command)g("cow")g(with)h(some)f(special)g(args)218 1800 y(SetArgs\(thingstodo[i],)e(&argv\);)218 1857 y (MPI_Spawn_independent_mpi\(")o(cow")g(argv,)i(1,)h(MPI_INFO_NULL,)385 1913 y(MPI_NONMPI,)f(0,)g(MPI_COMM_SELF,)f(&workers[i],)g(NULL\))218 1970 y(MPI_Monitor\(workers[i],)f(0,)i(MPI_PROCESS_DIED,)f (&obituaries[i]\);)147 2026 y(})147 2083 y(nthingsdone=i;)147 2195 y(/*)h(spawn)g(new)h(ones)f(as)h(old)f(ones)g(finish)h(*/)147 2252 y(while\(nthingsdone)d(<)j(nthingstodo\))147 2308 y({)218 2365 y(MPI_Waitany\(nslots,)d(obituaries,)i(&deadone,)f (NULL\);)218 2421 y(SetArgs\(thingstodo[nthingsd)o(one],)f(&argv\);)218 2478 y(MPI_Spawn_independent\("cow")o(,)g(argv,)i(1,)h(MPI_INFO_NULL,) 385 2534 y(MPI_NONMPI,)f(0,)g(MPI_COMM_SELF,)f(&workers[deadone],)g (NULL\);)218 2591 y(MPI_MONITOR\(&workers[deadon)o(e],)f(0,)j (MPI_PROCESS_DIED,)505 2647 y(&obituaries[nthingsdone)o(]\);)147 2704 y(})-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Fs(20)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)147 49 y Fg(MPI_Waitall\(nslots,)21 b(obituaries,)h(NULL\);)147 162 y(MPI_Finalize\(\);)75 219 y(})75 395 y Fk(PVM-st)o(yle)16 b(SPMD)f(example)75 481 y Fs(This)21 b(is)h(ho)o(w)e(man)o(y)g(PVM)h (programs)e(are)i(t)o(ypically)h(written.)36 b(There)21 b(is)h(no)e(reason)h(they)g(can't)75 537 y(b)q(e)e(done)f(with)g(MPI-1) g(\(except)g(for)f(p)q(erformance!\),)h(but)g(in)h(case)f(users)g(w)o (an)o(t)f(the)h(app)q(earance)g(of)75 594 y(minimal)f(c)o(hange,)e (here)g(is)h(is.)k(\(V)l(ery)15 b(similar)i(to)d(the)i(manager-w)o(ork) o(er)d(example)j(ab)q(o)o(v)o(e.\))75 700 y Fg(/*)24 b(myprog)f(*/)75 813 y(#include)g()75 869 y(main\(int)g(argc,)g (char)g(*argv[]\))75 926 y({)147 982 y(MPI_Comm)f(everyone;)238 b(/*)23 b(intercommunicator)f(*/)147 1039 y(MPI_Comm)g(new_world;)214 b(/*)23 b(corresponding)f(intracommunicator)g(*/)147 1095 y(int)h(size,)g(nslots,)g(flag;)147 1152 y(MPI_Init\(&argc,)e (&argv\);)147 1265 y(MPI_Comm_remote_size\(MP)o(I_COMM_)o(PARENT,)f (&size\);)147 1321 y(MPI_Attr_get\(MPI_COMM_W)o(ORLD,)h (MPI_UNIVERSE_SIZE,)g(&nslots,)i(&flag\);)147 1378 y(if)g(\(size)g(==)h (0\))47 b(/*)24 b(I'm)f(the)h(parent)f(*/)147 1434 y({)218 1490 y(MPI_Spawn\("myproc",)e(NULL,)j(-1,)f(nslots-1,)433 1547 y(NULL,)g(0,)h(MPI_COMM_SELF,)e(&everyone,)g(NULL\);)218 1603 y(MPI_Intercomm_merge\(everyon)o(e,)f(FALSE,)i(&new_world\);)147 1660 y(})147 1716 y(else)g({)218 1773 y(MPI_Intercomm_merge\(MPI_COM)o (M_PARENT)o(,)e(TRUE,)i(&new_world\);)147 1829 y(})170 1886 y(/*)h(SPMD)f(parallel)g(code)g(here,)h(using)f(new_world)f (instead)h(of)h(MPI_COMM_WORLD)e(*/)170 1999 y(MPI_Finalize\(\);)75 2055 y(})75 2255 y Fp(3.4)59 b(Other)20 b(routines)75 2403 y Fo(Curren)o(t)9 b(Status:)14 b Fn(Reading)9 b(and)g(\014rst)h(v) o(ote)f(on)g Fe(INF)o(O)f Fn(routines,)j Fe(SIGNAL)p Fn(,)d Fe(MONITOR)p Fn(,)g(and)h Fe(MPI)p 1666 2403 13 2 v 15 w(UNIVERSE)p 1879 2403 V 13 w(SIZE)p Fn(.)75 2460 y(Singleton)i Fe(MPI)p 332 2460 V 14 w(INIT)g Fn(has)g(b)q(een)h (discussed)h(extensiv)o(ely)f(in)f(committees)f(but)h(has)g(had)g(no)g (formal)e(\014rst)j(reading)75 2516 y(or)i(v)o(ote.)k Fe(MPI)p 309 2516 V 14 w(APPNUM)13 b Fn(has)h(not)g(had)g(a)f(\014rst)i (reading)f(but)g(has)g(b)q(een)h(discussed)g(in)f(sub)q(committee.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fi(3.4.)34 b(OTHER)16 b(R)o(OUTINES)1227 b Fs(21)75 49 y Fh(3.4.1)49 b(Info)17 b(Management)f(Routines)75 135 y Fs(Man)o(y)i(of)h(the)g(routines)h(in)g(this)f(c)o(hapter)g(tak)o (e)f(an)h(argumen)o(t)g Fk(info)p Fs(.)31 b Fk(info)19 b Fs(is)h(an)f(opaque)g(ob)s(ject)f(of)75 192 y(t)o(yp)q(e)g Fk(MPI)p 265 192 14 2 v 16 w(Info)g Fs(in)g(C)g(and)g(in)o(teger)h(in)f (F)l(ortran.)28 b(It)18 b(consists)g(of)f(\(k)o(ey)l(,v)m(alue\))i (pairs)f(\(b)q(oth)g(k)o(ey)g(and)75 248 y(v)m(alue)i(are)f(strings\),) g(with)g(a)f(unique)j(v)m(alue)f(for)e(ev)o(ery)h(de\014ned)h(k)o(ey)l (.)31 b(MPI)19 b(reserv)o(es)f(sev)o(eral)i(k)o(eys,)75 304 y(and)e(requires)i(that)d(if)i(an)f(implemen)o(tation)i(uses)e(a)g (reserv)o(ed)h(k)o(ey)l(,)f(it)h(m)o(ust)f(pro)o(vide)g(the)h(sp)q (eci\014ed)75 361 y(functionalit)o(y)l(.)26 b(An)17 b(implemen)o (tation)h(is)g(not)e(required)i(to)e(supp)q(ort)h(these)g(k)o(eys,)g (and)g(ma)o(y)f(supp)q(ort)75 417 y(an)o(y)j(others)g(not)g(reserv)o (ed)g(b)o(y)g(MPI.)g(Reserv)o(ed)h(k)o(eys)f(related)h(to)f Fk(SP)l(A)-5 b(WN)20 b Fs(calls)g(are)f(describ)q(ed)i(in)75 474 y(section)d(3.3.7.)24 b(Reserv)o(ed)18 b(k)o(eys)f(related)g(to)g (connecting)h(indep)q(enden)o(t)i(pro)q(cesses)d(are)g(describ)q(ed)i (in)75 530 y(section)d(3.5.7.)166 587 y(If)d(an)g(implemen)o(tation)i (do)q(es)e(not)g(recognize)h(a)f(k)o(ey)l(,)g(it)h(should)g(ignore)f (it,)h(though)f(it)g(ma)o(y)g(giv)o(e)g(a)75 643 y(w)o(arning)g(to)f(a) g(user.)19 b(If)13 b(an)g(implemen)o(tation)h(recognizes)f(a)g(k)o(ey)f (but)h(do)q(es)g(not)f(recognize)i(the)f(format,)75 700 y(the)i(result)h(is)g(unde\014ned.)1346 b Fm(>)16 b Fl(\(Sept\))166 756 y Fs(Keys)10 b(ha)o(v)o(e)g(an)g(implemen)o(tation-de\014ned)j (maxim)o(um)d(length)h(of)f Fe(MPI)p 1372 756 13 2 v 14 w(INF)o(O)p 1481 756 V 14 w(MAX)p 1587 756 V 15 w(KEYLEN)p Fs(,)g(whic)o(h)75 813 y(is)17 b(at)e(least)i(8)f(and)g(at)g(most)f (255.)23 b(V)l(alues)17 b(ha)o(v)o(e)f(an)g(implemen)o (tation-de\014ned)j(maxim)o(um)d(length)h(of)75 869 y Fe(MPI)p 152 869 V 14 w(INF)o(O)p 261 869 V 14 w(MAX)p 367 869 V 15 w(V)m(ALUELEN)p Fs(.)f(In)h(F)l(ortran,)f(leading)j(and)e (trailing)h(spaces)f(are)g(stripp)q(ed)h(from)e(b)q(oth.)75 925 y(Returned)g(v)m(alues)h(will)f(nev)o(er)g(b)q(e)g(larger)f(than)g (these)g(maxim)o(um)g(lengths.)189 1024 y Ff(R)n(ationale.)51 b Fs(The)18 b(set)f(of)f(kno)o(wn)h(k)o(eys)g(will)i(alw)o(a)o(ys)e(b)q (e)g(\014nite)i(and)e(is)h(kno)o(wn)e(to)h(the)g(imple-)189 1080 y(men)o(tation.)j(Also)c(there)f(is)h(no)g(reason)f(for)g(k)o(eys) g(to)g(b)q(e)h(complex.)22 b(The)15 b(small)i(maxim)o(um)e(size)189 1137 y(allo)o(ws)h(applications)i(to)d(declare)i(k)o(eys)f(of)g(size)h Fe(MPI)p 1108 1137 V 14 w(INF)o(O)p 1217 1137 V 14 w(MAX)p 1323 1137 V 15 w(KEYLEN)p Fs(.)e(The)i(limitation)g(on)189 1193 y(v)m(alue)j(sizes)g(is)g(so)f(that)g(an)g(implemen)o(tation)i(is) f(not)f(forced)g(to)g(deal)h(with)g(arbitrarily)g(long)189 1250 y(strings.)f(This)c(constan)o(t)e(migh)o(t)h(b)q(e)h(v)o(ery)f (large,)g(so)g(it)g(migh)o(t)g(not)g(b)q(e)h(wise)f(to)g(declare)h(a)f (string)189 1306 y(of)g(that)h(size.)189 1380 y(\()p Ff(End)g(of)i(r)n(ationale.)p Fs(\))1875 1433 y Fm(?)f Fl(\(Sept\))75 1573 y Fk(MPI)p 160 1573 14 2 v 16 w(INF)o(O)p 279 1573 V 16 w(CREA)l(TE\(info\))117 1650 y Fn(OUT)108 b Fk(info)516 b Fn(info)13 b(ob)r(ject)i(created)g(\(handle\))75 1775 y Fg(int)23 b(MPI)p 245 1775 15 2 v 17 w(Info)p 358 1775 V 17 w(create\(MPI)p 615 1775 V 16 w(Info)g(*info\))75 1861 y(MPI)p 150 1861 V 17 w(INFO)p 263 1861 V 16 w(CREATE\(INFO,)g (IERR\))170 1917 y(INTEGER)g(INFO,)h(IERR)166 2004 y Fs(Creates)14 b(a)h(new)h(info)f(ob)s(ject.)20 b(The)15 b(newly)h(created)f(ob)s(ject)g(con)o(tains)g(no)g(k)o(ey/v)m(alue)i (pairs.)75 2155 y Fk(MPI)p 160 2155 14 2 v 16 w(INF)o(O)p 279 2155 V 16 w(SET\(info,)e(k)o(ey)l(,)g(value\))117 2232 y Fn(INOUT)62 b Fk(info)516 b Fn(info)13 b(ob)r(ject)i(\(handle\)) 117 2306 y(IN)155 b Fk(k)o(ey)525 b Fn(k)o(ey)14 b(\(string\))117 2380 y(IN)155 b Fk(value)490 b Fn(v)n(alue)13 b(\(string\))75 2504 y Fg(int)23 b(MPI)p 245 2504 15 2 v 17 w(Info)p 358 2504 V 17 w(set\(MPI)p 543 2504 V 16 w(Info)g(info,)h(char*)f(key,) g(char*)g(value\))75 2591 y(MPI)p 150 2591 V 17 w(INFO)p 263 2591 V 16 w(SET\(INFO,)g(KEY,)g(VALUE,)g(IERR\))170 2647 y(INTEGER)g(INFO,)h(IERR)170 2704 y(CHARACTER*\(*\))f(KEY,)g (VALUE)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 22 24 22 23 bop 75 -100 a Fs(22)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)166 49 y Fs(Adds)i(the)g(\(k)o(ey)l(,v)m(alue\))h(pair)f(to)f(info.)26 b(Ov)o(errides)18 b(if)f(previously)h(set.)25 b Fk(k)o(ey)17 b Fs(and)g Fk(value)h Fs(are)e(n)o(ull-)75 106 y(terminated)d(strings)f (in)i(C.)e(In)h(F)l(ortran,)f(leading)i(and)f(trailing)h(spaces)e(in)i Fk(k)o(ey)f Fs(and)f Fk(value)h Fs(are)g(ignored.)75 162 y(If)22 b(either)h Fk(k)o(ey)f Fs(or)g Fk(value)g Fs(are)g(larger)g(than)g(the)g(allo)o(w)o(ed)h(maxim)o(ums,)g Fk(MPI)p 1451 162 14 2 v 16 w(INF)o(O)p 1570 162 V 16 w(SET)f Fs(returns)g(a)75 219 y(con)o(tin)o(uable)16 b(error)f(of)g(class)g Fe(MPI)p 664 219 13 2 v 15 w(ERR)p 758 219 V 14 w(LEN)h Fs(and)f(do)q(es)h(not)e(set)h(\(or)g(reset\))f (the)i(\(k)o(ey)l(,)e(v)m(alue\))i(pair.)75 370 y Fk(MPI)p 160 370 14 2 v 16 w(INF)o(O)p 279 370 V 16 w(GET\(info,)f(k)o(ey)l(,)g (valuelen,)h(value,)f(\015ag\))117 447 y Fn(IN)155 b Fk(info)516 b Fn(info)13 b(ob)r(ject)i(\(handle\))117 522 y(IN)155 b Fk(k)o(ey)525 b Fn(k)o(ey)14 b(\(string\))117 597 y(IN)155 b Fk(valuelen)436 b Fn(length)14 b(of)f(v)n(alue)h(arg)f (\(in)o(teger\))117 672 y(OUT)108 b Fk(value)490 b Fn(v)n(alue)13 b(\(string\))117 747 y(OUT)108 b Fk(\015ag)518 b Fe(true)15 b Fn(if)e(k)o(ey)h(de\014ned,)h Fe(false)e Fn(if)g(not)h(\(b)q(o)q (olean\))75 872 y Fg(int)23 b(MPI)p 245 872 15 2 v 17 w(Info)p 358 872 V 17 w(get\(MPI)p 543 872 V 16 w(Info)g(info,)h(char*) 47 b(key,)23 b(int)g(valuelen,)g(char*)g(value,)393 928 y(int*)g(flag\))75 1015 y(MPI)p 150 1015 V 17 w(INFO)p 263 1015 V 16 w(GET\(INFO,)g(KEY,)g(VALUELEN,)g(VALUE,)g(FLAG,)g (IERR\))170 1071 y(INTEGER)g(INFO,)h(VALUELEN,)e(IERR)170 1128 y(CHARACTER*\(*\))h(KEY,)g(VALUE)170 1184 y(LOGICAL)g(FLAG)166 1270 y Fs(Retriev)o(es)f(the)g(v)m(alue)g(asso)q(ciated)g(with)g(k)o (ey)f(in)h(a)f(previous)i(call)f(to)f Fk(MPI)p 1526 1270 14 2 v 16 w(INF)o(O)p 1645 1270 V 16 w(SET)p Fs(.)g(If)h(no)75 1327 y(suc)o(h)c(k)o(ey)l(,)f(sets)g Fk(\015ag)g Fs(to)f Fe(true)j Fs(\(nonzero)e(in)h(C\),)e(and)h(returns)g(the)g(v)m(alue)i (in)f Fk(value)p Fs(,)f(otherwise)h(it)f(sets)75 1383 y Fk(\015ag)h Fs(to)g Fe(false)g Fs(\(zero)g(in)h(C\))e(and)i(lea)o(v)o (es)f Fk(value)h Fs(unc)o(hanged.)29 b Fk(valuelen)19 b Fs(is)g(the)f(n)o(um)o(b)q(er)h(of)f(c)o(haracters)75 1440 y(a)o(v)m(ailable)f(in)f(v)m(alue.)22 b(If)15 b(it)h(is)g(less)g (than)f(the)g(actual)h(size)g(of)f(the)h(v)m(alue,)g(the)f(v)m(alue)i (is)f(truncated,)f(and)75 1496 y Fk(MPI)p 160 1496 V 16 w(INF)o(O)p 279 1496 V 16 w(GET)h Fs(returns)f(the)g(con)o(tin)o (uable)h(error)f Fe(MPI)p 1062 1496 13 2 v 14 w(ERR)p 1155 1496 V 15 w(V)m(ALUE)p Fs(.)-1427 b Fm(>)15 b Fl(\(Sept\))166 1553 y Fs(If)c Fk(k)o(ey)h Fs(is)g(larger)f(than)g Fe(MPI)p 627 1553 V 14 w(INF)o(O)p 736 1553 V 14 w(MAX)p 842 1553 V 15 w(KEYLEN)p Fs(,)f Fk(MPI)p 1121 1553 14 2 v 16 w(INF)o(O)p 1240 1553 V 17 w(GET)h Fs(returns)g(a)g(con)o(tin)o(uable)i(error)75 1609 y(of)i(class)g Fe(MPI)p 310 1609 13 2 v 15 w(ERR)p 404 1609 V 14 w(LEN)h Fs(and)f(sets)g Fk(\015ag)g Fs(to)g Fe(false)p Fs(.)75 1760 y Fk(MPI)p 160 1760 14 2 v 16 w(INF)o(O)p 279 1760 V 16 w(GET)p 383 1760 V 17 w(V)l(ALUELEN\(info,)g (k)o(ey)l(,)g(valuelen,)h(\015ag\))117 1837 y Fn(IN)155 b Fk(info)516 b Fn(info)13 b(ob)r(ject)i(\(handle\))117 1913 y(IN)155 b Fk(k)o(ey)525 b Fn(k)o(ey)14 b(\(string\))117 1988 y(OUT)108 b Fk(valuelen)436 b Fn(length)14 b(of)f(v)n(alue)h(arg)f (\(in)o(teger\))117 2063 y(OUT)108 b Fk(\015ag)518 b Fe(true)15 b Fn(if)e(k)o(ey)h(de\014ned,)h Fe(false)e Fn(if)g(not)h(\(b)q(o)q(olean\))75 2187 y Fg(int)23 b(MPI)p 245 2187 15 2 v 17 w(Info)p 358 2187 V 17 w(get)p 447 2187 V 17 w(valuelen\(MPI)p 752 2187 V 15 w(Info)g(info,char*)g (key,int*)g(valuelen,int*)f(flag\))75 2274 y(MPI)p 150 2274 V 17 w(INFO)p 263 2274 V 16 w(GET)p 351 2274 V 17 w(VALUELEN\(INFO,)g(KEY,)h(VALUELEN,)g(FLAG,)g(IERR\))170 2330 y(INTEGER)g(INFO,)h(VALUELEN,)e(IERR)170 2387 y(LOGICAL)h(FLAG)h (CHARACTER*\(*\))e(KEY)166 2473 y Fs(Retriev)o(es)14 b(the)g(length)g(of)f(the)h Fk(value)g Fs(asso)q(ciated)f(with)h Fk(k)o(ey)p Fs(.)20 b(If)13 b Fk(k)o(ey)h Fs(is)g(de\014ned,)h Fk(valuelen)f Fs(is)g(set)g(to)75 2530 y(the)h(length)g(of)f(its)h (asso)q(ciated)g(v)m(alue)h(and)f Fk(\015ag)f Fs(is)i(set)e(to)g Fe(true)p Fs(.)21 b(If)15 b Fk(k)o(ey)f Fs(is)i(not)e(de\014ned,)i Fk(valuelen)f Fs(is)g(not)75 2586 y(touc)o(hed)g(and)f Fk(\015ag)g Fs(is)h(set)f(to)g Fe(false)p Fs(.)20 b(The)14 b(length)h(returned)g(in)g(C)f(do)q(es)h(not)f(include)j(the)d (end-of-string)75 2642 y(c)o(haracter.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 23 25 23 24 bop 75 -100 a Fi(3.4.)34 b(OTHER)16 b(R)o(OUTINES)1227 b Fs(23)166 49 y(If)11 b Fk(k)o(ey)h Fs(is)g(larger)f(than)g Fe(MPI)p 627 49 13 2 v 14 w(INF)o(O)p 736 49 V 14 w(MAX)p 842 49 V 15 w(KEYLEN)p Fs(,)f Fk(MPI)p 1121 49 14 2 v 16 w(INF)o(O)p 1240 49 V 17 w(GET)h Fs(returns)g(a)g(con)o(tin)o(uable) i(error)75 106 y(of)i(class)g Fe(MPI)p 310 106 13 2 v 15 w(ERR)p 404 106 V 14 w(LEN)h Fs(and)f(sets)g Fk(\015ag)g Fs(to)g Fe(false)p Fs(.)75 257 y Fk(MPI)p 160 257 14 2 v 16 w(INF)o(O)p 279 257 V 16 w(GET)p 383 257 V 17 w(NKEYS\(info,)g(nk)o(eys\))117 334 y Fn(IN)155 b Fk(info)516 b Fn(info)13 b(ob)r(ject)i(\(handle\))117 409 y(OUT)108 b Fk(nk)o(eys)485 b Fn(n)o(um)o(b)q(er)13 b(of)h(de\014ned)h(k)o(eys)f (\(in)o(teger\))75 533 y Fg(int)23 b(MPI)p 245 533 15 2 v 17 w(Info)p 358 533 V 17 w(get)p 447 533 V 17 w(nkeys\(MPI)p 680 533 V 16 w(Info)g(info,)g(int*)g(nkeys\))75 620 y(MPI)p 150 620 V 17 w(INFO)p 263 620 V 16 w(GET)p 351 620 V 17 w(NKEYS\(INFO,)g(NKEYS,)g(IERR\))170 676 y(INTEGER)g(INFO,)h(NKEYS,) f(IERR)166 763 y Fs(Returns)16 b(the)f(n)o(um)o(b)q(er)g(of)g(curren)o (tly)h(de\014ned)g(k)o(eys)f(in)i Fk(info)p Fs(.)75 914 y Fk(MPI)p 160 914 14 2 v 16 w(INF)o(O)p 279 914 V 16 w(GET)p 383 914 V 17 w(NTHKEY\(info,)e(n,)g(k)o(ey\))117 991 y Fn(IN)155 b Fk(info)516 b Fn(info)13 b(ob)r(ject)i(\(handle\))117 1066 y(IN)155 b Fk(n)564 b Fn(k)o(ey)14 b(n)o(um)o(b)q(er)f(\(in)o (teger\))117 1141 y(OUT)108 b Fk(k)o(ey)525 b Fn(k)o(ey)14 b(\(string\))75 1266 y Fg(int)23 b(MPI)p 245 1266 15 2 v 17 w(Info)p 358 1266 V 17 w(get)p 447 1266 V 17 w(nthkey\(MPI)p 704 1266 V 15 w(Info)h(info,)f(int)g(n,)h(char*)f(key\))75 1352 y(MPI)p 150 1352 V 17 w(INFO)p 263 1352 V 16 w(GET)p 351 1352 V 17 w(NTHKEY\(INFO,)f(N,)i(KEY,)f(IERR\))170 1409 y(INTEGER)g(INFO,)h(N,)f(IERR)170 1465 y(CHARACTER*\(*\))g(KEY)166 1552 y Fs(Returns)c(the)g Fk(n)p Fs(th)g(de\014ned)h(k)o(ey)e(in)i Fk(info)p Fs(.)29 b(Keys)20 b(are)e(n)o(um)o(b)q(ered)h(0)8 b Fd(:)g(:)g(:)d(N)17 b Fm(\000)c Fs(1)18 b(where)h Fd(N)k Fs(is)c(the)75 1608 y(v)m(alue)c(returned)f(b)o(y)g Fk(MPI)p 521 1608 14 2 v 16 w(INF)o(O)p 640 1608 V 16 w(GET)p 744 1608 V 17 w(NKEYS)p Fs(.)g(All)h(k)o(eys)e(b)q(et)o(w)o(een)h(0)g (and)g Fd(N)e Fm(\000)7 b Fs(1)14 b(are)f(guaran)o(teed)h(to)75 1664 y(b)q(e)g(de\014ned.)21 b(The)14 b(n)o(um)o(b)q(ering)g(of)g(k)o (eys)f(do)q(es)h(not)f(c)o(hange,)h(so)f(that)g Fk(MPI)p 1352 1664 V 16 w(INF)o(O)p 1471 1664 V 16 w(GET)p 1575 1664 V 17 w(NTHKEY)h Fs(with)75 1721 y(a)h(giv)o(en)h(v)m(alue)g(of)f Fk(n)g Fs(will)i(alw)o(a)o(ys)e(return)g(the)g(same)g(k)o(ey)g(for)g (the)g(same)g Fk(info)g Fs(ob)s(ject.)189 1827 y Ff(R)n(ationale.)55 b Fs(Since)19 b(k)o(eys)f(cannot)f(b)q(e)i(deleted,)g(there)f(is)g(no)g (reason)f(to)g(ren)o(um)o(b)q(er.)28 b(\()p Ff(End)18 b(of)189 1884 y(r)n(ationale.)p Fs(\))1875 1945 y Fm(?)e Fl(\(Sept\))75 2084 y Fk(MPI)p 160 2084 V 16 w(INF)o(O)p 279 2084 V 16 w(DUP\(info,)f(newinfo\))117 2162 y Fn(IN)155 b Fk(info)516 b Fn(info)13 b(ob)r(ject)i(\(handle\))117 2237 y(OUT)108 b Fk(newinto)440 b Fn(info)13 b(ob)r(ject)i(\(handle\)) 75 2361 y Fg(int)23 b(MPI)p 245 2361 15 2 v 17 w(Info)p 358 2361 V 17 w(dup\(MPI)p 543 2361 V 16 w(Info)g(info,)h(MPI)p 894 2361 V 16 w(Info)g(*newinfo\))75 2448 y(MPI)p 150 2448 V 17 w(INFO)p 263 2448 V 16 w(DUP\(INFO,)f(NEWINFO,)g(IERR\))170 2504 y(INTEGER)g(INFO,)h(NEWINFO,)e(IERR)1875 2540 y Fm(>)16 b Fl(\(Sept\))166 2591 y Fs(Duplicates)h(an)e(existing)i(info)f (ob)s(ject.)21 b(Creates)15 b(a)h(new)g(ob)s(ject,)f(with)h(same)f(\(k) o(ey)l(,v)m(alue\))h(pairs,)75 2647 y(and)f(the)h(same)f(ordering)g(of) g(k)o(eys.)1190 b Fm(?)16 b Fl(\(Sept\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 24 26 24 25 bop 75 -100 a Fs(24)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(INF)o(O)p 279 49 V 16 w(FREE\(info\))117 126 y Fn(INOUT)62 b Fk(info)516 b Fn(info)13 b(ob)r(ject)i(\(handle\))75 251 y Fg(int)23 b(MPI)p 245 251 15 2 v 17 w(Info)p 358 251 V 17 w(free\(MPI)p 567 251 V 16 w(Info*)g(info\))75 337 y(MPI)p 150 337 V 17 w(INFO)p 263 337 V 16 w(FREE\(INFO,)g(IERR\))g(INTEGER)g(INFO,)g (IERR)-117 374 y Fm(>)15 b Fl(\(Sept\))166 424 y Fs(F)l(rees)g Fk(info)g Fs(and)h(sets)e(it)i(to)e Fe(MPI)p 724 424 13 2 v 15 w(INF)o(O)p 834 424 V 14 w(NULL)p Fs(.)166 528 y Fo(Discussion:)34 b Fn(If)14 b(there)h(is)f(a)g(p)q(ending)g (non-blo)q(c)o(king)e(op)q(eration,)i(should)g(w)o(e)g(sa)o(y)g(it)f (is)h(simply)e(mark)o(ed)75 584 y(for)i(deletion?)20 b(This)15 b(seems)f(lik)o(e)g(it)g(migh)o(t)e(b)q(e)k(an)e(implemen)o (tatio)o(n)e(headac)o(he)j(for)f(what)h(should)f(b)q(e)h(a)f(simple)75 640 y(ob)r(ject.)19 b(If)13 b(w)o(e)h(don't)g(sa)o(y)g(it,)f(then)h (info)f(parsing)h(is)f(sync)o(hronous)i(in)f(the)g(non)o(blo)q(c)o (king)f(calls.)-117 696 y Fm(?)i Fl(\(Sept\))75 809 y Fh(3.4.2)49 b(Signalling)19 b(Pro)q(cesses)75 895 y Fs(It)c(is)h(not)e (p)q(ossible)j(to)d(comm)o(unicate,)h(using)h(messages,)e(with)i(a)e (pro)q(cess)i(represen)o(ted)f(b)o(y)g(a)g(\(group,)75 952 y(rank\))g(pair,)h(except)h(through)e(a)g(comm)o(unicator)h(asso)q (ciated)g(with)g(that)f(group.)22 b(F)l(or)15 b(instance,)h(it)g(is)75 1008 y(not)f(p)q(ossible)i(to)d(send)i(a)f(message)g(to)f(an)h(MPI)h (pro)q(cess)f(spa)o(wned)g(b)o(y)75 1065 y Fk(MPI)p 160 1065 14 2 v 16 w(SP)l(A)-5 b(WN)p 326 1065 V 17 w(INDEPENDENT)p Fs(,)15 b(since)h(no)g(comm)o(unicator)e(is)i(a)o(v)m(ailable.)166 1121 y(Ho)o(w)o(ev)o(er,)e(MPI)h(pro)o(vides)h(simple)g(mec)o(hanisms)g (to)f(manage)f(suc)o(h)i(pro)q(cesses.)75 1177 y Fk(MPI)p 160 1177 V 16 w(SIGNAL)f Fs(sends)g(a)f(signal)h(to)f(a)g(pro)q(cess,)h (and)g Fk(MPI)p 1065 1177 V 15 w(PROCESS)p 1277 1177 V 18 w(MONITOR)h Fs(pro)o(vides)f(a)f(mec)o(ha-)75 1234 y(nism)i(to)e(detect)i(when)f(a)g(pro)q(cess)h(dies)g(or)f(c)o(hanges)g (state.)75 1385 y Fk(MPI)p 160 1385 V 16 w(SIGNAL\(group,)g(rank,)f (signal\))117 1462 y Fn(IN)155 b Fk(group)479 b Fn(group)14 b(con)o(taining)f(pro)q(cess)i(to)f(signal)f(\(handle\))117 1537 y(IN)155 b Fk(rank)504 b Fn(rank)14 b(of)f(pro)q(cess)j(to)e (signal)f(\(in)o(teger\))117 1612 y(IN)155 b Fk(signal)480 b Fn(signal)13 b(t)o(yp)q(e)h(\(in)o(teger\))75 1737 y Fg(int)23 b(MPI)p 245 1737 15 2 v 17 w(Signal\(MPI)p 502 1737 V 16 w(Group)g(group,)g(int)h(rank,)f(int)g(signal\))75 1823 y(MPI)p 150 1823 V 17 w(SIGNAL\(GROUP,)f(RANK,)h(SIGNAL,)g(IERR\)) 170 1880 y(INTEGER)g(GROUP,)g(RANK,)h(SIGNAL,)e(IERR)166 1966 y Fk(MPI)p 251 1966 14 2 v 16 w(SIGNAL)14 b Fs(sends)i(a)e(signal) h(to)f(a)g(pro)q(cess)h(represen)o(ted)g(b)o(y)f(a)h(\(group,)e(rank\)) h(pair.)20 b(If)15 b Fk(rank)f Fs(is)75 2023 y Fe(MPI)p 152 2023 13 2 v 14 w(ANY)p 251 2023 V 15 w(SOURCE)p Fs(,)f(the)i (signal)h(is)g(sen)o(t)f(to)g(ev)o(ery)g(pro)q(cess)g(in)h(the)f (group.)166 2079 y(Where)20 b(POSIX)i(signals)f(are)f(supp)q(orted,)i Fk(signal)f Fs(is)g(a)f(signal)h(de\014ned)h(b)o(y)e(POSIX.)h(It)g(is)g (the)75 2136 y(resp)q(onsibilit)o(y)d(of)d(an)h(implemen)o(tation)g(to) f(translate)h(b)q(et)o(w)o(een)f(signals;)i(in)f(other)f(w)o(ords,)g(a) g Fg(SIGINT)75 2192 y Fs(that)e(has)h(v)m(alue)i Fg(3)e Fs(on)g(system)g(A)g(m)o(ust)f(b)q(e)i(deliv)o(ered)h(as)e(a)g Fg(SIGINT)f Fs(on)h(system)g(B,)g(ev)o(en)g(if)h(system)f(B)75 2248 y(uses)i(the)g(v)m(alue)g Fg(5)g Fs(for)f Fg(SIGINT)p Fs(.)f(If)i(there)g(is)g(no)g(corresp)q(onding)g(signal,)g(the)g(op)q (eration)g(is)g(erroneous)75 2305 y(and)f(the)h(result)f(is)h (unde\014ned.)166 2361 y(An)f(MPI)h(implemen)o(tation)g(m)o(ust)f(also) g(supp)q(ort)g(the)g(MPI-de\014ned)i(signal)f(t)o(yp)q(e)75 2418 y Fk(MPI)p 160 2418 14 2 v 16 w(SIGNAL)p 331 2418 V 16 w(KILL)p Fs(.)21 b(Sending)j(this)f(\\signal")f(will)i(reliably)g (kill)h(a)d(pro)q(cess)g(and)h(attempt)e(to)h(do)75 2474 y(the)f(necessary)h(system-lev)o(el)g(clean)o(up.)39 b(In)22 b(order)e(to)h(allo)o(w)g(appropriate)g(clean)o(up)i(to)d(o)q (ccur,)j(an)75 2531 y(implemen)o(tation)d(could,)g(for)e(instance,)i (send)f(a)f(catc)o(hable)h(signal)g(\(e.g.,)f(SIGINT\))h(follo)o(w)o (ed)g(b)o(y)f(a)75 2587 y(noncatc)o(hable)i(signal)g(\(SIGKILL\))g(if)g (the)f(pro)q(cess)g(had)h(not)e(exited)i(after)f(a)f(short)h(time)g(in) o(terv)m(al.)75 2644 y(MPI)g(requires)h(only)f(that)f(the)h(pro)q(cess) h(b)q(e)f(reliably)i(killed.)33 b(Applications)21 b(should)f(not)f (rely)g(on)g(a)75 2700 y(sp)q(eci\014c)e(implemen)o(tation.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 25 27 25 26 bop 75 -100 a Fi(3.4.)34 b(OTHER)16 b(R)o(OUTINES)1227 b Fs(25)166 49 y(Signals)15 b(are)e(deliv)o(ered)i(async)o(hronously)l (.)20 b(That)13 b(is,)h(when)h(the)e Fk(MPI)p 1362 49 14 2 v 16 w(SIGNAL)h Fs(function)g(returns,)75 106 y(the)k(signal)g(ma) o(y)g(not)f(y)o(et)g(ha)o(v)o(e)h(b)q(een)h(deliv)o(ered.)29 b(If)18 b(a)g(signal)g(is)g(undeliv)o(erable)j(\(e.g.,)c(the)h(pro)q (cess)75 162 y(has)d(already)h(exited\),)f(the)g(signal)h(is)g(silen)o (tly)h(dropp)q(ed.)166 219 y(If)22 b(an)h(MPI)f(implemen)o(tation)h (can)g(determine,)h(at)e(the)g(time)h Fk(MPI)p 1399 219 V 15 w(SIGNAL)g Fs(is)f(called,)k(that)75 275 y(the)18 b(pro)q(cess)h(is)g(already)g(dead,)g(it)f(ma)o(y)g(return)g(the)h(con) o(tin)o(uable)g(error)f Fe(MPI)p 1467 275 13 2 v 14 w(ERR)p 1560 275 V 14 w(NOPROCESS)p Fs(.)f(If)75 332 y Fe(MPI)p 152 332 V 14 w(ANY)p 251 332 V 15 w(SOURCE)12 b Fs(had)i(b)q(een)g(sp)q (eci\014ed)i(and)e(some)f(of)h(the)f(pro)q(cesses)h(are)g(kno)o(wn)f (to)g(b)q(e)h(dead)g(while)75 388 y(others)h(are)g(not,)f(no)h(error)g (is)g(returned.)189 494 y Ff(A)n(dvic)n(e)e(to)i(users.)39 b Fs(Signals)15 b(do)e(not)g(pro)o(vide)h(an)o(y)g(of)f(the)g (reliabilit)o(y)j(or)d(guaran)o(tees)g(of)g(regular)189 551 y(MPI)j(comm)o(unication.)23 b(A)16 b(high)h(qualit)o(y)g(implemen) o(tation)g(will)h(deliv)o(er)f(signals)g(quic)o(kly)h(and)189 607 y(reliably)l(,)23 b(but)d(applications)i(should)f(nev)o(er)f(dep)q (end)i(on)e(ordering.)35 b(Since)21 b(signals)g(ma)o(y)e(not)189 664 y(b)q(e)h(queued,)i(when)e(m)o(ultiple)i(signals)f(are)e(sen)o(t,)i (only)f(one)g(ma)o(y)f(get)h(through.)33 b(Some)20 b(MPI)189 720 y(implemen)o(tations)e(ma)o(y)f(use)h(signals)g(in)o(ternally)l(.) 29 b(If)18 b(an)f(application)i(attempts)e(to)g(use)h(these)189 777 y(signals,)g(unexp)q(ected)i(b)q(eha)o(vior)e(ma)o(y)f(result.)27 b(Finally)l(,)20 b(it)e(ma)o(y)f(not)g(b)q(e)h(p)q(ossible)i(at)d(the)g (im-)189 833 y(plemen)o(tation)g(lev)o(el)i(to)d(restrict)h(the)g (e\013ects)g(of)f(a)h(signal)h(to)e(a)h(single)h(MPI)f(pro)q(cess,)g (so)g(that)189 889 y(a)d(side-e\013ect)i(of)f(sending)h(a)e(signal)i (to)e(a)h(pro)q(cess)g(ma)o(y)f(b)q(e)i(that)e(other)h(pro)q(cesses)g (receiv)o(e)h(that)189 946 y(signal)g(or)e(are)h(killed.)22 b(\()p Ff(End)16 b(of)g(advic)n(e)g(to)h(users.)p Fs(\))75 1068 y Fh(3.4.3)49 b(Noti\014cation)18 b(of)e(change)g(in)h(state)e(of) i(a)g(p)o(ro)q(cess)1875 1107 y Fm(>)f Fl(\(Sept\))75 1153 y Fs(In)d(order)f(to)f(manage)h(pro)q(cesses)g(an)g(MPI)h (application)g(m)o(ust)f(b)q(e)h(able)g(to)e(b)q(e)i(noti\014ed)g(when) g(a)f(pro)q(cess)75 1210 y(exits.)75 1361 y Fk(MPI)p 160 1361 14 2 v 16 w(PROCESS)p 373 1361 V 18 w(MONITOR\(group,)j(rank,) g(event,)h(request\))117 1438 y Fn(IN)155 b Fk(group)479 b Fn(a)14 b(group)g(of)f(pro)q(cesses)j(\(handle\))117 1513 y(IN)155 b Fk(rank)504 b Fn(rank)14 b(of)f(pro)q(cess)j(to)e(b)q (e)g(monitored)f(\(in)o(teger\))117 1588 y(IN)155 b Fk(event)487 b Fn(a)15 b(\015ag)f(sp)q(ecifying)h(what)g(ev)o(en)o(t)h(to)f(b)q(e)g (noti\014ed)h(ab)q(out)e(\(in-)905 1645 y(teger\))117 1720 y(OUT)108 b Fk(request)452 b Fn(a)14 b(request)h(ob)r(ject)g (\(handle\))75 1844 y Fg(int)23 b(MPI)p 245 1844 15 2 v 17 w(Process)p 430 1844 V 16 w(monitor\(MPI)p 710 1844 V 16 w(Group)g(group,)g(int)h(rank,)f(int)g(event,)393 1901 y(MPI)p 468 1901 V 17 w(Request*)g(request\))75 1987 y(MPI)p 150 1987 V 17 w(PROCESS)p 335 1987 V 16 w(MONITOR\(GROUP,)f(RANK,)h(EVENT,)g(REQUEST,)g(IERR\))170 2044 y(INTEGER)g(GROUP,)g(RANK,)h(EVENT,)f(REQUEST\(*\),)f(IERR)1875 2080 y Fm(?)16 b Fl(\(Sept\))166 2130 y Fs(This)d(function)h(pro)o (vides)f(a)f(general)i(metho)q(d)f(for)f(an)g(MPI)h(pro)q(cess)g(to)f (detect)h(a)f(c)o(hange)h(in)h(state)75 2187 y(of)h(another)h(pro)q (cess.)21 b(The)16 b(request)g(completes)g(when)h(the)f(pro)q(cess)g(c) o(hanges)f(state)g(as)h(indicated)h(b)o(y)75 2243 y Fk(event)p Fs(.)166 2300 y Fk(event)i Fs(is)f(an)f(in)o(teger)h(sp)q(ecifying)h(a) e(c)o(hange)h(of)f(state)f(that)h(the)g(calling)j(pro)q(cess)d(wishes)i (to)d(de-)75 2356 y(tect.)j(The)d(most)e(imp)q(ortan)o(t)g(state-c)o (hange)g(and)h(the)g(only)h(one)f(MPI)g(curren)o(tly)g(de\014nes)h(for) e Fk(event)i Fs(is)75 2412 y Fk(MPI)p 160 2412 14 2 v 16 w(PROCESS)p 373 2412 V 18 w(DIED)p Fs(.)f(F)l(or)h(this)h(case,)g Fk(request)h Fs(completes)f(when)g(the)g(pro)q(cess)g(terminates)f(or)g (b)q(e-)75 2469 y(comes)d(unreac)o(hable.)20 b(If)14 b(a)f(pro)q(cess)g(has)g(already)g(terminated)h(or)f(b)q(ecome)g (unreac)o(hable,)i(the)e(request)75 2525 y(completes)j(immediately)l(.) 22 b(If)16 b(an)f(application)i(w)o(an)o(ts)d(to)h(b)q(e)h(noti\014ed)g (of)f(sev)o(eral)g(di\013eren)o(t)h(ev)o(en)o(ts,)f(it)75 2582 y(m)o(ust)g(call)h Fk(MPI)p 356 2582 V 16 w(MONITOR)g Fs(once)g(for)f(eac)o(h)g(ev)o(en)o(t.)20 b(If)c(a)f(m)o(ultiple)i (requests)e(are)g(activ)o(e)h(for)f(a)g(single)75 2638 y(t)o(yp)q(e)g(of)g(ev)o(en)o(t,)g(all)h(requests)f(will)i(complete)f (when)f(the)h(ev)o(en)o(t)f(o)q(ccurs.)541 b Fm(>)16 b Fl(\(Sept\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 26 28 26 27 bop 75 -100 a Fs(26)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)166 49 y Fs(A)k(request)f(generated)h(b)o(y)g Fk(MPI)p 743 49 14 2 v 15 w(PROCESS)p 955 49 V 18 w(MONITOR)h Fs(is)f(not)f (required)i(to)e(progress)g(unless)75 106 y Fk(MPI)p 160 106 V 16 w(TEST)d Fs(or)g(mpifuncMPI)p 609 106 V 18 w(W)-5 b(AIT)15 b(\(or)g(a)g(related)h(function\))g(on)f(that)g (request)g(is)h(called.)22 b(In)16 b(par-)75 162 y(ticular,)e(a)f (handler)h(p)q(osted)f(on)g(that)f(request)h(ma)o(y)f(not)h(b)q(e)g (called)i(un)o(til)f(a)f Fk(MPI)p 1471 162 V 16 w(TEST)g Fs(or)f Fk(MPI)p 1749 162 V 16 w(W)l(AIT)75 219 y Fs(is)h(done.)19 b Fk(MPI)p 328 219 V 16 w(REQUEST)p 547 219 V 18 w(FREE)12 b Fs(on)g(suc)o(h)h(a)f(monitor)g(request)g(marks)f(the)i(request)f (complete)h(b)q(efore)75 275 y(deleting)k(it)e(so)g(that)f(no)i (further)f(action)g(is)h(p)q(erformed)f(b)o(y)g(MPI)g(on)h(b)q(ehalf)g (of)f(the)g(request.)-1832 b Fm(?)15 b Fl(\(Sept\))166 332 y Fs(F)l(uture)d(v)o(ersions)h(of)f(the)h(MPI)f(standard)h(ma)o(y)e (sp)q(ecify)j(new)f(t)o(yp)q(es)g(of)f(ev)o(en)o(ts,)g(and)h(implemen)o (ta-)75 388 y(tions)h(ma)o(y)f(de\014ne)i(new)f(ones)g(as)f(w)o(ell.)21 b(If)14 b(an)g(application)h(w)o(an)o(ts)e(to)g(b)q(e)i(noti\014ed)f (of)g(sev)o(eral)g(di\013eren)o(t)75 444 y(ev)o(en)o(ts,)h(it)g(m)o (ust)g(call)h Fk(MPI)p 552 444 V 16 w(MONITOR)g Fs(once)f(for)g(eac)o (h)g(ev)o(en)o(t.)166 501 y(F)l(or)c(a)h(request)h(generated)f(b)o(y)g Fk(MPI)p 784 501 V 16 w(PROCESS)p 997 501 V 18 w(MONITOR)p Fs(,)g Fk(MPI)p 1333 501 V 16 w(W)l(AIT)g Fs(returns)g(a)g(status)g (that)75 557 y(con)o(tains)j(no)g(useful)i(information)e(\(i.e.,)g(the) g(status)f(is)i(unde\014ned\).)189 664 y Ff(R)n(ationale.)43 b Fs(Another)15 b(p)q(ossibilit)o(y)j(w)o(ould)e(b)q(e)h(to)e(get)g (the)h(exit)g(co)q(de)g(from)f(the)h(status.)k(Ho)o(w-)189 720 y(ev)o(er,)14 b(the)g(pro)q(cess)h(ma)o(y)e(not)h(ha)o(v)o(e)g (exited)h(-)f(it)h(ma)o(y)f(ha)o(v)o(e)f(b)q(ecome)i(unreac)o(hable.)21 b(It)14 b(ma)o(y)g(also)189 777 y(ha)o(v)o(e)h(b)q(een)i(killed)h(b)o (y)e(a)g(signal)h(and)f(what)f(w)o(e'd)h(w)o(an)o(t)f(to)g(kno)o(w)g (is)i(what)e(signal)i(killed)h(it.)k(In)189 833 y(the)15 b(end,)g(w)o(e)g(migh)o(t)g(ha)o(v)o(e)g(to)g(reimplemen)o(t)h(w)o (ait\(\).)j(\()p Ff(End)d(of)g(r)n(ationale.)p Fs(\))-117 894 y Fm(>)f Fl(\(Sept\))75 1034 y Fk(MPI)p 160 1034 V 16 w(GROUP)p 328 1034 V 18 w(MONITOR\(group,)g(event,)h(a)o(rra)o(y)p 935 1034 V 14 w(of)p 986 1034 V 16 w(requests\))117 1111 y Fn(IN)155 b Fk(group)479 b Fn(a)14 b(group)g(of)f(pro)q(cesses)j (\(handle\))117 1186 y(IN)155 b Fk(rank)504 b Fn(rank)14 b(of)f(pro)q(cess)j(to)e(b)q(e)g(monitored)f(\(in)o(teger\))117 1261 y(IN)155 b Fk(event)487 b Fn(a)15 b(\015ag)f(sp)q(ecifying)h(what) g(ev)o(en)o(t)h(to)f(b)q(e)g(noti\014ed)h(ab)q(out)e(\(in-)905 1318 y(teger\))117 1393 y(OUT)108 b Fk(a)o(rra)o(y)p 416 1393 V 15 w(of)p 468 1393 V 16 w(requests)272 b Fn(arra)o(y)14 b(of)f(request)j(ob)r(jects)f(\(arra)o(y)e(of)h(handles\))75 1517 y Fg(int)23 b(MPI)p 245 1517 15 2 v 17 w(Monitor\(MPI)p 526 1517 V 16 w(Group)g(group,)g(int)h(rank,)f(int)g(event,)393 1574 y(MPI)p 468 1574 V 17 w(Request)g(array)p 796 1574 V 16 w(of)p 860 1574 V 17 w(requests[]\))75 1660 y(MPI)p 150 1660 V 17 w(MONITOR\(GROUP,)f(RANK,)h(EVENT,)g(REQUEST,)g(IERR\)) 170 1717 y(INTEGER)g(GROUP,)g(RANK,)h(EVENT,)f(REQUEST\(*\),)f(IERR)166 1803 y Fs(This)c(con)o(v)o(enience)h(function)f(is)g(equiv)m(alen)o(t)h (to)e(calling)i Fk(MPI)p 1258 1803 14 2 v 16 w(PROCESS)p 1471 1803 V 18 w(MONITOR)f Fs(once)g(for)75 1860 y(eac)o(h)c(pro)q (cess)h(in)g Fk(group)p Fs(.)k(The)c Fd(i)p Fs(th)f(elemen)o(t)h(\(n)o (um)o(b)q(ered)f(from)g(0\))f(of)h Fk(a)o(rra)o(y)p 1396 1860 V 15 w(of)p 1448 1860 V 16 w(requests)i Fs(is)f(asso)q(ciated)75 1916 y(with)g(rank)f Fd(i)h Fs(in)g(the)g(group.)k Fk(a)o(rra)o(y)p 687 1916 V 15 w(of)p 739 1916 V 16 w(requests)d Fs(m)o(ust)e(ha)o(v)o (e)h(a)f(n)o(um)o(b)q(er)h(of)f(elemen)o(ts)h(greater)f(than)g(or)75 1972 y(equal)i(to)f(the)g(size)h(of)f Fk(group)p Fs(.)-705 b Fm(?)15 b Fl(\(Sept\))75 2094 y Fh(3.4.4)49 b(Universe)16 b(Size)-117 2124 y Fm(>)f Fl(\(Sept\))75 2180 y Fs(Man)o(y)h (\\dynamic")g(MPI)h(applications)h(are)e(exp)q(ected)h(to)f(exist)h(in) g(a)f(static)g(run)o(time)h(en)o(vironmen)o(t,)75 2236 y(in)22 b(whic)o(h)g(resources)f(ha)o(v)o(e)g(b)q(een)h(allo)q(cated)g (b)q(efore)g(the)f(application)i(is)e(run.)38 b(When)22 b(a)f(user)g(\(or)75 2293 y(p)q(ossibly)c(a)f(batc)o(h)g(system\))f (runs)h(one)g(of)f(these)h(quasi-static)g(applications,)i(he/she/it)e (will)i(usually)75 2349 y(sp)q(ecify)g(a)e(n)o(um)o(b)q(er)i(of)e(pro)q (cesses)h(to)f(start)g(and)h(a)f(total)g(n)o(um)o(b)q(er)h(of)g(pro)q (cesses)g(that)f(are)g(exp)q(ected.)75 2406 y(An)f(application)h (simply)f(needs)h(to)d(kno)o(w)h(ho)o(w)g(man)o(y)g(slots)g(there)h (are,)f(i.e.,)g(ho)o(w)g(man)o(y)g(pro)q(cesses)h(it)75 2462 y(should)h(spa)o(wn.)166 2519 y(MPI)21 b(pro)o(vides)h(an)g(in)o (teger)f(attribute)h(on)f Fe(MPI)p 1033 2519 13 2 v 14 w(COMM)p 1177 2519 V 15 w(W)o(ORLD)p Fs(,)g Fe(MPI)p 1449 2519 V 14 w(UNIVERSE)p 1661 2519 V 14 w(SIZE)p Fs(,)g(that)75 2575 y(allo)o(ws)15 b(the)h(application)h(to)d(obtain)i(this)f (information)h(in)g(a)f(p)q(ortable)h(manner.)k(This)15 b(v)m(ariable)i(indi-)75 2632 y(cates)g(the)g(total)f(n)o(um)o(b)q(er)i (of)f(pro)q(cesses)g(that)f(are)h(exp)q(ected.)27 b(An)17 b(application)i(t)o(ypically)g(subtracts)75 2688 y(the)13 b(size)h(of)f Fk(MPI)p 370 2688 14 2 v 16 w(COMM)p 528 2688 V 17 w(W)o(ORLD)g Fs(from)f Fk(MPI)p 907 2688 V 16 w(UNIVERSE)p 1137 2688 V 18 w(SIZE)h Fs(to)f(\014nd)i(out)f(ho)o(w)g (man)o(y)g(pro)q(cesses)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 27 29 27 28 bop 75 -100 a Fi(3.4.)34 b(OTHER)16 b(R)o(OUTINES)1227 b Fs(27)75 49 y(it)20 b(should)i(spa)o(wn.)34 b Fk(MPI)p 530 49 14 2 v 16 w(UNIVERSE)p 760 49 V 17 w(SIZE)20 b Fs(is)h(initialized)j(in)d Fk(MPI)p 1296 49 V 15 w(INIT)f Fs(and)g(is)h(not)f(c)o(hanged)g(b)o(y)75 106 y(MPI.)166 162 y Fe(MPI)p 243 162 13 2 v 14 w(UNIVERSE)p 455 162 V 14 w(SIZE)15 b Fs(is)i(determined)g(b)o(y)e(the)h(application)i (startup)d(mec)o(hanism)h(in)h(a)f(w)o(a)o(y)e(not)75 219 y(sp)q(eci\014ed)h(b)o(y)f(MPI.)f(\(The)g(size)h(of)f Fe(MPI)p 753 219 V 14 w(COMM)p 897 219 V 15 w(W)o(ORLD)g Fs(is)h(another)f(example)h(of)f(suc)o(h)g(a)g(parameter\).)166 275 y(P)o(ossibilities)18 b(for)c(ho)o(w)h Fe(MPI)p 655 275 V 14 w(UNIVERSE)p 867 275 V 14 w(SIZE)g Fs(migh)o(t)g(b)q(e)g(set)g (include)143 368 y Fm(\017)23 b Fs(A)15 b Fg(-universe)p 457 368 15 2 v 16 w(size)f Fs(argumen)o(t)h(to)f(a)h(program)f(that)h (starts)f(MPI)h(pro)q(cesses.)143 462 y Fm(\017)23 b Fs(Automatic)13 b(in)o(teraction)h(with)g(a)f(batc)o(h)g(sc)o(heduler)i (to)e(\014gure)g(out)g(ho)o(w)g(man)o(y)g(pro)q(cessors)g(ha)o(v)o(e) 189 518 y(b)q(een)j(allo)q(cated)g(to)f(an)g(application.)143 612 y Fm(\017)23 b Fs(An)15 b(en)o(vironmen)o(t)h(v)m(ariable)g(set)f (b)o(y)g(the)h(user.)143 705 y Fm(\017)23 b Fs(Extra)14 b(information)h(passed)h(to)e Fk(MPI)p 848 705 14 2 v 16 w(SP)l(A)-5 b(WN)16 b Fs(through)f(the)g Fk(info)g Fs(argumen)o(t.)166 798 y(An)g(implemen)o(tation)i(m)o(ust)e(do)q (cumen)o(t)g(ho)o(w)g Fk(MPI)p 1067 798 V 16 w(UNIVERSE)p 1297 798 V 18 w(SIZE)g Fs(is)g(set.)20 b(An)c(implemen)o(ta-)75 855 y(tion)10 b(ma)o(y)g(not)g(supp)q(ort)g(the)g(abilit)o(y)h(to)f (set)g Fk(MPI)p 907 855 V 16 w(UNIVERSE)p 1137 855 V 17 w(SIZE)p Fs(,)g(in)h(whic)o(h)g(case)f Fk(MPI)p 1616 855 V 16 w(UNIVERSE)p 1846 855 V 17 w(SIZE)75 911 y Fs(is)16 b(equal)g(to)e Fe(MPI)p 373 911 13 2 v 15 w(UNDEFINED)p Fs(.)166 968 y Fk(MPI)p 251 968 14 2 v 16 w(UNIVERSE)p 481 968 V 17 w(SIZE)h Fs(is)h(a)e(recommendation,)h(not)g(necessarily)h (a)f(hard)g(limit.)21 b(F)l(or)15 b(instance,)75 1024 y(some)20 b(implemen)o(tations)h(ma)o(y)f(allo)o(w)g(an)g(application)i (to)e(spa)o(wn)g(50)f(pro)q(cesses)i(p)q(er)g(pro)q(cessor,)f(if)75 1081 y(they)15 b(are)f(requested.)20 b(Ho)o(w)o(ev)o(er,)14 b(it)h(is)g(lik)o(ely)h(that)e(the)h(user)g(only)g(w)o(an)o(ts)e(to)h (spa)o(wn)h(one)g(pro)q(cess)f(p)q(er)75 1137 y(pro)q(cessor.)166 1194 y Fk(MPI)p 251 1194 V 16 w(UNIVERSE)p 481 1194 V 17 w(SIZE)c Fs(is)h(assumed)f(to)g(ha)o(v)o(e)f(b)q(een)j(sp)q (eci\014ed)g(when)f(an)f(application)i(w)o(as)d(started,)75 1250 y(and)k(is)g(in)g(essence)h(a)e(p)q(ortable)h(mec)o(hanism)g(to)f (allo)o(w)h(the)g(user)f(to)g(pass)h(to)f(the)g(application)j (\(through)75 1307 y(the)g(MPI)h(pro)q(cess)f(startup)g(mec)o(hanism\)) h(a)f(piece)h(of)f(critical)i(run)o(time)f(information.)k(Note)15 b(that)g(no)75 1363 y(in)o(teraction)h(with)g(the)g(run)o(time)g(en)o (vironmen)o(t)g(is)g(required.)23 b(If)16 b(the)g(run)o(time)g(en)o (vironmen)o(t)g(c)o(hanges)75 1419 y(size)g(while)i(an)d(application)i (is)f(running)h Fk(MPI)p 880 1419 V 16 w(UNIVERSE)p 1110 1419 V 17 w(SIZE)f Fs(is)g(not)f(up)q(dated,)h(and)g(the)f(applica-)75 1476 y(tion)g(m)o(ust)g(\014nd)h(out)f(through)g(direct)h(comm)o (unication)f(with)h(the)f(run)o(time)h(system.)75 1598 y Fh(3.4.5)49 b(Singleton)17 b(MPI)p 517 1598 15 2 v 19 w(INIT)1875 1637 y Fm(>)f Fl(\(Sept\))75 1683 y Fs(A)i(high-qualit)o (y)h(implemen)o(tation)g(will)g(allo)o(w)f(an)o(y)g(pro)q(cess)g (\(including)i(those)d(not)g(started)g(with)h(a)75 1740 y(\\parallel)11 b(application")h(mec)o(hnism\))f(to)e(b)q(ecome)i(an)g (MPI)f(pro)q(cess)g(b)o(y)g(calling)i Fk(MPI)p 1532 1740 14 2 v 16 w(INIT)p Fs(.)d(Suc)o(h)i(a)f(pro-)75 1796 y(cess)15 b(can)g(then)g(connect)g(to)g(other)f(MPI)h(pro)q(cesses)g (using)g(the)g Fk(MPI)p 1273 1796 V 16 w(A)o(CCEPT)g Fs(and)g Fk(MPI)p 1648 1796 V 16 w(CONNECT)75 1853 y Fs(routines,)f(or)e(spa)o(wn)h(other)g(MPI)g(pro)q(cesses.)20 b(MPI)13 b(do)q(es)h(not)f(mandate)f(this)i(b)q(eha)o(vior,)g(but)f (strongly)75 1909 y(encourages)i(it)h(where)f(tec)o(hnically)i (feasible.)189 2015 y Ff(A)n(dvic)n(e)e(to)h(implementors.)40 b Fs(T)l(o)15 b(start)f(an)h(MPI-1)g(application)i(with)f(more)e(than)h (one)h(pro)q(cess)189 2071 y(requires)c(some)f(sp)q(ecial)j(co)q (ordination.)19 b(The)12 b(pro)q(cesses)g(m)o(ust)f(b)q(e)h(started)f (at)g(the)g(\\same")g(time,)189 2128 y(they)k(m)o(ust)f(ha)o(v)o(e)h(a) g(mec)o(hanism)g(to)g(establish)h(comm)o(unication,)g(etc.)j(Either)d (the)f(user)g(or)g(the)189 2184 y(op)q(erating)g(system)g(m)o(ust)f (tak)o(e)h(sp)q(ecial)i(steps)e(b)q(ey)o(ond)h(simply)g(starting)f(pro) q(cesses.)189 2259 y(When)j(an)g(application)i(en)o(ters)e Fk(MPI)p 849 2259 V 15 w(INIT)p Fs(,)f(clearly)i(it)g(m)o(ust)e(b)q(e)i (able)g(to)e(determine)i(if)g(these)189 2315 y(sp)q(ecial)c(steps)e(w)o (ere)f(tak)o(en.)19 b(MPI-1)13 b(do)q(es)h(not)e(sa)o(y)h(what)f(happ)q (ens)i(if)g(these)f(sp)q(ecial)i(steps)e(w)o(ere)189 2372 y(not)18 b(tak)o(en)g(-)h(presumably)g(this)h(is)f(treated)f(as)g (an)h(error)f(in)h(starting)f(the)h(MPI)g(application.)189 2428 y(MPI-2)c(recommends)g(the)g(follo)o(wing)h(b)q(eha)o(vior.)231 2534 y(1.)22 b(If)e(a)f(pro)q(cess)h(en)o(ters)g Fk(MPI)p 767 2534 V 15 w(INIT)f Fs(and)h(determines)h(that)e(no)h(sp)q(ecial)h (steps)f(w)o(ere)f(tak)o(en)289 2591 y(\(i.e.,)d(it)g(has)g(not)g(b)q (een)i(giv)o(en)e(the)h(information)f(to)g(form)f(an)i Fe(MPI)p 1466 2591 13 2 v 14 w(COMM)p 1610 2591 V 14 w(W)o(ORLD)g Fs(with)289 2647 y(other)e(pro)q(cesses\))g(it)h(succeeds) h(and)f(forms)e(a)i(singleton)g(MPI)g(program,)e(that)h(is,)g(one)h(in) 289 2704 y(whic)o(h)g Fe(MPI)p 496 2704 V 14 w(COMM)p 640 2704 V 15 w(W)o(ORLD)f Fs(has)g(size)h(1.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 28 30 28 29 bop 75 -100 a Fs(28)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)231 49 y Fs(2.)22 b(F)l(urthermore,)10 b(if)g(a)g(pro)q(cess)h(is)f(spa)o (wned)h(with)f Fk(MPI)p 1186 49 14 2 v 16 w(SP)l(A)-5 b(WN)p 1352 49 V 17 w(INDEPENDENT)11 b Fs(or)e Fk(MPI)p 1828 49 V 16 w(SP)l(A)-5 b(WN)p 1994 49 V 17 w(MUL)l(TIPLE)p 2228 49 V 16 w(INDEPENDENT)289 106 y Fs(and)15 b(the)g Fe(MPI)p 532 106 13 2 v 15 w(NONMPI)g Fs(\015ag,)f(it)h(will)i(form)e (a)g(singleton)h(MPI)f(pro)q(cess.)189 211 y(In)e(some)f(implemen)o (tations,)i(MPI)e(ma)o(y)g(not)g(b)q(e)h(able)g(to)f(function)h (without)g(an)f(\\MPI)g(en)o(viron-)189 268 y(men)o(t.")19 b(F)l(or)c(example,)g(MPI)h(ma)o(y)e(require)i(that)f(daemons)g(b)q(e)h (running)g(or)f(MPI)g(ma)o(y)f(not)h(b)q(e)189 324 y(able)g(to)f(w)o (ork)g(at)g(all)i(on)f(the)f(fron)o(t-end)h(of)f(an)h(MPP)l(.)f(In)i (this)f(case,)f(an)h(MPI)g(implemen)o(tation)189 381 y(ma)o(y)f(either)231 474 y(1.)22 b(Create)14 b(the)i(en)o(vironmen)o (t)f(\(e.g.,)f(start)g(a)g(daemon\))h(or)231 547 y(2.)22 b(Return)13 b(an)g(error)g(if)g(it)g(cannot)g(create)g(the)g(en)o (vironmen)o(t)h(and)f(the)g(en)o(vironmen)o(t)g(has)g(not)289 603 y(b)q(een)j(started)e(indep)q(enden)o(tly)m(.)189 697 y(A)20 b(high)g(qualit)o(y)h(implemen)o(tation)g(will)h(try)d(to)h (create)f(a)h(singleton)h(MPI)f(pro)q(cess)g(and)g(not)189 753 y(return)15 b(an)g(error.)189 828 y(On)20 b(certain)h(mac)o(hines,) h(c)o(hildren)g(created)e(b)o(y)g Fk(MPI)p 1136 828 14 2 v 16 w(SP)l(A)-5 b(WN)p 1302 828 V 17 w(INDEPENDENT)21 b Fs(ma)o(y)e(estab-)189 884 y(lish)f(comm)o(unication)f(b)q(efore)h Fk(MPI)p 816 884 V 15 w(INIT)p Fs(.)e(In)i(this)f(case,)g Fk(MPI)p 1291 884 V 16 w(INIT)f Fs(should)i(ignore)f(the)g(extra)189 941 y(structure)e(and)g(set)g Fk(MPI)p 628 941 V 16 w(COMM)p 786 941 V 16 w(W)o(ORLD)g Fs(to)g(a)g(duplicate)i(of)d Fk(MPI)p 1403 941 V 16 w(COMM)p 1561 941 V 17 w(SELF)p Fs(.)189 1016 y(\()p Ff(End)h(of)i(advic)n(e)f(to)g(implementors.)p Fs(\))-117 1076 y Fm(?)f Fl(\(Sept\))75 1186 y Fh(3.4.6)49 b Fe(MPI)p 301 1186 13 2 v 14 w(APPNUM)-117 1215 y Fm(>)15 b Fl(\(Sept\))75 1272 y Fs(There)e(is)g(a)f(prede\014ned)i(in)o(teger)f (attribute)f Fe(MPI)p 913 1272 V 15 w(APPNUM)f Fs(of)h Fe(MPI)p 1239 1272 V 14 w(COMM)p 1383 1272 V 15 w(W)o(ORLD)p Fs(.)g(If)h(a)f(pro)q(cess)h(w)o(as)75 1328 y(spa)o(wned)24 b(with)f Fk(MPI)p 463 1328 14 2 v 16 w(SP)l(A)-5 b(WN)p 629 1328 V 17 w(MUL)l(TIPLE)23 b Fs(or)g Fk(MPI)p 1032 1328 V 16 w(SP)l(A)-5 b(WN)p 1198 1328 V 17 w(MUL)l(TIPLE)p 1432 1328 V 15 w(INDEPENDENT)24 b Fs(\(and)75 1384 y(the)19 b Fe(MPI)p 234 1384 13 2 v 14 w(MPI)f Fs(\015ag\),)g Fe(MPI)p 536 1384 V 15 w(APPNUM)f Fs(is)i(the)g(command-line)h(n)o(um)o (b)q(er)f(that)f(generated)g(the)h(curren)o(t)75 1441 y(pro)q(cess.)42 b(Num)o(b)q(ering)24 b(starts)d(from)h(zero.)42 b(If)23 b(a)g(pro)q(cess)g(w)o(as)e(spa)o(wned)i(with)g Fk(MPI)p 1648 1441 14 2 v 16 w(SP)l(A)-5 b(WN)23 b Fs(or)75 1497 y Fk(MPI)p 160 1497 V 16 w(SP)l(A)-5 b(WN)p 326 1497 V 17 w(INDEPENDENT)15 b Fs(it)h(will)h(ha)o(v)o(e)e Fe(MPI)p 985 1497 13 2 v 14 w(APPNUM)f Fs(equal)i(to)f(zero.)166 1554 y(Additionally)l(,)g(if)f(the)f(pro)q(cess)g(w)o(as)f(not)g (started)g(b)o(y)h(a)f(spa)o(wn)h(call,)h(but)f(b)o(y)g(an)g(implemen)o (tation-)75 1610 y(sp)q(eci\014c)20 b(startup)e(mec)o(hanism)h(that)f (can)g(handle)i(m)o(ultiple)g(pro)q(cess)f(sp)q(eci\014cations,)i Fe(MPI)p 1688 1610 V 14 w(APPNUM)75 1667 y Fs(should)14 b(b)q(e)g(set)e(to)h(the)g(n)o(um)o(b)q(er)g(of)g(the)g(corresp)q (onding)h(pro)q(cess)f(sp)q(eci\014cation.)21 b(If)13 b(this)h(do)q(esn't)e(mak)o(e)75 1723 y(sense)i(in)g(the)f(con)o(text)g (of)g(the)g(implemen)o(tation,)i Fe(MPI)p 1016 1723 V 14 w(APPNUM)d Fs(should)i(b)q(e)g(set)f(to)g Fe(MPI)p 1618 1723 V 14 w(UNDEFINED)p Fs(.)189 1828 y Ff(R)n(ationale.)54 b Fs(When)18 b(a)f(single)i(application)h(is)e(started,)f(it)h(is)g (able)g(to)f(\014gure)h(out)f(ho)o(w)g(man)o(y)189 1885 y(pro)q(cesses)e(there)h(are)f(b)o(y)g(lo)q(oking)h(at)f(the)g(size)h (of)f Fe(MPI)p 1149 1885 V 15 w(COMM)p 1294 1885 V 14 w(W)o(ORLD)p Fs(.)g(An)h(application)h(con-)189 1941 y(sisting)h(of)g(m)o(ultiple)i(SPMD)e(sub-applications)i(has)e(no)g(w)o (a)o(y)f(to)g(\014nd)i(out)f(this)g(information.)189 1998 y(While)e(there)g(are)f(w)o(a)o(ys)f(to)h(\014gure)g(it)h(out)f (in)h(sp)q(ecial)h(cases,)e(there)h(is)g(no)f(general)h(mec)o(hanism.) 189 2054 y Fe(MPI)p 266 2054 V 14 w(APPNUM)e Fs(pro)o(vides)i(suc)o(h)f (a)g(general)h(mec)o(hanism.)k(\()p Ff(End)c(of)g(r)n(ationale.)p Fs(\))-117 2114 y Fm(?)f Fl(\(Sept\))75 2246 y Fp(3.5)59 b(Establishing)20 b(Communication)h(Bet)n(w)n(een)f(Indep)r(endent)g (Pro)r(cesses)75 2394 y Fo(Curren)o(t)14 b(Status:)j Fn(Reading)c(and)h(\014rst)g(v)o(ote)g(on)g(en)o(tire)g(section.)166 2498 y Fs(This)22 b(section)f(pro)o(vides)h(the)f(functions)h(needed)g (to)e(solv)o(e)i(the)f(general)g(case)g(of)g(creating)g(an)75 2554 y(in)o(tercomm)o(unicator)c(b)q(et)o(w)o(een)g(t)o(w)o(o)e(sets)h (of)h(MPI)f(pro)q(cesses)h(that)f(do)h(not)f(share)h(an)f(existing)i (com-)75 2611 y(m)o(unicator.)i(This)c(situation)f(arises)h(in)g(sev)o (eral)f(con)o(texts:)131 2704 y(1.)22 b(Tw)o(o)14 b(parts)g(of)h(an)g (application)i(that)e(are)g(started)f(indep)q(enden)o(tly)k(need)e(to)f (comm)o(unicate.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 29 31 29 30 bop 75 -100 a Fi(3.5.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fs(29)131 49 y(2.)22 b(A)15 b(visualization)i(to)q(ol)e(w)o(an)o(ts)f(to)h(attac)o (h)f(to)g(a)h(running)i(pro)q(cess.)131 143 y(3.)22 b(A)16 b(serv)o(er)f(ma)o(y)g(w)o(an)o(t)g(to)g(accept)h(connections)h(from)f (a)f(m)o(ultiple)j(clien)o(ts.)23 b(This)16 b(situation)h(can)189 200 y(b)q(e)f(more)e(complex)i(than)f(traditional)h(clien)o(t-serv)o (er)h(programs.)h(In)e(particular,)243 293 y Fm(\017)23 b Fs(Both)15 b(clien)o(t)h(and)g(serv)o(er)e(ma)o(y)h(b)q(e)h(parallel) g(programs.)243 366 y Fm(\017)23 b Fs(Multiple)18 b(instances)f(of)e (the)i(clien)o(t)g(ma)o(y)f(w)o(an)o(t)f(to)h(in)o(teract)g(with)g(the) g(same)g(instance)h(of)289 423 y(the)e(serv)o(er)g(at)f(the)i(same)e (time,)i(as)e(w)o(ell)j(as)d(serially)l(.)243 496 y Fm(\017)23 b Fs(Since)17 b(this)f(is)g(an)f(application)i(rather)e(than)g(a)g (system)h(service,)g(there)f(ma)o(y)g(b)q(e)h(m)o(ultiple)289 552 y(instances)g(of)e(the)i(serv)o(er)e(running)j(on)e(the)g(same)g (host.)75 646 y(In)i(eac)o(h)f(of)g(these)g(situations,)g(neither)h (set)f(of)g(pro)q(cesses)h(spa)o(wns)e(the)h(other,)g(but)g(they)h (wish)f(to)g(es-)75 703 y(tablish)d(comm)o(unication.)20 b(As)12 b(in)h(the)f(case)g(of)g Fk(MPI)p 971 703 14 2 v 16 w(SP)l(A)-5 b(WN)p Fs(,)13 b(w)o(e)f(will)h(use)g(MPI)f(in)o (tercomm)o(unicators)75 759 y(to)i(express)h(this)g(situation.)20 b(That)14 b(is,)h(w)o(e)g(will)h(de\014ne)g(MPI)f(functions)g(that)f (supp)q(ort)h(creation)g(of)f(an)75 816 y(in)o(tercomm)o(unicator)h(in) h(whic)o(h)g(the)g(lo)q(cal)g(and)f(remote)g(groups)g(are)g(the)g (original)h(sets)f(of)g(pro)q(cesses.)166 872 y(Establishing)21 b(con)o(tact)e(b)q(et)o(w)o(een)h(t)o(w)o(o)f(groups)g(of)h(pro)q (cesses)g(that)f(do)h(not)f(share)h(an)g(existing)75 928 y(comm)o(unicator)14 b(is)i(a)e(collectiv)o(e,)i(but)f(asymmetric)g (pro)q(cess.)20 b(One)c(group)e(of)g(pro)q(cesses)i(indicates)g(its)75 985 y(willingness)21 b(to)c(accept)h(connections)h(from)e(other)h (groups)f(of)h(pro)q(cesses.)28 b(W)l(e)18 b(will)i(call)f(this)f (group)75 1041 y(the)c(\(parallel\))h Ff(server)p Fs(,)f(ev)o(en)g(if)h (this)f(is)h(not)f(a)f(clien)o(t/serv)o(er)i(t)o(yp)q(e)f(of)g (application.)21 b(The)15 b(other)e(group)75 1098 y(connects)j(to)e (the)h(serv)o(er;)g(w)o(e)g(will)i(call)f(it)f(the)h Ff(client)p Fs(.)75 1220 y Fh(3.5.1)49 b(Names,)15 b(addresses,)g(p)q (o)o(rts,)g(and)h(all)i(that)75 1305 y Fs(Almost)i(all)i(of)e(the)g (complexit)o(y)h(in)g(MPI)g(clien)o(t/serv)o(er)g(routines)g(addresses) f(the)h(question)g(\\ho)o(w)75 1362 y(do)q(es)16 b(the)g(clien)o(t)i (\014nd)e(out)g(ho)o(w)f(to)h(con)o(tact)f(the)h(serv)o(er?")22 b(The)16 b(di\016cult)o(y)l(,)i(of)d(course,)h(is)h(that)e(there)75 1418 y(is)k(no)g(existing)h(comm)o(unication)f(c)o(hannel)h(b)q(et)o(w) o(een)f(them,)h(y)o(et)e(they)h(m)o(ust)f(someho)o(w)g(agree)h(on)f(a) 75 1475 y(rendezv)o(ous)e(p)q(oin)o(t)f(where)h(they)f(will)i (establish)f(comm)o(unication)g(-)g(Catc)o(h)e(22.)166 1531 y(Agreeing)19 b(on)g(a)f(rendezv)o(ous)h(p)q(oin)o(t)g(alw)o(a)o (ys)f(in)o(v)o(olv)o(es)h(a)f(third)i(part)o(y)l(.)29 b(The)19 b(third)g(part)o(y)f(ma)o(y)75 1588 y(b)q(e,)i(for)f (instance,)h(a)f(higher)g(lev)o(el)i(of)d(managemen)o(t,)h(whic)o(h)h (tells)g(them)f(b)q(oth,)g(\\thou)g(shalt)g(meet)75 1644 y(at)c(p)q(oin)o(t)i(X)f(at)f(high)h(no)q(on.")22 b(The)16 b(third)h(part)o(y)e(migh)o(t)h(b)q(e)g(a)g(go-b)q(et)o(w)o(een,)f (that)h(gets)f(a)g(rendezv)o(ous)75 1701 y(p)q(oin)o(t)i(from)f(the)h (serv)o(er)g(and)f(passes)h(it)g(to)f(the)h(clien)o(t.)26 b(Complicating)18 b(matters)e(migh)o(t)g(b)q(e)i(the)f(fact)75 1757 y(that)h(the)i(clien)o(t)g(do)q(esn't)f(really)h(care)g(what)e (serv)o(er)h(it)g(con)o(tacts,)g(only)h(that)f(it)g(b)q(e)h(able)g(to)e (get)h(in)75 1813 y(touc)o(h)c(with)h(one)f(that)f(can)i(handle)g(its)g (request.)166 1870 y(Ideally)l(,)21 b(MPI)d(can)h(accomo)q(date)f(a)g (wide)h(v)m(ariet)o(y)g(of)f(run-time)h(systems)f(while)i(retaining)f (the)75 1926 y(abilit)o(y)d(to)f(write)g(simple)i(p)q(ortable)f(co)q (de.)k(The)c(follo)o(wing)f(should)i(b)q(e)e(compatible)i(with)e(MPI:) 143 2033 y Fm(\017)23 b Fs(The)15 b(serv)o(er)g(resides)h(at)f(a)f(w)o (ell-kno)o(wn)j(in)o(ternet)e(address)g(host:p)q(ort.)143 2126 y Fm(\017)23 b Fs(The)18 b(serv)o(er)f(prin)o(ts)h(out)g(an)g (address)f(to)h(the)g(terminal,)g(the)g(user)g(giv)o(es)g(this)h (address)f(to)f(the)189 2183 y(clien)o(t)f(program.)143 2277 y Fm(\017)23 b Fs(The)14 b(serv)o(er)h(places)g(the)g(address)f (information)h(on)g(a)f(nameserv)o(er,)g(where)h(it)g(can)f(b)q(e)i (retriev)o(ed)189 2333 y(with)f(an)g(agreed-up)q(on)h(name.)143 2427 y Fm(\017)23 b Fs(The)17 b(serv)o(er)g(to)g(whic)o(h)h(the)f (clien)o(t)i(connects)e(is)h(actually)g(a)f(brok)o(er,)g(acting)g(as)g (a)g(middleman)189 2483 y(b)q(et)o(w)o(een)e(the)g(clien)o(t)i(and)e (the)h(real)f(serv)o(er.)166 2590 y(MPI)f(do)q(es)h(not)e(require)i(a)f (nameserv)o(er,)g(so)g(not)f(all)i(implemen)o(tations)h(will)g(b)q(e)e (able)h(to)f(supp)q(ort)75 2646 y(all)i(of)e(the)h(ab)q(o)o(v)o(e)f (scenarios.)20 b(Ho)o(w)o(ev)o(er,)14 b(MPI)g(pro)o(vides)h(an)g (optional)g(nameserv)o(er)g(in)o(terface,)g(and)f(is)75 2703 y(compatible)i(with)g(external)f(name)h(serv)o(ers.)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 30 32 30 31 bop 75 -100 a Fs(30)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)166 49 y Fs(A)k Fk(p)q(o)o(rt)p 300 49 14 2 v 17 w(name)g Fs(is)g(a)g Ff(system-supplie)n(d)g Fs(string)g(that)g(enco)q(des)h(a)f (lo)o(w-lev)o(el)i(net)o(w)o(ork)d(address)i(at)75 106 y(whic)o(h)f(a)e(serv)o(er)h(can)g(b)q(e)g(con)o(tacted.)28 b(T)o(ypically)20 b(this)e(is)g(an)g(IP)g(address)g(and)g(a)g(p)q(ort)g (n)o(um)o(b)q(er)g(but)75 162 y(an)d(implemen)o(tation)h(is)g(free)f (to)f(use)h(an)o(y)g(proto)q(col.)20 b(The)15 b(serv)o(er)g (establishes)h(a)e Fk(p)q(o)o(rt)p 1578 162 V 17 w(name)h Fs(with)g(the)75 219 y Fk(MPI)p 160 219 V 16 w(PORT)p 298 219 V 17 w(OPEN)j Fs(routine.)26 b(It)18 b(accepts)f(a)g (connection)h(to)f(a)g(giv)o(en)h(p)q(ort)e(with)i Fk(MPI)p 1624 219 V 16 w(A)o(CCEPT)p Fs(.)f(A)75 275 y(clien)o(t)f(uses)g Fk(p)q(o)o(rt)p 375 275 V 16 w(name)f Fs(to)g(connect)g(to)g(the)g (serv)o(er.)166 332 y(By)j(itself,)i(the)e Fk(p)q(o)o(rt)p 529 332 V 17 w(name)g Fs(mec)o(hanism)h(is)f(completely)i(p)q(ortable,) f(but)f(it)h(ma)o(y)f(b)q(e)g(clumsy)i(to)75 388 y(use,)g(b)q(ecause)g (of)e(the)h(necessit)o(y)h(to)e(comm)o(unicate)i Fk(p)q(o)o(rt)p 1095 388 V 16 w(name)f Fs(to)f(the)h(clien)o(t.)32 b(It)19 b(w)o(ould)h(b)q(e)f(more)75 444 y(con)o(v)o(enien)o(t)c(if)h(a)e(serv) o(er)h(could)g(sp)q(ecify)h(that)f(it)g(b)q(e)g(kno)o(wn)g(b)o(y)f(a)h Ff(applic)n(ation-supplie)n(d)g Fk(service)p 1760 444 V 16 w(name)75 501 y Fs(so)g(that)f(the)i(clien)o(t)g(could)g(connect)g (to)e(that)h Fk(service)p 1000 501 V 17 w(name)g Fs(without)g(kno)o (wing)g(the)g Fk(p)q(o)o(rt)p 1638 501 V 17 w(name)p Fs(.)166 557 y(An)j(MPI)f(implemen)o(tation)i(ma)o(y)d(allo)o(w)i(the)f (serv)o(er)g(to)g(publish)j(a)d(\()p Fk(p)q(o)o(rt)p 1469 557 V 16 w(name)p Fs(,)g Fk(service)p 1742 557 V 17 w(name)p Fs(\))75 614 y(pair)h(with)g Fk(MPI)p 362 614 V 16 w(NAME)p 507 614 V 17 w(PUBLISH)f Fs(and)h(the)g(clien)o(t)h (to)e(retriev)o(e)h(the)f(p)q(ort)h(name)f(from)g(the)h(service)75 670 y(name)g(with)g Fk(MPI)p 390 670 V 16 w(NAME)p 535 670 V 17 w(GET)p Fs(.)f(This)i(allo)o(ws)f(three)g(lev)o(els)h(of)e(p)q (ortabilit)o(y)l(,)i(with)g(increasing)g(lev)o(els)75 727 y(of)c(functionalit)o(y)l(.)131 821 y(1.)22 b(Applications)17 b(that)f(do)f(not)h(rely)g(on)g(the)g(abilit)o(y)h(to)e(publish)j (names)e(are)g(the)g(most)f(p)q(ortable.)189 877 y(T)o(ypically)h(the)g Fk(p)q(o)o(rt)p 549 877 V 16 w(name)f Fs(m)o(ust)g(b)q(e)g(transferred) g(\\b)o(y)g(hand")g(from)g(serv)o(er)g(to)f(clien)o(t.)131 971 y(2.)22 b(Applications)d(that)e(use)h(the)g Fk(MPI)p 803 971 V 15 w(NAME)p 947 971 V 17 w(PUBLISH)g Fs(mec)o(hanism)g(are)g (completely)g(p)q(ortable)189 1027 y(among)f(implemen)o(tations)j(that) d(pro)o(vide)i(this)g(service.)30 b(T)l(o)18 b(b)q(e)h(p)q(ortable)g (among)e(all)i(imple-)189 1084 y(men)o(tations,)f(these)g(applications) i(should)g(ha)o(v)o(e)d(a)h(fall-bac)o(k)i(mec)o(hanism)e(that)g(can)g (b)q(e)h(used)189 1140 y(when)c(names)g(are)g(not)g(published.)131 1234 y(3.)22 b(Applications)13 b(ma)o(y)d(ignore)i(MPI's)f(name)g (publishing)j(functionalit)o(y)f(and)e(use)h(their)g(o)o(wn)f(mec)o(h-) 189 1290 y(anism)17 b(\(p)q(ossibly)i(system-supplied\))g(to)e(publish) j(names.)26 b(This)18 b(allo)o(ws)g(arbitrary)e(\015exibili)q(t)o(y)189 1347 y(but)f(is)h(not)e(p)q(ortable.)75 1469 y Fh(3.5.2)49 b(Server)15 b(Routines)75 1554 y Fs(A)j(serv)o(er)f(mak)o(es)g(itself)i (a)o(v)m(ailable)g(with)f(t)o(w)o(o)e(routines.)28 b(First)17 b(it)h(m)o(ust)f(call)i Fk(MPI)p 1543 1554 V 16 w(PORT)p 1681 1554 V 17 w(OPEN)g Fs(to)75 1611 y(establish)14 b(a)e Fk(p)q(o)o(rt)h Fs(at)f(whic)o(h)i(it)f(ma)o(y)f(b)q(e)i(con)o (tacted.)k(Secondly)d(it)e(m)o(ust)f(call)i Fk(MPI)p 1496 1611 V 16 w(A)o(CCEPT)f Fs(to)f(accept)75 1667 y(connections)k (from)f(clien)o(ts.)75 1818 y Fk(MPI)p 160 1818 V 16 w(PORT)p 298 1818 V 17 w(OPEN\(info,)g(p)q(o)o(rt)p 631 1818 V 17 w(name\))117 1895 y Fn(IN)155 b Fk(info)516 b Fn(implemen)o(tation-sp)q(eci\014c)12 b(information)f(on)j(ho)o(w)g (to)f(estab-)905 1952 y(lish)h(an)f(address)i(\(string\))117 2027 y(OUT)108 b Fk(p)q(o)o(rt)p 399 2027 V 17 w(name)390 b Fn(newly)14 b(established)h(p)q(ort)f(\(string\))75 2152 y Fg(int)23 b(MPI)p 245 2152 15 2 v 17 w(Port)p 358 2152 V 17 w(open\(char*)f(info,)i(char)f(port)p 996 2152 V 16 w(name[MPI)p 1204 2152 V 17 w(MAX)p 1293 2152 V 16 w(NAME)p 1405 2152 V 17 w(LEN]\))75 2238 y(MPI)p 150 2238 V 17 w(PORT)p 263 2238 V 16 w(OPEN\(INFO,)g(PORT)p 638 2238 V 17 w(NAME,)g(IERR\))170 2294 y(CHARACTER*\(MPI)p 509 2294 V 16 w(MAX)p 597 2294 V 16 w(NAME)p 709 2294 V 17 w(LEN\))g(PORT)p 941 2294 V 17 w(NAME)170 2351 y(CHARACTER*\(*\))g (INFO)170 2407 y(INTEGER)g(IERR)166 2494 y Fs(This)14 b(function)h(establishes)g(a)f(net)o(w)o(ork)f(address,)h(enco)q(ded)h (in)g(the)f Fk(p)q(o)o(rt)p 1436 2494 14 2 v 16 w(name)g Fs(string,)g(at)f(whic)o(h)75 2550 y(the)20 b(serv)o(er)g(will)h(b)q(e) g(able)f(to)g(accept)g(connections)h(from)e(clien)o(ts.)35 b Fk(p)q(o)o(rt)p 1374 2550 V 17 w(name)19 b Fs(is)i(supplied)h(b)o(y)e (the)75 2607 y(system,)14 b(p)q(ossibly)j(using)f(information)f(in)h (the)g Fk(info)f Fs(argumen)o(t.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 31 33 31 32 bop 75 -100 a Fi(3.5.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fs(31)166 49 y Fk(p)q(o)o(rt)p 247 49 14 2 v 16 w(name)19 b Fs(is)g(a)f (system-supplied)j(string)e(that)f(iden)o(ti\014es)j(the)d(giv)o(en)i (p)q(ort)e(and)h(can)g(b)q(e)g(used)75 106 y(b)o(y)f(a)f(clien)o(t)i (to)f(con)o(tact)f(the)g(serv)o(er.)28 b(The)18 b(maxim)o(um)g(size)g (string)g(that)f(ma)o(y)g(b)q(e)i(supplied)h(b)o(y)e(the)75 162 y(system)d(is)g Fe(MPI)p 348 162 13 2 v 15 w(MAX)p 455 162 V 14 w(NAME)p 587 162 V 16 w(LEN)p Fs(.)166 219 y Fk(p)q(o)o(rt)p 247 219 14 2 v 16 w(name)21 b Fs(is)h(essen)o(tially) h(a)e(net)o(w)o(ork)f(address.)38 b(It)22 b(is)f(unique)i(within)g(the) e(comm)o(unication)75 275 y(univ)o(erse)15 b(to)f(whic)o(h)i(is)f(b)q (elongs)g(\(determined)h(b)o(y)e(the)h(implemen)o(tation\),)g(and)g(ma) o(y)f(b)q(e)h(used)g(b)o(y)f(an)o(y)75 332 y(clien)o(t)22 b(within)f(that)e(comm)o(unication)i(univ)o(erse.)36 b(F)l(or)20 b(instance,)i(if)f(it)f(is)h(an)f(in)o(ternet)h(\(host:p)q (ort\))75 388 y(address,)14 b(it)g(will)i(b)q(e)f(unique)h(on)e(the)g (in)o(ternet.)20 b(If)14 b(it)h(is)g(a)e(lo)o(w)h(lev)o(el)i(switc)o(h) e(address)h(on)f(an)g(IBM)g(SP2)75 444 y(it)j(will)i(b)q(e)e(unique)i (to)d(that)g(SP2.)25 b(The)17 b(precise)h(form)e(of)g(the)h(address)g (is)g(implemen)o(tation-de\014ned.)75 501 y(F)l(or)f(instance,)i(an)e (in)o(ternet)h(address)g(ma)o(y)f(b)q(e)h(a)g(host)f(name)g(or)h(ip)g (address,)g(or)f(an)o(ything)h(that)f(the)75 557 y(implemen)o(tation)g (can)e(deco)q(de)h(in)o(to)f(an)h(ip)g(address.)k(A)c(p)q(ort)e(name)i (ma)o(y)e(b)q(e)i(reused)g(after)e(it)i(is)g(freed)75 614 y(with)h Fk(MPI)p 264 614 V 15 w(PORT)p 401 614 V 17 w(CLOSE)g Fs(and)g(released)g(b)o(y)f(the)g(system.)189 720 y Ff(A)n(dvic)n(e)g(to)j(implementors.)43 b Fs(Since)17 b(the)f(user)g(ma)o(y)f(t)o(yp)q(e)h(in)h Fk(p)q(o)o(rt)p 1346 720 V 16 w(name)f Fs(b)o(y)f(hand,)h(it)h(is)f(useful)189 777 y(to)e(c)o(ho)q(ose)h(a)g(form)g(that)f(is)i(easily)g(readable)g (and)f(do)q(es)h(not)f(ha)o(v)o(e)g(em)o(b)q(edded)h(spaces.)189 852 y(\()p Ff(End)f(of)i(advic)n(e)f(to)g(implementors.)p Fs(\))166 958 y Fk(info)h Fs(is)g(an)g(implemen)o(tation-de\014ned)j (string)c(that)g(ma)o(y)h(b)q(e)g(used)h(to)e(tell)i(the)f(implemen)o (tation)75 1014 y(ho)o(w)12 b(to)g(establish)i(the)f(address.)19 b(It)13 b(ma)o(y)e(\(and)i(usually)h(will\))g(b)q(e)f(an)g(empt)o(y)f (string)h(\(NULL)g(or)f(empt)o(y)75 1071 y(string)g(in)g(C,)f Fg(' ')g Fs(in)i(F)l(ortran\))d(to)h(get)g(the)h(implemen)o(tation)h (defaults.)19 b(Section)12 b(3.5.8)e(giv)o(es)i(examples)75 1127 y(of)j(its)g(p)q(ossible)i(uses.)75 1278 y Fk(MPI)p 160 1278 V 16 w(PORT)p 298 1278 V 17 w(CLOSE\(p)q(o)o(rt)p 550 1278 V 17 w(name\))117 1355 y Fn(IN)155 b Fk(p)q(o)o(rt)p 399 1355 V 17 w(name)390 b Fn(a)14 b(p)q(ort)g(\(string\))75 1480 y Fg(int)23 b(MPI)p 245 1480 15 2 v 17 w(Port)p 358 1480 V 17 w(close\(char*)f(port)p 757 1480 V 17 w(name\))75 1566 y(MPI)p 150 1566 V 17 w(PORT)p 263 1566 V 16 w(CLOSE\(PORT)p 519 1566 V 16 w(NAME,)h(IERR\))170 1623 y(CHARACTER*\(*\))g(PORT)p 604 1623 V 16 w(NAME)170 1679 y(INTEGER)g(IERR)166 1766 y Fs(This)16 b(function)h(releases)f(the)g(net)o(w)o(ork)e(address)i (represen)o(ted)g(b)o(y)g Fk(p)q(o)o(rt)p 1416 1766 14 2 v 16 w(name)p Fs(.)21 b(If)16 b(there)g(are)f(an)o(y)75 1822 y(p)q(ending)h Fk(MPI)p 332 1822 V 16 w(IA)o(CCEPT)e Fs(op)q(erations)g(on)g(the)g(p)q(ort,)g(the)g(p)q(ort)g(is)h(mark)o (ed)f(for)f(deletion)j(but)e(it)h(is)f(not)75 1879 y(actually)i (released)g(un)o(til)g(the)g(op)q(erations)f(complete)h(or)f(are)g (cancelled.)75 2030 y Fk(MPI)p 160 2030 V 16 w(A)o(CCEPT\(p)q(o)o(rt)p 446 2030 V 16 w(name,)g(info,)f(ro)q(ot,)h(comm,)e(new)o(comm\))117 2107 y Fn(IN)155 b Fk(p)q(o)o(rt)p 399 2107 V 17 w(name)390 b Fn(p)q(ort)14 b(name)f(\(string,)h(used)h(only)e(on)g Fe(ro)q(ot)p Fn(\))117 2182 y(IN)155 b Fk(info)516 b Fn(implemen)o(tation-dep)q(enden)o(t)14 b(information)f(\(handle,)i (used)905 2238 y(only)e(on)h Fe(ro)q(ot)p Fn(\))117 2314 y(IN)155 b Fk(ro)q(ot)508 b Fn(rank)14 b(in)f Fe(comm)f Fn(of)i(ro)q(ot)g(no)q(de)g(\(in)o(teger\))117 2389 y(IN)155 b Fk(comm)470 b Fn(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o(h)g(call)f (is)h(collectiv)o(e)g(\(handle\))117 2464 y(OUT)108 b Fk(new)o(comm)397 b Fn(in)o(tercomm)o(unicator)12 b(with)i(clien)o(t)h (as)g(remote)f(group)g(\(han-)905 2520 y(dle\))75 2645 y Fg(int)23 b(MPI)p 245 2645 15 2 v 17 w(Accept\(char*)f(port,)i(MPI)p 788 2645 V 16 w(Info)g(info,)f(int)g(root,)h(MPI)p 1378 2645 V 16 w(Comm)g(comm,)393 2701 y(MPI)p 468 2701 V 17 w(Comm*)f(newcomm\))-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 32 34 32 33 bop 75 -100 a Fs(32)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)75 49 y Fg(MPI)p 150 49 15 2 v 17 w(ACCEPT\(PORT)p 431 49 V 15 w(NAME,)24 b(ROOT,)f(COMM,)g(NEWCOMM,)g(IERR\))170 106 y(CHARACTER*\(*\))g(PORT)p 604 106 V 16 w(NAME)170 162 y(INTEGER)g(COMM,)h(ROOT,)f(NEWCOMM,)f(IERR)166 249 y Fk(MPI)p 251 249 14 2 v 16 w(A)o(CCEPT)15 b Fs(establishes)i(comm)o (unication)f(with)g(a)f(clien)o(t.)21 b(It)15 b(is)h(collectiv)o(e)h(o) o(v)o(er)e(the)g(calling)75 305 y(comm)o(unicator.)k(It)12 b(returns)g(an)h(in)o(tercomm)o(unicator)f(that)g(allo)o(ws)h(comm)o (unication)g(with)g(the)f(clien)o(t.)166 362 y(The)j Fk(p)q(o)o(rt)p 340 362 V 17 w(name)g Fs(m)o(ust)f(ha)o(v)o(e)h(b)q (een)h(established)h(through)e(a)g(call)h(to)f Fk(MPI)p 1462 362 V 16 w(PORT)p 1600 362 V 17 w(OPEN)p Fs(.)166 418 y Fk(info)i Fs(is)h(a)f(implemen)o(tation-de\014ned)j(string)e (that)e(ma)o(y)h(allo)o(w)h(\014ne)g(con)o(trol)f(o)o(v)o(er)f(the)i Fk(A)o(CCEPT)75 474 y Fs(call.)j(See)16 b(section)g Fj(??)e Fs(for)h(examples.)75 596 y Fh(3.5.3)49 b(Client)17 b(Routines)75 682 y Fs(There)e(is)h(only)g(one)f(routine)h(on)f(the)g(clien)o(t)i (side.)75 833 y Fk(MPI)p 160 833 V 16 w(CONNECT\(p)q(o)o(rt)p 485 833 V 17 w(name,)d(info,)h(ro)q(ot,)g(comm,)e(new)o(comm\))117 910 y Fn(IN)155 b Fk(p)q(o)o(rt)p 399 910 V 17 w(name)390 b Fn(net)o(w)o(ork)14 b(address)h(\(string,)f(used)h(only)e(on)h Fe(ro)q(ot)p Fn(\))117 985 y(IN)155 b Fk(info)516 b Fn(implemen)o (tation-dep)q(enden)o(t)19 b(information)e(\(string,)22 b(used)905 1042 y(only)13 b(on)h Fe(ro)q(ot)p Fn(\))117 1117 y(IN)155 b Fk(ro)q(ot)508 b Fn(rank)14 b(in)f Fe(comm)f Fn(of)i(ro)q(ot)g(no)q(de)g(\(in)o(teger\))117 1192 y(IN)155 b Fk(comm)470 b Fn(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o(h)g(call)f (is)h(collectiv)o(e)g(\(string\))117 1267 y(OUT)108 b Fk(new)o(comm)397 b Fn(in)o(tercomm)o(unicator)11 b(with)i(serv)o(er)h (as)f(remote)g(group)g(\(han-)905 1324 y(dle\))75 1448 y Fg(int)23 b(MPI)p 245 1448 15 2 v 17 w(Connect\(char*)f(port)p 692 1448 V 17 w(name,)h(char*)g(info,)g(int)h(root,)f(MPI)p 1449 1448 V 17 w(Comm)g(comm,)393 1504 y(MPI)p 468 1504 V 17 w(Comm*)g(newcomm\))75 1591 y(MPI)p 150 1591 V 17 w(CONNECT\(PORT)p 455 1591 V 15 w(NAME,)g(INFO,)h(ROOT,)f(COMM,)g (NEWCOMM,)g(IERR\))170 1647 y(CHARACTER*\(*\))g(PORT)p 604 1647 V 16 w(NAME,)g(INFO)170 1704 y(INTEGER)g(ROOT,)h(COMM,)f (NEWCOMM,)f(IERR)166 1790 y Fs(This)g(routine)f(establishes)i(comm)o (unication)f(with)f(a)g(serv)o(er)g(sp)q(eci\014ed)i(b)o(y)e Fk(p)q(o)o(rt)p 1623 1790 14 2 v 17 w(name)p Fs(.)37 b(It)21 b(is)75 1847 y(collectiv)o(e)g(o)o(v)o(er)d(the)h(calling)i (comm)o(unicator)e(and)g(returns)g(an)g(in)o(tercomm)o(unicator)g(in)h (whic)o(h)g(the)75 1903 y(remote)15 b(group)g(participated)h(in)g(an)f Fk(MPI)p 812 1903 V 16 w(A)o(CCEPT)p Fs(.)166 1960 y(If)j(the)g(named)h (p)q(ort)f(do)q(es)g(not)g(exist)g(\(or)f(has)h(b)q(een)i(closed\))e Fk(MPI)p 1357 1960 V 16 w(CONNECT)h Fs(returns)f(a)g(soft)75 2016 y(error)c(of)h(class)h Fk(MPI)p 430 2016 V 16 w(ERR)p 531 2016 V 17 w(PORT)g Fs(and)f(sets)g Fk(new)o(comm)f Fs(to)g Fe(MPI)p 1196 2016 13 2 v 15 w(COMM)p 1341 2016 V 14 w(NULL)p Fs(.)166 2073 y(If)23 b(the)f(p)q(ort)h(exists,)h(but)f (do)q(es)g(not)f(ha)o(v)o(e)g(a)h(p)q(ending)h Fk(A)o(CCEPT)p Fs(,)f(the)f(connection)i(attempt)75 2129 y(will)18 b(ev)o(en)o(tually) f(time)f(out)g(after)f(an)h(implemen)o(tation-de\014ned)j(time,)e(or)e (succeed)j(when)e(the)g(serv)o(er)75 2185 y(calls)24 b Fk(A)o(CCEPT)p Fs(.)e(In)i(the)f(case)f(of)h(a)f(time)h(out,)h Fk(MPI)p 1051 2185 14 2 v 16 w(CONNECT)f Fs(returns)g(a)f(soft)g(error) g(of)h(class)75 2242 y Fk(MPI)p 160 2242 V 16 w(ERR)p 261 2242 V 17 w(PORT)16 b Fs(and)f(sets)g Fk(new)o(comm)f Fs(to)h Fe(MPI)p 927 2242 13 2 v 14 w(COMM)p 1071 2242 V 15 w(NULL)p Fs(.)166 2346 y Fo(Discussion:)34 b Fn(The)15 b(in)o(ten)o(t)f(is)g(for)g(timed-out)e(of)i(refused)h(connections)g (to)f(return)h(a)f(\\soft")f(error)i(-)f(not)75 2402 y(fatal)f(b)o(y)g(default.)18 b(Not)c(sure)h(what)f(is)f(the)i (standard)f(w)o(a)o(y)f(for)h(MPI)g(to)g(handle)g(this.)k(-)13 b(w)o(cs)189 2556 y Ff(A)n(dvic)n(e)k(to)i(implementors.)54 b Fs(The)18 b(time-out)g(p)q(erio)q(d)g(ma)o(y)f(b)q(e)i(arbitrarily)f (short)f(or)g(long.)27 b(A)189 2612 y(high)16 b(qualit)o(y)g(implemen)o (tation)h(will)g(ensure)f(that)f(a)g Fk(CONNECT)i Fs(will)g(usually)g (succeed)g(if)f(the)189 2669 y Fk(A)o(CCEPT)k Fs(happ)q(ens)h(at)e (\\ab)q(out)h(the)g(same)g(time.")35 b(A)20 b(high)h(qualit)o(y)f (implemen)o(tation)i(ma)o(y)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 33 35 33 34 bop 75 -100 a Fi(3.5.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fs(33)189 49 y(also)i(pro)o(vide)i(a)e(mec)o(hanism,)h(through)g(the)g Fk(info)f Fs(argumen)o(ts)g(to)g Fk(PORT)p 1451 49 14 2 v 18 w(OPEN)p Fs(,)h Fk(A)o(CCEPT)g Fs(and)189 106 y Fk(CONNECT)p Fs(,)h(for)g(the)g(user)h(to)e(sp)q(ecify)j(the)e(b)q (eha)o(vior.)20 b(\()p Ff(End)c(of)g(advic)n(e)g(to)h(implementors.)p Fs(\))166 193 y Fk(p)q(o)o(rt)p 247 193 V 16 w(name)g Fs(is)g(the)g(address)g(of)f(the)h(serv)o(er.)24 b(It)17 b(m)o(ust)f(b)q(e)i(the)e(same)h(as)f(the)h(name)g(returned)g(b)o(y)75 249 y Fk(MPI)p 160 249 V 16 w(PORT)p 298 249 V 17 w(OPEN)k Fs(on)f(the)g(serv)o(er.)34 b(Some)21 b(freedom)f(is)g(allo)o(w)o(ed)h (here.)35 b(If)20 b(there)g(are)g(equiv)m(alen)o(t)75 305 y(forms)13 b(of)g Fk(p)q(o)o(rt)p 330 305 V 16 w(name)p Fs(,)h(an)f(implemen)o(tation)i(ma)o(y)e(accept)h(them)f(as)g(w)o(ell.) 21 b(F)l(or)12 b(instance,)j(if)f Fk(p)q(o)o(rt)p 1760 305 V 16 w(name)75 362 y Fs(is)i(\(hostname:p)q(ort\),)d(an)i(implemen) o(tation)h(ma)o(y)f(accept)g(\(ip)p 1156 362 V 17 w(address:p)q(ort\))g (as)f(w)o(ell.)75 481 y Fh(3.5.4)49 b(Name)16 b(publishing)75 566 y Fs(The)i(routines)g(in)h(this)f(section)g(pro)o(vide)h(a)e(mec)o (hanism)h(for)g(publishing)i(names.)28 b(A)17 b(\()p Fk(service)p 1747 566 V 17 w(name)p Fs(,)75 623 y Fk(p)q(o)o(rt)p 156 623 V 17 w(name)p Fs(\))j(pair)i(is)g(published)j(b)o(y)c(the)h (serv)o(er,)g(and)g(ma)o(y)f(b)q(e)h(retriev)o(ed)g(b)o(y)f(a)h(clien)o (t)h(using)f(the)75 679 y Fk(service)p 203 679 V 17 w(name)17 b Fs(only)l(.)29 b(An)18 b(MPI)f(implemen)o(tation)i(de\014nes)g(the)f Ff(sc)n(op)n(e)f Fs(of)g(the)h Fk(service)p 1584 679 V 17 w(name)p Fs(,)g(that)f(is,)75 736 y(the)g(domain)g(o)o(v)o(er)f (whic)o(h)i(the)e Fk(service)p 756 736 V 17 w(name)h Fs(can)f(b)q(e)i(retriev)o(ed.)25 b(If)17 b(the)g(domain)g(is)g(the)g (empt)o(y)g(set,)75 792 y(that)f(is,)i(if)g(no)f(clien)o(t)h(can)f (retriev)o(e)g(the)h(information,)f(then)g(w)o(e)g(sa)o(y)f(that)h (name)g(publishing)j(is)d(not)75 849 y(supp)q(orted.)1589 b Fm(>)16 b Fl(\(Sept\))166 905 y Fs(If)10 b(name)g(publishin)q(g)i(is) f(not)f(supp)q(orted,)h(these)g(routines)f(return)g(the)g(soft)g(error) f(class)i Fe(MPI)p 1732 905 13 2 v 14 w(ERR)p 1825 905 V 14 w(NOPUBLISHING)p Fs(.)1875 962 y Fm(?)16 b Fl(\(Sept\))1875 1020 y Fm(>)g Fl(\(Sept\))75 1113 y Fk(MPI)p 160 1113 14 2 v 16 w(NAME)p 305 1113 V 17 w(PUBLISH\(p)q(o)o(rt)p 603 1113 V 16 w(name,)e(info,)h(service)p 972 1113 V 17 w(name\))117 1190 y Fn(IN)155 b Fk(p)q(o)o(rt)p 399 1190 V 17 w(name)390 b Fn(a)14 b(p)q(ort)g(name)f(\(string\))117 1262 y(IN)155 b Fk(info)516 b Fn(implemen)o(tation-sp)q(eci\014c)12 b(information)f(\(handle\))117 1334 y(IN)155 b Fk(service)p 446 1334 V 17 w(name)343 b Fn(a)14 b(service)h(name)e(to)h(asso)q (ciate)g(with)g(the)g(p)q(ort)g(\(string\))75 1458 y Fg(int)23 b(MPI)p 245 1458 15 2 v 17 w(NAME)p 358 1458 V 17 w(PUBLISH\(char*)f(port)p 805 1458 V 17 w(name,)h(MPI)p 1037 1458 V 17 w(Info)g(info,)g(char*)g(service)p 1627 1458 V 16 w(name\))75 1544 y(MPI)p 150 1544 V 17 w(NAME)p 263 1544 V 16 w(PUBLISH\(PORT)p 567 1544 V 16 w(NAME,)g(INFO,)g (SERVICE)p 1037 1544 V 17 w(NAME,)g(IERR\))170 1601 y(INTEGER)g(INFO,)h (IERR)170 1657 y(CHARACTER*\(*\))f(PORT)p 604 1657 V 16 w(NAME,)g(SERVICE)p 931 1657 V 17 w(NAME)1875 1694 y Fm(?)16 b Fl(\(Sept\))166 1744 y Fs(This)i(routine)g(publishes)i(the) d(pair)h(\()p Fk(p)q(o)o(rt)p 906 1744 14 2 v 16 w(name)p Fs(,)g Fk(service)p 1180 1744 V 16 w(name)p Fs(\))f(so)g(that)g(an)h (application)h(ma)o(y)75 1800 y(retriev)o(e)c(a)g(system-supplied)j Fk(p)q(o)o(rt)p 689 1800 V 16 w(name)d Fs(using)h(a)f(w)o(ell-kno)o(wn) h Fk(service)p 1340 1800 V 17 w(name)p Fs(.)166 1857 y(The)j(implemen)o(tation)i(m)o(ust)e(de\014ne)h(the)g Ff(sc)n(op)n(e)e Fs(of)h(a)g(published)j(service)e(name,)g(that)e(is,)j (the)75 1913 y(domain)16 b(o)o(v)o(er)g(whic)o(h)h(the)f(service)h (name)f(is)g(unique,)i(and)e(con)o(v)o(ersely)l(,)g(the)g(domain)h(o)o (v)o(er)e(whic)o(h)i(the)75 1970 y(\(p)q(ort)i(name,)i(service)g (name\))e(pair)h(ma)o(y)g(b)q(e)g(retriev)o(ed.)34 b(F)l(or)20 b(instance,)h(a)f(service)g(name)g(ma)o(y)f(b)q(e)75 2026 y(unique)c(to)d(a)h(job)g(\(where)h(job)f(is)h(de\014ned)g(b)o(y)f (a)g(distributed)i(op)q(erating)f(system)e(or)h(batc)o(h)g(sc)o (heduler\),)75 2083 y(unique)20 b(to)e(a)g(mac)o(hine,)i(or)e(unique)i (to)e(a)g(k)o(erb)q(eros)h(realm.)30 b(The)19 b(scop)q(e)g(ma)o(y)f (dep)q(end)i(on)e(the)h Fk(info)p Fm(>)d Fl(\(Sept\))75 2139 y Fs(argumen)o(t)e(to)h Fk(MPI)p 419 2139 V 16 w(NAME)p 564 2139 V 17 w(PUBLISH)p Fs(.)1875 2141 y Fm(?)h Fl(\(Sept\))166 2195 y Fs(It)k(is)g(allo)o(w)o(ed)g(to)g(publish)i(more)d(than)h(one)g Fk(service)p 1106 2195 V 16 w(name)g Fs(for)f(a)g(single)j Fk(p)q(o)o(rt)p 1570 2195 V 16 w(name)p Fs(.)33 b(On)21 b(the)75 2252 y(other)16 b(hand,)i(if)f Fk(service)p 495 2252 V 17 w(name)f Fs(has)h(already)g(b)q(een)g(established)i (within)f(the)f(curren)o(t)f(scop)q(e,)i(the)e(de-)75 2308 y(fault)22 b(b)q(eha)o(vior)g(is)h(that)e Fk(MPI)p 624 2308 V 16 w(PUBLISH)p 825 2308 V 16 w(NAME)i Fs(fails)f(and)g (returns)g(a)g(soft)f(error)g(of)g(error)h(class)75 2365 y Fk(MPI)p 160 2365 V 16 w(ERR)p 261 2365 V 17 w(SERVICE)p Fs(.)c(An)g(MPI)f(implemen)o(tation)i(ma)o(y)l(,)e(through)g(an)g(o)o (v)o(erride)h(mec)o(hanism)g(in)g(the)75 2421 y Fk(info)d Fs(argumen)o(t)f(to)g Fk(MPI)p 504 2421 V 16 w(OPEN)p 641 2421 V 17 w(PORT)p Fs(,)h(pro)o(vide)h(a)e(w)o(a)o(y)g(to)g(allo)o (w)h(m)o(ultiple)i(serv)o(ers)d(with)h(the)g(same)75 2478 y(service)i(in)g(the)f(same)f(name)h(space.)23 b(In)16 b(this)h(case,)f(an)g(implemen)o(tation-de\014ned)i(p)q(olicy)g(will)g (deter-)75 2534 y(mine)e(to)f(whic)o(h)h(of)e(sev)o(eral)i(serv)o(ers)f (a)f(clien)o(t)j(is)f(connected)g(\(see)f(examples\).)166 2591 y(Note)h(that)g(while)i Fk(service)p 625 2591 V 16 w(name)f Fs(has)f(a)g(limited)i(scop)q(e,)f(determined)h(b)o(y)e (the)h(implemen)o(tation,)75 2647 y Fk(p)q(o)o(rt)p 156 2647 V 17 w(name)f Fs(alw)o(a)o(ys)h(has)g(global)g(scop)q(e)h(within)g (the)f(comm)o(unication)h(univ)o(erse)g(used)g(b)o(y)f(the)g(imple-)75 2704 y(men)o(tation)e(\(i.e.,)f(it)i(is)g(globally)g(unique\).)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 34 36 34 35 bop 75 -100 a Fs(34)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)166 49 y Fs(If)j Fk(p)q(o)o(rt)p 295 49 14 2 v 16 w(name)g Fs(should)h(b)q(e)f(the)g(name)g(of)f(a)g(p)q(ort)h(established)h(b)o (y)f Fk(MPI)p 1427 49 V 16 w(PORT)p 1565 49 V 17 w(OPEN)g Fs(and)g(not)75 106 y(y)o(et)d(deleted)h(b)o(y)f Fk(MPI)p 455 106 V 16 w(PORT)p 593 106 V 17 w(CLOSE)p Fs(.)h(If)f(it)h(is)f (not,)g(the)g(results)h(are)e(unde\014ned.)189 212 y Ff(A)n(dvic)n(e)k(to)h(implementors.)60 b Fs(In)19 b(some)f(cases,)h (an)f(MPI)h(implemen)o(tation)g(ma)o(y)f(use)h(a)f(name)189 268 y(service)g(that)g(a)f(user)h(can)g(also)g(access)g(directly)l(.)30 b(In)18 b(this)h(case,)f(a)g(name)g(published)i(b)o(y)e(MPI)189 325 y(could)c(easily)g(con\015ict)g(with)f(a)g(name)g(published)j(b)o (y)d(a)g(user.)19 b(In)14 b(order)f(to)f(a)o(v)o(oid)h(suc)o(h)h (con\015icts,)189 381 y(MPI)d(implemen)o(tations)h(should)g(mangle)f (service)h(names)f(so)g(that)f(they)h(are)f(unlik)o(ely)k(to)c (con\015ict)189 438 y(with)j(user)g(co)q(de)h(that)e(mak)o(es)g(use)i (of)e(the)h(same)g(service.)20 b(Suc)o(h)14 b(name)f(mangling)h(will)g (of)f(course)189 494 y(b)q(e)j(completely)g(transparen)o(t)e(to)h(the)g (user.)189 569 y(The)d(follo)o(wing)i(situation)f(is)g(problematic)g (but)g(una)o(v)o(oidable,)h(if)f(w)o(e)f(w)o(an)o(t)f(to)h(allo)o(w)h (implemen-)189 626 y(tations)i(to)g(use)h(nameserv)o(ers.)k(Supp)q(ose) d(there)e(are)h(m)o(ultiple)h(instances)f(of)f(\\o)q(cean")h(running) 189 682 y(on)21 b(a)g(mac)o(hine.)38 b(If)22 b(the)f(scop)q(e)h(of)f(a) g(service)h(name)f(is)h(con\014ned)g(to)f(a)f(job,)j(then)e(m)o (ultiple)189 739 y(o)q(ceans)15 b(can)h(co)q(exist.)k(If)c(an)f (implemen)o(tation)h(pro)o(vides)g(site-wide)h(scop)q(e,)e(ho)o(w)o(ev) o(er,)f(m)o(ultiple)189 795 y(instances)i(are)f(not)g(p)q(ossible)i(as) f(all)g(calls)h(to)e Fk(MPI)p 1075 795 V 15 w(PUBLISH)p 1275 795 V 17 w(NAME)h Fs(after)f(the)g(\014rst)g(will)j(fail.)189 852 y(There)d(is)h(no)f(go)q(o)q(d)g(solution)h(to)f(this.)189 927 y(T)l(o)g(handle)h(these)g(situations,)f(a)g(high)h(qualit)o(y)g (implemen)o(tation)g(should)g(mak)o(e)f(it)h(p)q(ossible)h(to)189 983 y(limit)f(the)g(domain)f(o)o(v)o(er)f(whic)o(h)j(names)e(are)g (published.)22 b(\()p Ff(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Fs(\))75 1184 y Fk(MPI)p 160 1184 V 16 w(NAME)p 305 1184 V 17 w(DELETE\(p)q(o)o(rt)p 588 1184 V 16 w(name,)d(service)p 858 1184 V 17 w(name\))117 1261 y Fn(IN)155 b Fk(p)q(o)o(rt)p 399 1261 V 17 w(name)390 b Fn(a)14 b(p)q(ort)g(name)f(\(string\))117 1336 y(IN)155 b Fk(service)p 446 1336 V 17 w(name)343 b Fn(a)14 b(service)h(name)e(\(string\))75 1461 y Fg(int)23 b(MPI)p 245 1461 15 2 v 17 w(Name)p 358 1461 V 17 w(delete\(char*)f (port)p 781 1461 V 17 w(name,)h(char*)g(service)p 1252 1461 V 16 w(name\))75 1547 y(MPI)p 150 1547 V 17 w(NAME)p 263 1547 V 16 w(DELETE\(PORT)p 543 1547 V 16 w(NAME,)g(SERVICE)p 870 1547 V 16 w(NAME,)h(IERR\))170 1604 y(INTEGER)f(IERR)170 1660 y(CHARACTER*\(*\))g(PORT)p 604 1660 V 16 w(NAME,)g(SERVICE)p 931 1660 V 17 w(NAME)166 1747 y Fs(This)14 b(routine)g(deletes)g(a)f (service)i(name)e(that)g(has)g(b)q(een)i(previously)g(published.)21 b(A)o(ttempting)14 b(to)75 1803 y(delete)i(a)f(name)g(that)g(has)g(not) g(b)q(een)h(published)i(is)e(erroneous,)e(and)i(is)f(indicated)i(b)o(y) f(the)f(error)f(co)q(de)75 1860 y Fk(MPI)p 160 1860 14 2 v 16 w(ERR)p 261 1860 V 17 w(SERVICE)p Fs(.)166 1916 y(All)22 b(published)i(names)c(should)i(b)q(e)g(deleted)g(b)q(efore)f (the)g(corresp)q(onding)h(p)q(ort)f(is)g(closed)h(and)75 1972 y(b)q(efore)15 b(the)h(publishing)i(pro)q(cess)d(exits.)-890 b Fm(>)15 b Fl(\(Sept\))166 2029 y Fs(The)f(b)q(eha)o(vior)g(of)g Fk(MPI)p 578 2029 V 15 w(NAME)p 722 2029 V 17 w(DELETE)g Fs(when)g(a)f(pro)q(cess)i(tries)e(to)h(delete)h(a)e(name)h(that)f(it)h (did)75 2085 y(not)h(publish)i(is)f(implemen)o(tation)g(dep)q(enden)o (t.)-1018 b Fm(?)15 b Fl(\(Sept\))-117 2144 y Fm(>)g Fl(\(Sept\))189 2192 y Ff(A)n(dvic)n(e)i(to)h(implementors.)51 b Fs(Because)18 b(the)f(mec)o(hanism)h(for)e(name)h(publishing)j(ma)o (y)c(dep)q(end)189 2248 y(on)d(the)h Fk(info)f Fs(argumen)o(t)g(to)g Fk(MPI)p 752 2248 V 16 w(NAME)p 897 2248 V 16 w(PUBLISH)p Fs(,)h(and)g(indirectly)h(on)f(the)f Fk(info)h Fs(argumen)o(t)f(to)189 2305 y Fk(MPI)p 274 2305 V 15 w(PORT)p 411 2305 V 18 w(OPEN)p Fs(,)i(it)g(ma)o(y)f(b)q(e)h(necessary)g(to)f(cac)o(he)h (publishing)j(information)d(in)o(ternally)h(on)189 2361 y Fk(p)q(o)o(rt)p 270 2361 V 16 w(name)g Fs(or)g Fk(service)p 586 2361 V 17 w(name)g Fs(so)g(that)f(the)i(name)f(can)h(b)q(e)g(prop)q (erly)g(deleted.)25 b(\()p Ff(End)17 b(of)g(advic)n(e)189 2417 y(to)f(implementors.)p Fs(\))-117 2479 y Fm(?)f Fl(\(Sept\))-117 2537 y Fm(>)g Fl(\(Sept\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 35 37 35 36 bop 75 -100 a Fi(3.5.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fs(35)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(NAME)p 305 49 V 17 w(GET\(service)p 553 49 V 16 w(name,)k(info,)g(p)q(o)o(rt)p 876 49 V 16 w(name\))117 126 y Fn(IN)155 b Fk(service)p 446 126 V 17 w(name)343 b Fn(a)14 b(service)h(name)e(\(string\))117 202 y(IN)155 b Fk(info)516 b Fn(implemen)o(tation-sp)q(eci\014c)12 b(information)f(\(handle\))117 277 y(OUT)108 b Fk(p)q(o)o(rt)p 399 277 V 17 w(name)390 b Fn(a)14 b(p)q(ort)g(name)f(\(string\))75 401 y Fg(int)23 b(MPI)p 245 401 15 2 v 17 w(Name)p 358 401 V 17 w(get\(char*)f(service)p 781 401 V 17 w(name,)h(MPI)p 1013 401 V 17 w(Info)g(info,)393 458 y(char)g(port)p 611 458 V 17 w(name[MPI)p 820 458 V 16 w(MAX)p 908 458 V 17 w(NAME)p 1021 458 V 17 w(LEN]\))75 544 y(MPI)p 150 544 V 17 w(NAME)p 263 544 V 16 w(GET\(SERVICE)p 543 544 V 16 w(NAME,)g(INFO,)g(PORT)p 941 544 V 17 w(NAME,)g(IERR\))170 601 y(CHARACTER*\(*\))g(SERVICE)p 676 601 V 16 w(NAME)170 657 y(CHARACTER\(*\)\(MPI)p 557 657 V 15 w(MAX)p 644 657 V 17 w(NAME)p 757 657 V 17 w(LEN\))g(PORT)p 989 657 V 17 w(NAME)170 713 y(INTEGER)g(INFO,)h(IERR)1875 750 y Fm(?)16 b Fl(\(Sept\))166 800 y Fs(This)11 b(function)g(retriev)o(es) f(a)g Fk(p)q(o)o(rt)p 729 800 14 2 v 16 w(name)g Fs(published)j(b)o(y)d Fk(MPI)p 1197 800 V 16 w(NAME)p 1342 800 V 16 w(PUBLISH)h Fs(with)f Fk(service)p 1777 800 V 17 w(name)p Fs(.)75 856 y(If)h Fk(service)p 244 856 V 16 w(name)g Fs(has)f(not)g(b)q(een)i (published,)h(it)e(returns)f(a)h(soft)e(error)h(with)h(error)f(class)h Fe(MPI)p 1639 856 13 2 v 14 w(ERR)p 1732 856 V 14 w(NAME)p Fs(.)166 913 y(If)21 b(an)f(implemen)o(tation)i(allo)o(ws)f(m)o (ultiple)h(en)o(tries)f(with)g(the)f(same)g Fk(service)p 1529 913 14 2 v 17 w(name)g Fs(within)i(the)75 969 y(same)13 b(domain,)h(a)f(particular)h Fk(p)q(o)o(rt)p 685 969 V 17 w(name)f Fs(is)h(c)o(hosen)f(in)i(a)e(w)o(a)o(y)f(determined)j(b)o (y)e(the)h(implemen)o(tation.)q Fm(>)i Fl(\(Sept\))166 1026 y Fs(If)j(the)f Fk(info)g Fs(argumen)o(t)g(w)o(as)g(used)h(with)g Fk(MPI)p 982 1026 V 15 w(NAME)p 1126 1026 V 17 w(PUBLISH)g Fs(to)f(tell)h(the)g(implemen)o(tation)75 1082 y(ho)o(w)c(to)g(publish) i(names,)e(a)g(similar)h Fk(info)g Fs(argumen)o(t)e(ma)o(y)h(b)q(e)h (required)g(to)f Fk(MPI)p 1494 1082 V 16 w(NAME)p 1639 1082 V 17 w(GET)g Fs(to)g(tell)75 1139 y(the)g(information)h(ho)o(w)e (to)h(retriev)o(e)g(names.)1024 b Fm(?)16 b Fl(\(Sept\))75 1260 y Fh(3.5.5)49 b(Non-blo)q(cking)19 b(routines)75 1441 y Fk(MPI)p 160 1441 V 16 w(IA)o(CCEPT\(p)q(o)o(rt)p 459 1441 V 16 w(name,)14 b(info,)h(ro)q(ot,)f(comm,)f(new)o(comm,)h (request\))117 1518 y Fn(IN)155 b Fk(p)q(o)o(rt)p 399 1518 V 17 w(name)390 b Fn(p)q(ort)14 b(name)f(\(string,)h(used)h(only)e (on)g Fe(ro)q(ot)p Fn(\))117 1593 y(IN)155 b Fk(info)516 b Fn(implemen)o(tation-dep)q(enden)o(t)19 b(information)e(\(string,)22 b(used)905 1649 y(only)13 b(on)h Fe(ro)q(ot)p Fn(\))117 1725 y(IN)155 b Fk(ro)q(ot)508 b Fn(rank)14 b(in)f Fe(comm)f Fn(of)i(ro)q(ot)g(no)q(de)g(\(in)o(teger\))117 1800 y(IN)155 b Fk(comm)470 b Fn(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o(h)g(call)f (is)h(collectiv)o(e)g(\(string\))117 1875 y(OUT)108 b Fk(new)o(comm)397 b Fn(in)o(tercomm)o(unicator)12 b(with)i(clien)o(t)h (as)g(remote)f(group)g(\(han-)905 1931 y(dle\))117 2006 y(OUT)108 b Fk(request)452 b Fn(request)16 b(ob)r(ject)e(\(handle\))75 2131 y Fg(int)23 b(MPI)p 245 2131 15 2 v 17 w(Iaccept\(char*)f(port)p 692 2131 V 17 w(name,)h(char)g(*)h(info,)f(int)h(root,)f(MPI)p 1473 2131 V 17 w(Comm)g(comm,)393 2187 y(MPI)p 468 2187 V 17 w(Comm*)g(newcomm,)g(MPI)p 915 2187 V 17 w(Request*)f(request\))75 2274 y(MPI)p 150 2274 V 17 w(IACCEPT\(PORT)p 455 2274 V 15 w(NAME,)h(ROOT,)h(COMM,)f(NEWCOMM,)g(REQUEST,)f(IERR\))170 2330 y(CHARACTER*\(*\))h(PORT)p 604 2330 V 16 w(NAME)170 2387 y(INTEGER)g(ROOT,)h(COMM,)f(NEWCOMM,)f(REQUEST,)h(IERR)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 36 38 36 37 bop 75 -100 a Fs(36)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)75 49 y Fk(MPI)p 160 49 14 2 v 16 w(ICONNECT\(p)q(o)o(rt)p 498 49 V 17 w(name,)f(info,)h(ro)q(ot,)f(comm,)f(new)o(comm,)h (request\))117 126 y Fn(IN)155 b Fk(p)q(o)o(rt)p 399 126 V 17 w(name)390 b Fn(net)o(w)o(ork)14 b(address)h(\(string,)f(used) h(only)e(on)h Fe(ro)q(ot)p Fn(\))117 202 y(IN)155 b Fk(info)516 b Fn(implemen)o(tation-dep)q(enden)o(t)19 b(information)e(\(string,)22 b(used)905 258 y(only)13 b(on)h Fe(ro)q(ot)p Fn(\))117 333 y(IN)155 b Fk(ro)q(ot)508 b Fn(rank)14 b(in)f Fe(comm)f Fn(of)i(ro)q(ot)g(no)q(de)g(\(in)o(teger\))117 408 y(IN)155 b Fk(comm)470 b Fn(comm)o(unicator)11 b(o)o(v)o(er)j(whic)o(h)g(call)f (is)h(collectiv)o(e)g(\(string\))117 483 y(OUT)108 b Fk(new)o(comm)397 b Fn(in)o(tercomm)o(unicator)11 b(with)i(serv)o(er)h (as)f(remote)g(group)g(\(han-)905 540 y(dle\))117 615 y(OUT)108 b Fk(request)452 b Fn(request)16 b(ob)r(ject)e(\(handle\))75 740 y Fg(int)23 b(MPI)p 245 740 15 2 v 17 w(Iconnect\(char*)f(port)p 716 740 V 17 w(name,)h(char*)g(info,)g(int)h(root,)f(MPI)p 1473 740 V 17 w(Comm)g(comm,)393 796 y(MPI)p 468 796 V 17 w(Comm*)g(newcomm,)g(MPI)p 915 796 V 17 w(Request*)f(request\))75 882 y(MPI)p 150 882 V 17 w(ICONNECT\(PORT)p 479 882 V 15 w(NAME,)h(INFO,)h(ROOT,)f(COMM,)g(NEWCOMM,)g(REQUEST,)f(IERR\))170 939 y(CHARACTER*\(*\))h(PORT)p 604 939 V 16 w(NAME,)g(INFO)170 995 y(INTEGER)g(ROOT,)h(COMM,)f(NEWCOMM,)f(REQUEST,)h(IERR)166 1082 y Fs(These)c(functions)g(are)g(iden)o(tical)h(to)e(their)h(blo)q (c)o(king)h(coun)o(terparts)e(except)h(that)f(they)h(do)f(not)75 1138 y(blo)q(c)o(k.)i(An)13 b(application)h(ma)o(y)e(test)g(or)h(w)o (ait)f(for)g(completion)i(using)f Fk(MPI)p 1347 1138 14 2 v 16 w(W)l(AIT)p Fs(,)g Fk(MPI)p 1584 1138 V 15 w(TEST)p Fs(,)g(and)g(re-)75 1195 y(lated)e(routines.)18 b(These)11 b(non-blo)q(c)o(king)h(collectiv)o(e)g(call)f(ob)q(ey)f(the) h(same)e(seman)o(tics)i(as)e(the)i Fk(MPI)p 1731 1195 V 15 w(ISP)l(A)-5 b(WN)p 1909 1195 V 17 w(*)75 1251 y Fs(calls)13 b(except)f(that)f(they)h(ma)o(y)f(b)q(e)h(cancelled)i(if)e (there)g(w)o(as)e(no)i(mixture)g(of)f(blo)q(c)o(king)i(and)f(non-blo)q (c)o(king)75 1308 y(calls.)21 b(The)15 b(cancel)i(op)q(eration)e(is)h (collectiv)o(e)h(o)o(v)o(er)d(the)h(original)i(calling)g(comm)o (unicator.)75 1429 y Fh(3.5.6)49 b(Releasing)17 b(Connections)75 1515 y Fs(Before)i(a)g(clien)o(t)i(and)f(serv)o(er)f(connect,)h(they)g (are)f(indep)q(enden)o(t)i(MPI)f(applications.)34 b(An)19 b(error)g(in)75 1572 y(one)h(do)q(es)f(not)h(a\013ect)e(the)i(other.)32 b(After)20 b(establishing)h(a)e(connection)i(with)e Fk(MPI)p 1555 1572 V 16 w(CONNECT)i Fs(and)75 1628 y Fk(MPI)p 160 1628 V 16 w(A)o(CCEPT)p Fs(,)14 b(an)g(error)g(in)h(one)g(ma)o(y)f (a\013ect)f(the)i(other.)k(F)l(or)14 b(instance,)h(if)g(a)f(serv)o(er)g (tries)g(to)g(send)h(a)75 1684 y(message)h(to)g(a)g(clien)o(t)h(whic)o (h)h(has)e(died,)h(the)g(serv)o(er)f(will)i(get)e(an)g(error.)23 b(It)16 b(is)h(desirable)h(for)e(a)g(clien)o(t)75 1741 y(and)f(serv)o(er)f(to)g(b)q(e)h(able)h(to)e(disconnect,)h(so)g(that)e (an)i(error)f(in)h(one)g(will)h(not)f(a\013ect)f(the)g(other.)20 b(These)75 1797 y(same)15 b(considerations)h(apply)g(to)f(paren)o(ts)f (and)i(c)o(hildren.)143 1904 y Fm(\017)23 b Fs(Tw)o(o)14 b(pro)q(cesses)i(are)e Fj(connected)j Fs(if)231 1997 y(1.)22 b(They)c(b)q(oth)h(b)q(elong)g(to)e(the)i(same)f(in)o(tercomm)o (unicator)g(or)g(comm)o(unicator)g(\(including)289 2054 y Fk(MPI)p 374 2054 V 15 w(COMM)p 531 2054 V 17 w(W)o(ORLD)p Fs(\))d(or)231 2127 y(2.)22 b(They)15 b(are)g(are)g(b)q(oth)g (connected)h(to)f(a)g(third)g(pro)q(cess.)143 2221 y Fm(\017)23 b Fs(Tw)o(o)14 b(pro)q(cesses)i(are)e Fj(disconnected)j Fs(\(also)e Fj(indep)q(enden)o(t)p Fs(\))g(if)h(they)f(are)g(not)g (connected.)143 2315 y Fm(\017)23 b Fs(By)18 b(the)h(ab)q(o)o(v)o(e)f (de\014nitions,)i(connectivit)o(y)g(is)f(a)f(transitiv)o(e)h(prop)q (ert)o(y)l(,)f(and)h(divides)h(the)f(uni-)189 2371 y(v)o(erse)e(of)g (MPI)g(pro)q(cesses)h(in)o(to)f(disconnected)i(\(indep)q(enden)o(t\))g (sets)e(\(equiv)m(alence)j(classes\))d(of)189 2428 y(pro)q(cesses.)143 2521 y Fm(\017)23 b Fs(Pro)q(cesses)17 b(whic)o(h)i(are)e(connected,)h (but)g(don't)f(share)g(the)h(same)f Fk(MPI)p 1441 2521 V 16 w(COMM)p 1599 2521 V 16 w(W)o(ORLD)h Fs(ma)o(y)189 2578 y(b)q(ecome)e(disconnected)h(\(indep)q(enden)o(t\))g(if)f(the)f (comm)o(unication)h(path)f(b)q(et)o(w)o(een)h(them)f(is)h(bro-)189 2634 y(k)o(en)f(b)o(y)g(using)h Fk(MPI)p 539 2634 V 16 w(COMM)p 697 2634 V 16 w(DISCONNECT)p Fs(.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 37 39 37 38 bop 75 -100 a Fi(3.5.)28 b(EST)l(ABLISHING)12 b(COMMUNICA)l(TION) g(BETWEEN)e(INDEPENDENT)h(PR)o(OCESSES)p Fs(37)166 49 y(Note)h(that)g Fk(MPI)p 454 49 14 2 v 16 w(MONITOR)h Fs(and)g Fk(MPI)p 862 49 V 16 w(SIGNAL)g Fs(ma)o(y)e(still)j(b)q(e)g (able)f(to)f(in)o(teract)g(with)h(a)g(pro)q(cess)75 106 y(after)h(a)h(disconnect,)h(if)g(the)f(pro)q(cess)h(is)g(a)e(mem)o(b)q (er)i(of)f(a)f(group)h(ob)s(ject.)166 162 y(The)g(follo)o(wing)h (additional)h(rules)f(apply)g(to)e(MPI-1)h(functions:)143 256 y Fm(\017)23 b Fk(MPI)p 274 256 V 15 w(FINALIZE)15 b Fs(is)g(collectiv)o(e)i(o)o(v)o(er)e(a)g(set)g(of)f(connected)j(pro)q (cesses.)143 350 y Fm(\017)23 b Fk(MPI)p 274 350 V 15 w(ABORT)e Fs(do)q(es)e(not)f(ab)q(ort)g(indep)q(enden)o(t)k(pro)q (cesses.)31 b(As)19 b(in)g(MPI-1,)h(it)f(ma)o(y)f(ab)q(ort)g(all)189 406 y(pro)q(cesses)12 b(in)h Fk(MPI)p 517 406 V 16 w(COMM)p 675 406 V 17 w(W)o(ORLD)f Fs(\(ignoring)g(its)h Fk(comm)d Fs(argumen)o(t\).)18 b(Additionally)l(,)d(it)e(ma)o(y)189 463 y(ab)q(ort)k(connected)i(pro)q(cesses)g(as)f(w)o(ell,)i(though)e (it)g(mak)o(es)g(a)g(\\b)q(est)g(attempt")f(to)h(ab)q(ort)f(only)189 519 y(the)e(pro)q(cesses)h(in)g Fk(comm)p Fs(.)143 613 y Fm(\017)23 b Fs(If)10 b(a)h(pro)q(cess)f(exits)h(without)g(calling)h Fk(MPI)p 908 613 V 16 w(FINALIZE)e Fs(the)g(e\013ect)g(on)h(other)f (connected)i(pro)q(cesses)189 669 y(is)j(not)g(de\014ned.)75 858 y Fk(MPI)p 160 858 V 16 w(COMM)p 318 858 V 16 w(DISCONNECT\(comm\)) 117 935 y Fn(INOUT)62 b Fk(comm)470 b Fn(comm)o(unicator)11 b(\(handle\))75 1059 y Fg(int)23 b(MPI)p 245 1059 15 2 v 17 w(Comm)p 358 1059 V 17 w(disconnect\(MPI)p 711 1059 V 15 w(Comm*)g(comm\))75 1146 y(MPI)p 150 1146 V 17 w(COMM)p 263 1146 V 16 w(DISCONNECT\(COMM,)f(IERR\))170 1202 y(INTEGER)h(COMM,)h(IERR)166 1289 y Fs(This)14 b(function)h(w)o (aits)e(for)g(all)i(p)q(ending)g(comm)o(unication)f(on)g Fk(comm)e Fs(to)h(complete)h(and)g(then)g(sets)75 1345 y(the)i(comm)o(unicator)g(to)f Fk(MPI)p 593 1345 14 2 v 16 w(COMM)p 751 1345 V 16 w(NULL)h Fs(in)g(the)g(same)g(manner)g(as)g Fk(MPI)p 1447 1345 V 15 w(COMM)p 1604 1345 V 17 w(FREE)p Fs(.)g(It)g(is)g(a)75 1402 y(collectiv)o(e)h(op)q(eration.)k(It)16 b(ma)o(y)f(not)g(b)q(e)h(called)h(with)f(the)g(comm)o(unicator)f Fk(MPI)p 1480 1402 V 15 w(COMM)p 1637 1402 V 17 w(W)o(ORLD)p Fs(.)g(It)75 1458 y(is)g(erroneous)f(to)g(call)i Fk(MPI)p 547 1458 V 16 w(COMM)p 705 1458 V 16 w(DISCONNECT)f Fs(if)g(not)f(all)h (comm)o(unication)h(can)e(b)q(e)h(completed,)75 1515 y(for)e(instance,)i(if)f(there)g(has)g(b)q(een)h(an)f(send)h(op)q (eration)f(without)g(a)f(matc)o(hing)h(receiv)o(e,)h(or)f(a)f (collectiv)o(e)75 1571 y(op)q(eration)g(that)f(has)h(not)f(b)q(een)i (called)h(on)d(all)i(pro)q(cesses)f(in)h(the)f(comm)o(unicator.)19 b(It)12 b(is)i(not)e(erroneous)75 1627 y(to)h(ha)o(v)o(e)g(outstanding) g(non-blo)q(c)o(king)i(op)q(erations,)f(as)f(long)h(as)f(those)g(op)q (erations)g(can)h(complete.)20 b(Af-)75 1684 y(ter)c Fk(MPI)p 232 1684 V 15 w(COMM)p 389 1684 V 17 w(DISCONNECT)h Fs(returns,)e(all)i(non-blo)q(c)o(king)h(op)q(erations)e(on)g(that)f (comm)o(unicator)75 1740 y(will)h(b)q(e)g(complete,)f(and)g(all)g(asso) q(ciated)g(handlers)h(will)g(ha)o(v)o(e)e(b)q(een)i(called.)22 b(It)14 b(is)h(not)g(allo)o(w)o(ed)g(to)f(call)75 1797 y Fk(MPI)p 160 1797 V 16 w(COMM)p 318 1797 V 16 w(DISCONNECT)i Fs(on)f Fk(MPI)p 779 1797 V 16 w(COMM)p 937 1797 V 16 w(W)o(ORLD)p Fs(.)166 1853 y(Note)10 b(that)h(it)g(ma)o(y)f(b)q(e)h (necessary)g(to)f(disconnect)j(sev)o(eral)e(comm)o(unicators)f(b)q (efore)h(t)o(w)o(o)f(pro)q(cesses)75 1910 y(are)15 b(completely)h (indep)q(enden)o(t.)189 2016 y Ff(R)n(ationale.)54 b Fs(It)17 b(w)o(ould)h(b)q(e)g(nice)h(to)e(b)q(e)h(able)h(to)d(use)i Fk(MPI)p 1235 2016 V 16 w(COMM)p 1393 2016 V 17 w(FREE)g Fs(instead,)g(but)g(that)189 2072 y(function)i(explicitly)i(do)q(es)d (not)g(w)o(ait)g(for)g(p)q(ending)i(comm)o(unication)f(to)f(complete.) 32 b(\()p Ff(End)20 b(of)189 2129 y(r)n(ationale.)p Fs(\))75 2251 y Fh(3.5.7)49 b(Reserved)15 b(k)o(ey)h(values)75 2336 y Fs(The)e(follo)o(wing)h(k)o(ey)f(v)m(alues)i(are)e(reserv)o(ed)g (for)f Fk(info)i Fs(argumen)o(ts)e(in)i(section)g(3.5.)j(An)d(implemen) o(tation)75 2393 y(is)h(not)g(required)h(to)e(in)o(terpret)h(these)g(k) o(ey)f(v)m(alues,)i(but)f(if)g(it)g(do)q(es)g(in)o(terpret)g(the)g(k)o (ey)g(v)m(alue,)g(it)h(m)o(ust)75 2449 y(pro)o(vide)f(the)f (functionalit)o(y)h(describ)q(ed.)75 2556 y Fj(ip)p 121 2556 16 2 v 19 w(p)q(ort)24 b Fs(V)l(alue)11 b(con)o(tains)f(IP)g(p)q (ort)g(n)o(um)o(b)q(er)h(at)e(whic)o(h)i(to)f(establish)h(a)f Fk(p)q(o)o(rt)p Fs(.)18 b(\(Reserv)o(ed)11 b(for)e Fk(MPI)p 1771 2556 14 2 v 16 w(PORT)p 1909 2556 V 17 w(OPEN)189 2612 y Fs(only\).)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 38 40 38 39 bop 75 -100 a Fs(38)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)75 49 y Fh(3.5.8)49 b(Client/Server)16 b(Examples)75 182 y Fo(Discussion:)f Fn(These)h(examples)d(ha)o(v)o(e)g(not)h(b)q(een)h (up)q(dated)75 350 y Fk(Simplest)h(example)f(-)g(completely)g(p)q(o)o (rtable.)75 435 y Fs(The)j(follo)o(wing)h(example)g(sho)o(ws)f(the)g (simplest)i(w)o(a)o(y)d(to)g(use)i(the)f(clien)o(t/serv)o(er)h(in)o (terface.)29 b(It)19 b(do)q(es)75 492 y(not)c(use)g(service)h(names)g (at)e(all.)166 548 y(On)i(the)f(serv)o(er)g(side:)170 699 y Fg(char)24 b(myport[MPI_MAX_NAME_LEN])o(;)170 755 y(MPI_Comm)f(intercomm;)170 812 y(/*)h(...)f(*/)170 868 y(MPI_Port_open\(NULL,)f(&myport\);)170 924 y(printf\("port)h(name)g (is:)g(",)h(\045s\\n",)f(myport\);)170 1037 y(MPI_Accept\(myport,)f(0,) h(MPI_COMM_SELF,)f(&intercomm\);)170 1094 y(/*)i(do)g(something)e(with) i(intercomm)e(*/)75 1188 y Fs(The)d(serv)o(er)g(prin)o(ts)h(out)f(the)g (p)q(ort)g(name)g(to)g(the)g(terminal)h(and)f(the)h(user)f(m)o(ust)g(t) o(yp)q(e)g(it)g(in)h(when)75 1244 y(starting)g(up)h(the)g(clien)o(t)h (\(assuming)f(the)g(MPI)g(implemen)o(tation)h(supp)q(orts)f(stdin)g (suc)o(h)g(that)g(this)75 1301 y(w)o(orks\).)e(On)c(the)h(clien)o(t)g (side:)170 1394 y Fg(MPI_Comm)23 b(intercomm;)170 1451 y(char)h(name[MPI_MAX_NAME_LEN];)170 1507 y(printf\("enter)f(port)g (name:)g("\);)g(fflush\(stdout\);)170 1564 y(gets\(name\);)170 1620 y(MPI_Connect\(name,)f(NULL,)h(0,)h(MPI_COMM_SELF,)e (&intercomm\);)75 1740 y Fk(Ocean/A)o(tmosphere)17 b(-)e(relies)g(on)h (name)e(publishing)75 1826 y Fs(In)g(this)f(example,)h(the)g(\\o)q (cean")f(application)h(is)g(the)f(\\serv)o(er")g(side)h(of)e(a)h (coupled)i(o)q(cean-atmosphere)75 1882 y(climate)h(mo)q(del.)21 b(It)15 b(assumes)g(that)g(the)g(MPI)g(implemen)o(tation)i(publishes)g (names.)170 2045 y Fg(MPI_Port_open\(NULL,)22 b(&port_name\);)170 2102 y(MPI_Name_publish\(port_name,)f("ocean"\);)170 2215 y(MPI_Accept\(port_name,)g("ocean",)i(0,)h(MPI_COMM_SELF,)e (&intercomm\);)170 2271 y(/*)i(do)g(something)e(with)i(intercomm)e(*/) 170 2327 y(MPI_Name_delete\(port_name,)f("ocean"\);)75 2490 y Fs(On)16 b(the)f(clien)o(t)i(side:)170 2584 y Fg(MPI_Name_get\("ocean",)k(port_name\);)170 2640 y(MPI_Connect\()i (port_name,)f(NULL,)h(0,)h(MPI_COMM_SELF,)e(&intercomm\);)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 39 41 39 40 bop 75 -100 a Fi(3.5.)23 b(EST)l(ABLISHING)13 b(COMMUNICA)l(TION) g(BETWEEN)f(INDEPENDENT)g(PR)o(OCESSES)p Fs(39)75 49 y Fk(Ocean/A)o(tmosphere)17 b(-)e(uses)i(name)d(publishing)k(if)d(p)q (ossible)75 135 y Fs(This)d(example)g(is)f(similar)i(to)d(the)i (previous)f(one)h(except)f(that)g(is)h(mak)o(es)e(a)h(run)g(time)h (decision)h(\(compile)75 192 y(time)j(w)o(ould)f(also)g(b)q(e)h(p)q (ossible\))h(based)e(on)g(whether)h(names)f(are)g(published.)170 351 y Fg(MPI_Port_open\(NULL,)22 b(port_name\);)170 407 y(MPI_Publish_name\(port_name,"o)o(cean"\);)170 520 y (MPI_Accept\(port_name,)f(NULL,)j(0,)f(MPI_COMM_SELF,)f(&intercomm\);) 170 577 y(if)i(\(!MPI_NAMES_ARE_PUBLISHED\))c({)218 633 y(if)k(\(batch_mode\))e(error\(FATAL,)g("can't)h(transfer)g(port)g (name"\);)218 689 y(printf\("port)f(name)i(is:)f(",)h(\045s\\n",)f (port_name\);)170 746 y(})170 802 y(/*)h(do)g(something)e(with)i (intercomm)e(*/)170 859 y(MPI_Name_delete\(port_name,)f("ocean"\);)75 1018 y Fs(On)16 b(the)f(clien)o(t)i(side:)170 1109 y Fg(if)24 b(\(!MPI_NAMES_ARE_PUBLISHED\))c({)218 1165 y(if)k(\(batch_mode\))e(error\(FATAL,)g("can't)h(transfer)g(port)g (name"\);)218 1222 y(printf\("low)g(quality)f(mpi)i(implementation)e (encountered.)g(\\\\)409 1278 y(enter)h(port)h(name:)f("\);)g (fflush\(stdout\);)218 1335 y(gets\(port_name\);)170 1391 y(})h(else)f({)218 1448 y(MPI_Name_get\("ocean",)e(port_name\);) 170 1504 y(})170 1561 y(MPI_Connect\(port_name,)g(NULL,)i(0,)h (MPI_COMM_SELF,)e(&intercomm\);)75 1680 y Fk(Simple)15 b(client-server)h(example.)75 1766 y Fs(This)h(is)g(a)f(simple)h (example;)g(the)g(serv)o(er)f(accepts)g(only)h(a)f(single)h(connection) h(at)d(a)h(time)h(and)f(serv)o(es)75 1822 y(that)e(connection)i(un)o (til)g(the)f(clien)o(t)h(requests)f(to)f(b)q(e)i(disconnected.)21 b(The)15 b(serv)o(er)g(is)g(a)g(single)h(pro)q(cess.)166 1879 y(Here)21 b(is)h(the)g(serv)o(er.)38 b(It)21 b(accepts)h(a)f (single)h(connection)h(and)e(then)h(pro)q(cesses)g(data)e(un)o(til)j (it)75 1935 y(receiv)o(es)16 b(a)f(message)g(with)g(tag)f Fg(1)p Fs(.)20 b(A)15 b(message)g(with)h(tag)e Fg(0)h Fs(tells)h(the)g(serv)o(er)e(to)h(exit.)75 2026 y Fg(#include)23 b("mpi.h")75 2083 y(main\()g(int)h(argc,)f(char)g(**argv)g(\))75 2139 y({)170 2195 y(MPI_Comm)g(client;)170 2252 y(MPI_Status)g(status;) 170 2308 y(char)h(port_name[MPI_MAX_NAME_L)o(EN];)170 2365 y(double)f(buf[MAX_DATA];)170 2421 y(int)95 b(size,)24 b(again;)170 2534 y(MPI_Init\()f(&argc,)g(&argv)g(\);)170 2591 y(MPI_Comm_size\(MPI_COMM_WORLD,)d(size\);)170 2647 y(if)k(\(size)f(!=)h(0\))f(error\(FATAL,)g("Server)g(too)g(big"\);)170 2704 y(MPI_Port_open\(NULL,)f(port_name\);)-32 46 y Fl(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 40 42 40 41 bop 75 -100 a Fs(40)483 b Fi(CHAPTER)15 b(3.)35 b(PR)o(OCESS)16 b(CREA)l(TION)g(AND)f(MANA)o(GEMENT)170 49 y Fg(printf\("server)22 b(available)h(at)h(\045s\\n",port_name\);) 170 106 y(while)g(\(1\))f({)266 162 y(MPI_Accept\()f(port_name,)h (NULL,)g(0,)h(MPI_COMM_WORLD,)d(&client)i(\);)266 219 y(again)g(=)h(1;)266 275 y(while)f(\(again\))g({)361 332 y(MPI_Recv\()g(buf,)g(MAX_DATA,)g(MPI_DOUBLE,)f(MPI_ANY_SOURCE,)g (MPI_ANY_TAG,)600 388 y(client,)h(&status)g(\);)361 444 y(switch)g(\(status.tag\))g({)457 501 y(case)g(0:)h(MPI_Comm_free\()e (&client)h(\);)648 557 y(MPI_Finalize\(\);)648 614 y(return)g(0;)457 670 y(case)g(1:)h(MPI_Comm_disconnect\()d(&client)i(\);)648 727 y(again)g(=)h(0;)648 783 y(break;)457 840 y(case)f(2:)h(/*)f(do)h (something)f(*/)457 896 y(...)457 953 y(default:)648 1009 y(MPI_Abort\()f(MPI_COMM_WORLD,)g("Unexpected)g(message)h(type")g (\);)457 1065 y(})361 1122 y(})266 1178 y(})75 1235 y(})166 1319 y Fs(Here)15 b(is)h(the)f(clien)o(t.)75 1414 y Fg(#include)23 b("mpi.h")75 1471 y(main\()g(int)h(argc,)f(char)g(**argv)g(\))75 1527 y({)170 1584 y(MPI_Comm)g(server;)170 1640 y(double)g (buf[MAX_DATA];)170 1696 y(char)h(port_name[MAX_NAME_SIZE])o(;)170 1809 y(MPI_Init\()f(&argc,)g(&argv)g(\);)170 1866 y(strcpy\(port_name,) f(argv[1])h(\);/*)g(assume)g(server's)g(name)g(is)h(cmd-line)f(arg)g (*/)170 1979 y(MPI_Connect\()g(port_name,)f(NULL,)h(0,)h (MPI_COMM_WORLD,)e(&server)h(\);)170 2092 y(while)h(\(!done\))f({)266 2148 y(tag)g(=)h(2;)g(/*)f(Action)g(to)h(perform)f(*/)266 2205 y(MPI_Send\()g(buf,)g(n,)g(MPI_DOUBLE,)g(0,)g(tag,)h(server)f(\);) 266 2261 y(/*)g(etc)h(*/)266 2317 y(})170 2374 y(MPI_Send\()f(buf,)g (0,)h(MPI_DOUBLE,)e(0,)i(1,)g(server)f(\);)170 2430 y (MPI_Comm_disconnect\()f(&server)g(\);)170 2487 y(MPI_Finalize\(\);)75 2543 y(})166 2638 y Fs(If)17 b(the)g(serv)o(er)f(needs)i(to)e(manage)g (m)o(ultiple)i(connections)g(at)e(once,)h(it)g(can)g(use)g Fk(MPI)p 1675 2638 14 2 v 16 w(IA)o(CCEPT)75 2695 y Fs(instead.)j(The)c (clien)o(t)g(need)g(not)f(b)q(e)h(c)o(hanged.)1967 46 y Fl(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Mon Aug 19 12:25:05 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA00880; Mon, 19 Aug 1996 12:25:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA27006 for mpi-core-out; Mon, 19 Aug 1996 11:27:00 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA26999 for ; Mon, 19 Aug 1996 11:26:49 -0500 Message-Id: <199608191626.LAA26999@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: September MPI Forum meeting Date: Mon, 19 Aug 1996 11:26:45 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, Below is the usual invitation. Notable changes are that this will be a four-day meeting, starting at 1:30 on Tuesday, September 3 (the day after Labor Day). And to cover the extra expenses associated with the extra day, we will have to raise the registration fee to $115. Sorry about this, but that's the way the budget crumbles. Please let me know now for the head count whether you are coming. I know that some people already had their tickets before we voted to have the longer meeting. If you are not able to come until Wednesday, let me know that as well. Rusty The usual note, with details you now probably have memorized: The next meeting of the MPI Forum will take place on Tues.-Fri. Sep. 3-6. at the Ramada Hotel O'Hare (same place as the last few meetings). It is time to make your reservations. Register at the hotel by calling (847) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). Please let me know as soon as possible if you are coming, so that I can give the hotel a head count. Thanks. When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. We will follow a variation on the usual schedule, beginning at 1:30 on Tuesday and ending at noon on Friday. We will charge a registration fee of $115, which should be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. If you have any problems or questions, let me know, either by email (lusk@mcs.anl.gov) or phone (708-252-7852). Here is the schedule of future MPI Forum meetings, which are also scheduled for the Ramada O'Hare. Oct 8-11 Supercomputing '96 is Nov 17-22. Regards, Rusty From mpi-core-human@mcs.anl.gov Mon Aug 19 15:36:37 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA02132; Mon, 19 Aug 1996 15:36:31 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA01711 for mpi-core-out; Mon, 19 Aug 1996 14:39:36 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA01697; Mon, 19 Aug 1996 14:39:19 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id OAA26346; Mon, 19 Aug 1996 14:39:17 -0500 Date: Mon, 19 Aug 1996 14:39:16 -0500 (CDT) From: Tony Skjellum To: Tony Kimball cc: mpi-core@mcs.anl.gov, mpi-comm@mcs.anl.gov Subject: Re: event driven chapter In-Reply-To: <199608191904.OAA26197@compound.Think.COM> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Mr. Kimball: You should take this up with the MPI-CORE reflector. It is the policy to reflect the postscript, which is then archived. I understand the concerns you raise, and so I have taken the initiative to forward to the mpi-core group, and mpi-comm group, so it can be discussed by the appropriate people, rather than aiming it at a subset. This is my final communication with you on this subject. -Tony Anthony Skjellum, PhD, Asst. Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! On Mon, 19 Aug 1996, Tony Kimball wrote: > Quoth Tony Skjellum on Mon, 19 August: > : OK, you don't understand the MPI reflector list. We send out the > : postscript. The fact that I reflected it was unintentional. I do think > : you know how to hit delete, so ... > > I object strongly to the practice. This is a bandwidth-profligate > distribution technique, and therefore of detriment to the Internet > community as a whole. Moreover, it does material damage to those who > must pay for the packets to recieve the unwanted drafts. If it were > desirable to distribute drafts by email, which I strongly contend it > is not, it would nonetheless be no more than commonsense to separate > the distribution lists into drafts-only and discussion-only subsets. > I have no problem hitting delete. That is not an issue for me. > Locking down my Ethernet LAN for as much as a second to deliver unwanted > materials is a significant detriment, made worse by a factor of twenty > when I have twenty independent recipients on the LAN. Worse yet > is locking down the T1 external interface of my network to the > backbone for that time. Worse still is locking down my V.34 modem > port for 3 full minutes, particularly if I am paying premium rates > to read my mail in flight. > > > From mpi-core-human@mcs.anl.gov Mon Aug 19 15:49:43 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA02199; Mon, 19 Aug 1996 15:49:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA01991 for mpi-core-out; Mon, 19 Aug 1996 14:52:44 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA01986 for ; Mon, 19 Aug 1996 14:52:37 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id OAA25963; Mon, 19 Aug 1996 14:52:35 -0500 Date: Mon, 19 Aug 1996 14:52:35 -0500 Message-Id: <199608191952.OAA25963@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: MPI document available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. These directories also have all the sources and figures. The document is now well over 300 pages so send it to a big printer. This version is the first attempt to truely integrate the document in anticipation of the upcoming public release. Note that all the chapter numbers changed. Please check it over since it is likely to have oversights. If you want to make changes for the version to be printed for the meeting, you need to make sure the chapter authors can incorporate them by 10:00 AM (central US time) on Monday, August 26. Steve From mpi-core-human@mcs.anl.gov Mon Aug 26 13:21:34 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA12648; Mon, 26 Aug 1996 13:21:33 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA17622 for mpi-core-out; Mon, 26 Aug 1996 12:21:57 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA17616 for ; Mon, 26 Aug 1996 12:21:48 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id MAA25202; Mon, 26 Aug 1996 12:21:47 -0500 Date: Mon, 26 Aug 1996 12:21:47 -0500 Message-Id: <199608261721.MAA25202@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The version to be printed and distributed at the next meeting is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. These directories also have all the sources and figures. The document is now almost 300 pages so send it to a big printer. This version is an attempt to begin to get the document ready for the SC96 release. As editor, I would appreciate people taking the effort to look it over and give me feedback. Steve From mpi-core-human@mcs.anl.gov Thu Aug 29 12:36:25 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA18466; Thu, 29 Aug 1996 12:36:24 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA01542 for mpi-core-out; Thu, 29 Aug 1996 11:34:59 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA01537; Thu, 29 Aug 1996 11:34:47 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id LAA19890; Thu, 29 Aug 1996 11:34:47 -0500 Date: Thu, 29 Aug 1996 11:34:47 -0500 (CDT) From: Tony Skjellum To: Tony Kimball cc: mpi-core@mcs.anl.gov, mpi-comm@mcs.anl.gov Subject: Re: Newest realtime draft In-Reply-To: <199608291609.LAA01626@compound.Think.COM> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Mr. Kimball: I cannot thank you enough for reiterating your concern, but I remind you that your subscription is voluntary, but the method of sharing information is the convention of the Forum, of which I am but one voice among many. Once again, I have propagated your concern to the Forum. I cannot stop doing transmission in this way until the MPI Forum changes its policy. Please address comments to those bodies, and unsubscribe in the meantime to control bandwidth to your desktop. You will find periodic updates at the URLs below, and you can poll for them. If you were subscribing to other lists, you would get their postscript too. Perhaps you do, and are haranging others than me. If so, I feel that you are missing the opportunity to express your concerns directly to the entire MPI Forum body, which might listen to you. I clearly explained these things to you before. If you do not agree with the terms of our voluntary organization's activities, please complain to the group at large, not to one of those following its procedures. I personally am time writing this letter, because you are from a company that might contribute to the standard; not because i have not explained this before. To be frank, if you were a random lurker, I would have blown you off already, after having responded clearly twice before. However, I do find all of this tiresome and atavistic. Your employer would be better served by your contributing a technical response to the documents, rather than pursuing a crusade. The only other person who complained about this is connected to us by cellular modem, and reads his e-mail by downloading while flying cross country... Tony Skjellum On Thu, 29 Aug 1996, Tony Kimball wrote: > Date: Thu, 29 Aug 1996 11:09:32 -0500 (CDT) > From: Tony Kimball > To: tony@erc.msstate.edu > Subject: Newest realtime draft > > Quoth Tony Skjellum on Wed, 28 August: > : See also anonymous ftp > : as ftp://aurora.cs.msstate.edu/pub/mpi/rt-2-28aug96.ps > : or ftp://aurora.cs.msstate.edu/pub/mpi/rt-2-28aug96.ps.Z > : > > I very much appreciate the urls, and thank you for them > most enthusiastically; however, I must strenuously protest > the redudant inclusion of 203 kilobytes of uuencoded > binary data. > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Fri Aug 30 15:18:09 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA15955; Fri, 30 Aug 1996 15:18:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA24670 for mpi-core-out; Fri, 30 Aug 1996 14:19:36 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA24662 for ; Fri, 30 Aug 1996 14:19:25 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id OAA15484; Fri, 30 Aug 1996 14:19:07 -0500 Date: Fri, 30 Aug 1996 14:19:07 -0500 Message-Id: <199608301919.OAA15484@romano.cs.wisc.edu> From: Steve Huss-Lederman To: alk@think.com CC: mpi-core@mcs.anl.gov In-reply-to: <199608291739.MAA01723@compound.Think.COM> (message from Tony Kimball on Thu, 29 Aug 1996 12:39:19 -0500 (CDT)) Subject: Re: Newest realtime draft Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The issue of sending drafts via e-mail has now been raised several times. My hope was to send out a message soon to say a solution had been reached. I am actively working this problem and hope to have a posted solution soon. I will be discussing it with chapter authors at the next meeting. Please hang in there for a little while longer. Steve From mpi-core-human@mcs.anl.gov Fri Aug 30 15:19:28 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA15982; Fri, 30 Aug 1996 15:19:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA24798 for mpi-core-out; Fri, 30 Aug 1996 14:23:04 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA24788 for ; Fri, 30 Aug 1996 14:22:52 -0500 Received: from windsurf.kgn.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA24353; Fri, 30 Aug 1996 15:18:49 -0400 Received: by windsurf.kgn.ibm.com (AIX 4.1/UCB 5.64/4.03) id AA43398; Fri, 30 Aug 1996 15:21:47 -0400 Date: Fri, 30 Aug 1996 15:21:47 -0400 From: treumann@kgn.ibm.com (Dick Treumann) Message-Id: <9608301921.AA43398@windsurf.kgn.ibm.com> To: @minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov Subject: ATTENTION Chapter Authors Cc: @minnie.nic.kingston.ibm.com:treumann@mhv.net Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This note identifies a number of errors (at least in my view) in the current version of the MPI document. It covers several chapters so I hope it will be surveyed by at least the owners of the chapters on: Terms and Conventions MPI-1.2 Process Creation and Management One Sided External Interfaces Language Bindings Miscellany I did not attempt to review MPI-IO or Collective Communication The page and line numbers are from the version offered by Steve prior to the Sept Meeting. I hope they still fit the doc handed out on Tuesday Dick Treumann **************************************************************************** Pages 12,30,48,69 have lines which run off the page. At least with the printer I am using this results in the whole page printing blank. I found an alternate way to print thes pages so I have been able to see thta this is what all have in common. Pg 11(19,35) References to mapping C names to C++ names use the Fortran spellings. Though the Fortran spellings are generic in most contexts, here I think it makes better sense to say MPI_Datatype and MPI_Comm_compare become MPI::Datatype .... Pg15 When did it become the intent of MPI to be signal safe in the sense that it can run on a signal handler? I think any suggestion that MPI is signal safe is very dangerous. On AIX, malloc is not signal safe and trying to do MPI without malloc would be impractical. Is malloc signal safe on other Unix like systems? At best, any code which tries to do MPI on signal handlers is non-portable with timing related failures likely on systems that it does not port to. Any implied support in MPI for something nonportable but with no way for a non supporting system to detect the danger should be avoided. pg25(line30) Does this mean that any non-blocking receive which has been posted must be either successfully completed or be canceled else MPI_FINALIZE gives a fatal error? If I am using handlers it may not be easy to know about outstanding Irecvs which are posted to kick handlers. Does anyone have codes which post Irecvs expecting only a subset to complete in any given run? Given the ability to do MPI_Testsome on a set of requests, it seems like a reasonable thing to do. Why not just guarantee that any MPI message which HAS completed in the users perception (A wait succeeded.) will in fact complete under the covers too. A user who does MPI_Finalize with uncompleted messages can reasonably have decided these are DONTCARE cases. Any strengthening of MPI_Finalize should probably say how MPI_Isend/MPI_Request_free and early arrivals (small eager-protocol messages) with no matching receive fit in. pg25(line38) My understanding of the MPI 1.1 doc is that EVERY task of an MPI job must call MPI_Init. This implies that there can be a correct MPI MPMD job in which only 1 program calls MPI_Init. Is this really intended and if so, on what basis in the MPI 1.1 doc? pg27 The section on "New Datatypes" is really about "New Datatype Constructors" pg28(22) Add "MPI_Type_simple_struct should only be used for complete structures, never for portions of larger structures." pg30 "Example" This indicates that in some cases the wire protocol for 2 floats followed by a double will contain padding and in others in will not. If a message containing 2 floats followed by a double is received as MPI_PACKED and then digested by calls to MPI_Unpack, first for the pair of floats and then for the double, how does MPI know if there is a 4 byte pad? OR I receive the [cc]xx[ffff][ffff]xxxx[dddddddd] message into a halfword aligned buffer using MPI_PACKED then call MPI_Unpack for the [cc] first and next for [ffff][ffff]. Which call to MPI_Unpack decides it must discard two bytes and how does it decide this? pg46(40) Shouldn't MPI_PARENT be MPI_COMM_PARENT? pg47(12) The PVM rule is $HOME/pvm3/bin/$PVM_ARCH pg47(14) The "SP2" series is now simply called SP. pg58(1) A bit clearer if the word "spawn" were added. "...collective spawn routine..." pg58(44) add "MPI will spawn the largest number of processes it can, consistent with some number in the set. The order in which triplets are given is not significant." Also, should say if 16:8:-2 is a valid form since it may seem logical to a user who overlooks the point that order is not significant. pg62(29) info is an opaque object >>with a handle << of type MPI_Info. i.e. add the marked phrase because it is not the object which has this type. 62(38) Make it explicit that it is the format of "value" which may be unrecognizable. 62(40-42) Does the value length include the terminating \0 in C? I assume yes but it would be better to say. 63(34),64(12),64(35) How could a Fortran implementation recognize an overlong value? 65 Given the ability to dup INFO objects, the ability to replace key values and NO way to remove a key once defined -- It seems we need to define a NULL value that will apply to every key. 65 Maybe I just overlooked something but - Is there a "No info provided" predefined handle which is different from MPI_INFO_NULL. If not, how does a routine tell if it is correctly passed a "No info" or mistakenly passed a handle to an info which has been freed. Do we care? 66(1-3) Given that info is a modifiable object, I think we must state that it is parsed before return from the call which uses it, that MPI in a threaded environment guarantees that it is managed such that parsing by a thread is atomic re the modification calls and lastly that Free is immediate. The user will need to do the managing of the handle on his own if he uses an info on one thread while it might be freed on another. This is not unique to info though. 67(42) Macro mpifunc shows in text 67(44-45) Does MPI_Request_free on a monitor request pop any attached handler? Probably no but should say. 68(7) say UNIX wait() rather than just wait(). I first read this line as a shortcut way of saying MPI_Wait. 70(24-29) Make it clear what "this information" is. Does it mean how many instances of ProgA, how many of ProgB etc? It is clearly still possible to find out how many process in total even when MPMD. 72(48),73(1) The IBM SP2 is now simply the IBM SP (will not need SP3, SP4 ...) 72-73 MPI_PORT_OPEN - This still discusses INFO as a string rather than an MPI_Info handle. Is the arg really a string? Then it should get a new name. Otherwise the text needs fixing and so do the examples. 78(37) MPI_Recv is an even stronger example. A receive against a dead client could hang the server forever. 79(37-41) The semantic in which disconnect completes requests will leave dangling references. If I do an MPI_IRECV(........,req,..) then disconnect I will still have what seems to be a valid handle in the "req" variable. ONE SIDED 84 In the introduction it says "needs to compute the inverse mapping" then goes on to mention the option of polling for requests. Should probably say "needs to determine the.. 85(11) clean up double negative 85(19) add "after MPI_Init" to sentence ending "any MPI call." 86(advise to users) Consider pointing out that "void *base" in MPI_Mem_alloc and MPI_Mem_free are not the same parameter (in lieu of making Mem_alloc use "void **base") 87(9,10,27-29) Should rule out 0 as displacement unit or explicitly allow it with some hint as to why it is legit. 87(29,30) An RMA_INIT does not modify an existing communicator, it creates a new one. The first sentence should be more like: "Only a communicator created by an MPI_RMA_INIT call can be used for RMA commands." I did not add "or one duped from ..." because I slightly lean toward alternative 1 below which says dup does not carry this along. 89(Rational) Note that while RMA to own memory may be a plus on some versions of MPI it will probably be a minus on others. 91(Advise to impl) what does MPI do when it must truncate an MPI_PUT? Where and when is an error reported? 92(MPI_Get) Does the bounds checking indicated for MPI_PUT apply to the target window on a get as well (i.e. Do not read outside the target window)? I would expect the MPI-1 rules for a receive to apply to the origin window in this case. (no overwrite) Put, Get, Accumulate - target displacement should be nonnegative integer 96(48) replace "other" with non-RMA. Clearer that way. 97(32,41),102(21),103(6-12,22,23),108(19) - Uses of MPI_LOCK, MPI_UNLOCK, MPI_WINDOW_LOCK and MPI_WINDOW_UNLOCK should all be RMA_LOCK/UNLOCK. 99(24,25) Should it be explicit that two puts from the same location cannot occur in an epoch? 100(19-20) Should say "Each MPI_RMA_POST must be matched by an MPI_RMA_WAIT and each MPI_RMA_WAIT followed by a fresh MPI_RMA_POST if the widow is to be used again." 100(Alternative 1) Mentions info on MPI_RMA_INIT. This parameter no longer exists. 102 Is there an idea that there may be impl defined values beyond MPI_STRONG, _WEAK, _NOCHECK? If not then this might better be something other than an info. If it is to be an info then these are probably best defined as values for some as yet unnamed key. 106(27) add "after MPI_Init" to last sentence. 108(1-14) Is it legal to have a window at task three with 0,1,2,3 on the communicator. Task 0 does put; all barrier; task 1 put; all barrier; task 2 put; all barrier; task 3 does local load? The description of barrier as a toggle does not seem to allow this. Also, toggle semantic without a way to check a window's current mode is hard for a user to track. 110(9-10) To say a "high quality Fortran" will solve the problems introduced by MPI violations of the Fortran language rules is unwarranted. The only way I see for a Fortran compiler to do this would be to forgo optimizations that are proper by language rules. This is hardly a "quality" behavior for a Fortran compiler. 110(32-35) I presume this comment is based on the idea that "frombuf" was originally part of the MPI_RMA_INIT. This is not evident in the code fragment. 110(48) should "update_core(comm)" be "update_core(A)" and shifted to align with "for"? 117(23) left should be "right" 117(39) Suggest "Some Fortran implementations may provide compiler options to solve this problem" Again, it is not a matter of quality since it involves forgoing legal optimizations. 117(46,47) Suggest "A C compiler understands the implications. Some compilers do offer optional, aggressive optimization levels which may not be safe." EXTERNAL INTERFACES 164(14-17) The whole issue of macros for MPI functions needs broad consideration. It looses name shift profiling and if MPI allows macros for some functions it should probably be specific about which ones. 164(47-48) I could not decide what the last sentence means. Can it be clarified? 166(32-36) In a homogeneous MPI you only need 2 internal representations for datatypes, one covers contiguous, vector and hvector. The other covers indexed, hindexed and struct (incl new versions of struct). If I could return either hvector or struct based on internal form then I would not need to worry about keeping track of the combiner. Since this is ruled out (properly),I see little benefit in the flexibility that remains. 177 (Datatype transfer) - I realize this may be too late but I want to point out that the overloading of MPI_Pack and MPI_UNPACK to handle datatypes is a REALLY UGLY KLUDGE. I would like to see new functions, one which encapsulates an opaque object and one which reconstitutes it. We are adding so many new functions to MPI that this does not seem like the place to get by on the cheap. 181-182 is the "handle" arg to ...PUT, ...GET and ...DELETE really intended to be a reference parameter or is the "*" a typo? 186(6,7) Suggest "There is no requirement that an MPI program provide a name for any handle." 188(1-5)(6-10) These two paragraphs are not consistent. An MPI which used a global lock to serialize MPI calls could not allow MPI calls on one thread while another was blocked on some MPI call. 191(first Alt) This is not consistent with the stronger rules for MPI_FINALIZE unless it is accepted that all handler use will involve some policy in which the client for which a handler is posted takes the responsibility of telling the handler to run one last time in a cleanup mode which does not involve a repost. 192-193 I believe MPI_MUTEX_CREATE and MPI_MUTEX_FREE must both be collective. There must be some instantiation of the mutex which all members of the communicator deal with. and each must know where it is and how it is locked and unlocked. 194-195 MPI_COND_CREATE and MPI_COND_FREE must be collective. 195(25-26) I do not understand the comment that OS must contain the cond variable state. If so, please explain more fully. Also, if so, does this not rule out this function for MPI based on MPI not defining special OS or environment interfaces. 195 MPI_COND_WAIT - In a thread compliant MPI, can more than one COND_WAIT exist on a single MPI cond (2 or more threads)? I would advocate that it be disallowed but either way it should be explicit. FORTRAN 90 209 I would prefer to see a different include file name as for example mpif90.h. Since F90 is a superset of F77 there is a probability that the F90 include file will not be tolerated by an F77 compiler. An environment which has both F77 and F90 compilers in use should be able to go after suitable include files for each compiler. 212(10-18) This advise brings in the problem of MPI calls with hidden arguments that is discussed in the section about compiler opt and Fortran. MISCELLANY 231 (MPI_TRYRECV) This is still risky in the case of a threaded MPI unless the user is careful with tags and can guarantee that the QUERY will not find a message 100 words long and then will have that taken by another thread leaving the tryrecv to match a 200 word message. At least it will not deadlock though. ALSO - The "status" argument is missing from the IN/OUT list. 231 (continuable errors) What does it mean to say one of these errors is continuable when it occurs in a collective communication? From mpi-core-human@mcs.anl.gov Sun Sep 1 13:09:13 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA25939; Sun, 1 Sep 1996 13:09:12 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA14628 for mpi-core-out; Sun, 1 Sep 1996 12:07:47 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA14623 for ; Sun, 1 Sep 1996 12:07:38 -0500 Message-Id: <199609011707.MAA14623@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Attendance at September meeting Date: Sun, 01 Sep 1996 12:07:37 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I now need the final head count for the hotel. Here are the people who have told me they are coming to the meeting next week. Please let me know if you are coming but are not on the list (or the other way around.) Rusty Robert Babb Rajesh Bordawekar Pete Bradley Maciej Brodowicz Margaret Cahir Pang Chen Lyndon Clarke Dennis Cottel Jim Cownie (starting Wed) Raja Daoud David DiNucci Terry Dontje Mark Fallon Richard Frost Shane Hebert Rolf Hempel (starting Wed) HansChristian Hoppe Steve HussLederman Arkady Kanevsky (starting late Tues) Susan Kraus Steve Landherr Juan Leon Rusty Lusk Nick Nevin Bill Nitzberg Elsie Pierce JeanPierre Prost Anna Rounbehler Eric Salo (starting late Tue) Bill Saphir Andrew Sherman Anthony Skjellum (Tue-Wed) Marc Snir Jeff Squyres David Taylor Rajeev Thakur MaryDell Tholburn Dick Treumann Manuel Ujaldon Klaus Wolf Parkson Wong From mpi-core-human@mcs.anl.gov Sun Sep 1 16:00:12 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA26652; Sun, 1 Sep 1996 16:00:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA15942 for mpi-core-out; Sun, 1 Sep 1996 15:01:35 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA15937 for ; Sun, 1 Sep 1996 15:01:26 -0500 Message-Id: <199609012001.PAA15937@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: NSF support Date: Sun, 01 Sep 1996 15:01:19 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk For those university-based attendees who have been partially supported by the NSF grant through Tennesee, the ceiling on what reimbursement you can ask for has been raised for four-day meetings, from $500 to $650. Rusty From mpi-core-human@mcs.anl.gov Sun Sep 1 17:16:22 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA27198; Sun, 1 Sep 1996 17:16:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA16351 for mpi-core-out; Sun, 1 Sep 1996 16:19:18 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA16346 for ; Sun, 1 Sep 1996 16:19:12 -0500 Message-Id: <199609012119.QAA16346@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: highly tentative agenda Date: Sun, 01 Sep 1996 16:19:11 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is an agenda for this week's meeting, even more tentative than usual. My considerations in forming it were: Chapters that can be voted on should be, with a subcommittee meeting first to prepare for voting. (Dynamic, Binding, Misc) One-sided needs more subcommittee work, but if we can get to a vote that would be good. Chapters that have not been seen outside their subcommittee for a while should get a thorough reading in a plenary session, even if they are not quite ready for an official first reading, so that there won't be last-minute surprises (Collective, I/O, Realtime, External). I/O has specifically requested a plenary session, and I agree it would be a good idea. For several of these chapters we need to make tentative decisions about what is MPI-1.2, what is MPI-2, and what is JOD. I believe that the situation with realtime is particularly unclear on this, and maybe the threads stuff now in External. Some critical people will only be able to attend part of the meeting. I/O wants to have lots of subcommittee time. Comments welcome. Rusty ______________________________________________________________________ Agenda for MPI Forum meeting, September 3-6, 1996 Tuesday --------- 1:30-2:00 Discussion of agenda and Intro 2:00-3:30 Room A: One-sided Room B: I/O Room C: Bindings 3:30-4:00 Break 4:00-5:30 Room A: Dynamic Room B: External Room C: Collective 6:00-8:00 Dinner 8:00-10:00 Room A: I/O Room B: Misc Room C: as needed Wednesday -------- 8:30-10:00 Room A: Dynamic - voting 10:00-10:30 Break 10:30-12:00 Room A: I/O - general reading 12:00-1:00 Lunch 1:00-3:00 Room A: Misc - voting 3:00-3:30 Break 3:30-5:30 Room A: Collective - general reading 6:00-8:00 Dinner 8:00-10:00 Room A: Real-time Room B: One-sided Room C: as needed Thursday -------- 8:30-10:00 Room A: External - general reading 10:00-10:30 Break 10:30-12:00 Room A: Bindings - vote 12:00-1:00 Lunch 1:00-3:00 Room A: I/O Room B: One-sided Room C: as needed 3:00-3:30 Break 3:30-5:30 Room A: Real time - general reading 6:00-8:00 Dinner 8:00-10:00 Room A: as needed Room B: as needed Room C: as needed Friday ------ 8:30-10:00 Room A: One-sided - voting 10:00-10:15 Break 10:15-12:00 Room A: One-sided - voting 12:00 Adjourn From mpi-core-human@mcs.anl.gov Sun Sep 1 17:17:29 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA27211; Sun, 1 Sep 1996 17:17:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA16387 for mpi-core-out; Sun, 1 Sep 1996 16:21:10 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA16378 for ; Sun, 1 Sep 1996 16:21:00 -0500 Message-Id: <199609012121.QAA16378@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: updated tentative agenda Date: Sun, 01 Sep 1996 16:20:58 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The agenda in the preceding message was incomplete. Here is a slightly improved one: Tuesday --------- 1:30-2:00 Discussion of procedures, agenda, and Intro 2:00-3:30 Room A: One-sided Room B: I/O Room C: Bindings 3:30-4:00 Break 4:00-5:30 Room A: Dynamic Room B: External Room C: Collective 6:00-8:00 Dinner 8:00-10:00 Room A: I/O Room B: Misc Room C: as needed Wednesday -------- 8:30-10:00 Room A: Dynamic - voting 10:00-10:30 Break 10:30-12:00 Room A: I/O - general reading 12:00-1:00 Lunch 1:00-3:00 Room A: Misc - voting 3:00-3:30 Break 3:30-5:30 Room A: Collective - general reading 6:00-8:00 Dinner 8:00-10:00 Room A: Real-time Room B: One-sided Room C: as needed Thursday -------- 8:30-10:00 Room A: External - general reading 10:00-10:30 Break 10:30-12:00 Room A: Bindings - vote 12:00-1:00 Lunch 1:00-3:00 Room A: I/O Room B: One-sided Room C: as needed 3:00-3:30 Break 3:30-5:30 Room A: Real time - general reading 6:00-8:00 Dinner 8:00-10:00 Room A: I/O Room B: as needed Room C: as needed Friday ------ 8:30-10:00 Room A: One-sided - voting 10:00-10:15 Break 10:15-12:00 Room A: One-sided - voting 12:00 Adjourn From mpi-core-human@mcs.anl.gov Mon Sep 2 16:24:17 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA06714; Mon, 2 Sep 1996 16:24:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA25010 for mpi-core-out; Mon, 2 Sep 1996 15:27:06 -0500 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA24905; Mon, 2 Sep 1996 15:11:12 -0500 Message-Id: <199609022011.PAA24905@antares.mcs.anl.gov> Received: from ry20 (actually ry20.rz.uni-karlsruhe.de) by nz11.rz.uni-karlsruhe.de with SMTP (PP); Mon, 2 Sep 1996 18:40:41 +0200 Received: from ry71.rz.uni-karlsruhe.de by ry20 with SMTP (1.36.108.7/16.2) id AA02072; Sun, 1 Sep 1996 04:15:28 +0200 Subject: MPI-2 section 8.3 To: hanson@houston.vni.com, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Date: Mon, 2 Sep 1996 17:54:41 +0200 (CES) From: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 7bit Content-Length: 5915 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk From: Michael Hennecke To: Richard Hanson, mpi-bind, mpi-core Subject: MPI-2 section 8.3 - Proposal from Richard Hanson Date: 1996-09-02 Dear Richard, dear MPIF, having worked on proper F90 bindings to MPI for quite a while, I read with interest the proposal from VNI which made it into the 1996-08-26 draft for MPI-2 at pages 213+. Although the idea of introducing a derived type to circumvent the problems with MPI arguments seems to be a good way to go, I have some doubts that the proposal can be implemented on the Fortran 95 language level. The problems I see arise when looking into the assignment overloadings: extending the example given in the draft, suppose we have MODULE MPI2 TYPE(MPI_message) INTEGER :: flag = 0 ! ... 1 is INTEGER(:), 2 is REAL(:) ... INTEGER, POINTER :: i_arr(:) REAL, POINTER :: r_arr(:) END TYPE(MPI_message) INTERFACE ASSIGNMENT(=) MODULE PROCEDURE real2mpi, mpi2real MODULE PROCEDURE int2mpi, mpi2int END INTERFACE ASSIGNMENT(=) CONTAINS ! ... all the specific procedures ... END MODULE MPI2 and an application like this: USE MPI2 TYPE(MPI_message) :: msend REAL :: fsend(100) ! ... TARGET missing, see below ... ! ... sender ... msend=fsend CALL MPI_SEND(msend, dest,tag,comm) We want to use POINTER assignment to the MPI_message to avoid copying. The specific procedure to store the pointer to FSEND in MSEND can be: SUBROUTINE real2mpi ( lhs, rhs ) TYPE(MPI_message), INTENT(OUT) :: lhs REAL, TARGET, INTENT(IN) :: rhs(:) ! ... TARGET required ... lhs%flag = 2 lhs%r_arr => rhs END SUBROUTINE real2mpi Note that for the pointer assignment, TARGET for RHS is needed (or POINTER, but this makes no sense)! If we now start reading section 12.4.1.1 of the Fortran95 DIS about argument association, it is discovered that TARGET for the actual argument FSEND is not **required** by the letter of the standard, but working through the long list of 12.4.1.1 we realize that [F95, 201:6+] "If the dummy argument has the TARGET attribute and the corresponding actual argument does not have the TARGET attribute **or** is an array section with a vector subscript, any pointers associated with the dummy argument become undefined when execution of the procedure completes." So with the example as it stands in the MPI-2 draft (and above), we'll end up with a program that compiles well but looses the pointer association status we need immediately when REAL2MPI returns, simply because the programmer forgot the TARGET attribute for FSEND. Ooops... This is the first severe restriction: ALL ORIGINAL F95 SEND BUFFERS MUST HAVE THE TARGET (OR POINTER) ATTRIBUTE! FORGETTING THIS ATTRIBUTE MAY (no, will) CAUSE NASTY BUGS. Let's now look into the receiver: it's obvious that we also want to avoid copying, but that's impossible with Fortran95: Even if we give FRECV the POINTER attribute as in USE MPI2 TYPE(MPI_message) :: mrecv REAL, POINTER :: frecv(:) ! ... reciever ... frecv=mrecv CALL MPI_RECV(mrecv, source,tag,comm) we cannot overload the assignment because the LHS of ASSIGNMENT(=) must have INTENT(OUT) or INTENT(INOUT). But INTENT cannot be specified for POINTER dummy arguments (like LHS in our MPI2REAL), and INTENT(INOUT) is **not** the same as not specifying any intent. This is the second restriction: ASSIGNMENT OVERLOADING FOR MPI_MESSAGE RECEIVES MUST COPY THE BUFFER. The requirement to allow INTENT for POINTERs is in the F2000 requirements queue, but I doubt that this is of any use for MPI-2 :-) Best regards, Michael Hennecke PS: I might again point out that it **is** possible to provide proper generic interfaces to the MPI procedures, as described in http://www.uni-karlsruhe.de/~Michael.Hennecke/Publications/#MPI_F90 The count of specific procedures in the MPI-2 draft is not correct, since we do not need one version for each of the >=7 array dimensions: a BUF(*) dummy can be associated with A1(1), A2(1,1), A3(1,1,1), A4(1,1,1,1) in full conformance to F77 and F90, only for a scalar BUF we need a new specific procedure. On IBMs XLF and MPI 1.1, I count 35 choice procedures times (1 char + 4 int + 4 logical + 3 real + 3 complex) = 525 specific procedures, or twice = 1050 if scalar BUFs are included. This is large but not astronomically large. I have meanwhile developed a ``light'' version of my MPI module which does not include the extra stub calls for the procedures. For time-critical applications, this version can be used by replacing -I//mpi_f90/ -L// -l mpi_f90 with -I//mpi_f90_nochoice/ -L// -l mpi_f90_nochoice (and forcing the f90 compiler to be less strict in type checking). This eliminates the extra procedure calls, and the remaining interface blocks for non-choice arguments should not have any run-time costs. Should MPIF choose not to include such a solution in the MPI-2 document, then I would like to ask to at least reference the above work so people can generate a F90 binding themselves without too much effort. The sources (and soon a real Makefile) are publicly available. ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) No longer on BITNET :-( Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Tue Sep 3 12:11:50 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA02362; Tue, 3 Sep 1996 12:11:48 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA06552 for mpi-core-out; Tue, 3 Sep 1996 11:10:01 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA06537 for ; Tue, 3 Sep 1996 11:09:43 -0500 Message-Id: <199609031609.LAA06537@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: revised, updated, adjusted agenda Date: Tue, 03 Sep 1996 11:09:42 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk If you receive this, you are probably going to be late for the meeting. Rusty Room A = Salon A Room B = 4605 Room C = 4624 Tuesday --------- 1:30-2:00 Discussion of procedures, agenda, and Intro 2:00-3:30 Room A: One-sided Room B: I/O Room C: Bindings 3:30-4:00 Break 4:00-5:30 Room A: Dynamic Room B: External Room C: Collective 6:00-8:00 Dinner 8:00-10:00 Room A: I/O Room B: Misc Room C: as needed Wednesday -------- 8:30-10:00 Room A: Realtime - general reading 10:00-10:30 Break 10:30-12:00 Room A: I/O - general reading 12:00-1:00 Lunch 1:00-3:00 Room A: External - general reading (vote on parts) 3:00-3:30 Break 3:30-5:30 Room A: Collective - general reading 6:00-8:00 Dinner 8:00-10:00 Room A: Real-time Room B: One-sided Room C: as needed Thursday -------- 8:30-10:00 Room A: Misc - vote 10:00-10:30 Break 10:30-12:00 Room A: Dynamic - vote 12:00-1:00 Lunch 1:00-3:00 Room A: I/O Room B: One-sided Room C: as needed 3:00-3:30 Break 3:30-5:30 Room A: Bindings - vote (read Fortran) 6:00-8:00 Dinner 8:00-10:00 Room A: I/O Room B: as needed Room C: as needed Friday ------ 8:30-10:00 Room A: One-sided - voting 10:00-10:15 Break 10:15-12:00 Room A: External - voting 12:00 Adjourn From mpi-core-human@mcs.anl.gov Thu Sep 5 20:54:28 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA14743; Thu, 5 Sep 1996 20:54:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA04572 for mpi-core-out; Thu, 5 Sep 1996 19:57:06 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.22.180]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA04535; Thu, 5 Sep 1996 19:55:46 -0500 From: Andrew.Lumsdaine@nd.edu Received: from owl.lsc.nd.edu (owl.lsc.nd.edu [129.74.25.119]) by lsc.nd.edu (8.7.5/8.7.3) with ESMTP id TAA25853; Thu, 5 Sep 1996 19:55:43 -0500 (EST) Received: (from lums@localhost) by owl.lsc.nd.edu (8.7.4/8.7.3) id TAA02398; Thu, 5 Sep 1996 19:55:42 -0500 (EST) Date: Thu, 5 Sep 1996 19:55:42 -0500 (EST) Message-Id: <199609060055.TAA02398@owl.lsc.nd.edu> To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: C++ bindings Reply-to: Andrew.Lumsdaine@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A thought occurred to me on my drive back to South Bend this evening. The C bindings can not substitute fully for C++ bindings for the simple reason that ANSI C++ has native data types that are not part of C, namely long long, long double, Complex, Complex, and Complex. Cheers, Andrew Lumsdaine (from the ancient Celtic meaning one who thinks slowly on his feet) From mpi-core-human@mcs.anl.gov Fri Sep 6 15:55:18 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA24468; Fri, 6 Sep 1996 15:55:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA17537 for mpi-core-out; Fri, 6 Sep 1996 13:29:16 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA17495; Fri, 6 Sep 1996 13:28:05 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id NAA22419; Fri, 6 Sep 1996 13:28:16 -0500 Date: Fri, 6 Sep 1996 13:28:16 -0500 (CDT) From: Tony Skjellum To: mpi-core@mcs.anl.gov, mpi-comm@mcs.anl.gov cc: mpi-coll@mcs.anl.gov, mpi-realtime@mcs.anl.gov Subject: FYI Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk MPI Forum: I just wanted to thank everyone who expressed concern about my recent illness at the MPI Meeting. I am much better now, and I will be in touch with those of you who covered for me on Collective and Realtime. These will proceed apace for the next meeting: Andrew will send minutes for Collective soon, and Arkady for Realtime. I had more visitors and helpers at the hospital from the MPI Forum than could have possibly been expected ... it was some experience to be sure. Anyway, I was allowed to leave without staying their too long, and I was OK to travel on Thursday, and am 99% recovered today [Friday]. For realtime aficionados, we will be setting an interim MPI/RT meeting for late September. Arkady will announce in minutes on the realtime reflector. People interested in channels under the collective chapter should also refer to the channel activity in the realtime chapter to see how further evolution has gone on there. Thanks again for your concern, Tony Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Sun Sep 8 23:02:19 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA08486; Sun, 8 Sep 1996 23:02:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA11508 for mpi-core-out; Sun, 8 Sep 1996 22:03:04 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA11503 for ; Sun, 8 Sep 1996 22:02:57 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id WAA08738; Sun, 8 Sep 1996 22:02:56 -0500 Date: Sun, 8 Sep 1996 22:02:56 -0500 Message-Id: <199609090302.WAA08738@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: deadlines Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As was discussed at the meeting last week, we need to keep the deadlines for the October meeting and Supercomputing in mind. We have only 5 weeks until the October meeting so things will be tight. Given the tight deadlines, it would be greatly appreciated if people proof read as much of the document as possible in the next few weeks. It will be hard to make lots of changes after the next meeting. Remember, this will be a public release so we want it to look good. Here are the document deadlines for the October meeting: Friday, September 13: Informal date for chapter authors to get out a new draft of their chapters which reflect changes from the August meeting. Tuesday, September 24: A first cut at the next version of each chapter is due to me by 12:00 PM Central time (noon). When the full document is ready it will be announced on mpi-core and should be out by late afternoon. Chapter authors are locked out from making any edits until I announce the new release. Tuesday, October 1: The final version of each chapter is due to me by 10:00 AM Central time. This is the version that will be printed for the meeting. It will also be announced on mpi-core and should be out by late afternoon. Tuesday, October 8: MPI Forum meeting begins. Here are the deadlines for the Supercomputing release: Friday, October 11: MPI Forum meeting ends. Friday, October 18: Updated chapters are due to me by 12:00 PM Central time (noon). When the full document is ready it will be announced on mpi-core and should be out by late afternoon. Chapter authors are locked out from making any edits until I announce the new release. Chapter authors may also release their chapters individually. Friday, October 25: "Final" version of each chapter due to me by 12:00 PM Central time (noon). At this point I will begin editing the document for the Supercomputing release. Friday, November 1: By this day, I will release the full document to the Forum for review. Friday, November 8: All changes to chapters are due to me by 10:00 AM Central time. ONLY SMALL CHANGES TO FIX INTEGRATION PROBLEMS WILL BE MADE FROM NOVEMBER 1 UNTIL NOVEMBER 8. Monday, November 11: The Supercomputing version will be released. It will be announced to core, printed, and announced on the appropriate newsgroups.. The current (and future documents) are available via the web. See http://www.cs.wisc.edu/~lederman/mpi2 and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2. The full current document is in mpi2-report.ps.Z. The sources and scripts are also available. The official votes through the September meeting are available in the file votes-official. Steve From mpi-core-human@mcs.anl.gov Mon Sep 9 00:01:50 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA08755; Mon, 9 Sep 1996 00:01:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA12046 for mpi-core-out; Sun, 8 Sep 1996 23:04:53 -0500 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA12041 for ; Sun, 8 Sep 1996 23:04:44 -0500 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.18.106]) by franklin.sdsc.edu (8.7.5/8.7.3/SDSCserver-12) with SMTP id VAA03802; Sun, 8 Sep 1996 21:04:42 -0700 (PDT) Received: from localhost by rogue.sdsc.edu (940406.SGI/1.11-client) with SMTP id AA05612; Sun, 8 Sep 96 21:04:41 -0700 Date: Sun, 8 Sep 1996 21:04:41 -0700 (PDT) From: Richard Frost To: Steve Huss-Lederman Cc: mpi-core@mcs.anl.gov Subject: Re: deadlines and "XDR-like" functionalities In-Reply-To: <199609090302.WAA08738@romano.cs.wisc.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Given the tight deadlines, I propose that we adopt XDR RFC 1832 as a standard for the architecture-neutral encoding options discussed in MPI-2 I/O, external, and elsewhere. The spec can be reviewed at http://ds.internic.net/rfc/rfc1832.txt To be complete, we need to specify the mapping between MPI types and XDR encodings. Which chapter should this be submitted to? - Richard From mpi-core-human@mcs.anl.gov Mon Sep 9 00:48:35 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA09046; Mon, 9 Sep 1996 00:48:34 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA12341 for mpi-core-out; Sun, 8 Sep 1996 23:52:16 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA12336; Sun, 8 Sep 1996 23:52:08 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.7.5/NAS.6.1) id VAA11581; Sun, 8 Sep 1996 21:52:06 -0700 (PDT) Date: Sun, 8 Sep 1996 21:52:06 -0700 (PDT) From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199609090452.VAA11581@win233.nas.nasa.gov> To: mpi-bind@mcs.anl.gov Subject: question about C bindings Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A question has come up about C bindings for new functions. I currently have (at the request of someone, I think) char foo[] instead of char *foo where foo is a null-terminated string. So my questions are: is there a semantic difference, which one should we use, and why? The other question has to do with const. Should input strings be declared with const? I understand there was an MPI-1 decision against const, but I don't know what the reasons were and if they apply to string declarations. Opinions/facts welcome. Bill From mpi-core-human@mcs.anl.gov Mon Sep 9 18:10:38 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA23587; Mon, 9 Sep 1996 18:08:31 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA25147 for mpi-core-out; Mon, 9 Sep 1996 15:36:04 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id PAA25114; Mon, 9 Sep 1996 15:35:28 -0500 Received: from windsurf.kgn.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA68357; Mon, 9 Sep 1996 16:32:02 -0400 Message-Id: <32347F8C.2781@kgn.ibm.com> Date: Mon, 09 Sep 1996 16:35:24 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 2.01 (X11; I; AIX 1) Mime-Version: 1.0 To: mpi-misc@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Homogeneous??? Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There have been many comments made during MPI-2 deliberations that "If I know I am in a homogeneous environment I can do xxxx." One example is: If I know I am in a homogeneous environment I can get the effect of MPI_Contiguous_struct by simply sending/receiving with extent(struct) as a count and MPI_BYTE as type. Of course, this is not something anyone can "know" in a portable way. Has there been any discussion of prescribing a C #define / Fortran parameter which would be in the mpi.h/mpif.h header to say "This MPI implementation only supports homogeneous environments." With such a flag, a user could write code which does things one way if the environment is guaranteed homogeneous and another if there is potential mixing. I do not consider this idea very pretty but if it allows some codes to do things in a faster way without becoming nonportable it may be worthwhile. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@kgn.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.kgn.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Mon Sep 9 18:28:38 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA24127; Mon, 9 Sep 1996 18:28:37 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA27574 for mpi-core-out; Mon, 9 Sep 1996 17:32:04 -0500 Received: from relay.hp.com (relay.hp.com [15.255.152.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA27569; Mon, 9 Sep 1996 17:31:55 -0500 Received: from tbag.rsn.hp.com by relay.hp.com with ESMTP (1.37.109.16/15.5+ECS 3.3) id AA143958297; Mon, 9 Sep 1996 15:31:37 -0700 Message-Id: <199609092231.AA143958297@relay.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA114508296; Mon, 9 Sep 1996 17:31:36 -0500 From: Raja Daoud Subject: Re: Homogeneous??? To: garyo@avs.com (Gary Oberbrunner) Date: Mon, 09 Sep 1996 17:31:36 CDT Cc: treumann@kgn.ibm.com, mpi-misc@mcs.anl.gov, mpi-core@mcs.anl.gov In-Reply-To: <199609092146.RAA08400@darkstar.avs.com>; from "Gary Oberbrunner" at Sep 9, 96 5:46 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Heck, if you're going to go to the trouble, a runtime call makes it a > lot more flexible (same binary/library gets faster in hetero env). > And of course if it's too much trouble, the implementation just always > says "hetero", and you have to code it right. :-) Actually, it could be a communicator attribute, or for lower latency, a function (or a first in MPI, a macro). This way as communicators are split/joined, the flag is set/reset. Ah, what beauty! All are OK by me, the flag's going to be hardcoded to 1 for our MPI. :-) --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com Convex Division From mpi-core-human@mcs.anl.gov Mon Sep 9 19:54:46 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA24542; Mon, 9 Sep 1996 19:54:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA26766 for mpi-core-out; Mon, 9 Sep 1996 16:47:11 -0500 Received: from caffeine.avs.com (caffeine.avs.com [192.138.253.107]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id QAA26748; Mon, 9 Sep 1996 16:46:49 -0500 Received: from darkstar.avs.com by caffeine.avs.com (5.x/SMI-SVR4) id AA13041; Mon, 9 Sep 1996 17:46:16 -0400 Received: by darkstar.avs.com (SMI-8.6/SMI-SVR4) id RAA08400; Mon, 9 Sep 1996 17:46:16 -0400 Message-Id: <199609092146.RAA08400@darkstar.avs.com> Date: Mon, 9 Sep 1996 17:46:16 -0400 From: Gary Oberbrunner To: Dick Treumann Cc: mpi-misc@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Homogeneous??? In-Reply-To: <32347F8C.2781@kgn.ibm.com> References: <32347F8C.2781@kgn.ibm.com> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Date: Mon, 09 Sep 1996 16:35:24 -0400 > From: Dick Treumann > ... > Has there been any discussion of prescribing a C #define / Fortran > parameter which would be in the mpi.h/mpif.h header to say "This > MPI implementation only supports homogeneous environments." Heck, if you're going to go to the trouble, a runtime call makes it a lot more flexible (same binary/library gets faster in hetero env). And of course if it's too much trouble, the implementation just always says "hetero", and you have to code it right. :-) I agree, not too pretty, but could be useful. -- Gary Oberbrunner garyo@avs.com Advanced Visual Systems, Inc. http://www.avs.com/~garyo 300 Fifth Avenue (617)890-8192 x2133 TEL Waltham, MA 02154 (617)890-8287 FAX From mpi-core-human@mcs.anl.gov Wed Sep 11 16:11:29 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA27575; Wed, 11 Sep 1996 16:11:28 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA11826 for mpi-core-out; Wed, 11 Sep 1996 15:13:36 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA11821 for ; Wed, 11 Sep 1996 15:13:29 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id PAA02751; Wed, 11 Sep 1996 15:13:27 -0500 Date: Wed, 11 Sep 1996 15:13:27 -0500 Message-Id: <199609112013.PAA02751@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: sending postscript files Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There has been several occasions where a discussion has ensued about sending postscript to the MPI mail reflectors. The decision of the chapter authors is that it will be up each author to decide whether to send out postscript to the group. Some people prefer this system while others don't. To allow the option of not sending postscript and to allow people who may need later access to distributed documents, I have set up an anonymous ftp location. What is placed here is determined by the chapter authors. Official releases will still be made available in the original (and different) location. The WWW address is ftp://ftp.cs.wisc.edu/pub/mpi2. You can also access via anonymous ftp by: 1) ftp ftp.cs.wisc.edu 2) log on as anonymous 3) cd pub/mpi2 4) get the files you want At the current time no files are available. Chapter authors will announce their existence once s/he has placed them in the anonymous ftp repository. Let me know if you experience any problems. Steve From mpi-core-human@mcs.anl.gov Wed Sep 11 20:17:44 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA29755; Wed, 11 Sep 1996 20:17:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA17102 for mpi-core-out; Wed, 11 Sep 1996 19:20:33 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA17093 for ; Wed, 11 Sep 1996 19:20:25 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id RAA05249 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Wed, 11 Sep 1996 17:20:23 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id RAA14751; Wed, 11 Sep 1996 17:18:12 -0700 From: "Eric Salo" Message-Id: <9609111718.ZM14749@mrjones.engr.sgi.com> Date: Wed, 11 Sep 1996 17:18:11 -0700 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: MPI_MEM_ALLOC and MPI_MEM_FREE Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I would like to make the humble suggestion that we move these functions into MPI-1.2, and will propose this formally for a vote at the next meeting. My motives are completely selfish: Our MPI-1 implementation could make very good use of these today to increase performance, and SGI would like to start encouraging application writers to use them. We will do this at any rate, but it will be more difficult if the functions are only part of MPI-2. Arguments for: - Useful now for some high-performance MPI-1 implementations - Trivial to implement as wrappers around malloc() and free() Arguments against: - This is new functionality, not a correction, and so it probably belongs more logically as part MPI-2. Does anyone object strongly to this idea? I'm hoping that maybe I can ride a wave of apathy and get this approved... -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Wed Sep 11 21:34:00 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA00453; Wed, 11 Sep 1996 21:33:59 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA17993 for mpi-core-out; Wed, 11 Sep 1996 20:37:35 -0500 Received: from newton.ncsa.uiuc.edu (newton.ncsa.uiuc.edu [141.142.2.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA17988; Wed, 11 Sep 1996 20:37:28 -0500 Received: from shalom.ncsa.uiuc.edu (shalom.ncsa.uiuc.edu [141.142.21.11]) by newton.ncsa.uiuc.edu (8.6.11/8.6.12) with ESMTP id UAA01964; Wed, 11 Sep 1996 20:37:27 -0500 Received: by shalom.ncsa.uiuc.edu (940816.SGI.8.6.9) id UAA03058; Wed, 11 Sep 1996 20:37:26 -0500 From: "Albert Cheng" Message-Id: <9609112037.ZM3056@shalom.ncsa.uiuc.edu> Date: Wed, 11 Sep 1996 20:37:26 -0500 In-Reply-To: Richard Frost "Re: deadlines and "XDR-like" functionalities" (Sep 8, 9:04pm) References: X-Mailer: Z-Mail (3.1.0 22feb94 MediaMail) To: mpi-io@mcs.anl.gov Subject: Re: deadlines and "XDR-like" functionalities Cc: mpi-core@mcs.anl.gov Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Sep 8, 9:04pm, Richard Frost wrote: > Subject: Re: deadlines and "XDR-like" functionalities > > Given the tight deadlines, I propose that we adopt XDR RFC 1832 > as a standard for the architecture-neutral encoding options discussed in > MPI-2 I/O, external, and elsewhere. The spec can be reviewed at > > http://ds.internic.net/rfc/rfc1832.txt > > To be complete, we need to specify the mapping between MPI types > and XDR encodings. Which chapter should this be submitted to? XDR defines all data representations in blocks of 4 bytes. That would add a requirement that all etype sizes and displacement must be in multiples of 4 bytes. I am hesistate to impose such a requirement. And there are a few MPI basic datatypes that are not defined in XDR. E.g., MPI_SHORT, MPI_COMPLEX. Also, do MPI_INT and MPI_FLOAT map into XDR's 32 bits integer and floating point, even if the host machine, like Cray, uses 64 bits for those two types? Does an MPI_LONG map to XDR's integer or hyper integer? I think in order to define "MPI_DATA_XDR" access mode, we need to borrow something from XDR (floating points are in IEEE representation, byte orders are in Big-Endian order), and map every MPI basic type to defined byte sizes and orders (like typemap). We would need to deal with alignment problems such as "is there any unused bytes for {int, char, int}?" =Albert -- From mpi-core-human@mcs.anl.gov Wed Sep 11 22:04:08 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA00788; Wed, 11 Sep 1996 22:04:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA18385 for mpi-core-out; Wed, 11 Sep 1996 21:07:50 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA18372; Wed, 11 Sep 1996 21:07:34 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id VAA22111; Wed, 11 Sep 1996 21:07:42 -0500 Date: Wed, 11 Sep 1996 21:07:42 -0500 (CDT) From: Tony Skjellum To: Albert Cheng cc: mpi-io@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: deadlines and "XDR-like" functionalities In-Reply-To: <9609112037.ZM3056@shalom.ncsa.uiuc.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There is an international standard, ASN.1 BER (binary encoding rules). These seem to be more general than XDR; ELROS (LLNL) used this for automated heterogeneous conversion between Crays and workstations several years ago, before MPI-1 was even complete... -Tony On Wed, 11 Sep 1996, Albert Cheng wrote: > Date: Wed, 11 Sep 1996 20:37:26 -0500 > From: Albert Cheng > To: mpi-io@mcs.anl.gov > Cc: mpi-core@mcs.anl.gov > Subject: Re: deadlines and "XDR-like" functionalities > > On Sep 8, 9:04pm, Richard Frost wrote: > > Subject: Re: deadlines and "XDR-like" functionalities > > > > Given the tight deadlines, I propose that we adopt XDR RFC 1832 > > as a standard for the architecture-neutral encoding options discussed in > > MPI-2 I/O, external, and elsewhere. The spec can be reviewed at > > > > http://ds.internic.net/rfc/rfc1832.txt > > > > To be complete, we need to specify the mapping between MPI types > > and XDR encodings. Which chapter should this be submitted to? > > XDR defines all data representations in blocks of 4 bytes. That > would add a requirement that all etype sizes and displacement must be > in multiples of 4 bytes. I am hesistate to impose such a requirement. > And there are a few MPI basic datatypes that are not defined in XDR. > E.g., MPI_SHORT, MPI_COMPLEX. > > Also, do MPI_INT and MPI_FLOAT map into XDR's 32 bits integer and floating > point, even if the host machine, like Cray, uses 64 bits for those two > types? Does an MPI_LONG map to XDR's integer or hyper integer? > > I think in order to define "MPI_DATA_XDR" access mode, we need to borrow > something from XDR (floating points are in IEEE representation, byte orders > are in Big-Endian order), and map every MPI basic type to defined byte sizes > and orders (like typemap). We would need to deal with alignment problems > such as "is there any unused bytes for {int, char, int}?" > > =Albert > > -- > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Wed Sep 11 23:07:47 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA01269; Wed, 11 Sep 1996 23:07:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA18988 for mpi-core-out; Wed, 11 Sep 1996 22:11:18 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [129.34.139.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA18983 for ; Wed, 11 Sep 1996 22:11:10 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw2.watson.ibm.com (8.7.4/8.7.1) with ESMTP id XAA51646; Wed, 11 Sep 1996 23:11:36 -0400 Received: from watngi05.watson.ibm.com (watngi05.watson.ibm.com [9.2.235.16]) by mailhub1.watson.ibm.com (8.7.1/09-08-96) with SMTP id XAA529760; Wed, 11 Sep 1996 23:11:08 -0400 Received: by watngi05.watson.ibm.com(Lotus SMTP MTA Release 1.0) id 852563A3.00117020 ; Wed, 11 Sep 1996 23:10:28 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: salo@mrjones.engr.sgi.com cc: Mpi-Core@mcs.anl.gov Message-ID: <852563A3:00102365.00@watngi05.watson.ibm.com> Date: Wed, 11 Sep 1996 23:10:22 -0400 Subject: Re: MPI_MEM_ALLOC and MPI_MEM_FREE Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am inclined to suggest that the only new content of MPI 1.2 be the Version number calls. This, in addition to clarifications to MPI1. The reason is that, even if minor changes are made to a library, it still has to go through testing, documentation has to change, etc. I am not sure it is worthwhile do do so just in order to sneak in a new errror code or a few new datatypes. The version number is something that is needed, so as to distinguish "old" MPI libraries from new ones. On the other hand, me may reorganize MPI2, and move MPI_MEM_MALLOC and MPI_MEM_FREE to the "misc" chapter that I suggest to rename "Enhancements to MPI1". We may also move to there parts of external, such as new style attribute manipulation functions. Essentially, this section should contain everything that does not provide new functionality, but facilitates MPI1 coding: everything that replaces deprecated functions, everything that is a no brainer to all vendors but useful to some, everything that codifies existing user practice (such as language interoperability) and avoids multiple ports to implementation specific features of current MPI1 libraries. We may suggest that this part of MPI2 be added to MPI1 by implementors asap. From mpi-core-human@mcs.anl.gov Thu Sep 12 01:18:05 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id BAA02073; Thu, 12 Sep 1996 01:18:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id AAA20089 for mpi-core-out; Thu, 12 Sep 1996 00:21:11 -0500 Received: from hp.com (hp.com [15.255.152.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id AAA20084 for ; Thu, 12 Sep 1996 00:20:59 -0500 Received: from tbag.rsn.hp.com by hp.com with ESMTP (1.37.109.16/15.5+ECS 3.3) id AA277085655; Wed, 11 Sep 1996 22:20:56 -0700 Message-Id: <199609120520.AA277085655@hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA191035655; Thu, 12 Sep 1996 00:20:55 -0500 From: Raja Daoud Subject: Re: MPI_MEM_ALLOC and MPI_MEM_FREE To: mpi-core@mcs.anl.gov Date: Thu, 12 Sep 1996 0:20:55 CDT In-Reply-To: <852563A3:00102365.00@watngi05.watson.ibm.com>; from "Marc Snir" at Sep 11, 96 11:10 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I like Marc's suggestion. It's clean, simple, and shuts the door equally and fairly to all haggling over sliding this or that small functionality into 1.2. Eric, the push to have users favour one way of coding MPI over another in order to get "portable max performance" is currently not attainable and code will have to be tweaked as it moves from one architecture to another (but portable good-enough performance is achievable on the other hand). On the T3D, Cray will probably push for pt2pt 1sided, while MPI-1 pt2pt would be preferred on clusters. Currently for SGI, off-host BW is higher than on-host, but not for HP (curious: do you ask users to de-localize their application to gain performance?). With DMA engines (e.g. SP2) I*send() is better whereas blocking pt2pt is faster on systems that simulate concurrency using signals or daemons. All this to say that, while pushing users toward Mem_alloc() helps you (and HP incidentally :-) ), I'm not at ease adding it to MPI-1.2, especially since we will keep tweaking and #ifdefing code to milk performance out of it. --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com Convex Division From mpi-core-human@mcs.anl.gov Thu Sep 12 05:54:39 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA03033; Thu, 12 Sep 1996 05:54:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA26876 for mpi-core-out; Thu, 12 Sep 1996 04:55:40 -0500 Received: from win233.nas.nasa.gov (win233.nas.nasa.gov [129.99.66.80]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA26871 for ; Thu, 12 Sep 1996 04:55:33 -0500 Received: (from wcs@localhost) by win233.nas.nasa.gov (8.7.5/NAS.6.1) id CAA16443 for mpi-core@mcs.anl.gov; Thu, 12 Sep 1996 02:55:31 -0700 (PDT) Date: Thu, 12 Sep 1996 02:55:31 -0700 (PDT) From: wcs@nas.nasa.gov (William C. Saphir) Message-Id: <199609120955.CAA16443@win233.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: 2nd votes Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At the last meeting we had a couple of serious discussions during what were supposed to be second and final votes. In one case (the dynamic chapter) we voted out a few sections, and in another (the C++ chapter) we ended up postponing the second vote. Several people have expressed amazement that the process could go so far, only to be dealt a setback at the last minute. The purpose of this note is to defend, and encourage, serious consideration at the second vote. I assert (and have heard others assert) that the MPI process does not really allow consideration of global issues until the second vote. In subcommittee straw votes, attempts to remove entire sections are often met with "this is only a straw vote - let's give ourselves a chance to explore this issue - we can always get rid of it later on". In other cases, the proposal is being formed, and technical issues are being hashed out, and it can be quite difficult to step back and look at the global picture. Likewise, I assert that the first vote is pretty much a technical vote. When people vote yes on a first vote, what I think they are usually saying is that the technical details are fine. Because there is another vote coming up, people haven't been asking the question "do we really want this whole thing in the first place" since they're not really forced to decide. It is not until the second vote that you really have to ask yourself the big question. The second vote means you're committed. It's also not until the second vote that you have the complete picture. Even for people who know all along that they would like to get rid of a proposal, it might be difficult to make others realize the seriousness of the error they are about to make if it is only a first vote. What's the purpose of a second vote if there's no chance of reversing previous decisions? Finally... In this and in other areas, I would like to try to vote according to the following principle: We should not add anything to MPI for which there is not a compelling motivation and clear need. As an outsider to the MPI-1 process, it was my perception and that of many others that this principle was not invoked often enough. Whether or not that perception is correct, I think it is a useful principle. Adding things to MPI is sometimes a long process but there is no inherent barrier. Removing things is effectively impossible. So I would like to err on the side of not adding enough, rather than adding too much, and if there is a serious deficiency, it can be fixed later. At this point I am not concerned that MPI is lacking critical functionality, or that the lack of certain functionality might doom MPI, so we can afford to take a more relaxed and cautious approach. Just an opinion. Bill From mpi-core-human@mcs.anl.gov Thu Sep 12 08:59:58 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA04776; Thu, 12 Sep 1996 08:59:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA28441 for mpi-core-out; Thu, 12 Sep 1996 08:02:46 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA28436 for ; Thu, 12 Sep 1996 08:02:34 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id IAA24598; Thu, 12 Sep 1996 08:02:32 -0500 Date: Thu, 12 Sep 1996 08:02:32 -0500 Message-Id: <199609121302.IAA24598@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov In-reply-to: <199609120955.CAA16443@win233.nas.nasa.gov> (wcs@nas.nasa.gov) Subject: Re: 2nd votes Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Since the topic of second votes has been explicitly raised, I would like to raise the issue of voting rules. I and a number of other people have discussed (offline from the main group) that the current voting rules can allow items to become part of MPI with minimal support. Currently a 3-2-20 vote passes. We adopted the HPF rules at the start of MPI. Since that time, HPF has modified its voting rules to make it harder for things to pass. (Not active in HPF I can only report this second hand and do not recall the details.) I think the MPI Forum should also consider such a change. Some have pointed out people rightly abstain in MPI-2 because they do not fully understand the issue. Changing the voting rules might penalize this practice. However, I think raising the bar would be useful. Any change will seem arbitrary. However, as a starting point of discussion I propose the following rule: On a SECOND vote, a motion only passes if: - 60% of the yes/no votes are cast yes. - 40% of all votes are yes (including abstains) The first rule means more than a simple majority of those expressing an opinion must be in favor. The second rule means that a motion cannot pass with lots of apathy. After some thought I decided not to propose that the second rule be 40% of voting institutions present. This allows people to bow out if they want. I am open to thoughts and suggestions. There are certain to be ideas that I have missed. Steve From mpi-core-human@mcs.anl.gov Thu Sep 12 09:11:25 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA04995; Thu, 12 Sep 1996 09:11:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA28754 for mpi-core-out; Thu, 12 Sep 1996 08:14:42 -0500 Received: from mbunix.mitre.org (mbunix.mitre.org [129.83.20.100]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA28008 for ; Thu, 12 Sep 1996 07:19:47 -0500 Received: from TGATE3 (tgate3.mitre.org [129.83.20.27]) by mbunix.mitre.org (8.7.5/8.7.3/mitre.0) with SMTP id IAA09382; Thu, 12 Sep 1996 08:16:47 -0400 (EDT) Received: from mail11 (129.83.20.44) by tgate3.mitre.org (EMWAC SMTPRS 0.60) with SMTP id ; Thu, 12 Sep 1996 08:11:01 -0400 Received: by mail11; (5.65v3.2/1.1.8.2/22Jun94-0628PM) id AA08530; Thu, 12 Sep 1996 08:16:28 -0400 Subject: Re: MPI_MEM_ALLOC and MPI_MEM_FREE From: arkady@mail11.mitre.org (Arkady Kanevsky) To: raja@tbag.rsn.hp.com (Raja Daoud), mpi-core@mcs.anl.gov Message-Id: <960912081623.344@mail11.mitre.org.0> Date: Thu, 12 Sep 96 08:16:28 -0400 X-Mailer: MAILworks 1.7-A-2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I also like Mark's recommendation for its simplicity and fairness. Arkady > >I like Marc's suggestion. It's clean, simple, and shuts the door >equally and fairly to all haggling over sliding this or that small >functionality into 1.2. > >Eric, the push to have users favour one way of coding MPI over another >in order to get "portable max performance" is currently not attainable >and code will have to be tweaked as it moves from one architecture to >another (but portable good-enough performance is achievable on the other >hand). On the T3D, Cray will probably push for pt2pt 1sided, while >MPI-1 pt2pt would be preferred on clusters. Currently for SGI, off-host >BW is higher than on-host, but not for HP (curious: do you ask users to >de-localize their application to gain performance?). With DMA engines >(e.g. SP2) I*send() is better whereas blocking pt2pt is faster on >systems that simulate concurrency using signals or daemons. > >All this to say that, while pushing users toward Mem_alloc() helps you >(and HP incidentally :-) ), I'm not at ease adding it to MPI-1.2, >especially since we will keep tweaking and #ifdefing code to milk >performance out of it. > >--Raja > > >-=- >Raja Daoud Hewlett-Packard Co. >raja@rsn.hp.com Convex Division > > From mpi-core-human@mcs.anl.gov Thu Sep 12 09:12:59 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA05029; Thu, 12 Sep 1996 09:12:53 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA28804 for mpi-core-out; Thu, 12 Sep 1996 08:16:10 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA28632 for ; Thu, 12 Sep 1996 08:11:48 -0500 Received: by felix.dircon.co.uk id AA24062 (5.67b/IDA-1.5 for ); Thu, 12 Sep 1996 14:01:09 +0100 Message-Id: <199609121301.AA24062@felix.dircon.co.uk> Received: from gw2-177.pool.dircon.co.uk(194.112.35.177) by amnesiac via smap (V1.3) id sma024056; Thu Sep 12 14:00:56 1996 Received: from localhost by jim (SMI-8.6) id OAA01750; Thu, 12 Sep 1996 14:03:51 +0100 To: mpi-core@mcs.anl.gov Subject: ***Unofficial*** minutes Reply-To: jcownie@dolphinics.com Date: Thu, 12 Sep 1996 14:03:50 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Folks, here is a write up of the notes I took at the last meeting. I have to prepare a report for Esprit anyway to justify their funding half my expenses (and that report gets into the public domain anyway), so I thought other people might also like a copy. These are in no way official minutes, but I hope they have the votes and action points correct. Feel free to berate me if I got something wrong, or to use these in any way you feel like. Enjoy -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com MPI 2 Chicago O'Hare Sept 3-6 ============================= James Cownie 11 Sept This is a report on the MPI meeting held at the Ramada hotel Chicago O'Hare on September 3 - 6. This report only covers the meeting from lunch on September 4, since I was unable to attend the Tuesday meeting having booked flights prior to the decision to meet on Tuesday. Summary ======= One sided has now passed a first formal vote (for the second time !) Dynamic has now passed two formal votes, and (subject to correction of typographical and binding errors) is frozen. External Interfaces ------------------- Starting at 1p.m. Steve Huss-Ledermann presented the chapter for a formal vote. On a point of order James Cownie was allowed a formal vote as he personally had attended 2 of the last three meetings, though Dolphin Interconnect Solutions had not. There were therefore 19 organisations present with official votes. Voting on section 7.2: Generalised requests 7.2.1 Introduction 16: 0: 3 7.2.2 Functionality 18: 1: 3 7.2.3 Deferred pending handler discussions 7.2.4 example => no vote needed Voting on 7.3: Communicator ID Deferred. Voting on 7.4: Accessing MPI datatypes 7.4.1: Amendment to return exactly the info given to create the datatype 17: 0: 6 <=== Amendment passes 7.4.1 as amended 20: 0: 3 7.4.2: Deleted 7.4.3: Deferred for further discussion, over potential problems with char constants > 128, Fortran, ... Voting on 7.5: Packing of datatypes Deferred. Section 7.6: True extent of datatypes deleted, to be replaced by a better solution in the MISC chapter. Voting on 7.7: caching on MPI handles. Deferred, pending resolution of a variety of issues When a handle with attributes is deleted are the calls to the attribute delete functions ordered ? (Particularly when the handle is MPI_COMM_WORLD). There is continuing indecision on whether it makes sense to cache on MPI_GROUP objects. Voting on 7.8: Identifying requests 18: 0: 6 Voting on 7.9: Naming Objects deferred Voting on 7.10: Allowing User Errors. Deferred because of concerns over the meaning when the error handling mode is ERRORS_RETURN. It appears that there probably isn't a real problem, but this needs to be exhaustively checked. Voting on 7.11: Allowing user functions at MPI_FINALIZE. Deferred because of unclarified issues with order of * communicator deletion * attribute delete function invocation * precise definition of Finalize. Marc Snir introduced section 7.12: MPI and threads no votes were taken. 7.12.1: There was discussion on the need for functions MPI_Thread_initialize to be called before a thread can invoke MPI. (MPI_Init also performs MPI_Thread_init for the calling thread, so single threaded code doesn't change). MPI_Thread_finalize for symmetry with Thread_initialize MPI_Thread_initialized to allow a thread to know if it needs to call Thread_initialize 7.12.2: Much discussion on the precise requirements for error handler invocation. Should they be invoked 1) on any thread 2) on the thread which set up the operation 3) on the thread which tests/waits for completion No decision was reached. 7.12.4: Global thread functions. The whole section has been deleted. Broke for coffee 3.p.m. Extended Collective Operations ------------------------------ Andrew Lumsdaine stood in for Tony Skjellum who had got sick and was in hospital. (He's now recovered). All votes here were straw votes. Andrew presented the suggested new operations: Intercommunicator collective operations Non-blocking collective operations Persistent collective operations Miscellaneous new blocking collective operations There was some discussion on the meaning/implementability of cancel on collective operations. The current draft has a logical extension of the pt2pt cancel, however the net effect is that cancel is useless on collective operations. Some people want a more local cancel, however it is extremely unclear if this would be implementable. Straw votes were taken Remove most non-blocking collective operations 17: 6: 6 Remove all ditto 5:15: 9 (The most popular non-blocking collective seems to be the non-blocking barrier.) Persistent non-blocking collectives only match other persistent non-blocking collectives 24: 0: 3 Do we want persistient blocking collective calls 14: 4:11 Section 6.4: Intercommunicator constructors Do we want Comm_create, Comm_split on Intercommunicator 15: 3: 8 Do we want Intercommunicator partition 0:13:14 Section 6.5: Intercommunicator collective ops 19: 2: 5 Do we want in-place operations Simple cases 19: 1: 8 Hard cases 6:13: 8 Section 6.6: Extended functions Do we want persistent blocking functions as detailed here 1: 4:21 (Issues here are to do with when you bind blocking property to a persistent handle : at handle create time, or later). Delete ICALL, ICALL_SPLIT, COMM_DUP_LOCAL, COMM_SPLIT_LOCAL 15: 0:10 (This may have suffered from Tony's absence, since no one could understand what these functions were supposed to do !) Section 6.7: Channels Deferred pending thinking on the possibilty of constructing equivalent functionality by pairing persistent requests. 5-35p.m. adjourned. Miscellaneous chapter --------------------- Thursday 8-30 a.m. Rusty Lusk started formal voting on the Miscellaneous chapter. 21 voting organisations were present. Discussion references a new version of the chapter prepared by Rusty late on Wednesday night, and handed out at the meeting. Section 3.1: Version number An amendment was proposed to remove the Get_version function 1:13:9 (The argument being that all it allows you to do is check that you have linked against the correct library.) Section 3.1 as not amended 18: 0: 2 Section 3.2.1: STATUS_IGNORE 19: 1: 3 Section 3.2.2: Non-destructive test of status 23: 0: 1 Section 3.3: MPI_ERR_KEYVAL 21: 0: 3 Section 3.4: MPI_True_extent Section 3.5: MPI_Set_extent Discussion at length about the implications of changing a property of a datatype, rather than constructing a new one. Implementors disliked modifying an existing datatype since many implementations reference count datatypes, therefore achieving the required semantics could require significant changes to code. Amendment to replace set_extent with a new datatype constructor which would change the extent 15: 0: 6 Section 3.4 and 3.5 as amended 22: 0: 0 Section 3.6: Section 3.6.1:Clarification of finalize. Much discussion on whether finalize must return a) at all b) somewhere c) everywhere Problems arise becuase this is really a binding issue to do with how "an MPI process" is bound to an implementation (e.g. it could be a Posix process, or a thread, or ...). Different choices for finalize semantics depend on this choice of binding. Allow finalize not to return 11: 6: 8 Disallow all MPI calls after Finalize 15: 1: 4 Section deferred, pending further clarification, and possible addition of an MPI_Finalized call. Section 3.6.2: Clarification of Intercomm_create 22: 1: 1 Section 3.6.3: Clarification of MPI_TYPE_SIZE 19: 0: 3 Section 3.6.4: Clarification of MPI_REDUCE Discussion on whether the proposal overly restricts implementations. Amendment to delete lines 34-38 (which specify exactly which functions will be called) and replace with advice to someone which says that any combination of user functions may be called (from one to all). 22: 0: 1 Section 3.6.4 as amended 22: 0: 1 Section 3.7: New datatype constructors Section 3.7.1: Simple_struct the formal definition as given is wrong, the section has been postponed pending a correct definition. 3.7.2: Contiguous struct 6: 7:10 (i.e. rejected) 3.7.3: Indexed_block 10: 7: 7 Process Creation and Management ------------------------------- At 10-45a.m. Bill Saphir introduced this section for second formal vote. (So this should not change again...) Sections 4.3.1, 4.3.2: MPI_SPAWN 22: 0: 3 4.3.3: MPI_SPAWN_MULTIPLE 20: 0: 3 4.3.4: SPAWN_INDEPENDENT Much discussion on this and the associated MONITOR and SIGNAL functions in 4.5.2 and 4.5.3. Vote on deleting 4.3.4 SPAWN_INDEPENDENT 4.3.5 SPAWN_MULTIPLE_INDEPENDENT 4.3.6 non-blocking independent functions 4.5.2 Signalling 4.5.3 Process monitoring 14: 7: 4 <== DELETED Vote to move all of the stuff just deleted to the JOD, as a recommendation to "do it this way if you do it" 19: 1: 5 Miscellanea ----------- Starting after lunch at 1p.m. Steve Huss-Ledermann discussed issues with the MPI_STATUS object (many more functions need to access it, and some user functions need to set values in it). The proposal is to provide functions for setting the fields in an MPI_STATUS. Rusty Lusk then introduced the Misc chapter. Formal votes 23-25 votes. Section 10.1: Portable mpi process startup The command is to be called mpiexec (rather than mpirun) thus avoiding any compatibility issues with existing mpirun commands. The proopsal states that if unspecifed the number of processes requested should be one. An amendment was proposed to leave the default unspecified. 14: 4: 7 Section 10.1 as amended 22: 1: 2 10.3: Language inter-operability Marc Snir 10.3.1, 10.3.2: Intro and assumptions 21: 0: 1 10.3.3: Initialization 23: 0: 1 10.3.4: Handle transfer 22: 0: 0 10.3.5, 10.3.6:Opaque objects and datatypes 23: 0: 2 10.3.7: Addresses deferred 10.3.8, 10.3.9: Groups and communicators 23: 0: 1 10.3.10: Attributes deferred 10.3.11:,12,13: Requests, Error handlers, Reduce operators 23: 0: 0 10.3.14: Status deferred 10.3.15:,16: Constants, inter-language communiction 21: 0: 0 Dynamic ------- At 2-25 p.m. Bill Saphir continued the Dynamic chapter. Section 4.3.6: Non-blocking spawn (with independent functions removed, see above) 12: 1:10 4.4.7: Reserved key values 15: 1: 6 4.4.1: Names, addresses etc 15: 0: 8 4.4.2: Server routines 13: 0:10 4.4.3: Client routines 11: 0:11 4.4.4: Name publishing 11: 0:12 The function names are likely to change to Service_publish, Service_retract or Service_unpublish 4.4.5: Iaccept 12: 1:11 Iconnect 9: 3:12 4.4.6: Reserved keys Amendement to add "ip_address" as an additional reserved key 12: 0:11 4.4.6 as amended 13: 0:10 C++ Bindings ------------ At 3-40 p.m. Andrew Lumsdaine attempted to present the C++ bindings. There was a great deal of discussion as to whether MPI needs C++ bindings, or whether C++ programmers should simply use the existing C bindings. Discussion continued at such length that there was no time for any formal votes on the C++ bindings. An extremely informal straw poll was held on whether to keep the C++ bindings. This was slightly in favour of keeping them, but without a majority. (When I asked for this vote I explicitly stated that I would not publish the results, so you'll have to ask me in person if you want the numbers !) Fortran 90 Bindings ------------------- At 4-30p.m. Bill Spahir presented the current state of the F90 bindings. There are significant outstanding issues here, in particular because * F90 is a strongly type-checked language without any escape routes, * F90 explicitly does not say anything about the layout of data in store, while MPI has an implicit model of store layout. There are a number of possible approaches, but none achieves full MPI and full F90. Further work is clearly required. One Sided Communications ------------------------ At 8-30a.m. on Friday Marc Snir introduced the newly revised version of the One sided chapter for a first formal vote. (The version of one-sided which had previously received a first vote has been abandoned). 24 voting organisations present. Instead of a communicator the one-sided operations now take a window object. Section 1.2: Initialisation There was discussion on whether an info object was the correct way to pass optional/additional information into the RMA_init call. No amendment was proposed, so an info argument remains. There was discussion on the use (and sematics) of the displacement argument. Amendment to delete the displacement argument 3:14: 7 Amendment to replace the displacement argument with a datatype whos extent would be used 3:16: 5 1.2 as not amended 21: 2: 1 Section 1.3: Communication calls SGI proposed an amendment to re-order the arguments to the calls so that the buffer came after the datatype 3:14: 6 Amendment that Fortran character data need not be supported in these calls 12: 4: 7 1.3 as amended 21: 0: 3 Section 1.4: Synchronization calls Much discussion in 1.4.3 on the fact that this section (which allows third party transfers) requires an active agent at each node. Amendment Locks are restricted to windows which are in MEM_ALLOCed windows 16: 7: 2 (This allows a simpler implementation on machines which can share some regions of the address space, but not all of it). 1.4 as amended 21: 1: 2 Section 1.5: Progress There was a discussion of the progress rules. As written these are stronger than some people's view of the MPI-1 rules, while conforming to other people's view. (This really means that the MPI-1 progress rules are open to interpretation). Amendment The progress rules are the same as in MPI-1 19: 1: 4 Whole one-sided chapter 20: 2: 3 Meeting dates ------------- At 10-30a.m. Rusty Lusk presented the possible meeting dates October 8-11 (Tuesday-Friday) confirmed January 21-23 (Tuesday-Thursday) tentative March 5- 7 (Wednesday-Friday) " April 23-25 (Wednesday-Friday) " There was some question of moving to California for these meetings to avoid getting snowed in at O'Hare. California 11 Chicago 13 So the meetings will remain at the Ramada O'Hare. Miscellany ---------- At 11-05 Marc Snir presented the difficult inter-operability problems in a 64 bit environment (section 10.3.7). Much discussion ensued, Eric Salo will draft an alternative that provides 5 Fortran functions with non-portable bindings. (I.e. the user may have to change her source code between different conforming MPI implementations if she uses these functions in F77). A straw vote on the idea of these (without seeing them) 7: 5: 10 Section 10.3.10: Attributes. No conclusion was reached a possible alternative would be to have new functions. At 11-45a.m. Rusty introduced 10.7 on continuable errors. At 11-50a.m. Bill Saphir discussed the MPI_APPNUM attribute which would allow an application to work out how many similar processes there are to it when started in a multi-executable job. Meeting closed at noon. From mpi-core-human@mcs.anl.gov Thu Sep 12 09:13:27 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA05094; Thu, 12 Sep 1996 09:13:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA28812 for mpi-core-out; Thu, 12 Sep 1996 08:16:26 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA28632 for ; Thu, 12 Sep 1996 08:11:48 -0500 Received: by felix.dircon.co.uk id AA24062 (5.67b/IDA-1.5 for ); Thu, 12 Sep 1996 14:01:09 +0100 Message-Id: <199609121301.AA24062@felix.dircon.co.uk> Received: from gw2-177.pool.dircon.co.uk(194.112.35.177) by amnesiac via smap (V1.3) id sma024056; Thu Sep 12 14:00:56 1996 Received: from localhost by jim (SMI-8.6) id OAA01750; Thu, 12 Sep 1996 14:03:51 +0100 To: mpi-core@mcs.anl.gov Subject: ***Unofficial*** minutes Reply-To: jcownie@dolphinics.com Date: Thu, 12 Sep 1996 14:03:50 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Folks, here is a write up of the notes I took at the last meeting. I have to prepare a report for Esprit anyway to justify their funding half my expenses (and that report gets into the public domain anyway), so I thought other people might also like a copy. These are in no way official minutes, but I hope they have the votes and action points correct. Feel free to berate me if I got something wrong, or to use these in any way you feel like. Enjoy -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com MPI 2 Chicago O'Hare Sept 3-6 ============================= James Cownie 11 Sept This is a report on the MPI meeting held at the Ramada hotel Chicago O'Hare on September 3 - 6. This report only covers the meeting from lunch on September 4, since I was unable to attend the Tuesday meeting having booked flights prior to the decision to meet on Tuesday. Summary ======= One sided has now passed a first formal vote (for the second time !) Dynamic has now passed two formal votes, and (subject to correction of typographical and binding errors) is frozen. External Interfaces ------------------- Starting at 1p.m. Steve Huss-Ledermann presented the chapter for a formal vote. On a point of order James Cownie was allowed a formal vote as he personally had attended 2 of the last three meetings, though Dolphin Interconnect Solutions had not. There were therefore 19 organisations present with official votes. Voting on section 7.2: Generalised requests 7.2.1 Introduction 16: 0: 3 7.2.2 Functionality 18: 1: 3 7.2.3 Deferred pending handler discussions 7.2.4 example => no vote needed Voting on 7.3: Communicator ID Deferred. Voting on 7.4: Accessing MPI datatypes 7.4.1: Amendment to return exactly the info given to create the datatype 17: 0: 6 <=== Amendment passes 7.4.1 as amended 20: 0: 3 7.4.2: Deleted 7.4.3: Deferred for further discussion, over potential problems with char constants > 128, Fortran, ... Voting on 7.5: Packing of datatypes Deferred. Section 7.6: True extent of datatypes deleted, to be replaced by a better solution in the MISC chapter. Voting on 7.7: caching on MPI handles. Deferred, pending resolution of a variety of issues When a handle with attributes is deleted are the calls to the attribute delete functions ordered ? (Particularly when the handle is MPI_COMM_WORLD). There is continuing indecision on whether it makes sense to cache on MPI_GROUP objects. Voting on 7.8: Identifying requests 18: 0: 6 Voting on 7.9: Naming Objects deferred Voting on 7.10: Allowing User Errors. Deferred because of concerns over the meaning when the error handling mode is ERRORS_RETURN. It appears that there probably isn't a real problem, but this needs to be exhaustively checked. Voting on 7.11: Allowing user functions at MPI_FINALIZE. Deferred because of unclarified issues with order of * communicator deletion * attribute delete function invocation * precise definition of Finalize. Marc Snir introduced section 7.12: MPI and threads no votes were taken. 7.12.1: There was discussion on the need for functions MPI_Thread_initialize to be called before a thread can invoke MPI. (MPI_Init also performs MPI_Thread_init for the calling thread, so single threaded code doesn't change). MPI_Thread_finalize for symmetry with Thread_initialize MPI_Thread_initialized to allow a thread to know if it needs to call Thread_initialize 7.12.2: Much discussion on the precise requirements for error handler invocation. Should they be invoked 1) on any thread 2) on the thread which set up the operation 3) on the thread which tests/waits for completion No decision was reached. 7.12.4: Global thread functions. The whole section has been deleted. Broke for coffee 3.p.m. Extended Collective Operations ------------------------------ Andrew Lumsdaine stood in for Tony Skjellum who had got sick and was in hospital. (He's now recovered). All votes here were straw votes. Andrew presented the suggested new operations: Intercommunicator collective operations Non-blocking collective operations Persistent collective operations Miscellaneous new blocking collective operations There was some discussion on the meaning/implementability of cancel on collective operations. The current draft has a logical extension of the pt2pt cancel, however the net effect is that cancel is useless on collective operations. Some people want a more local cancel, however it is extremely unclear if this would be implementable. Straw votes were taken Remove most non-blocking collective operations 17: 6: 6 Remove all ditto 5:15: 9 (The most popular non-blocking collective seems to be the non-blocking barrier.) Persistent non-blocking collectives only match other persistent non-blocking collectives 24: 0: 3 Do we want persistient blocking collective calls 14: 4:11 Section 6.4: Intercommunicator constructors Do we want Comm_create, Comm_split on Intercommunicator 15: 3: 8 Do we want Intercommunicator partition 0:13:14 Section 6.5: Intercommunicator collective ops 19: 2: 5 Do we want in-place operations Simple cases 19: 1: 8 Hard cases 6:13: 8 Section 6.6: Extended functions Do we want persistent blocking functions as detailed here 1: 4:21 (Issues here are to do with when you bind blocking property to a persistent handle : at handle create time, or later). Delete ICALL, ICALL_SPLIT, COMM_DUP_LOCAL, COMM_SPLIT_LOCAL 15: 0:10 (This may have suffered from Tony's absence, since no one could understand what these functions were supposed to do !) Section 6.7: Channels Deferred pending thinking on the possibilty of constructing equivalent functionality by pairing persistent requests. 5-35p.m. adjourned. Miscellaneous chapter --------------------- Thursday 8-30 a.m. Rusty Lusk started formal voting on the Miscellaneous chapter. 21 voting organisations were present. Discussion references a new version of the chapter prepared by Rusty late on Wednesday night, and handed out at the meeting. Section 3.1: Version number An amendment was proposed to remove the Get_version function 1:13:9 (The argument being that all it allows you to do is check that you have linked against the correct library.) Section 3.1 as not amended 18: 0: 2 Section 3.2.1: STATUS_IGNORE 19: 1: 3 Section 3.2.2: Non-destructive test of status 23: 0: 1 Section 3.3: MPI_ERR_KEYVAL 21: 0: 3 Section 3.4: MPI_True_extent Section 3.5: MPI_Set_extent Discussion at length about the implications of changing a property of a datatype, rather than constructing a new one. Implementors disliked modifying an existing datatype since many implementations reference count datatypes, therefore achieving the required semantics could require significant changes to code. Amendment to replace set_extent with a new datatype constructor which would change the extent 15: 0: 6 Section 3.4 and 3.5 as amended 22: 0: 0 Section 3.6: Section 3.6.1:Clarification of finalize. Much discussion on whether finalize must return a) at all b) somewhere c) everywhere Problems arise becuase this is really a binding issue to do with how "an MPI process" is bound to an implementation (e.g. it could be a Posix process, or a thread, or ...). Different choices for finalize semantics depend on this choice of binding. Allow finalize not to return 11: 6: 8 Disallow all MPI calls after Finalize 15: 1: 4 Section deferred, pending further clarification, and possible addition of an MPI_Finalized call. Section 3.6.2: Clarification of Intercomm_create 22: 1: 1 Section 3.6.3: Clarification of MPI_TYPE_SIZE 19: 0: 3 Section 3.6.4: Clarification of MPI_REDUCE Discussion on whether the proposal overly restricts implementations. Amendment to delete lines 34-38 (which specify exactly which functions will be called) and replace with advice to someone which says that any combination of user functions may be called (from one to all). 22: 0: 1 Section 3.6.4 as amended 22: 0: 1 Section 3.7: New datatype constructors Section 3.7.1: Simple_struct the formal definition as given is wrong, the section has been postponed pending a correct definition. 3.7.2: Contiguous struct 6: 7:10 (i.e. rejected) 3.7.3: Indexed_block 10: 7: 7 Process Creation and Management ------------------------------- At 10-45a.m. Bill Saphir introduced this section for second formal vote. (So this should not change again...) Sections 4.3.1, 4.3.2: MPI_SPAWN 22: 0: 3 4.3.3: MPI_SPAWN_MULTIPLE 20: 0: 3 4.3.4: SPAWN_INDEPENDENT Much discussion on this and the associated MONITOR and SIGNAL functions in 4.5.2 and 4.5.3. Vote on deleting 4.3.4 SPAWN_INDEPENDENT 4.3.5 SPAWN_MULTIPLE_INDEPENDENT 4.3.6 non-blocking independent functions 4.5.2 Signalling 4.5.3 Process monitoring 14: 7: 4 <== DELETED Vote to move all of the stuff just deleted to the JOD, as a recommendation to "do it this way if you do it" 19: 1: 5 Miscellanea ----------- Starting after lunch at 1p.m. Steve Huss-Ledermann discussed issues with the MPI_STATUS object (many more functions need to access it, and some user functions need to set values in it). The proposal is to provide functions for setting the fields in an MPI_STATUS. Rusty Lusk then introduced the Misc chapter. Formal votes 23-25 votes. Section 10.1: Portable mpi process startup The command is to be called mpiexec (rather than mpirun) thus avoiding any compatibility issues with existing mpirun commands. The proopsal states that if unspecifed the number of processes requested should be one. An amendment was proposed to leave the default unspecified. 14: 4: 7 Section 10.1 as amended 22: 1: 2 10.3: Language inter-operability Marc Snir 10.3.1, 10.3.2: Intro and assumptions 21: 0: 1 10.3.3: Initialization 23: 0: 1 10.3.4: Handle transfer 22: 0: 0 10.3.5, 10.3.6:Opaque objects and datatypes 23: 0: 2 10.3.7: Addresses deferred 10.3.8, 10.3.9: Groups and communicators 23: 0: 1 10.3.10: Attributes deferred 10.3.11:,12,13: Requests, Error handlers, Reduce operators 23: 0: 0 10.3.14: Status deferred 10.3.15:,16: Constants, inter-language communiction 21: 0: 0 Dynamic ------- At 2-25 p.m. Bill Saphir continued the Dynamic chapter. Section 4.3.6: Non-blocking spawn (with independent functions removed, see above) 12: 1:10 4.4.7: Reserved key values 15: 1: 6 4.4.1: Names, addresses etc 15: 0: 8 4.4.2: Server routines 13: 0:10 4.4.3: Client routines 11: 0:11 4.4.4: Name publishing 11: 0:12 The function names are likely to change to Service_publish, Service_retract or Service_unpublish 4.4.5: Iaccept 12: 1:11 Iconnect 9: 3:12 4.4.6: Reserved keys Amendement to add "ip_address" as an additional reserved key 12: 0:11 4.4.6 as amended 13: 0:10 C++ Bindings ------------ At 3-40 p.m. Andrew Lumsdaine attempted to present the C++ bindings. There was a great deal of discussion as to whether MPI needs C++ bindings, or whether C++ programmers should simply use the existing C bindings. Discussion continued at such length that there was no time for any formal votes on the C++ bindings. An extremely informal straw poll was held on whether to keep the C++ bindings. This was slightly in favour of keeping them, but without a majority. (When I asked for this vote I explicitly stated that I would not publish the results, so you'll have to ask me in person if you want the numbers !) Fortran 90 Bindings ------------------- At 4-30p.m. Bill Spahir presented the current state of the F90 bindings. There are significant outstanding issues here, in particular because * F90 is a strongly type-checked language without any escape routes, * F90 explicitly does not say anything about the layout of data in store, while MPI has an implicit model of store layout. There are a number of possible approaches, but none achieves full MPI and full F90. Further work is clearly required. One Sided Communications ------------------------ At 8-30a.m. on Friday Marc Snir introduced the newly revised version of the One sided chapter for a first formal vote. (The version of one-sided which had previously received a first vote has been abandoned). 24 voting organisations present. Instead of a communicator the one-sided operations now take a window object. Section 1.2: Initialisation There was discussion on whether an info object was the correct way to pass optional/additional information into the RMA_init call. No amendment was proposed, so an info argument remains. There was discussion on the use (and sematics) of the displacement argument. Amendment to delete the displacement argument 3:14: 7 Amendment to replace the displacement argument with a datatype whos extent would be used 3:16: 5 1.2 as not amended 21: 2: 1 Section 1.3: Communication calls SGI proposed an amendment to re-order the arguments to the calls so that the buffer came after the datatype 3:14: 6 Amendment that Fortran character data need not be supported in these calls 12: 4: 7 1.3 as amended 21: 0: 3 Section 1.4: Synchronization calls Much discussion in 1.4.3 on the fact that this section (which allows third party transfers) requires an active agent at each node. Amendment Locks are restricted to windows which are in MEM_ALLOCed windows 16: 7: 2 (This allows a simpler implementation on machines which can share some regions of the address space, but not all of it). 1.4 as amended 21: 1: 2 Section 1.5: Progress There was a discussion of the progress rules. As written these are stronger than some people's view of the MPI-1 rules, while conforming to other people's view. (This really means that the MPI-1 progress rules are open to interpretation). Amendment The progress rules are the same as in MPI-1 19: 1: 4 Whole one-sided chapter 20: 2: 3 Meeting dates ------------- At 10-30a.m. Rusty Lusk presented the possible meeting dates October 8-11 (Tuesday-Friday) confirmed January 21-23 (Tuesday-Thursday) tentative March 5- 7 (Wednesday-Friday) " April 23-25 (Wednesday-Friday) " There was some question of moving to California for these meetings to avoid getting snowed in at O'Hare. California 11 Chicago 13 So the meetings will remain at the Ramada O'Hare. Miscellany ---------- At 11-05 Marc Snir presented the difficult inter-operability problems in a 64 bit environment (section 10.3.7). Much discussion ensued, Eric Salo will draft an alternative that provides 5 Fortran functions with non-portable bindings. (I.e. the user may have to change her source code between different conforming MPI implementations if she uses these functions in F77). A straw vote on the idea of these (without seeing them) 7: 5: 10 Section 10.3.10: Attributes. No conclusion was reached a possible alternative would be to have new functions. At 11-45a.m. Rusty introduced 10.7 on continuable errors. At 11-50a.m. Bill Saphir discussed the MPI_APPNUM attribute which would allow an application to work out how many similar processes there are to it when started in a multi-executable job. Meeting closed at noon. From mpi-core-human@mcs.anl.gov Thu Sep 12 10:07:05 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA05545; Thu, 12 Sep 1996 10:07:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA29967 for mpi-core-out; Thu, 12 Sep 1996 09:09:45 -0500 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA29960; Thu, 12 Sep 1996 09:09:35 -0500 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.18.106]) by franklin.sdsc.edu (8.7.5/8.7.3/SDSCserver-12) with SMTP id HAA10791; Thu, 12 Sep 1996 07:09:34 -0700 (PDT) Received: from localhost by rogue.sdsc.edu (940406.SGI/1.11-client) with SMTP id AA15156; Thu, 12 Sep 96 07:09:33 -0700 Date: Thu, 12 Sep 1996 07:09:33 -0700 (PDT) From: Richard Frost To: Albert Cheng Cc: mpi-io@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: deadlines and "XDR-like" functionalities In-Reply-To: <9609112037.ZM3056@shalom.ncsa.uiuc.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Albert, I agree. Can you write a proposal for such a mapping? - Richard > I think in order to define "MPI_DATA_XDR" access mode, we need to borrow > something from XDR (floating points are in IEEE representation, byte orders > are in Big-Endian order), and map every MPI basic type to defined byte sizes > and orders (like typemap). We would need to deal with alignment problems > such as "is there any unused bytes for {int, char, int}?" > > =Albert > > -- > From mpi-core-human@mcs.anl.gov Thu Sep 12 10:15:12 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA05635; Thu, 12 Sep 1996 10:15:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA00279 for mpi-core-out; Thu, 12 Sep 1996 09:17:54 -0500 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA00242; Thu, 12 Sep 1996 09:17:25 -0500 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.18.106]) by franklin.sdsc.edu (8.7.5/8.7.3/SDSCserver-12) with SMTP id HAA12433; Thu, 12 Sep 1996 07:17:23 -0700 (PDT) Received: from localhost by rogue.sdsc.edu (940406.SGI/1.11-client) with SMTP id AA15159; Thu, 12 Sep 96 07:17:22 -0700 Date: Thu, 12 Sep 1996 07:17:22 -0700 (PDT) From: Richard Frost Reply-To: Richard Frost To: Tony Skjellum Cc: Albert Cheng , mpi-io@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: ASN.1 BER (was: deadlines and "XDR-like" functionalities) In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I've read some general papers regarding ASN.1, discovering that BER (basic encoding rules) is CCITT standard X.209, derived from X.409. Are these online somewhere? I've found several Web documents which reference X.#09, but none that discuss encoding details. Thanks, Richard > There is an international standard, ASN.1 BER (binary encoding rules). > These seem to be more general than XDR; ELROS (LLNL) used this for > automated heterogeneous conversion between Crays and workstations several > years ago, before MPI-1 was even complete... > -Tony > From mpi-core-human@mcs.anl.gov Thu Sep 12 10:15:23 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA05639; Thu, 12 Sep 1996 10:15:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA00285 for mpi-core-out; Thu, 12 Sep 1996 09:17:56 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA00234; Thu, 12 Sep 1996 09:17:07 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id JAA25614; Thu, 12 Sep 1996 09:17:05 -0500 Date: Thu, 12 Sep 1996 09:17:05 -0500 Message-Id: <199609121417.JAA25614@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov, mpi-dynamic@mcs.anl.gov Subject: anonymous ftp back up Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Well, people must be looking at what is posted because I got several e-mails about our ftp being down. It never fails that problems occur the first day after you announce something. Anyway, our anonymous ftp service went down last night. It is back up now. If you tried to get files and it failed, you should be able to get them. Let me know if you have problems. Steve From mpi-core-human@mcs.anl.gov Thu Sep 12 10:53:24 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA05993; Thu, 12 Sep 1996 10:53:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA01397 for mpi-core-out; Thu, 12 Sep 1996 09:56:15 -0500 Received: from zach.fit.edu (zach.fit.edu [163.118.22.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id JAA01391 for ; Thu, 12 Sep 1996 09:56:07 -0500 Received: by zach.fit.edu (5.61/1.34/900923wah) id AA00652; Thu, 12 Sep 96 10:49:28 -0400 Date: Thu, 12 Sep 96 10:49:28 -0400 From: howell@zach.fit.edu ( Gary Howell) Message-Id: <9609121449.AA00652@zach.fit.edu> To: lederman@cs.wisc.edu, mpi-core@mcs.anl.gov Subject: Re: 2nd votes Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As someone who attended MPI-1 but not MPI-2, I am concerned that MPI is and has been evolving too rapidly. For those with long memories, it evokes comparison with PL/1. Or perhaps VAX assembly. These evolved into rather too complicated constructs, and arrived at the evolutionary dead-end common to extravagant species and are now more or less extinct. Higher bars for new change will probably be helpful to MPI's long run usefulness to a larger public. Regards, Gary Howell Florida Tech Applied Math From mpi-core-human@mcs.anl.gov Thu Sep 12 10:58:03 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA06003; Thu, 12 Sep 1996 10:57:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA01518 for mpi-core-out; Thu, 12 Sep 1996 10:01:08 -0500 Received: from ns.ge.com (ns.ge.com [192.35.39.24]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA01502; Thu, 12 Sep 1996 10:00:52 -0500 Received: from thomas.ge.com (thomas.ge.com [3.47.28.21]) by ns.ge.com (8.7.5/8.7.3) with ESMTP id LAA27845; Thu, 12 Sep 1996 11:03:15 -0400 (EDT) Received: from ss09.ae.ge.com (ss09.ae.ge.com [129.202.174.1]) by thomas.ge.com (8.7.5/8.6.12) with ESMTP id JAA12175; Thu, 12 Sep 1996 09:52:30 -0400 Received: from h0006.ae.ge.com by ss09.ae.ge.com with ESMTP (1.37.109.16/15.6) id AA167090413; Thu, 12 Sep 1996 11:00:13 -0400 Received: from h0006 (localhost) by h0006.ae.ge.com with SMTP (1.37.109.16/16.2) id AA015360409; Thu, 12 Sep 1996 11:00:09 -0400 Message-Id: <32382578.52E8@ae.ge.com> Date: Thu, 12 Sep 1996 11:00:08 -0400 From: "Craig J. Fischberg" Organization: GE Aircraft Engines: Info. Technology Services X-Mailer: Mozilla 2.0 (X11; I; HP-UX A.09.05 9000/720) Mime-Version: 1.0 To: Richard Frost Cc: Tony Skjellum , Albert Cheng , mpi-io@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: ASN.1 BER (was: deadlines and "XDR-like" functionalities) References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Richard Frost wrote: > > I've read some general papers regarding ASN.1, discovering that > BER (basic encoding rules) is CCITT standard X.209, derived from > X.409. > > Are these online somewhere? I've found several Web documents which > reference X.#09, but none that discuss encoding details. http://www.fsid.cvut.cz/pub/doc/ccitt/1992/x/x209.asc -- ==================================================================== Craig J Fischberg E-mail: Craig.Fischberg@ae.ge.com GE Aircraft Engines Phone: 513-552-2716 One Neumann Way M/D T207 Fax: 513-552-3153 Cincinnati, OH 45215-6301 ==================================================================== From mpi-core-human@mcs.anl.gov Thu Sep 12 17:34:51 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA10666; Thu, 12 Sep 1996 17:34:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA13344 for mpi-core-out; Thu, 12 Sep 1996 16:35:24 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA13325 for ; Thu, 12 Sep 1996 16:35:05 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA04877 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Thu, 12 Sep 1996 14:35:03 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id OAA16415; Thu, 12 Sep 1996 14:32:52 -0700 From: "Eric Salo" Message-Id: <9609121432.ZM16413@mrjones.engr.sgi.com> Date: Thu, 12 Sep 1996 14:32:51 -0700 In-Reply-To: Steve Huss-Lederman "Re: 2nd votes" (Sep 12, 8:02am) References: <199609121302.IAA24598@romano.cs.wisc.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: 2nd votes Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with both Bill and Steve. However, I think that it might be more useful if the 60/40 rule was worded so that it applied only to adding things to the standard. In fact, I think it should be retroactive, meaning that all second votes which have already passed but do not meet the stricter requirements should be voted on again. I can think of only one specific example where this would be needed - for the most part, we had very solid majorities at the last meeting. What *should* the policy be for removing things from the standard? If a motion is made to delete some text, do we need a 60% majority to keep it in or do we need a 60% majority to pass the motion? -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Thu Sep 12 17:35:03 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA10671; Thu, 12 Sep 1996 17:35:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA13290 for mpi-core-out; Thu, 12 Sep 1996 16:34:07 -0500 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA13274; Thu, 12 Sep 1996 16:33:47 -0500 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.18.106]) by franklin.sdsc.edu (8.7.5/8.7.3/SDSCserver-12) with SMTP id OAA17749; Thu, 12 Sep 1996 14:33:46 -0700 (PDT) Received: from localhost by rogue.sdsc.edu (940406.SGI/1.11-client) with SMTP id AA15582; Thu, 12 Sep 96 14:33:45 -0700 Date: Thu, 12 Sep 1996 14:33:45 -0700 (PDT) From: Richard Frost To: Albert Cheng Cc: mpi-io@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: deadlines and "XDR-like" functionalities In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi, I received the following blurb from a colleague. Any interest or experience with DCE-IDL ? Richard - has any discussion of DCE's IDL (used for DCE RPC-s) taken place? From what I've seen, it appears that this is a superset of XDR, and OSF has released a PD version of some of their sources, which includes the IDL compiler and RPC interface. I've not actually had any experience using this PD version, but I have it available if you want to experiment with it. Since DCE seems to be proliferating, it may be that the MPI group may be able to leverage off of work that's already been done and is freely available. From mpi-core-human@mcs.anl.gov Thu Sep 12 22:26:48 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA00461; Thu, 12 Sep 1996 22:26:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA16654 for mpi-core-out; Thu, 12 Sep 1996 20:55:06 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA16649 for ; Thu, 12 Sep 1996 20:54:58 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id SAA26135 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Thu, 12 Sep 1996 18:54:55 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id SAA17039; Thu, 12 Sep 1996 18:52:44 -0700 From: "Eric Salo" Message-Id: <9609121852.ZM17037@mrjones.engr.sgi.com> Date: Thu, 12 Sep 1996 18:52:43 -0700 In-Reply-To: Raja Daoud "Re: MPI_MEM_ALLOC and MPI_MEM_FREE" (Sep 12, 12:20am) References: <199609120520.AA277085655@hp.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: MPI_MEM_ALLOC and MPI_MEM_FREE Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Currently for SGI, off-host BW is higher than on-host, but not for HP > (curious: do you ask users to de-localize their application to gain > performance?) This question comes up rather frequently, and the answer is no. The reason is that the HIPPI boards are a shared resource. Within a host, the aggregate bandwidth of all of the CPUs (assuming a moderately large system) is still more than you can get out of the HIPPI boards. But it's certainly true that on some applications, you can get better performance by running (for example) with 8 processes on each of 2 hosts than by running with 16 processes on one host. I also like Marc's suggestion, since it does fairly treat other sections as well, so I guess we're all in agreement on that. So now that this is behind us, I'd like to propose a small change to the MEM_ALLOC function. This is gonna generate a lot of groans, but I think that the comm argument should be an info argument instead. (Yeah, I know...) The reason is that it serves merely as a performance hint, but I can think of many other types of possibly useful hints. Examples: optimize all messages sent within this communicator optimize all messages sent within this group optimize sends only optimize recvs only optimize messages sent to a particular architecture optimize messages sent to a particular host or hosts optimize messages sent over a particular medium optimize with preference to send/recv optimize with preference to get/put assert that other MPI processes will also issue this call specify 'soft' bounds if you can't get all of the memory that you want Since we've already opened the 'info' can of worms, we might as well be consistent with the other parts of MPI-2, especially in something as implementation-dependent as this. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Sep 13 08:35:17 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA03267; Fri, 13 Sep 1996 08:35:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA21676 for mpi-core-out; Fri, 13 Sep 1996 07:38:38 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA21671 for ; Fri, 13 Sep 1996 07:38:29 -0500 Received: (from Unknown UID 28630@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id HAA16313; Fri, 13 Sep 1996 07:38:25 -0500 Date: Fri, 13 Sep 1996 07:38:25 -0500 Message-Id: <199609131238.HAA16313@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov In-reply-to: <9609121432.ZM16413@mrjones.engr.sgi.com> (salo@mrjones.engr.sgi.com) Subject: Re: 2nd votes Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > What *should* the policy be for removing things from the standard? If a motion > is made to delete some text, do we need a 60% majority to keep it in or do we > need a 60% majority to pass the motion? I think removing anything from MPI-1 (I assume this is what was meant) is a motion. It would take 60% majority to pass a motion. Thus, you cannot delete text from MPI-1 without a 60% majority. Steve P.S. - I assume this is not going to happen since we have already decided not to break any MPI-1 programs. From mpi-core-human@mcs.anl.gov Fri Sep 13 11:12:12 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA05083; Fri, 13 Sep 1996 11:12:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA24586 for mpi-core-out; Fri, 13 Sep 1996 10:15:17 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [129.34.139.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA24576 for ; Fri, 13 Sep 1996 10:15:03 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw2.watson.ibm.com (8.7.4/8.7.1) with ESMTP id LAA77532; Fri, 13 Sep 1996 11:15:28 -0400 Received: from watngi05.watson.ibm.com (watngi05.watson.ibm.com [9.2.235.16]) by mailhub1.watson.ibm.com (8.7.1/09-08-96) with SMTP id LAA325948; Fri, 13 Sep 1996 11:15:00 -0400 Received: by watngi05.watson.ibm.com(Lotus SMTP MTA Release 1.0) id 852563A4.0053C2A7 ; Fri, 13 Sep 1996 11:14:53 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: salo@mrjones.engr.sgi.com cc: Mpi-Core@mcs.anl.gov Message-ID: <852563A4:00535F13.00@watngi05.watson.ibm.com> Date: Fri, 13 Sep 1996 11:12:41 -0400 Subject: Re: MPI_MEM_ALLOC and MPI_MEM_FREE Mime-Version: 1.0 Content-type: multipart/mixed; Boundary="0__=t7j1H0ATmAdDFX7pClCU8W8F2oHpZedMleohFtQuOpHpUYGJTZs03vrj" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk --0__=t7j1H0ATmAdDFX7pClCU8W8F2oHpZedMleohFtQuOpHpUYGJTZs03vrj Content-type: text/plain; charset=us-ascii (Embedded image moved salo @ mrjones.engr.sgi.com to file: 09/12/96 09:52 PM PIC20866.PCX) To: mpi-core @ mcs.anl.gov cc: (bcc: Marc Snir/Watson/IBM Research) Subject: Re: MPI_MEM_ALLOC and MPI_MEM_FREE --0__=t7j1H0ATmAdDFX7pClCU8W8F2oHpZedMleohFtQuOpHpUYGJTZs03vrj ... So now that this is behind us, I'd like to propose a small change to the MEM_ALLOC function. This is gonna generate a lot of groans, but I think that the comm argument should be an info argument instead. (Yeah, I know...) The reason is that it serves merely as a performance hint, but I can think of many other types of possibly useful hints. Examples: optimize all messages sent within this communicator optimize all messages sent within this group optimize sends only optimize recvs only optimize messages sent to a particular architecture optimize messages sent to a particular host or hosts optimize messages sent over a particular medium optimize with preference to send/recv optimize with preference to get/put assert that other MPI processes will also issue this call specify 'soft' bounds if you can't get all of the memory that you want Since we've already opened the 'info' can of worms, we might as well be consistent with the other parts of MPI-2, especially in something as implementation-dependent as this. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" ... ***** We can have an additional info argument (why not?), but we cannot stuff in it the information provided by the comm argument, namely the list of processes that are likely to use this memory for communication. ***** --0__=t7j1H0ATmAdDFX7pClCU8W8F2oHpZedMleohFtQuOpHpUYGJTZs03vrj Content-type: application/octet-stream; name="PIC20866.PCX" Content-transfer-encoding: base64 CgUBCAAAAABoACwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAABaQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAD1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPH E8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sT zRPHE8MTwhPwEwzIBgzYE8wTxhPDE8IT7hPOBtcTzBPGE8MTE+wTwgbCBwbCEgbCEgbCEsUG1hPL E8YTwxMT6hMMwgYHwgLCAwISwgfEEsMCwwbVE8sTxRPDExPpE8MGAwcCBwMCwhLDB8ISwgISwgLD BtUTyhPFE8MTE+gTwgIHA8ICEw4DDgLDE8USwwLCEMIG1BPKE8UTwxMT5xMCAwcDAg4TDgITwgIS D8ISD8ISBRICEcICwwbUE8oTxRPCExPmEwYCBwMCDgIOwgLDExITEhPCEg8GxgLDBtMMDAfJE8QT whMT5hMGwwITBgMCDhLFEw8SE8ISBgIDwhIDEsMGB9MDxwwHxRPDExPlEwYHAhESAg8CwhMPwhMP xBMPxRIQwgIDAgMCBtMDxwPEDAfDE8IT4RMHwwzCBgLCEhMCDxLIE8MSD8MSwwIQAwIDBgfSDMkD wgPCDAfCExPbEwfGDMIDDAIHERITEhMSwxMPwxMPwxPDEgIDAgMCwwMCBgzREwfHDMYDDMITE9YT B8UMyAMGB8ICBhLDAsYTEhMSExIPwhIHAgcCAwUQAgYRBgfSE8UTB8QMwgMMwhMT0hMHxAzLA8IM BsISDxESExITAw4DxBMSExITwxICBwPCAsMDDMIGB9ITyRMHwwzCExPPEwfDDMkDxQwHwhMGBxIT AhECEwMOAg7DExITDxMPwxIDAgMCBwMCDAYRBgfSE8kTwhPCDMITE8wTB8MMxwPEDMIHxxMGxBLD Ag4DDgIGwg/IEgIDwgIDAgwCEMIGB9ITyRMHDAcMwhMTyhMHwgzGA8MMwgfMEwYHwhLCEAIOAg4C DhDDAhIPxhIFAgXDAgUCEQYH0hPHEwfCDAcPDMITE8gTB8IMxQPDDAfQEwbDEhDEAhAOEA4QwgLG EgcSBhIGBcMCBcIGB9ATB8UMEwfCDA8HDwwHwhMTxhMHwgzEA8MMB9MTBgfCEhADEMICDhAOEMIC EQIDxxIGBwbCAgUCEQYHyxMHxAwHwhMHEwzCEwcPBw8MB8MTE8UTBwzEA8IMB9YTBsQSEAMCA8UC EQIDAgPDEgcSBgfCBgUQAhDCBgfGEwfEDAfGE8INEwzCEw8HwgwHwxPCE8QTBwzDA8IMB9gTBgfE EhACEMYCEQIDAsQSBhLDBsICEALCBgfCEwfDDAfKEwfCDRMHwhPCDAfEE8ITE8MTBwzCA8IMB9oT DBIHwxLDDBEDxQIDAgPDEgYSBgfCBgIQAhAGDAfCEwzDE8MHyRMHwhPCBxMHxRPDExPDEwzCAwwH 3RMGxxICEQPDAgMCA8MSBhIGBwYMBhACEAIGDMMTDBPCB8YTwwfHEwfGE8MTwhPDEwwDDAfeEwYH xxICEQPDAgMCwhIGEgYHBgwGEAIQAsIGB8MTDMYTwwfKEwzGE8MTwhPDE8IMB98TDBLCB8USAgMR xAISB8ISBgcGDAYQBhAGEAYMB8MMB8kTwwfHEwzGE8MTwhPDEwwPwgzfEwYSB8ISB8ISAhECAwID EgcSBwYHBgwGEAYQxgzDD8IHxRPDB8kTBwzGE8MTwhPDEwzDD8QM3BPCBhIGwxIGAhECAwIHBgcG yAzJDxMHzRMHwwwHxxPDE8ITwxMHDMYPxwwH1BMGEgYSBhLLDM4PwwwTDMcTwgfEDAfJE8QTwhMT xBMHwgzLD9sM0w/GDAfDEwzDEwfEDAfLE8YTwxMTxhMHxAztD8gMBgfIE8QMB84TxxPDE8ITyhMH xwzbD8sMEAUMBcIMwgYH1RPKE8UTwxMT0RMH2wwGEAYQBhACBQwFDAUMBgwHBgfWE8sTxRPDExPu EwYMBhAGEAIGDAYMwwYH1xPLE8YTwxMT8BPKBgfYE8wTxhPDExP1E9sTzRPHE8MTwhP1E9sTzRPH E8MTwhMMAAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD/ /wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8A AAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A //8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAA AP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA /wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCk gICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vw oKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw //vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzA psrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDA wNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICA wMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACA AICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACA gACA//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP////// --0__=t7j1H0ATmAdDFX7pClCU8W8F2oHpZedMleohFtQuOpHpUYGJTZs03vrj-- From mpi-core-human@mcs.anl.gov Fri Sep 13 14:15:51 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA07054; Fri, 13 Sep 1996 14:15:50 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA28698 for mpi-core-out; Fri, 13 Sep 1996 13:18:27 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA28693 for ; Fri, 13 Sep 1996 13:18:19 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id LAA21645 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Fri, 13 Sep 1996 11:18:17 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id LAA18054; Fri, 13 Sep 1996 11:16:06 -0700 From: "Eric Salo" Message-Id: <9609131116.ZM18052@mrjones.engr.sgi.com> Date: Fri, 13 Sep 1996 11:16:06 -0700 In-Reply-To: Steve Huss-Lederman "Re: 2nd votes" (Sep 13, 7:38am) References: <199609131238.HAA16313@romano.cs.wisc.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: 2nd votes Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I think removing anything from MPI-1 (I assume this is what was meant) > is a motion. It would take 60% majority to pass a motion. Thus, you > cannot delete text from MPI-1 without a 60% majority. No, I meant removing things from MPI-2 that have already been voted on twice and approved. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Sep 13 14:57:41 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA07435; Fri, 13 Sep 1996 14:57:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA29414 for mpi-core-out; Fri, 13 Sep 1996 14:00:14 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA29397 for ; Fri, 13 Sep 1996 14:00:03 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id MAA01797 for <@sgi.engr.sgi.com:Mpi-Core@Mcs.Anl.Gov>; Fri, 13 Sep 1996 12:00:01 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for Mpi-Core@Mcs.Anl.Gov id LAA18212; Fri, 13 Sep 1996 11:57:50 -0700 From: "Eric Salo" Message-Id: <9609131157.ZM18210@mrjones.engr.sgi.com> Date: Fri, 13 Sep 1996 11:57:49 -0700 In-Reply-To: "Marc Snir" "Re: MPI_MEM_ALLOC and MPI_MEM_FREE" (Sep 13, 11:12am) References: <852563A4:00535F13.00@watngi05.watson.ibm.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: Mpi-Core@mcs.anl.gov Subject: Re: MPI_MEM_ALLOC and MPI_MEM_FREE Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > We can have an additional info argument (why not?), but we cannot stuff > in it the information provided by the comm argument, namely the list of > processes that are likely to use this memory for communication. Sure we could. We could use a comma-delimited list of ranks within MPI_COMM_WORLD, for example. We could do whatever we wanted, limited only by the bounds of imagination and good taste. And maybe not even the latter... -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Mon Sep 16 09:42:25 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA21512; Mon, 16 Sep 1996 09:42:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA27767 for mpi-core-out; Mon, 16 Sep 1996 08:40:03 -0500 Received: from tbag.osc.edu (tbag.osc.edu [128.146.36.50]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA27762 for ; Mon, 16 Sep 1996 08:39:55 -0500 Received: for gdburns@tbag.osc.edu by tbag.osc.edu (8.7.1/950822.1) id JAA05860; Mon, 16 Sep 1996 09:39:51 -0400 (EDT) Date: Mon, 16 Sep 1996 09:39:51 -0400 (EDT) From: Greg Burns Message-Id: <199609161339.JAA05860@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: SC'96 BOF Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Does anybody know when the MPI gathering at SC'96 will be held? -- Greg From mpi-core-human@mcs.anl.gov Mon Sep 16 10:21:08 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA21857; Mon, 16 Sep 1996 10:21:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA28781 for mpi-core-out; Mon, 16 Sep 1996 09:23:48 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA28757 for ; Mon, 16 Sep 1996 09:23:32 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id JAA20029; Mon, 16 Sep 1996 09:23:29 -0500 Date: Mon, 16 Sep 1996 09:23:29 -0500 Message-Id: <199609161423.JAA20029@romano.cs.wisc.edu> From: Steve Huss-Lederman To: gdburns@tbag.osc.edu CC: mpi-core@mcs.anl.gov In-reply-to: <199609161339.JAA05860@tbag.osc.edu> (message from Greg Burns on Mon, 16 Sep 1996 09:39:51 -0400 (EDT)) Subject: Re: SC'96 BOF Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I have made the request for a BOF at SC96. I am waiting to hear what time we will actually get. I requested Wednesday afternoon but there are often lots of conflicts at Supercomputing. I'll let everyone know once I hear our time slot. Steve From mpi-core-human@mcs.anl.gov Tue Sep 17 13:11:17 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA18435; Tue, 17 Sep 1996 13:11:12 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA25231 for mpi-core-out; Tue, 17 Sep 1996 12:13:29 -0500 Received: from felix.dircon.co.uk (felix.dircon.co.uk [193.128.224.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id MAA24938; Tue, 17 Sep 1996 12:02:21 -0500 Received: by felix.dircon.co.uk id AA03732 (5.67b/IDA-1.5); Tue, 17 Sep 1996 17:35:51 +0100 Message-Id: <199609171635.AA03732@felix.dircon.co.uk> Received: from gw2-151.pool.dircon.co.uk(194.112.35.151) by amnesiac via smap (V1.3) id sma003698; Tue Sep 17 17:35:20 1996 Received: from localhost by jim (SMI-8.6) id RAA03777; Tue, 17 Sep 1996 17:41:04 +0100 To: Marc Snir/Watson/IBM Research Cc: mpi-comm , mpi-core Subject: Re: Request for information on Fortran systems that use MPI In-Reply-To: Your message of "16 Sep 1996 10:40:05." <9609171359.AA0226@watngi01.watson.ibm.com> Date: Tue, 17 Sep 1996 17:41:04 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > 1. Are there Fortran systems that do not support (integer) pointers at all? I believe that g77 is such a system. GNU F77 version 2.7.2 (sparc) compiled by GNU C version 2.7.2. GNU Fortran Front End version 0.5.16 compiled: May 30 1996 14:00:35 jcownie@jim: g77 -o foo foo.f foo.f: In program `prof': foo.f:4: POINTER (P, A(100,100)) ^ Unimplemented or invalid form of statement at (^) -- statement ignored (this is a catchall diagnostic that currently applies to a wide variety of errors, including things like invalid ordering of statements) -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Wed Sep 18 12:35:31 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA27908; Wed, 18 Sep 1996 12:35:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA14521 for mpi-core-out; Wed, 18 Sep 1996 10:30:36 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA14503; Wed, 18 Sep 1996 10:29:51 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id KAA10555; Wed, 18 Sep 1996 10:29:36 -0500 Date: Wed, 18 Sep 1996 10:29:36 -0500 (CDT) From: Tony Skjellum To: mpi-core@mcs.anl.gov, mpi-comm@mcs.anl.gov Subject: JOD as separate document Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Folks, I make the following proposal. Since the JOD document is the core of MPI-3, and since it may need chapter stratification, I suggest that it be separated from MPI-2 document, and be physically attached, rather than being jointly managed. Reasons: 1) Reduction of complexity 2) Reduction of work for Steve Lederman 3) Ability to begin structuring work for MPI-3 4) Ability to continue work on JOD contents without strictures of SC'96 deadline associated with rest of MPI-2 There is no loss of generality, provided the JOD gets distributed at Supercomputing with rest of document. Comments? Tony From mpi-core-human@mcs.anl.gov Wed Sep 18 14:19:59 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA29259; Wed, 18 Sep 1996 14:19:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA18198 for mpi-core-out; Wed, 18 Sep 1996 13:22:06 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA18193 for ; Wed, 18 Sep 1996 13:21:54 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id NAA29528; Wed, 18 Sep 1996 13:21:53 -0500 Date: Wed, 18 Sep 1996 13:21:53 -0500 Message-Id: <199609181821.NAA29528@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: SC96 BOF 11/20 at 3:30 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The MPI-2 BOF has been tentatively scheduled for Wednesday, Nov. 20 from 3:30 to 5:30. Note that the schedule can still change some as the final juggling occurs. I'll keep people updated to changes and final choice of room. Steve From mpi-core-human@mcs.anl.gov Fri Sep 20 02:06:30 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA24033; Fri, 20 Sep 1996 02:06:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA28092 for mpi-core-out; Thu, 19 Sep 1996 23:22:50 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA28086; Thu, 19 Sep 1996 23:22:42 -0500 Message-Id: <199609200422.XAA28086@antares.mcs.anl.gov> To: mpi-comm@antares.mcs.anl.gov, mpi-core@antares.mcs.anl.gov Subject: New Misc chapters Date: Thu, 19 Sep 1996 23:22:42 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The next two messages contain updates of the Misc 1.2 and Misc 2.0 chapters that reflect decisions made during the last meeting and email discussion since then. This chapter is highly miscellaneous, and I am not sure that I have correctly captured every issue that has been raised. If there is something that you have been following in Misc, please check that it has been properly addressed. Rusty From mpi-core-human@mcs.anl.gov Fri Sep 20 04:03:03 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA02621; Fri, 20 Sep 1996 04:02:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA28160 for mpi-core-out; Thu, 19 Sep 1996 23:27:03 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA28155; Thu, 19 Sep 1996 23:26:48 -0500 Message-Id: <199609200426.XAA28155@antares.mcs.anl.gov> To: mpi-comm@antares.mcs.anl.gov, mpi-core@antares.mcs.anl.gov Subject: New Misc-2.0 Date: Thu, 19 Sep 1996 23:26:46 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Thu Sep 19 23:25:17 1996 %%Pages: 21 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.09.19:2325 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 1 1 df0 D E /Fb 2 111 df<00180038001000000000000000000000000001C0022004300430086000600060 006000C000C000C000C001800180018001806300E300C60078000D1D80960E>106 D<383C0044C6004702004602008E06000C06000C06000C0C00180C00180C401818401818 80300880300F00120E7F8D15>110 D E /Fc 4 52 df<07C018303018701C600C600CE0 0EE00EE00EE00EE00EE00EE00EE00EE00E600C600C701C30181C7007C00F157F9412>48 D<03000700FF000700070007000700070007000700070007000700070007000700070007 00070007007FF00C157E9412>I<0F8030E040708030C038E03840380038007000700060 00C00180030006000C08080810183FF07FF0FFF00D157E9412>I<0FE030306018701C70 1C001C00180038006007E000300018000C000E000EE00EE00EC00C401830300FE00F157F 9412>I E /Fd 52 122 df<00E001C0038007000E000E001C001C003800380038007000 700070007000E000E000E000E000E000E000E000E000E000E000E000E000700070007000 70003800380038001C001C000E000E000700038001C000E00B2A7E9E10>40 DI<018001C0018001806186F99F7DBE1FF8 07E007E01FF87DBEF99F61860180018001C0018010127E9E15>I44 DI<03C00FF01FF83C3C381C700E700E700EE007 E007E007E007E007E007E007E007E007E007E007E007E007700E700E700E381C3C3C1FF8 0FF007E0101D7E9B15>48 D<010007003F00FF00C7000700070007000700070007000700 07000700070007000700070007000700070007000700070007000700FFF8FFF80D1C7C9B 15>I<07E01FF83C3C700E700EE007E007E007E007700E700E3C3C1FF807E01FF83C3C70 0E700EE007E007E007E007E007E007700E781E3C3C1FF807E0101D7E9B15>56 D58 D<7FFFFFC0FFFFFFE000 00000000000000000000000000000000000000000000000000000000000000FFFFFFE07F FFFFC01B0C7E8F20>61 D<001C0000003E0000003E0000002E0000006700000067000000 E7800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F00007 00F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E0038 001E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F00001E00003C00003C00 00780000780000780000F00000F00000F00000F00000F00000F00000F00000F00000F000 007800007800007800003C00003C00001E00000F000807801803C07800FFF0003F80151F 7D9D1B>IIII<00 3F8001FFF003C0F80780380F00181E00003C00003C0000780000780000780000F00000F0 0000F00000F00000F00000F00000F007F8F007F8F000387800387800387800383C00383C 00381E00380F003807803803C0F801FFF0003F80151F7D9D1C>III75 DIII<003F000001 FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F00780007807800078078 000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C078000780780007807C000F803C000F003E001F001F003E000F807C0007C0F80003 FFF00001FFE000003F00001A1F7E9D1F>II82 D<03F8000FFE001C0F00 380700700300600000E00000E00000E00000E00000F000007800007F00003FE0001FFC00 07FE0001FF00001F800007800003C00003C00001C00001C00001C00001C0C00180E00380 F007007C0E001FFC0007F000121F7E9D17>III87 D89 D<7FFFF07FFFF00001E00003E00003C00007C000 0780000F00001F00001E00003E00003C0000780000F80000F00001F00001E00003C00007 C0000780000F80000F00001E00003E00003C00007C0000780000FFFFF0FFFFF0141D7E9C 19>I<0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7F DC3F1C0E127E9114>97 DI<07E00FF81FFC3C1C70047000E000E000E000E000E000E000700070043C1C1FFC 0FF807E00E127E9112>I<000E000E000E000E000E000E000E000E000E000E000E0F8E1F EE3FFE7C3E700E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E0F1D7E 9C15>I<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C3FFC 1FF807E00E127E9112>I<00FC01FC03FC07000E000E000E000E000E000E000E00FFE0FF E00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E1D80 9C0D>I<03C3C00FFFC01FFFC01C3800381C00381C00381C00381C00381C001C38001FF8 001FF0003BC0003800003800001FFC001FFF003FFF80700780E001C0E001C0E001C0F003 C07C0F803FFF001FFE0007F800121B7F9115>III107 DIII<03F0000FFC001FFE003C0F007807 80700380E001C0E001C0E001C0E001C0E001C0F003C07003807807803C0F001FFE000FFC 0003F00012127F9115>II114 D<1FC03FF07FF0F030E000E000F0007F003FC01FE000F0003800388038F0 78FFF07FE01FC00D127F9110>I<1C001C001C001C001C001C00FFE0FFE01C001C001C00 1C001C001C001C001C001C001C001C001C001C201FF00FF007C00C187F970F>I118 D121 D E /Fe 48 123 df46 D<01F00007FC000FFE001F1F003C0780 3C07807803C07803C07803C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F001E0 F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C07803C07803C0 3C07803C07801F1F000FFE0007FC0001F00013237EA118>48 D<00C001C007C0FFC0FFC0 FBC003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C0FFFFFFFFFFFF10227CA118>I<03F0000FFC001F FE003C1F003007807007C06003C0E003E0C001E04001E04001E00001E00001E00001E000 03C00003C0000780000780000F00001E00003C0000780000F00001E00001C00003800007 00000E00001C0000380000700000FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC00 1FFF003E0F003807807003C02003C02003C00003C00003C00003C0000780000780000F00 001E0003FC0003F80003FE00000F000007800003C00003C00001E00001E00001E00001E0 0001E08001E0C003C0E003C07007803C0F801FFF000FFC0003F00013237EA118>I<001F 00001F00002F00002F00006F0000EF0000CF0001CF0001CF00038F00038F00078F00070F 000F0F000E0F001E0F003C0F003C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF8000F 00000F00000F00000F00000F00000F00000F00000F00000F0015217FA018>I<3FFF803F FF803FFF803C00003C00003C00003C00003C00003C00003C00003C00003C00003CF8003F FE003FFF003F0F803E07803C03C03803C00001E00001E00001E00001E00001E00001E000 01E04003C04003C0E003C07007807C1F003FFE000FFC0003F00013227EA018>I<007E00 01FF0003FF0007C1000F00001E00001E00003C00003C000078000078000078F800F1FE00 F7FF00FF0F80FC0780F803C0F803C0F801C0F001E0F001E0F001E0F001E0F001E07001E0 7001E07801E07803C03803C03C03801C07801F0F000FFE0007FC0001F00013237EA118> II<01 F00007FC000FFE001E0F003C07803C07807803C07803C07803C07803C07803C03803803C 07801E0F000F1E0007FC0003F8000FFE001E0F003C07807803C07803C0F001E0F001E0F0 01E0F001E0F001E0F001E07803C07803C03C07803E0F801FFF0007FC0001F00013237EA1 18>I<01F00007FC000FFE001E0F003C07003807807803807803C0F003C0F001C0F001E0 F001E0F001E0F001E0F001E0F001E07003E07803E07803E03C07E03E1FE01FFDE00FF1E0 03E3C00003C00003C0000380000780000700000F00101E00183C003FF8001FF00007C000 13237EA118>I<001F0000001F0000003F8000003F8000003B8000007BC0000073C00000 71C00000F1E00000F1E00000E0E00001E0F00001E0F00001C0F00003C0780003C0780003 80780007803C0007803C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF001C 000F003C0007803C00078038000780780003C0780003C0700003C0F00001E0F00001E0E0 0001E01B237EA220>65 DI<000FF000007FFC0000FFFF0001F01F0003C0070007 8000000F0000001E0000003E0000003C0000003C000000780000007800000078000000F0 000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000078 00000078000000780000003C0000003C0000003E0000001E0000000F0000000780008003 C0038001F00F8000FFFF00007FFC00000FF00019257DA31F>III<000FF0 00003FFE0000FFFF8001F80F8003E00380078000000F0000001E0000001E0000003C0000 003C000000780000007800000078000000F0000000F0000000F0000000F0000000F00000 00F0000000F000FFC0F000FFC0F000FFC0780003C0780003C0780003C03C0003C03C0003 C01E0003C01E0003C00F0003C0078003C003E003C001F807C000FFFFC0003FFF00000FF8 001A257DA321>71 D73 D77 D<001FC000007FF00001FFFC0003F07E00 07C01F000F800F801F0007C01E0003C03C0001E03C0001E0780000F0780000F0780000F0 70000070F0000078F0000078F0000078F0000078F0000078F0000078F0000078F0000078 F0000078780000F0780000F0780000F07C0001F03C0001E03E0003E01E0003C01F0007C0 0F800F8007C01F0003F07E0001FFFC00007FF000001FC0001D257DA324>79 DI82 D<00FE0003FFC007FFE00F81E01E00603C00003C00007800007800 007800007800007800007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007 E00001F00000F00000F8000078000078000078000078000078000078C000F0E000F0F801 E07E07C03FFF800FFF0001FC0015257EA31B>II<07E01FF83FFC381E201E000F00 0F000F000F00FF07FF1FFF3E0F780FF00FF00FF00FF00FF83F7FFF3FEF1F8F10167E9517 >97 DI<01FC0007FF000FFF801F03803C0180780000780000700000F00000F000 00F00000F00000F00000F000007800007800007800003C00401F03C00FFFC007FF8001FC 0012167E9516>I<0003C00003C00003C00003C00003C00003C00003C00003C00003C000 03C00003C00003C00003C003E3C00FFBC01FFFC03F0FC03C07C07803C07803C0F003C0F0 03C0F003C0F003C0F003C0F003C0F003C0F003C07803C07803C03C07C03E0FC01FFFC00F FBC003E3C012237EA219>I<03F00007FC001FFE003E0F003C0780780380780380F001C0 FFFFC0FFFFC0FFFFC0F00000F00000F000007000007800007800003C00801F07800FFF80 07FF0001F80012167E9516>I<003F00FF01FF03C0038007800780078007800780078007 800780FFF8FFF8FFF8078007800780078007800780078007800780078007800780078007 8007800780078007800780102380A20F>I<01F07807FFF80FFFF81F1F001E0F003C0780 3C07803C07803C07803C07801E0F001F1F000FFE001FFC0019F0003800003800003C0000 1FFE001FFFC01FFFE03FFFF07801F07800F8F00078F00078F00078F000787800F03E03E0 1FFFC00FFF8001FC0015217F9518>III<00F000F000F000F000 000000000000000000000000000000000000F000F000F000F000F000F000F000F000F000 F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F080 F0E3E0FFE07FC01F000C2D83A20D>I108 DII<01FC0007FF000FFF801F 07C03C01E07800F07800F0700070F00078F00078F00078F00078F00078F000787800F078 00F07C01F03E03E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07 C07C03C07803C0F803C0F003C0F003C0F003C0F003C0F003C0F003C0F803C07803C07803 C03C07C03F0FC01FFFC00FFBC003E3C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C012207E9519>II<07F01FFC 3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE 3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12 >II119 D<7801F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F800 00700000F80001FC0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F8 1516809516>II<7FFFC07FFFC07FFFC0000F80000F00001E00003E00007C0000780000F00001F00003 E00003C0000780000F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012167F95 15>I E /Ff 23 123 df<70F8F8F87005057C840D>58 D<70F8FCFC7404040408081010 2040060E7C840D>I<000001C00000078000001E00000078000001E00000078000000E00 000038000000F0000003C000000F0000003C000000F0000000F00000003C0000000F0000 0003C0000000F0000000380000000E0000000780000001E0000000780000001E00000007 80000001C01A1A7C9723>I62 D<0FFFFFFC1E03C0381803C0181003C0082003C008 20078008600780084007800840078008800F0010000F0000000F0000000F0000001E0000 001E0000001E0000001E0000003C0000003C0000003C0000003C00000078000000780000 007800000078000000F0000000F0000000F0000000F0000001F000007FFFC0001E1F7F9E 1B>84 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E 00F00E00F00E00F00E10F01C20F01C20703C20705C40308C400F078014147E9318>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207E9F14>I< 0000780003F80000700000700000700000700000E00000E00000E00000E00001C00001C0 00F1C00389C00707800E03801C03803C0380380700780700780700780700F00E00F00E00 F00E00F00E10F01C20F01C20703C20705C40308C400F078015207E9F18>100 D<007C01C207010E011C013C013802780C7BF07C00F000F000F000F00070007001700230 04183807C010147E9315>I<00007C0000CE00019E00039E00030C000700000700000700 000700000E00000E00000E0000FFF0000E00000E00001C00001C00001C00001C00001C00 00380000380000380000380000380000700000700000700000700000700000E00000E000 00E00000E00000C00001C000318000798000F300006200003C000017297E9F16>I<00E0 01E001E000C000000000000000000000000000000E001300238043804380438087000700 07000E000E001C001C001C20384038403840388019000E000B1F7E9E10>105 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207E9F0E> 108 D<1E07C07C00231861860023A032030043C034030043803803804380380380870070 07000700700700070070070007007007000E00E00E000E00E00E000E00E00E000E00E01C 101C01C01C201C01C038201C01C038401C01C0184038038018801801800F0024147E9328 >I<1E07802318C023A06043C0704380704380708700E00700E00700E00700E00E01C00E 01C00E01C00E03821C03841C07041C07081C03083803101801E017147E931B>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147E9316>I<03C1E00462180474 1C08781C08701E08701E10E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D819317>I<1E1E0023210023C38043C78043878043830087000007 00000700000700000E00000E00000E00000E00001C00001C00001C00001C000038000018 000011147E9315>114 D<007C018203010603060706060E00078007F803FC01FE001F00 077007F006F006E004400820301FC010147E9315>I<00C000E001C001C001C001C00380 0380FFF8038007000700070007000E000E000E000E001C001C001C001C10382038203820 384018800F000D1C7F9B10>I<0F00601180702180E021C0E041C0E04380E08381C00701 C00701C00701C00E03800E03800E03800E03840E07080C07080C07080E0F1006131003E1 E016147E931A>I<03C1C00C62201034701038F02038F020386040700000700000700000 700000E00000E00000E00000E02061C040F1C040F1C080E2C080446300383C0014147E93 1A>120 D<0F00601180702180E021C0E041C0E04380E08381C00701C00701C00701C00E 03800E03800E03800E03800E07000C07000C07000E0F00061E0003EE00000E00000E0000 1C0078180078380070700060600021C0001F0000141D7E9316>I<01E02003F04007F8C0 0C1F8008010000020000040000080000100000600000C000010000020000040080080100 1003003F060061FC0040F80080700013147E9315>I E /Fg 26 90 df<3078F8787005057C840D>46 D<007E0001C3000301800701C00E00C00E00E01C00E0 1C00E03C01E03801E07801E07801E07801E07801E07801E0F003C0F003C0F003C0F003C0 F003C0F00380F00780E00780E00700E00700E00E00600E00701C003038003870000FC000 131F7C9D17>48 D<000C001C00FC0F380038003800380038003800700070007000700070 007000E000E000E000E000E000E001C001C001C001C001C001C0038003C0FFFE0F1E7C9D 17>I<003F8000C1E00100F00200780400780400780F007C0F807C0F807C0F0078060078 0000F80000F00001E00001C0000380000700000E00001C0000380000600000C000018000 0300200600200800401000403FFFC07FFF80FFFF80161E7E9D17>I<007F000183C00201 E00400F00700F00F00F00F01F00F01F00001E00001E00003C0000380000700000E0000F8 00000E000007000007800007C00003C00007C03007C07807C0F807C0F807C0F00780800F 00400E00201C0018780007E000141F7D9D17>I<01803001FFE003FFC003FF0003FC0002 0000020000020000040000040000040000047C000587000603800C01800801C00001C000 01E00001E00001E00001E07003C0F803C0F003C0E00380800780400700400E00201C0018 700007C000141F7D9D17>53 D<0000100000001800000038000000380000007800000078 000000FC000001BC0000013C0000033C0000023C0000063C0000043E0000081E0000081E 0000101E0000101E0000201E0000200F0000400F0000400F0000FFFF0000800F0001000F 8001000780020007800200078004000780040007800C0007C03E0007C0FF807FFC1E207E 9F22>65 D<07FFFF00007C01C0003C01E0003C00F0007800F8007800F8007800F8007800 F8007800F8007800F000F001F000F001E000F003C000F00F8000FFFE0000F00F0001E007 C001E003C001E003E001E001E001E001E001E001E003C001E003C003E003C003E003C003 C003C007C003C00F8007800F0007803E00FFFFF0001D1F7E9E20>I<0001F808000E0618 00380138007000F801E0007803C0007007800030078000300F0000301F0000301E000030 3E0000203C0000007C0000007C0000007C0000007C000000F8000000F8000000F8000000 F8000000F80000007800004078000080780000803C0000803C0001001C0002000E000200 06000C000300100001C0E000003F00001D217B9F21>I<07FFFF00007C01E0003C00F000 3C00780078003C0078003C0078001E0078001E0078001E0078001F00F0001F00F0001F00 F0001F00F0001F00F0001F00F0001F01E0001E01E0003E01E0003E01E0003E01E0003C01 E0007C03C0007803C000F003C000F003C001E003C003C003C0078007800F0007803C00FF FFE000201F7E9E23>I<07FFFFF8007C0078003C0038003C001800780018007800080078 000800780008007800080078080800F0100000F0100000F0100000F0300000FFF00000F0 700001E0200001E0200001E0200001E0200001E0000801E0001003C0001003C0001003C0 002003C0002003C0006003C000C0078001C0078007C0FFFFFF801D1F7E9E1F>I<07FFFF F8007C0078003C0038003C00180078001800780008007800080078000800780008007800 0800F0100000F0100000F0100000F0300000F0700000FFF00001E0600001E0200001E020 0001E0200001E0200001E0000003C0000003C0000003C0000003C0000003C0000003C000 000780000007C00000FFFE00001D1F7E9E1E>I<0001FC04000F030C003C009C0070007C 00E0003C01C0003803800018078000180F0000181F0000181E0000183E0000103C000000 7C0000007C0000007C0000007C000000F8000000F8000000F8007FFCF80003E0780001E0 780001E0780003C0780003C03C0003C03C0003C01C0003C00E0007C007000B8003801180 01E06080003F80001E217B9F24>I<07FFC7FFC0007C00F800003C007800003C00780000 7800F000007800F000007800F000007800F000007800F000007800F00000F001E00000F0 01E00000F001E00000F001E00000FFFFE00000F001E00001E003C00001E003C00001E003 C00001E003C00001E003C00001E003C00003C007800003C007800003C007800003C00780 0003C007800003C007800007800F000007C00F8000FFF8FFF800221F7E9E22>I<07FFE0 007C00003C00003C0000780000780000780000780000780000780000F00000F00000F000 00F00000F00000F00001E00001E00001E00001E00001E00001E00003C00003C00003C000 03C00003C00003C00007800007C000FFFC00131F7F9E10>I<07FFF000007E0000003C00 00003C000000780000007800000078000000780000007800000078000000F0000000F000 0000F0000000F0000000F0000000F0000001E0000001E0000001E0000001E0000001E000 8001E0010003C0010003C0010003C0030003C0020003C0060003C0060007801E0007807C 00FFFFFC00191F7E9E1C>76 D<07FC0000FFC0007C0000F800003C00017800003C000178 00004E0002F000004E0002F000004E0004F000004E0004F000004E0008F000004E0008F0 0000870011E00000870011E00000870021E00000870021E00000870041E00000838041E0 0001038083C00001038083C00001038103C00001038203C0000101C203C0000101C403C0 000201C40780000201C80780000201C80780000201D00780000200F00780000600E00780 000600E00F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00 003E001800003E001800004F001000004F001000004780100000478010000043C0100000 43C010000083C020000081E020000081E020000080F020000080F0200000807820000100 78400001007C400001003C400001003C400001001E400001001E400002000F800002000F 800002000F800002000780000200078000060003800006000300000F00010000FFE00100 00221F7E9E22>I<0003F800001E0E000038070000E0038001C001C003C001E0078000E0 0F0000F00F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F8 7C0000F8F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0780007C0 7C0007803C000F003C001E001E001C000E0038000700F00003C3C00000FE00001D217B9F 23>I<07FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F8007800F800 7800F8007800F000F001F000F001E000F003C000F0078000F00F0000FFF80001E0000001 E0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003 C0000003C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C 03C0003C01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F0 03C000F0078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E0 1C0001E01E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C0 1F04FFFC0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C070018 0600080E00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE0 0007E00001E00000E00000F00000F04000E04000E04000E04000E06000C0600180E00380 F80300C60C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010 400F0010C00F0010800F0010800F0010800F0010001E0000001E0000001E0000001E0000 001E0000001E0000003C0000003C0000003C0000003C0000003C0000003C000000780000 007800000078000000780000007800000078000000F0000001F800007FFFE0001C1F7A9E 21>II87 D 89 D E /Fh 37 123 df<387CFEFEFE7C3807077C860F>46 D<01FC0007FF001F07C01E 03C03E03E07C01F07C01F07C01F0FC01F8FC01F8FC01F8FC01F8FC01F8FC01F8FC01F8FC 01F8FC01F8FC01F8FC01F8FC01F8FC01F87C01F07C01F07C01F03E03E01E03C01F8FC007 FF0001FC00151D7E9C1A>48 D<00E00001E0000FE000FFE000F3E00003E00003E00003E0 0003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E0 0003E00003E00003E00003E00003E00003E00003E000FFFF80FFFF80111D7C9C1A>I<07 F0001FFE00383F007C1F80FE0FC0FE0FC0FE0FE0FE07E07C07E03807E0000FE0000FC000 0FC0001F80001F00003E0000780000F00000E00001C0000380600700600E00601C00E01F FFC03FFFC07FFFC0FFFFC0FFFFC0131D7D9C1A>I<01FC0007FF000E0F801E0FC03F07E0 3F07E03F07E03F07E01E0FC0000FC0000F80001F0001FC0001FC00000F800007C00003E0 0003F00003F83803F87C03F8FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003FC00 151D7E9C1A>I<0001C00003C00007C00007C0000FC0001FC0003BC00073C00063C000C3 C00183C00383C00703C00E03C00C03C01803C03803C07003C0E003C0FFFFFEFFFFFE0007 C00007C00007C00007C00007C00007C000FFFE00FFFE171D7F9C1A>I<3803803FFF803F FF003FFE003FFC003FF0003F800030000030000030000030000033F80037FE003C1F0038 0F801007C00007C00007E00007E07807E0FC07E0FC07E0FC07E0FC07C0780FC0600F8038 1F001FFC0007F000131D7D9C1A>I<003F0001FFC007E0E00F81E01F03F01E03F03E03F0 7C03F07C01E07C0000FC1000FCFF00FDFFC0FD03E0FE01F0FE01F0FC01F8FC01F8FC01F8 FC01F87C01F87C01F87C01F83C01F03E01F01E03E00F07C007FF8001FE00151D7E9C1A> I<6000007FFFF87FFFF87FFFF07FFFE07FFFC0E00180C00300C00300C00600000C000018 0000380000380000780000700000F00000F00001F00001F00001F00001F00003F00003F0 0003F00003F00003F00003F00001E00000C000151E7D9D1A>I<01FC0007FF000F07801E 03C01C01E03C01E03C01E03E01E03F01E03FC3C01FE3801FFF000FFE0007FF8007FFC01F FFE03C3FF0780FF07803F8F001F8F000F8F00078F00078F000707800707C00E03E03C00F FF8003FC00151D7E9C1A>I<01FC000FFF001F07803E03C07C03E07C01E0FC01F0FC01F0 FC01F0FC01F8FC01F8FC01F8FC01F87C03F87C03F83E05F81FFDF807F9F80041F80001F0 3C01F07E01F07E03E07E03E07E07C03C0780381F001FFC0007F000151D7E9C1A>I<07F8 001FFE00381F80780F80FC0FC0FC0FC0FC0FC0780FC0301F80001F00003E00007C000070 0000E00000E00000C00000C00000C00000C00000C00000C0000000000000000000000000 0001C00003E00007F00007F00007F00003E00001C00012207D9F19>63 D<0007FC02003FFF0E00FE03DE03F000FE07E0003E0FC0001E1F80001E3F00000E3F0000 0E7F0000067E0000067E000006FE000000FE000000FE000000FE000000FE000000FE0000 00FE0000007E0000007E0000067F0000063F0000063F00000C1F80000C0FC0001807E000 3803F0007000FE01C0003FFF800007FC001F1F7D9E26>67 D69 D73 D77 D80 D<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003 E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803E0FC05E07E0DE03FF8FE0FE0 7E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC00 00FC0000FC00007C00007E00003E00603F00C01F81C007FF0001FC0013147E9317>I<01 FE0007FF800F83C01E01E03E00F07C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC 00007C00007C00003E00181E00180F807007FFE000FF8015147F9318>101 D<001F8000FFC001F3E003E7E003C7E007C7E007C3C007C00007C00007C00007C00007C0 00FFFC00FFFC0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC3C07 FFFE0F079E1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC0018 00001800001C00001FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78 003C78003C3F01F80FFFE001FF00171E7F931A>I<1C003E007F007F007F003E001C0000 0000000000000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F 001F001F001F001F001F00FFE0FFE00B217EA00E>105 D<0038007C00FE00FE00FE007C 003800000000000000000000000001FE01FE003E003E003E003E003E003E003E003E003E 003E003E003E003E003E003E003E003E003E003E003E303E783EFC3CFC7C78783FF01FC0 0F2A83A010>I108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC 007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>II<01F81807FE381F87783F01F83E01F87E00 F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87E00F87E00F83F01F81F87 F80FFEF803F8F80000F80000F80000F80000F80000F80000F80000F80007FF0007FF181D 7E931C>II< 0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81E FFFCC7F010147E9315>I<01800180018003800380038007800F803F80FFFCFFFC0F800F 800F800F800F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F 9C14>II120 D<3FFFE03FFFE03C07C0380F80701F80603F00603E 00607C0000F80001F80003F00003E06007C0600F80601F80E03F00C03E01C07C03C0FFFF C0FFFFC013147F9317>122 D E /Fi 37 120 df<00003F03E00000C386700001878CF0 0003879CF00003031860000700380000070038000007003800000E003800000E00700000 0E007000000E00700000FFFFFF80001C007000001C00E000001C00E000001C00E000001C 00E000003800E000003801C000003801C000003801C000003801C000007001C000007003 8000007003800000700380000070038000006003800000E007000000E007000000E00700 0000E007000000C006000001C00E000001C00E000031860C0000798F180000F31E100000 620C6000003C07C000002429829F1C>11 D<00003FE00000E01000018038000380780003 007800070030000700000007000000070000000E0000000E0000000E000000FFFFE0000E 00E0001C01C0001C01C0001C01C0001C01C0001C03800038038000380380003803800038 070000380700007007000070071000700E2000700E2000700E2000E00E2000E0064000E0 038000E0000000C0000001C0000001C000003180000079800000F3000000620000003C00 00001D29829F1A>I<0E1F3F3F1D0102020404081020C0080E779F0E>39 D<000100020004000800100020006000C0018001800300070006000E000C001C00180038 00380030007000700060006000E000E000C000C000C000C000C000C000C000C000C000C0 00C000C000C0004000600060002000100010000800102E79A113>I<0010000008000004 000006000002000003000003000003000001000001800001800001800001800001800001 80000180000380000380000380000300000300000300000700000700000600000600000E 00000C00000C00001C0000180000380000300000700000600000E00000C0000180000100 000300000600000C0000180000300000600000800000112E80A113>I<7FF0FFE07FE00C 037D8A10>45 D<70F8F8F0E005057B840E>I<00000040000000C0000001800000018000 00030000000300000006000000060000000C000000180000001800000030000000300000 0060000000C0000000C0000001800000018000000300000003000000060000000C000000 0C0000001800000018000000300000003000000060000000C0000000C000000180000001 8000000300000003000000060000000C0000000C00000018000000180000003000000030 00000060000000C0000000C0000000800000001A2D7FA117>I<000F800030E000E07001 C0700380300380380700380F00780F00780E00781E00781E00703C00F03C00F03C00F03C 00F07801E07801E07801E07801C07003C0F003C0F00380F00780F00700700700700E0070 1C003038001870000FC000151F7C9D17>I<000200020006000E003C00DC031C001C0038 003800380038007000700070007000E000E000E000E001C001C001C001C0038003800380 03800780FFF80F1E7B9D17>I<0007C0001C200030200060E000C1E00181E00380C00700 000F00000E00001E00001E78001D84003E06003E07003C07007C07807807807807807807 80700F00700F00F00F00F00E00F01E00701C00601C0070380030700010C0000F8000131F 7B9D17>54 D<070F1F1F0E0000000000000000000070F8F8F0E008147B930E>58 D<00000200000006000000060000000E0000001E0000001E0000003F0000002F0000004F 0000004F0000008F0000010F0000010F0000020F0000020F0000040F00000C0F0000080F 0000100F0000100F0000200F80003FFF800040078000C007800080078001000780010007 800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFC0001E00F0001E0078001E0038001E003C003C003C003C003C003C003C003C00 3C0078007800780078007800F0007801E000F0078000FFFE0000F00F8000F003C001E001 C001E001E001E001E001E001E003C001E003C001E003C001E003C001C0078003C0078007 8007800F0007801E000F007800FFFFE0001E1F7D9E20>I<01FFFFFE001E001C001E000C 001E0004001E0004003C0004003C0004003C0004003C0004007808080078080000780800 0078180000F0300000FFF00000F0300000F0300001E0200001E0200001E0200001E00010 03C0002003C0002003C0004003C00040078000800780018007800100078007000F001F00 FFFFFE001F1F7D9E1F>69 D<01FFFFFC001E0038001E0018001E0008001E0008003C0008 003C0008003C0008003C00080078001000780800007808000078080000F0100000F03000 00FFF00000F0300001E0200001E0200001E0200001E0200003C0000003C0000003C00000 03C00000078000000780000007800000078000000F800000FFF800001E1F7D9E1E>I<01 FFFF00001E03C0001E00E0001E0070001E0078003C0078003C0078003C0078003C007800 7800F0007800F0007801E0007801C000F0070000F01E0000FFF00000F0380001E01C0001 E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003C01E0007803C0007803C0807 803C0807803C100F801C10FFF00C20000007C01D207D9E21>82 D<0007E040001C18C000 3005800060038000C0038001C00180018001000380010003800100038001000380000003 C0000003C0000003F8000001FF800001FFE000007FF000001FF0000001F8000000780000 007800000038000000380020003800200038002000300060007000600060006000E00070 00C000E8038000C606000081F800001A217D9F1A>I<00F1800389C00707800E03801C03 803C0380380700780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C 40705C40308C800F070013147C9317>97 D<007E0001C1000300800E07801E07801C0700 3C0200780000780000780000F00000F00000F00000F00000F00000700100700200300400 18380007C00011147C9315>99 D<0000780003F80000700000700000700000700000E000 00E00000E00000E00001C00001C000F1C00389C00707800E03801C03803C038038070078 0700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F 070015207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000F000F0 007000700170023804183807C010147C9315>I<00007800019C00033C00033C00071800 0700000700000E00000E00000E00000E00000E0001FFE0001C00001C00001C00001C0000 380000380000380000380000380000700000700000700000700000700000700000E00000 E00000E00000E00000C00001C00001C0000180003180007B0000F300006600003C000016 29829F0E>I<003C6000E27001C1E00380E00700E00F00E00E01C01E01C01E01C01E01C0 3C03803C03803C03803C03803C07003C07001C0F001C17000C2E0003CE00000E00000E00 001C00001C00301C00783800F0700060E0003F8000141D7E9315>I<01E0000FE00001C0 0001C00001C00001C000038000038000038000038000070000070000071E000763000E81 800F01C00E01C00E01C01C03801C03801C03801C0380380700380700380700380E10700E 20700C20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C00000000000 0000000000000000000E003300230043804300470087000E000E000E001C001C001C0038 40388030807080310033001C000B1F7C9E0E>I<03C01FC0038003800380038007000700 070007000E000E000E000E001C001C001C001C0038003800380038007000700070007100 E200E200E200E200640038000A207C9F0C>108 D<1C0F80F0002630C318004740640C00 4780680E004700700E004700700E008E00E01C000E00E01C000E00E01C000E00E01C001C 01C038001C01C038001C01C038001C01C0708038038071003803806100380380E1003803 8062007007006600300300380021147C9325>I<1C0F802630C047406047806047007047 00708E00E00E00E00E00E00E00E01C01C01C01C01C01C01C038438038838030838070838 03107003303001C016147C931A>I<007C0001C3000301800E01C01E01C01C01E03C01E0 7801E07801E07801E0F003C0F003C0F003C0F00780F00700700F00700E00301800187000 07C00013147C9317>I<01C1E002621804741C04781C04701E04701E08E01E00E01E00E0 1E00E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0072380071E000700 000700000E00000E00000E00000E00001C00001C0000FFC000171D809317>I<1C1E0026 61004783804787804707804703008E00000E00000E00000E00001C00001C00001C00001C 000038000038000038000038000070000030000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E000E700EF00CF00CE00840 1020601F8010147D9313>I<018001C0038003800380038007000700FFF007000E000E00 0E000E001C001C001C001C003800380038003820704070407080708031001E000C1C7C9B 0F>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E03801C0700 1C07001C07001C07101C0E20180E20180E201C1E200C264007C38014147C9318>I<0E03 803307802307C04383C04301C04700C08700800E00800E00800E00801C01001C01001C01 001C02001C02001C04001C04001C08000E300003C00012147C9315>I<0E00C1C03300E3 C02301C3E04381C1E04301C0E04701C060870380400E0380400E0380400E0380401C0700 801C0700801C0700801C0701001C0701001C0602001C0F02000C0F04000E13080003E1F0 001B147C931E>I E /Fj 58 122 df<007F07F001FF1FF003FF3FF00780780007007000 0F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000FFF8FF80FFF8FF80 FFF8FF800F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000 0F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000 1C20809F1B>11 D<00F8F001F8F003F8F00780F00700F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F0FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F01420809F 18>13 D<007000E001C00380078007000E001E001E003C003C003C007800780078007800 7000F000F000F000F000F000F000F000F000F000F000F000F00070007800780078007800 3C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<0006000000060000 000600000006000000060000000600000006000000060000000600000006000000060000 00060000000600000006000000060000FFFFFFF0FFFFFFF0000600000006000000060000 000600000006000000060000000600000006000000060000000600000006000000060000 0006000000060000000600001C207D9A23>43 D<787878781830306060E0050A7D830D> III<00C001C00FC0FFC0FFC0F3C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C003C003C003C0FFFEFFFEFFFE0F1F7C9E17>49 D<07F0000FFC001FFE00383F00700F 00600780E00780E003C04003C04003C00003C00003C00003C0000780000780000F00000E 00001C00003C0000780000E00001C0000380000700000E00001C0000380000700000FFFF C0FFFFC0FFFFC0121F7E9E17>I61 D<001F0000001F0000003F8000003B8000003B8000007BC0000073C0000071C00000F1E0 0000E1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C 0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F001E000F003C000F803C0007 803C000780780007C0780003C0780003C0F00003E01B207F9F1E>65 DI<001FC000 FFF801FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000780000F0 0000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00003C 00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>IIII<001FE000FFF801FFFE03E03E07800E0F0000 1E00003E00003C00007C0000780000780000780000F00000F00000F00000F00000F00000 F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07801E 03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F000000FF C00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F00780007807800 078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F80007C07800078078000780780007803C000F003C000F001E001E001F00 3E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C 00007800007800007800007800007800007C00003C00003F00001FE0000FFC0007FE0001 FF00003F800007C00003C00003E00001E00001E00001E00001E00001E00001C0C003C0F0 07C0FC0F807FFF001FFE0003F80013227EA019>III II<780007807C000F003E001F00 1E001E000F003C000F807C000780780003C0F00003E1F00001F1E00000F3C000007FC000 007F8000003F0000001F0000001E0000003F0000007F8000007FC00000F3C00001F1E000 01E0F00003C0F80007C0780007803C000F003E001F001E001E000F003C000F807C0007C0 780003C0F00003E01B207F9F1E>II I<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7F FF3FEF1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000F0007800 78003C013E0F1FFF0FFE03F010147E9314>I<0007800007800007800007800007800007 8000078000078000078000078000078000078007C7800FF7801FFF803E1F807C07807807 80F80780F00780F00780F00780F00780F00780F00780F00780780780780F803E1F801FFF 800FF78007C78011207E9F17>I<03F0000FFC001FFE003E1F003C0700780700700380FF FF80FFFF80FFFF80F00000F00000F000007000007800003C01003E07001FFF0007FE0001 F80011147F9314>I<007E01FE03FE078007000F000F000F000F000F000F000F00FFF0FF F0FFF00F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F 000F20809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00780F00780F00780F00780F0078 0F003C1E003E3E001FFC003FF80033E0003000003800003FFE003FFF801FFFC03FFFE078 03F0F000F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE00 1FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000F07801E0 7801E03C03C03F0FC01FFF8007FE0001F80014147F9317>II114 D<07F01FFC3FFC780C78007800 78007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311>I<1E 001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E001E001E 001E001E001E001E201FF00FF007C00C1A7F9910>IIII<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F80000F0 0001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414809315 >II E /Fk 15 117 df<020408103020604040C0C0C0C0C0C0C0C0404060203010080402071A7F920C >40 D<8040201018080C0404060606060606060604040C081810204080071A7E920C>I< 1F00318060C04040C060C060C060C060C060C060C060C060404060C031801F000B107F8F 0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C000C000C000C00FF80 09107E8F0F>I<1F00618040C08060C0600060006000C00180030006000C00102020207F C0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F00008000400060C060C060 804060801F000B107F8F0F>I<0300030007000F000B001300330023004300C300FFE003 000300030003001FE00B107F8F0F>I<20803F002C002000200020002F00308020400060 00600060C06080C061801F000B107F8F0F>I<0780184030C060C06000C000CF00F080E0 40C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC08080808001000200 040004000C0008000800180018001800180018000B117E900F>I<1F00318060C060C060 C071803F000F00338061C0C060C060C060404060801F000B107F8F0F>I<1F00318060C0 C040C060C060C06040E021E01E600060004060C0608043003E000B107F8F0F>I<03E000 0C1800100400200200600300400100C00180C00180C00180C00180C00180600300600300 3006003006000C180003E00011117E9017>79 D<1F8030C06000C000C000C000C000C000 604030801F000A0B7F8A0E>99 D<10103030FE3030303030323232321C070F7F8E0C> 116 D E /Fl 7 104 df0 D21 D<00000C00000C0000180000180000300000300000600000600000C00000C00001800001 80000180000300000300000600000600000C00000C000018000018000030000030000060 0000600000C00000C0000180000180000300000300000600000600000600000C00000C00 00180000180000300000300000600000600000C00000400000162C7AA000>54 D62 D<00040000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000FFFFFFE0FFFFFFE01B1C7C9B23>I<000F0038006000E001C001C0 01C001C001C001C001C001C001C001C001C001C001C001C001C0038007001E00F8001E00 0700038001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C000E0 00600038000F102D7DA117>102 DI E /Fm 77 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070 F8F8F870051C779B18>33 D<4010E038F078E038E038E038E038E038E038E038E038E038 E03860300D0E7B9C18>I<01C00007E0000FF0000E70001C38001C38001C38001C38001C 73F01C73F01CE3F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0 F800E0F800E07070E0787070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F001E001C00380038007000700070007000E000E000E000E0 00E000E000E000E0007000700070007000380038001C001E000F00078003C001F000F000 700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000 E00070007000700070007000700070007000E000E000E000E001C001C0038007800F001E 003C007800F00060000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF 807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C0 0001C00011147D9718>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<7FFF00FFFF80FFFF807FFF0011047D8F18>I<3078FCFC78300606778518>I<00030000 0780000780000F80000F00001F00001E00001E00003E00003C00007C0000780000780000 F80000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F00001F 00001E00003E00003C00003C00007C0000780000F80000F00000F0000060000011247D9F 18>I<01F00007FC000FFE001F1F001C07003803807803C07001C07001C0E000E0E000E0 E000E0E000E0E000E0E000E0E000E0E000E0E000E0F001E07001C07001C07803C0380380 1C07001F1F000FFE0007FC0001F000131C7E9B18>I<01800380038007800F803F80FF80 FB8043800380038003800380038003800380038003800380038003800380038003800380 7FFCFFFE7FFC0F1C7B9B18>I<03F0000FFE003FFF007C0F807003C0E001C0F000E0F000 E06000E00000E00000E00001C00001C00003C0000780000F00001E00003C0000780000F0 0001E00007C0000F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001F FE003FFF007807807803C07801C03001C00001C00003C0000380000F0003FF0003FE0003 FF000007800003C00001C00000E00000E00000E0F000E0F000E0F001C0F003C07C07803F FF001FFE0003F800131C7E9B18>I<1FFF803FFF803FFF80380000380000380000380000 3800003800003800003800003BF8003FFE003FFF003C07801803C00001C00000E00000E0 6000E0F000E0F000E0E001C07003C07C0F803FFF001FFC0003F000131C7E9B18>53 D<007E0001FF0007FF800F83C01E03C01C03C0380180380000700000700000E1F800E7FE 00FFFF00FE0780F803C0F001C0F000E0E000E0F000E07000E07000E07000E03801C03C03 C01E07800FFF0007FE0001F800131C7E9B18>I<3078FCFC783000000000000000003078 FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78 F060071A789318>I<000300000780001F80003F00007E0001FC0003F00007E0001FC000 3F00007E0000FC0000FC00007E00003F00001FC00007E00003F00001FC00007E00003F00 001F8000078000030011187D9918>I<7FFFC0FFFFE0FFFFE0FFFFE00000000000000000 00000000FFFFE0FFFFE0FFFFE07FFFC0130C7E9318>I<600000F00000FC00007E00003F 00001FC00007E00003F00001FC00007E00003F00001F80001F80003F00007E0001FC0003 F00007E0001FC0003F00007E0000FC0000F0000060000011187D9918>I<0FF0003FFC00 7FFF00700F00F00380F00380600780000F00003E00007C0001F00001E00003C00003C000 03C00003C00003C00003800000000000000000000000000000000003800007C00007C000 07C000038000111C7D9B18>I<00700000F80000F80000D80000D80001DC0001DC0001DC 00018C00038E00038E00038E00038E000306000707000707000707000707000FFF800FFF 800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>65 DI<00F8E003FEE007FFE00F07E01E03E03C 01E03800E07000E07000E0700000E00000E00000E00000E00000E00000E00000E00000E0 00007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE0000F800131C7E9B 18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00E01C00F01C0070 1C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00E01C01E01C01C0 1C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FDC00FFFC01F0FC0 1C03C03803C03801C07001C07001C0700000E00000E00000E00000E00000E00000E00FF0 E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFFC003FDC001F1C0 141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C01C01C01C01C01 C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C01C01C01C01C01 C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00FFFF807FFF0001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C0007FFF00FFFF807F FF00111C7D9B18>I<7F07F0FF87F87F07F01C03C01C07801C07001C0E001C1E001C3C00 1C38001C70001CF0001DF0001DF0001FB8001FB8001F1C001E1C001C0E001C0E001C0700 1C07001C03801C03801C01C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00700E00700E00700E00 700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C0 1CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C0DC0 1C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F007007 00F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C 7D9B18>II<0FF8003FFE007FFF00780F00 700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00 001C00001E00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF001C0F 801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F 001C07001C03801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F87F00 F0161C7F9B18>I<03F3801FFF803FFF807C0F80700780E00380E00380E00380E0000070 00007800003F00001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E0 00E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8 E07038E07038E07038E07038007000007000007000007000007000007000007000007000 00700000700000700000700000700000700000700000700000700000700007FF0007FF00 07FF00151C7F9B18>IIII<7F8FE07F9F E07F8FE00E07000F0700070E00078E00039C0003DC0001F80001F80000F00000F0000070 0000F00000F80001F80001DC00039E00038E00070F000707000E07800E03801E03C07F07 F0FF8FF87F07F0151C7F9B18>II<3FFFE0 7FFFE07FFFE07001C07003C0700780700700000F00001E00001C00003C00007800007000 00F00001E00001C00003C0000780000700000F00001E00E01C00E03C00E07800E07000E0 FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E00 00070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F00F00781F00 3FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E 00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E 00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C809B18>I< 01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E00000E00000 E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80003F 80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80380F807007 80700380E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF F00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F00380780700380700380E0 01C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF8007FF0001 FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001C0007FFFC0 FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFF F81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC001FF80039E0 003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E000387800 F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00000E00000E 00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B 18>I<03800007C00007C00007C0000380000000000000000000000000007FC000FFC000 7FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I107 D<7FE000FFE0007FE00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E9B18>I<7CE0E000FFFBF8 007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C007F1F1F00FFBFBF807F1F1F 001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC17148093 18>I<01F0000FFE001FFF003E0F803803807001C07001C0E000E0E000E0E000E0E000E0 E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001F00013147E9318>I<7E3E 00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E00380E00380E00 380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00000E00000E00000E00000E00 000E00000E00007FC000FFE0007FC000151E809318>I<01E38007FB801FFF803E1F8038 0F80700780700780E00380E00380E00380E00380E00380E00380700780700780380F803C 1F801FFF800FFB8003E380000380000380000380000380000380000380000380003FF800 3FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F87803F03003E00003C00003C000 0380000380000380000380000380000380000380000380000380007FFE00FFFF007FFE00 15147F9318>I<07F7003FFF007FFF00780F00E00700E00700E007007C00007FE0001FFC 0003FE00001F00600780E00380E00380F00380F80F00FFFF00FFFC00E7F00011147D9318 >I<0180000380000380000380000380007FFFC0FFFFC0FFFFC003800003800003800003 80000380000380000380000380000380000380400380E00380E00380E001C1C001FFC000 FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F03E007FFFC03FFFE01FCFC 1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E03800E03800707000707000707 00038E00038E00038E00038E0001DC0001DC0001DC0000F80000F80000700015147F9318 >II<7F8FF0 7F9FF07F8FF0070700078E00039E0001DC0001F80000F80000700000F00000F80001DC00 039E00038E000707000F07807F8FF0FF8FF87F8FF015147F9318>I<7F8FF0FF8FF87F8F F00E01C00E03800E0380070380070700070700038700038600038E0001CE0001CE0000CC 0000CC0000DC0000780000780000780000700000700000700000F00000E00079E0007BC0 007F80003F00001E0000151E7F9318>I<3FFFF07FFFF07FFFF07001E07003C070078000 0F00001E00003C0000F80001F00003C0000780000F00701E00703C0070780070FFFFF0FF FFF0FFFFF014147F9318>I<0007E0001FE0007FE000780000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00001E0007FC000FF8000FF80007FC000 01E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 007800007FE0001FE00007E013247E9F18>I<7C0000FF0000FFC00003C00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000F000007FC0003F E0003FE0007FC000F00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00003C000FFC000FF00007C000013247E9F18>125 D E /Fn 48 124 df<004000800100020006000C000C0018001800300030007000600060 006000E000E000E000E000E000E000E000E000E000E000E000E000600060006000700030 003000180018000C000C00060002000100008000400A2A7D9E10>40 D<800040002000100018000C000C000600060003000300038001800180018001C001C001 C001C001C001C001C001C001C001C001C001C0018001800180038003000300060006000C 000C00180010002000400080000A2A7E9E10>I<00060000000600000006000000060000 000600000006000000060000000600000006000000060000000600000006000000060000 FFFFFFE0FFFFFFE000060000000600000006000000060000000600000006000000060000 0006000000060000000600000006000000060000000600001B1C7E9720>43 D<60F0F0701010101020204080040C7C830C>II<60F0F0600404 7C830C>I<07C01830201C400C400EF00FF80FF807F8077007000F000E000E001C001C00 380070006000C00180030006010C01180110023FFE7FFEFFFE101C7E9B15>50 D<0FE03038401CE00EF00EF00EF00E000C001C0030006000C00080018001000100010001 00010001000000000000000000000003000780078003000F1D7E9C14>63 D66 D<001F808000E0618001801980070007 800E0003801C0003801C00018038000180780000807800008070000080F0000000F00000 00F0000000F0000000F0000000F0000000F0000000F00000007000008078000080780000 80380000801C0001001C0001000E000200070004000180080000E03000001FC000191E7E 9C1E>III73 D77 DI<003F800000E0E0000380380007001C000E000E001C000700 3C00078038000380780003C0780003C0700001C0F00001E0F00001E0F00001E0F00001E0 F00001E0F00001E0F00001E0F00001E0700001C0780003C0780003C0380003803C000780 1C0007000E000E0007001C000380380000E0E000003F80001B1E7E9C20>II<07E0801C1980300580700380600180E00180E00080E0 0080E00080F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007C000 03C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E7E9C 17>83 D<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F0020 800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 001F800003FFFC001B1C7F9B1E>IIII<1FC000307000783800781C00301C00001C0000 1C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E801F870012 127E9115>97 DI<07E00C30187830 7870306000E000E000E000E000E000E00060007004300418080C3007C00E127E9112>I< 003F0000070000070000070000070000070000070000070000070000070000070003E700 0C1700180F00300700700700600700E00700E00700E00700E00700E00700E00700600700 700700300700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006E006 FFFEE000E000E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E06 1E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C38801818 00381C00381C00381C00381C001818001C38000C300013C0001000003000001800001FF8 001FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007F800121C 7F9215>II<18003C003C00180000 00000000000000000000000000FC001C001C001C001C001C001C001C001C001C001C001C 001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C00000000000000000 0000000000000FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>IIIII<03F0000E1C001806003003007003 80600180E001C0E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C 0003F00012127F9115>II114 D<1F9030704030C010C010E010F8007F803FE00FF000F880388018C018C018E010D0608F C00D127F9110>I<04000400040004000C000C001C003C00FFE01C001C001C001C001C00 1C001C001C001C001C101C101C101C101C100C100E2003C00C1A7F9910>IIII<7F8FF00F03800F030007020003840001C80001D8 0000F00000700000780000F800009C00010E00020E000607000403801E07C0FF0FF81512 809116>II<7FFC70386038407040F040E041C003C00380 07000F040E041C043C0C380870087038FFF80E127F9112>II E /Fo 16 118 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D<001FE02000FFF8E003F80FE007C003E00F8001E01F0000E03E0000E03E0000607E0000 607C000060FC000000FC000000FC000000FC000000FC000000FC000000FC000000FC0000 007C0000607E0000603E0000603E0000C01F0000C00F80018007C0030003F80E0000FFFC 00001FE0001B1C7D9B22>67 DI77 D<07F8201FFEE03C07E07801E07000E0F000E0F00060F00060F80000FE0000FFE0007FFE 003FFF003FFF800FFFC007FFE0007FE00003F00001F00000F0C000F0C000F0C000E0E000 E0F001C0FC03C0EFFF0083FC00141C7D9B1B>83 D<0FF8001C1E003E0F803E07803E07C0 1C07C00007C0007FC007E7C01F07C03C07C07C07C0F807C0F807C0F807C0780BC03E13F8 0FE1F815127F9117>97 D<03FC000E0E001C1F003C1F00781F00780E00F80000F80000F8 0000F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115>99 D<01FC000F07001C03803C01C07801C07801E0F801E0F801E0FFFFE0F80000F80000F800 007800007C00603C00601E00C00F038001FC0013127F9116>101 D<03F8F00E0F381E0F381C07303C07803C07803C07803C07801C07001E0F000E0E001BF8 001000001800001800001FFF001FFFC00FFFE01FFFF07801F8F00078F00078F000787000 707800F01E03C007FF00151B7F9118>103 D<1E003F003F003F003F001E000000000000 00000000000000FF00FF001F001F001F001F001F001F001F001F001F001F001F001F001F 001F00FFE0FFE00B1E7F9D0E>105 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800 F8F800F8F800F87800F07800F03C01E01E03C00F078001FC0015127F9118>I114 D<1FD830786018E018E018F000FF 807FE07FF01FF807FC007CC01CC01CE01CE018F830CFC00E127E9113>I<030003000300 0300070007000F000F003FFCFFFC1F001F001F001F001F001F001F001F001F001F0C1F0C 1F0C1F0C0F08079803F00E1A7F9913>II E /Fp 35 122 df46 D<007F000001FFC00007 FFF0000FFFF8000FC1F8001F007C003F007E003E003E003C001E007C001F007C001F007C 001F0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F8078000F007C 001F007C001F007C001F003E003E003E003E003F007E001F80FC000FC1F8000FFFF80007 FFF00001FFC000007F000019297EA71E>48 D<00180000380000F80007F800FFF800FFF8 00F8F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF07FFFF014287BA71E >I<00FE0003FFC007FFE00FFFF01F03F83C00FC38007E78003E70003EF0001FF0001F60 001F20001F00001F00001F00001F00003E00003E00007C00007C0000F80001F00001E000 03C0000780000F00001E00003C0000780000F00001E00003C0000780000F00001E00003C 00007FFFFF7FFFFF7FFFFF7FFFFF18287EA71E>I<007F000001FFC00007FFF0000FFFF8 001FC1F8003E007C003C003E0078003E0038003E0010003E0000003E0000003E0000003C 0000007C000000FC000001F8000007F00000FFE00000FFC00000FFE00000FFF0000001FC 0000007C0000003E0000001F0000001F0000000F8000000F8000000F8000000F8000000F 8040000F8060001F00F0001F00F8003F007E007E003F81FC001FFFF8000FFFF00003FFE0 00007F000019297EA71E>I<0003F0000007F0000005F000000DF000000DF000001DF000 0039F0000039F0000079F0000079F00000F1F00000F1F00001E1F00003E1F00003E1F000 07C1F00007C1F0000F81F0000F81F0001F01F0001F01F0003E01F0007C01F0007C01F000 F801F000FFFFFF80FFFFFF80FFFFFF80FFFFFF800001F0000001F0000001F0000001F000 0001F0000001F0000001F0000001F0000001F0000001F00019277EA61E>I<3FFFFC3FFF FC3FFFFC3FFFFC3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00 003E3F003EFFC03FFFE03FFFF03FE1F83F807C3F003E3E003E00003E00001F00001F0000 1F00001F00001F00001F00001F20001F60003E70003EF8007C7C00FC3F03F81FFFF00FFF E007FF8000FE0018287EA61E>I<0001FF00000FFFE0003FFFF8007FFFF800FE01F801F8 003003F0001007C000000F8000001F8000001F0000003E0000003E0000007E0000007C00 00007C0000007C000000F8000000F8000000F8000000F8000000F8000000F8000000F800 0000F8000000F8000000F80000007C0000007C0000007C0000007E0000003E0000003E00 00001F0000001F8000000F80000007C0000003F0000401F8001C00FE00FC007FFFFC003F FFF8000FFFE00001FF001E2C7CAA26>67 DII73 D76 DII80 D<007FC00001FFF80007FFFE 000FFFFF001FC07F003F000F007E0006007C0000007C000000F8000000F8000000F80000 00F8000000F8000000FC0000007E0000007F0000003F8000001FF800000FFF800007FFE0 0003FFF80000FFFC00000FFE000000FF0000003F0000001F8000000F8000000FC0000007 C0000007C0000007C0000007C0000007C0000007C000000F8060000F80F0001F00FC003F 00FF80FE007FFFFC001FFFF80007FFE00000FF80001A2C7DAA21>83 D<01FE000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F80000F80000F80000 F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F800F8F800F8FC01F87E07F87FFF F83FFFF81FFCF80FE0F8151B7E9A1D>97 DI<007FC001FFF007FFFC0FFFFC1FC07C1F00083E00007C00007C00007C00 00F80000F80000F80000F80000F80000F80000F800007C00007C00007E00003E00001F00 0C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<00003E00003E00003E00003E00 003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00FC3E03 FF3E07FFFE0FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8003EF8003EF8 003EF8003EF8003EF8003EFC003E7C003E7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03 FF3E00FC3E172A7EA91F>I<007E0003FF8007FFC00FFFE01F83F03F00F03E00787C0078 7C003878003CFFFFFCFFFFFCFFFFFCFFFFFCF80000F80000F800007800007C00007C0000 3E00003F000C1FC07C0FFFFC07FFFC01FFF0007F80161B7E9A1B>I<001FC0007FC000FF C001FFC003F00003E00007C00007C00007C00007C00007C00007C00007C00007C00007C0 00FFFE00FFFE00FFFE0007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C0 0007C00007C00007C000122A7FA912>I<00F8078003FE7FC00FFFFFC01FFFFFC01F07C0 003E03E0003E03E0007C01F0007C01F0007C01F0007C01F0007C01F0007C01F0003E03E0 003E03E0001F07C0001FFFC0003FFF80003BFE000038F8000078000000780000003C0000 003FFFC0003FFFF8001FFFFC001FFFFE003FFFFF007C007F00F8001F80F8000F80F8000F 80F8000F80FC001F807E003F003F80FE003FFFFE000FFFF80007FFF00000FF80001A287E 9A1E>I105 D108 DII<007F000001FFC00007FFF0000FFFF8001FC1FC003F 007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001FC1FC000F FFF80007FFF00001FFC000007F0000191B7E9A1E>II114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8 003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFF C07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007 C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003 FFE003FFE001FF8000FC0013227FA116>II119 D121 D E /Fq 9 122 df77 D<0007FFFC000000007FFFFFC0000001FFFFFFF8000003FFFFFFFE000007FE001FFF0000 07FF0003FFC0000FFF8001FFE0000FFF8000FFF0000FFF80007FF0000FFF80007FF8000F FF80007FF80007FF00003FFC0007FF00003FFC0003FE00003FFC0000F800003FFC000000 00003FFC00000000003FFC00000000003FFC00000000003FFC00000007FFFFFC000000FF FFFFFC000007FFFFFFFC00003FFFE03FFC0000FFFE003FFC0003FFF0003FFC0007FFC000 3FFC000FFF00003FFC001FFE00003FFC003FFC00003FFC007FF800003FFC007FF800003F FC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC00FFF000003FFC 00FFF000007FFC007FF80000FFFC007FF80001EFFC003FFC0003EFFC003FFF0007CFFF00 0FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC03FFFC007FFFF001FFFC0003FF80007FF836 2E7DAD3A>97 D<00001FFFC0000000FFFFF8000007FFFFFE00001FFFFFFF80007FFC00FF C000FFE001FFC001FFC003FFE003FF8003FFE007FF0003FFE00FFE0003FFE00FFE0003FF E01FFC0001FFC01FFC0001FFC03FFC0000FF803FFC00003E007FF8000000007FF8000000 007FF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF8000000 00FFF800000000FFF800000000FFF800000000FFF800000000FFF8000000007FF8000000 007FF8000000007FFC000000003FFC000000003FFC000000001FFC000000F81FFE000000 F80FFE000000F80FFF000001F007FF800003F003FFC00007E001FFE0000FC000FFF0001F 80007FFE00FF00001FFFFFFE000007FFFFF8000000FFFFE00000001FFE00002D2E7CAD35 >99 D<00001FFE00000001FFFFE0000007FFFFF800001FFFFFFE00007FFC07FF0000FFE0 01FF8001FFC0007FC003FF80003FE007FF00003FF00FFE00001FF01FFE00000FF81FFC00 000FF83FFC00000FFC3FFC000007FC7FFC000007FC7FF8000007FC7FF8000007FE7FF800 0007FEFFF8000007FEFFF8000007FEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFF FFFFFCFFF800000000FFF800000000FFF800000000FFF8000000007FF8000000007FF800 0000007FFC000000003FFC000000003FFC000000003FFC0000001C1FFE0000003E0FFE00 00003E07FF0000007E07FF000000FC03FF800001F801FFC00003F0007FF0001FE0003FFE 00FFC0001FFFFFFF800007FFFFFE000000FFFFF80000000FFF80002F2E7DAD36>101 D<00FC0001FE0003FF0007FF800FFFC01FFFE01FFFE01FFFE01FFFE01FFFE01FFFE00FFF C007FF8003FF0001FE0000FC000000000000000000000000000000000000000000000000 00000000000000000000007FC0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFFFFFFFFFF FFFFFFFF18497CC820>105 D<007FC000FFFFC000FFFFC000FFFFC000FFFFC000FFFFC0 0003FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC000FFFFFF80FFFFFF 80FFFFFF80FFFFFF80FFFFFF8019487CC720>108 D<007FC001FFC00000FFFFC00FFFF8 0000FFFFC03FFFFE0000FFFFC0FFFFFF0000FFFFC1FC07FF8000FFFFC3E003FFC00003FF C7C001FFC00001FFCF0001FFE00001FFDE0000FFE00001FFDC0000FFE00001FFFC0000FF F00001FFF80000FFF00001FFF00000FFF00001FFF00000FFF00001FFF00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF000FFFFFFC07FFFFFE0FFFF FFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B2E7CAD42> 110 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FFE01FC0001FE03F8000 0FE03F000007E07F000003E07F000003E0FF000003E0FF000003E0FF800003E0FFC00000 00FFF0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFFFFFC001FFFFFFF00 0FFFFFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FFFFF800001FFFF800 0000FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003FCFC000003FCFE00 0003F8FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0FFFC00FFC0FFFFFF FF80FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>115 D<7FFFFFC000FFFF807FFF FFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF8000FFF000000F E00000FFF800000FC00000FFF800000FC000007FFC00000F8000007FFC00001F8000003F FC00001F0000003FFE00003F0000001FFE00003E0000001FFF00007E0000000FFF00007C 0000000FFF8000FC00000007FF8000F800000007FFC001F800000003FFC001F000000003 FFE003F000000003FFE003F000000001FFF003E000000001FFF007E000000000FFF007C0 00000000FFF80FC0000000007FF80F80000000007FFC1F80000000003FFC1F0000000000 3FFE3F00000000001FFE3E00000000001FFF7E00000000000FFF7C00000000000FFFFC00 000000000FFFFC000000000007FFF8000000000007FFF8000000000003FFF00000000000 03FFF0000000000001FFE0000000000001FFE0000000000000FFC0000000000000FFC000 00000000007F800000000000007F800000000000003F000000000000003F000000000000 003F000000000000003E000000000000007E000000000000007C00000000000000FC0000 00001F8000F8000000003FC001F8000000007FE001F000000000FFF003F000000000FFF0 03E000000000FFF007E000000000FFF00FC000000000FFF01F8000000000FFF03F800000 00007FE07F00000000007F43FE00000000003FFFF800000000001FFFF0000000000007FF C0000000000001FE00000000000039427EAD3F>121 D E /Fr 9 117 df<0001FF8000000FFFF000003FFFFC0000FF81FF0001FE007F8003FC003FC007F8 001FE00FF8001FF00FF0000FF01FF0000FF81FF0000FF83FF0000FFC3FE00007FC3FE000 07FC7FE00007FE7FE00007FE7FE00007FE7FE00007FE7FE00007FEFFE00007FFFFE00007 FFFFE00007FFFFE00007FFFFE00007FFFFE00007FFFFE00007FFFFE00007FFFFE00007FF FFE00007FFFFE00007FFFFE00007FFFFE00007FFFFE00007FFFFE00007FFFFE00007FFFF E00007FFFFE00007FFFFE00007FF7FE00007FE7FE00007FE7FE00007FE7FE00007FE7FE0 0007FE3FE00007FC3FF0000FFC3FF0000FFC1FF0000FF81FF0000FF80FF0000FF00FF800 1FF007F8001FE003FC003FC001FE007F8000FF81FF00003FFFFC00000FFFF0000001FF80 0028397CB731>48 D<00001E000000003E00000000FE00000003FE0000003FFE0000FFFF FE0000FFFFFE0000FFFFFE0000FFCFFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE00 00000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000 000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE000000 0FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000F FE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE0000000FFE 0000000FFE0000000FFE0000000FFE00007FFFFFFFC07FFFFFFFC07FFFFFFFC07FFFFFFF C0223879B731>I<0000001FFF000030000001FFFFE000F000000FFFFFFC01F000007FFF FFFE03F00001FFFE007F87F00003FFE0000FCFF0000FFF000003FFF0001FFC000001FFF0 003FF80000007FF0007FF00000003FF000FFC00000003FF001FFC00000001FF003FF8000 00000FF007FF000000000FF00FFF0000000007F00FFE0000000007F01FFE0000000003F0 1FFE0000000003F03FFC0000000003F03FFC0000000001F03FFC0000000001F07FFC0000 000001F07FF80000000001F07FF80000000000007FF8000000000000FFF8000000000000 FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF80000 00000000FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000 FFF80000000000007FF80000000000007FF80000000000007FF80000000000007FFC0000 000000F03FFC0000000000F03FFC0000000000F03FFC0000000000F01FFE0000000000F0 1FFE0000000001E00FFE0000000001E00FFF0000000001E007FF0000000003C003FF8000 000003C001FFC0000000078000FFE00000000F00007FF00000001F00003FF80000003E00 001FFC0000007C00000FFF000001F8000003FFE00007F0000001FFFE003FC00000007FFF FFFF000000000FFFFFFC0000000001FFFFF000000000001FFF0000003C3D7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF00001FFC00FF80 001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF00007F0003FF0 0001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000000FFFFFF0 00007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF0003FE0003FF0 007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000FFC0003FF0 00FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF007FFFE0FFF F001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFFFE00 00FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03FF00007F83F F00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFFFFFFFCFFFF FFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE00000007FE000 00007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF800007807FC0000 F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE00026267DA52D >101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF0000000007FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF007FC00003FF 01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF6000FF8003FF E000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC3FFFFFFFFF FC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00FFFFFE03FF 00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF803FF00001F FC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF03FF000007 FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007 FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC03FF00000F FC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC003FFF001FF 8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000FFFFFC0000 00FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fs 73 123 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E743A040204020402080408041008 1008201040200F0E7E9F17>34 D<0020004000800100020006000C000C00180018003000 300030007000600060006000E000E000E000E000E000E000E000E000E000E000E000E000 6000600060007000300030003000180018000C000C000600020001000080004000200B2E 7DA112>40 D<800040002000100008000C00060006000300030001800180018001C000C0 00C000C000E000E000E000E000E000E000E000E000E000E000E000E000C000C000C001C0 01800180018003000300060006000C00080010002000400080000B2E7DA112>I<000600 000006000000060000000600000006000000060000000600000006000000060000000600 000006000000060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600 000006000000060000000600000006000000060000000600000006000000060000000600 00000600000006000000060000000600001C207D9A23>43 D<70F8FCFC74040404080810 102040060E7C840D>II<70F8F8F87005057C840D>I<00010003 0003000600060006000C000C000C00180018001800300030003000600060006000C000C0 00C00180018001800300030003000600060006000C000C000C0018001800180030003000 3000600060006000C000C000C000102D7DA117>I<03F0000E1C001C0E00180600380700 700380700380700380700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C07003807003807003807807803807001806001C0E00 0E1C0003F000121F7E9D17>I<018003800F80F380038003800380038003800380038003 80038003800380038003800380038003800380038003800380038003800380038007C0FF FE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F8 03C02007C00007C0000780000780000F00000E00001C0000380000700000600000C00001 80000300000600400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F000 0C1C00100E00200F00780F80780780780780380F80000F80000F00000F00000E00001C00 00380003F000003C00000E00000F000007800007800007C02007C0F807C0F807C0F807C0 F00780400780400F00200E001C3C0003F000121F7E9D17>I<000600000600000E00000E 00001E00002E00002E00004E00008E00008E00010E00020E00020E00040E00080E00080E 00100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E00000E00000E00000E 00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE00010000010000010 000010000010000010000011F000161C00180E001007001007800003800003800003C000 03C00003C07003C0F003C0F003C0E00380400380400700200600100E000C380003E00012 1F7E9D17>I<007C000182000701000E03800C07801C0780380300380000780000700000 700000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C0 7003C07003C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<4000 007FFFC07FFF807FFF804001008002008002008004000008000008000010000020000020 0000400000400000C00000C00001C0000180000380000380000380000380000780000780 00078000078000078000078000078000030000121F7D9D17>I<03F0000C0C0010060030 03002001806001806001806001807001807803003E03003F06001FC8000FF00003F80007 FC000C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0C0008060018020 01001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C00380600380700700700 700380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007C03807C0180BC0 0E13C003E3C0000380000380000380000700300700780600780E00700C00201800107000 0FC000121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F87005147C930D>I< 70F8F8F8700000000000000000000070F0F8F878080808101010202040051D7C930D>I< 7FFFFFE0FFFFFFF000000000000000000000000000000000000000000000000000000000 00000000FFFFFFF07FFFFFE01C0C7D9023>61 D<0FC0307040384038E03CF03CF03C603C 0038007000E000C001800180010003000200020002000200020002000000000000000000 000007000F800F800F8007000E207D9F15>63 D<00010000000380000003800000038000 0007C0000007C0000007C0000009E0000009E0000009E0000010F0000010F0000010F000 00207800002078000020780000403C0000403C0000403C0000801E0000801E0000FFFE00 01000F0001000F0001000F00020007800200078002000780040003C00E0003C01F0007E0 FFC03FFE1F207F9F22>65 DI<00 0FC040007030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C 0000407C00004078000040F8000000F8000000F8000000F8000000F8000000F8000000F8 000000F8000000F8000000780000007C0000407C0000403C0000401C0000401E0000800E 000080070001000380020001C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000 E004E0038002E0070001E00F0000E01E0000601E0000603C0000603C0000207C00002078 000020F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8007FFCF8 0003E0780001E07C0001E03C0001E03C0001E01E0001E01E0001E00F0001E0070001E003 8002E000E0046000781820000FE0001E217D9F24>III76 DII<001F800000F0F00001C0380007801E000F000F000E0007001E0007803C00 03C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F80001F0F80001F0F800 01F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C0003E03C0003C03C00 03C01E0007800E0007000F000F0007801E0001C0380000F0F000001F80001C217D9F23> II82 D<07E0800C1980100780300380600180600180E001 80E00080E00080E00080F00000F000007800007F00003FF0001FFC000FFE0003FF00001F 800007800003C00003C00001C08001C08001C08001C08001C0C00180C00380E00300F006 00CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F 0030800F0010800F0010800F0010800F0010000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000001F800007FFFE001C1F7E9E21> II87 D<7FF83FF80FE00FC007C0070003C0020001E0040001F00C00 00F0080000781000007C1000003C2000003E4000001E4000000F8000000F800000078000 0003C0000007E0000005E0000009F0000018F8000010780000207C0000603C0000401E00 00801F0001800F0001000780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I<08 0410082010201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>92 D<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E001E0E00380E 00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>I E /Ft 46 122 df<00200040008001000300060004000C000C0018001800300030003000 7000600060006000E000E000E000E000E000E000E000E000E000E000E000E000E000E000 6000600060007000300030003000180018000C000C000400060003000100008000400020 0B327CA413>40 D<800040002000100018000C0004000600060003000300018001800180 01C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00C000C000C001C0018001800180030003000600060004000C0018001000200040008000 0B327DA413>I<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<008003800F80F3800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 8003800380038003800380038007C0FFFE0F217CA018>49 D<03F0000C1C001007002007 804003C04003C08003E0F003E0F801E0F801E0F801E02003E00003E00003C00003C00007 80000700000E00001C0000180000300000600000C0000180000100000200200400200800 201800603000403FFFC07FFFC0FFFFC013217EA018>I<000200000600000E00000E0000 1E00001E00002E00004E00004E00008E00008E00010E00020E00020E00040E00040E0008 0E00100E00100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E0000 0E00000E00000E00001F0001FFF015217FA018>52 D<1000801E07001FFF001FFE001FF8 0013E00010000010000010000010000010000010000010F800130E001407001803801003 800001C00001C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C04003 C04003802007001006000C1C0003F00013227EA018>I<007E0001C1000300800601C00E 03C01C03C0180180380000380000780000700000700000F0F800F30C00F40600F40300F8 0380F801C0F001C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C038 01C01803801C03000C0600070C0001F00013227EA018>I<01F800060E00080300100180 2001802000C06000C06000C06000C07000C07801803E01003F02001FC4000FF80003F800 03FC00067F00083F80100F803007C06001C06000E0C000E0C00060C00060C00060C00060 6000406000C03000801803000E0E0003F00013227EA018>56 D<01F000060C000C060018 0700380380700380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E070 01E07003E03803E01805E00C05E00619E003E1E00001C00001C00001C000038000038030 0300780700780600700C002018001030000FC00013227EA018>I<000180000001800000 0180000003C0000003C0000003C0000005E0000005E000000DF0000008F0000008F00000 10F800001078000010780000203C0000203C0000203C0000401E0000401E0000401E0000 800F0000800F0000FFFF000100078001000780030007C0020003C0020003C0040003E004 0001E0040001E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E0000701E0000701C0000 303C0000303C0000307C0000107800001078000010F8000000F8000000F8000000F80000 00F8000000F8000000F8000000F800000078000000780000107C0000103C0000103C0000 101C0000201E0000200E000040070000400380008001C0010000E0020000381C000007E0 001C247DA223>67 DIII72 DI77 DI80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020 E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E0 0001E00000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0 C80180C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008 40078008C007800C80078004800780048007800480078004000780000007800000078000 000780000007800000078000000780000007800000078000000780000007800000078000 000780000007800000078000000780000007800000078000000780000007800000078000 000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700 00070000FF0007C7001E07003C0700780700700700F00708F00708F00708F00F08781708 3C23900FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E 003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F 00700C80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000 700000F00000F00000F00000F00000F00000F00000F000007000007800403800401C0080 0C010007060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800 E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800 E01801E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C078 01E07000E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C 00400E008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700 07000700070007000700FFF8070007000700070007000700070007000700070007000700 07000700070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C 07001C07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010 00001800001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860 00306000303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E0 0F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000 0000000000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E0 01C00000000000000000000000000000000001E00FE001E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0 F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E0200 0E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C0 0E01E00E00F00E00F8FFE3FE17237FA21A>I<0E1FC07F00FE60E183801E807201C00F00 3C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E0FFE3FF8FFE27157F942A>109 D<0E1F80FE60C01E80E00F00 700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E0038 0E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E0038 0F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E0000 0E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E 000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>I119 D121 D E /Fu 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E078 0000E0700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F8 0000007C0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000 FFFF00000FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F800 0000F88000007880000078800000788000007880000078C0000078C0000070E00000F0E0 0000E0F00000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125 >83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E 001E0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F 001E003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478 003E047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E060038018 0700040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000 F80000F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F0002 07000403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E0 000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0 000001E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E0 078005E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0 F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E0 780001E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF 003F81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C 000F003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F8 000000F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E 0000800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007 E0001C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF 007FFF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E 003C001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E 003C000F007800070070000780E00009C1C000087F000018000000180000001800000018 000000180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E030 0000F070000070E0000038E0000038E0000038E0000038E0000038700000707000007038 0000E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F80 0700000000000000000000000000000000000000000000000780FF80FF800F8007800780 078007800780078007800780078007800780078007800780078007800780078007800780 07800780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F0 00FF8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E00 07C000F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E000780 00F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Fv 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 947 a Fv(D)26 b(R)g(A)f(F)h(T)225 1038 y Fu(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n (terface)621 1232 y Ft(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum) 766 1359 y(Septem)o(b)q(er)g(19,)h(1996)190 1417 y(This)h(w)o(ork)f(w)o (as)h(supp)q(orted)g(in)f(part)g(b)o(y)g(NSF)g(and)h(ARP)l(A)e(under)h (NSF)g(con)o(tract)283 1475 y(CD)o(A-9115428)j(and)e(Esprit)f(under)h (pro)s(ject)e(HPC)i(Standards)g(\(21111\).)p eop %%Page: 1 2 1 1 bop 166 45 a Fs(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fr(Chapter)34 b(10)75 564 y Fq(Miscellan)m(y)75 786 y Fs(There)12 b(are)g(a)g(n)o(um)o(b)q(er)g(of)g(topics)g(that)f (do)h(not)g(\014t)g(con)o(v)o(enien)o(tly)h(in)o(to)f(the)g(other)g(c)o (hapters.)19 b(W)l(e)12 b(collect)75 843 y(them)j(here.)75 986 y Fp(10.1)59 b(P)n(o)n(rtable)20 b(MPI)g(Pro)r(cess)f(Sta)n(rtup)75 1135 y Fo(Curren)o(t)14 b(Status:)j Fn(P)o(assed)e(once.)166 1239 y Fs(A)f(n)o(um)o(b)q(er)g(of)f(implemen)o(tations)i(of)e(MPI-1)g (pro)o(vide)i(a)e(startup)g(command)g(for)h(MPI)f(programs)75 1295 y(that)h(is)i(of)f(the)g(form)170 1389 y Fm(mpirun)23 b()g()g()75 1483 y Fs(Separating)16 b(the)g(command)g(to)g(start)f(the)h(program)f (from)g(the)h(program)f(itself)i(pro)o(vides)g(\015exibilit)o(y)l(,)75 1539 y(particularly)23 b(for)e(net)o(w)o(ork)g(and)h(heterogeneous)g (implemen)o(tations.)41 b(F)l(or)21 b(example,)j(the)e(startup)75 1596 y(script)16 b(need)g(not)f(run)g(on)g(one)g(of)g(the)g(mac)o (hines)h(that)f(will)i(b)q(e)f(executing)g(the)f(MPI)g(program)f (itself.)166 1652 y(Ha)o(ving)i(a)f(standard)h(startup)f(mec)o(hanism)i (also)e(extends)i(the)f(p)q(ortabilit)o(y)h(of)e(F)l(ortran,)f(C,)i (and)75 1708 y(C++)f(programs)f(one)g(step)h(further,)f(to)g(the)h (command)g(lines)h(and)f(scripts)g(that)f(manage)g(them.)20 b(F)l(or)75 1765 y(example,)c(a)f(v)m(alidation)i(suite)f(script)g (that)e(runs)i(h)o(undreds)g(of)f(programs)f(can)i(b)q(e)g(a)f(p)q (ortable)h(script)75 1821 y(if)f(it)f(is)g(written)g(using)h Fm(mpirun)p Fs(.)k(In)c(order)e(that)h(the)g(\\statndard")f(command)h (not)f(b)q(e)i(confused)g(with)p Fl(>)h Fk(\(Oct\))75 1878 y Fs(existing)h(practice,)f(whic)o(h)h(is)f(not)g(standard)f(and)h (not)g(p)q(ortable)g(among)f(implemen)o(tations,)i(instead)75 1934 y(of)e Fm(mpirun)f Fs(w)o(e)h(sp)q(ecify)i Fm(mpiexec)p Fs(.)1191 b Fl(?)16 b Fk(\(Oct\))166 1991 y Fs(It)f(is)h(prop)q(osed)f (that)170 2085 y Fm(mpiexec)23 b(-n)h()e()75 2178 y Fs(b)q(e)c(at)f(least)g(one)g(w)o(a)o(y)g(to)f(start)g Fm()g Fs(with)i(an)f(initial)j Fj(MPI)p 1275 2178 14 2 v 15 w(COMM)p 1432 2178 V 17 w(W)o(ORLD)d Fs(whose)g(group)75 2235 y(con)o(tains)22 b Fm()f Fs(pro)q(cesses.)42 b(Other)22 b(argumen)o(ts)g(to)g Fm(mpiexec)f Fs(ma)o(y)h(b)q(e)h (implemen)o(tation-)75 2291 y(dep)q(enden)o(t.)166 2348 y(Curren)o(tly)14 b(this)g(is)g(prop)q(osed)g(as)f(advice)h(to)f (implemen)o(tors,)i(rather)e(than)g(as)g(a)h(required)g(part)f(of)75 2404 y(MPI-2.)20 b(It)15 b(is)h(not)e(suggested)i(that)e(this)i(b)q(e)g (the)f(only)h(w)o(a)o(y)e(to)g(start)g(MPI)h(programs.)189 2510 y Fi(A)n(dvic)n(e)c(to)j(implementors.)37 b Fs(Implemen)o(tors,)12 b(if)g(they)g(do)f(pro)o(vide)h(a)f(sp)q(ecial)i(startup)e(command)189 2567 y(for)j(MPI)i(programs,)e(are)h(advised)h(to)f(giv)o(e)g(it)h(the) g(follo)o(wing)g(form.)j(The)d(syn)o(tax)f(is)g(c)o(hosen)h(in)189 2623 y(order)g(that)g Fm(mpiexec)g Fs(b)q(e)i(able)f(to)f(b)q(e)i(view) o(ed)f(as)g(a)f(command-line)j(v)o(ersion)e(of)f Fj(MPI)p 1711 2623 V 16 w(SP)l(A)-5 b(WN)189 2680 y Fs(\(See)15 b(Section)h Fh(??)p Fs(\).)964 2828 y(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fs(2)1133 b Fg(CHAPTER)15 b(10.)35 b(MISCELLANY)284 45 y Fm(mpiexec)23 b(-n)95 b()451 102 y(-soft)47 b(<)191 b(>)451 158 y(-host)47 b(<)191 b(>)451 214 y(-arch)47 b(<)191 b(>)451 271 y(-wdir)47 b(<)191 b(>)451 327 y(-path)47 b(<)191 b(>)451 384 y(-file)47 b(<)191 b(>)475 440 y(...)451 497 y()189 585 y Fs(for)9 b(the)i(case)f(where)g(a)g(single)i(command)e(line)i (for)e(the)g(application)i(program)d(and)h(its)h(argumen)o(ts)189 641 y(will)23 b(su\016ce.)41 b(F)l(or)22 b(the)g(case)g(corresp)q (onding)h(to)f Fj(MPI)p 1180 641 14 2 v 16 w(SP)l(A)-5 b(WN)p 1346 641 V 17 w(MUL)l(TIPLE)21 b Fs(there)h(are)g(t)o(w)o(o)189 698 y(p)q(ossible)17 b(formats:)189 769 y(F)l(orm)d(A:)284 861 y Fm(mpiexec)23 b({)h()f(})i(:)g({)f(...)h(})g (:)f({)h(...)g(})f(:)h(...)f(:)h({)g(...)f(})189 953 y Fs(As)17 b(with)g Fj(MPI)p 448 953 V 16 w(SP)l(A)-5 b(WN)p Fs(,)18 b(all)g(the)f(argumen)o(ts)f(are)h(optional.)26 b(\(Ev)o(en)17 b(the)h Fm(-n)23 b(x)41 b Fs(argumen)o(t)16 b(is)189 1009 y(optional;)h(the)f(default)h(is)g(implemen)o(tation)h (dep)q(enden)o(t.)25 b(It)17 b(migh)o(t)f(b)q(e)h Fm(1)p Fs(,)f(it)h(migh)o(t)f(b)q(e)h(tak)o(en)189 1066 y(from)11 b(an)i(en)o(vironmen)o(t)g(v)m(ariable,)h(or)e(it)g(migh)o(t)h(b)q(e)g (sp)q(eci\014ed)h(at)e(compile)i(time\).)19 b(The)13 b(names)f(of)189 1122 y(the)k(argumen)o(ts)e(are)i(tak)o(en)f(from)g (the)h(k)o(eys)g(in)g(the)g Fm(info)f Fs(argumen)o(t)g(to)h Fj(MPI)p 1567 1122 V 15 w(SP)l(A)-5 b(WN)p Fs(.)17 b(There)189 1179 y(ma)o(y)d(b)q(e)i(other,)e(implemen)o(tation-dep)q(end)q(en)o(t)k (argumen)o(ts)c(as)h(w)o(ell.)189 1250 y(Note)k(that)g(F)l(orm)f(A,)h (though)h(con)o(v)o(enien)o(t)g(to)f(t)o(yp)q(e,)h(prev)o(en)o (tscolons)g(from)e(b)q(eing)j(program)189 1306 y(argumen)o(ts.)e (Therefore)c(and)g(alternate,)g(\014le-based)i(form)d(is)i(allo)o(w)o (ed:)189 1377 y(F)l(orm)e(B:)284 1470 y Fm(mpiexec)23 b(-runfile)g()189 1562 y Fs(where)15 b(the)g(lines)i(of)e Ff(<)p Fs(\014lename)p Ff(>)i Fs(are)e(of)g(the)g(form)f(separated)h(b) o(y)g(the)h(colons)f(in)h(F)l(orm)f(A.)189 1633 y(\()p Fi(End)g(of)i(advic)n(e)f(to)g(implementors.)p Fs(\))75 1710 y Fh(Example)i(10.1)23 b Fi(Start)16 b(16)h(instanc)n(es)d(of)i Fm(myprog)g Fi(on)g(the)g(curr)n(ent)h(or)f(default)h(machine:)170 1784 y Fm(mpiexec)23 b(-n)h(16)g(myprog)75 1861 y Fh(Example)18 b(10.2)23 b Fi(Start)16 b(10)h(pr)n(o)n(c)n(esses)d(on)i(the)h(machine) f(c)n(al)r(le)n(d)f Fm(ferrari)p Fi(:)170 1935 y Fm(mpiexec)23 b(-n)h(10)g(-host)f(ferrari)g(myprog)75 2012 y Fh(Example)18 b(10.3)23 b Fi(Start)11 b(the)h Fm(ocean)f Fi(pr)n(o)n(gr)n(am)h(on)f (\014ve)g(suns)g(and)h(the)f Fm(atmos)g Fi(pr)n(o)n(gr)n(am)h(on)g(10)g (RS/6000's:)170 2086 y Fm(mpiexec)23 b(-n)h(5)g(-arch)f(sun)g(ocean)g (:)h(-n)g(10)f(-arch)h(rs6000)f(atmos)75 2163 y Fh(Example)18 b(10.4)23 b Fi(Start)17 b(thr)n(e)n(e)h(c)n(opies)f(of)h(the)g(same)f (pr)n(o)n(gr)n(am)h(with)g(di\013er)n(ent)f(c)n(ommand-line)g(ar)n(gu-) 75 2220 y(ments:)170 2293 y Fm(mpiexec)23 b(myprog)g(infile1)g(:)h (myprog)f(infile2)g(:)h(myprog)f(infile3)75 2371 y Fh(Example)18 b(10.5)23 b Fi(Start)11 b(the)h Fm(ocean)f Fi(pr)n(o)n(gr)n(am)h(on)f (\014ve)g(suns)g(and)h(the)f Fm(atmos)g Fi(pr)n(o)n(gr)n(am)h(on)g(10)g (RS/6000's)75 2427 y(\(F)m(orm)k(B\):)170 2500 y Fm(mpiexec)23 b(-runfile)g(myfile)75 2574 y Fi(wher)n(e)16 b Fm(myfile)g Fi(c)n(ontains)170 2647 y Fm(-n)24 b(5)48 b(-arch)23 b(sun)95 b(ocean)170 2704 y(-n)24 b(10)g(-arch)f(rs6000)g(atmos)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fg(10.2.)34 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY) 898 b Fs(3)75 45 y Fp(10.2)59 b(Language)19 b(Interop)r(erabilit)n(y)75 148 y Fe(10.2.1)49 b(Intro)q(duction)75 281 y Fo(Curren)o(t)14 b(Status:)j Fn(P)o(assed)e(once.)166 385 y Fs(It)h(is)h(not)f(uncommon) h(for)e(library)i(dev)o(elop)q(ers)h(to)e(use)g(one)h(language)f(to)g (dev)o(elop)h(an)f(applica-)75 441 y(tions)g(library)g(to)f(b)q(e)h (used)h(b)o(y)e(applications)i(dev)o(elop)q(ers)g(using)f(another)g (language.)21 b(MPI)15 b(curren)o(tly)75 498 y(supp)q(orts)h(ANSI)g(C,) f(ANSI)h(C++,)g(and)g(F)l(ortran)e(77)h(bindings.)23 b(It)15 b(should)i(b)q(e)f(p)q(ossible)i(for)c(applica-)75 554 y(tions)h(in)h(an)o(y)f(of)g(the)g(supp)q(orted)h(languages)f(to)g (call)h(MPI-related)g(functions)g(in)g(another)f(language.)166 611 y(MPI)h(allo)o(ws)h(the)f(dev)o(elopmen)o(t)h(of)f(clien)o(t-serv)o (er)i(co)q(de,)f(with)g(MPI)f(comm)o(unication)h(used)g(b)q(e-)75 667 y(t)o(w)o(een)d(a)h(parallel)h(clien)o(t)g(and)f(a)g(parallel)h (serv)o(er.)k(It)14 b(should)i(b)q(e)g(p)q(ossible)g(to)e(co)q(de)i (the)f(serv)o(er)f(in)i(one)75 724 y(language,)j(and)g(the)f(clien)o (ts)i(in)f(another)f(language.)30 b(T)l(o)18 b(do)h(so,)f(in)o(tercomm) o(unications)i(should)f(b)q(e)75 780 y(p)q(ossible)e(b)q(et)o(w)o(een)e (applications)i(written)e(in)h(an)o(y)f(language.)166 837 y(There)g(are)g(sev)o(eral)h(issues)g(that)e(need)i(to)f(b)q(e)h (addressed,)f(to)f(ac)o(hiev)o(e)i(in)o(terop)q(erabilit)o(y)l(.)75 930 y Fh(Initializat)q(ion)26 b Fs(W)l(e)12 b(need)h(to)f(sp)q(ecify)i (ho)o(w)d(is)i(the)f(MPI)g(en)o(vironmen)o(t)h(initialized)i(for)d(all) h(languages.)75 1024 y Fh(Comm)o(unication)18 b(of)g(MPI)e(opaque)i(ob) s(jects)23 b Fs(W)l(e)13 b(need)h(to)f(sp)q(ecify)i(ho)o(w)e(are)g(MPI) g(ob)s(ject)g(han-)189 1081 y(dles)j(passed)g(b)q(et)o(w)o(een)g (languages.)21 b(W)l(e)16 b(also)f(need)i(to)e(sp)q(ecify)i(what)e (happ)q(ens)h(when)g(an)g(MPI)189 1137 y(ob)s(ject)k(is)i(accessed)g (in)g(one)f(language,)i(to)d(retriev)o(e)i(information)f(set)g(with)h (this)f(ob)s(ject)g(in)189 1194 y(another)14 b(language.)75 1287 y Fh(In)o(terlanguage)k(comm)o(unication)24 b Fs(W)l(e)18 b(need)g(to)e(sp)q(ecify)i(ho)o(w)f(messages)g(sen)o(t)g(in)g(one)h (language)189 1344 y(can)d(b)q(e)h(receiv)o(ed)g(in)g(another)f (language.)166 1438 y(It)f(is)h(highly)h(desirable)g(that)d(the)i (solution)g(for)f(in)o(terlanguage)h(in)o(terop)q(erabilit)o(y)h(b)q(e) f(extendable)75 1494 y(to)g(new)g(languages,)g(should)h(MPI)f(bindings) i(b)q(e)f(de\014ned)h(for)d(suc)o(h)i(languages.)75 1616 y Fe(10.2.2)49 b(Assumptions)1875 1655 y Fl(>)16 b Fk(\(Oct\))75 1702 y Fs(W)l(e)g(assume)g(that)f(con)o(v)o(en)o(tions)h(exist)h(for)e (MPI)h(programs)f(to)g(call)i(functions)g(in)g(MPI)f(programs)e(of)75 1758 y(another)e(language.)19 b(These)13 b(con)o(v)o(en)o(tions)f(sp)q (ecify)i(ho)o(w)e(to)f(link)j(routines)f(in)g(di\013eren)o(t)g (langauges)f(in)o(to)75 1815 y(one)j(program,)e(ho)o(w)g(to)h(call)i (functions)f(in)g(a)f(di\013eren)o(t)h(language,)f(ho)o(w)g(to)g(pass)g (argumen)o(ts)g(b)q(et)o(w)o(een)75 1871 y(languages,)23 b(and)f(the)f(corresp)q(ondence)i(b)q(et)o(w)o(een)f(basic)h(data)d(t)o (yp)q(es)i(of)f(di\013eren)o(t)h(languages.)39 b(In)75 1928 y(general,)18 b(these)f(con)o(v)o(en)o(tions)g(will)i(b)q(e)e (implemen)o(tation)i(dep)q(enden)o(t)f(\(ho)o(w)o(ev)o(er,)e(this)h(is) h(an)f(ongoing)75 1984 y(e\013ort)11 b(to)g(standardize)i(a)e(C)h (calling)i(in)o(terface)e(in)g(F)l(ortran\).)18 b(F)l(urthernote,)12 b(not)f(ev)o(ery)h(basic)g(datat)o(yp)q(e)75 2040 y(ma)o(y)h(ha)o(v)o (e)h(a)g(matc)o(hing)g(t)o(yp)q(e)g(in)h(other)e(languages.)20 b(F)l(or)14 b(example,)g(C/C++)g(c)o(haracter)g(strings)g(ma)o(y)75 2097 y(not)i(b)q(e)h(compatible)g(with)g(F)l(ortran)e Fm(CHARACTER)f Fs(v)m(ariables.)25 b(Ho)o(w)o(ev)o(er,)15 b(w)o(e)h(assume)g(that)f(a)h(F)l(ortran)75 2153 y Fm(INTEGER)c Fs(can)g(b)q(e)h(passed)g(to)f(a)g(C)h(or)f(C++)h(program.)18 b(W)l(e)12 b(also)h(assume)f(that)g(F)l(ortran,)g(C,)g(and)g(C++)75 2210 y(ha)o(v)o(e)j(address)g(sized)h(in)o(tegers.)1256 b Fl(?)16 b Fk(\(Oct\))75 2332 y Fe(10.2.3)49 b(Initializatio)q(n)75 2465 y Fo(Curren)o(t)14 b(Status:)j Fn(P)o(assed)e(once.)166 2568 y Fs(A)k(call)h(to)f Fm(MPI)p 441 2568 15 2 v 16 w(INIT\(\))p Fs(,)g(from)f(either)i(language,)g(initializes)i(MPI)d (for)f(execution)i(in)g(all)g(lan-)75 2625 y(guages.)1655 b Fl(>)16 b Fk(\(Oct\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fs(4)1138 b Fg(CHAPTER)15 b(10.)30 b(MISCELLANY)189 45 y Fi(A)n(dvic)n(e)17 b(to)i(users.)56 b Fs(Certain)17 b(implemen)o(tations)i(use)g(the)e(\(inout\))h(argc,)g (argv)e(argumen)o(ts)h(of)189 102 y(the)f(C/C++)g(v)o(ersion)g(of)g Fj(MPI)p 736 102 14 2 v 16 w(INIT)f Fs(in)i(order)f(to)f(propagate)h (the)g(righ)o(t)g(v)m(alue)h(for)e(argc,)h(argv)189 158 y(to)g(all)i(executing)h(pro)q(cesses.)26 b(Use)18 b(of)f(the)g(F)l (ortran)f(v)o(ersion)i(of)f Fj(MPI)p 1424 158 V 15 w(INIT)g Fs(to)g(initialize)j(MPI)189 214 y(ma)o(y)14 b(result)i(in)g(a)f(loss)g (of)g(this)g(abilit)o(y)l(.)22 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g (users.)p Fs(\))-117 276 y Fl(?)f Fk(\(Oct\))166 321 y Fs(The)g(function)h Fm(MPI)p 512 321 15 2 v 17 w(INITIALIZED)e Fs(returns)h(the)g(same)g(answ)o(er)g(in)h(an)o(y)f(language.)166 377 y(The)g(function)h Fm(MPI)p 512 377 V 17 w(FINALIZE)e Fs(\014nalizes)j(the)e(MPI)h(en)o(vironmen)o(ts)f(for)g(all)h (languages.)166 434 y(The)h(function)g Fm(MPI)p 515 434 V 17 w(ABORT)e Fs(kill)k(pro)q(cesses,)e(irresp)q(ectiv)o(e)h(of)e(the) g(language)h(used)g(b)o(y)f(the)h(caller)75 490 y(or)e(b)o(y)g(the)g (pro)q(cesses)h(killed.)166 547 y(The)e(MPI)g(en)o(vironmen)o(t)g(is)h (initialized)i(in)e(the)f(same)f(manner)h(for)f(all)i(languages)f(b)o (y)g Fj(MPI)p 1761 547 14 2 v 16 w(INIT)p Fs(.)75 603 y(I.e.,)d Fd(MPI)p 238 603 13 2 v 14 w(COMM)p 382 603 V 15 w(W)o(ORLD)f Fs(carries)g(the)g(same)g(information)g(regardless)h (of)f(language:)17 b(same)10 b(pro)q(cesses,)75 659 y(same)15 b(en)o(vironmen)o(tal)h(attributes,)e(same)h(error)g(handlers.)189 766 y Fi(A)n(dvic)n(e)h(to)h(users.)44 b Fs(The)16 b(use)g(of)g(sev)o (eral)g(languages)g(in)g(one)g(MPI)g(program)f(ma)o(y)g(require)i(the) 189 822 y(use)e(of)g(sp)q(ecial)i(options)e(at)g(compile)h(and/or)f (link)i(time.)j(\()p Fi(End)c(of)g(advic)n(e)g(to)h(users.)p Fs(\))189 928 y Fi(A)n(dvic)n(e)k(to)h(implementors.)79 b Fs(Implemen)o(tations)23 b(ma)o(y)e(selectiv)o(ely)i(link)g(language) f(sp)q(eci\014c)189 985 y(MPI)15 b(libraries)i(only)f(to)f(co)q(des)h (that)f(need)h(them,)g(so)f(as)g(not)g(to)g(increase)h(the)g(size)g(of) g(binaries)189 1041 y(for)21 b(co)q(des)i(that)e(use)i(only)f(one)g (language.)41 b(The)22 b(MPI)g(initializati)q(on)j(co)q(de)d(need)h(p)q (erform)189 1098 y(initialization)18 b(for)c(a)h(language)g(only)g(if)h (that)e(language)h(library)h(is)g(loaded.)k(\()p Fi(End)c(of)g(advic)n (e)g(to)189 1154 y(implementors.)p Fs(\))75 1276 y Fe(10.2.4)49 b(T)l(ransfer)16 b(of)g(handles)75 1409 y Fo(Curren)o(t)e(Status:)j Fn(P)o(assed)e(once.)-117 1456 y Fl(>)g Fk(\(Oct\))166 1513 y Fs(Handles)e(are)f(passed)h(across)f(languages)g(b)o(y)h(using)g (an)f(explicit)j(C)d(wrapp)q(er)h(to)e(con)o(v)o(ert)h(handles.)75 1569 y(The)j(con)o(v)o(ersion)h(is)f(automatic)g(b)q(et)o(w)o(een)h(C)f (and)g(C++.)166 1626 y(The)d(data)f(t)o(yp)q(e)h Fd(MPI)p 532 1626 V 14 w(Fint)g Fs(is)g(pro)o(vided)g(in)h(C)e(for)g(an)h(in)o (teger)g(of)f(the)h(size)h(that)d(matc)o(hes)i(a)f(F)l(ortran)75 1682 y Fd(INTEGER)p Fs(;)j(usually)l(,)i Fd(MPI)p 517 1682 V 15 w(Fint)f Fs(will)i(b)q(e)f(equiv)m(alen)o(t)h(to)d Fd(int)p Fs(.)166 1739 y(The)f(follo)o(wing)h(t)o(w)o(o)e(functions)i (are)f(pro)o(vided)h(in)g(C)f(to)g(con)o(v)o(ert)f(from)h(a)g(F)l (ortran)f(handle)i(\(whic)o(h)75 1795 y(is)i(an)f(in)o(teger\))g(to)f (a)h(C)g(handle,)h(and)g(vice)g(v)o(ersa.)75 1851 y Fm(MPI)p 150 1851 15 2 v 17 w(Handle)23 b(MPI)p 406 1851 V 17 w(Int2handle\()f(MPI)p 781 1851 V 17 w(Fint)h(f)p 941 1851 V 17 w(handle,)g(MPI)p 1221 1851 V 17 w(Handle)p 1382 1851 V 16 w(type)g(handle)p 1661 1851 V 17 w(kind\))75 1938 y(MPI)p 150 1938 V 17 w(Fint)g(MPI)p 358 1938 V 17 w(Handle2int\()f(MPI)p 733 1938 V 17 w(Handle)h(c)p 941 1938 V 17 w(handle,)g(MPI)p 1221 1938 V 17 w(Handle)p 1382 1938 V 16 w(type)g(handle)p 1661 1938 V 17 w(kind\))166 2024 y Fs(If)f Fj(f)p 235 2024 14 2 v 16 w(handle)g Fs(is)g(a)f(v)m (alid)i(F)l(ortran)e(handle)h(to)f(an)g(opaque)h(ob)s(ject,)g(and)f Fj(handle)p 1598 2024 V 18 w(kind)h Fs(sp)q(eci\014es)75 2081 y(the)17 b(t)o(yp)q(e)f(of)h(ob)s(ject,)f(then)h Fj(MPI)p 654 2081 V 16 w(INT2HANDLE)f Fs(returns)h(a)f(v)m(alid)i(C)f (handle)h(to)e(that)g(same)g(ob)s(ject;)75 2137 y(if)21 b Fj(f)p 139 2137 V 16 w(handle)h Fs(is)f(a)g(n)o(ull)h(F)l(ortran)d (handle,)k(then)e Fj(MPI)p 1016 2137 V 16 w(INT2HANDLE)f Fs(returns)h(a)f(n)o(ull)i(C)e(handle;)k(if)75 2194 y Fj(f)p 92 2194 V 16 w(handle)17 b Fs(is)e(an)g(in)o(v)m(alid)j(F)l (ortran)c(handle,)i(then)f Fj(MPI)p 1011 2194 V 16 w(INT2HANDLE)g Fs(returns)g(an)g(in)o(v)m(alid)i(C)e(handle;)75 2250 y(and)g(similarly)l(,)i(for)e Fj(MPI)p 513 2250 V 16 w(HANDLE2INT)p Fs(.)75 2356 y Fh(Example)j(10.6)23 b Fs(The)10 b(example)h(b)q(elo)o(w)g(illustrates)g(ho)o(w)e(the)i(F)l (ortran)e(MPI)h(function)h Fj(MPI)p 1677 2356 V 16 w(TYPE)p 1810 2356 V 16 w(COMMIT)75 2413 y Fs(can)k(b)q(e)h(implemen)o(ted)h(b)o (y)d(wrapping)i(the)f(C)g(MPI)g(function)h Fj(MPI)p 1244 2413 V 15 w(T)l(yp)q(e)p 1351 2413 V 18 w(commit)d Fs(with)i(a)g(C)g (wrapp)q(er)75 2469 y(to)g(do)g(handle)h(con)o(v)o(ersions.)75 2576 y Fm(!)24 b(FORTRAN)f(PROCEDURE)75 2632 y(SUBROUTINE)f (MPI_TYPE_COMMIT\()g(DATATYPE,)h(IERR\))75 2689 y(INTEGER)g(DATATYPE,)g (IERR)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fg(10.2.)34 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY) 898 b Fs(5)75 45 y Fm(CALL)23 b(MPI_X_TYPE_COMMIT\(DATATYPE,)d(IERR\)) 75 102 y(RETURN)75 158 y(END)75 271 y(!)k(C)f(wrapper)75 384 y(void)g(MPI_X_TYPE_COMMIT\()f(MPI_Fint)g(*f_handle,)h(*ierr\))75 440 y({)75 497 y(MPI_Datatype)f(datatype;)75 610 y(datatype)h(=)g (\(MPI_Datatype\)MPI_int2handle\()d(*f_handle,)j (MPI_DATATYPE_HANDLE\);)75 666 y(*ierr)g(=)h (\(MPI_Fint\)MPI_Type_commit)o(\()d(&datatype\);)75 723 y(*f_handle)i(=)g(MPI_handle2int\()f(\(MPI_Handle\)datatype,)75 779 y(MPI_DATATYPE_HANDLE\);)75 835 y(return\(\);)75 892 y(})166 994 y Fs(The)10 b(same)g(approac)o(h)g(can)g(b)q(e)h(used)g (for)e(all)i(other)f(MPI)g(functions.)19 b(The)11 b(call)g(to)e Fj(MPI)p 1637 994 14 2 v 16 w(INT2HANDLE)75 1050 y Fs(\(resp.)k Fj(MPI)p 285 1050 V 16 w(HANDLE2INT)p Fs(\))g(can)i(b)q(e)f(omitted)g (when)g(the)g(handle)i(is)e(an)g Fn(OUT)g Fs(\(resp.)f Fn(IN)p Fs(\))h(argumen)o(t,)75 1107 y(rather)h(than)g Fn(INOUT)p Fs(.)166 1256 y Fo(Discussion:)g Fn(Do)f(w)o(e)g(also)f(w)o (an)o(t)h(C++)g(bindings)f(for)h(these)h(functions?)189 1406 y Fi(R)n(ationale.)62 b Fs(The)19 b(design)h(here)g(pro)o(vides)f (a)g(con)o(v)o(enien)o(t)g(solution)h(for)e(the)h(prev)m(alen)o(t)h (case,)189 1462 y(where)f(a)f(C)h(wrapp)q(er)g(is)h(used)f(to)f(allo)o (w)i(F)l(ortran)d(co)q(de)j(to)e(call)i(a)f(C)f(library)l(,)j(or)d(C)h (co)q(de)h(to)189 1518 y(call)g(a)f(F)l(ortran)f(library)l(.)32 b(The)19 b(use)h(of)f(C)f(wrapp)q(ers)i(is)f(m)o(uc)o(h)g(more)g(lik)o (ely)i(than)e(the)g(use)g(of)189 1575 y(F)l(ortran)12 b(wrapp)q(ers,)i(b)q(ecause)h(it)f(is)g(m)o(uc)o(h)g(more)f(lik)o(ely)j (that)d(a)g(v)m(ariable)j(of)d(t)o(yp)q(e)h Fd(INTEGER)f Fs(can)189 1631 y(b)q(e)j(passed)f(to)g(C,)f(than)h(a)g(C)g(handle)h (can)g(b)q(e)g(passed)f(to)g(F)l(ortran.)189 1706 y(The)21 b(use)h(of)f(con)o(v)o(ersion)g(functions,)j(rather)c(than)i(pro)q (cedures,)h(allo)o(w)f(us)f(to)g(use)g(inlinin)q(g,)189 1762 y(esp)q(ecially)c(in)f(the)f(case)g(where)g(these)g(functions)h (are)e(the)h(iden)o(tit)o(y)l(.)21 b(The)15 b(con)o(v)o(ersion)g (function)189 1819 y(in)f(the)g(wrapp)q(er)g(do)q(es)g(not)f(catc)o(h)h (an)f(in)o(v)m(alid)j(handle)f(argumen)o(t.)k(Instead,)14 b(an)g(in)o(v)m(alid)i(handle)189 1875 y(is)11 b(passed)h(b)q(elo)o(w)g (to)f(the)g(library)h(function,)h(whic)o(h,)f(presumably)l(,)h(c)o(hec) o(ks)e(its)h(input)g(argumen)o(ts.)189 1932 y(\()p Fi(End)j(of)i(r)n (ationale.)p Fs(\))1875 1989 y Fl(>)f Fk(\(Oct\))189 2079 y Fi(A)n(dvic)n(e)d(to)h(users.)38 b Fs(The)13 b(user)g(needs)g (to)g(co)q(erce)g(the)g(result)g(returned)g(b)o(y)f Fj(MPI)p 1560 2079 V 16 w(int2handle)j Fs(from)189 2136 y Fd(MPI)p 266 2136 13 2 v 14 w(Handle)i Fs(to)d(a)h(sp)q(eci\014c)i(handle)g(t)o (yp)q(e.)j(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fs(\))1875 2193 y Fl(?)g Fk(\(Oct\))75 2301 y Fj(C)f(and)g(C++.)46 b Fs(The)15 b(C++)h(language)f(in)o(terface)g(pro)o(vides)g(the)g (functions)g(listed)h(b)q(elo)o(w)g(for)e(C/C++)75 2358 y(in)o(terop)q(erabilit)o(y)k(\(in)e(particular,)h(to)e(allo)o(w)i(syn) o(tactically)g(pleasing)g(calls)g(of)f(C)g(from)f(C++)i(and/or)75 2414 y(to)h(build)i(C/F)l(ortran)d(in)o(terfaces)i(for)f(C++\).)30 b(The)19 b(tok)o(en)f Fm(CLASS)g Fs(is)h(used)g(b)q(elo)o(w)g(to)f (indicate)i(an)o(y)75 2470 y(v)m(alid)d Fj(MPI)d Fs(opaque)i(handle)g (name)f(\(e.g.)f Fm(Comm)p Fs(\).)166 2527 y(The)f(follo)o(wing)g Fm(operator=\(\))e Fs(function)i(allo)o(ws)g(assignmen)o(t)f(from)g(a)g (C)h Fj(MPI)f Fs(handle)h(to)f(a)g(C++)75 2583 y Fj(MPI)j Fs(handle.)75 2663 y Fm(MPI::CLASS&)22 b(MPI::CLASS::operator=\(const)f (MPI)p 1105 2663 15 2 v 16 w(CLASS&)i(data\))1875 2704 y Fl(>)16 b Fk(\(Oct\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fs(6)1138 b Fg(CHAPTER)15 b(10.)30 b(MISCELLANY)166 45 y Fs(The)17 b(constructor)f(b)q(elo)o(w)h(creates)g (a)f(C++)i Fj(MPI)e Fs(ob)s(ject)g(from)g(a)h(C)f Fj(MPI)g Fs(handle.)26 b(This)18 b(allo)o(ws)75 102 y(the)d(automatic)g (promotion)g(of)g(a)f(C)h Fj(MPI)g Fs(handle)h(to)f(a)g(C++)h Fj(MPI)e Fs(handle.)75 176 y Fm(MPI::CLASS\(const)22 b(MPI)p 556 176 15 2 v 16 w(CLASS&)h(data\))75 290 y Fh(Example)18 b(10.7)23 b Fs(In)d(order)f(for)h(a)f(C)h(program)e(to)h (use)i(a)e(C++)h(library)l(,)i(the)e(C++)g(library)h(m)o(ust)75 347 y(exp)q(ort)16 b(a)h(C)f(in)o(terface)h(that)f(pro)o(vides)h (appropriate)g(con)o(v)o(ersions)f(b)q(efore)h(in)o(v)o(oking)g(the)g (underlying)75 403 y(C++)g(library)h(call.)26 b(This)17 b(example)h(sho)o(ws)e(a)g(C)h(in)o(terface)g(function)h(that)e(in)o(v) o(ok)o(es)g(a)h(C++)g(library)75 459 y(call)c(with)g(a)f(C)g(comm)o (unicator;)g(the)h(comm)o(unicator)e(is)i(automatically)g(promoted)f (to)f(a)h(C++)h(handle)75 516 y(when)j(the)f(underlying)i(C++)f (function)g(is)g(in)o(v)o(ok)o(ed.)75 600 y Fm(void)23 b(cpp_lib_call\(MPI::Comm)e(cpp_comm\);)75 713 y(extern)i("C")g({)75 769 y(void)g(c_interface\(MPI_Comm)e(c_comm\);)75 825 y(})75 938 y(void)i(c_interface\(MPI_Comm)e(c_comm\))75 995 y({)75 1051 y(cpp_lib_call\(c_comm\);)75 1108 y(})166 1191 y Fs(The)15 b(follo)o(wing)g(function)h(allo)o(ws)f(con)o(v)o (ersion)g(from)f(C++)h(ob)s(jects)f(to)g(C)h Fj(MPI)f Fs(handles.)21 b(In)16 b(this)75 1248 y(case,)f(the)g(casting)g(op)q (erator)g(is)g(o)o(v)o(erloaded)h(to)e(pro)o(vide)i(the)f(functionalit) o(y)l(.)75 1323 y Fm(MPI::CLASS::operator)21 b(MPI)p 651 1323 V 17 w(CLASS\(\))i(const)75 1436 y Fh(Example)18 b(10.8)23 b Fs(A)15 b(C)f(library)i(routine)g(is)f(called)i(from)d(a)h (C++)g(program.)k(The)c(C)g(library)g(routine)75 1493 y(is)h(protot)o(yp)q(ed)e(to)h(tak)o(e)f(an)i Fm(MPI)p 647 1493 V 16 w(Comm)f Fs(as)g(an)g(argumen)o(t.)75 1577 y Fm(extern)23 b("C")g({)75 1633 y(void)g(c_lib_call\(MPI_Comm)f (c_comm\);)75 1689 y(})75 1802 y(void)h(cpp_function\(\))75 1859 y({)75 1915 y(MPI::Comm)g(cpp_comm;)75 1972 y (cpp_comm.Dup\(MPI::COMM_WOR)o(LD\);)75 2028 y(c_lib_call\(cpp_comm\);) 75 2085 y(})189 2168 y Fi(R)n(ationale.)60 b Fs(Pro)o(viding)19 b(con)o(v)o(ersion)f(from)g(C)h(to)e(C++)i(via)g(constructors)f(and)h (from)e(C++)189 2225 y(to)e(C)h(via)g(casting)g(allo)o(ws)h(the)f (compiler)h(to)f(mak)o(e)f(automatic)h(con)o(v)o(ersions.)22 b(Calling)c(C)e(from)189 2281 y(C++)k(b)q(ecomes)g(trivial,)h(as)f(do)q (es)g(the)f(pro)o(vision)i(of)e(a)g(C)h(or)f(F)l(ortran)f(in)o(terface) i(to)f(a)g(C++)189 2338 y(library)l(.)i(\()p Fi(End)15 b(of)h(r)n(ationale.)p Fs(\))189 2421 y Fi(A)n(dvic)n(e)f(to)i (implementors.)43 b Fs(While)17 b(the)f(pro)o(vided)g(casting)g(and)g (promotion)f(functions)i(allo)o(w)189 2478 y(the)11 b(seamless)g(con)o (v)o(ersions)g(b)q(et)o(w)o(een)g(C)f(and)h(C++)h Fj(MPI)e Fs(handles,)i(there)f(are)g(sev)o(eral)g(situations)189 2534 y(in)16 b(the)f Fj(MPI)g Fs(C)g(bindings)i(\(and)e(third)h(part)o (y)e(C)h(library)h(calls\))g(where)g(a)f Fi(p)n(ointer)g Fs(to)f(an)i Fj(MPI)e Fs(C)189 2591 y(handle)g(is)h(required)f(\(e.g.,) f Fm(MPI)p 742 2591 V 16 w(CANCEL)p Fs(\).)f(Curren)o(tly)l(,)i(a)f (C++)h(user)g(w)o(ould)g(ha)o(v)o(e)f(to)g(explicitly)189 2647 y(cast)k(the)g(C++)h(handle)h(to)e(C)g(v)m(ariable,)i(and)f(then)f (use)h(the)g Fm(&)f Fs(op)q(erator)g(to)g(get)g(its)g(address.)189 2704 y(F)l(or)d(example,)i(the)f(C++)h(syn)o(tax)e(to)h(call)h(the)f(C) g Fm(MPI)p 1143 2704 V 17 w(Cancel\(\))f Fs(function)i(is)g(as)f(follo) o(ws:)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fg(10.2.)29 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY) 903 b Fs(7)189 45 y Fm(MPI::Request)22 b(cpp_request;)189 102 y(MPI_Request)g(c_request;)189 158 y(...)189 214 y(c_request)g(=)i(cpp_request;)189 271 y(MPI_Cancel\(&c_request\);)189 396 y Fs(A)12 b(cleaner)h(syn)o(tax)e(can)i(b)q(e)g(supp)q(orted)f (with)h(the)f(follo)o(wing)h(functions:)19 b(a)12 b(promotion)g(op)q (erator)189 452 y(from)i(a)h(C)g Fj(MPI)g Fs(handle)h(p)q(oin)o(ter,)f (and)h(a)f(casting)g(op)q(erator)f(to)h(a)g(C)g Fj(MPI)f Fs(handle)j(p)q(oin)o(ter.)189 527 y Fm(MPI::CLASS\(const)k(MPI)p 669 527 15 2 v 17 w(CLASS*)i(data\))189 633 y(MPI::CLASS::operator)e (MPI)p 765 633 V 17 w(CLASS*\(\))189 738 y Fs(If)g(these)g(functions)h (are)e(supp)q(orted)i(b)o(y)f(the)g(implemen)o(tation,)i(a)e(C++)g (program)f(can)h(call)189 794 y Fm(MPI)p 264 794 V 16 w(Cancel\(\))14 b Fs(as)h(follo)o(ws:)189 919 y Fm(MPI::Request)22 b(cpp_request;)189 976 y(...)189 1032 y(MPI_Cancel\(cpp_request\))o(;) 189 1157 y Fs(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fs(\))1875 1218 y Fl(?)g Fk(\(Oct\))75 1329 y Fe(10.2.5)49 b(MPI)17 b(Opaque)g(objects)75 1462 y Fo(Curren)o(t)d(Status:)j Fn(P)o(assed)e(once.)166 1565 y Fs(In)h(general,)f(opaque)g(ob)s(jects) f(are)h(\\the)g(same")f(in)i(all)g(languages:)k(they)15 b(carry)g(the)g(same)g(infor-)75 1622 y(mation,)j(and)f(ha)o(v)o(e)h (the)f(same)g(meaning)i(in)f(b)q(oth)g(languages.)27 b(The)18 b(mec)o(hanism)g(describ)q(ed)h(in)g(the)75 1678 y(previous)d(section)g(can)f(b)q(e)h(used)g(to)e(pass)h (references)h(to)f(MPI)g(ob)s(jects)g(from)f(language)i(to)e(language.) 75 1735 y(An)h(ob)s(ject)g(created)g(in)h(one)g(language)f(can)g(b)q(e) h(accessed,)g(mo)q(di\014ed)g(or)f(freed)h(in)g(another)e(language.)166 1791 y(W)l(e)h(examine)h(b)q(elo)o(w)g(in)g(more)f(detail,)h(issues)g (that)e(arise)i(for)e(eac)o(h)h(t)o(yp)q(e)h(of)f(MPI)g(ob)s(ject.)75 1913 y Fe(10.2.6)49 b(Datat)o(yp)q(es)75 2046 y Fo(Curren)o(t)14 b(Status:)j Fn(P)o(assed)e(once.)166 2150 y Fs(Datat)o(yp)q(es)d(enco)q (de)j(the)f(same)f(information)h(in)h(all)f(languages.)20 b(E.g.,)12 b(a)h(datat)o(yp)q(e)g(accessor)h(lik)o(e)75 2206 y Fj(MPI)p 160 2206 14 2 v 16 w(TYPE)p 293 2206 V 17 w(EXTENT)h Fs(will)h(return)f(the)g(same)f(information)h(in)h(all) f(three)g(languages.)20 b(If)15 b(a)f(datat)o(yp)q(e)75 2262 y(de\014ned)22 b(in)f(one)f(language)h(is)f(used)h(for)f(a)g(comm) o(unication)h(call)g(in)g(another)f(language,)h(then)g(the)75 2319 y(message)c(sen)o(t)h(will)h(b)q(e)f(iden)o(tical)i(to)d(the)g (message)h(that)f(w)o(ould)h(b)q(e)g(sen)o(t)f(from)g(the)h(\014rst)f (language:)75 2375 y(the)e(same)g(comm)o(unication)h(bu\013er)f(is)g (accessed,)g(and)h(the)f(same)g(represen)o(tation)g(con)o(v)o(ersion)g (is)g(p)q(er-)75 2432 y(formed,)c(if)h(needed.)20 b(All)13 b(basic)f(datat)o(yp)q(es)f(can)g(b)q(e)h(used)g(in)h(datat)o(yp)q(e)d (constructors)h(in)h(an)o(y)f(language.)75 2488 y(If)k(a)g(datat)o(yp)q (e)g(is)h(committed,)f(it)g(can)g(b)q(e)h(used)g(for)f(comm)o (unication)h(in)g(an)o(y)e(languages.)166 2545 y(The)19 b(function)h Fj(MPI)p 530 2545 V 16 w(ADDRESS)g Fs(returns)f(the)h (same)f(v)m(alue)h(in)g(all)g(languages.)32 b(On)20 b(the)f(other)75 2601 y(hand,)h(w)o(e)e(do)h(not)g(require)g(that)f(the)h(constan)o(t)f Fd(MPI)p 1032 2601 13 2 v 15 w(BOTTOM)g Fs(has)h(the)g(same)f(v)m(alue) i(in)g(all)g(three)75 2658 y(languages.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fs(8)1133 b Fg(CHAPTER)15 b(10.)35 b(MISCELLANY)75 45 y Fh(Example)18 b(10.9)75 134 y Fm(!)24 b(FORTRAN)f(CODE)75 190 y(REAL)g(R\(5\))75 247 y(INTEGER)g(TYPE,)g (ADDR,)g(IERR)75 360 y(!)h(create)f(an)g(absolute)g(datatype)g(for)g (array)h(R)75 416 y(CALL)f(MPI_ADDRESS\()g(R,)g(ADDR,)g(IERR\))75 473 y(CALL)g(MPI_TYPE_STRUCT\(1,)f(5,)h(ADDR,)g(MPI_REAL,)g(TYPE,)g (IERR\))75 529 y(CALL)g(C_ROUTINE\(TYPE\))75 642 y(/*)h(C)f(code)h(*/) 75 755 y(void)f(C_ROUTINE\(MPI_Fint)f(*ftype\))75 811 y({)75 868 y(int)h(count)h(=)f(5;)75 924 y(void)g(*handle;)75 981 y(int)g(lens[2])g(=)h({1,1};)75 1037 y(MPI_Aint)f(displs[2];)75 1094 y(MPI_Datatype)f(types[2],)h(newtype;)75 1207 y(/*)h(create)f(an)g (absolute)g(datatype)g(for)g(buffer)g(that)h(consists)70 b(*/)75 1263 y(/*)47 b(of)24 b(count,)f(followed)g(by)g(R\(5\))668 b(*/)75 1376 y(MPI_Addr\(&count,)22 b(displs[0]\);)75 1432 y(displs[1])h(=)g(0;)75 1489 y(types[0])g(=)g(MPI_INT;)75 1545 y(types[1])g(=)g(\(MPI_Datatype\)MPI_Int2handle\(f)o(type\);)75 1602 y(MPI_Type_struct\(2,)e(lens,)j(displs,)e(types,)i(&newtype\);)75 1658 y(MPI_Type_commit\(&newtype\);)75 1771 y(MPI_Send\(MPI_BOTTOM,)d (1,)j(newtype,)e(1,)i(0,)g(MPI_COMM_WORLD\);)75 1828 y(/*)g(the)f(message)g(sent)g(contains)g(an)h(int)f(count)g(of)h(5,)f (followed)47 b(*/)75 1884 y(/*)24 b(by)f(the)h(5)f(REAL)h(entries)f(of) g(the)h(Fortran)f(array)g(R.)238 b(*/)75 1940 y(})189 2040 y Fi(R)n(ationale.)58 b Fs(The)18 b(curren)o(t)g(MPI)g(standard)g (sp)q(eci\014es)i(that)d Fd(MPI)p 1363 2040 13 2 v 15 w(ADDRESS)g Fs(can)h(b)q(e)h(used)g(in)189 2096 y(initialization)i (expressions)f(in)g(C,)e(but)h(not)f(in)i(F)l(ortran.)29 b(Since)21 b(F)l(ortran)c(do)q(es)i(not)g(supp)q(ort)189 2153 y(normally)11 b(call)h(b)o(y)f(v)m(alue,)h(then)f Fd(MPI)p 815 2153 V 15 w(ADDRESS)f Fs(m)o(ust)g(b)q(e)i(in)f(F)l (ortran)f(the)h(name)f(of)h(a)f(prede\014ned)189 2209 y(static)19 b(v)m(ariable,)k(e.g.,)d(a)f(v)m(ariable)j(in)f(an)f(MPI)g (declared)h Fd(COMMON)f Fs(blo)q(c)o(k.)35 b(On)20 b(the)g(other)189 2266 y(hand,)e(in)h(C,)e(it)h(is)g(natural)f(to)g(tak)o(e)g Fd(MPI)p 930 2266 V 15 w(BOTTOM)f(=)g(0)h Fs(\(Ca)o(v)o(eat:)23 b(De\014ning)c Fd(MPI)p 1681 2266 V 14 w(BOTTOM)189 2322 y(=)e(0)h Fs(implies)i(that)e Fd(NULL)i Fs(p)q(oin)o(ter)e(cannot)h(b)q (e)g(distinguished)i(from)d Fd(MPI)p 1502 2322 V 14 w(BOTTOM)p Fs(;)g(ma)o(y)f(b)q(e)189 2378 y Fd(MPI)p 266 2378 V 14 w(BOTTOM)12 b(=)i(1)f Fs(is)i(b)q(etter)f Ff(:)8 b(:)g(:)e Fs(\))13 b(Requiring)j(that)e(the)g(F)l(ortran)f(and)h(C)g(v)m(alues)h (b)q(e)g(the)f(same)189 2435 y(will)i(complicate)h(the)e (initialization)j(pro)q(cess.)i(\()p Fi(End)c(of)g(r)n(ationale.)p Fs(\))189 2534 y Fi(A)n(dvic)n(e)e(to)h(implementors.)39 b Fs(The)14 b(follo)o(wing)g(implemen)o(tation)h(can)f(b)q(e)h(used:)k (MPI)14 b(addresses,)189 2591 y(as)e(returned)h(b)o(y)f Fj(MPI)p 569 2591 14 2 v 16 w(ADDRESS)p Fs(,)h(will)h(ha)o(v)o(e)f(the) f(same)h(v)m(alue)g(in)h(all)f(languages.)19 b(One)14 b(ob)o(vious)189 2647 y(c)o(hoice)e(is)g(that)g(MPI)f(addresses)h(b)q (e)g(iden)o(tical)i(to)d(regular)h(addresses.)19 b(The)12 b(address)g(is)g(stored)f(in)189 2704 y(the)k(datat)o(yp)q(e,)f(when)h (datat)o(yp)q(es)f(with)h(absolute)g(addresses)g(are)g(constructed.)20 b(When)15 b(a)f(send)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fg(10.2.)34 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY) 898 b Fs(9)189 45 y(or)13 b(receiv)o(e)i(op)q(eration)f(is)h(p)q (erformed,)f(then)g(addresses)g(stored)g(in)h(a)e(datat)o(yp)q(e)h(are) f(in)o(terpreted)189 102 y(as)h(displacemen)o(ts)j(that)d(are)g(all)i (augmen)o(ted)f(b)o(y)g(a)f(base)h(address.)20 b(This)c(base)f(address) g(is)g(\(the)189 158 y(address)f(of)t(\))e Fj(buf)p Fs(,)j(or)e(zero,)h (if)h Fj(buf)f(=)h(MPI)p 921 158 14 2 v 15 w(BOTTOM)p Fs(.)g(Th)o(us,)e(if)i Fd(MPI)p 1403 158 13 2 v 14 w(BOTTOM)f Fs(is)g(zero)g(then)g(a)189 214 y(send)g(or)f(receiv)o(e)i(call)g(with) f Fj(buf)g(=)g(MPI)p 885 214 14 2 v 16 w(BOTTOM)h Fs(is)f(implemen)o (ted)h(exactly)g(as)e(a)g(call)i(with)f(a)189 271 y(regular)h(bu\013er) g(argumen)o(t:)k(in)d(b)q(oth)f(cases)g(the)h(base)f(address)g(is)h Fj(buf)p Fs(.)21 b(On)15 b(the)g(other)g(hand,)g(if)189 327 y Fd(MPI)p 266 327 13 2 v 14 w(BOTTOM)e Fs(is)h(not)g(zero,)f(then) h(the)g(implemen)o(tation)h(has)e(to)g(b)q(e)h(sligh)o(tly)h (di\013eren)o(t.)20 b(A)14 b(test)189 384 y(is)h(p)q(erformed)f(to)g(c) o(hec)o(k)h(whether)f Fj(buf)i(=)f(MPI)p 1011 384 14 2 v 15 w(BOTTOM)p Fs(.)g(If)g(true,)f(then)h(the)f(base)h(address)f(is) 189 440 y(zero,)g(otherwise)i(it)f(is)h Fj(buf)p Fs(.)21 b(In)16 b(particular,)f(if)h Fd(MPI)p 1083 440 13 2 v 14 w(BOTTOM)f Fs(do)q(es)h(not)f(ha)o(v)o(e)f(the)i(same)f(v)m(alue)189 497 y(in)i(F)l(ortran)f(and)h(C/C++,)g(then)g(an)f(additional)i(test)f (for)f Fj(bu\013)h(=)g(MPI)p 1463 497 14 2 v 16 w(BOTTOM)g Fs(is)h(needed)189 553 y(in)e(at)e(least)i(one)f(of)g(the)g(languages.) 189 628 y(It)f(ma)o(y)g(b)q(e)i(desirable)g(to)e(use)h(a)g(v)m(alue)h (other)e(than)g(zero)h(for)f Fd(MPI)p 1339 628 13 2 v 14 w(BOTTOM)h Fs(ev)o(en)g(in)g(C/C++,)189 685 y(so)h(as)h(to)f (distinguish)j(it)e(from)f(a)h(NULL)g(p)q(oin)o(ter.)26 b(If)17 b Fd(MPI)p 1234 685 V 14 w(BOTTOM)e(=)h(c)h Fs(then)g(one)h (can)f(still)189 741 y(a)o(v)o(oid)f(the)g(test)g Fj(bu\013)h(=)f(MPI)p 705 741 14 2 v 16 w(BOTTOM)p Fs(,)h(b)o(y)f(using)h(the)g(displacemen)o (t)h(from)d Fd(MPI)p 1668 741 13 2 v 15 w(BOTTOM)p Fs(,)189 798 y(i.e.,)d(the)g(regular)f(address)h(-)g(c,)g(as)g(the)g(MPI)f (address)h(returned)g(b)o(y)g Fj(MPI)p 1441 798 14 2 v 16 w(ADDRESS)g Fs(and)g(stored)189 854 y(in)k(absolute)f(datat)o(yp)q (es.)k(\()p Fi(End)d(of)g(advic)n(e)h(to)f(implementors.)p Fs(\))75 976 y Fe(10.2.7)49 b(Addresses)75 1109 y Fo(Curren)o(t)14 b(Status:)j Fn(No)d(v)o(otes.)166 1213 y Fs(Some)i(of)g(the)h(datat)o (yp)q(e)e(accessors)h(and)h(constructors)e(ha)o(v)o(e)h(argumen)o(ts)g (of)f(t)o(yp)q(e)i Fd(MPI)p 1717 1213 13 2 v 14 w(Aint)f Fs(\(in)75 1269 y(C\))e(or)g Fd(MPI::Aint)f Fs(in)j(C++,)f(to)e(hold)j (addresses.)k(The)15 b(corresp)q(onding)g(argumen)o(ts,)f(in)h(F)l (ortran,)e(ha)o(v)o(e)75 1326 y(t)o(yp)q(e)20 b Fd(INTEGER)p Fs(.)g(This)h(causes)f(F)l(ortran)f(and)i(C/C++)f(to)g(b)q(e)h (incompatible,)i(in)e(an)g(en)o(vironmen)o(t)75 1382 y(where)15 b(addresses)h(ha)o(v)o(e)f(64)f(bits,)h(but)h(F)l(ortran)e Fd(INTEGER)p Fs(s)g(ha)o(v)o(e)h(32)g(bits.)166 1438 y(This)k(is)h(a)e(problem,)i(irresp)q(ectiv)o(e)h(of)d(in)o (terlanguage)h(issues.)32 b(Supp)q(ose)20 b(that)e(a)g(F)l(ortran)g (pro-)75 1495 y(cess)h(has)g(an)g(address)g(space)g(of)g Fl(\025)g Fs(4)g(GB.)g(What)f(should)i(b)q(e)g(the)f(v)m(alue)h (returned)f(in)h(F)l(ortran)e(b)o(y)75 1551 y Fj(MPI)p 160 1551 14 2 v 16 w(ADDRESS)p Fs(,)i(for)e(a)h(v)m(ariable)i(with)e (an)h(address)f(ab)q(o)o(v)o(e)g(2)1195 1535 y Fc(32)1232 1551 y Fs(?)33 b(The)19 b(prop)q(osed)h(design)g(aims)g(to)75 1608 y(solv)o(e)15 b(this)h(issue,)g(while)g(main)o(taining)h (compatibilit)o(y)f(with)g(curren)o(t)f(F)l(ortran)f(co)q(des.)166 1664 y(The)h(constan)o(t)g Fd(MPI)p 520 1664 13 2 v 14 w(ADDRESS)p 720 1664 V 14 w(KIND)g Fs(is)h(de\014ned)g(so)f(that,)f(in) i(F)l(ortran)e(90,)75 1721 y Fd(INTEGER\(KIND=MPI)p 474 1721 V 14 w(ADDRESS)p 674 1721 V 14 w(KIND\))22 b Fs(is)g(an)g(address) g(sized)h(in)o(teger)f(t)o(yp)q(e)g(\(t)o(ypically)l(,)i(but)e(not)75 1777 y(necessarily)l(,)f Fd(MPI)p 392 1777 V 15 w(ADDRESS)p 593 1777 V 14 w(KIND)e Fs(is)g(4)g(on)g(32)g(bit)g(address)h(mac)o (hines)g(and)f(8)g(on)g(64)f(bit)i(address)75 1834 y(mac)o(hines\).)g (Similarly)l(,)d(the)f(constan)o(t)75 1890 y Fd(MPI)p 152 1890 V 14 w(INTEGER)p 340 1890 V 15 w(KIND)g Fs(is)h(de\014ned)h (so)f(that)f Fd(INTEGER\(KIND=MPI)p 1231 1890 V 13 w(INTEGER)p 1418 1890 V 15 w(KIND\))g Fs(is)h(a)g(default)g(size)75 1947 y Fd(INTEGER)p Fs(.)166 2003 y(There)e(are)g(sev)o(en)h(functions) g(that)e(ha)o(v)o(e)h(address)g(argumen)o(ts:)k Fj(MPI)p 1380 2003 14 2 v 16 w(TYPE)p 1513 2003 V 17 w(HVECTOR)p Fs(,)75 2059 y Fj(MPI)p 160 2059 V 16 w(TYPE)p 293 2059 V 17 w(HINDEXED)p Fs(,)c Fj(MPI)p 647 2059 V 15 w(TYPE)p 779 2059 V 17 w(STRUCT)p Fs(,)h Fj(MPI)p 1083 2059 V 16 w(ADDRESS)p Fs(,)f Fj(MPI)p 1411 2059 V 16 w(TYPE)p 1544 2059 V 17 w(EXTENT)75 2116 y(MPI)p 160 2116 V 16 w(TYPE)p 293 2116 V 17 w(LB)g Fs(and)g Fj(MPI)p 550 2116 V 16 w(TYPE)p 683 2116 V 17 w(UB)p Fs(.)1104 b Fl(>)16 b Fk(\(Oct\))166 2172 y Fs(F)l(our)k(new)i(functions)f(are)g(pro)o (vided)h(to)e(supplemen)o(t)i(the)f(\014rst)g(four)g(functions)g(in)h (this)g(list.)75 2229 y(These)15 b(functions)h(are)f(describ)q(ed)i(in) e(Section)h(10.3,)d(page)i(13)g(.)k(The)d(remaining)g(three)f (functions)g(are)75 2285 y(supplemen)o(ted)g(b)o(y)e(the)g(new)g (function)h Fj(MPI)p 849 2285 V 15 w(GET)p 952 2285 V 17 w(EXTENT)p Fs(,)f(describ)q(ed)i(in)f(that)e(same)h(section.)20 b(The)75 2342 y(new)g(functions)g(are)f(synonimous)h(with)g(the)g(old)g (functions)g(in)g(C/C++,)h(or)e(on)g(F)l(ortran)f(systems)75 2398 y(where)23 b(default)g Fd(INTEGER)p Fs(s)e(are)i(address)f(sized.) 43 b(In)23 b(F)l(ortran,)g(they)f(accept)h(argumen)o(ts)f(of)g(t)o(yp)q (e)75 2455 y Fd(INTEGER\(KIND=MPI)p 474 2455 13 2 v 14 w(ADDRESS)p 674 2455 V 14 w(SIZE\))p Fs(,)d(wherev)o(er)g(argumen)o(ts) f(of)h(t)o(yp)q(e)g Fd(MPI)p 1479 2455 V 15 w(Aint)g Fs(are)g(used)h(in)g(C.)75 2511 y(On)j(F)l(ortran)e(77)h(systems)g (that)f(do)h(not)g(supp)q(ort)h(the)f(F)l(ortran)f(90)h Fd(KIND)g Fs(notation,)h(and)g(where)75 2568 y(addresses)e(are)g(64)f (bits)i(whereas)f(default)h Fd(INTEGER)p Fs(s)e(are)g(32)h(bits,)h (these)g(argumen)o(ts)e(will)j(b)q(e)e(of)75 2624 y(t)o(yp)q(e)16 b Fd(INTEGER*8)p Fs(.)k(The)c(old)h(functions)f(will)i(con)o(tin)o(ue)e (to)f(b)q(e)i(pro)o(vided,)f(for)f(p)q(ortabilit)o(y)l(.)23 b(Ho)o(w)o(ev)o(er,)75 2680 y(users)14 b(are)g(encouraged)g(to)f(switc) o(h)h(to)f(the)h(new)h(functions,)f(in)h(F)l(ortran,)e(so)g(as)h(to)f (a)o(v)o(oid)g(problems)i(on)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fs(10)1110 b Fg(CHAPTER)15 b(10.)35 b(MISCELLANY)75 45 y Fs(systems)15 b(with)g(an)g(address)h(range)e Ff(>)f Fs(2)767 29 y Fc(32)805 45 y Fs(,)h(and)i(to)e(pro)o(vide)i (compatibilit)o(y)h(across)d(languages.)75 165 y Fe(10.2.8)49 b(Groups)75 298 y Fo(Curren)o(t)14 b(Status:)j Fn(P)o(assed)e(once.)166 401 y Fs(A)j(group)g(ob)s(ject)g(represen)o(t)h(the)f(same)g(group)g (in)h(all)h(languages:)26 b(the)18 b(same)g(pro)q(cesses,)i(with)75 458 y(the)15 b(same)g(ranks.)20 b(Group)15 b(accessors)f(suc)o(h)i(as)f Fj(MPI)p 986 458 14 2 v 15 w(GROUP)p 1153 458 V 18 w(SIZE)g Fs(return)g(the)h(same)f(v)m(alue.)75 577 y Fe(10.2.9)49 b(Communicato)o(rs)75 710 y Fo(Curren)o(t)14 b(Status:)j Fn(P)o(assed)e(once.)166 814 y Fs(A)k(comm)o(unicator)g(represen)o(ts)h (the)f(same)g(group)g(and)h(same)f(comm)o(unication)h(domain)g(in)g (all)75 870 y(languages.)25 b(In)17 b(particular,)h(a)e(message)g(sen)o (t)h(from)f(one)h(language)g(using)g(a)g(comm)o(unicator)f(can)h(b)q(e) 75 927 y(receiv)o(ed)e(from)e(another)g(language,)h(using)h(the)f(matc) o(hing)g(comm)o(unicator)f(on)h(another)f(pro)q(cess;)h(col-)75 983 y(lectiv)o(e)j(comm)o(unication)f(can)g(in)o(v)o(olv)o(e)g(pro)q (cesses)g(executing)h(co)q(de)f(written)g(in)g(di\013eren)o(t)g (languages,)75 1040 y(using)f(matc)o(hing)g(comm)o(unicators;)f (accessors)g(suc)o(h)h(as)f Fj(MPI)p 1152 1040 V 16 w(COMM)p 1310 1040 V 16 w(SIZE)h Fs(return)f(the)h(same)f(v)m(alue;)75 1096 y(etc.)166 1153 y(T)l(op)q(ology)e(information)h(asso)q(ciated)f (to)g(a)g(comm)o(unicator)g(can)g(b)q(e)h(accessed)g(in)h(an)o(y)e(of)f (the)i(three)75 1209 y(languages.)75 1329 y Fe(10.2.10)50 b(A)o(ttributes)75 1462 y Fo(Curren)o(t)14 b(Status:)j Fn(No)d(v)o(otes.)166 1565 y Fs(A)o(ttribute)k(k)o(eys)g(can)h(b)q(e)g (allo)q(cated)g(in)g(one)f(language)h(and)f(freed)h(in)g(another.)28 b(Similarly)l(,)21 b(at-)75 1622 y(tributes)13 b(v)m(alues)i(can)e(b)q (e)h(set)e(in)i(one)g(langage)f(and)g(accessed)g(in)h(another.)19 b(T)l(o)13 b(ac)o(hiev)o(e)h(this,)f(attribute)75 1678 y(k)o(eys)j(will)i(b)q(e)f(allo)q(cated)g(in)g(an)g(in)o(teger)f(range) g(that)f(is)i(v)m(alid)h(all)f(languages.)24 b(The)16 b(same)g(holds)h(true)75 1735 y(for)e(system)f(de\014ned)j(attribute)e (v)m(alues)h(\(suc)o(h)f(as)g Fd(MPI)p 1032 1735 13 2 v 15 w(T)m(A)o(G)p 1127 1735 V 14 w(UB,)d(MPI)p 1297 1735 V 14 w(IO)p Fs(,)i(etc.\))166 1791 y(A)o(ttribute)d(k)o(eys)g (declared)h(in)g(one)f(language)g(are)g(asso)q(ciated)h(with)f(cop)o(y) g(and)g(delete)h(functions)g(in)75 1848 y(that)e(language)g(\(the)h (functions)g(pro)o(vided)g(b)o(y)f(the)h Fj(MPI)p 1035 1848 14 2 v 16 w(Keyval)p 1178 1848 V 15 w(create)g Fs(call\).)19 b(When)11 b(a)f(comm)o(unicator)75 1904 y(is)16 b(duplicated,)i(for)d (eac)o(h)h(attribute,)g(the)g(corresp)q(onding)h(cop)o(y)e(function)i (is)f(called,)i(using)e(the)g(righ)o(t)75 1960 y(calling)j(con)o(v)o (en)o(tion)e(for)g(the)h(language)f(of)g(that)g(function;)h(and)g (similarly)l(,)h(for)e(the)g(delete)i(callbac)o(k)75 2017 y(function.)189 2106 y Fi(A)n(dvic)n(e)i(to)h(implementors.)77 b Fs(This)22 b(requires)g(that)f(attributes)g(b)q(e)h(tagged)f(either)h (as)f(\\C",)189 2163 y(\\C++")15 b(or)g(\\F)l(ortran",)e(and)j(that)e (the)i(language)f(tag)g(b)q(e)h(c)o(hec)o(k)o(ed)g(in)g(order)f(to)g (use)h(the)f(righ)o(t)189 2219 y(calling)i(con)o(v)o(en)o(tion)e(for)f (the)i(callbac)o(k)g(function.)21 b(\()p Fi(End)15 b(of)i(advic)n(e)f (to)g(implementors.)p Fs(\))166 2308 y(The)21 b(attribute)g (manipulation)h(functions)g(describ)q(ed)g(in)g(section)f(5.7)f(of)h (the)g(MPI)f(standard)75 2365 y(de\014ne)d(attributes)e(argumen)o(ts)g (to)f(b)q(e)j(of)e(t)o(yp)q(e)g Fd(void*)h Fs(in)g(C,)f(and)h(of)f(t)o (yp)q(e)g Fd(INTEGER)p Fs(,)f(in)j(F)l(ortran.)i(On)75 2421 y(some)14 b(systems,)f Fd(INTEGER)p Fs(s)h(will)i(ha)o(v)o(e)e(32) f(bits,)i(while)g(C/C++)g(p)q(oin)o(ters)f(will)i(ha)o(v)o(e)e(64)g (bits.)20 b(This)14 b(is)75 2478 y(a)j(problem)g(if)g(comm)o(unicator)g (attributes)f(are)h(used)g(to)g(mo)o(v)o(e)f(information)h(from)f(a)g (F)l(ortran)g(caller)75 2534 y(to)f(a)f(C/C++)i(callee,)g(or)f(vice-v)o (ersa.)-858 b Fl(>)15 b Fk(\(Oct\))166 2591 y Fs(MPI)j(will)h(store,)e (in)o(ternally)l(,)j(address)d(sized)i(attributes.)27 b(If)18 b(F)l(ortran)e Fd(INTEGER)p Fs(s)h(are)h(smaller,)75 2647 y(then)g(the)f(F)l(ortran)g(function)h Fj(MPI)p 694 2647 V 16 w(A)l(TTR)p 827 2647 V 17 w(GET)g Fs(will)h(return)e(the) h(least)f(signi\014can)o(t)i(part)e(of)g(the)g(at-)75 2704 y(tribute)e(w)o(ord;)f(the)g(F)l(ortran)g(function)h Fj(MPI)p 855 2704 V 16 w(A)l(TTR)p 988 2704 V 17 w(PUT)g Fs(will)h(set)f(the)f(least)h(signi\014can)o(t)h(part)e(of)g(the)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fg(10.2.)34 b(LANGUA)o(GE)15 b(INTER)o(OPERABILITY) 876 b Fs(11)75 45 y(attribute)16 b(w)o(ord,)f(whic)o(h)i(will)g(b)q(e)g (sign)f(extended)h(to)f(the)g(en)o(tire)g(w)o(ord.)21 b(\(These)16 b(t)o(w)o(o)f(functions)i(ma)o(y)75 102 y(b)q(e)f(in)o(v)o(ok)o(ed)f(explicitly)j(b)o(y)d(user)h(co)q(de,)f(or) g(implicitly)l(,)j(b)o(y)d(attribute)g(cop)o(ying)h(callbac)o(k)g (functions.\))189 208 y Fi(A)n(dvic)n(e)f(to)i(users.)40 b Fs(Users)15 b(are)g(encouraged)g(to)g(use)g(the)g(t)o(w)o(o)f(new)i (functions)189 264 y Fj(MPI)p 274 264 14 2 v 15 w(HANDLE)p 468 264 V 17 w(A)l(TTR)p 602 264 V 17 w(GET)24 b Fs(and)f Fj(MPI)p 909 264 V 16 w(HANDLE)p 1104 264 V 17 w(A)l(TTR)p 1238 264 V 17 w(GET)p Fs(,)g(describ)q(ed)i(in)f(Section)g Fh(??)p Fs(,)189 321 y(page)17 b Fh(??)p Fs(.)26 b(Their)18 b(F)l(ortran)e(binding)k(sp)q(eci\014es)f(address)f(sized)g(in)o(teger) g(argumen)o(ts,)f(th)o(us)g(pro-)189 377 y(viding)f(language)g(in)o (terop)q(erabilit)o(y)l(.)22 b(\()p Fi(End)15 b(of)h(advic)n(e)h(to)f (users.)p Fs(\))1875 438 y Fl(?)g Fk(\(Oct\))75 549 y Fe(10.2.11)50 b(Requests)75 682 y Fo(Curren)o(t)14 b(Status:)j Fn(P)o(assed)e(once.)166 785 y Fs(A)g(v)m(alid)i(request)e(created)g (in)h(on)f(language)g(can)g(b)q(e)h(accessed,)f(up)q(dated,)h(freed,)f (or)f(canceled)j(in)75 842 y(another)e(language.)75 964 y Fe(10.2.12)50 b(Erro)o(r)15 b(handlers)75 1097 y Fo(Curren)o(t)f (Status:)j Fn(P)o(assed)e(once.)166 1200 y Fs(Error)c(handles)i(can)g (b)q(e)f(allo)q(cated)i(and)e(deallo)q(cated)h(in)g(an)o(y)f(com)o (bination)h(of)f(languages.)19 b(When)75 1257 y(an)c(MPI)h(exception)h (o)q(ccurs,)e(then)h(the)g(last)f(error)g(handler)i(asso)q(ciated)e (with)h(the)g(relev)m(an)o(t)g(comm)o(u-)75 1313 y(nicator)h(is)g(in)o (v)o(ok)o(ed,)g(irresp)q(ectiv)o(e)h(of)f(the)f(language)h(en)o (vironmen)o(t)g(where)g(the)g(error)f(o)q(ccurred)i(and)75 1370 y(the)h(en)o(vironmen)o(t)f(where)h(the)g(handler)h(w)o(as)d(asso) q(ciated)i(with)g(the)g(comm)o(unicator.)29 b(The)19 b(handler)75 1426 y(will)e(b)q(e)f(in)o(v)o(ok)o(ed)f(with)h(the)f (righ)o(t)g(argumen)o(t)f(passing)i(con)o(v)o(en)o(tion.)189 1532 y Fi(A)n(dvic)n(e)j(to)h(implementors.)65 b Fs(Error)18 b(handler)j(ob)s(jects)d(need)j(to)d(ha)o(v)o(e)h(a)g(language)h(tag,)f (lik)o(e)189 1589 y(attribute)e(k)o(eys,)h(so)f(as)h(to)f(use)h(the)g (righ)o(t)f(calling)j(con)o(v)o(en)o(tion.)27 b(These)18 b(handlers)h(ha)o(v)o(e,)f(in)g(C)189 1645 y(and)f(C++,)h(a)f(\\)p Fm(stdargs)p Fs(")e(argumen)o(t)i(list.)26 b(It)18 b(migh)o(t)f(b)q(e)g (useful)i(to)d(pro)o(vide)i(to)f(the)g(handler)189 1702 y(information)c(on)h(the)f(language)h(en)o(vironmen)o(t)g(where)f(the)h (error)e(o)q(ccurred.)20 b(\()p Fi(End)14 b(of)h(advic)n(e)g(to)189 1758 y(implementors.)p Fs(\))75 1880 y Fe(10.2.13)50 b(Reduce)16 b(op)q(erations)75 2013 y Fo(Curren)o(t)e(Status:)j Fn(P)o(assed)e(once.)166 2117 y Fs(Reduce)g(op)q(erations)e(de\014ned)i (in)f(one)f(language)h(\(b)o(y)e(a)h(call)i(to)d Fj(MPI)p 1352 2117 V 16 w(OP)p 1430 2117 V 17 w(CREA)l(TE)p Fs(\))i(can)f(b)q(e) h(used)75 2173 y(or)h(deallo)q(cated)h(in)g(an)o(y)f(other)g(language.) 189 2280 y Fi(A)n(dvic)n(e)h(to)h(users.)47 b Fs(Reduce)18 b(op)q(erations)e(receiv)o(e,)h(as)f(one)g(of)g(their)h(argumen)o(t,)e (the)h(datat)o(yp)q(e)189 2336 y(of)d(the)g(op)q(erands.)20 b(Th)o(us,)14 b(one)f(can)h(de\014ne)h(\\p)q(olymorphic")f(reduce)g(op) q(erations)g(that)f(w)o(ork)g(for)189 2392 y(C,)h(C++,)i(and)f(F)l (ortran)f(datat)o(yp)q(es.)19 b(\()p Fi(End)d(of)g(advic)n(e)g(to)h (users.)p Fs(\))189 2499 y Fi(A)n(dvic)n(e)c(to)i(implementors.)39 b Fs(Reduce)15 b(op)q(eration)f(ob)s(jects)f(need)h(to)f(carry)g(a)h (language)f(tag,)g(lik)o(e)189 2555 y(attribute)h(k)o(eys,)g(so)h(as)f (to)g(use)h(the)g(righ)o(t)f(calling)j(con)o(v)o(en)o(tion.)i(\()p Fi(End)d(of)g(advic)n(e)f(to)h(implemen-)189 2612 y(tors.)p Fs(\))-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fs(12)1115 b Fg(CHAPTER)15 b(10.)30 b(MISCELLANY)75 45 y Fe(10.2.14)50 b(Status)75 178 y Fo(Curren)o(t)14 b(Status:)j Fn(V)m(ote??)166 282 y Fs(MPI)i(do)q(es)g (not)g(pro)o(vide)g(a)g(mec)o(hanism)g(for)g(passing)g(status)f(ob)s (jects)g(from)g(one)h(language)h(to)75 338 y(another.)189 442 y Fi(R)n(ationale.)40 b Fs(The)15 b(MPI)g(status)g(ob)s(ject)f(is)i (not)f(really)h(an)f(opaque)g(ob)s(ject,)f(since)j(its)e(structure)189 499 y(is)k(partially)g(visible)i(to)c(the)i(user,)g(and)g(the)f(ob)s (ject)g(is)h(allo)q(cated)g(b)o(y)g(the)f(user.)30 b(A)19 b(C)f(status)189 555 y(ob)s(ject)12 b(is)h(a)f(structure,)g(a)h(C++)g (status)e(ob)s(ject)h(is)h(a)g(real)f(ob)s(ject,)h(and)f(a)g(F)l (ortran)g(status)g(ob)s(ject)189 612 y(is)j(an)f(arra)o(y)l(.)19 b(Instead)c(of)f(passing)h(status)f(ob)s(jects,)g(one)h(can)f(pass)h (requests,)f(and)h(use)g(them)g(to)189 668 y(buld)h(status)f(ob)s (jects.)k(\()p Fi(End)c(of)i(r)n(ationale.)p Fs(\))75 790 y Fe(10.2.15)50 b(Constants)75 923 y Fo(Curren)o(t)14 b(Status:)j Fn(P)o(assed)e(once.)166 1026 y Fs(MPI)i(constan)o(ts)f(ha) o(v)o(e)g(the)h(same)g(v)m(alue)h(in)g(all)g(languages.)25 b(This)18 b(includes,)h(error)d(co)q(des,)i(con-)75 1083 y(stan)o(ts)12 b(suc)o(h)h(as)g Fm(MPI)p 435 1083 15 2 v 17 w(PROC)p 548 1083 V 16 w(NULL)g Fs(or)f Fm(MPI)p 798 1083 V 17 w(ANY)p 887 1083 V 17 w(TAG)p Fs(,)g(etc.)19 b(This)14 b(do)q(es)f(not)f(apply)i(to)f(constan)o(t)f(handles)75 1139 y(\()p Fd(MPI)p 170 1139 13 2 v 14 w(INT,)17 b(MPI)p 357 1139 V 15 w(COMM)p 502 1139 V 14 w(W)o(ORLD,)h(MPI)p 769 1139 V 14 w(ERRORS)p 943 1139 V 14 w(RETURN,)f(MPI)p 1226 1139 V 14 w(SUM)p Fs(,)i(etc.\))33 b(These)19 b(handles)i(need)75 1196 y(to)15 b(b)q(e)h(con)o(v)o(erted,)e(as)h(explained)j(in)e (Section)g(10.2.4.)i Fd(MPI)p 1109 1196 V 15 w(BOTTOM)d Fs(ma)o(y)f(ha)o(v)o(e)h(di\013eren)o(t)h(v)m(alues)g(in)75 1252 y(F)l(ortran,)e(C,)g(and)i(C++.)166 1356 y Fo(Missing:)h Fn(Need)d(to)g(add)g(the)g(new)h(MPI2)f(sp)q(ecial)g(addresses)75 1525 y Fe(10.2.16)50 b(Interlanguage)17 b(communication)75 1658 y Fo(Curren)o(t)d(Status:)j Fn(P)o(assed)e(once.)166 1761 y Fs(The)i(t)o(yp)q(e)h(matc)o(hing)f(rules)h(for)e(comm)o (unications)i(in)g(MPI)f(are)g(not)g(c)o(hanged:)24 b(The)17 b(datat)o(yp)q(e)75 1818 y(sp)q(eci\014cation)g(for)e(eac)o(h)h(item)f (sen)o(t)h(should)g(matc)o(h,)f(textually)l(,)h(the)f(datat)o(yp)q(e)g (sp)q(eci\014cation)j(used)e(to)75 1874 y(receiv)o(e)g(this)g(item)f (\(unless)h(one)g(of)e(the)i(t)o(yp)q(es)f(is)h Fd(MPI)p 1025 1874 V 14 w(P)m(A)o(CKED)p Fs(\).)c(And)k(the)f(t)o(yp)q(e)g(of)g (a)g(message)g(item)75 1931 y(should)21 b(matc)o(h)e(the)h(t)o(yp)q(e)f (declaration)i(for)e(the)h(corresp)q(onding)g(comm)o(unication)h (bu\013er)f(lo)q(cation,)75 1987 y(unless)c(the)g(t)o(yp)q(e)f(is)h Fd(MPI)p 513 1987 V 14 w(BYTE)e Fs(or)h Fd(MPI)p 781 1987 V 14 w(P)m(A)o(CKED)p Fs(.)e(In)o(terlanguage)i(comm)o(unication)h (is)g(allo)o(w)o(ed,)f(if)h(it)75 2044 y(complies)h(with)e(these)h (rules.)75 2148 y Fh(Example)i(10.10)23 b Fs(In)16 b(the)g(example)g(b) q(elo)o(w,)g(a)f(F)l(ortran)g(arra)o(y)f(is)i(sen)o(t)g(from)f(F)l (ortran)f(and)i(receiv)o(ed)75 2204 y(in)g(C.)75 2308 y Fm(!)24 b(FORTRAN)f(CODE)75 2365 y(REAL)g(R\(5\))75 2421 y(INTEGER)g(TYPE,)g(ADDR,)g(IERR,)g(MYRANK)75 2534 y(!)h(create)f(an)g(absolute)g(datatype)g(for)g(array)h(R)75 2591 y(CALL)f(MPI_ADDRESS\()g(R,)g(ADDR\))75 2647 y(CALL)g (MPI_TYPE_STRUCT\(1,)f(5,)h(ADDR,)g(MPI_REAL,)g(TYPE,)g(IERR\))75 2704 y(CALL)g(MPI_TYPE_COMMIT\()f(TYPE\))1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fg(10.3.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPE)g (MANIPULA)l(TION)i(FUNCTIONS)574 b Fs(13)75 102 y Fm(CALL)23 b(MPI_COMM_RANK\()f(MPI_COMM_WORLD,)g(MYRANK,)h(IERR\))75 158 y(IF)h(\(MYRANK.EQ.0\))e(THEN)75 214 y(CALL)h(MPI_SEND\()g (MPI_BOTTOM,)f(1,)i(TYPE,)f(1,)h(0,)f(MPI_COMM_WORLD,)f(IERR\))75 271 y(ELSE)75 327 y(CALL)h(MPI_HANDLE2INT\(TYPE,)e(CODE\))75 384 y(CALL)i(C_ROUTINE\(CODE\))75 440 y(END)g(IF)75 610 y(/*)h(C)f(code)h(*/)75 723 y(void)f(C_ROUTINE\(MPI_Fint)f(*fhandle\)) 75 779 y({)75 835 y(void)h(*handle;)75 892 y(MPI_Datatype)f(type;)75 948 y(MPI_Status)g(status;)75 1061 y(type)h(=)h (\(MPI_Datatype\)MPI_Int2hand)o(le\(fhand)o(le\);)75 1174 y(MPI_Recv\()f(MPI_BOTTOM,)f(1,)i(type,)f(0,)g(0,)h (MPI_COMM_WORLD,)e(status\);)75 1231 y(})166 1337 y Fs(MPI)17 b(implemen)o(tors)h(ma)o(y)f(w)o(eak)o(en)g(these)h(t)o(yp)q(e)f(matc)o (hing)h(rules,)g(and)g(allo)o(w)g(messages)e(to)h(b)q(e)75 1393 y(sen)o(t)c(with)g(F)l(ortran)f(t)o(yp)q(es)h(and)g(receiv)o(ed)h (with)f(C)g(t)o(yp)q(es,)g(and)g(vice)h(v)o(ersa,)e(when)h(those)g(t)o (yp)q(es)g(matc)o(h.)75 1450 y(I.e.,)g(if)h(the)f(F)l(ortran)f(t)o(yp)q (e)h Fd(INTEGER)f Fs(is)i(iden)o(tical)h(to)d(the)i(C)e(t)o(yp)q(e)i Fd(int)p Fs(,)f(then)h(an)f(MPI)g(implemen)o(tation)75 1506 y(ma)o(y)22 b(allo)o(w)g(data)g(to)f(b)q(e)i(sen)o(t)g(with)f (datat)o(yp)q(e)g Fd(MPI)p 1035 1506 13 2 v 14 w(INTEGER)g Fs(and)g(b)q(e)h(receiv)o(ed)h(with)e(datat)o(yp)q(e)75 1563 y Fd(MPI)p 152 1563 V 14 w(INT)p Fs(.)15 b(Ho)o(w)o(ev)o(er,)f (suc)o(h)i(co)q(de)f(is)h(not)f(p)q(ortable.)906 b Fl(>)16 b Fk(\(Oct\))75 1706 y Fp(10.3)59 b(New)20 b(datat)n(yp)r(e)d (manipulation)i(functions)75 1807 y Fs(New)e(functions)h(are)e(pro)o (vided)i(to)f(supplemen)o(t)h(the)f(t)o(yp)q(e)g(manipulation)h (functions)g(that)e(ha)o(v)o(e)h(ad-)75 1864 y(dress)d(sized)i(in)o (teger)e(argumen)o(ts.)k(The)d(new)f(functions)h(will)h(use,)e(in)h (their)g(F)l(ortran)e(binding)j(address)75 1920 y(sized)g Fd(INTEGER)p Fs(s,)d(th)o(us)i(solving)g(problems)g(curren)o(tly)g (encoun)o(tered)h(when)f(the)g(application)h(address)75 1977 y(range)h(is)i Ff(>)e Fs(2)325 1960 y Fc(32)362 1977 y Fs(.)27 b(Also,)18 b(a)g(new,)g(more)f(con)o(v)o(enien)o(t)h(t)o (yp)q(e)g(constructor)f(is)h(pro)o(vided)h(to)e(mo)q(dify)h(the)75 2033 y(lo)o(w)o(er)d(b)q(ound)h(and)f(exten)o(t)g(of)g(a)g(datat)o(yp)q (e.)75 2155 y Fe(10.3.1)49 b(T)l(yp)q(e)17 b(constructo)o(rs)d(with)j (explicit)g(addresses)75 2241 y Fs(The)11 b(four)f(functions)h(b)q(elo) o(w)h(supplemen)o(t)f(the)g(four)f(corresp)q(onding)i(t)o(yp)q(e)f (constructor)e(functions)j(from)75 2297 y Fj(MPI-1)p Fs(.)18 b(The)c(new)f(functions)g(are)g(synon)o(ymous)g(with)g(the)g (old)g(functions)h(in)g(C/C++,)f(or)f(on)h(F)l(ortran)75 2354 y(systems)k(where)h(default)h Fd(INTEGER)p Fs(s)e(are)g(address)h (sized.)29 b(In)19 b(F)l(ortran,)d(they)i(accept)g(argumen)o(ts)f(of)75 2410 y(t)o(yp)q(e)e Fd(INTEGER\(KIND=MPI)p 576 2410 V 13 w(ADDRESS)p 775 2410 V 15 w(SIZE\))p Fs(,)f(wherev)o(er)g(argumen)o (ts)g(of)g(t)o(yp)q(e)h Fd(MPI)p 1558 2410 V 14 w(Aint)g Fs(are)f(used)h(in)75 2467 y(C.)i(On)i(F)l(ortran)d(77)i(systems)f (that)g(do)h(not)f(supp)q(ort)h(the)g(F)l(ortran)f(90)g Fd(KIND)h Fs(notation,)g(and)g(where)75 2523 y(addresses)c(are)g(64)f (bits)h(whereas)g(default)h Fd(INTEGER)p Fs(s)e(are)g(32)h(bits,)g (these)g(argumen)o(ts)f(will)j(b)q(e)e(of)g(t)o(yp)q(e)75 2580 y Fd(INTEGER*8)p Fs(.)19 b(The)c(old)g(functions)h(will)h(con)o (tin)o(ue)e(to)g(b)q(e)g(pro)o(vided,)h(for)e(p)q(ortabilit)o(y)l(.)21 b(Ho)o(w)o(ev)o(er,)14 b(users)75 2636 y(are)h(encouraged)g(to)g(switc) o(h)g(to)g(the)g(new)h(functions,)f(in)h(F)l(ortran.)166 2692 y(The)f(new)h(functions)g(are)f(listed)h(b)q(elo)o(w.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fs(14)1110 b Fg(CHAPTER)15 b(10.)35 b(MISCELLANY)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(TYPE)p 293 45 V 17 w(HVECTOR)p 521 45 V 17 w(X\()16 b(count,)g(blo)q (cklength,)g(stride,)g(oldt)o(yp)q(e,)g(newt)o(yp)q(e\))117 122 y Fn(IN)155 b Fj(count)482 b Fn(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(nonnegativ)o(e)f(in)o(teger\))117 197 y(IN)155 b Fj(blo)q(cklength)371 b Fn(n)o(um)o(b)q(er)16 b(of)h(elemen)o(ts)g(in)f(eac)o(h)i(blo)q(c)o(k)e(\(nonnegativ)o(e)h (in)o(te-)905 254 y(ger\))117 329 y(IN)155 b Fj(stride)484 b Fn(n)o(um)o(b)q(er)13 b(of)g(b)o(ytes)h(b)q(et)o(w)o(een)h(start)f (of)f(eac)o(h)h(blo)q(c)o(k)f(\(in)o(teger\))117 404 y(IN)155 b Fj(oldt)o(yp)q(e)450 b Fn(old)13 b(datat)o(yp)q(e)i (\(handle\))117 479 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fn(new)15 b(datat)o(yp)q(e)f(\(handle\))75 604 y Fm(int)23 b(MPI)p 245 604 15 2 v 17 w(Type)p 358 604 V 17 w(hvector)p 543 604 V 16 w(x\(int)g(count,)g(int)h(blocklength,)e(MPI)p 1347 604 V 17 w(Aint)h(stride,)393 660 y(MPI)p 468 660 V 17 w(Datatype)g(oldtype,)f(MPI)p 986 660 V 17 w(Datatype)h (*newtype\))75 747 y(MPI)p 150 747 V 17 w(TYPE)p 263 747 V 16 w(HVECTOR)p 447 747 V 17 w(X\(COUNT,)f(BLOCKLENGTH,)h(STIDE,)g (OLDTYPE,)f(NEWTYPE,)h(IERROR\))170 803 y(INTEGER)g(COUNT,)g (BLOCKLENGTH,)g(OLDTYPE,)f(NEWTYPE,)h(IERROR)170 860 y(INTEGER\(KIND=MPI)p 557 860 V 15 w(ADDRESS)p 740 860 V 17 w(SIZE\))g(STRIDE)75 1041 y Fj(MPI)p 160 1041 14 2 v 16 w(TYPE)p 293 1041 V 17 w(HINDEXED)p 537 1041 V 16 w(X\()14 b(count,)h(a)o(rra)o(y)p 843 1041 V 14 w(of)p 894 1041 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1267 1041 V 15 w(of)p 1319 1041 V 16 w(displacements,)f(oldt)o(yp)q(e,)g (new-)75 1097 y(t)o(yp)q(e\))117 1174 y Fn(IN)155 b Fj(count)482 b Fn(n)o(um)o(b)q(er)34 b(of)f(blo)q(c)o(ks)h({)g(also)f(n)o(um)o(b)q (er)h(of)f(en)o(tries)i(in)905 1231 y Fd(a)o(rra)o(y)p 992 1231 13 2 v 15 w(of)p 1041 1231 V 15 w(displacements)19 b Fn(and)f Fd(a)o(rra)o(y)p 1483 1231 V 15 w(of)p 1532 1231 V 14 w(blo)q(cklengths)i Fn(\(in)o(te-)905 1287 y(ger\))117 1362 y(IN)155 b Fj(a)o(rra)o(y)p 416 1362 14 2 v 15 w(of)p 468 1362 V 16 w(blo)q(cklengths)191 b Fn(n)o(um)o(b)q(er)15 b(of)g(elemen)o(ts)h(in)f(eac)o(h)h(blo)q(c)o (k)g(\(arra)o(y)f(of)g(nonnega-)905 1419 y(tiv)o(e)f(in)o(tegers\))117 1494 y(IN)155 b Fj(a)o(rra)o(y)p 416 1494 V 15 w(of)p 468 1494 V 16 w(displacements)164 b Fn(b)o(yte)14 b(displacemen)o(t)g (of)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 1569 y(IN)155 b Fj(oldt)o(yp)q(e)450 b Fn(old)13 b(datat)o(yp)q(e)i (\(handle\))117 1644 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fn(new)15 b(datat)o(yp)q(e)f(\(handle\))75 1769 y Fm(int)23 b(MPI)p 245 1769 15 2 v 17 w(Type)p 358 1769 V 17 w(hindexed)p 567 1769 V 16 w(x\(int)g(count,)g(int)h(*array)p 1133 1769 V 16 w(of)p 1197 1769 V 17 w(blocklengths,)393 1825 y(MPI)p 468 1825 V 17 w(Aint)f(*array)p 748 1825 V 17 w(of)p 813 1825 V 17 w(displacements,)e(MPI)p 1259 1825 V 17 w(Datatype)i(oldtype,)393 1881 y(MPI)p 468 1881 V 17 w(Datatype)g(*newtype\))75 1968 y(MPI)p 150 1968 V 17 w(TYPE)p 263 1968 V 16 w(HINDEXED)p 471 1968 V 16 w(X\(COUNT,)g(ARRAY)p 822 1968 V 17 w(OF)p 887 1968 V 17 w(BLOCKLENGTHS,)f(ARRAY)p 1358 1968 V 16 w(OF)p 1422 1968 V 17 w(DISPLACEMENTS,)393 2024 y(OLDTYPE,)h(NEWTYPE,)g(IERROR\)) 170 2081 y(INTEGER)g(COUNT,)g(ARRAY)p 651 2081 V 17 w(OF)p 716 2081 V 17 w(BLOCKLENGTHS\(*\),)e(OLDTYPE,)i(NEWTYPE,)g(IERROR)170 2137 y(INTEGER\(KIND=MPI)p 557 2137 V 15 w(ADDRESS)p 740 2137 V 17 w(SIZE\))g(ARRAY)p 1020 2137 V 16 w(OF)p 1084 2137 V 17 w(DISPLACEMENTS\(*\))1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fg(10.3.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPE)g (MANIPULA)l(TION)i(FUNCTIONS)574 b Fs(15)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(TYPE)p 293 45 V 17 w(STRUCT)p 486 45 V 17 w(X\(count,)12 b(a)o(rra)o(y)p 776 45 V 14 w(of)p 827 45 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1197 45 V 15 w(of)p 1249 45 V 16 w(displacements,)g(a)o(rra)o(y)p 1646 45 V 14 w(of)p 1697 45 V 16 w(t)o(yp)q(es,)f(new-)75 102 y(t)o(yp)q(e\))117 179 y Fn(IN)155 b Fj(count)482 b Fn(n)o(um)o(b)q(er)18 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))g({)g (also)f(n)o(um)o(b)q(er)h(of)f(en)o(tries)905 235 y(in)c(arra)o(ys)h Fd(a)o(rra)o(y)p 1167 235 13 2 v 15 w(of)p 1216 235 V 14 w(t)o(yp)q(es)p Fn(,)h Fd(a)o(rra)o(y)p 1432 235 V 15 w(of)p 1481 235 V 14 w(displacements)g Fn(and)e Fd(a)o(r-)905 292 y(ra)o(y)p 959 292 V 15 w(of)p 1008 292 V 15 w(blo)q(cklengths)117 367 y Fn(IN)155 b Fj(a)o(rra)o(y)p 416 367 14 2 v 15 w(of)p 468 367 V 16 w(blo)q(cklength)208 b Fn(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h(eac)o(h)g(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in) o(teger\))117 442 y(IN)155 b Fj(a)o(rra)o(y)p 416 442 V 15 w(of)p 468 442 V 16 w(displacements)164 b Fn(b)o(yte)14 b(displacemen)o(t)g(of)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in)o (teger\))117 517 y(IN)155 b Fj(a)o(rra)o(y)p 416 517 V 15 w(of)p 468 517 V 16 w(t)o(yp)q(es)327 b Fn(t)o(yp)q(e)20 b(of)f(elemen)o(ts)g(in)g(eac)o(h)h(blo)q(c)o(k)f(\(arra)o(y)g(of)g (handles)g(to)905 574 y(datat)o(yp)q(e)14 b(ob)r(jects\))117 649 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fn(new)15 b(datat)o(yp)q(e)f (\(handle\))75 773 y Fm(int)23 b(MPI)p 245 773 15 2 v 17 w(Type)p 358 773 V 17 w(struc)p 495 773 V 16 w(xt\(int)g(count,)g (int)h(*array)p 1085 773 V 16 w(of)p 1149 773 V 17 w(blocklengths,)393 830 y(MPI)p 468 830 V 17 w(Aint)f(*array)p 748 830 V 17 w(of)p 813 830 V 17 w(displacements,)e(MPI)p 1259 830 V 17 w(Datatype)i(*array)p 1635 830 V 16 w(of)p 1699 830 V 17 w(types,)393 886 y(MPI)p 468 886 V 17 w(Datatype)g(*newtype\)) 75 972 y(MPI)p 150 972 V 17 w(TYPE)p 263 972 V 16 w(STRUCT)p 423 972 V 17 w(X\(COUNT,)g(ARRAY)p 775 972 V 16 w(OF)p 839 972 V 17 w(BLOCKLENGTHS,)f(ARRAY)p 1310 972 V 17 w(OF)p 1375 972 V 16 w(DISPLACEMENTS,)393 1029 y(ARRAY)p 516 1029 V 17 w(OF)p 581 1029 V 17 w(TYPES,)h(NEWTYPE,)f(IERROR\))170 1085 y(INTEGER)h(COUNT,)g(ARRAY)p 651 1085 V 17 w(OF)p 716 1085 V 17 w(BLOCKLENGTHS\(*\),)e(ARRAY)p 1258 1085 V 17 w(OF)p 1323 1085 V 17 w(TYPES\(*\),)i(NEWTYPE,)170 1142 y(IERROR)170 1198 y(INTEGER\(KIND=MPI)p 557 1198 V 15 w(ADDRESS)p 740 1198 V 17 w(SIZE\))g(ARRAY)p 1020 1198 V 16 w(OF)p 1084 1198 V 17 w(DISPLACEMENTS\(*\),)75 1379 y Fj(MPI)p 160 1379 14 2 v 16 w(ADDRESS)p 378 1379 V 17 w(X\(lo)q(cation,)15 b(address\))117 1456 y Fn(IN)155 b Fj(lo)q(cation)437 b Fn(lo)q(cation)13 b(in)h(caller)f(memory)f(\(c)o (hoice\))117 1532 y(OUT)108 b Fj(address)449 b Fn(address)15 b(of)f(lo)q(cation)f(\(in)o(teger\))75 1656 y Fm(int)23 b(MPI)p 245 1656 15 2 v 17 w(Address)p 430 1656 V 16 w(x\(void*)g(location,)g(MPI)p 948 1656 V 17 w(Aint)g(*address\))75 1743 y(MPI)p 150 1743 V 17 w(ADDRESS)p 335 1743 V 16 w(X\(LOCATION,)f(ADDRESS,)h(IERROR\))170 1799 y()g(LOCATION\(*\)) 170 1855 y(INTEGER)g(IERROR)170 1912 y(INTEGER\(KIND=MPI)p 557 1912 V 15 w(ADDRESS)p 740 1912 V 17 w(SIZE\))g(ADDRESS)189 2048 y Fi(A)n(dvic)n(e)d(to)h(users.)72 b Fs(Curren)o(t)20 b(F)l(ortran)g(MPI)g(co)q(des)h(will)h(run)f(unmo)q(di\014ed,)i(and)e (will)h(p)q(ort)189 2105 y(to)f(an)o(y)g(system.)39 b(Ho)o(w)o(ev)o (er,)22 b(they)g(ma)o(y)f(fail)h(if)g(addresses)g(larger)g(than)f(2) 1558 2088 y Fc(32)1610 2105 y Fl(\000)15 b Fs(1)21 b(are)g(used)189 2161 y(in)f(the)f(program.)31 b(New)19 b(co)q(des)h(should)g(b)q(e)g (written)f(so)g(that)g(they)g(use)h(the)f(new)h(functions.)189 2218 y(This)e(pro)o(vides)h(compatibilit)o(y)g(with)f(C/C++)g(and)h(a)o (v)o(oids)e(errors)g(on)h(64)g(bit)g(arc)o(hitectures.)189 2274 y(Ho)o(w)o(ev)o(er,)13 b(suc)o(h)i(newly)g(written)g(co)q(des)g (ma)o(y)e(need)j(to)e(b)q(e)h(\(sligh)o(tly\))g(rewritten)f(to)g(p)q (ort)h(to)e(old)189 2330 y(F)l(ortran)k(77)g(en)o(vironmen)o(ts)i(that) f(do)g(not)g(supp)q(ort)g Fd(KIND)g Fs(declarations.)30 b(\()p Fi(End)18 b(of)i(advic)n(e)f(to)189 2387 y(users.)p Fs(\))1875 2448 y Fl(?)d Fk(\(Oct\))75 2558 y Fe(10.3.2)49 b(Extent)16 b(and)h(Bounds)f(of)h(Datat)o(yp)q(es)75 2644 y Fs(The)d(follo)o(wing)h(function)g(supplemen)o(ts)h(the)e(three) g(functions)h Fj(MPI)p 1266 2644 14 2 v 16 w(TYPE)p 1399 2644 V 17 w(UB,)f(MPI)p 1586 2644 V 16 w(TYPE)p 1719 2644 V 17 w(LB)g Fs(and)75 2701 y Fj(MPI)p 160 2701 V 16 w(TYPE)p 293 2701 V 17 w(EXTENT)p Fs(.)h(It)g(also)g(returns)h (address)f(sized)h(in)o(tegers,)f(in)h(the)f(F)l(ortran)f(binding.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Fs(16)1110 b Fg(CHAPTER)15 b(10.)35 b(MISCELLANY)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(GET)p 264 45 V 17 w(EXTENT)p 459 45 V 17 w(X\(datat)o(yp)q(e,)16 b(lb,)f(extent\))117 122 y Fn(IN)155 b Fj(datat)o(yp)q(e)424 b Fn(datat)o(yp)q(e)14 b(to)g(get)g(information)d(on)117 197 y(OUT)108 b Fj(lb)553 b Fn(lo)o(w)o(er)14 b(b)q(ound)g(of)f(datat)o (yp)q(e)117 273 y(OUT)108 b Fj(extent)471 b Fn(exten)o(t)15 b(of)e(datat)o(yp)q(e)75 397 y Fm(int)23 b(MPI)p 245 397 15 2 v 17 w(Get)p 334 397 V 17 w(extent)p 495 397 V 16 w(x\(MPI)p 631 397 V 17 w(Datatype)g(datatype,)f(MPI)p 1173 397 V 17 w(Aint)h(*lb,)h(MPI)p 1501 397 V 16 w(Aint)g(*extent\))75 483 y(MPI)p 150 483 V 17 w(GET)p 239 483 V 17 w(EXTENT)p 400 483 V 16 w(X\(DATATYPE,)e(LB,)i(EXTENT,)f(IERROR\))170 540 y(INTEGER)g(DATATYPE,)g(IERROR)170 596 y(INTEGER\(KIND)g(=)g(MPI)p 603 596 V 17 w(ADDRESS)p 788 596 V 16 w(SIZE\))h(LB,)f(EXTENT)-117 633 y Fl(>)15 b Fk(\(Oct\))75 683 y Fm(int)23 b(MPI::Datatype::Get)p 605 683 V 15 w(extent)p 764 683 V 17 w(x\(MPI::Aint&)f(lb,)h (MPI::Aint&)g(extent\))g(const)166 769 y Fs(Returns)16 b(the)f(lo)o(w)o(er)g(b)q(ound)h(and)f(the)g(exten)o(t)g(of)g Fj(datat)o(yp)q(e)h Fs(\(as)f(de\014ned)h(b)o(y)f(the)h(MPI)f (standard,)75 826 y(Section)h(3.12.2\).)166 882 y(MPI)21 b(allo)o(ws)h(one)f(to)g(c)o(hange)h(the)f(exten)o(t)g(of)g(a)h(datat)o (yp)q(e,)g(using)g(lo)o(w)o(er)f(b)q(ound)h(and)g(upp)q(er)75 939 y(b)q(ound)e(mark)o(ers)e(\()p Fd(MPI)p 490 939 13 2 v 14 w(LB)h Fs(and)g Fd(MPI)p 740 939 V 14 w(UB)p Fs(\).)f(This)h(is) h(useful,)g(as)f(it)g(allo)o(ws)g(to)f(con)o(trol)h(the)g(stride)g(of) 75 995 y(successiv)o(e)e(datat)o(yp)q(es)e(that)g(are)g(replicated)i(b) o(y)e(datat)o(yp)q(e)g(constructors,)g(or)g(are)g(replicated)i(b)o(y)f (the)75 1052 y Fj(count)k Fs(argumen)o(t)e(in)i(a)e(send)h(or)g(reciev) o(e)g(call.)32 b(Ho)o(w)o(ev)o(er,)19 b(the)f(curren)o(t)h(mec)o (hanism)h(for)e(ac)o(hieving)75 1108 y(it)h(is)h(painful;)i(also)d(it)h (is)f(restrictiv)o(e,)i(as)d Fd(MPI)p 906 1108 V 15 w(LB)h Fs(and)g Fd(MPI)p 1157 1108 V 14 w(UB)g Fs(are)f(\\stic)o(ky":)28 b(once)19 b(presen)o(t)g(in)h(a)75 1165 y(datat)o(yp)q(e,)d(they)h (cannot)g(b)q(e)g(o)o(v)o(eriden)g(\(e.g.,)f(the)h(upp)q(er)h(b)q(ound) f(can)g(b)q(e)g(mo)o(v)o(ed)g(up,)g(b)o(y)g(adding)g(a)75 1221 y(new)13 b Fd(MPI)p 243 1221 V 14 w(UB)g Fs(mark)o(er,)f(but)h (cannot)g(b)q(e)g(mo)o(v)o(ed)g(do)o(wn)f(b)q(elo)o(w)i(an)f(existing)h Fj(MPI)p 1474 1221 14 2 v 15 w(UB)g Fs(mark)o(er\).)k(A)13 b(new)-1992 b Fl(>)15 b Fk(\(Oct\))75 1277 y Fs(t)o(yp)q(e)g (constructor)g(is)g(pro)o(vided)h(to)f(facilitate)h(these)f(c)o (hanges.)75 1428 y Fj(MPI)p 160 1428 V 16 w(TYPE)p 293 1428 V 17 w(RESIZED)p 492 1428 V 16 w(X\(oldt)o(yp)q(e,)h(lb,)f (extent,)i(newt)o(yp)q(e\))117 1506 y Fn(IN)155 b Fj(oldt)o(yp)q(e)450 b Fn(input)14 b(datat)o(yp)q(e)g(\(handle\))117 1581 y(IN)155 b Fj(lb)553 b Fn(new)15 b(lo)o(w)o(er)e(b)q(ound)h(of)f(datat) o(yp)q(e)h(\(in)o(teger\))117 1656 y(IN)155 b Fj(extent)471 b Fn(new)15 b(exten)o(t)f(of)g(datat)o(yp)q(e)g(\(in)o(teger\))117 1731 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fn(output)14 b(datat)o(yp)q(e)g(\(handle\))75 1855 y Fm(int)23 b(MPI)p 245 1855 15 2 v 17 w(Type)p 358 1855 V 17 w(resized)p 543 1855 V 16 w(x\(MPI)p 679 1855 V 17 w(Datatype)f(oldtype,)h(MPI)p 1197 1855 V 17 w(Aint)g(lb,)h(MPI)p 1501 1855 V 16 w(Aint)g(extent,)393 1912 y(MPI)p 468 1912 V 17 w(Datatype)f(*newtype\))75 1998 y(MPI)p 150 1998 V 17 w(TYPE)p 263 1998 V 16 w(RESIZED)p 447 1998 V 17 w(X\(OLDTYPE,)f(LB,)i(EXTENT,)e(NEWTYPE,)h(IERROR\))170 2055 y(INTEGER)g(DATATYPE,)g(NEWTYPE,)g(IERROR)170 2111 y(INTEGER\(KIND=MPI)p 557 2111 V 15 w(ADDRESS)p 740 2111 V 17 w(SIZE\))g(LB,)g(EXTENT)75 2198 y(int)g(MPI::Datatype::Resized)p 701 2198 V 15 w(x\(const)g(MPI::Datatype&)e(type,)j(const)f(MPI::Aint&) 393 2254 y(lb,)h(const)f(MPI::Aint&)f(extent\))-117 2290 y Fl(?)15 b Fk(\(Oct\))166 2341 y Fs(Returns)f(in)g Fj(newt)o(yp)q(e)i Fs(a)d(handle)i(to)e(a)g(new)h(datat)o(yp)q(e)f(that)g(is)h(iden)o (tical)h(to)e Fj(oldt)o(yp)q(e)p Fs(,)i(except)f(that)75 2397 y(the)k(lo)o(w)o(er)f(b)q(ound)i(of)e(this)h(new)g(datat)o(yp)q(e) g(is)g(set)f(to)g(b)q(e)i Fj(lb)p Fs(,)f(and)g(its)g(upp)q(er)h(b)q (ound)f(is)h(set)e(to)g(b)q(e)i Fj(lb)75 2454 y Fs(+)h Fj(extent)p Fs(.)34 b(This)20 b(is)g(as)f(if)h(an)o(y)f(previous)h Fh(lb)g Fs(and)g Fh(ub)f Fs(mark)o(ers)g(w)o(ere)g(erased,)h(and)g(a)f (new)h(pair)f(of)-1992 b Fl(>)15 b Fk(\(Oct\))75 2510 y Fs(lo)o(w)o(er)i(b)q(ound)i(and)f(upp)q(er)h(b)q(ound)f(mark)o(ers)f (w)o(ere)h(put)g(in)g(the)g(p)q(ositions)h(indicated)g(b)o(y)f(the)g Fj(lb)g Fs(and)75 2566 y Fj(extent)g Fs(argumen)o(ts.)24 b(This)17 b(a\013ects)f(the)h(b)q(eha)o(vior)h(of)e(the)h(datat)o(yp)q (e)f(when)h(used)h(in)f(comm)o(unication)75 2623 y(op)q(erations,)e (with)h Fj(count)g Ff(>)d Fs(1,)h(and)i(when)g(used)f(in)h(the)g (construction)f(of)g(new)g(deriv)o(ed)i(datat)o(yp)q(es.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fg(10.3.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPE)g (MANIPULA)l(TION)i(FUNCTIONS)574 b Fs(17)189 45 y Fi(A)n(dvic)n(e)14 b(to)i(users.)40 b Fs(It)14 b(is)h(strongly)f(recommended)i(that)e (users)g(use)h(these)g(t)o(w)o(o)e(new)i(functions,)189 102 y(rah)o(ter)h(than)h(the)g(old)g(MPI)h(functions)f(to)g(set)g(and)g (access)g(lo)o(w)o(er)g(b)q(ound,)h(upp)q(er)g(b)q(ound)g(and)189 158 y(exten)o(t)d(of)f(datat)o(yp)q(es.)20 b(\()p Fi(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fs(\))1875 211 y Fl(>)g Fk(\(Oct\))75 320 y Fe(10.3.3)49 b(T)l(rue)16 b(Extent)g(of)g(Datat)o (yp)q(es)75 406 y Fs(Supp)q(ose)22 b(w)o(e)f(implemen)o(t)h(gather)f (as)f(a)h(spanning)h(tree)f(implemen)o(ted)i(on)e(top)g(of)f(p)q(oin)o (t-to-p)q(oin)o(t)75 462 y(routines.)f(Since)11 b(the)g(recv)f (bu\013er)g(is)h(only)g(v)m(alid)h(on)e(the)h(ro)q(ot)e(pro)q(cess,)i (one)g(will)g(need)h(to)d(allo)q(cate)i(some)75 519 y(temp)q(orary)i (space)i(for)e(receiving)j(data)d(on)h(in)o(termediate)h(no)q(des.)20 b(The)14 b(di\016cultly)i(is)f(in)f(determining)75 575 y(the)21 b(size)h(one)f(needs)h(to)f(allo)q(cate.)38 b(This)21 b(o)q(ccurs)h(since)g(the)f(user)g(can)h(mo)q(dify)f(the)g (exten)o(t)g(using)75 632 y(the)h Fd(MPI)p 237 632 13 2 v 14 w(UB)g Fs(and)g Fd(MPI)p 500 632 V 14 w(LB)g Fs(v)m(alues.)42 b(The)22 b(writer)g(of)f(the)i(gather)e(routine)h(could)h(determine)h (this)75 688 y(information)16 b(b)o(y)g(deco)q(ding)i(the)e(datat)o(yp) q(e.)21 b(Ho)o(w)o(ev)o(er,)15 b(this)i(is)f(more)g(w)o(ork)f(and)h (more)g(painful)h(than)75 745 y(desired.)k(Th)o(us,)15 b(a)g(new)g(function)h(is)g(pro)o(vided)g(whic)o(h)g(returns)f(the)g (true)g(exten)o(t)g(of)g(the)g(datat)o(yp)q(e.)75 895 y Fj(MPI)p 160 895 14 2 v 16 w(TRUE)p 294 895 V 17 w(EXTENT)p 489 895 V 17 w(X\(datat)o(yp)q(e,)h(true)p 821 895 V 17 w(lb,)f(true)p 975 895 V 17 w(extent\))117 973 y Fn(IN)155 b Fj(datat)o(yp)q(e)424 b Fn(datat)o(yp)q(e)14 b(to)g(get)g (information)d(on)117 1047 y(OUT)108 b Fj(true)p 396 1047 V 17 w(lb)461 b Fn(true)15 b(lo)o(w)o(er)e(b)q(ound)h(of)g(datat)o (yp)q(e)117 1121 y(OUT)108 b Fj(true)p 396 1121 V 17 w(extent)379 b Fn(true)15 b(size)g(of)e(datat)o(yp)q(e)75 1245 y Fm(int)23 b(MPI)p 245 1245 15 2 v 17 w(True)p 358 1245 V 17 w(extent)p 519 1245 V 16 w(x\(MPI)p 655 1245 V 17 w(Datatype)f(datatype,)h(MPI)p 1197 1245 V 17 w(Aint)g(*true)p 1453 1245 V 17 w(lb,)g(MPI)p 1637 1245 V 17 w(Aint)393 1301 y(*true)p 516 1301 V 17 w(extent\))75 1388 y(MPI)p 150 1388 V 17 w(TRUE)p 263 1388 V 16 w(EXTENT)p 423 1388 V 17 w(X\(DATATYPE,)f(TRUE)p 822 1388 V 17 w(LB,)h(TRUE)p 1030 1388 V 17 w(EXTENT,)g(IERROR\))170 1444 y(INTEGER)g(DATATYPE,)g (IERROR)170 1501 y(INTGER\(KIND)g(=)h(MPI)p 580 1501 V 16 w(ADDRESS)p 764 1501 V 17 w(SIZE\))f(TRUE)p 1020 1501 V 16 w(LB,)h(TRUE)p 1228 1501 V 16 w(EXTENT)1875 1537 y Fl(>)16 b Fk(\(Oct\))75 1587 y Fm(int)23 b(MPI::Datatype::True)p 629 1587 V 15 w(extent)p 788 1587 V 16 w(x\(MPI::Aint&)g(true)p 1211 1587 V 16 w(lb,)h(MPI::Aint&)e(true)p 1681 1587 V 17 w(extent\))393 1644 y(const)166 1730 y Fj(true)p 244 1730 14 2 v 17 w(lb)14 b Fs(returns)f(the)g(o\013set)g(of)g(the)g (lo)o(w)o(est)g(unit)h(of)f(store)g(whic)o(h)h(is)g(addressed)g(b)o(y)f (the)h(datat)o(yp)q(e,)75 1787 y(i.e.,)j(the)g(lo)o(w)o(er)f(b)q(ound)i (of)f(the)g(corresp)q(onding)h(t)o(yp)q(emap,)e(ignoring)i Fd(MPI)p 1391 1787 13 2 v 14 w(LB)f Fs(mark)o(ers.)24 b Fj(true)p 1743 1787 14 2 v 17 w(extent)75 1843 y Fs(returns)12 b(the)h(true)f(size)h(of)f(the)g(datat)o(yp)q(e,)g(i.e.,)h(the)f(exten) o(t)g(of)g(the)h(corresp)q(onding)g(t)o(yp)q(emap,)f(ignoring)75 1900 y Fd(MPI)p 152 1900 13 2 v 14 w(LB)17 b Fs(and)h Fj(MPI)p 407 1900 14 2 v 16 w(UB)f Fs(mark)o(ers,)f(and)i(p)q (erforming)f(no)g(rounding)h(for)e(alignmen)o(t.)27 b(If)17 b(the)g(t)o(yp)q(emap)75 1956 y(asso)q(ciated)e(with)h Fj(datat)o(yp)q(e)h Fs(is)189 2044 y Ff(T)6 b(y)r(pemap)12 b Fs(=)h Fl(f)p Fs(\()p Ff(ty)r(pe)562 2051 y Fc(0)581 2044 y Ff(;)8 b(disp)686 2051 y Fc(0)705 2044 y Fs(\))p Ff(;)g(:)g(:)g(:)t(;)g Fs(\()p Ff(ty)r(pe)926 2051 y Fb(n)p Fa(\000)p Fc(1)994 2044 y Ff(;)g(disp)1099 2051 y Fb(n)p Fa(\000)p Fc(1)1166 2044 y Fs(\))p Fl(g)75 2132 y Fs(Then)189 2219 y Ff(tr)q(ue)p 277 2219 V 16 w(l)q(b)p Fs(\()p Ff(T)e(y)r(pemap)p Fs(\))11 b(=)i Ff(min)691 2226 y Fb(j)709 2219 y Fl(f)p Ff(disp)816 2226 y Fb(j)861 2219 y Fs(:)28 b Ff(ty)r(pe)986 2226 y Fb(j)1017 2219 y Fl(6)p Fs(=)13 b Fh(lb)p Fl(g)p Ff(;)189 2317 y(tr)q(ue)p 277 2317 V 16 w(ub)p Fs(\()p Ff(T)6 b(y)r(pemap)p Fs(\))12 b(=)g Ff(max)709 2324 y Fb(j)728 2317 y Fl(f)p Ff(disp)835 2324 y Fb(j)862 2317 y Fs(+)f Ff(siz)r(eof)5 b Fs(\()p Ff(ty)r(pe)1140 2324 y Fb(j)1158 2317 y Fs(\))28 b(:)f Ff(ty)r(pe)1328 2324 y Fb(j)1360 2317 y Fl(6)p Fs(=)13 b Fh(ub)p Fl(g)p Ff(;)75 2415 y Fs(and)189 2503 y Ff(tr)q(ue)p 277 2503 V 16 w(extent)p Fs(\()p Ff(T)6 b(y)r(pemap)p Fs(\))13 b(=)g Ff(tr)q(ue)p 790 2503 V 17 w(ub)p Fs(\()p Ff(T)6 b(y)r(pemap)p Fs(\))j Fl(\000)h Ff(tr)q(ue)p 1216 2503 V 17 w(l)q(b)p Fs(\()p Ff(ty)r(pemap)p Fs(\))p Ff(:)75 2591 y Fs(\(Compares)k(this)i(with)f(the)h(de\014nitions)h(in)f (Section)g(3.12.3)d(of)i(the)g(MPI)g(standard.\))166 2647 y(The)25 b Fj(true)p 347 2647 V 17 w(extent)h Fs(is)e(the)h(minim) o(um)g(n)o(um)o(b)q(er)g(of)f(b)o(ytes)g(of)g(memory)g(necessary)g(to)g (hold)h(a)75 2704 y(datat)o(yp)q(e)15 b(\(uncompressed\).)1285 b Fl(?)16 b Fk(\(Oct\))-32 46 y(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Fs(18)1110 b Fg(CHAPTER)15 b(10.)35 b(MISCELLANY)75 45 y Fp(10.4)59 b(Continuable)19 b(Erro)n(rs)75 147 y Fs(T)o(ypically)l(,)j(MPI)d(sa)o(ys)g(nothing)h(ab)q(out)g(the)f (in)o(ternal)h(state)f(of)g(itself)i(after)d(an)i(error)f(handler)h (has)75 203 y(b)q(een)14 b(in)o(v)o(ok)o(ed.)20 b(There)13 b(are)g(a)g(n)o(um)o(b)q(er)g(of)g(error)f(classes)i(within)g(MPI)f (that)g(should)h(ha)o(v)o(e)f(no)g(negativ)o(e)75 259 y(impact)19 b(on)f(the)g(in)o(ternal)h(state)f(of)g(MPI.)g(That)g(is)g (to)g(sa)o(y)l(,)g(if)h(one)g(of)e(the)i("con)o(tin)o(uable")g(errors)e (is)75 316 y(detected)g(and)h(no)e(other)h(external)g(ev)o(en)o(ts)g (cause)g(degradation,)g(it)g(is)g(safe)g(to)f(con)o(tin)o(ue)h(using)h (MPI)75 372 y(within)e(an)g(application.)166 429 y(The)f("con)o(tin)o (uable")h(error)f(classes)g(are:)75 535 y Fm(MPI_ERR_BUFFER)237 b(/*)23 b(invalid)g(buffer)g(pointer)g(*/)75 592 y(MPI_ERR_COUNT)261 b(/*)23 b(invalid)g(count)g(argument)g(*/)75 648 y(MPI_ERR_TYPE)285 b(/*)23 b(invalid)g(datatype)g(argument)g(*/)75 704 y(MPI_ERR_TAG)309 b(/*)23 b(invalid)g(tag)h(argument)e(*/)75 761 y(MPI_ERR_COMM)285 b(/*)23 b(invalid)g(communicator)f(*/)75 817 y(MPI_ERR_KEYVAL)237 b(/*)23 b(invalid)g(key)h(value)f(*/)75 874 y(MPI_ERR_ROOT)285 b(/*)23 b(invalid)g(root)h(*/)75 930 y(MPI_ERR_RANK)285 b(/*)23 b(invalid)g(rank)h(*/)75 987 y(MPI_ERR_GROUP)261 b(/*)23 b(invalid)g(group)g(*/)75 1043 y(MPI_ERR_TOPOLOGY)189 b(/*)23 b(invalid)g(topology)g(*/)75 1100 y(MPI_ERR_DIMS)285 b(/*)23 b(invalid)g(dimension)g(argument)g(*/)75 1156 y(MPI_ERR_ARG)309 b(/*)23 b(invalid)g(argument)g(*/)75 1262 y Fs(The)13 b(follo)o(wing)h(errors)f(pro)o(vide)g(no)g(guaran)o (tees,)g(although)g(implemen)o(tors)h(are)f(encouraged)g(to)f(mak)o(e) 75 1319 y(errors)i("con)o(tin)o(uable")i(whenev)o(er)g(p)q(ossible:)75 1413 y Fm(MPI_ERR_OP)333 b(/*)23 b(invalid)g(operation)g(*/)75 1469 y(MPI_ERR_REQUEST)213 b(/*)23 b(invalid)g(request)g(handle)g(*/)75 1526 y(MPI_ERR_UNKNOWN)213 b(/*)23 b(unknown)g(error)g(*/)75 1582 y(MPI_ERR_TRUNCATE)189 b(/*)23 b(message)g(truncated)g(on)g (receive)g(*/)75 1638 y(MPI_ERR_OTHER)261 b(/*)23 b(Other)g(error)h(*/) 75 1695 y(MPI_ERR_INTERN)237 b(/*)23 b(internal)g(MPI)h(error)f(*/)75 1751 y(MPI_ERR_IN_STATUS)165 b(/*)23 b(error)g(code)h(is)f(in)h(status) f(*/)75 1808 y(MPI_ERR_PENDING)213 b(/*)23 b(pending)g(request)g(*/)75 1864 y(MPI_ERR_SYSRESOURCE)117 b(/*)23 b(out)h(of)f(system)g(resources) g(*/)h(/*)f(where)g(in)h(MPI?)f(*/)166 1958 y Fs(W)l(e)12 b(also)f(pro)o(vide)i(the)e(function)i(to)e(indicate)i(whether)f(MPI)g (b)q(eliev)o(es)h(an)f(error)f(is)h("con)o(tin)o(uable")75 2015 y(or)j(not:)75 2166 y Fj(MPI)p 160 2166 14 2 v 16 w(ERR)p 261 2166 V 17 w(IS)p 316 2166 V 16 w(CONTINUABLE)h(\(erro)o(r,) d(\015ag\))117 2243 y Fn(IN)155 b Fj(Erro)o(r)490 b Fn(MPI)14 b(Error)h(Status)f(\(actual)g(status)h(or)f(a)f(class\))117 2318 y(OUT)108 b Fj(Flag)505 b Fn(Bo)q(olean)13 b(indication)f(of)h (whether)h(the)g(implemen)o(tation)c(al-)905 2374 y(lo)o(ws)19 b(safe)h(con)o(tin)o(ued)g(op)q(eration)g(in)f(the)i(ev)o(en)o(t)f(of)f (Error)905 2431 y(b)q(eing)14 b(encoun)o(tered)75 2555 y Fm(int)23 b(MPI)p 245 2555 15 2 v 17 w(Err)p 334 2555 V 17 w(is)p 399 2555 V 17 w(continuable\()f(int)i(error,)f(int)g(*flag) g(\))75 2642 y(MPI)p 150 2642 V 17 w(ERR)p 239 2642 V 17 w(IS)p 304 2642 V 16 w(CONTINUABLE\()g(ERROR,)g(FLAG,)g(IERROR)g(\)) 170 2698 y(INTEGER)g(ERROR,)g(IERROR)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fg(10.5.)34 b(NEW)15 b(ELEMENT)l(AR)l(Y)h(D)o(A)l (T)l(A)l(TYPES)883 b Fs(19)170 45 y Fm(LOGICAL)23 b(FLAG)75 132 y(static)g(int)g(MPI::Err)p 532 132 15 2 v 17 w(is)p 597 132 V 16 w(continuable\()g(int)g(error,)g(bool&)g(flag)h(\))1875 172 y Fl(>)16 b Fk(\(Oct\))166 218 y Fs(There)c(is)h(a)f(new,)h(con)o (tin)o(uable)g(error)f(class,)h Fj(MPI)p 1026 218 14 2 v 16 w(ERR)p 1127 218 V 17 w(W)o(OULDBLOCK)p Fs(.)f(This)g(error)g (class)h(ma)o(y)75 274 y(b)q(e)j(used)g(b)o(y)g(a)f(non-blo)q(c)o(king) i(function)f(to)f(inform)h(the)f(caller)i(that)e(the)g(op)q(eration)h (cannot)f(pro)q(ceed)75 331 y(without)g(blo)q(c)o(king.)22 b(When)15 b(this)h(error)e(is)i(returned,)f(the)h(state)e(of)h(the)g (application)i(is)f(equiv)m(alen)o(t)h(to)75 387 y(the)e(one)h(prior)f (to)g(the)g(function)h(call.)166 444 y(An)d(implemen)o(tation)h(is)f (not)g(required)g(to)f(use)i(this)f(error)f(class,)h(but)g(if)g(it)g (do)q(es,)g(it)g(m)o(ust)f(pro)o(vide)75 500 y(the)j(functionalit)o(y)i (describ)q(ed.)166 604 y Fo(Discussion:)33 b Fn(What)13 b(do)q(es)h(it)f(mean)f(when)i(a)f(con)o(tin)o(uable)f(error)j(o)q (ccurs)f(in)f(a)g(collectiv)o(e)g(op)q(eration?)166 708 y Fs(There)k(is)g(a)f(prede\014ned)i(error)e(handler)h Fj(MPI)p 969 708 V 16 w(CONT)p 1110 708 V 17 w(ERRORS)p 1299 708 V 18 w(RETURN)p Fs(,)h(whic)o(h)f(allo)o(ws)g(con-)75 764 y(tin)o(uable)e(erorrs)e(to)f(return)i(error)f(co)q(des)h(and)f (non-con)o(tin)o(uable)j(errors)c(to)h(ab)q(ort.)19 b(The)14 b(default)g(error)75 821 y(handler)i(is)g Fj(MPI)p 370 821 V 16 w(ERRORS)p 558 821 V 18 w(ARE)p 662 821 V 17 w(F)l(A)l(T)l(AL)p Fs(.)75 927 y Fh(Example)i(10.11)118 b Fm(MPI_Errhandler_set\(MPI_COMM)o(_WORLD,)20 b (MPI_CONT_ERRORS_RETURN\);)170 983 y(err)k(=)g(MPI_Isend\()e(...)i(,)f (&req\);)170 1040 y(if)h(\()g(err)f(!=)h(MPI_SUCCESS\))e({)266 1096 y(MPI_Error_class\(err,&clas)o(s\);)266 1153 y(if)h(\(class)h(==)f (MPI_ERR_WOULDBLOCK\))e({)361 1209 y(MPI_Send\()i(...)g(\);)266 1266 y(else)361 1322 y(...)266 1379 y(})170 1435 y(})1875 1489 y Fl(?)16 b Fk(\(Oct\))75 1628 y Fp(10.5)59 b(New)20 b(Elementa)n(ry)e(Datat)n(yp)r(es)75 1777 y Fo(Discussion:)34 b Fn(Do)13 b(w)o(e)h(need)h Fd(MPI)p 644 1777 13 2 v 14 w(W)o(CHAR)p Fn(,)e(to)h(tak)o(e)g(care)h(of)e(Unico)q(de?)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Fri Sep 20 05:22:55 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA17638; Fri, 20 Sep 1996 05:22:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA28140 for mpi-core-out; Thu, 19 Sep 1996 23:26:13 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA28135; Thu, 19 Sep 1996 23:26:02 -0500 Message-Id: <199609200426.XAA28135@antares.mcs.anl.gov> To: mpi-comm@antares.mcs.anl.gov, mpi-core@antares.mcs.anl.gov Subject: Misc-1.2 Date: Thu, 19 Sep 1996 23:26:00 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Thu Sep 19 16:46:18 1996 %%Pages: 23 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.09.19:1646 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 1 106 df<030007800780030000000000000000007F807F80038003800380 038003800380038003800380038003800380FFFCFFFC0E187D9714>105 D E /Fb 12 122 df<387CFEFEFE7C3807077C860F>46 D<00E00001E0000FE000FFE000 F3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000 03E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000FFFF80 FFFF80111D7C9C1A>49 D<01FC0007FF000E0F801E0FC03F07E03F07E03F07E03F07E01E 0FC0000FC0000F80001F0001FC0001FC00000F800007C00003E00003F00003F83803F87C 03F8FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003FC00151D7E9C1A>51 D69 D<07FC001FFF003F0F803F07 C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803 E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 D<01FE0007FF800F83C01E01E03E00F0 7C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E0018 0F807007FFE000FF8015147F9318>101 D108 DI112 D<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F80 0F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>116 D120 DI E /Fc 4 103 df<70F8FCFC74040404080810102040060E7C840D>59 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E10F01C20F01C20703C20705C40308C400F078014147E9318>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207E9F14>I< 00007C0000CE00019E00039E00030C000700000700000700000700000E00000E00000E00 00FFF0000E00000E00001C00001C00001C00001C00001C00003800003800003800003800 00380000700000700000700000700000700000E00000E00000E00000E00000C00001C000 318000798000F300006200003C000017297E9F16>102 D E /Fd 33 122 df<00E001C0038007000E000E001C001C003800380038007000700070007000E0 00E000E000E000E000E000E000E000E000E000E000E00070007000700070003800380038 001C001C000E000E000700038001C000E00B2A7E9E10>40 DI44 D<001C0000003E0000003E0000 002E0000006700000067000000E7800000C7800000C3800001C3C0000183C0000181C000 0381E0000381E0000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C003C00 1C003C0018003C0038001E0038001E0070001F0070000F0070000F00E0000780191D7F9C 1C>65 DI<003FC000FFF003C0F007 80300F00001E00003C00003C0000780000780000780000F00000F00000F00000F00000F0 0000F00000F00000F00000F000007800007800007800003C00003C00001E00000F000807 801803C07800FFF0003F80151F7D9D1B>III<003F8001FFF003C0F807 80380F00181E00003C00003C0000780000780000780000F00000F00000F00000F00000F0 0000F00000F007F8F007F8F000387800387800387800383C00383C00381E00380F003807 803803C0F801FFF0003F80151F7D9D1C>71 D73 D76 DII<003F000001FFE00003FFF00007C0F8000F807C001E 001E003E001F003C000F00780007807800078078000780F00003C0F00003C0F00003C0F0 0003C0F00003C0F00003C0F00003C0F00003C0F80007C078000780780007807C000F803C 000F003E001F001F003E000F807C0007C0F80003FFF00001FFE000003F00001A1F7E9D1F >II<003F000001FFE00003FFF000 07C0F8000F807C001F003E003E001F003C000F00780007807800078078000780F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C078000780 78000780780E07803C0F0F003E079F001E03DE000F83FC0007C1F80003FFF00001FFF800 003F780000003C0000003E0000001F0000000F801A237E9D1F>II<03F8000FFE001C0F00380700700300600000E00000E000 00E00000E00000F000007800007F00003FE0001FFC0007FE0001FF00001F800007800003 C00003C00001C00001C00001C00001C0C00180E00380F007007C0E001FFC0007F000121F 7E9D17>III<78000E007C001E003C003C001E0038000F0070000F00F0000781E0 0003C1C00001C3C00001E7800000F70000007E0000003E0000003C0000003C0000007E00 000077000000E7800001E3800003C1C0000381E0000700F0000F00F8000E0078001C003C 003C003E0078001F0070000F00F0000F80191D7F9C1C>88 DI<0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC 7FDC3F1C0E127E9114>97 DI<000E000E000E000E000E000E000E000E000E000E000E0F8E1FEE3FFE7C3E70 0E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E0F1D7E9C15>100 D<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C3FFC1FF807 E00E127E9112>I108 D110 D112 D<1C001C001C001C001C001C00FFE0FFE01C001C001C001C001C001C00 1C001C001C001C001C001C001C201FF00FF007C00C187F970F>116 DI<7003807807003C0E001C1C000E1C0007380003F00001E00001C00001 E00003F0000738000E18000E1C001C0E00380700700380F003C01212809113>120 DI E /Fe 24 119 df<00003F03E00000C386700001878CF00003879CF00003031860000700380000 070038000007003800000E003800000E007000000E007000000E00700000FFFFFF80001C 007000001C00E000001C00E000001C00E000001C00E000003800E000003801C000003801 C000003801C000003801C000007001C00000700380000070038000007003800000700380 00006003800000E007000000E007000000E007000000E007000000C006000001C00E0000 01C00E000031860C0000798F180000F31E100000620C6000003C07C000002429829F1C> 11 D<70F8F8F0E005057B840E>46 D<00000200000006000000060000000E0000001E00 00001E0000003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F00 00020F0000040F00000C0F0000080F0000100F0000100F0000200F80003FFF8000400780 00C007800080078001000780010007800200078002000780060007801E000F80FF807FF8 1D207E9F22>65 D<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004 003C0004003C00040078080800780800007808000078180000F0300000FFF00000F03000 00F0300001E0200001E0200001E0200001E0001003C0002003C0002003C0004003C00040 078000800780018007800100078007000F001F00FFFFFE001F1F7D9E1F>69 D<01FFFF00001E03C0001E00E0001E0070001E0078003C0078003C0078003C0078003C00 78007800F0007800F0007801E0007801C000F0070000F01E0000FFF00000F0380001E01C 0001E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003C01E0007803C0007803C 0807803C0807803C100F801C10FFF00C20000007C01D207D9E21>82 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207B9F15>I< 007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F00000 F00000F00000F0000070010070020030040018380007C00011147C9315>I<0000780003 F80000700000700000700000700000E00000E00000E00000E00001C00001C000F1C00389 C00707800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E 20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C01C207010E011C01 3C013802780C7BF07C00F000F000F000F0007000700170023804183807C010147C9315> I<00007800019C00033C00033C000718000700000700000E00000E00000E00000E00000E 0001FFE0001C00001C00001C00001C000038000038000038000038000038000070000070 0000700000700000700000700000E00000E00000E00000E00000C00001C00001C0000180 003180007B0000F300006600003C00001629829F0E>I<003C6000E27001C1E00380E007 00E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C03803C07003C07001C 0F001C17000C2E0003CE00000E00000E00001C00001C00301C00783800F0700060E0003F 8000141D7E9315>I<01E0000FE00001C00001C00001C00001C000038000038000038000 038000070000070000071E000763000E81800F01C00E01C00E01C01C03801C03801C0380 1C0380380700380700380700380E10700E20700C20701C20700C40E00CC060070014207D 9F17>I<00C001E001E001C000000000000000000000000000000E003300230043804300 470087000E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E> I<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030 000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00800E00800E 00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C00012147C93 15>I E /Ff 50 123 df<000078007C7800FC7801FC7803C00003800007800007800007 8000078000078000078000078000078000FFFC78FFFC78FFFC7807807807807807807807 807807807807807807807807807807807807807807807807807807807807807807807807 8078078078078078078078152480A31A>12 D45 DI<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C0FFFFFFFFFFFF10227CA118>49 D<03F0000FFC001FFE003C1F003007807007C06003 C0E003E0C001E04001E04001E00001E00001E00001E00003C00003C0000780000780000F 00001E00003C0000780000F00001E00001C0000380000700000E00001C00003800007000 00FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC001FFF003E0F003807807003C020 03C02003C00003C00003C00003C0000780000780000F00001E0003FC0003F80003FE0000 0F000007800003C00003C00001E00001E00001E00001E00001E08001E0C003C0E003C070 07803C0F801FFF000FFC0003F00013237EA118>I<001F00001F00002F00002F00006F00 00EF0000CF0001CF0001CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F00 3C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00 000F00000F00000F00000F0015217FA018>I55 D<01F00007FC000FFE001E0F003C07803C0780 7803C07803C07803C07803C07803C03803803C07801E0F000F1E0007FC0003F8000FFE00 1E0F003C07807803C07803C0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C0 3C07803E0F801FFF0007FC0001F00013237EA118>I<01F00007FC000FFE001E0F003C07 003807807803807803C0F003C0F001C0F001E0F001E0F001E0F001E0F001E0F001E07003 E07803E07803E03C07E03E1FE01FFDE00FF1E003E3C00003C00003C00003800007800007 00000F00101E00183C003FF8001FF00007C00013237EA118>I<001F0000001F0000003F 8000003F8000003B8000007BC0000073C0000071C00000F1E00000F1E00000E0E00001E0 F00001E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C000F00 1E000F001E000FFFFE001FFFFF001FFFFF001C000F003C0007803C000780380007807800 03C0780003C0700003C0F00001E0F00001E0E00001E01B237EA220>65 DI<000FF000007FFC0000FFFF0001F01F0003C00700078000000F0000001E0000 003E0000003C0000003C000000780000007800000078000000F0000000F0000000F00000 00F0000000F0000000F0000000F0000000F0000000F00000007800000078000000780000 003C0000003C0000003E0000001E0000000F0000000780008003C0038001F00F8000FFFF 00007FFC00000FF00019257DA31F>IIII<000FF000003FFE0000 FFFF8001F80F8003E00380078000000F0000001E0000001E0000003C0000003C00000078 0000007800000078000000F0000000F0000000F0000000F0000000F0000000F0000000F0 00FFC0F000FFC0F000FFC0780003C0780003C0780003C03C0003C03C0003C01E0003C01E 0003C00F0003C0078003C003E003C001F807C000FFFFC0003FFF00000FF8001A257DA321 >I73 D76 DII<001FC000007FF00001FFFC0003F07E 0007C01F000F800F801F0007C01E0003C03C0001E03C0001E0780000F0780000F0780000 F070000070F0000078F0000078F0000078F0000078F0000078F0000078F0000078F00000 78F0000078780000F0780000F0780000F07C0001F03C0001E03E0003E01E0003C01F0007 C00F800F8007C01F0003F07E0001FFFC00007FF000001FC0001D257DA324>II< FFFC00FFFF80FFFFC0F003E0F000F0F00078F00038F0003CF0003CF0003CF0003CF0003C F00038F00078F000F0F003E0FFFFC0FFFF80FFFE00F01E00F00F00F00700F00780F00380 F003C0F001E0F001E0F000F0F000F0F00078F00038F0003CF0001EF0001EF0000F18237B A21F>82 D<00FE0003FFC007FFE00F81E01E00603C00003C000078000078000078000078 00007800007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F000 00F00000F8000078000078000078000078000078000078C000F0E000F0F801E07E07C03F FF800FFF0001FC0015257EA31B>III90 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF00FF0 0FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180 780000780000700000F00000F00000F00000F00000F00000F00000780000780000780000 3C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC01FFFC03F0F C03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C07803 C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC001FFE003E 0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F0000070000078 00007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01FF03C003 8007800780078007800780078007800780FFF8FFF8FFF807800780078007800780078007 80078007800780078007800780078007800780078007800780102380A20F>I<01F07807 FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F000FFE001F FC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8F00078F0 0078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>I 105 D107 DIII<01FC0007FF000FFF801F07C03C01E07800F078 00F0700070F00078F00078F00078F00078F00078F000787800F07800F07C01F03E03E01F 07C00FFF8007FF0001FC0015167F9518>II114 D<07F01FFC3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC0 0FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F00 0F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E0 0E1C7F9B12>III<7801F07C01E03E03C01E07C00F 0780078F0007DE0003FC0001FC0000F80000700000F80001FC0003DC00039E00078F000F 07801E07801E03C03C01E07800F0F800F81516809516>120 DI<7FFFC07FFFC07FFFC0000F80000F00 001E00003E00007C0000780000F00001F00003E00003C0000780000F80001F00001E0000 3C00007C0000FFFFC0FFFFC0FFFFC012167F9515>I E /Fg 32 90 df45 D<3078F8787005057C840D>I<007E0001C3000301800701 C00E00C00E00E01C00E01C00E03C01E03801E07801E07801E07801E07801E07801E0F003 C0F003C0F003C0F003C0F003C0F00380F00780E00780E00700E00700E00E00600E00701C 003038003870000FC000131F7C9D17>48 D<000C001C00FC0F3800380038003800380038 00700070007000700070007000E000E000E000E000E000E001C001C001C001C001C001C0 038003C0FFFE0F1E7C9D17>I<003F8000C1E00100F00200780400780400780F007C0F80 7C0F807C0F00780600780000F80000F00001E00001C0000380000700000E00001C000038 0000600000C0000180000300200600200800401000403FFFC07FFF80FFFF80161E7E9D17 >I<007F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E00003C000 0380000700000E0000F800000E000007000007800007C00003C00007C03007C07807C0F8 07C0F807C0F00780800F00400E00201C0018780007E000141F7D9D17>I<01803001FFE0 03FFC003FF0003FC00020000020000020000040000040000040000047C00058700060380 0C01800801C00001C00001E00001E00001E00001E07003C0F803C0F003C0E00380800780 400700400E00201C0018700007C000141F7D9D17>53 D<2000003FFFE07FFFC07FFF8040 0100C0020080020080040000080000100000200000400000400000800001800003000003 00000700000600000E00000E00001E00001C00001C00003C00003C00003C000078000078 0000780000300000131F799D17>55 D<003F0000C1C00100600200600400300C00300C00 300C00300C00600E00600F80C00FC18007F60003FC0001FC0001FF00063F800C0F801007 C03003C06001C06000C0C000C0C000C0C000C0C00080C0010060030030040018180007E0 00141F7D9D17>I<007E0001C3000301800601C00E01C01C00C03C00E03C00E03C01E078 01E07801E07801E07801E07803E07803E03803C03807C01C0BC00C13C003E38000078000 0780000700000E00600E00F01C00F01800E0300080600041C0003F0000131F7C9D17>I< 0000100000001800000038000000380000007800000078000000FC000001BC0000013C00 00033C0000023C0000063C0000043E0000081E0000081E0000101E0000101E0000201E00 00200F0000400F0000400F0000FFFF0000800F0001000F80010007800200078002000780 04000780040007800C0007C03E0007C0FF807FFC1E207E9F22>65 D<0001F808000E061800380138007000F801E0007803C0007007800030078000300F0000 301F0000301E0000303E0000203C0000007C0000007C0000007C0000007C000000F80000 00F8000000F8000000F8000000F80000007800004078000080780000803C0000803C0001 001C0002000E00020006000C000300100001C0E000003F00001D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E007800 1E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E01E000 3E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C003 C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038 003C001800780018007800080078000800780008007800080078080800F0100000F01000 00F0100000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001E00008 01E0001003C0001003C0001003C0002003C0002003C0006003C000C0078001C0078007C0 FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C0018007800180078000800 78000800780008007800080078000800F0100000F0100000F0100000F0300000F0700000 FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0000003C0000003 C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F7E9E1E>I<0001 FC04000F030C003C009C0070007C00E0003C01C0003803800018078000180F0000181F00 00181E0000183E0000103C0000007C0000007C0000007C0000007C000000F8000000F800 0000F8007FFCF80003E0780001E0780001E0780003C0780003C03C0003C03C0003C01C00 03C00E0007C007000B800380118001E06080003F80001E217B9F24>I<07FFC7FFC0007C 00F800003C007800003C007800007800F000007800F000007800F000007800F000007800 F000007800F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000F001E0 0001E003C00001E003C00001E003C00001E003C00001E003C00001E003C00003C0078000 03C007800003C007800003C007800003C007800003C007800007800F000007C00F8000FF F8FFF800221F7E9E22>I<07FFE0007C00003C00003C0000780000780000780000780000 780000780000F00000F00000F00000F00000F00000F00001E00001E00001E00001E00001 E00001E00003C00003C00003C00003C00003C00003C00007800007C000FFFC00131F7F9E 10>I<07FFE07FE0007C001F00003C000C00003C00180000780010000078004000007800 800000780100000078020000007804000000F008000000F010000000F060000000F0F000 0000F1F0000000F278000001E478000001E878000001F03C000001E03C000001E01E0000 01E01E000003C00F000003C00F000003C00F000003C007800003C007800003C003C00007 8003C00007C007E000FFFC3FFC00231F7E9E23>75 D<07FFF000007E0000003C0000003C 000000780000007800000078000000780000007800000078000000F0000000F0000000F0 000000F0000000F0000000F0000001E0000001E0000001E0000001E0000001E0008001E0 010003C0010003C0010003C0030003C0020003C0060003C0060007801E0007807C00FFFF FC00191F7E9E1C>I<07FC0000FFC0007C0000F800003C00017800003C00017800004E00 02F000004E0002F000004E0004F000004E0004F000004E0008F000004E0008F000008700 11E00000870011E00000870021E00000870021E00000870041E00000838041E000010380 83C00001038083C00001038103C00001038203C0000101C203C0000101C403C0000201C4 0780000201C80780000201C80780000201D00780000200F00780000600E00780000600E0 0F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E0018 00003E001800004F001000004F001000004780100000478010000043C010000043C01000 0083C020000081E020000081E020000080F020000080F020000080782000010078400001 007C400001003C400001003C400001001E400001001E400002000F800002000F80000200 0F800002000780000200078000060003800006000300000F00010000FFE0010000221F7E 9E22>I<0003F800001E0E000038070000E0038001C001C003C001E0078000E00F0000F0 0F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8 F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C000780 3C000F003C001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07 FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F8007800F8007800F800 7800F000F001F000F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001 E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003 C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C 01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F0 078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E0 1E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC 0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E 00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E000 01E00000E00000F00000F04000E04000E04000E04000E06000C0600180E00380F80300C6 0C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010 C00F0010800F0010800F0010800F0010001E0000001E0000001E0000001E0000001E0000 001E0000003C0000003C0000003C0000003C0000003C0000003C00000078000000780000 0078000000780000007800000078000000F0000001F800007FFFE0001C1F7A9E21>IIII< 03FFC0FFC0007F007E00003E003800001E003000001E002000000F004000000F00800000 0F81000000078200000007C600000003C400000003E800000001F000000001F000000000 F000000000F800000000F8000000017C000000023C000000043C0000000C1E000000081E 000000101F000000200F000000400F800000C0078000008007C000010003C000070003E0 001F8007E000FFE01FFE00221F7F9E22>II E /Fh 15 117 df<020408103020604040C0C0C0C0C0C0C0C0404060203010 080402071A7F920C>40 D<8040201018080C0404060606060606060604040C0818102040 80071A7E920C>I<1F00318060C04040C060C060C060C060C060C060C060C060404060C0 31801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C00 0C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006 000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F000080 00400060C060C060804060801F000B107F8F0F>I<0300030007000F000B001300330023 004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C00200020002000 2F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C060 00C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC0 8080808001000200040004000C0008000800180018001800180018000B117E900F>I<1F 00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F >I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B10 7F8F0F>I<03E0000C1800100400200200600300400100C00180C00180C00180C00180C0 01806003006003003006003006000C180003E00011117E9017>79 D<1F8030C06000C000C000C000C000C000604030801F000A0B7F8A0E>99 D<10103030FE3030303030323232321C070F7F8E0C>116 D E /Fi 3 64 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF8 0FF003C010127D9317>15 D62 D<00040000000C0000000C0000 000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000 000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000 000C0000000C0000000C0000000C0000000C0000FFFFFFE0FFFFFFE01B1C7C9B23>I E /Fj 60 123 df<00F8F001F8F003F8F00780F00700F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F0FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F01420809F 18>13 D<007000E001C00380078007000E001E001E003C003C003C007800780078007800 7000F000F000F000F000F000F000F000F000F000F000F000F00070007800780078007800 3C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DII<03F000 07F8001FFE001E1E003C0F00380700780780780780700380F003C0F003C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0780780780780 7807803807003C0F001E1E001FFE000FFC0003F00012207E9E17>48 D<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17>I58 D61 D<001F0000001F0000003F8000003B8000003B8000007BC0000073 C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0F00003C0780003C0 78000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F001E00 0F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B207F9F1E> 65 DI<001F C000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C00007800007800007800 00F00000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00 003C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>IIII<001FE000FFF801FFFE03E03E07800E0F 00001E00003E00003C00007C0000780000780000780000F00000F00000F00000F00000F0 0000F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07 801E03E03E01FFFE00FFF8001FC017227DA01E>III75 DI II<003F0000 00FFC00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F0078000780 7800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F80007C07800078078000780780007803C000F003C000F001E001E00 1F003E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC00003FF F00007E1F8000F807C001F003E001E001E003C000F003C000F0078000780780007807800 0780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE000F83 FC0007E1F80003FFF00000FFF800003F780000007C0000003E0000001E0000001F000000 0F801A277DA021>II<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800 007C00003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001 E00001E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F8001322 7EA019>IIIII<780007807C000F003E001F001E001E000F003C000F807C0007807800 03C0F00003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000 003F0000007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C00 0F003E001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>III<07E03FF87FFC701E401F000F000F00 0F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000 F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F8 0014147F9317>II<03C7800FF780 1FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F80780 7807807C0F803E1F801FFF800FF78007C780000780000780000780000780000780000780 000780000780000780111D7E9317>II<07F01FFC3FFC780C78 00780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311 >I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E00 1E001E001E001E001E001E201FF00FF007C00C1A7F9910>II< F003C0F003C07803807807807807803C0F003C0F003C0F001E0E001E1E001E1E000F1C00 0F3C000F3C0007380007380007B80003F00003F00001E00012147F9315>II<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F8 0000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414 809315>II<7FFF7FFF7FFF003E00 3C007800F800F001E003E007C007800F001F001E003C007C00FFFFFFFFFFFF10147F9314 >I E /Fk 73 126 df<030600078F00078F00078F00078F00078F00078F007FFFC0FFFF E0FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFF C01E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>35 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3 800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E078 7070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F001E001C 00380038007000700070007000E000E000E000E000E000E000E000E00070007000700070 00380038001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070007000 70007000700070007000E000E000E000E001C001C0038007800F001E003C007800F00060 000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007 F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<3078FCFC78300606778518>46 D<000300000780000780000F80000F00001F00001E00001E00003E00003C00007C000078 0000780000F80000F00001F00001E00003E00003C00003C00007C0000780000F80000F00 000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F000006000 0011247D9F18>I<01800380038007800F803F80FF80FB80438003800380038003800380 038003800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>49 D<03F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001 C00001C00003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00 E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C078 01C03001C00001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C000 00E00000E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B 18>I<001F00003F0000770000770000E70001E70001C7000387000787000707000E0700 1E07003C0700380700780700F00700FFFFF8FFFFF8FFFFF8000700000700000700000700 000700000700007FF000FFF8007FF0151C7F9B18>I<03F8000FFE001FFF003E0F803803 807001C07001C07001C07001C03803803C07801FFF0007FC000FFE001F1F003C07807001 C0F001E0E000E0E000E0E000E0E000E07001C07803C03E0F801FFF000FFE0003F800131C 7E9B18>56 D<3078FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<7FFF C0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C 7E9318>61 D<0FF0003FFC007FFF00700F00F00380F00380600780000F00003E00007C00 01F00001E00003C00003C00003C00003C00003C000038000000000000000000000000000 00000003800007C00007C00007C000038000111C7D9B18>63 D<00700000F80000F80000 D80000D80001DC0001DC0001DC00018C00038E00038E00038E00038E0003060007070007 07000707000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F 07F0151C7F9B18>65 DI<00F8E003FEE007 FFE00F07E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E0 0000E00000E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003 FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E0 1C00E01C00F01C00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E0 1C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C0 03FDC00FFFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000 E00000E00000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC0 0FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01 C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01 C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7F FF00FFFF807FFF0001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C0007FFF00FFFF807FFF00111C7D9B18>I<7F07F0FF87F87F07F01C03C01C07801C0700 1C0E001C1E001C3C001C38001C70001CF0001DF0001DF0001FB8001FB8001F1C001E1C00 1C0E001C0E001C07001C07001C03801C03801C01C07F03F0FF87F87F03F0151C7F9B18> 75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00700E00700E0070 0E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01C E1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C 0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F00 700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800 111C7D9B18>II<0FF8003FFE007FFF0078 0F00700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000F FC00001C00001E00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF00 1C0F801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE00 1C0F001C07001C03801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F8 7F00F0161C7F9B18>I<03F3801FFF803FFF807C0F80700780E00380E00380E00380E000 007000007800003F00001FF00007FE0000FF00000F800003C00001C00000E00000E06000 E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FF FFF8E07038E07038E07038E0703800700000700000700000700000700000700000700000 700000700000700000700000700000700000700000700000700000700000700007FF0007 FF0007FF00151C7F9B18>IIII<7F8FE0 7F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F80001F80000F00000F000 00700000F00000F80001F80001DC00039E00038E00070F000707000E07800E03801E03C0 7F07F0FF8FF87F07F0151C7F9B18>II<3F FFE07FFFE07FFFE07001C07003C0700780700700000F00001E00001C00003C0000780000 700000F00001E00001C00003C0000780000700000F00001E00E01C00E03C00E07800E070 00E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E00 00070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F00F00781F00 3FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E 00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E 00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C809B18>I< 01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E00000E00000 E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80003F 80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80380F807007 80700380E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF F00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F00380780700380700380E0 01C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF8007FF0001 FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001C0007FFFC0 FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFF F81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC001FF80039E0 003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E000387800 F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00000E00000E 00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B 18>I<03800007C00007C00007C0000380000000000000000000000000007FC000FFC000 7FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I<0038007C007C007C003800 000000000000000FFC1FFC0FFC001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C001C001C6038F078FFF07FE03F800E277E9C18 >II<7FE000FFE0007FE00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E 9B18>I<7CE0E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 7F1F1F00FFBFBF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3 FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E0 00E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001 F00013147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E0038 0E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E0000 0E00000E00000E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<01E3 8007FB801FFF803E1F80380F80700780700780E00380E00380E00380E00380E00380E003 80700780700780380F803C1F801FFF800FFB8003E3800003800003800003800003800003 80000380000380003FF8003FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F87803 F03003E00003C00003C00003800003800003800003800003800003800003800003800003 80007FFE00FFFF007FFE0015147F9318>I<07F7003FFF007FFF00780F00E00700E00700 E007007C00007FE0001FFC0003FE00001F00600780E00380E00380F00380F80F00FFFF00 FFFC00E7F00011147D9318>I<0180000380000380000380000380007FFFC0FFFFC0FFFF C00380000380000380000380000380000380000380000380000380000380400380E00380 E00380E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F 03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E0380 0E0380070700070700070700038E00038E00038E00038E0001DC0001DC0001DC0000F800 00F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F80000F80000 700000F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF87F8FF015147F93 18>I<7F8FF0FF8FF87F8FF00E01C00E03800E0380070380070700070700038700038600 038E0001CE0001CE0000CC0000CC0000DC00007800007800007800007000007000007000 00F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<3FFFF07FFFF07FFF F07001E07003C0700780000F00001E00003C0000F80001F00003C0000780000F00701E00 703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I<0007E0001FE0007FE000780000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00001E0007F C000FF8000FF80007FC00001E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E000007800007FE0001FE00007E013247E9F18>I<7C0000FF0000 FFC00003C00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000F000007FC0003FE0003FE0007FC000F00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00003C000FFC000FF00007C000013247E9F18> 125 D E /Fl 70 123 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E 01C0000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E 01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E 01C0000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C180030180 0703C00E03C00E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C07F87F8151D809C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<60 60F0F0F8F86868080808080808101010102020404080800D0C7F9C15>34 D<00E0000001900000030800000308000007080000070800000708000007080000071000 0007100000072000000740000003C03FE003800F00038006000380040005C0040009C008 0010E0100030E010006070200060702000E0384000E03C4000E01C8000E00F0020E00700 20700780403009C0401830E18007C03E001B1F7E9D20>38 D<004000800100020006000C 000C0018001800300030007000600060006000E000E000E000E000E000E000E000E000E0 00E000E000E000600060006000700030003000180018000C000C00060002000100008000 400A2A7D9E10>40 D<800040002000100018000C000C0006000600030003000380018001 80018001C001C001C001C001C001C001C001C001C001C001C001C0018001800180038003 000300060006000C000C00180010002000400080000A2A7E9E10>I<60F0F07010101010 20204080040C7C830C>44 DI<60F0F06004047C830C>I<03C00C 301818300C300C700E60066006E007E007E007E007E007E007E007E007E007E007E007E0 07E00760066006700E300C300C18180C3007E0101D7E9B15>48 D<030007003F00C70007 000700070007000700070007000700070007000700070007000700070007000700070007 000700070007000F80FFF80D1C7C9B15>I<07C01830201C400C400EF00FF80FF807F807 7007000F000E000E001C001C00380070006000C00180030006010C01180110023FFE7FFE FFFE101C7E9B15>I<07E01830201C201C781E780E781E381E001C001C00180030006007 E00030001C001C000E000F000F700FF80FF80FF80FF00E401C201C183007E0101D7E9B15 >I<000C00000C00001C00003C00003C00005C0000DC00009C00011C00031C00021C0004 1C000C1C00081C00101C00301C00201C00401C00C01C00FFFFC0001C00001C00001C0000 1C00001C00001C00001C0001FFC0121C7F9B15>I<300C3FF83FF03FC020002000200020 002000200023E024302818301C200E000E000F000F000F600FF00FF00FF00F800E401E40 1C2038187007C0101D7E9B15>I<00F0030C06040C0E181E301E300C700070006000E3E0 E430E818F00CF00EE006E007E007E007E007E007600760077006300E300C18180C3003E0 101D7E9B15>I<4000007FFF807FFF007FFF004002008004008004008008000010000010 0000200000600000400000C00000C00001C0000180000180000380000380000380000380 00078000078000078000078000078000078000030000111D7E9B15>I<03E00C30100820 0C20066006600660067006780C3E083FB01FE007F007F818FC307E601E600FC007C003C0 03C003C00360026004300C1C1007E0101D7E9B15>I<03C00C301818300C700C600EE006 E006E007E007E007E007E0076007700F300F18170C2707C700060006000E300C780C7818 7010203030C00F80101D7E9B15>I<60F0F0600000000000000000000060F0F06004127C 910C>I<60F0F0600000000000000000000060F0F0701010101020204080041A7C910C>I< 7FFFFFC0FFFFFFE000000000000000000000000000000000000000000000000000000000 00000000FFFFFFE07FFFFFC01B0C7E8F20>61 D<0FE03038401CE00EF00EF00EF00E000C 001C0030006000C000800180010001000100010001000100000000000000000000000300 0780078003000F1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F 00000017800000178000001780000023C0000023C0000023C0000041E0000041E0000041 E0000080F0000080F0000180F8000100780001FFF80003007C0002003C0002003C000600 3E0004001E0004001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E061800180198007000780 0E0003801C0003801C00018038000180780000807800008070000080F0000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000700000807800008078000080 380000801C0001001C0001000E000200070004000180080000E03000001FC000191E7E9C 1E>IIII<001F808000E0618001801980070007800E000380 1C0003801C00018038000180780000807800008070000080F0000000F0000000F0000000 F0000000F0000000F0000000F000FFF0F0000F8070000780780007807800078038000780 1C0007801C0007800E00078007000B800180118000E06080001F80001C1E7E9C21>II I<1FFF00F800780078007800780078007800780078007800780078007800780078007800 780078007800787078F878F878F878F0F040E021C01F00101D7F9B15>I76 DII<003F800000E0E000 0380380007001C000E000E001C0007003C00078038000380780003C0780003C0700001C0 F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0 780003C0780003C0380003803C0007801C0007000E000E0007001C000380380000E0E000 003F80001B1E7E9C20>II82 D<07E0801C1980300580700380600180E00180E00080E00080E000 80F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007C00003C00001 C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E7E9C17>I<7F FFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F0020800F002000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F800003 FFFC001B1C7F9B1E>II<0808101020204040404080 8080808080B0B0F8F8787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00 001C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E80 1F870012127E9115>97 DI<07E00C 301878307870306000E000E000E000E000E000E00060007004300418080C3007C00E127E 9112>I<003F000007000007000007000007000007000007000007000007000007000007 0003E7000C1700180F00300700700700600700E00700E00700E00700E00700E00700E007 00600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E 6006E006FFFEE000E000E000E00060007002300218040C1803E00F127F9112>I<00F801 8C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C 3880181800381C00381C00381C00381C001818001C38000C300013C00010000030000018 00001FF8001FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007 F800121C7F9215>II<18003C003C 0018000000000000000000000000000000FC001C001C001C001C001C001C001C001C001C 001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C000000000 000000000000000000000FE000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>III II<03F0000E1C0018060030 0300700380600180E001C0E001C0E001C0E001C0E001C0E001C060018070038030030018 06000E1C0003F00012127F9115>II<03C1000C3300180B 00300F00700700700700E00700E00700E00700E00700E00700E00700600700700700300F 00180F000C370007C700000700000700000700000700000700000700000700003FE0131A 7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000F88038 8018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C001C003C00FF E01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C100E2003C00C 1A7F9910>IIII<7F8FF00F03800F0300 07020003840001C80001D80000F00000700000780000F800009C00010E00020E00060700 0403801E07C0FF0FF81512809116>II<7FFC7038603840 7040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E127F9112>I E /Fm 17 119 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D<00038000000380000007C0000007C0000007C000000FE000000FE000001FF000001BF0 00001BF0000031F8000031F8000061FC000060FC0000E0FE0000C07E0000C07E0001803F 0001FFFF0003FFFF8003001F8003001F8006000FC006000FC00E000FE00C0007E0FFC07F FEFFC07FFE1F1C7E9B24>65 D<001FE02000FFF8E003F80FE007C003E00F8001E01F0000 E03E0000E03E0000607E0000607C000060FC000000FC000000FC000000FC000000FC0000 00FC000000FC000000FC0000007C0000607E0000603E0000603E0000C01F0000C00F8001 8007C0030003F80E0000FFFC00001FE0001B1C7D9B22>67 DI< 07F8201FFEE03C07E07801E07000E0F000E0F00060F00060F80000FE0000FFE0007FFE00 3FFF003FFF800FFFC007FFE0007FE00003F00001F00000F0C000F0C000F0C000E0E000E0 F001C0FC03C0EFFF0083FC00141C7D9B1B>83 D<0FF8001C1E003E0F803E07803E07C01C 07C00007C0007FC007E7C01F07C03C07C07C07C0F807C0F807C0F807C0780BC03E13F80F E1F815127F9117>97 D<03FC000E0E001C1F003C1F00781F00780E00F80000F80000F800 00F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115>99 D<01FC000F07001C03803C01C07801C07801E0F801E0F801E0FFFFE0F80000F80000F800 007800007C00603C00601E00C00F038001FC0013127F9116>101 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F001F 001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D108 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I114 D<1FD830786018E018E018F000FF807FE07FF01F F807FC007CC01CC01CE01CE018F830CFC00E127E9113>I<030003000300030007000700 0F000F003FFCFFFC1F001F001F001F001F001F001F001F001F001F0C1F0C1F0C1F0C0F08 079803F00E1A7F9913>II< FFC1FCFFC1FC1F00601F80E00F80C00FC0C007C18007C18003E30003E30001F60001F600 01FE0000FC0000FC0000780000780000300016127F9119>I E /Fn 53 123 df<00000F80003F8F80007F8F8000FF8F8001FF8F8003E0000003C0000007C000 0007C0000007C0000007C0000007C0000007C0000007C0000007C0000007C00000FFFF8F 80FFFF8F80FFFF8F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F 8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F 8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F80192B7F AA20>12 D45 DI<007F000001 FFC00007FFF0000FFFF8000FC1F8001F007C003F007E003E003E003C001E007C001F007C 001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F8078 000F007C001F007C001F007C001F003E003E003E003E003F007E001F80FC000FC1F8000F FFF80007FFF00001FFC000007F000019297EA71E>48 D<00180000380000F80007F800FF F800FFF800F8F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF07FFFF014 287BA71E>I<00FE0003FFC007FFE00FFFF01F03F83C00FC38007E78003E70003EF0001F F0001F60001F20001F00001F00001F00001F00003E00003E00007C00007C0000F80001F0 0001E00003C0000780000F00001E00003C0000780000F00001E00003C0000780000F0000 1E00003C00007FFFFF7FFFFF7FFFFF7FFFFF18287EA71E>I<007F000001FFC00007FFF0 000FFFF8001FC1F8003E007C003C003E0078003E0038003E0010003E0000003E0000003E 0000003C0000007C000000FC000001F8000007F00000FFE00000FFC00000FFE00000FFF0 000001FC0000007C0000003E0000001F0000001F0000000F8000000F8000000F8000000F 8000000F8040000F8060001F00F0001F00F8003F007E007E003F81FC001FFFF8000FFFF0 0003FFE000007F000019297EA71E>I<0003F0000007F0000005F000000DF000000DF000 001DF0000039F0000039F0000079F0000079F00000F1F00000F1F00001E1F00003E1F000 03E1F00007C1F00007C1F0000F81F0000F81F0001F01F0001F01F0003E01F0007C01F000 7C01F000F801F000FFFFFF80FFFFFF80FFFFFF80FFFFFF800001F0000001F0000001F000 0001F0000001F0000001F0000001F0000001F0000001F0000001F00019277EA61E>I<3F FFFC3FFFFC3FFFFC3FFFFC3E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00003E3F003EFFC03FFFE03FFFF03FE1F83F807C3F003E3E003E00003E00001F00 001F00001F00001F00001F00001F00001F20001F60003E70003EF8007C7C00FC3F03F81F FFF00FFFE007FF8000FE0018287EA61E>I<000FF000003FFC0000FFFC0001FFFC0003F8 0C0007E000000FC000000F8000001F0000001E0000003E0000003C0000007C0000007C00 00007C3FE000F8FFF000F9FFF800FBFFFC00FF807E00FF003E00FE003F00FC001F00FC00 1F00FC000F80F8000F80F8000F80F8000F80F8000F8078000F807C000F807C000F807C00 0F003E001F003E001F001F003E001F807C000FC1FC0007FFF80003FFF00001FFC000007F 000019297EA71E>II<007F000001FFC000 07FFF0000FFFF8001FC1FC003F007E003E003E007E003F007C001F007C001F007C001F00 7C001F007C001F003E003E003E003E001F007C000FC1F80007FFF00003FFE00003FFE000 0FFFF8001FC1FC003F007E003E003E007C001F007C001F00F8000F80F8000F80F8000F80 F8000F80F8000F80F8000F807C001F007C001F007E003F003F007E001FC1FC000FFFF800 07FFF00003FFE000007F000019297EA71E>I<007F000001FFC00003FFE0000FFFF0000F C1F8001F007C003E007C007C003E007C001E007C001F00F8001F00F8001F00F8000F00F8 000F80F8000F80F8000F80F8000F80F8001F807C001F807C001F807E003F803E007F803F 00FF801FFFEF800FFFCF8007FF8F8003FE1F0000001F0000001F0000001E0000003E0000 003E0000007C0000007C000000F8001801F0001E07E0003FFFC0001FFF80000FFE000003 F8000019297EA71E>I<0001F000000003F800000003F800000007FC00000007BC000000 07BC0000000F3E0000000F1E0000000F1E0000001F1F0000001E1F0000001E0F0000003E 0F8000003C0F8000003C078000007C07C000007807C00000F803E00000F803E00000F003 E00001F001F00001F001F00001E001F00003E000F80003E000F80003C000F80007FFFFFC 0007FFFFFC000FFFFFFE000F80003E000F80003E001F00003F001F00001F001E00001F00 3E00000F803E00000F803C00000F807C000007C07C000007C078000007C0F8000003E0F8 000003E0232A7EA928>65 D<0001FF00000FFFE0003FFFF8007FFFF800FE01F801F80030 03F0001007C000000F8000001F8000001F0000003E0000003E0000007E0000007C000000 7C0000007C000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000F8000000F80000007C0000007C0000007C0000007E0000003E0000003E000000 1F0000001F8000000F80000007C0000003F0000401F8001C00FE00FC007FFFFC003FFFF8 000FFFE00001FF001E2C7CAA26>67 DIII72 DI75 D77 DI<0001FC0000000FFF8000003FFFE000 007FFFF00001FE03FC0003F800FE0007E0003F0007C0001F000F80000F801F000007C01F 000007C03E000003E03E000003E07C000001F07C000001F07C000001F078000000F0F800 0000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F80000 00F8F8000000F8F8000000F87C000001F07C000001F07C000001F07E000003F03E000003 E03F000007E01F000007C01F80000FC00FC0001F8007E0003F0007F0007F0003F800FE00 01FE03FC0000FFFFF800003FFFE000000FFF80000001FC0000252C7DAA2C>II82 D<007FC00001FFF80007FFFE000F FFFF001FC07F003F000F007E0006007C0000007C000000F8000000F8000000F8000000F8 000000F8000000FC0000007E0000007F0000003F8000001FF800000FFF800007FFE00003 FFF80000FFFC00000FFE000000FF0000003F0000001F8000000F8000000FC0000007C000 0007C0000007C0000007C0000007C0000007C000000F8060000F80F0001F00FC003F00FF 80FE007FFFFC001FFFF80007FFE00000FF80001A2C7DAA21>II86 DI<01FE000FFF803FFFC03FFFE03C03F03001F00001F800 00F80000F80000F80000F80000F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F8 00F8F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1D>97 DI<007FC001FFF007FFFC0F FFFC1FC07C1F00083E00007C00007C00007C0000F80000F80000F80000F80000F80000F8 0000F800007C00007C00007E00003E00001F000C1FC07C0FFFFC07FFFC01FFF0007F8016 1B7E9A1B>I<00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E00FC3E03FF3E07FFFE0FFFFE1FC1FE3F007E3E003E 7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E 7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03FF3E00FC3E172A7EA91F>I<007E0003FF 8007FFC00FFFE01F83F03F00F03E00787C00787C003878003CFFFFFCFFFFFCFFFFFCFFFF FCF80000F80000F800007800007C00007C00003E00003F000C1FC07C0FFFFC07FFFC01FF F0007F80161B7E9A1B>I<001FC0007FC000FFC001FFC003F00003E00007C00007C00007 C00007C00007C00007C00007C00007C00007C000FFFE00FFFE00FFFE0007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C000122A7FA912>I< 00F8078003FE7FC00FFFFFC01FFFFFC01F07C0003E03E0003E03E0007C01F0007C01F000 7C01F0007C01F0007C01F0007C01F0003E03E0003E03E0001F07C0001FFFC0003FFF8000 3BFE000038F8000078000000780000003C0000003FFFC0003FFFF8001FFFFC001FFFFE00 3FFFFF007C007F00F8001F80F8000F80F8000F80F8000F80FC001F807E003F003F80FE00 3FFFFE000FFFF80007FFF00000FF80001A287E9A1E>III108 DII<007F000001FFC00007FFF0000FFFF8001FC1 FC003F007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F800 0F80F8000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001FC1 FC000FFFF80007FFF00001FFC000007F0000191B7E9A1E>II114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8 003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFF C07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007 C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003 FFE003FFE001FF8000FC0013227FA116>IIII<7C000FC03E001F803F001F001F803E000F807C0007C0FC0003E0F80001F1F00001 FBE00000FFC000007FC000003F8000001F0000001F0000003F8000007FC00000FBC00000 F3E00001F1F00003E0F80007C07C000F807C000F803E001F001F003E000F807E000FC0FC 0007E01B1B809A1C>III E /Fo 15 122 df<07E00FF03FFC3FFC7FFEFFFFFFFFFFFFFFFFFF FFFFFF7FFE3FFC3FFC0FF007E01010788F21>46 D<000001E00000000003F0000000000F F0000000003FF000000000FFF00000000FFFF0000003FFFFF00000FFFFFFF00000FFFFFF F00000FFFFFFF00000FFF0FFF00000FC00FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000003FFFFFFFFFC03FFFFFFFFFC03FFFFFFFFFC03FFFFFFFFFC03FFFFFFFFFC02A4478 C33B>49 D<0000FFE00000000FFFFE0000003FFFFFC00000FFFFFFF00003FFFFFFFC0007 FC01FFFE000FE0007FFF001FC0001FFF803F80000FFFC03FE00007FFE07FF80007FFE07F FC0003FFF0FFFC0003FFF0FFFE0001FFF8FFFE0001FFF8FFFE0001FFF8FFFE0000FFFCFF FE0000FFFC7FFC0000FFFC7FFC0000FFFC3FF80000FFFC1FF00000FFFC0FE00000FFFC00 000001FFFC00000001FFF800000001FFF800000001FFF000000003FFF000000003FFE000 000003FFE000000007FFC000000007FF800000000FFF800000001FFF000000001FFE0000 00003FFC000000007FF8000000007FE000000000FFC000000001FF8000000003FF000000 0007FC000000000FF8000000001FF0000000003FC0000000007F80007C0000FF00007C00 00FE00007C0001FC00007C0003F00000F80007E00000F8000FC00000F8001F800000F800 3F000001F8007C000003F800FFFFFFFFF801FFFFFFFFF003FFFFFFFFF007FFFFFFFFF00F FFFFFFFFF01FFFFFFFFFF03FFFFFFFFFF07FFFFFFFFFF07FFFFFFFFFF0FFFFFFFFFFE0FF FFFFFFFFE0FFFFFFFFFFE0FFFFFFFFFFE02E447AC33B>I77 D<0007FFFC000000007FFFFFC0000001FFFF FFF8000003FFFFFFFE000007FE001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000 FFF0000FFF80007FF0000FFF80007FF8000FFF80007FF80007FF00003FFC0007FF00003F FC0003FE00003FFC0000F800003FFC00000000003FFC00000000003FFC00000000003FFC 00000000003FFC00000007FFFFFC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC00 00FFFE003FFC0003FFF0003FFC0007FFC0003FFC000FFF00003FFC001FFE00003FFC003F FC00003FFC007FF800003FFC007FF800003FFC00FFF000003FFC00FFF000003FFC00FFF0 00003FFC00FFF000003FFC00FFF000003FFC00FFF000007FFC007FF80000FFFC007FF800 01EFFC003FFC0003EFFC003FFF0007CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC 03FFFC007FFFF001FFFC0003FF80007FF8362E7DAD3A>97 D<00001FFFC0000000FFFFF8 000007FFFFFE00001FFFFFFF80007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FF E007FF0003FFE00FFE0003FFE00FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF 803FFC00003E007FF8000000007FF8000000007FF800000000FFF800000000FFF8000000 00FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF8000000 00FFF800000000FFF8000000007FF8000000007FF8000000007FFC000000003FFC000000 003FFC000000001FFC000000F81FFE000000F80FFE000000F80FFF000001F007FF800003 F003FFC00007E001FFE0000FC000FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8 000000FFFFE00000001FFE00002D2E7CAD35>99 D<00001FFE00000001FFFFE0000007FF FFF800001FFFFFFE00007FFC07FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00 003FF00FFE00001FF01FFE00000FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC00 0007FC7FF8000007FC7FF8000007FE7FF8000007FEFFF8000007FEFFF8000007FEFFFFFF FFFFFEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFFFFFCFFF800000000FFF800000000FFF800 000000FFF8000000007FF8000000007FF8000000007FFC000000003FFC000000003FFC00 0000003FFC0000001C1FFE0000003E0FFE0000003E07FF0000007E07FF000000FC03FF80 0001F801FFC00003F0007FF0001FE0003FFE00FFC0001FFFFFFF800007FFFFFE000000FF FFF80000000FFF80002F2E7DAD36>101 D<000000FFC000000007FFF80000003FFFFC00 0000FFFFFF000001FFC1FF000007FF03FF80000FFC03FF80000FF807FFC0001FF807FFC0 003FF007FFC0003FF007FFC0003FE003FF80007FE003FF80007FE001FF00007FE000FE00 007FE0003800007FE0000000007FE0000000007FE0000000007FE0000000007FE0000000 007FE0000000007FE0000000007FE0000000007FE0000000007FE0000000FFFFFFFE0000 FFFFFFFE0000FFFFFFFE0000FFFFFFFE0000FFFFFFFE0000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF00000003FFFFFF800003FFFFFF80000 3FFFFFF800003FFFFFF800003FFFFFF800002A487DC724>I<00FC0001FE0003FF0007FF 800FFFC01FFFE01FFFE01FFFE01FFFE01FFFE01FFFE00FFFC007FF8003FF0001FE0000FC 00000000000000000000000000000000000000000000000000000000000000000000007F C0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF18497CC820>105 D<007FC000FFFFC000FFFFC000FFFFC000FFFFC000FFFFC00003FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF 8019487CC720>108 D<007FC001FFC00000FFFFC00FFFF80000FFFFC03FFFFE0000FFFF C0FFFFFF0000FFFFC1FC07FF8000FFFFC3E003FFC00003FFC7C001FFC00001FFCF0001FF E00001FFDE0000FFE00001FFDC0000FFE00001FFFC0000FFF00001FFF80000FFF00001FF F00000FFF00001FFF00000FFF00001FFF00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFF FFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B2E7CAD42>110 D<00000FFF0000000000 FFFFF000000007FFFFFE0000001FFFFFFF8000003FFC03FFC00000FFE0007FF00001FF80 001FF80003FF00000FFC0007FE000007FE000FFE000007FF000FFC000003FF001FFC0000 03FF803FFC000003FFC03FF8000001FFC03FF8000001FFC07FF8000001FFE07FF8000001 FFE07FF8000001FFE0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FF F0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0 FFF8000001FFF07FF8000001FFE07FF8000001FFE07FF8000001FFE07FF8000001FFE03F FC000003FFC03FFC000003FFC01FFC000003FF801FFE000007FF800FFE000007FF0007FF 00000FFE0003FF80001FFC0001FFC0003FF80000FFE0007FF000007FFC03FFE000001FFF FFFF80000007FFFFFE00000000FFFFF0000000000FFF000000342E7DAD3B>I<00FF803F 8000FFFF80FFF000FFFF83FFFC00FFFF87FFFE00FFFF8FC3FF00FFFF8F07FF0003FF9E0F FF8001FFBC0FFF8001FFB80FFF8001FFF80FFF8001FFF00FFF8001FFF007FF0001FFF007 FF0001FFE003FE0001FFE000F80001FFE000000001FFE000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC0000000FFFFFFE00000FFFFFFE0 0000FFFFFFE00000FFFFFFE00000FFFFFFE00000292E7CAD31>114 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FFE01FC0001FE03F80000FE0 3F000007E07F000003E07F000003E0FF000003E0FF000003E0FF800003E0FFC0000000FF F0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFFFFFC001FFFFFFF000FFF FFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FFFFF800001FFFF8000000 FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003FCFC000003FCFE000003 F8FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0FFFC00FFC0FFFFFFFF80 FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>I<7FFFFFC000FFFF807FFFFFC000FF FF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF8000FFF000000FE00000FF F800000FC00000FFF800000FC000007FFC00000F8000007FFC00001F8000003FFC00001F 0000003FFE00003F0000001FFE00003E0000001FFF00007E0000000FFF00007C0000000F FF8000FC00000007FF8000F800000007FFC001F800000003FFC001F000000003FFE003F0 00000003FFE003F000000001FFF003E000000001FFF007E000000000FFF007C000000000 FFF80FC0000000007FF80F80000000007FFC1F80000000003FFC1F00000000003FFE3F00 000000001FFE3E00000000001FFF7E00000000000FFF7C00000000000FFFFC0000000000 0FFFFC000000000007FFF8000000000007FFF8000000000003FFF0000000000003FFF000 0000000001FFE0000000000001FFE0000000000000FFC0000000000000FFC00000000000 007F800000000000007F800000000000003F000000000000003F000000000000003F0000 00000000003E000000000000007E000000000000007C00000000000000FC000000001F80 00F8000000003FC001F8000000007FE001F000000000FFF003F000000000FFF003E00000 0000FFF007E000000000FFF00FC000000000FFF01F8000000000FFF03F80000000007FE0 7F00000000007F43FE00000000003FFFF800000000001FFFF0000000000007FFC0000000 000001FE00000000000039427EAD3F>121 D E /Fp 8 117 df<0003FF8000001FFFF000 007FFFFE0000FE03FF0001F000FF8003C000FFC00780007FE00FF0007FF00FF8007FF01F FC007FF81FFE007FF81FFE007FF81FFE007FF81FFE007FF81FFE007FF80FFC007FF007F8 007FF003F0007FF0000000FFE0000000FFC0000001FF80000001FF00000003FE00000007 FC0000001FF000000FFFC000000FFF8000000FFFF800000003FE00000000FF800000007F E00000003FF00000003FF80000003FFC0000001FFC0000001FFE0000001FFE0200001FFF 1FC0001FFF3FE0001FFF7FF0001FFF7FF0001FFFFFF8001FFFFFF8001FFFFFF8001FFEFF F8001FFEFFF0001FFE7FF0003FFC7FE0003FFC3FC0003FF81F80007FF01FE000FFE007FC 03FFC003FFFFFF0001FFFFFE00003FFFF0000007FF800028397CB731>51 D<0000001FFF000030000001FFFFE000F000000FFFFFFC01F000007FFFFFFE03F00001FF FE007F87F00003FFE0000FCFF0000FFF000003FFF0001FFC000001FFF0003FF80000007F F0007FF00000003FF000FFC00000003FF001FFC00000001FF003FF800000000FF007FF00 0000000FF00FFF0000000007F00FFE0000000007F01FFE0000000003F01FFE0000000003 F03FFC0000000003F03FFC0000000001F03FFC0000000001F07FFC0000000001F07FF800 00000001F07FF80000000000007FF8000000000000FFF8000000000000FFF80000000000 00FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800 0000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF80000000000 007FF80000000000007FF80000000000007FF80000000000007FFC0000000000F03FFC00 00000000F03FFC0000000000F03FFC0000000000F01FFE0000000000F01FFE0000000001 E00FFE0000000001E00FFF0000000001E007FF0000000003C003FF8000000003C001FFC0 000000078000FFE00000000F00007FF00000001F00003FF80000003E00001FFC0000007C 00000FFF000001F8000003FFE00007F0000001FFFE003FC00000007FFFFFFF000000000F FFFFFC0000000001FFFFF000000000001FFF0000003C3D7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF00001FFC00FF80 001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF00007F0003FF0 0001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000000FFFFFF0 00007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF0003FE0003FF0 007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000FFC0003FF0 00FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF007FFFE0FFF F001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFFFE00 00FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03FF00007F83F F00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFFFFFFFCFFFF FFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE00000007FE000 00007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF800007807FC0000 F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE00026267DA52D >101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF0000000007FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF007FC00003FF 01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF6000FF8003FF E000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC3FFFFFFFFF FC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00FFFFFE03FF 00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF803FF00001F FC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF03FF000007 FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007 FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC03FF00000F FC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC003FFF001FF 8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000FFFFFC0000 00FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fq 83 124 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<001F81FF0000F06F070001C07C0F000380F80F000300 F00700070070070007007007000700700700070070070007007007000700700700070070 0700FFFFFFFF000700700700070070070007007007000700700700070070070007007007 000700700700070070070007007007000700700700070070070007007007000700700700 070070070007007007000700700700070070070007007007007FE3FE3FF02420809F26> I<70F8F8F8F8F8F8F8707070707070707070702020202020000000000070F8F8F8700521 7CA00D>33 D<7038F87CFC7EFC7E743A0402040204020804080410081008201040200F0E 7E9F17>I<00780000008400000184000003020000070200000702000007020000070200 00070400000704000007080000070800000310000003A00FFC03C003E0038001C001C000 8001C0010003E0010004E0020008F00200187004003078080070380800701C1000F01E10 00F00E2000F0074000F003C0087003C0087801C010380670301C18386007E00F801E227E A023>38 D<70F8FCFC74040404080810102040060E7C9F0D>I<00200040008001000200 06000C000C00180018003000300030007000600060006000E000E000E000E000E000E000 E000E000E000E000E000E0006000600060007000300030003000180018000C000C000600 020001000080004000200B2E7DA112>I<800040002000100008000C0006000600030003 0001800180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000 E000E000C000C000C001C001800180018003000300060006000C00080010002000400080 000B2E7DA112>I<01800180018001800180C183F18F399C0FF003C003C00FF0399CF18F C1830180018001800180018010147DA117>I<0006000000060000000600000006000000 060000000600000006000000060000000600000006000000060000000600000006000000 06000000060000FFFFFFF0FFFFFFF0000600000006000000060000000600000006000000 060000000600000006000000060000000600000006000000060000000600000006000000 0600001C207D9A23>I<70F8FCFC74040404080810102040060E7C840D>II<70F8F8F87005057C840D>I<000100030003000600060006000C000C000C0018 0018001800300030003000600060006000C000C000C00180018001800300030003000600 060006000C000C000C00180018001800300030003000600060006000C000C000C000102D 7DA117>I<03F0000E1C001C0E00180600380700700380700380700380700380F003C0F0 03C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C070 03807003807003807807803807001806001C0E000E1C0003F000121F7E9D17>I<018003 800F80F38003800380038003800380038003800380038003800380038003800380038003 800380038003800380038003800380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E 00200700400780800780F007C0F803C0F803C0F803C02007C00007C0000780000780000F 00000E00001C0000380000700000600000C0000180000300000600400C00401800401000 803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E00200F00780F8078078078 0780380F80000F80000F00000F00000E00001C0000380003F000003C00000E00000F0000 07800007800007C02007C0F807C0F807C0F807C0F00780400780400F00200E001C3C0003 F000121F7E9D17>I<000600000600000E00000E00001E00002E00002E00004E00008E00 008E00010E00020E00020E00040E00080E00080E00100E00200E00200E00400E00C00E00 FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0141E7F9D17>I<1803 001FFE001FFC001FF8001FE00010000010000010000010000010000010000011F000161C 00180E001007001007800003800003800003C00003C00003C07003C0F003C0F003C0E003 80400380400700200600100E000C380003E000121F7E9D17>I<007C000182000701000E 03800C07801C0780380300380000780000700000700000F1F000F21C00F40600F80700F8 0380F80380F003C0F003C0F003C0F003C0F003C07003C07003C070038038038038070018 07000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF807FFF80400100800200 80020080040000080000080000100000200000200000400000400000C00000C00001C000 018000038000038000038000038000078000078000078000078000078000078000078000 030000121F7D9D17>I<03F0000C0C001006003003002001806001806001806001807001 807803003E03003F06001FC8000FF00003F80007FC000C7E00103F00300F806003804001 C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C0003F000121F7E9D17 >I<03F0000E18001C0C00380600380700700700700380F00380F00380F003C0F003C0F0 03C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C000038000038000038000 0700300700780600780E00700C002018001070000FC000121F7E9D17>I<70F8F8F87000 00000000000000000070F8F8F87005147C930D>I<70F8F8F87000000000000000000000 70F0F8F878080808101010202040051D7C930D>I<7FFFFFE0FFFFFFF000000000000000 00000000000000000000000000000000000000000000000000FFFFFFF07FFFFFE01C0C7D 9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000C001800180010003 000200020002000200020002000000000000000000000007000F800F800F8007000E207D 9F15>63 D<000100000003800000038000000380000007C0000007C0000007C0000009E0 000009E0000009E0000010F0000010F0000010F00000207800002078000020780000403C 0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F0001000F00020007 800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0 038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C00004078000040 F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 780000007C0000407C0000403C0000401C0000401E0000800E0000800700010003800200 01C0040000703800000FC0001A217D9F21>IIII72 D I<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00203C 00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>I76 DII<001F800000F0 F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E07800 01E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F800 01F0F80001F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F00 0F0007801E0001C0380000F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F000 00F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001 C08001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19 >I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F 0010800F0010000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0 070003C0020001E0040001F00C0000F0080000781000007C1000003C2000003E4000001E 4000000F8000000F8000000780000003C0000007E0000005E0000009F0000018F8000010 780000207C0000603C0000401E0000801F0001800F0001000780020007C0070003C01F80 07E0FFE01FFE1F1F7F9E22>II< FEFEC0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0 C0C0C0C0C0C0C0FEFE072D7CA10D>91 D<080410082010201040204020804080408040B8 5CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE00030 3000781800781C00300E00000E00000E00000E0000FE00078E001E0E00380E00780E00F0 0E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>II E /Fr 46 122 df<00200040008001000300060004000C000C0018001800 3000300030007000600060006000E000E000E000E000E000E000E000E000E000E000E000 E000E000E0006000600060007000300030003000180018000C000C000400060003000100 0080004000200B327CA413>40 D<800040002000100018000C0004000600060003000300 01800180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000C000C000C001C0018001800180030003000600060004000C0018001000 2000400080000B327DA413>I<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<008003800F80F3800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 8003800380038003800380038007C0FFFE0F217CA018>49 D<03F0000C1C001007002007 804003C04003C08003E0F003E0F801E0F801E0F801E02003E00003E00003C00003C00007 80000700000E00001C0000180000300000600000C0000180000100000200200400200800 201800603000403FFFC07FFFC0FFFFC013217EA018>I<000200000600000E00000E0000 1E00001E00002E00004E00004E00008E00008E00010E00020E00020E00040E00040E0008 0E00100E00100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E0000 0E00000E00000E00001F0001FFF015217FA018>52 D<1000801E07001FFF001FFE001FF8 0013E00010000010000010000010000010000010000010F800130E001407001803801003 800001C00001C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C04003 C04003802007001006000C1C0003F00013227EA018>I<007E0001C1000300800601C00E 03C01C03C0180180380000380000780000700000700000F0F800F30C00F40600F40300F8 0380F801C0F001C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C038 01C01803801C03000C0600070C0001F00013227EA018>I<01F800060E00080300100180 2001802000C06000C06000C06000C07000C07801803E01003F02001FC4000FF80003F800 03FC00067F00083F80100F803007C06001C06000E0C000E0C00060C00060C00060C00060 6000406000C03000801803000E0E0003F00013227EA018>56 D<01F000060C000C060018 0700380380700380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E070 01E07003E03803E01805E00C05E00619E003E1E00001C00001C00001C000038000038030 0300780700780600700C002018001030000FC00013227EA018>I<000180000001800000 0180000003C0000003C0000003C0000005E0000005E000000DF0000008F0000008F00000 10F800001078000010780000203C0000203C0000203C0000401E0000401E0000401E0000 800F0000800F0000FFFF000100078001000780030007C0020003C0020003C0040003E004 0001E0040001E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E0000701E0000701C0000 303C0000303C0000307C0000107800001078000010F8000000F8000000F8000000F80000 00F8000000F8000000F8000000F800000078000000780000107C0000103C0000103C0000 101C0000201E0000200E000040070000400380008001C0010000E0020000381C000007E0 001C247DA223>67 DIII72 DI77 DI80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020 E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E0 0001E00000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0 C80180C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008 40078008C007800C80078004800780048007800480078004000780000007800000078000 000780000007800000078000000780000007800000078000000780000007800000078000 000780000007800000078000000780000007800000078000000780000007800000078000 000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700 00070000FF0007C7001E07003C0700780700700700F00708F00708F00708F00F08781708 3C23900FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E 003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F 00700C80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000 700000F00000F00000F00000F00000F00000F00000F000007000007800403800401C0080 0C010007060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800 E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800 E01801E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C078 01E07000E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C 00400E008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700 07000700070007000700FFF8070007000700070007000700070007000700070007000700 07000700070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C 07001C07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010 00001800001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860 00306000303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E0 0F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000 0000000000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E0 01C00000000000000000000000000000000001E00FE001E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0 F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E0200 0E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C0 0E01E00E00F00E00F8FFE3FE17237FA21A>I<0E1FC07F00FE60E183801E807201C00F00 3C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E0FFE3FF8FFE27157F942A>109 D<0E1F80FE60C01E80E00F00 700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E0038 0E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E0038 0F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E0000 0E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E 000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>I119 D121 D E /Fs 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E078 0000E0700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F8 0000007C0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000 FFFF00000FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F800 0000F88000007880000078800000788000007880000078C0000078C0000070E00000F0E0 0000E0F00000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125 >83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E 001E0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F 001E003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478 003E047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E060038018 0700040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000 F80000F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F0002 07000403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E0 000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0 000001E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E0 078005E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0 F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E0 780001E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF 003F81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C 000F003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F8 000000F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E 0000800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007 E0001C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF 007FFF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E 003C001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E 003C000F007800070070000780E00009C1C000087F000018000000180000001800000018 000000180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E030 0000F070000070E0000038E0000038E0000038E0000038E0000038700000707000007038 0000E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F80 0700000000000000000000000000000000000000000000000780FF80FF800F8007800780 078007800780078007800780078007800780078007800780078007800780078007800780 07800780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F0 00FF8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E00 07C000F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E000780 00F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Ft 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 947 a Ft(D)26 b(R)g(A)f(F)h(T)225 1038 y Fs(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n (terface)621 1232 y Fr(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum) 766 1359 y(Septem)o(b)q(er)g(19,)h(1996)190 1417 y(This)h(w)o(ork)f(w)o (as)h(supp)q(orted)g(in)f(part)g(b)o(y)g(NSF)g(and)h(ARP)l(A)e(under)h (NSF)g(con)o(tract)283 1475 y(CD)o(A-9115428)j(and)e(Esprit)f(under)h (pro)s(ject)e(HPC)i(Standards)g(\(21111\).)p eop %%Page: 1 2 1 1 bop 166 45 a Fq(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fp(Chapter)34 b(3)75 564 y Fo(Miscellan)m(y)41 b(for)g(1.2)75 805 y Fn(3.1)59 b(V)n(ersion)20 b(Numb)r(er)75 953 y Fm(Curren)o(t)12 b(Status:)k Fl(P)o(assed)d(once.)20 b Fq(In)14 b(order)f(to)f(cop)q(e)i(with)g(c)o(hanges)f(to)f(the)i(MPI) f(Standard,)g(there)g(are)75 1057 y(b)q(oth)j(compile-time)h(and)f (run-time)g(w)o(a)o(ys)e(to)h(determine)h(whic)o(h)g(v)o(ersion)g(of)f (the)h(standard)e(is)i(in)h(use)75 1114 y(in)f(the)f(en)o(vironmen)o(t) h(one)f(is)h(using.)166 1170 y(The)e(\\v)o(ersion")g(will)h(b)q(e)g (represen)o(ted)f(b)o(y)g(t)o(w)o(o)e(separate)i(in)o(tegers,)g(for)f (the)h(v)o(ersion)g(and)g(sub)o(v)o(er-)75 1227 y(sion:)166 1283 y(In)i(C,)170 1367 y Fk(#define)23 b(MPI_VERSION)94 b(1)170 1423 y(#define)23 b(MPI_SUBVERSION)f(2)75 1507 y Fq(in)16 b(F)l(ortran,)170 1591 y Fk(INTEGER)23 b(MPI_VERSION,)g (MPI_SUBVERSION)170 1648 y(PARAMETER)g(\(MPI_VERSION)94 b(=)24 b(1\))170 1704 y(PARAMETER)f(\(MPI_SUBVERSION)f(=)i(2\))75 1788 y Fq(and)15 b(in)h(C++,)170 1872 y Fk(const)24 b(int)f (MPI::VERSION)f(=)i(1)170 1928 y(const)g(int)f(MPI::SUBVERSION)f(=)i(2) 75 2012 y Fq(F)l(or)15 b(run)o(time)g(determination,)75 2163 y Fj(MPI)p 160 2163 14 2 v 16 w(GET)p 264 2163 V 17 w(VERSION\()h(version,)f(subversion)h(\))117 2240 y Fl(OUT)108 b Fj(version)456 b Fl(v)o(ersion)14 b(n)o(um)o(b)q(er)117 2313 y(OUT)108 b Fj(subversion)393 b Fl(sub)o(v)o(ersion)15 b(n)o(um)o(b)q(er)75 2438 y Fk(int)23 b(MPI)p 245 2438 15 2 v 17 w(Get)p 334 2438 V 17 w(version\()g(int)g(*version,)g(int)g (*subversion)g(\))75 2524 y(MPI)p 150 2524 V 17 w(GET)p 239 2524 V 17 w(VERSION\()f(VERSION,)h(SUBVERSION,)f(IERROR)h(\))170 2581 y(INTEGER)g(VERSION,)g(SUBVERSION,)f(IERROR)1875 2617 y Fi(>)16 b Fh(\(Oct\))75 2667 y Fk(static)23 b(int)g(MPI::Get)p 532 2667 V 17 w(version\()f(int&)i(version,)e(int&)i(subversion)e(\)) 1875 2704 y Fi(?)16 b Fh(\(Oct\))964 2828 y Fq(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fq(2)970 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)75 45 y Fn(3.2)59 b(T)-5 b(reatment)18 b(of)i(MPI)p 670 45 18 2 v 21 w(Status)75 147 y Fq(The)25 b(follo)o(wing)g(prop)q(osals)g(add)g(to,)g(but)g(do)g(not)f(c)o (hange,)i(the)f(functionalit)o(y)h(asso)q(ciated)f(with)75 203 y Fj(MPI)p 160 203 14 2 v 16 w(ST)l(A)l(TUS)p Fq(.)75 325 y Ff(3.2.1)49 b(P)o(assing)17 b(MPI)p 481 325 15 2 v 18 w(ST)l(A)l(TUS)p 676 325 V 18 w(IGNORE)g(fo)o(r)f(MPI)p 1044 325 V 18 w(Status)75 458 y Fm(Curren)o(t)e(Status:)j Fl(P)o(assed)e(once.)166 561 y Fq(Ev)o(ery)10 b(call)i(to)e Fj(MPI)p 507 561 14 2 v 16 w(RECV)h Fq(includes)i(a)e Fj(status)h Fq(argumen)o(t,)f(where)g(the)f(system)h(can)f(return)h (details)75 618 y(ab)q(out)18 b(the)g(message)g(receiv)o(ed.)31 b(There)18 b(are)g(also)g(a)g(n)o(um)o(b)q(er)h(of)f(other)g(MPI)g (calls,)i(particularly)f(in)-1991 b Fi(>)15 b Fh(\(Oct\))75 674 y Fq(MPI-2,)20 b(where)f Fj(status)i Fq(is)f(returned.)33 b(An)19 b(ob)s(ject)g(of)g(t)o(yp)q(e)g Fj(MPI)p 1234 674 V 16 w(ST)l(A)l(TUS)h Fq(is)g(not)f(an)g Fj(MPI)g Fq(opaque)-117 676 y Fi(?)c Fh(\(Oct\))75 731 y Fq(ob)s(ject;)h(its)h (structure)g(is)g(declared)h(in)f Fk(mpi.h)f Fq(and)h Fk(mpif.h)p Fq(,)e(and)i(it)g(exists)g(in)g(the)g(users')f(program.)75 787 y(In)i(man)o(y)e(cases)h(application)i(programs)d(are)g (constructed)i(so)e(that)h(it)g(is)g(unnecessary)h(for)f(them)g(to)75 844 y(examine)j(the)g Fk(status)f Fq(\014elds.)35 b(In)20 b(these)g(cases)f(it)h(is)g(a)g(w)o(aste)e(for)h(the)h(user)g(to)f (allo)q(cate)h(a)g(status)75 900 y(ob)s(ject,)f(and)h(it)f(is)h (particularly)g(w)o(asteful)f(for)g(the)g Fj(MPI)f Fq(implemen)o (tation)j(to)d(\014ll)j(in)f(\014elds)h(in)f(this)75 957 y(ob)s(ject.)166 1013 y(T)l(o)h(cop)q(e)h(with)g(this)g(problem,)i (there)d(is)h(a)g(pre-de\014ned)h(constan)o(t,)f Fj(MPI)p 1502 1013 V 16 w(ST)l(A)l(TUS)p 1683 1013 V 17 w(IGNORE)p Fq(,)75 1070 y(whic)o(h,)i(when)e(passed)g(to)f(a)h(receiv)o(e)h(or)e (test)g(function,)j(informs)e(the)g(implemen)o(tation)h(that)e(the)75 1126 y(status)13 b(\014elds)h(are)f(not)g(to)g(b)q(e)h(\014lled)i(in.)k (Note)13 b(that)g Fj(MPI)p 1059 1126 V 15 w(ST)l(A)l(TUS)p 1239 1126 V 18 w(IGNORE)i Fq(is)e(not)h(a)f(sp)q(ecial)i(t)o(yp)q(e)e (of)75 1183 y Fj(MPI)p 160 1183 V 16 w(ST)l(A)l(TUS)i Fq(ob)s(ject;)f(rather,)f(it)h(is)h(a)f(sp)q(ecial)i(v)m(alue)f(for)f (the)g(argumen)o(t.)k(That)c(is,)h(in)f(C)g(one)h(w)o(ould)75 1239 y(exp)q(ect)h(it)f(to)g(b)q(e)h(NULL,)g(not)e(the)i(address)f(of)g (a)g(sp)q(ecial)i Fj(MPI)p 1179 1239 V 15 w(ST)l(A)l(TUS)p Fq(.)166 1295 y(In)12 b(general,)h(this)f(optimization)g(can)g(apply)g (to)f(all)i(functions)f(for)f(whic)o(h)h Fj(MPI)p 1514 1295 V 16 w(Status)i Fq(or)d(an)g(arra)o(y)75 1352 y(of)k Fj(MPI)p 212 1352 V 16 w(Status)p Fq('s)i(is)f(an)g(argumen)o(t.)k (These)c(are)f(all)i(the)e(v)m(arious)h(forms)f(of)g Fj(MPI)p 1493 1352 V 16 w(RECV)p Fq(,)h Fj(MPI)p 1735 1352 V 16 w(TEST)p Fq(,)75 1408 y(and)i Fj(MPI)p 251 1408 V 15 w(W)l(AIT)p Fq(.)f(When)h(an)f(arra)o(y)g(is)g(passed,)h(as)f (in)h(the)p 1132 1408 V 34 w Fj(ANY)g Fq(and)p 1349 1408 V 34 w Fj(ALL)f Fq(functions,)h(a)f(separate)75 1465 y(constan)o(t,)d Fj(MPI)p 356 1465 V 16 w(ST)l(A)l(TUSES)p 589 1465 V 18 w(IGNORE)p Fq(,)i(is)f(passed)h(for)e(the)i(arra)o(y)e (argumen)o(t.)166 1521 y Fj(MPI)p 251 1521 V 16 w(ST)l(A)l(TUS)p 432 1521 V 17 w(IGNORE)g Fq(and)e Fj(MPI)p 794 1521 V 16 w(ST)l(A)l(TUSES)p 1027 1521 V 18 w(IGNORE)i Fq(are)e(not)g (required)h(to)f(ha)o(v)o(e)g(the)g(same)-1992 b Fi(>)15 b Fh(\(Oct\))75 1578 y Fq(v)m(alues)h(in)g(C)f(and)h(F)l(ortran.)166 1634 y(It)d(is)g(not)g(allo)o(w)o(ed)g(to)f(ha)o(v)o(e)g(some)h(of)f (the)h(statuses)f(in)i(an)f(arra)o(y)e(of)i(statuses)f(for)p 1574 1634 V 28 w Fj(ANY)i Fq(and)p 1782 1634 V 29 w Fj(ALL)75 1691 y Fq(functions)j(set)g(to)f Fj(MPI)p 487 1691 V 16 w(ST)l(A)l(TUS)p 668 1691 V 17 w(IGNORE)p Fq(;)i(one)e(either)i(sp)q (eci\014es)g(ignoring)f Fe(al)r(l)34 b Fq(of)17 b(the)f(statuses)g(in) 75 1747 y(suc)o(h)e(a)g(call)h(with)f Fj(MPI)p 482 1747 V 16 w(ST)l(A)l(TUSES)p 715 1747 V 18 w(IGNORE)p Fq(,)h(or)f Fe(none)i Fq(of)e(themm)g(b)o(y)f(passing)i(normal)f(statuses)f(in)75 1804 y(alll)k(p)q(ositions)f(in)g(the)f(arra)o(y)f(of)h(statuses.)-932 b Fi(?)15 b Fh(\(Oct\))75 1925 y Ff(3.2.2)49 b(Non-destructive)16 b(T)l(est)f(of)i(MPI)p 808 1925 15 2 v 18 w(Status)75 2058 y Fm(Curren)o(t)d(Status:)j Fl(P)o(assed)e(once.)166 2162 y Fq(This)22 b(call)g(is)g(useful)g(for)f(accessing)h(the)f (information)h(asso)q(ciated)f(with)h(a)f(request,)h(without)75 2218 y(deleting)c(the)f(request)f(\(in)h(case)g(the)g(user)f(is)i(exp)q (ected)f(to)f(access)h(it)g(after)f(the)g(handler\).)25 b(It)17 b(allo)o(ws)75 2275 y(one)11 b(to)e(la)o(y)o(er)i(libraries)h (more)e(con)o(v)o(enien)o(tly)l(,)i(since)g(m)o(ultiple)g(la)o(y)o(ers) e(of)h(soft)o(w)o(are)d(ma)o(y)i(access)h(the)f(same)75 2331 y(completed)17 b(request)f(and)g(extract)f(from)g(it)h(the)g (status)f(information.)21 b(This)c(will)g(also)f(b)q(e)g(imp)q(ortan)o (t)75 2388 y(for)g(language)i(in)o(terop)q(erabilit)o(y)h(if)e(w)o(e)g (decide)i(that)d(status)g(ob)s(jects)h(cannot)f(b)q(e)i(transferred)f (across)75 2444 y(language)d(b)q(oundaries,)g(since)h(with)f(this)g (function)g(the)g(request)f(ob)s(ject)g(can)h(b)q(e)g(transferred)f (instead.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fg(3.3.)34 b(ERR)o(OR)17 b(CLASS)f(F)o(OR)f(INV)-5 b(ALID)16 b(KEYV)-5 b(AL)818 b Fq(3)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(GET)p 264 45 V 17 w(ST)l(A)l(TUS\()16 b(request,)g(\015ag,)f(status)i(\))117 122 y Fl(IN)155 b Fj(request)452 b Fl(an)14 b Fd(MPI)p 1040 122 13 2 v 14 w(REQUEST)f Fl(ob)r(ject)117 197 y(OUT)108 b Fj(\015ag)518 b Fl(\015ag,)13 b(same)g(as)h(from)e Fd(MPI)p 1325 197 V 15 w(TEST)117 273 y Fl(OUT)108 b Fj(status)476 b Fd(MPI)p 982 273 V 15 w(ST)m(A)m(TUS)12 b Fl(ob)r(ject)j(if)e(\015ag)g(is)h (true)75 397 y Fk(int)23 b(MPI)p 245 397 15 2 v 17 w(Get)p 334 397 V 17 w(status\(MPI)p 591 397 V 16 w(Request)g(request,)f(int)i (*flag,)f(MPI)p 1347 397 V 17 w(Status)g(*status\))75 483 y(MPI)p 150 483 V 17 w(GET)p 239 483 V 17 w(STATUS\()f(REQUEST,)h (FLAG,)g(STATUS,)g(IERROR\))170 540 y(INTEGER)g(REQUEST,)g(FLAG,)g (STATUS\(MPI)p 962 540 V 16 w(STATUS)p 1122 540 V 16 w(SIZE\),)h(IERROR)75 626 y(int)f(MPI::Request::Get)p 581 626 V 15 w(status\(bool&)g(flag,)g(MPI::Status&)f(status\))h(const) 166 713 y Fq(Sets)18 b(\015ag=true)g(if)h(the)f(request)h(has)f (completed,)h(and,)g(if)g(so,)f(returns)g(in)h(status)f(the)g(request) 75 769 y(status.)34 b(Ho)o(w)o(ev)o(er,)20 b(unlik)o(e)i(test)d(or)h(w) o(ait,)g(it)h(do)q(es)f(not)g(deallo)q(cate)h(or)f(inactiv)m(ate)h(the) f(request;)i(a)75 826 y(subsequen)o(t)16 b(call)g(to)f(test,)f(w)o(ait) h(or)f(free)i(should)g(b)q(e)g(executed)g(with)f(that)g(request)75 969 y Fn(3.3)59 b(Erro)n(r)21 b(Class)f(fo)n(r)h(Invalid)e(Keyval)1875 1026 y Fi(>)d Fh(\(Oct\))75 1118 y Fm(Curren)o(t)e(Status:)j Fl(P)o(assed)e(once.)166 1221 y Fq(There)d(is)f(a)h(new)f(MPI)h(error)e (class:)19 b Fj(MPI)p 875 1221 14 2 v 15 w(ERR)p 975 1221 V 18 w(KEYV)l(AL)p Fq(.)11 b(It)h(can)f(b)q(e)h(returned)g(b)o(y)f Fj(MPI)p 1692 1221 V 16 w(A)o(ttr)p 1785 1221 V 17 w(put)p Fq(,)75 1278 y Fj(MPI)p 160 1278 V 16 w(A)o(ttr)p 253 1278 V 17 w(get)p Fq(,)16 b Fj(MPI)p 440 1278 V 16 w(A)o(ttr)p 533 1278 V 16 w(delete)p Fq(,)i Fj(MPI)p 772 1278 V 16 w(Keyval)p 915 1278 V 15 w(free)p Fq(,)e Fj(MPI)p 1111 1278 V 16 w(Comm)p 1253 1278 V 14 w(dup)p Fq(,)i(and)e Fj(MPI)p 1538 1278 V 16 w(Comm)p 1680 1278 V 14 w(free)p Fq(.)23 b(The)75 1334 y(last)15 b(t)o(w)o(o)f(are)h(b)q(ecause)h Fj(k)o(eyval)f Fq(is)h(an)f(argumen)o(t)f(to)h(the)g(cop)o(y)g(and)h (delete)g(functions)g(for)e(attributes.)8 b Fi(?)16 b Fh(\(Oct\))1875 1393 y Fi(>)g Fh(\(Oct\))1875 1451 y Fi(>)g Fh(\(Oct\))75 1478 y Fn(A)k(new)f(reduce)g(op)r(eration)75 1626 y Fm(Curren)o(t)14 b(Status:)j Fl(New)d(for)g(Octob)q(er.)166 1730 y Fq(A)e(new)h(prede\014ned)h(op)q(eration,)f Fd(MPI)p 811 1730 13 2 v 14 w(REPLA)o(CE)p Fq(,)d(is)j(de\014ned.)20 b(It)13 b(corresp)q(onds)f(to)g(the)h(asso)q(ciativ)o(e)75 1786 y(function)18 b Fc(f)5 b Fq(\()p Fc(a;)j(b)p Fq(\))15 b(=)h Fc(b)p Fq(;)i(i.e.,)f(the)g(curren)o(t)h(v)m(alue)g(in)g(the)g (target)e(memory)h(is)h(replaced)g(b)o(y)f(the)h(v)m(alue)75 1843 y(supplied)g(b)o(y)d(the)g(origin.)21 b(This)15 b(op)q(eration)h(can)f(b)q(e)h(applied)h(on)e(an)o(y)g(basic)h(datat)o (yp)q(e.)166 1947 y Fm(Discussion:)35 b Fl(This)15 b(needs)g(a)f (rationale.)19 b(It)14 b(can)h(b)q(e)g(used)g(to)f(do)g(one-sided)h (broadcasts)g(\(should)f(it)g(b)q(e)75 2003 y(f\(a,b\))i(=)h(a\)?)27 b(It)17 b(w)o(ould)f(b)q(e)i(wierd)f(in)f(a)h(reduce)i(op)q(eration,)e (and)f(could)h(b)q(e)h(easily)e(done)h(b)o(y)g(the)g(user)h(as)f(a)75 2059 y(user-de\014ned)f(function.)1875 2107 y Fi(?)g Fh(\(Oct\))75 2250 y Fn(3.4)59 b(A)20 b(F)n(o)n(rtran)h(Problem)e(with) h(Register)e(Optimization)75 2399 y Fm(Curren)o(t)c(Status:)j Fl(A)d(new)g(discussion.)189 2552 y Fe(A)n(dvic)n(e)h(to)i(users.)189 2627 y Fq(MPI)d(pro)o(vides)g(op)q(erations)g(whic)o(h)h(ma)o(y)e(b)q (e)i(hidden)h(from)d(the)h(user)g(co)q(de)h(and)f(run)h(in)f(parallel) 189 2684 y(with)g(it,)h(accessing)g(the)f(same)g(memory)g(as)g(user)g (co)q(de.)20 b(Examples)15 b(include)i(the)d(data)g(transfer)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fq(4)970 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fq(for)10 b(an)h Fj(MPI)p 398 45 14 2 v 16 w(RECV)h Fq(The)f(optimizer)i(of)d(a)h(compiler)i(will)f (assume)f(that)g(it)g(can)h(recognize)g(p)q(erio)q(ds)189 102 y(when)j(a)g(cop)o(y)g(of)g(a)f(v)m(ariable)j(can)e(b)q(e)h(k)o (ept)f(in)h(a)e(register)h(without)g(reloading)h(from)f(or)f(storing) 189 158 y(to)e(memory)l(.)18 b(When)13 b(the)g(user)g(co)q(de)g(is)g(w) o(orking)f(with)h(a)f(register)h(cop)o(y)f(of)g(some)h(v)m(ariable)g (while)189 214 y(the)i(hidden)i(op)q(eration)f(reads)f(or)g(writes)g (the)h(memory)f(cop)o(y)l(,)g(problems)h(o)q(ccur.)k(This)c(section)189 271 y(discusses)g(register)f(optimization)h(pitfalls.)189 346 y(When)f(a)f(v)m(ariable)i(is)f(lo)q(cal)h(to)e(a)g(F)l(ortran)f (subroutine)j(\(i.e.)k(not)14 b(in)h(a)g(COMMON)f(blo)q(c)o(k\),)h(the) 189 403 y(compiler)g(will)i(assume)d(that)g(it)h(cannot)f(b)q(e)h(mo)q (di\014ed)h(b)o(y)f(a)f(called)i(subroutine)f(unless)h(it)f(is)g(an)189 459 y(actual)g(argumen)o(t)g(of)h(the)f(call.)23 b(In)16 b(the)g(most)e(common)i(link)m(age)h(con)o(v)o(en)o(tion,)e(the)h (subroutine)189 515 y(is)d(exp)q(ected)i(to)d(sa)o(v)o(e)h(and)g (restore)g(certain)h(registers.)19 b(Th)o(us,)13 b(the)g(optimizer)i (will)g(assume)e(that)189 572 y(a)k(register)h(whic)o(h)h(held)g(a)e(v) m(alid)j(cop)o(y)d(of)h(suc)o(h)g(a)g(v)m(ariable)h(b)q(efore)f(the)g (call)h(will)g(still)h(hold)e(a)189 628 y(v)m(alid)e(cop)o(y)g(on)f (return.)189 703 y(Normally)22 b(users)g(are)g(not)g(a\017icted)g(with) h(this.)41 b(But)22 b(the)g(user)g(should)h(pa)o(y)f(atten)o(tion)g(to) 189 760 y(this)f(section)g(if)g(in)h(his/her)f(program)e(a)i(bu\013er)f (argumen)o(t)g(to)g(an)h Fj(MPI)p 1498 760 V 16 w(SEND)p Fq(,)f Fj(MPI)p 1746 760 V 16 w(RECV)189 816 y Fq(etc.)25 b(uses)18 b(a)f(name)g(whic)o(h)h(hides)g(the)f(actual)h(v)m(ariables)g (in)o(v)o(olv)o(ed.)27 b Fj(MPI)p 1493 816 V 16 w(BOTTOM)18 b Fq(with)f(an)189 873 y(MPI)p 281 873 V 16 w(Datat)o(yp)q(e)i(con)o (taining)i(absolute)g(addresses)f(is)h(one)f(example.)36 b(Creating)20 b(a)f(datat)o(yp)q(e)189 929 y(whic)o(h)14 b(uses)f(one)g(v)m(ariable)i(as)e(an)g(anc)o(hor)f(and)i(brings)f (along)h(others)e(b)o(y)h(using)h Fj(MPI)p 1659 929 V 16 w(ADDRESS)189 986 y Fq(to)h(determine)j(their)f(o\013sets)e(from)g (the)i(anc)o(hor)f(is)h(another.)22 b(The)17 b(anc)o(hor)f(v)m(ariable) i(w)o(ould)e(b)q(e)189 1042 y(the)g(only)h(one)f(men)o(tioned)h(in)g (the)g(call.)24 b(Also)16 b(atten)o(tion)g(m)o(ust)g(b)q(e)h(pa)o(y)o (ed)f(if)h(MPI)f(op)q(erations)189 1099 y(are)f(used)g(that)g(run)g(in) h(parallel)h(with)f(the)f(user's)g(application.)189 1174 y(The)g(follo)o(wing)h(example)g(sho)o(ws,)e(what)h(F)l(ortran)f (compilers)i(are)f(allo)o(w)o(ed)h(to)e(do:)224 1299 y(This)i(source)f(...)663 b(can)15 b(b)q(e)h(compiled)h(as:)224 1363 y(call)f(MPI)p 399 1363 V 17 w(ADDRESS\(buf,bufaddr,ierr\))189 b(call)16 b(MPI)p 1344 1363 V 17 w(ADDRESS\(buf,...\))224 1420 y(call)g(MPI)p 399 1420 V 17 w(TYPE)p 545 1420 V 16 w(STR)o(UCT\(1,1,bufaddr,)163 b(call)16 b(MPI)p 1344 1420 V 17 w(TYPE)p 1490 1420 V 16 w(STR)o(UCT\(...\))590 1476 y(MPI)p 682 1476 V 17 w(REAL,t)o(yp,ierr\))224 1533 y(call)g(MPI)p 399 1533 V 17 w(TYPE)p 545 1533 V 16 w(COMMIT\(t)o(yp\)) 308 b(call)16 b(MPI)p 1344 1533 V 17 w(TYPE)p 1490 1533 V 16 w(COMMIT\(...\))224 1589 y(v)m(al)p 283 1589 V 17 w(old)g(=)g(buf)681 b(register)15 b(=)h(buf)1169 1646 y(v)m(al)p 1228 1646 V 17 w(old)g(=)f(register)224 1702 y(call)h(MPI)p 399 1702 V 17 w(RECV\(MPI)p 654 1702 V 16 w(BOTTOM,1,t)o(yp,...\))107 b(call)16 b(MPI)p 1344 1702 V 17 w(RECV\(MPI)p 1599 1702 V 16 w(BOTTOM,...\))224 1759 y(v)m(al)p 283 1759 V 17 w(new)g(=)f(buf)664 b(v)m(al)p 1228 1759 V 17 w(new)16 b(=)f(register)189 1884 y(The)j(compiler)g(do)q (es)g(not)g(in)o(v)m(alidate)h(the)f(register)g(b)q(ecause)g(it)g (cannot)f(see)h(that)f Fj(MPI)p 1746 1884 V 16 w(RECV)189 1940 y Fq(c)o(hanges)h(the)g(v)m(alue)i(of)e Fj(buf)p Fq(.)30 b(The)19 b(access)f(of)g Fj(buf)i Fq(is)f(hidden)h(b)o(y)e(the) h(use)f(of)g Fj(MPI)p 1659 1940 V 16 w(ADDRESS)189 1997 y Fq(and)d Fj(MPI)p 362 1997 V 16 w(BOTTOM)p Fq(.)189 2072 y(The)g(next)g(example)h(sho)o(ws)f(extreme,)g(but)g(allo)o(w)o (ed)h(p)q(ossibilities!)224 2197 y(Source)424 b(compiled)17 b(as)322 b(or)15 b(compiled)i(as)224 2261 y(call)f(MPI)p 399 2261 V 17 w(IRECV\(buf,..req\))i(call)f(MPI)p 955 2261 V 16 w(IRECV\(buf,..req\))h(call)f(MPI)p 1510 2261 V 16 w(IRECV\(buf,..req\))779 2318 y(register)e(=)h(buf)278 b(b1)15 b(=)h(buf)224 2374 y(call)g(MPI)p 399 2374 V 17 w(W)-5 b(AIT\(req,..\))104 b(call)17 b(MPI)p 955 2374 V 16 w(W)-5 b(AIT\(req,..\))104 b(call)17 b(MPI)p 1510 2374 V 16 w(W)-5 b(AIT\(req,..\))224 2431 y(b1)15 b(=)h(buf)377 b(b1)15 b(:=)g(register)189 2556 y Fj(MPI)p 274 2556 V 15 w(W)l(AIT)e Fq(or)f(a)g(parallel)i(thread)e(mo)q(di\014es)i Fj(buf)f Fq(b)q(et)o(w)o(een)g(the)f(in)o(v)o(o)q(cation)h(of)f Fj(MPI)p 1648 2556 V 16 w(IRECV)h Fq(and)189 2612 y(the)j(\014nish)h (of)f Fj(MPI)p 528 2612 V 16 w(W)l(AIT)p Fq(.)g(But)g(the)g(compiler)i (cannot)e(see)g(an)o(y)g(p)q(ossibilit)o(y)i(that)e Fj(buf)h Fq(can)f(b)q(e)189 2669 y(c)o(hanged)g(after)e Fj(MPI)p 557 2669 V 16 w(IRECV)i Fq(has)g(\014nished.)23 b(And)16 b(therefore)f(it)h(can)g(sc)o(hedule)h(the)f(load)g(of)f Fj(buf)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fg(3.5.)34 b(TR)o(UE)15 b(EXTENT)g(OF)g(D)o(A)l(T)l (A)l(TYPES)958 b Fq(5)189 45 y(earlier)14 b(than)g(t)o(yp)q(ed)g(in)h (the)f(source.)20 b(It)13 b(has)h(no)g(reason)f(to)h(a)o(v)o(oid)f (using)i(a)e(register)h(to)f(hold)i Fj(buf)189 102 y Fq(across)f(the)h(call)i(to)d Fj(MPI)p 625 102 14 2 v 16 w(W)l(AIT)p Fq(.)h(It)g(also)g(can)h(reorder)f(the)g(instructions)h (as)f(in)h(the)f(left)h(case.)189 177 y(T)l(o)c(prev)o(en)o(t)h (instruction)h(reordering)g(or)e(the)h(allo)q(cation)h(of)f(a)g(v)m (ariable)h(or)f(bu\013er)g(in)h(a)e(register,)189 233 y(there)j(are)g(three)g(p)q(ossibilities)k(in)d(p)q(ortable)f(F)l (ortran)f(co)q(de:)243 327 y Fi(\017)23 b Fq(One)d(can)f(put)h(the)g(v) m(ariables)g(and)g(bu\013ers)f(in)o(to)h(common)f(blo)q(c)o(ks;)j(then) e(they)f(will)i(b)q(e)289 383 y(allo)q(cated)d(in)g(the)g(memory)e (while)j(an)o(y)e(library)h(subroutine)h(\(e.g.)25 b(an)18 b(MPI)f(routine\))g(is)289 440 y(called)c(pro)o(vided)f(that)f(the)h (compiler)h(cannot)e(analyze)h(that)f(the)h(library)g(subroutine)h(do)q (es)289 496 y(not)h(use)i(these)f(common)g(blo)q(c)o(ks,)h(or)243 569 y Fi(\017)23 b Fq(one)11 b(can)h(declare)g(them)f(as)g(V)o(OLA)l (TILE,)h(but)g(this)g(is)f(not)g(part)g(of)g(the)g(standards)g(F)l (ortran)289 626 y(77)f(and)h(90,)g(and)g(y)o(our)f(co)q(de)i(will)h (not)d(b)q(e)i(p)q(ortable)f(b)q(ecause)h(there)f(are)g(compilers)h (without)289 682 y(V)o(OLA)l(TILE,)k(and)f(normally)h(it)g(prev)o(en)o (ts)e(an)o(y)h(optimization,)h(or)243 755 y Fi(\017)23 b Fq(one)14 b(can)g(call)h(a)f(dumm)o(y)g(routine)g(with)g(the)h(v)m (ariable)g(or)e(bu\013er)h(as)g(an)g(actual)g(argumen)o(t;)289 812 y(if)22 b(the)h(dumm)o(y)f(routine)h(is)f(not)g(part)g(of)g(the)g (application)i(then)f(the)f(compiler)i(m)o(ust)289 868 y(allo)q(cate)16 b(the)f(v)m(ariable)i(or)e(bu\013er)h(in)g(the)f (memory)g(while)i(that)e(subroutine)h(is)g(executed;)289 925 y Fj(MPI)p 374 925 V 15 w(ADDRESS)h Fq(can)e(b)q(e)h(used)g(as)e (dumm)o(y)h(routine.)189 1019 y(High)e(qualit)o(y)g(F)l(ortran)f (implemen)o(tations)i(ma)o(y)e(pro)o(vide)i(mec)o(hanisms)f(to)f(a)o(v) o(oid)h(the)g(problem,)189 1075 y(without)18 b(disabling)j (optimizations.)31 b(Ho)o(w)o(ev)o(er,)18 b(co)q(de)h(that)f(do)q(es)h (not)f(use)h(one)g(of)f(the)h(three)189 1131 y(solutions)d(describ)q (ed)h(ab)q(o)o(v)o(e)d(ma)o(y)h(not)g(b)q(e)h(p)q(ortable.)189 1207 y(In)h(C,)f(subroutines)i(whic)o(h)g(mo)q(dify)f(v)m(ariables)h (that)e(are)h(not)f(in)i(the)f(parameter)f(list)i(will)g(not)189 1263 y(cause)e(register)h(optimization)g(problems.)25 b(This)17 b(is)g(b)q(ecause)g(taking)g(p)q(oin)o(ters)g(to)f(storage)f (ob-)189 1319 y(jects)d(b)o(y)g(using)h(the)g(&)f(op)q(erator)g(and)g (later)h(referencing)g(the)g(ob)s(jects)f(b)o(y)g(w)o(a)o(y)f(of)h(the) h(p)q(oin)o(ter)f(is)189 1376 y(an)j(in)o(tegral)g(part)g(of)f(the)i (language.)k(A)15 b(C)g(compiler)h(understands)g(the)f(implications,)i (so)e(that)189 1432 y(the)g(problem)h(should)h(not)e(o)q(ccur,)h(in)g (general.)21 b(Ho)o(w)o(ev)o(er,)14 b(some)i(compilers)g(do)g(o\013er)e (optional)189 1489 y(aggressiv)o(e)g(optimization)j(lev)o(els)f(whic)o (h)g(ma)o(y)f(not)f(b)q(e)i(safe.)189 1564 y(\()p Fe(End)f(of)i(advic)n (e)f(to)g(users.)p Fq(\))p 75 1670 827 2 v 75 1727 a(The)f(follo)o (wing)h(advice)h(m)o(ust)d(b)q(e)i(added)g(in)75 1795 y(-)f(MPI)g(2,)g(c)o(hapter)g(?.?)21 b(Handlers,)75 1851 y(-)15 b Fj(MPI)p 190 1851 14 2 v 16 w(ADDRESS)h Fq(\(MPI)f(1.1,)f (page)h(70,)f(line)j(16\),)d(and)h(in)75 1908 y(-)g Fj(MPI)p 190 1908 V 16 w(IRECV)h Fq(\(MPI)f(1.1,)e(\\Non)o(blo)q(c)o(king)k (comm)o(unication",)e(end)h(of)e(page)h(40\))189 2014 y Fe(A)n(dvic)n(e)i(to)i(users.)53 b Fq(T)l(o)17 b(prev)o(en)o(t)g (troubles)h(with)g(the)f(register)g(optimization)i(of)e(the)g(F)l (ortran)189 2071 y(compilers)i(please)g(pa)o(y)e(atten)o(tion)h(to)f (the)h(hin)o(ts)g(in)h(section)g(3.4)e(\\A)g(F)l(ortran)g(Problem)i (with)189 2127 y(register)c(optimization")h(on)f(page)g(3.)k(\()p Fe(End)d(of)g(advic)n(e)g(to)h(users.)p Fq(\))1875 2188 y Fi(?)f Fh(\(Oct\))75 2320 y Fn(3.5)59 b(T)-5 b(rue)20 b(Extent)d(of)k(Datat)n(yp)r(es)75 2469 y Fm(Curren)o(t)14 b(Status:)j Fl(P)o(assed)e(once.)166 2573 y Fq(Supp)q(ose)23 b(w)o(e)f(implemen)o(t)h(gather)f(as)g(a)f(spanning)i(tree)f(implemen)o (ted)i(on)e(top)g(of)g(p)q(oin)o(t-to-)75 2629 y(p)q(oin)o(t)e (routines.)34 b(Since)22 b(the)d(recv)h(bu\013er)g(is)g(only)h(v)m (alid)g(on)f(the)g(ro)q(ot)f(pro)q(cess,)i(one)f(will)h(need)g(to)75 2685 y(allo)q(cate)f(some)f(temp)q(orary)g(space)h(for)f(receiving)i (data)e(on)g(in)o(termediate)i(no)q(des.)33 b(The)20 b(di\016cultly)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fq(6)970 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)75 45 y Fq(is)20 b(in)g(determining)h(the)f(size)g(one)f (needs)i(to)d(allo)q(ciate.)34 b(This)20 b(o)q(ccurs)g(since)h(the)e (user)h(can)f(mo)q(dify)75 102 y(the)g(exten)o(t)h(using)g(the)f Fd(MPI)p 583 102 13 2 v 14 w(UB)g Fq(and)h Fd(MPI)p 841 102 V 14 w(LB)f Fq(v)m(alues.)34 b(The)20 b(writer)f(of)g(the)g(gather) g(routine)h(could)75 158 y(determine)14 b(this)f(information)g(b)o(y)g (deco)q(ding)h(the)f(datat)o(yp)q(e.)18 b(Ho)o(w)o(ev)o(er,)12 b(this)h(is)g(more)g(w)o(ork)e(and)i(more)75 214 y(painful)j(than)e (desired.)20 b(Th)o(us,)14 b(a)g(new)g(function)h(is)g(pro)o(vided)g (whic)o(h)f(returns)g(the)h(true)e(exten)o(t)h(of)g(the)75 271 y(datat)o(yp)q(e.)75 422 y Fj(MPI)p 160 422 14 2 v 16 w(TRUE)p 294 422 V 17 w(EXTENT\(datat)o(yp)q(e,)j(true)p 775 422 V 17 w(lb,)e(true)p 929 422 V 17 w(extent\))117 499 y Fl(IN)155 b Fj(datat)o(yp)q(e)424 b Fl(datat)o(yp)q(e)14 b(to)g(get)g(information)d(on)117 574 y(OUT)108 b Fj(true)p 396 574 V 17 w(lb)461 b Fl(true)15 b(lo)o(w)o(er)e(b)q(ound)h(of)g (datat)o(yp)q(e)117 649 y(OUT)108 b Fj(true)p 396 649 V 17 w(extent)379 b Fl(true)15 b(size)g(of)e(datat)o(yp)q(e)75 774 y Fk(int)23 b(MPI)p 245 774 15 2 v 17 w(True)p 358 774 V 17 w(extent\(MPI)p 615 774 V 16 w(Datatype)f(datatype,)h(MPI)p 1156 774 V 17 w(Aint)g(*true)p 1412 774 V 17 w(lb,)g(MPI)p 1596 774 V 17 w(Aint)393 830 y(*true)p 516 830 V 17 w(extent\))75 917 y(MPI)p 150 917 V 17 w(TRUE)p 263 917 V 16 w(EXTENT\(DATATYPE,)f (TRUE)p 781 917 V 17 w(LB,)h(TRUE)p 989 917 V 17 w(EXTENT,)g(IERROR\)) 170 973 y(INTEGER)g(DATATYPE,)g(TRUE)p 699 973 V 17 w(LB,)g(TRUE)p 907 973 V 17 w(EXTENT,)g(IERROR)166 1060 y Fj(true)p 244 1060 14 2 v 17 w(lb)14 b Fq(returns)f(the)g(o\013set)g(of)g(the)g (lo)o(w)o(est)g(unit)h(of)f(store)g(whic)o(h)h(is)g(addressed)g(b)o(y)f (the)h(datat)o(yp)q(e.)75 1116 y Fj(true)p 153 1116 V 17 w(extent)k Fq(returns)f(the)g(true)f(size)i(of)e(the)h(datat)o(yp)q (e.)24 b(The)17 b Fj(true)p 1244 1116 V 17 w(extent)i Fq(is)e(the)g(minim)o(um)h(n)o(um)o(b)q(er)75 1173 y(of)13 b(b)o(ytes)g(of)h(memory)f(necessary)g(to)g(hold)i(a)e(datat)o(yp)q(e)g (and)h(ignores)g(the)f Fj(LB)h Fq(and)g Fj(UB)g Fq(that)e(ma)o(y)h(ha)o (v)o(e)75 1229 y(b)q(een)j(used)g(in)g(creating)g(the)f(datat)o(yp)q (e.)166 1326 y Fm(Alternativ)o(es)o(:)166 1376 y Fl(If)e(w)o(e)i(w)o (an)o(t)e(to)h(b)q(e)g(consisten)o(t)h(with)f(the)g(MPI1)g(exten)o(t)h (functions,)f(w)o(e)g(w)o(ould)f(need)i(three)g(functions:)166 1426 y Fd(MPI)p 243 1426 13 2 v 14 w(TRUE)p 366 1426 V 15 w(LB\(datat)o(yp)q(e,)g(lb\))166 1475 y(MPI)p 243 1475 V 14 w(TRUE)p 366 1475 V 15 w(UB\(datat)o(yp)q(e,)f(ub\))166 1525 y(MPI)p 243 1525 V 14 w(TRUE)p 366 1525 V 15 w(EXTENT\(datat)o(yp) q(e,)h(extent\))166 1575 y Fl(On)f(the)h(other)f(hand,)f(w)o(e)i(ma)o (y)c(not)j(w)o(an)o(t)g(to)f(mak)o(e)g(the)h(same)f(mistak)o(e)g(t)o (wice...)75 1766 y Fn(3.6)59 b(Datat)n(yp)r(e)18 b(extent)75 1914 y Fm(Curren)o(t)c(Status:)j Fl(P)o(assed)e(once.)166 2018 y Fq(MPI)21 b(allo)o(ws)h(one)f(to)g(c)o(hange)h(the)f(exten)o(t)g (of)g(a)h(datat)o(yp)q(e,)g(using)g(lo)o(w)o(er)f(b)q(ound)h(and)g(upp) q(er)75 2074 y(b)q(ound)e(mark)o(ers)e(\()p Fd(MPI)p 490 2074 V 14 w(LB)h Fq(and)g Fd(MPI)p 740 2074 V 14 w(UB)p Fq(\).)f(This)h(is)h(useful,)g(as)f(it)g(allo)o(ws)g(to)f(con)o (trol)h(the)g(stride)g(of)75 2131 y(successiv)o(e)e(datat)o(yp)q(es)e (that)g(are)g(replicated)i(b)o(y)e(datat)o(yp)q(e)g(constructors,)g(or) g(are)g(replicated)i(b)o(y)f(the)75 2187 y Fj(count)k Fq(argumen)o(t)e(in)i(a)e(send)h(or)g(reciev)o(e)g(call.)32 b(Ho)o(w)o(ev)o(er,)19 b(the)f(curren)o(t)h(mec)o(hanism)h(for)e(ac)o (hieving)75 2244 y(it)h(is)h(painful;)i(also)d(it)h(is)f(restrictiv)o (e,)i(as)d Fd(MPI)p 906 2244 V 15 w(LB)h Fq(and)g Fd(MPI)p 1157 2244 V 14 w(UB)g Fq(are)f(\\stic)o(ky":)28 b(once)19 b(presen)o(t)g(in)h(a)75 2300 y(datat)o(yp)q(e,)d(they)h(cannot)g(b)q (e)g(o)o(v)o(eriden)g(\(e.g.,)f(the)h(upp)q(er)h(b)q(ound)f(can)g(b)q (e)g(mo)o(v)o(ed)g(up,)g(b)o(y)g(adding)g(a)75 2357 y(new)13 b Fd(MPI)p 243 2357 V 14 w(UB)g Fq(mark)o(er,)f(but)h(cannot)g(b)q(e)g (mo)o(v)o(ed)g(do)o(wn)f(b)q(elo)o(w)i(an)f(existing)h Fj(MPI)p 1474 2357 14 2 v 15 w(UB)g Fq(mark)o(er\).)k(A)13 b(new)75 2413 y(function)j(is)g(pro)o(vided)g(to)e(facilitate)i(these)g (c)o(hanges.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fg(3.7.)34 b(MPI-1.0)14 b(AND)i(MPI-1.1)e(CLARIFICA)l (TIONS)802 b Fq(7)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(SET)p 259 45 V 17 w(EXTENT\(datat)o(yp)q(e,)16 b(lb,)f(extent\))117 122 y Fl(INOUT)62 b Fj(datat)o(yp)q(e)424 b Fl(datat)o(yp)q(e)14 b(to)g(up)q(date)h(exten)o(t)117 197 y(IN)155 b Fj(true)p 396 197 V 17 w(lb)461 b Fl(new)15 b(lo)o(w)o(er)e(b)q(ound)h(of)f (datat)o(yp)q(e)117 273 y(IN)155 b Fj(true)p 396 273 V 17 w(extent)379 b Fl(new)15 b(exten)o(t)f(of)g(datat)o(yp)q(e)75 397 y Fk(int)23 b(MPI)p 245 397 15 2 v 17 w(Set)p 334 397 V 17 w(extent\(MPI)p 591 397 V 16 w(Datatype)g(datatype,)f(MPI)p 1132 397 V 17 w(Aint)h(lb,)h(MPI)p 1436 397 V 17 w(Aint)f(extent\))75 483 y(MPI)p 150 483 V 17 w(TRUE)p 263 483 V 16 w(EXTENT\(DATATYPE,)f (LB,)i(EXTENT,)e(IERROR\))170 540 y(INTEGER)h(DATATYPE,)g(LB,)g (EXTENT,)g(IERROR)166 626 y Fq(Resets)13 b(the)g(lo)o(w)o(er)g(b)q (ound)h(and)f(the)h(exten)o(t)e(\(and,)h(hence,)h(the)g(upp)q(er)g(b)q (ound\))f(of)g Fj(datat)o(yp)q(e)p Fq(.)21 b(The)75 683 y(new)e(settings)f(will)i(a\013ect)e(the)h(outcome)f(of)g(send)h(or)f (receiv)o(e)h(op)q(erations)g(that)e(use)i Fj(datat)o(yp)q(e)p Fq(,)i(and)75 739 y(a\013ects)14 b(the)i(v)m(alue)g(of)f(datat)o(yp)q (es)g(constructed)g(using)h Fj(datat)o(yp)q(e)h Fq(as)e(an)g(agumen)o (t,)f(after)h(the)g(call)i(w)o(as)75 796 y(made.)i(Ho)o(w)o(ev)o(er,)11 b(datat)o(yp)q(es)h(that)f(w)o(ere)h(previously)i(constructed)e(using)h Fj(datat)o(yp)q(e)h Fq(are)e(not)g(a\013ected.)189 902 y Fe(R)n(ationale.)189 977 y Fq(This)h(is)g(consisten)o(t)f(to)g(the)h (MPI)f(statemen)o(t)g(that)g(\\the)g(system)g(b)q(eha)o(v)o(es)h(as)f (if)h(input)g(datat)o(yp)q(e)189 1034 y(argumen)o(ts)h(to)h(deriv)o(ed) h(datat)o(yp)q(e)e(constructors)h(are)g(passed)g(b)o(y)g(v)m(alue".)189 1109 y(\()p Fe(End)g(of)i(r)n(ationale.)p Fq(\))189 1215 y Fe(A)n(dvic)n(e)g(to)h(users.)51 b Fq(It)17 b(is)g(recommended)h (that)f(uses)g(use)g(the)g(function)h Fj(MPI)p 1571 1215 14 2 v 16 w(SET)p 1670 1215 V 17 w(EXTENT)p Fq(,)189 1271 y(rather)c(than)h Fd(MPI)p 508 1271 13 2 v 15 w(LB)g Fq(and)g Fd(MPI)p 751 1271 V 15 w(UB)f Fq(mark)o(ers.)189 1347 y(\()p Fe(End)h(of)i(advic)n(e)f(to)g(users.)p Fq(\))1875 1408 y Fi(?)g Fh(\(Oct\))75 1540 y Fn(3.7)59 b(MPI-1.0)19 b(and)h(MPI-1.1)f(Cla)n(ri\014cations)75 1643 y Ff(3.7.1)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 1643 15 2 v 18 w(FINALIZE)1875 1672 y Fi(>)g Fh(\(Oct\))75 1776 y Fm(Curren)o(t)e(Status:)j Fl(Not)d(v)o(oted)g(on.)166 1880 y Fq(This)j(routine)f(cleans)h(up)f(all)h(MPI)f(state.)22 b(Eac)o(h)16 b(pro)q(cess)g(m)o(ust)f(call)i Fj(MPI)p 1485 1880 14 2 v 16 w(FINALIZE)e Fq(b)q(efore)i(it)75 1936 y(exits.)j(Once)c(this)f(routine)g(returns,)f(no)h(MPI)g(routine)g (\(ev)o(en)g(not)f Fj(MPI)p 1338 1936 V 16 w(INIT)p Fq(\))f(ma)o(y)h(b) q(e)i(called.)21 b(Eac)o(h)75 1992 y(pro)q(cess)11 b(m)o(ust)e (complete)j(an)o(y)e(p)q(ending)i(comm)o(unication)f(it)g(initiated)h (b)q(efore)e(it)h(calls)g Fj(MPI)p 1656 1992 V 16 w(FINALIZE)p Fq(.)75 2049 y(If)16 b(the)g(call)h(returns,)e(eac)o(h)h(pro)q(cess)g (ma)o(y)f(con)o(tin)o(ue)i(lo)q(cal)f(computations,)g(or)f(exit,)h (without)g(partici-)75 2105 y(pating)g(in)g(further)g Fj(MPI)f Fq(comm)o(unication)h(with)g(other)f(pro)q(cesses.)22 b Fj(MPI)p 1362 2105 V 16 w(FINALIZE)14 b Fq(is)j(collectiv)o(e)g(on)75 2162 y Fj(MPI)p 160 2162 V 16 w(COMM)p 318 2162 V 16 w(W)o(ORLD)p Fq(.)1369 b Fi(>)16 b Fh(\(Oct\))166 2218 y Fq(It)f(is)h(not)f(required)h(that)e Fj(MPI)p 703 2218 V 16 w(Finalize)h Fq(return,)g(ev)o(en)g(on)h(one)f(pro)q(cess.)1875 2220 y Fi(?)h Fh(\(Oct\))189 2325 y Fe(A)n(dvic)n(e)11 b(to)i(implementors.)38 b Fq(Ev)o(en)11 b(though)g(a)g(pro)q(cess)h (has)f(completed)i(all)f(the)f(comm)o(unication)189 2381 y(it)j(initiated,)h(suc)o(h)g(comm)o(unication)f(ma)o(y)f(not)h(y)o(et) g(b)q(e)g(completed)h(from)e(the)h(viewp)q(oin)o(t)h(of)f(the)189 2437 y(underlying)j(MPI)f(system.)k(E.g.,)14 b(a)h(blo)q(c)o(king)i (send)f(ma)o(y)f(ha)o(v)o(e)g(completed,)i(ev)o(en)e(though)h(the)189 2494 y(data)k(is)h(still)h(bu\013ered)f(at)f(the)h(sender.)37 b(The)20 b(MPI)h(implemen)o(tation)h(m)o(ust)e(ensure)h(that)f(a)189 2550 y(pro)q(cess)14 b(has)g(completed)h(an)o(y)f(in)o(v)o(olv)o(emen)o (t)g(in)h(MPI)f(comm)o(unication)h(b)q(efore)f Fj(MPI)p 1669 2550 V 16 w(FINALIZE)189 2607 y Fq(returns.)26 b(Th)o(us,)17 b(if)h(a)e(pro)q(cess)i(exits)g(after)e(the)h(call)i(to)d Fj(MPI)p 1271 2607 V 16 w(FINALIZE)p Fq(,)g(this)i(will)h(not)e(cause) 189 2663 y(an)e(ongoing)g(comm)o(unication)h(to)e(fail.)21 b(\()p Fe(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Fq(\))-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fq(8)970 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fe(A)n(dvic)n(e)i(to)i(implementors.)54 b Fq(Although)18 b(it)g(is)g(not)f(required,)i(it)f(has)f(b)q(een)i (found)f(that)f(users)189 102 y(exp)q(ect)f(that)f(at)f(least)i(one)f (pro)q(cess)h(return,)f(so)g(that)g(they)h(can)f(kno)o(w)g(that)g(the)g (MPI)h(p)q(ortion)189 158 y(of)e(the)g(computation)g(is)h(o)o(v)o(er.)k (In)c(addition,)g(in)g(a)f(POSIX)i(en)o(vironmen)o(t,)e(they)h(ma)o(y)e (desire)j(to)189 214 y(supply)f(an)f(exit)g(co)q(de)h(for)e(eac)o(h)h (pro)q(cess)g(that)f(returns)h(from)f Fj(MPI)p 1357 214 14 2 v 16 w(FINALIZE)p Fq(.)g(\()p Fe(End)h(of)h(advic)n(e)189 271 y(to)h(implementors.)p Fq(\))-117 332 y Fi(>)f Fh(\(Oct\))75 464 y Fn(3.8)59 b(Determining)19 b(Whether)g(MPI)h(Has)g(Finished)75 613 y Fm(Curren)o(t)14 b(Status:)j Fl(No)d(v)o(otes.)166 716 y Fq(One)i(of)f(the)h(goals)f(of)h Fj(MPI)f Fq(w)o(as)f(to)h(allo)o (w)h(for)f(la)o(y)o(ered)h(libraries.)23 b(In)16 b(order)f(for)g(a)h (library)g(to)f(do)75 773 y(this)k(cleanly)l(,)i(it)e(needs)g(to)f(kno) o(w)g(if)h Fj(MPI)f Fq(is)i(activ)o(e.)30 b(In)19 b Fj(MPI-1)f Fq(the)h(function)g Fj(MPI)p 1590 773 V 16 w(Initialized)g Fq(w)o(as)75 829 y(pro)o(vided)c(to)g(tell)g(if)g Fj(MPI)f Fq(had)h(b)q(een)h(initialzed.)23 b(The)15 b(problem)g(arises)g(in)g (kno)o(wing)g(if)g Fj(MPI)f Fq(has)h(b)q(een)75 886 y(\014nalized.)31 b(Once)19 b Fj(MPI)f Fq(has)g(b)q(een)h(\014nalized)h(it)f(is)g(no)f (longer)g(activ)o(e)h(and)f(cannot)g(b)q(e)h(restarted.)28 b(A)75 942 y(library)16 b(needs)g(to)e(b)q(e)i(able)g(to)e(determine)i (this)f(to)g(act)f(accordingly)l(.)22 b(Do)14 b(ac)o(hiev)o(e)i(this)f (the)g(follo)o(wing)75 999 y(function)h(is)g(needed:)75 1150 y Fj(MPI)p 160 1150 V 16 w(FINALIZED\(\015ag\))117 1227 y Fl(OUT)108 b Fj(\015ag)518 b Fl(true)15 b(if)e Fd(MPI)g Fl(w)o(as)h(\014nalized)g(\(logical\).)75 1351 y Fk(int)23 b(MPI)p 245 1351 15 2 v 17 w(Finalized\(int)f(*flag\))75 1438 y(MPI)p 150 1438 V 17 w(FINALIZED\(flag,)g(IERROR\)\))170 1494 y(FLAG,)i(IERROR)75 1581 y(int)f(MPI::Finalized\(bool)f (&flag\)\)?????)166 1667 y Fq(This)17 b(function)g(returns)f(true)g(if) h Fj(MPI)p 833 1667 14 2 v 16 w(FINALIZE)e Fq(has)h(returned.)23 b(Th)o(us,)16 b(it)h(will)h(return)e(false)g(if)75 1724 y(called)k(in)f(a)f(callbac)o(k)h(routine)f(in)o(v)o(ok)o(ed)h(b)o(y)f Fj(MPI)p 960 1724 V 16 w(FINALIZE)p Fq(.)f(It)h(is)g(legal)h(to)f(call) h Fj(MPI)p 1636 1724 V 16 w(FINALIZED)75 1780 y Fq(b)q(efore)c Fj(MPI)p 296 1780 V 16 w(INIT)g Fq(and)g(after)g Fj(MPI)p 694 1780 V 15 w(FINALIZE)p Fq(.)-1030 b Fi(?)15 b Fh(\(Oct\))-117 1839 y Fi(>)g Fh(\(Oct\))75 1902 y Ff(3.8.1)49 b(Cla)o(ri\014cation)18 b(of)e(status)f(after)h(MPI)p 875 1902 15 2 v 18 w(Isend)75 2035 y Fm(Curren)o(t)e(Status:)j Fl(No)d(v)o(otes.)166 2138 y Fq(A)22 b Fj(status)i Fq(returned)e(b)o(y)g(a)g(call)h(to)e Fj(MPI)p 900 2138 14 2 v 16 w(W)l(AIT)p Fq(,)g Fj(MPI)p 1145 2138 V 16 w(TEST)p Fq(,)h(or)f(an)o(y)h(of)f(the)h(other)g(deriv)o (ed)75 2195 y(functions,)j(where)f(the)f Fj(request)h Fq(corresp)q(onds)g(to)e(a)h(send)g(call,)j(is)e(unde\014ned,)i(with)d (t)o(w)o(o)f(excep-)75 2251 y(tions:)36 b(The)24 b(error)f(status)f (\014eld)j(will)g(con)o(tain)e(v)m(alid)i(information)f(if)f(the)h(w)o (ait)f(or)g(test)f(call)j(re-)75 2308 y(turned)20 b(with)g Fd(MPI)p 411 2308 13 2 v 15 w(ERROR)p 563 2308 V 13 w(IN)p 617 2308 V 15 w(ST)m(A)m(TUS)p Fq(;)f(and)h(the)g(returned)g(status)f (can)h(b)q(e)g(querried)h(b)o(y)f(the)g(call)75 2364 y Fj(MPI)p 160 2364 14 2 v 16 w(TEST)p 290 2364 V 16 w(CANCELLED)p Fq(.)-690 b Fi(?)15 b Fh(\(Oct\))75 2486 y Ff(3.8.2)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 2486 15 2 v 18 w(INTERCOMM)p 936 2486 V 20 w(CREA)l(TE)75 2619 y Fm(Curren)o(t)e(Status:)j Fl(P)o(assed)e(once.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fg(3.8.)34 b(DETERMINING)15 b(WHETHER)h Fj(MPI)e Fg(HAS)i(FINISHED)630 b Fq(9)75 45 y Fj(The)11 b(Problem:)44 b Fq(The)12 b(MPI)e(1.1)g(standard)h(sa)o(ys,)f(in)i(the) f(discussion)h(of)f Fj(MPI)p 1389 45 14 2 v 16 w(INTERCOMM)p 1679 45 V 16 w(CREA)l(TE)p Fq(,)75 102 y(b)q(oth)k(that)189 195 y(The)g(groups)g(m)o(ust)g(b)q(e)g(disjoin)o(t)75 289 y(and)g(that)189 383 y(The)g(leaders)h(ma)o(y)e(b)q(e)i(the)g(same) e(pro)q(cess.)75 477 y(T)l(o)d(further)h(m)o(uddy)g(the)g(w)o(aters,)e (the)i(reason)f(giv)o(en)i(for)e(\\The)g(groups)g(m)o(ust)h(b)q(e)g (disjoin)o(t")g(is)g(based)g(on)75 533 y(concerns)18 b(ab)q(out)g(the)g(implmen)o(tation)g(of)g Fj(MPI)p 922 533 V 15 w(INTERCOMM)p 1211 533 V 17 w(CREA)l(TE)h Fq(that)e(are)g(not) h(applicable)75 590 y(for)d(the)g(case)g(where)h(the)f(leaders)h(are)f (the)g(same)g(pro)q(cess.)75 710 y Fj(The)h(Fix:)45 b Fq(Delete)15 b(the)h(text:)189 804 y(\(the)f(t)o(w)o(o)e(leaders)j (could)h(b)q(e)e(the)h(same)f(pro)q(cess\))75 897 y(from)f(the)i (discussion)h(of)d Fj(MPI)p 610 897 V 16 w(INTERCOMM)p 900 897 V 17 w(CREA)l(TE)p Fq(.)166 954 y(Replace)j(the)e(text:)189 1048 y(All)f(in)o(ter-comm)o(unicator)g(constructors)e(are)h(blo)q(c)o (king)h(and)g(require)g(that)e(the)i(lo)q(cal)g(and)189 1104 y(remote)g(groups)h(b)q(e)h(disjoin)o(t)g(in)g(order)f(to)f(a)o(v) o(oid)h(deadlo)q(c)o(k.)75 1198 y(with)189 1292 y(All)f(in)o(ter-comm)o (unicator)g(constructors)e(are)h(blo)q(c)o(king)h(and)g(require)g(that) e(the)i(lo)q(cal)g(and)189 1348 y(remote)g(groups)h(b)q(e)h(disjoin)o (t.)289 1454 y Fe(A)n(dvic)n(e)e(to)h(users.)40 b Fq(The)14 b(groups)g(m)o(ust)f(b)q(e)i(disjoin)o(t)g(for)e(sev)o(eral)h(reasons.) 19 b(Primar-)289 1511 y(ily)l(,)12 b(this)f(is)g(the)f(in)o(ten)o(t)h (of)f(the)g(in)o(tercomm)o(unicators)h({)f(to)g(pro)o(vide)h(a)f(comm)o (unicator)289 1567 y(for)17 b(comm)o(unication)i(b)q(et)o(w)o(een)g (disjoin)o(t)f(groups.)29 b(This)19 b(is)f(re\015ected)h(in)g(the)g (de\014-)289 1624 y(nition)e(of)f Fj(MPI)p 559 1624 V 15 w(INTERCOMM)p 848 1624 V 17 w(MERGE)p Fq(,)g(whic)o(h)h(allo)o(ws)g (the)f(user)g(to)f(con)o(trol)h(the)289 1680 y(ranking)k(of)g(the)g (pro)q(cesses)g(in)h(the)g(created)f(in)o(tracomm)o(unicator;)h(this)g (ranking)289 1737 y(mak)o(es)c(little)i(sense)f(if)g(the)g(groups)g (are)f(not)h(disjoin)o(t.)28 b(In)18 b(addition,)h(the)f(natural)289 1793 y(extension)c(of)g(collectiv)o(e)i(op)q(erations)e(to)f(in)o (tercomm)o(unicators)h(\(b)q(eing)h(considered)289 1850 y(for)i(MPI-2\))h(mak)o(es)f(the)h(most)g(sense)g(when)h(the)f(groups)g (are)f(disjoin)o(t.)30 b(It)18 b(is)g(the)289 1906 y(in)o(ten)o(t)i(of) g(the)g(MPI)g(de\014nition)i(not)e(to)f(preclude)j(future)f (extensions.)35 b(\()p Fe(End)20 b(of)289 1963 y(advic)n(e)c(to)g (users.)p Fq(\))75 2084 y Ff(3.8.3)49 b(Cla)o(ri\014cation)18 b(of)e(Binding)i(of)f(MPI)p 854 2084 15 2 v 18 w(T)l(yp)q(e)p 971 2084 V 18 w(size)75 2217 y Fm(Curren)o(t)d(Status:)j Fl(P)o(assed)e(once.)166 2321 y Fq(This)i(clari\014cation)h(is)f (needed)h(in)f(the)g(MPI-1)f(description)i(of)e Fj(MPI)p 1369 2321 14 2 v 16 w(T)l(yp)q(e)p 1477 2321 V 17 w(size)p Fq(,)h(since)g(the)g(issue)75 2378 y(rep)q(eatedly)f(arises.)k(It)c(is) f(a)g(clari\014cation)i(of)e(the)g(binding.)189 2484 y Fe(A)n(dvic)n(e)20 b(to)j(users.)76 b Fq(The)21 b(binding)i(of)e Fj(MPI)p 1004 2484 V 16 w(T)l(yp)q(e)p 1112 2484 V 17 w(size)p Fq(,)i(sp)q(eci\014cally)l(,)i(whether)d(the)f(output)189 2540 y(argumen)o(t)e(in)i(C)g(is)f(of)g(t)o(yp)q(e)h Fk(int)f Fq(as)g(in)h(the)f(curren)o(t)h(standard)f(or)f(should)j(b)q (e)f(c)o(hanged)g(to)189 2597 y(something)14 b(else,)i(has)e(b)q(een)i (extensiv)o(ely)f(discussed)h(b)o(y)f(the)g(MPI)f(F)l(orum.)19 b(The)c(\014nal)g(decision)189 2653 y(is)g(to)g(lea)o(v)o(e)g(it)h(an)f Fk(int)p Fq(.)k(\()p Fe(End)d(of)g(advic)n(e)g(to)h(users.)p Fq(\))-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fq(10)952 b Fg(CHAPTER)16 b(3.)29 b(MISCELLANY)17 b(F)o(OR)e(1.2)75 45 y Ff(3.8.4)49 b(Cla)o (ri\014cation)18 b(of)e(MPI)p 627 45 15 2 v 18 w(REDUCE)75 131 y Fq(The)f(curren)o(t)h(text)e(on)h(p.)20 b(115)15 b(lines)i(25-28)d(from)g(MPI-1.1)h(\(June)h(12,)e(1995\))f(sa)o(ys:)166 187 y(The)j Fj(datat)o(yp)q(e)i Fq(argumen)o(t)e(of)f Fj(MPI)p 783 187 14 2 v 16 w(REDUCE)i Fq(m)o(ust)f(b)q(e)g(compatible)i (with)e Fj(op)p Fq(.)23 b(Prede\014ned)18 b(op-)75 244 y(erators)h(w)o(ork)g(only)i(with)f(the)g Fj(MPI)g Fq(t)o(yp)q(es)g (listed)h(in)g(Sec.)35 b(4.9.2)18 b(and)j(Sec.)35 b(4.9.3.)d (User-de\014ned)75 300 y(op)q(erators)14 b(ma)o(y)h(op)q(erate)g(on)g (general,)g(deriv)o(ed)h(datat)o(yp)q(es.)166 357 y(This)g(text)e(is)i (c)o(hanged)g(to:)166 413 y(The)g Fj(datat)o(yp)q(e)i Fq(argumen)o(t)e(of)f Fj(MPI)p 783 413 V 16 w(REDUCE)i Fq(m)o(ust)f(b)q(e)g(compatible)i(with)e Fj(op)p Fq(.)23 b(Prede\014ned)18 b(op-)75 470 y(erators)c(w)o(ork)h(only)h(with)f(the) h Fj(MPI)f Fq(t)o(yp)q(es)g(listed)i(in)f(Sec.)21 b(4.9.2)14 b(and)h(Sec.)21 b(4.9.3.)e(F)l(urthermore,)c(the)75 526 y Fj(datat)o(yp)q(e)20 b Fq(and)e Fj(op)h Fq(giv)o(en)f(for)g (prede\014ned)i(op)q(erators)d(m)o(ust)h(b)q(e)h(the)f(same)g(on)g(all) h(pro)q(cesses.)30 b(User-)75 583 y(de\014ned)15 b(op)q(erators)e(ma)o (y)g(op)q(erate)g(on)h(general,)g(deriv)o(ed)h(datat)o(yp)q(es)e (without)g(the)h(restrictions)g(ab)q(o)o(v)o(e)75 639 y(for)h(prede\014ned)h(op)q(erators.)-681 b Fi(>)15 b Fh(\(Oct\))166 695 y Fq(Note)10 b(that)f(it)i(is)f(p)q(ossible)i(for)e (users)g(to)f(supply)j(di\013eren)o(t)e(user-de\014ned)i(op)q(erations) e(to)g Fj(MPI)p 1750 695 V 16 w(REDUCE)75 752 y Fq(in)23 b(eac)o(h)e(pro)q(cess.)40 b Fj(MPI)21 b Fq(do)q(es)h(not)f(de\014ne)i (whic)o(h)g(op)q(erations)e(are)h(used)g(in)h(this)f(case)f(on)h(whic)o (h)75 808 y(op)q(erands.)189 915 y Fe(A)n(dvic)n(e)e(to)h(users.)71 b Fq(Users)20 b(should)i(mak)o(e)d(no)i(assumptions)f(ab)q(out)g(ho)o (w)g Fj(MPI)p 1638 915 V 16 w(RECUCE)h Fq(is)189 971 y(implemen)o(ted.)29 b(Safest)17 b(is)h(to)f(ensure)i(that)e(the)g (same)h(function)g(is)g(passed)g(to)f Fj(MPI)p 1685 971 V 16 w(REDUCE)189 1028 y Fq(b)o(y)e(eac)o(h)g(pro)q(cess.)20 b(\()p Fe(End)c(of)g(advic)n(e)g(to)h(users.)p Fq(\))-117 1089 y Fi(?)e Fh(\(Oct\))75 1221 y Fn(3.9)59 b(New)20 b(Datat)n(yp)r(es)75 1324 y Ff(3.9.1)49 b(Simple)17 b(Struct)d(T)l(yp)q (es)75 1457 y Fm(Curren)o(t)g(Status:)j Fl(Not)d(v)o(oted)g(on)f(y)o (et.)75 1655 y Fj(MPI)p 160 1655 V 16 w(TYPE)p 293 1655 V 17 w(SIMPLE)p 469 1655 V 15 w(STRUCT\(count,)k(a)o(rra)o(y)p 908 1655 V 14 w(of)p 959 1655 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1334 1655 V 14 w(of)p 1385 1655 V 16 w(t)o(yp)q(es,)e(newt)o(yp)q(e\)) 117 1732 y Fl(IN)155 b Fj(count)482 b Fl(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 1807 y(IN)155 b Fj(a)o(rra)o(y)p 416 1807 V 15 w(of)p 468 1807 V 16 w(blo)q(cklengths)191 b Fl(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h (eac)o(h)g(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 1882 y(IN)155 b Fj(a)o(rra)o(y)p 416 1882 V 15 w(of)p 468 1882 V 16 w(t)o(yp)q(es)327 b Fl(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h (in)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f(handle\))117 1958 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fl(new)15 b(datat)o(yp)q(e)f (\(handle\))166 2082 y Fq(Eac)o(h)c(t)o(yp)q(e)h(in)g(the)g Fj(a)o(rra)o(y)p 592 2082 V 14 w(of)p 643 2082 V 16 w(t)o(yp)q(es)h Fq(list)g(m)o(ust)e(b)q(e)h(a)f Fe(p)n(ortable)h Fq(datat)o(yp)q(e:)17 b(I.e.,)10 b(a)h(basic)g(datat)o(yp)q(e,)f(or)75 2138 y(a)g(datat)o(yp)q(e)f(constructed)i(from)e(a)h(p)q(ortable)h(datat)o (yp)q(e)e(using)i(one)f(of)g(the)g(constructors)g Fj(MPI)p 1681 2138 V 15 w(TYPE)p 1813 2138 V 17 w(CONTIGUOUS,)75 2195 y(MPI)p 160 2195 V 16 w(TYPE)p 293 2195 V 17 w(VECTOR,)16 b(MPI)p 600 2195 V 15 w(TYPE)p 732 2195 V 17 w(INDEXED,)f Fq(or)g Fj(MPI)p 1110 2195 V 16 w(TYPE)p 1243 2195 V 16 w(SIMPLE)p 1418 2195 V 16 w(STRUCT)p Fq(.)166 2251 y(This)22 b(t)o(yp)q(e)g(constructor)f(is)i(similar)f(to)g Fj(MPI)p 988 2251 V 15 w(TYPE)p 1120 2251 V 17 w(STRUCT)p Fq(,)g(except)h(that)e(the)h(user)g(do)q(es)75 2308 y(not)e(supply)h (an)f(arra)o(y)e(of)i(displacemen)o(ts.)35 b(Instead,)22 b(the)e(successiv)o(e)h(blo)q(c)o(ks)f(are)g(assumed)g(to)f(b)q(e)75 2364 y(con)o(tiguous.)g(P)o(adding)11 b(spaces)h(are)e(added,)j (according)e(to)g(the)g(default)h(alignmen)o(t)g(rules)g(for)e (structure)75 2421 y(comp)q(onen)o(ts.)21 b(This)c(facilitates)f(the)g (construction)g(of)f(datat)o(yp)q(es)g(for)g(structures,)g(as)h(the)f (user)h(need)75 2477 y(not)f(compute)g(displacemen)o(ts.)-743 b Fi(>)15 b Fh(\(Oct\))166 2534 y Fq(This)h(can)f(probably)h(b)q(e)g (more)e(elegan)o(tly)i(expressed)g(in)g(terms)f(of)g(exten)o(ts.)-117 2536 y Fi(?)g Fh(\(Oct\))75 2640 y Fb(Example)j(3.1)k Fq(The)14 b(follo)o(wing)g(co)q(de)f(declares)h(a)f(structure)g(t)o(yp) q(e)g(and)h(creates)f(a)g(datat)o(yp)q(e)f(for)h(that)75 2696 y(structure)i(t)o(yp)q(e.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fg(3.9.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPES)1245 b Fq(11)75 45 y Fk(struct)23 b(record)g({)147 102 y(char)g(name;)147 158 y(double)g(position[3];)147 214 y(float)47 b(mass;)147 271 y(})75 384 y(MPI_Datatype)22 b(record_type;)75 440 y(MPI_datatype)g(types[3])h(=)h({MPI_CHAR,)e(MPI_DOUBLE,)h(MPI_FLOAT};) 75 497 y(int)238 b(lengths[3])23 b(=)g({1,)h(3,)f(1};)75 610 y(MPI_Type_simple_struct\()e(3,)i(lengths,)g(types,)g (record_type\);)189 704 y Fe(R)n(ationale.)76 b Fq(The)21 b(co)q(de)h(ab)q(o)o(v)o(e)f(is)h(m)o(uc)o(h)f(simpler)h(and)g(more)e (natural)i(than)f(the)g(curren)o(t)189 760 y(approac)o(h,)14 b(that)h(uses)g Fj(MPI)p 677 760 14 2 v 16 w(TYPE)p 810 760 V 17 w(STRUCT)p Fq(,)g(and)h(requires)g(to)e(compute)h(displacemen) o(ts.)189 834 y(It)i(also)g(pro)o(vides)h(a)f(de\014nition)i(of)e(the)h (record)f(t)o(yp)q(e)h(that)e(is)i(arc)o(hitecture)g(indep)q(enden)o(t) i(\(as-)189 890 y(suming)c(that)e(compilers)j(don't)e(\014ddle)i(with)e (the)h(order)f(of)g(\014elds,)h(whic)o(h)g(is)g(true)f(in)i(C\))d({)i (this)189 947 y(facilitates)f(transfer)e(of)h(records)g(using)h(RMA,)f (an)g(\014ts)g(w)o(ell)h(with)g(v)m(arious)f(prop)q(osals)h(for)e (third)189 1003 y(part)o(y)h(transfers.)19 b(\()p Fe(End)d(of)g(r)n (ationale.)p Fq(\))189 1097 y Fe(A)n(dvic)n(e)h(to)h(users.)50 b Fq(Datat)o(yp)q(es)16 b(constructed)h(using)h Fj(MPI)p 1231 1097 V 16 w(TYPE)p 1364 1097 V 16 w(SIMPLE)p 1539 1097 V 16 w(STRUCT)g Fq(should)189 1154 y(b)q(e)i(used)g(only)g(for)f (structures)g(that)g(use)h(the)g(default)g(alignmen)o(t)g(rules.)34 b(They)20 b(should)g(not)189 1210 y(b)q(e)c(used)h(if)f(alignmen)o(t)g (rules)h(ha)o(v)o(e)f(b)q(een)h(c)o(hanged,)f(using)g(compiler)h (options)f(or)g(compilation)189 1267 y(pragmas.)j(\()p Fe(End)c(of)i(advic)n(e)f(to)g(users.)p Fq(\))189 1361 y Fe(A)n(dvic)n(e)g(to)h(implementors.)47 b Fq(The)16 b(datat)o(yp)q(e)g(constructor)f Fj(MPI)p 1330 1361 V 16 w(TYPE)p 1463 1361 V 17 w(SIMPLE)p 1639 1361 V 15 w(STRUCT)i Fq(is)189 1417 y(most)e(easily)i(implemen)o(ted)h(on)e (systems)g(that)f(use)h Fe(natur)n(al)g Fq(data)g(alignmen)o(t)h (rules.)23 b(On)17 b(suc)o(h)189 1474 y(systems,)12 b(eac)o(h)h(basic)g (datat)o(yp)q(e)f(has)g(an)h(exten)o(t)f(and)h(an)g(alignmen)o(t;)h (the)e(alignmen)o(t)i(of)e(a)g(com-)189 1530 y(p)q(ound)j(t)o(yp)q(e)g (is)h(the)f(strictest)f(alignmen)o(t)i(of)e(a)g(comp)q(onen)o(t.)20 b(P)o(adding)c(is)f(added)g(so)g(that)f(eac)o(h)189 1587 y(comp)q(onen)o(t)f(is)h(aligned)g(at)f(its)g(natural)g(alignmen)o(t.) 20 b(W)l(e)13 b(can)g(then)h(deriv)o(e)g(ob)o(vious)f(de\014nitions)189 1643 y(for)h(the)h(exten)o(t)g(and)h(alignmen)o(t)g(of)e(p)q(ortable)i (MPI)f(datat)o(yp)q(es:)189 1700 y(The)g(exten)o(t)f(and)h(alignmen)o (t)g(of)f(a)h(basic)g(datat)o(yp)q(e)f(is)i(the)e(exten)o(t)h(and)g (alignmen)o(t)g(of)f(the)h(cor-)189 1756 y(resp)q(onding)h(language)f (t)o(yp)q(e.)189 1812 y(The)10 b(alignmen)o(t)h(of)f(a)f(datat)o(yp)q (e)h(built)i(with)e(one)g(of)g(the)g(constructors)g Fj(MPI)p 1487 1812 V 15 w(TYPE)p 1619 1812 V 17 w(CONTIGUOUS,)189 1869 y(MPI)p 274 1869 V 15 w(TYPE)p 406 1869 V 17 w(VECTOR,)26 b Fq(and)e Fj(PI)p 780 1869 V 16 w(TYPE)p 913 1869 V 17 w(INDEXED)h Fq(is)g(the)f(alignmen)o(t)h(of)f(the)h(comp)q(onen)o(t) 189 1925 y(datat)o(yp)q(e;)14 b(the)h(exten)o(t)g(of)g(the)g(comp)q (ound)h(datat)o(yp)q(e)f(is)g(as)g(de\014ned)i(in)f(MPI1.)189 1982 y(The)h(alignmen)o(t)g(of)f(a)g(datat)o(yp)q(e)h(built)h(b)o(y)e (a)h(call)g(to)f Fj(MPI)p 1221 1982 V 16 w(TYPE)p 1354 1982 V 17 w(SIMPLE)p 1530 1982 V 16 w(STRUCT\(n,)h(lens,)189 2038 y(t)o(yp)q(es,)i(newt)o(yp)q(e\))h Fq(is)e(max)643 2045 y Fa(i)672 2038 y Fk(alignment)p Fq(\()p Fk(types)p Fq([)p Fk(i)p Fq(]\);)e(the)i(exten)o(t)g(is)g(computed)h(b)o(y)f(la)o (ying)g(out)189 2095 y(the)h(comp)q(onen)o(ts)g(with)h(minimal)h (padding)f(added)g(so)f(that)g(eac)o(h)h(comp)q(onen)o(t)f(starts)f(at) h(its)189 2151 y(alignmen)o(t.)189 2225 y(Some)g(compilers)h(treat)f (the)g(\014rst)g(comp)q(onen)o(t)g(of)g(a)g(structure)g(di\013eren)o (tly)l(.)33 b(E.g.,)18 b(the)i(IBM)189 2281 y(compiler)d(aligns)h (doubles)f(at)f(w)o(ord)g(b)q(oundaries,)i(but)f(align)g(a)f(structure) g(that)g(starts)f(with)i(a)189 2338 y(double)j(comp)q(onen)o(t)f(at)f (doublew)o(ord)i(b)q(oundaries.)32 b(The)19 b(de\014nition)i(of)d(the)h (alignmen)o(t)h(of)e(a)189 2394 y(datat)o(yp)q(e)h(built)i(with)f Fj(MPI)p 690 2394 V 16 w(TYPE)p 823 2394 V 16 w(SIMPLE)p 998 2394 V 16 w(STRUCT)h Fq(is)f(c)o(hanged,)h(accordingly)l(.)34 b(\()p Fe(End)20 b(of)189 2450 y(advic)n(e)c(to)g(implementors.)p Fq(\))75 2571 y Ff(3.9.2)49 b(Contiguous)17 b(Struct)e(T)l(yp)q(es)75 2704 y Fm(Curren)o(t)f(Status:)j Fl(F)m(ailed)c(6-7-10)f(at)i(Septem)o (b)q(er)g(meeting.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fq(12)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)166 45 y Fq(Data)j(structures)i(ma)o(y)f (con)o(tain)h(padding)g(holes,)i(added)e(for)f(alignmen)o(t.)34 b(The)20 b(de\014nition)h(of)75 102 y(an)f(MPI)g(datat)o(yp)q(e,)g (done)g(using)g(the)g Fj(MPI)p 865 102 14 2 v 16 w(TYPE)p 998 102 V 17 w(STRUCT)g Fq(constructor,)g(do)q(es)g(not)g(distinguish) 75 158 y(b)q(et)o(w)o(een)15 b(holes)g(that)f(con)o(tains)g(no)h (signi\014can)o(t)g(data,)f(and)h(are)f(there)h(only)g(for)f(alignmen)o (t)h(purp)q(oses,)75 214 y(and)k(b)q(et)o(w)o(een)g(holes)g(that)f(ma)o (y)f(con)o(tain)i(signi\014can)o(t)g(data)f(and)h(w)o(ere)f(purp)q (osefully)j(left)e(out)f(b)o(y)g(a)75 271 y(user)k(that)e(wishes)i(not) f(to)g(o)o(v)o(erwrite)g(some)g(of)g(the)g(information)h(in)g(a)f (structure.)38 b(As)22 b(a)f(result,)75 327 y(MPI)g(cannot)g(o)o(v)o (erwrite)f(these)h(holes,)i(this)f(prev)o(en)o(ts)e(the)i(ob)o(vious)f (optimization)h(of)e(shipping)j(a)75 384 y(data)13 b(structure)h(as)g (a)f(con)o(tiguous)h(blo)q(c)o(k)h(of)f(data,)f(padding)i(holes)f (included.)22 b(The)15 b(t)o(yp)q(e)f(constructor)75 440 y Fj(MPI)p 160 440 V 16 w(TYPE)p 293 440 V 17 w(CONTIGUOUS)p 598 440 V 18 w(STRUCT)d Fq(marks)f(an)o(y)g(suc)o(h)h(holes)g(as)g (\\don)o(t-care")e(lo)q(cations,)j(allo)o(wing)75 497 y(the)j(implemen)o(tation)i(to)d(o)o(v)o(erwrite)h(them.)75 648 y Fj(MPI)p 160 648 V 16 w(TYPE)p 293 648 V 17 w(CONTIGUOUS)p 598 648 V 18 w(STRUCT\(count,)j(a)o(rra)o(y)p 1041 648 V 14 w(of)p 1092 648 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1468 648 V 14 w(of)p 1519 648 V 16 w(displacements,)g(a)o(r-)75 704 y(ra)o(y)p 136 704 V 15 w(of)p 188 704 V 16 w(t)o(yp)q(es,)e(newt)o (yp)q(e\))117 781 y Fl(IN)155 b Fj(count)482 b Fl(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 856 y(IN)155 b Fj(a)o(rra)o(y)p 416 856 V 15 w(of)p 468 856 V 16 w(blo)q(cklengths) 191 b Fl(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h(eac)o(h)g(blo)q (c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 932 y(IN)155 b Fj(a)o(rra)o(y)p 416 932 V 15 w(of)p 468 932 V 16 w(displacements)164 b Fl(b)o(yte)14 b(displacemen)o(t)g(of)f(eac)o(h)h(blo)q(c)o(k)g (\(arra)o(y)g(of)f(in)o(teger\))117 1007 y(IN)155 b Fj(a)o(rra)o(y)p 416 1007 V 15 w(of)p 468 1007 V 16 w(t)o(yp)q(es)327 b Fl(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h(in)f(eac)o(h)h(blo)q(c)o(k)g (\(arra)o(y)g(of)f(handle\))117 1082 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fl(new)15 b(datat)o(yp)q(e)f(\(handle\))166 1206 y Fq(The)c(seman)o(tics)h(of)e Fj(MPI)p 587 1206 V 16 w(TYPE)p 720 1206 V 17 w(CONTIGUOUS)p 1025 1206 V 18 w(STRUCT)i Fq(are)f(iden)o(tical)i(to)d Fj(MPI)p 1611 1206 V 16 w(TYPE)p 1744 1206 V 17 w(STRUCT)p Fq(,)75 1263 y(except)15 b(that,)e(if)h(a)g(receiv)o(e)h(call)g(uses)g(a)e(datat)o(yp)q(e)h (constructed)g(with)h(this)f(constructor,)f(then)i(the)f(in-)75 1319 y(ternal)f(holes)h(in)f(the)g(structure)g(can)g(b)q(e)g(o)o(v)o (erwritten.)18 b(The)13 b(same)g(rule)h(\(namely)l(,)f(that)f(holes)i (b)q(et)o(w)o(een)75 1376 y(comp)q(onen)o(ts)c(can)g(b)q(e)h(o)o(v)o (erwritten\))e(applies)j(also)e(to)g(datat)o(yp)q(es)f(constructed)h (using)h Fj(MPI)p 1635 1376 V 16 w(TYPE)p 1768 1376 V 17 w(SIMPLE)p 1944 1376 V 16 w(STRUCT)p Fq(.)166 1479 y Fm(Discussion:)29 b Fl(It)10 b(is)f(reasonable)g(to)g(assume)g(that)h (holes)f(in)g(la)o(y)o(outs)f(de\014ned)j(using)e Fd(MPI)p 1576 1479 13 2 v 14 w(TYPE)p 1698 1479 V 14 w(SIMPLE)p 1857 1479 V 15 w(STRUCT)75 1536 y Fl(con)o(tain)14 b(no)h(signi\014can) o(t)f(information,)e(since)k(this)f(datat)o(yp)q(e)g(constructor)h(is)f (exp)q(ected)i(to)e(b)q(e)g(used)h(for)f(sp)q(ec-)75 1592 y(ifying)f(the)h(la)o(y)o(out)f(of)h(structures.)24 b(Users)16 b(will)e(b)q(e)i(required)g(to)f(use)h Fd(MPI)p 1271 1592 V 14 w(TYPE)p 1393 1592 V 14 w(STRUCT)f Fl(when)g(holes)h (hold)75 1649 y(signi\014can)o(t)d(information.)189 1802 y Fe(A)n(dvic)n(e)e(to)i(users.)38 b Fq(Users)11 b(should)i(arrange)d (the)i(\014elds)g(in)g(a)f(C)h(struct)e(t)o(yp)q(e)i(in)g(descending)h (order)189 1859 y(of)i(their)i(size.)24 b(This)17 b(arrangemen)o(t)e (has)h(t)o(w)o(o)f(adv)m(an)o(tages:)21 b(\014rstly)c(it)f(ma)o(y)g (reduce)h(the)f(size)h(of)189 1915 y(the)i(struct)f(b)o(y)h (eliminating)i(padding)f(b)q(et)o(w)o(een)f(elemen)o(ts)h(with)f (di\013eren)o(t)g(alignmen)o(ts)h(\(this)189 1972 y(is)c(generally)i (true)e(for)g(C)g(programs\).)22 b(Secondly)l(,)c(it)e(ma)o(y)g (increase)h(the)g(n)o(um)o(b)q(er)f(of)g(cases)h(for)189 2028 y(whic)o(h)d(the)f(implemen)o(tation)h(can)g(p)q(erform)f(blo)q(c) o(k)g(copies)i(b)o(y)e(ensuring)h(that)e(the)h(\014rst)g(elemen)o(t)189 2085 y(of)19 b(the)g(structure)g(has)h(the)f(most)g(stringen)o(t)g (alignmen)o(t)h(requiremen)o(ts.)33 b(\()p Fe(End)19 b(of)i(advic)n(e)f(to)189 2141 y(users.)p Fq(\))189 2247 y Fe(A)n(dvic)n(e)15 b(to)i(implementors.)39 b Fq(An)16 b(implemen)o(tation)g(ma)o(y)f(ignore)g(the)h(constructor)189 2304 y Fj(MPI)p 274 2304 14 2 v 15 w(TYPE)p 406 2304 V 17 w(CONTIGUOUS)p 711 2304 V 18 w(STRUCT)p Fq(,)21 b(and)f(handle)i(it)e(exactly)h(as)f Fj(MPI)p 1539 2304 V 16 w(TYPE)p 1672 2304 V 17 w(STRUCT)p Fq(.)189 2360 y(Th)o(us,)f(there)g(is)h(\(almost\))d(no)i(o)o(v)o(erhead)g(in)g(supp) q(orting)h(this)f(constructor,)g(without)g(taking)189 2417 y(adv)m(an)o(tage)f(of)h(it.)32 b(One)20 b(p)q(ossible)h(w)o(a)o (y)d(of)h(taking)g(adv)m(an)o(tage)g(of)g(it,)h(is)g(to)e(adopt)h(a)g (canoni-)189 2473 y(cal)d(\\wire")g(represen)o(tation)g(for)g (structures.)22 b(Supp)q(ose)17 b(that)f(the)g(compiler)h(uses,)g(b)o (y)f(default,)189 2530 y(natural)g(alignmen)o(t)i(rules)f(for)f (structures:)23 b(eac)o(h)16 b(basic)i(comp)q(onen)o(t)e(is)i(aligned)g (at)e(a)g(natural)189 2586 y(b)q(oundary)l(.)k(Supp)q(ose)14 b(that)f(one)h(adopts)f(natural)h(alignmen)o(t)g(rules)h(for)e (messages)g(on)g(the)h(wire,)189 2642 y(in)19 b(a)f(homogeneous)g(en)o (vironmen)o(t:)27 b(messages)18 b(are)g(padded)h(so)f(that)g(eac)o(h)g (basic)h(elemen)o(t)g(is)189 2699 y(aligned)d(at)f(a)g(natural)g(b)q (oundary)l(.)20 b(Then)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fg(3.9.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPES)1245 b Fq(13)189 45 y(\(1\))14 b(A)i(receiv)o(er)g(can)f(alw)o(a)o(ys)g (deco)q(de)h(an)g(incoming)h(message)d(and)i(retriev)o(e)g(correctly)f (the)h(ba-)189 102 y(sic)k(comp)q(onen)o(ts.)31 b(This)20 b(b)q(ecause)g(the)g(basic)g(datat)o(yp)q(e)e(of)h(the)g(elemen)o(t)h (indicates)h(at)e(what)189 158 y(b)q(oundary)c(it)h(is)f(aligned.)189 214 y(\(2\))f(A)h(sender)g(can)h(blo)q(c)o(k)f(cop)o(y)g(a)g(structure) g(on)o(to)f(the)h(\\wire")g(\(most)f(lik)o(ely)l(,)j(on)o(to)d(a)g (message)189 271 y(bu\013er\),)19 b(when)g(the)g(structure)g(is)g (naturally)h(aligned,)h Fe(r)n(elative)e(to)h(the)g(start)g(of)f(the)h (sending)189 327 y(bu\013er)p Fq(.)26 b(This)18 b(optimization)g(do)q (es)g(not)e(require)j(that)d(the)h(sending)i(datat)o(yp)q(e)e(b)q(e)h (built)g(using)189 384 y Fj(MPI)p 274 384 14 2 v 15 w(TYPE)p 406 384 V 17 w(CONTIGUOUS)p 711 384 V 18 w(STRUCT)p Fq(:)c(The)g(MPI)f (library)i(can)e(scan)h(datat)o(yp)q(es,)f(when)h(they)189 440 y(are)h(committed,)f(and)i(recognize)g(when)g(a)e(datat)o(yp)q(e)h (\(or)f(a)h(datat)o(yp)q(e)g(fragmen)o(t\))f(is)h(naturally)189 497 y(aligned.)24 b(Blo)q(c)o(k)17 b(cop)o(ying)g(can)g(b)q(e)g(used)g (ev)o(en)f(if)h(the)g(holes)g(in)g(the)f(datat)o(yp)q(e)g(con)o(tain)g (signi\014-)189 553 y(can)o(t)e(information.)189 610 y(\(3\))h(A)h(receiv)o(er)h(can)f(blo)q(c)o(k)h(cop)o(y)f(an)g (incoming)i(message)d(on)o(to)h(the)g(receiv)o(e)h(bu\013er)f(when)h (the)189 666 y(receiving)e(structure)f(is)h(naturally)g(aligned)h Fe(r)n(elative)f(to)g(the)h(start)g(of)f(the)h(r)n(e)n(c)n(eiving)d (bu\013er)i Fq(and)189 723 y(holes)10 b(can)h(b)q(e)g(ignored.)18 b(I.e.,)11 b(when)g(the)f(receiving)i(datat)o(yp)q(e)d(is)i(built)g (using)g Fj(MPI)p 1597 723 V 16 w(TYPE)p 1730 723 V 17 w(CONTIGUOUS)p 2035 723 V 18 w(STRUCT)189 779 y Fq(or)i Fj(MPI)p 328 779 V 16 w(TYPE)p 461 779 V 17 w(SIMPLE)p 637 779 V 16 w(STRUCT)h Fq(and)g(is)h(naturally)f(aligned,)i(relativ)o (e)e(to)g(its)g(\014rst)f(p)q(osition.)189 853 y(The)k(same)h(wire)g (represen)o(tation)f(ma)o(y)g(b)q(e)h(used)g(in)h(a)e(heterogenous)h (en)o(vironmen)o(t,)g(in)g(cases)189 909 y(where)h(data)e(con)o(v)o (ersion)i(is)g(done)g(at)f(the)h(destination;)i(the)e(destination)g (has)g(to)f(kno)o(w,)g(not)189 966 y(only)c(the)h(basic)g(datat)o(yp)q (es)e(used)i(b)o(y)f(the)h(source,)f(but)g(also)h(the)f(alignmen)o(t)h (rules)g(used)g(b)o(y)f(the)189 1022 y(source.)189 1096 y(Example:)34 b(Consider)23 b(a)f(C)g(structure)g Fk(struct)h({char)g (a[2];)g(float)g(b[2];)h(double)f(c})p Fq(;)189 1153 y(assume)16 b(that)g(\015oats)g(are)g(4)h(b)o(ytes)f(and)h(doubles)h (are)e(8)g(b)o(ytes,)h(and)g(structures)f(are)g(naturally)189 1209 y(aligned.)36 b(The)20 b(memory)f(la)o(y)o(out)h(is)44 b Fk([c][c]xx[ffff][ffff]xxxx[dddd)o(dddd])p Fq(,)18 b(and)i(the)189 1266 y(structure)15 b(is)g(aligned)i(to)d(an)i(8)e(b)o (yte)h(b)q(oundary)l(.)189 1340 y(Case)c(1:)18 b(The)12 b(en)o(tire)g(structure)f(is)h(comm)o(unicated.)20 b(Sender)12 b(constructs)f(a)h(datat)o(yp)q(e)f(for)g(struc-)189 1396 y(ture)f(using)h Fj(MPI)p 479 1396 V 15 w(TYPE)p 611 1396 V 17 w(STRUCT,)g(MPI)p 910 1396 V 15 w(TYPE)p 1042 1396 V 17 w(SIMPLE)p 1218 1396 V 16 w(STRUCT)g Fq(or)f Fj(MPI)p 1554 1396 V 15 w(TYPE)p 1686 1396 V 17 w(CONTIGUOUS)p 1991 1396 V 18 w(STRUCT)p Fq(;)189 1452 y(receiv)o(er)16 b(constructs)g(a)g(datat)o(yp)q(e)g(for)f(this)i(structure,)e(using)i Fj(MPI)p 1379 1452 V 16 w(CONTIGUOUS)p 1683 1452 V 18 w(STRUCT)189 1509 y Fq(or)12 b Fj(MPI)p 327 1509 V 16 w(TYPE)p 460 1509 V 16 w(SIMPLE)p 635 1509 V 16 w(STRUCT)p Fq(.)h(Then)g(the)g(la)o(y)o(outs)f(in)h(the)g(sender)g(memory)l(,)g (on)f(the)h(wire,)189 1565 y(and)i(in)h(the)f(receiv)o(er)h(memory)f (are)g(iden)o(tical,)h(and)g(data)e(can)i(b)q(e)g(simply)g(copied.)189 1639 y(Case)10 b(2:)17 b(comm)o(unication)12 b(in)o(v)o(olv)o(es)f (only)h(the)f(second)g(of)f(the)h(t)o(w)o(o)f(c)o(haracters,)g(and)h (the)g(remain-)189 1696 y(der)f(of)g(the)g(structure.)18 b(Receiv)o(er)11 b(constructs)f(the)g(datat)o(yp)q(e)g(using)h Fj(MPI)p 1441 1696 V 16 w(CONTIGUOUS)p 1745 1696 V 18 w(STRUCT)p Fq(.)189 1752 y(The)19 b(memory)f(la)o(y)o(out)g(is)h Fk([c]xx[ffff][ffff]xxxx[dddddd)o(dd])p Fq(,)d(and)j(the)g(wire)g(la)o (y)o(out)f(is)189 1809 y Fk([c]xxx[ffff][ffff]xxxx[)o(dddddddd)o(])p Fq(.)e(The)c(datat)o(yp)q(e)f(is)h(not)g(naturally)g(aligned,)h(so)f (that)189 1865 y(blo)q(c)o(k)k(cop)o(ying)g(cannot)f(b)q(e)h(used)g(at) f(the)h(sender)g(or)f(the)g(receiv)o(er.)22 b(Ho)o(w)o(ev)o(er,)14 b(a)h(smart)f(imple-)189 1922 y(men)o(tation)h(ma)o(y)g(realize)i(that) e(the)h(fragmen)o(t)f(starting)g(from)g(the)h(\015oat)f(is)i(naturally) f(aligned,)189 1978 y(relativ)o(e)e(to)e(the)i(start)e(of)h(the)g (bu\013er,)h(so)f(that)f(blo)q(c)o(k)j(cop)o(ying)e(can)h(b)q(e)g(used) g(for)f(this)g(fragmen)o(t.)189 2052 y(Case)f(3:)19 b(comm)o(unication) 14 b(in)o(v)o(olv)o(es)g(only)f(the)g(t)o(w)o(o)f(\015oats)h(and)g(the) g(double.)21 b(The)13 b(memory)g(la)o(y-)189 2108 y(out)c(is)i Fk([ffff][ffff]xxxx[dddddddd])o Fq(.)16 b(The)10 b(la)o(y)o(out)g(on)g (the)g(wire)g(is)h Fk([ffff][ffff][dddddddd])p Fq(.)189 2165 y(The)j(datat)o(yp)q(e)f(is)i(not)f(naturally)g(aligned,)i (relativ)o(e)e(to)g(its)g(start,)e(so)i(that)f(blo)q(c)o(k)i(cop)o (ying)g(can-)189 2221 y(not)f(b)q(e)i(used.)21 b(\()p Fe(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))166 2360 y Fm(Alternativ)o(es)o(:)134 2432 y Fl(1.)22 b(No)14 b(new)h(function)f Fd(MPI)p 581 2432 13 2 v 14 w(TYPE)p 703 2432 V 15 w(CONTIGUOUS)p 985 2432 V 12 w(STRUCT)p Fl(:)g(instead,)g(holes)h(can)g(b)q(e)g(o)o(v)o(erwritten)g(only)189 2482 y(for)f(datat)o(yp)q(es)i(built)e(with)g Fd(MPI)p 717 2482 V 15 w(TYPE)p 840 2482 V 14 w(SIMPLE)p 999 2482 V 15 w(STRUCT)p Fl(.)f(The)j(argumen)o(t)e(is)g(that,)h(at)g(least)g (in)f(the)189 2532 y(scenario)f(I)g(presen)o(ted,)i(hole)d(o)o(v)o (erwriting)h(is)g(tak)o(en)g(adv)n(an)o(tage)f(of)g(only)g(for)h (structures)i(with)e(a)f(natural)189 2581 y(alignmen)o(t.)j(But,)f (then,)g(their)h(la)o(y)o(out)d(can)i(b)q(e)h(sp)q(eci\014ed)g(using)f Fd(MPI)p 1308 2581 V 14 w(TYPE)p 1430 2581 V 15 w(SIMPLE)p 1590 2581 V 14 w(STRUCT)p Fl(.)134 2654 y(2.)22 b(Both)d(sender)h(and)f (receiv)o(er)h(are)g(required)f(to)g(use)h Fd(MPI)p 1126 2654 V 14 w(TYPE)p 1248 2654 V 14 w(CONTIGUOUS)p 1529 2654 V 13 w(STRUCT)p Fl(,)e(and)g(the)189 2704 y(datat)o(yp)q(es)c(on)f (b)q(oth)h(ends)g(ha)o(v)o(e)g(to)f(b)q(e)h(structurally)g(equiv)n (alen)o(t,)f(i.e.,)f(de\014ned)j(b)o(y)e(the)h(same)f(sequence)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fq(14)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fl(of)e(de\014nitions.)18 b(There)d(are)f(sev)o(eral)h(p)q(ossible)f(v)n(arian)o(ts)f(for)g(suc)o (h)i(prop)q(osal)189 107 y(\(a\))20 b(This)g(construct)i(can)e(b)q(e)h (only)e(used)i(in)f(a)g(homogeneous)f(en)o(vironmen)o(t.)36 b(But,)22 b(then,)g(w)o(e)e(lose)189 157 y(p)q(ortabilit)o(y)11 b(to)h(heterogeneous)j(en)o(vironmen)o(t,)c(and)i(w)o(e)g(ac)o(hiev)o (e)f(little)g(that)h(could)f(not)h(b)q(e)g(ac)o(hiev)o(ed)g(b)o(y)189 207 y(using)g Fd(MPI)p 374 207 13 2 v 15 w(BYTE)p Fl(.)189 269 y(\(b\))18 b(This)g(construct)i(can)e(b)q(e)g(used)h(p)q(ortably)m (,)f(with)g(enhanced)h(p)q(erformance)f(obtained)g(in)f(a)h(homo-)189 319 y(geneous)i(en)o(vironmen)o(t.)36 b(E.g.,)20 b(a)g(homogeneous)f (implem)o(en)o(tation)e(will)i(use)h(blo)q(c)o(k)g(cop)o(ying,)g(while) 189 369 y(a)d(heterogeneous)j(implemen)o(tatio)o(n)15 b(will)i(mo)o(v)o(e)f(elemen)o(t)h(b)o(y)h(elemen)o(t)f(\(at)h(least,)g (for)g(heterogeneous)189 419 y(comm)o(unicators\).)j(This)16 b(assumes)g(that,)g(in)f(a)h(homogeneous)e(en)o(vironmen)o(t,)h (di\013eren)o(t)i(pro)q(cesses)i(are)189 469 y(compiled)8 b(with)i(the)g(same)f(data)h(alignmen)o(t)d(options.)17 b(Also,)10 b(this)g(runs)g(coun)o(ter)h(the)g(datat)o(yp)q(e)f(matc)o (hing)189 518 y(rules)k(in)g(MPI,)f(where)i(only)e(signature)i(matc)o (hing)d(is)h(required.)189 581 y(\(c\))d Fd(MPI)p 326 581 V 14 w(TYPE)p 448 581 V 14 w(CONTIGUOUS)p 729 581 V 13 w(STRUCT)f Fl(can)g(b)q(e)i(matc)o(hed)d(with)h(other)i(datat)o (yp)q(es,)f(as)g(long)e(as)i(signa-)189 630 y(tures)g(matc)o(h;)f (impro)o(v)o(ed)f(p)q(erformance)h(is)g(ac)o(hiev)o(ed)h(when)f(b)q (oth)h(ends)g(use)g Fd(MPI)p 1463 630 V 14 w(TYPE)p 1585 630 V 15 w(CONTIGUOUS)p 1867 630 V 12 w(STRUCT)p Fl(.)189 680 y(But,)i(then,)h(the)g(wire)g(proto)q(col)f(m)o(ust)f(b)q(e)i(so)f (that)h(individual)d(elemen)o(ts)i(can)h(b)q(e)f(extracted,)i(in)e (case)h(the)189 730 y(receiv)o(er)k(do)q(es)f(not)f(use)i Fd(MPI)p 664 730 V 14 w(TYPE)p 786 730 V 14 w(CONTIGUOUS)p 1067 730 V 13 w(STRUCT)p Fl(.)d(In)i(this)f(case,)i(asking)e(the)h (sender)h(to)189 780 y(use)g Fd(MPI)p 340 780 V 14 w(TYPE)p 462 780 V 15 w(CONTIGUOUS)p 744 780 V 12 w(STRUCT)f Fl(do)q(es)h(not)g (pro)o(vide)f(additional)f(function)h(o)o(v)o(er)h(the)g(main)189 830 y(prop)q(osal.)g(The)d(di\013erence)h(is)f(only)e(whether)j(the)f (use)g(asserts)i(that)d(the)h(sending)g(datat)o(yp)q(e)f(has)h(a)f (nat-)189 879 y(ural)i(la)o(y)o(out)g(\(using)h Fd(MPI)p 608 879 V 14 w(TYPE)p 730 879 V 14 w(CONTIGUOUS)p 1011 879 V 13 w(STRUCT)p Fl(\),)f(or)h(whether)h(MPI)f(disco)o(v)o(ers)h (that)f(fact)189 929 y(b)o(y)d(itself.)75 1123 y Ff(3.9.3)49 b(Convenient)17 b(F)o(o)o(rm)d(of)j(MPI)p 731 1123 15 2 v 18 w(T)l(yp)q(e)p 848 1123 V 18 w(indexed)p 1018 1123 V 18 w(blo)q(ck)75 1256 y Fm(Curren)o(t)d(Status:)j Fl(P)o(assed)e(once.)166 1360 y Fq(This)d(prop)q(osal)g(is)g(to)f(allo) o(w)h(constan)o(t)f(blo)q(c)o(ksize)i(and)f(arbitrary)f(displacemen)o (ts)i(in)f(the)g(in)o(terface)75 1416 y(to)e Fj(MPI)p 211 1416 14 2 v 15 w(TYPE)p 343 1416 V 17 w(INDEXED)p Fq(.)g(The)g(routine)h(no)o(w)f(tak)o(es)f Fj(a)o(rra)o(y)p 1114 1416 V 14 w(of)p 1165 1416 V 16 w(blo)q(cklengths)k Fq(and)d Fj(a)o(rra)o(y)p 1605 1416 V 15 w(of)p 1657 1416 V 16 w(displacements)75 1473 y Fq(as)17 b(argumen)o(ts.)27 b(There)18 b(are)f(man)o(y)h(co)q(des)g(using)g(indirect)i(addressing)e (arising)g(from)f(unstructured)75 1529 y(grids)f(where)h(the)f(blo)q(c) o(ksize)i(is)e(alw)o(a)o(ys)f(1)h(\(gather/scatter\).)k(W)l(e)c(prop)q (ose)g(this)h(in)o(terface,)f(allo)o(wing)75 1586 y(for)f(constan)o(t)f (blo)q(c)o(ksize)j(and)e(arbitrary)g(displacemen)o(ts.)75 1737 y Fj(MPI)p 160 1737 V 16 w(TYPE)p 293 1737 V 17 w(INDEXED)p 505 1737 V 16 w(BLOCK\()g(count,)i(blo)q(cklength,)h(a)o (rra)o(y)p 1180 1737 V 14 w(of)p 1231 1737 V 16 w(displacements,)g (oldt)o(yp)q(e,)f(newt)o(yp)q(e\))117 1870 y Fl(IN)155 b Fj(count)482 b Fl(length)14 b(of)f(arra)o(y)h(of)f(displacemen)o(ts) 117 1945 y(IN)155 b Fj(blo)q(cklength)371 b Fl(size)15 b(of)e(blo)q(c)o(k)117 2021 y(IN)155 b Fj(a)o(rra)o(y)p 416 2021 V 15 w(of)p 468 2021 V 16 w(displacements)164 b Fl(arra)o(y)14 b(of)f(displacemen)o(ts)117 2096 y(IN)155 b Fj(oldt)o(yp)q(e)450 b Fl(old)13 b(datat)o(yp)q(e)117 2171 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fl(new)15 b(datat)o(yp)q(e)75 2295 y Fk(int)23 b(MPI)p 245 2295 15 2 v 17 w(Type)p 358 2295 V 17 w(Indexed)p 543 2295 V 16 w(Block\()g(int)h(count,)f(int) g(blocklength,)393 2352 y(int*)g(array)p 635 2352 V 17 w(of)p 700 2352 V 17 w(displacements,)f(MPI)p 1147 2352 V 17 w(Datatype)g(oldtype,)393 2408 y(MPI)p 468 2408 V 17 w(Datatype*)h(newtype)f(\))75 2495 y(MPI)p 150 2495 V 17 w(TYPE)p 263 2495 V 16 w(INDEXED)p 447 2495 V 17 w(BLOCK\(COUNT,)g(BLOCKLENGTH,)g(ARRAY)p 1204 2495 V 17 w(OF)p 1269 2495 V 16 w(DISPLACEMENTS,)g(OLDTYPE,)393 2551 y(NEWTYPE,)h(IERROR\))170 2608 y(INTEGER)g(COUNT,)g(INTEGER)g (BLOCKLENGTH,)g(ARRAY)p 1153 2608 V 16 w(OF)p 1217 2608 V 17 w(DISPLACEMENT\(*\),)f(OLDTYPE,)170 2664 y(NEWTYPE,)h(IERROR)-117 2700 y Fi(>)15 b Fh(\(Oct\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fg(3.10.)34 b(MINOR)16 b(CORRECTIONS)1109 b Fq(15)75 45 y Fk(int)23 b(MPI::Datatype::Indexed)p 701 45 15 2 v 15 w(Block\()g(int)g(count,)g(int)h(blocklength,)393 102 y(const)f(int)h(array)p 755 102 V 16 w(of)p 819 102 V 17 w(displacements[],)393 158 y(const)f(MPI::Datatype&)f(oldtype)h (\))1875 198 y Fi(?)16 b Fh(\(Oct\))75 331 y Fn(3.10)59 b(Mino)n(r)21 b(Co)n(rrections)75 480 y Fm(Curren)o(t)14 b(Status:)j Fl(P)o(assed)e(once.)166 584 y Fq(The)i(follo)o(wing)i (corrections)e(to)g Fj(MPI-1.1)f Fq(are)h(\(all)h(page)f(and)g(line)i (n)o(um)o(b)q(ers)f(are)f(for)g(the)g(June)75 640 y(12,)d(1995)g(v)o (ersion)i(without)f(c)o(hangebars\):)143 746 y Fi(\017)23 b Fq(P)o(age)14 b(19,)g(lines)j(1-2)e(reads)189 803 y(for)f(\(64)g (bit\))i(C)f(in)o(tegers)g(declared)i(to)d(b)q(e)i(of)f(t)o(yp)q(e)g Fj(longlong)g(int)189 859 y Fq(but)g(should)h(read)189 916 y(for)e(\(64)g(bit\))i(C)f(in)o(tegers)g(declared)i(to)d(b)q(e)i (of)f(t)o(yp)q(e)g Fj(long)g(long)g(int)189 1088 y Fm(Discussion)o(:) 189 1156 y Fl(Just)f(to)g(mak)o(e)f(it)g(correct)j(C)d(so)h(it)g(is)g (long)f(long.)1875 1212 y Fi(>)j Fh(\(Oct\))143 1297 y Fi(\017)23 b Fq(P)o(age)14 b(41,)g(lines)j(16-18)d(reads)189 1354 y(A)19 b Fb(empt)o(y)g Fq(status)g(is)h(a)f(status)g(whic)o(h)i (is)f(set)f(to)g(return)h Fj(tag)g(=)g(MPI)p 1460 1354 14 2 v 16 w(ANY)p 1568 1354 V 17 w(T)l(A)o(G)p Fq(,)f Fj(source)i(=)189 1410 y(MPI)p 274 1410 V 15 w(ANY)p 381 1410 V 18 w(SOURCE)p Fq(,)11 b(and)g(is)h(also)e(in)o(ternally)j (con\014gured)e(so)g(that)f(calls)h(to)g Fj(MPI)p 1601 1410 V 15 w(GET)p 1704 1410 V 17 w(COUNT)189 1467 y Fq(and)k Fj(MPI)p 362 1467 V 16 w(GET)p 466 1467 V 17 w(ELEMENTS)g Fq(return)g Fj(count)i(=)e(0)p Fq(.)189 1523 y(but)g(should)h(read)189 1580 y(A)j Fb(empt)o(y)g Fq(status)g(is)h(a)f(status)g(whic)o(h)i(is)f (set)f(to)g(return)h Fj(tag)g(=)g(MPI)p 1460 1580 V 16 w(ANY)p 1568 1580 V 17 w(T)l(A)o(G)p Fq(,)f Fj(source)i(=)189 1636 y(MPI)p 274 1636 V 15 w(ANY)p 381 1636 V 18 w(SOURCE)p Fq(,)c Fj(erro)o(r)e(=)i(MPI)p 842 1636 V 16 w(SUCCESS)p Fq(,)h(and)f(is)g(also)g(in)o(ternally)h(con\014gured)f(so)g(that)189 1693 y(calls)f(to)e Fj(MPI)p 430 1693 V 16 w(GET)p 534 1693 V 17 w(COUNT)i Fq(and)f Fj(MPI)p 893 1693 V 16 w(GET)p 997 1693 V 17 w(ELEMENTS)g Fq(return)g Fj(count)i(=)e(0)p Fq(.)189 1865 y Fm(Discussion)o(:)189 1933 y Fl(Filling)d(in)h(the)h (error)h(\014eld)f(got)g(forgotten.)1875 1989 y Fi(?)i Fh(\(Oct\))143 2074 y Fi(\017)23 b Fq(P)o(age)14 b(71,)g(line)j(10)e (reads)189 2131 y(and)g(do)g(not)g(a\013ect)f(the)i(the)f(con)o(ten)o (t)g(of)f(a)h(message)189 2187 y(but)g(should)h(read)189 2244 y(and)f(do)g(not)g(a\013ect)f(the)i(con)o(ten)o(t)e(of)h(a)g (message)189 2416 y Fm(Discussion)o(:)189 2484 y Fl(This)e(is)h(a)g (trivial)e(t)o(yp)q(o)i(of)f(ha)o(ving)g(the)h(2)g(times.)143 2626 y Fi(\017)23 b Fq(P)o(age)14 b(74,)g(lines)j(39-45)d(read)189 2682 y(A)i(datat)o(yp)q(e)f(ma)o(y)g(sp)q(ecify)i(o)o(v)o(erlapping)g (en)o(tries.)22 b(The)16 b(use)g(of)g(suc)o(h)g(a)f(datat)o(yp)q(e)h (in)g(a)g(receiv)o(e)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Fq(16)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fq(op)q(eration)f(is)h (erroneous.)k(\(This)14 b(is)h(erroneous)f(ev)o(en)g(if)h(the)f(actual) g(message)g(receiv)o(ed)h(is)g(short)189 102 y(enough)g(not)g(to)g (write)g(an)o(y)g(en)o(try)g(more)f(than)h(once.\))189 174 y(A)i(datat)o(yp)q(e)g(ma)o(y)g(sp)q(ecify)i(o)o(v)o(erlapping)g (en)o(tries.)27 b(If)18 b(suc)o(h)g(a)g(datat)o(yp)q(e)f(is)h(used)g (in)h(a)e(receiv)o(e)189 231 y(op)q(eration,)d(that)h(is,)g(if)g(some)g (part)f(of)h(the)f(receiv)o(e)i(bu\013er)f(is)h(written)f(more)f(than)h (once)g(b)o(y)g(the)189 287 y(receiv)o(e)h(op)q(eration,)f(then)g(the)h (call)g(is)g(erroneous.)189 359 y(The)i(\014rst)f(part)g(w)o(as)h(an)f Fj(MPI-1.1)g Fq(addition.)29 b(The)18 b(second)g(part)f(o)o(v)o(erlaps) h(with)g(it.)28 b(The)18 b(old)189 416 y(text)c(will)j(b)q(e)f(remo)o (v)o(ed)f(so)g(it)g(no)o(w)g(reads)189 472 y(A)h(datat)o(yp)q(e)f(ma)o (y)g(sp)q(ecify)i(o)o(v)o(erlapping)g(en)o(tries.)22 b(The)16 b(use)g(of)g(suc)o(h)g(a)f(datat)o(yp)q(e)h(in)g(a)g(receiv)o (e)189 529 y(op)q(eration)e(is)h(erroneous.)k(\(This)14 b(is)h(erroneous)f(ev)o(en)g(if)h(the)f(actual)g(message)g(receiv)o(ed) h(is)g(short)189 585 y(enough)g(not)g(to)g(write)g(an)o(y)g(en)o(try)g (more)f(than)h(once.\))189 698 y Fm(Discussion)o(:)189 764 y Fl(The)f(in)o(ten)o(t)h(is)f(to)g(remo)o(v)o(e)f(duplicate)h (text)h(whic)o(h)f(seems)h(to)f(ha)o(v)o(e)g(b)q(een)h(left.)k(No)14 b(c)o(hange)h(in)f(meaning)189 814 y(is)f(in)o(tended.)143 950 y Fi(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(3)e(reads)189 1006 y(MPI)p 281 1006 14 2 v 16 w(P)o(ac)o(k)p 393 1006 V 16 w(size\(coun)o(t,)g(MPI)p 724 1006 V 16 w(CHAR,)h(&k2\);)189 1063 y(but)f(should)h(read)189 1119 y(MPI)p 281 1119 V 16 w(P)o(ac)o(k)p 393 1119 V 16 w(size\(coun)o(t,)f(MPI)p 724 1119 V 16 w(CHAR,)h(comm,)e(&k2\);)189 1289 y Fm(Discussion)o(:)189 1355 y Fl(This)f(is)h(a)g(minor)e(\014x)i(to)f(an)h(example)f(for)g(a)h (missing)e(argumen)o(t.)143 1490 y Fi(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(10)e(reads)189 1547 y(MPI)p 281 1547 V 16 w(P)o(ac)o(k\(c)o(hr,)f(coun)o(t,)g(MPI)p 726 1547 V 17 w(CHAR,)h(&lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 1603 y(but)f(should)h(read)189 1660 y(MPI)p 281 1660 V 16 w(P)o(ac)o(k\(c)o(hr,)e(coun)o(t,)g(MPI)p 726 1660 V 17 w(CHAR,)h(lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 1829 y Fm(Discussion)o(:)189 1895 y Fl(This)e(is)h(a)f(minor)f(\014x)i (to)g(an)f(example)g(for)g(an)g(argumen)o(t)g(that)h(is)g(declared)g (to)g(b)q(e)g(a)g(p)q(oin)o(ter)g(and)f(then)189 1945 y(passed)h(with)e(&.)143 2081 y Fi(\017)23 b Fq(P)o(age)14 b(109,)g(lines)j(26/27)d(and)h(page)g(110,)f(lines)j(28/29)d(reads)189 2137 y(The)h Fk(j)p Fq(th)g(blo)q(c)o(k)g(of)g(data)f(sen)o(t)h(from)f (eac)o(h)h(pro)q(cess)g(is)h(receiv)o(ed)g(b)o(y)f(ev)o(ery)f(pro)q (cess)i(and)f(placed)189 2194 y(in)h(the)f Fk(j)p Fq(th)g(blo)q(c)o(k)h (of)f(the)g(bu\013er)g Fj(recvbuf)p Fq(.)189 2250 y(but)g(should)h (read)189 2307 y(The)g(blo)q(c)o(k)i(of)e(data)g(sen)o(t)g(from)g(the)g Fk(j)p Fq(th)h(pro)q(cess)g(is)g(receiv)o(ed)g(b)o(y)g(ev)o(ery)f(pro)q (cess)h(and)g(placed)189 2363 y(in)f(the)f Fk(j)p Fq(th)g(blo)q(c)o(k)h (of)f(the)g(bu\013er)g Fj(recvbuf)p Fq(.)189 2533 y Fm(Discussion)o(:) 189 2598 y Fl(This)e(is)g(related)h(to)f(a)f(subtle)i(p)q(oin)o(t)f(in) g(usage.)18 b(The)13 b(sendbuf)h(is)f(a)g(single)g(item)f(whereas)i (the)g(recvbuf)g(is)189 2648 y(an)f(\\arra)o(y")g(of)h(items.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fg(3.10.)34 b(MINOR)16 b(CORRECTIONS)1109 b Fq(17)143 45 y Fi(\017)23 b Fq(P)o(age)14 b(117,)g(lines)j(22/23)d (reads)189 102 y(MPI)h(pro)o(vides)g(sev)o(en)h(suc)o(h)f(prede\014ned) i(datat)o(yp)q(es.)189 158 y(but)e(should)h(read)189 214 y(MPI)f(pro)o(vides)g(nine)i(suc)o(h)e(prede\014ned)i(datat)o(yp)q (es.)189 442 y Fm(Discussion)o(:)189 509 y Fl(This)c(is)h(an)g(ob)o (vious)f(mistak)o(e.)k(There)e(are)f(nine)g(listed)g(on)f(line)h (28-40.)143 647 y Fi(\017)23 b Fq(P)o(age)14 b(122,)g(lines)j(35-36)d (read)204 704 y Fj(MPI)p 289 704 14 2 v 16 w(OP)p 367 704 V 16 w(FREE\()i(op\))230 893 y Fl(IN)156 b Fj(op)541 b Fl(op)q(eration)14 b(\(handle\))189 1035 y Fq(but)h(should)h(read)204 1091 y Fj(MPI)p 289 1091 V 16 w(OP)p 367 1091 V 16 w(FREE\()g(op\))230 1280 y Fl(INOUT)63 b Fj(op)541 b Fl(op)q(eration)14 b(\(handle\))189 1463 y Fm(Discussion)o(:)189 1530 y Fl(This)j(app)q(ears)g(to)g(ha)o(v) o(e)g(b)q(een)h(a)f(mistak)o(e.)26 b(This)17 b(function)g(frees)h(the)g (function)e(p)q(oin)o(ter)i(and)f(sets)h(its)189 1580 y(v)n(alue)10 b(to)h(MPI)p 424 1580 13 2 v 15 w(OP)p 499 1580 V 15 w(NULL.)g(Th)o(us,)g(it)g(is)g(b)q(oth)g(input)g(and)f (output)i(as)f(are)g(the)g(free)h(routines)g(for)e(Groups,)189 1629 y(Comms,)g(etc.)143 1768 y Fi(\017)23 b Fq(P)o(age)14 b(125,)g(line)j(1)e(reads)189 1824 y(CALL)h(MPI)p 420 1824 14 2 v 16 w(ALLREDUCE\(sum,)f(c,)g(n,)g(MPI)p 1040 1824 V 17 w(REAL,)g(MPI)p 1300 1824 V 17 w(SUM,)g(0,)f(comm,)g(ierr\)) 189 1881 y(but)h(should)h(read)189 1937 y(CALL)g(MPI)p 420 1937 V 16 w(ALLREDUCE\(sum,)f(c,)g(n,)g(MPI)p 1040 1937 V 17 w(REAL,)g(MPI)p 1300 1937 V 17 w(SUM,)g(comm,)f(ierr\))189 2108 y Fm(Discussion)o(:)189 2175 y Fl(The)g(extra)h(argumen)o(t)e(app) q(ears)j(to)e(b)q(e)h(left)f(o)o(v)o(er)g(from)f(the)i(previous)f (example)f(with)h(MPI)p 1670 2175 13 2 v 16 w(REDUCE.)189 2225 y(This)f(is)h(a)g(mistak)o(e.)143 2363 y Fi(\017)23 b Fq(P)o(age)14 b(194,)g(line)j(48)e(reads)189 2420 y Fj(MPI)p 274 2420 14 2 v 15 w(ERRHANDLER)p 582 2420 V 18 w(CREA)l(TE\(FUNCTION,)h(HANDLER,)g(IERROR\))189 2476 y Fq(but)f(should)h(read)189 2533 y Fj(MPI)p 274 2533 V 15 w(ERRHANDLER)p 582 2533 V 18 w(CREA)l(TE\(FUNCTION,)g(ERRHANDLER,) h(IERROR\))189 2704 y Fm(Discussion)o(:)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Fq(18)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fl(This)e(is)h(a)g(small)d(t)o (yp)q(o)j(to)g(matc)o(h)f(all)f(the)j(other)f(names)f(for)h(this)g (argumen)o(t.)143 186 y Fi(\017)23 b Fq(P)o(age)14 b(196,)g(lines)j (1-2)e(reads)204 243 y Fj(MPI)p 289 243 14 2 v 16 w(ERRHANDLER)p 598 243 V 17 w(FREE\()h(errhandler)f(\))230 433 y Fl(IN)156 b Fj(errhandler)397 b Fl(MPI)14 b(error)h(handler)f(\(handle\))189 576 y Fq(but)h(should)h(read)204 633 y Fj(MPI)p 289 633 V 16 w(ERRHANDLER)p 598 633 V 17 w(FREE\()g(errhandler)f(\))230 823 y Fl(INOUT)63 b Fj(errhandler)397 b Fl(MPI)14 b(error)h(handler)f (\(handle\))189 1007 y Fm(Discussion)o(:)189 1075 y Fl(This)c(app)q (ears)h(to)f(ha)o(v)o(e)h(b)q(een)g(a)f(mistak)o(e.)16 b(This)10 b(function)g(frees)i(the)f(function)f(p)q(oin)o(ter)g(and)h (sets)g(its)g(v)n(alue)189 1125 y(to)h(MPI)p 321 1125 13 2 v 15 w(ERRHANDLER)p 636 1125 V 14 w(NULL.)g(Th)o(us,)h(it)f(is)g (b)q(oth)h(input)f(and)g(output)h(as)f(are)h(the)g(free)g(routines)g (for)189 1175 y(Groups,)g(Comms,)e(etc.)143 1316 y Fi(\017)23 b Fq(P)o(age)14 b(203,)g(line)j(1)e(reads)189 1372 y Fj(MPI)p 274 1372 14 2 v 15 w(PCONTROL\(level\))189 1429 y Fq(but)g(should)h(read)189 1485 y Fj(MPI)p 274 1485 V 15 w(PCONTROL\(LEVEL\))189 1657 y Fm(Discussion)o(:)189 1726 y Fl(This)d(is)h(a)g(minor)e(t)o(yp)q(o.)143 1867 y Fi(\017)23 b Fq(P)o(age)14 b(210,)g(line)j(44)e(reads)189 1923 y Fd(MPI)p 266 1923 13 2 v 14 w(PENDING)189 1980 y Fq(but)g(should)h(read)189 2036 y Fd(MPI)p 266 2036 V 14 w(ERR)p 359 2036 V 14 w(PENDING)189 2209 y Fm(Discussion)o(:)189 2277 y Fl(This)d(is)h(a)g(t)o(yp)q(o.)k(See)c(page)g(197,)f(line)g(20)h (for)f(the)i(original)d(de\014nition.)143 2418 y Fi(\017)23 b Fq(P)o(age)14 b(211,)g(line)j(44)e(reads)189 2475 y Fd(MPI)p 266 2475 V 14 w(DOUBLE)p 445 2475 V 14 w(COMPLEX)189 2531 y Fq(but)g(should)h(b)q(e)g(mo)o(v)o(ed)f(to)f(P)o(age)h(212,)f (line)j(22)d(since)j(it)e(is)h(an)f(optional)h(F)l(ortran)e(datat)o(yp) q(e.)189 2647 y Fm(Discussion)o(:)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fg(3.10.)34 b(MINOR)16 b(CORRECTIONS)1109 b Fq(19)189 45 y Fl(On)20 b(page)h(19,)f(lines)h(2-3,)f(it)g(clearly)h (states)g(this)f(is)h(an)f(optional)f(F)m(ortran)h(datat)o(yp)q(e.)37 b(It)21 b(is)f(listed)189 95 y(incorrectly)14 b(in)g(the)g(app)q (endix.)1875 150 y Fi(>)i Fh(\(Oct\))143 236 y Fi(\017)23 b Fq(P)o(age)14 b(212,)g(add)h(new)h(lines)h(of)d(text)h(at)g(line)i (22)d(and)i(line)g(25)f(to)g(read:)189 292 y(etc.)189 349 y(Th)o(us,)f(the)i(text)e(will)j(no)o(w)e(read:)189 518 y Fk(/*)23 b(optional)g(datatypes)g(\(Fortran\))f(*/)189 574 y(MPI_INTEGER1)189 631 y(MPI_INTEGER2)189 687 y(MPI_INTEGER4)189 744 y(MPI_REAL2)189 800 y(MPI_REAL4)189 857 y(MPI_REAL8)189 913 y(etc.)189 1026 y(/*)h(optional)g(datatypes)g(\(C\))g(*/)189 1082 y(MPI_LONG_LONG_INT)189 1139 y(etc.)189 1292 y Fm(Discussion)o(:) 189 1360 y Fl(On)15 b(page)g(19,)f(line)g(6,)h(concludes)h(the)f(list)f (of)h(optional)e(datat)o(yp)q(es)j(with)e(etc.)22 b(It)15 b(w)o(as)g(not)g(in)o(tended)g(to)189 1410 y(b)q(e)f(a)g(complete)g (list)g(of)f(all)g(p)q(ossible)i(ones.)20 b(Some)13 b(ha)o(v)o(e)h(in)o (terpreted)i(the)e(app)q(endix)h(to)f(b)q(e)h(a)f(complete)189 1460 y(list)f(since)h(only)f(the)h(ones)h(explictly)e(on)g(page)h(19)f (are)h(listed.)k(Adding)13 b(etc.,)h(to)g(the)g(F)m(ortran)f(and)h(C)f (list)189 1510 y(in)g(the)i(app)q(endix)e(will)g(mak)o(e)f(it)i(clear)g (these)h(are)g(only)e(some)g(of)g(the)i(p)q(ossible)f(optional)e(datat) o(yp)q(es.)1875 1565 y Fi(?)k Fh(\(Oct\))143 1651 y Fi(\017)23 b Fq(P)o(age)14 b(213,)g(line)j(28.)i(The)d(follo)o(wing)g(text)e (should)i(b)q(e)g(added:)189 1707 y(/*)9 b(Prede\014ned)j(functions)f (in)g(C)e(and)i(F)l(ortran)e(*/)g(MPI)p 1137 1707 14 2 v 17 w(NULL)p 1278 1707 V 17 w(COPY)p 1428 1707 V 17 w(FN)g(MPI)p 1607 1707 V 17 w(NULL)p 1748 1707 V 17 w(DELETE)p 1954 1707 V 16 w(FN)189 1764 y(MPI)p 281 1764 V 16 w(DUP)p 397 1764 V 16 w(FN)189 1880 y Fm(Discussion)o(:)189 1948 y Fl(These)15 b(are)f(missing)e(and)i(de\014ned)h(on)e(pages)i(169,)d (line)i(17)f(and)h(p.)k(170,)13 b(line)g(4.)143 2089 y Fi(\017)23 b Fq(P)o(age)14 b(220,)g(lines)j(19-20)d(reads)189 2146 y Fj(int)i(double)g(MPI)p 479 2146 V 16 w(Wtime\(void\))189 2202 y(int)g(double)g(MPI)p 479 2202 V 16 w(Wtick\(void\))189 2258 y Fq(but)f(should)h(read)189 2315 y Fj(double)g(MPI)p 413 2315 V 16 w(Wtime\(void\))189 2371 y(double)g(MPI)p 413 2371 V 16 w(Wtick\(void\))189 2544 y Fm(Discussion)o(:)189 2612 y Fl(This)d(w)o(as)h(a)g(mistak)o(e)e(caused)j(b)o(y)f(the)g (macros.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Fq(20)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)143 45 y Fi(\017)23 b Fq(P)o(age)14 b(222,)g(line)j(34)e(reads)189 102 y Fj(INTEGER)f (REQUEST,)g(COUNT,)g(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)g(T)l(A)o(G,)f (COMM,)g(REQUEST,)i(IERROR)189 158 y Fq(but)g(should)h(read)189 214 y Fj(INTEGER)f(COUNT,)h(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)f(T)l(A)o(G,) g(COMM,)g(REQUEST,)h(IERROR)189 387 y Fm(Discussion)o(:)189 455 y Fl(An)e(extra)g(argumen)o(t)f(got)g(added.)19 b(It)14 b(needs)h(to)f(b)q(e)g(remo)o(v)o(ed.)143 596 y Fi(\017)23 b Fq(P)o(age)14 b(222,)g(line)j(38)e(reads)189 653 y Fj(INTEGER)f(REQUEST,)g(COUNT,)g(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)g(T)l(A) o(G,)f(COMM,)g(REQUEST,)i(IERROR)189 709 y Fq(but)g(should)h(read)189 766 y Fj(INTEGER)f(COUNT,)h(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)f(T)l(A)o(G,) g(COMM,)g(REQUEST,)h(IERROR)189 938 y Fm(Discussion)o(:)189 1006 y Fl(An)e(extra)g(argumen)o(t)f(got)g(added.)19 b(It)14 b(needs)h(to)f(b)q(e)g(remo)o(v)o(ed.)143 1147 y Fi(\017)23 b Fq(P)o(age)14 b(227,)g(lines)j(19-20)d(reads)189 1204 y Fj(MPI)p 274 1204 14 2 v 15 w(INTERCOMM)p 563 1204 V 17 w(MERGE\(INTERCOMM,)i(HIGH,)f(INTRA)o(COMM,)g(IERROR\))189 1260 y(INTEGER)g(INTERCOMM,)h(INTRA)o(COMM,)f(IERROR)189 1317 y Fq(but)g(should)h(read)189 1373 y Fj(MPI)p 274 1373 V 15 w(INTERCOMM)p 563 1373 V 17 w(MERGE\(INTERCOMM,)g(HIGH,)f (NEWINTRA)o(COMM,)g(IERROR\))189 1430 y(INTEGER)g(INTERCOMM,)h (NEWINTRA)o(COMM,)f(IERROR)189 1602 y Fm(Discussion)o(:)189 1670 y Fl(This)g(mak)o(es)e(it)i(matc)o(h)f(the)i(de\014nition)e(in)h (the)g(text)h(on)f(P)o(age)g(159,)f(lines)h(34-36.)20 b(It)15 b(only)g(c)o(hanges)g(the)189 1720 y(name)d(of)i(the)g(argumen) o(t.)143 1861 y Fi(\017)23 b Fq(P)o(age)14 b(228,)g(line)j(46)e(reads) 189 1918 y Fj(MPI)p 274 1918 V 15 w(ERRHANDLER)p 582 1918 V 18 w(CREA)l(TE\(FUNCTION,)h(HANDLER,)g(IERROR\))189 1974 y Fq(but)f(should)h(read)189 2031 y Fj(MPI)p 274 2031 V 15 w(ERRHANDLER)p 582 2031 V 18 w(CREA)l(TE\(FUNCTION,)g (ERRHANDLER,)h(IERROR\))189 2203 y Fm(Discussion)o(:)189 2271 y Fl(This)c(c)o(hange)i(mak)o(es)d(the)j(app)q(endix)f(consistan)o (t)g(with)g(c)o(hange)g(on)g(p.)k(194,)12 b(line)i(48.)143 2412 y Fi(\017)23 b Fq(P)o(age)14 b(229,)g(line)j(33)e(reads)189 2469 y Fj(MPI)p 274 2469 V 15 w(PCONTROL\(level\))189 2525 y Fq(but)g(should)h(read)189 2582 y Fj(MPI)p 274 2582 V 15 w(PCONTROL\(LEVEL\))1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fg(3.10.)34 b(MINOR)16 b(CORRECTIONS)1109 b Fq(21)189 45 y Fm(Discussion)o(:)189 114 y Fl(This)13 b(is)h(to)g(mak)o(e)e(the)j(language)e(binding)g(the)h(same)f(as)h(the) h(c)o(hange)f(to)g(P)o(age)f(203,)g(line)g(1.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Fri Sep 20 07:38:53 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA27358; Fri, 20 Sep 1996 07:38:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA28140 for mpi-core-out; Thu, 19 Sep 1996 23:26:13 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA28135; Thu, 19 Sep 1996 23:26:02 -0500 Message-Id: <199609200426.XAA28135@antares.mcs.anl.gov> To: mpi-comm@antares.mcs.anl.gov, mpi-core@antares.mcs.anl.gov Subject: Misc-1.2 Date: Thu, 19 Sep 1996 23:26:00 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk %!PS-Adobe-2.0 %%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%CreationDate: Thu Sep 19 16:46:18 1996 %%Pages: 23 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1996.09.19:1646 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@landscape{/isls true N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{ pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get} B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D }B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail} B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{ 3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{ 3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet TeXDict begin 40258431 52099146 1000 300 300 (/tmp_mnt/Net/antireo/antireo6/lusk/mpi2/report2/temp.dvi) @start /Fa 1 106 df<030007800780030000000000000000007F807F80038003800380 038003800380038003800380038003800380FFFCFFFC0E187D9714>105 D E /Fb 12 122 df<387CFEFEFE7C3807077C860F>46 D<00E00001E0000FE000FFE000 F3E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000 03E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000FFFF80 FFFF80111D7C9C1A>49 D<01FC0007FF000E0F801E0FC03F07E03F07E03F07E03F07E01E 0FC0000FC0000F80001F0001FC0001FC00000F800007C00003E00003F00003F83803F87C 03F8FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003FC00151D7E9C1A>51 D69 D<07FC001FFF003F0F803F07 C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803 E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 D<01FE0007FF800F83C01E01E03E00F0 7C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E0018 0F807007FFE000FF8015147F9318>101 D108 DI112 D<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F80 0F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>116 D120 DI E /Fc 4 103 df<70F8FCFC74040404080810102040060E7C840D>59 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E10F01C20F01C20703C20705C40308C400F078014147E9318>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207E9F14>I< 00007C0000CE00019E00039E00030C000700000700000700000700000E00000E00000E00 00FFF0000E00000E00001C00001C00001C00001C00001C00003800003800003800003800 00380000700000700000700000700000700000E00000E00000E00000E00000C00001C000 318000798000F300006200003C000017297E9F16>102 D E /Fd 33 122 df<00E001C0038007000E000E001C001C003800380038007000700070007000E0 00E000E000E000E000E000E000E000E000E000E000E00070007000700070003800380038 001C001C000E000E000700038001C000E00B2A7E9E10>40 DI44 D<001C0000003E0000003E0000 002E0000006700000067000000E7800000C7800000C3800001C3C0000183C0000181C000 0381E0000381E0000700F0000700F0000600F0000E0078000FFFF8000FFFF8001C003C00 1C003C0018003C0038001E0038001E0070001F0070000F0070000F00E0000780191D7F9C 1C>65 DI<003FC000FFF003C0F007 80300F00001E00003C00003C0000780000780000780000F00000F00000F00000F00000F0 0000F00000F00000F00000F000007800007800007800003C00003C00001E00000F000807 801803C07800FFF0003F80151F7D9D1B>III<003F8001FFF003C0F807 80380F00181E00003C00003C0000780000780000780000F00000F00000F00000F00000F0 0000F00000F007F8F007F8F000387800387800387800383C00383C00381E00380F003807 803803C0F801FFF0003F80151F7D9D1C>71 D73 D76 DII<003F000001FFE00003FFF00007C0F8000F807C001E 001E003E001F003C000F00780007807800078078000780F00003C0F00003C0F00003C0F0 0003C0F00003C0F00003C0F00003C0F00003C0F80007C078000780780007807C000F803C 000F003E001F001F003E000F807C0007C0F80003FFF00001FFE000003F00001A1F7E9D1F >II<003F000001FFE00003FFF000 07C0F8000F807C001F003E003E001F003C000F00780007807800078078000780F00003C0 F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C078000780 78000780780E07803C0F0F003E079F001E03DE000F83FC0007C1F80003FFF00001FFF800 003F780000003C0000003E0000001F0000000F801A237E9D1F>II<03F8000FFE001C0F00380700700300600000E00000E000 00E00000E00000F000007800007F00003FE0001FFC0007FE0001FF00001F800007800003 C00003C00001C00001C00001C00001C0C00180E00380F007007C0E001FFC0007F000121F 7E9D17>III<78000E007C001E003C003C001E0038000F0070000F00F0000781E0 0003C1C00001C3C00001E7800000F70000007E0000003E0000003C0000003C0000007E00 000077000000E7800001E3800003C1C0000381E0000700F0000F00F8000E0078001C003C 003C003E0078001F0070000F00F0000F80191D7F9C1C>88 DI<0FC03FF07FF87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC 7FDC3F1C0E127E9114>97 DI<000E000E000E000E000E000E000E000E000E000E000E0F8E1FEE3FFE7C3E70 0E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E0F1D7E9C15>100 D<07C01FE03FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C3FFC1FF807 E00E127E9112>I108 D110 D112 D<1C001C001C001C001C001C00FFE0FFE01C001C001C001C001C001C00 1C001C001C001C001C001C001C201FF00FF007C00C187F970F>116 DI<7003807807003C0E001C1C000E1C0007380003F00001E00001C00001 E00003F0000738000E18000E1C001C0E00380700700380F003C01212809113>120 DI E /Fe 24 119 df<00003F03E00000C386700001878CF00003879CF00003031860000700380000 070038000007003800000E003800000E007000000E007000000E00700000FFFFFF80001C 007000001C00E000001C00E000001C00E000001C00E000003800E000003801C000003801 C000003801C000003801C000007001C00000700380000070038000007003800000700380 00006003800000E007000000E007000000E007000000E007000000C006000001C00E0000 01C00E000031860C0000798F180000F31E100000620C6000003C07C000002429829F1C> 11 D<70F8F8F0E005057B840E>46 D<00000200000006000000060000000E0000001E00 00001E0000003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F00 00020F0000040F00000C0F0000080F0000100F0000100F0000200F80003FFF8000400780 00C007800080078001000780010007800200078002000780060007801E000F80FF807FF8 1D207E9F22>65 D<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004 003C0004003C00040078080800780800007808000078180000F0300000FFF00000F03000 00F0300001E0200001E0200001E0200001E0001003C0002003C0002003C0004003C00040 078000800780018007800100078007000F001F00FFFFFE001F1F7D9E1F>69 D<01FFFF00001E03C0001E00E0001E0070001E0078003C0078003C0078003C0078003C00 78007800F0007800F0007801E0007801C000F0070000F01E0000FFF00000F0380001E01C 0001E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003C01E0007803C0007803C 0807803C0807803C100F801C10FFF00C20000007C01D207D9E21>82 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207B9F15>I< 007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F00000 F00000F00000F0000070010070020030040018380007C00011147C9315>I<0000780003 F80000700000700000700000700000E00000E00000E00000E00001C00001C000F1C00389 C00707800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E 20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C01C207010E011C01 3C013802780C7BF07C00F000F000F000F0007000700170023804183807C010147C9315> I<00007800019C00033C00033C000718000700000700000E00000E00000E00000E00000E 0001FFE0001C00001C00001C00001C000038000038000038000038000038000070000070 0000700000700000700000700000E00000E00000E00000E00000C00001C00001C0000180 003180007B0000F300006600003C00001629829F0E>I<003C6000E27001C1E00380E007 00E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C03803C07003C07001C 0F001C17000C2E0003CE00000E00000E00001C00001C00301C00783800F0700060E0003F 8000141D7E9315>I<01E0000FE00001C00001C00001C00001C000038000038000038000 038000070000070000071E000763000E81800F01C00E01C00E01C01C03801C03801C0380 1C0380380700380700380700380E10700E20700C20701C20700C40E00CC060070014207D 9F17>I<00C001E001E001C000000000000000000000000000000E003300230043804300 470087000E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E> I<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C> 108 D<1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E0 1C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070 8038038071003803806100380380E10038038062007007006600300300380021147C9325 >I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C 01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C00 01C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0 F00780F00700700F00700E0030180018700007C00013147C9317>I<01C1E00262180474 1C04781C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380 780380700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00 001C0000FFC000171D809317>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030 000011147C9313>114 D<00FC030206010C030C070C060C000F800FF007F803FC003E00 0E700EF00CF00CE008401020601F8010147D9313>I<018001C003800380038003800700 0700FFF007000E000E000E000E001C001C001C001C003800380038003820704070407080 708031001E000C1C7C9B0F>I<0E00C03300E02301C04381C04301C04701C08703800E03 800E03800E03801C07001C07001C07001C07101C0E20180E20180E201C1E200C264007C3 8014147C9318>I<0E03803307802307C04383C04301C04700C08700800E00800E00800E 00801C01001C01001C01001C02001C02001C04001C04001C08000E300003C00012147C93 15>I E /Ff 50 123 df<000078007C7800FC7801FC7803C00003800007800007800007 8000078000078000078000078000078000FFFC78FFFC78FFFC7807807807807807807807 807807807807807807807807807807807807807807807807807807807807807807807807 8078078078078078078078152480A31A>12 D45 DI<00C001C007C0FFC0FFC0FBC003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C0 03C0FFFFFFFFFFFF10227CA118>49 D<03F0000FFC001FFE003C1F003007807007C06003 C0E003E0C001E04001E04001E00001E00001E00001E00003C00003C0000780000780000F 00001E00003C0000780000F00001E00001C0000380000700000E00001C00003800007000 00FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC001FFF003E0F003807807003C020 03C02003C00003C00003C00003C0000780000780000F00001E0003FC0003F80003FE0000 0F000007800003C00003C00001E00001E00001E00001E00001E08001E0C003C0E003C070 07803C0F801FFF000FFC0003F00013237EA118>I<001F00001F00002F00002F00006F00 00EF0000CF0001CF0001CF00038F00038F00078F00070F000F0F000E0F001E0F003C0F00 3C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF8000F00000F00000F00000F00000F00 000F00000F00000F00000F0015217FA018>I55 D<01F00007FC000FFE001E0F003C07803C0780 7803C07803C07803C07803C07803C03803803C07801E0F000F1E0007FC0003F8000FFE00 1E0F003C07807803C07803C0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C0 3C07803E0F801FFF0007FC0001F00013237EA118>I<01F00007FC000FFE001E0F003C07 003807807803807803C0F003C0F001C0F001E0F001E0F001E0F001E0F001E0F001E07003 E07803E07803E03C07E03E1FE01FFDE00FF1E003E3C00003C00003C00003800007800007 00000F00101E00183C003FF8001FF00007C00013237EA118>I<001F0000001F0000003F 8000003F8000003B8000007BC0000073C0000071C00000F1E00000F1E00000E0E00001E0 F00001E0F00001C0F00003C0780003C078000380780007803C0007803C0007003C000F00 1E000F001E000FFFFE001FFFFF001FFFFF001C000F003C0007803C000780380007807800 03C0780003C0700003C0F00001E0F00001E0E00001E01B237EA220>65 DI<000FF000007FFC0000FFFF0001F01F0003C00700078000000F0000001E0000 003E0000003C0000003C000000780000007800000078000000F0000000F0000000F00000 00F0000000F0000000F0000000F0000000F0000000F00000007800000078000000780000 003C0000003C0000003E0000001E0000000F0000000780008003C0038001F00F8000FFFF 00007FFC00000FF00019257DA31F>IIII<000FF000003FFE0000 FFFF8001F80F8003E00380078000000F0000001E0000001E0000003C0000003C00000078 0000007800000078000000F0000000F0000000F0000000F0000000F0000000F0000000F0 00FFC0F000FFC0F000FFC0780003C0780003C0780003C03C0003C03C0003C01E0003C01E 0003C00F0003C0078003C003E003C001F807C000FFFFC0003FFF00000FF8001A257DA321 >I73 D76 DII<001FC000007FF00001FFFC0003F07E 0007C01F000F800F801F0007C01E0003C03C0001E03C0001E0780000F0780000F0780000 F070000070F0000078F0000078F0000078F0000078F0000078F0000078F0000078F00000 78F0000078780000F0780000F0780000F07C0001F03C0001E03E0003E01E0003C01F0007 C00F800F8007C01F0003F07E0001FFFC00007FF000001FC0001D257DA324>II< FFFC00FFFF80FFFFC0F003E0F000F0F00078F00038F0003CF0003CF0003CF0003CF0003C F00038F00078F000F0F003E0FFFFC0FFFF80FFFE00F01E00F00F00F00700F00780F00380 F003C0F001E0F001E0F000F0F000F0F00078F00038F0003CF0001EF0001EF0000F18237B A21F>82 D<00FE0003FFC007FFE00F81E01E00603C00003C000078000078000078000078 00007800007C00003C00003F00001FC0000FFC0007FF0001FF80003FC00007E00001F000 00F00000F8000078000078000078000078000078000078C000F0E000F0F801E07E07C03F FF800FFF0001FC0015257EA31B>III90 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF00FF0 0FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C0180 780000780000700000F00000F00000F00000F00000F00000F00000780000780000780000 3C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC01FFFC03F0F C03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C07803 C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC001FFE003E 0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F0000070000078 00007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01FF03C003 8007800780078007800780078007800780FFF8FFF8FFF807800780078007800780078007 80078007800780078007800780078007800780078007800780102380A20F>I<01F07807 FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F000FFE001F FC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8F00078F0 0078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>I 105 D107 DIII<01FC0007FF000FFF801F07C03C01E07800F078 00F0700070F00078F00078F00078F00078F00078F000787800F07800F07C01F03E03E01F 07C00FFF8007FF0001FC0015167F9518>II114 D<07F01FFC3FFE3C0E7806780078007C003F003FF01FF80FFC01FE001F000F000F000FC0 0FF81EFFFE3FFC0FF010167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F00 0F000F000F000F000F000F000F000F000F000F000F000F000F000F080F1C07FC07F803E0 0E1C7F9B12>III<7801F07C01E03E03C01E07C00F 0780078F0007DE0003FC0001FC0000F80000700000F80001FC0003DC00039E00078F000F 07801E07801E03C03C01E07800F0F800F81516809516>120 DI<7FFFC07FFFC07FFFC0000F80000F00 001E00003E00007C0000780000F00001F00003E00003C0000780000F80001F00001E0000 3C00007C0000FFFFC0FFFFC0FFFFC012167F9515>I E /Fg 32 90 df45 D<3078F8787005057C840D>I<007E0001C3000301800701 C00E00C00E00E01C00E01C00E03C01E03801E07801E07801E07801E07801E07801E0F003 C0F003C0F003C0F003C0F003C0F00380F00780E00780E00700E00700E00E00600E00701C 003038003870000FC000131F7C9D17>48 D<000C001C00FC0F3800380038003800380038 00700070007000700070007000E000E000E000E000E000E001C001C001C001C001C001C0 038003C0FFFE0F1E7C9D17>I<003F8000C1E00100F00200780400780400780F007C0F80 7C0F807C0F00780600780000F80000F00001E00001C0000380000700000E00001C000038 0000600000C0000180000300200600200800401000403FFFC07FFF80FFFF80161E7E9D17 >I<007F000183C00201E00400F00700F00F00F00F01F00F01F00001E00001E00003C000 0380000700000E0000F800000E000007000007800007C00003C00007C03007C07807C0F8 07C0F807C0F00780800F00400E00201C0018780007E000141F7D9D17>I<01803001FFE0 03FFC003FF0003FC00020000020000020000040000040000040000047C00058700060380 0C01800801C00001C00001E00001E00001E00001E07003C0F803C0F003C0E00380800780 400700400E00201C0018700007C000141F7D9D17>53 D<2000003FFFE07FFFC07FFF8040 0100C0020080020080040000080000100000200000400000400000800001800003000003 00000700000600000E00000E00001E00001C00001C00003C00003C00003C000078000078 0000780000300000131F799D17>55 D<003F0000C1C00100600200600400300C00300C00 300C00300C00600E00600F80C00FC18007F60003FC0001FC0001FF00063F800C0F801007 C03003C06001C06000C0C000C0C000C0C000C0C00080C0010060030030040018180007E0 00141F7D9D17>I<007E0001C3000301800601C00E01C01C00C03C00E03C00E03C01E078 01E07801E07801E07801E07803E07803E03803C03807C01C0BC00C13C003E38000078000 0780000700000E00600E00F01C00F01800E0300080600041C0003F0000131F7C9D17>I< 0000100000001800000038000000380000007800000078000000FC000001BC0000013C00 00033C0000023C0000063C0000043E0000081E0000081E0000101E0000101E0000201E00 00200F0000400F0000400F0000FFFF0000800F0001000F80010007800200078002000780 04000780040007800C0007C03E0007C0FF807FFC1E207E9F22>65 D<0001F808000E061800380138007000F801E0007803C0007007800030078000300F0000 301F0000301E0000303E0000203C0000007C0000007C0000007C0000007C000000F80000 00F8000000F8000000F8000000F80000007800004078000080780000803C0000803C0001 001C0002000E00020006000C000300100001C0E000003F00001D217B9F21>67 D<07FFFF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E007800 1E0078001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E01E000 3E01E0003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C003 C003C0078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038 003C001800780018007800080078000800780008007800080078080800F0100000F01000 00F0100000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001E00008 01E0001003C0001003C0001003C0002003C0002003C0006003C000C0078001C0078007C0 FFFFFF801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C0018007800180078000800 78000800780008007800080078000800F0100000F0100000F0100000F0300000F0700000 FFF00001E0600001E0200001E0200001E0200001E0200001E0000003C0000003C0000003 C0000003C0000003C0000003C000000780000007C00000FFFE00001D1F7E9E1E>I<0001 FC04000F030C003C009C0070007C00E0003C01C0003803800018078000180F0000181F00 00181E0000183E0000103C0000007C0000007C0000007C0000007C000000F8000000F800 0000F8007FFCF80003E0780001E0780001E0780003C0780003C03C0003C03C0003C01C00 03C00E0007C007000B800380118001E06080003F80001E217B9F24>I<07FFC7FFC0007C 00F800003C007800003C007800007800F000007800F000007800F000007800F000007800 F000007800F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000F001E0 0001E003C00001E003C00001E003C00001E003C00001E003C00001E003C00003C0078000 03C007800003C007800003C007800003C007800003C007800007800F000007C00F8000FF F8FFF800221F7E9E22>I<07FFE0007C00003C00003C0000780000780000780000780000 780000780000F00000F00000F00000F00000F00000F00001E00001E00001E00001E00001 E00001E00003C00003C00003C00003C00003C00003C00007800007C000FFFC00131F7F9E 10>I<07FFE07FE0007C001F00003C000C00003C00180000780010000078004000007800 800000780100000078020000007804000000F008000000F010000000F060000000F0F000 0000F1F0000000F278000001E478000001E878000001F03C000001E03C000001E01E0000 01E01E000003C00F000003C00F000003C00F000003C007800003C007800003C003C00007 8003C00007C007E000FFFC3FFC00231F7E9E23>75 D<07FFF000007E0000003C0000003C 000000780000007800000078000000780000007800000078000000F0000000F0000000F0 000000F0000000F0000000F0000001E0000001E0000001E0000001E0000001E0008001E0 010003C0010003C0010003C0030003C0020003C0060003C0060007801E0007807C00FFFF FC00191F7E9E1C>I<07FC0000FFC0007C0000F800003C00017800003C00017800004E00 02F000004E0002F000004E0004F000004E0004F000004E0008F000004E0008F000008700 11E00000870011E00000870021E00000870021E00000870041E00000838041E000010380 83C00001038083C00001038103C00001038203C0000101C203C0000101C403C0000201C4 0780000201C80780000201C80780000201D00780000200F00780000600E00780000600E0 0F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A>I<07FC01FFC0003E003E00003E0018 00003E001800004F001000004F001000004780100000478010000043C010000043C01000 0083C020000081E020000081E020000080F020000080F020000080782000010078400001 007C400001003C400001003C400001001E400001001E400002000F800002000F80000200 0F800002000780000200078000060003800006000300000F00010000FFE0010000221F7E 9E22>I<0003F800001E0E000038070000E0038001C001C003C001E0078000E00F0000F0 0F0000F01E0000F01E0000F83E0000F83C0000F87C0000F87C0000F87C0000F87C0000F8 F80001F0F80001F0F80001F0F80001F0F80003E0780003E0780003C0780007C07C000780 3C000F003C001E001E001C000E0038000700F00003C3C00000FE00001D217B9F23>I<07 FFFF00007C03C0003C01E0003C00F0007800F0007800F8007800F8007800F8007800F800 7800F000F001F000F001E000F003C000F0078000F00F0000FFF80001E0000001E0000001 E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000003C0000003 C000000780000007C00000FFFC00001D1F7E9E1F>I<07FFFC00007C0700003C03C0003C 01E0007801E0007801F0007801F0007801F0007801F0007801E000F003E000F003C000F0 078000F00F0000F03C0000FFF00001E0300001E0380001E01C0001E01C0001E01C0001E0 1E0003C03E0003C03E0003C03E0003C03E0003C03E0003C03E0207803E0407C01F04FFFC 0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E 00080E00080E00080E00000F00000F80000FE00007FE0003FF8001FFC0007FE00007E000 01E00000E00000F00000F04000E04000E04000E04000E06000C0600180E00380F80300C6 0C0081F80016217D9F19>I<3FFFFFF03C0780F03007803060078030400F0010400F0010 C00F0010800F0010800F0010800F0010001E0000001E0000001E0000001E0000001E0000 001E0000003C0000003C0000003C0000003C0000003C0000003C00000078000000780000 0078000000780000007800000078000000F0000001F800007FFFE0001C1F7A9E21>IIII< 03FFC0FFC0007F007E00003E003800001E003000001E002000000F004000000F00800000 0F81000000078200000007C600000003C400000003E800000001F000000001F000000000 F000000000F800000000F8000000017C000000023C000000043C0000000C1E000000081E 000000101F000000200F000000400F800000C0078000008007C000010003C000070003E0 001F8007E000FFE01FFE00221F7F9E22>II E /Fh 15 117 df<020408103020604040C0C0C0C0C0C0C0C0404060203010 080402071A7F920C>40 D<8040201018080C0404060606060606060604040C0818102040 80071A7E920C>I<1F00318060C04040C060C060C060C060C060C060C060C060404060C0 31801F000B107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C00 0C000C000C00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006 000C00102020207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F000080 00400060C060C060804060801F000B107F8F0F>I<0300030007000F000B001300330023 004300C300FFE003000300030003001FE00B107F8F0F>I<20803F002C00200020002000 2F0030802040006000600060C06080C061801F000B107F8F0F>I<0780184030C060C060 00C000CF00F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC0 8080808001000200040004000C0008000800180018001800180018000B117E900F>I<1F 00318060C060C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F >I<1F00318060C0C040C060C060C06040E021E01E600060004060C0608043003E000B10 7F8F0F>I<03E0000C1800100400200200600300400100C00180C00180C00180C00180C0 01806003006003003006003006000C180003E00011117E9017>79 D<1F8030C06000C000C000C000C000C000604030801F000A0B7F8A0E>99 D<10103030FE3030303030323232321C070F7F8E0C>116 D E /Fi 3 64 df<03C00FF01FF83FFC7FFE7FFEFFFFFFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF8 0FF003C010127D9317>15 D62 D<00040000000C0000000C0000 000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000 000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000 000C0000000C0000000C0000000C0000000C0000FFFFFFE0FFFFFFE01B1C7C9B23>I E /Fj 60 123 df<00F8F001F8F003F8F00780F00700F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F0FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F01420809F 18>13 D<007000E001C00380078007000E001E001E003C003C003C007800780078007800 7000F000F000F000F000F000F000F000F000F000F000F000F00070007800780078007800 3C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DII<03F000 07F8001FFE001E1E003C0F00380700780780780780700380F003C0F003C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0780780780780 7807803807003C0F001E1E001FFE000FFC0003F00012207E9E17>48 D<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F7C9E17>I58 D61 D<001F0000001F0000003F8000003B8000003B8000007BC0000073 C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0F00003C0780003C0 78000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E000F001E00 0F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B207F9F1E> 65 DI<001F C000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C00007800007800007800 00F00000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00 003C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>IIII<001FE000FFF801FFFE03E03E07800E0F 00001E00003E00003C00007C0000780000780000780000F00000F00000F00000F00000F0 0000F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07 801E03E03E01FFFE00FFF8001FC017227DA01E>III75 DI II<003F0000 00FFC00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F0078000780 7800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F80007C07800078078000780780007803C000F003C000F001E001E00 1F003E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II<003F000000FFC00003FF F00007E1F8000F807C001F003E001E001E003C000F003C000F0078000780780007807800 0780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C07800078078000780781E07803C0F0F003C0F8F001E079E001F03FE000F83 FC0007E1F80003FFF00000FFF800003F780000007C0000003E0000001E0000001F000000 0F801A277DA021>II<01FC0007FF800FFFC01F03C03C00C03C00007800007800007800007800007800 007C00003C00003F00001FE0000FFC0007FE0001FF00003F800007C00003C00003E00001 E00001E00001E00001E00001E00001C0C003C0F007C0FC0F807FFF001FFE0003F8001322 7EA019>IIIII<780007807C000F003E001F001E001E000F003C000F807C0007807800 03C0F00003E1F00001F1E00000F3C000007FC000007F8000003F0000001F0000001E0000 003F0000007F8000007FC00000F3C00001F1E00001E0F00003C0F80007C0780007803C00 0F003E001F001E001E000F003C000F807C0007C0780003C0F00003E01B207F9F1E>III<07E03FF87FFC701E401F000F000F00 0F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7FFF3FEF1F8F10147E9316>97 DI<03F00FFC 1FFE3E0E3C0278007800F000F000F000F000F000F000780078003C013E0F1FFF0FFE03F0 10147E9314>I<0007800007800007800007800007800007800007800007800007800007 8000078000078007C7800FF7801FFF803E1F807C0780780780F80780F00780F00780F007 80F00780F00780F00780F00780780780780F803E1F801FFF800FF78007C78011207E9F17 >I<03F0000FFC001FFE003E1F003C0700780700700380FFFF80FFFF80FFFF80F00000F0 0000F000007000007800003C01003E07001FFF0007FE0001F80011147F9314>I<007E01 FE03FE078007000F000F000F000F000F000F000F00FFF0FFF0FFF00F000F000F000F000F 000F000F000F000F000F000F000F000F000F000F000F000F000F20809F0E>I<03E0F00F FFF01FFFF03E3E003C1E00780F00780F00780F00780F00780F003C1E003E3E001FFC003F F80033E0003000003800003FFE003FFF801FFFC03FFFE07803F0F000F0F000F0F000F0F8 01F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE001FFF803F0FC03C03C07801E07801E0F000 F0F000F0F000F0F000F0F000F0F000F07801E07801E03C03C03F0FC01FFF8007FE0001F8 0014147F9317>II<03C7800FF780 1FFF803E1F807C0F80780780780780F00780F00780F00780F00780F00780F00780F80780 7807807C0F803E1F801FFF800FF78007C780000780000780000780000780000780000780 000780000780000780111D7E9317>II<07F01FFC3FFC780C78 00780078007C003FC01FF00FF803F8007C003C003CC03CF07CFFF87FF00FC00E147F9311 >I<1E001E001E001E001E001E00FFF0FFF0FFF01E001E001E001E001E001E001E001E00 1E001E001E001E001E001E201FF00FF007C00C1A7F9910>II< F003C0F003C07803807807807807803C0F003C0F003C0F001E0E001E1E001E1E000F1C00 0F3C000F3C0007380007380007B80003F00003F00001E00012147F9315>II<7801E07C03C03E07801E0F000F0F00079E0003FC0003F80001F8 0000F00001F00001F80003FC00079E000F0F000E0F001E07803C03C07801E0F801F01414 809315>II<7FFF7FFF7FFF003E00 3C007800F800F001E003E007C007800F001F001E003C007C00FFFFFFFFFFFF10147F9314 >I E /Fk 73 126 df<030600078F00078F00078F00078F00078F00078F007FFFC0FFFF E0FFFFE07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFF C01E3C001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>35 D<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3F00FE3 800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E07070E078 7070FC707FFFE03FCFE00F03C0141C7F9B18>38 D<007000F001E003C007800F001E001C 00380038007000700070007000E000E000E000E000E000E000E000E00070007000700070 00380038001C001E000F00078003C001F000F000700C24799F18>40 D<6000F00078003C001E000F000780038001C001C000E000E000E000E000700070007000 70007000700070007000E000E000E000E001C001C0038007800F001E003C007800F00060 000C247C9F18>I<01C00001C00001C00001C000C1C180F1C780F9CF807FFF001FFC0007 F00007F0001FFC007FFF00F9CF80F1C780C1C18001C00001C00001C00001C00011147D97 18>I<1C3E7E7F3F1F070E1E7CF860080C788518>44 D<3078FCFC78300606778518>46 D<000300000780000780000F80000F00001F00001E00001E00003E00003C00007C000078 0000780000F80000F00001F00001E00003E00003C00003C00007C0000780000F80000F00 000F00001F00001E00003E00003C00003C00007C0000780000F80000F00000F000006000 0011247D9F18>I<01800380038007800F803F80FF80FB80438003800380038003800380 038003800380038003800380038003800380038003807FFCFFFE7FFC0F1C7B9B18>49 D<03F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E00000E00001 C00001C00003C0000780000F00001E00003C0000780000F00001E00007C0000F80001E00 E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807807803C078 01C03001C00001C00003C0000380000F0003FF0003FE0003FF000007800003C00001C000 00E00000E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F800131C7E9B 18>I<001F00003F0000770000770000E70001E70001C7000387000787000707000E0700 1E07003C0700380700780700F00700FFFFF8FFFFF8FFFFF8000700000700000700000700 000700000700007FF000FFF8007FF0151C7F9B18>I<03F8000FFE001FFF003E0F803803 807001C07001C07001C07001C03803803C07801FFF0007FC000FFE001F1F003C07807001 C0F001E0E000E0E000E0E000E0E000E07001C07803C03E0F801FFF000FFE0003F800131C 7E9B18>56 D<3078FCFC783000000000000000003078FCFC78300614779318>58 D<183C7E7E3C180000000000000000183C7E7E3E1E0E1C3C78F060071A789318>I<7FFF C0FFFFE0FFFFE0FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C 7E9318>61 D<0FF0003FFC007FFF00700F00F00380F00380600780000F00003E00007C00 01F00001E00003C00003C00003C00003C00003C000038000000000000000000000000000 00000003800007C00007C00007C000038000111C7D9B18>63 D<00700000F80000F80000 D80000D80001DC0001DC0001DC00018C00038E00038E00038E00038E0003060007070007 07000707000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F 07F0151C7F9B18>65 DI<00F8E003FEE007 FFE00F07E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E0 0000E00000E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003 FE0000F800131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E0 1C00E01C00F01C00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E0 1C00E01C01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C0 03FDC00FFFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000 E00000E00000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC0 0FFFC003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01 C01C01C01C01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01 C01C01C01C01C01C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7F FF00FFFF807FFF0001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001 C0007FFF00FFFF807FFF00111C7D9B18>I<7F07F0FF87F87F07F01C03C01C07801C0700 1C0E001C1E001C3C001C38001C70001CF0001DF0001DF0001FB8001FB8001F1C001E1C00 1C0E001C0E001C07001C07001C03801C03801C01C07F03F0FF87F87F03F0151C7F9B18> 75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00700E00700E0070 0E00700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01C E1C01CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C 0DC01C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F00 700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800 111C7D9B18>II<0FF8003FFE007FFF0078 0F00700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E0 0380E00380E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000F FC00001C00001E00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF00 1C0F801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE00 1C0F001C07001C03801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F8 7F00F0161C7F9B18>I<03F3801FFF803FFF807C0F80700780E00380E00380E00380E000 007000007800003F00001FF00007FE0000FF00000F800003C00001C00000E00000E06000 E0E000E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FF FFF8E07038E07038E07038E0703800700000700000700000700000700000700000700000 700000700000700000700000700000700000700000700000700000700000700007FF0007 FF0007FF00151C7F9B18>IIII<7F8FE0 7F9FE07F8FE00E07000F0700070E00078E00039C0003DC0001F80001F80000F00000F000 00700000F00000F80001F80001DC00039E00038E00070F000707000E07800E03801E03C0 7F07F0FF8FF87F07F0151C7F9B18>II<3F FFE07FFFE07FFFE07001C07003C0700780700700000F00001E00001C00003C0000780000 700000F00001E00001C00003C0000780000700000F00001E00E01C00E03C00E07800E070 00E0FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<7FFF00FFFF80FFFF807FFF0011047D7F18>95 D<1FE0003FF8007FFC00781E00300E00 00070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F00F00781F00 3FFFF01FFBF007E1F014147D9318>97 D<7E0000FE00007E00000E00000E00000E00000E 00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00380E00380E 00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C809B18>I< 01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E00000E00000 E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F80003F 80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80380F807007 80700380E00380E00380E00380E00380E00380E00380700780700780380F803C1F801FFF F00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F00380780700380700380E0 01C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF8007FF0001 FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001C0007FFFC0 FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007FFF80FFF F81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC001FF80039E0 003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E000387800 F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00000E00000E 00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C809B 18>I<03800007C00007C00007C0000380000000000000000000000000007FC000FFC000 7FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I<0038007C007C007C003800 000000000000000FFC1FFC0FFC001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C001C001C6038F078FFF07FE03F800E277E9C18 >II<7FE000FFE0007FE00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C7E 9B18>I<7CE0E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C00 1C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C00 7F1F1F00FFBFBF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F00 E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3 FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0E0 00E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE0001 F00013147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E0038 0E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E0000 0E00000E00000E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<01E3 8007FB801FFF803E1F80380F80700780700780E00380E00380E00380E00380E00380E003 80700780700780380F803C1F801FFF800FFB8003E3800003800003800003800003800003 80000380000380003FF8003FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F87803 F03003E00003C00003C00003800003800003800003800003800003800003800003800003 80007FFE00FFFF007FFE0015147F9318>I<07F7003FFF007FFF00780F00E00700E00700 E007007C00007FE0001FFC0003FE00001F00600780E00380E00380F00380F80F00FFFF00 FFFC00E7F00011147D9318>I<0180000380000380000380000380007FFFC0FFFFC0FFFF C00380000380000380000380000380000380000380000380000380000380400380E00380 E00380E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E00E 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E00F 03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E0380 0E0380070700070700070700038E00038E00038E00038E0001DC0001DC0001DC0000F800 00F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F80000F80000 700000F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF87F8FF015147F93 18>I<7F8FF0FF8FF87F8FF00E01C00E03800E0380070380070700070700038700038600 038E0001CE0001CE0000CC0000CC0000DC00007800007800007800007000007000007000 00F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<3FFFF07FFFF07FFF F07001E07003C0700780000F00001E00003C0000F80001F00003C0000780000F00701E00 703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I<0007E0001FE0007FE000780000 E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00001E0007F C000FF8000FF80007FC00001E00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E000007800007FE0001FE00007E013247E9F18>I<7C0000FF0000 FFC00003C00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E000 00E00000F000007FC0003FE0003FE0007FC000F00000E00000E00000E00000E00000E000 00E00000E00000E00000E00000E00000E00003C000FFC000FF00007C000013247E9F18> 125 D E /Fl 70 123 df<007E1F0001C1B1800303E3C00703C3C00E03C1800E01C0000E 01C0000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0000E01C0000E 01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E 01C0000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C180030180 0703C00E03C00E01800E00000E00000E00000E00000E0000FFFFC00E01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C07F87F8151D809C17>I<007FC001C1C00303C00703C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07FCFF8151D809C17>I<60 60F0F0F8F86868080808080808101010102020404080800D0C7F9C15>34 D<00E0000001900000030800000308000007080000070800000708000007080000071000 0007100000072000000740000003C03FE003800F00038006000380040005C0040009C008 0010E0100030E010006070200060702000E0384000E03C4000E01C8000E00F0020E00700 20700780403009C0401830E18007C03E001B1F7E9D20>38 D<004000800100020006000C 000C0018001800300030007000600060006000E000E000E000E000E000E000E000E000E0 00E000E000E000600060006000700030003000180018000C000C00060002000100008000 400A2A7D9E10>40 D<800040002000100018000C000C0006000600030003000380018001 80018001C001C001C001C001C001C001C001C001C001C001C001C0018001800180038003 000300060006000C000C00180010002000400080000A2A7E9E10>I<60F0F07010101010 20204080040C7C830C>44 DI<60F0F06004047C830C>I<03C00C 301818300C300C700E60066006E007E007E007E007E007E007E007E007E007E007E007E0 07E00760066006700E300C300C18180C3007E0101D7E9B15>48 D<030007003F00C70007 000700070007000700070007000700070007000700070007000700070007000700070007 000700070007000F80FFF80D1C7C9B15>I<07C01830201C400C400EF00FF80FF807F807 7007000F000E000E001C001C00380070006000C00180030006010C01180110023FFE7FFE FFFE101C7E9B15>I<07E01830201C201C781E780E781E381E001C001C00180030006007 E00030001C001C000E000F000F700FF80FF80FF80FF00E401C201C183007E0101D7E9B15 >I<000C00000C00001C00003C00003C00005C0000DC00009C00011C00031C00021C0004 1C000C1C00081C00101C00301C00201C00401C00C01C00FFFFC0001C00001C00001C0000 1C00001C00001C00001C0001FFC0121C7F9B15>I<300C3FF83FF03FC020002000200020 002000200023E024302818301C200E000E000F000F000F600FF00FF00FF00F800E401E40 1C2038187007C0101D7E9B15>I<00F0030C06040C0E181E301E300C700070006000E3E0 E430E818F00CF00EE006E007E007E007E007E007600760077006300E300C18180C3003E0 101D7E9B15>I<4000007FFF807FFF007FFF004002008004008004008008000010000010 0000200000600000400000C00000C00001C0000180000180000380000380000380000380 00078000078000078000078000078000078000030000111D7E9B15>I<03E00C30100820 0C20066006600660067006780C3E083FB01FE007F007F818FC307E601E600FC007C003C0 03C003C00360026004300C1C1007E0101D7E9B15>I<03C00C301818300C700C600EE006 E006E007E007E007E007E0076007700F300F18170C2707C700060006000E300C780C7818 7010203030C00F80101D7E9B15>I<60F0F0600000000000000000000060F0F06004127C 910C>I<60F0F0600000000000000000000060F0F0701010101020204080041A7C910C>I< 7FFFFFC0FFFFFFE000000000000000000000000000000000000000000000000000000000 00000000FFFFFFE07FFFFFC01B0C7E8F20>61 D<0FE03038401CE00EF00EF00EF00E000C 001C0030006000C000800180010001000100010001000100000000000000000000000300 0780078003000F1D7E9C14>63 D<000600000006000000060000000F0000000F0000000F 00000017800000178000001780000023C0000023C0000023C0000041E0000041E0000041 E0000080F0000080F0000180F8000100780001FFF80003007C0002003C0002003C000600 3E0004001E0004001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E061800180198007000780 0E0003801C0003801C00018038000180780000807800008070000080F0000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000700000807800008078000080 380000801C0001001C0001000E000200070004000180080000E03000001FC000191E7E9C 1E>IIII<001F808000E0618001801980070007800E000380 1C0003801C00018038000180780000807800008070000080F0000000F0000000F0000000 F0000000F0000000F0000000F000FFF0F0000F8070000780780007807800078038000780 1C0007801C0007800E00078007000B800180118000E06080001F80001C1E7E9C21>II I<1FFF00F800780078007800780078007800780078007800780078007800780078007800 780078007800787078F878F878F878F0F040E021C01F00101D7F9B15>I76 DII<003F800000E0E000 0380380007001C000E000E001C0007003C00078038000380780003C0780003C0700001C0 F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0 780003C0780003C0380003803C0007801C0007000E000E0007001C000380380000E0E000 003F80001B1E7E9C20>II82 D<07E0801C1980300580700380600180E00180E00080E00080E000 80F00000F800007C00007FC0003FF8001FFE0007FF0000FF80000F800007C00003C00001 C08001C08001C08001C0C00180C00180E00300D00200CC0C0083F800121E7E9C17>I<7F FFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F0020800F0020800F002000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F800003 FFFC001B1C7F9B1E>II<0808101020204040404080 8080808080B0B0F8F8787830300D0C7A9C15>92 D<1FC000307000783800781C00301C00 001C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E80 1F870012127E9115>97 DI<07E00C 301878307870306000E000E000E000E000E000E00060007004300418080C3007C00E127E 9112>I<003F000007000007000007000007000007000007000007000007000007000007 0003E7000C1700180F00300700700700600700E00700E00700E00700E00700E00700E007 00600700700700300700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E 6006E006FFFEE000E000E000E00060007002300218040C1803E00F127F9112>I<00F801 8C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4C00C38C01C 3880181800381C00381C00381C00381C001818001C38000C300013C00010000030000018 00001FF8001FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007 F800121C7F9215>II<18003C003C 0018000000000000000000000000000000FC001C001C001C001C001C001C001C001C001C 001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C000000000 000000000000000000000FE000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>III II<03F0000E1C0018060030 0300700380600180E001C0E001C0E001C0E001C0E001C0E001C060018070038030030018 06000E1C0003F00012127F9115>II<03C1000C3300180B 00300F00700700700700E00700E00700E00700E00700E00700E00700600700700700300F 00180F000C370007C700000700000700000700000700000700000700000700003FE0131A 7E9116>II<1F9030704030C010C010E010F8007F803FE00FF000F88038 8018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C001C003C00FF E01C001C001C001C001C001C001C001C001C001C101C101C101C101C100C100E2003C00C 1A7F9910>IIII<7F8FF00F03800F0300 07020003840001C80001D80000F00000700000780000F800009C00010E00020E00060700 0403801E07C0FF0FF81512809116>II<7FFC7038603840 7040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E127F9112>I E /Fm 17 119 df<78FCFCFCFC7800000000000078FCFCFCFC7806127D910D>58 D<00038000000380000007C0000007C0000007C000000FE000000FE000001FF000001BF0 00001BF0000031F8000031F8000061FC000060FC0000E0FE0000C07E0000C07E0001803F 0001FFFF0003FFFF8003001F8003001F8006000FC006000FC00E000FE00C0007E0FFC07F FEFFC07FFE1F1C7E9B24>65 D<001FE02000FFF8E003F80FE007C003E00F8001E01F0000 E03E0000E03E0000607E0000607C000060FC000000FC000000FC000000FC000000FC0000 00FC000000FC000000FC0000007C0000607E0000603E0000603E0000C01F0000C00F8001 8007C0030003F80E0000FFFC00001FE0001B1C7D9B22>67 DI< 07F8201FFEE03C07E07801E07000E0F000E0F00060F00060F80000FE0000FFE0007FFE00 3FFF003FFF800FFFC007FFE0007FE00003F00001F00000F0C000F0C000F0C000E0E000E0 F001C0FC03C0EFFF0083FC00141C7D9B1B>83 D<0FF8001C1E003E0F803E07803E07C01C 07C00007C0007FC007E7C01F07C03C07C07C07C0F807C0F807C0F807C0780BC03E13F80F E1F815127F9117>97 D<03FC000E0E001C1F003C1F00781F00780E00F80000F80000F800 00F80000F80000F800007800007801803C01801C03000E0E0003F80011127E9115>99 D<01FC000F07001C03803C01C07801C07801E0F801E0F801E0FFFFE0F80000F80000F800 007800007C00603C00601E00C00F038001FC0013127F9116>101 D<1E003F003F003F003F001E00000000000000000000000000FF00FF001F001F001F001F 001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B1E7F9D0E>105 D108 D110 D<01FC000F07801C01C03C01E07800F07800F0F800F8F800F8F800F8F800F8F800F8F800 F87800F07800F03C01E01E03C00F078001FC0015127F9118>I114 D<1FD830786018E018E018F000FF807FE07FF01F F807FC007CC01CC01CE01CE018F830CFC00E127E9113>I<030003000300030007000700 0F000F003FFCFFFC1F001F001F001F001F001F001F001F001F001F0C1F0C1F0C1F0C0F08 079803F00E1A7F9913>II< FFC1FCFFC1FC1F00601F80E00F80C00FC0C007C18007C18003E30003E30001F60001F600 01FE0000FC0000FC0000780000780000300016127F9119>I E /Fn 53 123 df<00000F80003F8F80007F8F8000FF8F8001FF8F8003E0000003C0000007C000 0007C0000007C0000007C0000007C0000007C0000007C0000007C0000007C00000FFFF8F 80FFFF8F80FFFF8F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F 8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F 8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F8007C00F80192B7F AA20>12 D45 DI<007F000001 FFC00007FFF0000FFFF8000FC1F8001F007C003F007E003E003E003C001E007C001F007C 001F007C001F0078000F00F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8 000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F80F8000F8078 000F007C001F007C001F007C001F003E003E003E003E003F007E001F80FC000FC1F8000F FFF80007FFF00001FFC000007F000019297EA71E>48 D<00180000380000F80007F800FF F800FFF800F8F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000 F80000F80000F80000F80000F80000F80000F80000F80000F8007FFFF07FFFF07FFFF014 287BA71E>I<00FE0003FFC007FFE00FFFF01F03F83C00FC38007E78003E70003EF0001F F0001F60001F20001F00001F00001F00001F00003E00003E00007C00007C0000F80001F0 0001E00003C0000780000F00001E00003C0000780000F00001E00003C0000780000F0000 1E00003C00007FFFFF7FFFFF7FFFFF7FFFFF18287EA71E>I<007F000001FFC00007FFF0 000FFFF8001FC1F8003E007C003C003E0078003E0038003E0010003E0000003E0000003E 0000003C0000007C000000FC000001F8000007F00000FFE00000FFC00000FFE00000FFF0 000001FC0000007C0000003E0000001F0000001F0000000F8000000F8000000F8000000F 8000000F8040000F8060001F00F0001F00F8003F007E007E003F81FC001FFFF8000FFFF0 0003FFE000007F000019297EA71E>I<0003F0000007F0000005F000000DF000000DF000 001DF0000039F0000039F0000079F0000079F00000F1F00000F1F00001E1F00003E1F000 03E1F00007C1F00007C1F0000F81F0000F81F0001F01F0001F01F0003E01F0007C01F000 7C01F000F801F000FFFFFF80FFFFFF80FFFFFF80FFFFFF800001F0000001F0000001F000 0001F0000001F0000001F0000001F0000001F0000001F0000001F00019277EA61E>I<3F FFFC3FFFFC3FFFFC3FFFFC3E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00003E3F003EFFC03FFFE03FFFF03FE1F83F807C3F003E3E003E00003E00001F00 001F00001F00001F00001F00001F00001F20001F60003E70003EF8007C7C00FC3F03F81F FFF00FFFE007FF8000FE0018287EA61E>I<000FF000003FFC0000FFFC0001FFFC0003F8 0C0007E000000FC000000F8000001F0000001E0000003E0000003C0000007C0000007C00 00007C3FE000F8FFF000F9FFF800FBFFFC00FF807E00FF003E00FE003F00FC001F00FC00 1F00FC000F80F8000F80F8000F80F8000F80F8000F8078000F807C000F807C000F807C00 0F003E001F003E001F001F003E001F807C000FC1FC0007FFF80003FFF00001FFC000007F 000019297EA71E>II<007F000001FFC000 07FFF0000FFFF8001FC1FC003F007E003E003E007E003F007C001F007C001F007C001F00 7C001F007C001F003E003E003E003E001F007C000FC1F80007FFF00003FFE00003FFE000 0FFFF8001FC1FC003F007E003E003E007C001F007C001F00F8000F80F8000F80F8000F80 F8000F80F8000F80F8000F807C001F007C001F007E003F003F007E001FC1FC000FFFF800 07FFF00003FFE000007F000019297EA71E>I<007F000001FFC00003FFE0000FFFF0000F C1F8001F007C003E007C007C003E007C001E007C001F00F8001F00F8001F00F8000F00F8 000F80F8000F80F8000F80F8000F80F8001F807C001F807C001F807E003F803E007F803F 00FF801FFFEF800FFFCF8007FF8F8003FE1F0000001F0000001F0000001E0000003E0000 003E0000007C0000007C000000F8001801F0001E07E0003FFFC0001FFF80000FFE000003 F8000019297EA71E>I<0001F000000003F800000003F800000007FC00000007BC000000 07BC0000000F3E0000000F1E0000000F1E0000001F1F0000001E1F0000001E0F0000003E 0F8000003C0F8000003C078000007C07C000007807C00000F803E00000F803E00000F003 E00001F001F00001F001F00001E001F00003E000F80003E000F80003C000F80007FFFFFC 0007FFFFFC000FFFFFFE000F80003E000F80003E001F00003F001F00001F001E00001F00 3E00000F803E00000F803C00000F807C000007C07C000007C078000007C0F8000003E0F8 000003E0232A7EA928>65 D<0001FF00000FFFE0003FFFF8007FFFF800FE01F801F80030 03F0001007C000000F8000001F8000001F0000003E0000003E0000007E0000007C000000 7C0000007C000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 F8000000F8000000F80000007C0000007C0000007C0000007E0000003E0000003E000000 1F0000001F8000000F80000007C0000003F0000401F8001C00FE00FC007FFFFC003FFFF8 000FFFE00001FF001E2C7CAA26>67 DIII72 DI75 D77 DI<0001FC0000000FFF8000003FFFE000 007FFFF00001FE03FC0003F800FE0007E0003F0007C0001F000F80000F801F000007C01F 000007C03E000003E03E000003E07C000001F07C000001F07C000001F078000000F0F800 0000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F8000000F8F80000 00F8F8000000F8F8000000F87C000001F07C000001F07C000001F07E000003F03E000003 E03F000007E01F000007C01F80000FC00FC0001F8007E0003F0007F0007F0003F800FE00 01FE03FC0000FFFFF800003FFFE000000FFF80000001FC0000252C7DAA2C>II82 D<007FC00001FFF80007FFFE000F FFFF001FC07F003F000F007E0006007C0000007C000000F8000000F8000000F8000000F8 000000F8000000FC0000007E0000007F0000003F8000001FF800000FFF800007FFE00003 FFF80000FFFC00000FFE000000FF0000003F0000001F8000000F8000000FC0000007C000 0007C0000007C0000007C0000007C0000007C000000F8060000F80F0001F00FC003F00FF 80FE007FFFFC001FFFF80007FFE00000FF80001A2C7DAA21>II86 DI<01FE000FFF803FFFC03FFFE03C03F03001F00001F800 00F80000F80000F80000F80000F8007FF807FFF81FFFF83FE0F87F00F8FC00F8F800F8F8 00F8F800F8FC01F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1D>97 DI<007FC001FFF007FFFC0F FFFC1FC07C1F00083E00007C00007C00007C0000F80000F80000F80000F80000F80000F8 0000F800007C00007C00007E00003E00001F000C1FC07C0FFFFC07FFFC01FFF0007F8016 1B7E9A1B>I<00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E 00003E00003E00003E00003E00003E00FC3E03FF3E07FFFE0FFFFE1FC1FE3F007E3E003E 7C003E7C003EFC003EF8003EF8003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E 7C003E3E007E3F00FE1FC1FE0FFFFE07FFBE03FF3E00FC3E172A7EA91F>I<007E0003FF 8007FFC00FFFE01F83F03F00F03E00787C00787C003878003CFFFFFCFFFFFCFFFFFCFFFF FCF80000F80000F800007800007C00007C00003E00003F000C1FC07C0FFFFC07FFFC01FF F0007F80161B7E9A1B>I<001FC0007FC000FFC001FFC003F00003E00007C00007C00007 C00007C00007C00007C00007C00007C00007C000FFFE00FFFE00FFFE0007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C000122A7FA912>I< 00F8078003FE7FC00FFFFFC01FFFFFC01F07C0003E03E0003E03E0007C01F0007C01F000 7C01F0007C01F0007C01F0007C01F0003E03E0003E03E0001F07C0001FFFC0003FFF8000 3BFE000038F8000078000000780000003C0000003FFFC0003FFFF8001FFFFC001FFFFE00 3FFFFF007C007F00F8001F80F8000F80F8000F80F8000F80FC001F807E003F003F80FE00 3FFFFE000FFFF80007FFF00000FF80001A287E9A1E>III108 DII<007F000001FFC00007FFF0000FFFF8001FC1 FC003F007E003E003E007C001F007C001F0078000F00F8000F80F8000F80F8000F80F800 0F80F8000F80F8000F80F8000F807C001F007C001F007E003F003E003E003F007E001FC1 FC000FFFF80007FFF00001FFC000007F0000191B7E9A1E>II114 D<03FC001FFF803FFFC07FFFC07C07C0F80080F80000F80000F80000FC00007F80007FF8 003FFE001FFF0007FF8000FFC0000FE00007E00003E00003E04003E0E007E0FC0FC0FFFF C07FFF801FFE0003F800131B7E9A17>I<07C00007C00007C00007C00007C00007C00007 C000FFFFC0FFFFC0FFFFC007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C04007E1C003 FFE003FFE001FF8000FC0013227FA116>IIII<7C000FC03E001F803F001F001F803E000F807C0007C0FC0003E0F80001F1F00001 FBE00000FFC000007FC000003F8000001F0000001F0000003F8000007FC00000FBC00000 F3E00001F1F00003E0F80007C07C000F807C000F803E001F001F003E000F807E000FC0FC 0007E01B1B809A1C>III E /Fo 15 122 df<07E00FF03FFC3FFC7FFEFFFFFFFFFFFFFFFFFF FFFFFF7FFE3FFC3FFC0FF007E01010788F21>46 D<000001E00000000003F0000000000F F0000000003FF000000000FFF00000000FFFF0000003FFFFF00000FFFFFFF00000FFFFFF F00000FFFFFFF00000FFF0FFF00000FC00FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000000000FFF000000000FFF000000000FFF000000000FFF000000000FFF000000000FF F000003FFFFFFFFFC03FFFFFFFFFC03FFFFFFFFFC03FFFFFFFFFC03FFFFFFFFFC02A4478 C33B>49 D<0000FFE00000000FFFFE0000003FFFFFC00000FFFFFFF00003FFFFFFFC0007 FC01FFFE000FE0007FFF001FC0001FFF803F80000FFFC03FE00007FFE07FF80007FFE07F FC0003FFF0FFFC0003FFF0FFFE0001FFF8FFFE0001FFF8FFFE0001FFF8FFFE0000FFFCFF FE0000FFFC7FFC0000FFFC7FFC0000FFFC3FF80000FFFC1FF00000FFFC0FE00000FFFC00 000001FFFC00000001FFF800000001FFF800000001FFF000000003FFF000000003FFE000 000003FFE000000007FFC000000007FF800000000FFF800000001FFF000000001FFE0000 00003FFC000000007FF8000000007FE000000000FFC000000001FF8000000003FF000000 0007FC000000000FF8000000001FF0000000003FC0000000007F80007C0000FF00007C00 00FE00007C0001FC00007C0003F00000F80007E00000F8000FC00000F8001F800000F800 3F000001F8007C000003F800FFFFFFFFF801FFFFFFFFF003FFFFFFFFF007FFFFFFFFF00F FFFFFFFFF01FFFFFFFFFF03FFFFFFFFFF07FFFFFFFFFF07FFFFFFFFFF0FFFFFFFFFFE0FF FFFFFFFFE0FFFFFFFFFFE0FFFFFFFFFFE02E447AC33B>I77 D<0007FFFC000000007FFFFFC0000001FFFF FFF8000003FFFFFFFE000007FE001FFF000007FF0003FFC0000FFF8001FFE0000FFF8000 FFF0000FFF80007FF0000FFF80007FF8000FFF80007FF80007FF00003FFC0007FF00003F FC0003FE00003FFC0000F800003FFC00000000003FFC00000000003FFC00000000003FFC 00000000003FFC00000007FFFFFC000000FFFFFFFC000007FFFFFFFC00003FFFE03FFC00 00FFFE003FFC0003FFF0003FFC0007FFC0003FFC000FFF00003FFC001FFE00003FFC003F FC00003FFC007FF800003FFC007FF800003FFC00FFF000003FFC00FFF000003FFC00FFF0 00003FFC00FFF000003FFC00FFF000003FFC00FFF000007FFC007FF80000FFFC007FF800 01EFFC003FFC0003EFFC003FFF0007CFFF000FFFC03F8FFFF807FFFFFF07FFFC01FFFFFC 03FFFC007FFFF001FFFC0003FF80007FF8362E7DAD3A>97 D<00001FFFC0000000FFFFF8 000007FFFFFE00001FFFFFFF80007FFC00FFC000FFE001FFC001FFC003FFE003FF8003FF E007FF0003FFE00FFE0003FFE00FFE0003FFE01FFC0001FFC01FFC0001FFC03FFC0000FF 803FFC00003E007FF8000000007FF8000000007FF800000000FFF800000000FFF8000000 00FFF800000000FFF800000000FFF800000000FFF800000000FFF800000000FFF8000000 00FFF800000000FFF8000000007FF8000000007FF8000000007FFC000000003FFC000000 003FFC000000001FFC000000F81FFE000000F80FFE000000F80FFF000001F007FF800003 F003FFC00007E001FFE0000FC000FFF0001F80007FFE00FF00001FFFFFFE000007FFFFF8 000000FFFFE00000001FFE00002D2E7CAD35>99 D<00001FFE00000001FFFFE0000007FF FFF800001FFFFFFE00007FFC07FF0000FFE001FF8001FFC0007FC003FF80003FE007FF00 003FF00FFE00001FF01FFE00000FF81FFC00000FF83FFC00000FFC3FFC000007FC7FFC00 0007FC7FF8000007FC7FF8000007FE7FF8000007FEFFF8000007FEFFF8000007FEFFFFFF FFFFFEFFFFFFFFFFFEFFFFFFFFFFFEFFFFFFFFFFFCFFF800000000FFF800000000FFF800 000000FFF8000000007FF8000000007FF8000000007FFC000000003FFC000000003FFC00 0000003FFC0000001C1FFE0000003E0FFE0000003E07FF0000007E07FF000000FC03FF80 0001F801FFC00003F0007FF0001FE0003FFE00FFC0001FFFFFFF800007FFFFFE000000FF FFF80000000FFF80002F2E7DAD36>101 D<000000FFC000000007FFF80000003FFFFC00 0000FFFFFF000001FFC1FF000007FF03FF80000FFC03FF80000FF807FFC0001FF807FFC0 003FF007FFC0003FF007FFC0003FE003FF80007FE003FF80007FE001FF00007FE000FE00 007FE0003800007FE0000000007FE0000000007FE0000000007FE0000000007FE0000000 007FE0000000007FE0000000007FE0000000007FE0000000007FE0000000FFFFFFFE0000 FFFFFFFE0000FFFFFFFE0000FFFFFFFE0000FFFFFFFE0000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000007FF0000000 007FF0000000007FF0000000007FF0000000007FF00000003FFFFFF800003FFFFFF80000 3FFFFFF800003FFFFFF800003FFFFFF800002A487DC724>I<00FC0001FE0003FF0007FF 800FFFC01FFFE01FFFE01FFFE01FFFE01FFFE01FFFE00FFFC007FF8003FF0001FE0000FC 00000000000000000000000000000000000000000000000000000000000000000000007F C0FFFFC0FFFFC0FFFFC0FFFFC0FFFFC003FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FFC001FF C001FFC001FFC001FFC001FFC0FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF18497CC820>105 D<007FC000FFFFC000FFFFC000FFFFC000FFFFC000FFFFC00003FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC00001FFC0 0001FFC00001FFC00001FFC00001FFC000FFFFFF80FFFFFF80FFFFFF80FFFFFF80FFFFFF 8019487CC720>108 D<007FC001FFC00000FFFFC00FFFF80000FFFFC03FFFFE0000FFFF C0FFFFFF0000FFFFC1FC07FF8000FFFFC3E003FFC00003FFC7C001FFC00001FFCF0001FF E00001FFDE0000FFE00001FFDC0000FFE00001FFFC0000FFF00001FFF80000FFF00001FF F00000FFF00001FFF00000FFF00001FFF00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FF F00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FFE00000FFF00001FF E00000FFF00001FFE00000FFF000FFFFFFC07FFFFFE0FFFFFFC07FFFFFE0FFFFFFC07FFF FFE0FFFFFFC07FFFFFE0FFFFFFC07FFFFFE03B2E7CAD42>110 D<00000FFF0000000000 FFFFF000000007FFFFFE0000001FFFFFFF8000003FFC03FFC00000FFE0007FF00001FF80 001FF80003FF00000FFC0007FE000007FE000FFE000007FF000FFC000003FF001FFC0000 03FF803FFC000003FFC03FF8000001FFC03FF8000001FFC07FF8000001FFE07FF8000001 FFE07FF8000001FFE0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FF F0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0FFF8000001FFF0 FFF8000001FFF07FF8000001FFE07FF8000001FFE07FF8000001FFE07FF8000001FFE03F FC000003FFC03FFC000003FFC01FFC000003FF801FFE000007FF800FFE000007FF0007FF 00000FFE0003FF80001FFC0001FFC0003FF80000FFE0007FF000007FFC03FFE000001FFF FFFF80000007FFFFFE00000000FFFFF0000000000FFF000000342E7DAD3B>I<00FF803F 8000FFFF80FFF000FFFF83FFFC00FFFF87FFFE00FFFF8FC3FF00FFFF8F07FF0003FF9E0F FF8001FFBC0FFF8001FFB80FFF8001FFF80FFF8001FFF00FFF8001FFF007FF0001FFF007 FF0001FFE003FE0001FFE000F80001FFE000000001FFE000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000000001FFC000 000001FFC000000001FFC000000001FFC000000001FFC0000000FFFFFFE00000FFFFFFE0 0000FFFFFFE00000FFFFFFE00000FFFFFFE00000292E7CAD31>114 D<000FFF00E0007FFFF3E001FFFFFFE007FFFFFFE00FF800FFE01FC0001FE03F80000FE0 3F000007E07F000003E07F000003E0FF000003E0FF000003E0FF800003E0FFC0000000FF F0000000FFFE000000FFFFF800007FFFFFC0007FFFFFF0003FFFFFFC001FFFFFFF000FFF FFFF8007FFFFFFC003FFFFFFE000FFFFFFF0003FFFFFF00003FFFFF800001FFFF8000000 FFFC0000001FFC7800000FFCF8000007FCF8000003FCFC000003FCFC000003FCFE000003 F8FE000003F8FF000003F8FF800007F0FFC0000FF0FFF0001FE0FFFC00FFC0FFFFFFFF80 FC7FFFFE00F81FFFF800E003FF8000262E7CAD2F>I<7FFFFFC000FFFF807FFFFFC000FF FF807FFFFFC000FFFF807FFFFFC000FFFF807FFFFFC000FFFF8000FFF000000FE00000FF F800000FC00000FFF800000FC000007FFC00000F8000007FFC00001F8000003FFC00001F 0000003FFE00003F0000001FFE00003E0000001FFF00007E0000000FFF00007C0000000F FF8000FC00000007FF8000F800000007FFC001F800000003FFC001F000000003FFE003F0 00000003FFE003F000000001FFF003E000000001FFF007E000000000FFF007C000000000 FFF80FC0000000007FF80F80000000007FFC1F80000000003FFC1F00000000003FFE3F00 000000001FFE3E00000000001FFF7E00000000000FFF7C00000000000FFFFC0000000000 0FFFFC000000000007FFF8000000000007FFF8000000000003FFF0000000000003FFF000 0000000001FFE0000000000001FFE0000000000000FFC0000000000000FFC00000000000 007F800000000000007F800000000000003F000000000000003F000000000000003F0000 00000000003E000000000000007E000000000000007C00000000000000FC000000001F80 00F8000000003FC001F8000000007FE001F000000000FFF003F000000000FFF003E00000 0000FFF007E000000000FFF00FC000000000FFF01F8000000000FFF03F80000000007FE0 7F00000000007F43FE00000000003FFFF800000000001FFFF0000000000007FFC0000000 000001FE00000000000039427EAD3F>121 D E /Fp 8 117 df<0003FF8000001FFFF000 007FFFFE0000FE03FF0001F000FF8003C000FFC00780007FE00FF0007FF00FF8007FF01F FC007FF81FFE007FF81FFE007FF81FFE007FF81FFE007FF81FFE007FF80FFC007FF007F8 007FF003F0007FF0000000FFE0000000FFC0000001FF80000001FF00000003FE00000007 FC0000001FF000000FFFC000000FFF8000000FFFF800000003FE00000000FF800000007F E00000003FF00000003FF80000003FFC0000001FFC0000001FFE0000001FFE0200001FFF 1FC0001FFF3FE0001FFF7FF0001FFF7FF0001FFFFFF8001FFFFFF8001FFFFFF8001FFEFF F8001FFEFFF0001FFE7FF0003FFC7FE0003FFC3FC0003FF81F80007FF01FE000FFE007FC 03FFC003FFFFFF0001FFFFFE00003FFFF0000007FF800028397CB731>51 D<0000001FFF000030000001FFFFE000F000000FFFFFFC01F000007FFFFFFE03F00001FF FE007F87F00003FFE0000FCFF0000FFF000003FFF0001FFC000001FFF0003FF80000007F F0007FF00000003FF000FFC00000003FF001FFC00000001FF003FF800000000FF007FF00 0000000FF00FFF0000000007F00FFE0000000007F01FFE0000000003F01FFE0000000003 F03FFC0000000003F03FFC0000000001F03FFC0000000001F07FFC0000000001F07FF800 00000001F07FF80000000000007FF8000000000000FFF8000000000000FFF80000000000 00FFF8000000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF800 0000000000FFF8000000000000FFF8000000000000FFF8000000000000FFF80000000000 007FF80000000000007FF80000000000007FF80000000000007FFC0000000000F03FFC00 00000000F03FFC0000000000F03FFC0000000000F01FFE0000000000F01FFE0000000001 E00FFE0000000001E00FFF0000000001E007FF0000000003C003FF8000000003C001FFC0 000000078000FFE00000000F00007FF00000001F00003FF80000003E00001FFC0000007C 00000FFF000001F8000003FFE00007F0000001FFFE003FC00000007FFFFFFF000000000F FFFFFC0000000001FFFFF000000000001FFF0000003C3D7BBB47>67 D<001FFF00000001FFFFF0000003FFFFFC000007F007FE00000FF801FF00001FFC00FF80 001FFC007FC0001FFC007FE0001FFC003FE0000FF8003FF0000FF8003FF00007F0003FF0 0001C0003FF0000000003FF0000000003FF0000000003FF0000000FFFFF000000FFFFFF0 00007FF83FF00001FF803FF00007FE003FF0000FF8003FF0001FF0003FF0003FE0003FF0 007FE0003FF0007FE0003FF000FFC0003FF000FFC0003FF000FFC0003FF000FFC0003FF0 00FFC0007FF0007FE0007FF0007FE000DFF0003FF0039FF8001FFC0F0FFFF007FFFE0FFF F001FFFC07FFF0003FE000FFF02C267DA530>97 D<0001FFC000000FFFF800003FFFFE00 00FF80FF0001FE003F8007FC001FC00FF8000FE00FF8000FF01FF00007F03FF00007F83F F00007F87FE00007F87FE00003FC7FE00003FC7FE00003FCFFE00003FCFFFFFFFFFCFFFF FFFFFCFFFFFFFFFCFFE0000000FFE0000000FFE0000000FFE00000007FE00000007FE000 00007FE00000003FE00000003FF000003C1FF000003C1FF000003C0FF800007807FC0000 F803FE0001F001FF0007E000FFC03FC0003FFFFF000007FFFC000000FFE00026267DA52D >101 D<00FF00000000FFFF00000000FFFF00000000FFFF00000000FFFF0000000007FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF 0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF007FC00003FF 01FFF80003FF07FFFC0003FF0F03FE0003FF1C01FF0003FF3001FF8003FF6000FF8003FF E000FFC003FFC000FFC003FF8000FFC003FF8000FFC003FF8000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF0000FFC003FF 0000FFC003FF0000FFC003FF0000FFC0FFFFFC3FFFFFFFFFFC3FFFFFFFFFFC3FFFFFFFFF FC3FFFFF303C7CBB37>104 D<00FF01FF8000FFFF0FFFF000FFFF3FFFFC00FFFFFE03FF 00FFFFF000FF8003FFC0007FC003FF80003FE003FF00003FF003FF00001FF803FF00001F FC03FF00000FFC03FF00000FFE03FF00000FFE03FF000007FE03FF000007FF03FF000007 FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007FF03FF000007 FF03FF000007FF03FF000007FE03FF000007FE03FF00000FFE03FF00000FFC03FF00000F FC03FF00001FF803FF00001FF803FF00003FF003FF80003FE003FFC0007FC003FFF001FF 8003FFFC07FF0003FF3FFFFC0003FF0FFFF00003FF01FF000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF0000000003FF000000 0003FF0000000003FF0000000003FF0000000003FF0000000003FF00000000FFFFFC0000 00FFFFFC000000FFFFFC000000FFFFFC00000030377DA537>112 D<00FE03F000FFFE0FFE00FFFE1FFF00FFFE3C3F80FFFE707FC007FE60FFE003FEE0FFE0 03FEC0FFE003FFC0FFE003FF807FC003FF807FC003FF803F8003FF800E0003FF00000003 FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF 00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00000003FF00 000003FF00000003FF00000003FF00000003FF00000003FF000000FFFFFE0000FFFFFE00 00FFFFFE0000FFFFFE000023267DA529>114 D<00078000000780000007800000078000 00078000000F8000000F8000000F8000000F8000001F8000001F8000003F8000003F8000 007F800000FF800001FF800007FF80001FFFFFF0FFFFFFF0FFFFFFF0FFFFFFF001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF800001FF8000 01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C01FF803C00FF8078 00FFC078007FC070003FE0E0001FFFC00007FF800001FF001E377EB626>116 D E /Fq 83 124 df<001F83E000F06E3001C078780380F8780300F03007007000070070 000700700007007000070070000700700007007000FFFFFF800700700007007000070070 000700700007007000070070000700700007007000070070000700700007007000070070 000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E00700000700000700000700000700000700 00FFFFE00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000 E0E001C1E00381E00700E00700E00700E00700E00700E00700E00700E00700E0FFFFE007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E07FE7FE1720809F19>I<001F81F80000F04F04 0001C07C06000380F80F000300F00F000700F00F00070070000007007000000700700000 070070000007007000000700700000FFFFFFFF0007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070 07007FE3FE3FF02420809F26>I<001F81FF0000F06F070001C07C0F000380F80F000300 F00700070070070007007007000700700700070070070007007007000700700700070070 0700FFFFFFFF000700700700070070070007007007000700700700070070070007007007 000700700700070070070007007007000700700700070070070007007007000700700700 070070070007007007000700700700070070070007007007007FE3FE3FF02420809F26> I<70F8F8F8F8F8F8F8707070707070707070702020202020000000000070F8F8F8700521 7CA00D>33 D<7038F87CFC7EFC7E743A0402040204020804080410081008201040200F0E 7E9F17>I<00780000008400000184000003020000070200000702000007020000070200 00070400000704000007080000070800000310000003A00FFC03C003E0038001C001C000 8001C0010003E0010004E0020008F00200187004003078080070380800701C1000F01E10 00F00E2000F0074000F003C0087003C0087801C010380670301C18386007E00F801E227E A023>38 D<70F8FCFC74040404080810102040060E7C9F0D>I<00200040008001000200 06000C000C00180018003000300030007000600060006000E000E000E000E000E000E000 E000E000E000E000E000E0006000600060007000300030003000180018000C000C000600 020001000080004000200B2E7DA112>I<800040002000100008000C0006000600030003 0001800180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000 E000E000C000C000C001C001800180018003000300060006000C00080010002000400080 000B2E7DA112>I<01800180018001800180C183F18F399C0FF003C003C00FF0399CF18F C1830180018001800180018010147DA117>I<0006000000060000000600000006000000 060000000600000006000000060000000600000006000000060000000600000006000000 06000000060000FFFFFFF0FFFFFFF0000600000006000000060000000600000006000000 060000000600000006000000060000000600000006000000060000000600000006000000 0600001C207D9A23>I<70F8FCFC74040404080810102040060E7C840D>II<70F8F8F87005057C840D>I<000100030003000600060006000C000C000C0018 0018001800300030003000600060006000C000C000C00180018001800300030003000600 060006000C000C000C00180018001800300030003000600060006000C000C000C000102D 7DA117>I<03F0000E1C001C0E00180600380700700380700380700380700380F003C0F0 03C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C070 03807003807003807807803807001806001C0E000E1C0003F000121F7E9D17>I<018003 800F80F38003800380038003800380038003800380038003800380038003800380038003 800380038003800380038003800380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E 00200700400780800780F007C0F803C0F803C0F803C02007C00007C0000780000780000F 00000E00001C0000380000700000600000C0000180000300000600400C00401800401000 803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E00200F00780F8078078078 0780380F80000F80000F00000F00000E00001C0000380003F000003C00000E00000F0000 07800007800007C02007C0F807C0F807C0F807C0F00780400780400F00200E001C3C0003 F000121F7E9D17>I<000600000600000E00000E00001E00002E00002E00004E00008E00 008E00010E00020E00020E00040E00080E00080E00100E00200E00200E00400E00C00E00 FFFFF0000E00000E00000E00000E00000E00000E00000E0000FFE0141E7F9D17>I<1803 001FFE001FFC001FF8001FE00010000010000010000010000010000010000011F000161C 00180E001007001007800003800003800003C00003C00003C07003C0F003C0F003C0E003 80400380400700200600100E000C380003E000121F7E9D17>I<007C000182000701000E 03800C07801C0780380300380000780000700000700000F1F000F21C00F40600F80700F8 0380F80380F003C0F003C0F003C0F003C0F003C07003C07003C070038038038038070018 07000C0E00061C0001F000121F7E9D17>I<4000007FFFC07FFF807FFF80400100800200 80020080040000080000080000100000200000200000400000400000C00000C00001C000 018000038000038000038000038000078000078000078000078000078000078000078000 030000121F7D9D17>I<03F0000C0C001006003003002001806001806001806001807001 807803003E03003F06001FC8000FF00003F80007FC000C7E00103F00300F806003804001 C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C0003F000121F7E9D17 >I<03F0000E18001C0C00380600380700700700700380F00380F00380F003C0F003C0F0 03C0F003C0F003C07007C07007C03807C0180BC00E13C003E3C000038000038000038000 0700300700780600780E00700C002018001070000FC000121F7E9D17>I<70F8F8F87000 00000000000000000070F8F8F87005147C930D>I<70F8F8F87000000000000000000000 70F0F8F878080808101010202040051D7C930D>I<7FFFFFE0FFFFFFF000000000000000 00000000000000000000000000000000000000000000000000FFFFFFF07FFFFFE01C0C7D 9023>61 D<0FC0307040384038E03CF03CF03C603C0038007000E000C001800180010003 000200020002000200020002000000000000000000000007000F800F800F8007000E207D 9F15>63 D<000100000003800000038000000380000007C0000007C0000007C0000009E0 000009E0000009E0000010F0000010F0000010F00000207800002078000020780000403C 0000403C0000403C0000801E0000801E0000FFFE0001000F0001000F0001000F00020007 800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0 038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C00004078000040 F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 780000007C0000407C0000403C0000401C0000401E0000800E0000800700010003800200 01C0040000703800000FC0001A217D9F21>IIII72 D I<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00203C 00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>I76 DII<001F800000F0 F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E07800 01E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F800 01F0F80001F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F00 0F0007801E0001C0380000F0F000001F80001C217D9F23>II82 D<07E0800C1980100780300380600180600180E00180E00080E00080E00080F000 00F000007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001 C08001C08001C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19 >I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F0030800F0010800F0010800F 0010800F0010000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000001F800007FFFE001C1F7E9E21>IIII<7FF83FF80FE00FC007C0 070003C0020001E0040001F00C0000F0080000781000007C1000003C2000003E4000001E 4000000F8000000F8000000780000003C0000007E0000005E0000009F0000018F8000010 780000207C0000603C0000401E0000801F0001800F0001000780020007C0070003C01F80 07E0FFE01FFE1F1F7F9E22>II< FEFEC0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0 C0C0C0C0C0C0C0FEFE072D7CA10D>91 D<080410082010201040204020804080408040B8 5CFC7EFC7E7C3E381C0F0E7B9F17>II<1FE00030 3000781800781C00300E00000E00000E00000E0000FE00078E001E0E00380E00780E00F0 0E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00 780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F0 10147E9314>I<000380003F800003800003800003800003800003800003800003800003 8000038000038003E380061B801C0780380380380380700380700380F00380F00380F003 80F00380F00380F003807003807003803803803807801C07800E1B8003E3F815207E9F19 >I<03F0000E1C001C0E00380700380700700700700380F00380F00380FFFF80F00000F0 0000F000007000007000003800801800800C010007060001F80011147F9314>I<007C00 C6018F038F07060700070007000700070007000700FFF007000700070007000700070007 00070007000700070007000700070007000700070007007FF01020809F0E>I<0000E003 E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E380033 E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000 00000000000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000E000000000000000000000 0000007007F000F000700070007000700070007000700070007000700070007000700070 00700070007000700070007000706070F060F0C061803F000C28829E0E>I<0E0000FE00 000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0FF00E03 C00E03000E02000E04000E08000E10000E30000E70000EF8000F38000E1C000E1E000E0E 000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E0 0E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000 E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C03 80070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E 00780E00780E00780E00780E00780E00780E00700E00F00E00E00F01E00F01C00EC3000E 3E000E00000E00000E00000E00000E00000E00000E00000E0000FFE000151D7F9319>I< 03E0800619801C05803C0780380380780380700380F00380F00380F00380F00380F00380 F003807003807803803803803807801C0B800E138003E380000380000380000380000380 000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E000E00 0E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F 800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E00 0E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC00007800003800003C 00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714809318>II<3FFF380E200E201C40384078407000 E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>II E /Fr 46 122 df<00200040008001000300060004000C000C0018001800 3000300030007000600060006000E000E000E000E000E000E000E000E000E000E000E000 E000E000E0006000600060007000300030003000180018000C000C000400060003000100 0080004000200B327CA413>40 D<800040002000100018000C0004000600060003000300 01800180018001C000C000C000C000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000C000C000C001C0018001800180030003000600060004000C0018001000 2000400080000B327DA413>I<70F8FCFC7404040404080810102040060F7C840E>44 DI<70F8F8F87005057C840E>I<008003800F80F3800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 8003800380038003800380038007C0FFFE0F217CA018>49 D<03F0000C1C001007002007 804003C04003C08003E0F003E0F801E0F801E0F801E02003E00003E00003C00003C00007 80000700000E00001C0000180000300000600000C0000180000100000200200400200800 201800603000403FFFC07FFFC0FFFFC013217EA018>I<000200000600000E00000E0000 1E00001E00002E00004E00004E00008E00008E00010E00020E00020E00040E00040E0008 0E00100E00100E00200E00200E00400E00800E00FFFFF8000E00000E00000E00000E0000 0E00000E00000E00001F0001FFF015217FA018>52 D<1000801E07001FFF001FFE001FF8 0013E00010000010000010000010000010000010000010F800130E001407001803801003 800001C00001C00001E00001E00001E00001E07001E0F001E0F001E0E001C08001C04003 C04003802007001006000C1C0003F00013227EA018>I<007E0001C1000300800601C00E 03C01C03C0180180380000380000780000700000700000F0F800F30C00F40600F40300F8 0380F801C0F001C0F001E0F001E0F001E0F001E0F001E07001E07001E07001E03801C038 01C01803801C03000C0600070C0001F00013227EA018>I<01F800060E00080300100180 2001802000C06000C06000C06000C07000C07801803E01003F02001FC4000FF80003F800 03FC00067F00083F80100F803007C06001C06000E0C000E0C00060C00060C00060C00060 6000406000C03000801803000E0E0003F00013227EA018>56 D<01F000060C000C060018 0700380380700380700380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E070 01E07003E03803E01805E00C05E00619E003E1E00001C00001C00001C000038000038030 0300780700780600700C002018001030000FC00013227EA018>I<000180000001800000 0180000003C0000003C0000003C0000005E0000005E000000DF0000008F0000008F00000 10F800001078000010780000203C0000203C0000203C0000401E0000401E0000401E0000 800F0000800F0000FFFF000100078001000780030007C0020003C0020003C0040003E004 0001E0040001E00C0000F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C00170038000F0070000F00E0000701E0000701C0000 303C0000303C0000307C0000107800001078000010F8000000F8000000F8000000F80000 00F8000000F8000000F8000000F800000078000000780000107C0000103C0000103C0000 101C0000201E0000200E000040070000400380008001C0010000E0020000381C000007E0 001C247DA223>67 DIII72 DI77 DI80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020 E00020E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E0 0001E00000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0 C80180C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008 40078008C007800C80078004800780048007800480078004000780000007800000078000 000780000007800000078000000780000007800000078000000780000007800000078000 000780000007800000078000000780000007800000078000000780000007800000078000 000FC00003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700 00070000FF0007C7001E07003C0700780700700700F00708F00708F00708F00F08781708 3C23900FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E 003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F 00700C80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000 700000F00000F00000F00000F00000F00000F00000F000007000007800403800401C0080 0C010007060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800 E07800E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800 E01801E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C078 01E07000E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C 00400E008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700 07000700070007000700FFF8070007000700070007000700070007000700070007000700 07000700070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C 07001C07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010 00001800001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860 00306000303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E0 0F00700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000 0000000000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E0 01C00000000000000000000000000000000001E00FE001E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0 F0C0F18061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E0200 0E04000E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C0 0E01E00E00F00E00F8FFE3FE17237FA21A>I<0E1FC07F00FE60E183801E807201C00F00 3C00E00F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E0038 00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800 E00E003800E00E003800E0FFE3FF8FFE27157F942A>109 D<0E1F80FE60C01E80E00F00 700F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C038 00E0700070700070F00078F00078F00078F00078F00078F00078F000787000707800F038 00E01C01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E0038 0E003C0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E0038 0F00700E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E0000 0E0000FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E 000E000E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C008E008F0007F803FE00FF001F8003C801C800C800CC00CC0 08E018D0308FC00E157E9413>I<02000200020002000600060006000E001E003E00FFF8 0E000E000E000E000E000E000E000E000E000E000E000E040E040E040E040E040E040708 030801F00E1F7F9E13>I<0E0070FE07F01E00F00E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00700E00700E00700E00700E00F00E00F006017003827800 FC7F18157F941B>I119 D121 D E /Fs 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E0006E03C0003E0380001E078 0000E0700000E070000060F0000060F0000060F0000020F0000020F0000020F8000020F8 0000007C0000007E0000003F0000003FC000001FF800000FFF800007FFF80003FFFC0000 FFFF00000FFF800000FFC000001FE0000007E0000003F0000001F0000000F0000000F800 0000F88000007880000078800000788000007880000078C0000078C0000070E00000F0E0 0000E0F00000E0F80001C0EC000380C7000700C1F01E00807FFC00800FF0001D337CB125 >83 D<00FE00000303C0000C00E00010007000100038003C003C003E001C003E001E003E 001E0008001E0000001E0000001E0000001E00000FFE0000FC1E0003E01E000F801E001F 001E003E001E003C001E007C001E00F8001E04F8001E04F8001E04F8003E04F8003E0478 003E047C005E043E008F080F0307F003FC03E01E1F7D9E21>97 D<003F8000E060038018 0700040F00041E001E1C003E3C003E7C003E7C0008780000F80000F80000F80000F80000 F80000F80000F80000F80000F800007800007C00007C00003C00011E00011E00020F0002 07000403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E0 000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0 000001E0000001E0000001E0000001E0000001E0000001E0001F81E000F061E001C019E0 078005E00F0003E00E0003E01E0001E03C0001E03C0001E07C0001E0780001E0F80001E0 F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001E0780001E0 780001E03C0001E03C0001E01C0001E01E0003E00E0005E0070009E0038011F000E061FF 003F81FF20327DB125>I<003F800000E0E0000380380007003C000E001E001E001E001C 000F003C000F007C000F0078000F8078000780F8000780F8000780FFFFFF80F8000000F8 000000F8000000F8000000F8000000F8000000780000007C0000003C0000003C0000801E 0000800E0001000F0002000780020001C00C0000F03000001FC000191F7E9E1D>I<0007 E0001C1000383800707C00E07C01E07C01C03803C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C000FFFFC0FFFFC003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00007E0007FFF 007FFF0016327FB114>I<000000F0007F030801C1C41C0380E81C070070080F0078001E 003C001E003C003E003E003E003E003E003E003E003E003E003E003E003E001E003C001E 003C000F007800070070000780E00009C1C000087F000018000000180000001800000018 000000180000001C0000000E0000000FFFF80007FFFF0003FFFF800E000FC0180001E030 0000F070000070E0000038E0000038E0000038E0000038E0000038700000707000007038 0000E01C0001C00700070001C01C00003FE0001E2F7E9F21>I<07000F801F801F800F80 0700000000000000000000000000000000000000000000000780FF80FF800F8007800780 078007800780078007800780078007800780078007800780078007800780078007800780 07800780078007800FC0FFF8FFF80D307EAF12>105 D<0780FE001FC000FF83078060F0 00FF8C03C18078000F9001E2003C0007A001E4003C0007A000F4001E0007C000F8001E00 07C000F8001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E000780 00F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000 F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0 001E000FC001F8003F00FFFC1FFF83FFF0FFFC1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A001E00007A000F00007C000F000 07C000F000078000F000078000F000078000F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F0000FC001 F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC00000F0780001C01C00070007000F 0007801E0003C01C0001C03C0001E03C0001E0780000F0780000F0780000F0F80000F8F8 0000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8780000F07C0001F03C 0001E03C0001E01E0003C01E0003C00F00078007800F0001C01C0000F07800001FC0001D 1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C03807C00007C00007C000078000 078000078000078000078000078000078000078000078000078000078000078000078000 0780000780000780000780000780000780000FC000FFFE00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E00010E00010F00010F800007E00 003FF0001FFF000FFFC003FFE0003FF00001F80000F880003C80003C80001CC0001CC000 1CE0001CE00018F00038F00030CC0060C301C080FE00161F7E9E1A>I<00400000400000 400000400000400000C00000C00000C00001C00001C00003C00007C0000FC0001FFFE0FF FFE003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C01003C01003C01003C01003C01003C01003C01003 C01001C02001E02000E0400078C0001F00142C7FAB19>I<078000F000FF801FF000FF80 1FF0000F8001F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078001F000 078001F000078001F000038002F00003C004F00001C008F800007030FF80001FC0FF8021 1F7E9E25>I E /Ft 5 85 df<00000000C00000000000E00000000001E00000000003E0 0000000003E00000000007E00000000007E0000000000FE0000000000FE0000000001FE0 000000001FE00000000037E00000000067E00000000067E000000000C7E000000000C7F0 0000000183F00000000183F00000000303F00000000703F00000000603F00000000C03F0 0000000C03F00000001803F00000001803F00000003003F00000003003F00000006003F0 000000C003F0000000C003F00000018003F00000018003F8000003FFFFF8000003FFFFF8 0000060001F800000E0001F800000C0001F80000180001F80000180001F80000300001F8 0000300001F80000600001F80000E00001F80000C00001F80001C00001F80001C00001F8 0007C00001FC001FC00003FC00FFF8007FFFE0FFF8007FFFE02B327BB135>65 D<000FFFFFFE0000000FFFFFFF800000007F000FE00000007E0003F00000007E0000F800 00007E0000FC0000007E00007C000000FC00003E000000FC00003E000000FC00003F0000 00FC00001F000001F800001F000001F800001F800001F800001F800001F800001F800003 F000001F800003F000001F800003F000001F800003F000001F800007E000003F800007E0 00003F800007E000003F800007E000003F80000FC000003F00000FC000007F00000FC000 007F00000FC000007F00001F8000007E00001F800000FE00001F800000FE00001F800000 FC00003F000001FC00003F000001F800003F000001F800003F000003F000007E000003E0 00007E000007E000007E00000FC000007E00000F800000FC00001F800000FC00003F0000 00FC00007E000000FC0000FC000001F80001F0000001F80003E0000001F8000FC0000003 F8007F000000FFFFFFFC000000FFFFFFE000000031317BB036>68 D<000FFFFFFFFC000FFFFFFFFC00007F0001FC00007E00007C00007E00003C00007E0000 3C00007E0000180000FC0000180000FC0000180000FC0000180000FC0000180001F80000 180001F80000180001F80000180001F80000180003F00080100003F00180000003F00180 000003F00180000007E00300000007E00300000007E00700000007E01F0000000FFFFE00 00000FFFFE0000000FC01E0000000FC00E0000001F800C0000001F800C0000001F800C00 00001F800C0000003F00180000003F00080000003F00000000003F00000000007E000000 00007E00000000007E00000000007E0000000000FC0000000000FC0000000000FC000000 0000FC0000000001F80000000001F80000000001F80000000003F800000000FFFFF00000 00FFFFF00000002E317BB02F>70 D<000FFFFFF000000FFFFFFE0000007F003F8000007E 000FC000007E0007E000007E0003F000007E0001F80000FC0001F80000FC0001F80000FC 0001F80000FC0001F80001F80003F80001F80003F80001F80003F80001F80003F00003F0 0007F00003F00007E00003F0000FC00003F0000FC00007E0001F000007E0007E000007E0 00FC000007E007F000000FFFFFC000000FFFFF0000000FC00F8000000FC003C000001F80 03E000001F8001F000001F8001F000001F8001F800003F0001F800003F0001F800003F00 01F800003F0001F800007E0003F800007E0003F800007E0003F000007E0003F00000FC00 07F00000FC0007F00000FC0007F00800FC0007F00C01F80007F01801F80007F01801F800 03F03003F80003F030FFFFE001F0E0FFFFE000FFC0000000003F002E327BB034>82 D<07FFFFFFFFF00FFFFFFFFFF00FC00FE003F01E000FC000F01C000FC000E018000FC000 E038000FC0006030001F8000E030001F8000E060001F8000C060001F8000C060003F0000 C0C0003F0000C0C0003F0000C0C0003F0000C080007E00008000007E00000000007E0000 0000007E0000000000FC0000000000FC0000000000FC0000000000FC0000000001F80000 000001F80000000001F80000000001F80000000003F00000000003F00000000003F00000 000003F00000000007E00000000007E00000000007E00000000007E0000000000FC00000 00000FC0000000000FC0000000000FC0000000001F80000000001F80000000001F800000 00001F80000000003F00000000003F00000000003F0000000000FF00000000FFFFFF0000 00FFFFFF0000002C3173B033>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 0 1 0 0 bop 795 947 a Ft(D)26 b(R)g(A)f(F)h(T)225 1038 y Fs(Do)r(cumen)n(t)20 b(for)i(a)f(Standard)g(Message-P)n(assing)f(In)n (terface)621 1232 y Fr(Message)c(P)o(assing)h(In)o(terface)e(F)l(orum) 766 1359 y(Septem)o(b)q(er)g(19,)h(1996)190 1417 y(This)h(w)o(ork)f(w)o (as)h(supp)q(orted)g(in)f(part)g(b)o(y)g(NSF)g(and)h(ARP)l(A)e(under)h (NSF)g(con)o(tract)283 1475 y(CD)o(A-9115428)j(and)e(Esprit)f(under)h (pro)s(ject)e(HPC)i(Standards)g(\(21111\).)p eop %%Page: 1 2 1 1 bop 166 45 a Fq(This)20 b(is)h(the)f(result)g(of)f(a)h(LaT)l(eX)g (run)g(of)g(a)f(draft)g(of)h(a)f(single)j(c)o(hapter)d(of)h(the)g(MPIF) f(Final)75 102 y(Rep)q(ort)d(do)q(cumen)o(t.)969 2828 y(i)p eop %%Page: 1 3 1 2 bop 75 356 a Fp(Chapter)34 b(3)75 564 y Fo(Miscellan)m(y)41 b(for)g(1.2)75 805 y Fn(3.1)59 b(V)n(ersion)20 b(Numb)r(er)75 953 y Fm(Curren)o(t)12 b(Status:)k Fl(P)o(assed)d(once.)20 b Fq(In)14 b(order)f(to)f(cop)q(e)i(with)g(c)o(hanges)f(to)f(the)i(MPI) f(Standard,)g(there)g(are)75 1057 y(b)q(oth)j(compile-time)h(and)f (run-time)g(w)o(a)o(ys)e(to)h(determine)h(whic)o(h)g(v)o(ersion)g(of)f (the)h(standard)e(is)i(in)h(use)75 1114 y(in)f(the)f(en)o(vironmen)o(t) h(one)f(is)h(using.)166 1170 y(The)e(\\v)o(ersion")g(will)h(b)q(e)g (represen)o(ted)f(b)o(y)g(t)o(w)o(o)e(separate)i(in)o(tegers,)g(for)f (the)h(v)o(ersion)g(and)g(sub)o(v)o(er-)75 1227 y(sion:)166 1283 y(In)i(C,)170 1367 y Fk(#define)23 b(MPI_VERSION)94 b(1)170 1423 y(#define)23 b(MPI_SUBVERSION)f(2)75 1507 y Fq(in)16 b(F)l(ortran,)170 1591 y Fk(INTEGER)23 b(MPI_VERSION,)g (MPI_SUBVERSION)170 1648 y(PARAMETER)g(\(MPI_VERSION)94 b(=)24 b(1\))170 1704 y(PARAMETER)f(\(MPI_SUBVERSION)f(=)i(2\))75 1788 y Fq(and)15 b(in)h(C++,)170 1872 y Fk(const)24 b(int)f (MPI::VERSION)f(=)i(1)170 1928 y(const)g(int)f(MPI::SUBVERSION)f(=)i(2) 75 2012 y Fq(F)l(or)15 b(run)o(time)g(determination,)75 2163 y Fj(MPI)p 160 2163 14 2 v 16 w(GET)p 264 2163 V 17 w(VERSION\()h(version,)f(subversion)h(\))117 2240 y Fl(OUT)108 b Fj(version)456 b Fl(v)o(ersion)14 b(n)o(um)o(b)q(er)117 2313 y(OUT)108 b Fj(subversion)393 b Fl(sub)o(v)o(ersion)15 b(n)o(um)o(b)q(er)75 2438 y Fk(int)23 b(MPI)p 245 2438 15 2 v 17 w(Get)p 334 2438 V 17 w(version\()g(int)g(*version,)g(int)g (*subversion)g(\))75 2524 y(MPI)p 150 2524 V 17 w(GET)p 239 2524 V 17 w(VERSION\()f(VERSION,)h(SUBVERSION,)f(IERROR)h(\))170 2581 y(INTEGER)g(VERSION,)g(SUBVERSION,)f(IERROR)1875 2617 y Fi(>)16 b Fh(\(Oct\))75 2667 y Fk(static)23 b(int)g(MPI::Get)p 532 2667 V 17 w(version\()f(int&)i(version,)e(int&)i(subversion)e(\)) 1875 2704 y Fi(?)16 b Fh(\(Oct\))964 2828 y Fq(1)p eop %%Page: 2 4 2 3 bop 75 -100 a Fq(2)970 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)75 45 y Fn(3.2)59 b(T)-5 b(reatment)18 b(of)i(MPI)p 670 45 18 2 v 21 w(Status)75 147 y Fq(The)25 b(follo)o(wing)g(prop)q(osals)g(add)g(to,)g(but)g(do)g(not)f(c)o (hange,)i(the)f(functionalit)o(y)h(asso)q(ciated)f(with)75 203 y Fj(MPI)p 160 203 14 2 v 16 w(ST)l(A)l(TUS)p Fq(.)75 325 y Ff(3.2.1)49 b(P)o(assing)17 b(MPI)p 481 325 15 2 v 18 w(ST)l(A)l(TUS)p 676 325 V 18 w(IGNORE)g(fo)o(r)f(MPI)p 1044 325 V 18 w(Status)75 458 y Fm(Curren)o(t)e(Status:)j Fl(P)o(assed)e(once.)166 561 y Fq(Ev)o(ery)10 b(call)i(to)e Fj(MPI)p 507 561 14 2 v 16 w(RECV)h Fq(includes)i(a)e Fj(status)h Fq(argumen)o(t,)f(where)g(the)f(system)h(can)f(return)h (details)75 618 y(ab)q(out)18 b(the)g(message)g(receiv)o(ed.)31 b(There)18 b(are)g(also)g(a)g(n)o(um)o(b)q(er)h(of)f(other)g(MPI)g (calls,)i(particularly)f(in)-1991 b Fi(>)15 b Fh(\(Oct\))75 674 y Fq(MPI-2,)20 b(where)f Fj(status)i Fq(is)f(returned.)33 b(An)19 b(ob)s(ject)g(of)g(t)o(yp)q(e)g Fj(MPI)p 1234 674 V 16 w(ST)l(A)l(TUS)h Fq(is)g(not)f(an)g Fj(MPI)g Fq(opaque)-117 676 y Fi(?)c Fh(\(Oct\))75 731 y Fq(ob)s(ject;)h(its)h (structure)g(is)g(declared)h(in)f Fk(mpi.h)f Fq(and)h Fk(mpif.h)p Fq(,)e(and)i(it)g(exists)g(in)g(the)g(users')f(program.)75 787 y(In)i(man)o(y)e(cases)h(application)i(programs)d(are)g (constructed)i(so)e(that)h(it)g(is)g(unnecessary)h(for)f(them)g(to)75 844 y(examine)j(the)g Fk(status)f Fq(\014elds.)35 b(In)20 b(these)g(cases)f(it)h(is)g(a)g(w)o(aste)e(for)h(the)h(user)g(to)f (allo)q(cate)h(a)g(status)75 900 y(ob)s(ject,)f(and)h(it)f(is)h (particularly)g(w)o(asteful)f(for)g(the)g Fj(MPI)f Fq(implemen)o (tation)j(to)d(\014ll)j(in)f(\014elds)h(in)f(this)75 957 y(ob)s(ject.)166 1013 y(T)l(o)h(cop)q(e)h(with)g(this)g(problem,)i (there)d(is)h(a)g(pre-de\014ned)h(constan)o(t,)f Fj(MPI)p 1502 1013 V 16 w(ST)l(A)l(TUS)p 1683 1013 V 17 w(IGNORE)p Fq(,)75 1070 y(whic)o(h,)i(when)e(passed)g(to)f(a)h(receiv)o(e)h(or)e (test)g(function,)j(informs)e(the)g(implemen)o(tation)h(that)e(the)75 1126 y(status)13 b(\014elds)h(are)f(not)g(to)g(b)q(e)h(\014lled)i(in.)k (Note)13 b(that)g Fj(MPI)p 1059 1126 V 15 w(ST)l(A)l(TUS)p 1239 1126 V 18 w(IGNORE)i Fq(is)e(not)h(a)f(sp)q(ecial)i(t)o(yp)q(e)e (of)75 1183 y Fj(MPI)p 160 1183 V 16 w(ST)l(A)l(TUS)i Fq(ob)s(ject;)f(rather,)f(it)h(is)h(a)f(sp)q(ecial)i(v)m(alue)f(for)f (the)g(argumen)o(t.)k(That)c(is,)h(in)f(C)g(one)h(w)o(ould)75 1239 y(exp)q(ect)h(it)f(to)g(b)q(e)h(NULL,)g(not)e(the)i(address)f(of)g (a)g(sp)q(ecial)i Fj(MPI)p 1179 1239 V 15 w(ST)l(A)l(TUS)p Fq(.)166 1295 y(In)12 b(general,)h(this)f(optimization)g(can)g(apply)g (to)f(all)i(functions)f(for)f(whic)o(h)h Fj(MPI)p 1514 1295 V 16 w(Status)i Fq(or)d(an)g(arra)o(y)75 1352 y(of)k Fj(MPI)p 212 1352 V 16 w(Status)p Fq('s)i(is)f(an)g(argumen)o(t.)k (These)c(are)f(all)i(the)e(v)m(arious)h(forms)f(of)g Fj(MPI)p 1493 1352 V 16 w(RECV)p Fq(,)h Fj(MPI)p 1735 1352 V 16 w(TEST)p Fq(,)75 1408 y(and)i Fj(MPI)p 251 1408 V 15 w(W)l(AIT)p Fq(.)f(When)h(an)f(arra)o(y)g(is)g(passed,)h(as)f (in)h(the)p 1132 1408 V 34 w Fj(ANY)g Fq(and)p 1349 1408 V 34 w Fj(ALL)f Fq(functions,)h(a)f(separate)75 1465 y(constan)o(t,)d Fj(MPI)p 356 1465 V 16 w(ST)l(A)l(TUSES)p 589 1465 V 18 w(IGNORE)p Fq(,)i(is)f(passed)h(for)e(the)i(arra)o(y)e (argumen)o(t.)166 1521 y Fj(MPI)p 251 1521 V 16 w(ST)l(A)l(TUS)p 432 1521 V 17 w(IGNORE)g Fq(and)e Fj(MPI)p 794 1521 V 16 w(ST)l(A)l(TUSES)p 1027 1521 V 18 w(IGNORE)i Fq(are)e(not)g (required)h(to)f(ha)o(v)o(e)g(the)g(same)-1992 b Fi(>)15 b Fh(\(Oct\))75 1578 y Fq(v)m(alues)h(in)g(C)f(and)h(F)l(ortran.)166 1634 y(It)d(is)g(not)g(allo)o(w)o(ed)g(to)f(ha)o(v)o(e)g(some)h(of)f (the)h(statuses)f(in)i(an)f(arra)o(y)e(of)i(statuses)f(for)p 1574 1634 V 28 w Fj(ANY)i Fq(and)p 1782 1634 V 29 w Fj(ALL)75 1691 y Fq(functions)j(set)g(to)f Fj(MPI)p 487 1691 V 16 w(ST)l(A)l(TUS)p 668 1691 V 17 w(IGNORE)p Fq(;)i(one)e(either)i(sp)q (eci\014es)g(ignoring)f Fe(al)r(l)34 b Fq(of)17 b(the)f(statuses)g(in) 75 1747 y(suc)o(h)e(a)g(call)h(with)f Fj(MPI)p 482 1747 V 16 w(ST)l(A)l(TUSES)p 715 1747 V 18 w(IGNORE)p Fq(,)h(or)f Fe(none)i Fq(of)e(themm)g(b)o(y)f(passing)i(normal)f(statuses)f(in)75 1804 y(alll)k(p)q(ositions)f(in)g(the)f(arra)o(y)f(of)h(statuses.)-932 b Fi(?)15 b Fh(\(Oct\))75 1925 y Ff(3.2.2)49 b(Non-destructive)16 b(T)l(est)f(of)i(MPI)p 808 1925 15 2 v 18 w(Status)75 2058 y Fm(Curren)o(t)d(Status:)j Fl(P)o(assed)e(once.)166 2162 y Fq(This)22 b(call)g(is)g(useful)g(for)f(accessing)h(the)f (information)h(asso)q(ciated)f(with)h(a)f(request,)h(without)75 2218 y(deleting)c(the)f(request)f(\(in)h(case)g(the)g(user)f(is)i(exp)q (ected)f(to)f(access)h(it)g(after)f(the)g(handler\).)25 b(It)17 b(allo)o(ws)75 2275 y(one)11 b(to)e(la)o(y)o(er)i(libraries)h (more)e(con)o(v)o(enien)o(tly)l(,)i(since)g(m)o(ultiple)g(la)o(y)o(ers) e(of)h(soft)o(w)o(are)d(ma)o(y)i(access)h(the)f(same)75 2331 y(completed)17 b(request)f(and)g(extract)f(from)g(it)h(the)g (status)f(information.)21 b(This)c(will)g(also)f(b)q(e)g(imp)q(ortan)o (t)75 2388 y(for)g(language)i(in)o(terop)q(erabilit)o(y)h(if)e(w)o(e)g (decide)i(that)d(status)g(ob)s(jects)h(cannot)f(b)q(e)i(transferred)f (across)75 2444 y(language)d(b)q(oundaries,)g(since)h(with)f(this)g (function)g(the)g(request)f(ob)s(ject)g(can)h(b)q(e)g(transferred)f (instead.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 3 5 3 4 bop 75 -100 a Fg(3.3.)34 b(ERR)o(OR)17 b(CLASS)f(F)o(OR)f(INV)-5 b(ALID)16 b(KEYV)-5 b(AL)818 b Fq(3)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(GET)p 264 45 V 17 w(ST)l(A)l(TUS\()16 b(request,)g(\015ag,)f(status)i(\))117 122 y Fl(IN)155 b Fj(request)452 b Fl(an)14 b Fd(MPI)p 1040 122 13 2 v 14 w(REQUEST)f Fl(ob)r(ject)117 197 y(OUT)108 b Fj(\015ag)518 b Fl(\015ag,)13 b(same)g(as)h(from)e Fd(MPI)p 1325 197 V 15 w(TEST)117 273 y Fl(OUT)108 b Fj(status)476 b Fd(MPI)p 982 273 V 15 w(ST)m(A)m(TUS)12 b Fl(ob)r(ject)j(if)e(\015ag)g(is)h (true)75 397 y Fk(int)23 b(MPI)p 245 397 15 2 v 17 w(Get)p 334 397 V 17 w(status\(MPI)p 591 397 V 16 w(Request)g(request,)f(int)i (*flag,)f(MPI)p 1347 397 V 17 w(Status)g(*status\))75 483 y(MPI)p 150 483 V 17 w(GET)p 239 483 V 17 w(STATUS\()f(REQUEST,)h (FLAG,)g(STATUS,)g(IERROR\))170 540 y(INTEGER)g(REQUEST,)g(FLAG,)g (STATUS\(MPI)p 962 540 V 16 w(STATUS)p 1122 540 V 16 w(SIZE\),)h(IERROR)75 626 y(int)f(MPI::Request::Get)p 581 626 V 15 w(status\(bool&)g(flag,)g(MPI::Status&)f(status\))h(const) 166 713 y Fq(Sets)18 b(\015ag=true)g(if)h(the)f(request)h(has)f (completed,)h(and,)g(if)g(so,)f(returns)g(in)h(status)f(the)g(request) 75 769 y(status.)34 b(Ho)o(w)o(ev)o(er,)20 b(unlik)o(e)i(test)d(or)h(w) o(ait,)g(it)h(do)q(es)f(not)g(deallo)q(cate)h(or)f(inactiv)m(ate)h(the) f(request;)i(a)75 826 y(subsequen)o(t)16 b(call)g(to)f(test,)f(w)o(ait) h(or)f(free)i(should)g(b)q(e)g(executed)g(with)f(that)g(request)75 969 y Fn(3.3)59 b(Erro)n(r)21 b(Class)f(fo)n(r)h(Invalid)e(Keyval)1875 1026 y Fi(>)d Fh(\(Oct\))75 1118 y Fm(Curren)o(t)e(Status:)j Fl(P)o(assed)e(once.)166 1221 y Fq(There)d(is)f(a)h(new)f(MPI)h(error)e (class:)19 b Fj(MPI)p 875 1221 14 2 v 15 w(ERR)p 975 1221 V 18 w(KEYV)l(AL)p Fq(.)11 b(It)h(can)f(b)q(e)h(returned)g(b)o(y)f Fj(MPI)p 1692 1221 V 16 w(A)o(ttr)p 1785 1221 V 17 w(put)p Fq(,)75 1278 y Fj(MPI)p 160 1278 V 16 w(A)o(ttr)p 253 1278 V 17 w(get)p Fq(,)16 b Fj(MPI)p 440 1278 V 16 w(A)o(ttr)p 533 1278 V 16 w(delete)p Fq(,)i Fj(MPI)p 772 1278 V 16 w(Keyval)p 915 1278 V 15 w(free)p Fq(,)e Fj(MPI)p 1111 1278 V 16 w(Comm)p 1253 1278 V 14 w(dup)p Fq(,)i(and)e Fj(MPI)p 1538 1278 V 16 w(Comm)p 1680 1278 V 14 w(free)p Fq(.)23 b(The)75 1334 y(last)15 b(t)o(w)o(o)f(are)h(b)q(ecause)h Fj(k)o(eyval)f Fq(is)h(an)f(argumen)o(t)f(to)h(the)g(cop)o(y)g(and)h (delete)g(functions)g(for)e(attributes.)8 b Fi(?)16 b Fh(\(Oct\))1875 1393 y Fi(>)g Fh(\(Oct\))1875 1451 y Fi(>)g Fh(\(Oct\))75 1478 y Fn(A)k(new)f(reduce)g(op)r(eration)75 1626 y Fm(Curren)o(t)14 b(Status:)j Fl(New)d(for)g(Octob)q(er.)166 1730 y Fq(A)e(new)h(prede\014ned)h(op)q(eration,)f Fd(MPI)p 811 1730 13 2 v 14 w(REPLA)o(CE)p Fq(,)d(is)j(de\014ned.)20 b(It)13 b(corresp)q(onds)f(to)g(the)h(asso)q(ciativ)o(e)75 1786 y(function)18 b Fc(f)5 b Fq(\()p Fc(a;)j(b)p Fq(\))15 b(=)h Fc(b)p Fq(;)i(i.e.,)f(the)g(curren)o(t)h(v)m(alue)g(in)g(the)g (target)e(memory)h(is)h(replaced)g(b)o(y)f(the)h(v)m(alue)75 1843 y(supplied)g(b)o(y)d(the)g(origin.)21 b(This)15 b(op)q(eration)h(can)f(b)q(e)h(applied)h(on)e(an)o(y)g(basic)h(datat)o (yp)q(e.)166 1947 y Fm(Discussion:)35 b Fl(This)15 b(needs)g(a)f (rationale.)19 b(It)14 b(can)h(b)q(e)g(used)g(to)f(do)g(one-sided)h (broadcasts)g(\(should)f(it)g(b)q(e)75 2003 y(f\(a,b\))i(=)h(a\)?)27 b(It)17 b(w)o(ould)f(b)q(e)i(wierd)f(in)f(a)h(reduce)i(op)q(eration,)e (and)f(could)h(b)q(e)h(easily)e(done)h(b)o(y)g(the)g(user)h(as)f(a)75 2059 y(user-de\014ned)f(function.)1875 2107 y Fi(?)g Fh(\(Oct\))75 2250 y Fn(3.4)59 b(A)20 b(F)n(o)n(rtran)h(Problem)e(with) h(Register)e(Optimization)75 2399 y Fm(Curren)o(t)c(Status:)j Fl(A)d(new)g(discussion.)189 2552 y Fe(A)n(dvic)n(e)h(to)i(users.)189 2627 y Fq(MPI)d(pro)o(vides)g(op)q(erations)g(whic)o(h)h(ma)o(y)e(b)q (e)i(hidden)h(from)d(the)h(user)g(co)q(de)h(and)f(run)h(in)f(parallel) 189 2684 y(with)g(it,)h(accessing)g(the)f(same)g(memory)g(as)g(user)g (co)q(de.)20 b(Examples)15 b(include)i(the)d(data)g(transfer)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 4 6 4 5 bop 75 -100 a Fq(4)970 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fq(for)10 b(an)h Fj(MPI)p 398 45 14 2 v 16 w(RECV)h Fq(The)f(optimizer)i(of)d(a)h(compiler)i(will)f (assume)f(that)g(it)g(can)h(recognize)g(p)q(erio)q(ds)189 102 y(when)j(a)g(cop)o(y)g(of)g(a)f(v)m(ariable)j(can)e(b)q(e)h(k)o (ept)f(in)h(a)e(register)h(without)g(reloading)h(from)f(or)f(storing) 189 158 y(to)e(memory)l(.)18 b(When)13 b(the)g(user)g(co)q(de)g(is)g(w) o(orking)f(with)h(a)f(register)h(cop)o(y)f(of)g(some)h(v)m(ariable)g (while)189 214 y(the)i(hidden)i(op)q(eration)f(reads)f(or)g(writes)g (the)h(memory)f(cop)o(y)l(,)g(problems)h(o)q(ccur.)k(This)c(section)189 271 y(discusses)g(register)f(optimization)h(pitfalls.)189 346 y(When)f(a)f(v)m(ariable)i(is)f(lo)q(cal)h(to)e(a)g(F)l(ortran)f (subroutine)j(\(i.e.)k(not)14 b(in)h(a)g(COMMON)f(blo)q(c)o(k\),)h(the) 189 403 y(compiler)g(will)i(assume)d(that)g(it)h(cannot)f(b)q(e)h(mo)q (di\014ed)h(b)o(y)f(a)f(called)i(subroutine)f(unless)h(it)f(is)g(an)189 459 y(actual)g(argumen)o(t)g(of)h(the)f(call.)23 b(In)16 b(the)g(most)e(common)i(link)m(age)h(con)o(v)o(en)o(tion,)e(the)h (subroutine)189 515 y(is)d(exp)q(ected)i(to)d(sa)o(v)o(e)h(and)g (restore)g(certain)h(registers.)19 b(Th)o(us,)13 b(the)g(optimizer)i (will)g(assume)e(that)189 572 y(a)k(register)h(whic)o(h)h(held)g(a)e(v) m(alid)j(cop)o(y)d(of)h(suc)o(h)g(a)g(v)m(ariable)h(b)q(efore)f(the)g (call)h(will)g(still)h(hold)e(a)189 628 y(v)m(alid)e(cop)o(y)g(on)f (return.)189 703 y(Normally)22 b(users)g(are)g(not)g(a\017icted)g(with) h(this.)41 b(But)22 b(the)g(user)g(should)h(pa)o(y)f(atten)o(tion)g(to) 189 760 y(this)f(section)g(if)g(in)h(his/her)f(program)e(a)i(bu\013er)f (argumen)o(t)g(to)g(an)h Fj(MPI)p 1498 760 V 16 w(SEND)p Fq(,)f Fj(MPI)p 1746 760 V 16 w(RECV)189 816 y Fq(etc.)25 b(uses)18 b(a)f(name)g(whic)o(h)h(hides)g(the)f(actual)h(v)m(ariables)g (in)o(v)o(olv)o(ed.)27 b Fj(MPI)p 1493 816 V 16 w(BOTTOM)18 b Fq(with)f(an)189 873 y(MPI)p 281 873 V 16 w(Datat)o(yp)q(e)i(con)o (taining)i(absolute)g(addresses)f(is)h(one)f(example.)36 b(Creating)20 b(a)f(datat)o(yp)q(e)189 929 y(whic)o(h)14 b(uses)f(one)g(v)m(ariable)i(as)e(an)g(anc)o(hor)f(and)i(brings)f (along)h(others)e(b)o(y)h(using)h Fj(MPI)p 1659 929 V 16 w(ADDRESS)189 986 y Fq(to)h(determine)j(their)f(o\013sets)e(from)g (the)i(anc)o(hor)f(is)h(another.)22 b(The)17 b(anc)o(hor)f(v)m(ariable) i(w)o(ould)e(b)q(e)189 1042 y(the)g(only)h(one)f(men)o(tioned)h(in)g (the)g(call.)24 b(Also)16 b(atten)o(tion)g(m)o(ust)g(b)q(e)h(pa)o(y)o (ed)f(if)h(MPI)f(op)q(erations)189 1099 y(are)f(used)g(that)g(run)g(in) h(parallel)h(with)f(the)f(user's)g(application.)189 1174 y(The)g(follo)o(wing)h(example)g(sho)o(ws,)e(what)h(F)l(ortran)f (compilers)i(are)f(allo)o(w)o(ed)h(to)e(do:)224 1299 y(This)i(source)f(...)663 b(can)15 b(b)q(e)h(compiled)h(as:)224 1363 y(call)f(MPI)p 399 1363 V 17 w(ADDRESS\(buf,bufaddr,ierr\))189 b(call)16 b(MPI)p 1344 1363 V 17 w(ADDRESS\(buf,...\))224 1420 y(call)g(MPI)p 399 1420 V 17 w(TYPE)p 545 1420 V 16 w(STR)o(UCT\(1,1,bufaddr,)163 b(call)16 b(MPI)p 1344 1420 V 17 w(TYPE)p 1490 1420 V 16 w(STR)o(UCT\(...\))590 1476 y(MPI)p 682 1476 V 17 w(REAL,t)o(yp,ierr\))224 1533 y(call)g(MPI)p 399 1533 V 17 w(TYPE)p 545 1533 V 16 w(COMMIT\(t)o(yp\)) 308 b(call)16 b(MPI)p 1344 1533 V 17 w(TYPE)p 1490 1533 V 16 w(COMMIT\(...\))224 1589 y(v)m(al)p 283 1589 V 17 w(old)g(=)g(buf)681 b(register)15 b(=)h(buf)1169 1646 y(v)m(al)p 1228 1646 V 17 w(old)g(=)f(register)224 1702 y(call)h(MPI)p 399 1702 V 17 w(RECV\(MPI)p 654 1702 V 16 w(BOTTOM,1,t)o(yp,...\))107 b(call)16 b(MPI)p 1344 1702 V 17 w(RECV\(MPI)p 1599 1702 V 16 w(BOTTOM,...\))224 1759 y(v)m(al)p 283 1759 V 17 w(new)g(=)f(buf)664 b(v)m(al)p 1228 1759 V 17 w(new)16 b(=)f(register)189 1884 y(The)j(compiler)g(do)q (es)g(not)g(in)o(v)m(alidate)h(the)f(register)g(b)q(ecause)g(it)g (cannot)f(see)h(that)f Fj(MPI)p 1746 1884 V 16 w(RECV)189 1940 y Fq(c)o(hanges)h(the)g(v)m(alue)i(of)e Fj(buf)p Fq(.)30 b(The)19 b(access)f(of)g Fj(buf)i Fq(is)f(hidden)h(b)o(y)e(the) h(use)f(of)g Fj(MPI)p 1659 1940 V 16 w(ADDRESS)189 1997 y Fq(and)d Fj(MPI)p 362 1997 V 16 w(BOTTOM)p Fq(.)189 2072 y(The)g(next)g(example)h(sho)o(ws)f(extreme,)g(but)g(allo)o(w)o (ed)h(p)q(ossibilities!)224 2197 y(Source)424 b(compiled)17 b(as)322 b(or)15 b(compiled)i(as)224 2261 y(call)f(MPI)p 399 2261 V 17 w(IRECV\(buf,..req\))i(call)f(MPI)p 955 2261 V 16 w(IRECV\(buf,..req\))h(call)f(MPI)p 1510 2261 V 16 w(IRECV\(buf,..req\))779 2318 y(register)e(=)h(buf)278 b(b1)15 b(=)h(buf)224 2374 y(call)g(MPI)p 399 2374 V 17 w(W)-5 b(AIT\(req,..\))104 b(call)17 b(MPI)p 955 2374 V 16 w(W)-5 b(AIT\(req,..\))104 b(call)17 b(MPI)p 1510 2374 V 16 w(W)-5 b(AIT\(req,..\))224 2431 y(b1)15 b(=)h(buf)377 b(b1)15 b(:=)g(register)189 2556 y Fj(MPI)p 274 2556 V 15 w(W)l(AIT)e Fq(or)f(a)g(parallel)i(thread)e(mo)q(di\014es)i Fj(buf)f Fq(b)q(et)o(w)o(een)g(the)f(in)o(v)o(o)q(cation)h(of)f Fj(MPI)p 1648 2556 V 16 w(IRECV)h Fq(and)189 2612 y(the)j(\014nish)h (of)f Fj(MPI)p 528 2612 V 16 w(W)l(AIT)p Fq(.)g(But)g(the)g(compiler)i (cannot)e(see)g(an)o(y)g(p)q(ossibilit)o(y)i(that)e Fj(buf)h Fq(can)f(b)q(e)189 2669 y(c)o(hanged)g(after)e Fj(MPI)p 557 2669 V 16 w(IRECV)i Fq(has)g(\014nished.)23 b(And)16 b(therefore)f(it)h(can)g(sc)o(hedule)h(the)f(load)g(of)f Fj(buf)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 5 7 5 6 bop 75 -100 a Fg(3.5.)34 b(TR)o(UE)15 b(EXTENT)g(OF)g(D)o(A)l(T)l (A)l(TYPES)958 b Fq(5)189 45 y(earlier)14 b(than)g(t)o(yp)q(ed)g(in)h (the)f(source.)20 b(It)13 b(has)h(no)g(reason)f(to)h(a)o(v)o(oid)f (using)i(a)e(register)h(to)f(hold)i Fj(buf)189 102 y Fq(across)f(the)h(call)i(to)d Fj(MPI)p 625 102 14 2 v 16 w(W)l(AIT)p Fq(.)h(It)g(also)g(can)h(reorder)f(the)g(instructions)h (as)f(in)h(the)f(left)h(case.)189 177 y(T)l(o)c(prev)o(en)o(t)h (instruction)h(reordering)g(or)e(the)h(allo)q(cation)h(of)f(a)g(v)m (ariable)h(or)f(bu\013er)g(in)h(a)e(register,)189 233 y(there)j(are)g(three)g(p)q(ossibilities)k(in)d(p)q(ortable)f(F)l (ortran)f(co)q(de:)243 327 y Fi(\017)23 b Fq(One)d(can)f(put)h(the)g(v) m(ariables)g(and)g(bu\013ers)f(in)o(to)h(common)f(blo)q(c)o(ks;)j(then) e(they)f(will)i(b)q(e)289 383 y(allo)q(cated)d(in)g(the)g(memory)e (while)j(an)o(y)e(library)h(subroutine)h(\(e.g.)25 b(an)18 b(MPI)f(routine\))g(is)289 440 y(called)c(pro)o(vided)f(that)f(the)h (compiler)h(cannot)e(analyze)h(that)f(the)h(library)g(subroutine)h(do)q (es)289 496 y(not)h(use)i(these)f(common)g(blo)q(c)o(ks,)h(or)243 569 y Fi(\017)23 b Fq(one)11 b(can)h(declare)g(them)f(as)g(V)o(OLA)l (TILE,)h(but)g(this)g(is)f(not)g(part)g(of)g(the)g(standards)g(F)l (ortran)289 626 y(77)f(and)h(90,)g(and)g(y)o(our)f(co)q(de)i(will)h (not)d(b)q(e)i(p)q(ortable)f(b)q(ecause)h(there)f(are)g(compilers)h (without)289 682 y(V)o(OLA)l(TILE,)k(and)f(normally)h(it)g(prev)o(en)o (ts)e(an)o(y)h(optimization,)h(or)243 755 y Fi(\017)23 b Fq(one)14 b(can)g(call)h(a)f(dumm)o(y)g(routine)g(with)g(the)h(v)m (ariable)g(or)e(bu\013er)h(as)g(an)g(actual)g(argumen)o(t;)289 812 y(if)22 b(the)h(dumm)o(y)f(routine)h(is)f(not)g(part)g(of)g(the)g (application)i(then)f(the)f(compiler)i(m)o(ust)289 868 y(allo)q(cate)16 b(the)f(v)m(ariable)i(or)e(bu\013er)h(in)g(the)f (memory)g(while)i(that)e(subroutine)h(is)g(executed;)289 925 y Fj(MPI)p 374 925 V 15 w(ADDRESS)h Fq(can)e(b)q(e)h(used)g(as)e (dumm)o(y)h(routine.)189 1019 y(High)e(qualit)o(y)g(F)l(ortran)f (implemen)o(tations)i(ma)o(y)e(pro)o(vide)i(mec)o(hanisms)f(to)f(a)o(v) o(oid)h(the)g(problem,)189 1075 y(without)18 b(disabling)j (optimizations.)31 b(Ho)o(w)o(ev)o(er,)18 b(co)q(de)h(that)f(do)q(es)h (not)f(use)h(one)g(of)f(the)h(three)189 1131 y(solutions)d(describ)q (ed)h(ab)q(o)o(v)o(e)d(ma)o(y)h(not)g(b)q(e)h(p)q(ortable.)189 1207 y(In)h(C,)f(subroutines)i(whic)o(h)g(mo)q(dify)f(v)m(ariables)h (that)e(are)h(not)f(in)i(the)f(parameter)f(list)i(will)g(not)189 1263 y(cause)e(register)h(optimization)g(problems.)25 b(This)17 b(is)g(b)q(ecause)g(taking)g(p)q(oin)o(ters)g(to)f(storage)f (ob-)189 1319 y(jects)d(b)o(y)g(using)h(the)g(&)f(op)q(erator)g(and)g (later)h(referencing)g(the)g(ob)s(jects)f(b)o(y)g(w)o(a)o(y)f(of)h(the) h(p)q(oin)o(ter)f(is)189 1376 y(an)j(in)o(tegral)g(part)g(of)f(the)i (language.)k(A)15 b(C)g(compiler)h(understands)g(the)f(implications,)i (so)e(that)189 1432 y(the)g(problem)h(should)h(not)e(o)q(ccur,)h(in)g (general.)21 b(Ho)o(w)o(ev)o(er,)14 b(some)i(compilers)g(do)g(o\013er)e (optional)189 1489 y(aggressiv)o(e)g(optimization)j(lev)o(els)f(whic)o (h)g(ma)o(y)f(not)f(b)q(e)i(safe.)189 1564 y(\()p Fe(End)f(of)i(advic)n (e)f(to)g(users.)p Fq(\))p 75 1670 827 2 v 75 1727 a(The)f(follo)o (wing)h(advice)h(m)o(ust)d(b)q(e)i(added)g(in)75 1795 y(-)f(MPI)g(2,)g(c)o(hapter)g(?.?)21 b(Handlers,)75 1851 y(-)15 b Fj(MPI)p 190 1851 14 2 v 16 w(ADDRESS)h Fq(\(MPI)f(1.1,)f (page)h(70,)f(line)j(16\),)d(and)h(in)75 1908 y(-)g Fj(MPI)p 190 1908 V 16 w(IRECV)h Fq(\(MPI)f(1.1,)e(\\Non)o(blo)q(c)o(king)k (comm)o(unication",)e(end)h(of)e(page)h(40\))189 2014 y Fe(A)n(dvic)n(e)i(to)i(users.)53 b Fq(T)l(o)17 b(prev)o(en)o(t)g (troubles)h(with)g(the)f(register)g(optimization)i(of)e(the)g(F)l (ortran)189 2071 y(compilers)i(please)g(pa)o(y)e(atten)o(tion)h(to)f (the)h(hin)o(ts)g(in)h(section)g(3.4)e(\\A)g(F)l(ortran)g(Problem)i (with)189 2127 y(register)c(optimization")h(on)f(page)g(3.)k(\()p Fe(End)d(of)g(advic)n(e)g(to)h(users.)p Fq(\))1875 2188 y Fi(?)f Fh(\(Oct\))75 2320 y Fn(3.5)59 b(T)-5 b(rue)20 b(Extent)d(of)k(Datat)n(yp)r(es)75 2469 y Fm(Curren)o(t)14 b(Status:)j Fl(P)o(assed)e(once.)166 2573 y Fq(Supp)q(ose)23 b(w)o(e)f(implemen)o(t)h(gather)f(as)g(a)f(spanning)i(tree)f(implemen)o (ted)i(on)e(top)g(of)g(p)q(oin)o(t-to-)75 2629 y(p)q(oin)o(t)e (routines.)34 b(Since)22 b(the)d(recv)h(bu\013er)g(is)g(only)h(v)m (alid)g(on)f(the)g(ro)q(ot)f(pro)q(cess,)i(one)f(will)h(need)g(to)75 2685 y(allo)q(cate)f(some)f(temp)q(orary)g(space)h(for)f(receiving)i (data)e(on)g(in)o(termediate)i(no)q(des.)33 b(The)20 b(di\016cultly)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 6 8 6 7 bop 75 -100 a Fq(6)970 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)75 45 y Fq(is)20 b(in)g(determining)h(the)f(size)g(one)f (needs)i(to)d(allo)q(ciate.)34 b(This)20 b(o)q(ccurs)g(since)h(the)e (user)h(can)f(mo)q(dify)75 102 y(the)g(exten)o(t)h(using)g(the)f Fd(MPI)p 583 102 13 2 v 14 w(UB)g Fq(and)h Fd(MPI)p 841 102 V 14 w(LB)f Fq(v)m(alues.)34 b(The)20 b(writer)f(of)g(the)g(gather) g(routine)h(could)75 158 y(determine)14 b(this)f(information)g(b)o(y)g (deco)q(ding)h(the)f(datat)o(yp)q(e.)18 b(Ho)o(w)o(ev)o(er,)12 b(this)h(is)g(more)g(w)o(ork)e(and)i(more)75 214 y(painful)j(than)e (desired.)20 b(Th)o(us,)14 b(a)g(new)g(function)h(is)g(pro)o(vided)g (whic)o(h)f(returns)g(the)h(true)e(exten)o(t)h(of)g(the)75 271 y(datat)o(yp)q(e.)75 422 y Fj(MPI)p 160 422 14 2 v 16 w(TRUE)p 294 422 V 17 w(EXTENT\(datat)o(yp)q(e,)j(true)p 775 422 V 17 w(lb,)e(true)p 929 422 V 17 w(extent\))117 499 y Fl(IN)155 b Fj(datat)o(yp)q(e)424 b Fl(datat)o(yp)q(e)14 b(to)g(get)g(information)d(on)117 574 y(OUT)108 b Fj(true)p 396 574 V 17 w(lb)461 b Fl(true)15 b(lo)o(w)o(er)e(b)q(ound)h(of)g (datat)o(yp)q(e)117 649 y(OUT)108 b Fj(true)p 396 649 V 17 w(extent)379 b Fl(true)15 b(size)g(of)e(datat)o(yp)q(e)75 774 y Fk(int)23 b(MPI)p 245 774 15 2 v 17 w(True)p 358 774 V 17 w(extent\(MPI)p 615 774 V 16 w(Datatype)f(datatype,)h(MPI)p 1156 774 V 17 w(Aint)g(*true)p 1412 774 V 17 w(lb,)g(MPI)p 1596 774 V 17 w(Aint)393 830 y(*true)p 516 830 V 17 w(extent\))75 917 y(MPI)p 150 917 V 17 w(TRUE)p 263 917 V 16 w(EXTENT\(DATATYPE,)f (TRUE)p 781 917 V 17 w(LB,)h(TRUE)p 989 917 V 17 w(EXTENT,)g(IERROR\)) 170 973 y(INTEGER)g(DATATYPE,)g(TRUE)p 699 973 V 17 w(LB,)g(TRUE)p 907 973 V 17 w(EXTENT,)g(IERROR)166 1060 y Fj(true)p 244 1060 14 2 v 17 w(lb)14 b Fq(returns)f(the)g(o\013set)g(of)g(the)g (lo)o(w)o(est)g(unit)h(of)f(store)g(whic)o(h)h(is)g(addressed)g(b)o(y)f (the)h(datat)o(yp)q(e.)75 1116 y Fj(true)p 153 1116 V 17 w(extent)k Fq(returns)f(the)g(true)f(size)i(of)e(the)h(datat)o(yp)q (e.)24 b(The)17 b Fj(true)p 1244 1116 V 17 w(extent)i Fq(is)e(the)g(minim)o(um)h(n)o(um)o(b)q(er)75 1173 y(of)13 b(b)o(ytes)g(of)h(memory)f(necessary)g(to)g(hold)i(a)e(datat)o(yp)q(e)g (and)h(ignores)g(the)f Fj(LB)h Fq(and)g Fj(UB)g Fq(that)e(ma)o(y)h(ha)o (v)o(e)75 1229 y(b)q(een)j(used)g(in)g(creating)g(the)f(datat)o(yp)q (e.)166 1326 y Fm(Alternativ)o(es)o(:)166 1376 y Fl(If)e(w)o(e)i(w)o (an)o(t)e(to)h(b)q(e)g(consisten)o(t)h(with)f(the)g(MPI1)g(exten)o(t)h (functions,)f(w)o(e)g(w)o(ould)f(need)i(three)g(functions:)166 1426 y Fd(MPI)p 243 1426 13 2 v 14 w(TRUE)p 366 1426 V 15 w(LB\(datat)o(yp)q(e,)g(lb\))166 1475 y(MPI)p 243 1475 V 14 w(TRUE)p 366 1475 V 15 w(UB\(datat)o(yp)q(e,)f(ub\))166 1525 y(MPI)p 243 1525 V 14 w(TRUE)p 366 1525 V 15 w(EXTENT\(datat)o(yp) q(e,)h(extent\))166 1575 y Fl(On)f(the)h(other)f(hand,)f(w)o(e)i(ma)o (y)c(not)j(w)o(an)o(t)g(to)f(mak)o(e)g(the)h(same)f(mistak)o(e)g(t)o (wice...)75 1766 y Fn(3.6)59 b(Datat)n(yp)r(e)18 b(extent)75 1914 y Fm(Curren)o(t)c(Status:)j Fl(P)o(assed)e(once.)166 2018 y Fq(MPI)21 b(allo)o(ws)h(one)f(to)g(c)o(hange)h(the)f(exten)o(t)g (of)g(a)h(datat)o(yp)q(e,)g(using)g(lo)o(w)o(er)f(b)q(ound)h(and)g(upp) q(er)75 2074 y(b)q(ound)e(mark)o(ers)e(\()p Fd(MPI)p 490 2074 V 14 w(LB)h Fq(and)g Fd(MPI)p 740 2074 V 14 w(UB)p Fq(\).)f(This)h(is)h(useful,)g(as)f(it)g(allo)o(ws)g(to)f(con)o (trol)h(the)g(stride)g(of)75 2131 y(successiv)o(e)e(datat)o(yp)q(es)e (that)g(are)g(replicated)i(b)o(y)e(datat)o(yp)q(e)g(constructors,)g(or) g(are)g(replicated)i(b)o(y)f(the)75 2187 y Fj(count)k Fq(argumen)o(t)e(in)i(a)e(send)h(or)g(reciev)o(e)g(call.)32 b(Ho)o(w)o(ev)o(er,)19 b(the)f(curren)o(t)h(mec)o(hanism)h(for)e(ac)o (hieving)75 2244 y(it)h(is)h(painful;)i(also)d(it)h(is)f(restrictiv)o (e,)i(as)d Fd(MPI)p 906 2244 V 15 w(LB)h Fq(and)g Fd(MPI)p 1157 2244 V 14 w(UB)g Fq(are)f(\\stic)o(ky":)28 b(once)19 b(presen)o(t)g(in)h(a)75 2300 y(datat)o(yp)q(e,)d(they)h(cannot)g(b)q (e)g(o)o(v)o(eriden)g(\(e.g.,)f(the)h(upp)q(er)h(b)q(ound)f(can)g(b)q (e)g(mo)o(v)o(ed)g(up,)g(b)o(y)g(adding)g(a)75 2357 y(new)13 b Fd(MPI)p 243 2357 V 14 w(UB)g Fq(mark)o(er,)f(but)h(cannot)g(b)q(e)g (mo)o(v)o(ed)g(do)o(wn)f(b)q(elo)o(w)i(an)f(existing)h Fj(MPI)p 1474 2357 14 2 v 15 w(UB)g Fq(mark)o(er\).)k(A)13 b(new)75 2413 y(function)j(is)g(pro)o(vided)g(to)e(facilitate)i(these)g (c)o(hanges.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 7 9 7 8 bop 75 -100 a Fg(3.7.)34 b(MPI-1.0)14 b(AND)i(MPI-1.1)e(CLARIFICA)l (TIONS)802 b Fq(7)75 45 y Fj(MPI)p 160 45 14 2 v 16 w(SET)p 259 45 V 17 w(EXTENT\(datat)o(yp)q(e,)16 b(lb,)f(extent\))117 122 y Fl(INOUT)62 b Fj(datat)o(yp)q(e)424 b Fl(datat)o(yp)q(e)14 b(to)g(up)q(date)h(exten)o(t)117 197 y(IN)155 b Fj(true)p 396 197 V 17 w(lb)461 b Fl(new)15 b(lo)o(w)o(er)e(b)q(ound)h(of)f (datat)o(yp)q(e)117 273 y(IN)155 b Fj(true)p 396 273 V 17 w(extent)379 b Fl(new)15 b(exten)o(t)f(of)g(datat)o(yp)q(e)75 397 y Fk(int)23 b(MPI)p 245 397 15 2 v 17 w(Set)p 334 397 V 17 w(extent\(MPI)p 591 397 V 16 w(Datatype)g(datatype,)f(MPI)p 1132 397 V 17 w(Aint)h(lb,)h(MPI)p 1436 397 V 17 w(Aint)f(extent\))75 483 y(MPI)p 150 483 V 17 w(TRUE)p 263 483 V 16 w(EXTENT\(DATATYPE,)f (LB,)i(EXTENT,)e(IERROR\))170 540 y(INTEGER)h(DATATYPE,)g(LB,)g (EXTENT,)g(IERROR)166 626 y Fq(Resets)13 b(the)g(lo)o(w)o(er)g(b)q (ound)h(and)f(the)h(exten)o(t)e(\(and,)h(hence,)h(the)g(upp)q(er)g(b)q (ound\))f(of)g Fj(datat)o(yp)q(e)p Fq(.)21 b(The)75 683 y(new)e(settings)f(will)i(a\013ect)e(the)h(outcome)f(of)g(send)h(or)f (receiv)o(e)h(op)q(erations)g(that)e(use)i Fj(datat)o(yp)q(e)p Fq(,)i(and)75 739 y(a\013ects)14 b(the)i(v)m(alue)g(of)f(datat)o(yp)q (es)g(constructed)g(using)h Fj(datat)o(yp)q(e)h Fq(as)e(an)g(agumen)o (t,)f(after)h(the)g(call)i(w)o(as)75 796 y(made.)i(Ho)o(w)o(ev)o(er,)11 b(datat)o(yp)q(es)h(that)f(w)o(ere)h(previously)i(constructed)e(using)h Fj(datat)o(yp)q(e)h Fq(are)e(not)g(a\013ected.)189 902 y Fe(R)n(ationale.)189 977 y Fq(This)h(is)g(consisten)o(t)f(to)g(the)h (MPI)f(statemen)o(t)g(that)g(\\the)g(system)g(b)q(eha)o(v)o(es)h(as)f (if)h(input)g(datat)o(yp)q(e)189 1034 y(argumen)o(ts)h(to)h(deriv)o(ed) h(datat)o(yp)q(e)e(constructors)h(are)g(passed)g(b)o(y)g(v)m(alue".)189 1109 y(\()p Fe(End)g(of)i(r)n(ationale.)p Fq(\))189 1215 y Fe(A)n(dvic)n(e)g(to)h(users.)51 b Fq(It)17 b(is)g(recommended)h (that)f(uses)g(use)g(the)g(function)h Fj(MPI)p 1571 1215 14 2 v 16 w(SET)p 1670 1215 V 17 w(EXTENT)p Fq(,)189 1271 y(rather)c(than)h Fd(MPI)p 508 1271 13 2 v 15 w(LB)g Fq(and)g Fd(MPI)p 751 1271 V 15 w(UB)f Fq(mark)o(ers.)189 1347 y(\()p Fe(End)h(of)i(advic)n(e)f(to)g(users.)p Fq(\))1875 1408 y Fi(?)g Fh(\(Oct\))75 1540 y Fn(3.7)59 b(MPI-1.0)19 b(and)h(MPI-1.1)f(Cla)n(ri\014cations)75 1643 y Ff(3.7.1)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 1643 15 2 v 18 w(FINALIZE)1875 1672 y Fi(>)g Fh(\(Oct\))75 1776 y Fm(Curren)o(t)e(Status:)j Fl(Not)d(v)o(oted)g(on.)166 1880 y Fq(This)j(routine)f(cleans)h(up)f(all)h(MPI)f(state.)22 b(Eac)o(h)16 b(pro)q(cess)g(m)o(ust)f(call)i Fj(MPI)p 1485 1880 14 2 v 16 w(FINALIZE)e Fq(b)q(efore)i(it)75 1936 y(exits.)j(Once)c(this)f(routine)g(returns,)f(no)h(MPI)g(routine)g (\(ev)o(en)g(not)f Fj(MPI)p 1338 1936 V 16 w(INIT)p Fq(\))f(ma)o(y)h(b) q(e)i(called.)21 b(Eac)o(h)75 1992 y(pro)q(cess)11 b(m)o(ust)e (complete)j(an)o(y)e(p)q(ending)i(comm)o(unication)f(it)g(initiated)h (b)q(efore)e(it)h(calls)g Fj(MPI)p 1656 1992 V 16 w(FINALIZE)p Fq(.)75 2049 y(If)16 b(the)g(call)h(returns,)e(eac)o(h)h(pro)q(cess)g (ma)o(y)f(con)o(tin)o(ue)i(lo)q(cal)f(computations,)g(or)f(exit,)h (without)g(partici-)75 2105 y(pating)g(in)g(further)g Fj(MPI)f Fq(comm)o(unication)h(with)g(other)f(pro)q(cesses.)22 b Fj(MPI)p 1362 2105 V 16 w(FINALIZE)14 b Fq(is)j(collectiv)o(e)g(on)75 2162 y Fj(MPI)p 160 2162 V 16 w(COMM)p 318 2162 V 16 w(W)o(ORLD)p Fq(.)1369 b Fi(>)16 b Fh(\(Oct\))166 2218 y Fq(It)f(is)h(not)f(required)h(that)e Fj(MPI)p 703 2218 V 16 w(Finalize)h Fq(return,)g(ev)o(en)g(on)h(one)f(pro)q(cess.)1875 2220 y Fi(?)h Fh(\(Oct\))189 2325 y Fe(A)n(dvic)n(e)11 b(to)i(implementors.)38 b Fq(Ev)o(en)11 b(though)g(a)g(pro)q(cess)h (has)f(completed)i(all)f(the)f(comm)o(unication)189 2381 y(it)j(initiated,)h(suc)o(h)g(comm)o(unication)f(ma)o(y)f(not)h(y)o(et) g(b)q(e)g(completed)h(from)e(the)h(viewp)q(oin)o(t)h(of)f(the)189 2437 y(underlying)j(MPI)f(system.)k(E.g.,)14 b(a)h(blo)q(c)o(king)i (send)f(ma)o(y)f(ha)o(v)o(e)g(completed,)i(ev)o(en)e(though)h(the)189 2494 y(data)k(is)h(still)h(bu\013ered)f(at)f(the)h(sender.)37 b(The)20 b(MPI)h(implemen)o(tation)h(m)o(ust)e(ensure)h(that)f(a)189 2550 y(pro)q(cess)14 b(has)g(completed)h(an)o(y)f(in)o(v)o(olv)o(emen)o (t)g(in)h(MPI)f(comm)o(unication)h(b)q(efore)f Fj(MPI)p 1669 2550 V 16 w(FINALIZE)189 2607 y Fq(returns.)26 b(Th)o(us,)17 b(if)h(a)e(pro)q(cess)i(exits)g(after)e(the)h(call)i(to)d Fj(MPI)p 1271 2607 V 16 w(FINALIZE)p Fq(,)g(this)i(will)h(not)e(cause) 189 2663 y(an)e(ongoing)g(comm)o(unication)h(to)e(fail.)21 b(\()p Fe(End)16 b(of)g(advic)n(e)g(to)h(implementors.)p Fq(\))-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 8 10 8 9 bop 75 -100 a Fq(8)970 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fe(A)n(dvic)n(e)i(to)i(implementors.)54 b Fq(Although)18 b(it)g(is)g(not)f(required,)i(it)f(has)f(b)q(een)i (found)f(that)f(users)189 102 y(exp)q(ect)f(that)f(at)f(least)i(one)f (pro)q(cess)h(return,)f(so)g(that)g(they)h(can)f(kno)o(w)g(that)g(the)g (MPI)h(p)q(ortion)189 158 y(of)e(the)g(computation)g(is)h(o)o(v)o(er.)k (In)c(addition,)g(in)g(a)f(POSIX)i(en)o(vironmen)o(t,)e(they)h(ma)o(y)e (desire)j(to)189 214 y(supply)f(an)f(exit)g(co)q(de)h(for)e(eac)o(h)h (pro)q(cess)g(that)f(returns)h(from)f Fj(MPI)p 1357 214 14 2 v 16 w(FINALIZE)p Fq(.)g(\()p Fe(End)h(of)h(advic)n(e)189 271 y(to)h(implementors.)p Fq(\))-117 332 y Fi(>)f Fh(\(Oct\))75 464 y Fn(3.8)59 b(Determining)19 b(Whether)g(MPI)h(Has)g(Finished)75 613 y Fm(Curren)o(t)14 b(Status:)j Fl(No)d(v)o(otes.)166 716 y Fq(One)i(of)f(the)h(goals)f(of)h Fj(MPI)f Fq(w)o(as)f(to)h(allo)o (w)h(for)f(la)o(y)o(ered)h(libraries.)23 b(In)16 b(order)f(for)g(a)h (library)g(to)f(do)75 773 y(this)k(cleanly)l(,)i(it)e(needs)g(to)f(kno) o(w)g(if)h Fj(MPI)f Fq(is)i(activ)o(e.)30 b(In)19 b Fj(MPI-1)f Fq(the)h(function)g Fj(MPI)p 1590 773 V 16 w(Initialized)g Fq(w)o(as)75 829 y(pro)o(vided)c(to)g(tell)g(if)g Fj(MPI)f Fq(had)h(b)q(een)h(initialzed.)23 b(The)15 b(problem)g(arises)g(in)g (kno)o(wing)g(if)g Fj(MPI)f Fq(has)h(b)q(een)75 886 y(\014nalized.)31 b(Once)19 b Fj(MPI)f Fq(has)g(b)q(een)h(\014nalized)h(it)f(is)g(no)f (longer)g(activ)o(e)h(and)f(cannot)g(b)q(e)h(restarted.)28 b(A)75 942 y(library)16 b(needs)g(to)e(b)q(e)i(able)g(to)e(determine)i (this)f(to)g(act)f(accordingly)l(.)22 b(Do)14 b(ac)o(hiev)o(e)i(this)f (the)g(follo)o(wing)75 999 y(function)h(is)g(needed:)75 1150 y Fj(MPI)p 160 1150 V 16 w(FINALIZED\(\015ag\))117 1227 y Fl(OUT)108 b Fj(\015ag)518 b Fl(true)15 b(if)e Fd(MPI)g Fl(w)o(as)h(\014nalized)g(\(logical\).)75 1351 y Fk(int)23 b(MPI)p 245 1351 15 2 v 17 w(Finalized\(int)f(*flag\))75 1438 y(MPI)p 150 1438 V 17 w(FINALIZED\(flag,)g(IERROR\)\))170 1494 y(FLAG,)i(IERROR)75 1581 y(int)f(MPI::Finalized\(bool)f (&flag\)\)?????)166 1667 y Fq(This)17 b(function)g(returns)f(true)g(if) h Fj(MPI)p 833 1667 14 2 v 16 w(FINALIZE)e Fq(has)h(returned.)23 b(Th)o(us,)16 b(it)h(will)h(return)e(false)g(if)75 1724 y(called)k(in)f(a)f(callbac)o(k)h(routine)f(in)o(v)o(ok)o(ed)h(b)o(y)f Fj(MPI)p 960 1724 V 16 w(FINALIZE)p Fq(.)f(It)h(is)g(legal)h(to)f(call) h Fj(MPI)p 1636 1724 V 16 w(FINALIZED)75 1780 y Fq(b)q(efore)c Fj(MPI)p 296 1780 V 16 w(INIT)g Fq(and)g(after)g Fj(MPI)p 694 1780 V 15 w(FINALIZE)p Fq(.)-1030 b Fi(?)15 b Fh(\(Oct\))-117 1839 y Fi(>)g Fh(\(Oct\))75 1902 y Ff(3.8.1)49 b(Cla)o(ri\014cation)18 b(of)e(status)f(after)h(MPI)p 875 1902 15 2 v 18 w(Isend)75 2035 y Fm(Curren)o(t)e(Status:)j Fl(No)d(v)o(otes.)166 2138 y Fq(A)22 b Fj(status)i Fq(returned)e(b)o(y)g(a)g(call)h(to)e Fj(MPI)p 900 2138 14 2 v 16 w(W)l(AIT)p Fq(,)g Fj(MPI)p 1145 2138 V 16 w(TEST)p Fq(,)h(or)f(an)o(y)h(of)f(the)h(other)g(deriv)o (ed)75 2195 y(functions,)j(where)f(the)f Fj(request)h Fq(corresp)q(onds)g(to)e(a)h(send)g(call,)j(is)e(unde\014ned,)i(with)d (t)o(w)o(o)f(excep-)75 2251 y(tions:)36 b(The)24 b(error)f(status)f (\014eld)j(will)g(con)o(tain)e(v)m(alid)i(information)f(if)f(the)h(w)o (ait)f(or)g(test)f(call)j(re-)75 2308 y(turned)20 b(with)g Fd(MPI)p 411 2308 13 2 v 15 w(ERROR)p 563 2308 V 13 w(IN)p 617 2308 V 15 w(ST)m(A)m(TUS)p Fq(;)f(and)h(the)g(returned)g(status)f (can)h(b)q(e)g(querried)h(b)o(y)f(the)g(call)75 2364 y Fj(MPI)p 160 2364 14 2 v 16 w(TEST)p 290 2364 V 16 w(CANCELLED)p Fq(.)-690 b Fi(?)15 b Fh(\(Oct\))75 2486 y Ff(3.8.2)49 b(Cla)o(ri\014cation)18 b(of)e(MPI)p 627 2486 15 2 v 18 w(INTERCOMM)p 936 2486 V 20 w(CREA)l(TE)75 2619 y Fm(Curren)o(t)e(Status:)j Fl(P)o(assed)e(once.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 9 11 9 10 bop 75 -100 a Fg(3.8.)34 b(DETERMINING)15 b(WHETHER)h Fj(MPI)e Fg(HAS)i(FINISHED)630 b Fq(9)75 45 y Fj(The)11 b(Problem:)44 b Fq(The)12 b(MPI)e(1.1)g(standard)h(sa)o(ys,)f(in)i(the) f(discussion)h(of)f Fj(MPI)p 1389 45 14 2 v 16 w(INTERCOMM)p 1679 45 V 16 w(CREA)l(TE)p Fq(,)75 102 y(b)q(oth)k(that)189 195 y(The)g(groups)g(m)o(ust)g(b)q(e)g(disjoin)o(t)75 289 y(and)g(that)189 383 y(The)g(leaders)h(ma)o(y)e(b)q(e)i(the)g(same) e(pro)q(cess.)75 477 y(T)l(o)d(further)h(m)o(uddy)g(the)g(w)o(aters,)e (the)i(reason)f(giv)o(en)i(for)e(\\The)g(groups)g(m)o(ust)h(b)q(e)g (disjoin)o(t")g(is)g(based)g(on)75 533 y(concerns)18 b(ab)q(out)g(the)g(implmen)o(tation)g(of)g Fj(MPI)p 922 533 V 15 w(INTERCOMM)p 1211 533 V 17 w(CREA)l(TE)h Fq(that)e(are)g(not) h(applicable)75 590 y(for)d(the)g(case)g(where)h(the)f(leaders)h(are)f (the)g(same)g(pro)q(cess.)75 710 y Fj(The)h(Fix:)45 b Fq(Delete)15 b(the)h(text:)189 804 y(\(the)f(t)o(w)o(o)e(leaders)j (could)h(b)q(e)e(the)h(same)f(pro)q(cess\))75 897 y(from)f(the)i (discussion)h(of)d Fj(MPI)p 610 897 V 16 w(INTERCOMM)p 900 897 V 17 w(CREA)l(TE)p Fq(.)166 954 y(Replace)j(the)e(text:)189 1048 y(All)f(in)o(ter-comm)o(unicator)g(constructors)e(are)h(blo)q(c)o (king)h(and)g(require)g(that)e(the)i(lo)q(cal)g(and)189 1104 y(remote)g(groups)h(b)q(e)h(disjoin)o(t)g(in)g(order)f(to)f(a)o(v) o(oid)h(deadlo)q(c)o(k.)75 1198 y(with)189 1292 y(All)f(in)o(ter-comm)o (unicator)g(constructors)e(are)h(blo)q(c)o(king)h(and)g(require)g(that) e(the)i(lo)q(cal)g(and)189 1348 y(remote)g(groups)h(b)q(e)h(disjoin)o (t.)289 1454 y Fe(A)n(dvic)n(e)e(to)h(users.)40 b Fq(The)14 b(groups)g(m)o(ust)f(b)q(e)i(disjoin)o(t)g(for)e(sev)o(eral)h(reasons.) 19 b(Primar-)289 1511 y(ily)l(,)12 b(this)f(is)g(the)f(in)o(ten)o(t)h (of)f(the)g(in)o(tercomm)o(unicators)h({)f(to)g(pro)o(vide)h(a)f(comm)o (unicator)289 1567 y(for)17 b(comm)o(unication)i(b)q(et)o(w)o(een)g (disjoin)o(t)f(groups.)29 b(This)19 b(is)f(re\015ected)h(in)g(the)g (de\014-)289 1624 y(nition)e(of)f Fj(MPI)p 559 1624 V 15 w(INTERCOMM)p 848 1624 V 17 w(MERGE)p Fq(,)g(whic)o(h)h(allo)o(ws)g (the)f(user)g(to)f(con)o(trol)h(the)289 1680 y(ranking)k(of)g(the)g (pro)q(cesses)g(in)h(the)g(created)f(in)o(tracomm)o(unicator;)h(this)g (ranking)289 1737 y(mak)o(es)c(little)i(sense)f(if)g(the)g(groups)g (are)f(not)h(disjoin)o(t.)28 b(In)18 b(addition,)h(the)f(natural)289 1793 y(extension)c(of)g(collectiv)o(e)i(op)q(erations)e(to)f(in)o (tercomm)o(unicators)h(\(b)q(eing)h(considered)289 1850 y(for)i(MPI-2\))h(mak)o(es)f(the)h(most)g(sense)g(when)h(the)f(groups)g (are)f(disjoin)o(t.)30 b(It)18 b(is)g(the)289 1906 y(in)o(ten)o(t)i(of) g(the)g(MPI)g(de\014nition)i(not)e(to)f(preclude)j(future)f (extensions.)35 b(\()p Fe(End)20 b(of)289 1963 y(advic)n(e)c(to)g (users.)p Fq(\))75 2084 y Ff(3.8.3)49 b(Cla)o(ri\014cation)18 b(of)e(Binding)i(of)f(MPI)p 854 2084 15 2 v 18 w(T)l(yp)q(e)p 971 2084 V 18 w(size)75 2217 y Fm(Curren)o(t)d(Status:)j Fl(P)o(assed)e(once.)166 2321 y Fq(This)i(clari\014cation)h(is)f (needed)h(in)f(the)g(MPI-1)f(description)i(of)e Fj(MPI)p 1369 2321 14 2 v 16 w(T)l(yp)q(e)p 1477 2321 V 17 w(size)p Fq(,)h(since)g(the)g(issue)75 2378 y(rep)q(eatedly)f(arises.)k(It)c(is) f(a)g(clari\014cation)i(of)e(the)g(binding.)189 2484 y Fe(A)n(dvic)n(e)20 b(to)j(users.)76 b Fq(The)21 b(binding)i(of)e Fj(MPI)p 1004 2484 V 16 w(T)l(yp)q(e)p 1112 2484 V 17 w(size)p Fq(,)i(sp)q(eci\014cally)l(,)i(whether)d(the)f(output)189 2540 y(argumen)o(t)e(in)i(C)g(is)f(of)g(t)o(yp)q(e)h Fk(int)f Fq(as)g(in)h(the)f(curren)o(t)h(standard)f(or)f(should)j(b)q (e)f(c)o(hanged)g(to)189 2597 y(something)14 b(else,)i(has)e(b)q(een)i (extensiv)o(ely)f(discussed)h(b)o(y)f(the)g(MPI)f(F)l(orum.)19 b(The)c(\014nal)g(decision)189 2653 y(is)g(to)g(lea)o(v)o(e)g(it)h(an)f Fk(int)p Fq(.)k(\()p Fe(End)d(of)g(advic)n(e)g(to)h(users.)p Fq(\))-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 10 12 10 11 bop 75 -100 a Fq(10)952 b Fg(CHAPTER)16 b(3.)29 b(MISCELLANY)17 b(F)o(OR)e(1.2)75 45 y Ff(3.8.4)49 b(Cla)o (ri\014cation)18 b(of)e(MPI)p 627 45 15 2 v 18 w(REDUCE)75 131 y Fq(The)f(curren)o(t)h(text)e(on)h(p.)20 b(115)15 b(lines)i(25-28)d(from)g(MPI-1.1)h(\(June)h(12,)e(1995\))f(sa)o(ys:)166 187 y(The)j Fj(datat)o(yp)q(e)i Fq(argumen)o(t)e(of)f Fj(MPI)p 783 187 14 2 v 16 w(REDUCE)i Fq(m)o(ust)f(b)q(e)g(compatible)i (with)e Fj(op)p Fq(.)23 b(Prede\014ned)18 b(op-)75 244 y(erators)h(w)o(ork)g(only)i(with)f(the)g Fj(MPI)g Fq(t)o(yp)q(es)g (listed)h(in)g(Sec.)35 b(4.9.2)18 b(and)j(Sec.)35 b(4.9.3.)d (User-de\014ned)75 300 y(op)q(erators)14 b(ma)o(y)h(op)q(erate)g(on)g (general,)g(deriv)o(ed)h(datat)o(yp)q(es.)166 357 y(This)g(text)e(is)i (c)o(hanged)g(to:)166 413 y(The)g Fj(datat)o(yp)q(e)i Fq(argumen)o(t)e(of)f Fj(MPI)p 783 413 V 16 w(REDUCE)i Fq(m)o(ust)f(b)q(e)g(compatible)i(with)e Fj(op)p Fq(.)23 b(Prede\014ned)18 b(op-)75 470 y(erators)c(w)o(ork)h(only)h(with)f(the) h Fj(MPI)f Fq(t)o(yp)q(es)g(listed)i(in)f(Sec.)21 b(4.9.2)14 b(and)h(Sec.)21 b(4.9.3.)e(F)l(urthermore,)c(the)75 526 y Fj(datat)o(yp)q(e)20 b Fq(and)e Fj(op)h Fq(giv)o(en)f(for)g (prede\014ned)i(op)q(erators)d(m)o(ust)h(b)q(e)h(the)f(same)g(on)g(all) h(pro)q(cesses.)30 b(User-)75 583 y(de\014ned)15 b(op)q(erators)e(ma)o (y)g(op)q(erate)g(on)h(general,)g(deriv)o(ed)h(datat)o(yp)q(es)e (without)g(the)h(restrictions)g(ab)q(o)o(v)o(e)75 639 y(for)h(prede\014ned)h(op)q(erators.)-681 b Fi(>)15 b Fh(\(Oct\))166 695 y Fq(Note)10 b(that)f(it)i(is)f(p)q(ossible)i(for)e (users)g(to)f(supply)j(di\013eren)o(t)e(user-de\014ned)i(op)q(erations) e(to)g Fj(MPI)p 1750 695 V 16 w(REDUCE)75 752 y Fq(in)23 b(eac)o(h)e(pro)q(cess.)40 b Fj(MPI)21 b Fq(do)q(es)h(not)f(de\014ne)i (whic)o(h)g(op)q(erations)e(are)h(used)g(in)h(this)f(case)f(on)h(whic)o (h)75 808 y(op)q(erands.)189 915 y Fe(A)n(dvic)n(e)e(to)h(users.)71 b Fq(Users)20 b(should)i(mak)o(e)d(no)i(assumptions)f(ab)q(out)g(ho)o (w)g Fj(MPI)p 1638 915 V 16 w(RECUCE)h Fq(is)189 971 y(implemen)o(ted.)29 b(Safest)17 b(is)h(to)f(ensure)i(that)e(the)g (same)h(function)g(is)g(passed)g(to)f Fj(MPI)p 1685 971 V 16 w(REDUCE)189 1028 y Fq(b)o(y)e(eac)o(h)g(pro)q(cess.)20 b(\()p Fe(End)c(of)g(advic)n(e)g(to)h(users.)p Fq(\))-117 1089 y Fi(?)e Fh(\(Oct\))75 1221 y Fn(3.9)59 b(New)20 b(Datat)n(yp)r(es)75 1324 y Ff(3.9.1)49 b(Simple)17 b(Struct)d(T)l(yp)q (es)75 1457 y Fm(Curren)o(t)g(Status:)j Fl(Not)d(v)o(oted)g(on)f(y)o (et.)75 1655 y Fj(MPI)p 160 1655 V 16 w(TYPE)p 293 1655 V 17 w(SIMPLE)p 469 1655 V 15 w(STRUCT\(count,)k(a)o(rra)o(y)p 908 1655 V 14 w(of)p 959 1655 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1334 1655 V 14 w(of)p 1385 1655 V 16 w(t)o(yp)q(es,)e(newt)o(yp)q(e\)) 117 1732 y Fl(IN)155 b Fj(count)482 b Fl(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 1807 y(IN)155 b Fj(a)o(rra)o(y)p 416 1807 V 15 w(of)p 468 1807 V 16 w(blo)q(cklengths)191 b Fl(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h (eac)o(h)g(blo)q(c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 1882 y(IN)155 b Fj(a)o(rra)o(y)p 416 1882 V 15 w(of)p 468 1882 V 16 w(t)o(yp)q(es)327 b Fl(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h (in)f(eac)o(h)h(blo)q(c)o(k)g(\(arra)o(y)g(of)f(handle\))117 1958 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fl(new)15 b(datat)o(yp)q(e)f (\(handle\))166 2082 y Fq(Eac)o(h)c(t)o(yp)q(e)h(in)g(the)g Fj(a)o(rra)o(y)p 592 2082 V 14 w(of)p 643 2082 V 16 w(t)o(yp)q(es)h Fq(list)g(m)o(ust)e(b)q(e)h(a)f Fe(p)n(ortable)h Fq(datat)o(yp)q(e:)17 b(I.e.,)10 b(a)h(basic)g(datat)o(yp)q(e,)f(or)75 2138 y(a)g(datat)o(yp)q(e)f(constructed)i(from)e(a)h(p)q(ortable)h(datat)o (yp)q(e)e(using)i(one)f(of)g(the)g(constructors)g Fj(MPI)p 1681 2138 V 15 w(TYPE)p 1813 2138 V 17 w(CONTIGUOUS,)75 2195 y(MPI)p 160 2195 V 16 w(TYPE)p 293 2195 V 17 w(VECTOR,)16 b(MPI)p 600 2195 V 15 w(TYPE)p 732 2195 V 17 w(INDEXED,)f Fq(or)g Fj(MPI)p 1110 2195 V 16 w(TYPE)p 1243 2195 V 16 w(SIMPLE)p 1418 2195 V 16 w(STRUCT)p Fq(.)166 2251 y(This)22 b(t)o(yp)q(e)g(constructor)f(is)i(similar)f(to)g Fj(MPI)p 988 2251 V 15 w(TYPE)p 1120 2251 V 17 w(STRUCT)p Fq(,)g(except)h(that)e(the)h(user)g(do)q(es)75 2308 y(not)e(supply)h (an)f(arra)o(y)e(of)i(displacemen)o(ts.)35 b(Instead,)22 b(the)e(successiv)o(e)h(blo)q(c)o(ks)f(are)g(assumed)g(to)f(b)q(e)75 2364 y(con)o(tiguous.)g(P)o(adding)11 b(spaces)h(are)e(added,)j (according)e(to)g(the)g(default)h(alignmen)o(t)g(rules)g(for)e (structure)75 2421 y(comp)q(onen)o(ts.)21 b(This)c(facilitates)f(the)g (construction)g(of)f(datat)o(yp)q(es)g(for)g(structures,)g(as)h(the)f (user)h(need)75 2477 y(not)f(compute)g(displacemen)o(ts.)-743 b Fi(>)15 b Fh(\(Oct\))166 2534 y Fq(This)h(can)f(probably)h(b)q(e)g (more)e(elegan)o(tly)i(expressed)g(in)g(terms)f(of)g(exten)o(ts.)-117 2536 y Fi(?)g Fh(\(Oct\))75 2640 y Fb(Example)j(3.1)k Fq(The)14 b(follo)o(wing)g(co)q(de)f(declares)h(a)f(structure)g(t)o(yp) q(e)g(and)h(creates)f(a)g(datat)o(yp)q(e)f(for)h(that)75 2696 y(structure)i(t)o(yp)q(e.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 11 13 11 12 bop 75 -100 a Fg(3.9.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPES)1245 b Fq(11)75 45 y Fk(struct)23 b(record)g({)147 102 y(char)g(name;)147 158 y(double)g(position[3];)147 214 y(float)47 b(mass;)147 271 y(})75 384 y(MPI_Datatype)22 b(record_type;)75 440 y(MPI_datatype)g(types[3])h(=)h({MPI_CHAR,)e(MPI_DOUBLE,)h(MPI_FLOAT};) 75 497 y(int)238 b(lengths[3])23 b(=)g({1,)h(3,)f(1};)75 610 y(MPI_Type_simple_struct\()e(3,)i(lengths,)g(types,)g (record_type\);)189 704 y Fe(R)n(ationale.)76 b Fq(The)21 b(co)q(de)h(ab)q(o)o(v)o(e)f(is)h(m)o(uc)o(h)f(simpler)h(and)g(more)e (natural)i(than)f(the)g(curren)o(t)189 760 y(approac)o(h,)14 b(that)h(uses)g Fj(MPI)p 677 760 14 2 v 16 w(TYPE)p 810 760 V 17 w(STRUCT)p Fq(,)g(and)h(requires)g(to)e(compute)h(displacemen) o(ts.)189 834 y(It)i(also)g(pro)o(vides)h(a)f(de\014nition)i(of)e(the)h (record)f(t)o(yp)q(e)h(that)e(is)i(arc)o(hitecture)g(indep)q(enden)o(t) i(\(as-)189 890 y(suming)c(that)e(compilers)j(don't)e(\014ddle)i(with)e (the)h(order)f(of)g(\014elds,)h(whic)o(h)g(is)g(true)f(in)i(C\))d({)i (this)189 947 y(facilitates)f(transfer)e(of)h(records)g(using)h(RMA,)f (an)g(\014ts)g(w)o(ell)h(with)g(v)m(arious)f(prop)q(osals)h(for)e (third)189 1003 y(part)o(y)h(transfers.)19 b(\()p Fe(End)d(of)g(r)n (ationale.)p Fq(\))189 1097 y Fe(A)n(dvic)n(e)h(to)h(users.)50 b Fq(Datat)o(yp)q(es)16 b(constructed)h(using)h Fj(MPI)p 1231 1097 V 16 w(TYPE)p 1364 1097 V 16 w(SIMPLE)p 1539 1097 V 16 w(STRUCT)g Fq(should)189 1154 y(b)q(e)i(used)g(only)g(for)f (structures)g(that)g(use)h(the)g(default)g(alignmen)o(t)g(rules.)34 b(They)20 b(should)g(not)189 1210 y(b)q(e)c(used)h(if)f(alignmen)o(t)g (rules)h(ha)o(v)o(e)f(b)q(een)h(c)o(hanged,)f(using)g(compiler)h (options)f(or)g(compilation)189 1267 y(pragmas.)j(\()p Fe(End)c(of)i(advic)n(e)f(to)g(users.)p Fq(\))189 1361 y Fe(A)n(dvic)n(e)g(to)h(implementors.)47 b Fq(The)16 b(datat)o(yp)q(e)g(constructor)f Fj(MPI)p 1330 1361 V 16 w(TYPE)p 1463 1361 V 17 w(SIMPLE)p 1639 1361 V 15 w(STRUCT)i Fq(is)189 1417 y(most)e(easily)i(implemen)o(ted)h(on)e (systems)g(that)f(use)h Fe(natur)n(al)g Fq(data)g(alignmen)o(t)h (rules.)23 b(On)17 b(suc)o(h)189 1474 y(systems,)12 b(eac)o(h)h(basic)g (datat)o(yp)q(e)f(has)g(an)h(exten)o(t)f(and)h(an)g(alignmen)o(t;)h (the)e(alignmen)o(t)i(of)e(a)g(com-)189 1530 y(p)q(ound)j(t)o(yp)q(e)g (is)h(the)f(strictest)f(alignmen)o(t)i(of)e(a)g(comp)q(onen)o(t.)20 b(P)o(adding)c(is)f(added)g(so)g(that)f(eac)o(h)189 1587 y(comp)q(onen)o(t)f(is)h(aligned)g(at)f(its)g(natural)g(alignmen)o(t.) 20 b(W)l(e)13 b(can)g(then)h(deriv)o(e)g(ob)o(vious)f(de\014nitions)189 1643 y(for)h(the)h(exten)o(t)g(and)h(alignmen)o(t)g(of)e(p)q(ortable)i (MPI)f(datat)o(yp)q(es:)189 1700 y(The)g(exten)o(t)f(and)h(alignmen)o (t)g(of)f(a)h(basic)g(datat)o(yp)q(e)f(is)i(the)e(exten)o(t)h(and)g (alignmen)o(t)g(of)f(the)h(cor-)189 1756 y(resp)q(onding)h(language)f (t)o(yp)q(e.)189 1812 y(The)10 b(alignmen)o(t)h(of)f(a)f(datat)o(yp)q (e)h(built)i(with)e(one)g(of)g(the)g(constructors)g Fj(MPI)p 1487 1812 V 15 w(TYPE)p 1619 1812 V 17 w(CONTIGUOUS,)189 1869 y(MPI)p 274 1869 V 15 w(TYPE)p 406 1869 V 17 w(VECTOR,)26 b Fq(and)e Fj(PI)p 780 1869 V 16 w(TYPE)p 913 1869 V 17 w(INDEXED)h Fq(is)g(the)f(alignmen)o(t)h(of)f(the)h(comp)q(onen)o(t) 189 1925 y(datat)o(yp)q(e;)14 b(the)h(exten)o(t)g(of)g(the)g(comp)q (ound)h(datat)o(yp)q(e)f(is)g(as)g(de\014ned)i(in)f(MPI1.)189 1982 y(The)h(alignmen)o(t)g(of)f(a)g(datat)o(yp)q(e)h(built)h(b)o(y)e (a)h(call)g(to)f Fj(MPI)p 1221 1982 V 16 w(TYPE)p 1354 1982 V 17 w(SIMPLE)p 1530 1982 V 16 w(STRUCT\(n,)h(lens,)189 2038 y(t)o(yp)q(es,)i(newt)o(yp)q(e\))h Fq(is)e(max)643 2045 y Fa(i)672 2038 y Fk(alignment)p Fq(\()p Fk(types)p Fq([)p Fk(i)p Fq(]\);)e(the)i(exten)o(t)g(is)g(computed)h(b)o(y)f(la)o (ying)g(out)189 2095 y(the)h(comp)q(onen)o(ts)g(with)h(minimal)h (padding)f(added)g(so)f(that)g(eac)o(h)h(comp)q(onen)o(t)f(starts)f(at) h(its)189 2151 y(alignmen)o(t.)189 2225 y(Some)g(compilers)h(treat)f (the)g(\014rst)g(comp)q(onen)o(t)g(of)g(a)g(structure)g(di\013eren)o (tly)l(.)33 b(E.g.,)18 b(the)i(IBM)189 2281 y(compiler)d(aligns)h (doubles)f(at)f(w)o(ord)g(b)q(oundaries,)i(but)f(align)g(a)f(structure) g(that)g(starts)f(with)i(a)189 2338 y(double)j(comp)q(onen)o(t)f(at)f (doublew)o(ord)i(b)q(oundaries.)32 b(The)19 b(de\014nition)i(of)d(the)h (alignmen)o(t)h(of)e(a)189 2394 y(datat)o(yp)q(e)h(built)i(with)f Fj(MPI)p 690 2394 V 16 w(TYPE)p 823 2394 V 16 w(SIMPLE)p 998 2394 V 16 w(STRUCT)h Fq(is)f(c)o(hanged,)h(accordingly)l(.)34 b(\()p Fe(End)20 b(of)189 2450 y(advic)n(e)c(to)g(implementors.)p Fq(\))75 2571 y Ff(3.9.2)49 b(Contiguous)17 b(Struct)e(T)l(yp)q(es)75 2704 y Fm(Curren)o(t)f(Status:)j Fl(F)m(ailed)c(6-7-10)f(at)i(Septem)o (b)q(er)g(meeting.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 12 14 12 13 bop 75 -100 a Fq(12)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)166 45 y Fq(Data)j(structures)i(ma)o(y)f (con)o(tain)h(padding)g(holes,)i(added)e(for)f(alignmen)o(t.)34 b(The)20 b(de\014nition)h(of)75 102 y(an)f(MPI)g(datat)o(yp)q(e,)g (done)g(using)g(the)g Fj(MPI)p 865 102 14 2 v 16 w(TYPE)p 998 102 V 17 w(STRUCT)g Fq(constructor,)g(do)q(es)g(not)g(distinguish) 75 158 y(b)q(et)o(w)o(een)15 b(holes)g(that)f(con)o(tains)g(no)h (signi\014can)o(t)g(data,)f(and)h(are)f(there)h(only)g(for)f(alignmen)o (t)h(purp)q(oses,)75 214 y(and)k(b)q(et)o(w)o(een)g(holes)g(that)f(ma)o (y)f(con)o(tain)i(signi\014can)o(t)g(data)f(and)h(w)o(ere)f(purp)q (osefully)j(left)e(out)f(b)o(y)g(a)75 271 y(user)k(that)e(wishes)i(not) f(to)g(o)o(v)o(erwrite)g(some)g(of)g(the)g(information)h(in)g(a)f (structure.)38 b(As)22 b(a)f(result,)75 327 y(MPI)g(cannot)g(o)o(v)o (erwrite)f(these)h(holes,)i(this)f(prev)o(en)o(ts)e(the)i(ob)o(vious)f (optimization)h(of)e(shipping)j(a)75 384 y(data)13 b(structure)h(as)g (a)f(con)o(tiguous)h(blo)q(c)o(k)h(of)f(data,)f(padding)i(holes)f (included.)22 b(The)15 b(t)o(yp)q(e)f(constructor)75 440 y Fj(MPI)p 160 440 V 16 w(TYPE)p 293 440 V 17 w(CONTIGUOUS)p 598 440 V 18 w(STRUCT)d Fq(marks)f(an)o(y)g(suc)o(h)h(holes)g(as)g (\\don)o(t-care")e(lo)q(cations,)j(allo)o(wing)75 497 y(the)j(implemen)o(tation)i(to)d(o)o(v)o(erwrite)h(them.)75 648 y Fj(MPI)p 160 648 V 16 w(TYPE)p 293 648 V 17 w(CONTIGUOUS)p 598 648 V 18 w(STRUCT\(count,)j(a)o(rra)o(y)p 1041 648 V 14 w(of)p 1092 648 V 16 w(blo)q(cklengths,)h(a)o(rra)o(y)p 1468 648 V 14 w(of)p 1519 648 V 16 w(displacements,)g(a)o(r-)75 704 y(ra)o(y)p 136 704 V 15 w(of)p 188 704 V 16 w(t)o(yp)q(es,)e(newt)o (yp)q(e\))117 781 y Fl(IN)155 b Fj(count)482 b Fl(n)o(um)o(b)q(er)13 b(of)h(blo)q(c)o(ks)g(\(in)o(teger\))117 856 y(IN)155 b Fj(a)o(rra)o(y)p 416 856 V 15 w(of)p 468 856 V 16 w(blo)q(cklengths) 191 b Fl(n)o(um)o(b)q(er)13 b(of)h(elemen)o(ts)f(in)h(eac)o(h)g(blo)q (c)o(k)g(\(arra)o(y)g(of)f(in)o(teger\))117 932 y(IN)155 b Fj(a)o(rra)o(y)p 416 932 V 15 w(of)p 468 932 V 16 w(displacements)164 b Fl(b)o(yte)14 b(displacemen)o(t)g(of)f(eac)o(h)h(blo)q(c)o(k)g (\(arra)o(y)g(of)f(in)o(teger\))117 1007 y(IN)155 b Fj(a)o(rra)o(y)p 416 1007 V 15 w(of)p 468 1007 V 16 w(t)o(yp)q(es)327 b Fl(t)o(yp)q(e)15 b(of)e(elemen)o(ts)h(in)f(eac)o(h)h(blo)q(c)o(k)g (\(arra)o(y)g(of)f(handle\))117 1082 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fl(new)15 b(datat)o(yp)q(e)f(\(handle\))166 1206 y Fq(The)c(seman)o(tics)h(of)e Fj(MPI)p 587 1206 V 16 w(TYPE)p 720 1206 V 17 w(CONTIGUOUS)p 1025 1206 V 18 w(STRUCT)i Fq(are)f(iden)o(tical)i(to)d Fj(MPI)p 1611 1206 V 16 w(TYPE)p 1744 1206 V 17 w(STRUCT)p Fq(,)75 1263 y(except)15 b(that,)e(if)h(a)g(receiv)o(e)h(call)g(uses)g(a)e(datat)o(yp)q(e)h (constructed)g(with)h(this)f(constructor,)f(then)i(the)f(in-)75 1319 y(ternal)f(holes)h(in)f(the)g(structure)g(can)g(b)q(e)g(o)o(v)o (erwritten.)18 b(The)13 b(same)g(rule)h(\(namely)l(,)f(that)f(holes)i (b)q(et)o(w)o(een)75 1376 y(comp)q(onen)o(ts)c(can)g(b)q(e)h(o)o(v)o (erwritten\))e(applies)j(also)e(to)g(datat)o(yp)q(es)f(constructed)h (using)h Fj(MPI)p 1635 1376 V 16 w(TYPE)p 1768 1376 V 17 w(SIMPLE)p 1944 1376 V 16 w(STRUCT)p Fq(.)166 1479 y Fm(Discussion:)29 b Fl(It)10 b(is)f(reasonable)g(to)g(assume)g(that)h (holes)f(in)g(la)o(y)o(outs)f(de\014ned)j(using)e Fd(MPI)p 1576 1479 13 2 v 14 w(TYPE)p 1698 1479 V 14 w(SIMPLE)p 1857 1479 V 15 w(STRUCT)75 1536 y Fl(con)o(tain)14 b(no)h(signi\014can) o(t)f(information,)e(since)k(this)f(datat)o(yp)q(e)g(constructor)h(is)f (exp)q(ected)i(to)e(b)q(e)g(used)h(for)f(sp)q(ec-)75 1592 y(ifying)f(the)h(la)o(y)o(out)f(of)h(structures.)24 b(Users)16 b(will)e(b)q(e)i(required)g(to)f(use)h Fd(MPI)p 1271 1592 V 14 w(TYPE)p 1393 1592 V 14 w(STRUCT)f Fl(when)g(holes)h (hold)75 1649 y(signi\014can)o(t)d(information.)189 1802 y Fe(A)n(dvic)n(e)e(to)i(users.)38 b Fq(Users)11 b(should)i(arrange)d (the)i(\014elds)g(in)g(a)f(C)h(struct)e(t)o(yp)q(e)i(in)g(descending)h (order)189 1859 y(of)i(their)i(size.)24 b(This)17 b(arrangemen)o(t)e (has)h(t)o(w)o(o)f(adv)m(an)o(tages:)21 b(\014rstly)c(it)f(ma)o(y)g (reduce)h(the)f(size)h(of)189 1915 y(the)i(struct)f(b)o(y)h (eliminating)i(padding)f(b)q(et)o(w)o(een)f(elemen)o(ts)h(with)f (di\013eren)o(t)g(alignmen)o(ts)h(\(this)189 1972 y(is)c(generally)i (true)e(for)g(C)g(programs\).)22 b(Secondly)l(,)c(it)e(ma)o(y)g (increase)h(the)g(n)o(um)o(b)q(er)f(of)g(cases)h(for)189 2028 y(whic)o(h)d(the)f(implemen)o(tation)h(can)g(p)q(erform)f(blo)q(c) o(k)g(copies)i(b)o(y)e(ensuring)h(that)e(the)h(\014rst)g(elemen)o(t)189 2085 y(of)19 b(the)g(structure)g(has)h(the)f(most)g(stringen)o(t)g (alignmen)o(t)h(requiremen)o(ts.)33 b(\()p Fe(End)19 b(of)i(advic)n(e)f(to)189 2141 y(users.)p Fq(\))189 2247 y Fe(A)n(dvic)n(e)15 b(to)i(implementors.)39 b Fq(An)16 b(implemen)o(tation)g(ma)o(y)f(ignore)g(the)h(constructor)189 2304 y Fj(MPI)p 274 2304 14 2 v 15 w(TYPE)p 406 2304 V 17 w(CONTIGUOUS)p 711 2304 V 18 w(STRUCT)p Fq(,)21 b(and)f(handle)i(it)e(exactly)h(as)f Fj(MPI)p 1539 2304 V 16 w(TYPE)p 1672 2304 V 17 w(STRUCT)p Fq(.)189 2360 y(Th)o(us,)f(there)g(is)h(\(almost\))d(no)i(o)o(v)o(erhead)g(in)g(supp) q(orting)h(this)f(constructor,)g(without)g(taking)189 2417 y(adv)m(an)o(tage)f(of)h(it.)32 b(One)20 b(p)q(ossible)h(w)o(a)o (y)d(of)h(taking)g(adv)m(an)o(tage)g(of)g(it,)h(is)g(to)e(adopt)h(a)g (canoni-)189 2473 y(cal)d(\\wire")g(represen)o(tation)g(for)g (structures.)22 b(Supp)q(ose)17 b(that)f(the)g(compiler)h(uses,)g(b)o (y)f(default,)189 2530 y(natural)g(alignmen)o(t)i(rules)f(for)f (structures:)23 b(eac)o(h)16 b(basic)i(comp)q(onen)o(t)e(is)i(aligned)g (at)e(a)g(natural)189 2586 y(b)q(oundary)l(.)k(Supp)q(ose)14 b(that)f(one)h(adopts)f(natural)h(alignmen)o(t)g(rules)h(for)e (messages)g(on)g(the)h(wire,)189 2642 y(in)19 b(a)f(homogeneous)g(en)o (vironmen)o(t:)27 b(messages)18 b(are)g(padded)h(so)f(that)g(eac)o(h)g (basic)h(elemen)o(t)g(is)189 2699 y(aligned)d(at)f(a)g(natural)g(b)q (oundary)l(.)20 b(Then)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 13 15 13 14 bop 75 -100 a Fg(3.9.)34 b(NEW)15 b(D)o(A)l(T)l(A)l(TYPES)1245 b Fq(13)189 45 y(\(1\))14 b(A)i(receiv)o(er)g(can)f(alw)o(a)o(ys)g (deco)q(de)h(an)g(incoming)h(message)d(and)i(retriev)o(e)g(correctly)f (the)h(ba-)189 102 y(sic)k(comp)q(onen)o(ts.)31 b(This)20 b(b)q(ecause)g(the)g(basic)g(datat)o(yp)q(e)e(of)h(the)g(elemen)o(t)h (indicates)h(at)e(what)189 158 y(b)q(oundary)c(it)h(is)f(aligned.)189 214 y(\(2\))f(A)h(sender)g(can)h(blo)q(c)o(k)f(cop)o(y)g(a)g(structure) g(on)o(to)f(the)h(\\wire")g(\(most)f(lik)o(ely)l(,)j(on)o(to)d(a)g (message)189 271 y(bu\013er\),)19 b(when)g(the)g(structure)g(is)g (naturally)h(aligned,)h Fe(r)n(elative)e(to)h(the)g(start)g(of)f(the)h (sending)189 327 y(bu\013er)p Fq(.)26 b(This)18 b(optimization)g(do)q (es)g(not)e(require)j(that)d(the)h(sending)i(datat)o(yp)q(e)e(b)q(e)h (built)g(using)189 384 y Fj(MPI)p 274 384 14 2 v 15 w(TYPE)p 406 384 V 17 w(CONTIGUOUS)p 711 384 V 18 w(STRUCT)p Fq(:)c(The)g(MPI)f (library)i(can)e(scan)h(datat)o(yp)q(es,)f(when)h(they)189 440 y(are)h(committed,)f(and)i(recognize)g(when)g(a)e(datat)o(yp)q(e)h (\(or)f(a)h(datat)o(yp)q(e)g(fragmen)o(t\))f(is)h(naturally)189 497 y(aligned.)24 b(Blo)q(c)o(k)17 b(cop)o(ying)g(can)g(b)q(e)g(used)g (ev)o(en)f(if)h(the)g(holes)g(in)g(the)f(datat)o(yp)q(e)g(con)o(tain)g (signi\014-)189 553 y(can)o(t)e(information.)189 610 y(\(3\))h(A)h(receiv)o(er)h(can)f(blo)q(c)o(k)h(cop)o(y)f(an)g (incoming)i(message)d(on)o(to)h(the)g(receiv)o(e)h(bu\013er)f(when)h (the)189 666 y(receiving)e(structure)f(is)h(naturally)g(aligned)h Fe(r)n(elative)f(to)g(the)h(start)g(of)f(the)h(r)n(e)n(c)n(eiving)d (bu\013er)i Fq(and)189 723 y(holes)10 b(can)h(b)q(e)g(ignored.)18 b(I.e.,)11 b(when)g(the)f(receiving)i(datat)o(yp)q(e)d(is)i(built)g (using)g Fj(MPI)p 1597 723 V 16 w(TYPE)p 1730 723 V 17 w(CONTIGUOUS)p 2035 723 V 18 w(STRUCT)189 779 y Fq(or)i Fj(MPI)p 328 779 V 16 w(TYPE)p 461 779 V 17 w(SIMPLE)p 637 779 V 16 w(STRUCT)h Fq(and)g(is)h(naturally)f(aligned,)i(relativ)o (e)e(to)g(its)g(\014rst)f(p)q(osition.)189 853 y(The)k(same)h(wire)g (represen)o(tation)f(ma)o(y)g(b)q(e)h(used)g(in)h(a)e(heterogenous)h (en)o(vironmen)o(t,)g(in)g(cases)189 909 y(where)h(data)e(con)o(v)o (ersion)i(is)g(done)g(at)f(the)h(destination;)i(the)e(destination)g (has)g(to)f(kno)o(w,)g(not)189 966 y(only)c(the)h(basic)g(datat)o(yp)q (es)e(used)i(b)o(y)f(the)h(source,)f(but)g(also)h(the)f(alignmen)o(t)h (rules)g(used)g(b)o(y)f(the)189 1022 y(source.)189 1096 y(Example:)34 b(Consider)23 b(a)f(C)g(structure)g Fk(struct)h({char)g (a[2];)g(float)g(b[2];)h(double)f(c})p Fq(;)189 1153 y(assume)16 b(that)g(\015oats)g(are)g(4)h(b)o(ytes)f(and)h(doubles)h (are)e(8)g(b)o(ytes,)h(and)g(structures)f(are)g(naturally)189 1209 y(aligned.)36 b(The)20 b(memory)f(la)o(y)o(out)h(is)44 b Fk([c][c]xx[ffff][ffff]xxxx[dddd)o(dddd])p Fq(,)18 b(and)i(the)189 1266 y(structure)15 b(is)g(aligned)i(to)d(an)i(8)e(b)o (yte)h(b)q(oundary)l(.)189 1340 y(Case)c(1:)18 b(The)12 b(en)o(tire)g(structure)f(is)h(comm)o(unicated.)20 b(Sender)12 b(constructs)f(a)h(datat)o(yp)q(e)f(for)g(struc-)189 1396 y(ture)f(using)h Fj(MPI)p 479 1396 V 15 w(TYPE)p 611 1396 V 17 w(STRUCT,)g(MPI)p 910 1396 V 15 w(TYPE)p 1042 1396 V 17 w(SIMPLE)p 1218 1396 V 16 w(STRUCT)g Fq(or)f Fj(MPI)p 1554 1396 V 15 w(TYPE)p 1686 1396 V 17 w(CONTIGUOUS)p 1991 1396 V 18 w(STRUCT)p Fq(;)189 1452 y(receiv)o(er)16 b(constructs)g(a)g(datat)o(yp)q(e)g(for)f(this)i(structure,)e(using)i Fj(MPI)p 1379 1452 V 16 w(CONTIGUOUS)p 1683 1452 V 18 w(STRUCT)189 1509 y Fq(or)12 b Fj(MPI)p 327 1509 V 16 w(TYPE)p 460 1509 V 16 w(SIMPLE)p 635 1509 V 16 w(STRUCT)p Fq(.)h(Then)g(the)g(la)o(y)o(outs)f(in)h(the)g(sender)g(memory)l(,)g (on)f(the)h(wire,)189 1565 y(and)i(in)h(the)f(receiv)o(er)h(memory)f (are)g(iden)o(tical,)h(and)g(data)e(can)i(b)q(e)g(simply)g(copied.)189 1639 y(Case)10 b(2:)17 b(comm)o(unication)12 b(in)o(v)o(olv)o(es)f (only)h(the)f(second)g(of)f(the)h(t)o(w)o(o)f(c)o(haracters,)g(and)h (the)g(remain-)189 1696 y(der)f(of)g(the)g(structure.)18 b(Receiv)o(er)11 b(constructs)f(the)g(datat)o(yp)q(e)g(using)h Fj(MPI)p 1441 1696 V 16 w(CONTIGUOUS)p 1745 1696 V 18 w(STRUCT)p Fq(.)189 1752 y(The)19 b(memory)f(la)o(y)o(out)g(is)h Fk([c]xx[ffff][ffff]xxxx[dddddd)o(dd])p Fq(,)d(and)j(the)g(wire)g(la)o (y)o(out)f(is)189 1809 y Fk([c]xxx[ffff][ffff]xxxx[)o(dddddddd)o(])p Fq(.)e(The)c(datat)o(yp)q(e)f(is)h(not)g(naturally)g(aligned,)h(so)f (that)189 1865 y(blo)q(c)o(k)k(cop)o(ying)g(cannot)f(b)q(e)h(used)g(at) f(the)h(sender)g(or)f(the)g(receiv)o(er.)22 b(Ho)o(w)o(ev)o(er,)14 b(a)h(smart)f(imple-)189 1922 y(men)o(tation)h(ma)o(y)g(realize)i(that) e(the)h(fragmen)o(t)f(starting)g(from)g(the)h(\015oat)f(is)i(naturally) f(aligned,)189 1978 y(relativ)o(e)e(to)e(the)i(start)e(of)h(the)g (bu\013er,)h(so)f(that)f(blo)q(c)o(k)j(cop)o(ying)e(can)h(b)q(e)g(used) g(for)f(this)g(fragmen)o(t.)189 2052 y(Case)f(3:)19 b(comm)o(unication) 14 b(in)o(v)o(olv)o(es)g(only)f(the)g(t)o(w)o(o)f(\015oats)h(and)g(the) g(double.)21 b(The)13 b(memory)g(la)o(y-)189 2108 y(out)c(is)i Fk([ffff][ffff]xxxx[dddddddd])o Fq(.)16 b(The)10 b(la)o(y)o(out)g(on)g (the)g(wire)g(is)h Fk([ffff][ffff][dddddddd])p Fq(.)189 2165 y(The)j(datat)o(yp)q(e)f(is)i(not)f(naturally)g(aligned,)i (relativ)o(e)e(to)g(its)g(start,)e(so)i(that)f(blo)q(c)o(k)i(cop)o (ying)g(can-)189 2221 y(not)f(b)q(e)i(used.)21 b(\()p Fe(End)15 b(of)i(advic)n(e)f(to)g(implementors.)p Fq(\))166 2360 y Fm(Alternativ)o(es)o(:)134 2432 y Fl(1.)22 b(No)14 b(new)h(function)f Fd(MPI)p 581 2432 13 2 v 14 w(TYPE)p 703 2432 V 15 w(CONTIGUOUS)p 985 2432 V 12 w(STRUCT)p Fl(:)g(instead,)g(holes)h(can)g(b)q(e)g(o)o(v)o(erwritten)g(only)189 2482 y(for)f(datat)o(yp)q(es)i(built)e(with)g Fd(MPI)p 717 2482 V 15 w(TYPE)p 840 2482 V 14 w(SIMPLE)p 999 2482 V 15 w(STRUCT)p Fl(.)f(The)j(argumen)o(t)e(is)g(that,)h(at)g(least)g (in)f(the)189 2532 y(scenario)f(I)g(presen)o(ted,)i(hole)d(o)o(v)o (erwriting)h(is)g(tak)o(en)g(adv)n(an)o(tage)f(of)g(only)g(for)h (structures)i(with)e(a)f(natural)189 2581 y(alignmen)o(t.)j(But,)f (then,)g(their)h(la)o(y)o(out)d(can)i(b)q(e)h(sp)q(eci\014ed)g(using)f Fd(MPI)p 1308 2581 V 14 w(TYPE)p 1430 2581 V 15 w(SIMPLE)p 1590 2581 V 14 w(STRUCT)p Fl(.)134 2654 y(2.)22 b(Both)d(sender)h(and)f (receiv)o(er)h(are)g(required)f(to)g(use)h Fd(MPI)p 1126 2654 V 14 w(TYPE)p 1248 2654 V 14 w(CONTIGUOUS)p 1529 2654 V 13 w(STRUCT)p Fl(,)e(and)g(the)189 2704 y(datat)o(yp)q(es)c(on)f (b)q(oth)h(ends)g(ha)o(v)o(e)g(to)f(b)q(e)h(structurally)g(equiv)n (alen)o(t,)f(i.e.,)f(de\014ned)j(b)o(y)e(the)h(same)f(sequence)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 14 16 14 15 bop 75 -100 a Fq(14)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fl(of)e(de\014nitions.)18 b(There)d(are)f(sev)o(eral)h(p)q(ossible)f(v)n(arian)o(ts)f(for)g(suc)o (h)i(prop)q(osal)189 107 y(\(a\))20 b(This)g(construct)i(can)e(b)q(e)h (only)e(used)i(in)f(a)g(homogeneous)f(en)o(vironmen)o(t.)36 b(But,)22 b(then,)g(w)o(e)e(lose)189 157 y(p)q(ortabilit)o(y)11 b(to)h(heterogeneous)j(en)o(vironmen)o(t,)c(and)i(w)o(e)g(ac)o(hiev)o (e)f(little)g(that)h(could)f(not)h(b)q(e)g(ac)o(hiev)o(ed)g(b)o(y)189 207 y(using)g Fd(MPI)p 374 207 13 2 v 15 w(BYTE)p Fl(.)189 269 y(\(b\))18 b(This)g(construct)i(can)e(b)q(e)g(used)h(p)q(ortably)m (,)f(with)g(enhanced)h(p)q(erformance)f(obtained)g(in)f(a)h(homo-)189 319 y(geneous)i(en)o(vironmen)o(t.)36 b(E.g.,)20 b(a)g(homogeneous)f (implem)o(en)o(tation)e(will)i(use)h(blo)q(c)o(k)g(cop)o(ying,)g(while) 189 369 y(a)d(heterogeneous)j(implemen)o(tatio)o(n)15 b(will)i(mo)o(v)o(e)f(elemen)o(t)h(b)o(y)h(elemen)o(t)f(\(at)h(least,)g (for)g(heterogeneous)189 419 y(comm)o(unicators\).)j(This)16 b(assumes)g(that,)g(in)f(a)h(homogeneous)e(en)o(vironmen)o(t,)h (di\013eren)o(t)i(pro)q(cesses)i(are)189 469 y(compiled)8 b(with)i(the)g(same)f(data)h(alignmen)o(t)d(options.)17 b(Also,)10 b(this)g(runs)g(coun)o(ter)h(the)g(datat)o(yp)q(e)f(matc)o (hing)189 518 y(rules)k(in)g(MPI,)f(where)i(only)e(signature)i(matc)o (hing)d(is)h(required.)189 581 y(\(c\))d Fd(MPI)p 326 581 V 14 w(TYPE)p 448 581 V 14 w(CONTIGUOUS)p 729 581 V 13 w(STRUCT)f Fl(can)g(b)q(e)i(matc)o(hed)d(with)h(other)i(datat)o (yp)q(es,)f(as)g(long)e(as)i(signa-)189 630 y(tures)g(matc)o(h;)f (impro)o(v)o(ed)f(p)q(erformance)h(is)g(ac)o(hiev)o(ed)h(when)f(b)q (oth)h(ends)g(use)g Fd(MPI)p 1463 630 V 14 w(TYPE)p 1585 630 V 15 w(CONTIGUOUS)p 1867 630 V 12 w(STRUCT)p Fl(.)189 680 y(But,)i(then,)h(the)g(wire)g(proto)q(col)f(m)o(ust)f(b)q(e)i(so)f (that)h(individual)d(elemen)o(ts)i(can)h(b)q(e)f(extracted,)i(in)e (case)h(the)189 730 y(receiv)o(er)k(do)q(es)f(not)f(use)i Fd(MPI)p 664 730 V 14 w(TYPE)p 786 730 V 14 w(CONTIGUOUS)p 1067 730 V 13 w(STRUCT)p Fl(.)d(In)i(this)f(case,)i(asking)e(the)h (sender)h(to)189 780 y(use)g Fd(MPI)p 340 780 V 14 w(TYPE)p 462 780 V 15 w(CONTIGUOUS)p 744 780 V 12 w(STRUCT)f Fl(do)q(es)h(not)g (pro)o(vide)f(additional)f(function)h(o)o(v)o(er)h(the)g(main)189 830 y(prop)q(osal.)g(The)d(di\013erence)h(is)f(only)e(whether)j(the)f (use)g(asserts)i(that)d(the)h(sending)g(datat)o(yp)q(e)f(has)h(a)f (nat-)189 879 y(ural)i(la)o(y)o(out)g(\(using)h Fd(MPI)p 608 879 V 14 w(TYPE)p 730 879 V 14 w(CONTIGUOUS)p 1011 879 V 13 w(STRUCT)p Fl(\),)f(or)h(whether)h(MPI)f(disco)o(v)o(ers)h (that)f(fact)189 929 y(b)o(y)d(itself.)75 1123 y Ff(3.9.3)49 b(Convenient)17 b(F)o(o)o(rm)d(of)j(MPI)p 731 1123 15 2 v 18 w(T)l(yp)q(e)p 848 1123 V 18 w(indexed)p 1018 1123 V 18 w(blo)q(ck)75 1256 y Fm(Curren)o(t)d(Status:)j Fl(P)o(assed)e(once.)166 1360 y Fq(This)d(prop)q(osal)g(is)g(to)f(allo) o(w)h(constan)o(t)f(blo)q(c)o(ksize)i(and)f(arbitrary)f(displacemen)o (ts)i(in)f(the)g(in)o(terface)75 1416 y(to)e Fj(MPI)p 211 1416 14 2 v 15 w(TYPE)p 343 1416 V 17 w(INDEXED)p Fq(.)g(The)g(routine)h(no)o(w)f(tak)o(es)f Fj(a)o(rra)o(y)p 1114 1416 V 14 w(of)p 1165 1416 V 16 w(blo)q(cklengths)k Fq(and)d Fj(a)o(rra)o(y)p 1605 1416 V 15 w(of)p 1657 1416 V 16 w(displacements)75 1473 y Fq(as)17 b(argumen)o(ts.)27 b(There)18 b(are)f(man)o(y)h(co)q(des)g(using)g(indirect)i(addressing)e (arising)g(from)f(unstructured)75 1529 y(grids)f(where)h(the)f(blo)q(c) o(ksize)i(is)e(alw)o(a)o(ys)f(1)h(\(gather/scatter\).)k(W)l(e)c(prop)q (ose)g(this)h(in)o(terface,)f(allo)o(wing)75 1586 y(for)f(constan)o(t)f (blo)q(c)o(ksize)j(and)e(arbitrary)g(displacemen)o(ts.)75 1737 y Fj(MPI)p 160 1737 V 16 w(TYPE)p 293 1737 V 17 w(INDEXED)p 505 1737 V 16 w(BLOCK\()g(count,)i(blo)q(cklength,)h(a)o (rra)o(y)p 1180 1737 V 14 w(of)p 1231 1737 V 16 w(displacements,)g (oldt)o(yp)q(e,)f(newt)o(yp)q(e\))117 1870 y Fl(IN)155 b Fj(count)482 b Fl(length)14 b(of)f(arra)o(y)h(of)f(displacemen)o(ts) 117 1945 y(IN)155 b Fj(blo)q(cklength)371 b Fl(size)15 b(of)e(blo)q(c)o(k)117 2021 y(IN)155 b Fj(a)o(rra)o(y)p 416 2021 V 15 w(of)p 468 2021 V 16 w(displacements)164 b Fl(arra)o(y)14 b(of)f(displacemen)o(ts)117 2096 y(IN)155 b Fj(oldt)o(yp)q(e)450 b Fl(old)13 b(datat)o(yp)q(e)117 2171 y(OUT)108 b Fj(newt)o(yp)q(e)433 b Fl(new)15 b(datat)o(yp)q(e)75 2295 y Fk(int)23 b(MPI)p 245 2295 15 2 v 17 w(Type)p 358 2295 V 17 w(Indexed)p 543 2295 V 16 w(Block\()g(int)h(count,)f(int) g(blocklength,)393 2352 y(int*)g(array)p 635 2352 V 17 w(of)p 700 2352 V 17 w(displacements,)f(MPI)p 1147 2352 V 17 w(Datatype)g(oldtype,)393 2408 y(MPI)p 468 2408 V 17 w(Datatype*)h(newtype)f(\))75 2495 y(MPI)p 150 2495 V 17 w(TYPE)p 263 2495 V 16 w(INDEXED)p 447 2495 V 17 w(BLOCK\(COUNT,)g(BLOCKLENGTH,)g(ARRAY)p 1204 2495 V 17 w(OF)p 1269 2495 V 16 w(DISPLACEMENTS,)g(OLDTYPE,)393 2551 y(NEWTYPE,)h(IERROR\))170 2608 y(INTEGER)g(COUNT,)g(INTEGER)g (BLOCKLENGTH,)g(ARRAY)p 1153 2608 V 16 w(OF)p 1217 2608 V 17 w(DISPLACEMENT\(*\),)f(OLDTYPE,)170 2664 y(NEWTYPE,)h(IERROR)-117 2700 y Fi(>)15 b Fh(\(Oct\))1967 46 y(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 15 17 15 16 bop 75 -100 a Fg(3.10.)34 b(MINOR)16 b(CORRECTIONS)1109 b Fq(15)75 45 y Fk(int)23 b(MPI::Datatype::Indexed)p 701 45 15 2 v 15 w(Block\()g(int)g(count,)g(int)h(blocklength,)393 102 y(const)f(int)h(array)p 755 102 V 16 w(of)p 819 102 V 17 w(displacements[],)393 158 y(const)f(MPI::Datatype&)f(oldtype)h (\))1875 198 y Fi(?)16 b Fh(\(Oct\))75 331 y Fn(3.10)59 b(Mino)n(r)21 b(Co)n(rrections)75 480 y Fm(Curren)o(t)14 b(Status:)j Fl(P)o(assed)e(once.)166 584 y Fq(The)i(follo)o(wing)i (corrections)e(to)g Fj(MPI-1.1)f Fq(are)h(\(all)h(page)f(and)g(line)i (n)o(um)o(b)q(ers)f(are)f(for)g(the)g(June)75 640 y(12,)d(1995)g(v)o (ersion)i(without)f(c)o(hangebars\):)143 746 y Fi(\017)23 b Fq(P)o(age)14 b(19,)g(lines)j(1-2)e(reads)189 803 y(for)f(\(64)g (bit\))i(C)f(in)o(tegers)g(declared)i(to)d(b)q(e)i(of)f(t)o(yp)q(e)g Fj(longlong)g(int)189 859 y Fq(but)g(should)h(read)189 916 y(for)e(\(64)g(bit\))i(C)f(in)o(tegers)g(declared)i(to)d(b)q(e)i (of)f(t)o(yp)q(e)g Fj(long)g(long)g(int)189 1088 y Fm(Discussion)o(:) 189 1156 y Fl(Just)f(to)g(mak)o(e)f(it)g(correct)j(C)d(so)h(it)g(is)g (long)f(long.)1875 1212 y Fi(>)j Fh(\(Oct\))143 1297 y Fi(\017)23 b Fq(P)o(age)14 b(41,)g(lines)j(16-18)d(reads)189 1354 y(A)19 b Fb(empt)o(y)g Fq(status)g(is)h(a)f(status)g(whic)o(h)i (is)f(set)f(to)g(return)h Fj(tag)g(=)g(MPI)p 1460 1354 14 2 v 16 w(ANY)p 1568 1354 V 17 w(T)l(A)o(G)p Fq(,)f Fj(source)i(=)189 1410 y(MPI)p 274 1410 V 15 w(ANY)p 381 1410 V 18 w(SOURCE)p Fq(,)11 b(and)g(is)h(also)e(in)o(ternally)j (con\014gured)e(so)g(that)f(calls)h(to)g Fj(MPI)p 1601 1410 V 15 w(GET)p 1704 1410 V 17 w(COUNT)189 1467 y Fq(and)k Fj(MPI)p 362 1467 V 16 w(GET)p 466 1467 V 17 w(ELEMENTS)g Fq(return)g Fj(count)i(=)e(0)p Fq(.)189 1523 y(but)g(should)h(read)189 1580 y(A)j Fb(empt)o(y)g Fq(status)g(is)h(a)f(status)g(whic)o(h)i(is)f (set)f(to)g(return)h Fj(tag)g(=)g(MPI)p 1460 1580 V 16 w(ANY)p 1568 1580 V 17 w(T)l(A)o(G)p Fq(,)f Fj(source)i(=)189 1636 y(MPI)p 274 1636 V 15 w(ANY)p 381 1636 V 18 w(SOURCE)p Fq(,)c Fj(erro)o(r)e(=)i(MPI)p 842 1636 V 16 w(SUCCESS)p Fq(,)h(and)f(is)g(also)g(in)o(ternally)h(con\014gured)f(so)g(that)189 1693 y(calls)f(to)e Fj(MPI)p 430 1693 V 16 w(GET)p 534 1693 V 17 w(COUNT)i Fq(and)f Fj(MPI)p 893 1693 V 16 w(GET)p 997 1693 V 17 w(ELEMENTS)g Fq(return)g Fj(count)i(=)e(0)p Fq(.)189 1865 y Fm(Discussion)o(:)189 1933 y Fl(Filling)d(in)h(the)h (error)h(\014eld)f(got)g(forgotten.)1875 1989 y Fi(?)i Fh(\(Oct\))143 2074 y Fi(\017)23 b Fq(P)o(age)14 b(71,)g(line)j(10)e (reads)189 2131 y(and)g(do)g(not)g(a\013ect)f(the)i(the)f(con)o(ten)o (t)g(of)f(a)h(message)189 2187 y(but)g(should)h(read)189 2244 y(and)f(do)g(not)g(a\013ect)f(the)i(con)o(ten)o(t)e(of)h(a)g (message)189 2416 y Fm(Discussion)o(:)189 2484 y Fl(This)e(is)h(a)g (trivial)e(t)o(yp)q(o)i(of)f(ha)o(ving)g(the)h(2)g(times.)143 2626 y Fi(\017)23 b Fq(P)o(age)14 b(74,)g(lines)j(39-45)d(read)189 2682 y(A)i(datat)o(yp)q(e)f(ma)o(y)g(sp)q(ecify)i(o)o(v)o(erlapping)g (en)o(tries.)22 b(The)16 b(use)g(of)g(suc)o(h)g(a)f(datat)o(yp)q(e)h (in)g(a)g(receiv)o(e)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 16 18 16 17 bop 75 -100 a Fq(16)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fq(op)q(eration)f(is)h (erroneous.)k(\(This)14 b(is)h(erroneous)f(ev)o(en)g(if)h(the)f(actual) g(message)g(receiv)o(ed)h(is)g(short)189 102 y(enough)g(not)g(to)g (write)g(an)o(y)g(en)o(try)g(more)f(than)h(once.\))189 174 y(A)i(datat)o(yp)q(e)g(ma)o(y)g(sp)q(ecify)i(o)o(v)o(erlapping)g (en)o(tries.)27 b(If)18 b(suc)o(h)g(a)g(datat)o(yp)q(e)f(is)h(used)g (in)h(a)e(receiv)o(e)189 231 y(op)q(eration,)d(that)h(is,)g(if)g(some)g (part)f(of)h(the)f(receiv)o(e)i(bu\013er)f(is)h(written)f(more)f(than)h (once)g(b)o(y)g(the)189 287 y(receiv)o(e)h(op)q(eration,)f(then)g(the)h (call)g(is)g(erroneous.)189 359 y(The)i(\014rst)f(part)g(w)o(as)h(an)f Fj(MPI-1.1)g Fq(addition.)29 b(The)18 b(second)g(part)f(o)o(v)o(erlaps) h(with)g(it.)28 b(The)18 b(old)189 416 y(text)c(will)j(b)q(e)f(remo)o (v)o(ed)f(so)g(it)g(no)o(w)g(reads)189 472 y(A)h(datat)o(yp)q(e)f(ma)o (y)g(sp)q(ecify)i(o)o(v)o(erlapping)g(en)o(tries.)22 b(The)16 b(use)g(of)g(suc)o(h)g(a)f(datat)o(yp)q(e)h(in)g(a)g(receiv)o (e)189 529 y(op)q(eration)e(is)h(erroneous.)k(\(This)14 b(is)h(erroneous)f(ev)o(en)g(if)h(the)f(actual)g(message)g(receiv)o(ed) h(is)g(short)189 585 y(enough)g(not)g(to)g(write)g(an)o(y)g(en)o(try)g (more)f(than)h(once.\))189 698 y Fm(Discussion)o(:)189 764 y Fl(The)f(in)o(ten)o(t)h(is)f(to)g(remo)o(v)o(e)f(duplicate)h (text)h(whic)o(h)f(seems)h(to)f(ha)o(v)o(e)g(b)q(een)h(left.)k(No)14 b(c)o(hange)h(in)f(meaning)189 814 y(is)f(in)o(tended.)143 950 y Fi(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(3)e(reads)189 1006 y(MPI)p 281 1006 14 2 v 16 w(P)o(ac)o(k)p 393 1006 V 16 w(size\(coun)o(t,)g(MPI)p 724 1006 V 16 w(CHAR,)h(&k2\);)189 1063 y(but)f(should)h(read)189 1119 y(MPI)p 281 1119 V 16 w(P)o(ac)o(k)p 393 1119 V 16 w(size\(coun)o(t,)f(MPI)p 724 1119 V 16 w(CHAR,)h(comm,)e(&k2\);)189 1289 y Fm(Discussion)o(:)189 1355 y Fl(This)f(is)h(a)g(minor)e(\014x)i(to)f(an)h(example)f(for)g(a)h (missing)e(argumen)o(t.)143 1490 y Fi(\017)23 b Fq(P)o(age)14 b(90,)g(line)j(10)e(reads)189 1547 y(MPI)p 281 1547 V 16 w(P)o(ac)o(k\(c)o(hr,)f(coun)o(t,)g(MPI)p 726 1547 V 17 w(CHAR,)h(&lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 1603 y(but)f(should)h(read)189 1660 y(MPI)p 281 1660 V 16 w(P)o(ac)o(k\(c)o(hr,)e(coun)o(t,)g(MPI)p 726 1660 V 17 w(CHAR,)h(lbuf,)h(k,)f(&p)q(osition,)h(comm\);)189 1829 y Fm(Discussion)o(:)189 1895 y Fl(This)e(is)h(a)f(minor)f(\014x)i (to)g(an)f(example)g(for)g(an)g(argumen)o(t)g(that)h(is)g(declared)g (to)g(b)q(e)g(a)g(p)q(oin)o(ter)g(and)f(then)189 1945 y(passed)h(with)e(&.)143 2081 y Fi(\017)23 b Fq(P)o(age)14 b(109,)g(lines)j(26/27)d(and)h(page)g(110,)f(lines)j(28/29)d(reads)189 2137 y(The)h Fk(j)p Fq(th)g(blo)q(c)o(k)g(of)g(data)f(sen)o(t)h(from)f (eac)o(h)h(pro)q(cess)g(is)h(receiv)o(ed)g(b)o(y)f(ev)o(ery)f(pro)q (cess)i(and)f(placed)189 2194 y(in)h(the)f Fk(j)p Fq(th)g(blo)q(c)o(k)h (of)f(the)g(bu\013er)g Fj(recvbuf)p Fq(.)189 2250 y(but)g(should)h (read)189 2307 y(The)g(blo)q(c)o(k)i(of)e(data)g(sen)o(t)g(from)g(the)g Fk(j)p Fq(th)h(pro)q(cess)g(is)g(receiv)o(ed)g(b)o(y)g(ev)o(ery)f(pro)q (cess)h(and)g(placed)189 2363 y(in)f(the)f Fk(j)p Fq(th)g(blo)q(c)o(k)h (of)f(the)g(bu\013er)g Fj(recvbuf)p Fq(.)189 2533 y Fm(Discussion)o(:) 189 2598 y Fl(This)e(is)g(related)h(to)f(a)f(subtle)i(p)q(oin)o(t)f(in) g(usage.)18 b(The)13 b(sendbuf)h(is)f(a)g(single)g(item)f(whereas)i (the)g(recvbuf)g(is)189 2648 y(an)f(\\arra)o(y")g(of)h(items.)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 17 19 17 18 bop 75 -100 a Fg(3.10.)34 b(MINOR)16 b(CORRECTIONS)1109 b Fq(17)143 45 y Fi(\017)23 b Fq(P)o(age)14 b(117,)g(lines)j(22/23)d (reads)189 102 y(MPI)h(pro)o(vides)g(sev)o(en)h(suc)o(h)f(prede\014ned) i(datat)o(yp)q(es.)189 158 y(but)e(should)h(read)189 214 y(MPI)f(pro)o(vides)g(nine)i(suc)o(h)e(prede\014ned)i(datat)o(yp)q (es.)189 442 y Fm(Discussion)o(:)189 509 y Fl(This)c(is)h(an)g(ob)o (vious)f(mistak)o(e.)k(There)e(are)f(nine)g(listed)g(on)f(line)h (28-40.)143 647 y Fi(\017)23 b Fq(P)o(age)14 b(122,)g(lines)j(35-36)d (read)204 704 y Fj(MPI)p 289 704 14 2 v 16 w(OP)p 367 704 V 16 w(FREE\()i(op\))230 893 y Fl(IN)156 b Fj(op)541 b Fl(op)q(eration)14 b(\(handle\))189 1035 y Fq(but)h(should)h(read)204 1091 y Fj(MPI)p 289 1091 V 16 w(OP)p 367 1091 V 16 w(FREE\()g(op\))230 1280 y Fl(INOUT)63 b Fj(op)541 b Fl(op)q(eration)14 b(\(handle\))189 1463 y Fm(Discussion)o(:)189 1530 y Fl(This)j(app)q(ears)g(to)g(ha)o(v) o(e)g(b)q(een)h(a)f(mistak)o(e.)26 b(This)17 b(function)g(frees)h(the)g (function)e(p)q(oin)o(ter)i(and)f(sets)h(its)189 1580 y(v)n(alue)10 b(to)h(MPI)p 424 1580 13 2 v 15 w(OP)p 499 1580 V 15 w(NULL.)g(Th)o(us,)g(it)g(is)g(b)q(oth)g(input)g(and)f (output)i(as)f(are)g(the)g(free)h(routines)g(for)e(Groups,)189 1629 y(Comms,)g(etc.)143 1768 y Fi(\017)23 b Fq(P)o(age)14 b(125,)g(line)j(1)e(reads)189 1824 y(CALL)h(MPI)p 420 1824 14 2 v 16 w(ALLREDUCE\(sum,)f(c,)g(n,)g(MPI)p 1040 1824 V 17 w(REAL,)g(MPI)p 1300 1824 V 17 w(SUM,)g(0,)f(comm,)g(ierr\)) 189 1881 y(but)h(should)h(read)189 1937 y(CALL)g(MPI)p 420 1937 V 16 w(ALLREDUCE\(sum,)f(c,)g(n,)g(MPI)p 1040 1937 V 17 w(REAL,)g(MPI)p 1300 1937 V 17 w(SUM,)g(comm,)f(ierr\))189 2108 y Fm(Discussion)o(:)189 2175 y Fl(The)g(extra)h(argumen)o(t)e(app) q(ears)j(to)e(b)q(e)h(left)f(o)o(v)o(er)g(from)f(the)i(previous)f (example)f(with)h(MPI)p 1670 2175 13 2 v 16 w(REDUCE.)189 2225 y(This)f(is)h(a)g(mistak)o(e.)143 2363 y Fi(\017)23 b Fq(P)o(age)14 b(194,)g(line)j(48)e(reads)189 2420 y Fj(MPI)p 274 2420 14 2 v 15 w(ERRHANDLER)p 582 2420 V 18 w(CREA)l(TE\(FUNCTION,)h(HANDLER,)g(IERROR\))189 2476 y Fq(but)f(should)h(read)189 2533 y Fj(MPI)p 274 2533 V 15 w(ERRHANDLER)p 582 2533 V 18 w(CREA)l(TE\(FUNCTION,)g(ERRHANDLER,) h(IERROR\))189 2704 y Fm(Discussion)o(:)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 18 20 18 19 bop 75 -100 a Fq(18)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)189 45 y Fl(This)e(is)h(a)g(small)d(t)o (yp)q(o)j(to)g(matc)o(h)f(all)f(the)j(other)f(names)f(for)h(this)g (argumen)o(t.)143 186 y Fi(\017)23 b Fq(P)o(age)14 b(196,)g(lines)j (1-2)e(reads)204 243 y Fj(MPI)p 289 243 14 2 v 16 w(ERRHANDLER)p 598 243 V 17 w(FREE\()h(errhandler)f(\))230 433 y Fl(IN)156 b Fj(errhandler)397 b Fl(MPI)14 b(error)h(handler)f(\(handle\))189 576 y Fq(but)h(should)h(read)204 633 y Fj(MPI)p 289 633 V 16 w(ERRHANDLER)p 598 633 V 17 w(FREE\()g(errhandler)f(\))230 823 y Fl(INOUT)63 b Fj(errhandler)397 b Fl(MPI)14 b(error)h(handler)f (\(handle\))189 1007 y Fm(Discussion)o(:)189 1075 y Fl(This)c(app)q (ears)h(to)f(ha)o(v)o(e)h(b)q(een)g(a)f(mistak)o(e.)16 b(This)10 b(function)g(frees)i(the)f(function)f(p)q(oin)o(ter)g(and)h (sets)g(its)g(v)n(alue)189 1125 y(to)h(MPI)p 321 1125 13 2 v 15 w(ERRHANDLER)p 636 1125 V 14 w(NULL.)g(Th)o(us,)h(it)f(is)g (b)q(oth)h(input)f(and)g(output)h(as)f(are)h(the)g(free)g(routines)g (for)189 1175 y(Groups,)g(Comms,)e(etc.)143 1316 y Fi(\017)23 b Fq(P)o(age)14 b(203,)g(line)j(1)e(reads)189 1372 y Fj(MPI)p 274 1372 14 2 v 15 w(PCONTROL\(level\))189 1429 y Fq(but)g(should)h(read)189 1485 y Fj(MPI)p 274 1485 V 15 w(PCONTROL\(LEVEL\))189 1657 y Fm(Discussion)o(:)189 1726 y Fl(This)d(is)h(a)g(minor)e(t)o(yp)q(o.)143 1867 y Fi(\017)23 b Fq(P)o(age)14 b(210,)g(line)j(44)e(reads)189 1923 y Fd(MPI)p 266 1923 13 2 v 14 w(PENDING)189 1980 y Fq(but)g(should)h(read)189 2036 y Fd(MPI)p 266 2036 V 14 w(ERR)p 359 2036 V 14 w(PENDING)189 2209 y Fm(Discussion)o(:)189 2277 y Fl(This)d(is)h(a)g(t)o(yp)q(o.)k(See)c(page)g(197,)f(line)g(20)h (for)f(the)i(original)d(de\014nition.)143 2418 y Fi(\017)23 b Fq(P)o(age)14 b(211,)g(line)j(44)e(reads)189 2475 y Fd(MPI)p 266 2475 V 14 w(DOUBLE)p 445 2475 V 14 w(COMPLEX)189 2531 y Fq(but)g(should)h(b)q(e)g(mo)o(v)o(ed)f(to)f(P)o(age)h(212,)f (line)j(22)d(since)j(it)e(is)h(an)f(optional)h(F)l(ortran)e(datat)o(yp) q(e.)189 2647 y Fm(Discussion)o(:)1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 19 21 19 20 bop 75 -100 a Fg(3.10.)34 b(MINOR)16 b(CORRECTIONS)1109 b Fq(19)189 45 y Fl(On)20 b(page)h(19,)f(lines)h(2-3,)f(it)g(clearly)h (states)g(this)f(is)h(an)f(optional)f(F)m(ortran)h(datat)o(yp)q(e.)37 b(It)21 b(is)f(listed)189 95 y(incorrectly)14 b(in)g(the)g(app)q (endix.)1875 150 y Fi(>)i Fh(\(Oct\))143 236 y Fi(\017)23 b Fq(P)o(age)14 b(212,)g(add)h(new)h(lines)h(of)d(text)h(at)g(line)i (22)d(and)i(line)g(25)f(to)g(read:)189 292 y(etc.)189 349 y(Th)o(us,)f(the)i(text)e(will)j(no)o(w)e(read:)189 518 y Fk(/*)23 b(optional)g(datatypes)g(\(Fortran\))f(*/)189 574 y(MPI_INTEGER1)189 631 y(MPI_INTEGER2)189 687 y(MPI_INTEGER4)189 744 y(MPI_REAL2)189 800 y(MPI_REAL4)189 857 y(MPI_REAL8)189 913 y(etc.)189 1026 y(/*)h(optional)g(datatypes)g(\(C\))g(*/)189 1082 y(MPI_LONG_LONG_INT)189 1139 y(etc.)189 1292 y Fm(Discussion)o(:) 189 1360 y Fl(On)15 b(page)g(19,)f(line)g(6,)h(concludes)h(the)f(list)f (of)h(optional)e(datat)o(yp)q(es)j(with)e(etc.)22 b(It)15 b(w)o(as)g(not)g(in)o(tended)g(to)189 1410 y(b)q(e)f(a)g(complete)g (list)g(of)f(all)g(p)q(ossible)i(ones.)20 b(Some)13 b(ha)o(v)o(e)h(in)o (terpreted)i(the)e(app)q(endix)h(to)f(b)q(e)h(a)f(complete)189 1460 y(list)f(since)h(only)f(the)h(ones)h(explictly)e(on)g(page)h(19)f (are)h(listed.)k(Adding)13 b(etc.,)h(to)g(the)g(F)m(ortran)f(and)h(C)f (list)189 1510 y(in)g(the)i(app)q(endix)e(will)g(mak)o(e)f(it)i(clear)g (these)h(are)g(only)e(some)g(of)g(the)i(p)q(ossible)f(optional)e(datat) o(yp)q(es.)1875 1565 y Fi(?)k Fh(\(Oct\))143 1651 y Fi(\017)23 b Fq(P)o(age)14 b(213,)g(line)j(28.)i(The)d(follo)o(wing)g(text)e (should)i(b)q(e)g(added:)189 1707 y(/*)9 b(Prede\014ned)j(functions)f (in)g(C)e(and)i(F)l(ortran)e(*/)g(MPI)p 1137 1707 14 2 v 17 w(NULL)p 1278 1707 V 17 w(COPY)p 1428 1707 V 17 w(FN)g(MPI)p 1607 1707 V 17 w(NULL)p 1748 1707 V 17 w(DELETE)p 1954 1707 V 16 w(FN)189 1764 y(MPI)p 281 1764 V 16 w(DUP)p 397 1764 V 16 w(FN)189 1880 y Fm(Discussion)o(:)189 1948 y Fl(These)15 b(are)f(missing)e(and)i(de\014ned)h(on)e(pages)i(169,)d (line)i(17)f(and)h(p.)k(170,)13 b(line)g(4.)143 2089 y Fi(\017)23 b Fq(P)o(age)14 b(220,)g(lines)j(19-20)d(reads)189 2146 y Fj(int)i(double)g(MPI)p 479 2146 V 16 w(Wtime\(void\))189 2202 y(int)g(double)g(MPI)p 479 2202 V 16 w(Wtick\(void\))189 2258 y Fq(but)f(should)h(read)189 2315 y Fj(double)g(MPI)p 413 2315 V 16 w(Wtime\(void\))189 2371 y(double)g(MPI)p 413 2371 V 16 w(Wtick\(void\))189 2544 y Fm(Discussion)o(:)189 2612 y Fl(This)d(w)o(as)h(a)g(mistak)o(e)e(caused)j(b)o(y)f(the)g (macros.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 20 22 20 21 bop 75 -100 a Fq(20)947 b Fg(CHAPTER)15 b(3.)35 b(MISCELLANY)17 b(F)o(OR)e(1.2)143 45 y Fi(\017)23 b Fq(P)o(age)14 b(222,)g(line)j(34)e(reads)189 102 y Fj(INTEGER)f (REQUEST,)g(COUNT,)g(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)g(T)l(A)o(G,)f (COMM,)g(REQUEST,)i(IERROR)189 158 y Fq(but)g(should)h(read)189 214 y Fj(INTEGER)f(COUNT,)h(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)f(T)l(A)o(G,) g(COMM,)g(REQUEST,)h(IERROR)189 387 y Fm(Discussion)o(:)189 455 y Fl(An)e(extra)g(argumen)o(t)f(got)g(added.)19 b(It)14 b(needs)h(to)f(b)q(e)g(remo)o(v)o(ed.)143 596 y Fi(\017)23 b Fq(P)o(age)14 b(222,)g(line)j(38)e(reads)189 653 y Fj(INTEGER)f(REQUEST,)g(COUNT,)g(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)g(T)l(A) o(G,)f(COMM,)g(REQUEST,)i(IERROR)189 709 y Fq(but)g(should)h(read)189 766 y Fj(INTEGER)f(COUNT,)h(D)o(A)l(T)l(A)l(TYPE,)g(DEST,)f(T)l(A)o(G,) g(COMM,)g(REQUEST,)h(IERROR)189 938 y Fm(Discussion)o(:)189 1006 y Fl(An)e(extra)g(argumen)o(t)f(got)g(added.)19 b(It)14 b(needs)h(to)f(b)q(e)g(remo)o(v)o(ed.)143 1147 y Fi(\017)23 b Fq(P)o(age)14 b(227,)g(lines)j(19-20)d(reads)189 1204 y Fj(MPI)p 274 1204 14 2 v 15 w(INTERCOMM)p 563 1204 V 17 w(MERGE\(INTERCOMM,)i(HIGH,)f(INTRA)o(COMM,)g(IERROR\))189 1260 y(INTEGER)g(INTERCOMM,)h(INTRA)o(COMM,)f(IERROR)189 1317 y Fq(but)g(should)h(read)189 1373 y Fj(MPI)p 274 1373 V 15 w(INTERCOMM)p 563 1373 V 17 w(MERGE\(INTERCOMM,)g(HIGH,)f (NEWINTRA)o(COMM,)g(IERROR\))189 1430 y(INTEGER)g(INTERCOMM,)h (NEWINTRA)o(COMM,)f(IERROR)189 1602 y Fm(Discussion)o(:)189 1670 y Fl(This)g(mak)o(es)e(it)i(matc)o(h)f(the)i(de\014nition)e(in)h (the)g(text)h(on)f(P)o(age)g(159,)f(lines)h(34-36.)20 b(It)15 b(only)g(c)o(hanges)g(the)189 1720 y(name)d(of)i(the)g(argumen) o(t.)143 1861 y Fi(\017)23 b Fq(P)o(age)14 b(228,)g(line)j(46)e(reads) 189 1918 y Fj(MPI)p 274 1918 V 15 w(ERRHANDLER)p 582 1918 V 18 w(CREA)l(TE\(FUNCTION,)h(HANDLER,)g(IERROR\))189 1974 y Fq(but)f(should)h(read)189 2031 y Fj(MPI)p 274 2031 V 15 w(ERRHANDLER)p 582 2031 V 18 w(CREA)l(TE\(FUNCTION,)g (ERRHANDLER,)h(IERROR\))189 2203 y Fm(Discussion)o(:)189 2271 y Fl(This)c(c)o(hange)i(mak)o(es)d(the)j(app)q(endix)f(consistan)o (t)g(with)g(c)o(hange)g(on)g(p.)k(194,)12 b(line)i(48.)143 2412 y Fi(\017)23 b Fq(P)o(age)14 b(229,)g(line)j(33)e(reads)189 2469 y Fj(MPI)p 274 2469 V 15 w(PCONTROL\(level\))189 2525 y Fq(but)g(should)h(read)189 2582 y Fj(MPI)p 274 2582 V 15 w(PCONTROL\(LEVEL\))1967 46 y Fh(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 21 23 21 22 bop 75 -100 a Fg(3.10.)34 b(MINOR)16 b(CORRECTIONS)1109 b Fq(21)189 45 y Fm(Discussion)o(:)189 114 y Fl(This)13 b(is)h(to)g(mak)o(e)e(the)j(language)e(binding)g(the)h(same)f(as)h(the) h(c)o(hange)f(to)g(P)o(age)f(203,)g(line)g(1.)-32 46 y Fh(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Tue Sep 24 06:45:25 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA20101; Tue, 24 Sep 1996 06:45:24 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA03136 for mpi-core-out; Tue, 24 Sep 1996 05:48:09 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id FAA03131 for ; Tue, 24 Sep 1996 05:48:01 -0500 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id DAA27528 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Tue, 24 Sep 1996 03:47:58 -0700 Received: from mrjones.engr.sgi.com (mrjones.engr.sgi.com [150.166.49.18]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id DAA13042 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Tue, 24 Sep 1996 03:47:58 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id DAA04817; Tue, 24 Sep 1996 03:44:27 -0700 From: "Eric Salo" Message-Id: <9609240344.ZM4815@mrjones.engr.sgi.com> Date: Tue, 24 Sep 1996 03:44:26 -0700 In-Reply-To: Tony Skjellum "JOD as separate document" (Sep 18, 10:29am) References: X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: JOD as separate document Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I make the following proposal. Since the JOD document is the core of MPI-3, > and since it may need chapter stratification, I suggest that it be separated > from MPI-2 document, and be physically attached, rather than being jointly > managed. I think that it would be a *very* big mistake to say anything at all about the JOD with respect to MPI-3. In fact I think that even mentioning MPI-3 at this point should be grounds for...I dunno, but something very unpleasant! MPI-2 is already so large and so complex that we can't even get firm commitments to implement all or even most of it from the current MPI-1 implementors. The JOD is for things that have been taken *out* of MPI, and I am personally opposed to even including it as part of the SC'96 release. So I favor anything that puts distance between it and the "real" MPI chapters. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Tue Sep 24 09:40:28 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA25241; Tue, 24 Sep 1996 09:40:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA05203 for mpi-core-out; Tue, 24 Sep 1996 08:42:41 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA05194 for ; Tue, 24 Sep 1996 08:42:28 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id IAA11533; Tue, 24 Sep 1996 08:32:35 -0500 Date: Tue, 24 Sep 1996 08:32:35 -0500 (CDT) From: Tony Skjellum To: Eric Salo cc: mpi-core@mcs.anl.gov Subject: Re: JOD as separate document In-Reply-To: <9609240344.ZM4815@mrjones.engr.sgi.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The RT chapter has chosen to be in JOD on purpose, and the tone of Eric's note is that JOD is a pejorative. In many cases, being in JOD just means we are not ready, or it is wiser to think more than stamp "standard by SC'96" on the material... -Tony On Tue, 24 Sep 1996, Eric Salo wrote: > Date: Tue, 24 Sep 1996 03:44:26 -0700 > From: Eric Salo > To: mpi-core@mcs.anl.gov > Subject: Re: JOD as separate document > > > I make the following proposal. Since the JOD document is the core of MPI-3, > > and since it may need chapter stratification, I suggest that it be separated > > from MPI-2 document, and be physically attached, rather than being jointly > > managed. > > I think that it would be a *very* big mistake to say anything at all about the > JOD with respect to MPI-3. In fact I think that even mentioning MPI-3 at this > point should be grounds for...I dunno, but something very unpleasant! MPI-2 is > already so large and so complex that we can't even get firm commitments to > implement all or even most of it from the current MPI-1 implementors. > > The JOD is for things that have been taken *out* of MPI, and I am personally > opposed to even including it as part of the SC'96 release. So I favor anything > that puts distance between it and the "real" MPI chapters. > > -- > Eric Salo Silicon Graphics Inc. "Do you know what the > (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just > salo@sgi.com Mountain View, CA 94043-1389 before he died?" > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Tue Sep 24 10:13:12 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA25884; Tue, 24 Sep 1996 10:13:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA05988 for mpi-core-out; Tue, 24 Sep 1996 09:16:03 -0500 Received: from mbunix.mitre.org (mbunix.mitre.org [129.83.20.100]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA05699 for ; Tue, 24 Sep 1996 09:05:38 -0500 Received: from TGATE3 (tgate3.mitre.org [129.83.20.27]) by mbunix.mitre.org (8.7.6/8.7.3/mitre.0) with SMTP id KAA08286; Tue, 24 Sep 1996 10:01:49 -0400 (EDT) Received: from mail11 (129.83.20.44) by tgate3.mitre.org (EMWAC SMTPRS 0.60) with SMTP id ; Tue, 24 Sep 1996 09:53:35 -0400 Received: by mail11; (5.65v3.2/1.1.8.2/22Jun94-0628PM) id AA25482; Tue, 24 Sep 1996 09:59:23 -0400 Subject: Re: JOD as separate document From: arkady@mail11.mitre.org (Arkady Kanevsky) To: salo@mrjones.engr.sgi.com (Eric Salo), mpi-core@mcs.anl.gov Message-Id: <960924095919.7408@mail11.mitre.org.0> Date: Tue, 24 Sep 96 09:59:22 -0400 X-Mailer: MAILworks 1.7-A-2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Eric, I am in favor of removing JoD from the final MPI-2 document, but I strongly suggest that we distribute it for SC'96. The goal of SC'96 is to give a clear picture to all interested parties, that includes ongoing work and work that we do not want all implementation to have, but that can be added by individual implementors for added benefit for users... It is true that we should not create MPI-3 after MPI-2 right away and let user to get familiar with MPI-2 features... However, real-time MPI will continue to work on its chapter(s) since it is not required for all MPI implementations. I invision that MPI I/O will continue its work if it will not be approved as part of MPI-2. Sincerely yours, Arkady >> I make the following proposal. Since the JOD document is the core of MPI-3, >> and since it may need chapter stratification, I suggest that it be separated >> from MPI-2 document, and be physically attached, rather than being jointly >> managed. > >I think that it would be a *very* big mistake to say anything at all about the >JOD with respect to MPI-3. In fact I think that even mentioning MPI-3 at this >point should be grounds for...I dunno, but something very unpleasant! MPI-2 is >already so large and so complex that we can't even get firm commitments to >implement all or even most of it from the current MPI-1 implementors. > >The JOD is for things that have been taken *out* of MPI, and I am personally >opposed to even including it as part of the SC'96 release. So I favor anything >that puts distance between it and the "real" MPI chapters. > >-- >Eric Salo Silicon Graphics Inc. "Do you know what the >(415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just >salo@sgi.com Mountain View, CA 94043-1389 before he died?" > > From mpi-core-human@mcs.anl.gov Tue Sep 24 11:35:05 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA27030; Tue, 24 Sep 1996 11:35:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA08268 for mpi-core-out; Tue, 24 Sep 1996 10:37:01 -0500 Received: from hp.com (hp.com [15.255.152.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA08262 for ; Tue, 24 Sep 1996 10:36:53 -0500 Received: from tbag.rsn.hp.com by hp.com with ESMTP (1.37.109.16/15.5+ECS 3.3) id AA122509410; Tue, 24 Sep 1996 08:36:51 -0700 Message-Id: <199609241536.AA122509410@hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA297189409; Tue, 24 Sep 1996 10:36:49 -0500 From: Raja Daoud Subject: Re: JOD as separate document To: mpi-core@mcs.anl.gov Date: Tue, 24 Sep 1996 10:36:49 CDT In-Reply-To: ; from "Tony Skjellum" at Sep 24, 96 8:32 am X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > The RT chapter has chosen to be in JOD on purpose, and the tone of Eric's > note is that JOD is a pejorative. In many cases, being in JOD just means > we are not ready, or it is wiser to think more than stamp "standard by > SC'96" on the material... I think the main point is that the JOD won't be a standard by spring 97 when the MPI-2 standard is officially released. Since the public comment period is meant to get feedback on the standard, it stands to reason that we ought to use it optimally and focus the attention of "our customers" on the MPI 1.2 and 2.0 documents and not dilute their effort by asking them to also read and comment on the JOD. So I propose that we not release the JOD at SC'96, but release it instead in the spring as a separate document along with the official standard. --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com Convex Division From mpi-core-human@mcs.anl.gov Tue Sep 24 11:43:55 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA27138; Tue, 24 Sep 1996 11:43:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA08603 for mpi-core-out; Tue, 24 Sep 1996 10:47:02 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA08597; Tue, 24 Sep 1996 10:46:53 -0500 Message-Id: <199609241546.KAA08597@antares.mcs.anl.gov> To: Raja Daoud cc: mpi-core@mcs.anl.gov Subject: Re: JOD as separate document In-reply-to: Your message of "Tue, 24 Sep 1996 10:36:49 CDT." <199609241536.AA122509410@hp.com> Date: Tue, 24 Sep 1996 10:46:51 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | | So I propose that we not release the JOD at SC'96, I disagree. In MPI-1, one thing that happened during the public comment period was that potential users strongly requested the inclusion of some things that we had discussed and decided not to include. I think that the JOD now contains some things that we have put a lot of work on but have set aside. If the SC'96 "release" is to be a snapshot of our work, then the public should see this work as well. If they want independent processes, for example, then they should see what our thoughts were on what it would look like. If they want shared buffers, they should see a writeup on them. I hope to be able to make it clear in the introduction, once we decide at the next meeting the shape of the SC version, just what is what. Rusty From mpi-core-human@mcs.anl.gov Tue Sep 24 12:06:54 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA27278; Tue, 24 Sep 1996 12:06:48 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA09340 for mpi-core-out; Tue, 24 Sep 1996 11:09:52 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA09327 for ; Tue, 24 Sep 1996 11:09:39 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id LAA12453; Tue, 24 Sep 1996 11:00:37 -0500 Date: Tue, 24 Sep 1996 11:00:37 -0500 (CDT) From: Tony Skjellum To: Raja Daoud cc: mpi-core@mcs.anl.gov Subject: Re: JOD as separate document In-Reply-To: <199609241536.AA122509410@hp.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk We should obviously not put everything we don't want into JOD. We should not have a pejorative view of it. I am not happy to see RT in JOD if this is an indicator that the forum will say that it went to the junk heap because it is in JOD. Whether with sanction of the forum or not, RT is ready to be shown around to realtime community by SC'96 timeframe. It will be shown around... That is different from collective. People have expressed reservations about collective and the majority of its functions are going to JOD to provide continuity of thought when and if we should do MPI-3... -Tony On Tue, 24 Sep 1996, Raja Daoud wrote: > Date: Tue, 24 Sep 1996 10:36:49 CDT > From: Raja Daoud > To: mpi-core@mcs.anl.gov > Subject: Re: JOD as separate document > > > The RT chapter has chosen to be in JOD on purpose, and the tone of Eric's > > note is that JOD is a pejorative. In many cases, being in JOD just means > > we are not ready, or it is wiser to think more than stamp "standard by > > SC'96" on the material... > > I think the main point is that the JOD won't be a standard by spring 97 > when the MPI-2 standard is officially released. Since the public comment > period is meant to get feedback on the standard, it stands to reason that > we ought to use it optimally and focus the attention of "our customers" > on the MPI 1.2 and 2.0 documents and not dilute their effort by asking > them to also read and comment on the JOD. > > So I propose that we not release the JOD at SC'96, but release it instead > in the spring as a separate document along with the official standard. > > --Raja > > -=- > Raja Daoud Hewlett-Packard Co. > raja@rsn.hp.com Convex Division > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Wed Sep 25 13:21:26 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA14553; Wed, 25 Sep 1996 13:21:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA03175 for mpi-core-out; Wed, 25 Sep 1996 12:18:50 -0500 Received: from hpcsos.col.hp.com (hpcsos.col.hp.com [15.255.240.16]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA03141 for ; Wed, 25 Sep 1996 12:17:18 -0500 Received: from tbag.rsn.hp.com by hpcsos.col.hp.com with ESMTP (1.37.109.16/15.5+IOS 3.14) id AA184761829; Wed, 25 Sep 1996 11:17:09 -0600 Message-Id: <199609251717.AA184761829@hpcsos.col.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA045811753; Wed, 25 Sep 1996 12:15:53 -0500 From: Raja Daoud Subject: Re: JOD as separate document To: mpi-core@mcs.anl.gov Date: Wed, 25 Sep 1996 12:15:53 CDT In-Reply-To: ; from "Tony Skjellum" at Sep 24, 96 11:00 am X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > We should obviously not put everything we don't want into JOD. We should > not have a pejorative view of it. It's not a pejorative view but it does indicate that, in the opinion of the Forum, the idea or the design is not ready to be in the standard. We can find for each JOD section some who disagree with its move to the JOD. However, unless the section is voted back into MPI-2, we need to agree that it is not part of the standard. Obviously individuals are free to pass around copies of any document and pitch any idea, MPI-ish or not. We're debating the official actions of the MPI Forum at SC'96. I feel it's important for the Forum to send a clear signal that JOD is not MPI-2, and ideas in it do not have higher priority or an easier glide-path into MPI-2 than any other idea. Some JOD sections were hashed and re-hashed, that's true. But in the end they were moved to JOD because they still had nasty unresolved issues. Others were directly moved to JOD because they needed much more rethinking or just didn't fit with the scope of the current MPI work. JOD is a place holder for potentially nice ideas or incomplete designs so they are not lost. Sections that try to move back from JOD into MPI-2 will still have the same unresolved issues to deal with, the same problems to address. So JOD is not a place for users to do "window shopping" and select functions to force back into MPI-2. For these reasons, I am in favour of not officially releasing JOD at SC'96. Free free to personally take a copy, show it to your friends, and urge them to flood the 1-800-2ASK-MPI number with requests for this or that function. :-) But I'd like the MPI Forum position to be: "Here are the 1.2 and 2.0 documents to be standardized, please review them and comment on them. Thank you!" --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com Convex Division From mpi-core-human@mcs.anl.gov Wed Sep 25 13:26:51 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA14601; Wed, 25 Sep 1996 13:26:50 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA03405 for mpi-core-out; Wed, 25 Sep 1996 12:30:11 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA03399 for ; Wed, 25 Sep 1996 12:29:56 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id MAA12167; Wed, 25 Sep 1996 12:29:54 -0500 Date: Wed, 25 Sep 1996 12:29:54 -0500 Message-Id: <199609251729.MAA12167@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: new version of MPI document available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. (Note: This is not where the chapter releases have occurred.) These directories also have all the sources and figures. This version is a dry run for the SC'96 release. There is a new appendix for constants that still needs some work. Not all the shifting of text has occured. Exactly what happens is likely to depend on the outcome of recent discussions (so vote early and vote often!) Please check over the document since it is likely to have oversights. If you want to make changes for the version to be printed for the meeting, you need to make sure the chapter authors can incorporate them by 10:00 AM (central US time) on Tuesday, October 1. Steve From mpi-core-human@mcs.anl.gov Wed Sep 25 14:17:23 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA15630; Wed, 25 Sep 1996 14:17:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA04617 for mpi-core-out; Wed, 25 Sep 1996 13:20:41 -0500 Received: from newton.ncsa.uiuc.edu (newton.ncsa.uiuc.edu [141.142.2.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA04610 for ; Wed, 25 Sep 1996 13:20:29 -0500 Received: from shalom.ncsa.uiuc.edu (shalom.ncsa.uiuc.edu [141.142.21.11]) by newton.ncsa.uiuc.edu (8.6.11/8.6.12) with ESMTP id NAA15488; Wed, 25 Sep 1996 13:20:27 -0500 Received: by shalom.ncsa.uiuc.edu (940816.SGI.8.6.9) id NAA00746; Wed, 25 Sep 1996 13:20:27 -0500 From: "Albert Cheng" Message-Id: <9609251320.ZM744@shalom.ncsa.uiuc.edu> Date: Wed, 25 Sep 1996 13:20:26 -0500 In-Reply-To: Raja Daoud "Re: JOD as separate document" (Sep 25, 12:15pm) References: <199609251717.AA184761829@hpcsos.col.hp.com> X-Mailer: Z-Mail (3.1.0 22feb94 MediaMail) To: Raja Daoud , mpi-core@mcs.anl.gov Subject: Re: JOD as separate document Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Sep 25, 12:15pm, Raja Daoud wrote: > For these reasons, I am in favour of not officially releasing JOD at > SC'96. Free free to personally take a copy, show it to your friends, > and urge them to flood the 1-800-2ASK-MPI number with requests for this > or that function. :-) But I'd like the MPI Forum position to be: > "Here are the 1.2 and 2.0 documents to be standardized, please review > them and comment on them. Thank you!" [I would prefer 1-900-MPI-4YOU, $20.00/min. That would pay for all the printing cost. :-)] May I propose a compromise: make the MPI-2 Standard containing only the current Part 1 and 2 and make the JOD a companion document. Together they form the official report by the MPI-2 forum. The whole report is distributed in SC'96 for public comments. As I understand, JOD is a time-dependent document--the intention is to attach it to MPI-2 Standard for now and then it will "migrate" to MPI-3 (maybe) or somewhere else (MPI-2++) or just vanishes (hope not). Whatever eventually happen to the JOD, the MPI-2 standard should stay unchanged. Just my two cents. =Albert -- From mpi-core-human@mcs.anl.gov Wed Sep 25 14:40:48 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA15917; Wed, 25 Sep 1996 14:40:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA05312 for mpi-core-out; Wed, 25 Sep 1996 13:44:20 -0500 Received: from tbag.osc.edu (tbag.osc.edu [128.146.36.50]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA05307 for ; Wed, 25 Sep 1996 13:44:09 -0500 Received: for gdburns@tbag.osc.edu by tbag.osc.edu (8.7.1/950822.1) id OAA18806; Wed, 25 Sep 1996 14:44:04 -0400 (EDT) Date: Wed, 25 Sep 1996 14:44:04 -0400 (EDT) From: Greg Burns Message-Id: <199609251844.OAA18806@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: Re: JOD as separate document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with Raja. I see the SC'96 release and subsequent public input as part of the latter stages of MPI-2. I don't see it as the opportunity for one big MPI Forum meeting where we rehash everything that was ever on the table. The public has access to the Forum, the mailing lists, the current draft document/chapters for this type of input. The people who will read the SC'96 document for the first time have probably made a conscious decision to avoid the "preliminaries". They want to concentrate on what is probably going to be in the final document and thereby serve as a late stage sanity check. -- Greg From mpi-core-human@mcs.anl.gov Wed Sep 25 14:53:22 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA15986; Wed, 25 Sep 1996 14:53:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA05666 for mpi-core-out; Wed, 25 Sep 1996 13:56:55 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA05659 for ; Wed, 25 Sep 1996 13:56:42 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id NAA00325; Wed, 25 Sep 1996 13:56:49 -0500 Date: Wed, 25 Sep 1996 13:56:49 -0500 (CDT) From: Tony Skjellum To: Greg Burns cc: mpi-core@mcs.anl.gov Subject: Re: JOD as separate document In-Reply-To: <199609251844.OAA18806@tbag.osc.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Again, that assumes that the only customers are those that care about MPI as of January 1, 1997, versus those looking for the long term roadmap and thinking of programs with durations of longer than a few months. Most of the questions I get is to see the big picture. -Tony On Wed, 25 Sep 1996, Greg Burns wrote: > Date: Wed, 25 Sep 1996 14:44:04 -0400 (EDT) > From: Greg Burns > To: mpi-core@mcs.anl.gov > Subject: Re: JOD as separate document > > > I agree with Raja. I see the SC'96 release and subsequent public input > as part of the latter stages of MPI-2. I don't see it as the opportunity > for one big MPI Forum meeting where we rehash everything that was ever > on the table. The public has access to the Forum, the mailing lists, > the current draft document/chapters for this type of input. The people > who will read the SC'96 document for the first time have probably > made a conscious decision to avoid the "preliminaries". They want > to concentrate on what is probably going to be in the final document and > thereby serve as a late stage sanity check. > > -- > Greg > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Wed Sep 25 14:55:28 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA16004; Wed, 25 Sep 1996 14:55:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA05741 for mpi-core-out; Wed, 25 Sep 1996 13:59:15 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA05727 for ; Wed, 25 Sep 1996 13:58:49 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id NAA00281; Wed, 25 Sep 1996 13:47:20 -0500 Date: Wed, 25 Sep 1996 13:47:19 -0500 (CDT) From: Tony Skjellum To: Raja Daoud cc: mpi-core@mcs.anl.gov Subject: Re: JOD as separate document In-Reply-To: <199609251717.AA184761829@hpcsos.col.hp.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Raja, it is important for the MPI Forum to advance to the level of other standards bodies, such as IETF, or face extinction, or fragmentation, or simply stalemate. Groups like IETF, with many of the same goals as MPI Forum, recognize several levels of standard: Request for Comments (RFC) Drafts Standards For your information (FYI) This body meets quarterly and allow these document to progress in their lives in this way. They do not hide, but rather encourage, things below the standard track to get lots of public comment. Lots of drafts and FYI's have made impact on the history of the Internet and TCP/IP, and the literally thousands of documents provide a written basis for much of the layering of software that is "below" MPI. Many of the documents are much poorer in formatting and detail than what we produce, btw. They provide valuable continuity and history to further generations of workers in the field, as well. POSIX is a similar situation, drafts, committees, working notes, standards, etc. While MPI straddles POSIX-like and IETF-like issues below it, the process of standardization is itself analogous. Given the positive successes of these organizations (and others like ANSI) with their standards and other type documents, I think that the JOD needs to have this type of circulation, that is enjoyed by the less-blessed, but still living documents being put forward by the working groups of the standard. It serves the role of the FYI's etc in IETF. So, if it should be true that the MPI Forum puts things in JOD as a consolation prize to authors of proposals who haven't the votes to get them into the standard, rather than only putting things there to be continued later, then we need to reconsider JOD. It is clear that these things should be the first-order ideas for MPI-3, and certainly will be de facto, if not de jure, given the import of what is written down. Because there is no stable, expected mechanism for convening MPI-3, JOD represents the one point of stability, that marks where we left off technically, if not politically. Also, the growth of the MPI Forum to the demands of so many activities, including, dynamic, parallel client-server, one-sided, distributed shared memory, signals, I/O, support for debuggers (external), collective extensions :-) and, of course, realtime :-), as well as the more traditional MPI-1-type role, calls into question its organization, which is geared to O(20-40) people, and O(3-5) simultaneous sub-threads. As part of a more formal standards organization, we would not have to establish the rules as we go along, but rather leverage that infrastructure, and experience of a permanent organization. We would not have to fight to make sure that longer term concepts are allowed to mature at their own pace. It is or will soon be time to accept this fact, and join a real standards body, that would also give the work a higher level of acceptance. Furthermore, I would imagine that interactions of the subgroups would have defined semantics, and the special interest subgroups would not vie with each other for "places in the sun" as they must do now. We would also not be tied to artificial deadlines of SC'9X, which are arguably less and less relevant, given the potential user base of MPI-2 and beyond, and even some burgeoning users of MPI-1. The disagreement about JOD is part of the organizational hole that comes from the HPF model we have adapted (and grownup with). It leaves open the recent discussion on changing the meaning of votes, what voting is, and how we actually adopt parts of MPI. It says little about conflict resolution, if anything. This is so, in addition to failing to offer continuity between versions of MPI (because their is no governing body to assure that), and a governing body to offload the organizational efforts, and provide guidance. It also puts unreasonable burdens on single individuals (convenor, major editor). So, to summarize, it is my view that: * JOD should get wide circulation as ideas in progress * We should drop ideas that were handled differently in MPI-2 from JOD, if any such ideas should be agreeable and severable * We should move to a formal standards organization for MPI-3 to address the growth of complexity and areas of interest, and let them work with coarser grain interactions, * The roadmap for future development should be more distributed across areas of interest. -Tony On Wed, 25 Sep 1996, Raja Daoud wrote: > Date: Wed, 25 Sep 1996 12:15:53 CDT > From: Raja Daoud > To: mpi-core@mcs.anl.gov > Subject: Re: JOD as separate document > > > We should obviously not put everything we don't want into JOD. We should > > not have a pejorative view of it. > > It's not a pejorative view but it does indicate that, in the opinion of > the Forum, the idea or the design is not ready to be in the standard. > We can find for each JOD section some who disagree with its move to the > JOD. However, unless the section is voted back into MPI-2, we need to > agree that it is not part of the standard. > > Obviously individuals are free to pass around copies of any document and > pitch any idea, MPI-ish or not. We're debating the official actions of > the MPI Forum at SC'96. I feel it's important for the Forum to send a > clear signal that JOD is not MPI-2, and ideas in it do not have higher > priority or an easier glide-path into MPI-2 than any other idea. > > Some JOD sections were hashed and re-hashed, that's true. But in the > end they were moved to JOD because they still had nasty unresolved > issues. Others were directly moved to JOD because they needed much more > rethinking or just didn't fit with the scope of the current MPI work. > JOD is a place holder for potentially nice ideas or incomplete designs > so they are not lost. Sections that try to move back from JOD into > MPI-2 will still have the same unresolved issues to deal with, the same > problems to address. So JOD is not a place for users to do "window > shopping" and select functions to force back into MPI-2. > > For these reasons, I am in favour of not officially releasing JOD at > SC'96. Free free to personally take a copy, show it to your friends, > and urge them to flood the 1-800-2ASK-MPI number with requests for this > or that function. :-) But I'd like the MPI Forum position to be: > "Here are the 1.2 and 2.0 documents to be standardized, please review > them and comment on them. Thank you!" > > --Raja > > -=- > Raja Daoud Hewlett-Packard Co. > raja@rsn.hp.com Convex Division > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Fri Sep 27 11:17:16 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA16310; Fri, 27 Sep 1996 11:17:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA18145 for mpi-core-out; Fri, 27 Sep 1996 10:18:38 -0500 Received: from daedalus.epcc.ed.ac.uk (root@daedalus.epcc.ed.ac.uk [129.215.56.21]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA15961 for ; Fri, 27 Sep 1996 08:33:20 -0500 Received: from subnode.epcc.ed.ac.uk (lyndon@onyx.epcc.ed.ac.uk [129.215.158.4]) by daedalus.epcc.ed.ac.uk (8.6.13/8.6.12) with SMTP id OAA17649 for ; Fri, 27 Sep 1996 14:32:50 +0100 Date: Fri, 27 Sep 96 14:32:48 BST Message-Id: <18945.9609271332@subnode.epcc.ed.ac.uk> From: L J Clarke Subject:  MPI Tech To: mpi-core@mcs.anl.gov Reply-To: lyndon@epcc.ed.ac.uk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk http://www.mpitech.com Enjoy - Lyndon /-----------------------------------------------------------------------\ |e||)| Lyndon J Clarke Edinburgh Parallel Computing Centre | Cymru | |c||c| Tel: [+ 44] (0)131 650 5021 Email: lyndon@epcc.ed.ac.uk | am Byth| \----| URL: http://www.epcc.ed.ac.uk/~lyndon/ |-------------------------/ \----------------------------------------/ From mpi-core-human@mcs.anl.gov Mon Sep 30 16:35:23 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA16678; Mon, 30 Sep 1996 16:35:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA29214 for mpi-core-out; Mon, 30 Sep 1996 15:38:08 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA29204 for ; Mon, 30 Sep 1996 15:37:52 -0500 Message-Id: <199609302037.PAA29204@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: voting rules Date: Mon, 30 Sep 1996 15:37:49 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am opposed to changing the voting rules. We are making good progress now and I see no abuses that need correction. I do not identify abstention with apathy at all. We have more abstentions than we often did in MPI-1 for two reasons. We voted to add the MPI-IO community into the process at a relatively late date, and many of them do not want to express an opinion on matters that they are unfamiliar with. This is good. In addition, because of the success of MPI, several attendees are there to observe but not participate. Since MPI is an open process, they should be welcome to do so, and the presence of observers (abstainers) should not impact the result of the votes of those who do want to express an opinion. Rusty From mpi-core-human@mcs.anl.gov Mon Sep 30 16:49:30 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA17116; Mon, 30 Sep 1996 16:49:28 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA29868 for mpi-core-out; Mon, 30 Sep 1996 15:50:25 -0500 Received: from relay2.UU.NET (relay2.UU.NET [192.48.96.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA29857; Mon, 30 Sep 1996 15:49:59 -0500 Received: from sky.com by relay2.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQbjlf24599; Mon, 30 Sep 1996 15:52:12 -0400 (EDT) Received: from anna.sky.com by sky.com (4.1/SMI-4.1-DNI-7.0.1) id AA23146; Mon, 30 Sep 96 15:52:21 EDT Received: from anna (localhost) by anna.sky.com (4.1/SMI-4.1) id AA07789; Mon, 30 Sep 96 15:48:51 EDT Message-Id: <32502422.2F1CF0FB@sky.com> Date: Mon, 30 Sep 1996 15:48:51 -0400 From: Anna Rounbehler Organization: Sky Computers X-Mailer: Mozilla 3.0Gold (X11; I; SunOS 4.1.4 sun4c) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov, mpi-realtime@mcs.anl.gov Subject: JOD & Some Simple Thoughts Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk JOD -------- Comments on the Journal of Development can be constructively funneled to individual workgroups by category to a specific group member. The member can summarize the comments for his/her category and post the summary to the group. That could work. Categories for real time could be by section. Comments for each summary may be organized as follows: Bindings typos, grammatical user implementor additions deletions re-write This is how lots of committees provide service to their technical communitites and avoid going crazy from information overload. The "person" to contact and category can be provided on the web. MPI RT & MPI-3 --------------- There is a great deal of interest in MPI RT by user communities that have applications requirements that are not satisifed by MPI-1, and MPI-2. In addition, there are architectures that are more frequently used in accordance with these applications. In the future, more architectures will be "slipping" into this category. For this reason, it is imperative that MPI-3 be a reality to continue some of the work yet to be done. In addition, there may be timing requirements for document changes, turn around, etc that conflict with MPI-2 progress. All these issues must be considered when decisions are being made with respect to the future of the standard and how to regulate changes. There are different needs developing. Implementors have committed to providing this product to customers, therefore the ability to accomodate some special requirements when needed is important. The purpose of these requirements is not to undermine the current effort by the standards committee, but to accelerate the unfinished work. They call it "cutting your losses to promote progress". Regards, Anna ----------------------------------------------------------------- Anna Rounbehler SKY Computers anna@sky.com From mpi-core-human@mcs.anl.gov Mon Sep 30 17:25:57 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA17603; Mon, 30 Sep 1996 17:25:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA28323 for mpi-core-out; Mon, 30 Sep 1996 15:03:40 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA28283; Mon, 30 Sep 1996 15:01:25 -0500 Message-Id: <199609302001.PAA28283@antares.mcs.anl.gov> To: Dick Treumann cc: mpi-misc@mcs.anl.gov, mpi-core@mcs.anl.gov, gropp@antares.mcs.anl.gov Subject: Re: Homogeneous??? In-reply-to: Your message of "Mon, 09 Sep 1996 16:35:24 EDT." <32347F8C.2781@kgn.ibm.com> Date: Mon, 30 Sep 1996 15:01:19 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I hate to disagree with Dick twice in one day, but... | There have been many comments made during MPI-2 deliberations that "If I | know I am in a homogeneous environment I can do xxxx." One example is: | If I know I am in a homogeneous environment I can get the effect of | MPI_Contiguous_struct by simply sending/receiving with extent(struct) as | a count and MPI_BYTE as type. | | Of course, this is not something anyone can "know" in a portable way. | MPI is supposed to be an aid in the writing of portable programs. | Has there been any discussion of prescribing a C #define / Fortran | parameter which would be in the mpi.h/mpif.h header to say "This MPI | implementation only supports homogeneous environments." | | With such a flag, a user could write code which does things one way if | the environment is guaranteed homogeneous and another if there is | potential mixing. I do not consider this idea very pretty but if it | allows some codes to do things in a faster way without becoming | nonportable it may be worthwhile. There are two points here, one about efficiency and the other about programmer convenience. Surely it is for the implementation to know whether it is homogeneous and optimize accordingly. For example, some communicators might be homogeneous and others not, and the situation might change from run to run. The implementation figures this out according to where processes are actually running. It doesn't need the user to tell it, and wouldn't trust the user anyway. I think that having an attribute or other mechanism for the user to tell the implementation of his intention to cheat and use MPI_BYTE when he knows better is a bad idea on philosophical grounds, which are important. MPI went to some effort to provide a way for the user to write really portable code, and bypassing it would be like telling a C compiler not to do type checking. You might have less trouble getting your code to compile, and it might run anyway, but it wouldn't (even) be C. To be realistic as well as philosophical, programmers are lazy, and the MPI Forum did not find a way to make the construction of a complex datatype easy. (Although the current mechanism is incredibly elegant compared with some we looked at!) The problem is that MPI is a library, not a compiler. What is needed to address the convenience issue is a tool that behaves a little bit like a compiler. It would read the declarations of datatypes in the C code and then produce the sequence of MPI calls needed to construct the corresponding MPI datatype. I can envision this being bound to a keystroke in my editor in such a way that the construction of complex MPI datatypes would be a pleasure. Wouldn't this be a fun tool to write? How can you resist? Rusty From mpi-core-human@mcs.anl.gov Tue Oct 1 09:00:36 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA07547; Tue, 1 Oct 1996 09:00:34 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA09429 for mpi-core-out; Tue, 1 Oct 1996 08:03:18 -0500 Received: from msr.EPM.ORNL.GOV (geist@msr.EPM.ORNL.gov [128.219.8.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA09424 for ; Tue, 1 Oct 1996 08:03:11 -0500 Received: (from geist@localhost) by msr.EPM.ORNL.GOV (8.7.6/8.7.3) id JAA20969 for mpi-core@mcs.anl.gov; Tue, 1 Oct 1996 09:03:05 -0400 (EDT) Date: Tue, 1 Oct 1996 09:03:05 -0400 (EDT) From: Al Geist Message-Id: <199610011303.JAA20969@msr.EPM.ORNL.GOV> To: mpi-core@mcs.anl.gov Subject: Re: Homogeneous??? Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >I think that having an attribute or other mechanism for the user to tell >the implementation of his intention to cheat and use MPI_BYTE... Rusty, I think that Dick may have been asking for the other way around. I agree that the above is a very bad idea, but what about a query to ask if the present system is homogeneous? An application could check this parameter and respond accordingly. My thoughts on this: a. Such a query exists in PVM and was placed there after several users requested such a feature. It is useful, but... b. This query really falls into the area of resource management/info and we have already voted not to tackle RM stuff in MPI-2. If we step back on the slippery slope of RM, we are backtracking. Al Geist From mpi-core-human@mcs.anl.gov Tue Oct 1 09:48:11 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA08176; Tue, 1 Oct 1996 09:48:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA10083 for mpi-core-out; Tue, 1 Oct 1996 08:51:57 -0500 Received: from sun1.ccrl-nece.technopark.gmd.de (sun1.ccrl-nece.technopark.gmd.de [193.175.160.67]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA10077 for ; Tue, 1 Oct 1996 08:51:50 -0500 Received: from sgi7.ccrl-nece.technopark.gmd.de (sgi7.ccrl-nece.technopark.gmd.de [193.175.160.89]) by sun1.ccrl-nece.technopark.gmd.de (8.7/3.4W296021412) with SMTP id OAA05170; Tue, 1 Oct 1996 14:50:00 +0100 (MET) Received: by sgi7.ccrl-nece.technopark.gmd.de (940816.SGI.8.6.9/940406.SGI.AUTO) id PAA02370; Tue, 1 Oct 1996 15:55:20 +0200 Date: Tue, 1 Oct 1996 15:55:20 +0200 From: hempel@ccrl-nece.technopark.gmd.de (Rolf Hempel) Message-Id: <199610011355.PAA02370@sgi7.ccrl-nece.technopark.gmd.de> To: mpi-core@mcs.anl.gov Subject: Re: Request for information on Fortran systems that use MPI Cc: sakon@lang1.bs1.fc.nec.co.jp, hempel@ccrl-nece.technopark.gmd.de Reply-To: hempel@ccrl-nece.technopark.gmd.de Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In his mail of Sept 16th, Marc Snir asked about the availability of pointer support by Fortran compilers on HPC platforms. With this mail I would like to answer his questions from NEC's perspective. > The current proposal for 1sided communication in MPI2 makes the use of some > functions (lock/unlock, for shared memory emulation) > contingent upon the availability of dynamic memory allocation and C-like > pointers. The type of code that is assumed to work in Fortran is shown below: > > REAL A > POINTER (P, A(100,100)) ! declare P to be an (integer) pointer to array A > CALL MPI_MEM_ALLOC(400,comm,P,ierr) ! allocate space for array A and returns > in P address of allocated memory; > ! now array A can be used > > This is not part of standard Fortran or Fortran 90 (Fortran 90 pointers cannot > be used for this type of code), but is widely available in many Fortran > environments. > > Questions: > >1. Are there Fortran systems that do not support (integer) pointers at all? >2. Are there systems that support such pointers, but require a syntax that is >different than the one shown in the example above? On the NEC SX-4 the syntax is the same as in the presented example. >3. On 64 bit systems, what is the datatype used for (integer) pointers? I.e., >what should be the type of the 3rd formal argument of MPI_MEM_ALLOC? > The SX-4 supports several different data formats (IEEE, IBM, CRAY). For all of them, You can choose that standard integers have 64 bit length (in the CRAY case, it's the only choice). Then, pointers can just be declared as integers. If you choose 32 bit integers as standard ints, then pointers have to be declared explicitely as integer*8. This feature will be available with NEC's Fortran 90 compiler by the end of this year. So, with the current Fortran 77 compiler, you have to use 64 bit length as standard integer mode. >If your pet Fortran has problems with this kind of code, please answer; >otherwise you will be stuck with an MPI feature that you cannot use on your >system. -- Rolf ------------------------------------------------------------------------ Rolf Hempel (email: hempel@ccrl-nece.technopark.gmd.de) C&C Research Laboratories, NEC Europe Ltd., Rathausallee 10, 53757 Sankt Augustin, Germany Tel.: +49 (0) 2241 - 92 52 - 95 Fax: +49 (0) 2241 - 92 52 - 99 From mpi-core-human@mcs.anl.gov Tue Oct 1 12:10:29 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA11386; Tue, 1 Oct 1996 12:10:28 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA13400 for mpi-core-out; Tue, 1 Oct 1996 11:13:21 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA13378 for ; Tue, 1 Oct 1996 11:13:03 -0500 Message-Id: <199610011613.LAA13378@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: headcount for October MPI Forum meeting Date: Tue, 01 Oct 1996 11:13:01 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, Below is the usual invitation. This will be a four-day meeting like the September meeting, starting at 1:30 on Tuesday, October 8. To cover the extra expenses associated with the extra day, the registration fee will be $115. Please let me know now for the head count whether you are coming. Thanks. Rusty The usual note, with details you now probably have memorized: The next meeting of the MPI Forum will take place on Tues.-Fri. Oct. 8-11. at the Ramada Hotel O'Hare (same place as the last few meetings). It is time to make your reservations. Register at the hotel by calling (847) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. We will follow a variation on the usual schedule, beginning at 1:30 on Tuesday and ending at noon on Friday. We will charge a registration fee of $115, which should be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. If you have any problems or questions, let me know, either by email (lusk@mcs.anl.gov) or phone (708-252-7852). Supercomputing '96 is Nov 17-22. Regards, Rusty ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Tue Oct 1 12:30:59 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA12116; Tue, 1 Oct 1996 12:30:57 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA14280 for mpi-core-out; Tue, 1 Oct 1996 11:34:32 -0500 Received: from moat.pweh.com (moat.pweh.com [192.54.250.131]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA14275 for ; Tue, 1 Oct 1996 11:34:22 -0500 Received: (from uucp@localhost) by moat.pweh.com (8.7.6/8.7.3) id MAA04363 for ; Tue, 1 Oct 1996 12:33:50 -0400 (EDT) Received: from unknown(191.29.71.250) by moat.pweh.com via smap (V1.3) id sma004325; Tue Oct 1 12:33:40 1996 Received: (from uucp@localhost) by drawbridge.eh.pweh.com (8.7.6/8.7.3) id MAA03608 for ; Tue, 1 Oct 1996 12:33:39 -0400 (EDT) Received: from fs17005.eh.pweh.com(191.29.170.5) by drawbridge.eh.pweh.com via smap (V1.3) id sma003585; Tue Oct 1 12:33:29 1996 Received: from ehposrv5.eh.pweh.com by pweh011.eh.pweh.com (SMI-8.6/SMI-SVR4) id MAA23515; Tue, 1 Oct 1996 12:33:25 -0400 Received: by ehposrv5.eh.pweh.com with SMTP (Microsoft Exchange Server Internet Mail Connector Version 4.0.993.5) id <01BBAF94.C2659B40@ehposrv5.eh.pweh.com>; Tue, 1 Oct 1996 12:33:33 -0400 Message-ID: From: "Bradley, Peter C. (MIS/CFD)" To: "'mpi-core@mcs.anl.gov'" Subject: Re: Homogeneous?? Date: Tue, 1 Oct 1996 12:33:30 -0400 X-Mailer: Microsoft Exchange Server Internet Mail Connector Version 4.0.993.5 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >Rusty wrote: > > >>|I There have been many comments made during MPI-2 deliberations that "If I >>| know I am in a homogeneous environment I can do xxxx." One > >>MPI is supposed to be an aid in the writing of portable programs. > >>| Has there been any discussion of prescribing a C #define / Fortran >>| parameter which would be in the mpi.h/mpif.h header to say "This MPI >>| implementation only supports homogeneous environments." >. >>Surely it is for the implementation to know whether it is homogeneous and >>optimize accordingly. This is very important. As an end-user, it is disappointing (though somewhat understandable from hardware vendors) to hear suggestions that users should write different code for different MPI implementations. For me, one of the biggest selling points of MPI over home-grown systems is its its potential for transparent use on heterogeneous clusters. A lot of our programmers are scientists with a very limited knowledge (or interest) of message passing technology. What they want is a black box message passing package that will run on a variety of platforms. I think it would be a selling point for a vendor MPI package if it was offered like: "This package is optimized for our systems, but it's also available for ...)" For example, I could crank a CFD code on a high-end machine while visualizing it on my own workstation. I know of at least one company using PVM for this kind of thing to connect a Pentium PC to SGI boxes. Pete Bradley Pratt & Whitney From mpi-core-human@mcs.anl.gov Tue Oct 1 12:33:43 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA12480; Tue, 1 Oct 1996 12:33:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA14358 for mpi-core-out; Tue, 1 Oct 1996 11:37:34 -0500 Received: from moat.pweh.com (moat.pweh.com [192.54.250.131]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA14353 for ; Tue, 1 Oct 1996 11:37:24 -0500 Received: (from uucp@localhost) by moat.pweh.com (8.7.6/8.7.3) id MAA04972 for ; Tue, 1 Oct 1996 12:36:51 -0400 (EDT) Received: from unknown(191.29.71.250) by moat.pweh.com via smap (V1.3) id sma004932; Tue Oct 1 12:36:40 1996 Received: (from uucp@localhost) by drawbridge.eh.pweh.com (8.7.6/8.7.3) id MAA04243 for ; Tue, 1 Oct 1996 12:36:39 -0400 (EDT) Received: from fs17005.eh.pweh.com(191.29.170.5) by drawbridge.eh.pweh.com via smap (V1.3) id sma004169; Tue Oct 1 12:36:23 1996 Received: from ehposrv5.eh.pweh.com by pweh011.eh.pweh.com (SMI-8.6/SMI-SVR4) id MAA23815; Tue, 1 Oct 1996 12:36:20 -0400 Received: by ehposrv5.eh.pweh.com with SMTP (Microsoft Exchange Server Internet Mail Connector Version 4.0.993.5) id <01BBAF95.2A761430@ehposrv5.eh.pweh.com>; Tue, 1 Oct 1996 12:36:28 -0400 Message-ID: From: "Bradley, Peter C. (MIS/CFD)" To: "'mpi-core@mcs.anl.gov'" Subject: Re: voting rules Date: Tue, 1 Oct 1996 12:36:25 -0400 X-Mailer: Microsoft Exchange Server Internet Mail Connector Version 4.0.993.5 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >I'm with Rusty on this one. If someone has an incomplete knowledge of an >issue, it makes absolutely no sense for them to vote on it. Should they flip >a coin? Alternate between yes and no? > >I attend the MPI forum partly as an observer and partly to represent the >specific needs and experience of my company. As an end user rather than a >software vendor, it is certain that there are items that do not affect us, >and >that I do not feel qualified to comment on. In this case, the only >responsible course of action is abstention. > > Pete Bradley > Pratt & Whitney > > >>---------- >>From: Rusty Lusk[SMTP:lusk@mcs.anl.gov] >>Sent: Monday, September 30, 1996 4:37PM >>To: mpi-core@antares.mcs.anl.gov >>Subject: voting rules >> >>I am opposed to changing the voting rules. We are making good progress now >>and I see no abuses that need correction. I do not identify abstention with >>apathy at all. We have more abstentions than we often did in MPI-1 for two >>reasons. We voted to add the MPI-IO community into the process at a >>relatively late date, and many of them do not want to express an opinion on >>matters that they are unfamiliar with. This is good. In addition, because >>of >>the success of MPI, several attendees are there to observe but not >>participate. Since MPI is an open process, they should be welcome to do so, >>and the presence of observers (abstainers) should not impact the result of >>the >>votes of those who do want to express an opinion. >> >>Rusty >> > > From mpi-core-human@mcs.anl.gov Tue Oct 1 14:48:24 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA16005; Tue, 1 Oct 1996 14:48:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA17238 for mpi-core-out; Tue, 1 Oct 1996 13:49:32 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA17219; Tue, 1 Oct 1996 13:49:04 -0500 Received: (tony@localhost); by Aurora.CS.MsState.Edu (SMI-8.6/7.0m-FWP-MsState); id NAA14832; Tue, 1 Oct 1996 13:48:59 -0500 Date: Tue, 1 Oct 1996 13:48:59 -0500 From: Tony Skjellum Message-Id: <199610011848.NAA14832@Aurora.CS.MsState.Edu> To: mpi-comm@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: New collective chapter Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The new collective chapter, as mentioned on mpi-coll group, is available. -Tony See ftp://aurora.cs.msstate.edu/pub/mpi/coll-2-30sep96.ps (or .ps.gz) Please review as it is on time and ready for firm consideration, following updates since last meeting. From mpi-core-human@mcs.anl.gov Thu Oct 3 11:36:25 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA27221; Thu, 3 Oct 1996 11:36:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA23558 for mpi-core-out; Thu, 3 Oct 1996 10:36:34 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA23552 for ; Thu, 3 Oct 1996 10:36:19 -0500 Message-Id: <199610031536.KAA23552@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: headcount for meeting (list) Date: Thu, 03 Oct 1996 10:36:18 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here are the people who have told me they are coming to the MPI Forum meeting next week. If you are coming and are not on this list, please let me know immediately. Rusty Robert Babb Ron Brightwell Pang Chen Albert Cheng Lyndon Clarke Dennis Cottel Raja Daoud Nathan Doss Richard Frost Shane Hebert Steve HussLederman Arkady Kanevsky Steve Landherr Lloyd Lewins Rusty Lusk Susan Kraus Steve Kubica Tyce McLarty Nick Nevin Bill Nitzberg Elsie Pierce JeanPierre Prost Heidi Poxon Rolf Rabenseifner Anna Rounbehler Eric Salo Bill Saphir Andrew Sherman Tony Skjellum Marc Snir Jeff Squyres Rajeev Thakur MaryDell Tholburn Dick Treumann Manuel Ujaldon Jerrell Watts Klaus Wolf Parkson Wong Kim Yates From mpi-core-human@mcs.anl.gov Fri Oct 4 11:25:08 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA11308; Fri, 4 Oct 1996 11:25:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA10894 for mpi-core-out; Fri, 4 Oct 1996 10:25:36 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA10889 for ; Fri, 4 Oct 1996 10:25:24 -0500 Message-Id: <199610041525.KAA10889@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: FINAL head count for next week's meeting Date: Fri, 04 Oct 1996 10:25:22 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This is the current list. I have to make the commitment to the hotel today. If you are coming but are not on the list, please let me know immediately. Thanks. Rusty ___________________________________ Robert Babb Ron Brightwell Pang Chen Albert Cheng Lyndon Clarke Dennis Cottel Raja Daoud Nathan Doss Mark Fallon Richard Frost Shane Hebert Steve Huss-Lederman Arkady Kanevsky Steve Landherr Lloyd Lewins Rusty Lusk Susan Kraus Steve Kubica Tyce McLarty Harish Nag Nick Nevin Bill Nitzberg Elsie Pierce Jean-Pierre Prost Heidi Poxon Rolf Rabenseifner Anna Rounbehler Eric Salo Bill Saphir Andrew Sherman Tony Skjellum Marc Snir Jeff Squyres Rajeev Thakur MaryDell Tholburn Dick Treumann Manuel Ujaldon Jerrell Watts Klaus Wolf Parkson Wong Kim Yates From mpi-core-human@mcs.anl.gov Fri Oct 4 14:18:40 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA13207; Fri, 4 Oct 1996 14:18:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA14733 for mpi-core-out; Fri, 4 Oct 1996 13:21:17 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA14728 for ; Fri, 4 Oct 1996 13:21:09 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id UAA12237 (8.6.13/IDA-1.6 for ); Fri, 4 Oct 1996 20:21:06 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for mpi-core@mcs.anl.gov) id UAA06257; Fri, 4 Oct 1996 20:21:06 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199610041821.UAA06257@awsrr.rus.uni-stuttgart.de> Subject: MPI-2 Interoperability To: mpi-core@mcs.anl.gov Date: Fri, 4 Oct 1996 20:21:05 +0200 (DST) Content-Type: text Content-Length: 1248 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I want to discuss at our meeting the following statement: Amendment -- Additional Section A high quality implementation of MPI allows that some processes use most of the other MPI implementations and publishes the protocol used for this purpose and allows that other MPI implementations are using the same protocol for interoperability. Discussion I believe we must strongly ask the implementators to implement interoperability over all platforms. Otherwise it is expected that proprietary standards, e.g. for Windows NT, will play this role in the future, i.e. the lack of interoperability invites others to push a new standard for message passing. Because I will leave now for the meeting, I hope we can discuss it at the meeting. I think there must be a sign for the users and a chance for the customers to buy computers with high quality MPI in this sense. Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Fri Oct 4 15:06:53 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA13540; Fri, 4 Oct 1996 15:06:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA15710 for mpi-core-out; Fri, 4 Oct 1996 14:10:03 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA15701 for ; Fri, 4 Oct 1996 14:09:54 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (SMI-8.6/7.0m-FWP-MsState); id OAA25340; Fri, 4 Oct 1996 14:09:55 -0500 Date: Fri, 4 Oct 1996 14:09:55 -0500 (CDT) From: Tony Skjellum To: Rolf Rabenseifner cc: mpi-core@mcs.anl.gov Subject: Re: MPI-2 Interoperability In-Reply-To: <199610041821.UAA06257@awsrr.rus.uni-stuttgart.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Rolf, We, of course, have talked about these things for years. It is not a new thing to think about... 1) high performance - implementing a least common denominator protocol and envelope make some implementations slower at the expense of others 2) different collective communication algorithms - everyone must achieve concensus on collective algorithms over a communicator with different implementations Where some concensus can be reached... Windows/NT MPI from MSU (MPICH) currently does not use a P4-style envelope, though we plan to amend that. It does not make use of the P4 networking approach. LAM does not use this envelope approach either. On our Myrinet version, we could also use the P4-style envelope. 3) Process startup. There will be different mechanisms for startup. a) different queueing systems b) different proprietary rsh-type mechanisms c) implementations that exploit threads masquerading as processes for SPMD groups. These should not be disallowed for interoperability. -Tony On Fri, 4 Oct 1996, Rolf Rabenseifner wrote: > Date: Fri, 4 Oct 1996 20:21:05 +0200 (DST) > From: Rolf Rabenseifner > To: mpi-core@mcs.anl.gov > Subject: MPI-2 Interoperability > > I want to discuss at our meeting the following statement: > > Amendment -- Additional Section > > A high quality implementation of MPI allows that some processes use > most of the other MPI implementations and publishes the protocol used > for this purpose and allows that other MPI implementations are using > the same protocol for interoperability. > > Discussion > > I believe we must strongly ask the implementators > to implement interoperability over all platforms. > Otherwise it is expected that proprietary standards, > e.g. for Windows NT, will play this role in the future, > i.e. the lack of interoperability invites others to push > a new standard for message passing. > > Because I will leave now for the meeting, I hope we can discuss it > at the meeting. I think there must be a sign for the users and > a chance for the customers to buy computers with high quality MPI > in this sense. > > Rolf > > > Rolf Rabenseifner (Computer Center ) > Rechenzentrum Universitaet Stuttgart (University of Stuttgart) > Allmandring 30 Phone: ++49 711 6855530 > D-70550 Stuttgart 80 FAX: ++49 711 6787626 > Germany rabenseifner@rus.uni-stuttgart.de > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; Quote: "What a rain of ashes falls on him that sees the new and cannot leave the old."-Shakespeare ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Fri Oct 4 16:59:10 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA14810; Fri, 4 Oct 1996 16:59:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA18507 for mpi-core-out; Fri, 4 Oct 1996 16:01:46 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA18497 for ; Fri, 4 Oct 1996 16:01:35 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA06075 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Fri, 4 Oct 1996 14:01:33 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id NAA14030; Fri, 4 Oct 1996 13:59:22 -0700 From: "Eric Salo" Message-Id: <9610041359.ZM14028@mrjones.engr.sgi.com> Date: Fri, 4 Oct 1996 13:59:22 -0700 In-Reply-To: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) "MPI-2 Interoperability" (Oct 4, 8:21pm) References: <199610041821.UAA06257@awsrr.rus.uni-stuttgart.de> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: MPI-2 Interoperability Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I want to discuss at our meeting the following statement: > > Amendment -- Additional Section > > A high quality implementation of MPI allows that some processes use > most of the other MPI implementations and publishes the protocol used > for this purpose and allows that other MPI implementations are using > the same protocol for interoperability. Hold on a second, let me go and put on my Cynical Hat... I see this as meaningless addition to the text. What exactly does it accomplish? Everyone already knows that the users would like interoperability between implementations, but I just don't see any way that this is going to happen any time soon. (Well, maybe with one exception...) Tony has already pointed out some of the technical problems, and there are political ones as well. We can't very well say that someone isn't complying with the standard because they refuse to publish their protocol. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Oct 11 12:26:57 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA12199; Fri, 11 Oct 1996 12:26:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA11921 for mpi-core-out; Fri, 11 Oct 1996 11:25:25 -0500 Received: from daedalus.epcc.ed.ac.uk (root@[129.215.56.21]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA11912; Fri, 11 Oct 1996 11:25:08 -0500 Received: from subnode.epcc.ed.ac.uk (smith@serpentine.epcc.ed.ac.uk [129.215.158.2]) by daedalus.epcc.ed.ac.uk (8.6.13/8.6.12) with SMTP id RAA01956; Fri, 11 Oct 1996 17:25:04 +0100 Date: Fri, 11 Oct 96 17:25:01 BST Message-Id: <16004.9610111625@subnode.epcc.ed.ac.uk> To: Rusty Lusk Cc: mpi-misc@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Homogeneous??? In-Reply-To: <199609302001.PAA28283@antares.mcs.anl.gov> References: <32347F8C.2781@kgn.ibm.com> <199609302001.PAA28283@antares.mcs.anl.gov> From: "A Gordon Smith" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Rusty Lusk writes: (30/Sep/96) > I think that having an attribute or other mechanism for the user to tell > the implementation of his intention to cheat and use MPI_BYTE when he knows > better is a bad idea on philosophical grounds, which are important. MPI > went to some effort to provide a way for the user to write really portable > code, and bypassing it would be like telling a C compiler not to do type > checking. You might have less trouble getting your code to compile, and it > might run anyway, but it wouldn't (even) be C. ********** > To be realistic as well as philosophical, programmers are lazy, and the MPI > Forum did not find a way to make the construction of a complex datatype easy. > (Although the current mechanism is incredibly elegant compared with some we > looked at!) The problem is that MPI is a library, not a compiler. What is > needed to address the convenience issue is a tool that behaves a little bit > like a compiler. It would read the declarations of datatypes in the C code > and then produce the sequence of MPI calls needed to construct the > corresponding MPI datatype. I can envision this being bound to a keystroke in > my editor in such a way that the construction of complex MPI datatypes would > be a pleasure. Wouldn't this be a fun tool to write? How can you resist? *********** > Rusty [ Sorry so slow, I'm just catching up on my MPI e-mail. ] On Rusty's last point, Werner Augustin of Uni Karlsruhe, just developed a toolset not unlike that which Rusty envisages as part of EPCC's Summer Scholarship Programme this year. There are separate pre- processor tools for C (converting marked structs) and for Fortran 77 (converting marked COMMON blocks); both tools are based on public- domain lex/yacc code and grammars. The project report is available via the web at URL: http://www.epcc.ed.ac.uk/ssp/1996/projects96.html "EPCC-SS96-10 MPI Datatypes Toolset" Werner Augustin I hope to do further development and testing of the code and to make it freely available sometime soon. (I like the idea of integrating with an editor: "mpiify-region"?). Please contact me if you have any questions on this work. thanks, -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- -=-=- A. Gordon Smith -=- Edinburgh Parallel Computing Centre -=-=- =- http://www.epcc.ed.ac.uk/~smith -=- Phone +44 (0)131 650 6712 -= =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= From mpi-core-human@mcs.anl.gov Mon Oct 14 11:08:47 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA04473; Mon, 14 Oct 1996 11:08:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA15320 for mpi-core-out; Mon, 14 Oct 1996 10:09:22 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA15315 for ; Mon, 14 Oct 1996 10:09:13 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id KAA18079; Mon, 14 Oct 1996 10:09:12 -0500 Date: Mon, 14 Oct 1996 10:09:12 -0500 Message-Id: <199610141509.KAA18079@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: schedule for SC96 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As was discussed at the meeting last week, we need to keep the deadlines for Supercomputing in mind. Here are the deadlines for the Supercomputing release: Friday, October 11: MPI Forum meeting ended. Friday, October 18: Updated chapters are due to me by 12:00 PM Central US time (noon). When the full document is ready it will be announced on mpi-core. This version should reflect all changes decided at the last meeting. The Forum should veiw this as a chance to look over the full document. Friday, October 25: "Final" version of each chapter due to me by 12:00 PM Central US time (noon). This is the last input that should have substantial changes. I will take this verison and work on it to make it "nice" for the SC96 release. As a result, no changes can be made by others during the following week. Friday, November 1: By this day, I will release the full document to the Forum for review. This is the opportunity for the Forum to proof the SC96 document. Friday, November 8: All changes from chapter authors are due to me by 10:00 AM Central US time. ONLY SMALL CHANGES TO FIX INTEGRATION PROBLEMS WILL BE MADE FROM NOVEMBER 1 UNTIL NOVEMBER 8. Monday, November 11: The Supercomputing version will be released. It will be announced to core, printed, and announced on the appropriate newsgroups. Keep in mind this will be a major public release. Looking the document over for the next 10 days can help make our work look as it should. The BOF will be Wednesday, November 20 from 3:30 to 5:30 in the Washington Room at SC96. The current (and future documents) are available via the web. See http://www.cs.wisc.edu/~lederman/mpi2 and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2. The full current document is in mpi2-report.ps.Z. The sources and scripts are also available. The official votes through the September meeting are available in the file votes-official. The October votes will be added soon. Steve From mpi-core-human@mcs.anl.gov Fri Oct 18 09:09:29 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA09272; Fri, 18 Oct 1996 09:09:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA14763 for mpi-core-out; Fri, 18 Oct 1996 08:12:29 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA14723; Fri, 18 Oct 1996 08:11:06 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id IAA29295; Fri, 18 Oct 1996 08:11:04 -0500 Date: Fri, 18 Oct 1996 08:11:04 -0500 Message-Id: <199610181311.IAA29295@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-bind@mcs.anl.gov CC: mpi-core@mcs.anl.gov In-reply-to: <9610171334.ZM1200@win233.nas.nasa.gov> (wcs@nas.nasa.gov) Subject: problems with retriving files (was: new Fortran 90 section ready) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Several times in the last month, people have been unable to retrive files left by chapter authors because the server ftp.cs.wisc.edu will not accept more connections. This problems has surfaced due to the beginning of classes here at the University. I am trying to have this corrected. I have created another access method which I hope will eliminate the problem. You can also reach the files via http://www.cs.wisc.edu/~lederman/mpif. I have tested doing this myself from a remote site and it seems to work fine. I believe this will still work even when the ftp server is at its limit of 50 logins. If you have any problems (and it seems to happen to me a lot lately) please drop me a note and I will try to fix it. Please do not confuse this address with the location where the official files are released for each update of the document (http://www.cs.wisc.edu/~lederman/mpi2). Chapter authors should note that no changes have been made in the mechanism they should use to place documents here. Glad to see that people are actually trying to get the documents! Steve P.S. - The ftp directions are: The WWW address is ftp://ftp.cs.wisc.edu/pub/mpi2. You can also access via anonymous ftp by: 1) ftp ftp.cs.wisc.edu 2) log on as anonymous 3) cd pub/mpi2 4) get the files you want From mpi-core-human@mcs.anl.gov Fri Oct 18 16:33:52 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA14051; Fri, 18 Oct 1996 16:33:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA24869 for mpi-core-out; Fri, 18 Oct 1996 15:37:09 -0500 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA24730; Fri, 18 Oct 1996 15:34:34 -0500 Date: Fri, 18 Oct 1996 15:34:33 -0500 Message-Id: <199610182034.PAA14982@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov, mpi-external@mcs.anl.gov, mpi-io@mcs.anl.gov Subject: caching on MPI handles Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At the last MPI meeting, the Forum voted out the section on "Caching on MPI Handles" in the external interfaces chapter. Part of the reason may be that the section allowed caching on all MPI handles. Though it may not be necessary to support caching on all handles, it is definitely needed on some handles. As explained below, I need caching on datatypes in my MPI-IO implementation. I can also see a need for caching on file handles. Therefore, I suggest that at the next MPI meeting we reconsider the issue of allowing caching on select MPI handles. My short list includes datatypes and file handles. I heard that there may be some technical difficulties in caching on datatypes, but those need to be addressed. Rajeev -----------for those interested in the details---------- 1. Caching on datatypes: In my "layerable" implementation of MPI-IO, I create a flattened version of derived datatypes, and I need to store it somewhere for use later in the program. The ideal way would be to cache the flattened version on the datatype itself, so that when the datatype is freed, the flattened version is also freed. Since caching on datatypes is not currently available, I maintain a global linked list (maybe a hash table later on) of all flattened datatypes, indexed by the datatype handle. In order that this list gets freed at the end of the program, I maintain an attribute on MPI_COMM_WORLD (could be MPI_COMM_SELF---anything that would be freed by MPI_Finalize) and provide a delete function. This works fine, except that when the user frees a derived datatype, the flattened version doesn't get freed automatically. If the system reuses the same file handle for a newly created derived datatype, it will match the key for the flattened version of the old datatype in my linked list! One solution to this problem is to define a new MPI function, MPI_Type_free_callback(datatype, callback_fn), which allows me to supply a callback function that will be called when the datatype is freed. The callback function will delete the flattened version in my linked list. A cleaner solution, I think, is to support caching on datatypes. Wasn't attribute caching intended for these kinds of purposes in the first place? 2. Caching on file handles: In the I/O chapter, there is a proposal by Leslie Hart on I/O Service Interception (Section 10.8). This proposal allows the user to add alternate service routines for each of the I/O functions in the chapter. This is intended to support special purpose I/O services such as remote I/O, data compression, etc. The ability to cache on file handles is essential for this proposal to work. In general, I think special-purpose I/O libraries built on top of MPI-IO would need caching on file handles. From mpi-core-human@mcs.anl.gov Fri Oct 18 16:57:47 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA14168; Fri, 18 Oct 1996 16:57:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA25720 for mpi-core-out; Fri, 18 Oct 1996 16:01:16 -0500 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA25715 for ; Fri, 18 Oct 1996 16:01:07 -0500 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA11776 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Fri, 18 Oct 1996 14:01:04 -0700 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id NAA08992; Fri, 18 Oct 1996 13:58:48 -0700 From: "Eric Salo" Message-Id: <9610181358.ZM8990@mrjones.engr.sgi.com> Date: Fri, 18 Oct 1996 13:58:48 -0700 In-Reply-To: Rajeev Thakur "caching on MPI handles" (Oct 18, 3:34pm) References: <199610182034.PAA14982@abacus> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: caching on MPI handles Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I'll pass on the question of file handles, since I don't claim to understand anything about the I/O chapter. But in general, adding dedicated routines for each object class to support attribute caching feels like a much better solution to me, so this is an approach that I would personally favor. Regarding datatypes, it does seem like this would be a useful bit of functionality, but I'm a bit concerned about the effect on derived datatypes. Consider the following sequence of events: 1) An attribute is cached on Datatype A 2) Datatype A is used to create Datatype B 3) The value of the attribute on Datatype A is altered. What is the defined behavior? Perhaps there is a simple solution to this problem, but if so it's not yet obvious to me. Let's continue this thread on mpi-external... -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Sat Oct 19 15:55:23 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA22096; Sat, 19 Oct 1996 15:55:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA08760 for mpi-core-out; Sat, 19 Oct 1996 14:57:06 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA08755 for ; Sat, 19 Oct 1996 14:56:57 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id OAA00162; Sat, 19 Oct 1996 14:56:55 -0500 Date: Sat, 19 Oct 1996 14:56:55 -0500 Message-Id: <199610191956.OAA00162@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. (Note: This is not where the chapter releases have occurred.) These directories also have all the sources and figures. This version is a dry run for the SC'96 release. The collective chapter has been removed pending an updated version. Also, the realtime chapter has the same section twice - this will be fixed. PLEASE CAREFULLY READ THIS VERSION SINCE THIS IS EVERYONES LAST CHANCE TO MAKE CHANGES TO THE DOCUMENT BEFORE THE SC'96 RELEASE. ALL CHANGES MUST BE TO ME (VIA CHAPTER AUTHORS) BY TUESDAY, OCTOBER 28 AT 8:00 AM CENTRAL TIME. AFTER THAT DATE ONLY CHANGES TO FIX FORMATTING WILL BE ACCEPTED. Note that I have extended the deadline from Friday to make sure everyone has time to look over the document. I suspect chapter authors will need changes no later than Monday morning to have time to incorporate them into their chapters. Each author will decide her/his own time frame. There is a known problem that all the chapters after realtime have letters instead of numbers for their label. I am working on that. We have all spent a lot of time to make this document what it is today. Let's go the extra yard and check it over to make it great for the public release. Steve From mpi-core-human@mcs.anl.gov Sat Oct 19 16:13:36 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA22206; Sat, 19 Oct 1996 16:13:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA09002 for mpi-core-out; Sat, 19 Oct 1996 15:16:38 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA08996 for ; Sat, 19 Oct 1996 15:16:28 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id PAA00198; Sat, 19 Oct 1996 15:16:27 -0500 Date: Sat, 19 Oct 1996 15:16:27 -0500 Message-Id: <199610192016.PAA00198@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: corrected date of next dealine Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Ok, I had one too many notes to send. The deadline for input from chapter authors is Tuesday, October 29 at 8:00 AM Central US time. The last note I sent out had the correct day but the wrong date. Sorry. Steve From mpi-core-human@mcs.anl.gov Sun Oct 20 18:40:01 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA00814; Sun, 20 Oct 1996 18:39:59 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA19655 for mpi-core-out; Sun, 20 Oct 1996 17:41:02 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA19650; Sun, 20 Oct 1996 17:40:54 -0500 Message-Id: <199610202240.RAA19650@antares.mcs.anl.gov> To: mpi-impl@antares.mcs.anl.gov, mpi-core@antares.mcs.anl.gov Subject: interoperability Date: Sun, 20 Oct 1996 17:40:53 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At the last MPI Forum meeting, there was a discussion of how one might deal with the issue of interoperability among separate MPI implementations. While seamless integration of multiple implementations seems to be a quite complex research issue, there was interest in exporing a very limited solution, in which substantial reductions in functionality were traded for a design in which some useful application work could be done. All of the proposals discussed involved some sort of standard "bridge" protocol that would allow at least one process on a machine talk to at least one process on other machines. It appeared that by greatly simplifying what would be expected of an implmentation in order to participate in a multi-vendor MPI computation, and offloading many of the issues (e.g., routing) to the user application, it might be possible to define something useful that would not be too much of a burden to implementors. The discussion resulted in an agreement to include a sentence in the Supercomputing version of the document to the effect that the Forum was still considering the interoperability issue and the decision to use the mpi-impl mailing list to continue the discussion, rather than starting a new list. One can join mpi-impl@mcs.anl.gov by sending the message "subscribe mpi-impl" to majordomo@mcs.anl.gov. Regards, Rusty From mpi-core-human@mcs.anl.gov Tue Oct 22 10:27:08 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA02645; Tue, 22 Oct 1996 10:27:05 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA20783 for mpi-core-out; Tue, 22 Oct 1996 09:27:27 -0500 Received: from relay.hp.com (relay.hp.com [15.255.152.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA20550 for ; Tue, 22 Oct 1996 09:19:35 -0500 Received: from dink.rsn.hp.com by relay.hp.com with ESMTP (1.37.109.16/15.5+ECS 3.3) id AA004493968; Tue, 22 Oct 1996 07:19:29 -0700 Message-Id: <199610221419.AA004493968@relay.hp.com> Received: by dink.rsn.hp.com (1.39.111.2/16.2) id AA064614344; Tue, 22 Oct 1996 09:25:44 -0500 From: Suresh Damodaran-Kamal To: mpi-core@mcs.anl.gov Date: Tue, 22 Oct 1996 9:25:44 CDT X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk unsubscribe suresh From mpi-core-human@mcs.anl.gov Wed Oct 23 09:06:02 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA00213; Wed, 23 Oct 1996 09:05:57 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA12419 for mpi-core-out; Wed, 23 Oct 1996 08:06:48 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA12414 for ; Wed, 23 Oct 1996 08:06:41 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id IAA21976; Wed, 23 Oct 1996 08:06:40 -0500 Date: Wed, 23 Oct 1996 08:06:40 -0500 Message-Id: <199610231306.IAA21976@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: new version with collective Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. The major difference from the version released on Saturday (Oct. 19) is that the collective chapter has been added. Other minor fixups of formatting have also occurred. The schedule for producing the SC96 document is not effected by this updated version. Steve From mpi-core-human@mcs.anl.gov Thu Oct 24 10:44:58 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA19429; Thu, 24 Oct 1996 10:44:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA09480 for mpi-core-out; Thu, 24 Oct 1996 09:47:12 -0500 Received: from letterbox.rl.ac.uk (letterbox.rl.ac.uk [130.246.8.100]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id JAA08839 for ; Thu, 24 Oct 1996 09:18:26 -0500 Received: from jkr.cc.rl.ac.uk by letterbox.rl.ac.uk with SMTP (PP) id ; Thu, 24 Oct 1996 15:18:11 +0100 Received: by jkr.cc.rl.ac.uk (4.1/SMI-4.1) id AA19134; Thu, 24 Oct 96 15:18:10 BST Date: Thu, 24 Oct 96 15:18:10 BST From: jkr@letterbox.rl.ac.uk (John Reid) Message-Id: <9610241418.AA19134@jkr.cc.rl.ac.uk> To: mpi-core@mcs.anl.gov Subject: Access to MPI standard Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk ----- Begin Included Message ----- It was a very tedious business to access the MPI standard. May I suggest that you provide it in compressed format? It is far easier to get a file of length 357088 than one of length 1383303. Thanks, John Reid. From mpi-core-human@mcs.anl.gov Thu Oct 24 11:20:10 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA20614; Thu, 24 Oct 1996 11:20:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA11118 for mpi-core-out; Thu, 24 Oct 1996 10:23:44 -0500 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA11110 for ; Thu, 24 Oct 1996 10:23:17 -0500 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id KAA22468; Thu, 24 Oct 1996 10:22:53 -0500 Date: Thu, 24 Oct 1996 10:22:53 -0500 Message-Id: <199610241522.KAA22468@romano.cs.wisc.edu> From: Steve Huss-Lederman To: jkr@letterbox.rl.ac.uk CC: mpi-core@mcs.anl.gov In-reply-to: <9610241418.AA19134@jkr.cc.rl.ac.uk> Subject: Re: Access to MPI standard Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I'll carbon the group just to let you know this is being followed up on. After this, I will take it offline. > Date: Thu, 24 Oct 96 15:18:10 BST > From: jkr@letterbox.rl.ac.uk (John Reid) > Sender: owner-mpi-core@mcs.anl.gov > Precedence: bulk > > > ----- Begin Included Message ----- > > > It was a very tedious business to access the MPI standard. May > I suggest that you provide it in compressed format? It is far easier > to get a file of length 357088 than one of length 1383303. > > Thanks, > > John Reid. > > > > > Dear John Reid, As editor of MPI-2, I should take care of your concern. As far as I know, all the versions of MPI I distribute are in compressed format. I suspect you are getting a version from somewhere else. I don't recognize the size of the file you are refering to, so I am not sure what version of MPI you are trying to get and where you got it. Let me know (lederman@cs.wisc.edu) and we can fix it. Steve From mpi-core-human@mcs.anl.gov Thu Oct 31 10:46:37 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA24887; Thu, 31 Oct 1996 10:46:35 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA18530 for mpi-core-out; Thu, 31 Oct 1996 09:48:29 -0600 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA18525 for ; Thu, 31 Oct 1996 09:48:26 -0600 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id JAA23063; Thu, 31 Oct 1996 09:47:30 -0600 Date: Thu, 31 Oct 1996 09:47:30 -0600 Message-Id: <199610311547.JAA23063@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: SC'96 proof document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The proposed public release version for SC'96 is now available at http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. I will accept corrections to the document until Thursday, November 7 at 10:00 AM Central US time. This is a day earlier than previously announced since this version is coming out a day early. This will allow me to release the document for SC'96 sooner. Unlike previous changes, all corrections must be sent to me as chapter authors can no longer edit the document. I ONLY WANT FORMATTING AND MINOR CORRECTIONS FOR THESE FINAL CHANGES TO THE SC'96 RELEASE. NO TECHNICAL CHANGES ARE EXPECTED TO BE MADE. Attached is the proposed announcement I will post to the usual places when the release occurs next week. Steve ----------------------------------------------------------------------- This is to announce that there will be a Birds-of-a-Feather about MPI-2 at Supercomputing '96. The session will be from 3:30-5:30 on Wednesday, November 20, in the Washington Room (which is located in the DoubleTree Hotel). The session will consist of an overview of the current thinking in the MPI-2 standards effort and will be followed by an opportunity for public comment. The current draft of the proposed MPI-2 standard is available at http://www.mcs.anl.gov/mpi. With the release of this version, a formal public comment period begins. It is anticipated that the final version of the MPI-2 standard will be released in May 1997. Hope to see you there. The MPI Forum ---------------------------------------------------------------------- Abstract of the MPI-2 document: This document describes the efforts of the MPI Forum to enhance the original MPI standard. Known as MPI-2, this effort began in March 1995. A draft version of the document is being released for public input at Supercomputing '96. The final version is anticipated in May 1997. The document has three parts: MPI-1.2, MPI-2, and MPI Journal of Development. Topics being explored for inclusion in MPI-2 are dynamic processes, one-sided communications, extended collective operations, external interfaces, additional language bindings, miscellaneous topics, and I/O. From mpi-core-human@mcs.anl.gov Thu Nov 7 18:03:15 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA00252; Thu, 7 Nov 1996 18:03:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA19350 for mpi-core-out; Thu, 7 Nov 1996 17:04:41 -0600 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA19345 for ; Thu, 7 Nov 1996 17:04:38 -0600 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id RAA28016; Thu, 7 Nov 1996 17:02:52 -0600 Date: Thu, 7 Nov 1996 17:02:52 -0600 Message-Id: <199611072302.RAA28016@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: MPI-2 SC'96 release done Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am pleased to announce that the SC'96 version of the document has been released. The public releases are available from http://www.mcs.anl.gov/mpi by following the links. All public copies of the standard will reside at this home page. For the Forum, the source files, etc are available in their usual location: http://www.cs.wisc.edu/~lederman/mpi2/ and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/. Chapter authors should get the latest sources to begin edits for the version for the next meeting. I would like to that everyone for their efforts in making this document what it is. Steve From mpi-core-human@mcs.anl.gov Mon Nov 11 13:33:57 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA10845; Mon, 11 Nov 1996 13:33:32 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA11590 for mpi-core-out; Mon, 11 Nov 1996 12:34:50 -0600 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA11584 for ; Mon, 11 Nov 1996 12:34:44 -0600 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.6.12/8.6.12) id MAA01243; Mon, 11 Nov 1996 12:32:22 -0600 Date: Mon, 11 Nov 1996 12:32:22 -0600 Message-Id: <199611111832.MAA01243@romano.cs.wisc.edu> From: Steve Huss-Lederman To: treumann@kgn.ibm.com CC: mpi-core@mcs.anl.gov In-reply-to: <32876CCC.41C6@kgn.ibm.com> (message from Dick Treumann on Mon, 11 Nov 1996 13:13:32 -0500) Subject: Re: SC96 Version Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dick, The same day I announced to mpi-core I put it on the news groups (comp.parallel.mpi, comp.parallel, comp.sys.super). I checked and it is there for two different sites. It is not on comp.parallel yet but it is moderated and often slow in getting out posts. Thus, I think it should be generally available - if there was a glitch I do not see it. I did not put it on comp.parallel.pvm but will do that now. Steve > Sender: treumann@minnie.nic.kingston.ibm.com > Date: Mon, 11 Nov 1996 13:13:32 -0500 > From: Dick Treumann > Organization: IBM Power Parallel Systems - RS6000 Division > X-Mailer: Mozilla 3.0 (X11; I; AIX 1) > Mime-Version: 1.0 > Cc: mpi-core@mcs.anl.gov > Content-Type: text/plain; charset=us-ascii > Content-Transfer-Encoding: 7bit > > Steve, > > Has the availability announcement for the SC96 draft been posted to > comp.parallel and comp.parallel.mpi? I do not remember seeing it there? > > Certainly we are at a stage where we want the parallel community at > large to take greater notice even though they do not subscribe to the > mpi-core listserv. Maybe it should even be posted to comp.parallel.pvm > with the explanation that reaction from PVM users is also desired. > > Dick > -- > Dick Treumann IBM RS/6000 Division > (Internet) treumann@kgn.ibm.com Poughkeepsie, NY > (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 > (internal) treumann@windsurf.kgn.ibm.com Fax: (914) 433-8363 > From mpi-core-human@mcs.anl.gov Tue Nov 12 04:42:21 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA04727; Tue, 12 Nov 1996 04:42:20 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA25506 for mpi-core-out; Tue, 12 Nov 1996 03:40:04 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id DAA25501 for ; Tue, 12 Nov 1996 03:39:58 -0600 Received: from windsurf.kgn.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA17098; Mon, 11 Nov 1996 13:08:54 -0500 Message-Id: <32876CCC.41C6@kgn.ibm.com> Date: Mon, 11 Nov 1996 13:13:32 -0500 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.0 (X11; I; AIX 1) Mime-Version: 1.0 To: lederman@cs.wisc.edu Cc: mpi-core@mcs.anl.gov Subject: SC96 Version Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve, Has the availability announcement for the SC96 draft been posted to comp.parallel and comp.parallel.mpi? I do not remember seeing it there? Certainly we are at a stage where we want the parallel community at large to take greater notice even though they do not subscribe to the mpi-core listserv. Maybe it should even be posted to comp.parallel.pvm with the explanation that reaction from PVM users is also desired. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@kgn.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.kgn.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue Nov 26 11:23:16 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA16048; Tue, 26 Nov 1996 11:23:13 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA08228 for mpi-core-out; Tue, 26 Nov 1996 10:25:33 -0600 Received: from daedalus.epcc.ed.ac.uk (root@daedalus.epcc.ed.ac.uk [129.215.56.21]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA08180; Tue, 26 Nov 1996 10:23:53 -0600 Received: from subnode.epcc.ed.ac.uk (mpi2euro@serpentine.epcc.ed.ac.uk [129.215.158.2]) by daedalus.epcc.ed.ac.uk (8.6.13/8.6.12) with SMTP id QAA24867 for ; Tue, 26 Nov 1996 16:09:19 GMT Date: Tue, 26 Nov 96 16:09:16 GMT Message-Id: <15335.9611261609@subnode.epcc.ed.ac.uk> From: MPI-2 European Information Meeting Subject: ANNOUNCE 3rd European MPI Workshop To: mpi2_euro_list@epcc.ed.ac.uk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Announcement ------------ Please distribute this announcement to all interested colleagues. 3rd European MPI Workshop February 13-15, 1997 Edinburgh, Scotland, United Kingdom MPI (Message Passing Interface) is rapidly gaining wide acceptance as the standard for message-passing programming. All major HPC hardware vendors support MPI or have announced that they will do so. Mature public domain implementations are available and are in wide use. The MPI Forum reconvened during 1995 to address new functionality and to develop an MPI-2 standard. MPI-2 is a superset of MPI and is expected to be finalized in the first half of 1997. The new functionality in MPI-2 includes: dynamic process creation; single sided communication; parallel input/output; real time. MPI-2 was described during a successful BOF at Supercomputing 1996. The current draft standard can be obtained from . The 3rd European MPI Workshop is the successor of the successful 2nd European MPI Workshop held in January 1996 at VCPC and 1st European MPI Workshop held in January 1994 at INRIA. This workshop is organized by EPCC, and sponsored by the ESPRIT project "HPC Standards". The workshop will include presentations by a number of key members of the MPI Forum: Rusty Lusk, Convener; Steve Huss-Lederman, Editor; Bill Saphir, Chapter Author, Dynamic Processes; Marc Snir, Chapter Author, One Sided Communication; Tony Skjellum, Chapter Author, Real Time; Bill Nitzberg, Chapter Author, Parallel I/O. The workshop will also be attended by European members of the MPI Forum. We plan to include a session for vendor and third party developers. The workshop will provide an excellent opportunity for users and developers to learn more about MPI-2 and meet with members of the MPI-2 Forum and the MPI developer community. The workshop is also an excellent and timely opportunity to provide feedback to the MPI-2 Forum. We welcome offers to contribute a short user perspective presentation and will schedule a user perspective session if there is sufficient interest. If you wish to contribute to a user perspective session, please check the box on the registration form. Who should attend: European software developers with experience in message passing programming or tool development. Participants from universities, research organizations, and industry are welcome to attend. Venue: John MacIntrye Centre, Pollock Halls, The University of Edinburgh. Fees: GBP 50 (to be paid in cash at registration). Fees include refreshments and lunch. Programme: To be announced, the programme will appear on the World Wide Web at . Registration ------------ To register for the workshop, please use the form on the World Wide Web at , or e-mail the attached form to . Accommodation ------------- If you need help with accommodation, please fill out the arrival and departure information on the registration form. We will send you an accommodation booking form, and we will attempt to place you in a hotel close to the location of the events. (The typical cost of a single room is GBP 20-30.) Please note that the cost of accommodation is not included in the registration fee. Information ----------- For further information, please check the information on the WWW, or e-mail to . ----------------------------------------------------------- %< Cut Here +-------------------------------------------------------+ | MPI Workshop Registration | +-------------------------------------------------------+ --------------------- Start of registration information --------------------- Title : First Name : Surname : Email : Department : Country : Phone : Fax : Address ------- Accommodation ------------- O I would like help with accommodation arrangements O I will make my own accommodation arrangements Presentations -------------- O I would like to contribute to a user perspective session. Topic: -------------------- End of registration information -------------------- ----------------------------------------------------------- %< Cut Here From mpi-core-human@mcs.anl.gov Wed Nov 27 08:51:44 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA29179; Wed, 27 Nov 1996 08:51:43 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA08746 for mpi-core-out; Wed, 27 Nov 1996 07:53:23 -0600 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA08741 for ; Wed, 27 Nov 1996 07:53:20 -0600 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.7.6/8.7.3) id HAA13233; Wed, 27 Nov 1996 07:53:19 -0600 (CST) Date: Wed, 27 Nov 1996 07:53:19 -0600 (CST) Message-Id: <199611271353.HAA13233@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: timeline for January meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy MPIers! With the conclusion of SC96, it is time to begin thinking about continued work on the MPI-2 standard. So everyone is aware of the timeline, here is the schedule for the January meeting: Tuesday, 7 January: A first cut at the next version of each chapter is due to me by 12:00 PM Central US time (noon). When the full document is ready it will be announced on mpi-core and should be out by late afternoon. Tuesday, 14 January: The final version of each chapter is due to me by 10:00 AM Central US time. This is the version that will be printed for the meeting. It will also be announced on mpi-core and should be out by late afternoon. Tuesday, 21 January: MPI Forum meeting begins. Seeing light at the end of the tunnel... Steve From mpi-core-human@mcs.anl.gov Mon Dec 9 12:43:36 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA14009; Mon, 9 Dec 1996 12:43:33 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA29392 for mpi-core-out; Mon, 9 Dec 1996 11:41:11 -0600 Received: from win44.nas.nasa.gov (win44.nas.nasa.gov [129.99.33.40]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA29387 for ; Mon, 9 Dec 1996 11:41:08 -0600 Received: (from parkson@localhost) by win44.nas.nasa.gov (8.8.3/NAS.6.1) id JAA11856 for mpi-core@mcs.anl.gov; Mon, 9 Dec 1996 09:41:06 -0800 (PST) From: parkson@nas.nasa.gov (Parkson Wong) Message-Id: <199612091741.JAA11856@win44.nas.nasa.gov> Subject: MPI_TYPE_DUP To: mpi-core@mcs.anl.gov Date: Mon, 9 Dec 1996 09:41:05 -0800 (PST) X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi: I would like to propose a routine be added to the external or misc chapter as people see fit: MPI_TYPE_DUP(dtype, newdtype) IN dtype data type (handle) OUT newdtype copy of dtype (handle) int MPI_Type_dup(MPI_Datatype dtype, MPI_Datatype *newdtype) MPI_TYPE_DUP(DTYPE, NEWDTYPE, IERROR) INTEGER DTYPE, NEWDTYPE, IERROR MPI_Type_dup duplicates the existing datatype dtype and returns it in newdtype. Newdtype has the same typemap as dtype, and if operated on by the low level datatype rountines will produces identical results. Motivation: Most of the MPI implementation keeps a reference count for the datatypes internally, so this function should be very cheap. If implementated by a layered library, like the I/O routines, it is very expensive. Even though I/O is slow to begin with, I hate to make things slower than necessay. A MPI_Type_contiguous of count one is not the same datatype even though they has the same typemap. -- parkson -- Parkson Wong Address: Numerical Aerodynamic Simulation MRJ, Inc. NASA Ames Research Center M/S 258-6 Supercomputer Applications Segment Moffett Field, CA 94035-1000 e-mail: parkson@nas.nasa.gov Phone: (415)604-3988 Fax: (415)966-8669 From mpi-core-human@mcs.anl.gov Mon Dec 9 20:57:11 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA14552; Mon, 9 Dec 1996 20:57:10 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA09069 for mpi-core-out; Mon, 9 Dec 1996 19:59:09 -0600 Received: from palrel1.hp.com (palrel1.hp.com [15.253.72.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA09064 for ; Mon, 9 Dec 1996 19:59:06 -0600 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com with ESMTP (8.7.5/8.7.3) id RAA12606; Mon, 9 Dec 1996 17:59:03 -0800 (PST) Message-Id: <199612100159.RAA12606@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA072803142; Mon, 9 Dec 1996 19:59:02 -0600 From: Raja Daoud Subject: Re: timeline for January meeting To: lederman@cs.wisc.edu (Steve Huss-Lederman) Date: Mon, 09 Dec 1996 19:59:02 CST Cc: mpi-core@mcs.anl.gov In-Reply-To: <199611271353.HAA13233@romano.cs.wisc.edu>; from "Steve Huss-Lederman" at Nov 27, 96 7:53 am X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Tuesday, 21 January: MPI Forum meeting begins. Steve, I may be mistaken, but I was under the impression that we voted to extend the pre-SC'96 MPI meetings to 4 days, but not the 1997 meetings. Is the date above correct or should it start on Wed.? Thanks, --Raja From mpi-core-human@mcs.anl.gov Mon Dec 9 22:30:26 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA15325; Mon, 9 Dec 1996 22:30:26 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA10128 for mpi-core-out; Mon, 9 Dec 1996 21:34:40 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA10123; Mon, 9 Dec 1996 21:34:37 -0600 Message-Id: <199612100334.VAA10123@antares.mcs.anl.gov> To: Raja Daoud Cc: mpi-core@antares.mcs.anl.gov Subject: Re: timeline for January meeting In-reply-to: Your message of "Mon, 09 Dec 1996 19:59:02 CST." <199612100159.RAA12606@palrel1.hp.com> Date: Mon, 09 Dec 1996 21:34:37 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | I may be mistaken, but I was under the impression that we voted to | extend the pre-SC'96 MPI meetings to 4 days, but not the 1997 meetings. | Is the date above correct or should it start on Wed.? The January meeting is an unusual one, due to scheduling difficulties at the hotel. It starts on Tuesday at 1:30 and ends on Thursday at noon. Rusty From mpi-core-human@mcs.anl.gov Wed Dec 11 01:58:21 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id BAA18784; Wed, 11 Dec 1996 01:58:20 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA03795 for mpi-core-out; Wed, 11 Dec 1996 01:00:28 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA03790 for ; Wed, 11 Dec 1996 01:00:24 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id XAA11401 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Tue, 10 Dec 1996 23:00:20 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id WAA05517; Tue, 10 Dec 1996 22:58:09 -0800 From: "Eric Salo" Message-Id: <9612102258.ZM5515@mrjones.engr.sgi.com> Date: Tue, 10 Dec 1996 22:58:09 -0800 In-Reply-To: parkson@nas.nasa.gov (Parkson Wong) "MPI_TYPE_DUP" (Dec 9, 9:41am) References: <199612091741.JAA11856@win44.nas.nasa.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: MPI_TYPE_DUP Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I would like to propose a routine be added to the external or misc > chapter as people see fit: > > MPI_TYPE_DUP(dtype, newdtype) > IN dtype data type (handle) > OUT newdtype copy of dtype (handle) Seems harmless enough. For implementations, it would mean adding a reference count (which is probably already there anyway) and some sort of indirection mechanism to point the new type at the original (which is also probably there already). So if we give you this, then you won't be needing the ability to cache attributes on datatypes, right? :-) -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Wed Dec 11 12:34:46 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA28016; Wed, 11 Dec 1996 12:34:43 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA12035 for mpi-core-out; Wed, 11 Dec 1996 11:35:54 -0600 Received: from win44.nas.nasa.gov (win44.nas.nasa.gov [129.99.33.40]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA12030 for ; Wed, 11 Dec 1996 11:35:51 -0600 Received: (from parkson@localhost) by win44.nas.nasa.gov (8.8.3/NAS.6.1) id JAA25474; Wed, 11 Dec 1996 09:32:05 -0800 (PST) From: parkson@nas.nasa.gov (Parkson Wong) Message-Id: <199612111732.JAA25474@win44.nas.nasa.gov> Subject: Re: MPI_TYPE_DUP To: salo@mrjones.engr.sgi.com (Eric Salo) Date: Wed, 11 Dec 1996 09:32:04 -0800 (PST) Cc: mpi-core@mcs.anl.gov In-Reply-To: <9612102258.ZM5515@mrjones.engr.sgi.com> from "Eric Salo" at Dec 10, 96 10:58:09 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > > I would like to propose a routine be added to the external or misc > > chapter as people see fit: > > > > MPI_TYPE_DUP(dtype, newdtype) > > IN dtype data type (handle) > > OUT newdtype copy of dtype (handle) > > Seems harmless enough. For implementations, it would mean adding a reference > count (which is probably already there anyway) and some sort of indirection > mechanism to point the new type at the original (which is also probably there > already). > > So if we give you this, then you won't be needing the ability to cache > attributes on datatypes, right? :-) > Eric, you are so smooth. :-) With this, at least I could build my own cache. The only problem is that this cache is not really portable. There is no guarantee that the implementation will use reference count, it may choose to make an exact copy of the datatype and return it to be, so when the application pass the old filetype to me, it is different from (but has the same type map as) the copy I have and cached. There are also other uses of the cache, especially with the complicated type constructors (helper functions) currently listed in the I/O chapters. When those functions return the datatype, it knows what it created and could tag them so when the I/O library sees them again later, could identify the I/O access pattern easier. I will take anything I could get, so give me MPI_Type_dup first. It is needed even if we have caching attributes on datatype. After the I/O routines returns, the applications should be allowed to free the datatypes, so the I/O routines need to make a copy of the datatype. It is less error prone this way, otherwise we need to specify in the spec that the application cannot free the datatype until they closed the file. -- parkson -- Parkson Wong Address: Numerical Aerodynamic Simulation MRJ, Inc. NASA Ames Research Center M/S 258-6 Supercomputer Applications Segment Moffett Field, CA 94035-1000 e-mail: parkson@nas.nasa.gov Phone: (415)604-3988 Fax: (415)966-8669 From mpi-core-human@mcs.anl.gov Wed Dec 11 13:08:17 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA28681; Wed, 11 Dec 1996 13:08:16 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA13000 for mpi-core-out; Wed, 11 Dec 1996 12:12:28 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA12995 for ; Wed, 11 Dec 1996 12:12:25 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id KAA01210 for ; Wed, 11 Dec 1996 10:11:50 -0800 (PST) Message-ID: Date: 11 Dec 1996 10:09:59 -0800 From: "Lewins, Lloyd J" Subject: RE: MPI_TYPE_DUP To: "Eric Salo" , "mpi-core" X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk We need to define the semantics of this operation, NOT one possible implementation. Following MPI-1, I would assume the semantics to be something like: "MPI_TYPE_DUP creates a new MPI type, whose type map is the same as the type map of the old type." I don't think this is what what Parkson Wong wants however!! He wants a function which increments the reference count, and returns the same handle. However, this assumes a particular implementation -- an assumption which may not be valid. The ripple on effect of redefining the semantics of all handle types might be significant. Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com _______________________________________________________________________________ From: Eric Salo on Tue, Dec 10, 1996 11:07 PM Subject: Re: MPI_TYPE_DUP To: mpi-core@mcs.anl.gov > I would like to propose a routine be added to the external or misc > chapter as people see fit: > > MPI_TYPE_DUP(dtype, newdtype) > IN dtype data type (handle) > OUT newdtype copy of dtype (handle) Seems harmless enough. For implementations, it would mean adding a reference count (which is probably already there anyway) and some sort of indirection mechanism to point the new type at the original (which is also probably there already). So if we give you this, then you won't be needing the ability to cache attributes on datatypes, right? :-) -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" ------------------ RFC822 Header Follows ------------------ Received: by MSMAIL4.HAC.COM with SMTP;10 Dec 1996 23:06:58 -0800 Received: from antares.mcs.anl.gov (140.221.10.64) by EDEN.HAC.COM (PMDF V4.3-7 #5884) id <01ICV1SSDQAO00LV6R@EDEN.HAC.COM>; Tue, 10 Dec 1996 23:06:54 PST Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA03795 for mpi-core-out; Wed, 11 Dec 1996 01:00:28 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA03790 for ; Wed, 11 Dec 1996 01:00:24 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id XAA11401 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Tue, 10 Dec 1996 23:00:20 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id WAA05517; Tue, 10 Dec 1996 22:58:09 -0800 Date: Tue, 10 Dec 1996 22:58:09 -0800 From: Eric Salo Subject: Re: MPI_TYPE_DUP In-reply-to: parkson@nas.nasa.gov (Parkson Wong) "MPI_TYPE_DUP" (Dec 9, 9:41am) Sender: owner-mpi-core@mcs.anl.gov To: mpi-core@mcs.anl.gov Message-id: <9612102258.ZM5515@mrjones.engr.sgi.com> MIME-version: 1.0 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Precedence: bulk References: <199612091741.JAA11856@win44.nas.nasa.gov> From mpi-core-human@mcs.anl.gov Wed Dec 11 13:31:45 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA29795; Wed, 11 Dec 1996 13:31:44 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA13568 for mpi-core-out; Wed, 11 Dec 1996 12:34:14 -0600 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA13562; Wed, 11 Dec 1996 12:34:10 -0600 Message-Id: <199612111834.MAA13562@antares.mcs.anl.gov> To: "Lewins, Lloyd J" cc: "Eric Salo" , "mpi-core" Subject: Re: MPI_TYPE_DUP In-Reply-to: Message from "Lewins, Lloyd J" of "11 Dec 1996 10:09:59 -0800." Date: Wed, 11 Dec 1996 12:34:11 -0600 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | We need to define the semantics of this operation, NOT one possible | implementation. I agree with Lloyd, though this is likely to cause the semantics to severely restrict the implementation choices, as several MPI 1.1 choices have done. Bill From mpi-core-human@mcs.anl.gov Wed Dec 11 13:40:12 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA29967; Wed, 11 Dec 1996 13:40:09 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA13764 for mpi-core-out; Wed, 11 Dec 1996 12:44:22 -0600 Received: from win44.nas.nasa.gov (win44.nas.nasa.gov [129.99.33.40]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA13758 for ; Wed, 11 Dec 1996 12:44:18 -0600 Received: (from parkson@localhost) by win44.nas.nasa.gov (8.8.3/NAS.6.1) id KAA25749; Wed, 11 Dec 1996 10:39:11 -0800 (PST) From: parkson@nas.nasa.gov (Parkson Wong) Message-Id: <199612111839.KAA25749@win44.nas.nasa.gov> Subject: Re: MPI_TYPE_DUP To: llewins@msmail4.hac.com (Lewins, Lloyd J) Date: Wed, 11 Dec 1996 10:39:10 -0800 (PST) Cc: salo@mrjones.engr.sgi.com, mpi-core@mcs.anl.gov In-Reply-To: from "Lewins, Lloyd J" at Dec 11, 96 10:09:59 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > We need to define the semantics of this operation, NOT one possible > implementation. > > Following MPI-1, I would assume the semantics to be something like: > > "MPI_TYPE_DUP creates a new MPI type, whose type map is the same as the > type > map of the old type." > > I don't think this is what what Parkson Wong wants however!! He wants a > function > which increments the reference count, and returns the same handle. However, > this assumes a particular implementation -- an assumption which may not be > valid. The ripple on effect of redefining the semantics of all handle types > might be significant. > > Lloyd Lewins > Hughes Aircraft Co., > llewins@msmail4.hac.com > _______________________________________________________________________________ The intent is to have MPI_Type_dup return a new copy of the datatype if I was not clear on the original message. The implementation could choose to use reference count, but is not mandated to do so. At least I am not forced to do a disassemble and reassemble of the datatype while it could easily be done by adding one to a reference count that already exist in most implementation. This is the basic functionality that I needed. Now, onto what I really want. I really want the reference count so that the new and old datatype actually is the same one so that I could do attribute caching myself as well as keeping the applications' datatypes around. However I am not requiring that MPI_Type_dup been implemented with reference count. And of course, no one wants to redefine the semantics of all handle types. The portable code will look something like rc = MPI_Type_dup(filetype, &my_ftype); if (my_ftype == filetype) { /* great, they are actually the same */ } else { /* Oh well, I lose, do something funcky */ /* * Or I am happy to turn all optimization with * datatype off. However I still have a valid * copy of the datatype, and the application could * free his copy. */ } Hopefully, most implementation will implement it with a reference count and I don't need to exercise the funcky code too often. -- parkson -- Parkson Wong Address: Numerical Aerodynamic Simulation MRJ, Inc. NASA Ames Research Center M/S 258-6 Supercomputer Applications Segment Moffett Field, CA 94035-1000 e-mail: parkson@nas.nasa.gov Phone: (415)604-3988 Fax: (415)966-8669 From mpi-core-human@mcs.anl.gov Wed Dec 11 13:41:06 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA29976; Wed, 11 Dec 1996 13:41:03 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA13791 for mpi-core-out; Wed, 11 Dec 1996 12:45:21 -0600 Received: from win44.nas.nasa.gov (win44.nas.nasa.gov [129.99.33.40]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA13785; Wed, 11 Dec 1996 12:45:17 -0600 Received: (from parkson@localhost) by win44.nas.nasa.gov (8.8.3/NAS.6.1) id KAA25782; Wed, 11 Dec 1996 10:45:18 -0800 (PST) From: parkson@nas.nasa.gov (Parkson Wong) Message-Id: <199612111845.KAA25782@win44.nas.nasa.gov> Subject: Re: MPI_TYPE_DUP To: gropp@mcs.anl.gov (William Gropp) Date: Wed, 11 Dec 1996 10:45:18 -0800 (PST) Cc: llewins@msmail4.hac.com, salo@mrjones.engr.sgi.com, mpi-core@mcs.anl.gov In-Reply-To: <199612111834.MAA13562@antares.mcs.anl.gov> from "William Gropp" at Dec 11, 96 12:34:11 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > | We need to define the semantics of this operation, NOT one possible > | implementation. > > I agree with Lloyd, though this is likely to cause the semantics to severely > restrict the implementation choices, as several MPI 1.1 choices have done. > > Bill > A new copy does not necessarily mean a distinct copy, so the implementation should still be able to use a reference count. Am I still missing something? -- parkson -- Parkson Wong Address: Numerical Aerodynamic Simulation MRJ, Inc. NASA Ames Research Center M/S 258-6 Supercomputer Applications Segment Moffett Field, CA 94035-1000 e-mail: parkson@nas.nasa.gov Phone: (415)604-3988 Fax: (415)966-8669 From mpi-core-human@mcs.anl.gov Wed Dec 11 13:53:38 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA00303; Wed, 11 Dec 1996 13:53:37 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA14113 for mpi-core-out; Wed, 11 Dec 1996 12:57:39 -0600 Received: from win44.nas.nasa.gov (win44.nas.nasa.gov [129.99.33.40]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA14108; Wed, 11 Dec 1996 12:57:36 -0600 Received: (from parkson@localhost) by win44.nas.nasa.gov (8.8.3/NAS.6.1) id KAA25782; Wed, 11 Dec 1996 10:45:18 -0800 (PST) From: parkson@nas.nasa.gov (Parkson Wong) Message-Id: <199612111845.KAA25782@win44.nas.nasa.gov> Subject: Re: MPI_TYPE_DUP To: gropp@mcs.anl.gov (William Gropp) Date: Wed, 11 Dec 1996 10:45:18 -0800 (PST) Cc: llewins@msmail4.hac.com, salo@mrjones.engr.sgi.com, mpi-core@mcs.anl.gov In-Reply-To: <199612111834.MAA13562@antares.mcs.anl.gov> from "William Gropp" at Dec 11, 96 12:34:11 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > | We need to define the semantics of this operation, NOT one possible > | implementation. > > I agree with Lloyd, though this is likely to cause the semantics to severely > restrict the implementation choices, as several MPI 1.1 choices have done. > > Bill > A new copy does not necessarily mean a distinct copy, so the implementation should still be able to use a reference count. Am I still missing something? -- parkson -- Parkson Wong Address: Numerical Aerodynamic Simulation MRJ, Inc. NASA Ames Research Center M/S 258-6 Supercomputer Applications Segment Moffett Field, CA 94035-1000 e-mail: parkson@nas.nasa.gov Phone: (415)604-3988 Fax: (415)966-8669 From mpi-core-human@mcs.anl.gov Wed Dec 11 14:35:25 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA00724; Wed, 11 Dec 1996 14:35:22 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA15474 for mpi-core-out; Wed, 11 Dec 1996 13:39:29 -0600 Received: from win44.nas.nasa.gov (win44.nas.nasa.gov [129.99.33.40]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA15469 for ; Wed, 11 Dec 1996 13:39:26 -0600 Received: (from parkson@localhost) by win44.nas.nasa.gov (8.8.3/NAS.6.1) id LAA25920; Wed, 11 Dec 1996 11:35:13 -0800 (PST) From: parkson@nas.nasa.gov (Parkson Wong) Message-Id: <199612111935.LAA25920@win44.nas.nasa.gov> Subject: Re: MPI_TYPE_DUP (Actually, what I really wanted) To: llewins@msmail4.hac.com (Lewins Lloyd J) Date: Wed, 11 Dec 1996 11:35:12 -0800 (PST) Cc: salo@mrjones.engr.sgi.com, mpi-core@mcs.anl.gov In-Reply-To: from "Lewins, Lloyd J" at Dec 11, 96 10:09:59 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi everyone: Maybe it is time that I stated what the problem is and see if people have ideas other than MPI_Type_dup. What the problem is on MPI_Read or MPI_Write a buftype is passed in to describe the memory layout, the code could be optimized if the same buftype is used over and over again, which is probably going to be the common case. With collective buffering, the buftype needed to be sent to all the nodes that is buffering the data, and so on so forth (quite a bit of work and communication need to be done). What is needed is a unique identifier of the datatype so that the I/O code could remember the unique identifier and match it with the next read/write call. If the buftype is free by the application and recreated a new buftype, they need to be different. So if this datatype unique identifier wrapped around, I need to be notified. Possible solutions: 1. Give me this unique identifier. 2. cache my own unique identifier to the datatype, so when I was called again, I could check the unique identifier. 3. Give me a reference count on the datatype, I increment the reference count to make sure it won't go away, so that I could do the pointer comparison to see if the buftype is the same. 4. Give me this MPI_Type_dup, and if it is not implemented by a reference count, I will punt. 5. Other suggestions, ideas are welcome. -- parkson -- Parkson Wong Address: Numerical Aerodynamic Simulation MRJ, Inc. NASA Ames Research Center M/S 258-6 Supercomputer Applications Segment Moffett Field, CA 94035-1000 e-mail: parkson@nas.nasa.gov Phone: (415)604-3988 Fax: (415)966-8669 From mpi-core-human@mcs.anl.gov Wed Dec 11 14:46:08 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA00862; Wed, 11 Dec 1996 14:46:01 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA15728 for mpi-core-out; Wed, 11 Dec 1996 13:49:57 -0600 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA15723; Wed, 11 Dec 1996 13:49:53 -0600 Date: Wed, 11 Dec 1996 13:49:55 -0600 Message-Id: <199612111949.NAA07643@abacus> From: Rajeev Thakur To: parkson@nas.nasa.gov CC: llewins@msmail4.hac.com, salo@mrjones.engr.sgi.com, mpi-core@mcs.anl.gov In-reply-to: <199612111935.LAA25920@win44.nas.nasa.gov> (parkson@nas.nasa.gov) Subject: Re: MPI_TYPE_DUP (Actually, what I really wanted) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Well, it seems to me that what you really need is only attribute caching on the datatype :-) When the buftype is passed to you for the first time, you create whatever extra information out of it and cache it on the datatype itself. Whenever it is passed again, you can just check the cache. When the datatype is freed, the cached information is also freed, so there is no issue of using the old information for a new datatype. I need MPI_Type_dup for a different reason: The etype and filetype are specified only at open time and used thereafter. For safety, I need to maintain a copy of them, because the user could free them while the file is still open. I currently get around it by using MPI_Type_contiguous with count=1, but MPI_Type_Dup would be cleaner. If we were to define a new MPI_Type_dup function, we cannot say anything about reference count in the definition, because that specifies a particular implementation. Therefore, we also cannot assume that the new handle returned by such a function will be identical to the old handle. Rajeev > From: parkson@nas.nasa.gov (Parkson Wong) > Date: Wed, 11 Dec 1996 11:35:12 -0800 (PST) > Cc: salo@mrjones.engr.sgi.com, mpi-core@mcs.anl.gov > X-Mailer: ELM [version 2.4 PL25] > MIME-Version: 1.0 > Content-Type: text/plain; charset=US-ASCII > Content-Transfer-Encoding: 7bit > Sender: owner-mpi-core@mcs.anl.gov > Precedence: bulk > > Hi everyone: > > Maybe it is time that I stated what the problem is and see if people have > ideas other than MPI_Type_dup. > > What the problem is on MPI_Read or MPI_Write a buftype is passed in to > describe the memory layout, the code could be optimized if the same buftype > is used over and over again, which is probably going to be the common case. > With collective buffering, the buftype needed to be sent to all the nodes > that is buffering the data, and so on so forth (quite a bit of work and > communication need to be done). > > What is needed is a unique identifier of the datatype so that the I/O code > could remember the unique identifier and match it with the next read/write > call. If the buftype is free by the application and recreated a new buftype, > they need to be different. So if this datatype unique identifier wrapped > around, I need to be notified. > > Possible solutions: > > 1. Give me this unique identifier. > > 2. cache my own unique identifier to the datatype, so when I was called > again, I could check the unique identifier. > > 3. Give me a reference count on the datatype, I increment the reference > count to make sure it won't go away, so that I could do the pointer > comparison to see if the buftype is the same. > > 4. Give me this MPI_Type_dup, and if it is not implemented by a reference > count, I will punt. > > 5. Other suggestions, ideas are welcome. > > -- parkson From mpi-core-human@mcs.anl.gov Thu Dec 19 12:05:10 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA03974; Thu, 19 Dec 1996 12:05:08 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA01114 for mpi-core-out; Thu, 19 Dec 1996 11:07:46 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA01094 for ; Thu, 19 Dec 1996 11:07:40 -0600 Message-Id: <199612191707.LAA01094@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Spring meeting dates Date: Thu, 19 Dec 1996 11:07:38 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Spring MPI Forum meetings are Jan 21-23 (yes, that is Tues-Wed-Thurs, because of scheduling problems) Mar 5-7 Apr 23-25 - Rusty From mpi-core-human@mcs.anl.gov Fri Dec 20 09:20:49 1996 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA22751; Fri, 20 Dec 1996 09:20:47 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA16326 for mpi-core-out; Fri, 20 Dec 1996 08:24:53 -0600 Received: from acacia.sucs.soton.ac.uk (acacia.sucs.soton.ac.uk [152.78.128.232]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA16275; Fri, 20 Dec 1996 08:22:35 -0600 Received: from bright.ecs.soton.ac.uk (bright.ecs.soton.ac.uk [152.78.64.201]) by acacia.sucs.soton.ac.uk (8.8.2/server) with SMTP id OAA00851; Fri, 20 Dec 1996 14:22:22 GMT Received: from bacchus by bright.ecs.soton.ac.uk; Fri, 20 Dec 96 14:21:39 GMT Message-Id: <32BAA133.59E2B600@ecs.soton.ac.uk> Date: Fri, 20 Dec 1996 14:22:43 +0000 From: Robin Allen Organization: ECS X-Mailer: Mozilla 2.0 (X11; I; SunOS 4.1.3_U1 sun4c) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Cc: mpi-coll@mcs.anl.gov, mpi-comm@mcs.anl.gov, hpff@cs.rice.edu Subject: Announcement: Upcoming Software Tools Workshop Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi all. This is to announce an upcoming European software tools workshop, Brussels, 5 February 1997. If you are interested in attending, please reply to the address at the end of the message, not the mail address of this message! Thanks. European Commission workshop for High-Performance Computing and Networking (HPCN) Software Tools Brussels, 5 February 1997 The European Commission recently engaged Smith System Engineering and the PAC to undertake a survey of HPCN software tools. The Commission's objectives in performing this survey were to assess the current status of HPCN tools and to develop a strategy for further investment and exploitation. The results of the survey will be presented by Smith and the PAC at the forthcoming workshop in Brussels. Details of the venue and agenda will be distributed in January. In addition to providing feedback on the findings of the survey, the workshop will be used to stimulate discussion on the 'way ahead' for European HPCN software tools. In particular, it will provide a forum for debate on software tool development, the EC's framework programme and the newly forming Technology Transfer Node (TTN) network. The workshop will last one day with the morning dedicated to presentation of the study's results and the EC's perspectives. In the afternoon, two parallel sessions will debate the current issues in HPCN software tool development. The findings of the two sessions will then be consolidated and discussed. If you are interested in taking part in the workshop please mail your details To: Dr Daron Green Fax: +44 1483 442304 Name: Address: Company: Tel: email: Stuart A Haire System Engineer ///////////////////////////////// / Smith System Engineering Ltd / / Surrey Research Park / / Guildford / / Surrey / / GU2 5YP / / e-mail: sahaire@smithsys.co.uk/ / tel: +44 (0)1483 442103 / / fax: +44 (0)1483 442304 / / mobile: 0468 946960 / ///////////////////////////////// From mpi-core-human@mcs.anl.gov Sat Jan 4 15:08:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA18327; Sat, 4 Jan 1997 15:08:20 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA08762 for mpi-core-out; Sat, 4 Jan 1997 14:10:45 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA08757 for ; Sat, 4 Jan 1997 14:10:42 -0600 Message-Id: <199701042010.OAA08757@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: January MPI Forum meeting Date: Sat, 04 Jan 1997 14:10:41 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, The next MPI Forum meeting is January 21-23 (Tues-Wed-Thurs) at the Ramada O'Hare in Chicago (the usual place). This is a midweek meeting because of scheduling restrictions at the hotel. As agreed, we return now to the normal schedule of three-day meetings (acutally, one full day and two half-days) instead of the four-day meetings we had in the fall. The registration fee will revert to $75. Please let me know now for the head count whether you are coming. Thanks. Rusty --------------------------------------------------------------------------- The next meeting of the MPI Forum will take place on Tues.-Thurs. Jan. 21-23. at the Ramada Hotel O'Hare (same place as the last few meetings). It is time to make your reservations. Register at the hotel by calling (847) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. The registration fee will be $75, which should be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. If you have any problems or questions, let me know, either by email (lusk@mcs.anl.gov) or phone (708-252-7852). The last two MPI Forum meetings are March 5-7 and April 23-25. Regards, Rusty From mpi-core-human@mcs.anl.gov Mon Jan 6 11:03:44 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA00774; Mon, 6 Jan 1997 11:03:43 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA27181 for mpi-core-out; Mon, 6 Jan 1997 10:05:15 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA27175 for ; Mon, 6 Jan 1997 10:05:11 -0600 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA28320; Mon, 6 Jan 1997 10:59:27 -0500 Message-Id: <32D122B4.41C6@kgn.ibm.com> Date: Mon, 06 Jan 1997 11:05:08 -0500 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.0 (X11; I; AIX 1) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: Public comment on MPI-2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am getting very curious about whether there has been any public reaction to the MPI-2 draft presented at SC-96. Right after SC96 I susbcribed to the mpi-comment list which was suggested as one means of public comment. To date, the only mail I have seen on it is the subscription confirmation. SO - Is there some body of comments being collected? Are other forum members getting comments directly? Or, has the public reaction been thunderous silence? Happy New Year - Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Mon Jan 6 12:14:30 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA01609; Mon, 6 Jan 1997 12:14:29 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA28657 for mpi-core-out; Mon, 6 Jan 1997 11:16:59 -0600 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA28651 for ; Mon, 6 Jan 1997 11:16:53 -0600 Received: from jim (gw2-163.pool.dircon.co.uk [194.112.35.163]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id RAA15657; Mon, 6 Jan 1997 17:19:00 GMT Message-Id: <199701061719.RAA15657@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id RAA05856; Mon, 6 Jan 1997 17:15:16 GMT To: Dick Treumann cc: mpi-core@mcs.anl.gov Subject: Re: Public comment on MPI-2 In-reply-to: Your message of "Mon, 06 Jan 1997 11:05:08 EST." <32D122B4.41C6@kgn.ibm.com> Date: Mon, 06 Jan 1997 17:15:16 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > SO - Is there some body of comments being collected? Are other forum > members getting comments directly? Or, has the public reaction been > thunderous silence? Right, we must be getting to the point where we can say "So, no-one *really* wants any of this !" and then we can go for the big simplification. (After all Steve is used to having his chapter removed, so now that he's editor of the whole document...) -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Mon Jan 6 14:12:33 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA03456; Mon, 6 Jan 1997 14:12:27 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA00961 for mpi-core-out; Mon, 6 Jan 1997 13:15:55 -0600 Received: from relay1.UU.NET (relay1.UU.NET [192.48.96.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA00950 for ; Mon, 6 Jan 1997 13:15:49 -0600 Received: from sky.com by relay1.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQbxiz00604; Mon, 6 Jan 1997 14:15:22 -0500 (EST) Received: from anna.sky.com by sky.com (SMI-8.6/SMI-SVR4) id OAA12874; Mon, 6 Jan 1997 14:14:19 -0500 Received: from anna by anna.sky.com (SMI-8.6/SMI-SVR4) id OAA02974; Mon, 6 Jan 1997 14:11:44 -0500 Message-ID: <32D14E6F.F7A@sky.com> Date: Mon, 06 Jan 1997 14:11:43 -0500 From: Anna Rounbehler Organization: Sky Computers X-Mailer: Mozilla 3.0Gold (X11; I; SunOS 5.5.1 sun4c) MIME-Version: 1.0 To: Dick Treumann CC: mpi-core@mcs.anl.gov Subject: Re: Public comment on MPI-2 References: <32D122B4.41C6@kgn.ibm.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dick Treumann wrote: > > I am getting very curious about whether there has been any public > reaction to the MPI-2 draft presented at SC-96. Right after SC96 I > susbcribed to the mpi-comment list which was suggested as one means of > public comment. To date, the only mail I have seen on it is the > subscription confirmation. > > SO - Is there some body of comments being collected? Are other forum > members getting comments directly? Or, has the public reaction been > thunderous silence? > > Happy New Year - Dick > > -- > Dick Treumann IBM RS/6000 Division > (Internet) treumann@pok.ibm.com Poughkeepsie, NY > (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 > (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 Most likely, the public is still digesting the 380 pages. Also,there were major holidays in November-December and that always sets things back. The absence of negative comments can be a very good indicator. In the past, when I have asked application developers to review a standard, they often can not commit the time. In addition, they are assuming that the implementors are looking out for their interests by participating in the standards committee. Were you hoping for comments from users and/or the academic community? Regards, ---------------------- Anna Rounbehler SKY Computers anna@sky.com From mpi-core-human@mcs.anl.gov Mon Jan 6 18:58:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA07801; Mon, 6 Jan 1997 18:58:23 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA06968 for mpi-core-out; Mon, 6 Jan 1997 17:59:02 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA06955; Mon, 6 Jan 1997 17:58:55 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id PAA13556; Mon, 6 Jan 1997 15:58:54 -0800 (PST) Date: Mon, 6 Jan 1997 15:58:54 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701062358.PAA13556@win182.nas.nasa.gov> To: Dick Treumann , Anna Rounbehler Subject: Re: Public comment on MPI-2 Cc: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov References: <32D122B4.41C6@kgn.ibm.com> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Anna Rounbehler wrote: > Dick Treumann wrote: > > > > I am getting very curious about whether there has been any public > > reaction to the MPI-2 draft presented at SC-96. Right after SC96 I > > susbcribed to the mpi-comment list which was suggested as one means of > > public comment. To date, the only mail I have seen on it is the > > subscription confirmation. > > > > SO - Is there some body of comments being collected? Are other forum > > members getting comments directly? Or, has the public reaction been > > thunderous silence? Judging from the total lack of comments and discussion about MPI-2 on the comp.parallel.mpi newsgroup, I would imagine that either (a) most of that group does not even know that there is a public comment period, or (b) they just assume that the forum must know what it is doing. (By the way, it appeared that a large percentage (40%?) of the audience at the SC-96 session were present or former members of the MPI forum.) > Most likely, the public is still digesting the 380 pages. Most likely, hardly anyone is digesting those pages. I would guess that there are only a few members of the forum who have truly digested and understood the entire 380 pages. And if you only read part of it and find it troublesome, you may have the feeling that it must be that way because of something in some other part, or some minor point that you don't understand (and in some cases, you'd be right). > The absence of negative comments can be a very good indicator. I suppose through the right glasses, almost anything can look like a good indicator. > Were you hoping for comments from users and/or the academic community? Perhaps this question should be directed to the forum. If they are not concerned with the lack of public comment, one might ask why they requested it. (I think it's like the question in the wedding ceremony where the audience is given the opportunity to object to the wedding, but nobody ever does.) For my part, I still have serious concerns about the one-sided chapter, but I believe that I have expressed them clearly at the meetings and to the one-sided list, and have even made an alternative proposal available on a web page. I could hope that external reviewers would take the last two into consideration, but I don't think it can be expected that all of us with grievances will repeat all of our concerns in the public forum. Without access to the discussions within the meetings, outside reviewers are at an extreme disadvantage when understanding the pros and cons of the various features, and I would guess that they know this. Even if all of the meeting's minutes were available, reviewers could not be expected to read and understand all of them and all of the mailing list archives during this public review period. I think that the best that can be expected is for the public reviewers to make comments based upon whether they are prepared to accept the standard in its present form, and to express the reasons that they will or will not do so. If all the reasons for accepting it are "Because it is MPI" or "Because it was formulated by experts who I trust", then it's acceptance won't really depend upon whether or not the forum did a good job in developing the standard, and there is little reason for the forum to have any more meetings. Rather than viewing the absence of negative comments as a very good indicator, it can also be viewed as the first hint of MPI's excessive size and complexity, especially related to its rather limited scope. End users are supposed to understand MPI just as a small adjunct to their real jobs as nuclear physicists or aerodynamicists or whatever. The forum seems to have shied away from any attempts to simplify MPI if it might result in even the slightest performance degradation on some (perhaps fictional) machines. I believe that, as a result, the forum has ended up with a standard which can theoretically be efficiently implemented on a large number of machines, but it doesn't really matter because few people with any real work to do will be able to learn to use it effectively. I have seen new architecture-independent graphical parallel languages described in 100 pages; the P4 book is about 270 pages; MPI, which only helps to write standardized message-passing programs, is 220 + 380 = 600 pages! Another communication interface which I have worked on has only 28 functions but is more portable than MPI and duplicates much of its functionality, while MPI has hundreds of function calls! The success of MPI will depend upon its ability to compete with these other approaches in the hands of scientists, not just in terms of efficiency of message-passing programs, but in flexibility and ease of use and learning. For these reasons, I doubt that I, personally, will be using MPI. Well, I guess that could be considered as public comment, so I'm sending this on to mpi-comment. -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Mon Jan 6 19:09:42 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA07948; Mon, 6 Jan 1997 19:09:40 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA07277 for mpi-core-out; Mon, 6 Jan 1997 18:13:19 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA07262; Mon, 6 Jan 1997 18:13:04 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id PAA13556; Mon, 6 Jan 1997 15:58:54 -0800 (PST) Date: Mon, 6 Jan 1997 15:58:54 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701062358.PAA13556@win182.nas.nasa.gov> To: Dick Treumann , Anna Rounbehler Subject: Re: Public comment on MPI-2 Cc: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov References: <32D122B4.41C6@kgn.ibm.com> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Anna Rounbehler wrote: > Dick Treumann wrote: > > > > I am getting very curious about whether there has been any public > > reaction to the MPI-2 draft presented at SC-96. Right after SC96 I > > susbcribed to the mpi-comment list which was suggested as one means of > > public comment. To date, the only mail I have seen on it is the > > subscription confirmation. > > > > SO - Is there some body of comments being collected? Are other forum > > members getting comments directly? Or, has the public reaction been > > thunderous silence? Judging from the total lack of comments and discussion about MPI-2 on the comp.parallel.mpi newsgroup, I would imagine that either (a) most of that group does not even know that there is a public comment period, or (b) they just assume that the forum must know what it is doing. (By the way, it appeared that a large percentage (40%?) of the audience at the SC-96 session were present or former members of the MPI forum.) > Most likely, the public is still digesting the 380 pages. Most likely, hardly anyone is digesting those pages. I would guess that there are only a few members of the forum who have truly digested and understood the entire 380 pages. And if you only read part of it and find it troublesome, you may have the feeling that it must be that way because of something in some other part, or some minor point that you don't understand (and in some cases, you'd be right). > The absence of negative comments can be a very good indicator. I suppose through the right glasses, almost anything can look like a good indicator. > Were you hoping for comments from users and/or the academic community? Perhaps this question should be directed to the forum. If they are not concerned with the lack of public comment, one might ask why they requested it. (I think it's like the question in the wedding ceremony where the audience is given the opportunity to object to the wedding, but nobody ever does.) For my part, I still have serious concerns about the one-sided chapter, but I believe that I have expressed them clearly at the meetings and to the one-sided list, and have even made an alternative proposal available on a web page. I could hope that external reviewers would take the last two into consideration, but I don't think it can be expected that all of us with grievances will repeat all of our concerns in the public forum. Without access to the discussions within the meetings, outside reviewers are at an extreme disadvantage when understanding the pros and cons of the various features, and I would guess that they know this. Even if all of the meeting's minutes were available, reviewers could not be expected to read and understand all of them and all of the mailing list archives during this public review period. I think that the best that can be expected is for the public reviewers to make comments based upon whether they are prepared to accept the standard in its present form, and to express the reasons that they will or will not do so. If all the reasons for accepting it are "Because it is MPI" or "Because it was formulated by experts who I trust", then it's acceptance won't really depend upon whether or not the forum did a good job in developing the standard, and there is little reason for the forum to have any more meetings. Rather than viewing the absence of negative comments as a very good indicator, it can also be viewed as the first hint of MPI's excessive size and complexity, especially related to its rather limited scope. End users are supposed to understand MPI just as a small adjunct to their real jobs as nuclear physicists or aerodynamicists or whatever. The forum seems to have shied away from any attempts to simplify MPI if it might result in even the slightest performance degradation on some (perhaps fictional) machines. I believe that, as a result, the forum has ended up with a standard which can theoretically be efficiently implemented on a large number of machines, but it doesn't really matter because few people with any real work to do will be able to learn to use it effectively. I have seen new architecture-independent graphical parallel languages described in 100 pages; the P4 book is about 270 pages; MPI, which only helps to write standardized message-passing programs, is 220 + 380 = 600 pages! Another communication interface which I have worked on has only 28 functions but is more portable than MPI and duplicates much of its functionality, while MPI has hundreds of function calls! The success of MPI will depend upon its ability to compete with these other approaches in the hands of scientists, not just in terms of efficiency of message-passing programs, but in flexibility and ease of use and learning. For these reasons, I doubt that I, personally, will be using MPI. Well, I guess that could be considered as public comment, so I'm sending this on to mpi-comment. -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Mon Jan 6 19:24:35 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA07997; Mon, 6 Jan 1997 19:24:34 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA07432 for mpi-core-out; Mon, 6 Jan 1997 18:28:08 -0600 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA07427; Mon, 6 Jan 1997 18:28:05 -0600 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.32.106]) by franklin.sdsc.edu (8.8.3/8.8.3/SDSCserver-13) with SMTP id QAA02393; Mon, 6 Jan 1997 16:28:03 -0800 (PST) Received: from localhost by rogue.sdsc.edu (950413.SGI.8.6.12/1.11-client) with SMTP id QAA20824; Mon, 6 Jan 1997 16:28:03 -0800 Date: Mon, 6 Jan 1997 16:28:02 -0800 (PST) From: Richard Frost To: "Dr. David C. DiNucci" cc: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Public comment on MPI-2 In-Reply-To: <199701062358.PAA13556@win182.nas.nasa.gov> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Mon, 6 Jan 1997, Dr. David C. DiNucci wrote (among other things): > Rather than viewing the absence of negative comments as a very good indicator, > it can also be viewed as the first hint of MPI's excessive size and complexity, > especially related to its rather limited scope. End users are supposed to > understand MPI just as a small adjunct to their real jobs as nuclear > physicists or aerodynamicists or whatever. I would have to agree that most users at our site will probably never touch MPI-2, with a possible exception of some I/O functions. On the other hand, I thought MPI-2 was for parallel pre-processors, library writers and tool builders! Those who regularly get their hands dirty developing portable parallel tools will certainly benefit from much of MPI-2. - Richard From mpi-core-human@mcs.anl.gov Mon Jan 6 22:15:54 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA09307; Mon, 6 Jan 1997 22:15:53 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA09180 for mpi-core-out; Mon, 6 Jan 1997 21:19:40 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA09175; Mon, 6 Jan 1997 21:19:36 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id TAA13827; Mon, 6 Jan 1997 19:19:27 -0800 (PST) Date: Mon, 6 Jan 1997 19:19:27 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701070319.TAA13827@win182.nas.nasa.gov> To: Richard Frost Subject: Re: Public comment on MPI-2 Cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Richard Frost > On Mon, 6 Jan 1997, Dr. David C. DiNucci wrote (among other things): > > Rather than viewing the absence of negative comments as a very good > > indicator, > > it can also be viewed as the first hint of MPI's excessive size and > > complexity, > > especially related to its rather limited scope. End users are supposed to > > understand MPI just as a small adjunct to their real jobs as nuclear > > physicists or aerodynamicists or whatever. > I would have to agree that most users at our site will probably > never touch MPI-2, with a possible exception of some I/O functions. > On the other hand, I thought MPI-2 was for parallel pre-processors, > library writers and tool builders! Those who regularly get their > hands dirty developing portable parallel tools will certainly benefit > from much of MPI-2. Since I consider myself a tool builder, I'll answer. There is no doubt that the whole "communicator" idea is an aid to library writers. This can be added to any message-passing system. In fact, didn't it start in Zipcode? I agree that the MPI-I/O stuff is pretty good. I am not sure that making it part of MPI has increased its value, however. I think I might like it more if I didn't need to hassle with MPI to use it. As for using MPI for preprocessors and tools, I would have to say that MPI is only a good choice based on the absence of other choices. In fact, when MPI-1 first started heading toward a non-layered system with no decent support for shared memory, and it began collecting operations like "ready send", I recognized that I would not want to build tools upon it, and started my own substrate, CDS. This decision was further cemented by decisions made in MPI-2, especially concerning restrictions in dynamic process creation, and the fact that one-sided operations still involve copying. (Would you want to program with an HPF compiler which produced programs which internally copied data even if the processes which were communicating were on the same node?) If MPI was developed for the tool builder and not the applications programmer, then it sure is a lot of work for a relatively small end-user population, and I, as a tool-builder, would have liked to know that I might have had a stronger voice. Regardless of who the end-user will be, the answer will be in the marketplace. If MPI can benefit the tool-builder or the preprocessor writer more than other approaches (like CDS, or Nexus, or ...), then great. MPI has some leverage in that the forum calls it a standard, but this is mostly marketing. (I heard MPI called a "de facto standard" even when MPICH was the only extant implementation, and there were virtually no programs written using MPI yet.) MPI's partial success over PVM can be attributed to MPI's ability to pass messages with fewer calls and less copying in some cases -- i.e. market forces -- not because it was "more standard" in some sense. We shall see, then, whether MPI can succeed on its own merits against new lean and mean entries into the field, especially considering the years and PhD-months it apparently takes to update MPI. As always, I could be wrong... -Dave (P.S. If there are followups on mpi-comment, I can't see them because majordomo has delayed my subscription.) ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Mon Jan 6 22:20:47 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA09350; Mon, 6 Jan 1997 22:20:46 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA09242 for mpi-core-out; Mon, 6 Jan 1997 21:25:04 -0600 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA07909; Mon, 6 Jan 1997 19:10:34 -0600 From: Andrew.Lumsdaine.1@nd.edu Received: (from lums@localhost) by lsc.nd.edu (8.8.3/8.8.2) id UAA01109; Mon, 6 Jan 1997 20:10:20 -0500 (EST) Date: Mon, 6 Jan 1997 20:10:20 -0500 (EST) Message-Id: <199701070110.UAA01109@lsc.nd.edu> To: frost@sdsc.edu CC: dinucci@nas.nasa.gov, mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: (message from Richard Frost on Mon, 6 Jan 1997 16:28:02 -0800 (PST)) Subject: Re: Public comment on MPI-2 Reply-to: Andrew.Lumsdaine.1@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode Richard Frost wrote: > I would have to agree that most users at our site will probably > never touch MPI-2, with a possible exception of some I/O functions. > On the other hand, I thought MPI-2 was for parallel pre-processors, > library writers and tool builders! Those who regularly get their > hands dirty developing portable parallel tools will certainly benefit > from much of MPI-2. In following comp.parallel.mpi, there have been various things not in MPI-1 that users consistently seem to ask for. At the top of the list would have to be dynamic process control. I have also seen many requests for C++ bindings (but then again, that's what I would be especially sensitive to). Best Wishes, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Mon Jan 6 22:23:39 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA09395; Mon, 6 Jan 1997 22:23:33 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA09333 for mpi-core-out; Mon, 6 Jan 1997 21:28:07 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA09328; Mon, 6 Jan 1997 21:28:04 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id TAA13827; Mon, 6 Jan 1997 19:19:27 -0800 (PST) Date: Mon, 6 Jan 1997 19:19:27 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701070319.TAA13827@win182.nas.nasa.gov> To: Richard Frost Subject: Re: Public comment on MPI-2 Cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Richard Frost > On Mon, 6 Jan 1997, Dr. David C. DiNucci wrote (among other things): > > Rather than viewing the absence of negative comments as a very good > > indicator, > > it can also be viewed as the first hint of MPI's excessive size and > > complexity, > > especially related to its rather limited scope. End users are supposed to > > understand MPI just as a small adjunct to their real jobs as nuclear > > physicists or aerodynamicists or whatever. > I would have to agree that most users at our site will probably > never touch MPI-2, with a possible exception of some I/O functions. > On the other hand, I thought MPI-2 was for parallel pre-processors, > library writers and tool builders! Those who regularly get their > hands dirty developing portable parallel tools will certainly benefit > from much of MPI-2. Since I consider myself a tool builder, I'll answer. There is no doubt that the whole "communicator" idea is an aid to library writers. This can be added to any message-passing system. In fact, didn't it start in Zipcode? I agree that the MPI-I/O stuff is pretty good. I am not sure that making it part of MPI has increased its value, however. I think I might like it more if I didn't need to hassle with MPI to use it. As for using MPI for preprocessors and tools, I would have to say that MPI is only a good choice based on the absence of other choices. In fact, when MPI-1 first started heading toward a non-layered system with no decent support for shared memory, and it began collecting operations like "ready send", I recognized that I would not want to build tools upon it, and started my own substrate, CDS. This decision was further cemented by decisions made in MPI-2, especially concerning restrictions in dynamic process creation, and the fact that one-sided operations still involve copying. (Would you want to program with an HPF compiler which produced programs which internally copied data even if the processes which were communicating were on the same node?) If MPI was developed for the tool builder and not the applications programmer, then it sure is a lot of work for a relatively small end-user population, and I, as a tool-builder, would have liked to know that I might have had a stronger voice. Regardless of who the end-user will be, the answer will be in the marketplace. If MPI can benefit the tool-builder or the preprocessor writer more than other approaches (like CDS, or Nexus, or ...), then great. MPI has some leverage in that the forum calls it a standard, but this is mostly marketing. (I heard MPI called a "de facto standard" even when MPICH was the only extant implementation, and there were virtually no programs written using MPI yet.) MPI's partial success over PVM can be attributed to MPI's ability to pass messages with fewer calls and less copying in some cases -- i.e. market forces -- not because it was "more standard" in some sense. We shall see, then, whether MPI can succeed on its own merits against new lean and mean entries into the field, especially considering the years and PhD-months it apparently takes to update MPI. As always, I could be wrong... -Dave (P.S. If there are followups on mpi-comment, I can't see them because majordomo has delayed my subscription.) ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Tue Jan 7 09:23:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA18900; Tue, 7 Jan 1997 09:23:36 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA14173 for mpi-core-out; Tue, 7 Jan 1997 08:27:02 -0600 Received: from relay1.UU.NET (relay1.UU.NET [192.48.96.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA14159; Tue, 7 Jan 1997 08:26:54 -0600 Received: from sky.com by relay1.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQbxlx09125; Tue, 7 Jan 1997 09:26:32 -0500 (EST) Received: from anna.sky.com by sky.com (SMI-8.6/SMI-SVR4) id JAA22408; Tue, 7 Jan 1997 09:25:27 -0500 Received: from anna by anna.sky.com (SMI-8.6/SMI-SVR4) id JAA03515; Tue, 7 Jan 1997 09:22:54 -0500 Message-ID: <32D25C3D.442C@sky.com> Date: Tue, 07 Jan 1997 09:22:53 -0500 From: Anna Rounbehler Organization: Sky Computers X-Mailer: Mozilla 3.0Gold (X11; I; SunOS 5.5.1 sun4c) MIME-Version: 1.0 To: dinucci@nas.nasa.gov CC: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Public Defined Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Ref: David Dinucci wrote: > think that the best that can be expected is for the public reviewers to > make comments based upon whether they are prepared to accept the standard in > its present form, and to express the reasons that they will or will not do so. You have raised a question that I think needs to be answered before we can judge the acceptance, apathy or rejection to the MPI-2 standard. We first need to: 1. define who the public is 2. decide if all subgroups in this "public" are aware of MPI-2 3. determine if they are aware that their comments are significant to the MPI-2 forum 4. Be sure the public is encouraged to give comments if and when they are made aware. 5. Provide a questionnaire for commentors to fill out. This is a standard product market strategy. Although this may seem like a trivial approach, it would certainly help to define the problem. Perhaps providers and vendors should be polling their users and ask them to comment. When discussing MPI with potential users, I have found it necessary to encapsulate the functionality and provide a brief description of how it may be used. For at least one class of uers, this may be a necessity. They will not take the time to evaluate it. Also, there is a broad spectrum of potential users. It is unclear how to get them involved for commenting since they do not know who they are yet. And finally, the current users are probably well represented at the Forum already and constantly give their comments. My opinion about MPI products in general is that they are becoming more popular. Like any product, they must be marketed to increase awareness. Local communities of potential users are made aware through local IEEE and/or ACM chapter talks at special interest group meetings; vendor literature; published texts and tutorials. As consumers apply the product to their problems, more uses may evolve. As a provider and supplier of MPI, we usually invest time to consult with potential customers on how the product may be used. There are so many diverse applications, with a need for different MPI functionality. This is especially true for MPI-2. In general, it may be more effective to create a list of criteria and send out a questionnaire to users and potential users, rather than expect them to read 380 pages and comment. Perhaps the process needs some order. ------------------------------ Anna Rounbehler SKY Computers anna@sky.com From mpi-core-human@mcs.anl.gov Tue Jan 7 14:02:07 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA22618; Tue, 7 Jan 1997 14:02:05 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA19711 for mpi-core-out; Tue, 7 Jan 1997 12:42:09 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id MAA19706 for ; Tue, 7 Jan 1997 12:41:59 -0600 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA25538; Tue, 7 Jan 1997 13:36:14 -0500 Message-Id: <32D298F4.167E@kgn.ibm.com> Date: Tue, 07 Jan 1997 13:41:56 -0500 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.0 (X11; I; AIX 1) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: Expectations for public comment Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I began participation in the MPI-2 Forum in Jan 96 and at that time it was already a given that a document would be presented for public comment at SC96. We added a day to each meeting and pushed hard to be ready for that presentation. To me that implied that at least many Forum members had expectations that the public (whoever that may be) would react. So, I must turn the "What were you expecting" question that Anna addressed to me into a question addressed to those members of the Forum who participated in the decision to target SC96. ====================================================================== To all: If you are one of those who felt SC96 was an important target event; What were you expecting? ====================================================================== Perhaps answer enough is, "To prove to the public that the MPI-2 Forum has not been sleeping or disbanded". Regards - Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue Jan 7 16:14:32 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA23958; Tue, 7 Jan 1997 16:14:22 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA23449 for mpi-core-out; Tue, 7 Jan 1997 15:17:24 -0600 Received: from romano.cs.wisc.edu (romano.cs.wisc.edu [128.105.66.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA23438 for ; Tue, 7 Jan 1997 15:17:20 -0600 Received: (from lederman@localhost) by romano.cs.wisc.edu (8.7.6/8.7.3) id PAA05240; Tue, 7 Jan 1997 15:17:18 -0600 (CST) Date: Tue, 7 Jan 1997 15:17:18 -0600 (CST) Message-Id: <199701072117.PAA05240@romano.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: new version of document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. These directories also have all the sources and figures. In addition, there is the file interop-970107.ps.Z which is an informal proposal from Rolf Rabenseifner for interoperability which he would like to discuss at the next meeting. The chapters that have changes are one-sided, external, real time and the JOD on caching. If you want to make changes for the version to be printed for the meeting, you need to make sure the chapter authors can incorporate them by 10:00 AM (central US time) on Tuesday, January 14. Steve From mpi-core-human@mcs.anl.gov Tue Jan 7 16:56:32 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA24412; Tue, 7 Jan 1997 16:56:27 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA24261 for mpi-core-out; Tue, 7 Jan 1997 15:58:45 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA24243; Tue, 7 Jan 1997 15:58:36 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id NAA15451; Tue, 7 Jan 1997 13:58:28 -0800 (PST) Date: Tue, 7 Jan 1997 13:58:28 -0800 (PST) From: dinucci@nas.nasa.gov (David C. DiNucci) Message-Id: <199701072158.NAA15451@win182.nas.nasa.gov> To: thomasd@netcom.com (Thomas Donaldson) Subject: Re: Public comment on MPI-2 Cc: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Thomas Donaldson writes: > To be portable, a standard for communicators must be accepted and available on > a wide range of machines. It seems to me that this is not something that any > company, even Microsoft, can successfully bring about. I think that, once the concept of communicators is "out there", any company can successfully bring about *their version* of an implementation. The only problem is in standardizing it. I'm not sure that standardizing communicators is any different than standardizing anything else. > I will even agree that MPI (even MPI 1.1) seems a bit corpulent for the > basic job. But then besides the notion of communicators, it is also a > library of communication routines, all of which a programmer would probably > have to invent at one time or another. MPI started with a very sensible goal: "Let's standardize message-passing so that programmers who want to pass messages can all speak the same language". There were at least two problems. First, no group will stand for having their favorite feature omitted from The Standard. Second, if you get *lots* of *smart* people together in a big room every six weeks for years, they are going to do more than standardize common practice. I believe that both of these problems could be remedied by the "smaller is better" philosophy. If different groups want different features, first standardize the lower-level layer with which everyone's features can be built, and only after that provide one or more higher-level layers which can be defined totally in terms of the low-level layer but which may be optimized further on some architectures. One or more of these higher-level layers could be standardized as well, after they have been played around with for a while. Tool-builders might just want to stick mostly to the low-level layer, while applications programmers might use higher-level layers exclusively. Different higher-level layers may be desirable for different end-user groups. It's rather like a RISC instruction set with a few commonly-used sequences built into CISC instructions. Some of the forum might claim that MPI-1 forms this RISC level, but I think that this is still far too high. An example: When this RISC philosophy was proposed for the dynamic processes chapter, it was met with disapproval, primarily from a small number (1?) of vendors because their parallel architecture wasn't geared to create processes first and then wire them up (into communicators) later -- it all had to be done at the same time to ensure the best performance. So now, the standard is forever changed in spite of the fact that this "feature" of the architecture may well be transient, and there may be no other architectures for which it is a problem. If the layered approach had been used, the higher-level layers could be changed as architectures change while the low-level layers could be left intact. This vendor(s) could have strongly recommended that its programmers utilize a higher-level layer. That is, over time, the older, lower layers become more concrete and stable while the upper layers remain flexible to accomodate new architectures and ideas. The ratio of developers and vendors to users on the MPI Forum is probably about 1 to 1, instead of 1 to hundreds or thousands in the real world. I believe that this has led to a standard which developers can be proud of, with all sorts of neat doodads to make things more efficient on this box or that, but with no public voice warning that things might be getting too complex (although there have been a few vendors trying to make this point). Perhaps it would be beneficial if the forum now spent a significant amount of time on simplification. After all, the ratio will return to normal once MPI hits the masses. This is its first major exposure to those masses. Problem: Maybe I am just one lone soul who feels this way. I do not see other postings corroborating my impressions. I guess the forum must just estimate whether MPI can compete with other packages. As the old saying goes, the nice thing about standards is that there are so many to choose from. "Beware the Attack of the Killer Micro-APIs" (with apologies to Eugene Brooks). -Dave (P.S. I am now subscribed to mpi-comment.) ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Tue Jan 7 21:07:41 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA28127; Tue, 7 Jan 1997 21:07:40 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA28010 for mpi-core-out; Tue, 7 Jan 1997 20:11:10 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA28005; Tue, 7 Jan 1997 20:11:06 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id SAA16182; Tue, 7 Jan 1997 18:11:05 -0800 (PST) Date: Tue, 7 Jan 1997 18:11:05 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701080211.SAA16182@win182.nas.nasa.gov> To: Anna Rounbehler Subject: Re: Public Defined Cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Anna Rounbehler writes: > You have raised a question that I think needs to be answered before we > can judge > the acceptance, apathy or rejection to the MPI-2 standard. We first > need to: > 1. define who the public is > 2. decide if all subgroups in this "public" are aware of MPI-2 > 3. determine if they are aware that their comments are significant to > the MPI-2 forum > 4. Be sure the public is encouraged to give comments if and when they > are made aware. > 5. Provide a questionnaire for commentors to fill out. This is a > standard product market strategy. > Although this may seem like a trivial approach, it would certainly help > to define the problem. Perhaps providers and vendors should be polling > their users and ask them to comment. No matter who the public is, they must understand the standard before they can know how much they like it, and they must know how much they like it before they can express how much they like it. To understand it, they could try to write some programs with it, and/or read the 380-page document, and/or attend a presentation or tutorial, and/or discuss it with others who are familiar with it. And that's just for people who already understand MPI-1. For almost everyone, reading the document is the only available option. So, this public comment period is first a test of whether people can read the document. Regardless of how wonderful MPI might be, reading a 380-page standards book, containing justifications based upon details about machines with which the reader may not be familiar, may be an imposing task, especially if it is being read in one's spare time for no pay. Even the forum hasn't contented itself with some parts. The SC'96 BOF session might have helped a little, but it was very short, and left me a little confused. (e.g. Did Marc say that multiple non-overlapping PUTs to the same window were disallowed in some cases? Between WIN_BARRIERS? I can't seem to find a statement one way or the other in the document, but it's probably in there, and this seems like an important point.) On the other hand, I don't want to make excuses for those who do find the time and motivation to read the document. My guess is, therefore, that a questionnaire won't help for a review of *THE* standard, though it could theoretically help to determine what people would like in *A* standard. It does seem a little late for a step like this, but maybe the current standard could be tailored to suit the results. -Dave (Someone (in addition to Juan) sent me some personal email about one of my previous posts this afternoon and I accidentally deleted it before reading it. If they see this, please resend if possible. Thanks.) ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Tue Jan 7 21:09:25 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA28148; Tue, 7 Jan 1997 21:09:24 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA28074 for mpi-core-out; Tue, 7 Jan 1997 20:13:07 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA28069; Tue, 7 Jan 1997 20:13:03 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id SAA16182; Tue, 7 Jan 1997 18:11:05 -0800 (PST) Date: Tue, 7 Jan 1997 18:11:05 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701080211.SAA16182@win182.nas.nasa.gov> To: Anna Rounbehler Subject: Re: Public Defined Cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Anna Rounbehler writes: > You have raised a question that I think needs to be answered before we > can judge > the acceptance, apathy or rejection to the MPI-2 standard. We first > need to: > 1. define who the public is > 2. decide if all subgroups in this "public" are aware of MPI-2 > 3. determine if they are aware that their comments are significant to > the MPI-2 forum > 4. Be sure the public is encouraged to give comments if and when they > are made aware. > 5. Provide a questionnaire for commentors to fill out. This is a > standard product market strategy. > Although this may seem like a trivial approach, it would certainly help > to define the problem. Perhaps providers and vendors should be polling > their users and ask them to comment. No matter who the public is, they must understand the standard before they can know how much they like it, and they must know how much they like it before they can express how much they like it. To understand it, they could try to write some programs with it, and/or read the 380-page document, and/or attend a presentation or tutorial, and/or discuss it with others who are familiar with it. And that's just for people who already understand MPI-1. For almost everyone, reading the document is the only available option. So, this public comment period is first a test of whether people can read the document. Regardless of how wonderful MPI might be, reading a 380-page standards book, containing justifications based upon details about machines with which the reader may not be familiar, may be an imposing task, especially if it is being read in one's spare time for no pay. Even the forum hasn't contented itself with some parts. The SC'96 BOF session might have helped a little, but it was very short, and left me a little confused. (e.g. Did Marc say that multiple non-overlapping PUTs to the same window were disallowed in some cases? Between WIN_BARRIERS? I can't seem to find a statement one way or the other in the document, but it's probably in there, and this seems like an important point.) On the other hand, I don't want to make excuses for those who do find the time and motivation to read the document. My guess is, therefore, that a questionnaire won't help for a review of *THE* standard, though it could theoretically help to determine what people would like in *A* standard. It does seem a little late for a step like this, but maybe the current standard could be tailored to suit the results. -Dave (Someone (in addition to Juan) sent me some personal email about one of my previous posts this afternoon and I accidentally deleted it before reading it. If they see this, please resend if possible. Thanks.) ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Tue Jan 7 22:45:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA28917; Tue, 7 Jan 1997 22:45:00 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA28847 for mpi-core-out; Tue, 7 Jan 1997 21:48:10 -0600 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA28834; Tue, 7 Jan 1997 21:48:03 -0600 From: Andrew.Lumsdaine.1@nd.edu Received: (from lums@localhost) by lsc.nd.edu (8.8.3/8.8.2) id WAA04267; Tue, 7 Jan 1997 22:47:48 -0500 (EST) Date: Tue, 7 Jan 1997 22:47:48 -0500 (EST) Message-Id: <199701080347.WAA04267@lsc.nd.edu> To: dinucci@nas.nasa.gov CC: anna@sky.com, mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov In-reply-to: <199701080211.SAA16182@win182.nas.nasa.gov> (dinucci@nas.nasa.gov) Subject: Re: Public Defined Reply-to: Andrew.Lumsdaine.1@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode Dr. David C. DiNucci wrote: > No matter who the public is, they must understand the standard > before they can know how much they like it, and they must know how > much they like it before they can express how much they like it. To > understand it, they could try to write some programs with it, and/or > read the 380-page document, and/or attend a presentation or > tutorial, and/or discuss it with others who are familiar with it. > And that's just for people who already understand MPI-1. For almost > everyone, reading the document is the only available option. Regarding "writing the programs with it" -- I seem to recall that there was an early version of MPICH available during the MPI-1 public comment period -- people actually could write programs with it. Is there such software available for at least a subset of MPI-2? I think people would try some of the things they have been waiting for -- and provide feedback -- if an implementation were available. I am not quite so cynical about MPI-2 -- I think there were some legitimate needs expressed by the user community that are being met by MPI-2. I think it would be helpful, though, for those users who have expressed needs (I am thinking in particular of dynamic processes) to have an implementation to try out. MPI-2 was clearly necessary. Without dynamic process control, MPI would always be perceived as being inferior to PVM. ;-) Best Wishes, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Tue Jan 7 22:54:39 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA28978; Tue, 7 Jan 1997 22:54:38 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA28952 for mpi-core-out; Tue, 7 Jan 1997 21:58:15 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA28947; Tue, 7 Jan 1997 21:58:12 -0600 Message-Id: <199701080358.VAA28947@antares.mcs.anl.gov> To: dinucci@nas.nasa.gov (Dr. David C. DiNucci) cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Subject: Re: Public Defined In-reply-to: Your message of "Tue, 07 Jan 1997 18:11:05 PST." <199701080211.SAA16182@win182.nas.nasa.gov> Date: Tue, 07 Jan 1997 21:58:12 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | My guess is, therefore, that a questionnaire won't help for a review of *THE* | standard, though it could theoretically help to determine what people would | like in *A* standard. It does seem a little late for a step like this, but | maybe the current standard could be tailored to suit the results. After MPI-1 seemed to be gaining some acceptance, there was indeed a questionnaire, administered by the Ohio Supercomputer Center folks. It had a number of parts, but one interesting part asked what people felt was lacking in MPI. There were many responses, but most broke down into easily recognizable areas: parallel I/O, dynamic process management, and one-sided operations were the major ones, followed by C++ and Fortran 90 bindings, a few requests for non-blocking collective operations, and some odds and ends that are addressed generally by the external chapter. The results were solicited and posted on comp.parallel.mpi, and also, I think, on comp.parallel. There were on the order of a hundred responses. So I believe that we are indeed addressing topics that people would like in *A* standard. I confess that I am a little disappointed by the lack of response to the call for comments, but I am not distressed by it. I speak often in public about MPI, both 1 and 2, and the people I talk to seem happy about MPI-1, and happy about the topics being addressed in MPI-2, without being familiar with the current MPI-2 details. Rather, I believe that MPI-1 garnered sufficient credibility that the MPI Forum is the beneficiary of considerable confidence that it will do a reasonable job on the MPI-2 topics, which are indeed what people want to see addressed. I hope we are worthy of such confidence, but I have no doubt that we are working on the right topics. I am not particularly concerned with the size of the document. There has been a lot of work to do, and the Forum has done a lot of work. Just as MPI-1 exposed some issues that require some complexity for a complete treatment (buffering, for example), so will MPI-2 expose some issues that require some complexity for a complete treatment (1-sided, I/O, and external, particularly). Maybe the presentation can be made more concise; maybe not. Standards definitions are notorious for being bad tutorial documents; the MPI-2 tutorial writings will no doubt emerge later, and will make it less intimidating for "the public". Rusty From mpi-core-human@mcs.anl.gov Wed Jan 8 09:25:16 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA04769; Wed, 8 Jan 1997 09:25:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA03013 for mpi-core-out; Wed, 8 Jan 1997 08:27:39 -0600 Received: from relay1.UU.NET (relay1.UU.NET [192.48.96.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA03008 for ; Wed, 8 Jan 1997 08:27:36 -0600 Received: from sky.com by relay1.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQbxpp05035; Wed, 8 Jan 1997 09:27:15 -0500 (EST) Received: from anna.sky.com by sky.com (SMI-8.6/SMI-SVR4) id JAA00307; Wed, 8 Jan 1997 09:26:10 -0500 Received: from anna by anna.sky.com (SMI-8.6/SMI-SVR4) id RAA03908; Tue, 7 Jan 1997 17:25:08 -0500 Message-ID: <32D2CD44.353D@sky.com> Date: Tue, 07 Jan 1997 17:25:08 -0500 From: Anna Rounbehler Organization: Sky Computers X-Mailer: Mozilla 3.0Gold (X11; I; SunOS 5.5.1 sun4c) MIME-Version: 1.0 To: treumann@kgn.ibm.com CC: mpi-core@mcs.anl.gov Subject: As a Courtesy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > Perhaps answer enough is, "To prove to the public that the MPI-2 Forum > has not been sleeping or disbanded". > But more important - As a courtesy to anyone who would like to comment and be represented. Anna -------------------------- Anna Rounbehler SKY Computers anna@sky.com From mpi-core-human@mcs.anl.gov Wed Jan 8 17:57:10 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA11835; Wed, 8 Jan 1997 17:56:57 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA13610 for mpi-core-out; Wed, 8 Jan 1997 16:59:21 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA13605; Wed, 8 Jan 1997 16:59:16 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id OAA17942; Wed, 8 Jan 1997 14:59:14 -0800 (PST) Date: Wed, 8 Jan 1997 14:59:14 -0800 (PST) From: dinucci@nas.nasa.gov (David C. DiNucci) Message-Id: <199701082259.OAA17942@win182.nas.nasa.gov> To: Rusty Lusk Subject: Re: Public Defined Cc: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Rusty Lusk writes: > | My guess is, therefore, that a questionnaire won't help for a review of *THE* > | standard, though it could theoretically help to determine what people would > | like in *A* standard. It does seem a little late for a step like this, but > | maybe the current standard could be tailored to suit the results. > > After MPI-1 seemed to be gaining some acceptance, there was indeed a > questionnaire, administered by the Ohio Supercomputer Center folks. ... > ... There were on the order of a hundred responses. So I believe > that we are indeed addressing topics that people would like in *A* standard. I do also believe that the MPI forum is probably addressing topics that people would like to see in a standard. But how many of these people thought that they were asking for a 600-page standard containing hundreds of function calls? > I confess that I am a little disappointed by the lack of response to the call > for comments, but I am not distressed by it. I speak often in public about > MPI, both 1 and 2, and the people I talk to seem happy about MPI-1, and happy > about the topics being addressed in MPI-2, without being familiar with the > current MPI-2 details. Right. Again, they are happy about the topics. So am I. > I am not particularly concerned with the size of the document. There has been > a lot of work to do, and the Forum has done a lot of work. There is no doubt that the forum has done a lot of work, and I'm sure that there has been much benefit from it. Many researchers and vendors and implementors have spoken with each other who otherwise might not have, and they may have uncovered new semantic and syntactic problems related to message passing, and they may have come up with new ways of handling old problems, new software and hardware approaches, etc. I am definitely not arguing that the MPI forum meetings were a waste of time. But the public is not being asked to evaluate or comment on the meetings right now, if I understand correctly. They are being asked to comment on the standard which the forum has proposed. I, for one, think that that standard is too large and complex. By "too large", I don't necessarily mean the number of pages in the standards document. I mean something that might be called "meme size" -- the number of things which I must remember in order to understand and use the standard. For example, if lots of functions have exactly the same argument list, then I only need to remember the argument list and then the name and purpose of each routine, whereas if every routine has a different argument list, I must remember the name, description, and argument list for each separately. The meme size would be closely related to the number of non-inherited methods in an OO system. I suggest that the "meme size" for MPI is much too large for your average application programmer, and that it could be made smaller. Even though the MPI forum has one committee for each chapter, often containing a number of parallel processing experts, the typical application program will have but one programmer (or if more than one, each may need to understand MPI in toto). To be workable, the entire MPI standard should be totally comprehensible by any single person interested in using it, and certainly by each of the members of the MPI forum. (Maybe this is latter part is already the case, but I did not get that impression in the few meetings which I attended.) Note that with larger APIs, such as X and Motif, for example, it is quite common to be an "X hacker" who builds GUIs and has little or no application-specific expertise. I'm not sure that it will make sense to be an "MPI hacker". (Xt has about 231 functions and macros.) I may be overstressing the importance of this simplification. I will leave it to the forum to decide the target audience of MPI. It may be that there will be enough projects, with enough person-hours devoted to learning the parts of MPI that they need, to keep *this* MPI standard afloat. My guess is that most people will use a subset consisting of a few functions from MPI-1, plus some dynamic processes and maybe some I/O. But, in any case, I think the standard would greatly benefit from any simplification, for both users and implementors, even if it requires some modest compromises in performance on some architectures. And in any case, I will still continue working on the RISC end, and will expect CDS and its cousins to be a formidable competitor to MPI if the standard is not simplified. (In case you lost the pointer I gave you at SC'96, CDS is at http://www.nas.nasa.gov/NAS/Tools/Projects/CDS/) > Just as MPI-1 > exposed some issues that require some complexity for a complete treatment > (buffering, for example), so will MPI-2 expose some issues that require some > complexity for a complete treatment (1-sided, I/O, and external, > particularly). Maybe the presentation can be made more concise; maybe not. > Standards definitions are notorious for being bad tutorial documents; the > MPI-2 tutorial writings will no doubt emerge later, and will make it less > intimidating for "the public". The standard cannot be judged until it is understood. The standards document can be judged by just trying to understand the standard. > Rusty -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Thu Jan 9 00:55:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA17003; Thu, 9 Jan 1997 00:55:11 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA18220 for mpi-core-out; Wed, 8 Jan 1997 23:59:26 -0600 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA18215; Wed, 8 Jan 1997 23:59:22 -0600 Received: from genie.sdsc.edu (genie.sdsc.edu [132.249.40.160]) by franklin.sdsc.edu (8.8.3/8.8.3/SDSCserver-14) with SMTP id VAA23955; Wed, 8 Jan 1997 21:59:20 -0800 (PST) Received: from localhost by genie.sdsc.edu (950413.SGI.8.6.12/1.11-client) with SMTP id VAA26286; Wed, 8 Jan 1997 21:59:20 -0800 Date: Wed, 8 Jan 1997 21:59:20 -0800 (PST) From: Richard Frost To: "David C. DiNucci" cc: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Public Defined In-Reply-To: <199701082259.OAA17942@win182.nas.nasa.gov> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Wed, 8 Jan 1997, David C. DiNucci wrote: > > Even though the MPI forum has one committee for each chapter, often containing > a number of parallel processing experts, the typical application program will > have but one programmer (or if more than one, each may need to understand MPI > in toto). To be workable, the entire MPI standard should be totally > comprehensible by any single person interested in using it, and certainly by > each of the members of the MPI forum. I disagree with the notion "the entire standard should be totally comprehensible by any single person" -- especially if you mean that any person should be able to learn a majority of MPI with little effort. Today, if an application programmer (who is not a tool builder) needs more than send/receive or some other small functionality provided by MPI, then they often rely on libraries or whole applications built by parallel tool experts. It is arguably foolish for them to do otherwise. This is what our staff and external users do now with many fundamental but otherwise extensive software systems. I expect the trend to continue. A number of our users recently began testing applications on our T3E. I am on the mail reflector for problem reports, general announcements, etc. Some of the ported applications made use of MPI libraries authored by external parties. A few of the more robust users have their own, complex MPI-based libraries and applications. I did not encounter any MPI-related reports in code ports. So, several of our T3E users do not understand much about MPI, only that the MPI-based libraries that they use recompiled without error on the T3E and completed all confidence tests successfully. Let's look at the Unix graphics world for a moment. Your arguments contradict common practice in X, (Open)GL, Motif, etc. Every quarter a member of our staff teaches some fundamentals of these graphic languages to 40 or so UCSD undergraduate students in the context of application-based programming assignments. Certainly these students don't study all 2100 pages of X+Motif! But they do achieve surprising levels of functionality. The two reasons I see for this are (a) they master the fundamental concepts and primitives of graphics programming and (b) they learn how to use some very powerful tools along with simple primitives to achieve their goals. If you are a tool builder and think that some higher-level systems than MPI should be targeted at application areas: please, please, please go build them. In my opinion, building them on MPI primitives would be a good idea. From mpi-core-human@mcs.anl.gov Thu Jan 9 14:34:38 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA07812; Thu, 9 Jan 1997 14:34:35 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA29034 for mpi-core-out; Thu, 9 Jan 1997 13:34:45 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA29023; Thu, 9 Jan 1997 13:34:39 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id LAA26528; Thu, 9 Jan 1997 11:34:41 -0800 (PST) Date: Thu, 9 Jan 1997 11:34:41 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701091934.LAA26528@win182.nas.nasa.gov> To: Richard Frost Subject: Re: Public Defined Cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Richard Frost wrote: > On Wed, 8 Jan 1997, David C. DiNucci wrote: > > > > Even though the MPI forum has one committee for each chapter, often > > containing a number of parallel processing experts, the typical > > application program will have but one programmer (or if more than one, > > each may need to understand MPI > > in toto). To be workable, the entire MPI standard should be totally > > comprehensible by any single person interested in using it, and certainly > > by each of the members of the MPI forum. > > I disagree with the notion "the entire standard should be totally > comprehensible by any single person" -- especially if you mean that any > person should be able to learn a majority of MPI with little effort. > > Today, if an application programmer (who is not a tool builder) needs more > than send/receive or some other small functionality provided by MPI, > then they often rely on libraries or whole applications built by > parallel tool experts. It is arguably foolish for them to do otherwise. > This is what our staff and external users do now with many fundamental > but otherwise extensive software systems. I expect the > trend to continue. If I understand your point, it is that applications users often won't use the standard interface in any case: They will use instead some more application-specific software built upon the standard interface. I agree, and I stated so in an earlier message. If you are suggesting that this is a justification for making the standard interface complex, I disagree. In fact, if this premise is taken to be true, I think it makes sense to leave all of the niceties targetted to the applications programmer out of the standard, because they can just as well be included in the layers between the standard and the applications programmer *if they are needed for that application domain*. > Let's look at the Unix graphics world for a moment. Your arguments > contradict common practice in X, (Open)GL, Motif, etc. Every quarter a > member of our staff teaches some fundamentals of these graphic > languages to 40 or so UCSD undergraduate students in the context of > application-based programming assignments. Certainly these students > don't study all 2100 pages of X+Motif! But they do achieve surprising > levels of functionality. The two reasons I see for this are (a) they > master the fundamental concepts and primitives of graphics programming > and (b) they learn how to use some very powerful tools along with > simple primitives to achieve their goals. It is certainly possible for an application programmer with expertise in some certain field to use X+Motif. (In fact, everyone in my group has done this at some time or other, and we could probably all be considered experts in something.) But each big package that must be learned is one more obstacle to getting things done, and one more piece of brain clutter. Just think of all the combined neurons and minutes across the world that will be devoted to MPI that could be going to what they are really trying to get done :-). I think it's fine for people like those on the forum to use their neurons this way. I just don't like the idea of every message-passing programmer in the world doing so, and I think its possible to avoid it. (A neuron is a terrible thing to waste.) > If you are a tool builder and think that some higher-level systems than > MPI should be targeted at application areas: please, please, please go > build them. In my opinion, building them on MPI primitives would be > a good idea. If the MPI standard combines actions which are not necessarily related (e.g. creating processes and manipulating communicators) into a single primitive, this restricts the interfaces being built upon the standard to have the same combined actions. I believe that it is much more effective to build a specialized interface out of small and easy-to-understand primitive actions than it is to refashion complex actions into a different form, and I would posit that it is difficult and/or inefficient to try to refashion complex actions into simple primitives through an intermediate interface. Again, I don't want to overstate the case. I just want to say that there is power in simplicity, and I hope that the forum is taking this into account at every opportunity. -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Thu Jan 9 14:39:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA07843; Thu, 9 Jan 1997 14:39:45 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA29308 for mpi-core-out; Thu, 9 Jan 1997 13:43:05 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA29302; Thu, 9 Jan 1997 13:43:02 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id LAA26528; Thu, 9 Jan 1997 11:34:41 -0800 (PST) Date: Thu, 9 Jan 1997 11:34:41 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701091934.LAA26528@win182.nas.nasa.gov> To: Richard Frost Subject: Re: Public Defined Cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Richard Frost wrote: > On Wed, 8 Jan 1997, David C. DiNucci wrote: > > > > Even though the MPI forum has one committee for each chapter, often > > containing a number of parallel processing experts, the typical > > application program will have but one programmer (or if more than one, > > each may need to understand MPI > > in toto). To be workable, the entire MPI standard should be totally > > comprehensible by any single person interested in using it, and certainly > > by each of the members of the MPI forum. > > I disagree with the notion "the entire standard should be totally > comprehensible by any single person" -- especially if you mean that any > person should be able to learn a majority of MPI with little effort. > > Today, if an application programmer (who is not a tool builder) needs more > than send/receive or some other small functionality provided by MPI, > then they often rely on libraries or whole applications built by > parallel tool experts. It is arguably foolish for them to do otherwise. > This is what our staff and external users do now with many fundamental > but otherwise extensive software systems. I expect the > trend to continue. If I understand your point, it is that applications users often won't use the standard interface in any case: They will use instead some more application-specific software built upon the standard interface. I agree, and I stated so in an earlier message. If you are suggesting that this is a justification for making the standard interface complex, I disagree. In fact, if this premise is taken to be true, I think it makes sense to leave all of the niceties targetted to the applications programmer out of the standard, because they can just as well be included in the layers between the standard and the applications programmer *if they are needed for that application domain*. > Let's look at the Unix graphics world for a moment. Your arguments > contradict common practice in X, (Open)GL, Motif, etc. Every quarter a > member of our staff teaches some fundamentals of these graphic > languages to 40 or so UCSD undergraduate students in the context of > application-based programming assignments. Certainly these students > don't study all 2100 pages of X+Motif! But they do achieve surprising > levels of functionality. The two reasons I see for this are (a) they > master the fundamental concepts and primitives of graphics programming > and (b) they learn how to use some very powerful tools along with > simple primitives to achieve their goals. It is certainly possible for an application programmer with expertise in some certain field to use X+Motif. (In fact, everyone in my group has done this at some time or other, and we could probably all be considered experts in something.) But each big package that must be learned is one more obstacle to getting things done, and one more piece of brain clutter. Just think of all the combined neurons and minutes across the world that will be devoted to MPI that could be going to what they are really trying to get done :-). I think it's fine for people like those on the forum to use their neurons this way. I just don't like the idea of every message-passing programmer in the world doing so, and I think its possible to avoid it. (A neuron is a terrible thing to waste.) > If you are a tool builder and think that some higher-level systems than > MPI should be targeted at application areas: please, please, please go > build them. In my opinion, building them on MPI primitives would be > a good idea. If the MPI standard combines actions which are not necessarily related (e.g. creating processes and manipulating communicators) into a single primitive, this restricts the interfaces being built upon the standard to have the same combined actions. I believe that it is much more effective to build a specialized interface out of small and easy-to-understand primitive actions than it is to refashion complex actions into a different form, and I would posit that it is difficult and/or inefficient to try to refashion complex actions into simple primitives through an intermediate interface. Again, I don't want to overstate the case. I just want to say that there is power in simplicity, and I hope that the forum is taking this into account at every opportunity. -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Thu Jan 9 15:18:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA08423; Thu, 9 Jan 1997 15:18:41 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA00406 for mpi-core-out; Thu, 9 Jan 1997 14:22:09 -0600 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA00390; Thu, 9 Jan 1997 14:21:54 -0600 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.32.106]) by franklin.sdsc.edu (8.8.3/8.8.3/SDSCserver-14) with SMTP id MAA26882; Thu, 9 Jan 1997 12:21:56 -0800 (PST) Received: from localhost by rogue.sdsc.edu (950413.SGI.8.6.12/1.11-client) with SMTP id MAA23732; Thu, 9 Jan 1997 12:21:56 -0800 Date: Thu, 9 Jan 1997 12:21:55 -0800 (PST) From: Richard Frost To: dinucci@nas.nasa.gov cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Subject: unnecessary complexities (was: Public Defined) In-Reply-To: <199701091934.LAA26528@win182.nas.nasa.gov> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Thu, 9 Jan 1997, Dr. David C. DiNucci wrote: > If I understand your point, it is that applications users often won't use > the standard interface in any case: They will use instead some more > application-specific software built upon the standard interface. I agree, > and I stated so in an earlier message. If you are suggesting that this is > a justification for making the standard interface complex, I disagree. There can be a difference between a complex software system and one with a variety of features. I believe that members of the MPI-2 Forum are striving for the latter. In my view, there is only one good reason why any component of a software system is complex: the operation is inherently complex. I disagree with the argument that MPI-2 as a whole is unnecessarily complex. I'm interested in what components (chapter sections) you believe could achieve the same functionality with less complex semantics or calling interface. > It is certainly possible for an application programmer with expertise in > some certain field to use X+Motif. (In fact, everyone in my group has done > this at some time or other, and we could probably all be considered experts > in something.) But each big package that must be learned is one more obstacle > to getting things done, and one more piece of brain clutter. The point was that these students don't learn the package. Instead, they learn the concepts and primitives of graphics programming and then get a good tutorial on how to implement with X+Motif. Many of them never look farther than a man page. Certainly none of them are studying 2100 pages in 10 weeks! For those that do pull the O'Reilly manuals off the shelf, I believe their excursions are limited to a few pages. From mpi-core-human@mcs.anl.gov Thu Jan 9 16:46:53 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA10528; Thu, 9 Jan 1997 16:46:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA02732 for mpi-core-out; Thu, 9 Jan 1997 15:51:02 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA02714; Thu, 9 Jan 1997 15:50:53 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id NAA26677; Thu, 9 Jan 1997 13:50:55 -0800 (PST) Date: Thu, 9 Jan 1997 13:50:55 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701092150.NAA26677@win182.nas.nasa.gov> To: Richard Frost Subject: Re: unnecessary complexities (was: Public Defined) Cc: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > There can be a difference between a complex software system and one with > a variety of features. I believe that members of the MPI-2 > Forum are striving for the latter. > In my view, there is only one good reason why any component of a > software system is complex: the operation is inherently complex. > I disagree with the argument that MPI-2 as a whole is unnecessarily > complex. I'm interested in what components (chapter sections) you > believe could achieve the same functionality with less complex > semantics or calling interface. The primary problem, I believe, is in how the standard is broken down into components. One-sided communication is taken to be different from message-passing, rather than a version of it or a component of it. Communicators (or "hypertags", as I have heard them called) are taken to be a primitive part of the standard, rather than a construction on top of tags (so now there are multiple kinds of communicators and multiple constructors for them *in the standard*). Types are taken to be opaque objects which are constructed using MPI operations, rather than just simple arguments (so now there are multiple kinds of types and multiple constructors for them *in the standard*). I am not saying that these features should not be available to users, just that they should not be primitives in the lowest level of the standard. Even with the current breakdown of functionality, I have often and vocally described my objections and suggestions for the one-sided chapter on the mailing list (see the archives for more info), and was sorry to see the dynamic processes chapter get so large. Those new "info objects" just open the floodgates, allowing even different vendors to make things as complex as they want them to be, apparently because the argument lists couldn't contain all the complexity desired in the standard. My most complete answer is to ask you to see CDS1 (pointer in my previous mail). CDS is not perfect, but it is the level that I was hoping for, and I expect that all the brain power of the forum could improve on the idea. It is based on a different philosophy, one where almost all of the design effort was focussed on keeping the interface as small and simple as possible. Only after that were some concessions made to increase efficiency on some architectures. I think that the decision to base MPI-2 on an unchanged MPI-1 made a similar philosophy difficult to adopt by the forum. > The point was that these students don't learn the package. Instead, > they learn the concepts and primitives of graphics programming and then > get a good tutorial on how to implement with X+Motif. Many of them > never look farther than a man page. Certainly none of them are > studying 2100 pages in 10 weeks! For those that do pull the O'Reilly > manuals off the shelf, I believe their excursions are limited to > a few pages. Maybe I'm just getting old and mentally feeble. Still, even X is built in layers: X, Xt, Motif or OL or Athena or whatever. The user can take the layer that best suits the job at hand. Layering was under discussion for MPI when it was first being formulated, but was (unfortunately) discarded. -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Fri Jan 10 10:28:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA20070; Fri, 10 Jan 1997 10:28:22 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA13728 for mpi-core-out; Fri, 10 Jan 1997 09:32:12 -0600 Received: from aloisius.vcpc.univie.ac.at (aloisius.vcpc.univie.ac.at [193.171.58.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA13173; Fri, 10 Jan 1997 09:08:55 -0600 Received: (from smap@localhost) by aloisius.vcpc.univie.ac.at (8.7.5/8.7.3) id QAA24473; Fri, 10 Jan 1997 16:06:26 +0100 (MET) From: Ian Glendinning X-Authentication-Warning: aloisius.vcpc.univie.ac.at: smap set sender to using -f Received: from beavis(193.171.58.38) by aloisius via smap (V1.3) id sma024463; Fri Jan 10 16:06:09 1997 Received: (from ian@localhost) by beavis.vcpc.univie.ac.at (8.7.1/8.7.1) id QAA04550; Fri, 10 Jan 1997 16:06:08 +0100 (MET) Date: Fri, 10 Jan 1997 16:06:08 +0100 (MET) Message-Id: <199701101506.QAA04550@beavis.vcpc.univie.ac.at> To: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Subject: Re: Public Defined Cc: frost@sdsc.edu, dinucci@nas.nasa.gov X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear all, it's been a while since I have contributed to discussion in these lists, though I still keep an eye on them, but I have found David DiNucci's comments sufficiently provocative that I feel I must join Richard Frost in voicing dissent with his views! Luckily Richard has saved me some time, by having already expressed many of my own opinions on the issue of complexity, so on that point I would simply like to say that I agree with him (though I have some extra comments of my own - see later). Specifically I agree with Richard's comments that: > I disagree with the notion "the entire standard should be totally > comprehensible by any single person" -- especially if you mean that any > person should be able to learn a majority of MPI with little effort. and also with a subsequent message: > I disagree with the argument that MPI-2 as a whole is unnecessarily > complex. I'm interested in what components (chapter sections) you > believe could achieve the same functionality with less complex > semantics or calling interface. I also have some comments of my own. David DiNucci wrote: > If the MPI standard combines actions which are not necessarily related (e.g. > creating processes and manipulating communicators) into a single primitive, > this restricts the interfaces being built upon the standard to have the > same combined actions. I believe that it is much more effective to build > a specialized interface out of small and easy-to-understand primitive actions > than it is to refashion complex actions into a different form, and I would > posit that it is difficult and/or inefficient to try to refashion complex > actions into simple primitives through an intermediate interface. I think the issue here is one of pragmatism and commercial influences rather than anything technical. I actually agree with David that it would be a much more elegant and flexible approach to define a small and easy-to-understand set of primitive actions, but unfortunately any such set of primitives is inevitably going to be better suited to some vendors architectures than others, and more to the point is probably going to be rather *badly* suited to one or two. For this reason, it has never been possible to get the vendors to agree on a standard based on that philosophy, and I believe that the backing of the vendors has been crucial to the success of MPI. [Aside: Not so long ago it would still have been premature to speak of the success of MPI, but I think we can all agree that it's now safe to do so - I was amused to notice recently that last year's "EuroPVM 1996" meeting will this year be called "EuroPVM-MPI'97" :-] Anyway, back to the main theme - the small set of primitives approach has been tried, but no such interface has ever become widely adopted. On the other hand, the approach that MPI took, as I see it, was basically to standardise at a slightly higher level than a set of neat primitives, which does indeed mean that it's a bit more hard-wired, so to speak, and new functionality does require additions to the standard (MPI-2) rather than a new library layer, and this is also why MPI contains a relatively large number of functions, so as to cover most, though not all, of the features that people might conceivably construct out of a small set of primitives. However, being specified at that very slightly higher level confers the advantage that each vendor is free to choose their own set of low-level primitives to construct the MPI layer with, allowing them to come as close as is reasonably possible to achieving maximum performance from a portable interface. Ian -- Ian Glendinning European Centre for Parallel Computing at Vienna (VCPC) ian@vcpc.univie.ac.at Liechtensteinstr. 22, A-1090 Vienna, Austria Tel: +43 1 310 939612 WWW: http://www.vcpc.univie.ac.at/~ian/ From mpi-core-human@mcs.anl.gov Fri Jan 10 12:14:48 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA21222; Fri, 10 Jan 1997 12:14:46 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA16297 for mpi-core-out; Fri, 10 Jan 1997 11:18:45 -0600 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA16292 for ; Fri, 10 Jan 1997 11:18:42 -0600 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.32.106]) by franklin.sdsc.edu (8.8.3/8.8.3/SDSCserver-14) with SMTP id JAA19071; Fri, 10 Jan 1997 09:18:14 -0800 (PST) Received: from localhost by rogue.sdsc.edu (950413.SGI.8.6.12/1.11-client) with SMTP id JAA23990; Fri, 10 Jan 1997 09:18:14 -0800 Date: Fri, 10 Jan 1997 09:18:14 -0800 (PST) From: Richard Frost Reply-To: Richard Frost To: Ian Glendinning cc: mpi-core@mcs.anl.gov Subject: the MPI approach Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Ian, Thanks for your kind remarks. Also, your reply includes a fine characterization of the MPI approach. I believe it would be helpful to include an edited version of it in the preface to the MPI standard document(s), and generally in any text on MPI. - Richard ---------- Forwarded message ---------- I think the issue here is one of pragmatism and commercial influences rather than anything technical. I actually agree with David that it would be a much more elegant and flexible approach to define a small and easy-to-understand set of primitive actions, but unfortunately any such set of primitives is inevitably going to be better suited to some vendors architectures than others, and more to the point is probably going to be rather *badly* suited to one or two. For this reason, it has never been possible to get the vendors to agree on a standard based on that philosophy, and I believe that the backing of the vendors has been crucial to the success of MPI. [Aside: Not so long ago it would still have been premature to speak of the success of MPI, but I think we can all agree that it's now safe to do so - I was amused to notice recently that last year's "EuroPVM 1996" meeting will this year be called "EuroPVM-MPI'97" :-] Anyway, back to the main theme - the small set of primitives approach has been tried, but no such interface has ever become widely adopted. On the other hand, the approach that MPI took, as I see it, was basically to standardise at a slightly higher level than a set of neat primitives, which does indeed mean that it's a bit more hard-wired, so to speak, and new functionality does require additions to the standard (MPI-2) rather than a new library layer, and this is also why MPI contains a relatively large number of functions, so as to cover most, though not all, of the features that people might conceivably construct out of a small set of primitives. However, being specified at that very slightly higher level confers the advantage that each vendor is free to choose their own set of low-level primitives to construct the MPI layer with, allowing them to come as close as is reasonably possible to achieving maximum performance from a portable interface. -- Ian Glendinning European Centre for Parallel Computing at Vienna (VCPC) ian@vcpc.univie.ac.at Liechtensteinstr. 22, A-1090 Vienna, Austria Tel: +43 1 310 939612 WWW: http://www.vcpc.univie.ac.at/~ian/ From mpi-core-human@mcs.anl.gov Fri Jan 10 13:40:58 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA22135; Fri, 10 Jan 1997 13:40:55 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA18100 for mpi-core-out; Fri, 10 Jan 1997 12:43:18 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA18076; Fri, 10 Jan 1997 12:43:01 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id KAA28454; Fri, 10 Jan 1997 10:31:09 -0800 (PST) Date: Fri, 10 Jan 1997 10:31:09 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701101831.KAA28454@win182.nas.nasa.gov> To: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov, Ian Glendinning Subject: Re: Public Defined Cc: frost@sdsc.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Ian Glendinning writes: > Dear all, > it's been a while since I have contributed to discussion in these > lists, though I still keep an eye on them, but I have found David DiNucci's > comments sufficiently provocative that I feel I must join Richard Frost in > voicing dissent with his views! Well at least my comments may have helped in one way: to start some discussion on these matters, out of a formerly rather silent "public discussion period". > I think the issue here is one of pragmatism and commercial influences rather > than anything technical. I actually agree with David that it would be a much > more elegant and flexible approach to define a small and easy-to-understand > set of primitive actions, but unfortunately any such set of primitives is > inevitably going to be better suited to some vendors architectures than > others, and more to the point is probably going to be rather *badly* suited to > one or two. For this reason, it has never been possible to get the vendors to > agree on a standard based on that philosophy, and I believe that the backing > of the vendors has been crucial to the success of MPI. If one accepts the last phrase of the above as fact, then the rest of what you say is perhaps substantiated, making my goal of simplicity impossible. I hate to think of this in an "us vs. them" framework, but if "we" (the users and toolbuilders) are relying on "them" (the vendors) to determine the standard interface, then "they" are certainly going to put more importance on the performance on their particular architecture and less on simplicity than "we" are. And, as I have already stated, the balance of vendors to users on the forum is certainly far out of balance to that in the real world. However, I should also add that even implementors, who generally work for vendors, are often on the side of simplicity, since they must implement and maintain the standard on that architecture. I also want to repeat the time argument. These vendors are fighting for an interface which will be efficient on their architecture *right now*, but I presume the standard is to last into the future, where the users needs will probably become even more diverse, and the architectures are likely to change. I think it would be good if the standards reflected enough of the users' needs that the architecture designers had a target for future development, rather than just an interface to run efficiently on todays architectures. > Anyway, back to the main theme - the small set of > primitives approach has been tried, but no such interface has ever become > widely adopted. Has such a large group of qualified people ever put their collective minds to it? > On the other hand, the approach that MPI took, as I see it, > was basically to standardise at a slightly higher level than a set of neat > primitives, which does indeed mean that it's a bit more hard-wired, so to > speak, and new functionality does require additions to the standard (MPI-2) > rather than a new library layer, and this is also why MPI contains a > relatively large number of functions, so as to cover most, though not all, of > the features that people might conceivably construct out of a small set of > primitives. However, being specified at that very slightly higher level > confers the advantage that each vendor is free to choose their own set of > low-level primitives to construct the MPI layer with, allowing them to come as > close as is reasonably possible to achieving maximum performance from a > portable interface. I, also, believe that each vendor should be free to choose their own set of low-level primitives to construct the "MPI" layer with. I just don't believe that the MPI layer should be the lowest layer available to the user. Even in the CDS work which I continually reference, there is a proposed layer, CDS2, which is very similar to MPI in many ways (including "communicators" and topologies and the rest), it just isn't the lowest level of interface available to users. CDS2 is defined in terms of CDS1 (the kernel level), but vendors are free to implement CDS2 using any means they desire. (There is an overview of this in the last MPI Developer's Conference Proceedings.) This contradicts some personal email I have received from Richard, which suggests that I find MPI "repulsive". I do not. I think that there are many fine ideas in MPI, which I am happy to steal^H^H^H^H^Hborrow for my work. I'm just not convinced that they belong in the lowest level of a standard user interface. So it could very well be that my primary argument is with the decision, made long ago, to base MPI-2 on an unchanged MPI-1. -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Fri Jan 10 15:12:26 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA23188; Fri, 10 Jan 1997 15:12:24 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA20582 for mpi-core-out; Fri, 10 Jan 1997 14:13:23 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA20570 for ; Fri, 10 Jan 1997 14:12:59 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id MAA28971 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Fri, 10 Jan 1997 12:13:04 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id MAA04202; Fri, 10 Jan 1997 12:10:53 -0800 From: "Eric Salo" Message-Id: <9701101210.ZM4200@mrjones.engr.sgi.com> Date: Fri, 10 Jan 1997 12:10:51 -0800 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: MPI naming rules Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Introduction: With the upcoming increase in the number of functions defined by MPI, there is a real need for a consistent naming convention. To address this need, Jeff and I have been working to identify a set of rules which both summarize the current practice in MPI-1 and suggest changes that might be appropriate for MPI-2. That effort is now essentially completed, and is presented below. Those with short attention spans should just skip to the section marked (******) to see the bottom line. Executive summary: What we found is that 108 of the 128 routines in MPI-1 could be described by the following three rules: Rule 1: All routines must be of the form Object_Action_Subset unless they qualify for abbreviation. Subsets may be omitted where none exist. Rule 2: The object may be inferred if the action cannot be applied to any other object type. Rule 3: Subsets which are fundamentally read-only in nature need not have an explicit 'Get' action. There are some subrules associated with the above which are a bit subjective and deal with questions such as which objects may be reasonably inferred from which actions. For example, we take the religious position that the 'Test' action should only apply to request objects even though there are two routines in MPI-1 which do not follow this rule. Of the 20 MPI-1 routines which do not follow the above rules, we distinguish between different degrees of badness and conclude that there are only a few which absolutely need to have new, alternate names. The full summary of the rules and how they apply to both MPI-1 and MPI-2 follows. Disclaimer: We did our best to be objective but that was not always possible. This is still very much a work-in-progress (and only a set of suggestions at any rate), so please tear into it mercilessly! ------------------------------------------------------------------------------- MPI-1 ------------------------------------------------------------------------------- Rule 1: All routines must be of the form Object_Action_Subset unless they qualify for abbreviation. Subsets may be omitted where none exist. In addition, actions should have some degree regularity/reasonableness across objects: Subrule 1a: The action for reading a value is "get". Subrule 1b: The action for writing a value is "set". Subrule 1c: The action for testing a boolean is "is". Subrule 1d: The action for destroying an object is "free". Rule 2: The object may be inferred if the action cannot be applied to any other object type. Subrule 2a: The "global" object has the following unique actions: buffer_attach buffer_detach finalize init pcontrol Subrule 2b: The comm object has the following unique actions: abort allgather (includes allgatherv) allreduce alltoall (includes alltoallv) barrier bcast gather (includes gatherv) probe (includes iprobe) recv (includes all variations) reduce scan scatter (includes scatterv) send (includes all variations) Subrule 2c: The request object has the following unique actions: cancel start (includes startall) test (includes all variations) wait (includes all variations) Subrule 2d: The type object has the following unique actions: pack pack_size unpack Rule 3: The following object subsets, being fundamentally read-only in nature, need not have an explicit "get" action: class (error codes) coords (cart) group (comms) neighbors (graph) neighbors_count (graph) rank (cart, comms and groups) remote_group (comms) remote_size (comms) size (comms and groups) string (error codes) wtick (global) wtime (global) ------ The following is a list of how each MPI-1 routine conforms to the above rules. If the routine "passes", then the appropriate rules are listed. If it does not pass, then a "correct" name is provided. This is all just theoretical for the moment. MPI-1 Rules ----- ----- mpi_abort 1, 2(comm) mpi_address 1, 2(global) mpi_allgather 1, 2(comm) mpi_allgatherv 1, 2(comm) mpi_allreduce 1, 2(comm) mpi_alltoall 1, 2(comm) mpi_alltoallv 1, 2(comm) mpi_attr_delete mpi_comm_delete_attr mpi_attr_get mpi_comm_get_attr mpi_attr_put mpi_comm_put_attr mpi_barrier 1, 2(comm) mpi_bcast 1, 2(comm) mpi_bsend 1, 2(comm) mpi_bsend_init 1, 2(comm) mpi_buffer_attach 1, 2(global) mpi_buffer_detach 1, 2(global) mpi_cancel 1, 2(request) mpi_cart_coords 1, 3 mpi_cart_create 1 mpi_cart_get mpi_cart_get_topo mpi_cart_map 1 mpi_cart_rank 1, 3 mpi_cart_shift 1 mpi_cart_sub 1 mpi_cartdim_get mpi_cart_get_dim mpi_comm_compare 1 mpi_comm_create 1 mpi_comm_dup 1 mpi_comm_free 1 mpi_comm_group 1, 3 mpi_comm_rank 1, 3 mpi_comm_remote_group 1, 3 mpi_comm_remote_size 1, 3 mpi_comm_size 1, 3 mpi_comm_split 1 mpi_comm_test_inter mpi_comm_is_inter mpi_dims_create 1 mpi_errhandler_create 1 mpi_errhandler_free 1 mpi_errhandler_get mpi_comm_get_errhandler mpi_errhandler_set mpi_comm_set_errhandler mpi_error_class 1, 3 mpi_error_string 1, 3 mpi_finalize 1, 2(global) mpi_gather 1, 2(comm) mpi_gatherv 1, 2(comm) mpi_get_count mpi_status_get_count mpi_get_elements mpi_status_get_elements mpi_get_processor_name mpi_global_get_processor_name mpi_graph_create 1 mpi_graph_get mpi_graph_get_topo mpi_graph_map 1 mpi_graph_neighbors 1, 3 mpi_graph_neighbors_count 1, 3 mpi_graphdims_get mpi_graph_get_dims mpi_group_compare 1 mpi_group_difference 1 mpi_group_excl 1 mpi_group_free 1 mpi_group_incl 1 mpi_group_intersection 1 mpi_group_range_excl 1 mpi_group_range_incl 1 mpi_group_rank 1, 3 mpi_group_size 1, 3 mpi_group_translate_ranks 1 mpi_group_union 1 mpi_ibsend 1, 2(comm) mpi_init 1, 2(global) mpi_initialized mpi_global_is_initialized mpi_intercomm_create 1 mpi_intercomm_merge 1 mpi_iprobe 1, 2(comm) mpi_irecv 1, 2(comm) mpi_irsend 1, 2(comm) mpi_isend 1, 2(comm) mpi_issend 1, 2(comm) mpi_keyval_create 1 mpi_keyval_free 1 mpi_op_create 1 mpi_op_free 1 mpi_pack 1, 2(type) mpi_pack_size 1, 2(type) mpi_pcontrol 1, 2(global) mpi_probe 1, 2(comm) mpi_recv 1, 2(comm) mpi_recv_init 1, 2(comm) mpi_reduce 1, 2(comm) mpi_reduce_scatter 1, 2(comm) mpi_request_free 1 mpi_rsend 1, 2(comm) mpi_rsend_init 1, 2(comm) mpi_scan 1, 2(comm) mpi_scatter 1, 2(comm) mpi_scatterv 1, 2(comm) mpi_send 1, 2(comm) mpi_send_init 1, 2(comm) mpi_sendrecv 1, 2(comm) mpi_sendrecv_replace 1, 2(comm) mpi_ssend 1, 2(comm) mpi_ssend_init 1, 2(comm) mpi_start 1, 2(comm) mpi_startall 1, 2(comm) mpi_test 1, 2(request) mpi_test_cancelled mpi_request_is_cancelled mpi_testall 1, 2(request) mpi_testany 1, 2(request) mpi_testsome 1, 2(request) mpi_topo_test mpi_comm_get_topo mpi_type_commit 1 mpi_type_contiguous 1 mpi_type_extent mpi_type_get_extent mpi_type_free 1 mpi_type_hindexed 1 mpi_type_hvector 1 mpi_type_indexed 1 mpi_type_lb mpi_type_get_lb mpi_type_size mpi_type_get_size mpi_type_struct 1 mpi_type_ub mpi_type_get_ub mpi_type_vector 1 mpi_unpack 1, 2(type) mpi_wait 1, 2(request) mpi_waitall 1, 2(request) mpi_waitany 1, 2(request) mpi_waitsome 1, 2(request) mpi_wtick 1, 3 mpi_wtime 1, 3 (******) So, given that we have 20 MPI-1 routines which don't fit into our nice clean model, what should be done about them? Case 1: Definitely broken, should be fixed by providing new, alternate names and depricating the old names: mpi_get_count -> mpi_status_get_count mpi_get_elements -> mpi_status_get_elements Case 2: Need to be changed in MPI-2 for other reasons, so we might as well fix the names while we're at it (and depricate the old names): mpi_attr_delete -> mpi_comm_delete_attr mpi_attr_get -> mpi_comm_get_attr mpi_attr_put -> mpi_comm_put_attr Case 3: "Reserved" actions, should probably be fixed/depricated: mpi_comm_test_inter -> mpi_comm_is_inter mpi_test_cancelled -> mpi_request_is_cancelled Case 4: Goofy "global" actions that are technically incorrect but should probably just be left as is: mpi_get_processor_name mpi_initialized Case 5: Definitely broken, but since they appear in the topology chapter we'll just ignore them like the rest of the world does: mpi_cart_get mpi_cartdim_get mpi_graph_get mpi_graphdims_get mpi_topo_test Case 6: ??? (Some feedback here would be nice) mpi_errhandler_get -> (?) mpi_comm_get_errhandler mpi_errhandler_set -> (?) mpi_comm_set_errhandler mpi_type_extent -> (?) mpi_type_get_extent mpi_type_lb -> (?) mpi_type_get_lb mpi_type_size -> (?) mpi_type_get_size mpi_type_ub -> (?) mpi_type_get_ub ------------------------------------------------------------------------------- MPI-2 ------------------------------------------------------------------------------- For brevity (too late, I know) I'll just list the MPI-2 routines which are "broken" and suggest replacements. I've almost certainly missed a few. Note that the entire I/O chapter passes! Can that possibly be right??? :-) Chapter 4: MPI_Name_publish -> MPI_Port_publish_name MPI_Name_unpublish -> MPI_Port_unpublish_name MPI_Name_get -> MPI_Port_lookup_name Chapter 5: MPI_Put -> MPI_Win_put MPI_Get -> MPI_Win_get MPI_Accumulate -> MPI_Win_accumulate Chapter 7: MPI_Set_elements_status -> MPI_Status_set_elements MPI_Ident_put -> MPI_Comm_put_ident MPI_Ident_get -> MPI_Comm_get_ident MPI_Get_type_envelope -> MPI_Type_get_envelope MPI_Get_type_contents -> MPI_Type_get_contents MPI_Post_handler -> (?) MPI_Request_put_handler Chapter 9: MPI_Get_status -> MPI_Request_get_status MPI_Type_hvector_x MPI_Type_hindexed_x MPI_Type_struct_x MPI_Address_x MPI_Get_extent_x MPI_Type_resized_x MPI_True_extent_x MPI_Keyval_create_x MPI_Attr_put_x MPI_Attr_get_x These '_x' routines are a terrible mess and should be changed to whatever we select as the solutions for their MPI-1 counterparts. For example, suggesting MPI_Type_get_extent as the new name for MPI_Type_extent would both fix the ambiguous name in MPI-1 and provide a new binding that solved the 64-bit problem. Some good analysis is needed here. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Jan 10 16:05:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA23640; Fri, 10 Jan 1997 16:05:35 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA21753 for mpi-core-out; Fri, 10 Jan 1997 15:08:51 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA21748 for ; Fri, 10 Jan 1997 15:08:47 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id NAA11323 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Fri, 10 Jan 1997 13:08:49 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id NAA04349; Fri, 10 Jan 1997 13:06:38 -0800 From: "Eric Salo" Message-Id: <9701101306.ZM4347@mrjones.engr.sgi.com> Date: Fri, 10 Jan 1997 13:06:38 -0800 In-Reply-To: "Eric Salo" "MPI naming rules" (Jan 10, 12:10pm) References: <9701101210.ZM4200@mrjones.engr.sgi.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: MPI naming rules Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Two separate people have already informed me that I misspelled the word 'deprecate' in my post, so the rest of you need not bother. Your collective concern for my education is appreciated. :-) -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Jan 10 21:06:53 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA26225; Fri, 10 Jan 1997 21:06:52 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA27729 for mpi-core-out; Fri, 10 Jan 1997 20:08:09 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA27724; Fri, 10 Jan 1997 20:08:04 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id SAA15602; Fri, 10 Jan 1997 18:08:11 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) id SAA05678; Fri, 10 Jan 1997 18:05:20 -0800 From: "Eric Salo" Message-Id: <9701101805.ZM5676@mrjones.engr.sgi.com> Date: Fri, 10 Jan 1997 18:05:19 -0800 In-Reply-To: Ian Glendinning "Re: Public Defined" (Jan 10, 4:06pm) References: <199701101506.QAA04550@beavis.vcpc.univie.ac.at> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Public Defined Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I actually agree with David that it would be a much more elegant and flexible > approach to define a small and easy-to-understand set of primitive actions, > but unfortunately [...] it has never been possible to get the vendors to agree > on a standard based on that philosophy I guess it's about time I joined in on the fun... My experience at the MPI Forum meetings has been exactly the reverse of the above. In general, it is the vendors and implementors who have been arguing in favor of simplicity, and the users who have been saying "but I really need this fancy whiz-bang function". I would welcome anyone who disagrees to try getting something deleted from any chapter in MPI-2 at the next meeting and watch the results. Prediction: most of the implementors will support you, and most of the users will not. For what it's worth, I agree with virtually everything that David has said on this topic (and think of him as sort of an honorary implementor because of his background with CDS). But rather than arguing about the history of MPI-1 I would be much more interested in talking about ways to improve MPI-2, which as I recall is what started this whole thread in the first place. So here's a sweeping suggestion for improving the simplicity of the entire document: let's delete *all* of the non-blocking collective routines from every chapter. I personally know of several implementors who would just love to see this happen, myself included. Rusty, could we reserve some time at the upcoming meeting for discussion on this specific topic? -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Jan 10 22:43:56 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA26903; Fri, 10 Jan 1997 22:43:55 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA28754 for mpi-core-out; Fri, 10 Jan 1997 21:48:18 -0600 Received: from msr.EPM.ORNL.GOV (msr.epm.ornl.gov [128.219.8.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA28741; Fri, 10 Jan 1997 21:48:12 -0600 Received: (from geist@localhost) by msr.EPM.ORNL.GOV (8.8.3/8.8.3) id WAA25509; Fri, 10 Jan 1997 22:48:21 -0500 (EST) Date: Fri, 10 Jan 1997 22:48:21 -0500 (EST) From: Al Geist Message-Id: <199701110348.WAA25509@msr.EPM.ORNL.GOV> To: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: premature to speak of the success of MPI Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >Ian wrote: >[Aside: Not so long >ago it would still have been premature to speak of the success of MPI, but I >think we can all agree that it's now safe to do so - I was amused to notice >recently that last year's "EuroPVM 1996" meeting will this year be called >"EuroPVM-MPI'97" :-] Actually this was too amusing to let pass. (-: The facts are that EuroPVM 1996 was the largest PVM user's group meeting to date. And while the use of PVM continues to grow at a rate far larger than MPI, it is also true that there is a growing body of MPI users who would benefit from a meeting like EuroPVM so next year it is being called EuroPVM-MPI'97. And to add my two cents in to the Dave and MPI-is-too-complex story. YES, MPI is too complex. Not only is it a great deal of work for the vendors and implementors who must support and maintain their versions, but also for the users who are presented with many ways of moving data in their application. Why so many ways? Well one argument Ian gave is so every vendor will have a "fast" way to move data. And I agree. Unfortunately the price we paid in MPI for "Performance at all costs" is a very large API. The very richness of MPI may hurt us in the long run by decreasing its portability in the following way: Scenario: I am an application developer. My site has two different computers Machine1 and Machine2 I discover that the fastest way to move data on Machine1 is MPI routines x, y My tests show that the fastest way to move data on Machine2 is MPI routines a, b So to make sure my application always get peak performance I develop it to use both methods with a compile or runtime switch to choose between them. Flashback: Before I started designing MPI-1 with the forum (with a goal of developing a portable message-passing interface) my application looked very similar except instead of using MPI routines x, y and MPI routines a, b it used Vendor1 routines x,y and Vendor2 routines a,b What have I gained by using MPI? Well maybe if these vendors go out of business my application will still run. That's a plus. But I may find I need to change routines once again to use the fastest MPI routines the remaining vendors supply. deja vu. Dave is arguing that we, the forum, should consider simplifying the MPI API. Realisticly, I do not believe we can do this. We are just too far down the design path. We designed MPI block by block following a particular set of goals. A key goal was making sure all vendors could produce high performance implementations. I think we have done a reasonable job within our constraints. Al Geist From mpi-core-human@mcs.anl.gov Sat Jan 11 20:36:33 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA01713; Sat, 11 Jan 1997 20:36:32 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA12230 for mpi-core-out; Sat, 11 Jan 1997 19:39:41 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA12211; Sat, 11 Jan 1997 19:39:35 -0600 Message-Id: <199701120139.TAA12211@antares.mcs.anl.gov> To: "Eric Salo" cc: mpi-comment@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Public Defined In-reply-to: Your message of "Fri, 10 Jan 1997 18:05:19 PST." <9701101805.ZM5676@mrjones.engr.sgi.com> Date: Sat, 11 Jan 1997 19:39:49 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | let's delete *all* of the non-blocking collective routines from every | chapter. | | Rusty, could we reserve some time at the upcoming meeting for discussion on | this specific topic? It seems to me that both the benefits and implementation difficulties of non-blocking operations are quite different in each chapter (dynamic, I/O, and collective) and so it remains more appropriate to discuss them in their own contexts. But we can certainly arrange the agenda however we want. From mpi-core-human@mcs.anl.gov Sun Jan 12 03:34:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA04236; Sun, 12 Jan 1997 03:34:42 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA16204 for mpi-core-out; Sun, 12 Jan 1997 02:37:16 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA16199 for ; Sun, 12 Jan 1997 02:37:11 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id AAA03106 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Sun, 12 Jan 1997 00:37:19 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id AAA07082; Sun, 12 Jan 1997 00:35:08 -0800 From: "Eric Salo" Message-Id: <9701120035.ZM7080@mrjones.engr.sgi.com> Date: Sun, 12 Jan 1997 00:35:07 -0800 In-Reply-To: Rusty Lusk "Re: Public Defined" (Jan 11, 7:39pm) References: <199701120139.TAA12211@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: Public Defined Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > It seems to me that both the benefits and implementation difficulties of > non-blocking operations are quite different in each chapter (dynamic, I/O, and > collective) and so it remains more appropriate to discuss them in their own > contexts. But we can certainly arrange the agenda however we want. I would argue that the implementation issues are quite similar to the extent that once we have even a single one of these critters, it opens up a whole Pandora's Box of pain. For example, someone (Greg, maybe?) has already pointed out that mandating ispawn is equivalent to mandating iallreduce. So it seems reasonable to address this in the full group, if for no other reason than to prevent it from coming up separately in each chapter. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Sun Jan 12 17:01:29 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA00569; Sun, 12 Jan 1997 17:01:28 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA23591 for mpi-core-out; Sun, 12 Jan 1997 13:56:59 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA23586; Sun, 12 Jan 1997 13:56:49 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id LAA09702; Sun, 12 Jan 1997 11:57:05 -0800 (PST) Date: Sun, 12 Jan 1997 11:57:05 -0800 (PST) From: dinucci@nas.nasa.gov (David C. DiNucci) Message-Id: <199701121957.LAA09702@win182.nas.nasa.gov> To: Al Geist , mpi-comment@mcs.anl.gov Subject: Conclusion? Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Al Geist writes: > Dave is arguing that we, the forum, should consider simplifying > the MPI API. Realisticly, I do not believe we can do this. > We are just too far down the design path. > We designed MPI block by block following a particular set of goals. > A key goal was making sure all vendors could produce high performance > implementations. I think we have done a reasonable job within our constraints. I suppose it is a little late for me to be pointing out blemishes if they cannot be remedied. I was perhaps taking the term "open comment period", and my favorite quote, a little too literally: "The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man." -- G. B. Shaw I believe that the current standard being presented by the forum will fulfill a valuable role, and I hope that they can put the finishing touches on it soon. After all, it's the only standard we've got (aside from PVM and P4 :-). Lots of good people have put lots of work into it, and it appears to be as promised; a widely-supported API which allows users to write efficient message-passing programs. Even if it is possible to provide a simplified API by using a radically different design, I'm sure that the experience gained by the development and use of MPI will prove very valuable to that effort, and that MPI will be the yardstick by which it will be measured. I am willing to discuss more, but since I see that the forum is meeting again soon, I thank the forum and others for offering this discussion period, for putting up with my sometimes rather harsh comments, and for taking the time to answer them intelligently. -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Sun Jan 12 17:53:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA00680; Sun, 12 Jan 1997 17:53:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA25824 for mpi-core-out; Sun, 12 Jan 1997 16:57:12 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA25819 for ; Sun, 12 Jan 1997 16:57:08 -0600 Message-Id: <199701122257.QAA25819@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: attendance at January meeting Date: Sun, 12 Jan 1997 16:57:28 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the list of people who have told me that they are coming to the January meeting. If you are coming but not on the list, or vice versa, please let me know. - Rusty Pang Chen Albert Cheng Dennis Cottel Raja Daoud Judith Devaney Terry Dontje Richard Frost Al Geist Bill Gropp John Hagedorn Leslie Hart Rolf Hempel Hans-Christian Hoppe Steve Huss-Lederman Arkady Kanevsky Steve Landherr Lloyd Lewins Andrew Lumsdaine Rusty Lusk John May Tyce McLarty Harish Nag Nick Nevin Heidi Poxon Jean-Pierre Prost Rolf Rabenseifner T Robey Anna Rounbehler Eric Salo Tony Skjellum Marc Snir Rajeev Thakur MaryDell Tholburn Dick Treumann From mpi-core-human@mcs.anl.gov Sun Jan 12 19:41:09 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA01987; Sun, 12 Jan 1997 19:41:08 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA26961 for mpi-core-out; Sun, 12 Jan 1997 18:44:23 -0600 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA26956 for ; Sun, 12 Jan 1997 18:44:19 -0600 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (8.7.6/7.0m-FWP-MsState); id SAA24150; Sun, 12 Jan 1997 18:04:44 -0600 (CST) Date: Sun, 12 Jan 1997 18:04:44 -0600 (CST) From: Tony Skjellum To: Eric Salo cc: mpi-core@mcs.anl.gov Subject: Re: Public Defined In-Reply-To: <9701120035.ZM7080@mrjones.engr.sgi.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Folks, >From the point of view of parliamentary procedure, Eric is asking to have time on the agenda to make a motion. There will certainly be a second if he makes this motion. I think it an open and fair way to attempt changes of directions to declare one's approach in advance, and this should get floor time. In fact, it seems like a fixed amount of time for such motions to strike sets of features or concepts from the standard is not only in order, but should be scheduled, possibly at each meeting to the end of MPI-2. Since MPI is concept-driven, and the concept of non-blocking operations has significant related impacts on implementations, this seems like a well-formed motion. As I recall, this broad stroke motion was brought up earlier and not allowed to proceed because of parliamentary arguments during the first reading of the collective chapter. It should get a chance to come up in the appropriate session. Please note that I am not speaking either for or against this proposition, just for the openness of getting it considered. -Tony On Sun, 12 Jan 1997, Eric Salo wrote: > Date: Sun, 12 Jan 1997 00:35:07 -0800 > From: Eric Salo > To: mpi-core@mcs.anl.gov > Subject: Re: Public Defined > > > It seems to me that both the benefits and implementation difficulties of > > non-blocking operations are quite different in each chapter (dynamic, I/O, > and > > collective) and so it remains more appropriate to discuss them in their own > > contexts. But we can certainly arrange the agenda however we want. > > I would argue that the implementation issues are quite similar to the extent > that once we have even a single one of these critters, it opens up a whole > Pandora's Box of pain. For example, someone (Greg, maybe?) has already pointed > out that mandating ispawn is equivalent to mandating iallreduce. So it seems > reasonable to address this in the full group, if for no other reason than to > prevent it from coming up separately in each chapter. > > -- > Eric Salo Silicon Graphics Inc. "Do you know what the > (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just > salo@sgi.com Mountain View, CA 94043-1389 before he died?" > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; "Persistence is fertile." ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Mon Jan 13 11:36:31 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA12639; Mon, 13 Jan 1997 11:36:30 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA09156 for mpi-core-out; Mon, 13 Jan 1997 10:37:16 -0600 Received: from relay1.UU.NET (relay1.UU.NET [192.48.96.5]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA09150 for ; Mon, 13 Jan 1997 10:37:11 -0600 Received: from sky.com by relay1.UU.NET with SMTP (peer crosschecked as: sky.SKY.COM [198.4.246.2]) id QQbyik22796; Mon, 13 Jan 1997 11:33:41 -0500 (EST) Received: from anna.sky.com by sky.com (SMI-8.6/SMI-SVR4) id LAA24348; Mon, 13 Jan 1997 11:32:31 -0500 Received: from anna by anna.sky.com (SMI-8.6/SMI-SVR4) id LAA06602; Mon, 13 Jan 1997 11:30:03 -0500 Message-ID: <32DA630B.67C@sky.com> Date: Mon, 13 Jan 1997 11:30:03 -0500 From: Anna Rounbehler Organization: Sky Computers X-Mailer: Mozilla 3.0Gold (X11; I; SunOS 5.5.1 sun4c) MIME-Version: 1.0 To: Eric Salo CC: mpi-core@mcs.anl.gov Subject: Re: Public Defined References: <199701120139.TAA12211@antares.mcs.anl.gov> <9701120035.ZM7080@mrjones.engr.sgi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Eric Salo wrote: > I would argue that the implementation issues are quite similar to the extent > that once we have even a single one of these critters, it opens up a whole > Pandora's Box of pain. For example, someone (Greg, maybe?) has already pointed > out that mandating ispawn is equivalent to mandating iallreduce. So it seems > reasonable to address this in the full group, if for no other reason than to > prevent it from coming up separately in each chapter. > > I agree. In fact, in September I thought we agreed that "cross referenced" issues would be addressed by the larger group as required. I brought up the suggestion, as a result of concern fo inconsistencies and/or design conflicts across chapters. This will help us to address possible overlapping logic and either agree to accept it or reject it. ------------ Anna Rounbehler anna@sky.com From mpi-core-human@mcs.anl.gov Mon Jan 13 12:33:40 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA13617; Mon, 13 Jan 1997 12:33:34 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA11046 for mpi-core-out; Mon, 13 Jan 1997 11:37:39 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA11041 for ; Mon, 13 Jan 1997 11:37:34 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id MAA07700; Mon, 13 Jan 1997 12:38:29 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/11-26-96) with SMTP id MAA71806; Mon, 13 Jan 1997 12:37:47 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA Release 1.0.1) id 8525641E.0060D1F5 ; Mon, 13 Jan 1997 12:37:32 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: dinucci@nas.nasa.gov cc: Mpi-Core@mcs.anl.gov Message-ID: <8525641E:005FDD9A.00@watngi01.watson.ibm.com> Date: Mon, 13 Jan 1997 12:36:24 -0400 Subject: Re: unnecessary complexities (was: Public Defined) Mime-Version: 1.0 Content-type: multipart/mixed; Boundary="0__=xWJBzENSroYfOJ770eZruaQN4d4spAN8cvpDi8EljOuY6BYdSyFI3Vxm" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk --0__=xWJBzENSroYfOJ770eZruaQN4d4spAN8cvpDi8EljOuY6BYdSyFI3Vxm Content-type: text/plain; charset=us-ascii From: Marc Snir on 01-13-97 12:36 PM (Embedded image moved dinucci @ nas.nasa.gov to file: 01/09/97 04:50 PM PIC32460.PCX) To: fdinucci@nas.nasa.com cc: mpi-comment @ mcs.anl.gov, mpi-core @ mcs.anl.gov (bcc: Marc Snir/Watson/IBM Research) Subject: Re: unnecessary complexities (was: Public Defined) --0__=xWJBzENSroYfOJ770eZruaQN4d4spAN8cvpDi8EljOuY6BYdSyFI3Vxm You basically argue that MPI1 was not designed correctly, and that MPI2 compounds the crime. MPI1 had heated discussions on some of the issues you mention -- in particular communicators vs tags. The decision went one way -- the main arguments were that this way leaves more freedom to implementors -- in particular it allows for different divisions between hardware/firmware supported functions and software supported functions, and that this design is safer. The decisions may have been wrong, and CDS1 may be a superior design. But this is clearly not a fruitful debate for the MPI forum. Clearly, the MPI forum has to design MPI2 using the same design principles as MPI1, while other groups or individuals may push designs built on totally different principles. ***************************** The primary problem, I believe, is in how the standard is broken down into components. One-sided communication is taken to be different from message-passing, rather than a version of it or a component of it. Communicators (or "hypertags", as I have heard them called) are taken to be a primitive part of the standard, rather than a construction on top of tags (so now there are multiple kinds of communicators and multiple constructors for them *in the standard*). Types are taken to be opaque objects which are constructed using MPI operations, rather than just simple arguments (so now there are multiple kinds of types and multiple constructors for them *in the standard*). I am not saying that these features should not be available to users, just that they should not be primitives in the lowest level of the standard. Even with the current breakdown of functionality, I have often and vocally described my objections and suggestions for the one-sided chapter on the mailing list (see the archives for more info), and was sorry to see the dynamic processes chapter get so large. Those new "info objects" just open the floodgates, allowing even different vendors to make things as complex as they want them to be, apparently because the argument lists couldn't contain all the complexity desired in the standard. My most complete answer is to ask you to see CDS1 (pointer in my previous mail). CDS is not perfect, but it is the level that I was hoping for, and I expect that all the brain power of the forum could improve on the idea. It is based on a different philosophy, one where almost all of the design effort was focussed on keeping the interface as small and simple as possible. Only after that were some concessions made to increase efficiency on some architectures. I think that the decision to base MPI-2 on an unchanged MPI-1 made a similar philosophy difficult to adopt by the forum. -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 --0__=xWJBzENSroYfOJ770eZruaQN4d4spAN8cvpDi8EljOuY6BYdSyFI3Vxm Content-type: application/octet-stream; name="PIC32460.PCX" Content-transfer-encoding: base64 CgUBCAAAAABoACwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAABaQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAD1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPH E8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sT zRPHE8MTwhPwEwzIBgzYE8wTxhPDE8IT7hPOBtcTzBPGE8MTE+wTwgbCBwbCEgbCEgbCEsUG1hPL E8YTwxMT6hMMwgYHwgLCAwISwgfEEsMCwwbVE8sTxRPDExPpE8MGAwcCBwMCwhLDB8ISwgISwgLD BtUTyhPFE8MTE+gTwgIHA8ICEw4DDgLDE8USwwLCEMIG1BPKE8UTwxMT5xMCAwcDAg4TDgITwgIS D8ISD8ISBRICEcICwwbUE8oTxRPCExPmEwYCBwMCDgIOwgLDExITEhPCEg8GxgLDBtMMDAfJE8QT whMT5hMGwwITBgMCDhLFEw8SE8ISBgIDwhIDEsMGB9MDxwwHxRPDExPlEwYHAhESAg8CwhMPwhMP xBMPxRIQwgIDAgMCBtMDxwPEDAfDE8IT4RMHwwzCBgLCEhMCDxLIE8MSD8MSwwIQAwIDBgfSDMkD wgPCDAfCExPbEwfGDMIDDAIHERITEhMSwxMPwxMPwxPDEgIDAgMCwwMCBgzREwfHDMYDDMITE9YT B8UMyAMGB8ICBhLDAsYTEhMSExIPwhIHAgcCAwUQAgYRBgfSE8UTB8QMwgMMwhMT0hMHxAzLA8IM BsISDxESExITAw4DxBMSExITwxICBwPCAsMDDMIGB9ITyRMHwwzCExPPEwfDDMkDxQwHwhMGBxIT AhECEwMOAg7DExITDxMPwxIDAgMCBwMCDAYRBgfSE8kTwhPCDMITE8wTB8MMxwPEDMIHxxMGxBLD Ag4DDgIGwg/IEgIDwgIDAgwCEMIGB9ITyRMHDAcMwhMTyhMHwgzGA8MMwgfMEwYHwhLCEAIOAg4C DhDDAhIPxhIFAgXDAgUCEQYH0hPHEwfCDAcPDMITE8gTB8IMxQPDDAfQEwbDEhDEAhAOEA4QwgLG EgcSBhIGBcMCBcIGB9ATB8UMEwfCDA8HDwwHwhMTxhMHwgzEA8MMB9MTBgfCEhADEMICDhAOEMIC EQIDxxIGBwbCAgUCEQYHyxMHxAwHwhMHEwzCEwcPBw8MB8MTE8UTBwzEA8IMB9YTBsQSEAMCA8UC EQIDAgPDEgcSBgfCBgUQAhDCBgfGEwfEDAfGE8INEwzCEw8HwgwHwxPCE8QTBwzDA8IMB9gTBgfE EhACEMYCEQIDAsQSBhLDBsICEALCBgfCEwfDDAfKEwfCDRMHwhPCDAfEE8ITE8MTBwzCA8IMB9oT DBIHwxLDDBEDxQIDAgPDEgYSBgfCBgIQAhAGDAfCEwzDE8MHyRMHwhPCBxMHxRPDExPDEwzCAwwH 3RMGxxICEQPDAgMCA8MSBhIGBwYMBhACEAIGDMMTDBPCB8YTwwfHEwfGE8MTwhPDEwwDDAfeEwYH xxICEQPDAgMCwhIGEgYHBgwGEAIQAsIGB8MTDMYTwwfKEwzGE8MTwhPDE8IMB98TDBLCB8USAgMR xAISB8ISBgcGDAYQBhAGEAYMB8MMB8kTwwfHEwzGE8MTwhPDEwwPwgzfEwYSB8ISB8ISAhECAwID EgcSBwYHBgwGEAYQxgzDD8IHxRPDB8kTBwzGE8MTwhPDEwzDD8QM3BPCBhIGwxIGAhECAwIHBgcG yAzJDxMHzRMHwwwHxxPDE8ITwxMHDMYPxwwH1BMGEgYSBhLLDM4PwwwTDMcTwgfEDAfJE8QTwhMT xBMHwgzLD9sM0w/GDAfDEwzDEwfEDAfLE8YTwxMTxhMHxAztD8gMBgfIE8QMB84TxxPDE8ITyhMH xwzbD8sMEAUMBcIMwgYH1RPKE8UTwxMT0RMH2wwGEAYQBhACBQwFDAUMBgwHBgfWE8sTxRPDExPu EwYMBhAGEAIGDAYMwwYH1xPLE8YTwxMT8BPKBgfYE8wTxhPDExP1E9sTzRPHE8MTwhP1E9sTzRPH E8MTwhMMAAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD/ /wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8A AAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A //8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAA AP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA /wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCk gICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vw oKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw //vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzA psrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDA wNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICA wMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACA AICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACA gACA//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP////// --0__=xWJBzENSroYfOJ770eZruaQN4d4spAN8cvpDi8EljOuY6BYdSyFI3Vxm-- From mpi-core-human@mcs.anl.gov Mon Jan 13 14:40:47 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA14937; Mon, 13 Jan 1997 14:40:45 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA14673 for mpi-core-out; Mon, 13 Jan 1997 13:42:45 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA14665 for ; Mon, 13 Jan 1997 13:42:42 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id LAA11401; Mon, 13 Jan 1997 11:28:08 -0800 (PST) Date: Mon, 13 Jan 1997 11:28:08 -0800 (PST) From: dinucci@nas.nasa.gov (Dr. David C. DiNucci) Message-Id: <199701131928.LAA11401@win182.nas.nasa.gov> To: "Marc Snir" Subject: Re: unnecessary complexities (was: Public Defined) Cc: Mpi-Core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > You basically argue that MPI1 was not designed correctly, and that MPI2 > compounds the crime. MPI1 had heated discussions on some of the issues > you mention -- in particular communicators vs tags. The decision went > one way -- the main arguments were that this way leaves more freedom to > implementors -- in particular it allows for different divisions between > hardware/firmware supported functions and software supported functions, > and that this design is safer. The decisions may have been wrong, and > CDS1 may be a superior design. But this is clearly not a fruitful debate > for the MPI forum. Clearly, the MPI forum has to design MPI2 using the > same design principles as MPI1, while other groups or individuals may > push designs built on totally different principles. Since the wording appears to address me, I will briefly respond, but I see the possibility that some parts are actually addressed to the forum. I'm not sure that I would say that MPI1 was not designed correctly. But when pressed by Richard to point out ways in which it could be made simpler, I responded with information that is available to me today, rather than that which was available when MPI1 was in its initial stages of design. In other words, I was trying to answer the question "How can it be made simpler" rather than "How could it have been made simpler". I didn't feel it proper to restrict my answer based on the practicality of having the forum entertain changes at this point, since I have neither the authority nor the information to decide this practicality. I hope it is clear that my comments and suggestions are being made with the benefit of 20-20 hindsight, and that the efforts of the MPI forum have made this possible, to a large extent. I hope it is also clear that CDS has not reached a stage of development yet to assure that it is superior to MPI at all, though I personally believe that it has some advantages. I was primarily using it as a data point for comparison and illustration. Being no longer actively involved in the forum, I cannot be involved in deciding discussion topics for the forum. I did agree earlier with the possibility that this may not be fruitful debate for the MPI forum at this point, when this was suggested by Al Geist, but I have also seen some other views, so I remain neutral. Thank you for the response, -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Mon Jan 13 21:30:54 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA20504; Mon, 13 Jan 1997 21:30:53 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA25569 for mpi-core-out; Mon, 13 Jan 1997 20:34:43 -0600 Received: from tbag.osc.edu (tbag.osc.edu [128.146.36.50]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA25564 for ; Mon, 13 Jan 1997 20:34:40 -0600 Received: for gdburns@tbag.osc.edu by tbag.osc.edu (8.7.1/950822.1) id VAA25673; Mon, 13 Jan 1997 21:35:02 -0500 (EST) Date: Mon, 13 Jan 1997 21:35:02 -0500 (EST) From: Greg Burns Message-Id: <199701140235.VAA25673@tbag.osc.edu> To: mpi-core@mcs.anl.gov Subject: Re: Public Defined Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I would like to throw my support behind the rebel David DiNucci's original remarks. I am not surprised by the silence of the public comment period, though it is necessary that we have such a period for the sake of credibility. The reason for the lack of feedback is that not enough people care. Parallel computing is a niche technology dominated by transient users in the academic sector. The vendors all attend the Forum. What is lacking is the business sector who intend to build products on or around MPI, but for one reason or another do not attend the Forum. If they existed in greater numbers, they would certainly care enough to dive into the MPI-2 document, no matter how large or complex. I am quite disappointed that every HPC cycle center does not have a person carefully reviewing MPI-2 and returning comments. Users at these centers are not being properly informed or prepared for key developments in HPC software, of which the MPI standard is certainly one. MPI has supplanted PVM for the traditional parallel applications. It still seems like there are more PVM users in the world and many are not using it for scientific computing. Interestingly, PVM's future may be brighter because of this. Non-blocking operations, beginning in MPI-1 and culminating in generalized requests, are turning MPI into an operating system wanna-be. Synchronization and blocking processes are major parts of what an OS does. We want to wait on an arbitrary subset of communication events and preferably release the processor in the meantime. In MPI-2, we also include I/O devices. Oh, and because we want underlying progress, don't forget to watch all other outstanding requests at the same time. These things are best and most efficiently coordinated at the lowest level, the operating system that is fielding all the interrupts from the devices. But no OS lets the user tap into all of this in a completely flexible way [Win32 WaitForMultipleObjects() is more powerful than UNIX select(); let's require NT :-)]. So we decided to force all of this into MPI, at the cost of considerable complexity for both the user and the implementor. Fortunately, the total lack of correct implementations has resulted in the acceptance of the weak interpretation of the progress rule. Otherwise, you would need to have or to build an OS to implement MPI - forget about lean, mean embedded implementations running with the interrupts turned off. But this has caused all sorts of agony in designing one-sided to be suitable to distributed memory machines. Many of the implementations have single-threaded progress engines. Every time MPI-2 adds another non-blocking operation, we have to extend the progress engine with an additional state machine for that operation. This only increases the opportunity for users to create multi-flavoured request arrays that cause the implementation to degrade to polling the operating system from a user process. Very few users realise that this is going to suck in terms of chewing up the CPU and increasing mean latency to respond to a synchronization on one of the requests. Of course it all looks yummy at the MPI layer. My wish, in hindsight, is that MPI_I*() only provided information to the implementation for possible overlapping or idle-consuming progress. There would be zero guarantee of progress until a Wait or Test was called. This would at least allow implementations to concentrate on the request array used in the Wait/Test call, and not the entire outstanding request list. Even in MPI-1, we surpassed the goal of standardizing common practice. We must keep in mind that MPI's number one selling point is that it is a standard. Any particular chapter or function is a distant number two. The _standard_ is what can catalize the niche parallel computing industry, which is the primary goal in my opinion. The very existence of MPI-2 shows that we want more. We want to design an all-encompassing parallel interface. It's message-passing. It's shared memory (subtle distinctions aside). It's I/O. Did we forget anything? No worry, it can be layered on GR. Now we may get this all bang-on right, be adopted as POSIX.n+1 and turn parallel computing into the next WWW. Or we might muck it up, scare off the users with the complexity, piss off the implementors with work, save PVM, and get bumped off by some new DCE-2 standard with more hype. I don't know which is right. My gut tells me to keep it simple, let the market have a go at it, see where they really get stuck and then step in. -=- tidbit MPI-2 feedback: I, like many, felt that dynamic processes were a major draw-back, marketing-wise, for MPI-1. As an (the only?) implementor of MPI-2 dynamic processes, I can report that we have had near-zero feedback on these functions, which leads me to believe that they are not being used very much. Maybe Marc's unconventional wisdom on this chapter will prove correct... -- Greg From mpi-core-human@mcs.anl.gov Mon Jan 13 21:44:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA20534; Mon, 13 Jan 1997 21:44:10 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA25751 for mpi-core-out; Mon, 13 Jan 1997 20:48:25 -0600 Received: from win182.nas.nasa.gov (win182.nas.nasa.gov [129.99.66.88]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA25746 for ; Mon, 13 Jan 1997 20:48:21 -0600 Received: (from dinucci@localhost) by win182.nas.nasa.gov (8.8.3/NAS.6.1) id SAA01618 for mpi-core@mcs.anl.gov; Mon, 13 Jan 1997 18:48:47 -0800 (PST) Date: Mon, 13 Jan 1997 18:48:47 -0800 (PST) From: dinucci@nas.nasa.gov (David C. DiNucci) Message-Id: <199701140248.SAA01618@win182.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: Re: Public Defined Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Greg Burns writes: > I would like to throw my support behind the rebel David DiNucci's > original remarks. Gee, this is the first time I can remember being called a rebel. And by a man who himself is on the LAM :-) And on my 40th birthday, no less. Just wait 'til I'm 60! > Non-blocking operations, beginning in MPI-1 and culminating in > generalized requests, are turning MPI into an operating system > wanna-be. Synchronization and blocking processes are major parts > of what an OS does. We want to wait on an arbitrary subset > of communication events and preferably release the processor in > the meantime. In MPI-2, we also include I/O devices. Oh, and > because we want underlying progress, don't forget to watch all > other outstanding requests at the same time. These things are > best and most efficiently coordinated at the lowest level, the > operating system that is fielding all the interrupts from the > devices. But no OS lets the user tap into all of this in > a completely flexible way [Win32 WaitForMultipleObjects() is more > powerful than UNIX select(); let's require NT :-)]. So we decided > to force all of this into MPI, at the cost of considerable complexity > for both the user and the implementor. I think this is an extremely important point, and one of the reasons that I have begun proposing that CDS1 (or something like it) belongs in the OS. (See my paper, and probably others, at the upcoming CANPC workshop in San Antonio.) MPI, with its current size and complexity, is not a viable candidate for kernel-level implementation, but if it were built in a layered fashion, its lower levels might be. Still, the current MPI demonstrates the kinds of functionality which it must be possible to build upon the OS layer, whatever it is, and any standard API proposed today cannot count on functionality that might not be in the OS until tomorrow. (And MPI is certainly not alone in its creeping featurism leading to OS-like powers. Have you seen Netscape lately?) > Fortunately, the total lack of correct implementations has resulted > in the acceptance of the weak interpretation of the progress rule. > Otherwise, you would need to have or to build an OS to implement MPI > - forget about lean, mean embedded implementations running with > the interrupts turned off. But this has caused all sorts of agony > in designing one-sided to be suitable to distributed memory machines. > My wish, in hindsight, is that MPI_I*() only provided information > to the implementation for possible overlapping or idle-consuming progress. > There would be zero guarantee of progress until a Wait or Test was called. > This would at least allow implementations to concentrate on the request > array used in the Wait/Test call, and not the entire outstanding request list. Yes, these are exactly some of the drawbacks I was alluding to when I said that one-sided had suffered from the policy of basing MPI-2 on an unchanged MPI-1. > tidbit MPI-2 feedback: I, like many, felt that dynamic processes were > a major draw-back, marketing-wise, for MPI-1. As an (the only?) implementor > of MPI-2 dynamic processes, I can report that we have had near-zero > feedback on these functions, which leads me to believe that they are > not being used very much. Maybe Marc's unconventional wisdom on this > chapter will prove correct... At some level, all processes on all machines could be considered to be dynamic processes -- i.e. they weren't there, and then they were -- so I argue that the basic functionality is indeed useful. So the question is: How many restrictions can you place upon that basic functionality, with the goal of making process creation and/or inter-process communication more efficient, before users give up and find another way? I think that MPI should give us simple dynamic processes, with few or no strings attached, or not bother. (I'm not sure I've heard Marc's comments to which you refer.) -Dave ======================================================================== David C. DiNucci | MRJ, Inc., Rsrch Scntst | NASA Ames Rsrch Ctr dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2 (415)604-4430 | Parallel Tools Team | Moffett Field, CA 94035 From mpi-core-human@mcs.anl.gov Tue Jan 14 17:05:39 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA02565; Tue, 14 Jan 1997 17:05:38 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA14171 for mpi-core-out; Tue, 14 Jan 1997 16:08:44 -0600 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA14166 for ; Tue, 14 Jan 1997 16:08:41 -0600 From: Andrew.Lumsdaine.1@nd.edu Received: (from lums@localhost) by lsc.nd.edu (8.8.3/8.8.2) id RAA11031; Tue, 14 Jan 1997 17:09:10 -0500 (EST) Date: Tue, 14 Jan 1997 17:09:10 -0500 (EST) Message-Id: <199701142209.RAA11031@lsc.nd.edu> To: mpi-core@mcs.anl.gov Subject: Pedagogy and MPI example code Reply-to: Andrew.Lumsdaine.1@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In reading through some of the examples in the current MPI document, I notice that most (if not all) of the examples involving main() use the common but incorrect practice of returning from main() without a value. Main() is declared to be int (or defaults to int with no declaration). Results are undefined if no value is explicitly returned by main() -- typically some sort of garbage value is obtained. Thus, I would urge all example authors to return a value of 0 from main() after MPI_Finalize() -- or return MPI_Finalize(). The reasons for this are more than pedagogical, I think, in the case of MPI. The return value of main() is returned to the operating system, and may be checked by invoking processes. It seems that batch systems in particular might infer failure of a job that did not fail and take inappropriate action as a result. Maybe it's just because the term is about to start and I am feeling the pressure of curious young minds, but it seems that examples in the document should be examples not only of the particular point being made at the time, but also of good MPI programming practice. Best Regards, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Tue Jan 14 17:34:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA02927; Tue, 14 Jan 1997 17:34:48 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA14959 for mpi-core-out; Tue, 14 Jan 1997 16:38:05 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA14949 for ; Tue, 14 Jan 1997 16:37:59 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA23677 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Tue, 14 Jan 1997 14:38:27 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id OAA18771; Tue, 14 Jan 1997 14:36:16 -0800 From: "Eric Salo" Message-Id: <9701141436.ZM18769@mrjones.engr.sgi.com> Date: Tue, 14 Jan 1997 14:36:15 -0800 In-Reply-To: Andrew.Lumsdaine.1@nd.edu "Pedagogy and MPI example code" (Jan 14, 5:09pm) References: <199701142209.RAA11031@lsc.nd.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: Pedagogy and MPI example code Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Sounds to me like Andrew has just volunteered to fix all of the examples in the standard! Please try to get them all done by next week so that we can review them in Chicago. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Wed Jan 15 11:18:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA28250; Wed, 15 Jan 1997 11:18:10 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA27778 for mpi-core-out; Wed, 15 Jan 1997 10:15:50 -0600 Received: from aloisius.vcpc.univie.ac.at (aloisius.vcpc.univie.ac.at [193.171.58.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA27767 for ; Wed, 15 Jan 1997 10:15:28 -0600 Received: (from smap@localhost) by aloisius.vcpc.univie.ac.at (8.7.5/8.7.3) id RAA24720 for ; Wed, 15 Jan 1997 17:15:56 +0100 (MET) From: Ian Glendinning X-Authentication-Warning: aloisius.vcpc.univie.ac.at: smap set sender to using -f Received: from beavis(193.171.58.38) by aloisius via smap (V1.3) id sma024708; Wed Jan 15 17:15:51 1997 Received: (from ian@localhost) by beavis.vcpc.univie.ac.at (8.7.1/8.7.1) id RAA03013 for mpi-core@mcs.anl.gov; Wed, 15 Jan 1997 17:15:50 +0100 (MET) Date: Wed, 15 Jan 1997 17:15:50 +0100 (MET) Message-Id: <199701151615.RAA03013@beavis.vcpc.univie.ac.at> To: mpi-core@mcs.anl.gov Subject: Re: Public Defined X-Sun-Charset: US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Eric Salo wrote: > My experience at the MPI Forum meetings has been exactly the reverse of the > above. In general, it is the vendors and implementors who have been arguing in > favor of simplicity, and the users who have been saying "but I really need this > fancy whiz-bang function". I see - the balance has obviously shifted since MPI-1 days. It sounds like this is a question of how high or low-level you are talking. I was talking about a relatively low level. Perhaps I should have made it clear I was speaking about the philosopy behind MPI-1. However, as Marc Snir pointed out in his very clear perspective of the situation: > and that this design is safer. The decisions may have been wrong, and > CDS1 may be a superior design. But this is clearly not a fruitful debate > for the MPI forum. Clearly, the MPI forum has to design MPI2 using the > same design principles as MPI1, while other groups or individuals may > push designs built on totally different principles. Since I am no longer actively involved in the forum, I will now resume my former silence, and continue to track developments as best I can from what arrives in the mail :-) Ian -- Ian Glendinning European Centre for Parallel Computing at Vienna (VCPC) ian@vcpc.univie.ac.at Liechtensteinstr. 22, A-1090 Vienna, Austria Tel: +43 1 310 939612 WWW: http://www.vcpc.univie.ac.at/~ian/ From mpi-core-human@mcs.anl.gov Wed Jan 15 20:55:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA05454; Wed, 15 Jan 1997 20:55:42 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA10082 for mpi-core-out; Wed, 15 Jan 1997 19:58:21 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA10077 for ; Wed, 15 Jan 1997 19:58:18 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id TAA00071; Wed, 15 Jan 1997 19:58:49 -0600 (CST) Date: Wed, 15 Jan 1997 19:58:49 -0600 (CST) Message-Id: <199701160158.TAA00071@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document for meeting ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. This is the version that is going to be handed out at the meeting next week. Remember that lots of votes are likely to be taken so start having fun early and read it over the weekend :-). See you all soon in the cold, snowy Midwest! Steve From mpi-core-human@mcs.anl.gov Mon Jan 20 11:20:09 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA27918; Mon, 20 Jan 1997 11:20:08 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA10870 for mpi-core-out; Mon, 20 Jan 1997 10:20:12 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA10865 for ; Mon, 20 Jan 1997 10:20:09 -0600 Message-Id: <199701201620.KAA10865@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: tentative agenda for meeting Date: Mon, 20 Jan 1997 10:21:11 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, Here is a tentative agenda for the MPI Forum meeting this week. Our goals are to finish the chapters that are very close to finishing, see whether we can get major parts of I/O solidified, and make a few decisions that are still hanging (External, C++). Let me know if you see major problems with the following plan. Rusty Tuesday --------- 1:30-2:00 Discussion Issues Function names Interoperability Schedule 2:00-3:00 Room A: External Room B: Misc Room C: F90 Bindings 3:00-3:30 Break 3:30-5:30 Room A: I/O Room B: Real-time Room C: 6:00-8:00 Dinner 8:00-9:00 Room A: Voting on Dynamic, F-90 Binding, Misc 9:00-10:00 Room A: Voting on F90 Binding Wednesday -------- 8:30-10:00 Room A: Reading of I/O 10:00-10:30 Break 10:30-12:00 Room A: Reading of I/O 12:00-1:00 Lunch 1:00-3:00 Room A: Collective Room B: External Room C: C++ Bindings 3:00-3:30 Break 3:30-5:30 Room A: 1-sided Room B: I/O Room C: Real-time 6:00-8:00 Dinner 8:00-10:00 Room A: Voting on C++ Bindings, 1-sided, External Thursday -------- 8:30-10:00 Room A: Voting on I/O 10:00-10:15 Break 10:15-12:00 Room A: Voting on Collective 12:00 Adjourn From mpi-core-human@mcs.anl.gov Mon Jan 20 12:18:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA29211; Mon, 20 Jan 1997 12:18:47 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA12494 for mpi-core-out; Mon, 20 Jan 1997 11:21:36 -0600 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA12484 for ; Mon, 20 Jan 1997 11:21:31 -0600 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id SAA00007 (8.6.13/IDA-1.6); Mon, 20 Jan 1997 18:22:31 +0100 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id RAA25098; Mon, 20 Jan 1997 17:22:31 GMT From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199701201722.RAA25098@awsrr.rus.uni-stuttgart.de> Subject: Re: tentative agenda for meeting To: lusk@mcs.anl.gov (Rusty Lusk) Date: Mon, 20 Jan 1997 18:22:31 +0100 (MEZ) Cc: mpi-core@antares.mcs.anl.gov In-Reply-To: <199701201620.KAA10865@antares.mcs.anl.gov> from "Rusty Lusk" at Jan 20, 97 10:21:11 am Content-Type: text Content-Length: 860 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hello, > Tuesday > > 1:30-2:00 Discussion Issues > Function names > Interoperability > Schedule Because my plane is landing 1:10, I make my proposal per mail: Please put the Interoperability-Discussion into the empty slot 3:30-5:30 Room C. > 2:00-3:00 Room A: External > Room B: Misc > Room C: F90 Bindings > > 3:30-5:30 Room A: I/O > Room B: Real-time > Room C: ... Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Mon Jan 20 13:59:51 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA01179; Mon, 20 Jan 1997 13:57:15 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA14892 for mpi-core-out; Mon, 20 Jan 1997 12:59:49 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA14884 for ; Mon, 20 Jan 1997 12:59:43 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id LAA27020 for <@sgi.engr.sgi.com:mpi-core@antares.mcs.anl.gov>; Mon, 20 Jan 1997 11:00:44 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@antares.mcs.anl.gov id KAA00191; Mon, 20 Jan 1997 10:58:33 -0800 From: "Eric Salo" Message-Id: <9701201058.ZM188@mrjones.engr.sgi.com> Date: Mon, 20 Jan 1997 10:58:33 -0800 In-Reply-To: Rusty Lusk "tentative agenda for meeting" (Jan 20, 10:21am) References: <199701201620.KAA10865@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@antares.mcs.anl.gov Subject: Re: tentative agenda for meeting Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Here is a tentative agenda for the MPI Forum meeting this week. Our goals > are to finish the chapters that are very close to finishing, see whether we > can get major parts of I/O solidified, and make a few decisions that are still > hanging (External, C++). Let me know if you see major problems with the > following plan. As there is currently no time scheduled to discuss the issue of non-blocking collective routines across the entire document, I will certainly propose that we make some time to discuss that. To prevent redundant discussions in the subcommittees, it should be done early in the meeting. -- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 8U-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?" From mpi-core-human@mcs.anl.gov Fri Jan 24 09:10:33 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA27848; Fri, 24 Jan 1997 09:10:32 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA08908 for mpi-core-out; Fri, 24 Jan 1997 08:08:44 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA08903 for ; Fri, 24 Jan 1997 08:08:41 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id IAA08032; Fri, 24 Jan 1997 08:09:04 -0600 (CST) Date: Fri, 24 Jan 1997 08:09:04 -0600 (CST) Message-Id: <199701241409.IAA08032@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: schedule for March meeting Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Howdy MPIers! Below are the deadlines for the March meeting. As chapters and the document begin to take their final shape, I encourage people to begin carefully reading the document for mistakes. There is likely to be a time crunch at the end so doing it early would be greatly appreciated. Friday, 7 February: A first cut at the next version of each chapter is due to me by 12:00 PM Central US time (noon). When the full document is ready it will be announced on mpi-core and should be out by late afternoon. This is a little early due to the MPI-2 European meeting which will make a number of us unavailable during the following week. Wednesday, 26 February: The final version of each chapter is due to me by 10:00 AM Central US time. This is the version that will be printed for the meeting. It will also be announced on mpi-core and should be out by late afternoon. Wednesday, 5 March: MPI Forum meeting begins. Steve From mpi-core-human@mcs.anl.gov Mon Jan 27 18:26:44 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA06988; Mon, 27 Jan 1997 18:26:42 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA04791 for mpi-core-out; Mon, 27 Jan 1997 17:28:50 -0600 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA04786 for ; Mon, 27 Jan 1997 17:28:46 -0600 Received: from Bubbaport5.ERC.MsState.Edu (bubbaport5 [192.208.145.175]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id RAA11855; Mon, 27 Jan 1997 17:06:52 -0600 (CST) Received: by Bubbaport5.ERC.MsState.Edu with Microsoft Mail id <01BC0C74.A24769A0@Bubbaport5.ERC.MsState.Edu>; Mon, 27 Jan 1997 17:07:53 -0600 Message-ID: <01BC0C74.A24769A0@Bubbaport5.ERC.MsState.Edu> From: Shane Hebert To: "'mpi-core@mcs.anl.gov'" Subject: quick question about 1-sided Date: Mon, 27 Jan 1997 17:07:51 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Suppose I have a library that uses MPI-2 that I intend to call inside my own MPI-2 program. What happens if I use one of the synchronization methods to complete put/get-s in my code and the library uses another and we try to do them in an interleaved fashion? Shane From mpi-core-human@mcs.anl.gov Mon Jan 27 19:13:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA07494; Mon, 27 Jan 1997 19:13:08 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA05535 for mpi-core-out; Mon, 27 Jan 1997 18:15:25 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA05530 for ; Mon, 27 Jan 1997 18:15:20 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id QAA14572 for ; Mon, 27 Jan 1997 16:16:25 -0800 (PST) Message-ID: Date: 27 Jan 1997 16:13:33 -0800 From: "Lewins, Lloyd J" Subject: RE: quick question about 1-sided To: "'mpi-core@mcs.anl.gov'" , "Shane Hebert" X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The same problem occurs in MPI-1 if you have only one communication domain (MPI_COMM_WORLD), and is exactly why communicators exist for libraries. In MPI-2, a library should use its own window, distinct from the windows used by the application. This window will provide a distinct one-sided communication domain for the library. A problem occurs since we have strange semantics on overlapping windows, and the library/application may not be aware of this overlap. Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com _______________________________________________________________________________ From: Shane Hebert on Mon, Jan 27, 1997 3:38 PM Subject: quick question about 1-sided To: 'mpi-core@mcs.anl.gov' Suppose I have a library that uses MPI-2 that I intend to call inside my own MPI-2 program. What happens if I use one of the synchronization methods to complete put/get-s in my code and the library uses another and we try to do them in an interleaved fashion? Shane ------------------ RFC822 Header Follows ------------------ Received: by MSMAIL4.HAC.COM with SMTP;27 Jan 1997 15:37:52 -0800 Received: from antares.mcs.anl.gov (140.221.9.6) by EDEN.HAC.COM (PMDF V4.3-7 #5884) id <01IEPO3U8J0W00LD26@EDEN.HAC.COM>; Mon, 27 Jan 1997 15:37:15 PST Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA04791 for mpi-core-out; Mon, 27 Jan 1997 17:28:50 -0600 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA04786 for ; Mon, 27 Jan 1997 17:28:46 -0600 Received: from Bubbaport5.ERC.MsState.Edu (bubbaport5 [192.208.145.175]) ; by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id RAA11855; Mon, 27 Jan 1997 17:06:52 -0600 (CST) Received: by Bubbaport5.ERC.MsState.Edu with Microsoft Mail id <01BC0C74.A24769A0@Bubbaport5.ERC.MsState.Edu>; Mon, 27 Jan 1997 17:07:53 -0600 Date: Mon, 27 Jan 1997 17:07:51 -0600 From: Shane Hebert Subject: quick question about 1-sided Sender: owner-mpi-core@mcs.anl.gov To: "'mpi-core@mcs.anl.gov'" Message-id: <01BC0C74.A24769A0@Bubbaport5.ERC.MsState.Edu> MIME-version: 1.0 Content-type: text/plain; charset="us-ascii" Content-transfer-encoding: 7bit Precedence: bulk From mpi-core-human@mcs.anl.gov Tue Jan 28 16:30:29 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA03850; Tue, 28 Jan 1997 16:30:27 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA23470 for mpi-core-out; Tue, 28 Jan 1997 15:32:40 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA23455 for ; Tue, 28 Jan 1997 15:32:34 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id QAA09586; Tue, 28 Jan 1997 16:34:43 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id QAA21838; Tue, 28 Jan 1997 16:33:55 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05b4 (287.3 12-16-1996)) id 8525642D.00766F61 ; Tue, 28 Jan 1997 16:33:37 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: shane@erc.msstate.edu cc: Mpi-Core@mcs.anl.gov Message-ID: <8525642D.00763613.00@watngi01.watson.ibm.com> Date: Tue, 28 Jan 1997 16:33:34 -0400 Subject: Re: quick question about 1-sided Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Suppose I have a library that uses MPI-2 that I intend to call inside my own MPI-2 program. What happens if I use one of the synchronization methods to complete put/get-s in my code and the library uses another and we try to do them in an interleaved fashion? Shane --------------------------------- If the library uses a different window handle than the main program, then no harm is done, since all rules are per communication domain. If the library uses the same window handle as the main program, then cahos ensues. This is similar to having pending nonblocking communications on a communicator while calling a library that communicates using the same communicator: calls outside the library may inadvertendly match calls inside the library. From mpi-core-human@mcs.anl.gov Thu Jan 30 11:03:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA06616; Thu, 30 Jan 1997 11:03:01 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA27883 for mpi-core-out; Thu, 30 Jan 1997 10:06:36 -0600 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA27870 for ; Thu, 30 Jan 1997 10:06:31 -0600 Received: from hydra.cs.msstate.edu (hydra.cs.msstate.edu [192.208.157.132]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id KAA18896; Thu, 30 Jan 1997 10:06:28 -0600 (CST) Received: by hydra.cs.msstate.edu with Microsoft Mail id <01BC0E95.65DEFDC0@hydra.cs.msstate.edu>; Thu, 30 Jan 1997 10:07:28 -0600 Message-ID: <01BC0E95.65DEFDC0@hydra.cs.msstate.edu> From: Shane Hebert To: "'mpi-core'" Subject: Trollius Date: Thu, 30 Jan 1997 10:07:26 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Are any of you out there familiar with Trollius libraries? I have been exchanging email with a user who likes the way Trollius transfers data around and was wondering whether or not MPI supported that type data manipulation and that he liked it a lot and wondered if MPI-2 could support that type of functionality. It doesn't look like it is the short answer. I told him that I would at least mention it to mpi-core for discussion. If anyone is interested in this topic, I can send you the emails that we exchanged so you can follow the discussion. Shane From mpi-core-human@mcs.anl.gov Thu Jan 30 12:13:16 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA07220; Thu, 30 Jan 1997 12:13:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA29629 for mpi-core-out; Thu, 30 Jan 1997 11:16:39 -0600 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA29623 for ; Thu, 30 Jan 1997 11:16:35 -0600 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (8.7.6/7.0m-FWP-MsState); id LAA09139; Thu, 30 Jan 1997 11:17:09 -0600 (CST) Date: Thu, 30 Jan 1997 11:17:08 -0600 (CST) From: Tony Skjellum To: Shane Hebert cc: "'mpi-core'" Subject: Re: Trollius In-Reply-To: <01BC0E95.65DEFDC0@hydra.cs.msstate.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk That was what Greg Burns did before LAM> -Tony On Thu, 30 Jan 1997, Shane Hebert wrote: > Date: Thu, 30 Jan 1997 10:07:26 -0600 > From: Shane Hebert > To: 'mpi-core' > Subject: Trollius > > Are any of you out there familiar with Trollius libraries? > > I have been exchanging email with a user who likes the way Trollius transfers data > around and was wondering whether or not MPI supported that type data manipulation > and that he liked it a lot and wondered if MPI-2 could support that type of functionality. > It doesn't look like it is the short answer. I told him that I would at least mention it > to mpi-core for discussion. If anyone is interested in this topic, I can send you > the emails that we exchanged so you can follow the discussion. > > Shane > Anthony Skjellum, PhD, Associate Professor of Computer Science; Mississippi State University, Department of Computer Science & NSF ERC Butler, Rm 300, PO Box 9637, Corner of Perry&Barr, Mississippi State,MS 39762 (601)325-8435 FAX: (601)325-8997; http://www.erc.msstate.edu/~tony; "Persistence is fertile." ; e-mail: tony@cs.msstate.edu; Try MPI! From mpi-core-human@mcs.anl.gov Thu Jan 30 13:33:13 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA08561; Thu, 30 Jan 1997 13:33:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA01896 for mpi-core-out; Thu, 30 Jan 1997 12:35:31 -0600 Received: from palrel1.hp.com (palrel1.hp.com [15.253.72.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA01884 for ; Thu, 30 Jan 1997 12:35:19 -0600 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com with ESMTP (8.7.5/8.7.3) id KAA11764 for ; Thu, 30 Jan 1997 10:35:17 -0800 (PST) Message-Id: <199701301835.KAA11764@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA027499314; Thu, 30 Jan 1997 12:35:14 -0600 From: Raja Daoud Subject: Re: Trollius To: mpi-core@mcs.anl.gov Date: Thu, 30 Jan 1997 12:35:14 CST X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > That was what Greg Burns did before LAM> Actually it was more of an evolutionary path: LAM is a port of Trollius to UNIX clusters, plus some extensions, plus MPI (and for one year a layered PVM API before it was dropped). Trollius' CUBIX and Brenda (a Linda-like tuple space API) were unbundled and CUBIX later ported to MPI. >> I have been exchanging email with a user who likes the way Trollius >> transfers data around and was wondering whether or not MPI supported >> that type data manipulation and that he liked it a lot and wondered >> if MPI-2 could support that type of functionality. Shane, I'm curious which Trollius send/recv features did that user miss having in MPI? Maybe we can easily accommodate him. We already went out of our way adding non-prior-practice functionality. Here's a case of prior-practice need by a non-imaginary user, I think it behooves us to make an effort. --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com Convex Division From mpi-core-human@mcs.anl.gov Thu Jan 30 19:32:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA13237; Thu, 30 Jan 1997 19:30:20 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA09784 for mpi-core-out; Thu, 30 Jan 1997 18:33:42 -0600 Received: from nimble.llnl.gov (nimble.llnl.gov [134.9.13.62]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA09758; Thu, 30 Jan 1997 18:33:04 -0600 Received: by nimble.llnl.gov (4.1/LLNL-1.20) id AA03228; Thu, 30 Jan 97 16:33:02 PST Date: Thu, 30 Jan 97 16:33:02 PST From: trj@nimble.llnl.gov (Terry R. Jones) Message-Id: <9701310033.AA03228@nimble.llnl.gov> To: mpi-core@mcs.anl.gov, mpi-external@mcs.anl.gov, mpi-io@mcs.anl.gov Subject: non-blocking collective I/O Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am interested in discussing if we should have non-blocking collective (asynchronous) functions as part of I/O. I am not interested in pursuing non-blocking collective calls for communication; I wish only to discuss these functions with regard to I/O. I realize that time is short and that we'll need a proposal by Feb 7 if we are able to agree that such calls are both technically possible and useful. To expedite the discussion, I propose that we first focus our discussion to whether or not such I/O calls are technically feasible (i.e. we are unable to find a reason that would preclude a reasonably efficient implementation). I believe that there is utility in such I/O calls -- we can cover the necessary discussion on whether or not these calls are beneficial once we have first made a positive finding on if they are possible. So, in trying to determine if there are technical reasons to prohibit non-blocking collective (asynchronous) I/O functions, let me begin this discussion by listing the difficulties associated with such calls. This list is probably incomplete, please add what I am missing. After the list you will find my initial response to the stated problem. 1. The List: (Difficulties associated with non-blocking collective calls) 1.1) May require large amounts of state to be passed around. 1.2) Fortran 90 array slices may be implemented on stack and therefore may cause inappropriate behavior. For example, in the following code snippet, a compiler could make a new copy of 50 elements from array x (every other element) onto the stack. However, this stack may only be valid inside the scope of foo. (While this kind of behavior is similar to C programmers misusing automatic variables, the difference is that a C program would be erroneous but the Fortran program is valid.) program example1 real x(100) call foo(x(::2)) end subroutine foo(yy) real yy(*) return end 1.3) May place additional constraints on the progress_fn. For example, a multi-phase function like barrier (which requires some kind of reduce comprised of multiple messages from a given node) would need to progress until a "sane" state. 1.4) Would allow a non-blocking read into a variable ``i'' which the compiler could actually have in a register. The compiler could later flush its (old) value from the register to ``i'' and overwrite the result of the non-blocking read. 1.5) Increases potential for deadlock from interleaved collective operations. 1.6) Matching collective operations becomes a concern. 1.7) What are the rules for the progress engine? When do reads and writes happen? 1.8) (I must be very vague here because I do not understand the issue. Would some kind soul please elaborate.) I believe I heard allusion to some kind of issue with the completion operations MPI_{TEST|WAIT}{ANY|SOME|ALL} at the recent MPI-forum meeting earlier this month. If I understand the concern correctly, only MPI_Wait can reasonably be used with non-blocking collective operations. Someone will have to fill in the details here. 2. Discussion of the list Since difficulties 1.1 through 1.4 apply to non-blocking point-to-point communications in MPI 1.1, I'll assume that they have been determined to not be prohibitive there. Therefore, for non-blocking collective I/O (possibly implemented as a layered library), I think we can say that problems 1.1 through 1.4 reduce to "can we get Generalized Requests right" which is an ongoing topic in the mpi-external mailing-list. Regarding 1.5 and 1.6, it seems like maintaining a sequence number with the request state would be sufficient (i.e. read 16 on node xx must match read 16 on node yy, and so forth). Assuming that the implementation is able to match the operation orderings, any remaining potential for problems is a program correctness issue. Regarding 1.7, is this only a problem for overlapping reads/writes? If so, I think any method which provides the following two requirements should work: a) an ordering of the data accesses; b) atomic reads/writes of the entire buffer. Requirement a) can be met through either using globally synchronized timestamps or using the order of messages received at a single location. BOTTOM LINE: I think we have reasonable ways to address 1.1 through 1.6. For other problems, someone else will have to provide some further description of the problem. -terry From mpi-core-human@mcs.anl.gov Sat Feb 1 15:06:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA03258; Sat, 1 Feb 1997 15:06:33 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA19199 for mpi-core-out; Sat, 1 Feb 1997 14:07:20 -0600 Received: from zeta.ece.nwu.edu (zeta.ece.nwu.edu [129.105.5.105]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA17580; Sat, 1 Feb 1997 11:13:07 -0600 Received: (from daemon@localhost) by zeta.ece.nwu.edu (8.8.5/8.8.3) id LAA27848; Sat, 1 Feb 1997 11:13:03 -0600 (CST) Received: from bellport2.ece.nwu.edu(129.105.5.222) by zeta.ece.nwu.edu via smap (V2.0beta) id xmah27817; Sat, 1 Feb 97 11:12:50 -0600 Message-Id: <2.2.32.19970201170650.006b2f6c@ece.nwu.edu> X-Sender: choudhar@ece.nwu.edu (Unverified) X-Mailer: Windows Eudora Pro Version 2.2 (32) Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Sat, 01 Feb 1997 11:06:50 -0600 To: trj@nimble.llnl.gov (Terry R. Jones) From: Alok Choudhary Subject: Re: non-blocking collective I/O Cc: mpi-core@mcs.anl.gov, mpi-external@mcs.anl.gov, mpi-io@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Just a warning from experience of PASSION. Non-blockingcollective I/O call are extremely difficult to implement and get working in general. However, I would bevery interested in seeing something defined in this area. Alok At 04:33 PM 1/30/97 PST, you wrote: > >I am interested in discussing if we should have non-blocking collective >(asynchronous) functions as part of I/O. I am not interested in pursuing >non-blocking collective calls for communication; I wish only to discuss >these functions with regard to I/O. > >I realize that time is short and that we'll need a proposal by Feb 7 if we >are able to agree that such calls are both technically possible and useful. >To expedite the discussion, I propose that we first focus our discussion to >whether or not such I/O calls are technically feasible (i.e. we are unable >to find a reason that would preclude a reasonably efficient implementation). >I believe that there is utility in such I/O calls -- we can cover >the necessary discussion on whether or not these calls are beneficial >once we have first made a positive finding on if they are possible. > >So, in trying to determine if there are technical reasons to prohibit >non-blocking collective (asynchronous) I/O functions, let me begin >this discussion by listing the difficulties associated with such calls. >This list is probably incomplete, please add what I am missing. >After the list you will find my initial response to the stated problem. > >1. The List: (Difficulties associated with non-blocking collective calls) > > 1.1) May require large amounts of state to be passed around. > > 1.2) Fortran 90 array slices may be implemented on stack and therefore > may cause inappropriate behavior. For example, in the following > code snippet, a compiler could make a new copy of 50 elements from > array x (every other element) onto the stack. However, this stack > may only be valid inside the scope of foo. (While this kind of > behavior is similar to C programmers misusing automatic variables, > the difference is that a C program would be erroneous but the Fortran > program is valid.) > > program example1 > real x(100) > call foo(x(::2)) > > end > > subroutine foo(yy) > real yy(*) > > return > end > > 1.3) May place additional constraints on the progress_fn. For example, a > multi-phase function like barrier (which requires some kind of reduce > comprised of multiple messages from a given node) would need to > progress until a "sane" state. > > 1.4) Would allow a non-blocking read into a variable ``i'' which the > compiler could actually have in a register. The compiler could > later flush its (old) value from the register to ``i'' and overwrite > the result of the non-blocking read. > > 1.5) Increases potential for deadlock from interleaved collective > operations. > > 1.6) Matching collective operations becomes a concern. > > 1.7) What are the rules for the progress engine? When do reads and > writes happen? > > 1.8) (I must be very vague here because I do not understand the issue. > Would some kind soul please elaborate.) > I believe I heard allusion to some kind of issue with the completion > operations MPI_{TEST|WAIT}{ANY|SOME|ALL} at the recent MPI-forum > meeting earlier this month. If I understand the concern correctly, > only MPI_Wait can reasonably be used with non-blocking collective > operations. Someone will have to fill in the details here. > >2. Discussion of the list > > Since difficulties 1.1 through 1.4 apply to non-blocking point-to-point > communications in MPI 1.1, I'll assume that they have been determined > to not be prohibitive there. Therefore, for non-blocking collective I/O > (possibly implemented as a layered library), I think we can say that > problems 1.1 through 1.4 reduce to "can we get Generalized Requests right" > which is an ongoing topic in the mpi-external mailing-list. > > Regarding 1.5 and 1.6, it seems like maintaining a sequence number > with the request state would be sufficient (i.e. read 16 on node xx > must match read 16 on node yy, and so forth). Assuming that the > implementation is able to match the operation orderings, any remaining > potential for problems is a program correctness issue. > > Regarding 1.7, is this only a problem for overlapping reads/writes? > If so, I think any method which provides the following two requirements > should work: a) an ordering of the data accesses; b) atomic reads/writes > of the entire buffer. Requirement a) can be met through either using > globally synchronized timestamps or using the order of messages received > at a single location. > > BOTTOM LINE: I think we have reasonable ways to address 1.1 through 1.6. > For other problems, someone else will have to provide some further > description of the problem. > >-terry > > From mpi-core-human@mcs.anl.gov Mon Feb 3 10:16:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA19926; Mon, 3 Feb 1997 10:16:13 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA09964 for mpi-core-out; Mon, 3 Feb 1997 09:19:41 -0600 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA09959 for ; Mon, 3 Feb 1997 09:19:37 -0600 Received: from jim (gw2-134.pool.dircon.co.uk [194.112.35.134]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id PAA14199 for ; Mon, 3 Feb 1997 15:19:31 GMT Message-Id: <199702031519.PAA14199@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id PAA02633; Mon, 3 Feb 1997 15:16:29 GMT To: mpi-core@mcs.anl.gov Subject: Informal minutes of last meeting Reply-to: jcownie@dolphinics.com Date: Mon, 03 Feb 1997 15:16:29 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here are the informal minutes. I don't think I have mis-represented anyone, but feel free to shout if you feel they need correction (even though they are informal !) -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com MPI 2 Chicago O'Hare January 21-23 ================================== James Cownie 28 January This is a report on the MPI meeting held at the Ramada hotel Chicago O'Hare on January 21-23. Summary ======= Most of I/O received a first formal vote. Those parts of one sided which had not already got it received a second formal vote. The collective chapter was significantly cut down, has a new chair (Bill Gropp), and some new ideas. Rusty Lusk is to be congratulated on having the foresight to call the meeting a day earlier than normal meetings, since O'Hare was closed by snow on Friday ! Miscellaneous Subcommittee -------------------------- Starting at 1-30p.m. Rusty Lusk started the discussion on the Miscellaneous chapter 3 (MPI 1.2 clarifications) and 9 (MPI 2 Miscellaneous items). In Clarification of MPI_Finalize (3.2.1) the sub-committee decided to remove the ordering requirements on destructor callbacks. The logic being that either libraries are dependent and co-operating, in which case a single finalize callback can be used to close down the libraries in the correct order, or libraries are independent, in which case unordered finalize callbacks are sufficient. A discussion on the "Salo/Squyres" naming conventions followed. I/O subcommittee ---------------- Bill Nitzberg chaired the I/O sub-committee discussions. The discussion commenced by looking at RESIZE and PREALLOCATE. There was discussion on whether PREALLOCATE was required, general opinion was in favour of keeping the function. Straw votes were then taken on whether RESIZE and PREALLOCATE chould be collective or independent Collective Independent Abstain RESIZE 10 2 8 PREALLOCATE 13 3 5 Both functions will therefore become collective. Section 10.6.3 on interleaving collective operations was discussed. Minor wording amendments were made. Discussion moved on to 10.8 Error handlers A proposal was made to remove error handlers from the FileHandle, but add an additional I/O error handler to a communicator (along with functions for setting and getting it). This allows handling of errors in OPEN in a sensible way, depsite there not yet being a file handle. This new error handler would handle all error classes defined in the I/O chapter (to be defined). Its default behaviour would be ERRORS_RETURN. A straw vote was taken 17:0:4 (Note that later decisions about whether it is possible to extract a communicator from a filehandle may require this decision to be reconsidered). Discussion moved on to the locking proopsal in 10.2.1. There was agreement that it would be useful to implementations to know that a file was being exclusively accessed. However requiring locking seemed hard, (it would raise many questions such as * Is the locking mandatory or advisory ? * How is locking to be enforced ? * How does the lock get released if the program crashes ? * ...) Given bad experiences with Unix Lock daemons, an alternative proposal was made to add a an "Open Unique" amode, by which the user program asserts that it is the only accessor of the file. This provides the same information to the system, while giving the user code the reponsibility of implementing locking in whatever way it requires. (This could be as simple as just asserting that only one copy of the program will run at one time, or could involve use of any other existing lock system). Add open unique Amode 16:1:2 Dynamic Formal votes -------------------- At 8-30p.m. Bill Saphir started formal votes on the remaining issues in the Dynamic chapter. There were (initially) 23 voting organisations present. 4.1 Introduction 22:0:1 4.2 Process model 20:0:3 4.5.3 Singleton MPI_Init 15:2:5 4.5.4 MPI_Appnum 14:5:4 4.5.5 Releasing connections An amendment to change the names of the functions MPI_Accept, MPI_Connect, MPI_Disconnect to MPI_Comm_* 19:1:2 4.5.5 as amended 16:0:6 Some function naming issues remain to be resolved, for instance MPI_Name_get should clearly be called something else (MPI_Name_lookup ?) Miscellaneous Formal Votes -------------------------- At 9-10p.m. Rusty Lusk started formal votes on chapter 3. 3.2.1 MPI_Finalize 22:0:1 3.2.1(b) Allowing user functions at MPI_Finalize time Delete ordering requirements on attribute callback functions 13:6:4 (as amended) 16:3:3 3.2.2 Clarification of MPI_Status result from send operations 20:0:2 3.2.4 Clarification of MPI_Intercomm_merge 23:0:0 Discussion on whether the C types MPI_LONG_LONG_INT and MPI_UNSIGNED_LONG_LONG_INT should be mandatory or optional types. Optional 15 Required 3 Abstain 5 More discussion on the S/S naming rules. The general conclusion was that 1) The rules are sensible 2) Retro-fitting them to MPI-1 is dangerous because of the amount of literature and tutorial material which already exists. 3) Applying them literally to MPI-2 would also be perverse (For instance it would lead to MPI_Initialize and MPI_Initialized, but MPI_Finalize and MPI_Is_finalized ) At 9-50p.m. we moved on to Chapter 9 (MPI-2 Misceallaneous). 9.5 MPI_Finalized 21: 0:0 9.10 Continuable errors 4:11:7 *** Continuable errors are out *** 9.11.{1,2} Wide characters, Arithmetic byte types 21: 0:1 Finished at 10-15p.m. Wednesday January 22 ==================== I/O --- At 8-40a.m. Bill Nitzberg started on the formal voting on I/O. There were 23 institutions present (though this increased as people got up). 10.2.1 Open Siginificant discussion on whether the file name given at all of the processes needs 1) to be textually identical 2) refer to the same file An amendment was proposed to have a sinlge file name provided at the root process, and provide an additional root argument 9:11:4 (defeated) A second amendment was proposed that all filenames must be both textually identical, and refer to the same file 11:9:4 (The issues here are to do with how you name files to ensure unique names, in an environment in which you may be expanding the set of sites at which your program is running by using MPI_Spawn. Re-opening a given file name on a larger set of processes may be invalid, even though the name was valid on the original set). 10.2.1 as amended 15: 5:5 10.2.2 Iopen 3:18:6 10.2.3 Close 24: 0:2 10.2.5 Delete Amendment to add an info argument 13: 5:9 Errors to be handled in COMM_WORLD (accepted as friendly) 10.2.5 as amended 12: 5:9 Broke for coffee at 10-30a.m. Miscellaneous ------------- At 10-40a.m. Marc Snir started the formal voting on the language inter-operability sections of the Miscellaneous chapter. 9.6, 9.7.4 Handle types An amendement was proposed to remove handle types, and instead have 12 additional functions for transferring specific handles between C and Fortran 12:1:11 9.7.4 Transfer of handles (as amended) 20:0:4 9.7.6 Addresses 18:1:4 Rest of 9.7.6 19:0:6 9.7.7 Reduction operations 21:0:4 9.7.8 Transfer of Status objects 9.7.9 Transfer of constants Both delayed for further discussion 9.7.10 Inter language send/receive 21:0:4 9.8 Datatype constructors for 64 bit Fortran environments 9.8.1 HVECTOR_X and friends 20:0:6 9.8.2 Extent and bounds 18:0:7 9.8.3 True_extent 17:0:6 9.9 New caching functions for 64 bit Fortran environments 20:0:5 At 11-30a.m. Rusty Lusk took over again, and Bill Gropp presented some issues raised by the Intel test suite, in particular the specification of what happens if an attribute delete function fails during a Comm_free. Straw polls were taken For clarification Unanimous Any failure makes process erroneous 18 A stronger specification 6 A formal vote was then held to endorse suitable text to be written by Marc Snir and Bill Gropp 19:3:0 A proposal to add Group_world and Group_self 1:8:17 (defeated apathetically) 12-00 break for lunch. Discussions on Meta issues -------------------------- At 1p.m. Rusty reconvened the meeting to discuss administrative issues. In his absence (and since time is running short), a formal vote was taken to replace Tony Skjellum as chair of the collective sub-committee with Bill Gropp 25:1:4 Interoperability issues were raised by Rolf Rabenseifner A vote was taken on whether to consider interoperability in MPI-2 3:15:0 => MPI-2 will not attempt to standardise interoperability. Eric Salo argued for removing all non-blocking collective operations. A vote was taken on "Non-blocking collective operations are discouraged" 26:0:8 => Chapter authors will reconsider how their chapter can survive without non-blocking collective operations. 1-50p.m. The meeting split into subcommittees. External Interfaces Subcommitee ------------------------------- Steve Huss-Ledermann chaired the external interfaces sub-committee. Much discussion on Generalised Requests, which ultimately lead to proposals from Marc Snir, Raja Daoud and Eric Salo which will appear in the mail. This should be consulted for the current state, which has already moved on significantly since the meeting ! One Sided Subcommittee ---------------------- Win_free becomes collective, all RMA ops must be complete before a window can be freed. Replace MPI_Int with MPI_Aint in alloc and many other places, so that one can expose a window > 2GB. C++ Bindings ------------ At 8-05p.m. Andrew Lumsdaine started the formal votes on the C++ bindings. 22 voting organisations present. Proposal to adopt the S/S naming convention for the C++ bindings 11:3:7 Accept the binding style of appendx C (the "new" C++ bindings) 16:0:5 Accept C++ bindings 18:0:3 8.1.5 type additions 15:0:4 8.1.7 C++ Profiling interface 8:0:13 One sided --------- Marc Snir took over at 8-55p.m. to lead the committee through formal votes on the changes to the one-sided chapter. Make Win_free a collective operation 18:0:2 Window size and displacements become Aint 15:2:2 5.6 Semantics 7:2:9 Fortran 90 Bindings ------------------- Bill Saphir started into formal voting on the F90 bindings at 9-05p.m. 8.2.{1,2} Overview and basic F90 support 13:0:7 Move 8.2.2 (basic F90 support) to MPI 1.2 4:2:11 Remove requirement for overloaded functions in the interface (becomes a recommendation) 7:2:10 8.2.3 Extended F90 support 9:1:9 External Interfaces ------------------- Steve H-L 7.6 Error codes and classes 14:0:6 14.2 Allow attribute caching on datatypes 17:0:2 7.7 Accessing datatypes (intro) 17:0:3 7.7.1 Get_type_envelope, Get_type_contents 8:3:9 7.7.2 Datatype encode and decode 8:10:5 => Datatype encode and decode do *not* exist in MPI-2 7.7.3 Canonical type representation postponed. 7.4 Setting fields in MPI_Status 14:0:5 Type_dup 15:0:7 7.5 Comm_set_name, Comm_get_name 12:1:9 Meeting closed about 10-15p.m. Thursday ======== Collective ---------- At 8-30a.m. Bill Gropp started discussion and formal votes on the collective chapter. 22 voting organisations. 6.2 Persistent collective operations. Removed by sub-committee. 6.3 Non-blocking collective operations. Removed by sub-committee. 6.4 Comm_create, Comm_split on intercommunicators 13:0:9 6.5 Intercomm collectives (through 6.5.3) 15:2:5 In place operations only for Intra communicators 17:0:6 6.5.{4,5} AllToAll_W 16:0:8 6.6 Exclusive scan 6:1:17 6.7 Channels Removed by sub-committee. Discussion of "Two phase operations" as an alternative to non-blocking operations. Such operations have a start and an end function, but do not produce a handle object. Only a single two-phase operation is permitted in any communicator at one time. Either the start, or the end, or both may synchronise. Straw vote to pursue this idea 15:7:7 I/O --- Bill Nitzberg started back into I/O votes at 9-30a.m. 10.4.2 Write explicit (without non-blocking form) 16:0:5 10.4.3 Independent file pointers 19:0:2 Get_position becomes two separate functions 10.3 File views 18:0:2 Absolute address in an etype or ftype is erroneous. 10.2.6 Resize (renamed to File_set_size) 13:0:5 10.2.{7,8} Prealloc, Get_size Amendment that resize can only truncate 1:5:13 (fails) 10.2.9 Querying file parameters Get_file_comm becomes Get_file_group 6:5:9 Get_file_group function wanted 11:4:5 Get_amode 13:0:7 10.2.10 File info access. set 14:1:6 get 13:2:6 10.4.4 Shared file pointers 13:4:3 10.5 Interoperability Pend file_get_type_size 16:0:6 Desire for a canonical file format 13:4:4 Meeting finished at noon. From mpi-core-human@mcs.anl.gov Tue Feb 4 19:28:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA18366; Tue, 4 Feb 1997 19:28:43 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA19184 for mpi-core-out; Tue, 4 Feb 1997 18:31:08 -0600 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id SAA19177 for ; Tue, 4 Feb 1997 18:31:03 -0600 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA03501; Tue, 4 Feb 97 17:31:01 MST From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9702050031.AA03501@nipmuc.fsl.noaa.gov> Subject: Canonical (External?) Data Representation To: mpi-core@mcs.anl.gov Date: Tue, 4 Feb 1997 17:31:00 -0700 (MST) Cc: hart@antares.mcs.anl.gov (Leslie Hart) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 3397 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Some random thoughts regarding canonical data representations. When we are discussing CANONICAL data representation(s) for MPI I/O perhaps the following questions are useful to look at: 1) Is there a specific bit pattern I can expect in a file when I read/write? 2) Can I write a file using machine A and use the result with the XYZ graphics package on machine B? 3) Can I interoperate with other MPI I/O packages? 4) Is there an existing standard that is adequate? 5) Do we want to leave open the possibililty of larger word machines? 6) Do we want to allow canonical in MPI_PACK/UNPACK? 7) Do we want to introduce it for MPI interoperability? Question 1: I think it is reasonable for a the user to have some minimal level of control regarding the input and output format of his/her files. Question 2: We can probably craft something to solve this problem. My original proposal of October (I think) went quite a way in that direction. It requires the user to be able to define the size of each language type independently. We could do this with a number of pre-defined attributes on a special communicator, but it gets messy pretty quickly. Question 3: Not so different from question 1. Question 4: There is XDR and ASN.1: For www details: XDR: ftp://ds.internic.net/rfc/rfc1832.txt ASN.1: http://www.inria.fr/rodeo/personnel/hoschka/asn1.html XDR vs ASN.1: http://ganges.cs.tcd.ie/4ba2/presentation/xdrandber.html What are the short comings? They don't map language types to their defined data types. We still have to make a mapping from language types to bit representations. XDR has some packing issues with types shorter than 32 bits, Padding is added to the end of shorter types to round up to 32 bit boundaries (three 1-byte characters are packed into a 4-byte area). ASN.1 includes some meta-data that may make it more difficult for the average program to interpret. Do we want to use XDR or ASN.1? I don't think either are the most suitable for an external representation in MPI I/O. Question 5: This probably means an extensible solution, which gets as messy as something that would solve question 2. Question 6: This is the current proposal. Question 7: This is an issue for the separate group considering interoperability. So, what are we left with? We can define a single CANONICAL representation (Albert and I proposed two at the last meeting, a 32 bit based representation and a 64 bit based representation.) If we do this I would favor one that loses the least amount of information, something like the CANONICAL64 we proposed in January. We can define several EXTERNAL representations (this is essentially what Albert and I did for the last meeting). We can define method for mapping language data types to a small number of primitive data types. This involves a pre-defined attribute per language on a "special" communicator. Each decimal (hex, octal?) of the value would be the log(2) of the number of bytes of the primitive types that the user can control with limits on the valid choices. This will get messy for implementors pretty quick and will most likely not be performance oriented :-) Comments, suggestions? Regards, Leslie Hart (hart@fsl.noaa.gov) NOAA/FSL From mpi-core-human@mcs.anl.gov Wed Feb 5 15:25:57 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA01514; Wed, 5 Feb 1997 15:25:46 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA05378 for mpi-core-out; Wed, 5 Feb 1997 14:28:53 -0600 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA05252; Wed, 5 Feb 1997 14:24:58 -0600 Message-Id: <199702052024.OAA05252@antares.mcs.anl.gov> Received: from ry73.rz.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Wed, 5 Feb 1997 21:24:56 +0100 Received: by ry73.rz.uni-karlsruhe.de (1.38.193.4/16.2) id AA02398; Wed, 5 Feb 1997 21:24:51 +0100 Subject: Some Fortraner comments... To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Date: Wed, 5 Feb 1997 21:24:50 +0100 (CET) From: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 7bit Content-Length: 4148 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Some Fortraner Comments on MPI-2 Draft 07-Nov-96: ------------------------------------------------- The following suggestions might be worth considering when preparing future drafts of the MPI-2 document. Sorry I did not check if any of these have been addressed in newer drafts... 1. The document uses IERR in some places, IERROR in other places to denote the Fortran binding's error argument. This is irregular, and a Fortran90/95 explicit interface which makes dummy argument names visible to the user would unravel this irregularity. Please stick with IERROR as it is in MPI 1.1. 2. Fortran90/95 elements are used in some places (KIND=, ! for comments, DO/END DO, ...). It would be a significant improvement for people concerned about Fortran90/95 bindings if the INTENT attribute would also be used in the specification of the Fortran bindings. This does not solve the -type and other problems, but would be a good first step (and easy to ignore for FORTRAN77-ers). The attribute comes in three forms, INTENT(IN|OUT|INOUT), and should be easily derived from the language-independent specs. Lots of typing, of course, and a separate section for corresponding MPI 1.1 Fortran interfaces might be necessary... 3. page 9:34+ says: "Discussion: Did we want 30 or 31? Assume upper limit is 32 and need 1 for profiling??" Fortran90/95 only guarantees 31 characters in a name, minus one for profiling means I would advocate THIRTY significant characters. 4. page 72:44 talks about "``true'' (C-like, Cray-like) pointers". I would resist calling Fortran90/95 pointers ``wrong'' pointers, since there is a law (ISO/IEC 1539) which defines them to be ``true'' pointers, too. ``true ( )'' should be deleted from MPI-2. 5. I would like to see the following problems in MPI 1.1 fixed as part of MPI 1.2 (references are to MPI 1.1, 12-June-95, with changebars): page 122:1 and 215:8 Change "FUNCTION USER_FUNCTION( INVEC(*), INOUTVEC(*), LEN, TYPE)" to "SUBROUTINE USER_FUNCTION(INVEC, INOUTVEC, LEN, TYPE)" to have the correct classification (I think) as a subroutine, and to make the dummy arg declarations correct Fortran. page 170:5+ and 170:46+ The INTENT of the arguments of the Fortran declarations should be given. For the main MPI procedures, intent may be derived from the language- independent specs, but not for these dummy procedures. page 215:13 and 215:21 Change "PROCEDURE" to "SUBROUTINE", just to make it FORTRAN. page 215:23 There is no Fortran example for a Handler_Function. It should. 6. Minor fixes in MPI-2: pages 79-83: Assuming we do not talk about such revolutionary things as internal procedures or module procedures which might access 'mpif.h' by host association, an "INCLUDE 'mpif.h'" statement is required in these three subroutines to bring the MPI names into scope. Also note that subroutines end with END, not with RETURN (which is redundant here). page 80:7 "END DO" is missing. page 133:48 add ", IERR" page 181:11+ add "CALL " (four times) page 334+ add "CALL " to all subroutine references Thanks, Michael PS: ISO's Fortran/C interoperability project mentioned on page 335:45 will probably forward its current draft to the first round of ballots in March 1997. I'm the project editor, and will write up a summary of what can and can't be done for MPI bindings, using that standard. I'll let MPIF know when (and where) this write-up will be available... ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) No longer on BITNET :-( Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Fri Feb 7 03:58:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA24463; Fri, 7 Feb 1997 03:58:53 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA09373 for mpi-core-out; Fri, 7 Feb 1997 03:02:19 -0600 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA09357; Fri, 7 Feb 1997 03:01:47 -0600 Received: by nsgi3.lbl.gov (8.7/ner-1) id BAA07541; Fri, 7 Feb 1997 01:01:46 -0800 (PST) Date: Fri, 7 Feb 1997 01:01:46 -0800 (PST) From: wcs@nersc.gov ( W. Saphir) Message-Id: <199702070901.BAA07541@nsgi3.lbl.gov> To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Subject: Re: Some Fortraner comments... Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > 1. The document uses IERR in some places, IERROR in other places to > denote the Fortran binding's error argument. This is irregular, > and a Fortran90/95 explicit interface which makes dummy argument > names visible to the user would unravel this irregularity. > Please stick with IERROR as it is in MPI 1.1. Good point. This is now fixed in the dynamic chapter. In a brief glance through the document this looks like the only place where it was wrong. > 2. Fortran90/95 elements are used in some places (KIND=, ! for comments, > DO/END DO, ...). It would be a significant improvement for people > concerned about Fortran90/95 bindings if the INTENT attribute would > also be used in the specification of the Fortran bindings. This does > not solve the -type and other problems, but would be a good > first step (and easy to ignore for FORTRAN77-ers). The attribute comes > in three forms, INTENT(IN|OUT|INOUT), and should be easily derived from > the language-independent specs. Lots of typing, of course, and a separate > section for corresponding MPI 1.1 Fortran interfaces might be necessary... The warrants further discussion. It seems like basically a good idea, but I'm not sure how to handle some details. 1. Do we now need to specify separate F77 and F90 bindings? 2. Am I correct that these functions now require explicit interfaces and therefore would require using the mpi module? 3. For routines with choice arguments, do we just leave out the intent information? A related change would be to use assumed-shape arrays rather than assumed-size arrays for dummy arguments. This seems to involve the same issues. > 3. page 9:34+ says: > "Discussion: Did we want 30 or 31? > Assume upper limit is 32 and need 1 for profiling??" > Fortran90/95 only guarantees 31 characters in a name, minus one for > profiling means I would advocate THIRTY significant characters. Sounds good. > > 4. page 72:44 talks about "``true'' (C-like, Cray-like) pointers". > I would resist calling Fortran90/95 pointers ``wrong'' pointers, > since there is a law (ISO/IEC 1539) which defines them to be > ``true'' pointers, too. ``true ( )'' should be deleted from MPI-2. Agreed. > > 5. I would like to see the following problems in MPI 1.1 fixed as part of > MPI 1.2 (references are to MPI 1.1, 12-June-95, with changebars): > > page 122:1 and 215:8 > Change "FUNCTION USER_FUNCTION( INVEC(*), INOUTVEC(*), LEN, TYPE)" > to "SUBROUTINE USER_FUNCTION(INVEC, INOUTVEC, LEN, TYPE)" > to have the correct classification (I think) as a subroutine, > and to make the dummy arg declarations correct Fortran. > > page 170:5+ and 170:46+ > The INTENT of the arguments of the Fortran declarations should be given. > For the main MPI procedures, intent may be derived from the language- > independent specs, but not for these dummy procedures. > > page 215:13 and 215:21 > Change "PROCEDURE" to "SUBROUTINE", just to make it FORTRAN. > > page 215:23 > There is no Fortran example for a Handler_Function. It should. I don't have an MPI 1.1 spec in front of me, but I do remember the first example quite clearly. I intended to request that this be added to MPI 1.2. Rusty, do we still have an opportunity to add these to the list of minor corrections? > 6. Minor fixes in MPI-2: > The page numbers you give here don't correspond to my January draft. Could you give a little more context for the location? > PS: ISO's Fortran/C interoperability project mentioned on page 335:45 > will probably forward its current draft to the first round of ballots > in March 1997. I'm the project editor, and will write up a summary > of what can and can't be done for MPI bindings, using that standard. > I'll let MPIF know when (and where) this write-up will be available... Again, the reference doesn't correspond to my text. What chapter/section is this in? Thanks for the comments. Bill From mpi-core-human@mcs.anl.gov Fri Feb 7 08:48:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA23480; Fri, 7 Feb 1997 08:48:18 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA11826 for mpi-core-out; Fri, 7 Feb 1997 07:51:26 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA11821; Fri, 7 Feb 1997 07:51:22 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id HAA00628; Fri, 7 Feb 1997 07:51:19 -0600 (CST) Date: Fri, 7 Feb 1997 07:51:19 -0600 (CST) Message-Id: <199702071351.HAA00628@rap.cs.wisc.edu> From: Steve Huss-Lederman To: wcs@nersc.gov CC: hennecke@rz.uni-karlsruhe.de, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <199702070901.BAA07541@nsgi3.lbl.gov> (wcs@nersc.gov) Subject: Re: Some Fortraner comments... Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > > 3. page 9:34+ says: > > "Discussion: Did we want 30 or 31? > > Assume upper limit is 32 and need 1 for profiling??" > > Fortran90/95 only guarantees 31 characters in a name, minus one for > > profiling means I would advocate THIRTY significant characters. > > Sounds good. > > > > > 5. I would like to see the following problems in MPI 1.1 fixed as part of > > MPI 1.2 (references are to MPI 1.1, 12-June-95, with changebars): > > > > page 122:1 and 215:8 > > Change "FUNCTION USER_FUNCTION( INVEC(*), INOUTVEC(*), LEN, TYPE)" > > to "SUBROUTINE USER_FUNCTION(INVEC, INOUTVEC, LEN, TYPE)" > > to have the correct classification (I think) as a subroutine, > > and to make the dummy arg declarations correct Fortran. > > > > page 170:5+ and 170:46+ > > The INTENT of the arguments of the Fortran declarations should be given. > > For the main MPI procedures, intent may be derived from the language- > > independent specs, but not for these dummy procedures. > > > > page 215:13 and 215:21 > > Change "PROCEDURE" to "SUBROUTINE", just to make it FORTRAN. > > > > page 215:23 > > There is no Fortran example for a Handler_Function. It should. > > I don't have an MPI 1.1 spec in front of me, but I do remember the > first example quite clearly. I intended to request that this be added > to MPI 1.2. Rusty, do we still have an opportunity to add these > to the list of minor corrections? I did not broadcast my response to these. As editor, I have made all the above corrections. They will show up in the next draft. The only exception is INTENT and a Fortran binding for Handler_Function. I wanted to understand about what was wanted for the INTENT (IN, INOUT, OUT; or some Fortran indication). Also, it was not clear what to do with Handler_Function since the C binding takes a variable number of arguments. I was not sure how to do this in Fortran. I was waiting for a response on how to do this and whether the omission in MPI-1 was deliberate. I welcome ideas here. > > > 6. Minor fixes in MPI-2: > > > The page numbers you give here don't correspond to > my January draft. Could you give a little more context for the > location? > > > PS: ISO's Fortran/C interoperability project mentioned on page 335:45 > > will probably forward its current draft to the first round of ballots > > in March 1997. I'm the project editor, and will write up a summary > > of what can and can't be done for MPI bindings, using that standard. > > I'll let MPIF know when (and where) this write-up will be available... > > Again, the reference doesn't correspond to my text. What chapter/section > is this in? > As Michael pointed out the number are the SC96 draft. You can find it on the Argonne MPI WWW page. I made the one fix to external. I assume chapter authors will do the others. Steve From mpi-core-human@mcs.anl.gov Fri Feb 7 10:27:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA24738; Fri, 7 Feb 1997 10:27:53 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA13456 for mpi-core-out; Fri, 7 Feb 1997 09:30:51 -0600 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id FAA10523; Fri, 7 Feb 1997 05:25:02 -0600 Message-Id: <199702071125.FAA10523@antares.mcs.anl.gov> Received: from ry73.rz.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Fri, 7 Feb 1997 12:23:55 +0100 Received: by ry73.rz.uni-karlsruhe.de (1.38.193.4/16.2) id AA06959; Fri, 7 Feb 1997 12:23:52 +0100 Subject: Re2: Some Fortraner comments... To: wcs@nersc.gov (W. Saphir) Date: Fri, 7 Feb 1997 12:23:51 +0100 (CET) Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-Reply-To: <199702070901.BAA07541@nsgi3.lbl.gov> from "W. Saphir" at Feb 7, 97 01:01:46 am From: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 7bit Content-Length: 3171 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk According to W. Saphir: > > > 2. Fortran90/95 elements are used in some places (KIND=, ! for comments, > > DO/END DO, ...). It would be a significant improvement for people > > concerned about Fortran90/95 bindings if the INTENT attribute would > > also be used in the specification of the Fortran bindings. This does > > not solve the -type and other problems, but would be a good > > first step (and easy to ignore for FORTRAN77-ers). The attribute comes > > in three forms, INTENT(IN|OUT|INOUT), and should be easily derived from > > the language-independent specs. Lots of typing, of course, and a separate > > section for corresponding MPI 1.1 Fortran interfaces might be necessary... > > The warrants further discussion. > It seems like basically a good idea, but I'm not sure how to > handle some details. > 1. Do we now need to specify separate F77 and F90 bindings? > 2. Am I correct that these functions now require explicit interfaces > and therefore would require using the mpi module? > 3. For routines with choice arguments, do we just leave out > the intent information? I would like *one* text for F77 and F90/F95 binding, saying that the INTENT attribute may be ignored by the F77 user, and *not* requiring that the MPI procedures *must* have an explicit interface. Still it would be illegal to pass a constant to an INTENT(IN) dummy argument. The compiler is not required to detect this (either with or without INTENT given). But *if* an implementation wants to provide explicit interfaces, it needs the information about the INTENT it should use. INTENT for may be given as for ordinary dummies, again *not* requiring that the actual code provides an explicit interface. > A related change would be to use assumed-shape arrays rather > than assumed-size arrays for dummy arguments. This > seems to involve the same issues. NO. This would need significant changes in MPI code (as opposed to document changes), which I think are too complex to be done within the current MPI-2 schedule (all that descriptor stuff, choice, ...). > > 6. Minor fixes in MPI-2: > The page numbers you give here don't correspond to > my January draft. Could you give a little more context for the > location? > > > PS: ISO's Fortran/C interoperability project mentioned on page 335:45 > Again, the reference doesn't correspond to my text. What chapter/section > is this in? The first line of my email says "Comments on MPI-2 Draft 07-Nov-96", that should be sufficient information. This was the SC'96 review draft, I guess. Thanks, Michael ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) No longer on BITNET :-( Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Fri Feb 7 18:10:14 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA02490; Fri, 7 Feb 1997 18:10:07 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA25161 for mpi-core-out; Fri, 7 Feb 1997 17:13:31 -0600 Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA24847; Fri, 7 Feb 1997 17:00:00 -0600 Received: from [152.136.26.29] ([152.136.26.29]) by cs.rice.edu (8.8.5/8.7.1) with ESMTP id QAA29941; Fri, 7 Feb 1997 16:59:50 -0600 (CST) X-Sender: chk@titan.cs.rice.edu Message-Id: In-Reply-To: <199702070901.BAA07541@nsgi3.lbl.gov> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Fri, 7 Feb 1997 16:29:40 -0600 To: wcs@nersc.gov ( W. Saphir) From: Chuck Koelbel Subject: Re: Some Fortraner comments... Cc: hennecke@rz.uni-karlsruhe.de (Michael Hennecke), mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >> 2. Fortran90/95 elements are used in some places (KIND=, ! for comments, >> DO/END DO, ...). It would be a significant improvement for people >> concerned about Fortran90/95 bindings if the INTENT attribute would >> also be used in the specification of the Fortran bindings. This does >> not solve the -type and other problems, but would be a good >> first step (and easy to ignore for FORTRAN77-ers). The attribute comes >> in three forms, INTENT(IN|OUT|INOUT), and should be easily derived from >> the language-independent specs. Lots of typing, of course, and a separate >> section for corresponding MPI 1.1 Fortran interfaces might be >>necessary... > >The warrants further discussion. >It seems like basically a good idea, but I'm not sure how to >handle some details. >1. Do we now need to specify separate F77 and F90 bindings? >2. Am I correct that these functions now require explicit interfaces >and therefore would require using the mpi module? >3. For routines with choice arguments, do we just leave out >the intent information? I haven't had a chance to download the chapter, but can offer a few comments: 1. KIND= requires explicit subprogram interfaces; therefore, it cannot be part of an F77 binding. If you use KIND=, you already need separate F77 and F90 bindings (although the F90 bindings could be a superset of the F77). 2. INTENT does not require explicit subprogram interfaces. It is basically advice to the compiler and/or programmer. So it does not require a separate binding, but it may look a bit odd to people who only know F77. I think that adding INTENT would be a good idea. It would probably be wise to add a one-sentence (maybe one-paragraph) explanation for F77 programmers as well. I'm doing this from memory, but I'm sure somebody will quickly correct me if I misspoke.:-) >A related change would be to use assumed-shape arrays rather >than assumed-size arrays for dummy arguments. This >seems to involve the same issues. Yes, assumed-shape arrays require explicit interfaces. But there are other issues with assumed-shape dummies as well (e.g. they can be passed strided sections, which interact poorly with MPI data types). Chuck ********************************************************************** Charles Koelbel CRPC, MS 132 Center for Research on Parallel Computation Rice University Rice University 6100 Main Street chk@cs.rice.edu Houston, TX 77005 ********************************************************************** From mpi-core-human@mcs.anl.gov Fri Feb 7 22:06:46 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA03912; Fri, 7 Feb 1997 22:06:45 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA27806 for mpi-core-out; Fri, 7 Feb 1997 21:10:17 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA27801 for ; Fri, 7 Feb 1997 21:10:14 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id VAA05745; Fri, 7 Feb 1997 21:10:12 -0600 (CST) Date: Fri, 7 Feb 1997 21:10:12 -0600 (CST) Message-Id: <199702080310.VAA05745@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: new document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. It is the first shot at a version of the next meeting. I encourage you all to carefully check this over since it is getting down to the wire for making changes to MPI-2. The final version of the chapters is due to me on 26 February. Due to current difficulties with my computer systems, I have not been able to preview the document. If anyone finds problems with it please let me know. Steve From mpi-core-human@mcs.anl.gov Sat Feb 8 17:16:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA08509; Sat, 8 Feb 1997 17:16:26 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA05665 for mpi-core-out; Sat, 8 Feb 1997 16:19:24 -0600 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA04976; Sat, 8 Feb 1997 15:09:01 -0600 Message-Id: <199702082109.PAA04976@antares.mcs.anl.gov> Received: from ry70.rz.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Sat, 8 Feb 1997 13:21:20 +0100 Received: by ry70.rz.uni-karlsruhe.de (1.37.109.16/16.2) id AA105684473; Sat, 8 Feb 1997 13:21:13 +0100 Subject: KIND and INTENT To: chk@cs.rice.edu (Chuck Koelbel) Date: Sat, 8 Feb 1997 13:21:13 +0100 (CET) Cc: wcs@nersc.gov, hennecke@rz.uni-karlsruhe.de, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-Reply-To: from "Chuck Koelbel" at Feb 7, 97 04:29:40 pm From: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 7bit Content-Length: 2249 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk According to Chuck Koelbel: > 1. KIND= requires explicit subprogram interfaces; therefore, it cannot be > part of an F77 binding. If you use KIND=, you already need separate F77 > and F90 bindings (although the F90 bindings could be a superset of the F77). No, KIND doesn't require explicit subprogram interfaces. You just have to get the KIND of the actual argument right. Only that the KIND= keyword itself requires an f90 compiler :-) > 2. INTENT does not require explicit subprogram interfaces. It is basically > advice to the compiler and/or programmer. So it does not require a > separate binding, but it may look a bit odd to people who only know F77. Yes and no. It does not require explicit subprogram interfaces, but it is more than an advice: e.g. it causes INTENT(OUT) arguments to become undefined on entry into the procedure. (Exception: when in F95 there is a component-initialization of an ) ( INTENT(OUT) derived type component, this init. is done.) This makes a call with a constant actual argument non-std-conforming at the point of the *call*, not only at the point where the procedure actually tries to *assign* to the INTENT(OUT) dummy argument. Of course, for the compiler to *detect* this the interface probably has to be explicit, but this is not a requirement of the F90/F95 standard. > I think that adding INTENT would be a good idea. It would probably be wise > to add a one-sentence (maybe one-paragraph) explanation for F77 programmers > as well. Yes, this could be in the introduction section. > I'm doing this from memory, but I'm sure somebody will quickly correct me > if I misspoke.:-) Done. Thanks, Michael ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) No longer on BITNET :-( Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Mon Feb 10 14:28:09 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA27604; Mon, 10 Feb 1997 14:28:03 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA00332 for mpi-core-out; Mon, 10 Feb 1997 13:28:10 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id NAA00326 for ; Mon, 10 Feb 1997 13:28:07 -0600 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA42875; Mon, 10 Feb 1997 14:27:49 -0500 Message-Id: <32FF76C4.167E@kgn.ibm.com> Date: Mon, 10 Feb 1997 14:28:04 -0500 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: New text in MPI-2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Though I am not a procedure expert, as I understand the rules for MPI-2 there is one meeting left at which formal votes can be taken. To be in MPI-2 a piece of text, or at least the concept it represents, must be voted twice. Therefor - unless we can reasonably argue that we are taking a second vote on text substantially equivalent to something already passed once, text in question cannot go in. I am already very uncomfortable with the level of change MPI-2 is seeing right down to the wire. In my view, the idea that MPI-2 content will all have received two votes is already being stretched to the limit. Let's stick to what we already have and try to get that right. -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Wed Feb 12 15:37:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA12075; Wed, 12 Feb 1997 15:36:54 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA18677 for mpi-core-out; Wed, 12 Feb 1997 14:40:57 -0600 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA18668; Wed, 12 Feb 1997 14:40:51 -0600 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA12726; Wed, 12 Feb 97 13:40:42 MST Date: Wed, 12 Feb 97 13:40:42 MST From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9702122040.AA12726@nipmuc.fsl.noaa.gov> To: mpi-core@mcs.anl.gov Subject: Re: Canonical (External?) Data Representation Cc: mpi-io@mcs.anl.gov X-Mailer: [XMailTool v3.1.2b] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Although there has been some useful discussion regarding a Canonical or multiple External formats, there seems to be no consensus. We need to make some decision in the short term (no decision is a decision to have no explicitly defined data representation). Below is a (potentially biased) summary of the choices we have, I am willing to write up at least one formal proposal: 1) Have exactly one Canonical representation. We need some consensus on what this should look like. 2) Assemble all the relevant standards and some how support them. We need a comprehensive list of these standards and mappings from all language types to the standard(s) types. 3) Target a large number of native machine types and provide translation on input and output. We need syntax and semantics on how to make this happen. I think the multiple language mappings can make this messy. 4) Have a small number of External representations. We need a consensus on how many and what they should look like. 5) Forget it (this proposal is the easiest to write up :-). Pros and cons: 1: The syntax is easy for this one. At most one translator needs to be written. It will either consume more memory/disk or loose precision depending on whether we choose something short or long. 2: There can be a lot of standards out there (although, I only found 2 official looking ones in my looking.) XDR: We have to work out the details of MPI_BYTE and MPI_CHAR, I think one choice could lead to each taking 32 bits per octet. The other choice would lead to padding that the user may have to be aware of (1 byte takes the same space as 2, 3, or 4 bytes). 3: I personally don't see the vendors buying this one, but I am willing to be surprised. It does seem to solve a lot of problems. The syntax could get rather complicated, although we could predefine data set types such as 32-bit SPARC, 64-bit MIPS, ... 4: This requires more than one set of translators. It allows for potentially a "small" representation and a "big" representation (perhaps making a few more people happy, while at the same time making more miserable :-) 5: We should at least strongly suggest that the vendor enumerate the nature of internal and native representations in their documentation so that a non-MPI program has a shot at reading data. BOTTOM LINE: We need one or more VERY SPECIFIC proposals before the next meeting. Regards, Leslie Hart Leslie Hart (hart@fsl.noaa.gov) (303) 497 7253 CIRA NOAA/Forecast Systems Laboratories Aviation Divison Model Verification and Production Assistance Branch High Performance Computing Section R/E/FS5 325 Broadway Boulder, CO 80303 From mpi-core-human@mcs.anl.gov Wed Feb 12 16:31:31 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA12689; Wed, 12 Feb 1997 16:31:30 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA19882 for mpi-core-out; Wed, 12 Feb 1997 15:36:10 -0600 Received: from ncsa.uiuc.edu (sdgmail.ncsa.uiuc.edu [141.142.103.66]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA19855; Wed, 12 Feb 1997 15:35:46 -0600 Received: from shalom.ncsa.uiuc.edu (shalom [141.142.21.11]) by ncsa.uiuc.edu (8.8.5/8.8.5) with SMTP id PAA10920; Wed, 12 Feb 1997 15:35:37 -0600 (CST) Received: by shalom.ncsa.uiuc.edu (940816.SGI.8.6.9) id PAA21497; Wed, 12 Feb 1997 15:35:44 -0600 From: "Albert Cheng" Message-Id: <9702121535.ZM21495@shalom.ncsa.uiuc.edu> Date: Wed, 12 Feb 1997 15:35:43 -0600 In-Reply-To: hart@nipmuc.fsl.noaa.gov (Leslie Hart) "Re: Canonical (External?) Data Representation" (Feb 12, 1:40pm) References: <9702122040.AA12726@nipmuc.fsl.noaa.gov> X-Mailer: Z-Mail (3.1.0 22feb94 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: Canonical (External?) Data Representation Cc: mpi-io@mcs.anl.gov Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk [Leslie, thank you to staying on this issue.] > 5) Forget it (this proposal is the easiest to write up :-). This could be very harmful for the MPIO interface. Notice currently there are far more sequential machines than parallel machines and zillion times more software using sequential I/O than MPIO. Scientists using MPI to calculate a complex problem would like to write the result to a file for visualization and analysis by other existing packages. Similarly, they also may want their newly coded MPI application to read data files from other sequential application. Without a defined canonical format, they are stuck. Their only alternative would be to funnel all data through one process which does I/O in the traditional way. If the MPIO interface cannot interact with sequential I/O application, it will be living on an island. Someone has suggested to use the native machine format for data exchange since majority data files are shared within the same platform. This avoids unnecessarily conversions and loss of precision. It is up to the reader program to do the conversion. I think that is exactly what the "Native format" is. That one provides speed with the risk that the file is not exchangable with another platform, or even another MPIO implementation. The canonical format is supposedly to provide a definite way to do file exchange between platforms with the potential risk of loss of speed and precision. My feeling is that couple canonical formats would be sufficient for data exchange purpose. -- Albert -- From mpi-core-human@mcs.anl.gov Mon Feb 17 06:19:16 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA18202; Mon, 17 Feb 1997 06:19:15 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA04950 for mpi-core-out; Mon, 17 Feb 1997 05:23:02 -0600 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id FAA04925; Mon, 17 Feb 1997 05:21:57 -0600 Received: from jim (gw2-155.pool.dircon.co.uk [194.112.35.155]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id LAA15650; Mon, 17 Feb 1997 11:21:53 GMT Message-Id: <199702171121.LAA15650@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id KAA00845; Mon, 17 Feb 1997 10:35:02 GMT To: "Albert Cheng" cc: mpi-core@mcs.anl.gov, mpi-io@mcs.anl.gov Subject: Re: Canonical (External?) Data Representation In-reply-to: Your message of "Wed, 12 Feb 1997 15:35:43 CST." <9702121535.ZM21495@shalom.ncsa.uiuc.edu> Date: Mon, 17 Feb 1997 10:35:01 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Albert points out the desire to be able to read files written with MPI/IO on sequential machines, or read files written on sequential machines via MPI-IO. Unfortunately even the canonical representation cannot make any promises about either of these operations. The canonical representation allows a file written by MPI-IO on one machine to be read by MPI-IO on another machine. It does not make any statements about reading such a file from a non MPI-IO environment. The reason for this omission is that there are many native file formats (particularly for files written from Fortran), and therefore MPI cannot be both canonical and inter-operable. The simplest solution which is guaranteed to work is very likely going to remain "read on one process using MPI-IO, and write using the normal mechanism". While this is a depressing conclusion, please note that this is *not* an MPI problem, but an existing problem that MPI happens to have bumped into. -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Mon Feb 17 12:33:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA24171; Mon, 17 Feb 1997 12:33:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA10671 for mpi-core-out; Mon, 17 Feb 1997 11:31:40 -0600 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA10666; Mon, 17 Feb 1997 11:31:37 -0600 Received: by nipmuc.fsl.noaa.gov (4.1/SMI-4.1) id AA17324; Mon, 17 Feb 97 10:31:25 MST From: hart@nipmuc.fsl.noaa.gov (Leslie Hart) Message-Id: <9702171731.AA17324@nipmuc.fsl.noaa.gov> Subject: Re: Canonical (External?) Data Representation To: jcownie@dolphinics.com (James Cownie) Date: Mon, 17 Feb 1997 10:31:25 -0700 (MST) Cc: acheng@ncsa.uiuc.edu, mpi-core@mcs.anl.gov, mpi-io@mcs.anl.gov In-Reply-To: <199702171121.LAA15650@mailhost.dircon.co.uk> from "James Cownie" at Feb 17, 97 10:35:01 am X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 2152 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The proposed standard does provide for the equivalents of copy and move functions to be present. The exact name and syntax are not provided, but they are required. There is also verbage regarding import/export of files from/to the "native" file system to/from the "MPI" system (no-one precludes this from being cp/mv/ln -s in Un*x). "High-quality" implementations are expected to provide retention of seek locations when a file is imported/exported. This still does not solve the record oriented language/system problem, but does solve a large number of problems that the user may face. In short, the Canonical (or External) representation(s) together with import/ export (and in a high-quality implementaion, retention of seek addresses) gives the user a strong clue as to what is in a sequential file. Hopefully the user will only have to resort to "read on one process using MPI-IO, and write using the normal mechanism" on a very small number of "low quality" implementations. Leslie Hart (hart@fsl.noaa.gov) > > Albert points out the desire to be able to read files written with > MPI/IO on sequential machines, or read files written on sequential > machines via MPI-IO. > > Unfortunately even the canonical representation cannot make any > promises about either of these operations. > > The canonical representation allows a file written by MPI-IO on one > machine to be read by MPI-IO on another machine. It does not make any > statements about reading such a file from a non MPI-IO environment. > > The reason for this omission is that there are many native file > formats (particularly for files written from Fortran), and therefore > MPI cannot be both canonical and inter-operable. > > The simplest solution which is guaranteed to work is very likely going > to remain "read on one process using MPI-IO, and write using the > normal mechanism". > > While this is a depressing conclusion, please note that this is *not* > an MPI problem, but an existing problem that MPI happens to have > bumped into. > > -- Jim > > James Cownie > Dolphin Interconnect Solutions > Phone : +44 117 9071438 > E-Mail: jcownie@dolphinics.com > From mpi-core-human@mcs.anl.gov Mon Feb 17 22:10:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA01134; Mon, 17 Feb 1997 22:10:22 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA21556 for mpi-core-out; Mon, 17 Feb 1997 21:13:19 -0600 Received: from taurus.cus.cam.ac.uk (cusexim@taurus.cus.cam.ac.uk [131.111.8.48]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id GAA05393; Mon, 17 Feb 1997 06:22:32 -0600 Received: from nmm1 by taurus.cus.cam.ac.uk with smtp (Exim 1.598 #1) id 0vwS5M-0007U0-00; Mon, 17 Feb 1997 12:22:20 +0000 To: mpi-core@mcs.anl.gov, mpi-io@mcs.anl.gov Subject: Re: Canonical (External?) Data Representation Date: Mon, 17 Feb 1997 12:22:19 +0000 From: Nick Maclaren Message-Id: Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk James Cownie writes: > Albert points out the desire to be able to read files written with > MPI/IO on sequential machines, or read files written on sequential > machines via MPI-IO. > > Unfortunately even the canonical representation cannot make any > promises about either of these operations. > > The reason for this omission is that there are many native file > formats (particularly for files written from Fortran), and therefore > MPI cannot be both canonical and inter-operable. > > The simplest solution which is guaranteed to work is very likely going > to remain "read on one process using MPI-IO, and write using the > normal mechanism". Unfortunately, I have to agree with this - except for one thing. The reason that Fortran is more of a problem is PRECISELY because almost all current workstations run Unix or Unix-like systems, which use a byte-stream representation for files, and use one or more bytes to represent newline. And this is the C model. On systems that are based on a record model (such as MVS, CMS, VMS, and many older ones), exactly the converse is true. There is usually only one representation for Fortran, but one per compiler for C. Yes, I know that recent versions of VMS have added a revolting kludge for C. And there are probably still some specialised systems that can support neither model in a "natural" way, though I am not sure how many will be appropriate for MPI. Nick Maclaren, University of Cambridge Computer Laboratory, New Museums Site, Pembroke Street, Cambridge CB2 3QG, England. Email: nmm1@cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679 From mpi-core-human@mcs.anl.gov Tue Feb 18 18:48:15 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA24467; Tue, 18 Feb 1997 18:48:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA10736 for mpi-core-out; Tue, 18 Feb 1997 17:51:10 -0600 Received: from ncsa.uiuc.edu (sdgmail.ncsa.uiuc.edu [141.142.103.66]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA10731; Tue, 18 Feb 1997 17:51:06 -0600 Received: from narnia.ncsa.uiuc.edu (narnia.ncsa.uiuc.edu [141.142.21.165]) by ncsa.uiuc.edu (8.8.5/8.8.5) with SMTP id RAA08930; Tue, 18 Feb 1997 17:51:00 -0600 (CST) Message-Id: <3.0.32.19970218175102.00696978@sdgmail.ncsa.uiuc.edu> X-Sender: acheng@sdgmail.ncsa.uiuc.edu X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Tue, 18 Feb 1997 17:51:03 -0600 To: mpi-core@mcs.anl.gov From: Albert Cheng Subject: Re: Canonical (External?) Data Representation Cc: mpi-io@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 10:35 AM 2/17/97 +0000, James Cownie wrote: >Albert points out the desire to be able to read files written with >MPI/IO on sequential machines, or read files written on sequential >machines via MPI-IO. > >Unfortunately even the canonical representation cannot make any >promises about either of these operations. > >The canonical representation allows a file written by MPI-IO on one >machine to be read by MPI-IO on another machine. It does not make any >statements about reading such a file from a non MPI-IO environment. Jim, you are correct that the paragraph of the MPI_DATA_CANONICAL access mode does not specify compatibility with non-MPI environments. But in view of other context of the IO chapter, it can be concluded that way. Let me explain. First of all, the canonical representation defines the bytes size and bit pattern of each data value. So, for each MPI-view, there can be only one bit pattern of the data file if one walks through it byte by byte. (It is not as absolute as I present because the bytes under the displacement can be any random values and LOGICAL value TRUE can be any non-zero patterns. But you know what I mean. :-) Secondly, it is expected the MPI implementation would provide the mechanism/tools for conversion between files in and outside the MPI-environment with the requirement of maintaining correspondance between byte offsets (p.249, line 38-42). Therefore, for each MPI-view, there can be only one bit pattern of the data file outside of the MPI-environment, implying a "unique outside" data file. With this defined uniqueness, a non-MPI-IO environment has a defined way to interpret the data file for all MPI-environments. >The reason for this omission is that there are many native file >formats (particularly for files written from Fortran), and therefore >MPI cannot be both canonical and inter-operable. True. The canonical data representation cannot solve those problems but it gives the users a defined means to convert the native files into the canonical-formatted files which would work for all MPI-environments. I would expect a high-qualify vendor would provid tools for conversion in and out of their own MPI-environment but if a user has to move data files between different platforms, the canonical formatted file is a means they can rely on. -- Albert From mpi-core-human@mcs.anl.gov Wed Feb 19 11:24:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA03794; Wed, 19 Feb 1997 11:24:02 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA23731 for mpi-core-out; Wed, 19 Feb 1997 10:27:03 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA23726 for ; Wed, 19 Feb 1997 10:26:59 -0600 Message-Id: <199702191626.KAA23726@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: March MPI Forum meeting Date: Wed, 19 Feb 1997 10:26:50 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, The next MPI Forum meeting is March 5-7 (Wed-Thurs-Fri) at the Ramada O'Hare in Chicago (the usual place). You must make your hotel reservations by tomorrow to qualify for the group rate. Please let me know now for the head count whether you are coming. Thanks. Rusty --------------------------------------------------------------------------- Here is the usual blurb: The next meeting of the MPI Forum will take place on Wed.-Fri. March 5-7. at the Ramada Hotel O'Hare (same place as the last few meetings). It is time to make your reservations. Register at the hotel by calling (847) 827-5131. When you register, tell them you are from the MPI Forum to get the special rate (around $95). When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. The registration fee will be $75, which should be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. If you have any problems or questions, let me know, either by email (lusk@mcs.anl.gov) or phone (708-252-7852). The next (and last) MPI Forum meeting will be April 23-25. Regards, Rusty From mpi-core-human@mcs.anl.gov Fri Feb 21 15:24:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA18932; Fri, 21 Feb 1997 15:24:41 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA16118 for mpi-core-out; Fri, 21 Feb 1997 14:20:23 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA16113 for ; Fri, 21 Feb 1997 14:18:50 -0600 Message-Id: <199702212018.OAA16113@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: attendee list for March meeting Date: Fri, 21 Feb 1997 14:18:47 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This the list of people who have told me they are coming to the March MPI Forum meeting. If you are coming and are not on the list, please let me know. Thanks. Rusty Robert Babb Ed Benson Rajesh Bordawekar Peter Brennan Ron Brightwell Ying Chen Albert Cheng Dennis Cottel Raja Daoud Terry Dontje Nathan Doss Mark Fallon Richard Frost Al Geist Bill Gropp Shane Hebert Rolf Hempel Steve Huss-Lederman Steve Landherr Rusty Lusk John May Nick Nevin Jean-Pierre Prost Rolf Rabenseifner Tom Robey Marc Snir David Taylor Rajeev Thakur MaryDell Tholburn Dick Treumann From mpi-core-human@mcs.anl.gov Wed Feb 26 16:26:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA24531; Wed, 26 Feb 1997 16:26:19 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA23745 for mpi-core-out; Wed, 26 Feb 1997 15:24:57 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA23739 for ; Wed, 26 Feb 1997 15:24:52 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id PAA06024; Wed, 26 Feb 1997 15:24:57 -0600 (CST) Date: Wed, 26 Feb 1997 15:24:57 -0600 (CST) Message-Id: <199702262124.PAA06024@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. This is the version that is going to be handed out at the meeting next week. In addition, with this release, I will begin making each chapter available in postscript form. The page numbering of these chapters is the same as the full document. Some chapters have an extra page or two at the end because there are extra pages between chapters in the document. The names for each chapter are: misc-1.2.ps.Z: Miscellaneous 1.2 dynamic-2.ps.Z: Dynamic one-side-2.ps.Z: One-sided collective-2.ps.Z: Collective ei-2.ps.Z: External Interfaces binding-2.ps.Z: Language Binding misc-2.ps.Z: Miscellaneous 2.0 io-2.ps.Z: I/O rt-jod.ps.Z: Realtime spawnindep-jod.ps.Z: JOD for Spawning Independent Processes threads-jod.ps.Z: JOD for Threads and MPI commid-jod.ps.Z: JOD for Communicator ID misc-jod.ps.Z: JOD for Miscellany f90-jod.ps.Z: JOD for Towards a Full Fortran-90 Interface They are available in the same directory as the full report. Steve From mpi-core-human@mcs.anl.gov Wed Feb 26 22:48:28 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA01103; Wed, 26 Feb 1997 22:48:27 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA00722 for mpi-core-out; Wed, 26 Feb 1997 21:48:49 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA00717 for ; Wed, 26 Feb 1997 21:48:44 -0600 Received: from mailhub2.watson.ibm.com (mailhub2.watson.ibm.com [9.2.250.15]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id WAA17616; Wed, 26 Feb 1997 22:49:48 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub2.watson.ibm.com (8.8.2/01-15-97) with SMTP id WAA40445; Wed, 26 Feb 1997 22:48:49 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 8525644B.0014EED6 ; Wed, 26 Feb 1997 22:48:38 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: hart@fsl.noaa.gov cc: mpi-core@mcs.anl.gov Message-ID: <0525644B.00100D60.00@watngi01.watson.ibm.com> Date: Wed, 26 Feb 1997 22:23:29 -0400 Subject: canonical data representation Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk We can postpone dealing with canonical data representation, but we cannot avoid altogether this issue, especially if we want to leave the door open for interoperability. We have three choices 1, Ignore the issue 2. Put hooks in MPI that can be used to generate one or more specific data encodings but postpone the specification for that encoding. 3. Specify a canonical encoding. If we do not feel we can do 3, we should at least specify the MPI syntax for 2, so that we can revisit the issue. I.e., if we cannot agree on a canonical data encoding, we must, to the least, convince ourselves that this can be later retrofited into MPI without requiring new functions. By the way, there is one language that requires a canonical data representation for its basic datatypes -- this is Java. So, if we were doing an MPI binding for Java, the issue of canonical data representation would not arise. Since Java implementations interoperate with C and C interoperates with Fortran this, in principe, defines a "canonical" data representation. It is canonical in the sense that each processor has a well defined mechanism for mapping C or Fortran types to a sequence of bytes. It is not canonical in the sense that there is a fixed encoding for 32 bit integers, but not a fixed encoding for C values of type int. There is a single representation of a 32 bit integer or of a 64 bit integer, but different representations for int according as it is 32 bit or 64 bt long. But this might well be what applications need: I doubt very much that users will want 64 bit real numbers to be transparently truncated to 32 bits. We were looking for a definition of "canonical" were the user will only have to know that a file contains 5 floats, followed by 4 ints, followed by 20 chars. I suggest that we require the user to know that the file contains 5 64 bit floating point numbers, followed by 4 32 bit integers, followed by 20 byte characters, know what is the local size of its datatypes, and declare variables accordingly. From mpi-core-human@mcs.anl.gov Thu Feb 27 07:08:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA27998; Thu, 27 Feb 1997 07:08:02 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id GAA06193 for mpi-core-out; Thu, 27 Feb 1997 06:07:31 -0600 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA06188 for ; Thu, 27 Feb 1997 06:07:27 -0600 Received: by nsgi3.lbl.gov (8.7/ner-1) id EAA04741; Thu, 27 Feb 1997 04:07:35 -0800 (PST) Date: Thu, 27 Feb 1997 04:07:35 -0800 (PST) Message-Id: <199702271207.EAA04741@nsgi3.lbl.gov> From: "W. Saphir" To: mpi-core@mcs.anl.gov Cc: wcs@nsgi3.lbl.gov Subject: new dynamic chapter available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, The version of the dynamic process management chapter that will be handed out and discussed at next week's meeting is now available at: http://www.nersc.gov/research/FTG/wcs/dynamic.2.27.ps I have tried to look at the text very closely and have found a large number of "leftovers" from two years of evolution. Thanks also to a few folks who gave me details comments. Enclosed is a complete list of all the changes made between this version and the previous (2/7). Bill ========================================== - Fixed various small typos and reworded several sentences with awkward grammar or style. - Deleted: A process may or may not be an ``MPI process'' in that it may or may not call \mpifunc{MPI\_INIT} and thus be assigned an \mpifunc{MPI\_COMM\_WORLD}. An \MPI/ process cannot communicate directly with another process unless it has a communicator. because the first sentence is no longer correct and the second was added to clarify issues of signalling and independent processes that no longer exist. - Rewrote (with no change in meaning) a confusing description of maxprocs - There were many many places where the description of erroneous behavior was wrong because the error was described as soft or continuable and/or because specific error behavior was mandated. These all seemed to me to be historical artifacts that have been missed because the document is so familiar. The only questionable language that remains (I hope) is of the form "if situation X, the function returns an error of class MPI_ERR_X." MPI-1 does not specify error class in the text, but in a separate error class section. o Removed mandated error behavior: "If MPI is unable to spawn maxprocs processes, it returns an error ... and a half-empty communicator." We shouldn't specify what is contained in the communicator arg. o Removed same mandated error behavior for soft spawns o Removed soft error for CONNECT if port does not exist and mandated error behavior of setting newcomm to MPI_COMM_NULL o Same for CONNECT time out. o Removed soft error MPI_ERR_NOPUBLISHING o Remove soft error when publishing a name that is already published. Decided the right thing here is to leave behavior undefined rather than say it is erroneous. o Removed soft error when looking up a name that has not been published. o Removed soft error when getting a key that is not defined o There is an absurd situation with MPI_INFO_GET. It has an IN argument valuelen that specified the length of the user-supplied buffer into which the value is placed. If the value is larger than valuelen, the call returns an error (rather than truncating). The whole point of putting in valuelen is to allow the implementation to cleanly detect the error, but since it is an MPI error, the default behavior is to abort and MPI doesn't guarantee recovery even for MPI_ERRORS_RETURN. My current theory is that we should either make it easy on the user and punt valuelen (since it adds no value) or the function should truncate based on valuelen. - In fixing up the examples, deleted the SPMD code that starts itself example because it is very similar to the previous example and because the text recommends against doing this. - Renamed several functions as discussed. We did not come up with new names, so I have supplied the following, which are open to discussion: MPI_NAME_PUBLISH -> MPI_NAMESERVICE_PUBLISH_NAME MPI_NAME_GET -> MPI_NAMESERVICE_LOOKUP_NAME MPI_NAME_DELETE -> MPI_NAMESERVICE_DELETE_NAME I have included "NAMESERVICE" even though it is not an MPI C handle in order to make avoid further top-level namespace pollution and suggest that in C++ it might make sense to have a singleton [what's the right term?] nameservice object. I have kept MPI_SPAWN MPI_SPAWN_MULTIPLE MPI_ACCEPT MPI_CONNECT on the theory that they are similar to send/recv/put/get and that adding "COMM" does not actually clarify. As per the general rule, I've also changed PORT_OPEN -> OPEN_PORT PORT_CLOSE -> CLOSE_PORT In this case it does not make sense to keep PORT_OPEN on the grounds that there is an implicit singleton PORT object (as with NAMESERVICE) because the most natural approach along these lines would have a separate PORT object for each port, which we have rejected. - Changed advice to implementors after MPI_CONNECT as per mail of 2/5. - I was asked to say something about fairness and progress in connection attempts. Since we don't have nonblocking calls I'm not sure it is necessary to talk about progress, but I added a small bit of text on fairness saying there is no guarantee of fairness. - For MPI_ERR_NOPUBLISHING came to the conclusion that it doesn't really make sense to have an attribute on COMM_WORLD. For instance, an implementation may be perfectly capable of publishing, but only if you give it the name of a nameserver host in the INFO argument. Does such an implementation publish names? Decided instead to add some text saying that implementations should document what they do. - I got some feedback that the definition of "connected" was not quite correct. I think it was actually correct (except for the important issue below), but was deductive rather than inductive, and therefore hard to understand. I've changed it slightly. - With or without the above clarification, the text for MPI_COMM_DISCONNECT has some huge (unintended, I think) holes in it. The idea in the introduction is that "it might be desirable for a parent and child to disconnect, so that errors in the child to not affect the parent, and vice-versa". This is fine, but the text that actually describes MPI_COMM_DISCONNECT does not do this. In particular, it does not say specifically that disconnected processes do not affect each other. It says only that connected processes may affect each other. Moreover, if you look closely at the definition of "disconnected" you'll see that an MPI_COMM_FREE technically disconnects processes because it gets rid of the communicators. So I've added two clarifications which I believe are what we intended: 1. changed the definition of "connected" so that MPI_COMM_FREE cannot make processes disconnected 2. added text to say that early termination of a process does not affect independent processes. - added MPI_Join proposal - added some text on SPAWN_START and SPAWN_END From mpi-core-human@mcs.anl.gov Thu Feb 27 09:36:26 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA29770; Thu, 27 Feb 1997 09:36:22 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA08119 for mpi-core-out; Thu, 27 Feb 1997 08:38:35 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA08111 for ; Thu, 27 Feb 1997 08:38:30 -0600 Received: from mailhub2.watson.ibm.com (mailhub2.watson.ibm.com [9.2.250.15]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id JAA13274 for ; Thu, 27 Feb 1997 09:39:36 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub2.watson.ibm.com (8.8.2/01-15-97) with SMTP id JAA42007 for ; Thu, 27 Feb 1997 09:38:38 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 8525644B.00506C22 ; Thu, 27 Feb 1997 09:38:25 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: mpi-core@mcs.anl.gov Message-ID: <8525644B.00504C04.00@watngi01.watson.ibm.com> Date: Thu, 27 Feb 1997 09:37:33 -0400 Subject: Re: canonical data representation Mime-Version: 1.0 Content-type: multipart/mixed; Boundary="0__=MeYILNDb93o4TAfQFFuhjdCY7Za06kng1mNSFc5kqan3DIkcnELuLhSz" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk --0__=MeYILNDb93o4TAfQFFuhjdCY7Za06kng1mNSFc5kqan3DIkcnELuLhSz Content-type: text/plain; charset=US-ASCII Sorry for the mistypo, Jean-Pierre ---------------------- Forwarded by Jean-Pierre Prost/Watson/IBM Research on 02/27/97 09:37 AM --------------------------- From: Jean-Pierre Prost on 02/27/97 09:30 AM To: snir @ watson.ibm.com cc: mi-core @ mcs.anl.gov, mpi-io @ mcs.anl.gov Subject: Re: canonical data representation (Document link not converted) I fully agree with Marc on this issue. We should at least achieve 2. And requiring the user to remember if the numbers present in the file are 64 bit or 32 bit besides the type itself is not unreasonable. Jean-Pierre (Embedded image moved snir @ watson.ibm.com to file: 02/26/97 09:23 PM PIC31079.PCX) To: hart @ fsl.noaa.gov cc: mpi-core @ mcs.anl.gov (bcc: Jean-Pierre Prost/Watson/IBM Research) Subject: canonical data representation We can postpone dealing with canonical data representation, but we cannot avoid altogether this issue, especially if we want to leave the door open for interoperability. We have three choices 1, Ignore the issue 2. Put hooks in MPI that can be used to generate one or more specific data encodings but postpone the specification for that encoding. 3. Specify a canonical encoding. If we do not feel we can do 3, we should at least specify the MPI syntax for 2, so that we can revisit the issue. I.e., if we cannot agree on a canonical data encoding, we must, to the least, convince ourselves that this can be later retrofited into MPI without requiring new functions. By the way, there is one language that requires a canonical data representation for its basic datatypes -- this is Java. So, if we were doing an MPI binding for Java, the issue of canonical data representation would not arise. Since Java implementations interoperate with C and C interoperates with Fortran this, in principe, defines a "canonical" data representation. It is canonical in the sense that each processor has a well defined mechanism for mapping C or Fortran types to a sequence of bytes. It is not canonical in the sense that there is a fixed encoding for 32 bit integers, but not a fixed encoding for C values of type int. There is a single representation of a 32 bit integer or of a 64 bit integer, but different representations for int according as it is 32 bit or 64 bt long. But this might well be what applications need: I doubt very much that users will want 64 bit real numbers to be transparently truncated to 32 bits. We were looking for a definition of "canonical" were the user will only have to know that a file contains 5 floats, followed by 4 ints, followed by 20 chars. I suggest that we require the user to know that the file contains 5 64 bit floating point numbers, followed by 4 32 bit integers, followed by 20 byte characters, know what is the local size of its datatypes, and declare variables accordingly. ########################################## Jean-Pierre Prost IBM T.J. Watson Research Center PO Box 218 Yorktown Heights, NY 10598 USA Phone: (914) 945 3225 - Fax: (914) 945 2141 Lotus Notes: Jean-Pierre Prost @ IBM Research Internet: jpprost@watson.ibm.com URL: http://www.research.ibm.com/people/p/prost/ ########################################## ########################################## Jean-Pierre Prost IBM T.J. Watson Research Center PO Box 218 Yorktown Heights, NY 10598 USA Phone: (914) 945 3225 - Fax: (914) 945 2141 Lotus Notes: Jean-Pierre Prost @ IBM Research Internet: jpprost@watson.ibm.com URL: http://www.research.ibm.com/people/p/prost/ ########################################## --0__=MeYILNDb93o4TAfQFFuhjdCY7Za06kng1mNSFc5kqan3DIkcnELuLhSz Content-type: application/octet-stream; name="PIC31079.PCX" Content-transfer-encoding: base64 CgUBCAAAAABoACwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAABaQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAD1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPH E8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sT zRPHE8MTwhPwEwzIBgzYE8wTxhPDE8IT7hPOBtcTzBPGE8MTE+wTwgbCBwbCEgbCEgbCEsUG1hPL E8YTwxMT6hMMwgYHwgLCAwISwgfEEsMCwwbVE8sTxRPDExPpE8MGAwcCBwMCwhLDB8ISwgISwgLD BtUTyhPFE8MTE+gTwgIHA8ICEw4DDgLDE8USwwLCEMIG1BPKE8UTwxMT5xMCAwcDAg4TDgITwgIS D8ISD8ISBRICEcICwwbUE8oTxRPCExPmEwYCBwMCDgIOwgLDExITEhPCEg8GxgLDBtMMDAfJE8QT whMT5hMGwwITBgMCDhLFEw8SE8ISBgIDwhIDEsMGB9MDxwwHxRPDExPlEwYHAhESAg8CwhMPwhMP xBMPxRIQwgIDAgMCBtMDxwPEDAfDE8IT4RMHwwzCBgLCEhMCDxLIE8MSD8MSwwIQAwIDBgfSDMkD wgPCDAfCExPbEwfGDMIDDAIHERITEhMSwxMPwxMPwxPDEgIDAgMCwwMCBgzREwfHDMYDDMITE9YT B8UMyAMGB8ICBhLDAsYTEhMSExIPwhIHAgcCAwUQAgYRBgfSE8UTB8QMwgMMwhMT0hMHxAzLA8IM BsISDxESExITAw4DxBMSExITwxICBwPCAsMDDMIGB9ITyRMHwwzCExPPEwfDDMkDxQwHwhMGBxIT AhECEwMOAg7DExITDxMPwxIDAgMCBwMCDAYRBgfSE8kTwhPCDMITE8wTB8MMxwPEDMIHxxMGxBLD Ag4DDgIGwg/IEgIDwgIDAgwCEMIGB9ITyRMHDAcMwhMTyhMHwgzGA8MMwgfMEwYHwhLCEAIOAg4C DhDDAhIPxhIFAgXDAgUCEQYH0hPHEwfCDAcPDMITE8gTB8IMxQPDDAfQEwbDEhDEAhAOEA4QwgLG EgcSBhIGBcMCBcIGB9ATB8UMEwfCDA8HDwwHwhMTxhMHwgzEA8MMB9MTBgfCEhADEMICDhAOEMIC EQIDxxIGBwbCAgUCEQYHyxMHxAwHwhMHEwzCEwcPBw8MB8MTE8UTBwzEA8IMB9YTBsQSEAMCA8UC EQIDAgPDEgcSBgfCBgUQAhDCBgfGEwfEDAfGE8INEwzCEw8HwgwHwxPCE8QTBwzDA8IMB9gTBgfE EhACEMYCEQIDAsQSBhLDBsICEALCBgfCEwfDDAfKEwfCDRMHwhPCDAfEE8ITE8MTBwzCA8IMB9oT DBIHwxLDDBEDxQIDAgPDEgYSBgfCBgIQAhAGDAfCEwzDE8MHyRMHwhPCBxMHxRPDExPDEwzCAwwH 3RMGxxICEQPDAgMCA8MSBhIGBwYMBhACEAIGDMMTDBPCB8YTwwfHEwfGE8MTwhPDEwwDDAfeEwYH xxICEQPDAgMCwhIGEgYHBgwGEAIQAsIGB8MTDMYTwwfKEwzGE8MTwhPDE8IMB98TDBLCB8USAgMR xAISB8ISBgcGDAYQBhAGEAYMB8MMB8kTwwfHEwzGE8MTwhPDEwwPwgzfEwYSB8ISB8ISAhECAwID EgcSBwYHBgwGEAYQxgzDD8IHxRPDB8kTBwzGE8MTwhPDEwzDD8QM3BPCBhIGwxIGAhECAwIHBgcG yAzJDxMHzRMHwwwHxxPDE8ITwxMHDMYPxwwH1BMGEgYSBhLLDM4PwwwTDMcTwgfEDAfJE8QTwhMT xBMHwgzLD9sM0w/GDAfDEwzDEwfEDAfLE8YTwxMTxhMHxAztD8gMBgfIE8QMB84TxxPDE8ITyhMH xwzbD8sMEAUMBcIMwgYH1RPKE8UTwxMT0RMH2wwGEAYQBhACBQwFDAUMBgwHBgfWE8sTxRPDExPu EwYMBhAGEAIGDAYMwwYH1xPLE8YTwxMT8BPKBgfYE8wTxhPDExP1E9sTzRPHE8MTwhP1E9sTzRPH E8MTwhMMAAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD/ /wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8A AAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A //8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAA AP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA /wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCk gICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vw oKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw //vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzA psrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDA wNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICA wMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACA AICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACA gACA//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP////// --0__=MeYILNDb93o4TAfQFFuhjdCY7Za06kng1mNSFc5kqan3DIkcnELuLhSz-- From mpi-core-human@mcs.anl.gov Fri Feb 28 14:01:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA29255; Fri, 28 Feb 1997 14:01:07 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA09804 for mpi-core-out; Fri, 28 Feb 1997 12:59:53 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA09798 for ; Fri, 28 Feb 1997 12:59:48 -0600 Message-Id: <199702281859.MAA09798@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: final(?) list of attendees for next week's meeting Date: Fri, 28 Feb 1997 13:00:03 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the list I have now in order to give the hotel a headcount. Let me know if I am missing anyone. -Rusty Robert Babb Ed Benson Rajesh Bordawekar Peter Brennan Ron Brightwell Pang Chen Ying Chen Albert Cheng Lyndon Clarke Dennis Cottel Zhenqian Cui Raja Daoud Judy Devaney Terry Dontje Nathan Doss Mark Fallon Richard Frost Al Geist Bill Gropp John Hagedorn Leslie Hart Shane Hebert Rolf Hempel Hans-Christian Hoppe Steve Huss-Lederman Terry Jones Arkady Kanevsky Susan Kraus Steve Landherr Lloyd Lewins Andrew Lumsdaine Rusty Lusk John May Nick Nevin Bill Nitzberg Jean-Pierre Prost Rolf Rabenseifner Tom Robey Anna Rounbehler Eric Salo Bill Saphir Marc Snir Ian Stockdale David Taylor Rajeev Thakur MaryDell Tholburn Dick Treumann Parkson Wong From mpi-core-human@mcs.anl.gov Fri Feb 28 18:50:46 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA03154; Fri, 28 Feb 1997 18:50:45 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA17568 for mpi-core-out; Fri, 28 Feb 1997 17:49:54 -0600 Received: from ncsa.uiuc.edu (sdgmail.ncsa.uiuc.edu [141.142.103.66]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA17435; Fri, 28 Feb 1997 17:44:41 -0600 Received: from narnia.ncsa.uiuc.edu (narnia.ncsa.uiuc.edu [141.142.21.165]) by ncsa.uiuc.edu (8.8.5/8.8.5) with SMTP id RAA14064; Fri, 28 Feb 1997 17:44:49 -0600 (CST) Message-Id: <3.0.32.19970228174835.0069e3d4@sdgmail.ncsa.uiuc.edu> X-Sender: acheng@sdgmail.ncsa.uiuc.edu X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Fri, 28 Feb 1997 17:48:35 -0600 To: mpi-io@mcs.anl.gov From: Albert Cheng Subject: Re: canonical data representation Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I also agree at least getting the hooks in. But I am not sure if I follow the later part. Currently, MPI types do not say anything about the sizes of the data. So, an application that writes 1 MPI_INT, 1 MPI_LONG may end up with a file of 32 bits integer, 32 bit integer (32 bit workstation) 32 bits integer, 64 bit integer (64 bit SGI) 64 bits integer, 64 bit integer (64 bit Cray) Another application, e.g., a visualization tool, knows that it needs to read 1 MPI_INT, 1 MPI_LONG and if it has to accept MPI files created in various machine types, it is impossible to program it using mere MPIO functions without interactive input asking "what kind of file you give me this time?" The read data segment looks rather complicated--it has to figure out how many bytes to bring in, sign-extension or not, little or big endian, swab-byte or not. The reader application basically has to read MPI-data in by bytes, then start doing byte movement here and there. That would require an experience programmer who knows all the specific of each machine out there. If we can have certain external data representation that defines 1 MPI_INT is stored as a 32-bit-big-endian integer, 1 MPI_LONG is stored as a 64-bit-big-endian integer, etc. Then the MPIO implementation can do all the above bytes movements and conversion for the users. The trade-off is potential loss of precision if the machine involved is not a good match with the defined data representation. If the precision loss is unacceptable to a certain application, it can choose to use the NATIVE-mode and never move the data files to a different platform or it can write data out in MPI_BYTE and program their own conversion. The defined external data representation just makes life easier for the subset of MPIO applications that can tolerate the potential precision loss. At 09:30 AM 2/27/97 -0400, Jean-Pierre Prost wrote: > >I fully agree with Marc on this issue. We should at least >achieve 2. >And requiring the user to remember if the numbers present in the >file are 64 bit or 32 bit besides the type itself is not unreasonable. >Jean-Pierre > >From: "Marc Snir" >... >But this might well be what applications need: I doubt very much that users >will want 64 bit real numbers to be transparently truncated to 32 bits. >We were looking for a definition of "canonical" were the user will only >have to know that a file contains 5 floats, followed by 4 ints, followed by >20 chars. I suggest that we require the user to know that the file >contains 5 64 bit floating point numbers, followed by 4 32 bit integers, >followed by 20 byte characters, know what is the local size of its >datatypes, and declare variables accordingly. From mpi-core-human@mcs.anl.gov Sun Mar 2 16:09:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA12046; Sun, 2 Mar 1997 16:08:59 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA15550 for mpi-core-out; Sun, 2 Mar 1997 15:09:16 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA15544 for ; Sun, 2 Mar 1997 15:09:12 -0600 Received: from mailhub2.watson.ibm.com (mailhub2.watson.ibm.com [9.2.250.15]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id QAA17954; Sun, 2 Mar 1997 16:10:36 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub2.watson.ibm.com (8.8.2/01-15-97) with SMTP id QAA18678; Sun, 2 Mar 1997 16:09:36 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 8525644E.007433DF ; Sun, 2 Mar 1997 16:09:14 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: acheng@ncsa.uiuc.edu cc: mpi-core@mcs.anl.gov Message-ID: <8525644E.0071C836.00@watngi01.watson.ibm.com> Date: Sun, 2 Mar 1997 16:02:17 -0400 Subject: Re: canonical data representation Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My take is that a program that generates data that is later used by a visualization package, is responsible for generating data of the "right size": e.g. 32 bit integers or 64 bit floats, if that's what the visualization package wants. The canonical representation will specify a unique bit representation for a 32 bit integer (say, 2-complement, little endian), so that users do not have to worry about data representation, beyond the size of each datum. This leaves room for two possible interfacing mechanisms: 1. Visualisation package specifies the format it expects for data; the client program has to dump data in files in this format. Since the only issue is whether to use int or long or long long, float or double, the programming effort required from the user is quite small. Code is not portable, but the system dependent part can be encapsulated in a few datatype definitions. 2. Visualization package is interfaced by a client library. The library provides one portable interface (say, accepts arguments of type float). The data conversion is dealt with inside the library. The user code is fully portable. The library code is not portable, but the system dependnet part can be encapsulated in a few datatype definitions. I suggest not to go beyond this level of portability since, in many instances, users will not want 64 bit values to be transparently truncated to 32 bit values. From mpi-core-human@mcs.anl.gov Wed Mar 5 10:21:25 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04275; Wed, 5 Mar 1997 10:21:22 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA18258 for mpi-core-out; Wed, 5 Mar 1997 09:18:43 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA18253; Wed, 5 Mar 1997 09:18:38 -0600 Message-Id: <199703051518.JAA18253@antares.mcs.anl.gov> To: "Marc Snir" cc: acheng@ncsa.uiuc.edu, mpi-core@mcs.anl.gov Subject: Re: canonical data representation In-reply-to: Your message of "Sun, 02 Mar 1997 16:02:17 -0400." <8525644E.0071C836.00@watngi01.watson.ibm.com> Date: Wed, 05 Mar 1997 09:19:20 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | | 1. Visualisation package specifies the format it expects for data; the | client program has to dump data in files in this format. Since the only | issue is whether to use int or long or long long, float or double, the | programming effort required from the user is quite small. Code is not | portable, but the system dependent part can be encapsulated in a few | datatype definitions. A group of potential MPI-IO users that Jean-Pierre and I recently talked to in the ASCI program want the ability for MPI-IO to be told the name of a format for the data to be dumped in, precisely for this purpose (visualization). The idea is that a program that dumps data in a format expected by the vusualization program could then be portable. It would be a generalization of the three options we currently have when the file is opened. From mpi-core-human@mcs.anl.gov Fri Mar 7 13:02:51 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA17679; Fri, 7 Mar 1997 13:02:50 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA15257 for mpi-core-out; Fri, 7 Mar 1997 12:02:15 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA15252; Fri, 7 Mar 1997 12:02:02 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id KAA02658; Fri, 7 Mar 1997 10:02:18 -0800 (PST) Message-ID: Date: 7 Mar 1997 10:00:26 -0800 From: "Lewins, Lloyd J" Subject: Semantics of MPI Cancel? To: "'mpi-core@mcs.anl.gov'" , "mpi-misc" X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk What is the correct behaviour of the following program? Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Isend (...,1,...) MPI_Cancel () MPI_Wait () MPI_Test_cancelled () MPI_Barrier () MPI_Barrier () MPI_Probe () Must test cancelled return true? Must probe return false? I believe test cancelled may return either true of false. If it returns true, probe returns false. If it returns false, probe returns true. Consider an implementation which uses an eager send protocol for short messages. The Isend may be complete by the time the cancel call occurs. I don't think the intention is that cancel is actually "unsend" in this case. Rationale: The standard says that cancel "marks for cancellation a PENDING, nonblocking communication operation." (my emphasis). In the above (eager) case, the send is no longer pending, it is complete (but not yet completed). The meat of the definition of cancel is then to define that a subsequent wait is local, not that a probe after the cancel is guaranteed to return false. I would argue that if we wanted a way to "unsend", we would have this functionality for blocking sends as well. A related issue, consider: Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Isend (...,1,...) MPI_Probe () MPI_Cancel () MPI_Wait () MPI_Test_cancelled () MPI_Barrier () MPI_Barrier () MPI_Recv () Assuming the probe returns true, should the cancel go ahead or not? If it does, then the standard is violated with regards to the recv (a probe showed a message exists, but the receive doesn't get it). If it doesn't, the standard is violated with regards to the cancel (the wait is not local). Catch 22. I believe that the send should be canceled, and that we need to add clarifying text to probe (the recv gets the message, assuming no other recv has grabbed it AND the send was not successfully canceled). Comments? Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com From mpi-core-human@mcs.anl.gov Sat Mar 8 05:32:53 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA26778; Sat, 8 Mar 1997 05:32:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA02235 for mpi-core-out; Sat, 8 Mar 1997 04:33:19 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA02230 for ; Sat, 8 Mar 1997 04:33:11 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id CAA06457 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Sat, 8 Mar 1997 02:34:08 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id CAA07350; Sat, 8 Mar 1997 02:31:58 -0800 From: "Eric Salo" Message-Id: <9703080231.ZM7348@mrjones.engr.sgi.com> Date: Sat, 8 Mar 1997 02:31:58 -0800 In-Reply-To: "Lewins, Lloyd J" "Semantics of MPI Cancel?" (Mar 7, 10:00am) References: X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: Semantics of MPI Cancel? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > What is the correct behaviour of the following program? Good grief! Is this a real problem for a code, or are you just being evil??? > Process 0 Process 1 > ------------- -------------- > MPI_Init () MPI_Init () > MPI_Isend (...,1,...) > MPI_Cancel () > MPI_Wait () > MPI_Test_cancelled () > MPI_Barrier () MPI_Barrier () > MPI_Probe () > > Must test cancelled return true? Must probe return false? My belief is that if we go with a strict interpretation of the standard, test_cancelled must return true and probe must return false. Meaning, that for 100% "correctness" an UNSEND message is needed here. Which is a big chunk of why, after more than two years now, I still haven't bothered to implement cancel for sends... > The standard says that cancel "marks for cancellation a PENDING, nonblocking > communication operation." (my emphasis). In the above (eager) case, the send > is no longer pending, it is complete (but not yet completed). The meat of the > definition of cancel is then to define that a subsequent wait is local, not > that a probe after the cancel is guaranteed to return false. Well, the standard says that either the cancel must succeed or the request must be guaranteed to complete. But until a matching receive is posted, there is no way to ensure that this completion will ever occur. I think you are using the common (and more useful) defintion of "complete" rather than the official one. > I would argue that if we wanted a way to "unsend", we would have this > functionality for blocking sends as well. > > A related issue, consider: > > Process 0 Process 1 > ------------- -------------- > MPI_Init () MPI_Init () > MPI_Isend (...,1,...) > MPI_Probe () > MPI_Cancel () > MPI_Wait () > MPI_Test_cancelled () > MPI_Barrier () MPI_Barrier () > MPI_Recv () You are a twisted man. Please seek professional help before it's too late. -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Sat Mar 8 19:22:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA02891; Sat, 8 Mar 1997 19:22:54 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA10817 for mpi-core-out; Sat, 8 Mar 1997 18:23:32 -0600 Received: from palrel3.hp.com (palrel3.hp.com [15.253.88.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA10811 for ; Sat, 8 Mar 1997 18:23:20 -0600 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel3.hp.com with ESMTP (8.7.5/8.7.3) id QAA28298 for ; Sat, 8 Mar 1997 16:24:20 -0800 (PST) Message-Id: <199703090024.QAA28298@palrel3.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA048577059; Sat, 8 Mar 1997 18:24:19 -0600 From: Raja Daoud Subject: Re: Semantics of MPI Cancel? To: mpi-core@mcs.anl.gov Date: Sat, 08 Mar 1997 18:24:18 CST In-Reply-To: <9703080231.ZM7348@mrjones.engr.sgi.com>; from "Eric Salo" at Mar 8, 97 2:31 am X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Lloyd, in both cases cancel must succeed and the message destroyed. Eager protocol or not, is an implementation detail. The user doesn't (and shouldn't) distinguish between the two. Cancelling sends requires a non-deadlocking (stateless) RPC mechanism and full access to the list of unreceived messages at the destination. Your posting got me thinking about the following example: rank 0 rank 1 =============================================== MPI_Init MPI_Init MPI_Isend(tag = 5) MPI_Probe(tag = 6) MPI_Cancel MPI_Finalize MPI_Wait exit MPI_Test_cancelled MPI_Finalize exit In each process, Finalize is called after all pending (non-completed) communication that the process knows about is terminated. But the probe may (most probably) cause the eager short-message to be pulled out of the communication medium (shmem, switch, socket) and buffered in the process' local memory. If Finalize in rank 1 doesn't delay returning, then cancel cannot complete (i.e. isn't "local"). Comments? Cancel is a very tricky beast. We have it implemented (finally) but I'm sure there are a few corner cases we still haven't thought about. Bsend's ugliness pales in comparison (one tiny plus-point for the Bsend lovers :-) ). [editorial comment] This may give a hint why I think actual implementors ought to be proud of their aggressive and unrelenting resistance to similar requirements in the MPI Forum: on the surface they appear easy and logical and get the support of unsuspecting users, in reality they're horrible, and get worse the more optimized the implementation is. So kudos all around to the fightin' implementors! Carry on, resistance is not futile! :-) --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/wsg/ssa/mpi/mpihome.html From mpi-core-human@mcs.anl.gov Sat Mar 8 20:23:51 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA03058; Sat, 8 Mar 1997 20:23:50 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA11470 for mpi-core-out; Sat, 8 Mar 1997 19:24:59 -0600 Received: from palrel1.hp.com (palrel1.hp.com [15.253.72.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA11465 for ; Sat, 8 Mar 1997 19:24:53 -0600 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com with ESMTP (8.7.5/8.7.3) id RAA00801 for ; Sat, 8 Mar 1997 17:25:46 -0800 (PST) Message-Id: <199703090125.RAA00801@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA050090745; Sat, 8 Mar 1997 19:25:45 -0600 From: Raja Daoud Subject: Re: Semantics of MPI Cancel? To: mpi-core@mcs.anl.gov Date: Sat, 08 Mar 1997 19:25:45 CST In-Reply-To: <199703090024.QAA28298@palrel3.hp.com>; from "Raja Daoud" at Mar 08, 97 6:24 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Correction: I meant Iprobe instead of Probe. > rank 0 rank 1 > =============================================== > MPI_Init MPI_Init > MPI_Isend(tag = 5) MPI_Probe(tag = 6) ^^^^^^^^^MPI_Iprobe > MPI_Cancel MPI_Finalize > MPI_Wait exit > MPI_Test_cancelled > MPI_Finalize > exit We may need to modify the Finalize clarification to make the meaning of "completed" more tight and disallow this case. --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/wsg/ssa/mpi/mpihome.html From mpi-core-human@mcs.anl.gov Sun Mar 9 21:31:31 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA10580; Sun, 9 Mar 1997 21:31:30 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA25588 for mpi-core-out; Sun, 9 Mar 1997 20:31:50 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA25569; Sun, 9 Mar 1997 20:30:51 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id VAA09584; Sun, 9 Mar 1997 21:21:54 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id VAA50538; Sun, 9 Mar 1997 21:31:51 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256456.000DE2E5 ; Sun, 9 Mar 1997 21:31:40 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: llewins@msmail4.hac.com cc: Mpi-Core@mcs.anl.gov, Mpi-Misc@mcs.anl.gov Message-ID: <85256456.000BB996.00@watngi01.watson.ibm.com> Date: Sun, 9 Mar 1997 21:24:53 -0400 Subject: Re: Semantics of MPI Cancel? Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk l What is the correct behaviour of the following program? Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Isend (...,1,...) MPI_Cancel () MPI_Wait () MPI_Test_cancelled () MPI_Barrier () MPI_Barrier () MPI_Probe () Must test cancelled return true? Must probe return false? I believe test cancelled may return either true of false. If it returns true, probe returns false. If it returns false, probe returns true. **** Since there is no receive to match the send, the cancel must succeed, and test_cancelled must return true. It clearly will not do to tell the sender that the send was cancelled, if it might be received at the other end. It will not do either to refuse to cancel, if there might be no matching receive, as in this example. This would negate the usefulness of cancel if, at some arbitrary time after the send was posted, the send becomes uncancellable, even if no matching receive will ever be posted. I would prefer to say that probe must return false, in this case. But this probably points out to something we miss in the spec. Namely, that if a user probe for an incoming messagfe, then it is erroneous not to receive it -- we shaould add this as a clarification to MPI-1. The reason is that we do not want to cancel a message after it has been probed, so it must be received. So, I propose the following clarification: "A message that has been matched by a call to MPI-PROBE must be received by a subsequent call to MPI_RECV or MPI_IRECV." **** Consider an implementation which uses an eager send protocol for short messages. The Isend may be complete by the time the cancel call occurs. I don't think the intention is that cancel is actually "unsend" in this case. *** The Isend is complete at the sender side and, yes, the cancel may require a communication with the receiving side to "undo" the send. This much has been clear since cancel was defined by the MPI forum. Indeed, this is the precise reason why many implementors were against the cancel function in MPI1. **** Rationale: The standard says that cancel "marks for cancellation a PENDING, nonblocking communication operation." (my emphasis). In the above (eager) case, the send is no longer pending, it is complete (but not yet completed). The meat of the definition of cancel is then to define that a subsequent wait is local, not that a probe after the cancel is guaranteed to return false. **** The send is pending until it has completed at both ends. *** I would argue that if we wanted a way to "unsend", we would have this functionality for blocking sends as well. *** You have no "request" to represent the operation, in the blocking case, so that you cannot have a cancel. Otherwise, yes, it makes sense for blocking sends, too. ****** A related issue, consider: Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Isend (...,1,...) MPI_Probe () MPI_Cancel () MPI_Wait () MPI_Test_cancelled () MPI_Barrier () MPI_Barrier () MPI_Recv () Assuming the probe returns true, should the cancel go ahead or not? If it does, then the standard is violated with regards to the recv (a probe showed a message exists, but the receive doesn't get it). If it doesn't, the standard is violated with regards to the cancel (the wait is not local). Catch 22. *** See clarification above. ******* I believe that the send should be canceled, and that we need to add clarifying text to probe (the recv gets the message, assuming no other recv has grabbed it AND the send was not successfully canceled). Comments? Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com **** Marc ******* From mpi-core-human@mcs.anl.gov Sun Mar 9 21:55:38 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA10727; Sun, 9 Mar 1997 21:55:37 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA25859 for mpi-core-out; Sun, 9 Mar 1997 20:54:27 -0600 Received: from hitiij.hitachi.co.jp (root@hitiij.hitachi.co.jp [133.145.224.3]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA25853 for ; Sun, 9 Mar 1997 20:54:17 -0600 Received: from hcrlgw92.crl.hitachi.co.jp by hitiij.hitachi.co.jp (8.8.5+2.7Wbeta5/3.5W-hitiij) id LAA00922; Mon, 10 Mar 1997 11:54:52 +0900 (JST) Received: from hcrlgw2.crl.hitachi.co.jp by hcrlgw92.crl.hitachi.co.jp (4.1/6.4J.6) id AA10219; Mon, 10 Mar 97 11:55:17 JST Received: from [133.144.18.33] by hcrlgw2.crl.hitachi.co.jp (5.61/6.4J.6) id AA05124; Mon, 10 Mar 97 11:55:14 +0900 Message-Id: <33237828.3283@crl.hitachi.co.jp> Date: Mon, 10 Mar 1997 11:55:36 +0900 From: Nobutoshi Sagawa Reply-To: sagawa@crl.hitachi.co.jp Organization: Hitachi Central Research Lab. X-Mailer: Mozilla 3.01 (Win95; I) Mime-Version: 1.0 To: "Lewins, Lloyd J" Cc: "'mpi-core@mcs.anl.gov'" Subject: Re: Semantics of MPI Cancel? References: Content-Type: text/plain; charset=iso-2022-jp Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Lewins, Lloyd J wrote: > > Process 0 Process 1 > ------------- -------------- > MPI_Init () MPI_Init () > MPI_Isend (...,1,...) > MPI_Probe () > MPI_Cancel () > MPI_Wait () > MPI_Test_cancelled () > MPI_Barrier () MPI_Barrier () > MPI_Recv () > > Assuming the probe returns true, should the cancel go ahead or not? If it > does, then the standard is violated with regards to the recv (a probe showed a > message exists, but the receive doesn't get it). If it doesn't, > the standard is violated with regards to the cancel (the wait is not > local). Catch 22. > > I believe that the send should be canceled, and that we need to add clarifying > text to probe (the recv gets the message, assuming no other recv has grabbed > it AND the send was not successfully canceled). > It would seem to me that the standard is NOT saying "a probed message must be received by the immediately following MPI_Recv". It merely says "MPI_Probe must give the same result as if the corresponding MPI_Recv were issued at the same point". If I understand the definition correctly, it should be OK that the MPI_Probe() returns true, AND the MPI_Recv() still fails to receive the message. Besides, the MPI_Probe() may either return true or block in the above example, depending on the order the cancel and probe are actually processed. I appreciate if anybody could correct me if I am wrong, because we are under a struggle to decide how to implement MPI_Cancel on our machine. --------------------------------------------------------------- Nobutoshi Sagawa Processor System Department Central Research Laboratory, Hitachi Ltd. tel: +81-423-23-1111 ext 3850 fax: +81-423-27-7743 --------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Sun Mar 9 23:00:18 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA11412; Sun, 9 Mar 1997 23:00:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA26525 for mpi-core-out; Sun, 9 Mar 1997 21:58:55 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA26520 for ; Sun, 9 Mar 1997 21:58:51 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id WAA08412 for ; Sun, 9 Mar 1997 22:49:56 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id WAA35261 for ; Sun, 9 Mar 1997 22:59:58 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256456.0015F497 ; Sun, 9 Mar 1997 22:59:48 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: mpi-core@mcs.anl.gov Message-ID: <85256456.0015B444.00@watngi01.watson.ibm.com> Date: Sun, 9 Mar 1997 22:58:24 -0400 Subject: Re: Semantics of MPI Cancel? Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Tony complains he got a blank message -- so here I try again What is the correct behaviour of the following program? Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Isend (...,1,...) MPI_Cancel () MPI_Wait () MPI_Test_cancelled () MPI_Barrier () MPI_Barrier () MPI_Probe () Must test cancelled return true? Must probe return false? I believe test cancelled may return either true of false. If it returns true, probe returns false. If it returns false, probe returns true. **** Since there is no receive to match the send, the cancel must succeed, and test_cancelled must return true. It clearly will not do to tell the sender that the send was cancelled, if it might be received at the other end. It will not do either to refuse to cancel, if there might be no matching receive, as in this example. This would negate the usefulness of cancel if, at some arbitrary time after the send was posted, the send becomes uncancellable, even if no matching receive will ever be posted. I would prefer to say that probe must return false, in this case. But this probably points out to something we miss in the spec. Namely, that if a user probe for an incoming messagfe, then it is erroneous not to receive it -- we shaould add this as a clarification to MPI-1. The reason is that we do not want to cancel a message after it has been probed, so it must be received. So, I propose the following clarification: "A message that has been matched by a call to MPI-PROBE must be received by a subsequent call to MPI_RECV or MPI_IRECV." **** Consider an implementation which uses an eager send protocol for short messages. The Isend may be complete by the time the cancel call occurs. I don't think the intention is that cancel is actually "unsend" in this case. *** The Isend is complete at the sender side and, yes, the cancel may require a communication with the receiving side to "undo" the send. This much has been clear since cancel was defined by the MPI forum. Indeed, this is the precise reason why many implementors were against the cancel function in MPI1. **** Rationale: The standard says that cancel "marks for cancellation a PENDING, nonblocking communication operation." (my emphasis). In the above (eager) case, the send is no longer pending, it is complete (but not yet completed). The meat of the definition of cancel is then to define that a subsequent wait is local, not that a probe after the cancel is guaranteed to return false. **** The send is pending until it has completed at both ends. *** I would argue that if we wanted a way to "unsend", we would have this functionality for blocking sends as well. *** You have no "request" to represent the operation, in the blocking case, so that you cannot have a cancel. Otherwise, yes, it makes sense for blocking sends, too. ****** A related issue, consider: Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Isend (...,1,...) MPI_Probe () MPI_Cancel () MPI_Wait () MPI_Test_cancelled () MPI_Barrier () MPI_Barrier () MPI_Recv () Assuming the probe returns true, should the cancel go ahead or not? If it does, then the standard is violated with regards to the recv (a probe showed a message exists, but the receive doesn't get it). If it doesn't, the standard is violated with regards to the cancel (the wait is not local). Catch 22. *** See clarification above. ******* I believe that the send should be canceled, and that we need to add clarifying text to probe (the recv gets the message, assuming no other recv has grabbed it AND the send was not successfully canceled). Comments? Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com **** Marc ******* From mpi-core-human@mcs.anl.gov Sun Mar 9 23:56:51 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA11708; Sun, 9 Mar 1997 23:56:49 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA27099 for mpi-core-out; Sun, 9 Mar 1997 22:57:54 -0600 Received: from palrel3.hp.com (palrel3.hp.com [15.253.88.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA27083; Sun, 9 Mar 1997 22:57:38 -0600 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel3.hp.com with ESMTP (8.7.5/8.7.3) id UAA26483; Sun, 9 Mar 1997 20:58:44 -0800 (PST) Message-Id: <199703100458.UAA26483@palrel3.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA064419917; Sun, 9 Mar 1997 22:58:37 -0600 From: Raja Daoud Subject: Re: Semantics of MPI Cancel? To: snir@watson.ibm.com (Marc Snir) Date: Sun, 09 Mar 1997 22:58:37 CST Cc: llewins@msmail4.hac.com, Mpi-Core@mcs.anl.gov, Mpi-Misc@mcs.anl.gov In-Reply-To: <85256456.000BB996.00@watngi01.watson.ibm.com>; from "Marc Snir" at Mar 9, 97 9:24 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > So, I propose the following clarification: "A message that has been > matched by a call to MPI-PROBE must be received by a subsequent call to > MPI_RECV or MPI_IRECV." Marc, in the example I presented, I made the Iprobe() call ask for a different tag than the message's. This is to make sure that we cannot use such an escape hatch by saying that the user knew the message was there. This forces the correct implementation to delay Finalize until it is sure that all of the source processes of its unreceived messages have called Finalize themselves (i.e. told it that they won't be cancelling any of these messages). Of course, a barrier in Finalize would also do the trick, but possibly require more communication than strictly needed (if Finalize is to be kept as lightweight as possible). In a strict sense, Finalize must do a Buffer_detach, then send to the unreceived messages' source processes "may-I-exit?" requests, reply to any such requests it may have received, and then wait for all the "cleared-for-exit" ACKs to come back. The other option would be to make the user not call Finalize in any process until all pending communications application-wide are completed. This forces the user to make a global decision, not just one based on each process' own incomplete requests. In practice, I expect such users to punt and just add a barrier before Finalize (like some public test suites codes do). If possible, I still prefer to reduce the complexity of the Finalize clarification for users, and just tell the implementors to add the required synchronization inside Finalize. Unless somebody wants to optimize Finalize, my guess is that implementations will just add a barrier inside Finalize, in lieu of the Buffer_detach/send-ACKs/Waitall. In this scenario, the clarification becomes an advice to implementors, pointing out that Finalize must cleanup the loose-ends caused by the asynchronous behaviour of Bsend and Cancel. Users can skip it and keep doing what they have been doing. --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/wsg/ssa/mpi/mpihome.html From mpi-core-human@mcs.anl.gov Mon Mar 10 10:56:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA17299; Mon, 10 Mar 1997 10:56:48 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA05498 for mpi-core-out; Mon, 10 Mar 1997 09:56:21 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA05493 for ; Mon, 10 Mar 1997 09:56:16 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id JAA00245; Mon, 10 Mar 1997 09:57:27 -0600 (CST) Date: Mon, 10 Mar 1997 09:57:27 -0600 (CST) Message-Id: <199703101557.JAA00245@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document credits & T-shirts Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am in the active process of trying to finalize the MPI document. Attached is the current list of acknowledgements in the document. I have tried my best to find all the names I could. However, I am sure that some names are missing or incorrectly listed. If you have attended at least one MPI Forum meeting and would like to be listed and are missing, please drop me a note and I will add you to the document. Please take a minute now to check your entry. Also, recall that the effort to get a MPI T-shirt is going on. We hope to get a design by the end of the month. Right now we don't have any. If you have ideas, send them to me (ASCII, postscript) and I will make them available on the web. Unless someone comes up with an idea we won't have T-shirts. Thanks, Steve ---------------------------------------------------------------------- Rusty Lusk, Convener and Meeting Chair Steve Huss-Lederman, Editor Bill Saphir, Process Creation and Management Marc Snir, One-Sided Communications Bill Gropp, Extended Collective Operations Steve Huss-Lederman, External Interfaces Andrew Lumsdaine and Jeff Squyres, Language Bindings Rusty Lusk, Miscellany Bill Nitzberg, I/O Anthony Skjellum and Arkady Kanevsky, Real-Time MPI The following list includes some of the active participants who attended \MPIII/ Forum meetings and are not mentioned above. Greg Astfalk Robert Babb Rajesh Bordawekar Pete Bradley Ron Brightwell Maciej Brodowicz Eric Brunner Greg Burns Margaret Cahir Pang Chen Ying Chen Albert Cheng Yong Cho Joel Clark Lyndon Clarke Laurie Costello Dennis Cottel Jim Cownie Raja Daoud David DiNucci Doug Doefler Jack Dongarra Terry Dontje Nathan Doss Anne Elster Mark Fallon Sam Fineberg Stephen Fleischman Ian Foster Hubertus Franke Richard Frost Al Geist Robert George David Greenberg Kei Harada Leslie Hart Shane Hebert Rolf Hempel Tom Henderson Alex Ho Hans-Christian Hoppe Joefon Jann Suresh Damondovan Kamal Karl Kesselman Koichi Konishi Susan Kraus Steve Landherr Mario Lauria Mark Law Juan Leon Lloyd Lewins Ziyang Lu Peter Madams John May Oliver McBryan Brian McCandless Thom McMahon Harish Nag Nick Nevin Jarek Nieplocha Ron Oldfield Steve Otto Pratap Pattnaik Peter Pacheco Yoonho Park Perry Partow Elsie Pierce Paul Pierce Heidi Poxon JeanPierre Prost James Pruyve Rolf Rabenseifner Joe Rieken Peter Rigsbee Anna Rounbehler Nobutoshi Sagawa Arindam Saha Eric Salo Eric Sharakan Andrew Sherman Fred Shirley Lance Shuler Anthony Skjellum A. Gordon Smith Jeff Squyres David Taylor Steven Taylor Greg Tensa Rajeev Thakur Marydell Tholburn Dick Treumann Simon Tsang Manuel Ujaldon David Walker Jerrell Watts Klaus Wolf Parkson Wong Dave Wright From mpi-core-human@mcs.anl.gov Mon Mar 10 11:58:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA18110; Mon, 10 Mar 1997 11:58:58 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA07666 for mpi-core-out; Mon, 10 Mar 1997 10:56:49 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA07647; Mon, 10 Mar 1997 10:56:37 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id IAA26692; Mon, 10 Mar 1997 08:57:06 -0800 (PST) Message-ID: Date: 10 Mar 1997 08:53:07 -0800 From: "Lewins, Lloyd J" Subject: RE: Semantics of MPI Cancel? To: "mpi-core" , "mpi-misc" X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >Lewins, Lloyd J wrote: > > Process 0 Process 1 > ------------- -------------- > MPI_Init () MPI_Init () > MPI_Isend (...,1,...) > MPI_Probe () > MPI_Cancel () > MPI_Wait () > MPI_Test_cancelled () > MPI_Barrier () MPI_Barrier () > MPI_Recv () > > Assuming the probe returns true, should the cancel go ahead or not? If it > does, then the standard is violated with regards to the recv (a probe showed a > message exists, but the receive doesn't get it). If it doesn't, > the standard is violated with regards to the cancel (the wait is not > local). Catch 22. > > I believe that the send should be canceled, and that we need to add clarifying > text to probe (the recv gets the message, assuming no other recv has grabbed > it AND the send was not successfully canceled). > Nobutoshi Sagawa wrote: It would seem to me that the standard is NOT saying "a probed message must be received by the immediately following MPI_Recv". It merely says "MPI_Probe must give the same result as if the corresponding MPI_Recv were issued at the same point". If I understand the definition correctly, it should be OK that the MPI_Probe() returns true, AND the MPI_Recv() still fails to receive the message. ******* LJL Start I disagree. Page 52, line 1 appears to make it clear that a probed message is guaranteed to be received by a following receive (and by implication that the following receive is guaranteed to be local after a successful probe). ******* LJL End Besides, the MPI_Probe() may either return true or block in the above example, depending on the order the cancel and probe are actually processed. ******** LJL Start True, I am only arguing about what should happen IF the probe returns true. If it blocks, the problem goes away. ******* LJL End I appreciate if anybody could correct me if I am wrong, because we are under a struggle to decide how to implement MPI_Cancel on our machine. ******* LJL Start Exactly what I am trying to do to. ******* LJL End Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com From mpi-core-human@mcs.anl.gov Mon Mar 10 11:58:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA18111; Mon, 10 Mar 1997 11:58:58 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA07665 for mpi-core-out; Mon, 10 Mar 1997 10:56:48 -0600 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA07623; Mon, 10 Mar 1997 10:56:24 -0600 Received: by nsgi3.lbl.gov (8.7/ner-1) id IAA00702; Mon, 10 Mar 1997 08:57:33 -0800 (PST) From: " W. Saphir" Message-Id: <9703100857.ZM700@nsgi3.lbl.gov> Date: Mon, 10 Mar 1997 08:57:32 -0800 In-Reply-To: Raja Daoud "Re: Semantics of MPI Cancel?" (Mar 9, 10:58pm) References: <199703100458.UAA26483@palrel3.hp.com> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: Mpi-Misc@mcs.anl.gov Subject: Re: Semantics of MPI Cancel? Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In some sense, this discussion is moot. From a practical point of view, MPI_Cancel (for sends) is not a part of MPI. Because it is extremely difficult to implement, it doesn't not work in many implementations and users can't count on it. Moreover, I have yet to think of a good reason why an application might want to cancel a send. When I teach classes in MPI, I tell users not to use it. If a user were to come to me complaining that MPI was broken because cancel doesn't work (fortunately none have, possibly because they've heard my tutorials) I would suggest they rewrite their program rather than complain to the vendor. I think it's already unfortunate that vendors have had to put significant resources into dealing with it. The point of this is I think the Forum has more pressing things to worry about at the moment than worrying about cancel (especially, worrying about the absurd examples that were given). We should devote our energy to careful readings of the MPI-2 document so that we can avoid this type of problem in the future. Bill From mpi-core-human@mcs.anl.gov Mon Mar 10 12:56:32 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA19056; Mon, 10 Mar 1997 12:56:31 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA09119 for mpi-core-out; Mon, 10 Mar 1997 11:54:31 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA09114; Mon, 10 Mar 1997 11:54:26 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id JAA04248; Mon, 10 Mar 1997 09:54:51 -0800 (PST) Message-ID: Date: 10 Mar 1997 09:51:25 -0800 From: "Lewins, Lloyd J" Subject: RE: Semantics of MPI Cancel? To: "Marc Snir" Cc: Mpi-Core@mcs.anl.gov, Mpi-Misc@mcs.anl.gov X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk **** Marc Since there is no receive to match the send, the cancel must succeed, and test_cancelled must return true. It clearly will not do to tell the sender that the send was cancelled, if it might be received at the other end. It will not do either to refuse to cancel, if there might be no matching receive, as in this example. This would negate the usefulness of cancel if, at some arbitrary time after the send was posted, the send becomes uncancellable, even if no matching receive will ever be posted. ****** LJL Start I disagree, it depends how you define the semantics of cancel. Looking at page 51, line 20-21, "Posting a send or a receive ties up USER resources (send or receive buffers), and a cancel may be need to free these resources gracefully." So the intent of cancel is to ensure that the user can stop the ongoing communication which prevents him from touching his own local buffers (page 40, line 44 and 47). An implementation can acheive this by ensuring that the send is locally complete after the cancel (the message is buffered). I think you are arguing that there is an implicit semantic missing from the standard, which should be added to page 54, line 17: "If a call to cancel is made for a send, then any receive, which matches the send, made after the cancel is complete , must block (unless another send call is made which matches the receive)." But we DO NOT have this text currently in the standard. And therefore I think your interpretation (while possibly attractive for users) goes beyond the current letter of the standard. ****** LJL End So, I propose the following clarification: "A message that has been matched by a call to MPI-PROBE must be received by a subsequent call to MPI_RECV or MPI_IRECV." ****** LJL Start I don't think this is the correct solution. Better to ignore the probe when cancelling messages, and add my clarification to page 52 line 2: "... if no other intervening receive occurs after the probe, AND THE SEND IS NOT SUCCESSFULLY CANCELLED BEFORE THE RECEIVE." a) I think this is more in the spirit of MPI-1 b) The semanitics are simply defined c) No additional requirements are placed on the user (like all probed messages must be received). d) The semantics are useful and obvious (if you probe a message, but then cancel it, what do you expect to get ... no message.) ****** LJL End Consider an implementation which uses an eager send protocol for short messages. The Isend may be complete by the time the cancel call occurs. I don't think the intention is that cancel is actually "unsend" in this case. *** The Isend is complete at the sender side and, yes, the cancel may require a communication with the receiving side to "undo" the send. This much has been clear since cancel was defined by the MPI forum. Indeed, this is the precise reason why many implementors were against the cancel function in MPI1. ****** LJL Start It is clear that a rendezvous protocol requires a communication with the receiving side to "unblock" the partial send. I accept this, although it complicates sends. My concern is to allow a fast buffered eager send for short messages. I think that your interpretation forces all non-blocking sends to use a slower protocol. Nothing else in the standard prevents a non-blocking send from being implemented as a buffered send (this is made clear on page 41 line 5). ****** LJL End Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com **** Marc ******* ------------------ RFC822 Header Follows ------------------ Received: by MSMAIL4.HAC.COM with SMTP;9 Mar 1997 18:32:05 -0800 Received: from 129.34.139.18 by EDEN.HAC.COM (PMDF V4.3-7 #5884) id <01IGB45Q08A800DKKC@EDEN.HAC.COM>; Sun, 9 Mar 1997 18:32:03 PST Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id VAA09584; Sun, 9 Mar 1997 21:21:54 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id VAA50538; Sun, 9 Mar 1997 21:31:51 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256456.000DE2E5 ; Sun, 9 Mar 1997 21:31:40 -0400 Date: Sun, 09 Mar 1997 21:24:53 -0400 From: Marc Snir Subject: Re: Semantics of MPI Cancel? To: llewins@msmail4.hac.com Cc: Mpi-Core@Mcs.Anl.Gov, Mpi-Misc@Mcs.Anl.Gov Message-id: <85256456.000BB996.00@watngi01.watson.ibm.com> MIME-version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT X-Lotus-FromDomain: IBM RESEARCH From mpi-core-human@mcs.anl.gov Mon Mar 10 13:31:29 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA19778; Mon, 10 Mar 1997 13:31:24 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA09968 for mpi-core-out; Mon, 10 Mar 1997 12:29:36 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA09958; Mon, 10 Mar 1997 12:29:28 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id KAA07851; Mon, 10 Mar 1997 10:30:00 -0800 (PST) Message-ID: Date: 10 Mar 1997 10:28:08 -0800 From: "Lewins, Lloyd J" Subject: RE: Semantics of MPI Cancel? To: "Marc Snir" Cc: Mpi-Core@mcs.anl.gov, Mpi-Misc@mcs.anl.gov X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc wrote: >The send is pending until it has completed at both ends. Thus you argue that the following is an erroneous program: Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Send (...,1,...) MPI_Finalize () MPI_Finalize () Since page 199 line 47 says: "The user must ensure that all PENDING communications involving a process completes before the process calls MPI_FINALIZE." Even if you argue that process 0 is legal (it's send is complete before finalize, your definition of PENDING leaves process 1 involved in a "pending communication" and calling finalize without completing it. Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com From mpi-core-human@mcs.anl.gov Mon Mar 10 13:36:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA19912; Mon, 10 Mar 1997 13:36:52 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA10135 for mpi-core-out; Mon, 10 Mar 1997 12:35:31 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA10129 for ; Mon, 10 Mar 1997 12:35:27 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id KAA08456 for ; Mon, 10 Mar 1997 10:35:57 -0800 (PST) Message-ID: Date: 10 Mar 1997 10:34:21 -0800 From: "Lewins, Lloyd J" Subject: RE: Semantics of MPI Cancel? To: mpi-core@mcs.anl.gov, "Raja Daoud" X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Raja wrote: > rank 0 rank 1 > =============================================== > MPI_Init MPI_Init > MPI_Isend(tag = 5) MPI_Probe(tag = 6) ^^^^^^^^^MPI_Iprobe > MPI_Cancel MPI_Finalize > MPI_Wait exit > MPI_Test_cancelled > MPI_Finalize > exit With or without the probe, I think the semantics are clear, and the code is legal. The users has completed the outstanding operations, and can safely call finalize (just as the following is legal:) Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Send (...,1,...) MPI_Finalize () MPI_Finalize () and so it: Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Finalize () sleep 10 minutes MPI_Send (...,1,...) MPI_Finalize () Yes, that does mean that finalize is a non-local operation. Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com From mpi-core-human@mcs.anl.gov Mon Mar 10 15:01:15 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA20976; Mon, 10 Mar 1997 15:01:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA12314 for mpi-core-out; Mon, 10 Mar 1997 14:00:02 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA12299 for ; Mon, 10 Mar 1997 13:59:38 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id MAA13527 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Mon, 10 Mar 1997 12:00:40 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id LAA10707; Mon, 10 Mar 1997 11:58:29 -0800 From: "Eric Salo" Message-Id: <9703101158.ZM10705@mrjones.engr.sgi.com> Date: Mon, 10 Mar 1997 11:58:29 -0800 In-Reply-To: "Lewins, Lloyd J" "RE: Semantics of MPI Cancel?" (Mar 10, 8:53am) References: X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: "mpi-core" Subject: Re: Semantics of MPI Cancel? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I appreciate if anybody could correct me if I am wrong, because we are > under a struggle to decide how to implement MPI_Cancel on our machine. > > ******* LJL Start > Exactly what I am trying to do to. Don't waste your time on it, life is too short! Why don't we (the MPI Forum) just admit our error and add a single line of clarifying text to MPI-1.2 which allows implementations to not support cancelling sends? I agree 100% with Bill that trying to fix the current cancel model is a moot exercise. Here's a wonderful opportunity for us to standardize common practice. :-) -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Mon Mar 10 15:35:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA21352; Mon, 10 Mar 1997 15:35:50 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA13272 for mpi-core-out; Mon, 10 Mar 1997 14:36:35 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA13262 for ; Mon, 10 Mar 1997 14:36:28 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id MAA23684 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Mon, 10 Mar 1997 12:37:37 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id MAA10775; Mon, 10 Mar 1997 12:35:26 -0800 From: "Eric Salo" Message-Id: <9703101235.ZM10773@mrjones.engr.sgi.com> Date: Mon, 10 Mar 1997 12:35:25 -0800 In-Reply-To: Steve Huss-Lederman "document credits & T-shirts" (Mar 10, 9:57am) References: <199703101557.JAA00245@rap.cs.wisc.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: document credits & T-shirts Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Top 10 Slogans for the MPI-2 T-shirt ------------------------------------ 10. "From the creators of BSEND and GRAPH_CREATE." 9. "Brought to you by a resounding 5-4-18 vote." 8. "One-sided, two-faced, and proud of it!" 7. "Don't blame me, I voted against that." 6. "Spawn this, buddy!" 5. "Is this our first or second second-vote?" 4. "Behave yourself or we'll move you into the JOD." 3. (Implementors only) "F*** the users!" 2. "Violent pain prevents me from continuing." 1. "The new phonebook is here!" -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Mon Mar 10 16:39:28 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA22118; Mon, 10 Mar 1997 16:39:24 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA14839 for mpi-core-out; Mon, 10 Mar 1997 15:37:37 -0600 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA14833 for ; Mon, 10 Mar 1997 15:37:30 -0600 Received: from denna.cs.msstate.edu (denna.cs.msstate.edu [192.208.157.80]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id PAA24080; Mon, 10 Mar 1997 15:38:23 -0600 (CST) Received: by denna.cs.msstate.edu with Microsoft Mail id <01BC2D69.431F1DE0@denna.cs.msstate.edu>; Mon, 10 Mar 1997 15:39:38 -0600 Message-ID: <01BC2D69.431F1DE0@denna.cs.msstate.edu> From: Shane Hebert To: "mpi-core@mcs.anl.gov" , "'Eric Salo'" Subject: RE: document credits & T-shirts Date: Mon, 10 Mar 1997 15:39:36 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Wait... At least one more: 11. MPI_WINS but MPI_AINT_KIND Top 10 Slogans for the MPI-2 T-shirt ------------------------------------ 10. "From the creators of BSEND and GRAPH_CREATE." 9. "Brought to you by a resounding 5-4-18 vote." 8. "One-sided, two-faced, and proud of it!" 7. "Don't blame me, I voted against that." 6. "Spawn this, buddy!" 5. "Is this our first or second second-vote?" 4. "Behave yourself or we'll move you into the JOD." 3. (Implementors only) "F*** the users!" 2. "Violent pain prevents me from continuing." 1. "The new phonebook is here!" -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Mon Mar 10 17:54:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA23159; Mon, 10 Mar 1997 17:53:54 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA17570 for mpi-core-out; Mon, 10 Mar 1997 16:49:36 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA17561 for ; Mon, 10 Mar 1997 16:49:31 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id RAA04922; Mon, 10 Mar 1997 17:16:57 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id RAA59436; Mon, 10 Mar 1997 17:26:53 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256456.007B49E0 ; Mon, 10 Mar 1997 17:26:38 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: raja@tbag.rsn.hp.com cc: Mpi-Core@mcs.anl.gov Message-ID: <85256456.0078CAB7.00@watngi01.watson.ibm.com> Date: Mon, 10 Mar 1997 17:25:30 -0400 Subject: Re: Semantics of MPI Cancel? Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > So, I propose the following clarification: "A message that has been > matched by a call to MPI-PROBE must be received by a subsequent call to > MPI_RECV or MPI_IRECV." Marc, in the example I presented, I made the Iprobe() call ask for a different tag than the message's. This is to make sure that we cannot use such an escape hatch by saying that the user knew the message was there. This forces the correct implementation to delay Finalize until it is sure that all of the source processes of its unreceived messages have called Finalize themselves (i.e. told it that they won't be cancelling any of these messages). *** Raja, I missed your example. You seem to worry about the case that process A calls finalize and, some time later, process B sends a message to process A, next cancels the send. This should be legal, the cancel should succeed. Good reason, indeed, to put a barrier inside the finalize, to make sure that process A is around until B finishes its send/cancel games. ***** -=- From mpi-core-human@mcs.anl.gov Mon Mar 10 18:01:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA23248; Mon, 10 Mar 1997 18:01:54 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA18328 for mpi-core-out; Mon, 10 Mar 1997 17:00:40 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA18320 for ; Mon, 10 Mar 1997 17:00:36 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id RAA08098; Mon, 10 Mar 1997 17:50:51 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id SAA07551; Mon, 10 Mar 1997 18:00:47 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256456.007E5C01 ; Mon, 10 Mar 1997 18:00:10 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: llewins@msmail4.hac.com cc: Mpi-Core@mcs.anl.gov Message-ID: <85256456.007DF154.00@watngi01.watson.ibm.com> Date: Mon, 10 Mar 1997 17:59:01 -0400 Subject: RE: Semantics of MPI Cancel? Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc wrote: >The send is pending until it has completed at both ends. Thus you argue that the following is an erroneous program: Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Send (...,1,...) MPI_Finalize () MPI_Finalize () Since page 199 line 47 says: "The user must ensure that all PENDING communications involving a process completes before the process calls MPI_FINALIZE." Even if you argue that process 0 is legal (it's send is complete before finalize, your definition of PENDING leaves process 1 involved in a "pending communication" and calling finalize without completing it. **** Was there ever a doubt that the program above is illegal? We clearly intended that sends and receives must be matched before finalize is called. ***** From mpi-core-human@mcs.anl.gov Mon Mar 10 18:23:15 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA23621; Mon, 10 Mar 1997 18:23:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA19025 for mpi-core-out; Mon, 10 Mar 1997 17:21:57 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA19013 for ; Mon, 10 Mar 1997 17:21:50 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id RAA08258; Mon, 10 Mar 1997 17:45:40 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id RAA21705; Mon, 10 Mar 1997 17:55:35 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256456.007DEBFC ; Mon, 10 Mar 1997 17:55:24 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: llewins@msmail4.hac.com cc: mpi-core@mcs.anl.gov Message-ID: <85256456.007BA374.00@watngi01.watson.ibm.com> Date: Mon, 10 Mar 1997 17:54:18 -0400 Subject: RE: Semantics of MPI Cancel -- try again Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > Process 0 Process 1 > ------------- -------------- > MPI_Init () MPI_Init () > MPI_Isend (...,1,...) > MPI_Probe () > MPI_Cancel () > MPI_Wait () > MPI_Test_cancelled () > MPI_Barrier () MPI_Barrier () > MPI_Recv () > I quote the standard: "Either the cancellation succeds or the communication succeeds, but not both. If a request is marked for cancellation, then it must be the case that either the send completes normally, in which case the message sent was received at the destination process, or that the send is successfully cancelled, in which case no part of the message was received at the destination." Thus, if MPI_Test_cancelled succeeds, it must be the case that the cancelled message will not be received by process 1. An implementation that allows MPI_Cancel to return success once the message has left the local node and has been buffered at the remote node is incorrect, since this will allow cancel to succeed, yet will allow the message to be received. A correct implementation of MPI_CANCEL requires a hand-shake with the remote process: if the message to be cancelled has already left process 0, then process 0 sends a request to cancel to process 1; process 1 either acks and deletes the message from its queue, or nacks, if the message has already been matched to a receive. The cancel call returns, and the appropriate information is returned later by Test_cancelled. This is explained in the "advise to implementors", which says explicitly that this may require an interrupt. This is why the "advise to user" warns that "Cancel can be an expensive operation". So, in this example, either cancel succeeds and the message is never received by process 1, or it fails, and the message is received by process 1. If my change is accepted, namely, that a process has to post a receive for a message that has been probed successfully, then the implementation may be actually simplified: a match by a probe call has the same effect as a match by a receive call, making the matched message ineligible for cancelation. I also think that this is what most users will want: not having to deal with the hazard of a message being probed successfully, but then disapearing before it is received, because of a cancelation. It is more natural to require that messages must be received once matched by a call to MPI_PROBE. From mpi-core-human@mcs.anl.gov Tue Mar 11 06:01:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA06087; Tue, 11 Mar 1997 06:01:26 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA26719 for mpi-core-out; Tue, 11 Mar 1997 04:59:29 -0600 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA26691; Tue, 11 Mar 1997 04:58:22 -0600 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id LAA21478 (8.6.13/IDA-1.6); Tue, 11 Mar 1997 11:59:34 +0100 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id KAA04152; Tue, 11 Mar 1997 10:59:34 GMT From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199703111059.KAA04152@awsrr.rus.uni-stuttgart.de> Subject: Re: Semantics of MPI Cancel? To: llewins@msmail4.hac.com (Lewins, Lloyd J) Date: Tue, 11 Mar 1997 11:59:33 +0100 (MEZ) Cc: mpi-core@mcs.anl.gov, mpi-misc@mcs.anl.gov In-Reply-To: from "Lewins, Lloyd J" at Mar 10, 97 08:53:07 am Content-Type: text Content-Length: 2803 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >>Lewins, Lloyd J wrote: >> >> Process 0 Process 1 >> ------------- -------------- >> MPI_Init () MPI_Init () >> MPI_Isend (...,1,...) >> MPI_Probe () >> MPI_Cancel () >> MPI_Wait () >> MPI_Test_cancelled () >> MPI_Barrier () MPI_Barrier () >> MPI_Recv () > Marc wrote: > So, I propose the following clarification: "A message that has been > matched by a call to MPI-PROBE must be received by a subsequent call > to MPI_RECV or MPI_IRECV." > LJL wrote: > I don't think this is the correct solution. Better to ignore the probe when > cancelling messages, and add my clarification to page 52 line 2: "... if no > other intervening receive occurs after the probe, AND THE SEND IS NOT > SUCCESSFULLY CANCELLED BEFORE THE RECEIVE." Both are modifications to MPI 1.1. The messages until now do not show a inconsistency and therefore there is no need for modifications. But a need for clarification. We had the problem that the intel testsuite expects that the cancel is successfull, but CRAY's MPI on our T3E does not cancel. We agreed with CRAY due to the text of MPI 1.1 in p. 41 Advice to users after def. of MPI_WAIT p. 54 definition of MPI_CANCEL p. 52 line 1, the relation of IPROBE and a subsequent RECV looking at all mail it seems to need a lot of time to understand the standard, but looking at all related text it seems to me, that the standard does not need any modification, because it says: - If on process 1 MPI_PROBE returns, then there is no possiblity to cancel the message. - Therefore MPI_WAIT has to return. (No problem because the local buffer can be reused due to "eager" implementation and due to p. 41, line 5-6.) - And MPI_Test_cancelled has to return "not cancelled". - The program above must be modified by using IPROBE instead of PROBE because the cancel may succeed and PROBE would never return in that case. - But with IPROBE there is no chance to guarantee that the IPROBE is issued before the message has arrived. - And also an additional IPROBE directly before RECV does not guarantee to return flag=true if the message was not cancelled I did not find such a rule in the standard. Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Tue Mar 11 06:45:30 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA06548; Tue, 11 Mar 1997 06:45:29 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA27349 for mpi-core-out; Tue, 11 Mar 1997 05:44:12 -0600 Received: from palrel3.hp.com (palrel3.hp.com [15.253.88.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id FAA27333; Tue, 11 Mar 1997 05:44:03 -0600 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel3.hp.com with ESMTP (8.7.5/8.7.3) id DAA15603; Tue, 11 Mar 1997 03:45:10 -0800 (PST) Message-Id: <199703111145.DAA15603@palrel3.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA083920695; Tue, 11 Mar 1997 05:44:55 -0600 From: Raja Daoud Subject: Re: Semantics of MPI Cancel? To: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Date: Tue, 11 Mar 1997 5:44:55 CST Cc: llewins@msmail4.hac.com, mpi-core@mcs.anl.gov, mpi-misc@mcs.anl.gov In-Reply-To: <199703111059.KAA04152@awsrr.rus.uni-stuttgart.de>; from "Rolf Rabenseifner" at Mar 11, 97 11:59 am X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > We had the problem that the intel testsuite expects that the cancel > is successfull, but CRAY's MPI on our T3E does not cancel. > > We agreed with CRAY due to the text of MPI 1.1 in > p. 41 Advice to users after def. of MPI_WAIT > p. 54 definition of MPI_CANCEL > p. 52 line 1, the relation of IPROBE and a subsequent RECV This interpretation implies that probing a message somehow "marks" it as reserved or untouchable, beyond the reach of Cancel. This is clearly not the intention of MPI, since it allows another thread to receive the message even though it was successfully probed. I think you're reading too much into line 1 on p. 52, it's is not an exhaustive listing of all possible cases, just an example warning or what might happen if other events affect the existance of the message. I think we can agree that messages must be atomic, they are either buffered "out-there", or have been consumed. Messages are consumed when they are received or when they are successfully cancelled. These two options are mutually exclusive. Probing returns temporary information about the existance of a message, but does not alter its state, reserve it, or otherwise guarantee its future existance. I think this model is intuitive and users most probably understood the stardard this way. To keep MPI-1 unchanged, I propose we clarify to *implementors* that this is the model expected, and to be helpful, we can spell out what this model may require in their implementation: a stateless RPC mechanism to the location of the buffered message and a delayed Finalize to catch race-hazards and late cancels. --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/wsg/ssa/mpi/mpihome.html From mpi-core-human@mcs.anl.gov Tue Mar 11 07:57:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA08078; Tue, 11 Mar 1997 07:57:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id GAA28033 for mpi-core-out; Tue, 11 Mar 1997 06:54:57 -0600 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA28018; Tue, 11 Mar 1997 06:54:46 -0600 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id NAA23478 (8.6.13/IDA-1.6); Tue, 11 Mar 1997 13:55:59 +0100 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id MAA04239; Tue, 11 Mar 1997 12:55:59 GMT From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199703111255.MAA04239@awsrr.rus.uni-stuttgart.de> Subject: Re: Semantics of MPI Cancel? To: raja@tbag.rsn.hp.com (Raja Daoud) Date: Tue, 11 Mar 1997 13:55:58 +0100 (MEZ) Cc: mpi-core@mcs.anl.gov, mpi-misc@mcs.anl.gov In-Reply-To: <199703111145.AA126910700@hplb.hpl.hp.com> from "Raja Daoud" at Mar 11, 97 05:44:55 am Content-Type: text Content-Length: 5297 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >> p. 52 line 1, the relation of IPROBE and a subsequent RECV > > This interpretation implies that probing a message somehow "marks" it as > reserved or untouchable, beyond the reach of Cancel. This is clearly > not the intention of MPI, since it allows another thread to receive the > message even though it was successfully probed. I think you're reading > too much into line 1 on p. 52, it's is not an exhaustive listing of all > possible cases, just an example warning or what might happen if other > events affect the existance of the message. > > I think we can agree that messages must be atomic, they are either > buffered "out-there", or have been consumed. Messages are consumed when > they are received or when they are successfully cancelled. These two > options are mutually exclusive. Probing returns temporary information > about the existance of a message, but does not alter its state, reserve > it, or otherwise guarantee its future existance. ... > --Raja i.e. you would also change page 52 according to LJL's proposal: clarification to page 52 line 2: "... if no other intervening receive occurs after the probe, AND THE SEND IS NOT SUCCESSFULLY CANCELLED BEFORE THE RECEIVE." because this is not the wording, but the intent of MPI 1.1. Then I believe that in the example below we can have two different choices for implementation (A and B): Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Isend (...,1,...) MPI_Probe () MPI_Barrier () MPI_Barrier () MPI_Cancel () MPI_Wait () MPI_Test_cancelled (flag) MPI_Barrier () MPI_Barrier () MPI_Iprobe (flag) if (flag) MPI_Recv () - Probe always shows the message from MPI_Isend. - MPI_Test_cancelled can return flag=true or flag=false -- in the case of MPI_Test_cancelled returns flag=true (choice A) then MPI_Iprobe returns flag=false -- in the case of MPI_Test_cancelled returns flag=false (choice B) then MPI_Iprobe returns flag=true (this behavior is implied because the previous call to MPI_Probe guarantees that the message is already at process 1 Or does someone believe that an implementation does not have these two choices? But what are the consequences for the following simple example: Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Isend (...,1,...) MPI_Cancel () MPI_Wait () MPI_Test_cancelled (flag) MPI_Barrier () MPI_Barrier () MPI_Iprobe (flag) if (flag) MPI_Recv () There are the following cases: 1. The MPI_Wait returns (i.e. act as a local call), MPI_Test_cancelled returns flag=false (i.e. the message is buffered and choice B), the MPI_Iprobe returns flag=true (i.e. by chance the buffered message is already at process 1), and MPI_Recv receives the message. An additional MPI_Iprobe before the MPI_Barrier in process 1 may return flag=true or flag=false. 2. The MPI_Wait returns (i.e. act as a local call), MPI_Test_cancelled returns flag=false (i.e. the message is buffered and choice B), the MPI_Iprobe returns flag=false (i.e. by chance the buffered message is still on the way), and MPI_Recv is not called, and the example ends in the state that the message has yet to be received at process 1, i.e. it is pending. The application can e.g. issue an additional message from process 0 to process 1 to allow a blocking receive of the pending message. The analyse of its tag, etc. must show, that this was not the receive of the additional message, and therefore the additional message needs also to be received. An additional MPI_Iprobe before the MPI_Barrier in process 1 returns flag=false. 3. The MPI_Wait returns (i.e. act as a local call), MPI_Test_cancelled returns flag=true (i.e. the message was not buffered but cancelled (another choice), or the message was buffered and cancelled (choice B)), the MPI_Iprobe returns flag=false, and therefore MPI_Recv is not called. An additional MPI_Iprobe before the MPI_Barrier in process 1 may return flag=true or flag=false. According to Bills opinion I would therefore propose to add the following Advice to users. It is strongly recommended not to cancel messages sent by MPI_ISEND or MPI_IBSEND because there are different choices for the implementation to handle such cancellation and it is very difficult to catch all cases correctly by the receiving process. Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Tue Mar 11 08:14:06 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA08378; Tue, 11 Mar 1997 08:14:05 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA28257 for mpi-core-out; Tue, 11 Mar 1997 07:15:04 -0600 Received: from palrel1.hp.com (palrel1.hp.com [15.253.72.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA28242; Tue, 11 Mar 1997 07:14:55 -0600 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com with ESMTP (8.7.5/8.7.3) id FAA01377; Tue, 11 Mar 1997 05:16:09 -0800 (PST) Message-Id: <199703111316.FAA01377@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA086176161; Tue, 11 Mar 1997 07:16:01 -0600 From: Raja Daoud Subject: Re: Semantics of MPI Cancel? To: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Date: Tue, 11 Mar 1997 7:16:01 CST Cc: mpi-core@mcs.anl.gov, mpi-misc@mcs.anl.gov In-Reply-To: <199703111255.MAA04239@awsrr.rus.uni-stuttgart.de>; from "Rolf Rabenseifner" at Mar 11, 97 1:55 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > MPI_Init () MPI_Init () > MPI_Isend (...,1,...) > MPI_Probe () > MPI_Barrier () MPI_Barrier () > MPI_Cancel () > MPI_Wait () > MPI_Test_cancelled (flag) > MPI_Barrier () MPI_Barrier () > MPI_Iprobe (flag) > if (flag) MPI_Recv () > > - Probe always shows the message from MPI_Isend. > > - MPI_Test_cancelled can return flag=true or flag=false It must return true, no matter where the message is buffered. I don't mind incomplete implementations in real life (since I believe Cancel of both send or receive requests to be useless, users can always send themselves a dummy "terminate" message to cleanup dangling receives and not open the cancel can-of-worms), but if we bother having Cancel in the standard, then let's not fudge its semantics. > (this behavior is implied because the previous call to MPI_Probe > guarantees that the message is already at process 1 That's the complexity and ugliness of implementing Cancel correctly: it must hunt down and destroy the message no matter where it resides, even if the destination process buffers it locally inside MPI, or may be it is scattered in multiple buffers along the way (store-and-forward design). --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/wsg/ssa/mpi/mpihome.html From mpi-core-human@mcs.anl.gov Tue Mar 11 11:35:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA11839; Tue, 11 Mar 1997 11:35:54 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA03167 for mpi-core-out; Tue, 11 Mar 1997 10:34:07 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA03162 for ; Tue, 11 Mar 1997 10:34:03 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id IAA21587 for ; Tue, 11 Mar 1997 08:34:40 -0800 (PST) Message-ID: Date: 11 Mar 1997 08:30:01 -0800 From: "Lewins, Lloyd J" Subject: RE: Semantics of MPI Cancel? To: "Marc Snir" Cc: Mpi-Core@mcs.anl.gov X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Thus you argue that the following is an erroneous program: Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Send (...,1,...) MPI_Finalize () MPI_Finalize () **** Was there ever a doubt that the program above is illegal? We clearly intended that sends and receives must be matched before finalize is called. ***** Thanks for the clarification. Yes there has been a doubt in my mind for the past three years (just plain dumb I guess). The problem I have always had with your interpretation is that it appears to involve a race condition, i.e., can process 0 safely call Finalize, even though its send is still pending? By interpreting the requirement using local semantics, I have avoided this problem. If the semanitics you suggest are the intention of the forum, then I suggest the following change in text for MPI_Finalize: "The user must ensure that all pending non-blocking communications involving a process are (locally) complete before calling MPI_Finalize. Further, at the instant at which the last process calls MPI_Finalize(), all pending sends must be matched by a receive, and all pending receives must be matched by a send. Note: since MPI_Finalize is a collective call, a correct MPI program will naturally ensure that all participants in pending collective operations have made the call before calling MPI_Finalize." ------------------ RFC822 Header Follows ------------------ Received: by MSMAIL4.HAC.COM with SMTP;10 Mar 1997 15:02:50 -0800 Received: from 129.34.139.18 by EDEN.HAC.COM (PMDF V4.3-7 #5884) id <01IGCB3S22VK00DNYV@EDEN.HAC.COM>; Mon, 10 Mar 1997 15:02:06 PST Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id RAA08098; Mon, 10 Mar 1997 17:50:51 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id SAA07551; Mon, 10 Mar 1997 18:00:47 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256456.007E5C01 ; Mon, 10 Mar 1997 18:00:10 -0400 Date: Mon, 10 Mar 1997 17:59:01 -0400 From: Marc Snir Subject: RE: Semantics of MPI Cancel? To: llewins@msmail4.hac.com Cc: Mpi-Core@Mcs.Anl.Gov Message-id: <85256456.007DF154.00@watngi01.watson.ibm.com> MIME-version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT X-Lotus-FromDomain: IBM RESEARCH From mpi-core-human@mcs.anl.gov Tue Mar 11 12:52:07 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA13391; Tue, 11 Mar 1997 12:52:06 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA05063 for mpi-core-out; Tue, 11 Mar 1997 11:50:06 -0600 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA05058 for ; Tue, 11 Mar 1997 11:50:02 -0600 Received: from msmail4.hac.com ([147.17.106.42]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id JAA01115 for ; Tue, 11 Mar 1997 09:50:40 -0800 (PST) Message-ID: Date: 11 Mar 1997 09:48:58 -0800 From: "Lewins, Lloyd J" Subject: RE: Semantics of MPI Cancel -- try again To: "Marc Snir" Cc: mpi-core@mcs.anl.gov X-Mailer: Mail*Link SMTP-MS 3.0.2 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc wrote: >If my change is accepted, namely, that a process has to post a receive >for a message that has been probed successfully, then the implementation >may be actually simplified: a match by a probe call has the same effect as >a match by a receive call, making the matched message ineligible for >cancelation. I also think that this is what most users will want: not >having to deal with the hazard of a message being probed successfully, but >then disapearing before it is received, because of a cancelation. It is >more natural to require that messages must be received once matched by a >call to MPI_PROBE. So you propose that the following code deadlocks (since the Wait is no longer local)? Process 0 Process 1 ------------- -------------- MPI_Init () MPI_Init () MPI_Issend (...,1,...) MPI_Probe () MPI_Barrier () MPI_Barrier () MPI_Cancel () MPI_Wait () MPI_Barrier () MPI_Barrier () MPI_Recv () I.e., the standard must be changed so that page 54 line 13 reads: "If a communication is marked for cancellation, then a MPI_WAIT call for that communication is guaranteed to complete, irrespective of other processes (i.e., MPI_WAIT behaves as a local function) UNLESS A PROBE (OR IPROBE) CALL HAS BEEN MADE WHICH DETECTED THE PRESENCE OF THE MESSSAGE ON THE RECEIVE SIDE;" I would oppose this. Lloyd Lewins Hughes Aircraft Co., llewins@msmail4.hac.com From mpi-core-human@mcs.anl.gov Tue Mar 11 15:08:44 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA14788; Tue, 11 Mar 1997 15:08:43 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA08902 for mpi-core-out; Tue, 11 Mar 1997 14:05:09 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA08864 for ; Tue, 11 Mar 1997 14:05:01 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id OAA09110; Tue, 11 Mar 1997 14:56:16 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id PAA31813; Tue, 11 Mar 1997 15:06:11 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256457.006E65CB ; Tue, 11 Mar 1997 15:05:50 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: llewins@msmail4.hac.com cc: Mpi-Core@mcs.anl.gov Message-ID: <85256457.006E3A16.00@watngi01.watson.ibm.com> Date: Tue, 11 Mar 1997 15:04:44 -0400 Subject: RE: Semantics of MPI Cancel? Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk ... I suggest the following change in text for MPI_Finalize: "The user must ensure that all pending non-blocking communications involving a process are (locally) complete before calling MPI_Finalize. Further, at the instant at which the last process calls MPI_Finalize(), all pending sends must be matched by a receive, and all pending receives must be matched by a send. Note: since MPI_Finalize is a collective call, a correct MPI program will naturally ensure that all participants in pending collective operations have made the call before calling MPI_Finalize." ***** Amen From mpi-core-human@mcs.anl.gov Tue Mar 11 16:34:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA15930; Tue, 11 Mar 1997 16:34:06 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA11194 for mpi-core-out; Tue, 11 Mar 1997 15:32:25 -0600 Received: from hdl.ie (galois.hdl.ie [192.122.222.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA25255 for ; Tue, 11 Mar 1997 02:09:29 -0600 Received: from ur (ur.hdl.ie [192.122.222.40]) by hdl.ie (8.7.3/8.7.3) with SMTP id IAA17643; Tue, 11 Mar 1997 08:08:41 GMT Message-ID: <33251297.2781@hdl.ie> Date: Tue, 11 Mar 1997 08:06:47 +0000 From: Mark Fallon Organization: Hitachi Dublin Laboratory X-Mailer: Mozilla 2.02 (X11; I; AIX 1) MIME-Version: 1.0 To: Shane Hebert CC: "mpi-core@mcs.anl.gov" Subject: Re: document credits & T-shirts References: <01BC2D69.431F1DE0@denna.cs.msstate.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk And the list goes on: 13: (Forum member) "I have this user who....." 12: (Users only) "F*** the implementors!" > > Wait... At least one more: > > 11. MPI_WINS but MPI_AINT_KIND > > Top 10 Slogans for the MPI-2 T-shirt > ------------------------------------ > > 10. "From the creators of BSEND and GRAPH_CREATE." > > 9. "Brought to you by a resounding 5-4-18 vote." > > 8. "One-sided, two-faced, and proud of it!" > > 7. "Don't blame me, I voted against that." > > 6. "Spawn this, buddy!" > > 5. "Is this our first or second second-vote?" > > 4. "Behave yourself or we'll move you into the JOD." > > 3. (Implementors only) "F*** the users!" > > 2. "Violent pain prevents me from continuing." > > 1. "The new phonebook is here!" > > -- > Eric Salo Silicon Graphics Inc. salo@sgi.com -- ___________________________________________________________________ Mark Fallon Researcher Hitachi Dublin Laboratory O'Reilly Institute, Tel : +353-1-6798911 Trinity College, Fax : +353-1-6798926 Dublin Email : mark.fallon@hdl.ie Ireland ___________________________________________________________________ From mpi-core-human@mcs.anl.gov Tue Mar 11 22:01:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA22724; Tue, 11 Mar 1997 22:01:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA17039 for mpi-core-out; Tue, 11 Mar 1997 21:02:38 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA17034 for ; Tue, 11 Mar 1997 21:02:34 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id VAA11504; Tue, 11 Mar 1997 21:53:35 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id WAA42093; Tue, 11 Mar 1997 22:03:36 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256458.0010C7BF ; Tue, 11 Mar 1997 22:03:17 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: mark@hdl.ie cc: Mpi-Core@mcs.anl.gov Message-ID: <85256458.00107550.00@watngi01.watson.ibm.com> Date: Tue, 11 Mar 1997 22:02:08 -0400 Subject: Re: document credits & T-shirts Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk 13. We have fewer functions that X-Windows ... but not for long. 14. No Java binding and proud of it. From mpi-core-human@mcs.anl.gov Fri Mar 14 11:05:28 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA00657; Fri, 14 Mar 1997 11:05:28 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA12436 for mpi-core-out; Fri, 14 Mar 1997 10:05:10 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA12430 for ; Fri, 14 Mar 1997 10:05:06 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id KAA02026; Fri, 14 Mar 1997 10:06:38 -0600 (CST) Date: Fri, 14 Mar 1997 10:06:38 -0600 (CST) Message-Id: <199703141606.KAA02026@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: Re: document credits & T-shirts Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Since there has been real activity in choosing a t-shirt, I will let everyone know the current ideas are in ftp://ftp.cs.wisc.edu/pub/mpi2/tshirt/. See ideas.txt for the e-mails so far. The images are also located there (hc.ps for Hans-Christian's recent one). Steve From mpi-core-human@mcs.anl.gov Fri Mar 14 11:23:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA00842; Fri, 14 Mar 1997 11:23:06 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA13029 for mpi-core-out; Fri, 14 Mar 1997 10:23:45 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA13021 for ; Fri, 14 Mar 1997 10:23:40 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id KAA02132; Fri, 14 Mar 1997 10:25:11 -0600 (CST) Date: Fri, 14 Mar 1997 10:25:11 -0600 (CST) Message-Id: <199703141625.KAA02132@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: http alternate address for t-shirts Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I have been quickly informed that some got a message that the max # of ftp users is exceeded at our ftp site. To get around this, you can use the http address http://www.cs.wisc.edu/~lederman/mpif/tshirt/. Sorry for any problems. Steve From mpi-core-human@mcs.anl.gov Fri Mar 14 11:50:57 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA01145; Fri, 14 Mar 1997 11:50:38 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA11505 for mpi-core-out; Fri, 14 Mar 1997 09:30:57 -0600 Received: from gatekeeper.pallas.de (gatekeeper.pallas.de [194.45.33.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA11500 for ; Fri, 14 Mar 1997 09:30:50 -0600 Received: from mailhost.pallas.de by gatekeeper.pallas.de id QAA12104; Fri, 14 Mar 1997 16:32:06 +0100 Received: from dilbert.pallas.de by mailhost.pallas.de (SMI-8.6/SMI-SVR4) id QAA02580; Fri, 14 Mar 1997 16:32:08 +0100 Received: from localhost by dilbert.pallas.de (SMI-8.6/SMI-SVR4) id QAA00218; Fri, 14 Mar 1997 16:32:04 +0100 Date: Fri, 14 Mar 1997 16:32:03 +0100 (MET) From: Hans-Christian Hoppe X-Sender: hch@dilbert To: Steve Huss-Lederman cc: mpi-core@mcs.anl.gov Subject: Re: document credits & T-shirts In-Reply-To: <199703101557.JAA00245@rap.cs.wisc.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Mon, 10 Mar 1997, Steve Huss-Lederman wrote: > Also, recall that the effort to get a MPI T-shirt is going on. We > hope to get a design by the end of the month. Right now we don't have > any. If you have ideas, send them to me (ASCII, postscript) and I > will make them available on the web. Unless someone comes up with an > idea we won't have T-shirts. Dear Forum, I'd like to propose a somewhat more graphical design for the MPI-2 T-shirt. I append my first go at the "shoe"-design, just using clip-art from CorelDraw. We would probably need something nicer for PVM (ballerina slippers), but I couldn't find such a thing in a hurry. Best regards Hans-Christian begin 640 MPi2-shirt.ps.gz M'XL("$!N*3, W1S:&ER="TQ+G!S .V]>9/;2)8G^#\^!;9W59/:D2)P$215 M-F9YJCJG*U6:5%:.:C5E;2 1K#%(*-(AA11''WW]>L]=_\YG& HIZ;'K,7( M%$G X?[NRQ_ )__7ZS?/O^FVB_YY<9$E3YY\M^N;PW;W(NT^K&[W[]/)Q62V M3+_;WC[L5E?7AS2?S^IGXM]YE?[<=*NV6:<_//3IF^WR\+'9]6*&7U:'=?\B M7>ZVF\.%F$4<>MU<]?L7:6X^_FG7]6*%;_9MO^G$L6^W=YMNM;GZ=GO_(LW$ MWV1>I[.J$*>^W[9W-_WF\%),)F;XYW[]H3^(59U3KYO;?O=F]7>Y0E.)$S]L MNN^V-_+4/GGR_:\_OGXCOS:;[H^K36\02Y^_3G?]NF_V??I\FQ[VUZO=X7E^ M(29%7+_*ZF$B\^ZO5YO5NV[[I#R_20W_? M7MSNMLFEN.C[57M(BTF6=O)#UR]3.KB0%Z67KX[BX"=YXO+;XV*UZ>2@3Z_2 MRS=',<_UIU?)Y=OCF_35IV_3RU]^/AYVS6:_;@Z]'++:KP6:S5K@+KY]V O: MI7F>3HOTYFXM#UVK0[.+B3B6F&-K0<#;]=U^GAW5I7KMKX54K#9'.>/Q798Z M@_+T>?[I^#Q/\T_):MG+Q21KL[^VVTW;'#ZMEG+!G_O]=GUW6&TW M,/\LBO M<&C37Z5[(69]HA9Q5G!'B@OE8(V+A#A+%1F.SICG>E!R;0=],I#]\K.$QQGK MSJTG-5>G>=ITG;PZ^>7G=S?-8;>Z3]N[W4Y(@?YV[.YN4_G_3HIUNK];I,UB MGV87F7CEZ?IP5"?$@I^2Y7;7K-=FI 38^?C7?6]FE#S[6B(NZ'#;'S7_#KN[ M7K(WN?Q:R/9=LU[V?7?<'YK#W5X)S:4]K <+(932\'6[O5WU^^/E_ZT_I&_E M)"]_:@[O*S$5:IO;\\W/9I*4?+;S]IPBTWA]T]'?M6Z'\JEDQ?)7MQ=G.5 M7BZD?KY-F]VN>1""OA+7"0U]*S[>K=;==]?-[BC_4=_ZG:33#YMV*VU)*H!\ ME?12+^YNCY?+[385]%R*93X59CZ!OOCG@UQ_O6TZ@8J 6&HJ(?9.DJM;"K3V M2\%W,;N&5])*?#-8ZT'[HQ0..?"M&?9.?):D$V]2AB4-$^^B'XZWVUNYIA*8 M?BFLDX2/X11#VNOG'U?=X?HH/G3-H9%#DW6_N3IS-*!P24-EN_F@@=YP<#80HY)Y-B\F#KCN_NAT3F L;H1OH & M)G+@05:=%L'=1_F>7.Y:_?WJULBH M>?^#?E,\$>^.U!SWS03)=8M4&-?^7GU^8V50?]6 &53? MII*?K!6*#DF6&H*GAHXILTJ1PSFK.*V0,T,3(0EMTU[W7?]AU?9VC)U"*K95 MV^<7.4]H)E?S7"C2_U42B^AN:,933M7"LW0EQ=BHGQSBD3,QY#RJPX(YXEUS M50Y\F^X^?CK^P=C43VJ9FT;$$,)9BI"BE^SZP_&XZA(QFZ*)O$Z"]%9+U-518G0K+I335I9Q MJXUPX1]$J"*TFS_+Y<0P,4#LEQ,5D0H>*2?6)W8$$5IX3=ESBI]ZW.Z:6=D0J4$@,)+OQ M-8N97L!95*BB.98H"KZ1HTZL1GBWZQVOIBFGQ2!QB6;&"L$T8Y>K]7JO)'Z$ MWH0472 8;,0L$S#DTZ/!46(BX1*H?!+N40K?[?K=45R=2N&^^I38SZEPH\?< MJ,R5\U&=D,,$V_@#']S)@XGYY,PA(3QJ2;8'!:H\\7JG)RCT&E,)#7D<=_TR#-O4N68 MQ-5&R!+/U"HA,%S12B2&*Z6Z4_$7^SK7K H6_7A,VM8<^%X>6&QOCW?[?J?C M"O'M^?5V^SY]O]E^W!SIJYSQ\LV/J3*GKU(=&2;2,LG(^+!-+W]-A^(EK?<('O'>#WU]N/M\)FJ7!+!%*[@X.T^JZ&)OI*>T"14EI^-V@448#W+1?! MGZ+]I4@0A&5,WR8RCBHFM8E&7BD]$-)_%(>E?3#:H?6]-&R4+&L_J#!%RE92 M3Z:S_&):ZZEUI/HV]8_J>/>M1.OV*)&4GW[VX[U$!7C??K\5#J_.7 =PMQ;+ M9O3Q07_\<#3?WM*(M^FO4E)^38[R[>=?4R<2-5&@R&VZ._'UXW6_ZY6ZT!$G M2)@*KA^S=)JX9D2<_^K[U?YVW3P\[?]&XE&YIN:K5_W;7^1)8>34W#I),:G' M)Y.S&(D18>'Q>*4$]9>?E=<4_XE/N22[S#52C93&45V:2((=)8$#91%L@:$_+64JJ3Y= M"?%0'ZZ/YLCJ: ;]V]%<]_Z85/K3QZ,'R_IXFXIU/LK/-_)SJ3]OY.="?][* MS[G^_+=C(HV3^KP[2G.J/N[%1W/EX2A##?7Q7JSUQEWM0G.1_MQ<'_[^+'U]D?[4M/_2/#Q+OQ%Y9/I?^^5RUXMOWUZD_[S= M;9ZE_W*1?MOO=@__]$Q-\J'?[:7=$K-D%[4Y*(+<7A[)J_2;V]UJ+2L:$W-. MKK]I;M3YV>Y"@&5.",D6(\71]XNOV_W%W2G]/2':QFG-[NFE56C="/T M4!Q^$.:[7=]U8B)A"=TUQ#3-!T$BA8H$1L8#(N80ANWJ(OWE6DPO_A/\ZMO# MZD,O9H(5Q'RJ?BG,045ZSZU+&[C^G@C3I'QLQFUCP/PL+(:R#>_5RM[U) M_WC7KKKF0B/U'9U4)$Z;_7YUM=$UKH_";@L7KHX*6-?K[26\K[-.LY/DN=GN#XI* APQ>K7SQHMI;K?[E728^V>"\&TC*)_N MMS=V6NF/]VG;;#;"4?7-?B7HL>N5$O1R1C&%HNHS%>^N#@+:][WD@O"$@H0B MZ!!IE)#KK1B[2_TC)+H"#7;RW\;O8I<1*CC M7G%13.#P,7W^7!C8A9!C02>1*'=]*XS^^NDS<-@.NKO,[$L?UV M>;A^N+WN-XJNT_+I1?JWN^VAWXNIY-H"LZN=DC?)#EEU3/_M;G]@BJT._VDO MXC_A"QHYQ6JW6PE?+-59(B$D58(D"'"C9+%_:ZEN1."KXFGZLU3%0 BRWHK 2,RR7:ICPAM=]8(?'[8K)9+9,Q$+*21R M&=0(9@J0/PI$>A7MW2P4 60MF: IGZ9_^=U?C)2FW^HZM-&GO98/+=<'\>: M^WLC9F*.ZZU@AN:59OUK(8EOVMWJ]D "<]OOKF5TW4'(5"YIF? B^ED> M4N'#516H4U-) !0[-^E7$O[EG; C3\4<=[>".MUS6886;)7BN.L_K/9:;60< M*RC2B8CR1D:^P+VK?M/+C.3!M3L$V^1IJ@4O;6YO^T9<:JV=W=K!N9 MY/QQOVY$JB>R>/DN1VZOMIO^O7#JJLYH9I4GA$(*U;J\66WN]G8Y ;TP$GO2 M?&,N#A^WLO!ON'FW"5&3ZBS,@Q .:50^]B Y%W+%_Z]M=A+DO^MW1:@\2T1( MI(\+3-?"2ZX4C^X$6'#$G=1"98^VPBZL-D*(72ND[)J2?2E!ZIN" M5DXD%&=>:T-13;3N?M3V3C'@<"TFH_;I17[J45$2&C,I>7 MKA555"Z$.&H!5BG*7A))V^)+$4NOFQMS3;=8BQ!1V"89%EUM%"G7ZV8G-2V$,^XW"4'_4UNNRVUJ M*Z;2XBI!+:5 _+W?B31:"*X(.X7H71ZNA?,33-S>[:062!SW(J&^W L1EE#J M&N&ED,Y>+K&6TK3O;U;FHQ0&,>AO4IX$C:3=VTGP12@J=$@M6\EE!>4OOQ'9 M87KYG4A?T\L?TLN7LJ)Z^<_IY8_IY7]-+_]%J)@(]--+D0[\25TXD1>^%OF+ M2 ;3RS?IY2_IY9]E">#ROXL3(FW\BQ!\J6U-^[X_: (MQ&>CJ.8X4ZM=K=K5 MKA4AW:6TDKM]NY6;C6*A6BYDY>A24'$AD!4L$J@)J@C,!%U%DB#2 Y$8"*R% M<@N:"#JJZZ?R>I&Z_$W8 D$=052QA,AP1>H@3@AB/@BZ:X!Z G-GOKO@'60Q MV=&2)\;E_]Z82!'6[]6"LVQ0EZQPJOJ/,&A;89L2*X'ZL+ @J]N]" DON^;J M2O)+OXL!TF(H&BUE+B=E2(0H=UHFWQA;4WF4=Z-ZY*2)F55;C4ZBK5;RV+F^* *D:U@LT/4O46N^W[?J-X MO.^5,Y'J;M:26D0[X9?;7;<4RJ2U2T*][F^V!T.']?9*;HX+B!)6ZRBK5-_V[B ]VITR08TPAK?79&):E7*I9=J^$R@TRK[8R022-T*^ M[]8^ELR;C30?.V4XQ.?K9KTT )C#>VM0)/T5%M])++XQ1OD; ]XWKNQ^8\3C M&TOX;[2G_D8(['<&5#'+#V:6'\PL/[BS_, B(NR4&?BC&?BC._!'3Y84D7\0 ML(MS(#5V;@RAVX\HCW4L+72^)M#%FVYN(M$6_K7KVE41;8;O5A MI1#8&M+=F1GNS/)W'NGN[)4/9L!!D^Z!(?MKJLR9K*J\5-T=8>_%[5XU7T"; MQ6YY%!%YI_9EO:U,6][@1HEY#E<]INOOQ)>M!VGZC+ MI65TRK$4F1_U2=Z5D 55(7.T+6% U3L*B2Y&K]7A3\=WA,#1V=(W97L!BEG= M.2*13SRD>1#!2GL#"M)>M[6\HTT,YUQBSM&F>RG9//2>R-*=.2]2.M+CL[J9OM"M\?\W"LR]VJ0[4T@?JAY8O4^%L[]K5#,9CTDGF]TGU"64<_% M&_?LUV_TE=_WRT88I/WQ\GJ?UGDA-W(_[-/IO%#-/EM3TS?OU_M6-SM\H ^- MVJ23N^9__/$U;;>OWO3"57$7T>Y:';!M1<+M"_-_E'L-P@GVZD-3J;=U?]6L MQ2?5O*(*XW>;U4%ND<@+O[Y6AX[.&;.-T>J-WJ\_X(#$['[0 +7CH;!]:X#. M=;'UZP_ZS(?P3"OBJF.BCQ7FV/5VN91;G9)&9FDZ\H&.-#*=."H:Z0."-,QZ=>11UWK4M3]*4&>]OC]>BG_,W"+1D5\?S->[G3@K M_N&O#_*K.7O37.F^*36=%$>I,I=?_]1(L?JSNZ-WTYGM,.^0U%II!I)+>4@9 MA/YO=H@6,C&.;&*F.Z/$?\V].78E*'.CCCF;DT5FK:?:\I6:L-+@B9%&>(?D MQPA.\DKZB^/A_E96X.0FULU!*;>4.RTQ80P/A4> C M2"$1^O1)_FN,*^T9M2(W/ARV*L[6NXV-WA>]O&K3^TYO7PINE/,DTTT!6;K\ MNZ#KWGY/_]5O;X3>:;PJ+)K2]A#D>@?ENOFZOA:Y/6';]W#WFQ]W=[NTD9MU6J7LCMNMDN9RR@7\T8YK3=RUTQ[F^=YHJZ3 M@G$OQZ4/ZE_I0^&"?)8)@ASD[H2\SEFOE#*"T(O&XZO&Z6Z)DU.GI^GMF[PH=+7?6)B?;+SPFYYF(Z M@/\0W0RP\O^YO20)218!?81D24 R1FQPY'WWQK-B?R_8@W?C)'6*JD4<=,BTK IB+>:UB*$U(+1LD(4HK M9)]2PM$4M1JDMS>\\_=*&D -FS9ZLK"E/9W7Z*JKJZ,<]O^\$3%X]T;1V;1X M<%>%-?CN?C^!D;(# MS)Q2#3C._ 8H,S^=E;/+LY(R)"W*!_:[1EIZ\TW*'W^1>]*2S.SBY<%NJR,; MAV)$2**O6LRC%/L8\B"5EO1*)#8ZS-6M!9RH"$$W70/A!_9-::M4V+A@XU]? MJ>*0$309 H"L6?6X_%I7:750>U2A;D*!L#XHKR/I-&T3)T0.XV/M)V6"JMZ% M<'3ZD_I7BK90BO:]UH^OA1H94)*CB21%>$0$<_J>KO=>\X6(/>F+...U7"0^ MJ:2ZBLA3A)K"PLCP5T:X&A49;AJC9V++HPPU)4,%DQ.21Q-1'F6 *9GLR*KJ M$?3LH+Q2P2-':91P.=102(/]0 (COZJE]%8I]LAYK)$DSV7!=X*:RVUU_VS%9FW MR!7?@KF3Q_XBQ/B5S-[[@YR@;5@!M9$_[@^RX)FJZQ-]A=7,Y6I]5&U >GH\ MK8O1(L:7Y5VUK?(VT8[#?$LO'W;"I8KW>_,NE=*$Z(/-@*^DK,O!B;I22U>F M>N.FSI3N+1-!7NWGSN4\G\P*$;=-)N4LFPO_K5KT:O/_5WS_TM-$ M-P.FER\;,6;RKJ[3NGPGKY3_%.]DWEJ^*\NT*-2Y.JWF[XY)&NR!NM6 ]-,T MS?/YQ5R\JEEZ:?MZ=D*@7R[2O)R_*T3,\:Z8B34J,6DMWXMT*O_)Y9&B?C?) MQ=*325+7[XYC"\[367E1RM<4UG-)M=Y>405"$$UD;4^>O!0B?K?K7Z3_KQ-U M"")UMRN?J.K^+G,WEKD92Y*KET_77TE^/N&?$@%>\B0U M+8D"!+FAMDM5)]]%JMI)=)/]2M:3U"#=^:/'J6*^[$#8ZPZ$#ZO^HZR4RXZ# MG6IH-+T*^[M;D2@=]&7/=.O"Q]5>;GRM#O[DS5XVA3PH?;D0T%TN[F3#2SK) M"^XDE46KQ8-*$]Q#:O9_O54;;O+V#^?4[>I>6/!T6L_^X!'.777J>RI,97=A/LJ]C0U]6:SO>OJL M(\P7J=Z'E>TE!CJ]!R)W157+R6IOR/S55C5L-.NGM+0Q-;)9+?6PWO5-=]W? M&QQ-E"XO,+?Z28$7Z<1!')*(IK0;_*\&!!5WB1.?>("[E*)YN(3*3X9F4^5! M76(N54'13)X:?YJJAF#_&EV?I=7=B3Q$W89\.:/L5>+A&H?[5B=<"=":80;IMBM=9E+ :5U1%2A+/V%G;C8B5)+-'S-UZ)WZUYX2 MZFR.J$$FDI+M6_+87]6_QS0058FU*E*6J5W=X]T3_6_Z:NN,2+?"-,E;8W\O M(5#5#]EBX"B=O?1_&?!_$+,/(< PDTAX[!F\Z@P-/8'9"<5U]=91VW]7K74& MZ2I =E',Y^I& CB=F].3V92J^SBD,$-RX6V<(1]6LGE'4HD&4EG_$88C_T<9 MC@&3D9]M,ERDA,7X#99"BN+K?7_7;4^)HC/B\T51[="\,!LU AK=DJ@X>=/< M_H.\R B'+O5V@::J:\M]ITM IF9XZ=U$11+U1:C_#Q'JSQ'H_]"2^R6(.BU% M\2#*%20OB!J2HU8.>Y%FS]VKA"#DS_^ !M2CK,X4)&U5EX]VH@?9!F7N:%83 M<1.!<;+JH,?/\4F]:;O^]G"-TQKKL^= R+("QE-/=.!U#,50QE%UF*W]R+,QY':@!#2 MVIPP;ZZBT$FIW.8TL@+C,CDT%I.-AOU$V\'HV3)XB,7#3"8V2T8'%CN6!&@N M6UB>T/OG) /V\G\ 2H.1E2.MGJD;SA.,F3DOA1-R^"AC=)_^+GU(Z;DSPO9? M>">L-1*'ULVB7Z>J:BG[,^BP)K4ZZ1\B>OU.D8;.L?D#L'/7'O)@ZY3D)3_? M;;1_U04S^0 A]EEJBE=;YQ!-R]&VLYQQ]0;^\=PO[C M5GG4K,P[G.7R5?]1WM=WMWMXTUZO%^^?_[R]:38I=]79*Y5(N$] ,:CJ+I;' M^BPC(L/(*N7Z+?@.N\3' 6@E+.I:?=.E%%7HLE?4H(.LZ[07X>JZW[23>(HO MC4Y>B4BZFB?YC#]FXF^I7E7?U?(ORV:U_,NRJI1_699UZH]>Q70F_[),7S== M]I7\R[)D7LJ_+)NH/WY-ZX7\XPL6RUS^95G3R[\LH_.J:BVN3\R%#4UD+IRU M<_DG9C00T 7F-:MK>2S+YF*,^)S31$4YD7\\446HSM7K]$0*HK95,XH)U'M! MWVF!8B;_>(&,4*0%:O6RJ,WDK.X*L( X5M<.1#E!9,[G"4*0-_*/(*A;@J!5 M+PL!+4S,2>@+G01(JA%("@(@(3DQ$,QIY?F<5C[)+8L:G:"9$7<)CX-Z3F,! MHC(A&AB(*A0(&1)#(*(X$6/HQG)<46RB ;BE@R=>96$8H(2"A/EI+0QHO-$ M**'GHD( 1)66G &AHLQ3'D) $[.I)11(CB+$1TA+5Y ]%4'042!1;LCO$>0L M1S&: '%SFBZT1Q%=&Q.+0(Y0-8AXC*+G<2!%QH@:@SA!"$;D*!!( MV*$B"Q.X48XAC63E@/U$=HAI('V1RYT3*VG"WD@D&-LI@G],,@ZAUA8XYF1:S/1IR BZ$$2OQ>.,_$F_/ J7%H"%F[ ,+&6/SB"#V>#Y! M7Q^SW3%SPRH2H\F(7/5!Z <^/+#%,50#"WF:%OEDQ/.RQ3P;-=3Z@-@^[AJ" M(4\Z9NC.5-J\'?-SR6G[4Y2G4]5)-Z+]!49XV,_F2:81E'-"UQE>)HH/R$*9911FHA&=>\C8H _E129\ D3M( MHPKB)B[D6XU\EG"=R$ 0*8DM:"*B89!E^P4HP7: !(GMTRRO++&-KK!6 R1C MV7<0:/G*&$*&!LZG;9XE,5T"^5H0BK'4==3W^]H?0AJ@YANJK$,E9;GQ:=@X M[NAT$2Z658^4QJQ@QE 89DXV3\[4]D @,;A@>S02O899MF]).\BR"U)#X%8V MB15]"46:B.2$ W>$<*SZ%X1^:$+!=F.1A3&Q$WDXVP+XD(3+%T$@W8'Z 2"0D#YB2$"F;78X7,L[+LS_$FHTH;B2'CJ&'% M<]1VXV;F;ZW1!NP_MXXTC#)7M#CXC.E<@84#5.K1?=HSRXOCI?IS:R98],5H M%4.]*.I!26,L6!B-CS[743ZZQ#H6[2)J41J,;-C][Z\>\_8\TPZBW@1,:A0U MJ!L%WN2$&3E915X&T0A$KXM(]!I B/XN"M&8%<&!37*U8D^V&:+=" M[@5*ZQLL>8%?P1JS5T3L"14N@3ML)LB;X 0D#DSLY7(^;]LX!!$2E&1^:5<&LG3&K4@ATN2!"Q8(&78$L('I@7AD#+[J>9%3DJ@0Y- MUCV G--U+#CQBBA/L""WAB:@"ES2@!672"N0ISI:AC8K3FT;[.D%#8U*3%Y0 M?J:X(,J;87\^H95A0(2X90F2G\->]O#*[@5FX9*8@VW5$2*7P41^2<,NS'X- M[ RMS-W/O-$"!"&E6!.0$:-;$% M"#7K^\T* #HF@CE$= U:2$2AHOC(O"J?^!7ORO-$8(.C*)CS)82(%>0B%5H^ M+D"1Q(_1"'..$IB 06B)&"2X%3@2F1%J.3"G1-0X;D:43N=O(LV*K(C;&J5O M7GC!BF_.H15/UVJ#T"^HO">X$D 8H!A1;NM%8I+KTL3A(@;V)119LC;6%H2Q M ;BE'':S*FRM*F&A#FLCD/?S2A.@62"(,'$?=(UA> P33,8F9@PU M )H)60&$ )G$Q 84F2:0;N5(&\QIES&(Z+Y'3R^'K8%-^(A6V*)7TX0$&2AQ M$>,F3X3=&,168C,J,>1M3; O H'5E$NO/ML#E&D7HJ<5X(*:5 #"X2E!CC<]FA 805TBGP!0O(]6:*L$(GFCF%PARWMX@ MN<."%"?'-,"G45Z.T,9J/T'@*V4Y0] A\0N2F\2/-LH9A"U@4FT5)VC203<# M*0))?+"=ZN^6EG,,M""ZE7;'=SNHO(P:[=IC>!.C'49LV%<[12/FIU]!1\L, MHI<^O.W,)_X8I*BP/0Q,[I.*-LLB(._2)HO8'M0X#2= ""DK-M'4Z$+2 =M-?2A')Q"4 M..B)3"6R&VF E2Q2:H0(7+$?;[NA'B=!GG>I)NC7Q@Q;11"0@6-= T&$QH$< ME7<"8L%I>Z1^Q#I5D^[!41N)A'5Z0'"BBTDHH9/ M7T!(S'$.@Q UAF"X.#?)$!*,CU""25EC%8@)+40"BD]E*'%%D!>^_\@LP/W: MV*(7K;61A83[_Z=\$J0@/H@N$LN^; G;-_5V7*IN5=97KGP>J)].I6"#-BSPK9U7I_["+>WWXPR[7 MV_[+[[I\^5V7]#_Z(\F__*[+E]]U^?*[+E]^ N/+[[I\^5V7_X!"_>5W7;[\ MKLN7WW7Y\KLN7W[7Y(,?KRNRZ/-;-??M?ER^^Z_#O]KDLQ*=-\ M6LK?=:&/]G==\IH:#MM6;TC2[W'(?W6AN30-OO3NUT+;4@.% ,@=(>$*S0$/? M$]@=S-TN*^<"Z,WM)[A A=4A*+=F/!I:)E@B:\FHBMB1 =(\ABJ"4$"1.0! MYE4A4V"'-@O8C_*#%\0@3WS0NVZ$2R4MY$[L<0TW 5V!TQ#\;T#-[")ZJ,WI M>P0UWM-W%W*;$1I$A28X%[+D'%3<":U\*0QJYAI=8$"?A.;"F[B/,2$Y%Q)2 M[@B7+?M=*^ 0OT2B$N1(=.I#+<:)[4-@ 0B([2-'GU%>?'%IEWP_O-V_].4% ME!86RCW)]XB-!LX%_82IM1,1D0'DCHZC>%AN^MPC8CN]MS[;QVA'$*/-]K@6 MBD.[,.-S%(/$)VK>QB"+T<;Q(OB<8.*>=4\"DAE]IY>%6$-&J,%#O'+6*2N8 MIVF5N&QUIZ(+(X(XL;_596AD!M;(;A_E(CM%.S410C F>!%N%HF]?R(T8+5J M=!VB84%M@)9&,3D:4QWTQ";06BR)J/0RWV=(.U_N"N[/".*C,;MS0FF'B+?H M"'2 L'$G<&@V24ZQ=6CE&.V2TX*W:%#P,%*C%S20V4;5,9N-$"9#*&A447D] M2*>!RS:@+D(;/02)A3B *")P.2KQ&*0DD!3:D>5K8UXD NDLROZX_1G1_C.) MS %]0".#"OXKDD=XDXXSD2UOOEH%G!/'.9M?+L[!FA$HS*&(EA$]L3GJDB03B&W+/&U MO!D5:>81FQT$7/:XC\& 7_-HXK@=%[(R4)E8Q(:^'1_PZ;GT@32KJ4.'Z9GB M E'%#+(>$TQ[WIH\&83HXJ[7!&T'!80R]"+8_A;L[/QU!WPF.W M J$;GN5W^P.?+@1E8!4T2.MV=+MD%Q@BCZ3"DX M:&5/&PL.Z/PIR1[(LILN4Y M@XH2[G,KSS%>8A7Q\_AL3M]]"$-4@K &42%N#0F>I=V\"R(V W*%-**!<-][ M$Y.G(#F.J$P1([*!L!E((5QYR2>(@I4G5]>R16)5P.>& :2+^3$@1>$$6D-: M/9\-!^XRF?9IRI&_ ;$#2\2L :(YY,*9$_E[Q.M/6\@0PEC] MB",T]/$&0JZU&0@7;(^L/+F&3=[E<-*P<7226$'TV1G& I9F#NTLC0QH+1)O MF'9Y&;%/"PS8^4'H*(!8Y.FNAZI=WKEW2,P[2;H+FQDQ4-'&!R[R7+ZC6 M.41RVFQAMS<\^9H,IQ93[2 '=&EY)O'MLR> ^D&.$8M6,&@=B&I]EPW!147? MB13L(",J,J7O./&P8-9]4#]"VN@)BD$WY@H!>FX0Y]QD(&)3-,+;^R(+Y'/6 M-:0-6$IO>\,!? X:D3OVR&:,CH7,7>45*%8Q,7 VH=1[D'KZ"Q6#H9^:R/?] MN1-(N>N5+:!,&-1GAL=U'N%6'50B_!6*,L8M7TGS( 0T$Y5!S<.ED2.($V0& MLU^O,.$XFZGB,V$&VDXZ68(<^5["08%K)9#WXYW8PD4 M5(?NTR=3 ME'P#Y]*I1/@S&Y"#(DI$\CFP3P!DFC8'8@>H ,03=MEH\?0T>>#?7$>IGPAI M;#:(^C)"W,H5R'#"+&2_+WB3RC_+D!9@5C+8%RD"[?=M>L'$!MTL<0^2R\Q# MQQWB%H/YF@ UNN5L(1TV+R3 R0B1BQBJ%G(.(@AT3PF#XIV!/$C?6=<(]#,E MN8T0V_[X"9V .(@>QY+[N4>E*./& )X7<4!GUPP&CP)YM-GS)$)D;[?*$8M@ MKQNX5BDM=CVH@7"*Q/65N9@RU^P SR;7$4CG.+&!M!NTD!)"C,@,A($Y8<-F M03PMN0;2(@+I(JK][CX;EEHM":INP/@KT)L8Z 1Y3'#QP8S+B.1V$1,<0N0+ M6-7Z%&.(%V/$G@U+;-9&B+M$P63#ACM_PU:A6D0@;0>XYA%W4&Z<4IK[HQ4N MR%4.(%/DOXQ8 =YM]\R("^HXI('-'I)@6Z('B#GZ\*U"[I;J?;-@0.=PD];W M([>\&]!^'_3S3*_=/X&O?D+X==B(J1 MB;BC@:UJ7T4H&&45B7"-FKR"3<]@%P)0"OK; $6N-W$N@J4,=[O,W>G#^,B\ M\J!:X[.W@IIL-++K>7?=9P_+TSQ&7$,S^]PI=V:'K3F"CN-\ <[+1Q;K*B?+ MU@Z5X/-JM?*)6A@;NG+EQ !Q[?JXHZ'B M&S\QE&DQK#*V\A5$M3ZQH^:$=I@YCXNQ/Z8RP=8T^K6@2Q5I$"$!/S:%)1L" MK1@MO U@5P,(M7P8E=)KPS]%@H&^D6$:V#!ZF 0#S5X>#?AIA79"30(YWIO( M-P=%?KZ* $2V'N0B6#Q"1=Q+4X3 M&NC18O:Y*C(AE: 7D6),14I4$2O97@$J5)%AU/)EC!8Q4@S>+B0^=6-BX*.X5\<:(#X1QQJ(KH!LMP %\6]!T%ED@!/>R4!#,- MH8:4*N1>H"OBY^4 KPR 7 X$, MM-^:"8][RQ@$A"** Z%6/Y)K0<$<*Z,CW LK6IS4#'D/;6]8D6OB$G+;H7;OCBX5&F;CK4RQT M1T4,PI"[9[DC?7^1B]J0'7*]B=%^?VH Y1S9C=Q&FSWB M=L* RY/L.%$GO3F?^R6RHJ/O3&R#8U!XQ M13LBJ&WB!I4QHCZ*:S$:^9:S'FQW=+:@BS8B:'U$\KW*A$P,HQ!!!E"'$%L5 M<8D=+8R#BC3#AC#7$[DSGT8A9UU#N\22;7L>*':,ZZ [D<&H27!F.[&G$@/; M9 JE@ETVG0Z55JPHQ8C9 M+[#9>70/VY>K?(;RQ_4C-Z*W%TS+,^5K%K3H^6&OW>$[;8KSEE0D4CRQFYQ MW+#$:E (HMF(/,VP;DG?C:G-ZW-1B,D3]A\M;1 1:KM+0S_LF;:!%P&Y"K(D M@\$,,0AJ(X!*=E!K1J3,P':("I$S-R'+/BM $LC7T)G WW:@X9L@E; >W")_/6+,=L,&+ % M=79J0EH,J]QF!969!P\Z M00<9/+0%4,AC&&!GW30&>H3XMH?=VF"7"_K;&?)4$09>&'SCADL2-LYV]1EM"M3H8I1G7 XSO M;\X-CXO8@MZNJ&-"40#S,VCD3A#>WQA18MR++'@3"H-0<,F!BE",Z:6B"&HH MZ=;@"F?.>HUN]40*)H9WEVEDJ8B$ZCSL-_Q:7G\;G M[%>H$M%$!#/&[@!":*V*WNX!NM@$,0#VC/)O@AA(%U9 3T.<(,AH3NAS+$QF M4^N'+7..%2..DVEI,+#]1P:%[EP40FM@DN,AN?!1\4<8R(/S.)'3B4#O@Q/6 MP41T!G$V*"_'4?:,O\79Y\E0>N6/X+#&1%SS$6X%#2AD#1P:Z8&T@'79H7MR MV,X_(&UBSSQ!&EA+.6ST(:QA+H]TULVK!^_NSC!]RMXQPJ;9,"CSMH+V)<@F]3V3#=QK$FI,0SH.)#&H3@*"+ MH!C/^V,HQCQR0",Z >6<' 41:47?'5US)\C8%0/*^(N@"T(9(K;\D:X[7Q)7 M R]RVGO4L5)K'HFSCD;@YJ6FL$ ' M*-6$46*K?;[N8-$EGL\9I:4+/A-%Z_M!0L>XU\!"W(/#/^E'RH?QD#4C,:[Y MJ 5/ZXBA9K84@R8>I^;O<:OW)\SY200$>5 :(W\5Y\Z@^0B?-T* G]:QR3R6 M97M)C25:$4Q$J,$$[$T&;X(?J(F4+K?B$%NNM8@:RE,$9:X3C#[[T* <-.4 MZN7@L\8T:E[6Q%,8E(-[M!R;[4NJ09GO"347!7D:Q-E!.X?CX]T%"L\.>1,9 M'">(HE]6S/EYH@:U&IW$@,L>]!9-;*LZF$A?6)W<\@D7LC<^\;T0!L4:5P04 MH816N&$-$GD0M;E+$W\A<) ^R-4BAJ)O#0JZ;\GT5"C+8,-&'SLM1/1HU?Q+:0OF.4R M^BP-W[N4A3N!;#8-:!0#/90C'T(D?J!KI"H&DD5,)1!R(T>E\S1S#;J=T*M# M@CAX7//[1M UC\A/[DSD$35VBP)(>@FQ9=X.%GU#%(HERHWY;DMC;@2O5_11 M(:CH?"PH)6*72!N4$R !W+)G'G^5.3%A##5 J73=DX>:1>ET>HZ0&C-C]R#/ MBTK*UIW(597@ 0F-<3ACG%03+&2KHYPR9[LF,,U M_P+8C> 5$%5SW81US:[@H=@@:EBHHK-\A^^944B!$Q.JP5W02%13LP740I23 M",XUHNSOFMK&%$8-[0ID M%-3-1!<$>9XR 'V>S<"JKO_O%08S^%?!2X<_.R><6VIAL,:QH$V2WB#2NO M7GBDQ!H^R#2F"41L]E<16M012(@D9;"]2O$VJ<18)8ORMH!&9 4B44I07&$+ MZ1JVN-G( ]U"VB8@-^":2Y1LHEGPA!V4HPJC$4K729Y& BV;SP,$"XR#( SJ MHO5L5]>'-;2!![LY]QS#0)HT!]H%?6M46B7:!3>= MNJIB)[*]-V:!%IF2^**>QYK?@UHN'N?'7YWCFDN.1MR4U40CEJT>K3CM-54A9CGP)SC$>ZD^'@,X3!\R,03R$4*FHD1Q&&W0!=3![^7^OLB07)#1)P<8,<5,DIB* M( V"IS4BQ&1&Z(8W/F%H49\K!MXNA$,+'D3F)(3(0NR:VF!GAH**H<#K%(V< M#91!&E%'0HX3 (WRP) 9&LU17F)NBR;RDN.RM,_KBZ%BDVAMTTU28V]/-,2G MI&;FTL@E-N\*RQ,8,6TSDR;"QXR$4(/KD#"L*?((,D1"A9AL>H%?R; M8ABX![L0()@.309I$SPA+K=*J/,V%S6K>V6#W.(,\DQN#=[U0\0>^+FKO,2H MU2=V68"ARSW)SJP'G5#ETR7J@(7$/O_2\R(#1.6P&2;,(X:M"%"")Q!F(9&M M2KD-*,$O4?J16SFU%Y) ^H()E8BB A1SE!M D1]?A$46?K8X3: AGBQ/Q]LB M70]!]FVQ)3Y91/5>N/3,9*T6+"-O5T14H@#[51F(@D?R<-U:HU2SY+HFV*)4 M46=4GL2XX*[L0&#BIG#3'&Z$JXOA@&L>7.B'-_58$#%WS(L][DS :$?HB FJF,0ZPFFUK"% M SV( F[Y#Q5RXJ<^\7ETY53J%^-'%RV&$NN5S53/)<]GR&*S^NBC.Q M AD/L'S/ZW)S0/OG7IWQG)J:#U$U3T9H$B3)!$F$_4WUVR JE]Z=4,[*06?O M*5H%P:BU0\WB3 B'[X1R?/Q %'L2X@*WZ7, &1Y.OL"RH=O(]+AT"O(ZON.. MW1&&-68"GM Z4)>'G 0Q1"=1M4+E($R3.@7@1/97!I(%8J MAB$KFZ3?=,F3)S]LNN^W[=U-OSDD22J._=+?OUQ=)6E>9F4J,I$R;=*7S5>O M?_WIZ;0JTL57/[W^\7\419T_S6=UE1;S,DL?TI>+K[YMVJ>;K]X_S64<64ZF ML_0A22>3>965\W123D5X.TTS\9?+?HG)5'PH9$/>?)KN#\WNP N+ZU-U?2/@ M^[:_6FT(PA?I_OUJL=T>+OK;_3))DB?)D_3[U?YVW3P(,-OM>KM+5S?-57^1 MIK]<]_ISNMJGG1[4=^EJ8\9M-^+BU]O]8=_N5K>'],.J_]CO]JDX=;M;;0[R M\^&Z.:3[N]O;[>Z@+WN6;@_7_>[C:M^+RU<'?_)FGU[MFH=]VZS["P'=Y>)N MN>QWZ20O!))BUJNTZY?B\,.A3W/OD)K]7V^;]GU_2$OOU.WJOE_OTVD]OU77),4G3)_+_] ]BEB9U1J1ZZ@L>\;K9-3>]1/.%^IZF MN[Z[,!^O=GV_H2^+]5U/G]?]YNIP_2+=W-TL!&+;96J@NUEM[@3Q-KT\=K@6 M9-%D_FI[>UAM-\WZ*2W=WNUV@IG+U;I//:QW?=-=]_<&Q]OMK?Q?7K"]N=WU M^[V81HC/U4$+7AU=R(/T=N[@Q*P#\U:S;C<[A(>GL&< M CP>_"E=K#9=R/X?I9P[W+/=$_YN^VCHCTNUMOVL.V]WO)02;#[W0R\/653I[Z?\RX/\@9A]"@&$F MD?#8,WC5&1IZ K,3BNOJK:.V_ZY:ZPQ26I9F%\5\+G4)3^?F]&0V5:HFE Z' M%&9(GE?ND \K\>^EI!(-5.JF=?5]?Z#?QK[A,N$O-R9OF]A_D148X=+E=+O6W"]!U&DIB@=1KB!Y0=20'+5RV(LT>^Y> M)00A?_X'-* >976F(&F[7@DITD[TL'W?;Q2)+]6\; C(R:J#'C_')_6F[?K; MPS5.:ZS/G@,E)^\(0BLUPV\(KUR ]:JA@#G!5A @^BPP3)"2LG?55!^1Z1^J MM7.%./)"1E//="#U3,501I&UV.V]"/-QI#8@A+0V)\R;JRAT4BJW.8VLP+A, M#HW%9*-A/]%V,'JV#!YB\3"3B"6N-Q*%UL^C70E"$ MH=RO_LZ'-:G52?\0T>MWBC1TCLT?@)V[]I '6Z. MSU)3O-HZAV@2(>BABDJ"7.V;#_UCS>R]&VLYQQ]0;^\=PO[C5GG4K,P[G.7R M5?_Q.S'-W>[A37N]7KQ__O/VIMFD2R&&R^WF8+FN14(=$Y&$?#>HJN./]EE& M1(:15 M+\C+B?RS"]%$N&!B5IP@+6B@SX2FHO.T )$DH8'$;N!.5I7RSUXXZ>2?%0N$ MJ$+VTL0&DIQH0:@#^R<)<8%6H!61-@@! < JPEPP WRVSGLZ3I 030PD,X(\ MH1D-+7*807NI+4Q( UJB!:%,YW'!@-AH'GQEGC M67%&QXF6OJ#.NL3V='@K5^XMX.J ISIY@J8UP;>*V3.G]9D]G MK.02[J J<[1+9L$%/^_"K#1%P02QR$D0>2'0?D;-%\A9 ^(Q=1_TY](N(>+1 M1+@2+13AULRY\<$CX@R)2! 2:C01"2;+$:P5#D.[8"PP:N,0L9%N:DGS%%0$S-SCI:1+J0% /*& MF$.O!$#/AOV8<(C^0CE!RL8?V6Y6G")JPTR8\F,(";4)X6JY,0P!!EH<9QL( M*L391UFN[+)?/E14CV-B1R!@022BG[9'A17($5ITPW8I;PFR!&E#-"&(B"L8 MK1C5X*R;V4\7T(O#8@/1L%@460"1H1$;W1B7 *))D!T14!GEAQ:(F5@V(ZSMAK@,4NDO8!8L*&!7]]@[D(*#E'=+GH:0Q ,K$A > M9PVXY@!",/[\&T;4B[\8%KBB156(:4*"Y@"M 4!<8=3+$T5 G@(Q%\,"688" M&0,Y)IA&#'+V_<1V8#-'8@C9L$"648$L(Q,OPHF-.](#JH&5W0F"\TV0T\8' M#A$W'(<0-;$!>-Z/_*>QR+^:(21G8#EK@CP!T"T1FB)-'$KF*0?I8I:5- M!?.LHKJ+!!'G$Q?%P6YGP(;=8\4B.9/('<5'(-&?3>P",P(*)GZS9/.VF!\X M210\R>X_4[)EGJ?=#5=O3+AL+HB:E7,M9,RL=.C7SD8!,>#2&$0A9QO[#@=254!<\ZHY%QEN3*JAI%IU$4E.?:,YAB!"E""T*O^1: M+V/5FJ HAW:(!#8F+ERMB;!WBJ4QGV9UA[61H&(>,Q\DT5CBH+VC'$#O?-H5 MT#"03\ T-T&-#2% :X!6@ 42V\BPM0'2JE@%M;1](Z8B8=A<0N6SA/.\[\^; M4&![ZS&B1X*+(FJS<=,R8K.;2 9IN>-ODF%Z'Y_A"N11I.BBF&0G$;! M]F3QMIE!!3$X6T4PT8/P9YK$-MX>RP1JKBA1P!Y+_"#+QNYG AU48H(0)^?) M35Z-01JDHA$B%^#WZLA^?PCJF9 R;9//)6ZTDP62E#JR4QR%_-]/LB,5TCHF MV8\F/A=]L91A7F>;EYB%S(X[FNGS4BPSO.=;3X1RXF)GX**]166/Y&D+&P2CVL&.S*7$1PQZT MK)1!=K%* UA,;HG!/ENJC M9E[<<]LI'B?:T(OMD6]JRUAK\((?"1NYG3H00,P B#;$ M!(!PPG>P(/$,[?*(_,P#4XMV9R2YP5O2;0\[N6CH4"G U$)IOER H/ WXBQM*,@P M-)JC*B5H[ EG*.[6&/WB=;'RX3(B/PWV;6,EH@5VEA .SUGB(WL#7$"@2-Y\ MS^$[;&(6M4U^S&8FF5249 SU0'GS0"#!>W#M=?@^?_LL#4AN*LY%P&;WD3@) M58-M/4:U^3 JMMQCF4'CZ48XB%XILN?-2LA1\,X[;HB+I%>! /K!14&_C\KM M( EP!Y*5@@-R4B4BKEF [FIMF-B@_=A+$[TKD00Z 2)66+=FZD;DB_=I07>X M'ADK7-($UF*:?DB*1B@*(0#0X&'V#=%P'=S?#]R9T'?SFB.*X/L+IA09JIC* M& A:R.\JMI#(!=\B.O>.&M3QR1:,&D&"@3BV#\68PU4_RPTR*UZB5P(J.68" M[(XB>7[P) MB DUD7EUP2U5LPXY>8(?X 3G!?=E8.Z,+^'$S/E-LE@TGENA) MZ87),-**;_.@\=A3@ZX90T';$16QV5!8X!S7D* D,V*(7_%$!J SO4B!U< 0 MHDA>WT;ZUUB^D-B+85=MPQEZ!35_O'O5O"HP7"TR <-GCFK))8_8G0*B%)+T MDC?J#+&C'9@&16XGPCO($R2:.=$19' /.S.#E#K8@X25@R !O0PT.=M-*" N M&K(@MV6JDV1C_(,A'9;L(W)F[9&_>U7PO>D^!/D4S8XAQ0(=)&;;\(2O.!/8 M98.$!_$U)8@T#I]KP[TUM *]H(B'3J#'@ L;O8,4@5]PGS:0H KN/$#[,]Q7 M4N#M^_G(;; Y;$'G<[0"J+0EIE%C=HB?[@&1_]SGFK4_L?HC.,R2W9%/]!Q; MT N,DU"%$MQ,BMDG+$/[Q,^7^ 24H' 984*)JD(EC>!1.[A]BA"B9^;$#^T+ MU48B>5N/3."")C[S(,KVP1@@Y\:!:!4FQOY@QP\+4*>?A))'^K6=)^F,< G2 M^)QOL,1[_+@!%U2$'[X!1YV8%.2XRA>2H&\Z*ARID?"S-5FR8]H/-*I1<$'.EO;)7GX*VG!/:.:C'HGH\B#-&FE$B?9O MVZJ?7S#@,K/ED@?1V<^MF<7B(EH80T4F-M748LUPC##D8/9/H]N<$#:T,L;L$ MW0MI.T)"+AQR$&Z/Y9H_O6+=8EC Q I8 D1NPDXZGW:HM&S8:&D >8E^#M+S MP*Q@!T)!$X/OYQ0TEFHP1!!P%;"''>&BK;PGMD_$HQ4T[2!TC+LT.*"#P TM"JP4,ZHT42QV\E00#'%8*Y! M6MY%4H5 C@@#WA;#S!&Y8GL@AB$:,R-YY+%I&(7D*$>+F./#A-#($T4G=@\R M8JAJG' )W3WTXKR?B!S99>"$$)Z&/B/:T@MUK0XEV&^#'7XN:5FB&4$))@FG M0M0\XM_JH#(:J=WRK7@D/Z@R8R76&=YJ!7YM-B!''BT@0RRQ0I$C;:.FEBI= MF.N27<)G(&))8^9ON(1B '%1PS'D\'U$N5,]'E*A;.%[XC):/T(:Y",DF/-= MA\053(8QYZ7S8&8&V#^2'0&DC<-^?RN9NC* _;'G1'+W*[8-!;GJL.KDN&$W M3]#H1]H:[\H"8S:87B $_=3BB"5UB+2*AIE"%NWLJ#MR-2^?-<'(&)-D]<@&3EDB< MS1O$"4!2^Y#4492QQ8'3+,@@&]"E8$+HC&J#75$(;S(T)P;R"4)N2QH&HF K M&K9^(GTE=73#%W^Q B0_L K4I>$TF'C$KQ %6@@"^6EPFT 8YVPY/1(4SL1^2PP\U:/F3DSH@5A@#,DE5I074-H)27C$E4\XA: KL#L# MY 8A1AI9PQ99L1F[S1ISVAD^)/'NEYJ;!H/F45P#' M2(+&)0]*:GROPQ.3[ZP>]6!WG9^B1\5;8G>$.Y7/[F(6BT:"7PPP*$QC M@AC3_F#?%>]@P2UHJ!864*TI./2#E )^A6$6=);'502L $%*XX!F3>*7Z(.5 MY[$M:/0^F(M4L6H,K1PN;'J/_8&SV).96!S0Q@=WBPVG!L4"(1B^!7361>^% MP.<=1U#EVQI)((MA@9S%>O]RA#R)R TXQ'EA50(ATM\3;%[VPY@6([5J^*&< MBR4&H]6PV5CT$@+R"U4_#&E;Q8XG2;_IDB=/?MATWV_;NYM^B@M_V36K=;]3T_"O AMEZ*;\^OM]OW^M>RC_3UTVHIU_K3R^3_!^6JD+3UQ $ end ------------------------------------------------------------- ---/--- Hans-Christian Hoppe phone : +49-2232-1896-0 / / PALLAS GmbH direct line: +49-2232-1896-11 / / / Hermuelheimer Strasse 10 fax : +49-2232-1896-29 / / / / D-50321 Bruehl email : hch@pallas.de / / / Germany URL : http://www.pallas.de / / PALLAS ------------------------------------------------------------- ---/--- From mpi-core-human@mcs.anl.gov Tue Mar 18 21:31:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA00736; Tue, 18 Mar 1997 21:31:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA07939 for mpi-core-out; Tue, 18 Mar 1997 20:31:20 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA07934 for ; Tue, 18 Mar 1997 20:31:17 -0600 Message-Id: <199703190231.UAA07934@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Winding things up, but not completely Date: Tue, 18 Mar 1997 20:33:08 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This is just to communicate to the whole list the discussion we had at the last Forum meeting on simultaneously terminating and continuing. There seemed to be general agreement that the following was a reasonable plan. Termination ----------- 1. April meeting 2. Chapter authors' editing, as usual 3. Distribution to mpi-core 4. Chapter author's final meeting, which produces current document minus JOD 5. Official Web page, completed in late May or early June a) MPI 1.2 and 2.0 document in postscript, HTML, and downloadable HTML b) JOD c) Pointers to unofficial MPI pages d) Description of Continuation (below) Continuation ------------ 1. Discussion lists continue to exist 2. Chapter authors constitiute an MPI advisory committee 3. Public comment address needed, in addition to comp.parallel.mpi 4. It may happen that the MPI advisory committee will want to poll the Forum to make a decision, although this is not anticipated. If any official votes are needed, eligibility is determined by eligibility at last meeting. 5. Plan single meeting in one year to discuss MPI 2.1, if necessary for corrections or clarifications. Rusty From mpi-core-human@mcs.anl.gov Wed Mar 19 13:54:16 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA14159; Wed, 19 Mar 1997 13:54:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA21361 for mpi-core-out; Wed, 19 Mar 1997 12:51:10 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA21356 for ; Wed, 19 Mar 1997 12:51:06 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id MAA10031; Wed, 19 Mar 1997 12:53:06 -0600 (CST) Date: Wed, 19 Mar 1997 12:53:06 -0600 (CST) Message-Id: <199703191853.MAA10031@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the full document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. As before, the individual chapters are also available (the JOD ones got mangled with the changes so are missing. I may work on this later). The C++ part of the binding chapter and Appendix B have not been updated yet but should be released separately soon. Also, real-time has a meeting and will also be releasing separately. This release is an attempt to make the document look as it will in the final release. As a result, the JOD is now split off and is in the file mpi2-jod.ps.Z. I urge everyone to carefully read this version. It will become increasingly difficult to make lots of changes as time goes on. Steve From mpi-core-human@mcs.anl.gov Wed Mar 19 15:12:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA15856; Wed, 19 Mar 1997 15:12:19 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA23531 for mpi-core-out; Wed, 19 Mar 1997 14:10:01 -0600 Received: from nimble.llnl.gov (nimble.llnl.gov [134.9.13.62]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA23518 for ; Wed, 19 Mar 1997 14:09:53 -0600 Received: by nimble.llnl.gov (4.1/LLNL-1.20) id AA03353; Wed, 19 Mar 97 12:11:53 PST Date: Wed, 19 Mar 97 12:11:53 PST From: trj@nimble.llnl.gov (Terry R. Jones) Message-Id: <9703192011.AA03353@nimble.llnl.gov> To: mpi-core@mcs.anl.gov Subject: glossary Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hello, I think the document would be improved if we included a glossary. Personally, I'd like to see a glossary which includes terms from MPI 1.2 as well as new concepts from MPI 2. Perhaps the chapter authors could contribute any new MPI 2 terms introduced in their chapter... Any thoughts? -terry ----------------------------------------------------------------------------- terry jones | lawrence livermore lab | POB 808 MS: L-61 | The scientific theory I like best is that the Livermore, CA 94550 | rings of Saturn are composed entirely of lost email: trj@llnl.gov | airline luggage. voice: (415) 423-9834 | -- Mark Russel fax: (415) 423-6961 | ----------------------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Wed Mar 19 16:29:01 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA16981; Wed, 19 Mar 1997 16:28:49 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA25254 for mpi-core-out; Wed, 19 Mar 1997 15:27:08 -0600 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA25141 for ; Wed, 19 Mar 1997 15:21:22 -0600 Received: from ptah.erc.msstate.edu (mott@ptah-eth [192.208.143.115]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with ESMTP; id PAA11111; Wed, 19 Mar 1997 15:23:21 -0600 (CST) Received: (from mott@localhost); by ptah.erc.msstate.edu (8.8.5/8.8.5/v1c); id PAA05637; Wed, 19 Mar 1997 15:23:20 -0600 (CST) From: "Michael Mott" Message-Id: <9703191523.ZM5635@unknown.zmail.host> Date: Wed, 19 Mar 1997 15:23:19 -0600 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: mpi-core@mcs.anl.gov Subject: MPI_2 Logo Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk To the MPI list: I'm the designer who created the MPI logo which is now in common usage. I created that design about two years ago, for Tony Skjellum, who has recommended that I post my concept for the MPI_2 logo online for your examination. Two versions can be seen at http://www.erc.msstate.edu/~mott/MPI.html . If you think it fitting, please feel free to grab it for your own usage. Also, please let Tony know your opinion, by emailing him at tony@aurora.cs.msstate.edu. Thanks --Michael Mott -- Michael Mott--mott@erc.msstate.edu or MottGrafix@aol.com -------------------------------------------------------------- http://www.erc.msstate.edu/~mott (Cyberscuro)--Also check out: http://www.erc.msstate.edu/~mott/1_Dirigibiliosity.html http://www.scifi.com/scifi.con/gallery/mottstuff http://www.metatools.com/spotlight/spotlight2.html http://www.ac-online.com/new01.html -------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Wed Mar 19 16:53:22 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA17215; Wed, 19 Mar 1997 16:53:20 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA25858 for mpi-core-out; Wed, 19 Mar 1997 15:51:07 -0600 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA25848 for ; Wed, 19 Mar 1997 15:50:59 -0600 Received: from denna.cs.msstate.edu (denna.cs.msstate.edu [192.208.157.80]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id PAA11992; Wed, 19 Mar 1997 15:52:58 -0600 (CST) Received: by denna.cs.msstate.edu with Microsoft Mail id <01BC347D.CC5ACB70@denna.cs.msstate.edu>; Wed, 19 Mar 1997 15:54:16 -0600 Message-ID: <01BC347D.CC5ACB70@denna.cs.msstate.edu> From: Shane Hebert To: "mpi-core@mcs.anl.gov" Subject: RE: MPI_2 Logo Date: Wed, 19 Mar 1997 15:54:15 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk heh heh heh... That is about the size of it.... MPI^^2 (MPI squared) :-) (pun intended) From: Michael Mott Sent: Wednesday, March 19, 1997 3:23 PM To: mpi-core@mcs.anl.gov Subject: MPI_2 Logo To the MPI list: I'm the designer who created the MPI logo which is now in common usage. I created that design about two years ago, for Tony Skjellum, who has recommended that I post my concept for the MPI_2 logo online for your examination. Two versions can be seen at http://www.erc.msstate.edu/~mott/MPI.html . If you think it fitting, please feel free to grab it for your own usage. Also, please let Tony know your opinion, by emailing him at tony@aurora.cs.msstate.edu. Thanks --Michael Mott -- Michael Mott--mott@erc.msstate.edu or MottGrafix@aol.com -------------------------------------------------------------- http://www.erc.msstate.edu/~mott (Cyberscuro)--Also check out: http://www.erc.msstate.edu/~mott/1_Dirigibiliosity.html http://www.scifi.com/scifi.con/gallery/mottstuff http://www.metatools.com/spotlight/spotlight2.html http://www.ac-online.com/new01.html -------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Wed Mar 19 16:56:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA17239; Wed, 19 Mar 1997 16:56:11 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA25937 for mpi-core-out; Wed, 19 Mar 1997 15:54:12 -0600 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA25928 for ; Wed, 19 Mar 1997 15:54:07 -0600 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.32.106]) by franklin.sdsc.edu (8.8.3/8.8.3/SDSCserver-14) with SMTP id NAA20698; Wed, 19 Mar 1997 13:56:07 -0800 (PST) Received: from localhost by rogue.sdsc.edu (950413.SGI.8.6.12/1.11-client) with SMTP id NAA28510; Wed, 19 Mar 1997 13:56:07 -0800 Date: Wed, 19 Mar 1997 13:56:06 -0800 (PST) From: Richard Frost To: Michael Mott cc: mpi-core@mcs.anl.gov Subject: Re: MPI_2 Logo In-Reply-To: <9703191523.ZM5635@unknown.zmail.host> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk IMHO: I like the MPI^2 part of the logo ... the graphic on the left seems out of place. Perhaps it could be moved (or disapear)? On Wed, 19 Mar 1997, Michael Mott wrote: > To the MPI list: > > I'm the designer who created the MPI logo which is now in common usage. I > created that design about two years ago, for Tony Skjellum, who has recommended > that I post my concept for the MPI_2 logo online for your examination. Two > versions can be seen at http://www.erc.msstate.edu/~mott/MPI.html . > > If you think it fitting, please feel free to grab it for your own usage. Also, > please let Tony know your opinion, by emailing him at > tony@aurora.cs.msstate.edu. > > Thanks > > --Michael Mott From mpi-core-human@mcs.anl.gov Wed Mar 19 17:05:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA17334; Wed, 19 Mar 1997 17:05:35 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA26192 for mpi-core-out; Wed, 19 Mar 1997 16:03:29 -0600 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA26187 for ; Wed, 19 Mar 1997 16:03:25 -0600 Received: from ptah.erc.msstate.edu (mott@ptah-eth [192.208.143.115]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with ESMTP; id QAA12498; Wed, 19 Mar 1997 16:05:18 -0600 (CST) Received: (from mott@localhost); by ptah.erc.msstate.edu (8.8.5/8.8.5/v1c); id QAA05668; Wed, 19 Mar 1997 16:05:17 -0600 (CST) From: "Michael Mott" Message-Id: <9703191605.ZM5666@unknown.zmail.host> Date: Wed, 19 Mar 1997 16:05:17 -0600 In-Reply-To: Richard Frost "Re: MPI_2 Logo" (Mar 19, 1:56pm) References: X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: Richard Frost Subject: Re: MPI_2 Logo Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Thanks. It was orignally designed without the graphic you mentioned. That was a later recommendation (actually added today). As soon as the group decides which way it prefers it, I can provide you with a final version (with or without the rotating icon). Thanks --Michael Mott -- Michael Mott--mott@erc.msstate.edu or MottGrafix@aol.com -------------------------------------------------------------- http://www.erc.msstate.edu/~mott (Cyberscuro)--Also check out: http://www.erc.msstate.edu/~mott/1_Dirigibiliosity.html http://www.scifi.com/scifi.con/gallery/mottstuff http://www.metatools.com/spotlight/spotlight2.html http://www.ac-online.com/new01.html -------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Wed Mar 19 17:06:40 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA17339; Wed, 19 Mar 1997 17:06:38 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA26289 for mpi-core-out; Wed, 19 Mar 1997 16:07:01 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA26282; Wed, 19 Mar 1997 16:06:54 -0600 Message-Id: <199703192206.QAA26282@antares.mcs.anl.gov> To: "Michael Mott" cc: mpi-core@mcs.anl.gov Subject: Re: MPI_2 Logo In-reply-to: Your message of "Wed, 19 Mar 1997 15:23:19 CST." <9703191523.ZM5635@unknown.zmail.host> Date: Wed, 19 Mar 1997 16:08:56 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I have always liked the original version of this logo, which we use on the MPI home page at Argonne (http://www.mcs.anl.gov/mpi). There at least used to be also a black-and-white version of this available at one time from the Miss. State MPI home page. I like the graphic, which is supposed to represent both point-to-point and collective communication. I am happy that Michael has offered it to us. Rusty From mpi-core-human@mcs.anl.gov Wed Mar 19 18:05:09 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA17782; Wed, 19 Mar 1997 18:05:07 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA27883 for mpi-core-out; Wed, 19 Mar 1997 17:04:55 -0600 Received: from bedrock.osc.edu (bedrock.osc.edu [128.146.36.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA27878 for ; Wed, 19 Mar 1997 17:04:51 -0600 Received: for nevin@osc.edu by bedrock.osc.edu (8.8.3/950822.1) id SAA27996; Wed, 19 Mar 1997 18:06:50 -0500 (EST) Date: Wed, 19 Mar 1997 18:06:50 -0500 (EST) Message-Id: <199703192306.SAA27996@bedrock.osc.edu> From: Nick Nevin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: mpi-core@antares.mcs.anl.gov Subject: Re:Winding things up, but not completely In-Reply-To: <199703190231.UAA07934@antares.mcs.anl.gov> References: <199703190231.UAA07934@antares.mcs.anl.gov> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Termination > ----------- > 1. April meeting > 2. Chapter authors' editing, as usual > 3. Distribution to mpi-core > 4. Chapter author's final meeting, which produces current > document minus JOD > 5. Official Web page, completed in late May or early June > a) MPI 1.2 and 2.0 document in postscript, HTML, and > downloadable HTML > b) JOD > c) Pointers to unofficial MPI pages > d) Description of Continuation (below) I would like to see the MPI Home page be situated at a neutral site and not shamelessly funnel people visitors towards certain implementations. ---nick. From mpi-core-human@mcs.anl.gov Thu Mar 20 02:28:15 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA01974; Thu, 20 Mar 1997 02:28:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA03680 for mpi-core-out; Thu, 20 Mar 1997 01:25:11 -0600 Received: from volitans.MorningStar.Com (volitans.MorningStar.Com [137.175.2.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA03675 for ; Thu, 20 Mar 1997 01:25:07 -0600 Received: by volitans.MorningStar.Com (8.7.1/95070701) id CAA08579; Thu, 20 Mar 1997 02:27:09 -0500 (EST) Date: Thu, 20 Mar 1997 02:27:09 -0500 (EST) From: Greg Burns Message-Id: <199703200727.CAA08579@volitans.MorningStar.Com> To: mpi-core@mcs.anl.gov Subject: Re: MPI_2 Logo Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Since people are publicly supporting this logo, I dare to give an opposing view. In the first place, I appreciate the time and effort that the original designer has contributed, probably for free, in producing an MPI logo. We used the logo for a long time. However, I grew tired of its, IMHO, geeky look. (I like the version two aspect of it.) I think we need something completely new. I haven't been able to look at Hans-Christian's suggestion yet. I really like the PTOOLS logo, and was therefore only mildly perturbed when somebody affixed one to our OSC booth at SC96 despite the fact that we had nothing to do with PTOOLS and we build MPI-specific tools. -- Greg From mpi-core-human@mcs.anl.gov Thu Mar 20 05:38:35 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA10918; Thu, 20 Mar 1997 05:38:34 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA05124 for mpi-core-out; Thu, 20 Mar 1997 04:36:14 -0600 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA05119 for ; Thu, 20 Mar 1997 04:36:10 -0600 Received: from jim (gw2-181.pool.dircon.co.uk [194.112.35.181]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id KAA24232; Thu, 20 Mar 1997 10:38:03 GMT Message-Id: <199703201038.KAA24232@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id JAA00568; Thu, 20 Mar 1997 09:34:51 GMT To: Nick Nevin cc: mpi-core@mcs.anl.gov Subject: Re: Winding things up, but not completely In-reply-to: Your message of "Wed, 19 Mar 1997 18:06:50 EST." <199703192306.SAA27996@bedrock.osc.edu> Date: Thu, 20 Mar 1997 09:34:51 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Nick, Why don't you provide the web magic to randomly re-order the list, as happens in the netscape "People" or "Search the web" pages, where the order of the different search engines is different each time you go there... That sounds an easier solution than moving the whole site. (Especially since there is already lots of printed material which references the existing site). -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Thu Mar 20 09:54:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA14500; Thu, 20 Mar 1997 09:54:01 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA07732 for mpi-core-out; Thu, 20 Mar 1997 08:51:45 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA07727 for ; Thu, 20 Mar 1997 08:51:41 -0600 Message-Id: <199703201451.IAA07727@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Michael Resch: MPI_Extent/MPI_UB Question/Comment Date: Thu, 20 Mar 1997 08:53:47 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is a late comment. ------- Forwarded Message From: Resch@RUS.Uni-Stuttgart.DE (Michael Resch) Subject: MPI_Extent/MPI_UB Question/Comment To: mpi-comment@mcs.anl.gov Date: Thu, 20 Mar 1997 15:44:40 +0100 (MEZ) hello, to be able to judge the correctness of an MPI implementation i have two short questions concerning UB and EXTENT ========================================================================= QUESTION 1: ========================================================================= the standard provides a formula EXTNET = UB - LB my personal point of view is that: normally UB and EXTENT are the same. the only need for two values arises from the possibility to choose LB not to be zero. (e.g. EXTENT=8, UB=8 for a type consisting of one float and then one byte or EXTENT=5 and UB=5 for the same type depending on whether alignment is done or not) one might however argue: EXTENT would describe memory used (taking into account alignment) while UB decribes the size in bytes which can differ from EXTENT in that UB could be used to send only bytes used and not bytes that were added to EXTENT to reach alignment boundaries. (e.g. EXTENT =8, UB=5 for a type consisting of one float and then one byte) my question would be: which one comes closer to the standard? and if one is correct, which one is it? personally i find it a bit confusing that the standard initially defines UB depending on an epsilon. then defines EXTENT depending on UB and finally defines the epsilon depending on EXTENT. if my interpretation of the definition is correct i would suggest to say - ------------------------------------------------------------------------ LB = as defined in the standard (MPI1.1) if UB not set explicitely then EXTENT = max_j(displ_j)+sizeof(type_j)+EPS-LB where EPS = as defined in the standard (MPI1.1) UB = LB+EXTENT else EXTENT = UB-LB end if - ------------------------------------------------------------------------ ========================================================================= QUESTION 2 ========================================================================= UB and EXTENT have never come to my knowledge in numerical applications. is there a special need for those things in any numerical libraries that are around (e.g. ScaLAPACK)? regards michael - -- - ------------------------------------------------------------------------------- Michael Resch Phone: ++49 (0)711-685-5834 Rechenzentrum Universitaet Stuttgart Fax: ++49 (0)711-678-7626 Parallel Computing email: resch@rus.uni-stuttgart.de Allmandring 30 D-70550 Stuttgart http://www.uni-stuttgart.de/pcg/pcg/resch/resch.html FRG (RB/1995/LmI) - ------------------------------------------------------------------------------- ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Thu Mar 20 10:17:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA14781; Thu, 20 Mar 1997 10:17:58 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA08267 for mpi-core-out; Thu, 20 Mar 1997 09:15:38 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA08262 for ; Thu, 20 Mar 1997 09:15:35 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id JAA15887; Thu, 20 Mar 1997 09:17:28 -0600 (CST) Date: Thu, 20 Mar 1997 09:17:28 -0600 (CST) Message-Id: <199703201517.JAA15887@rap.cs.wisc.edu> From: Steve Huss-Lederman To: trj@nimble.llnl.gov CC: mpi-core@mcs.anl.gov In-reply-to: <9703192011.AA03353@nimble.llnl.gov> (trj@nimble.llnl.gov) Subject: Re: glossary Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >From you humble editor: I agree a glossary would be nice. However, I don't think it can happen at this point. Not only would some people have to come up with a list of words but they would have to define them. To then go into the MPI-2 document, I strongly feel the Forum would need to agree to them. Given the fact that there is only one more meeting and we have lots to do at it, it seems impractical to do this. A request was also made to integrate the MPI-1.2 document into MPI-1.1. At the last meeting I stated that I did not want this done since one has to be careful about wording and the full Forum should review any such changes. There simply is not the time. A glossary, integration, index to important terms are all very good ideas. I hope they come in future books and reference materials. However, I continue to feel they cannot happen officially in the standard at this time. Steve > I think the document would be improved if we included a glossary. > Personally, I'd like to see a glossary which includes terms from > MPI 1.2 as well as new concepts from MPI 2. > > Perhaps the chapter authors could contribute any new MPI 2 terms > introduced in their chapter... From mpi-core-human@mcs.anl.gov Thu Mar 20 10:35:49 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA15046; Thu, 20 Mar 1997 10:35:47 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA08767 for mpi-core-out; Thu, 20 Mar 1997 09:33:41 -0600 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA08748; Thu, 20 Mar 1997 09:33:27 -0600 Message-Id: <199703201533.JAA08748@antares.mcs.anl.gov> To: Resch@RUS.Uni-Stuttgart.DE cc: mpi-core@antares.mcs.anl.gov Subject: Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment In-Reply-to: Message from Rusty Lusk of "Thu, 20 Mar 1997 08:53:47 -0600." <199703201451.IAA07727@antares.mcs.anl.gov> Date: Thu, 20 Mar 1997 09:35:31 -0600 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | ========================================================================= | QUESTION 1: | ========================================================================= The purpose of EXTENT is to decide where to take the next item. For contiguous data with no holes or padding, it is the same as the size. But this is only true for the simple case, and can mislead. EXTENT was not the best choice of name, but it is too late to fix that. | ========================================================================= | QUESTION 2 | ========================================================================= | | UB and EXTENT have never come to my knowledge in numerical | applications. is there a special need for those things in | any numerical libraries that are around (e.g. ScaLAPACK)? Setting the UB to change the extent is VERY useful in creating a strided vector type where the number of elements sent is specified with the "count" argument to the send/recv. It is also useful in a number of data re-arrangment operations that come up in setting up the data for many numerical calculations. Bill From mpi-core-human@mcs.anl.gov Thu Mar 20 14:11:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA18324; Thu, 20 Mar 1997 14:11:00 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA14316 for mpi-core-out; Thu, 20 Mar 1997 13:08:43 -0600 Received: from Mpi.Mpich.Com (mpi.MPIch.Com [208.201.179.97]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA06928 for ; Thu, 20 Mar 1997 08:03:02 -0600 Received: from Sockets.MPIch.Com (sockets.MPIch.Com [208.201.179.99]); by Mpi.Mpich.Com using SMTP (8.7.6/7.0m-FWP-MsState); id IAA17624; Thu, 20 Mar 1997 08:05:52 -0600 (CST) From: Tony Skjellum Received: by Sockets.MPIch.Com (SMI-8.6/6.0c-FWP); id IAA09433; Thu, 20 Mar 1997 08:05:22 -0600 Date: Thu, 20 Mar 1997 08:05:22 -0600 Message-Id: <199703201405.IAA09433@Sockets.MPIch.Com> To: mpi-core@mcs.anl.gov Subject: Proposal for Permanent "MPI Organization Site" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At the end of MPI-1, I organized the drive for the comp.parallel.mpi. This provided one useful, neutral discussion medium. Over 500 people voted for the site with a dozen or so against, and it has been useful ever since... A new kind of effort is needed now with MPI terminating with no clear intent to resume a Forum (eg., MPI-3). I propose that like BSP worldwide, the mainpage should be moved to "mpi-worldwide.org." Unfortunately, we are not going to have by default an organization capable of supporting the page that does not have actual people with their own emphases. Here is a suggestion that offers MPI vendors (and possibly other orgs) with an opportunity to support a neutral but vital site: I offer that: 1) MPI SoftTech is willing to create a neutral, mpi-worldwide.org site separate from its own pages - 2) Steve and the MPI inner circle of editors define content to a web manager we provide at MPI SoftTech's cost It is important to have a high quality page, not just have a page. 3) MPI SoftTech would maintain the domain name and pages, and 4) take contributions from organziations who are eager to help us defray the costs of particularly item 2/3. By mutual consent, they would be listed on the web page as supporters if they gave (eg, $500 per year subscription to defray cost of web services, machine maintainenance and upgrade for the site). Our firm can provide the server to for immediate years as we expect the hits to be reasonable. Such contributors will get equivalent links from the site to their MPI page, eg, one per vendor. Any such contributions would be fully accounted for in terms of actual costs incurred. The vendor's and other contributors, if any, would get an annual accounting, or as often as otherwise agreed. 5) Any other links that Steve et al decide to have, that link to information about implementations must be even-handed, according to a charter posted on the site, and a simple mechanism for getting listed as a new impl., vendor or product is provided as part of the web mechanism, subject to simple verification by Steve et al, to avoid crank links. Such listings are maintained current by communication between vendors and the web manager. 6) We would want a visible early notice that we provide the community support for the page with our logo, but our link will be with all other vendors. Other contributors will get similar conspicuous notice immediately below that notice. Since Steve H-L himself is not an MPI developer, and would determine content with the advice and consent of the inner circle of editors and the MPI Forum at large, we would have a nice, sustainable site to look at, with appropriate credit to vendors for helping. This seems as neutral as possible, and improves the MPI image as a sustainable activity, without requiring more meetings to prove it :-). What is more, it provides sustainability without government subsidy as the main mechanism, is fully international, and avoids some of us writing proposals periodically for funds to do such support from an otherwise research organization like MSU or Argonne or OSC. -Tony Anthony Skjellum, PhD, President (tony@mpich.com - soon tony@mpi-softtech.com) MPI Software Technology, Inc., Suite 201, 1 Research Blvd., Starkville, MS 39759. (601)320-4300; FAX: (601)320-4301; http://www.mpich.com * The source for MPI support, MPI-2, & High Performance Middleware (tm) * From mpi-core-human@mcs.anl.gov Thu Mar 20 14:39:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA18651; Thu, 20 Mar 1997 14:39:43 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA15219 for mpi-core-out; Thu, 20 Mar 1997 13:37:24 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA15214 for ; Thu, 20 Mar 1997 13:37:21 -0600 Message-Id: <199703201937.NAA15214@antares.mcs.anl.gov> To: mpi-core@mcs.anl.gov Subject: Re: Proposal for Permanent "MPI Organization Site" In-reply-to: Your message of "Thu, 20 Mar 1997 08:05:22 CST." <199703201405.IAA09433@Sockets.MPIch.Com> Date: Thu, 20 Mar 1997 13:39:25 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | A new kind of effort is needed now with MPI terminating with no clear | intent to resume a Forum (eg., MPI-3). | Several months ago, several people began thinking about ways to provide a "continuing existence" for the MPI Forum after it stopped its every-six-weeks meetings. The Center for Research in Parallel Computation, an NSF Science and Technology Center, decided to earmark a small amount of funding for this purpose. The idea would be to pay for a small amount of someone's time to ensure that the official documents and mailing lists were maintained and accessible, and that there be a continuing existence for the Forum. Such a person could also be responsible for organizing future meetings if the Forum wanted to reconvene (for example for the 2.1 meeting we discussed at the last meeting.) DOE has also expressed willingness to back this up if necessary. Since Argonne is a CRPC member, and I am an MPI "something", I was asked by the CRPC executive committee (of which I am not a member) to organize this. I asked Steve Huss-Lederman (the obvious choice) about doing the job and he agreed. This will make it Steve's official job to do what he has been doing a good job of for the last two years, and seems consistent with the "termination and continuation" discussions we had at the last meeting. I think that he should also have the responsibility of maintaining the official MPI Forum Web page, which would contain the official documents, errata, and mailing lists, and little else. I don't think the "official" MPI Forum page should be particularly elaborate, and I now even think that it should not point to any other, unofficial pages, lest anyone become concerned about neutrality of the contents. Rather, anyone who wants to should be welcome to put up their own informative and useful MPI pages whereever they like, and those pages could point to the "official" Forum page for the official documents. I agree that the Form official page should be at a neutral site like mpi-forum.org, and the funding referred to above can be used to support the site in terms of net fees, disk space, etc. I think the Steve's first job should be to set this up. Since I envision the "official" home page to be quite straightforward, we can even vote on its contents at the next meeting. Rusty From mpi-core-human@mcs.anl.gov Thu Mar 20 15:19:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA19596; Thu, 20 Mar 1997 15:19:46 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA16428 for mpi-core-out; Thu, 20 Mar 1997 14:17:07 -0600 Received: from franklin.sdsc.edu (franklin.sdsc.edu [132.249.40.106]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA16423 for ; Thu, 20 Mar 1997 14:17:03 -0600 Received: from rogue.sdsc.edu (rogue.sdsc.edu [132.249.32.106]) by franklin.sdsc.edu (8.8.3/8.8.3/SDSCserver-14) with SMTP id MAA16903; Thu, 20 Mar 1997 12:18:56 -0800 (PST) Received: from localhost by rogue.sdsc.edu (950413.SGI.8.6.12/1.11-client) with SMTP id MAA29087; Thu, 20 Mar 1997 12:18:56 -0800 Date: Thu, 20 Mar 1997 12:18:56 -0800 (PST) From: Richard Frost To: Tony Skjellum cc: mpi-core@mcs.anl.gov Subject: Re: Proposal for Permanent "MPI Organization Site" In-Reply-To: <199703201405.IAA09433@Sockets.MPIch.Com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I'm in full support of the main theme here. A few things to consider: SDSC (and I'm sure others) would support www.mpi***.org at no cost. It seems reasonable that any institution providing such support could also provide access to external personnel for the purpose of editing web pages. The domain name "www.mpi.org" seems a bit more natural. As some of us have experienced with PTools, it's the establishment of the non-profit organization to qualify for xxx.org which requires the most time. - Richard On Thu, 20 Mar 1997, Tony Skjellum wrote: > At the end of MPI-1, I organized the drive for the comp.parallel.mpi. > This provided one useful, neutral discussion medium. Over 500 people > voted for the site with a dozen or so against, and it has been useful > ever since... > > A new kind of effort is needed now with MPI terminating with no clear > intent to resume a Forum (eg., MPI-3). > > I propose that like BSP worldwide, the mainpage should be moved to > "mpi-worldwide.org." Unfortunately, we are not going to have by default an > organization capable of supporting the page that does not have actual > people with their own emphases. Here is a suggestion that offers MPI > vendors (and possibly other orgs) with an opportunity to support a neutral > but vital site: > > I offer that: > 1) MPI SoftTech is willing to create a neutral, mpi-worldwide.org > site separate from its own pages - > > 2) Steve and the MPI inner circle of editors define content to a web > manager we provide at MPI SoftTech's cost > > It is important to have a high quality page, not just have a page. > > 3) MPI SoftTech would maintain the domain name and pages, and > > 4) take contributions from organziations who are eager to help us defray > the costs of particularly item 2/3. By mutual consent, they would > be listed on the web page as supporters if they gave (eg, $500 > per year subscription to defray cost of web services, machine > maintainenance and upgrade for the site). Our firm can provide the > server to for immediate years as we expect the hits to be reasonable. > Such contributors will get equivalent links from the site to their > MPI page, eg, one per vendor. > > Any such contributions would be fully accounted for in terms > of actual costs incurred. The vendor's and other > contributors, if any, would get an annual accounting, or as > often as otherwise agreed. > > 5) Any other links that Steve et al decide to have, that link to > information about implementations must be even-handed, > according to a charter posted on the site, and a simple > mechanism for getting listed as a new impl., vendor or product is > provided as part of the web mechanism, subject to simple > verification by Steve et al, to avoid crank links. Such listings > are maintained current by communication between vendors and the > web manager. > > 6) We would want a visible early notice that we provide the community > support for the page with our logo, but our link will be with all > other vendors. Other contributors will get similar conspicuous > notice immediately below that notice. > > Since Steve H-L himself is not an MPI developer, and would determine > content with the advice and consent of the inner circle of editors and > the MPI Forum at large, we would have a nice, sustainable site to look > at, with appropriate credit to vendors for helping. This seems as > neutral as possible, and improves the MPI image as a sustainable activity, > without requiring more meetings to prove it :-). What is more, it provides > sustainability without government subsidy as the main mechanism, is fully > international, and avoids some of us writing proposals periodically for > funds to do such support from an otherwise research organization like > MSU or Argonne or OSC. > > -Tony > > Anthony Skjellum, PhD, President (tony@mpich.com - soon tony@mpi-softtech.com) > MPI Software Technology, Inc., Suite 201, 1 Research Blvd., > Starkville, MS 39759. (601)320-4300; FAX: (601)320-4301; http://www.mpich.com > * The source for MPI support, MPI-2, & High Performance Middleware (tm) * > > > > From mpi-core-human@mcs.anl.gov Thu Mar 20 16:29:32 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA20704; Thu, 20 Mar 1997 16:29:30 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA18101 for mpi-core-out; Thu, 20 Mar 1997 15:26:43 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA18096 for ; Thu, 20 Mar 1997 15:26:40 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id NAA19970 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Thu, 20 Mar 1997 13:28:46 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id NAA03949; Thu, 20 Mar 1997 13:26:35 -0800 From: "Eric Salo" Message-Id: <9703201326.ZM3947@mrjones.engr.sgi.com> Date: Thu, 20 Mar 1997 13:26:35 -0800 In-Reply-To: Steve Huss-Lederman "Re: glossary" (Mar 20, 9:17am) References: <199703201517.JAA15887@rap.cs.wisc.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: glossary Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk While I am highly sympathetic to concerns about time, we should also remember that there is absolutely nothing at all magical about the deadline we have chosen; it simply marks the last meeting that we have planned, although there is also Rusty's sanity to consider. :-) I agree that a glossary would be too much work just now, as would any attempt to merge MPI-2 with MPI-1. But I think it would be a big mistake for us to not integrate MPI-1.2 into MPI-1.1. If no one else wants to do the merge, I'll volunteer to do it myself (after the next meeting) and then you can all throw rocks at it. Do we really think it necessary to meet again as a whole group just to re-approve a few simple cut-and-paste changes? I have heard arguments that the MPI-1.2 changes could impact other sections of MPI-1, and while this is a serious concern it also seems to me that we should be aware of these before we decide to adapt any clarifications! -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Thu Mar 20 16:39:18 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA20961; Thu, 20 Mar 1997 16:39:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA18362 for mpi-core-out; Thu, 20 Mar 1997 15:36:52 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA18357 for ; Thu, 20 Mar 1997 15:36:48 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id NAA22701 for <@sgi.engr.sgi.com:mpi-core@antares.mcs.anl.gov>; Thu, 20 Mar 1997 13:38:53 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@antares.mcs.anl.gov id NAA03975; Thu, 20 Mar 1997 13:36:42 -0800 From: "Eric Salo" Message-Id: <9703201336.ZM3973@mrjones.engr.sgi.com> Date: Thu, 20 Mar 1997 13:36:41 -0800 In-Reply-To: Rusty Lusk "Michael Resch: MPI_Extent/MPI_UB Question/Comment" (Mar 20, 8:53am) References: <199703201451.IAA07727@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@antares.mcs.anl.gov Subject: Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My understanding is that extent ONLY affects the placement of a datatype when you use it in a call to MPI_Type_contiguous(). That's it. -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Thu Mar 20 17:21:16 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA22192; Thu, 20 Mar 1997 17:21:13 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA19445 for mpi-core-out; Thu, 20 Mar 1997 16:18:54 -0600 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA19440; Thu, 20 Mar 1997 16:18:50 -0600 Message-Id: <199703202218.QAA19440@antares.mcs.anl.gov> To: "Eric Salo" cc: mpi-core@antares.mcs.anl.gov Subject: Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment In-Reply-to: Message from "Eric Salo" of "Thu, 20 Mar 1997 13:36:41 -0800." <9703201336.ZM3973@mrjones.engr.sgi.com> Date: Thu, 20 Mar 1997 16:20:56 -0600 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | My understanding is that extent ONLY affects the placement of a datatype when | you use it in a call to MPI_Type_contiguous(). That's it. But since the effect of the count argument is defined in terms of MPI_Type_contiguous (see "Use of general datatypes in communication"): -- from the standard Thus, MPI_SEND(buf, count, datatype, dest, tag, comm) is equivalent to, MPI_TYPE_CONTIGUOUS(count, datatype, newtype) MPI_TYPE_COMMIT(newtype) MPI_SEND(buf, 1, newtype, dest, tag, comm). Similar statements apply to all other communication functions that have a count and datatype argument. -- end of extract This makes extent fairly pervasive. Bill From mpi-core-human@mcs.anl.gov Thu Mar 20 17:53:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA22446; Thu, 20 Mar 1997 17:53:49 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA20239 for mpi-core-out; Thu, 20 Mar 1997 16:51:37 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA20233 for ; Thu, 20 Mar 1997 16:51:32 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA11251 for <@sgi.engr.sgi.com:mpi-core@antares.mcs.anl.gov>; Thu, 20 Mar 1997 14:53:37 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@antares.mcs.anl.gov id OAA04188; Thu, 20 Mar 1997 14:51:27 -0800 From: "Eric Salo" Message-Id: <9703201451.ZM4186@mrjones.engr.sgi.com> Date: Thu, 20 Mar 1997 14:51:26 -0800 In-Reply-To: William Gropp "Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment" (Mar 20, 4:20pm) References: <199703202218.QAA19440@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@antares.mcs.anl.gov Subject: Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Thus, MPI_SEND(buf, count, datatype, dest, tag, comm) is equivalent to, > > MPI_TYPE_CONTIGUOUS(count, datatype, newtype) > MPI_TYPE_COMMIT(newtype) > MPI_SEND(buf, 1, newtype, dest, tag, comm). Oops. Time to go check my code... -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Thu Mar 20 17:56:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA22491; Thu, 20 Mar 1997 17:56:26 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA20312 for mpi-core-out; Thu, 20 Mar 1997 16:54:22 -0600 Received: from ncsa.uiuc.edu (sdgmail.ncsa.uiuc.edu [141.142.103.66]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA20306 for ; Thu, 20 Mar 1997 16:54:18 -0600 Received: from narnia.ncsa.uiuc.edu (narnia.ncsa.uiuc.edu [141.142.21.165]) by ncsa.uiuc.edu (8.8.5/8.8.5) with SMTP id QAA20812 for ; Thu, 20 Mar 1997 16:56:27 -0600 (CST) Message-Id: <3.0.32.19970320165740.006c7de0@sdgmail.ncsa.uiuc.edu> X-Sender: acheng@sdgmail.ncsa.uiuc.edu X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Thu, 20 Mar 1997 16:57:40 -0600 To: mpi-core@antares.mcs.anl.gov From: Albert Cheng Subject: Re:Winding things up, but not completely Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 06:06 PM 3/19/97 -0500, Nick Nevin wrote: > > 5. Official Web page, completed in late May or early June > > a) MPI 1.2 and 2.0 document in postscript, HTML, and > > downloadable HTML > > b) JOD > > c) Pointers to unofficial MPI pages > > d) Description of Continuation (below) > >I would like to see the MPI Home page be situated at a neutral site and >not shamelessly funnel people visitors towards certain implementations. My interpretation to Rusty's meaning of home page is the home page for the MPI2 specification documentation. Since Rusty has guided this MPI-2 effort all along, people expect to find the official document at his site, Argonne National Lab. Of course, many others have also contributed much to this MPI-2 work, it is still natural to expect A.N.L. to provide the official documents. From mpi-core-human@mcs.anl.gov Thu Mar 20 19:33:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA23503; Thu, 20 Mar 1997 19:33:01 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA22100 for mpi-core-out; Thu, 20 Mar 1997 18:30:53 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA22095 for ; Thu, 20 Mar 1997 18:30:50 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id TAA13598; Thu, 20 Mar 1997 19:22:59 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id TAA55306; Thu, 20 Mar 1997 19:32:56 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256461.0002F811 ; Thu, 20 Mar 1997 19:32:25 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: salo@mrjones.engr.sgi.com cc: mpi-core@mcs.anl.gov Message-ID: <85256461.00029A9E.00@watngi01.watson.ibm.com> Date: Thu, 20 Mar 1997 19:32:39 -0400 Subject: Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment Mime-Version: 1.0 Content-type: text/plain; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Your understanding is wrong (I hope your implementation is better -:) ) Extend appears in the description of all the datatype constructors, since all of them have blocks of contiguous dataytpes, and the meaning of such blocks depends on extend. To: mpi-core @ antares.mcs.anl.gov cc: (bcc: Marc Snir/Watson/IBM Research) Subject: Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment My understanding is that extent ONLY affects the placement of a datatype when you use it in a call to MPI_Type_contiguous(). That's it. -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Thu Mar 20 19:45:31 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA23603; Thu, 20 Mar 1997 19:45:30 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA22276 for mpi-core-out; Thu, 20 Mar 1997 18:43:12 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA22270 for ; Thu, 20 Mar 1997 18:43:08 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id QAA07874 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Thu, 20 Mar 1997 16:45:13 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id QAA04507; Thu, 20 Mar 1997 16:43:02 -0800 From: "Eric Salo" Message-Id: <9703201643.ZM4505@mrjones.engr.sgi.com> Date: Thu, 20 Mar 1997 16:43:02 -0800 In-Reply-To: "Marc Snir" "Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment" (Mar 20, 7:32pm) References: <85256461.00029A9E.00@watngi01.watson.ibm.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: Michael Resch: MPI_Extent/MPI_UB Question/Comment Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Your understanding is wrong (I hope your implementation is better -:) ) Couldn't resist, could you? :-) > Extend appears in the description of all the datatype constructors, since > all of them have blocks of contiguous dataytpes, and the meaning of such > blocks depends on extend. Yes, I know. I consider these to be internal calls to type_contiguous, and as such they are consistent with my original post. But Bill's earlier point is definitely something that I overlooked... -- Eric Salo Silicon Graphics Inc. salo@sgi.com From mpi-core-human@mcs.anl.gov Thu Mar 20 21:37:07 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA24258; Thu, 20 Mar 1997 21:37:06 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA23517 for mpi-core-out; Thu, 20 Mar 1997 20:37:23 -0600 Received: from palrel1.hp.com (palrel1.hp.com [15.253.72.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA23512 for ; Thu, 20 Mar 1997 20:37:20 -0600 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com with ESMTP (8.7.5/8.7.3) id SAA24649 for ; Thu, 20 Mar 1997 18:39:21 -0800 (PST) Message-Id: <199703210239.SAA24649@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA028121959; Thu, 20 Mar 1997 20:39:19 -0600 From: Raja Daoud Subject: MPI fine print (chap. 2) To: mpi-core@mcs.anl.gov Date: Thu, 20 Mar 1997 20:39:19 CST X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Dick and Marc's suggestions, and to avoid having users equate a lack of explicit prohibition to mean a mandate, I propose adding this (or equivalent) text to chapter 2: Unless otherwise stated in the specification of the standard, MPI places no requirements on the result of its interaction with external mechanisms that provide similar or equivalent functionality. This includes, but is not limited to, interactions with external mechanisms for process control, shared and remote memory access, filesystem access and control, interprocess communication, process signaling, and terminal I/O. High quality implementations should strive to make the results of such interactions intuitive to users, and attempt to document restrictions where deemed necessary. Comments? --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/wsg/ssa/mpi/mpihome.html From mpi-core-human@mcs.anl.gov Fri Mar 21 00:41:09 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA25687; Fri, 21 Mar 1997 00:41:09 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA25398 for mpi-core-out; Thu, 20 Mar 1997 23:41:18 -0600 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA25393 for ; Thu, 20 Mar 1997 23:41:13 -0600 Received: by nsgi3.lbl.gov (8.7/ner-1) id VAA05997; Thu, 20 Mar 1997 21:43:20 -0800 (PST) Date: Thu, 20 Mar 1997 21:43:20 -0800 (PST) From: wcs@nersc.gov ( W. Saphir) Message-Id: <199703210543.VAA05997@nsgi3.lbl.gov> To: Greg Burns Subject: Re: MPI_2 Logo Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Since people are publicly supporting this logo, I dare to give an > opposing view. I second your view. I too appreciate the effort here. The logo doesn't quite work for me, though. In an attempt to be constructive, I've put an idea at http://www.nersc.gov/research/FTG/wcs/tshirt.html. This is a bit hokey, but maybe conveys a bit about what MPI is and also has an easily-recognized symbol. I'm beginning to think we should have *two* MPI t-shirts. One should be essentially a joke t-shirt - perhaps the shoes in front and the 10-best in the back. But we also need a more serious t-shirt, for those occasions when we want to promote MPI. The idea above would be for this "serious" version (and possibly the web site). I approve in principal of Eric's top-ten (possibly modified according to subsequent suggestions) for the non-serious shirt. For the shoes picture, I particularly like the MPI-2 shoe. I would suggest moving the second pair of shoes to the first slot (possibly losing the heel) and put pair of normal dress shoes in the second slot. Bill From mpi-core-human@mcs.anl.gov Fri Mar 21 10:50:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA02093; Fri, 21 Mar 1997 10:50:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA02056 for mpi-core-out; Fri, 21 Mar 1997 09:45:36 -0600 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA02051 for ; Fri, 21 Mar 1997 09:45:32 -0600 Date: Fri, 21 Mar 1997 09:47:42 -0600 Message-Id: <199703211547.JAA11657@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov In-reply-to: <199703210543.VAA05997@nsgi3.lbl.gov> (wcs@nersc.gov) Subject: Re: MPI_2 Logo Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I like the colorful upper one of Michael Mott's logos, but I'd prefer if the 2 made it look like MPI 2 rather than MPI squared. The logo really comes across as MPI squared. Rajeev From mpi-core-human@mcs.anl.gov Fri Mar 21 11:43:22 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA03053; Fri, 21 Mar 1997 11:43:21 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA03472 for mpi-core-out; Fri, 21 Mar 1997 10:40:57 -0600 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA03467 for ; Fri, 21 Mar 1997 10:40:52 -0600 Date: Fri, 21 Mar 1997 10:42:59 -0600 Message-Id: <199703211642.KAA11715@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov Subject: Re: MPI_2 Logo Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I'm beginning to think we should have *two* MPI t-shirts. > One should be essentially a joke t-shirt - perhaps the shoes > in front and the 10-best in the back. But we also need a > more serious t-shirt, for those occasions when we want > to promote MPI. If we have two versions, we have no control over who will wear which version on which occasion. The top-10 slogans are a good internal joke within the MPI Forum, but by having them on a T-shirt, we would, in my opinion, be advertizing the wrong message to the public. Rajeev From mpi-core-human@mcs.anl.gov Fri Mar 21 12:36:33 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA03552; Fri, 21 Mar 1997 12:36:32 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA04646 for mpi-core-out; Fri, 21 Mar 1997 11:36:03 -0600 Received: from nimble.llnl.gov (nimble.llnl.gov [134.9.13.62]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA04632; Fri, 21 Mar 1997 11:35:58 -0600 Received: by nimble.llnl.gov (4.1/LLNL-1.20) id AA04327; Fri, 21 Mar 97 09:38:07 PST Date: Fri, 21 Mar 97 09:38:07 PST From: trj@nimble.llnl.gov (Terry R. Jones) Message-Id: <9703211738.AA04327@nimble.llnl.gov> To: mpi-core@mcs.anl.gov, thakur@mcs.anl.gov Subject: Re: MPI_2 Logo Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >I like the colorful upper one of Michael Mott's logos, but >I'd prefer if the 2 made it look like MPI 2 rather than MPI squared. >The logo really comes across as MPI squared. I like the MPI squared, and I prefer color to black-and-white, but I'm not too crazy about the graphic on the left. I like some aspects of Bill Saphir's graphic better -- but I *don't* like the chattering teeth at the center. Furthermore, I think the MPI 2 is not immediately obvious. If we went with a variation on this theme, I'd support adding the MPI squared to the side and removing the M P I from the graphic. I think it would be a plus if I/O and communication were represented in the MPI 2 graphic. Perhaps we could replace the chattering teeth in Bill's graphic with the symbol for secondary storage (overlapped ovals) and have bi-directional arrows going from the 3 workstations to the storage... I like the idea of a joke t-shirt and a t-shirt used to promote MPI. -terry ----------------------------------------------------------------------------- terry jones | lawrence livermore lab | POB 808 MS: L-61 | New York... when civilization falls apart, Livermore, CA 94550 | remember, we were way ahead of you. email: trj@llnl.gov | voice: (510) 423-9834 | - David Letterman fax: (510) 423-6961 | ----------------------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Fri Mar 21 13:31:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA04566; Fri, 21 Mar 1997 13:31:07 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA05753 for mpi-core-out; Fri, 21 Mar 1997 12:28:54 -0600 Received: from xipe-totec.cacr.caltech.edu (root@xipe-totec.cacr.caltech.edu [131.215.145.166]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA05748 for ; Fri, 21 Mar 1997 12:28:50 -0600 Received: from willow.cacr.caltech.edu (rajesh@willow.cacr.caltech.edu [131.215.151.104]) by xipe-totec.cacr.caltech.edu (8.8.5/8.6.4) with SMTP id KAA28385 for ; Fri, 21 Mar 1997 10:29:48 -0800 (PST) Received: by willow.cacr.caltech.edu (8.6.12) id KAA20433; Fri, 21 Mar 1997 10:32:29 -0800 Date: Fri, 21 Mar 1997 10:32:19 -0800 (PST) From: "Rajesh R. Bordawekar" X-Sender: rajesh@willow To: mpi-core@mcs.anl.gov Subject: Re: MPI_2 Logo In-Reply-To: <199703211642.KAA11715@abacus> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > > I'm beginning to think we should have *two* MPI t-shirts. > > One should be essentially a joke t-shirt - perhaps the shoes > > in front and the 10-best in the back. But we also need a > > more serious t-shirt, for those occasions when we want > > to promote MPI. > I agree completely. On the serious T-shirt, we can have a artwork illustrating a MPI operation (similar to the FORALL execution graph on the HPF T-shirt...) rajesh From mpi-core-human@mcs.anl.gov Mon Mar 24 18:47:22 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA18108; Mon, 24 Mar 1997 18:47:21 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA25134 for mpi-core-out; Mon, 24 Mar 1997 17:44:01 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA25129 for ; Mon, 24 Mar 1997 17:43:57 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id RAA17050; Mon, 24 Mar 1997 17:46:27 -0600 (CST) Date: Mon, 24 Mar 1997 17:46:27 -0600 (CST) Message-Id: <199703242346.RAA17050@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: input deadline Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This is just a quick note to let everyone know that I need the new input for final cut of the document to be handed out at the meeting (23-25 April) by Tuesday, 15 April at 10:00 AM Central US time. We are in the process of determining the schedule for the completion of the MPI document. I will send it around once it is worked out. Steve From mpi-core-human@mcs.anl.gov Tue Mar 25 11:53:56 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA05916; Tue, 25 Mar 1997 11:53:55 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA09454 for mpi-core-out; Tue, 25 Mar 1997 10:51:55 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA09449 for ; Tue, 25 Mar 1997 10:51:50 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id KAA21950; Tue, 25 Mar 1997 10:54:23 -0600 (CST) Date: Tue, 25 Mar 1997 10:54:23 -0600 (CST) Message-Id: <199703251654.KAA21950@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov In-reply-to: <199703210239.SAA24649@palrel1.hp.com> (message from Raja Daoud on Thu, 20 Mar 1997 20:39:19 CST) Subject: Re: MPI fine print (chap. 2) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Since there has not been any objection to Raja's proposed change, I have added it to section 2.7 of the current document. The new text reads: ---------------------------------------------------------------------- \section{Processes} An \MPI/ program consists of autonomous processes, executing their own code, in an MIMD style. The codes executed by each process need not be identical. The processes communicate via calls to \MPI/ communication primitives. Typically, each process executes in its own address space, although shared-memory implementations of \MPI/ are possible. \begchangeapr This document specifies the behavior of a parallel program assuming that only \MPI/ calls are used. The interaction of an \MPI/ program with other possible means of communication, I/O, and process management is not specified. Unless otherwise stated in the specification of the standard, \MPI/ places no requirements on the result of its interaction with external mechanisms that provide similar or equivalent functionality. This includes, but is not limited to, interactions with external mechanisms for process control, shared and remote memory access, filesystem access and control, interprocess communication, process signaling, and terminal I/O. High quality implementations should strive to make the results of such interactions intuitive to users, and attempt to document restrictions where deemed necessary. \begin{implementors} Implementations that support such additional mechanisms for functionality supported within \mpi/ are expected to document how these interact with \mpi/. \end{implementors} \endchangeapr \begchangeapr The interaction of \mpi/ and threads is defined in section~\ref{sec:ei-threads}. \endchangeapr ---------------------------------------------------------------------- For reference, the current text is: ---------------------------------------------------------------------- \section{Processes} An \MPI/ program consists of autonomous processes, executing their own code, in an MIMD style. The codes executed by each process need not be identical. The processes communicate via calls to \MPI/ communication primitives. Typically, each process executes in its own address space, although shared-memory implementations of \MPI/ are possible. This document specifies the behavior of a parallel program assuming that only \MPI/ calls are used for communication. The interaction of an \MPI/ program with other possible means of communication (e.g., shared memory) is not specified. \begchangeapr \begin{implementors} The interaction of an \mpi/ program with other possible means of communication between \mpi/ processes, such as shared memory or sockets, is not specified by this document. Implementations that support such additional communication means are expected to document how these interact with \mpi/ communication. \end{implementors} \endchangeapr \begchangeapr The interaction of \mpi/ and threads is defined in section~\ref{sec:ei-threads}. \endchangeapr ---------------------------------------------------------------------- If you have any comments let us know. Steve From mpi-core-human@mcs.anl.gov Wed Mar 26 10:21:05 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA20390; Wed, 26 Mar 1997 10:21:03 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA29663 for mpi-core-out; Wed, 26 Mar 1997 09:19:23 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA29658 for ; Wed, 26 Mar 1997 09:19:20 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id JAA28441; Wed, 26 Mar 1997 09:21:50 -0600 (CST) Date: Wed, 26 Mar 1997 09:21:50 -0600 (CST) Message-Id: <199703261521.JAA28441@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document corrections to be made Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi, I have looked over the appendix and chapters to make a quick check of the function definitions. Here are a list of problems. I also include a few other small things. I am including core on this mail so everyone can see the corrections already noted. As you can see the document still needs careful reading! Thanks, Steve P.S. - Can each chapter author make these changes. Intro ----- 1:22 - text runs over line Terms ----- 14:9,15 - text runs over line Misc-1.2 -------- 34:4 - text runs over line Dynamic ------- 59:115 - I think INFO should be of type INTEGER as it is everywhere else 62:13 - Lookup should be lower case in the C++ binding One-sided --------- 76:39 - There is a extra C++ binding for Mem_alloc under the Mem_free declaration. Needs deletion. 76:44 - Need to add "INTEGER IERROR" line the Fortran declaration of MEM_FREE. Collective ---------- 112:32 - line too long pp. 112-127 - All the MPI-1 functions that now can take intercommunicators only have language independent bindings. I suspect we want to add all the language bindings too. I think it looks funny to only have one however I see how the bindings are the same as MPI-1 except the communicator can be either type now. We need to decide how we are going to list these. (MPI_BARRIER is the exception. It also has the C binding.) External -------- 134:36 - add MPI_ prefix to query_fn 135:5 - add MPI_ prefix to free_fn 135:17 - add MPI_ prefix to cancel_fn 146:28 - line too long 148:6,18 - lines too long 152:38 - Fortran binding has THREAD_QUERY instead of THREAD_INIT 154:38 - C binding need capital of first letter so it is MPI_Thread_main 158:34 - Fortran binding missing the FLAG argument 160:1 - The C++ binding uses the C macro so it shows up in the wrong list in the appendix 161:34,42 - SUBROUTINE shows up twice Bindings -------- 170:31 - line too long Misc-2 ------ 187+ - lots of the transfer functions are messed up in the text or the appendix. It is because the macro used takes two arguments and only one was given. (fix the ones with mpiemptybind) 197:19,42;198:23,35 - the new constructors and address functions are missing C++ bindings (4 routines) 203:3 - As noted, need C++ for PACK and need UNPACK definitions. I/O --- 210:1 - should the C++ binding really be MPI::File::Delete instead of MPI::File_delete? 243:13+ - I thing the register function should be MPI_SET_REGISTERED_FILTER. Fits naming scheme and then like the get function. 244:1,5,30 - All C version should have MPI_ prefix. The Fortran and C++ bindings are missing. 245:31,36,44 - missing get_ in all language bindings From mpi-core-human@mcs.anl.gov Wed Apr 2 12:14:54 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA05601; Wed, 2 Apr 1997 12:14:53 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA14113 for mpi-core-out; Wed, 2 Apr 1997 11:13:43 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA14107; Wed, 2 Apr 1997 11:13:31 -0600 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA22047; Wed, 2 Apr 1997 12:15:28 -0500 Message-Id: <3342947B.41C6@kgn.ibm.com> Date: Wed, 02 Apr 1997 12:16:43 -0500 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 Cc: mpi-core@mcs.anl.gov, mpi-impl@mcs.anl.gov Subject: Intel Suite vs. MPI_GRAPH_CREATE Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am looking for comments from anyone who understands the intent and usefulness of MPI_GRAPH_CREATE well enough to express an opinion on one of the "errors" the Intel Test Suite expects will be detected. The Intel test calls MPI_GRAPH_CREATE with an edge which is a direct loop back for one node. (Intel's comment header in the testcase calls this a NULL edge). Specifically, the test has a edge for node 0 which is directed to node 0. Is providing such an edge to MPI_GRAPH_CREATE 1) reasonable and useful 2) meaningful but a bit strange 3) well defined but pointless (if a program did it, the user really must have made a mistake) 4) so bogus that a program which tries has no right to continue I should add that the testcase goes on to specify a broken graph after the loopback edge is allowed. This is due to a shortcut in creating the test. The test takes the example in MPI-1.1 (sec 6.5.3) and changes edges[0] from 0->1 into 0->0 while leaving the reverse edge 1->0 in the graph. Thanks - Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Wed Apr 2 14:51:35 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA10861; Wed, 2 Apr 1997 14:51:33 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA19347 for mpi-core-out; Wed, 2 Apr 1997 13:52:41 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA19342 for ; Wed, 2 Apr 1997 13:52:38 -0600 Message-Id: <199704021952.NAA19342@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Institutions Date: Wed, 02 Apr 1997 13:55:55 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am trying to assemble a list of institutions to credit at the beginning of the document. I have made a very preliminary cut at this, causing me to realize how many acronyms I don't know the meaning of. This is sorted by institution name. If you have come to one of the MPI Forum meetings at your institution's expense, your institution should be here. If it isn't, or if I have mangled its name in some way, please send me a correction. Thanks, Rusty ? Peter Brennan ? Zhenqian Cui Argonne National Laboratory Bill Gropp Argonne National Laboratory Rajeev Thakur Argonne National Laboratory Rusty Lusk California Institute of Technology Jerrel Watts California Institute of Technology Rajesh Bordawekar Cray Research Heidi Poxon Cray Research Laurie Costello Cray Research Margaret Cahir Digital Equipment Corporation Ed Benson Dolphin Interconnect Solutions, Inc. Jim Cownie Edinburgh Parallel Computing Centre Lyndon Clarke GMD Klaus Wolf Hewlett-Packard Raja Daoud Hewlett-Packard Steve Landherr Hitachi Mark Fallon Hughes Aircraft Corporation Lloyd Lewins Intel Corporation Harish Nag Intel Corporation Joel Clark International Business Machines Dick Treumann International Business Machines Jean-Pierre Prost International Business Machines Marc Snir International Business Machines xDick Treumann Khoral Research Steve Kubica Lawrence Livermore National Laboratory Elsie Pierce Lawrence Livermore National Laboratory John May Lawrence Livermore National Laboratory Terry Jones Los Alamos National Laboratory MaryDell Tholburn Los Alamos National Laboratory Tyce McClarty Mississippi State University Boris Protopopov Mississippi State University Tony Skjellum Mitre Corporation Arkady Kanevsky National Aeronautics and Space Administration Bill Nitzberg National Aeronautics and Space Administration Ian Stockdale National Aeronautics and Space Administration Parkson Wong National Energy Research Supercomputing Center Bill Saphir National Institute of Standards and Technology John Hagedorn National Institute of Standards and Technology Judy Devaney National Oceanic and Atmospheric Adminstration Leslie Hart Nippon Electronics Corporation Koichi Konishi Nippon Electronics Corporation Rolf Hempel Nippon Electronics Corporation Susan Kraus Oak Ridge National Laboratory Al Geist Ohio State University Nick Nevin PALLAS GmbH Hans-Christian Hoppe Pratt & Whitney Pete Bradley Rechenzentrum Universitaet Stuttgart Rolf Rabenseifner Sanders Nathan Doss Sandia National Laboratory Pang Chen Sandia National Laboratory Ron Brightwell Silicon Graphics Incorporated Eric Salo Sky Computers Anna Rounbehler Sun Microsystems David Taylor Sun Microsystems Terry Dontje United States Navy Dennis Cottel University of California at San Diego Richard Frost University of Denver Robert Babb University of Houston Darren Sanders University of Illinois Albert Cheng University of Illinois Ying Chen University of Maryland Manuel Ujaldon University of New Mexico Tom Robey University of New Mexico Tom Robey University of Notre Dame Andrew Lumsdaine University of Notre Dame Jeff Squyres University of Wisconsin Steve Huss-Lederman From mpi-core-human@mcs.anl.gov Wed Apr 2 16:28:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA12314; Wed, 2 Apr 1997 16:28:17 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA23464 for mpi-core-out; Wed, 2 Apr 1997 15:28:23 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA23404; Wed, 2 Apr 1997 15:27:05 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id PAA18801; Wed, 2 Apr 1997 15:30:19 -0600 (CST) Date: Wed, 2 Apr 1997 15:30:19 -0600 (CST) Message-Id: <199704022130.PAA18801@rap.cs.wisc.edu> From: Steve Huss-Lederman To: treumann@kgn.ibm.com CC: mpi-core@mcs.anl.gov, mpi-impl@mcs.anl.gov In-reply-to: <3342947B.41C6@kgn.ibm.com> (message from Dick Treumann on Wed, 02 Apr 1997 12:16:43 -0500) Subject: Re: Intel Suite vs. MPI_GRAPH_CREATE Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I am looking for comments from anyone who understands the intent and > usefulness of MPI_GRAPH_CREATE well enough to express an opinion on one > of the "errors" the Intel Test Suite expects will be detected. I was at the subcommittee meeting where these functions were set so I will try to relate what I think was the intent from my (biased) position. > The Intel test calls MPI_GRAPH_CREATE with an edge which is a direct > loop back for one node. (Intel's comment header in the testcase calls > this a NULL edge). Specifically, the test has a edge for node 0 which > is directed to node 0. > > Is providing such an edge to MPI_GRAPH_CREATE > 1) reasonable and useful > 2) meaningful but a bit strange > 3) well defined but pointless (if a program did it, the user really must > have made a mistake) > 4) so bogus that a program which tries has no right to continue I would answer 2/3. Most likely 3 but 2 is possible. MPI by its definition supports a fully connected graph of comnumication, e.g., you can send a message between any pairs of processors. The topology functions offer two things. First, it provides the user with a standard and easy method of defining and getting topologies. Second, it allows MPI to optimize placement of processes in the group to optimize these communications. There was a lot of discussion about achieving the second goal with the current functions. Even taking a simple case of saying you have a 2D grid does not really tell you the important communication patterns or when they occur. Are the along rows, columns, diagonals, all-to-all and do they occur simultaneously. My feelings about this were described on p. 253 and 254 of "MPI: The Complete Reference." What does this all mean? Saying you are going to send messages to yourself really isn't very interesting. However, if you look at it from the user wanting to define a topology, then it is legitimate to say this and why should MPI reject it? If you view it as a hint to MPI for optimizing process placement, then it isn't a fatal mistake - MPI can ignore this information. (In reality I suspect most implementations don't optimize process placement because of what I said above.) Thus, I think it should not be an error and certainly not fatal. > I should add that the testcase goes on to specify a broken graph after > the loopback edge is allowed. This is due to a shortcut in creating the > test. The test takes the example in MPI-1.1 (sec 6.5.3) and changes > edges[0] from 0->1 into 0->0 while leaving the reverse edge 1->0 in the > graph. I think the text here is clear. This is an erroneous call. The graph is supposed to be symmetric. The full graph is given for ease of usage and error checking. Whether an implementation catches this error is another question. Steve From mpi-core-human@mcs.anl.gov Thu Apr 3 02:17:45 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA19197; Thu, 3 Apr 1997 02:17:44 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA07906 for mpi-core-out; Thu, 3 Apr 1997 01:17:53 -0600 Received: from sun1.ccrl-nece.technopark.gmd.de (sun1.ccrl-nece.technopark.gmd.de [193.175.160.67]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA07901; Thu, 3 Apr 1997 01:17:47 -0600 Received: from sgi7.ccrl-nece.technopark.gmd.de (sgi7.ccrl-nece.technopark.gmd.de [193.175.160.89]) by sun1.ccrl-nece.technopark.gmd.de (8.7/3.4W296021412) with SMTP id JAA12526; Thu, 3 Apr 1997 09:19:56 +0200 (MET DST) Received: (from hempel@localhost) by sgi7.ccrl-nece.technopark.gmd.de (950413.SGI.8.6.12/950213.SGI.AUTOCF) id JAA02042; Thu, 3 Apr 1997 09:27:15 +0200 Date: Thu, 3 Apr 1997 09:27:15 +0200 From: hempel@ccrl-nece.technopark.gmd.de (Rolf Hempel) Message-Id: <199704030727.JAA02042@sgi7.ccrl-nece.technopark.gmd.de> To: treumann@kgn.ibm.com Subject: Re: Intel Suite vs. MPI_GRAPH_CREATE Cc: mpi-impl@mcs.anl.gov, mpi-core@mcs.anl.gov, hempel@ccrl-nece.technopark.gmd.de Reply-To: hempel@ccrl-nece.technopark.gmd.de Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As the MPI-1 subcommittee chairman of the process topology chapter, I was very much involved in the related discussions. As far as I remember, the question raised by Dick (and the Intel test suite) has never been brought up at the meetings. I guess we all tacitly assumed that the graph does not contain any loop back edge. Now, thinking about it again, I suggest that including such an edge should not be treated as a fatal error. I would put this into the same category as sending a message to yourself which - as far as I can see - is not forbidden by the standard either. I don't expect a user to explicitly create a topology with a loop back edge. It could, however, be the result of some automatic topology creation. Some algorithm could set up a communication structure, including loop back messages, and then weight the process links with the number of message to be exchanged. This, then, could be the input for the graph_create function. I admit that this is a highly speculative scenario, but I don't see why the call to the topology function should be forbidden while the send-to-yourself is not. So, I think that option 2) in Dick's list is most appropriate. Since an implementation of graph_create can just ignore a loop back edge, it does not further complicate implementations. Needless to say, the non-symmetric graph supplied by the Intel suite is, as Steve already pointed out, definitively violating the standard. By the way: I don't agree with the statement, repeated many times, that the optimizations for process placement are useless on most machines. At the time when MPI-1 was defined it was more valid than with today's collection of hardware platforms. For our clustered SX-4 computer, for example, we are looking into ways to minimize the inter-cluster communication, and the topology functions will help us very much. I expect the same to be true for networks with SMP nodes. - Rolf ------------------------------------------------------------------------ Rolf Hempel (email: hempel@ccrl-nece.technopark.gmd.de) C&C Research Laboratories, NEC Europe Ltd., Rathausallee 10, 53757 Sankt Augustin, Germany Tel.: +49 (0) 2241 - 92 52 - 95 Fax: +49 (0) 2241 - 92 52 - 99 From mpi-core-human@mcs.anl.gov Thu Apr 3 10:45:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA23892; Thu, 3 Apr 1997 10:45:49 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA15981 for mpi-core-out; Thu, 3 Apr 1997 09:46:59 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA15976 for ; Thu, 3 Apr 1997 09:46:56 -0600 Message-Id: <199704031546.JAA15976@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: institutions, second version Date: Thu, 03 Apr 1997 09:50:23 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear group, I received many corrections, additions, and clarifications after yesterday's draft list of institutions. Please check and see whether or not this now is beginning to resemble a complete list of the institutions (now with their correct names) that supported MPI-2 by sending their employees to meetings. I think it is also getting close to a complete list of people, as well. If you detect any omissions, please let me know. Thanks. Rusty Argonne National Laboratory Bill Gropp Argonne National Laboratory Rajeev Thakur Argonne National Laboratory Rusty Lusk California Institute of Technology Jerrel Watts California Institute of Technology Rajesh Bordawekar Convex Computer Corporation Steve Fleischman Cray Research Heidi Poxon Cray Research Laurie Costello Cray Research Margaret Cahir Digital Equipment Corporation Ed Benson Dolphin Interconnect Solutions, Inc. Jim Cownie Edinburgh Parallel Computing Centre Lyndon Clarke GEC-Marconi Research Centre Bob Madahar GMD Klaus Wolf General Electric Craig Fischberg Hewlett-Packard Greg Astfalk Hewlett-Packard Raja Daoud Hewlett-Packard Steve Landherr Hitachi Mark Fallon Hughes Aircraft Company Lloyd Lewins Intel Corporation Harish Nag Intel Corporation Joel Clark International Business Machines Dick Treumann International Business Machines Jean-Pierre Prost International Business Machines Marc Snir Khoral Research Steve Kubica Khoral Research Tom Robey Lawrence Livermore National Laboratory Elsie Pierce Lawrence Livermore National Laboratory John May Lawrence Livermore National Laboratory Terry Jones Los Alamos National Laboratory MaryDell Tholburn Los Alamos National Laboratory Tyce McLarty MPI Software Techology, Inc. Peter Brennan Mississippi State University Boris Protopopov Mississippi State University Shane Hebert Mississippi State University Tony Skjellum Mississippi State University Zhenqian Cui NEC Corporation Koichi Konishi NEC Corporation Rolf Hempel NEC Corporation Susan Kraus National Aeronautics and Space Administration Bill Nitzberg National Aeronautics and Space Administration Ian Stockdale National Aeronautics and Space Administration Parkson Wong National Energy Research Scientific Computing Center Bill Saphir National Institute of Standards and Technology John Hagedorn National Institute of Standards and Technology Judy Devaney National Oceanic and Atmospheric Adminstration Leslie Hart National Oceanic and Atmospheric Adminstration Tom Henderson Oak Ridge National Laboratory Al Geist Ohio State University Nick Nevin PALLAS GmbH Hans-Christian Hoppe Pacific Northwest National Laboratory Jarek Nieplocha Pratt & Whitney Dave Wright Pratt & Whitney Pete Bradley San Diego Supercomputer Center Richard Frost Sanders, A Lockheed-Martin Company Nathan Doss Sandia National Laboratories Pang Chen Sandia National Laboratories Ron Brightwell Schlumberger Anne Elster Scientific Computing Associates, Inc. Andrew Sherman Silicon Graphics Incorporated Eric Salo Sky Computers Anna Rounbehler Sun Microsystems David Taylor Sun Microsystems Terry Dontje Syracuse University Stephen Taylor The MITRE Corporation Arkady Kanevsky United States Navy Dennis Cottel University of California at Santa Barbara Ambuj Singh University of Colorado Oliver McBryan University of Denver Robert Babb University of Houston Darren Sanders University of Illinois Albert Cheng University of Illinois Ying Chen University of Maryland Manuel Ujaldon University of Notre Dame Andrew Lumsdaine University of Notre Dame Jeff Squyres University of San Fransisco Peter Pacheco University of Stuttgart Computing Center Rolf Rabenseifner University of Wisconsin Steve Huss-Lederman From mpi-core-human@mcs.anl.gov Thu Apr 3 12:56:01 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA25515; Thu, 3 Apr 1997 12:56:00 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA19465 for mpi-core-out; Thu, 3 Apr 1997 11:56:39 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA19460 for ; Thu, 3 Apr 1997 11:56:36 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id MAA24521; Thu, 3 Apr 1997 12:00:06 -0600 (CST) Date: Thu, 3 Apr 1997 12:00:06 -0600 (CST) Message-Id: <199704031800.MAA24521@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: T-shirt decision Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Well, it is time to decide about T-shirts. I have checked about cost and deadlines. To get it done so I can bring it to the meeting, we need to decide in the next week. The cost varies by what we want and how many we get. However, to give everyone a range (assuming about 35 total shirts), the cost is as low as $6.50 for a single sided/one color design on a white T-shirt to $9.10 for a two sided/two color design on a colored T-shirt. After all expenses it should be $10 or less. I think the main choice now is whether we can decide on a design. I will narrow the potential choices to ones that I think are realistic to put on the shirt (Top 10 is probably too big), can be worn in a family setting, and give a reasonable number to choose from. Can people send ME (lederman@cs.wisc.edu) their first and second choice (if have two) and I will then try and narrow it down early next week to ask for final commitments to buy the chosen one. If you don't plan to buy a shirt at all then you probably should not vote. If there isn't enough consensus or interest I will drop the effort. Choices: 1) MPI_WINS but MPI_AINT_KIND 2) No Java binding and proud of it. 3) Shoe design (see http://www.cs.wisc.edu/~lederman/mpif/tshirt/hc.ps) 4) New MPI-2 logo (http://www.erc.msstate.edu/~mott/MPI.html) We would have to fix up the final choice to make it pretty. Thanks, Steve From mpi-core-human@mcs.anl.gov Thu Apr 3 13:28:35 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA25838; Thu, 3 Apr 1997 13:28:31 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA20171 for mpi-core-out; Thu, 3 Apr 1997 12:30:00 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA20165 for ; Thu, 3 Apr 1997 12:29:55 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id NAA08132; Thu, 3 Apr 1997 13:23:32 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id NAA21980; Thu, 3 Apr 1997 13:33:24 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 8525646E.0065E70D ; Thu, 3 Apr 1997 13:33:02 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: lederman@cs.wisc.edu cc: mpi-core@mcs.anl.gov Message-ID: <8525646E.00652EC3.00@watngi01.watson.ibm.com> Date: Thu, 3 Apr 1997 13:32:26 -0400 Subject: Re: T-shirt decision Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve, I have the following suggestion: 2 sided color shirt (let us shoot for $10.00 a piece): in the front (pocket size), choice 4 (color logo with symbol on the side = bottom logo on the web page) in the back (larger size), choice 3 revisited: After PVM MPI-1 Comes MPI-2 with the same logos (except that PVM as suggested earlier could display ballerina shoes). Please let me know what you think of that. Jean-Pierre lederman @ cs.wisc.edu 04/03/97 01:00 PM To: mpi-core @ mcs.anl.gov cc: (bcc: Jean-Pierre Prost/Watson/IBM Research) Subject: T-shirt decision Well, it is time to decide about T-shirts. I have checked about cost and deadlines. To get it done so I can bring it to the meeting, we need to decide in the next week. The cost varies by what we want and how many we get. However, to give everyone a range (assuming about 35 total shirts), the cost is as low as $6.50 for a single sided/one color design on a white T-shirt to $9.10 for a two sided/two color design on a colored T-shirt. After all expenses it should be $10 or less. I think the main choice now is whether we can decide on a design. I will narrow the potential choices to ones that I think are realistic to put on the shirt (Top 10 is probably too big), can be worn in a family setting, and give a reasonable number to choose from. Can people send ME (lederman@cs.wisc.edu) their first and second choice (if have two) and I will then try and narrow it down early next week to ask for final commitments to buy the chosen one. If you don't plan to buy a shirt at all then you probably should not vote. If there isn't enough consensus or interest I will drop the effort. Choices: 1) MPI_WINS but MPI_AINT_KIND 2) No Java binding and proud of it. 3) Shoe design (see http://www.cs.wisc.edu/~lederman/mpif/tshirt/hc.ps) 4) New MPI-2 logo (http://www.erc.msstate.edu/~mott/MPI.html) We would have to fix up the final choice to make it pretty. Thanks, Steve From mpi-core-human@mcs.anl.gov Thu Apr 3 13:48:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA26151; Thu, 3 Apr 1997 13:48:02 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA20555 for mpi-core-out; Thu, 3 Apr 1997 12:49:36 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id MAA20550 for ; Thu, 3 Apr 1997 12:49:31 -0600 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA19531; Thu, 3 Apr 1997 13:51:38 -0500 Message-Id: <3343FC85.167E@kgn.ibm.com> Date: Thu, 03 Apr 1997 13:52:53 -0500 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: Steve Huss-Lederman Cc: mpi-core@mcs.anl.gov Subject: Re: T-shirt decision References: <199704031800.MAA24521@rap.cs.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I like the PVM=ballerina shoes; MPI-1=walking shoes or hightop tennis shoes maybe and MPI-2=combat boots with the laces tied together. Doesn't anyone know an artist? It seems like spending $50 on the labors of an impoverished art student would be worthwhile. It should not take someone who knows how to draw more than 1 hour. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Thu Apr 3 14:35:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA26664; Thu, 3 Apr 1997 14:35:23 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA21540 for mpi-core-out; Thu, 3 Apr 1997 13:36:28 -0600 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA21532 for ; Thu, 3 Apr 1997 13:36:22 -0600 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id OAA02772; Thu, 3 Apr 1997 14:29:58 -0500 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id OAA04775; Thu, 3 Apr 1997 14:39:50 -0500 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 8525646E.006BFC64 ; Thu, 3 Apr 1997 14:39:29 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: wcs@nsgi3.lbl.gov cc: Mpi-Core@mcs.anl.gov Message-ID: <8525646E.006B5FD6.00@watngi01.watson.ibm.com> Date: Thu, 3 Apr 1997 14:39:01 -0400 Subject: Re: intent, derived types, explicit interfaces Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The note below was sent by Bill, wrt F90 interfaces. I think that these comments are not about F90 interfaces, but about the generic (language independent) definitions of MPI functions. So, new generic bindings are required for all send/receive functions in MPI 1.1 and/or 1.2, according to (1&3), and new generic bindings for many collective functions are needed according to (2). *************** Every time I think about the Fortran 90 interface it seems there is some additional issue to nail down. Currently I have a few questions/comments about apecifying INTENT and explicit interfaces in the MPI module. It was claimed that we could use the IN/OUT/INOUT intent given by the generic interface. I'm going through the bindings to make sure this is really true, and have come up with a few things exceptions: 1. For many functions with a status argument the status argument is declared OUT but in fact we now allow you to pass MPI_STATUS_IGNORE, so it must be INOUT. Same for array_of_errcodes arguments. 2. As is noted in the collective chapter, the receive buffer on collective operations must be INOUT once we add the inplace option. 3. On any call in which MPI_BOTTOM is a valid value for a receive buffer, the buffer must be declared INOUT. (this includes all of the basic communication routines). Cases 1 and 3 are disputable on technical grounds. The implementation of MPI_STATUS_IGNORE and MPI_BOTTOM as special "addresses" is foreign to Fortran and technically not legal (because of the copy-in/out problem) to rely on. The contents of these "buffers" are not observed, though the base address of the buffer is, so perhaps technically they can be "IN" arguments. I can add these cases to the text, as exceptions (2 is actually a subset of 3), though I'm not yet confident that we have all the exceptions. ..... Comments? Bill From mpi-core-human@mcs.anl.gov Thu Apr 3 15:37:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA27317; Thu, 3 Apr 1997 15:36:57 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA23098 for mpi-core-out; Thu, 3 Apr 1997 14:38:09 -0600 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA23093 for ; Thu, 3 Apr 1997 14:37:57 -0600 Received: from denna.cs.msstate.edu (denna.cs.msstate.edu [192.208.157.80]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id OAA17568; Thu, 3 Apr 1997 14:41:16 -0600 (CST) Received: by denna.cs.msstate.edu with Microsoft Mail id <01BC403D.485ABC60@denna.cs.msstate.edu>; Thu, 3 Apr 1997 14:42:41 -0600 Message-ID: <01BC403D.485ABC60@denna.cs.msstate.edu> From: Shane Hebert To: "mpi-core@mcs.anl.gov" , "'Steve Huss-Lederman'" Subject: RE: T-shirt decision Date: Thu, 3 Apr 1997 14:42:39 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >| I think the main choice now is whether we can decide on a design. I >| will narrow the potential choices to ones that I think are realistic >| to put on the shirt (Top 10 is probably too big), can be worn in a >| family setting, and give a reasonable number to choose from. Can >| people send ME (lederman@cs.wisc.edu) their first and second choice >| (if have two) and I will then try and narrow it down early next week >| to ask for final commitments to buy the chosen one. If you don't plan >| to buy a shirt at all then you probably should not vote. If there >| isn't enough consensus or interest I will drop the effort. Top 10 should fit OK on a shirt. I see tons of them walking around the campus from different Frats,Sororities, and other organizations. I'd vote for the Top 10 if it was present. Since that isn't here, I'll cast my vote for: Main choice: MPI-2 Logo Second choice: MPI_WINS but MPI_AINT_KIND >| Choices: >| >| 1) MPI_WINS but MPI_AINT_KIND >| >| 2) No Java binding and proud of it. >| >| 3) Shoe design (see http://www.cs.wisc.edu/~lederman/mpif/tshirt/hc.ps) >| >| 4) New MPI-2 logo (http://www.erc.msstate.edu/~mott/MPI.html) >| >| We would have to fix up the final choice to make it pretty. >| >| Thanks, >| Steve From mpi-core-human@mcs.anl.gov Thu Apr 3 16:16:09 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA28053; Thu, 3 Apr 1997 16:16:06 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA24287 for mpi-core-out; Thu, 3 Apr 1997 15:17:27 -0600 Received: from cs.sandia.gov (cs.sandia.gov [132.175.13.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id PAA24277 for ; Thu, 3 Apr 1997 15:17:21 -0600 Received: from cheetah.cs.sandia.gov.noname by cs.sandia.gov with smtp (Smail3.1.28.1 #5) id m0wCsrI-000XQEC; Thu, 3 Apr 97 13:11 MST Received: by cheetah.cs.sandia.gov.noname (4.1/SMI-4.1) id AA21103; Thu, 3 Apr 97 13:11:43 MST From: bright@cs.sandia.gov (Ron Brightwell) Message-Id: <9704032011.AA21103@cheetah.cs.sandia.gov.noname> Subject: T-shirt decision To: mpi-core@mcs.anl.gov Date: Thu, 3 Apr 97 13:11:42 MST X-Mailer: ELM [version 2.3 PL11] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I like the simple idea of just displaying an MPI-2 logo. I don't like the idea of ever having to explain my clothing. What's the saying--fashion should be a statement, not a question? People didn't seem to be completely happy with Mike Mott's logo, so I modified it a little. The result is at http://www.cs.sandia.gov/~bright/mpi2logo.html It's his logo with the 2 moved so that it doesn't look like a squared, with the graphic on both sides connected to each other. This is supposed to represent the interapplication communication provided by the dynamic proposal. If people like this, then we should probably see if Mike will make the modifications to his original logo, since mine was hacked together from his, and it was his to begin with. -Ron From mpi-core-human@mcs.anl.gov Thu Apr 3 16:32:28 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA28255; Thu, 3 Apr 1997 16:32:27 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA24664 for mpi-core-out; Thu, 3 Apr 1997 15:33:44 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA24658; Thu, 3 Apr 1997 15:33:40 -0600 Message-Id: <199704032133.PAA24658@antares.mcs.anl.gov> To: bright@cs.sandia.gov (Ron Brightwell) cc: mpi-core@mcs.anl.gov Subject: Re: T-shirt decision In-reply-to: Your message of "Thu, 03 Apr 1997 13:11:42 MST." <9704032011.AA21103@cheetah.cs.sandia.gov.noname> Date: Thu, 03 Apr 1997 15:37:09 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The latest suggestion http://www.cs.sandia.gov/~bright/mpi2logo.html is my new favorite. From mpi-core-human@mcs.anl.gov Thu Apr 3 18:16:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA29570; Thu, 3 Apr 1997 18:15:57 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA27090 for mpi-core-out; Thu, 3 Apr 1997 17:17:04 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA27085 for ; Thu, 3 Apr 1997 17:17:00 -0600 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA13654; Thu, 3 Apr 1997 18:19:10 -0500 Message-Id: <33443B3E.446B@kgn.ibm.com> Date: Thu, 03 Apr 1997 18:20:30 -0500 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: IN, OUT, INOUT Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The subject of IN vs. OUT vs INOUT is a quagmire. I hope someone has a rope handy. Take the example of MPI_BOTTOM that Bill S. mentions. MPI_BOTTOM seems to be something special because we have no way to indicate the real object being identified in an MPI_Send, MPI_Recv etc. When someone calls MPI_Recv(mybuf,n,mytype,....) it seems that "mybuf" is the parameter which represents the OUT. In fact, "mybuf" by itself does not represent anything except an anchor point for the object defined by {buffer,count,datatype}. This hidden thing is always an OUT for a receive. There are cases where we pass a value (eg count) and these are clearly IN arguments. Once we move past this point things get confusing. In some cases we pass a value that identifies an object (may or may not be an MPI opaque object) as with MPI_Start(request). In others we pass a value which identifies a reference to an object (eg. MPI_Request_free(&request) or MPI_Buffer_detach(&buf_ptr,...). In the second and third cases, the initial value is an IN which we choose to ignore. MPI_STATUS_IGNORE seems to muddy the water However, if we picture MPI_STATUS_IGNORE as a value that identifies a predefined MPI_Status which happens to be very cheap to fill in and impossible to dereference, perhaps we can ignore this value like any other initial value and continue to label MPI_Status as an OUT where that is the label today. We have been fortunate in doing the labeling in the cases of a value that identifies a reference to an object. I do not know of a case where the IN/OUT/INOUT label would differ depending on whether the handle or the object is considered the parameter. MPI_Irecv takes a reference to a request handle. That handle is itself a reference to an object. Mercifully, both the request handle and the request object are OUTs so the label can serve both. As usual, the initial value which is an IN gets ignored. Then there are calls like MPI_Isend(b,c,mytype,....) in which we label the MPI_Datatype as an IN. The value passed in the datatype field does not change but it is really a handle which is a reference to an opaque object. The reference count in the opaque object must be bumped so the object is an INOUT. We ignore that too. THE QUICKSAND IS AT MY NECK ALREADY! Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Thu Apr 3 19:21:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA00221; Thu, 3 Apr 1997 19:21:49 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA28154 for mpi-core-out; Thu, 3 Apr 1997 18:23:13 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA28149 for ; Thu, 3 Apr 1997 18:23:10 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id SAA26827; Thu, 3 Apr 1997 18:26:36 -0600 (CST) Date: Thu, 3 Apr 1997 18:26:36 -0600 (CST) Message-Id: <199704040026.SAA26827@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: MPI Forum home page Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There was recent mail about setting up the official MPI home page. The last main proposal came from Rusty to set up a neutral domain and have me administer it. There has been very little mail on this topic since Rusty's note so I would like to go forward with setting up the home page. I have looked into what it takes to get our own domain. I can take care of the logistics. I have checked on available domain names. Unfortunately, mpi.org and mpif.org are taken. Since this is going to be the forum's home page, the domain name mpi-forum.org (www.mpi-forum.org for the home page) seems the best choice given what is taken. Unless there are objections, I propose to proceed to register this name for us. (The actual computer where the domain is rooted will be handled by name resolution and will not be obvious to the public. As Rusty's note indicated, Argonne has volunteered to provide these facilities at no charge.) The content and basic design of the home page will be discussed at the next forum meeting. The group will get a chance to see a proposed page and content and discuss/vote on it. Steve From mpi-core-human@mcs.anl.gov Thu Apr 3 19:37:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA00315; Thu, 3 Apr 1997 19:37:26 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA28338 for mpi-core-out; Thu, 3 Apr 1997 18:38:56 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA28333 for ; Thu, 3 Apr 1997 18:38:53 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id SAA26900; Thu, 3 Apr 1997 18:42:22 -0600 (CST) Date: Thu, 3 Apr 1997 18:42:22 -0600 (CST) Message-Id: <199704040042.SAA26900@rap.cs.wisc.edu> From: Steve Huss-Lederman To: treumann@kgn.ibm.com CC: mpi-core@mcs.anl.gov In-reply-to: <33443B3E.446B@kgn.ibm.com> (message from Dick Treumann on Thu, 03 Apr 1997 18:20:30 -0500) Subject: Re: IN, OUT, INOUT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > There are cases where we pass a value (eg count) and these are clearly > IN arguments. Once we move past this point things get confusing. In > some cases we pass a value that identifies an object (may or may not be > an MPI opaque object) as with MPI_Start(request). In others we pass a > value which identifies a reference to an object (eg. > MPI_Request_free(&request) or MPI_Buffer_detach(&buf_ptr,...). In the > second and third cases, the initial value is an IN which we choose to > ignore. I think the last two are errors. The current minor corrections has a similar fix for p. 122/lines 35-6 for MPI_OP_FREE and p. 196/lines 1-2 for MPI_ERRORHANDLER_FREE. There it argues that it should be INOUT since the handle gets "NULLed". To be consistent, we should do the same here. > Then there are calls like MPI_Isend(b,c,mytype,....) in which we label > the MPI_Datatype as an IN. The value passed in the datatype field does > not change but it is really a handle which is a reference to an opaque > object. The reference count in the opaque object must be bumped so the > object is an INOUT. We ignore that too. I'm not sure this is really a problem. I view the IN vs. OUT status as being what the user visably sees. When you free a handle, it gets "NULLed" and the user sees this so it has OUT properties (IN too). In the case above, the change is opaque to the user. I think it may actually be cleaner and clearer to label by what is visible to the user and not what implementations do under the cover. All that said, I have not check other functions to see if this view is consistent with what is done. Steve From mpi-core-human@mcs.anl.gov Thu Apr 3 20:09:41 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA00487; Thu, 3 Apr 1997 20:09:40 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA28683 for mpi-core-out; Thu, 3 Apr 1997 19:11:16 -0600 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA28676 for ; Thu, 3 Apr 1997 19:11:12 -0600 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id TAA27050; Thu, 3 Apr 1997 19:14:43 -0600 (CST) Date: Thu, 3 Apr 1997 19:14:43 -0600 (CST) Message-Id: <199704040114.TAA27050@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: questions in terms chapter Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There are a few outstanding points in the terms chapter that need to be addressed. Since we need clean text for the next meeting, I am asking for help/input since I am not sure what is correct. 1) p. 6 lines 44-46 say: In Fortran, the handle can be an index into a table of opaque objects in a system table; in C it can be such an index or a pointer to the object. C++ handles can simply ``wrap up'' a table index or pointer. The point has been raised that they really cannot be an index or pointer if they are constants. What is correct? 2) p. 11 lines 37-38 say: MPI_WAIT or a parallel thread modifies buf between the invocation of MPI_IRECV and the finish of MPI_WAIT. I don't understand why it begins with MPI_WAIT. Should it be: A parallel thread modifies buf between the invocation of MPI_IRECV and the finish of MPI_WAIT. 3) p. 17 line 17 notes that others promised to create text about the effect on routines undefined if use signal handlers and signals. Can someone help here? Thanks, Steve From mpi-core-human@mcs.anl.gov Thu Apr 3 20:20:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA00557; Thu, 3 Apr 1997 20:20:20 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA28774 for mpi-core-out; Thu, 3 Apr 1997 19:21:40 -0600 Received: from ncsa.uiuc.edu (sdgmail.ncsa.uiuc.edu [141.142.103.66]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA28769 for ; Thu, 3 Apr 1997 19:21:38 -0600 Received: from narnia.ncsa.uiuc.edu (narnia.ncsa.uiuc.edu [141.142.21.165]) by ncsa.uiuc.edu (8.8.5/8.8.5) with SMTP id TAA22644; Thu, 3 Apr 1997 19:25:13 -0600 (CST) Message-Id: <3.0.32.19970403192741.006bc75c@sdgmail.ncsa.uiuc.edu> X-Sender: acheng@sdgmail.ncsa.uiuc.edu X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Thu, 03 Apr 1997 19:27:42 -0600 To: Steve Huss-Lederman , mpi-core@mcs.anl.gov From: Albert Cheng Subject: Re: T-shirt decision Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 12:00 PM 4/3/97 -0600, Steve Huss-Lederman wrote: >Well, it is time to decide about T-shirts. I have checked about cost >and deadlines. To get it done so I can bring it to the meeting, we >need to decide in the next week. The cost varies by what we want and >how many we get. However, to give everyone a range (assuming about 35 >total shirts), the cost is as low as $6.50 for a single sided/one >color design on a white T-shirt to $9.10 for a two sided/two color >design on a colored T-shirt. After all expenses it should be $10 or >less. Black on white T-shirt is rather dull. I would cough up another $3.50 for something colorful. Does 2-side/2-colors means totally 2 colors per shirt OR 2 colors on front and 2 different other colors on back? If we stay just one side (like front), can we have more colors for about the same price? >4) New MPI-2 logo (http://www.erc.msstate.edu/~mott/MPI.html) First choice: color MPI2 logo. (5th design on the above html) Second choice: color MPI2 with small square on upper left corner (1st design of html) Since the color logos (and the shoes too) take multiple colors as shown now, does the above 2colors price permit that? I would not be happy with just a one or 2 colors logo. It would lose the flashy artistic appeal, IMO. -- Albert From mpi-core-human@mcs.anl.gov Thu Apr 3 20:28:53 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA00625; Thu, 3 Apr 1997 20:28:52 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA28969 for mpi-core-out; Thu, 3 Apr 1997 19:30:22 -0600 Received: from k2.llnl.gov (johnmay@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA28963 for ; Thu, 3 Apr 1997 19:30:17 -0600 Received: (from johnmay@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id RAA22625; Thu, 3 Apr 1997 17:33:48 -0800 (PST) Message-ID: <19970403173348.XF28820@k2.llnl.gov> Date: Thu, 3 Apr 1997 17:33:48 -0800 From: johnmay@llnl.gov (John May) To: mpi-core@mcs.anl.gov Subject: Re: MPI Forum home page References: <199704040026.SAA26827@rap.cs.wisc.edu> X-Mailer: Mutt 0.60e Mime-Version: 1.0 In-Reply-To: <199704040026.SAA26827@rap.cs.wisc.edu>; from Steve Huss-Lederman on Apr 3, 1997 18:26:36 -0600 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve Huss-Lederman writes: > I have looked into what it takes to get our own domain... > I have checked on available domain names. > Unfortunately, mpi.org and mpif.org are taken. I was too curious to let that pass. In can anyone else is interested, mpi.org is registered to Minnesota Project Innovation (no web page; don't know what they do). And mpif.org is home to the Metal Powder Industries Federation. From mpi-core-human@mcs.anl.gov Fri Apr 4 10:17:34 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA07371; Fri, 4 Apr 1997 10:17:32 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA09953 for mpi-core-out; Fri, 4 Apr 1997 09:17:04 -0600 Received: from mbunix.mitre.org (mbunix.mitre.org [129.83.20.100]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA08984 for ; Fri, 4 Apr 1997 08:32:04 -0600 Received: from TGATE3 (tgate3.mitre.org [129.83.20.27]) by mbunix.mitre.org (8.8.5/8.8.5/mitre.0) with SMTP id JAA13645 for ; Fri, 4 Apr 1997 09:35:38 -0500 (EST) Received: from mail11 (129.83.20.44) by tgate3.mitre.org (EMWAC SMTPRS 0.60) with SMTP id ; Fri, 04 Apr 1997 09:30:15 -0500 Received: by mail11; (5.65v3.2/1.1.8.2/22Jun94-0628PM) id AA29029; Fri, 4 Apr 1997 09:34:52 -0500 Subject: Re: T-shirt decision From: arkady@mail11.mitre.org (Arkady Kanevsky) cc: mpi-core@mcs.anl.gov Message-Id: <970404093444.20705@mail11.mitre.org.0> Date: Fri, 4 Apr 97 09:34:46 -0500 X-Mailer: MailWorks 2.0-3 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am also in favor of Ron's logo on one side and 10 MPI commandments on another. Arkady >The latest suggestion > > http://www.cs.sandia.gov/~bright/mpi2logo.html > >is my new favorite. > > From mpi-core-human@mcs.anl.gov Fri Apr 4 10:49:16 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA08031; Fri, 4 Apr 1997 10:49:14 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA10851 for mpi-core-out; Fri, 4 Apr 1997 09:50:14 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id JAA10846 for ; Fri, 4 Apr 1997 09:50:11 -0600 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA09281; Fri, 4 Apr 1997 10:52:24 -0500 Received: by windsurf.pok.ibm.com (AIX 4.1/UCB 5.64/4.03) id AA13272; Fri, 4 Apr 1997 10:53:44 -0500 Date: Fri, 4 Apr 1997 10:53:44 -0500 From: treumann@kgn.ibm.com (Dick Treumann) Message-Id: <9704041553.AA13272@windsurf.pok.ibm.com> To: @minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov Subject: Re: IN, OUT, INOUT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The main point of my ramblings about IN/OUT/INOUT is that as far as I can see they do not have a rigorous meaning. We should simply acknowledge that and move on to issues which matter more than whether MPI_STATUS_IGNORE makes status arguments an INOUT where before they were only OUT. Let us indicate that the IN/OUT/INOUT labels are hints to the reader about how to think about the argument and not pretend that someone who was writing a Lint or a specialized compiler for MPI programs could assume each label told him all he needed to know about that argument. Changes to these labels should be discussed from the point of view of making the issue clearer to the reader and not much more. The changes related to the new InPlace collectives are a good example. Dick Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Fri Apr 4 10:55:23 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA08104; Fri, 4 Apr 1997 10:55:21 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA11068 for mpi-core-out; Fri, 4 Apr 1997 09:56:53 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA11060; Fri, 4 Apr 1997 09:56:45 -0600 Message-Id: <199704041556.JAA11060@antares.mcs.anl.gov> To: treumann@kgn.ibm.com (Dick Treumann) Cc: mpi-core@antares.mcs.anl.gov Subject: Re: IN, OUT, INOUT In-reply-to: Your message of "Fri, 04 Apr 1997 10:53:44 EST." <9704041553.AA13272@windsurf.pok.ibm.com> Date: Fri, 04 Apr 1997 10:00:19 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | The main point of my ramblings about IN/OUT/INOUT is that as far | as I can see they do not have a rigorous meaning. We should | simply acknowledge that and move on to issues which matter | more than whether MPI_STATUS_IGNORE makes status arguments an | INOUT where before they were only OUT. | | Let us indicate that the IN/OUT/INOUT labels are hints to the | reader about how to think about the argument and not pretend | that someone who was writing a Lint or a specialized compiler | for MPI programs could assume each label told him all he needed | to know about that argument. I heartily agree with this point of view. The disclaimer of precision in the IN/OUT/INOUT labels can go in Terms. Rusty From mpi-core-human@mcs.anl.gov Fri Apr 4 11:15:04 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA08325; Fri, 4 Apr 1997 11:15:03 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA11594 for mpi-core-out; Fri, 4 Apr 1997 10:16:22 -0600 Received: from mailhost.lanl.gov (mailhost.lanl.gov [128.165.3.12]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA11589 for ; Fri, 4 Apr 1997 10:16:17 -0600 Received: from wrangler.lanl.gov (wrangler.lanl.gov [128.165.113.131]) by mailhost.lanl.gov (8.8.5/8.8.3) with ESMTP id JAA07477 for ; Fri, 4 Apr 1997 09:19:49 -0700 (MST) Received: from sooks.lanl.gov (sooks.lanl.gov [128.165.115.96]) by wrangler.lanl.gov (8.7.6/8.7.3) with SMTP id JAA08171 for ; Fri, 4 Apr 1997 09:19:48 -0700 (MST) Message-Id: <2.2.32.19970404162428.006a78e0@wrangler.lanl.gov> X-Sender: marydell@wrangler.lanl.gov X-Mailer: Windows Eudora Pro Version 2.2 (32) Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Fri, 04 Apr 1997 09:24:28 -0700 To: mpi-core@mcs.anl.gov From: MaryDell Tholburn Subject: RE: T-shirt decision Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with Shane on the top ten, that would certainly get my vote. I also like Richard's suggestion: MPI-2 5-yes, 4-no 17-abstain 1-sleeping But in the absents of those choices I vote for the MPI logo. MaryDell From mpi-core-human@mcs.anl.gov Fri Apr 4 11:36:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA08657; Fri, 4 Apr 1997 11:36:23 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA12611 for mpi-core-out; Fri, 4 Apr 1997 10:37:46 -0600 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA12606 for ; Fri, 4 Apr 1997 10:37:43 -0600 From: Andrew.Lumsdaine.1@nd.edu Received: (from lums@localhost) by lsc.nd.edu (8.8.5/8.8.5) id LAA01376; Fri, 4 Apr 1997 11:41:13 -0500 (EST) Date: Fri, 4 Apr 1997 11:41:13 -0500 (EST) Message-Id: <199704041641.LAA01376@lsc.nd.edu> To: lusk@mcs.anl.gov CC: treumann@kgn.ibm.com, mpi-core@antares.mcs.anl.gov In-reply-to: <199704041556.JAA11060@antares.mcs.anl.gov> (message from Rusty Lusk on Fri, 04 Apr 1997 10:00:19 -0600) Subject: Re: IN, OUT, INOUT Reply-to: Andrew.Lumsdaine.1@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode Rusty Lusk wrote: > | Let us indicate that the IN/OUT/INOUT labels are hints to the > | reader about how to think about the argument and not pretend > | that someone who was writing a Lint or a specialized compiler > | for MPI programs could assume each label told him all he needed > | to know about that argument. > I heartily agree with this point of view. The disclaimer of precision > in the IN/OUT/INOUT labels can go in Terms. I am not so sure that I agree. In particular, the IN/OUT/INOUT labels can be formalized in the C++ bindings via const for IN. That is, a parameter is either const or it is not -- there is no ambiguity in the binding declaration. Best Regards, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Fri Apr 4 12:07:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA09132; Fri, 4 Apr 1997 12:06:59 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA13636 for mpi-core-out; Fri, 4 Apr 1997 11:08:23 -0600 Received: from sgi.sgi.com (SGI.COM [192.48.153.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA13630 for ; Fri, 4 Apr 1997 11:08:17 -0600 Received: from mrjones.engr.sgi.com ([150.166.49.18]) by sgi.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id JAA09738 for <@sgi.engr.sgi.com:mpi-core@mcs.anl.gov>; Fri, 4 Apr 1997 09:11:50 -0800 Received: by mrjones.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) for mpi-core@mcs.anl.gov id JAA12215; Fri, 4 Apr 1997 09:09:40 -0800 From: "Eric Salo" Message-Id: <9704040909.ZM12213@mrjones.engr.sgi.com> Date: Fri, 4 Apr 1997 09:09:39 -0800 In-Reply-To: treumann@kgn.ibm.com (Dick Treumann) "Re: IN, OUT, INOUT" (Apr 4, 10:53am) References: <9704041553.AA13272@windsurf.pok.ibm.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: IN, OUT, INOUT Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with Dick. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Fri Apr 4 12:58:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA09655; Fri, 4 Apr 1997 12:58:58 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA15164 for mpi-core-out; Fri, 4 Apr 1997 11:59:10 -0600 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA15158 for ; Fri, 4 Apr 1997 11:59:04 -0600 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA22593; Fri, 4 Apr 1997 13:01:18 -0500 Received: by windsurf.pok.ibm.com (AIX 4.1/UCB 5.64/4.03) id AA13300; Fri, 4 Apr 1997 13:02:38 -0500 Date: Fri, 4 Apr 1997 13:02:38 -0500 From: treumann@kgn.ibm.com (Dick Treumann) Message-Id: <9704041802.AA13300@windsurf.pok.ibm.com> To: @minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov Subject: Re: IN, OUT, INOUT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Consider MPI_START: Fortran: CALL MPI_START(REQ) C: rc = MPI_Start(req); In C, req is a value parameter and therefor from the compiler's viewpoint cannot be anything but an IN. The C compiler has no knowledge of the opaque object it represents. In Fortran without an ability to tell the compiler to view it as an IN the compiler will treat it as an INOUT. F90 provides a way to tell the compiler that the argument it is tracking (the handle) is an IN so that is what the compiler should be told. To the user trying to write an MPI program the argument of interest is the request object and it is quite clearly an INOUT. We can continue to call the MPI_Datatype on an Irecv an IN because the user does not really care that MPI_RECV may leave the datatype object exactly as it found it while MPI_Irecv must bump a reference count. The explicit language bindings are the place for telling the compilers which of the variables it is tracking can be presumed unmodified across an MPI call. That does not need to exactly match the IN/OUT/INOUT labels unless we back away from the "one special case" described in MPI 1.1 section 2.2 related to MPI opaque objects. Dick Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Fri Apr 4 16:02:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA12579; Fri, 4 Apr 1997 16:02:48 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA19554 for mpi-core-out; Fri, 4 Apr 1997 15:03:09 -0600 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA19226; Fri, 4 Apr 1997 14:53:31 -0600 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id MAA01946; Fri, 4 Apr 1997 12:57:02 -0800 Date: Fri, 4 Apr 1997 12:57:02 -0800 From: wcs@nsgi3.lbl.gov (William Saphir) Message-Id: <199704042057.MAA01946@nsgi3.lbl.gov> To: mpi-bind@mcs.anl.gov Subject: Re: IN, OUT, INOUT Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Date: Fri, 4 Apr 1997 10:53:44 -0500 > From: treumann@kgn.ibm.com (Dick Treumann) > > The main point of my ramblings about IN/OUT/INOUT is that as far > as I can see they do not have a rigorous meaning. We should > simply acknowledge that and move on to issues which matter > more than whether MPI_STATUS_IGNORE makes status arguments an > INOUT where before they were only OUT. > > Let us indicate that the IN/OUT/INOUT labels are hints to the > reader about how to think about the argument and not pretend > that someone who was writing a Lint or a specialized compiler > for MPI programs could assume each label told him all he needed > to know about that argument. > > Changes to these labels should be discussed from the point of > view of making the issue clearer to the reader and not much > more. The changes related to the new InPlace collectives > are a good example. > > Dick The original issue here was that the current text says that implementations could add INTENT in Fortran 90 interfaces and that the INTENT is the same as the generic bindings. There had been assertions that the generic bindings were sufficient. The point of my message was that it is not so simple, and there seems to be agreement on this. While IN/OUT/INOUT in the generic bindings is vaguely defined and may be impossible to define rigorously, they have rigorous meaning in language bindings, so it remains possible to say something definite. For the Fortran section, I see three choices: 1. drop the discussion of INTENT 2. explicitly say that INTENT may be specified in the mpi module, and that the correct designation is up to the the implementation (the restriction is that legal MPI programs should work). 3. add explicit bindings, with intent, for Fortran 90 - for all functions (or, equivalently, have very explicit text that lists every exception to the generic specification). >From the point of view of requirements, 1 and 2 are probably equivalent. I would prefer 2, since many people have expressed an interest in saying something about INTENT. If someone would like to do a thorough job on 3, then we should consider such a proposal if one can be submitted early enough to get discussion. [I do not have enough confidence in my own understanding of all the corner cases and implementation issues to feel comfortable doing this, but others are welcome.] Comments? Bill From mpi-core-human@mcs.anl.gov Fri Apr 4 16:40:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA12949; Fri, 4 Apr 1997 16:40:05 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA20725 for mpi-core-out; Fri, 4 Apr 1997 15:41:05 -0600 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA20719 for ; Fri, 4 Apr 1997 15:41:01 -0600 Message-Id: <199704042141.PAA20719@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: institutions, version 3 Date: Fri, 04 Apr 1997 15:44:34 -0600 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Convergence is well under way, but may not be complete. Please check. Thanks. Rusty Argonne National Laboratory Bill Gropp Argonne National Laboratory Rajeev Thakur Argonne National Laboratory Rusty Lusk California Institute of Technology Jerrell Watts California Institute of Technology Rajesh Bordawekar Center for Computing Sciences Steve Huss-Lederman Convex Computer Corporation Steve Fleischman Cray Research Heidi Poxon Cray Research Laurie Costello Cray Research Margaret Cahir Cray Research Peter Rigsbee Digital Equipment Corporation Ed Benson Dolphin Interconnect Solutions, Inc. Jim Cownie Edinburgh Parallel Computing Centre Lyndon Clarke General Electric Corporation Bob Madahar General Electric Corporation Craig Fischberg German National Research Center for Information Technology Klaus Wolf Hewlett-Packard Greg Astfalk Hewlett-Packard Raja Daoud Hewlett-Packard Steve Landherr Hitachi Mark Fallon Hughes Aircraft Company Lloyd Lewins Intel Corporation Harish Nag Intel Corporation Joel Clark International Business Machines Dick Treumann International Business Machines Jean-Pierre Prost International Business Machines Marc Snir Khoral Research Steve Kubica Khoral Research Tom Robey Lawrence Livermore National Laboratory Elsie Pierce Lawrence Livermore National Laboratory John May Lawrence Livermore National Laboratory Terry Jones Los Alamos National Laboratory MaryDell Tholburn Los Alamos National Laboratory Suresh Damodaran-Kamal Los Alamos National Laboratory Tyce McLarty MPI Software Techology, Inc. Peter Brennan Mississippi State University Boris Protopopov Mississippi State University Robert George Mississippi State University Shane Hebert Mississippi State University Tony Skjellum Mississippi State University Zhenqian Cui NEC Corporation Koichi Konishi NEC Corporation Rolf Hempel NEC Corporation Susan Kraus National Aeronautics and Space Administration Bill Nitzberg National Aeronautics and Space Administration David DiNucci National Aeronautics and Space Administration Ian Stockdale National Aeronautics and Space Administration Parkson Wong National Energy Research Scientific Computing Center Bill Saphir National Institute of Standards and Technology John Hagedorn National Institute of Standards and Technology Judy Devaney National Oceanic and Atmospheric Adminstration Leslie Hart National Oceanic and Atmospheric Adminstration Tom Henderson Oak Ridge National Laboratory Al Geist Ohio State University Nick Nevin PALLAS GmbH Hans-Christian Hoppe Pacific Northwest National Laboratory Jarek Nieplocha Pratt & Whitney Dave Wright Pratt & Whitney Pete Bradley San Diego Supercomputer Center Richard Frost Sanders, A Lockheed-Martin Company Nathan Doss Sandia National Laboratories Pang Chen Sandia National Laboratories Ron Brightwell Schlumberger Anne Elster Scientific Computing Associates, Inc. Andrew Sherman Silicon Graphics Incorporated Eric Salo Sky Computers Anna Rounbehler Sun Microsystems Computer Corporation David Taylor Sun Microsystems Computer Corporation Terry Dontje Syracuse University Stephen Taylor The MITRE Corporation Arkady Kanevsky Thinking Machines Corporation Eric Sharakan Thinking Machines Corporation Peter Ossadnik United States Navy Dennis Cottel University of Colorado Oliver McBryan University of Denver Robert Babb University of Houston Darren Sanders University of Houston Maciej Brodowicz University of Houston Yoonho Park University of Illinois Albert Cheng University of Illinois Ying Chen University of Maryland Manuel Ujaldon University of Notre Dame Andrew Lumsdaine University of Notre Dame Brian McCandless University of Notre Dame Jeff Squyres University of San Fransisco Peter Pacheco University of Stuttgart Computing Center Rolf Rabenseifner University of Wisconsin Steve Huss-Lederman From mpi-core-human@mcs.anl.gov Tue Apr 8 11:04:01 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA07139; Tue, 8 Apr 1997 11:04:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA29603 for mpi-core-out; Tue, 8 Apr 1997 10:02:16 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA29598 for ; Tue, 8 Apr 1997 10:02:12 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id KAA28403; Tue, 8 Apr 1997 10:02:24 -0500 (CDT) Date: Tue, 8 Apr 1997 10:02:24 -0500 (CDT) Message-Id: <199704081502.KAA28403@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: T-shirt choices Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is my analysis of the the votes for the T-shirt. The largest number of votes by far was for having a logo. Initially it was for the one from Michael Mott but the revised one from Ron Brightwell got most of the later votes after it was introduced. There was also a modest number of votes for the top 10 list. This is notable since I did not give it as an option. The shoes logo did ok but not as well and some objected to it. To narrow this down, I propose the following. The front of the T-shirt will have a logo. We need to decide which one. The second choice is if we should also have a top 10 list on the back. To make it concrete, here is the list: 10. "From the creators of BSEND and GRAPH_CREATE." 9. "Brought to you by a resounding 5 yes/4 no/17 abstain/1 sleeping vote." 8. "One-sided, two-faced, and proud of it!" 7. "Don't blame me, I voted against that." 6. "Spawn this, buddy!" 5. "Is this our first or second second-vote?" 4. "No Java binding and proud of it" 3. "MPI_WINS but MPI_AINT_KIND" 2. "Violent pain prevents me from continuing." 1. "The new phonebook is here!" My removed 2 that some might find objectionable and put in two other choices. We can modify the list if people think I chose wrong. I would like people to vote on the following by Thursday (if possible). Soon I will be asking people for orders. I will need the size desired (S/M/L/XL) and how many of each. If you are not coming to the next meeting, then I will need to add several dollars to cover the cost to send it to you. Thanks, Steve ---------------------------------------------------------------------- CUT HERE ---------------------------------------------------------------------- My choice for logo on the front is: 1) http://www.cs.sandia.gov/~bright/mpi2logo.html : 2) http://www.erc.msstate.edu/~mott/MPI.html : On the back I want: 1) Top 10 : 2) nothing : ---------------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Tue Apr 8 11:22:32 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA07455; Tue, 8 Apr 1997 11:22:31 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA00260 for mpi-core-out; Tue, 8 Apr 1997 10:21:35 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA00255; Tue, 8 Apr 1997 10:21:30 -0500 Message-Id: <199704081521.KAA00255@antares.mcs.anl.gov> To: Steve Huss-Lederman cc: mpi-core@mcs.anl.gov Subject: Re: T-shirt choices In-reply-to: Your message of "Tue, 08 Apr 1997 10:02:24 CDT." <199704081502.KAA28403@rap.cs.wisc.edu> Date: Tue, 08 Apr 1997 10:21:45 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My choice for logo on the front is: 1) http://www.cs.sandia.gov/~bright/mpi2logo.html : X On the back I want: 2) nothing : X Actually, I enjoy "MPI_WINS but MPI_AINT_KIND" and would vote for having it on the back. But at the risk of sounding stuffy, I prefer not to make negative comments in public about what we all worked so hard on and took very seriously, lest they be misperceived by those who didn't participate. Therefore I would prefer nothing to the "Top 10" list. Rusty From mpi-core-human@mcs.anl.gov Tue Apr 8 13:25:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA09372; Tue, 8 Apr 1997 13:25:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA03682 for mpi-core-out; Tue, 8 Apr 1997 12:29:55 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA03677 for ; Tue, 8 Apr 1997 12:29:51 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id NAA15658; Tue, 8 Apr 1997 13:20:16 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id NAA47671; Tue, 8 Apr 1997 13:30:05 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256473.00601338 ; Tue, 8 Apr 1997 13:29:23 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: lederman@cs.wisc.edu cc: mpi-core@mcs.anl.gov Message-ID: <85256473.005FA3F1.00@watngi01.watson.ibm.com> Date: Tue, 8 Apr 1997 13:28:45 -0400 Subject: Re: T-shirt choices Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I back up Rusty on this. The top 10 list is pretty funny, especially to participants to the MPI forum. However, it could be misinterpreted and could give a negative image about the quality of MPI-2. Therefore, I would prefer to leave the back of the T-shirt empty. Jean-Pierre lusk @ mcs.anl.gov 04/08/97 11:21 AM To: lederman @ cs.wisc.edu cc: mpi-core @ mcs.anl.gov (bcc: Jean-Pierre Prost/Watson/IBM Research) Subject: Re: T-shirt choices My choice for logo on the front is: 1) http://www.cs.sandia.gov/~bright/mpi2logo.html : X On the back I want: 2) nothing : X Actually, I enjoy "MPI_WINS but MPI_AINT_KIND" and would vote for having it on the back. But at the risk of sounding stuffy, I prefer not to make negative comments in public about what we all worked so hard on and took very seriously, lest they be misperceived by those who didn't participate. Therefore I would prefer nothing to the "Top 10" list. Rusty From mpi-core-human@mcs.anl.gov Tue Apr 8 13:42:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA09709; Tue, 8 Apr 1997 13:42:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA04108 for mpi-core-out; Tue, 8 Apr 1997 12:47:10 -0500 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA04102 for ; Tue, 8 Apr 1997 12:47:02 -0500 Received: from denna.cs.msstate.edu (denna.cs.msstate.edu [192.208.157.80]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id MAA17896; Tue, 8 Apr 1997 12:47:12 -0500 (CDT) Received: by denna.cs.msstate.edu with Microsoft Mail id <01BC441B.2EEC11C0@denna.cs.msstate.edu>; Tue, 8 Apr 1997 12:48:40 -0500 Message-ID: <01BC441B.2EEC11C0@denna.cs.msstate.edu> From: Shane Hebert Cc: "mpi-core@mcs.anl.gov" Subject: RE: T-shirt choices Date: Tue, 8 Apr 1997 12:48:38 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >| I back up Rusty on this. The top 10 list is pretty funny, >| especially to participants to the MPI forum. However, it >| could be misinterpreted and could give a negative image about >| the quality of MPI-2. >| Therefore, I would prefer to leave the back of the T-shirt >| empty. >| Jean-Pierre I like the idea (can't remember who first said it) to have two T-shirts. One 'official' shirt that has just a cool logo on the front and nothing on the back and another 'joke' shirt that has the Top 10 List on the back and something (maybe nothing but definitely not the official MPI-2 Logo) on the front. From mpi-core-human@mcs.anl.gov Tue Apr 8 13:53:25 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA10041; Tue, 8 Apr 1997 13:53:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA04468 for mpi-core-out; Tue, 8 Apr 1997 12:57:55 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA04463 for ; Tue, 8 Apr 1997 12:57:51 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id NAA13758; Tue, 8 Apr 1997 13:48:13 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id NAA56190; Tue, 8 Apr 1997 13:58:02 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256473.0062A488 ; Tue, 8 Apr 1997 13:57:26 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: shane@erc.msstate.edu cc: mpi-core@mcs.anl.gov Message-ID: <85256473.0062857D.00@watngi01.watson.ibm.com> Date: Tue, 8 Apr 1997 13:56:45 -0400 Subject: RE: T-shirt choices Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This would be fine with me. Jean-Pierre shane @ erc.msstate.edu 04/08/97 01:48 PM To: cc: mpi-core @ mcs.anl.gov (bcc: Jean-Pierre Prost/Watson/IBM Research) Subject: RE: T-shirt choices >| I back up Rusty on this. The top 10 list is pretty funny, >| especially to participants to the MPI forum. However, it >| could be misinterpreted and could give a negative image about >| the quality of MPI-2. >| Therefore, I would prefer to leave the back of the T-shirt >| empty. >| Jean-Pierre I like the idea (can't remember who first said it) to have two T-shirts. One 'official' shirt that has just a cool logo on the front and nothing on the back and another 'joke' shirt that has the Top 10 List on the back and something (maybe nothing but definitely not the official MPI-2 Logo) on the front. From mpi-core-human@mcs.anl.gov Tue Apr 8 14:26:35 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA10923; Tue, 8 Apr 1997 14:26:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA05414 for mpi-core-out; Tue, 8 Apr 1997 13:30:44 -0500 Received: from ncsa.uiuc.edu (sdgmail.ncsa.uiuc.edu [141.142.103.66]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA05407 for ; Tue, 8 Apr 1997 13:30:37 -0500 Received: from narnia.ncsa.uiuc.edu (narnia.ncsa.uiuc.edu [141.142.21.165]) by ncsa.uiuc.edu (8.8.5/8.8.5) with SMTP id NAA19607 for ; Tue, 8 Apr 1997 13:30:57 -0500 (CDT) Message-Id: <3.0.32.19970408133251.006b2e88@sdgmail.ncsa.uiuc.edu> X-Sender: acheng@sdgmail.ncsa.uiuc.edu X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Tue, 08 Apr 1997 13:32:52 -0500 To: mpi-core@mcs.anl.gov From: Albert Cheng Subject: Re: T-shirt choices Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 01:28 PM 4/8/97 -0400, Jean-Pierre Prost wrote: >I back up Rusty on this. The top 10 list is pretty funny, >especially to participants to the MPI forum. However, it >could be misinterpreted and could give a negative image about >the quality of MPI-2. >Therefore, I would prefer to leave the back of the T-shirt >empty. >Jean-Pierre I am kind of neutral about the top 10 list but I am not sure if we can ever decide on which top 10. Also, if it matters, I would rather move the cost for printing the back to make sure the logo is printed in multiple colors instead just two colors. I like the current flashy colors and I don't know what it looks like when it is reduced to two or three only. From mpi-core-human@mcs.anl.gov Tue Apr 8 15:53:39 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA12144; Tue, 8 Apr 1997 15:53:36 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA08745 for mpi-core-out; Tue, 8 Apr 1997 14:54:32 -0500 Received: from bedrock.osc.edu (bedrock.osc.edu [128.146.36.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA08739 for ; Tue, 8 Apr 1997 14:54:28 -0500 Received: for nevin@osc.edu by bedrock.osc.edu (8.8.3/950822.1) id PAA20830; Tue, 8 Apr 1997 15:51:29 -0400 (EDT) Date: Tue, 8 Apr 1997 15:51:29 -0400 (EDT) Message-Id: <199704081951.PAA20830@bedrock.osc.edu> From: Nick Nevin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Steve Huss-Lederman Cc: mpi-core@mcs.anl.gov Subject: Re:questions in terms chapter In-Reply-To: <199704040114.TAA27050@rap.cs.wisc.edu> References: <199704040114.TAA27050@rap.cs.wisc.edu> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve writes: > There are a few outstanding points in the terms chapter that need to > be addressed. Since we need clean text for the next meeting, I am > asking for help/input since I am not sure what is correct. > 1) p. 6 lines 44-46 say: > In Fortran, the handle can be an index into a table of opaque objects > in a system table; in C it can be such an index or a pointer to the > object. C++ handles can simply ``wrap up'' a table index or pointer. > The point has been raised that they really cannot be an index or > pointer if they are constants. What is correct? MPI-1 requires that "all named MPI constants can be used in initialization expressions or assignments like C constants." This does not require them to be integers (indices). For example the following works and fulfills the above requirements. struct _dtype { ... }; typedef struct _dtype *MPI_Datatype; struct _dtype mpi_int_dtype; #define MPI_INT ((MPI_Datatype) &mpi_int_dtype) ---nick. From mpi-core-human@mcs.anl.gov Tue Apr 8 17:17:01 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA13273; Tue, 8 Apr 1997 17:17:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA10981 for mpi-core-out; Tue, 8 Apr 1997 16:21:35 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA09985 for ; Tue, 8 Apr 1997 15:42:07 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id NAA12700; Tue, 8 Apr 1997 13:42:18 -0700 From: "William Saphir" Message-Id: <9704081342.ZM12699@nsgi3.lbl.gov> Date: Tue, 8 Apr 1997 13:42:17 -0700 In-Reply-To: Nick Nevin "Re:questions in terms chapter" (Apr 8, 3:51pm) References: <199704040114.TAA27050@rap.cs.wisc.edu> <199704081951.PAA20830@bedrock.osc.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: Nick Nevin , Steve Huss-Lederman Subject: Re: Re:questions in terms chapter Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Apr 8, 3:51pm, Nick Nevin wrote: > MPI-1 requires that "all named MPI constants can be used in > initialization expressions or assignments like C constants." > > This does not require them to be integers (indices). For example the > following works and fulfills the above requirements. > > > struct _dtype { > ... > }; > > typedef struct _dtype *MPI_Datatype; > > struct _dtype mpi_int_dtype; > > #define MPI_INT ((MPI_Datatype) &mpi_int_dtype) > > > ---nick. >-- End of excerpt from Nick Nevin It's helpful to keep in mind the canonical example: extern FILE __sF[]; #define stdin (&__sF[0]) #define stdout (&__sF[1]) #define stderr (&__sF[2]) Bill From mpi-core-human@mcs.anl.gov Tue Apr 8 17:56:07 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA13682; Tue, 8 Apr 1997 17:56:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA12557 for mpi-core-out; Tue, 8 Apr 1997 16:58:50 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA12552 for ; Tue, 8 Apr 1997 16:58:45 -0500 From: Andrew.Lumsdaine.1@nd.edu Received: from dartagnan.lsc.nd.edu (dartagnan.lsc.nd.edu [129.74.22.187]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id QAA02099; Tue, 8 Apr 1997 16:58:42 -0500 (EST) Received: (from lums@localhost) by dartagnan.lsc.nd.edu (8.8.5/8.8.5) id QAA22008; Tue, 8 Apr 1997 16:58:42 -0500 (EST) Date: Tue, 8 Apr 1997 16:58:42 -0500 (EST) Message-Id: <199704082158.QAA22008@dartagnan.lsc.nd.edu> To: wcs@nsgi3.lbl.gov CC: nevin@osc.edu, lederman@cs.wisc.edu, mpi-core@mcs.anl.gov, jsquyres@lsc.nd.edu In-reply-to: <9704081342.ZM12699@nsgi3.lbl.gov> (message from William Saphir on Tue, 8 Apr 1997 13:42:17 -0700) Subject: Re: questions in terms chapter Reply-to: Andrew.Lumsdaine.1@nd.edu Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode William Saphir wrote: > > MPI-1 requires that "all named MPI constants can be used in > > initialization expressions or assignments like C constants." > > > This does not require them to be integers (indices). For example the > > following works and fulfills the above requirements. > It's helpful to keep in mind the canonical example: > extern FILE __sF[]; > #define stdin (&__sF[0]) > #define stdout (&__sF[1]) > #define stderr (&__sF[2]) One big caveat here -- when one says the constants can be used for initialization like C constants, I believe that means they can be used before MPI_Init() is called. So, implementations using pointers may have a problem (and many do), in that the values of the pointers (and the structures they reference) are set when MPI_Init() is called. In the canonical example, the file table is initialized before main() is called, so stdin can be copied (or more precisely, the contents of stdin can be copied) at any point in the program. Regards, Andrew Lumsdaine From mpi-core-human@mcs.anl.gov Tue Apr 8 18:09:05 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA13787; Tue, 8 Apr 1997 18:09:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA13007 for mpi-core-out; Tue, 8 Apr 1997 17:13:49 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA12900; Tue, 8 Apr 1997 17:10:25 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id PAA12827; Tue, 8 Apr 1997 15:10:38 -0700 From: "William Saphir" Message-Id: <9704081510.ZM12826@nsgi3.lbl.gov> Date: Tue, 8 Apr 1997 15:10:37 -0700 In-Reply-To: johnmay@llnl.gov (John May) "Re: reserved file hints question" (Apr 8, 2:16pm) References: <19970407103222.CC53285@k2.llnl.gov> <199704080033.TAA24189@rap.cs.wisc.edu> <19970408141649.TP20301@k2.llnl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: johnmay@llnl.gov (John May), mpi-io@mcs.anl.gov Subject: Re: reserved file hints question Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Apr 8, 2:16pm, John May wrote: > Subject: Re: reserved file hints question > Steve noticed some ambiguity in the section on File Hints in the > I/O chapter. I propose the following clarification, to be inserted > in Section 10.2.8, just before the list of reserved file hints. > > John > > > MPI info values are always passed as strings. When the descriptions > below refer to a value as being a boolean, an integer, or a list, > they mean the string representation of these types. Specifically, > legal values for a boolean are the strings ``true'' and ``false'' > (all lowercase). For integers, legal values are string representations > of decimal values of integers that are within the range of a standard > integer type in the program. (However it is > possible that not every legal integer is a legal value for a > given key.) Leading zeros are permitted, and $+$ signs are optional > on positive numbers. No space may appear between a $+$ or $-$ sign > and the first digit of a number. > For lists, the string must contain legal elements > separated by commas. Leading and trailing spaces are stripped > automatically from all file info values. Spaces are permitted before > and after commas in lists, but spaces are not permitted within > individual elements of any type. > > \begin{discuss} > We define don't specify any floating-point info values. Should > we specify a format for any implementations that use them? > > Also, should we permit hex or octal integer representations? > \end{discuss} >-- End of excerpt from John May Nice try, but this is a quagmire as we found earlier in the dynamic chapter (*months* of discussion). Also, we should have the same rules for the dynamic and I/O chapters. I think the above is too restrictive. For example, the "path" key for MPI_SPAWN might be colon-separated list; some strings might have spaces in them; in some cases it might be reasonable to have a base-8 integer (e.g. "755" for permissions). I prefer: "The interpretation of string as boolean, integer, real-number, string or list values is implementation-defined." Whatever text we have, I'd also like to attach it to the info section and, as discussed before, the info section can move to a more neutral place (misc). Bill I prefer From mpi-core-human@mcs.anl.gov Tue Apr 8 19:38:18 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA14459; Tue, 8 Apr 1997 19:38:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA14972 for mpi-core-out; Tue, 8 Apr 1997 18:42:53 -0500 Received: from k2.llnl.gov (johnmay@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA14967; Tue, 8 Apr 1997 18:42:50 -0500 Received: (from johnmay@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id QAA16395; Tue, 8 Apr 1997 16:43:06 -0700 (PDT) Message-ID: <19970408164306.RP18633@k2.llnl.gov> Date: Tue, 8 Apr 1997 16:43:06 -0700 From: johnmay@llnl.gov (John May) To: mpi-io@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: reserved file hints question References: <19970407103222.CC53285@k2.llnl.gov> <199704080033.TAA24189@rap.cs.wisc.edu> <19970408141649.TP20301@k2.llnl.gov> <9704081510.ZM12826@nsgi3.lbl.gov> X-Mailer: Mutt 0.60e Mime-Version: 1.0 In-Reply-To: <9704081510.ZM12826@nsgi3.lbl.gov>; from William Saphir on Apr 8, 1997 15:10:37 -0700 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk William Saphir writes: > Nice try, but this is a quagmire as we found earlier in the > dynamic chapter (*months* of discussion). Also, we should have > the same rules for the dynamic and I/O chapters. I think the above > is too restrictive. For example, the "path" key for MPI_SPAWN might > be colon-separated list; some strings might have spaces in > them; in some cases it might be reasonable to have a base-8 > integer (e.g. "755" for permissions). > > I prefer: > > "The interpretation of string as boolean, integer, real-number, > string or list values is implementation-defined." > > Whatever text we have, I'd also like to attach it to the info > section and, as discussed before, the info section can move > to a more neutral place (misc). I appreciate the problems you're describing, but the main purpose of the addition was to clarify that values given as "boolean", "integer", and "list of ..." for these particular info keys are really string-representations of the given types. Perhaps we can avoid the problems you describe by stipulating that the description applies *only* to these particular items, and that implementations are free to specify the forms of other info values. I think as long as we're reserving some hints to enhance portability, we owe it to the user to make the values portable too. (Unfortunately, this doesn't work for file_perm, but it's an important enough key that it should probably still be reserved.) For the string-type hints, I think the rules I gave (or a similar set) will work pretty well. All but one of the string-type hints accept only specific strings, so the ambiguity with commas and spaces doesn't come up. My rule about internal spaces was intended to remind Fortaners that these spaces aren't ignored; perhaps that reminder isn't necessary. I don't see any problem stripping leading and trailing spaces. Filenames are only returned; they can't be passed in, so the formatting rules don't apply. The one potentially troublesome hint is the list of I/O nodes. Node names could conceivably include commas, though I don't know of any systems where this would really happen. If it seems useful, we can add a way to specify literal commas. Likewise, for the integers, I think the rules I gave will work. The only reserved hint that really needs an octal value is file permission, and that's already specified to have an implementation-defined value. The implementation is free to say that the string contains an octal integer in that case. John From mpi-core-human@mcs.anl.gov Tue Apr 8 19:56:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA14575; Tue, 8 Apr 1997 19:56:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA15202 for mpi-core-out; Tue, 8 Apr 1997 19:00:58 -0500 Received: from palrel1.hp.com (palrel1.hp.com [15.253.72.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA15197 for ; Tue, 8 Apr 1997 19:00:54 -0500 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com with ESMTP (8.7.5/8.7.3) id RAA07546; Tue, 8 Apr 1997 17:01:06 -0700 (PDT) Message-Id: <199704090001.RAA07546@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA100434064; Tue, 8 Apr 1997 19:01:04 -0500 From: Raja Daoud Subject: Re: questions in terms chapter To: Andrew.Lumsdaine.1@nd.edu Date: Tue, 08 Apr 1997 19:01:04 CDT Cc: wcs@nsgi3.lbl.gov, nevin@osc.edu, lederman@cs.wisc.edu, mpi-core@mcs.anl.gov, jsquyres@lsc.nd.edu In-Reply-To: <199704082158.QAA22008@dartagnan.lsc.nd.edu>; from "Andrew.Lumsdaine.1@nd.edu" at Apr 8, 97 4:58 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > One big caveat here -- when one says the constants can be used for > initialization like C constants, I believe that means they can be used > before MPI_Init() is called. So, implementations using pointers may > have a problem (and many do), in that the values of the pointers (and > the structures they reference) are set when MPI_Init() is called. Your belief is correct, they can be used for initialization before MPI_Init() is called. It works as Nick described it, and we use this method as well. The difference is between making the objects be pointer whose values are filled by MPI_Init() [this is not 1.1 compliant] or be constant addresses of static structures, the structures being filled by MPI_Init() [this is 1.1 compliant]. So in summary: don't mandate objects to be integers. Keep the 1.1 text as is in this respect. It gives users what they want, and allows one less indirection for C codes. --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/wsg/ssa/mpi/mpihome.html From mpi-core-human@mcs.anl.gov Wed Apr 9 05:33:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA20228; Wed, 9 Apr 1997 05:33:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA22333 for mpi-core-out; Wed, 9 Apr 1997 04:34:16 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA22328 for ; Wed, 9 Apr 1997 04:34:12 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id LAA26736 (8.6.13/IDA-1.6 for ); Wed, 9 Apr 1997 11:34:29 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for mpi-core@mcs.anl.gov) id LAA04402; Wed, 9 Apr 1997 11:34:29 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199704090934.LAA04402@awsrr.rus.uni-stuttgart.de> Subject: F77&90 problems/advices in Chap.8? (fwd) To: mpi-core@mcs.anl.gov Date: Wed, 9 Apr 1997 11:34:29 +0200 (DST) Content-Type: text Content-Length: 3521 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Forum, the following proposal moves the section "A Fortran Problem with Register Optimization" from the Terms Chapter to the F90 Binding Chapter. It also adds the therefore necessary item on the list on page 10. It is discussed in mpi-bind without any objection and positive 'votes' by Bill Saphir and John Reid. Therefore I bring it now to mpi-core. If there is no objection I will ask Steve and Bill to make the movement in the draft for the next meeting. Rolf Forwarded message: | From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) | Subject: F77&90 problems/advices in Chap.8? | To: mpi-bind@mcs.anl.gov | Date: Tue, 8 Apr 1997 14:19:45 +0200 (DST) | | Dear group, | | I also agree that we need the detailed text from John about | "A Fortran Problem with Copying Dummy Arguments". | | Now we have two advices which are valid for Fortran 77 and 90 | voted or proposed in chapter 2 | and about 4 further advices only for Fortran 90 in chapter 8. | | I agree -- as mentioned in some mails to that topics -- | that we should put together all the text in Chapter 8. | | | Therefore I propose: | | - Making the subsubsection of 8.2.2 to ordinary subsections, | - moving the subsection 2.6.2 into that "list" of subsections, | - mentioning in 8.2.2 which topics are also valid for Fortran 77, | - adding the following item to list in "Fortran Binding Issues" | on page 10: | | Using non-blocking MPI routines or persistent requests, or | passing a buffer by using MPI_BOTTOM or another anchor | as argument (i.e. the compiler cannot see that the buffer | is passed to an MPI routine) it is assumed that Fortran 77 and | 90 passes actual arguments by address and that in some cases no | register optimization is done. The advices to users in the | sections {\bf 8.2.xx "A Fortran Problem with Register Optimization"} | and {\bf 8.2.yy "A Fortran Problem with Copying Dummy Arguments"} | are valid also for Fortran 77 although they are part of the | Fortran 90 binding section. | They should be carefully considered in writing portable Fortran 77 | and 90 applications. | | | Then we have most of the Fortran text together but also the | Fortran 77 users have a chance to find the new text due to the | bold faced references. | | Because this is a proposal for a larger change in Chapter Terms | I shall propose it to mpi-core only if it is agreed on mpi-bind. | | Regards, | Rolf : From: "William Saphir" : : I feel the same about the register problem. Originally : the idea was to keep this in chapter 2 because chapter : 8 talks about Fortran 90 where this is a problem for : any Fortran. As per discussion at last meeting (or before - : I forget) the chapter 8 discussion is no longer : Fortran 90 specific (one thing that remains to be changed : is the title of section 8.2 - Fortran 90 Support). : From: jkr@letterbox.rl.ac.uk (John Reid) : : I support Rolf on this. I have always thought it was very awkward : to split this material between chapters 2 and 8. I have changed : my proposed new section 2.6.2a to edits for 8. Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Wed Apr 9 12:02:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA25425; Wed, 9 Apr 1997 12:01:58 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA01062 for mpi-core-out; Wed, 9 Apr 1997 11:05:58 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA01055; Wed, 9 Apr 1997 11:05:53 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id LAA10718; Wed, 9 Apr 1997 11:56:22 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id MAA26859; Wed, 9 Apr 1997 12:06:11 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256474.00586902 ; Wed, 9 Apr 1997 12:05:40 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: squyres@cse.nd.edu cc: mpi-io@mcs.anl.gov, mpi-core@mcs.anl.gov Message-ID: <85256474.005830FE.00@watngi01.watson.ibm.com> Date: Wed, 9 Apr 1997 12:05:05 -0400 Subject: Re: A bunch of IO issues Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As far as I know, the 30-character limit for MPI routine names was imposed for FORTRAN. Therefore, 30 is fine, but 35 would not be fine. Jean-Pierre squyres @ cse.nd.edu 04/09/97 11:40 AM Please respond to squyres@cse.nd.edu To: mpi-io @ mcs.anl.gov cc: (bcc: Jean-Pierre Prost/Watson/IBM Research) Subject: Re: A bunch of IO issues Jean-Pierre Prost wrote: > Before worrying about finding a shortening scheme which is satisfactory, > I would like to know how many of us are strongly opposed to slightly > breaking the new naming rules (the same way they are already broken by > MPI_SEND). Personally, I am not opposed to it. The difference is that MPI_SEND is already written in stone. The majority of MPI programs that are already in existence most likely use some form of MPI_?SEND. Therefore, it really couldn't be renamed without breaking existing MPI code. But since all the MPI-2 names are not fixed yet, there should be an attempt to make them conform to the rest of MPI-2 rather than be different. Since MPI is a standard, it would certainly seem to make sense that it should be consistent with itself, no? Also, the name will include the "File" in the C++ binding anyway. It won't be in the formal name, but it is in the scoping. So if the IO names are not fixed, the bindings between C and Fortran will be "different" from the C++ bindings. I'm not attached to the shortening scheme that I proposed earlier, but if 30 characters is "not too long", yet 35 is, I fail to see a major difference. :-) {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to Notre Dame for 4 years and ended up staying for a decade." From mpi-core-human@mcs.anl.gov Fri Apr 11 18:00:05 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA02504; Fri, 11 Apr 1997 18:00:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA04794 for mpi-core-out; Fri, 11 Apr 1997 17:04:13 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA04789 for ; Fri, 11 Apr 1997 17:04:07 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id RAA21238; Fri, 11 Apr 1997 17:04:39 -0500 (CDT) Date: Fri, 11 Apr 1997 17:04:39 -0500 (CDT) Message-Id: <199704112204.RAA21238@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: T-SHIRT ORDER FORM Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Well, the votes are in and I have arranged for the T-shirts. At the end of this note is an order form. If you want a T-shirt, please fill it in and return it to me. IF YOU WANT TO BE GUARANTEED A SHIRT, I NEED TO HAVE YOUR ORDER BY TUESDAY, APRIL 15 AT 15:00 CENTRAL. I may be able to add items until Friday, April 18 but I cannot promise. I will be bringing the shirts to the meeting in two weeks and collecting money there. Keep in mind that I am personally paying for these shirts so ordering one is a hard commitment to pay for it. The vote for the front and the back of the T-shirt was both 2 to 1. The vote was to use the revised logo on the front and nothing on the back. If you want to see what the front of the T-shirt will look like go to http://www.cs.wisc.edu/~lederman/mpif/tshirt/tshirt.jpg or ftp://ftp.cs.wisc.edu/pub/mpi2/tshirt/tshirt.jpg. The back will be blank. Many thanks to all who put forth ideas and especially Michael Mott for his original design and Ron Brightwell for his modification. The T-shirt will be made by doing a four color separation so the color should be pretty close to what is in the above image. The graphics company says it will look best on a white T-shirt. Natural is ok but the colors in the design will be slightly different. If you want 100% cotton it is 25 cents extra (the default price is for 50% cotton/50% polyester). I got a university discount and they are running a special, so the short sleeve shirts are cheaper than I said before. Prices are in US$. All price include tax and my small delivery charge :-). So order now for this limited time offer. At these prices you should get one for everyone in your family! Steve ---------------------------------------------------------------------- ORDER FORM ---------------------------------------------------------------------- If you want the item indicated, please place the size or sizes desired (S,M,L,XL). White short sleeve 50/50 shirt at $5.50/shirt: Natural short sleeve 50/50 shirt at $5.50/shirt: White long sleeve 50/50 shirt at $9.25/shirt: Natural long sleeve 50/50 shirt at $9.25/shirt: Mark the following items with an X if it applies to your order. Make shirts 100% cotton at $0.25/shirt: (Note: if you order more than one and do not want them all to be 100% cotton then please list which ones are to be 100% cotton.) My shirt cannot be picked up so it will need to be mailed: (Note: I will send you the final cost with postage and you can then mail me a check and then I will send the shirt to you. I expect it will cost a couple of dollars extra for postage within the U.S.) send to: lederman@cs.wisc.edu ---------------------------------------------------------------------- From mpi-core-human@mcs.anl.gov Sat Apr 12 14:53:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA15481; Sat, 12 Apr 1997 14:53:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA16957 for mpi-core-out; Sat, 12 Apr 1997 13:57:12 -0500 Received: from xipe-totec.cacr.caltech.edu (root@xipe-totec.cacr.caltech.edu [131.215.145.166]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA16952 for ; Sat, 12 Apr 1997 13:57:08 -0500 Received: from willow.cacr.caltech.edu (rajesh@willow.cacr.caltech.edu [131.215.151.104]) by xipe-totec.cacr.caltech.edu (8.8.5/8.8.5) with SMTP id LAA03769; Sat, 12 Apr 1997 11:56:12 -0700 (PDT) Received: by willow.cacr.caltech.edu (8.6.12) id LAA12009; Sat, 12 Apr 1997 11:57:42 -0700 Date: Sat, 12 Apr 1997 11:57:32 -0700 (PDT) From: "Rajesh R. Bordawekar" X-Sender: rajesh@willow To: Steve Huss-Lederman cc: mpi-core@mcs.anl.gov Subject: Re: T-SHIRT ORDER FORM In-Reply-To: <199704112204.RAA21238@rap.cs.wisc.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi Steve, Here's my order rajesh > ---------------------------------------------------------------------- > ORDER FORM > ---------------------------------------------------------------------- > > White short sleeve 50/50 shirt at $5.50/shirt: Large > Make shirts 100% cotton at $0.25/shirt: X > From mpi-core-human@mcs.anl.gov Sun Apr 13 21:42:09 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA25974; Sun, 13 Apr 1997 21:42:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA04925 for mpi-core-out; Sun, 13 Apr 1997 20:44:57 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA04912; Sun, 13 Apr 1997 20:44:50 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id SAA25533; Sun, 13 Apr 1997 18:45:35 -0700 Date: Sun, 13 Apr 1997 18:45:35 -0700 Message-Id: <199704140145.SAA25533@nsgi3.lbl.gov> From: William Saphir To: mpi-bind@mcs.anl.gov, mpi-io@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: more on naming Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Some comments on the current naming discussion: 1. I second the proposal of Jean-Pierre for I/O chapter names. For MPI_FILE_OPEN in particular let's not use bad precedent from MPI-1 and leave out "FILE". Can't speak for other chapters, but in the case of dynamic, it will be MPI_COMM_SPAWN, MPI_COMM_ATTACH and MPI_COMM_JOIN for the next draft. The abbreviations seem reasonable and avoid the >31 character problem. They also preserve FILE in the name. 2. Except: For all two-phase collective functions, I propose we use "BEGIN" instead of "START", which is associated with persistent operations. 3. There are a number of functions that create objects where we are debating what object to attach them to. John May asks a good question: > What possible business does a communicator object have knowing > how to initialize a file object? Is is also true that MPI::File f; f.Open(...) is bizarre (and we have explicitly rejected before). IMHO the right solution (stated by others as well) is to make these pseudo-constructors static class methods, not regular member functions. There are several functions in MPI-2 that fall into this category. These include: static MPI::Comm::Inter MPI::Comm::Join(int fd) static MPI::File MPI::File::Open(comm, ...) static MPI::Info MPI::Info::Create() static MPI::Win MPI::Win::Create(comm, ...) These are used like: comm = MPI::Comm::Join(fd); fh = MPI::File::Open(...); info = MPI::Info::Create(...); win = MPI::Win::Create(...); The C/Fortran names follow naturally from this model. MPI_COMM_JOIN MPI_FILE_OPEN MPI_INFO_CREATE MPI_WIN_CREATE I'm leaning towards putting MPI_TYPE_CREATE_F90 in this category too, even though there's already a datatype object it could be attached to. This is because it can only be called with named MPI datatypes, and not general datatypes. Bill From mpi-core-human@mcs.anl.gov Tue Apr 15 23:00:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA07358; Tue, 15 Apr 1997 23:00:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA02448 for mpi-core-out; Tue, 15 Apr 1997 22:04:03 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA02443 for ; Tue, 15 Apr 1997 22:03:53 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id WAA14246; Tue, 15 Apr 1997 22:03:53 -0500 (CDT) Date: Tue, 15 Apr 1997 22:03:53 -0500 (CDT) Message-Id: <199704160303.WAA14246@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: T-shrit verification and ordering Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Everyone who has ordered a T-shirt should have received a verification message to check over. Due to the large number of orders (over 100), the price is less than what I said before. The new costs are: $4.75 for short sleeve $8.50 for long sleeve $0.25 extra for each 100% cotton Thus, the new cost is 75 cents less per shirt. If you already wrote a check don't worry - I will give back the difference at the meeting. I should be able to accept corrections and additional orders until 12:00 (noon) Central US on Friday, April 18. I already know there is a 25 cent mistake somewhere so hats off to whoever finds it. However, this means I may have made other mistakes so please take a minute to check your order over. I don't have extra money to cover mistakes. Thanks, Steve From mpi-core-human@mcs.anl.gov Thu Apr 17 11:26:10 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA29347; Thu, 17 Apr 1997 11:26:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA10707 for mpi-core-out; Thu, 17 Apr 1997 10:24:03 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA10688 for ; Thu, 17 Apr 1997 10:23:54 -0500 Message-Id: <199704171523.KAA10688@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: April MPI Forum meeting Date: Thu, 17 Apr 1997 10:23:50 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group, I have been remiss in not sending out the usual reminder to register early. If you have not already made your reservations, you will find that the time has expired for obtaining the special rate. The normal rate is $147. Please let me know now for the head count whether you are coming. Thanks. Rusty The usual details: ______________________________________________________________________ The next meeting of the MPI Forum will take place on Wed.-Fri. April 23-25. at the Ramada Hotel O'Hare. Register at the hotel by calling (847) 827-5131. When you arrive at the airport, follow the new, red signs to the bus and taxi stand. For those of you familiar with O'Hare, this is the old Terminal 4, where the international terminal used to be, on the first floor of the parking garage, across from the Hilton lobby. There a red bus from the Ramada Hotel stops every 15 minutes. The Hotel is about five minutes away. If you arrive at a strange time, you might need to call the hotel (number above) to ask to be picked up. The meeting will start at 1:30 on Wednesday and finish at noon on Friday. The registration fee will be $75, which should be paid at the meeting with cash or a check made out to Argonne National Laboratory. We will have a receipt for you. If you have any problems or questions, let me know, either by email (lusk@mcs.anl.gov) or phone (708-252-7852). From mpi-core-human@mcs.anl.gov Thu Apr 17 17:00:35 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA04328; Thu, 17 Apr 1997 17:00:34 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA20549 for mpi-core-out; Thu, 17 Apr 1997 16:05:34 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA20544 for ; Thu, 17 Apr 1997 16:05:30 -0500 Message-Id: <199704172105.QAA20544@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Are you on the list? Date: Thu, 17 Apr 1997 16:05:26 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the list of people who have told me that they are coming to the MPI Forum meeting next week. I need this number to commit by tomorrow to the number of meals. If you are coming and are not listed here, please let me know. Thanks. Rusty Ying Chen Dennis Cottel Terry Dontje Mark Fallon Al Geist Bill Gropp Leslie Hart Shane Hebert Rolf Hempel Hans-Christian Hoppe Steve Landherr John Hagedorn Steve Huss-Lederman Lloyd Lewins Andrew Lumsdaine Rusty Lusk John May Tyce McLarty Harish Nag Bill Nitzberg Heidi Poxon Tom Robey Anna Rounbehler Eric Salo Tony Skjellum Jeff Squyres Ian Stockdale Rajeev Thakur MaryDell Tholburn Dick Treumann Parkson Wong From mpi-core-human@mcs.anl.gov Thu Apr 17 17:29:20 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA04739; Thu, 17 Apr 1997 17:29:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA19771 for mpi-core-out; Thu, 17 Apr 1997 15:32:29 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA19759 for ; Thu, 17 Apr 1997 15:32:22 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id PAA25492; Thu, 17 Apr 1997 15:32:18 -0500 (CDT) Date: Thu, 17 Apr 1997 15:32:18 -0500 (CDT) Message-Id: <199704172032.PAA25492@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A new version of the MPI-2 document is now available on http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. The JOD is also available as the file mpi2-jod.ps.Z. This is the version that is going to be handed out at the meeting next week. In addition, each chapter is available in postscript form. We are fast approaching the end so I encourage everyone to carefully look the document over. From experience, I can tell you the easiest way to give people corrections is to hand them a marked up copy at the meeting. Steve From mpi-core-human@mcs.anl.gov Thu Apr 17 22:05:54 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA08069; Thu, 17 Apr 1997 22:05:53 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA25267 for mpi-core-out; Thu, 17 Apr 1997 21:09:46 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA25261 for ; Thu, 17 Apr 1997 21:09:42 -0500 Message-Id: <199704180209.VAA25261@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: MPI 2 draft in HTML form Date: Thu, 17 Apr 1997 21:09:39 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I've automated the process of producing an HTML version from the official LaTeX version. The current draft is available at http://www.mcs.anl.gov/Projects/mpi/mpi2/mpi2-report/mpi2-report.html I load this thing into my laptop. There are a few bugs; for example, the section numbers are advanced by part as well as chapter, so they don't match the LaTeX version. There are a few bugs in handling reserved HTML characters (& in particular). On the other hand, the index works, so that it is easy to find functions. Bill From mpi-core-human@mcs.anl.gov Fri Apr 18 10:36:51 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA16794; Fri, 18 Apr 1997 10:36:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA03924 for mpi-core-out; Fri, 18 Apr 1997 09:41:27 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA03918 for ; Fri, 18 Apr 1997 09:41:23 -0500 Message-Id: <199704181441.JAA03918@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: are you on the revised list? Date: Fri, 18 Apr 1997 09:41:23 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Here is the revised list of people attending next week's meeting. There are a few people who have ordered T-shirts but are not on this list so I suspect it is still incomplete. Today I need to tell the hotel how many are coming, so please let me know if you are coming and are not listed here. Terry? Brian? Richard? Albert? Peter? Anyone else? Thanks. Rusty Ron Brightwell Pang Chen Ying Chen Lyndon Clarke Dennis Cottel Raja Daoud Judy Devaney Terry Dontje Mark Fallon Al Geist Bill Gropp Leslie Hart Shane Hebert Rolf Hempel Hans-Christian Hoppe Steve Landherr John Hagedorn Steve Huss-Lederman Lloyd Lewins Andrew Lumsdaine Rusty Lusk John May Tyce McLarty Harish Nag Nick Nevin Bill Nitzberg Heidi Poxon Jean-Pierre Prost Rolf Rabenseifner Tom Robey Anna Rounbehler Eric Salo William Saphir Marc Snir Tony Skjellum Jeff Squyres Ian Stockdale Rajeev Thakur MaryDell Tholburn Dick Treumann Parkson Wong From mpi-core-human@mcs.anl.gov Fri Apr 18 15:43:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA23333; Fri, 18 Apr 1997 15:43:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA11787 for mpi-core-out; Fri, 18 Apr 1997 14:46:58 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA11781 for ; Fri, 18 Apr 1997 14:46:51 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id OAA29890; Fri, 18 Apr 1997 14:46:52 -0500 (CDT) Date: Fri, 18 Apr 1997 14:46:52 -0500 (CDT) Message-Id: <199704181946.OAA29890@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: T-shirt - paying and picking up Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I have now sent in the final order for the T-shirts. In total, we ordered 132 shirts. As a result, the logistics of handing them out will be fun. To clarify what I see as happening and to answer some questions I have beeen given, I tell everyone the following. You are welcome to bring a check to pay for your order. Make it out to "Steve Huss-Lederman". It needs to be in US$ drawn from a US bank. Alternatively, you can pay me in cash. As the flight attendants say: "Exact change is always appreciated". I should be in our room after 12:00 on Wednesday if you want to buy your shirt before the meeting. After we start, I will distribute during breaks, etc. Given the large number of shirts, I cannot carefully inspect each one. Take a minute to do that when you get your order. It will be easier on both of us if I can deal with mistakes at the meeting. I look forward to seeing you all next week. Steve From mpi-core-human@mcs.anl.gov Mon Apr 21 11:16:49 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA24018; Mon, 21 Apr 1997 11:16:48 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA20786 for mpi-core-out; Mon, 21 Apr 1997 10:19:58 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA20778 for ; Mon, 21 Apr 1997 10:19:51 -0500 Message-Id: <199704211519.KAA20778@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: The JOD Date: Mon, 21 Apr 1997 10:20:01 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Although I would not like to spend any significant time on it at the upcoming meeting, the MPI Journal of Development needs one brief look before we are finished with it. It will be distributed as a compeletely separate document at the meeting, and I do believe that we should read and approve of at least the introduction, in which I have tried to define just what the JOD is and, perhaps more importantly, what it is not. I would also like the "chapter authors" of sections of the JOD to update their sections one more time. There are only minor problems, mainly having to do with continuity. In some cases it is a little too obvious that the material was ripped out of the middle of something else. It would also help if the placeholder introductory paragraphs I have supplied were updated by their authors, now with the context of their being in the JOD. There is a lot of good reading in the JOD, and one more brief pass by the authors can raise its quality one more notch. The JOD is currently contained in several .tex files in the distribution. intro-2-jod.tex - (new) introduction to the jod misc-jod.tex - six smallish chapters rt-2.tex - real-time I propose that everyone who is an author of a part of misc-jod.tex get the current version from Steve, update his chapter of it, and send the result to me. I will coordinate the reassembly into one misc-jod.tex. Tony will remain in charge of the real-time material. The misc chapters and the people who have responsibility for them are: Spawning Independent Processes - Bill Saphir Threads and MPI - Marc Snir Communicator ID - Steve Huss-Lederman Miscellany - Rusty Lusk Torwards a Full Fortran-90 Interface - Bill Saphir Two-phase Collective Operations - Bill Gropp If anyone has suggestions or corrections on the JOD chapters, please send them to the people above (soon). I will try to pull the whole thing back together in the week right after the meeting. Rusty From mpi-core-human@mcs.anl.gov Mon Apr 21 15:21:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA01936; Mon, 21 Apr 1997 15:21:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA27395 for mpi-core-out; Mon, 21 Apr 1997 14:23:39 -0500 Received: from ncsa.uiuc.edu (sdgmail.ncsa.uiuc.edu [141.142.103.66]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA27376; Mon, 21 Apr 1997 14:23:17 -0500 Received: from narnia.ncsa.uiuc.edu (narnia.ncsa.uiuc.edu [141.142.21.165]) by ncsa.uiuc.edu (8.8.5/8.8.5) with SMTP id OAA02193; Mon, 21 Apr 1997 14:23:43 -0500 (CDT) Message-Id: <3.0.32.19970421142449.00696e2c@sdgmail.ncsa.uiuc.edu> X-Sender: acheng@sdgmail.ncsa.uiuc.edu X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Mon, 21 Apr 1997 14:24:50 -0500 To: mpi-io@mcs.anl.gov From: Albert Cheng Subject: Re: Standard Character representation Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 01:47 PM 4/21/97 -0500, Albert Cheng wrote: >Is it acceptable to use website address as reference? E.g., > "The External representation for MPI_CHAR and MPI_CHARACTER should > be of ISO-8859:1987 (see "http://www.iso.ch")." > >Actually, can I say that ISO is so well known that it is unnecesarily to >make a formal reference for it? Never mind. Bill was couple steps ahead of me and put in a formal reference already. (Thanks, Bill.) From mpi-core-human@mcs.anl.gov Mon Apr 21 15:29:26 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA02058; Mon, 21 Apr 1997 15:29:24 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA26379 for mpi-core-out; Mon, 21 Apr 1997 13:47:29 -0500 Received: from ncsa.uiuc.edu (sdgmail.ncsa.uiuc.edu [141.142.103.66]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA26331; Mon, 21 Apr 1997 13:46:09 -0500 Received: from narnia.ncsa.uiuc.edu (narnia.ncsa.uiuc.edu [141.142.21.165]) by ncsa.uiuc.edu (8.8.5/8.8.5) with SMTP id NAA01798; Mon, 21 Apr 1997 13:46:28 -0500 (CDT) Message-Id: <3.0.32.19970421134735.006a6e54@sdgmail.ncsa.uiuc.edu> X-Sender: acheng@sdgmail.ncsa.uiuc.edu X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Mon, 21 Apr 1997 13:47:35 -0500 To: mpi-io@mcs.anl.gov From: Albert Cheng Subject: Re: Standard Character representation Cc: mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 04:33 PM 4/18/97 -0700, Ian E. Stockdale wrote: >> I need a reference about the character representation standard >> quoted on 10.5.2, p241, line 18. I hunt around the net and >> found a reference of ISO-8859-1. (Here is the URL of one of the >> references: "http://www.mrc-apu.cam.ac.uk/html/iso_table.html"). >> Is this the correct one to use? > >You should go to the ISO home page (http://www.iso.ch), and follow >the "catalogue" link. This eventually gets you to: > >================================================================= >ISO 8859-1:1987 >Information processing -- 8-bit single-byte coded graphic character >sets -- Part 1: Latin alphabet No. 1 > >Edition: 1 (monolingual) >Number of pages: 7 >Price code: D >Technical committee / subcommittee: JTC 1 / SC 2 >ICS: 35.040 > >Descriptors: coded character sets, data codes, data processing, data >transmission, information interchange, Latin characters >================================================================= > [Thanks Ian, for the reference.] Is it acceptable to use website address as reference? E.g., "The External representation for MPI_CHAR and MPI_CHARACTER should be of ISO-8859:1987 (see "http://www.iso.ch")." Actually, can I say that ISO is so well known that it is unnecesarily to make a formal reference for it? From mpi-core-human@mcs.anl.gov Mon Apr 21 15:39:18 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA02246; Mon, 21 Apr 1997 15:39:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA27930 for mpi-core-out; Mon, 21 Apr 1997 14:42:33 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA27925 for ; Mon, 21 Apr 1997 14:42:29 -0500 Message-Id: <199704211942.OAA27925@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: small T-shirt subsidy Date: Mon, 21 Apr 1997 14:42:46 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The MPI Forum meeting fund (your $75) has been operating pretty close to even. Nonetheless, it looks like we might have a few bucks left over (not many) after the upcoming meeting. In an effort to keep this balance as low as possible, the fund will buy everyone's first T-shirt. This applies only to T-shirts *already ordered*. Those who thought $5 was too expensive but do want a freebie are out of luck. Rusty From mpi-core-human@mcs.anl.gov Mon Apr 21 22:34:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA12500; Mon, 21 Apr 1997 22:34:12 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA06080 for mpi-core-out; Mon, 21 Apr 1997 21:38:38 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA06074 for ; Mon, 21 Apr 1997 21:38:34 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id VAA13911; Mon, 21 Apr 1997 21:38:55 -0500 (CDT) Date: Mon, 21 Apr 1997 21:38:55 -0500 (CDT) Message-Id: <199704220238.VAA13911@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: new T-shirt price & verification Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Everyone who ordered a T-shirt should have recently received a new verification. It should be the same as your last one except that the price now reflects the updated price. The new price is due to subsidy of the first shirt as described in Rusty's recent mail. For those who already have checks from others I will refund the difference to you at the meeting. Steve From mpi-core-human@mcs.anl.gov Wed Apr 23 01:41:33 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id BAA22568; Wed, 23 Apr 1997 01:41:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id AAA06868 for mpi-core-out; Wed, 23 Apr 1997 00:45:26 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id AAA06862 for ; Wed, 23 Apr 1997 00:45:21 -0500 Message-Id: <199704230545.AAA06862@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: agenda Date: Wed, 23 Apr 1997 00:45:48 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk For those who like to have the agenda ahead of time ( :-) ), here is the highly tentative agenda for tomorrow's meeting. Rusty ______________________________________________________________________ April 23-25, 1997 Room A = Salon A Room B = 3605 Room C = 4605 Unless we decide otherwise, all sessions are in Room A Wednesday --------- 1:30-2:00 Discussion of Agenda and Assorted Issues 2:00-2:30 Introduction and JOD Introduction - votes 2:30-3:30 Terms - second vote 3:30-4:00 Break 4:00-5:30 I/O - new material 6:00-8:00 Dinner 8:00-9:00 One-sided - final approval 9:00-10:00 Dynamic - final approval Thursday -------- 8:30-10:00 I/O second votes 10:00-10:30 Break 10:30-12:00 External second votes 12:00-1:00 Lunch 1:00-3:00 I/O 3:00-3:30 Break 3:30-4:30 Collective and Miscellany 4:30-5:30 Bindings (esp. Fortran) 6:00-8:00 Dinner 8:00-10:00 I/O Friday -------- 8:30-10:00 Room A: Votes to be scheduled 10:00-10:15 Break 10:15-12:00 Room A: Votes to be scheduled 12:00 Adjourn From mpi-core-human@mcs.anl.gov Thu Apr 24 08:55:22 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA05945; Thu, 24 Apr 1997 08:55:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA06104 for mpi-core-out; Thu, 24 Apr 1997 07:57:47 -0500 Received: from mailhost.lanl.gov (mailhost.lanl.gov [128.165.3.12]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA06099 for ; Thu, 24 Apr 1997 07:57:43 -0500 Received: from wrangler.lanl.gov (wrangler.lanl.gov [128.165.113.131]) by mailhost.lanl.gov (8.8.5/8.8.3) with ESMTP id GAA18565 for ; Thu, 24 Apr 1997 06:58:17 -0600 (MDT) Received: from sooks.lanl.gov (transitory92.lanl.gov [128.165.7.108]) by wrangler.lanl.gov (8.7.6/8.7.3) with SMTP id GAA19826 for ; Thu, 24 Apr 1997 06:58:13 -0600 (MDT) Message-Id: <2.2.32.19970424130318.006aeb94@wrangler.lanl.gov> X-Sender: marydell@wrangler.lanl.gov X-Mailer: Windows Eudora Pro Version 2.2 (32) Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Thu, 24 Apr 1997 07:03:18 -0600 To: mpi-core@mcs.anl.gov From: MaryDell Tholburn Subject: ISCOPE '97 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk CALL FOR PAPERS International Scientific Computing in Object-Oriented Parallel Environments The first International Scientific Computing in Object-Oriented Parallel Environments (ISCOPE) Conference will be held at the Marina Beach Marriott Hotel in Marina del Rey, California, from December 8 to December 11, 1997. This conference represents the combination of the POOMA, IWPC++, and OONSCI conferences. ISCOPE '97 seeks to encompass a broad range of topics which address object-orientation and high-performance scientific computing in a forum which spans government research institutions, academic institutions, and commercial developers. Topics represented at this conference include: Scientific applications enabled by object technology. Parallel, distributed, and serial problem solving environments. Software portability, components, and reuse. Heterogeneous computing environments. Compiler technology. Parallel extensions to C++. Java - performance and parallelism. High-performance run-time systems. Object-oriented parallel languages. Performance issues related to object-oriented programming. Object-oriented tools and class libraries. Standards for object interoperability If you would like to present a paper at ISCOPE '97, please submit a four page extended abstract in either HTML, pdf, Microsoft Word, Adobe FrameMaker, or PostScript to anonymous ftp at ftp://ftp.acl.lanl.gov/iscope no later than June, 6 1997. Notification of acceptance will occur by July 17, 1997. There will be a published proceedings for ISCOPE '97 in Springer-Verlag for which final papers will be due by Aug 14th, 1997. Final paper acceptance is contingent upon receipt of the whole paper by Aug 14th, 1997. If you have any questions about submitting a paper, please send email to iscope97@acl.lanl.gov or call Marydell Tholburn at 505-667-0619. ISCOPE '97 Organizing Committee: Scott Baden - University of California - San Diego Denis Caromel - University of Nice - INRIA Sophia Antipolis Dennis Gannon - Indiana University Yutaka Ishikawa - Real World Computing Partnership Carl Kesselman - California Institute of Technology/USC-ISI Satoshi Matsuoka - Tokyo Institute of Technology Rod Oldehoeft - US Dept of Energy/Colorado State University John Reynders - Los Alamos National Laboratory Tony Skjellum - Mississippi State University Further information is available at http://www.acl.lanl.gov/iscope MaryDell From mpi-core-human@mcs.anl.gov Mon Apr 28 00:38:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA06594; Mon, 28 Apr 1997 00:38:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA16695 for mpi-core-out; Sun, 27 Apr 1997 23:39:36 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA16690; Sun, 27 Apr 1997 23:39:32 -0500 Message-Id: <199704280439.XAA16690@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Cc: dongarra@cs.utk.edu Subject: Publishing MPI-2 Date: Sun, 27 Apr 1997 23:40:26 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear group, I have received an offer from the editor, Jack Dongarra, to publish the MPI-2 Standard in the International Journal of Supercomputer Applications and High-Performance Computing, which published the MPI-1 standard. The author would be "The MPI Forum", as it was for MPI-1. I believe this is a good idea, to provide an archival version and a standard reference for the document. If there are no objections, I will agree to this and arrange for the very final version of the document to be published in IJSA. Rusty From mpi-core-human@mcs.anl.gov Tue Apr 29 15:38:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA14692; Tue, 29 Apr 1997 15:38:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA27022 for mpi-core-out; Tue, 29 Apr 1997 14:39:38 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA27017 for ; Tue, 29 Apr 1997 14:39:35 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id OAA09233; Tue, 29 Apr 1997 14:40:38 -0500 (CDT) Date: Tue, 29 Apr 1997 14:40:38 -0500 (CDT) Message-Id: <199704291940.OAA09233@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: schedule for final document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As we discussed at the meeting last week, here is the planned schedule for finishing up the MPI-2 documents: Wednesday, May 7 at 12:00: All input from chapter authors is due to me from chapter authors. By Friday, May 9: I will prepare and release a version to the Forum for review. This will begin the last chance for people to make detailed comments on the draft. Friday, May 23: All comments should be to the chapter authors. They begin final edits in preparation for the chapter authors meeting. Friday, May 30: Chapter authors get me their chapters. This is the frozen version from which we will make the final edits. At this point I have control over the complete document. Saturday/Sunday, June 7/8: The chapter authors will met at Argonne to do the final document cleanup. Tuesday, June 10: The final check version of the document will be released to the Forum. All comment must come to me since I have control of the document. ONLY MINOR CHANGES WILL BE MADE THAT CAME ABOUT FROM THE FINAL EDITS. No further revisions will occur at this point. Tuesday, June 17 at 10:00: All comments are due. The small changes will be made to produce the final documents. Around Thursday, June 19: The final document will be released. All times are Central US. You should not publicly distribute the document until you have received mail from me indicating that the final version is done and its location. We are trying hard to avoid the situation with MPI-1.1 there were several "final" versions. To do this, it is important that people get their comments in by May 23. Steve From mpi-core-human@mcs.anl.gov Tue May 6 16:44:47 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA05902; Tue, 6 May 1997 16:44:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA22234 for mpi-core-out; Tue, 6 May 1997 15:46:47 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA22229; Tue, 6 May 1997 15:46:43 -0500 Received: from porthos.lsc.nd.edu (porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id PAA21275; Tue, 6 May 1997 15:46:57 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id PAA18278; Tue, 6 May 1997 15:46:56 -0500 (EST) Date: Tue, 6 May 1997 15:46:56 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: mpi-external@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: New errhandler text Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I bring this text to the external and core lists because it has been passed around internally to several people, but everyone on the list should see it. Here's the context: I'm not happy with the new errhandler and caching functions. The only problem that I have with them is that they are essentially the same functions, but times 3 (because we define the same operations on MPI_Comm, MPI_Datatype, and MPI_File). I am requesting a minor change that would ease a lot of implementations (C and C++) while still providing the 3x functions (since it's too late to change that). I appologize in advance for the length of this letter. :-) If you don't care about the details about *why* am asking for all of this, skip to "THE BOTTOM LINE" near the bottom. ===== My main problem with having essentially the same functions 3 times is that there is no potential for code reuse within the implementation (specifically, the C++ implementation, regardless of whether it is layered on C bindings or not). The main reason that there is no possibility for code reuse is the user callback function typedefs. The present scheme has user callback functions, that are typedef'ed something like (I'll show just the C bindings, for simplicity): ------ typedef int MPI_Comm_copy_attr_function(MPI_Comm comm, int comm_keyval, void *extra_state, void *attr_val_in, void *attr_val_out, int *flag); typedef int MPI_Win_copy_attr_function(MPI_Win win, int comm_keyval, void *extra_state, void *attr_val_in, void *attr_val_out, int *flag); typedef int MPI_Type_copy_attr_function(MPI_Datatype type, int comm_keyval, void *extra_state, void *attr_val_in, void *attr_val_out, int *flag); ------ Notice that they only have the first parameter different in each of the three typedefs. This makes the bindings for the MPI_*_CREATE_KEYVAL function be something like this: ------ int MPI_Comm_create_keyval(MPI_Comm_copy_attr_function *copy_fn, ...); int MPI_Win_create_keyval(MPI_File_copy_attr_function *copy_fn, ...); int MPI_Type_create_keyval(MPI_Type_copy_attr_function *copy_fn, ...); ------ Notice that the first parameter is different in all three, only because the first parameter of the respective typedefs are different. Here's what I am proposing: have only *one* typedef for the user callback functions (that takes (void *) as its first argument vs. an MPI opaque handle), and then have the MPI_*_CREATE_KEYVAL functions all take that one typedef as their argument: ------ typedef MPI_Copy_attr_function(void *handle, int keyval, void *extra_state, void *attr_val_in, void *attr_val_out, int *flag); int MPI_Comm_create_keyval(MPI_Copy_attr_function *copy_fn, ...); int MPI_Win_create_keyval(MPI_Copy_attr_function *copy_fn, ...); int MPI_Type_create_keyval(MPI_Copy_attr_function *copy_fn, ...); ------ Notice how they all use that one typedef. The only type checking that we lose is in the first parameter of the user's callback function. This is very minor. The same reasoning can be applied to all the user callback functions for attribute caching and error hander functions. Here's a synopsis of exactly what I am asking for: ----- PART 1: Eliminate all the object-dependant typedefs: MPI_*_copy_attr_function * = { Comm, Win, Type } MPI_*_delete_attr_function MPI_*_handler_function (<-- was there one for for File as well?) They would be replaced with: MPI_Copy_attr_function MPI_Delete_attr_function MPI_Errorhandler_function (can't use MPI_Handler_function -- it's used in MPI-1) ----- PART 2: Change the types of parameters in functions to match these new typedefs. The following functions are affected: MPI_*_Create_keyval * = { Comm, Win, Type } MPI_*_Create_errhandler (<-- File as well? I don't remember) --- This allows the C++ implementation to lump together all the attribute, error handler, and name functions and only implement them once (rather than 3 times). *** THE BOTTOM LINE: I want one callback typedef (of each type -- copy, delete, etc.), that takes a (void *) as its first argument, not an MPI opaque handle. I want this functionality because this allows for an elegant implmentation in C++ (specifically, we can use inheritance -- I can supply the details if anyone wants to know), and the cost is very small (i.e. the change is minor). It can also help in C implementations. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Tue May 6 17:39:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA07066; Tue, 6 May 1997 17:39:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA24193 for mpi-core-out; Tue, 6 May 1997 16:44:19 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA23851; Tue, 6 May 1997 16:33:38 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id OAA01387; Tue, 6 May 1997 14:33:34 -0700 From: "William Saphir" Message-Id: <9705061433.ZM1386@nsgi3.lbl.gov> Date: Tue, 6 May 1997 14:33:33 -0700 In-Reply-To: Jeff Squyres "New errhandler text" (May 6, 3:46pm) References: X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: Jeff Squyres Subject: Re: New errhandler text Cc: mpi-external@mcs.anl.gov, mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On May 6, 3:46pm, Jeff Squyres wrote: > *** THE BOTTOM LINE: I want one callback typedef (of each type -- copy, > delete, etc.), that takes a (void *) as its first argument, not an MPI > opaque handle. I support this. There is no change in functionality from what has been voted on in the final meeting - it just cleans up syntax. The only question I had when reading over your note was whether there would be conflicts with things in MPI-1. I believe there are none. Am I correct that the situation is: 1. We are already proposing functions with new names that reproduce MPI-1 functionality for communicators and extend it for other types. (The old functions remain, but are effectively deprecated, as with the datatype constructors with MPI_Aint arguments). 2. The names of your new proposed typedefs have been carefully chosen to also not conflict with the MPI-1 names. Bill From mpi-core-human@mcs.anl.gov Tue May 6 17:53:07 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA07502; Tue, 6 May 1997 17:53:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA24578 for mpi-core-out; Tue, 6 May 1997 16:57:14 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA24573; Tue, 6 May 1997 16:57:07 -0500 Received: from porthos.lsc.nd.edu (porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id QAA21442; Tue, 6 May 1997 16:57:02 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id QAA18327; Tue, 6 May 1997 16:57:03 -0500 (EST) Date: Tue, 6 May 1997 16:57:03 -0500 (EST) From: Jeff Squyres To: William Saphir cc: mpi-external@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: New errhandler text In-Reply-To: <9705061433.ZM1386@nsgi3.lbl.gov> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Tue, 6 May 1997, William Saphir wrote: > The only question I had when reading over your note was whether > there would be conflicts with things in MPI-1. I believe there > are none. Am I correct that the situation is: > 1. We are already proposing functions with new names that > reproduce MPI-1 functionality for communicators and extend > it for other types. (The old functions remain, but are effectively > deprecated, as with the datatype constructors with MPI_Aint arguments). Correct. In all truthfulness, I would prefer just *one* set of functions (i.e. not essentially the same functions replicated 3 times), but I think it's too late to change that. So therefore, I'm proposing this compromise. > 2. The names of your new proposed typedefs have been carefully chosen to > also not conflict with the MPI-1 names. Correct. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Tue May 6 21:03:28 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA08897; Tue, 6 May 1997 21:03:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA28503 for mpi-core-out; Tue, 6 May 1997 20:07:47 -0500 Received: from win62.nas.nasa.gov (win62.nas.nasa.gov [129.99.50.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA28497; Tue, 6 May 1997 20:07:42 -0500 Received: (from ies@localhost) by win62.nas.nasa.gov (8.8.3/NAS.6.1) id SAA06269; Tue, 6 May 1997 18:07:39 -0700 (PDT) Date: Tue, 6 May 1997 18:07:39 -0700 (PDT) Message-Id: <199705070107.SAA06269@win62.nas.nasa.gov> From: "Ian E. Stockdale" To: mpi-external@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <9705061433.ZM1386@nsgi3.lbl.gov> (wcs@nsgi3.lbl.gov) Subject: Re: New errhandler text Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On May 6, 3:46pm, Jeff Squyres wrote: > *** THE BOTTOM LINE: I want one callback typedef (of each type -- copy, > delete, etc.), that takes a (void *) as its first argument, not an MPI > opaque handle. There seems to be one drawback to this approach, at least in C, which is that the compiler can no longer use the typedef to verify that the first argument is of the correct type. -Ian -- Ian E. Stockdale MRJ Technology Solutions, NASA Ames Research Center ies@nas.nasa.gov (415) 604-4628 From mpi-core-human@mcs.anl.gov Wed May 7 04:45:41 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA03168; Wed, 7 May 1997 04:45:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA03548 for mpi-core-out; Wed, 7 May 1997 03:50:51 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA03543; Wed, 7 May 1997 03:50:47 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id KAA11643 (8.6.13/IDA-1.6); Wed, 7 May 1997 10:50:35 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id KAA00841; Wed, 7 May 1997 10:50:35 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199705070850.KAA00841@awsrr.rus.uni-stuttgart.de> Subject: Re: New errhandler text To: jsquyres@lsc.nd.edu Date: Wed, 7 May 1997 10:50:34 +0200 (DST) Cc: mpi-external@mcs.anl.gov, mpi-core@mcs.anl.gov, nitzberg@nas.nasa.gov (Bill Nitzberg) In-Reply-To: from "Jeff Squyres" at May 6, 97 03:46:56 pm Content-Type: text Content-Length: 710 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >*** THE BOTTOM LINE: I want one callback typedef (of each type -- copy, >delete, etc.), that takes a (void *) as its first argument, not an MPI >opaque handle. >{+} Jeff Squyres If we make this changes, do we then have problems in implementing MPI-I/O by layering on existing rest of MPI-2? And the same question probabely for 1-sided? Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Wed May 7 05:43:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA14125; Wed, 7 May 1997 05:43:01 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA03921 for mpi-core-out; Wed, 7 May 1997 04:46:50 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA03916; Wed, 7 May 1997 04:46:39 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id LAA13009 (8.6.13/IDA-1.6); Wed, 7 May 1997 11:46:28 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id LAA00927; Wed, 7 May 1997 11:46:28 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199705070946.LAA00927@awsrr.rus.uni-stuttgart.de> Subject: Re: New errhandler text To: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Date: Wed, 7 May 1997 11:46:27 +0200 (DST) Cc: jsquyres@lsc.nd.edu, mpi-external@mcs.anl.gov, mpi-core@mcs.anl.gov, nitzberg@nas.nasa.gov In-Reply-To: <199705070850.KAA00841@awsrr.rus.uni-stuttgart.de> from "Rolf Rabenseifner" at May 7, 97 10:50:34 am Content-Type: text Content-Length: 502 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > If we make this changes, do we then have problems in > implementing MPI-I/O by layering on existing rest of MPI-2? Sorry, my question makes no sense, because Jeff only wanted to change the typedefs for the callback functions. He did not want to merge e.g. three MPI_*_ERRHANDLER_CREATE routines to one routine. Therefore there is no problem with layerability. And therefore I explicitly support Jeff's proposal. It reduces pages and the drawback marked by Ian seems to be minor for me. Rolf From mpi-core-human@mcs.anl.gov Wed May 7 10:13:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA17122; Wed, 7 May 1997 10:13:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA07451 for mpi-core-out; Wed, 7 May 1997 09:17:52 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA07446; Wed, 7 May 1997 09:17:47 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id JAA22599; Wed, 7 May 1997 09:17:47 -0500 (CDT) Date: Wed, 7 May 1997 09:17:47 -0500 (CDT) Message-Id: <199705071417.JAA22599@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-external@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: MPI_STATUS_IGNORE implications Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I sent this to core and external since the issue of returning error codes effects several chapters. The details after the first paragraph are the implications for external. A change toward the end of the last meeting was to allow passing MPI_STATUS_IGNORE to MPI_GET_STATUS. We also clarified that WAIT_{ALL|SOME} can still return MPI_ERR_IN_STATUS even if you gave MPI_STATUSES_IGNORE. I believe this means that we wish to return the error status in these cases but not any other information. Thus, passing MPI_STATUS_IGNORE to MPI_GET_STATUS is a test for completion and error code without getting other status info and allowing one to get it later. I think one can also argue that the user didn't want the error code but I don't think this is what the group went for. Doing this would make such calls even cheaper. To return the error code for a generalized request means calling the query_fn even if MPI_STATUS_IGNORE is provided. The current text does not allow this. The first change is to delete the sentence on p. 3/21-23: The callback function is not invoked if the user indicated that no status needs be returned for this request, by supplying \const{MPI\_STATUS\_IGNORE} or \const{MPI\_STATUSES\_IGNORE}. And adding text at p. 3/41: \begin{users} If the user provided \const{MPI\_STATUS\_IGNORE} or \const{MPI\_STATUSES\_IGNORE} to the \MPI/ function that causes the \mpifunc{query\_fn} to be called, then \MPI/ will pass \const{MPI\_STATUS\_IGNORE} to the \mpifunc{query\_fn}. In this case, only the error code is returned (return value in C and C++ and in \mpiarg{IERROR} in Fortran). Implementors of a \mpifunc{query\_fn} should be careful to test for the case where \mpiarg{status} is \const{MPI\_STATUS\_IGNORE} since accessing it as a normal status handle is erroneous. \end{users} From mpi-core-human@mcs.anl.gov Wed May 7 10:47:05 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA18506; Wed, 7 May 1997 10:47:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA08250 for mpi-core-out; Wed, 7 May 1997 09:51:14 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id JAA08229; Wed, 7 May 1997 09:50:59 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA26296; Wed, 7 May 1997 10:48:46 -0400 Received: by windsurf.pok.ibm.com (AIX 4.1/UCB 5.64/4.03) id AA25178; Wed, 7 May 1997 10:50:57 -0400 Date: Wed, 7 May 1997 10:50:57 -0400 From: treumann@kgn.ibm.com (Dick Treumann) Message-Id: <9705071450.AA25178@windsurf.pok.ibm.com> To: @minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov, @minnie.nic.kingston.ibm.com:mpi-external@mcs.anl.gov Subject: Re: New errhandler text Cc: treumann@windsurf.pok.ibm.com Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Does not Jeff's proposal imply that an MPI object handle is "really" a pointer to the object. If an MPI_Comm is really a Comm_Obj* and an MPI_Win is a Win_Obj* then both can be passed as void*. The MPI standard is explicit in saying they NEED NOT BE POINTERS. I do not know if this suggestion runs into problems when a handle is something other than a pointer (an int probably) but those considering this change must assure everyone no problem will show up. I think suitable use of casting can solve the problem in C but I do not know for C++. Within my MPI implementation's C code which calls these functions I will need to cast my handle to void* to be able to pass it to such a callback. If the impact in C++ is no worse then I have no objection. Dick Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Wed May 7 12:29:23 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA20015; Wed, 7 May 1997 12:28:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA11151 for mpi-core-out; Wed, 7 May 1997 11:31:46 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA11145; Wed, 7 May 1997 11:31:41 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id MAA13684; Wed, 7 May 1997 12:32:01 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id MAA25245; Wed, 7 May 1997 12:31:40 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256490.005AC3A0 ; Wed, 7 May 1997 12:31:23 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: jsquyres@lsc.nd.edu cc: Mpi-External@mcs.anl.gov, Mpi-Core@mcs.anl.gov Message-ID: <85256490.005A8534.00@watngi01.watson.ibm.com> Date: Wed, 7 May 1997 12:31:04 -0400 Subject: Re: New errhandler text Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Jeff wrote: They would be replaced with: MPI_Copy_attr_function MPI_Delete_attr_function MPI_Errorhandler_function (can't use MPI_Handler_function -- it's used in MPI-1) *************** I would prefer MPI_Errhandler_function for the third one. *************** ----- PART 2: Change the types of parameters in functions to match these new typedefs. The following functions are affected: MPI_*_Create_keyval * = { Comm, Win, Type } MPI_*_Create_errhandler (<-- File as well? I don't remember) ************** For error handlers, the set of types are Comm, Win and File (Type is not there). ************** Jean-Pierre jsquyres @ lsc.nd.edu 05/06/97 04:46 PM Please respond to jsquyres@lsc.nd.edu To: mpi-external @ mcs.anl.gov, mpi-core @ mcs.anl.gov cc: (bcc: Jean-Pierre Prost/Watson/IBM Research) Subject: New errhandler text I bring this text to the external and core lists because it has been passed around internally to several people, but everyone on the list should see it. Here's the context: I'm not happy with the new errhandler and caching functions. The only problem that I have with them is that they are essentially the same functions, but times 3 (because we define the same operations on MPI_Comm, MPI_Datatype, and MPI_File). I am requesting a minor change that would ease a lot of implementations (C and C++) while still providing the 3x functions (since it's too late to change that). I appologize in advance for the length of this letter. :-) If you don't care about the details about *why* am asking for all of this, skip to "THE BOTTOM LINE" near the bottom. ===== My main problem with having essentially the same functions 3 times is that there is no potential for code reuse within the implementation (specifically, the C++ implementation, regardless of whether it is layered on C bindings or not). The main reason that there is no possibility for code reuse is the user callback function typedefs. The present scheme has user callback functions, that are typedef'ed something like (I'll show just the C bindings, for simplicity): ------ typedef int MPI_Comm_copy_attr_function(MPI_Comm comm, int comm_keyval, void *extra_state, void *attr_val_in, void *attr_val_out, int *flag); typedef int MPI_Win_copy_attr_function(MPI_Win win, int comm_keyval, void *extra_state, void *attr_val_in, void *attr_val_out, int *flag); typedef int MPI_Type_copy_attr_function(MPI_Datatype type, int comm_keyval, void *extra_state, void *attr_val_in, void *attr_val_out, int *flag); ------ Notice that they only have the first parameter different in each of the three typedefs. This makes the bindings for the MPI_*_CREATE_KEYVAL function be something like this: ------ int MPI_Comm_create_keyval(MPI_Comm_copy_attr_function *copy_fn, ...); int MPI_Win_create_keyval(MPI_File_copy_attr_function *copy_fn, ...); int MPI_Type_create_keyval(MPI_Type_copy_attr_function *copy_fn, ...); ------ Notice that the first parameter is different in all three, only because the first parameter of the respective typedefs are different. Here's what I am proposing: have only *one* typedef for the user callback functions (that takes (void *) as its first argument vs. an MPI opaque handle), and then have the MPI_*_CREATE_KEYVAL functions all take that one typedef as their argument: ------ typedef MPI_Copy_attr_function(void *handle, int keyval, void *extra_state, void *attr_val_in, void *attr_val_out, int *flag); int MPI_Comm_create_keyval(MPI_Copy_attr_function *copy_fn, ...); int MPI_Win_create_keyval(MPI_Copy_attr_function *copy_fn, ...); int MPI_Type_create_keyval(MPI_Copy_attr_function *copy_fn, ...); ------ Notice how they all use that one typedef. The only type checking that we lose is in the first parameter of the user's callback function. This is very minor. The same reasoning can be applied to all the user callback functions for attribute caching and error hander functions. Here's a synopsis of exactly what I am asking for: ----- PART 1: Eliminate all the object-dependant typedefs: MPI_*_copy_attr_function * = { Comm, Win, Type } MPI_*_delete_attr_function MPI_*_handler_function (<-- was there one for for File as well?) They would be replaced with: MPI_Copy_attr_function MPI_Delete_attr_function MPI_Errorhandler_function (can't use MPI_Handler_function -- it's used in MPI-1) ----- PART 2: Change the types of parameters in functions to match these new typedefs. The following functions are affected: MPI_*_Create_keyval * = { Comm, Win, Type } MPI_*_Create_errhandler (<-- File as well? I don't remember) --- This allows the C++ implementation to lump together all the attribute, error handler, and name functions and only implement them once (rather than 3 times). *** THE BOTTOM LINE: I want one callback typedef (of each type -- copy, delete, etc.), that takes a (void *) as its first argument, not an MPI opaque handle. I want this functionality because this allows for an elegant implmentation in C++ (specifically, we can use inheritance -- I can supply the details if anyone wants to know), and the cost is very small (i.e. the change is minor). It can also help in C implementations. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Wed May 7 13:35:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA20745; Wed, 7 May 1997 13:35:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA12651 for mpi-core-out; Wed, 7 May 1997 12:39:48 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA12644; Wed, 7 May 1997 12:39:42 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id MAA23191; Wed, 7 May 1997 12:39:40 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id MAA21555; Wed, 7 May 1997 12:39:41 -0500 (EST) Date: Wed, 7 May 1997 12:39:41 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: Steve Huss-Lederman cc: mpi-external@mcs.anl.gov, mpi-core@mcs.anl.gov, Andrew Lumsdaine Subject: Re: MPI_STATUS_IGNORE implications In-Reply-To: <199705071417.JAA22599@rap.cs.wisc.edu> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Wed, 7 May 1997, Steve Huss-Lederman wrote: > And adding text at p. 3/41: > > \begin{users} > If the user provided \const{MPI\_STATUS\_IGNORE} or > \const{MPI\_STATUSES\_IGNORE} to the \MPI/ function that causes the > \mpifunc{query\_fn} to be called, then \MPI/ will pass > \const{MPI\_STATUS\_IGNORE} to the \mpifunc{query\_fn}. In this case, > only the error code is returned (return value in C and C++ and in > \mpiarg{IERROR} in Fortran). Implementors of a \mpifunc{query\_fn} > should be careful to test for the case where \mpiarg{status} is > \const{MPI\_STATUS\_IGNORE} since accessing it as a normal status > handle is erroneous. > \end{users} Err... we have a problem with C++. There is no C++ MPI::STATUS[ES]_IGNORE. So passing this in to the Grequest::Query_function (the user callback) will be problematic. The way that it is handled in C++ is to have a second [overloaded] function that simply does not contain the MPI::Status argument. A possible solutions for this case: Have two typedefs for the query_fn, one with the MPI::Status, and one without. The one that the user passes in will be used appropriately (the implementation will have to save a flag about which to use) {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Wed May 7 14:39:23 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA21922; Wed, 7 May 1997 14:39:21 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA14332 for mpi-core-out; Wed, 7 May 1997 13:44:32 -0500 Received: from win153.nas.nasa.gov (win153.nas.nasa.gov [129.99.33.68]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA14322 for ; Wed, 7 May 1997 13:44:25 -0500 Received: from win153.nas.nasa.gov (nitzberg@localhost) by win153.nas.nasa.gov (8.7.6/NAS.6.1) with ESMTP id LAA15660 for ; Wed, 7 May 1997 11:44:25 -0700 (PDT) Message-Id: <199705071844.LAA15660@win153.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: I/O Datarep Clarification: "re-entrant (thread-safe)" Date: Wed, 07 May 1997 11:44:25 -0700 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In I/O section 10.5.3 User Defined Data Representations, my notes say that the following requirement of the user-defined data conversion callbacks should be clarified. The text said: "The filter functions must be re-entrant (thread-safe)." Can anyone recall the problem (and, hopefully, the proposed solution)? Thanks, - bill From mpi-core-human@mcs.anl.gov Wed May 7 16:58:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA24391; Wed, 7 May 1997 16:58:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA20491 for mpi-core-out; Wed, 7 May 1997 16:02:16 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id PAA18903; Wed, 7 May 1997 15:35:44 -0500 Received: from timbuk-fddi.cray.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA20056; Wed, 7 May 1997 16:33:25 -0400 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.2) with ESMTP id PAA09266; Wed, 7 May 1997 15:35:30 -0500 (CDT) Received: from mrjones.cray.com (fsgi026 [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-serv-news-1.2) with ESMTP id PAA25198; Wed, 7 May 1997 15:35:28 -0500 (CDT) From: Eric Salo Received: by mrjones.cray.com (8.8.0/btd-b3) id UAA01145; Wed, 7 May 1997 20:35:28 GMT Message-Id: <199705072035.UAA01145@mrjones.cray.com> Date: Wed, 7 May 1997 20:35:28 GMT To: @minnie.nic.kingston.ibm.com:mpi-external@mcs.anl.gov, @minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov, treumann@minnie.nic.kingston.ibm.com (Dick Treumann) Subject: Re: New errhandler text Cc: treumann@windsurf.pok.ibm.com Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Does not Jeff's proposal imply that an MPI object handle is "really" > a pointer to the object. Nnnnnnnnno, I don't think so... Jeff wants to replace a (MPI_Comm *) with a (void *). From the point of view of the application, we're just passing the address of a communicator handle. That opaque handle can be an int, a pointer, or whatever; we're just pointing to whatever it is. I think you're short one level of indirection, or else I'm very confused! -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Thu May 8 14:39:30 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA12944; Thu, 8 May 1997 14:39:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA13910 for mpi-core-out; Thu, 8 May 1997 13:41:28 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA13894 for ; Thu, 8 May 1997 13:41:19 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id NAA00634; Thu, 8 May 1997 13:41:22 -0500 (CDT) Date: Thu, 8 May 1997 13:41:22 -0500 (CDT) Message-Id: <199705081841.NAA00634@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: error codes and MPI_STATUS_IGNORE Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The attached text was just put into the external chapter. I think it outlines the current situation with MPI_STATUS_IGNORE. We need to decide which of the two choice we are going to use. Comments are welcome (and encouraged!) Steve ---------------------------------------------------------------------- The new text allows for the query_fn to be called when MPI_STATUS_IGNORE is supplied (choice 1 below). The alternative is not to call the query_fn. To be consistent, this decision has ramifications for wait/test/get_status for regular requests. I do not think we really decided this issue. Here is an outline of what I view as the two consistent choices. 1) A call with MPI_STATUS_IGNORE both checks for completion of the request and returns the error code. For a generalized request this means that the query_fn is invoked and supplied MPI_STATUS_IGNORE. Issues: If you are in a wait/test call for all or any then the error code goes in the status. Since you don't have a status we decided you still return MPI_ERR_IN_STATUS which tells the user that there was an error but the error code is lost. A second issue is that C++ does not have the value MPI_STATUS_IGNORE. We are working on possible solutions for how to define the callbacks for C++. A third issue is you call the query_fn so you won't make this case very fast if you have a generalized request and the callback always needs to test for this special case. 2) A call with MPI_STATUS_IGNORE checks for completion of the request and nothing else. No error code is returned. Since no error code is needed you don't call the query_fn. If you want the error code you must supply a status and get it filled in. Issues: Passing MPI_STATUS_IGNORE means that errors for this request are likely to be lost. This would presumably happen even if abort is the current error handler. It basically implies if you want to have a code that gets errors back then you must supply a status so good code can never use this optimization. An alternative is to allow an implementation to return an error if they want. In the case of generalized requests this will not happen since we would say the query_fn is never called. This choice has the negative attributes that the result is fuzzy and depends on the type of request. From mpi-core-human@mcs.anl.gov Thu May 8 15:28:26 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA13504; Thu, 8 May 1997 15:28:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA16072 for mpi-core-out; Thu, 8 May 1997 14:33:27 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA16065 for ; Thu, 8 May 1997 14:33:23 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA19356; Thu, 8 May 1997 15:30:33 -0400 Message-Id: <33722A5D.41C6@kgn.ibm.com> Date: Thu, 08 May 1997 15:32:45 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: Steve Huss-Lederman Cc: mpi-core@mcs.anl.gov Subject: Re: error codes and MPI_STATUS_IGNORE References: <199705081841.NAA00634@rap.cs.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve Huss-Lederman wrote: > > 1) A call with MPI_STATUS_IGNORE both checks for completion of the > request and returns the error code. For a generalized request this > means that the query_fn is invoked and supplied MPI_STATUS_IGNORE. I think this approach is broken in Fortran. In Fortran, MPI_STATUS_IGNORE will be one of those funny critters like MPI_BOTTOM. This means that there is no way for Fortran code to tell that it has been passed MPI_STATUS_IGNORE rather than a real status. SUBROUTINE QUERY(STATE,STATUS,IERROR) if (STATUS .ne. MPI_STATUS_IGNORE) Will not work. Only the code inside MPI which is written, probably in C, with the knowledge of how the implementation finagled the support for MPI_STATUS_IGNORE in Fortran can recognize that a parameter it has been passed is really the Fortran MPI_STATUS_IGNORE. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Thu May 8 16:15:42 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA14213; Thu, 8 May 1997 16:15:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA17909 for mpi-core-out; Thu, 8 May 1997 15:20:29 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA17899 for ; Thu, 8 May 1997 15:20:17 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id PAA02080; Thu, 8 May 1997 15:20:21 -0500 (CDT) Date: Thu, 8 May 1997 15:20:21 -0500 (CDT) Message-Id: <199705082020.PAA02080@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: documents ready Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The last version of the MPI-2 document for substantive checking is now available at http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. The JOD is available at the same location as the file mpi2-jod.ps.Z. We are asking everyone to look this version over and make comments by Friday, May 23. This is everyones last chance to make real comments about the document. After the input the chapter authors will work on making last minute changes. When the document is presented again on June 10 there will be a few days to point out formatting errors made in the final edits but not to make other changes. Have fun. Steve P.S. - The Appendix of constants still needs some work. I plan to split off the C++ constants. I also hope to add a section with new types. We never carefully checked the appendix so I welcome comments on it. From mpi-core-human@mcs.anl.gov Thu May 8 18:18:13 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA16070; Thu, 8 May 1997 18:18:12 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA22708 for mpi-core-out; Thu, 8 May 1997 17:23:02 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA22703 for ; Thu, 8 May 1997 17:22:57 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id SAA08684; Thu, 8 May 1997 18:23:26 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id SAA25707; Thu, 8 May 1997 18:23:04 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256491.007AF017 ; Thu, 8 May 1997 18:22:48 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: lederman@cs.wisc.edu cc: Mpi-Core@mcs.anl.gov Message-ID: <85256491.0077B9C7.00@watngi01.watson.ibm.com> Date: Thu, 8 May 1997 18:22:30 -0400 Subject: Re: error codes and MPI_STATUS_IGNORE Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My preference is for 2 -- do not call query_fn when MPI_STATUS_IGNORE is provided. For a regular request the implementation still have the option to return a significant error code to MPI_GET_STATUS or to MPI_WAIT. In any case, returning a useful error status is a quality of implementation issue. So, what we loose if we go this way is the ability for generalized request to return a meaningful error code (provided by query_fn) when the user has specified MPI_STATUS_IGNORE in a call to MPI_WAIT or MPI_GET_STATUS. What we gain is that query_fn is not invoked when the user did not ask for a status does not have to be coded so as to deal with an MPI_STATUS_IGNORE input argument. From mpi-core-human@mcs.anl.gov Mon May 12 12:19:41 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA00409; Mon, 12 May 1997 12:18:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA28612 for mpi-core-out; Mon, 12 May 1997 11:19:29 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA28601; Mon, 12 May 1997 11:19:21 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id SAA19892 (8.6.13/IDA-1.6); Mon, 12 May 1997 18:19:16 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id SAA05068; Mon, 12 May 1997 18:19:15 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199705121619.SAA05068@awsrr.rus.uni-stuttgart.de> Subject: errors in draft-8-may To: gropp@mcs.anl.gov (William Gropp), lederman@cs.wisc.edu (Steve Huss-Lederman), lusk@mcs.anl.gov (Rusty Lusk), nitzberg@nas.nasa.gov (Bill Nitzberg), wcs@nas.nasa.gov (William C. Saphir), SNIR@watson.ibm.com (Marc Snir) Date: Mon, 12 May 1997 18:19:15 +0200 (DST) Cc: mpi-core@mcs.anl.gov Content-Type: text Content-Length: 7134 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear authors, I have found the following typos, small inconsistencies, errors, ... in MPI-2 and the JOD. The only major problem I found: page 93 (1-SIDED) and page 272 (I/O) have chosen an incosistent way to handle MPI_???_FREE_ERRHANDLER(errhandler) Because of the argument of layerability of I/O on top of rest-of-MPI-2 I think the way of I/O should be chosen for 1-SIDED, i.e. 1-SIDED should define MPI_WIN_FREE_ERRHANDLER(errhandler). The alternative is to remove MPI_FILE_FREE_ERRHANDLER on page 272 and to add in __both__ sections a sentence, that MPI_FREE_ERRHANDLER must be called to mark an error handler for deallocation. It would be nice to get a short response whether you have done or rejected some of my corrections. Rolf ------------- detailed list -------------------- INTRO 3:6 MPI\_PACK\_EXTERNAL instead of MPI\_PACK 3:7 MPI\_UNPACK\_EXTERNAL instead of MPI\_UNPACK MPI 1.2 19:6 completion). instead of completion. 24:21-22 A Problem with Register Optimization instead of A Fortran Problem with register optimization ------- - - 24:22 pages~\pageref{sec:misc-sequence} and \pageref{sec:misc-register}. instead of page~\pageref{sec:misc-register}. 25:21-25 Same text as the now corrected text 24:19-24 26:1-4 Same text as the now corrected text 24:19-24 DYNAMIC 51:43 port_name[], const MPI::Info& info, instead of port_name[], 52:3 that it did not publish or had already unpublished. instead of that it did not publish. 52:25 MPI_LOOKUP_NAME and MPI_UNPUBLISH_NAME instead of MPI_LOOKUP_NAME 52:26 how to retrieve or where to unpublish names. instead of how to retrieve names. 1-SIDED 65:37-45, 66:4,11 the name 'base' should be modified to 94:10 reflect the 'one less dereference' comparing 'base' of MPI_ALLOC_MEM 66:27 MPI_ALLOC_MEM instead of MPI_MALLOC_MEM = 89:21 MPI\_MODE\_NOPUT instead of MPI\_NOPUT 89:23 MPI\_MODE\_NOPRECEDE instead of MPI\_NOPRECEDE 89:26 ...SUCCEED instead of ...SUCCED - 90:4 MPI_MODE_NOPRECEDE instead of MPI_NOPRECEDE 90:8 MPI_MODE_NOSTORE instead of MPI_NOSTORE and MPI_MODE_NOSUCCEED instead of MPI_NOSUCCEED 90:22 MPI_MODE_NOPUT instead of MPI_NOPUT and MPI_MODE_NOPRECEDE instead of MPI_NOPRECEDE 90:28 MPI_MODE_NOSUCCEED instead of MPI_NOSUCCEED 91:5 MPI_MODE_NOPUT instead of MPI_NOPUT 91:22 MPI_MODE_NOCHECK instead of MPI_NOCHECK and MPI_MODE_NOPUT instead of MPI_NOPUT 91:29 MPI_MODE_NOCHECK instead of MPI_NOCHECK 91:35 MPI_MODE_NOCHECK instead of MPI_NOCHECK and MPI_MODE_NOPUT instead of MPI_NOPUT 91:41 MPI_MODE_NOCHECK instead of MPI_NOCHECK 91:48 MPI_MODE_NOCHECK instead of MPI_NOCHECK and MPI_MODE_NOPUT instead of MPI_NOPUT 93:6 MPI_WIN_FREE_ERRHANDLER(errhandler) should be defined (see details above). 94:10 must reflect the changes on page 65, i.e. an additional class: "MPI_ERR_MEM invalid mem argument" or modified MPI_ERR_BASE: "MPI_ERR_BASE invalid base or "mem" argument" 100:25-27 For additional Fortran problems see also "..." on page 175 instead of See also, "..." on page 175 additional Fortran problems. COLLECT. 122:30 \subsection{Other Operations} instead of \subsubsection{Other operations} --- - EXTERNAL 129:21 completes instead of completed (or is "completed" here a really correct english?) 152:34 delete the second and wrong C++ definition F90 180:36 ...REAL and MPI_TYPE_CREATE_F90_COMPLEX instead of ...REAL MISC 195:24 MPI_OFFSET_KIND instead of MPIOFFSETKIND 202:17,23,29 \subsection instead of \subsubsection ?! 212:7-15 should be moved directly bevor section 9.12 because there is a relation between these two sections. 213:47 9.10). instead of 9.10. - I/O 232:9-11 The rationale should move after line 20 because it is related to "reading the end of file". (It was my error, I switched the line numbers on the slide.) 232:21-24 The Advice should move after line 7. The content should be corrected into the following text (same as corrected text page 24, see above): \begin{users} To prevent problems with the argument copying and register optimization done by Fortran compilers, please note the hints in subsections ``Problems due to data copying and sequence association'', and ``A Problem with Register Optimization'' in Section~\ref{sec:misc-register} of this document, pages~\pageref{sec:misc-sequence} and \pageref{sec:misc-register}. \end{users} 272:6-21 see all text above related to page 93 ANNEX-A 286:24-28 MPI:: should be removed (cause: page 88:38-39) 288:6-17 do we have these constants? 286-289: (it would be nice to have page numbers for referencing the definition of the constants) JOD-MISC 29:12 please add some heading like \section{Continuable Errors} JOD-COLL 34-38: please modify start --> begin Start --> Begin START --> BEGIN 35:16 caution: stat --> begin 35:28 caution: started --> begun 35:25 MPI-2, section 8.2.2 instead of section 2.5.2 ------- ----- 38:33,35: MPI_REAL instead of REALx Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Mon May 12 12:58:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA00916; Mon, 12 May 1997 12:58:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA29942 for mpi-core-out; Mon, 12 May 1997 12:01:58 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA29937 for ; Mon, 12 May 1997 12:01:55 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id MAA20384; Mon, 12 May 1997 12:01:52 -0500 (CDT) Date: Mon, 12 May 1997 12:01:52 -0500 (CDT) Message-Id: <199705121701.MAA20384@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov In-reply-to: <199705121619.SAA05068@awsrr.rus.uni-stuttgart.de> (Rabenseifner@RUS.Uni-Stuttgart.DE) Subject: Re: errors in draft-8-may Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Rolf, Thanks for the comments. Here are my responses: > EXTERNAL 129:21 completes instead of completed > (or is "completed" here a really correct english?) The completes and completed are all tied up in the definitions we agreed to at the last meeting. This is why I have not changed this. I think it is ok English as it is. > 152:34 delete the second and wrong C++ definition I think this came up before and I still think it is correct. There are 2 C++ definitions since the two argument argc & argv are optional. This is being done via different function prototypes. See how MPI_Init is done on 332:35 and 332:37. > ANNEX-A 286:24-28 MPI:: should be removed (cause: page 88:38-39) Actually I think this is a bigger issue. As I said in my announcement, the C++ constants are not properly listed. Since every MPI-1 constant now has a C++ version we need to do the same for MPI-2. Thus, these few represent the C++ for the others that should be coming. > 288:6-17 do we have these constants? I know some are definitely used. I will check them over. > 286-289: (it would be nice to have page numbers for > referencing the definition of the constants) It would be nice to have a much better index in general. This would not be trivial to produce and I do not plan to do it now. Steve From mpi-core-human@mcs.anl.gov Mon May 12 20:09:42 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA05322; Mon, 12 May 1997 20:09:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA11307 for mpi-core-out; Mon, 12 May 1997 19:14:30 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA11301 for ; Mon, 12 May 1997 19:14:27 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id TAA05590; Mon, 12 May 1997 19:14:24 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id TAA18000; Mon, 12 May 1997 19:14:23 -0500 (EST) Date: Mon, 12 May 1997 19:14:22 -0500 (EST) From: Jeff Squyres To: Steve Huss-Lederman cc: mpi-core@mcs.anl.gov Subject: Re: errors in draft-8-may In-Reply-To: <199705121701.MAA20384@rap.cs.wisc.edu> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Mon, 12 May 1997, Steve Huss-Lederman wrote: > > 152:34 delete the second and wrong C++ definition > > I think this came up before and I still think it is correct. There > are 2 C++ definitions since the two argument argc & argv are > optional. This is being done via different function prototypes. See > how MPI_Init is done on 332:35 and 332:37. Yes, Steve is correct. There is an Advice to Users that describes the use of the second C++ bindings. > > ANNEX-A 286:24-28 MPI:: should be removed (cause: page 88:38-39) > > Actually I think this is a bigger issue. As I said in my > announcement, the C++ constants are not properly listed. Since every > MPI-1 constant now has a C++ version we need to do the same for > MPI-2. Thus, these few represent the C++ for the others that should > be coming. I think that I would agree -- a separate list would be good, especially since the constants might not all be the same. Right now, I think that there are only two that are not -- there is no MPI::STATUS_IGNORE nor MPI::STATUSES_IGNORE. Thanks for checking so closely, Rolf! {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Tue May 13 10:27:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA16086; Tue, 13 May 1997 10:27:01 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA21135 for mpi-core-out; Tue, 13 May 1997 09:30:07 -0500 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA20679; Tue, 13 May 1997 09:08:09 -0500 Message-Id: <199705131408.JAA20679@antares.mcs.anl.gov> Received: from ry73.rz.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Tue, 13 May 1997 15:17:17 +0200 Received: by ry73.rz.uni-karlsruhe.de (1.38.193.4/16.2) id AA07325; Tue, 13 May 1997 15:17:16 +0200 Subject: Comment on new INTENT paragraph To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Date: Tue, 13 May 1997 15:17:15 +0200 (CES) From: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) X-Mailer: ELM [version 2.4ME+ PL31 (25)] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 2407 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk A comment on the MPI-2 Draft (April 17, 1997): Section 2.3, page 5 contains a new (Apr) paragraph on INTENT: "MPI's use of OUT, IN and INOUT is intended to indicate to the user how an argument is to be used, but does not provide a rigorous classification that can be translated directly into language bindings (e.g. INTENT in Fortran 90 bindings or const in C bindings). ..." I would strongly recommend that this paragraph be removed from the document. It does nothing than confusing the user, and makes *any* Fortran90 binding which wants to specify INTENT (for performance and/or safety) completely impossible. If there are problems with the proper specification of some arguments as being OUT or INOUT, why not simply specify them as INOUT??? Which are these controversial procedures/arguments? I also hope that there is no procedure where a specified intent IN may in some cases be interpreted as INOUT or OUT: this would imply that C arguments which are specified in the C language binding to be passed by value then had to be passed by reference, which sounds strange... Thanks, Michael PS: A "const" qualification does not change the representation of the C type, and does nothing to the way the C argument is passed. Especially it does not change the definition status of the argument. On the other hand, a Fortran INTENT attribute allows the compiler to avoid copy[in|out] if the intent is specified as [OUT|IN]. It also influences the definition status of the argument in that an INTENT(OUT) dummy argument becomes undefined on entry, so testing its value (MPI_BOTTOM, MPI_STATUS_IGNORE, or whatever) is *illegal*. It is therefore very important that MPI gets its argument intents right if there should be any future for a proper Fortran 90/95 binding. ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) No longer on BITNET :-( Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Tue May 13 12:01:06 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA18109; Tue, 13 May 1997 12:00:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA24096 for mpi-core-out; Tue, 13 May 1997 11:04:39 -0500 Received: from borris.khoros.unm.edu (borris.khoros.unm.edu [198.59.155.2]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA23607 for ; Tue, 13 May 1997 10:50:20 -0500 Received: by borris.khoros.unm.edu (8.7.5/KHOROS/Jan 16 1996) id <199705131550.JAA13738@borris.khoros.unm.edu>; Tue, 13 May 1997 09:50:08 -0600 (MDT) From: trobey@borris.khoros.unm.edu (Tom Robey) Posted-Date: Tue, 13 May 1997 09:50:08 -0600 (MDT) Message-Id: <199705131550.JAA13738@borris.khoros.unm.edu> Subject: Re: Comment on new INTENT paragraph To: hennecke@rz.uni-karlsruhe.de Date: Tue, 13 May 1997 09:50:08 -0600 (MDT) Cc: mpi-core@mcs.anl.gov In-Reply-To: <199705131408.JAA20679@antares.mcs.anl.gov> from "Michael Hennecke" at May 13, 97 03:17:15 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Michael Hennecke wrote > > A comment on the MPI-2 Draft (April 17, 1997): > > Section 2.3, page 5 contains a new (Apr) paragraph on INTENT: > > "MPI's use of OUT, IN and INOUT is intended to indicate to the user how > an argument is to be used, but does not provide a rigorous classification > that can be translated directly into language bindings (e.g. INTENT in > Fortran 90 bindings or const in C bindings). ..." > > I would strongly recommend that this paragraph be removed from the document. > It does nothing than confusing the user, and makes *any* Fortran90 binding > which wants to specify INTENT (for performance and/or safety) completely > impossible. > This was thoroughly discussed at the last meeting and this paragraph is consistent with that discussion. The problem is that something in an opaque object may change even though nothing accessible by the user changes. Thus a rigorous classification may not be possible. If a rigorous classification is used then every argument would be INOUT which would defeat the purpose of INTENT. I would suggest adding that the rigor of the classification is implementation dependent since most implementations probably do not want rigorous classification, but some may want to do this if they want to address FORTRAN 90 issues. -- Thomas H. Robey trobey@khoral.com Khoral Research, Inc. URL http://www.khoral.com/ 6001 Indian School Road N.E. phone 505 837-6500 x487 Suite 200 fax 505 881-3842 |/ ) | Albuquerque NM, 87110 |\ \ | From mpi-core-human@mcs.anl.gov Tue May 13 12:35:44 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA19461; Tue, 13 May 1997 12:35:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA25110 for mpi-core-out; Tue, 13 May 1997 11:38:24 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA25096; Tue, 13 May 1997 11:38:09 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA29391; Tue, 13 May 1997 12:35:28 -0400 Message-Id: <337898DB.41C6@kgn.ibm.com> Date: Tue, 13 May 1997 12:37:48 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: Michael Hennecke Cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: Comment on new INTENT paragraph References: <199705131408.JAA20679@antares.mcs.anl.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Michael Hennecke wrote: > If there are problems with the proper specification of some arguments as > being OUT or INOUT, why not simply specify them as INOUT??? Which are these > controversial procedures/arguments? One area where problems exist relate to the fact that many MPI routines are passed handles that represent opaque objects. To the user thinking about his code it is often the state of the object which is of primary interest. That is, is the object created or modified by the call. All the compiler cares about (and all INTENT in Fortran refers to in the case of handles and objects) is the handle. If, from the start of MPI 1 it had been decided that the IN, OUT, INOUT refered only to the actual parameter then more of the labels would have been correct for C const and Fortran INTENT. For better or worse, the decision was made that in some cases IN/OUT/INOUT would reflect the impact on the object. A second area is arguments to calls like MPI_Start or MPI_Type_commit where some MPI implementations may actually modify the handle while others will not. I suspect that most deviations do involve labeling arguments as INOUT when the language binding can be declared INTENT IN or const. I think a quality implementation could use the optimal qualifier but the author of portable MPI code must sometimes assume the more general case. Does anyone argue that the MPI Standard saying the argument to MPI_Start is an INOUT rules out an MPI implementation choosing to provide the function prototype: int MPI_Start(const MPI_Request *handle) if that implementation does not change the handle when an MPI_Start is done? Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue May 13 13:09:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA20084; Tue, 13 May 1997 13:09:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA25909 for mpi-core-out; Tue, 13 May 1997 12:13:28 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA25862; Tue, 13 May 1997 12:11:56 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id MAA08747; Tue, 13 May 1997 12:11:53 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id MAA25141; Tue, 13 May 1997 12:11:53 -0500 (EST) Date: Tue, 13 May 1997 12:11:52 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: Re: Comment on new INTENT paragraph In-Reply-To: <199705131550.JAA13738@borris.khoros.unm.edu> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Tue, 13 May 1997, Tom Robey wrote: > > I would strongly recommend that this paragraph be removed from the document. > > It does nothing than confusing the user, and makes *any* Fortran90 binding > > which wants to specify INTENT (for performance and/or safety) completely > > impossible. > > This was thoroughly discussed at the last meeting and this paragraph is > consistent with that discussion. The problem is that something in an > opaque object may change even though nothing accessible by the user > changes. Thus a rigorous classification may not be possible. If a > rigorous classification is used then every argument would be INOUT which > would defeat the purpose of INTENT. I would suggest adding that the > rigor of the classification is implementation dependent since most > implementations probably do not want rigorous classification, but some > may want to do this if they want to address FORTRAN 90 issues. I actually missed the discussion about this, but here's my $0.02: Can't we get a compromise by removing the clause, p5 32-36: "There is one special case -- if an argument is a handle to an opaque object (these terms are define din Section 2.5.1), and the object is updated by the procedure call, then the argument is marked OUT. It is marked this way even though the handle itself is not modified -- we use the OUT attribute to denote that what the randle references is updated." It would seem that this is the clause that is the source of the controversey, since it dictates that the internal state of the MPI objects is (or is not) allowed to change. But at the last meeting, it was agreed that it should only affect "what the user perceives as changing.". So doesn't that effectively mean just the handle? So the *handle* could be strictly IN, OUT, or INOUT, but not necessarily what the handle points to in the implementation (which will probably be different in all implementations anyway, so it may be pointless to try to regulate this anyway). Marking the INTENT of the handle should be easy, even in C and C++ (pass by value when IN, reference when OUT or INOUT). This seems to be consistent with what was decided at the last meeting. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Tue May 13 14:23:20 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA21275; Tue, 13 May 1997 14:23:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA27818 for mpi-core-out; Tue, 13 May 1997 13:26:54 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA26785; Tue, 13 May 1997 12:51:51 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id MAA22106; Tue, 13 May 1997 12:51:42 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id MAA20337; Tue, 13 May 1997 12:51:40 -0500 (CDT) From: Eric Salo Received: by mrjones.cray.com (8.8.0/btd-b3) id RAA04308; Tue, 13 May 1997 17:51:39 GMT Message-Id: <199705131751.RAA04308@mrjones.cray.com> Date: Tue, 13 May 1997 17:51:39 GMT To: Michael Hennecke , Dick Treumann Subject: Re: Comment on new INTENT paragraph Cc: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov References: <199705131408.JAA20679@antares.mcs.anl.gov> Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Does anyone argue that the MPI Standard saying the argument to MPI_Start > is an INOUT rules out an MPI implementation choosing to provide the > function prototype: I think that in general, implementations can assign const arguments wherever they please because it does not impact the applications at all. Certain arguments would of course be very silly, such as the buffer passed to MPI_RECV... -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Tue May 13 14:42:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA21627; Tue, 13 May 1997 14:42:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA28455 for mpi-core-out; Tue, 13 May 1997 13:46:58 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id NAA28450; Tue, 13 May 1997 13:46:53 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA21469; Tue, 13 May 1997 14:44:04 -0400 Message-Id: <3378B6FF.2781@kgn.ibm.com> Date: Tue, 13 May 1997 14:46:23 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: Eric Salo Cc: Michael Hennecke , Dick Treumann , mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: Re: Comment on new INTENT paragraph References: <199705131408.JAA20679@antares.mcs.anl.gov> <199705131751.RAA04308@mrjones.cray.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Eric Salo wrote: > > I think that in general, implementations can assign const arguments > wherever they please because it does not impact the applications at all. > Certain arguments would of course be very silly, such as the buffer > passed to MPI_RECV... > -- > Eric Salo Silicon Graphics salo@sgi.com I think your comment would apply to: int MPI_Recv(void const *bufptr, ..... which says that bufptr is a constant pointer to a void. Since the pointer is passed by value anyway, the fact that it is const is not news to the compiler. int MPI_Recv(const void *bufptr,... says bufptr is a pointer to a const void. This is clearly incorrect and any MPI which put this in their function prototype would get lots of bug reports. In the case MPI_Start(const MPI_Request *handle) the standard allows an implementation to modify the handle if it wishes but many will not. This means that the handle is passed by reference to make the overlay possible but a prototype using const attribute to indicate a "pointer to a const MPI_Request" as I indicate is promising the compiler it does not change the handle even though it has the information it needs to do so. Perhaps there would be a value to providing: int MPI_Send(const void *bufptr,..... if you have a smart enough C compiler to exploit the extra information. Dick I hope I have the const subtleties right here! -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue May 13 15:09:57 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA22443; Tue, 13 May 1997 15:09:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA29249 for mpi-core-out; Tue, 13 May 1997 14:14:06 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA29238; Tue, 13 May 1997 14:13:59 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA21328; Tue, 13 May 1997 15:11:30 -0400 Message-Id: <3378BD6E.446B@kgn.ibm.com> Date: Tue, 13 May 1997 15:13:50 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: Jeff Squyres Cc: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: Re: Comment on new INTENT paragraph References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Jeff Squyres wrote: > Can't we get a compromise by removing the clause, p5 32-36: "There is one > special case -- if an argument is a handle to an opaque object (these > terms are define din Section 2.5.1), and the object is updated by the > procedure call, then the argument is marked OUT. It is marked this way > even though the handle itself is not modified -- we use the OUT attribute > to denote that what the randle references is updated." It is not the statement above which is the problem. It is the fact that in MPI 1.1 at least, this statement guided those who put IN/OUT/INOUT labels on arguments. I have a good idea how to decide when a function's parameter is const and when it is not. I do not really understand how to judge "what the user perceives as changing". Did that phrase go into the document somewhere? Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue May 13 15:27:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA22682; Tue, 13 May 1997 15:26:58 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA29640 for mpi-core-out; Tue, 13 May 1997 14:31:05 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA29608; Tue, 13 May 1997 14:30:25 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id OAA09111; Tue, 13 May 1997 14:30:19 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id OAA26400; Tue, 13 May 1997 14:30:17 -0500 (EST) Date: Tue, 13 May 1997 14:30:17 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: Dick Treumann cc: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: Re: Comment on new INTENT paragraph In-Reply-To: <3378BD6E.446B@kgn.ibm.com> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Tue, 13 May 1997, Dick Treumann wrote: > > Can't we get a compromise by removing the clause, p5 32-36: "There is one > > special case -- if an argument is a handle to an opaque object (these > > terms are define din Section 2.5.1), and the object is updated by the > > procedure call, then the argument is marked OUT. It is marked this way > > even though the handle itself is not modified -- we use the OUT attribute > > to denote that what the randle references is updated." > > It is not the statement above which is the problem. It is the fact that > in MPI 1.1 at least, this statement guided those who put IN/OUT/INOUT > labels on arguments. I have a good idea how to decide when a function's > parameter is const and when it is not. I do not really understand how I think that it may have been the misunderstanding of this statement that led to the bad placement of IN/OUT/INOUT labels in MPI-1. For example, the statement says that the Comm argument on an MPI_Send should probably be INOUT, because it is certainly conceivable that an implementation would need to change something internal to the MPI_Comm object when posting a send, although it doesn't need to change the handle itself. > to judge "what the user perceives as changing". Did that phrase go into > the document somewhere? I don't think so, but it was certainly the context that it was discussed in (I remember hearing that phrase several times). {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Tue May 13 16:10:17 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA23588; Tue, 13 May 1997 16:10:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA29751 for mpi-core-out; Tue, 13 May 1997 14:34:56 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA29738; Tue, 13 May 1997 14:34:40 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id OAA09115; Tue, 13 May 1997 14:34:31 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id OAA26405; Tue, 13 May 1997 14:34:30 -0500 (EST) Date: Tue, 13 May 1997 14:34:30 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: Re: Comment on new INTENT paragraph In-Reply-To: <3378B6FF.2781@kgn.ibm.com> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Tue, 13 May 1997, Dick Treumann wrote: > Perhaps there would be a value to providing: > int MPI_Send(const void *bufptr,..... > if you have a smart enough C compiler to exploit the extra information. Unfortunately not, since you have to cast away the (void *) anyway (you can't use a (void *) to do anything -- it has to be cast to something else before you can use it). So if you passed in (const void *), you'd have to cast that away as well, so the (const) information is lost. The real use for const is when passing in handles that will not change, such as for IN arguments that are passed by reference, ala C++. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Wed May 14 04:36:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA15502; Wed, 14 May 1997 04:36:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA11945 for mpi-core-out; Wed, 14 May 1997 03:41:42 -0500 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA11929; Wed, 14 May 1997 03:41:24 -0500 Message-Id: <199705140841.DAA11929@antares.mcs.anl.gov> Received: from ry73.rz.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Wed, 14 May 1997 10:41:05 +0200 Received: by ry73.rz.uni-karlsruhe.de (1.38.193.4/16.2) id AA11266; Wed, 14 May 1997 10:41:04 +0200 Subject: INTENT take two: handles and targets In-Reply-To: from Jeff Squyres at "May 13, 97 02:30:17 pm" To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Date: Wed, 14 May 1997 10:41:04 +0200 (CES) From: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) X-Mailer: ELM [version 2.4ME+ PL31 (25)] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 2339 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk According to Jeff Squyres: > Can't we get a compromise by removing the clause, p5 32-36: "There is one > special case -- if an argument is a handle to an opaque object (these > terms are define din Section 2.5.1), and the object is updated by the > procedure call, then the argument is marked OUT. It is marked this way > even though the handle itself is not modified -- we use the OUT attribute > to denote that what the randle references is updated." This is one paragraph whose removal would clarify the binding a lot: As mentioned by others, the intent of the argument, from the compiler's view, is that of the handle and not of anything it "points to" in the MPI system. "what the user perceives as changing" is exactly the argument he or she supplied, and not something behind the scenes which is inaccessible to the user. Otherwise, we could not even pass a SEND buffer as intent(in) since it will definitely change the state of something behind the scenes :-) In my opinion, the correct way to say that the handle's target changes is by a scentence in the description of either the handle argument, or the procedure as a whole. INTENT is not the right place to document this. But this handle/target stuff is not all of the discussion: another problem comes from the arguments which are documented OUT, but MPI procedures need to read the value supplied by the user. MPI_IN_PLACE is an example where on object and not a handle is passed as OUT RECVBUF but needs to be *read* by the procedure. So the recommendation in the advice to users in 6.3.2 (page 118 in the April-17 draft) to use INTENT(INOUT) should also be reflected in the procedure specifications. I guess MPI_STATUS_IGNORE is a similar case, and there may be others. Thanks, Michael ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) No longer on BITNET :-( Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Wed May 14 05:10:46 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA00564; Wed, 14 May 1997 05:10:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA12255 for mpi-core-out; Wed, 14 May 1997 04:16:13 -0500 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA12239; Wed, 14 May 1997 04:15:41 -0500 Received: from jim (gw2-173.pool.dircon.co.uk [194.112.35.173]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id KAA04737; Wed, 14 May 1997 10:15:16 +0100 (BST) Message-Id: <199705140915.KAA04737@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id KAA00629; Wed, 14 May 1997 10:13:16 +0100 To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: Re: Comment on new INTENT paragraph Date: Wed, 14 May 1997 10:13:16 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk For C or C++, constness is a tidiness issue. Getting it right allows the user to try to make her program const correct, but it doesn't constrain the implementation at all, because it can always cast away the constness and do what it likes. Indeed Stroustroup himself recommends using constness on methods in C++ to signify "perceptual" rather than bit-level constness, thus requiring casting away the const. However the situation is very different in Fortran, since here INTENT IN arguments could be passed by a completely different mechanism than INTENT OUT or INTENT INOUT, and compilers may be clever enough to realise that they need not invalidate (register) copies of actual arguments passed to INTENT IN formals. I don't actually think that there's a big problem, though, since most of the places where there's ambiguity appear to be where MPI specifies something as being INOUT, but in fact what is being expressed is that something referred to by the actual argument may be changed, so the argument itself could be classed as the stricter IN. This doesn't seem like a problem to me, the problem would be if we had the opposite requirement. Dick wrote > I hope I have the const subtleties right here! Nearly. The concepts are right, the syntax you used doesn't mean what you thought. "const void * foo;" and "void const * foo;" both declare a changeable pointer to a non-changeable object of type void. "void * const foo;" declares a non changeable pointer to a changeable object. "void const * const foo;" and "const void * const foo;" declare non-changeable pointers to non-changeable objects. e.g. void foo (int * p) { int const * q = p; int * const qq = p; const int * qqq = p; int const * const qqqq = p; q++; *q = 1; /* line 9: error: expression must be a modifiable lvalue */ qq++; /* line 11: error: expression must be a modifiable lvalue */ *qq = 1; qqq++; *qqq = 1; /* line 15: error: expression must be a modifiable lvalue */ qqqq++; /* line 17: error: expression must be a modifiable lvalue */ *qqqq = 1; /* line 18: error: expression must be a modifiable lvalue */ } -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Wed May 14 07:29:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA02569; Wed, 14 May 1997 07:29:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id GAA13843 for mpi-core-out; Wed, 14 May 1997 06:34:14 -0500 Received: from daedalus.epcc.ed.ac.uk (root@daedalus.epcc.ed.ac.uk [129.215.56.21]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA13838 for ; Wed, 14 May 1997 06:34:04 -0500 Received: from onyx.epcc.ed.ac.uk (lyndon@onyx.epcc.ed.ac.uk [129.215.158.4]) by daedalus.epcc.ed.ac.uk (8.6.13/8.6.12) with ESMTP id MAA24466; Wed, 14 May 1997 12:35:58 +0100 Message-Id: <5980.199705141131@onyx.epcc.ed.ac.uk> Subject: Further errors in May8 - Chapter 5 To: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Date: Wed, 14 May 1997 12:31:16 +0100 (BST) From: "Lyndon J. Clarke" Cc: mpi-core@mcs.anl.gov In-Reply-To: <199705121619.SAA05068@awsrr.rus.uni-stuttgart.de> from "Rolf Rabenseifner" at May 12, 97 06:19:15 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Group Here some more corrections. I tried to delete overlap with Rolf's list. ------------------------------------------------------------ %< cut here Chapter 5, One-Sided Communication 65:45 " base" should (?) be " BASE(*)" to be consistent with create_win. 66:41 "as specified in the init call" should (?) be "as specified in the create_win calls" 67:12 "MPI_Win_create_win" should be "MPI_Comm_create_win" if 67:1 is correct. 67:14 "MPI_WIN_CREATE" should be "MPI_COMM_CREATE_WIN" if 67:1 is correct. 67:19 "MPI::Win::Create" should be "MPI::Comm::Create_win" if 67:1 is correct. 67:31 "spawn" should be "span" (? :-) 68:16 } "MPI_WIN_INIT" then should be "MPI_COMM_CREATE_WIN" 68:20 } 73:18 } 74:42 } 76:27 } 68:18 } "MPI_MEM_ALLOC" should be "MPI_ALLOC_MEM" 71:41 } 73:37 "END DO" is missing 74:19 } assert in MPI_WIN_FENCE is missing in Fortran examples 74:24 } "_FENCE(" should be "_FENCE(0," for example 74:44 } 75:4 } 76:31 } 76:38 } 78:8 "This mechanism is more efficient" should (?) be "This mechanism may be more efficient". MPI does not mandate relative efficiency. ------------------------------------------------------------ %< cut here -- +-|epcc|----------------------------+-----+ | Lyndon J Clarke, PhD | Tel: [+ 44] (0)131 650 5021 | | EPCC, The University of Edinburgh | Fax: [+ 44] (0)131 650 6555 | +-----------------------------------+-----------------------------+ From mpi-core-human@mcs.anl.gov Wed May 14 08:11:57 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA03038; Wed, 14 May 1997 08:11:57 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA14251 for mpi-core-out; Wed, 14 May 1997 07:17:13 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA14246 for ; Wed, 14 May 1997 07:17:08 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id OAA22017 (8.6.13/IDA-1.6); Wed, 14 May 1997 14:17:04 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id OAA07307; Wed, 14 May 1997 14:17:05 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199705141217.OAA07307@awsrr.rus.uni-stuttgart.de> Subject: Re: errors in draft-8-may To: SNIR@watson.ibm.com (Marc Snir) Date: Wed, 14 May 1997 14:17:04 +0200 (DST) Cc: mpi-core@mcs.anl.gov In-Reply-To: <199705121619.SAA05068@awsrr.rus.uni-stuttgart.de> from "Rolf Rabenseifner" at May 12, 97 06:19:15 pm Content-Type: text Content-Length: 1061 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc, I want to modify my own proposal, that was: >1-SIDED 65:37-45, 66:4,11 the name 'base' should be modified to > 94:10 reflect the 'one less dereference' > comparing 'base' of MPI_ALLOC_MEM Lyndon showed me that "base" is used in the same way in MPI_MEM_FREE and MPI_WIN_CREATE. Therefore I think it is better to change the name in MPI_ALLOC_MEM e.g. into "baseptr", affected lines: 65:19,23,25,27,29 66:7 And the explanation 65:23 into "pointer to the initial ..." And 94:10 invalid base or baseptr argument Rolf PS: I'm not sure that the name MPI_COMM_CREATE_WIN was wanted. I thought MPI_WIN_CREATE is the correct one. Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Wed May 14 09:07:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA04241; Wed, 14 May 1997 09:07:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA14800 for mpi-core-out; Wed, 14 May 1997 08:11:54 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA14795; Wed, 14 May 1997 08:11:49 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA26070; Wed, 14 May 1997 09:09:23 -0400 Received: by windsurf.pok.ibm.com (AIX 4.1/UCB 5.64/4.03) id AA12356; Wed, 14 May 1997 09:11:43 -0400 Date: Wed, 14 May 1997 09:11:43 -0400 From: treumann@kgn.ibm.com (Dick Treumann) Message-Id: <9705141311.AA12356@windsurf.pok.ibm.com> To: @minnie.nic.kingston.ibm.com:mpi-bind@mcs.anl.gov, @minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov Subject: MPI_STATUS_IGNORE etc. Cc: treumann@windsurf.pok.ibm.com Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk As an MPI implementer I find the whole subject of MPI_STATUS_IGNORE, MPI_BOTTOM etc scares me. Most discussion of these magic names in Fortran 90 discuss them as if a call which can be passed MPI_STATUS_IGNORE is getting a status array which contains some value which makes it detectable as the magic name. That is what suggestions that it be labeled INOUT rather than OUT imply to me. This is simply unworkable. With the following call: CALL MPI_WAIT(REQUEST,STATUS) the assumption is that STATUS is something the user has declared and at entry it contains undefined bits. The code which carries out the MPI_WAIT has access to the address of STATUS because "no real compiler would use copy in/copy out". The MPI_WAIT code could look at the address of the argument it was passed and recognize it had either a real MPI_STATUS or had the magic argument, MPI_STATUS_IGNORE. (MPI-1 did not have MPI_STATUS_IGNORE so consider MPI_BOTTOM if you prefer. A send or receive buffer also has undefined bits at entry.) As long as the bit content of an argument which can either be the real thing or a magic name is undefined, there is no cookie that can be used as the recognizable content of the magic object. Some trick inside the implementation code is needed. The reliable trick in F77 depends on getting a recognizable address when passed MPI_BOTTOM and assuming any address that did not match was a real buffer. Once the door is open to F90's increased use of copying it may no longer safe to assume that the address of the MPI_STATUSES_IGNORE pretend array can be recognized. As far as I can see, as long as the implementation can count on getting these arguments in the form "addresses of the actual argument" rather than as addresses of copy temps it will not have any reason to view an MPI_STATUS_IGNORE as anything but an OUT. If ever it becomes impossible to count on pass by "address of the actual argument" then MPI will be "up you know what creek without a paddle" and it will not help that the MPI_STATUS_IGNORE argument is identified as an INOUT rather than an OUT. The MPI and Fortran saga will continue. Dick Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Thu May 15 08:15:46 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA11580; Thu, 15 May 1997 08:15:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA07225 for mpi-core-out; Thu, 15 May 1997 07:16:19 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA07220 for ; Thu, 15 May 1997 07:16:15 -0500 Received: (tony@localhost); by Aurora.CS.MsState.Edu (8.8.5/7.0m-FWP-MsState); id HAA04619; Thu, 15 May 1997 07:16:58 -0500 (CDT) Date: Thu, 15 May 1997 07:16:58 -0500 (CDT) From: Tony Skjellum Message-Id: <199705151216.HAA04619@Aurora.CS.MsState.Edu> To: mpi-core@mcs.anl.gov Subject: Another Gordon Conference Announcement Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The program looks very good. WE have room for more attendees. Please attend. -Tony ----- The deadline for applying to attend the 1997 Gordon conference in high performance computing is June 1. If you are interested in attending, please apply as soon as possible. The simplest way to apply is to download the application form from the web site indicated below, or to use the online registration option. If you have any problems with either of these, please contact the organizers at tony@cs.msstate.edu and worleyph@ornl.gov. ------------------------------------------------------------------------------- The 1997 Gordon Conference on High Performance Computing and Information Infrastructure: "Practical Revolutions in HPC and NII" Chair, Anthony Skjellum, Mississippi State University, tony@cs.msstate.edu, 601-325-8435 Co-Chair, Pat Worley, Oak Ridge National Laboratory, worleyph@ornl.gov, 615-574-3128 Conference web page: http://www.erc.msstate.edu/conferences/gordon97 July 13-17, 1997 Plymouth State College Plymouth NH The now bi-annual Gordon conference series in HPC and NII commenced in 1992 and has had its second meeting in 1995. The Gordon conferences are an elite series of conferences designed to advance the state-of-the-art in covered disciplines. Speakers are assured of anonymity and referencing presentations done at Gordon conferences is prohibited by conference rules in order to promote science, rather than publication lists. Previous meetings have had good international participation, and this is always encouraged. Experts, novices, and technically interested parties from other fields interested in HPC and NII are encouraged to apply to attend. All attendees, including speakers, poster presenters, and session chairs must apply to attend. We *strongly* encourage all poster presenters to have their poster proposals in by May 13, 1997, though we will consider poster presentations up to six weeks prior to the conference. Application to attend the conference is also six weeks in advance. More information on the conference can be found at the web page listed above, including the list of speakers and poster presenters and information on applying for attendance. From mpi-core-human@mcs.anl.gov Thu May 15 13:57:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA18982; Thu, 15 May 1997 13:57:15 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA15010 for mpi-core-out; Thu, 15 May 1997 13:00:49 -0500 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA15004 for ; Thu, 15 May 1997 13:00:42 -0500 Message-Id: <199705151800.NAA15004@antares.mcs.anl.gov> Received: from ry70.rz.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Thu, 15 May 1997 20:00:03 +0200 Received: by ry70.rz.uni-karlsruhe.de (1.37.109.16/16.2) id AA295369191; Thu, 15 May 1997 19:59:52 +0200 Subject: may8-typos: chapter0..3 To: mpi-core@mcs.anl.gov Date: Thu, 15 May 1997 19:59:51 +0200 (CES) From: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) X-Mailer: ELM [version 2.4ME+ PL31 (25)] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1881 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk MPI2 draft May 8, 1997 typos: ----------------------------- ABSTRACT "standard. They" --> "standards. They" "is describes" --> "describes" ACKNOWLEDGEMENTS maybe drop Real-Time on page viii? (is not in this document) CH.1 2:16 " and MPI-1.2" --> " and MPI-1.0" (or drop completely) 2:26, 2:39, 2:42 chapter titles do not match CH.2 11:20 fix font of MPI_SUCCESS 15:36 fix font of MPI_INIT and MPI_FINALIZE 15:40 fix font of MPI_COMM_WORLD 15:43 don't pass void* and void*** (NULL cast to these types, if anything) 16:19 "SIG_ALARM" --> "SIG_ALRM" CH.3 17:15 "here is" --> "in version 1.2 is" 17+ 3.1: why is a run-time method needed if a compile-time method is already there? Changing the C #define-s to const int would also give the correct type, so the MPI_GET_VERSION is unnecessary (or do I miss something???) 18:1 drop "is one of the few functions that" (this is a null information) 18:3 "with this" --> "with all of this" 18:42 "MPI_Finalize" --> "MPI_FINALIZE" 19:6 add ")" 19:28 "In no" --> "If no" 19:43+ Print all procedure names in uppercase 19:46 add "()" after MPI_Iprobe (to lign up with 19:43) 20 fix font of several "MPI" 20:38 "return so" --> "returns so" 20:42 "process return" --> "process returns" 21:16 "MPI_Status" --> "MPI_STATUS" ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) No longer on BITNET :-( Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Thu May 15 14:33:10 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA19578; Thu, 15 May 1997 14:33:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA16179 for mpi-core-out; Thu, 15 May 1997 13:38:15 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA16174 for ; Thu, 15 May 1997 13:38:12 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id NAA21291; Thu, 15 May 1997 13:38:05 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id NAA27402; Thu, 15 May 1997 13:38:03 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id SAA13998; Thu, 15 May 1997 18:38:02 GMT From: "Eric Salo" Message-Id: <9705151338.ZM13996@sgi.com> Date: Thu, 15 May 1997 13:38:00 -0500 In-Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) "may8-typos: chapter0..3" (May 15, 7:59pm) References: <199705151800.NAA15004@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke), mpi-core@mcs.anl.gov Subject: Re: may8-typos: chapter0..3 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > CH.3 > 17+ 3.1: why is a run-time method needed if a compile-time method > is already there? Changing the C #define-s to const int > would also give the correct type, so the MPI_GET_VERSION > is unnecessary (or do I miss something???) I asked this same question a while back. The answer I was given is that in the presence of dynamic libraries, there might be a mismatch between the header used during your build and the library used when you run. So depending on your own particular degree of paranoia, you may want to verify one, or the other, or both...or neither. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Fri May 16 10:01:14 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA09736; Fri, 16 May 1997 10:01:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA03193 for mpi-core-out; Fri, 16 May 1997 09:05:02 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA03183 for ; Fri, 16 May 1997 09:04:56 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id JAA13714; Fri, 16 May 1997 09:04:54 -0500 (CDT) Date: Fri, 16 May 1997 09:04:54 -0500 (CDT) Message-Id: <199705161404.JAA13714@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: what is language neutral binding for void *? Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I noticed several external functions do not have the language neural type definition. I have filled most of them in but several I am not sure what to do. They all involve void * arguments. Unfortunitely in MPI-1 we left them off so I don't have a model to go by. The ones in question are: extra_state for MPI_GREQUEST_START, MPI_*_CREATE_KEYVAL attribute_val for MPI_*_SET_ATTR, MPI_*_GET_ATTR They are somewhat like the choice arguments but here they are always an integer in Fortran. Thoughts? Steve From mpi-core-human@mcs.anl.gov Fri May 16 16:20:15 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA13783; Fri, 16 May 1997 16:20:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA13252 for mpi-core-out; Fri, 16 May 1997 15:24:06 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA13233 for ; Fri, 16 May 1997 15:23:54 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id PAA14911; Fri, 16 May 1997 15:23:48 -0500 (CDT) Date: Fri, 16 May 1997 15:23:48 -0500 (CDT) Message-Id: <199705162023.PAA14911@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: proposed text change for MPI_STATUS_IGNORE Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Several people have spoken up against returning error codes when one supplies MPI_STATUS_IGNORE. The reasons ranged from taste to problems with the Fortran and C++ bindings. I now proposed we change the MPI-2 document to state that an error code is not required to be returned when you give MPI_STATUS_IGNORE. The specific change is: 192:30 Add: An MPI implementation is not required to return any error codes when MPI_STATUS_IGNORE or MPI_STATUSES_IGNORE is given. These calls only check that the operation associated with the request is complete. An implementation my choose to return an error code if it wishes but there is no requirement to do so. In the case where a request is a generalized request, the query_fn is not invoked. In this case the error code cannot ever be returned. Advice to users: Using MPI_STATUS_IGNORE or MPI_STATUSES_IGNORE means that error codes are likely to be lost. If there is any chance that an error can occur with the request(s) given then a normal status argument should be provided so the error codes are guaranteed to be returned. Add back to 129:23 the sentence: The callback function is not invoked if the user indicated that no status needs be returned for this request, by supplying MPI_STATUS_IGNORE or MPI_STATUSES_IGNORE. and delete 129:40-46 Unless people indicate otherwise, this will become the new text. Steve From mpi-core-human@mcs.anl.gov Fri May 16 16:28:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA13838; Fri, 16 May 1997 16:28:58 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA13533 for mpi-core-out; Fri, 16 May 1997 15:33:13 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA13525; Fri, 16 May 1997 15:33:06 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id PAA14936; Fri, 16 May 1997 15:33:04 -0500 (CDT) Date: Fri, 16 May 1997 15:33:04 -0500 (CDT) Message-Id: <199705162033.PAA14936@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov In-reply-to: <199705131408.JAA20679@antares.mcs.anl.gov> (hennecke@rz.uni-karlsruhe.de) Subject: Re: Comment on new INTENT paragraph Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There has been a fair amount of discussion about intent and IN/OUT/INOUT. The text is in terms which I edit. I have not seen a conclusion that we should change any text. The waters are muddy here but we did discuss and vote on the text that is currently there. Unless someone gets support for a specific proposal I am not going to change the current text. Steve From mpi-core-human@mcs.anl.gov Fri May 16 17:27:07 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA14452; Fri, 16 May 1997 17:27:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA15190 for mpi-core-out; Fri, 16 May 1997 16:31:52 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA15184 for ; Fri, 16 May 1997 16:31:37 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id RAA04370; Fri, 16 May 1997 17:30:17 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id RAA24990; Fri, 16 May 1997 17:30:43 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.05 (305.3 1-15-1997)) id 85256499.00762758 ; Fri, 16 May 1997 17:30:33 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: lederman@cs.wisc.edu cc: Mpi-Core@mcs.anl.gov Message-ID: <85256499.00755114.00@watngi01.watson.ibm.com> Date: Fri, 16 May 1997 17:29:50 -0400 Subject: Re: what is language neutral binding for void *? Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk We cannot really have a language neutral type for attributes, because an attribute in an integer in Fortran and a pointer in C; and the C pointer is not the same as the Fortran integer, it is a pointer to the Fortran integer. I.e., we have broken the language independence of the MPI binding. We have done this, because the MPI1 binding was not consistent, in Fortran and in C, and this asymmetry was the only way to deal with MPI1 C and Fortran binding and with predefined attributes. I suppose the same could be say about extra_state, although the problem is less visible there, because there is no language conversion. So call it a new name, and explain what this new name means. I noticed several external functions do not have the language neural type definition. I have filled most of them in but several I am not sure what to do. They all involve void * arguments. Unfortunitely in MPI-1 we left them off so I don't have a model to go by. The ones in question are: extra_state for MPI_GREQUEST_START, MPI_*_CREATE_KEYVAL attribute_val for MPI_*_SET_ATTR, MPI_*_GET_ATTR They are somewhat like the choice arguments but here they are always an integer in Fortran. Thoughts? Steve From mpi-core-human@mcs.anl.gov Fri May 16 21:52:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id VAA15932; Fri, 16 May 1997 21:52:48 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id UAA20415 for mpi-core-out; Fri, 16 May 1997 20:56:12 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id UAA20410 for ; Fri, 16 May 1997 20:56:08 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id UAA04638 for ; Fri, 16 May 1997 20:55:59 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id UAA20538 for ; Fri, 16 May 1997 20:55:57 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id BAA19687; Sat, 17 May 1997 01:55:54 GMT From: "Eric Salo" Message-Id: <9705162055.ZM19685@sgi.com> Date: Fri, 16 May 1997 20:55:54 -0500 In-Reply-To: Steve Huss-Lederman "proposed text change for MPI_STATUS_IGNORE" (May 16, 3:23pm) References: <199705162023.PAA14911@rap.cs.wisc.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: proposed text change for MPI_STATUS_IGNORE Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I now proposed we change the MPI-2 document to state that an error code > is not required to be returned when you give MPI_STATUS_IGNORE. By definition, error codes are only returned when an error occurs, and MPI already states in no uncertain terms that the behavior under error conditions is undefined. So while I certainly agree with this sentiment, I can't say that I see a real need for any extra text. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Sat May 17 11:58:42 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA20250; Sat, 17 May 1997 11:58:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA27252 for mpi-core-out; Sat, 17 May 1997 11:02:22 -0500 Received: from home.gis.net (home.gis.net [206.42.64.20]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA18501 for ; Fri, 16 May 1997 18:22:43 -0500 Received: (from nevin@localhost) by home.gis.net (8.8.5/8.6.11) id TAA11051 for mpi-core@mcs.anl.gov; Fri, 16 May 1997 19:21:56 -0400 (EDT) Date: Fri, 16 May 1997 19:21:56 -0400 (EDT) Message-Id: <199705162321.TAA11051@home.gis.net> From: home!nevin@gis.net (Nick Nevin) To: mpi-core@mcs.anl.gov Subject: Re: proposed text change for MPI_STATUS_IGNORE Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Several people have spoken up against returning error codes when one > supplies MPI_STATUS_IGNORE. The reasons ranged from taste to problems > with the Fortran and C++ bindings. I now proposed we change the MPI-2 > document to state that an error code is not required to be returned > when you give MPI_STATUS_IGNORE. The specific change is: I would like to know more about these objections, in particular the bindings problems. I don't like this proposed change at all. If an error occurs then an error should be returned. What happens if the error handler is set to be MPI_ERRORS_RETURN and an error occurs? Seems to me that in this case under the proposed change the user won't be able to detect that the error occurred. --nick. -- Nick Nevin nevin@acm.org From mpi-core-human@mcs.anl.gov Sat May 17 14:46:49 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA21343; Sat, 17 May 1997 14:46:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA28786 for mpi-core-out; Sat, 17 May 1997 13:51:43 -0500 Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA28765; Sat, 17 May 1997 13:50:28 -0500 Message-Id: <199705171850.NAA28765@antares.mcs.anl.gov> Received: from ry70.rz.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Sat, 17 May 1997 20:50:23 +0200 Received: by ry70.rz.uni-karlsruhe.de (1.37.109.16/16.2) id AA113255015; Sat, 17 May 1997 20:50:15 +0200 Subject: Is MPI_Status now a handle ??? To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Date: Sat, 17 May 1997 20:50:14 +0200 (CES) From: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) Reply-To: hennecke@rz.uni-karlsruhe.de (Michael Hennecke) X-Mailer: ELM [version 2.4ME+ PL31 (25)] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1563 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk MPI v1.1 clearly distinguished opaque *types* from *handles* to opaque objects: MPI_Aint and MPI_Status are *types*, but MPI_Comm, MPI_Group, MPI_Datatype, MPI_Request MPI_Op and MPI_Errhandler are *handles*. They are also listed separately in A.2 of MPI v1.1. But: The May-08 draft labels MPI_Status arguments as (handle), see e.g. MPI_STATUS_SET_ELEMENTS and MPI_STATUS_SET_CANCELLED. Is this an error, or an intended change of the binding? ( This has also implications to the INTENT stuff: whereas one may ) ( argue about handle intent being for the handle or for its target ) ( object, for MPI_Status it is *always* for the object since there ) ( is no handle. E.g. in Fortran, STATUS is an array not an integer ) ( pointing to a status structure... ) Thanks, Michael PS: The May-08 draft does not list the new handles MPI_Info, MPI_Win and MPI_File (maybe others) in appendix A. It should. If does list the null handle values, though. ====================================================================== Michael Hennecke http://www.uni-karlsruhe.de/~Michael.Hennecke/ ---------------------------------------------------------------------- University of Karlsruhe RFC822: hennecke@rz.uni-karlsruhe.de Computing Center (G20.21 R210) No longer on BITNET :-( Zirkel 2 * P.O. Box 69 80 Phone: +49 721 608-4862 D-76128 Karlsruhe Fax: +49 721 32550 ====================================================================== From mpi-core-human@mcs.anl.gov Mon May 19 07:26:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA08971; Mon, 19 May 1997 07:26:01 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id GAA08481 for mpi-core-out; Mon, 19 May 1997 06:30:06 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA08476 for ; Mon, 19 May 1997 06:30:01 -0500 Message-Id: <199705191130.GAA08476@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Corrections to May 8 Document Date: Mon, 19 May 1997 06:29:52 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I have read the document cover to cover (while flying back and forth from Norway), and have a lot of corrections. These vary from "remove a comma" to "section is wrong". I have appended them below. I have NOT integrated the comments that others have sent in; certainly some of these are duplicates. I've also noted some general issues while reading it that I have noted first. I have tried not to duplicate the fixes in the examples that I have already made and returned; but to make it easier to identify the exact location of the changes, I have used the May 8th version of the document. Bill All pages/lines relative to version of May 8th General comments The section/subsection titles have inconsistant capitalization. The majority view is to capitalize everything except prepositions and articles. These are easily spotted on the contents pages (ii-vii) Chapters are not being forced to fall on pages of the same parity (in two-sided mode, they should always start on the right, odd-numbered page). How should references to sections in the text appear? The following three are used in the document: Section 1.2, section 1.2, Sec. 1.2 . What font should be used for examples? Some use \rm; others use \em (the default)? In a number of places, different rules for C and Fortran are given, with no mention of C++. In others, "C and C++" or "C/C++" are explicitly mentioned. We could add some text to chapter 2 stating that if rules are provided for C and not for C++, the C rules apply. The phrase "null string" and "empty string" are used in several places. What is usually meant in the document is the string "" in C, which contains the single null character. This is different from (char *)0, which contains no characters. I believe that we should use "empty string" instead of "null string" to avoid confusion with (char *)0. In describing the arguments to the language neutral bindings, we have usually NOT capitalized the phrase describing the arguments. However, we have not been consistent with this (e.g., see page 135, lines 23-25). When describing Fortran INTEGER, we sometimes use integer and sometimes use INTEGER (and sometimes a sans serif INTEGER). In the bindings (even in C), we sometimes use "char *name" and sometimes use "char name[]". We need to decide and stick to it. A similar choice should be made for other arrays (e.g., int *r or int r[]). Specific comments Section 1.2 Organization of this Document Page 2, line 16. "clarifications to MPI-1.1 and MPI-1.2" should be just "clarifications to MPI-1.1". Clarifications to MPI-1.2 should not be necessary in the same chapter that defines MPI-1.2. Page 2, line 48. Needs a sentance on the constant index to parallel the function index. Page 2, line 27. Change "routines to allow" to "routines that allow". Page 2, line 30. Change "(put/get), and" to "(put/get) and" (remove comma). Page 3, line 31. Did "two-phase" change to "split" in the title of the JOD section on two-phase/split communicators? Chapter 2 MPI-2 Terms and Conventions Page 4, line 18. The chapter no longer mentions POSIX. I recommend deleting ", and POSIX interaction". Section 2.1 Document Notation Page 4, line 27. Change "and illustrates" to "and that illustrates" Section 2.2 Naming Conventions Page 4, line 39. Change "In many cases MPI-1" to "In many cases, MPI-1" Page 4, line 41. Change "In MPI-2 an" to "In MPI-2, an" Page 5, lines 19-20. With several of the I/O routines, it is no longer true the identifiers are limited to 30 characters. Either this section or their names must change. Section 2.3 Procedure Specification Page 6, line 22. "... function is show followed ..." should be "... shown ..." I also recommend that after line 23, a sentance along the lines of: "Fortran in this document refers to Fortran 90; see Section 2.6." Section 2.4 Semantic Terms Page 6, line 27. Change "two" to "five" Page 7, line 8. Change "is portable, because" to "is portable because" Section 2.5.1 Opague objects Page 8, line 2. Change "was created, and" to "was created and" Section 2.5.2 Array Arguments Page 9, line 6. Add a period at the end of MPI_STATUSES_IGNORE Section 2.5.4 Named Constants Page 9, lines 22-22. The range of regular values only for TAG are provided by environment inquiry (chapter 7) functions. The range for source, for example, depends on the size of the communicator. The last sentance of this section needs to be reworded. Section 2.6.3 C++ Binding Issues Page 11, line 41. "The file mpi.h will ..." should be "The file mpi.h may ..." since the C and C++ compilers may access completely separate files (each named mpi.h, perhaps in different directories), at the discretion of the implementor. Page 12, line 4. Change "sematic descritions" to "semantic descriptions" Page 13, line 28. Add a "void" in front of "MPI::Attach_buffer". Section 2.8 Error Handling Page 14, line 26. Change ", etc.) This" to ", etc.). This" Page 14, line 36. Change "abort except" to "abort, except" Section 2.9.2 Interaction with Signals Page 16, line 18. "reccomended" should be "recommended" Page 16, line 19. "SIGALARM" should be "SIGALRM" Section 3.1 Version Number Page 17, line 37. Add text similar to that on page 18, lines 3-4, indicating that the definitions and parameters have value 2 for version and 0 for subversion for MPI-2 implementations. The current text can be read as applying only to the MPI_GET_VERSION function. Page 17, line 48. The C++ binding for MPI_GET_VERSION is missing (and also missing from the appendix, A.5, MPI-1.2 C++ bindings (page 289, line 37). Section 3.2.2 Clarification of MPI_FINALIZE Page 19, line 38. Change "In no" to "If no". Page 19, line 38. Change "MPI_BSEND" to "MPI_BSEND (or other buffered send)" Page 20, line 22. "even not" should be "not even". Section 3.2.4 Clarification of MPI_INTERCOMM_CREATE Page 21, line 32. "implmentation" should be "implementation" Section 3.2.10 Minor Corrections Pages 24-32 general comment. Font usage, particularly for examples, is inconsistant. Also, it would help if the old and new text were more clearly marked, perhaps by being used in a narrower or blockquote environment. Also, there were a number of discussion points here; perhaps some of them should be turned into text, comment, or advice to users. Page 24, line 23. Change "Section 8.2.2 of this document" to "Section 8.2.2 of the MPI-2 Standard". Since the text is a change to MPI-1.1, "this document" refers to MPI-1.1, which is not what is intended. This occurs in several places (page 25 line 23 and page 26 line 2). Page 24, line 14. Shouldn't "long long int" be just "long long"? Page 28, lines 29-37. Why not declare error handlers as SUBROUTINE HANDLER_FUNCTION( COMM, ERROR_CODE ) INTEGER COMM, ERROR_CODE and require the MPI implementation to use the calling sequence that matches the language that the error handler was created in. Alternately, a rationale that says that this was considered too burdensome for the limited benefit should be added. See also Page 30, lines 37-41. Page 29, line 12. Note that changing the bindings will break user code that makes use of the profiling interface. This is particularly bad for the routine here, MPI_Pcontrol, since this routine is SUPPOSED to be replaced by the user. Specifically, if the implementation's mpi.h file prototypes MPI_Pcontrol, then this change will break user code (though users can use #if MPI_SUBVERSION = 2 || MPI_VERSION > 1 to test for it). I mention this because (a) it deserves an advice to users and (b) suggests that recent comments on the discussion list about the ability to add "const" willy nilly to the prototypes is in fact prohibited by the profiling interface. The easiest fix is probably: Add Advice to users. This change will require profiling libraries that define an MPI_Pcontrol routine to be re-written. The MPI_VERSION and MPI_SUBVERSION constants in C and C++ may be used to write code that will compile correctly on MPI 1.1, MPI 1-2, and MPI-2. (better would be to drop the change, but it is probably too late for that.) See also Page 31, lines 16-19. Section 4.2.2 The Runtime Environment Page 35, lines 42-43. Change "MPI_COMM_SPAWN, MPI_COMM_SPAWN_MULTIPLE" to "MPI_COMM_SPAWN and MPI_COMM_SPAWN_MULTIPLE". Section 4.3.2 Starting Processes and Establishing Communication Page 37, line 16. Change "the remote." to "the remote group." Page 38, lines 11-14. Change "C" to "C and C++" in several places. Page 40, line 13. MPI_ERRCODES_IGNORE in Fortran is another thing like MPI_BOTTOM or MPI_INPLACE that cannot be assigned. Like MPI_BOTTOM, it cannot be used in initialization expressions (see Named Constants in MPI-1.1 standard). Some discussion of this should perhaps be added to MPI-2 section 2.5.4: Named constants. Section 4.3.3 Starting Multiple Executables and Establishing Communication Page 41, lines 8 and 38. In Fortran, will count need to be positive, even on non-root nodes, because of the ARRAY_OF_ARGV(COUNT,*) declaration? Otherwise, line 8 needs "significant only at root". Page 42, line 32. There is a missing period before "Error codes" (after the formula). Section 4.3.4 Reserved Keys Page 43, line 23. Add a period at the end of "implementation" Page 43, line 26. Add a period at the end of "implementation" Page 43, lines 28-29. There should be some mention that wdir is refering to the "working directory". This just refers to ANY directory on the machine the program is to execute on. Page 43, line 47. Do we require that c divide (b-a) exactly? Or do we mean "a, a+c, a+2c, ..., a+kc, where a+kc <= b and a+(k+1)c > b, for c > 0, and a+kc >= b and a+(k+1)c < b for c < 0, and k integer. Section 4.4 Establishing Communication Page 46, line 6. Change "but asymmetric process" to "but asymmetric, process" Section 4.4.2 Server Routines Page 47, line 29. Change (string) to (handle) on the info argument. Page 47, line 33, 38, and 45. As defined, the system does NOT supply the string, it supplies the VALUE IN the string. Thus, the user must pass in a string of the correct size. This should be char port_name[MPI_MAX_PORT_LEN+1] in C and C++, and character*MPI_MAX_PORT_LEN in Fortran (though Fortran can accept character*(*) since the length of the user's variable is available to Fortran). Page 48, lines 10,11. Change "ip" to "IP" Section 4.4.3 Client Routines Page 49, lines 20 and 24. Change "string" to "handle" for the info and comm arguments. Page 49, line 40. Change "been closed) MPI_COMM_CONNECT" to "been closed), MPI_COMM_CONNECT" Section 4.4.4 Name Publishing Page 51, line 13. Change "results is" to "result is" Page 51, line 45. Change "unpublishss" to "unpublishes" Page 52, line 12. Change "MPI_lookup_name" to "MPI_Lookup_name". Page 52, line 12. As for page 47, lines 33, 38, 45, an array of char large enough to hold the largest possible port name must be passed. Section 4.4.6 Client/Server Examples Page 54, line 19. Add MPI_Close_port( port_name ); Section 4.5.2 Universe Size Page 59, line 38. Change "running MPI_UNIVERSE_SIZE" to "running, MPI_UNIVERSE_SIZE" Page 59, line 39. Change "find out" to "find out about the change" Section 4.5.3 Singleton MPI_INIT Page 60, line 7. Use em dash (--- in LaTeX instead of -) Page 60, line 10. An enumerate with a single item. Remove enumerate and just let the text be a paragraph. Section 4.5.4 MPI_APPNUM Page 60, line 37. "In particular, it is ..." should be "In particular, if it is ..." Page 61, line 4. There is an orphaned period. Section 4.5.6 Another Way to Establish MPI Communication Page 64, line 3. Change "inter-communicator" to "intercommunicator" Page 63, line 21. Do we mean non-blocking or asynchronous? I suspect asynchronous; non-blocking sockets simply allow read and write calls on the socket to return 0/EAGAIN/EWOULDBLOCK (depending on the implementation). Page 63, line 29. "else.." should be "else." Page 63, line 30. The "(see below)." should be something like (See below.) or the period on line 29 after "quiescent" should be removed. Page 63, line 31. Change "and and" to "and" Chapter 5 One-Sided Communication Section 5.1 Introduction Page 64, line 24. I'd change "In our situation" to "In some situations" (I found that I was trying to determine what "our" situation was, while with "some", I know that there isn't a specific example). Section 5.2.1 Memory Allocation Page 66, line 2. I don't like this change. It would seem to make more sense to provide this function only when the "pointer" extension is present; in that case, I'm not sure that the Fortran binding for MPI_ALLOC_MEM is correct (shouldn't it be base instead of INTEGER base?). I view the change as making the Fortran MPI_FREE_MEM inconsistent with MPI_ALLOC_MEM. Page 66, line 27. Change MPI_MALLOC_MEM to MPI_ALLOC_MEM Section 5.2.2 Window Creation Page 67, lines 1, 12, 15. The routine is declared with the names MPI_COMM_CREATE_WIN, MPI_Win_create_win, MPI_WIN_CREATE, and MPI::Win:Create. The language neutral and C names should be changed to MPI_WIN_CREATE and MPI_Win_create. Page 68, line 3. Change "gets and puts" to "gets, puts, and accumulates" Page 68, lines 17 and 21. Change MPI_WIN_INIT to MPI_WIN_CREATE Page 68, line 22. Change "decide" to "decides" Section 5.2.3 Window Attributes Page 69, lines 4-8. MPI_WIN_xxxx need to have \const entries Page 69, line 6. Change "size" to "size in bytes". Page 69, lines 9-10. There needs to be a forward reference to the definitions of MPI_Win_get_attr (in section 7.8.2), since they aren't defined in this chapter. Page 69, lines 14-16. All Fortran routines need an additional ", ierror" argument. Section 5.3.1 Put Page 72, line 4. "weighted" should be "weighed" Section 5.3.3 Examples Page 73, line 17. Change "MPI_EXTENT" to "MPI_TYPE_EXTENT" Page 73, line 18. Change "MPI_WIN_INIT" to "MPI_WIN_CREATE" Page 74, line 32. Change "written, that does not require to build a datatype" to "written that does not requires that a datatype be built" Page 74, line 33. Change "But, then one" to "But, one then" Page 74, line 41. Change MPI_WIN_INIT to MPI_WIN_CREATE Section 5.3.4 Accumulate Functions Page 75, line 11. "This will allow, for example, to accumulate a sum ..." should be "This will allow, for example, the accumulation of a sum ... " Page 76, lines 3-4. There is a paragraph break after "target_disp, in" and before "the target". Page 76, line 10. I'm unclear on whether a basic datatype includes the datatypes provided for MPI_MAXLOC and MPI_MINLOC (e.g., MPI_2INT or MPI_DOUBLE_INT). Should line 6 exclude these two operations? Alternately, do we need to add the datatypes from the MPI-1.2 section "MINLOC and MAXLOC" to the list of acceptable types? Page 76, line 28. Change MPI_WIN_INIT to MPI_WIN_CREATE Section 5.4.1 Fence Page 82, line 20. Change "epoch, if" to "epoch if" Page 82, line 35. Add an advice to users that MPI_WIN_FENCE should be called BEFORE any RMA operations (this is implied by lines 20-28, but users may overlook that). Section 5.4.2 General 2-Party Synchronization Page 83, line 12. Add an advice to users that each process may specify a different group (more precisely, a group with different members) as long as the other restrictions hold (i.e., those in lines 5-6 about corresponding calls to MPI_WIN_POST). Page 83, line 29. Change "Consider a" to "Consider the" Section 5.4.3 Lock Page 87, lines 11-28 (Question). I agree with Marc's rational. Section 5.4.4 Assertions Page 89, lines 7-9. The text for MPI_MODE_NOCHECK for MPI_WIN_POST is unclear. Does "the nocheck information provided by MPI_WIN_POST must be the same as the information provided by the matchine MPI_WIN_START calls" mean that both or neither WIN_POST and WIN_START must use MPI_MODE_NOCHECK in their assertion (this is unlike ready-receive, where only one side must indicate the special case). This needs to be reworded. Section 5.5 Examples Page 90, line 10. Change "Same code" to "The same code" Page 90, line 34. Change "This, because" to "This is required because" Page 92, line 11. Change "(resp. update(A0, A1)" to "(resp. update(A0, A1))" Section 5.6.1 Error handlers Page 92, line 44. Change "int *." to "int *," Page 93, line 4. Is this the correct C++ binding? It looks like this says that the user must define the errhandler in the MPI namespace, which we specifically disallow. I would have thought it would be typedef void (Errhandler_fn)( MPI::Win *, int *, ... ) Also, the generalized request callback spell out function instead of abbreviating it as "fn". MPI-1.1 uses MPI_Handler_function. Page 93, lines 12 and 29. Change "Errhanlder" to "Errhandler" Page 93, line 43. Change "trace" to "track" Section 5.7 Semantics and Correctness Page 96, lines 18,21. The clauses starting with "this, " are confusing. What does "this" refer to? Page 96, line 34. It looks like the \end{enumerate} should go here instead of Page 97, line 26. Page 97, line 21. Change "And two concurrent" to "In addition, two concurrent" Section 5.7.2 Progress Page 98, line 22. Change "occur" to "occurs" Page 98, line 28. Change "next accesses" to "then accesses" Page 98, line 29. Change "calls occurred" to "calls occur" Page 98, line 48. Change "communication, and" to "communication and" Section 5.7.3 Registers and Compiler Optimizations Page 99, line 45. "advise" should be "advice" Page 100, line 8. Change "CALL_MPI_WIN_FENCE" to "call MPI_WIN_FENCE" Page 100, line 24. Change "Details, and" to "Details and" Page 100, line 26. Change "page 175 additional" to "page 175 for additional" Section 5.7.4 Implementation Model Page 100, line 32. Change "coherent, or" to "coherent or" Page 100, line 37. "advise" should be "advice" Page 101, line 28. Remove paragraph break between "thread, " and "daemon". "daemon" should also be "demon". Page 101, line 38. Remove paragraph break between "data)." and "The call to MPI_WIN_WAIT". Page 102, line 14. Change "process have to" to "process must" Page 103, line 29. Remove paragraph break between "and updates" and "are not atomic". Page 103, line 30. Remove paragraph break between "is needed)." and "Then, correct" Section 6 Extended Collective Operations Several places: in-place should be "in place" 6.2 Intercommunicator Constructors Page 106, lines 2-5. Each parameter is a (handle). Page 106, line 42. Remove () from figure caption Page 106, lines 46-47. Remove "" around left and right Page 107, line 26. Change "color of subset assignment" to "control of subset assignment (integer)" Page 107, line 27. Change "color of rank assignment" to "control of rank assignment (integer)". Page 108, line 41. Remove () from figure caption. Section 6.3.2 Operations that Move Data Page 112, line 20. Change "restritions" to "restrictions" Page 112, line 30. Change "all processes" to "all other processes" Page 116, lines 6, 9, 11. Add (significant only at root). Page 117, line 41. Change "direction, by" to "direction by" Page 119, line 26. Change "communication, by" to "communication by" Section 6.3.4 Generalized All-to-all Function Page 123, line 20. Change "indicating" to "indicates" Chapter 7 External Interfaces Section 7.1 Introduction Page 127, line 25. Change "on top of MPI it" to "on top of MPI, it" Section 7.2 Generalized Requests Page 127, line 40. Change "ie." to "i.e." Page 129, line 43 and above. The attribute callbacks can only return MPI_SUCCESS. Note also that MPI defines error CLASSES, not error codes. Exactly what values are legal return values from these callbacks, and, if they are error classes, how are they converted into error codes (note that the statement in MPI-1 that error classes are a subset of error codes means only that there may be more error codes than classes, NOT that all error classes are valid error codes) needs to be specified. Also, if these return values are error codes/classes, shouldn't the C++ version throw an exception instead? Remember that we went through this with the attribute callbacks; see the final decision in chapter 2. Page 129, lines 31-38. Move to page 130, line 21 (after the Question). Page 131, line 7. Place a tie (~) between "bool" and "complete" in the C++ binding for Cancel_function. Page 131, lines 14-22. Error code/class problem. See page 129, line 43. Page 131, line 41. Change "Section 2.4)" to "Section 2.4.)" Page 133, line 6. Change (MPI_SUCCESS) to MPI_SUCCESS Page 134, line 16. Insert "return MPI_SUCCESS" before the end of the routine. Section 7.3 Associating Information with Status Page 134, line 23. Change "In MPI-1 requests" to "In MPI-1, requests" Page 135, line 11 and line 30. Change "INTEGER STATUS," to "INTEGER STATUS(MPI_STATUS_SIZE)" Page 135, after line 16. Do we require MPI_Get_elements and MPI_Get_count to return the count value only if they are called with the same datatype as MPI_Status_set_elements? A statement that it is erroneous to call MPI_Get_elements or MPI_Get_count with a datatype other than that specified in MPI_Status_set_elements. If the intent is that MPI_Get_elements and MPI_Get_count can be called with ANY datatype, that needs to be specified, since the description says that "count" is returned regardless of the datatype. Perhaps a change from "a call to MPI_GET_ELEMENTS" to "a call to MPI_GET_ELEMENTS with the same datatype" and the addition of "It is erroneous to call MPI_GET_ELEMENTS or MPI_GET_COUNT on a status that has been acted on by MPI_STATUS_SET_SIZE with a different datatype.". Section 7.4 Naming Communicators Page 135, line 46. Change "a print identifier" to "a printable identifier" Page 136, line 32. Does the value of MPI_MAX_NAME_STRING include the null in C/C++? I expect not, since the min value is 63. Page 137, line 21. Replace "null string" with "empty string". Page 137, lines 32 and 39-41. These two points (about the difficulties in passing a string as an attribute value in Fortran) seem related. Shouldn't they be merged? Section 7.5 Adding New Error Classes and Error Codes Page 141, line 24. MPI_ADD_ERROR_STRING should be MPI_ERROR_STRING. "null string" should be "empty string". Page 142, line 44. Remove period in "MPI_WIN_CALL_ERRHANDLER. is" Section 7.6 Decoding a Datatype Page 144, lines 25-32. The text that starts "For calls with address..." and ends with a rationale should be moved, possibly to Page 146, line 14 or so. Page 144, line 26. Change "or and address" to "or an address" Page 145, lines 47-48. It is very awkward for the user to have the requirement that the datatype must be free unless it is a basic datatype. Why not always require the user to free the datatype? Page 146, line 15. Change "In the MPI-1 datatype" to "In MPI-1, the Fortran datatype" Section 7.7.2 Clarifications Page 152, line 39. This says that argc and argv are optional in C, but does not specify how that is achived. The probably means that (int *)0 and (char ***)0 are valid arguments, but that needs to be said. Page 152, line 41. The sentance, "This call initializes MPI as MPI_INIT." needs to be clarified. I know what is intended, but not what this sentance says. Page 153, lines 35-37. The paragraph "If provided is not ... by the MPI library." should be an advice to implementors; it has nothing to do with the MPI standard itself. Page 154, lines 41 and 46. These refer to MPI_INITIALIZED in a way that suggests that there is a corresponding MPI_THREAD_INITIALIZED. Page 155, line 2. Here MPI_THREAD_INITIALIZED is mentioned. It isn't mentioned anywhere else. It should be deleted. Page 155, line 8. Change "party library" to "party libraries" Section 7.8.1 Communicators Page 155, line 42. Add (integer) to the comm_keyval definition. Page 156, line 6. Change MPI::Comm_ to MPI::Comm:: Page 156, line 7. Change MPI::Comm_ to MPI::Comm:: Page 156, line 14. Change MPI_Comm_copy_attr_fn to MPI_Comm_copy_attr_function to match use in MPI_Comm_create_keyval. Page 156, line 18. Change MPI_Comm_delete_attr_fn to MPI_Comm_delete_attr_function. Page 156, line 25. Change IERR to IERROR. Page 156, line 33. Change IERR to IERROR. Page 156, lines 36 and 41. Should the C++ bindings for these types be in the MPI namespace? Page 157, line 11. Change "communicator specific" to "communicator-specific" Page 157, line 15. Add (handle) to the comm argument Page 157, line 16. Add (integer) to the comm_keyval argument Page 157, line 32. Add (handle) to the comm argument Page 157, line 33. Add (integer) to the comm_keyval argument Page 158, line 2. Add (handle) to the comm argument Page 158, line 3. Add (integer) to the comm_keyval argument Section 7.8.2 Windows Page 158, line 23. Add (integer) to the win_keyval argument Page 158, line 35. Change MPI::Win_ to MPI::Win:: Page 158, line 36. Change MPI::Win_ to MPI::Win:: Page 158, line 40. Change MPI_Win_copy_attr_fn to MPI_Win_copy_attr_function Page 158, line 44. Change MPI_Win_delete_attr_fn to MPI_Win_delete_attr_function Page 159, line 3. Change IERR to IERROR Page 159, line 10. Change IERR to IERROR Page 159, line 14. Change Copy_attr_fn to Copy_attr_function Page 159, line 19. Change Delete_attr_fn to Delete_attr_function Page 159, line 24. Add (integer) to win_keyval argument Page 159, line 35. Add (handle) to win argument Page 159, line 36. Add (integer) to win_keyval argument Page 160, line 2. Add (handle) to win argument Page 160, line 3. Add (integer) to win_keyval argument Page 160, line 21. Add (handle) to win argument Page 160, line 22. Add (integer) to win_keyval argument Section 7.8.3 Datatypes Page 160, line 41. Add (integer) to type_keyval argument Page 161, line 6. Change MPI::Datatype_c to MPI::Datatype::C Page 161, line 7. Change MPI::Datatype_d to MPI::Datatype::D Page 161, line 11. Change MPI_Type_copy_attr_fn t MPI_Type_copy_attr_function Page 161, line 15. Change MPI_Type_delete_attr_fn to MPI_Type_delete_attr_function Page 161, line 21. Change IERR to IERROR Page 161, line 32. Change Copy_attr_fn to Copy_attr_function Page 161, line 36. Change Delete_attr_fn to Delete_attr_function Page 162, line 5. Add (handle) to type argument Page 162, line 6. Add (integer) to type_keyval argument Page 162, line 21. Add (handle) to type argument Page 162, line 22. Add (integer) to type_keyval argument Page 162, line 39. Add (handle) to type argument Page 162, line 40. Add (integer) to type_keyval argument Page 162, line 47. Change MPI::Type to MPI::Datatype Chapter 8 Language Bindings Section 8.1.2 Design Page 164, line 36 (point 3). Now that status is optional, there is no longer a one-to-one mapping between MPI functions and their C++ bindings. Adding a sentance about this exception would fix this. Section 8.1.5 Semantics Page 167, lines 7-14. Unfortunately, MPI::COMM_NULL is an invalid argument for MPI::Comm::Compare. Simply test bar == MPI::COMM_NULL instead. Section 8.1.6 C++ Datatypes Page 168, line 28. Change "MPI basic datatypes" to "MPI basic datatypes that may be used in the predefined reduce operations. Page 168, lines 30-40. This should be updated section 9.9.2 (MPI_SIGNED_CHAR and MPI_UNSIGNED_CHAR). Page 169, table 8.1. MPI::CHAR should be MPI::SIGNED_CHAR; note that MPI::CHAR should refer to the C "char", which in MPI is viewed as a character, not integer, type (particularly in heterogeneous environments; consider ASCII to/from EBCDIC or an environment where char is Unicode). Also, the datatypes "signed short int" should be "signed short" etc. (no int at end). Page 170, line 9. Should there be an MPI::LONG_LONG? Page 170, line 28. What datatypes may be used the MPI_MINLOC and MPI_MAXLOC? Section 8.1.7 Exceptions Page 170, line 32. Change "MPI::ERRORS_THROW_EXCEPTIONS, for" to "MPI::ERRORS_THROW_EXCEPTIONS for" Section 8.1.9 Profiling page 172, line 27. The "static const" here conflicts with the "extern const" in example 8.4 (page 172, line 48). Which is correct? Page 173, line 5 "calss" should be "class" Page 173, line 6. We should note that we do not mandate a libmpi++.a. Section 8.2.2 Problems with Fortran bindings for MPI Page 175, line 5 "the void *" should be "the use of void *" Page 175, line 23. Remove the word "unlikely" (I relied on this to test the examples!) Page 175, line 37. Change "the section" to "the array section" Page 175, line 42. There is an extra space before the footnote number Page 176, lines 10,11,16,44. 'simple' should be `simple' (improper leading quote when typeset). Page 176, line 13. Change "(we know" to "(though we know" Page 176, line 26. There is an extra space before the footnote number Page 176, line 27. Change "Some compilers, Fortran 77 included," to "Some compilers, even for Fortran 77,". Fortran 77 is not a compiler. (Note that Fortran 77 specifies value-result, not reference, handling of arguments) Page 177, line 1. Change "inbibits" to "inhibits" Page 177, line 19. There is an extra space before the footnote number Page 178, line 4. Add "call MPI_TYPE_COMMIT( newtype, ierror )" Page 178, line 26. Change "etc. uses" to "etc.\ uses" Page 178, line 39. Change "(type)" to "(type, ierr)" Page 180, line 1. Change "planned" to "under consideration". Nonblocking collective was "planned" for MPI-2. Section 8.2.3 Basic Fortran Support Page 180, line 23. Change "to meet requirement 2 by" to "that requirement two be met by" Section 8.2.5 Routines to Support Fortran Parameterized Types Page 182, line 41. Change "the declaration a" to "the declaration, a" Page 183, line 7. Change "MPI_Datatype newtype" to "MPI_Datatype *newtype" Page 183, lines 9-10. Change IERR to IERROR Page 183, line 25. Change "MPI_Datatype newtype" to "MPI_Datatype *datatype" Page 183, lines 27-28. Change IERR to IERROR Page 183, lines 41-46. Insert "CALL" and add ", ierror" as necessary Page 185, line 5. If there is REAL*16, shouldn't there be COMPLEX*32? Page 185, line 12. Change "In MPI-2 one" to "In MPI-2, one" Page 185, line 26. Add a \\ in " X\\INTEGER" Page 185, lines 47-48. Make Fortran binding all uppercase. Page 186, lines 44-47. I agree. Simplest to change "will not" to "often will not" in text. Page 187, lines 5,6. Add ", ierr" Page 187, line 40. Add ", ierror" Page 187, line 3. Add ", ierror" Chapter 9 Miscellany Section 9.1 Portable MPI Process Startup Page 189, lines 37-38 and lines 43-46. These are inconsistant; one says that implementors are advised only to provide "-n " and the other that implementors are advised to provide "-soft < > etc.". The simplest fix is to delete lines 32-38. Page 190, line 34. Change \ in the LaTeX to {\tt \char`\\} Page 191, line 46. Remove "const" from scalar arguments. Page 191, line 46. Change "Create_Indexed_Block" to "Create_indexed_block" Section 9.3.1 Passing MPI_STATUS_IGNORE for Status Page 192, line 11. Change "many cases application" to "many cases, application" Page 192, line 12. Change "In these cases it" to "In these cases, it" Page 192, line 20. Note that MPI_STATUS_IGNORE is an object like MPI_BOTTOM in Fortran (not usable for initialization or assignment). Page 192, line 25. The sentance starting "These are". What does "These" refer to? Section 9.4 Error Class of Invalid Keyval Page 193, line 27. Change "MPI{_TYPE,_COMM,_WIN}" to "MPI_{TYPE,COMM,WIN}" Section 9.6 Determining Whether MPI Has Finished Page 194, lines 14-23. Note that the C and Fortran names are MPI_Finalized and C++ is MPI::Is_finalized. Either the function should be named MPI_FINALIZED or MPI_IS_FINALIZED. Page 194, line 25. Change "function" to "routine" (the value returned is the error code in C, the result in C++, and none in Fortran). Section 9.7.1 Introduction Page 194, line 44. Change "language, and the clients" to "language and the clients" Section 9.7.2 Assumptions Page 195, line 24. Change "(KIND=MPIOFFSETKIND" to "(KIND=MPI_OFFSET_KIND)" Section 9.7.5 Status Page 199, lines 33- page 200 line 21: The material added to handle MPI_STATUS_IGNORE and MPI_STATUSES_IGNORE will not work. For example, in MPI_Status_f2c, the binding is (correctly) MPI_Status_f2c( MPI_Fint *f_status, MPI_Status *c_status). Note that the TYPE of MPI_STATUS_IGNORE is (MPI_Status *) (the most likely value is ((MPI_Status *)0)). This value CAN NOT be RETURNED in a type of (MPI_Status *); for that, we'd need (MPI_Status **). But if we make MPI_Status_f2c( MPI_Fint *f_status, MPI_Status **c_status), then either MPI_Status_f2c must allocate the status (this is bad) or the user must do something like MPI_Status s, *sp = &s; MPI_Status_f2c( f_status, &sp ); if (sp != MPI_STATUS_IGNORE) { ... s contains a valid status .... } This is ok, though ugly. Interestingly enough, with this change, the binding for MPI_Statuses_f2c need not be changed, though I believe that wasn't intended. The type for an array of MPI_Status that can be assigned into in C is (MPI_Status *), not (MPI_Status **). Again, the usage may not be obvious to users. For example, MPI_Status s[10], *sp = s; MPI_Statuses_f2c( 10, f_status, &sp ); if (sp != MPI_STATUSES_IGNORE) { ... s contains 10 valid status entries ... } These should be added as examples, if this is truely how we want to do this. Page 200, lines 6-8. Exactly how is the implementation to make the Fortran value of MPI_STATUS_IGNORE available to the C library? It can't be called MPI_STATUS_IGNORE (well, almost not - not if the Fortran doesn't have the ability to set a pointer to a value like 0). Is it MPI_F_STATUS_IGNORE? Something else? Does initialization time refer to MPI_INIT/MPI_THREAD_INIT (I hope so!)? Section 9.7.6 MPI Opaque Objects Page 200, line 40. "The function MPI_ADDRESS returns" should be "The function MPI_GET_ADDRESS returns". Note that if the default integer kind and MPI_ADDRESS_KIND are different, it is impossible to implement this. Page 200, line 48. Change INTEGER TYPE, ADDR, IERR to INTEGER TYPE, IERR INTEGER (KIND=MPI_ADDRESS_KIND) ADDR Page 201, line 3. Change MPI_ADDRESS to MPI_GET_ADDRESS Page 201, lines 32. Change MPI_ADDRESS to MPI_GET_ADDRESS Page 201, line 43. Change "buff" to "buf" Page 201, line 47. Change "buff" to "buf". Page 202, line 1. Change MPI_ADDRESS to MPI_GET_ADDRESS. Section 9.7.7 Callback functions Page 202, line 9. Change "language will be" to "language may be" Section 9.7.8 Attributes Page 203, line 19. Change MPI{_TYPE,_COMM,_WIN} to MPI_{TYPE,COMM,WIN} Page 203, line 37. Change "manipulate in Fortran" to "manipulate Fortran" Page 203, line 41. Change "address valued" to "address-valued" Page 204, line 34. Insert a space before the arguments of MPI_Comm_get_attr to allow line breaks. Page 204, line 37. Add ",ierr" to the argument list for MPI_WIN_GET_ATTR Section 9.7.11 Interlanguage Communication Page 205, line 33. The word "textually" appears to refer to the type signature. The word "textually" is used nowhere in the MPI-1 specification. The phrase "type signature" should be used instead. Section 9.8 New Datatype Manipulation Functions Page 206, line 32. Change "Fortran binding address" to "Fortran binding, address" Page 206, line 34. Change "more convenient type" to "more convenient, type" Section 9.8.1 Type Constructors with Explicit Addresses Page 206, lines 41-42. The statement claims that the names are not available in C++, but MPI::Address is. In addition, the other three "new" routines appear in the MPI-1 C++ binding in Annex B. They should not. Page 208, line 23. Change "MPI::Type_create" to "MPI::Datatype:Create" Section 9.8.2 Extent and Bounds of a Datatype Page 209, line 7. Add (handle) to datatype argument Page 209, line 8. Add (integer) to lb argument Page 209, line 10. Add (integer) to extent argument Page 209, line 27. Change "overiden" to "overidden" Page 209, line 46. Remove "const" from scalar arguments Section 9.8.3 True Extent of Datatypes Page 210, line 23. Add (handle) to datatype argument Page 210, line 24. Add (integer) to true_lb argument Page 210, line 25. Add (integer) to true_extent argument Section 9.9.3 Unsigned long long Type Page 212, line 6. Should we have MPI_LONG_LONG since we have MPI_UNSIGNED_LONG_LONG? Section 9.10 Functions and Macros Page 212, line 15. Add an advice to users: If these routines are implemented as macros, they will not work with the MPI profiling interface. Section 9.11 Cannonical MPI_PACK and MPI_UNPACK Page 212, line 42. INTEGER OUTSIZE and POSITION should be INTEGER(KIND=MPI_ADDRESS_KIND) since they are MPI_Aint in C. Page 212, line 47, 48. Change "MPI_Aint" to "MPI::Aint" (2 places) Page 213, line 18. INTEGER INSIZE and POSITION should be INTEGER(KIND=MPI_ADDRESS_KIND) since they are MPI_Aint in C. Page 213, line 23. Change "MPI_Aint" to "MPI::Aint" (2 places) Page 213, line 35. "MPI_Aint size" should be "MPI_Aint *size" Page 213, line 38. "INTEGER SIZE" should be "INTEGER (KIND=MPI_ADDRESS_TYPE) SIZE" if MPI_Aint corresponds to this type of Fortran integer. Page 213, line 40. "void MPI::Datatype::Pack_external_size( ..., MPI_Aint size)" should be "MPI_Aint MPI::Datatype::Pack_external_size(...)" Section 9.12 Profiling Interface Page 213, line 47. Change "(See Section 9.10" to "(See Section 9.10)" Page 214, lines 3-4. "The exemption of MPI_WTIME and MPI_WTICK from the profiling interface is a change from MPI-1.1." This statement is unclear. MPI-1.2 (and MPI-2) still requies a PMPI_WTIME and PMPI_WTICK that produce the correct results. The exemption is only for the ability to replace, at link time, the user of MPI_WTIME and MPI_WTICK with user-defined versions (see 9.10). Chapter 10 I/O Section 10.2.8 File Info Page 226, lines 35-37. Why is "datarep" ignored on input, even to MPI_FILE_OPEN? Page 227, lines 2, 6, 9, 12, 17. Hints are variously "most relevant" or "only relevant" Is most == only? Section 10.3 File Views Page 228, line 16. Change "file), where" to "file) where" Section 10.4.1 Data Access Routines Page 230, lines 25-28. This paragraph says that "Implementation of data access routines may buffer data to improve performance." This is very misleading; it seems to imply that the routines are more like the level 3 Unix routines (fread/fwrite) than the level 2 (read/write). The mention of MPI_FILE_SYNC suggests the intent is that data may be "in the system" but not on permanent storage, as long as all other semantics, including any concurrent access from other MPI programs that have opened the same file, are maintained. Looking at the section on semantics, I see that the fread/fwrite semantics are the defaults. Page 230, lines 27-28. (See also page 261). Does MPI_FILE_SYNC really guarentee that the file is written to "permanent" storage? The "sync" routine in Unix does not guarentee that much. What it usually means is that any data or commands that are destined for the disk, but that have not yet been issued by the application, are sent. Few if any implementations of sync wait for the data to be physically written to the physical storage device. Page 230, line 41. Change MPI_XXX to MPI_FILE_XXX. Page 231, line 21. There needs to be some mention of the split collective here, if only a forward reference. Page 232, line 9. The rationale is strange. Is this really an advice to users? Also, this is the first mention of pipe; do we really need this? Section 10.4.2 Data Access with Explicit Offsets Page 236, line 23. The argument definitions (from lines 2-8 above) need to be here. Page 238, line 24. The argument definitions (from lines 2-8 above) need to be here. Section 10.4.4 Data Access with Shared Filed Pointers Page 244, lines 1-3. Why is the implementation being constrained here? Why is it required that the call can only return after all the processes within the group have called it? I agree that a programmer must assume that the call has blocking collective semantics, but why can it not behave as an MPI_BCAST, for example? To fix this most simply, change "the call can" to "the call may" Page 245, line 27. "in this section". Which section? I believe that this refers only to MPI_FILE_SEEK_SHARED and MPI_FILE_GET_POSITION_SHARED, but I'm not sure (the "Seek" heading has no section number). Page 246, lines 3-4. Another implementation constraint. Why is MPI_FILE_SEEK_SHARED {\em required} to synchronized the group with a barrier? If it has collective semantics it might (already specified in line 1), but I see no reason to require it. Section 10.5.1 Datatypes for File Interoperability Page 253, line 41. Change MPI_ADDRESS_SIZE to MPI_ADDRESS_KIND Section 10.5.2 External Data Representation: "external32" Page 254, line 11. How are bits ordered in each byte? Page 255, lines 20-21. These lines are blank. They shouldn't be. Section 10.5.3 User Defined Data Representations Page 257, line 15. Change "USERBUF, FILEBUF" to "USERBUF(*), FILEBUF(*)" Section 10.6.1 File Consistency Page 261, line 27. What does it mean for the data to be written to "permanent storage"? This is far stronger than the Unix "sync" system call. Section 10.6.4 Collective File Operaitons Page 262, lines 18-26. The word "interleaved" is not used in the MPI-1.1 document. Section 4.12, MPI-1.1, details the ordering of collective operations and the requirement that point-to-point operations, mixed in with collective operations, should not cause deadlock when used properly. It isn't clear what exactly is meant here. It might be simpler just to leave out the word "interleaved" and indicate that the non-collective I/O routines are treated in the same way as the point-to-point operations in 4.12, MPI-1.1. Section 10.6.9 File Size Page 263, lines 31, 33, 35, 39, 41, 45. MPI routines should be \mpifunc and all upper case. Page 264, lines 5, 6. MPI routines should be upper case. Section 10.7.2 Local Array Filetype Constructor Page 265, line 10. "const enum order" should be just "enum order". Page 265, line 10. Actually, it needs an enum name, or a typedef that is an enum. "enum" by itself isn't a valid C type. Page 265, line 42. The sentance is garbled. Perhaps change "... ndims = 0, oldtype is an" to " ndims = 0. Let oldtype be an". Page 266, lines 1-44. Change lb to MPI_LB and ub to MPI_UB. Section 10.7.3 Distributed Array Filetype Constructors Page 268, line 3. "enum order" isn't a valid declaration. Page 268, line 8. There is a > in a font that doesn't have a ">" character (has upside down question mark instead). Page 268, line 21. Change "ARRAY(CYCLIC(15))" to "the HPF layout ARRAY(CYCLIC(15))" Page 268, lines 30-- page 270, line 20. Needs to be reformatted to use verbatim only for code fragments, not text (e.g., lines 30-31). Section 10.8 I/O Error Handling Page 271, line 16. "user defined" should be "user-defined" Page 271, line 21. Change "second argument" to "first argument" Page 271, line 46. Make MPI_File_errhandler_fn an \mpifunc{} reference, all uppercase. Page 272, line 4. Add a rationale for why void * is used instead of MPI_File *. Section 10.9 I/O Error Classes Page 273, line 6. Change "implementation dependent" to "implementation-dependent" Section 10.10.1 Double Buffering with Split Collective I/O Example Page 273, line 44. Remove the "Example" from the end of the title Page 273, line 45. Change these two sentances to read better. For example, "This example shows how to overlap computation and output. The computation is performed by the function {\tt compute\_buffer()}." Bibliography Page 284, line 27. Change "C++" to "{C++}" in the .bib file Page 284, line 36. Change "Fortran" to "{Fortran}" and "MPI" to "{MPI}" in the .bib file Page 285, line 8-9. Change "BSD" to "{BSD}" and "PSD" to "{PSD}" in the .bib file. Change "Unix" to "{U}nix". Page 285, line 29. Replace "Draft version" with report number, if any, or change to unpublished. Page 285, lines 33-34. Either change "in press" to "In press" or update reference. Annex A Language Binding Section A.1 Introduction Page 286, line 19. Change "MPI-2 binding are" to "MPI-2 bindings are" Most of the rest of these indicate an error somewhere else in the document. I'm marking them here so that they aren't lost. Section A.2.1 Constants Page 288, lines 6-16. What are "c_order" through "write_once" doing here? Should they be categorized under some other heading? Section A.5 MPI-1.2 C++ bindings Page 289, line 38. As noted above, MPI::Get_version is missing. Section A.6.1 Process Creation and Management Page 290, line 33. MPI_lookup_name should be MPI_Lookup_name. Section A.7 MPI-2 C Functions Page 297, lines 29-34. These are C++ functions, not C functions (MPI::Datarep_conversion_function and MPI::Datarep_extent_function) Section A.9 MPI-2 Fortran Subroutines Page 310, line 23. IERR should be IERROR Page 310, line 30. IERR should be IERROR Page 310, line 46. IERR should be IERROR Page 311, line 4. IERR should be IERROR Page 311, line 9. IERR should be IERROR Page 311, line 15. IERR should be IERROR Section A.10 MPI-2 C++ Bindings In general, shouldn't many of the "char foo[]" be "char foo[MPI_MAX_FOO_LEN]" or "foo[MPI_MAX_FOO_LEN+1]"? Section A.10.4 External Interfaces Page 315, line 33. MPI::Type should be MPI::Datatype. Section A.10.5 Miscellany Page 316, lines 19-20. MPI_Aint should be MPI::Aint Page 316, line 28. MPI_Aint should be MPI::Aint. Section B Note that Section 9.8.1 claims that 4 MPI 1.1 functions have no C++ equivalents. This should be mentioned if true, or corrected. (These are on page 328). Section B.4 C++ Bindings for Point-to-Point Communication Page 328, lines 19-20 and 25-33. The functions Create_hvector, Create_hindexed, Create_struct use their MPI-2 versions, and Address is not supposed to exist, replaced by Get_address. See 9.8.1. Section B.8 C++ Bindings for Environmental Inquiry Page 332, line 21. "char name[]" should be "char name[MPI_MAX_PROCESSOR_NAME]" Page 332, line 29. "char name[]" should be "char name[MPI_MAX_ERROR_STRING]". Page 332, line 40. Should Is_initialized be Initialized? Section MPI Function Index Page 336-337. Note that MPI_FILE_READ_EXPLICIT_ALL_BEGIN has 32 characters, and MPI_FILE_WRITE_EXPLICIT_ALL has 33. I observed this by having a Fortran 90 compiler signal these as fatal errors. From mpi-core-human@mcs.anl.gov Mon May 19 10:13:31 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA10620; Mon, 19 May 1997 10:13:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA10400 for mpi-core-out; Mon, 19 May 1997 09:18:15 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id JAA10387 for ; Mon, 19 May 1997 09:18:09 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA07758; Mon, 19 May 1997 10:15:27 -0400 Message-Id: <3380604A.41C6@kgn.ibm.com> Date: Mon, 19 May 1997 10:17:52 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: William Gropp Cc: mpi-core@antares.mcs.anl.gov Subject: Re: Corrections to May 8 Document References: <199705191130.GAA08476@antares.mcs.anl.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk William Gropp wrote: > Page 29, line 12. > > Note that changing the bindings will break user code that makes use of > the profiling interface. This is particularly bad for the routine > here, MPI_Pcontrol, since this routine is SUPPOSED to be replaced by > the user. Specifically, if the implementation's mpi.h file prototypes > MPI_Pcontrol, then this change will break user code (though users can > use > #if MPI_SUBVERSION = 2 || MPI_VERSION > 1 > to test for it). I mention this because (a) it deserves an advice to > users and (b) suggests that recent comments on the discussion list > about the ability to add "const" willy nilly to the prototypes is in > fact prohibited by the profiling interface. > > The easiest fix is probably: > > Add > > Advice to users. This change will require profiling libraries > that define an MPI_Pcontrol routine to be re-written. The > MPI_VERSION and MPI_SUBVERSION constants in C and C++ may be used > to write code that will compile correctly on MPI 1.1, MPI 1-2, and > MPI-2. > > (better would be to drop the change, but it is probably too late for > that.) See also Page 31, lines 16-19. It is unfortunate that the MPI C bindings do not include the const attribute at all the appropriate places because Bill is absolutely correct. A profiler author must be able to assume either that the first argument of his MPI_Send will be declared with the const attribute or without it. Even if it OUGHT to be declared with const, the profiling interface becomes non-protable if some mpi.h files have const and others do not. I would like to see an advise to implementers added to the C bindings discussion 2.6.2: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ There are many places in MPI C bindings where the const attribute would be appropriate since the argument is passed by reference and the MPI code will never modify its value. However, profiling interface code must be written to match the prototype in mpi.h. If some implementation were to supply the const attribute on an appropriate argument while another did not, a profiler routine would not be portable between the two. Implementations should use the const attribute exactly where it is specified by the binding in the standard. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ I also would like to see the binding of MPI_Pcontrol retain const and the 1.2 document simply state that the forum has recognized that it is superfluous but harmless. It is retained to avoid breaking existing profilers. ********** Can you imagine what Bill would have come up with if he had flown to Australia and back? ************* Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Mon May 19 10:26:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA10724; Mon, 19 May 1997 10:26:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA10722 for mpi-core-out; Mon, 19 May 1997 09:31:43 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA10715; Mon, 19 May 1997 09:31:38 -0500 Message-Id: <199705191431.JAA10715@antares.mcs.anl.gov> To: Dick Treumann cc: mpi-core@antares.mcs.anl.gov Subject: Re: Corrections to May 8 Document In-Reply-to: Message from Dick Treumann of "Mon, 19 May 1997 10:17:52 -0400." <3380604A.41C6@kgn.ibm.com> Date: Mon, 19 May 1997 09:31:36 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I second both of Dick's suggestions. The advice for implementors is easy; is there any way that we can retract the change to MPI_PCONTROL? Bill From mpi-core-human@mcs.anl.gov Mon May 19 11:10:15 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA11066; Mon, 19 May 1997 11:10:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA11961 for mpi-core-out; Mon, 19 May 1997 10:13:09 -0500 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA11942 for ; Mon, 19 May 1997 10:12:58 -0500 Received: from jim (gw2-131.pool.dircon.co.uk [194.112.35.131]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id QAA05815; Mon, 19 May 1997 16:12:47 +0100 (BST) Message-Id: <199705191512.QAA05815@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id QAA05877; Mon, 19 May 1997 16:14:15 +0100 To: William Gropp cc: Dick Treumann , mpi-core@antares.mcs.anl.gov Subject: Re: Corrections to May 8 Document In-reply-to: Your message of "Mon, 19 May 1997 09:31:36 CDT." <199705191431.JAA10715@antares.mcs.anl.gov> Date: Mon, 19 May 1997 16:14:15 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I think you're all getting a bit too paranoid. The profile writer should *always* put the const in, and it won't break anything, because both C and C++ always allow you to pass a non-const actual to a const formal. (This is an example of exceeding the specification, which is OK, you're passing something which could be modified, but the subroutine is promising not to modify it. Fine). What's not allowed is to pass a const actual to a non-const formal, because there you may be breaking something. Of course, most of the code which uses the profile interface probably hasn't been written with const in it, so it will break. However once it's written with const, then it can be used on libraries independently of whether they have const in their headers. So, it's a once only change, not a once per MPI implementation change. (It's still not great, but it's not as bad as you seem to be making out). -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Mon May 19 11:25:41 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA11200; Mon, 19 May 1997 11:25:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA12529 for mpi-core-out; Mon, 19 May 1997 10:30:48 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA12522; Mon, 19 May 1997 10:30:44 -0500 Message-Id: <199705191530.KAA12522@antares.mcs.anl.gov> To: James Cownie cc: Dick Treumann , mpi-core@antares.mcs.anl.gov Subject: Re: Corrections to May 8 Document In-Reply-to: Message from James Cownie of "Mon, 19 May 1997 16:14:15 +0100." <199705191512.QAA05815@mailhost.dircon.co.uk> Date: Mon, 19 May 1997 10:30:41 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | So, it's a once only change, not a once per MPI implementation change. | (It's still not great, but it's not as bad as you seem to be making | out). True, as long as it is restricted to MPI_PCONTROL. If we let it spread beyond that, it will defeat the intent of the profiling interface - to encourage the development of libraries and tools that work with any MPI implementation. Bill From mpi-core-human@mcs.anl.gov Mon May 19 11:39:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA11432; Mon, 19 May 1997 11:39:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA12894 for mpi-core-out; Mon, 19 May 1997 10:45:18 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA12889 for ; Mon, 19 May 1997 10:45:15 -0500 Message-Id: <199705191545.KAA12889@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: More changes Date: Mon, 19 May 1997 10:45:13 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My C++ compiler doesn't like const in front of function pointer definitions, such as int Create_keyval(const Copy_attr_function *type_copy_attr_fn, ... ^^^^^ Can someone determine if this is a compiler error or if const is meaningless in this context? Bill From mpi-core-human@mcs.anl.gov Mon May 19 11:50:40 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA11620; Mon, 19 May 1997 11:50:39 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA13193 for mpi-core-out; Mon, 19 May 1997 10:55:57 -0500 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA13188 for ; Mon, 19 May 1997 10:55:52 -0500 Received: from jim (gw2-121.pool.dircon.co.uk [194.112.35.121]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id QAA07995; Mon, 19 May 1997 16:55:42 +0100 (BST) Message-Id: <199705191555.QAA07995@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id QAA06266; Mon, 19 May 1997 16:57:09 +0100 To: William Gropp cc: James Cownie , Dick Treumann , mpi-core@antares.mcs.anl.gov Subject: Re: Corrections to May 8 Document In-reply-to: Your message of "Mon, 19 May 1997 10:30:41 CDT." <199705191530.KAA12522@antares.mcs.anl.gov> Date: Mon, 19 May 1997 16:57:09 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > True, as long as it is restricted to MPI_PCONTROL. If we let it > spread beyond that, it will defeat the intent of the profiling > interface - to encourage the development of libraries and tools that > work with any MPI implementation. I don't think so, you can make it work everywhere *if* you are slightly grubby in the profile wrappers. So, a wrapper which looks like this will work everywhere (I claim) int mpi_foo (const type_whatever arg) { return pmpi_foo ((type_whatever) arg); } The points being that 1) It has a const formal argument, so can be called with either const or non-const actual. 2) It calls the underlying function with a non-const, which will work whether or not the underlying function has a const specified formal. It's entirely reasonable for a compiler to warn about dropping out the constness, but it should compile this without a problem... In C++ there's some more amusement, because the constness can be used in choosing which overloaded function to call, however provided people are reasonable, it should still all work. -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Mon May 19 12:21:23 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA11987; Mon, 19 May 1997 12:21:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA14415 for mpi-core-out; Mon, 19 May 1997 11:25:49 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA14410 for ; Mon, 19 May 1997 11:25:44 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA25249; Mon, 19 May 1997 12:22:48 -0400 Message-Id: <33807EE9.41C6@kgn.ibm.com> Date: Mon, 19 May 1997 12:25:13 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: James Cownie Cc: William Gropp , Dick Treumann , mpi-core@antares.mcs.anl.gov Subject: Re: Corrections to May 8 Document References: <199705191555.QAA07995@mailhost.dircon.co.uk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk James Cownie wrote: > I don't think so, you can make it work everywhere *if* you are > slightly grubby in the profile wrappers. > > So, a wrapper which looks like this will work everywhere (I claim) > > int mpi_foo (const type_whatever arg) > { > return pmpi_foo ((type_whatever) arg); > } > If Bob's MPI shop provides an mpi.h file the gives: int MPI_Send(const void *buff,....); and Joe's MPI Shoppe provides mpi.h with: int MPI_Send(void *buff,.........); The correct MPI program can use either mpi.h file and perhaps get a bit better optimization from Bob's version. Bob will use const when writing his MPI_Send code and Joe omit it on his so each will be happy with the way their mpi.h is written. Their code and mpi.h files match. What does a profiler writer who wants to write: #include "mpi.h" int MPI_Send(????? ,...) { return( PMPI_Send(......); } do to for the definition of formal argument number 1 to compile with either mpi.h and not get fatal compile errors from one or the other? This is the reason for my advice that const should not be added by the implementer if the standard does not say it is part of the binding. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Mon May 19 16:20:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA15075; Mon, 19 May 1997 16:20:09 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA21890 for mpi-core-out; Mon, 19 May 1997 15:24:23 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id PAA21805; Mon, 19 May 1997 15:22:04 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA08661; Mon, 19 May 1997 16:19:27 -0400 Message-Id: <3380B661.167E@kgn.ibm.com> Date: Mon, 19 May 1997 16:21:53 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: William Saphir Cc: mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: dynamic chapter changes References: <199705082116.OAA08170@nsgi3.lbl.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk William Saphir wrote: > > - changed MPI_COMM_GET_PARENT to return MPI_COMM_NULL if parent > has been freed or disconnected. Changed advice to users to reflect > this. > It seems to me that MPI_COMM_FREE on the communicator returned by MPI_COMM_GET_PARENT is about as reasonable as MPI_REQUEST_FREE on a receive request. MPI-1 does not call it erroneous but does point out that no reasonable program will do this. Is there a circumstance where an MPI spawn group could SAFELY follow a return from MPI_Comm_free on this communicator with a call to MPI_Abort(MPI_COMM_WORLD) or a segfault? I am refering to SAFE for the parent. If not, I would suggest either pointing this out in the new advice or even declaring MPI_COMM_FREE on this communicator an error. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Mon May 19 16:42:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA15518; Mon, 19 May 1997 16:42:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA22496 for mpi-core-out; Mon, 19 May 1997 15:48:15 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA22476; Mon, 19 May 1997 15:47:59 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id NAA07112; Mon, 19 May 1997 13:47:42 -0700 From: "William Saphir" Message-Id: <9705191347.ZM7111@nsgi3.lbl.gov> Date: Mon, 19 May 1997 13:47:41 -0700 In-Reply-To: Dick Treumann "Re: dynamic chapter changes" (May 19, 4:21pm) References: <199705082116.OAA08170@nsgi3.lbl.gov> <3380B661.167E@kgn.ibm.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: Dick Treumann Subject: Re: dynamic chapter changes Cc: mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On May 19, 4:21pm, Dick Treumann wrote: > Subject: Re: dynamic chapter changes > William Saphir wrote: > > > > > - changed MPI_COMM_GET_PARENT to return MPI_COMM_NULL if parent > > has been freed or disconnected. Changed advice to users to reflect > > this. > > > It seems to me that MPI_COMM_FREE on the communicator returned by > MPI_COMM_GET_PARENT is about as reasonable as MPI_REQUEST_FREE on a > receive request. MPI-1 does not call it erroneous but does point out > that no reasonable program will do this. > Is there a circumstance where an MPI spawn group could SAFELY follow a > return from MPI_Comm_free on this communicator with a call to > MPI_Abort(MPI_COMM_WORLD) or a segfault? I am refering to SAFE for the > parent. > > If not, I would suggest either pointing this out in the new advice or > even declaring MPI_COMM_FREE on this communicator an error. > I agree that MPI_COMM_FREE on the parent intercommunicator is fairly useless. We have essentially deprecated MPI_COMM_FREE in favor of MPI_COMM_DISCONNECT. I'd be happy to put in an advice to users to this effect if there is no objection. Seems to me though that disallowing COMM_FREE would just confuse the issue (because it would be a special case with no precedent). Do any implementors expect that MPI_COMM_DISCONNECT might be substantially more expensive than MPI_COMM_FREE? If so, there is one case where a library might want to use COMM_FREE for its side effect of hiding the fact that there is a parent. For instance, user code might do: MPI_Comm_get_parent(&parent); if (parent == MPI_COMM_NULL) { /* I'm a parent... */ spawn children... } else { /* I'm a child ... */ be a child } Now imagine that the parent is spawned by a library that wants to allow the above user code to work. The hiding can also be done with MPI_COMM_DISCONNECT, so the only benefit to COMM_FREE might be performance. I'm not advocating we recommend COMM_FREE for this, just trying to make sure we don't miss anything. Bill From mpi-core-human@mcs.anl.gov Mon May 19 16:47:07 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA15594; Mon, 19 May 1997 16:47:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA22601 for mpi-core-out; Mon, 19 May 1997 15:52:32 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA12409; Mon, 19 May 1997 10:27:39 -0500 Received: from mcs.anl.gov by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA23733; Mon, 19 May 1997 11:25:03 -0400 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA12401; Mon, 19 May 1997 10:27:27 -0500 Message-Id: <199705191527.KAA12401@antares.mcs.anl.gov> To: treumann@minnie.nic.kingston.ibm.com (Dick Treumann) Cc: mpi-bind <@minnie.nic.kingston.ibm.com:mpi-bind@mcs.anl.gov>, mpi-core <@minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov>, treumann@windsurf.pok.ibm.com Subject: Re: MPI_STATUS_IGNORE etc. In-Reply-To: Message from treumann@kgn.ibm.com (Dick Treumann) of "Wed, 14 May 1997 09:11:43 -0400." <9705141311.AA12356@windsurf.pok.ibm.com> Date: Mon, 19 May 1997 10:27:25 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dick has raised the excellent question of MPI_STATUS_IGNORE in Fortran. There is a solution (I think): Fortran now has optional arguments. This would make the Fortran bindings like the C++ ones: there is no special "status" value, just a different interface. For example, for MPI_WAIT, the following INTERFACE specification works just fine: interface subroutine mpi_wait( req, status, ierror ) optional :: status integer req, ierror integer status(4) end subroutine end interface The following test program also works --- start example subroutine mpi_wait( req, status, ierror ) OPTIONAL:: status Integer req, status(4), ierror if (present(status)) then print *, 'status present' else print *, 'no status' endif req = 0 ierror = 0 return end program main integer a, ierr integer status(4) interface subroutine mpi_wait( req, status, ierror ) optional :: status integer req, ierror integer status(4) end subroutine end interface call mpi_wait( a, status, ierr ) call mpi_wait( a, ierror=ierr ) end --- end example Note that if we make ierror optional as well, you can do "call mpi_wait(a)". I suggest that we eliminate MPI_STATUS_IGNORE and MPI_STATUS_IGNORE from the Fortran bindings, and make use of the languages "optional" feature. This works most easily only for the last argument in the argument list (except for ierror). Note that we could even eliminate MPI_BOTTOM if we forced user to use the argument keyword approach for this case. We could even eliminate MPI_STATUS_IGNORE from C by forcing a varargs binding... Bill From mpi-core-human@mcs.anl.gov Mon May 19 17:43:40 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA16181; Mon, 19 May 1997 17:43:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA23986 for mpi-core-out; Mon, 19 May 1997 16:48:57 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA23963; Mon, 19 May 1997 16:48:24 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id OAA07257; Mon, 19 May 1997 14:48:05 -0700 From: "William Saphir" Message-Id: <9705191448.ZM7256@nsgi3.lbl.gov> Date: Mon, 19 May 1997 14:48:05 -0700 In-Reply-To: William Gropp "Re: MPI_STATUS_IGNORE etc." (May 19, 10:27am) References: <199705191527.KAA12401@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: William Gropp , treumann@minnie.nic.kingston.ibm.com (Dick Treumann) Subject: Re: MPI_STATUS_IGNORE etc. Cc: mpi-bind , mpi-core Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On May 19, 10:27am, William Gropp wrote: > Subject: Re: MPI_STATUS_IGNORE etc. > Dick has raised the excellent question of MPI_STATUS_IGNORE in Fortran. There > is a solution (I think): Fortran now has optional arguments. We considered this at one point (in the context of ierror), and the consensus (I think) was to delay consideration of optional arguments until the "full fortran support" of MPI 2.x Optional arguments are potentially a win, but they're not completely straightforward. For instance: - The MPI_STATUS argument is one candidate. Probably the most useful one would be IERROR. There's also potential for the DATATYPE argument (which can for contiguous arrays be inferred from the type of the buffer argument), the count argument (same thing) the communicator argument (MPI_COMM_WORLD) the tag argument (MPI_ANY_TAG) the source argument (MPI_ANY_SOURCE). I'm not arguing for these, just pointing out that it's a slippery slope, and that we might want to address all of them at once. - Optional arguments require a run time determination of whether an argument exists. This is in critical path code. A possibly better approach (because it can be done at compile time) would be to have overloaded functions, one with the argument and the other without (as is proposed in C++). The same issue goes for varargs in C. [In the past, the Forum has worried about things as small as adding an extra entry to MPI_Status because it might cause a cache miss. The figure that's thrown around is that on "some systems" a cache miss takes a microsecond, but these systems are probably not the ones with 3-4 microsecond MPI latency. Any implementors with really low latencies care to comment on how important these issues are?] - Adding optional arguments in any form requires explicit interfaces. At this point we allow explicit interfaces but do not require them. This was controversial, and voted on. Bill From mpi-core-human@mcs.anl.gov Mon May 19 20:05:23 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA17624; Mon, 19 May 1997 20:05:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA27476 for mpi-core-out; Mon, 19 May 1997 19:09:35 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id RAA25282; Mon, 19 May 1997 17:19:01 -0500 Received: from timbuk-fddi.cray.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA09180; Mon, 19 May 1997 18:16:30 -0400 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id RAA01756; Mon, 19 May 1997 17:18:55 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id RAA03554; Mon, 19 May 1997 17:18:53 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id WAA07858; Mon, 19 May 1997 22:18:52 GMT From: "Eric Salo" Message-Id: <9705191718.ZM7856@sgi.com> Date: Mon, 19 May 1997 17:18:52 -0500 In-Reply-To: William Gropp "Re: MPI_STATUS_IGNORE etc." (May 19, 10:27am) References: <199705191527.KAA12401@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-bind <@minnie.nic.kingston.ibm.com:mpi-bind@mcs.anl.gov>, mpi-core <@minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov> Subject: Re: MPI_STATUS_IGNORE etc. Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I suggest that we eliminate MPI_STATUS_IGNORE and MPI_STATUS_IGNORE from the > Fortran bindings, and make use of the languages "optional" feature. I completely agree. > We could even eliminate MPI_STATUS_IGNORE from C by forcing a varargs > binding... We only added the magic word to get around the problem of passing a NULL pointer in Fortran. Since this is now a moot point, we can just following common practice and let C programmers pass NULL. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Tue May 20 04:54:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA17715; Tue, 20 May 1997 04:54:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA03086 for mpi-core-out; Tue, 20 May 1997 03:59:18 -0500 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA03080 for ; Tue, 20 May 1997 03:59:10 -0500 Received: from jim (gw2-111.pool.dircon.co.uk [194.112.35.111]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id JAA15138; Tue, 20 May 1997 09:59:05 +0100 (BST) Message-Id: <199705200859.JAA15138@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id JAA00912; Tue, 20 May 1997 09:33:05 +0100 To: William Gropp cc: mpi-core@antares.mcs.anl.gov Subject: Re: More changes In-reply-to: Your message of "Mon, 19 May 1997 10:45:13 CDT." <199705191545.KAA12889@antares.mcs.anl.gov> Date: Tue, 20 May 1997 09:33:05 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > My C++ compiler doesn't like const in front of function pointer definitions, > such as > > int Create_keyval(const Copy_attr_function *type_copy_attr_fn, ... > ^^^^^ > > Can someone determine if this is a compiler error or if const is meaningless > in this context? I think it's meaningless. (What do you think it should mean ? You can never indirect through the function pointer and do anything [other than apply the function] anyway). int Create_keyval(Copy_attr_function * const type_copy_attr_fn, ... would be maningful, though. (It means you can't change the pointer value). -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Tue May 20 05:16:41 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA24022; Tue, 20 May 1997 05:16:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA03264 for mpi-core-out; Tue, 20 May 1997 04:22:12 -0500 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA03259 for ; Tue, 20 May 1997 04:22:07 -0500 Received: from jim (gw2-138.pool.dircon.co.uk [194.112.35.138]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id KAA17988 for ; Tue, 20 May 1997 10:22:03 +0100 (BST) Message-Id: <199705200922.KAA17988@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id KAA01109; Tue, 20 May 1997 10:02:01 +0100 To: mpi-core@mcs.anl.gov Subject: Re: Corrections to May 8 Document In-reply-to: Your message of "Mon, 19 May 1997 12:25:13 EDT." <33807EE9.41C6@kgn.ibm.com> Date: Tue, 20 May 1997 10:02:01 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dick writes :- > If Bob's MPI shop provides an mpi.h file the gives: > int MPI_Send(const void *buff,....); > > and Joe's MPI Shoppe provides mpi.h with: > int MPI_Send(void *buff,.........); > What does a profiler writer who wants to write: > #include "mpi.h" > int MPI_Send(????? ,...) > { > return( PMPI_Send(......); > } > do to for the definition of formal argument number 1 to compile with > either mpi.h and not get fatal compile errors from one or the other? He does what I put before :- int MPI_Send (const void *buff,...) { return PMPI_Send ((void *)buff, ...); } This will work in C (unless there's some clever name mangling so that the linker can check function prototypes against implementations, in which case the MPI_Send (const void *) won't satisfy the calls to MPI_Send(void *)). The fact that the implementation of MPI_Send may not match the prototype in the header should be at worst a warning. -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Tue May 20 09:27:48 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA27623; Tue, 20 May 1997 09:27:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA05777 for mpi-core-out; Tue, 20 May 1997 08:32:30 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA05769 for ; Tue, 20 May 1997 08:32:25 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA25659; Tue, 20 May 1997 09:29:28 -0400 Message-Id: <3381A7CB.41C6@kgn.ibm.com> Date: Tue, 20 May 1997 09:31:55 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: James Cownie Cc: mpi-core@mcs.anl.gov Subject: Re: Corrections to May 8 Document References: <199705200922.KAA17988@mailhost.dircon.co.uk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk James Cownie wrote: > The fact that the implementation of MPI_Send may not match the > prototype in the header should be at worst a warning. > > -- Jim It seems the author's of xlc (the compiler I use) do not agree that this mismatch is only worthy of a warning. The following example produces a serious error on xlc. int foo( int*); int foo(const int *i) {} I understand the argument that it should be a warning but am not sure the ISO C standard supports the argument. Clearly, int foo(const int*); int foo(int *i){} should be a serious error and I did not find a place in the older ANSI document I have which distinguishes one redeclaration from the other. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue May 20 09:37:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA27781; Tue, 20 May 1997 09:37:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA06007 for mpi-core-out; Tue, 20 May 1997 08:43:12 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA05982; Tue, 20 May 1997 08:42:35 -0500 Message-Id: <199705201342.IAA05982@antares.mcs.anl.gov> To: "William Saphir" cc: treumann@minnie.nic.kingston.ibm.com (Dick Treumann), mpi-bind , mpi-core Subject: Re: MPI_STATUS_IGNORE etc. In-Reply-to: Message from "William Saphir" of "Mon, 19 May 1997 14:48:05 -0700." <9705191448.ZM7256@nsgi3.lbl.gov> Date: Tue, 20 May 1997 08:42:33 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | - Optional arguments require a run time determination of whether | an argument exists. This is in critical path code. But detecting the presence of MPI_STATUS_IGNORE also requires a test; in some cases, this test will be MORE expensive than testing for the presence of an optional argument (testing the presence of the optional arg is probably a test of a local register; testing in Fortran for the address of MPI_STATUS_IGNORE may require testing against a globally known address, which often requires a separate load). The varargs case in C is weaker, but having MPI_STATUS_IGNORE already mandates a test that was not necessary (though may often have been performed to catch user errors) in MPI 1.x. Bill From mpi-core-human@mcs.anl.gov Tue May 20 09:43:40 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA27901; Tue, 20 May 1997 09:43:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA06158 for mpi-core-out; Tue, 20 May 1997 08:49:05 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA06142; Tue, 20 May 1997 08:48:49 -0500 Message-Id: <199705201348.IAA06142@antares.mcs.anl.gov> To: "William Saphir" cc: treumann@minnie.nic.kingston.ibm.com (Dick Treumann), mpi-bind , mpi-core Subject: Re: MPI_STATUS_IGNORE etc. In-Reply-to: Message from "William Saphir" of "Mon, 19 May 1997 14:48:05 -0700." <9705191448.ZM7256@nsgi3.lbl.gov> Date: Tue, 20 May 1997 08:48:47 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | - The MPI_STATUS argument is one candidate. ... The situation is slightly different from the previous case where we considered all possible uses of optional arguments. We have chosen to use optional arguments in C++ for status, even though the values used in C could have worked (though I agree with the decision the Forum took on the C++ bindings). We should consider the MPI_STATUS case on its own merits, without worrying about other candidates. I believe that a good case has been made that we are ALREADY on the slipperly slope with "special case values" (MPI_BOTTOM, MPI_STATUS_IGNORE, MPI_ERRCODES_IGNORE, ...). Bill From mpi-core-human@mcs.anl.gov Tue May 20 10:52:14 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA28978; Tue, 20 May 1997 10:52:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA07840 for mpi-core-out; Tue, 20 May 1997 09:56:56 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA07828 for ; Tue, 20 May 1997 09:56:50 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id QAA19301 (8.6.13/IDA-1.6); Tue, 20 May 1997 16:56:46 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id QAA12729; Tue, 20 May 1997 16:56:46 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199705201456.QAA12729@awsrr.rus.uni-stuttgart.de> Subject: Re: Corrections to May 8 Document To: gropp@mcs.anl.gov (William Gropp) Date: Tue, 20 May 1997 16:56:45 +0200 (DST) Cc: mpi-core@antares.mcs.anl.gov In-Reply-To: <199705191130.GAA08476@antares.mcs.anl.gov> from "William Gropp" at May 19, 97 06:29:52 am Content-Type: text Content-Length: 1183 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Page 5, lines 19-20. With several of the I/O routines, it is no > longer true the identifiers are limited to 30 characters. Either this > section or their names must change. > Section MPI Function Index > Page 336-337. > Note that MPI_FILE_READ_EXPLICIT_ALL_BEGIN has 32 characters, and > MPI_FILE_WRITE_EXPLICIT_ALL has 33. I observed this by having a > Fortran 90 compiler signal these as fatal errors. There was a proposal at the last meeting to change "EXPLICIT" into "AT" and this would solve the problem, because we have it only with the routines MPI_FILE_WRITE_EXPLICIT_ALL_BEGIN MPI_FILE_READ_EXPLICIT_ALL_BEGIN MPI_FILE_WRITE_EXPLICIT_ALL_END ------------------------------ 123456789012345678901234567890 Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Tue May 20 16:43:13 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA06255; Tue, 20 May 1997 16:42:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA17331 for mpi-core-out; Tue, 20 May 1997 15:46:55 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id PAA17325 for ; Tue, 20 May 1997 15:46:50 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA06736; Tue, 20 May 1997 16:44:20 -0400 Message-Id: <33820DB7.41C6@kgn.ibm.com> Date: Tue, 20 May 1997 16:46:47 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: comments on first 80 pages Content-Type: multipart/mixed; boundary="------------167E2781446B" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This is a multi-part message in MIME format. --------------167E2781446B Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 --------------167E2781446B Content-Type: text/plain; charset=us-ascii; name="final" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="final" I have had little time to review the May 8 dreft and what little time I have had in now over. Here are comments on the first part of the document. The rest I have not reviewed at all this time. references are page:line-line 2:38 Remove: and handlers 3:8-9 The last sentence is no longer true 4:41 Fotran -> Fortran 6:27 The first two -> The first five 7:46 This says that an opaque object is no longer accessible after a MPI_Whatever_free is made on it. The new, never voted, semantic of MPI_Request_free on a GR directly violates this. 9:8-13 Is there really any use of this concept in the body of the document? I looked at MPI_ERRHANDLER_SET and did not see any discussion of the errorhandler being a "state". I do not recall the concept elsewhere either. 11:17 add: To avoid conflicts with case insensative languages which may use lower case for external names, also avoid mpi_ and pmpi_. 12:4 spelling: descriptions, not descritions 18:16 "or not" adds nothing to the intent. 18:21 I suggest second sentence become "Unless there has been a call to MPI_ABORT, each ....." 18:40-43 repeats same info that is above the example. 18:48 Consider adding word "operation" before "completion" 19:28 first word shoind be If, not In 21:12 Make {Test,Wait}_{all,some} caps and add any 21:18 TEST_ANY -> TESTANY 22:20-23 Wording is awkward - maybe: The MPI 1 standard specifies the output argument of MPI_TYPE_SIZE in the C binding as type int. The MPI Forum has considered proposals to change this and decided to reiterate the original decision. 23:1-6 Does this say that it is erroneous for a copy or delete function to return anything other than MPI_SUCCESS? We may have chosen to beg the issue of what sort of error is reported when there is a non-0 return but did we really intend to label user programs which return error codes as no longer correct? 24:22, 25:24, 26:3 remove the word Fortran 29 The handle type MPI_Errhandler should be added to list of handle types. Pg 214 in changebar copy of 1.1 which is what I have handy. 33:20 Chapter 4 does not read like it has "relaxed" something. How about saying we "extended" the model? To me, "relax" is something like saying "MPI 1.1 did not allow MPI_Foo to be used on fribbles but MPI 2 relaxed this restriction." 36:10-11 "(group,rank) identification is not unique" is ambiguous because (group,rank) does identify a unique process, it is not the unique identifier for the process. Maybe: "(group,rank) is not a unique identifier" Several pages not reviewed ......... 55:12-? Are keys and values case sensitive? I did not see an explicit statement on this and I think there should be one. 63:1-2 What does an MPI which does not exist in a Berkley sockets environment do? Does it not provide the entry point at all, treat a call as a fatal error or politely return MPI_COMM_NULL? I do not care which but think the standard should say. 63:3-4 Since MPI states that interaction between MPI and non-MPI modes of interprocess communication is undefined, can we imply that MPI_COMM_JOIN is OK between processes which share a MPI_COMM_WORLD or MPI parent/child relationship? I think this should say: The processes must not have a preexisting MPI connection. 67:32 span, not spawn 67:36-40 MPI_Type_extent should be mentioned. 68:30 All other MPI_Whatever_free functions are local and lazy and not collective. MPI_File_close is similar to MPI_Win_free in that it is collective and non-local, possibly blocking. Resources are recovered before it returns. I think the use of "free" in the name MPI_Win_free confuses the issue and MPI_Win_close would be better. 69:44 Suggest rephrasing "than we need to needlessly add" to say "then we must add otherwise unneeded" 72:1-5 Note that a remote window is sized in bytes while a put/get uses datatypes, remotely interpreted. The amount of information and evaluation required to locally decide if a put/get of a arbitrary type at a remote task in a heterogeneous environment may be quite substantial. 82:28 I do not understand what this sentence is trying to convey. there are certainly similarities but not "equivelence" as far as I can see. That is as far as I got. Vacation is upon me. --------------167E2781446B-- From mpi-core-human@mcs.anl.gov Tue May 20 19:04:31 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA07417; Tue, 20 May 1997 19:04:30 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA21228 for mpi-core-out; Tue, 20 May 1997 18:09:34 -0500 Received: from win62.nas.nasa.gov (win62.nas.nasa.gov [129.99.50.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA21219 for ; Tue, 20 May 1997 18:09:27 -0500 Received: (from ies@localhost) by win62.nas.nasa.gov (8.8.3/NAS.6.1) id QAA06922; Tue, 20 May 1997 16:09:24 -0700 (PDT) Date: Tue, 20 May 1997 16:09:24 -0700 (PDT) Message-Id: <199705202309.QAA06922@win62.nas.nasa.gov> From: "Ian E. Stockdale" To: mpi-core@mcs.anl.gov In-reply-to: <3381A7CB.41C6@kgn.ibm.com> (message from Dick Treumann on Tue, 20 May 1997 09:31:55 -0400) Subject: prototypes and const [was Re: Corrections to May 8 Document] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk James Cownie wrote: >> The fact that the implementation of MPI_Send may not match the >> prototype in the header should be at worst a warning. Dick Treumann followed up: >It seems the authors of xlc (the compiler I use) do not agree that this >mismatch is only worthy of a warning. The following example produces a >serious error on xlc. > >int foo( int*); > >int foo(const int *i) {} > >I understand the argument that it should be a warning but am not sure >the ISO C standard supports the argument. Clearly, > >int foo(const int*); > >int foo(int *i){} > >should be a serious error and I did not find a place in the older ANSI >document I have which distinguishes one redeclaration from the other. Here is what I managed to extract from the ANSI C document. I also took a quick look through summaries of the ISO addendum and corrigenda to see if there were relevant changes. (My apologies to those trying to correlate ISO section numbers with obsolete ANSI section numbers.) Given: a1. int foo( int * ); b1. int foo( const int *i ) {} *or* a2. int foo( const int *i); b2. int foo( int *i ) {} 1. Both pairs of declarations are compatible. (ANSI 3.5.4.3) [ Note especially regarding parameter type comparisons in determining compatibility of function types: "For each parameter declared with qualified type, its type for these comparisons is the unqualified version of its declared type."] 2. Since they are compatible, declaration a1(a2) is a legal prototype for definition b1(b2). (ANSI 3.1.2.6) *Assuming* I haven't made a mistake in reading 3.5.4.3, or missed some additional relevant text in the standard, it looks like: Given source code which uses a1 (a2) as a prototype for b1 (b2) and does the appropriate casting, a compiler: (i) shouldn't issue an error, since neither (a1,b1) nor (a2,b2) violates a syntax rule or constraint (ii) may issue a warning for one, both, or neither of (a1,b1) and (a2,b2) This agrees with Jim and disagrees with xlc. -Ian P.S. The above notwithstanding, if a program (say in foo or a function called by foo) attempts to modify an object which was declared "const int bar" and whose address was then passed to foo, "the behavior is undefined". (ANSI 3.5.3) You also aren't going to get much benefit from const-based optimizations by writing this type of code. -- Ian E. Stockdale MRJ Technology Solutions, NASA Ames Research Center ies@nas.nasa.gov (415) 604-4628 From mpi-core-human@mcs.anl.gov Wed May 21 06:07:14 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id GAA13021; Wed, 21 May 1997 06:07:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id FAA27240 for mpi-core-out; Wed, 21 May 1997 05:11:03 -0500 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id FAA27235 for ; Wed, 21 May 1997 05:10:57 -0500 Received: from jim (gw2-106.pool.dircon.co.uk [194.112.35.106]) by mailhost.dircon.co.uk (8.8.4/8.7.3) with SMTP id LAA12005 for ; Wed, 21 May 1997 11:10:52 +0100 (BST) Message-Id: <199705211010.LAA12005@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id KAA00766; Wed, 21 May 1997 10:52:15 +0100 nTo: Hans-Christian Hoppe CC: mpi-core@mcs.anl.gov Subject: Re: Corrections to May 8 Document In-reply-to: Your message of "Wed, 21 May 1997 10:39:04 +0200." Date: Wed, 21 May 1997 10:52:14 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Well, despite Ian's reading of the ANSI standard, it seems that many of the compilers do not accept mixed const/non-const functions and prototypes, so we have a problem. Therefore, it seems to me, we have a number of possibilities 1) Forbid const in mpi.h + it's pretty much where we are now + it's consistent with (at least one) compiler which accepts prototypes, but not const ! + profile writers have something consistent to write to - it hurts users who are trying to enforce const correctness 2) Require const in mpi.h + it allows users to write const correct code. + profile writers have something consistent to write to 3) Allow either consistent const, or no-const in mpi.h. + no need to change existing header files - profile writers have a problem (but it can at least be fixed by #define __MPI__CONST const or #define __MPI__CONST and using __MPI__CONST in the profile function declarations). 4) Anarchy, allow const on some functions and non-const on other "const-able" functions. - no advantages for anyone. Thinking about 3, does suggest another possible solution, which is to require mpi.h to #define MPI_CONST (or whatever the naming police would like this to be called). So how about this for a proposal :- 1) mpi.h must either consistently use const, or consistently not use const. 2) mpi.h must contain a definition of the pre-processor symbol MPI_CONST, which must have one of the two values "const" or "" (without the "" of course), set in the appropriate way. The advantages of this are :- 1) The only change needed to existing mpi.h files is the addition of a suitable definition of MPI_CONST. 2) Profile code can be written to be consistent with const or non-const headers. -- Jim James Cownie Dolphin Interconnect Solutions Phone : +44 117 9071438 E-Mail: jcownie@dolphinics.com From mpi-core-human@mcs.anl.gov Wed May 21 13:49:29 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA21260; Wed, 21 May 1997 13:49:28 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA05122 for mpi-core-out; Wed, 21 May 1997 12:47:43 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA05092 for ; Wed, 21 May 1997 12:47:18 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id NAA16752 for ; Wed, 21 May 1997 13:46:14 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id NAA04783 for ; Wed, 21 May 1997 13:46:37 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 8525649E.00619693 ; Wed, 21 May 1997 13:45:55 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: mpi-core@mcs.anl.gov Message-ID: <8525649E.0054E19C.00@watngi01.watson.ibm.com> Date: Wed, 21 May 1997 11:33:09 -0400 Subject: chapter 5 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Thanks for the detailed comments (Rolf, Bill, Lyndon). I plan to incorporate all corrections. There are two pending issues. 1. How to deal with error handlers -- see my previous message. 2. Change in Fortran binding for MPI_FREE_MEM. Two people objected to it, and nobody expressed support for it. Thus, unless I hear soon support for the change, I shall revert to the previous binding, which is what we voted officially on. I don't think that the issue is very important, although I would have prefered more consistency between C and Fortran. From mpi-core-human@mcs.anl.gov Wed May 21 14:34:07 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA22859; Wed, 21 May 1997 14:34:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA06238 for mpi-core-out; Wed, 21 May 1997 13:35:36 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA06225 for ; Wed, 21 May 1997 13:35:18 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id OAA17196 for ; Wed, 21 May 1997 14:34:14 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/01-15-97) with SMTP id OAA25167 for ; Wed, 21 May 1997 14:34:39 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 8525649E.006606B5 ; Wed, 21 May 1997 14:34:23 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: mpi-core@mcs.anl.gov Message-ID: <8525649E.0065DB54.00@watngi01.watson.ibm.com> Date: Wed, 21 May 1997 14:33:30 -0400 Subject: error handling Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I not sure we have closed on the design for error handlers. Let me outline what I believe to be the logical choices and ask for oppinions. There are two entities to consider: the opaque error handler object, and the error handler callback function. The first is an MPI opaque object, accessed by a handle (of type MPI_Errhandler, in MPI1). The second is a function provided by the user. This function has, in C, a typedef of errhandler_fn(MPI_xxx handle, void* extra_state,...), where xxx is, respectively, Com, File, or Win. There are three possible designs: A) 3/3: 3 different types of opaque error handler objects and 3 types of callback functions. We introduce new (C) types MPI_Win_errhandler, MPI_Comm_errhandler (same as MPI1 MPI_Errhandler) and MPI_File_errhandler. We introduce new constants MPI_WIN_ERRORS_RETURN, MPI_COMM_ERRORS_RETURN, etc. We triplicate all error handler functions. In C++ each type of error handler, and each callback typedef is in the namespace of the corresponding class (Comm, Win or File). One advantage of this proposal is layerability and extensibility: implementors of MPI-IO can deal with error handling on thir own; libraries that add some new opaque objects, can add their new error handlers, too. B) 3/1: 3 different types of callback functions, but only one type of opaque error handler object. In C++ the three typedefs for the callback functions are in the namespace of the corresponding class. We triplicate all functions that deall with callbacks, or with communicators, windows of files; namely all error handler functions, except MPI_Errhandler_free. [[There is no point in triplicating this last function, if there is only one MPI_Errhandler object type -- there is no gain in layerability.]] In terms of implementation, this means that the opaque error handler object registers which kind of callback function is attached to it. We have a run-time type matching, rather than compile-time matching: MPI can check, when MPI_XXX_ERRHANDLER_SET(handle, errhandler) is invoked, that the callback function attached to errhandler is one that takes handles to XXX. C) 1/1: one type of callback function, one type of opaque object. The typedef of the callback function becomes, in C, errhandler_fn(void * handleptr, void * extra_state,...), where handleptr can be, in one version, the MPI handle coerced to void* or, in another version, a pointer to the MPI handle coerced to void*. We still triplicate the functions that associate error handlers with objects (MPI_XXX_ERRHANDLER_SET, MPI_XXX_ERRHANDLER_GET), but we do not need to replicate MPI_Errhandler_create and MPI_Errhandler_free. We get, with this proposal, a lower function count. We get layerability and extensibility: if an implementer adds a new FOO opaque object (e.g., FILE), then it provides MPI_FOO_ERRHANDLER_GET and MPI_FOO_ERRHANDLER_SET. There is no way for MPI to do type matching. It's up to the user to invoke MPI_XXX_ERRHANDLER_SET(handle, errhanlder) with an errhandler object with an attached callback function that takes arguments of type XXX. I think we agreed on (B), but there are still some inconsistencies among chapters (e.g., do we triplicate MPI_Errhandler_free), and there is still a proposal from Squires to go to C. I am not sure that we discussed the issue with a clear view of the alternatives and their pros and cons. Hence, this mesage. From mpi-core-human@mcs.anl.gov Wed May 21 22:04:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA02238; Wed, 21 May 1997 22:04:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA14230 for mpi-core-out; Wed, 21 May 1997 21:09:56 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA09132; Wed, 21 May 1997 15:28:53 -0500 From: squyres@cse.nd.edu Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id PAA02190; Wed, 21 May 1997 15:28:48 -0500 (EST) Received: (from jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) id PAA29273; Wed, 21 May 1997 15:28:48 -0500 (EST) Date: Wed, 21 May 1997 15:28:48 -0500 (EST) Message-Id: <199705212028.PAA29273@porthos.lsc.nd.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: 1.2 C++ function names In-Reply-To: <199705211417.JAA00769@rap.cs.wisc.edu> References: <199705211417.JAA00769@rap.cs.wisc.edu> X-Mailer: VM 6.31 under Emacs 19.34.1 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Andy Lumsdaine and I think that be a Good Thing to have a cross reference table in the C++ annex for the function names. That is, it will cross reference the language neutral names to the C++ names for the MPI 1.2 functions. This is not new material -- it simply will be an aid for C++ programmers because it collates a bunch of information from around the document into a single list. We think that this is a Good Thing for the following reasons: - Some of the names are different between C and C++ in 1.2 (the C++ names are J&E conformant, while the C names are not) - Some of the C/Fortran 1.1/1.2 functions are being discouraged, and are not included in the C++ bindings. These should be specifically noted. - There are some new functions that do not have counterparts in C/Fortran for 1.2, such as the Status access functions. These also should be specifically noted. Any objections? (I'm volunteering free work here, who in their right mind would turn it down? :-) {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Wed May 21 22:06:35 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA02263; Wed, 21 May 1997 22:06:34 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA14276 for mpi-core-out; Wed, 21 May 1997 21:12:09 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA11420 for ; Wed, 21 May 1997 17:08:43 -0500 From: squyres@cse.nd.edu Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id RAA02507; Wed, 21 May 1997 17:08:40 -0500 (EST) Received: (from jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) id RAA29507; Wed, 21 May 1997 17:08:39 -0500 (EST) Date: Wed, 21 May 1997 17:08:39 -0500 (EST) Message-Id: <199705212208.RAA29507@porthos.lsc.nd.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Marc Snir" Cc: mpi-core@mcs.anl.gov Subject: error handling In-Reply-To: <8525649E.0065DB54.00@watngi01.watson.ibm.com> References: <8525649E.0065DB54.00@watngi01.watson.ibm.com> X-Mailer: VM 6.31 under Emacs 19.34.1 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In our last exciting episode, Marc Snir wrote: > A) [munched] Yuk. This option is not attractive because it has too many new types. > B) 3/1: 3 different types of callback functions, but only one type > of opaque error handler object. In C++ the three typedefs for the > callback functions are in the namespace of the corresponding class. C would have 3 typedefs as well -- that's not special for C++. > We triplicate all functions that deall with callbacks, or with > communicators, windows of files; namely all error handler functions, > except MPI_Errhandler_free. [[There is no point in triplicating > this last function, if there is only one MPI_Errhandler object type > -- there is no gain in layerability.]] In terms of implementation, > this means that the opaque error handler object registers which kind > of callback function is attached to it. We have a run-time type > matching, rather than compile-time matching: MPI can check, when > MPI_XXX_ERRHANDLER_SET(handle, errhandler) is invoked, that the > callback function attached to errhandler is one that takes handles > to XXX. But not very elegant. :-) > C) 1/1: one type of callback function, one type of opaque object. > The typedef of the callback function becomes, in C, > errhandler_fn(void * handleptr, void * extra_state,...), where > handleptr can be, in one version, the MPI handle coerced to void* > or, in another version, a pointer to the MPI handle coerced to Here's a breakdown of the two versions: 1) (MPI_Handle) cast to (void *): This works for implementations that implement handles as integers or pointers. This does not work for implementations that implement handles as structures (such as the C++ bindings -- a class is essentially a struct). 2) (MPI_Handle *) cast to (void *): This works for everyone. So I vote for #2. :-) > void*. We still triplicate the functions that associate error > handlers with objects (MPI_XXX_ERRHANDLER_SET, > MPI_XXX_ERRHANDLER_GET), but we do not need to replicate > MPI_Errhandler_create and MPI_Errhandler_free. We get, with this > proposal, a lower function count. We get layerability and > extensibility: if an implementer adds a new FOO opaque object (e.g., > FILE), then it provides MPI_FOO_ERRHANDLER_GET and > MPI_FOO_ERRHANDLER_SET. The main reason that I want this is so that implementations can utilize some code reuse and provide a bit more elegant solution (especially in C++, whether your C++ implementation is a layer on top of C or not). There is one minor drawback (see below), but I think that it is a small tradeoff for the benefit for the implementor. See my original post for all the nitty gritty details, along with some corrections by Jean-Pierre: http://parallel.nas.nasa.gov/MPI-2/mpi-core/0634.html http://parallel.nas.nasa.gov/MPI-2/mpi-core/0642.html > There is no way for MPI to do type matching. It's up to the user to > invoke MPI_XXX_ERRHANDLER_SET(handle, errhanlder) with an errhandler > object with an attached callback function that takes arguments of > type XXX. Which, I think, is a reasonable request. After all, why would you invoke a File errhandler on a Comm? {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Wed May 21 22:07:50 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA02267; Wed, 21 May 1997 22:07:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA14311 for mpi-core-out; Wed, 21 May 1997 21:13:24 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA12087; Wed, 21 May 1997 17:47:03 -0500 From: squyres@cse.nd.edu Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id RAA02638; Wed, 21 May 1997 17:46:59 -0500 (EST) Received: (from jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) id RAA29912; Wed, 21 May 1997 17:46:58 -0500 (EST) Date: Wed, 21 May 1997 17:46:58 -0500 (EST) Message-Id: <199705212246.RAA29912@porthos.lsc.nd.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: C++ typedef convention X-Mailer: VM 6.31 under Emacs 19.34.1 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Throughout the MPI-2 document, we used a particular notation for C++ function pointer typedefs. Here's an example (Section 7.2, page 129): --- typedef int MPI::Grequest::Query_function(void* extra_state, MPI::Status& status) --- This is technically not correct, as this is not a valid C++ statement. The correct statement would be: --- namespace MPI { class Request { // ... }; class Grequest : public Request { // ... typedef Query_function(void* extra_state, MPI::Status& status); }; }; --- which is too bulky to put for all the C++ function pointer typedefs in MPI-2. I would like to propose that we put a statement in the terms chapter (I'll write it) that explains our current use of the typdef for C++ function pointers, and how it is shorthand for what the declarations really should be. Any objections? {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Thu May 22 04:56:14 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA11572; Thu, 22 May 1997 04:56:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA17319 for mpi-core-out; Thu, 22 May 1997 04:00:23 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA17314 for ; Thu, 22 May 1997 04:00:18 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id LAA24133 (8.6.13/IDA-1.6); Thu, 22 May 1997 11:00:10 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id LAA14670; Thu, 22 May 1997 11:00:10 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199705220900.LAA14670@awsrr.rus.uni-stuttgart.de> Subject: Re: chapter 5 To: snir@watson.ibm.com (Marc Snir) Date: Thu, 22 May 1997 11:00:09 +0200 (DST) Cc: mpi-core@mcs.anl.gov In-Reply-To: <8525649E.0054E19C.00@watngi01.watson.ibm.com> from "Marc Snir" at May 21, 97 11:33:09 am Content-Type: text Content-Length: 3383 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc wrote: > 2. Change in Fortran binding for MPI_FREE_MEM. Two people objected to it, I withdraw my objection. > and nobody expressed support for it. Thus, unless I hear soon support for > the change, I shall revert to the previous binding, which is what we voted > officially on. I don't think that the issue is very important, although I > would have prefered more consistency between C and Fortran. I support explicitly Marc's change. It solves the major inconsistency. But please also remove the minor inconsistencies noted below. (The are in both drafts, voted April 17 and current May 8.) Lyndon (?) has shown to me, that we use the "base"-argument in three routines: MPI_ALLOC_MEM, MPI_FREE_MEM and MPI_WIN_CREATE, but with two different levels of dereferencing. My changes are changes to the draft May 8. - All name the argument "base". This is a inconsistency. I propose on 65:19-31: use "base_pointer" in MPI_ALLOC_MEM - The description is different: MPI_ALLOC_MEM: initial address of memory allocated (address) MPI_FREE_MEM: initial address of memory segment allocated by MPI_ALLOC_MEM [April 17] MPI_ALLOC_MEM (choice) [May 8] MPI_WIN_CREATE: initial address of window (choice) This is a inconsistency. I propose the following change on 65:23 in MPI_ALLOC_MEM: OUT base_pointer pointer to the initial address of memory allocated for usage in MPI_WIN_CREATE (address) - The C-binding is always "void *base" and the text in both drafts say: In the C binding, the base arguments of the calls MPI_Alloc_mem and MPI_Free_mem are not the same variable: a call to MPI_Alloc_mem(&base) should be paired with a call to MPI_Free_mem (base) (one less dereference). Both arguments are declared to be of the same type void*, so as to facilitate type casting. There is no additional text about the usage of base in WIN_CREATE, but the notation of "(choice)" in the descriptions of MPI_FREE_MEM and MPI_WIN_CREATE is a hint, that MPI_Win_create(base) would be correct. I propose to change only the first sentences (reflecting "base_pointer") into (66:6-7): In the C binding, the base and base_pointer arguments of the calls MPI_Alloc_mem and MPI_Free_mem are not the same variable: - The Fortran binding reflects the C-binding in the routines MPI_ALLOC_MEM and MPI_WIN_CREATE both drafts. In MPI_FREE_MEM the draft April 17 had an inconsistency between Fortran and C. This is solved with marc's changes. It has now the same interface as MPI_WIN_CREATE! - But please can you add also a example written in C (same content as Example 5.1). (Sorry about not writing it, but I'm to bad in writing C) - And please can you add also the MPI_WIN_CREATE with all arguments and MPI_FREE_MEM to the C and Fortran version of Exa.5.1 - If there is someone who can write Exa.5.1 in C++ this would be great. Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Thu May 22 10:27:18 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA20745; Thu, 22 May 1997 10:27:17 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA21034 for mpi-core-out; Thu, 22 May 1997 09:27:19 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA20927; Thu, 22 May 1997 09:22:25 -0500 Message-Id: <199705221422.JAA20927@antares.mcs.anl.gov> To: squyres@cse.nd.edu cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: C++ typedef convention In-Reply-to: Message from squyres@cse.nd.edu of "Wed, 21 May 1997 17:46:58 -0500." <199705212246.RAA29912@porthos.lsc.nd.edu> Date: Thu, 22 May 1997 09:22:23 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | I would like to propose that we put a statement in the terms | chapter (I'll write it) that explains our current use of the typdef | for C++ function pointers, and how it is shorthand for what the | declarations really should be. | | Any objections? I think that Jeff's solution is a good one and I am for it. Bill From mpi-core-human@mcs.anl.gov Thu May 22 10:35:04 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA21115; Thu, 22 May 1997 10:35:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA21245 for mpi-core-out; Thu, 22 May 1997 09:35:03 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA21238; Thu, 22 May 1997 09:34:57 -0500 Message-Id: <199705221434.JAA21238@antares.mcs.anl.gov> To: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) cc: snir@watson.ibm.com (Marc Snir), mpi-core@mcs.anl.gov Subject: Re: chapter 5 In-Reply-to: Message from Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) of "Thu, 22 May 1997 11:00:09 +0200." <199705220900.LAA14670@awsrr.rus.uni-stuttgart.de> Date: Thu, 22 May 1997 09:34:54 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | - But please can you add also a example written in C (same content | as Example 5.1). | I'm not sure that it is worth it, but it is simply (with the Win_create's added): float *p; MPI_Win win; MPI_Alloc_mem( sizeof(float)*100*100, MPI_INFO_NULL, &p ); ... MPI_Win_create( p, sizeof(float)*100*100, sizeof(float), MPI_INFO_NULL, MPI_COMM_WORLD, &win ); ... MPI_Free_mem( p ); In C++, it would be: float *p; p = MPI::Alloc_mem( sizeof(float)*100*100, MPI::INFO_NULL ); ... win = MPI::Win_create( p, sizeof(float)*100*100, sizeof(float), MPI::INFO_NULL, MPI::COMM_WORLD ); ... MPI::Free_mem( p ); Note that in both cases, we are using "void *" where "void **" has the appropriate intent because of a "feature" in C/C++ that makes "void *" friendlier for users. Bill From mpi-core-human@mcs.anl.gov Thu May 22 11:25:37 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA21871; Thu, 22 May 1997 11:25:36 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA20913 for mpi-core-out; Thu, 22 May 1997 09:21:05 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA20906; Thu, 22 May 1997 09:21:00 -0500 Message-Id: <199705221421.JAA20906@antares.mcs.anl.gov> To: squyres@cse.nd.edu cc: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: Re: 1.2 C++ function names In-Reply-to: Message from squyres@cse.nd.edu of "Wed, 21 May 1997 15:28:48 -0500." <199705212028.PAA29273@porthos.lsc.nd.edu> Date: Thu, 22 May 1997 09:20:57 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | Any objections? (I'm volunteering free work here, who in their right | mind would turn it down? :-) I agree that this is a good idea (after all, I just added the constant index), and I am happy to let Jeff do it. Bill From mpi-core-human@mcs.anl.gov Thu May 22 11:52:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA27223; Thu, 22 May 1997 11:52:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA24801 for mpi-core-out; Thu, 22 May 1997 10:54:40 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA24796; Thu, 22 May 1997 10:54:37 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id KAA04459; Thu, 22 May 1997 10:54:34 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id KAA04814; Thu, 22 May 1997 10:54:33 -0500 (EST) Date: Thu, 22 May 1997 10:54:33 -0500 (EST) From: Jeff Squyres To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: C++ function name cross reference Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk For this C++ function name cross reference, where do you think it should go? Should it be its own Annex, or should it be the last section of Annex B? I'd almost prefer that it be it's own annex, but I suppose that arguments could be made for either. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Thu May 22 11:53:17 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA27396; Thu, 22 May 1997 11:53:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA24859 for mpi-core-out; Thu, 22 May 1997 10:56:23 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA24843; Thu, 22 May 1997 10:56:12 -0500 Message-Id: <199705221556.KAA24843@antares.mcs.anl.gov> To: Jeff Squyres cc: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: C++ function name cross reference In-Reply-to: Message from Jeff Squyres of "Thu, 22 May 1997 10:54:33 -0500." Date: Thu, 22 May 1997 10:56:08 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | For this C++ function name cross reference, where do you think it should | go? Should it be its own Annex, or should it be the last section of Annex | B? Last section of Annex B. Bill From mpi-core-human@mcs.anl.gov Thu May 22 17:27:17 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA12657; Thu, 22 May 1997 17:27:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA03265 for mpi-core-out; Thu, 22 May 1997 16:28:54 -0500 Received: from k2.llnl.gov (johnmay@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA03260 for ; Thu, 22 May 1997 16:28:51 -0500 Received: (from johnmay@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id OAA13238; Thu, 22 May 1997 14:28:48 -0700 (PDT) Message-ID: <19970522142848.HI03348@k2.llnl.gov> Date: Thu, 22 May 1997 14:28:48 -0700 From: johnmay@llnl.gov (John May) To: mpi-core@mcs.anl.gov Subject: Re: error handling References: <8525649E.0065DB54.00@watngi01.watson.ibm.com> X-Mailer: Mutt 0.60e Mime-Version: 1.0 In-Reply-To: <8525649E.0065DB54.00@watngi01.watson.ibm.com>; from Marc Snir on May 21, 1997 14:33:30 -0400 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc Snir writes: [(A) means full elaboration of error handlers and function types for Comms, Files, and Wins. (B) means full elaboration of function types but a single error handler type that keeps track of what kind of handle it can be attached to. (C) means a single error handler function type that takes a void * for a handle pointer and leaves it to the user to associate the handler with the right kind of handle. ] > I think we agreed on (B), but there are still some inconsistencies among > chapters (e.g., do we triplicate MPI_Errhandler_free), and there is still a > proposal from Squires to go to C. I am not sure that we discussed the > issue with a clear view of the alternatives and their pros and cons. > Hence, this mesage. A pro and a con of C: Con: since no type checking is possible even at runtime, and since all error handlers look alike, there is a danger that the user will, for example, associate the same error handler with all three kinds of object. This won't be detected until an actual error occurs and the handler is invoked, at which time the program will probably crash. This is just what the user wanted to avoid. Using option A or B allows the user to discover the sooner. Pro: perhaps the user *wants* to write one error handler for all three handles. If the handler ignores the handle passed in, it can serve all three types of handle. On balance, I favor B slightly. John From mpi-core-human@mcs.anl.gov Fri May 23 05:55:57 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id FAA00740; Fri, 23 May 1997 05:55:50 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id AAA08633 for mpi-core-out; Fri, 23 May 1997 00:30:35 -0500 Received: from win153.nas.nasa.gov (win153.nas.nasa.gov [129.99.33.68]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id AAA08626 for ; Fri, 23 May 1997 00:30:16 -0500 Received: from win153.nas.nasa.gov (nitzberg@localhost) by win153.nas.nasa.gov (8.7.6/NAS.6.1) with ESMTP id WAA26234 for ; Thu, 22 May 1997 22:30:13 -0700 (PDT) Message-Id: <199705230530.WAA26234@win153.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: Updated I/O Consistency & Semantics Section Date: Thu, 22 May 1997 22:30:12 -0700 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk After considerable discussion, the "Consistency and Semantics" section (10.6.1) of the I/O chapter has been re-written for clarity, and examples (10.6.10) have been added. Since a significant number of small changes have been made, I am sending out a new version of the section to the whole forum. Below is a postscript version of the new text. Comments, as always, are solicited. Enjoy, - bill ----- cut here ----- io-consistency.ps ----- cut here ----- %!PS-Adobe-2.0 %%Creator: dvipsk 5.58f Copyright 1986, 1994 Radical Eye Software %%Title: temp.dvi %%Pages: 10 %%PageOrder: Ascend %%BoundingBox: 0 0 612 792 %%EndComments %DVIPSCommandLine: /usr/local/bin/dvips -o temp.ps temp %DVIPSParameters: dpi=300, comments removed %DVIPSSource: TeX output 1997.05.22:2218 %%BeginProcSet: tex.pro /TeXDict 250 dict def TeXDict begin /N{def}def /B{bind def}N /S{exch}N /X{S N}B /TR{translate}N /isls false N /vsize 11 72 mul N /hsize 8.5 72 mul N /landplus90{false}def /@rigin{isls{[0 landplus90{1 -1}{-1 1} ifelse 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[matrix currentmatrix{dup dup round sub abs 0.00001 lt{round}if} forall round exch round exch]setmatrix}N /@landscape{/isls true N}B /@manualfeed{statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{ /nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{ /sf 1 N /fntrx FMat N df-tail}B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0] N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{ 128 ch-data dup length 3 sub get sub}B /ch-yoff{ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 sub]{ch-image}imagemask restore}B /D{/cc X dup type /stringtype ne{]} if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{ cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto /V matrix currentmatrix dup 1 get dup mul exch 0 get dup mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N /eop{SI restore userdict /eop-hook known{eop-hook}if showpage}N /@start{userdict /start-hook known{start-hook}if pop /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for 65781.76 div /vsize X 65781.76 div /hsize X}N /p{show}N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V {}B /RV statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval dup(Display)eq exch 0 4 getinterval(NeXT)eq or}{pop false} ifelse}{false}ifelse end{{gsave TR -.1 .1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 .1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /QV{gsave newpath transform round exch round exch itransform moveto rulex 0 rlineto 0 ruley neg rlineto rulex neg 0 rlineto fill grestore}B /a{moveto}B /delta 0 N /tail {dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{S p tail}B /c{-4 M} B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{ 4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w}B /q{ p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{SS restore}B end %%EndProcSet %%BeginProcSet: special.pro TeXDict begin /SDict 200 dict N SDict begin /@SpecialDefaults{/hs 612 N /vs 792 N /ho 0 N /vo 0 N /hsc 1 N /vsc 1 N /ang 0 N /CLIP 0 N /rwiSeen false N /rhiSeen false N /letter{}N /note{}N /a4{}N /legal{}N}B /@scaleunit 100 N /@hscale{@scaleunit div /hsc X}B /@vscale{@scaleunit div /vsc X}B /@hsize{/hs X /CLIP 1 N}B /@vsize{/vs X /CLIP 1 N}B /@clip{ /CLIP 2 N}B /@hoffset{/ho X}B /@voffset{/vo X}B /@angle{/ang X}B /@rwi{ 10 div /rwi X /rwiSeen true N}B /@rhi{10 div /rhi X /rhiSeen true N}B /@llx{/llx X}B /@lly{/lly X}B /@urx{/urx X}B /@ury{/ury X}B /magscale true def end /@MacSetUp{userdict /md known{userdict /md get type /dicttype eq{userdict begin md length 10 add md maxlength ge{/md md dup length 20 add dict copy def}if end md begin /letter{}N /note{}N /legal{} N /od{txpose 1 0 mtx defaultmatrix dtransform S atan/pa X newpath clippath mark{transform{itransform moveto}}{transform{itransform lineto} }{6 -2 roll transform 6 -2 roll transform 6 -2 roll transform{ itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll curveto}}{{ closepath}}pathforall newpath counttomark array astore /gc xdf pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{PaintBlack}if}N /txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 -1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop 90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr 1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr 2 get ppr 0 get neg sub neg 0 S TR}if}ifelse scaleby96{ppr aload pop 4 -1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S TR}if}N /cp {pop pop showpage pm restore}N end}if}if}N /normalscale{Resolution 72 div VResolution 72 div neg scale magscale{DVImag dup scale}if 0 setgray} N /psfts{S 65781.76 div N}N /startTexFig{/psf$SavedState save N userdict maxlength dict begin /magscale true def normalscale currentpoint TR /psf$ury psfts /psf$urx psfts /psf$lly psfts /psf$llx psfts /psf$y psfts /psf$x psfts currentpoint /psf$cy X /psf$cx X /psf$sx psf$x psf$urx psf$llx sub div N /psf$sy psf$y psf$ury psf$lly sub div N psf$sx psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub TR /showpage{}N /erasepage{}N /copypage{}N /p 3 def @MacSetUp}N /doclip{ psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto closepath clip newpath moveto}N /endTexFig{end psf$SavedState restore}N /@beginspecial{SDict begin /SpecialSave save N gsave normalscale currentpoint TR @SpecialDefaults count /ocount X /dcount countdictstack N}N /@setspecial {CLIP 1 eq{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs neg 0 rlineto closepath clip}if ho vo TR hsc vsc scale ang rotate rwiSeen{rwi urx llx sub div rhiSeen{rhi ury lly sub div}{dup}ifelse scale llx neg lly neg TR }{rhiSeen{rhi ury lly sub div dup scale llx neg lly neg TR}if}ifelse CLIP 2 eq{newpath llx lly moveto urx lly lineto urx ury lineto llx ury lineto closepath clip}if /showpage{}N /erasepage{}N /copypage{}N newpath }N /@endspecial{count ocount sub{pop}repeat countdictstack dcount sub{ end}repeat grestore SpecialSave restore end}N /@defspecial{SDict begin} N /@fedspecial{end}B /li{lineto}B /rl{rlineto}B /rc{rcurveto}B /np{ /SaveX currentpoint /SaveY X N 1 setlinecap newpath}N /st{stroke SaveX SaveY moveto}N /fil{fill SaveX SaveY moveto}N /ellipse{/endangle X /startangle X /yrad X /xrad X /savematrix matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc savematrix setmatrix}N end %%EndProcSet TeXDict begin 40258437 52099154 1000 300 300 (temp.dvi) @start /Fa 1 82 df81 D E /Fb 1 1 df0 D E /Fc 9 116 df<071018F0307060706060C060C060C06080C080C480C4C1C446C838 700E0E7E8D13>97 D<7C0018001800180018003000300030003000678068C070406060C0 60C060C060C06080C080C08180C10046003C000B177E960F>I<003E000C000C000C000C 0018001800180018073018F0307060706060C060C060C06080C080C480C4C1C446C83870 0F177E9612>100 D<000E0013003700360060006000600060006007FC00C000C000C000 C000C001800180018001800180018003000300030003006600E600CC007800101D7E9611 >102 D<1F0006000600060006000C000C000C000C0018F01B181C081808381830183018 30306030603160616062C022C03C10177E9614>104 D<03000380030000000000000000 00000000001C002400460046008C000C0018001800180031003100320032001C0009177F 960C>I<383C1E0044C6630047028100460301008E0703000C0603000C0603000C060600 180C0600180C0620180C0C20180C0C4030180440301807801B0E7F8D1F>109 D<383C0044C6004702004602008E06000C06000C06000C0C00180C00180C401818401818 80300880300F00120E7F8D15>I<07C00C201870187038001E000FC003E000606060E060 C0C0C1803F000C0E7E8D10>115 D E /Fd 4 62 df<07C018303018701C600C600CE00E E00EE00EE00EE00EE00EE00EE00EE00E600C600C701C30181C7007C00F157F9412>48 D<03000700FF000700070007000700070007000700070007000700070007000700070007 00070007007FF00C157E9412>I<0F8030E040708030C038E03840380038007000700060 00C00180030006000C08080810183FF07FF0FFF00D157E9412>I61 D E /Fe 32 123 df<70F8F8F87005057C840D>58 D<70F8FCFC74040404080810102040 060E7C840D>I<000002000000060000000E0000000E0000001E0000001F0000002F0000 002F0000004F0000008F0000008F0000010F0000010F0000020F0000040F0000040F0000 080F8000080780001007800020078000200780007FFF8000400780008007800180078001 00078002000780020007C0040003C00C0003C01E0007C0FF807FFC1E207E9F22>65 D<00FFFFE000000F007800000F001C00000F000E00000F000700001E000700001E000380 001E000380001E000380003C000380003C000380003C000380003C000380007800078000 780007800078000780007800078000F0000F0000F0000F0000F0000E0000F0001E0001E0 001C0001E0003C0001E000380001E000700003C000E00003C001C00003C003800003C007 000007803C0000FFFFF00000211F7E9E26>68 D<00FFFFFF000F000E000F0006000F0002 000F0002001E0002001E0002001E0002001E0002003C0404003C0400003C0400003C0C00 00781800007FF800007818000078180000F0100000F0100000F0100000F0000401E00008 01E0000801E0001001E0001003C0002003C0006003C0004003C001C0078007C0FFFFFF80 201F7E9E22>I<00FFFFFF000F000E000F0006000F0002000F0002001E0002001E000200 1E0002001E0002003C0004003C0400003C0400003C04000078080000781800007FF80000 78180000F0100000F0100000F0100000F0100001E0000001E0000001E0000001E0000003 C0000003C0000003C0000003C0000007C00000FFFE0000201F7E9E1D>I<00FFF9FFF000 0F801F00000F001E00000F001E00000F001E00001E003C00001E003C00001E003C00001E 003C00003C007800003C007800003C007800003C007800007800F000007FFFF000007800 F000007800F00000F001E00000F001E00000F001E00000F001E00001E003C00001E003C0 0001E003C00001E003C00003C007800003C007800003C007800003C007800007C00F8000 FFF8FFF800241F7E9E26>72 D<0001FC0000070700001C01C0003000E000E0006001C000 700380007007800038070000380E0000381E0000381C0000383C0000383C000038780000 78780000787800007878000078F00000F0F00000F0F00000E0F00001E0F00001C0F00003 C0700003807000070078000F0038001E0038003C001C0070000E00E0000783800001FC00 001D217E9F23>79 D<0001FC0000070700001C01C0003000E000E000E001C00070038000 7007800078070000380F0000381E0000381E0000383C0000383C00007878000078780000 787800007878000078F00000F0F00000F0F00000E0F00001E0F00001C0F00003C0700003 80701C070070600F0038811E0038813C001C8170000E81E0000783808001FD0080000101 800001010000038300000386000003FE000003FC000001F8000000F0001D297E9F24>81 D<0007E0800018118000300B000060070000C0070001C003000180020003800200038002 0003800200038000000380000003C0000003F8000003FF800001FFC00000FFE000003FF0 000003F0000000F0000000700000007000000070002000700020007000200060006000E0 006000C0006001C00070018000E8030000C60E000081F8000019217D9F1C>83 D<00FFF83FF8000FC00F80000F80060000078004000007C008000003C010000003C02000 0003E040000001E080000001F100000000F300000000F600000000FC0000000078000000 007C000000007C000000007C00000000BE000000011E000000021E000000061F0000000C 0F000000080F800000100780000020078000004007C000008003C000010003E000030003 E0000F0007E000FFE01FFE00251F7F9E26>88 D<00F1800389C00707800E03801C03803C 0380380700780700780700780700F00E00F00E00F00E00F00E10F01C20F01C20703C2070 5C40308C400F078014147E9318>97 D<07803F8007000700070007000E000E000E000E00 1C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE01EE01CE03CE038 607060E031C01F0010207E9F14>I<007C01C207010E0F1E0F1C0E3C04780078007800F0 00F000F000F000F00070017002300418380FC010147E9314>I<0000780003F800007000 00700000700000700000E00000E00000E00000E00001C00001C000F1C00389C00707800E 03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E10F01C20F0 1C20703C20705C40308C400F078015207E9F18>I<007C01C207010E011C013C01380278 0C7BF07C00F000F000F000F0007000700170023004183807C010147E9315>I<00007C00 00CE00019E00039E00030C000700000700000700000700000E00000E00000E0000FFF000 0E00000E00001C00001C00001C00001C00001C0000380000380000380000380000380000 700000700000700000700000700000E00000E00000E00000E00000C00001C00031800079 8000F300006200003C000017297E9F16>I<01E0000FE00001C00001C00001C00001C000 038000038000038000038000070000070000071F000761800E80C00F00C00E00E00E00E0 1C01C01C01C01C01C01C01C0380380380380380380380704700708700E08700E10700610 E006206003C016207E9F1A>104 D<00E001E001E000C000000000000000000000000000 000E00130023804380438043808700070007000E000E001C001C001C2038403840384038 8019000E000B1F7E9E10>I<03C01FC0038003800380038007000700070007000E000E00 0E000E001C001C001C001C0038003800380038007000700070007100E200E200E200E200 640038000A207E9F0E>108 D<1E07C07C00231861860023A032030043C0340300438038 0380438038038087007007000700700700070070070007007007000E00E00E000E00E00E 000E00E00E000E00E01C101C01C01C201C01C038201C01C038401C01C018403803801880 1801800F0024147E9328>I<1E07802318C023A06043C0704380704380708700E00700E0 0700E00700E00E01C00E01C00E01C00E03821C03841C07041C07081C03083803101801E0 17147E931B>I<007C0001C3000301800E01C01E01C01C01E03C01E07801E07801E07801 E0F003C0F003C0F003C0F00780F00700700F00700E0030180018700007C00013147E9316 >I<03C1E004621804741C08781C08701E08701E10E01E00E01E00E01E00E01E01C03C01 C03C01C03C01C0380380780380700380E003C1C0072380071E000700000700000E00000E 00000E00000E00001C00001C0000FFC000171D819317>I<1E1E0023210023C38043C780 4387804383008700000700000700000700000E00000E00000E00000E00001C00001C0000 1C00001C000038000018000011147E9315>114 D<007C018203010603060706060E0007 8007F803FC01FE001F00077007F006F006E004400820301FC010147E9315>I<00C000E0 01C001C001C001C003800380FFF8038007000700070007000E000E000E000E001C001C00 1C001C10382038203820384018800F000D1C7F9B10>I<0F00601180702180E021C0E041 C0E04380E08381C00701C00701C00701C00E03800E03800E03800E03840E07080C07080C 07080E0F1006131003E1E016147E931A>I<0F006060118070F02180E0F821C0E07841C0 E0384380E0188381C0100701C0100701C0100701C0100E0380200E0380200E0380200E03 80400E0380400E0380800E078080060781000709860001F078001D147E9321>119 D<03C1C00C62201034701038F02038F020386040700000700000700000700000E00000E0 0000E00000E02061C040F1C040F1C080E2C080446300383C0014147E931A>I<0F006011 80702180E021C0E041C0E04380E08381C00701C00701C00701C00E03800E03800E03800E 03800E07000C07000C07000E0F00061E0003EE00000E00000E00001C0078180078380070 700060600021C0001F0000141D7E9316>I<01E02003F04007F8C00C1F80080100000200 00040000080000100000600000C0000100000200000400800801001003003F060061FC00 40F80080700013147E9315>I E /Ff 55 123 df<00FC1F8001FC3F8003FC7F800700E0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C000FFE1FC00FFE1FC 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C000191D809C18>11 D<0001C001F1C003F1C007F0000700000E00000E00000E00000E00000E00000E00000E00 00FFF1C0FFF1C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C0121E809D16>I<03C00FF01FF83C3C381C 700E700E700EE007E007E007E007E007E007E007E007E007E007E007E007E007700E700E 700E381C3C3C1FF80FF007E0101D7E9B15>48 D<010007003F00FF00C700070007000700 070007000700070007000700070007000700070007000700070007000700070007000700 FFF8FFF80D1C7C9B15>I<07C01FF03878701C601EE00EC00F400F400700070007000F00 0E001E001C003C007800F001E001C0038007000E001C0038007000FFFFFFFF101C7E9B15 >I<003C00007C00005C0000DC0001DC00019C00039C00039C00071C00071C000E1C000E 1C001C1C003C1C00381C00781C00701C00F01C00FFFFC0FFFFC0001C00001C00001C0000 1C00001C00001C00001C00121B7F9A15>52 D<7FFE7FFE70007000700070007000700070 0073E07FF07FF87E3C7C1E781E780F000F000F000F000F000F000F401EC01E603C38781F F00FC0101C7E9A15>I<001C0000003E0000003E0000002E0000006700000067000000E7 800000C7800000C3800001C3C0000183C0000181C0000381E0000381E0000700F0000700 F0000600F0000E0078000FFFF8000FFFF8001C003C001C003C0018003C0038001E003800 1E0070001F0070000F0070000F00E0000780191D7F9C1C>65 DI<003FC000FFF003C0F00780300F00001E00003C00003C0000 780000780000780000F00000F00000F00000F00000F00000F00000F00000F00000F00000 7800007800007800003C00003C00001E00000F000807801803C07800FFF0003F80151F7D 9D1B>IIII<003F 8001FFF003C0F80780380F00181E00003C00003C0000780000780000780000F00000F000 00F00000F00000F00000F00000F007F8F007F8F000387800387800387800383C00383C00 381E00380F003807803803C0F801FFF0003F80151F7D9D1C>III75 DIII<003F000001 FFE00003FFF00007C0F8000F807C001E001E003E001F003C000F00780007807800078078 000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F8 0007C078000780780007807C000F803C000F003E001F001F003E000F807C0007C0F80003 FFF00001FFE000003F00001A1F7E9D1F>II<003F000001FFE00003FFF00007C0F8000F807C001F003E003E001F003C000F00 780007807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0 F00003C0F00003C0F00003C07800078078000780780E07803C0F0F003E079F001E03DE00 0F83FC0007C1F80003FFF00001FFF800003F780000003C0000003E0000001F0000000F80 1A237E9D1F>II<03F8000FFE001C 0F00380700700300600000E00000E00000E00000E00000F000007800007F00003FE0001F FC0007FE0001FF00001F800007800003C00003C00001C00001C00001C00001C0C00180E0 0380F007007C0E001FFC0007F000121F7E9D17>II< F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070 F00070F00070F00070F00070F00070F00070F00070F00070F00070F00070F000707800E0 7800E03C01C01E03800F078007FE0001F800141E7C9C1D>I87 D<78000E007C001E003C003C001E0038000F0070000F00F0000781E00003C1C00001C3C0 0001E7800000F70000007E0000003E0000003C0000003C0000007E00000077000000E780 0001E3800003C1C0000381E0000700F0000F00F8000E0078001C003C003C003E0078001F 0070000F00F0000F80191D7F9C1C>II<0FC03FF07F F87038401C001C001C00FC0FFC3FFC781CE01CE01CE01CF07C7FFC7FDC3F1C0E127E9114 >97 DI<07E00FF8 1FFC3C1C70047000E000E000E000E000E000E000700070043C1C1FFC0FF807E00E127E91 12>I<000E000E000E000E000E000E000E000E000E000E000E0F8E1FEE3FFE7C3E700E70 0EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E0F1D7E9C15>I<07C01FE0 3FF078787018601CFFFCFFFCFFFCE000E000E000700070043C1C3FFC1FF807E00E127E91 12>I<00FC01FC03FC07000E000E000E000E000E000E000E00FFE0FFE00E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E1D809C0D>I<03C3C00F FFC01FFFC01C3800381C00381C00381C00381C00381C001C38001FF8001FF0003BC00038 00003800001FFC001FFF003FFF80700780E001C0E001C0E001C0F003C07C0F803FFF001F FE0007F800121B7F9115>III<03C003C003C003C0000000000000000000000000000001C001C001C001C001 C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C0C3 C0FF80FF003E000A25839C0B>IIIII<03F0000FFC001FFE003C0F00780780 700380E001C0E001C0E001C0E001C0E001C0F003C07003807807803C0F001FFE000FFC00 03F00012127F9115>II<078E1F EE3FFE7C3E781E700EE00EE00EE00EE00EE00EE00EF00E701E7C3E3FFE1FEE0F8E000E00 0E000E000E000E000E000E000E0F1A7E9115>II<1FC03FF07FF0F030E0 00E000F0007F003FC01FE000F0003800388038F078FFF07FE01FC00D127F9110>I<1C00 1C001C001C001C001C00FFE0FFE01C001C001C001C001C001C001C001C001C001C001C00 1C001C201FF00FF007C00C187F970F>IIII121 D<7FFC7FFC7FFC007800 F000E001E003C0038007000F001E001C003C007800FFFCFFFCFFFC0E127F9112>I E /Fg 89 126 df<70F8F8F8F8F8F8F8F8F8F8F8F8F8F8F8F870000000000070F8F8F870 051C779B18>33 D<4010E038F078E038E038E038E038E038E038E038E038E038E0386030 0D0E7B9C18>I<030600078F00078F00078F00078F00078F00078F007FFFC0FFFFE0FFFF E07FFFC00F1E000F1E000F1E000F1E000F1E000F1E007FFFC0FFFFE0FFFFE07FFFC01E3C 001E3C001E3C001E3C001E3C001E3C000C1800131C7E9B18>I<00C00001C00001C00001 C00003F0000FFC003FFE007DCF0071C700E1C380E1C780E1C780E1C780F1C00079C0003D C0001FE0000FF80003FC0001DE0001CF0001C70061C380F1C380F1C380E1C380E1C70071 C70079DE003FFE001FF80007E00001C00001C00001C00000C00011247D9F18>I<380300 7C07807C0780EE0F80EE0F00EE0F00EE1F00EE1E00EE1E00EE3E007C3C007C3C00387C00 00780000780000F80000F00001F00001E00001E00003E00003C00003C00007C000078380 0787C00F87C00F0EE00F0EE01F0EE01E0EE01E0EE03E0EE03C07C03C07C018038013247E 9F18>I<01C00007E0000FF0000E70001C38001C38001C38001C38001C73F01C73F01CE3 F00FE3800FC7000F87000F07001F0E003F0E007B8E0073DC00E1DC00E0F800E0F800E070 70E0787070FC707FFFE03FCFE00F03C0141C7F9B18>I<387C7C7E3E0E0E0E1C1C38F8F0 C0070E789B18>I<007000F001E003C007800F001E001C00380038007000700070007000 E000E000E000E000E000E000E000E0007000700070007000380038001C001E000F000780 03C001F000F000700C24799F18>I<6000F00078003C001E000F000780038001C001C000 E000E000E000E00070007000700070007000700070007000E000E000E000E001C001C003 8007800F001E003C007800F00060000C247C9F18>I<01C00001C00001C00001C000C1C1 80F1C780F9CF807FFF001FFC0007F00007F0001FFC007FFF00F9CF80F1C780C1C18001C0 0001C00001C00001C00011147D9718>I<00600000F00000F00000F00000F00000F00000 F00000F0007FFFC0FFFFE0FFFFE07FFFC000F00000F00000F00000F00000F00000F00000 F00000600013147E9718>I<1C3E7E7F3F1F070E1E7CF860080C788518>I<7FFF00FFFF80 FFFF807FFF0011047D8F18>I<3078FCFC78300606778518>I<000300000780000780000F 80000F00001F00001E00001E00003E00003C00007C0000780000780000F80000F00001F0 0001E00003E00003C00003C00007C0000780000F80000F00000F00001F00001E00003E00 003C00003C00007C0000780000F80000F00000F0000060000011247D9F18>I<01F00007 FC000FFE001F1F001C07003803807803C07001C07001C0E000E0E000E0E000E0E000E0E0 00E0E000E0E000E0E000E0E000E0F001E07001C07001C07803C03803801C07001F1F000F FE0007FC0001F000131C7E9B18>I<01800380038007800F803F80FF80FB804380038003 80038003800380038003800380038003800380038003800380038003807FFCFFFE7FFC0F 1C7B9B18>I<03F0000FFE003FFF007C0F807003C0E001C0F000E0F000E06000E00000E0 0000E00001C00001C00003C0000780000F00001E00003C0000780000F00001E00007C000 0F80001E00E03C00E07FFFE0FFFFE07FFFE0131C7E9B18>I<07F8001FFE003FFF007807 807803C07801C03001C00001C00003C0000380000F0003FF0003FE0003FF000007800003 C00001C00000E00000E00000E0F000E0F000E0F001C0F003C07C07803FFF001FFE0003F8 00131C7E9B18>I<001F00003F0000770000770000E70001E70001C70003870007870007 07000E07001E07003C0700380700780700F00700FFFFF8FFFFF8FFFFF800070000070000 0700000700000700000700007FF000FFF8007FF0151C7F9B18>I<1FFF803FFF803FFF80 3800003800003800003800003800003800003800003800003BF8003FFE003FFF003C0780 1803C00001C00000E00000E06000E0F000E0F000E0E001C07003C07C0F803FFF001FFC00 03F000131C7E9B18>I<007E0001FF0007FF800F83C01E03C01C03C03801803800007000 00700000E1F800E7FE00FFFF00FE0780F803C0F001C0F000E0E000E0F000E07000E07000 E07000E03801C03C03C01E07800FFF0007FE0001F800131C7E9B18>I<03F8000FFE001F FF003E0F803803807001C07001C07001C07001C03803803C07801FFF0007FC000FFE001F 1F003C07807001C0F001E0E000E0E000E0E000E0E000E07001C07803C03E0F801FFF000F FE0003F800131C7E9B18>56 D<03F0000FFC001FFE003C0F00780780700380E001C0E001 C0E001C0E001E0E001E07001E07803E03C0FE01FFFE00FFEE003F0E00000E00001C00001 C00001C0300380780780780F00783E003FFC001FF00007C000131C7E9B18>I<3078FCFC 783000000000000000003078FCFC78300614779318>I<183C7E7E3C1800000000000000 00183C7E7E3E1E0E1C3C78F060071A789318>I<000300000780001F80003F00007E0001 FC0003F00007E0001FC0003F00007E0000FC0000FC00007E00003F00001FC00007E00003 F00001FC00007E00003F00001F8000078000030011187D9918>I<7FFFC0FFFFE0FFFFE0 FFFFE0000000000000000000000000FFFFE0FFFFE0FFFFE07FFFC0130C7E9318>I<6000 00F00000FC00007E00003F00001FC00007E00003F00001FC00007E00003F00001F80001F 80003F00007E0001FC0003F00007E0001FC0003F00007E0000FC0000F000006000001118 7D9918>I<0FF0003FFC007FFF00700F00F00380F00380600780000F00003E00007C0001 F00001E00003C00003C00003C00003C00003C00003800000000000000000000000000000 000003800007C00007C00007C000038000111C7D9B18>I<00700000F80000F80000D800 00D80001DC0001DC0001DC00018C00038E00038E00038E00038E00030600070700070700 0707000707000FFF800FFF800FFF800E03800E03801C01C01C01C07F07F0FF8FF87F07F0 151C7F9B18>65 DI<00F8E003FEE007FFE0 0F07E01E03E03C01E03800E07000E07000E0700000E00000E00000E00000E00000E00000 E00000E00000E000007000007000E07000E03800E03C00E01E01C00F07C007FF8003FE00 00F800131C7E9B18>I<7FF800FFFE007FFF001C0F801C03C01C03C01C01E01C00E01C00 E01C00F01C00701C00701C00701C00701C00701C00701C00701C00701C00F01C00E01C00 E01C01E01C01C01C03C01C0F807FFF00FFFE007FF800141C7F9B18>III<01F1C003FD C00FFFC01F0FC01C03C03803C03801C07001C07001C0700000E00000E00000E00000E000 00E00000E00FF0E01FF0E00FF07001C07001C07003C03803C03803C01C07C01F0FC00FFF C003FDC001F1C0141C7E9B18>I<7F07F0FF8FF87F07F01C01C01C01C01C01C01C01C01C 01C01C01C01C01C01C01C01C01C01FFFC01FFFC01FFFC01C01C01C01C01C01C01C01C01C 01C01C01C01C01C01C01C01C01C01C01C07F07F0FF8FF87F07F0151C7F9B18>I<7FFF00 FFFF807FFF0001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 01C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C000 7FFF00FFFF807FFF00111C7D9B18>I<7F07F0FF87F87F07F01C03C01C07801C07001C0E 001C1E001C3C001C38001C70001CF0001DF0001DF0001FB8001FB8001F1C001E1C001C0E 001C0E001C07001C07001C03801C03801C01C07F03F0FF87F87F03F0151C7F9B18>75 D<7FE000FFE0007FE0000E00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E00000E00000E00000E00700E00700E00700E00 700E00707FFFF0FFFFF07FFFF0141C7F9B18>II<7E07F0FF0FF87F07F01D81C01D81C01D81C01DC1C01CC1C01CC1C01CE1C01CE1C0 1CE1C01C61C01C71C01C71C01C31C01C39C01C39C01C39C01C19C01C19C01C1DC01C0DC0 1C0DC01C0DC07F07C0FF87C07F03C0151C7F9B18>I<0FF8003FFE007FFF00780F007007 00F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380E003 80E00380E00380E00380E00380E00380F00780700700780F007FFF003FFE000FF800111C 7D9B18>II<0FF8003FFE007FFF00780F00 700700F00780E00380E00380E00380E00380E00380E00380E00380E00380E00380E00380 E00380E00380E00380E00380E1E380E1E380F0E78070F700787F007FFF003FFE000FFC00 001C00001E00000E00000F0000070000070011227D9B18>I<7FF800FFFE007FFF001C0F 801C03801C03C01C01C01C01C01C01C01C03C01C03801C0F801FFF001FFE001FFE001C0F 001C07001C03801C03801C03801C03801C03801C039C1C039C1C039C7F01F8FF81F87F00 F0161C7F9B18>I<03F3801FFF803FFF807C0F80700780E00380E00380E00380E0000070 00007800003F00001FF00007FE0000FF00000F800003C00001C00000E00000E06000E0E0 00E0E001E0F001C0F80780FFFF80FFFE00E7F800131C7E9B18>I<7FFFF8FFFFF8FFFFF8 E07038E07038E07038E07038007000007000007000007000007000007000007000007000 00700000700000700000700000700000700000700000700000700000700007FF0007FF00 07FF00151C7F9B18>IIII<7F8FE07F9F E07F8FE00E07000F0700070E00078E00039C0003DC0001F80001F80000F00000F0000070 0000F00000F80001F80001DC00039E00038E00070F000707000E07800E03801E03C07F07 F0FF8FF87F07F0151C7F9B18>II<3FFFE0 7FFFE07FFFE07001C07003C0700780700700000F00001E00001C00003C00007800007000 00F00001E00001C00003C0000780000700000F00001E00E01C00E03C00E07800E07000E0 FFFFE0FFFFE0FFFFE0131C7E9B18>II93 D<018007C01FF07EFCF83EE00E0F067C9B18>I<7FFF00FFFF80FFFF807FFF0011047D7F 18>I<061E3E387070E0E0E0F8FC7C7C38070E789E18>I<1FE0003FF8007FFC00781E0030 0E0000070000070000FF0007FF001FFF007F0700780700E00700E00700E00700F00F0078 1F003FFFF01FFBF007E1F014147D9318>I<7E0000FE00007E00000E00000E00000E0000 0E00000E00000E3E000EFF800FFFC00FC1E00F80E00F00700E00700E00380E00380E0038 0E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF80063E00151C809B18> I<01FE0007FF001FFF803E0780380300700000700000E00000E00000E00000E00000E000 00E000007000007001C03801C03E03C01FFF8007FF0001FC0012147D9318>I<001F8000 3F80001F8000038000038000038000038000038003E3800FFB801FFF803C1F80380F8070 0780700380E00380E00380E00380E00380E00380E00380700780700780380F803C1F801F FFF00FFBF803E3F0151C7E9B18>I<01F00007FC001FFE003E0F00380780700380700380 E001C0E001C0FFFFC0FFFFC0FFFFC0E000007000007001C03801C03E03C01FFF8007FF00 01FC0012147D9318>I<001F80007FC000FFE000E1E001C0C001C00001C00001C0007FFF C0FFFFC0FFFFC001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C00001C0007FFF007FFF007FFF00131C7F9B18>I<01E1F007FFF80F FFF81E1E301C0E003807003807003807003807003807001C0E001E1E001FFC001FF80039 E0003800001C00001FFE001FFFC03FFFE07801F0700070E00038E00038E00038E0003878 00F07E03F01FFFC00FFF8001FC00151F7F9318>I<7E0000FE00007E00000E00000E0000 0E00000E00000E00000E3E000EFF800FFFC00FC1C00F80E00F00E00E00E00E00E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07FC3FCFFE7FE7FC3FC171C80 9B18>I<03800007C00007C00007C0000380000000000000000000000000007FC000FFC0 007FC00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C00001C0 0001C00001C00001C000FFFF00FFFF80FFFF00111D7C9C18>I<0038007C007C007C0038 00000000000000000FFC1FFC0FFC001C001C001C001C001C001C001C001C001C001C001C 001C001C001C001C001C001C001C001C001C001C001C6038F078FFF07FE03F800E277E9C 18>II<7FE000FFE0007FE00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E0 0000E00000E00000E00000E00000E00000E00000E00000E0007FFFC0FFFFE07FFFC0131C 7E9B18>I<7CE0E000FFFBF8007FFFF8001F1F1C001E1E1C001E1E1C001C1C1C001C1C1C 001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C001C1C1C 007F1F1F00FFBFBF807F1F1F001914819318>I<7E3E00FEFF807FFFC00FC1C00F80E00F 00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E07F C3FCFFE7FE7FC3FC1714809318>I<01F0000FFE001FFF003E0F803803807001C07001C0 E000E0E000E0E000E0E000E0E000E0F001E07001C07803C03C07803E0F801FFF000FFE00 01F00013147E9318>I<7E3E00FEFF807FFFC00FC1E00F80E00F00700E00700E00380E00 380E00380E00380E00380E00380F00700F00700F80E00FC1E00FFFC00EFF800E3E000E00 000E00000E00000E00000E00000E00000E00007FC000FFE0007FC000151E809318>I<01 E38007FB801FFF803E1F80380F80700780700780E00380E00380E00380E00380E00380E0 0380700780700780380F803C1F801FFF800FFB8003E38000038000038000038000038000 0380000380000380003FF8003FF8003FF8151E7E9318>I<7F87E0FF9FF07FBFF803F878 03F03003E00003C00003C000038000038000038000038000038000038000038000038000 0380007FFE00FFFF007FFE0015147F9318>I<07F7003FFF007FFF00780F00E00700E007 00E007007C00007FE0001FFC0003FE00001F00600780E00380E00380F00380F80F00FFFF 00FFFC00E7F00011147D9318>I<0180000380000380000380000380007FFFC0FFFFC0FF FFC00380000380000380000380000380000380000380000380000380000380400380E003 80E00380E001C1C001FFC000FF80003E0013197F9818>I<7E07E0FE0FE07E07E00E00E0 0E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E00E01E0 0F03E007FFFC03FFFE01FCFC1714809318>I<7F8FF0FF8FF87F8FF01E03C00E03800E03 800E0380070700070700070700038E00038E00038E00038E0001DC0001DC0001DC0000F8 0000F80000700015147F9318>II<7F8FF07F9FF07F8FF0070700078E00039E0001DC0001F80000F800 00700000F00000F80001DC00039E00038E000707000F07807F8FF0FF8FF87F8FF015147F 9318>I<7F8FF0FF8FF87F8FF00E01C00E03800E03800703800707000707000387000386 00038E0001CE0001CE0000CC0000CC0000DC000078000078000078000070000070000070 0000F00000E00079E0007BC0007F80003F00001E0000151E7F9318>I<3FFFF07FFFF07F FFF07001E07003C0700780000F00001E00003C0000F80001F00003C0000780000F00701E 00703C0070780070FFFFF0FFFFF0FFFFF014147F9318>I<0007E0001FE0007FE0007800 00E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00001E000 7FC000FF8000FF80007FC00001E00000E00000E00000E00000E00000E00000E00000E000 00E00000E00000E00000E000007800007FE0001FE00007E013247E9F18>I<60F0F0F0F0 F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0600424769F18 >I<7C0000FF0000FFC00003C00000E00000E00000E00000E00000E00000E00000E00000 E00000E00000E00000E00000F000007FC0003FE0003FE0007FC000F00000E00000E00000 E00000E00000E00000E00000E00000E00000E00000E00000E00003C000FFC000FF00007C 000013247E9F18>I E /Fh 47 123 df<007E1F0001C1B1800303E3C00703C3C00E03C1 800E01C0000E01C0000E01C0000E01C0000E01C0000E01C000FFFFFC000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0000E01C0 000E01C0000E01C0000E01C0000E01C0000E01C0007F87FC001A1D809C18>11 D<007E0001C1800301800703C00E03C00E01800E00000E00000E00000E00000E0000FFFF C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C07F87F8151D809C17>I<007FC001C1C00303C00703C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFFFC00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C07F CFF8151D809C17>I<004000800100020006000C000C0018001800300030007000600060 006000E000E000E000E000E000E000E000E000E000E000E000E000600060006000700030 003000180018000C000C00060002000100008000400A2A7D9E10>40 D<800040002000100018000C000C000600060003000300038001800180018001C001C001 C001C001C001C001C001C001C001C001C001C0018001800180038003000300060006000C 000C00180010002000400080000A2A7E9E10>I<60F0F0701010101020204080040C7C83 0C>44 DI<60F0F06004047C830C>I66 D<001F808000E0618001801980070007800E0003801C0003801C0001 8038000180780000807800008070000080F0000000F0000000F0000000F0000000F00000 00F0000000F0000000F0000000700000807800008078000080380000801C0001001C0001 000E000200070004000180080000E03000001FC000191E7E9C1E>I69 D73 D75 DI78 D<003F800000E0E0000380380007001C000E000E001C0007003C 00078038000380780003C0780003C0700001C0F00001E0F00001E0F00001E0F00001E0F0 0001E0F00001E0F00001E0F00001E0700001C0780003C0780003C0380003803C0007801C 0007000E000E0007001C000380380000E0E000003F80001B1E7E9C20>I<07E0801C1980 300580700380600180E00180E00080E00080E00080F00000F800007C00007FC0003FF800 1FFE0007FF0000FF80000F800007C00003C00001C08001C08001C08001C0C00180C00180 E00300D00200CC0C0083F800121E7E9C17>83 D<7FFFFFC0700F01C0600F00C0400F0040 400F0040C00F0020800F0020800F0020800F0020000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000001F800003FFFC001B1C7F9B1E>III89 D<1FC000307000783800781C00301C00001C00001C0001FC000F1C00381C00701C00601C 00E01C40E01C40E01C40603C40304E801F870012127E9115>97 DI<07E00C301878307870306000E000E000E000E000E000E0 0060007004300418080C3007C00E127E9112>I<003F0000070000070000070000070000 070000070000070000070000070000070003E7000C1700180F00300700700700600700E0 0700E00700E00700E00700E00700E00700600700700700300700180F000C370007C7E013 1D7E9C17>I<03E00C301818300C700E6006E006FFFEE000E000E000E000600070023002 18040C1803E00F127F9112>I<00F8018C071E061E0E0C0E000E000E000E000E000E00FF E00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E007FE00F 1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C00381C00381C00181800 1C38000C300013C0001000003000001800001FF8001FFF001FFF803003806001C0C000C0 C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C0018000000000000000000000000000000FC001C 001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80091D7F 9C0C>I<00C001E001E000C000000000000000000000000000000FE000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0 F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180E001C0E001C0E001C0E001C0 E001C0E001C06001807003803003001806000E1C0003F00012127F9115>II<03C1000C3300180B00300F00700700700700E00700E00700E00700E0 0700E00700E00700600700700700300F00180F000C370007C70000070000070000070000 0700000700000700000700003FE0131A7E9116>II<1F9030704030C010 C010E010F8007F803FE00FF000F880388018C018C018E010D0608FC00D127F9110>I<04 000400040004000C000C001C003C00FFE01C001C001C001C001C001C001C001C001C001C 101C101C101C101C100C100E2003C00C1A7F9910>III< FF3FCFE03C0F03801C0701801C0701001C0B01000E0B82000E0B82000E1182000711C400 0711C4000720C40003A0E80003A0E80003C0680001C0700001C070000180300000802000 1B127F911E>I<7F8FF00F03800F030007020003840001C80001D80000F0000070000078 0000F800009C00010E00020E000607000403801E07C0FF0FF81512809116>II<7FFC70386038407040F040E041C003C0038007000F040E041C043C 0C380870087038FFF80E127F9112>I E /Fi 14 107 df0 D<70F8F8F87005057C8D0D>I<400004C0000C6000183000301800600C00C0060180 03030001860000CC0000780000300000300000780000CC000186000303000601800C00C0 180060300030600018C0000C40000416187A9623>I<03C00FF01FF83FFC7FFE7FFEFFFF FFFFFFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D21 D<003FF800FFF803C0000700000C0000180000300000300000600000 600000C00000C00000C00000FFFFF8FFFFF8C00000C00000C00000600000600000300000 3000001800000C000007000003C00000FFF8003FF8151C7C981E>50 D<00000C00000C0000180000180000300000300000600000600000C00000C00001800001 80000180000300000300000600000600000C00000C000018000018000030000030000060 0000600000C00000C0000180000180000300000300000600000600000600000C00000C00 00180000180000300000300000600000600000C00000400000162C7AA000>54 D<00020000060003F600061C000C0C00180E00300F00301B00701B806019806031806031 80E031C0E031C0E061C0E061C0E061C0E061C0E0C1C0E0C1C0E0C1C0E181C0E181C0E181 C0E181C0E301C06301806301807303807603803603003603001C06000C0C000E18000FF0 0018000018000018000012277EA217>59 D62 D<00040000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C00 00000C0000000C0000000C0000000C0000000C0000000C0000000C0000000C0000FFFFFF E0FFFFFFE01B1C7C9B23>I<00FE0003FF800F01E01C007030001860000C60000CC00006 C00006C00006C00006C00006C00006C00006C00006C00006C00006C00006C00006C00006 C00006C00006C00006C00006C00006C00006C00006400002171C7D9A1E>92 D<000F0038006000E001C001C001C001C001C001C001C001C001C001C001C001C001C001 C001C0038007001E00F8001E000700038001C001C001C001C001C001C001C001C001C001 C001C001C001C001C001C000E000600038000F102D7DA117>102 DI106 D E /Fj 34 90 df<3078F8787005057C840D>46 D<00000400000C00000C0000180000180000300000300000600000600000C00000C00001 80000180000300000300000600000600000C00000C000018000018000030000030000060 0000600000C00000C0000180000180000300000300000600000600000600000C00000C00 00180000180000300000300000600000600000C00000C00000800000162D7EA117>I<00 7E0001C3000301800701C00E00C00E00E01C00E01C00E03C01E03801E07801E07801E078 01E07801E07801E0F003C0F003C0F003C0F003C0F003C0F00380F00780E00780E00700E0 0700E00E00600E00701C003038003870000FC000131F7C9D17>I<000C001C00FC0F3800 38003800380038003800700070007000700070007000E000E000E000E000E000E001C001 C001C001C001C001C0038003C0FFFE0F1E7C9D17>I<003F8000C1E00100F00200780400 780400780F007C0F807C0F807C0F00780600780000F80000F00001E00001C00003800007 00000E00001C0000380000600000C0000180000300200600200800401000403FFFC07FFF 80FFFF80161E7E9D17>I<007F000183C00201E00400F00700F00F00F00F01F00F01F000 01E00001E00003C0000380000700000E0000F800000E000007000007800007C00003C000 07C03007C07807C0F807C0F807C0F00780800F00400E00201C0018780007E000141F7D9D 17>I<0000600000600000E00001C00003C00005C0000DC00009C00011C0002380004380 00C380008380010380020380040700080700180700100700200700400700FFFFF0000E00 000E00000E00000E00000E00001C00001E0001FFE0141E7E9D17>I<01803001FFE003FF C003FF0003FC00020000020000020000040000040000040000047C000587000603800C01 800801C00001C00001E00001E00001E00001E07003C0F803C0F003C0E003808007804007 00400E00201C0018700007C000141F7D9D17>I<000F8000704000C0200180E00301E007 01E00E00C01E00001C00003C000038000078F800790E007A07007C0300F80380F80380F0 03C0F003C0F003C0F003C0F00780E00780E00780E00700E00F00600E00701C0030180018 700007C000131F7C9D17>I<2000003FFFE07FFFC07FFF80400100C00200800200800400 0008000010000020000040000040000080000180000300000300000700000600000E0000 0E00001E00001C00001C00003C00003C00003C0000780000780000780000300000131F79 9D17>I<003F0000C1C00100600200600400300C00300C00300C00300C00600E00600F80 C00FC18007F60003FC0001FC0001FF00063F800C0F801007C03003C06001C06000C0C000 C0C000C0C000C0C00080C0010060030030040018180007E000141F7D9D17>I<007E0001 C3000301800601C00E01C01C00C03C00E03C00E03C01E07801E07801E07801E07801E078 03E07803E03803C03807C01C0BC00C13C003E380000780000780000700000E00600E00F0 1C00F01800E0300080600041C0003F0000131F7C9D17>I<000010000000180000003800 0000380000007800000078000000FC000001BC0000013C0000033C0000023C0000063C00 00043E0000081E0000081E0000101E0000101E0000201E0000200F0000400F0000400F00 00FFFF0000800F0001000F8001000780020007800200078004000780040007800C0007C0 3E0007C0FF807FFC1E207E9F22>65 D<07FFFF00007C01C0003C01E0003C00F0007800F8 007800F8007800F8007800F8007800F8007800F000F001F000F001E000F003C000F00F80 00FFFE0000F00F0001E007C001E003C001E003E001E001E001E001E001E001E003C001E0 03C003E003C003E003C003C003C007C003C00F8007800F0007803E00FFFFF0001D1F7E9E 20>I<0001F808000E061800380138007000F801E0007803C0007007800030078000300F 0000301F0000301E0000303E0000203C0000007C0000007C0000007C0000007C000000F8 000000F8000000F8000000F8000000F80000007800004078000080780000803C0000803C 0001001C0002000E00020006000C000300100001C0E000003F00001D217B9F21>I<07FF FF00007C01E0003C00F0003C00780078003C0078003C0078001E0078001E0078001E0078 001F00F0001F00F0001F00F0001F00F0001F00F0001F00F0001F01E0001E01E0003E01E0 003E01E0003E01E0003C01E0007C03C0007803C000F003C000F003C001E003C003C003C0 078007800F0007803C00FFFFE000201F7E9E23>I<07FFFFF8007C0078003C0038003C00 1800780018007800080078000800780008007800080078080800F0100000F0100000F010 0000F0300000FFF00000F0700001E0200001E0200001E0200001E0200001E0000801E000 1003C0001003C0001003C0002003C0002003C0006003C000C0078001C0078007C0FFFFFF 801D1F7E9E1F>I<07FFFFF8007C0078003C0038003C0018007800180078000800780008 00780008007800080078000800F0100000F0100000F0100000F0300000F0700000FFF000 01E0600001E0200001E0200001E0200001E0200001E0000003C0000003C0000003C00000 03C0000003C0000003C000000780000007C00000FFFE00001D1F7E9E1E>I<0001FC0400 0F030C003C009C0070007C00E0003C01C0003803800018078000180F0000181F0000181E 0000183E0000103C0000007C0000007C0000007C0000007C000000F8000000F8000000F8 007FFCF80003E0780001E0780001E0780003C0780003C03C0003C03C0003C01C0003C00E 0007C007000B800380118001E06080003F80001E217B9F24>I<07FFC7FFC0007C00F800 003C007800003C007800007800F000007800F000007800F000007800F000007800F00000 7800F00000F001E00000F001E00000F001E00000F001E00000FFFFE00000F001E00001E0 03C00001E003C00001E003C00001E003C00001E003C00001E003C00003C007800003C007 800003C007800003C007800003C007800003C007800007800F000007C00F8000FFF8FFF8 00221F7E9E22>I<07FFE0007C00003C00003C0000780000780000780000780000780000 780000F00000F00000F00000F00000F00000F00001E00001E00001E00001E00001E00001 E00003C00003C00003C00003C00003C00003C00007800007C000FFFC00131F7F9E10>I< 07FFF000007E0000003C0000003C00000078000000780000007800000078000000780000 0078000000F0000000F0000000F0000000F0000000F0000000F0000001E0000001E00000 01E0000001E0000001E0008001E0010003C0010003C0010003C0030003C0020003C00600 03C0060007801E0007807C00FFFFFC00191F7E9E1C>76 D<07FC0000FFC0007C0000F800 003C00017800003C00017800004E0002F000004E0002F000004E0004F000004E0004F000 004E0008F000004E0008F00000870011E00000870011E00000870021E00000870021E000 00870041E00000838041E00001038083C00001038083C00001038103C00001038203C000 0101C203C0000101C403C0000201C40780000201C80780000201C80780000201D0078000 0200F00780000600E00780000600E00F00000F00C00F8000FFE0C1FFF8002A1F7E9E2A> I<07FC01FFC0003E003E00003E001800003E001800004F001000004F0010000047801000 00478010000043C010000043C010000083C020000081E020000081E020000080F0200000 80F020000080782000010078400001007C400001003C400001003C400001001E40000100 1E400002000F800002000F800002000F8000020007800002000780000600038000060003 00000F00010000FFE0010000221F7E9E22>I<0003F800001E0E000038070000E0038001 C001C003C001E0078000E00F0000F00F0000F01E0000F01E0000F83E0000F83C0000F87C 0000F87C0000F87C0000F87C0000F8F80001F0F80001F0F80001F0F80001F0F80003E078 0003E0780003C0780007C07C0007803C000F003C001E001E001C000E0038000700F00003 C3C00000FE00001D217B9F23>I<07FFFF00007C03C0003C01E0003C00F0007800F00078 00F8007800F8007800F8007800F8007800F000F001F000F001E000F003C000F0078000F0 0F0000FFF80001E0000001E0000001E0000001E0000001E0000001E0000003C0000003C0 000003C0000003C0000003C0000003C000000780000007C00000FFFC00001D1F7E9E1F> I<07FFFC00007C0700003C03C0003C01E0007801E0007801F0007801F0007801F0007801 F0007801E000F003E000F003C000F0078000F00F0000F03C0000FFF00001E0300001E038 0001E01C0001E01C0001E01C0001E01E0003C03E0003C03E0003C03E0003C03E0003C03E 0003C03E0207803E0407C01F04FFFC0F18000003E01F207E9E21>82 D<003F040060CC01803C03801C03001C0700180600080E00080E00080E00080E00000F00 000F80000FE00007FE0003FF8001FFC0007FE00007E00001E00000E00000F00000F04000 E04000E04000E04000E06000C0600180E00380F80300C60C0081F80016217D9F19>I<3F FFFFF03C0780F03007803060078030400F0010400F0010C00F0010800F0010800F001080 0F0010001E0000001E0000001E0000001E0000001E0000001E0000003C0000003C000000 3C0000003C0000003C0000003C0000007800000078000000780000007800000078000000 78000000F0000001F800007FFFE0001C1F7A9E21>IIII<03FFC0FFC0007F007E00003E00 3800001E003000001E002000000F004000000F008000000F81000000078200000007C600 000003C400000003E800000001F000000001F000000000F000000000F800000000F80000 00017C000000023C000000043C0000000C1E000000081E000000101F000000200F000000 400F800000C0078000008007C000010003C000070003E0001F8007E000FFE01FFE00221F 7F9E22>II E /Fk 15 111 df<020408103020604040C0C0C0C0C0C0C0C0404060203010080402071A 7F920C>40 D<8040201018080C0404060606060606060604040C081810204080071A7E92 0C>I<1F00318060C04040C060C060C060C060C060C060C060C060404060C031801F000B 107F8F0F>48 D<0C003C00CC000C000C000C000C000C000C000C000C000C000C000C000C 00FF8009107E8F0F>I<1F00618040C08060C0600060006000C00180030006000C001020 20207FC0FFC00B107F8F0F>I<1F00218060C060C000C0008001800F00008000400060C0 60C060804060801F000B107F8F0F>I<0300030007000F000B001300330023004300C300 FFE003000300030003001FE00B107F8F0F>I<20803F002C002000200020002F00308020 40006000600060C06080C061801F000B107F8F0F>I<0780184030C060C06000C000CF00 F080E040C060C060C060406060C030801F000B107F8F0F>I<40007FE07FC08080808001 000200040004000C0008000800180018001800180018000B117E900F>I<1F00318060C0 60C060C071803F000F00338061C0C060C060C060404060801F000B107F8F0F>I<1F0031 8060C0C040C060C060C06040E021E01E600060004060C0608043003E000B107F8F0F>I< FFFE18061802180318011821182018601FE01860182018201800180018001800FF001011 7F9014>70 D<20702000000000F0303030303030303030FC0612809108>105 D110 D E /Fl 33 123 df<00003F03E00000C386700001878CF00003879CF000030318600007 00380000070038000007003800000E003800000E007000000E007000000E00700000FFFF FF80001C007000001C00E000001C00E000001C00E000001C00E000003800E000003801C0 00003801C000003801C000003801C000007001C000007003800000700380000070038000 0070038000006003800000E007000000E007000000E007000000E007000000C006000001 C00E000001C00E000031860C0000798F180000F31E100000620C6000003C07C000002429 829F1C>11 D<00003FE00000E01000018038000380780003007800070030000700000007 000000070000000E0000000E0000000E000000FFFFE0000E00E0001C01C0001C01C0001C 01C0001C01C0001C03800038038000380380003803800038070000380700007007000070 071000700E2000700E2000700E2000E00E2000E0064000E0038000E0000000C0000001C0 000001C000003180000079800000F3000000620000003C0000001D29829F1A>I<000F00 000030800C0060401E00C0203C01C0201C0180E0040381E0080303C00807018010070000 2007000040060001800678028007C40C4007041020060420200F0440101D0840103CF058 1038007C10780078107800301078000010F0000020F0000020F0000040F0000040700000 807000010038000300180004000E0018000780E00000FF00001F227BA023>38 D<7FF0FFE07FE00C037D8A10>45 D<70F8F8F0E005057B840E>I<000002000000060000 00060000000E0000001E0000001E0000003F0000002F0000004F0000004F0000008F0000 010F0000010F0000020F0000020F0000040F00000C0F0000080F0000100F0000100F0000 200F80003FFF800040078000C00780008007800100078001000780020007800200078006 0007801E000F80FF807FF81D207E9F22>65 D<01FFFFFE001E001C001E000C001E000400 1E0004003C0004003C0004003C0004003C00040078080800780800007808000078180000 F0300000FFF00000F0300000F0300001E0200001E0200001E0200001E0001003C0002003 C0002003C0004003C00040078000800780018007800100078007000F001F00FFFFFE001F 1F7D9E1F>69 D<01FFFF00001E03C0001E00E0001E0070001E0078003C0078003C007800 3C0078003C0078007800F0007800F0007801E0007801C000F0070000F01E0000FFF00000 F0380001E01C0001E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003C01E0007 803C0007803C0807803C0807803C100F801C10FFF00C20000007C01D207D9E21>82 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E 00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F38 0F700F700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207B9F15>I< 007E0001C1000300800E07801E07801C07003C0200780000780000780000F00000F00000 F00000F00000F0000070010070020030040018380007C00011147C9315>I<0000780003 F80000700000700000700000700000E00000E00000E00000E00001C00001C000F1C00389 C00707800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E 20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C01C207010E011C01 3C013802780C7BF07C00F000F000F000F0007000700170023804183807C010147C9315> I<00007800019C00033C00033C000718000700000700000E00000E00000E00000E00000E 0001FFE0001C00001C00001C00001C000038000038000038000038000038000070000070 0000700000700000700000700000E00000E00000E00000E00000C00001C00001C0000180 003180007B0000F300006600003C00001629829F0E>I<003C6000E27001C1E00380E007 00E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C03803C07003C07001C 0F001C17000C2E0003CE00000E00000E00001C00001C00301C00783800F0700060E0003F 8000141D7E9315>I<01E0000FE00001C00001C00001C00001C000038000038000038000 038000070000070000071E000763000E81800F01C00E01C00E01C01C03801C03801C0380 1C0380380700380700380700380E10700E20700C20701C20700C40E00CC060070014207D 9F17>I<00C001E001E001C000000000000000000000000000000E003300230043804300 470087000E000E000E001C001C001C003840388030807080310033001C000B1F7C9E0E> I<01E0000FE00001C00001C00001C00001C0000380000380000380000380000700000700 000703C00704200E08E00E11E00E21E00E40C01C80001D00001E00001FC00038E0003870 00387000383840707080707080707080703100E03100601E0013207D9F15>107 D<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C 0038003800380038007000700070007100E200E200E200E200640038000A207C9F0C>I< 1C0F80F0002630C318004740640C004780680E004700700E004700700E008E00E01C000E 00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01C070803803 8071003803806100380380E10038038062007007006600300300380021147C9325>I<1C 0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C01C01C 01C01C03843803883803083807083803107003303001C016147C931A>I<007C0001C300 0301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0F00780 F00700700F00700E0030180018700007C00013147C9317>I<01C1E002621804741C0478 1C04701E04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380780380 700380E003C1C0072380071E000700000700000E00000E00000E00000E00001C00001C00 00FFC000171D809317>I<00F0400388C00705800E03801C03803C038038070078070078 0700780700F00E00F00E00F00E00F00E00F01C00F01C00703C00705C0030B8000F380000 380000380000700000700000700000700000E00000E0000FFE00121D7C9315>I<1C1E00 2661004783804787804707804703008E00000E00000E00000E00001C00001C00001C0000 1C000038000038000038000038000070000030000011147C9313>I<00FC030206010C03 0C070C060C000F800FF007F803FC003E000E700EF00CF00CE008401020601F8010147D93 13>I<018001C0038003800380038007000700FFF007000E000E000E000E001C001C001C 001C003800380038003820704070407080708031001E000C1C7C9B0F>I<0E00C03300E0 2301C04381C04301C04701C08703800E03800E03800E03801C07001C07001C07001C0710 1C0E20180E20180E201C1E200C264007C38014147C9318>I<0E03803307802307C04383 C04301C04700C08700800E00800E00800E00801C01001C01001C01001C02001C02001C04 001C04001C08000E300003C00012147C9315>I<0E00C1C03300E3C02301C3E04381C1E0 4301C0E04701C060870380400E0380400E0380400E0380401C0700801C0700801C070080 1C0701001C0701001C0602001C0F02000C0F04000E13080003E1F0001B147C931E>I<03 83800CC4401068E01071E02071E02070C040E00000E00000E00000E00001C00001C00001 C00001C040638080F38080F38100E5810084C60078780013147D9315>I<0E00C03300E0 2301C04381C04301C04701C08703800E03800E03800E03801C07001C07001C07001C0700 1C0E00180E00180E001C1E000C3C0007DC00001C00001C00003800F03800F07000E06000 C0C0004380003E0000131D7C9316>I<01C04003E08007F1800C1F000802000004000008 000010000020000040000080000100000200000401000802001002003E0C0063FC0041F8 0080E00012147D9313>I E /Fm 67 123 df<007F07F001FF1FF003FF3FF00780780007 0070000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F000FFF8FF80FF F8FF80FFF8FF800F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F 00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F00F0000F 00F0001C20809F1B>11 D<0000F000F8F001F8F003F8F00780000700000F00000F00000F 00000F00000F00000F00000F0000FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F 00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F 00F0142180A018>I<00F8F001F8F003F8F00780F00700F00F00F00F00F00F00F00F00F0 0F00F00F00F00F00F0FFF8F0FFF8F0FFF8F00F00F00F00F00F00F00F00F00F00F00F00F0 0F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F00F0142080 9F18>I<007000E001C00380078007000E001E001E003C003C003C007800780078007800 7000F000F000F000F000F000F000F000F000F000F000F000F00070007800780078007800 3C003C003C001E001E000E0007000780038001C000E000700C2E7EA112>40 DI<7878787818303060 60E0050A7D830D>44 DI<00010003000300060006000600 0C000C000C00180018001800300030003000600060006000C000C000C001800180018003 00030003000600060006000C000C000C00180018001800300030003000600060006000C0 00C000C000102D7DA117>47 D<03F00007F8001FFE001E1E003C0F003807007807807807 80700380F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003 C0F003C0F003C0F003C07807807807807807803807003C0F001E1E001FFE000FFC0003F0 0012207E9E17>I<00C001C00FC0FFC0FFC0F3C003C003C003C003C003C003C003C003C0 03C003C003C003C003C003C003C003C003C003C003C003C003C003C0FFFEFFFEFFFE0F1F 7C9E17>I<07F0000FFC001FFE00383F00700F00600780E00780E003C04003C04003C000 03C00003C00003C0000780000780000F00000E00001C00003C0000780000E00001C00003 80000700000E00001C0000380000700000FFFFC0FFFFC0FFFFC0121F7E9E17>I<03F000 0FFC001FFE003C1F00780F00300780200780000780000780000780000F00000F00003E00 03FC0003F80003FC00001E00000F000007800003800003C00003C00003C00003C08003C0 C003C0C00780700F807C1F003FFE000FFC0003F00012207E9E17>I<03F00007F8000FFC 001E1E003C0F00780700780780F00780F00380F003C0F003C0F003C0F003C0F003C0F003 C07807C07807C07C0FC03E1FC01FFBC00FF3C007C780000780000780000700000F00001E 00201E00307C007FF8003FF0000FC00012207E9E17>57 DI61 D<0F000F000F000F000000000000000000000006000F000F000E000E000E000E000E001E 001C003C003C007800F800F000F000F000F000F008783C7FFC3FF81FC00E207D9615>I< 001F0000001F0000003F8000003B8000003B8000007BC0000073C0000071C00000F1E000 00E1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C00 07803C0007003C000FFFFE000FFFFE000FFFFE001E000F001E000F003C000F803C000780 3C000780780007C0780003C0780003C0F00003E01B207F9F1E>65 DI<001FC000 FFF801FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000780000F0 0000F00000F00000F00000F00000F00000F00000F000007800007800007800007C00003C 00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>IIII<001FE000FFF801FFFE03E03E07800E0F0000 1E00003E00003C00007C0000780000780000780000F00000F00000F00000F00000F00000 F00000F003FEF003FE7803FE78001E78001E7C001E3C001E3E001E1E001E0F001E07801E 03E03E01FFFE00FFF8001FC017227DA01E>III75 DIII<003F000000FF C00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F00780007807800 078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F000 03C0F00003C0F80007C07800078078000780780007803C000F003C000F001E001E001F00 3E000F807C0007E1F80003FFF00000FFC000003F00001A227DA021>II82 D<01FC0007FF800FFFC01F03C03C00C03C 00007800007800007800007800007800007C00003C00003F00001FE0000FFC0007FE0001 FF00003F800007C00003C00003E00001E00001E00001E00001E00001E00001C0C003C0F0 07C0FC0F807FFF001FFE0003F80013227EA019>III II<780007807C000F003E001F00 1E001E000F003C000F807C000780780003C0F00003E1F00001F1E00000F3C000007FC000 007F8000003F0000001F0000001E0000003F0000007F8000007FC00000F3C00001F1E000 01E0F00003C0F80007C0780007803C000F003E001F001E001E000F003C000F807C0007C0 780003C0F00003E01B207F9F1E>II II93 D<07E03FF87FFC701E401F000F000F000F003F07FF1FFF7E0FF80FF00FF00FF00FF83F7F FF3FEF1F8F10147E9316>97 DI<03F00FFC1FFE3E0E3C0278007800F000F000F000F000F000F0007800 78003C013E0F1FFF0FFE03F010147E9314>I<0007800007800007800007800007800007 8000078000078000078000078000078000078007C7800FF7801FFF803E1F807C07807807 80F80780F00780F00780F00780F00780F00780F00780F00780780780780F803E1F801FFF 800FF78007C78011207E9F17>I<03F0000FFC001FFE003E1F003C0700780700700380FF FF80FFFF80FFFF80F00000F00000F000007000007800003C01003E07001FFF0007FE0001 F80011147F9314>I<007E01FE03FE078007000F000F000F000F000F000F000F00FFF0FF F0FFF00F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F000F 000F20809F0E>I<03E0F00FFFF01FFFF03E3E003C1E00780F00780F00780F00780F0078 0F003C1E003E3E001FFC003FF80033E0003000003800003FFE003FFF801FFFC03FFFE078 03F0F000F0F000F0F000F0F801F07E07E03FFFC00FFF0003FC00141E7F9317>III107 DIII<01F80007FE00 1FFF803F0FC03C03C07801E07801E0F000F0F000F0F000F0F000F0F000F0F000F07801E0 7801E03C03C03F0FC01FFF8007FE0001F80014147F9317>II<03C7800FF7801FFF803E1F807C0F80780780780780F00780F0 0780F00780F00780F00780F00780F807807807807C0F803E1F801FFF800FF78007C78000 0780000780000780000780000780000780000780000780000780111D7E9317>II<07F01FFC3FFC780C7800780078007C003FC01FF00FF803F8007C003C 003CC03CF07CFFF87FF00FC00E147F9311>I<1E001E001E001E001E001E00FFF0FFF0FF F01E001E001E001E001E001E001E001E001E001E001E001E001E001E201FF00FF007C00C 1A7F9910>IIII<7801E07C03C03E0780 1E0F000F0F00079E0003FC0003F80001F80000F00001F00001F80003FC00079E000F0F00 0E0F001E07803C03C07801E0F801F01414809315>II<7FFF7FFF7FFF003E003C007800F800F001E003E007C007800F001F00 1E003C007C00FFFFFFFFFFFF10147F9314>I E /Fn 59 123 df<003F83F800FF8FF801 FF9FF803E03E000380380007807800078078000780780007807800078078000780780007 80780007807800FFFC7FC0FFFC7FC0FFFC7FC00780780007807800078078000780780007 807800078078000780780007807800078078000780780007807800078078000780780007 80780007807800078078000780780007807800078078001D2380A21C>11 D<000078007C7800FC7801FC7803C0000380000780000780000780000780000780000780 00078000078000FFFC78FFFC78FFFC780780780780780780780780780780780780780780 780780780780780780780780780780780780780780780780780780780780780780780780 78152480A31A>I<783C783C783C783C180C3018301860306030E0700E0A7FA218>34 D46 D<0000800001800001800003000003000003000006000006 00000600000C00000C00000C000018000018000018000030000030000030000060000060 0000600000C00000C00000C0000180000180000180000180000300000300000300000600 000600000600000C00000C00000C00001800001800001800003000003000003000006000 00600000600000C00000C00000C0000011317DA418>I<01F00007FC000FFE001F1F003C 07803C07807803C07803C07803C07001C0F001E0F001E0F001E0F001E0F001E0F001E0F0 01E0F001E0F001E0F001E0F001E0F001E0F001E0F001E0F001E07803C07803C07803C078 03C03C07803C07801F1F000FFE0007FC0001F00013237EA118>I<00C001C007C0FFC0FF C0FBC003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003C003 C003C003C003C003C003C003C003C003C0FFFFFFFFFFFF10227CA118>I<03F0000FFC00 1FFE003C1F003007807007C06003C0E003E0C001E04001E04001E00001E00001E00001E0 0003C00003C0000780000780000F00001E00003C0000780000F00001E00001C000038000 0700000E00001C0000380000700000FFFFE0FFFFE0FFFFE013227EA118>I<01F00007FC 001FFF003E0F003807807003C02003C02003C00003C00003C00003C0000780000780000F 00001E0003FC0003F80003FE00000F000007800003C00003C00001E00001E00001E00001 E00001E08001E0C003C0E003C07007803C0F801FFF000FFC0003F00013237EA118>I<00 1F00001F00002F00002F00006F0000EF0000CF0001CF0001CF00038F00038F00078F0007 0F000F0F000E0F001E0F003C0F003C0F00780F00780F00F00F00FFFFF8FFFFF8FFFFF800 0F00000F00000F00000F00000F00000F00000F00000F00000F0015217FA018>I<3FFF80 3FFF803FFF803C00003C00003C00003C00003C00003C00003C00003C00003C00003CF800 3FFE003FFF003F0F803E07803C03C03803C00001E00001E00001E00001E00001E00001E0 0001E04003C04003C0E003C07007807C1F003FFE000FFC0003F00013227EA018>I<007E 0001FF0003FF0007C1000F00001E00001E00003C00003C000078000078000078F800F1FE 00F7FF00FF0F80FC0780F803C0F803C0F801C0F001E0F001E0F001E0F001E0F001E07001 E07001E07801E07803C03803C03C03801C07801F0F000FFE0007FC0001F00013237EA118 >II< 01F00007FC000FFE001E0F003C07803C07807803C07803C07803C07803C07803C0380380 3C07801E0F000F1E0007FC0003F8000FFE001E0F003C07807803C07803C0F001E0F001E0 F001E0F001E0F001E0F001E07803C07803C03C07803E0F801FFF0007FC0001F00013237E A118>I<01F00007FC000FFE001E0F003C07003807807803807803C0F003C0F001C0F001 E0F001E0F001E0F001E0F001E0F001E07003E07803E07803E03C07E03E1FE01FFDE00FF1 E003E3C00003C00003C0000380000780000700000F00101E00183C003FF8001FF00007C0 0013237EA118>II< 001F0000001F0000003F8000003F8000003B8000007BC0000073C0000071C00000F1E000 00F1E00000E0E00001E0F00001E0F00001C0F00003C0780003C078000380780007803C00 07803C0007003C000F001E000F001E000FFFFE001FFFFF001FFFFF001C000F003C000780 3C00078038000780780003C0780003C0700003C0F00001E0F00001E0E00001E01B237EA2 20>65 DI<000FF000007FFC0000FFFF0001F01F0003C00700078000000F000000 1E0000003E0000003C0000003C000000780000007800000078000000F0000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000F00000007800000078000000 780000003C0000003C0000003E0000001E0000000F0000000780008003C0038001F00F80 00FFFF00007FFC00000FF00019257DA31F>IIII73 D76 DI<001FC000007FF00001FFFC0003F07E0007C01F000F800F801F0007 C01E0003C03C0001E03C0001E0780000F0780000F0780000F070000070F0000078F00000 78F0000078F0000078F0000078F0000078F0000078F0000078F0000078780000F0780000 F0780000F07C0001F03C0001E03E0003E01E0003C01F0007C00F800F8007C01F0003F07E 0001FFFC00007FF000001FC0001D257DA324>79 DI<001FC000007FF00001FFFC 0003F07E0007C01F000F800F801F0007C01E0003C03C0001E03C0001E0780000F0780000 F0780000F0780000F0F0000078F0000078F0000078F0000078F0000078F0000078F00000 78F0000078F000007870000070780000F0780000F0780780F03C07C1E03C03C1E01E01E3 C01F01F7C00F80FF8007C07F0003F07E0001FFFE00007FFE00001FDF0000000F80000007 C0000003C0000003E0000001F01D2A7DA324>II<00FE0003FFC007FFE00F81E0 1E00603C00003C00007800007800007800007800007800007C00003C00003F00001FC000 0FFC0007FF0001FF80003FC00007E00001F00000F00000F8000078000078000078000078 000078000078C000F0E000F0F801E07E07C03FFF800FFF0001FC0015257EA31B>I II87 D<381C3018301860306030C060F078F078F078F0780E0A78A218> 92 D<07E01FF83FFC381E201E000F000F000F000F00FF07FF1FFF3E0F780FF00FF00FF0 0FF00FF83F7FFF3FEF1F8F10167E9517>97 DI<01FC0007FF000FFF801F03803C 0180780000780000700000F00000F00000F00000F00000F00000F0000078000078000078 00003C00401F03C00FFFC007FF8001FC0012167E9516>I<0003C00003C00003C00003C0 0003C00003C00003C00003C00003C00003C00003C00003C00003C003E3C00FFBC01FFFC0 3F0FC03C07C07803C07803C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0 7803C07803C03C07C03E0FC01FFFC00FFBC003E3C012237EA219>I<03F00007FC001FFE 003E0F003C0780780380780380F001C0FFFFC0FFFFC0FFFFC0F00000F00000F000007000 007800007800003C00801F07800FFF8007FF0001F80012167E9516>I<003F00FF01FF03 C0038007800780078007800780078007800780FFF8FFF8FFF80780078007800780078007 800780078007800780078007800780078007800780078007800780102380A20F>I<01F0 7807FFF80FFFF81F1F001E0F003C07803C07803C07803C07803C07801E0F001F1F000FFE 001FFC0019F0003800003800003C00001FFE001FFFC01FFFE03FFFF07801F07800F8F000 78F00078F00078F000787800F03E03E01FFFC00FFF8001FC0015217F9518>I II107 DIII<01FC0007FF000FFF801F07C03C01E0 7800F07800F0700070F00078F00078F00078F00078F00078F000787800F07800F07C01F0 3E03E01F07C00FFF8007FF0001FC0015167F9518>II<03E3C007FBC00FFFC01F0FC03C07C07C03C078 03C0F803C0F003C0F003C0F003C0F003C0F003C0F003C0F803C07803C07803C03C07C03F 0FC01FFFC00FFBC003E3C00003C00003C00003C00003C00003C00003C00003C00003C000 03C00003C012207E9519>II<07F01FFC3FFE3C0E78 06780078007C003F003FF01FF80FFC01FE001F000F000F000FC00FF81EFFFE3FFC0FF010 167F9513>I<0F000F000F000F000F000F00FFF8FFF8FFF80F000F000F000F000F000F00 0F000F000F000F000F000F000F000F000F080F1C07FC07F803E00E1C7F9B12>IIII< 7801F07C01E03E03C01E07C00F0780078F0007DE0003FC0001FC0000F80000700000F800 01FC0003DC00039E00078F000F07801E07801E03C03C01E07800F0F800F81516809516> II<7FFFC07F FFC07FFFC0000F80000F00001E00003E00007C0000780000F00001F00003E00003C00007 80000F80001F00001E00003C00007C0000FFFFC0FFFFC0FFFFC012167F9515>I E /Fo 41 123 df<000FF07F00007FF9FF8000F83FC7C001E07F8FC003E07F0FC007C07F 0FC007C03F078007C01F000007C01F000007C01F000007C01F000007C01F0000FFFFFFF8 00FFFFFFF80007C01F000007C01F000007C01F000007C01F000007C01F000007C01F0000 07C01F000007C01F000007C01F000007C01F000007C01F000007C01F000007C01F000007 C01F000007C01F000007C01F00003FF8FFF0003FF8FFF0002220809F1F>11 D<000FE000007FF80000F81C0001E07C0003E07C0007C07C0007C07C0007C0380007C000 0007C0000007C0000007C1FE00FFFFFE00FFFFFE0007C03E0007C03E0007C03E0007C03E 0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E0007C03E 0007C03E0007C03E0007C03E003FF9FFC03FF9FFC01A20809F1D>I<3807007C0F80FE1F C0FF1FE0FF1FE07F0FE03B07600300600300600600C00600C00E01C00C0180180300700E 0020040013107E9F1B>34 D<0018007000E001C00380038007000E000E001E001C003C00 3C007800780078007800F800F000F000F000F000F000F000F000F000F000F80078007800 780078003C003C001C001E000E000E0007000380038001C000E0007000180D2D7DA114> 40 DI<387CFEFEFE7C3807 077C860F>46 D<01FC0007FF001F07C01E03C03E03E07C01F07C01F07C01F0FC01F8FC01 F8FC01F8FC01F8FC01F8FC01F8FC01F8FC01F8FC01F8FC01F8FC01F8FC01F8FC01F87C01 F07C01F07C01F03E03E01E03C01F8FC007FF0001FC00151D7E9C1A>48 D<00E00001E0000FE000FFE000F3E00003E00003E00003E00003E00003E00003E00003E0 0003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E0 0003E00003E00003E000FFFF80FFFF80111D7C9C1A>I<07F0001FFE00383F007C1F80FE 0FC0FE0FC0FE0FE0FE07E07C07E03807E0000FE0000FC0000FC0001F80001F00003E0000 780000F00000E00001C0000380600700600E00601C00E01FFFC03FFFC07FFFC0FFFFC0FF FFC0131D7D9C1A>I<01FC0007FF000E0F801E0FC03F07E03F07E03F07E03F07E01E0FC0 000FC0000F80001F0001FC0001FC00000F800007C00003E00003F00003F83803F87C03F8 FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003FC00151D7E9C1A>I<387CFEFEFE 7C38000000000000387CFEFEFE7C3807147C930F>58 D<07F8001FFE00381F80780F80FC 0FC0FC0FC0FC0FC0780FC0301F80001F00003E00007C0000700000E00000E00000C00000 C00000C00000C00000C00000C00000000000000000000000000001C00003E00007F00007 F00007F00003E00001C00012207D9F19>63 D<0000E000000000E000000001F000000001 F000000001F000000003F800000003F800000006FC00000006FC0000000EFE0000000C7E 0000000C7E000000183F000000183F000000303F800000301F800000701FC00000600FC0 0000600FC00000C007E00000FFFFE00001FFFFF000018003F000018003F000030001F800 030001F800060001FC00060000FC000E0000FE00FFE00FFFE0FFE00FFFE0231F7E9E28> 65 D69 D77 D<03FC080FFF381E03F83800F8700078700038F00038F00018F00018F80000FC 00007FC0007FFE003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC0 003CC0003CC0003CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>83 D91 D<0400800E01C0180300300600700E00600C00 600C00C01800C01800DC1B80FE1FC0FF1FE0FF1FE07F0FE03E07C01C038013107B9F1B> II<07FC001FFF003F0F803F07C03F03E03F03E00C 03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803E0FC05E07E0DE03F F8FE0FE07E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC 0000FC0000FC0000FC00007C00007E00003E00603F00C01F81C007FF0001FC0013147E93 17>I<0007F80007F80000F80000F80000F80000F80000F80000F80000F80000F80000F8 0000F801F8F80FFEF81F83F83E01F87E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8 FC00F8FC00F87C00F87C00F87E00F83E01F81F07F80FFEFF03F8FF18207E9F1D>I<01FE 0007FF800F83C01E01E03E00F07C00F07C00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00 007C00007C00003E00181E00180F807007FFE000FF8015147F9318>I<001F8000FFC001 F3E003E7E003C7E007C7E007C3C007C00007C00007C00007C00007C000FFFC00FFFC0007 C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC3C07FFFE0F079E1E03DE 3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC001800001800001C0000 1FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78003C78003C3F01F8 0FFFE001FF00171E7F931A>II<1C003E007F007F007F003E001C00000000000000000000000000FF00 FF001F001F001F001F001F001F001F001F001F001F001F001F001F001F001F001F00FFE0 FFE00B217EA00E>I108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC00 7EFC007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>II114 D<0FE63FFE701E600EE006E006F800FFC07FF8 3FFC1FFE03FE001FC007C007E007F006F81EFFFCC7F010147E9315>I<01800180018003 800380038007800F803F80FFFCFFFC0F800F800F800F800F800F800F800F800F800F800F 860F860F860F860F8607CC03F801F00F1D7F9C14>I 118 DIII<3FFFE03F FFE03C07C0380F80701F80603F00603E00607C0000F80001F80003F00003E06007C0600F 80601F80E03F00C03E01C07C03C0FFFFC0FFFFC013147F9317>I E /Fp 42 122 df46 D<00000C00001C00001C000038000038 0000380000700000700000700000E00000E00000E00001C00001C00001C0000380000380 000380000700000700000700000E00000E00000E00001C00001C00001C00003800003800 00380000700000700000700000E00000E00000E00001C00001C00001C000038000038000 0380000700000700000700000E00000E00000E00001C00001C00001C0000380000380000 380000700000700000700000E00000E00000C00000163C7DAC1D>I<007E0001FF8007FF E00FFFF01FC3F81F00F83E007C3E007C3C003C7C003E7C003E78001E78001EF8001FF800 1FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF8001FF800 1FF8001F78001E7C003E7C003E7C003E7C003E3E007C3F00FC1F00F81FC3F80FFFF007FF E001FF80007E0018297EA71D>I<00300000700001F0000FF000FFF000FFF000F1F00001 F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001 F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001F00001 F00001F00001F00001F00001F00001F000FFFFE0FFFFE0FFFFE013287BA71D>I<00FE00 03FF800FFFC01FFFE01F07F03C01F87800FC78007C70007CF0003EF0003E60003E20003E 00003E00003E00003E00007C00007C0000780000F80001F00001E00003E00007C0000F80 001F00003E00007C0000F80001F00003E00003C0000780000F00001E00003C00007FFFFE 7FFFFE7FFFFE7FFFFE17287EA71D>I<007E0003FF8007FFE01FFFF03F83F83E00F87C00 7C38007C30007C10007C00007C00007C0000F80000F80001F80003F0000FE001FFC001FF 8001FFC001FFE00001F80000FC00007C00003E00003E00001F00001F00001F00001F0000 1F80001FC0003EE0003EF0007E7C00FC3F01F81FFFF00FFFE003FFC000FE0018297EA71D >I<0007E0000FE0000BE0001BE0001BE0003BE0003BE00073E00073E000F3E000F3E001 E3E001E3E003E3E003C3E007C3E00F83E00F83E01F03E01F03E03E03E03E03E07C03E07C 03E0F803E0FFFFFFFFFFFFFFFFFFFFFFFF0003E00003E00003E00003E00003E00003E000 03E00003E00003E00003E018277EA61D>I<3FFFF83FFFF83FFFF83FFFF83E00003E0000 3E00003E00003E00003E00003E00003E00003E00003E00003E3E003EFF803FFFE03FFFF0 3FC1F03F00F83F007C3E007C00007C00003E00003E00003E00003E00003E00003E00003E 40003E60007CE0007CF000F87801F87F07F03FFFE00FFFC007FF0001FC0017287EA61D> I<000FE0003FF800FFF801FFF803F01807E0000F80000F00001F00003E00003E00003C00 007C00007C02007C3FC0F8FFE0F9FFF0FBFFF8FF80FCFE007CFE003EFC003EFC003FFC00 1FF8001FF8001FF8001FF8001F78001F7C001F7C001F7C001E3C003E3E003E1E007C1F00 FC0FC1F807FFF003FFE001FF80007E0018297EA71D>II<007E0003FFC007FFE00FFFF01F81F83F00FC3E007C7E007E7C003E7C003E7C003E 7C003E7C003E3E007C3E007C1F00F80FC3F007FFE003FFC003FFC00FFFF01F81F83F00FC 3E007C7C003E7C003EF8001FF8001FF8001FF8001FF8001FF8001F7C003E7C003E7E007E 3F00FC1F81F80FFFF007FFE003FFC0007E0018297EA71D>I<007E0001FF8007FFC00FFF E01F83F01F00F83E00787C007C7C003C78003EF8003EF8003EF8001EF8001FF8001FF800 1FF8001FF8003FFC003F7C003F7C007F3E007F3F01FF1FFFDF0FFF9F07FF1F03FC1E0040 3E00003E00003E00007C00007C0000F80000F80001F01803E01E0FC03FFF801FFF000FFE 0003F80018297EA71D>I<0003F000000003F000000007F800000007F800000007780000 000F3C0000000F3C0000000F3C0000001E3E0000001E1E0000001E1E0000003E1F000000 3C1F0000007C0F8000007C0F800000780F800000F807C00000F807C00000F007C00001F0 03E00001F003E00001E003E00003E001F00003E001F00003C001F00007C000F80007FFFF F80007FFFFF8000FFFFFFC000F80007C000F00007C001F00003E001F00003E001E00003E 003E00001F003E00001F003C00001F007C00000F807C00000F80F800000FC0F8000007C0 F8000007C0222A7EA927>65 D<0003FF00000FFFE0003FFFF800FFFFF801FE01F803F800 7007E0001007C000000F8000001F8000001F0000003E0000003E0000007C0000007C0000 007C0000007C000000F8000000F8000000F8000000F8000000F8000000F8000000F80000 00F8000000F8000000F80000007C0000007C0000007C0000007C0000003E0000003E0000 001F0000001F8000000F80000007C0000007E0000803F8003801FE01F800FFFFF8003FFF F0000FFFC00003FE001D2C7CAA25>67 DIII72 DI77 D<0003FC0000000FFF0000003FFFC00000FF FFF00001FE07F80003F801FC0007E0007E000FC0003F000F80001F001F00000F801F0000 0F803E000007C03E000007C07C000003E07C000003E07C000003E078000001E0F8000001 F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0F8000001F0 F8000001F0F8000001F07C000003E07C000003E07C000003E07E000007E03E000007C03E 000007C01F00000F801F80001F800F80001F000FC0003F0007E0007E0003F801FC0001FE 07F80000FFFFF000003FFFC000001FFF80000003FC0000242C7DAA2B>79 D<007FC00001FFF80007FFFE000FFFFF001FC07F003F000F007E0006007C0000007C0000 00F8000000F8000000F8000000F8000000F8000000FC0000007E0000007F0000003F8000 001FF800000FFF000007FFE00003FFF80000FFFC00000FFE000000FF0000003F0000001F 8000000F8000000FC0000007C0000007C0000007C0000007C0000007C0000007C000000F 8060000F80F0001F00FC003F00FF80FE007FFFFC001FFFF80007FFE00000FF80001A2C7D AA21>83 D86 D<01FC000FFF803FFFC03FFFE03C03F03001F00001F80000F80000F80000 F80000F80001F800FFF807FFF81FFFF83FC0F87E00F8FC00F8F800F8F800F8F800F8FC01 F87E07F87FFFF83FFFF81FFCF80FE0F8151B7E9A1C>97 DI<007F8001FFE007FFF80FFFF81FC0783F00103E00007C00 007C00007C0000F80000F80000F80000F80000F80000F80000F800007C00007C00007C00 003E00003F00181FC0780FFFF807FFF801FFE0007F80151B7E9A1A>I<00003E00003E00 003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00 003E01FC3E03FF3E0FFFBE1FFFFE1FC1FE3F007E3E003E7C003E7C003EFC003EF8003EF8 003EF8003EF8003EF8003EF8003EF8003EFC003E7C003E7C003E7E007E3F00FE1FC1FE1F FFFE0FFFBE03FE3E01F83E172A7EA91E>I<00FC0003FF0007FF800FFFC01F87E03F01E0 3E00F07C00F07C0070F80078FFFFF8FFFFF8FFFFF8FFFFF8F80000F80000F80000780000 7C00007C00003E00003F00181FC0F80FFFF807FFF801FFE0007F00151B7E9A1A>I<007C 078001FF3FC007FFFFC00FFFFFC00F83E0001F01F0001F01F0003E00F8003E00F8003E00 F8003E00F8003E00F8001F01F0001F01F0000F83E0000FFFE0001FFFC0001DFF00001C7C 00003C0000003C0000003E0000001FFFE0001FFFF8000FFFFE001FFFFF003FFFFF807E00 3F807C000FC0F80007C0F80007C0F80007C0F80007C0FC000FC07E001F803F807F001FFF FE000FFFFC0003FFF000007F80001A287F9A1D>103 D 105 D108 DII<007E0001 FF8007FFE00FFFF01F81F83F00FC3E007C7C003E7C003E78001EF8001FF8001FF8001FF8 001FF8001FF8001FF8001F7C003E7C003E7C003E3E007C3F00FC1FC3F80FFFF007FFE003 FFC0007E00181B7E9A1D>II114 D<07F8001FFF003FFF807FFF807C0780F801 00F80000F80000F80000FC00007F00007FF0003FFC001FFE0007FF0000FF80001FC0000F C00007C00007C04007C0E007C0FC1F80FFFF807FFF001FFE0007F800121B7E9A16>I<0F 80000F80000F80000F80000F80000F80000F8000FFFF80FFFF80FFFF800F80000F80000F 80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F 80000F80000F80000F80000F80800FC38007FFC007FFC003FF0001F80012227FA115>I< F800F8F800F8F800F8F800F8F800F8F800F8F800F8F800F8F800F8F800F8F800F8F800F8 F800F8F800F8F800F8F800F8F800F8F800F8F800F8F800F8F801F8F803F8FC07F8FFFFF8 7FFEF83FF8F81FC0F8151B7C9A1E>I119 D<7C001F807E003F003F003E001F007C000F80FC00 07C0F80007E1F00003E3E00001F7E00000FFC000007F8000007F0000003E0000003E0000 003F0000007F800000FFC00001F3E00003E1E00003E1F00007C0F8000F807C001F807C00 1F003E003E001F007E001F80FC000FC01A1B809A1B>II E /Fq 3 80 df<0000000000700000000000F80000000001F80000000001F80000 000003F80000000003F00000000007F00000000007E00000000007E0000000000FE00000 00000FC0000000001FC0000000001F80000000001F80000000003F80000000003F000000 00007F00000000007E00000000007E0000000000FE0000000000FC0000000001FC000000 0001F80000000001F80000000003F80000000003F00000000007F00000000007E0000000 0007E0000000000FE0000000000FC0000000001FC0000000001F80000000001F80000000 003F80000000003F00000000007F00000000007E00000000007E0000000000FE00000000 00FC0000000001FC0000000001F80000000001F80000000003F80000000003F000000000 07F00000000007E00000000007E0000000000FE0000000000FC0000000001FC000000000 1F80000000003F80000000003F00000000003F00000000007F00000000007E0000000000 FE0000000000FC0000000000FC0000000001FC0000000001F80000000003F80000000003 F00000000003F00000000007F00000000007E0000000000FE0000000000FC0000000000F C0000000001FC0000000001F80000000003F80000000003F00000000003F00000000007F 00000000007E0000000000FE0000000000FC0000000000FC0000000001FC0000000001F8 0000000003F80000000003F00000000003F00000000007F00000000007E0000000000FE0 000000000FC0000000000FC0000000001FC0000000001F80000000003F80000000003F00 000000003F00000000007F00000000007E0000000000FE0000000000FC0000000000FC00 00000000F800000000007800000000002D677ACC3A>47 D73 D<00000003FFF0000000 000000007FFFFF80000000000003FFFFFFF000000000000FFF003FFC00000000003FF800 07FF0000000000FFE00001FFC000000003FF8000007FF000000007FF0000003FF8000000 0FFE0000001FFC0000001FFC0000000FFE0000003FF800000007FF0000007FF000000003 FF800000FFF000000003FFC00001FFE000000001FFE00003FFE000000001FFF00003FFC0 00000000FFF00007FFC000000000FFF80007FF80000000007FF8000FFF80000000007FFC 000FFF80000000007FFC001FFF00000000003FFE001FFF00000000003FFE003FFF000000 00003FFF003FFF00000000003FFF003FFF00000000003FFF007FFF00000000003FFF807F FE00000000001FFF807FFE00000000001FFF807FFE00000000001FFF807FFE0000000000 1FFF80FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00 000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFF C0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE000000 00001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC0FFFE00000000001FFFC07F FE00000000001FFF807FFF00000000003FFF807FFF00000000003FFF807FFF0000000000 3FFF803FFF00000000003FFF003FFF00000000003FFF003FFF00000000003FFF001FFF80 000000007FFE001FFF80000000007FFE001FFF80000000007FFE000FFFC000000000FFFC 000FFFC000000000FFFC0007FFC000000000FFF80003FFE000000001FFF00003FFE00000 0001FFF00001FFF000000003FFE00000FFF800000007FFC000007FF800000007FF800000 7FFC0000000FFF8000003FFE0000001FFF0000000FFF0000003FFC00000007FFC00000FF F800000003FFE00001FFF000000000FFF80007FFC0000000007FFF807FFF80000000000F FFFFFFFC000000000003FFFFFFF00000000000007FFFFF8000000000000003FFF0000000 004A497AC757>79 D E /Fr 9 117 df<0001FF0000000FFFE000003FFFF80000FF83FE 0001FE00FF0003F8003F8007F8003FC00FF0001FE00FE0000FE01FE0000FF01FE0000FF0 3FE0000FF83FE0000FF83FC00007F87FC00007FC7FC00007FC7FC00007FC7FC00007FC7F C00007FCFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC0 0007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC000 07FEFFC00007FEFFC00007FEFFC00007FEFFC00007FEFFC00007FE7FC00007FC7FC00007 FC7FC00007FC7FC00007FC7FC00007FC3FE0000FF83FE0000FF83FE0000FF81FE0000FF0 1FE0000FF00FF0001FE00FF0001FE007F8003FC003FC007F8001FE00FF0000FF83FE0000 7FFFFC00001FFFF0000001FF000027397CB730>48 D<00003C000000007C00000001FC00 000007FC0000003FFC0000FFFFFC0000FFFFFC0000FFFFFC0000FFCFFC0000000FFC0000 000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC000000 0FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000F FC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC 0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC00 00000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000 000FFC0000000FFC0000000FFC0000000FFC0000000FFC0000000FFC00007FFFFFFF807F FFFFFF807FFFFFFF807FFFFFFF80213879B730>I<0000001FFE000060000003FFFFE000 E000001FFFFFF801E000007FFFFFFE03E00001FFFC00FF07E00007FFC0001FCFE0000FFF 000007FFE0001FFC000001FFE0003FF0000000FFE0007FE00000007FE000FFC00000003F E001FF800000001FE003FF000000000FE007FF000000000FE00FFE0000000007E00FFE00 00000007E01FFC0000000003E01FFC0000000003E03FF80000000003E03FF80000000001 E03FF80000000001E07FF80000000001E07FF80000000001E07FF00000000000007FF000 0000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF00000000000 00FFF0000000000000FFF0000000000000FFF0000000000000FFF0000000000000FFF000 0000000000FFF0000000000000FFF00000000000007FF00000000000007FF00000000000 007FF80000000000007FF80000000001E03FF80000000001E03FF80000000001E03FF800 00000001E01FFC0000000001E01FFC0000000003C00FFE0000000003C00FFE0000000003 C007FF00000000078003FF00000000078001FF800000000F0000FFC00000001F00007FE0 0000003E00003FF00000007C00001FFC000000F800000FFF000003F0000007FFC0000FE0 000001FFFC007F800000007FFFFFFF000000001FFFFFFC0000000003FFFFE00000000000 1FFE0000003B3D7BBB46>67 D<003FFE00000001FFFFE0000007FFFFF800000FE007FC00 000FF001FE00001FF800FF00001FF8007F80001FF8007FC0001FF8003FC0000FF0003FE0 0007E0003FE00003C0003FE0000000003FE0000000003FE0000000003FE0000000003FE0 000000FFFFE000001FFFFFE000007FF83FE00003FF803FE00007FC003FE0000FF0003FE0 001FE0003FE0003FE0003FE0007FC0003FE0007FC0003FE000FF80003FE000FF80003FE0 00FF80003FE000FF80003FE000FF80007FE0007FC0007FE0007FC000DFE0003FE0039FF0 001FF80F0FFFE007FFFE0FFFE001FFFC07FFE0003FE000FFE02B267DA52F>97 D<0001FF8000000FFFF000007FFFFC0000FF81FE0003FE007F8007F8003F800FF0001FC0 0FF0000FE01FE0000FE03FE0000FF03FE00007F07FC00007F07FC00007F87FC00007F8FF C00007F8FFC00007F8FFFFFFFFF8FFFFFFFFF8FFFFFFFFF8FFC0000000FFC0000000FFC0 000000FFC00000007FC00000007FC00000007FC00000003FE00000003FE00000781FE000 00781FF00000780FF00000F007F80001F003FC0003E001FE000FC000FFC07F80003FFFFE 00000FFFF8000000FFC00025267DA52C>101 D<00FE00000000FFFE00000000FFFE0000 0000FFFE00000000FFFE0000000007FE0000000003FE0000000003FE0000000003FE0000 000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000 000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE0000 000003FE0000000003FE00FF800003FE03FFF00003FE0FFFF80003FE1E03FC0003FE3801 FE0003FE6001FF0003FEC000FF0003FFC000FF8003FF8000FF8003FF0000FF8003FF0000 FF8003FF0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000 FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF8003FE0000FF80FFFFF83F FFFEFFFFF83FFFFEFFFFF83FFFFEFFFFF83FFFFE2F3C7CBB36>104 D<00FE01FF0000FFFE1FFFF000FFFE7FFFFC00FFFEFC07FE00FFFFF001FF0007FFC0007F 8003FF00003FC003FE00003FE003FE00001FF003FE00001FF803FE00000FF803FE00000F FC03FE00000FFC03FE000007FC03FE000007FE03FE000007FE03FE000007FE03FE000007 FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007FE03FE000007 FC03FE00000FFC03FE00000FFC03FE00000FFC03FE00000FF803FE00001FF003FE00001F F003FF00003FE003FF80007FC003FFC000FF8003FFE001FF0003FEFC07FE0003FE7FFFF8 0003FE1FFFE00003FE03FE000003FE0000000003FE0000000003FE0000000003FE000000 0003FE0000000003FE0000000003FE0000000003FE0000000003FE0000000003FE000000 0003FE0000000003FE0000000003FE00000000FFFFF8000000FFFFF8000000FFFFF80000 00FFFFF80000002F377DA536>112 D<01FC03F000FFFC0FFC00FFFC1FFF00FFFC3C3F80 FFFC707F8007FCE0FFC003FCC0FFC003FD80FFC003FD80FFC003FF807F8003FF003F0003 FF001E0003FF00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE 00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00 000003FE00000003FE00000003FE00000003FE00000003FE00000003FE00000003FE0000 0003FE000000FFFFFC0000FFFFFC0000FFFFFC0000FFFFFC000022267DA528>114 D<000F0000000F0000000F0000000F0000000F0000001F0000001F0000001F0000001F00 00003F0000003F0000007F0000007F000000FF000001FF000003FF000007FF00001FFFFF F0FFFFFFF0FFFFFFF0FFFFFFF001FF000001FF000001FF000001FF000001FF000001FF00 0001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF000001FF00 0001FF000001FF000001FF000001FF000001FF003C01FF003C01FF003C01FF003C01FF00 3C01FF003C01FF003C01FF003C00FF007800FF8078007F80F0003FC1E0001FFFC0000FFF 800001FE001E377EB626>116 D E /Fs 82 125 df<001F83E000F06E3001C078780380 F8780300F03007007000070070000700700007007000070070000700700007007000FFFF FF8007007000070070000700700007007000070070000700700007007000070070000700 700007007000070070000700700007007000070070000700700007007000070070000700 70007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007000007 0000070000070000070000070000FFFFE00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07F C3FE1720809F19>I<003FE000E0E001C1E00381E00700E00700E00700E00700E00700E0 0700E00700E00700E0FFFFE00700E00700E00700E00700E00700E00700E00700E00700E0 0700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FE7FE172080 9F19>I<001F81F80000F04F040001C07C06000380F80F000300F00F000700F00F000700 70000007007000000700700000070070000007007000000700700000FFFFFFFF00070070 070007007007000700700700070070070007007007000700700700070070070007007007 000700700700070070070007007007000700700700070070070007007007000700700700 0700700700070070070007007007007FE3FE3FF02420809F26>I<7038F87CFC7EFC7E74 3A0402040204020804080410081008201040200F0E7E9F17>34 D<007800000084000001 840000030200000702000007020000070200000702000007040000070400000708000007 0800000310000003A00FFC03C003E0038001C001C0008001C0010003E0010004E0020008 F00200187004003078080070380800701C1000F01E1000F00E2000F0074000F003C00870 03C0087801C010380670301C18386007E00F801E227EA023>38 D<70F8FCFC7404040408 0810102040060E7C9F0D>I<0020004000800100020006000C000C001800180030003000 30007000600060006000E000E000E000E000E000E000E000E000E000E000E000E0006000 600060007000300030003000180018000C000C000600020001000080004000200B2E7DA1 12>I<800040002000100008000C00060006000300030001800180018001C000C000C000 C000E000E000E000E000E000E000E000E000E000E000E000E000C000C000C001C0018001 80018003000300060006000C00080010002000400080000B2E7DA112>I<018001800180 01800180C183F18F399C0FF003C003C00FF0399CF18FC183018001800180018001801014 7DA117>I<00060000000600000006000000060000000600000006000000060000000600 0000060000000600000006000000060000000600000006000000060000FFFFFFF0FFFFFF F00006000000060000000600000006000000060000000600000006000000060000000600 000006000000060000000600000006000000060000000600001C207D9A23>I<70F8FCFC 74040404080810102040060E7C840D>II<70F8F8F87005057C84 0D>I<000100030003000600060006000C000C000C001800180018003000300030006000 60006000C000C000C00180018001800300030003000600060006000C000C000C00180018 001800300030003000600060006000C000C000C000102D7DA117>I<03F0000E1C001C0E 00180600380700700380700380700380700380F003C0F003C0F003C0F003C0F003C0F003 C0F003C0F003C0F003C0F003C0F003C0F003C0F003C07003807003807003807807803807 001806001C0E000E1C0003F000121F7E9D17>I<018003800F80F3800380038003800380 038003800380038003800380038003800380038003800380038003800380038003800380 0380038007C0FFFE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0 F803C0F803C0F803C02007C00007C0000780000780000F00000E00001C00003800007000 00600000C0000180000300000600400C00401800401000803FFF807FFF80FFFF80121E7E 9D17>I<03F0000C1C00100E00200F00780F80780780780780380F80000F80000F00000F 00000E00001C0000380003F000003C00000E00000F000007800007800007C02007C0F807 C0F807C0F807C0F00780400780400F00200E001C3C0003F000121F7E9D17>I<00060000 0600000E00000E00001E00002E00002E00004E00008E00008E00010E00020E00020E0004 0E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E00000E00000E0000 0E00000E00000E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001FF8001FE000 10000010000010000010000010000010000011F000161C00180E00100700100780000380 0003800003C00003C00003C07003C0F003C0F003C0E00380400380400700200600100E00 0C380003E000121F7E9D17>I<007C000182000701000E03800C07801C07803803003800 00780000700000700000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003 C0F003C0F003C07003C07003C07003803803803807001807000C0E00061C0001F000121F 7E9D17>I<4000007FFFC07FFF807FFF8040010080020080020080040000080000080000 100000200000200000400000400000C00000C00001C00001800003800003800003800003 8000078000078000078000078000078000078000078000030000121F7D9D17>I<03F000 0C0C001006003003002001806001806001806001807001807803003E03003F06001FC800 0FF00003F80007FC000C7E00103F00300F806003804001C0C001C0C000C0C000C0C000C0 C000806001802001001002000C0C0003F000121F7E9D17>I<03F0000E18001C0C003806 00380700700700700380F00380F00380F003C0F003C0F003C0F003C0F003C07007C07007 C03807C0180BC00E13C003E3C0000380000380000380000700300700780600780E00700C 002018001070000FC000121F7E9D17>I<70F8F8F8700000000000000000000070F8F8F8 7005147C930D>I<70F8F8F8700000000000000000000070F0F8F8780808081010102020 40051D7C930D>I<7FFFFFE0FFFFFFF00000000000000000000000000000000000000000 000000000000000000000000FFFFFFF07FFFFFE01C0C7D9023>61 D<000100000003800000038000000380000007C0000007C0000007C0000009E0000009E0 000009E0000010F0000010F0000010F00000207800002078000020780000403C0000403C 0000403C0000801E0000801E0000FFFE0001000F0001000F0001000F0002000780020007 8002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F22>65 DI<000FC040007030C001C009C0 038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C00004078000040 F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000 780000007C0000407C0000403C0000401C0000401E0000800E0000800700010003800200 01C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E0070001E0 0F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E07C0001E0 3C0001E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E0046000781820 000FE0001E217D9F24>III75 DIII<001F800000F0F00001C0380007801E000F000F00 0E0007001E0007803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0 F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E0 7C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0380000F0F000 001F80001C217D9F23>II<001F 800000F0F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C00 03E07C0003E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F800 01F0F80001F0F80001F0780001E0780001E07C0003E03C0003C03C0F03C01E1087800E20 47000F204F0007A03E0001E0380000F0F010001FB0100000301000003830000038700000 3FF000001FE000001FE000000FC0000007801C297D9F23>II<07E0800C1980100780300380600180600180E001 80E00080E00080E00080F00000F000007800007F00003FF0001FFC000FFE0003FF00001F 800007800003C00003C00001C08001C08001C08001C08001C0C00180C00380E00300F006 00CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F 0030800F0010800F0010800F0010800F0010000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F 0000000F0000000F0000000F0000000F0000000F0000001F800007FFFE001C1F7E9E21> IIII<7FF83FF80FE00FC007C0070003C0020001E0040001F00C0000F0080000781000007C 1000003C2000003E4000001E4000000F8000000F8000000780000003C0000007E0000005 E0000009F0000018F8000010780000207C0000603C0000401E0000801F0001800F000100 0780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>II91 D<080410082010201040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>I< FEFE06060606060606060606060606060606060606060606060606060606060606060606 06060606060606FEFE072D7FA10D>I<1FE000303000781800781C00300E00000E00000E 00000E0000FE00078E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867 200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E0078 0E00780E00780E00780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E00 15207F9F19>I<03F80E0C1C1E381E380C70007000F000F000F000F000F000F000700070 00380138011C020E0C03F010147E9314>I<000380003F80000380000380000380000380 00038000038000038000038000038000038003E380061B801C0780380380380380700380 700380F00380F00380F00380F00380F00380F003807003807003803803803807801C0780 0E1B8003E3F815207E9F19>I<03F0000E1C001C0E00380700380700700700700380F003 80F00380FFFF80F00000F00000F000007000007000003800801800800C010007060001F8 0011147F9314>I<007C00C6018F038F07060700070007000700070007000700FFF00700 070007000700070007000700070007000700070007000700070007000700070007007FF0 1020809F0E>I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F 00380E001C1C001E380033E0002000002000003000003000003FFE001FFF800FFFC03001 E0600070C00030C00030C00030C000306000603000C01C038003FC00141F7F9417>I<0E 0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E 3E000E43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E00 1E001C000000000000000000000000000E007E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E00FFC00A1F809E0C>I<00E001F001F001F000 E0000000000000000000000000007007F000F00070007000700070007000700070007000 700070007000700070007000700070007000700070007000706070F060F0C061803F000C 28829E0E>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E0000 0E00000E00000E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF800 0F38000E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18> I<0E00FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I< 0E1F01F000FE618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00 E00E000E00E00E000E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E 3E00FE43000E81800F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC16147F9319>I<01F800070E00 1C03803801C03801C07000E07000E0F000F0F000F0F000F0F000F0F000F0F000F07000E0 7000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00FEC3800F01C00F00 E00E00E00E00F00E00700E00780E00780E00780E00780E00780E00780E00700E00F00E00 E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E00 00FFE000151D7F9319>I<03E0800619801C05803C0780380380780380700380F00380F0 0380F00380F00380F00380F003807003807803803803803807801C0B800E138003E38000 0380000380000380000380000380000380000380000380003FF8151D7E9318>I<0E78FE 8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FF E00F147F9312>I<1F9030704030C010C010C010E00078007F803FE00FF0007080388018 8018C018C018E030D0608F800D147E9312>I<020002000200060006000E000E003E00FF F80E000E000E000E000E000E000E000E000E000E000E000E080E080E080E080E08061003 1001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01 C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E03C00603C0030DC001F1FC1614 7F9319>II< FF9FE1FC3C0780701C0300601C0380200E0380400E0380400E03C0400707C0800704C080 0704E080038861000388710003C8730001D0320001D03A0000F03C0000E01C0000E01C00 00601800004008001E147F9321>I<7FC3FC0F01E00701C007018003810001C20000E400 00EC00007800003800003C00007C00004E000087000107000303800201C00601E01E01E0 FF07FE1714809318>II<3FFF380E 200E201C40384078407000E001E001C00380078007010E011E011C0338027006700EFFFE 10147F9314>I124 D E /Ft 46 122 df<00200040008001000300060004000C000C0018001800300030003000700060006000 6000E000E000E000E000E000E000E000E000E000E000E000E000E000E000600060006000 7000300030003000180018000C000C0004000600030001000080004000200B327CA413> 40 D<800040002000100018000C000400060006000300030001800180018001C000C000 C000C000E000E000E000E000E000E000E000E000E000E000E000E000E000E000C000C000 C001C0018001800180030003000600060004000C00180010002000400080000B327DA413 >I<70F8FCFC7404040404080810102040060F7C840E>44 DI<70 F8F8F87005057C840E>I<008003800F80F3800380038003800380038003800380038003 800380038003800380038003800380038003800380038003800380038003800380038003 8007C0FFFE0F217CA018>49 D<03F0000C1C001007002007804003C04003C08003E0F003 E0F801E0F801E0F801E02003E00003E00003C00003C0000780000700000E00001C000018 0000300000600000C0000180000100000200200400200800201800603000403FFFC07FFF C0FFFFC013217EA018>I<000200000600000E00000E00001E00001E00002E00004E0000 4E00008E00008E00010E00020E00020E00040E00040E00080E00100E00100E00200E0020 0E00400E00800E00FFFFF8000E00000E00000E00000E00000E00000E00000E00001F0001 FFF015217FA018>52 D<1000801E07001FFF001FFE001FF80013E0001000001000001000 0010000010000010000010F800130E001407001803801003800001C00001C00001E00001 E00001E00001E07001E0F001E0F001E0E001C08001C04003C04003802007001006000C1C 0003F00013227EA018>I<4000006000007FFFE07FFFC07FFFC0400080C0010080010080 020080020000040000080000080000100000300000200000600000600000600000E00000 C00000C00001C00001C00001C00001C00003C00003C00003C00003C00003C00003C00003 C00003C00001800013237DA118>55 D<01F800060E000803001001802001802000C06000 C06000C06000C07000C07801803E01003F02001FC4000FF80003F80003FC00067F00083F 80100F803007C06001C06000E0C000E0C00060C00060C00060C000606000406000C03000 801803000E0E0003F00013227EA018>I<01F000060C000C060018070038038070038070 0380F001C0F001C0F001C0F001E0F001E0F001E0F001E0F001E07001E07003E03803E018 05E00C05E00619E003E1E00001C00001C00001C000038000038030030078070078060070 0C002018001030000FC00013227EA018>I<0001800000018000000180000003C0000003 C0000003C0000005E0000005E000000DF0000008F0000008F0000010F800001078000010 780000203C0000203C0000203C0000401E0000401E0000401E0000800F0000800F0000FF FF000100078001000780030007C0020003C0020003C0040003E0040001E0040001E00C00 00F00C0000F03E0001F8FF800FFF20237EA225>65 D<0007E0100038183000E0063001C0 0170038000F0070000F00E0000701E0000701C0000303C0000303C0000307C0000107800 001078000010F8000000F8000000F8000000F8000000F8000000F8000000F8000000F800 000078000000780000107C0000103C0000103C0000101C0000201E0000200E0000400700 00400380008001C0010000E0020000381C000007E0001C247DA223>67 DI< FFFFFFC00F8007C0078001C0078000C00780004007800040078000600780002007800020 0780002007802020078020000780200007802000078060000780E00007FFE0000780E000 078060000780200007802000078020000780200807800008078000080780001007800010 07800010078000300780003007800070078000E00F8003E0FFFFFFE01D227EA121>II72 DI 77 DI 80 D82 D<03F0200C0C601802603001E07000E0600060E00060E00060E00020E00020 E00020F00000F000007800007F00003FF0001FFE000FFF0003FF80003FC00007E00001E0 0000F00000F0000070800070800070800070800070C00060C00060E000C0F000C0C80180 C6070081FC0014247DA21B>I<7FFFFFF878078078600780184007800840078008400780 08C007800C80078004800780048007800480078004000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000780 000007800000078000000780000007800000078000000780000007800000078000000FC0 0003FFFF001E227EA123>I<0FE0001838003C0C003C0E00180700000700000700000700 00FF0007C7001E07003C0700780700700700F00708F00708F00708F00F087817083C2390 0FC1E015157E9418>97 D<0E0000FE00001E00000E00000E00000E00000E00000E00000E 00000E00000E00000E00000E00000E00000E1F000E61C00E80600F00300E00380E003C0E 001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E001C0E003C0E00380F00700C 80600C41C0083F0017237FA21B>I<01FE000703000C07801C0780380300780000700000 F00000F00000F00000F00000F00000F00000F000007000007800403800401C00800C0100 07060001F80012157E9416>I<0000E0000FE00001E00000E00000E00000E00000E00000 E00000E00000E00000E00000E00000E00000E001F8E00704E00C02E01C01E03800E07800 E07000E0F000E0F000E0F000E0F000E0F000E0F000E0F000E07000E07800E03800E01801 E00C02E0070CF001F0FE17237EA21B>I<01FC000707000C03801C01C03801C07801E070 00E0F000E0FFFFE0F00000F00000F00000F00000F000007000007800203800201C00400E 008007030000FC0013157F9416>I<003C00C6018F038F030F0700070007000700070007 00070007000700FFF8070007000700070007000700070007000700070007000700070007 00070007000700070007807FF8102380A20F>I<00007001F198071E180E0E181C07001C 07003C07803C07803C07803C07801C07001C07000E0E000F1C0019F00010000010000018 00001800001FFE000FFFC00FFFE03800F0600030400018C00018C00018C0001860003060 00303800E00E038003FE0015217F9518>I<0E0000FE00001E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E00000E1F800E60C00E80E00F0070 0F00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E0070 0E00700E00700E00700E0070FFE7FF18237FA21B>I<1C001E003E001E001C0000000000 0000000000000000000000000E00FE001E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E00FFC00A227FA10E>I<01C003E003E003E001C000 00000000000000000000000000000001E00FE001E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0C0F1 8061803E000B2C82A10F>I<0E0000FE00001E00000E00000E00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E03FC0E01F00E01C00E01800E02000E0400 0E08000E10000E38000EF8000F1C000E1E000E0E000E07000E07800E03C00E01C00E01E0 0E00F00E00F8FFE3FE17237FA21A>I<0E1FC07F00FE60E183801E807201C00F003C00E0 0F003C00E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E 003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E003800E00E00 3800E00E003800E0FFE3FF8FFE27157F942A>109 D<0E1F80FE60C01E80E00F00700F00 700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00 700E00700E00700E0070FFE7FF18157F941B>I<01FC000707000C01801800C03800E070 0070700070F00078F00078F00078F00078F00078F00078F000787000707800F03800E01C 01C00E038007070001FC0015157F9418>I<0E1F00FE61C00E80600F00700E00380E003C 0E001C0E001E0E001E0E001E0E001E0E001E0E001E0E001E0E003C0E003C0E00380F0070 0E80E00E41C00E3F000E00000E00000E00000E00000E00000E00000E00000E00000E0000 FFE000171F7F941B>I<0E3CFE461E8F0F0F0F060F000E000E000E000E000E000E000E00 0E000E000E000E000E000E000F00FFF010157F9413>114 D<0F8830786018C018C008C0 08E008F0007F803FE00FF001F8003C801C800C800CC00CC008E018D0308FC00E157E9413 >I<02000200020002000600060006000E001E003E00FFF80E000E000E000E000E000E00 0E000E000E000E000E000E040E040E040E040E040E040708030801F00E1F7F9E13>I<0E 0070FE07F01E00F00E00700E00700E00700E00700E00700E00700E00700E00700E00700E 00700E00700E00700E00700E00F00E00F006017003827800FC7F18157F941B>I119 D121 D E /Fu 20 118 df45 D68 D73 D77 D80 D<007F802001FFE02007C078600F001C601E00 06E03C0003E0380001E0780000E0700000E070000060F0000060F0000060F0000020F000 0020F0000020F8000020F80000007C0000007E0000003F0000003FC000001FF800000FFF 800007FFF80003FFFC0000FFFF00000FFF800000FFC000001FE0000007E0000003F00000 01F0000000F0000000F8000000F88000007880000078800000788000007880000078C000 0078C0000070E00000F0E00000E0F00000E0F80001C0EC000380C7000700C1F01E00807F FC00800FF0001D337CB125>83 D<00FE00000303C0000C00E00010007000100038003C00 3C003E001C003E001E003E001E0008001E0000001E0000001E0000001E00000FFE0000FC 1E0003E01E000F801E001F001E003E001E003C001E007C001E00F8001E04F8001E04F800 1E04F8003E04F8003E0478003E047C005E043E008F080F0307F003FC03E01E1F7D9E21> 97 D<003F8000E0600380180700040F00041E001E1C003E3C003E7C003E7C0008780000 F80000F80000F80000F80000F80000F80000F80000F80000F800007800007C00007C0000 3C00011E00011E00020F000207000403801800E060003F80181F7D9E1D>99 D<000001E000003FE000003FE0000003E0000001E0000001E0000001E0000001E0000001 E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001E0000001 E0000001E0001F81E000F061E001C019E0078005E00F0003E00E0003E01E0001E03C0001 E03C0001E07C0001E0780001E0F80001E0F80001E0F80001E0F80001E0F80001E0F80001 E0F80001E0F80001E0F80001E0780001E0780001E03C0001E03C0001E01C0001E01E0003 E00E0005E0070009E0038011F000E061FF003F81FF20327DB125>I<003F800000E0E000 0380380007003C000E001E001E001E001C000F003C000F007C000F0078000F8078000780 F8000780F8000780FFFFFF80F8000000F8000000F8000000F8000000F8000000F8000000 780000007C0000003C0000003C0000801E0000800E0001000F0002000780020001C00C00 00F03000001FC000191F7E9E1D>I<0007E0001C1000383800707C00E07C01E07C01C038 03C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C000 FFFFC0FFFFC003C00003C00003C00003C00003C00003C00003C00003C00003C00003C000 03C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C000 03C00003C00003C00003C00007E0007FFF007FFF0016327FB114>I<000000F0007F0308 01C1C41C0380E81C070070080F0078001E003C001E003C003E003E003E003E003E003E00 3E003E003E003E003E003E001E003C001E003C000F007800070070000780E00009C1C000 087F000018000000180000001800000018000000180000001C0000000E0000000FFFF800 07FFFF0003FFFF800E000FC0180001E0300000F070000070E0000038E0000038E0000038 E0000038E00000387000007070000070380000E01C0001C00700070001C01C00003FE000 1E2F7E9F21>I<07000F801F801F800F8007000000000000000000000000000000000000 00000000000780FF80FF800F800780078007800780078007800780078007800780078007 800780078007800780078007800780078007800780078007800FC0FFF8FFF80D307EAF12 >105 D<0780FE001FC000FF83078060F000FF8C03C18078000F9001E2003C0007A001E4 003C0007A000F4001E0007C000F8001E0007C000F8001E00078000F0001E00078000F000 1E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E 00078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E00 078000F0001E00078000F0001E00078000F0001E00078000F0001E00078000F0001E0007 8000F0001E00078000F0001E00078000F0001E000FC001F8003F00FFFC1FFF83FFF0FFFC 1FFF83FFF0341F7E9E38>109 D<0780FE0000FF83078000FF8C03C0000F9001E00007A0 01E00007A000F00007C000F00007C000F000078000F000078000F000078000F000078000 F000078000F000078000F000078000F000078000F000078000F000078000F000078000F0 00078000F000078000F000078000F000078000F000078000F000078000F000078000F000 078000F000078000F0000FC001F800FFFC1FFF80FFFC1FFF80211F7E9E25>I<001FC000 00F0780001C01C00070007000F0007801E0003C01C0001C03C0001E03C0001E0780000F0 780000F0780000F0F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8F80000F8 F80000F8780000F07C0001F03C0001E03C0001E01E0003C01E0003C00F00078007800F00 01C01C0000F07800001FC0001D1F7E9E21>I<0783E0FF8C18FF907C0F907C07A07C07C0 3807C00007C00007C0000780000780000780000780000780000780000780000780000780 000780000780000780000780000780000780000780000780000780000780000FC000FFFE 00FFFE00161F7E9E19>114 D<01FC100E03301800F0300070600030E00030E00010E000 10E00010F00010F800007E00003FF0001FFF000FFFC003FFE0003FF00001F80000F88000 3C80003C80001CC0001CC0001CE0001CE00018F00038F00030CC0060C301C080FE00161F 7E9E1A>I<00400000400000400000400000400000C00000C00000C00001C00001C00003 C00007C0000FC0001FFFE0FFFFE003C00003C00003C00003C00003C00003C00003C00003 C00003C00003C00003C00003C00003C00003C00003C00003C00003C01003C01003C01003 C01003C01003C01003C01003C01001C02001E02000E0400078C0001F00142C7FAB19>I< 078000F000FF801FF000FF801FF0000F8001F000078000F000078000F000078000F00007 8000F000078000F000078000F000078000F000078000F000078000F000078000F0000780 00F000078000F000078000F000078000F000078000F000078000F000078000F000078000 F000078000F000078001F000078001F000078001F000038002F00003C004F00001C008F8 00007030FF80001FC0FF80211F7E9E25>I E /Fv 5 85 df<00000000600000000000E0 0000000000E00000000001E00000000003E00000000003E00000000007E00000000007F0 000000000DF0000000000DF00000000019F00000000039F00000000031F00000000061F0 0000000061F000000000C1F000000000C1F00000000181F00000000381F00000000301F0 0000000601F00000000601F00000000C01F80000000C00F80000001800F80000003800F8 0000003000F80000006000F80000006000F8000000C000F8000000C000F8000001FFFFF8 000001FFFFF80000030000F80000060000F80000060000F800000C0000F800000C0000FC 00001800007C00001800007C00003000007C00007000007C00006000007C0000C000007C 0000C000007C0001C000007C0003C000007C000FE00000FE00FFF8001FFFE0FFF8001FFF E02B327CB134>65 D<000FFFFFFC0000000FFFFFFF800000007E000FE00000007C0003F0 0000007C0000F80000007C00007C0000007C00007C000000F800003E000000F800001E00 0000F800001F000000F800001F000001F000001F000001F000000F800001F000000F8000 01F000000F800003E000000F800003E000000F800003E000000F800003E000001F800007 C000001F800007C000001F800007C000001F800007C000001F80000F8000001F00000F80 00003F00000F8000003F00000F8000003F00001F0000007E00001F0000007E00001F0000 007E00001F0000007C00003E000000FC00003E000000F800003E000001F800003E000001 F000007C000003E000007C000003E000007C000007C000007C00000F800000F800000F00 0000F800001E000000F800003C000000F8000078000001F00001F0000001F00003E00000 01F0000F80000003F0007F000000FFFFFFFC000000FFFFFFC000000031317BB035>68 D<000FFFFFFFF8000FFFFFFFF800007E0003F800007C0000F800007C00007800007C0000 7800007C0000300000F80000300000F80000300000F80000300000F80000300001F00000 300001F00000300001F00000300001F00000600003E00180600003E00180000003E00180 000003E00180000007C00300000007C00300000007C00700000007C01F0000000FFFFE00 00000FFFFE0000000F801E0000000F800E0000001F000C0000001F000C0000001F000C00 00001F000C0000003E00180000003E00180000003E00000000003E00000000007C000000 00007C00000000007C00000000007C0000000000F80000000000F80000000000F8000000 0000F80000000001F00000000001F00000000001F00000000003F000000000FFFFE00000 00FFFFE00000002D317BB02E>70 D<000FFFFFE000000FFFFFFC0000007E003F0000007C 000F8000007C0007C000007C0003E000007C0003E00000F80001F00000F80001F00000F8 0001F00000F80003F00001F00003F00001F00003F00001F00003F00001F00003E00003E0 0007E00003E00007C00003E0000FC00003E0000F800007C0001F000007C0003E000007C0 0078000007C001E000000F800F8000000FFFFC0000000F800E0000000F80070000001F00 038000001F0003C000001F0001E000001F0001E000003E0001F000003E0001F000003E00 01F000003E0001F000007C0003F000007C0003F000007C0003E000007C0003E00000F800 07E00000F80007E00000F80007E00000F80007E01801F00007E03001F00007E03001F000 07E02003F00003E060FFFF8001E0C0FFFF8000F180000000003E002D327BB033>82 D<07FFFFFFFFF007FFFFFFFFF00FE007E007F00F8007C001F00E0007C000E00C0007C000 E01C0007C000E018000F8000E018000F8000E030000F8000C030000F8000C030001F0000 C060001F0000C060001F0000C060001F0000C0C0003E00018000003E00000000003E0000 0000003E00000000007C00000000007C00000000007C00000000007C0000000000F80000 000000F80000000000F80000000000F80000000001F00000000001F00000000001F00000 000001F00000000003E00000000003E00000000003E00000000003E00000000007C00000 000007C00000000007C00000000007C0000000000F80000000000F80000000000F800000 00000F80000000001F00000000001F00000000001F00000000003F000000003FFFFF0000 003FFFFF0000002C3174B032>84 D E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300dpi TeXDict begin %%EndSetup %%Page: 45 1 45 46 bop 75 -100 a Fj(10.6.)34 b(CONSISTENCY)16 b(AND)f(SEMANTICS)886 b Fs(45)75 49 y Fn(10.5.4)49 b(Matching)18 b(Data)e(Rep)o(resentations) 75 135 y Fs(It)f(is)f(the)h(user's)f(resp)q(onsibilit)o(y)j(to)d (ensure)h(that)f(the)g(data)g(represen)o(tation)h(used)g(to)e(read)i (data)f(from)75 192 y(a)j(\014le)g(is)h Fl(c)n(omp)n(atible)e Fs(with)h(the)g(data)f(represen)o(tation)h(whic)o(h)h(w)o(as)e(used)h (to)f(write)h(that)g(data)f(to)g(the)75 248 y(\014le.)166 304 y(In)i(general,)g(using)g(the)f(same)g(data)g(represen)o(tation)g (name)h(when)g(writing)f(and)h(reading)g(a)f(\014le)75 361 y(do)q(es)g(not)g(guaran)o(tee)f(that)g(the)h(represen)o(tation)g (is)g(compatible.)26 b(Similarly)l(,)19 b(using)f(di\013eren)o(t)f (repre-)75 417 y(sen)o(tation)e(names)g(on)g(t)o(w)o(o)f(di\013eren)o (t)i(implemen)o(tations)g(ma)o(y)e(yield)j(compatible)g(represen)o (tations.)166 474 y(Compatibilit)o(y)23 b(is)e(guaran)o(teed)g(when)h (\\external32")f(represen)o(tation)g(is)h(used)g(on)f(b)q(oth)h(sys-)75 530 y(tems,)14 b(although)h(precision)h(ma)o(y)d(b)q(e)j(lost)e(and)h (the)f(p)q(erformance)h(ma)o(y)f(b)q(e)h(less)g(than)f(when)h(\\nativ)o (e")75 587 y(represen)o(tation)g(is)h(used.)166 643 y(User)h(de\014ned) h(datat)o(yp)q(es)e(ma)o(y)g(b)q(e)h(used)g(to)f(pro)o(vide)h(an)g (implemen)o(tation)h(compatiblit)o(y)g(with)75 700 y(another)d (implemen)o(tation's)h(\\nativ)o(e")f(or)f(\\in)o(ternal")i(represen)o (tation.)189 802 y Fl(A)n(dvic)n(e)f(to)h(users.)40 b Fs(The)15 b Fm(MPI)f Fs(F)l(ortran)g(binding)j(\()p Fo(??)o Fs(\))e(pro)o(vides)g(some)g(supp)q(ort)g(for)f(matc)o(hing)189 858 y(datat)o(yp)q(es)g(in)i(heterogeneous)g(en)o(vironmen)o(ts.)k(\()p Fl(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fs(\))75 1001 y Fp(10.6)60 b(Consistency)20 b(and)g(Semantics)75 1104 y Fn(10.6.1)49 b(File)17 b(Consistency)75 1190 y Fs(Consistency)23 b(seman)o(tics)f(de\014ne)h(the)g(outcome)e(of)h(m)o(ultiple)i (accesses)f(to)e(a)h(single)i(\014le.)41 b(All)24 b(\014le)75 1246 y(accesses)19 b(in)g Fm(MPI)f Fs(are)h(relativ)o(e)g(to)f(a)g(sp)q (eci\014c)j(\014le)e(handle)h(created)f(from)f(a)g(collectiv)o(e)j(op)q (en.)31 b Fm(MPI)75 1303 y Fs(pro)o(vides)13 b(three)g(lev)o(els)h(of)f (consistency:)19 b(sequen)o(tial)14 b(consistency)g(among)e(all)i (accesses)f(using)g(a)g(single)75 1359 y(\014le)g(handle,)h(sequen)o (tial)f(consistency)g(among)f(all)h(accesses)g(using)f(\014le)i (handles)f(created)f(from)g(a)g(single)75 1416 y(collectiv)o(e)20 b(op)q(en)f(with)g(atomic)f(mo)q(de)h(enabled,)h(and)f(user-imp)q(osed) h(consistency)f(among)f(accesses)p Fi(>)e Fk(\(Fin2\))75 1472 y Fs(other)e(than)f(the)i(ab)q(o)o(v)o(e.)k(Sequen)o(tial)c (consistency)g(means)f(the)g(b)q(eha)o(vior)h(of)e(a)h(set)g(of)f(op)q (erations)h(will)1875 1474 y Fi(?)i Fk(\(Fin2\))75 1529 y Fs(b)q(e)g(as)f(if)g(the)h(op)q(erations)f(w)o(ere)g(p)q(erformed)h (in)g(some)e(serial)j(order)e(consisten)o(t)g(with)g(program)g(order;) 75 1585 y(eac)o(h)j(access)g(app)q(ears)g(atomic,)g(although)h(the)f (exact)f(ordering)i(of)e(accesses)h(is)h(unsp)q(eci\014ed.)31 b(User-)p Fi(>)16 b Fk(\(Fin2\))75 1642 y Fs(imp)q(osed)k(consistency)f (ma)o(y)f(b)q(e)h(obtained)g(using)g(program)f(order)g(and)h(calls)g (to)f Fm(MPI)p 1626 1642 14 2 v 16 w(FILE)p 1733 1642 V 16 w(SYNC)p Fs(.)1875 1698 y Fi(?)e Fk(\(Fin2\))1875 1757 y Fi(>)g Fk(\(Fin1\))166 1754 y Fs(Let)i Fe(F)6 b(H)323 1761 y Fd(1)361 1754 y Fs(b)q(e)18 b(the)g(set)f(of)g(\014le)i (handles)g(created)f(from)f(one)g(particular)i(collectiv)o(e)g(op)q(en) g(of)e(the)75 1811 y(\014le)j Fe(F)6 b(O)q(O)q Fs(,)21 b(and)e Fe(F)6 b(H)458 1818 y Fd(2)498 1811 y Fs(b)q(e)20 b(the)f(set)g(of)g(\014le)h(handles)h(created)e(from)g(a)g(di\013eren)o (t)g(collectiv)o(e)i(op)q(en)f(of)75 1867 y Fe(F)6 b(O)q(O)q Fs(.)31 b(Note)18 b(that)h(nothing)g(restrictiv)o(e)g(is)g(said)h(ab)q (out)e Fe(F)6 b(H)1181 1874 y Fd(1)1220 1867 y Fs(and)19 b Fe(F)6 b(H)1385 1874 y Fd(2)1405 1867 y Fs(:)27 b(the)19 b(sizes)h(of)e Fe(F)6 b(H)1763 1874 y Fd(1)1802 1867 y Fs(and)75 1924 y Fe(F)g(H)148 1931 y Fd(2)181 1924 y Fs(ma)o(y)12 b(b)q(e)i(di\013eren)o(t,)f(the)h(groups)e(of)h(pro)q (cesses)g(used)h(for)e(eac)o(h)i(op)q(en)f(ma)o(y)g(or)f(ma)o(y)g(not)h (in)o(tersect,)75 1980 y(the)k(\014le)h(handles)g(in)g Fe(F)6 b(H)524 1987 y Fd(1)561 1980 y Fs(ma)o(y)16 b(b)q(e)h(destro)o (y)o(ed)g(b)q(efore)g(those)f(in)i Fe(F)6 b(H)1316 1987 y Fd(2)1353 1980 y Fs(are)16 b(created,)h(etc.)25 b(Consider)75 2037 y(the)18 b(follo)o(wing)h(three)f(cases:)25 b(a)18 b(single)h(\014le)g(handle)g(\(e.g.,)e Fe(f)5 b(h)1172 2044 y Fd(1)1210 2037 y Fi(2)17 b Fe(F)6 b(H)1330 2044 y Fd(1)1350 2037 y Fs(\),)18 b(t)o(w)o(o)f(\014le)i(handles)g(created) 75 2093 y(from)13 b(a)g(single)h(collectiv)o(e)i(op)q(en)e(\(e.g.,)e Fe(f)5 b(h)810 2100 y Fd(1)p Fc(a)861 2093 y Fi(2)13 b Fe(F)6 b(H)977 2100 y Fd(1)1011 2093 y Fs(and)13 b Fe(f)5 b(h)1150 2100 y Fd(1)p Fc(b)1198 2093 y Fi(2)13 b Fe(F)6 b(H)1314 2100 y Fd(1)1334 2093 y Fs(\),)13 b(and)h(t)o(w)o(o)e (\014le)i(handles)h(from)75 2150 y(di\013eren)o(t)g(collectiv)o(e)i(op) q(ens)f(\(e.g.,)e Fe(f)5 b(h)746 2157 y Fd(1)778 2150 y Fi(2)13 b Fe(F)6 b(H)894 2157 y Fd(1)930 2150 y Fs(and)15 b Fe(f)5 b(h)1071 2157 y Fd(2)1104 2150 y Fi(2)13 b Fe(F)6 b(H)1220 2157 y Fd(2)1240 2150 y Fs(\).)604 b Fi(?)16 b Fk(\(Fin1\))1875 2208 y Fi(>)g Fk(\(Fin2\))166 2206 y Fs(F)l(or)d(the)h(purp)q(ose)h(of)e(consistency)i(seman)o(tics,)f(a)f (matc)o(hed)h(pair)g(\()p Fl(cf.)19 b Fs(10.4.5\))12 b(of)h(split)i(collectiv)o(e)75 2263 y(data)h(access)h(op)q(erations)f (\(e.g.,)g Fm(MPI)p 739 2263 V 15 w(FILE)p 845 2263 V 16 w(READ)p 980 2263 V 17 w(ALL)p 1077 2263 V 16 w(BEGIN)h Fs(and)g Fm(MPI)p 1414 2263 V 16 w(FILE)p 1521 2263 V 15 w(READ)p 1655 2263 V 17 w(ALL)p 1752 2263 V 16 w(END)p Fs(\))75 2319 y(comp)q(ose)j(a)g(single)i(data)e(access)g(op)q (eration.)36 b(Similarly)l(,)23 b(a)d(pair)h(of)f(a)g(non-blo)q(c)o (king)i(data)e(access)75 2375 y(routine)14 b(\(e.g.,)f Fm(MPI)p 428 2375 V 16 w(FILE)p 535 2375 V 15 w(IREAD)p Fs(\))h(and)g(a)f(matc)o(hing)h(request)g(complete)h(routine)f(\(e.g.,) f Fm(MPI)p 1732 2375 V 15 w(W)l(AIT)p Fs(\))75 2432 y(also)j(comp)q (ose)g(a)g(single)h(data)f(access)g(op)q(eration.)23 b(The)16 b(request)g(complete)h(routine)g(m)o(ust)e(complete)75 2488 y(the)i(matc)o(hing)g(data)g(access)g(op)q(eration.)25 b(F)l(or)17 b(all)h(cases)f(b)q(elo)o(w,)g(these)h(data)e(access)h(op)q (erations)g(are)75 2545 y(sub)s(ject)e(to)g(the)g(same)g(constrain)o (ts)g(as)f(blo)q(c)o(king)j(data)d(access)i(op)q(erations.)189 2647 y Fl(A)n(dvic)n(e)h(to)h(users.)52 b Fs(F)l(or)17 b(a)f Fm(MPI)p 768 2647 V 16 w(FILE)p 875 2647 V 16 w(IREAD)h Fs(and)h Fm(MPI)p 1213 2647 V 15 w(W)l(AIT)f Fs(pair,)h(the)f(op)q (eration)h(b)q(egins)189 2704 y(when)c Fm(MPI)p 391 2704 V 16 w(FILE)p 498 2704 V 16 w(IREAD)g Fs(is)g(called)i(and)e(ends)h (when)g Fm(MPI)p 1222 2704 V 15 w(W)l(AIT)f Fs(returns.)20 b(\()p Fl(End)14 b(of)i(advic)n(e)f(to)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 46 2 46 47 bop 75 -100 a Fs(46)1342 b Fj(CHAPTER)15 b(10.)35 b(I/O)189 49 y Fl(users.)p Fs(\))166 156 y(Assume)19 b(that)f Fe(A)474 163 y Fd(1)513 156 y Fs(and)h Fe(A)639 163 y Fd(2)678 156 y Fs(are)g(t)o(w)o(o)e(data)i(access)g(op)q (erations.)31 b(Let)19 b Fe(D)1466 163 y Fd(1)1504 156 y Fs(\()p Fe(D)1560 163 y Fd(2)1579 156 y Fs(\))g(b)q(e)h(the)f(set)f (of)-1992 b Fi(?)15 b Fk(\(Fin2\))-117 214 y Fi(>)g Fk(\(Fin1\))75 212 y Fs(absolute)22 b(b)o(yte)f(displacemen)o(ts)i(of)e(ev)o(ery)g(b)o (yte)g(accessed)h(in)g Fe(A)1234 219 y Fd(1)1276 212 y Fs(\()p Fe(A)1328 219 y Fd(2)1347 212 y Fs(\).)38 b(The)22 b(t)o(w)o(o)e(data)h(accesses)75 268 y Fl(overlap)15 b Fs(if)h Fe(D)312 275 y Fd(1)342 268 y Fi(\\)10 b Fe(D)420 275 y Fd(2)452 268 y Fi(6)p Fs(=)j Fi(;)p Fs(.)-117 272 y Fi(?)i Fk(\(Fin1\))166 325 y Fs(Let)i Fe(S)s(E)s(Q)353 332 y Fc(f)t(h)411 325 y Fs(b)q(e)h(a)e(sequence)i(of)f(\014le)h(op)q (erations)f(on)f(a)h(single)h(\014le)g(handle,)g(brac)o(k)o(eted)f(b)o (y)75 381 y Fm(MPI)p 160 381 14 2 v 16 w(FILE)p 267 381 V 15 w(SYNC)p Fs(s)j(of)e(that)g(\014le)i(handle.)31 b(\(Both)18 b(op)q(ening)i(and)f(closing)g(a)g(\014le)g(implicitl)q(y)i (p)q(erform)-1992 b Fi(>)15 b Fk(\(Fin2\))75 438 y Fs(an)i Fm(MPI)p 225 438 V 15 w(FILE)p 331 438 V 16 w(SYNC)p Fs(.\))g Fe(S)s(E)s(Q)612 445 y Fc(f)t(h)670 438 y Fs(is)g(a)g(\\write) f(sequence")i(if)f(an)o(y)f(of)h(the)f(data)g(access)h(op)q(erations)g (in)-117 440 y Fi(?)e Fk(\(Fin2\))75 494 y Fs(the)f(sequence)h(are)f (writes)g(or)g(if)g(an)o(y)g(of)g(the)g(\014le)h(manipulation)g(op)q (erations)g(in)g(the)f(sequence)h(c)o(hange)75 551 y(the)22 b(state)g(of)f(the)i(\014le)g(\(i.e.,)g Fm(MPI)p 700 551 V 16 w(FILE)p 807 551 V 15 w(SET)p 905 551 V 17 w(SIZE)f Fs(or)g Fm(MPI)p 1182 551 V 15 w(FILE)p 1288 551 V 16 w(PREALLOCA)l(TE)p Fs(\).)h(Giv)o(en)f(t)o(w)o(o)-1993 b Fi(>)15 b Fk(\(Fin2\))-117 609 y Fi(?)g Fk(\(Fin2\))75 607 y Fs(sequences,)h Fe(S)s(E)s(Q)398 614 y Fd(1)431 607 y Fs(and)f Fe(S)s(E)s(Q)623 614 y Fd(2)642 607 y Fs(,)f(w)o(e)h(sa)o(y)f(they)h(are)g(not)g Fl(c)n(oncurr)n(ent)f Fs(if)i(one)f(sequence)h(is)f(guaran)o(teed)-117 668 y Fi(>)g Fk(\(Fin1\))75 664 y Fs(to)g(completely)h(precede)g(the)g (other)e(\(temp)q(orally\).)-117 726 y Fi(?)h Fk(\(Fin1\))-117 785 y Fi(>)g Fk(\(Fin2\))75 784 y Fm(Case)k(1:)25 b Fe(f)5 b(h)296 791 y Fd(1)333 784 y Fi(2)18 b Fe(F)6 b(H)454 791 y Fd(1)519 784 y Fs(All)20 b(op)q(erations)e(on)g Fe(f)5 b(h)940 791 y Fd(1)978 784 y Fs(are)18 b(sequen)o(tially)i (consisten)o(t)e(if)g(atomic)g(mo)q(de)g(is)75 840 y(set.)h(If)13 b(non-atomic)h(mo)q(de)f(is)h(set,)f(then)h(all)g(op)q(erations)f(on)h Fe(f)5 b(h)1185 847 y Fd(1)1218 840 y Fs(are)13 b(sequen)o(tially)i (consisten)o(t)e(if)h(they)75 897 y(are)h(non-concurren)o(t,)g(non-o)o (v)o(erlapping,)h(or)f(b)q(oth.)-1101 b Fi(?)15 b Fk(\(Fin2\))-117 955 y Fi(>)g Fk(\(Fin1\))75 1017 y Fm(Case)i(2:)23 b Fe(f)5 b(h)292 1024 y Fd(1)p Fc(a)347 1017 y Fi(2)15 b Fe(F)6 b(H)465 1024 y Fd(1)502 1017 y Fm(and)18 b Fe(f)5 b(h)641 1024 y Fd(1)p Fc(b)692 1017 y Fi(2)15 b Fe(F)6 b(H)810 1024 y Fd(1)876 1017 y Fs(Assume)17 b Fe(A)1080 1024 y Fd(1)1117 1017 y Fs(is)g(a)g(data)f(access)h(op)q(eration)g (using)h Fe(f)5 b(h)1824 1024 y Fd(1)p Fc(a)1862 1017 y Fs(,)75 1073 y(and)14 b Fe(A)196 1080 y Fd(2)230 1073 y Fs(is)h(a)f(data)f(access)i(op)q(eration)f(using)h Fe(f)5 b(h)921 1080 y Fd(1)p Fc(b)956 1073 y Fs(.)20 b(If)14 b Fe(A)1067 1080 y Fd(1)1101 1073 y Fs(and)h Fe(A)1223 1080 y Fd(2)1256 1073 y Fs(do)g(not)e(o)o(v)o(erlap,)h Fm(MPI)g Fs(guaran)o(tees)-1993 b Fi(>)15 b Fk(\(Fin2\))-117 1132 y Fi(?)g Fk(\(Fin2\))75 1130 y Fs(that)f(they)i(are)f(sequen)o (tially)i(consisten)o(t,)e(and)g(hence,)h(app)q(ear)f(to)g(the)g (application)i(as)e(atomic.)166 1186 y(Ho)o(w)o(ev)o(er,)d(unlik)o(e)i (UNIX,)f(the)g(default)g(seman)o(tics)g(for)f(o)o(v)o(erlapping)h (accesses)g(do)f(not)h(guaran)o(tee)75 1242 y(sequen)o(tial)k (consistency)l(.)24 b(If)17 b Fe(A)625 1249 y Fd(1)661 1242 y Fs(and)f Fe(A)784 1249 y Fd(2)820 1242 y Fs(o)o(v)o(erlap,)g (sequen)o(tial)h(consistency)g(can)f(b)q(e)h(guaran)o(teed)f(b)o(y)75 1299 y(either)k(enabling)g(atomic)f(mo)q(de)h(via)f(the)g Fm(MPI)p 918 1299 V 16 w(FILE)p 1025 1299 V 16 w(SET)p 1124 1299 V 16 w(A)l(TOMICITY)h Fs(routine,)g(or)e(meeting)i(the)75 1355 y(condition)c(describ)q(ed)i(in)e(case)f(3)g(b)q(elo)o(w.)-906 b Fi(>)15 b Fk(\(Fin2\))-117 1414 y Fi(?)g Fk(\(Fin2\))-117 1472 y Fi(?)g Fk(\(Fin1\))-117 1531 y Fi(>)g Fk(\(Fin1\))-117 1590 y Fi(?)g Fk(\(Fin1\))75 1475 y Fm(Case)g(3:)k Fe(f)5 b(h)286 1482 y Fd(1)318 1475 y Fi(2)13 b Fe(F)6 b(H)434 1482 y Fd(1)468 1475 y Fm(and)15 b Fe(f)5 b(h)604 1482 y Fd(2)637 1475 y Fi(2)13 b Fe(F)6 b(H)753 1482 y Fd(2)818 1475 y Fs(Consider)15 b(access)f(to)g(a)g(single)h(\014le)g(using)g (\014le)g(handles)h(from)75 1532 y(distinct)23 b(collectiv)o(e)g(op)q (ens.)40 b(In)22 b(order)f(to)g(guaran)o(tee)g(sequen)o(tial)i (consistency)l(,)h Fm(MPI)p 1639 1532 V 16 w(FILE)p 1746 1532 V 15 w(SYNC)-117 1648 y Fi(>)15 b Fk(\(Fin1\))-117 1707 y Fi(?)g Fk(\(Fin1\))-117 1765 y Fi(>)g Fk(\(Fin2\))-117 1824 y Fi(?)g Fk(\(Fin2\))75 1588 y Fs(m)o(ust)g(b)q(e)g(used)h(\(b)q (oth)f(op)q(ening)i(and)e(closing)h(a)f(\014le)h(implicitl)q(y)i(p)q (erform)d(an)g Fm(MPI)p 1521 1588 V 16 w(FILE)p 1628 1588 V 15 w(SYNC)p Fs(\).)-117 1882 y Fi(>)g Fk(\(Fin2\))-117 1941 y Fi(?)g Fk(\(Fin2\))166 1645 y Fs(Sequen)o(tial)21 b(consistency)g(is)f(guaran)o(teed)g(among)f(accesses)h(to)f(a)h (single)h(\014le)g(if)f(for)f(an)o(y)h(write)75 1701 y(sequence)d Fe(S)s(E)s(Q)368 1708 y Fd(1)402 1701 y Fs(to)e(the)g(\014le,)i(there)e(is)i(no)e(sequence)i Fe(S)s(E)s(Q)1142 1708 y Fd(2)1176 1701 y Fs(to)e(the)h(\014le)g(whic)o (h)h(is)f Fl(c)n(oncurr)n(ent)f Fs(with)-117 1999 y Fi(>)g Fk(\(Fin1\))75 1758 y Fe(S)s(E)s(Q)179 1765 y Fd(1)197 1758 y Fs(.)34 b(T)l(o)19 b(guaran)o(tee)g(sequen)o(tial)i(consistency) g(when)f(there)g(are)f(write)h(sequences,)-117 2058 y Fi(?)15 b Fk(\(Fin1\))-117 2116 y Fi(>)g Fk(\(Fin2\))75 1814 y Fm(MPI)p 160 1814 V 16 w(FILE)p 267 1814 V 15 w(SYNC)h Fs(m)o(ust)e(b)q(e)h(used)g(together)f(with)h(a)f(mec)o (hanism)h(that)f(guaran)o(tees)g(nonconcurrency)75 1871 y(of)h(the)g(sequences.)166 1927 y(See)h(the)f(examples)i(in)f(Sect.)k (10.6.10)14 b(for)g(further)i(clari\014cation)g(of)f(some)g(of)g(these) h(consistency)75 1984 y(seman)o(tics.)-117 2175 y Fi(?)f Fk(\(Fin2\))75 2134 y Fm(MPI)p 160 2134 V 16 w(FILE)p 267 2134 V 15 w(SET)p 365 2134 V 17 w(A)l(TOMICITY\(fh,)g(\015ag\))117 2212 y Fh(INOUT)62 b Fm(fh)550 b Fh(\014le)14 b(handle)g(\(handle\))117 2287 y(IN)155 b Fm(\015ag)518 b Ff(true)14 b Fh(to)f(set)h(atomic)d(mo) q(de,)g Ff(false)i Fh(to)f(set)i(non-atomic)d(mo)q(de)905 2343 y(\(b)q(o)q(olean\))75 2468 y Fg(int)23 b(MPI)p 245 2468 15 2 v 17 w(File)p 358 2468 V 17 w(set)p 447 2468 V 17 w(atomicity\(MPI)p 776 2468 V 15 w(File)g(fh,)h(int)f(flag\)) 75 2554 y(MPI)p 150 2554 V 17 w(FILE)p 263 2554 V 16 w(SET)p 351 2554 V 17 w(ATOMICITY\(FH,)f(FLAG,)h(IERROR\))170 2611 y(INTEGER)g(FH,)h(IERROR)170 2667 y(LOGICAL)f(FLAG)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 47 3 47 48 bop 75 -100 a Fj(10.6.)34 b(CONSISTENCY)16 b(AND)f(SEMANTICS)886 b Fs(47)75 49 y Fg(void)23 b(MPI::File::Set)p 533 49 15 2 v 16 w(atomicity\(bool)f(flag\))166 136 y Fs(Let)f Fe(F)6 b(H)24 b Fs(b)q(e)e(the)f(set)f(of)g(\014le)i(handles)g(created) e(b)o(y)h(one)g(collectiv)o(e)h(op)q(en.)37 b(The)21 b(consistency)75 192 y(seman)o(tics)f(for)g(data)f(access)i(op)q (erations)f(using)h Fe(F)6 b(H)24 b Fs(is)c(set)g(b)o(y)g(collectiv)o (ely)j(calling)75 249 y Fm(MPI)p 160 249 14 2 v 16 w(FILE)p 267 249 V 15 w(SET)p 365 249 V 17 w(A)l(TOMICITY)15 b Fs(on)f Fe(F)6 b(H)t Fs(.)20 b Fm(MPI)p 897 249 V 16 w(FILE)p 1004 249 V 15 w(SET)p 1102 249 V 17 w(A)l(TOMICITY)15 b Fs(is)g(collectiv)o(e;)h(all)g(pro)q(cesses)75 305 y(in)g(the)f(group)g(m)o(ust)f(pass)h(iden)o(tical)i(v)m(alues)g(for)d Fm(fh)h Fs(and)h Fm(\015ag)p Fs(.)j(If)d Fm(\015ag)f Fs(is)g Ff(true)p Fs(,)h(atomic)f(mo)q(de)g(is)h(set,)e(if)75 362 y Fm(\015ag)h Fs(is)h Ff(false)p Fs(,)e(non-atomic)i(mo)q(de)f(is)h (set.)166 418 y(Changing)k(the)g(consistency)h(seman)o(tics)g(for)e(an) h(op)q(en)h(\014le)g(only)f(a\013ects)f(new)i(data)e(accesses.)75 474 y(All)e(completed)f(data)e(accesses)i(are)f(guaran)o(teed)g(to)g (abide)h(b)o(y)f(the)g(consistency)i(seman)o(tics)e(in)h(e\013ect)75 531 y(during)i(their)f(execution.)26 b(Non)o(blo)q(c)o(king)18 b(data)e(accesses)h(that)f(ha)o(v)o(e)g(not)h(completed)h(\(e.g.,)d (via)75 587 y Fm(MPI)p 160 587 V 16 w(W)l(AIT)p Fs(\))g(are)g(only)g (guaran)o(teed)g(to)g(abide)h(b)o(y)f(non-atomic)g(mo)q(de)h (consistency)g(seman)o(tics.)189 694 y Fl(A)n(dvic)n(e)11 b(to)i(implementors.)37 b Fs(Since)12 b(the)g(seman)o(tics)f(guaran)o (teed)f(b)o(y)h(atomic)g(mo)q(de)g(are)g(stronger)189 750 y(than)17 b(those)g(guaran)o(teed)g(b)o(y)h(non-atomic)g(mo)q(de,)g (an)f(implemen)o(tation)i(is)f(free)g(to)f(adhere)h(to)189 807 y(the)f(more)f(stringen)o(t)h(atomic)g(mo)q(de)h(seman)o(tics)f (for)f(outstanding)h(requests.)26 b(\()p Fl(End)17 b(of)h(advic)n(e)189 863 y(to)e(implementors.)p Fs(\))75 1064 y Fm(MPI)p 160 1064 V 16 w(FILE)p 267 1064 V 15 w(GET)p 370 1064 V 17 w(A)l(TOMICITY\(fh,)f(\015ag\))117 1141 y Fh(IN)155 b Fm(fh)550 b Fh(\014le)14 b(handle)g(\(handle\))117 1216 y(OUT)108 b Fm(\015ag)518 b Ff(true)11 b Fh(if)d(atomic)f(mo)q(de,)i Ff(false)g Fh(if)g(non-atomic)e(mo)q(de)h(\(b)q(o)q(olean\))75 1341 y Fg(int)23 b(MPI)p 245 1341 15 2 v 17 w(File)p 358 1341 V 17 w(get)p 447 1341 V 17 w(atomicity\(MPI)p 776 1341 V 15 w(File)g(fh,)h(int)f(*flag\))75 1427 y(MPI)p 150 1427 V 17 w(FILE)p 263 1427 V 16 w(GET)p 351 1427 V 17 w(ATOMICITY\(FH,)f(FLAG,)h(IERROR\))170 1483 y(INTEGER)g(FH,)h (IERROR)170 1540 y(LOGICAL)f(FLAG)75 1626 y(bool)g(MPI::File::Get)p 533 1626 V 16 w(atomicity\(void\))e(const)166 1713 y Fm(MPI)p 251 1713 14 2 v 16 w(FILE)p 358 1713 V 15 w(GET)p 461 1713 V 17 w(A)l(TOMICITY)16 b Fs(returns)g(the)h(curren)o(t)f (consistency)h(seman)o(tics)f(for)f(data)h(access)75 1769 y(op)q(erations)i(on)f(the)g(set)h(of)f(\014le)h(handles)h (created)e(b)o(y)h(one)f(collectiv)o(e)i(op)q(en.)28 b(If)17 b Fm(\015ag)h Fs(is)g Ff(true)p Fs(,)g(atomic)75 1826 y(mo)q(de)d(is)h(enabled,)g(if)g Fm(\015ag)f Fs(is)h Ff(false)p Fs(,)f(non-atomic)g(mo)q(de)g(is)h(enabled.)75 1977 y Fm(MPI)p 160 1977 V 16 w(FILE)p 267 1977 V 15 w(SYNC\(fh\))117 2054 y Fh(INOUT)62 b Fm(fh)550 b Fh(\014le)14 b(handle)g(\(handle\))75 2178 y Fg(int)23 b(MPI)p 245 2178 15 2 v 17 w(File)p 358 2178 V 17 w(sync\(MPI)p 567 2178 V 16 w(File)g(fh\))75 2265 y(MPI)p 150 2265 V 17 w(FILE)p 263 2265 V 16 w(SYNC\(FH,)g(IERROR\))170 2321 y(INTEGER)g(FH,)h(IERROR)75 2408 y(void)f(MPI::File::Sync\(void\))166 2494 y Fs(Calling)17 b Fm(MPI)p 408 2494 14 2 v 16 w(FILE)p 515 2494 V 15 w(SYNC)g Fs(with)f Fm(fh)g Fs(causes)f(all)i(previous)f (writes)g(to)f Fm(fh)h Fs(b)o(y)g(the)f(calling)j(pro)q(cess)75 2551 y(to)12 b(b)q(e)i(written)f(to)f(p)q(ermanen)o(t)h(storage.)18 b Fm(MPI)p 882 2551 V 15 w(FILE)p 988 2551 V 16 w(SYNC)c Fs(ma)o(y)e(b)q(e)h(necessary)g(to)f(ensure)i(sequen)o(tial)q Fi(>)i Fk(\(Fin1\))1875 2609 y Fi(>)g Fk(\(Fin2\))75 2607 y Fs(consistency)g(in)g(certain)g(cases)f(\(see)g(ab)q(o)o(v)o (e\).)1875 2668 y Fi(?)h Fk(\(Fin2\))1875 2726 y Fi(?)g Fk(\(Fin1\))166 2664 y Fm(MPI)p 251 2664 V 16 w(FILE)p 358 2664 V 15 w(SYNC)g Fs(is)g(a)f(collectiv)o(e)i(op)q(eration.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 48 4 48 49 bop 75 -100 a Fs(48)1342 b Fj(CHAPTER)15 b(10.)35 b(I/O)166 49 y Fs(The)18 b(user)f(is)h(resp)q(onsible)h(for)e(ensuring) h(that)f(all)h(non)o(blo)q(c)o(king)h(requests)e(and)h(split)g (collectiv)o(e)75 106 y(op)q(erations)g(on)f Fm(fh)h Fs(ha)o(v)o(e)g(b)q(een)g(completed)h(b)q(efore)f(calling)h Fm(MPI)p 1226 106 14 2 v 16 w(FILE)p 1333 106 V 16 w(SYNC)p Fs(|otherwise,)g(the)f(call)75 162 y(to)d Fm(MPI)p 216 162 V 15 w(FILE)p 322 162 V 16 w(SYNC)h Fs(is)g(erroneous.)75 281 y Fn(10.6.2)49 b(Random)17 b(Access)e(vs.)22 b(Sequential)16 b(Files)75 367 y Fm(MPI)h Fs(distinguishes)i(ordinary)f(random)e (access)i(\014les)g(from)f(sequen)o(tial)h(stream)f(\014les,)h(suc)o(h) g(as)e(pip)q(es)75 423 y(and)j(tap)q(e)h(\014les.)32 b(Sequen)o(tial)21 b(stream)d(\014les)j(m)o(ust)d(b)q(e)i(op)q(ened)g (with)g(the)f Ff(MPI)p 1475 423 13 2 v 14 w(MODE)p 1611 423 V 15 w(SEQUENTIAL)75 480 y Fs(\015ag)14 b(set)h(in)h(the)e(amo)q (de.)20 b(F)l(or)14 b(these)h(\014les,)h(the)f(only)g(p)q(ermitted)g (data)g(access)f(op)q(erations)h(are)g(shared)75 536 y(\014le)j(p)q(oin)o(ter)g(reads)g(and)f(writes.)27 b(Filet)o(yp)q(es) 18 b(and)g(et)o(yp)q(es)f(with)h(holes)g(are)f(erroneous.)26 b(In)18 b(addition,)75 593 y(the)13 b(notion)f(of)g(\014le)i(p)q(oin)o (ter)e(is)h(not)f(meaningful;)j(therefore,)d(calls)i(to)d Fm(MPI)p 1365 593 14 2 v 16 w(FILE)p 1472 593 V 16 w(SEEK)p 1599 593 V 16 w(SHARED)j Fs(and)75 649 y Fm(MPI)p 160 649 V 16 w(FILE)p 267 649 V 15 w(GET)p 370 649 V 17 w(POSITION)p 596 649 V 17 w(SHARED)20 b Fs(are)e(erroneous,)g(and)h(the)f(p)q(oin)o (ter)h(up)q(date)g(rules)g(sp)q(eci\014ed)75 706 y(for)e(the)g(data)g (access)g(routines)h(do)g(not)f(apply)l(.)27 b(The)18 b(amoun)o(t)e(of)h(data)g(accessed)h(b)o(y)f(a)g(data)g(access)75 762 y(op)q(eration)c(will)h(b)q(e)f(the)g(amoun)o(t)f(requested)h (unless)h(the)f(end-of-\014le)h(is)f(reac)o(hed)g(or)f(an)h(error)f(is) h(raised.)166 819 y(Finally)l(,)23 b(for)d(some)g(sequen)o(tial)i (\014les,)g(suc)o(h)f(as)f(those)g(corresp)q(onding)h(to)f(magnetic)h (tap)q(es)f(or)75 875 y(streaming)f(net)o(w)o(ork)g(connections,)i (writes)e(to)g(the)g(\014le)i(ma)o(y)e(b)q(e)h(destructiv)o(e.)33 b(In)20 b(other)f(w)o(ords,)g(a)75 931 y(write)f(ma)o(y)g(act)f(as)h(a) g(truncate)g(\(a)f Fm(MPI)p 802 931 V 16 w(FILE)p 909 931 V 15 w(SET)p 1007 931 V 17 w(SIZE)h Fs(with)g Fm(size)h Fs(set)f(to)f(the)h(curren)o(t)g(p)q(osition\))75 988 y(follo)o(w)o(ed)e(b)o(y)f(the)g(write.)75 1107 y Fn(10.6.3)49 b(Progress)75 1193 y Fs(The)17 b(progress)g(rules)g(of)g Fm(MPI)f Fs(are)h(b)q(oth)g(a)g(promise)g(to)g(users,)g(and)g(a)g(set)f (of)h(constrain)o(ts)f(on)h(imple-)75 1249 y(men)o(tors.)28 b(In)18 b(cases)g(where)h(the)f(progress)f(rules)i(restrict)f(p)q (ossible)i(implemen)o(tation)f(c)o(hoices)g(more)75 1306 y(than)c(the)g(in)o(terface)h(sp)q(eci\014cation)h(alone,)e(the)h (progress)e(rules)i(tak)o(e)e(precedence.)166 1362 y(All)f(blo)q(c)o (king)h(routines)e(m)o(ust)f(complete)i(in)g(\014nite)g(time)f(unless)h (an)f(exceptional)i(condition)f(\(suc)o(h)75 1418 y(as)i(resource)g (exhaustion\))h(causes)f(an)g(error.)166 1475 y(Non)o(blo)q(c)o(king)i (data)e(access)h(routines)g(inherit)h(the)f(follo)o(wing)g(progress)f (rule)i(from)e(non)o(blo)q(c)o(king)75 1531 y(p)q(oin)o(t)h(to)e(p)q (oin)o(t)i(comm)o(unication:)21 b(a)14 b(non)o(blo)q(c)o(king)j(write)e (is)h(equiv)m(alen)o(t)h(to)d(a)h(non)o(blo)q(c)o(king)i(send)f(for)75 1588 y(whic)o(h)f(a)f(receiv)o(e)i(is)f(ev)o(en)o(tually)g(p)q(osted,)g (and)f(a)g(non)o(blo)q(c)o(king)i(read)f(is)g(equiv)m(alen)o(t)h(to)d (a)i(non)o(blo)q(c)o(king)75 1644 y(receiv)o(e)h(for)f(whic)o(h)h(a)f (send)g(is)h(ev)o(en)o(tually)g(p)q(osted.)166 1701 y(Finally)l(,)g(an) f(implemen)o(tation)i(is)e(free)g(to)g(dela)o(y)g(progress)g(of)f (collectiv)o(e)j(routines)f(un)o(til)g(all)g(pro-)75 1757 y(cesses)i(in)h(the)f(group)g(asso)q(ciated)g(with)g(the)h (collectiv)o(e)g(call)g(ha)o(v)o(e)f(in)o(v)o(ok)o(ed)g(the)g(routine.) 29 b(Once)19 b(all)75 1814 y(pro)q(cesses)g(in)g(the)f(group)g(ha)o(v)o (e)g(in)o(v)o(ok)o(ed)h(the)f(routine,)h(the)g(progress)e(rule)i(of)f (the)h(equiv)m(alen)o(t)h(non-)75 1870 y(collectiv)o(e)d(routine)f(m)o (ust)e(b)q(e)i(follo)o(w)o(ed.)75 1989 y Fn(10.6.4)49 b(Collective)18 b(File)e(Op)q(erations)75 2075 y Fs(Collectiv)o(e)j (\014le)f(op)q(erations)g(are)f(sub)s(ject)g(to)g(the)g(same)g (restrictions)h(as)f(collectiv)o(e)i(comm)o(unication)75 2131 y(op)q(erations.)30 b(F)l(or)18 b(a)g(complete)i(discussion)g(of)e (in)o(terlea)o(v)o(ed)h(collectiv)o(e)i(op)q(erations,)e(please)g (refer)g(to)75 2188 y(the)c(seman)o(tics)h(set)f(forth)f(in)i([)p Fo(?)o Fs(],)f(section)h(4.12.)166 2244 y(Note)f(that)f(non-collectiv)o (e)k(op)q(erations)d(ma)o(y)f(b)q(e)i(in)o(terlea)o(v)o(ed)g(with)g (collectiv)o(e)h(op)q(erations.)189 2333 y Fl(A)n(dvic)n(e)f(to)h (users.)46 b Fs(Consider)17 b(op)q(ening)g(the)f(same)g(\014le)h(m)o (ultiple)h(times)f(if)f(y)o(ou)g(\014nd)h(that)e(y)o(ou)189 2389 y(ha)o(v)o(e)c(need)h(for)f(m)o(ultiple)i(in)o(terlea)o(v)o(ed)f (collectiv)o(e)h(op)q(erations)e(to)g(the)h(same)f(\014le.)19 b(\()p Fl(End)12 b(of)h(advic)n(e)189 2446 y(to)j(users.)p Fs(\))166 2534 y(Collectiv)o(e)i(\014le)h(op)q(erations)e(are)g (collectiv)o(e)i(o)o(v)o(er)d(a)h(dup)g(of)g(the)g(comm)o(unicator)g (used)h(to)e(op)q(en)75 2591 y(the)g(\014le|this)j(duplicate)f(comm)o (unicator)e(is)g(implicitl)q(y)j(sp)q(eci\014ed)f(via)f(the)f(\014le)i (handle)f(argumen)o(ts.)75 2647 y(Di\013eren)o(t)11 b(pro)q(cesses)g (can)g(pass)g(di\013eren)o(t)g(v)m(alues)h(for)f(other)f(argumen)o(ts)g (of)h(a)g(collectiv)o(e)h(routine)g(unless)75 2704 y(sp)q(eci\014ed)17 b(otherwise.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 49 5 49 50 bop 75 -100 a Fj(10.6.)34 b(CONSISTENCY)16 b(AND)f(SEMANTICS)886 b Fs(49)75 49 y Fn(10.6.5)49 b(T)l(yp)q(e)17 b(Matching)75 135 y Fs(The)h(t)o(yp)q(e)h(matc)o(hing)f(rules)h(for)e(I/O)i(mimic)g (the)g(t)o(yp)q(e)f(matc)o(hing)g(rules)h(for)f(comm)o(unication)h (with)75 192 y(one)14 b(exception:)20 b(the)14 b(t)o(yp)q(e)g Ff(MPI)p 632 192 13 2 v 14 w(BYTE)f Fs(matc)o(hes)g(an)o(y)h(t)o(yp)q (e)g(in)g(a)g(data)f(access.)19 b(In)c(general,)f(the)g(et)o(yp)q(e)75 248 y(of)e(data)g(items)h(written)g(m)o(ust)f(matc)o(h)g(the)h(et)o(yp) q(e)g(used)g(to)f(read)g(the)h(items,)g(and)g(for)f(eac)o(h)h(data)f (access)75 304 y(op)q(eration,)h(the)h(curren)o(t)f(et)o(yp)q(e)g(m)o (ust)g(also)g(matc)o(h)g(the)g(t)o(yp)q(e)g(declaration)h(of)f(the)g (data)g(access)g(bu\013er.)75 361 y(An)i(et)o(yp)q(e)g(or)e(data)h (access)h(bu\013er)f(t)o(yp)q(e)h(constructed)f(en)o(tirely)i(from)e Ff(MPI)p 1385 361 V 14 w(BYTE)g Fs(matc)o(hes)g(all)h(t)o(yp)q(es.)189 464 y Fl(A)n(dvic)n(e)20 b(to)i(users.)74 b Fs(In)22 b(most)e(cases,)i(use)f(of)g Ff(MPI)p 1122 464 V 14 w(BYTE)f Fs(as)h(a)f(wild)j(card)d(will)j(defeat)e(the)189 521 y(\014le)15 b(in)o(terop)q(erabilit)o(y)i(features)d(of)g Fm(MPI)p Fs(.)g(File)i(in)o(terop)q(erabilit)o(y)g(can)f(only)g(p)q (erform)g(automatic)189 577 y(con)o(v)o(ersion)f(b)q(et)o(w)o(een)h (heterogeneous)g(data)f(represen)o(tations)g(when)h(the)g(exact)f (datat)o(yp)q(es)g(ac-)189 633 y(cessed)i(are)e(explicitly)k(sp)q (eci\014ed.)k(\()p Fl(End)16 b(of)g(advic)n(e)g(to)h(users.)p Fs(\))75 755 y Fn(10.6.6)49 b(Miscellaneous)18 b(Cla)o(ri\014cations)75 840 y Fs(Once)g(an)f(I/O)g(routine)g(completes,)h(it)f(is)g(safe)g(to)f (free)h(an)o(y)f(opaque)h(ob)s(jects)f(passed)h(as)g(argumen)o(ts)75 897 y(to)e(that)g(routine.)21 b(F)l(or)15 b(example,)i(the)e Fm(comm)f Fs(and)i Fm(info)f Fs(used)i(in)f(an)f Fm(MPI)p 1369 897 14 2 v 16 w(FILE)p 1476 897 V 16 w(OPEN)p Fs(,)h(or)f(the)h Fm(et)o(yp)q(e)75 953 y Fs(and)i Fm(\014let)o(yp)q(e)h Fs(used)f(in)h(an)e Fm(MPI)p 632 953 V 16 w(FILE)p 739 953 V 16 w(SET)p 838 953 V 16 w(VIEW)p Fs(,)h(can)g(b)q(e)g(freed)g (without)f(a\013ecting)h(access)g(to)f(the)75 1010 y(\014le.)k(Note)13 b(that)h(for)f(non-blo)q(c)o(king)j(routines)f(and)f(split)h(collectiv) o(e)h(op)q(erations,)e(the)g(op)q(eration)h(m)o(ust)75 1066 y(b)q(e)h(\\completed")f(b)q(efore)h(it)f(is)h(safe)f(to)g(reuse)g (data)g(bu\013ers)g(passed)g(as)g(parameters.)166 1123 y(As)k(in)h(comm)o(unication,)g(datat)o(yp)q(es)e(m)o(ust)h(b)q(e)g (committed)g(b)q(efore)h(they)f(can)g(b)q(e)h(used)f(in)h(\014le)75 1179 y(manipulation)k(or)e(data)g(access)g(op)q(erations.)42 b(F)l(or)22 b(example,)i(the)f Fm(et)o(yp)q(e)g Fs(and)75 1236 y Fm(\014let)o(yp)q(e)18 b Fs(m)o(ust)e(b)q(e)h(committed)g(b)q (efore)g(calling)h Fm(MPI)p 1002 1236 V 16 w(FILE)p 1109 1236 V 16 w(SET)p 1208 1236 V 16 w(VIEW)p Fs(,)f(and)f(the)h Fm(datat)o(yp)q(e)h Fs(m)o(ust)e(b)q(e)75 1292 y(committed)f(b)q(efore) h(calling)h Fm(MPI)p 668 1292 V 16 w(FILE)p 775 1292 V 15 w(READ)f Fs(or)f Fm(MPI)p 1063 1292 V 15 w(FILE)p 1169 1292 V 16 w(WRITE)p Fs(.)75 1413 y Fn(10.6.7)49 b(MPI)p 337 1413 15 2 v 19 w(O\013set)15 b(T)l(yp)q(e)75 1499 y Fm(MPI)p 160 1499 14 2 v 16 w(O\013set)h Fs(is)f(an)g(in)o (teger)h(t)o(yp)q(e)f(of)f(size)i(su\016cien)o(t)g(to)e(represen)o(t)h (the)h(size)g(\(in)f(b)o(ytes\))g(of)f(the)h(largest)75 1556 y(\014le)h(supp)q(orted)g(b)o(y)f Fm(MPI)p Fs(.)f(Displacemen)o (ts)i(and)g(o\013sets)e(are)h(alw)o(a)o(ys)f(sp)q(eci\014ed)k(b)o(y)d (an)g Fm(MPI)p 1676 1556 V 16 w(O\013set)p Fs(.)166 1612 y(In)i(F)l(ortran)e(90)h(\(as)g(part)g(of)g(the)h(\\extended)g(F)l (ortran)f(supp)q(ort"\),)g(the)g(corresp)q(onding)i(in)o(teger)75 1669 y(is)i(an)f(in)o(teger)h(of)f(kind)h Ff(MPI)p 586 1669 13 2 v 15 w(OFFSET)p 756 1669 V 14 w(KIND)p Fs(,)e(de\014ned)j(in) f(mpif.h)g(and)g(the)f(mpi)h(mo)q(dule.)34 b(F)l(ortran)75 1725 y(bindings)17 b(in)f(this)g(c)o(hapter)f(use)g(this)h(F)l(ortran)e (90)h(feature.)166 1781 y(In)22 b(F)l(ortran)e(77)h(en)o(vironmen)o(ts) g(that)g(do)g(not)g(supp)q(ort)g(KIND)h(parameters,)75 1838 y Fm(MPI)p 160 1838 14 2 v 16 w(O\013set)16 b Fs(argumen)o(ts)e (should)j(b)q(e)f(declared)g(as)f(INTEGER*8)g(if)h(supp)q(orted)g(b)o (y)f(the)g(compiler,)h(or)75 1894 y(INTEGER)f(otherwise.)75 2016 y Fn(10.6.8)49 b(Logical)19 b(vs.)i(Physical)c(File)g(La)o(y)o (out)75 2101 y Fs(This)c(standard)f(sp)q(eci\014es)i(ho)o(w)e(the)h (data)f(should)h(b)q(e)g(laid)h(out)e(in)h(a)f(virtual)h(\014le)h (structure)e(\(the)g(view\),)75 2158 y(not)g(ho)o(w)g(that)g(\014le)h (structure)g(is)g(to)e(b)q(e)j(stored)e(on)g(one)h(or)f(more)g(disks.) 19 b(Sp)q(eci\014cation)c(of)d(the)h(ph)o(ysical)75 2214 y(\014le)18 b(structure)e(w)o(as)g(a)o(v)o(oided)h(b)q(ecause)g(it)g (is)g(exp)q(ected)h(that)e(the)h(mapping)g(of)f(\014les)i(to)e(disks)h (will)h(b)q(e)75 2271 y(system)g(sp)q(eci\014c,)i(and)f(an)o(y)f(sp)q (eci\014c)i(con)o(trol)e(o)o(v)o(er)f(\014le)j(la)o(y)o(out)d(w)o(ould) i(therefore)f(restrict)g(program)75 2327 y(p)q(ortabilit)o(y)l(.)i(Ho)o (w)o(ev)o(er,)12 b(there)h(are)g(still)h(cases)f(where)g(some)f (information)h(will)i(b)q(e)e(necessary)h(in)f(order)75 2384 y(to)h(optimize)i(disk)g(la)o(y)o(out.)j(This)d(information)f(is)g (pro)o(vided)h(as)e Fl(hints)g Fs(sp)q(eci\014ed)j(via)f Fl(info)e Fs(when)h(a)g(\014le)75 2440 y(is)h(created)f(\(see)g (section)h(10.2.8\).)1203 b Fi(>)16 b Fk(\(Fin1\))75 2561 y Fn(10.6.9)49 b(File)17 b(Size)75 2647 y Fs(The)f(size)i(of)e(a)f (\014le)j(ma)o(y)d(b)q(e)i(increased)h(b)o(y)e(writing)h(to)e(the)i (\014le)g(after)e(the)i(curren)o(t)f(end)h(of)f(\014le.)24 b(The)75 2704 y(size)17 b(ma)o(y)e(also)g(b)q(e)i(c)o(hanged)f(b)o(y)g (calling)h Fm(MPI)e Fl(size)h(changing)f Fs(routines,)h(suc)o(h)g(as)f (MPI)p 1614 2704 V 17 w(File)p 1705 2704 V 17 w(set)p 1778 2704 V 16 w(size.)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 50 6 50 51 bop 75 -100 a Fs(50)1347 b Fj(CHAPTER)15 b(10.)30 b(I/O)75 49 y Fs(A)19 b(call)i(to)d(a)h(size)h(c)o(hanging)g(routine)f (do)q(es)h(not)f(necessarily)h(c)o(hange)g(the)f(\014le)h(size.)33 b(F)l(or)18 b(example,)75 106 y(calling)e(MPI)p 310 106 14 2 v 17 w(File)p 401 106 V 17 w(preallo)q(cate)g(with)f(a)f(size)i (less)f(than)g(the)g(curren)o(t)f(size)i(do)q(es)f(not)f(c)o(hange)h (the)f(size.)166 162 y(Consider)i(a)e(set)h(of)g(b)o(ytes)g(whic)o(h)g (has)g(b)q(een)i(written)e(to)f(a)h(\014le)h(since)g(the)f(most)g (recen)o(t)g(call)h(to)e(a)75 219 y(size)j(c)o(hanging)g(routine,)g(or) e(since)j(MPI)p 786 219 V 16 w(Op)q(en)g(if)f(no)f(suc)o(h)h(routine)f (has)h(b)q(een)g(called.)25 b(Let)16 b(the)h Fl(high)75 275 y(byte)e Fs(b)q(e)h(the)f(b)o(yte)g(in)h(that)f(set)g(with)g(the)h (largest)e(displacemen)o(t.)22 b(The)15 b(\014le)i(size)f(is)f(the)h (larger)f(of)143 369 y Fi(\017)23 b Fs(One)16 b(plus)g(the)f (displacemen)o(t)i(of)e(the)g(high)h(b)o(yte.)143 463 y Fi(\017)23 b Fs(The)15 b(size)h(immediately)h(after)d(the)i(size)g(c) o(hanging)g(routine,)f(or)g(MPI)p 1407 463 V 16 w(Op)q(en,)h(returned.) 166 557 y(In)g(applying)h(consistency)g(seman)o(tics)f(to)f(determine)i (the)f(\014le)h(size,)f(calls)h(to)e(MPI)p 1627 557 V 16 w(File)p 1717 557 V 18 w(set)p 1791 557 V 16 w(size)75 613 y(and)g(MPI)p 255 613 V 17 w(File)p 346 613 V 17 w(preallo)q(cate)h(are)f(considered)i(writes)e(to)g(the)g(\014le.)189 719 y Fl(A)n(dvic)n(e)22 b(to)i(users.)87 b Fs(An)o(y)23 b(sequence)h(of)f(op)q(erations)g(con)o(taining)h(the)f(collectiv)o(e)i (routines)189 776 y(MPI)p 281 776 V 16 w(File)p 371 776 V 18 w(set)p 445 776 V 16 w(size)18 b(and)g(MPI)p 729 776 V 17 w(File)p 820 776 V 17 w(preallo)q(cate)h(is)f(a)g(write)g (sequence.)29 b(As)17 b(suc)o(h,)i(sequen)o(tial)189 832 y(consistency)d(in)g(non-atomic)f(mo)q(de)g(is)h(not)f(guaran)o (teed)f(unless)j(the)e(conditions)h(in)g(10.6.1)d(are)189 889 y(satis\014ed.)20 b(\()p Fl(End)c(of)g(advic)n(e)g(to)h(users.)p Fs(\))166 995 y(File)g(p)q(oin)o(ter)g(up)q(date)f(seman)o(tics)g(\()p Fl(i.e.)22 b Fs(\014le)17 b(p)q(oin)o(ters)g(are)f(up)q(dated)g(b)o(y)g (the)g(amoun)o(t)g(accessed\))75 1051 y(are)f(only)h(guaran)o(teed)e (if)i(\014le)g(size)g(c)o(hanges)g(are)e(sequen)o(tially)j(consisten)o (t.)189 1158 y Fl(A)n(dvic)n(e)h(to)i(users.)63 b Fs(Consider)19 b(the)g(follo)o(wing)h(example.)32 b(Giv)o(en)19 b(t)o(w)o(o)f(op)q (erations)h(made)g(b)o(y)189 1214 y(separate)d(pro)q(cesses)h(to)g(a)f (\014le)i(con)o(taining)g(100)e(b)o(ytes:)23 b(an)17 b(MPI)p 1345 1214 V 16 w(File)p 1435 1214 V 18 w(iread)g(of)g(10)f(b)o (ytes)h(and)189 1271 y(an)12 b(MPI)p 341 1271 V 16 w(File)p 431 1271 V 18 w(set)p 505 1271 V 16 w(size)h(to)f(0)g(b)o(ytes.)19 b(If)13 b(the)f(user)h(do)q(es)f(not)h(enforce)f(sequen)o(tial)i (consistency)f(b)q(e-)189 1327 y(t)o(w)o(een)g(these)h(t)o(w)o(o)e(op)q (erations,)i(the)f(\014le)i(p)q(oin)o(ter)f(ma)o(y)f(b)q(e)h(up)q (dated)g(b)o(y)g(the)g(amoun)o(t)e(requested)189 1383 y(\(10)f(b)o(ytes\))g(rather)h(than)g(the)g(amoun)o(t)f(accessed)i (\(whic)o(h)g(ma)o(y)e(b)q(e)i(zero)f(b)o(ytes\).)18 b(\()p Fl(End)13 b(of)g(advic)n(e)189 1440 y(to)j(users.)p Fs(\))-117 1501 y Fi(?)f Fk(\(Fin1\))-117 1560 y Fi(>)g Fk(\(Fin2\))75 1611 y Fn(10.6.10)50 b(Examples)75 1697 y Fs(The)13 b(examples)g(in)h(this)f(section)g(illustrate)h(the)e (application)j(of)d(the)g Fm(MPI)g Fs(consistency)i(and)f(seman)o(tics) 75 1754 y(guaran)o(tees.)19 b(These)d(address)143 1847 y Fi(\017)23 b Fs(o)o(v)o(erlapping)15 b(accesses)h(on)f(\014le)h (handles)h(obtained)f(from)e(a)h(single)h(collectiv)o(e)h(op)q(en,)f (and)143 1941 y Fi(\017)23 b Fs(all)16 b(accesses)f(on)g(\014le)i (handles)f(obtained)g(from)e(t)o(w)o(o)g(separate)h(collectiv)o(e)i(op) q(ens.)166 2035 y(The)c(simplest)g(w)o(a)o(y)f(to)g(ac)o(hiev)o(e)h (consistency)g(for)f(o)o(v)o(erlapping)h(accesses)g(is)g(to)f(obtain)h (sequen)o(tial)75 2092 y(consistency)k(b)o(y)e(setting)h(atomic)f(mo)q (de.)22 b(F)l(or)15 b(the)g(co)q(de)i(b)q(elo)o(w,)f(pro)q(cess)g(1)f (will)i(either)f(read)g Ff(0)f Fs(or)g Ff(10)75 2148 y Fs(b)o(ytes.)23 b(If)17 b(the)g(latter,)f(ev)o(ery)g(elemen)o(t)h(of) g Fg(b)f Fs(will)i(b)q(e)f Ff(5)p Fs(.)24 b(If)16 b(non-atomic)h(mo)q (de)g(is)g(set,)f(the)h(results)f(of)75 2204 y(the)f(read)g(are)g (unde\014ned.)75 2298 y Fg(/*)24 b(Process)e(0)i(*/)75 2355 y(int)47 b(i,)24 b(a[10])f(;)75 2411 y(for)g(\()h(i=0;i<10;i++\)) 147 2468 y(a[i])f(=)h(5)f(;)75 2581 y(MPI_File_open\()f (MPI_COMM_WORLD,)g('workfile',)433 2637 y(MPI_MODE_RDWR)g(|)i (MPI_MODE_CREATE,)d(MPI_INFO_NULL,)h(&fh0)i(\))f(;)75 2693 y(MPI_File_set_view\()e(fh0,)j(0,)f(MPI_INT,)g(MPI_INT,)g (``native'',)f(MPI_INFO_NULL)g(\))i(;)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 51 7 51 52 bop 75 -100 a Fj(10.6.)34 b(CONSISTENCY)16 b(AND)f(SEMANTICS)886 b Fs(51)75 49 y Fg(MPI_File_set_atomicity\()21 b(fh0,)i(TRUE)g(\))h(;) 75 106 y(MPI_File_write_explicit\(fh)o(0,)d(0,)j(a,)f(10,)h(MPI_INT,)e (&status\))h(;)75 162 y(/*)h(MPI_Barrier\()e(MPI_COMM_WORLD)g(\))h(;)h (*/)75 275 y(/*)g(Process)e(1)i(*/)75 332 y(int)47 b(b[10])23 b(;)75 388 y(MPI_File_open\()f(MPI_COMM_WORLD,)g('workfile',)433 444 y(MPI_MODE_RDWR)g(|)i(MPI_MODE_CREATE,)d(MPI_INFO_NULL,)h(&fh1)i (\))f(;)75 501 y(MPI_File_set_view\()e(fh1,)j(0,)f(MPI_INT,)g(MPI_INT,) g(``native'',)f(MPI_INFO_NULL)g(\))i(;)75 557 y (MPI_File_set_atomicity\()d(fh1,)i(TRUE)g(\))h(;)75 614 y(/*)g(MPI_Barrier\()e(MPI_COMM_WORLD)g(\))h(;)h(*/)75 670 y(MPI_File_read_explicit\(fh1)o(,)d(0,)j(b,)f(10,)h(MPI_INT,)f (&status\))f(;)75 755 y Fs(A)13 b(user)f(ma)o(y)g(guaran)o(tee)g(that)g (the)g(write)h(on)f(pro)q(cess)h Ff(0)f Fs(preceed)i(the)e(read)h(on)f (pro)q(cess)h Ff(1)f Fs(b)o(y)h(imp)q(osing)75 812 y(temp)q(oral)i (order)g(with,)g(for)g(example,)h(calls)g(to)e Fm(MPI)p 1005 812 14 2 v 16 w(BARRIER)p Fs(.)189 907 y Fl(A)n(dvic)n(e)e(to)i (users.)37 b Fs(Routines)14 b(other)d(than)h Fm(MPI)p 1027 907 V 16 w(BARRIER)h Fs(ma)o(y)e(b)q(e)i(used)g(to)e(imp)q(ose)i (temp)q(oral)189 964 y(order.)18 b(In)13 b(the)g(example)g(ab)q(o)o(v)o (e,)f(pro)q(cess)h(0)f(could)h(use)g Fm(MPI)p 1238 964 V 16 w(SEND)g Fs(to)e(send)i(a)f(0)h(b)o(yte)f(message,)189 1020 y(receiv)o(ed)k(b)o(y)f(pro)q(cess)h(1)e(using)i Fm(MPI)p 826 1020 V 16 w(RECV)p Fs(.)g(\()p Fl(End)f(of)i(advic)n(e)f (to)h(users.)p Fs(\))166 1116 y(Alternately)l(,)f(a)f(user)g(can)g(imp) q(ose)h(consistency)g(with)g(non-atomic)f(mo)q(de)h(set:)75 1201 y Fg(/*)24 b(Process)e(0)i(*/)75 1258 y(int)47 b(i,)24 b(a[10])f(;)75 1314 y(for)g(\()h(i=0;i<10;i++\))147 1371 y(a[i])f(=)h(5)f(;)75 1483 y(MPI_File_open\()f(MPI_COMM_WORLD,)g ('workfile',)433 1540 y(MPI_MODE_RDWR)g(|)i(MPI_MODE_CREATE,)d (MPI_INFO_NULL,)h(&fh0)i(\))f(;)75 1596 y(MPI_File_set_view\()e(fh0,)j (0,)f(MPI_INT,)g(MPI_INT,)g(``native'',)f(MPI_INFO_NULL)g(\))i(;)75 1653 y(MPI_File_write_explicit\(fh)o(0,)d(0,)j(a,)f(10,)h(MPI_INT,)e (&status)h(\))h(;)75 1709 y(MPI_File_sync\()e(fh0)h(\))h(;)75 1766 y(MPI_Barrier\()e(MPI_COMM_WORLD)g(\))i(;)75 1822 y(MPI_File_sync\()e(fh0)h(\))h(;)75 1935 y(/*)g(Process)e(1)i(*/)75 1992 y(int)47 b(b[10])23 b(;)75 2048 y(MPI_File_open\()f (MPI_COMM_WORLD,)g('workfile',)433 2104 y(MPI_MODE_RDWR)g(|)i (MPI_MODE_CREATE,)d(MPI_INFO_NULL,)h(&fh1)i(\))f(;)75 2161 y(MPI_File_set_view\()e(fh1,)j(0,)f(MPI_INT,)g(MPI_INT,)g (``native'',)f(MPI_INFO_NULL)g(\))i(;)75 2217 y(MPI_File_sync\()e(fh1)h (\))h(;)75 2274 y(MPI_Barrier\()e(MPI_COMM_WORLD)g(\))i(;)75 2330 y(MPI_File_sync\()e(fh1)h(\))h(;)75 2387 y (MPI_File_read_explicit\(fh1)o(,)d(0,)j(b,)f(10,)h(MPI_INT,)f(&status)f (\))i(;)75 2472 y Fs(The)15 b(\\sync-barrier-sync")h(construct)f(is)h (required)g(b)q(ecause:)143 2557 y Fi(\017)23 b Fs(The)15 b(barrier)g(ensures)h(that)f(the)g(write)g(on)g(pro)q(cess)h(0)f(o)q (ccurs)g(b)q(efore)h(the)f(read)g(on)g(pro)q(cess)h(1.)143 2647 y Fi(\017)23 b Fs(The)f(\014rst)g(sync)h(guaran)o(tees)f(that)g (the)g(data)g(written)g(b)o(y)h(all)g(pro)q(cesses)g(is)g(committed)g (to)189 2704 y(p)q(ermanen)o(t)15 b(storage.)k(\(This)c(do)q(es)h(not)e (a\013ect)h(pro)q(cess)g(1)g(in)h(this)g(example.\))-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 52 8 52 53 bop 75 -100 a Fs(52)1347 b Fj(CHAPTER)15 b(10.)30 b(I/O)143 49 y Fi(\017)23 b Fs(The)18 b(second)h(sync)g(guaran)o(tees)f (that)f(all)j(data)d(whic)o(h)j(has)e(b)q(een)h(committed)g(to)f(p)q (ermanen)o(t)189 106 y(storage)c(is)h(visible)j(to)c(all)i(pro)q (cesses.)21 b(\(This)15 b(do)q(es)h(not)e(a\013ect)h(pro)q(cess)g(0)g (in)h(this)g(example.\))166 198 y(The)d(follo)o(wing)g(program)e (represen)o(ts)i(an)f(erroneous)h(attempt)e(to)h(ac)o(hiev)o(e)i (consistency)f(b)o(y)g(elim-)75 254 y(inating)j(the)f(apparen)o(tly)h (sup)q(er\015uous)g(second)g(\\sync")f(call)h(for)f(eac)o(h)g(pro)q (cess.)75 346 y Fg(/*)24 b(----------------)45 b(THIS)23 b(EXAMPLE)g(IS)h(ERRONEOUS)f(---------------)e(*/)75 403 y(/*)j(Process)e(0)i(*/)75 459 y(int)47 b(i,)24 b(a[10])f(;)75 516 y(for)g(\()h(i=0;i<10;i++\))147 572 y(a[i])f(=)h(5)f(;)75 685 y(MPI_File_open\()f(MPI_COMM_WORLD,)g('workfile',)433 742 y(MPI_MODE_RDWR)g(|)i(MPI_MODE_CREATE,)d(MPI_INFO_NULL,)h(&fh0)i (\))f(;)75 798 y(MPI_File_set_view\()e(fh0,)j(0,)f(MPI_INT,)g(MPI_INT,) g(``native'',)f(MPI_INFO_NULL)g(\))i(;)75 855 y (MPI_File_write_explicit\(fh)o(0,)d(0,)j(a,)f(10,)h(MPI_INT,)e(&status) h(\))h(;)75 911 y(MPI_File_sync\()e(fh0)h(\))h(;)75 967 y(MPI_Barrier\()e(MPI_COMM_WORLD)g(\))i(;)75 1080 y(/*)g(Process)e(1)i (*/)75 1137 y(int)47 b(b[10])23 b(;)75 1193 y(MPI_File_open\()f (MPI_COMM_WORLD,)g('workfile',)433 1250 y(MPI_MODE_RDWR)g(|)i (MPI_MODE_CREATE,)d(MPI_INFO_NULL,)h(&fh1)i(\))f(;)75 1306 y(MPI_File_set_view\()e(fh1,)j(0,)f(MPI_INT,)g(MPI_INT,)g (``native'',)f(MPI_INFO_NULL)g(\))i(;)75 1363 y(MPI_Barrier\()e (MPI_COMM_WORLD)g(\))i(;)75 1419 y(MPI_File_sync\()e(fh1)h(\))h(;)75 1476 y(MPI_File_read_explicit\(fh1)o(,)d(0,)j(b,)f(10,)h(MPI_INT,)f (&status)f(\))i(;)75 1588 y(/*)g(----------------)45 b(THIS)23 b(EXAMPLE)g(IS)h(ERRONEOUS)f(---------------)e(*/)75 1681 y Fs(This)g(program)e(violates)h(the)g Fm(MPI)g Fs(rulest)h(for)e(not)h(in)o(terlea)o(ving)h(collectiv)o(e)h(op)q (erations,)f(and)f(will)75 1737 y(deadlo)q(c)o(k)c(for)f(implemen)o (tations)h(in)g(whic)o(h)g Fm(MPI)p 943 1737 14 2 v 16 w(FILE)p 1050 1737 V 16 w(SYNC)g Fs(blo)q(c)o(ks.)189 1841 y Fl(A)n(dvic)n(e)c(to)h(users.)38 b Fs(Some)12 b(implemen)o(tations)h(ma)o(y)e(c)o(ho)q(ose)h(to)f(implemen)o(t)i Fm(MPI)p 1586 1841 V 16 w(FILE)p 1693 1841 V 16 w(SYNC)f Fs(as)189 1898 y(a)g(temp)q(orally)i(sync)o(hronizing)g(function.)20 b(When)14 b(using)f(suc)o(h)h(an)e(implemen)o(tation,)j(the)e(\\sync-) 189 1954 y(barrier-sync")22 b(construct)f(ab)q(o)o(v)o(e)g(can)h(b)q(e) g(replaced)h(b)o(y)f(a)f(single)i(\\sync".)39 b(The)22 b(results)g(of)189 2011 y(using)15 b(suc)o(h)g(co)q(de)g(with)f(an)h (implemen)o(tation)h(for)d(whic)o(h)j Fm(MPI)p 1285 2011 V 15 w(FILE)p 1391 2011 V 16 w(SYNC)f Fs(is)g(not)f(temp)q(orally)189 2067 y(sync)o(hronizing)i(is)g(unde\014ned.)22 b(\()p Fl(End)15 b(of)i(advic)n(e)f(to)g(users.)p Fs(\))75 2187 y Fm(Asynchronous)h(I/O)75 2273 y Fs(The)e(b)q(eha)o(vior)g(of)f(a)o (ysnc)o(hronous)g(I/O)h(op)q(erations)f(is)h(determined)h(b)o(y)e (applying)i(the)f(rules)g(sp)q(eci\014ed)75 2329 y(ab)q(o)o(v)o(e)g (for)f(sync)o(hronous)i(I/O)f(op)q(erations.)166 2386 y(The)c(follo)o(wing)h(examples)g(all)h(access)e(a)g(pre-existing)i (\014le)f(\\m)o(y\014le".)19 b(W)l(ord)11 b(10)f(in)i(m)o(y\014le)g (initially)75 2442 y(con)o(tains)j(the)h(in)o(teger)f Ff(2)p Fs(.)20 b(Eac)o(h)15 b(example)h(b)q(oth)f(writes)g(to,)f(and)i (reads)f(from,)f(w)o(ord)g(10.)166 2499 y(First)h(consider)h(the)f (follo)o(wing)h(co)q(de)g(fragmen)o(t:)75 2591 y Fg(int)23 b(a)h(=)g(4;)75 2647 y(MPI_File_open\()e(MPI_COMM_WORLD,)g('myfile',) 433 2704 y(MPI_MODE_RDWR,)g(MPI_INFO_NULL,)g(fh)h(\))h(;)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Page: 53 9 53 54 bop 75 -100 a Fj(10.6.)34 b(CONSISTENCY)16 b(AND)f(SEMANTICS)886 b Fs(53)75 49 y Fg(MPI_File_set_view\()21 b(fh,)j(0,)f(MPI_INT,)g (MPI_INT,)g(``native'',)f(MPI_INFO_NULL)g(\))i(;)75 106 y(/*)g(MPI_File_set_atomicity\()c(fh,)k(TRUE)f(\))h(;)71 b(Use)24 b(this)f(to)h(set)f(atomic)g(mode.)g(*/)75 162 y(MPI_File_Iwrite_explicit\(f)o(h,)e(10,)i(a,)h(1,)g(MPI_INT,)e (&reqs[0]\))h(;)75 219 y(MPI_File_Iread_explicit\(fh)o(,)45 b(10,)23 b(b,)h(1,)g(MPI_INT,)e(&reqs[1]\))h(;)75 275 y(MPI_Wait_all\(2,)f(reqs,)h(statuses\))g(;)75 369 y Fs(F)l(or)f(async)o(hronous)g(data)g(access)h(op)q(erations,)h Fm(MPI)e Fs(sp)q(eci\014es)i(that)e(the)g(access)h(o)q(ccurs)g(an)o (ytime)75 425 y(b)q(et)o(w)o(een)c(the)g(call)g(to)f(the)h(async)o (hronous)g(data)e(access)i(routine)g(and)g(the)g(return)g(from)e(the)i (corre-)75 482 y(sp)q(onding)c(request)e(complete)i(routine.)k(Th)o (us,)14 b(either)g(executing)h(the)e(read)h(b)q(efore)g(the)f(write,)h (or)f(the)75 538 y(write)19 b(b)q(efore)f(the)h(read)f(is)i(consisten)o (t)e(with)h(program)e(order.)30 b(If)19 b(atomic)f(mo)q(de)h(is)g(set,) g(then)g Fm(MPI)75 595 y Fs(guaran)o(tees)14 b(sequen)o(tial)i (consistency)l(,)f(and)g(the)g(program)f(will)i(read)f(either)g Ff(2)f Fs(or)h Ff(4)f Fs(in)o(to)h Fg(b)p Fs(.)k(If)c(atomic)75 651 y(mo)q(de)i(is)g(not)f(set,)g(then)h(sequen)o(tial)h(consistency)f (is)g(not)f(guaran)o(teed)g(and)h(the)g(program)e(ma)o(y)h(read)75 708 y(something)f(other)g(than)g Ff(2)g Fs(or)g Ff(4)g Fs(due)h(to)e(the)h(o)o(v)o(erlapping)h(data)f(access.)166 764 y(Similarly)l(,)i(the)e(follo)o(wing)h(co)q(de)g(fragmen)o(t)e(do)q (es)i(not)e(de\014ne)j(a)e(program)f(order:)75 858 y Fg(int)23 b(a)h(=)g(4;)75 914 y(MPI_File_open\()e(MPI_COMM_WORLD,)g ('myfile',)433 971 y(MPI_MODE_RDWR,)g(MPI_INFO_NULL,)g(fh)h(\))h(;)75 1027 y(MPI_File_set_view\()d(fh,)j(0,)f(MPI_INT,)g(MPI_INT,)g (``native'',)f(MPI_INFO_NULL)g(\))i(;)75 1084 y(/*)g (MPI_File_set_atomicity\()c(fh,)k(TRUE)f(\))h(;)71 b(Use)24 b(this)f(to)h(set)f(atomic)g(mode.)g(*/)75 1140 y (MPI_File_iwrite_explicit\(f)o(h,)e(10,)i(a,)h(1,)g(MPI_INT,)e (&reqs[0]\))h(;)75 1197 y(MPI_File_iread_explicit\(fh)o(,)45 b(10,)23 b(b,)h(1,)g(MPI_INT,)e(&reqs[1]\))h(;)75 1253 y(MPI_Wait\(&reqs[0],)e(&status\))i(;)75 1310 y(MPI_Wait\(&reqs[1],)e (&status\))i(;)75 1403 y Fs(If)c(atomic)g(mo)q(de)h(is)f(set,)h(either) g Ff(2)f Fs(or)f Ff(4)h Fs(will)i(b)q(e)f(read)f(in)o(to)g Fg(b)p Fs(.)32 b(Again,)20 b Fm(MPI)e Fs(do)q(es)i(not)f(guaran)o(tee) 75 1460 y(sequen)o(tial)d(consistency)h(in)f(non-atomic)f(mo)q(de.)166 1516 y(On)h(the)f(other)g(hand,)g(the)g(follo)o(wing)h(co)q(de)g (fragmen)o(t:)75 1610 y Fg(int)23 b(a)h(=)g(4;)75 1667 y(MPI_File_open\()e(MPI_COMM_WORLD,)g('myfile',)433 1723 y(MPI_MODE_RDWR,)g(MPI_INFO_NULL,)g(fh)h(\))h(;)75 1779 y(MPI_File_set_view\()d(fh,)j(0,)f(MPI_INT,)g(MPI_INT,)g(``native'',)f (MPI_INFO_NULL)g(\))i(;)75 1836 y(MPI_File_iwrite_explicit\(f)o(h,)d (10,)i(a,)h(1,)g(MPI_INT,)e(&reqs[0]\))h(;)75 1892 y (MPI_Wait\(&reqs[0],)e(&status\))i(;)75 1949 y (MPI_File_iread_explicit\(fh)o(,)45 b(10,)23 b(b,)h(1,)g(MPI_INT,)e (&reqs[1]\))h(;)75 2005 y(MPI_Wait\(&reqs[1],)e(&status\))i(;)75 2099 y Fs(de\014nes)16 b(the)g(same)e(ordering)i(as:)75 2193 y Fg(int)23 b(a)h(=)g(4;)75 2249 y(MPI_File_open\()e (MPI_COMM_WORLD,)g('myfile',)433 2306 y(MPI_MODE_RDWR,)g (MPI_INFO_NULL,)g(fh)h(\))h(;)75 2362 y(MPI_File_set_view\()d(fh,)j(0,) f(MPI_INT,)g(MPI_INT,)g(``native'',)f(MPI_INFO_NULL)g(\))i(;)75 2419 y(MPI_File_write_explicit\(fh)o(,)d(10,)j(a,)f(1,)h(MPI_INT,)f (&status)f(\))i(;)75 2475 y(MPI_File_read_explicit\(fh,)44 b(10,)24 b(b,)f(1,)h(MPI_INT,)f(&status)f(\))i(;)75 2569 y Fs(Since)143 2663 y Fi(\017)f Fs(non-concurren)o(t)15 b(op)q(erations)h(on)f(a)g(single)h(\014le)g(handle)h(are)e(sequen)o (tially)i(consisten)o(t,)e(and)-32 46 y Fk(1)-32 103 y(2)-32 159 y(3)-32 215 y(4)-32 272 y(5)-32 328 y(6)-32 385 y(7)-32 441 y(8)-32 498 y(9)-40 554 y(10)-40 611 y(11)-40 667 y(12)-40 724 y(13)-40 780 y(14)-40 836 y(15)-40 893 y(16)-40 949 y(17)-40 1006 y(18)-40 1062 y(19)-40 1119 y(20)-40 1175 y(21)-40 1232 y(22)-40 1288 y(23)-40 1345 y(24)-40 1401 y(25)-40 1457 y(26)-40 1514 y(27)-40 1570 y(28)-40 1627 y(29)-40 1683 y(30)-40 1740 y(31)-40 1796 y(32)-40 1853 y(33)-40 1909 y(34)-40 1966 y(35)-40 2022 y(36)-40 2078 y(37)-40 2135 y(38)-40 2191 y(39)-40 2248 y(40)-40 2304 y(41)-40 2361 y(42)-40 2417 y(43)-40 2474 y(44)-40 2530 y(45)-40 2587 y(46)-40 2643 y(47)-40 2699 y(48)p eop %%Page: 54 10 54 55 bop 75 -100 a Fs(54)1342 b Fj(CHAPTER)15 b(10.)35 b(I/O)143 49 y Fi(\017)23 b Fs(the)15 b(program)f(fragmen)o(ts)g(sp)q (ecify)i(an)g(order)f(for)f(the)h(op)q(erations,)75 143 y Fm(MPI)e Fs(guaran)o(tees)g(that)h(b)q(oth)g(program)f(fragmen)o(ts)f (will)k(read)e(the)g(v)m(alue)h Ff(4)e Fs(in)o(to)h Fg(b)p Fs(.)20 b(There)14 b(is)g(no)g(need)75 200 y(to)h(set)g(atomic)g(mo)q (de)g(for)g(this)g(example.)166 256 y(Similar)i(considerations)f(apply) g(to)e(o)o(v)o(erlapping)i(accesses)g(of)e(the)i(form:)75 350 y Fg(MPI_File_write_all_begin\(f)o(h,...\))k(;)75 406 y(MPI_File_iread\(fh,...\))h(;)75 463 y(MPI_Wait\(fh,...\))h(;)75 519 y(MPI_File_write_all_end\(fh,)o(...\))f(;)166 613 y Fs(Recall)h(that)e(constrain)o(ts)f(go)o(v)o(erning)h(consistency)i (and)e(seman)o(tics)h(are)f(not)g(relev)o(en)o(t)g(to)g(the)75 669 y(follo)o(wing:)75 763 y Fg(MPI_File_write_all_begin\(f)o(h,...\))g (;)75 820 y(MPI_File_read_all_begin\(fh)o(,...\))h(;)75 876 y(MPI_File_read_all_end\(fh,.)o(..\))g(;)75 933 y (MPI_File_write_all_end\(fh,)o(...\))g(;)75 1026 y Fs(since)16 b(split)h(collectiv)o(e)g(op)q(erations)e(on)g(the)g(same)g(\014le)h (handle)h(ma)o(y)d(not)h(o)o(v)o(erlap)g(\()p Fl(cf.)20 b Fs(Sect.)g(10.4.5\).)-1987 b Fi(?)15 b Fk(\(Fin2\))75 1170 y Fp(10.7)60 b(Filet)n(yp)r(e)19 b(Constructo)n(rs)75 1273 y Fn(10.7.1)49 b(Intro)q(duction)75 1359 y Fs(Common)20 b(I/O)h(op)q(erations)f(\(e.g.,)h(rank-ordered)f(blo)q(c)o(ks,)i (structured)f(arra)o(y)e(accesses,)j(etc.\))35 b(are)75 1415 y(easily)20 b(expressed)g(using)g(the)f(previously)h(de\014ned)h (read/write)e(op)q(erations)g(and)g(carefully)i(de\014ned)75 1471 y(\014let)o(yp)q(es.)i(In)17 b(order)f(to)f(simplify)j(generation) e(of)g(common)f(\014let)o(yp)q(es,)i(the)f(follo)o(wing)h Fm(MPI)e Fs(datat)o(yp)q(e)75 1528 y(constructors)f(are)h(pro)o(vided.) 166 1584 y(Eac)o(h)j(new)g(\014let)o(yp)q(e,)i Fm(newt)o(yp)q(e)p Fs(,)g(consists)f(of)e(zero)h(or)g(more)g(copies)h(of)f(the)g(base)g(t) o(yp)q(e)g Fm(oldt)o(yp)q(e)p Fs(,)75 1641 y(p)q(ossibly)h(separated)e (b)o(y)g(holes.)27 b(The)17 b(datat)o(yp)q(es)g(created)g(b)o(y)g (these)h(routines)f(are)g(not)g(committed.)75 1697 y Fm(Oldt)o(yp)q(e)e Fs(can)f(b)q(e)h(an)o(y)e Fm(MPI)g Fs(basic)h(or)f(deriv)o(ed)i(datat)o(yp)q(e,)e(and)h(need)g(not)g(b)q (e)g(con)o(tiguous.)19 b(The)14 b(exten)o(t)75 1754 y(of)19 b(the)g(new)h(\014let)o(yp)q(e)g(is)g(a)e(nonnegativ)o(e)i(in)o(teger)f (m)o(ultiple)i(of)e(the)g(exten)o(t)g(of)g(the)g(base)h(t)o(yp)q(e.)32 b(All)75 1810 y(\014let)o(yp)q(e)16 b(constructors)f(return)g(a)g (success)h(or)e(failure)i(co)q(de.)166 1867 y(Although)f(it)f(is)h(exp) q(ected)g(that)f(these)g(routines)g(will)i(b)q(e)f(commonly)g(used)f (in)h(a)f(collectiv)o(e)i(fash-)75 1923 y(ion,)h(all)g(the)f(\014let)o (yp)q(e)h(constructors)e(are)h(indep)q(enden)o(t.)25 b(This)17 b(facilitates)g(using)f(these)h(constructors)75 1980 y(for)e(de\014ned)h(datat)o(yp)q(es)f(as)g(w)o(ell)h(as)f(\014let) o(yp)q(es.)1967 46 y Fk(1)1967 103 y(2)1967 159 y(3)1967 215 y(4)1967 272 y(5)1967 328 y(6)1967 385 y(7)1967 441 y(8)1967 498 y(9)1959 554 y(10)1959 611 y(11)1959 667 y(12)1959 724 y(13)1959 780 y(14)1959 836 y(15)1959 893 y(16)1959 949 y(17)1959 1006 y(18)1959 1062 y(19)1959 1119 y(20)1959 1175 y(21)1959 1232 y(22)1959 1288 y(23)1959 1345 y(24)1959 1401 y(25)1959 1457 y(26)1959 1514 y(27)1959 1570 y(28)1959 1627 y(29)1959 1683 y(30)1959 1740 y(31)1959 1796 y(32)1959 1853 y(33)1959 1909 y(34)1959 1966 y(35)1959 2022 y(36)1959 2078 y(37)1959 2135 y(38)1959 2191 y(39)1959 2248 y(40)1959 2304 y(41)1959 2361 y(42)1959 2417 y(43)1959 2474 y(44)1959 2530 y(45)1959 2587 y(46)1959 2643 y(47)1959 2699 y(48)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF From mpi-core-human@mcs.anl.gov Fri May 23 14:13:14 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA08531; Fri, 23 May 1997 14:13:13 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA21218 for mpi-core-out; Fri, 23 May 1997 13:14:33 -0500 Received: from win62.nas.nasa.gov (win62.nas.nasa.gov [129.99.50.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA21213 for ; Fri, 23 May 1997 13:14:26 -0500 Received: (from ies@localhost) by win62.nas.nasa.gov (8.8.3/NAS.6.1) id LAA25322; Fri, 23 May 1997 11:14:17 -0700 (PDT) Date: Fri, 23 May 1997 11:14:17 -0700 (PDT) Message-Id: <199705231814.LAA25322@win62.nas.nasa.gov> From: "Ian E. Stockdale" To: mpi-core@mcs.anl.gov In-reply-to: <19970522142848.HI03348@k2.llnl.gov> (johnmay@llnl.gov) Subject: Re: error handling Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Regarding John May's comments on Marc Snir's option "C": >Con: since no type checking is possible even at runtime, and since >all error handlers look alike, there is a danger that the user >will, for example, associate the same error handler with all >three kinds of object. This won't be detected until an actual ... I agree with this assessment. While we are dealing with admittedly rare circumstances - a user who *does* write an error handler - it seems at least a little undesireable to introduce an additional runtime failure mode into a situation which only exists when you already have an error. >Pro: perhaps the user *wants* to write one error handler for all >three handles. If the handler ignores the handle passed in, it can >serve all three types of handle. This isn't a large burden. Also, I expect that users will only be writing error handlers for one of the three possibilities (comm, file, win) anyways. >On balance, I favor B slightly. Agreed. My guess is that most error handlers (not that there will be a lot written) will be written in C rather than Fortran or C++. Taking advantage of C type checking seems like a good thing. -Ian -- Ian E. Stockdale MRJ Technology Solutions, NASA Ames Research Center ies@nas.nasa.gov (415) 604-4628 From mpi-core-human@mcs.anl.gov Fri May 23 16:48:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA09809; Fri, 23 May 1997 16:48:23 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA25180 for mpi-core-out; Fri, 23 May 1997 15:52:37 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA25175 for ; Fri, 23 May 1997 15:52:33 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id PAA08590; Fri, 23 May 1997 15:52:30 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id PAA12992; Fri, 23 May 1997 15:52:27 -0500 (EST) Date: Fri, 23 May 1997 15:52:27 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: "Ian E. Stockdale" cc: mpi-core@mcs.anl.gov Subject: Re: error handling In-Reply-To: <199705231814.LAA25322@win62.nas.nasa.gov> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Fri, 23 May 1997, Ian E. Stockdale wrote: > Regarding John May's comments on Marc Snir's option "C": > >Con: since no type checking is possible even at runtime, and since > >all error handlers look alike, there is a danger that the user > >will, for example, associate the same error handler with all > >three kinds of object. This won't be detected until an actual ... > > I agree with this assessment. While we are dealing with admittedly > rare circumstances - a user who *does* write an error handler - it > seems at least a little undesireable to introduce an additional > runtime failure mode into a situation which only exists when you > already have an error. I'm not so sure. When you write an error handler, you're going to write it for a specific type, right? For example, if I write a communicator error handler, I will only use it on comms. Who's going to try to use a communicator error handler on a window? Using (void *) is a very minor loss of type matching. > >Pro: perhaps the user *wants* to write one error handler for all > >three handles. If the handler ignores the handle passed in, it can > >serve all three types of handle. There is an actual use for this -- having a centralized error handler that can dispatch the error to an appropriate routine. This can be useful in a library environment (e.g. OOMPI) that can catch *all* errors, do some processing (perhaps profiling types of things) and then pass the error on to the user. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Fri May 23 19:05:28 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA10958; Fri, 23 May 1997 19:05:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA28322 for mpi-core-out; Fri, 23 May 1997 18:08:08 -0500 Received: from win62.nas.nasa.gov (win62.nas.nasa.gov [129.99.50.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA28317 for ; Fri, 23 May 1997 18:08:03 -0500 Received: (from ies@localhost) by win62.nas.nasa.gov (8.8.3/NAS.6.1) id QAA26144; Fri, 23 May 1997 16:07:54 -0700 (PDT) Date: Fri, 23 May 1997 16:07:54 -0700 (PDT) Message-Id: <199705232307.QAA26144@win62.nas.nasa.gov> From: "Ian E. Stockdale" To: mpi-core@mcs.anl.gov In-reply-to: (message from Jeff Squyres on Fri, 23 May 1997 15:52:27 -0500 (EST)) Subject: Re: error handling Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Jeff writes: >error handler, I will only use it on comms. Who's going to try to use a >communicator error handler on a window? No one. On purpose, that is. I can imagine doing it by accident. > Using (void *) is a very minor loss of type matching. "Minor" here is one of those things that can't be proved. Type matching is a good thing, but it is not the "be all and end all". We have to guess at the consequences of keeping or dropping it here. >can dispatch the error to an appropriate routine. This can be useful in a >library environment (e.g. OOMPI) that can catch *all* errors, do some >processing (perhaps profiling types of things) and then pass the error on >to the user. I think you can achieve the same results with "B", although perhaps it is more difficult. My guess is that the common case will be served by C type matching, and that the benefits obtained via (void *) don't make up the deficit. We'll have to agree to disagree. -Ian -- Ian E. Stockdale MRJ Technology Solutions, NASA Ames Research Center ies@nas.nasa.gov (415) 604-4628 From mpi-core-human@mcs.anl.gov Tue May 27 08:09:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA22568; Tue, 27 May 1997 08:09:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA18648 for mpi-core-out; Tue, 27 May 1997 07:11:56 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA18638 for ; Tue, 27 May 1997 07:11:49 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id HAA03074; Tue, 27 May 1997 07:11:46 -0500 (CDT) Date: Tue, 27 May 1997 07:11:46 -0500 (CDT) Message-Id: <199705271211.HAA03074@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: MPI-2 typos (forwarded) Cc: Alexander.Supalov@gmd.de (Alexander Supalov) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The following comments were received. I am forwarding them to core for completeness and any comments Steve ---------------------------------------------------------------------- Dear all, since I don't know whom to send suggested corrections to Section 1 - 3 of the MPI-2 draft of May 8, 1997, here they are. 7/2-4 update marks are in the wrong margin 9/29 (void *) is bracketed not everywhere in the text (e.g., 11/26) 9/35,41 different fonts are used for "INTEGER (KIND=... )" 10/3,4,10-12 would look better in 2.6.1 10/10-12 these two sentences seem to contradict each other. Why avoiding only the lowercase prefixes? 11/34 not all C++ compilers can do namespacing yet 11/43,44 IMHO, a C compiler is unlikely to see anything that has been discarded by the preprocessor. How can the compiler "require that the source protected this way be legal C code"? What particular compiler was caught doing this? 12/4 "semantic descriptions" 13/36 would do better before 13/32 15/43.5 syntax error. Better: MPI_Init(&argc,&argv) 18/40-43 these lines echo 18/21-24 almost literally 19/6 closing ) is missing 21/32 "implementation" 22/20-23 strictly speaking, the mentioned output argument has type (int *) rather than just (int) 24/22.5 reference to "this document" from within MPI 1.1 would look strange unless MPI 1.1 and MPI-2 documents were going to be merged; perhaps, "MPI-2 specs" are meant here 25/24, 26/2 see 24/22.5 above 26/35-41 I found that at least on IBM SP2 a jth process cannot send data of length 0; perhaps, a clarification for all the collective calls of this kind is necessary Generally, it's strange (although historically clear) to see "I/O" chapter behind "Miscellany". I'd suggest the "I/O" to be put before "Language bindings". Hopefully, it's not too late. Best regards. Alexander Dr. Alexander Supalov GMD -- German National Research Center for Information Technology SCAI -- Institute for Algorithms and Scientific Computing Schloss Birlinghoven phone: +49 2241 14 2371 53754 Sankt Augustin fax: +49 2241 14 2102 Germany e-mail: supalov@gmd.de From mpi-core-human@mcs.anl.gov Thu May 29 13:52:15 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA20293; Thu, 29 May 1997 13:52:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA17268 for mpi-core-out; Thu, 29 May 1997 12:55:14 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA17262; Thu, 29 May 1997 12:55:10 -0500 Message-Id: <199705291755.MAA17262@antares.mcs.anl.gov> To: Bill Nitzberg cc: mpi-core@mcs.anl.gov Subject: Re: Updated I/O Consistency & Semantics Section In-Reply-to: Message from Bill Nitzberg of "Thu, 22 May 1997 22:30:12 -0700." <199705230530.WAA26234@win153.nas.nasa.gov> Date: Thu, 29 May 1997 12:55:07 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This still contains text about the data being "written to permanent storage" (page 47, line 45). I'm uncomfortable with this. For example, it is stronger than my reading of POSIX 1003.1, 6.6, File Synchronization. See in particular lines 599-601 and 614-617 in that document. Basically, POSIX defines "file synchronization" by its observable effect, not a particular implementation strategy. I believe we should do the same. Also, "interleaved collective operations" are mentioned without definition. This is too informal. Bill From mpi-core-human@mcs.anl.gov Thu May 29 17:22:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA22314; Thu, 29 May 1997 17:21:50 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA23239 for mpi-core-out; Thu, 29 May 1997 16:26:12 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA23213; Thu, 29 May 1997 16:25:24 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id OAA13303; Thu, 29 May 1997 14:24:59 -0700 From: "William Saphir" Message-Id: <9705291424.ZM13302@nsgi3.lbl.gov> Date: Thu, 29 May 1997 14:24:58 -0700 In-Reply-To: Dick Treumann "Re: (Fwd) Re: MPI_JOIN" (May 29, 4:49pm) References: <9705291301.ZM13086@nsgi3.lbl.gov> <338DEBEC.59E2@pok.ibm.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: Dick Treumann Subject: Re: MPI_JOIN Cc: mpi-dynamic@mcs.anl.gov, mpi-core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On May 29, 4:49pm, Dick Treumann wrote: > > Your example is the sort of thing I had in mind. I am not sure what the > difference is between describing something as erroneous and saying its > outcome is undefined. I expect correct programs only to do things which > have defined outcomes. I agree this is a subtle difference. Obviously a program that uses JOIN to connect connected processes is not guaranteed to work. By calling it undefined, though, we allow implementations to support it and applications to target this specific implementation. We label the application as "not portable" rather than "incorrect", which is kinder and gentler. On the other hand, I do think it was a big mistake, for example, not to label as "incorrect" programs that rely on buffering to avoid deadlock. This has caused lots of headaches and incorrect assumptions about what MPI requires. I guess I wouldn't have too much of a problem declaring it to be erroneous. I suspect my original instinct to make it undefined was partly based on several prior experiences trying to label something as "erroneous" and getting "corrected" by the Forum. This is cc'd to mpi-dynamic/core in case someone can give a clear explanation of the reason for distinguishing between undefined and erroneous. Bill ps. Examples in the current dynamic chapter of "undefined" behavior. If the program named in \mpiarg{command} does not call \mpifunc{MPI\_INIT}, but instead forks a process that calls \mpifunc{MPI\_INIT}, the results are undefined. Implementations may allow this case to work but are not required to. if \mpifunc{service\_name} has already been published within the scope determined by \mpiarg{info}, the behavior of \mpifunc{MPI\_PUBLISH\_NAME} is undefined. \mpiarg{port\_name} should be the name of a port established by \mpifunc{MPI\_OPEN\_PORT} and not yet deleted by \mpifunc{MPI\_CLOSE\_PORT}. If it is not, the result is undefined. \item[\info{ip\_address}] Value contains IP address at which to establish a \mpiarg{port}. If the address is not a valid IP address of the host on which the \mpifunc{MPI\_OPEN\_PORT} call is made, the results are undefined. (Reserved for \mpifunc{MPI\_OPEN\_PORT} only). If a function does not recognize a key, it will ignore it, unless otherwise specified. If an implementation recognizes a key but does not recognize the format of the corresponding value, the result is undefined. From mpi-core-human@mcs.anl.gov Tue Jun 3 19:14:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA12558; Tue, 3 Jun 1997 19:14:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA06626 for mpi-core-out; Tue, 3 Jun 1997 18:16:29 -0500 Received: from palrel3.hp.com (palrel3.hp.com [156.153.255.219]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA19362; Thu, 29 May 1997 14:13:48 -0500 Received: from mailhost.rsn.hp.com (daemon@mailhost.rsn.hp.com [15.99.1.2]) by palrel3.hp.com with ESMTP (8.7.5/8.7.3) id MAA28716; Thu, 29 May 1997 12:13:44 -0700 (PDT) Received: from getbent (getbent.rsn.hp.com) by mailhost.rsn.hp.com with SMTP (1.37.109.18/15.5+ECS 3.4 ) id AA070083063; Thu, 29 May 1997 14:11:03 -0500 Message-Id: <338DD4CB.6EC6@convex.hp.com> Date: Thu, 29 May 1997 14:11:07 -0500 From: Steve Landherr Organization: Convex Division of Hewlett-Packard Company X-Mailer: Mozilla 3.0 (X11; I; HP-UX B.10.01 9000/712) Mime-Version: 1.0 To: William Gropp Cc: Bill Nitzberg , mpi-core@mcs.anl.gov Subject: Re: Updated I/O Consistency & Semantics Section References: <199705291755.MAA17262@antares.mcs.anl.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk William Gropp wrote: > > This still contains text about the data being "written to permanent storage" > (page 47, line 45). I'm uncomfortable with this. For example, it is stronger > than my reading of POSIX 1003.1, 6.6, File Synchronization. See in particular > lines 599-601 and 614-617 in that document. Basically, POSIX defines "file > synchronization" by its observable effect, not a particular implementation > strategy. I believe we should do the same. The POSIX systems to which I have access all include moving modified file data and attributes to permanent storage in their description of the fsync(2) system call. And from my knowledge of the internals of three of these systems (HP-UX, SPP-UX, and ConvexOS), I know that the system call does not return until the writes have completed. This functionality is necessary to support applications that periodically checkpoint themselves. They don't want to remove the old checkpoint file until they are sure the new file is on disk. -- Steve Landherr -- landherr@convex.hp.com -- (972)497-4104 Kernel Software Development -- File Systems Convex Division of Hewlett-Packard Company -- Richardson, Texas From mpi-core-human@mcs.anl.gov Fri Jun 6 18:26:06 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA10738; Fri, 6 Jun 1997 18:25:34 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA24819 for mpi-core-out; Fri, 6 Jun 1997 17:27:03 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA24814 for ; Fri, 6 Jun 1997 17:26:59 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id RAA16075 for ; Fri, 6 Jun 1997 17:26:55 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id RAA04339 for ; Fri, 6 Jun 1997 17:26:42 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id WAA10569; Fri, 6 Jun 1997 22:26:41 GMT From: "Eric Salo" Message-Id: <9706061726.ZM10567@sgi.com> Date: Fri, 6 Jun 1997 17:26:40 -0500 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: MPI_ADDRESS_KIND Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Okay, so now that I'm actually trying to implement some of this stuff, I've run into the following problem: - We define a new constant, MPI_ADDRESS_KIND, for Fortran codes - We require that the Fortran header file be called mpif.h - Some implementations (like ours) can support multiple pointer sizes At the moment, we have a single mpif.h and two different libmpi's: a 32-bit version and a 64-bit version. So, my question to y'all is, what value should we be using for MPI_ADDRESS_KIND??? There is no single answer which works. I can't even just make it equal to the larger value of 8 bytes, because it's defined to be the *same size* as a C pointer. So it would appear that in order to support MPI-2, I'll need two different versions of mpif.h. Either that, or we need to add support for conditional compiling to our f77 front-end (since mpif.h must also work for f77). Am I missing something? -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Sat Jun 7 10:29:47 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA16217; Sat, 7 Jun 1997 10:29:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA01149 for mpi-core-out; Sat, 7 Jun 1997 08:44:10 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA01144; Sat, 7 Jun 1997 08:44:06 -0500 Message-Id: <199706071344.IAA01144@antares.mcs.anl.gov> To: "Eric Salo" cc: mpi-core@mcs.anl.gov Subject: Re: MPI_ADDRESS_KIND In-Reply-to: Message from "Eric Salo" of "Fri, 06 Jun 1997 17:26:40 -0500." <9706061726.ZM10567@sgi.com> Date: Sat, 07 Jun 1997 08:44:04 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | So it would appear that in order to support MPI-2, I'll need two different | versions of mpif.h. Either that, or we need to add support for conditional | compiling to our f77 front-end (since mpif.h must also work for f77). Surely this problem isn't restricted to mpif.h. Doesn't the compiler have separate system-include directories for the different sizes? If not, can't you specify an alternate include search path? Bill From mpi-core-human@mcs.anl.gov Sun Jun 8 16:35:10 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA28698; Sun, 8 Jun 1997 16:35:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA13325 for mpi-core-out; Sun, 8 Jun 1997 15:39:01 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA11499; Sun, 8 Jun 1997 12:21:40 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id MAA05223; Sun, 8 Jun 1997 12:21:37 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id MAA16403; Sun, 8 Jun 1997 12:21:35 -0500 (CDT) From: Eric Salo Received: by mrjones.cray.com (8.8.0/btd-b3) id RAA16640; Sun, 8 Jun 1997 17:21:35 GMT Message-Id: <199706081721.RAA16640@mrjones.cray.com> Date: Sun, 8 Jun 1997 17:21:35 GMT To: "Eric Salo" , William Gropp Subject: Re: MPI_ADDRESS_KIND Cc: mpi-core@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Surely this problem isn't restricted to mpif.h. Doesn't the compiler have > separate system-include directories for the different sizes? If not, can't > you specify an alternate include search path? The system headers are all in C, so #ifdef is our friend here and you can get by with a single copy of each. Yes, we could use '-I' to point at alternate include directories at compile time but my feeling is that this solution would be both error-prone and annoying for users. Surely I'm not the first vendor to run into this issue - what are the rest of y'all doing? -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Mon Jun 9 08:37:41 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA04556; Mon, 9 Jun 1997 08:37:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA21106 for mpi-core-out; Mon, 9 Jun 1997 07:40:25 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA21101 for ; Mon, 9 Jun 1997 07:40:19 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id HAA18103; Mon, 9 Jun 1997 07:40:17 -0500 (CDT) Date: Mon, 9 Jun 1997 07:40:17 -0500 (CDT) Message-Id: <199706091240.HAA18103@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: revised schedule for document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The chapter authors had a long a productive meeting this past weekend at Argonne. Lots of clean up work was done and items were resolved. With the goal of trying to release as close as possible to a finished version to the Forum, we have decided to delay the release of the document to the full Forum (originally scheduled for June 10). The new schedule is: Friday, June 20: release the final draft version to the Forum. All comments should be posted to mpi-core in addition to any other places. The goal of this release is for everyone to check changes and look over the document before the public release. There will be two versions supplied. One will have changebars for all significant changes since the May 8 release. The other will not and will look like the final document will look. Friday, June 27: last day to make comments. The (hopefully) small edits will be made after June 27. The hope is to have the public release by early July. Steve From mpi-core-human@mcs.anl.gov Mon Jun 9 10:57:45 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA06997; Mon, 9 Jun 1997 10:57:37 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA24212 for mpi-core-out; Mon, 9 Jun 1997 09:58:48 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id JAA24207 for ; Mon, 9 Jun 1997 09:58:41 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA79976; Mon, 9 Jun 1997 10:55:24 -0400 Message-Id: <339C1A0C.167E@pok.ibm.com> Date: Mon, 09 Jun 1997 10:58:20 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: Its done with mirrors Content-Type: multipart/mixed; boundary="------------2781446B794B" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This is a multi-part message in MIME format. --------------2781446B794B Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I thought you all might like to see the most basic Message Passing system so far to identify itself as MPI. It does not appear to include MPI_GET_VERSION() so I do not know what level it may be. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 --------------2781446B794B Content-Type: image/gif; name="mpi.gif" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="mpi.gif" R0lGODlh8wENAfcAAAAAAIAAAACAAICAAAAAgIAAgACAgICAgAkJCYcZDVJcUI6NckkKBZyj tAkkRIZPWkwhRI9EMRI0OEFcdrSRiUc6K83DuCgHBY0jLI2QkSE9SmcPBWg3Sd7cw2NdQ2Bm dkhBVZysl4hlXwkYJ6YiE2iHhIVyf3N8abM9HEhDZSccLSBAZZ6SlOHi3GFhZGgnKaq9uUkl Kmo1MywiRmp4kbEwLgocGIonEKpiXbI+N1VCVbm1m9LS1TY/SQsmMqgtEyUcC2gdCJyPc0tN OXVkYndsd7/AvaOhpzI1Ku7jw5GUpYZ6Y6p/gzYbLxgMCUoeGHmKme/s3cPEzDdCZSkxS9DQ xlVDZTlMSohDRGZQZ3VnUs3CqoiBglccKkg+OKEjLIkpIDdON2t2eKSgljlOZzM0XreytjgH BWhuZngbJ0NcaElOV6mumLNdI7IyRrFZODg4OI6PgJKdlmRTSBgbKFdRUa0vIHceCJZ8Ypkc HVgMBWdtU7u4qcPNvoqElQsvTPn24KouWrSZn4xaSKlTXUgzXWRTO1BfddTc0IdpcKcjHqZi bvj01KI+LbWrk5qtqaSEnHmdnKFFQVFdZ6FXQ6qttVYvSIsvL3YRCHqGebw9KHk0KxocDLw+ NjYpG6SdglocCXZMV6muqSE1N0hQaCkpLhooMTYrLtzUxFZRZ1koLqEvLRowS6BOLJscElVC KyE/WZZkXuTs3Xd5kLwyLpsmE6xvXNzU1+/p1NDNrLa0xKxUSgkPF0gQFpmythohRUYzTczE yCYTFmYRFXU1R2VfUAsbNZdvf7M9KiRPZ2cqNUgnNq9ERzZAWbUwFCgiGWkgGJ6SgFhNOXdw ZHd5e5Kepoh+bxYSF8TNzjcySzlNWpaCglRDN5soHjhQdTYPF3cpLJKcgWRSVxgbNnghGZSD cVQUF8DEq/v77LWrpVNueOHi06xubowxP3qHiHc3ORodGVciGZRUVGxqdrQkFHyAaMS4nDw2 K8DAwP8AAAD/AP//AAAA//8A/wD//////ywAAAAA8wENAUAI/wCjRAEEKAquggcJDhQoEBAu gwYVMpwYUSCuFhQnojMoS9Y6jLKigBQpUFYLdBhLRjG5smTIlhNDontJM8pMmyFNdoS5MufL kh9byszZEmNKhjsFouzItMXHFiyb7mTZoipUq1E+PiT5U2BQhk4Nrnu4LqvTqjqddvy4bl1H qG7BrgSp1uRHj1dDliUJNWtWk3kD35WFCG1au2GvVkWU1i1UlGhFWp0ssu1do1a7MrxYNaNn mxk3Kh298ahA0zZPGkUJeeW/Fv9kzZwtG3bt17I1ljx5WiQ61ilRvrb5u/i/4sjRAUpefDly 58mPb5S4FSFEXIy8FqSIq/vm6wsPbv/1fHO0UMkxWzzUnHL81IY4C+4sy1LlT3Rt7cu6SPLm VaREhURXbhtBplNll02llmNvJbgOKuhZJiEuHuWX0lEYdfcUYIQB5taHVX2o01tQebRfCx10 IBKJBl3o1lu4lJWYToyVGKJhfXEY1GRo5djZiitmtqCMOdkolIFHiQbaZ+RN9I95wN3WF0Yz 1abaTFVBJlxtsry2VG1YmjRblmAqyVCBGB0XhXS/Qcdcc8m5GSdBb4JGZUHobIVaQyhBRBGG no33GaByAUKfSuqhV19qPM2mEkN7PfQjT42dl1qVAf6kGFNE4TQXYDiu+JJTe1WYmYAS3kJh C7dYaJZfZcn/GKNXIWb1IVu2xkrqhKRiFNSIiFh21llP7WWUjOsUdqBdh6n1n1WXdcbjsphR RteFBD5WW38nyVSgnWead+aVS4arUbeypfulcKqttJS7Nm75m4BaqrsTcKzldilObTIHSBKM yPnmc8s5J+dCFBVE50AKb0eQQgoH+vBD25kJn2jtwRdFdgw5zCRFmoHW6U8CUojUSuvNpSd/ e4I81HsuWVsXqaAadhhgJFlmBDxOqODFKKOUQkUdpQjdAwhUUNFDM9lQAUs2Ta+wQjbAlLHC 0yCwoo0GVCB9xRXaNAO2Nj1csQbYX6+hjRou9LDGGmqoPYk2U2izxgekNDPFJIes/zEFKZNY QcoapBRORjanxPOEDODc0M3jl/xgxyrdXPL4KmCgcIkiYKQhziF/kzFJKt4cMroL3nxwSAqk TOBCKmuQcYgLeAOewiQTpKL7IaTcnkrhIHwAAilrp6JDCoRbkcrwpAwvOuyTkLK8NoVP4QIZ 1Rd+SCrapII33n+T8joZrB+ShfYpfMD6Gikc4k0WunvjDQi/t55CKimkQAYZvzdDhhqs0wbu rHAIMriggLzTXTJUpwZSkEF1pLiCE0CwDuVYcGAY7FcGkbOxjq2DGjyLgyeewQdsnAIecniG E7ihIhiogBNaKEsVJvGMZ+wAEXGAxzM+0Z2LPKQKYXgGPP8qkAFUeEckH4CHJ1DRAjHocAu4 6MMzrgEPTnjiE+5AABDSsQ456HAH6wjBNYDAgnX0oRScmMY6sDEKIJQDERlQIhsQIUZ4fKIq KYTHGF4EmEpcgxf1cAc85lCFZjjhE5PgBDzS4RR18MITVZBFH5CAAE4MIRPnYCMCnKAATFah B/AoRrDcYcU+rAMG8FDiCUIQybaUKFiKOYutjEAHBKxgCv6bgDf2t78C+i982tBf9GRXuDWs 4ANW8F8yXJCCKZDBG9LTHxmaAbo1lKEZpJgCCJA3TWzaTRtkIBwI4FY4KuigAl2AwAvA8Lgv XCIPinhcN76wCkXUggSPs0M3nFH/C0Wswg4/iKdASZCHguahG7XIQxowsE0dpMIKu9SBFfI3 AdalwAUTINwU8EcKb2gzFdubQDMO2FHATUB2vHOg6kCwPxcor5i6K9ztUjDRVADQovrbqO72 dz8riM6igEtFRV0gPQeObpggOITrsqkGb5yuo0nF3xT2ZgUd4HICpNCBTPMm0/0B75ms64ET ZlDBCypHYHNCKwaX0xCHGCQOVnTEKZ8BhAXgogpweMYrOnBXJDyjGElIWBRS+AwhIGwgVfAC J55Rh1SoAAhXXEkOOVEHNCDBCc+wRxSq0MZpBHYrmXgGJ4DwibG0wAyleEY0ZJGE04qWEx5A BIVEUcVn/yABCYpcAFSOoMQ72scIqV2CR8TwjHpQiLg2XIkYPIEEC8iiCmt4Rgw7gogreKIY 9DHjEIQ42jCwoS1GCAMSFuCRKnwFVCCyjCw6gIgzIqAZazjeFMqQgm1OwRspaAYIdkmGKaQA mimQGhmox1/9YpWlohvmMqdQwHCuAXTzw10KyqCMV6gCGuBgJ0JX8YMfyLMb9rRnNzxcCw+D WBGu6OdBEZrig5KABAJN6IsLCuNauKIbq8BnHm6cgEtoQQfHw6ol8Me6bTaTdROdKhnsm81m cK9/2ewdGbDavAfi98G521/0iAq4BzqQddFrX0wfyLqTFnUKanioN9Tm5DIMjv8MVljDTieh tsFRj3+Di94kiMpllYoDnNJLhf96l4rngaCBqRBHM3hBBVTA6U3QkZPB4vSc4oCoL/hwgRc2 zQLv8Mcr4oliHcCIrBZkQI0US1hBMPIIQ6hIQBPJz0QE5ZFZkQQs+zGUKBbgFvespA9w4INl 5pISk8yq2NCNRmGqwY2vtCVTY/ELNpBwDqhUoQ9gYYsrQxSsW9nILYwhEVyenSwgIqAHXFWD AFmqTRf47W9rAMFUe5c3/BWCDNe0wn5xOeFJTFPQqSiDF7rwgk1gAAwdnpzk9FkDO0xOn6to RA4kQTl/6hOfHF5FLTYezzyQwBUz3rg8PUwCG8P4B2D/2LEivvDxg4LcFR53hSLs+YNadCMP tUjDJoBROBeAYHi8m4RWvTHS5WHPpSwtaTNGx7qNVtWjgnOoN9zMv13S2xtYheYheHeIMkSP FEqdRPRoMLsP0CADSlCCHGBgBnxgQwpw5wEPpIANbMjd7jyou9yDIfe98yASHxCD8f5XuKU3 MJu/Q7Du+Ca9OvBCBTyY9Fr9dVazVpqD22EIKopQjFx8JkaoYMMQrLGOKoBArnIwBISioIQ5 JCGxjsgKKoIRBVG8ogoPOcIrjGgdI6LCDEOIAy4qUYEtZGUMzQb2OdaRBDPwIQoZiKFIMmCP LkpjHWa4xxZKD4fY98ELjhhJ/wsawA1ErMQISDDlOrgwhou4YLVRaEAFcoGIVJR2HaKYBiL4 cIrtt+AA5dACeMVI66AE0qAiLSAH00ANcZAsMDB/LYAIkyBXtfIUudJtwyIhiLAGjLY+TZYC d2ZnsTMJw7MCibZkU7AGwPBzZZANvWAJjHMJNTBPDUdPNbAKzqAIyLAK9UQP8WQHM6cIdnBy tfBPJ/YDJHCDAAVi3QBjIFZP90RQM1dyMteEIEdQJbdxMvdxNhZQ9lRQL+cKMPdxH5diMCeG rpAAL8AB4nA0fqMDApRNWuUCdLN02+M/k3A/hRNTo5MKtCMGs8AFGdAAlWAGFmABbyd3VRAM UsB3PP/AiI74iHzXiHNXiY8od1LQd36niZwYiZz4iEpQBGIgdsqjNnLGPVImaMPTPfiTCtdQ ChagQZE2i5OmVszBJjGyFzzRMjDxEhdRFqA3FoLSMZ5hKPARFNaxWVVgAcFgBIdoXn2weuoh KXIREwvBCOvxXFy0DkawR8dWFG9hF9H2K/QxMjEiC3xQBGPBFFGACEXAB+zoE8DyAXdkGHeB IC9SLIhxFhKIAHAACzoQN/0Vb3rjX83gPyDQBcoQAy/QDl/wBYpQA/QAhD44kUJoBxZpBzVw kQ73TxcphCQAhCeWY/aUY1noYRzWDRCHUB2HUEv4OIrwcfE0YjAWTxrncij/NmMiVnIwhnNR aHMHZU82BwZ2YHM82AgRMAgikAgZ0JSCwAfpgA+6EAxVIAX4sIiXiJU8sIhVoIhdmXdydwsC uJW3gAq3cImFgYl91wKcmImaaHd694k8wJZ4x5ZyKZfYcASzQHZOpgb9NWBgVz+DgzeJRwd0 YAQD4yYF82iVhla0GDHu4AXWIGy/OAl2hQu2lwst8ICeJxC6IA2diX0HMAQx4HliIAK6eFdw cA/SUAdD4FmHZRHb4QJLYCzu4AGBhQguIFcZcA/FsAcusAfPtxDrwAdo4AXFEHuyAAP34Hnk NxbY4AVVYHq5sA5H4AmZEAdyEAfngAv4cArCBhRZ/9EHcoAGr4l7oiCd67AGpUUY7LkO1VAB jmAEZmAEfZAZ6/AB1rAY+CAHLiANwuZKAroY0OIUiFAFHwAAcDBVOgAMlgAO4BAByFADDecM mmAHzkALHEkLdsCh9EALNfAFG0mhFOpwGtlwJoqhQDhzDucMQOhwJOBh+hSTAiWEkuNPiiA5 AVViSIhPLxaTL9aEXohjACWEPLgKWCAPx+AHx7ANgmAGZqALbRcMjIgPcEelVomli0ilVLmI cDedjEh3XQl3caeWbhmmiaiIniiXbqmJbGmX2GCXd9l3cTmXnCinYdl3Yqmnb6kEUPABOsBT zUMKyTBVcrYCkwBNLDUC1/+QDgxjeRtUeZBKeaLhECByGrA2F1/xKn6RK7qoHjKSEtvRApkH EWOhi1wRERBDjL7mJx7REgcRFJzRECkRF4rCF7hQGHNRFEbhEtoWK/nhIalqImhhGeHWK3Bx Ix5SLLrqFDywDmJwAdBQA26QA27gBrQAol+QAxSardn6oTWQrQ73oRxKociQoR2qkRN6kea6 kQ5HCxO5cPTggykKUC/mg15IrznqhfWkkkdICduQDoZolfhgBlSKD/QZDPhgAfhQpQTbiFla pXtHpW+HD1Yad1j6pVLwlTyAD33HiHh3d5e4iXeJd7cAlybrpmB5p3LJKp/osjxwllUhd1Vx ljH/K5Y4yyq3cJZ7Knd+4A7qsGQ64DdLJmhHNlFWMAV0sEiLOamUtkG2WGxoIQoVUAVZ8QFC gH33cA63UAUusH1yUAxVgArVIA2osA4uYA2IkASVUAo7oB5nkQEeQCHUUAzdwQM9EH7Ckp5G lJ/WkASquQVJELYdgH/zl2ry17d1C15GlGzrgA/SiQswwA0W8LhwcA5ZoQSGYH4ZAFjtuAZy tQ6VoJ4fMA1GUAUZcHt44RcnUgLYJQvYAAdskBUhULXQZQ3pwAZoMAdtgQiF8RR0gSDS0ha/ qG1Q4ARBkAOdsLzZWgOd4LzLS6Fu0HDhOr3h2glukK4c6qHZypEOt6G0/wCRX9ChiqChF8mh L5qjNKeS+lRztaBPS4hPF4djb8AEAgsDumAERpC/bWcG6SAIkrCiinAJyNAIEfcOY7C/ZrCI DSumWwl32FCVENx3VzqxjpiJGEzBK9umOzuJPauJNguWcmoVNGunKmuXM5unbBnCerqzYqmz LmynKbyzeeenXRc7gmY2dPM2hNMMpXANLCCpZwUIAUMnk9ccD8OYSlIxXmGrkHK2s0a8PdQd SdAqgRWbn1cQ65AdboUsWzEekiLFEPETPWQQFJNqmxUM1MBrqJooh+VWemIRcnEXF+F8vSYh DNKrDfJsyXoryaJefcwWN9IryZIBTrABzku9tP+QA7SgCd7ayA7nyM5rB5KcvfBKoV9QruOr oSeKDCm6kSRADzUIY0PIg86ADPHkrkC4kRWnT4/jYUK4rzAGhY0QDU8KpbhsBg0ApVJ6BCyw DYkgAoJwBPjLy+lgBBZrsacbsSNLw5vId9B8lp54C5HIwjFriZ/ozNesp29KwjH7pjJMs3g6 w+BcziHMKm+6s+O8wlvpB7PwAQfZDFRgTWGjXzpQBqbgBBkwEJenQf0cqY1JEv9Sei4wBtMQ Q+tXB9JABBYgEEag0MVADbsHKevgB9KwB0OwBK2CD9wQoEUwBHtQDGsgXFFgBtJwAgrgARAi KSFAQ8XABqflBfUQB3X/YA0Qgld7QAQLgHsGgQ09MAdx0NLSdwTS0FpUGwJ8sAelMAabBQdD wAbnsLullwFIwJ2AoiHVaBKAQBT8kV16YTJ4ESKEkSynIiyZYSuDbCvdVoAXoAc1MKEgmq2a UAOS7KKaUL7wOq+NrAkXqpGZnKG0YMnaSr13DZEOt5Hj63BeSJEmWk/fy74k0K8rCpMiVk9f 0IQwlgPHwAJ8UIhH8NlQWgmVcAShXQmEmMuFyHbJLAVTyZV715VcGsJVUJYi+4mz/c18SrM4 y4kubBUyC7MgbJeFgcK1XcLcLLO7HZa7/du6LacfjA1y4A4ucJDZ0AM9UAbxxnMz4ARc8KiS /7qYTRupitkvC5F5qAC63QEI+DAH1sAFdeAIDzEG0lAPnGe2X/xBHjANLjAHxmcB3ABFt2AC r0AExeACeNAdRqAAc1AHS6AibIELwVAMK50ERSAN0uABelsFaIC5B8EQiHAAO9AdVUB6uGAE C3DFY4AGezAGVxB768CdPzQGgSULbDAGfPAQ/OEQCLGqsRYpXrEo4bEWwCIhHaJeetG7xBIW p5osY7BJDAAKDHADnqwJGTrXlOwM4drIiUzXKKAJb6AJ9EDljUwLFuoMKJDX9OCiGeoMbI6i d33YQ4jlK4cCGPqRzoCEORqj7wtiJedPG2dzM9cNObAKyMAEDZAOR/8gCKLdAEeQDpXg6Ltc iGZQiPmrv2bQiFQJiXQniRvLd7B9zV3JsbiNp195lojQKl2rp+sgdx+B6t28p92ssiUc68Z9 lzALzjJszjbLzqvOsycbCbMgBmWTDbBABV+DNLxQBDYRaRfUtAYjaWvlHOU97XK8GVOMfGe7 DvaHHb1R7daeBNSAalFgRAhjgfg47p6WFVM8bNlxqrjnnXAgCMGC7X1gATzNGYhgBqfgCLIV BxLOB3wAIZbRBwIbKSnhjAJRBUYwFohgBCwQB+kgWwJhBnHA2RCx1U5hBHEwDX5wDr6bAS4w BFxgAdDiu8WCx6zyi6piBvAAAIw6ArzgBE7/cAZPfgPX68iU7K0deuaOvL2d4K0/36HbS8nI QAsTugol6nCe/KKS49g3+gUTuaIsFwB6cAYXYPVncAZOcAEzL/NOsElOAABfjwAX4KRH0ACM /tlpL7BnXwn5W7BTGQz23rCQaLFVSqVVaaVjOokc2+l2l4mfvul32bNsueo8kJZ2h6dzasIy e83LbZc4u+szi87qPJfanMJryYlKMAuHkA1lwAo/BzVO4ALEMSeQ1ibPbsT+/BwdPhAxIg7R AIzUUJtRRASokASiYN+IoADRUOLzlxW72QLUEEO4EJ3VxwUeUBaIMATt1wcVcOOIwAbcYF4Z YAiBhbY8dESDFdLs/1XgcnVXXgD9XuCNVCvwYlAOOA64wOYIZxsH1xcjk9D7VSANewRdOxBr JzAN0ykGp9AHuODvAGEB1REvudbFqWMhmAshuKIAkrXOiBcWRqr12NFinZxiiHCtw9UiSosW sjSSHBllHciVGteZgQfgFx1TvODRQTCCFy8nTi4wYJCAlh1aNYwis1ODliY79Io6VbpUE62l SYkSraHIztatq7gq+oHixw87iuiRIEF2KxhVz5wgAAAAQU8ATuqe0cNAzxkGPS/87CWCRbox RwwfSXek0hEzjfFVMoIvmGN8UswYMaPLDL7GZmAYwawLny7QZoIZkVJZMg8pUqqwbs0jWP8w Kdh4yO5ze/br27db9OaB7Tci4L9vE+9tnEeLW79vNb99K/py6tBJTvddHDrw4tlDzPrQI4wO ENmc6IiCDhA69u3VA1oPXz379fPr08efH/7DKA/h/wfEP/hwweUIP7ZYJwM++mNEo/4C7K+/ dUZah5od+NvPDCI6kIWNaRyCjxH++qtikh1w+U+lKtJJgsAHo0ARxQhXciEah444hxFGAgyp hQBT/JHH/mTpT0aQZAREI2rY4G/HGSeEMUgJXVppHZdSKqklXExSyUpcblGJwFtC+lIiFQCg Ik1WJPilFAlMGcGGa3RCAIGdzsAriFXcKEqRoqga6k+uaPkiK6L/pgok0aJqQAqZrLSyqoYv 6GkqqS/GUsSZVfKI568zLnACrw3y+jSuuOq8hhc6VCCChcMEefUIUSqpRBAz0mkAsXQSM0yQ StKppAFIFPuV1s7MqMSzzDDjzAgLIpNsNmmnnU22YGTDlrveruVBOt207RY66cb1dtxuSRJX ueWuk86554zz1jnlnuuWByXmmUeBbDRAAASH3ItPvoDlw6+++ACer739AAFJpTFcqCccT3Lh kUhRuOlApVQ+waWPefb4xB1pUFmJjwUUqIPJdWp0qAo4LMBFlApyaeEIjKvw4pOVqgDhkygQ ucLnKETxogpczEDCHgJdjOhICKPowAVr/0aaZ4kjYezAC0ckNHrlBbascJoW16linT5yXgcR L8bAJQk0psYljmKS+NkIjxBRqY89XABJRETWiAYRRD6wxsqVYDxcpP5aILmllKIAc6UkWqji GQSyoaKMMqhoJptRYBmFlRkkkAAeY2zQaSc7AcDzDEy+oMUZomjpJPaivjjDVLnkousCMJzR qgarkMmhBtnt+KIsRb7IY9RRGbjA1DoRgHNVB5AYZZxRti/Fh1JMocPNeVh45IgGYDVMifRd NawwJVhAnwX2x2ChsF5jXWyxXBsDNtnGkgVNaS6DD8mYwQIEnA1oolUF2uCjClVojRS+FQze 9AYV3qrgu3pzi/8Hkms7zSEXc6aDCHVhQxe6kNUYKqEEF2TDFE54Bi9MMYoeZKMHl1sHgNyj sP/s0GD22aEPR+Q4LCUpCkRq2OMc4pCHoAhCBCJSSp4GIUDIKEKLw1oS6TaSpa2jAxOqEooO BxGSBCgiSoSRST5SpSaaBEpEahoblwilI4bkiE0DBCPceMR1nHFEVYwjc77kIi5iaZDMESSF vgQjkSSBbKVAABUm0YMykGENzdBGMkjRuWZMIXPZgMWbqGAKeKCOF3XiyQX4soFuIAMFzlgK PfzEKKtcRRFI4Yrs0rABXublAnMBgKpUAAcqsKI8zSjDCsoDjDI0o4ZUWEEzqIA5KsD/ohTZ +MUpqKCCUdKhDio8AvsOo77ErHAM6luM+URxGHair1fqO0JhxGmYxazTMGY4jCjwqRh89tMM tjKgEdJxmciURjIHpOC1XnOtW1wwOuu4BXLWsRza6OJXA62EH+ZRATqkCgCpsoEN4OEDH5hC AqMoJhV60AMQrEAbayCDDpIxgZdqYxKTAEEzyOCCVKyBFyqoQsIURh+C5UeoQwWYwGDUpSNR YwmywIXadIYNODhiaWyQxhfXwDFGVOEeHMPZiQARAmm0yAgzi1AecYEGGw0NrSoZiBe+hosi aAGqsojDK+LAAj94wh7rmIY4XOACBKFhCR9hSFThsAc4tBUX/y7wQhHmIY4KZOyIAeKBKKhx imKsxB1zU5IWcOEyRwSoD/ewx9/uMYchDCGroz2CC4pRii38rKouEYMHilGM2i6tSGMC02/F BKPm9OgjH0HFDcNABjJMoRkuWIM2ppAMbeh0EmToQSZTwblmJIOS0oSFA1yYKl7YIFSqZOVW aEGWbgwDE7wMQjzigUonrCobrOjBNEmxgp6WoRDaUMMaYFEGbbwUkwWewktBMAVYrKEHdahm M+AAgvxukwozMEUpSkEK+B1BfdVwlSDK9754elgJ56yGEqpRmFwpQX2isF/7ZBXOwtgvBLKy 5xjWiT5wpk8UDSjnsY6VzkfQ6ghFSP8FHU7JC7nYwE42GMVMTprfURS4GWvQwCTWMAVS3JQ8 ZCBFCnA6BTKAQMsp8MYaJpCKFKQizKmYghXc7I2YkoLLZbbCTsnQDF6UwgLpKdh7fngw/RDM 0AXbD4j2AyUhwYiJIIKRKAwBP4itQ0RIqiIh03rF/RTJ0Z6+4oSWuNQXPY0/UP3Ri/wThRY5 RERPjIKOAgSSAI1BbCrhzzqIcCFVR+lBLiIQI1wEIZJQqSVV6uOEHOSlkRR7JT0ySURagAgF 8GIGpACBnUlRZjunYMGWLMOCpzAFAk8h283AZDPUgMxmwIIUZWiTBrRRCmPQJMO/WCkVYJrT K2ujDGyGKZf/QZBTUlghBWQOcwrKoANsk6EM2E5wdS+5hiuE+6U9gAUme0DDYpoCDhIYQRY+ XJhqpPjELj5CNTzsYZXXD+UNcJWKV64+FbuYBQ1oAIlX+GIYmMEPRSAFPK5RJ7lwwgbltUH4 TBGGUczgCg0e+BqwzWbmNmMSVsipNlIBghRoI8xrSIHUSaGNNpOC4ejG+razTYptS50MPZX6 IdqsBjc/l+1rWMMh2J4Khtt5EjpIxQTe7lMVnKOHB7sPEBVv1PcsPkRV1BEu5rGALbABtYxQ WzHOkYsTeIBAIfCCPTqwgA+tQwxC6AAbvLCFHVURHwVhhCikcdjHNkQOSHAE2epg/6N1bHW0 QYtqMTgmCk+cqANv01HkZZ+L0Y6sA3vYAi6ydiKXcSwKITBEEngkfQUsACQyYz41vpYgD2h/ R7g4wTRAcg6OrQMEO0iCLNDQENl/sQ+lOEcU5IAxxH5CSgACECfZEih6tRghEIZBAwRwAC0j MzL4O7FzQDDTHDXYJL6js2bAtkJwrmYoBBBIBjJIgS8jA6/DwDTbpOtiO1KAKVJAAxBcAzVw ATIIMDXYKVJIhetCg1TApJ36tpc6QWxruxDUBuxag2xAtyP0Ag1Ik1EwKRBQEJeLJzlQAjkw jBBwMSVogEeQg0oYA2qYh1JIlemZnpAqLzqwN48LHVi4Av9g0AGdwrYUADMQzLPrIgMr2Dtv uLNJ8AYbVIObArNtu67rWjMH3DZtcAG9syQXsEEwk7o1mIS7uzK+Yy6p6ylSoMFUgDM1IDtt gyk7dAEUFDubuqQu07KbWBD/QBikMhjE46Egag/DaaIJ4QPxs4dzkAOSYQOaiYIqYIMOaBFb DIcdQATD4QM5+ARU+IgxIZCV6IMqeaAiQYRzAKNh/IiyWaMqOIc+qIIh8QjEuStR25IH6oCP UAlqTJsjYTYi2Y8WcJFna5qQMEc7YphkwyNcC4k+ipDjsiKRMCP+KAkzgjRiG8DDSTaHiIil EQMnKIXoagZgWAMFwLhmUACKu4L/fiOFigQGnbokEMgkaaqDZlgBKriCIqQCDVgDYEiT7FoG OBiFbMgGYiqFGTgFYjKFGYiBX8DJGVgGNRmHGZgBnPyFGSiFXxiHNhkHYxiHUqCDcaADeBAG migvYXgGJeOFa3gGeKjKaxCGnbgJJ1AdVJILotud6QlLuDglshTDtCQ6tURLuKgTJ3vLtHyL J5OTOnGCkBI6O7mGuawTrHQyTshKTrgGTnjLaxg6oRO6a4AHxEzMm1jM8FnMUUACeFBK7CkF JDAFJNDMUYADOGC67QmD0ByFK1CA0kSDPtAhxUs8onJFH9KPXltGoxERVMiY0aqCPkiHPqCb RDOCedAC/z6AEiMwRiNwohgxgzkYxyjIgLJChxcRBQs4gmmIhgc6AOlcohBwhMlJhQWAkSow gWKYBnvoD0Tggt2yBmQEkAqpgzkgglwYmiVoEUQQhzHoD1tLgiogAgWQhjmorQywBiYygjoY x2f7j4i4grYChA7oGQJxB6sxG26oAoahhmLIhD2oA9EKkscTkE6ThQC5tIXptFSTkg09QCYC wAEJEicCEBUtzlf7kR1x0UxrkodoPScBkF4TkP6gG1kgoShB0SYS0RzdPiriNE6DtF8rkiqa ov0QERvdtPRIjxDdPiR5CPXoD/YAtAx9ECp6j1W7jyhtziBRR/zgD0HzUUILmP8qHZihuo8f SkWhIrTW/NJScwgzqIBzqAKZ8ZkqUAAPwNMMKIayaZh1+Cw8XRlruE3f7MaRmIg4eBbf/KJp ZJl1gIGZWYlZsJoWEAP1QwWpQYQqQINpcITl7INbyIA5CAFBcIE9GMYiEZkxOIBiGALgXIcG CAOf4YFJQM6VyAAvsIZ5mAZpGAMrcZl66MZ1IBy6uTREcIFPMMfPGkcj0E8XiIMhyAUeBT0F GKw9SJw3ahj4iDZ6PByoGpIjUhyIgBITvaI6WqKSuCNakwV1VNJyTat5fbYhgSo7whBHM8g0 IpI1cgiSgCN0iCI1olc+GhImOiIKOVg0aqIRURwsWlf/KZqRfpWQCGHHOWIicoWIKHoROloY c02PeDVXfrwjKG3OhW1Ogj1ZKJUFlo3SIVnZl03ZmEVZQDsipLrS1oTTNh00NGW8hbm0TwMT kXAQcmyJKlLGj3CqeyUQ5liJeRCCZ2MjU2NHK3kSUpsQokFGCYGoexy1fOTXCTk/e/wIUTCI hkGJCZkIC9C+ZVyHsWHHIsGFJjUDL8i/LkpXO9IIJhKcYPsbjnm0KvGRtrmF+GMishFULFoH VGiBi0EFC9gCjwAjVDgHPmgcqKmCDrAAR4AZCTkHRxCExuED8PwrlRhVQbC0PKoilRCFOIgG 92SYKhgDayiHPwsQPhiDdCgC/2tIrTggAhMwmijgAy4YAzQgAsFViWgwzz/bxfKcva6B3XU4 ASS4h2IwgipwAfEk3mLQz2nYRuUETyIYgmJABVXjkZWZBiJAgyEohwM4ATTQGhiJg2kQiUwg AtkSBfmchmJAgz/jgjoIGWkQhPr0AiI4Afp8iHUohiP4iD2A0Cg4hx4wgnX4pnUIAQUQB1Ew GzgoBs/YAyI4V1NDBSI4AEBrzirIBBfog5FwAS5YCfzNIaKBgRbogyEo4BZYYURAKp7l2dSE U0TLj2UMttESAy34osHRgpeAg3NArlxABGjtgA6ghmkgmb6lWlQQA2sQnCoghWjo4BNRCRdY AI8wgv84QAU5kIayqQTUWocjuFMY0BpEoJs5QoUPsJqPUIJdBSxpHIOySWNuQAVE6BlESAdn JRvAkYhS+CtROAVp5JlPmLaeKRkvsAAm5oM6dqQ+gIM4WAmiQRA/8IBIlRoaiQMCobb6vStA zZiJyIUjkAbm44NTEE8+jYa7Qpph7dRyiNsqJhmiaUYjSBovOtUOgAHcQ4UxKIU4GMdNPQdE uOExQIWLKZulQoRiqAcjOIc1YIPRGgJr4ANRWFDlJILJqYKUAQk+WGMMfoY5SIdepVXQi4aH QYOV2L8MYAEiUL8kqGKICoEKIIJn/qaPWE5rmAYduBBcAFTH7VQFjliSWIf/IhCEjwjQ/FuH MagHK8mAjF4HFshojUAEapBkPoIBNPAAaaCG5BQFjl7XaYsDOVgJM3AHmtEIOeCDksAHOZAD HkCEaqgCkcCGI0CJkqCcEMiEDBAFo83Z38AHLmkBfFgn4TAJdOABGBAFXVAck/jUHl0cIdrZ VnRTozq0xluPqJIcghTUxonbw/W1JFjaNTIcRyITkIgcMZmcyH1YZGvchnEIiLqaZ+ODSy4S lmAjOZJrRnI0ZFzGOnaJbYi+vY2RiMW1ewVpuE3cuO2i4xIJSyPivnaBrTlaxKG1ZVQJNBjW OubrI/EIURAtuq1bliAQ7XWEWzgSl/iIT02Fe2CS/6ddKnwAAffk6yPoLEeCEjqiUQ8F2DCC 6yrJAPVztEGlo2GjW2WDa3Kckc3G2LU9HFzjblJbRpHoETAiG5ZQCdteCUSg7ZUogmh4Ebcp YET4IlxoHKJJBysJEERQ4woOCWwMBli8BT4QqDoQNSNgAyMoCQvggQgRiCQ4h6LZ0BNN0yv1 0iuyDx06tKLqITb9WYIR7ZVAhWl+LV6U32noAFTgBqVp6CTAB/PlgzqgzRUpAgW4hy1IgiPQ ApJBhUn4BEc6gjlQbBgJhgzI31PYAlTIAC1wJFEYZVdWCTHg5VYzAm5IW5LpgAwgYxdYgk+t gxPpv6U5giXoVPhDhFTIvf8xCFREMAMX74M1QJAqiIPCWaIqWAOlWcYO+IBpOAdAZQkjsAb3 rfJi0MYHErWVMO11kDRkxJlckPPsbFxUqIIi8L6z+jOcOYeGPtuROYc79SIjQIUoR0ZEYKs1 6tv4rAMu2BU2UAAbyQDaywW8XZnsxAU28PEowQVq4OXRwtskEAP2vmgTSAJRmAOPqAIKeFtW exiSae60kfMTUbY3jgMxAE5UyITCaQEjAIYt6ID1/ggx2IbREgcnzlj55gKY0V6dWYADYFoT SJvPnpE9AN4J6YA6SAcC+YRpcIEK3uYKHoOiwYVM6PYuOgEW6IAxkAZtnIR0iIKzIc7yjgLV 6+//PpiDIzAbeZeFEKiD7KUGOLhNL8BcarAEQV1hh0aFONjGKEoRdCg2iS3TK9qSTVPHl31w ABQ0gTk8CM80kICqPrJsupGcIg6Tg6gHJ+ZRqgUJaoTbuD2cpdHb86ZaAjnsydFrpt/rc1xG wen5pG+aJIBmVKCbYeR5Ol6jAY3NzU1ud9T5LRnHtukbgAWb176aun3tVjtIxK3jj0DtQX82 fTyuKur54XX6oy/RhhE2pfeIj8jX+PPFzD5AHoWitXc1Uht8R7t5ZZTrLqrYAQmjpSITGHk1 YeP8kWjdMRFtTxv0pZpbhlmqfHWRS1NHclxGGR18vo6QfO2Qe0CCU0CC/1KAg39HnLa5ZjgY VnK92rYX7aXJV1DN/WeoABboo3SAg8t8BjgwgzBBV1wIgWe4zFEIA2pYiTi4zCF4BiSgT5uH 8Jmv+TWVecgb/KVxGSSoh1JYADkoBQ+gkVPYgXXIBCRoCLgffCqGgwUACFFIKlhIsm4Mkh3u kBRbtw6NpyUd0iG5lwsXoySi7nnogAtXBy6lPFUUtK4KnFKfqqA51fEjRkYxAX1kJJMRTUA6 a9qkiRFQlCgYh+KsmYRRlZqAitK8ybMnTJk6l8rElSSmzShQdy6lGrQrTqk4l/4c2/Xs1KVa gY6NWTMozihnceXsilFuTK10y1JlhFdnz6qA///2BBzXr2G0dBEDRZt2qlmmZtNOflz5MWbI gDM/xnUODhwjDtfJwrVuDxI5DjvsOcViHRskdaqwMSFIViZ4RRD1OWLmIyBZHeK44HPiyJ5S GfogKcUmChtPIBA1xnviHosoVUadElVlyLMj6xYgcUGXMyB06HSqT79e/Xr26efHd79+r82j H6/q1z80CYBE7YeLUALChB9NBQ4VlV5DNZWXU34JWNVNWhW2V102hUVVWT/lxBdiGZ0HIlt7 gajVWl2tdZdki0mmoV59/VRWUWPZqCFkG4IVlk0ugrWjhoFtpuOPifGoWY054eWXkEMOSSSR mRXmGHpUZVbfVOhcdln/lVPJIldwX0Uhy5dhTiXXmGTq9CWaUWBZplmyoENmmljGNyaaagL1 JTp51vceoPTNl1Z79gUKKHw5wsVkU0wBB1xhNWKl4YRMZWhpg301OiOFL0ZxVFxNUbbZYmQ5 dpOoI6q4qZVqHdmjkThq5mSRUcp6qmW0WjmZjT9OGdmOmyE5KmdHEmvZlMWi1euxVHZJaJV2 PovZn/JRK+yVykYb6LVXSktfe4haCx+5hkL7nqHlAhcUu0IFRSC7QMX7VWPwvkvXu0Dhqy+7 fUaxjrxz4tmCm18ajCebCLc7Z8JBGewvnmP2KQvBQQkcRcUNu9sunWPam3DDGh9scAuyOIQx /50kt0AawSVTXHILKztk2r8Yr1Mxu+tgvHNQMRO4MtAux0zay2zCTLTODqU82s0Uj+lQyT2P RqbPTVNM5tVUQy20yUeb7LDJVCNSNNVYx1wyy//eso5pP8NMMCo7t9xzu3XjXDfe/eLs7z95 082nnC2gIzjh/8gZOMP/EM7nnIM7HnPjhNNNt+RRKN5nyYmSuznn5XYeLufsxuymxQTL6/dX LdirOuW4VLw3xv5qXLObUc9J8MEZvxx1yg6jDrbvBh/MsMR0rpxmwQmTlqbTA2M9eJotZw0z 1mSy3MLYuCPy9tdQs93yyh8lfXPrqoOPMmmINK2zzDcr7f7ZRZM2v//T3AOddde0t3zz2aPz v/z94ne8Ma0MfyiLGsa4hjbICe1sBDRb2XjWN9L1C2J+g9ic+iY5WfQNegks3eUgFziKDc5w LVAcB+XUODeV0GWIE9wLSdenGVasTx3kYLo65x5wbe4fn9ucXPylHnrVrU/VaYxO2nREfSWo TWjy11TgRaB9vcuJebtYz77Eu4iVzmIUCxOboAc4LtYPT0LEHRdrFzmBSW9w1IOaw/pXP5UZ bYHtW8ct+gc09sktgXfUo8yGdjLx/WsdSfiX67ZGMKZ5bzSC3OM6UHE29+VRfTxopPqw58i1 nWQd20PE9obGg+3dAhE8qAIqPVkFHoxylVX/wAYqXYnKYFSBllIwQh+CYQF89AEf+LCAEYJp BD4YwQzDFGY60mGGZDJzDGNIxxEEMYYjONOZLKgmC7J5zQywIANxyAAXwhkHLmTgAOMswgEO MA1qFOEEaJjGNFzwThcUgwgusGcdiiGOfNahDi7oZx2kMQRuCNQLBoXDQUFzinvcAwmnQAJJ RgJRT7iAhT/U3EUz6h53SZEanOAEEICwA7adoxQfBYIhEBEFGDzjGZwgCTxAqgVPugMeQPjE FN8lCpMiYQ8Q4UQFchGc0gRlZh4Dis5OV9SdLa8SaEDDDgikM3Z9EWE6E96YquFSNizPeo94 hhPCwAlDVOEkpXCC/0uBcA4ymWEHMJgqxViGNbaRr2SgrAIiyrpApnENbFEjGtoc0odSAIAM h0jFGiYxATWkoBmGVcMhSEEKbTTjEFO4rBpIsYYVJDYVpJgCYlOwBjJ4FgSkIANopzAJ1KYC BIcgQxmakQJvpIKyqUgGKdRAhs9OYg1T0Oxuk+ECyYKgGa/oQi9isIla2KEW3VhFN0igiG58 oRvOkK4istuNWkzXDnlQBAmqCwYwEEMHKYisFVIxhQ9MIAWS/UAqvHGIw7IXvqo9RBYO4d5J kOIDkh0tfFOw2+G6lxQ64C0prHDaVKTCCmRQQ3oLUVsQfGAK4viteg88iQOnwAUT8CwZ/P8L YslqYxLDPa1/m0GKCcwDsZIlxYlJkQL+ujcVhz1vgr0B2hRYwRtk0EGDyeAN9/72tDoe7m8P sQYnAANgOxTX53KoQ9AZsTq4cAdId1BST6wBCGyoQiku8IoORAEfpUBpFVBhSLop4Rk3LRCa +sCHdEgSBnVwAhC0kIQWoAEenlDzAUC6hRYMFghLgKs7nvGEELSUDa4TxTOuEQ2TGcGl0UgF EKbhOiN44aXhqMY1nDBpWRyBE8IYA/tw8SUYnHkJiMCymz+xjg/gmQ8mo7UnqhAFbMCBE1ow jcmy4YRiuK4F2BgFEIigPizDIx3rYKmk+bfIsiltkuMzQikQsIL/KRx4CgJuxmhT0QwVS3YC ZACBN1Q8icvCgr/NmEIyJpEKMtDbvbttbG4nwWNtkFbfvv0tCEBwWVKAoBDZgEAXVPECcHSj 4dNFQS2c243mSrwWJJAuCSLeDUVIPLocb/jF80ACV+Sh4bX4QR7y4ApXdGMT4lCwDnTgbfem AAQpKISMSdHgzwrYszqWcXw/m3Ml/3YSh/WGGoabCsjqQLGHVfAEXMDg3er8AynYbyp0EPAE zzu0prWCDsidYG1o1tu5/S0ZhjsJfrd2td72Z21BS4reulizL+etFaxwWRe4duY6J0UznAAH VExZc4IyV5Q9hzHTKHV0vrNbImsmLwIR/zVn9hLT6E43O8ffrV9VxBPl8XQ6890trieLYxkF ySatxZGA5HOaFuGKvMVTFX78c177gibt9MHsXzLjNS96INsim9e0UyiueacwbmAUogzJmAIw dLCCCUyhDCC4d/WnQNsVRNYb/H5tkZthiSZUQBXvaPgqftAIO6ziEj9IfysiXotVTPcSJGg4 +rmbgOdKtxsijzjLXZzFgZf/KQLJGaB2jZwBkhwJqNzJkQAYgEMoANm7pUIKtFYKgBZ/FcKB fVYZWOBs6VwGeoM3AFkZRFbfudhsZR1tnVt8eRbZ6dxrnVZ/0WCIWeB7eZYO8pfUpQJ/ldYk yNsHuIAYuIMfKP+BIFQCDPhSMPCAFGCDFPAADFBDCbib1K3BZIGADgjXGqiYcOkAvenciZEB L6gAD2yOufAQ6LBHRpFLzQRNJq2UF+yA6NwCCMRBFfSBAsgapznCOkhBD6xEH1yBB0TSLNSD J01COWiHF6AaIrjAmLHJs8HBOawDPhSDBUTBTo1BB1SBC0yDQaQCHSLCHiJCA8BBH7QAD4hi FGRAIfJAJvCBJcIBV/EaV8WREiDBOZRMH1DiSazBJ2CPC6DaOhyBF+DVJMhaFbiDspnBPViA SlXDDshCL+5AycgBN6hUCWwD9qgTIogCKlaBEcDBGNhMzKgU1QTQHu1RFTQDAoyCNkz/gTbM 22g1g241AxhOAOAhlo6BW8E5VmsBA8ClQDbc3DKogsJhwCXYwXPR3ypAF/spwip8F/1J1w/8 wMbZAQnYwRsgQ8P9wANunMi5AnhxHEZyVy3cwP1lVy3kAXc1YMaBVwMuIHTVwsoxYMa5QgN+ V0tOFzmkgQ4Ugrcl1tWplrytQWdJVh0IWNplHSl4g9JdViq4gGV52wfgF4/pl2fJGBmk1wd8 wCxQgztkwBhUQiWYwTkEAzY44Vo24RM+IStFIRSyEjasJSu5JSvdAg80ISv15RM2ABSoQ80N HwtqA5DRIxYG3BosHVI6gSkEA5UVCufkEBu2ob/oDIEgAh/w/8E5nANnxg2aAEUfsMEOeMS/ lJX4eGYHrOZV/Esf9AGZvQtvdMDXxMtHzOYhRYER3MMWIIJpwovJTNUUjQZMyMLkvYtRnZ7N GMxVFdLJfAkAeUxwkhEjOYyq3R7J/A9gEY3pBWfXxJUCBefNIMIaIIAGaGEz9ABlgYA2tOe4 NUNrjRu/sWczAAMwdEEXbEI3YMDE2UENKIIdZJczsN8q2MGABmiAAmiCOpx2ZZdEFmB44Z8d MOT9dcPKjVzEXdwPgJfFKaBMYtd2beTIpVxJqlyDkoBKkqQifJfIgaRNomQ33MALcIDMISVo kYE4nNYhNJ0LfIAYZII7yMEYsAEbiP+CEQRDMEhBFeADLa1SE75SEzahXUpBFPrlXfIlXPal lm7pXnJpX66lKXnplgKmGCiWZs2dPupcmrrYGlCYFYDANdCBGWTJGkYmoXBLfEwZ3sSMGXhT JrpOJWQiod2GLFTBGBgqC4gCRFiDLBpBERDBNIhGUBjBCbiANVSB6qxDBohDMUQDH6FJUhVV MVgDZsqCO0hDEXiAF0xD3LTAEQTUEHwCmFANF0jDHngAqcoCp+WCq3JDB8AAN/SB61QBCFjj q9aDAjTEOvRiERTDHnSA0hDrCYTDHrgAtIpCBVRBB5hBGGSCHCSHH7aAGEwDG5zAbJjMJ30A qfZiHJxDBnj/gQVcjct4EvYcjSA9IgL0gG8BntUVlzZkwQuogjLIwEMGKC0AqDPUAC3QA4Ie 7IR+AS3QQg0EaA3UgDMUKIBOLP1NqB0gw/spwoZinMT9QMZ5HEjynx2gnMqVbE9ynE1O18Vd 3EdS3MYZ4ENGV4hunMc1giTIgwhYwzYowTWNAT5AITDhwywhaRQm7S04KZWekltCYZTCEiuh Eg/Y5SqdktaOUl96El3ygCperZhqKTaErdhyqdluadi2gF7qJSu1wCPQQIXRIykkA1KqwRqo QYnN3X+pAR3I6aAU3mSioRpOZor8iwuUAypRw0ytQyYQgTi6QAWgAi4YwSvkwlcY/4E0SNJG HAEfoEEFvEYVSIMJpIMfGAKZiYI0jIESSIMfroOl5tRUwe4SnIemzoH4uIAjjEkczEGzLkES WNkJ+K44PGsU9MEQYK4odMTToIEHoIIW8YEC9NMCqA8iFAMfUFXLyIE0SIMChINDnMM0dAD2 xIECeEA4ZAIdUmM9eABUqQ94hgBXLesCnEA9jBT8mKPtHY9DIEIRIAAoXMIq5IAbuEHF0oId dEIO5IAdaILELmzEKmwOQDAtOMPCTiw9LKx/TqzF2gEKOKxGKsIXgKxE0oOCbugqYNwPTKgi 0AMJbOhFvrBFkqxzZVyFdgMlLME2cME2HIEopIM94IMZSP8BkwaDEfiSFJiBBeiCLhBxMDCp FDSx0iYplbplFbsSlWKDWj4hLV2tFCSp2M6lldrl2H7t2aptlZ6xlqYt2PYl24ZtHrltG7+t G7ctD7Qt297CLSjBLMxgBpJCGdBgM1zf8ZFBHZDCM9DBESCK4XmOIzMyuigeZhKN5AnF3eDM 3UwVTbANLqCEPSBnC6SDI5jIIUnRvYgCWXlS4pKPmNiNFRVSBjRuy4CmJaMJ2hhBBWyBafhm UbnPJUWBKW0PXRBM0+KOvE4V9ojBpfIGHAhCXmHq0NxCMZXV9iiNH2lRXHVN/3hPcLqDEwyD Ah9wxCasJtSAJmywwtqBxKZzxXL/rCKs8whHLC1ALMf6J4LawRcw7EUiqCJY8ISacD2LXHaN cDdgpAmz5EpGnCQcgyCkAz4U00P70hE7sS4IMRMQAiFIQiJAwpHSEj58cS1VwdLyANJKQR+s EhTig9jypZZGIS0h6RmvZZWytJi6LRmbsV6u7dvGjBvzdBurYh5rqRyz8R0DdRvbcVHnUQtU AxR8wBqQXTPwVxe6FsFRnSdcwxioIZ0KrlZjFHwo0aR6AeY+xAIYRBUAiB4KgabugSGtgxKg AirUgax18hw60el+hDsQGy6gwhqMMpylw+RahQsALyOmAy5kQDEcUhWIA+WyCwx4AfSuAzWg 7m5uASPK/yow18EYFENsIkI27C4uYKNHyEFDIK6s6Wa2HrYWqE8lSANtplLOrIM7TMMnkaPJ iMIxIkIcKNs6+MEeaJJ4zk/vuQ8jEffQyIETbMACO/AD0wIydILCakIOdEInlHMNVLczIEMN uME/t/PEIjA6c3CBUqyCbjCAljfIbqiBvnD6vV9BkyzGrkIt1MDGyTeAdsMusIAgGJMZ9Hcx wUB/Z0AOpB+ARkBD3oAIsEB/fzQRi2MwfbGS9oFLk7EWrzSXfjEZV6lMxyVM9yVS0/Eae/gc gy0c73Qega0e3/Edu+0bG3Ucn7gqxvGKs/iKs1IkzMIHgEAPgMBTh93a7fjcwf8DHWRHoaSL lCWeRs0LJ7syv+QMMP8GqPZyTt0Cnd0NLigTgSCVUPjERwiFGdjGRzSPKRdSyYRe3oxBEUyR 8hBMFegAH2Q5qEL57LLygRCMqjm5zQBnbdbe8vzPHaHr0ICPLFRz0KxjC8jBBegBMpyzAyus MzS6PJ8zBGuCdUfwF/hnPl+wOQ/oxPonCiQsPteAdAXoFzgDCnfXx7IwPazwhKbwC2MkCc+w RE4XRuYABRyBb5iBWep6f/d3Jfhwr5sBDFRCMPU6EyJpSO9l1tpxmgUDKrz01vKlXgaDHcvx llr7KbVtiqt4Xvp0iBP1iLvxGp84Hsdx/yg1ugf1i+v/JRS4gwtkQyrAATBkgzaAgBdkA1KW wjVwQRAFbrkUrkblEBEFBQ/AQVT9CwhUQDEsvBagAh/cA+XiguoKVU4ZwTQMQTGg2krdwydT Q+0SCDWQqhGcQj0sQBzEAR3eCy7r8siD9iMeWhWcwicfJyKAgKziAgwENvciAip4wSgDwjr0 Nfb0AGjjtkdUNpjgTXXMC0ehyJZfXru0z9OQDxy5D7qahrRBEvaomfrIASdcADmswnXnQDqj gDo/OgowtyZkMKVrwqNb8Nu7MALD/cFqMAc7w4HaM94/l4HifYBuqDNkFwu38Hk/YHa9MIA6 1+FPVze0Ag4IQgP8+hGYZSWk/8NZ+jrlB7sxGUFFNziHU20tNS3UdnEt2ZIY17ErNSEq3GWK 63FOkzvQnDiJ67Ee/bROh/hP17jsbzuKy7iMv60uNSHt534IQAE19AAV6CspwMEU9MAknMI1 UEOWGLl8WKadhgtX4EsrH6eY4AuBIMbn2cteVFG9KAnqsE0rY0jOIAK8NJHrrIvofb9pBNbM DNLo9UzFuE4VVYxwThVT5QxAtJC1LgouXFEQRhm4bp2sFi0aroOICBUiRAwZPmR4C6NEVBA7 HuEFwMaICxcYMADTyU2NGrRosaRVw44mWvSc2anpLCYtO7Rs0nIWlJ5QmDBz2kHm0xnNHzp/ 5FRkx/8ZCTskfmT9sQqrzlquEuTBhCnBnQRjA2zApEdPyjNAjqUTdaRBAzNHBNU9cqQS30qV zOgyEwyfFCn4ghk+zAMbDynYpFThwaNKsCoteNyaPLnFLcmZb4UONllzRR7rLnPWzKMFqnUc 1/FAhBkz68nYanPenNs26dWZN4fuXBvzLdrGORcH/lp36M3VZnHR1gNWNmArJpniNS8KIO+A 0KED/z08+PLexY8Pv968ePbd4R9cl2pJEkC4MmjBtU7UK1S4+usginXqiCYJUbipArUeHFmn iPrWQSUa1w7QgqAqhmjwCE9yWQcRPjw5J4kMDEFkvzXGMKigFjKYZp0QykH/Y4t1JvkkiT7g 4GOdPrJxxCA5uOkAlxbQiCahg3C0pyASBZwvRVE8iYMFP+Aw8qAoWnDBmiTWMaOCXKIQBQki HEGkoCpGmWMaInooJgmEIJrEGonWgcIQBTMoRkFZsGRoIIg06ui1QVEx4xoAeKGDDl6u4WUE Xpxg4Aw97qhBkZeOeumnmewoCqai6NEkVHo6GSopZ4a6tFM7psJJJ0WQYfUpWqxSJCo7utED gV2dcAIBAHZFwAkAAHDiJGN9/bVXEY4wo6+5GtiLr2bNMKJaM/DJNtvDECNMCl0S4/aww4w4 TJdsLQgGm9EcqyIydoMZrbHNGGMMm8bupVdfRCab/01f3VgrrjbNVsuN4OE0K84455QLjjjW fgOOByVogKIOYIBpppkKeEmlu/LWYy/kkNUDeWTz2kuoBYQMalnFg1yWKCEsaT7ooZkZKmjA FmAu6E1c3kzoviFz3g8hQASCE2ZAEOoIvoQIwoXplV8+uumjpSaI5XUOYnqgAbHOOurupD5o IO8GksUgppmWyKGIGLoobaf5JEjQ/dZJAjaOjIAHAQdM6aEUU0YwxodGeUmc2Av0OKPSGpzJ QROXfILppaJagmlTyDVBYaYaSEC1BmSe0mmVH5BpA4U2fMKp1qe2AuOJXo1164xhUZIUFLYk 3bXYUMaQNp0j5uqrWcAAu//WjOXN4MNZ5p39CwYzpg/MCBj+wnZbw7IN97DELIPsMMiqmNex 4EgLLuLNLFJ/tYQlfgi0WxBBzrjhdkvYufQDTq6zgPc3Bj+goRk9WEMYSuGCdZRsZCBjYANR lh6Tfcc7MOMBGupwgiFsKT4ZGIIn9mAPVCjgEy2oQirqEQcFLMAgGZDGAuLgBR8hog5eWMAY XGAIrh2hAq6phjSSgIt0eCEabHCBJz4RhSp4QRpsEAUaPJAEI8ThHp/ow5Ck5p3uaBEkLljA OozggmlUgUtR84MHcoEIakBoEhUoU5eGUIUOuGMOQnLBGFAjQ9TIKYhG4IZrHtFDRIQBj+sQ QxL/4bMOd5QQES6Ig9MIAhKw8UxvA3LZLZLAJUxaQAUAyIYGkkEFVvxiFBoYxS98MIoR0OFR j+KFsM4wqQ3Q4yc0QZUdKMeTGnyhBkGQlLGE1Ss9dCMHOqHHLqfiDFrSQxGoYuYXWGUVrChi LBtgwAYucDtiAQsBiRIGK0dRilMoCgniGMPwjjA8QShBEHMRxBEgQRdpSYud6WhA8NKZjr7s c3hzYZ4+01GtgGLrXMEwQkHjRZjRJDReVahMvHiw0MkwlF0TrcL7CMaaYGR0YQe7KGk6IxyC dXR+ClOfb0KzPz/M4gCluMIonACMBboHPeppT3sk6B4J4vSmTFOIifqg/wAPnCAOqdCCa7QG gzHuBw2OiAIMilGMO2ppHeeAoiGmcQ6D+OETUkNFEVDBiGBQ4z/4yATXEJEnF8gBDmzYTwb4 MJA+ZKADLUCEKOTgVIW47WZTYwgfzmGiKoiCD1UYyH5wUQU2hMAIJkpsB+rmIVGkIxdSGxAf +uChKuCCEQMRECNw0Yc+KGg/iKjCaBtyEEbszKEfwWQLksCzK62MZjy7BS5usbOM9OlMM0DA DLShDRAUIhs9aAYVQEAF6syAClQYxy9GQLjEKQ4BKLnABhShCJvkRBO47MYLmvAMXtjACYxS QTwaQROa+EQnNcjBUxRxg2FsQA8bOMMFfnUNBP+s8hd0mMEvsuFcWAC4DDNoBiuokI1xNOMP 45iBA2YwDjpkYQx3OYISjsCCueylASGY54eVUI29vPMIIu6LEjBMz0fsRRR9aedehme8a/UF eskjzPSMkC0j7Lgw2mpoRNtVGXpFDH/ok83+UgqxW8SLeYTNQBxccIpnBMsGixrBCMJpClFq IAxUWAMIQKAAbazhGnAgCE/RwQj0TJCmFMypTr3DCHRwza8IEcMCOiDHhrBMRQWhc9tgdhAM 2QMXqHjTfSrxJRW1bQ1CQAUq2OCfg3TAAhZYYNsYUgXX8MAI6bC0T3MWBUQcohxSOwcqlAgH H+HCCDpKbBzikI7Z0tb/DNuwBgvcxocxjOEE5ahC01iwg3IQ4RNckywRijEhiPABDXMoho8A gYhp8OEARDA0lrCY2yvNjGZYki3PdHZbhOS2Cj1AwCiacYgpaCwZk9BYM5KhjXUbF7kJbu4o qGAM6E5XBaqQ1DWpAjmfdEITPKFHUeyQlbPcob4MuEB5E0eHUsCiB15uxgq8TAoQTKEMEwDB cadAiox9vAzZaMYagJENBDdjBjEY5S9KoYIZiAPD8bywEliwlxQrAZ17GcNchofhDed8nkoI nj7f+fMPf9jDI3Zns9JJrUroE3qB0Z4RGDpRiAZDF3wQxbkqIQcXzAEOjdqVDeBR5RH4wBSl /6CCBhLcDFg0Axgfn0IKmqGDFEwhGWQAgQ6mQAa/p0AbpCBDMww/iRSQgRSkOIQO+n4NFaAa zuRB2XjWjFM4i6zNQkOIOzwgoHLvILFeIP2AHvHHDqBhCftJxyka1AcZImQMrxCQGZAwo9mu Y41DSnQusKHqPM7JkNMwmhEqoCMYeGELPlXZGiXyARYiAgTRQE2j+VOBc1TBGvqBGhq0YAE+ 1EEaFsCFHJ6xgGB4EBXRhkMFBAGDOnwxCnzgRjTS4YJo4IIP94gGr22IIdagQQAlajRiSLAN b1YEAYekM7hmZR4Ct64PAYAh8MggBcSBDCaADLxBBw6B8aZAGybBG/+mYApgYeNIQWOoABbG jBUoTgN8wBimrHZ4h3d6obwapRRKYQaKqww+Tgd6IAVAwAc/TmNSIBXWYA2EKwXHbApSQQRT EATIQBsKqBlAYAVAgN7ezpRmQOb6Cw0y4MJ07sIawOjEEMNwji5QLOeiRQyXTgxFjMN8bi+M Z+l+Lh0wzMMqAXtKLBUqwG9GghNeKQevwcFMAQ40gBWAgQqaoQxOsASDKwVIIeTWoBlSYQom Qd4Sz/DWYBIcbwpTgQw+cA0ar/Ga4QNIQRJB4ANAYO9IIRVSwQoY7wjzzhtSYRLpQAXMYM3Q IQo4T6fSI4JqymSC8TxWi2ksi7OexvPWpmz/hmZo1gYQQIuCFg0+mGa1ouAIdEhn/KwYxqAg nA8Zj0AahMQa7+M7zJGz1MwcKWiL7kNnaGgHdOYcLeto7kPN6nGL1AwZlREbryTOsNFq2rEg rmRt9nFpnoZtumMg0OC3lHANrCAF1EANGg8EvOECRW7xnFDvSHDjyiAFsJDjKnIKVoAULXEK QGANMFFjpHASLnEN1GANUuEQKtIlY7ETDwEYYJEUvmwSQOAJtSEV9K4ZtAEDy0AJU7IMUgHM 1qAMmmEKEgwWWCHAsqHt4GENWCAddi0OpaUaRGHDPozEooUFKqEMSYwFeK4v4qAO7sFvuKmb XmkE4GGVTIEVTsEB/zSgB0ouGbxh45phEiZg8aCw8QqPDFLBE8nABUhhEgSPIm9xJ0HgEKwg CwRP8VIgGVLxAkGAJ1MxFJEwCRsvCVOSJwuv8ZDQExtvEg4BCafgA2BxDUQRBCZhEhTFCNjm zXqqzXSzgYAxPLIozoakCIrBHvoABJxqHbygq1qgErihQ9agqxDCNSrhHvrAIizCsu4DH6Th HIigA8YgDvxgTmQhBLxAQdZhHqxhP1JBCNbhCF7BRGTBCGZEFCrg16LgAFrPbhzE+9rTAowA DnZAFjoADvCoCrIhRVqgAYJEZ1AjFRoktAD0FiYhDlYGH7aBa/gkPi3AIWDgHuyhCnJEIv/k YBtk4RHmwEwQoQj4IAqAZLNkAQ3iQCESohcV0vkUQm34pDtyRiAIAhHQgBe8jAz4cgK0wQUw kgwmASUD7xCSEN5CThyecgVSMPAMrwQnceO8gRSsQAc4UOMs8vBQMhUoUghdIOTIYAq4lCbJ ACdJoe9GEATU4BD2rkgdLwRdIAVUUmMOM0+b9CiN68BAYBRmwAfoIBtSwQ+MLgvEwbe66S0T xwYShy5NoZTgQJRgoRJTcAVSId0mMeQ+IAUKwTBhsRW19BZLkfHWQE4PYQKsIBQnIEutoPE+ UBKn4AMbzxsQ8wNnkwwock29QTEXbwJcIBWy1CVLkSclcylVU+P/YHHMDFMHeNIwrWAzhQsJ RzE1D5MUSgEe0sEXcXOniBGCNI8Y57FpzGa24AMioCaTZHRFtKavMgKx/AwiGhBO3pW2+Iwh 5CNv+iRqcvRrZPSwZAEROiCIEGvPDIJedUY+cEFt3lVmJKmM4MZsMGIMWCADPuGcogYZ+Yy3 wKZhuc1qQrZh/1VnZOFfB3YhLgJuWBY1DEJfMcLbMOIiBqQjXCNmGIJLXgYaOWJlIYl+JJBr iMauPIQiIgIkJEIj4ORmmoYjVAZf+1UicishcmtQ6KRntg0jQAJfAWVmtGhpsSRHkZFHwzZo fPFpZAEdcjRgTzYhTpa2MlQgHiJHG4JP/3rR2+x2bt32a7QGIcSD274WToTm8niTgXJzXB0o ZM6sbA6CPsGERroKERrtTPRoZfKGGiwEF3gITELL9HoGTnKB2YrhzJDRBboKF5YP1aJgjSBC DPLTBfBAFsagFKTBA2roE2ShD/bAC04gHcDtPriGD4pgD4ZAP86PCPpARtwhHYzgRHEBEcaA CKZBAbTgTURhdK8XO1kmkRSgHCJCFDwUaDKBCC4EDgILc2EmTxqWD3agfTtgtbBRGlWET96R ZdARawBBbY6RgqTxGPexgr7jN91RfuOsO/aRbBpWGrfIGStIGcGRgv7XGu2RHdGRgcFxHdkm HM1RZLdoZmj0H/872Gt9aoSXUYsymB0/poNJ+G9NGDyWMYTb1vOYJjzgoxdp2Ib91m+TZj1y +GMQl0aJMYIgqHBpKly/dTyEpmwYtGaHBBUcKbZiC4v2FSOq0Wa0Zj/EDWqppmabZmUwbYpr lk4UVl8P4hbe8UqMjba4Zmv6dq9uhl4NQrFITyLWxpLIZnuxZGigFl2bpgPSKGbX9mrQcc8g sGlsFGz/EXiNDWaZ2I4LQm1edmmmhmV+9xiT+Iq9+IS3OApA6461xk/aFdS4LWGvOEIuQ21q ywKcdme8TSBtZmZWBh0ikLe+9pOFyHdV5mmfptC2DSECWSC5DWAH5I4KwgJ+DRCqQBz/LABw 1Tgd6sAQhqAIeMCDo8AIdGAP7JOH3SOHZZmG98I8ouAfbBgYb+o8zoM8hvhkQCYge8YMvMAe kgAGkABCUuET8mwIBEEW+AAO7GFH6sB4OSuLWqYIPMA1jOAeGqQKSmFO9oMLloAh8iTPluA/ kA9M6HMLuGAa/iMKDLZlXKAYhORA6qiavUBH6uBC+W8L9iM510EOynNA0kFIzgEOVLqqBKsO vpGGUgRAekiKKoDQDAK0LAAOkoj/7uEc1sEFWs8gxKAOLqJAqkoaDO16hQQNWOg+XMOnUKEa vOAcBsQL2IBlzqEOKqulpfocToEP5EMBLvQcKsAI9kMO5mCR/82gpYoaDdITP+7hADL6FLr1 HcnzP4KIS06AfA+CDbzA/OJgGgQEEYZA1sJkDv4jBIDhP1pADuqgA8wgRw6ChuIgsabBC9BA ECKpCExgP+KAfFEDp3EhDqQaIWTPCKIgDkZ3QMwpZL2NCDLgIPjAC4LhIBb7IaqNa6xBgTrb BbjAbuRAB7TmHLxA1kJAB1R3ZjqgGI5ADsagD4jACNbAqxFBGjIAS07ABBCiCsKgW6NgDOqg jatgmREiF+DgCNq1BT7BBVamt48ZEbzgCNZhD4qAtsaAGwhCDsThIe72bv+Bm4exnIWYnHsT p3KWiU2o1XbgHFzAe1+jCs4hF4SEUP9yIQke7SFQwQ8yS19tiyI64AElIojgpCI6/D8iBJN4 wNC4BpPwRmY+NtSuNopVJm844mX/A8h5hiE2um+Lpk/242Z3az8AJbGG4BszYmz0lUFfVtdc wEf87AHNBMn97GZj60iIZh1Aq5OHZkBei4rpuJHtqmWaxkT8LKgRC4y1FxnPzJOb8ZPBCA0U pGly5mG50fPMFY3BjWqPJIuxLQG7jWYOubZaOdTw9sjxFWZwHJYj6WaipmHFOEfxdZJp5m5D 1kZjuQVkeWxx24ZXJsE9PWS7+Wxx2G83D3Exr6aEcT1cpsxFYaSNoAhcYBv+AxG2IR10nQVQ oQq4ANVwwQL/TAAVLKAIFMAFBGE/REFKXMACOuAApEEcxKCOmtECPnoIeH0dWIAIkI2IPIQP QCCq1tNlAKQOlkAB4oBL5qEY6oEbtAoRdOAAXIA9XYAa6sAa4uChn/fZ5mMH1mEMvGAaxqAY piGIxpENFuAKSu2Mj2AOxiAcJDupP4HZHkn2HGEH9sD6GinjSShfm4o/qigXArwi8iQXzqH8 1qFI4tMLXGOJOmQMxGHYi6EOuOSOOkAORrcDcLoD9nn/lPEeCyIeIbdxLz4E4MDSMoEa1qED 6mG9wfF4Fwi/k6AIFoBn4sAEkmCsOyQciCDQWNRFRsRFdqSrj0RKPKTkqcF7+S+x/9dhGkpo HTLBSKpgAGULt5KgDnANNd7kAIxkHeKAC563DlatCopBrZktQhQgHdYhHIoANcQBrv8DFQze NYrgs7UoCdCAC7wRIYB+RWXv1yjW5i9Cve2mHkwAEU5gt/kPBLa8CvJPHDhi2nCBC+QbEJIa FX75a9VWIYB4i1Ld1X1qbdvM8tiDF2PdzZAYbwwiCRDBDKShCtx51+rAAyygD+qA9L7qEy6i CiahGFChA1ABDabhCMRgCCZkDLQgxQuFG6zBHepAsl+2BQ6gDjK631FBDjLXDABiTockR4ag cVGs3Lp1qBChGlPHRZ0FqIwUq0MkTp0dVYjsQIRoHTVHiP+StEAkysu5heuK7FjHZwmqdX2s dVhX5USdaTwdtpCFqEqcYsXYrEP0aY+1c3HYdOBTbxrFJDQXWPsUMgnQpxZwflz3tAOip58+ nQuZK5fYXGwcVemzsIoRNnxQ5cK1ls/ZJFGSVNmCClUHXH5n4jq8Lslhwg7XMVqnluU6vB0q T8b1eCEunJOjLOS7eV0Uz5oPk0a8kNFmRIsvH+3jYuXm0AtHY8YVBReg3C1at8hdO3RvzbxN e7ZtuvfoKMpx9T7MOrfp0sHX/cYlS/Ro7biZi/5O2zvz4VWiZNfuGRebYi4sjN4NqIV2WaN/ bg/t/Deg3frtL7cdxW7v7UdggQb/AoIOgQkiuKCBrbGECz5wIOGBWHGMosVNiyl2G2pH4eQC EqbcsxKHHfhRSil7DJTBKPfUowASxXQgygInWHNCGKV8so4opyARjhjPeJALIqkgcUIIV5zy CYem3aZaFKoRdhlmU36WGyOKUZXYYoAsdtiWToZp5WKqXXkYI1OqOZpqSax5ZodJJAEIm2l+ mWacuB2Gp5pmmvklnnjyyciXqtWp5pmMSIndoocWSticmDGqW5l+6uano4mWWWeAmSKqG4GP HlpnpYWquV+A8KUK36IGFrofrAeeWmestIpaK6yn0opqrLqdMwoSZ4lSyilOKfDMKDuuU0wp Y+ByDhJw/7AxCRJxrPNJKbHFUQoRfHUarSmlXJvlOVcgIU4pa9xFIHbrZIJEvM+YYg1YYTwj Ci7hPIOGqgciCLCBCTbIYMAForOglJuqVsUVFezQwQeewGPITWvcY08ScsCxxGE3KYaIOxTu wIfIQ+TyiQtEoHGFJ/dYgwo1z0xDIxIV3IXLGKdUvBg1nmRoxIT2dPBJHENTg0THf05aa6ix 1hroqXceiilmgkKaq6moYgpp11QnqpihVuYqKqqFZjm2p1Yq+i+iuZrdodlRQv30olyTfTfU p0opZaeh6unoqmv3zarTsmL6r650y1og468mXivBT5dNd9u47ocLDKKsIwt2nf8zck4IfWBG 01lxnQMDH1Xg9hrqVcgSpSznwXZtdrLT1MdgiLIpe3ZBscZI71GAJHsUAx+8X4MIM4jw8gEv qDzA6DB66W1iZ4rbmtRXuph0Xweom21RQ014+FE/CmapUxq6W6WBVr7m01Zzj2mA1MvNaKlM 79Z134n+nyuFHUphb9ud3PRGtkQlsFeOa1sDG8c4xx2wgA+8ldt2dzld3UpWFpyc5RpnOeh9 MFZvGmHyFDQ5yX3QXyo0oQsVdDwUnnCGKoye82iIwuPp8EDKa54PF1S1M2GOV6lS4NrU9z5a VW1xHmRcEC9IxF6161WZkhuddoUrBeqKcARSFQOTqDj/Ig6xcrO63ONA6MHHKY6MZnzhvxbU QTfKcY4fjOEbQ9i2G6oQVnZEHvNEKDA6thCGeJxhIO3YR+n9UIcD8+Efk9e8PzJykcszH58A xL33/cd+fMMcnnLTvk1OL3zTW85vtKOcTY4GHfSRxW9GQ5/lxFKWsGSleWwpO1aWspUtQMdv sgPL/9iSluYJpi5l4UtYttI8vbTPLM3jytvhZJm2a8FPePkTa4qmc/KpD3qYA87/tOAW3bPO Nc+ZnW7K5zfaXIh8oonNdsrHOq7Mpmd6Y82uiIad1ozmNZlTz/NYxzPwtGY2Xfmhn7jzoNk8 5XjS8517sjMx7BzPLS4Kzleq/7I+G1VlKTsaTmG24B/W1GUzf+JLVv4DmSNNKUoR1kxdLsel Hu1lSKeXUuNRcqeK9OE/eArJBA3IftNrn4B0ox0B2Q8+e7pkqsQXylVKdaaxdGUwV/nMqm6y eKKkDyBi+VWq0oeV9OllWZfZzdxg1ZdpLeYybWnWZCozo+k0Dz051093BnSe85wMS0gjmfrg RqOmdA4uyLlOzrkSEbLLpmIVek3tDLSaBG2oPT30S2s6B6DWDIllC1rN3lnVoNbBq0H7OZ56 klah4xkoals52vEQFqS03eQ/ZtrRUv7SpSQlqS9ZKguS9jKlyMxlTIPLUs6aNJkfFadcb1vM nUpXj/+NRMdPp1tKuTa3Oc+5jFpNydHrqPJLs5Xl9Lj6y2gqM7395Kpor1q85nYHmnPVqvHs Wdx72teqrfwtYdFqvOL+dq/OhOd51tk7hgJ2n32tjUMZbErrzLe1C2UsZB3LObzerr3vLK1q l1NaeuLTteaMJj3Vm01EXLOxB2WmaPn7S4DK1jsGHSs/D1pQ/963ubXNLVaF61KTrvenx81l f0eK0rgi2Twr7V2QX/qb6Z0UpiTFbZMdKV1JSvK6kazuwMSpVB4rlZPvmSo5l9O+52x2zKq8 TpRredMYg1N4WIXlK3dZ5v9gxz5hpeUsh0lna9bXltt0sVttjM0Q21WhGb7/K0gUO57NAKc2 LJllbdIjHufwZaKS0WtpSSMf/LDknQEVMV/nSeDWss61/gz1aJ0JTcda9rSoFbR9zinbKZ/2 FgZlzkqVG0vrRgG6yzmMRnGqU+bgNJmE1W2Qgwtlmv52wElm7jVNytLdytmXvrWpRm+b0upK cpEHw+64efxRZBuPzEod81HZDB80D/Wp7zEOm0VaPK1m98iq5JwwgWnf42C1ldjBzkwzW8wA +znAxHVxPVVca2ZmGJ5HCWg9Q5wfd4LTne6U9HDWOVG7lhbim0lsg0+ZV1sbtAoDvas5Byoe hJ6avyU+SUNbW9F+9prW7c31aVObXBsLGq1l7bHR/7nK41VG+dbDPSa4Z1l0FsvSybCNtWgb nvCkLxun58ZulhvZU6GiWd0BKmW8yWy+paLdUaBsu1HdPd94u3ur/9kNgG3Z54TXGb7glCvf 6a7RsTK34ValNmTTiU0X33qvFD7lW6ZBBBe4QByRL4Y4dHCROtSBG3WwxBW44YXQw8ELFYDD MuBwDzicwhOnaH0pPJEiT8j+GaV4hu1v/wx46P4ZnOAEPDjhBHgEv/cIcILxOVF8BPzeCTZw QvEBgIDoR9/50ne+8Y2ffOxff/vUn770EQD974s/+rxAQPnHz4vwnx8eCLCB9NkPj2u0HwGc cL8NfK/7/MPDBnSAR/99AP8PpUAHpjACKpAiPjACdNADdEAHI2AKCziAo3AKESgtSCABxYIE oyAuyzAKGpgNcDAKrDAKVJAN2aABrEAFVNAMsEAFrNAM2tAMGvCCzfCCatAMPaANybAGYbAG ObgGdaAA2qANZaAAXnANIFB2XbdIXEZJ58Y8apcbgVEFU7gdUVAFFhAMVYAKUWgERmABXWgG XZEfRpAO6bA638QcR0ANRFAMRJAOocJNwFRsD1Vm9PFNeGUGaIAGfMBfqlRVZ+VMR+AF3JAO ipVhleAFp6B6goBQPmN7SLAFC+ECnuAFiGB3C9c5y2QeKtZYlTYZjbVooAhPzEQacgEPvNAM pED/BpMwCSlACtrwATpAClNACi6QAmtACrn4iqlwCCmgBmSQCqSQAt4wBbSYAsNojKngDd6Q iykwAZOQi2WgBrlIBqsojIdABgpACmowjSCgDWTwAbo4AS5wCMFICmuQCsAQA+sIDmCwCu/4 A93QDatgBz9QC4qwCvFIjySgCP1oB90ABsQgDqkQjCkQjOZIkJNwkNAYjApJCuNoi6TgDWhA kNvokLlIkBNACjrAi95wkMOIkVaQAqtoBVZAi5MAAiBACr3okSupjBNwCBOgjOUYk4dAk5Pw ARMQjmrAix9wCLOokMHoDS5AkB5JkKkwAVawBmSQi1aQCslACq2YCqvo/41TsAZX+Yt1kIrn mIvq4ALaoJA82QPXAAzrgGXk9kPjpoRpqUrzwAlA4ALPUAyTQQ3XIA2lwAmvgAhRAAOlAARa IBo8IAac4AljEDKc8AwU0FSjMZicEA2TUQnxEg2eMQYHsA0LcQQmYAIzUQVFUASCkIdjwBP1 EgU8MA/FcC1mUAfiYBRycHl84BzUgHkz4QdewHvPUAGP+RNy8AxA4AjOZE2iIHwnAAfPYBRQ 4ATSMAlO8Ax8YB2TgJssVwWj8AzWgAbGWQVwAA/TEJdjIAvE4nu+9wwsYE0wQHsL0AOeYAT4 VVCR9R3rgA+mwAtlkAqTsJRTAI26SI3n2AxkwP+MOpACU0AGawACqSCgTJkKw5gCvbgGtEgG hQACwBigrggCJkkGvaiKP0kGCzoBU+ANZTAJzOgNTEkKqQABTdAFL3AJX/AFikACP/CiLqoI 3fAD/dgNLtqiNdANtdAN/9iPeTCj83gDGBAKJakDBEoKJakNupgK2vCTU2AFpBCOxeiSsCiM URmLzvgBRLkGzFiLIDABHqmQU+CKZVqQhzAFOhCl3pCUVuANzmiQLZmkQ6mMtFiLI5qLk2CT rliOUvqQOuANH5Cg3yiorzgJViAOTLkGKeACZGAFKjkJG3qL50imtFiMIBCgQniM+KkBTlCW QdVlXuZ1TfiEsWI/1AD/D75JDZwgAnJwDYZgBKXgBK8wOn3Ze/AwMc+ABDzSAn6Qqo+JGLyx DmYAnW/5CmzAF4jwAbjaECXAnOdwC1IgqxUjH32Al/VQBcvpAQshnJywAJ4hJJ6AChngBJ6w BfVUBUcwD8LgBI7JOUeAq7x6UDAgDPBgDX1AnVxwDRXQAZPQe1/hr+K6Djwgq4BpHYgwCk5Q DCoWBdhAnQu7DuRqnOtQCfAgDNeCYBKnc5LxSzRRCgCQDT34ilOQCrAQqRuqBpPwgiA6oLBA iyCgBrg4BS8YqSAgoFIZoAgaoMxYoFMQpj2rkc0wkgkqoClQBpZABV1gDl1ADpegCPFYj/NY /ws8eqMkMLXdQAKu4Ard4ApXqwhUO4/9OLUuOrV54Aok0A3goAykIA5GS6FWQJ8gUAjFaJC5 qA2bap8kOwUucJUlSpC4mAogEI4OeQj2aZ8euQaCOpLgiKkImgoN2osGepASGaUe6g2Vewhv OpVIaZQXSowZiqYSyYpeCoze8JMgEKIEaQVkMAE6MKUD+gEpAALNgIuYiov8GYzeeA1wwAMF g5ZpuUO/q2WrYhvukKovgQapyq/TCQT8igvWCgQesJgB0gIs8HrRsD6j4Q69OQajAb2ecBNo QJgz4Q6IuQV+cQpOMA3zVQVIAA/FAA/S0AFfVQnPIAyPmR1ocA22+f8MLyEL7uAEQJALPXIN jtkBLXAE13ANnxBjP8EHflkvseoESJALsoAGzPkS6yC+niCdcMAJc8k5iFCc2+oZVTAEnDAH VcAD5VsK5yALRgAPz7AAnbZNpzVaHFsFHzsDr2iQtAiDAUoKs/uKmFoGV1kG33iVNLgGV9Ck tNgM6kAKQruKa7CVE5CoLvCmAzoJOiCgG6kDp6AKp9AFeQAGACmPZwyk95gHXyCPQXqjN9qP inAJO/qiaLujMFoLJJC1YrsKtfADedANbCyP5PAAJTm7qQCgtBu4wJCKKtkMuTuLwXi7rmiV 1bi5qmin20iNRPmm9JmL6DgJWOwCwjgByeD/AiW5jKo7jAR5jMzYi654pSopkWRAsqWrszGZ iheqi95wkSUKjT9Zi6yIkYB7jrR8yqtrxbyQDaggqmAHPVgmPczjuz+0VF/ClyeQCbmQG3Jg E54hB3HAI94cDo4QKnnWB44wBtosd6NhBHFwAicQByvxSi0AA58gB32ACH0AF8PzFjPxH2CR O5zjLn1QBbB2FIyVUTjRBzPBVwlVPKGmX3S1EP40V5KhifVRUZ140CU2agdmWiC3Yai2EH0A BwAABylAg4FrBdpglTpQg80AAkf8w0U8BbBgs2XAw1OwAkvZoEv6isVMCiZLxaroAsAADKew DPGwCd1wCShQCxGA/7Z20AqtMLX/KI96HMhcm8dmrMcz+rU7KrZfqwg+6goy6sbdkAdTW9ZY e8ZgAAbicKX5KYxXCYyP+o3BaKEFCqFCGMU68I3CqJLBqA2ou8V3HcQpW7SH4AIqmQyxSNfl mJ+MqpApoAPYGIyAyoyqS8vCqIz+mQqFwMtImQrzQA1Q4AdKwAIZIAazoA60a43UWAe6DIx6 CgLegLqHsJS1+Lh0wLvN3GVhF6peR2Tm4S6kiGlqpVFoOFgZpVaSxh9FtR8ahRuzJGt5Nktn Nx9oJoeecWbB1Eo4gQralB8PXXTzfBRmYAY4YQYbJ2PFNE+bITvhvRl8QJ4YbTsaRl8CZf+I 7+RZfOVZm0HSCPALyQCN3riKOkAGSLzZs7vZsiugEQrTSpySVwCDBAqWAdoM05gNy9AELyAD 4LAK80iP9WgHNdAI9UiPJ/4DfczUP2CPVOvHIc61WOuiO3q181i1rgCkT6vWd2yPWpu1eZwH JACkaOvWOqCSsUgGzSCoCUqSCImRtuu3TbqhpAChm82RmTySKCm06DiVUYmOCW6OV7m4RBmO MpkK4biSHyCoLvABszALGcACR9AAuoANUiAFPIANe84DPIDnex4Mes4DulAEUNCLtKyMU7mh vByieRqiGlqfk3ANp8ADkRRU06xlvi1d36VQIMAjzwsH5CxOZFj/F9BhBh0AHKIwBnwQBx6A CodRBSzwCUagKutwBFzgCHIgDQxbcsHABdOwDe7xwtHABzzhCLEUscWgBY4wUEYwDXVQDOeA G6KgEDAhDkNQB7pJ0sc+WdUwDSxQB9cCA5rnBVywDkagAHRQAXuwDroeEsHgAgqgAIagzesw Bl5wAntQDKhQCciiANYgCxngAYhgBkPAcueeDfawDn6gABmgAETwOgd2TvIREiFcWgolCw2D AHAwASOZizMblSmQg6c7BWCJnyW/lbObDWRAxN4otFMAAsBABTFgDsoADpfwjy06o/Gox/FY o/loBzxqB6uQA5SwCmxsB/9ojzPa1WiN/7Uk0Md5HI9TG481buNae6ND/seKkNZZC8hd28dc z9ZpmwWMugaTAJUE2gx6eqQpW6LVuMUaiosfgKlrH7gwb+EbmQquTZCom4t+jdskmp+H8AHz IAbuEAfVMAaiUAlGsOd57udScOd6Dvl9LuiW3+eVb/m34OezYOhBnKCLW5AS2cMxWaBSeghl EIBVQEiTpOl/xISWjiDdMWGlWQQesAVkFgVoMAcZEAwDVQXA8CxjIL/MkQFzmZ3iUATTAALb KgoVsIVRcATS8E2ocAJeMA3psIXDejO5cQTQfw7P4AV1MAdeAPDHv2nMgQsCbx2xngXNsgMn MS2BJ4h72QLE4v8Be6D/z7IOkxAN1gQQRyp0qAJnz4l69cKtE1hlXZQWUWTxgJOuRYsM3Na1 6DPESBUQfdZVuycnBJsdfaKsY3nR5UVZLzeyRHURkTYEXpIdItWsGSk1pKaUmURKKBltP5tN AdFsglAQ2pBOSQUi26kYyshh6NbNzipFq2qFVVR2lR1FaLu56lbWjp1uYMsqQtYIrqIf3Ui4 skOibbcaP2p9yUMiTy1XZREnbouBhGFFeVzRIxG5sOE8ilxtTlyrWx7Bng/fUOYiRRkQUw6l SmF0QioypFJNKUqKDG2jh9ZoM+oi1RqjH1JZeZ3K94cPYmZxyaBElBl8uoLhk4KNRzD/Hjyw ScFnPbuU69p5gM8+Ppj38rduZccevvz1YDCgzPpgZcoU2aSsGLVyiGprb7zR4ZAUArRBhWDQ AQQdBhdkUEFAHFQQwgcrlLBCDKOIAhcNo0CkDiGqQCUYVFABpMNb6nAElXWKmAaXKq5wJIoq vPgEFSO88GCdJJQoxoI+XFgAF0RcWMICI9Z4pYMOabTRiDG8SAcXGJ6Zg48o2eBQCQ/GYIGb c2B0QYsxqJEGlSQyMGSdPrxYgoVphpgmF5Bm7LAFUaR5KCY55HShmA5YcmEMDkXhhsU+6ihG 0XMkEmWIRafpQJZ1xJBmGoaKWYdDRD6YBhGWEIFBgSEUGCMi/5lagsmlmVgKtAVE1EAADjLK 6MkoMlIgIzZvhMIvNqF+wg8EYCxpAhoZwFnFq1XAsgOFsL5Ay44f0CJrlRr06kbaxzwri4Ra fhB3lRxQ6MazsTb7DK0fXMmMBMrYetfdsTQba9698HLlMcP4dVeyfbvaV15XwFCGA1J0SIGp 2SaJDbbagFvjg9U+IIUGapZr7ohKjtBFiiqukwK7KkB2D7uRy2sPO3zay67l91S+br2Y4csu ZPBo/g6fWZRIJZVDdBNOqIl1gI0MF9ZQWLZUeFHBiAgpXPBCCxt0MGoMs2aQpXVk0bAFSh+K 4sSIToRo7IiiYEQiXLhukqWzF9Sw1f8mwRZ7w5k0xKWPPXKRqOu3Ab/ba1lwicjrDs1uoevC KaXUJcdRJfyhxbvGRRZKOaT07K5ZanulyfeUSCKwY+rcdMxjCrvzru3eSHWWMA91EgSWSSaZ NVIAwZtkQEihGV2bqTW1ZgrJpot4ZLikLTuy/aKGaemB3i24noX+C7fSSqubH5ZVhITuCwvX s79qIcFZb+1g63s7MutLkbEa+QHc79fyzC/za7n3MbbG+mwtxXTlM/rTzLn0VwswgCMUqdDB JJAzjyJwwQ9KUMIYzGAEC4SMZNfZDg9CVoWQ8QARHsRGFWhGHuuQZ4QerFl2vGOdF2oHPeKp WQta2AKdjYf/CyVYTR1cYBukpcIopChKUYrxs9eMQBgwgNDUGvRECTkRQ1OjWobiFgV0fK0F ZtMQF7HYJL2ZwQtx8NtKWvA5MKbxbhxKm+FWcg4j8OEK0+BDHxziDi0kISK4YOOGVtJFDSGu SZirAh82Ioc5IOKPo7vTRdz2R0rpcVOy2xzXQHW5yiEiFWzoHKxuAbbFqQ4boJIFIlABwtZh rnKwihzXKleFSQDgF6QAQWqm8LtmrGEKhdAGCJbRhTQo4xJfoEUNjPkFYiqiBtizg/SgRws7 FDMtaDHmWaplB2dYi1pl+YH8zpcZ89WvLeESzAC5R7+xgC9c3CSgIuKyr7noJQ/e/9NMZCID LvLBzzPNioA8EmGCaLDgCEowA3TwcdCVVSEYwQBPCd9DIhBeR4PlAU9Fb1Yeh4anCjZsITY8 +p4ZgjSkLcwODkmKjYzJ5gqk4A1LJ6GNFADnA3XoiQtcUAc60MEME6oaTyOEi6hV8UES+kfW uHgiRKwhGixBwyc8lAohsMRQkzKDJxbQAVCJogIsqkIxctECM9zDb0d4RRVgJA0+dOoTREKD NTY1Nj/yERCMWMcklhC7dbjDEIHqgKlwIQovBMoIcLBHkyoh1rbtYFKigMNG5cANrIphAV3z wwkQYYR7nANWa/gER+BAqMM5Cgl8uIhD1vGIClQBEZNwq/8s+nCFT6xDFJnlAQy8MAZEqA4R 6lgCIvqAhAwggpSyWwciWDkTu72kSAgwBinWMAldkkEaL3iBDLDwlrfUwA2aeEsx7cDdZSaz BsWsgTPoEc3mreKaNfgWdqX1A+15ZZ1i2VctqLXNH2STX/DVH//WYpj5cc++XSHgOivDvGo1 ohHywEMstsEcQYyBDwWFjhSMgFAWLjTDwSiZBxXKHQ1WxzsaBg92NgjSm4WQB6hwIQ9uscKO klSGGBXPRcpj4xbSLIc8wIcfIpGKXt4qKToowx50uQYkA1kBpXiGKCjU06FaTcoTipBQJzQ2 XDDCbG1jExq08BAhraQSr0AFlSr/4LcTncMLLDKU3/BRAQvgAo+THIMFEOGFGeHCCILgkB87 lOUTucAaSdBbBjzAoXW4wBF4esWm1mENCySuCoTykJFkAQOxRsFQqIDIEayRgQVEpA9wyAXm xPAJ13qBD2eTiCz4cNfiriEOsvXCHT+1DiOMorOjPsc6YNnZ0iFCDNZoAQykIRJhu9V1LVFV q7gmBgTQwRIvUAYxGmHM5uUgB8hoHi2IGU1oOqMGmtAEPZwZzWWO15nshV4NzsLetDxPEc64 5irABz64zK8y7MOuV8pCj29xL+BxERdcrt0ISsjDGsdgARMinA4zBOPCumgZdy7MHRAzFDwt 287IStay/w92fGVSqM53SpZCko+Yhi2O2UhJylEZ8wDmJb1xzGZecxvmcD0t0MUs3LGGZgAZ BFfIJbGkorCfqKEZdFBBOqIYZShKucpT1hoUx3YiQHCoCuewx1s15FuHIIKPbetz1uHWNlR0 gMsdYkkS+rg4PrJ9i3pbIx8J3SSzZX1DY4er3uGauMT1OY1glJwWMXenmKzkIXsiXOXMSPbD Pa51G1lcclX5OONCxHSmpXzlWSIGJ5CjBjkwphtooQnodcIZq4+mImjx+vF2grztRm8xaZHN t3z73K4HOPbOqwhpNXOa2NWes9ISmG26W5154d4lcMCC5+jCCNO/4HQOelCSj/9MCtJpGUOt 732QpbDiCAWZxrHTwZuRB2QkV5lFU7jyFuuMPCunGcw9akOO3rzm6+H/J0uKQ/4rKfXYOfVI jwE0glmAAhDQgV2ZBB1IjTUAgZ4ough8BjpwuquxMp/iqSfiQKqRG8TxmhO5m0UaQb3xMw+5 u7aJiHWwgFvos1uQBRACKsMBhLRRvLYpnElaBzOItC7Cuiqbmw0BhIeIO1zjs7g6u7ITvHXg gU1pgRAoBzZaHFZLPFkABMJJvFYzI7AxHFnAIIjogzWwhxB0pNTxPLyivDNMri6MCdcxLWoI PWR4PWeAJtijBW4bLzeogfNaJlpwPWdIi9c7L2j6rmz/wrbcQ8R3ewtucgOAMzfoIQFaALjv UQR6yAtucobK8Iu4gAtr2gWBwgfosD4jWCjq6L5gsIBggA4z4A6EwgeFgpkqwAYSo0UQsg58 EJnzcI8Oq6gQGhlaZD8Xc4+YiyEZwz8bmzlldLEBhDmXkDn/u7H6yzlm1BmO8IN5ICIQWIPY QDLgeMDdUAMQuIZrEAQnqjKs0ZqnqzoLcZA90kIjiAMTiYIxmJIoMAIiUAAi4IPPMYM/mYY4 i4JbOIA1KIY4ICy2iwI5WAIO8YMloIZiCIPeapIMgAMgGAJHKJJi+BMveJGV4II62AM8eyuy 8QMvOIGbOrR1yIA5WAA04IZc/0i0aOgzalgCPkCDZ9iDT0AEP/CAScmACjgBNnCHCpiHBWCD SSgGsesDEBgDUVCAcgAVd9CU1im2STAENqiCPiiFOqgjNCiGKuiAaiiGPjACqMytylGdxmMV G2Kci+CcxckAJ9gAZOgETbDLOyxEaWk3OwQ4bComegAvepjESWwe4nuLwfy97IIeRjyv6Gme tICvuCgL5omMtpifL1iFL+CeGng+iLugg9IFC6AOVzQDZmCvs3iLHJAHSChFhaK46vgwjDOP imIonAEp9uOw8Jg/8nAZFnKPHSvA7FgP9DCp/3PGYfS/nGuBlBlGaMy/aBzGnMuh5bShPjgA GhCeSf/wiZ+YBHGYAjKALheYAjqAhzFIRyniQHbsQCuLEEDCwr86s5VAgzhomzjwAiJgAbNq ESSQBgWoA2nYAlTAM63zgh2IuzNyBw/QowMoBg7pgDXIs7gThVfwm3X4AGvgoxohQzSogHqw hgWIAyahwSNYE1xIAjnwyb8qhmlggzCggMIBNVw4NQ2pkR3YkAyQBrWTAwddiTrorEs7syJB g1w4hz7og8uBCLe7k0zAlHXAhs9aCa3qA0RQgOBSLc4zI4jAq7lBLkp5Fa6hFLkEBWbohE6o AWTQhNfjrr8sxNr7LmNSt27jvev5Q3fCLjxNi2zKL/j6CmphL/gii/typ7b/ODBrGovUzItV aASHE4QLKyh8mL7oqIZFyAGwwB4UaJZuuIRLYAIjcEVY7D6SC7GIYqGUGRmGoqjx2M0Wo8WP Gk4NY8b0sBn8S06Zy7+LUE6bA8Dh5KgCxLHlVA8AjE6dew9dcIdZ6IkGBAFYsA0kC08QMIVr yIB0pLKoW09s5ak+y6ITwYYn8QMkWIAkqAIFiAM+YIEhWDULKIZyOAI0WAK3A6w4YAEXQILC cs8o4BJC8wMtQLRUcIQkuIW9y5E5WKqa5JQ6KKx14AJDcAFpIBQEXYcQ8AJFOQEtCBR3oFhr iAMPKKwoMINTsAfPc4e2+qtBi4JKqM+VyIADpZE4/1Ak2fpKeaQUUXABJkm8J/0EORCJR/gq GhEFrhGFOHAHoI2dymGSZsskLqscZmMJudyAHNjD0xsv6EE92Ts9dIMmY3K94aPEZirEaXpE xsSWacGW4Yu3PZ0LemsLa1Ind8KLPqUWTlwWWxCEIzADGKC+SqA4MziCWXgAAlOmbgCHRKiE 0BzF84iOlslFkvNN8Agh3/xNk0k5FYs5Ww2GACxAHds5mqtVl+BcmNsom8M5A+xVk2oBE0JO G9IFKKABqohA2+gBCNwNbaCDa4gDRtAyD1TH9Zw6o5IbiDicsvkaN2I7uDkjLHsbvXnBhEwb s2M1thM8s/M61+GQ4l0JQv+73j1aJOvFG1PyGri5E0BKo7dcJIi4nI3YELD5m86pwtAiHM0D X68xQ0eCG65ZPDPqJMpRlRC4AD0gPWN6PTeIJjPthD2sgauNJmcgN+gZTHAzt+Y5REYsRGRY hWwaPjzdJmqRTBJgF3jSYHt7n8d4i7woDLxQBEIQhOcwg8+shIJKBxhwYRZugErY24LSBRpm 4YIaGekwsfEwIQ9ajxDLGRXTsFhlD5qrmcoVzgHcMWh8D1x1sfwb3c5N4k8iQGS8Ys4dQJlz Tp6ZBaBrhh6YgmbwHS/YFRCggmvgAiyiIvbswN2Fuqi7mkD6u+0V368BpIeohCHgtBEcO68T SEP/SpuLwIUPmKyxCRugyjo+QgQ+ODNU8RBOQ14+osK7ucF1ENivEZvVuRuK2IESSV8evIct eCsuK8IbvBNO1hyuuRxBYiTGccPOkx3SMZ0/ciTIYVpXOgIn0IM0nVrSu73TU1Nwe71iUtM4 fSbYc70AfiYMRos/nB/tqZZpwgtGLAt4Cgt2+R5p1gtNvLdamKeCwwFBcISOMQMZbgAz0AVd QGdB0AUZvlvDvSAK6z4NWpkgdhnsEOLxSJmJMiEd41XwuDkdQ4UCbA8t7mL9gznj2tXhVOhm tLFPMk5kVOgubkbhlA9qsAJtmII1AAZmLWMdEAcJmFYsguMNvJpsXU/C/6MROHArlpiHOICI OACUgsgzOUgtthOjRVsHNmARGPCEMOlJREiCdagDt6qCHuCDsHQBKczfHCmzUXOERh6CaJDB 0WI7pLqtTTGDJYmCEDiUKvjRDUEEOOism9aQR1E7zGoULxq8q/u7E+yzuLNeoPIaTAokt1HD NgzfxQGV5JoJsVucMXACBmBmPDQmcjNTdaOFMx3MNV1TyB7g7zJMTRC3ZoKeOswmzDZMC26e TcwvazE30U5belun7wkX+fIf+GkWQhgDu2XhShAFjqkEI0iHIxAEGe5bCjOCgrq46RhV+Chi 9ggZmvGOUq0ZlIGx33yPFdLiZMQh6T7diZ7imP+7OQCEOQKUVYme6P7LVePEBhqYhe28Airo gWbIBiQDgV+4BhO4OmuluvR8svak46vrMx7oAapum0lYqyg4h3JYB3zwAns4UYEos+xFhTFY vA/4MiMIWYzwyZUwkiTw1h0gNC4Qgu792DVrG6jqgHHtgbWy0cDzEBCIrY+FLFw4AshaB2ro 149FgrVaB2DYbzFSO2ywV0CSXqwDvLf+qcHbMlcyHWezmxwsHVhhNuOq31BhCAQw7DvAlmMm Lzws4PGqQ2POgQU+PWPGci435ttzhjNFYMQsJnrDrvPCvT29r2nJr/OS5kqEnx+4hNQuIPZ5 AxYQBIg7gtx2YT5HZxf/VmeIa2d6Jk3sm9zcZLH2UKj3UDHvENabYTHpLA+DVg9EuOJM5lWb kGLjvFVPh2LrqNUufg8d+45qZEbprkZfvehkrC0o8INkuAI4gIVm0Ib1ZgVeoIaxGaqpq1Yo W+mhGkK92fAtw7s/AyO7Fry5VvYTHPa6Md8/W7tNLkL3JBvAW6OsA6xtkAMXkBQ26vETcaM9 6rPy/TMqfJvDa6S61jssREPHya3HAaV4J3K2RMPFEYVrAAB4uAAGYIANkPJxi6bLNia8HL1x c2wzJTdjRuDyMubELERzK7fmcUzDfIs61J4LztOz0J/uwZ+8AJc8CPkEuIM7GPkEGIRt4Bjc /+bz2j6CdKjhI2iAu6XnUuSwSF2oEhNGHtNNfn7ogK4/F/ugW+XcEboFgG7iJkzdzmVoHEvi Fgj1ZdRu/uv0K15OKY7OoOdcFTMDd6jIbEDvWheeHuCFeWhjlY6y+TaqlX5r6807GsRXDTEC FrCAdWCBLRg7t6fBY28bQiNCa2iUvsvjleCQrCv8Li1Cug53XMCHOdiClUgHRYqCI5iDLGM1 RKMRFigzXOBXSSKdNrrjtKkkui7Ct0IVU34bULKJTJKDVxgDzlND02FBZhPYh0AEUeAFBOCF EdB9/9WDDfgC2euEHHg9u+yEbhu3wWxgbxtMcltgZ+C2Y4bTPCwv0v+DzGbpYGZAC0mMzLlQ ix9IAEwIAEzABD0ofz1gAH4/gwu4ACdQfydofw5ggXQQhZXnc9zuWz6fMBaePolTxdHkDoAI hk8KQR4GpeATWMXgwlvBsGGTYtDgQx5VglVZiOrWLR4dO07k0YJHsJALRRocGXLlSJUpQ94a 2bHFR48tP9IEOZOjTY9VON5ah9PgLXxcZunIlo1KmGzaRvHKEgUQIHRV0VmlipVq1atdu1rF GlYsV6tTcQFCu65OHBfccq2jVmcPtx3ruHjx5IENqjWfcOFi4cFasWnrolRJVWyBtAVVFnip UG9LlTWOcK2DYkhOOC9/cbm7N43NggpbcEX/ibKuzxB7Ufp40bJu3REPHarcc1RlDJIFfHCN 8bQkFx84jqKgXleFTakxqNbFKeUIFQwkY9YhIvUJkSg4uVSjboGm3GwY9+C6s11FTrHZ1JZ0 QESkHOooLWRlkFalReU4szMUUwUiLcxG4DoGtoCggUHNtg4f8ABABy8j8FIhLxfooUcQNSBT g4fO2EFLiJrYQQ8ttGiiCS010GLiiXaEyCI9Hrrx4heK0ENiDXYoUiKMP9gBIow9KgKkHatg cgECFzjhBAJOAOCkkwAgsCSVT0KJgCoU8HGEl0dUIsqXlVRihplGwJAQQvgkZAFCwQQjRZwE JYSPEfhUISebeZa0/xA2PPxZxZ9yTiTRn0QBShRILx36p0sqYeMSSi0hOtFMCXokUkwg0aQp Sjx9OpFKmIKETSVFZHDFUtqA0IwGCMwTRVZk0VqrV2R5pZVYYk3Va1qI+AVYFC5IY401ByyA ShKVSNMBZn51cMQ9C8TxyRjndODCAvUhh8sBHiSBmBdj4JLBK6ggh48XdmVgm2p1fJJar4gM cc4854hijRlzOFsFHPbgwgMwbKAWghfoJuFCNKgBklofp7iWRAbcJIFZMX9VUgEqiGB3GLfj rYOLEV7kghmBfRxQQS5VjIsZKlXU1wIiLlgzGyJQBNgCe6gYaN99h9E0YBRB3YLZOkkExf/H NQiMMA4dNtjAyzUVLskAAxu4EmSLKDqjyY4sauLMiVyPnaIdNSgiojPInA2j2yg4Q48iq8C4 45CKKAJiDat0E8QFAESJAOBPEu7EGWdccHiTgFOpyhiCfJmOGV+aIXmZZhhhphlscn4nm0Zk rosZa27OpkCfYyTFoHH+iQ1GBh1aEg8FrTRporVPdGjtiMAkE1EuxSQTTTItGlNInSp6PPIi 4UONElc001QdoyDggixg7QoW9rnuWmtYYMlrn2q3uCDEgbhUgYY4REjjCIFmeIAKZmj81QIb 0kwyh/mzxSFNHdLsAF1mKEYx+JC+HchiNfWYwxymUYXZHMEa4UL/RRzSMRtc/KwKcbDLOkQR DkSk5hafsACBziGHT6zDCGxYRwL5cI4oJBCGsjgHG3aACAucI0EtMGCC1gGDMazwQKmRhWr6 wIZzZISFskDEOc7RB0QYrQp9OAfHDgNDFspMigOSxRKhyEKgtUB8DToQgQiEwQOhQmkA0IAp fECHp0mIFzYQnBMuwIAz6CEBX0NBDbwmpLE5w0MwUpsdSEQLFKAgbzua0d6QYSRFtKINkmyE ImrwBWfg7Qs1+EE3bhCDLAHgDE1ykpKcwAA9bCCVGcrQHZ+QgS+JohJHEIQozEAmW4JOc2aw QDBsqTl8iK5MwLxl5jQHA8+xiSBregg+/0gSEdcJynUWgd1ESsK7l6wEJNfMpko2dbxMAa9T mMrUS7pJKY6MU4e/G4kRxAAFbUijGdoIwzXE0QJd4ap7Y8neVrqHPbJMpWFUYQRaBMoIgaIF LchpWPgYxlCuhC81aSFoWsRnUYgyDDAh61VAw6jQtIQMLQdlxFlUs1CRUmUqIZWFsBpG0rPg gou9Qk1yIspFgQZ0KvfxFcMkipkrNohACUTQYRIoC4+GjEDjGxouioY0I8ADAaOgwihYgQRj jAIeFJoQAnjxpDueYQPdaATbwna2FTkDBYOkRyEHyaGv0cINgXDDF2jRoyPZrUU6ClGQ5raj H5BgFZeIxxmedP8BDB32cHqwmh4Oy7grqWAbR0iHl5QgijGkowGVSEeZbunLyZFJEDAAreU2 ezld8iFzqq0TPswQp9eqbk6zI0hJajtNklQKd5Wqiag89RLj6RCdnNoUUBJkPJgARVPDO25L 8FEEPwyhGc0YAi/qMBVa/TO73uunPvMJ0dRghqQpZegRpCE/YLnPQNjp1X2MlpoW+CVBEo3C IzQWspAxdDYmjQIqjJNEtCCiGipDDREBId6ZejQ+ypmENZTVMDIigg/OQU4SLCAgMxymCpXI AB+sSJs4ZKAP6AuBNvYgipCtxxpjQARB+5ABNFDjNyVdBzYy4AIXZKAKRzVqGO1TVHn/Haho 9tFoU4WSwmcgwAFU0MAfsiEBU4xiBFHe6lbpeLgNXEJEg0xbiFZUIhFZEkY3MMcwhsGANPgV Gd1wG4hWASIQpQ1vbnPz3S7RBcVezWqCY5xXp3YNOpjiGXSAgwkaAKZZKgFySgCTGQw9Sy/J EkyUPUID0jFpMknOlp7trC7YhDnMJcS1roVTMHRhBFKjGtUesa1EQhKnanoEVLG+RUM4IhSd 8KQmH9lUS4ZnqeTumnjJ4wgbJiDdZsABASDAxT5nxU+t6AqfV+Hu91IahSQcdR07SEILRKEy htrnfsrqALwQ0wek5Sc+hYFiEtcxCSEgxwgVsAAuGlCBPqxD/w7c6MA6QBCNkGEDDihkmSNm I4pz4aI4WwBvavjAhYfvIKGq0VZM0WANmqZmHT0QRH08Vr7wUIMbB+ACGlQ2m92goRTxWsd4 UBMaIhBhEkjosBxEcwR0zZdA+AAQEmqm3vuaMUG4kJ9QABPGAiVBKEkwQikAAAsq9KAM2oAF K6ZaClaYgg6loNAIusq0KCEuj3XVWtdeJDY7QOMZnIAS45b0jDNkjUiVDNKOaEECtvrIR4pw hSoX+zfAeZUXdHjGCOjgCVaUQgWloALWS3EKYLxy0l+C3BGUYGjKL/pLXqpGAzIfSy/ZknKf L5PlKldMM/DBDH2wQGsxByfTJeR0Uv/IyGsp0ifZzZ4i5Gx1cmvd22na+teZ4vWuY22RXN9i IyrRRSWYL0sXsAIOyHYCMNahXa5Me9rZ+0o/ub8VhR5mHe3qQBTMcJ7woIZZzlqHX9YxjyWE LAOGQMQtivAb2rwCLgoDjBHOU/576Ji38RsaTAPHiEIp/BsigAAKRQEMVMAD4cPMrQMjhIu8 EJFDTdwCpIYYTMP6xVQSiEExbAyAOIu2wBAuwAA3nMMtjEGzGAESfMJuIOBsoEEBoc8QjEEV KMEcbAzNgI4L1IOzxBSwTAMfZAAcFEMfDI3H9Fj4tAAGGR1TMWGD9EEpIAAVSNcKSBcVUEEZ NAPjsUIY0oH/MRjDCGyVVz3JGVhNAnQZ15hVDbxAKV1JljjBMLiNltmBWv0V3dhBLWBCKm3A Yi0OlYzANcCDD0zVL4wClCliNrBCD4yCA/zCLyCBKTRNKcSB5n3JGFieo1We5mVeZbEApX0J 53mJIBiamICJZkFaOtQS6ZlJmRxBLm2OqdkJ6OAi5oAOnIwan6SO7JxEclXKqHzETxzfNgFX 8IBEMIwENujCaOlC5bFAKsABHSCADTgJPEBNKcDDVFFBqzQDL8DBLZTF93RX92EXtXXXWQBC AoFQGJVRghyVT+0XSWFcwxwdcpgUYEyUQgVUfTwYLoiCF8TBGIjDxfWYsKiGUg3V/xDtlExl HDwWCGawFFDVx1DdR0UukWro11QUmEdqlBkV1JCx1ETqowUeiFA8YdGspELCoz62ZJA1Vcfx jL8AQA+AAAiQgTc0Q05OwdNpQzY0Qxj2QCnMgANk3YRYCJackivsiIqciIeswjssQ1Uugwyk wSXQDTKo1V25jSLcQN/dkZN01Z9R1QwAAxciQTMAQw9woVvCwTicwijMwCK6JaDRARXMwAxw gSYumhJEWuVNzhGMQeXJEuVt4hGwgKEpwRgsZjVonqNFmmbJEmex4hjE0mbVEuZcDi2aGuh8 jhFcRDLRHm4Zn6LsR0/gFm/dgpw0GvOBDgtwgQuYAidUyf81XuM1jIApQNlU9UAPwAIsaEMP rAEZaAMpNMMkpMIakEIP8MIpVMH16VNZaA8+7ZP2yEtMHYgsKEG8IEhDYUYEhQtDfRRiiMNp PBQgrAPAUEVBCeQcaEEH8EGzMMKBCdRhMBS3cdGBIEIFduQ6fAB9yILCtAA2rAtgmExy4GfG WRFHCVEZzVdGGVXDVCQ+pmRMVWQUHJS8tEDSMRVNgWhEvZf41AdLhpTRVUE2IEB0TQEZpEAq gMAUrMAaHBssNEM2lEEXLgUrUME4qIAZBh4vCIMaBsENwIgm0EPXGNIfeWUgdQMmYEIAoBID KEmF6CYduOVS/GYZ0Cg4gsAVeKH/UHJhWmaDBlBBNsBBXdIlHIxAKZjCDJiAlxTmKI6iYham l6iiEkie51UeYFYC5KDiLDlmA3giKVaW5lEeoU5WJWiWIGjaadWi6OxinbwOqYHKTxzX7FWB GYyWKIQAC7iAJWBjlUTNNdiA1pVCKUiABijFDPRAUIJAGSCndE0BKUzBcbooKZDBJNiqrpLC GjAnKTxndFZn9m2ftJ0jP6FD+JxPFFCDF5xAOCgAEfAMbOzBHizAOoyBB4SMKAzBJ7DBHiDB ZfQBHHjADvDBJGhBEmiQJ3xCLuBUOijANJxDB5kXaMDHOSjAwuCCGFTAAmBLH4CAI7iYfiho giDCB8Cf/7tla8DFIDVYA8c0QBJWgTsUwxAdRggYxjpkAhEcBhcgQQ7BxnX0wQxsSwtkwMEg wnvMhju4gHJMQy60QLlaQBQgAhws7GzcAsdgEHI04ZDJS9HcQsU8YUvewpDxhxfwAhLoqhog J3KSASlowCSAgBY2AyngKCvIUw8QJeLtJoU4wTMAAZVqyNydTQ6EiCJ8wRckwB8G4mJVydSM gw9cAeMdGwjAAnL2wBWQwRqkgDyRwdVOwRRI16x+4RqAANRhYSmMQjakatYRGmYqAQtwYmVx oiqOYih2HmIqZgOIQuYFZp2KgqI5mhkkGisapqIuZqSFliwZU2dmjpyQGvPpQv8DmMkYzAI1 nEIpoOEcRQ08wMOTmQIkUh3iSlcZuIA2pEDULqc3kMLz9qoV6CQZ6AApWMGvgoCtAusUpII2 1AFzrsFxrgEvqAA+TGezlSM6lqP2SRs+pkXDuEO2qsYkxIu/AAxqHMEr3IYXOEK4KOAn8Acc RJxASkO4ZMzCVdT8aGAUZAxchCAfzAx9xIVhoIYSGIKCPZBqENETAugSxNQHbAsieIE1wIBn rEMHTAJ5YAa9ZSc1dGAUAAi2yV+4rIMYeDAiZMNlrIMZzBwivNvJjUEHDCRcGMG/REHApcN9 qFcUAsbRvlfPDllIreTRGghm4AKwIAArtAoZGKeulsH/JGhDM5DBFKhBUE6B1DUDLISvjapx NvxCGCYlhdgRA/RChpDDBgxDIG4AA3yDMPjZCJyCUnRh9oLABCRDGaRCciZDM/ytrk7B8j5v MhQnl5bxGYcvFYQxHEydAoxCKYyDA4RB4sHpl0wupSXaotGpYlJuY9apErxSYSpBKBLmJ4rC I4yJJh7B5LCA5XkJZRnapZFJLFVDNZAJmSiBC4iDNd7mHHGCbprCqdLBKMCCKUjXFYTxFEjy cWrDIqvBBDzvrh5CsILvIShn+CqnDhynC6RCHYCAGoDAJABrChQCcy6vDujk1YoDCFxDKVhA tJkjtM1Kdt2KP+GKRKXUEdQD/zUQkF3wVybYg8fwwQLwGyrIwbWGQxi4zy3EwXcAggX8BX+J wgZxaBQYwQvZLBs8UB/IgRDYQx8E0Do0EcMhAhuEwA7wG0fJCyI8B2LID39t1BKqRi5YQE+T KGbkAsdAES7IAb9khETd7HVcUH1gBhQ1jHZyiwUKHXicUUc24YkeHU0YjdFtcNK52xWCAPjm 5BqoATo3A3Oi8xkvcvgy8hfXaOD2QJkininAg4W83TU4wdS06SnMQBnIajbEat9aATBMgTdY gRWkwGODgDeUwWPrgDdMQfYyZzLUajMk8hhLVw+M8RRQQXFyLRVcQdWNAyuowAxQA6IpAWRm buXBcv/l/uUuw7LmQaaX7DItVV4GGBoQsbIq5jZmhgnqjYEfrEEpSEiV7DWFIEDiAVoPwAEc aC1b6gAZgMDf6kAKkEIK1OrzVnZ3e0P1esNOrkEqJEMqBK568+ojk8IHTEKLKievkkEq8Cop pEIq6OokWIF67+oaTEIKgIA2KGdOasM1CIMuXFd3ad9YNLizOXj26KOBgRdgENRC2SORKSRy bJRCYp8+XjV5JhQUSvV8RRR+zkYFTkWGB9RLnWRaZNT7BpS1gddzIGh70jj20XhLZSeI8uNV Y9DPzAYXsRSRZ1tHitGQE9GSGxV2TAIAXG2rTEF8r0Ey9K2A920YpwDirkH/GUyBNUtXGNOo XLfKGvzmZ4P2F/YkKYAjr4pDT6qBgCvn9R6nDqTCYh+CnZOCZZPCdu+kgKeCrNrq8kb2YkvX GqwAGcDCXS/6DJiCIo5CX86ymFTDKKqioRKqEoTA5G364+i2HNAp5TqmnjoCH4hCEbhAHUTV ldwmL2jjCDyDD5gCFZiCmYe5C6xBrZ539z6vfh9CfpPCBMCzfjuvDhxCCjhvKhxCdz8vd1/t BOQ3gTsvd+f3FBxC1LoAGXyAZav1IVhB1PJqtqsBrxaD0/7qJAS4Oq8BghtBQDnb9lDnVqxv tU0njuMCQeFChT1QQh3NOgQDutw7ZvSBEcgPjFeB/wtp6HihAiocFC4M9a/AzBjMGxuwwG8A RjrATFqIAhewQMngAht4QTpkQBGQy4ojaB9wwTRwAcMzQhUcgAtMQzpwOBc4XDEQwQsp5DoM AcnNgQvQGyOInwLUwTQUPFekjzRIQzHULCD0ARF4wBCIwATKQnkVwxCwAGqMwRxQwx4UgzRw NPYVlEtROEXN+EOBFyAgAhogQA/cKikoJ5fyKvgiZ3HqADoDayEkcvhOgT0jp5hjd+FOQbCy ynkjsn2fu3qnwCTgOikkw43BKH6ngAtAbzNMgLc/r/OCQCoge0/qgBW0KHbrgBj/PRWEPhdG n6OzghiAohLIwRGICSfeaf81sP6m96kcjEE1GCELUAMcWOFtxi0CmCoC6HXwYunwwkEPNAPy ozMphLHkP6822DeBVz57+yoZqEMik8Kvw3PUToHz+7rl73mc//qe4zcIWIE3TMCLcnf626p6 4zf16jcpqMEaZLvk962wP++4p8IHPO8kNIMNAIQKPoCiAAKEDp1BhAcZJjTYcKFChBMPRnxY 0CCudYjGwHGEC5GXj7hESUOFSNwnXFE04jKD5JMoQT2EMMKVkZoQXEV2uMiFj1uuKEdeddCI RkiSdWuErGtQwSjLlaIq5AKEi9qSm7i4itGakWAUPgfqlPpYBc6OqdKSbNW4Ri2uKj0cIVpz T5r/AmnF1kXxGwVRJi/F4qCKYqSUtD2L4+DKgGQPmmkntkSRI60DSzTR/hL0bNAvo7+jWxSU upLlug8ImpGZpCAZqTWkFLgG0dq1DtwgUkyRDYIUGRBTtJGZ0kxcM1LaqOhupg0EcBDatDVL 1WwSKevasveYTWa5i+CkJh0ilYKMGvIgyHhr5mJ4ihSTtPkmRep4qhTDta0BkW2FZIBpZgZW qDDlFBBYOKIaJTJ4MBIlxpDjiBDmcSGbaxDYkJcOeRmhQxt4McWUUiSQoBkqetAGludgWUOb MmabbTptUiBFh+B8uzE2/dAj48YpsguOjNnu04aUVFyr7xAyXACPFG+G/yRjAt+U9IaUD6zw zRssgUtBvx1B8AbIVNjzxjcQUsESy0mwnMKbVEhRjwwyUlGDFxXSwSghiiRqqKKFJhrUT4fQ QeQmz9YxAgl7DoPjkxaq8GIMv6jKZR13PDBinSMUyAURBVi4BZ81HFmnhZXWydSLaOSSZg2h cHmkgipWFcMakFwop4oqFFgiGDOu2GGdS3FpIY5pVj12HRiGsMYMLuaogg8ktkmHmjAS6fWK U3HhwySpNprklQNYqKMeo/oYwoQM6vhkHVmikKXYIYogq5542Rhimnk8EEqWTBQoQoEFVnrE gw5akOWExmSRZSVA6J3XL4LopXfZeZddFZGN0P9AgBUYlVyDDO4mAK9k/NQgoxmSUynDN9fw Q3K9ZpqZQg35wJtkDTDX0ME4ENZYIxkydNBPjTWsAy7nO11IZQ051xgyOPGGnGQCI8k45L3e 6jzEzOJScG4/m6mgohQ6OtwQAV4Q4IQXJz5UYYRSHAjjl1Fm6OGKHspYYY0pevCtZ21O/m4S NasG4UnZXAA7hamt0K/I/eQUGjgdhpYtFSu0UYO3QuQMTvGbk1GSZzJkzK5lNyOvY4rY/Suj DFhS0aG3MnC0AmbgpqAdGOOyAQGYbKagAphlskmxmWws6eGaZ8zASCGGICI0UIoMHfQhQFpA NbVVv/cLfIJSXUnetlr/igLVvsbnqoXv42fJL1lKo5/9vrxfZ6X423c/fqApTf/Qt6p5tYR/ q1Ig+Srml9JAzC+4kFcB88c/eiWhfusgCKokmD9V/eV+BeFKxSDGv5tQDBAGPKC85CUximlk YhyEoaoyAY8RwENEvBARDnXICzr4wAYjoIMN1EYHI36IiESEhyngQQcmlsIUSGBiFEcBRTiM Aouj6FsYNKABEKwIFv0hWjL6owY1cEcd6jBcGidhRnWgQR1qSKMY4KgOMbijBOoogTv4WII9 ugMKJYgEFAgJhUj4IRKJ9EM1IlSNajzCF46sRCSr0QBRPAKS1RAFDCYJAzPAAJSVMIIRQCmF /z5gAwZ9gAE2+oAPbGCDB6/kASJi+coW8ACX2KgCInjZS14urAW/jF/8HiYLRCzsmMC0Hw8e hkwFBvOZwQRmMptJTGB+z37f6wtpQjhM+0VhYeiIQp8qIhFCaQ+d5+zTOsWZsYyNz3v3459U UjUaXIxBEIj4y8buh5p1HGAIJ9iDF3ZAvvhJcJjsK009uZIJeK1jEjo5VgRVSD6NzBOCI9RI CKMgmhQu7KIdvajFFApBcK7qYhrxQhw6sI4OhOMUfIjYRW/CQq5ctDRhkUpnCKKRA9YvYhQT zcNwMVRsnoYrJuxpUfeJmpbclH4r4YpN3DLVgjDiKldJqkaZas+Kof/mL+s7oQRZyFHTjKaB o9GpU88aVrPib6ydkStUPwMaceoULKbRKlbHiRG1prWvnjHNXf2KV7+IE7F/EWdBCCsVj46z InzqHp8EVdntWdaygTJnOsvZURFyRRRFMeYkPrKOOtQBVJnwQFvC0gExaCEziKhDUzIF2wgG 1QJjqMNHPoOLVCwhXo84hVrWgYZyaMQrEowoLvrgBQtEobn2WMeuUCUKV/lVFGzwizu0Eiq1 hOAEfOkAaXFhBLUgwh3TSIJdKoMLF7ChpvgriF0MsYQiFIMbY7ipGKaxkj7AYQuAkMMcTCiG YvilD1XIRS64ktWMiGZessirQfha2Ickqq7/FwEEXzeM1Qx7WMMOvnBWTcNhCnd0wyJ1MIbP 6mC9lliqXxUsaBp4kc8mqsVh0TH+JhsxwYrwM0AGDUEIC5aEFDawdz0yZO3q1wYudsaIXSeT DyIvyPYVsTa2XjkRclhAFYp7m+UsOdmJWRpf9S9UQYVGUqES2X6CJZMiVgLXQQ0ttOURr2Dw YURiwk5tA4Oy+K8jIgiYd6EKBrVaVVZuNQ1eusAaSRCFFwxzTyNIigerGsM9qkA/REAaVe7Q wvcQkQo5FAMRZpgGNeBlF1e1QNSqYvUHCloQ8El1I3VYwF9kuwClUKMYGnnJOdZxGcMUly/+ lCpXNfbNq1i5fvWD/zZaMXzmHt9UYmCdsU4Le8J+0i+nQQYyS7RKsbMuFNroaCFLIBZj00ws rQsDZ1i+eT/PLBSt+EZrvkH4zRHOS5tq5XZY47XNg7DP3It9mLkBbu/6mXWxLVA3OkpDcVmI c+F9HQ3FFR7tLo8zIS2oMsjBzM7sffkh3Ctzl4MKYVzg4xNjQIURLLCOKjAlHSEg7rI0ggoL 5GLNuEAFH86BCqXY0+Z86AMq+gKStkBTVdvM30k3pkBc3AIRSXgf+xCVVHFZXX7sO3ZfyB5h nivQCHWouVOnKpplX1SB28R1FLRuBAVEowpjAAEb+gIxj4LUxvL+i7ya/k37tRBiNobwRf+s 3MH6tYURGNNwxqlnE9BIlV5bAfFZ0Ue9nm5TfDUGRFSAWsCV2ASEYL1JCG9dYxG+dd8Vi/iz F3o/f8MwrPhDfLRBLm4JTxvhJKVetA++bidrnLAVPyzyFzuoi89rIs9XZ0S2t+Xrqfyc1QvN bcm3TYYmcHwVXcctyN5Pf5I/4PJ1IAibzr+enm8dwWgp+5b9vlWFpenghCr49I+KI9gK9xSq gvJP7r7t9K4iCtYs7k4IfORuNKRuHeahUsxnn9DKfKzMhFBDp2ShEvhCNNzJgAyIpkIK3LTK yjIgHd7PojSP9xTq3ySLAkUI4cBpn8DHrGxP3+7ne0JgDFYlDor/IH6qQA5y4RYsigaXTwYl LvXehwSP0HyqoA8myqBSZYDIB1P06d1o0OEwSP9SBWO+BxVKY7q2oTSaiw8QIRiq4Jvuig9A oAoIwhrqYPnQoQqmwQWC4S9uYZSK8MjWwQzG4NIMwgjQgAjMQAm7LJ2yj/oE5Xogwnoowqr0 BxeCYQziwAj44LkAIQnSgQu2wQLUpwq24QAcISswaNk8CyuWIAOIYAksgCuqAFpYIBGM4ghc oBjKwTCiYAzEQRrE4SOSgCiMrgqIgA/IjcLqLBpMoA7s4Z7qwAOGgAK4Ih3qoBgUoBiEYh18 UejmoRhOQAHKYR3YoBhKYQ587VwyYQ9M/wIkpMEFpqEe+KUD2GAPSqEYPqJiJkUawiEcWmUl LGAN6oEN6sEQqkAjFABe0ADOrsIFWGBRKKUPTkBZ1qEYQqAPQsC5AOMK6iAciC0dkMAacgEG +BErfKIK4oAbUEEWqsUaqgAV0ODOJHIMOqC5BAEXOkAyUHJSiG3SUMHDoqAKQGAMZKEDXEC/ +MALzgEk0EAEuCIO6oDp/sthQqAO+OcIYiopIw0V6oAhcSEk+aC4DqAvrGtVisAaUiUT+CIB QeAj0sG5VmIMLGHNMkAc+kK2xuDHFisKFnIlzuEi1+EHVyUTpsEvDsAFuFAiueAWuGIMvIAg o2AH4OC5FjMXz/+KDVxgUaYhDo6gBQ4ALE0rKIuLC/wCLaYnCthAHJTvwj7hBIaADLEsCljA BSSODRhznCblCKaLGppODqYyCqLBBeZQFv4BEROL5LDvESurehJRq3ChLVZlHoBrVT5AWRCj GPrFC6wBETLAAxSIu1rqe2KMK6jBGlrqFophAdbBDFfCLu7htLxgIJOgDzLABergHlxRFJ7B CypgJOqQCIhgGlrqA5ZAfTLAE9pxD07AUUpyCOaACxQQBFRiHfgADfaiB5PACOCAGdchHdox BCDFJx+UK4xyzfABQ29KNOZCLaLgHIYLESYhV1giAzDDtK4rAxYAFVxALZLABVzlKlL/ZSit 4Rz6IAyKcSngbF6M0laSYAy4oQOM4B74AIOWoinOAQ5kKjv5ogrqgFcmZQyU4gSAyzHmAFHQ U4NGaCe5IheMIAncIdnuidL88j8lBS5C1AsQhSoI0jGKoQNINB3o8kFbYOm+ZzZrTjz5JwPQ EhXWoEvZsjEZNQkOleyIcv0cqDT0UkW9IBj4Jw7QgH3igAj45wA+tTRulAvaxyiNQCMWE1FC QAcqDX+KxQUmYQuq4ATQIDdXZQ0ygH2oYRqC0AvYoDREwSkplQ/CYE+KjTY1ThRiM4WK4QAQ xV2OyQi84AzTrlIsQxo4RbEkjjgVyxFPziG4DFwNZWOYM5gm/80R8q4UtGAdeMArLCDtTsUI hiAOguEIwiDZYIAIMIWm3AEJ4oAPqGEOUAEVJiUdUAMG5iAOxiAVFqADRAEOuCADBmoJqkAU 7uEcOqAKhoAhxWdZcKV9UIFN7ws7W+AIpMEOpQFeUKHNUKEP9qAOiEAa6qFXqsAFpGEl68EL xGEaVOsM0QC+quAcioEladU8FQgR5mIa9mAOpuEkVoUP9sAD9qAuEAEVPoFY7LQCziF8zmEB pmEBqkDrQiATyuEc0uETmE4UcgGbEIENYCAOooEZmTMJLOAToiEdsi4BzSCB+qAPWqoPRGEH eoXYNmLpOmYdckEUkDI1RmgjcqEKOv+ghB63D4oQ/6KA/G4q7hABYvugY8Ynf75znhSIC2uQ 7BLTp7pP9Y4FrObnCF9XrhCOBMvO+0a3rVKjCs3nfOTt8FKjCwfvftTtsLhV4irO4SAOyzhu eLvs4oi348Sp4iaOW0GO+QxFs7AnOcvpT1J3HZiue6HgtA6tBZLgFtSHfCaTJeTH6FJFfK4u MTMw7uapBco3fOiWfd/3pMRwHTAIdMdvm8gvfMhv/C6qLSpIVepp6taHNE6KpkI3dDeqe7sO p1T3gKMOcZuOdecpfOqX/oxCheYpqxrs7erX/tyvezf43z7QcUFvzfQvdl1vNGzifP5NhVgP /FDjO30Mduv/qYYZyAZn0IReV+r+QoZ1EHyqoAgqQ369T4jjgCG/ihr44DuDiQg4BRHSQQxp 8BwEwYE0eP2sDAZ484UVC+DasNNu18bwEi8pkHmLsyFugsy2jMzKTLMagueEblWOYA48OH5v aoVRBYEHcJ6SqnxzWBQI7LYUiPxM0XG9l+mYkxoagyuUYn/PrlzhzqKU4o9NyOhoSn2iLnO5 jzltbkxX5RaqoAptLg3pUi5sBalsLoEIkuY6Rj35gG814gQ6du2aVAFz4QhyThyS4CrMAAXL NO/iYENVYqxUhTnHYBrYoA6MgissoAiW4AgusgqOAA0OQC6ViiVEgQgU4AAqLQNY/4AaYva5 FoUXp6HTpEoOvOAeimHNWKAai6EYKwYViABPd4AIOgAV7gtmZWodTkC/cOEAoiHXyCIa5lcj 6DkO9qAIjoUaWEAj4mAbNKIYiMAFPkEOPlUjXCAdOkAB0iEXhiAYFLMI/NIa+KAeLHMdMqEx MCIJ0OAIECEOTgARRPoM2QAO2IAgbeKlQMArs3QMEGEHpAEfcAGajaADTuBNiXEdPmEewrEI /CIXphIQcmENDIPIJIt6rfeutM/LDo4Rv6ysUa57qqecuKp7ScIQNNYFLCA7l0BjdWBlXSBd xQBekmAe4CUOHCFNgyIJlOAVtkDruMAaXCpGMbhTpKu5tv9gHTIAtpKgEhKmCoDhI240Dj75 5qSrCjLBAoZu0c5zKRxBKT4tGowiK3NB0vR0ukx7DOR5UQa2A9YgQ8VThJdiB5JAFiygpVal Cg4g2MygAmTukeXAA2yZC/RUczf6WzBDLigtS0vrt5GiIZluUragRU8FGPcYBu6hHg5gGrwA JE8SJE7AVTQKFcdgDoI0DkwiCeIg0qIAG7xg6VKhtNiAwJYNIdWTWgf6E9oiAxJ7DEphCIqh DvZA9Tp60RpjI0xFg6uAaPmACDpmo9lnUvgWDSgaK1ylCsRBpuzpFsYgHTuWGraBfRDb0EKc JP5zHvjAmIrhYLOUD0KgVCP7vWb/01asgQuEOYLQwAXe24rroE9RAVMnKlWEtXtDQBxs5aWL ABFOgAyb5SSJugqg2TD4YBr6wDJBozCWT/lir2LOrFvJ2q/CNa0tYp3+JOW6Bx2SqpQRgQ+m ZdJY2w9ga2MPtrh6EBXiNhPS9eZi0gy8YAtQYcBWBRXmoTDWYQyGIOjQc7cQ4UKLTrL5J2Fd dgj6tAqMa2Om9RysdhL4YBV5ib96xVSYE7WTqiUzQFnWC68ZfVoiFAQK1iCTVjpb6lgQYcM3 Aq9FgRtsJQ5QTRxtxddz4RzqwFaUQFluStd78JBtZWNrDiKFNiU+Gxep9Awz4EmLbV9tlrw3 PQ6qgJ4j/5cwxB0OEDq4t2CGD1kBw5HS1oUFlrVKSUIajmAMFIDQ9xckUOEbaxEcB3obVkUO nPYWIFIuC/rqOMIL0MC30aAYzjkOVkV9bK4Y0CAMba4exIEIdosrHEEasmAapsHDXcAe5ld9 WoAPELwOIr68PKAOwDbiUeEAvBKQtRwpuxdgxbAcCjocp0EL0BZeRIHQKmgHUDUKRCEm2aDm IpQNhOKkhk4UvFcUwqGoxacPxiAE4BXR2UAUhLZw08GjrQyVJxWcWM+ncoqMabDvhuwg7C2t 377N1RqsLwxjUAppha4PgG6XlCIJmtOEqmAedsA7EZ3wNTl+JZjZTxjRPYiS4f+8e+OXkluq Azom8uPXkhG9D7jAwZelD/hH685ODvYg51ygcBNIc0fXz6R5VTKDEVyK2f24qDoA1y9II2h5 hLLu9G8KY7LO6lQlSvXWFFu/LX77gs6HESCP59qtqDIv96UC+FG/qJKKr0SQfjKGfZRfgrtO 0o5AYUDvgwRr8f4Nfmx37tiXohoMYpjfdz8vgrpuBsnNCDvoouTlfTIOrCwopPKPXsLPfpgl /dMfINYhQjSmyrp1UXDJkiWwRZQoCxcqZHiwBUOLBwUaRAgIkCxcuB5G6UiypMmT6DqmPAkI 1zqGSaIcRISqCqIk66roXJcE5JYkPXExAgnyo0YzZiz/ZFzXAdG6PhYMguxgZowgfAermBFl Jl06MxurjOESJ53NJIiOcDGRIRgikAfh4urJM+E6kDLvutTLlGhcnG+jDJ37li7fuwdbuhQq d7Fcu3pxoerZ4i4jnJHXDR1ad29Il3bfyuWs9+1gxRkXR948+OPQhYiFbpaVkJFtWUlaInwY klFCxoNxdfS9eGjMxoASJu+YpEVIxS5bSme8ruPz5CBbJnQYUiTez9bhfhSqnKj1kQnLpxfO CBCj3R9b8nkGxwiic/eejVnHx6wRMx2IckQVH4nFRQZGZGREBkW4U8ktuBVVDxJxuMBGPXt0 sM4eSIwhSzjPUCMLSQrhEk4p/0cgcstLjHRwTymiyLJAKUSMWFJ7KwGyUo4qmbRjjzqik5Jg jBHVBxylPLNEBxk8o0VPVUzjxRBDRAOUbUGBJAoc0ohRCh4dwMUCEpM844GGLjxTDy6V3FNB LiABlQESS+IiSikeoHKnNB2cU8o956wTRymGwLnZdNMVOVRLmwnlHpFE2WYbeqw5CpJtsv0m G3ntMcaop+7Ndal7pGIqqWKSSjpqpe35RqpspL7aanuhPipdo8Ml56pQmoV6GaeIXkreSK0y yihxLLnqm22xstdpsdJF6x6zJBVLq47V5notSydtO5y3yYYaBR9IlBLGEAqcEGgHe5Syn0su lPIJLv9sIFGMUxvVQ8cp0oRhijSItDeSlkPA80wxGhLxTBx9ZPIMEcKVJBMPCPVRLh+5wOGJ KLgMCjG32AqJLZBApvTjSZt+WgUwSOwgxj314MkUCPfY08KcSxDllxFelOKCC6OcskAHRwht BDWlzGFEB2gouY4onngBp2R1IGEPY+48k2EuPXjCxzrn7DGENO16kKWwvpJKLLOOCrbooiOh Klymh86t6nSd2sqerKkCtzfb1lI767TMFp5t4LTO6iitmZKUhLW1xWqdtsOlzHi0hce0qKKb Jn645NnGnSvI2ZJMeqmnl36tt+B2lDEc5/D2ErulhAARu6OwAHYYcHCxxjP/cshSLxwZxFuE LO0dFA4SXvQwRClr9NHBAkggUccpSGTQ3hj3zBPSDnAg0QP2JwwUjielIAkHDCyZbDLJIgvJ o0o57libqbhUMQQcjuSvwD1nwkUI4DAED4RBGucASRyGUI6nDCEMjoDSGrwwr6DIwQvFiEkV 0HAPONxjDjsgCjW8sIAirSMT+wPgDoBihD14wQsKKOHm8gY3tcUKV29D1LMitrjwvG1vhGvc 3iwlMEzZzT09sZasSjVETOVGcKfiFtsGZqyIoY5ygbvR6FL1rRtCizzVAg6yeihFaN2wWyZp XbjUSDpqmS51CYlIdibil0wphCMPOQh64tORiFhH/2AigYgVTQKRkfjRI7IQUiIXEgUevY9+ kOxR/Xb0PvkJyW9+C85g1qMz/P2qPEOpzWfa9psq1mqOVLTUKbFDnDkWzoqJA87kCDeYZSnr bZOaS+WMJRgdfrGL03rc3LD4qWlNS4hu7KHh3AjMxHHxjKXTludQtzppUvNGrKPl6KgpuDSi EXQoKUn9wMktkYlzZNuEo0gk0pFCooM27SwkRODJSHmK00YkSWTcCpnPktATnn0ciZAeQsmQ /eiRB52k+yxpzh0+R5S5hCh7JNopuw3zUJSK2KSaJbBbPcuI+CumqbT4x7/JRlKCIZZHI1Ye 9jyHh750aShTpapqei6Ks/9aZrWgtcws7jR1ZmRjF3VKzG2a8adelCI5z5m6pgoVnSE73bXm 90alnvNkTfWRJEk3TpI4MkiVjN/IFlrQSkaVoe87jeXkNrlhanSHN8Tk4I4oS7UJ7qSnWtwz ocnTUvXVmYRD6rdaRaRvDs6mQfWpUKupRcCmcXBXzKo3Jde6m3Zrmlt8ajgly9n2XfV08+uq WLMaWq2iUbSbDZJXuwq/g4LVoJaU5Gjdp1q0opUk6UEPehSjUt0G8qHeya1ulyOS5IjEIeuM gnPkyUjaWESe0KVNIKXrEH0SFJGBtAhtpAsIjORxux4p5Hbn+U6IlPchzUWvctErXUY6pDJO mef/dmXRguqitwWIoO9CKlNfl7z3vXnZTXZlApJbhMQ59a2vfvWbl/zWdx0Kfol+nxsRCL8E vxeBMG+U6xALK3jCC44wRSoDkZdYuCIR4S+EK7Pi9f5XuRomMC463GLlNge6x70Fco/L40D6 +MeB/Ac6gttIgkbhHwRtATr+oeT6KhcdFrHIPxLZAiZP2SJQfuc7m5zkISv5uE2ur5eHTFD6 gtW2DKWfbVXL5jOnp5025A1RqLjP3T4klLoSjnLiaVwyT1ee/22vkdULz3kaV7rkHbRIyNxe LedRvuscskKYC+QnF1mfzh3yeBWs3cpEhL7HvUiKq/vcvLC4xRRer6oD/3kXDs+TwrBJ8W4k fJAJQ3i7Jq7uhVksExg/OCOk/vB7ZU3fl0AkyhfptK0nDOBhRxm5Fp7xjIetXLy42MkArrS2 f0zmbvsYuX7OMpapLO4s01fLi1QylenL5C1HpJH29TO4d3xcISmZrGjOt/yEfFuG7ta4D4Gz KgNuZ+NiZ7jDFQ6ZQ+Ln5A5YvO8cb3Kt6+Pxcne+512IIiuu3keLd5EdNzbIuXxsiM93vZ9m CIMxQt+J6NfBGl7Kih88Yw7T+9v+he+CV7zyKOcaJMt9L7ARouxsX7jYAM71gvd76wzvGiNN fzbPkf1cqWObIS5+yItbgAqtr5rb2w47kh8ydv9FD7nc5E57k69cZTGTW76OfvK5mxzub9dd kfkW8pnjJ9Y1y8/NxcWt4H+z3uQgdzkAJ3idEy/ghmvdz9T1Mj3VW2raKETT4jUvkBHdcczv GNGTRm56Jf7jL1u85NptL2weXOxaI2TpwOYvaE7da1f3mMd6ZIhTHoyIBKdcIBQBjcqjnJfV gzrGTtEjjQ8886M7WNepfvZ+e19f6j9buczmNEYSzH2vj/rY4BbJ2MsO5LHXvciWLrKXldvu FJt7y+JepMarvG50y7/9qT+7qv3cbnsfuZEjom+x9XdmxW98h2/0tWNDwhFzs1x64XXJIWB3 YXgy4WQ9lmXe93n/RWr/+tVi1LVd9nVuqudbgjZiifaB+5UX6zSCIMdemVdP2DdPH4di2Kdr r7dfnJZhwuccwFZzObdeFwYZNXdsFJF8HvZ0R6hsQQh8JoZ1qpaAt8dhOAhrrid9W0Z1EwZl FMZ9IMZhzed7CUhv6gZqyIVc5Bd2jvdjVPZqNeho9dVulhZm8Ed3Iph2UyZudPeGPlZuXwZl SWaAr9VvfLd3aUaIaUhcBOccnwFk4NZdPraIWmdcDoF5mhdpMch5gqZ1zvVnmbdOyTEiJYeJ 4vVl30Z55qVxoAZp2xVxkrdOvodsIEgbTQhi/JVs/CV7VVdiFXhgZEhgw+Zfurd7zjUTt6hd /xd2dMd4g1lXa8QHFwIWR8jmYrZWayVWXaNWhoWUg9z3XKiGg+8WhnyIgXsYhQ2nf0kmd+Go XuimbnOIitoVcfVHh2Poh+d2bN6Gfm23ZO91h393VmimZgL4j+inaBvmHcA1XNjRHd2xXIGU eN21Y9Y2cZQoXRLGXNg2idt2egupaZBHhB3ZiSBYXmWYifX2adYoC/llfKdGa8B2fHqEECeW Ec61F062GzoWcxVYhDwHk1GACkfIjMkna1NXi/wlT7FRe8WXYL12dLDmXMPoc1yodRt4hb7m ZGHYbGM2d+KogAMZSI7nbQnIedm1fhw2ZmImZel2blqmbls4buaGZf9j2JXn2HBIxo8BOYh3 iWYE2XDdkXiVJom+5XgPhZE31x0O94nZdYOIZpWTOHof+RGeRnALEV4fyGMliG3ZhYV013EM xnRhSZSwWBEaJnQrkhEwNprM5xC3sIyi6RKtxwO1iBDNIXMV4YVEJxNMd2Ksd4vy1B259orQ 5nPuRYTbR4ZQuHXOFobGyWlaeJZsuZZk+X/2Zpjqt2iLtl+0UY5Pxo5r95zyVY9naW7gN49U Jof5CG+OxmRFhmSaxp4JpVAEyFAGGJA3hx7amYaBmZByRmTahmC/9WMHVorbCY+vRom/ZV9S eXrbF3Ek1mvSBX8sKH2kWJmamGDUF2KFVGv/IJh6m0ibp0Zd/gWTo8kdHeZpw6di0AZhA7Ei CMaEI7liXSdzp3aRMXdiYcl0PndrwbZ648lsOHhtEhqNCQaPUKZP4oZj1Ymf3OaH4UeYIplu 77dIViYLdyiDIpiHUdpcmUZu99ZIcXd2XmaXa2aI++Z3+WZk4WaOihdw+jd4PiZw2mFwCJeQ vlFcXsmJpCcTfqiJRfaQDmdpJ2eJ6mVsK1hejTaK4omgbFiEsDh8OnqLdciTKgZsS4kYEIYL iGBzM7ZiX0ibMEljcTGjDcqpRjh1vRdgUsl9FGN9oPpgr+hh2/iKCQirNneV3LenSrmh91WV 9Rl2xNWcFhhqJeeF/5hJhrF6ixWBX+vAA9QHmywGmzzgrMzKrDZxC7dQEzoRDFWADdpaBd26 rcFApkuWZvOZb2cGeEPSkAOZHCKzT8SFiAEHr24lp+8qY9rhW/iajRPXcda5gn9qd5VGihz3 p1pqjZInsEi3YK/mXlRHfFh3kQ+GCjSRE8sqsTxQE9haBdgaDFERDDygE1GRrRZgAUZAsiNr BPhwDhZwDkbQsv/BB13BB/2RDnxwBGwwBjZrFTg7BiEwBj7rsxkwBiwQDXEQDRnAAnEQB2Rx ANawDWRhDSZQDyZgDUUwDVW7BNQwDdNABESwtVyLBtNQDC5QDEQwtuJQDHUwB+KQtjrADf9e YAncAAIv5LYV8EIddA/3gATos7fPoLel4AnPELiCC7icELjwAASckLiJ6wSK6wROAA9O8AzX 4ATX8AyJiwCOew2cwLjXgLkIsLlO4LmcgACkG7qli7mm67mke7pOAACk67qrywmdS7o24AS8 sLqpi7nXwAuhO7o2gAA2AA/wgADwUAf3Vq7jmpcHeKY9aaf8uRyS5q6+lXhzwxsHp1t6hh0M +VCG6ZeWCIOal4lOlnGPZo0E+4ScmL7opWnup27Z6JmopmDeB2rLuRQ0ZgR0MLnXMLuO27un e7uwe7urK8AI4LoEnLu8sLvAe7u+a7sIcLtO8Lu/C7m8wAvCa8H/8EAHwjsCGZzBpkAHpWAK PuADvJAk++sETwAKT5DC0BAPoAAKQbABMQzDQRAEoKAH0OAJv0AFpfBCVPDDXkAF2QAL2dAM P0zEzTAFrFAGzQACKwALzUAKIEAG2tAMaxDFatADVTwJU2DF2jAJV0AGk7AGa6ANajAJqbAG ySDGaUwGZAACH3AI2jAF2vABXTwFazAJ2gACayDFc5wKpKAD2kAGqXAIIEAKE5AChzAFk3AI pEAKU+DIa5AChOwCjTwBZOACpPAB3pDJk0AKLtDHh5DJfezGHzDJ3uANqVDIpEAGKcDJj1zI 3qADqzwB3uAFCAAC65Bm8klJAiiu5hSv/0AGZ4tnbfHEn+CheHARrFr3UtZLcN4LZMlKaPyK Xs8IjTxQmoK2DtoaDG7BSFrhsi+BC/hgBgmCpy44XyyGEU4xfDvnlJ12mqf5FHTgBFE8CTpA BslACmsAAmdcymncz8ngAl/cymM8xlv8z1OgBmTQx5MwCcnwxYDsxpMADGswBY9cB4f8yFLc yqTgDY8M0pBMCocgy2QQy94wBcsQA0/QBWCwCjfQDT+wCt3QDbXwA3ZQ0zVdC4rQ0zR9CZsQ ClkQyaSQCi6QAqTwxVmQCikAAoacCoTcx4CcAqmgDfxM1WTgyA1NCp+M0Z+syYRMBmgA1YA8 CS4wBanwyZ881v+IDMqr7A0lrclwbQWHgNSRXNIpYAVwDchZQApITciA3NapQNSkkAUToA4e 3cipgMkfAMoTMAFfPQFF/cif3NiPXNKADNlZMAneoA5qUMh4fAVOoMuwxcuCuLynPSTEhQuz UAEVcA+vgAoLUQU6cA+e4AnF8BJmkLeeUD3VQwQJcjN/uwNEB20mYLiWew+fIKwsFhJN92N3 0V8V9gjP4Ake8m58KkgxJ2hyELhsoKEWIQqQy7hLMBNroLjPkAsoOQmeMAQd8JTQRaivRo3K xmAf12lRdmCfcRB9QAe8INIpkAKYnAJTQNjesM+Q3MlrANWNXQYf/QGT0AwUPQlqQNL/X3zS dHwIaAzIFc7IfTzILmAFzTDITD3iVk0KUdzRaI3iINAEntAF8QAO3XAJq9AI3ZDTPk0C3aAI drAKilALO77jJJAHq3AJYEAMRp0KVoDILiAOej0BRq3X3lAIH0DVU2AFH2AFOgDlgLzKpKDX hHwIUi7gJC3ZKf0BUI7Jmlzhkh3HqSDSaFzSn7zJXp7Vbx3mS17Udc3UJE3VRc3QG60DffzI 4vDQhawDXO3XcHwIiE4GH9DGRf3QJU3LbJzPRZ0K+UwGaO0FvOAFiBCfshVWeJnatHUdDyEG wgAESPAM0XAQR2Aw8MAJr1AxzwAPHnAO5xACYQAPnnAOLeAO/4U7Lx7nDogrBE7hMJ7gCDCW HgIRqqZmczEWBSrCzhSjXBp7jD55YdgwsxbAaXLA62MQgi3wCJq7B7ze7TBgMArACUDAB/RV s+lAdOkwBkYgFueACILAAnxQBSwQKIjAB0XgAnNABGOwe33ABdbwH9NgASknXyQGffTMC4g+ yH6dAmasDX18yG48BZq+Bh1fBqXM8UW9z2qc1JMAAsnAyJ/cyWMM1STt1EVN4JJN1ZQ8BSmQ BZ3M8TpgBTbfDF0ADV3wAmDQDTS9Cj+g0yQA5DlNAiTwBYqw0zVtB0+v4zxODmAQCi4Q2Gmt A68c2N7w6Cnw9UuN0aRQCHvu14As6P+P3MSIntSEzM+TcNJVrQ4XzeCD/ciPfuKHoMmOnOir XMglvdSIbAUuoANT4A0LTuCpbOUfbQUTsOKHPwXiEPYf8MbikAyHMMofnfUgrceH0MmtjNiP TNSKXNcXfdQ6cAUIAAwtMFb/OIDx+csicx0uQQ1AAATU8AwVYBCpwAknUArwwA1dZwSeIAy5 PWNiwAnw8AlVkAHXAATLvR0PoQTP4ASv8DWIwAZsoN7r8AG63wEtkPue0Act0AelwAnTEAb6 MQlA4AlG0AIwALifsA4NYLnhLt5A0OrrAALwcC+VYLkAUUEBEiel+ERpEQLeMzbr1kWRJauF KHhOqLngFAf/ERp4JyYheHbOoQsnnqrIwlbKyT14QMZUGeUECSd4YxB9gOcpQ4gw8KQhmgiP 0zN4SM61QCQxYgumSpniYoqozzNeZCaASJXK2xQyqUCkmJIsRYpmXKdMATFlQgoyKbSm8JaM lDcypJJNIVV3K6kJqQ4d0kaGzCFSaadMGjs31VxSpNaATWEFRDNgqrooe4Ehj508imp160aC xCpFohX9UESvVmlFnlu3Fv0lT7fPGC6F+pA1RWMdflOlOJTVitu+unensqJ1yiErVryRGuuc lFZvi7VOJ6XjOYjGLqwKbhycDIhD6lJx/RCc1KFJVtbkTrWGVHv3061kkd/4Nym8/4Wcp3jM imYmAaEt+VLRZo3FSGlGgbQcm2QNMrRx4T3wQPigMfkIy8aJbFpAB5AQ0QlRxBFNNHFEEldk sUVAUgQkiiheRIgaJxia5JlPzKDDEzZMceKVKqIw4plruGGBBZw4qaCKdfwwkoIocIEKlyjW EWUIeEAiwgwZr/wAiCZbEMMJYbaQpQqhnlmAjT4eISoaRKjh5J4qEopzHUSucOInihhaB4Zw 5KgCET+uuSaOdWSphqgxWngIoShgeMYJa0R5pgKK4OmDJGH4WKcFjDzpY50qVLITEVNPcaKC XELtoxROpgHKHXjo4AMRI4SaxkmJmmpBqaUgDRZSXenghf+KvLh7rrEJDEyLjGZSKUsbvCa7 Ai1SylhDmzKsnWTCvFKRdpLGSNGGsLMmUUcHcEmpsK41XNBhihTg4MayFy4BDTRF+q1llX5F W62b02oh4V87fiDBjtIQdiXhbvIgwZVaLhFHBwZzG2sC88A6JDf9UnHBLckQm05j6OjNq7HI BFsMr+m2Egc6/rIiY43HyEVjjSnUIGMSHcxVg+RDuEshw7GSjiy4xVIoma0pUtFBY+YKS2Et as+by61xvXHLm0O2SqUvbyZobLDfJtHmA43rqoMXOG4pse4UW1zxbrz3XnHKL6c8wAkn2Dji mlem4cQafFQI0skiOQFiiCHqoIb/D1RaiCIDIJ6JBheHvnQIkSoyQAIITp7YAUwgPHnVRk9E wqeUa5awEtJUOJGmj2wSf2jTztepZqF0xHjGC1QkMiKLZ54Jw4lrPnGoUU4ceUgiGWG45hlr OpiEk2JmRUQdJ4DYAdJJcrq8Cjiu0cKhFng4xfvqT3ViDz0zuHGHQIWKhinMIQ3VopjyuVD5 bx34QNa0FvOsCNULBChz2VnWAAIrJMMbIChEgc4yFq40YzwVnII3miGhxSSDO+IhhRq8oZ1D kIErLjhECsoAh2V0ARrKAMMlUACaWnxGEQLr4b8M1hnQCKwbPzCYHfrVDSWKphsCe1gPXeEK cMxBBzqr/9BkggYC72hrHvw5zCF0cJxU1CwL0LHKGPOCmPOMzS3AwcturNA1xWzlEIfJC2Em oRVzOWYCLiCX2j5ABm+0J0PxIYXSuoWd+MiHDFngjs6y0wx0YSVBjmHLHSkJnUWexwWNqYM2 0iaOHjhhbi+yG95EpDe+5c2VIvpSjCDiByCQDxEgqKUntlCF+L0iCVGQwjOA4IFIyVJGctgc BahkpRhxwROecISMKiHMV3RgHR+4gCdQgQso3GgLuDgVJ5YgCytFQRTXAAI1rvG6KWGKE9GA CCJ6gAAiwIETC1gHLuQglDGsoxLCeKdDCgcP6EHEb5iCRz3WIQdOjI8PZMrfSP+Ewbo0lUIY WgBKFKqwu/bByp7FQAQiMpA9UMGgJYpyyOeK5T+EAFBVUIkVAsowhQ3tRhtq2CNNJzG1C5bF XGgZYTbKQIoMnuU3IEiGtAxEhmzcrA7NYIvOKLmGEaphgjOIQQye8AJyRACK3fgCGH6Amicy UYj/UkQeCobEWvzAFbRZmME+I5o8fGE1CPOXZ7pxCY1NoWpWQFsdwKK0cAFyjoWMWWHSM5YB FQY7hdmju7SBHCuI8jfu6qPX2NKVETamPi2cTyIVxBdS1KGP3nCBOMg1BUC2bEHnGht0nAO2 9rCFOxTEy2WtshWNrYVsR1PLYtaAgGysQ5UkutsqX+n/ylb2DRcxeq4sNAcER8hiDKbTgiwQ AYdnVDMKfUDCM4rxS0b8LQohEGbnmCmjIwhzGttsLyeIuQ4xDOUcNxnfDnARKyBMQxZfQoQC gOAFICwhUpgCwgJktFBONPgJJ1mHOyDHAiPMw3TRcAh6dfSQFuDiv5QSp6kE7AFVlQkIfIjI B4aSizRxtxgcrkIP5CcLhxCvIwdQiSGAAgNPJFhVeooIRAYYEYk8xJoOgQkArEqG3q6hB40R 4RomMdOgZacMSQXBU8/CHXsViD8UvAtkQojlFKxgLiDQQTLCRZllxKMLXAUDwH7Qw7Yu0WA8 7AYyFPHWhwHMFbOZog8rlrDW/3wGYWlNax7yQA4OiAOqnHTh1gLUGLa54CvkoleB5rWYukBt N8yJ4XCsMDbyROaCvwFOVwIJNnJ9AGx2ceNzglMdwpQtOHhkD4SmgAYNTQANnDbXb6y6m65I ppLUagwIsjBCa5GiB/FRQ4KagVMQZOMac9ubclGEohPxjZXInVG4M7c8R+CCB2F4RuoQ4YVn cMNK4PXEeP0WoxiNwRP3gGe40SGjMWhJmEXB54xgcI9neGII9fCEF+zRgioM4R7TKGc7JTcE FEMKBgoYwicgoicFeMADQhDgOsJRASR4gA1oKEY/W8CHE9QDxQKUUR8WYI2GtMAICziILEQR jnAMaf9S4fjENhHBhhCIJFjrOAcbRLLxdfRBDvVYQCbuGypEnKMPTvrxophewABCRU9NhwMA qFAGTqcgGWoAQSEZBLTGUPIQ8iHLFLSBGBDAYkBfESV0NK3Bx7grLykAgc7KAIwKvEAV5ODX bFbTCjvg9V8EmxgJ6DyxWsxGrZ8JzWfy4IrSgGY2dogY5j/DsMyL5q03uAQH6lKWE55lWm+H WV34k4JJ4BQvsp8A7KeQGyuEcI/kmo5fqsOs38u+y+MpjM4kmYpJsG08g2mZC/ZYl0l4Yw1F S9sHhJZCQH7yLx84wTzEoP2n7cYtcYQOWsBCFwqa2V5RvUYTbqFc5LJolaz/xP+3V2QlKplX RjEipy+xkkiRkXIyJr8ZwBb4pViSJVzIKIP6EswRsgj0Gwk8QBmBlAIkwAgMFUmRFPdBiP+y nmCRERFkCggkwQXrwOopwRIkQQ6TwBbEnCJznwAKFhqjsQISQRwEihO0QYfAhnmiAhDQhmYQ h6DxGTLAqTUog2aYu8lwDGmRMgHxhiKklu2glmd5I7aYDibcjQw6BVVQBRxqBDtQoh9iIjto hUa4MzRcBRKgGNo4qyOKmD2rBYt5q4Shq4RZhblSPItZhTzMvDxAIkUgBwy4Ih24gmbovcBT g93IhuZLmz1SkCqblrbIC47hC0CqA6sgGXJZiw/4/wrqGw7P8oahmpnDQI4JAJsYgq2ucJYq QxtKqzVSUIcPmIVZgAI/kANRqAQzwAdswIdK4AIEmTtxSAFRcgusyI5JgKrE0IbdWAMrEAwQ uLYqUJH76zYYWS79w5sclAgAnMAOw0Aaq50rQcdyUqkK/BxJAYTnmpKHkKX/+hsJlAVjWi8a +T8MjIJ9C0HzYqkgW7AhM6j/wcD/IcETbKkS/C+HEEh6vJIOg0HMIcAg68ATdIpiobpQycGQ 88GUAsKwGwWyoIuyoCQXWpcJkbu8AJopGBCrkjtpyYucabb4CItp1IFmKAMMUoV4UIUXAAM7 WAWHUQQ7QIGxCqJGQEomOv+i1eAz2ggYg2mNyxPEzui8f7lDhCGBiXEFJJqiiKkY0HCiykuD UECOtIujBxkP7sgLncKprQiXD3iW8DiP5rs18ViD4BAHufs9bUiQrLAXmiKM4cDE6ojGkRmP R0wFNSOhSUADNBCDIpCDMTgCX5QCbMBMKbhMHuBMKeABbPCFeVACwHKMeXgbVeMPUrACcRgq wBsXauGFU+ABGNE/brO/5sKbDmvAK+EBIzCDUuk/D/Sc//OcI0AFhxjAdTCCdMgn/zOm/zKC bdC6f5yScGzBYrqSdkTHGRnAL5GDJVAVgRQybLAEkRCgIKNHYlGKePKDg1uoOVgUcGxB0KEx jSr/AgvIJzkoBivZwYeIT4dcFA/ryBxUz5Q6Ov80FSpAACQYoQbtFlgglxGSSVgIlzIggxX4 pGbwoAjxGXThUP5QAMdApBTwAstQBRm4hFVAIja0A2RAhlVQyqGEUTtghtaYM8+QvChqSrEs DdIIGFcQyn6BjX+xytm4s9CYIrSiMz7LgxcAB3EohEKgpLTgFtW8C43ZDQLhjwiBKgqpA3Sh EOkTDNYSI7cgDB2QvQsCJKlhjgoCDh0oEExckFTzjUOggSJwBz+ozAbAB13IzM/kzM/EhkDd zCrgTGwYHSgwj6zIC0BakKwQj2YIDp3BkBTQgWtQAWysv01VEVdiBE5d/5F/aBFRBbf4jIh1 mAcPsLo4qIAt8L/tnJKh6wAORIQ4wKj+g4p9BATrOYJX6MArKackqM8kkMB8+q939Jxb8M8I RAUeEMCMgpQqcBIZwQc42AEcvBJfqYLjMRU+MIKH0BU4iINCGQNp6IBoFYWXgAjtMoIxeAmG G4NSiIOHqgIUs4BcoTFESIdSQYQxoIYMMIKjy1enKKCmOFUAcogO0AYEAIbbe0a8WIHAm5aI 9Rb+cCFKWkwwQqo7akJRYkJlVAY4UAYZAAeBCVLUkFElchgk+oFVcIYaaIRd6Jeveo05a7zM cw3Jm7yxCkvX4DPOI4EE6AbO27yK8Qyt3Lwb6P8hciCGVLAEanGOqoEqrtgNe+kKsjkX4/sL rEiGDKGpKlPC+XCBZqG94sjL4FsQLkoOkvkAE8iEOFCCMTADMzACzbzMy8QGzQxUQd1bbAiG QeXMYAhUGKCBWWiGTwKQ96AWShyXwniWD8iCLHiW2NRU+mOu4wJVEiFVVYpBzEGENdAfc4MD ZZoRzDkCadgDBSACCwAnLyifdciAITiB1CIxIqmDE0ADbnBVXFACL8gEMSgGafhVhquDaZAD dZiDWeWxYmADd1A4GWGBPRCFDPCCgEWEOqgHGHAHQ0CFKFAC+ESESYgDp7uHc0AJOHCEE2QK OTgF6e0DRHCB8a2GCrj/LyNQgPFdKG5QFXegFSOYBCHQrjXQiD6og4aIFUcACjkAXy8AlXU4 gjnoA2xg4BaAgQp4KInoQQMilgLqQY5EChAAgFHAC/eQoMHDsm1p0CngFhAoEGkboUVEqlR4 MqqqjHh4gXfQIdKogW6oAYVhIh8lDYP5ATsgtEbIgRyISiL2F6NdIkWQjbQ6tH+ZohxlGKNN KzykvM3rjK+UKyXNAwwAB2KYjKPqgRFKAXHw2CkQEK/QUPmoj2VZlroglwQpjp16jg/4tUZ1 gZJJpFT4gHmYhwOAgmo4gnSoBLvlAbv1zM/0TMDVW8Dd21vYW8HlAUnuzGrwg0GqMu/Avk+a /5rm2w/5+KQ6wNRg0MZurL/k4pvN3VQYlJF9Ywoe+IBi6N4v6QMvGIPjTAeRqIIwoJ4jaJJQ yQSMaoE1+AREuAVq0IIkMIMY6IMpiYRXqGUrMQNPGAMeMBUrqQRd0hM54AZU6INTSIcqwAcX MLA4mIPubYFXka45QITsrIIjQALqqQJgYAODbQE5MJ5g2WYWiNtsiAPtSoVPgJT57QBsyAYi 8IMMWAMSEwUksCYSPF9RQAg5EF6dIwJE6AAXYANEkANPEAcicAEkuOcA4sgQJJYBMiCqq4I1 AABgOEU1QAsBcaFsoJawKIMxKoMVIIOZKoNnOYsVOA+TrAMdWAYxBP8HDDArh1lqNlSEHkar IBVK0jiNbnCDN1gF0viBG2CNHgoNgUEisPKM2UArshbrPfuziokYqURayTO9rSw0xesGcHgB HSiELxuOsdEY7pCWC9osAOFryCibuegKwjiEWSSDxcrED2DsEvADyqzMulXkQc3bRCbUvu3M zlzkwOXszrZszhQFGhADKZO9vFyQruU0MqgQKUOOSeAFFTBlVOJU2bbcVD4RUmXlvnnVKpiH YnDVfZSRKliAydkGJzm3cvve7sUFFhgv9ZmGcHDXhqgEaZYmbljAAqwCNjgBLyCCbTKCU7AA GRGFCugAI/CET9iCPjiH7s0AYlJB3jWEkZj/hj444HXgATgQBIVsgSP4ZoiohHuwgJDKhXcO X+ixLi+QVjiAnnVABScZ7+N8n5NQHza4En3mVjSgzD1QFVFA8JAyg2AInYK9kl+5BQDa4JTi geEahfRTRu7YazOejCIsBMmqi8lQ4TKY8WyIgTGMsx/6gh+qATOkaoepARXdyiF+DWcoSrEk jS9gUUObysYjK9agPNMoNKOtPDjcsz0jGIEp2iv+SoiBSs+YogS4AWIoBJrioqlhxqVySbeY tvG4LAnZDcewKkxrvg+gBiiQg8kUBW8NBkPVWx4Q3EUu9M0edEZO9EYm9Emu5M+25L2VAiWY hQrJWAhZD0pjG8KA/8UvSgFMxQfatu1tk+3bvNy9Ae4p8YNi6AN4FDdGWAcWUOf2VhUQKDcO V+cD8IAkgN9PKE4tIJJT2N048FVGQFYj0IJtagE0ULBqLrd1OACIu15fR1UhwIXxdlUeqAPq yQBD6IAHbwEzOIVHqQJcNq9q8IIORISB7oAq0IGGWHeNm4gKuBx30AJpLQJr0JM1WAALsIAr oGc4iAZ8WIcQ4AYnsTlpWAOREJ1J4IJ2BYFXGXE9wecDHaCUAorPRYBRgJn0m4Bp+Qp7OVMO cskxkqFmAAFg6IIdB4ciaoQfsoMvKHKHsYMa+PEaEA0U6GGpLLSWPY3SmDNXgFEe2sqtNP8r sRQNPQwiHJ3iiumhpHf6K68YipniBBiNt3qrzbMYRSOHF2i0tIMXVAMOpGkMrpAtsKCsszmE LJiHWfADKGCBQoYBXQgGKTBUzjRUQxVcSvbsKqj7Rid0RAd8S+b7vQ3Uws/sYDADGqCGtAAQ r7BCc8mZ17J01b4GYTCCFyn1VNb8V7q/zsfN+cQFVLiCCuAGbhgCbhACcYsCOfCAaUADVY0C HlAA/aqCOBiCYtiDetiDJAAnNJAGQ3gvKrk4aSiGOJgG34cuWXgEFzD+gqrmBSiGYgiHcqrV PdiDhoCIPoiDPTgBkZCFPuina68HqRMUVBi6PhAWlHgog0IEGNj/ieOUiHMYEj1ZuqZjAzY4 iQXrgz/PKO0CCESIoghs0UKWQETrDC7E1qfPwSiy1lFssU6WRYsYJ2JcWBGRAgQ9phxao43M FBfNmpEhlUrHiilkCpGaMgUEqRQggHWJ8eIFBjt2FBFVJHRVN6FCf6wiscrOj26KkkbNQ8JO t6hYFZGopehL0m4oakntRsIV2Vo/SJDoZrVr2VquXE1V5IqEIq9y6eXJe5ft3L/d0NaymhYM WlerwIB5oeNDipuTJqUitYbyJFIuSU3+8IEalwxHRsMwg09KMGxSVvOQwkM1jyrBXLuugg1b sNdVeOTezZs3bB6thd+KTVw48uTKl7+G/zLrQypvLslEp66Z1ARSZMi4mHAoFRl4KoygA1Te PLr06AGxL+9effr36N/HRxcF18UogHBF6e+/PyD6/TfgOvz9hwt+CAZ4ICAF6ocgfw7yN1ES +0VxTgxG4NcCf/vhwqEs/cmyYIgDCihiFBRFERFFIR60Tn8apSgRjAZNlKIsuMiyI45RdEAR fhxd1IJEKBp0ZEcHvZiiRgxN5BEiEx25EEIdpIIAFSWpUdMUJpECAgjbkbEGCCyVsYYOEMQA gQxgXLLKF4rUUIMddD61yio14GnHKkURtZRaivwwFFS1CEVWN7UoyqehP5DVlStSKfrDn7V0 pZalcrU1lVeKvP9Fl2JtSdWXV5vehWhecs2laC3gkGOJS2tUNkEqLszjQgli+KEEC2wccVow p0lRxW74EDuscr4hF4xvyCZHrHKIvMZcC9Qih41wqWHLwy3FIdftEQccQIo44mijgBogpNLM JCaBoA1l2pByyCF1XDNefOu519567dWX777/ohckkf/dOCAgBR9YcIEc3qcwgCfOqF9+Md73 IcQblvihfxiRGKDCPPZnoEQMY8RkjiHm52KOFulI0UURsrzjRi3e+OKQO9pH845K7nZQzyEK DWNFBym0Y5QLIdLBJAicdFMyh5BRhkyHkFJGu2NOUQcE8agCziV6dtPnUIPaoQmfhD7/pUgj e9bwZ1FSeSqoXlwBhtcPeXeTKFGS5lWWVKciZXceY3vl6GBcjUoUXVIhVtfeU+3Nd1x5M7XJ Ay4UcUAGnY8hihlGGHFatq0xmy1qv73m2rO35Kb6tcdNG3ty2zJHu+08SJtcC95mm8Fz2YEH QgpkpKAZeNfpQIbVOvCiAj70wTfffPD9K9/18UUoMYrdr2iixeuMgUY6KcKI8H8Rfc9ewgBa FEUVdXDjhSdwVOCFFuv4oYVBIxdp8YkKtqAZicILqECEEgwxJBwdBB1Ao9FFGlSinvWnYjQT RTEWiLQ1sME/CaKZQaLkjk/AbGgXGZLSlLSQFiDCIhRBRNN+/0EFk2hjJakIw7pgsYZmkGIZ cHgBOLCQJ3p8gU5E/IpR7JCDOtHDDvTIkyLW1idF0ENOX+jTqCZnqbn1JYpdMUqgLMUWVlEK L3n4S1Sm4ihSocoVeSCL4hJ3Fkvd4IuIKlVZViEJeVgjEVxggShEcQTTGIFZzDLWcIJxOuHs xjfKEg5qirOb1yHHkb/ZXe0yyYNq3UY41ZpdC3JXLUp+slu+40El/DCL432pJmv4UgqaYYU1 rIEM2lBXMibxDGGYQXrTUw8jGGEe9p0ne8Ysz/pABj4BlWhiKyrQEaTRB0TgRyE1OpGD3ocL HqDCg+sYYBQYEc51TGIJIfoQFzyACP8LnON8FGGnEYi0ICKtwwjpsEBEEBEMjMiBGx2oghEu siMjVAEhooiDICywwnRUwQxjaJFEjGCNIpSjoFEwQhyGdAQiTGMMCnhoC6pAjWKggQ8s3EgL 4vDQdWBQAcVYKc3W0UKYAe2kCoEhAuCQjBXo4GreIAMIdjgJYKhCFcpoxBe+oBRa0Ikoc3rb U97GVGfMSShXsYNShfKnp6xlLYrKyiqs4oo0Fm4rRiGKoubClslxZS1mcdRZ9qaoN96FcWNL SqTemBW5YrERWEhEOY4RDSUcgQ/pkAI+UBEsYPEGWbJBjbJSl63ZPDI1VaBNI5fjm+Jgy3bb yt0mO0m7TyL/55Ol/NbtpKAEGohBVqTwRkrWEBnOqKtLKbitNuhwDSNUj1+99W19qJe9+cTI QOvIwBAWkAkvWAAhdYgDfkQhjQ60AAb3QMMeFhAONHiiHudAhAvqEYITFIMHCFnDEnZA3iq0 4AjFYEMIvLADjlnIPwFahwvK4aB1uAMJn+iDKO5hD1lUQQGfqEI/u2mfFpwDCTtYBwy80Ico mKECEIkvIrChgPLJQhRDyEUVOFhPOJi0D3DYw4+G1IJHvKIPsuiDNNfxiApUAREj/REMTjGG F/fApFIIQzqchIgPWAMRMEACRBCRiXPwrIUZYe8KYSZTdSBgHF1qBk6mkAovWOIF/xAAx1Pq VCdnOKOJZ3NGEp1IDxLQScx1WgUyhKKnJBJlT1oFi1v31lWzZEUtQ1nzms0ylS+idVB1ISum 8BI4SgFObo2QxCBEcIxtsIDS6TDDpY2AGkUaS5GxIdZlY7MtQ152NaF21mtyQ0lLQms3kuTB Om6hrGqFsnaiXY4olWOQaVXrFr3jFmm5dYtzzGIWpFAAuyyjBuvooJYpCSpQn6ECMwiXmMHV V3ADdj3iRowgOliAQvqACvzUYQEyrcQrxm1dexioEv5MUQeSsI5K3KMKUXCHFgx0DlRUAQ4s qEIf5pE/7uFini5YwkAs4odiIAgRdfhEFOTwijGkIwOn2P/Bf6rgBSLwoWJmuEcucPEIblDE vRRBw0MfAYdpZKIIcMioiTFeIor0wQXSiENALSKKCuQCEWv4xEEK/NAxlMILQ1DAPcbQwp6h wRoylcMQpMGFPlCpZyvMSEWUJlMX8GIUOQHBTZTxtUtEwA1oowVT6fHUOTGVFnKqgVKdwdS4 y/kodnDGnY6iJ0INhQSUKopakpIXO6glb4RnS168mni/KHooRcEKn/AyFDy1AwexKAcXtiEI FqTj0qvZ9OfxMZvUKHI3rHnsJHNzGnyoZjatcY20XBOc3MmeW7GRNWmlEOxgC8ezmsz17TZZ 2l4X59ex8YMcUlGmeZGiB9opCSn/XNBTHr7kGXSg9jGzjW1+Zf/aAWJPikJAhCFMwwIpSoUQ Ci6KdMuiEl7IhYBgkG5cIKII7xWFJ8wvhiV4s9//7UMVdIB96cf3xcjBEc1xzUGErIEjRJw0 oMILdVsFYdQ9OAIumMEpmN/InRDKiYIWRIkowAEqGIQF2Bs2wMGDvYiO9M5ChKAjIEIlVAAE TkI5TEQHgEA0yBiNydQYGMFMWYQYWAOBdRwLTUKRJQkLhZBHVASsDRkCzMBOeIEyXAIV2gEy 1AAtyJ0d0AI90IImaAJVbaFQMFVVvV0nCAUy5MnZDEUONJVTKQIKmA2eqJFRuEVRTNFQjM0P VNGgrQJT/9QFW7SFV/EJUzRCDuxCLCwCF5hABgjCEVSCGZiGIiEWPugCp0FWsHwebgyL6CmS J+LGZakGbMhebuAGttSea1DS6wkfJC2HQfja8PGOKwYf782i8BWfLKKSO7gDZbwSGUzC8mwH u5REClwGKVzD9fnStvmScGUfejwIgjDCOhyA+cnCEVTAReQXgozBKwhghZlff5iBAUXcHEAg Bm6ByKUb/HiBPazDB0SDRPjBEsAIKnTTyCzIOhDBEnRIFGTAHCQBQaTCJ7xYBbSTEQAD/IGf ETBXSKGfLBhBDFQBLhwBNwxEFMBYKvDBjuBXHAAUMGCcidmD+kgEDFQAGxjBEf94Qcc9ghcU 1M6NgRkUARJEg0WgARGIgh8MAdUdCSKIQT2ElAIUwRggVzqckItI2ZRYxC3QnxggwDfIwDu0 Qw24gRtg4VS6QZ1YJRaiXdvdHS1sYQ3QQxNx4Ri6WVi+DV4QEVQJitpAxaBIhaHRg+H1id8F YlMURqdEUV00QitEwCDEwjHgAQuMAQVEohFYgGmInughliFlIiKBXm6sxibKBj4A4OitDuqw DilqZiLBDusE3ymxYmnxju/QGi3i4nGQlrcUn/F1i3D4AQ24gHW0i2Y0g0o0Q5dEX5fsUiUo I8BoH8BUj8AQ1wAFCB94QB2Iwx7AXxQcZx1klxZ0AC7/9MEedFOAVME0AOQ6LIAh7MEOxMES COA5TEMxTIM9xMgYTMMevOBCfEIDFpyKJAEu8MF59gcj5AK79Yc9CCB+7EA47IB0BogwAUIH 7AAbAGgFQSAgLI1x/RPHIM05nMOP6IhClMzErAMq9MGEVpNEJMHSpNhCvEyNRZmUuNAS0lSB CFRGmOgzyZSNXcALLFENdEInTCUK0EIbIgMXoh09fOFWfqUmKMJXCkUXdmGZOdGQOsMVEmkV OR6laFXehJlSvI0dsAXhEYWhCUpXwEWiYAUOZAALCII9iE4kOqYlip4RWGKwjB6nqV6nQZJj qQbpNBbreKJxzMYpUpIoYkuo//nOZ/be7ewaJO2ea+airvUOLBZHt5gmatKaohJHr40BFIhB MoyJZWzH8cALZ8hEZegWtfmLMuoLqC7jMFlPeSjIikDIxdzHf4ToqvKHgZCMhcAqrC7TN40M fygMkXAILjACgkTMguTq90ggkRTcwniExZyMgexq0GwEAK0IRuzMMynJjLwPPcEIx0zrlBSN RKCU+bTIkbDQ1TGETK2DGFwAODBVJ6AdVdLJVtpBVf7ojP4o2lUVV2bhV1bl2dTAFZ4VFVFR lgoFmt1dk1ppU2TlUHjFUNRAoETF2DQCEwjCpeGDEYTO6FRsIWXipimSLqCGYo4ePvDGJMpG auiCsf8URyYOx+lli2pIQXC43mfKxu2AluzQbO4UByKc0qJ+i6B60q71DqKilq+ZknKg7Cz4 gbLRUjNUBpnASyr0QCq0SykIwxGMavcJ53AKTL40k3+cj38ozIcYa/8809diK35gK5CQzH8M EAIuq//cYwFyz7BOTK7+qoMUyTx9rXEVDIhkhIgQidA4UEQ8yY0IDeBupEKdT7MqydG4ELU6 q4uwqIvMlEJQgxMMQw7UaCcU0VbW6LzSAufeqyZ85VeKZZF+JZm9zVm+nVb9CRNpFdpIxd3h RVspApoZxVM4A6XQZVS8ydjsAgtUgsnigxlQIj4gpiJlLMfiQycOb8hubGz/zMZios5lEQvz uizLVhJrvA5rTNZyqNq15I7tYK9olpa0/AwmpWaijqZrGp/QDi0ueksVCAIU0ACZtEQPoAQt rUEyWIY2dAcdwEPVqkdv+aYxYe31FAzGIEyDQOOvzm2KPLBBTKcOoGMFHdc+6oc2ke3IGAEL UKgLLMBIfh/4/c+EWEwG5JuKzAh7nE+ASEkfiAP8rYMcTEMLVaj5PGvI3AwKxZQYlEO4wUFR pgyVhKuTccRGDE2ROAkKmahMZcIFkENW1sDZoc2M5mtVzUlVoh28ul3psh0WfsGOrgXc0UmZ wd2fCUWcICkVrXGakQAfYsVaPMVeJcUUSULEGibz/xLv6Hii6FmiLuiCIEBCmRLUmkKLp/HG ZU2icAjLbUhmcqjicITvqC1L8IGWaJmWJ42moW6y+7LirzGqarYv8dkeNqjSJJBBM5CJFbAE S4AAupSBNmjDM1zDEdCHcCIwcF2te9xIwrSM155MBcGaceFHFYwbrPJACRoBHLBbhFBEN53t OlnALXgtIFRBBrxCIZHTEqAkG6ACPqbDESRux6TIEWQAH8RBvuFCFRwBF3ABH+DHEdjbh6Ck EWRCKZjAEbDUQ+ECDLCAGdwKQ0GBOBABPk2EKExCHRxAFZQcC6zMOswDEtyDC/BBH5xCPbjA EOgAG1BEFZTAEAyBCwQUuP/2DNCoaIsCLkPQEyJwgRNAQ+bOiSasKy24AVfSQ7zKGZUSaene HRam8ZA6VRMZxZBWqZQqhRNBRZVS0VUwRZ9IURGtQuDhiaMgAxMowT2laSVawCUeryAvwp60 zQNsg/JS7CEPC24wi8siy1qr4qqxjuldSyLbHu+Q0szqGidDKitiC9D+hvu25ibhYs8i6moi anXNwjwQDw/dhA40gxpMQRnoAAgcAgiogC1rralmmwFLD/UoTIHgQgzCnyy4QDSw8xpYgyhk QrqtAzbUQTiwQTkJINSxwTlMgifUZ42k8EJQQwWMQTt7whacTyjJQQXwQY25ADd81yPEwBZc lBf/fJc7GAI9r4iJdXQfrIEHyNQkOMI6VEEPWGAkaAFA+twYtAA+lAIfEEQGKJAs+IE0FBQ2 eEGRIcLOQWAGTENDy4EhdACBUd0EtQA1aAF7mVgOrgNFBqAYxIFF4JuTuZDhakTRIAm5rqg7 XMAdTGUnaIJVcmW8vusUd6ERfaW9bqEzZGUWDoUbVJFaiuFZvU0TmdlVlQ3cPIXffUVeggXh PQUOUMAgjQ4h7THxVgI74O4fWiEYPEAlFNLqLSYjb1psIJZxtIbtUFbv1YYl344kh2Zo4nXw JUdpTrImHwlpKirQekvQmhY2uAMNTAII1AGW7cRKgEkPLO0awIMtE6D1/xCTLme2Zv/LiURI aK+ICwhBPd3DgcFMFGSCFpyDERD3FlTBFZDQdLLj2OpPMSQBgOcP/Q2BBerIfaBbN60DGuAB gvTbgFHDK2xDBqBBBWzBAFVDN/ZHBmS3MbMBEZQCCYHXg20UdZl6f+h3gSiBOq3IGoxBdLUk OXkCNwyBF7yCvbWMi6xIJhDBRWgcd8tCA9BYFYCAF7iUAuxBQx+NkDRrRXRE1n3rcV3ABtAo 2mwh2mm4F5LhhpMhLXhuF35BF25hj3IhnQxKVSG1UhgFUxBKU1FKDdDFWvAJ3GGKnyQ8V6jF KuQAE4COaVBimiJWA2xDDuCJHwrFw24CE1Ri8f/GBumM3p/KLLaodSr6hushx59OC82iluy4 4peTls3fYmATbWBvUijr9aIyqu64pmALRyXMAg3IBE6A3RQAQ02A3SvVMguAqp6fx55zdgHH B4Dwo/xJZBSgQTlQzDmEgzTMgUKNVC6cfS509xUIAn+YutiuQxzMAYzgm3yigjQ0oAeNgQym CBrsI/01syxQwzTATEPvR4Bw42fPend7AQmtw88hyDkUQx/UAXP2W3rT8APKwhj8I34I5EVU 2Ag2XbVngHTKTIm0gKL/SB94QUf7c0vCUEatgxkcgEJM+LcqyZPU/tJg3ZDIwQUEQSdk7hWy nZJ+oRcaP9uNLp2Qbhj/b2WL18AS1Un0L6ya8UlR4J0dNMJZYdWMp81W1UKY9cmgDMoq4MAY HAGZhk4fjI4uVAMTNEL2I0U3pGE3YMA7sACQ94HHjo6pTe+TpwZA8AgmRQoPgzwKGqyC7aBB hg0PJrw1sSHFWwYvQmzR4uBGjjxa3PoYUiTHiRxJ8hAJcqLIjBsvjuw4EluGWd7W9ADWbI0O ENl46kjVzNQ1JYAAoUOXNKlSpk2V/lO6dGpVq1ajZI0CKAoufF5yRUG0plwUI3NQrVv3YQku OWi9csu1Do21dVFEebKnNWsGQ3fdabmLaogjXLi64q1gAde6eW0BVfGy95yXc+vedjjMVfIn /1RGvBjqYKSC5yNIyq3DJSvOnAWIW1SBY41PizHSOtiegxiRC8NRYHhBJauPlzhHXCxQnWEO oqwtosjKIK3IOR5exqwWJQ1VFRgKinAZwgbRuhZq1yHaiEjW+fPoPZpHbx5RhgsbcuToZIcW rRqd+qOlEzcCrMG/Tmqwo4YFabFDwQYdpEdBOyS04wcH7WgEmRoUsaNDO1bBsENFLqwlRGcU 6aYWRVBUpBYSfvihmx8UIXEVEkH84osaKBjjCDOMMMIMM3TBZ8gMQlnlRhlX6WaVd5gY0ggp qggGH4LwsZIHbILhUqAtGQpGoGDElELMMbfkoQqEIDrozDZvOZMijP/mbAglkEICyaCP9Myo oz1ZwkilkD5yKU+TYtLTl1nUqQMEUkjxQg1tSAGhDlK00UYFIDJAaimkQKWKKqiumopUqJrK SpasuJJFjmL2+CQDR7IyAw1YBUniMCNOICIaVLpqgYUT6kmHBVS40uqcdLLqg4+scOGjisYQ U80IPvpYB59cDovisq4QgWGMyxDrVpZ1nK1C3XL7mEuWabM6K1voongvq7vK3apevu5a9Vz0 VkXkBLWCPezcVdfxV1X43NvoX/jke0++iCdWC5E4gNAjhwY1MZA/TfoDkBZnRvavY3oU8a9C lSFMkJ6RQUTGDmd+CLEGFO34wkIPV5GwEUX/SKjRjm6AdnJJJoEmgQSeaazZyV0EOaKSIYsc csiplQglgkbsuCSWbcYAEh9djNCljyzNuHKggdJcc6GC3uSySjcbejMiN+WsSCUxXbLzJT71 BDzRhwTtyCLAD63zT5RuwUcJKCZZo5lJQGhGmzV6aiaVNUq5Jg5cTi21VFJHZ2oqVflKlq+t uDIviruQaiwKRqCtVjXVW0CML3oTA0S11eXrFr18ETnsd1ygS9a9u2h/7nU5PJgGFeSjQOc5 1a1HSqvV9gWd4q4eVng1+KKr1/qEo9vIPIehO+/1hitmWGL3MnBigxrcYPC/jvurQRNk+uMM AwXIDgR6GT12NEAD/y2IP/5JEAo2RA8YfchBMXKGhy7kIaL9IGgfctGLalQ0D8FoRTOSETsE wQermSEdVpvaCqdWCSPpYmpBsho+qlQFglCJTAZZx0TE9JAq9dAgXfqSmtp0xCIKqgp+mpMT 77QRPFWEIwy5k57+lChE5UklFokJTEgiEkTw4AizKMEV6tCDKdQBGJHTBghSMQk6XGMbpjOV qZpiOtJdhXSeuhe0oAObrFhPK+dpXmK6whXqIfJ1ibnLHwu5ldxtBhC6CyTBdOfISjYGOgTT iiJXhTy1cIV39TpXbB7JFVAhBhD04h4jk7Cq8rlSVS1AGOpkiTqP2LI90VGL+g6GL/Skr/9e EyuPHDgBivyBrIG00ATI+mMHaAZQEwoaYIRg9qBqeghCMkOGAD2UoJuRAAUeMucPTHQjmnXQ QiSqUdCGpgglkSgPNyJBN3YBiXS00AyV8KcLaXg1GOhihf184dikcCVUnIlvc0LFRcykEoUs 8RZrkmgSk5iRJnZRilT8E6BkckVAZTFQJ8HTF6NoUpIQSiCzgIICmsGTnKxhBS4ABgi0QQUn +GErovLpqEQnqj3ycXWs6t7qbgG6auHCDGjZnu7wVUnQWAB4YihHLg+j1HJpKxsn2IFq3FfM 7QXPedCSZQuq0ZZG+q6RUbgOs7bCStCERX2u00p7cmnKc2klYQb/g47C/rieX/7SlsB02F3g s1f3EbYFcriAHvYnzY9pQj8AqmaAOrZA/9CCHgtKUIMYCLILKkhm5lSEhBJ0IxCF8EIhFNoI u+EkO3SwRuhc0WzzkCIadWMRUevnEVpYCalNLR2iaAAM+ykkIaVNChYgUw8vEoyLXCSHaloT NhLypjhdFCIJgdN2BwWnQ2F0pA8BXKD21FFssFSKFIFJS0zCxT1VglHNqJwXwqANYGRjEpNo xjiuwQVQ2bGPTuHj6Ep1vUJObx3AqlcVLECwSlbBCFUQxSuARcp6GSEdFnDwOiJ8F0SkYwxG QEM56GWGDFAgLF3ZVRE8EY0tdMUMXFiC/2eggwhBnGMMLChPVhDBAntkgAhLoKoszLANxPCA BS4oxhiosRcewIEL06jDHPbChjo8Yw7RKKVqbrG7SD5HlOXqJF9kEczzyEI9bCZswtaXsE7a 8i65O09abqEWx0K2EwgK0DcB6ExNOONlM7uZNAvdn5Rxlj8XdMbJMLQgCQmtQzTjYAVZ5KEQ ksAVHeoGjLoxQkXkFkZBi+2naaQkSAhCuK2WWj/N8KMj0NCfMjRokWRYJDKtSUwWVciaKlo4 IdLN1108iJ8eemyITHdQKTGvncYIqPXa6dh3aklMVnKolEgU293NgB/UQIYelCEbayBDNqaQ jDpQIcDV89S7gf/qqaBWxY5YEWWw+nAKQ4RDFOsQRTFy0QcFpIYu0+iDwCsAr9z1AQ5jQIQR hmCPFhjhFOdoAQzusYNwIcEa7XFBNKpQHFpVCxF8WIx5XDCN8ojiHnzABcO1UAVPisULCyhP CF7hHDlwozximMZdzOCJMawDG3D4hFpCwI0OyMII99hLMfEFrX2VkndZrVYvAVsvXsbnPYqF X53TQ+dTpict55GDE/QgTRQwKIC0IBA0B0gyBpYMtDAzmTNQRCAHedaaCUyQg0I0W8HXSIL0 aNoIldY0oNmhFihg0qdrEepu4CBqLTzC5Ynb6t/+9oVDggGRLLDDLH2JTNvtYRPzrGz/gxTE bYRrSLGT2ERCGarZWJwin66I+z5x1It0CtRIVtoS24OEDbOYBgh60AM4UKEH2ejBGrQxDl5w wd1AveO8fRo6eqtOKzyogGEaowBrpIMP1HjFOjBuAV1tB1i6q4ICiJCOtOzqHhGmhmBeNw9r eMUT0yiCCbygLXQnCebq5TIOF5JgHepg/6oAGPaikpJlHa5g5MygAoBlDLhhHWRj5FqgDj4B FxCh4RCDD5TOLE5hxvTlk4xqdbSHETajdlYnl9pj69wjzdYMsdpjsHRQPuolLcwjBJyAAVbh mTymP57JCActQOhhmjoGZF4mQUhG0PCuQfBuZrbJnBxktCJk/7QU4e9QBkMspGZiBGhehARu y0RWhATyINR2gQmGqxIs7wiOy58awJ8EIR1qbYXwAQYQqkrMZG6OCBUmqtiCQV2ODRHAS1B6 DYmqDSPM44tYwiPSS4qkCBus6LzOa05mjyWmC4xgYor4JvcMwqUOoXKAARhWIKbsixWuoQhY Z/uUghF+yiqGqt7eTZVYBRG84Dc6YA2EAD2mZa664sKmhy9w4RZyQQ5ewTCMAAnsARcCAzao wS5I4xk10MHKhTTUrw/ugRdTQQiiQDZ2QOqyogpAAPwqcOkwMAnGIhqALBs+geZGjhijABsq YMZwkXX0RXsqaR/jagVZKQUXhs1y0P888IpiZAEEV+MWymN9GsYH1eIIruE+LkETdqRBvgkJ /YeaJisJG8SZ+sdABm3uQKYGZIaZwPAHJESCVLJjkKFDKiRBKKgWOOgLXCRGJI9ogEaehsYW xgAP5fDVpObVpubVrmZIEipLdAi7CGL1GPH0BoL1zsS7Vqo73Cu+uuiH1oET+2QlAochJxEi sEEWZoLaVAL15OujwOikOMKiIJFPRKEI5iH5moEKgOIKKmcGrsEEqu+O8uj65i2P/tIpdCcC v89gMkALiofIkuD9okEtisACVQc0nrEKXAAPogAf7pEAK+Cr+AAOfo4uloDCFIACauflVIGq omDn+OAc/ID/G9KiCrjBEbhPLNbgA3FBFOQCF47gFTSjEuAgGtLBBU7BHceCVoBD6UDQC4jg A02JVUDHH/URNjIJlKrTYJAnzRpSYthRLU6JziJGYiomz47ACYKQAWoBGTZkfwBkgPpMJAGk E57pmfqs0fDu0aaQZCiEsyBkZvxjZhwEQliEHi6IRdZptmLEQZRm02gyRfLAFe4ptkjgZyiB An5EFKSmAfgAQ4drSIJrSCwgbcwgofywKQVCCsLEbgQC9pYt2KKrovJE+EACEX2IJVAh2rCt BdQjLDViS87LcAyHT7wyvbCoo75yUPKscaCABsShB0ZBA5ohGyoHBFjBCVwRj57C/8D0CPtE hymMqlocwQJUxwLiIBrsQVeiIAnYIBo+oQrOVF8QAxV2YAwcIQnuRf2M5xzsARX4VJFQYQsY I1gCKU3HygLOAV60Mgm4j86U6jy0Kit47Ea/4hnLRyAz6a6q7h9V0Hd6CZEaA1+i815mUAfL ozz+ZQbd5z3KQ30eEgRFwQkA4BvOQA9AIQ8wJApRAAU2S7JCRhNQQBM6oQ36YwkvaD6jcGRI wD/uU5rEqQo/ix7oYZ4m5LXAsEFrREVmxAxXBJ1U5AZqAQxQCPMC6vJg7fJEwWrSQbmMAIeC YUqopCAiSkzY5qIsgl5DYk1+KCNCcbqyjUjVRfi88k6izf8sAeUink1vMMIj4Mv2hI8SXUIk fsggaLSMMqEZesC+kg8OJkEvXVHehAreDKx0uNSokiWT7PQf+3F7FAk6M4mVuoV2SjYx7FSR ukXrzCOrPglnt0p3ApJeVGcdVEmR1oENTmAaPEN4jLEG0cx19oqv/iUx2qdfooOTEkN46Aw8 4ec9rrawdLTB8mxVjQkGeAEAruEMLqAXGGAD+MNACCRAOkE/F2Q/6HOaXmY/jBBBGo2A9HO0 UOCC/m60DK+dHMSDhmYV1DBFVCQBXIEcEiABhuEOAiABMAETJBcctsERYk0OVQi46nBqOhdI xMZMjMBKdkiHCCJMvqSJFpEHHir/I5hNTahLUBKli67tFlAhjNyLvABlYDGqvZwovYZUomSC 9riL9wwCH6BACVzAC7KBDJpBB9YABKgAAeahp7T0U0I2VPBoFg/MBaNzZY1xU1enWhghZaGF dgqTHF1sBfmlKzxJlXi2mMIsBfsRXR5JkPhUdxKyVSbJrqrgHD71OeiF6oC2rdBsVTYCMfzl kaCFlH4Jrx4mGKgBDfjggcFOaymmk9JDFK4BAODhGnjhDGZVD7qhP3IAGfwMQZiJCgGIP/xT QUjmVxvIWQfNVz2mBsopQDHkC5dGaVahaWrGRVZBcjEhcic3ADYgAPRADxhAhC/gAsz2AhKB BdJhuI5A/wlEodaO4Fw9L22CZG2eK6EKIkx0KIeISLoMkRGBTW8q6ne3jdkUh9pEavgQdvdO wnWFV4tc9xMLZbryFdg+4gioIQNgCigcZQ1+4RqIwHo9lsBCNt4QTI9WdpNydpNWdleGYA/4 ABGkwRFc8KkOo3m4QlHXIQ7KgTdc4Olwh3dy53zvhYEb2MUmjBq4RTJcLgpCQBpONpWopThm DBcS84HfB2mhipT+6n7XIQFlbl+Gh5HSRz1+iQfUwh2KwVTpwyAhZiP4lFRfFQBsgA7ggRcu AAjOYANA4UFMGJr4w4A2q203q88yyw0+BoLUEww9pJxeEoNWQSZFZCdBhEZuoP+IGYABoNhs z4CgL8AJLoABEEChFRoBeoEFhrISMDRzi6uK/Wm5gsQChij0yGR0d+i5riRLTo8HRg9eD4Ih 3Ma7WFR3CyeJKnGkiDRhjU1QUCIlajdxEsWJ1hgigsGlPkADKEeNcsoJqtcWn0IwHxlLAVNU /LF5+JTCVGMdzEAQjCArLMAdKiAd1KU0q6USMmDoXCwYxoALxqAD1uEIruxY1uFYnoMF/E+s o+AIJJoIiiEa9HcdjMAE0OMA8NospMcyz8EMiuEZAnAdQqAC4kAcvKAO4OV1tsOwryoDuOFW hgAt1CId6mAPhmD/XqfOXCAOXocPXkHm+AAEpqEevID/BSBVB+rhBCrgqgupClJhGuRgD+pg Lgj5HqwBgAtrPsauYlogCYa75BDguOlgBEDYCdCWAcyhGzZStKIQP0kmZTAEswboZUaLCiWE Y2418MCwQCdEaWqhBkggAfQgoZ3YPBO6odkbANzbCY77AojgDhugAY4gauRQuH6EoIZkpK3k uQYCH7iETEoXIaYkIkw0KosIXt9VIVjPpX/3pdtk2iJx+EyCE/n1YVdKp8Po9+pEEw2iEtzB HSSnB+qgGdCgFJzABarHKYbK+mgRVfbIj+AUfdegAthgekTBA+YiEwQDsaVhLtbhrnHBHHcg CYLjGW8jFzqAGNehCLRAM64j/zsYzjAQgQu4Yx0yoALmggeuwDnf5+PWoRKKoQ7SghqWQK/h IB3QBTvOYwwssDGSw5FeLgbOITq4QDRs8+jMQDg0EM7xpQWSI2Eq4R7Sogg8YFoQAYDXYQ1Y oAMQAcgLWCxcoB7ojBqKQeZeRebk7Jc+W5vRI8/mA/2uAQF4gQ6Se47MMwg3gBzYLtFKxkHa FrUCJAoHqCSlSYD+Tpz47kL4GWeERib1QL4voKGP2zzhGwBa3aEvgNnl+xvwQAn2yUfyGw6B 69VUKG0CvF0F3MBzCB/E3duNRCmjUofUBEXLZEvG+Eu8y6X1xMJ9tE3mWL4okU78pKN0mvaE FI73mP+lqgAKMmASlE8DegD5WpyRPZYWIzl7QdYpWBZ5PPBePsAQMiAcIpNbRMEQgKUFFCAa kiAEkKAeMqEITsERLDM18mUdDkAL7NQcHWEd5LwYm+6rMsADqkUcPjBlCbsK4kBW2hQEuMUe 90IykPMIlDMKPgAca9YCJqMv5sBO18EFjk4USmEaiGUakLNVXKDjcIE0UCGWzOAEvKAYxrEK TsG2x4BOH4nN1iAO2KMFpqMKEEEJigGt04diGqzU84y406IDKgEeEEAC6CDVbYAXxhYIyJkB 9CABstAInamZmullIt+ZCK1BktVZR+aCOkuCRuZDyqlDOGS2bmQVLoGh5Rv/AeAbVpf92RHg iZ+4oQFg9d+BBSoBv1mNvy+vQ72YbHTBTMgGH744KcU9oX4/S8Rd3CvMSq4k3YMoKo2oTWDv uiIi2lzvo3o0RrWI3veY22Sai/q9SIsUESqBBiKBFKgAp5rhFFocFyC5qQvswH4KVFy5Me76 XtBgzVVD5nITNhsjFQAiWpJHr6qswxVs3ToX5XBFSWIG1boMHtZFQdXDEa4jFXI9xOfJHq4M hg6uS/UpShRGKlu4+EQEFbZicZY4rAKHTxREXjRGOfKqQ5R1aBqqfFjl1I4oI+d0WNdhUrR1 MCqgwrWOms8ogFzaXGfmlAVcYzSu63NvByIFU3HJ/1liMUqLdXGKIVKIxppCOVoQRZHVQhZg uXItzm2B9dZhhUauAaAzwseIEbx4jUCAwMkFPXo2KKJHqwYtWprs2BHtbHSO0aTddGJtJ3SN 2M5qzKanCZntH4pqODP9mwQtOySIn7az6l0FBAAAOMEMAPNzBGcY6LmA/cwFJ857HWNxJLz4 SkfImzGjCx96fEaMpFdvRj0+9jCMxDfCXheM9+2DSZHiHw9SYFOFf1LgIwUPPATDA4EKPqhg FTzc8iCFEF7Y4IUtaIihghS2YCEPIIro4YYeQnjLhyEyKKKJJEZ4QAbFZNNMM2powIs4LQAC CDo9ogPkjz4GCWSRQx4ppP+RQzKlUlc71RHNUaJIYwEi1cyhkCivSLRQNLgg4kI0VVTihUii VCAKNhnMIVEG0jiCCg91pLROEcXwYYYL1iQxUUnrgOklYkeZcQ9cWVVwzk1XOLLTGjStc4QX qAwlxjYHMYVLFdLMoZccTsmy0CegysHNPHXAdZQsUYjiRR3iLDBELmcVU0cd0gihUB97cDOE FhK19KccCkgzp0LriGINIoAFto6qhc011Fzr3LIOKi3MZYZjDphiygg28GIMLwhURgd2epyB gW2h6SbabKwpMhxupZVmmiKmmSaaabrZe+8qyLzRRhv32kEPcvb+0IgynjDHMGYXROfEGZzp gcn/uRdUp4c5oRyRjiDilZfOEQ2cl44Z7p3X3nkomyGFLmZU0p7LMBsBAz66GMFHMPPhYwGC BuKDDYNSSIgNgFX8dyE2EJqISIdOX8giibc07eKHCoK4YYolKnaihYqFCGKIEz4IAw1c9AAM LM1UwIsOUShZ5I9C8kjkkUTSXXeP0CYxlELnVIELVtXyccQWhq1jwTotRFFFFYsjgnPiTLbQ hwUGIdYCKohcZZBcXzZ+kLSiYxUtU6DKghVgiis+VOC4XGvRYExd6tBRxmK6uLEKqTRYYCoF Ltjqvs8uePCsM8k7s7ArK7rioF67uLS3SHRLEi1YvzqfjSFQCremSAbZ/7gIXDNCxHpY100N OejGGi1fiDZabTWwH9pwyKBwr20F27aKM/b6GzAUIOM0X7jX+7phh1pAAwjNycwZzuCE5wCg OgyoYGes80DNbGMMlRAEecRDskqYbIQqK6HK7POyEtpHhOw5Dz6CYQQpmOyFVUDQ0Bb0H6Ul 6GgZElCDJCS2EIkNRRXiwbQmNDWtVehrKbpWihRjIihSSGtTjOKLvobEDQXDHVBwQQ+yAYxR XKMOb1MSj+QGtx6pEW52kxsgWAIIrqwEF4AIHI+4Qkc6/q52TjpIHFuyEqYsjo4HoSNLSKdH xMQxjq6bHR6PcpQ6OgSReuTKH/24yDP+BXlxef9j4Jj0JTpKMo95DOTumhTI1h3FjitZpPEU AqpmNQsxq5slsBQ3l9cloXqCWwc+4AGAX5iCCg6gwigmw4tvWcZhm9EDGGqAAk1oYjaaqA1o quku07AGmrnZnxtOU4OCFeczP7ADCtowQILRw3+9KacdugGOC2TmAhV84HY0Y518QrA50dHB J8bTQY9VQoTnGajKDPqykA20EulIx8jMUDKXwSCF9XHZfP6zM53Npw8KQhA+eHC0BLGoQCyK moJQISGTgvRFV5PQ2I44oRFhUYpVlJoUK5S1qyGRple7xRhm8QFgrKEHy7iGDtbRxjWycalM jdsdzyjJRVrSkrhgxCT/H/m7xR1lcXfU6kVYWcehICKXO5JqHK06F8BNlTBNIp1VwxpIafHh KmG9I+sswoi8tlKODlHVJ23HFLvC7qmV9KtZLYnLWkIPdoJ7XfSiZ5FpCTJwfKqWEYCZjVGM ggpU4NY44EGZyogPAfREXw2mORx40SI1/iNNamJzr3jZIRC0tY0b/HUa99mBXl9QxG/swK9y lrMbaYDgdhiAz4md72H8bA4nEKACE4TsY+UJD0MJKghRGNS6RxCEGcbzsvK8DKEi1EU67MOe /lwUpMEw0IIUxKACwXdsRHxvh4R4op2qyGtja2LWnvggJ04RRUpsItdUtKEqUAMKdQBGD8Jg /9Qd3c2pZrSbkZI0pCRBda9SZQRUWWKGCiwgFx2owxi0GpdVbpV3UJLjH0UxBMClYwlJ4FEl F1kFL4xBj1ZdBx8kBchH/nGTUXDEFur4ElYyC3menKMcC+OQOoIqsHIBFCtr11djRdmS0kKE l2eJOln2zXfQUhwubiE4NB+keus4Bx148YdRNGMGM5DAOEoBWsqUrzLi0swZNpAA2wh6NKYB zXDsUJvQ3GuaefhCIL5ph3TWoID3+kE4iaOIHxx6FZlWBAlWAQ1zRPCBG+AMcpnTHHEh4Fsj uAYdXOCH7n63AdYVYXXNIIpae1Cg1fWYeI/wsgYQdKEoXA96jQDDZP8vqL3BICmDbtG4Zk+I QSal9n35q8QJwTSmS4RigAcsogE/0UQjQqJNncgDCnzgAz1YQymcAAIJO5VuaDSjGjFcYafq 9SA8YAN4JqVJI1DjFWNABSrW8AnGxIEaLEBFHKswBmpQYwwmWcMCxjCNaWxhKCGwiizMYJSR uKAYYyiCBXbSA2uIgVg7gJQ0SjGHlESSsK2TxQfE5IJSVMADkxpKFM7hgjrsYQ5bwIqeXCAN wx2gGGjwQkrcIg1q7EEaKEXDPbwwjaeIYRoZGIJe1jGLOcghDl7YOFeulYE6hEAOXmAD7MgK O8LMRc1RQHOK6q5mVPShFADIRjaoAAtYUGH/BqZghSnoUAo+83me1dnAaehRMGkiOjSQv5ci anEHBtjTCegzTTXxhejTwKvQiii9vehBgtmQIA9pYECpN/DA5mYGASOABy9UcPjDj4ILJRNZ Q6k7ayXkugG0PsIYRNbdj3nwux5LhwhLVrKVoZBnMbNAe9nD7PYa3D8l9Y9LBfRsBSXopM8G 27Sb/UStyXRp5pbpga0Wbq8BONwu4sERDjEJEMBhDbzwwjrwNjcadmEZFiT4llRwFQX44AWy cgTc4HA8ci1jIA2ygggIdyy+sg4hYAhPoQDloBDusAB/sgZLgAi4IAr3ICtnYi0ZEBREQYKI IArP4AgtgBNxYBAc/9EBSVAJiAI8baVHd/QlDLEOSfABNvEQgQMmQlCCRvB1LoAqiJABWlAF iMAREjFXspBjjqAQeuIQIVAMTxEWfJBj5eAX61AFcgQYoGMEk5AsxgI7oZNLClE9dndmrlOH Z1EKCHAFPVAGf9cDM0AFfzgOpuAApTAO1yBavBBB9KQHCXAcq5UatuEM1TQctdAL30AH15AZ zqEZG7AKKKAImtBbwHFokLc/9FAD7kQCmPd614EZDTM+dPBmpUCLuKcCTWAKzcACEMUxSvAx vngEucYCvjhdwBiMvrZQDgUyouAx3wV930VQKqMePZMeyKZRZpB92YdDA1JSBSIhIpVf3P+m IuIIbdMjf1dzLVnERAA2f0rkUlWzIA0gB0fgAmsADFSAAMCAVBZGgPyYN2uENz4SgDziLFhR BUdQB0jgcMDSANLQAXMxJ4dhBu7gBUPwFHIwBCSXEFiBEiphAafgEWzQEZASFDzxCdeShV8C DE8XYldhBIiiEm8lCntAk4xiV3rCFGhgE5W0Dl6wFEO2EAvwPNQwBAuwAOGwADfoAicwBnDg CEhmDYHDBdIwBmxglbkQOH1QD0NQB2agR4ozBhZ3DmzILK+jO46FFbvkOpjTN7ukEFXAd7+g AWqwAiBABRrQA4FIBRIwCqbwC5bhBOLCCwCwHRejB19waLmFDLj/8VvxgGrN5RzXcAEJUE70 sArHgS+vVUDulAfdsAGYUGrXwVyp9i3w0AR0wAqsoAJwMA4qMA531pozMAu9RmvkAR7hoQTH Jx66KR7z2F0NUA3koV3hoV0D5UEc00HOV0LDljImIwUW0B7scVEvhCCNMyBDYzQB8iDjRyHP llNGVH/U8iCIAG7hRiIj4iJhcyFNMyC6QGvClgEZMAlXgHhwoAFrcA1w8H9100ZIMmECWYAW 5kaXhAuVwA07ABFWUTvXAwMOCSoKkHBrIityQHVmGDhVcAIOuQ6TIAQOgQ0dUaAgShASMQRT cRZwoIQmGRh8MJIwgAT2UJCoE0mltBDW/6ASOtksrlOBnxA4RlAHT8EQrzMSxWARVSAOuQAD MXAOOwEHPLoO87AnURACWMI4PlkFLrBxiBAHDlhHNAgHY9A0elJL1CKkWoUYifE6KjEtu8Qn aWUKCNADzbAGzbACzaANzUAFrPAHeDoDkmEKliFa0rEZDJAAozcap5UatwUK0QGZDfQN6fNb s/EDzjBA7ARcigAGN7ABm6p53OEYvEAH8DAOzzACowCIpjAD2VAKKqBZM1AKI8AtdCAOkBAe vBkeLOBrSgAevnibxsgC1VANucZdwYh8IQRC0FgJ5nUeLqMyFmUG1oce8REM0Hkz1uczyDY0 3jdf2ypE55ie2f+mRDnFNR2CDdhgBr7ABkfAB0ogBpNQCjaAAPBgA99iA5sFB9rwp8uACAAJ oAJJbwPIr2wkN6I0O+tQDdyQC6jgDp7QcryzKlvSAoiAEuvgDsVgAUA3BOdAFNNwDkYgBl+4 EB0aBRawgKvSEVHQAEGxKk43Bi7gCZ+QBBWohbhgBNzQB2fhBVowBryDFbUjR4wwFx+wAEkg CxngBS6AlbDUB3VQDNPgkwoxDwvQVy3gDtJQDMXABqAiBq9QB/WQCUknpVfbAbjwCMRSDC13 LEgnDScAcGYpCpMgDdYgCtOQDnXnENGjYirBZj5Xd7qDknxHBTUyBaTQDKxAp1dQBoH/2wOs UGeSYQN/WhkAIAzYoXl5sFuQWBrDQQ7Q8BzcMXsXcA2XEFv2MhvAMUAk0A2g2Rnnwx0AIJgj gASlMAOsQAVw4HecRQV/BwdUUAo9IAGI5wmo+gG4eqvAqATH212+2ABjQLzGiJvFJx7VMAai IKzT+3vBqF0l0wCiYAZ48nzxYUIm05xGAJ34IZ01dCAF8h/eCF8SYkXkho461UTf1l8WIgXJ amuVIAdoMAm2p2qKOBnwIIujoAGA1wx1QAragADZsI/31o/8eTcBGTcAG1hI2CxVIAoaWwU3 i0tMYRAOsTtmeA6T8iW01Ad9MFYqkQQ1hkdhdaY1ZwRsoDk7/7hxuIA6iBFLuaMsXAEYqtLD PBLCB6Eqq9NLqbI6gsEkChGxJeg6UmXDq2RjgLA6ano7jMQSuUM6iEGHaJZVoNSzrDNJFmGW ZjgKeggCK1AGcqoNK0AF2qANPQALb8xZfdktDkAZ3KGIm3EGw7AKo4GKsGUHl/ACwNAEnlAK MaAKXQAGwGEvpPsbl+kKGxAAobkdzCEuNgAPpVBMBAzHdkkjhptZ9gkHrKlZrDoKpaADx8er R4C8yXebHjMGwCgKuemLthoeyyvL1CWsuca9ISNsEDVdIiTM4NusUpAyNjNRPaMzzWZt8fUg R1R/5eYhYBMi0xgfLsMCLtC/TvBcNv9QPrZnDIaoWb+Ap4CnDWtAClOgDaRACvhZCregVPdm gBPcjwG7YV/pPM6CxIYBSUfBEjNnO3FBOjHZZGcVR15VGGywBNPwCQ/Zc0LaN3FxPMPTEkDs O2JMOkPMOl1FZFklCzF7KahDSrIUSwpRO80SZlPGZAdNS5MkpHXIxVp8t4Jzpm4oFxKBEwgw A81ACnU6BXZaBmusDWncDHDsAIv7PRLwDHzGiRjTDfijCcPhG8iwCu0ADi/wAt2wCpfJL6Fn TnaQAKrLGWfwirTHC5q1Waxwp5wFC6QQiM2QuICXl7tLu0gAB33JCrFbBErwQaz8i7fMyh7T AMD4QblJnMD/yTGiAL3BKGzBOFDjAWzf5djflb3QBwPcy6zhO0PMdjTN9kLthURVMGDl1q0U ogvmVQl8IAhKkAHAAAcqgACeoHil4GoSAAescNtw0AzZMKdrQAbrPAUgkAKDO7h1QAbJwAsz UAX+qGECiCRuRM89gkrLcjuqAz1pWrB81SRdVQUiDUlQtWGMMxaIYxFxlGKeA8JF3DdUHD2i oBOQwgcRywIW0KUTXd4HDUupQladpKaTNMRUfNKlY5Yehkp627OCJChB1hIu/TWYc2Z9m1aj 4ASjcKcgYKduLKewsAbJkMYaQAZU8AusMAqG5wPGEFrzRE9nEA/6wlqy4VqK8D6H/4YMOdAb XzAMr1cd8sRnzzAOo/ALgSu4P70GbZwMgcsK6cxZGrCHhstZPTAKIOCqptADkPBdvgiM1XCb u2p81OXXuSoeVl6rIRAe2SUevLzlxCoKyihC3iVeEDVQNmMy+zEfFuAyAJKN0MYiUxQnUZQ1 0YgPs8wFOadqz2V7mAwPEoAEfXkFsHAFwGCnyQACk2AFc9oMqdDOZLAGaqADIEAGINDO2gAC 13AKwYAOZeSvA0jPD/zASbJWWiUGDWHTMSklNhHFdeRhgNAHZYdKcWQGG1hVdxQFaKAFuZAE WvKABHtWMTlJjMXEX3IpRLEATGEEk4ITjCI4SRA4Jq1HU//GSHIBPYhgEEN46/+sZVJ8PX9S ZozzwVaFKWYoh1cVOFy8YoB0Zlos0yDic1XQAwhwCoRLBsENAiAwBVMQp81wxoGrNqwgeIYn AawAqJPhCd9gQfYS1bvVCflSejXQG7+lCEFQMZyxHc7BC8KAeKfwC7DAh2dcI2kcx3Ad1FdQ I20cBmvdA3GqARJAeBowAyMABx/AMeUB5l8+5quNfIIAnNVA9Lc82EpP2NUQHs73MdOLfI5N HtNLUJFdUJUgCijErJ9tAXYOX3n+IAxy2oQDUQdwAmugAvK6apXhzT7wDKYgAXk6BHlZI+jc DJh+6ZOQAingDWSQCqlABt5ACqn/cAhTQAaHAAKHkAKkkALNwAvvXEYB268QLKD1XCRVtVWL IwZLwALTsA0WsDhVkAEWIAgCYQQsQEctMAZ7MA1jYA1FVwUV8Al2ogUbdwTF8AzFQAFyhArF cA8VoAUdwAYTiAsWgHS9IhS44A7lQA1zwAbrMAZDoABeMBVcwTqAoZOoEyZf8qVnYWLg7gJo 0HQ7sKBRYATSEAdxEAbX7w4VgAZysAc8R4P3MARyIAf3EA1gKA3hgJFTARAdJi2AkUlaB1yI 4MzJNAZViyhRcE3EBTHiRYsSJ96CCPFWxYktIFbpgYAVqTVlSGkDkXJNMxBTqIBo1mwKKyrN qMAa9WuU/4NSI3gNdRKDAYMzG2rUoGenUydNdhTR+kJLUR5MmDbo0cPACQBe14CoKNUsG4hs zcoAmwIzhc0yZdo2WzEFRIpssKZ4KcOqJpVsEnqwKmXKh6lnPfyMOdLYcTXHR1goOSJIsmMW mDM3ZuxY1BHKDTAfqXSkQefSoBukc1zaTLp0rs3wIW3GjC4zFvDtDhaMRzApv4MZaeALd2k0 LuBwGtqc04hrI36ZolKKFayaPbR5s6mN1CFSU7y58DaJDBlSpCZNIkVGDfoUKLWpIZVijbYp LsSBuFYqGKAo0AEEHQEHJBAQBAUsUEEDCXTwQQQNbAGXiCqKogUxPMilAzOQ4P9jHSMqWAIh QI4wBKEilkgCFzZKcQQXVOD4hMIMDFkHFxgqsCAKRqIAMAo0hJhIlAqqiGIMe6JYJ0gKxThx oiOk6WMdGO7hQ0lEbqliHUTQWAKXdeqIYx2SWPCjmA6UFMOaKhCR45U0ZZkQEVRk4eGAYtZp IZM5EFlnSWvW6cOLMSKC4RQ+EHGBzT784CYJiWSpogFPtoiiimwc+XMdFrjwY5tgovjoogpv vHAjkCaECNU/ESnJkvNIAaGOFM4DAZgy6kqmkBRWsCmbvtRiZVjERjBmqDOcOOMMBsCwQxNN aHlWqgQ2CGADTPTo5QwEromulFIAyyYVYLLpgQwQ0Or/oZlCQFh3irNsaqYHu8pIKwVYWDG3 jGaGzQaoUeiYwYXOlBDtM8oKpgwyy0yjrDElIKsEEiUqbqxh0TprbLPUlGCtGtFaEy0dMyox uRLb8DHZDCPwYVkXXSoRxQxRqlFiHhBU4AUBnne2AR5egiqslGxGAYGKnHoYV5v7mqm1jkm8 m0QH89Aj4wNZU/Dmg1RSAO+QVEBgL4VJpugaBFJSCTrUAAc0MEIHE1RwbgYfPBDCCgGhMApq yqFwnVSiudSLHXDR+5FXyAzDkYgQ8cIRSQlnBBdRbAQkR1R8RFCiD4SMgg8vcrk0AyLQ8GIB MKnRgqIMKkBjmnrq2WIdOeJY/yCcDtYRQ0VZXFgAEERAuKcYDxAB0wUPwvkkHE1/jKIPIqbZ IZM81+EizxZkoWaaKBCpoHBc+jhlhw7qsKaPc/roIwlF9zjHDDjs4ZJwPWFEBZWPPIL01L1L xeUWUWmEIi3QE5dKooFmpGINL9FG2EAAC6bhpRnaaAYpdgILuQAGaVT4xXSEggBeXMAJF7jA BsjRjVV0oxtZ2UALXxADJzjhGmRhRQ/gABO1pAJd6SIFd9YQk5r8UC0V9AIwsFMTYPSgDA+k Qg96YEQq9MQ6v1gDFI5AG9AcoWCQGYMolDCGhI0mBKDR2GMokw4WZIBhWtxYwzjDxs40QBRY rE3MYv+GMtFUAgYZ8MM8TmEDBHAChN2Chw2CZgpT4IQb9OIXCOQyBVKoZArw0WF9vAHJ9Hgj PemZgKy80x72kMEF32lP2tjTDPSkwgVtmWGo6uY2uMmtQQmSpd3gViCJ+Cgi1MDDOuQkjk8M TlNgOgKcsCENFkCqCjJaBzbiNyQ4ycIMpzhH/8A0D9RFoUoWqMIpNLWORf1Jdb6MQgheUQVc 8KAISboIgHS3hD+FM3ifQEQGpIGKDmwvCUngwxzSJJF1DARMs/DAjTLwilzIAhE6CCYPKiCE P0UpF+twBzzXcQQPdEAOXkDFOkRRimo6DnIT+p+F1pG/Cq1qVRQSiagGeKP/+rUAEWtAABxI 0YxJpKI+KZhAKpwGAjJMoQziSMEU2uJItcACGP3K1y9mYIxjgdAJDOjKVlrIFQaIcCh0MEUp wkAFn/ZgDVOY2lnUExP2ABWS6bKVFcpABm2UIRVpMQte4DCKdfVgHKUYRWHowNdpZOZhEIsM ZR4WAhaEjDGbWewR1riZBggiMyDbjGEf4xjGmKwxoqhEAyBzhJL5gRoueMY1eEEHniFgBCNQ 7QiuAK4oasMLE4TJJMSWAhAoIG2eZAkpCqENMqQgbO0hQyqsEB9Oom0NPm2grMjwQ1JMYGyg XMNK1JMKBKhgR26L291u2aAD0ZK7EZolqXTpjgqM/+EcXLhnFHjwOJYWs6OiGMIY0uGCUwSz Cl6wx5AMkblBTUMQFJrIOvpmKG7MDg3c4MI0XCANTY3zb3yahjQWsA5SoUNJFRVnObrngk/c SA7F6GgcpFGMYmyBwBI5hwLmMA027CEa6zhoJjwwBzZQSCH12IM0llAFSHl0D8WIw4pot4d6 nIMNY2qBHHaE4Qnl76QWiggB8wfAjigJTHq6EAEJWAWaNpAmUzjE1CZp2/Sgy6j4UetLloq0 WyFtFDPY686u8Q1zcEUPyQJhdAQGC3PVpC3pQZsoF9gMTabCtpdcoFvJhttm6CCobdFBIepi FhCwIi4z4CAV6MBVKmyjMf/VGMNmbFZYxzomYpm1LGe+eIQxHgESIWAMY774sMlk0TSpLk5n G4OGVJQWAKlFgBNWew0fjOIZpugBLMJwNFTi1Ca4/UBR08O1uxzCGz7tZK3SFrYJkKEQOqiV N4RrNvigbZLxISsZOkmGSbgA3s9NBVlT0QNeaBdA+6blguQ2t/HezbsPChCpfrSOJHQAERVS Ui7BVODXIAIRoFOxwy/C0v5VWZcav1GBJ/Ijj4cEw3tznnk14rwfHXwiEin5xXPZowwU1OIx elGFSN4/kG/cvH8SCZcJCKaGR2RTFskIRPSUvS5zaQ1Bi64LXgJv9bwkPLYVT3HHjJ64qkWo 2EH/S77AtVqiDMUGI5jBKAATxGasQc04nQAItL2eFbBnCmtIhguKOp9JWOE+N70PTcRmE5o4 cSaAkQBhTKGCEZjCBZvJgGHBeOvQsIDWlHlEruVo65CBpmKlASNkRFMxg9n6CDCAwRhEY9tr EBssPwsLIq0zimZgcO5Pr+4avJEMlKiHPrZHQ3zOk9NSAr8+ZN2kDgidNhBQ0IFMI4UOrOAd dPm+umRrfnp0Wl06CGO7PgJ4eBkEXoF/l2645FFEAPRxivxtbxPp0Z/GELs4oCLjets4S82f y82N3OGby/iEcmle/rs/izM/wxFAHzGcnGMEvQEJXFDAJDAVg4NACgGE/3XYt3ZiufOTk1LZ FA7sQKOTBZ6TEznhQBBsKREkk0lAgLEihfcghRXQqblLj+6oDx2YgmQYKx2oiWQAPH4RIn1R mmyIolL4BVaYM2CgAgqKlzW4giVCiZcAATVYLvpwAZxCibeQLnWbJG0YJblAPn5Jjxy8qXIx i2ywDlaggwqYAT+gDEEwLCXIAIipBj9wLIUZgxCohDBqgNL4LFEYg2rgIlWbIxj4jAOoAzgA JAAApG4ZikL6NEQaBTg4wnWJNGmroG1LvvRAt0ycBPCwD1JyOvUQJUsSpa4hA/Cwu7Txmt0C jwkoBJ3qIRDQoUoCw/OoFbHppJvKNzPgvoDzPv9bWpBXAr+4QZCPU0CKIMb6Sz8AgRRAYARI AQnzoz93WrljvL8+0BQBioIkGIMtADlZIEAjKAfjicZmpBAFbEZS+Ti9aUZ+25x9q8AkcIEd YLge6REMZEd3rBBGOEdz5JEJBECGO78okAVckAUAsUdj1CWQ+8eLMEiCDCgEMAVSuILneq69 u6lJmICnk0EFmqQp+ABH0oZksK13waEKoq2YyIa3yAbfKiq5siS7SIVU0KTwKIQya4+6k5Xy WCCoWwPzGKWWSAZtoJpmSAaxSoE3Q4u7KrtRaIJ5YCwlYAE7tBhViwzC+ozIEIXP2IwjcAQW MAJR6JQ68IShIDZeMCT/eEi8gNEApJmXo0Kb9BjKWomPQsNFHdI2s1GDQ5gASDqEMts29liD TkwPF/DLn7wu9CAFF9Ahn4TBxnxFcUiP5aqkyWS+6qsDUriGZ9hF8gpGCOlMWfq3fyuQzTFG Y0TAj7M5lmtABNzHAiQwBDHG87NAfpsIc0qT9ZOFc0CIHjnGiUCchGqnCEHAk9u39OORyRG6 ClxHgFoDTQGydQSEFqgCdGpG2VSSKugDVKjHKEAFPrAAwzEcWbAAPugDcySTDjACNqjO8NSc dkRNaXTP4UQQcEKAZECXVFiBD0iGMlgD6bIuFgQq4DOPptGGSVCJc3mJSXiLl8hBd9EtHQCG /yv4oZf4IQVaA+dTrnPTIZXYpODqGgVCD5ySOgP1qUmovZF8iSnQBpZ4IJ2AA1P4BaLxmMvg Iq6ELDmwmMk4tapkAZo5gHnwBHggtkCyAUCCB04wBToYhb4KA1awLSOaO5oArpxKAe84Nwvd pPpYCbSJxTXIhjVIAacrrupLzGb4AI08j0OolUMwRXlb0+WagMUsNEUzvlP0RLR5xa7xhlgk A0QrLh0Ygc3UnO6iG9H8RfATxgJcx8kxAheogyEQx+4hggxAA2moA/BBBD9QAA9wgWrCBSOY hGKQhm1Ykc1hgzwBhGVinCjIgGnABTlwHR6rgGoKkxnBhQwYggVAg/85kD9R8IRiCIcF+J5y BBBESIUliLFXxYXd2QETswBGUAgcE4ViIIJFjdY1kAY2OIc9mIYboVZ7OIdiiAYF9McayQVU OIFXlYVMGJE+GIJ5LJ8dcBNukL8Q8AQ2SIIqqC8C5Df3NJzTVEhdak0QRAMEmIGW8NPcu09Z WQm4qqDmMqoTvSkDZRqmaQn1YAlEsy3j0yRxqDe4SpuxgqQUkLRXrC7jeq5vS0xZOQQ1sLaM BFkdGqXgMtBVug9tYEJtuCtWkIBSeIYsyIBKyAwwkgMtikqDSaM3jMoMgIIxYIwGcAcXGAUE GDaeScTm4IVHlAAfUBq8egkn0qmWoA+gMkX/EHABoFKgxTQqsDkEK/gO37MCb/BLUsDMTepT 85i78ggq9WAPb/BJAw03tTulMgMbMpNMbcCa5FMDRbOPqQmbE12uMM29GdpFQoWlfhu48YOQ zUXGzZGIA7CHB6yDGSGJYPIoOFmHeYCn8NGCDsAGLwDP8nmR/MPUVo2DPEEETJWFN8FNNJgx 3Q0xGIgBI2ERPqCcCsgcA0Od8MSFTFidieCDDqDAFYGfF1GImhMFaViHHtGbdXDOiRCfHQAn azgHI2Cv6T2/DCAUdLqRKkCCOPgMF4AnORiCDBiDOBgfXAiBg7gQF4iGfySvCPGRfRzIiJCF b2zP3lQSF+AFKvgh/00Cqgo6BPHAvaBSA/6slaqpLlxUu/wIWQMlA5VoBgUYq0kQhwlNBjTQ AaBi3FLET+FaoHqTXJRIO5mkoDDF2fjIhlO6KXmrifmoifNgmrRjSzjQgGzYBsaADDmYjIiR DCWAgmoQBTkQBTFYgxkAIau9WkBaLUEdgVGQAFPQAA24giNKO/cY4owMDwNNs+pCiTUlm8S1 glRgPplcAzUgvvCYO99DiRnWqfOoLhDAPRDQJDXI46m7j9+Ly/RYj0lwGvMIG53qJLtLgVrh Gh0QSrCpYzqAhyvxPsztPtA0VLvBG3fcHFQYAzTYAxdJpx5gHFyohHqdKcFxuEpAgmLYA/9d pt29OYJpqAIXqAJqcIRHyChckC81ubDvDaY3yRwCO+bkHUhqsIb+eafmVZLrUS84GIPIeRHK 4QYMk06lG19cwBRHmKkFqAKFM5Kd64MRQxMjgIPxnc4qkIUM8CcuIadq8ILckQUxiIZvNGD3 FFiBPuDzK8iTUxReaAK4esWxrZW0w4+9e6D7ICtYaI+2UAn8OLOXqCAndA+pQZc39ujzmAJa 8Ya/NT7xqA8KjuBSKiruMBsyq4k6CLe5mAShVFhYUIMyyAkIyoY/iCJT8AI/qIYMOOoqboCb KQUVINLUK0vT0lofABcHcKK8wI64YtEKKoOoES41eGSPPlH0UIP/+VAPlj6E6qJbMbVklOAa b8O26BKuu6BJSEq0ZECuCbikmUyFvN7SQ8A9UEoznCqbMnMB80irSE6b93A7HXpkeLiGXfQ3 URa4Uh7N7nOQ5cRHfLCShPAC/XrliZDlNnGBbbiRPhCHXAiRHWkBLmABhlsHgcgAeIIBB9uB ByymXCAgMSiHB5yEcqCSGDiHP5mH3yaSXKDAinrASMmAesUF2N0BUbgHSzGCUhgDMnkcDBMF jgIQAkIEBUgEPzmC0EGEONCCjhqDYli4LdseLonuiUKDbUAEYB6Td5UfXZCGhAoBjgLBIrgw gztgcryQGyHIFpgchMMwEOSSBv4FtO0V//iQtEuKpPBwGkgKrkuCJOASqlhpC7VDJfp4rg/4 W7ii2TIwULRG7AbCrcekGvZwASuQtGYQB1d0j7uID+OzCZTw0wUyJZbwFReYRKbBq1FQ0hny lp05ckMaAeZIyxHwgWegg1P4hZzIhiOUF7uwNkzSgbwmA2/QAW2jyZkED/WY20mbBK45D8Oe BE0CPvCQyUPgGkVjD+Rr6FSoQeDqU5RFm8A0Pka2PXn7cEVjQfygYJrVKRdYJbI5dD1v5Jw6 BRVIh0QF5VKm7LeZpe7CsAOsiH2l1A8YgmLYzTUIsSgwgiFABURoMSI4AAUQgo6SAwUgAhfY g47CdCXR7vHtEv8tYMZKmIMqIKBMUGY0WACFEwUPqANpqAeEgAES0xs5UOb6WYcYU4A9GF+N 8oA9YAM2OIE2iQPhFpRwSIKDW4cQOIcFOIFwmHVZgIF6WAA28JOL+BMYWAB2RwiFgoFPCIEp +cZ12IEQIB8A6QDzLEjqHMg/sTm9+UbujpSC/5sC+hMwQQMnmIEqJJu9o3gh1rs1c8W5C66g 0morZaCoaSCL1fPq+qQ2ThtRNKrvYGExd4Et1NIUePFaXINCCKpYLIQ9tmRvk7QmrDcKciS4 aqKqNoURoIMjb45roIMRKIyyg4UZmIFT0ABWmIF2QRpWqANOY1IQgIOnj0QvKLtTmAH/OKAC FciGU4CDX1ABOCgFOvsFs88GVhiHp2+GXzAGts8GOpsBtucqFVCBpC+FayBLOvCWZwAaXuCE UmgOFQAa6Bh8oTAtYxuKqCYKLR4kIr38nbl8nuEEQOKFZ4B0WCJNYbylSfdFCAG641QSmMJ0 AjKV1t8ykuIyAecRntOIKnP41ZQpghS6ioCyPyFIgvwR/0NoofM/AsqlCZHA41cSEBQgoNOl 5v/93q9AAgrogid4DLt+MAlooUP4giQnQOD+CwGooCs4i9iy7yen7S9INEBLoLGByT8tXjB8 nolqzTfL+79a1ataYnOCzL98gACAgBevgfAQIEQgMCHDhDYY/3JCePAhL04Er/FyQvHaNScc OWakcw0eR3h0bNAZYfKkyZWlnpmiExPJqFGlRoWpWTPnqB5XfGrQpkAbUaGT1GhTM+moGnVo 1KlRI0bMB3XqMokpIeZqia4lMnh1lyFSCXeRIsmJBCXt2EjVQjw6+0hU3GqPHvnKCwNGpb34 YPiC0afP3j5VsGErjK0P4sbYeEDGtg4QIHSUK1tGp1kzZs6WMYPeLFr0OlyAGEWJgihKC1mp 161LzToKriixOySBjau2a9a6d8Omnbp2i9jCo/R+3dt48ddJZJcm/jo4LlnrluPSHbt0alm4 Wgw3fV12a0C/kyDaXX069+zercMOLv+cUfLutV93Tw1IVm/w+5EjB5tr/L3HX3UwdIVWW2ud FUkGj1RTjVu+uFWNL3ddCIModD3CB12+VOIhYIIR5gs+55gBgxGLLYaPYYnh41gfPCBmBCI3 0nhjFYhgcyOPO8Lm443FIbJOC0cWaWSRVRhZ3JFPxneLkd+tI2WQt6BSZWlb1gablUfC1sJ3 vokpW2pPojmck989aSYg+gFY35H87SdLa/whh2Zx/DnZpp3gDdjadFDeYiZrRxr6JJ5RfHaZ Z5R9JhqknY22WWWh7RYFZbis5huA5NWWGy5J1JZpccbJlt19tqV2yxh+bGPEc4A055qRtoFn 2zqMZOfHGKz/zfMJb5rqql+ZxRbrWq7+7YaarrEx2xyumhbX5W/XdXBFBXt4oEAFJpAKSKmT efqmdbu59uatsxEbm3UtULbop7RdV52713lnb26xMbIpe6UCvG614vKaXRS8+vumuLhk6SWx 2cWnG22nGhmfmREbuWhy5nE3LbFnerkqcVEKB3Cqv6V6ZqZwomNoai0fF6CqxCrMqMtvorOx m/l9irOhLS+as2ziaroqZZrC/PJlQ1/qaNORZlYppZ5Jyhmk0Tmramlv3nfqfUksnR01S3Rq W8O1HS2LEXDskB0aRJQ6sZjSrcNDbEQjgkoLiBQjbHVa18ZIcUyunGoVO063cGnw/1XhHSJG pFOFaVWgsqoFpn0qi7iIePEJd4iEEc3BRliwnRnXAYIICxmkYwYfaJ/G9NLdkVt0zbcfjLmm 4nKN+Zuo+c60aacB/zvm/ap3GWUIm9Z8pkRflikuvKL25nC7wzk9lUMzjT2cRH9MM5zXo338 cLgrbD3XCrdcM8xL19w8r7u7L5tlu7d/e/TW73w/zta3j1GVwV76jqa8pzXKaQpM4KRGM7VK VW0zmmJEvygYBVEYogO7IYIjslOHabigGNKwx/NwIQYk3KMYucDFEaRxAjR4gIT9UlflqhOH YsDONnUohjXkUIxiTGMMcujcbuZhDdtMQghiO4EoxGAIVP9EIQMKEAUfXCA6YjHiCB7owzmS iAsjVKByRvCCNXbjDmt0YAx7oJwL8JAEM2TwTRmYRqbS1S4QRGMd6BmDF3JBm7HVpg99jIIZ KnAOWfShB1po3mVmeECi3c12vJtgamZIwN0xsoKPDB7vhtdI08xweBQ0X7+KVrDoTZB5nKSf /oSDp+VxsmYfMyD0xve99OmONvCjH/eg90j4xY5mpSSW/9JHTPWZKX/h04/PIiVAm0Hqmcpj ZqNsFsFHXdNq2XQgBC2lmeGMcjei8AIqrCOOaHBqCErExTnusQX12CYTdMRFJe5hAV6ZwQsk tI2qEDGJCkjjctFbxyQ+kZpC+rH/BWJYwm7QUMZ1uGAB6xCFIatghDA4Yh2ZkIYR9DY05HkH Hy5YpCyokUc/sMEFqODcCr9ThTFIQwveyQQ6OSfQ3w30CgqoB0+HQLbqUGOeVfDC5dwBN/pk QAvr6IAChjCEOaCieo0k3yaNiclK6m+SvkTeLz35y9Psb4I4HSv1TglWSZZSfdP8GMKCib6j fS93XlWr+JipPqPZ9ZI+o+UB77NL/n1GgMAcoLl6M6n+YcZm9kNOy4QGTWLlapeCFaADO4NA BGqTat3kpmxUKYpXVA6inltHHfzGOUeg4jliU+o6QkBO2gw1o3EbTxT4oM+TuYAF1cnnClsw tuegYQkd/4CoREVxjx0M5hwdeA0fMkHG2ADnCENgQwfkuRsjSOMcRKhCCD4hh3p0ABEZmMM5 NAo3QFShDsHIQBw0F55UIWIN2+BOtjKKCGosMgp9gAMf1uEOHFaHC8XAl3uGB93rCQeu+QFf nFzjyF8SsJG8+6rypre806znwgZEHv12Q0vYKSxT97NMDmmT1kvGlZiGeuSK+Zcqv4bPkakh VV09DMBZhg9t3nNxd1qQNOS4prEDjFNjgZZM+xmZUbJAh4+XPMAgv+yblwKapabszaqFBmqb ldT6moWLceYCNmsQ1jrWUA5SjdEe18Lvdcb4zi97YQsHhqkfCQkHEm6nCpNA5/86dFEBybXA uusQwzRW6gJrrGNtfFBNKjxHjTjABgZEHQ4iUgHpKnxAqYjwLREQnd419HcdCvB0CIvUghAQ QRpVkM0Y0pG9+EoUNnrGw3DdMYc+GEEM97CHLPLJgj4cAQlz6NiUbKNBzJ3KlROLE5CHNp7s zG5RcCWQe/CV1wb/bjgEIpadohO970zSOPfiNr4E1eM3JcGO5rZOs8/UmlzxDMZmQtSZeMaq ea+KOelyzXq2BCBzwWk5O8uTvVmTHPDArLGz4ZNm7NQdoTn2m4q1JqNa5uPNBHnJiuXmlB9I qaZ9PDMPTJoBo1CFNRBhGmhQgBDQs4YhANELHdz0a6r/IY1i9NcMOyxGHfhQqOnIQQFe4MYe zvEaMd2CGo5IDQwKfSQ5GLQFUIdNBtgAGyNkwgWZ6G9x5OCCPcShA4jaDTYycIIQVEEO6bjO S/sQaT50GxFyiAMfEMGHc6QHES4YQ8f6sMJNWedw2jkcImSBiCqcow+Hd41Fz5F4h4qJ3ZoK U/z2ZPABuWtA/85wydinLt7wyzr85qdY/8PX6HmHreVbmrRyLK7eyCt95u6O9YzjVzwVDk/Q hact5+17/EBH8nhyeLPdox/v8M+95BFywAOVM4vzh8msPNOPK65xx+JJghP/psIrzuR/VJxR 4Lfm1CAVwfKL3Mrd5Jp+QFmy/4rBpg5CCFnEIBad49zHONaj0ndWRSqDtQqcSIyu7IaU2N9r 5FvRpAp4jInE1J9tKEv/bc06mAEIQVpndRbmZErEVEuCsca4+JAC9I2AGEp05JJ8kIdxEIjk vYl/LMyyQYfBQBcjmKCqJE9hRcFyfRKAYI53tMCs8BJ+GBOvcEdzjEfawAvt0IuZ6A6qcA28 HR33pMzEuYx/CAcUgkx+FI7K5BL1uYmC5YqdGJndLNib3IKyuAzIBEMV5EqRvYkFWAC8JZbE iUb4bdzIWY35WQ0FOcqVXVNn2EaIbeBvQIw9lM6tyEd2pEeVkM+ZJAF43EJuVOHvsct04Ioe 0csgQv9MvmkgA0IXeFDHOhTJp4AedYhLm9wbbYSK9TAgAz5L/y2husBJlpAHdLhM13jKmHBP FeBDjtkfdVBJaTQHEeZL6mggZcAbjB2NvCHjLcWgLILKtNBKmbSgbPQGceRfEUTDkRiBGaRG eqVD/qWhwmDhixENFi5LEhwAot0NvMlJEjxHGu6MAUGhoKxDPbzOOqABC7yJIL3OzeRgHcRB H/CBB/jBYkVBHLiAE0aTxG1cy8SBCURZkWmWH1pWZinQx1lW+Ygbd5RGoTDid0DRB57JEVBA sd1C4BSHH3jO9aDKbOQKqvzcOV7QHBAOrTCHqRjL/YkJ+yHiOqTXGJiaoNj/ixGIAOHQS3Cs y8IMzzmIg+S4my5uD0mqh3jogEHJozG1XnPUY3EcwbANB/zRmP0VyW60Hg+ymyygwRVNSZKE 2GwAh8EBIC35Bjx5jbUAJQ8cSddUwc8x4OEhZQsIXtJtV5YwiWxYVEfJBipkiRmkQ+VM3LFY gEWNQailAwvwASgagRGIiQXwQR8IQnmJQjeywKp1QDRQgwlkJmWYwRgYAesQCyJsgwXQhiiw AHiggjWcDhekAyJMwyl4wTRUQR/cAxekmgt4o5BZwK+IixyIg4/ZzDSMwinUAW2eQzGEkDgY gWWswwHUARoUQxHERj0MARzsQTBs2fpxE5eFBshx/1nz9KRZZgmNTUe4mEeXtIA7TIMkViXE iEEZIQKWOAdtPIeRUA6qyBoUgQdFQWYkAoeHRcF8ZsmoVAEfCBRsOQIfQGaiZYlgpsMYqBnD ZMA9fAJtrkM6xAELlI5tpEMupEM0RIMYcYEnbAMJcc06YGYcFEFL0sY5WEOnvdk6XGUVlIMF WM8R8NkRFIMHiABt4kI68FwxUECXEMEOHIAKGZc1NJUI1FAVoIE0KEAdLF2ikVExKIA0dBAu sEEd7IECQFUHuAAKucAKZYDWSUM5TMb+yYE0ZAIMhdY0FIMcbJTRJaQXnIAcnEBwZkAIFINS fRkS7AEbxEEFGAFtyEEdsP8BH+wBEZRGBtRBOvQBGnABbTjUOvABGvRcFXQACMycAnDBcLpA OXxHHHhB3Y2TPUTZySjAdq1DsLGAKJZqFBiVbcTBKRwBFHXAJHgB3kXBGIkCbGTCsOFCCMDB EcRG0mSCNchCFbiACxjBOrCBOKyUAvyKPh6RyVUAGxSHp7bhpeCCNUhDHVRBwrXMGBABeJwD MPQiINiWBchCPbhAa6yDNWRBbcRBFkBnFIBf1ChcZnFcx1FNxz0NKN1HbXTAPPBQMXgBQ31Z HSzACRRD5aACeIbDEqzBEsyKl+UONVTAAuzAHoQsLlQBHBRDPXhOPjnCmqJTUJ5DHHADFI0T Y87/wwLE4/m0ABpwwwKcQy5YQB3swDpkwBNBFHh5aTQwAiKAgLAcQR9NlDRkiXHtwHC5gCMk QRVcgSBwyskOVwiEUa/dmYV9Yzawoxz8LNTOgQaRVjTkRmnhghwQAWwtK9QqFSCcgweslDWs 2lBtgW2QEb9JmpytwwloweF1TmzkAhyMwRfdWacOmCzsgORUAauyBhrMExsM2zoI0uISDR+E UXbsAB+o493SBh9BURwQQXp0wBqwQWqkw89GARt4geTgQhzUwTqcwymsXQ5eAd9tVDqACW2c AEOtg+3qESJIg9WxgA5ExznAAW0e7GsUgyC8jKEkQW6lxhitWgtQbxJk/wKndueA7QbfXGAU hIPv4gIMLIM38tFLIocg9YEocAEbcAGc/koVFEMGsMY0HEB1oAIcuFoUjIE4XGs0oQa4egFn mkkIZEFqsAEwoGfqDIG1FoMJMOAY6MAtAEI4BCyUvU/DIk17Vpk3qR82ZUbuRGhpJIE7aIEG 8Y1wgdHTcusCoELUhlYR7GfJBM5uUIM1kMqqXtoptI1tuMASGME5+MErrJoeDbE7TVQFZECc 1QYqcEERUMM2LJdD7UYS+AE3jMEYKAEU+5cXjIGqAgerJgE+eAERPGamrA2uPgcqHME8lIJB lZls2ZbkGMEpvFPBmJwc1wY+8BciiMMCqGIc3P+tP6ETnNpDB3DBAlSHFTUPYtYpEZwCrpaZ +Kabzg0XLlTDzx7vouFCssrq2kDueJVGBYIdHHhOEqDBNlTHFy9AOEBdLnQSG/wsPOFCEQzu BVVAmNlulnSADizdBXGDBtHud3yCvIIR3MnsGvwqUPbsEBRBlpzANHSqBDOuIKzDJ+gAFH0R peJCBogDSJovrRgKLriA34xRH9BGBsBNoLnvAXBqlxQDF8THGHADoPEBHHwrQq9a+EwvF5zA t+6dvO6NB9xm5BbBN3rBEXBw8ZrJJ6CzzHqBGUBny4TAc+pXSaeGBZT0OixAKqwGLnDBPICH SPMAQo6vG4pcZcEwlvX/dGWEivbAxhQHSXB1AAx4Qhx8whiEQH9lQDEwieBuR5OookKVA5Po GZ5C8jSvQypYw2bygRFYVB2cwDlQ1DuJQinEATV4AFTqkXYE15ZkgDSEl2DWEFBK1zSE1xrk EWygQh+4gxc8bR+j7hrEwWGM2d6MmXXkUx/c77KuDFDKMZrx19TqhoDBxp7VhhloQR9AVSzH apcg7jitmiDhanwJy0ERFdUVQwfElsWyKpmqmXjdrWEP1RikmxGTis1VjiQvbqmcA1GJTTlE QQgkdHbcUGlQ71KJaW1WMwSXgprpowkUiThAGgXG2ToUAaT1mhdANjWIp90aZmkB5RDw3X25 /688Q5HeUcAk4sI+dvakQa05d+dAG5VuwOl8lYbeTQNa10EGZMcY1AHNoWELCJI43Eg4YG6T FAML2EkmcMOrVkEPmPQF1UFO9wYifKkRc0MGwFvLsME8KAsM7OoJ1EEltIZ4DQERfCcPuAYf 7JDzPp+hLJmQ6WH69XTEXuSUDXWpIAIXeK0opsLkonZudPc6yIHU+nd8EI6WDJpUrwMef24V ZEMUrwMXFFprvXXUUs4YnIKGhgAX29rZaE0RLIFuwMDmPi4mH56nqqpAokIIBFQVSJojWBTm WsAYfULanYIQI4J5r0MlcO3axAHkLgkccEOmTnGR4LE18MEX95fexf8BxAx25n5HHwxBOdhr PZDpNMQBGlyBcOXuJ5SlGaSQNaD4m8GANOwBvCZ2og2Bmr3Ua+sdNxQDGqDBHMjZnpKXkt+c QHKJOLkAmCJadogCz90pd4xBdiNCJqwdgltDloQAEoR6pHYKIoQAGoDdk4/BCZxABtSQEWSU yY0BG7jdaIbxEcRBBpzDLYBHFZzOkVgAFNmzu0UoKqgL5YAiGR6OmNzIeABlxdDGtg7nYXaJ xwAIwXeb6MUHoLxbzvjYu2kMdLgjo5x0ngBNG7KGxIth9Z00wgkWRSpPUFuZ+vV4ZVhMxcjs AQzBDk1CmifaHtTBJP+2HHgAEMUBHsSGGQT/1A/COy40uzU0qpq1gBgv7muwwTRogRCggpR8 wjRQcQgIQRVYAGrpUTqwAS1KzN15JCqwwSd8rq6cQ1OrWW4wbWlUARtU5nABdi5MteNpdYIC 9ko1zF1DTG4fzr4IB2OmhyIqJSIcgXBai3c0jG9sSZG4PV5uYrh4mVm+Rko2SXYcPXDQoipG AXpMB+a0RyKSz6pgET+pS73AhmmDicD8C+jLEoJCC3Swfr09YRTeDLJ5ZOvTi5iISXDMTcrA hizhynoECHSFCbS4opnYyZKJvDUqFngACsb3nssIzbJYnM1wvMgnnNLoeNQ8zfbn+KUsVVVf XR2MAbBtd3z8X3ZI/4lsfIkUxGgUSMleeomYPKL94b524EJK/mBs7AtwwEYSGCACAkSUdS3W RRFYMEoLgQZxrSu4LsnBdbgaOqTocOJAigkNJnzo8CCugwQFdnDhyCKiiQYtRql4McO0aB02 DmQZJaLFlxUxgpxIEVAURhQZueypsSVIiiR5yhIpsAXQlS/XyXLZUaRCXICCNozStaBXhyRJ AiLYIknLgy4BPV2osAXcjiBZrlRoMC5erAqDBtX7dCJBjQzX2oxiVWLHKKjiRpW1rsrAuwRx yWqst+NdxFGqmGmBziDow6ETorPaQRAqxQa5ZhYdBbRC0Z87ogME2jbr21xt297d+3Zv3//C d+/G+RDXLYeojBixcCttw1s1W6ByG9XhLcrJcaFt2dDrU7OoUHVIq/ziwSS40oZVCZghWpHR BddVelE1Rq3SXYpcypol7H7CaKTuLILoJe9uuQo8ivqo4iK3VoqPoi2qQMS+hihzyIIdRAKE kbba6g8XRFiw4DuEKhpjCQgVqwyfJVS7C5cjrBHJKoo+5AhAr4ZqcKUkEEHECHswZGmjBYNS MUSXSHJJuSgWxCssyrQacDoDnSyorSaNqtKlKtAwwjCveOzOJWpYCMssNHYgcJ0M+ljHCGn6 iCurMVwYcC7MouCjDoRW62izKIzwIhjddGsNttc6uk0xvoLjjav/4tAZrlLegPuN09/AWyws M5YoCBWLvCOKIvRE+qm6n7zy7suJRCFivbaSy+i9ogIMCyRq4tCRqgOjCykkjHQ8qiE0zhm2 v7Wc1JGHYwfMSCcdVfoIqPSE/OSc5RgM69iGqIlmpbGMxUiUYiKKsCuChnThk6SOOkcaaqbp yag+vLDAI4dCKKZUrNbhgTGG2IMssnU6WAeVYModqIM4iIisoeZQSas1yDZKwoKGqhBlDDJx skC1KCyoQiAzLOADxbCqGCOOMVTDhWWKasZlDAvYYIGNdATCRRQ+Osjg5G0wXMeCmaooophp +GioHhZ2xAWNdJJYOh1EpjHinGJKqcMR/4bG8AKJYjpOtxgPWDAIlROKKQaNOqz7F40hBIli gTG+WqcYI6LQN5hJMx18uK8wDe6rTIW7lFLGFS9u0uEuBc5Vo1oQxRBE0kIk4fkaulg6YCFi uAoI/XMo4XXweXmOJIq6qDMzrMMFldL5MIJiIo8w4oQ41jvWCBbSya8KFlgwY7110umMD+Wr 4OMIM4xoKIkjUDyiiiSC56MIEyzISYl5MhhDEJFer8IPF4hIhyIzPhCHiI5rrKOOiSkSxQVp ipiZPVyK0JsPRHBHMaTBBdRlQQEFVBgaopGEPsgNMnXIBWRSUYw9uKAYE0lHHU4gjfJFYQxz qMce4LC2r3xlK/8XKUI4qiCNjgECEQqoxxiKAYdcyIIPXvjEGIbQur7g4hwK4BYabhQFNHjB EV3znUk+YY9iSM0ouXCb0A61g1wswAWo0NU69vC0DnDBGutgQx360IEFTEMlRZBaB6iRAVy0 0AVbYExGXLCmXIijVNZw4zriwAVcdKAOaKAJoCBUBbmhog580NoYJnICRlrjHC3ggw6ipcet 7G0aYxCFOMhkyEo4UAfy88gYVZJDlCFiDyxogTVMMBE26IAmRsJFCMSxjjuibAdF+AoixMED 2ISGUrUxXFB8A6lLIeZSv/zl4CrluGYWk5mT281gNhIFUcyhA6hIxQI+UYcGdgAExXD/RBzC YA9DbkEgcljC0KaxgzgYQjVy8MIe2NCBSnCDD4CCp39wkQFroEIU3MjF0Cqwgz4coBilc4E1 ngeHcgRpHS4QAiLMMAdU9GEOuUBEEaJBuzoUY4IRnYM91nEOueHCAhBcgyOq54UJikIaC3MB uaqwUP8AYh1oWEBDXGCkaeQCF3LQwjpgujBRNNAIdejA0JawLoe4YAxJEIUXrNOHKwx0pg45 wJvEgFSX4kJfKEKDEKoiBw90oApe2MI6+FAB5riUYTOtJgoNcqhSxeFeLYiJVfoAh7XCKwkd GIM4IvI6MWyDImm1Rwfm8YmGxGFUAU0HHzIwB/8kZAxEcEgm/wyYBFQoQBARwcms1tEHcWiU CKoEKzBIioa1rSMT2xhYIqvZEHlyoXkNia1A+shTn+FETXHqKCLqIAqa4ZEPeGzIEaaxBi9A iLNXAYQs0FCBUsShKutIpCzS2ge8NEQUcrtFCLJQkBbEoQg5XRMQdVCqAE7jBIGSRQYysI74 BaUF0/BlZkhzFcU8c1CiEfAyofnMxkEOwc0MynraUyrMIQIV3aoCNdZZhTV4CBEnGe86H7iF 5FxtbCgaw1kZxsCBjUEa/KtIFTKAhC0kIQNDpVkG8aHWhohBCP1TnUvbA5EO9CEDXrCGkC5M x446xB0UQEQGvjkJlPwLQjCIaVq/t/9ka5AInE/7Ge1CQIQraGFhGRiCFhwRZiRIQxoKuFdH EIEGx17TOthIsSHt4Z/AdpVIVIVMD1CUCkeAN6ZmOEU94nCAOHRGGhBKQleRc1lciOHM0oCD J9bqjodyRq2IKMYY7te6S/qPXJBZQzrWoUb2ZGAJQXVh6VDmFjkUY0jU2EZaAMnImojBHkwe Umodoi+SiiFe6/DVOohrhtr2BxFTfRNnIeLPP7rgt39LpAtQVgVxNG8dwP4fWKXRvHNYAmWZ eCg/XVuFYtRX0z6jMlQMEl7G8GEIVSAIGrCbCWssLB3PPVQwcDHGyBjSn4lLyF0Mp8yb6MUv jvILMQPMTAX/d8rhzrzURkp10Q6YIdFLa6cYtHBREPBBSI2FDBEoe28eZKAYbJjqWo/ggVJl WMnrOEKKdVSFAd7Or0uexsL4gM1DrTXb5VhYr/fFMD4g2h6oEMM6ETGJ8iWLXOvpgwvOIQ4I MXEdNOwAkbiBVtYiWQSaEwkqLvzHI0xWz4P1LIo6EAJuWCCEGDpH1C/iWqmm2IFzsIBJULIO d3yibXFABQz2pT2/ZjvUcuBG7dSKNBfkgsoMq4NsqwOV9fw8I5kgAiIya9QKkEncKsnA20Wb BCizFQQa5Sh7xjANsBaeDbQ6Fi37gAg+SAN37p4KIvpQDKsTe4NGAHIWhc2FPMtW/7vG/p1L 0EC0pJJaYn0QxRCKoJw9CMIncfDCazNM6tJ6IYtvVEA5xoAGOBwBFX00GCJOsKYqzIEF5j4C ZIpRhHREpWJhIPk6wjGHaRDhXiDDA8QBDfZgDkpnD+pgD05AAZAvBLzAbwyi9yKQ4AjFvzhC IWTBLx5j4ABkNR6OK0IkmgjH4W5jJy6iEhItBFwOEdyh41Lqz3KKXJJtGuJI1Bwh2bygW2io VOJkD9CKGl5hdm4w2ZCApDKACBbGDOApw7aBOXogjFbMBQTvnixADqQBH85hDaahdMDGITKM XFwCEeLAm8ojolDiEQ6QSCyKj8QhHeQpANECEa7wCDKgDv8s4BzgYHxcwAv44IG4gAUWitjQ YBrsEAepxc3YyqLQ6g7ZShrmwQXiCBF8hXMwiBB1sLR8zwX2wAOqoAPSQQGKwQulag7eRgsc q6Zy69QWIAnaow9+KutCMRrqYYLMyAOIIBwWYOgUhq2WoBh0sUQ+Adv4ACVwwQggiwJ+ByME S2be6BMWACW2oyLUqCdQYQw+4Xt0ZAuSjjzWATqmQz4qIsK2ABEognP2jtgmIi06oj20xFh2 gmHI41o+B0kSg0D+47tIYiJUol8owjJ6QhaEZCowogq4IAMy0C8m4zCyIiQc4y7MIlIwY+GC QgMdTgRHsAQVjD0e41jw4acmrH7/RCGTUOEA7CEtMuDPGqIIFqAHjWAapsERxiAamCOqaEfm rAG3avJziicOLMAMZMYIcgsVcgsRHIEFbqdjepBhjGAMHIFijIAN+AAVSIbYIiMJZMEM2YOi imEXeREj5hGH/NACcIwfK2KpFsYpjqIqAqtUnEIlHkNhnKJBXuIqEURz1PE9eGIXk8ch2vEi EMRVgGU+riUwUeVcHGIoBFIZXUUpOPI4NiI80sN8PmdVwvEeLSIJDgANqIFiFqZBCOPTjKI/ 8M8bT2cq8M9MJJMxL9BgosNVRuRTMgIujUJHOLAr4BJZ8qIi3mI0dnMiZKEDRAFF5iIDn0IW EOM4K6Mh/yxjM6oiPg6jBTRwNL7iODMymqDpcDLlR4wlOsKlJ1CFetjqE35QJZInPEtlPnTC VYZCIN4jLVqRnz7lYF5iFwkTM7/yQIJk6MJrD0pFSLzjJzgSdjLBgnagxzzzL49lPQS0x7Ys QnCBERiBweRyPiNTK+VSPZGFPWgiQd8IPP8SQR/DViA0XBihA5wiQoHTMCezIhihWsLiOIQi JJIARGjzQQGhO5NzKHCTRD2CKLjiQxhlHUZkPYhlVcKlNhnkL9siRxbSH4XCH29qK3KEPZn0 Jz4xn9JheX5TOX8zJueEVx5CIKxiNl/iMV5mCLigW9qCrawBDQ6geQ7DVqqCD//szd6sgZEU Jg4UYAj2IAOqoEYfxToHFeIG5490ojvBQw5ftDsRxCW2DiN+xz8DSz2vcmPwsy8Hgi+jQym/ kGHUklqopRzL01oOxlXyEjDnMT0tQhYwpCE80zkf9D4sAqcYkyICa2P+UhlrRStFJz2R7SJo wigYDEEilES/8JKkcSiGAkEbhBEewynYdCtAZC5R9KaMglpXoihAxFHlk3qygijEUC7ZdCJ+ hEQFU0XoQkVGYygggkmxwikEYl10hD1PKF5JJF5TEzEqY0jFqBRK4QTqwQOeAQ46xiOG7hnB dFfElCeW0ylkQQ6ewQUU4ASQAATm5ARKAQ34YA9KwYD/KtIlZOETTCETzsFk+wA4peG6+qBj qSFxQjDiCJVSIM432vVQe00c6iCq4kQHApAiOHQj2JNSkQwOTiGhLFVhzMC57gWs5gEOkKAU um0Oi7ZoT+EevqkI/hUJkCADFmYM4KAUkMATJEhWy5VELgJAI8I1f+I4PLUwF2Y93iMd1yMc 6bUi/NJznJUnfmc7CUQobHZYfyQJoiIvUcVFXad/DDc8HzSFlvVHuqRh7bYVeRUxdcRWXMJc I5QttiJEUGVEXsd8bCVEXkdGlzWFssVym6RzDUIE/QtEQpAtXhZIbeV1h6I1RrdSvmJ0ObdS RLB1DYIPTuEUCikMSqG+vIYa/7yga10ADjitzcIWCdZgevpAAZDgHpCgeTdXFsJBYosB21yi HkrBD1AhfN2oUtIyDp5hHtBgCgG1A7ygFJ4mDkphHhKnwEqwmLLTwPKXWufz8sDWEwwqA0rB A9YSEXLBQtbDWBO3tHoACRTAE8pMR/qgB57hGYaqA8TAE8BMDpBACHexD0BAGnLB9C44F8yg FF4hF7Bh0harBJDgrCCTPWfYWF9ngSEUPknUdHUESiUUQm2XRIPiR3a4Rpe1h3+YKCK0hk23 S3yYiZn4h7tkdv+WWou4inW4c0WkcUVXhy2Xc3cXVUj4hkd3WZtYizsXKDRXVy7SdyNUioH4 dS/Jd/9rNyMv0iJ7145LcI4HNY8NFURgYAhKQRyKYWsVYGEUoBSKAQaWCpEZiQ2+ZgyFCLae gTPH4GvWoShStIPhAQm40CUs4ApKIQzgoQ6MwHaBVBb2NBOAjGBLB37/LBxKwQUuEjsTTMEO hzg0ZTfOFVj05YErwAo9gStxgQuQYA8yYQheoc6Q+CLGYIPlYINp4mquYBoy4YIX5gOQoKlE AQ4qAKh+NhXuYVnEr5NdAAl08Ryst9I8wdO6+I2D9m9/lJ8W2FiPmFofN4hRiIyVGHJ/OIuX OEJrhYhB5EeE4odSyIx3d3DaeJ9fN44JGmZF0HzcuHMzuX9Z96G/mHMbN4r/R1dG+4Kg+Rl3 1Vh141h36digc9eNB+ek61hm9ZhQ+7g1uOIcSgEORKEP5iQtG3kf9wAJxkAWHrmWDiQTRHkT dWpIwYLYxGAPTKEU0iEXwuAZ5o8NLBYVmMkCzIl2oDYd+qB4RWEdFkCWpxUER5BxFufAIIdx zrWGqwBs2cAL6uATSgGbOuAKXGwd5OAebgRxraV6nZJsBAGtKggVMuAZMmgdsnlUqvqrGsIP TsF3KOIIutkRYpmqOqBjvSAVkOAU5mBIcRg877l2jbWKMXeIa1iHUWjs4FhzIZqMp1WJS9d0 d3ikXbt2pxVCgfSfrXihYTakf9igeZegIZSuROR1/1N6pHHXn+W4i1k3uPH4IhF6W2F7drvC uENwpcm6tx36pbtbO/fYt5mJjWnapruFKjqWBY4zw0ohBJKgqtegD+b3APiPfj8GDjKIW9ch YzMADcYAkUOgD4aADsY3DkyhlhgheNhHFsS6CPAhBAj2QTyADorACFygFFwWBIHDcTT8N3J5 UzYlaP0DFcLgHnaAD+Cgej3As9bgHuqsg5tKRxhMFKA2DJ72GVzOHbA3G+DgGYRZ6TyhHtjq FASKImDgFBo7CaihFCoMfg20tPqgsOFAxQkac407iEk7W+B4o7M4uDM5jd84uu+Zoklbyx3X cSP6lHFbRBRaiWdbvLFbpP9t97UpekS4+7iTW3fJeM1VWojpandbcc2rXIg9F7pnt7aVe6XD G6K9285hmtEzUl+GwLuWs0RIKAQqo83CgJEQIRPu4R7CYA+qoFXDYQg+vRjO4VgVBpltuouq ogqQeQgUIA46IETqNBwOAxFCYA+KQQEyQdT9pQBPQBTsd2Y1XH83Ra05pTd4mUSrAARKvHqg 1hBk6h4+YcLuIcdwZho+IS1c4B4WACKMwHpJqjyGhoAXpho8OBo6tnUk1B3uYRoUGGisdoTO rFTE4B48wBq8AImauHHdmHf1fJ+ReLcfGuDv2Z8NfrTjuLaDNotdW6Ht/HH/+ZS3dbT5edFt 25//M9m0uZvN6TjRKVqOYZfO1TjgC72iZ1e4VRq8Bf7NH73RY1qmZzaiDfVhKyO/pUUWXBRa c8RKb54tyHUoStZe+TVx8ng0mDRaD+M4RaMi+XdxzjpyBOfDLYVxMNduOyAXgSoJdiAcWAoQ OmAH6oHsWYoizoENDBQV2MARZgcX7MGKgGUcm50P0v6bh2Iq+/pHcmEHks4wEViMETqIT77k +xxyKyW3NxrxuVi3qXu25Xy0hVuJv1jkczvMsbt3a/5Hx9yh2fi1Lb+IQX/R99mO59j0Rb/k Dx1Ij1hzj367wVviexvjH33mCfV+yxrxkzNkUeg4NbD3c9sqnDNkrULp/zOF+DOF6Zm+Ih2u Ohfikpj+mB4u6ifH2Dtc2a3+rDlacJ34dzbacLtfWuPTi3u0iOlKRrdcd6G0i3eYrQX/cZFf iNecrLc8iHX7hwIdt/+5h4e786vbovUfIAAxwgWIICOBBAEBisLo4EGGCgciFHgwYkWBGB1W 3NgwosKPGD821CiSY0iKG1GGTBkFZEeQHlPGzOgS5kuYOHPeVIiO58eeP4HmBCo0Z5SjgGTh aolLFtIoThNGQQfV6VFZshZmBYmUq1OFUAGh69lyLFWrXxViHSt2qtifb9nGDSq2p9m4d82i M7i04EBcuAYOZMiIYeDDgBEOTtIwsESCBJnKLP8IeaHDloKXjpxo2KHlzCdXNvbId2RljUv7 Hi5cUCDExxkPt5YNuqPtg45fXvSsceTumZ491pxMMvhKkTGJ386oPLTt0NBpRsd50WPRnG+x g7xOPSFO7guvtvzoNK3aFlVbfDwKdfz67QthljXK3mf8vD7t4rWv9y7M/npld5pBfjHUmUCM JXQYRIAJNpqCrTFoYEOdLfVaZ4896FdkreHkXmIOugbabAUZphpKiSUkWGu4NQZYYJhFBCKB m1m0knc68TZTTTwmh1xw1SFHk447bmaccDo5N5124GnHU1FBDpXdXE5+V+WV/1UpF3h2CUWU fXXpB1eY2X1Zpn5d9nf/pppmFfbQm5BllluDghlmZxIluulZYgoFxqKfE70ZaIFyUmSRbMLJ 1FFXp2FEWIN+MWcQb4s6OBpzP6JU3Em9CWnjp0Mmyulw2hkXpU1CTubkqdixuhOWRMnFpJNe 1pXkd1tiSeuU1bEKF1vAwkdllrEGGKyYaM5l7H74AdUph42N9leGvfn5oqWI/RmtnxJJK62j Lgb6nEO0xUbhby0K2hulBOF526ajwjuujkAaWlxKvj3Hqa+o3mqSp8C9iiSSQf6m65I7Ngnm wtgpzPCUv856paxi8uhlxfBdV+vFG0OM5pfFpulfl2BFa6Rp5J6rqWLM+RYYY/jq1rK9KK+c Tehm5e5EKc2HTtqroSwPurNo8BbpkqlA5zikqiSVOmaPyf2bqdO6qurwrh5DfHC/Ty/ccZZY a1wmw00CaybZdCm735MiVxyrrW3nBVRAADs= --------------2781446B794B-- From mpi-core-human@mcs.anl.gov Mon Jun 9 11:53:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA08056; Mon, 9 Jun 1997 11:53:42 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA26008 for mpi-core-out; Mon, 9 Jun 1997 10:58:04 -0500 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA26003 for ; Mon, 9 Jun 1997 10:57:59 -0500 Received: from denna.cs.msstate.edu (denna.cs.msstate.edu [192.208.157.80]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id KAA01852; Mon, 9 Jun 1997 10:57:53 -0500 (CDT) Received: by denna.cs.msstate.edu with Microsoft Mail id <01BC74C4.3915E700@denna.cs.msstate.edu>; Mon, 9 Jun 1997 10:59:37 -0500 Message-ID: <01BC74C4.3915E700@denna.cs.msstate.edu> From: Shane Hebert To: "mpi-core@mcs.anl.gov" , "'Dick Treumann'" Subject: RE: Its done with mirrors Date: Mon, 9 Jun 1997 10:59:36 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This is great! Let's make a T-shirt! :) From: Dick Treumann[SMTP:treumann@kgn.ibm.com] Sent: Monday, June 09, 1997 9:58 AM To: mpi-core@mcs.anl.gov Subject: Its done with mirrors <> I thought you all might like to see the most basic Message Passing system so far to identify itself as MPI. It does not appear to include MPI_GET_VERSION() so I do not know what level it may be. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Mon Jun 9 16:18:16 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA11528; Mon, 9 Jun 1997 16:18:15 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA03469 for mpi-core-out; Mon, 9 Jun 1997 15:22:38 -0500 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA03463 for ; Mon, 9 Jun 1997 15:22:32 -0500 Received: from denna.cs.msstate.edu (denna.cs.msstate.edu [192.208.157.80]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id PAA14206; Mon, 9 Jun 1997 15:21:57 -0500 (CDT) Received: by denna.cs.msstate.edu with Microsoft Mail id <01BC74E9.1C8E96C0@denna.cs.msstate.edu>; Mon, 9 Jun 1997 15:23:41 -0500 Message-ID: <01BC74E9.1C8E96C0@denna.cs.msstate.edu> From: Shane Hebert To: "mpi-core@mcs.anl.gov" , "'Dick Treumann'" Subject: RE: Its done with mirrors Date: Mon, 9 Jun 1997 15:23:39 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Just out of curiosity, what are the latency and bandwidth numbers for this implementation? :) From: Dick Treumann[SMTP:treumann@kgn.ibm.com] Sent: Monday, June 09, 1997 9:58 AM To: mpi-core@mcs.anl.gov Subject: Its done with mirrors <> I thought you all might like to see the most basic Message Passing system so far to identify itself as MPI. It does not appear to include MPI_GET_VERSION() so I do not know what level it may be. Dick From mpi-core-human@mcs.anl.gov Wed Jun 18 15:20:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA07259; Wed, 18 Jun 1997 15:20:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA22679 for mpi-core-out; Wed, 18 Jun 1997 13:54:22 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA22671 for ; Wed, 18 Jun 1997 13:53:59 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id OAA06430 for ; Wed, 18 Jun 1997 14:53:04 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/05-30-97) with SMTP id OAA43465 for ; Wed, 18 Jun 1997 14:53:17 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564BA.0067BB04 ; Wed, 18 Jun 1997 14:53:00 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: mpi-core@mcs.anl.gov Message-ID: <852564BA.006740DC.00@watngi01.watson.ibm.com> Date: Wed, 18 Jun 1997 14:52:33 -0400 Subject: Minor correction to MPI 1.1 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk To the list of minor corrections to MPI 1.1 (Section 3.2.10), I would add the following (unless I missed it in the document): Page 85, line 36 reads: "specified by outbuf and outcount" but should read "specified by outbuf and outsize". Jean-Pierre From mpi-core-human@mcs.anl.gov Thu Jun 19 19:30:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA04253; Thu, 19 Jun 1997 19:30:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA26103 for mpi-core-out; Thu, 19 Jun 1997 18:33:58 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA26098 for ; Thu, 19 Jun 1997 18:33:53 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id SAA04926; Thu, 19 Jun 1997 18:33:51 -0500 (CDT) Date: Thu, 19 Jun 1997 18:33:51 -0500 (CDT) Message-Id: <199706192333.SAA04926@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: Final MPI-2 proof documents available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The final proof version of the MPI-2 document is now available at http://www.cs.wisc.edu/~lederman/mpi2/mpi2-report.ps.Z and ftp://ftp.cs.wisc.edu/pub/lederman/mpi2/mpi2-report.ps.Z. The JOD is available at the same location as the file mpi2-jod.ps.Z. We are asking everyone to look this version over and make comments by Friday, June 27. This is everyones last chance to make any comments about the document before it is publically released. Please keep in mind that this is still a draft version and will change before the final release so please do not distributed the document beyond those who are proof reading it. The document has an number of changes. The only outstanding issue is some details of the C++ bindings. Though the chapter authors have tried hard to get everything right it is inevitable with this many changes and this large a document that some (presumably minor) mistakes will be made. The document has changebars for major changes. Most are marked but small changes are not, in general, and some changes were not marked in the final rush. PLEASE TAKE THE TIME TO HELP MAKE MPI-2 THE BEST POSSIBLE BY READING THE DOCUMENT NOW. As an incentive, I am offering the one extra MPI-2 t-shirt I have to the person who finds the most corrections to the document. You should post all comments to mpi-core@mcs.anl.gov. We want everyone to see the discussion from now on. Steve Contest rules: - Enter as often as you can stand - Bribing the judge is encourage but useless - Members of the MPI-2 productions staff (aka - chapter authors) are not eligible - The criteria of judging is a secret - In case of a tie, I'm split the shirt in half - My decisions are final - The winning shirt will be mailed unless you want to come to Madison and give a lecture (at your expense) and pick it up. From mpi-core-human@mcs.anl.gov Fri Jun 20 07:29:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA00401; Fri, 20 Jun 1997 07:29:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA28982 for mpi-core-out; Thu, 19 Jun 1997 23:36:18 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA28977 for ; Thu, 19 Jun 1997 23:36:13 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id XAA02993 for ; Thu, 19 Jun 1997 23:36:09 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id XAA07922 for ; Thu, 19 Jun 1997 23:36:10 -0500 (EST) Date: Thu, 19 Jun 1997 23:36:10 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov Subject: Problem with error handlers Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There is a problem with error handlers -- what is the type of MPI_ERRORS_RETURN? It used to be MPI_Handler_function, but that is now deprecated in favor of MPI_Comm_errhandler_fn, MPI_File_errhandler_fn, and MPI_Win_errhandler_fn. Since there are now three types of errorhandler functions, MPI_ERRORS_RETURN, MPI_ERRORS_ABORT, and MPI_ERRORS_THROW_EXCEPTIONS will have type matching problems. I see two choices: 1) Have three types of each errorhandler function (yuk!): MPI_COMM_ERRORS_RETURN, MPI_FILE_ERRORS_RETURN, MPI_WIN_ERRORS_RETURN, etc. 2) Consolidate the errorhandler typedefs into one typedef with the first argument being (void *): typedef void MPI_Errhandler_fn(void *, int *, ...) This would also eliminate the need for different functions to create an errhandler: MPI_Comm_create_errhandler, MPI_Win_create_errhandler, and MPI_File_create_errhandler can all be consolidated into one function. In fact, if the MPI_Handler_function typedef from MPI-1 is relaxed to have its first argument be (void *), the original binding (from MPI-1) for MPI_Errhandler_create would be sufficient. Note that this would *not* break any existing MPI-1 codes. And if the Create errhandler functions are consolidated, there's no reason not to consolidate the Get and Set errhandler functions as well... {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Fri Jun 20 09:46:40 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA03228; Fri, 20 Jun 1997 09:46:39 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA04706 for mpi-core-out; Fri, 20 Jun 1997 08:52:06 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA04701; Fri, 20 Jun 1997 08:52:02 -0500 Message-Id: <199706201352.IAA04701@antares.mcs.anl.gov> To: Jeff Squyres cc: mpi-core@mcs.anl.gov Subject: Re: Problem with error handlers In-Reply-to: Message from Jeff Squyres of "Thu, 19 Jun 1997 23:36:10 -0500." Date: Fri, 20 Jun 1997 08:51:59 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | There is a problem with error handlers -- what is the type of | MPI_ERRORS_RETURN? It used to be MPI_Handler_function, but that is now | deprecated in favor of MPI_Comm_errhandler_fn, MPI_File_errhandler_fn, and | MPI_Win_errhandler_fn. No, MPI_ERRORS_RETURN is an MPI_Errhandler, not a particular function. There is no way to extract the actual function (it isn't possible in Fortran; there are no variables that can hold a function). Bill From mpi-core-human@mcs.anl.gov Fri Jun 20 11:02:32 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA04076; Fri, 20 Jun 1997 11:02:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA06607 for mpi-core-out; Fri, 20 Jun 1997 10:08:05 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA06598 for ; Fri, 20 Jun 1997 10:08:00 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id KAA03830 for ; Fri, 20 Jun 1997 10:07:55 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id KAA08509 for ; Fri, 20 Jun 1997 10:07:54 -0500 (EST) Date: Fri, 20 Jun 1997 10:07:54 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov Subject: Re: Problem with error handlers In-Reply-To: <199706201352.IAA04701@antares.mcs.anl.gov> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Fri, 20 Jun 1997, William Gropp wrote: > | There is a problem with error handlers -- what is the type of > | MPI_ERRORS_RETURN? It used to be MPI_Handler_function, but that is now > | deprecated in favor of MPI_Comm_errhandler_fn, MPI_File_errhandler_fn, and > | MPI_Win_errhandler_fn. > > No, MPI_ERRORS_RETURN is an MPI_Errhandler, not a particular function. There > is no way to extract the actual function (it isn't possible in Fortran; there > are no variables that can hold a function). Well, I guess that identifies an error in Annex B then. Doh! Steve: Annex B - 342:33 The type for the MPI::ERRORS_* constants should be listed as MPI::Errhandler, not MPI::Handler_function. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Fri Jun 20 11:49:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA05269; Fri, 20 Jun 1997 11:49:01 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA07849 for mpi-core-out; Fri, 20 Jun 1997 10:52:56 -0500 Received: from k2.llnl.gov (linda@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA07633 for ; Fri, 20 Jun 1997 10:49:26 -0500 Received: (from linda@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id IAA10163 for mpi-core@mcs.anl.gov; Fri, 20 Jun 1997 08:49:24 -0700 (PDT) Date: Fri, 20 Jun 1997 08:49:24 -0700 (PDT) From: Linda Stanberry Message-Id: <199706201549.IAA10163@k2.llnl.gov> To: mpi-core@mcs.anl.gov Subject: revised question Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Let me revise my question to ask, why is the type of the fh arg to some of the FILE RW functions IN? I'm thinking that the INOUT designation indicates that the object for which fh is the handle can have updated state for most of the RW operations, and that is the motivation for the INOUT designation. But if that is so, then it seems that could be the case for each of the operations since an implementation is free to cache state information associated with any RW operation in the object for which fh is the handle. I do find it somewhat disconcerting (as a user) that this arg to the RW ops does not have the same designation for all RW ops. Linda Stanberry lstanberry@llnl.gov From mpi-core-human@mcs.anl.gov Fri Jun 20 11:53:42 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA05378; Fri, 20 Jun 1997 11:53:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA07796 for mpi-core-out; Fri, 20 Jun 1997 10:52:06 -0500 Received: from k2.llnl.gov (linda@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA06942 for ; Fri, 20 Jun 1997 10:24:05 -0500 Received: (from linda@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id IAA06827 for mpi-core@mcs.anl.gov; Fri, 20 Jun 1997 08:24:01 -0700 (PDT) Date: Fri, 20 Jun 1997 08:24:01 -0700 (PDT) From: Linda Stanberry Message-Id: <199706201524.IAA06827@k2.llnl.gov> To: mpi-core@mcs.anl.gov Subject: fh arguments to FILE RWs Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Why is the type of the fh to some of the FILE RW routines designated as INOUT? I always thought the fh argument was strictly an IN. Linda Stanberry lstanberry@llnl.gov From mpi-core-human@mcs.anl.gov Fri Jun 20 12:00:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA05773; Fri, 20 Jun 1997 12:00:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA08295 for mpi-core-out; Fri, 20 Jun 1997 11:04:32 -0500 Received: from win147.nas.nasa.gov (win147.nas.nasa.gov [129.99.33.11]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA08289 for ; Fri, 20 Jun 1997 11:04:26 -0500 Received: (from parkson@localhost) by win147.nas.nasa.gov (8.7.6/NAS.6.1) id JAA18315; Fri, 20 Jun 1997 09:03:45 -0700 (PDT) From: parkson@nas.nasa.gov (Parkson Wong) Message-Id: <199706201603.JAA18315@win147.nas.nasa.gov> Subject: Re: fh arguments to FILE RWs To: linda@k2.llnl.gov (Linda Stanberry) Date: Fri, 20 Jun 1997 09:03:45 -0700 (PDT) Cc: mpi-core@mcs.anl.gov In-Reply-To: <199706201524.IAA06827@k2.llnl.gov> from "Linda Stanberry" at Jun 20, 97 08:24:01 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > Why is the type of the fh to some of the FILE RW routines designated as > INOUT? I always thought the fh argument was strictly an IN. > > Linda Stanberry > lstanberry@llnl.gov > > The file pointer and file size changes. If the content of the object identified by the handle changes, the handle is designated as INOUT. At least that is what I believe is the rule. -- parkson -- Parkson Wong Address: Numerical Aerodynamic Simulation MRJ, Inc. NASA Ames Research Center M/S 258-6 Supercomputer Applications Segment Moffett Field, CA 94035-1000 e-mail: parkson@nas.nasa.gov Phone: (415)604-3988 Fax: (415)966-8669 From mpi-core-human@mcs.anl.gov Fri Jun 20 15:32:44 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA10308; Fri, 20 Jun 1997 15:32:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA13498 for mpi-core-out; Fri, 20 Jun 1997 14:37:35 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA13492 for ; Fri, 20 Jun 1997 14:37:16 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id PAA07702 for ; Fri, 20 Jun 1997 15:36:13 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id PAA21006 for ; Fri, 20 Jun 1997 15:36:26 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564BC.006BB2D2 ; Fri, 20 Jun 1997 15:36:21 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: mpi-core@mcs.anl.gov Message-ID: <852564BC.006AA8C7.00@watngi01.watson.ibm.com> Date: Fri, 20 Jun 1997 15:36:05 -0400 Subject: Few comments on latest draft document Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk 61/42-43: The text refers to C and C++ only. The same applies to FORTRAN. However, instead of typedef's, FORTRAN defines three types of user routines. I think this should be mentioned. 65/21: The text should say: "The first argument is the handle of the file in use, ..." 73/8: I would move here the subarray example out of the I/O chapter (pages 268-269). In addition, I would move the remaining example on pages 266-267 to the end of section 9.4.5 (Split Collective Data Access Routines) on page 246. As a result, Section 9.9 disppears. 259/29: poor Latex formatting. I am pretty sure better formatting can be achieved. 266/8-9: extra blank line. Could be a side effect from the marginpar which will disappear in the final version of the document. Jean-Pierre From mpi-core-human@mcs.anl.gov Fri Jun 20 17:57:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA11832; Fri, 20 Jun 1997 17:57:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA16617 for mpi-core-out; Fri, 20 Jun 1997 16:35:18 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA16608 for ; Fri, 20 Jun 1997 16:34:57 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id RAA11344; Fri, 20 Jun 1997 17:33:40 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id RAA47682; Fri, 20 Jun 1997 17:33:53 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564BC.00766EBE ; Fri, 20 Jun 1997 17:33:35 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: jsquyres@lsc.nd.edu cc: Mpi-Core@mcs.anl.gov Message-ID: <852564BC.0074C376.00@watngi01.watson.ibm.com> Date: Fri, 20 Jun 1997 17:22:09 -0400 Subject: Re: Problem with error handlers Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There is a problem with error handlers -- what is the type of MPI_ERRORS_RETURN? It used to be MPI_Handler_function, but that is now deprecated in favor of MPI_Comm_errhandler_fn, MPI_File_errhandler_fn, and MPI_Win_errhandler_fn. Since there are now three types of errorhandler functions, MPI_ERRORS_RETURN, MPI_ERRORS_ABORT, and MPI_ERRORS_THROW_EXCEPTIONS will have type matching problems. I see two choices: ************ The current desing is that there are three types of error handling functions, and three typedefs of error handling callbacks, but only one type of error handler opaque object, namely MPI_Errhandler. MPI_ERRORS_RETURN is not the callback function, it is the opaque error handler object, that is associated with an error handling callback function. Normally, if an error handling obejct was created with MPI_COMM_CREATE_ERRHANDLER, then it can be used only with communicators (same for files and windows). The predefined error handler objects are anomalous in that they can be used with communicators, files and windows. This is explicitly spelled out (pg 61, line 23). No binding problem that I can see. From mpi-core-human@mcs.anl.gov Fri Jun 20 18:07:25 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA12082; Fri, 20 Jun 1997 18:07:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA16646 for mpi-core-out; Fri, 20 Jun 1997 16:36:22 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA16628 for ; Fri, 20 Jun 1997 16:35:50 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id RAA12408; Fri, 20 Jun 1997 17:34:23 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id RAA42135; Fri, 20 Jun 1997 17:34:35 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564BC.00768278 ; Fri, 20 Jun 1997 17:34:26 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: linda@k2.llnl.gov cc: Mpi-Core@mcs.anl.gov Message-ID: <852564BC.0075EB7C.00@watngi01.watson.ibm.com> Date: Fri, 20 Jun 1997 17:31:43 -0400 Subject: Re: revised question Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Let me revise my question to ask, why is the type of the fh arg to some of the FILE RW functions IN? I'm thinking that the INOUT designation indicates that the object for which fh is the handle can have updated state for most of the RW operations, and that is the motivation for the INOUT designation. But if that is so, then it seems that could be the case for each of the operations since an implementation is free to cache state information associated with any RW operation in the object for which fh is the handle. I do find it somewhat disconcerting (as a user) that this arg to the RW ops does not have the same designation for all RW ops. Linda Stanberry lstanberry@llnl.gov *********** The IN or INOUT designation in the generic binding does not refer to what an implementation might do, but to what is the behavior of the functions, as defined by the standard. If the call modifies the opaque file object in a way that is observable to the user, i.e., if the call modifies a property of the opaque file object that gen be retrieved by an MPI call, then the file handle argument is INOUT. Otherwise, it is IN. The implementation is free to update information associated with a file, even if the file argument is IN, as long as the user is not aware of this update. From mpi-core-human@mcs.anl.gov Fri Jun 20 18:48:34 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA12330; Fri, 20 Jun 1997 18:48:33 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA18421 for mpi-core-out; Fri, 20 Jun 1997 17:53:38 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA18406 for ; Fri, 20 Jun 1997 17:53:31 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id PAA06365; Fri, 20 Jun 1997 15:53:25 -0700 From: "William Saphir" Message-Id: <9706201553.ZM6364@nsgi3.lbl.gov> Date: Fri, 20 Jun 1997 15:53:25 -0700 In-Reply-To: Marc Snir "Re: revised question" (Jun 20, 5:31pm) References: <852564BC.0075EB7C.00@watngi01.watson.ibm.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: Marc Snir , linda@k2.llnl.gov Subject: Re: revised question Cc: Mpi-Core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Jun 20, 5:31pm, Marc Snir wrote: > *********** > The IN or INOUT designation in the generic binding does not refer to what > an implementation might do, but to what is the behavior of the functions, > as > defined by the standard. If the call modifies the opaque file object in a > way that is observable to the user, i.e., if the call modifies a property > of the opaque file > object that gen be retrieved by an MPI call, then the file handle argument > is INOUT. Otherwise, it is IN. The implementation is free to update > information > associated with a file, even if the file argument is IN, as long as the > user is not aware of this update. Code examples are a nice way to explain this, for handles. In the following, a and b are handles. a has been initialized to a valid handle, and b has not been initialized. b = a MPI_xxx(a) If the argument to MPI_xxx is IN, then b is valid afterwards If the argument to MPI_xxx is OUT or INOUT, then b may no longer be a valid handle. MPI_xxx(b) If the argument to MPI_xxx is OUT, this is ok, and the result does not depend on the value of b. If the argument fo MPI_xxx is IN or INOUT, this is erroneous. For buffer arguments, it is a bit more complicated, because the *location* of the buffer matters even for OUT arguments, and because the buffer argument itself doesn't uniquely describe the data. It is the data described by the (buffer, datatatype) pair that is IN or OUT. I'm not sure if MPI is rigorously consistent about this, but it's the general intent, as I understand it. Bill From mpi-core-human@mcs.anl.gov Sun Jun 22 19:04:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA25030; Sun, 22 Jun 1997 19:04:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA08273 for mpi-core-out; Sun, 22 Jun 1997 18:08:41 -0500 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA08268 for ; Sun, 22 Jun 1997 18:08:37 -0500 Date: Sun, 22 Jun 1997 18:08:33 -0500 Message-Id: <199706222308.SAA16218@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov In-reply-to: <199706192333.SAA04926@rap.cs.wisc.edu> (message from Steve Huss-Lederman on Thu, 19 Jun 1997 18:33:51 -0500 (CDT)) Subject: Re: Final MPI-2 proof documents available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk These are my comments. I have arranged them in the order: I/O chapter, miscellany chapter(for subarray and darray constructors), external chapter, and one comment that applies to the whole document. Rajeev /************* I/O Chapter *************/ 210/1: No comma before "and" (because a rule in writing says that if "and" is used to connect two clauses of a sentence, no comma should be used before "and" if the subject of the second clause lies in the first clause. e.g., I did this and then did that. (no comma) I did this, and then I did that. (comma) ) 210/4: The words "These displacements" were fine in the previous version, because the previous sentence refered to displacements. Now that the previous sentence has changed, "These displacements" should be replaced by "The displacements in the typemap of the filetype" 210/16: "during program execution" 211/47: No comma before "and" 211/48: "Furthermore, the use of" instead of "Further, use of" 212/2: no comma before "and" 212/38: OR should be in capitals 214/35: no comma before "and" 215/22: "that" instead of "which" 215/22-23: A better sentence: The values of data in the new regions in the file (those locations with displacements between old file size and \mpiarg{size}) are undefined. 215/36: "that" instead of "which" 215/38: should be plural "sizes" 216/6: A better sentence: \func{MPI\_FILE\_PREALLOCATE} ensures that storage space is allocated for the first \mpiarg{size} bytes of the file associated with \mpiarg{fh}. 219/42: no comma after "group" Also, "These target nodes coalesce" instead of "These coalesce" 219/44: Which additional hints? They should be listed. I suggest the foll. sentence instead: Collective buffering parameters are further directed via the additional hints: \info{cb\_block\_size}, \info{cb\_buffer\_size}, and \info{cb\_nodes}. 219/47: "distributed among" instead of "distributed to" 220/2: comma instead of semi-colon after "node" 220/26: A better sentence: Setting this hint is only useful when passed to \func{MPI\_FILE\_OPEN} with an \mpiarg{amode} that includes \const{MPI\_MODE\_CREATE}. 220/44-45: Split the sentence into two. period after "devices" followed by: It is expressed in bytes. 221/29: no comma before "and" 221/42: delete "the" after "follow" 222/15.5: "by using" instead of "using" 222/39: no comma after "null info" 223/20: comma before "respectively" 223/18-26: Are the etype and filetype returned by this function committed or noncommitted? A clarifying statement is needed. Note that the etype and filetype passed to MPI_File_set_view are required to be committed. 223/33-35: Why are the curly braces "}" there on these three lines after "vs."? In Latex, you should use vs.\ to prevent extra space after the period. 224/21: no comma after "operations" 224/25: Add "For writes" as follows: For writes, however, the \func{MPI\_FILE\_SYNC} routine ... 224/39: "that" instead of "which" 226/4: Before "The layout of data ..." add this important sentence: The \mpiarg{datatype} passed to the routine must be a committed datatype. 226/24: "by using" instead of "using" 235/16: Use "To set" instead of "In order to set" 240/30: Use "To set" instead of "In order to set" 240/37: no comma after "data accesses" 241/2: no comma after "open" 241/5: no comma after "open" 246/26: The increased interoperability outside the environment is also due to tbe external data representation, not just the data conversion functions. Therefore, I suggest changing the last part of the sentence to: and supports increased interoperability outside that environment through the external data representation (Section ?, page ?) as well as the data conversion functions (Section ?, page ?) (Fill in the labels for section and page nos.) 246/30: add comma after "process" and add comma after "page255)" 246/32: "Furthermore" instead of "Further" 246/47: "Furthermore" instead of "Further" 247/21-22: "on each file open" is not accurate, because the data representation is specified in file_view. Use the foll. sentence instead: Therefore, to correctly retrieve file data, an \MPI/ application is responsible for specifying the same data representation as was used to create the file. 247/27: no comma after "conversions" use "within a purely ..." instead of "in a purely ..." 248/7: Give a reference to where the "data conversion rules for communication" are. They are in Section 3.3.2, pg. 25-27, of the MPI-1 document. 249/32: comma after "formats" 249/33: comma after "112 fraction bits" 249/34: add "in" between "are" and "two's" 251/3: comma after MPI_TYPE_CREATE_F90_COMPLEX 251/11: "User-Defined" (add hyphen) 251/18: "user-defined" 252/29-30: This sentence should be: \MPI/ will only call this routine with predefined datatypes employed by the user. ("employed by the user" is important here) 253/5: Add this sentence after "native representation." \mpiarg{datatype} will be equivalent to the datatype that the user passed to the read or write function. (This sentence is specified for the write_conversion_fn on the next page, 254/11, but should be specified for the read_conversion_fn as well. ) 254/29: "Furthermore" instead of "Further" 254/30: no comma after "routines" 254/38: "that" instead of "which" 254/43: Since there no "two systems" discussed in the previous sentences, delete "on both systems" The meaning is clear even without it. 255/1-8: The three conditions essential for compatibility are listed here. Since the second condition is required only for Fortran 90 programs, it should say so. i.e., the second bullet should begin with: In the case of Fortran 90 programs, the programs participating .... 255/9: "User-defined" (hyphen) 255/12: "that" instead of "which" 255/18: Add "either" before "nonconcurrent" to make it clear 255/26: "Case 3" instead of "case 3" 257/8: semi-colon instead of comma after "set" 257/36: semi-colon instead of comma after "enabled" 258/48: There isn't a section 4.13 in the MPI-1.1 document. The correct section is Section 4.12. 259/1-2: This advice to users no longer makes sense, because the sentence before it and one critical word in it have both been deleted in the final edits. I suggest you delete the advice altogether. As it currently stands, it is wrong. For multiple collective operations to the same file, you don't need to open the file multiple times. You just open it once and issue the collective calls one after another on the same file handle. 259/12-14: The part beginning with "the current etype must..." is not precise enough. The precise version is on 226/7. Therefore, use the foll. sentences instead: In general, the etype of data items written must match the etype used to read the items. For each data access operation, the type signature of the datatype used in the read or write call must match the type signature of some number of contiguous copies of the current etype. 259/36: Use "as values of type" instead of "by an" 259/45: "\MPI/" sounds better, instead of "This standard" 260/20: "file sizes" (plural) 262/41: "The above program also violates" instead of "This program violates" 262/41: no comma after "operations" 264/47: "that" instead of "which" 265/30: Instead of "open" use \mpifunc{MPI\_FILE\_OPEN} /***************** Miscellany Chapter *************/ 72/5: Since this datatype constructor is not just for constructing filetypes, I suggest the foll. sentence instead: The \mpiarg{order} argument specifies the storage order for the subarray as well as the full array. 74/41: for the same reason as above, just delete the last part of this sentence: "for multidimensional arrays in memory and in the file" /***************** External Interfaces *************/ 169/21: comma after "windows" 170/2: "operations" (plural) 170/5: no comma after "system" 171/45: "user-allocated" (hyphen) 172/15: typo: should be \mpifunc{free\_fn} 172/18: comma after "point" 181/48: comma after "way" 182/2: comma after "used" 182/11: "user-defined" (hyphen) 182/12-13: The foll. sentence reads better: The value returned by this key is always greater than or equal to \consti{MPI\_ERR\_LASTCODE}. 182/33: no comma after errorclass 183/15-16: Since this sentence is not related to MPI_Add_error_string, there should be a blank line preceding it. 185/44: an error that a spell-checker will not catch: "latter" instead of "later" 186/27: comma after "Fortran" add "used" after "and the latter is" 186/30: comma after "call" 187/46: "returned datatypes" (plural) 187/48: no comma after "predefined datatype" 207/32-33: "and yields" instead of "as well as yielding" /************* For the whole document ************/ In general, when something in quotes is followed by a comma or a period, the comma or period must be included in the quotes. i.e., ``foo,'' instead of ``foo'', (It doesn't seem logical, but it's a rule in writing.) This rule has been correctly followed in some places in the document, but in some places it is not. grep \'\'\, *.tex and grep \'\'\\. *.tex should catch the errors. From mpi-core-human@mcs.anl.gov Mon Jun 23 10:10:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04208; Mon, 23 Jun 1997 10:10:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA15401 for mpi-core-out; Mon, 23 Jun 1997 09:15:01 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA15391; Mon, 23 Jun 1997 09:14:47 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id KAA12394; Mon, 23 Jun 1997 10:13:56 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id KAA33135; Mon, 23 Jun 1997 10:14:08 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564BF.004E2E3D ; Mon, 23 Jun 1997 10:13:56 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: thakur@mcs.anl.gov cc: Mpi-Core@mcs.anl.gov Message-ID: <852564BF.004C8A45.00@watngi01.watson.ibm.com> Date: Mon, 23 Jun 1997 10:13:31 -0400 Subject: Re: Final MPI-2 proof documents available Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk When Rajeev wrote: 255/18: Add "either" before "nonconcurrent" to make it clear 255/26: "Case 3" instead of "case 3" he meant: 256/18: Add "either" before "nonconcurrent" to make it clear 256/26: "Case 3" instead of "case 3". Whe he wrote: 172/18: comma after "point" he meant: 172/18: comma after first occurrence of "point" (there are two "point"s on this line) Pretty good, Rajeev. You are in good shape for winning the free T-shirt contest. Jean-Pierre From mpi-core-human@mcs.anl.gov Mon Jun 23 10:15:18 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04254; Mon, 23 Jun 1997 10:15:16 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA15554 for mpi-core-out; Mon, 23 Jun 1997 09:20:51 -0500 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA15543 for ; Mon, 23 Jun 1997 09:20:37 -0500 Date: Mon, 23 Jun 1997 09:20:33 -0500 Message-Id: <199706231420.JAA16537@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov In-reply-to: <852564BF.004C8A45.00@watngi01.watson.ibm.com> (jpprost@watson.ibm.com) Subject: Re: Final MPI-2 proof documents available Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Pretty good, Rajeev. You are in good shape for winning the free T-shirt > contest. > > Jean-Pierre Well, that was not my intention though. Rajeev From mpi-core-human@mcs.anl.gov Mon Jun 23 10:27:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04583; Mon, 23 Jun 1997 10:27:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA16001 for mpi-core-out; Mon, 23 Jun 1997 09:32:31 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA15996 for ; Mon, 23 Jun 1997 09:32:27 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id JAA24773; Mon, 23 Jun 1997 09:32:22 -0500 (CDT) Date: Mon, 23 Jun 1997 09:32:22 -0500 (CDT) Message-Id: <199706231432.JAA24773@rap.cs.wisc.edu> From: Steve Huss-Lederman To: jpprost@watson.ibm.com CC: mpi-core@mcs.anl.gov In-reply-to: <852564BC.006AA8C7.00@watngi01.watson.ibm.com> (jpprost@watson.ibm.com) Subject: Re: Few comments on latest draft document Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > 259/29: > poor Latex formatting. I am pretty sure better formatting can be achieved. Yes, this is a known side effect of another change. Before, we had items running over the margin. To correct this, we told LaTeX to break lines if they got to the margin. In some cases, this causes breaks where you would not think they are necessary. When I noticed this several months ago, the authors decided it was better to have extra space at the end of the line rather than running over the line. If it is possible to fix globally we will. > 266/8-9: > extra blank line. Could be a side effect from the marginpar which will > disappear > in the final version of the document. Yes, it is due to the change markers in the margin. This reminded me that in my rush to get the document out I forgot to put out a version without change markers. This version is what we expect the final one to look like except it won't have the questions. It is in the same location but under the names mpi2-jod-unmarked.ps and mpi2-report-unmarked.ps. I ask that people give comments on the marked version (as previously announced) so we all use consistent page references. If the problem is only with the unmarked verison then people can use it but indicate you are doing that. Steve From mpi-core-human@mcs.anl.gov Mon Jun 23 10:27:20 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04587; Mon, 23 Jun 1997 10:27:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA15972 for mpi-core-out; Mon, 23 Jun 1997 09:31:57 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA15960; Mon, 23 Jun 1997 09:31:42 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id KAA15318; Mon, 23 Jun 1997 10:30:51 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id KAA34690; Mon, 23 Jun 1997 10:31:03 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564BF.004FBDA3 ; Mon, 23 Jun 1997 10:30:58 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: thakur@mcs.anl.gov cc: mpi-core@mcs.anl.gov Message-ID: <852564BF.004FA5FD.00@watngi01.watson.ibm.com> Date: Mon, 23 Jun 1997 10:30:38 -0400 Subject: Re: Final MPI-2 proof documents available Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I know. I was just kidding. Jean-Pierre thakur @ mcs.anl.gov 06/23/97 10:20 AM To: mpi-core @ mcs.anl.gov cc: (bcc: Jean-Pierre Prost/Watson/IBM Research) Subject: Re: Final MPI-2 proof documents available > Pretty good, Rajeev. You are in good shape for winning the free T-shirt > contest. > > Jean-Pierre Well, that was not my intention though. Rajeev From mpi-core-human@mcs.anl.gov Mon Jun 23 10:34:40 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA04771; Mon, 23 Jun 1997 10:34:38 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA16264 for mpi-core-out; Mon, 23 Jun 1997 09:40:21 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA16259 for ; Mon, 23 Jun 1997 09:40:16 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id JAA24821; Mon, 23 Jun 1997 09:40:06 -0500 (CDT) Date: Mon, 23 Jun 1997 09:40:06 -0500 (CDT) Message-Id: <199706231440.JAA24821@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: correct pthread call Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On 174:29, the sample code has the call: pthreadinit(reduce_thread, (void*)&args); The correct syntax may be: pthread_t thread; ... pthread_create(&thread, NULL, reduce_thread, args); but we are not posative. If anyone knows for sure, please let us know. Thanks, Steve From mpi-core-human@mcs.anl.gov Mon Jun 23 10:55:41 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA05111; Mon, 23 Jun 1997 10:55:41 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA16743 for mpi-core-out; Mon, 23 Jun 1997 10:00:56 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA16738 for ; Mon, 23 Jun 1997 10:00:40 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id KAA14658; Mon, 23 Jun 1997 10:59:08 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id KAA34787; Mon, 23 Jun 1997 10:59:20 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564BF.00525210 ; Mon, 23 Jun 1997 10:59:09 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: wcs@nsgi3.lbl.gov cc: Linda@k2.llnl.gov, Mpi-Core@mcs.anl.gov Message-ID: <852564BF.0051E814.00@watngi01.watson.ibm.com> Date: Mon, 23 Jun 1997 10:58:49 -0400 Subject: Re: revised question Mime-Version: 1.0 Content-type: multipart/mixed; Boundary="0__=ZJvMT9zZiAC6Cs4BqVFREA769Ep3NLZPWJuYDTw9hwqG2znrPIq9iiaB" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk --0__=ZJvMT9zZiAC6Cs4BqVFREA769Ep3NLZPWJuYDTw9hwqG2znrPIq9iiaB Content-type: text/plain; charset=us-ascii (Embedded image moved wcs @ nsgi3.lbl.gov to file: 06/20/97 06:53 PM PIC30914.PCX) To: snir @ watson.ibm.com, linda @ k2.llnl.gov cc: Mpi-Core @ mcs.anl.gov (bcc: Marc Snir/Watson/IBM Research) Subject: Re: revised question Code examples are a nice way to explain this, for handles. In the following, a and b are handles. a has been initialized to a valid handle, and b has not been initialized. b = a MPI_xxx(a) If the argument to MPI_xxx is IN, then b is valid afterwards If the argument to MPI_xxx is OUT or INOUT, then b may no longer be a valid handle. ********* Even if the call to MPI_xxx(a) does not change the handle, so that MPI_xxx(b) can be safely invoked afterward, the argument may still be labeled INOUT, if it modifies the opaque object accessed by a. This is how we choose in MPI1 to use IN, OUT and INOUT for language independent bindings -- probably to fit some existing Posix standard on language independent bindings. ************************ --0__=ZJvMT9zZiAC6Cs4BqVFREA769Ep3NLZPWJuYDTw9hwqG2znrPIq9iiaB Content-type: application/octet-stream; name="PIC30914.PCX" Content-transfer-encoding: base64 CgUBCAAAAABoACwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAABaQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAD1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPH E8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sTzRPHE8MTwhP1E9sT zRPHE8MTwhPwEwzIBgzYE8wTxhPDE8IT7hPOBtcTzBPGE8MTE+wTwgbCBwbCEgbCEgbCEsUG1hPL E8YTwxMT6hMMwgYHwgLCAwISwgfEEsMCwwbVE8sTxRPDExPpE8MGAwcCBwMCwhLDB8ISwgISwgLD BtUTyhPFE8MTE+gTwgIHA8ICEw4DDgLDE8USwwLCEMIG1BPKE8UTwxMT5xMCAwcDAg4TDgITwgIS D8ISD8ISBRICEcICwwbUE8oTxRPCExPmEwYCBwMCDgIOwgLDExITEhPCEg8GxgLDBtMMDAfJE8QT whMT5hMGwwITBgMCDhLFEw8SE8ISBgIDwhIDEsMGB9MDxwwHxRPDExPlEwYHAhESAg8CwhMPwhMP xBMPxRIQwgIDAgMCBtMDxwPEDAfDE8IT4RMHwwzCBgLCEhMCDxLIE8MSD8MSwwIQAwIDBgfSDMkD wgPCDAfCExPbEwfGDMIDDAIHERITEhMSwxMPwxMPwxPDEgIDAgMCwwMCBgzREwfHDMYDDMITE9YT B8UMyAMGB8ICBhLDAsYTEhMSExIPwhIHAgcCAwUQAgYRBgfSE8UTB8QMwgMMwhMT0hMHxAzLA8IM BsISDxESExITAw4DxBMSExITwxICBwPCAsMDDMIGB9ITyRMHwwzCExPPEwfDDMkDxQwHwhMGBxIT AhECEwMOAg7DExITDxMPwxIDAgMCBwMCDAYRBgfSE8kTwhPCDMITE8wTB8MMxwPEDMIHxxMGxBLD Ag4DDgIGwg/IEgIDwgIDAgwCEMIGB9ITyRMHDAcMwhMTyhMHwgzGA8MMwgfMEwYHwhLCEAIOAg4C DhDDAhIPxhIFAgXDAgUCEQYH0hPHEwfCDAcPDMITE8gTB8IMxQPDDAfQEwbDEhDEAhAOEA4QwgLG EgcSBhIGBcMCBcIGB9ATB8UMEwfCDA8HDwwHwhMTxhMHwgzEA8MMB9MTBgfCEhADEMICDhAOEMIC EQIDxxIGBwbCAgUCEQYHyxMHxAwHwhMHEwzCEwcPBw8MB8MTE8UTBwzEA8IMB9YTBsQSEAMCA8UC EQIDAgPDEgcSBgfCBgUQAhDCBgfGEwfEDAfGE8INEwzCEw8HwgwHwxPCE8QTBwzDA8IMB9gTBgfE EhACEMYCEQIDAsQSBhLDBsICEALCBgfCEwfDDAfKEwfCDRMHwhPCDAfEE8ITE8MTBwzCA8IMB9oT DBIHwxLDDBEDxQIDAgPDEgYSBgfCBgIQAhAGDAfCEwzDE8MHyRMHwhPCBxMHxRPDExPDEwzCAwwH 3RMGxxICEQPDAgMCA8MSBhIGBwYMBhACEAIGDMMTDBPCB8YTwwfHEwfGE8MTwhPDEwwDDAfeEwYH xxICEQPDAgMCwhIGEgYHBgwGEAIQAsIGB8MTDMYTwwfKEwzGE8MTwhPDE8IMB98TDBLCB8USAgMR xAISB8ISBgcGDAYQBhAGEAYMB8MMB8kTwwfHEwzGE8MTwhPDEwwPwgzfEwYSB8ISB8ISAhECAwID EgcSBwYHBgwGEAYQxgzDD8IHxRPDB8kTBwzGE8MTwhPDEwzDD8QM3BPCBhIGwxIGAhECAwIHBgcG yAzJDxMHzRMHwwwHxxPDE8ITwxMHDMYPxwwH1BMGEgYSBhLLDM4PwwwTDMcTwgfEDAfJE8QTwhMT xBMHwgzLD9sM0w/GDAfDEwzDEwfEDAfLE8YTwxMTxhMHxAztD8gMBgfIE8QMB84TxxPDE8ITyhMH xwzbD8sMEAUMBcIMwgYH1RPKE8UTwxMT0RMH2wwGEAYQBhACBQwFDAUMBgwHBgfWE8sTxRPDExPu EwYMBhAGEAIGDAYMwwYH1xPLE8YTwxMT8BPKBgfYE8wTxhPDExP1E9sTzRPHE8MTwhP1E9sTzRPH E8MTwhMMAAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD/ /wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8A AAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A //8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA/wAA AP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCkgICA /wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vwoKCk gICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw//vw oKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzApsrw //vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDAwNzA psrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICAwMDA wNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACAAICA wMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACAgACA AICAwMDAwNzApsrw//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP//////AAAAgAAAAIAAgIAAAACA gACA//vwoKCkgICA/wAAAP8A//8AAAD//wD/AP////// --0__=ZJvMT9zZiAC6Cs4BqVFREA769Ep3NLZPWJuYDTw9hwqG2znrPIq9iiaB-- From mpi-core-human@mcs.anl.gov Mon Jun 23 12:26:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA08524; Mon, 23 Jun 1997 12:26:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA20150 for mpi-core-out; Mon, 23 Jun 1997 11:32:02 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA20116; Mon, 23 Jun 1997 11:31:11 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id LAA10427; Mon, 23 Jun 1997 11:31:07 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id LAA06848; Mon, 23 Jun 1997 11:31:06 -0500 (EST) Date: Mon, 23 Jun 1997 11:31:06 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: C++ Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In doing our proof-of-concept C++ implementation (layered on C), we found 3 minor errors: 1. Does comparing MPI::Status object make sense? It would be easier if we could just say that you can't do it, because the implementation may hide more than just MPI_TAG, MPI_SOURCE, and MPI_ERROR. Unless there's strong feeling otherwise, I'd like to add a sentence or two in the bindings chapter that says that == and != are not defined on MPI::Status objects. OR, if people strongly want it, I'd like to say that == and != *only* compare the three "public" pieces of information. Either way, something needs to be said. 2. The name Comm::Get_topo should change -- it is virtual, and compilers generate warnings that it is masked by Graphcomm::Get_topo. So the name Comm::Get_topo should change to Comm::Get_topo_type. 3. For similar reasons, Cartcomm::Get_rank should also change (it masks the virutal function Comm::Get_rank). It should change to Cartcomm::Get_cart_rank. (Aside -- Steve: do you want to make these changes, or do you want me to do it?) {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Mon Jun 23 13:00:37 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA09171; Mon, 23 Jun 1997 13:00:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA21258 for mpi-core-out; Mon, 23 Jun 1997 12:06:17 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA21225; Mon, 23 Jun 1997 12:05:02 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id MAA05811; Mon, 23 Jun 1997 12:04:43 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id MAA06586; Mon, 23 Jun 1997 12:04:41 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id RAA17636; Mon, 23 Jun 1997 17:04:41 GMT From: "Eric Salo" Message-Id: <9706231204.ZM17634@sgi.com> Date: Mon, 23 Jun 1997 12:04:40 -0500 In-Reply-To: Jeff Squyres "C++" (Jun 23, 11:31am) References: X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: C++ Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Unless there's strong > feeling otherwise, I'd like to add a sentence or two in the bindings > chapter that says that == and != are not defined on MPI::Status objects. > OR, if people strongly want it, I'd like to say that == and != *only* > compare the three "public" pieces of information. Either way works for me. If we go with the first option, we can always relax the restriction later if and when anyone complains. > 2. The name Comm::Get_topo should change -- it is virtual, and compilers > generate warnings that it is masked by Graphcomm::Get_topo. So the name > Comm::Get_topo should change to Comm::Get_topo_type. 'type' is bad, let's try to reserve that for datatypes. Maybe get_topo_kind? No, wait, we use 'kind' for something else as well. Hmm...how about get_topology? -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Mon Jun 23 13:26:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA09487; Mon, 23 Jun 1997 13:26:58 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA21916 for mpi-core-out; Mon, 23 Jun 1997 12:32:05 -0500 Received: from mcs.anl.gov (donner.mcs.anl.gov [140.221.5.134]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA21909; Mon, 23 Jun 1997 12:31:51 -0500 Message-Id: <199706231731.MAA21909@antares.mcs.anl.gov> To: Steve Huss-Lederman cc: mpi-core@mcs.anl.gov Subject: Re: correct pthread call In-reply-to: Your message of "Mon, 23 Jun 1997 09:40:06 CDT." <199706231440.JAA24821@rap.cs.wisc.edu> Date: Mon, 23 Jun 1997 12:31:47 -0500 From: Rusty Lusk Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | On 174:29, the sample code has the call: | | pthreadinit(reduce_thread, (void*)&args); | | The correct syntax may be: | | pthread_t thread; | ... | pthread_create(&thread, NULL, reduce_thread, args); | | but we are not positive. If anyone knows for sure, please let us | know. Using man pthread_create: ______________________________________________________________________ On a Sun: pthread_create(3T) Threads Library pthread_create(3T) NAME pthread_create - thread creation SYNOPSIS POSIX #include int pthread_create(pthread_t *new_thread_ID, const pthread_attr_t *attr, void * (*start_func)(void *), void *arg); ______________________________________________________________________ On an RS6000: Threads Library (libpthreads.a) Syntax #include int pthread_create (thread, attr, start_routine, arg) pthread_t *thread; const pthread_attr_t *attr; void *(*start_routine) (void *); void *arg; From mpi-core-human@mcs.anl.gov Mon Jun 23 13:42:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA09587; Mon, 23 Jun 1997 13:42:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA23066 for mpi-core-out; Mon, 23 Jun 1997 12:47:28 -0500 Received: from igw3.watson.ibm.com (igw3.watson.ibm.com [129.34.139.18]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA23059 for ; Mon, 23 Jun 1997 12:47:13 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw3.watson.ibm.com (8.7.6/8.7.1) with ESMTP id NAA07258 for ; Mon, 23 Jun 1997 13:46:20 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id NAA29261 for ; Mon, 23 Jun 1997 13:46:31 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564BF.0061A157 ; Mon, 23 Jun 1997 13:46:22 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: mpi-core@mcs.anl.gov Message-ID: <852564BF.0061077D.00@watngi01.watson.ibm.com> Date: Mon, 23 Jun 1997 13:45:59 -0400 Subject: Section 9.6.5 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I definitely prefer John's wording and am supportive to the replacement of current section 9.6.5 by his wording, if Steve thinks it is not too late to do so. John's wording specifies things much more accurately. Just one slight editing in the rationale's text: \begin{rationale} Although it may seem desirable to impose stronger restrictions on the type matching between file data and file views, doing so would require implicitly defining the type of each element in a file. Unlike variables in a program, data in a file typically has no type independent of what an MPI read or write operation imposes on it. It would be possible to require that the MPI datatype used to read a file must match the MPI datatype that was used to write it, but this would make it erroneous to read any file not written by MPI, and it would not account for the possibility that programs might read and write a file using different data representations. \end{rationale} In addition, I would replace all occurrences of READ and WRITE with read and write, respectively. Jean-Pierre From mpi-core-human@mcs.anl.gov Mon Jun 23 14:01:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA09769; Mon, 23 Jun 1997 14:01:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA17005 for mpi-core-out; Mon, 23 Jun 1997 10:07:29 -0500 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [192.208.145.139]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA16993 for ; Mon, 23 Jun 1997 10:07:19 -0500 Received: from ultraman.erc.msstate.edu (shane@ultraman [192.208.142.31]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with ESMTP; id KAA19216; Mon, 23 Jun 1997 10:07:10 -0500 (CDT) From: Shane Hebert Received: (from shane@localhost); by ultraman.erc.msstate.edu (8.8.5/8.8.5/v1c); id KAA14393; Mon, 23 Jun 1997 10:07:10 -0500 (CDT) Message-Id: <199706231507.KAA14393@ultraman.erc.msstate.edu> Subject: Re: [MPI #2507] Creating Communicators To: mark@hdl.ie (Mark Fallon) Date: Mon, 23 Jun 1997 10:07:10 -0500 (CDT) Cc: mpi-core@mcs.anl.gov In-Reply-To: <33AE8801.60C868D0@hdl.ie> from "Mark Fallon" at Jun 23, 97 03:28:17 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi Mark, Is this even a correct program? Rank 1 executes two MPI_COMM_CREATEs on COMM_WORLD while Rank 0 executes only one MPI_COMM_CREATE on COMM_WORLD. I think I would expect this one to hang with Rank 1 in the second MPI_COMM_CREATE. Shane Mark Fallon previously wrote: >| >| Hi, >| >| When looking at the best way to create communicators for >| my own implementation I came across the following in the >| MPICH code. The code I was looking at was v1.1 but I have >| only tested this up to version 1.0.13. The following >| segment of code causes the system to hang. >| >| #include >| >| void main(int argc, char *argv[]){ >| MPI_Comm NewComm, NewComm2; >| MPI_Group Group, Group2; >| MPI_Status status; >| int MyRank,Dest; >| >| MPI_Init(&argc,&argv); >| MPI_Comm_rank(MPI_COMM_WORLD, &MyRank); >| if (MyRank == 1){ >| MPI_Comm_group(MPI_COMM_SELF,&Group2); >| MPI_Comm_create(MPI_COMM_WORLD,Group2,&NewComm2); >| } >| MPI_Comm_group(MPI_COMM_WORLD,&Group); >| MPI_Comm_create(MPI_COMM_WORLD,Group,&NewComm); >| if (MyRank == 0) >| MPI_Send(&MyRank,1,MPI_INT,1,99,NewComm); >| else >| MPI_Recv(&Dest,1,MPI_INT,0,99,NewComm,&status); >| MPI_Finalize(); >| } >| >| (The code should be run on two processors) >| >| The reason for this is that the value of NewComm is different >| on rank 1 from rank 0. I would be very interested to find >| out if this an error in my interpretation of the standard or >| an error in MPICH. I would love to be able to avoid global >| communication on MPI_Comm_create. >| >| Regards, >| >| Mark. >| _________________________________________________________________ >| Mark Fallon >| Researcher >| Hitachi Dublin Laboratory >| O'Reilly Institute, Tel : +353-1-6798911 >| Trinity College, Fax : +353-1-6798926 >| Dublin Email : mark.fallon@hdl.ie >| Ireland >| __________________________________________________________________ >| From mpi-core-human@mcs.anl.gov Mon Jun 23 14:04:16 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA09806; Mon, 23 Jun 1997 14:04:15 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA24036 for mpi-core-out; Mon, 23 Jun 1997 13:09:56 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA24031; Mon, 23 Jun 1997 13:09:52 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id NAA11067; Mon, 23 Jun 1997 13:09:43 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id NAA08197; Mon, 23 Jun 1997 13:09:42 -0500 (EST) Date: Mon, 23 Jun 1997 13:09:41 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: C++ In-Reply-To: <9706231204.ZM17634@sgi.com> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Mon, 23 Jun 1997, Eric Salo wrote: > > Unless there's strong > > feeling otherwise, I'd like to add a sentence or two in the bindings > > chapter that says that == and != are not defined on MPI::Status objects. > > OR, if people strongly want it, I'd like to say that == and != *only* > > compare the three "public" pieces of information. > > Either way works for me. If we go with the first option, we can always relax > the restriction later if and when anyone complains. Sounds reasonable -- anyone else? > > 2. The name Comm::Get_topo should change -- it is virtual, and compilers > > generate warnings that it is masked by Graphcomm::Get_topo. So the name > > Comm::Get_topo should change to Comm::Get_topo_type. > > 'type' is bad, let's try to reserve that for datatypes. Maybe get_topo_kind? > No, wait, we use 'kind' for something else as well. Hmm...how about > get_topology? Get_topology sounds good to me. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Tue Jun 24 02:07:31 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA22208; Tue, 24 Jun 1997 02:07:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA01913 for mpi-core-out; Mon, 23 Jun 1997 16:37:57 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA01891 for ; Mon, 23 Jun 1997 16:37:07 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id QAA13062 for ; Mon, 23 Jun 1997 16:37:02 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id QAA08566 for ; Mon, 23 Jun 1997 16:37:03 -0500 (EST) Date: Mon, 23 Jun 1997 16:37:03 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov Subject: Annex B correction Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk One more minor correction: 358:36 The function returns "int index", not "int outcount". (Too bad I'm a chapter author, or I'd be in serious contention for that t-shirt! Actually, in all fairness -- most of the mistakes posted are really from our grad student who is doing the C++ implementation :-) {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Tue Jun 24 02:18:56 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA22391; Tue, 24 Jun 1997 02:18:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id BAA01915 for mpi-core-out; Tue, 24 Jun 1997 01:24:33 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id BAA01903; Tue, 24 Jun 1997 01:24:23 -0500 Received: from dumas.lsc.nd.edu (jsiek@dumas.lsc.nd.edu [129.74.22.162]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id SAA13231; Mon, 23 Jun 1997 18:00:44 -0500 (EST) Received: (from jsiek@localhost) by dumas.lsc.nd.edu (8.8.5/8.8.5) id SAA05004; Mon, 23 Jun 1997 18:00:45 -0500 (EST) Date: Mon, 23 Jun 1997 18:00:45 -0500 (EST) Message-Id: <199706232300.SAA05004@dumas.lsc.nd.edu> From: jeremy siek To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Subject: Alpha release of a C++ bindings implementation Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I have just completed the initial coding of the C++ bindings as a layer on top of the C bindings. This initial version is meant to be a "proof of concept", as well as an aid to fleshing out any remaining issues in the MPI C++ bindings. This is an alpha-release -- it demonstrates that the C++ bindings work, but has not yet been fully tested and ported. This implementation was developed and tested in Solaris 2.5.1, using LAM 6.1. It has not yet been tested with other versions of MPI, other compilers, or other operating systems. A complete version of this implementation is expected to be released by the end of the summer. You can obtain a copy of this implementation at http://www.nd.edu/~lsc/mpi2/. -Jeremy Siek From mpi-core-human@mcs.anl.gov Tue Jun 24 02:39:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id CAA22988; Tue, 24 Jun 1997 02:39:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA24727 for mpi-core-out; Mon, 23 Jun 1997 13:29:01 -0500 Received: from k2.llnl.gov (johnmay@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA24716 for ; Mon, 23 Jun 1997 13:28:44 -0500 Received: (from johnmay@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id LAA06091; Mon, 23 Jun 1997 11:28:40 -0700 (PDT) Message-ID: <19970623112840.UG06459@k2.llnl.gov> Date: Mon, 23 Jun 1997 11:28:40 -0700 From: johnmay@llnl.gov (John May) To: mpi-core@mcs.anl.gov Subject: Re: Section 9.6.5 References: <852564BF.0061077D.00@watngi01.watson.ibm.com> X-Mailer: Mutt 0.60e Mime-Version: 1.0 In-Reply-To: <852564BF.0061077D.00@watngi01.watson.ibm.com>; from Jean-Pierre Prost on Jun 23, 1997 13:45:59 -0400 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Jean-Pierre Prost writes: > > I definitely prefer John's wording and am supportive to the replacement of > current section > 9.6.5 by his wording, if Steve thinks it is not too late to do so. > John's wording specifies things much more accurately. I orginally sent my suggestion out to some of the MPI-IO implementors to get their opinion on it. To avoid keeping the mpi-core group in the dark, I'll include a copy here so you know what I had in mind. The confusion I refer to below is whether it's legal to have MPI_BYTE as a "wild card" etype that would match any buftype or filetype. BTW, I like Jean-Pierre's suggested editorial change to the rationale, and I have incorporated it below. --- It may be too late to get this in, but I want to present it anyway and try to convince you that it's important. Section 10.6.5 is too vague on the type matching rules, and it doesn't clearly maintain the distinction between matching of MPI types with each other and matching of MPI types with files and memory buffers. As evidence of this, there have already been significantly different interpretations of the type matching rules not only between Rajeev and our group at LLNL but also between different members of the LLNL group! The following is a longer but much more specific version of 10.6.5. My intent was to say what I thought the chapter implied, without adding any new restrictions. I found that I did have to relax the restriction on matching between data written and data read because, given the possibility of different datareps and of reading files not written by MPI (which I don't think we want to outlaw), the current rule is neither necessary nor sufficient to guarantee correct reading of data. So here goes. John In general, the type matching rules for I/O mimic the type matching rules for communication. As with communication, type matching must be considered at several phases (compare Section 3.3.1 of the MPI-1 reference document). For I/O, these phases are: \begin{enumerate} \item Between the memory buffer and the MPI datatype passed to a READ or WRITE command. \item Between the datatype and the file view. \item Between the file view and the MPI file. \end{enumerate} For Phase 1, the rules for matching the MPI datatype to the type of data in the buffer are identical to the rules for matching MPI datatypes to send and receive buffers in communication calls. For Phase 2, the type signature of the datatype passed to a READ or WRITE operation must equal the type signature of some number of contiguous copies of the etype used in the current file view. More formally, let $D$ be the type signature of the \mpiarg{datatype} passed to a READ or WRITE operation on file handle $fh$. Let $E$ be the type signature of the etype of the current file view for $fh$. Then for any MPI READ or WRITE operation, $D = E^r$ must hold for some positive integer $r$, where $E^r$ is the sequence of MPI basic types that consists of $r$ repetitions of $E$. For Phase 3, MPI imposes type matching only on READ operations. Each data item read from a file must represent a legal value of the corresponding MPI basic type in the view. Specifically, let $V = \{(type_0, disp_0), (type_1, disp_1),\ldots \}$ be the (arbitrarily long) type map that represents the current file view on file handle $fh$ (where $type_i$ is an MPI basic type and $disp_i$ is its byte displacement with respect to the start of the file). Let $B = \{b_0, b_1,\ldots, b_{n-1}\}$ be the sequence of bytes in the file that $fh$ points to. An MPI read operation that accesses element $i$ of $V$ will read the byte sequence $B_i = \{b_{disp_i}, \ldots, b_{disp_{i} + s - 1}\}$ from the file, where $s$ is the size of $type_i$ in bytes, as computed by calling MPI\_FILE\_GET\_TYPE\_EXTENT for $type_i$ and $fh$. Then for each read operation of an element $i$ in $V$, the byte sequence $B_i$ must be a legal value for $type_i$ in the current data representation for $fh$. \begin{users} The type matching rules between views and file data are quite lax. A program may write a data item as a floating-point number and then read the same bytes back as an integer. An MPI implementation is not required to recognize such a mismatch or to perform any conversion between these types. Also, the type matching rules between MPI datatypes and memory buffers permit a memory buffer of any type to be accessed using MPI\_BYTE. All of these type mismatches, though legal, will defeat the file interoperability features of MPI. READ and WRITE functions can only perform automatic conversion between heterogeneous data representations when the exact MPI basic types read and written are equal and are explicitly specified. \end{users} \begin{rationale} Although it may seem desirable to impose stronger restrictions on the type matching between file data and file views, doing so would require implicitly defining the type of each element in a file. Unlike variables in a program, data in a file typically has no type independent of what an MPI read or write operation imposes on it. It would be possible to require that the MPI datatype used to read a file must match MPI datatype that was used to write it, but this would make it erroneous to read any file not written by MPI, and it would not account for the possibility that programs might read and write a file using different data representations. \end{rationale} From mpi-core-human@mcs.anl.gov Tue Jun 24 04:23:57 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA05164; Tue, 24 Jun 1997 04:23:56 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA04076 for mpi-core-out; Tue, 24 Jun 1997 03:29:33 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA04071 for ; Tue, 24 Jun 1997 03:29:29 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id KAA12600 (8.6.13/IDA-1.6); Tue, 24 Jun 1997 10:29:25 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id KAA12469; Tue, 24 Jun 1997 10:29:24 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199706240829.KAA12469@awsrr.rus.uni-stuttgart.de> Subject: Re: Final MPI-2 ... some corrections To: lederman@cs.wisc.edu (Steve Huss-Lederman) Date: Tue, 24 Jun 1997 10:29:24 +0200 (DST) Cc: mpi-core@mcs.anl.gov In-Reply-To: <199706192333.SAA04926@rap.cs.wisc.edu> from "Steve Huss-Lederman" at Jun 19, 97 06:33:51 pm Content-Type: text Content-Length: 2681 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk MPI-2 from June 19, 1997: 47:32 (at the end of 4.9 The INFO Object) I propose to add if there is no other place where it is already said: Functions with info input arguments may return an error code of class MPI_ERR_INFO to indicate an invalid info argument. 48:32 I propose to add (compare also MPI_ERR_BASE on 139:38): The function MPI_FREE_MEM may return an error code of class MPI_ERR_BASE to indicate an invalid base argument. 133:47 MPI_ALLOC_MEM instead of MPI_MEM_ALLOC (Steve, it may be good to check by egrep on *.tex all the function names that where changed since the last version(s). This include the obsolete function names MPI_MEM_ALLOC, MPI_MEM_FREE, MPI_COMM_CREATE_WIN, MPI_WIN_INIT, MPI_FILE_FREE_ERRHANDLER, MPI_FILE.*EXPLICIT, MPI_TYPE_RESIZED and the obsolete constants MPI_COMBINER_FUNDAMENTAL and ...) 145:15 "Fortran problems" instead of "Fortran 90 problems" because this section handles also an important Fortran 77 problem. 226:32 The following Rationale was voted explicitely in the last meeting and was lost in the last editing period. In Draft May 8, it was on 232:9-11. This was the wrong place. It should be moved to the end of this section. Therefore please bring back 226:32: Rationale. This implies that reading on a pipe will always wait until the requested amount of data is available or until the process writing to the pipe has issued an end of file. (End of rationale.) 300:41 because my copy does not have the date "August" and because we should clearly distinguish from Standard 854, please write ANSI/IEEE Std 754-1985 instead of ANSI/IEEE, August 1985 303:2-34 (Steve it may be good to check in the tex-source, whether all these classes are mentioned at __all__ places where they can be used. (compare my corrections for MPI_ERR_INFO and MPI_ERR_BASE).) 330: I miss some C++ section for MPI-2 in Annex A.10, corresponding to the C++ MPI-1 sections in Annex B: B.1 C++ Classes (e.g. Win, Info, ...) B.3 Typedefs (if there are some new with MPI-2) (may be done with 304:13-14 ?) B.13.5 Function Name Cross Reference 356-358 (No corrections, only ...) thanks for the work of building the Cross Reference Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Tue Jun 24 13:11:28 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA03652; Tue, 24 Jun 1997 13:11:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA18999 for mpi-core-out; Tue, 24 Jun 1997 12:14:10 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA18990; Tue, 24 Jun 1997 12:14:00 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id MAA28374; Tue, 24 Jun 1997 12:13:49 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id MAA03710; Tue, 24 Jun 1997 12:13:47 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id RAA21337; Tue, 24 Jun 1997 17:13:46 GMT From: "Eric Salo" Message-Id: <9706241213.ZM21335@sgi.com> Date: Tue, 24 Jun 1997 12:13:44 -0500 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov, mpi-external@mcs.anl.gov Subject: final draft Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk >From the "now that we're actually trying to implement this" category: It turns out that MPI_TYPE_GET_ENVELOPE() is defined rather awkwardly because it does not return the 'count' argument that was used when the supplied datatype was created. This is not a fatal flaw, because the count can still be derived from the other fields, but it is awkward because it depends on the combiner. So I would argue that we really should add a 'count' return value for this function. Observe that if we have count, we don't even need num_addresses, num_integers or num_datatypes, since they can be derived easily from the count and combiner. So, my minimal-change proposal is to add 'count' as an OUT argument. And my maximal-elegance proposal is the above plus deleting the other three num_XXX args. I view this almost as a bindings issue, since we are providing exactly the same information as we were before. It's just in a more convenient form, and has better symmetry with the type constructors. It also makes for a cleaner separation between MPI_TYPE_GET_ENVELOPE() and MPI_TYPE_GET_CONTENTS(). Comments? I know it's awfully late, but this would be a very localized change to the document - certainly less sweeping than some of the other errors which have been caught this week... -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Tue Jun 24 14:22:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA04652; Tue, 24 Jun 1997 14:22:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA21077 for mpi-core-out; Tue, 24 Jun 1997 13:27:48 -0500 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA20990; Tue, 24 Jun 1997 13:25:39 -0500 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (SMI-8.6/mail.byaddr) with SMTP id LAA18833; Tue, 24 Jun 1997 11:48:04 -0700 Received: from suneast.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id OAA17323; Tue, 24 Jun 1997 14:24:51 -0400 Received: from thokk.East.Sun.COM by suneast.East.Sun.COM (SMI-8.6/SMI-SVR4) id OAA13383; Tue, 24 Jun 1997 14:24:52 -0400 Received: from thokk by thokk.East.Sun.COM (SMI-8.6/SMI-SVR4) id OAA18508; Tue, 24 Jun 1997 14:24:50 -0400 Message-Id: <199706241824.OAA18508@thokk.East.Sun.COM> To: "Eric Salo" cc: mpi-core@mcs.anl.gov, mpi-external@mcs.anl.gov Subject: Re: final draft Date: Tue, 24 Jun 1997 14:24:50 -0400 From: David Taylor - SMCC High Performance Computing Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk From: "Eric Salo" Date: Tue, 24 Jun 1997 12:13:44 -0500 From the "now that we're actually trying to implement this" category: It turns out that MPI_TYPE_GET_ENVELOPE() is defined rather awkwardly because it does not return the 'count' argument that was used when the supplied datatype was created. This is not a fatal flaw, because the count can still be derived from the other fields, but it is awkward because it depends on the combiner. So I would argue that we really should add a 'count' return value for this function. Well, it used to return the count argument... Observe that if we have count, we don't even need num_addresses, num_integers or num_datatypes, since they can be derived easily from the count and combiner. Count was deleted and those were added in a mad rush during one meeting to fix some problems -- type-get-contents didn't handle some of the new constructors (e.g., type-darray). I believe that argument for changing type-get-envelope was that it might be necessary and that even if it wasn't, it was 'more general'... [And at the time type-darray took arrays of two sizes -- ndims and pdims.] So, my minimal-change proposal is to add 'count' as an OUT argument. And my maximal-elegance proposal is the above plus deleting the other three num_XXX args. I prefer the maximal-elegance proposal. With it, the type-get-envelope binding would go back to what it was in the February 26th draft (except for the updated list of combiners). I view this almost as a bindings issue, since we are providing exactly the same information as we were before. It's just in a more convenient form, and has better symmetry with the type constructors. It also makes for a cleaner separation between MPI_TYPE_GET_ENVELOPE() and MPI_TYPE_GET_CONTENTS(). I'm in favor of it, but worry somewhat about the slippery slope... Comments? I know it's awfully late, but this would be a very localized change to the document - certainly less sweeping than some of the other errors which have been caught this week... -- Eric Salo Silicon Graphics salo@sgi.com David -- david.taylor@East.Sun.COM From mpi-core-human@mcs.anl.gov Tue Jun 24 14:28:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA04697; Tue, 24 Jun 1997 14:28:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA21275 for mpi-core-out; Tue, 24 Jun 1997 13:33:59 -0500 Received: from ormail.intel.com (ormail.intel.com [134.134.248.3]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA21133 for ; Tue, 24 Jun 1997 13:29:35 -0500 Received: from babel.co.intel.com (babel.co.intel.com [137.46.3.28]) by ormail.intel.com (8.8.4/8.8.4) with ESMTP id LAA29461; Tue, 24 Jun 1997 11:29:07 -0700 (PDT) Received: from knight.co.intel.com (knight.co.intel.com [137.46.134.46]) by babel.co.intel.com (8.8.5/8.8.4) with ESMTP id LAA04200; Tue, 24 Jun 1997 11:29:23 -0700 (PDT) Received: (from knighten@localhost) by knight.co.intel.com (8.8.5/8.8.4) id LAA06325; Tue, 24 Jun 1997 11:29:22 -0700 (PDT) Date: Tue, 24 Jun 1997 11:29:22 -0700 (PDT) Message-Id: <199706241829.LAA06325@knight.co.intel.com> From: Bob Knighten To: lederman@cs.wisc.edu Cc: mpi-core@mcs.anl.gov Subject: Re: correct pthread call In-Reply-To: <199706231440.JAA24821@rap.cs.wisc.edu> References: <199706231440.JAA24821@rap.cs.wisc.edu> Reply-To: knighten@co.intel.com (Bob Knighten) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve Huss-Lederman writes: > On 174:29, the sample code has the call: > > pthreadinit(reduce_thread, (void*)&args); > > The correct syntax may be: > > pthread_t thread; > ... > pthread_create(&thread, NULL, reduce_thread, args); > > but we are not posative. If anyone knows for sure, please let us > know. > > Thanks, > Steve >From the standard (ISO/IEC 9945-1:1996, Sec. 16.2.2 Thread Creation, p. 336): int pthread_create(pthread_t *thread, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg); ----- NULL for the attr means the default thread attributes are used. -- Bob -- Robert L. Knighten | knighten@co.intel.com Server Architecture Lab | Intel Corporation | 5200 N.E. Elam Young Parkway, CO3-202 | (503) 677-4315 Hillsboro, OR 97124-6497 | (503) 677-6700 [FAX] From mpi-core-human@mcs.anl.gov Tue Jun 24 15:23:29 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA05198; Tue, 24 Jun 1997 15:23:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA23302 for mpi-core-out; Tue, 24 Jun 1997 14:28:48 -0500 Received: from k2.llnl.gov (johnmay@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA23295 for ; Tue, 24 Jun 1997 14:28:42 -0500 Received: (from johnmay@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id MAA13815; Tue, 24 Jun 1997 12:28:38 -0700 (PDT) Message-ID: <19970624122837.IK35219@k2.llnl.gov> Date: Tue, 24 Jun 1997 12:28:37 -0700 From: johnmay@llnl.gov (John May) To: mpi-core@mcs.anl.gov Subject: more MPI-2 corrections X-Mailer: Mutt 0.60e Mime-Version: 1.0 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk 219/38 true/false info values are called "logical" here, but 44/5 defines such values as being "boolean" 223/34 word "and" would look better than "&" 226/32 Rolf asks for rationale here. It's still in the document at 258/20, where it also makes sense. 238/15 change to "the call might return only after..." 246/24 Not sure what "increased interoperability" means here, especially wrt "full interoperability." Also, comma shouldn't join these clauses as they are. How about "MPI guarantees full interoperability within a single MPI environment, and it also supports interoperability outside ..." 247/9 Sounds like "typing information" has "desires". Maybe drop "not only desired but also well-". 247/20 Further correction on item Rajeev noted: Data reps are specified when setting views, not when opening or creating files files. Also, this could be interpreted as saying that all processes in running a given app need to specify the same view, which is already required. I think the point of this advice is that the reading process and writing process need to specifying the same data rep. So: "MPI is not guaranteed to retain knowledge of what data representation was used when a file is written. Therefore, an MPI application reading a file is responsible for ensuring that it uses the same data representation that was used to write the file." 251/22 funny line break after ( 251/28 I'm not sure what the restriction about byte alignment for all types means. The user has no control over the alignment in the file, and buftype controls the alignment in memory. Is this trying to say that data in the intermediate buffer has to be read and written contiguously? 251/30 omit comma and add "to": "routines or to free" 252/45 Are these conditions ANDed or ORed? These conditions are not clear to me. 255/28 change "which" to "that" 255/5 " 255/12 " From mpi-core-human@mcs.anl.gov Tue Jun 24 15:43:38 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA05382; Tue, 24 Jun 1997 15:43:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA24180 for mpi-core-out; Tue, 24 Jun 1997 14:48:50 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA24129; Tue, 24 Jun 1997 14:48:16 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id OAA02858; Tue, 24 Jun 1997 14:47:52 -0500 (CDT) Date: Tue, 24 Jun 1997 14:47:52 -0500 (CDT) Message-Id: <199706241947.OAA02858@rap.cs.wisc.edu> From: Steve Huss-Lederman To: salo@sgi.com CC: mpi-core@mcs.anl.gov, mpi-external@mcs.anl.gov In-reply-to: <9706241213.ZM21335@sgi.com> (salo@sgi.com) Subject: Re: final draft Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Eric has proposed changing the datatype decoding functions to return a count and remove the array_size arguments. David Taylor gave a good description of how this change occurred and some reasons for it. They both support this change. I am not sure that I would like to see it. Frist, Eric argues that array_sizes are harder for the implemenation to return because they are not input argument. This is one very good reason to keep them. If they are not returned by GET_ENVELOPE then the user must do the calculation. There was concern that errors in doing this would lead to mistakes in array sizes passed to GET_CONTENTS. Second, the issue of generality was real. As David pointed out, you needed two counts for one of the new datatypes. If we return a single count and then add a constructor in the future (did I say that, really, aahhh!) it may mean changing these calls too. Third, not all calls have a count or it is called something else. For example, DUP would return count=0 I assume? Fourth, the user can get the count from the combiner and other array sizes returned if they need it without calling GET_CONTENTS (and needing it is less likely then the array sizes in my opinion). Finally, one gets the count as the first returned value in the integer array from GET_CONTENTS so not giving it in GET_ENVELOPE does not seem a big deal to me. All that said, I am currently thinking that leaving it alone is best right now. Steve From mpi-core-human@mcs.anl.gov Tue Jun 24 16:17:55 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA05922; Tue, 24 Jun 1997 16:17:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA25727 for mpi-core-out; Tue, 24 Jun 1997 15:17:56 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [129.34.139.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA25716 for ; Tue, 24 Jun 1997 15:17:47 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw2.watson.ibm.com (8.8.5/06-04-97) with ESMTP id QAA05996; Tue, 24 Jun 1997 16:18:29 -0400 Received: from watngi04.watson.ibm.com (watngi04.watson.ibm.com [9.2.235.45]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id QAA22553; Tue, 24 Jun 1997 16:16:25 -0400 Received: by watngi04.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564C0.006F5C77 ; Tue, 24 Jun 1997 16:16:21 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: johnmay@llnl.gov cc: mpi-core@mcs.anl.gov Message-ID: <852564C0.006F193D.00@watngi04.watson.ibm.com> Date: Tue, 24 Jun 1997 16:15:52 -0400 Subject: Re: more MPI-2 corrections Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk John, Your comments associated with page 251 are in fact relative to page 254. And I could not locate the next comments. You may want to check page and line numbers of all comments started on page 251 (which is in fact page 254). Jean-Pierre johnmay @ llnl.gov 06/24/97 03:28 PM To: mpi-core @ mcs.anl.gov cc: (bcc: Jean-Pierre Prost/Watson/IBM Research) Subject: more MPI-2 corrections 219/38 true/false info values are called "logical" here, but 44/5 defines such values as being "boolean" 223/34 word "and" would look better than "&" 226/32 Rolf asks for rationale here. It's still in the document at 258/20, where it also makes sense. 238/15 change to "the call might return only after..." 246/24 Not sure what "increased interoperability" means here, especially wrt "full interoperability." Also, comma shouldn't join these clauses as they are. How about "MPI guarantees full interoperability within a single MPI environment, and it also supports interoperability outside ..." 247/9 Sounds like "typing information" has "desires". Maybe drop "not only desired but also well-". 247/20 Further correction on item Rajeev noted: Data reps are specified when setting views, not when opening or creating files files. Also, this could be interpreted as saying that all processes in running a given app need to specify the same view, which is already required. I think the point of this advice is that the reading process and writing process need to specifying the same data rep. So: "MPI is not guaranteed to retain knowledge of what data representation was used when a file is written. Therefore, an MPI application reading a file is responsible for ensuring that it uses the same data representation that was used to write the file." 251/22 funny line break after ( 251/28 I'm not sure what the restriction about byte alignment for all types means. The user has no control over the alignment in the file, and buftype controls the alignment in memory. Is this trying to say that data in the intermediate buffer has to be read and written contiguously? 251/30 omit comma and add "to": "routines or to free" 252/45 Are these conditions ANDed or ORed? These conditions are not clear to me. 255/28 change "which" to "that" 255/5 " 255/12 " From mpi-core-human@mcs.anl.gov Tue Jun 24 16:30:05 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA06191; Tue, 24 Jun 1997 16:29:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA26329 for mpi-core-out; Tue, 24 Jun 1997 15:34:17 -0500 Received: from k2.llnl.gov (johnmay@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA26309 for ; Tue, 24 Jun 1997 15:34:02 -0500 Received: (from johnmay@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id NAA19040; Tue, 24 Jun 1997 13:33:53 -0700 (PDT) Message-ID: <19970624133353.JY54076@k2.llnl.gov> Date: Tue, 24 Jun 1997 13:33:53 -0700 From: johnmay@llnl.gov (John May) To: mpi-core@mcs.anl.gov Subject: Re: more MPI-2 corrections References: <852564C0.006F193D.00@watngi04.watson.ibm.com> X-Mailer: Mutt 0.60e Mime-Version: 1.0 In-Reply-To: <852564C0.006F193D.00@watngi04.watson.ibm.com>; from Jean-Pierre Prost on Jun 24, 1997 16:15:52 -0400 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Jean-Pierre Prost writes: > Your comments associated with page 251 are in fact relative to page 254. > And I could not locate the next comments. You may want to check page and > line numbers of all comments started on page 251 (which is in fact page > 254). Thanks. The corrected page numbers are below. > 254/28 I'm not sure what the restriction about byte alignment > for all types means. The user has no control over the > alignment in the file, and buftype controls the alignment > in memory. Is this trying to say that data in the > intermediate buffer has to be read and written > contiguously? > 254/30 omit comma and add "to": "routines or to free" > 254/38 change "which" to "that" > 254/45 Are these conditions ANDed or ORed? These conditions > are not clear to me. > 255/1 Change "which" to "that" > 255/5 " > 255/12 " From mpi-core-human@mcs.anl.gov Tue Jun 24 17:24:40 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA06899; Tue, 24 Jun 1997 17:24:39 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA28354 for mpi-core-out; Tue, 24 Jun 1997 16:28:56 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA28349; Tue, 24 Jun 1997 16:28:51 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id QAA18244; Tue, 24 Jun 1997 16:28:36 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id QAA11920; Tue, 24 Jun 1997 16:28:33 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id VAA21955; Tue, 24 Jun 1997 21:28:32 GMT From: "Eric Salo" Message-Id: <9706241628.ZM21953@sgi.com> Date: Tue, 24 Jun 1997 16:28:32 -0500 In-Reply-To: Steve Huss-Lederman "Re: final draft" (Jun 24, 2:47pm) References: <199706241947.OAA02858@rap.cs.wisc.edu> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov, mpi-external@mcs.anl.gov Subject: Re: final draft Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > I am not sure that I would like to see it. Fine by me. Considering the lateness of the hour, I don't think we should change anything that doesn't have universal support. But I'll try to answer the points you raise anyway... > First, Eric argues that array_sizes are harder for the implemenation to > return because they are not input argument. What I said was that it is awkward for applications to derive the count from the array sizes and the combiner. It's pretty trivial to implement either way, actually. Slightly more work to return the sizes, but my primary objection is just the ugliness of the user interface. > If they are not returned by GET_ENVELOPE then the user must do the > calculation. There was concern that errors in doing this would lead > to mistakes in array sizes passed to GET_CONTENTS. True. But this still does not free the application writer from having to get things right, because they must correctly index into the arrays returned by GET_CONTENTS in order to use that information. So we're not really saving them all that much work. > Second, the issue of generality was real. As David pointed out, you needed > two counts for one of the new datatypes. If we return a single count and then > add a constructor in the future (did I say that, really, aahhh!) it > may mean changing these calls too. Sounds like a solid argument for going with the count! :-) I really don't see generality as an issue. Fundamentally, I think it makes perfect sense to view datatypes as some sort of combiner plus some number of things to combine. Anything fancier than that could easily be folded into GET_CONTENTS should the need arise. > Third, not all calls have a count or it is called something else. For > example, DUP would return count=0 I assume? For DARRAY and SUBARRAY, I'm assuming that we would pass back ndims. For DUP we could choose 0, 1 (my own preference), or even MPI_UNDEFINED. Pick your favorite poison. > Fourth, the user can get the count from the combiner and other array sizes > returned if they need it without calling GET_CONTENTS But not easily. At the moment, the code to do that would look something like this: GET_ENVELOPE() switch (combiner) { case CONTIGUOUS: count = f1(numi, numa, numd); break; case VECTOR: count = f2(numi, numa, numd); break; ... } So yeah, it's possible, but it's ugly. > (and needing it is less likely then the array sizes in my opinion). Well, I did suggest passing back both in my minimal-change proposal. I think that count is important regardless of whether the array sizes are also important, and I think that logically it makes more sense in the envelope information than in the contents. But that's just personal taste, I suppose. > Finally, one gets the count as the first returned value in the integer > array from GET_CONTENTS so not giving it in GET_ENVELOPE does not seem > a big deal to me. Not always! For DARRAY, it is the third integer in the array, not the first. So you'll still need the switch() statement. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Wed Jun 25 00:24:20 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id AAA11071; Wed, 25 Jun 1997 00:24:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id XAA05505 for mpi-core-out; Tue, 24 Jun 1997 23:29:45 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id XAA05479; Tue, 24 Jun 1997 23:28:57 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id XAA02724; Tue, 24 Jun 1997 23:28:48 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id XAA00435; Tue, 24 Jun 1997 23:28:48 -0500 (EST) Date: Tue, 24 Jun 1997 23:28:48 -0500 (EST) From: Jeff Squyres To: OOMPI Release Subject: OOMPI version 1.0.2 Message-ID: X-URL: http://www.cse.nd.edu/~lsc/research/oompi MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The Laboratory for Scientific Computing of the Department of Computer Science and Engineering at the University of Notre Dame is proud to announce version 1.0.2 of Object Oriented MPI (OOMPI): A full-featured class library for MPI. Version 1.0.2 includes formal support for several more operating systems and MPI implementations: Hardware Operating System MPI Implementation =============== ======================= ========================= Sun SPARC SunOS 4.1.3 LAM 6.1, MPICH 1.1.0 Sun SPARC Solaris 2.5.1 LAM 6.1, MPICH 1.1.0 Sun UltraSPARC Solaris 2.5.1 LAM 6.1, MPICH 1.1.0 --------------- ----------------------- ------------------------- IBM RS/6000 AIX 3.2.5 LAM 6.1, MPICH 1.1.0 IBM RS/6000 AIX 4.1.1 LAM 6.1, MPICH 1.1.0 IBM SP-2 AIX 4.1.1 IBM, maint level 2.1.0.14 --------------- ----------------------- ------------------------- SGI Irix 6.2 LAM 6.1, MPICH 1.1.0 SGI Irix 6.2 SGI MPI release 3.0 --------------- ----------------------- ------------------------- Intel x86 Linux 2.0.0 LAM 6.1, MPICH 1.1.0 --------------- ----------------------- ------------------------- HP HP-UX 10.20 LAM 6.1, MPICH 1.1.0 HP HP-UX 10.20 HP MPI release 1.2 --------------- ----------------------- ------------------------- If you don't have the MPI implementation listed above, it is "highly recomended" that you upgrade to that release. :-) Various bugs have also been fixed (the HISTORY file in the distribution details all the changes). Some of OOMPI's features include: - Full MPI-1.1 functionality - Implemented as a thin layer on top of the C MPI bindings - Offers convienent and intuitive object-oriented abstractions for message passing - Uses many of the powerful semantic features of the C++ language, such as data typing, polymorphism, etc. Full information, documentation, and the source code distribution for OOMPI Version 1.0.2 can be obtained from: http://www.cse.nd.edu/~lsc/research/oompi/ Any questions, comments, and most especially feedback can be directed to oompi@lsc.nd.edu. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "Make today an OOMPI day!" From mpi-core-human@mcs.anl.gov Wed Jun 25 04:10:35 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA18242; Wed, 25 Jun 1997 04:10:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA07884 for mpi-core-out; Wed, 25 Jun 1997 03:16:06 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA07879 for ; Wed, 25 Jun 1997 03:15:57 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id KAA02998 (8.6.13/IDA-1.6); Wed, 25 Jun 1997 10:15:47 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id KAA15215; Wed, 25 Jun 1997 10:15:49 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199706250815.KAA15215@awsrr.rus.uni-stuttgart.de> Subject: Re: more MPI-2 corrections To: johnmay@llnl.gov (John May) Date: Wed, 25 Jun 1997 10:15:48 +0200 (DST) Cc: mpi-core@mcs.anl.gov In-Reply-To: <19970624122837.IK35219@k2.llnl.gov> from "John May" at Jun 24, 97 12:28:37 pm Content-Type: text Content-Length: 607 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk John May wrote: >226/32 Rolf asks for rationale here. It's still in the document > at 258/20, where it also makes sense. Thank you John. I didn't saw it. Therefore my topic on 262/32 is obsolete. (The rest of my mail stays valid.) Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Wed Jun 25 04:28:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA26398; Wed, 25 Jun 1997 04:28:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA08086 for mpi-core-out; Wed, 25 Jun 1997 03:33:29 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA08080 for ; Wed, 25 Jun 1997 03:33:24 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de with ESMTP id KAA03286 (8.6.13/IDA-1.6 for ); Wed, 25 Jun 1997 10:33:20 +0200 Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for mpi-core@mcs.anl.gov) id KAA15233; Wed, 25 Jun 1997 10:33:22 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199706250833.KAA15233@awsrr.rus.uni-stuttgart.de> Subject: some MPI-2 corr. To: mpi-core@mcs.anl.gov Date: Wed, 25 Jun 1997 10:33:21 +0200 (DST) Content-Type: text Content-Length: 1096 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve, - if I print your postscript documents on a duplex printer often I have the problem that page numbers, running headings and the linenumbers are all on the wrong side of each page. The marked MPI-2 from Jun 19 was wrong. The marked JOD from Jun 19 was correct. It seems that it is not automatically guaranteed that the number of roman numbered pages is even, i.e. Latex forgets to put an additional empty page in front of Chapter 1. In the past it looked like that you have added an empty page by hand before copying it for the meeting. Steve or Bill, - 295:14 A comma after "112 fraction bits" this is the analogous correction as on 249:33 noticed by ...(?) Regards Rolf Rolf Rabenseifner (Computer Center ) Rechenzentrum Universitaet Stuttgart (University of Stuttgart) Allmandring 30 Phone: ++49 711 6855530 D-70550 Stuttgart 80 FAX: ++49 711 6787626 Germany rabenseifner@rus.uni-stuttgart.de From mpi-core-human@mcs.anl.gov Wed Jun 25 17:42:49 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA22596; Wed, 25 Jun 1997 17:42:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA29124 for mpi-core-out; Wed, 25 Jun 1997 16:45:53 -0500 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA29113 for ; Wed, 25 Jun 1997 16:45:40 -0500 From: llewins@ccgate.hac.com Received: from ises01.ES.HAC.COM ([147.16.5.2]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id OAA01250 for ; Wed, 25 Jun 1997 14:44:51 -0700 (PDT) Received: by ises01.ES.HAC.COM; id AA11040; Wed, 25 Jun 1997 14:45:05 -0700 Received: from cc:Mail by CCGATE.HAC.COM id AA867275046; Wed, 25 Jun 97 14:38:20 PST8 Date: Wed, 25 Jun 97 14:38:20 PST8 Encoding: 27 Text Message-Id: <9705258672.AA867275046@CCGATE.HAC.COM> To: mpi-core@mcs.anl.gov Subject: Definition of "Byte" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I believe we are missing a definition for the term "byte" used various places throughout the document. This presumably should go in the "Semantic Terms" section. For simplicity, the definition should be the same as that used in the ANSI C standard, to quote: "Byte -- the unit of data storage large enough to hold any member of the basic character set of the execution environment. It shall be possible to express the address of each individual byte of an object uniquely. A byte is composed of a contiguous sequence of bits, the number of which is implementation defined..." Without this definition, a implementor may interpret byte to mean an 8-bit quantity. This would render the following code non-portable: size_t size; char *buf; MPI_Pack_size (1, MPI_INT, MPI_COMM_WORLD, &size); buf = (char *)malloc(size); Lloyd Lewins Hughes Aircraft Co. llewins@ccgate.hac.com P.S. I note this since on my current machine (a dsp) "bytes" are 32 quantities, and sizeof(char) == sizeof(short) == sizeof(int) == sizeof(long) == sizeof(float) == sizeof(double) == 1!! From mpi-core-human@mcs.anl.gov Wed Jun 25 22:05:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA25115; Wed, 25 Jun 1997 22:05:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA03305 for mpi-core-out; Wed, 25 Jun 1997 21:11:19 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.208.91]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA03300 for ; Wed, 25 Jun 1997 21:11:14 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (8.8.5/7.0m-FWP-MsState); id VAA21571; Wed, 25 Jun 1997 21:10:43 -0500 (CDT) Date: Wed, 25 Jun 1997 21:10:43 -0500 (CDT) From: Tony Skjellum To: llewins@ccgate.hac.com cc: mpi-core@mcs.anl.gov Subject: Re: Definition of "Byte" In-Reply-To: <9705258672.AA867275046@CCGATE.HAC.COM> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Lloyd your point suggests that we should also support an octet type, specifically signifying non-interpretted, network standard units. -Tony On Wed, 25 Jun 1997 llewins@ccgate.hac.com wrote: > Date: Wed, 25 Jun 97 14:38:20 PST8 > From: llewins@ccgate.hac.com > To: mpi-core@mcs.anl.gov > Subject: Definition of "Byte" > > I believe we are missing a definition for the term "byte" used various > places throughout the document. This presumably should go in the > "Semantic Terms" section. For simplicity, the definition should be > the same as that used in the ANSI C standard, to quote: > > "Byte -- the unit of data storage large enough to hold any member of > the basic character set of the execution environment. It shall be > possible to express the address of each individual byte of an object > uniquely. A byte is composed of a contiguous sequence of bits, the > number of which is implementation defined..." > > Without this definition, a implementor may interpret byte to mean an > 8-bit quantity. This would render the following code non-portable: > > size_t size; > char *buf; > > MPI_Pack_size (1, MPI_INT, MPI_COMM_WORLD, &size); > buf = (char *)malloc(size); > > Lloyd Lewins > Hughes Aircraft Co. > llewins@ccgate.hac.com > > P.S. I note this since on my current machine (a dsp) "bytes" are 32 > quantities, and sizeof(char) == sizeof(short) == sizeof(int) == > sizeof(long) == sizeof(float) == sizeof(double) == 1!! > > A. Skjellum, PhD, Assoc. Prof. of Computer Science; Mississippi State University http://www.cs.msstate.edu/~tony; tony@cs.msstate.edu; 601-325-8435 (FAX -8997) "You are in a maze of twisty little passages, all alike." Support MPI & MPI/RT! From mpi-core-human@mcs.anl.gov Wed Jun 25 23:54:29 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA26182; Wed, 25 Jun 1997 23:54:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA04450 for mpi-core-out; Wed, 25 Jun 1997 22:59:51 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA04445 for ; Wed, 25 Jun 1997 22:59:47 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id UAA22840; Wed, 25 Jun 1997 20:59:08 -0700 From: "William Saphir" Message-Id: <9706252059.ZM22839@nsgi3.lbl.gov> Date: Wed, 25 Jun 1997 20:59:07 -0700 In-Reply-To: llewins@ccgate.hac.com "Definition of "Byte"" (Jun 25, 2:38pm) References: <9705258672.AA867275046@CCGATE.HAC.COM> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: llewins@ccgate.hac.com, mpi-core@mcs.anl.gov Subject: Re: Definition of "Byte" Cc: wcs@nsgi3.lbl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Jun 25, 2:38pm, llewins@ccgate.hac.com wrote: > Subject: Definition of "Byte" > I believe we are missing a definition for the term "byte" used various > places throughout the document. This presumably should go in the > "Semantic Terms" section. For simplicity, the definition should be > the same as that used in the ANSI C standard, to quote: > > "Byte -- the unit of data storage large enough to hold any member of > the basic character set of the execution environment. It shall be > possible to express the address of each individual byte of an object > uniquely. A byte is composed of a contiguous sequence of bits, the > number of which is implementation defined..." Ack! By sending this note to the 100-or-so folks on mpi-core, you have probably increased by an order of magnitude the number of people in the world who are aware of this fact about C. :-) This is an MPI-1 issue - has anyone ever brought up the issue before? I make the alternate proposal: if we make any clarification, we should clarify that MPI_BYTE refers to 8 bits as do other references to "byte" (possibly with a clarification that there may be unusual computers where strange things happen) The reasons are: 1. ANSI C has this weird definition, but the (nearly) universally understood meaning of "byte" is "8 bits" (unless qualified, as in "4-bit bytes"). MPI is universal :-) 2. MPI also addresses Fortran and other languages. We now have an MPI_SIZEOF function in Fortran that is not tied to C. [An interesting experiment: do you have a Fortran compiler on your machine, and does it support REAL*N, etc. If so, is REAL*N of size N*32 bits or N*8 bits?] 3. Both the MPI-1 MPI_BYTE and the new Fortran datatypes require that the size of a "byte" be the same on all machines. MPI_BYTE is for transferring "raw" (untyped) data. If bytes are different sizes, transferring raw data is impossible. 4. Throughout the MPI document and the "complete reference" there are potential problem areas which we would need to think about carefully. Fixing all this at this late date is a practical impossibility. --------------------------- That having been said, I agree that for your type of machine there are problems. Many things in MPI-1 are defined in terms of bytes or byte displacements or byte addresses. Bill From mpi-core-human@mcs.anl.gov Thu Jun 26 08:46:51 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA13910; Thu, 26 Jun 1997 08:46:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA09250 for mpi-core-out; Thu, 26 Jun 1997 07:52:01 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA09244 for ; Thu, 26 Jun 1997 07:51:54 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id HAA12316; Thu, 26 Jun 1997 07:51:41 -0500 (CDT) Date: Thu, 26 Jun 1997 07:51:41 -0500 (CDT) Message-Id: <199706261251.HAA12316@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: minor corrections Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk 43:34 refers to C. It should say C and C++. 172:15 has mpifunc which should be \mpifunc 304:20 missing MPI_SIGNED_CHAR. It is definitely in C. The text on 78:15+ talks about C but not C++. Should this also be in C++? Steve From mpi-core-human@mcs.anl.gov Thu Jun 26 10:24:47 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA15799; Thu, 26 Jun 1997 10:24:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA11407 for mpi-core-out; Thu, 26 Jun 1997 09:27:46 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA11401; Thu, 26 Jun 1997 09:27:41 -0500 Message-Id: <199706261427.JAA11401@antares.mcs.anl.gov> To: knighten@co.intel.com (Bob Knighten) cc: lederman@cs.wisc.edu, mpi-core@mcs.anl.gov Subject: Re: correct pthread call In-Reply-to: Message from Bob Knighten of "Tue, 24 Jun 1997 11:29:22 -0700." <199706241829.LAA06325@knight.co.intel.com> Date: Thu, 26 Jun 1997 09:27:33 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I should have been more specific when I told Steve that I wasn't positive about the pthread_create call. I formulated this by checking both the POSIX standard and some of the man pages on the local systems. The SYNTAX (with the possible exception of the return type of the start_routine) is correct, as is the choice of NULL for the pthread_attr. The real question is with the surrounding code. Is it ok to allocate the first argument off the stack? Does anything else need to be called? I didn't have time to chase down all of the details of using the POSIX thread calls; I simply corrected the syntax of this single call. It also appeared that no thread termination call was needed (exit from the start_routine provides the correct behavior), but again, I haven't used POSIX threads and I'm not positive that I haven't missed something. Bill From mpi-core-human@mcs.anl.gov Thu Jun 26 10:44:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA16445; Thu, 26 Jun 1997 10:44:58 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA13102 for mpi-core-out; Thu, 26 Jun 1997 09:50:17 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [129.34.139.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA13093 for ; Thu, 26 Jun 1997 09:50:12 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw2.watson.ibm.com (8.8.5/06-04-97) with ESMTP id KAA08018; Thu, 26 Jun 1997 10:50:44 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-20-97) with SMTP id KAA32872; Thu, 26 Jun 1997 10:50:02 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564C2.0051777D ; Thu, 26 Jun 1997 10:49:50 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: llewins@ccgate.hac.com cc: Mpi-Core@mcs.anl.gov Message-ID: <852564C2.0050B44D.00@watngi01.watson.ibm.com> Date: Thu, 26 Jun 1997 10:49:22 -0400 Subject: Re: Definition of "Byte" Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I quote from MPI1 (18:42). "A value of type MPI_BYTE consists of a byte (8 binary digits)". We had this discussion in MPI1, in the context of Cray pointers. I think we really wanted a byte to be 8 bits, and byte addressing to be in 8 bit displacements, otherwise one has problems with datatypes. I would argue that in your case, the MPI definition of a byte will be different than the machine definition, and the code example you put will not be portable. From mpi-core-human@mcs.anl.gov Thu Jun 26 14:23:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA20242; Thu, 26 Jun 1997 14:22:57 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA22089 for mpi-core-out; Thu, 26 Jun 1997 13:22:47 -0500 Received: from igate1.hac.com (igate1.HAC.COM [192.48.33.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA22064 for ; Thu, 26 Jun 1997 13:22:28 -0500 From: llewins@ccgate.hac.com Received: from ises01.ES.HAC.COM ([147.16.5.2]) by igate1.hac.com (8.8.4/8.8.4) with SMTP id LAA07861 for ; Thu, 26 Jun 1997 11:21:21 -0700 (PDT) Received: by ises01.ES.HAC.COM; id AA02207; Thu, 26 Jun 1997 11:21:23 -0700 Received: from cc:Mail by CCGATE.HAC.COM id AA867349182; Thu, 26 Jun 97 10:52:06 PST8 Date: Thu, 26 Jun 97 10:52:06 PST8 Encoding: 61 Text Message-Id: <9705268673.AA867349182@CCGATE.HAC.COM> To: "Marc Snir" Cc: Mpi-Core@mcs.anl.gov Subject: Re[2]: Definition of "Byte" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc, I stand corrected -- thank you (-: So now I understand the difference between C "sizeof" and MPI_Type_size!! Thus, the correct (portable) code should be: ------------------ int charsize; size_t size; char *buf; /* Find number of 8-bit bytes in a C byte or storage unit */ MPI_Type_size (MPI_CHAR, &charsize); /* Find the required number of bytes to hold the packed message */ MPI_Pack_size (1, MPI_INT, MPI_COMM_WORLD, &size); /* Allocate the correct number of C storage units to hold the message */ buf = (char *)malloc(size/charsize); ------------------- Given the above, the definitions of some of the collective functions are unclear. For example, MPI_Gather states: "and the root had executed n calls to MPI_Recv(recvbuf + i.recvcount.extent(recvtype),...)" Since recvbuf is a choice, i.e. a native machine address, and i.recvcount.extent(recvtype) is a byte displacement, it is not obvious what "+" means here. Certainly the implementors of MPICH have it wrong when they code: mpi_errno = MPI_Recv( (void*)(((char*)recvbuf)+i*extent*recvcount), Lloyd Lewins Hughes Aircraft Co. llewins@ccgate.hac.com P.S. Bill - No Fortran compiler available for this machine (-: ______________________________ Reply Separator _________________________________ Subject: Re: Definition of "Byte" Author: "Marc Snir" at CCGATE Date: 6/26/97 7:54 AM I quote from MPI1 (18:42). "A value of type MPI_BYTE consists of a byte (8 binary digits)". We had this discussion in MPI1, in the context of Cray pointers. I think we really wanted a byte to be 8 bits, and byte addressing to be in 8 bit displacements, otherwise one has problems with datatypes. I would argue that in your case, the MPI definition of a byte will be different than the machine definition, and the code example you put will not be portable. From mpi-core-human@mcs.anl.gov Thu Jun 26 15:50:53 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA21059; Thu, 26 Jun 1997 15:50:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA27542 for mpi-core-out; Thu, 26 Jun 1997 14:53:44 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA27537 for ; Thu, 26 Jun 1997 14:53:40 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id MAA24574; Thu, 26 Jun 1997 12:53:03 -0700 From: "William Saphir" Message-Id: <9706261253.ZM24573@nsgi3.lbl.gov> Date: Thu, 26 Jun 1997 12:53:02 -0700 In-Reply-To: llewins@ccgate.hac.com "Re[2]: Definition of "Byte"" (Jun 26, 10:52am) References: <9705268673.AA867349182@CCGATE.HAC.COM> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: llewins@ccgate.hac.com, "Marc Snir" Subject: Re: Re[2]: Definition of "Byte" Cc: Mpi-Core@mcs.anl.gov Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Jun 26, 10:52am, llewins@ccgate.hac.com wrote: > Subject: Re[2]: Definition of "Byte" > Marc, I stand corrected -- thank you (-: So now I understand the > difference between C "sizeof" and MPI_Type_size!! Marc wrote: > We had this discussion in MPI1, in the context of Cray pointers. I think > we really wanted a byte to be 8 bits, and byte addressing to be in 8 bit > displacements, otherwise one has problems with datatypes. [We're all in agreement, so this is just additional commentary...] What Lloyd brought up is actually a separate issue, since even on Cray vector machines, an ANSI C byte is 8 bits. The issue for Cray machines is whether addresses are word addresses or byte addresses. Lloyd has found a truly weird machine. While the MPI standard appears to be consistent, I noticed that Marc's book (the "Complete" reference) has an error. :-) Under MPI_TYPE_EXTENT it says: "For example. MPI_TYPE_EXTENT(MPI_INT, extent) will return in extent the size, in bytes, of an int -- the same value that would be returned by the C call sizeof(int)". This is untrue, as Lloyd points out. Bill From mpi-core-human@mcs.anl.gov Thu Jun 26 15:58:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA21095; Thu, 26 Jun 1997 15:57:59 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA27891 for mpi-core-out; Thu, 26 Jun 1997 15:02:49 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA27886 for ; Thu, 26 Jun 1997 15:02:44 -0500 Received: from porthos.lsc.nd.edu (jsiek@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id PAA06505 for ; Thu, 26 Jun 1997 15:02:40 -0500 (EST) Received: (from jsiek@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) id PAA12774; Thu, 26 Jun 1997 15:02:39 -0500 (EST) Date: Thu, 26 Jun 1997 15:02:39 -0500 (EST) Message-Id: <199706262002.PAA12774@porthos.lsc.nd.edu> From: jeremy siek To: mpi-core@mcs.anl.gov Subject: C++ version, Create_errhandler correction Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk 62:31 The argument to MPI::Comm::Create_errhandler should be const MPI::Comm::Errhandler*, not const MPI_Comm_errhandler. Jeremy From mpi-core-human@mcs.anl.gov Fri Jun 27 14:45:02 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA04360; Fri, 27 Jun 1997 14:45:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA26462 for mpi-core-out; Fri, 27 Jun 1997 13:49:11 -0500 Received: from win62.nas.nasa.gov (win62.nas.nasa.gov [129.99.50.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA26457 for ; Fri, 27 Jun 1997 13:49:07 -0500 Received: (from ies@localhost) by win62.nas.nasa.gov (951211.SGI.8.6.12.PATCH1502/NAS.6.1) id LAA25742; Fri, 27 Jun 1997 11:49:03 -0700 Date: Fri, 27 Jun 1997 11:49:03 -0700 Message-Id: <199706271849.LAA25742@win62.nas.nasa.gov> From: "Ian E. Stockdale" To: mpi-core@mcs.anl.gov Subject: Re: I/O Discussion Items Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Marc Snir pointed out that several points in the i/o consistency and semantics section needed clarifying. The following two changes address these points. Change 256:11-13 to: "The requirements for guaranteeing sequential consistency among all accesses to a particular file are divided into the three cases given below. If any of these requirements are not met, then the value of all data in that file is implementation dependent." Change existing 256:21-23 ] If $A_1$ and $A_2$ do not conflict, \MPI/ guarantees that they ] are sequentially consistent, and hence, appear to the application ] as atomic. to: If for any access $A_1$, there is no access $A_2$ that conflicts with $A_1$, then \MPI/ guarantees sequential consistency. - Ian -- Ian E. Stockdale MRJ Technology Solutions, NASA Ames Research Center ies@nas.nasa.gov (415) 604-4628 From mpi-core-human@mcs.anl.gov Fri Jun 27 15:05:30 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA05020; Fri, 27 Jun 1997 15:05:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA27411 for mpi-core-out; Fri, 27 Jun 1997 14:10:53 -0500 Received: from win153.nas.nasa.gov (win153.nas.nasa.gov [129.99.33.68]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA27401 for ; Fri, 27 Jun 1997 14:10:49 -0500 Received: from win153.nas.nasa.gov (nitzberg@localhost) by win153.nas.nasa.gov (8.7.6/NAS.6.1) with ESMTP id MAA29741 for ; Fri, 27 Jun 1997 12:10:44 -0700 (PDT) Message-Id: <199706271910.MAA29741@win153.nas.nasa.gov> To: mpi-core@mcs.anl.gov Subject: I/O Type Matching Clarifications Date: Fri, 27 Jun 1997 12:10:43 -0700 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk After some discussion regarding the type matching rules for I/O, the following minor clarifications are suggested for section 9.6.5: ADD the following clarification after "matches any type in a data access." (259:12): Specifically, if \mpiarg{etype} is \consti{MPI\_BYTE}, then this matches any \mpiarg{datatype} in a data access operation. and DELETE the last sentence of the paragraph at 259:15: An etype or data access buffer constructed entirely from MPI_BYTE matches all types. - bill From mpi-core-human@mcs.anl.gov Tue Jul 1 11:13:48 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA16799; Tue, 1 Jul 1997 11:13:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA01528 for mpi-core-out; Tue, 1 Jul 1997 10:16:07 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id KAA01523 for ; Tue, 1 Jul 1997 10:16:02 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA15892; Tue, 1 Jul 1997 11:12:02 -0400 Message-Id: <33B91F20.41C6@pok.ibm.com> Date: Tue, 01 Jul 1997 11:15:44 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: MPI_TYPE_GET_ENVELOPE combiners Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Page 186 lists the combiners to be returned and for the MPI-1 calls MPI_TYPE_HVECTOR, MPI_TYPE_HINDEXED and MPI_TYPE_STRUCT. Because there is a potential problem with information loss when these old calls are used from Fortran in an MPI which has 64 bit addresses but 32 bit Fortran INTEGERS, all MPI implementations are currently required to record that fact that the Fortran constructor was used and to return something like MPI_COMBINER_HINDEXED_INTEGER rather than MPI_COMBINER_HINDEXED. In a threaded MPI which layers Fortran on top of C bindings it can be expensive to have each of the Fortran layer routines place a flag in thread specific storage and then have each underlying C routine check that thread specific flag. For a non-thread MPI a simple global flag will do but not for a threaded. Threads require pthread_getspecific() which can be expensive. Because the ONLY REASON for identifying the Fortran call with special combiners relates to versions of MPI which have different lengths for MPI_Aint and INTEGER, I would like to eliminate this requirement for MPI implementations which CANNOT have the truncation problem. This would be implementations today which have 32 bits for both types and maybe in a few years, implementations with 64 bit addresses and INTEGERS. I propose the following additional paragraph after line 26 on page 186. I have discussed this with Marc Snir and he does not see any problem. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ The only reason for returning MPI\_COMBINER\_HVECTOR\_INTEGER rather than MPI\_COMBINER\_HVECTOR for a MPI\_TYPE\_HVECTOR call from Fortran is to warn of possible information loss. An MPI implementation in which MPI\_Aint and Fortran INTEGER are the same size may return the same combiner for Fortran as it does for C and the new MPI\_TYPE\_CREATE_xxx routines. In such an implementation there will be no information loss. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue Jul 1 12:13:01 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA17483; Tue, 1 Jul 1997 12:13:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA05412 for mpi-core-out; Tue, 1 Jul 1997 11:15:06 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA05407 for ; Tue, 1 Jul 1997 11:15:02 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA10146; Tue, 1 Jul 1997 12:11:11 -0400 Message-Id: <33B92CFD.167E@pok.ibm.com> Date: Tue, 01 Jul 1997 12:14:53 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: Nicholas Nevin -SMCC - High Performance Computing Cc: mpi-core@mcs.anl.gov Subject: Re: MPI_TYPE_GET_ENVELOPE combiners References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Nicholas Nevin -SMCC - High Performance Computing wrote: > Why do you need to use thread specific storage? This is a property of the > datatype and you should be able to just set a flag in the datatype structure > while in the C wrapper. > > -nick > > -- > Nick Nevin Sun Microsystems - HPC Group > (508) 442-0219 nicholas.nevin@sun.com The Fortran program calls MPI_TYPE_HVECTOR. This is wrapper code which does the right things to target C bindings and calls MPI_Type_hvector. A C program simply calls MPI_Type_hvector. There is no way for the code of MPI_Type_hvector to know whether it was called from C or via the Fortran wrapper. In a non-thread MPI I would have a global flag called "fort_layer" exported from the core library of C binding routines. For the routines which track Fortran vs. C, the Fortran wrapper would set "fort_layer", the C routine would read and clear it. When the C routine is called directly from C it will find "fort_layer==FALSE". Without a way for the C code to tell whether it is called by the user's C code or by the wrapper, it cannot record the info in the MPI_Datatype object it is creating. In a threaded MPI this flag for telling the C code it was called fron the Fortran layer must be in thread specific storage. The locks to protect MPI internal state are not needed until the C routine is entered so I do not aquire a lock at the Fortran layer. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue Jul 1 13:26:20 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA18294; Tue, 1 Jul 1997 13:26:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA08216 for mpi-core-out; Tue, 1 Jul 1997 12:30:12 -0500 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA04944 for ; Tue, 1 Jul 1997 11:00:56 -0500 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (SMI-8.6/mail.byaddr) with SMTP id JAA07058; Tue, 1 Jul 1997 09:24:52 -0700 Received: from suneast.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id MAA01497; Tue, 1 Jul 1997 12:00:04 -0400 Received: from caja.East.Sun.COM by suneast.East.Sun.COM (SMI-8.6/SMI-SVR4) id MAA22707; Tue, 1 Jul 1997 12:00:05 -0400 Received: from caja by caja.East.Sun.COM (SMI-8.6/SMI-SVR4) id MAA18477; Tue, 1 Jul 1997 12:00:03 -0400 Date: Tue, 1 Jul 1997 12:00:01 -0400 (EDT) From: Nicholas Nevin -SMCC - High Performance Computing Reply-To: Nicholas Nevin -SMCC - High Performance Computing Subject: Re: MPI_TYPE_GET_ENVELOPE combiners To: mpi-core@mcs.anl.gov, treumann@kgn.ibm.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Hf5yKr3A/pAP5c1falolIA== X-Mailer: dtmail 1.1.0 CDE Version 1.1 SunOS 5.5.1 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Page 186 lists the combiners to be returned and for the MPI-1 calls > MPI_TYPE_HVECTOR, MPI_TYPE_HINDEXED and MPI_TYPE_STRUCT. Because there > is a potential problem with information loss when these old calls are > used from Fortran in an MPI which has 64 bit addresses but 32 bit > Fortran INTEGERS, all MPI implementations are currently required to > record that fact that the Fortran constructor was used and to return > something like MPI_COMBINER_HINDEXED_INTEGER rather than > MPI_COMBINER_HINDEXED. > > In a threaded MPI which layers Fortran on top of C bindings it can be > expensive to have each of the Fortran layer routines place a flag in > thread specific storage and then have each underlying C routine check > that thread specific flag. For a non-thread MPI a simple global flag > will do but not for a threaded. Threads require pthread_getspecific() > which can be expensive. Why do you need to use thread specific storage? This is a property of the datatype and you should be able to just set a flag in the datatype structure while in the C wrapper. -nick -- Nick Nevin Sun Microsystems - HPC Group (508) 442-0219 nicholas.nevin@sun.com From mpi-core-human@mcs.anl.gov Tue Jul 1 13:36:21 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA18412; Tue, 1 Jul 1997 13:36:20 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA08581 for mpi-core-out; Tue, 1 Jul 1997 12:42:01 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id MAA08575 for ; Tue, 1 Jul 1997 12:41:54 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA09925; Tue, 1 Jul 1997 13:38:02 -0400 Message-Id: <33B94158.15FB@pok.ibm.com> Date: Tue, 01 Jul 1997 13:41:44 -0400 From: Dick Treumann Organization: IBM Power Parallel Systems - RS6000 Division X-Mailer: Mozilla 3.01 (X11; U; AIX 1) Mime-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: Re: MPI_TYPE_GET_ENVELOPE combiners Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I do not intend to push for a rewrite of the discussion of MPI_COMBINER_HVECTOR_INTEGER etc but I will add this extra point in support of the limited change I proposed: Portable code calling MPI_TYPE_GET_ENVELOPE and getting back the combiner MPI_COMBINER_xxxxx_INTEGER must assume the worst. There is no way, as far as I know, for a call which gets back this sort of combiner to test whether the combiner indicates real potential for lost information or only means there was a perfectly good call from Fortran. It would actually be reasonable to say that these special combiners really mean "DANGER" rather than "called from Fortran". In this view, a high quality implementation of MPI will want to avoid raising the DANGER flag any time it can be certain there is no possible data loss. Dick -- Dick Treumann IBM RS/6000 Division (Internet) treumann@pok.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue Jul 1 13:39:48 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA18457; Tue, 1 Jul 1997 13:39:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA08707 for mpi-core-out; Tue, 1 Jul 1997 12:45:28 -0500 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA08390 for ; Tue, 1 Jul 1997 12:35:34 -0500 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (SMI-8.6/mail.byaddr) with SMTP id KAA29016 for ; Tue, 1 Jul 1997 10:59:38 -0700 Received: from suneast.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id NAA12624; Tue, 1 Jul 1997 13:34:48 -0400 Received: from caja.East.Sun.COM by suneast.East.Sun.COM (SMI-8.6/SMI-SVR4) id NAA26985; Tue, 1 Jul 1997 13:34:50 -0400 Received: from caja by caja.East.Sun.COM (SMI-8.6/SMI-SVR4) id NAA18619; Tue, 1 Jul 1997 13:34:49 -0400 Date: Tue, 1 Jul 1997 13:34:49 -0400 (EDT) From: Nicholas Nevin -SMCC - High Performance Computing Reply-To: Nicholas Nevin -SMCC - High Performance Computing Subject: Re: MPI_TYPE_GET_ENVELOPE combiners To: mpi-core@mcs.anl.gov Message-ID: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Uvl6g65/NUl/0BmWh7jcBQ== X-Mailer: dtmail 1.1.0 CDE Version 1.1 SunOS 5.5.1 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Lest anyone construe my earlier posting as an objection to Dick's proposal. It isn't. ------------- Begin Forwarded Message ------------- From: Nicholas Nevin -SMCC - High Performance Computing Subject: Re: MPI_TYPE_GET_ENVELOPE combiners To: treumann@kgn.ibm.com Mime-Version: 1.0 Content-MD5: Awbz8EAcNa+lXaMgFfYKiw== > OOPS - just read your note again. I should not have said "exactly". We > do have a routine called MPI_TYPE_HVECTOR which gets control from > fortran and then calls MPI_Type_hvector. No compiler magic. > -- Hi Dick. Thanks for the explanation. I understand the problem now :-) For what it's worth I have no objections to the new text you proposed. -nick. -- Nick Nevin Sun Microsystems - HPC Group (508) 442-0219 nicholas.nevin@sun.com ------------- End Forwarded Message ------------- -- Nick Nevin Sun Microsystems - HPC Group (508) 442-0219 nicholas.nevin@sun.com From mpi-core-human@mcs.anl.gov Tue Jul 1 15:19:30 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA19413; Tue, 1 Jul 1997 15:19:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA11889 for mpi-core-out; Tue, 1 Jul 1997 14:24:24 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA11884 for ; Tue, 1 Jul 1997 14:24:21 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id OAA03675 for ; Tue, 1 Jul 1997 14:24:16 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id OAA03680 for ; Tue, 1 Jul 1997 14:24:16 -0500 (EST) Date: Tue, 1 Jul 1997 14:24:16 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov Subject: constant problems :) (fwd) Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Our studious grad student just noticed a minor problem with the "const" constants. It seems that C++ will not allow MPI_COMM_WORLD and MPI_COMM_SELF to be const, because it is necessary to be able to invoke non-const member functions on it. See below. --- MPI::COMM_WORLD, MPI::COMM_SELF can't be constant because functions like Set_errhandler are not const Need to remove the const from the arguments because function pointers are not const Op::Init(const User_function* fn, bool commute); Comm::Create_errhandler(const Comm::Errhandler_fn* func); Comm::Create_keyval(const Copy_attr_function*, const Delete_attr_function*, void*); I've glanced through the other constants and I don't think there are any other problems. --- {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Mon Jul 7 23:07:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id XAA16845; Mon, 7 Jul 1997 23:07:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id WAA23044 for mpi-core-out; Mon, 7 Jul 1997 22:11:25 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id WAA23039 for ; Mon, 7 Jul 1997 22:11:21 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id WAA14637; Mon, 7 Jul 1997 22:11:14 -0500 (CDT) Date: Mon, 7 Jul 1997 22:11:14 -0500 (CDT) Message-Id: <199707080311.WAA14637@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov In-reply-to: <199706192333.SAA04926@rap.cs.wisc.edu> (message from Steve Huss-Lederman on Thu, 19 Jun 1997 18:33:51 -0500 (CDT)) Subject: Re: T-shirt winner & proof version coming Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I am pleased to announce that Rajeev Thakur is the winner of the MPI-2 T-shirt for the most comments made about the draft version. He had the largest number and the most words too. Many thanks to him and others who made comments. I plan to release a final proof version soon (hopefully Wednesday). People will then have a few days to check over the last minute changes to make sure all was done correctly. Steve From mpi-core-human@mcs.anl.gov Fri Jul 11 12:08:43 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA00205; Fri, 11 Jul 1997 12:08:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA27211 for mpi-core-out; Fri, 11 Jul 1997 11:06:34 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA27198 for ; Fri, 11 Jul 1997 11:06:26 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id LAA28599; Fri, 11 Jul 1997 11:06:21 -0500 (CDT) Date: Fri, 11 Jul 1997 11:06:21 -0500 (CDT) Message-Id: <199707111606.LAA28599@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: document ready for final check Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The (really) final proof version of the MPI-2 document is now available (The JOD has not changed). Due to the number of changes, the authors felt it was best to give everyone a chance to review these changes to make sure everything was done correctly. Please keep in mind this is not an opportunity to raise new points - that has already occurred. It is meant to be a chance to review the last set of changes for possible mistakes. I have created a master list of the comments received and the actions taken. It can be got at http://www.cs.wisc.edu/~lederman/mpif/do (or equivalently at ftp://ftp.cs.wisc.edu/pub/mpi2/do). We now have the domain mpi-forum.org set up with the MPI Forum home page at http://www.mpi-forum.org. The latest version of the document is there for your review as well as getting feedback on the home page. You can ignore the warning about not using this page - that is meant for the public. To keep the ball rolling, I need to have all comments by 12:00 Central US time on Tuesday, July 15. The goal is to make the public announcement by the middle of next week. The proposed text of that announcement is attached for your review. Steve ---------------------------------------------------------------------- To: comp.parallel.mpi, comp.parallel The MPI Forum is pleased to announce the completion of MPI-2, a set of extensions to the Message Passing Interface standard. Official information about MPI, including the new standard, can be found on the MPI Forum home page, located at http://www.mpi-forum.org. This page will also serve as the official location for corrections and future official MPI documents. The MPI-2 document describes two new versions of the standard: MPI-1.2 and MPI-2.0. MPI-1.2 contains clarifications and corrections to MPI-1.1. MPI-2.0 adds to MPI-1.2 substantial new functionality, including process creation and management, one-sided communications, extended collective operations, external interfaces, I/O, language bindings for C++ and Fortran 90, and miscellaneous topics. Other information available on the MPI Forum home page includes the MPI Journal of Development, previous MPI documents, errata, and contact information. From mpi-core-human@mcs.anl.gov Sun Jul 13 15:35:30 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA20283; Sun, 13 Jul 1997 15:35:30 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA11105 for mpi-core-out; Sun, 13 Jul 1997 14:40:55 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA11083; Sun, 13 Jul 1997 14:39:37 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id MAA04528; Sun, 13 Jul 1997 12:39:35 -0700 Date: Sun, 13 Jul 1997 12:39:35 -0700 From: wcs@nsgi3.lbl.gov (William Saphir) Message-Id: <199707131939.MAA04528@nsgi3.lbl.gov> To: raja@tbag.rsn.hp.com, mpi-core@mcs.anl.gov Subject: INTEGER(KIND=MPI_ADDRESS_KIND) Cc: mpi-bind@mcs.anl.gov Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > From: wcs (William Saphir) > > > From: Raja Daoud > > > > Bill, > > A request for clarification: is it the general rule that, for new MPI-2 > > functions that have Fortran arguments of this type, F77 users who want > > to call them must pass: > > > > 1) INTEGER*8 always > > 1) INTEGER*8 or *4 arguments as appropriate > > 3) INTEGER always and it's up to the implementation to have different > > wrappers for F77 and F90, the F77 ones being "lossy" on 64-bit machines? > > > > I see this rule in section 4.13.1 (p. 66) but it's only talking about the > > 4 new datatype creation functions, not every function with address_kind. > > > > Nick says: > > > > > My understanding is that F77 users must pass an address sized integer to > > > these routines. On 32 bit machines they can use INTEGER or INTEGER*4 and on > > > 64 bit machines must use INTEGER*8 or can use MPI_INTEGER if it's 64 bits. > > > How exactly they make their F77 code portable now I am not sure. Use the > > > C preprocessor I guess. > > > > > > This means only one wrapper is needed which assumes the > > > incoming ints are address sized. > > > > Which is it? Thanks. > > > > --Raja > > > > The short answer: option 2 (labeled as the 2nd option "1") > INTEGER*8, INTEGER*4 or INTEGER as appropriate > > The long answer: > - Portability of F77 codes is simply not possible without > seriously breaking something. (I.e. the old bindings were > technically portable, but seriously broken in other ways). > - My answer of option 2 is based on what I just read on the text > (which seems pretty clear in several places, including terms and > i/o). But before reading, I thought we had settled upon: always > INTEGER*8 unless the compiler doesn't support this, in which > case it would be implementation dependent. I had preferred > this because it is somewhat more portable, though internally > there is more baggage on 32-bit systems that support > INTEGER*8 (such as Nick's systems). However, the current > text will work, so we're ok. > > Bill > > > ps. Should we forward this to mpi-core, to place the discussion > in the record? > > From: Raja Daoud > Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) > To: wcs (William Saphir) > Date: Sun, 13 Jul 1997 13:51:40 CDT > Cc: nevin@caja.east.sun.com > > > > ps. Should we forward this to mpi-core, to place the discussion > > in the record? > > Thanks Bill. Please email the answer to mpi-core, it would help users > and implementors avoid misunderstandings. Done. > > In your opinion, how big a violation of the std would it be if some > vendors decide to tell their customers to always use integer*8 as > "the appropriate thing" on their systems, for both 32 and 64-bit > versions of the library? Reason: starting with our next release (1.3) > we will be supporting both 32/64-bit because HP-UX will have both > modes. It would be nice to have users not modify their source code no > matter how they choose to compile their apps. Same issue would be faced > by SGI. So if a vendor is willing to handle the "if (sizeof(void *) < 8)" > scenario in some wrappers and at least guarantee source code portability > between its own versions of the library, is that a big boo-boo? My opinion is that an implementation that requires integer*8 for the new functions is within the law and spirit of the standard, though this is apparently not required by the standard. Presumably all vendors/implementors will want to do this except those who have no 64-bit capability at all (and even there it might make sense). Hopefully users will move to F90 and when they use the new functions they will also use the F90 way of doing things. Bill From mpi-core-human@mcs.anl.gov Mon Jul 14 10:24:47 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA01636; Mon, 14 Jul 1997 10:24:45 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA25750 for mpi-core-out; Mon, 14 Jul 1997 09:29:28 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [198.81.209.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA25679; Mon, 14 Jul 1997 09:27:07 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw2.watson.ibm.com (8.8.5/06-04-97) with ESMTP id KAA18174; Mon, 14 Jul 1997 10:26:33 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/06-28-97-beta) with SMTP id KAA17016; Mon, 14 Jul 1997 10:27:03 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564D4.004F5F22 ; Mon, 14 Jul 1997 10:26:56 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: wcs@nsgi3.lbl.gov cc: raja@tbag.rsn.hp.com, mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov Message-ID: <852564D4.004EB8E0.00@watngi01.watson.ibm.com> Date: Mon, 14 Jul 1997 10:26:47 -0400 Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > In your opinion, how big a violation of the std would it be if some > vendors decide to tell their customers to always use integer*8 as > "the appropriate thing" on their systems, for both 32 and 64-bit > versions of the library? Reason: starting with our next release (1.3) > we will be supporting both 32/64-bit because HP-UX will have both > modes. It would be nice to have users not modify their source code no > matter how they choose to compile their apps. Same issue would be faced > by SGI. So if a vendor is willing to handle the "if (sizeof(void *) < 8)" > scenario in some wrappers and at least guarantee source code portability > between its own versions of the library, is that a big boo-boo? My opinion is that an implementation that requires integer*8 for the new functions is within the law and spirit of the standard, though this is apparently not required by the standard. Presumably all vendors/implementors will want to do this except those who have no 64-bit capability at all (and even there it might make sense). Hopefully users will move to F90 and when they use the new functions they will also use the F90 way of doing things. Bill ************ Such a choice clearly violates the current MPI2 standard. I quote (66:25) "The new functions [with address sized arguments] are synonymous with the old functions in C/C++, or on Fortran systems where default INTEGERs are address sized." Thus, on a 32 bit system, MPI_TYPE_CREATE_HVECTOR is synonymous (in Fortran) with MPI_TYPE_HVECTOR. Since the later has a 32 bit INTEGER stride argument, so must the former. As a matter of fact, we just modified the type decode functions so that, on 32 bit machines, they need not distinguish MPI_TYPE_CREATE_HVECTOR from MPI_TYPE_HVECTOR (Fortran). ************ From mpi-core-human@mcs.anl.gov Mon Jul 14 14:25:44 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA06212; Mon, 14 Jul 1997 14:25:43 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA06176 for mpi-core-out; Mon, 14 Jul 1997 13:26:23 -0500 Received: from win62.nas.nasa.gov (win62.nas.nasa.gov [129.99.50.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA06150 for ; Mon, 14 Jul 1997 13:26:09 -0500 Received: (from ies@localhost) by win62.nas.nasa.gov (951211.SGI.8.6.12.PATCH1502/NAS.6.1) id LAA06747; Mon, 14 Jul 1997 11:25:54 -0700 Date: Mon, 14 Jul 1997 11:25:54 -0700 Message-Id: <199707141825.LAA06747@win62.nas.nasa.gov> From: "Ian E. Stockdale" To: mpi-core@mcs.anl.gov In-reply-to: <852564D4.004EB8E0.00@watngi01.watson.ibm.com> (snir@watson.ibm.com) Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My reading of pp. 65-66 agrees with Marc's (and, I think, Nick's). The best solution to the problem Nick points out: > > > How exactly they make their F77 code portable now I am not sure. Use the > > > C preprocessor I guess. is for users and vendors to switch to f90. With f77, users must be aware of how big addresses are on their system(s). -Ian From mpi-core-human@mcs.anl.gov Mon Jul 14 16:32:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA07353; Mon, 14 Jul 1997 16:32:06 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA11046 for mpi-core-out; Mon, 14 Jul 1997 15:35:20 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA11041 for ; Mon, 14 Jul 1997 15:35:13 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id PAA09152 for ; Mon, 14 Jul 1997 15:35:01 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id PAA19483 for ; Mon, 14 Jul 1997 15:34:59 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id UAA27732; Mon, 14 Jul 1997 20:34:58 GMT From: "Eric Salo" Message-Id: <9707141534.ZM27730@sgi.com> Date: Mon, 14 Jul 1997 15:34:58 -0500 In-Reply-To: "Ian E. Stockdale" "Re: INTEGER(KIND=MPI_ADDRESS_KIND)" (Jul 14, 11:25am) References: <199707141825.LAA06747@win62.nas.nasa.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > My reading of pp. 65-66 agrees with Marc's (and, I think, Nick's). Ditto. > The best solution to the problem Nick points out: > > >>> How exactly they make their F77 code portable now I am not sure. Use the > >>> C preprocessor I guess. > > is for users and vendors to switch to f90. Except, of course, that you can't portably use goodies like MPI_Irecv() from within f90. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Mon Jul 14 22:33:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA11267; Mon, 14 Jul 1997 22:33:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA17351 for mpi-core-out; Mon, 14 Jul 1997 21:38:27 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA17325; Mon, 14 Jul 1997 21:37:11 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id TAA19719; Mon, 14 Jul 1997 19:36:47 -0700 From: "William Saphir" Message-Id: <9707141936.ZM19718@nsgi3.lbl.gov> Date: Mon, 14 Jul 1997 19:36:45 -0700 In-Reply-To: "Marc Snir" "Re: INTEGER(KIND=MPI_ADDRESS_KIND)" (Jul 14, 10:26am) References: <852564D4.004EB8E0.00@watngi01.watson.ibm.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: "Marc Snir", nevin@caja.East.Sun.COM Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Cc: wcs@nsgi3.lbl.gov, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov, raja@tbag.rsn.hp.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Nick writes: > I believe that the beginning blurb of section 4.13 states > quite clearly that MPI_TYPE_CREATE_HVECTOR etc. take *address > sized* integer arguments for the > INTEGER(KIND=MPI_ADDRESS_KIND) arguments. It doesn't say > address sized or larger meaning that on systems with 32 bit > addresses an INTEGER*4 is required and an INTEGER*8 is > erroneous. If this is contradicted somewhere else in the > standard, for example in the terms discussion (though I saw > nothing contradictory there), then we need to fix the standard > to remove the contradiction and make it clear what is meant. Marc writes: > Such a choice clearly violates the current MPI2 standard. I quote (66:25) > "The new functions [with address sized arguments] are synonymous with the > old functions in C/C++, or on Fortran systems where default INTEGERs are > address sized." > > Thus, on a 32 bit system, MPI_TYPE_CREATE_HVECTOR is synonymous (in > Fortran) with MPI_TYPE_HVECTOR. Since the later has a 32 bit INTEGER > stride argument, so must the former. As a matter of fact, we just modified > the type decode functions so that, on 32 bit machines, they need not > distinguish MPI_TYPE_CREATE_HVECTOR from MPI_TYPE_HVECTOR (Fortran). I disagree with the interpretation of Nick and Marc. I believe the above quote is taken out of context, and in the full context the decision is left to the implementation. The misinterpretation stems from confusing when we are talking about Fortran 90 and when we are talking about Fortran 77. Let me walk through the current text, line by line. The new functions will use, in their Fortran binding, address-sized INTEGERS. Yes. Fortran means Fortran 90. INTEGER(KIND=MPI_ADDRESS_KIND) is by definition an address-sized integer. ... The new functions are synonymous with the old functions in C/C++, no problem here or on Fortran systems where default INTEGERs are address sized. Yes. This is true. Remember Fortran means Fortran 90. This sentence says that if MPI_INTEGER_KIND == MPI_ADDRESS_KIND the two functions are equivalent, with which I agree. I assert that 1. we're talking about Fortran 90 not Fortran 77 and 2. "address-sized" is defined by the implementation in the context of Fortran 90 i.e. by MPI_ADDRESS_KIND. Marc is right that if "Fortran" means "Fortran 77" there is a problem. Quoting from 13.25: When the term Fortran is used it means Fortran 90. Continuing with the text in question: In Fortran, these functions accept arguments of type INTEGER(KIND=MPI_ADDRESS_KIND), wherever arguments of type MPI_Aint are used in C. No problem here. Further confirmation that "Fortran" really means Fortran 90, as this statement makes no sense for Fortran 77. On Fortran 77 systems that do not support the Fortran 90 KIND notation, and where addresses are 64 bits whereas default INTEGERS are 32 bits, these arguments will by of type INTEGER*8. Now the text talks about F77. The text says that if addresses are 64 bits, the arguments are INTEGER*8. It also uses "will" rather than "must" leaving the possibility that this is a practical requirement. It does *not* say that if addresses are (in C) 32 bits, arguments are INTEGER. The problem is that "address-sized" is not well-defined in F77 because it is defined by MPI_ADDRESS_KIND. Therefore an implementation is free to choose. Obviously in an environment where F77 and F90 interoperate, the F90 definition of MPI_ADDRESS_KIND determines what "address-sized" means in Fortran 77. Put another way: It is certainly ok on a "32-bit" system to define MPI_ADDRESS_KIND such that the corresponding integer is 8 bytes long. Incidentally, there are two good practical reasons why an implementation might choose to use 8-byte integers when addresses are 32 bits. 1. Mixed-mode architectures (e.g. HP, SGI) where you can have 32-bit or 64-bit mode. Users should not have to port. 2. 32 bits means 32 bits. Fortran integers are signed, so the last bit is not available. To be able to do address arithmetic you must have 32 real bits. Two other important sections: 14.27: (terms section) "On systems that don't support Fortran 90-style parameterized types, INTEGER*8 or INTEGER should be used instead." This does not appear to specify which one must be used. In fact it is deliberately vague. 11.16 Some MPI procedures use {\em address} arguments that represent an absolute address in the calling program. The datatype of such an argument is MPI_Aint in C, MPI::Aint in C++, and INTEGER(KIND=MPI_ADDRESS_KIND) in Fortran. The text does not say that the datatypes have to be the same size as the machine representation of an address - just that they "represent" the absolute address. Bill From mpi-core-human@mcs.anl.gov Tue Jul 15 08:17:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA29359; Tue, 15 Jul 1997 08:17:01 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA22987 for mpi-core-out; Tue, 15 Jul 1997 07:22:34 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [198.81.209.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA22982 for ; Tue, 15 Jul 1997 07:22:28 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw2.watson.ibm.com (8.8.5/07-11-97) with ESMTP id IAA15886; Tue, 15 Jul 1997 08:21:51 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/07-14-97) with SMTP id IAA21908; Tue, 15 Jul 1997 08:22:24 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564D5.0043EF0C ; Tue, 15 Jul 1997 08:22:01 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: salo@sgi.com cc: mpi-core@mcs.anl.gov Message-ID: <852564D4.0075B4BD.00@watngi01.watson.ibm.com> Date: Mon, 14 Jul 1997 17:30:11 -0400 Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > My reading of pp. 65-66 agrees with Marc's (and, I think, Nick's). Ditto. > The best solution to the problem Nick points out: > > >>> How exactly they make their F77 code portable now I am not sure. Use the > >>> C preprocessor I guess. > > is for users and vendors to switch to f90. Except, of course, that you can't portably use goodies like MPI_Irecv() from within f90. -- Eric Salo Silicon Graphics salo@sgi.com ********* You can use portably MPI_Irecv from within F90 if you use the "F77 subset" of F90. The F90 issues are not portability issues, programs that break are likely to break in a fairly consistent manner for all implementations. I don't think that the F90 issue is such a big deal, in practice. SP users have been using a F90 compiler with MPI for years and they seem smart enough not to hit the pitfalls of MPI + F90. From mpi-core-human@mcs.anl.gov Tue Jul 15 09:58:33 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA02132; Tue, 15 Jul 1997 09:58:32 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA25690 for mpi-core-out; Tue, 15 Jul 1997 09:01:01 -0500 Received: from minnie.nic.kingston.ibm.com (minnie.nic.kingston.ibm.com [204.149.7.252]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id JAA25680 for ; Tue, 15 Jul 1997 09:00:49 -0500 Received: from windsurf.pok.ibm.com by minnie.nic.kingston.ibm.com (AIX 3.2/UCB 5.64/4.03) id AA13134; Tue, 15 Jul 1997 09:56:27 -0400 Received: by windsurf.pok.ibm.com (AIX 4.1/UCB 5.64/4.03) id AA25796; Tue, 15 Jul 1997 10:00:34 -0400 Date: Tue, 15 Jul 1997 10:00:34 -0400 From: treumann@kgn.ibm.com (Dick Treumann) Message-Id: <9707151400.AA25796@windsurf.pok.ibm.com> To: @minnie.nic.kingston.ibm.com:lederman@cs.wisc.edu, @minnie.nic.kingston.ibm.com:mpi-core@mcs.anl.gov Subject: Final draft comments Cc: treumann@windsurf.pok.ibm.com Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk comments on final draft 47:30 "function info frees" -> "function frees info" 47:30-31 Does the second sentence mean to say "The value of an info argument is interpreted each time the info is passed to a routine. Changes to an info after return from a routine do not affect that interpretation." If YES, I suggest a wording like above. If NO, then what does it mean? 51:2 comma after Fortran should be replace by word "and". (We are indicating Fortran as one catagory and C,C++ as the other. "between" is only used when there are 2 entities. If we are refering to 3 entities "Fortran, C or C++" then it would be "among". We do not mean Fortran <-> C, Fortran <-> C++, C <-> C++ but rather {Fortran} <-> {C, C++}. ) 54:24 shouldn't *MPI_Fint be MPI_Fint* ? The whole paragraph is a bit vague. Maybe replace "In order ... Fortran" with "To allow code written in C which may be called from Fortran and passed MPI_STATUS_IGNORE or MPI_STATUSES_IGNORE to detect these usages," Then, break line 25 after MPI_FINALIZE and start new sentence with "They can" rather than running on with "that can". 68:38-41 Break runon sentence. After "restrictive" change comma to period and drop "as". Begin next sentence with MPI_LB and replace the colon after "sticky" with the word "so". I would also change "painful; also it is restrictive" on line 39 to "painful and restrictive". 80:26-28 I cannot determine what is ment by the clause "it is required that the PMPI_ version be supplied and work as expected" Can someone give a clue WHAT is still expected? 126:37 The name "MPI_FENCE" still appears and should be "MPI_WIN_FENCE" 130:14 delete the word "current". No longer an evolving draft being discussed. 137:7-8 I think the picture should indicate "public copy" rather than "window copy". Text discusses "public window copy" vs. "private window copy" so "window copy" is ambiguous. "process memory" could become "private copy" but I think it SHOULD NOT be changed since "process memory" is unambiguous and may give a bit of extra clarity. 171:25-28 I do not see any need for this advise on this particular routine. Why is it more appropriate here than on any other of the 300? routines of MPI 1 & 2? 183:41 (I am embarassed at this one. Glad I cought it now.) MPI_Create_type_hindexed should be MPI_Type_create_hindexed. 183:43 Last 2 sentences work better as one now. "information. That was" -> "information and was" 206:3-4 should say "yields the same net result when fully decoded" It is not "the same result when decoded" because the constructor sequence in this case will include the MPI_TYPE_DUP call. It is the same "net" because any usage of the info from a full decode will usually ignore the MPI_TYPE_DUP step anyway. Regards - Dick Sorry I could not do the whole doc. I was unable to get a copy until yesterday. Dick Treumann IBM RS/6000 Division (Internet) treumann@kgn.ibm.com Poughkeepsie, NY (VNET) TREUMANN at KGNVMC Tel: (914) 433-7846 (internal) treumann@windsurf.pok.ibm.com Fax: (914) 433-8363 From mpi-core-human@mcs.anl.gov Tue Jul 15 10:10:51 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA02495; Tue, 15 Jul 1997 10:10:49 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA26298 for mpi-core-out; Tue, 15 Jul 1997 09:16:25 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [198.81.209.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA26293 for ; Tue, 15 Jul 1997 09:16:21 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw2.watson.ibm.com (8.8.5/07-11-97) with ESMTP id KAA19138 for ; Tue, 15 Jul 1997 10:15:43 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/07-14-97) with SMTP id KAA23177 for ; Tue, 15 Jul 1997 10:16:17 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564D5.004E6118 ; Tue, 15 Jul 1997 10:16:06 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Jean-Pierre Prost" To: mpi-core@mcs.anl.gov Message-ID: <852564D5.004E3C17.00@watngi01.watson.ibm.com> Date: Tue, 15 Jul 1997 10:15:52 -0400 Subject: Typo Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Let me add to Dick's list the following: 45/20: "the call raises" instead of "the call raisess" Jean-Pierre ########################################## Jean-Pierre Prost IBM T.J. Watson Research Center PO Box 218 Yorktown Heights, NY 10598 USA (914) 945 3225 Fax: (914) 945 2141 http://www.research.ibm.com/people/p/prost/ jpprost@watson.ibm.com ########################################## From mpi-core-human@mcs.anl.gov Tue Jul 15 11:56:10 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA04264; Tue, 15 Jul 1997 11:56:07 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA00265 for mpi-core-out; Tue, 15 Jul 1997 10:58:15 -0500 Received: from igw2.watson.ibm.com (igw2.watson.ibm.com [198.81.209.6]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA00258 for ; Tue, 15 Jul 1997 10:58:10 -0500 Received: from mailhub1.watson.ibm.com (mailhub1.watson.ibm.com [9.2.249.31]) by igw2.watson.ibm.com (8.8.5/07-11-97) with ESMTP id LAA20354; Tue, 15 Jul 1997 11:54:42 -0400 Received: from watngi01.watson.ibm.com (watngi01.watson.ibm.com [9.2.235.20]) by mailhub1.watson.ibm.com (8.8.2/07-14-97) with SMTP id LAA21754; Tue, 15 Jul 1997 11:55:16 -0400 Received: by watngi01.watson.ibm.com(Lotus SMTP MTA v1.06 (346.4 3-18-1997)) id 852564D5.005770E0 ; Tue, 15 Jul 1997 11:55:05 -0400 X-Lotus-FromDomain: IBM RESEARCH From: "Marc Snir" To: wcs@nsgi3.lbl.gov cc: nevin@caja.East.Sun.COM, mpi-core@mcs.anl.gov, raja@tbag.rsn.hp.com Message-ID: <852564D5.0051D0A3.00@watngi01.watson.ibm.com> Date: Tue, 15 Jul 1997 11:54:48 -0400 Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk To: snir@watson.ibm.com, nevin@caja.East.Sun.COM cc: wcs@nsgi3.lbl.gov, mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov, raja@tbag.rsn.hp.com (bcc: Marc Snir/Watson/IBM Research) Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Marc writes: > Such a choice clearly violates the current MPI2 standard. I quote (66: 25) > "The new functions [with address sized arguments] are synonymous with the > old functions in C/C++, or on Fortran systems where default INTEGERs are > address sized." > > Thus, on a 32 bit system, MPI_TYPE_CREATE_HVECTOR is synonymous (in > Fortran) with MPI_TYPE_HVECTOR. Since the later has a 32 bit INTEGER > stride argument, so must the former. As a matter of fact, we just modified > the type decode functions so that, on 32 bit machines, they need not > distinguish MPI_TYPE_CREATE_HVECTOR from MPI_TYPE_HVECTOR (Fortran). I disagree with the interpretation of Nick and Marc. I believe the above quote is taken out of context, and in the full context the decision is left to the implementation. The misinterpretation stems from confusing when we are talking about Fortran 90 and when we are talking about Fortran 77. ************(Marc) Maybe our disagreement is apparent, not real. If vendors want to define "address-sized" as meaning 64 bits, even on a 32 bit machine, and define BOTH MPI_ADDRESS_KIND and MPI_Aint accordingly, God bless them. This is not a debate about F77, but a debate about the meaning of "address-sized". Vendors may, consistenly, define MPI_ADDRESS_KIND do be distinct from MPI_INTEGER_KIND in F90 on 32 bit machines, if they arbitrarily decided that "address-sized" means 8 bytes. Such vendors defined an artificial and somewhat surprising notion of address-sized, but probably break nothing. Note that the constant MPI_ADDRESS_KIND and MPI_INTEGER_KIND should preferably be defined, even in F77. This, a test of MPI_ADDRESS_KIND.EQ.MPI_INTEGER_KIND can tell, even in F77 whether an INTEGER is "address_sized". On the other hand, if you imply that one can have MPI_Aint be a 32 bit int in C, but have address arguments be 64 bits in F77 Fortran, then I strongly disagreee We agreed that Fortran means Fortran 90: Since this is the current Fortran standard, MPI Fortran binding is defined to the current Fortran standard. We understand that a lot of constructs are possible in F90 that will screw up MPI, and list them. If users avoid those, they are OK. We accept that some vendors may have obsolete Fortran compilers and try to accomodate them: whenever the MPI standard mandates a declaration of the form INTEGER(KIND=MPI_ADDRESS_KIND) then these vendors may instead indicate that their MPI library takes INTEGER or INTEGER*8, as is the case. Now, clearly, this is not intended as an escape hatch so that F77 libraries will, all at once, introduce a behavior that is totally different than the behavior in F90! The clear intention is that F77 vendors will mimick, to the best of their ability, the F90 behavior, which is the MPI standard. If we follow up this logic, then the MPI2 standard does not define in any way the behavior of the MPI2 Fortran functions in F77, since Fortran means Fortran 90, and the text says nothing of the behavior of these functions in F77. My interpretation is that, unless said explicitly otherwise, the F77 behavior is the same as the F90 behavior. Thus, "the new functions have the same functionality as the old functions on Fortran systems where default INTEGERs are address sized" is the required Fortran behavior and F77 libraries should stick to it. In any case, the new Fortran functions should interoperate with the corrsponding C functions, so that, if MPI_Aint is 32 bit, then the Fortran functions sould take 32 bit address arguments. After all, we introduce MPI_ADDRESS_KIND to resolve a language interoperability problem (page 57). ************** From mpi-core-human@mcs.anl.gov Tue Jul 15 14:16:30 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA06180; Tue, 15 Jul 1997 14:16:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA04208 for mpi-core-out; Tue, 15 Jul 1997 13:20:12 -0500 Received: from win62.nas.nasa.gov (win62.nas.nasa.gov [129.99.50.13]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA04096; Tue, 15 Jul 1997 13:17:43 -0500 Received: (from ies@localhost) by win62.nas.nasa.gov (951211.SGI.8.6.12.PATCH1502/NAS.6.1) id LAA01077; Tue, 15 Jul 1997 11:17:39 -0700 Date: Tue, 15 Jul 1997 11:17:39 -0700 Message-Id: <199707151817.LAA01077@win62.nas.nasa.gov> From: "Ian E. Stockdale" To: mpi-core@mcs.anl.gov, mpi-bind@mcs.anl.gov In-reply-to: <9707141936.ZM19718@nsgi3.lbl.gov> (wcs@nsgi3.lbl.gov) Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Bill Saphir wrote: > Let me walk through the current text, line by line. MPI-2: >> The new functions will use, in their Fortran binding, address-sized >> INTEGERS. Bill: > Yes. Fortran means Fortran 90. INTEGER(KIND=MPI_ADDRESS_KIND) is by > definition an address-sized integer. I agree. Now suppose a vendor of a 32 bit system has a Fortran 77 compiler and a Fortran 90 compiler, each of which may be used with the supported MPI-2 library. Further suppose that the declaration INTEGER(KIND=MPI_ADDRESS_KIND) in the f90 compiler denotes a 32 bit word. If an addresss-sized word in the sibling f77 compiler is not also 32 bits, then we have compiler-dependent address-sized words. This may not be explicit in the document, but I thought that address-sized words in MPI-2 always had the same size. MPI-2: >> ... The new functions are synonymous with the old functions >> in C/C++, or on Fortran systems where default INTEGERs are address >> sized. Bill: > Yes. This is true. Remember Fortran means Fortran 90. This sentence > says that if MPI_INTEGER_KIND == MPI_ADDRESS_KIND the two functions > are equivalent, with which I agree. I assert that 1. we're talking > about Fortran 90 not Fortran 77 and 2. "address-sized" is defined by the > implementation in the context of Fortran 90 i.e. by MPI_ADDRESS_KIND. And if an f90 compiler gives you 32 bit address-sized words, or would if it existed, then the f77 compiler ought to do the same thing. Bill: > Marc is right that if "Fortran" means "Fortran 77" there is a > problem. Quoting from 13.25: I don't think Marc's argument depended on "Fortran" meaning anything other than Fortran 90. -Ian From mpi-core-human@mcs.anl.gov Tue Jul 15 20:14:00 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id UAA10365; Tue, 15 Jul 1997 20:14:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id TAA15211 for mpi-core-out; Tue, 15 Jul 1997 19:17:50 -0500 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id TAA15171; Tue, 15 Jul 1997 19:16:14 -0500 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id RAA22789; Tue, 15 Jul 1997 17:15:57 -0700 From: "William Saphir" Message-Id: <9707151715.ZM22788@nsgi3.lbl.gov> Date: Tue, 15 Jul 1997 17:15:56 -0700 In-Reply-To: "Ian E. Stockdale" "Re: INTEGER(KIND=MPI_ADDRESS_KIND)" (Jul 15, 11:17am) References: <199707151817.LAA01077@win62.nas.nasa.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: snir@watson.ibm.com, "Ian E. Stockdale" , mpi-bind@mcs.anl.gov, mpi-core@mcs.anl.gov Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I believe I am in agreement with the followup messages of Marc and Ian. The key elements are: 1. the address datatype must be "the same thing" in all languages. 2. implementations may choose what this thing is as long as it can represent a machine address. Let me summarize what I think this means by some examples. The following are allowed: 1. Machine address is 32 bits. MPI_Aint is a 4-byte integer. MPI_ADDRESS_KIND in F90 gives 4-byte integer. F77 users required to declare INTEGER or INTEGER*4 (if they are equivalent). 2. Machine address is 64 bits. MPI_Aint is an 8-byte integer. MPI_ADDRESS_KIND in F90 gives 8-byte integer, F77 users required to declare INTEGER*8 for the new functions. 3. Machine address is 32 bits. MPI_Aint is an 8-byte integer MPI_ADDRESS_KIND in F90 gives 8 byte integer. F77 users required to declare INTEGER*8 for the new functions. Note that for examples 2 and 3 the new functions are synonymous with the old in C/C++ but probably not in Fortran. The following are not allowed: 1. Machine address is 32 bits. MPI_Aint is a 4-byte integer. MPI_ADDRESS_KIND in F90 gives an 8-byte integer. F77 users are required to use INTEGER*8 for the new functions. [*] 2. Machine address is 32 bits. MPI_Aint is a 4-byte integer. MPI_ADDRESS_KIND in F90 gives a 4-byte integer. F77 users are required to use INTEGER*8 for the new functions. 3. Machine address is 64 bits. MPI_Aint is a 4-byte integer... [Item 1 is only disallowed if the languages interoperate. But if applications are restricted to Fortran only or C only, this is ok, but weird.] Bill From mpi-core-human@mcs.anl.gov Fri Jul 18 16:25:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA21752; Fri, 18 Jul 1997 16:25:18 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA15538 for mpi-core-out; Fri, 18 Jul 1997 15:29:17 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA15533 for ; Fri, 18 Jul 1997 15:29:12 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id PAA21961; Fri, 18 Jul 1997 15:29:06 -0500 (CDT) Date: Fri, 18 Jul 1997 15:29:06 -0500 (CDT) Message-Id: <199707182029.PAA21961@rap.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-core@mcs.anl.gov Subject: MPI-2 available!!!! Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The announcement of the official release of MPI-2 was sent to comp.parallel.mpi and comp.parallel. I would like to thank everyone for their tireless work in making this happen. We can all be proud of the new versions of MPI. From mpi-core-human@mcs.anl.gov Mon Jul 21 11:47:45 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA13322; Mon, 21 Jul 1997 11:47:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA04956 for mpi-core-out; Mon, 21 Jul 1997 10:52:20 -0500 Received: from rap.cs.wisc.edu (rap.cs.wisc.edu [128.105.66.35]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA04947 for ; Mon, 21 Jul 1997 10:52:12 -0500 Received: (from lederman@localhost) by rap.cs.wisc.edu (8.7.6/8.7.3) id KAA29248; Mon, 21 Jul 1997 10:52:05 -0500 (CDT) Date: Mon, 21 Jul 1997 10:52:05 -0500 (CDT) Message-Id: <199707211552.KAA29248@rap.cs.wisc.edu> From: Steve Huss-Lederman To: nevin@caja.East.Sun.COM CC: mpi-core@mcs.anl.gov In-reply-to: (message from Nicholas Nevin -SMCC - High Performance Computing on Mon, 21 Jul 1997 11:48:09 -0400 (EDT)) Subject: Re: MPI-2 available!!!! Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Nick, > > The announcement of the official release of MPI-2 was sent to > > comp.parallel.mpi and comp.parallel. I would like to thank everyone > > for their tireless work in making this happen. We can all be proud of > > the new versions of MPI. > > Steve I still haven't seen the announcement appear on comp.parallel.mpi. > > Has someone else seen it? Where can one get the document? I also noticed it has not yet come out. I am hoping it is due to moderator delays. I will keep and eye out. Anyway, all info can be got from http://www.mpi-forum.org. Steve From mpi-core-human@mcs.anl.gov Sun Jul 27 14:26:54 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA07940; Sun, 27 Jul 1997 14:26:53 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA08571 for mpi-core-out; Sun, 27 Jul 1997 13:31:39 -0500 Received: from ERC.MsState.Edu (root@Sniper.ERC.MsState.Edu [130.18.14.12]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA20568 for ; Thu, 10 Jul 1997 11:03:08 -0500 Received: from python.ERC.MsState.Edu (puri@python [192.208.142.42]); by ERC.MsState.Edu (8.8.5/8.8.5/v1h) with SMTP; id LAA15764; Thu, 10 Jul 1997 11:02:53 -0500 (CDT) From: Purushotham Bangalore Received: by python.ERC.MsState.Edu (8.6.8.1/6.0c-FWP); id LAA17448; Thu, 10 Jul 1997 11:02:53 -0500 Date: Thu, 10 Jul 1997 11:02:53 -0500 Message-Id: <199707101602.LAA17448@python.ERC.MsState.Edu> To: mpi-core@mcs.anl.gov Subject: New C bindings Cc: lush@erc.msstate.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-MD5: HtNNvFoSHAznkfNRtIEIqg== Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Sorry to bring this issue so late. I looked at the current MPI-2 draft and realized that there serious performance problem in the implementation of the MPI C++ bindings. The C++ bindings enforce a good programming practice by making all the IN arguments "const". Where as the C bindings were left unchanged and the reason for this being that such a change will break the existing implementations as well as user code (which i don't completely agree). Without making the corresponding changes to the C bindings it will be impossible to develop a high-performance implementation of C++ bindings on top of the existing C bindings. The only way to implement the current C++ bindings will involve making extra unnecessary copies to remove the constness. So i strongly feel that a revised C bindings are necessary and it will not break either existing implementations or user code if a corrected prototype file (in arguments are changed to const where appropriate) is provided for the end users. This is what i had to do inorder to get a high-performance C++ binding implementation. If the MPI forum doesnot provide a revised C bindings i believe that the implementors will be forced to make their own changes to the C bindings. Also the inclusion of const will be in line with the changes adopted in the POSIX standard for standard I/O calls (for example fopen). Puri -------------------------------------------------------------------------- Purushotham Bangalore Office Phone: 601-325-4587; Fax: 601-325-7692 PO Box 2298 Home Phone: 601-324-2626 Mississippi State, MS 39762 URL: http://www.erc.msstate.edu/~puri From mpi-core-human@mcs.anl.gov Sun Jul 27 15:47:23 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA08738; Sun, 27 Jul 1997 15:47:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA09468 for mpi-core-out; Sun, 27 Jul 1997 14:52:45 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.25.119]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA09463 for ; Sun, 27 Jul 1997 14:52:42 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.5/8.8.5) with ESMTP id OAA00577; Sun, 27 Jul 1997 14:52:37 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.5/8.8.5) with SMTP id OAA00563; Sun, 27 Jul 1997 14:52:38 -0500 (EST) Date: Sun, 27 Jul 1997 14:52:38 -0500 (EST) From: Jeff Squyres Reply-To: Jeff Squyres To: Purushotham Bangalore cc: mpi-core@mcs.anl.gov, lush@erc.msstate.edu Subject: Re: New C bindings In-Reply-To: <199707101602.LAA17448@python.ERC.MsState.Edu> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Thu, 10 Jul 1997, Purushotham Bangalore wrote: > Sorry to bring this issue so late. I looked at the current MPI-2 draft and > realized that there serious performance problem in the implementation of > the MPI C++ bindings. I think that this is a delayed message from Puri -- I seem to recall seeing this message a long time ago. We had a discussion about it a few weeks ago. There is no performance problem with the C++ bindings. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Tue Jul 29 13:10:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA18704; Tue, 29 Jul 1997 13:10:36 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA27273 for mpi-core-out; Tue, 29 Jul 1997 12:14:36 -0500 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA23970; Mon, 14 Jul 1997 08:57:19 -0500 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (SMI-8.6/mail.byaddr) with SMTP id HAA13275; Mon, 14 Jul 1997 07:24:19 -0700 Received: from suneast.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id JAA10800; Mon, 14 Jul 1997 09:56:33 -0400 Received: from caja.East.Sun.COM by suneast.East.Sun.COM (SMI-8.6/SMI-SVR4) id JAA11374; Mon, 14 Jul 1997 09:56:33 -0400 Received: from caja by caja.East.Sun.COM (SMI-8.6/SMI-SVR4) id JAA24755; Mon, 14 Jul 1997 09:56:28 -0400 Date: Mon, 14 Jul 1997 09:56:28 -0400 (EDT) From: Nicholas Nevin -SMCC - High Performance Computing Reply-To: Nicholas Nevin -SMCC - High Performance Computing Subject: Re: INTEGER(KIND=MPI_ADDRESS_KIND) To: raja@tbag.rsn.hp.com, mpi-core@mcs.anl.gov, wcs@nsgi3.lbl.gov Cc: mpi-bind@mcs.anl.gov Message-ID: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: U8xCpzZmmdXxWSz2U/yGaw== X-Mailer: dtmail 1.1.0 CDE Version 1.1 SunOS 5.5.1 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > > > In your opinion, how big a violation of the std would it be if some > > vendors decide to tell their customers to always use integer*8 as > > "the appropriate thing" on their systems, for both 32 and 64-bit > > versions of the library? Reason: starting with our next release (1.3) > > we will be supporting both 32/64-bit because HP-UX will have both > > modes. It would be nice to have users not modify their source code no > > matter how they choose to compile their apps. Same issue would be faced > > by SGI. So if a vendor is willing to handle the "if (sizeof(void *) < 8)" > > scenario in some wrappers and at least guarantee source code portability > > between its own versions of the library, is that a big boo-boo? > > My opinion is that an implementation that requires integer*8 for the > new functions is within the law and spirit of the standard, though > this is apparently not required by the standard. Presumably all > vendors/implementors will want to do this except those who have no > 64-bit capability at all (and even there it might make sense). > Hopefully users will move to F90 and when they use the new functions > they will also use the F90 way of doing things. > > Bill I believe that the beginning blurb of section 4.13 states quite clearly that MPI_TYPE_CREATE_HVECTOR etc. take *address sized* integer arguments for the INTEGER(KIND=MPI_ADDRESS_KIND) arguments. It doesn't say address sized or larger meaning that on systems with 32 bit addresses an INTEGER*4 is required and an INTEGER*8 is erroneous. If this is contradicted somewhere else in the standard, for example in the terms discussion (though I saw nothing contradictory there), then we need to fix the standard to remove the contradiction and make it clear what is meant. If we really intended, as Bill mentioned earlier, that INTEGER(KIND=MPI_ADDRESS_KIND) should mean INTEGER*8 unless it is not supported in which case it is system dependent, then we should amend the standard to say precisely this. -nick. -- Nick Nevin Sun Microsystems, HPC Engineering (508) 442-0219 nicholas.nevin@sun.com From mpi-core-human@mcs.anl.gov Tue Aug 12 09:31:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA10373; Tue, 12 Aug 1997 09:31:52 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA02832 for mpi-core-out; Tue, 12 Aug 1997 08:36:49 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA02826 for ; Tue, 12 Aug 1997 08:36:44 -0500 Message-Id: <199708121336.IAA02826@antares.mcs.anl.gov> To: mpi-core@antares.mcs.anl.gov Subject: Clarification for 1.3/2.1 Date: Tue, 12 Aug 1997 08:36:41 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk An implementor of heterogeneous systems raised the following issue: | I have a question about the use of MPI_Pack_size(). In the standard, | MPI_Pack_size() is defined as: | | "A call to MPI_PACK_SIZE(incount, datatype, comm, size) returns in size | an upper bound on the increment in position that is effected by a call | to MPI_PACK(inbuf, incount, datatype, outbuf, outcount, position, comm)." | | This seems to indicate to me that the call was designed to be used to | determine how big of a buffer to allocate before calling | MPI_Send()/MPI_Pack(). | | The above definition, however, doesn't say anything about the unpack side. | So it is not clear to me from the standard that it is legal to call | MPI_Pack_size() to determine how big of a buffer to allocate for | MPI_Recv()/MPI_Unpack(). It is very possible that MPI_Pack() and | MPI_Unpack() will increment the position parameter differently. My answer was that the intent was | "A call to MPI_PACK_SIZE(incount, datatype, comm, size) returns in size | an upper bound on the increment in position that is effected by a call | to MPI_PACK(inbuf, incount, datatype, outbuf, outcount, position, comm)" and MPI_UNPACK(...). That is, MPI_PACK_SIZE provides a value that is at least as large as the increment in position effected by either MPI_PACK or MPI_UNPACK. But the standard doesn't say this. Any comments? Bill From mpi-core-human@mcs.anl.gov Tue Aug 12 16:03:19 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA16281; Tue, 12 Aug 1997 16:03:19 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA16080 for mpi-core-out; Tue, 12 Aug 1997 15:07:34 -0500 Received: from lms03.us1.ibm.com (lms03.ny.us.ibm.com [198.133.22.39]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAB14874 for ; Tue, 12 Aug 1997 14:31:58 -0500 Received: from d01lms04.pok.ibm.com by lms03.us1.ibm.com (AIX 4.1/UCB 5.64/4.03) id AA22296; Tue, 12 Aug 1997 18:24:49 GMT Received: by US.IBM.COM (Soft-Switch LMS 2.0) with snapi via D01AU002 id 5010400008231487; Tue, 12 Aug 1997 15:34:57 -0400 From: Marc Snir To: Cc: Subject: Re: Clarification for 1.3/2.1 Message-Id: <5010400008231487000002L072*@MHS> Date: Tue, 12 Aug 1997 15:34:57 -0400 Mime-Version: 1.0 Content-Type: text/plain Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with your interpretation. There may be another problem, however: Is it the case that if one sends n elements of type MPI_PACKED, then the other side receives n elements of type MPI_PACKED? If this is the assumption, then it must be the case that, in a heterogeneous environment, data conversion will be done at packing time, so that any resizing needed because of conversions is done when the data is packed. Is this what we want(ed)? Marc Snir, Senior Manager, Scalable Parallel Systems Tel: 914 945 3204 (8 862 3204) Fax: 914 945 4425 (8 862 4425) E-mail: snir@watson.ibm.com From mpi-core-human@mcs.anl.gov Tue Aug 12 17:03:15 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA16925; Tue, 12 Aug 1997 17:03:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA18457 for mpi-core-out; Tue, 12 Aug 1997 16:09:12 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA18449 for ; Tue, 12 Aug 1997 16:09:07 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id QAA01614 for ; Tue, 12 Aug 1997 16:08:58 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id QAA19976 for ; Tue, 12 Aug 1997 16:08:56 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id VAA01669; Tue, 12 Aug 1997 21:08:56 GMT From: "Eric Salo" Message-Id: <9708121608.ZM1667@sgi.com> Date: Tue, 12 Aug 1997 16:08:55 -0500 In-Reply-To: Marc Snir "Re: Clarification for 1.3/2.1" (Aug 12, 3:34pm) References: <5010400008231487000002L072*@MHS> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: Clarification for 1.3/2.1 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I think we need a MPI_UNPACK_SIZE() for symmetry. (Only half of a smiley here.) -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Wed Aug 13 09:25:08 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA25226; Wed, 13 Aug 1997 09:25:08 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA01481 for mpi-core-out; Wed, 13 Aug 1997 08:27:30 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA01471; Wed, 13 Aug 1997 08:27:22 -0500 Message-Id: <199708131327.IAA01471@antares.mcs.anl.gov> To: Marc Snir cc: owner-mpi-core@mcs.anl.gov, mpi-core@antares.mcs.anl.gov Subject: Re: Clarification for 1.3/2.1 In-Reply-to: Message from Marc Snir of "Tue, 12 Aug 1997 15:34:57 -0400." <5010400008231487000002L072*@MHS> Date: Wed, 13 Aug 1997 08:27:15 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | I agree with your interpretation. There may be another problem, however: Is | it the case that if one sends n elements of type MPI_PACKED, then the other | side receives n elements of type MPI_PACKED? If this is the assumption, then | it must be the case that, in a heterogeneous environment, data conversion will | be done at packing time, so that any resizing needed because of conversions is | done when the data is packed. Is this what we want(ed)? | We definitely don't want to enforce data conversion on the sender's side. But I don't see the problem. If the internal representation of MPI_PACKED is just byte encodings, then both sender and receiver see the same sized message. The only way in which different sizes would appear is if some conversion happened with the packed data before being delivered to the user (for example, an optimization that allowed MPI_UNPACK to simply do memcpy with no further conversion). For example, using XDR encoding sends and delivers the same amount of data of type MPI_PACKED; only in the MPI_PACK and MPI_UNPACK do different data lengths on the systems come into play. Bill From mpi-core-human@mcs.anl.gov Fri Aug 22 13:12:56 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA17504; Fri, 22 Aug 1997 13:12:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA08007 for mpi-core-out; Fri, 22 Aug 1997 12:15:08 -0500 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA08002 for ; Fri, 22 Aug 1997 12:15:02 -0500 Date: Fri, 22 Aug 1997 12:14:58 -0500 Message-Id: <199708221714.MAA14601@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov Subject: Fortran question Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk MPI_File_get_view and MPI_Info_get_nthkey both return strings, but don't have a resultlen argument (as MPI_Error_string has, MPI 1.1, pg. 197). I thought the resultlen was needed for Fortran because Fortran strings are not necessarily null-terminated, so there is no knowing what is the actual length of the string returned. Rajeev From mpi-core-human@mcs.anl.gov Fri Aug 22 13:55:28 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA17912; Fri, 22 Aug 1997 13:55:25 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA09865 for mpi-core-out; Fri, 22 Aug 1997 13:01:24 -0500 Received: from mailhost.lanl.gov (mailhost.lanl.gov [128.165.3.12]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA09860; Fri, 22 Aug 1997 13:01:20 -0500 Received: from [128.165.115.37] (ttm-mac.lanl.gov [128.165.115.37]) by mailhost.lanl.gov (8.8.7/8.8.7) with ESMTP id MAA12278; Fri, 22 Aug 1997 12:01:16 -0600 (MDT) X-Sender: u095182@cic-mail.lanl.gov Message-Id: In-Reply-To: <199708221714.MAA14601@abacus> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Fri, 22 Aug 1997 12:01:38 -0700 To: Rajeev Thakur , mpi-core@mcs.anl.gov From: Tyce McLarty Subject: Re: Fortran question Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 10:14 AM -0700 8/22/97, Rajeev Thakur wrote: >MPI_File_get_view and MPI_Info_get_nthkey both return strings, but >don't have a resultlen argument (as MPI_Error_string has, MPI 1.1, >pg. 197). I thought the resultlen was needed for Fortran because Fortran >strings are not necessarily null-terminated, so there is no knowing >what is the actual length of the string returned. > >Rajeev The f90 book says that character variables MUST have a declared maximum length. That means the maximum length has to be declared in the calling routine. Also when a string less than the length is assigned to a character variable, the string is blank filled. The function LEN(STRING) returns the declared length of a string for use by a called routine using a CHARACTER*(*) declaration, so a fortran wrapper program could find the maximum length. You should not have to worry about this. The function LEN_TRIM(STRING) returns the "used" length of a string not counting trailing blanks. So if a string is blank filled, a calling program can find the usable length. This takes care of the user. It sound like to be f90 friendly you should probably blank fill any returned strings. I don't think having a null before the blanks hurts anything, so you could have the string, a null, and blanks. I think if you initialize the string in the fortran wrapper e.g. string = '' that f90 will blank-fill it for you. Then your c code can replace the first part as needed and the trailing blanks will remain. Hope this helps, Tyce From mpi-core-human@mcs.anl.gov Tue Sep 16 13:11:04 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA00570; Tue, 16 Sep 1997 13:11:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA14744 for mpi-core-out; Tue, 16 Sep 1997 12:13:54 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA14738 for ; Tue, 16 Sep 1997 12:13:38 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id MAA16817 for ; Tue, 16 Sep 1997 12:13:26 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id MAA06672 for ; Tue, 16 Sep 1997 12:13:24 -0500 (CDT) Received: by mrjones.cray.com (8.8.0/btd-b3) id RAA03064; Tue, 16 Sep 1997 17:13:14 GMT From: "Eric Salo" Message-Id: <9709161213.ZM3062@sgi.com> Date: Tue, 16 Sep 1997 12:13:13 -0500 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: MPI-1.2 question Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On page 34, lines 18-22 of the April 17, 1997 draft (the one we used for the final MPI Forum meeting) we had the following text: * Page 202, line 48 reads int MPI_Pcontrol(const int level, ...) but should read int MPI_Pcontrol(int level, ...) Discussion: This is the only function with const and the ramifications are not liked. I distinctly recall that this change was voted in as part of MPI-1.2. However, somewhere between the final meeting and the release of the MPI-1.2 draft, this text was yanked. My question is, can anyone tell me when and why that happened? I don't recall any discussion but I haven't checked the mailing list archives yet. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Fri Sep 19 15:11:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA25169; Fri, 19 Sep 1997 15:11:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA09311 for mpi-core-out; Fri, 19 Sep 1997 14:15:10 -0500 Received: from hdl.ie (galois.hdl.ie [192.122.222.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA09140 for ; Fri, 19 Sep 1997 14:12:19 -0500 Received: from hdl.ie (ur.hdl.ie [192.122.222.40]) by hdl.ie (8.7.3/8.7.3) with ESMTP id UAA09611 for ; Fri, 19 Sep 1997 20:12:01 +0100 (BST) Message-ID: <3422CD70.A628DD16@hdl.ie> Date: Fri, 19 Sep 1997 20:07:29 +0100 From: Mark Fallon Organization: Hitachi Dublin Laboratory X-Mailer: Mozilla 4.03 [en] (X11; I; AIX 4.1) MIME-Version: 1.0 To: mpi-core@mcs.anl.gov Subject: Possible Typo. Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi, In the C++ in the appendix I can not seem to find the following two commands: void MPI::Status::Set_cancelled(bool flag) void MPI::Status::Set_elements(const MPI::Datatype& datatype int count) This is not currently listed in the errata. Regards. Mark. P.S. How about putting the archives somewhere where they are readable by everyone? lovelace.nas.nasa.gov is one of those machines only available in the States. ___________________________________________________________________ Mark Fallon Tel : +353-1-6798911 Researcher Fax : +353-1-6798926 Hitachi Dublin Laboratory Email : mark.fallon@hdl.ie ___________________________________________________________________ From mpi-core-human@mcs.anl.gov Fri Sep 19 15:14:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA25200; Fri, 19 Sep 1997 15:14:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA09739 for mpi-core-out; Fri, 19 Sep 1997 14:20:45 -0500 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA09733 for ; Fri, 19 Sep 1997 14:20:38 -0500 Date: Fri, 19 Sep 1997 14:20:34 -0500 Message-Id: <199709191920.OAA27144@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov In-reply-to: <3422CD70.A628DD16@hdl.ie> (message from Mark Fallon on Fri, 19 Sep 1997 20:07:29 +0100) Subject: Re: Possible Typo. Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > How about putting the archives somewhere where they are > readable by everyone? lovelace.nas.nasa.gov is one of those > machines only available in the States. It is not accessible even in the States. From mpi-core-human@mcs.anl.gov Fri Sep 19 16:14:27 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA26573; Fri, 19 Sep 1997 16:14:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA13282 for mpi-core-out; Fri, 19 Sep 1997 15:20:17 -0500 Received: from breeze.cs.wisc.edu (breeze.cs.wisc.edu [128.105.75.36]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA13265 for ; Fri, 19 Sep 1997 15:20:00 -0500 Received: (from lederman@localhost) by breeze.cs.wisc.edu (8.8.6/8.8.6) id PAA19996; Fri, 19 Sep 1997 15:19:41 -0500 (CDT) Date: Fri, 19 Sep 1997 15:19:41 -0500 (CDT) Message-Id: <199709192019.PAA19996@breeze.cs.wisc.edu> From: Steve Huss-Lederman To: mark.fallon@hdl.ie CC: mpi-core@mcs.anl.gov In-reply-to: <3422CD70.A628DD16@hdl.ie> (message from Mark Fallon on Fri, 19 Sep 1997 20:07:29 +0100) Subject: Re: Possible Typo. Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > In the C++ in the appendix I can not seem to find the following > two commands: > > void MPI::Status::Set_cancelled(bool flag) > void MPI::Status::Set_elements(const MPI::Datatype& datatype > int count) I am confused here. In Annex A it is listed on 337:19 and 337:20. If the remark is in reference to Annex B, which is MPI-1 functions, then I don't think it should be there since it is an MPI-2 function. Did I miss something? > How about putting the archives somewhere where they are > readable by everyone? lovelace.nas.nasa.gov is one of those > machines only available in the States. I assume this is refering to the mail archives. This is an item that has been brought up before but has yet to happen. I'll see what I can do. Steve From mpi-core-human@mcs.anl.gov Sat Sep 20 12:29:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA03167; Sat, 20 Sep 1997 12:29:51 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA00683 for mpi-core-out; Sat, 20 Sep 1997 11:34:05 -0500 Received: from hdl.ie (galois.hdl.ie [192.122.222.37]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA00677 for ; Sat, 20 Sep 1997 11:33:59 -0500 Received: from hdl.ie (ur.hdl.ie [192.122.222.40]) by hdl.ie (8.7.3/8.7.3) with ESMTP id RAA10562; Sat, 20 Sep 1997 17:33:08 +0100 (BST) Message-ID: <3423F9B5.8D0779CE@hdl.ie> Date: Sat, 20 Sep 1997 17:28:37 +0100 From: Mark Fallon Organization: Hitachi Dublin Laboratory X-Mailer: Mozilla 4.03 [en] (X11; I; AIX 4.1) MIME-Version: 1.0 To: Steve Huss-Lederman CC: mpi-core@mcs.anl.gov Subject: Re: Possible Typo. References: <199709192019.PAA19996@breeze.cs.wisc.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve Huss-Lederman wrote: > > > In the C++ in the appendix I can not seem to find the following > > two commands: > > > > void MPI::Status::Set_cancelled(bool flag) > > void MPI::Status::Set_elements(const MPI::Datatype& datatype > > int count) > > I am confused here. In Annex A it is listed on 337:19 and 337:20. If > the remark is in reference to Annex B, which is MPI-1 functions, then I > don't think it should be there since it is an MPI-2 function. Did I > miss something? No, I think I did or we missed some discussion point. When I saw the commands to set various elements of the status I was assuming they were part of MPI-2 (maybe for Generalised Requests), so I did not look at the title on the top of the page. So maybe the question should be rephrased where did MPI::Status::set_tag, set_source and set_error come from? Mark. ___________________________________________________________________ Mark Fallon Tel : +353-1-6798911 Researcher Fax : +353-1-6798926 Hitachi Dublin Laboratory Email : mark.fallon@hdl.ie ___________________________________________________________________ From mpi-core-human@mcs.anl.gov Sun Sep 21 19:06:12 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA13450; Sun, 21 Sep 1997 19:06:11 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA02961 for mpi-core-out; Sun, 21 Sep 1997 18:09:44 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.22.171]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA02955 for ; Sun, 21 Sep 1997 18:09:38 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.7/8.8.7) with ESMTP id SAA05006 for ; Sun, 21 Sep 1997 18:09:35 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.7/8.8.7) with SMTP id SAA21082 for ; Sun, 21 Sep 1997 18:09:34 -0500 (EST) Date: Sun, 21 Sep 1997 18:09:34 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov Subject: Re: Possible Typo. In-Reply-To: <3423F9B5.8D0779CE@hdl.ie> Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Sat, 20 Sep 1997, Mark Fallon wrote: > No, I think I did or we missed some discussion point. When I saw the > commands to set various elements of the status I was assuming they were > part of MPI-2 (maybe for Generalised Requests), so I did not look at the > title on the top of the page. So maybe the question should be rephrased > where did MPI::Status::set_tag, set_source and set_error come from? They are the counterparts to the fact that in C, the tag, source, and error are public data fields in MPI_Status. In C++, public data members are highly discouraged, so this was the next best thing. That is, a user/library could change the values that come back from MPI (if they wanted to, for some reason), just as they can in C and Fortran. Additionally, depending on how it is implemented, it may ease layered C++ implementations, particularly in the generalized request areana. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Wed Oct 1 17:48:22 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA14604; Wed, 1 Oct 1997 17:48:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA20122 for mpi-core-out; Wed, 1 Oct 1997 16:50:22 -0500 Received: from palrel3.hp.com (palrel3.hp.com [156.153.255.219]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA20111 for ; Wed, 1 Oct 1997 16:50:07 -0500 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel3.hp.com (8.8.5/8.8.5tis) with ESMTP id OAA00859 for ; Wed, 1 Oct 1997 14:49:57 -0700 (PDT) Message-Id: <199710012149.OAA00859@palrel3.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA084902633; Wed, 1 Oct 1997 16:50:33 -0500 From: Raja Daoud Subject: Re: MPI-1.2 question To: mpi-core@mcs.anl.gov Date: Wed, 01 Oct 1997 16:50:33 CDT In-Reply-To: <9709161213.ZM3062@sgi.com>; from "Eric Salo" at Sep 16, 97 12:13 (noon) X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > * Page 202, line 48 reads > int MPI_Pcontrol(const int level, ...) > but should read > int MPI_Pcontrol(int level, ...) > > Discussion: > This is the only function with const and the ramifications > are not liked. > >I distinctly recall that this change was voted in as part of MPI-1.2. However, >somewhere between the final meeting and the release of the MPI-1.2 draft, this >text was yanked. My question is, can anyone tell me when and why that happened? >I don't recall any discussion but I haven't checked the mailing list archives >yet. Good point! My recollection was that the "const" was voted out of MPI_Pcontrol(). I think it should be included in the MPI-2 errata. --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/go/mpi From mpi-core-human@mcs.anl.gov Wed Oct 1 18:19:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA14983; Wed, 1 Oct 1997 18:19:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA21879 for mpi-core-out; Wed, 1 Oct 1997 17:25:05 -0500 Received: from palrel3.hp.com (palrel3.hp.com [156.153.255.219]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA21874 for ; Wed, 1 Oct 1997 17:24:58 -0500 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel3.hp.com (8.8.5/8.8.5tis) with ESMTP id PAA06254 for ; Wed, 1 Oct 1997 15:24:54 -0700 (PDT) Message-Id: <199710012224.PAA06254@palrel3.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA085594729; Wed, 1 Oct 1997 17:25:29 -0500 From: Raja Daoud Subject: status for completed send To: mpi-core@mcs.anl.gov Date: Wed, 01 Oct 1997 17:25:29 CDT X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Does a wait on a send request have to touch the status object at all? It has to if a send was cancelled, in order to provide the necessary info to MPI_Test_cancelled(). But does it have to in the sweet-spot of no calls to MPI_Cancel()? Another way to say it: is a user allowed to call MPI_Test_cancelled() if MPI_Cancel() wasn't called on the request associated with the status object? Example: MPI_Isend(..., &req); MPI_Wait(&req, &status); MPI_Test_cancelled(&status, &flag); /* is this valid? */ If it's valid, wait on a send must at least touch one field in the status (potential extra cache miss). Comments? --Raja -=- Raja Daoud Hewlett-Packard Co., Convex Division raja@rsn.hp.com http://www.hp.com/go/mpi From mpi-core-human@mcs.anl.gov Wed Oct 1 19:03:36 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA15547; Wed, 1 Oct 1997 19:03:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA23927 for mpi-core-out; Wed, 1 Oct 1997 18:09:42 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA23915 for ; Wed, 1 Oct 1997 18:09:29 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id SAA08319 for ; Wed, 1 Oct 1997 18:08:17 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id SAA12465 for ; Wed, 1 Oct 1997 18:08:15 -0500 (CDT) Received: by mrjones.cray.com (8.8.5/btd-b3) id SAA13747; Wed, 1 Oct 1997 18:08:12 -0500 (CDT) From: "Eric Salo" Message-Id: <9710011808.ZM13745@sgi.com> Date: Wed, 1 Oct 1997 18:08:11 -0500 In-Reply-To: Raja Daoud "status for completed send" (Oct 1, 5:25pm) References: <199710012224.PAA06254@palrel3.hp.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: status for completed send Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Does a wait on a send request have to touch the status object at all? > It has to if a send was cancelled, in order to provide the necessary > info to MPI_Test_cancelled(). But does it have to in the sweet-spot > of no calls to MPI_Cancel()? Another way to say it: is a user allowed > to call MPI_Test_cancelled() if MPI_Cancel() wasn't called on the > request associated with the status object? Example: > > MPI_Isend(..., &req); > MPI_Wait(&req, &status); > MPI_Test_cancelled(&status, &flag); /* is this valid? */ > > If it's valid, wait on a send must at least touch one field in the > status (potential extra cache miss). Comments? I think that this is a *beautiful* interpretation! Unfortunately, it isn't particularly intuitive, so my guess is that you/we would have a hard time getting it approved. I think that the only way to avoid the extra cache miss is probably just to teach applications programmers to pass a NULL status pointer to MPI_Wait(). -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Thu Oct 2 09:05:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA23036; Thu, 2 Oct 1997 09:05:10 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA25411 for mpi-core-out; Thu, 2 Oct 1997 08:10:37 -0500 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA25367 for ; Thu, 2 Oct 1997 08:10:06 -0500 Received: from jim (th-eng09-176.pool.dircon.co.uk [194.112.56.176]) by mailhost.dircon.co.uk (8.8.5/8.8.7) with SMTP id OAA22392; Thu, 2 Oct 1997 14:09:41 +0100 (BST) Message-Id: <199710021309.OAA22392@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id KAA00628; Thu, 2 Oct 1997 10:08:24 +0100 To: Raja Daoud cc: mpi-core@mcs.anl.gov Subject: Re: status for completed send In-reply-to: Your message of "Wed, 01 Oct 1997 17:25:29 CDT." <199710012224.PAA06254@palrel3.hp.com> Date: Thu, 02 Oct 1997 10:08:24 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Raja, > Does a wait on a send request have to touch the status object at all? > It has to if a send was cancelled, in order to provide the necessary > info to MPI_Test_cancelled(). But does it have to in the sweet-spot > of no calls to MPI_Cancel()? Another way to say it: is a user allowed > to call MPI_Test_cancelled() if MPI_Cancel() wasn't called on the > request associated with the status object? Example: > > MPI_Isend(..., &req); > MPI_Wait(&req, &status); > MPI_Test_cancelled(&status, &flag); /* is this valid? */ > > If it's valid, wait on a send must at least touch one field in the > status (potential extra cache miss). Comments? I believe that the above code is valid. Of course you could do some tricky implementation something like this... boolean cancelled_ever_called = false; MPI_Cancel (...) { cancelled_ever_called = true; ... and so on ... } MPI_Wait (...) { ... if (it's a send && cancelled_ever_called) set the status to reflect whether this was cancelled ... } MPI_Test_cancelled (...) { if (!cancelled_ever_called) result is false; else get the result from the status. } Of course, this introduces another test in the MPI_Wait, *but* if wait is happening a lot, then cancelled_ever_called will be in the cache, so you do avoid the cache miss... *Not* elegant, but maybe worth a try, it might speed up codes which don't call MPI_Cancel at all (i.e. probably most codes :-). -- Jim James Cownie Dolphin Interconnect Solutions +44 117 9071438 http://www.dolphinics.com/toolworks From mpi-core-human@mcs.anl.gov Thu Oct 2 09:43:48 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA23418; Thu, 2 Oct 1997 09:43:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA27156 for mpi-core-out; Thu, 2 Oct 1997 08:50:09 -0500 Received: from mcs.anl.gov (godzilla.mcs.anl.gov [140.221.5.136]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA27150; Thu, 2 Oct 1997 08:50:03 -0500 Message-Id: <199710021350.IAA27150@antares.mcs.anl.gov> To: Raja Daoud cc: mpi-core@mcs.anl.gov Subject: Re: MPI-1.2 question In-Reply-to: Message from Raja Daoud of "Wed, 01 Oct 1997 16:50:33 -0500." <199710012149.OAA00859@palrel3.hp.com> Date: Thu, 02 Oct 1997 08:50:01 -0500 From: William Gropp Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk | > * Page 202, line 48 reads | > int MPI_Pcontrol(const int level, ...) | > but should read | > int MPI_Pcontrol(int level, ...) | > | > Discussion: | > This is the only function with const and the ramifications | > are not liked. | > | >I distinctly recall that this change was voted in as part of MPI-1.2. However, | >somewhere between the final meeting and the release of the MPI-1.2 draft, this | >text was yanked. My question is, can anyone tell me when and why that happened? | >I don't recall any discussion but I haven't checked the mailing list archives | >yet. The problem with changing it is this: MPI_Pcontrol is the ONE MPI routine that users are EXPECTED to implement as part of a profiling interface. Changing the prototype makes EXISTING MPI applications break (the compiler refuses to accept the difference between the old prototype and new prototype). This violates the MPI Forum "rule" of not breaking things that aren't wrong. Yes, it would have been nice if MPI 1 had been consistent in the use of const (everywhere or nowhere). But making this change will break existing programs for no good reason. It is true that users could use the MPI_VERSION macros to fix code that used to work, but this seems like a bad idea. Bill From mpi-core-human@mcs.anl.gov Thu Oct 2 09:50:26 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA23468; Thu, 2 Oct 1997 09:50:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA27494 for mpi-core-out; Thu, 2 Oct 1997 08:56:49 -0500 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA27489 for ; Thu, 2 Oct 1997 08:56:44 -0500 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (SMI-8.6/mail.byaddr) with SMTP id GAA18492 for ; Thu, 2 Oct 1997 06:56:48 -0700 Received: from suneast.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id JAA29147; Thu, 2 Oct 1997 09:56:09 -0400 Received: from caja.East.Sun.COM by suneast.East.Sun.COM (SMI-8.6/SMI-SVR4) id JAA04077; Thu, 2 Oct 1997 09:56:09 -0400 Received: from caja by caja.East.Sun.COM (SMI-8.6/SMI-SVR4) id JAA06230; Thu, 2 Oct 1997 09:56:05 -0400 Date: Thu, 2 Oct 1997 09:56:05 -0400 (EDT) From: Nicholas Nevin -SMCC - High Performance Computing Reply-To: Nicholas Nevin -SMCC - High Performance Computing Subject: Re: MPI-1.2 question To: mpi-core@mcs.anl.gov Message-ID: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: XNSqsqbAnP9dUZ/oQTDb9A== X-Mailer: dtmail 1.1.0 CDE Version 1.1 SunOS 5.5.1 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > > > * Page 202, line 48 reads > > int MPI_Pcontrol(const int level, ...) > > but should read > > int MPI_Pcontrol(int level, ...) > > > > Discussion: > > This is the only function with const and the ramifications > > are not liked. > > > >I distinctly recall that this change was voted in as part of MPI-1.2. However, > >somewhere between the final meeting and the release of the MPI-1.2 draft, this > >text was yanked. My question is, can anyone tell me when and why that happened? > >I don't recall any discussion but I haven't checked the mailing list archives > >yet. > > Good point! My recollection was that the "const" was voted out of > MPI_Pcontrol(). I think it should be included in the MPI-2 errata. > > --Raja Yes "const" was voted out. I agree that this should be put in the MPI-2 errata. -nick. From mpi-core-human@mcs.anl.gov Thu Oct 2 15:34:46 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA27786; Thu, 2 Oct 1997 15:34:46 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA10868 for mpi-core-out; Thu, 2 Oct 1997 14:37:49 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA10859 for ; Thu, 2 Oct 1997 14:37:38 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.5/CRI-gate-news-1.3) with ESMTP id MAA07821 for ; Thu, 2 Oct 1997 12:21:24 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id MAA05706 for ; Thu, 2 Oct 1997 12:21:21 -0500 (CDT) Received: by mrjones.cray.com (8.8.5/btd-b3) id MAA14674; Thu, 2 Oct 1997 12:21:20 -0500 (CDT) From: "Eric Salo" Message-Id: <9710021221.ZM14672@sgi.com> Date: Thu, 2 Oct 1997 12:21:19 -0500 In-Reply-To: William Gropp "Re: MPI-1.2 question" (Oct 2, 8:50am) References: <199710021350.IAA27150@antares.mcs.anl.gov> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: Re: MPI-1.2 question Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > The problem with changing it is this: > > MPI_Pcontrol is the ONE MPI routine that users are EXPECTED to implement > as part of a profiling interface. Changing the prototype makes EXISTING > MPI applications break (the compiler refuses to accept the difference > between the old prototype and new prototype). This violates the MPI Forum > "rule" of not breaking things that aren't wrong. > > Yes, it would have been nice if MPI 1 had been consistent in the use of const > (everywhere or nowhere). But making this change will break existing programs > for no good reason. > > It is true that users could use the MPI_VERSION macros to fix code that used > to work, but this seems like a bad idea. The real issue in my mind is that this change was *voted upon* by the forum. If everyone had the right to unilaterally yank text they didn't like from the standard, there wouldn't be anything left! So I would argue very strongly that we should rip out the const in an errata, just on principle. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Thu Oct 2 17:22:03 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA29304; Thu, 2 Oct 1997 17:22:02 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA10748 for mpi-core-out; Thu, 2 Oct 1997 16:27:19 -0500 Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.235]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA10729 for ; Thu, 2 Oct 1997 16:27:11 -0500 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com (8.8.6/8.8.5tis) with ESMTP id OAA27208; Thu, 2 Oct 1997 14:27:00 -0700 (PDT) Message-Id: <199710022127.OAA27208@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA097457657; Thu, 2 Oct 1997 16:27:37 -0500 From: Raja Daoud Subject: Re: MPI-1.2 question To: salo@sgi.com (Eric Salo) Date: Thu, 02 Oct 1997 16:27:37 CDT Cc: mpi-core@mcs.anl.gov In-Reply-To: <9710021221.ZM14672@sgi.com>; from "Eric Salo" at Oct 2, 97 12:21 (noon) X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with Eric, the text must reflect the results of the committee's votes. Otherwise the credibility and fairness of the MPI-2 effort are undermined. The technical merit of the argument is secondary here. I suggest we treat this issue as a "typo" in the production of the final book, expedite its fix in the errata, and put the matter behind us. --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com http://www.hp.com/go/mpi From mpi-core-human@mcs.anl.gov Fri Oct 3 10:52:33 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA09668; Fri, 3 Oct 1997 10:52:33 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA25970 for mpi-core-out; Fri, 3 Oct 1997 09:54:20 -0500 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA25826 for ; Fri, 3 Oct 1997 09:51:44 -0500 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (SMI-8.6/mail.byaddr) with SMTP id HAA17575; Fri, 3 Oct 1997 07:49:51 -0700 Received: from suneast.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id KAA19634; Fri, 3 Oct 1997 10:48:57 -0400 Received: from caja.East.Sun.COM by suneast.East.Sun.COM (SMI-8.6/SMI-SVR4) id KAA03497; Fri, 3 Oct 1997 10:48:58 -0400 Received: from caja by caja.East.Sun.COM (SMI-8.6/SMI-SVR4) id KAA07549; Fri, 3 Oct 1997 10:48:56 -0400 Date: Fri, 3 Oct 1997 10:48:56 -0400 (EDT) From: Nicholas Nevin -SMCC - High Performance Computing Reply-To: Nicholas Nevin -SMCC - High Performance Computing Subject: Re: MPI-1.2 question To: mpi-core@mcs.anl.gov, salo@sgi.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: F0+9BXu2uHX+AjTSeYgpCw== X-Mailer: dtmail 1.1.0 CDE Version 1.1 SunOS 5.5.1 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > From: "Eric Salo" > To: mpi-core@mcs.anl.gov > Subject: Re: MPI-1.2 question > The real issue in my mind is that this change was *voted upon* by the forum. > If everyone had the right to unilaterally yank text they didn't like from > the standard, there wouldn't be anything left! So I would argue very > strongly that we should rip out the const in an errata, just on principle. > > -- > Eric Salo Silicon Graphics salo@sgi.com Can we get an explanation from those responsible as to why the text was yanked without prior discussion and agreement from the members of the forum? As Eric points out there was an *explicit* vote from the forum to remove const from the binding. This change is thus not just a correction of an inadvertent or typographical error. As already suggested this should be corrected in the errata. -nick. From mpi-core-human@mcs.anl.gov Tue Oct 14 04:17:04 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA09145; Tue, 14 Oct 1997 04:17:04 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA06472 for mpi-core-out; Tue, 14 Oct 1997 03:21:30 -0500 Received: from gatekeeper.pallas.de (gatekeeper.pallas.de [194.45.33.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA06467 for ; Tue, 14 Oct 1997 03:21:15 -0500 Received: from mailhost.pallas.de by gatekeeper.pallas.de id KAA19487; Tue, 14 Oct 1997 10:21:04 +0200 Received: from dilbert.pallas.de by mailhost.pallas.de (SMI-8.6/SMI-SVR4) id KAA21262; Tue, 14 Oct 1997 10:21:03 +0200 Received: from localhost by dilbert.pallas.de (SMI-8.6/SMI-SVR4) id KAA03217; Tue, 14 Oct 1997 10:20:47 +0200 Date: Tue, 14 Oct 1997 10:20:46 +0200 (MET DST) From: Hans-Christian Hoppe X-Sender: hch@dilbert Reply-To: Hans-Christian Hoppe To: Nicholas Nevin -SMCC - High Performance Computing cc: mpi-core@mcs.anl.gov, salo@sgi.com Subject: Re: MPI-1.2 question In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MimeMultipartBoundary" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk --MimeMultipartBoundary Content-Type: TEXT/PLAIN; charset=US-ASCII Dear Nick, recently, a client mentioned that Sun has announced (or released?) a thread-safe MPI-Implementation for the Enterprise server line. As I'm about to embark on an MPI implementation project on a platform supporting pthreads, I'd be interested to learn more about the thread model supported by Sun's MPI, especially with regards to the discussion in the MPI-2 standard, page 196/197 (MPI_THREAD_FUNNELED etc.). Best regards Hans-Christian ------------------------------------------------------------- ---/--- Hans-Christian Hoppe phone : +49-2232-1896-0 / / PALLAS GmbH direct line: +49-2232-1896-11 / / / Hermuelheimer Strasse 10 fax : +49-2232-1896-29 / / / / D-50321 Bruehl email : hch@pallas.de / / / Germany URL : http://www.pallas.de / / PALLAS ------------------------------------------------------------- ---/--- --MimeMultipartBoundary-- From mpi-core-human@mcs.anl.gov Tue Oct 21 07:33:48 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA12740; Tue, 21 Oct 1997 07:33:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id GAA18582 for mpi-core-out; Tue, 21 Oct 1997 06:38:12 -0500 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA18565 for ; Tue, 21 Oct 1997 06:37:52 -0500 Received: from jim (th-eng14-140.pool.dircon.co.uk [194.112.58.140]) by mailhost.dircon.co.uk (8.8.5/8.8.7) with SMTP id MAA28235 for ; Tue, 21 Oct 1997 12:37:34 +0100 (BST) Message-Id: <199710211137.MAA28235@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id MAA01503; Tue, 21 Oct 1997 12:40:13 +0100 To: mpi-core@mcs.anl.gov Subject: SC97 Tutorials Reply-to: jcownie@dolphinics.com Date: Tue, 21 Oct 1997 12:40:13 +0100 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk It's nice to see that MPI is the FIRST mentioned subject in the process release :-) "SC97 Tutorials Offer Instruction in MPI, Mining, Metacomputing, and More" http://www.supercomp.org/sc97/Fhome2.html -- Jim James Cownie Dolphin Interconnect Solutions +44 117 9071438 http://www.dolphinics.com/toolworks From mpi-core-human@mcs.anl.gov Tue Oct 28 04:33:23 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA03627; Tue, 28 Oct 1997 04:33:22 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id DAA22991 for mpi-core-out; Tue, 28 Oct 1997 03:38:06 -0600 Received: from mailhost.dircon.co.uk (mailhost.dircon.co.uk [194.112.32.10]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id DAA22986 for ; Tue, 28 Oct 1997 03:37:55 -0600 Received: from jim (th-eng12-214.pool.dircon.co.uk [194.112.57.214]) by mailhost.dircon.co.uk (8.8.5/8.8.7) with SMTP id JAA11656 for ; Tue, 28 Oct 1997 09:37:34 GMT Message-Id: <199710280937.JAA11656@mailhost.dircon.co.uk> Received: from localhost by jim (SMI-8.6) id JAA00704; Tue, 28 Oct 1997 09:18:06 GMT To: mpi-core@mcs.anl.gov Subject: Tony Skjellum: November 5 - initial meeting of "DARPA Data reorganization standard effort" Date: Tue, 28 Oct 1997 09:18:05 +0000 From: James Cownie Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk ------- Forwarded Message Date: Mon, 27 Oct 1997 12:41:49 -0600 (CST) From: Tony Skjellum To: mpi-realtime@mcs.anl.gov Subject: November 5 - initial meeting of "DARPA Data reorganization standard effort" Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-realtime@MPIRT.Org Precedence: bulk X-UIDL: 029c8f70a77354eac9ee8788fd985757 * PLEASE ATTEND, PLEASE CIRCULATE, PLEASE COMMENT * DARPA has sponsored MPI Software Technology, Inc to hold a series of *open* meetings to informally standardize, improve, discuss, and document high performance data reorganization primitives, algorithms, and API's for transpose/reshape/remapping, of interest in the HPC and embedded HPC space. I am chairing this group, which will meet every six-ten weeks over a period of a year, and produce a report. Ideally, this effort will impact Extensions to MPI 1.2 or MPI 2.0 MPI/RT Existing vendor libraries (non-standard) Compiler-oriented libraries The MPI/RT group, and a few known vendors are invited, but this is a public meeting, and you are welcome to broadcast this notice more widely. People wanting time on the agenda to present their ideas is encouraged. Local host: Arkady Kanevsky, arkady@mitre.org, 617-271-5352. Attendance: to recover costs, $25, payable to MPI Software Technology, Inc, if by check. MITRE is fully reimbursed for any expenses it has as host. Lunch is included. Schedule: meet 8:30am at MITRE "M" building, on November 5, then escort to meeting room. Complete by 5pm, depending on final agenda. Final agenda to be published this week, based in part on requests to speak. At this time, we expect comments from MPI/RT people working on reorganization for RT (Anna Rouenbehler et al) comments from researchers and vendors (mercury, sky, cspi) on their data reorganization libraries at present discussion of format, scope, schedule and operation of other meetings time from users on requests for specific capabilities determination of other people to get involved at further meetings There will be a web site, www.data-re.org, set up soon, and all the meeting information, including resources will be provided there. Access to all the online references about any transpose, reshape, corner-turn, etc, may be sent to me, for collection. The web page will be maintained indefinitely by MPI Softtech, so that this information is widely available. The final report will be made public. There will also be a majordomo list set up, to widen access to this info. Please suggest names for the list, which is going to be up by November 5. Regards, Tony Anthony Skjellum, PhD, President (tony@mpi-softtech.com) MPI Software Technology, Inc., Suite 201, 1 Research Blvd., Starkville, MS 39759 +1-(601)320-4300; FAX: +1-(601)320-4301; http://www.mpi-softtech.com *The source for MPI support, MPI-2, Embedded & High Performance Middleware (tm)* ------- End of Forwarded Message From mpi-core-human@mcs.anl.gov Thu Nov 13 16:13:11 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA13630; Thu, 13 Nov 1997 16:13:11 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA17397 for mpi-core-out; Thu, 13 Nov 1997 15:15:19 -0600 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA17390 for ; Thu, 13 Nov 1997 15:15:08 -0600 Date: Thu, 13 Nov 1997 15:15:06 -0600 Message-Id: <199711132115.PAA05535@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov Subject: MPI_Aint and MPI_Offset Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In the I/O chapter, file sizes, offsets, and displacements are of type MPI_Offset. To support files larger than 2Gbytes (which are already available even on machines with 32-bit addresses), an MPI implementation would have to define MPI_Offset as a 64-bit integer. MPI datatype constructors, however, take displacements of type MPI_Aint. On machines with 32-bit addresses, I find that MPI implementations define MPI_Aint as a 32-bit integer. But then datatypes constructed with these functions cannot be used as filetypes for 64-bit files. Therefore, to support 64-bit files, does it mean that both MPI_Offset and MPI_Aint must be defined as 64-bit integers, even though memory addresses are only 32 bits? In that case, the new type MPI_Offset is not needed. Rajeev From mpi-core-human@mcs.anl.gov Thu Nov 13 17:40:24 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA14904; Thu, 13 Nov 1997 17:40:23 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA21712 for mpi-core-out; Thu, 13 Nov 1997 16:45:13 -0600 Received: from nsgi3.lbl.gov (nsgi3.lbl.gov [128.3.11.51]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA21707; Thu, 13 Nov 1997 16:45:07 -0600 Received: (from wcs@localhost) by nsgi3.lbl.gov (950413.SGI.8.6.12/950213.SGI.AUTOCF) id OAA19302; Thu, 13 Nov 1997 14:45:03 -0800 From: "William Saphir" Message-Id: <9711131445.ZM19301@nsgi3.lbl.gov> Date: Thu, 13 Nov 1997 14:45:03 -0800 In-Reply-To: Rajeev Thakur "MPI_Aint and MPI_Offset" (Nov 13, 3:15pm) References: <199711132115.PAA05535@abacus> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: Rajeev Thakur , mpi-core@mcs.anl.gov Subject: Re: MPI_Aint and MPI_Offset Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Sounds like we have a volunteer to lead MPI 2.1. Thanks, Rajeev! Bill ps. This is a good observation. Though possibly (hopefully?) not as much of a problem as one might expect because files really are "larger" than datatypes - in the sense that a file is an arbitrary number of contiguous filetypes, not a single filetype. Of course you can imagine scenarios where this will be a problem. On Nov 13, 3:15pm, Rajeev Thakur wrote: > Subject: MPI_Aint and MPI_Offset > In the I/O chapter, file sizes, offsets, and displacements are of type > MPI_Offset. To support files larger than 2Gbytes (which are already > available even on machines with 32-bit addresses), an MPI > implementation would have to define MPI_Offset as a 64-bit integer. > > MPI datatype constructors, however, take displacements of type > MPI_Aint. On machines with 32-bit addresses, I find that MPI > implementations define MPI_Aint as a 32-bit integer. But then > datatypes constructed with these functions cannot be used as filetypes > for 64-bit files. Therefore, to support 64-bit files, does it mean > that both MPI_Offset and MPI_Aint must be defined as 64-bit integers, > even though memory addresses are only 32 bits? In that case, the new > type MPI_Offset is not needed. > > Rajeev >-- End of excerpt from Rajeev Thakur From mpi-core-human@mcs.anl.gov Thu Nov 13 18:00:59 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA15046; Thu, 13 Nov 1997 18:00:59 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA22328 for mpi-core-out; Thu, 13 Nov 1997 17:07:21 -0600 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA22323 for ; Thu, 13 Nov 1997 17:07:14 -0600 Date: Thu, 13 Nov 1997 17:07:12 -0600 Message-Id: <199711132307.RAA05765@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov In-reply-to: <9711131445.ZM19301@nsgi3.lbl.gov> (wcs@nsgi3.lbl.gov) Subject: Re: MPI_Aint and MPI_Offset Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > ps. This is a good observation. Though possibly (hopefully?) > not as much of a problem as one might expect because files really > are "larger" than datatypes - in the sense that a file is > an arbitrary number of contiguous filetypes, not a single > filetype. Of course you can imagine scenarios where this > will be a problem. Well, I encountered such a scenario. If you want the details... I have implemented MPI_Type_create_darray and MPI_Type_create_subarray by layering them on top of other MPI datatype constructors (hindexed and struct). I can't do it for arrays >= 2Gbytes, and such arrays are common. A 512 x 1K x 1K single precision array is 2Gbytes. Users are already doing I/O this way --- defining the file view with subarray or darray and writing the array to the file. Rajeev From mpi-core-human@mcs.anl.gov Tue Nov 25 16:27:33 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA26048; Tue, 25 Nov 1997 16:27:32 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA19247 for mpi-core-out; Tue, 25 Nov 1997 15:32:18 -0600 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA19242 for ; Tue, 25 Nov 1997 15:32:07 -0600 Date: Tue, 25 Nov 1997 15:32:04 -0600 Message-Id: <199711252132.PAA13492@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov Subject: MPI_Type_size Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk If an implementation supports 64-bit file sizes, a user could create a filetype of size larger than 2 Gbytes. MPI_Type_size, however, returns the size of a datatype as an integer. On most machines, integers are of size 32 bits, which is a problem. Rajeev From mpi-core-human@mcs.anl.gov Tue Nov 25 19:24:01 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA28449; Tue, 25 Nov 1997 19:24:00 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA24492 for mpi-core-out; Tue, 25 Nov 1997 18:29:22 -0600 Received: from k2.llnl.gov (johnmay@k2.llnl.gov [134.9.1.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA24487 for ; Tue, 25 Nov 1997 18:29:17 -0600 Received: (from johnmay@localhost) by k2.llnl.gov (8.8.5/8.8.5/LLNL-Jun96) id QAA13345; Tue, 25 Nov 1997 16:29:14 -0800 (PST) Message-ID: <19971125162914.42320@k2.llnl.gov> Date: Tue, 25 Nov 1997 16:29:14 -0800 From: John May To: mpi-core@mcs.anl.gov Subject: Re: MPI_Type_size References: <199711252132.PAA13492@abacus> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.79e In-Reply-To: <199711252132.PAA13492@abacus>; from Rajeev Thakur on Tue, Nov 25, 1997 at 03:32:04PM -0600 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk On Tue, Nov 25, 1997 at 03:32:04PM -0600, Rajeev Thakur wrote: > If an implementation supports 64-bit file sizes, a user could create a > filetype of size larger than 2 Gbytes. MPI_Type_size, however, returns > the size of a datatype as an integer. On most machines, integers are > of size 32 bits, which is a problem. This is a general problem on machines with 32-bit integers and 64-bit pointers, regardless of whether the type is used for a filetype or something else. For example, on our DEC cluster the following call... MPI_Type_vector( (1<<30) + 1, 1, 8, MPI_FLOAT, &big ); ...does not produce an error, but MPI_Type_size reports 4 (i.e., the correct value minus 2^32), and MPI_Type_extent reports the correct extent. John From mpi-core-human@mcs.anl.gov Tue Nov 25 19:29:52 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA28620; Tue, 25 Nov 1997 19:29:52 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA24600 for mpi-core-out; Tue, 25 Nov 1997 18:36:39 -0600 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA24595 for ; Tue, 25 Nov 1997 18:36:34 -0600 Date: Tue, 25 Nov 1997 18:36:32 -0600 Message-Id: <199711260036.SAA13752@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov In-reply-to: <19971125162914.42320@k2.llnl.gov> (message from John May on Tue, 25 Nov 1997 16:29:14 -0800) Subject: Re: MPI_Type_size Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Yes, but what I am saying is that because of 64-bit file sizes, a user may have reason to construct large filetypes. Earlier, when datatypes were used only for communication, it was unlikely that one would need to construct a datatype of size larger than 2 Gbytes. Rajeev > Date: Tue, 25 Nov 1997 16:29:14 -0800 > From: John May > > On Tue, Nov 25, 1997 at 03:32:04PM -0600, Rajeev Thakur wrote: > > If an implementation supports 64-bit file sizes, a user could create a > > filetype of size larger than 2 Gbytes. MPI_Type_size, however, returns > > the size of a datatype as an integer. On most machines, integers are > > of size 32 bits, which is a problem. > > This is a general problem on machines with 32-bit integers and > 64-bit pointers, regardless of whether the type is used for > a filetype or something else. For example, on our DEC cluster > the following call... > > MPI_Type_vector( (1<<30) + 1, 1, 8, MPI_FLOAT, &big ); > > ...does not produce an error, but MPI_Type_size reports > 4 (i.e., the correct value minus 2^32), and MPI_Type_extent > reports the correct extent. > > John From mpi-core-human@mcs.anl.gov Wed Nov 26 16:52:37 1997 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA12791; Wed, 26 Nov 1997 16:52:37 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA21697 for mpi-core-out; Wed, 26 Nov 1997 15:56:33 -0600 Received: from abacus (abacus.mcs.anl.gov [140.221.6.130]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA21692 for ; Wed, 26 Nov 1997 15:56:28 -0600 Date: Wed, 26 Nov 1997 15:56:25 -0600 Message-Id: <199711262156.PAA15243@abacus> From: Rajeev Thakur To: mpi-core@mcs.anl.gov Subject: MPI_OFFSET and MPI_AINT Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk File displacements, sizes, and offsets are of type MPI_Offset. If the user wants to communicate file offsets among processes, there should be an MPI_OFFSET type defined that can be used as an argument to MPI_Send. Otherwise the program will not be portable. MPI_Offset off; MPI_Send(&off, 1, MPI_OFFSET, ...); Similarly, there should be an MPI_AINT, say if the user wants to communicate datatype extents among processes. MPI_Aint extent; MPI_Send(&extent, 1, MPI_AINT, ...); Rajeev From mpi-core-human@mcs.anl.gov Mon Jan 26 17:27:21 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA15372; Mon, 26 Jan 1998 17:27:21 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA13662 for mpi-core-out; Mon, 26 Jan 1998 16:31:40 -0600 Received: from Mpi.Mpi-softtech.Com (mpi.MPI-SoftTech.Com [208.152.187.97]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA13655 for ; Mon, 26 Jan 1998 16:31:32 -0600 Received: from tcgmsg.mpi-softtech.com (tcgmsg.MPI-SoftTech.Com [208.152.187.115]); by Mpi.Mpi-softtech.Com using SMTP (8.8.5/7.0m-FWP-MsState); id QAA14696; Mon, 26 Jan 1998 16:31:52 -0600 (CST) Message-ID: <033b01bd2aa9$0047ff80$73bb98d0@tcgmsg.mpi-softtech.com> From: "Clay Taylor, Jr." To: Subject: Data Reorganization Effort Date: Mon, 26 Jan 1998 16:23:12 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.2106.4 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2106.4 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The EMAIL list for Data-reorg (http://www.data-re.org) is now available for sign-on. If you are not already on the list and would like to join, the instructions are at the end of thehome frame. thanks. From mpi-core-human@mcs.anl.gov Wed Apr 8 10:28:58 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA11599; Wed, 8 Apr 1998 10:28:58 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA04298 for mpi-core-out; Wed, 8 Apr 1998 09:35:26 -0500 Received: from gropp.isdn.mcs.anl.gov (gropp-16.win.mcs.anl.gov [140.221.16.17]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id JAA04292 for ; Wed, 8 Apr 1998 09:35:17 -0500 Message-Id: <3.0.1.32.19980408093518.035eedc0@mcs.anl.gov> X-Sender: gropp@mcs.anl.gov X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Wed, 08 Apr 1998 09:35:18 -0500 To: mpi-core@antares.mcs.anl.gov From: William Gropp Subject: Missing Fortran MPI types Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk In listing size-specific MPI types (10.2.5), we list REAL, INTEGER, and COMPLEX. Missing is LOGICAL. In fact, one of the common choices for a byte-type in Fortran is LOGICAL*1. Since we have said that MPI_ is reserved, implementations should not provide MPI_LOGICAL1 etc. I propose that we add MPI_LOGICAL1 MPI_LOGICAL2 MPI_LOGICAL4 MPI_LOGICAL8 MPI_LOGICAL16 paralleling the INTEGER types. Bill From mpi-core-human@mcs.anl.gov Wed Apr 8 14:44:22 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA15502; Wed, 8 Apr 1998 14:44:22 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA13508 for mpi-core-out; Wed, 8 Apr 1998 13:51:56 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA09872 for ; Wed, 8 Apr 1998 11:59:23 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.8/CRI-gate-news-1.3) with ESMTP id LAA03016; Wed, 8 Apr 1998 11:59:20 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id LAA16664; Wed, 8 Apr 1998 11:59:17 -0500 (CDT) From: Eric Salo Received: by mrjones.cray.com (8.8.5/btd-b3) id LAA23584; Wed, 8 Apr 1998 11:59:16 -0500 (CDT) Message-Id: <199804081659.LAA23584@mrjones.cray.com> Date: Wed, 8 Apr 1998 11:59:16 -0500 (CDT) To: mpi-core@antares.mcs.anl.gov, William Gropp Subject: Re: Missing Fortran MPI types Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Since we have said that MPI_ is reserved, implementations should not > provide MPI_LOGICAL1 etc. I gotta disagree here; implementations are free to add whatever extensions they please, such as new datatypes, error codes, or whatever. And it only makes sense that these should also begin with MPI_, with the understanding that anyone who does this risks a conflict in the future if the MPI Forum should happen to grab one of those names in a later standard. So I would argue that it is only *applications* which are forbidden to use the MPI_ prefix. That said, I agree that it would make sense to add the new LOGICAL types that you propose. -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Wed Apr 8 22:18:13 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id WAA21804; Wed, 8 Apr 1998 22:18:12 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id VAA26918 for mpi-core-out; Wed, 8 Apr 1998 21:25:58 -0500 Received: from smtp3.ny.us.ibm.com (smtp3.ny.us.ibm.com [198.133.22.42]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id VAA26909 for ; Wed, 8 Apr 1998 21:25:44 -0500 Received: from relay1.server.ibm.com (relay1.server.ibm.com [9.14.2.98]) by smtp3.ny.us.ibm.com (8.8.7/8.8.7) with ESMTP id WAA42454 for ; Wed, 8 Apr 1998 22:18:59 -0400 Received: from US.IBM.COM (d01lms04.pok.ibm.com [9.117.30.25]) by relay1.server.ibm.com (8.8.7/8.8.7) with SMTP id WAA55444 for ; Wed, 8 Apr 1998 22:23:29 -0400 Received: by US.IBM.COM (Soft-Switch LMS 2.0) with snapi via D01AU004 id 5010400021406958; Wed, 8 Apr 1998 22:29:51 -0400 From: Richard Treumann To: Subject: Re: Missing Fortran MPI types Message-ID: <5010400021406958000002L082*@MHS> Date: Wed, 8 Apr 1998 22:29:51 -0400 MIME-Version: 1.0 Content-Type: text/plain Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Bill Gropp said & Eric Salo replied: > > Since we have said that MPI_ is reserved, implementations should not > > provide MPI_LOGICAL1 etc. > > I gotta disagree here; implementations are free to add whatever > extensions they please, such as new datatypes, error codes, or > whatever. And it only makes sense that these should also begin > with MPI_ I consider both statements to strong. MPI-1 said that an implementation should provide MPI predefined types for those types supported by the compiler. It listed several optional types, each with the MPI_ prefix. That list ended with "etc." which I view as an invitation to continue the pattern by adding MPI_LOGICAL1 if the compiler has LOGICAL*1. MPI-2 uses a weaker form of the same implication by saying "the list of names for such types includes" rather than "is limited to" (pg 297). On the other hand, I believe one MPI implementation defined MPI_TRUE and MPI_FALSE as constants. There is no invitation in the standard to do this so I do not think it was appropriate to use the MPI_ prefix. My overall view is, therefore, closer to Bill's than to Eric's. MPI for the SP does define MPI_LOGICAL1 on the basis of the invitation in MPI 1.1 section 3.2.2. We also provide an non-standard error handler which is not suggested by the standard. That error handler is like MPI_ERRORS_RETURN except it issues an error message. We named it MPE_ERRORS_WARN and avoided the MPI_ prefix. Dick Treumann From mpi-core-human@mcs.anl.gov Thu Apr 9 03:17:29 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA26386; Thu, 9 Apr 1998 03:17:28 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA01826 for mpi-core-out; Thu, 9 Apr 1998 02:25:48 -0500 Received: from timbuk.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA01821 for ; Thu, 9 Apr 1998 02:25:36 -0500 Received: from ironwood.cray.com (root@ironwood-fddi.cray.com [128.162.21.36]) by timbuk.cray.com (8.8.8/CRI-gate-news-1.3) with ESMTP id CAA21860 for ; Thu, 9 Apr 1998 02:25:32 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.147.156]) by ironwood.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id CAA14338 for ; Thu, 9 Apr 1998 02:25:30 -0500 (CDT) Received: by mrjones.cray.com (8.8.5/btd-b3) id CAA24573; Thu, 9 Apr 1998 02:25:30 -0500 (CDT) From: "Eric Salo" Message-Id: <9804090225.ZM24571@sgi.com> Date: Thu, 9 Apr 1998 02:25:30 -0500 In-Reply-To: Richard Treumann "Re: Missing Fortran MPI types" (Apr 8, 10:29pm) References: <5010400021406958000002L082*@MHS> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: Subject: Re: Missing Fortran MPI types Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dick - My reply was poorly worded, thanks for pointing that out. What I was thinking (but didn't type) was that implementations need to use the MPI_ prefix in order to have a protected namespace for globally-scoped internal symbols. For example, we prefix all such symbols with MPI_SGI_ in order to prevent potential name collisions with applications, and I believe you once said that you do something similar. So now we get to the slippery slope. For example, one issue that we're currently struggling with is how best to add some sort of feedback to MPI processes so that they can take advantage of NUMA information. Should we ever decide to officially support such a mechanism, I will be greatly tempted to use that same MPI_SGI_ prefix, which I would claim is sufficiently vendor-specific to qualify as a reasonable naming convention. On the other hand, names like MPI_TRUE and MPI_FALSE are just plain Evil... - Eric -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Thu Apr 9 04:59:14 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id EAA01184; Thu, 9 Apr 1998 04:59:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id EAA03210 for mpi-core-out; Thu, 9 Apr 1998 04:07:31 -0500 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.18.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA03201 for ; Thu, 9 Apr 1998 04:07:22 -0500 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de (8.8.7/8.8.7) with SMTP id LAA04356; Thu, 9 Apr 1998 11:07:17 +0200 (MET DST) env-from (rusrabe@awsrr.rus.uni-stuttgart.de) Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id LAA02400; Thu, 9 Apr 1998 11:07:18 +0200 From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199804090907.LAA02400@awsrr.rus.uni-stuttgart.de> Subject: Re: Missing Fortran MPI types To: gropp@mcs.anl.gov (William Gropp) Date: Thu, 9 Apr 1998 11:07:17 +0200 (DST) Cc: mpi-core@antares.mcs.anl.gov In-Reply-To: <3.0.1.32.19980408093518.035eedc0@mcs.anl.gov> from "William Gropp" at Apr 8, 98 09:35:18 am Content-Type: text Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk First I want to say that I agree with Dick and his citations of the MPI standard. But especially for LOGICAL* there is a technical problem with the sense of LOGICAL* for heterogenous systems. > In listing size-specific MPI types (10.2.5), we list REAL, INTEGER, > and COMPLEX. Missing is LOGICAL. In fact, one of the common > choices for a byte-type in Fortran is LOGICAL*1. It cannot be viewed as " bytes" !!!!!! I will say, the user must expect data conversion ! Only the two values .FALSE. and .TRUE. are allowed. Here three examples (cited from the official manuals) FALSE .FALSE. TRUE .TRUE. range default range default CRAY PVP >= 0 0 < 0 -1 CRAY T3E = 0 0 != 0 [no information] NEC = 0 0 = 1 1 References: CRAY: Vol. 3, CF90 Fortran Language Reference Manual, SR-3905 2.0 PVP: http://www.hlrs.de:8000/library/all/SR-3905_2.0/33727 Section 8.1.7 T3E: http://www.hlrs.de:8000/library/all/SR-3905_2.0/35078 Section 8.3.5 NEC: FORTRAN90/SX Programmer's Guide, Sect. 6.16.9 Therefore if we add --------- > MPI_LOGICAL1 > MPI_LOGICAL2 > MPI_LOGICAL4 > MPI_LOGICAL8 > MPI_LOGICAL16 then I propose to also add a rationale: In heterogeneous MPI implementations with different internal representations the implementation has to convert true and false values from a valid representation on one machine into a valid representation of another machine. Typical valid internal representation for (false/true) values are (0/1), (0/-1), (0/nonzero), and (nonnegative/negative). and an advice to users: The Fortran LOGICAL types should only be used to store valid values for Fortran .TRUE. and .FALSE. because the MPI implementation may transfer only these values. If we do not add MPI_LOGICAL1, etc. then I propose to add a short ---------===---- additional section in 10.2.2 "Problems with Fortran Bindings in MPI" with the content of the rationale and the advice. Otherwise programmers who use LOGICAL*1 as 8 bits do not have a chance to see, that its application can run into touble. Rolf PS: This was also the reason whether we do not have MPI_TYPE_CREATE_F90_LOGICAL. Rolf Rabenseifner Computing Center Rechenzentrum University of Stuttgart Universitaet Stuttgart Phone: ++49 711 6855530 Allmandring 30 FAX: ++49 711 6787626 D-70550 Stuttgart 80 rabenseifner@rus.uni-stuttgart.de Germany http://www.hlrs.de/people/rabenseifner From mpi-core-human@mcs.anl.gov Wed Apr 15 12:53:26 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA22679; Wed, 15 Apr 1998 12:53:26 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA17890 for mpi-core-out; Wed, 15 Apr 1998 12:01:18 -0500 Received: from lsc.nd.edu (lsc.nd.edu [129.74.22.171]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA17862 for ; Wed, 15 Apr 1998 12:00:58 -0500 Received: from porthos.lsc.nd.edu (jsquyres@porthos.lsc.nd.edu [129.74.22.188]) by lsc.nd.edu (8.8.8/8.8.8) with ESMTP id MAA04370 for ; Wed, 15 Apr 1998 12:00:50 -0500 (EST) Received: from localhost (jsquyres@localhost) by porthos.lsc.nd.edu (8.8.7/8.8.7) with SMTP id MAA11530 for ; Wed, 15 Apr 1998 12:00:50 -0500 (EST) Date: Wed, 15 Apr 1998 12:00:50 -0500 (EST) From: Jeff Squyres To: mpi-core@mcs.anl.gov Subject: Oops Message-ID: X-URL: http://www.cse.nd.edu/~jsquyres/ MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Oops. We've found a name inconsistency in the C++ bindings: MPI_TYPE_CREATE_RESIZED has a C binding of: MPI_Type_create_resized(...) Fortran binding of: MPI_TYPE_CREATE_RESIZED(...) C++ binding of: MPI::Datatype::Resized(...) All the other C++ bindings for the type constructors are of the form: MPI::Datatype::Create_whatever(...) Don't know how that one slipped by. {+} Jeff Squyres {+} squyres@cse.nd.edu {+} Perpetual Obsessive Notre Dame Student Craving Utter Madness {+} "I came to ND for 4 years and ended up staying for a decade" From mpi-core-human@mcs.anl.gov Fri Jul 31 17:51:54 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA02041; Fri, 31 Jul 1998 17:51:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA06572 for mpi-core-out; Fri, 31 Jul 1998 14:02:37 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.209.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA06550 for ; Fri, 31 Jul 1998 14:02:05 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (8.8.8/7.0m-FWP-MsState); id OAA15447; Fri, 31 Jul 1998 14:02:00 -0500 (CDT) Date: Fri, 31 Jul 1998 14:02:00 -0500 (CDT) From: Tony Skjellum To: Richard Treumann cc: mpi-core@mcs.anl.gov, mpi-comments@mpi-forum.org Subject: Re: MPI_ALLOC_MEM & no mem In-Reply-To: <5010400025483016000002L062*@MHS> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dick, it seems too drastic to me. I would use malloc() in the way you described without expecting anything unreasonable to happen (ie, exponential backoff). _Tony On Fri, 31 Jul 1998, Richard Treumann wrote: > Date: Fri, 31 Jul 1998 14:42:46 -0400 > From: Richard Treumann > To: mpi-core@mcs.anl.gov, mpi-comments@mpi-forum.org > Subject: MPI_ALLOC_MEM & no mem > > The standard says that MPI_ALLOC_MEM **may** return an error code if the > requested mem is not available. If it returns an error and the user has left > MPI_ERRORS_ARE_FATAL in effect, the job gets killed. > > Does anyone think this is too drastic? Should there be a way for a user to > TRY_ALLOC? After all, malloc is designed to allow failure and users may wish > to try for a big hunk of mem but settle for a smaller one if they need to. > Using MPI_ERRORS_RETURN will probably work but cannot be relied upon because > the standard still says that the internal state of MPI is no longer guarenteed > after an error. > > Dick > > Dick Treumann IBM RS/6000 SP Development > Tele (914) 433-7846 Fax (914) 433-8363 > A. Skjellum, PhD, Assoc. Prof. of Computer Science; Mississippi State Univ. http://www.cs.msstate.edu/~tony; tony@cs.msstate.edu; 601-325-8435 (FAX -8997) "Mississippi (n): small US state where opportunities abound." [Try MPI/RT!] From mpi-core-human@mcs.anl.gov Fri Jul 31 17:51:55 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA02043; Fri, 31 Jul 1998 17:51:55 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA05719 for mpi-core-out; Fri, 31 Jul 1998 13:38:44 -0500 Received: from smtp4.ny.us.ibm.COM (smtp4.ny.us.ibm.com [198.133.22.43]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA05714 for ; Fri, 31 Jul 1998 13:38:39 -0500 Received: from relay1.server.ibm.com (relay1.server.ibm.com [9.14.2.98]) by smtp4.ny.us.ibm.COM (8.8.7/8.8.7) with ESMTP id OAA90140; Fri, 31 Jul 1998 14:28:02 -0400 Received: from US.IBM.COM (d01lms04.pok.ibm.com [9.117.30.25]) by relay1.server.ibm.com (8.8.7/8.8.7) with SMTP id OAA45796; Fri, 31 Jul 1998 14:34:59 -0400 Received: by US.IBM.COM (Soft-Switch LMS 2.0) with snapi via D01AU004 id 5010400025483016; Fri, 31 Jul 1998 14:42:46 -0400 From: Richard Treumann To: , Subject: MPI_ALLOC_MEM & no mem Message-ID: <5010400025483016000002L062*@MHS> Date: Fri, 31 Jul 1998 14:42:46 -0400 MIME-Version: 1.0 Content-Type: text/plain Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The standard says that MPI_ALLOC_MEM **may** return an error code if the requested mem is not available. If it returns an error and the user has left MPI_ERRORS_ARE_FATAL in effect, the job gets killed. Does anyone think this is too drastic? Should there be a way for a user to TRY_ALLOC? After all, malloc is designed to allow failure and users may wish to try for a big hunk of mem but settle for a smaller one if they need to. Using MPI_ERRORS_RETURN will probably work but cannot be relied upon because the standard still says that the internal state of MPI is no longer guarenteed after an error. Dick Dick Treumann IBM RS/6000 SP Development Tele (914) 433-7846 Fax (914) 433-8363 From mpi-core-human@mcs.anl.gov Sat Aug 1 18:27:03 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA22434; Sat, 1 Aug 1998 18:27:03 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA04792 for mpi-core-out; Sat, 1 Aug 1998 17:25:05 -0500 Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA04785 for ; Sat, 1 Aug 1998 17:24:54 -0500 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com (8.8.6/8.8.5tis) with ESMTP id PAA07169; Sat, 1 Aug 1998 15:24:47 -0700 (PDT) Message-Id: <199808012224.PAA07169@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA267320807; Sat, 1 Aug 1998 17:33:27 -0500 From: Raja Daoud Subject: Re: MPI_ALLOC_MEM & no mem To: treumann@us.ibm.com (Richard Treumann) Date: Sat, 01 Aug 1998 17:33:27 CDT Cc: mpi-core@mcs.anl.gov, mpi-comments@mpi-forum.org In-Reply-To: <5010400025483016000002L062*@MHS>; from "Richard Treumann" at Jul 31, 98 2:42 pm X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Some approaches: 1- No change to MPI, implementors provide an option to set the amount of "special memory" needed. 2- Add a try_alloc routine. 3- Add a routine that allocates up to the given size and returns the actual size (avoid loop of try_alloc; lose flexibility in deciding the desired sizes). 4- Reconsider a tiny set of continuable errors. I prefer #4 and believe it can be done in a manner to address the concerns of the users and implementors who voted it down: a- The default MPI_ERRORS_ARE_FATAL stays: users who are concerned about application error checking complexity are happy. b- The small set of continuable errors is decided by consensus: only relevant cases are handled, and only if everybody agrees they can be trivially implemented (an if() very early inside the routine). c- The small set is then mandated: portability concerns are addressed. d- We have sufficient time to debate and evaluate till MPI 2.1. With this MPI gains some flexibility for applications beyond the core of scientific computing. If (b) leads to the empty set, we're back to #1 with no additional harm done. Comments? --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com http://www.hp.com/go/mpi From mpi-core-human@mcs.anl.gov Sun Aug 2 07:10:32 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id HAA02175; Sun, 2 Aug 1998 07:10:29 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id GAA14627 for mpi-core-out; Sun, 2 Aug 1998 06:09:35 -0500 Received: from timbuk-fddi.cray.com (timbuk-fddi.cray.com [128.162.8.102]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id GAA14614 for ; Sun, 2 Aug 1998 06:09:22 -0500 Received: from ledzep.cray.com (ledzep.cray.com [137.38.226.97]) by timbuk-fddi.cray.com (8.8.8/CRI-gate-news-1.3) with ESMTP id GAA10662; Sun, 2 Aug 1998 06:09:19 -0500 (CDT) Received: from ironwood-e185.cray.com (root@ironwood-e185.cray.com [128.162.185.212]) by ledzep.cray.com (8.8.8/craymail-smart) with ESMTP id GAA1202896; Sun, 2 Aug 1998 06:09:18 -0500 (CDT) Received: from mrjones.cray.com (mrjones [128.162.190.117]) by ironwood-e185.cray.com (8.8.4/CRI-ironwood-news-1.0) with ESMTP id GAA28055; Sun, 2 Aug 1998 06:09:16 -0500 (CDT) Received: by mrjones.cray.com (8.8.5/btd-b3) id GAA22410; Sun, 2 Aug 1998 06:09:16 -0500 (CDT) From: "Eric Salo" Message-Id: <9808020609.ZM22408@sgi.com> Date: Sun, 2 Aug 1998 06:09:15 -0500 In-Reply-To: Raja Daoud "Re: MPI_ALLOC_MEM & no mem" (Aug 1, 5:33pm) References: <199808012224.PAA07169@palrel1.hp.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-comments@mpi-forum.org, mpi-core@mcs.anl.gov Subject: Re: MPI_ALLOC_MEM & no mem Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > Some approaches: > > 1- No change to MPI, implementors provide an option to set the amount of > "special memory" needed. > > 2- Add a try_alloc routine. > > 3- Add a routine that allocates up to the given size and returns the > actual size (avoid loop of try_alloc; lose flexibility in deciding > the desired sizes). > > 4- Reconsider a tiny set of continuable errors. 5- Document that a value of NULL returned from MPI_Alloc_mem() is in fact *not* an "error", it is rather the way in which MPI successfully tells the application that their request for memory was not granted. This seems like the simplest and most straightforward way around the problem, but I can't seem to locate Dick's post so perhaps there's a good reason why this would be a Bad Idea...? 6- Define a second error handler for each communicator to handle "non-fatal" errors. The set of conditions which satisfy the "non-fatal" condition is implementation dependent (in fine MPI tradition) but perhaps a subset of required cases could be defined by the MPI Forum (also in fine MPI tradition). The "non-fatal" handler would track all changes to the "fatal" handler in order to preserve compatibility with existing code, but we would define two new routines to set/get it explicitly: MPI_Comm_set_nonfatal_errhandler() MPI_Comm_get_nonfatal_errhandler() -- Eric Salo Silicon Graphics salo@sgi.com From mpi-core-human@mcs.anl.gov Sun Aug 2 16:52:35 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA05861; Sun, 2 Aug 1998 16:52:35 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA22066 for mpi-core-out; Sun, 2 Aug 1998 15:51:29 -0500 Received: from gropp.isdn.mcs.anl.gov (gropp.isdn.mcs.anl.gov [146.137.172.89]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id PAA22060; Sun, 2 Aug 1998 15:51:23 -0500 Message-Id: <3.0.1.32.19980802155217.03575488@mcs.anl.gov> X-Sender: gropp@mcs.anl.gov X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Sun, 02 Aug 1998 15:52:17 -0500 To: Raja Daoud From: William Gropp Subject: Re: MPI_ALLOC_MEM & no mem Cc: treumann@us.ibm.com (Richard Treumann), mpi-core@mcs.anl.gov, mpi-comments@mpi-forum.org In-Reply-To: <199808012224.PAA07169@palrel1.hp.com> References: <5010400025483016000002L062*@MHS> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk One other thing to keep in mind: the reason ERRORS_ARE_FATAL is the default is that users often fail to check error returns. How many codes fail to check the return from malloc? The issue of whether the error is continuable has always been taken by the MPI Forum as a quality-of-implementation issue. A good implementation will strive to make all errors continuable, modulo concerns on performance. I'd rather not get into the issue of how this impacts portability, other than to remind everyone that any Fortran program that expects character*2 or an open of unit=2 to work is also beyond what the standard specifies. In C, malloc could *always* return null. Perhaps MPI is simply a little more up front with the limitations. Bill From mpi-core-human@mcs.anl.gov Mon Aug 3 01:39:01 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id BAA11387; Mon, 3 Aug 1998 01:39:00 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id AAA01660 for mpi-core-out; Mon, 3 Aug 1998 00:35:27 -0500 Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id AAA01647 for ; Mon, 3 Aug 1998 00:35:13 -0500 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com (8.8.6/8.8.5tis) with ESMTP id WAA13722; Sun, 2 Aug 1998 22:35:07 -0700 (PDT) Message-Id: <199808030535.WAA13722@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA278643029; Mon, 3 Aug 1998 00:43:49 -0500 From: Raja Daoud Subject: Re: MPI_ALLOC_MEM & no mem To: salo@sgi.com (Eric Salo) Date: Mon, 03 Aug 1998 0:43:48 CDT Cc: mpi-comments@mpi-forum.org, mpi-core@mcs.anl.gov In-Reply-To: <9808020609.ZM22408@sgi.com>; from "Eric Salo" at Aug 2, 98 6:09 am X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > 5- Document that a value of NULL returned from MPI_Alloc_mem() is in fact > *not* an "error", it is rather the way in which MPI successfully tells This wouldn't work for the default error handlers. I think users want the NULL to be mapped to an auto-abort error in this case. MPI's quality-of-implementation position is broad and has served implementors well, providing a good shield, letting us deal with our "fear, uncertainty, and doubt" at the expense of added value to the users. I don't want to remove it, I think it still has a good function to play in several cases. At the same time, several years have passed since MPI-1. The public implementations have matured, and most if not all vendors have mature implementations as well. I think we have learned that we can provide a good QoI level to users, without fear of seriously impacting performance or complexity of implementation. I think by far most MPIs provide a high level of error detection, and continuation from most of these errors. So we're at a point where implementations provide users extra value that cannot be safely used because it's theoretically not portable. It would be nice if we pass this value on to users, with our thanks for putting up with the sweeping impact of the QoI clause :-). This would allow ISVs to write more sophisticated code that can deal with a wider scope of resource limitation problems. There are a few approaches to do this, without adding new routines for each case: - Update the MPI standard to reflect this higher level of error handling. - Implementors reach a QoI agreement independent of the MPI standard and publish it for ISVs to feel safe about portability. - Let ISVs discover (maybe with some help from implementors) that they can portably get away with many instances of error continuation, thus establishing a de-facto higher QoI level as their applications are ported to other systems. This last scenario is where we're currently heading. It's not a very efficient model and introduces unnecessary uncertainty for users, with some people saying "it's erroneous code," and others saying "just do it, it works in real life." Without changing implementations, and just added text to document the current common-denominator QoI level, we can say: if you detect an error case from that small list, it must be continuable." This doesn't mandate you have to detect it (still a big protection), and if you do, just make sure the if() is upstream in the code (chances are it's already there if the case is on the small list). --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com http://www.hp.com/go/mpi From mpi-core-human@mcs.anl.gov Mon Aug 3 08:31:28 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA20583; Mon, 3 Aug 1998 08:31:27 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA09693 for mpi-core-out; Mon, 3 Aug 1998 07:31:31 -0500 Received: from breeze.cs.wisc.edu (breeze.cs.wisc.edu [128.105.75.36]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA09680 for ; Mon, 3 Aug 1998 07:31:20 -0500 Received: (from lederman@localhost) by breeze.cs.wisc.edu (8.8.6/8.8.6) id HAA22325; Mon, 3 Aug 1998 07:31:18 -0500 (CDT) Date: Mon, 3 Aug 1998 07:31:18 -0500 (CDT) Message-Id: <199808031231.HAA22325@breeze.cs.wisc.edu> From: Steve Huss-Lederman To: mpi-comments@mpi-forum.org, mpi-core@mcs.anl.gov In-reply-to: <199808030535.WAA13722@palrel1.hp.com> (message from Raja Daoud on Mon, 03 Aug 1998 0:43:48 CDT) Subject: Re: MPI_ALLOC_MEM & no mem Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk This whole discussion of continuing after an error is making brain cells tingle in my head. I tried to find the text this morning but failed. I am pretty certain the JOD (and maybe external interfaces) had a discussion of continuable errors. I don't think we ever came to a satisfactory conclusion but recall the text and discussion. I want to say the Leslie Hart was one of the leads but am not certain. Can someone help out here to see if we can recall the history to avoid recreating the wheel. Steve From mpi-core-human@mcs.anl.gov Mon Aug 3 11:38:54 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA25736; Mon, 3 Aug 1998 11:38:54 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA16362 for mpi-core-out; Mon, 3 Aug 1998 10:37:35 -0500 Received: from smtp3.ny.us.ibm.com (smtp3.ny.us.ibm.com [198.133.22.42]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA16339 for ; Mon, 3 Aug 1998 10:36:47 -0500 Received: from relay1.server.ibm.com (relay1.server.ibm.com [9.14.2.98]) by smtp3.ny.us.ibm.com (8.8.7/8.8.7) with ESMTP id LAA73346; Mon, 3 Aug 1998 11:24:46 -0400 Received: from US.IBM.COM (d01lms04.pok.ibm.com [9.117.30.25]) by relay1.server.ibm.com (8.8.7/8.8.7) with SMTP id LAA45792; Mon, 3 Aug 1998 11:33:27 -0400 Received: by US.IBM.COM (Soft-Switch LMS 2.0) with snapi via D01AU004 id 5010400025536207; Mon, 3 Aug 1998 11:41:07 -0400 From: Richard Treumann To: , Subject: Re: MPI_ALLOC_MEM & no mem Message-ID: <5010400025536207000002L072*@MHS> Date: Mon, 3 Aug 1998 11:41:07 -0400 MIME-Version: 1.0 Content-Type: text/plain Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I see that most Forum members who responded prefer the malloc model for MPI_ALLOC_MEM. I do too at this point but it looks to me like the standard, as written, rules it out. I think MPI_TRYALLOC_MEM with an additional flag output would be better if we could do it over but we cannot. It is not clear to me why Raja tells Eric that advising the user to check the returned baseptr for == NULL will not work. We have already introduced at least one case where the standard says a user's not getting all she asked is *not really* an error. The user who reads from a file is advised that she may get no data and still have an MPI_SUCCESS return code if the read is past end_of_file. The user is advised to check how much data was actually read if going past end_of_file is possible. I suggest that the description of MPI_ALLOC_MEM say something like this: "the basptr will be returned as NULL if sufficient memory is not available. MPI_MEM_ALLOC will never return a MPI_ERR_NO_MEM class error code when it might be possible to satisfy some smaller request." I considered " will never return a MPI_ERR_NO_MEM class error code." but some implementation might really be returning "special memory" and only be able to do so if some precondition is met, like an environment variable set to pre-construct a pool of "special memory". If there is a situation where even the smallest request would fail then maybe it is a NO_MEM error. Raja suggests reviving the discussion of "recoverable errors". I do not want to go anywhere near that one. It was a quagmire when we were meeting face to face and I do not want to try to define it in errata or even MPI 2.1. I do not know where the material that was dropped from the standard went if it did not get to the JOD. Dick Dick Treumann IBM RS/6000 SP Development Tele (914) 433-7846 Fax (914) 433-8363 From mpi-core-human@mcs.anl.gov Mon Aug 3 11:44:47 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA25887; Mon, 3 Aug 1998 11:44:47 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA16818 for mpi-core-out; Mon, 3 Aug 1998 10:44:58 -0500 Received: from keymaster.fsl.noaa.gov (root@keymaster.fsl.noaa.gov [137.75.131.182]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA16802 for ; Mon, 3 Aug 1998 10:44:41 -0500 Received: from nipmuc.fsl.noaa.gov (nipmuc.fsl.noaa.gov [137.75.70.27]) by keymaster.fsl.noaa.gov (8.8.5/8.8.5) with ESMTP id PAA25106; Mon, 3 Aug 1998 15:44:38 GMT Received: from localhost (hart@localhost) by nipmuc.fsl.noaa.gov (8.8.5/8.8.5) with SMTP id PAA14818; Mon, 3 Aug 1998 15:44:36 GMT Date: Mon, 3 Aug 1998 09:44:36 -0600 (MDT) From: Leslie Hart To: Steve Huss-Lederman cc: mpi-comments@mpi-forum.org, mpi-core@mcs.anl.gov Subject: Re: MPI_ALLOC_MEM & no mem In-Reply-To: <199808031231.HAA22325@breeze.cs.wisc.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Steve, I did make the original proposal for "continuable" errors. There were a number of arguments along the lines of Eric's that perhaps any "error" that MPIF wants to label as continuable is perhaps not an error. Returning a "NULL" value from MPI_ALLOC_MEM can be labeled to not be an error. On the otherhand, there may be a desire to abort here. I have the TEX of the original proposal. It had a list of error classes that were defined to be continuable and a list that bore no guarantees. There was a function int MPI_Err_is_continuable (int Error, int Flag) where Flag came back true if the error was continuable, false otherwise. A later proposal (I can't remember from whom) suggested an error handler to return on continuable errors and to abort on non-continuable. Perhaps a proposal consistent with Raja's comments would be to revamp the list of continuable errors, have MPI_Err_is_continuable and a new error handler. Have we established a mechanism to udate MPI 2? Do we want to? As an applications developer, I have always been uncomfortable with the notion of no guarantees regarding the state of MPI after an error was encountered. Regards, Leslie Leslie Hart (hart@fsl.noaa.gov) (303) 497 7253 http://www-ad.fsl.noaa.gov/ac FAX 6301 NOAA/Forecast Systems Laboratories Aviation Division Advanced Computing Branch R/E/FS5 325 Broadway Boulder, CO 80303 On Mon, 3 Aug 1998, Steve Huss-Lederman wrote: > This whole discussion of continuing after an error is making brain > cells tingle in my head. I tried to find the text this morning but > failed. I am pretty certain the JOD (and maybe external interfaces) > had a discussion of continuable errors. I don't think we ever came to > a satisfactory conclusion but recall the text and discussion. I want > to say the Leslie Hart was one of the leads but am not certain. Can > someone help out here to see if we can recall the history to avoid > recreating the wheel. > > Steve > From mpi-core-human@mcs.anl.gov Mon Aug 3 12:53:54 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA27682; Mon, 3 Aug 1998 12:53:53 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA19184 for mpi-core-out; Mon, 3 Aug 1998 11:53:56 -0500 Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA19176 for ; Mon, 3 Aug 1998 11:53:43 -0500 Received: from tbag.rsn.hp.com (tbag.rsn.hp.com [15.99.150.164]) by palrel1.hp.com (8.8.6/8.8.5tis) with ESMTP id JAA02197; Mon, 3 Aug 1998 09:53:33 -0700 (PDT) Message-Id: <199808031653.JAA02197@palrel1.hp.com> Received: by tbag.rsn.hp.com (1.39.111.2/16.2) id AA282513735; Mon, 3 Aug 1998 12:02:15 -0500 From: Raja Daoud Subject: Re: MPI_ALLOC_MEM & no mem To: treumann@us.ibm.com (Richard Treumann) Date: Mon, 03 Aug 1998 12:02:15 CDT Cc: mpi-comments@mpi-forum.org, mpi-core@mcs.anl.gov In-Reply-To: <5010400025536207000002L072*@MHS>; from "Richard Treumann" at Aug 3, 98 11:41 am X-Mailer: Elm [revision: 111.1] Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk My answer to Eric is based on two assumptions as to what users really want. I could be wrong, so we'll need to hear from users: 1) users want a safe run with the default error handler and no checking: MPI_Alloc_mem(....); /* use mem if call didn't abort, won't SEGV */ 2) users want to be able to re-call alloc if MPI_ERRORS_RETURN and call failed to find requested memory: MPI_Errhandler_set(MPI_COMM_WOLRD, MPI_ERRORS_RETURN); while ( ! done) { if (MPI_Alloc_mem(...) == MPI_SUCCESS) break; /* shrink mem size, set "done" */ } If users want both (1) and (2), and we know that implementors can easily provide them, then why let anything get in the way? The QoI clause protects implementors from users. If users want a solution that implementors can easily deliver (and in most cases is already delivered in current implementations) then we shouldn't allow the QoI clause to stop us. If on the other hand users don't want (1) and (2), then forget what I proposed. The other solutions presented would also work. --Raja -=- Raja Daoud Hewlett-Packard Co. raja@rsn.hp.com http://www.hp.com/go/mpi From mpi-core-human@mcs.anl.gov Tue Aug 4 19:05:48 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA04482; Tue, 4 Aug 1998 19:05:48 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA10359 for mpi-core-out; Tue, 4 Aug 1998 18:05:16 -0500 Received: from smtp4.ny.us.ibm.COM (smtp4.ny.us.ibm.com [198.133.22.43]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA07416 for ; Tue, 4 Aug 1998 16:49:24 -0500 From: snir@us.ibm.com Received: from relay1.server.ibm.com (relay1.server.ibm.com [9.14.2.98]) by smtp4.ny.us.ibm.COM (8.8.7/8.8.7) with ESMTP id RAA26684; Tue, 4 Aug 1998 17:39:17 -0400 Received: from us.ibm.com (d51mta01.pok.ibm.com [9.117.30.75]) by relay1.server.ibm.com (8.8.7/8.8.7) with SMTP id RAA32588; Tue, 4 Aug 1998 17:46:05 -0400 Received: by us.ibm.com(Lotus SMTP MTA v4.6.1 (569.2 2-6-1998)) id 85256656.00774786 ; Tue, 4 Aug 1998 17:42:50 -0400 X-Lotus-FromDomain: IBMUS To: Richard Treumann cc: mpi-comments@mpi-forum.org, mpi-core@mcs.anl.gov Message-ID: <85256655.006A74B0.00@us.ibm.com> Date: Mon, 3 Aug 1998 15:23:08 -0400 Subject: Re: MPI_ALLOC_MEM & no mem Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk The minimum change needed in the MPI standard to accomodate the issue that Dick is raising is to state that mpi_alloc_mem is associated by default with mpi_error_returns. Users will have to check the error code returned by the code, before they use the pointer returned by the call. This is somewhat different than the practice for malloc in C, but is consistent with MPI practice. If one wants to handle this issue with no changes in the standard, and no additional functions, then we can do the following: (1) have a new error handler that is not fatal for mpi_alloc_mem. This handler is not fatal for mpi_alloc_mem, and for any other mpi function where there is a consensus that fatal error is the wrong choice. (2) agree that a call to mpi_alloc_mem that fails does not cause any damage, expect perhaps returning a null pointer and certainly returning a suitable error code. We might like the new error handler to be the default, and the standard states that mpi_errors_are_fatal is the default. But an implementation can provide an argument to mpiexec in order to change the default, so no major pain, here. The major pain is lack of standardization, if each vendor starts deciding on its own which errors are nonfatal for which error handler. On the other hand, we could not agree in the forum meeting which errors are continuable and which are not, so is it realistic to do this by email? The following might be realistic: 1. agree on a name for this new error handler (e.g., mpi_errors_are_handled). 2. agree on a mimimum, and perhaps maximum list of errors that would be handled by this handler without being fatal and also agreeing on the process state upon return from an errorneous call. 3. hope (or fear) that customer pressure will lead vendors to standardize on the maximum list. Marc Snir, Senior Manager Scalable Parallel Systems, IBM T.J. Watson Research Center http://www.ibm.research.com/people/s/snir Tel: 914-945-3204 (8-862-3204) Fax: 914-945-4425 (8-862-4425) From mpi-core-human@mcs.anl.gov Wed Sep 2 08:44:15 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA28101; Wed, 2 Sep 1998 08:44:14 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA29451 for mpi-core-out; Wed, 2 Sep 1998 07:47:12 -0500 Received: from smtp3.ny.us.ibm.com (smtp3.ny.us.ibm.com [198.133.22.42]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id HAA29442 for ; Wed, 2 Sep 1998 07:46:57 -0500 Received: from relay2.server.ibm.com (relay2.server.ibm.com [9.14.2.99]) by smtp3.ny.us.ibm.com (8.8.7/8.8.7) with ESMTP id IAA40420 for ; Wed, 2 Sep 1998 08:31:21 -0400 Received: from US.IBM.COM (d01lms04.pok.ibm.com [9.117.30.25]) by relay2.server.ibm.com (8.8.7/8.8.7) with SMTP id IAA35624 for ; Wed, 2 Sep 1998 08:40:26 -0400 Received: by US.IBM.COM (Soft-Switch LMS 2.0) with snapi via D01AU004 id 5010400026508548; Wed, 2 Sep 1998 08:52:07 -0400 From: Richard Treumann To: Subject: Full thread support MPI Message-ID: <5010400026508548000002L082*@MHS> Date: Wed, 2 Sep 1998 08:52:07 -0400 MIME-Version: 1.0 Content-Type: text/plain Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At the Forum meetings we discussed MPI and threads quite a bit. At that time I understood the IBM SP version of MPI to be the only one which allowed users to create multiple threads and call MPI freely from any thread. Is anyone aware of any other MPI implementation which has this level of threads support? Was there another threads MPI at that time that I was not aware of? Have any been offered since? IBM's threaded MPI was MPI-1 based so does not include the new functions in MPI-2 like MPI_INIT_THREAD. We will add them at some point but did find them essential in the original design. (They did not exist yet) Thanks - Dick Dick Treumann IBM RS/6000 SP Development Tele (914) 433-7846 Fax (914) 433-8363 From mpi-core-human@mcs.anl.gov Wed Sep 2 10:26:41 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id KAA29761; Wed, 2 Sep 1998 10:26:40 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id JAA04347 for mpi-core-out; Wed, 2 Sep 1998 09:29:49 -0500 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id JAA04340 for ; Wed, 2 Sep 1998 09:29:38 -0500 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (SMI-8.6/mail.byaddr) with SMTP id HAA09833; Wed, 2 Sep 1998 07:28:20 -0700 Received: from suneast.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id KAA21783; Wed, 2 Sep 1998 10:28:17 -0400 Received: from focus-tm.East.Sun.COM by suneast.East.Sun.COM (SMI-8.6/SMI-SVR4) id KAA10423; Wed, 2 Sep 1998 10:28:17 -0400 Received: from focus-tm by focus-tm.East.Sun.COM (SMI-8.6/SMI-SVR4) id KAA16024; Wed, 2 Sep 1998 10:28:16 -0400 Date: Wed, 2 Sep 1998 10:28:16 -0400 (EDT) From: Terry Dontje - Sun High Performance Computing Reply-To: Terry Dontje - Sun High Performance Computing Subject: Re: Full thread support MPI To: mpi-core@mcs.anl.gov, treumann@us.ibm.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Xbp5HIukngFIJNUvP8UQWw== X-Mailer: dtmail 1.1.0 CDE Version 1.1 SunOS 5.5.1 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk From: Richard Treumann To: Subject: Full thread support MPI MIME-Version: 1.0 At the Forum meetings we discussed MPI and threads quite a bit. At that time I understood the IBM SP version of MPI to be the only one which allowed users to create multiple threads and call MPI freely from any thread. Is anyone aware of any other MPI implementation which has this level of threads support? Was there another threads MPI at that time that I was not aware of? Have any been offered since? IBM's threaded MPI was MPI-1 based so does not include the new functions in MPI-2 like MPI_INIT_THREAD. We will add them at some point but did find them essential in the original design. (They did not exist yet) Sun's MPI library has supported threads since November '97 this was an MPI-1 based library. --td |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Terry D. Dontje \http://hpcweb.east.sun.com/~tdd/index.html \ Keeper of the Focus of Energy /|\ SMCC - HPC Group > Holder of Past Knowledge <-X-> / I'll *ROCK* till I drop!!! \|/ (978) 442-3113 / tdd@east.sun.com |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| From mpi-core-human@mcs.anl.gov Wed Sep 2 12:21:45 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA01814; Wed, 2 Sep 1998 12:21:44 -0400 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA10771 for mpi-core-out; Wed, 2 Sep 1998 11:26:27 -0500 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.209.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA10764 for ; Wed, 2 Sep 1998 11:26:12 -0500 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu using SMTP (8.8.8/7.0m-FWP-MsState); id LAA06062; Wed, 2 Sep 1998 11:25:49 -0500 (CDT) Date: Wed, 2 Sep 1998 11:25:49 -0500 (CDT) From: Tony Skjellum To: Richard Treumann cc: mpi-core@mcs.anl.gov Subject: Re: Full thread support MPI In-Reply-To: <5010400026508548000002L082*@MHS> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Forthcoming versions of MPI/Pro products are thread safe. -Tony On Wed, 2 Sep 1998, Richard Treumann wrote: > Date: Wed, 2 Sep 1998 08:52:07 -0400 > From: Richard Treumann > To: mpi-core@mcs.anl.gov > Subject: Full thread support MPI > > At the Forum meetings we discussed MPI and threads quite a bit. At that time I > understood the IBM SP version of MPI to be the only one which allowed users to > create multiple threads and call MPI freely from any thread. > > Is anyone aware of any other MPI implementation which has this level of threads > support? Was there another threads MPI at that time that I was not aware of? > Have any been offered since? > > IBM's threaded MPI was MPI-1 based so does not include the new functions in > MPI-2 like MPI_INIT_THREAD. We will add them at some point but did find them > essential in the original design. (They did not exist yet) > > Thanks - Dick > > Dick Treumann IBM RS/6000 SP Development > Tele (914) 433-7846 Fax (914) 433-8363 > A. Skjellum, PhD, Assoc. Prof. of Computer Science; Mississippi State Univ. http://www.cs.msstate.edu/~tony; tony@cs.msstate.edu; 601-325-8435 (FAX -8997) "Mississippi (n): small US state where opportunities abound." [Try MPI/RT!] From mpi-core-human@mcs.anl.gov Wed Nov 18 16:20:18 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA23267; Wed, 18 Nov 1998 16:20:17 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA14039 for mpi-core-out; Wed, 18 Nov 1998 15:19:46 -0600 Received: from smtp3.ny.us.ibm.com (smtp3.ny.us.ibm.com [198.133.22.42]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id OAA11315; Wed, 18 Nov 1998 14:44:00 -0600 From: jpprost@us.ibm.com Received: from northrelay01.pok.ibm.com (northrelay01.pok.ibm.com [9.117.200.21]) by smtp3.ny.us.ibm.com (8.8.7/8.8.7) with ESMTP id PAA34458; Wed, 18 Nov 1998 15:25:22 -0500 Received: from us.ibm.com (d51mta03.pok.ibm.com [9.117.200.31]) by northrelay01.pok.ibm.com (8.8.7/NCO v1.2) with SMTP id PAA06136; Wed, 18 Nov 1998 15:43:16 -0500 Received: by us.ibm.com(Lotus SMTP MTA Internal build v4.6.2 (651.2 6-10-1998)) id 852566C0.0071D02F ; Wed, 18 Nov 1998 15:43:08 -0500 X-Lotus-FromDomain: IBMUS To: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov Message-ID: <852566C0.0071CE1F.00@us.ibm.com> Date: Wed, 18 Nov 1998 15:43:00 -0500 Subject: Double extended format defined by MPI2 I/O for external32 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk It seems to me that there is a typo in the description of the MPI format for IEEE Double-extended" numbers. The bias stipulated by the standard is +10383. It should be +16383, since there are 15 exponent bits: 16383 = 2^(15-1)-1. Could you please confirm to me that this is really a typo. Thanks, Jean-Pierre Prost ################################################### Jean-Pierre Prost IBM T.J. Watson Research Center PO Box 218 Yorktown Heights, NY 10598 USA Phone: (914) 945 3225 Fax: (914) 945 2141 Internet: jpprost@us.ibm.com Lotus Notes: Jean-Pierre Prost/Watson/IBM @ IBMUS VM: IBMUSM11(JPPROST) Home page: http://www.research.ibm.com/people/p/prost/ ################################################### From mpi-core-human@mcs.anl.gov Thu Nov 19 03:58:37 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id DAA08334; Thu, 19 Nov 1998 03:58:37 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id CAA08043 for mpi-core-out; Thu, 19 Nov 1998 02:58:57 -0600 Received: from artemis.rus.uni-stuttgart.de (artemis.rus.uni-stuttgart.de [129.69.1.28]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA08022; Thu, 19 Nov 1998 02:58:34 -0600 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by artemis.rus.uni-stuttgart.de (8.8.8/8.8.8) with SMTP id JAA24376; Thu, 19 Nov 1998 09:58:30 +0100 (MET) env-from (rusrabe@awsrr.rus.uni-stuttgart.de) Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id IAA03468; Thu, 19 Nov 1998 08:58:29 GMT From: Rabenseifner@RUS.Uni-Stuttgart.DE (Rolf Rabenseifner) Message-Id: <199811190858.IAA03468@awsrr.rus.uni-stuttgart.de> Subject: Re: Double extended format defined by MPI2 I/O for external32 To: jpprost@us.ibm.com Date: Thu, 19 Nov 1998 09:58:28 +0100 (MEZ) Cc: mpi-core@mcs.anl.gov, mpi-comment@mcs.anl.gov In-Reply-To: <852566C0.0071CE1F.00@us.ibm.com> from "jpprost@us.ibm.com" at Nov 18, 98 03:43:00 pm Content-Type: text Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Yes, it is a typo and it is already corrected in the offical errata, see http://www.mpi-forum.org/docs/errata-20.ps.Z (last modified 20 May 1998) page 3, lines 25-28. > It seems to me that there is a typo in the description of the MPI format > for IEEE Double-extended" numbers. The bias stipulated by the standard > is +10383. It should be +16383, since there are 15 exponent bits: > 16383 = 2^(15-1)-1. > Could you please confirm to me that this is really a typo. > Thanks, > Jean-Pierre Prost > > ################################################### > Jean-Pierre Prost > IBM T.J. Watson Research Center > PO Box 218 > Yorktown Heights, NY 10598 > USA > Phone: (914) 945 3225 > Fax: (914) 945 2141 > Internet: jpprost@us.ibm.com > Lotus Notes: Jean-Pierre Prost/Watson/IBM @ IBMUS > VM: IBMUSM11(JPPROST) > Home page: http://www.research.ibm.com/people/p/prost/ > ################################################### > Rolf Rabenseifner High Performance Computing Parallel Computing Center Stuttgart (HLRS) Rechenzentrum Universitaet Stuttgart (RUS) Phone: ++49 711 6855530 Allmandring 30 FAX: ++49 711 6787626 D-70550 Stuttgart rabenseifner@rus.uni-stuttgart.de Germany http://www.hlrs.de/people/rabenseifner From mpi-core-human@mcs.anl.gov Thu Nov 19 11:05:06 1998 Return-Path: Received: from antares.mcs.anl.gov by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA18402; Thu, 19 Nov 1998 11:05:05 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA19588 for mpi-core-out; Thu, 19 Nov 1998 10:05:49 -0600 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.209.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA19583 for ; Thu, 19 Nov 1998 10:05:41 -0600 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu (8.9.1/8.9.1/MsState-Aurora/evision: 1.3 $) with SMTP; id KAA26076; Thu, 19 Nov 1998 10:05:37 -0600 (CST) Date: Thu, 19 Nov 1998 10:05:37 -0600 (CST) From: Tony Skjellum To: mpi-realtime@Mpirt.org, mpi-core@mcs.anl.gov Subject: Reminder: CFP for MPIDC99 Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk First Call for Papers and Participation MPIDC'99 - Message Passing Interface Developer's and User's Conference March 10-12, 1999, Airport Radisson, Atlanta, GA, USA Preconference Tutorials March 8-9, 1999. This is the third in a series of successful conferences centering on design, implementation, and realization of cluster parallel and dedicated parallel message passing systems, applications, and related software technology, centering originally on the standards of the MPI Forum. The first two meetings were held at the University of Notre Dame in 1996 and 1997. This conference is broadened to include the real-time MPI (MPI/RT) standard as well. This conference features preconference tutorials on MPI-1, MPI-2, and MPI/RT as well as a formal proceedings that will be made available at the conference. Each papers and poster proposed for inclusion will be refereed by at least two anonymous referees. Topics of Interest for submission and presentation include but are not limited to: * Applications * Efficient implementations of MPI-1, MPI-2, and MPI/RT * Utilization of MPI with new cluster interconnects, such as VI Architecture * Message passing parallel computing in new environments, such as Java * Comparisons of technology such as DCOM and CORBA with MPI * Integration of MPI with scheduling * Performance, security, and scheduling issues for MPI in cluster settings * Embedded and real-time utilizations of MPI and MPI/RT * Interoperability Approaches (IMPI) * Progress, Multi-protocol, and Thread-safe MPIs * Data Reorganization Strategies and Emerging Standards * Tools for Middleware Validation, Program Performance Monitoring, and Debugging * Extensions to Message Passing Standards, Integration with Other standards Up-to-date Information: www.mpidc.org, or mpidc-info@mpidc.org (both will be up by October 15, 1998) Organizing Committee Mark Baker, University Portsmouth, mab@sis.port.ac.uk Judy Devaney, NIST, judy@cam.nist.gov Richard Games, The MITRE Corporation, rg@mitre.org Arkady Kanevsky, The MITRE Corporation, arkady@mitre.org David Likfa, Cornell University, lifka@tc.cornell.edu Andrew Lumsdaine, University of Notre Dame Ewing Lusk, Argonne National Laboratory, lusk@mcs.anl.gov Anthony Skjellum, Mississippi State University and MPI Software Technology, Inc Chairs Anthony Skjellum, Conference Co-Chair, tony@cs.msstate.edu Andrew Lumsdaine, Conference Co-Chair, lums@lsc.nd.edu Deadlines Contributed papers are due by December 7, 1998. These should follow the two-column, IEEE-type style; full papers, representing mature projects should be 10 pages or less in length. Short papers should follow the same guidelines, and be 6 pages or less in length, representing work that is more preliminary or lacking specific results. Poster presentation proposals should include a two-page paper of the same format, describing the main contents of the poster. Papers must be original in content and not be simultaneously submitted elsewhere. Authors will be notified of acceptability of manuscripts by January 15, and final papers will be due by February 1, 1999. The proceedings will be distributed at the meeting. Proposals for pre-conference tutorials should be received by November 15, 1998. Notification of the acceptability of the tutorials will be made by December 15, 1998. Tutorials will have to be self-supporting, but excess over costs will be provided to tutorial proposers to defray travel expenses, and registration expenses at MPIDC'99, insofar as possible.. Participation The widest possible participation is sought. International, student, government, and industrial participants are encouraged to attend, in addition to academic participants. Attendance at the three-day meeting is $300, for registration by February 1, 1999, and $450 thereafter. This includes three conference lunches, and a conference banquet, as well as one copy of the proceeedings. Conference tutorial topics and schedule will announced in early January, 1999. On-site registration is strongly discouraged in favor of advance registration to help control costs. Registration payments made payable to MPIDC, % Anthony Skjellum, Mississippi State University, PO Box 9637, Mississippi State, MS 39762, USA. +1-601-325-8435. Visa/Mastercard will also be accepted. The conference proceedings is sponsored by MPI Software Technology, Inc. From mpi-core-human@mcs.anl.gov Tue Feb 2 09:56:17 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA09996; Tue, 2 Feb 1999 09:56:17 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA00922 for mpi-core-out; Tue, 2 Feb 1999 08:54:35 -0600 Received: from www.ccrl-nece.technopark.gmd.de (www.ccrl-nece.technopark.gmd.de [193.175.160.84]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id EAA26739 for ; Tue, 2 Feb 1999 04:52:46 -0600 Received: from ccrl-nece.technopark.gmd.de (root@gauss.ccrl-nece.technopark.gmd.de [193.175.160.57]) by www.ccrl-nece.technopark.gmd.de (8.8.5/8.8.5) with ESMTP id MAA01061 for ; Tue, 2 Feb 1999 12:53:56 +0100 Received: from sgi8.ccrl-nece.technopark.gmd.de (sgi8.ccrl-nece.technopark.gmd.de [193.175.160.90]) by ccrl-nece.technopark.gmd.de (8.8.5/8.8.5) with SMTP id KAA26798 for ; Tue, 2 Feb 1999 10:52:13 GMT Received: from sgi8.ccrl-nece.technopark.gmd.de by sgi8.ccrl-nece.technopark.gmd.de (950413.SGI.8.6.12) id LAA04919; Tue, 2 Feb 1999 11:52:06 +0100 Message-Id: <199902021052.LAA04919@sgi8.ccrl-nece.technopark.gmd.de> X-Mailer: exmh version 2.0.2 2/24/98 To: mpi-core@mcs.anl.gov Subject: Question to MPI/IO Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 02 Feb 1999 11:52:05 +0100 From: Hubert Ritzdorf Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi, I am implementing MPI for NEC SX computers and I have a question to Datarep Conversion functions (chapter 9.5.3 of the MPI-2 document) (*) If MPI cannot allocate a (file) buffer large enough to hold all the the data to be converted, the standard says: For the first call, MPI will call the (conversion) function with ``position'' set to zero. ... Then in subsequent calls to the conversion function, MPI will increment the value in ``position'' by the ``count'' of items converted in the previous call. My question is: Is the user buffer ``buf'' incremented correspondingly in the second call (and further calls) of the conversion function (cf. Version i below) or must the user find the correct position in the user buffer ``buf'' within the conversion function (cf. Version ii below)? i.e. /* first call */ position = 0; write_fn (buf, datatype, count, filebuf, position, extra_state); /* write filebuf to the file */ write (.., filebuf, count*datarep_size); /* (Version i) second call, the user must find the correct position in buf */ position = count; write_fn (buf, datatype, count, filebuf, position, extra_state); or /* (Version ii) second call, MPI transfers the new user buffer location */ position = count; next_buf = (char *) buf + count * native_size; write_fn (next_buf, datatype, count, filebuf, position, extra_state); /* write filebuf to the file */ write (.., filebuf, count*datarep_size); Best regards Hubert ______________________________________________________________________________ Hubert Ritzdorf NEC Europe Ltd. C&C Research Laboratories Rathausallee 10 D-53757 Sankt Augustin Tel.: +49 (0) 2241 / 92 52 65 Fax : +49 (0) 2241 / 92 52 99 e-mail : ritzdorf@ccrl-nece.technopark.gmd.de InPerson: ritzdorf@193.175.160.90 or ritzdorf@sgi8.ccrl-nece.technopark.gmd.de WWW : http://www.ccrl-nece.technopark.gmd.de/~ritzdorf ______________________________________________________________________________ From mpi-core-human@mcs.anl.gov Tue Feb 2 11:16:26 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id LAA11185; Tue, 2 Feb 1999 11:16:26 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id KAA03956 for mpi-core-out; Tue, 2 Feb 1999 10:17:06 -0600 Received: from smtp3.ny.us.ibm.com (smtp3.ny.us.ibm.com [198.133.22.42]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id KAA03323 for ; Tue, 2 Feb 1999 10:01:02 -0600 From: jpprost@us.ibm.com Received: from northrelay01.pok.ibm.com (northrelay01.pok.ibm.com [9.117.200.21]) by smtp3.ny.us.ibm.com (8.8.7/8.8.7) with ESMTP id KAA25178; Tue, 2 Feb 1999 10:36:50 -0500 Received: from D51MTA03.pok.ibm.com (d51mta03.pok.ibm.com [9.117.200.31]) by northrelay01.pok.ibm.com (8.8.7m1/NCO v1.8) with SMTP id LAA95248; Tue, 2 Feb 1999 11:00:05 -0500 Received: by D51MTA03.pok.ibm.com(Lotus SMTP MTA Internal build v4.6.2 (651.2 6-10-1998)) id 8525670C.0057E1A7 ; Tue, 2 Feb 1999 10:59:53 -0500 X-Lotus-FromDomain: IBMUS To: Hubert Ritzdorf cc: mpi-core@mcs.anl.gov Message-ID: <8525670C.0057DAD5.00@D51MTA03.pok.ibm.com> Date: Tue, 2 Feb 1999 10:59:34 -0500 Subject: Re: Question to MPI/IO Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hubert, Version (i) is the version intended by the standard, which says: "The function must begin copying at the location in userbuf specified by position into the (tiled) datatype." Note that your note was confusing, because the descriptions of the two alternatives before the code and within the code are opposite. The right description is the one given within the code. Instead of saying "the user must find...", I would rather say "the user defined conversion routine must determine..." Jean-Pierre ################################################### Jean-Pierre Prost IBM T.J. Watson Research Center PO Box 218 Yorktown Heights, NY 10598 USA Phone: (914) 945 3225 Fax: (914) 945 2141 Internet: jpprost@us.ibm.com Lotus Notes: Jean-Pierre Prost/Watson/IBM @ IBMUS VM: IBMUSM11(JPPROST) Home page: http://www.research.ibm.com/people/p/prost/ ################################################### Hubert Ritzdorf on 02/02/99 05:52:05 To: mpi-core@mcs.anl.gov cc: (bcc: Jean-Pierre Prost/Watson/IBM) Subject: Question to MPI/IO Hi, I am implementing MPI for NEC SX computers and I have a question to Datarep Conversion functions (chapter 9.5.3 of the MPI-2 document) (*) If MPI cannot allocate a (file) buffer large enough to hold all the the data to be converted, the standard says: For the first call, MPI will call the (conversion) function with ``position'' set to zero. ... Then in subsequent calls to the conversion function, MPI will increment the value in ``position'' by the ``count'' of items converted in the previous call. My question is: Is the user buffer ``buf'' incremented correspondingly in the second call (and further calls) of the conversion function (cf. Version i below) or must the user find the correct position in the user buffer ``buf'' within the conversion function (cf. Version ii below)? i.e. /* first call */ position = 0; write_fn (buf, datatype, count, filebuf, position, extra_state); /* write filebuf to the file */ write (.., filebuf, count*datarep_size); /* (Version i) second call, the user must find the correct position in buf */ position = count; write_fn (buf, datatype, count, filebuf, position, extra_state); or /* (Version ii) second call, MPI transfers the new user buffer location */ position = count; next_buf = (char *) buf + count * native_size; write_fn (next_buf, datatype, count, filebuf, position, extra_state); /* write filebuf to the file */ write (.., filebuf, count*datarep_size); Best regards Hubert ___________________________________________________________________________ ___ Hubert Ritzdorf NEC Europe Ltd. C&C Research Laboratories Rathausallee 10 D-53757 Sankt Augustin Tel.: +49 (0) 2241 / 92 52 65 Fax : +49 (0) 2241 / 92 52 99 e-mail : ritzdorf@ccrl-nece.technopark.gmd.de InPerson: ritzdorf@193.175.160.90 or ritzdorf@sgi8.ccrl-nece.technopark.gmd.de WWW : http://www.ccrl-nece.technopark.gmd.de/~ritzdorf ___________________________________________________________________________ ___ From mpi-core-human@mcs.anl.gov Tue Feb 2 12:26:47 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA12516; Tue, 2 Feb 1999 12:26:47 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA07210 for mpi-core-out; Tue, 2 Feb 1999 11:25:47 -0600 Received: from riesz.llnl.gov (riesz.llnl.gov [134.9.14.41]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id LAA07200 for ; Tue, 2 Feb 1999 11:25:35 -0600 Received: (from johnmay@localhost) by riesz.llnl.gov (8.9.1/8.9.1/LLNL-Sep98) id JAA09689; Tue, 2 Feb 1999 09:24:55 -0800 (PST) Message-ID: <19990202092455.C9533@llnl.gov> Date: Tue, 2 Feb 1999 09:24:55 -0800 From: John May To: jpprost@us.ibm.com, Hubert Ritzdorf Cc: mpi-core@mcs.anl.gov Subject: Re: Question to MPI/IO References: <8525670C.0057DAD5.00@D51MTA03.pok.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: <8525670C.0057DAD5.00@D51MTA03.pok.ibm.com>; from jpprost@us.ibm.com on Tue, Feb 02, 1999 at 10:59:34AM -0500 Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with Jean-Pierre that version (i) is correct. Version (ii) would not be as simple as it looks because the user might call MPI_File_write with complex derived data type, and the first call to the conversion function might return after traversing only part of the type. Having the MPI implementation figure out the right buffer location for the second call would be pretty complicated, but the user function can just cache this information somewhere when it finishes the first call and then pick it up when it starts the second call. Since you're working on the datarep converters, you might be interested in a test code we posted on the web at http://www.llnl.gov/sccd/lc/piop/ . Look for the links at the end of the MPI-IO section. John On Tue, Feb 02, 1999 at 10:59:34AM -0500, jpprost@us.ibm.com wrote: > Hubert, > > Version (i) is the version intended by the standard, which says: > "The function must begin copying at the location in userbuf specified by > position > into the (tiled) datatype." > Note that your note was confusing, because the descriptions of the two > alternatives before > the code and within the code are opposite. The right description is the one > given > within the code. Instead of saying "the user must find...", I would rather > say "the user defined > conversion routine must determine..." > Jean-Pierre > > ################################################### > Jean-Pierre Prost > IBM T.J. Watson Research Center > PO Box 218 > Yorktown Heights, NY 10598 > USA > Phone: (914) 945 3225 > Fax: (914) 945 2141 > Internet: jpprost@us.ibm.com > Lotus Notes: Jean-Pierre Prost/Watson/IBM @ IBMUS > VM: IBMUSM11(JPPROST) > Home page: http://www.research.ibm.com/people/p/prost/ > ################################################### > > > > Hubert Ritzdorf on 02/02/99 05:52:05 > > To: mpi-core@mcs.anl.gov > cc: (bcc: Jean-Pierre Prost/Watson/IBM) > Subject: Question to MPI/IO > > > > > > > Hi, > > I am implementing MPI for NEC SX computers and I have a question to > Datarep Conversion functions (chapter 9.5.3 of the MPI-2 document) > > (*) If MPI cannot allocate a (file) buffer large enough to hold all the > the data to be converted, the standard says: > > For the first call, MPI will call the (conversion) function with > ``position'' set to zero. ... > Then in subsequent calls to the conversion function, > MPI will increment the value in ``position'' by the ``count'' > of items converted in the previous call. > > My question is: > Is the user buffer ``buf'' incremented correspondingly > in the second call (and further calls) of the conversion function > (cf. Version i below) or > must the user find the correct position in the user buffer ``buf'' > within the conversion function (cf. Version ii below)? > > i.e. > > /* first call */ > position = 0; > write_fn (buf, datatype, count, filebuf, position, extra_state); > > /* write filebuf to the file */ > > write (.., filebuf, count*datarep_size); > > > > /* (Version i) > second call, the user must find the correct position in buf */ > > position = count; > write_fn (buf, datatype, count, filebuf, position, extra_state); > > > or > > /* (Version ii) > second call, MPI transfers the new user buffer location */ > > position = count; > next_buf = (char *) buf + count * native_size; > write_fn (next_buf, datatype, count, filebuf, position, > extra_state); > /* write filebuf to the file */ > > write (.., filebuf, count*datarep_size); > > > > Best regards > > Hubert > > > > ___________________________________________________________________________ > ___ > > Hubert Ritzdorf > > NEC Europe Ltd. > C&C Research Laboratories > Rathausallee 10 > D-53757 Sankt Augustin > > Tel.: +49 (0) 2241 / 92 52 65 > Fax : +49 (0) 2241 / 92 52 99 > e-mail : ritzdorf@ccrl-nece.technopark.gmd.de > InPerson: ritzdorf@193.175.160.90 or > ritzdorf@sgi8.ccrl-nece.technopark.gmd.de > WWW : http://www.ccrl-nece.technopark.gmd.de/~ritzdorf > ___________________________________________________________________________ > ___ > > > > > > > > From mpi-core-human@mcs.anl.gov Tue Feb 2 13:55:17 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA14772; Tue, 2 Feb 1999 13:55:17 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA10534 for mpi-core-out; Tue, 2 Feb 1999 12:55:51 -0600 Received: from win153.nas.nasa.gov (win153.nas.nasa.gov [129.99.33.68]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA10521 for ; Tue, 2 Feb 1999 12:55:35 -0600 Received: from win153.nas.nasa.gov (nitzberg@localhost) by win153.nas.nasa.gov (8.9.1a/NAS8.8.7n) with ESMTP id KAA14832; Tue, 2 Feb 1999 10:55:13 -0800 (PST) Message-Id: <199902021855.KAA14832@win153.nas.nasa.gov> To: Hubert Ritzdorf cc: mpi-core@mcs.anl.gov Subject: Re: Question to MPI/IO Date: Tue, 02 Feb 1999 10:55:12 -0800 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with Jean-Pierre and John, version (i) is correct. - bill Bill Nitzberg, PhD Work: nitzberg@nas.nasa.gov MRJ Technology Solutions Tel: (650) 604-4513 NASA Ames Research Center, M/S 258-6 FAX: (650) 966-8669 Moffett Field, CA 94035-1000 Personal: nitzberg@netcom.com Jean-Pierre Prost replied: > Version (i) is the version intended by the standard, which says: > "The function must begin copying at the location in userbuf specified > by position into the (tiled) datatype." > Note that your note was confusing, because the descriptions of the two > alternatives before the code and within the code are opposite. The > right description is the one given within the code. Instead of saying > "the user must find...", I would rather say "the user defined conversion > routine must determine..." John May replied: > I agree with Jean-Pierre that version (i) is correct. Version > (ii) would not be as simple as it looks because the user might > call MPI_File_write with complex derived data type, and the > first call to the conversion function might return after > traversing only part of the type. Having the MPI implementation > figure out the right buffer location for the second call would > be pretty complicated, but the user function can just cache > this information somewhere when it finishes the first call and > then pick it up when it starts the second call. > > Since you're working on the datarep converters, you might be > interested in a test code we posted on the web at > http://www.llnl.gov/sccd/lc/piop/ . Look for the links at > the end of the MPI-IO section. Hubert Ritzdorf wrote: > > I am implementing MPI for NEC SX computers and I have a question to > Datarep Conversion functions (chapter 9.5.3 of the MPI-2 document) > > (*) If MPI cannot allocate a (file) buffer large enough to hold all the > the data to be converted, the standard says: > > For the first call, MPI will call the (conversion) function with > ``position'' set to zero. ... > Then in subsequent calls to the conversion function, > MPI will increment the value in ``position'' by the ``count'' > of items converted in the previous call. > > My question is: > Is the user buffer ``buf'' incremented correspondingly > in the second call (and further calls) of the conversion function > (cf. Version i below) or > must the user find the correct position in the user buffer ``buf'' > within the conversion function (cf. Version ii below)? > > i.e. > > /* first call */ > position = 0; > write_fn (buf, datatype, count, filebuf, position, extra_state); > > /* write filebuf to the file */ > > write (.., filebuf, count*datarep_size); > > > > /* (Version i) > second call, the user must find the correct position in buf */ > > position = count; > write_fn (buf, datatype, count, filebuf, position, extra_state); > > > or > > /* (Version ii) > second call, MPI transfers the new user buffer location */ > > position = count; > next_buf = (char *) buf + count * native_size; > write_fn (next_buf, datatype, count, filebuf, position, > extra_state); > /* write filebuf to the file */ > > write (.., filebuf, count*datarep_size); > From mpi-core-human@mcs.anl.gov Tue Feb 2 13:55:25 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id NAA14776; Tue, 2 Feb 1999 13:55:24 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id MAA10535 for mpi-core-out; Tue, 2 Feb 1999 12:55:51 -0600 Received: from win153.nas.nasa.gov (win153.nas.nasa.gov [129.99.33.68]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id MAA10522 for ; Tue, 2 Feb 1999 12:55:36 -0600 Received: from win153.nas.nasa.gov (nitzberg@localhost) by win153.nas.nasa.gov (8.9.1a/NAS8.8.7n) with ESMTP id KAA14832; Tue, 2 Feb 1999 10:55:13 -0800 (PST) Message-Id: <199902021855.KAA14832@win153.nas.nasa.gov> To: Hubert Ritzdorf cc: mpi-core@mcs.anl.gov Subject: Re: Question to MPI/IO Date: Tue, 02 Feb 1999 10:55:12 -0800 From: Bill Nitzberg Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I agree with Jean-Pierre and John, version (i) is correct. - bill Bill Nitzberg, PhD Work: nitzberg@nas.nasa.gov MRJ Technology Solutions Tel: (650) 604-4513 NASA Ames Research Center, M/S 258-6 FAX: (650) 966-8669 Moffett Field, CA 94035-1000 Personal: nitzberg@netcom.com Jean-Pierre Prost replied: > Version (i) is the version intended by the standard, which says: > "The function must begin copying at the location in userbuf specified > by position into the (tiled) datatype." > Note that your note was confusing, because the descriptions of the two > alternatives before the code and within the code are opposite. The > right description is the one given within the code. Instead of saying > "the user must find...", I would rather say "the user defined conversion > routine must determine..." John May replied: > I agree with Jean-Pierre that version (i) is correct. Version > (ii) would not be as simple as it looks because the user might > call MPI_File_write with complex derived data type, and the > first call to the conversion function might return after > traversing only part of the type. Having the MPI implementation > figure out the right buffer location for the second call would > be pretty complicated, but the user function can just cache > this information somewhere when it finishes the first call and > then pick it up when it starts the second call. > > Since you're working on the datarep converters, you might be > interested in a test code we posted on the web at > http://www.llnl.gov/sccd/lc/piop/ . Look for the links at > the end of the MPI-IO section. Hubert Ritzdorf wrote: > > I am implementing MPI for NEC SX computers and I have a question to > Datarep Conversion functions (chapter 9.5.3 of the MPI-2 document) > > (*) If MPI cannot allocate a (file) buffer large enough to hold all the > the data to be converted, the standard says: > > For the first call, MPI will call the (conversion) function with > ``position'' set to zero. ... > Then in subsequent calls to the conversion function, > MPI will increment the value in ``position'' by the ``count'' > of items converted in the previous call. > > My question is: > Is the user buffer ``buf'' incremented correspondingly > in the second call (and further calls) of the conversion function > (cf. Version i below) or > must the user find the correct position in the user buffer ``buf'' > within the conversion function (cf. Version ii below)? > > i.e. > > /* first call */ > position = 0; > write_fn (buf, datatype, count, filebuf, position, extra_state); > > /* write filebuf to the file */ > > write (.., filebuf, count*datarep_size); > > > > /* (Version i) > second call, the user must find the correct position in buf */ > > position = count; > write_fn (buf, datatype, count, filebuf, position, extra_state); > > > or > > /* (Version ii) > second call, MPI transfers the new user buffer location */ > > position = count; > next_buf = (char *) buf + count * native_size; > write_fn (next_buf, datatype, count, filebuf, position, > extra_state); > /* write filebuf to the file */ > > write (.., filebuf, count*datarep_size); > From mpi-core-human@mcs.anl.gov Wed Feb 3 15:29:41 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id PAA14629; Wed, 3 Feb 1999 15:29:40 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id OAA22450 for mpi-core-out; Wed, 3 Feb 1999 14:30:04 -0600 Received: from gropp.mcs.anl.gov (gropp-9.mcs.anl.gov [140.221.9.25]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id OAA22389; Wed, 3 Feb 1999 14:29:28 -0600 Message-Id: <4.1.19990203141018.00c0e100@localhost> X-Sender: gropp@localhost X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Date: Wed, 03 Feb 1999 14:29:14 -0600 To: "Pedro N.T. Silva" From: William Gropp Subject: Re: Clarification on MPI_PROC_NULL processes Cc: mpi-coll@mcs.anl.gov, mpi-core@mcs.anl.gov In-Reply-To: <199902031945.TAA06209@dsg.dei.uc.pt> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 08:04 PM 2/3/99 +0000, Pedro N.T. Silva wrote: >Hi, > >The descriptions on rooted operations do not mention alot on the processes >that pass MPI_PROC_NULL as the value of root. >I suppose these processes do nothing. >However, since I could not find anything explicitily stating this in the >archive of this mailing list and the MPI-2 standard document, I would like >to be sure about it. The processes may participate in the movement of data, even though they are not contributing or receiving any data. The Forum did discuss whether the root==MPI_PROC_NULL processes should have to call the routine, and it was decided that relaxing the requirement that all processes in a communicator participate in collective calls would be too confusing. Further, scalable implementations of the intercommunicator collective operations might require the participation of some of the MPI_PROC_NULL processes; the Forum did not want to constrain the implementations. > >Also, regarding the restriction on matching arguments for all members of >the same group. >Why would the user wish to allocate space for the arguments of processes >that have a passive role (ie the ones that pass MPI_PROC_NULL in root) in >the rooted operation? >I'm not talking so much about the data buffers but more about the >displacement buffers and such. Since these processes probably are operating >on the same data and thus already have had to allocate memory space. >So wouldn't it be ok to relax the restriction to include only the local >group active participants of the operation? > The text in the standard is a little unclear here. The text uses "significant only at root" for some of these, and is silent on arguments that are in the root's group in the intercommunicator case but are significant for the group that the root does not belong to. The text in the Complete Reference, while not the standard, clarifies this by using "significant only at the root process" and "significant only at processes of the leaf group" to indicate that where arguments are significant. This clarification should go into the errata. Bill From mpi-core-human@mcs.anl.gov Fri Feb 5 14:31:48 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA00777; Fri, 5 Feb 1999 14:31:47 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA02054 for mpi-core-out; Fri, 5 Feb 1999 13:32:12 -0600 Received: from gropp.mcs.anl.gov (gropp-9.mcs.anl.gov [140.221.9.25]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id NAA01986; Fri, 5 Feb 1999 13:30:57 -0600 Message-Id: <4.1.19990205132451.00c30220@localhost> X-Sender: gropp@localhost X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Date: Fri, 05 Feb 1999 13:30:40 -0600 To: "Pedro N.T. Silva" From: William Gropp Subject: Re: Clarification on Reduce_scatter Cc: mpi-coll@mcs.anl.gov, mpi-core@mcs.anl.gov In-Reply-To: <199902051720.RAA14842@dsg.dei.uc.pt> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 05:38 PM 2/5/99 +0000, Pedro N.T. Silva wrote: >Hello again, > >Thank you for the last reply. > >The question I have regards MPI_Reduce_scatter. >I don't see clearly the advantage one obtains from restricting the sum of >recvcounts entries to be the same for the 2 groups. >The standard implies that the recvcounts are not exchanged between the two >groups, so each group does not know how many elements each process of the >other group is to receive. However the total amount is known to both. > >Generally speaking, it actually depends on the implementation, this means >that there is some probability that a process will receive data it was not >meant to receive. Thus such processes will have to send it to a local >process according to the local recvcounts. > >I suppose that if one can keep the probability low the advantage is clear. >Maybe most applications have a uniform distribution of data so there will >be very few unbalances with regards to data distribution. > >However for some implementations this may not be true. For instance, a >simple (non scalable) implementation will have one process receive all the >results of the reduce from the other group. This process will then send the >data to the local processes according to the recvcounts. In this case, it >is probably better for the masters to exchange the recvcounts rather than >all the results from the local reduction and then do an inter distribution. > >Taking into account what you said in the previous answer, I'm guessing that >the restriction is there to give room for implementations that can exploit >it. Since it is not too restrictive for other implementations then it is >better to have it... > >Anyway, I'd like to hear your thoughts on this and what drived the decision >of the committee. > Basically, REDUCE_SCATTER in the intercommunicator version should be thought of as equivalent to an intERcommunicator reduce (where all you need is sum(recvcounts)), followed by an intRAcommunicator scatterv . This matches the description of the effect of reduce_scatter in MPI-1, along with the rational for requiring the sum of the recvcounts to be the same in both groups in the intercommunicator case. It also is the interpretation that keeps the recvcounts with the buffers into which data is received. The text as it stands could be clearer about the recvcounts. There was no extensive discussion of this routine in the intercommunicator case, just the straightforward extension of the MPI-1 description. Bill From mpi-core-human@mcs.anl.gov Mon Feb 15 14:33:46 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id OAA04424; Mon, 15 Feb 1999 14:33:46 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id NAA28606 for mpi-core-out; Mon, 15 Feb 1999 13:36:28 -0600 Received: from MPI-Softtech.Com (mpi.MPI-SoftTech.Com [208.152.187.97]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id NAA28479 for ; Mon, 15 Feb 1999 13:33:05 -0600 Received: from localhost (tony@localhost); by MPI-Softtech.Com (8.9.1/8.9.1/MPI-Softtech/evision: 1.3 $) with ESMTP; id NAA14005; Mon, 15 Feb 1999 13:33:02 -0600 (CST) Date: Mon, 15 Feb 1999 13:33:02 -0600 (CST) From: Tony Skjellum X-Sender: tony@mpi.mpi-softtech.com To: mpi-core@mcs.anl.gov, mpi-realtime@mpirt.org Subject: Exciting MPIDC99 Meeting in Atlanta, March 10-12, 1999 Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear colleagues, please attend the MPIDC99 meeting, March 10-12, 1999. The conference is described at www.mpidc.org Two tutorials, one on MPI-2, and one on MPI/RT, will be offered on March 9. We are looking forward to having you there. If you register this week, you can still register at the $300 rate for the meeting. The list of papers is online at the URL above. See you all in Atlanta. Thanks, Tony Anthony Skjellum, PhD, President (tony@mpi-softtech.com) MPI Software Technology, Inc., Ste 201, 1 Research Blvd., Starkville, MS 39759 +1-(601)320-4300 x15; FAX: +1-(601)320-4301; http://www.mpi-softtech.com *The source for MPI, MPI-2, MPI/RT, Embedded & High Performance Middleware(tm)* From mpi-core-human@mcs.anl.gov Fri Mar 12 12:23:07 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id MAA20199; Fri, 12 Mar 1999 12:23:07 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id LAA28547 for mpi-core-out; Fri, 12 Mar 1999 11:30:27 -0600 Received: from gropp.mcs.anl.gov (gropp-9.mcs.anl.gov [140.221.9.25]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id LAA28538 for ; Fri, 12 Mar 1999 11:30:19 -0600 Message-Id: <4.1.19990312112105.00bba100@localhost> X-Sender: gropp@localhost X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Date: Fri, 12 Mar 1999 11:30:38 -0600 To: mpi-core@mcs.anl.gov From: William Gropp Subject: No MPI_ERR_INFO class Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk There doesn't seem to be a class for info that corresponds to the MPI_ERR_COMM, MPI_ERR_TYPE, etc. classes. There are classes for various errors in using an info object (key,value,no matching key), but none for a bad Info object itself. MPI_ERR_INFO was present in the list of constants, but that didn't make it into the final copy of the standard. I'd like to define and use the class MPI_ERR_INFO in our implementation. Can MPI_ERR_INFO be considered a clarification, or must I use somethign like MPI_ERR_ARG or MPI_ERR_OTHER? Bill From mpi-core-human@mcs.anl.gov Wed Mar 17 17:32:47 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id RAA09157; Wed, 17 Mar 1999 17:32:46 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id QAA06756 for mpi-core-out; Wed, 17 Mar 1999 16:41:52 -0600 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id QAA06159 for ; Wed, 17 Mar 1999 16:23:31 -0600 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with SMTP id OAA09742 for ; Wed, 17 Mar 1999 14:23:30 -0800 (PST) Received: from suneast.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id RAA13960; Wed, 17 Mar 1999 17:23:25 -0500 Received: from balon.East.Sun.COM (balon [129.148.180.113]) by suneast.East.Sun.COM (8.9.1b+Sun/8.8.8) with SMTP id RAA11090 for ; Wed, 17 Mar 1999 17:23:26 -0500 (EST) Received: from balon by balon.East.Sun.COM (SMI-8.6/SMI-SVR4) id RAA00801; Wed, 17 Mar 1999 17:23:24 -0500 Date: Wed, 17 Mar 1999 17:23:24 -0500 (EST) From: Rolf vandeVaart - Sun HPC High Performance Computing Reply-To: Rolf vandeVaart - Sun HPC High Performance Computing Subject: MPI 1.1 Specification Clarification To: mpi-core@mcs.anl.gov Cc: nevin@balon.East.Sun.COM, rolfv@balon.East.Sun.COM Message-ID: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: n3lOC1bWzC0hWvjTY1sJQg== X-Mailer: dtmail 1.1.0 CDE Version 1.1_50 SunOS 5.5.1 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear Sirs: I am looking to get some clarification with respect to the behavior of MPI error handlers and the functions MPI_Errhandler_get() and MPI_Errhandler_free(). In Section 7.2 of the MPI 1.1 specification, with respect to MPI_Errhandler_free() it says: "Marks the error handler associated with errhandler for deallocation and sets errhandler to MPI_ERRHANDLER_NULL. The error handler will be deallocated after all communicators associated with it have been deallocated." This implies to me that in the following code snippet, 'errhandler' should be deallocated when the communicator is freed. MPI_Comm_dup(MPI_COMM_WORLD, newcomm); MPI_Errhandler_create(errfunc, &errhandler); MPI_Errhandler_set(newcomm, errhandler); MPI_Errhandler_get(newcomm, &olderrhandler); MPI_Errhandler_free(errhandler); MPI_Comm_free(newcomm); In this example, one would think that 'olderrhandler' would no longer pointing to a valid MPI_Errhandler as it would have been deallocated by the call to MPI_Comm_free(). However, in MPICH 1.1.2, this is not the case. Instead, one needs to call MPI_Errhandler_free() on the 'olderrhandler'. I am curious if MPICH is operating correctly and that I am misinterpreting the specification. Can someone clarify for me? Thank you, Rolf vandeVaart Sun Microsystems ========================= rolfv@east.sun.com 781-442-3043 ========================= From mpi-core-human@mcs.anl.gov Thu Mar 18 09:16:34 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA23087; Thu, 18 Mar 1999 09:16:34 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA20113 for mpi-core-out; Thu, 18 Mar 1999 08:26:19 -0600 Received: from gropp (gropp-9.mcs.anl.gov [140.221.9.25]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id IAA20105; Thu, 18 Mar 1999 08:26:12 -0600 Message-Id: <4.1.19990318074531.00bd27c0@localhost> X-Sender: gropp@localhost X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Date: Thu, 18 Mar 1999 07:59:04 -0600 To: Rolf vandeVaart - Sun HPC High Performance Computing From: William Gropp Subject: Re: MPI 1.1 Specification Clarification Cc: mpi-core@mcs.anl.gov, nevin@balon.East.Sun.COM, rolfv@balon.East.Sun.COM, mpi-bugs@mcs.anl.gov In-Reply-To: Mime-Version: 1.0 Content-Type: text/html; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 05:23 PM 3/17/99 -0500, Rolf vandeVaart - Sun HPC High Performance Computing wrote:
>
>Dear Sirs:
>
>I am looking to get some clarification with respect to the behavior of
>MPI error handlers and the functions MPI_Errhandler_get() and
>MPI_Errhandler_free().
>
>In Section 7.2 of the MPI 1.1 specification, with respect to
>MPI_Errhandler_free() it says:
>
>"Marks the error handler associated with errhandler for deallocation
>and sets errhandler to MPI_ERRHANDLER_NULL.  The error handler will
>be deallocated after all communicators associated with it have been
>deallocated."
>
>This implies to me that in the following code snippet, 'errhandler'
>should be deallocated when the communicator is freed.
>
>       MPI_Comm_dup(MPI_COMM_WORLD, newcomm);
>       MPI_Errhandler_create(errfunc, &errhandler);
>       MPI_Errhandler_set(newcomm, errhandler);
>       MPI_Errhandler_get(newcomm, &olderrhandler);
>       MPI_Errhandler_free(errhandler);
>       MPI_Comm_free(newcomm);
>
>In this example, one would think that 'olderrhandler' would no longer
>pointing to a valid MPI_Errhandler as it would have been deallocated
>by the call to MPI_Comm_free().  However, in MPICH 1.1.2, this is not
>the case.  Instead, one needs to call MPI_Errhandler_free() on the
>'olderrhandler'.
>
>I am curious if MPICH is operating correctly and that I am
>misinterpreting the specification.  Can someone clarify for me?

There are two parts to this.  First, should the object referred to by olderrhandler be invalid after the MPI_Comm_free?  The answer to this is yes; MPICH has a bug here.  The MPICH interpretation is incrementing the reference count on the error handler when a MPI_Errhandler_get retrieves it.  I my opinion (obviously), this is what the standard should have required, because it requires the user to explicitly free the reference to it before it can be made invalid.  However, this isn't what the standard says, and the next release of MPICH will fix this.

Second, should the value of olderrhandler change to MPI_ERRHANDLER_NULL?  The answer to this is no.  The relevant part of the standard is (from "Opaque Objects" in the "Terms and Conventions"):
A user may accidently create a dangling reference by assigning to a handle the value of another handle, and then deallocating the object associated with these handles. Conversely, if a handle variable is deallocated before the associated object is freed, then the object becomes inaccessible (this may occur, for example, if the handle is a local variable within a subroutine, and the subroutine is exited before the associated object is deallocated). It is the user's responsibility to avoid adding or deleting references to opaque objects, except as a result of calls that allocate or deallocate such objects. ( End of advice to users.)

The ease by which the user can create a dangling reference with MPI_Errhandler_set is the reason that MPICH (erroneously) increments the reference count on an MPI_Errhandler_set.

Bill From mpi-core-human@mcs.anl.gov Fri Mar 19 08:08:41 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA13677; Fri, 19 Mar 1999 08:08:40 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA19028 for mpi-core-out; Fri, 19 Mar 1999 07:17:42 -0600 Received: from www.ccrl-nece.technopark.gmd.de (www.ccrl-nece.technopark.gmd.de [193.175.160.84]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA16482 for ; Fri, 19 Mar 1999 02:45:57 -0600 Received: from ccrl-nece.technopark.gmd.de (root@gauss4.ccrl-nece.technopark.gmd.de [192.168.4.1]) by www.ccrl-nece.technopark.gmd.de (8.8.5/8.8.5) with ESMTP id KAA00427 for ; Fri, 19 Mar 1999 10:46:50 +0100 Received: from sgi9.ccrl-nece.technopark.gmd.de (sgi9.ccrl-nece.technopark.gmd.de [193.175.160.93]) by ccrl-nece.technopark.gmd.de (8.8.5/8.8.5) with SMTP id IAA27249; Fri, 19 Mar 1999 08:45:10 GMT Received: by sgi9.ccrl-nece.technopark.gmd.de (950413.SGI.8.6.12) id JAA01619; Fri, 19 Mar 1999 09:45:09 +0100 From: "Falk Zimmermann" Message-Id: <9903190945.ZM1617@sgi9.ccrl-nece.technopark.gmd.de> Date: Fri, 19 Mar 1999 09:45:07 +0000 X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: mpi-core@mcs.anl.gov Subject: MPI-2 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Dear ALL, I have a question concerning the definition of the new collective function MPI_EXSCAN: Why doesn't the definition allow the "in place" option, while the ordinary (inclusive) MPI_SCAN supports it ? Best regards Falk Zimmermann **************************************************************** Falk Zimmermann E-mail: falk@ccrl-nece.technopark.gmd.de Tel. +49 2241 925235 Fax. +49 2241 925299 Research Staff Member C & C Research Laboratories NEC Europe Ltd. Rathausallee 10 D-53757 Sankt Augustin Germany ***************************************************************** From mpi-core-human@mcs.anl.gov Fri Mar 19 19:33:42 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id TAA26148; Fri, 19 Mar 1999 19:33:42 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id SAA10256 for mpi-core-out; Fri, 19 Mar 1999 18:43:17 -0600 Received: from Aurora.CS.MsState.Edu (aurora.cs.msstate.edu [130.18.209.4]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id SAA10251 for ; Fri, 19 Mar 1999 18:43:11 -0600 Received: from localhost (tony@localhost); by Aurora.CS.MsState.Edu (8.9.3/8.9.1/MsState-Aurora/evision: 1.3 $) with ESMTP; id SAA05677; Fri, 19 Mar 1999 18:43:05 -0600 (CST) Date: Fri, 19 Mar 1999 18:43:05 -0600 (CST) From: Tony Skjellum To: Falk Zimmermann cc: mpi-core@mcs.anl.gov Subject: Re: MPI-2 In-Reply-To: <9903190945.ZM1617@sgi9.ccrl-nece.technopark.gmd.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk I recall that there is an issue with the identify operator being undefined and would need to be applicable for rank 0... On Fri, 19 Mar 1999, Falk Zimmermann wrote: > Date: Fri, 19 Mar 1999 09:45:07 +0000 > From: Falk Zimmermann > To: mpi-core@mcs.anl.gov > Subject: MPI-2 > > Dear ALL, > > I have a question concerning the definition of the new collective function > MPI_EXSCAN: Why doesn't the definition allow the "in place" option, while > the ordinary (inclusive) MPI_SCAN supports it ? > > Best regards > Falk Zimmermann > > **************************************************************** > Falk Zimmermann > E-mail: falk@ccrl-nece.technopark.gmd.de > Tel. +49 2241 925235 > Fax. +49 2241 925299 > Research Staff Member > C & C Research Laboratories > NEC Europe Ltd. > Rathausallee 10 > D-53757 Sankt Augustin > Germany > ***************************************************************** > > A. Skjellum, PhD, Assoc. Prof. of Computer Science; Mississippi State Univ. http://www.cs.msstate.edu/~tony; tony@cs.msstate.edu; 601-325-8435 (FAX -8997) "Unpopularity := A failure to poll-test policy before implementation." From mpi-core-human@mcs.anl.gov Thu Mar 25 16:26:41 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA15211; Thu, 25 Mar 1999 16:26:40 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA07549 for mpi-core-out; Thu, 25 Mar 1999 15:37:26 -0600 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA07526 for ; Thu, 25 Mar 1999 15:37:13 -0600 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with SMTP id NAA29257 for ; Thu, 25 Mar 1999 13:37:14 -0800 (PST) Received: from caja.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id QAA06494; Thu, 25 Mar 1999 16:37:06 -0500 Received: (from nevin@localhost) by caja.East.Sun.COM (8.8.8+Sun/8.8.8) id QAA03613; Thu, 25 Mar 1999 16:37:05 -0500 (EST) Date: Thu, 25 Mar 1999 16:37:05 -0500 (EST) Message-Id: <199903252137.QAA03613@caja.East.Sun.COM> From: Nicholas Nevin - Sun HPC High Performance Computing To: mpi-core@mcs.anl.gov Subject: clarification on inter-language interoperability of attributes Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Hi, All. I need some clarification on the behaviour of MPI_Comm_get_attr in the case of an architecture with 32 bit ints and 64 bit pointers. Section 4.12.7 of the MPI-2 standard states that MPI will store internally *address* sized attributes. It also states that an integer valued attribute when accessed from C via MPI_Comm_get_attr will return the address of a pointer to the integer valued attribute. This seems to imply that MPI_Comm_get_attr will return a pointer to a 64 bit quantity. This however will break codes which do this. int *tagub; int ub; MPI_Comm_get_attr(MPI_COMM_WORLD, MPI_TAG_UB, &tagub, &flag); ub = *tagub; Under this interpretation the code should be written something like this MPI_Aint *tagub; MPI_Aint ub; MPI_Comm_get_attr(MPI_COMM_WORLD, MPI_TAG_UB, &tagub, &flag); ub = *tagub; The same applies for MPI_Attr_get() since the bindings are defined to be identical. Is this what was intended? Thanks, Nick. From mpi-core-human@mcs.anl.gov Thu Mar 25 18:11:55 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id SAA16371; Thu, 25 Mar 1999 18:11:55 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id RAA11160 for mpi-core-out; Thu, 25 Mar 1999 17:23:31 -0600 Received: from smtp7.ny.us.ibm.com (smtp7.ny.us.ibm.com [198.133.22.19]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id RAA11151 for ; Thu, 25 Mar 1999 17:23:24 -0600 From: snir@us.ibm.com Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.117.200.22]) by smtp7.ny.us.ibm.com (8.8.7/8.8.7) with ESMTP id SAA135692; Thu, 25 Mar 1999 18:22:54 -0500 Received: from D51MTA03.pok.ibm.com (d51mta03.pok.ibm.com [9.117.200.31]) by northrelay02.pok.ibm.com (8.8.7m1/NCO v1.8) with SMTP id SAA88450; Thu, 25 Mar 1999 18:22:50 -0500 Received: by D51MTA03.pok.ibm.com(Lotus SMTP MTA v4.6.4 (817.1 3-4-1999)) id 8525673F.00806A66 ; Thu, 25 Mar 1999 18:22:38 -0500 X-Lotus-FromDomain: IBMUS To: Nicholas Nevin - Sun HPC High Performance Computing cc: mpi-core@mcs.anl.gov Message-ID: <8525673F.0080699E.00@D51MTA03.pok.ibm.com> Date: Thu, 25 Mar 1999 18:22:37 -0500 Subject: Re: clarification on inter-language interoperability of attributes Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk It's a bad idea to store and retrieve integer attributes in C -- since C stores and retrieves attributes of type void* (pointers). You can either 1. Trust type coertion to work OK. I.e. int ub; void* tagub; MPI_Comm_put_attr(..., (void*)ub,...); MPI_Comm_get_attr(..., &tagub, ...); ub = (int)tagub; this code, of course, is not portable. 2. Allocate memory for the integer attribute, and cache a pointer to the allocated memory. int ub; int *tagub tagub = (*int)malloc(sizeof ub); *tagub = ub; MPI_Comm_put_attr(..., tagub,...); MPI_Comm_get_attr(..., &tagub,...); ub = *tagub; Marc Snir Senior Manager, Scalable Parallel Systems IBM T J Watson Research Center, PO Box 218, Yorktown Heights 10598 Tel: 914 945 3204 (862 3204) Fax: 914 945 4425 (862 44245) URL: http://www.research.ibm.com/people/s/snir From mpi-core-human@mcs.anl.gov Fri Mar 26 08:35:42 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id IAA27625; Fri, 26 Mar 1999 08:35:42 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id HAA22245 for mpi-core-out; Fri, 26 Mar 1999 07:46:50 -0600 Received: from gropp (gropp-9.mcs.anl.gov [140.221.9.25]) by antares.mcs.anl.gov (8.6.10/8.6.10) with SMTP id HAA22234; Fri, 26 Mar 1999 07:46:36 -0600 Message-Id: <4.1.19990326074417.00bb7670@localhost> X-Sender: gropp@localhost X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Date: Fri, 26 Mar 1999 07:47:53 -0600 To: Nicholas Nevin - Sun HPC High Performance Computing From: William Gropp Subject: Re: clarification on inter-language interoperability of attributes Cc: mpi-core@mcs.anl.gov In-Reply-To: <199903252137.QAA03613@caja.East.Sun.COM> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk At 04:37 PM 3/25/99 -0500, Nicholas Nevin - Sun HPC High Performance Computing wrote: > >Hi, All. I need some clarification on the behaviour of MPI_Comm_get_attr >in the case of an architecture with 32 bit ints and 64 bit pointers. > >Section 4.12.7 of the MPI-2 standard states that MPI will store >internally *address* sized attributes. It also states that an integer >valued attribute when accessed from C via MPI_Comm_get_attr will return >the address of a pointer to the integer valued attribute. > >This seems to imply that MPI_Comm_get_attr will return a pointer to >a 64 bit quantity. This however will break codes which do this. No, it means that the attribute is stored as the address itself. The thing that the attribute points to may be an int, a long long, or a structure. That's why an MPI_TAG_UB is returned as the address of an int. We've been doing this for a long time with MPICH on many architectures with 32 bit C ints and 64 bit pointers (it is one of the reasons why Get_attr returns a pointer to the integer containing the tag UB, rather than the value of the UB itself). Bill From mpi-core-human@mcs.anl.gov Fri Mar 26 09:19:52 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA28286; Fri, 26 Mar 1999 09:19:51 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA23406 for mpi-core-out; Fri, 26 Mar 1999 08:31:32 -0600 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id IAA23396; Fri, 26 Mar 1999 08:31:23 -0600 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with SMTP id GAA21019; Fri, 26 Mar 1999 06:31:24 -0800 (PST) Received: from caja.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id JAA22997; Fri, 26 Mar 1999 09:31:15 -0500 Received: from caja (caja [129.148.180.111]) by caja.East.Sun.COM (8.8.8+Sun/8.8.8) with SMTP id JAA04878; Fri, 26 Mar 1999 09:31:15 -0500 (EST) Message-Id: <199903261431.JAA04878@caja.East.Sun.COM> Date: Fri, 26 Mar 1999 09:31:15 -0500 (EST) From: Nicholas Nevin - Sun HPC High Performance Computing Reply-To: Nicholas Nevin - Sun HPC High Performance Computing Subject: Re: clarification on inter-language interoperability of attributes To: gropp@mcs.anl.gov Cc: mpi-core@mcs.anl.gov MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: SnVfQtdUlxzACsd6XLHBTg== X-Mailer: dtmail 1.2.1 CDE Version 1.2.1 SunOS 5.6 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > X-Sender: gropp@localhost > Date: Fri, 26 Mar 1999 07:47:53 -0600 > To: Nicholas Nevin - Sun HPC High Performance Computing > From: William Gropp > Subject: Re: clarification on inter-language interoperability of attributes > Cc: mpi-core@mcs.anl.gov > > At 04:37 PM 3/25/99 -0500, Nicholas Nevin - Sun HPC High Performance > Computing wrote: > > > >Hi, All. I need some clarification on the behaviour of MPI_Comm_get_attr > >in the case of an architecture with 32 bit ints and 64 bit pointers. > > > >Section 4.12.7 of the MPI-2 standard states that MPI will store > >internally *address* sized attributes. It also states that an integer > >valued attribute when accessed from C via MPI_Comm_get_attr will return > >the address of a pointer to the integer valued attribute. > > > >This seems to imply that MPI_Comm_get_attr will return a pointer to > >a 64 bit quantity. This hoWhen an integer valued attribute is accessed from C or C++, then MPI_xxx_get_attr will return the address of (a pointer to) the integer valued attribute.wever will break codes which do this. > > No, it means that the attribute is stored as the address itself. The thing This might work for predefined attributes where the address can point to the integer value in static storage, but won't work for attributes set by the user in Fortran code. If you store a pointer to the integer as the attribute it may point to a temporary which might no longer exist when you try and access it later. > that the attribute points to may be an int, a long long, or a structure. > That's why an MPI_TAG_UB is returned as the address of an int. We've been > doing this for a long time with MPICH on many architectures with 32 bit C > ints and 64 bit pointers (it is one of the reasons why Get_attr returns a > pointer to the integer containing the tag UB, rather than the value of the > UB itself). > Unfortunately I left out some parentheses in my paraphrase of the standard, it actually reads "When an integer valued attribute is accessed from C or C++, then MPI_xxx_get_attr will return the address of (a pointer to) the integer valued attribute." This is what I had meant to say. So my question still stands. Assuming Fortran INTEGER is 32 bits and MPI_ADDRESS_KIND is 8, do I get back from MPI_xxx_get_attr a pointer to a 32 bit integer or a 64 bit integer? I see three cases and the answer may not be the same for all of them. 1. predefined attributes. The answer may vary according to the attribute (window size is of type MPI_ADDRESS_KIND in Fortran). 2. attributes set in Fortran with MPI_ATTR_PUT 3. attributes set in Fortran with MPI_COMM_SET_ATTR Of course it would be nice if the answer was the same for all the above cases which would require that a pointer to a 64 bit integer be returned in order to avoid possible data truncation. -nick. From mpi-core-human@mcs.anl.gov Fri Mar 26 09:34:12 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id JAA28695; Fri, 26 Mar 1999 09:34:11 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id IAA23843 for mpi-core-out; Fri, 26 Mar 1999 08:45:55 -0600 Received: from skylla.rus.uni-stuttgart.DE (root@skylla.rus.uni-stuttgart.de [129.69.1.8]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id CAA18793 for ; Fri, 26 Mar 1999 02:53:45 -0600 Received: from awsrr.rus.uni-stuttgart.de (awsrr.rus.uni-stuttgart.de [129.69.14.187]) by skylla.rus.uni-stuttgart.DE (8.8.8/8.8.8) with SMTP id JAA00810; Fri, 26 Mar 1999 09:53:41 +0100 (MET) env-from (rusrabe@awsrr.rus.uni-stuttgart.de) Received: by awsrr.rus.uni-stuttgart.de (940816.SGI.8.6.9/BelWue-1.0SG(subsidiary)) (for ) id IAA26049; Fri, 26 Mar 1999 08:53:40 GMT From: rusrabe@awsrr.rus.uni-stuttgart.de (Rolf Rabenseifner) Message-Id: <199903260853.IAA26049@awsrr.rus.uni-stuttgart.de> Subject: Re: clarification on inter-language interoperability of attributes To: snir@us.ibm.com Date: Fri, 26 Mar 1999 09:53:40 +0100 (MEZ) Cc: nevin@caja.East.Sun.COM, mpi-core@mcs.anl.gov In-Reply-To: <8525673F.0080699E.00@D51MTA03.pok.ibm.com> from "snir@us.ibm.com" at Mar 25, 99 06:22:37 pm Content-Type: text Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Nick Nevin mentioned: > MPI_Comm_get_attr(MPI_COMM_WORLD, MPI_TAG_UB, &tagub, &flag); ========== Marc Snir stated: > It's a bad idea to store and retrieve integer attributes in C -- since C stores > and retrieves attributes of type void* (pointers). You can either ========== [...] > int ub; > MPI_Comm_put_attr(..., (void*)ub,...); [or] > int ub; > int *tagub > *tagub = ub; > MPI_Comm_put_attr(..., tagub,...); But MPI 1.1 has defined 4 predefined INTEGER/int-typed attributes in the "7.1.1 Environmental Inquiries", and therefore, the question is, which "MPI_Comm_put_attr" is done by all MPI implementations. === Rolf PS: I have no opinion, what is the right answer, but when it will be clear, it would be nice, if we add as clarification an example, how to use MPI_Comm_get_attr with the predefined valus in Fortran and C. From mpi-core-human@mcs.anl.gov Fri Mar 26 16:02:13 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA07861; Fri, 26 Mar 1999 16:02:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA08391 for mpi-core-out; Fri, 26 Mar 1999 15:14:01 -0600 Received: from smtp3.ny.us.ibm.com (smtp3.ny.us.ibm.com [198.133.22.42]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA08386 for ; Fri, 26 Mar 1999 15:13:53 -0600 From: snir@us.ibm.com Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.117.200.22]) by smtp3.ny.us.ibm.com (8.8.7/8.8.7) with ESMTP id QAA27310; Fri, 26 Mar 1999 16:12:35 -0500 Received: from D51MTA03.pok.ibm.com (d51mta03.pok.ibm.com [9.117.200.31]) by northrelay02.pok.ibm.com (8.8.7m1/NCO v1.8) with SMTP id QAA167766; Fri, 26 Mar 1999 16:13:17 -0500 Received: by D51MTA03.pok.ibm.com(Lotus SMTP MTA v4.6.4 (817.1 3-4-1999)) id 85256740.00748C6E ; Fri, 26 Mar 1999 16:13:01 -0500 X-Lotus-FromDomain: IBMUS To: rusrabe@awsrr.rus.uni-stuttgart.de (Rolf Rabenseifner) cc: nevin@caja.East.Sun.COM, mpi-core@mcs.anl.gov Message-ID: <85256740.00748B7D.00@D51MTA03.pok.ibm.com> Date: Fri, 26 Mar 1999 16:02:15 -0500 Subject: Re: clarification on inter-language interoperability of attributes Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Again, look at the MPI document, on language interoperability 4-12-7. It makes clear how integer attributes (aka Fortran attributes) are accessed in C. It's as shown in my second example -- namely MPI_Comm_Attr_get returns a pointer to the integer variable. Marc Snir Senior Manager, Scalable Parallel Systems IBM T J Watson Research Center, PO Box 218, Yorktown Heights 10598 Tel: 914 945 3204 (862 3204) Fax: 914 945 4425 (862 44245) URL: http://www.research.ibm.com/people/s/snir From mpi-core-human@mcs.anl.gov Fri Mar 26 16:17:42 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA08007; Fri, 26 Mar 1999 16:17:41 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA09033 for mpi-core-out; Fri, 26 Mar 1999 15:29:34 -0600 Received: from smtp7.ny.us.ibm.com (smtp7.ny.us.ibm.com [198.133.22.19]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA09023; Fri, 26 Mar 1999 15:29:20 -0600 From: snir@us.ibm.com Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.117.200.22]) by smtp7.ny.us.ibm.com (8.8.7/8.8.7) with ESMTP id QAA129458; Fri, 26 Mar 1999 16:28:50 -0500 Received: from D51MTA03.pok.ibm.com (d51mta03.pok.ibm.com [9.117.200.31]) by northrelay02.pok.ibm.com (8.8.7m1/NCO v1.8) with SMTP id QAA24536; Fri, 26 Mar 1999 16:28:46 -0500 Received: by D51MTA03.pok.ibm.com(Lotus SMTP MTA v4.6.4 (817.1 3-4-1999)) id 85256740.0075F77B ; Fri, 26 Mar 1999 16:28:30 -0500 X-Lotus-FromDomain: IBMUS To: Nicholas Nevin - Sun HPC High Performance Computing cc: gropp@mcs.anl.gov, mpi-core@mcs.anl.gov Message-ID: <85256740.0075F6E8.00@D51MTA03.pok.ibm.com> Date: Fri, 26 Mar 1999 16:20:21 -0500 Subject: Re: clarification on inter-language interoperability of attributes Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk MPI_xxx_get_attr returns a void* value in "attr". If the attribute you accessed is a Fortran attribute then (int*)attr will point to an INTEGER. MPI_ADDRESS_KIND is irrelevant. > Unfortunately I left out some parentheses in my paraphrase of the standard, it actually reads "When an integer valued attribute is accessed from C or C++, then MPI_xxx_get_attr will return the address of (a pointer to) the integer valued attribute." This is what I had meant to say. So my question still stands. Assuming Fortran INTEGER is 32 bits and MPI_ADDRESS_KIND is 8, do I get back from MPI_xxx_get_attr a pointer to a 32 bit integer or a 64 bit integer? I see three cases and the answer may not be the same for all of them. 1. predefined attributes. The answer may vary according to the attribute (window size is of type MPI_ADDRESS_KIND in Fortran). 2. attributes set in Fortran with MPI_ATTR_PUT 3. attributes set in Fortran with MPI_COMM_SET_ATTR Of course it would be nice if the answer was the same for all the above cases which would require that a pointer to a 64 bit integer be returned in order to avoid possible data truncation. -nick. From mpi-core-human@mcs.anl.gov Fri Mar 26 16:27:44 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id QAA08128; Fri, 26 Mar 1999 16:27:43 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id PAA09390 for mpi-core-out; Fri, 26 Mar 1999 15:39:19 -0600 Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id PAA09384 for ; Fri, 26 Mar 1999 15:39:12 -0600 Received: from East.Sun.COM ([129.148.1.241]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with SMTP id NAA06084; Fri, 26 Mar 1999 13:39:00 -0800 (PST) Received: from caja.East.Sun.COM by East.Sun.COM (SMI-8.6/SMI-5.3) id QAA13081; Fri, 26 Mar 1999 16:38:51 -0500 Received: from caja (caja [129.148.180.111]) by caja.East.Sun.COM (8.8.8+Sun/8.8.8) with SMTP id QAA05742; Fri, 26 Mar 1999 16:38:51 -0500 (EST) Message-Id: <199903262138.QAA05742@caja.East.Sun.COM> Date: Fri, 26 Mar 1999 16:38:51 -0500 (EST) From: Nicholas Nevin - Sun HPC High Performance Computing Reply-To: Nicholas Nevin - Sun HPC High Performance Computing Subject: Re: clarification on inter-language interoperability of attributes To: snir@us.ibm.com Cc: mpi-core@mcs.anl.gov MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: WbB7DRyFXc8swew+ZeC6PA== X-Mailer: dtmail 1.2.1 CDE Version 1.2.1 SunOS 5.6 sun4u sparc Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk > From: snir@us.ibm.com > X-Lotus-FromDomain: IBMUS > To: Nicholas Nevin - Sun HPC High Performance Computing > cc: gropp@mcs.anl.gov, mpi-core@mcs.anl.gov > Date: Fri, 26 Mar 1999 16:20:21 -0500 > Subject: Re: clarification on inter-language interoperability of attributes > Content-Disposition: inline > > > MPI_xxx_get_attr returns a void* value in "attr". If the attribute you accessed > is a Fortran attribute then (int*)attr will point to an INTEGER. > MPI_ADDRESS_KIND is irrelevant. > > > I'm afraid I have to disagree. MPI_ADDRESS_KIND is relevant. Here is the binding for MPI_COMM_SET_ATTR. MPI_COMM_SET_ATTR(COMM, COMM_KEYVAL, ATTRIBUTE_VAL, IERROR) INTEGER COMM, COMM_KEYVAL, IERROR INTEGER(KIND=MPI_ADDRESS_KIND) ATTRIBUTE_VAL ^^^^^^^^^^^^^^^^ ????? -nick From mpi-core-human@mcs.anl.gov Mon Mar 29 01:30:12 1999 Return-Path: Received: from antares.mcs.anl.gov (mcs.anl.gov [140.221.9.6]) by netlib2.cs.utk.edu with ESMTP (cf v2.9t-netlib) id BAA08526; Mon, 29 Mar 1999 01:30:12 -0500 Received: (from listserv@localhost) by antares.mcs.anl.gov (8.6.10/8.6.10) id AAA24981 for mpi-core-out; Mon, 29 Mar 1999 00:40:13 -0600 Received: from smtp4.server.ibm.com (smtp4.ny.us.ibm.com [198.133.22.43]) by antares.mcs.anl.gov (8.6.10/8.6.10) with ESMTP id AAA24974 for ; Mon, 29 Mar 1999 00:39:59 -0600 From: snir@us.ibm.com Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.117.200.22]) by smtp4.server.ibm.com (8.8.7/8.8.7) with ESMTP id BAA24314; Mon, 29 Mar 1999 01:39:14 -0500 Received: from D51MTA03.pok.ibm.com (d51mta03.pok.ibm.com [9.117.200.31]) by northrelay02.pok.ibm.com (8.8.8m2/NCO v1.8) with SMTP id BAA63342; Mon, 29 Mar 1999 01:39:13 -0500 Received: by D51MTA03.pok.ibm.com(Lotus SMTP MTA v4.6.4 (817.1 3-4-1999)) id 85256743.002488EA ; Mon, 29 Mar 1999 01:39:03 -0500 X-Lotus-FromDomain: IBMUS To: Nicholas Nevin - Sun HPC High Performance Computing cc: mpi-core@mcs.anl.gov Message-ID: <85256743.0024877A.00@D51MTA03.pok.ibm.com> Date: Sun, 28 Mar 1999 19:24:47 -0500 Subject: Re: clarification on inter-language interoperability of attributes Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-mpi-core@mcs.anl.gov Precedence: bulk Sorry -- I was wrong. Internally, Fortran attributes are of type MPI_ADDRESS_KIND, indeed. If such an attribute is retrieved in C, then MPI_xxx_get_attr will return a pointer (of type void*) that will point to an integer of suitable size. Where I said INTEGER, below, I should have said INTEGER(KIND=MPI_ADDRESS_KIND). Marc Snir Senior Manager, Scalable Parallel Systems IBM T J Watson Research Center, PO Box 218, Yorktown Heights 10598 Tel: 914 945 3204 (862 3204) Fax: 914 945 4425 (862 44245) URL: http://www.research.ibm.com/people/s/snir Nicholas Nevin - Sun HPC High Performance Computing on 03/26/99 04:38:51 PM Please respond to Nicholas Nevin - Sun HPC High Performance Computing To: Marc Snir/Watson/IBM@IBMUS cc: mpi-core@mcs.anl.gov Subject: Re: clarification on inter-language interoperability of attributes > From: snir@us.ibm.com > X-Lotus-FromDomain: IBMUS > To: Nicholas Nevin - Sun HPC High Performance Computing > cc: gropp@mcs.anl.gov, mpi-core@mcs.anl.gov > Date: Fri, 26 Mar 1999 16:20:21 -0500 > Subject: Re: clarification on inter-language interoperability of attributes > Content-Disposition: inline > > > MPI_xxx_get_attr returns a void* value in "attr". If the attribute you accessed > is a Fortran attribute then (int*)attr will point to an INTEGER. > MPI_ADDRESS_KIND is irrelevant. > > > I'm afraid I have to disagree. MPI_ADDRESS_KIND is relevant. Here is the binding for MPI_COMM_SET_ATTR. MPI_COMM_SET_ATTR(COMM, COMM_KEYVAL, ATTRIBUTE_VAL, IERROR) INTEGER COMM, COMM_KEYVAL, IERROR INTEGER(KIND=MPI_ADDRESS_KIND) ATTRIBUTE_VAL ^^^^^^^^^^^^^^^^ ????? -nick .